2024-04-25T08:38:02.169 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T08:38:02.169 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_neesingh@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} --archive /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997 --name neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --verbose -- /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997/orig.config.yaml 2024-04-25T08:38:02.212 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T08:38:02.296 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7672997' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: 5787d42d10024ba410e822a8aba474331cb9af02 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5869 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 5787d42d10024ba410e822a8aba474331cb9af02 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMnwStv2ELVuA8XyN2z919cNNU9QuT8hb+z+dsJoCU8UDOotF/inBaQsVTrG4h3m0s4Htqa4bpDDycVjXfUt4lE= smithi203.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL2qEriPyJ6CItPjKUMyzVb/XnJ8y8GlJhIh02+rWs09iGUimc8iLUpOTQ25/3O5URX3FiPYroh9HvocYrYK7Dk= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_08:15:18 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T08:38:02.296 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa; will attempt to use it 2024-04-25T08:38:02.297 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks 2024-04-25T08:38:02.297 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T08:38:02.299 INFO:teuthology.task.internal:Checking packages... 2024-04-25T08:38:02.321 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T08:38:02.322 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T08:38:02.322 INFO:teuthology.packaging:ref: None 2024-04-25T08:38:02.322 INFO:teuthology.packaging:tag: None 2024-04-25T08:38:02.322 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T08:38:02.322 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:38:02.322 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T08:38:02.492 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:38:02.494 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T08:38:02.500 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T08:38:02.501 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T08:38:02.521 INFO:teuthology.task.internal:Saving configuration 2024-04-25T08:38:02.551 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T08:38:02.557 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T08:38:02.580 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:32:08.221304', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIH7WPy5kLdqQGIiOIamJxWB7X95bYaRjPaFl4X9pLj3S'} 2024-04-25T08:38:02.599 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi203.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:32:08.223234', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBi2zc9ckwy1a/gL7sFskLm6fOKhj8n7md/R48CvilW8'} 2024-04-25T08:38:02.599 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T08:38:02.605 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T08:38:02.605 INFO:teuthology.task.internal:roles: ubuntu@smithi203.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T08:38:02.605 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T08:38:02.673 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4a2b6ef0d0>, signals=[15]) 2024-04-25T08:38:02.674 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T08:38:02.679 INFO:teuthology.task.internal:Opening connections... 2024-04-25T08:38:02.680 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:38:02.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:38:02.762 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:38:02.763 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:38:02.843 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T08:38:02.850 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-04-25T08:38:02.855 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-04-25T08:38:02.856 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="Ubuntu" 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="22.04" 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_CODENAME=jammy 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:ID=ubuntu 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE=debian 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T08:38:02.905 INFO:teuthology.orchestra.run.smithi037.stdout:UBUNTU_CODENAME=jammy 2024-04-25T08:38:02.906 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-04-25T08:38:02.930 DEBUG:teuthology.orchestra.run.smithi203:> uname -m 2024-04-25T08:38:02.936 INFO:teuthology.orchestra.run.smithi203.stdout:x86_64 2024-04-25T08:38:02.936 DEBUG:teuthology.orchestra.run.smithi203:> cat /etc/os-release 2024-04-25T08:38:02.984 INFO:teuthology.orchestra.run.smithi203.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T08:38:02.984 INFO:teuthology.orchestra.run.smithi203.stdout:NAME="Ubuntu" 2024-04-25T08:38:02.984 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_ID="22.04" 2024-04-25T08:38:02.984 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T08:38:02.984 INFO:teuthology.orchestra.run.smithi203.stdout:VERSION_CODENAME=jammy 2024-04-25T08:38:02.984 INFO:teuthology.orchestra.run.smithi203.stdout:ID=ubuntu 2024-04-25T08:38:02.984 INFO:teuthology.orchestra.run.smithi203.stdout:ID_LIKE=debian 2024-04-25T08:38:02.984 INFO:teuthology.orchestra.run.smithi203.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T08:38:02.984 INFO:teuthology.orchestra.run.smithi203.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T08:38:02.984 INFO:teuthology.orchestra.run.smithi203.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T08:38:02.985 INFO:teuthology.orchestra.run.smithi203.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T08:38:02.985 INFO:teuthology.orchestra.run.smithi203.stdout:UBUNTU_CODENAME=jammy 2024-04-25T08:38:02.985 INFO:teuthology.lock.ops:Updating smithi203.front.sepia.ceph.com on lock server 2024-04-25T08:38:03.034 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T08:38:03.043 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T08:38:03.049 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T08:38:03.049 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:38:03.052 DEBUG:teuthology.orchestra.run.smithi203:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:38:03.056 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T08:38:03.061 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T08:38:03.061 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:38:03.098 DEBUG:teuthology.orchestra.run.smithi203:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:38:03.113 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T08:38:03.168 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T08:38:03.248 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T08:38:03.248 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T08:38:03.249 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:38:03.249 DEBUG:teuthology.orchestra.run.smithi203:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:38:03.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:38:03.254 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-04-25T08:38:03.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:38:03.255 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-04-25T08:38:03.300 INFO:teuthology.orchestra.run.smithi203.stdout:5.15.0-56-generic 2024-04-25T08:38:03.301 INFO:teuthology.task.kernel:Running kernel on smithi203: 5.15.0-56-generic 2024-04-25T08:38:03.301 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-get clean 2024-04-25T08:38:03.302 INFO:teuthology.orchestra.run.smithi037.stdout:5.15.0-56-generic 2024-04-25T08:38:03.302 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.15.0-56-generic 2024-04-25T08:38:03.302 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-get clean 2024-04-25T08:38:03.426 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-get update 2024-04-25T08:38:03.428 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-get update 2024-04-25T08:38:03.582 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:38:03.598 INFO:teuthology.orchestra.run.smithi203.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T08:38:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T08:38:03.687 INFO:teuthology.orchestra.run.smithi037.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T08:38:03.703 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:38:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T08:38:03.785 INFO:teuthology.orchestra.run.smithi203.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T08:38:03.793 INFO:teuthology.orchestra.run.smithi203.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T08:38:03.825 INFO:teuthology.orchestra.run.smithi037.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T08:38:03.905 INFO:teuthology.orchestra.run.smithi203.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T08:38:03.907 INFO:teuthology.orchestra.run.smithi037.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T08:38:03.910 INFO:teuthology.orchestra.run.smithi203.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T08:38:03.913 INFO:teuthology.orchestra.run.smithi203.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T08:38:03.913 INFO:teuthology.orchestra.run.smithi203.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T08:38:03.936 INFO:teuthology.orchestra.run.smithi203.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T08:38:03.936 INFO:teuthology.orchestra.run.smithi203.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T08:38:03.940 INFO:teuthology.orchestra.run.smithi203.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T08:38:03.940 INFO:teuthology.orchestra.run.smithi203.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T08:38:03.948 INFO:teuthology.orchestra.run.smithi203.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T08:38:03.955 INFO:teuthology.orchestra.run.smithi037.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T08:38:03.957 INFO:teuthology.orchestra.run.smithi203.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T08:38:03.959 INFO:teuthology.orchestra.run.smithi203.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T08:38:03.960 INFO:teuthology.orchestra.run.smithi203.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T08:38:03.960 INFO:teuthology.orchestra.run.smithi203.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T08:38:03.964 INFO:teuthology.orchestra.run.smithi037.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T08:38:03.965 INFO:teuthology.orchestra.run.smithi037.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T08:38:03.980 INFO:teuthology.orchestra.run.smithi203.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T08:38:03.996 INFO:teuthology.orchestra.run.smithi037.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T08:38:03.997 INFO:teuthology.orchestra.run.smithi037.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T08:38:04.001 INFO:teuthology.orchestra.run.smithi037.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T08:38:04.001 INFO:teuthology.orchestra.run.smithi037.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T08:38:04.005 INFO:teuthology.orchestra.run.smithi203.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T08:38:04.012 INFO:teuthology.orchestra.run.smithi037.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T08:38:04.018 INFO:teuthology.orchestra.run.smithi037.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T08:38:04.021 INFO:teuthology.orchestra.run.smithi037.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T08:38:04.053 INFO:teuthology.orchestra.run.smithi037.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T08:38:04.054 INFO:teuthology.orchestra.run.smithi037.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T08:38:04.054 INFO:teuthology.orchestra.run.smithi037.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T08:38:04.054 INFO:teuthology.orchestra.run.smithi037.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T08:38:04.055 INFO:teuthology.orchestra.run.smithi037.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T08:38:04.055 INFO:teuthology.orchestra.run.smithi037.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T08:38:04.056 INFO:teuthology.orchestra.run.smithi037.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T08:38:04.060 INFO:teuthology.orchestra.run.smithi037.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T08:38:04.082 INFO:teuthology.orchestra.run.smithi037.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T08:38:04.082 INFO:teuthology.orchestra.run.smithi037.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T08:38:04.082 INFO:teuthology.orchestra.run.smithi037.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T08:38:04.082 INFO:teuthology.orchestra.run.smithi037.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T08:38:04.152 INFO:teuthology.orchestra.run.smithi203.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T08:38:04.166 INFO:teuthology.orchestra.run.smithi037.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T08:38:04.290 INFO:teuthology.orchestra.run.smithi203.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T08:38:04.455 INFO:teuthology.orchestra.run.smithi037.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T08:38:04.489 INFO:teuthology.orchestra.run.smithi203.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T08:38:04.623 INFO:teuthology.orchestra.run.smithi037.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T08:38:04.636 INFO:teuthology.orchestra.run.smithi037.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T08:38:04.637 INFO:teuthology.orchestra.run.smithi037.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T08:38:04.639 INFO:teuthology.orchestra.run.smithi037.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T08:38:04.640 INFO:teuthology.orchestra.run.smithi203.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T08:38:04.654 INFO:teuthology.orchestra.run.smithi203.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T08:38:04.655 INFO:teuthology.orchestra.run.smithi203.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T08:38:04.723 INFO:teuthology.orchestra.run.smithi037.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T08:38:04.730 INFO:teuthology.orchestra.run.smithi037.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T08:38:04.730 INFO:teuthology.orchestra.run.smithi037.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T08:38:04.750 INFO:teuthology.orchestra.run.smithi037.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T08:38:04.752 INFO:teuthology.orchestra.run.smithi203.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T08:38:04.753 INFO:teuthology.orchestra.run.smithi203.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T08:38:04.761 INFO:teuthology.orchestra.run.smithi203.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T08:38:04.925 INFO:teuthology.orchestra.run.smithi203.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T08:38:04.927 INFO:teuthology.orchestra.run.smithi037.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T08:38:05.332 INFO:teuthology.orchestra.run.smithi037.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T08:38:05.363 INFO:teuthology.orchestra.run.smithi037.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T08:38:05.388 INFO:teuthology.orchestra.run.smithi037.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T08:38:05.388 INFO:teuthology.orchestra.run.smithi037.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T08:38:05.389 INFO:teuthology.orchestra.run.smithi037.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T08:38:05.559 INFO:teuthology.orchestra.run.smithi203.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T08:38:05.597 INFO:teuthology.orchestra.run.smithi203.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T08:38:05.610 INFO:teuthology.orchestra.run.smithi203.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T08:38:05.611 INFO:teuthology.orchestra.run.smithi203.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T08:38:05.613 INFO:teuthology.orchestra.run.smithi203.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T08:38:05.614 INFO:teuthology.orchestra.run.smithi203.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T08:38:05.614 INFO:teuthology.orchestra.run.smithi203.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T08:38:05.615 INFO:teuthology.orchestra.run.smithi203.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [90.8 kB] 2024-04-25T08:38:05.643 INFO:teuthology.orchestra.run.smithi203.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [82.9 kB] 2024-04-25T08:38:05.647 INFO:teuthology.orchestra.run.smithi203.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T08:38:05.812 INFO:teuthology.orchestra.run.smithi203.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T08:38:05.894 INFO:teuthology.orchestra.run.smithi203.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [18.5 kB] 2024-04-25T08:38:05.980 INFO:teuthology.orchestra.run.smithi203.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [29.8 kB] 2024-04-25T08:38:06.066 INFO:teuthology.orchestra.run.smithi203.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T08:38:06.097 INFO:teuthology.orchestra.run.smithi203.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T08:38:10.345 INFO:teuthology.orchestra.run.smithi203.stdout:Fetched 13.3 MB in 3s (5,109 kB/s) 2024-04-25T08:38:10.347 INFO:teuthology.orchestra.run.smithi037.stdout:Fetched 13.3 MB in 2s (6,979 kB/s) 2024-04-25T08:38:11.644 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2024-04-25T08:38:11.661 DEBUG:teuthology.orchestra.run.smithi203:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T08:38:11.722 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2024-04-25T08:38:11.873 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2024-04-25T08:38:11.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T08:38:11.930 INFO:teuthology.orchestra.run.smithi203.stdout:Building dependency tree... 2024-04-25T08:38:11.931 INFO:teuthology.orchestra.run.smithi203.stdout:Reading state information... 2024-04-25T08:38:11.958 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2024-04-25T08:38:12.098 INFO:teuthology.orchestra.run.smithi203.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:38:12.099 INFO:teuthology.orchestra.run.smithi203.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:38:12.099 INFO:teuthology.orchestra.run.smithi203.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:38:12.099 INFO:teuthology.orchestra.run.smithi203.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:38:12.099 INFO:teuthology.orchestra.run.smithi203.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T08:38:12.099 INFO:teuthology.orchestra.run.smithi203.stdout: python2.7-minimal 2024-04-25T08:38:12.100 INFO:teuthology.orchestra.run.smithi203.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:38:12.100 INFO:teuthology.orchestra.run.smithi203.stdout:The following additional packages will be installed: 2024-04-25T08:38:12.100 INFO:teuthology.orchestra.run.smithi203.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T08:38:12.100 INFO:teuthology.orchestra.run.smithi203.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T08:38:12.101 INFO:teuthology.orchestra.run.smithi203.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T08:38:12.102 INFO:teuthology.orchestra.run.smithi203.stdout:Suggested packages: 2024-04-25T08:38:12.102 INFO:teuthology.orchestra.run.smithi203.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T08:38:12.102 INFO:teuthology.orchestra.run.smithi203.stdout:Recommended packages: 2024-04-25T08:38:12.102 INFO:teuthology.orchestra.run.smithi203.stdout: thermald 2024-04-25T08:38:12.158 INFO:teuthology.orchestra.run.smithi203.stdout:The following NEW packages will be installed: 2024-04-25T08:38:12.158 INFO:teuthology.orchestra.run.smithi203.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T08:38:12.158 INFO:teuthology.orchestra.run.smithi203.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T08:38:12.158 INFO:teuthology.orchestra.run.smithi203.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T08:38:12.160 INFO:teuthology.orchestra.run.smithi203.stdout:The following packages will be upgraded: 2024-04-25T08:38:12.160 INFO:teuthology.orchestra.run.smithi203.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T08:38:12.164 INFO:teuthology.orchestra.run.smithi037.stdout:Building dependency tree... 2024-04-25T08:38:12.165 INFO:teuthology.orchestra.run.smithi037.stdout:Reading state information... 2024-04-25T08:38:12.217 INFO:teuthology.orchestra.run.smithi203.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T08:38:12.217 INFO:teuthology.orchestra.run.smithi203.stdout:Need to get 113 MB of archives. 2024-04-25T08:38:12.217 INFO:teuthology.orchestra.run.smithi203.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T08:38:12.217 INFO:teuthology.orchestra.run.smithi203.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T08:38:12.348 INFO:teuthology.orchestra.run.smithi037.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:38:12.348 INFO:teuthology.orchestra.run.smithi037.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:38:12.348 INFO:teuthology.orchestra.run.smithi037.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:38:12.348 INFO:teuthology.orchestra.run.smithi037.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:38:12.349 INFO:teuthology.orchestra.run.smithi037.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T08:38:12.349 INFO:teuthology.orchestra.run.smithi037.stdout: python2.7-minimal 2024-04-25T08:38:12.349 INFO:teuthology.orchestra.run.smithi037.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:38:12.350 INFO:teuthology.orchestra.run.smithi037.stdout:The following additional packages will be installed: 2024-04-25T08:38:12.350 INFO:teuthology.orchestra.run.smithi037.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T08:38:12.350 INFO:teuthology.orchestra.run.smithi037.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T08:38:12.350 INFO:teuthology.orchestra.run.smithi037.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T08:38:12.352 INFO:teuthology.orchestra.run.smithi037.stdout:Suggested packages: 2024-04-25T08:38:12.352 INFO:teuthology.orchestra.run.smithi037.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T08:38:12.352 INFO:teuthology.orchestra.run.smithi037.stdout:Recommended packages: 2024-04-25T08:38:12.352 INFO:teuthology.orchestra.run.smithi037.stdout: thermald 2024-04-25T08:38:12.401 INFO:teuthology.orchestra.run.smithi037.stdout:The following NEW packages will be installed: 2024-04-25T08:38:12.401 INFO:teuthology.orchestra.run.smithi037.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T08:38:12.401 INFO:teuthology.orchestra.run.smithi037.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T08:38:12.401 INFO:teuthology.orchestra.run.smithi037.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T08:38:12.402 INFO:teuthology.orchestra.run.smithi037.stdout:The following packages will be upgraded: 2024-04-25T08:38:12.403 INFO:teuthology.orchestra.run.smithi037.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T08:38:12.583 INFO:teuthology.orchestra.run.smithi037.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T08:38:12.583 INFO:teuthology.orchestra.run.smithi037.stdout:Need to get 113 MB of archives. 2024-04-25T08:38:12.584 INFO:teuthology.orchestra.run.smithi037.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T08:38:12.584 INFO:teuthology.orchestra.run.smithi037.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T08:38:12.740 INFO:teuthology.orchestra.run.smithi203.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T08:38:12.914 INFO:teuthology.orchestra.run.smithi203.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T08:38:14.003 INFO:teuthology.orchestra.run.smithi037.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi203.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi203.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T08:38:14.224 INFO:teuthology.orchestra.run.smithi203.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T08:38:14.388 INFO:teuthology.orchestra.run.smithi203.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T08:38:14.417 INFO:teuthology.orchestra.run.smithi203.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T08:38:14.417 INFO:teuthology.orchestra.run.smithi037.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T08:38:14.803 INFO:teuthology.orchestra.run.smithi203.stdout:Fetched 113 MB in 2s (50.3 MB/s) 2024-04-25T08:38:14.962 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T08:38:15.853 INFO:teuthology.orchestra.run.smithi037.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T08:38:15.882 INFO:teuthology.orchestra.run.smithi037.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T08:38:15.919 INFO:teuthology.orchestra.run.smithi037.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T08:38:16.146 INFO:teuthology.orchestra.run.smithi037.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T08:38:16.228 INFO:teuthology.orchestra.run.smithi037.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T08:38:16.665 INFO:teuthology.orchestra.run.smithi037.stdout:Fetched 113 MB in 4s (29.7 MB/s) 2024-04-25T08:38:16.846 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T08:38:17.007 INFO:teuthology.orchestra.run.smithi203.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T08:38:17.011 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:38:17.100 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:18.817 INFO:teuthology.orchestra.run.smithi037.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T08:38:18.822 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:38:18.984 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:19.185 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T08:38:19.206 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:38:19.294 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:19.771 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T08:38:19.792 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:38:19.830 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:21.660 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T08:38:21.681 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:38:21.797 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:22.263 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T08:38:22.284 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:38:22.329 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:26.902 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:38:27.029 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:38:27.362 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:38:27.475 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:38:27.784 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T08:38:27.806 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T08:38:27.844 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T08:38:28.944 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:38:29.056 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:38:29.389 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:38:29.534 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:38:29.845 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T08:38:29.869 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T08:38:29.911 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T08:38:33.777 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T08:38:33.814 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:38:33.851 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:35.745 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T08:38:35.783 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T08:38:35.829 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:36.535 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:38:36.661 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:38:37.003 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T08:38:37.124 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:37.250 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T08:38:37.379 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:38.696 INFO:teuthology.orchestra.run.smithi203.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:38:38.696 INFO:teuthology.orchestra.run.smithi203.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T08:38:38.696 INFO:teuthology.orchestra.run.smithi203.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:38:38.696 INFO:teuthology.orchestra.run.smithi203.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T08:38:38.891 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:38.931 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T08:38:39.055 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T08:38:40.264 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T08:38:40.385 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:40.511 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T08:38:40.544 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:40.681 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:41.936 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T08:38:42.045 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T08:38:42.067 INFO:teuthology.orchestra.run.smithi037.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:38:42.067 INFO:teuthology.orchestra.run.smithi037.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T08:38:42.067 INFO:teuthology.orchestra.run.smithi037.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:38:42.067 INFO:teuthology.orchestra.run.smithi037.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T08:38:42.147 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:42.202 INFO:teuthology.orchestra.run.smithi203.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T08:38:42.202 INFO:teuthology.orchestra.run.smithi203.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:38:42.360 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:43.796 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:45.647 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T08:38:46.240 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T08:38:46.341 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T08:38:46.386 INFO:teuthology.orchestra.run.smithi037.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T08:38:46.386 INFO:teuthology.orchestra.run.smithi037.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:38:56.658 INFO:teuthology.orchestra.run.smithi203.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T08:38:56.658 INFO:teuthology.orchestra.run.smithi203.stdout:Sourcing file `/etc/default/grub' 2024-04-25T08:38:56.671 INFO:teuthology.orchestra.run.smithi203.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:38:56.695 INFO:teuthology.orchestra.run.smithi203.stdout:Generating grub configuration file ... 2024-04-25T08:38:56.996 INFO:teuthology.orchestra.run.smithi203.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:38:57.011 INFO:teuthology.orchestra.run.smithi203.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:38:57.307 INFO:teuthology.orchestra.run.smithi203.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:38:57.312 INFO:teuthology.orchestra.run.smithi203.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:38:57.379 INFO:teuthology.orchestra.run.smithi203.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:38:57.384 INFO:teuthology.orchestra.run.smithi203.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:38:57.542 INFO:teuthology.orchestra.run.smithi203.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:38:57.542 INFO:teuthology.orchestra.run.smithi203.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:38:57.542 INFO:teuthology.orchestra.run.smithi203.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:38:57.578 INFO:teuthology.orchestra.run.smithi203.stdout:done 2024-04-25T08:38:58.037 DEBUG:teuthology.orchestra.run.smithi203:> dpkg -s linux-image-generic 2024-04-25T08:38:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:Package: linux-image-generic 2024-04-25T08:38:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:Status: install ok installed 2024-04-25T08:38:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:Priority: optional 2024-04-25T08:38:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:Section: kernel 2024-04-25T08:38:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:Installed-Size: 21 2024-04-25T08:38:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T08:38:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:Architecture: amd64 2024-04-25T08:38:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:Source: linux-meta 2024-04-25T08:38:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:Version: 5.15.0.105.102 2024-04-25T08:38:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T08:38:58.056 INFO:teuthology.orchestra.run.smithi203.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T08:38:58.057 INFO:teuthology.orchestra.run.smithi203.stdout:Recommends: thermald 2024-04-25T08:38:58.057 INFO:teuthology.orchestra.run.smithi203.stdout:Description: Generic Linux kernel image 2024-04-25T08:38:58.057 INFO:teuthology.orchestra.run.smithi203.stdout: This package will always depend on the latest generic kernel image 2024-04-25T08:38:58.057 INFO:teuthology.orchestra.run.smithi203.stdout: available. 2024-04-25T08:38:58.057 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T08:38:58.057 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T08:38:58.057 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T08:38:58.057 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T08:38:58.057 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi203.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T08:38:58.057 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-get clean 2024-04-25T08:38:58.219 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-get update 2024-04-25T08:38:58.341 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:38:58.341 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T08:38:58.353 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T08:38:58.373 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T08:38:59.619 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2024-04-25T08:38:59.637 DEBUG:teuthology.orchestra.run.smithi203:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T08:38:59.687 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2024-04-25T08:38:59.865 INFO:teuthology.orchestra.run.smithi203.stdout:Building dependency tree... 2024-04-25T08:38:59.865 INFO:teuthology.orchestra.run.smithi203.stdout:Reading state information... 2024-04-25T08:39:00.025 INFO:teuthology.orchestra.run.smithi203.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T08:39:00.026 INFO:teuthology.orchestra.run.smithi203.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:39:00.026 INFO:teuthology.orchestra.run.smithi203.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:39:00.026 INFO:teuthology.orchestra.run.smithi203.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:39:00.026 INFO:teuthology.orchestra.run.smithi203.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:39:00.026 INFO:teuthology.orchestra.run.smithi203.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:39:00.026 INFO:teuthology.orchestra.run.smithi203.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:39:00.026 INFO:teuthology.orchestra.run.smithi203.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:39:00.062 INFO:teuthology.orchestra.run.smithi203.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T08:39:00.064 DEBUG:teuthology.orchestra.run.smithi203:> dpkg -s linux-image-generic 2024-04-25T08:39:00.118 INFO:teuthology.orchestra.run.smithi203.stdout:Package: linux-image-generic 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Status: install ok installed 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Priority: optional 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Section: kernel 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Installed-Size: 21 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Architecture: amd64 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Source: linux-meta 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Version: 5.15.0.105.102 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Recommends: thermald 2024-04-25T08:39:00.119 INFO:teuthology.orchestra.run.smithi203.stdout:Description: Generic Linux kernel image 2024-04-25T08:39:00.120 INFO:teuthology.orchestra.run.smithi203.stdout: This package will always depend on the latest generic kernel image 2024-04-25T08:39:00.120 INFO:teuthology.orchestra.run.smithi203.stdout: available. 2024-04-25T08:39:00.120 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-25T08:39:00.168 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.kGK33YZj0Q 2024-04-25T08:39:00.169 DEBUG:teuthology.orchestra.run.smithi203:> sudo cp /boot/grub/grub.cfg /tmp/tmp.kGK33YZj0Q 2024-04-25T08:39:00.223 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.kGK33YZj0Q 2024-04-25T08:39:00.336 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.kGK33YZj0Q is 10KB 2024-04-25T08:39:00.349 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.kGK33YZj0Q 2024-04-25T08:39:00.354 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T08:39:00.407 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:39:00.407 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T08:39:00.407 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T08:39:00.475 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T08:39:00.475 DEBUG:teuthology.orchestra.run.smithi203:> sudo update-grub 2024-04-25T08:39:01.246 INFO:teuthology.orchestra.run.smithi037.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T08:39:01.246 INFO:teuthology.orchestra.run.smithi037.stdout:Sourcing file `/etc/default/grub' 2024-04-25T08:39:01.258 INFO:teuthology.orchestra.run.smithi037.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:39:01.286 INFO:teuthology.orchestra.run.smithi037.stdout:Generating grub configuration file ... 2024-04-25T08:39:01.582 INFO:teuthology.orchestra.run.smithi037.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:39:01.597 INFO:teuthology.orchestra.run.smithi037.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:39:01.702 INFO:teuthology.orchestra.run.smithi203.stderr:Sourcing file `/etc/default/grub' 2024-04-25T08:39:01.703 INFO:teuthology.orchestra.run.smithi203.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:39:01.706 INFO:teuthology.orchestra.run.smithi203.stderr:Generating grub configuration file ... 2024-04-25T08:39:01.884 INFO:teuthology.orchestra.run.smithi203.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:39:01.890 INFO:teuthology.orchestra.run.smithi203.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:39:01.951 INFO:teuthology.orchestra.run.smithi037.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:39:01.956 INFO:teuthology.orchestra.run.smithi037.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:39:02.022 INFO:teuthology.orchestra.run.smithi037.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:39:02.028 INFO:teuthology.orchestra.run.smithi037.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:39:02.122 INFO:teuthology.orchestra.run.smithi203.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:39:02.128 INFO:teuthology.orchestra.run.smithi203.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:39:02.173 INFO:teuthology.orchestra.run.smithi203.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:39:02.179 INFO:teuthology.orchestra.run.smithi203.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:39:02.194 INFO:teuthology.orchestra.run.smithi037.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:39:02.194 INFO:teuthology.orchestra.run.smithi037.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:39:02.194 INFO:teuthology.orchestra.run.smithi037.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:39:02.235 INFO:teuthology.orchestra.run.smithi037.stdout:done 2024-04-25T08:39:02.313 INFO:teuthology.orchestra.run.smithi203.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:39:02.313 INFO:teuthology.orchestra.run.smithi203.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:39:02.313 INFO:teuthology.orchestra.run.smithi203.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:39:02.329 INFO:teuthology.orchestra.run.smithi203.stderr:done 2024-04-25T08:39:02.337 DEBUG:teuthology.orchestra.run.smithi203:> sudo shutdown -r now 2024-04-25T08:39:02.713 DEBUG:teuthology.orchestra.run.smithi037:> dpkg -s linux-image-generic 2024-04-25T08:39:02.743 INFO:teuthology.orchestra.run.smithi037.stdout:Package: linux-image-generic 2024-04-25T08:39:02.743 INFO:teuthology.orchestra.run.smithi037.stdout:Status: install ok installed 2024-04-25T08:39:02.743 INFO:teuthology.orchestra.run.smithi037.stdout:Priority: optional 2024-04-25T08:39:02.743 INFO:teuthology.orchestra.run.smithi037.stdout:Section: kernel 2024-04-25T08:39:02.743 INFO:teuthology.orchestra.run.smithi037.stdout:Installed-Size: 21 2024-04-25T08:39:02.743 INFO:teuthology.orchestra.run.smithi037.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T08:39:02.743 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: amd64 2024-04-25T08:39:02.743 INFO:teuthology.orchestra.run.smithi037.stdout:Source: linux-meta 2024-04-25T08:39:02.743 INFO:teuthology.orchestra.run.smithi037.stdout:Version: 5.15.0.105.102 2024-04-25T08:39:02.743 INFO:teuthology.orchestra.run.smithi037.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T08:39:02.743 INFO:teuthology.orchestra.run.smithi037.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T08:39:02.744 INFO:teuthology.orchestra.run.smithi037.stdout:Recommends: thermald 2024-04-25T08:39:02.744 INFO:teuthology.orchestra.run.smithi037.stdout:Description: Generic Linux kernel image 2024-04-25T08:39:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: This package will always depend on the latest generic kernel image 2024-04-25T08:39:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: available. 2024-04-25T08:39:02.744 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T08:39:02.744 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T08:39:02.744 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T08:39:02.744 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T08:39:02.744 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi037.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T08:39:02.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-get clean 2024-04-25T08:39:02.899 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-get update 2024-04-25T08:39:03.027 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:39:03.029 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T08:39:03.042 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T08:39:03.063 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T08:39:04.430 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2024-04-25T08:39:04.448 DEBUG:teuthology.orchestra.run.smithi037:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T08:39:04.509 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2024-04-25T08:39:04.721 INFO:teuthology.orchestra.run.smithi037.stdout:Building dependency tree... 2024-04-25T08:39:04.721 INFO:teuthology.orchestra.run.smithi037.stdout:Reading state information... 2024-04-25T08:39:04.905 INFO:teuthology.orchestra.run.smithi037.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T08:39:04.905 INFO:teuthology.orchestra.run.smithi037.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:39:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:39:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:39:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:39:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:39:04.906 INFO:teuthology.orchestra.run.smithi037.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:39:04.907 INFO:teuthology.orchestra.run.smithi037.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:39:04.947 INFO:teuthology.orchestra.run.smithi037.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T08:39:04.949 DEBUG:teuthology.orchestra.run.smithi037:> dpkg -s linux-image-generic 2024-04-25T08:39:04.966 INFO:teuthology.orchestra.run.smithi037.stdout:Package: linux-image-generic 2024-04-25T08:39:04.966 INFO:teuthology.orchestra.run.smithi037.stdout:Status: install ok installed 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout:Priority: optional 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout:Section: kernel 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout:Installed-Size: 21 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: amd64 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout:Source: linux-meta 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout:Version: 5.15.0.105.102 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout:Recommends: thermald 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout:Description: Generic Linux kernel image 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout: This package will always depend on the latest generic kernel image 2024-04-25T08:39:04.967 INFO:teuthology.orchestra.run.smithi037.stdout: available. 2024-04-25T08:39:04.968 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-04-25T08:39:05.012 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.OYzqJ2qW8W 2024-04-25T08:39:05.012 DEBUG:teuthology.orchestra.run.smithi037:> sudo cp /boot/grub/grub.cfg /tmp/tmp.OYzqJ2qW8W 2024-04-25T08:39:05.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.OYzqJ2qW8W 2024-04-25T08:39:05.200 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.OYzqJ2qW8W is 10KB 2024-04-25T08:39:05.213 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.OYzqJ2qW8W 2024-04-25T08:39:05.218 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T08:39:05.273 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:39:05.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T08:39:05.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T08:39:05.464 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T08:39:05.464 DEBUG:teuthology.orchestra.run.smithi037:> sudo update-grub 2024-04-25T08:39:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:Sourcing file `/etc/default/grub' 2024-04-25T08:39:06.715 INFO:teuthology.orchestra.run.smithi037.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T08:39:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:Generating grub configuration file ... 2024-04-25T08:39:06.946 INFO:teuthology.orchestra.run.smithi037.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T08:39:06.953 INFO:teuthology.orchestra.run.smithi037.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T08:39:07.172 INFO:teuthology.orchestra.run.smithi037.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T08:39:07.177 INFO:teuthology.orchestra.run.smithi037.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T08:39:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T08:39:07.224 INFO:teuthology.orchestra.run.smithi037.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T08:39:07.334 INFO:teuthology.orchestra.run.smithi037.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T08:39:07.334 INFO:teuthology.orchestra.run.smithi037.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T08:39:07.334 INFO:teuthology.orchestra.run.smithi037.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T08:39:07.344 INFO:teuthology.orchestra.run.smithi037.stderr:done 2024-04-25T08:39:07.353 DEBUG:teuthology.orchestra.run.smithi037:> sudo shutdown -r now 2024-04-25T08:39:32.360 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T08:39:32.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-25T08:39:32.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:39:37.361 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T08:39:37.361 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T08:39:37.361 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:39:50.815 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-04-25T08:39:55.682 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-04-25T08:39:59.818 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-25T08:39:59.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:40:02.878 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.203 2024-04-25T08:40:04.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T08:40:04.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:40:14.891 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-25T08:40:14.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:40:23.071 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-04-25T08:40:35.083 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T08:40:35.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:40:38.146 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-04-25T08:40:53.149 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T08:40:53.150 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:40:53.354 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-04-25T08:40:53.915 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T08:40:53.916 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T08:40:53.916 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-04-25T08:40:53.963 INFO:teuthology.orchestra.run.smithi037.stdout:5.15.0-105-generic 2024-04-25T08:40:53.963 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T08:40:53.963 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T08:40:53.963 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T08:40:54.965 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T08:40:54.965 DEBUG:teuthology.orchestra.run.smithi037:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T08:40:55.022 INFO:teuthology.orchestra.run.smithi037.stdout:ttyS1 2024-04-25T08:40:55.037 DEBUG:teuthology.parallel:result is None 2024-04-25T08:41:14.894 DEBUG:teuthology.orchestra.remote:timed out 2024-04-25T08:41:29.896 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-25T08:41:29.896 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:41:30.157 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-04-25T08:41:30.712 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-25T08:41:30.712 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T08:41:30.712 DEBUG:teuthology.orchestra.run.smithi203:> uname -r 2024-04-25T08:41:30.757 INFO:teuthology.orchestra.run.smithi203.stdout:5.15.0-105-generic 2024-04-25T08:41:30.757 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T08:41:30.757 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T08:41:30.757 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T08:41:31.758 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T08:41:31.759 DEBUG:teuthology.orchestra.run.smithi203:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T08:41:32.114 INFO:teuthology.orchestra.run.smithi203.stdout:ttyS1 2024-04-25T08:41:32.151 DEBUG:teuthology.parallel:result is None 2024-04-25T08:41:32.151 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T08:41:32.158 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T08:41:32.158 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:41:32.161 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:41:32.166 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T08:41:32.180 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T08:41:32.191 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T08:41:32.191 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:41:32.208 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:41:32.227 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T08:41:32.233 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T08:41:32.233 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:41:32.268 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:41:32.286 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:41:32.287 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:41:32.295 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:41:32.296 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:41:32.298 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T08:41:32.304 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T08:41:32.304 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:41:32.344 DEBUG:teuthology.orchestra.run.smithi203:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:41:32.356 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T08:41:32.364 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T08:41:32.364 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:41:32.400 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:41:32.405 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:41:32.451 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:41:32.499 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:41:32.499 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:41:32.562 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:41:32.568 DEBUG:teuthology.orchestra.run.smithi203:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:41:32.613 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:41:32.613 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:41:32.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-04-25T08:41:32.685 DEBUG:teuthology.orchestra.run.smithi203:> sudo service rsyslog restart 2024-04-25T08:41:32.754 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T08:41:32.761 INFO:teuthology.task.internal:Starting timer... 2024-04-25T08:41:32.761 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T08:41:32.771 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T08:41:32.779 DEBUG:teuthology.task.selinux:Excluding smithi037: OS 'ubuntu' does not support SELinux 2024-04-25T08:41:32.779 DEBUG:teuthology.task.selinux:Excluding smithi203: OS 'ubuntu' does not support SELinux 2024-04-25T08:41:32.779 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T08:41:32.779 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T08:41:32.780 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T08:41:32.780 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T08:41:32.788 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T08:41:32.794 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T08:41:32.938 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T08:41:32.953 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T08:41:32.954 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi203.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T08:48:34.982 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi203.front.sepia.ceph.com')] 2024-04-25T08:48:34.984 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T08:48:34.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:48:35.067 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-04-25T08:48:35.146 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-25T08:48:35.150 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-25T08:48:35.151 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi203.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:48:35.234 DEBUG:teuthology.orchestra.run.smithi203:> true 2024-04-25T08:48:35.310 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi203.front.sepia.ceph.com' 2024-04-25T08:48:35.310 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T08:48:35.324 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T08:48:35.324 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:48:35.325 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:48:35.327 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:48:35.327 DEBUG:teuthology.orchestra.run.smithi203:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:48:35.355 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T08:48:35.355 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: Command line: ntpd -gq 2024-04-25T08:48:35.355 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: ---------------------------------------------------- 2024-04-25T08:48:35.356 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T08:48:35.356 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T08:48:35.356 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: corporation. Support and training for ntp-4 are 2024-04-25T08:48:35.356 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: available at https://www.nwtime.org/support 2024-04-25T08:48:35.356 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: ---------------------------------------------------- 2024-04-25T08:48:35.356 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: proto: precision = 0.046 usec (-24) 2024-04-25T08:48:35.357 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: basedate set to 2022-02-04 2024-04-25T08:48:35.357 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: gps base set to 2022-02-06 (week 2196) 2024-04-25T08:48:35.357 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:48:35.357 INFO:teuthology.orchestra.run.smithi037.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:48:35.358 INFO:teuthology.orchestra.run.smithi037.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:48:35.359 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:48:35.359 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T08:48:35.359 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T08:48:35.360 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T08:48:35.360 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: Listen normally on 3 ens1f0 172.21.15.37:123 2024-04-25T08:48:35.360 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: Listen normally on 4 lo [::1]:123 2024-04-25T08:48:35.360 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:fe88:7794%4]:123 2024-04-25T08:48:35.360 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:35 ntpd[17138]: Listening on routing socket on fd #22 for interface updates 2024-04-25T08:48:35.388 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T08:48:35.388 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: Command line: ntpd -gq 2024-04-25T08:48:35.388 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: ---------------------------------------------------- 2024-04-25T08:48:35.388 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T08:48:35.388 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T08:48:35.388 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: corporation. Support and training for ntp-4 are 2024-04-25T08:48:35.388 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: available at https://www.nwtime.org/support 2024-04-25T08:48:35.388 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: ---------------------------------------------------- 2024-04-25T08:48:35.388 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: proto: precision = 0.060 usec (-24) 2024-04-25T08:48:35.388 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: basedate set to 2022-02-04 2024-04-25T08:48:35.389 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: gps base set to 2022-02-06 (week 2196) 2024-04-25T08:48:35.389 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:48:35.389 INFO:teuthology.orchestra.run.smithi203.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T08:48:35.389 INFO:teuthology.orchestra.run.smithi203.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:48:35.390 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T08:48:35.390 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T08:48:35.390 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T08:48:35.390 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T08:48:35.390 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: Listen normally on 3 enp3s0f1 172.21.15.203:123 2024-04-25T08:48:35.391 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: Listen normally on 4 lo [::1]:123 2024-04-25T08:48:35.391 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:4a23%5]:123 2024-04-25T08:48:35.391 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:35 ntpd[17196]: Listening on routing socket on fd #22 for interface updates 2024-04-25T08:48:36.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:36.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:36.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:36.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:36.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:36 ntpd[17138]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:36.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:36 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:36.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:36 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:36.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:36 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:36.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:36 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:36.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:36 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:36.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:36 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:37.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:37.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:37.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:37 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:38 ntpd[17138]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:38.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:38 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:38.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:38 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:38.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:38 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:38.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:38 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:38.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:38 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:38.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:38 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:39.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:39.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:39.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:39.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:39.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:39.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:39.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:39.389 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:39.389 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:39.389 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:39.389 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:39.389 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:39 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:40 ntpd[17138]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:40.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:40 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:40.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:40 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:40.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:40 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:40.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:40 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:40.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:40 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:40.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:40 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:41.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:41.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:41.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:41.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:41.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:41.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:41.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:41.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:41.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:41.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:41.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:41.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:41 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:42.354 INFO:teuthology.orchestra.run.smithi037.stdout:25 Apr 08:48:42 ntpd[17138]: ntpd: time slew -0.000620 s 2024-04-25T08:48:42.355 INFO:teuthology.orchestra.run.smithi037.stdout:ntpd: time slew -0.000620s 2024-04-25T08:48:42.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T08:48:42.355 INFO:teuthology.orchestra.run.smithi037.stderr:25 Apr 08:48:42 ntpd[17138]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T08:48:42.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:42 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:42.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:42 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:42.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:42 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:42.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:42 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:42.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:42 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:42.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:42 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:42.406 INFO:teuthology.orchestra.run.smithi037.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T08:48:42.406 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================== 2024-04-25T08:48:42.406 INFO:teuthology.orchestra.run.smithi037.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:48:42.406 INFO:teuthology.orchestra.run.smithi037.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:48:42.406 INFO:teuthology.orchestra.run.smithi037.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:48:42.406 INFO:teuthology.orchestra.run.smithi037.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:48:43.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:43 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:43.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:43 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:43.387 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:43 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:43.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:43 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:43.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:43 ntpd[17196]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T08:48:43.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:43 ntpd[17196]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T08:48:43.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:43 ntpd[17196]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T08:48:43.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:43 ntpd[17196]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T08:48:43.388 INFO:teuthology.orchestra.run.smithi203.stdout:25 Apr 08:48:43 ntpd[17196]: ntpd: time slew -0.002548 s 2024-04-25T08:48:43.388 INFO:teuthology.orchestra.run.smithi203.stdout:ntpd: time slew -0.002548s 2024-04-25T08:48:43.388 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:43 ntpd[17196]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T08:48:43.389 INFO:teuthology.orchestra.run.smithi203.stderr:25 Apr 08:48:43 ntpd[17196]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T08:48:43.484 INFO:teuthology.orchestra.run.smithi203.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T08:48:43.484 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================== 2024-04-25T08:48:43.484 INFO:teuthology.orchestra.run.smithi203.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:48:43.484 INFO:teuthology.orchestra.run.smithi203.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:48:43.484 INFO:teuthology.orchestra.run.smithi203.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:48:43.485 INFO:teuthology.orchestra.run.smithi203.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T08:48:43.485 INFO:teuthology.run_tasks:Running task install... 2024-04-25T08:48:43.493 DEBUG:teuthology.task.install:project ceph 2024-04-25T08:48:43.493 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T08:48:43.493 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T08:48:43.493 INFO:teuthology.task.install:Using flavor: default 2024-04-25T08:48:43.499 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T08:48:43.499 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T08:48:43.499 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-key list | grep Ceph 2024-04-25T08:48:43.500 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-key list | grep Ceph 2024-04-25T08:48:43.569 INFO:teuthology.orchestra.run.smithi037.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T08:48:43.585 INFO:teuthology.orchestra.run.smithi203.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T08:48:43.600 INFO:teuthology.orchestra.run.smithi037.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T08:48:43.601 INFO:teuthology.orchestra.run.smithi037.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T08:48:43.601 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T08:48:43.602 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T08:48:43.602 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:48:43.615 INFO:teuthology.orchestra.run.smithi203.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T08:48:43.615 INFO:teuthology.orchestra.run.smithi203.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T08:48:43.615 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T08:48:43.616 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T08:48:43.616 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:48:43.752 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T08:48:43.752 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:48:43.756 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default/ 2024-04-25T08:48:43.756 INFO:teuthology.task.install.deb:Package version is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:48:43.874 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:48:43.874 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T08:48:43.888 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:48:43.888 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T08:48:43.889 DEBUG:teuthology.orchestra.run.smithi037:> sudo apt-get update 2024-04-25T08:48:43.903 DEBUG:teuthology.orchestra.run.smithi203:> sudo apt-get update 2024-04-25T08:48:44.069 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:48:44.084 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T08:48:44.084 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T08:48:44.091 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T08:48:44.104 INFO:teuthology.orchestra.run.smithi037.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T08:48:44.104 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T08:48:44.126 INFO:teuthology.orchestra.run.smithi203.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T08:48:44.129 INFO:teuthology.orchestra.run.smithi037.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T08:48:44.136 INFO:teuthology.orchestra.run.smithi203.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi203.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T08:48:44.166 INFO:teuthology.orchestra.run.smithi037.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T08:48:44.193 INFO:teuthology.orchestra.run.smithi037.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T08:48:44.194 INFO:teuthology.orchestra.run.smithi203.stdout:Ign:6 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T08:48:44.202 INFO:teuthology.orchestra.run.smithi203.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T08:48:44.218 INFO:teuthology.orchestra.run.smithi203.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T08:48:44.229 INFO:teuthology.orchestra.run.smithi037.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T08:48:44.247 INFO:teuthology.orchestra.run.smithi203.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-04-25T08:48:44.259 INFO:teuthology.orchestra.run.smithi037.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,398 B] 2024-04-25T08:48:44.441 INFO:teuthology.orchestra.run.smithi037.stdout:Fetched 28.9 kB in 0s (78.5 kB/s) 2024-04-25T08:48:44.649 INFO:teuthology.orchestra.run.smithi203.stdout:Fetched 139 kB in 1s (249 kB/s) 2024-04-25T08:48:45.417 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2024-04-25T08:48:45.434 DEBUG:teuthology.orchestra.run.smithi037:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:48:45.489 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2024-04-25T08:48:45.644 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2024-04-25T08:48:45.659 DEBUG:teuthology.orchestra.run.smithi203:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1628-g97ae6cc9-1jammy cephadm=19.0.0-1628-g97ae6cc9-1jammy ceph-mds=19.0.0-1628-g97ae6cc9-1jammy ceph-mgr=19.0.0-1628-g97ae6cc9-1jammy ceph-common=19.0.0-1628-g97ae6cc9-1jammy ceph-fuse=19.0.0-1628-g97ae6cc9-1jammy ceph-test=19.0.0-1628-g97ae6cc9-1jammy ceph-volume=19.0.0-1628-g97ae6cc9-1jammy radosgw=19.0.0-1628-g97ae6cc9-1jammy python3-rados=19.0.0-1628-g97ae6cc9-1jammy python3-rgw=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy python3-rbd=19.0.0-1628-g97ae6cc9-1jammy libcephfs2=19.0.0-1628-g97ae6cc9-1jammy libcephfs-dev=19.0.0-1628-g97ae6cc9-1jammy librados2=19.0.0-1628-g97ae6cc9-1jammy librbd1=19.0.0-1628-g97ae6cc9-1jammy rbd-fuse=19.0.0-1628-g97ae6cc9-1jammy python3-cephfs=19.0.0-1628-g97ae6cc9-1jammy cephfs-shell=19.0.0-1628-g97ae6cc9-1jammy cephfs-top=19.0.0-1628-g97ae6cc9-1jammy cephfs-mirror=19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:48:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:Building dependency tree... 2024-04-25T08:48:45.670 INFO:teuthology.orchestra.run.smithi037.stdout:Reading state information... 2024-04-25T08:48:45.713 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2024-04-25T08:48:45.852 INFO:teuthology.orchestra.run.smithi037.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:48:45.852 INFO:teuthology.orchestra.run.smithi037.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:48:45.852 INFO:teuthology.orchestra.run.smithi037.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:48:45.852 INFO:teuthology.orchestra.run.smithi037.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:48:45.853 INFO:teuthology.orchestra.run.smithi037.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:48:45.853 INFO:teuthology.orchestra.run.smithi037.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:48:45.853 INFO:teuthology.orchestra.run.smithi037.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout:The following additional packages will be installed: 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T08:48:45.855 INFO:teuthology.orchestra.run.smithi037.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout:Suggested packages: 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T08:48:45.856 INFO:teuthology.orchestra.run.smithi037.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T08:48:45.857 INFO:teuthology.orchestra.run.smithi037.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T08:48:45.857 INFO:teuthology.orchestra.run.smithi037.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T08:48:45.857 INFO:teuthology.orchestra.run.smithi037.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T08:48:45.857 INFO:teuthology.orchestra.run.smithi037.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T08:48:45.857 INFO:teuthology.orchestra.run.smithi037.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T08:48:45.857 INFO:teuthology.orchestra.run.smithi037.stdout:Recommended packages: 2024-04-25T08:48:45.857 INFO:teuthology.orchestra.run.smithi037.stdout: btrfs-tools 2024-04-25T08:48:45.889 INFO:teuthology.orchestra.run.smithi203.stdout:Building dependency tree... 2024-04-25T08:48:45.890 INFO:teuthology.orchestra.run.smithi203.stdout:Reading state information... 2024-04-25T08:48:45.964 INFO:teuthology.orchestra.run.smithi037.stdout:The following NEW packages will be installed: 2024-04-25T08:48:45.964 INFO:teuthology.orchestra.run.smithi037.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T08:48:45.964 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T08:48:45.964 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T08:48:45.964 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T08:48:45.964 INFO:teuthology.orchestra.run.smithi037.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:48:45.965 INFO:teuthology.orchestra.run.smithi037.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T08:48:45.965 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T08:48:45.965 INFO:teuthology.orchestra.run.smithi037.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T08:48:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T08:48:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T08:48:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T08:48:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T08:48:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T08:48:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T08:48:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T08:48:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T08:48:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T08:48:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T08:48:45.966 INFO:teuthology.orchestra.run.smithi037.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T08:48:45.967 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T08:48:45.967 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T08:48:45.967 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T08:48:45.967 INFO:teuthology.orchestra.run.smithi037.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T08:48:45.967 INFO:teuthology.orchestra.run.smithi037.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T08:48:45.967 INFO:teuthology.orchestra.run.smithi037.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T08:48:45.967 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T08:48:45.967 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T08:48:45.967 INFO:teuthology.orchestra.run.smithi037.stdout: socat unzip xmlstarlet zip 2024-04-25T08:48:46.032 INFO:teuthology.orchestra.run.smithi037.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T08:48:46.032 INFO:teuthology.orchestra.run.smithi037.stdout:Need to get 176 MB of archives. 2024-04-25T08:48:46.032 INFO:teuthology.orchestra.run.smithi037.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T08:48:46.033 INFO:teuthology.orchestra.run.smithi037.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T08:48:46.054 INFO:teuthology.orchestra.run.smithi037.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T08:48:46.061 INFO:teuthology.orchestra.run.smithi203.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:48:46.061 INFO:teuthology.orchestra.run.smithi203.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:48:46.062 INFO:teuthology.orchestra.run.smithi203.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:48:46.062 INFO:teuthology.orchestra.run.smithi203.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:48:46.062 INFO:teuthology.orchestra.run.smithi203.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:48:46.062 INFO:teuthology.orchestra.run.smithi203.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:48:46.062 INFO:teuthology.orchestra.run.smithi203.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:48:46.063 INFO:teuthology.orchestra.run.smithi203.stdout:The following additional packages will be installed: 2024-04-25T08:48:46.063 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T08:48:46.063 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T08:48:46.063 INFO:teuthology.orchestra.run.smithi203.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:48:46.063 INFO:teuthology.orchestra.run.smithi203.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T08:48:46.064 INFO:teuthology.orchestra.run.smithi203.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T08:48:46.064 INFO:teuthology.orchestra.run.smithi203.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T08:48:46.065 INFO:teuthology.orchestra.run.smithi203.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T08:48:46.066 INFO:teuthology.orchestra.run.smithi203.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T08:48:46.066 INFO:teuthology.orchestra.run.smithi203.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T08:48:46.066 INFO:teuthology.orchestra.run.smithi203.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T08:48:46.066 INFO:teuthology.orchestra.run.smithi203.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T08:48:46.066 INFO:teuthology.orchestra.run.smithi037.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T08:48:46.066 INFO:teuthology.orchestra.run.smithi203.stdout:Suggested packages: 2024-04-25T08:48:46.067 INFO:teuthology.orchestra.run.smithi203.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T08:48:46.067 INFO:teuthology.orchestra.run.smithi203.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T08:48:46.067 INFO:teuthology.orchestra.run.smithi203.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T08:48:46.067 INFO:teuthology.orchestra.run.smithi203.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T08:48:46.067 INFO:teuthology.orchestra.run.smithi203.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T08:48:46.067 INFO:teuthology.orchestra.run.smithi203.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T08:48:46.067 INFO:teuthology.orchestra.run.smithi203.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T08:48:46.067 INFO:teuthology.orchestra.run.smithi203.stdout:Recommended packages: 2024-04-25T08:48:46.067 INFO:teuthology.orchestra.run.smithi203.stdout: btrfs-tools 2024-04-25T08:48:46.122 INFO:teuthology.orchestra.run.smithi037.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T08:48:46.125 INFO:teuthology.orchestra.run.smithi037.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T08:48:46.145 INFO:teuthology.orchestra.run.smithi037.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T08:48:46.149 INFO:teuthology.orchestra.run.smithi037.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T08:48:46.163 INFO:teuthology.orchestra.run.smithi037.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T08:48:46.171 INFO:teuthology.orchestra.run.smithi203.stdout:The following NEW packages will be installed: 2024-04-25T08:48:46.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T08:48:46.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T08:48:46.171 INFO:teuthology.orchestra.run.smithi203.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T08:48:46.171 INFO:teuthology.orchestra.run.smithi203.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T08:48:46.172 INFO:teuthology.orchestra.run.smithi203.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T08:48:46.172 INFO:teuthology.orchestra.run.smithi203.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T08:48:46.172 INFO:teuthology.orchestra.run.smithi203.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T08:48:46.173 INFO:teuthology.orchestra.run.smithi203.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T08:48:46.174 INFO:teuthology.orchestra.run.smithi203.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T08:48:46.174 INFO:teuthology.orchestra.run.smithi203.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T08:48:46.174 INFO:teuthology.orchestra.run.smithi203.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T08:48:46.174 INFO:teuthology.orchestra.run.smithi203.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T08:48:46.174 INFO:teuthology.orchestra.run.smithi203.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T08:48:46.174 INFO:teuthology.orchestra.run.smithi203.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T08:48:46.174 INFO:teuthology.orchestra.run.smithi203.stdout: socat unzip xmlstarlet zip 2024-04-25T08:48:46.228 INFO:teuthology.orchestra.run.smithi037.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T08:48:46.234 INFO:teuthology.orchestra.run.smithi037.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T08:48:46.247 INFO:teuthology.orchestra.run.smithi037.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T08:48:46.252 INFO:teuthology.orchestra.run.smithi037.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T08:48:46.252 INFO:teuthology.orchestra.run.smithi037.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T08:48:46.262 INFO:teuthology.orchestra.run.smithi037.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T08:48:46.264 INFO:teuthology.orchestra.run.smithi037.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T08:48:46.266 INFO:teuthology.orchestra.run.smithi203.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T08:48:46.266 INFO:teuthology.orchestra.run.smithi203.stdout:Need to get 176 MB of archives. 2024-04-25T08:48:46.267 INFO:teuthology.orchestra.run.smithi203.stdout:After this operation, 732 MB of additional disk space will be used. 2024-04-25T08:48:46.267 INFO:teuthology.orchestra.run.smithi203.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T08:48:46.267 INFO:teuthology.orchestra.run.smithi037.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T08:48:46.268 INFO:teuthology.orchestra.run.smithi037.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T08:48:46.269 INFO:teuthology.orchestra.run.smithi037.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T08:48:46.293 INFO:teuthology.orchestra.run.smithi203.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T08:48:46.298 INFO:teuthology.orchestra.run.smithi037.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T08:48:46.301 INFO:teuthology.orchestra.run.smithi037.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T08:48:46.304 INFO:teuthology.orchestra.run.smithi037.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T08:48:46.305 INFO:teuthology.orchestra.run.smithi037.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T08:48:46.306 INFO:teuthology.orchestra.run.smithi037.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T08:48:46.307 INFO:teuthology.orchestra.run.smithi037.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T08:48:46.308 INFO:teuthology.orchestra.run.smithi037.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T08:48:46.308 INFO:teuthology.orchestra.run.smithi037.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T08:48:46.309 INFO:teuthology.orchestra.run.smithi037.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T08:48:46.314 INFO:teuthology.orchestra.run.smithi203.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,533 kB] 2024-04-25T08:48:46.318 INFO:teuthology.orchestra.run.smithi203.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T08:48:46.325 INFO:teuthology.orchestra.run.smithi037.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T08:48:46.339 INFO:teuthology.orchestra.run.smithi037.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T08:48:46.339 INFO:teuthology.orchestra.run.smithi037.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T08:48:46.343 INFO:teuthology.orchestra.run.smithi037.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T08:48:46.344 INFO:teuthology.orchestra.run.smithi037.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T08:48:46.345 INFO:teuthology.orchestra.run.smithi037.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T08:48:46.346 INFO:teuthology.orchestra.run.smithi037.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T08:48:46.347 INFO:teuthology.orchestra.run.smithi037.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T08:48:46.347 INFO:teuthology.orchestra.run.smithi037.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T08:48:46.352 INFO:teuthology.orchestra.run.smithi037.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T08:48:46.369 INFO:teuthology.orchestra.run.smithi037.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T08:48:46.371 INFO:teuthology.orchestra.run.smithi037.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T08:48:46.371 INFO:teuthology.orchestra.run.smithi037.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T08:48:46.379 INFO:teuthology.orchestra.run.smithi037.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T08:48:46.380 INFO:teuthology.orchestra.run.smithi037.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T08:48:46.380 INFO:teuthology.orchestra.run.smithi037.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T08:48:46.381 INFO:teuthology.orchestra.run.smithi037.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T08:48:46.382 INFO:teuthology.orchestra.run.smithi203.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T08:48:46.383 INFO:teuthology.orchestra.run.smithi037.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T08:48:46.386 INFO:teuthology.orchestra.run.smithi037.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T08:48:46.401 INFO:teuthology.orchestra.run.smithi037.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T08:48:46.404 INFO:teuthology.orchestra.run.smithi037.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T08:48:46.405 INFO:teuthology.orchestra.run.smithi037.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T08:48:46.407 INFO:teuthology.orchestra.run.smithi203.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T08:48:46.411 INFO:teuthology.orchestra.run.smithi203.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T08:48:46.423 INFO:teuthology.orchestra.run.smithi037.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T08:48:46.424 INFO:teuthology.orchestra.run.smithi037.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T08:48:46.425 INFO:teuthology.orchestra.run.smithi037.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T08:48:46.428 INFO:teuthology.orchestra.run.smithi037.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T08:48:46.434 INFO:teuthology.orchestra.run.smithi203.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T08:48:46.502 INFO:teuthology.orchestra.run.smithi037.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T08:48:46.503 INFO:teuthology.orchestra.run.smithi037.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T08:48:46.505 INFO:teuthology.orchestra.run.smithi037.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T08:48:46.505 INFO:teuthology.orchestra.run.smithi037.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T08:48:46.507 INFO:teuthology.orchestra.run.smithi037.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T08:48:46.544 INFO:teuthology.orchestra.run.smithi037.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T08:48:46.548 INFO:teuthology.orchestra.run.smithi037.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T08:48:46.548 INFO:teuthology.orchestra.run.smithi037.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T08:48:46.556 INFO:teuthology.orchestra.run.smithi203.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T08:48:46.559 INFO:teuthology.orchestra.run.smithi203.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T08:48:46.571 INFO:teuthology.orchestra.run.smithi203.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T08:48:46.576 INFO:teuthology.orchestra.run.smithi037.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T08:48:46.577 INFO:teuthology.orchestra.run.smithi037.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T08:48:46.577 INFO:teuthology.orchestra.run.smithi037.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T08:48:46.578 INFO:teuthology.orchestra.run.smithi037.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T08:48:46.579 INFO:teuthology.orchestra.run.smithi037.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T08:48:46.579 INFO:teuthology.orchestra.run.smithi037.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T08:48:46.594 INFO:teuthology.orchestra.run.smithi203.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T08:48:46.601 INFO:teuthology.orchestra.run.smithi203.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T08:48:46.602 INFO:teuthology.orchestra.run.smithi203.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T08:48:46.602 INFO:teuthology.orchestra.run.smithi037.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T08:48:46.603 INFO:teuthology.orchestra.run.smithi037.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T08:48:46.608 INFO:teuthology.orchestra.run.smithi037.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T08:48:46.621 INFO:teuthology.orchestra.run.smithi203.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T08:48:46.625 INFO:teuthology.orchestra.run.smithi037.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T08:48:46.625 INFO:teuthology.orchestra.run.smithi203.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T08:48:46.627 INFO:teuthology.orchestra.run.smithi037.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T08:48:46.627 INFO:teuthology.orchestra.run.smithi037.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T08:48:46.629 INFO:teuthology.orchestra.run.smithi203.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T08:48:46.631 INFO:teuthology.orchestra.run.smithi203.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T08:48:46.631 INFO:teuthology.orchestra.run.smithi203.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T08:48:46.632 INFO:teuthology.orchestra.run.smithi037.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T08:48:46.635 INFO:teuthology.orchestra.run.smithi203.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T08:48:46.636 INFO:teuthology.orchestra.run.smithi037.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T08:48:46.636 INFO:teuthology.orchestra.run.smithi037.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T08:48:46.638 INFO:teuthology.orchestra.run.smithi037.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T08:48:46.640 INFO:teuthology.orchestra.run.smithi203.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T08:48:46.643 INFO:teuthology.orchestra.run.smithi203.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T08:48:46.644 INFO:teuthology.orchestra.run.smithi203.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T08:48:46.653 INFO:teuthology.orchestra.run.smithi037.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T08:48:46.654 INFO:teuthology.orchestra.run.smithi203.stdout:Get:24 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T08:48:46.659 INFO:teuthology.orchestra.run.smithi037.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T08:48:46.659 INFO:teuthology.orchestra.run.smithi037.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T08:48:46.661 INFO:teuthology.orchestra.run.smithi203.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T08:48:46.663 INFO:teuthology.orchestra.run.smithi203.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T08:48:46.663 INFO:teuthology.orchestra.run.smithi203.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T08:48:46.664 INFO:teuthology.orchestra.run.smithi203.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T08:48:46.673 INFO:teuthology.orchestra.run.smithi037.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T08:48:46.674 INFO:teuthology.orchestra.run.smithi037.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T08:48:46.675 INFO:teuthology.orchestra.run.smithi037.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T08:48:46.675 INFO:teuthology.orchestra.run.smithi037.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T08:48:46.676 INFO:teuthology.orchestra.run.smithi203.stdout:Get:29 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T08:48:46.677 INFO:teuthology.orchestra.run.smithi037.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T08:48:46.678 INFO:teuthology.orchestra.run.smithi037.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T08:48:46.684 INFO:teuthology.orchestra.run.smithi203.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T08:48:46.684 INFO:teuthology.orchestra.run.smithi203.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T08:48:46.685 INFO:teuthology.orchestra.run.smithi203.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T08:48:46.685 INFO:teuthology.orchestra.run.smithi203.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T08:48:46.687 INFO:teuthology.orchestra.run.smithi037.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T08:48:46.688 INFO:teuthology.orchestra.run.smithi203.stdout:Get:34 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T08:48:46.689 INFO:teuthology.orchestra.run.smithi203.stdout:Get:35 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T08:48:46.689 INFO:teuthology.orchestra.run.smithi203.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T08:48:46.690 INFO:teuthology.orchestra.run.smithi203.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T08:48:46.695 INFO:teuthology.orchestra.run.smithi037.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T08:48:46.695 INFO:teuthology.orchestra.run.smithi037.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T08:48:46.698 INFO:teuthology.orchestra.run.smithi037.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T08:48:46.700 INFO:teuthology.orchestra.run.smithi203.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T08:48:46.704 INFO:teuthology.orchestra.run.smithi203.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T08:48:46.705 INFO:teuthology.orchestra.run.smithi203.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T08:48:46.706 INFO:teuthology.orchestra.run.smithi203.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T08:48:46.707 INFO:teuthology.orchestra.run.smithi203.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T08:48:46.713 INFO:teuthology.orchestra.run.smithi203.stdout:Get:43 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T08:48:46.715 INFO:teuthology.orchestra.run.smithi037.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T08:48:46.729 INFO:teuthology.orchestra.run.smithi203.stdout:Get:44 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T08:48:46.732 INFO:teuthology.orchestra.run.smithi203.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T08:48:46.734 INFO:teuthology.orchestra.run.smithi203.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T08:48:46.736 INFO:teuthology.orchestra.run.smithi203.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T08:48:46.737 INFO:teuthology.orchestra.run.smithi203.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T08:48:46.747 INFO:teuthology.orchestra.run.smithi203.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T08:48:46.748 INFO:teuthology.orchestra.run.smithi203.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T08:48:46.753 INFO:teuthology.orchestra.run.smithi203.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T08:48:46.754 INFO:teuthology.orchestra.run.smithi203.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T08:48:46.756 INFO:teuthology.orchestra.run.smithi203.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T08:48:46.760 INFO:teuthology.orchestra.run.smithi203.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T08:48:46.777 INFO:teuthology.orchestra.run.smithi203.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T08:48:46.780 INFO:teuthology.orchestra.run.smithi203.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T08:48:46.784 INFO:teuthology.orchestra.run.smithi203.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T08:48:46.789 INFO:teuthology.orchestra.run.smithi203.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T08:48:46.790 INFO:teuthology.orchestra.run.smithi203.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T08:48:46.791 INFO:teuthology.orchestra.run.smithi203.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T08:48:46.798 INFO:teuthology.orchestra.run.smithi203.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T08:48:46.822 INFO:teuthology.orchestra.run.smithi037.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1628-g97ae6cc9-1jammy [3,217 kB] 2024-04-25T08:48:46.872 INFO:teuthology.orchestra.run.smithi203.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T08:48:46.872 INFO:teuthology.orchestra.run.smithi203.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T08:48:46.874 INFO:teuthology.orchestra.run.smithi203.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T08:48:46.874 INFO:teuthology.orchestra.run.smithi203.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T08:48:46.875 INFO:teuthology.orchestra.run.smithi203.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T08:48:46.893 INFO:teuthology.orchestra.run.smithi203.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T08:48:46.895 INFO:teuthology.orchestra.run.smithi203.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T08:48:46.895 INFO:teuthology.orchestra.run.smithi203.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T08:48:46.910 INFO:teuthology.orchestra.run.smithi203.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T08:48:46.910 INFO:teuthology.orchestra.run.smithi203.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T08:48:46.921 INFO:teuthology.orchestra.run.smithi203.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T08:48:46.922 INFO:teuthology.orchestra.run.smithi203.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T08:48:46.923 INFO:teuthology.orchestra.run.smithi203.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T08:48:46.924 INFO:teuthology.orchestra.run.smithi203.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T08:48:46.942 INFO:teuthology.orchestra.run.smithi203.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T08:48:46.942 INFO:teuthology.orchestra.run.smithi203.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T08:48:46.946 INFO:teuthology.orchestra.run.smithi203.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T08:48:46.948 INFO:teuthology.orchestra.run.smithi203.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T08:48:46.950 INFO:teuthology.orchestra.run.smithi203.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T08:48:46.950 INFO:teuthology.orchestra.run.smithi203.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T08:48:46.965 INFO:teuthology.orchestra.run.smithi203.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T08:48:46.968 INFO:teuthology.orchestra.run.smithi203.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T08:48:46.969 INFO:teuthology.orchestra.run.smithi203.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T08:48:46.969 INFO:teuthology.orchestra.run.smithi203.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T08:48:46.980 INFO:teuthology.orchestra.run.smithi203.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T08:48:46.983 INFO:teuthology.orchestra.run.smithi203.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T08:48:46.985 INFO:teuthology.orchestra.run.smithi203.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T08:48:46.991 INFO:teuthology.orchestra.run.smithi203.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T08:48:46.991 INFO:teuthology.orchestra.run.smithi203.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T08:48:46.995 INFO:teuthology.orchestra.run.smithi203.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T08:48:46.995 INFO:teuthology.orchestra.run.smithi203.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T08:48:46.995 INFO:teuthology.orchestra.run.smithi203.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T08:48:46.997 INFO:teuthology.orchestra.run.smithi203.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T08:48:47.006 INFO:teuthology.orchestra.run.smithi203.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T08:48:47.008 INFO:teuthology.orchestra.run.smithi203.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T08:48:47.009 INFO:teuthology.orchestra.run.smithi203.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T08:48:47.018 INFO:teuthology.orchestra.run.smithi203.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T08:48:47.026 INFO:teuthology.orchestra.run.smithi203.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T08:48:47.027 INFO:teuthology.orchestra.run.smithi203.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T08:48:47.029 INFO:teuthology.orchestra.run.smithi203.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T08:48:47.029 INFO:teuthology.orchestra.run.smithi203.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T08:48:47.630 INFO:teuthology.orchestra.run.smithi037.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1628-g97ae6cc9-1jammy [709 kB] 2024-04-25T08:48:47.724 INFO:teuthology.orchestra.run.smithi037.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1628-g97ae6cc9-1jammy [356 kB] 2024-04-25T08:48:47.772 INFO:teuthology.orchestra.run.smithi037.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1628-g97ae6cc9-1jammy [32.8 kB] 2024-04-25T08:48:47.774 INFO:teuthology.orchestra.run.smithi037.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1628-g97ae6cc9-1jammy [180 kB] 2024-04-25T08:48:47.813 INFO:teuthology.orchestra.run.smithi037.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1628-g97ae6cc9-1jammy [64.8 kB] 2024-04-25T08:48:47.816 INFO:teuthology.orchestra.run.smithi037.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1628-g97ae6cc9-1jammy [330 kB] 2024-04-25T08:48:47.864 INFO:teuthology.orchestra.run.smithi037.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1628-g97ae6cc9-1jammy [6,575 kB] 2024-04-25T08:48:48.031 INFO:teuthology.orchestra.run.smithi203.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T08:48:48.176 INFO:teuthology.orchestra.run.smithi203.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T08:48:48.189 INFO:teuthology.orchestra.run.smithi203.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T08:48:48.196 INFO:teuthology.orchestra.run.smithi203.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T08:48:48.258 INFO:teuthology.orchestra.run.smithi203.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T08:48:48.528 INFO:teuthology.orchestra.run.smithi203.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T08:48:48.748 INFO:teuthology.orchestra.run.smithi037.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1628-g97ae6cc9-1jammy [112 kB] 2024-04-25T08:48:48.773 INFO:teuthology.orchestra.run.smithi037.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1628-g97ae6cc9-1jammy [467 kB] 2024-04-25T08:48:48.838 INFO:teuthology.orchestra.run.smithi037.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1628-g97ae6cc9-1jammy [24.1 MB] 2024-04-25T08:48:49.599 INFO:teuthology.orchestra.run.smithi203.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T08:48:49.599 INFO:teuthology.orchestra.run.smithi203.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T08:48:49.618 INFO:teuthology.orchestra.run.smithi203.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T08:48:49.658 INFO:teuthology.orchestra.run.smithi203.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T08:48:49.667 INFO:teuthology.orchestra.run.smithi203.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T08:48:49.669 INFO:teuthology.orchestra.run.smithi203.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T08:48:49.720 INFO:teuthology.orchestra.run.smithi203.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T08:48:50.077 INFO:teuthology.orchestra.run.smithi203.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T08:48:50.077 INFO:teuthology.orchestra.run.smithi203.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T08:48:52.644 INFO:teuthology.orchestra.run.smithi203.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T08:48:52.646 INFO:teuthology.orchestra.run.smithi203.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T08:48:52.651 INFO:teuthology.orchestra.run.smithi203.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T08:48:52.652 INFO:teuthology.orchestra.run.smithi203.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T08:48:52.652 INFO:teuthology.orchestra.run.smithi203.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T08:48:52.653 INFO:teuthology.orchestra.run.smithi203.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T08:48:53.057 INFO:teuthology.orchestra.run.smithi203.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T08:48:53.203 INFO:teuthology.orchestra.run.smithi037.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1628-g97ae6cc9-1jammy [5,076 kB] 2024-04-25T08:48:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1628-g97ae6cc9-1jammy [243 kB] 2024-04-25T08:48:53.423 INFO:teuthology.orchestra.run.smithi037.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [124 kB] 2024-04-25T08:48:53.435 INFO:teuthology.orchestra.run.smithi037.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1628-g97ae6cc9-1jammy [1,520 kB] 2024-04-25T08:48:53.509 INFO:teuthology.orchestra.run.smithi037.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1628-g97ae6cc9-1jammy [6,237 kB] 2024-04-25T08:48:53.642 INFO:teuthology.orchestra.run.smithi203.stdout:Fetched 176 MB in 7s (25.6 MB/s) 2024-04-25T08:48:53.756 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T08:48:53.811 INFO:teuthology.orchestra.run.smithi203.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T08:48:53.816 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T08:48:53.838 INFO:teuthology.orchestra.run.smithi037.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1628-g97ae6cc9-1jammy [22.7 MB] 2024-04-25T08:48:53.847 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T08:48:54.108 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T08:48:54.134 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T08:48:54.174 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:48:54.494 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T08:48:54.520 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T08:48:54.560 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T08:48:54.922 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T08:48:54.948 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T08:48:55.031 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:48:55.219 INFO:teuthology.orchestra.run.smithi037.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1628-g97ae6cc9-1jammy [14.0 kB] 2024-04-25T08:48:55.220 INFO:teuthology.orchestra.run.smithi037.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [866 kB] 2024-04-25T08:48:55.237 INFO:teuthology.orchestra.run.smithi037.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1628-g97ae6cc9-1jammy [2,191 kB] 2024-04-25T08:48:55.338 INFO:teuthology.orchestra.run.smithi037.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1628-g97ae6cc9-1jammy [776 kB] 2024-04-25T08:48:55.368 INFO:teuthology.orchestra.run.smithi037.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1628-g97ae6cc9-1jammy [144 kB] 2024-04-25T08:48:55.375 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T08:48:55.381 INFO:teuthology.orchestra.run.smithi037.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1628-g97ae6cc9-1jammy [1,921 kB] 2024-04-25T08:48:55.401 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T08:48:55.442 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:48:55.463 INFO:teuthology.orchestra.run.smithi037.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1628-g97ae6cc9-1jammy [8,625 kB] 2024-04-25T08:48:55.703 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T08:48:55.718 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T08:48:55.761 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:48:55.880 INFO:teuthology.orchestra.run.smithi037.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1628-g97ae6cc9-1jammy [14.2 kB] 2024-04-25T08:48:55.880 INFO:teuthology.orchestra.run.smithi037.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1628-g97ae6cc9-1jammy [49.9 MB] 2024-04-25T08:48:56.207 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T08:48:56.233 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:48:56.281 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:48:56.701 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T08:48:56.728 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:48:56.768 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:48:57.079 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T08:48:57.105 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:48:57.171 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:48:57.524 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T08:48:57.539 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T08:48:57.582 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:48:57.910 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package librados2. 2024-04-25T08:48:57.936 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:48:58.161 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:48:58.433 INFO:teuthology.orchestra.run.smithi037.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1628-g97ae6cc9-1jammy [121 kB] 2024-04-25T08:48:58.496 INFO:teuthology.orchestra.run.smithi037.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1628-g97ae6cc9-1jammy [228 kB] 2024-04-25T08:48:58.563 INFO:teuthology.orchestra.run.smithi037.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1628-g97ae6cc9-1jammy [34.0 kB] 2024-04-25T08:48:58.564 INFO:teuthology.orchestra.run.smithi037.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1628-g97ae6cc9-1jammy [23.4 kB] 2024-04-25T08:48:58.565 INFO:teuthology.orchestra.run.smithi037.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1628-g97ae6cc9-1jammy [38.4 kB] 2024-04-25T08:48:58.566 INFO:teuthology.orchestra.run.smithi037.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1628-g97ae6cc9-1jammy [13.0 MB] 2024-04-25T08:48:58.638 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package librbd1. 2024-04-25T08:48:58.664 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:48:58.704 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:48:58.988 INFO:teuthology.orchestra.run.smithi037.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1628-g97ae6cc9-1jammy [91.5 kB] 2024-04-25T08:48:59.124 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T08:48:59.139 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:48:59.174 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:48:59.409 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-rados. 2024-04-25T08:48:59.432 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:48:59.476 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:48:59.630 INFO:teuthology.orchestra.run.smithi037.stdout:Fetched 176 MB in 13s (13.6 MB/s) 2024-04-25T08:48:59.774 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T08:48:59.795 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T08:48:59.822 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:48:59.850 INFO:teuthology.orchestra.run.smithi037.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T08:48:59.855 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T08:48:59.862 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:48:59.888 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T08:49:00.123 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T08:49:00.138 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:00.141 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T08:49:00.157 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T08:49:00.181 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:00.191 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:49:00.475 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T08:49:00.490 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:00.511 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T08:49:00.525 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:00.538 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T08:49:00.577 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T08:49:00.837 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T08:49:00.863 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T08:49:00.895 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T08:49:00.931 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T08:49:00.957 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T08:49:01.039 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:49:01.156 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T08:49:01.182 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:01.222 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:01.384 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T08:49:01.411 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T08:49:01.451 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:49:01.567 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T08:49:01.593 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T08:49:01.634 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:49:01.737 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T08:49:01.764 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T08:49:01.795 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:49:01.886 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T08:49:01.901 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T08:49:01.944 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:49:02.248 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T08:49:02.274 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T08:49:02.282 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T08:49:02.309 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:49:02.314 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:49:02.358 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:49:02.659 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package lua5.1. 2024-04-25T08:49:02.685 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T08:49:02.725 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:49:02.752 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T08:49:02.779 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:49:02.927 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:49:03.020 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package lua-any. 2024-04-25T08:49:03.035 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T08:49:03.103 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T08:49:03.255 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T08:49:03.282 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T08:49:03.313 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:49:03.358 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package zip. 2024-04-25T08:49:03.385 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T08:49:03.425 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T08:49:03.675 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T08:49:03.702 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T08:49:03.728 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package unzip. 2024-04-25T08:49:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:49:03.755 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T08:49:03.795 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:49:04.183 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package luarocks. 2024-04-25T08:49:04.209 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T08:49:04.236 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package librados2. 2024-04-25T08:49:04.249 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:49:04.263 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../010-librados2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:04.294 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:04.641 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package librgw2. 2024-04-25T08:49:04.668 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:04.708 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:04.747 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package librbd1. 2024-04-25T08:49:04.774 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../011-librbd1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:04.822 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:05.235 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T08:49:05.251 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:05.283 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T08:49:05.285 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:05.310 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:05.350 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:05.627 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-rados. 2024-04-25T08:49:05.643 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:05.655 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T08:49:05.677 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:05.681 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T08:49:05.722 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:49:05.972 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T08:49:05.998 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:06.038 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:06.050 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T08:49:06.076 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:06.117 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:06.291 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T08:49:06.318 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:06.357 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:06.579 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-common. 2024-04-25T08:49:06.605 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:06.645 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:06.694 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T08:49:06.720 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:06.760 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:07.097 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T08:49:07.124 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T08:49:07.163 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T08:49:07.382 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T08:49:07.398 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:07.432 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:07.760 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T08:49:07.787 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T08:49:07.827 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:49:07.909 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-base. 2024-04-25T08:49:07.936 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:07.984 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:08.113 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T08:49:08.129 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T08:49:08.171 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:49:08.474 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T08:49:08.490 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T08:49:08.524 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:49:08.764 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T08:49:08.790 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T08:49:08.827 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package lua5.1. 2024-04-25T08:49:08.830 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:49:08.854 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T08:49:08.894 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:49:09.125 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T08:49:09.151 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T08:49:09.192 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:49:09.372 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package lua-any. 2024-04-25T08:49:09.399 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T08:49:09.438 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T08:49:09.461 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T08:49:09.488 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T08:49:09.528 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:49:09.691 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package zip. 2024-04-25T08:49:09.708 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T08:49:09.749 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T08:49:09.831 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T08:49:09.857 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T08:49:09.897 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:49:10.019 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package unzip. 2024-04-25T08:49:10.046 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T08:49:10.078 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:49:10.209 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T08:49:10.236 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T08:49:10.276 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:49:10.507 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package luarocks. 2024-04-25T08:49:10.534 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T08:49:10.554 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T08:49:10.573 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:49:10.580 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T08:49:10.612 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T08:49:10.915 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-portend. 2024-04-25T08:49:10.939 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package librgw2. 2024-04-25T08:49:10.941 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T08:49:10.966 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../027-librgw2_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:10.981 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T08:49:11.005 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:11.302 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T08:49:11.328 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T08:49:11.368 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T08:49:11.567 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T08:49:11.583 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:11.616 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:11.638 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T08:49:11.665 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T08:49:11.696 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:49:11.911 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T08:49:11.938 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T08:49:11.977 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:49:12.042 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T08:49:12.069 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T08:49:12.108 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T08:49:12.297 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T08:49:12.324 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:12.364 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:12.420 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T08:49:12.447 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T08:49:12.487 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T08:49:12.740 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T08:49:12.755 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T08:49:12.776 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-common. 2024-04-25T08:49:12.798 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:49:12.803 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:12.842 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:13.093 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-mako. 2024-04-25T08:49:13.120 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T08:49:13.159 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:49:13.496 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T08:49:13.523 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T08:49:13.562 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:49:13.866 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T08:49:13.881 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T08:49:13.916 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:49:14.114 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-base. 2024-04-25T08:49:14.141 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:14.190 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:14.194 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-webob. 2024-04-25T08:49:14.220 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T08:49:14.261 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:49:14.639 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T08:49:14.665 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T08:49:14.708 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:49:14.970 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T08:49:14.986 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T08:49:15.008 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T08:49:15.028 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:49:15.035 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T08:49:15.075 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:49:15.298 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T08:49:15.325 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T08:49:15.364 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:49:15.386 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-paste. 2024-04-25T08:49:15.413 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T08:49:15.453 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:49:15.684 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T08:49:15.711 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T08:49:15.750 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:49:15.781 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T08:49:15.796 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T08:49:15.839 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:49:16.054 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T08:49:16.081 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T08:49:16.120 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:49:16.176 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T08:49:16.203 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T08:49:16.243 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:49:16.381 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T08:49:16.397 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T08:49:16.431 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:49:16.513 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T08:49:16.539 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T08:49:16.571 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T08:49:16.701 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T08:49:16.728 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T08:49:16.767 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T08:49:16.882 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T08:49:16.909 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T08:49:16.949 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:49:17.070 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-portend. 2024-04-25T08:49:17.098 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T08:49:17.137 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T08:49:17.304 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T08:49:17.331 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T08:49:17.371 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T08:49:17.432 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T08:49:17.459 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T08:49:17.498 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T08:49:17.699 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-idna. 2024-04-25T08:49:17.726 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T08:49:17.751 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T08:49:17.765 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T08:49:17.778 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T08:49:17.817 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:49:18.085 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T08:49:18.112 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T08:49:18.129 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T08:49:18.145 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T08:49:18.152 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:49:18.179 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T08:49:18.480 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-requests. 2024-04-25T08:49:18.507 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T08:49:18.547 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:49:18.850 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T08:49:18.877 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T08:49:18.916 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:49:19.116 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T08:49:19.144 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T08:49:19.183 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T08:49:19.370 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T08:49:19.386 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:19.428 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:19.461 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T08:49:19.488 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T08:49:19.519 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:49:19.805 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-mako. 2024-04-25T08:49:19.832 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T08:49:19.872 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:49:20.183 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T08:49:20.211 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T08:49:20.249 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:49:20.553 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T08:49:20.580 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T08:49:20.592 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T08:49:20.619 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:20.619 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:49:20.667 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:20.863 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-webob. 2024-04-25T08:49:20.891 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T08:49:20.922 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:49:21.003 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T08:49:21.030 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T08:49:21.078 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:49:21.250 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T08:49:21.277 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T08:49:21.320 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:49:21.356 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T08:49:21.372 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T08:49:21.406 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T08:49:21.586 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T08:49:21.614 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T08:49:21.644 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:49:21.734 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T08:49:21.762 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:21.801 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:21.931 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-paste. 2024-04-25T08:49:21.958 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T08:49:21.997 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:49:22.213 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T08:49:22.240 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:22.279 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:22.292 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T08:49:22.319 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T08:49:22.350 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:49:22.628 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T08:49:22.656 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T08:49:22.694 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:49:22.850 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T08:49:22.877 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:22.916 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:22.998 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T08:49:23.025 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T08:49:23.066 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T08:49:23.325 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T08:49:23.341 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T08:49:23.383 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:49:23.714 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T08:49:23.741 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T08:49:23.780 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T08:49:23.986 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph. 2024-04-25T08:49:24.013 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:24.053 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:24.082 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-idna. 2024-04-25T08:49:24.099 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T08:49:24.141 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T08:49:24.364 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T08:49:24.390 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:24.430 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:24.461 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T08:49:24.488 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T08:49:24.644 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:49:24.809 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T08:49:24.835 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:24.884 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:25.031 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-requests. 2024-04-25T08:49:25.058 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T08:49:25.097 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:49:25.254 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package cephadm. 2024-04-25T08:49:25.280 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:25.320 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:25.375 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T08:49:25.402 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T08:49:25.433 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:49:25.656 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T08:49:25.683 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T08:49:25.723 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T08:49:25.728 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T08:49:25.755 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:25.794 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:26.060 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T08:49:26.086 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:49:26.126 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:49:26.141 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T08:49:26.168 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:26.199 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:26.519 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T08:49:26.546 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T08:49:26.585 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:49:26.872 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T08:49:26.899 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T08:49:26.938 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T08:49:27.148 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-babel. 2024-04-25T08:49:27.175 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:49:27.214 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:49:27.266 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T08:49:27.293 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:27.332 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:27.543 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T08:49:27.570 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T08:49:27.609 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:49:27.686 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T08:49:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:27.752 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:27.946 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T08:49:27.973 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:28.012 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:28.349 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T08:49:28.376 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T08:49:28.416 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T08:49:28.506 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T08:49:28.533 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:28.572 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:28.727 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-routes. 2024-04-25T08:49:28.754 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T08:49:28.793 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:49:29.130 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T08:49:29.158 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:29.197 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:29.642 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T08:49:29.657 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T08:49:29.659 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph. 2024-04-25T08:49:29.686 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../068-ceph_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:29.709 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:49:29.725 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:29.978 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T08:49:30.008 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:30.045 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:30.213 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T08:49:30.240 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T08:49:30.279 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:49:30.414 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T08:49:30.442 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:30.481 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:30.633 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T08:49:30.660 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T08:49:30.802 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:49:30.876 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package cephadm. 2024-04-25T08:49:30.903 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../071-cephadm_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:30.942 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:31.113 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T08:49:31.140 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T08:49:31.180 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:49:31.237 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T08:49:31.264 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T08:49:31.303 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T08:49:31.622 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T08:49:31.639 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:49:31.703 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:49:31.948 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T08:49:31.975 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:32.014 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:32.761 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-babel. 2024-04-25T08:49:32.789 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T08:49:32.827 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:49:32.968 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T08:49:32.995 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T08:49:33.034 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T08:49:33.155 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T08:49:33.183 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T08:49:33.222 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:49:33.338 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T08:49:33.364 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T08:49:33.404 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:49:33.533 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T08:49:33.561 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:33.600 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:33.741 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T08:49:33.768 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T08:49:33.807 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:49:33.920 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T08:49:33.936 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T08:49:33.969 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T08:49:34.094 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T08:49:34.121 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T08:49:34.161 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T08:49:34.272 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-routes. 2024-04-25T08:49:34.300 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T08:49:34.339 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:49:34.440 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T08:49:34.467 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T08:49:34.498 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T08:49:34.692 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T08:49:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:34.750 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:35.043 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T08:49:35.070 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T08:49:35.110 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:49:35.195 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T08:49:35.212 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T08:49:35.253 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:49:35.422 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T08:49:35.448 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T08:49:35.488 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T08:49:35.757 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T08:49:35.785 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T08:49:35.823 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:49:35.833 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T08:49:35.860 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T08:49:35.899 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:49:36.194 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T08:49:36.222 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T08:49:36.346 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:49:36.497 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T08:49:36.514 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:36.555 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:36.649 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T08:49:36.676 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T08:49:36.715 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:49:36.867 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T08:49:36.894 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T08:49:36.933 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:49:37.245 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T08:49:37.261 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:49:37.303 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:49:37.424 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T08:49:37.452 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:37.490 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:37.581 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package jq. 2024-04-25T08:49:37.597 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:49:37.639 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T08:49:37.892 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package socat. 2024-04-25T08:49:37.919 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T08:49:37.958 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:49:38.287 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T08:49:38.303 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T08:49:38.345 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:49:38.427 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T08:49:38.454 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T08:49:38.493 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T08:49:38.750 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-test. 2024-04-25T08:49:38.777 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:38.796 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T08:49:38.824 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T08:49:38.824 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:38.888 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:49:39.200 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T08:49:39.227 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T08:49:39.266 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:49:39.536 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T08:49:39.564 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T08:49:39.594 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T08:49:39.915 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T08:49:39.942 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T08:49:39.981 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T08:49:40.510 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T08:49:40.537 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T08:49:40.576 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:49:40.838 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T08:49:40.854 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T08:49:40.887 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T08:49:41.123 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T08:49:41.150 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:41.191 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T08:49:41.198 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:41.218 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T08:49:41.257 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:49:41.527 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T08:49:41.555 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:41.585 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:41.914 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T08:49:41.941 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T08:49:41.980 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T08:49:42.013 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T08:49:42.029 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:42.071 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:42.242 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T08:49:42.269 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:42.308 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:42.390 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T08:49:42.418 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T08:49:42.482 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:49:42.577 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T08:49:42.593 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:42.635 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:42.760 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T08:49:42.776 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:49:42.810 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:49:42.863 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T08:49:42.879 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:42.913 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:43.088 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package jq. 2024-04-25T08:49:43.104 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T08:49:43.146 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T08:49:43.250 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T08:49:43.277 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T08:49:43.316 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:49:43.398 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package socat. 2024-04-25T08:49:43.415 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T08:49:43.457 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:49:43.646 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T08:49:43.673 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T08:49:43.712 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:49:43.751 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T08:49:43.768 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T08:49:43.809 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:49:44.050 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T08:49:44.077 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T08:49:44.108 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:49:44.172 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-test. 2024-04-25T08:49:44.189 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:44.230 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:44.531 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package pkg-config. 2024-04-25T08:49:44.558 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T08:49:44.589 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:49:44.842 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T08:49:44.858 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T08:49:44.900 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:49:45.295 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T08:49:45.323 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T08:49:45.370 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T08:49:45.673 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T08:49:45.700 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T08:49:45.739 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T08:49:46.043 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T08:49:46.071 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T08:49:46.109 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:49:46.397 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T08:49:46.424 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T08:49:46.463 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T08:49:46.618 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T08:49:46.646 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:46.684 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:46.824 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-py. 2024-04-25T08:49:46.852 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T08:49:46.891 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T08:49:47.072 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T08:49:47.100 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:47.138 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:47.220 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T08:49:47.247 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T08:49:47.286 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:49:47.408 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T08:49:47.425 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T08:49:47.458 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T08:49:47.710 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T08:49:47.727 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:47.732 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T08:49:47.760 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T08:49:47.768 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:47.798 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:49:48.046 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T08:49:48.071 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1628-g97ae6cc9-1jammy_all.deb ... 2024-04-25T08:49:48.113 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:48.118 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-toml. 2024-04-25T08:49:48.146 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T08:49:48.184 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T08:49:48.365 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T08:49:48.393 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:48.432 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:48.488 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T08:49:48.515 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T08:49:48.554 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:49:48.710 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T08:49:48.738 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T08:49:48.768 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:49:48.925 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T08:49:48.952 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T08:49:48.991 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:49:49.081 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T08:49:49.109 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T08:49:49.147 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:49:49.277 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T08:49:49.293 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T08:49:49.335 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:49:49.501 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T08:49:49.529 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T08:49:49.568 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:49:50.006 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package pkg-config. 2024-04-25T08:49:50.035 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T08:49:50.073 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:49:50.107 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package radosgw. 2024-04-25T08:49:50.134 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:50.198 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:50.326 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T08:49:50.354 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T08:49:50.392 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:49:50.837 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T08:49:50.866 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T08:49:50.904 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T08:49:50.960 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T08:49:50.987 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:51.026 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:51.182 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T08:49:51.210 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T08:49:51.239 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T08:49:51.339 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T08:49:51.535 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T08:49:51.563 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T08:49:51.590 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:49:51.592 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:49:51.716 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:49:51.881 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T08:49:51.909 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T08:49:51.947 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T08:49:51.976 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T08:49:52.267 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-py. 2024-04-25T08:49:52.295 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T08:49:52.305 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T08:49:52.333 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T08:49:52.653 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T08:49:52.682 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T08:49:52.720 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:49:52.723 INFO:teuthology.orchestra.run.smithi203.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T08:49:52.785 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:53.182 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T08:49:53.211 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T08:49:53.235 INFO:teuthology.orchestra.run.smithi203.stdout:Adding system user cephadm....done 2024-04-25T08:49:53.248 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:49:53.391 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:49:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-toml. 2024-04-25T08:49:53.605 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T08:49:53.643 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T08:49:53.668 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:49:53.887 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T08:49:53.910 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:49:53.915 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T08:49:53.954 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:49:54.029 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:49:54.272 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T08:49:54.283 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T08:49:54.311 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T08:49:54.349 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:49:54.523 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:49:54.618 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T08:49:54.635 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T08:49:54.649 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T08:49:54.676 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:49:54.967 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:49:55.353 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T08:49:55.439 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package radosgw. 2024-04-25T08:49:55.467 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../122-radosgw_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:55.505 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:55.596 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T08:49:55.831 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T08:49:56.099 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:49:56.358 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T08:49:56.367 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:49:56.372 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1628-g97ae6cc9-1jammy_amd64.deb ... 2024-04-25T08:49:56.416 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:56.627 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:49:56.721 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T08:49:56.886 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:49:57.096 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T08:49:57.179 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:57.231 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T08:49:57.447 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:49:57.493 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T08:49:57.573 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:49:57.699 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T08:49:57.827 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T08:49:57.958 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:49:58.084 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:49:58.213 INFO:teuthology.orchestra.run.smithi037.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T08:49:58.274 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:49:58.343 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:49:58.720 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T08:49:58.750 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T08:49:58.757 INFO:teuthology.orchestra.run.smithi037.stdout:Adding system user cephadm....done 2024-04-25T08:49:58.814 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:49:58.922 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T08:49:58.940 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T08:49:59.198 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T08:49:59.300 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:49:59.426 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T08:49:59.449 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T08:49:59.559 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T08:49:59.702 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:49:59.818 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T08:49:59.953 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:50:00.078 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T08:50:00.203 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T08:50:00.246 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T08:50:00.372 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:50:00.522 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T08:50:00.941 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T08:50:01.083 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:50:01.183 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T08:50:01.342 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:50:01.435 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T08:50:01.468 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:50:01.586 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:50:01.694 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T08:50:01.871 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T08:50:01.962 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T08:50:02.230 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T08:50:02.247 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T08:50:02.498 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:50:02.506 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T08:50:02.749 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-ceph-argparse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:02.883 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:50:03.009 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T08:50:03.109 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T08:50:03.209 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T08:50:03.219 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T08:50:03.478 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T08:50:03.519 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:50:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T08:50:03.846 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:50:03.889 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T08:50:04.088 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:50:04.215 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:04.269 INFO:teuthology.orchestra.run.smithi037.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T08:50:04.319 INFO:teuthology.orchestra.run.smithi037.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T08:50:04.367 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T08:50:04.493 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T08:50:04.526 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T08:50:04.850 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:50:04.852 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T08:50:04.978 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T08:50:05.246 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T08:50:05.664 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T08:50:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T08:50:06.032 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:50:06.083 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T08:50:06.292 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:50:06.426 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T08:50:06.702 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:50:06.793 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T08:50:06.804 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:50:06.913 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:50:07.052 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T08:50:07.178 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T08:50:07.206 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:50:07.304 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T08:50:07.466 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T08:50:07.597 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T08:50:07.717 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:50:07.843 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:50:07.965 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T08:50:08.136 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:50:08.216 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T08:50:08.363 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T08:50:08.592 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T08:50:08.622 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T08:50:08.882 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:50:08.894 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T08:50:09.141 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T08:50:09.203 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T08:50:09.401 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:50:09.527 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T08:50:09.554 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T08:50:09.803 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:50:09.805 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T08:50:09.912 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:50:09.915 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:10.200 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T08:50:10.205 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:50:10.506 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:50:10.509 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T08:50:10.774 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:50:10.900 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:50:11.026 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:50:11.116 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T08:50:11.353 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:50:11.454 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:50:11.715 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T08:50:11.839 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:50:11.966 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:50:12.099 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T08:50:12.100 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T08:50:12.225 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T08:50:12.368 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T08:50:12.493 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:50:12.494 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T08:50:12.602 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:50:12.620 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T08:50:12.911 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:50:12.947 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T08:50:13.190 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T08:50:13.196 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:50:13.416 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:50:13.472 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T08:50:13.517 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T08:50:13.684 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T08:50:13.826 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T08:50:13.944 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T08:50:14.086 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T08:50:14.195 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:50:14.353 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T08:50:14.439 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:50:14.613 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T08:50:14.849 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T08:50:14.856 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T08:50:15.115 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T08:50:15.142 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:50:15.241 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T08:50:15.469 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:50:15.526 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T08:50:15.579 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:15.651 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T08:50:15.705 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:15.831 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:50:15.961 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T08:50:16.218 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:16.336 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:16.446 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T08:50:16.540 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T08:50:16.706 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:16.796 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T08:50:16.835 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:16.898 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T08:50:16.958 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:16.998 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T08:50:17.088 INFO:teuthology.orchestra.run.smithi037.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T08:50:17.177 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:50:17.177 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:50:17.317 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T08:50:17.443 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T08:50:17.643 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:17.769 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:17.852 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T08:50:17.878 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:17.996 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:18.105 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:50:18.128 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T08:50:18.255 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T08:50:18.555 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T08:50:18.657 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T08:50:18.975 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T08:50:19.284 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T08:50:19.309 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:19.460 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:19.570 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:19.576 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T08:50:19.679 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:19.788 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T08:50:19.789 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:19.915 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:20.048 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T08:50:20.332 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T08:50:20.467 INFO:teuthology.orchestra.run.smithi203.stdout:Adding group ceph....done 2024-04-25T08:50:20.592 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T08:50:20.818 INFO:teuthology.orchestra.run.smithi203.stdout:Adding system user ceph....done 2024-04-25T08:50:20.977 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T08:50:20.978 INFO:teuthology.orchestra.run.smithi203.stdout:Setting system user ceph properties....done 2024-04-25T08:50:20.990 INFO:teuthology.orchestra.run.smithi203.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T08:50:21.126 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T08:50:21.262 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T08:50:21.431 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T08:50:21.608 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T08:50:21.715 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up librados2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:21.841 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:21.934 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:21.967 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T08:50:22.143 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:50:22.143 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:50:22.328 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libcephfs2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:22.446 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libradosstriper1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:22.556 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T08:50:22.610 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:22.711 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:22.816 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up librbd1 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:22.917 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr-modules-core (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:23.034 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:23.170 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:50:23.170 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:50:23.252 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:50:23.252 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T08:50:23.605 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:23.727 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libcephfs-dev (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:23.853 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-rados (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:23.957 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T08:50:23.979 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up librgw2 (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:24.088 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up cephfs-top (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:24.198 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T08:50:24.423 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:24.637 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:50:24.637 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:50:25.117 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:25.338 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:50:25.339 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:50:25.493 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-rbd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:25.619 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up rbd-fuse (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:25.770 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-rgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:25.827 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:25.896 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up python3-cephfs (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:26.022 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up cephfs-shell (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:26.091 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:50:26.091 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:50:26.140 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-common (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:26.633 INFO:teuthology.orchestra.run.smithi037.stdout:Adding group ceph....done 2024-04-25T08:50:26.860 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:26.977 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:26.983 INFO:teuthology.orchestra.run.smithi037.stdout:Adding system user ceph....done 2024-04-25T08:50:27.120 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:27.136 INFO:teuthology.orchestra.run.smithi037.stdout:Setting system user ceph properties....done 2024-04-25T08:50:27.153 INFO:teuthology.orchestra.run.smithi037.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T08:50:27.294 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T08:50:27.331 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:50:27.331 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:50:27.613 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T08:50:27.788 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:27.939 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:28.065 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:28.074 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up cephfs-mirror (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:28.216 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:28.288 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:50:28.288 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:50:28.569 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:50:28.662 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:50:28.767 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-test (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:28.797 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:50:28.893 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up radosgw (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:29.361 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:50:29.361 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:50:29.836 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-base (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:29.856 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:50:30.184 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T08:50:30.511 INFO:teuthology.orchestra.run.smithi203.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:50:30.514 DEBUG:teuthology.orchestra.run.smithi203:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T08:50:30.607 INFO:teuthology.orchestra.run.smithi203.stdout:Reading package lists... 2024-04-25T08:50:30.663 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mds (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:30.801 INFO:teuthology.orchestra.run.smithi203.stdout:Building dependency tree... 2024-04-25T08:50:30.802 INFO:teuthology.orchestra.run.smithi203.stdout:Reading state information... 2024-04-25T08:50:30.884 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:50:30.885 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T08:50:30.974 INFO:teuthology.orchestra.run.smithi203.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T08:50:30.974 INFO:teuthology.orchestra.run.smithi203.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T08:50:30.974 INFO:teuthology.orchestra.run.smithi203.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T08:50:30.974 INFO:teuthology.orchestra.run.smithi203.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:50:30.974 INFO:teuthology.orchestra.run.smithi203.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:50:30.974 INFO:teuthology.orchestra.run.smithi203.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:50:30.974 INFO:teuthology.orchestra.run.smithi203.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:50:30.975 INFO:teuthology.orchestra.run.smithi203.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:50:30.976 INFO:teuthology.orchestra.run.smithi203.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:50:30.976 INFO:teuthology.orchestra.run.smithi203.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:50:30.976 INFO:teuthology.orchestra.run.smithi203.stdout:The following additional packages will be installed: 2024-04-25T08:50:30.976 INFO:teuthology.orchestra.run.smithi203.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T08:50:30.976 INFO:teuthology.orchestra.run.smithi203.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T08:50:30.976 INFO:teuthology.orchestra.run.smithi203.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T08:50:30.976 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T08:50:30.976 INFO:teuthology.orchestra.run.smithi203.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:50:30.977 INFO:teuthology.orchestra.run.smithi203.stdout:Suggested packages: 2024-04-25T08:50:30.977 INFO:teuthology.orchestra.run.smithi203.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T08:50:30.977 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T08:50:30.977 INFO:teuthology.orchestra.run.smithi203.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T08:50:31.016 INFO:teuthology.orchestra.run.smithi203.stdout:The following NEW packages will be installed: 2024-04-25T08:50:31.017 INFO:teuthology.orchestra.run.smithi203.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T08:50:31.017 INFO:teuthology.orchestra.run.smithi203.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T08:50:31.017 INFO:teuthology.orchestra.run.smithi203.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T08:50:31.017 INFO:teuthology.orchestra.run.smithi203.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T08:50:31.017 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T08:50:31.017 INFO:teuthology.orchestra.run.smithi203.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T08:50:31.017 INFO:teuthology.orchestra.run.smithi203.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:50:31.018 INFO:teuthology.orchestra.run.smithi203.stdout:The following packages will be upgraded: 2024-04-25T08:50:31.019 INFO:teuthology.orchestra.run.smithi203.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T08:50:31.076 INFO:teuthology.orchestra.run.smithi203.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T08:50:31.077 INFO:teuthology.orchestra.run.smithi203.stdout:Need to get 55.7 MB of archives. 2024-04-25T08:50:31.077 INFO:teuthology.orchestra.run.smithi203.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T08:50:31.077 INFO:teuthology.orchestra.run.smithi203.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T08:50:31.241 INFO:teuthology.orchestra.run.smithi203.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T08:50:31.248 INFO:teuthology.orchestra.run.smithi203.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T08:50:31.288 INFO:teuthology.orchestra.run.smithi203.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T08:50:31.288 INFO:teuthology.orchestra.run.smithi203.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T08:50:31.289 INFO:teuthology.orchestra.run.smithi203.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T08:50:31.300 INFO:teuthology.orchestra.run.smithi203.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T08:50:31.301 INFO:teuthology.orchestra.run.smithi203.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T08:50:31.303 INFO:teuthology.orchestra.run.smithi203.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T08:50:31.304 INFO:teuthology.orchestra.run.smithi203.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T08:50:31.305 INFO:teuthology.orchestra.run.smithi203.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T08:50:31.306 INFO:teuthology.orchestra.run.smithi203.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T08:50:31.309 INFO:teuthology.orchestra.run.smithi203.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T08:50:31.328 INFO:teuthology.orchestra.run.smithi203.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T08:50:31.329 INFO:teuthology.orchestra.run.smithi203.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T08:50:31.330 INFO:teuthology.orchestra.run.smithi203.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T08:50:31.365 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:31.583 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:50:31.584 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T08:50:31.746 INFO:teuthology.orchestra.run.smithi203.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T08:50:31.749 INFO:teuthology.orchestra.run.smithi203.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T08:50:31.749 INFO:teuthology.orchestra.run.smithi203.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T08:50:31.752 INFO:teuthology.orchestra.run.smithi203.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T08:50:31.752 INFO:teuthology.orchestra.run.smithi203.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T08:50:31.758 INFO:teuthology.orchestra.run.smithi203.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T08:50:31.760 INFO:teuthology.orchestra.run.smithi203.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T08:50:31.768 INFO:teuthology.orchestra.run.smithi203.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T08:50:31.769 INFO:teuthology.orchestra.run.smithi203.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T08:50:31.795 INFO:teuthology.orchestra.run.smithi203.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T08:50:31.795 INFO:teuthology.orchestra.run.smithi203.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T08:50:31.822 INFO:teuthology.orchestra.run.smithi203.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T08:50:31.869 INFO:teuthology.orchestra.run.smithi203.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T08:50:31.875 INFO:teuthology.orchestra.run.smithi203.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T08:50:31.876 INFO:teuthology.orchestra.run.smithi203.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T08:50:31.877 INFO:teuthology.orchestra.run.smithi203.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T08:50:31.877 INFO:teuthology.orchestra.run.smithi203.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T08:50:31.898 INFO:teuthology.orchestra.run.smithi203.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T08:50:31.900 INFO:teuthology.orchestra.run.smithi203.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T08:50:32.214 INFO:teuthology.orchestra.run.smithi203.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T08:50:32.214 INFO:teuthology.orchestra.run.smithi203.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T08:50:32.215 INFO:teuthology.orchestra.run.smithi203.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T08:50:32.215 INFO:teuthology.orchestra.run.smithi203.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T08:50:32.215 INFO:teuthology.orchestra.run.smithi203.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T08:50:32.215 INFO:teuthology.orchestra.run.smithi203.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T08:50:32.229 INFO:teuthology.orchestra.run.smithi203.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T08:50:32.229 INFO:teuthology.orchestra.run.smithi203.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T08:50:32.321 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-osd (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:32.613 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:50:32.613 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T08:50:32.625 INFO:teuthology.orchestra.run.smithi203.stdout:Preconfiguring packages ... 2024-04-25T08:50:32.817 INFO:teuthology.orchestra.run.smithi203.stdout:Fetched 55.7 MB in 1s (46.3 MB/s) 2024-04-25T08:50:33.093 INFO:teuthology.orchestra.run.smithi203.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T08:50:33.099 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:50:33.113 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:33.219 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:50:33.223 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:33.366 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mon (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:33.586 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:50:33.586 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T08:50:33.866 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T08:50:34.067 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr-cephadm (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:34.218 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:34.344 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-mgr-dashboard (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:34.495 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ceph-volume (19.0.0-1628-g97ae6cc9-1jammy) ... 2024-04-25T08:50:34.848 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:50:34.859 INFO:teuthology.orchestra.run.smithi203.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T08:50:34.940 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:50:35.093 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:50:35.133 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package flex. 2024-04-25T08:50:35.234 INFO:teuthology.orchestra.run.smithi203.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T08:50:35.240 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T08:50:35.281 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T08:50:35.814 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:50:35.987 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T08:50:36.168 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:50:36.415 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:50:36.838 INFO:teuthology.orchestra.run.smithi037.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:50:36.841 DEBUG:teuthology.orchestra.run.smithi037:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T08:50:36.907 INFO:teuthology.orchestra.run.smithi037.stdout:Reading package lists... 2024-04-25T08:50:36.937 INFO:teuthology.orchestra.run.smithi203.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T08:50:36.943 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:50:37.070 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:50:37.122 INFO:teuthology.orchestra.run.smithi037.stdout:Building dependency tree... 2024-04-25T08:50:37.122 INFO:teuthology.orchestra.run.smithi037.stdout:Reading state information... 2024-04-25T08:50:37.328 INFO:teuthology.orchestra.run.smithi037.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T08:50:37.328 INFO:teuthology.orchestra.run.smithi037.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T08:50:37.328 INFO:teuthology.orchestra.run.smithi037.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T08:50:37.328 INFO:teuthology.orchestra.run.smithi037.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T08:50:37.328 INFO:teuthology.orchestra.run.smithi037.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T08:50:37.328 INFO:teuthology.orchestra.run.smithi037.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T08:50:37.329 INFO:teuthology.orchestra.run.smithi037.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T08:50:37.329 INFO:teuthology.orchestra.run.smithi037.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T08:50:37.329 INFO:teuthology.orchestra.run.smithi037.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T08:50:37.329 INFO:teuthology.orchestra.run.smithi037.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T08:50:37.330 INFO:teuthology.orchestra.run.smithi037.stdout:The following additional packages will be installed: 2024-04-25T08:50:37.330 INFO:teuthology.orchestra.run.smithi037.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T08:50:37.330 INFO:teuthology.orchestra.run.smithi037.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T08:50:37.331 INFO:teuthology.orchestra.run.smithi037.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T08:50:37.331 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T08:50:37.331 INFO:teuthology.orchestra.run.smithi037.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:50:37.332 INFO:teuthology.orchestra.run.smithi037.stdout:Suggested packages: 2024-04-25T08:50:37.332 INFO:teuthology.orchestra.run.smithi037.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T08:50:37.332 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T08:50:37.332 INFO:teuthology.orchestra.run.smithi037.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T08:50:37.376 INFO:teuthology.orchestra.run.smithi037.stdout:The following NEW packages will be installed: 2024-04-25T08:50:37.376 INFO:teuthology.orchestra.run.smithi037.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T08:50:37.376 INFO:teuthology.orchestra.run.smithi037.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T08:50:37.377 INFO:teuthology.orchestra.run.smithi037.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T08:50:37.377 INFO:teuthology.orchestra.run.smithi037.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T08:50:37.377 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T08:50:37.377 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T08:50:37.377 INFO:teuthology.orchestra.run.smithi037.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T08:50:37.378 INFO:teuthology.orchestra.run.smithi037.stdout:The following packages will be upgraded: 2024-04-25T08:50:37.379 INFO:teuthology.orchestra.run.smithi037.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T08:50:37.462 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:50:37.465 INFO:teuthology.orchestra.run.smithi037.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T08:50:37.465 INFO:teuthology.orchestra.run.smithi037.stdout:Need to get 55.7 MB of archives. 2024-04-25T08:50:37.465 INFO:teuthology.orchestra.run.smithi037.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T08:50:37.465 INFO:teuthology.orchestra.run.smithi037.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T08:50:37.576 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:50:37.706 INFO:teuthology.orchestra.run.smithi037.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T08:50:37.715 INFO:teuthology.orchestra.run.smithi037.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T08:50:37.778 INFO:teuthology.orchestra.run.smithi037.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T08:50:37.779 INFO:teuthology.orchestra.run.smithi037.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T08:50:37.780 INFO:teuthology.orchestra.run.smithi037.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T08:50:37.798 INFO:teuthology.orchestra.run.smithi037.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T08:50:37.799 INFO:teuthology.orchestra.run.smithi037.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T08:50:37.802 INFO:teuthology.orchestra.run.smithi037.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T08:50:37.805 INFO:teuthology.orchestra.run.smithi037.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T08:50:37.806 INFO:teuthology.orchestra.run.smithi037.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T08:50:37.808 INFO:teuthology.orchestra.run.smithi037.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T08:50:37.816 INFO:teuthology.orchestra.run.smithi037.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T08:50:37.840 INFO:teuthology.orchestra.run.smithi037.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T08:50:37.841 INFO:teuthology.orchestra.run.smithi037.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T08:50:37.843 INFO:teuthology.orchestra.run.smithi037.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T08:50:37.895 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:50:38.294 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package bison. 2024-04-25T08:50:38.353 INFO:teuthology.orchestra.run.smithi037.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T08:50:38.357 INFO:teuthology.orchestra.run.smithi037.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T08:50:38.358 INFO:teuthology.orchestra.run.smithi037.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T08:50:38.363 INFO:teuthology.orchestra.run.smithi037.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T08:50:38.363 INFO:teuthology.orchestra.run.smithi037.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T08:50:38.372 INFO:teuthology.orchestra.run.smithi037.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T08:50:38.377 INFO:teuthology.orchestra.run.smithi037.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T08:50:38.389 INFO:teuthology.orchestra.run.smithi037.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T08:50:38.389 INFO:teuthology.orchestra.run.smithi037.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T08:50:38.390 INFO:teuthology.orchestra.run.smithi203.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T08:50:38.396 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T08:50:38.424 INFO:teuthology.orchestra.run.smithi037.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T08:50:38.424 INFO:teuthology.orchestra.run.smithi037.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T08:50:38.447 INFO:teuthology.orchestra.run.smithi037.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T08:50:38.455 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:50:38.494 INFO:teuthology.orchestra.run.smithi037.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T08:50:38.500 INFO:teuthology.orchestra.run.smithi037.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T08:50:38.501 INFO:teuthology.orchestra.run.smithi037.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T08:50:38.502 INFO:teuthology.orchestra.run.smithi037.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T08:50:38.502 INFO:teuthology.orchestra.run.smithi037.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T08:50:38.523 INFO:teuthology.orchestra.run.smithi037.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T08:50:38.526 INFO:teuthology.orchestra.run.smithi037.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T08:50:38.902 INFO:teuthology.orchestra.run.smithi037.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T08:50:38.903 INFO:teuthology.orchestra.run.smithi037.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T08:50:38.903 INFO:teuthology.orchestra.run.smithi037.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T08:50:38.903 INFO:teuthology.orchestra.run.smithi037.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T08:50:38.903 INFO:teuthology.orchestra.run.smithi037.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T08:50:38.903 INFO:teuthology.orchestra.run.smithi037.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T08:50:38.913 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T08:50:38.922 INFO:teuthology.orchestra.run.smithi037.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T08:50:38.923 INFO:teuthology.orchestra.run.smithi037.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T08:50:38.941 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T08:50:38.980 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:50:39.333 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package dump. 2024-04-25T08:50:39.346 INFO:teuthology.orchestra.run.smithi037.stdout:Preconfiguring packages ... 2024-04-25T08:50:39.361 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T08:50:39.400 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T08:50:39.503 INFO:teuthology.orchestra.run.smithi037.stdout:Fetched 55.7 MB in 2s (36.3 MB/s) 2024-04-25T08:50:39.670 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package indent. 2024-04-25T08:50:39.686 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:39.720 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:50:39.790 INFO:teuthology.orchestra.run.smithi037.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T08:50:39.796 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:50:39.916 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:50:39.981 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T08:50:40.009 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T08:50:40.047 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:50:40.385 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T08:50:40.413 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T08:50:40.451 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:50:40.554 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T08:50:40.696 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T08:50:40.712 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T08:50:40.746 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:50:41.091 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T08:50:41.119 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T08:50:41.158 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:50:41.410 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T08:50:41.438 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T08:50:41.477 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:50:41.509 INFO:teuthology.orchestra.run.smithi037.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T08:50:41.729 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T08:50:41.757 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T08:50:41.761 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package flex. 2024-04-25T08:50:41.796 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T08:50:41.860 INFO:teuthology.orchestra.run.smithi037.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173655 files and directories currently installed.) 2024-04-25T08:50:41.866 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T08:50:41.902 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T08:50:42.083 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T08:50:42.110 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T08:50:42.149 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:50:42.411 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:50:42.583 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T08:50:43.028 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:50:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T08:50:43.563 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:50:43.709 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:50:43.819 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T08:50:43.847 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:50:43.885 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:50:44.144 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T08:50:44.154 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T08:50:44.170 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:50:44.213 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:50:44.273 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T08:50:44.574 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T08:50:44.602 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:50:44.635 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:50:44.640 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:50:45.002 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T08:50:45.007 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package bison. 2024-04-25T08:50:45.030 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T08:50:45.068 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:50:45.115 INFO:teuthology.orchestra.run.smithi037.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173728 files and directories currently installed.) 2024-04-25T08:50:45.121 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T08:50:45.168 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:50:45.463 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T08:50:45.491 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:50:45.538 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:50:45.610 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T08:50:45.639 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T08:50:45.677 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:50:45.958 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T08:50:45.986 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:46.021 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package dump. 2024-04-25T08:50:46.032 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:50:46.050 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T08:50:46.088 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T08:50:46.417 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package indent. 2024-04-25T08:50:46.444 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T08:50:46.445 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:46.472 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:46.500 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:50:46.510 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:50:46.795 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T08:50:46.811 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T08:50:46.846 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T08:50:46.852 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:50:46.874 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:46.921 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:50:47.181 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T08:50:47.210 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T08:50:47.224 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T08:50:47.252 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:50:47.265 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:50:47.299 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:50:47.568 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T08:50:47.599 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T08:50:47.642 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:50:47.794 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T08:50:47.822 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T08:50:47.861 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:50:48.021 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T08:50:48.049 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T08:50:48.087 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:50:48.289 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package modemmanager. 2024-04-25T08:50:48.316 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:50:48.339 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T08:50:48.356 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T08:50:48.363 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:50:48.398 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:50:48.659 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T08:50:48.676 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T08:50:48.717 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T08:50:49.004 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package network-manager. 2024-04-25T08:50:49.033 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:50:49.037 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T08:50:49.065 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T08:50:49.071 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:50:49.103 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:50:49.728 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package ppp. 2024-04-25T08:50:49.756 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T08:50:50.022 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:50:50.341 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T08:50:50.369 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T08:50:50.408 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T08:50:50.660 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T08:50:50.676 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T08:50:50.718 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T08:50:50.807 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T08:50:50.835 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:50:50.881 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:50:50.980 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T08:50:51.007 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T08:50:51.046 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T08:50:51.192 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T08:50:51.221 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T08:50:51.267 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:50:51.308 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T08:50:51.336 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:51.367 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:50:51.653 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T08:50:51.682 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:50:51.720 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:50:52.048 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T08:50:52.076 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T08:50:52.114 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:50:52.155 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T08:50:52.183 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T08:50:52.240 INFO:teuthology.orchestra.run.smithi203.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T08:50:52.269 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking postgresql-common (238) ... 2024-04-25T08:50:52.484 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T08:50:52.512 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:50:52.567 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:50:52.637 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T08:50:52.666 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:52.699 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:50:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T08:50:52.998 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:53.037 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:50:53.389 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T08:50:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:53.456 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:50:53.776 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T08:50:53.804 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:53.850 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:50:54.040 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package postgresql. 2024-04-25T08:50:54.069 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T08:50:54.115 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking postgresql (14+238) ... 2024-04-25T08:50:54.153 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T08:50:54.182 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T08:50:54.228 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:50:54.409 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T08:50:54.438 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T08:50:54.484 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T08:50:54.736 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T08:50:54.765 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T08:50:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T08:50:54.802 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T08:50:54.840 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:50:55.130 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T08:50:55.159 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T08:50:55.205 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:50:55.218 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package modemmanager. 2024-04-25T08:50:55.247 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T08:50:55.285 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:50:55.541 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package tcl. 2024-04-25T08:50:55.570 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T08:50:55.621 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T08:50:55.933 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package network-manager. 2024-04-25T08:50:55.961 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T08:50:56.008 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:50:56.035 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T08:50:56.064 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T08:50:56.102 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:50:56.490 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T08:50:56.508 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T08:50:56.548 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T08:50:56.623 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package ppp. 2024-04-25T08:50:56.652 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T08:50:56.910 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:50:56.926 INFO:teuthology.orchestra.run.smithi203.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T08:50:56.954 INFO:teuthology.orchestra.run.smithi203.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T08:50:57.001 INFO:teuthology.orchestra.run.smithi203.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:50:57.245 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T08:50:57.273 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T08:50:57.289 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T08:50:57.311 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T08:50:57.507 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T08:50:57.614 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T08:50:57.633 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:50:57.642 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T08:50:57.689 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T08:50:57.759 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:50:57.861 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:50:57.961 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:50:58.025 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T08:50:58.054 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T08:50:58.071 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:50:58.092 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T08:50:58.354 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T08:50:58.382 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:58.412 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:50:58.548 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T08:50:58.643 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:50:58.970 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:50:58.971 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:50:58.975 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T08:50:59.004 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T08:50:59.052 INFO:teuthology.orchestra.run.smithi037.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T08:50:59.081 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking postgresql-common (238) ... 2024-04-25T08:50:59.640 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T08:50:59.669 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T08:50:59.710 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:50:59.805 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:50:59.930 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:51:00.056 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:51:00.166 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T08:51:00.283 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:51:00.409 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:51:00.535 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:51:00.661 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:51:00.976 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package postgresql. 2024-04-25T08:51:00.993 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T08:51:01.034 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking postgresql (14+238) ... 2024-04-25T08:51:01.228 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T08:51:01.245 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T08:51:01.274 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:51:01.278 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T08:51:01.401 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T08:51:01.486 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T08:51:01.496 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T08:51:01.514 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T08:51:01.544 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:51:01.580 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T08:51:01.645 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:51:01.763 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T08:51:01.888 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T08:51:01.916 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T08:51:01.944 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T08:51:01.982 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:51:02.014 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:51:02.106 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T08:51:02.175 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:51:02.300 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:51:02.310 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package tcl. 2024-04-25T08:51:02.338 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T08:51:02.381 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T08:51:02.468 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:51:02.569 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:51:02.663 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T08:51:02.670 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T08:51:02.691 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T08:51:02.721 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:51:02.830 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T08:51:02.948 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:51:03.075 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T08:51:03.092 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T08:51:03.133 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T08:51:03.409 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:03.409 INFO:teuthology.orchestra.run.smithi203.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T08:51:03.409 INFO:teuthology.orchestra.run.smithi203.stdout:which means they are currently configured by ifupdown: 2024-04-25T08:51:03.409 INFO:teuthology.orchestra.run.smithi203.stdout:- enp3s0f1 2024-04-25T08:51:03.409 INFO:teuthology.orchestra.run.smithi203.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T08:51:03.409 INFO:teuthology.orchestra.run.smithi203.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T08:51:03.409 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:03.524 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T08:51:03.826 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T08:51:03.936 INFO:teuthology.orchestra.run.smithi037.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T08:51:03.965 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T08:51:04.011 INFO:teuthology.orchestra.run.smithi037.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:51:04.170 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T08:51:04.299 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T08:51:04.508 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T08:51:04.634 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T08:51:04.659 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:51:04.760 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:51:04.886 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T08:51:05.012 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T08:51:05.142 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:51:05.269 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:51:05.411 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T08:51:05.546 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up postgresql-common (238) ... 2024-04-25T08:51:05.689 INFO:teuthology.orchestra.run.smithi037.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T08:51:05.777 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T08:51:06.163 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:51:06.163 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T08:51:06.647 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T08:51:06.748 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T08:51:06.843 INFO:teuthology.orchestra.run.smithi203.stdout:Adding user postgres to group ssl-cert 2024-04-25T08:51:06.848 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T08:51:06.958 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T08:51:07.084 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T08:51:07.109 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:07.109 INFO:teuthology.orchestra.run.smithi203.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T08:51:07.210 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T08:51:07.301 INFO:teuthology.orchestra.run.smithi203.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T08:51:07.301 INFO:teuthology.orchestra.run.smithi203.stdout:Removing obsolete dictionary files: 2024-04-25T08:51:07.345 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T08:51:07.470 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T08:51:07.848 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T08:51:08.050 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T08:51:08.176 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T08:51:08.271 INFO:teuthology.orchestra.run.smithi037.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T08:51:08.277 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:51:08.336 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:51:08.462 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T08:51:08.588 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T08:51:08.605 INFO:teuthology.orchestra.run.smithi203.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T08:51:08.623 INFO:teuthology.orchestra.run.smithi203.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T08:51:08.667 INFO:teuthology.orchestra.run.smithi203.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T08:51:08.667 INFO:teuthology.orchestra.run.smithi203.stdout:This user must also own the server process. 2024-04-25T08:51:08.667 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:08.668 INFO:teuthology.orchestra.run.smithi203.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T08:51:08.668 INFO:teuthology.orchestra.run.smithi203.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T08:51:08.668 INFO:teuthology.orchestra.run.smithi203.stdout:The default text search configuration will be set to "english". 2024-04-25T08:51:08.668 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:08.668 INFO:teuthology.orchestra.run.smithi203.stdout:Data page checksums are disabled. 2024-04-25T08:51:08.668 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:08.668 INFO:teuthology.orchestra.run.smithi203.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T08:51:08.668 INFO:teuthology.orchestra.run.smithi203.stdout:creating subdirectories ... ok 2024-04-25T08:51:08.668 INFO:teuthology.orchestra.run.smithi203.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T08:51:08.684 INFO:teuthology.orchestra.run.smithi203.stdout:selecting default max_connections ... 100 2024-04-25T08:51:08.704 INFO:teuthology.orchestra.run.smithi203.stdout:selecting default shared_buffers ... 128MB 2024-04-25T08:51:08.714 INFO:teuthology.orchestra.run.smithi203.stdout:selecting default time zone ... Etc/UTC 2024-04-25T08:51:08.714 INFO:teuthology.orchestra.run.smithi203.stdout:creating configuration files ... ok 2024-04-25T08:51:08.830 INFO:teuthology.orchestra.run.smithi203.stdout:running bootstrap script ... ok 2024-04-25T08:51:08.881 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T08:51:09.224 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T08:51:09.298 INFO:teuthology.orchestra.run.smithi037.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T08:51:09.358 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T08:51:09.465 INFO:teuthology.orchestra.run.smithi203.stdout:performing post-bootstrap initialization ... ok 2024-04-25T08:51:09.485 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T08:51:09.661 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T08:51:09.787 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:51:09.913 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T08:51:10.080 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T08:51:10.190 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T08:51:10.331 INFO:teuthology.orchestra.run.smithi203.stdout:syncing data to disk ... ok 2024-04-25T08:51:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:10.630 INFO:teuthology.orchestra.run.smithi037.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T08:51:10.630 INFO:teuthology.orchestra.run.smithi037.stdout:which means they are currently configured by ifupdown: 2024-04-25T08:51:10.630 INFO:teuthology.orchestra.run.smithi037.stdout:- ens1f0 2024-04-25T08:51:10.630 INFO:teuthology.orchestra.run.smithi037.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T08:51:10.630 INFO:teuthology.orchestra.run.smithi037.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T08:51:10.630 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:10.728 INFO:teuthology.orchestra.run.smithi203.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T08:51:10.753 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T08:51:11.080 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T08:51:11.400 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T08:51:11.884 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T08:51:12.044 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T08:51:12.170 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T08:51:12.321 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up postgresql-common (238) ... 2024-04-25T08:51:13.176 INFO:teuthology.orchestra.run.smithi037.stdout:Adding user postgres to group ssl-cert 2024-04-25T08:51:13.417 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:13.418 INFO:teuthology.orchestra.run.smithi037.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T08:51:13.452 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:51:13.570 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T08:51:13.594 INFO:teuthology.orchestra.run.smithi037.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T08:51:13.594 INFO:teuthology.orchestra.run.smithi037.stdout:Removing obsolete dictionary files: 2024-04-25T08:51:13.687 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up postgresql (14+238) ... 2024-04-25T08:51:13.789 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:51:13.890 INFO:teuthology.orchestra.run.smithi203.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:51:14.135 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T08:51:14.143 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:51:14.143 INFO:teuthology.orchestra.run.smithi203.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:51:14.521 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T08:51:14.685 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:51:14.793 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T08:51:14.846 INFO:teuthology.orchestra.run.smithi037.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T08:51:14.963 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T08:51:15.050 INFO:teuthology.orchestra.run.smithi037.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T08:51:15.051 INFO:teuthology.orchestra.run.smithi037.stdout:This user must also own the server process. 2024-04-25T08:51:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:15.051 INFO:teuthology.orchestra.run.smithi037.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T08:51:15.051 INFO:teuthology.orchestra.run.smithi037.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T08:51:15.051 INFO:teuthology.orchestra.run.smithi037.stdout:The default text search configuration will be set to "english". 2024-04-25T08:51:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:15.051 INFO:teuthology.orchestra.run.smithi037.stdout:Data page checksums are disabled. 2024-04-25T08:51:15.051 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:15.051 INFO:teuthology.orchestra.run.smithi037.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T08:51:15.051 INFO:teuthology.orchestra.run.smithi037.stdout:creating subdirectories ... ok 2024-04-25T08:51:15.051 INFO:teuthology.orchestra.run.smithi037.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T08:51:15.068 INFO:teuthology.orchestra.run.smithi037.stdout:selecting default max_connections ... 100 2024-04-25T08:51:15.092 INFO:teuthology.orchestra.run.smithi037.stdout:selecting default shared_buffers ... 128MB 2024-04-25T08:51:15.102 INFO:teuthology.orchestra.run.smithi037.stdout:selecting default time zone ... Etc/UTC 2024-04-25T08:51:15.103 INFO:teuthology.orchestra.run.smithi037.stdout:creating configuration files ... ok 2024-04-25T08:51:15.104 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:51:15.223 INFO:teuthology.orchestra.run.smithi037.stdout:running bootstrap script ... ok 2024-04-25T08:51:15.249 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:51:15.876 INFO:teuthology.orchestra.run.smithi037.stdout:performing post-bootstrap initialization ... ok 2024-04-25T08:51:16.482 INFO:teuthology.orchestra.run.smithi037.stdout:syncing data to disk ... ok 2024-04-25T08:51:16.745 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T08:51:16.845 INFO:teuthology.orchestra.run.smithi203.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:51:16.869 INFO:teuthology.orchestra.run.smithi037.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T08:51:17.357 INFO:teuthology.orchestra.run.smithi203.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:51:17.363 DEBUG:teuthology.parallel:result is None 2024-04-25T08:51:19.645 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:51:19.762 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T08:51:19.888 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up postgresql (14+238) ... 2024-04-25T08:51:20.014 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T08:51:20.140 INFO:teuthology.orchestra.run.smithi037.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T08:51:20.383 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:51:20.383 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T08:51:20.901 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T08:51:20.976 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T08:51:21.304 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T08:51:21.455 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T08:51:22.866 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T08:51:22.966 INFO:teuthology.orchestra.run.smithi037.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T08:51:23.473 INFO:teuthology.orchestra.run.smithi037.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T08:51:23.478 DEBUG:teuthology.parallel:result is None 2024-04-25T08:51:23.479 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:51:23.620 DEBUG:teuthology.orchestra.run.smithi037:> dpkg-query -W -f '${Version}' ceph 2024-04-25T08:51:23.639 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:51:23.640 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:51:23.640 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T08:51:23.642 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:51:23.807 DEBUG:teuthology.orchestra.run.smithi203:> dpkg-query -W -f '${Version}' ceph 2024-04-25T08:51:23.826 INFO:teuthology.orchestra.run.smithi203.stdout:19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:51:23.826 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628-g97ae6cc9-1jammy 2024-04-25T08:51:23.827 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628-g97ae6cc9-1jammy is installed. 2024-04-25T08:51:23.828 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T08:51:23.828 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:23.829 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:51:23.843 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:23.843 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:51:23.884 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T08:51:23.884 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:23.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:51:23.898 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:51:23.952 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:23.953 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:51:23.966 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:51:24.022 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T08:51:24.022 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:24.023 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:51:24.036 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:51:24.093 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:24.093 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:51:24.106 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:51:24.161 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T08:51:24.161 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:24.161 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:51:24.175 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:51:24.229 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:24.229 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:51:24.242 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:51:24.298 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T08:51:24.387 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T08:51:24.387 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /var/log/ceph 2024-04-25T08:51:24.389 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 777 /var/log/ceph 2024-04-25T08:51:24.402 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T08:51:24.402 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:51:24.446 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:51:24.459 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T08:51:24.459 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:51:24.502 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:51:24.522 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T08:51:24.522 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T08:51:24.522 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T08:51:24.522 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:24.522 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:51:24.561 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:51:24.637 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:24.637 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:51:24.643 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:51:24.724 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T08:51:24.725 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T08:51:24.725 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7672997', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': '5787d42d10024ba410e822a8aba474331cb9af02'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5869, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '5787d42d10024ba410e822a8aba474331cb9af02', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi037.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMnwStv2ELVuA8XyN2z919cNNU9QuT8hb+z+dsJoCU8UDOotF/inBaQsVTrG4h3m0s4Htqa4bpDDycVjXfUt4lE=', 'smithi203.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL2qEriPyJ6CItPjKUMyzVb/XnJ8y8GlJhIh02+rWs09iGUimc8iLUpOTQ25/3O5URX3FiPYroh9HvocYrYK7Dk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_08:15:18', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T08:51:24.725 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:51:24.728 DEBUG:teuthology.orchestra.run.smithi203:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:51:24.767 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:51:24.774 DEBUG:teuthology.orchestra.run.smithi203:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:51:24.822 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:24.822 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:51:24.833 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:51:24.833 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-04-25T08:51:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:51:24.881 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:51:24.881 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 887 Links: 1 2024-04-25T08:51:24.882 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:51:24.882 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 08:45:14.559407775 +0000 2024-04-25T08:51:24.882 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 08:45:14.275410311 +0000 2024-04-25T08:51:24.882 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 08:45:14.275410311 +0000 2024-04-25T08:51:24.882 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: - 2024-04-25T08:51:24.882 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:51:24.935 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T08:51:24.936 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T08:51:24.936 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000371025 s, 1.4 MB/s 2024-04-25T08:51:24.937 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:51:24.987 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-04-25T08:51:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:51:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:51:25.033 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-04-25T08:51:25.033 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:51:25.033 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 08:45:15.059403309 +0000 2024-04-25T08:51:25.033 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 08:45:14.795405667 +0000 2024-04-25T08:51:25.033 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 08:45:14.795405667 +0000 2024-04-25T08:51:25.033 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: - 2024-04-25T08:51:25.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:51:25.087 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T08:51:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T08:51:25.088 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000420156 s, 1.2 MB/s 2024-04-25T08:51:25.089 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:51:25.138 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-04-25T08:51:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:51:25.185 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:51:25.185 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-04-25T08:51:25.185 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:51:25.185 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 08:45:15.539399021 +0000 2024-04-25T08:51:25.185 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 08:45:15.291401236 +0000 2024-04-25T08:51:25.186 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 08:45:15.291401236 +0000 2024-04-25T08:51:25.186 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: - 2024-04-25T08:51:25.186 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:51:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T08:51:25.239 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T08:51:25.240 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000354925 s, 1.4 MB/s 2024-04-25T08:51:25.241 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:51:25.291 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-04-25T08:51:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:51:25.337 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:51:25.337 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-04-25T08:51:25.337 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:51:25.337 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-25 08:45:16.023394698 +0000 2024-04-25T08:51:25.337 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-25 08:45:15.779396877 +0000 2024-04-25T08:51:25.337 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-25 08:45:15.779396877 +0000 2024-04-25T08:51:25.338 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: - 2024-04-25T08:51:25.338 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:51:25.391 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-25T08:51:25.391 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-25T08:51:25.391 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000349997 s, 1.5 MB/s 2024-04-25T08:51:25.393 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:51:25.442 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:51:25.442 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:25.442 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:51:25.448 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:51:25.448 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_1 2024-04-25T08:51:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:51:25.495 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:51:25.495 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 885 Links: 1 2024-04-25T08:51:25.495 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:51:25.496 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-25 08:45:12.315823251 +0000 2024-04-25T08:51:25.496 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-25 08:45:12.063825503 +0000 2024-04-25T08:51:25.496 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-25 08:45:12.063825503 +0000 2024-04-25T08:51:25.496 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2024-04-25T08:51:25.496 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:51:25.549 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-25T08:51:25.550 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-25T08:51:25.550 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000449158 s, 1.1 MB/s 2024-04-25T08:51:25.551 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:51:25.600 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_2 2024-04-25T08:51:25.647 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:51:25.647 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:51:25.647 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 907 Links: 1 2024-04-25T08:51:25.647 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:51:25.647 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-25 08:45:12.551821143 +0000 2024-04-25T08:51:25.647 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-25 08:45:12.547821179 +0000 2024-04-25T08:51:25.647 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-25 08:45:12.547821179 +0000 2024-04-25T08:51:25.647 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2024-04-25T08:51:25.647 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:51:25.701 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-25T08:51:25.701 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-25T08:51:25.701 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.00043631 s, 1.2 MB/s 2024-04-25T08:51:25.702 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:51:25.752 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_3 2024-04-25T08:51:25.799 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:51:25.799 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:51:25.799 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 921 Links: 1 2024-04-25T08:51:25.799 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:51:25.799 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-25 08:45:13.307814390 +0000 2024-04-25T08:51:25.799 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-25 08:45:13.047816712 +0000 2024-04-25T08:51:25.799 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-25 08:45:13.047816712 +0000 2024-04-25T08:51:25.799 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2024-04-25T08:51:25.799 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:51:25.853 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-25T08:51:25.853 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-25T08:51:25.853 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000365871 s, 1.4 MB/s 2024-04-25T08:51:25.854 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:51:25.907 DEBUG:teuthology.orchestra.run.smithi203:> stat /dev/vg_nvme/lv_4 2024-04-25T08:51:25.958 INFO:teuthology.orchestra.run.smithi203.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:51:25.959 INFO:teuthology.orchestra.run.smithi203.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:51:25.959 INFO:teuthology.orchestra.run.smithi203.stdout:Device: 5h/5d Inode: 938 Links: 1 2024-04-25T08:51:25.959 INFO:teuthology.orchestra.run.smithi203.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:51:25.959 INFO:teuthology.orchestra.run.smithi203.stdout:Access: 2024-04-25 08:45:13.835809675 +0000 2024-04-25T08:51:25.959 INFO:teuthology.orchestra.run.smithi203.stdout:Modify: 2024-04-25 08:45:13.551812212 +0000 2024-04-25T08:51:25.959 INFO:teuthology.orchestra.run.smithi203.stdout:Change: 2024-04-25 08:45:13.551812212 +0000 2024-04-25T08:51:25.959 INFO:teuthology.orchestra.run.smithi203.stdout: Birth: - 2024-04-25T08:51:25.959 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:51:26.014 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records in 2024-04-25T08:51:26.014 INFO:teuthology.orchestra.run.smithi203.stderr:1+0 records out 2024-04-25T08:51:26.014 INFO:teuthology.orchestra.run.smithi203.stderr:512 bytes copied, 0.000405029 s, 1.3 MB/s 2024-04-25T08:51:26.015 DEBUG:teuthology.orchestra.run.smithi203:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:51:26.064 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:51:26.064 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:51:26.064 INFO:tasks.ceph:Generating config... 2024-04-25T08:51:26.066 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T08:51:26.066 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T08:51:26.066 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T08:51:26.066 INFO:tasks.ceph:[client] pid file = 2024-04-25T08:51:26.066 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T08:51:26.066 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T08:51:26.066 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T08:51:26.066 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T08:51:26.066 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T08:51:26.066 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T08:51:26.066 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T08:51:26.066 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T08:51:26.067 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T08:51:26.067 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T08:51:26.067 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T08:51:26.067 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T08:51:26.067 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T08:51:26.067 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T08:51:26.067 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T08:51:26.067 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T08:51:26.067 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T08:51:26.067 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T08:51:26.067 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T08:51:26.067 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T08:51:26.067 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T08:51:26.067 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T08:51:26.067 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T08:51:26.067 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T08:51:26.068 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T08:51:26.068 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T08:51:26.068 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T08:51:26.068 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T08:51:26.068 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T08:51:26.068 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T08:51:26.068 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T08:51:26.069 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T08:51:26.069 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T08:51:26.105 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T08:51:26.112 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T08:51:26.192 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:51:26.248 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.37'), ('mon.b', '172.21.15.203'), ('mon.c', '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]')] 2024-04-25T08:51:26.249 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T08:51:26.249 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:26.249 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:51:26.297 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.37 --add b 172.21.15.203 --addv c '[v2:172.21.15.203:3301,v1:172.21.15.203:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:26.383 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:26.383 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: generated fsid 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T08:51:26.383 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-04-25T08:51:26.383 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 0 2024-04-25T08:51:26.384 INFO:teuthology.orchestra.run.smithi037.stdout:fsid 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T08:51:26.384 INFO:teuthology.orchestra.run.smithi037.stdout:last_changed 2024-04-25T08:51:26.394288+0000 2024-04-25T08:51:26.384 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-25T08:51:26.394288+0000 2024-04-25T08:51:26.384 INFO:teuthology.orchestra.run.smithi037.stdout:min_mon_release 17 (quincy) 2024-04-25T08:51:26.384 INFO:teuthology.orchestra.run.smithi037.stdout:election_strategy: 1 2024-04-25T08:51:26.384 INFO:teuthology.orchestra.run.smithi037.stdout:0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-25T08:51:26.384 INFO:teuthology.orchestra.run.smithi037.stdout:1: [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] mon.b 2024-04-25T08:51:26.384 INFO:teuthology.orchestra.run.smithi037.stdout:2: [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] mon.c 2024-04-25T08:51:26.384 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T08:51:26.385 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:51:26.433 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3ecc7551-d5a4-47e7-a9cf-6943742cae5b... 2024-04-25T08:51:26.434 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:51:26.479 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:51:26.508 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-25T08:51:26.508 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-04-25T08:51:26.509 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:51:26.510 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-04-25T08:51:26.511 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-04-25T08:51:26.512 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:51:26.513 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 20 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 20 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 10 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:51:26.514 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T08:51:26.515 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:51:26.516 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: mon op complaint time = 120 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: client mount timeout = 600 2024-04-25T08:51:26.517 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout:[mds] 2024-04-25T08:51:26.518 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds = 20 2024-04-25T08:51:26.519 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds balancer = 20 2024-04-25T08:51:26.519 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T08:51:26.519 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug frag = True 2024-04-25T08:51:26.519 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug scatterstat = True 2024-04-25T08:51:26.519 INFO:teuthology.orchestra.run.smithi037.stdout: mds op complaint time = 180 2024-04-25T08:51:26.519 INFO:teuthology.orchestra.run.smithi037.stdout: mds verify scatter = True 2024-04-25T08:51:26.519 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T08:51:26.519 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T08:51:26.519 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.520 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:51:26.521 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2024-04-25T08:51:26.522 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2024-04-25T08:51:26.523 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2024-04-25T08:51:26.524 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 20 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 20 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 10 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:51:26.525 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T08:51:26.526 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:51:26.527 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2024-04-25T08:51:26.528 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T08:51:26.529 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2024-04-25T08:51:26.530 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = True 2024-04-25T08:51:26.530 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2024-04-25T08:51:26.530 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = True 2024-04-25T08:51:26.530 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T08:51:26.530 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T08:51:26.530 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T08:51:26.531 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T08:51:26.531 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T08:51:26.623 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T08:51:26.623 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:26.623 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T08:51:26.628 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:26.628 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T08:51:26.677 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:51:26.677 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:26.677 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:51:26.677 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:51:26.741 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:26.742 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:26.789 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:51:26.789 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:26.790 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:51:26.790 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:51:26.815 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:26.815 DEBUG:teuthology.orchestra.run.smithi203:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:26.863 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T08:51:26.863 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T08:51:26.863 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:51:26.908 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:51:26.911 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:51:26.949 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:51:26.959 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T08:51:26.959 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:51:27.003 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:51:27.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T08:51:27.062 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:51:27.147 INFO:teuthology.orchestra.run.smithi037.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:51:27.150 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T08:51:27.205 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:51:27.249 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:51:27.252 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T08:51:27.307 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:51:27.391 INFO:teuthology.orchestra.run.smithi203.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:51:27.394 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T08:51:27.453 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T08:51:27.453 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T08:51:27.492 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T08:51:27.504 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T08:51:27.582 INFO:teuthology.orchestra.run.smithi037.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T08:51:27.592 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T08:51:27.627 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T08:51:27.638 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T08:51:27.713 INFO:teuthology.orchestra.run.smithi203.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T08:51:27.723 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T08:51:27.723 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi037.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi203.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:51:27.723 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T08:51:27.739 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:51:27.739 INFO:tasks.ceph:role: osd.0 2024-04-25T08:51:27.739 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:51:27.739 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:51:27.801 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:51:27.801 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:51:27.801 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:51:27.801 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:51:27.802 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:51:27.802 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:51:27.802 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:51:27.802 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:51:27.802 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:51:27.802 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:51:28.256 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-25T08:51:28.258 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-25T08:51:28.259 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T08:51:28.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T08:51:28.357 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:51:28.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:51:28.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T08:51:28.411 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:51:28.411 INFO:tasks.ceph:role: osd.1 2024-04-25T08:51:28.411 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:51:28.411 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:51:28.472 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:51:28.473 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:51:28.473 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:51:28.473 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:51:28.473 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:51:28.473 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:51:28.473 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:51:28.473 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:51:28.473 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:51:28.473 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:51:28.938 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-25T08:51:28.940 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-25T08:51:28.940 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T08:51:29.112 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T08:51:29.124 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:51:29.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:51:29.125 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T08:51:29.181 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:51:29.181 INFO:tasks.ceph:role: osd.2 2024-04-25T08:51:29.181 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:51:29.181 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:51:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:51:29.244 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:51:29.244 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:51:29.244 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:51:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:51:29.244 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:51:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:51:29.244 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:51:29.245 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:51:29.245 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:51:29.721 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-25T08:51:29.723 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-25T08:51:29.724 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T08:51:29.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T08:51:29.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:51:29.859 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:51:29.859 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T08:51:29.912 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:51:29.913 INFO:tasks.ceph:role: osd.3 2024-04-25T08:51:29.913 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:51:29.913 DEBUG:teuthology.orchestra.run.smithi037:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:51:29.977 INFO:teuthology.orchestra.run.smithi037.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:51:29.977 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:51:29.977 INFO:teuthology.orchestra.run.smithi037.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:51:29.977 INFO:teuthology.orchestra.run.smithi037.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:51:29.977 INFO:teuthology.orchestra.run.smithi037.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:51:29.977 INFO:teuthology.orchestra.run.smithi037.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:51:29.977 INFO:teuthology.orchestra.run.smithi037.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:51:29.977 INFO:teuthology.orchestra.run.smithi037.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:51:29.978 INFO:teuthology.orchestra.run.smithi037.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:51:29.978 INFO:teuthology.orchestra.run.smithi037.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:51:30.403 INFO:teuthology.orchestra.run.smithi037.stdout:Discarding blocks...Done. 2024-04-25T08:51:30.405 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi037.front.sepia.ceph.com -o noatime 2024-04-25T08:51:30.406 DEBUG:teuthology.orchestra.run.smithi037:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T08:51:30.500 DEBUG:teuthology.orchestra.run.smithi037:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T08:51:30.510 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:51:30.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:51:30.511 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:30.637+0000 7f1b694867c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T08:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:30.641+0000 7f1b694867c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T08:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:30.641+0000 7f1b694867c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:30.641+0000 7f1b694867c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:30.641+0000 7f1b694867c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:51:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:30.641+0000 7f1b694867c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T08:51:33.628 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T08:51:33.643 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:33.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:33.729+0000 7f49643e17c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T08:51:33.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:33.733+0000 7f49643e17c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T08:51:33.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:33.733+0000 7f49643e17c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:51:33.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:33.733+0000 7f49643e17c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:51:33.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:33.733+0000 7f49643e17c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:51:33.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:33.733+0000 7f49643e17c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T08:51:36.703 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T08:51:36.761 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:36.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:36.857+0000 7efe2fb1d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T08:51:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:36.857+0000 7efe2fb1d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T08:51:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:36.857+0000 7efe2fb1d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:51:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:36.857+0000 7efe2fb1d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:51:36.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:36.857+0000 7efe2fb1d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:51:36.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:36.857+0000 7efe2fb1d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T08:51:39.815 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T08:51:39.870 DEBUG:teuthology.orchestra.run.smithi037:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:39.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:39.961+0000 7fad4506e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T08:51:39.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:39.961+0000 7fad4506e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T08:51:39.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:39.961+0000 7fad4506e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:51:39.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:39.961+0000 7fad4506e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:51:39.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:39.961+0000 7fad4506e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:51:39.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:51:39.961+0000 7fad4506e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T08:51:42.940 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T08:51:42.997 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T08:51:43.012 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:51:43.012 INFO:tasks.ceph:role: osd.4 2024-04-25T08:51:43.012 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:51:43.012 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:51:43.074 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:51:43.074 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:51:43.074 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:51:43.075 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:51:43.075 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:51:43.075 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:51:43.075 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:51:43.075 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:51:43.075 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:51:43.075 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:51:43.534 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-25T08:51:43.536 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-25T08:51:43.536 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T08:51:43.619 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T08:51:43.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:51:43.671 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:51:43.671 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T08:51:43.724 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:51:43.724 INFO:tasks.ceph:role: osd.5 2024-04-25T08:51:43.724 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:51:43.724 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:51:43.787 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:51:43.788 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:51:43.788 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:51:43.788 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:51:43.788 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:51:43.788 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:51:43.788 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:51:43.788 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:51:43.788 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:51:43.788 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:51:44.264 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-25T08:51:44.266 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-25T08:51:44.266 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T08:51:44.422 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T08:51:44.434 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:51:44.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:51:44.435 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T08:51:44.489 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:51:44.490 INFO:tasks.ceph:role: osd.6 2024-04-25T08:51:44.490 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:51:44.490 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:51:44.554 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:51:44.555 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:51:44.555 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:51:44.555 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:51:44.555 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:51:44.555 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:51:44.555 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:51:44.555 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:51:44.555 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:51:44.555 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:51:45.022 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-25T08:51:45.024 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-25T08:51:45.024 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T08:51:45.174 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T08:51:45.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:51:45.187 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:51:45.187 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T08:51:45.242 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:51:45.242 INFO:tasks.ceph:role: osd.7 2024-04-25T08:51:45.242 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:51:45.243 DEBUG:teuthology.orchestra.run.smithi203:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:51:45.306 INFO:teuthology.orchestra.run.smithi203.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:51:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:51:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:51:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T08:51:45.307 INFO:teuthology.orchestra.run.smithi203.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:51:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:51:45.307 INFO:teuthology.orchestra.run.smithi203.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:51:45.307 INFO:teuthology.orchestra.run.smithi203.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T08:51:45.307 INFO:teuthology.orchestra.run.smithi203.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:51:45.307 INFO:teuthology.orchestra.run.smithi203.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:51:45.732 INFO:teuthology.orchestra.run.smithi203.stdout:Discarding blocks...Done. 2024-04-25T08:51:45.734 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi203.front.sepia.ceph.com -o noatime 2024-04-25T08:51:45.735 DEBUG:teuthology.orchestra.run.smithi203:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T08:51:45.834 DEBUG:teuthology.orchestra.run.smithi203:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T08:51:45.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:51:45.844 INFO:teuthology.orchestra.run.smithi203.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T08:51:45.845 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:45.956 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:45.964+0000 7f8acff7d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T08:51:45.957 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:45.964+0000 7f8acff7d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T08:51:45.957 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:45.964+0000 7f8acff7d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:51:45.957 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:45.964+0000 7f8acff7d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:51:45.957 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:45.964+0000 7f8acff7d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:51:45.958 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:45.968+0000 7f8acff7d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T08:51:48.926 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T08:51:48.940 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:49.020 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:49.028+0000 7f3e738e47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T08:51:49.022 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:49.028+0000 7f3e738e47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T08:51:49.022 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:49.028+0000 7f3e738e47c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:51:49.022 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:49.028+0000 7f3e738e47c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:51:49.022 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:49.028+0000 7f3e738e47c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:51:49.022 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:49.028+0000 7f3e738e47c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T08:51:51.991 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T08:51:52.046 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:52.130 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:52.140+0000 7fa79bfd07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T08:51:52.130 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:52.140+0000 7fa79bfd07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T08:51:52.130 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:52.140+0000 7fa79bfd07c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:51:52.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:52.140+0000 7fa79bfd07c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:51:52.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:52.140+0000 7fa79bfd07c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:51:52.131 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:52.140+0000 7fa79bfd07c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T08:51:54.727 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:51:54.730 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:51:55.128 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T08:51:55.141 DEBUG:teuthology.orchestra.run.smithi203:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:51:55.224 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:55.232+0000 7f0c374b87c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T08:51:55.224 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:55.232+0000 7f0c374b87c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T08:51:55.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:55.232+0000 7f0c374b87c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:51:55.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:55.232+0000 7f0c374b87c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:51:55.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:55.232+0000 7f0c374b87c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:51:55.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:51:55.232+0000 7f0c374b87c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T08:51:58.192 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T08:51:58.251 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T08:51:58.252 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:58.252 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T08:51:58.266 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:58.266 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T08:51:58.320 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:58.320 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T08:51:58.377 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:58.377 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T08:51:58.434 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:58.434 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T08:51:58.488 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:58.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T08:51:58.544 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:58.544 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T08:51:58.600 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:58.600 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T08:51:58.614 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:58.614 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T08:51:58.670 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:58.670 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T08:51:58.725 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:58.725 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T08:51:58.780 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:58.781 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T08:51:58.833 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:58.833 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T08:51:58.889 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:58.889 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T08:51:58.941 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:58.941 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T08:51:58.947 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:51:58.947 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T08:51:58.993 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:58.993 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T08:51:58.999 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:51:58.999 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T08:51:59.047 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T08:51:59.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:51:59.049 DEBUG:teuthology.orchestra.run.smithi203:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:51:59.092 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.y] 2024-04-25T08:51:59.092 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCOGSpm3NLHNhAAeqSA658L6P9OcrSO5JVzLg== 2024-04-25T08:51:59.092 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-25T08:51:59.092 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpm2pLfABAAJbn2dGKdl8SHS8pApIWC1Q== 2024-04-25T08:51:59.092 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-25T08:51:59.092 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmiXRqCRAACtY8cFxwqOT4WNzj9WmeKA== 2024-04-25T08:51:59.092 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.0] 2024-04-25T08:51:59.092 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCSGSpmwhMiJhAAbmQz36LujHeYXaKIOdLLSA== 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.1] 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCVGSpm/OLPKxAA9ZmrrWxvt4QDj2lHATnzSg== 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.2] 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCYGSpmq7tEMxAA/wjwgkrtnTkrMoC6G+YIlQ== 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.3] 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCbGSpmReB1ORAAkXje/r11I3ArZHErIvijgw== 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr.x] 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCOGSpmLChTORAAo6fs/9PDJ5YqqcbfHfPO9Q== 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmqB+jDxAAmRnBzN3vqO8HIf/cd3QoJQ== 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmX3ARGBAAW/ZUr85NsKtd8HzxE+Yefw== 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.4] 2024-04-25T08:51:59.093 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQChGSpm4XfLORAAGj+R1Nk/EGmvEdpt8kQJlg== 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.5] 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQClGSpmd5f1ARAAKF3XG//v3Uqx4bckcD4JMw== 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.6] 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCoGSpmi9OECBAAJBskMWGCNlAaej9xUc6g8w== 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout:[osd.7] 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCrGSpmunQiDhAA3UZo2vzT44HEYUBUia75NA== 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA== 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout:[client.2] 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmGVVdIxAAY7/VOuqF5OoomqqVxE1Ifw== 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout:[client.1] 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmu8UoJhAAyutD5hzKLZmr57WuAWScuQ== 2024-04-25T08:51:59.094 INFO:teuthology.orchestra.run.smithi037.stdout:[client.3] 2024-04-25T08:51:59.095 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmjFNDKxAAmo6DB9W17TZaGlvkdqkSuQ== 2024-04-25T08:51:59.100 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.y] 2024-04-25T08:51:59.100 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCOGSpm3NLHNhAAeqSA658L6P9OcrSO5JVzLg== 2024-04-25T08:51:59.100 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-25T08:51:59.100 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPGSpm2pLfABAAJbn2dGKdl8SHS8pApIWC1Q== 2024-04-25T08:51:59.100 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-25T08:51:59.100 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPGSpmiXRqCRAACtY8cFxwqOT4WNzj9WmeKA== 2024-04-25T08:51:59.100 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.0] 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCSGSpmwhMiJhAAbmQz36LujHeYXaKIOdLLSA== 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.1] 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCVGSpm/OLPKxAA9ZmrrWxvt4QDj2lHATnzSg== 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.2] 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCYGSpmq7tEMxAA/wjwgkrtnTkrMoC6G+YIlQ== 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.3] 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCbGSpmReB1ORAAkXje/r11I3ArZHErIvijgw== 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr.x] 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCOGSpmLChTORAAo6fs/9PDJ5YqqcbfHfPO9Q== 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPGSpmqB+jDxAAmRnBzN3vqO8HIf/cd3QoJQ== 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPGSpmX3ARGBAAW/ZUr85NsKtd8HzxE+Yefw== 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.4] 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQChGSpm4XfLORAAGj+R1Nk/EGmvEdpt8kQJlg== 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.5] 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQClGSpmd5f1ARAAKF3XG//v3Uqx4bckcD4JMw== 2024-04-25T08:51:59.101 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.6] 2024-04-25T08:51:59.102 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCoGSpmi9OECBAAJBskMWGCNlAaej9xUc6g8w== 2024-04-25T08:51:59.102 INFO:teuthology.orchestra.run.smithi203.stdout:[osd.7] 2024-04-25T08:51:59.102 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCrGSpmunQiDhAA3UZo2vzT44HEYUBUia75NA== 2024-04-25T08:51:59.102 INFO:teuthology.orchestra.run.smithi203.stdout:[client.0] 2024-04-25T08:51:59.102 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA== 2024-04-25T08:51:59.102 INFO:teuthology.orchestra.run.smithi203.stdout:[client.2] 2024-04-25T08:51:59.102 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPGSpmGVVdIxAAY7/VOuqF5OoomqqVxE1Ifw== 2024-04-25T08:51:59.102 INFO:teuthology.orchestra.run.smithi203.stdout:[client.1] 2024-04-25T08:51:59.102 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPGSpmu8UoJhAAyutD5hzKLZmr57WuAWScuQ== 2024-04-25T08:51:59.102 INFO:teuthology.orchestra.run.smithi203.stdout:[client.3] 2024-04-25T08:51:59.102 INFO:teuthology.orchestra.run.smithi203.stdout: key = AQCPGSpmjFNDKxAAmo6DB9W17TZaGlvkdqkSuQ== 2024-04-25T08:51:59.103 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:51:59.138 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:51:59.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:51:59.193 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:51:59.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:51:59.241 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:51:59.323 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:51:59.326 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:51:59.403 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:51:59.410 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:51:59.482 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:51:59.498 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:51:59.572 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:51:59.586 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:51:59.652 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:51:59.669 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:51:59.735 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:51:59.758 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:51:59.825 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:51:59.829 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:51:59.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:51:59.918 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:51:59.960 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:52:00.002 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:52:00.051 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:52:00.089 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:52:00.141 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:52:00.144 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:52:00.225 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:52:00.228 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:52:00.315 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:52:00.318 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:52:00.362 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:52:00.364 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:52:00.410 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:52:00.412 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:52:00.491 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T08:52:00.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T08:52:00.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:52:00.593 DEBUG:teuthology.orchestra.run.smithi037:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T08:52:00.605 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T08:52:00.619 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:52:00.710 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T08:52:00.720 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T08:52:00.770 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:52:00.863 DEBUG:teuthology.orchestra.run.smithi203:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T08:52:00.873 DEBUG:teuthology.orchestra.run.smithi037:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:52:00.875 DEBUG:teuthology.orchestra.run.smithi203:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:52:00.921 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T08:52:00.921 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T08:52:00.921 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T08:52:00.924 INFO:tasks.ceph.mon.a:Started 2024-04-25T08:52:00.924 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T08:52:00.924 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T08:52:00.967 INFO:tasks.ceph.mon.b:Started 2024-04-25T08:52:00.968 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T08:52:00.968 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T08:52:00.970 INFO:tasks.ceph.mon.c:Started 2024-04-25T08:52:00.970 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T08:52:00.970 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T08:52:00.970 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T08:52:00.972 INFO:tasks.ceph.mgr.y:Started 2024-04-25T08:52:00.973 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T08:52:00.973 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T08:52:00.974 INFO:tasks.ceph.mgr.x:Started 2024-04-25T08:52:00.975 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T08:52:00.975 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T08:52:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7d1f7640 1 Processor -- start 2024-04-25T08:52:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7d1f7640 1 -- start start 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7d1f7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7d1f7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7d1f7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7d1f7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7809a950 con 0x7fdd780b0440 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7d1f7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7809ac30 con 0x7fdd7815b920 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7d1f7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780ae380 con 0x7fdd780aeb80 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd6e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd6e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44444/0 (socket says 172.21.15.37:44444) 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd6e59c640 1 -- 172.21.15.37:0/2837544610 learned_addr learned my addr 172.21.15.37:0/2837544610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd6e59c640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 msgr2=0x7fdd78099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd6e59c640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 msgr2=0x7fdd78099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:52:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd76d9d640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 msgr2=0x7fdd78099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T08:52:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd76d9d640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 msgr2=0x7fdd78099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7759e640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 msgr2=0x7fdd780aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T08:52:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7759e640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 msgr2=0x7fdd780aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:52:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.097+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:52:01.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.297+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:01.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.297+0000 7fdd6e59c640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 msgr2=0x7fdd78099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.297+0000 7fdd6e59c640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 msgr2=0x7fdd78099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.297+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.297+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.297+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.297+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.301+0000 7fdd76d9d640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 msgr2=0x7fdd78099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.301+0000 7fdd76d9d640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 msgr2=0x7fdd78099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.301+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.301+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.301+0000 7fdd7759e640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 msgr2=0x7fdd780aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.301+0000 7fdd7759e640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 msgr2=0x7fdd780aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.301+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:01.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.301+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T08:52:01.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:01.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd6e59c640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 msgr2=0x7fdd78099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T08:52:01.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd6e59c640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 msgr2=0x7fdd78099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:01.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:01.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T08:52:01.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:01.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:01.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd76d9d640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 msgr2=0x7fdd78099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T08:52:01.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd76d9d640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 msgr2=0x7fdd78099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:01.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:01.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd7759e640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 msgr2=0x7fdd780aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T08:52:01.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd7759e640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 msgr2=0x7fdd780aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:01.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:01.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T08:52:01.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:01.701+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T08:52:02.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:02.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd6e59c640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 msgr2=0x7fdd78099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T08:52:02.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd6e59c640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 msgr2=0x7fdd78099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:02.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:02.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T08:52:02.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd76d9d640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 msgr2=0x7fdd78099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T08:52:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd76d9d640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 msgr2=0x7fdd78099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T08:52:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd7759e640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 msgr2=0x7fdd780aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T08:52:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd7759e640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 msgr2=0x7fdd780aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:52:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:52:02.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:02.501+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T08:52:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:04.097+0000 7fdd7559a640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 msgr2=0x7fdd780aef90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:52:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:04.097+0000 7fdd7559a640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd780aeb80 0x7fdd780aef90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:04.097+0000 7fdd7559a640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 msgr2=0x7fdd78099a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:52:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:04.097+0000 7fdd7559a640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.203:3300/0 conn(0x7fdd7815b920 0x7fdd78099a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:04.097+0000 7fdd7559a640 1 -- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 msgr2=0x7fdd78099fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:52:04.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:04.097+0000 7fdd7559a640 1 --2- 172.21.15.37:0/2837544610 >> v2:172.21.15.37:3300/0 conn(0x7fdd780b0440 0x7fdd78099fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:04.097+0000 7fdd7559a640 1 -- 172.21.15.37:0/2837544610 --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd780ae660 con 0x7fdd64007310 2024-04-25T08:52:04.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:04.097+0000 7fdd7559a640 1 -- 172.21.15.37:0/2837544610 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6400c080 con 0x7fdd640036c0 2024-04-25T08:52:06.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.369+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378426214 0 0) 0x7fdd780ae660 con 0x7fdd64007310 2024-04-25T08:52:06.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.369+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd58003530 con 0x7fdd64007310 2024-04-25T08:52:06.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801492992 0 0) 0x7fdd6400c080 con 0x7fdd640036c0 2024-04-25T08:52:06.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd780ae660 con 0x7fdd640036c0 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1493297715 0 0) 0x7fdd780ae660 con 0x7fdd640036c0 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 >> v1:172.21.15.37:6789/0 conn(0x7fdd64007310 legacy=0x7fdd640097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6400d2c0 con 0x7fdd640036c0 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fdd6800f040 con 0x7fdd640036c0 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 >> v1:172.21.15.203:6789/0 conn(0x7fdd640036c0 legacy=0x7fdd64005ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7759e640 1 -- 172.21.15.37:0/2837544610 reap_dead start 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58003c60 0x7fdd58004040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd58004810 0x7fdd58004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd580099c0 0x7fdd5800bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5800c3d0 con 0x7fdd58003c60 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5800c580 con 0x7fdd580099c0 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5800c780 con 0x7fdd58004810 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58003c60 0x7fdd58004040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:06.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd58004810 0x7fdd58004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd7759e640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd580099c0 0x7fdd5800bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd76d9d640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd58004810 msgr2=0x7fdd58004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd58004810 0x7fdd58004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd76d9d640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd580099c0 msgr2=0x7fdd5800bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd580099c0 0x7fdd5800bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.373+0000 7fdd76d9d640 1 -- 172.21.15.37:0/2837544610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd780ae660 con 0x7fdd58003c60 2024-04-25T08:52:06.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.381+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58003c60 0x7fdd58004040 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800efa0 tx=0x7fdd6801d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea922e8d1c175e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.385+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6800ebf0 con 0x7fdd58003c60 2024-04-25T08:52:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.385+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd680a76f0 con 0x7fdd58003c60 2024-04-25T08:52:06.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.385+0000 7fdd7659c640 1 -- 172.21.15.37:0/2837544610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd680a7ae0 con 0x7fdd58003c60 2024-04-25T08:52:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.385+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/2837544610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58003c60 msgr2=0x7fdd58004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.385+0000 7fdd7d1f7640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58003c60 0x7fdd58004040 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdd6800efa0 tx=0x7fdd6801d810 comp rx=0 tx=0).stop 2024-04-25T08:52:06.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/2837544610 shutdown_connections 2024-04-25T08:52:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd580099c0 0x7fdd5800bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd58004810 0x7fdd58004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 --2- 172.21.15.37:0/2837544610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd58003c60 0x7fdd58004040 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/2837544610 >> 172.21.15.37:0/2837544610 conn(0x7fdd780b3010 msgr2=0x7fdd780b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/2837544610 shutdown_connections 2024-04-25T08:52:06.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/2837544610 wait complete. 2024-04-25T08:52:06.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 Processor -- start 2024-04-25T08:52:06.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 -- start start 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd58003c60 0x7fdd78156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd58004810 0x7fdd780cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd580099c0 0x7fdd780cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78099bf0 con 0x7fdd580099c0 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78099a70 con 0x7fdd58004810 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7d1f7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd78099d70 con 0x7fdd58003c60 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd580099c0 0x7fdd780cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd580099c0 0x7fdd780cf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44492/0 (socket says 172.21.15.37:44492) 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7759e640 1 -- 172.21.15.37:0/32242191 learned_addr learned my addr 172.21.15.37:0/32242191 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd76d9d640 1 --2- 172.21.15.37:0/32242191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd58003c60 0x7fdd78156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd6e59c640 1 --2- 172.21.15.37:0/32242191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd58004810 0x7fdd780cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:06.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7759e640 1 -- 172.21.15.37:0/32242191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd58003c60 msgr2=0x7fdd78156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7759e640 1 --2- 172.21.15.37:0/32242191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd58003c60 0x7fdd78156380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7759e640 1 -- 172.21.15.37:0/32242191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd58004810 msgr2=0x7fdd780cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7759e640 1 --2- 172.21.15.37:0/32242191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd58004810 0x7fdd780cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.389+0000 7fdd7759e640 1 -- 172.21.15.37:0/32242191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd780cab60 con 0x7fdd580099c0 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.393+0000 7fdd7759e640 1 --2- 172.21.15.37:0/32242191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd580099c0 0x7fdd780cf2f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdd64007280 tx=0x7fdd70015700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.393+0000 7fdd6ffff640 1 -- 172.21.15.37:0/32242191 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7001c030 con 0x7fdd580099c0 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.393+0000 7fdd6ffff640 1 -- 172.21.15.37:0/32242191 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd700046a0 con 0x7fdd580099c0 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.393+0000 7fdd6ffff640 1 -- 172.21.15.37:0/32242191 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70021630 con 0x7fdd580099c0 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.393+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/32242191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd780cae40 con 0x7fdd580099c0 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.393+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/32242191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd780cb400 con 0x7fdd580099c0 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.393+0000 7fdd6ffff640 1 -- 172.21.15.37:0/32242191 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fdd7001f030 con 0x7fdd580099c0 2024-04-25T08:52:06.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.393+0000 7fdd6ffff640 1 -- 172.21.15.37:0/32242191 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdd700203c0 con 0x7fdd580099c0 2024-04-25T08:52:06.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.393+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/32242191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7804efa0 con 0x7fdd580099c0 2024-04-25T08:52:06.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.397+0000 7fdd6ffff640 1 -- 172.21.15.37:0/32242191 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fdd70026030 con 0x7fdd580099c0 2024-04-25T08:52:06.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.461+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/32242191 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fdd7809a940 con 0x7fdd580099c0 2024-04-25T08:52:06.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.461+0000 7fdd6ffff640 1 -- 172.21.15.37:0/32242191 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fdd70010050 con 0x7fdd580099c0 2024-04-25T08:52:06.454 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-04-25T08:52:06.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.461+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/32242191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd580099c0 msgr2=0x7fdd780cf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:06.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.461+0000 7fdd7d1f7640 1 --2- 172.21.15.37:0/32242191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd580099c0 0x7fdd780cf2f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fdd64007280 tx=0x7fdd70015700 comp rx=0 tx=0).stop 2024-04-25T08:52:06.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.461+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/32242191 shutdown_connections 2024-04-25T08:52:06.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.461+0000 7fdd7d1f7640 1 --2- 172.21.15.37:0/32242191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd580099c0 0x7fdd780cf2f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.461+0000 7fdd7d1f7640 1 --2- 172.21.15.37:0/32242191 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdd58004810 0x7fdd780cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.461+0000 7fdd7d1f7640 1 --2- 172.21.15.37:0/32242191 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdd58003c60 0x7fdd78156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.461+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/32242191 >> 172.21.15.37:0/32242191 conn(0x7fdd780b3010 msgr2=0x7fdd7815aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:06.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.461+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/32242191 shutdown_connections 2024-04-25T08:52:06.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:06.461+0000 7fdd7d1f7640 1 -- 172.21.15.37:0/32242191 wait complete. 2024-04-25T08:52:06.473 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T08:52:06.473 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T08:52:06.473 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:52:06.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T08:52:06.484 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:52:06.484 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T08:52:06.538 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:52:06.538 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T08:52:06.590 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:52:06.590 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T08:52:06.591 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:06.600+0000 7f61c5ec3180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:52:06.646 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:52:06.646 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T08:52:06.660 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:52:06.660 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T08:52:06.714 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:52:06.714 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T08:52:06.770 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:52:06.770 DEBUG:teuthology.orchestra.run.smithi203:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T08:52:06.799 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:06.805+0000 7fcee9afa180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:52:06.824 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 7eda0189-e0df-47ea-99d9-17bbe2fe40cd 0 2024-04-25T08:52:06.989 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.995+0000 7f0cf55e4640 1 Processor -- start 2024-04-25T08:52:06.990 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cf55e4640 1 -- start start 2024-04-25T08:52:06.991 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cf55e4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 0x7f0cf00cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:06.991 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cf55e4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0cf015d180 0x7f0cf015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:06.991 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cf55e4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf006b710 con 0x7f0cf006fc50 2024-04-25T08:52:06.991 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cf55e4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf0161560 con 0x7f0cf015d180 2024-04-25T08:52:06.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cf55e4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf01617b0 con 0x7f0cf00cf840 2024-04-25T08:52:06.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cee7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 0x7f0cf00cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:06.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0ceeffd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0cf006fc50 0x7f0cf006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56394/0 (socket says 172.21.15.203:56394) 2024-04-25T08:52:06.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cef7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0cf015d180 0x7f0cf015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:06.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0ceeffd640 1 -- 172.21.15.203:0/4158757400 learned_addr learned my addr 172.21.15.203:0/4158757400 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-04-25T08:52:06.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cee7fc640 1 -- 172.21.15.203:0/4158757400 >> v2:172.21.15.203:3300/0 conn(0x7f0cf015d180 msgr2=0x7f0cf015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:06.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cee7fc640 1 --2- 172.21.15.203:0/4158757400 >> v2:172.21.15.203:3300/0 conn(0x7f0cf015d180 0x7f0cf015d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cee7fc640 1 -- 172.21.15.203:0/4158757400 >> v1:172.21.15.203:6789/0 conn(0x7f0cf006fc50 legacy=0x7f0cf006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:06.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cedffb640 1 -- 172.21.15.203:0/4158757400 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462064825 0 0) 0x7f0cf006b710 con 0x7f0cf006fc50 2024-04-25T08:52:06.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cee7fc640 1 -- 172.21.15.203:0/4158757400 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf0161a90 con 0x7f0cf00cf840 2024-04-25T08:52:06.992 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:06.999+0000 7f0cef7fe640 1 --2- 172.21.15.203:0/4158757400 >> v2:172.21.15.203:3300/0 conn(0x7f0cf015d180 0x7f0cf015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:06.993 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cee7fc640 1 --2- 172.21.15.203:0/4158757400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 0x7f0cf00cfca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40097c0 tx=0x7f0ce401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fb8477957da3496 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:06.993 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cedffb640 1 -- 172.21.15.203:0/4158757400 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40b2070 con 0x7f0cf00cf840 2024-04-25T08:52:06.994 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cedffb640 1 -- 172.21.15.203:0/4158757400 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce40ad680 con 0x7f0cf00cf840 2024-04-25T08:52:06.994 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cedffb640 1 -- 172.21.15.203:0/4158757400 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce40ada50 con 0x7f0cf00cf840 2024-04-25T08:52:06.995 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 -- 172.21.15.203:0/4158757400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 msgr2=0x7f0cf00cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:06.995 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 --2- 172.21.15.203:0/4158757400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 0x7f0cf00cfca0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ce40097c0 tx=0x7f0ce401c470 comp rx=0 tx=0).stop 2024-04-25T08:52:06.995 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 -- 172.21.15.203:0/4158757400 shutdown_connections 2024-04-25T08:52:06.995 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 --2- 172.21.15.203:0/4158757400 >> v2:172.21.15.203:3300/0 conn(0x7f0cf015d180 0x7f0cf015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.995 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 --2- 172.21.15.203:0/4158757400 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 0x7f0cf00cfca0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:06.995 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 -- 172.21.15.203:0/4158757400 >> 172.21.15.203:0/4158757400 conn(0x7f0cf0099a70 msgr2=0x7f0cf00d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:06.995 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 -- 172.21.15.203:0/4158757400 shutdown_connections 2024-04-25T08:52:06.995 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 -- 172.21.15.203:0/4158757400 wait complete. 2024-04-25T08:52:06.996 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 Processor -- start 2024-04-25T08:52:06.996 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 -- start start 2024-04-25T08:52:06.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf006fc50 0x7f0cf0152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:06.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 0x7f0cf00b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:06.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf015d180 0x7f0cf00b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:06.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf0070030 con 0x7f0cf006fc50 2024-04-25T08:52:06.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf00d0e30 con 0x7f0cf015d180 2024-04-25T08:52:06.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cf55e4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf00701b0 con 0x7f0cf00cf840 2024-04-25T08:52:06.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cee7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 0x7f0cf00b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:06.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cef7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf015d180 0x7f0cf00b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:06.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cef7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf015d180 0x7f0cf00b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48846/0 (socket says 172.21.15.203:48846) 2024-04-25T08:52:06.999 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cee7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 0x7f0cf00b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33884/0 (socket says 172.21.15.203:33884) 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.003+0000 7f0cef7fe640 1 -- 172.21.15.203:0/721651002 learned_addr learned my addr 172.21.15.203:0/721651002 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0ceeffd640 1 --2- 172.21.15.203:0/721651002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf006fc50 0x7f0cf0152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cef7fe640 1 -- 172.21.15.203:0/721651002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 msgr2=0x7f0cf00b3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cef7fe640 1 --2- 172.21.15.203:0/721651002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 0x7f0cf00b3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cef7fe640 1 -- 172.21.15.203:0/721651002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf006fc50 msgr2=0x7f0cf0152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cef7fe640 1 --2- 172.21.15.203:0/721651002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf006fc50 0x7f0cf0152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cef7fe640 1 -- 172.21.15.203:0/721651002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf00b46f0 con 0x7f0cf015d180 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cee7fc640 1 --2- 172.21.15.203:0/721651002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 0x7f0cf00b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0ceeffd640 1 --2- 172.21.15.203:0/721651002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf006fc50 0x7f0cf0152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cef7fe640 1 --2- 172.21.15.203:0/721651002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf015d180 0x7f0cf00b4070 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800efc0 tx=0x7f0ce800c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cd77fe640 1 -- 172.21.15.203:0/721651002 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8018030 con 0x7f0cf015d180 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cd77fe640 1 -- 172.21.15.203:0/721651002 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ce800cdf0 con 0x7f0cf015d180 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cf55e4640 1 -- 172.21.15.203:0/721651002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf00b49d0 con 0x7f0cf015d180 2024-04-25T08:52:07.000 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cd77fe640 1 -- 172.21.15.203:0/721651002 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8003bc0 con 0x7f0cf015d180 2024-04-25T08:52:07.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cf55e4640 1 -- 172.21.15.203:0/721651002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf00b4f90 con 0x7f0cf015d180 2024-04-25T08:52:07.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cd77fe640 1 -- 172.21.15.203:0/721651002 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0ce801b070 con 0x7f0cf015d180 2024-04-25T08:52:07.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cd77fe640 1 -- 172.21.15.203:0/721651002 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0ce801d830 con 0x7f0cf015d180 2024-04-25T08:52:07.001 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.007+0000 7f0cf55e4640 1 -- 172.21.15.203:0/721651002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf004efa0 con 0x7f0cf015d180 2024-04-25T08:52:07.011 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.019+0000 7f0cd77fe640 1 -- 172.21.15.203:0/721651002 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0ce8010040 con 0x7f0cf015d180 2024-04-25T08:52:07.080 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.087+0000 7f0cf55e4640 1 -- 172.21.15.203:0/721651002 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7eda0189-e0df-47ea-99d9-17bbe2fe40cd", "id": 0} v 0) v1 -- 0x7f0cf00b6480 con 0x7f0cf015d180 2024-04-25T08:52:07.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.095+0000 7f0cd77fe640 1 -- 172.21.15.203:0/721651002 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7eda0189-e0df-47ea-99d9-17bbe2fe40cd", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0ce801db00 con 0x7f0cf015d180 2024-04-25T08:52:07.086 INFO:teuthology.orchestra.run.smithi203.stdout:0 2024-04-25T08:52:07.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.095+0000 7f0cf55e4640 1 -- 172.21.15.203:0/721651002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf015d180 msgr2=0x7f0cf00b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.095+0000 7f0cf55e4640 1 --2- 172.21.15.203:0/721651002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf015d180 0x7f0cf00b4070 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0ce800efc0 tx=0x7f0ce800c490 comp rx=0 tx=0).stop 2024-04-25T08:52:07.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.095+0000 7f0cf55e4640 1 -- 172.21.15.203:0/721651002 shutdown_connections 2024-04-25T08:52:07.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.095+0000 7f0cf55e4640 1 --2- 172.21.15.203:0/721651002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cf015d180 0x7f0cf00b4070 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.095+0000 7f0cf55e4640 1 --2- 172.21.15.203:0/721651002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cf00cf840 0x7f0cf00b3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.095+0000 7f0cf55e4640 1 --2- 172.21.15.203:0/721651002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cf006fc50 0x7f0cf0152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.088 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.095+0000 7f0cf55e4640 1 -- 172.21.15.203:0/721651002 >> 172.21.15.203:0/721651002 conn(0x7f0cf0099a70 msgr2=0x7f0cf00aed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:07.088 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.095+0000 7f0cf55e4640 1 -- 172.21.15.203:0/721651002 shutdown_connections 2024-04-25T08:52:07.088 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.095+0000 7f0cf55e4640 1 -- 172.21.15.203:0/721651002 wait complete. 2024-04-25T08:52:07.094 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:07.103+0000 7f61c5ec3180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:52:07.103 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 8b0293a9-db94-43c9-bd5b-fe75890d1599 1 2024-04-25T08:52:07.217 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.223+0000 7fcecf946640 1 Processor -- start 2024-04-25T08:52:07.217 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fcecf946640 1 -- start start 2024-04-25T08:52:07.219 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fcecf946640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcec815b920 0x7fcec8099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.219 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fcecf946640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.219 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fcecf946640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec809a950 con 0x7fcec80aeb80 2024-04-25T08:52:07.219 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fcecf946640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec809ac30 con 0x7fcec815b920 2024-04-25T08:52:07.219 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fcecf946640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80ae380 con 0x7fcec80b0440 2024-04-25T08:52:07.219 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fceccee1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.219 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fcecdee3640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fcec80aeb80 0x7fcec80aeff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56410/0 (socket says 172.21.15.203:56410) 2024-04-25T08:52:07.219 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fceccee1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8099fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33900/0 (socket says 172.21.15.203:33900) 2024-04-25T08:52:07.219 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fcecdee3640 1 -- 172.21.15.203:0/2665229534 learned_addr learned my addr 172.21.15.203:0/2665229534 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-04-25T08:52:07.219 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fceccee1640 1 -- 172.21.15.203:0/2665229534 >> v2:172.21.15.37:3300/0 conn(0x7fcec815b920 msgr2=0x7fcec8099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:07.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fceccee1640 1 --2- 172.21.15.203:0/2665229534 >> v2:172.21.15.37:3300/0 conn(0x7fcec815b920 0x7fcec8099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fceccee1640 1 -- 172.21.15.203:0/2665229534 >> v1:172.21.15.203:6789/0 conn(0x7fcec80aeb80 legacy=0x7fcec80aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fcebffff640 1 -- 172.21.15.203:0/2665229534 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321473622 0 0) 0x7fcec809a950 con 0x7fcec80aeb80 2024-04-25T08:52:07.220 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fceccee1640 1 -- 172.21.15.203:0/2665229534 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec80ae660 con 0x7fcec80b0440 2024-04-25T08:52:07.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fceccee1640 1 --2- 172.21.15.203:0/2665229534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8099fb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fceb80097c0 tx=0x7fceb801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcecb537fd2b7d4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:07.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fcebffff640 1 -- 172.21.15.203:0/2665229534 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb80b0070 con 0x7fcec80b0440 2024-04-25T08:52:07.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.227+0000 7fcebffff640 1 -- 172.21.15.203:0/2665229534 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb80ab600 con 0x7fcec80b0440 2024-04-25T08:52:07.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcebffff640 1 -- 172.21.15.203:0/2665229534 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb80ab9d0 con 0x7fcec80b0440 2024-04-25T08:52:07.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 -- 172.21.15.203:0/2665229534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 msgr2=0x7fcec8099fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 --2- 172.21.15.203:0/2665229534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8099fb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fceb80097c0 tx=0x7fceb801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 -- 172.21.15.203:0/2665229534 shutdown_connections 2024-04-25T08:52:07.222 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 --2- 172.21.15.203:0/2665229534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8099fb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 --2- 172.21.15.203:0/2665229534 >> v2:172.21.15.37:3300/0 conn(0x7fcec815b920 0x7fcec8099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 -- 172.21.15.203:0/2665229534 >> 172.21.15.203:0/2665229534 conn(0x7fcec80b3010 msgr2=0x7fcec80b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:07.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 -- 172.21.15.203:0/2665229534 shutdown_connections 2024-04-25T08:52:07.223 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 -- 172.21.15.203:0/2665229534 wait complete. 2024-04-25T08:52:07.225 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 Processor -- start 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 -- start start 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec80aeb80 0x7fcec8154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec815b920 0x7fcec80b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec8168850 con 0x7fcec80aeb80 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec81686d0 con 0x7fcec815b920 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec81689d0 con 0x7fcec80b0440 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fceccee1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fceccee1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8145860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33904/0 (socket says 172.21.15.203:33904) 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fceccee1640 1 -- 172.21.15.203:0/3735262141 learned_addr learned my addr 172.21.15.203:0/3735262141 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecdee3640 1 --2- 172.21.15.203:0/3735262141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec815b920 0x7fcec80b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecd6e2640 1 --2- 172.21.15.203:0/3735262141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec80aeb80 0x7fcec8154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.226 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fceccee1640 1 -- 172.21.15.203:0/3735262141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec815b920 msgr2=0x7fcec80b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fceccee1640 1 --2- 172.21.15.203:0/3735262141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec815b920 0x7fcec80b4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fceccee1640 1 -- 172.21.15.203:0/3735262141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec80aeb80 msgr2=0x7fcec8154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fceccee1640 1 --2- 172.21.15.203:0/3735262141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec80aeb80 0x7fcec8154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fceccee1640 1 -- 172.21.15.203:0/3735262141 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec815c180 con 0x7fcec80b0440 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecd6e2640 1 --2- 172.21.15.203:0/3735262141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec80aeb80 0x7fcec8154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fceccee1640 1 --2- 172.21.15.203:0/3735262141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8145860 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fceb80ae040 tx=0x7fceb801ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcebdffb640 1 -- 172.21.15.203:0/3735262141 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb80b0050 con 0x7fcec80b0440 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcebdffb640 1 -- 172.21.15.203:0/3735262141 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb80bc070 con 0x7fcec80b0440 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 -- 172.21.15.203:0/3735262141 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec815c460 con 0x7fcec80b0440 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcebdffb640 1 -- 172.21.15.203:0/3735262141 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb80b5a70 con 0x7fcec80b0440 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcecf946640 1 -- 172.21.15.203:0/3735262141 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec815ca20 con 0x7fcec80b0440 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcebdffb640 1 -- 172.21.15.203:0/3735262141 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fceb80ab4c0 con 0x7fcec80b0440 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.231+0000 7fcebdffb640 1 -- 172.21.15.203:0/3735262141 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fceb80c1df0 con 0x7fcec80b0440 2024-04-25T08:52:07.227 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.235+0000 7fcecf946640 1 -- 172.21.15.203:0/3735262141 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec809a300 con 0x7fcec80b0440 2024-04-25T08:52:07.228 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.235+0000 7fcebdffb640 1 -- 172.21.15.203:0/3735262141 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fceb8016090 con 0x7fcec80b0440 2024-04-25T08:52:07.293 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:07.299+0000 7f61c5ec3180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:52:07.295 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.303+0000 7fcecf946640 1 -- 172.21.15.203:0/3735262141 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8b0293a9-db94-43c9-bd5b-fe75890d1599", "id": 1} v 0) v1 -- 0x7fcec804efa0 con 0x7fcec80b0440 2024-04-25T08:52:07.303 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.311+0000 7fcebdffb640 1 -- 172.21.15.203:0/3735262141 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8b0293a9-db94-43c9-bd5b-fe75890d1599", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fceb80c1460 con 0x7fcec80b0440 2024-04-25T08:52:07.303 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-25T08:52:07.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.311+0000 7fcecf946640 1 -- 172.21.15.203:0/3735262141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 msgr2=0x7fcec8145860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.311+0000 7fcecf946640 1 --2- 172.21.15.203:0/3735262141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8145860 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fceb80ae040 tx=0x7fceb801ace0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.311+0000 7fcecf946640 1 -- 172.21.15.203:0/3735262141 shutdown_connections 2024-04-25T08:52:07.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.311+0000 7fcecf946640 1 --2- 172.21.15.203:0/3735262141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcec815b920 0x7fcec80b4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.311+0000 7fcecf946640 1 --2- 172.21.15.203:0/3735262141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcec80b0440 0x7fcec8145860 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.311+0000 7fcecf946640 1 --2- 172.21.15.203:0/3735262141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcec80aeb80 0x7fcec8154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.311+0000 7fcecf946640 1 -- 172.21.15.203:0/3735262141 >> 172.21.15.203:0/3735262141 conn(0x7fcec80b3010 msgr2=0x7fcec80cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:07.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.311+0000 7fcecf946640 1 -- 172.21.15.203:0/3735262141 shutdown_connections 2024-04-25T08:52:07.305 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.311+0000 7fcecf946640 1 -- 172.21.15.203:0/3735262141 wait complete. 2024-04-25T08:52:07.320 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 9c8f8bd3-2f80-401c-8091-4d29cb03cf4f 2 2024-04-25T08:52:07.362 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:07.369+0000 7fcee9afa180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:52:07.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.439+0000 7f10f3c28640 1 Processor -- start 2024-04-25T08:52:07.432 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.439+0000 7f10f3c28640 1 -- start start 2024-04-25T08:52:07.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f3c28640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10ec0cd350 0x7f10ec15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f3c28640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f10ec0cf0e0 0x7f10ec15b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f3c28640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ec0ce320 con 0x7f10ec15aac0 2024-04-25T08:52:07.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f3c28640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ec0ce0d0 con 0x7f10ec0cf0e0 2024-04-25T08:52:07.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f3c28640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ec0ce600 con 0x7f10ec0cd350 2024-04-25T08:52:07.434 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f11c3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10ec0cd350 0x7f10ec15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.435 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f11c3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10ec0cd350 0x7f10ec15aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48864/0 (socket says 172.21.15.203:48864) 2024-04-25T08:52:07.435 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f11c3640 1 -- 172.21.15.203:0/2226783613 learned_addr learned my addr 172.21.15.203:0/2226783613 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T08:52:07.435 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f21c5640 1 --2- 172.21.15.203:0/2226783613 >> v2:172.21.15.37:3300/0 conn(0x7f10ec0cf0e0 0x7f10ec15b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.435 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f11c3640 1 -- 172.21.15.203:0/2226783613 >> v2:172.21.15.37:3300/0 conn(0x7f10ec0cf0e0 msgr2=0x7f10ec15b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:07.435 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f11c3640 1 --2- 172.21.15.203:0/2226783613 >> v2:172.21.15.37:3300/0 conn(0x7f10ec0cf0e0 0x7f10ec15b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.435 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f11c3640 1 -- 172.21.15.203:0/2226783613 >> v1:172.21.15.37:6789/0 conn(0x7f10ec15aac0 legacy=0x7f10ec0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:07.435 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f11c3640 1 -- 172.21.15.203:0/2226783613 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ec0ce8e0 con 0x7f10ec0cd350 2024-04-25T08:52:07.437 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f11c3640 1 --2- 172.21.15.203:0/2226783613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cd350 0x7f10ec15aea0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0cadc0 tx=0x7f10dc01ebb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f79d09126f0ce6f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:07.437 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f09c2640 1 -- 172.21.15.203:0/2226783613 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0b4070 con 0x7f10ec0cd350 2024-04-25T08:52:07.437 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f09c2640 1 -- 172.21.15.203:0/2226783613 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10dc0a7a90 con 0x7f10ec0cd350 2024-04-25T08:52:07.437 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.443+0000 7f10f09c2640 1 -- 172.21.15.203:0/2226783613 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0a7e60 con 0x7f10ec0cd350 2024-04-25T08:52:07.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- 172.21.15.203:0/2226783613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cd350 msgr2=0x7f10ec15aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 --2- 172.21.15.203:0/2226783613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cd350 0x7f10ec15aea0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f10ec0cadc0 tx=0x7f10dc01ebb0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- 172.21.15.203:0/2226783613 shutdown_connections 2024-04-25T08:52:07.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 --2- 172.21.15.203:0/2226783613 >> v2:172.21.15.37:3300/0 conn(0x7f10ec0cf0e0 0x7f10ec15b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 --2- 172.21.15.203:0/2226783613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cd350 0x7f10ec15aea0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- 172.21.15.203:0/2226783613 >> 172.21.15.203:0/2226783613 conn(0x7f10ec099a70 msgr2=0x7f10ec0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:07.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- 172.21.15.203:0/2226783613 shutdown_connections 2024-04-25T08:52:07.438 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- 172.21.15.203:0/2226783613 wait complete. 2024-04-25T08:52:07.439 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 Processor -- start 2024-04-25T08:52:07.439 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- start start 2024-04-25T08:52:07.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cd350 0x7f10ec14f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.441 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ec0cf0e0 0x7f10ec157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec15aac0 0x7f10ec06e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec06b830 con 0x7f10ec15aac0 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec06b6b0 con 0x7f10ec0cd350 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ec06b9b0 con 0x7f10ec0cf0e0 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f19c4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cd350 0x7f10ec14f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f11c3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ec0cf0e0 0x7f10ec157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f19c4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cd350 0x7f10ec14f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48870/0 (socket says 172.21.15.203:48870) 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f19c4640 1 -- 172.21.15.203:0/1725356999 learned_addr learned my addr 172.21.15.203:0/1725356999 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f19c4640 1 -- 172.21.15.203:0/1725356999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ec0cf0e0 msgr2=0x7f10ec157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f19c4640 1 --2- 172.21.15.203:0/1725356999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ec0cf0e0 0x7f10ec157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f19c4640 1 -- 172.21.15.203:0/1725356999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec15aac0 msgr2=0x7f10ec06e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.442 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f21c5640 1 --2- 172.21.15.203:0/1725356999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec15aac0 0x7f10ec06e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f19c4640 1 --2- 172.21.15.203:0/1725356999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec15aac0 0x7f10ec06e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f19c4640 1 -- 172.21.15.203:0/1725356999 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ec159e60 con 0x7f10ec0cd350 2024-04-25T08:52:07.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f19c4640 1 --2- 172.21.15.203:0/1725356999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cd350 0x7f10ec14f630 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f10e800b440 tx=0x7f10e800b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:07.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10e27fc640 1 -- 172.21.15.203:0/1725356999 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8015020 con 0x7f10ec0cd350 2024-04-25T08:52:07.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10e27fc640 1 -- 172.21.15.203:0/1725356999 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e8011890 con 0x7f10ec0cd350 2024-04-25T08:52:07.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- 172.21.15.203:0/1725356999 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ec158dc0 con 0x7f10ec0cd350 2024-04-25T08:52:07.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10e27fc640 1 -- 172.21.15.203:0/1725356999 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8011c80 con 0x7f10ec0cd350 2024-04-25T08:52:07.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- 172.21.15.203:0/1725356999 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ec159380 con 0x7f10ec0cd350 2024-04-25T08:52:07.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10e27fc640 1 -- 172.21.15.203:0/1725356999 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f10e8018020 con 0x7f10ec0cd350 2024-04-25T08:52:07.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10e27fc640 1 -- 172.21.15.203:0/1725356999 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f10e801a820 con 0x7f10ec0cd350 2024-04-25T08:52:07.443 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.447+0000 7f10f3c28640 1 -- 172.21.15.203:0/1725356999 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ec159aa0 con 0x7f10ec0cd350 2024-04-25T08:52:07.444 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.451+0000 7f10e27fc640 1 -- 172.21.15.203:0/1725356999 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f10e801f030 con 0x7f10ec0cd350 2024-04-25T08:52:07.511 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.519+0000 7f10f3c28640 1 -- 172.21.15.203:0/1725356999 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "9c8f8bd3-2f80-401c-8091-4d29cb03cf4f", "id": 2} v 0) v1 -- 0x7f10ec0cdc90 con 0x7f10ec0cd350 2024-04-25T08:52:07.518 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.527+0000 7f10e27fc640 1 -- 172.21.15.203:0/1725356999 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9c8f8bd3-2f80-401c-8091-4d29cb03cf4f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f10e800d040 con 0x7f10ec0cd350 2024-04-25T08:52:07.519 INFO:teuthology.orchestra.run.smithi203.stdout:2 2024-04-25T08:52:07.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.527+0000 7f10f3c28640 1 -- 172.21.15.203:0/1725356999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cd350 msgr2=0x7f10ec14f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.527+0000 7f10f3c28640 1 --2- 172.21.15.203:0/1725356999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cd350 0x7f10ec14f630 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f10e800b440 tx=0x7f10e800b900 comp rx=0 tx=0).stop 2024-04-25T08:52:07.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.527+0000 7f10f3c28640 1 -- 172.21.15.203:0/1725356999 shutdown_connections 2024-04-25T08:52:07.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.527+0000 7f10f3c28640 1 --2- 172.21.15.203:0/1725356999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10ec15aac0 0x7f10ec06e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.527+0000 7f10f3c28640 1 --2- 172.21.15.203:0/1725356999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10ec0cf0e0 0x7f10ec157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.527+0000 7f10f3c28640 1 --2- 172.21.15.203:0/1725356999 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10ec0cd350 0x7f10ec14f630 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.527+0000 7f10f3c28640 1 -- 172.21.15.203:0/1725356999 >> 172.21.15.203:0/1725356999 conn(0x7f10ec099a70 msgr2=0x7f10ec0ae480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:07.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.527+0000 7f10f3c28640 1 -- 172.21.15.203:0/1725356999 shutdown_connections 2024-04-25T08:52:07.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.527+0000 7f10f3c28640 1 -- 172.21.15.203:0/1725356999 wait complete. 2024-04-25T08:52:07.535 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 38a1085b-a817-4560-8f4e-297ab640dd94 3 2024-04-25T08:52:07.643 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f376291a640 1 Processor -- start 2024-04-25T08:52:07.643 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f376291a640 1 -- start start 2024-04-25T08:52:07.645 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f376291a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0af280 0x7f375c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.645 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f376291a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c066c30 con 0x7f375c0b2c30 2024-04-25T08:52:07.645 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f376291a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c0af660 con 0x7f375c06c3b0 2024-04-25T08:52:07.645 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f376291a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c06daa0 con 0x7f375c0af280 2024-04-25T08:52:07.645 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f375bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0af280 0x7f375c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.645 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f3760eb7640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f375c06c3b0 0x7f375c06df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56420/0 (socket says 172.21.15.203:56420) 2024-04-25T08:52:07.645 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f3760eb7640 1 -- 172.21.15.203:0/544556304 learned_addr learned my addr 172.21.15.203:0/544556304 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-04-25T08:52:07.645 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f375affd640 1 -- 172.21.15.203:0/544556304 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1283213374 0 0) 0x7f375c0af660 con 0x7f375c06c3b0 2024-04-25T08:52:07.645 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f375bfff640 1 -- 172.21.15.203:0/544556304 >> v1:172.21.15.203:6789/0 conn(0x7f375c06c3b0 legacy=0x7f375c06df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.645 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f375bfff640 1 -- 172.21.15.203:0/544556304 >> v1:172.21.15.37:6789/0 conn(0x7f375c0b2c30 legacy=0x7f375c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:07.645 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.651+0000 7f375bfff640 1 -- 172.21.15.203:0/544556304 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c15e640 con 0x7f375c0af280 2024-04-25T08:52:07.648 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.655+0000 7f375bfff640 1 --2- 172.21.15.203:0/544556304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0af280 0x7f375c15df10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f37440097f0 tx=0x7f374401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4544b6692f35c300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:07.648 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.655+0000 7f375affd640 1 -- 172.21.15.203:0/544556304 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37440b0070 con 0x7f375c0af280 2024-04-25T08:52:07.648 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.655+0000 7f375affd640 1 -- 172.21.15.203:0/544556304 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37440ab600 con 0x7f375c0af280 2024-04-25T08:52:07.648 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.655+0000 7f375affd640 1 -- 172.21.15.203:0/544556304 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37440ab9d0 con 0x7f375c0af280 2024-04-25T08:52:07.649 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.655+0000 7f376291a640 1 -- 172.21.15.203:0/544556304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0af280 msgr2=0x7f375c15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.649 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.655+0000 7f376291a640 1 --2- 172.21.15.203:0/544556304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0af280 0x7f375c15df10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f37440097f0 tx=0x7f374401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.649 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.655+0000 7f376291a640 1 -- 172.21.15.203:0/544556304 shutdown_connections 2024-04-25T08:52:07.649 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.655+0000 7f376291a640 1 --2- 172.21.15.203:0/544556304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0af280 0x7f375c15df10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.649 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.655+0000 7f376291a640 1 -- 172.21.15.203:0/544556304 >> 172.21.15.203:0/544556304 conn(0x7f375c071810 msgr2=0x7f375c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:07.649 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.655+0000 7f376291a640 1 -- 172.21.15.203:0/544556304 shutdown_connections 2024-04-25T08:52:07.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.655+0000 7f376291a640 1 -- 172.21.15.203:0/544556304 wait complete. 2024-04-25T08:52:07.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f376291a640 1 Processor -- start 2024-04-25T08:52:07.650 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f376291a640 1 -- start start 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f376291a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f375c06c3b0 0x7f375c14a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f376291a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f375c0af280 0x7f375c1535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f376291a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0b2c30 0x7f375c138010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f376291a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c06f1e0 con 0x7f375c0af280 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f376291a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c06f060 con 0x7f375c06c3b0 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f376291a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c16b8b0 con 0x7f375c0b2c30 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f375bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f375c06c3b0 0x7f375c14a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3760eb7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0b2c30 0x7f375c138010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3760eb7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0b2c30 0x7f375c138010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33926/0 (socket says 172.21.15.203:33926) 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f375bfff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f375c06c3b0 0x7f375c14a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48876/0 (socket says 172.21.15.203:48876) 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3760eb7640 1 -- 172.21.15.203:0/2680100838 learned_addr learned my addr 172.21.15.203:0/2680100838 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T08:52:07.656 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f375b7fe640 1 --2- 172.21.15.203:0/2680100838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f375c0af280 0x7f375c1535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3760eb7640 1 -- 172.21.15.203:0/2680100838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f375c06c3b0 msgr2=0x7f375c14a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3760eb7640 1 --2- 172.21.15.203:0/2680100838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f375c06c3b0 0x7f375c14a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3760eb7640 1 -- 172.21.15.203:0/2680100838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f375c0af280 msgr2=0x7f375c1535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3760eb7640 1 --2- 172.21.15.203:0/2680100838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f375c0af280 0x7f375c1535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3760eb7640 1 -- 172.21.15.203:0/2680100838 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c152b30 con 0x7f375c0b2c30 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f375b7fe640 1 --2- 172.21.15.203:0/2680100838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f375c0af280 0x7f375c1535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3760eb7640 1 --2- 172.21.15.203:0/2680100838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0b2c30 0x7f375c138010 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f375400c7a0 tx=0x7f375400cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3758ff9640 1 -- 172.21.15.203:0/2680100838 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3754018070 con 0x7f375c0b2c30 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3758ff9640 1 -- 172.21.15.203:0/2680100838 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375400a640 con 0x7f375c0b2c30 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f376291a640 1 -- 172.21.15.203:0/2680100838 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c14df00 con 0x7f375c0b2c30 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3758ff9640 1 -- 172.21.15.203:0/2680100838 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375400aab0 con 0x7f375c0b2c30 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f376291a640 1 -- 172.21.15.203:0/2680100838 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c1530d0 con 0x7f375c0b2c30 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3758ff9640 1 -- 172.21.15.203:0/2680100838 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f375401b070 con 0x7f375c0b2c30 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f3758ff9640 1 -- 172.21.15.203:0/2680100838 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f37540078a0 con 0x7f375c0b2c30 2024-04-25T08:52:07.657 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.659+0000 7f376291a640 1 -- 172.21.15.203:0/2680100838 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c1532d0 con 0x7f375c0b2c30 2024-04-25T08:52:07.658 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.663+0000 7f3758ff9640 1 -- 172.21.15.203:0/2680100838 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f375c1532d0 con 0x7f375c0b2c30 2024-04-25T08:52:07.714 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:07.721+0000 7fcee9afa180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:52:07.720 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.727+0000 7f376291a640 1 -- 172.21.15.203:0/2680100838 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "38a1085b-a817-4560-8f4e-297ab640dd94", "id": 3} v 0) v1 -- 0x7f375c04efa0 con 0x7f375c0b2c30 2024-04-25T08:52:07.727 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.735+0000 7f3758ff9640 1 -- 172.21.15.203:0/2680100838 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "38a1085b-a817-4560-8f4e-297ab640dd94", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f375400ae20 con 0x7f375c0b2c30 2024-04-25T08:52:07.727 INFO:teuthology.orchestra.run.smithi203.stdout:3 2024-04-25T08:52:07.728 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.735+0000 7f376291a640 1 -- 172.21.15.203:0/2680100838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0b2c30 msgr2=0x7f375c138010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.728 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.735+0000 7f376291a640 1 --2- 172.21.15.203:0/2680100838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0b2c30 0x7f375c138010 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f375400c7a0 tx=0x7f375400cc60 comp rx=0 tx=0).stop 2024-04-25T08:52:07.728 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.735+0000 7f376291a640 1 -- 172.21.15.203:0/2680100838 shutdown_connections 2024-04-25T08:52:07.728 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.735+0000 7f376291a640 1 --2- 172.21.15.203:0/2680100838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f375c0b2c30 0x7f375c138010 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.728 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.735+0000 7f376291a640 1 --2- 172.21.15.203:0/2680100838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f375c0af280 0x7f375c1535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.728 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.735+0000 7f376291a640 1 --2- 172.21.15.203:0/2680100838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f375c06c3b0 0x7f375c14a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.728 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.735+0000 7f376291a640 1 -- 172.21.15.203:0/2680100838 >> 172.21.15.203:0/2680100838 conn(0x7f375c071810 msgr2=0x7f375c0afb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:07.729 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.735+0000 7f376291a640 1 -- 172.21.15.203:0/2680100838 shutdown_connections 2024-04-25T08:52:07.729 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.735+0000 7f376291a640 1 -- 172.21.15.203:0/2680100838 wait complete. 2024-04-25T08:52:07.744 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 501d204e-2b71-406d-8505-6fe0dd15cff8 4 2024-04-25T08:52:07.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.871+0000 7f9f30793640 1 Processor -- start 2024-04-25T08:52:07.865 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.871+0000 7f9f30793640 1 -- start start 2024-04-25T08:52:07.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f30793640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9f2815aac0 0x7f9f280ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f30793640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9f280cf0e0 0x7f9f280cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f30793640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f280ce320 con 0x7f9f2815aea0 2024-04-25T08:52:07.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f30793640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280ce0d0 con 0x7f9f280cf0e0 2024-04-25T08:52:07.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f30793640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f280ce600 con 0x7f9f2815aac0 2024-04-25T08:52:07.866 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2e52f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9f2815aac0 0x7f9f280ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2ed30640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9f2815aea0 0x7f9f2815b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56436/0 (socket says 172.21.15.203:56436) 2024-04-25T08:52:07.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2e52f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9f2815aac0 0x7f9f280ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48880/0 (socket says 172.21.15.203:48880) 2024-04-25T08:52:07.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2ed30640 1 -- 172.21.15.203:0/304239838 learned_addr learned my addr 172.21.15.203:0/304239838 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-04-25T08:52:07.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2dd2e640 1 --2- 172.21.15.203:0/304239838 >> v2:172.21.15.37:3300/0 conn(0x7f9f280cf0e0 0x7f9f280cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2e52f640 1 -- 172.21.15.203:0/304239838 >> v2:172.21.15.37:3300/0 conn(0x7f9f280cf0e0 msgr2=0x7f9f280cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:07.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2e52f640 1 --2- 172.21.15.203:0/304239838 >> v2:172.21.15.37:3300/0 conn(0x7f9f280cf0e0 0x7f9f280cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2e52f640 1 -- 172.21.15.203:0/304239838 >> v1:172.21.15.203:6789/0 conn(0x7f9f2815aea0 legacy=0x7f9f2815b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.867 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2e52f640 1 -- 172.21.15.203:0/304239838 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f280ce8e0 con 0x7f9f2815aac0 2024-04-25T08:52:07.867 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f61c5ec3180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:52:07.868 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2e52f640 1 --2- 172.21.15.203:0/304239838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f2815aac0 0x7f9f280ccc40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9f28072f80 tx=0x7f9f2401eb30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9640b22209dd497e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:07.868 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2d52d640 1 -- 172.21.15.203:0/304239838 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400b2d0 con 0x7f9f2815aac0 2024-04-25T08:52:07.869 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2d52d640 1 -- 172.21.15.203:0/304239838 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f240a7810 con 0x7f9f2815aac0 2024-04-25T08:52:07.869 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f2d52d640 1 -- 172.21.15.203:0/304239838 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f240a7be0 con 0x7f9f2815aac0 2024-04-25T08:52:07.869 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f30793640 1 -- 172.21.15.203:0/304239838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f2815aac0 msgr2=0x7f9f280ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f30793640 1 --2- 172.21.15.203:0/304239838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f2815aac0 0x7f9f280ccc40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9f28072f80 tx=0x7f9f2401eb30 comp rx=0 tx=0).stop 2024-04-25T08:52:07.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f30793640 1 -- 172.21.15.203:0/304239838 shutdown_connections 2024-04-25T08:52:07.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f30793640 1 --2- 172.21.15.203:0/304239838 >> v2:172.21.15.37:3300/0 conn(0x7f9f280cf0e0 0x7f9f280cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f30793640 1 --2- 172.21.15.203:0/304239838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f2815aac0 0x7f9f280ccc40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.875+0000 7f9f30793640 1 -- 172.21.15.203:0/304239838 >> 172.21.15.203:0/304239838 conn(0x7f9f28099a70 msgr2=0x7f9f280d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:07.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 -- 172.21.15.203:0/304239838 shutdown_connections 2024-04-25T08:52:07.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 -- 172.21.15.203:0/304239838 wait complete. 2024-04-25T08:52:07.870 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 Processor -- start 2024-04-25T08:52:07.871 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 -- start start 2024-04-25T08:52:07.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f280cf0e0 0x7f9f28150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f2815aea0 0x7f9f281f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f281f6d60 0x7f9f28153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:07.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2806b830 con 0x7f9f281f6d60 2024-04-25T08:52:07.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2806b6b0 con 0x7f9f2815aea0 2024-04-25T08:52:07.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2806b9b0 con 0x7f9f280cf0e0 2024-04-25T08:52:07.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2dd2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f2815aea0 0x7f9f281f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.873 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2e52f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f280cf0e0 0x7f9f28150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2dd2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f2815aea0 0x7f9f281f6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48886/0 (socket says 172.21.15.203:48886) 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2dd2e640 1 -- 172.21.15.203:0/34589133 learned_addr learned my addr 172.21.15.203:0/34589133 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2ed30640 1 --2- 172.21.15.203:0/34589133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f281f6d60 0x7f9f28153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2dd2e640 1 -- 172.21.15.203:0/34589133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f280cf0e0 msgr2=0x7f9f28150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2dd2e640 1 --2- 172.21.15.203:0/34589133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f280cf0e0 0x7f9f28150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2dd2e640 1 -- 172.21.15.203:0/34589133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f281f6d60 msgr2=0x7f9f28153510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2dd2e640 1 --2- 172.21.15.203:0/34589133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f281f6d60 0x7f9f28153510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2dd2e640 1 -- 172.21.15.203:0/34589133 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f28151430 con 0x7f9f2815aea0 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2ed30640 1 --2- 172.21.15.203:0/34589133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f281f6d60 0x7f9f28153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f2dd2e640 1 --2- 172.21.15.203:0/34589133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f2815aea0 0x7f9f281f6820 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800d800 tx=0x7f9f1800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f1effd640 1 -- 172.21.15.203:0/34589133 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800b6c0 con 0x7f9f2815aea0 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f1effd640 1 -- 172.21.15.203:0/34589133 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f18019070 con 0x7f9f2815aea0 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 -- 172.21.15.203:0/34589133 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f28151710 con 0x7f9f2815aea0 2024-04-25T08:52:07.874 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f1effd640 1 -- 172.21.15.203:0/34589133 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1800ba60 con 0x7f9f2815aea0 2024-04-25T08:52:07.875 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 -- 172.21.15.203:0/34589133 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f28153a50 con 0x7f9f2815aea0 2024-04-25T08:52:07.875 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f1effd640 1 -- 172.21.15.203:0/34589133 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9f1800bc00 con 0x7f9f2815aea0 2024-04-25T08:52:07.875 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f1effd640 1 -- 172.21.15.203:0/34589133 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f9f18014a30 con 0x7f9f2815aea0 2024-04-25T08:52:07.875 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.879+0000 7f9f30793640 1 -- 172.21.15.203:0/34589133 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2804efa0 con 0x7f9f2815aea0 2024-04-25T08:52:07.875 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.883+0000 7f9f1effd640 1 -- 172.21.15.203:0/34589133 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9f18014d30 con 0x7f9f2815aea0 2024-04-25T08:52:07.923 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:07.931+0000 7f61c5ec3180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:52:07.939 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.947+0000 7f9f30793640 1 -- 172.21.15.203:0/34589133 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "501d204e-2b71-406d-8505-6fe0dd15cff8", "id": 4} v 0) v1 -- 0x7f9f2809e320 con 0x7f9f2815aea0 2024-04-25T08:52:07.946 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.955+0000 7f9f1effd640 1 -- 172.21.15.203:0/34589133 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "501d204e-2b71-406d-8505-6fe0dd15cff8", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9f180145f0 con 0x7f9f2815aea0 2024-04-25T08:52:07.946 INFO:teuthology.orchestra.run.smithi203.stdout:4 2024-04-25T08:52:07.947 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.955+0000 7f9f30793640 1 -- 172.21.15.203:0/34589133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f2815aea0 msgr2=0x7f9f281f6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:07.947 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.955+0000 7f9f30793640 1 --2- 172.21.15.203:0/34589133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f2815aea0 0x7f9f281f6820 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800d800 tx=0x7f9f1800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.947 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.955+0000 7f9f30793640 1 -- 172.21.15.203:0/34589133 shutdown_connections 2024-04-25T08:52:07.947 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.955+0000 7f9f30793640 1 --2- 172.21.15.203:0/34589133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f281f6d60 0x7f9f28153510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.947 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.955+0000 7f9f30793640 1 --2- 172.21.15.203:0/34589133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9f2815aea0 0x7f9f281f6820 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f9f1800d800 tx=0x7f9f1800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.947 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.955+0000 7f9f30793640 1 --2- 172.21.15.203:0/34589133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9f280cf0e0 0x7f9f28150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:07.948 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.955+0000 7f9f30793640 1 -- 172.21.15.203:0/34589133 >> 172.21.15.203:0/34589133 conn(0x7f9f28099a70 msgr2=0x7f9f280ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:07.948 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.955+0000 7f9f30793640 1 -- 172.21.15.203:0/34589133 shutdown_connections 2024-04-25T08:52:07.948 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:07.955+0000 7f9f30793640 1 -- 172.21.15.203:0/34589133 wait complete. 2024-04-25T08:52:07.962 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 6b27cd03-a272-441c-95eb-af689e2ffc1f 5 2024-04-25T08:52:07.980 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:07.987+0000 7f61c5ec3180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:52:08.062 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:08.067+0000 7f61c5ec3180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:52:08.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.083+0000 7fb644042640 1 Processor -- start 2024-04-25T08:52:08.077 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.083+0000 7fb644042640 1 -- start start 2024-04-25T08:52:08.078 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb644042640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb63c15b920 0x7fb63c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb644042640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb63c099fb0 0x7fb63c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb644042640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63c09a3c0 con 0x7fb63c0b0440 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb644042640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c0cd680 con 0x7fb63c099fb0 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb644042640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63c09a950 con 0x7fb63c15b920 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb641dde640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb63c15b920 0x7fb63c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb6415dd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fb63c0b0440 0x7fb63c0aeb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.203:56438/0 (socket says 172.21.15.203:56438) 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb6415dd640 1 -- 172.21.15.203:0/678929408 learned_addr learned my addr 172.21.15.203:0/678929408 (peer_addr_for_me v1:172.21.15.203:0/0) 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb6425df640 1 --2- 172.21.15.203:0/678929408 >> v2:172.21.15.37:3300/0 conn(0x7fb63c099fb0 0x7fb63c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb641dde640 1 -- 172.21.15.203:0/678929408 >> v2:172.21.15.37:3300/0 conn(0x7fb63c099fb0 msgr2=0x7fb63c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb641dde640 1 --2- 172.21.15.203:0/678929408 >> v2:172.21.15.37:3300/0 conn(0x7fb63c099fb0 0x7fb63c0cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb641dde640 1 -- 172.21.15.203:0/678929408 >> v1:172.21.15.203:6789/0 conn(0x7fb63c0b0440 legacy=0x7fb63c0aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb640ddc640 1 -- 172.21.15.203:0/678929408 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2498771238 0 0) 0x7fb62c001120 con 0x7fb63c0b0440 2024-04-25T08:52:08.079 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb641dde640 1 -- 172.21.15.203:0/678929408 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c09ac30 con 0x7fb63c15b920 2024-04-25T08:52:08.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb641dde640 1 --2- 172.21.15.203:0/678929408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb63c15b920 0x7fb63c099a70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0d1480 tx=0x7fb63801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9167871a0244410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:08.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb640ddc640 1 -- 172.21.15.203:0/678929408 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6380b0070 con 0x7fb63c15b920 2024-04-25T08:52:08.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb640ddc640 1 -- 172.21.15.203:0/678929408 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6380ab600 con 0x7fb63c15b920 2024-04-25T08:52:08.081 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.087+0000 7fb640ddc640 1 -- 172.21.15.203:0/678929408 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6380ab9d0 con 0x7fb63c15b920 2024-04-25T08:52:08.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- 172.21.15.203:0/678929408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb63c15b920 msgr2=0x7fb63c099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 --2- 172.21.15.203:0/678929408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb63c15b920 0x7fb63c099a70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb63c0d1480 tx=0x7fb63801c480 comp rx=0 tx=0).stop 2024-04-25T08:52:08.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- 172.21.15.203:0/678929408 shutdown_connections 2024-04-25T08:52:08.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 --2- 172.21.15.203:0/678929408 >> v2:172.21.15.37:3300/0 conn(0x7fb63c099fb0 0x7fb63c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 --2- 172.21.15.203:0/678929408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb63c15b920 0x7fb63c099a70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- 172.21.15.203:0/678929408 >> 172.21.15.203:0/678929408 conn(0x7fb63c0b3010 msgr2=0x7fb63c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:08.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- 172.21.15.203:0/678929408 shutdown_connections 2024-04-25T08:52:08.082 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- 172.21.15.203:0/678929408 wait complete. 2024-04-25T08:52:08.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 Processor -- start 2024-04-25T08:52:08.083 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- start start 2024-04-25T08:52:08.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb63c099fb0 0x7fb63c165f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb63c0b0440 0x7fb63c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb63c15b920 0x7fb63c156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.085 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0afc20 con 0x7fb63c099fb0 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0afaa0 con 0x7fb63c15b920 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63c0afda0 con 0x7fb63c0b0440 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6425df640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb63c15b920 0x7fb63c156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6415dd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb63c0b0440 0x7fb63c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6425df640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb63c15b920 0x7fb63c156ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48900/0 (socket says 172.21.15.203:48900) 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6415dd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb63c0b0440 0x7fb63c1569b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33954/0 (socket says 172.21.15.203:33954) 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6425df640 1 -- 172.21.15.203:0/3367925505 learned_addr learned my addr 172.21.15.203:0/3367925505 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb641dde640 1 --2- 172.21.15.203:0/3367925505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb63c099fb0 0x7fb63c165f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6425df640 1 -- 172.21.15.203:0/3367925505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb63c0b0440 msgr2=0x7fb63c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6425df640 1 --2- 172.21.15.203:0/3367925505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb63c0b0440 0x7fb63c1569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6425df640 1 -- 172.21.15.203:0/3367925505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb63c099fb0 msgr2=0x7fb63c165f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.086 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6425df640 1 --2- 172.21.15.203:0/3367925505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb63c099fb0 0x7fb63c165f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6425df640 1 -- 172.21.15.203:0/3367925505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63c15ab80 con 0x7fb63c15b920 2024-04-25T08:52:08.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb641dde640 1 --2- 172.21.15.203:0/3367925505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb63c099fb0 0x7fb63c165f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:08.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6425df640 1 --2- 172.21.15.203:0/3367925505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb63c15b920 0x7fb63c156ef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb63400cc30 tx=0x7fb63400a5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:08.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6327fc640 1 -- 172.21.15.203:0/3367925505 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63400ce40 con 0x7fb63c15b920 2024-04-25T08:52:08.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6327fc640 1 -- 172.21.15.203:0/3367925505 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6340039a0 con 0x7fb63c15b920 2024-04-25T08:52:08.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6327fc640 1 -- 172.21.15.203:0/3367925505 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb634003d50 con 0x7fb63c15b920 2024-04-25T08:52:08.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- 172.21.15.203:0/3367925505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63c143310 con 0x7fb63c15b920 2024-04-25T08:52:08.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- 172.21.15.203:0/3367925505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63c140740 con 0x7fb63c15b920 2024-04-25T08:52:08.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6327fc640 1 -- 172.21.15.203:0/3367925505 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb634004300 con 0x7fb63c15b920 2024-04-25T08:52:08.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb6327fc640 1 -- 172.21.15.203:0/3367925505 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fb634018d60 con 0x7fb63c15b920 2024-04-25T08:52:08.087 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.091+0000 7fb644042640 1 -- 172.21.15.203:0/3367925505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb63c04efa0 con 0x7fb63c15b920 2024-04-25T08:52:08.088 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.095+0000 7fb6327fc640 1 -- 172.21.15.203:0/3367925505 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb634014030 con 0x7fb63c15b920 2024-04-25T08:52:08.116 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:08.123+0000 7f61c5ec3180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:52:08.151 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.159+0000 7fb644042640 1 -- 172.21.15.203:0/3367925505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6b27cd03-a272-441c-95eb-af689e2ffc1f", "id": 5} v 0) v1 -- 0x7fb63c0b3770 con 0x7fb63c15b920 2024-04-25T08:52:08.158 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.167+0000 7fb6327fc640 1 -- 172.21.15.203:0/3367925505 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6b27cd03-a272-441c-95eb-af689e2ffc1f", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb63400f3f0 con 0x7fb63c15b920 2024-04-25T08:52:08.159 INFO:teuthology.orchestra.run.smithi203.stdout:5 2024-04-25T08:52:08.160 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.167+0000 7fb644042640 1 -- 172.21.15.203:0/3367925505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb63c15b920 msgr2=0x7fb63c156ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.160 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.167+0000 7fb644042640 1 --2- 172.21.15.203:0/3367925505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb63c15b920 0x7fb63c156ef0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb63400cc30 tx=0x7fb63400a5c0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.160 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.167+0000 7fb644042640 1 -- 172.21.15.203:0/3367925505 shutdown_connections 2024-04-25T08:52:08.160 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.167+0000 7fb644042640 1 --2- 172.21.15.203:0/3367925505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb63c15b920 0x7fb63c156ef0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.160 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.167+0000 7fb644042640 1 --2- 172.21.15.203:0/3367925505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb63c0b0440 0x7fb63c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.160 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.167+0000 7fb644042640 1 --2- 172.21.15.203:0/3367925505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb63c099fb0 0x7fb63c165f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.160 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.167+0000 7fb644042640 1 -- 172.21.15.203:0/3367925505 >> 172.21.15.203:0/3367925505 conn(0x7fb63c0b3010 msgr2=0x7fb63c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:08.161 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.167+0000 7fb644042640 1 -- 172.21.15.203:0/3367925505 shutdown_connections 2024-04-25T08:52:08.161 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.167+0000 7fb644042640 1 -- 172.21.15.203:0/3367925505 wait complete. 2024-04-25T08:52:08.175 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new 2eee4199-0713-4ff2-bd6d-d8b2f3e10151 6 2024-04-25T08:52:08.197 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:08.203+0000 7f61c5ec3180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:52:08.253 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:08.259+0000 7f61c5ec3180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:52:08.319 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.323+0000 7f71c0a4f640 1 Processor -- start 2024-04-25T08:52:08.319 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.323+0000 7f71c0a4f640 1 -- start start 2024-04-25T08:52:08.319 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71c0a4f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f71bc06bca0 0x7f71bc0cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.319 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71c0a4f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc0af370 0x7f71bc0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.319 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71c0a4f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71bc06ec30 con 0x7f71bc066e30 2024-04-25T08:52:08.319 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71c0a4f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71bc0669a0 con 0x7f71bc06bca0 2024-04-25T08:52:08.319 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71c0a4f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0cd5f0 con 0x7f71bc0af370 2024-04-25T08:52:08.319 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71bad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc0af370 0x7f71bc0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.319 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71bad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc0af370 0x7f71bc0ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33968/0 (socket says 172.21.15.203:33968) 2024-04-25T08:52:08.320 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71bad9d640 1 -- 172.21.15.203:0/224942560 learned_addr learned my addr 172.21.15.203:0/224942560 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T08:52:08.320 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71ba59c640 1 --2- 172.21.15.203:0/224942560 >> v2:172.21.15.37:3300/0 conn(0x7f71bc06bca0 0x7f71bc0cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.320 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71bad9d640 1 -- 172.21.15.203:0/224942560 >> v2:172.21.15.37:3300/0 conn(0x7f71bc06bca0 msgr2=0x7f71bc0cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:08.320 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71bad9d640 1 --2- 172.21.15.203:0/224942560 >> v2:172.21.15.37:3300/0 conn(0x7f71bc06bca0 0x7f71bc0cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.320 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71bad9d640 1 -- 172.21.15.203:0/224942560 >> v1:172.21.15.37:6789/0 conn(0x7f71bc066e30 legacy=0x7f71bc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.320 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71bad9d640 1 -- 172.21.15.203:0/224942560 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc06f230 con 0x7f71bc0af370 2024-04-25T08:52:08.321 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.327+0000 7f71bad9d640 1 --2- 172.21.15.203:0/224942560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc0af370 0x7f71bc0ceaf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009e50 tx=0x7f71b401c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=847115e0a6e487ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:08.321 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71b9d9b640 1 -- 172.21.15.203:0/224942560 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b40a5730 con 0x7f71bc0af370 2024-04-25T08:52:08.321 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71b9d9b640 1 -- 172.21.15.203:0/224942560 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b40a58d0 con 0x7f71bc0af370 2024-04-25T08:52:08.321 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71b9d9b640 1 -- 172.21.15.203:0/224942560 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b40a5ca0 con 0x7f71bc0af370 2024-04-25T08:52:08.322 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 -- 172.21.15.203:0/224942560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc0af370 msgr2=0x7f71bc0ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.322 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 --2- 172.21.15.203:0/224942560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc0af370 0x7f71bc0ceaf0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009e50 tx=0x7f71b401c8e0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.323 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 -- 172.21.15.203:0/224942560 shutdown_connections 2024-04-25T08:52:08.323 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 --2- 172.21.15.203:0/224942560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc0af370 0x7f71bc0ceaf0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.323 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 --2- 172.21.15.203:0/224942560 >> v2:172.21.15.37:3300/0 conn(0x7f71bc06bca0 0x7f71bc0cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.323 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 -- 172.21.15.203:0/224942560 >> 172.21.15.203:0/224942560 conn(0x7f71bc071810 msgr2=0x7f71bc071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:08.323 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 -- 172.21.15.203:0/224942560 shutdown_connections 2024-04-25T08:52:08.323 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 -- 172.21.15.203:0/224942560 wait complete. 2024-04-25T08:52:08.323 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 Processor -- start 2024-04-25T08:52:08.324 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 -- start start 2024-04-25T08:52:08.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc066e30 0x7f71bc158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc06bca0 0x7f71bc147c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc0af370 0x7f71bc148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc1687b0 con 0x7f71bc06bca0 2024-04-25T08:52:08.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc168630 con 0x7f71bc0af370 2024-04-25T08:52:08.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc168930 con 0x7f71bc066e30 2024-04-25T08:52:08.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71ba59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc066e30 0x7f71bc158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71bad9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc0af370 0x7f71bc148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71ba59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc066e30 0x7f71bc158f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.203:33980/0 (socket says 172.21.15.203:33980) 2024-04-25T08:52:08.326 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71ba59c640 1 -- 172.21.15.203:0/27842018 learned_addr learned my addr 172.21.15.203:0/27842018 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71b3fff640 1 --2- 172.21.15.203:0/27842018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc06bca0 0x7f71bc147c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71ba59c640 1 -- 172.21.15.203:0/27842018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc0af370 msgr2=0x7f71bc148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71ba59c640 1 --2- 172.21.15.203:0/27842018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc0af370 0x7f71bc148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71ba59c640 1 -- 172.21.15.203:0/27842018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc06bca0 msgr2=0x7f71bc147c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71ba59c640 1 --2- 172.21.15.203:0/27842018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc06bca0 0x7f71bc147c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71ba59c640 1 -- 172.21.15.203:0/27842018 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc14e9a0 con 0x7f71bc066e30 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71bad9d640 1 --2- 172.21.15.203:0/27842018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc0af370 0x7f71bc148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71ba59c640 1 --2- 172.21.15.203:0/27842018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc066e30 0x7f71bc158f00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00d5f0 tx=0x7f71ac00dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71b2ffd640 1 -- 172.21.15.203:0/27842018 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0125f0 con 0x7f71bc066e30 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71b2ffd640 1 -- 172.21.15.203:0/27842018 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac019070 con 0x7f71bc066e30 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 -- 172.21.15.203:0/27842018 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc14ec80 con 0x7f71bc066e30 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71b2ffd640 1 -- 172.21.15.203:0/27842018 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac0129b0 con 0x7f71bc066e30 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.331+0000 7f71c0a4f640 1 -- 172.21.15.203:0/27842018 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc14f240 con 0x7f71bc066e30 2024-04-25T08:52:08.327 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.335+0000 7f71b2ffd640 1 -- 172.21.15.203:0/27842018 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f71ac012b50 con 0x7f71bc066e30 2024-04-25T08:52:08.328 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.335+0000 7f71b2ffd640 1 -- 172.21.15.203:0/27842018 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f71ac014a20 con 0x7f71bc066e30 2024-04-25T08:52:08.328 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.335+0000 7f71c0a4f640 1 -- 172.21.15.203:0/27842018 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71bc04efa0 con 0x7f71bc066e30 2024-04-25T08:52:08.328 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.335+0000 7f71b2ffd640 1 -- 172.21.15.203:0/27842018 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f71ac014cc0 con 0x7f71bc066e30 2024-04-25T08:52:08.342 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:08.349+0000 7fcee9afa180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:52:08.394 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.403+0000 7f71c0a4f640 1 -- 172.21.15.203:0/27842018 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2eee4199-0713-4ff2-bd6d-d8b2f3e10151", "id": 6} v 0) v1 -- 0x7f71bc072400 con 0x7f71bc066e30 2024-04-25T08:52:08.398 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.407+0000 7f71b2ffd640 1 -- 172.21.15.203:0/27842018 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2eee4199-0713-4ff2-bd6d-d8b2f3e10151", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f71ac014430 con 0x7f71bc066e30 2024-04-25T08:52:08.398 INFO:teuthology.orchestra.run.smithi203.stdout:6 2024-04-25T08:52:08.399 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:08.405+0000 7fcee9afa180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:52:08.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.407+0000 7f71c0a4f640 1 -- 172.21.15.203:0/27842018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc066e30 msgr2=0x7f71bc158f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.407+0000 7f71c0a4f640 1 --2- 172.21.15.203:0/27842018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc066e30 0x7f71bc158f00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00d5f0 tx=0x7f71ac00dab0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.407+0000 7f71c0a4f640 1 -- 172.21.15.203:0/27842018 shutdown_connections 2024-04-25T08:52:08.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.407+0000 7f71c0a4f640 1 --2- 172.21.15.203:0/27842018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71bc0af370 0x7f71bc148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.407+0000 7f71c0a4f640 1 --2- 172.21.15.203:0/27842018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71bc06bca0 0x7f71bc147c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.407+0000 7f71c0a4f640 1 --2- 172.21.15.203:0/27842018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71bc066e30 0x7f71bc158f00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.407+0000 7f71c0a4f640 1 -- 172.21.15.203:0/27842018 >> 172.21.15.203:0/27842018 conn(0x7f71bc071810 msgr2=0x7f71bc155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:08.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.407+0000 7f71c0a4f640 1 -- 172.21.15.203:0/27842018 shutdown_connections 2024-04-25T08:52:08.400 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.407+0000 7f71c0a4f640 1 -- 172.21.15.203:0/27842018 wait complete. 2024-04-25T08:52:08.410 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:08.419+0000 7f61c5ec3180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:52:08.414 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --cluster ceph osd new bb76d7a8-1a44-4aec-a592-8112bfeab2c8 7 2024-04-25T08:52:08.454 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:08.461+0000 7fcee9afa180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:52:08.476 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:08.483+0000 7f61c5ec3180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:52:08.519 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da7746640 1 Processor -- start 2024-04-25T08:52:08.519 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da7746640 1 -- start start 2024-04-25T08:52:08.520 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da7746640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4da009a180 0x7f4da00cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.521 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da7746640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da009a950 con 0x7f4da015b920 2024-04-25T08:52:08.521 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da7746640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da009ac30 con 0x7f4da00afb90 2024-04-25T08:52:08.521 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da7746640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da00ae380 con 0x7f4da009a180 2024-04-25T08:52:08.521 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da4ce1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4da009a180 0x7f4da00cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.521 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da4ce1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4da009a180 0x7f4da00cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48918/0 (socket says 172.21.15.203:48918) 2024-04-25T08:52:08.521 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da4ce1640 1 -- 172.21.15.203:0/4093735573 learned_addr learned my addr 172.21.15.203:0/4093735573 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T08:52:08.521 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da4ce1640 1 -- 172.21.15.203:0/4093735573 >> v1:172.21.15.203:6789/0 conn(0x7f4da00afb90 legacy=0x7f4da00cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.521 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da4ce1640 1 -- 172.21.15.203:0/4093735573 >> v1:172.21.15.37:6789/0 conn(0x7f4da015b920 legacy=0x7f4da0099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:08.521 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.527+0000 7f4da4ce1640 1 -- 172.21.15.203:0/4093735573 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da00ae660 con 0x7f4da009a180 2024-04-25T08:52:08.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4da4ce1640 1 --2- 172.21.15.203:0/4093735573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da009a180 0x7f4da00cc130 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009870 tx=0x7f4d9001c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f232b6b1f629b006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:08.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4d97fff640 1 -- 172.21.15.203:0/4093735573 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d900ab430 con 0x7f4da009a180 2024-04-25T08:52:08.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4d97fff640 1 -- 172.21.15.203:0/4093735573 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d900b0070 con 0x7f4da009a180 2024-04-25T08:52:08.523 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4d97fff640 1 -- 172.21.15.203:0/4093735573 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d900ab7f0 con 0x7f4da009a180 2024-04-25T08:52:08.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4da7746640 1 -- 172.21.15.203:0/4093735573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da009a180 msgr2=0x7f4da00cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4da7746640 1 --2- 172.21.15.203:0/4093735573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da009a180 0x7f4da00cc130 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4d90009870 tx=0x7f4d9001c520 comp rx=0 tx=0).stop 2024-04-25T08:52:08.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4da7746640 1 -- 172.21.15.203:0/4093735573 shutdown_connections 2024-04-25T08:52:08.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4da7746640 1 --2- 172.21.15.203:0/4093735573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da009a180 0x7f4da00cc130 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4da7746640 1 -- 172.21.15.203:0/4093735573 >> 172.21.15.203:0/4093735573 conn(0x7f4da00b3010 msgr2=0x7f4da00b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:08.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4da7746640 1 -- 172.21.15.203:0/4093735573 shutdown_connections 2024-04-25T08:52:08.524 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4da7746640 1 -- 172.21.15.203:0/4093735573 wait complete. 2024-04-25T08:52:08.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4da7746640 1 Processor -- start 2024-04-25T08:52:08.525 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.531+0000 7f4da7746640 1 -- start start 2024-04-25T08:52:08.527 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da7746640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da009a180 0x7f4da0166710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da7746640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4da00afb90 0x7f4da01535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da7746640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4da015b920 0x7f4da015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da7746640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da0169860 con 0x7f4da00afb90 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da7746640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da01696e0 con 0x7f4da009a180 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da7746640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da01699e0 con 0x7f4da015b920 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da54e2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da009a180 0x7f4da0166710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da54e2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da009a180 0x7f4da0166710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.203:48922/0 (socket says 172.21.15.203:48922) 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da54e2640 1 -- 172.21.15.203:0/754020171 learned_addr learned my addr 172.21.15.203:0/754020171 (peer_addr_for_me v2:172.21.15.203:0/0) 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da5ce3640 1 --2- 172.21.15.203:0/754020171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4da015b920 0x7f4da015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da4ce1640 1 --2- 172.21.15.203:0/754020171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4da00afb90 0x7f4da01535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da54e2640 1 -- 172.21.15.203:0/754020171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4da015b920 msgr2=0x7f4da015a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da54e2640 1 --2- 172.21.15.203:0/754020171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4da015b920 0x7f4da015a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da54e2640 1 -- 172.21.15.203:0/754020171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4da00afb90 msgr2=0x7f4da01535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da54e2640 1 --2- 172.21.15.203:0/754020171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4da00afb90 0x7f4da01535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.528 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da54e2640 1 -- 172.21.15.203:0/754020171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da0157ae0 con 0x7f4da009a180 2024-04-25T08:52:08.529 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da54e2640 1 --2- 172.21.15.203:0/754020171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da009a180 0x7f4da0166710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00c450 tx=0x7f4d9c00c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:08.529 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4d95ffb640 1 -- 172.21.15.203:0/754020171 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c012830 con 0x7f4da009a180 2024-04-25T08:52:08.529 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4d95ffb640 1 -- 172.21.15.203:0/754020171 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9c017070 con 0x7f4da009a180 2024-04-25T08:52:08.529 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da7746640 1 -- 172.21.15.203:0/754020171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da0157d80 con 0x7f4da009a180 2024-04-25T08:52:08.529 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4d95ffb640 1 -- 172.21.15.203:0/754020171 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c012c30 con 0x7f4da009a180 2024-04-25T08:52:08.529 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da7746640 1 -- 172.21.15.203:0/754020171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da0158340 con 0x7f4da009a180 2024-04-25T08:52:08.529 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4d95ffb640 1 -- 172.21.15.203:0/754020171 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4d9c01c460 con 0x7f4da009a180 2024-04-25T08:52:08.529 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4d95ffb640 1 -- 172.21.15.203:0/754020171 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f4d9c01bd90 con 0x7f4da009a180 2024-04-25T08:52:08.529 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.535+0000 7f4da7746640 1 -- 172.21.15.203:0/754020171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da004efa0 con 0x7f4da009a180 2024-04-25T08:52:08.531 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.539+0000 7f4d95ffb640 1 -- 172.21.15.203:0/754020171 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f4d9c012dd0 con 0x7f4da009a180 2024-04-25T08:52:08.535 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:08.541+0000 7fcee9afa180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:52:08.590 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:08.597+0000 7fcee9afa180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:52:08.594 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.603+0000 7f4da7746640 1 -- 172.21.15.203:0/754020171 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd new", "uuid": "bb76d7a8-1a44-4aec-a592-8112bfeab2c8", "id": 7} v 0) v1 -- 0x7f4da015b150 con 0x7f4da009a180 2024-04-25T08:52:08.602 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.611+0000 7f4d95ffb640 1 -- 172.21.15.203:0/754020171 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bb76d7a8-1a44-4aec-a592-8112bfeab2c8", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4d9c01b730 con 0x7f4da009a180 2024-04-25T08:52:08.603 INFO:teuthology.orchestra.run.smithi203.stdout:7 2024-04-25T08:52:08.604 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.611+0000 7f4da7746640 1 -- 172.21.15.203:0/754020171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da009a180 msgr2=0x7f4da0166710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.604 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.611+0000 7f4da7746640 1 --2- 172.21.15.203:0/754020171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da009a180 0x7f4da0166710 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c00c450 tx=0x7f4d9c00c910 comp rx=0 tx=0).stop 2024-04-25T08:52:08.604 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.611+0000 7f4da7746640 1 -- 172.21.15.203:0/754020171 shutdown_connections 2024-04-25T08:52:08.604 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.611+0000 7f4da7746640 1 --2- 172.21.15.203:0/754020171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4da015b920 0x7f4da015a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.604 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.611+0000 7f4da7746640 1 --2- 172.21.15.203:0/754020171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4da00afb90 0x7f4da01535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.604 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.611+0000 7f4da7746640 1 --2- 172.21.15.203:0/754020171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4da009a180 0x7f4da0166710 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.604 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.611+0000 7f4da7746640 1 -- 172.21.15.203:0/754020171 >> 172.21.15.203:0/754020171 conn(0x7f4da00b3010 msgr2=0x7f4da0156fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:08.604 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.611+0000 7f4da7746640 1 -- 172.21.15.203:0/754020171 shutdown_connections 2024-04-25T08:52:08.605 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:52:08.611+0000 7f4da7746640 1 -- 172.21.15.203:0/754020171 wait complete. 2024-04-25T08:52:08.619 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T08:52:08.619 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T08:52:08.621 INFO:tasks.ceph.osd.0:Started 2024-04-25T08:52:08.622 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T08:52:08.622 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T08:52:08.624 INFO:tasks.ceph.osd.1:Started 2024-04-25T08:52:08.624 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T08:52:08.624 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T08:52:08.626 INFO:tasks.ceph.osd.2:Started 2024-04-25T08:52:08.627 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T08:52:08.627 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T08:52:08.629 INFO:tasks.ceph.osd.3:Started 2024-04-25T08:52:08.629 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T08:52:08.629 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T08:52:08.631 INFO:tasks.ceph.osd.4:Started 2024-04-25T08:52:08.633 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T08:52:08.633 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T08:52:08.635 INFO:tasks.ceph.osd.5:Started 2024-04-25T08:52:08.635 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T08:52:08.635 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T08:52:08.637 INFO:tasks.ceph.osd.6:Started 2024-04-25T08:52:08.637 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T08:52:08.637 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T08:52:08.641 INFO:tasks.ceph.osd.7:Started 2024-04-25T08:52:08.641 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:08.700 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:08.705+0000 7fcee9afa180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:52:08.758 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:08.765+0000 7fcee9afa180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:52:08.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.781+0000 7f9424e2c640 1 Processor -- start 2024-04-25T08:52:08.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.781+0000 7f9424e2c640 1 -- start start 2024-04-25T08:52:08.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f9424e2c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f94200af280 0x7f942015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f9424e2c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94200b2c30 0x7f942006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f9424e2c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9420066c30 con 0x7f942006c1e0 2024-04-25T08:52:08.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f9424e2c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942006daa0 con 0x7f94200af280 2024-04-25T08:52:08.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f9424e2c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94200af660 con 0x7f94200b2c30 2024-04-25T08:52:08.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f941ed9d640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f942006c1e0 0x7f9420161500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:58462/0 (socket says 172.21.15.37:58462) 2024-04-25T08:52:08.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f941ed9d640 1 -- 172.21.15.37:0/2438222083 learned_addr learned my addr 172.21.15.37:0/2438222083 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:08.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f9417fff640 1 --2- 172.21.15.37:0/2438222083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94200b2c30 0x7f942006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f941e59c640 1 --2- 172.21.15.37:0/2438222083 >> v2:172.21.15.203:3300/0 conn(0x7f94200af280 0x7f942015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f9417fff640 1 -- 172.21.15.37:0/2438222083 >> v2:172.21.15.203:3300/0 conn(0x7f94200af280 msgr2=0x7f942015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:08.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f9417fff640 1 --2- 172.21.15.37:0/2438222083 >> v2:172.21.15.203:3300/0 conn(0x7f94200af280 0x7f942015df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f9417fff640 1 -- 172.21.15.37:0/2438222083 >> v1:172.21.15.203:6789/0 conn(0x7f942006c1e0 legacy=0x7f9420161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f9417fff640 1 -- 172.21.15.37:0/2438222083 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942015e640 con 0x7f94200b2c30 2024-04-25T08:52:08.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f941e59c640 1 --2- 172.21.15.37:0/2438222083 >> v2:172.21.15.203:3300/0 conn(0x7f94200af280 0x7f942015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:08.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f9417fff640 1 --2- 172.21.15.37:0/2438222083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94200b2c30 0x7f942006bca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f94080097c0 tx=0x7f940801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f0b4a41a0570b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:08.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f941dd9b640 1 -- 172.21.15.37:0/2438222083 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94080b0070 con 0x7f94200b2c30 2024-04-25T08:52:08.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f941dd9b640 1 -- 172.21.15.37:0/2438222083 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94080ab600 con 0x7f94200b2c30 2024-04-25T08:52:08.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.785+0000 7f941dd9b640 1 -- 172.21.15.37:0/2438222083 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94080ab9d0 con 0x7f94200b2c30 2024-04-25T08:52:08.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 -- 172.21.15.37:0/2438222083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94200b2c30 msgr2=0x7f942006bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 --2- 172.21.15.37:0/2438222083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94200b2c30 0x7f942006bca0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f94080097c0 tx=0x7f940801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 -- 172.21.15.37:0/2438222083 shutdown_connections 2024-04-25T08:52:08.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 --2- 172.21.15.37:0/2438222083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94200b2c30 0x7f942006bca0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 --2- 172.21.15.37:0/2438222083 >> v2:172.21.15.203:3300/0 conn(0x7f94200af280 0x7f942015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 -- 172.21.15.37:0/2438222083 >> 172.21.15.37:0/2438222083 conn(0x7f9420071810 msgr2=0x7f9420071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:08.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 -- 172.21.15.37:0/2438222083 shutdown_connections 2024-04-25T08:52:08.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 -- 172.21.15.37:0/2438222083 wait complete. 2024-04-25T08:52:08.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 Processor -- start 2024-04-25T08:52:08.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 -- start start 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f942006c1e0 0x7f94201391f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94200af280 0x7f942013bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94200b2c30 0x7f942013a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942016aab0 con 0x7f94200af280 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942006c5e0 con 0x7f942006c1e0 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942016ac30 con 0x7f94200b2c30 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9417fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94200af280 0x7f942013bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9417fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94200af280 0x7f942013bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44524/0 (socket says 172.21.15.37:44524) 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9417fff640 1 -- 172.21.15.37:0/344996144 learned_addr learned my addr 172.21.15.37:0/344996144 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f941e59c640 1 --2- 172.21.15.37:0/344996144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f942006c1e0 0x7f94201391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f941ed9d640 1 --2- 172.21.15.37:0/344996144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94200b2c30 0x7f942013a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:08.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9417fff640 1 -- 172.21.15.37:0/344996144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94200b2c30 msgr2=0x7f942013a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9417fff640 1 --2- 172.21.15.37:0/344996144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94200b2c30 0x7f942013a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9417fff640 1 -- 172.21.15.37:0/344996144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f942006c1e0 msgr2=0x7f94201391f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9417fff640 1 --2- 172.21.15.37:0/344996144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f942006c1e0 0x7f94201391f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9417fff640 1 -- 172.21.15.37:0/344996144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94201398b0 con 0x7f94200af280 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9417fff640 1 --2- 172.21.15.37:0/344996144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94200af280 0x7f942013bcb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94080097c0 tx=0x7f9408019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9416ffd640 1 -- 172.21.15.37:0/344996144 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94080bb070 con 0x7f94200af280 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9416ffd640 1 -- 172.21.15.37:0/344996144 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94080b7460 con 0x7f94200af280 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 -- 172.21.15.37:0/344996144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9420139b90 con 0x7f94200af280 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9416ffd640 1 -- 172.21.15.37:0/344996144 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94080b0040 con 0x7f94200af280 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9424e2c640 1 -- 172.21.15.37:0/344996144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942013a150 con 0x7f94200af280 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.789+0000 7f9416ffd640 1 -- 172.21.15.37:0/344996144 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f94080be070 con 0x7f94200af280 2024-04-25T08:52:08.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.793+0000 7f9416ffd640 1 -- 172.21.15.37:0/344996144 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f94080b5430 con 0x7f94200af280 2024-04-25T08:52:08.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.793+0000 7f9414ff9640 1 -- 172.21.15.37:0/344996144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f80052a0 con 0x7f94200af280 2024-04-25T08:52:08.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.805+0000 7f9416ffd640 1 -- 172.21.15.37:0/344996144 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9408016090 con 0x7f94200af280 2024-04-25T08:52:08.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.861+0000 7f9414ff9640 1 -- 172.21.15.37:0/344996144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93f8005860 con 0x7f94200af280 2024-04-25T08:52:08.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.861+0000 7f9416ffd640 1 -- 172.21.15.37:0/344996144 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f94080c0450 con 0x7f94200af280 2024-04-25T08:52:08.856 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:08.856 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:08.607946+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:08.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.865+0000 7f9414ff9640 1 -- 172.21.15.37:0/344996144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94200af280 msgr2=0x7f942013bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:08.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.865+0000 7f9414ff9640 1 --2- 172.21.15.37:0/344996144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94200af280 0x7f942013bcb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f94080097c0 tx=0x7f9408019240 comp rx=0 tx=0).stop 2024-04-25T08:52:08.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.865+0000 7f9414ff9640 1 -- 172.21.15.37:0/344996144 shutdown_connections 2024-04-25T08:52:08.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.865+0000 7f9414ff9640 1 --2- 172.21.15.37:0/344996144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f94200b2c30 0x7f942013a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.865+0000 7f9414ff9640 1 --2- 172.21.15.37:0/344996144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94200af280 0x7f942013bcb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.865+0000 7f9414ff9640 1 --2- 172.21.15.37:0/344996144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f942006c1e0 0x7f94201391f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:08.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.865+0000 7f9414ff9640 1 -- 172.21.15.37:0/344996144 >> 172.21.15.37:0/344996144 conn(0x7f9420071810 msgr2=0x7f942015bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:08.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.865+0000 7f9414ff9640 1 -- 172.21.15.37:0/344996144 shutdown_connections 2024-04-25T08:52:08.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:08.865+0000 7f9414ff9640 1 -- 172.21.15.37:0/344996144 wait complete. 2024-04-25T08:52:08.872 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T08:52:08.873 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T08:52:08.919 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:08.925+0000 7fcee9afa180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:52:08.984 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:08.993+0000 7fcee9afa180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:52:09.120 INFO:tasks.ceph.mgr.x.smithi203.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T08:52:09.120 INFO:tasks.ceph.mgr.x.smithi203.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T08:52:09.120 INFO:tasks.ceph.mgr.x.smithi203.stderr: from numpy import show_config as show_numpy_config 2024-04-25T08:52:09.185 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:09.191+0000 7f61c5ec3180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:52:09.226 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-25T08:52:09.233+0000 7f50ae7a27c0 -1 Falling back to public interface 2024-04-25T08:52:09.233 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-25T08:52:09.241+0000 7f107970c7c0 -1 Falling back to public interface 2024-04-25T08:52:09.255 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-25T08:52:09.263+0000 7f6f854047c0 -1 Falling back to public interface 2024-04-25T08:52:09.255 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-25T08:52:09.263+0000 7fc51d7ac7c0 -1 Falling back to public interface 2024-04-25T08:52:09.256 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-25T08:52:09.263+0000 7f6f843c4640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-25T08:52:09.256 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-25T08:52:09.263+0000 7f6f843c4640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2024-04-25T08:52:09.256 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-25T08:52:09.263+0000 7ffbc7c047c0 -1 Falling back to public interface 2024-04-25T08:52:09.256 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-25T08:52:09.265+0000 7fd1aee397c0 -1 Falling back to public interface 2024-04-25T08:52:09.261 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-25T08:52:09.269+0000 7fdf727b27c0 -1 Falling back to public interface 2024-04-25T08:52:09.266 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-25T08:52:09.275+0000 7f58231187c0 -1 Falling back to public interface 2024-04-25T08:52:09.353 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:09.359+0000 7f61c5ec3180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:52:09.531 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:09.539+0000 7f61c5ec3180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:52:09.602 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:09.611+0000 7f61c5ec3180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:52:09.677 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:09.692 INFO:tasks.ceph.mgr.y.smithi037.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T08:52:09.692 INFO:tasks.ceph.mgr.y.smithi037.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T08:52:09.692 INFO:tasks.ceph.mgr.y.smithi037.stderr: from numpy import show_config as show_numpy_config 2024-04-25T08:52:09.750 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:09.759+0000 7f61c5ec3180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:52:09.758 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:09.765+0000 7fcee9afa180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:52:09.800 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.805+0000 7f25b6b6c640 1 Processor -- start 2024-04-25T08:52:09.800 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b6b6c640 1 -- start start 2024-04-25T08:52:09.802 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b6b6c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f25b00cd9e0 0x7f25b0099a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:09.802 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b6b6c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f25b00cfd80 0x7f25b00cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:09.802 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b6b6c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b00713c0 con 0x7f25b0099f70 2024-04-25T08:52:09.802 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b6b6c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b009a3d0 con 0x7f25b00cfd80 2024-04-25T08:52:09.802 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b6b6c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b00cddc0 con 0x7f25b00cd9e0 2024-04-25T08:52:09.802 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25affff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f25b0099f70 0x7f25b00cf670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41540/0 (socket says 172.21.15.37:41540) 2024-04-25T08:52:09.802 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25affff640 1 -- 172.21.15.37:0/3291937967 learned_addr learned my addr 172.21.15.37:0/3291937967 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:09.802 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b5109640 1 --2- 172.21.15.37:0/3291937967 >> v2:172.21.15.37:3300/0 conn(0x7f25b00cfd80 0x7f25b00cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:09.802 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b4908640 1 --2- 172.21.15.37:0/3291937967 >> v2:172.21.15.203:3300/0 conn(0x7f25b00cd9e0 0x7f25b0099a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:09.802 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25af7fe640 1 -- 172.21.15.37:0/3291937967 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609387787 0 0) 0x7f25b00713c0 con 0x7f25b0099f70 2024-04-25T08:52:09.802 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25af7fe640 1 -- 172.21.15.37:0/3291937967 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2598003530 con 0x7f25b0099f70 2024-04-25T08:52:09.803 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b5109640 1 -- 172.21.15.37:0/3291937967 >> v2:172.21.15.203:3300/0 conn(0x7f25b00cd9e0 msgr2=0x7f25b0099a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:09.803 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b5109640 1 --2- 172.21.15.37:0/3291937967 >> v2:172.21.15.203:3300/0 conn(0x7f25b00cd9e0 0x7f25b0099a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:09.803 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b5109640 1 -- 172.21.15.37:0/3291937967 >> v1:172.21.15.37:6789/0 conn(0x7f25b0099f70 legacy=0x7f25b00cf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:09.803 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b5109640 1 -- 172.21.15.37:0/3291937967 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b006fe80 con 0x7f25b00cfd80 2024-04-25T08:52:09.804 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.809+0000 7f25b5109640 1 --2- 172.21.15.37:0/3291937967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b00cfd80 0x7f25b00cbdd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f25b0071590 tx=0x7f25a801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb68b81da28014a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:09.804 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25af7fe640 1 -- 172.21.15.37:0/3291937967 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80b2070 con 0x7f25b00cfd80 2024-04-25T08:52:09.804 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25af7fe640 1 -- 172.21.15.37:0/3291937967 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a80a58e0 con 0x7f25b00cfd80 2024-04-25T08:52:09.804 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25af7fe640 1 -- 172.21.15.37:0/3291937967 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80a5cb0 con 0x7f25b00cfd80 2024-04-25T08:52:09.805 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 -- 172.21.15.37:0/3291937967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b00cfd80 msgr2=0x7f25b00cbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:09.805 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 --2- 172.21.15.37:0/3291937967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b00cfd80 0x7f25b00cbdd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f25b0071590 tx=0x7f25a801c930 comp rx=0 tx=0).stop 2024-04-25T08:52:09.805 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 -- 172.21.15.37:0/3291937967 shutdown_connections 2024-04-25T08:52:09.805 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 --2- 172.21.15.37:0/3291937967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b00cfd80 0x7f25b00cbdd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:09.805 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 --2- 172.21.15.37:0/3291937967 >> v2:172.21.15.203:3300/0 conn(0x7f25b00cd9e0 0x7f25b0099a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:09.805 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 -- 172.21.15.37:0/3291937967 >> 172.21.15.37:0/3291937967 conn(0x7f25b00b3060 msgr2=0x7f25b00b3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:09.805 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 -- 172.21.15.37:0/3291937967 shutdown_connections 2024-04-25T08:52:09.805 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 -- 172.21.15.37:0/3291937967 wait complete. 2024-04-25T08:52:09.806 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 Processor -- start 2024-04-25T08:52:09.806 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 -- start start 2024-04-25T08:52:09.808 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25b0099f70 0x7f25b009f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:09.808 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25b00cd9e0 0x7f25b00c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:09.808 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b00cfd80 0x7f25b00b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:09.808 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00ccb20 con 0x7f25b00cfd80 2024-04-25T08:52:09.808 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00d02c0 con 0x7f25b0099f70 2024-04-25T08:52:09.808 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00ccca0 con 0x7f25b00cd9e0 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b5109640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b00cfd80 0x7f25b00b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b5109640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b00cfd80 0x7f25b00b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53000/0 (socket says 172.21.15.37:53000) 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b5109640 1 -- 172.21.15.37:0/2386740184 learned_addr learned my addr 172.21.15.37:0/2386740184 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b4908640 1 --2- 172.21.15.37:0/2386740184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25b0099f70 0x7f25b009f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b5109640 1 -- 172.21.15.37:0/2386740184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25b00cd9e0 msgr2=0x7f25b00c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b5109640 1 --2- 172.21.15.37:0/2386740184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25b00cd9e0 0x7f25b00c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b5109640 1 -- 172.21.15.37:0/2386740184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25b0099f70 msgr2=0x7f25b009f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b5109640 1 --2- 172.21.15.37:0/2386740184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25b0099f70 0x7f25b009f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b5109640 1 -- 172.21.15.37:0/2386740184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b00c6ae0 con 0x7f25b00cfd80 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b4908640 1 --2- 172.21.15.37:0/2386740184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25b0099f70 0x7f25b009f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b5109640 1 --2- 172.21.15.37:0/2386740184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b00cfd80 0x7f25b00b2240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f25a8019540 tx=0x7f25a80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25ad7fa640 1 -- 172.21.15.37:0/2386740184 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80b2050 con 0x7f25b00cfd80 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25ad7fa640 1 -- 172.21.15.37:0/2386740184 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a80be070 con 0x7f25b00cfd80 2024-04-25T08:52:09.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 -- 172.21.15.37:0/2386740184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b015b1d0 con 0x7f25b00cfd80 2024-04-25T08:52:09.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25ad7fa640 1 -- 172.21.15.37:0/2386740184 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80ad890 con 0x7f25b00cfd80 2024-04-25T08:52:09.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25b6b6c640 1 -- 172.21.15.37:0/2386740184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b00c7030 con 0x7f25b00cfd80 2024-04-25T08:52:09.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.813+0000 7f25ad7fa640 1 -- 172.21.15.37:0/2386740184 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f25a80ada30 con 0x7f25b00cfd80 2024-04-25T08:52:09.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.817+0000 7f25ad7fa640 1 -- 172.21.15.37:0/2386740184 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f25a80add20 con 0x7f25b00cfd80 2024-04-25T08:52:09.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.817+0000 7f25b6b6c640 1 -- 172.21.15.37:0/2386740184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b0074d80 con 0x7f25b00cfd80 2024-04-25T08:52:09.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.817+0000 7f25ad7fa640 1 -- 172.21.15.37:0/2386740184 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f25a8016520 con 0x7f25b00cfd80 2024-04-25T08:52:09.868 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.877+0000 7f25b6b6c640 1 -- 172.21.15.37:0/2386740184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25b00cf240 con 0x7f25b00cfd80 2024-04-25T08:52:09.869 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.877+0000 7f25ad7fa640 1 -- 172.21.15.37:0/2386740184 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f25a80c3460 con 0x7f25b00cfd80 2024-04-25T08:52:09.869 INFO:teuthology.misc.health.smithi037.stdout: 2024-04-25T08:52:09.869 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":10,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:08.607946+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:09.870 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.877+0000 7f25b6b6c640 1 -- 172.21.15.37:0/2386740184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b00cfd80 msgr2=0x7f25b00b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:09.870 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.877+0000 7f25b6b6c640 1 --2- 172.21.15.37:0/2386740184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b00cfd80 0x7f25b00b2240 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f25a8019540 tx=0x7f25a80196d0 comp rx=0 tx=0).stop 2024-04-25T08:52:09.870 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.877+0000 7f25b6b6c640 1 -- 172.21.15.37:0/2386740184 shutdown_connections 2024-04-25T08:52:09.870 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.877+0000 7f25b6b6c640 1 --2- 172.21.15.37:0/2386740184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25b00cfd80 0x7f25b00b2240 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:09.871 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.877+0000 7f25b6b6c640 1 --2- 172.21.15.37:0/2386740184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f25b00cd9e0 0x7f25b00c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:09.871 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.877+0000 7f25b6b6c640 1 --2- 172.21.15.37:0/2386740184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f25b0099f70 0x7f25b009f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:09.871 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.877+0000 7f25b6b6c640 1 -- 172.21.15.37:0/2386740184 >> 172.21.15.37:0/2386740184 conn(0x7f25b00b3060 msgr2=0x7f25b009a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:09.871 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.877+0000 7f25b6b6c640 1 -- 172.21.15.37:0/2386740184 shutdown_connections 2024-04-25T08:52:09.871 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:09.877+0000 7f25b6b6c640 1 -- 172.21.15.37:0/2386740184 wait complete. 2024-04-25T08:52:09.885 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T08:52:09.902 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:09.909+0000 7fcee9afa180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:52:10.075 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:10.081+0000 7fcee9afa180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:52:10.141 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:10.149+0000 7fcee9afa180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:52:10.282 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:10.289+0000 7fcee9afa180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:52:10.385 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:10.391+0000 7f61c5ec3180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:52:10.462 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:10.471+0000 7f61c5ec3180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:52:10.643 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:10.651+0000 7f61c5ec3180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:52:10.717 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:10.723+0000 7f61c5ec3180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:52:10.926 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:10.933+0000 7fcee9afa180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:52:11.005 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:11.013+0000 7fcee9afa180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:52:11.009 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:11.015+0000 7f61c5ec3180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:52:11.071 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:11.079+0000 7f61c5ec3180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:52:11.196 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:11.205+0000 7fcee9afa180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:52:11.274 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:11.281+0000 7fcee9afa180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:52:11.581 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:11.589+0000 7fcee9afa180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:52:11.646 INFO:tasks.ceph.mgr.y.smithi037.stderr:2024-04-25T08:52:11.653+0000 7fcee9afa180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:52:12.108 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:12.115+0000 7f616d411640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T08:52:12.108 INFO:tasks.ceph.mgr.x.smithi203.stderr:2024-04-25T08:52:12.115+0000 7f616d411640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T08:52:12.899 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-25T08:52:12.905+0000 7f50ae7a27c0 -1 osd.3 0 log_to_monitors true 2024-04-25T08:52:12.920 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-25T08:52:12.927+0000 7ffbc7c047c0 -1 osd.5 0 log_to_monitors true 2024-04-25T08:52:13.067 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-25T08:52:13.073+0000 7fdf727b27c0 -1 osd.0 0 log_to_monitors true 2024-04-25T08:52:13.110 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-25T08:52:13.119+0000 7f6f854047c0 -1 osd.4 0 log_to_monitors true 2024-04-25T08:52:13.171 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-25T08:52:13.177+0000 7f107970c7c0 -1 osd.2 0 log_to_monitors true 2024-04-25T08:52:13.202 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-25T08:52:13.211+0000 7f58231187c0 -1 osd.7 0 log_to_monitors true 2024-04-25T08:52:13.209 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-25T08:52:13.217+0000 7fd1aee397c0 -1 osd.1 0 log_to_monitors true 2024-04-25T08:52:13.276 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-25T08:52:13.283+0000 7fc51d7ac7c0 -1 osd.6 0 log_to_monitors true 2024-04-25T08:52:16.636 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-25T08:52:16.643+0000 7ffbc3bad640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T08:52:16.638 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-25T08:52:16.643+0000 7f6f813ad640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T08:52:16.646 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-25T08:52:16.651+0000 7fc519755640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T08:52:16.655 INFO:tasks.ceph.osd.4.smithi203.stderr:2024-04-25T08:52:16.663+0000 7f6f791e2640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:52:16.655 INFO:tasks.ceph.osd.5.smithi203.stderr:2024-04-25T08:52:16.663+0000 7ffbbb9e2640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:52:16.662 INFO:tasks.ceph.osd.6.smithi203.stderr:2024-04-25T08:52:16.671+0000 7fc51158a640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:52:16.670 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-25T08:52:16.677+0000 7f10756b5640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T08:52:16.691 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-25T08:52:16.699+0000 7f581f0c1640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T08:52:16.691 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:16.692 INFO:tasks.ceph.osd.2.smithi037.stderr:2024-04-25T08:52:16.701+0000 7f106d4ea640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:52:16.694 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-25T08:52:16.701+0000 7f50aa74b640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T08:52:16.701 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-25T08:52:16.709+0000 7fdf6e75b640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T08:52:16.707 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-25T08:52:16.717+0000 7fd1aade2640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T08:52:16.727 INFO:tasks.ceph.osd.7.smithi203.stderr:2024-04-25T08:52:16.735+0000 7f5816ef6640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:52:16.732 INFO:tasks.ceph.osd.0.smithi037.stderr:2024-04-25T08:52:16.741+0000 7fdf66590640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:52:16.740 INFO:tasks.ceph.osd.3.smithi037.stderr:2024-04-25T08:52:16.749+0000 7f50a2580640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:52:16.752 INFO:tasks.ceph.osd.1.smithi037.stderr:2024-04-25T08:52:16.761+0000 7fd1a2c17640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:52:16.807 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.813+0000 7fe4585af640 1 Processor -- start 2024-04-25T08:52:16.807 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.813+0000 7fe4585af640 1 -- start start 2024-04-25T08:52:16.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe450145130 0x7fe45006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe45015fff0 0x7fe45015f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4500ce2a0 con 0x7fe45015f990 2024-04-25T08:52:16.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45015fde0 con 0x7fe45015fff0 2024-04-25T08:52:16.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45006bcf0 con 0x7fe450145130 2024-04-25T08:52:16.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe455b4a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe45015fff0 0x7fe45015f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:16.809 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe455b4a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe45015fff0 0x7fe45015f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53044/0 (socket says 172.21.15.37:53044) 2024-04-25T08:52:16.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe455b4a640 1 -- 172.21.15.37:0/4167970691 learned_addr learned my addr 172.21.15.37:0/4167970691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:16.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe45634b640 1 --2- 172.21.15.37:0/4167970691 >> v2:172.21.15.203:3300/0 conn(0x7fe450145130 0x7fe45006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:16.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe455b4a640 1 -- 172.21.15.37:0/4167970691 >> v2:172.21.15.203:3300/0 conn(0x7fe450145130 msgr2=0x7fe45006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:16.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe455b4a640 1 --2- 172.21.15.37:0/4167970691 >> v2:172.21.15.203:3300/0 conn(0x7fe450145130 0x7fe45006b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe455b4a640 1 -- 172.21.15.37:0/4167970691 >> v1:172.21.15.37:6789/0 conn(0x7fe45015f990 legacy=0x7fe45006d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:16.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe455b4a640 1 -- 172.21.15.37:0/4167970691 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4500cc1f0 con 0x7fe45015fff0 2024-04-25T08:52:16.810 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe45634b640 1 --2- 172.21.15.37:0/4167970691 >> v2:172.21.15.203:3300/0 conn(0x7fe450145130 0x7fe45006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:16.811 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe455b4a640 1 --2- 172.21.15.37:0/4167970691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe45015fff0 0x7fe45015f450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe440009870 tx=0x7fe44001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc63c76723e4baea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:16.811 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe455349640 1 -- 172.21.15.37:0/4167970691 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4400b0070 con 0x7fe45015fff0 2024-04-25T08:52:16.811 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe455349640 1 -- 172.21.15.37:0/4167970691 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4400ab600 con 0x7fe45015fff0 2024-04-25T08:52:16.811 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe455349640 1 -- 172.21.15.37:0/4167970691 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4400ab9d0 con 0x7fe45015fff0 2024-04-25T08:52:16.812 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 -- 172.21.15.37:0/4167970691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe45015fff0 msgr2=0x7fe45015f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:16.812 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 --2- 172.21.15.37:0/4167970691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe45015fff0 0x7fe45015f450 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe440009870 tx=0x7fe44001c520 comp rx=0 tx=0).stop 2024-04-25T08:52:16.812 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 -- 172.21.15.37:0/4167970691 shutdown_connections 2024-04-25T08:52:16.812 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 --2- 172.21.15.37:0/4167970691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe45015fff0 0x7fe45015f450 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.812 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 --2- 172.21.15.37:0/4167970691 >> v2:172.21.15.203:3300/0 conn(0x7fe450145130 0x7fe45006b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.812 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 -- 172.21.15.37:0/4167970691 >> 172.21.15.37:0/4167970691 conn(0x7fe45006ac90 msgr2=0x7fe45006b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:16.812 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 -- 172.21.15.37:0/4167970691 shutdown_connections 2024-04-25T08:52:16.813 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.817+0000 7fe4585af640 1 -- 172.21.15.37:0/4167970691 wait complete. 2024-04-25T08:52:16.815 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe4585af640 1 Processor -- start 2024-04-25T08:52:16.815 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe4585af640 1 -- start start 2024-04-25T08:52:16.816 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe4585af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe450145130 0x7fe45015af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.816 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe4585af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe45015f990 0x7fe45006b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.816 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe4585af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe45015fff0 0x7fe45006b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.816 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe4585af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45015bfd0 con 0x7fe450145130 2024-04-25T08:52:16.816 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe4585af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45015be50 con 0x7fe45015f990 2024-04-25T08:52:16.816 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe4585af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4501687d0 con 0x7fe45015fff0 2024-04-25T08:52:16.816 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe45634b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe450145130 0x7fe45015af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:16.817 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe45634b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe450145130 0x7fe45015af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53052/0 (socket says 172.21.15.37:53052) 2024-04-25T08:52:16.817 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe45634b640 1 -- 172.21.15.37:0/3828587798 learned_addr learned my addr 172.21.15.37:0/3828587798 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:16.817 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe455b4a640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe45015f990 0x7fe45006b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:16.817 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe456b4c640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe45015fff0 0x7fe45006b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:16.817 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe45634b640 1 -- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe45015fff0 msgr2=0x7fe45006b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:16.817 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe45634b640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe45015fff0 0x7fe45006b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.817 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe45634b640 1 -- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe45015f990 msgr2=0x7fe45006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:16.818 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe45634b640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe45015f990 0x7fe45006b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.818 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe45634b640 1 -- 172.21.15.37:0/3828587798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe450160620 con 0x7fe450145130 2024-04-25T08:52:16.818 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe455b4a640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe45015f990 0x7fe45006b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:16.818 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe456b4c640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe45015fff0 0x7fe45006b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:16.818 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe45634b640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe450145130 0x7fe45015af20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00cf90 tx=0x7fe44c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:16.818 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe446ffd640 1 -- 172.21.15.37:0/3828587798 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c01a070 con 0x7fe450145130 2024-04-25T08:52:16.818 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe446ffd640 1 -- 172.21.15.37:0/3828587798 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c00d840 con 0x7fe450145130 2024-04-25T08:52:16.818 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe4585af640 1 -- 172.21.15.37:0/3828587798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe450165ca0 con 0x7fe450145130 2024-04-25T08:52:16.819 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe446ffd640 1 -- 172.21.15.37:0/3828587798 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00dc10 con 0x7fe450145130 2024-04-25T08:52:16.819 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe4585af640 1 -- 172.21.15.37:0/3828587798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe450166540 con 0x7fe450145130 2024-04-25T08:52:16.819 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.821+0000 7fe4585af640 1 -- 172.21.15.37:0/3828587798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe450166090 con 0x7fe450145130 2024-04-25T08:52:16.820 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.825+0000 7fe446ffd640 1 -- 172.21.15.37:0/3828587798 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) v1 ==== 86671+0+0 (secure 0 0 0) 0x7fe44c01d070 con 0x7fe450145130 2024-04-25T08:52:16.820 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.825+0000 7fe446ffd640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe434066ed0 0x7fe434069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:16.820 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.825+0000 7fe446ffd640 1 -- 172.21.15.37:0/3828587798 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fe44c084840 con 0x7fe450145130 2024-04-25T08:52:16.820 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.825+0000 7fe455b4a640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe434066ed0 0x7fe434069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:16.820 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.825+0000 7fe455b4a640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe434066ed0 0x7fe434069390 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe4400ad000 tx=0x7fe440008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:16.821 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.829+0000 7fe446ffd640 1 -- 172.21.15.37:0/3828587798 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe44c056100 con 0x7fe450145130 2024-04-25T08:52:16.985 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.993+0000 7fe4585af640 1 -- 172.21.15.37:0/3828587798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe450074d80 con 0x7fe450145130 2024-04-25T08:52:16.986 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.993+0000 7fe446ffd640 1 -- 172.21.15.37:0/3828587798 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fe44c05a1a0 con 0x7fe450145130 2024-04-25T08:52:16.986 INFO:teuthology.misc.health.smithi037.stdout: 2024-04-25T08:52:16.986 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":12,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:14.393261+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:16.988 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.993+0000 7fe4585af640 1 -- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe434066ed0 msgr2=0x7fe434069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:16.988 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.993+0000 7fe4585af640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe434066ed0 0x7fe434069390 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe4400ad000 tx=0x7fe440008040 comp rx=0 tx=0).stop 2024-04-25T08:52:16.988 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.993+0000 7fe4585af640 1 -- 172.21.15.37:0/3828587798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe450145130 msgr2=0x7fe45015af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:16.988 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.993+0000 7fe4585af640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe450145130 0x7fe45015af20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe44c00cf90 tx=0x7fe44c00a7d0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.988 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.993+0000 7fe4585af640 1 -- 172.21.15.37:0/3828587798 shutdown_connections 2024-04-25T08:52:16.988 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.997+0000 7fe4585af640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe434066ed0 0x7fe434069390 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.988 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.997+0000 7fe4585af640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe45015fff0 0x7fe45006b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.988 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.997+0000 7fe4585af640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe45015f990 0x7fe45006b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.988 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.997+0000 7fe4585af640 1 --2- 172.21.15.37:0/3828587798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe450145130 0x7fe45015af20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:16.988 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.997+0000 7fe4585af640 1 -- 172.21.15.37:0/3828587798 >> 172.21.15.37:0/3828587798 conn(0x7fe45006ac90 msgr2=0x7fe45015ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:16.989 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.997+0000 7fe4585af640 1 -- 172.21.15.37:0/3828587798 shutdown_connections 2024-04-25T08:52:16.989 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:16.997+0000 7fe4585af640 1 -- 172.21.15.37:0/3828587798 wait complete. 2024-04-25T08:52:17.003 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T08:52:23.811 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:23.936 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.941+0000 7fd51c3de640 1 Processor -- start 2024-04-25T08:52:23.936 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51c3de640 1 -- start start 2024-04-25T08:52:23.937 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51c3de640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5141440d0 0x7fd51406d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51c3de640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd5141615d0 0x7fd514161a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51c3de640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd5140ae110 0x7fd5140ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51c3de640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5140aee80 con 0x7fd5140ae110 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51c3de640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5140b27e0 con 0x7fd5141615d0 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51c3de640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5140b2ac0 con 0x7fd5141440d0 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51a97b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd5140ae110 0x7fd5140ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51a97b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd5140ae110 0x7fd5140ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51448/0 (socket says 172.21.15.37:51448) 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51a97b640 1 -- 172.21.15.37:0/2279018786 learned_addr learned my addr 172.21.15.37:0/2279018786 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd519979640 1 --2- 172.21.15.37:0/2279018786 >> v2:172.21.15.203:3300/0 conn(0x7fd5141615d0 0x7fd514161a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51a17a640 1 --2- 172.21.15.37:0/2279018786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5141440d0 0x7fd51406d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51a97b640 1 -- 172.21.15.37:0/2279018786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5141440d0 msgr2=0x7fd51406d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:23.938 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51a97b640 1 --2- 172.21.15.37:0/2279018786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5141440d0 0x7fd51406d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:23.939 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51a97b640 1 -- 172.21.15.37:0/2279018786 >> v2:172.21.15.203:3300/0 conn(0x7fd5141615d0 msgr2=0x7fd514161a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:23.939 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51a97b640 1 --2- 172.21.15.37:0/2279018786 >> v2:172.21.15.203:3300/0 conn(0x7fd5141615d0 0x7fd514161a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:23.939 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51a97b640 1 -- 172.21.15.37:0/2279018786 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5140b2da0 con 0x7fd5140ae110 2024-04-25T08:52:23.939 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.945+0000 7fd51a97b640 1 --2- 172.21.15.37:0/2279018786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140ae110 0x7fd5140ae5a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd514072bb0 tx=0x7fd50c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41bcc7b79ac5eb92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:23.940 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd519178640 1 -- 172.21.15.37:0/2279018786 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c0a5670 con 0x7fd5140ae110 2024-04-25T08:52:23.940 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd519178640 1 -- 172.21.15.37:0/2279018786 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50c0a5810 con 0x7fd5140ae110 2024-04-25T08:52:23.940 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd519178640 1 -- 172.21.15.37:0/2279018786 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50c0a5be0 con 0x7fd5140ae110 2024-04-25T08:52:23.941 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 -- 172.21.15.37:0/2279018786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140ae110 msgr2=0x7fd5140ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:23.941 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 --2- 172.21.15.37:0/2279018786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140ae110 0x7fd5140ae5a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd514072bb0 tx=0x7fd50c01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:52:23.941 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 -- 172.21.15.37:0/2279018786 shutdown_connections 2024-04-25T08:52:23.941 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 --2- 172.21.15.37:0/2279018786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140ae110 0x7fd5140ae5a0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:23.941 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 --2- 172.21.15.37:0/2279018786 >> v2:172.21.15.203:3300/0 conn(0x7fd5141615d0 0x7fd514161a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:23.941 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 --2- 172.21.15.37:0/2279018786 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5141440d0 0x7fd51406d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:23.941 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 -- 172.21.15.37:0/2279018786 >> 172.21.15.37:0/2279018786 conn(0x7fd51406ac90 msgr2=0x7fd51406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:23.941 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 -- 172.21.15.37:0/2279018786 shutdown_connections 2024-04-25T08:52:23.941 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 -- 172.21.15.37:0/2279018786 wait complete. 2024-04-25T08:52:23.942 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 Processor -- start 2024-04-25T08:52:23.942 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 -- start start 2024-04-25T08:52:23.944 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140ae110 0x7fd51415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:23.944 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5141440d0 0x7fd51413d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:23.944 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5141615d0 0x7fd51413bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:23.944 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5140b3e60 con 0x7fd5140ae110 2024-04-25T08:52:23.944 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5140b3ce0 con 0x7fd5141615d0 2024-04-25T08:52:23.944 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51c3de640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd51415dfe0 con 0x7fd5141440d0 2024-04-25T08:52:23.944 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51a17a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140ae110 0x7fd51415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:23.944 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51a17a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140ae110 0x7fd51415cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51452/0 (socket says 172.21.15.37:51452) 2024-04-25T08:52:23.944 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51a17a640 1 -- 172.21.15.37:0/1706876323 learned_addr learned my addr 172.21.15.37:0/1706876323 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51a97b640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5141615d0 0x7fd51413bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd519979640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5141440d0 0x7fd51413d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51a17a640 1 -- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5141440d0 msgr2=0x7fd51413d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51a17a640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5141440d0 0x7fd51413d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51a17a640 1 -- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5141615d0 msgr2=0x7fd51413bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51a17a640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5141615d0 0x7fd51413bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51a17a640 1 -- 172.21.15.37:0/1706876323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd51413c740 con 0x7fd5140ae110 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51a97b640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5141615d0 0x7fd51413bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.949+0000 7fd51a17a640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140ae110 0x7fd51415cef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd51000c7f0 tx=0x7fd51000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.953+0000 7fd50affd640 1 -- 172.21.15.37:0/1706876323 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510015070 con 0x7fd5140ae110 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.953+0000 7fd50affd640 1 -- 172.21.15.37:0/1706876323 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd510004220 con 0x7fd5140ae110 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.953+0000 7fd51c3de640 1 -- 172.21.15.37:0/1706876323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51413ca20 con 0x7fd5140ae110 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.953+0000 7fd50affd640 1 -- 172.21.15.37:0/1706876323 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd510003620 con 0x7fd5140ae110 2024-04-25T08:52:23.945 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.953+0000 7fd51c3de640 1 -- 172.21.15.37:0/1706876323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51413d2c0 con 0x7fd5140ae110 2024-04-25T08:52:23.946 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.953+0000 7fd51c3de640 1 -- 172.21.15.37:0/1706876323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51413cbf0 con 0x7fd5140ae110 2024-04-25T08:52:23.946 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.953+0000 7fd50affd640 1 -- 172.21.15.37:0/1706876323 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd5100039d0 con 0x7fd5140ae110 2024-04-25T08:52:23.946 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.953+0000 7fd50affd640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4f8066f60 0x7fd4f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:23.947 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.953+0000 7fd50affd640 1 -- 172.21.15.37:0/1706876323 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd510084e60 con 0x7fd5140ae110 2024-04-25T08:52:23.947 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.953+0000 7fd519979640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4f8066f60 0x7fd4f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:23.947 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.953+0000 7fd519979640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4f8066f60 0x7fd4f8069420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5140ad060 tx=0x7fd504008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:23.950 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:23.957+0000 7fd50affd640 1 -- 172.21.15.37:0/1706876323 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd510056130 con 0x7fd5140ae110 2024-04-25T08:52:24.111 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.117+0000 7fd51c3de640 1 -- 172.21.15.37:0/1706876323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd514074d80 con 0x7fd5140ae110 2024-04-25T08:52:24.113 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd50affd640 1 -- 172.21.15.37:0/1706876323 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12848 (secure 0 0 0) 0x7fd510055a30 con 0x7fd5140ae110 2024-04-25T08:52:24.113 INFO:teuthology.misc.health.smithi037.stdout: 2024-04-25T08:52:24.113 INFO:teuthology.misc.health.smithi037.stdout:{"epoch":17,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:21.706223+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:24.115 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 -- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4f8066f60 msgr2=0x7fd4f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.115 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4f8066f60 0x7fd4f8069420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd5140ad060 tx=0x7fd504008040 comp rx=0 tx=0).stop 2024-04-25T08:52:24.115 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 -- 172.21.15.37:0/1706876323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140ae110 msgr2=0x7fd51415cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.115 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140ae110 0x7fd51415cef0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd51000c7f0 tx=0x7fd51000ccb0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.115 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 -- 172.21.15.37:0/1706876323 shutdown_connections 2024-04-25T08:52:24.115 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4f8066f60 0x7fd4f8069420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.115 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5141615d0 0x7fd51413bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.115 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd5141440d0 0x7fd51413d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.115 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 --2- 172.21.15.37:0/1706876323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5140ae110 0x7fd51415cef0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.116 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 -- 172.21.15.37:0/1706876323 >> 172.21.15.37:0/1706876323 conn(0x7fd51406ac90 msgr2=0x7fd51406b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:24.116 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 -- 172.21.15.37:0/1706876323 shutdown_connections 2024-04-25T08:52:24.116 INFO:teuthology.misc.health.smithi037.stderr:2024-04-25T08:52:24.121+0000 7fd51c3de640 1 -- 172.21.15.37:0/1706876323 wait complete. 2024-04-25T08:52:24.131 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T08:52:24.131 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T08:52:24.131 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:52:24.131 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:52:24.133 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:52:24.134 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:52:24.134 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:52:24.136 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:52:24.136 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:52:24.136 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:52:24.139 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:52:24.139 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:52:24.139 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:52:24.141 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:52:24.141 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T08:52:24.142 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:24.219 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T08:52:24.220 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T08:52:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95f437640 1 Processor -- start 2024-04-25T08:52:24.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95f437640 1 -- start start 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95f437640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc95815da80 0x7fc9580d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95f437640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95806be60 con 0x7fc9580d1cd0 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95f437640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95815de60 con 0x7fc9580d07f0 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95f437640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95806c440 con 0x7fc95815da80 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95d1d3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc95815da80 0x7fc9580d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95c9d2640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc9580d1cd0 0x7fc9580d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34772/0 (socket says 172.21.15.37:34772) 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95d1d3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc95815da80 0x7fc9580d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51472/0 (socket says 172.21.15.37:51472) 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95c9d2640 1 -- 172.21.15.37:0/2730262489 learned_addr learned my addr 172.21.15.37:0/2730262489 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95d1d3640 1 -- 172.21.15.37:0/2730262489 >> v1:172.21.15.203:6789/0 conn(0x7fc9580d07f0 legacy=0x7fc95806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95d1d3640 1 -- 172.21.15.37:0/2730262489 >> v1:172.21.15.37:6789/0 conn(0x7fc9580d1cd0 legacy=0x7fc9580d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc94ffff640 1 -- 172.21.15.37:0/2730262489 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4187979877 0 0) 0x7fc95806be60 con 0x7fc9580d1cd0 2024-04-25T08:52:24.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.265+0000 7fc95d1d3640 1 -- 172.21.15.37:0/2730262489 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9580d2450 con 0x7fc95815da80 2024-04-25T08:52:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95d1d3640 1 --2- 172.21.15.37:0/2730262489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc95815da80 0x7fc9580d4230 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc95806c5c0 tx=0x7fc95401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882f2b1d00b4a49e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc94ffff640 1 -- 172.21.15.37:0/2730262489 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9540b1070 con 0x7fc95815da80 2024-04-25T08:52:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc94ffff640 1 -- 172.21.15.37:0/2730262489 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9540ac410 con 0x7fc95815da80 2024-04-25T08:52:24.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc94ffff640 1 -- 172.21.15.37:0/2730262489 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9540ac7e0 con 0x7fc95815da80 2024-04-25T08:52:24.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 -- 172.21.15.37:0/2730262489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc95815da80 msgr2=0x7fc9580d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 --2- 172.21.15.37:0/2730262489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc95815da80 0x7fc9580d4230 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc95806c5c0 tx=0x7fc95401c450 comp rx=0 tx=0).stop 2024-04-25T08:52:24.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 -- 172.21.15.37:0/2730262489 shutdown_connections 2024-04-25T08:52:24.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 --2- 172.21.15.37:0/2730262489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc95815da80 0x7fc9580d4230 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 -- 172.21.15.37:0/2730262489 >> 172.21.15.37:0/2730262489 conn(0x7fc95808b000 msgr2=0x7fc9580d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:24.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 -- 172.21.15.37:0/2730262489 shutdown_connections 2024-04-25T08:52:24.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 -- 172.21.15.37:0/2730262489 wait complete. 2024-04-25T08:52:24.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 Processor -- start 2024-04-25T08:52:24.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 -- start start 2024-04-25T08:52:24.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9580d07f0 0x7fc9580c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9580d1cd0 0x7fc9580c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc95815da80 0x7fc9580c68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95806d970 con 0x7fc9580d1cd0 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95806d7f0 con 0x7fc95815da80 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95f437640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95806daf0 con 0x7fc9580d07f0 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.269+0000 7fc95c9d2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9580d1cd0 0x7fc9580c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95c9d2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9580d1cd0 0x7fc9580c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51488/0 (socket says 172.21.15.37:51488) 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95c9d2640 1 -- 172.21.15.37:0/2409058209 learned_addr learned my addr 172.21.15.37:0/2409058209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95d1d3640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9580d07f0 0x7fc9580c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95d9d4640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc95815da80 0x7fc9580c68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95c9d2640 1 -- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9580d07f0 msgr2=0x7fc9580c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95c9d2640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9580d07f0 0x7fc9580c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95c9d2640 1 -- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc95815da80 msgr2=0x7fc9580c68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95c9d2640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc95815da80 0x7fc9580c68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95c9d2640 1 -- 172.21.15.37:0/2409058209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9581570b0 con 0x7fc9580d1cd0 2024-04-25T08:52:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95c9d2640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9580d1cd0 0x7fc9580c6360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c760 tx=0x7fc94800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc94dffb640 1 -- 172.21.15.37:0/2409058209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948007270 con 0x7fc9580d1cd0 2024-04-25T08:52:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc94dffb640 1 -- 172.21.15.37:0/2409058209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc948007430 con 0x7fc9580d1cd0 2024-04-25T08:52:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95f437640 1 -- 172.21.15.37:0/2409058209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc958157370 con 0x7fc9580d1cd0 2024-04-25T08:52:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc94dffb640 1 -- 172.21.15.37:0/2409058209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9480059f0 con 0x7fc9580d1cd0 2024-04-25T08:52:24.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95f437640 1 -- 172.21.15.37:0/2409058209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc958157930 con 0x7fc9580d1cd0 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95f437640 1 -- 172.21.15.37:0/2409058209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95804efa0 con 0x7fc9580d1cd0 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc94dffb640 1 -- 172.21.15.37:0/2409058209 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc948005b90 con 0x7fc9580d1cd0 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc94dffb640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc93c066f90 0x7fc93c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc94dffb640 1 -- 172.21.15.37:0/2409058209 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc9480848c0 con 0x7fc9580d1cd0 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95d1d3640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc93c066f90 0x7fc93c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.273+0000 7fc95d1d3640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc93c066f90 0x7fc93c069450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc9540ae000 tx=0x7fc954008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.277+0000 7fc94dffb640 1 -- 172.21.15.37:0/2409058209 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc948055b90 con 0x7fc9580d1cd0 2024-04-25T08:52:24.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.437+0000 7fc95f437640 1 -- 172.21.15.37:0/2409058209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9580cf0a0 con 0x7fc9580d1cd0 2024-04-25T08:52:24.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc94dffb640 1 -- 172.21.15.37:0/2409058209 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12848 (secure 0 0 0) 0x7fc948001d10 con 0x7fc9580d1cd0 2024-04-25T08:52:24.432 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:24.433 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:21.706223+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:24.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 -- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc93c066f90 msgr2=0x7fc93c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc93c066f90 0x7fc93c069450 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc9540ae000 tx=0x7fc954008040 comp rx=0 tx=0).stop 2024-04-25T08:52:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 -- 172.21.15.37:0/2409058209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9580d1cd0 msgr2=0x7fc9580c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9580d1cd0 0x7fc9580c6360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c760 tx=0x7fc94800cc20 comp rx=0 tx=0).stop 2024-04-25T08:52:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 -- 172.21.15.37:0/2409058209 shutdown_connections 2024-04-25T08:52:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc93c066f90 0x7fc93c069450 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc95815da80 0x7fc9580c68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9580d1cd0 0x7fc9580c6360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 --2- 172.21.15.37:0/2409058209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc9580d07f0 0x7fc9580c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 -- 172.21.15.37:0/2409058209 >> 172.21.15.37:0/2409058209 conn(0x7fc95808b000 msgr2=0x7fc9581553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 -- 172.21.15.37:0/2409058209 shutdown_connections 2024-04-25T08:52:24.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.441+0000 7fc95f437640 1 -- 172.21.15.37:0/2409058209 wait complete. 2024-04-25T08:52:24.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:52:24.452 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:52:24.537 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-25T08:52:24.545 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T08:52:24.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b37fac640 1 Processor -- start 2024-04-25T08:52:24.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b37fac640 1 -- start start 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b37fac640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1b3006bca0 0x7f1b300cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b37fac640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1b30066e30 0x7f1b3006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b37fac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b300af370 0x7f1b3006d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b37fac640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b300669a0 con 0x7f1b3006bca0 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b37fac640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b3006ec30 con 0x7f1b30066e30 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b37fac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b300cd5f0 con 0x7f1b300af370 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b35d48640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1b3006bca0 0x7f1b300cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b35d48640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1b3006bca0 0x7f1b300cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51502/0 (socket says 172.21.15.37:51502) 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b35d48640 1 -- 172.21.15.37:0/4168092759 learned_addr learned my addr 172.21.15.37:0/4168092759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b36549640 1 --2- 172.21.15.37:0/4168092759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b300af370 0x7f1b3006d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b35547640 1 --2- 172.21.15.37:0/4168092759 >> v2:172.21.15.203:3300/0 conn(0x7f1b30066e30 0x7f1b3006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b35d48640 1 -- 172.21.15.37:0/4168092759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b300af370 msgr2=0x7f1b3006d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b35d48640 1 --2- 172.21.15.37:0/4168092759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b300af370 0x7f1b3006d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b35d48640 1 -- 172.21.15.37:0/4168092759 >> v2:172.21.15.203:3300/0 conn(0x7f1b30066e30 msgr2=0x7f1b3006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:24.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b35d48640 1 --2- 172.21.15.37:0/4168092759 >> v2:172.21.15.203:3300/0 conn(0x7f1b30066e30 0x7f1b3006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b35d48640 1 -- 172.21.15.37:0/4168092759 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b3006c5c0 con 0x7f1b3006bca0 2024-04-25T08:52:24.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b36549640 1 --2- 172.21.15.37:0/4168092759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b300af370 0x7f1b3006d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:24.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.589+0000 7f1b35547640 1 --2- 172.21.15.37:0/4168092759 >> v2:172.21.15.203:3300/0 conn(0x7f1b30066e30 0x7f1b3006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:24.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b35d48640 1 --2- 172.21.15.37:0/4168092759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b3006bca0 0x7f1b300cd0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b300731a0 tx=0x7f1b2c01c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8787750afe0eccbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b34d46640 1 -- 172.21.15.37:0/4168092759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0b0070 con 0x7f1b3006bca0 2024-04-25T08:52:24.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b34d46640 1 -- 172.21.15.37:0/4168092759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2c0ab600 con 0x7f1b3006bca0 2024-04-25T08:52:24.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b34d46640 1 -- 172.21.15.37:0/4168092759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2c0ab9d0 con 0x7f1b3006bca0 2024-04-25T08:52:24.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 -- 172.21.15.37:0/4168092759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b3006bca0 msgr2=0x7f1b300cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 --2- 172.21.15.37:0/4168092759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b3006bca0 0x7f1b300cd0b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1b300731a0 tx=0x7f1b2c01c110 comp rx=0 tx=0).stop 2024-04-25T08:52:24.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 -- 172.21.15.37:0/4168092759 shutdown_connections 2024-04-25T08:52:24.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 --2- 172.21.15.37:0/4168092759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b300af370 0x7f1b3006d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 --2- 172.21.15.37:0/4168092759 >> v2:172.21.15.203:3300/0 conn(0x7f1b30066e30 0x7f1b3006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 --2- 172.21.15.37:0/4168092759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b3006bca0 0x7f1b300cd0b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 -- 172.21.15.37:0/4168092759 >> 172.21.15.37:0/4168092759 conn(0x7f1b30071810 msgr2=0x7f1b30071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:24.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 -- 172.21.15.37:0/4168092759 shutdown_connections 2024-04-25T08:52:24.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 -- 172.21.15.37:0/4168092759 wait complete. 2024-04-25T08:52:24.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 Processor -- start 2024-04-25T08:52:24.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 -- start start 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30066e30 0x7f1b30151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b3006bca0 0x7f1b30148570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b300af370 0x7f1b30158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3015cc80 con 0x7f1b3006bca0 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3015cb00 con 0x7f1b30066e30 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.593+0000 7f1b37fac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b300ceaf0 con 0x7f1b300af370 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35547640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b3006bca0 0x7f1b30148570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35d48640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30066e30 0x7f1b30151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35547640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b3006bca0 0x7f1b30148570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51518/0 (socket says 172.21.15.37:51518) 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35d48640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30066e30 0x7f1b30151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:51604/0 (socket says 172.21.15.37:51604) 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35547640 1 -- 172.21.15.37:0/3798684147 learned_addr learned my addr 172.21.15.37:0/3798684147 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:24.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b36549640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b300af370 0x7f1b30158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35547640 1 -- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b300af370 msgr2=0x7f1b30158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35547640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b300af370 0x7f1b30158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35547640 1 -- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30066e30 msgr2=0x7f1b30151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35547640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30066e30 0x7f1b30151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35547640 1 -- 172.21.15.37:0/3798684147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b30151b60 con 0x7f1b3006bca0 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35d48640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30066e30 0x7f1b30151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35547640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b3006bca0 0x7f1b30148570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000c7f0 tx=0x7f1b2000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b267fc640 1 -- 172.21.15.37:0/3798684147 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20015070 con 0x7f1b3006bca0 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b267fc640 1 -- 172.21.15.37:0/3798684147 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b20007a30 con 0x7f1b3006bca0 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b37fac640 1 -- 172.21.15.37:0/3798684147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b300ac8e0 con 0x7f1b3006bca0 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b267fc640 1 -- 172.21.15.37:0/3798684147 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20007e00 con 0x7f1b3006bca0 2024-04-25T08:52:24.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b37fac640 1 -- 172.21.15.37:0/3798684147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b300acea0 con 0x7f1b3006bca0 2024-04-25T08:52:24.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b37fac640 1 -- 172.21.15.37:0/3798684147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3004efa0 con 0x7f1b3006bca0 2024-04-25T08:52:24.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b267fc640 1 -- 172.21.15.37:0/3798684147 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b20018070 con 0x7f1b3006bca0 2024-04-25T08:52:24.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b267fc640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b14066f60 0x7f1b14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b267fc640 1 -- 172.21.15.37:0/3798684147 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1b20084c20 con 0x7f1b3006bca0 2024-04-25T08:52:24.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35d48640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b14066f60 0x7f1b14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.597+0000 7f1b35d48640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b14066f60 0x7f1b14069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0ad000 tx=0x7f1b2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.601+0000 7f1b267fc640 1 -- 172.21.15.37:0/3798684147 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b20055820 con 0x7f1b3006bca0 2024-04-25T08:52:24.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.761+0000 7f1b37fac640 1 -- 172.21.15.37:0/3798684147 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1b30072400 con 0x7f1b3006bca0 2024-04-25T08:52:24.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.761+0000 7f1b267fc640 1 -- 172.21.15.37:0/3798684147 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1b200598c0 con 0x7f1b3006bca0 2024-04-25T08:52:24.755 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:52:24.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 -- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b14066f60 msgr2=0x7f1b14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b14066f60 0x7f1b14069420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1b2c0ad000 tx=0x7f1b2c008040 comp rx=0 tx=0).stop 2024-04-25T08:52:24.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 -- 172.21.15.37:0/3798684147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b3006bca0 msgr2=0x7f1b30148570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b3006bca0 0x7f1b30148570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b2000c7f0 tx=0x7f1b2000ccb0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 -- 172.21.15.37:0/3798684147 shutdown_connections 2024-04-25T08:52:24.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b14066f60 0x7f1b14069420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b300af370 0x7f1b30158df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b3006bca0 0x7f1b30148570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 --2- 172.21.15.37:0/3798684147 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b30066e30 0x7f1b30151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 -- 172.21.15.37:0/3798684147 >> 172.21.15.37:0/3798684147 conn(0x7f1b30071810 msgr2=0x7f1b30152950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:24.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 -- 172.21.15.37:0/3798684147 shutdown_connections 2024-04-25T08:52:24.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.765+0000 7f1b37fac640 1 -- 172.21.15.37:0/3798684147 wait complete. 2024-04-25T08:52:24.774 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:24.806 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:52:24.825 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:52:24.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f22536f9640 1 Processor -- start 2024-04-25T08:52:24.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f22536f9640 1 -- start start 2024-04-25T08:52:24.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f22536f9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f224c15d850 0x7f224c0d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f22536f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f224c15dc30 0x7f224c0d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f22536f9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224c0ce6b0 con 0x7f224c0d1de0 2024-04-25T08:52:24.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f22536f9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f224c0ce990 con 0x7f224c15d850 2024-04-25T08:52:24.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f22536f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224c0cec70 con 0x7f224c15dc30 2024-04-25T08:52:24.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f2251495640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f224c15d850 0x7f224c0d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f2251495640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f224c15d850 0x7f224c0d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:51620/0 (socket says 172.21.15.37:51620) 2024-04-25T08:52:24.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f2251495640 1 -- 172.21.15.37:0/2888570388 learned_addr learned my addr 172.21.15.37:0/2888570388 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:24.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f2250c94640 1 --2- 172.21.15.37:0/2888570388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f224c15dc30 0x7f224c0d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f2251495640 1 -- 172.21.15.37:0/2888570388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f224c15dc30 msgr2=0x7f224c0d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:24.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f2251495640 1 --2- 172.21.15.37:0/2888570388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f224c15dc30 0x7f224c0d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f2251495640 1 -- 172.21.15.37:0/2888570388 >> v1:172.21.15.203:6789/0 conn(0x7f224c0d1de0 legacy=0x7f224c0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f2243fff640 1 -- 172.21.15.37:0/2888570388 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1256215845 0 0) 0x7f224c0ce6b0 con 0x7f224c0d1de0 2024-04-25T08:52:24.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.909+0000 7f2251495640 1 -- 172.21.15.37:0/2888570388 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c0cef50 con 0x7f224c15d850 2024-04-25T08:52:24.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f2251495640 1 --2- 172.21.15.37:0/2888570388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f224c15d850 0x7f224c0d0160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2248009870 tx=0x7f224801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4161216d9a6cc28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f2243fff640 1 -- 172.21.15.37:0/2888570388 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22480aa5f0 con 0x7f224c15d850 2024-04-25T08:52:24.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f2243fff640 1 -- 172.21.15.37:0/2888570388 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22480af070 con 0x7f224c15d850 2024-04-25T08:52:24.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f2243fff640 1 -- 172.21.15.37:0/2888570388 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22480aa9b0 con 0x7f224c15d850 2024-04-25T08:52:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f22536f9640 1 -- 172.21.15.37:0/2888570388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f224c15d850 msgr2=0x7f224c0d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f22536f9640 1 --2- 172.21.15.37:0/2888570388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f224c15d850 0x7f224c0d0160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2248009870 tx=0x7f224801c560 comp rx=0 tx=0).stop 2024-04-25T08:52:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f22536f9640 1 -- 172.21.15.37:0/2888570388 shutdown_connections 2024-04-25T08:52:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f22536f9640 1 --2- 172.21.15.37:0/2888570388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f224c15dc30 0x7f224c0d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f22536f9640 1 --2- 172.21.15.37:0/2888570388 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f224c15d850 0x7f224c0d0160 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f22536f9640 1 -- 172.21.15.37:0/2888570388 >> 172.21.15.37:0/2888570388 conn(0x7f224c08b000 msgr2=0x7f224c0d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f22536f9640 1 -- 172.21.15.37:0/2888570388 shutdown_connections 2024-04-25T08:52:24.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.913+0000 7f22536f9640 1 -- 172.21.15.37:0/2888570388 wait complete. 2024-04-25T08:52:24.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f22536f9640 1 Processor -- start 2024-04-25T08:52:24.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f22536f9640 1 -- start start 2024-04-25T08:52:24.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f22536f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f224c0d1de0 0x7f224c14b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f22536f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f224c15d850 0x7f224c0b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f22536f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224c15dc30 0x7f224c14c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f22536f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c16a4c0 con 0x7f224c15dc30 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f22536f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c16a340 con 0x7f224c0d1de0 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f22536f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224c16a640 con 0x7f224c15d850 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2251c96640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224c15dc30 0x7f224c14c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2251c96640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224c15dc30 0x7f224c14c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51528/0 (socket says 172.21.15.37:51528) 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2251c96640 1 -- 172.21.15.37:0/325813899 learned_addr learned my addr 172.21.15.37:0/325813899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2250c94640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f224c15d850 0x7f224c0b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2251495640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f224c0d1de0 0x7f224c14b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2251c96640 1 -- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f224c15d850 msgr2=0x7f224c0b8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2251c96640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f224c15d850 0x7f224c0b8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2251c96640 1 -- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f224c0d1de0 msgr2=0x7f224c14b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2251c96640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f224c0d1de0 0x7f224c14b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2251c96640 1 -- 172.21.15.37:0/325813899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c1562a0 con 0x7f224c15dc30 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2251c96640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224c15dc30 0x7f224c14c400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f224400c7b0 tx=0x7f224400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2241ffb640 1 -- 172.21.15.37:0/325813899 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224400ce60 con 0x7f224c15dc30 2024-04-25T08:52:24.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2241ffb640 1 -- 172.21.15.37:0/325813899 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2244003db0 con 0x7f224c15dc30 2024-04-25T08:52:24.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f22536f9640 1 -- 172.21.15.37:0/325813899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224c0b7b10 con 0x7f224c15dc30 2024-04-25T08:52:24.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f2241ffb640 1 -- 172.21.15.37:0/325813899 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22440052b0 con 0x7f224c15dc30 2024-04-25T08:52:24.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f22536f9640 1 -- 172.21.15.37:0/325813899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224c0b80d0 con 0x7f224c15dc30 2024-04-25T08:52:24.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.917+0000 7f22536f9640 1 -- 172.21.15.37:0/325813899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224c04efa0 con 0x7f224c15dc30 2024-04-25T08:52:24.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.921+0000 7f2241ffb640 1 -- 172.21.15.37:0/325813899 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2244005450 con 0x7f224c15dc30 2024-04-25T08:52:24.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.921+0000 7f2241ffb640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2230066f60 0x7f2230069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:24.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.921+0000 7f2241ffb640 1 -- 172.21.15.37:0/325813899 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2244084a10 con 0x7f224c15dc30 2024-04-25T08:52:24.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.921+0000 7f2251495640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2230066f60 0x7f2230069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:24.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.921+0000 7f2251495640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2230066f60 0x7f2230069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22480ae000 tx=0x7f22480a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:24.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:24.925+0000 7f2241ffb640 1 -- 172.21.15.37:0/325813899 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2244055720 con 0x7f224c15dc30 2024-04-25T08:52:25.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.117+0000 7f22536f9640 1 -- 172.21.15.37:0/325813899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f224c0d0e60 con 0x7f224c15dc30 2024-04-25T08:52:25.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.117+0000 7f2241ffb640 1 -- 172.21.15.37:0/325813899 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7f22440597c0 con 0x7f224c15dc30 2024-04-25T08:52:25.111 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:25.111 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T08:52:25.111 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-04-25T08:52:25.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 -- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2230066f60 msgr2=0x7f2230069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2230066f60 0x7f2230069420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f22480ae000 tx=0x7f22480a7000 comp rx=0 tx=0).stop 2024-04-25T08:52:25.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 -- 172.21.15.37:0/325813899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224c15dc30 msgr2=0x7f224c14c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:25.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224c15dc30 0x7f224c14c400 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f224400c7b0 tx=0x7f224400cc70 comp rx=0 tx=0).stop 2024-04-25T08:52:25.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 -- 172.21.15.37:0/325813899 shutdown_connections 2024-04-25T08:52:25.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2230066f60 0x7f2230069420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224c15dc30 0x7f224c14c400 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f224c15d850 0x7f224c0b8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 --2- 172.21.15.37:0/325813899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f224c0d1de0 0x7f224c14b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:25.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 -- 172.21.15.37:0/325813899 >> 172.21.15.37:0/325813899 conn(0x7f224c08b000 msgr2=0x7f224c0cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:25.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 -- 172.21.15.37:0/325813899 shutdown_connections 2024-04-25T08:52:25.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:25.121+0000 7f22536f9640 1 -- 172.21.15.37:0/325813899 wait complete. 2024-04-25T08:52:27.131 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:27.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4137454640 1 Processor -- start 2024-04-25T08:52:27.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4137454640 1 -- start start 2024-04-25T08:52:27.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4137454640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4130144ca0 0x7f413006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4137454640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41300ccc70 con 0x7f413006ec60 2024-04-25T08:52:27.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4137454640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f413006dc40 con 0x7f41300ce510 2024-04-25T08:52:27.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4137454640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41300cc2b0 con 0x7f4130144ca0 2024-04-25T08:52:27.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41359f1640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f413006ec60 0x7f4130142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34780/0 (socket says 172.21.15.37:34780) 2024-04-25T08:52:27.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41359f1640 1 -- 172.21.15.37:0/2060463017 learned_addr learned my addr 172.21.15.37:0/2060463017 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:27.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657315426 0 0) 0x7f41300ccc70 con 0x7f413006ec60 2024-04-25T08:52:27.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41349ef640 1 --2- 172.21.15.37:0/2060463017 >> v2:172.21.15.203:3300/0 conn(0x7f4130144ca0 0x7f413006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f411c003530 con 0x7f413006ec60 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 226175658 0 0) 0x7f411c003530 con 0x7f413006ec60 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 >> v2:172.21.15.203:3300/0 conn(0x7f4130144ca0 msgr2=0x7f413006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 --2- 172.21.15.37:0/2060463017 >> v2:172.21.15.203:3300/0 conn(0x7f4130144ca0 0x7f413006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 >> v1:172.21.15.203:6789/0 conn(0x7f41300ce510 legacy=0x7f41300ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f413006df20 con 0x7f413006ec60 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f4128001a00 con 0x7f413006ec60 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 >> v1:172.21.15.37:6789/0 conn(0x7f413006ec60 legacy=0x7f4130142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f411c003a30 0x7f411c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f411c004810 0x7f411c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 0x7f411c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41300cc2b0 con 0x7f411c0051d0 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c00c0e0 con 0x7f411c004810 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c00c2e0 con 0x7f411c003a30 2024-04-25T08:52:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41359f1640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 0x7f411c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41349ef640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f411c004810 0x7f411c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41351f0640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f411c003a30 0x7f411c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41359f1640 1 -- 172.21.15.37:0/2060463017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f411c003a30 msgr2=0x7f411c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:27.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41359f1640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f411c003a30 0x7f411c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41359f1640 1 -- 172.21.15.37:0/2060463017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f411c004810 msgr2=0x7f411c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:27.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41359f1640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f411c004810 0x7f411c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41359f1640 1 -- 172.21.15.37:0/2060463017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c003530 con 0x7f411c0051d0 2024-04-25T08:52:27.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.273+0000 7f41351f0640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f411c003a30 0x7f411c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:27.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f41359f1640 1 -- 172.21.15.37:0/2060463017 reap_dead start 2024-04-25T08:52:27.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f41359f1640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 0x7f411c00b9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f412800a850 tx=0x7f412801d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2aa21a8b46abf41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412800a4a0 con 0x7f411c0051d0 2024-04-25T08:52:27.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41280ad650 con 0x7f411c0051d0 2024-04-25T08:52:27.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4127fff640 1 -- 172.21.15.37:0/2060463017 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41280ada40 con 0x7f411c0051d0 2024-04-25T08:52:27.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4137454640 1 -- 172.21.15.37:0/2060463017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 msgr2=0x7f411c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4137454640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 0x7f411c00b9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f412800a850 tx=0x7f412801d3a0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4137454640 1 -- 172.21.15.37:0/2060463017 shutdown_connections 2024-04-25T08:52:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4137454640 1 --2- 172.21.15.37:0/2060463017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 0x7f411c00b9d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4137454640 1 -- 172.21.15.37:0/2060463017 >> 172.21.15.37:0/2060463017 conn(0x7f41300b11a0 msgr2=0x7f41300b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4137454640 1 -- 172.21.15.37:0/2060463017 shutdown_connections 2024-04-25T08:52:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4137454640 1 -- 172.21.15.37:0/2060463017 wait complete. 2024-04-25T08:52:27.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4137454640 1 Processor -- start 2024-04-25T08:52:27.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.277+0000 7f4137454640 1 -- start start 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4137454640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 0x7f41300cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4137454640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41300b0360 0x7f41300c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4137454640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41300b07e0 0x7f41300c4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4137454640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41301451f0 con 0x7f411c0051d0 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4137454640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413006e290 con 0x7f41300b07e0 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4137454640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4130145370 con 0x7f41300b0360 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41351f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 0x7f41300cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41351f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 0x7f41300cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51540/0 (socket says 172.21.15.37:51540) 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41351f0640 1 -- 172.21.15.37:0/2786581853 learned_addr learned my addr 172.21.15.37:0/2786581853 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41349ef640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41300b0360 0x7f41300c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41359f1640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41300b07e0 0x7f41300c4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41351f0640 1 -- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41300b0360 msgr2=0x7f41300c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41351f0640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41300b0360 0x7f41300c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41351f0640 1 -- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41300b07e0 msgr2=0x7f41300c4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41351f0640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41300b07e0 0x7f41300c4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41351f0640 1 -- 172.21.15.37:0/2786581853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41301386f0 con 0x7f411c0051d0 2024-04-25T08:52:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41351f0640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 0x7f41300cae30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f412c00d540 tx=0x7f412c00da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4125ffb640 1 -- 172.21.15.37:0/2786581853 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c005480 con 0x7f411c0051d0 2024-04-25T08:52:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4125ffb640 1 -- 172.21.15.37:0/2786581853 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412c017070 con 0x7f411c0051d0 2024-04-25T08:52:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4137454640 1 -- 172.21.15.37:0/2786581853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41301389d0 con 0x7f411c0051d0 2024-04-25T08:52:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4125ffb640 1 -- 172.21.15.37:0/2786581853 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412c003840 con 0x7f411c0051d0 2024-04-25T08:52:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4137454640 1 -- 172.21.15.37:0/2786581853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4130138f90 con 0x7f411c0051d0 2024-04-25T08:52:27.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4137454640 1 -- 172.21.15.37:0/2786581853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41301393d0 con 0x7f411c0051d0 2024-04-25T08:52:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4125ffb640 1 -- 172.21.15.37:0/2786581853 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f412c0039e0 con 0x7f411c0051d0 2024-04-25T08:52:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4125ffb640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4114066f60 0x7f4114069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f4125ffb640 1 -- 172.21.15.37:0/2786581853 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f412c084b20 con 0x7f411c0051d0 2024-04-25T08:52:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.281+0000 7f41349ef640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4114066f60 0x7f4114069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.285+0000 7f41349ef640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4114066f60 0x7f4114069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41300c5d40 tx=0x7f4120008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.285+0000 7f4125ffb640 1 -- 172.21.15.37:0/2786581853 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f412c019310 con 0x7f411c0051d0 2024-04-25T08:52:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.481+0000 7f4137454640 1 -- 172.21.15.37:0/2786581853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f413004efa0 con 0x7f411c0051d0 2024-04-25T08:52:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.481+0000 7f4125ffb640 1 -- 172.21.15.37:0/2786581853 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3697 (secure 0 0 0) 0x7f412c055e80 con 0x7f411c0051d0 2024-04-25T08:52:27.476 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:27.477 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2902565757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2902565757},{"type":"v1","addr":"172.21.15.203:6835","nonce":2902565757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/822172299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":822172299},{"type":"v1","addr":"172.21.15.203:6837","nonce":822172299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3681855890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3681855890},{"type":"v1","addr":"172.21.15.37:6833","nonce":3681855890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T08:52:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 -- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4114066f60 msgr2=0x7f4114069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4114066f60 0x7f4114069420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f41300c5d40 tx=0x7f4120008040 comp rx=0 tx=0).stop 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 -- 172.21.15.37:0/2786581853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 msgr2=0x7f41300cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 0x7f41300cae30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f412c00d540 tx=0x7f412c00da00 comp rx=0 tx=0).stop 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 -- 172.21.15.37:0/2786581853 shutdown_connections 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4114066f60 0x7f4114069420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41300b07e0 0x7f41300c4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41300b0360 0x7f41300c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 --2- 172.21.15.37:0/2786581853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f411c0051d0 0x7f41300cae30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 -- 172.21.15.37:0/2786581853 >> 172.21.15.37:0/2786581853 conn(0x7f41300b11a0 msgr2=0x7f41300b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 -- 172.21.15.37:0/2786581853 shutdown_connections 2024-04-25T08:52:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.485+0000 7f4137454640 1 -- 172.21.15.37:0/2786581853 wait complete. 2024-04-25T08:52:27.494 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:27.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0988699640 1 Processor -- start 2024-04-25T08:52:27.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0988699640 1 -- start start 2024-04-25T08:52:27.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0988699640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09800b3f50 0x7f098015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0988699640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09800b24a0 0x7f09800b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0988699640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09800b2090 con 0x7f098015a700 2024-04-25T08:52:27.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0988699640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098015f200 con 0x7f09800b24a0 2024-04-25T08:52:27.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0988699640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09800b43e0 con 0x7f09800b3f50 2024-04-25T08:52:27.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0986435640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f098015a700 0x7f09800cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34794/0 (socket says 172.21.15.37:34794) 2024-04-25T08:52:27.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0986435640 1 -- 172.21.15.37:0/903461757 learned_addr learned my addr 172.21.15.37:0/903461757 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:27.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0986c36640 1 --2- 172.21.15.37:0/903461757 >> v2:172.21.15.37:3300/0 conn(0x7f09800b24a0 0x7f09800b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0985c34640 1 --2- 172.21.15.37:0/903461757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09800b3f50 0x7f098015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0986c36640 1 -- 172.21.15.37:0/903461757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09800b3f50 msgr2=0x7f098015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:27.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0986c36640 1 --2- 172.21.15.37:0/903461757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09800b3f50 0x7f098015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0986c36640 1 -- 172.21.15.37:0/903461757 >> v1:172.21.15.37:6789/0 conn(0x7f098015a700 legacy=0x7f09800cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0985433640 1 -- 172.21.15.37:0/903461757 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375676654 0 0) 0x7f09800b2090 con 0x7f098015a700 2024-04-25T08:52:27.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.621+0000 7f0986c36640 1 -- 172.21.15.37:0/903461757 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09800b46c0 con 0x7f09800b24a0 2024-04-25T08:52:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0986c36640 1 --2- 172.21.15.37:0/903461757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09800b24a0 0x7f09800b2930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0978009d90 tx=0x7f097801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9194f299cd354f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0985433640 1 -- 172.21.15.37:0/903461757 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09780b2070 con 0x7f09800b24a0 2024-04-25T08:52:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0985433640 1 -- 172.21.15.37:0/903461757 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09780a5800 con 0x7f09800b24a0 2024-04-25T08:52:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0985433640 1 -- 172.21.15.37:0/903461757 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09780a5bd0 con 0x7f09800b24a0 2024-04-25T08:52:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0988699640 1 -- 172.21.15.37:0/903461757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09800b24a0 msgr2=0x7f09800b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0988699640 1 --2- 172.21.15.37:0/903461757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09800b24a0 0x7f09800b2930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0978009d90 tx=0x7f097801c850 comp rx=0 tx=0).stop 2024-04-25T08:52:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0988699640 1 -- 172.21.15.37:0/903461757 shutdown_connections 2024-04-25T08:52:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0988699640 1 --2- 172.21.15.37:0/903461757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09800b24a0 0x7f09800b2930 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0988699640 1 --2- 172.21.15.37:0/903461757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09800b3f50 0x7f098015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0988699640 1 -- 172.21.15.37:0/903461757 >> 172.21.15.37:0/903461757 conn(0x7f09800b11a0 msgr2=0x7f09800b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0988699640 1 -- 172.21.15.37:0/903461757 shutdown_connections 2024-04-25T08:52:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.625+0000 7f0988699640 1 -- 172.21.15.37:0/903461757 wait complete. 2024-04-25T08:52:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0988699640 1 Processor -- start 2024-04-25T08:52:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0988699640 1 -- start start 2024-04-25T08:52:27.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0988699640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09800b24a0 0x7f0980154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0988699640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f098015a700 0x7f09800a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0988699640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09801fa450 0x7f09800b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0988699640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800cd310 con 0x7f098015a700 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0988699640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800ce600 con 0x7f09800b24a0 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0988699640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09800cd490 con 0x7f09801fa450 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0985c34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f098015a700 0x7f09800a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0985c34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f098015a700 0x7f09800a8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51554/0 (socket says 172.21.15.37:51554) 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0985c34640 1 -- 172.21.15.37:0/1125270548 learned_addr learned my addr 172.21.15.37:0/1125270548 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0986435640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09800b24a0 0x7f0980154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0985c34640 1 -- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09801fa450 msgr2=0x7f09800b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0986c36640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09801fa450 0x7f09800b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0985c34640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09801fa450 0x7f09800b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0985c34640 1 -- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09800b24a0 msgr2=0x7f0980154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0985c34640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09800b24a0 0x7f0980154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0985c34640 1 -- 172.21.15.37:0/1125270548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09800a8670 con 0x7f098015a700 2024-04-25T08:52:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0986435640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09800b24a0 0x7f0980154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0985c34640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f098015a700 0x7f09800a8ea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f097000d830 tx=0x7f097000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0976ffd640 1 -- 172.21.15.37:0/1125270548 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097000b660 con 0x7f098015a700 2024-04-25T08:52:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0976ffd640 1 -- 172.21.15.37:0/1125270548 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0970019070 con 0x7f098015a700 2024-04-25T08:52:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0976ffd640 1 -- 172.21.15.37:0/1125270548 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097000ba00 con 0x7f098015a700 2024-04-25T08:52:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0988699640 1 -- 172.21.15.37:0/1125270548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09800a8910 con 0x7f098015a700 2024-04-25T08:52:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0988699640 1 -- 172.21.15.37:0/1125270548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09800a93e0 con 0x7f098015a700 2024-04-25T08:52:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0988699640 1 -- 172.21.15.37:0/1125270548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098004efa0 con 0x7f098015a700 2024-04-25T08:52:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.629+0000 7f0976ffd640 1 -- 172.21.15.37:0/1125270548 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f097001e460 con 0x7f098015a700 2024-04-25T08:52:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.633+0000 7f0976ffd640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0964066f60 0x7f0964069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.633+0000 7f0976ffd640 1 -- 172.21.15.37:0/1125270548 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0970085410 con 0x7f098015a700 2024-04-25T08:52:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.633+0000 7f0986435640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0964066f60 0x7f0964069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.633+0000 7f0986435640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0964066f60 0x7f0964069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f097c000f70 tx=0x7f097c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.637+0000 7f0976ffd640 1 -- 172.21.15.37:0/1125270548 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09700566d0 con 0x7f098015a700 2024-04-25T08:52:27.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.793+0000 7f0988699640 1 -- 172.21.15.37:0/1125270548 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f098006f260 con 0x7f098015a700 2024-04-25T08:52:27.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.793+0000 7f0976ffd640 1 -- 172.21.15.37:0/1125270548 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12848 (secure 0 0 0) 0x7f0970055fd0 con 0x7f098015a700 2024-04-25T08:52:27.788 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:27.788 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:21.706223+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 -- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0964066f60 msgr2=0x7f0964069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0964066f60 0x7f0964069420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f097c000f70 tx=0x7f097c009040 comp rx=0 tx=0).stop 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 -- 172.21.15.37:0/1125270548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f098015a700 msgr2=0x7f09800a8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f098015a700 0x7f09800a8ea0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f097000d830 tx=0x7f097000dcf0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 -- 172.21.15.37:0/1125270548 shutdown_connections 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0964066f60 0x7f0964069420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09801fa450 0x7f09800b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f098015a700 0x7f09800a8ea0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 --2- 172.21.15.37:0/1125270548 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09800b24a0 0x7f0980154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 -- 172.21.15.37:0/1125270548 >> 172.21.15.37:0/1125270548 conn(0x7f09800b11a0 msgr2=0x7f0980159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 -- 172.21.15.37:0/1125270548 shutdown_connections 2024-04-25T08:52:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.797+0000 7f0988699640 1 -- 172.21.15.37:0/1125270548 wait complete. 2024-04-25T08:52:27.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:52:27.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:52:27.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f3476178640 1 Processor -- start 2024-04-25T08:52:27.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f3476178640 1 -- start start 2024-04-25T08:52:27.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f3476178640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f347015b800 0x7f34700cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f3476178640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34700cfad0 0x7f3470072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f3476178640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34700cbd20 con 0x7f347015b420 2024-04-25T08:52:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f3476178640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34700cff40 con 0x7f347015b800 2024-04-25T08:52:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f3476178640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34700cc930 con 0x7f34700cfad0 2024-04-25T08:52:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f347015b420 0x7f34700ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34796/0 (socket says 172.21.15.37:34796) 2024-04-25T08:52:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346f7fe640 1 -- 172.21.15.37:0/2961888697 learned_addr learned my addr 172.21.15.37:0/2961888697 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346effd640 1 --2- 172.21.15.37:0/2961888697 >> v2:172.21.15.203:3300/0 conn(0x7f347015b800 0x7f34700cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346ffff640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34700cfad0 0x7f3470072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901159016 0 0) 0x7f34700cbd20 con 0x7f347015b420 2024-04-25T08:52:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3458003530 con 0x7f347015b420 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496193986 0 0) 0x7f3458003530 con 0x7f347015b420 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34700cfad0 msgr2=0x7f3470072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34700cfad0 0x7f3470072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 >> v2:172.21.15.203:3300/0 conn(0x7f347015b800 msgr2=0x7f34700cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 --2- 172.21.15.37:0/2961888697 >> v2:172.21.15.203:3300/0 conn(0x7f347015b800 0x7f34700cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34700cc000 con 0x7f347015b420 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3460001680 con 0x7f347015b420 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 >> v1:172.21.15.37:6789/0 conn(0x7f347015b420 legacy=0x7f34700ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3458003a30 0x7f3458003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34700cfad0 0x7f3458004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458004d50 0x7f34580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700cff40 con 0x7f3458004d50 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700cc930 con 0x7f3458003a30 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345800b9d0 con 0x7f34700cfad0 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346ffff640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458004d50 0x7f34580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346f7fe640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3458003a30 0x7f3458003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346effd640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34700cfad0 0x7f3458004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346ffff640 1 -- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34700cfad0 msgr2=0x7f3458004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346ffff640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34700cfad0 0x7f3458004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346ffff640 1 -- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3458003a30 msgr2=0x7f3458003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346ffff640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3458003a30 0x7f3458003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346ffff640 1 -- 172.21.15.37:0/2961888697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3458003530 con 0x7f3458004d50 2024-04-25T08:52:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.937+0000 7f346f7fe640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3458003a30 0x7f3458003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f346ffff640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458004d50 0x7f34580051d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3468009fc0 tx=0x7f346801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70992afced479fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34680b2070 con 0x7f3458004d50 2024-04-25T08:52:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34680a5c90 con 0x7f3458004d50 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f346e7fc640 1 -- 172.21.15.37:0/2961888697 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3468009c10 con 0x7f3458004d50 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f3476178640 1 -- 172.21.15.37:0/2961888697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458004d50 msgr2=0x7f34580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f3476178640 1 --2- 172.21.15.37:0/2961888697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3458004d50 0x7f34580051d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3468009fc0 tx=0x7f346801cc80 comp rx=0 tx=0).stop 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f346ffff640 1 -- 172.21.15.37:0/2961888697 reap_dead start 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f3476178640 1 -- 172.21.15.37:0/2961888697 shutdown_connections 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f3476178640 1 -- 172.21.15.37:0/2961888697 >> 172.21.15.37:0/2961888697 conn(0x7f347006ae60 msgr2=0x7f347006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f3476178640 1 -- 172.21.15.37:0/2961888697 shutdown_connections 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f3476178640 1 -- 172.21.15.37:0/2961888697 wait complete. 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f3476178640 1 Processor -- start 2024-04-25T08:52:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.941+0000 7f3476178640 1 -- start start 2024-04-25T08:52:27.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3476178640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3458004d50 0x7f347015a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3476178640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34700c8230 0x7f3470159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3476178640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3470159c70 0x7f34701e6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3476178640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700cb9f0 con 0x7f3470159c70 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3476178640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34700cc5c0 con 0x7f34700c8230 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3476178640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34700cadb0 con 0x7f3458004d50 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3470159c70 0x7f34701e6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3470159c70 0x7f34701e6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51576/0 (socket says 172.21.15.37:51576) 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346ffff640 1 -- 172.21.15.37:0/3785111594 learned_addr learned my addr 172.21.15.37:0/3785111594 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346f7fe640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3458004d50 0x7f347015a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346effd640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34700c8230 0x7f3470159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346ffff640 1 -- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3458004d50 msgr2=0x7f347015a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346ffff640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3458004d50 0x7f347015a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346ffff640 1 -- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34700c8230 msgr2=0x7f3470159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346ffff640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34700c8230 0x7f3470159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:27.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346ffff640 1 -- 172.21.15.37:0/3785111594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34700c8e70 con 0x7f3470159c70 2024-04-25T08:52:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346effd640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34700c8230 0x7f3470159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346f7fe640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3458004d50 0x7f347015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346ffff640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3470159c70 0x7f34701e6b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3468002240 tx=0x7f3468009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3457fff640 1 -- 172.21.15.37:0/3785111594 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34680bd070 con 0x7f3470159c70 2024-04-25T08:52:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3457fff640 1 -- 172.21.15.37:0/3785111594 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34680a5a90 con 0x7f3470159c70 2024-04-25T08:52:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3457fff640 1 -- 172.21.15.37:0/3785111594 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34680b2040 con 0x7f3470159c70 2024-04-25T08:52:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3476178640 1 -- 172.21.15.37:0/3785111594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34700c9150 con 0x7f3470159c70 2024-04-25T08:52:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3476178640 1 -- 172.21.15.37:0/3785111594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34701e7090 con 0x7f3470159c70 2024-04-25T08:52:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3476178640 1 -- 172.21.15.37:0/3785111594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34700c7b40 con 0x7f3470159c70 2024-04-25T08:52:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3457fff640 1 -- 172.21.15.37:0/3785111594 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34680039a0 con 0x7f3470159c70 2024-04-25T08:52:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3457fff640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f345c066f90 0x7f345c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f3457fff640 1 -- 172.21.15.37:0/3785111594 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f346811ede0 con 0x7f3470159c70 2024-04-25T08:52:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.945+0000 7f346f7fe640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f345c066f90 0x7f345c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.949+0000 7f346f7fe640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f345c066f90 0x7f345c069450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34600018f0 tx=0x7f3460009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:27.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:27.949+0000 7f3457fff640 1 -- 172.21.15.37:0/3785111594 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3468016990 con 0x7f3470159c70 2024-04-25T08:52:28.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.109+0000 7f3476178640 1 -- 172.21.15.37:0/3785111594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f347004efa0 con 0x7f3470159c70 2024-04-25T08:52:28.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.109+0000 7f3457fff640 1 -- 172.21.15.37:0/3785111594 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34680efaf0 con 0x7f3470159c70 2024-04-25T08:52:28.103 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:52:28.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 -- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f345c066f90 msgr2=0x7f345c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f345c066f90 0x7f345c069450 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34600018f0 tx=0x7f3460009040 comp rx=0 tx=0).stop 2024-04-25T08:52:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 -- 172.21.15.37:0/3785111594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3470159c70 msgr2=0x7f34701e6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3470159c70 0x7f34701e6b50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3468002240 tx=0x7f3468009750 comp rx=0 tx=0).stop 2024-04-25T08:52:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 -- 172.21.15.37:0/3785111594 shutdown_connections 2024-04-25T08:52:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f345c066f90 0x7f345c069450 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3470159c70 0x7f34701e6b50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34700c8230 0x7f3470159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 --2- 172.21.15.37:0/3785111594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3458004d50 0x7f347015a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 -- 172.21.15.37:0/3785111594 >> 172.21.15.37:0/3785111594 conn(0x7f347006ae60 msgr2=0x7f34700af110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 -- 172.21.15.37:0/3785111594 shutdown_connections 2024-04-25T08:52:28.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.113+0000 7f3476178640 1 -- 172.21.15.37:0/3785111594 wait complete. 2024-04-25T08:52:28.123 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:52:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.249+0000 7f23da91e640 1 Processor -- start 2024-04-25T08:52:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.249+0000 7f23da91e640 1 -- start start 2024-04-25T08:52:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.249+0000 7f23da91e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f23d4144ca0 0x7f23d406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.249+0000 7f23da91e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f23d406ec60 0x7f23d4142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.249+0000 7f23da91e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d40ccc70 con 0x7f23d40ce510 2024-04-25T08:52:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.249+0000 7f23da91e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d40cc2b0 con 0x7f23d4144ca0 2024-04-25T08:52:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.249+0000 7f23da91e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d406e290 con 0x7f23d406ec60 2024-04-25T08:52:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.249+0000 7f23d3fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f23d40ce510 0x7f23d40ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34812/0 (socket says 172.21.15.37:34812) 2024-04-25T08:52:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.249+0000 7f23d3fff640 1 -- 172.21.15.37:0/1403321549 learned_addr learned my addr 172.21.15.37:0/1403321549 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.249+0000 7f23d37fe640 1 --2- 172.21.15.37:0/1403321549 >> v2:172.21.15.37:3300/0 conn(0x7f23d4144ca0 0x7f23d406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23d8ebb640 1 --2- 172.21.15.37:0/1403321549 >> v2:172.21.15.203:3300/0 conn(0x7f23d406ec60 0x7f23d4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23d37fe640 1 -- 172.21.15.37:0/1403321549 >> v2:172.21.15.203:3300/0 conn(0x7f23d406ec60 msgr2=0x7f23d4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23d2ffd640 1 -- 172.21.15.37:0/1403321549 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80055771 0 0) 0x7f23d40ccc70 con 0x7f23d40ce510 2024-04-25T08:52:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23d37fe640 1 --2- 172.21.15.37:0/1403321549 >> v2:172.21.15.203:3300/0 conn(0x7f23d406ec60 0x7f23d4142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23d37fe640 1 -- 172.21.15.37:0/1403321549 >> v1:172.21.15.37:6789/0 conn(0x7f23d40ce510 legacy=0x7f23d40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23d37fe640 1 -- 172.21.15.37:0/1403321549 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d406dc40 con 0x7f23d4144ca0 2024-04-25T08:52:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23d37fe640 1 --2- 172.21.15.37:0/1403321549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23d4144ca0 0x7f23d406e720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23d40b48f0 tx=0x7f23c401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ebdcf75e42fac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23d2ffd640 1 -- 172.21.15.37:0/1403321549 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40b5070 con 0x7f23d4144ca0 2024-04-25T08:52:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23d2ffd640 1 -- 172.21.15.37:0/1403321549 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c40a8970 con 0x7f23d4144ca0 2024-04-25T08:52:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23d2ffd640 1 -- 172.21.15.37:0/1403321549 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40a8d40 con 0x7f23d4144ca0 2024-04-25T08:52:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23da91e640 1 -- 172.21.15.37:0/1403321549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23d4144ca0 msgr2=0x7f23d406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23da91e640 1 --2- 172.21.15.37:0/1403321549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23d4144ca0 0x7f23d406e720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f23d40b48f0 tx=0x7f23c401ec70 comp rx=0 tx=0).stop 2024-04-25T08:52:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23da91e640 1 -- 172.21.15.37:0/1403321549 shutdown_connections 2024-04-25T08:52:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23da91e640 1 --2- 172.21.15.37:0/1403321549 >> v2:172.21.15.203:3300/0 conn(0x7f23d406ec60 0x7f23d4142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23da91e640 1 --2- 172.21.15.37:0/1403321549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23d4144ca0 0x7f23d406e720 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23da91e640 1 -- 172.21.15.37:0/1403321549 >> 172.21.15.37:0/1403321549 conn(0x7f23d40b11a0 msgr2=0x7f23d40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23da91e640 1 -- 172.21.15.37:0/1403321549 shutdown_connections 2024-04-25T08:52:28.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.253+0000 7f23da91e640 1 -- 172.21.15.37:0/1403321549 wait complete. 2024-04-25T08:52:28.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23da91e640 1 Processor -- start 2024-04-25T08:52:28.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23da91e640 1 -- start start 2024-04-25T08:52:28.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23da91e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23d406ec60 0x7f23d41f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23da91e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23d40ce510 0x7f23d41e50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23da91e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23d4144ca0 0x7f23d41e5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23da91e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d41451f0 con 0x7f23d406ec60 2024-04-25T08:52:28.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23da91e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40cc900 con 0x7f23d40ce510 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23da91e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d4145370 con 0x7f23d4144ca0 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23d406ec60 0x7f23d41f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23d406ec60 0x7f23d41f44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51586/0 (socket says 172.21.15.37:51586) 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d3fff640 1 -- 172.21.15.37:0/1378139431 learned_addr learned my addr 172.21.15.37:0/1378139431 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d37fe640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23d40ce510 0x7f23d41e50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d8ebb640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23d4144ca0 0x7f23d41e5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d3fff640 1 -- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23d4144ca0 msgr2=0x7f23d41e5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d3fff640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23d4144ca0 0x7f23d41e5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d3fff640 1 -- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23d40ce510 msgr2=0x7f23d41e50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d3fff640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23d40ce510 0x7f23d41e50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d3fff640 1 -- 172.21.15.37:0/1378139431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40ad940 con 0x7f23d406ec60 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d3fff640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23d406ec60 0x7f23d41f44f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0011f0 tx=0x7f23bc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d0ff9640 1 -- 172.21.15.37:0/1378139431 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc018070 con 0x7f23d406ec60 2024-04-25T08:52:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d0ff9640 1 -- 172.21.15.37:0/1378139431 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc0074e0 con 0x7f23d406ec60 2024-04-25T08:52:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23da91e640 1 -- 172.21.15.37:0/1378139431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d40adc20 con 0x7f23d406ec60 2024-04-25T08:52:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d0ff9640 1 -- 172.21.15.37:0/1378139431 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc004a10 con 0x7f23d406ec60 2024-04-25T08:52:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23da91e640 1 -- 172.21.15.37:0/1378139431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d40ae1e0 con 0x7f23d406ec60 2024-04-25T08:52:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23da91e640 1 -- 172.21.15.37:0/1378139431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d404efa0 con 0x7f23d406ec60 2024-04-25T08:52:28.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.257+0000 7f23d0ff9640 1 -- 172.21.15.37:0/1378139431 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23bc01b070 con 0x7f23d406ec60 2024-04-25T08:52:28.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.261+0000 7f23d0ff9640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f23b4066f60 0x7f23b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.261+0000 7f23d0ff9640 1 -- 172.21.15.37:0/1378139431 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f23bc084140 con 0x7f23d406ec60 2024-04-25T08:52:28.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.261+0000 7f23d37fe640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f23b4066f60 0x7f23b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.261+0000 7f23d37fe640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f23b4066f60 0x7f23b4069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f23c40b2000 tx=0x7f23c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:28.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.265+0000 7f23d0ff9640 1 -- 172.21.15.37:0/1378139431 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23bc055400 con 0x7f23d406ec60 2024-04-25T08:52:28.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.425+0000 7f23da91e640 1 -- 172.21.15.37:0/1378139431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f23d40b1cc0 con 0x7f23d406ec60 2024-04-25T08:52:28.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.425+0000 7f23d0ff9640 1 -- 172.21.15.37:0/1378139431 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f23bc054d00 con 0x7f23d406ec60 2024-04-25T08:52:28.418 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:28.419 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T08:52:28.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:52:28.419 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T08:52:28.419 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T08:52:28.419 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:52:28.419 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T08:52:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 -- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f23b4066f60 msgr2=0x7f23b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f23b4066f60 0x7f23b4069420 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f23c40b2000 tx=0x7f23c4008040 comp rx=0 tx=0).stop 2024-04-25T08:52:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 -- 172.21.15.37:0/1378139431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23d406ec60 msgr2=0x7f23d41f44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23d406ec60 0x7f23d41f44f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f23bc0011f0 tx=0x7f23bc00c9b0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 -- 172.21.15.37:0/1378139431 shutdown_connections 2024-04-25T08:52:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f23b4066f60 0x7f23b4069420 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23d4144ca0 0x7f23d41e5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23d40ce510 0x7f23d41e50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 --2- 172.21.15.37:0/1378139431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23d406ec60 0x7f23d41f44f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 -- 172.21.15.37:0/1378139431 >> 172.21.15.37:0/1378139431 conn(0x7f23d40b11a0 msgr2=0x7f23d406d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:28.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 -- 172.21.15.37:0/1378139431 shutdown_connections 2024-04-25T08:52:28.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.429+0000 7f23da91e640 1 -- 172.21.15.37:0/1378139431 wait complete. 2024-04-25T08:52:28.437 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:52:28.438 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:52:28.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.565+0000 7f9615228640 1 Processor -- start 2024-04-25T08:52:28.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.565+0000 7f9615228640 1 -- start start 2024-04-25T08:52:28.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f9615228640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f961006eea0 0x7f961015b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f9615228640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9610066e10 con 0x7f961015ed60 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f9615228640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961015e280 con 0x7f96100cf190 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f9615228640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961015e560 con 0x7f961006eea0 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960e59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f961015ed60 0x7f96100cea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:34818/0 (socket says 172.21.15.37:34818) 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960e59c640 1 -- 172.21.15.37:0/1409359862 learned_addr learned my addr 172.21.15.37:0/1409359862 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798341322 0 0) 0x7f9610066e10 con 0x7f961015ed60 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95f8003530 con 0x7f961015ed60 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960ed9d640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f961006eea0 0x7f961015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3703326659 0 0) 0x7f95f8003530 con 0x7f961015ed60 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f961006eea0 msgr2=0x7f961015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f961006eea0 0x7f961015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 >> v1:172.21.15.203:6789/0 conn(0x7f96100cf190 legacy=0x7f96100b3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961015e840 con 0x7f961015ed60 2024-04-25T08:52:28.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f9604001680 con 0x7f961015ed60 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 >> v1:172.21.15.37:6789/0 conn(0x7f961015ed60 legacy=0x7f96100cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f961006eea0 0x7f95f80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f8004020 0x7f95f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95f8004d50 0x7f95f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961015e560 con 0x7f95f8004020 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f800bc10 con 0x7f95f8004d50 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f800be10 con 0x7f961006eea0 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960e59c640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f8004020 0x7f95f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960ed9d640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f961006eea0 0x7f95f80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960e59c640 1 -- 172.21.15.37:0/1409359862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f961006eea0 msgr2=0x7f95f80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960f59e640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95f8004d50 0x7f95f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960e59c640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f961006eea0 0x7f95f80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960e59c640 1 -- 172.21.15.37:0/1409359862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95f8004d50 msgr2=0x7f95f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960e59c640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95f8004d50 0x7f95f80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960e59c640 1 -- 172.21.15.37:0/1409359862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f8003530 con 0x7f95f8004020 2024-04-25T08:52:28.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960e59c640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f8004020 0x7f95f8004810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f960400c240 tx=0x7f960401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=998cd1e8f2215166 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604007d20 con 0x7f95f8004020 2024-04-25T08:52:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96040a9070 con 0x7f95f8004020 2024-04-25T08:52:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.569+0000 7f960dd9b640 1 -- 172.21.15.37:0/1409359862 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604005d80 con 0x7f95f8004020 2024-04-25T08:52:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 -- 172.21.15.37:0/1409359862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f8004020 msgr2=0x7f95f8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 --2- 172.21.15.37:0/1409359862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f8004020 0x7f95f8004810 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f960400c240 tx=0x7f960401cc80 comp rx=0 tx=0).stop 2024-04-25T08:52:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960f59e640 1 -- 172.21.15.37:0/1409359862 reap_dead start 2024-04-25T08:52:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 -- 172.21.15.37:0/1409359862 shutdown_connections 2024-04-25T08:52:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 -- 172.21.15.37:0/1409359862 >> 172.21.15.37:0/1409359862 conn(0x7f96100cc680 msgr2=0x7f96100cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 -- 172.21.15.37:0/1409359862 shutdown_connections 2024-04-25T08:52:28.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 -- 172.21.15.37:0/1409359862 wait complete. 2024-04-25T08:52:28.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 Processor -- start 2024-04-25T08:52:28.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 -- start start 2024-04-25T08:52:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f8004020 0x7f9610158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9610153300 0x7f96101567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96101f71e0 0x7f96101f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9610066b20 con 0x7f96101f71e0 2024-04-25T08:52:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96100669a0 con 0x7f9610153300 2024-04-25T08:52:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9610166480 con 0x7f95f8004020 2024-04-25T08:52:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96101f71e0 0x7f96101f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96101f71e0 0x7f96101f7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51594/0 (socket says 172.21.15.37:51594) 2024-04-25T08:52:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960f59e640 1 -- 172.21.15.37:0/3945423435 learned_addr learned my addr 172.21.15.37:0/3945423435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960ed9d640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f8004020 0x7f9610158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960e59c640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9610153300 0x7f96101567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960f59e640 1 -- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f8004020 msgr2=0x7f9610158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960f59e640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f8004020 0x7f9610158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960f59e640 1 -- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9610153300 msgr2=0x7f96101567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960f59e640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9610153300 0x7f96101567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960f59e640 1 -- 172.21.15.37:0/3945423435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96101577f0 con 0x7f96101f71e0 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960e59c640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9610153300 0x7f96101567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f960f59e640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96101f71e0 0x7f96101f7660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f960800d770 tx=0x7f960800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f95f77fe640 1 -- 172.21.15.37:0/3945423435 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608018070 con 0x7f96101f71e0 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f95f77fe640 1 -- 172.21.15.37:0/3945423435 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9608003b80 con 0x7f96101f71e0 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f95f77fe640 1 -- 172.21.15.37:0/3945423435 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9608005070 con 0x7f96101f71e0 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 -- 172.21.15.37:0/3945423435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9610156f50 con 0x7f96101f71e0 2024-04-25T08:52:28.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.573+0000 7f9615228640 1 -- 172.21.15.37:0/3945423435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9610157510 con 0x7f96101f71e0 2024-04-25T08:52:28.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.577+0000 7f9615228640 1 -- 172.21.15.37:0/3945423435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9610068340 con 0x7f96101f71e0 2024-04-25T08:52:28.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.577+0000 7f95f77fe640 1 -- 172.21.15.37:0/3945423435 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f960801b070 con 0x7f96101f71e0 2024-04-25T08:52:28.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.577+0000 7f95f77fe640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95fc066f60 0x7f95fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:28.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.577+0000 7f95f77fe640 1 -- 172.21.15.37:0/3945423435 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f96080843e0 con 0x7f96101f71e0 2024-04-25T08:52:28.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.577+0000 7f960ed9d640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95fc066f60 0x7f95fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:28.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.577+0000 7f960ed9d640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95fc066f60 0x7f95fc069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f960000b9b0 tx=0x7f9600008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:28.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.581+0000 7f95f77fe640 1 -- 172.21.15.37:0/3945423435 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f96101577f0 con 0x7f96101f71e0 2024-04-25T08:52:28.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:28.733+0000 7f9615228640 1 -- 172.21.15.37:0/3945423435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f96100cd4b0 con 0x7f96101f71e0 2024-04-25T08:52:29.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.176+0000 7f95f77fe640 1 -- 172.21.15.37:0/3945423435 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9608055740 con 0x7f96101f71e0 2024-04-25T08:52:29.171 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T08:52:29.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 -- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95fc066f60 msgr2=0x7f95fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95fc066f60 0x7f95fc069420 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f960000b9b0 tx=0x7f9600008040 comp rx=0 tx=0).stop 2024-04-25T08:52:29.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 -- 172.21.15.37:0/3945423435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96101f71e0 msgr2=0x7f96101f7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96101f71e0 0x7f96101f7660 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f960800d770 tx=0x7f960800dc30 comp rx=0 tx=0).stop 2024-04-25T08:52:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 -- 172.21.15.37:0/3945423435 shutdown_connections 2024-04-25T08:52:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95fc066f60 0x7f95fc069420 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96101f71e0 0x7f96101f7660 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9610153300 0x7f96101567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 --2- 172.21.15.37:0/3945423435 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f8004020 0x7f9610158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 -- 172.21.15.37:0/3945423435 >> 172.21.15.37:0/3945423435 conn(0x7f96100cc680 msgr2=0x7f96100b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 -- 172.21.15.37:0/3945423435 shutdown_connections 2024-04-25T08:52:29.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.180+0000 7f9615228640 1 -- 172.21.15.37:0/3945423435 wait complete. 2024-04-25T08:52:29.199 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T08:52:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.332+0000 7f941e2fb640 1 Processor -- start 2024-04-25T08:52:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.332+0000 7f941e2fb640 1 -- start start 2024-04-25T08:52:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f941e2fb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f94180cf590 0x7f94180cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f941e2fb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9418072000 0x7f9418072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f941e2fb640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94180cbd20 con 0x7f941815b6b0 2024-04-25T08:52:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f941e2fb640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941815ba90 con 0x7f94180cf590 2024-04-25T08:52:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f941e2fb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94180cc000 con 0x7f9418072000 2024-04-25T08:52:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f940ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f94180cf590 0x7f94180cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f940ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f94180cf590 0x7f94180cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39086/0 (socket says 172.21.15.37:39086) 2024-04-25T08:52:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f940ffff640 1 -- 172.21.15.37:0/3565534740 learned_addr learned my addr 172.21.15.37:0/3565534740 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f9417fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f941815b6b0 0x7f94180ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37100/0 (socket says 172.21.15.37:37100) 2024-04-25T08:52:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f940ffff640 1 -- 172.21.15.37:0/3565534740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9418072000 msgr2=0x7f9418072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f941c898640 1 --2- 172.21.15.37:0/3565534740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9418072000 0x7f9418072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f940ffff640 1 --2- 172.21.15.37:0/3565534740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9418072000 0x7f9418072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f940ffff640 1 -- 172.21.15.37:0/3565534740 >> v1:172.21.15.37:6789/0 conn(0x7f941815b6b0 legacy=0x7f94180ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f940ffff640 1 -- 172.21.15.37:0/3565534740 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180cc2e0 con 0x7f94180cf590 2024-04-25T08:52:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f94177fe640 1 -- 172.21.15.37:0/3565534740 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591876122 0 0) 0x7f94180cbd20 con 0x7f941815b6b0 2024-04-25T08:52:29.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f940ffff640 1 --2- 172.21.15.37:0/3565534740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94180cf590 0x7f94180cfa20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9400009870 tx=0x7f940001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cac02f24f5578d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:29.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f94177fe640 1 -- 172.21.15.37:0/3565534740 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94000ab650 con 0x7f94180cf590 2024-04-25T08:52:29.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f94177fe640 1 -- 172.21.15.37:0/3565534740 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94000b0070 con 0x7f94180cf590 2024-04-25T08:52:29.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.336+0000 7f94177fe640 1 -- 172.21.15.37:0/3565534740 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94000aba10 con 0x7f94180cf590 2024-04-25T08:52:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 -- 172.21.15.37:0/3565534740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94180cf590 msgr2=0x7f94180cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 --2- 172.21.15.37:0/3565534740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94180cf590 0x7f94180cfa20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9400009870 tx=0x7f940001c560 comp rx=0 tx=0).stop 2024-04-25T08:52:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 -- 172.21.15.37:0/3565534740 shutdown_connections 2024-04-25T08:52:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 --2- 172.21.15.37:0/3565534740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9418072000 0x7f9418072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 --2- 172.21.15.37:0/3565534740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f94180cf590 0x7f94180cfa20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 -- 172.21.15.37:0/3565534740 >> 172.21.15.37:0/3565534740 conn(0x7f941806ae60 msgr2=0x7f941806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 -- 172.21.15.37:0/3565534740 shutdown_connections 2024-04-25T08:52:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 -- 172.21.15.37:0/3565534740 wait complete. 2024-04-25T08:52:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 Processor -- start 2024-04-25T08:52:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 -- start start 2024-04-25T08:52:29.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9418072000 0x7f9418159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94180cf590 0x7f9418098300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941815b6b0 0x7f94180ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180caf30 con 0x7f941815b6b0 2024-04-25T08:52:29.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94180cadb0 con 0x7f94180cf590 2024-04-25T08:52:29.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94181682a0 con 0x7f9418072000 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941c898640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941815b6b0 0x7f94180ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941c898640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941815b6b0 0x7f94180ad1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39090/0 (socket says 172.21.15.37:39090) 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941c898640 1 -- 172.21.15.37:0/1462874876 learned_addr learned my addr 172.21.15.37:0/1462874876 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f9417fff640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9418072000 0x7f9418159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941c898640 1 -- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9418072000 msgr2=0x7f9418159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941c898640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9418072000 0x7f9418159bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941c898640 1 -- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94180cf590 msgr2=0x7f9418098300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f940ffff640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94180cf590 0x7f9418098300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941c898640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94180cf590 0x7f9418098300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941c898640 1 -- 172.21.15.37:0/1462874876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94180b3610 con 0x7f941815b6b0 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941c898640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941815b6b0 0x7f94180ad1c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f941000cd70 tx=0x7f941000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f94157fa640 1 -- 172.21.15.37:0/1462874876 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410003ba0 con 0x7f941815b6b0 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f94157fa640 1 -- 172.21.15.37:0/1462874876 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9410003d40 con 0x7f941815b6b0 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 -- 172.21.15.37:0/1462874876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94180b33a0 con 0x7f941815b6b0 2024-04-25T08:52:29.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f94157fa640 1 -- 172.21.15.37:0/1462874876 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410004240 con 0x7f941815b6b0 2024-04-25T08:52:29.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.340+0000 7f941e2fb640 1 -- 172.21.15.37:0/1462874876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f941815a490 con 0x7f941815b6b0 2024-04-25T08:52:29.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.344+0000 7f941e2fb640 1 -- 172.21.15.37:0/1462874876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941804ef80 con 0x7f941815b6b0 2024-04-25T08:52:29.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.344+0000 7f94157fa640 1 -- 172.21.15.37:0/1462874876 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f94100043e0 con 0x7f941815b6b0 2024-04-25T08:52:29.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.344+0000 7f94157fa640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f93f8066f60 0x7f93f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:29.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.344+0000 7f94157fa640 1 -- 172.21.15.37:0/1462874876 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f94100859b0 con 0x7f941815b6b0 2024-04-25T08:52:29.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.344+0000 7f9417fff640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f93f8066f60 0x7f93f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:29.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.344+0000 7f9417fff640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f93f8066f60 0x7f93f8069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9408000f70 tx=0x7f9408009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.348+0000 7f94157fa640 1 -- 172.21.15.37:0/1462874876 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9410056ba0 con 0x7f941815b6b0 2024-04-25T08:52:29.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:29.512+0000 7f941e2fb640 1 -- 172.21.15.37:0/1462874876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f94180b2ec0 con 0x7f941815b6b0 2024-04-25T08:52:30.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.180+0000 7f94157fa640 1 -- 172.21.15.37:0/1462874876 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7f94100564a0 con 0x7f941815b6b0 2024-04-25T08:52:30.175 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T08:52:30.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 -- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f93f8066f60 msgr2=0x7f93f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f93f8066f60 0x7f93f8069420 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9408000f70 tx=0x7f9408009040 comp rx=0 tx=0).stop 2024-04-25T08:52:30.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 -- 172.21.15.37:0/1462874876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941815b6b0 msgr2=0x7f94180ad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941815b6b0 0x7f94180ad1c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f941000cd70 tx=0x7f941000a660 comp rx=0 tx=0).stop 2024-04-25T08:52:30.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 -- 172.21.15.37:0/1462874876 shutdown_connections 2024-04-25T08:52:30.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f93f8066f60 0x7f93f8069420 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f941815b6b0 0x7f94180ad1c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f94180cf590 0x7f9418098300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 --2- 172.21.15.37:0/1462874876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9418072000 0x7f9418159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 -- 172.21.15.37:0/1462874876 >> 172.21.15.37:0/1462874876 conn(0x7f941806ae60 msgr2=0x7f94180afe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:30.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 -- 172.21.15.37:0/1462874876 shutdown_connections 2024-04-25T08:52:30.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.184+0000 7f941e2fb640 1 -- 172.21.15.37:0/1462874876 wait complete. 2024-04-25T08:52:30.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:52:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b45984640 1 Processor -- start 2024-04-25T08:52:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b45984640 1 -- start start 2024-04-25T08:52:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b45984640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2b400af280 0x7f2b4015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b45984640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 0x7f2b4006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b45984640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b40066c30 con 0x7f2b4006c1e0 2024-04-25T08:52:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b45984640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b4006daa0 con 0x7f2b400af280 2024-04-25T08:52:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b45984640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b400af660 con 0x7f2b400b2c30 2024-04-25T08:52:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 0x7f2b4006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b3effd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2b400af280 0x7f2b4015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 0x7f2b4006bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:42088/0 (socket says 172.21.15.37:42088) 2024-04-25T08:52:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b3e7fc640 1 -- 172.21.15.37:0/2941872065 learned_addr learned my addr 172.21.15.37:0/2941872065 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b3f7fe640 1 --1- 172.21.15.37:0/2941872065 >> v1:172.21.15.203:6789/0 conn(0x7f2b4006c1e0 0x7f2b40161500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:43122/0 (socket says 172.21.15.37:43122) 2024-04-25T08:52:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b3e7fc640 1 -- 172.21.15.37:0/2941872065 >> v2:172.21.15.203:3300/0 conn(0x7f2b400af280 msgr2=0x7f2b4015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b3e7fc640 1 --2- 172.21.15.37:0/2941872065 >> v2:172.21.15.203:3300/0 conn(0x7f2b400af280 0x7f2b4015df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b3e7fc640 1 -- 172.21.15.37:0/2941872065 >> v1:172.21.15.203:6789/0 conn(0x7f2b4006c1e0 legacy=0x7f2b40161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.348+0000 7f2b3e7fc640 1 -- 172.21.15.37:0/2941872065 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b4015e640 con 0x7f2b400b2c30 2024-04-25T08:52:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b3e7fc640 1 --2- 172.21.15.37:0/2941872065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 0x7f2b4006bca0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2b30009e50 tx=0x7f2b3001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a8f7fe63bee5970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b3dffb640 1 -- 172.21.15.37:0/2941872065 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300a5700 con 0x7f2b400b2c30 2024-04-25T08:52:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b3dffb640 1 -- 172.21.15.37:0/2941872065 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b300a58a0 con 0x7f2b400b2c30 2024-04-25T08:52:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b3dffb640 1 -- 172.21.15.37:0/2941872065 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300a5c70 con 0x7f2b400b2c30 2024-04-25T08:52:30.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b45984640 1 -- 172.21.15.37:0/2941872065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 msgr2=0x7f2b4006bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b45984640 1 --2- 172.21.15.37:0/2941872065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 0x7f2b4006bca0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2b30009e50 tx=0x7f2b3001ca30 comp rx=0 tx=0).stop 2024-04-25T08:52:30.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b45984640 1 -- 172.21.15.37:0/2941872065 shutdown_connections 2024-04-25T08:52:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b45984640 1 --2- 172.21.15.37:0/2941872065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 0x7f2b4006bca0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b45984640 1 --2- 172.21.15.37:0/2941872065 >> v2:172.21.15.203:3300/0 conn(0x7f2b400af280 0x7f2b4015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b45984640 1 -- 172.21.15.37:0/2941872065 >> 172.21.15.37:0/2941872065 conn(0x7f2b40071810 msgr2=0x7f2b40071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b45984640 1 -- 172.21.15.37:0/2941872065 shutdown_connections 2024-04-25T08:52:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b45984640 1 -- 172.21.15.37:0/2941872065 wait complete. 2024-04-25T08:52:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.352+0000 7f2b45984640 1 Processor -- start 2024-04-25T08:52:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b45984640 1 -- start start 2024-04-25T08:52:30.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b45984640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b4006c1e0 0x7f2b4014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b45984640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b400af280 0x7f2b40160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b45984640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 0x7f2b40160580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b45984640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4016aab0 con 0x7f2b400af280 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b45984640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4006c5e0 con 0x7f2b4006c1e0 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b45984640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b4016ac30 con 0x7f2b400b2c30 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b400af280 0x7f2b40160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b400af280 0x7f2b40160040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39106/0 (socket says 172.21.15.37:39106) 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3e7fc640 1 -- 172.21.15.37:0/2618285492 learned_addr learned my addr 172.21.15.37:0/2618285492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3f7fe640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 0x7f2b40160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3effd640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b4006c1e0 0x7f2b4014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3e7fc640 1 -- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 msgr2=0x7f2b40160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3e7fc640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 0x7f2b40160580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3e7fc640 1 -- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b4006c1e0 msgr2=0x7f2b4014e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3e7fc640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b4006c1e0 0x7f2b4014e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3e7fc640 1 -- 172.21.15.37:0/2618285492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b400ad790 con 0x7f2b400af280 2024-04-25T08:52:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3effd640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b4006c1e0 0x7f2b4014e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b3e7fc640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b400af280 0x7f2b40160040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2b3001a660 tx=0x7f2b300197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b37fff640 1 -- 172.21.15.37:0/2618285492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300bc070 con 0x7f2b400af280 2024-04-25T08:52:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b37fff640 1 -- 172.21.15.37:0/2618285492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b300b8640 con 0x7f2b400af280 2024-04-25T08:52:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b45984640 1 -- 172.21.15.37:0/2618285492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b40152690 con 0x7f2b400af280 2024-04-25T08:52:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b37fff640 1 -- 172.21.15.37:0/2618285492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b300b89f0 con 0x7f2b400af280 2024-04-25T08:52:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b45984640 1 -- 172.21.15.37:0/2618285492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b40152c50 con 0x7f2b400af280 2024-04-25T08:52:30.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b45984640 1 -- 172.21.15.37:0/2618285492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b4004efa0 con 0x7f2b400af280 2024-04-25T08:52:30.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b37fff640 1 -- 172.21.15.37:0/2618285492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b300b8b90 con 0x7f2b400af280 2024-04-25T08:52:30.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.356+0000 7f2b37fff640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b20066f60 0x7f2b20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:30.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.360+0000 7f2b37fff640 1 -- 172.21.15.37:0/2618285492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2b3011fbf0 con 0x7f2b400af280 2024-04-25T08:52:30.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.360+0000 7f2b3effd640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b20066f60 0x7f2b20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:30.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.360+0000 7f2b3effd640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b20066f60 0x7f2b20069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009b10 tx=0x7f2b28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:30.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.364+0000 7f2b37fff640 1 -- 172.21.15.37:0/2618285492 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b30016620 con 0x7f2b400af280 2024-04-25T08:52:30.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:30.556+0000 7f2b45984640 1 -- 172.21.15.37:0/2618285492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2b400727c0 con 0x7f2b400af280 2024-04-25T08:52:31.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.208+0000 7f2b37fff640 1 -- 172.21.15.37:0/2618285492 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2b300f0740 con 0x7f2b400af280 2024-04-25T08:52:31.203 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T08:52:31.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.212+0000 7f2b45984640 1 -- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b20066f60 msgr2=0x7f2b20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:31.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.212+0000 7f2b45984640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b20066f60 0x7f2b20069420 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009b10 tx=0x7f2b28008040 comp rx=0 tx=0).stop 2024-04-25T08:52:31.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.212+0000 7f2b45984640 1 -- 172.21.15.37:0/2618285492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b400af280 msgr2=0x7f2b40160040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:31.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.212+0000 7f2b45984640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b400af280 0x7f2b40160040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2b3001a660 tx=0x7f2b300197d0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.216+0000 7f2b45984640 1 -- 172.21.15.37:0/2618285492 shutdown_connections 2024-04-25T08:52:31.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.216+0000 7f2b45984640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b20066f60 0x7f2b20069420 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.216+0000 7f2b45984640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b400b2c30 0x7f2b40160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.216+0000 7f2b45984640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b400af280 0x7f2b40160040 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.216+0000 7f2b45984640 1 --2- 172.21.15.37:0/2618285492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b4006c1e0 0x7f2b4014e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.216+0000 7f2b45984640 1 -- 172.21.15.37:0/2618285492 >> 172.21.15.37:0/2618285492 conn(0x7f2b40071810 msgr2=0x7f2b400b07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:31.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.216+0000 7f2b45984640 1 -- 172.21.15.37:0/2618285492 shutdown_connections 2024-04-25T08:52:31.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.216+0000 7f2b45984640 1 -- 172.21.15.37:0/2618285492 wait complete. 2024-04-25T08:52:31.239 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T08:52:31.240 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T08:52:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.376+0000 7fa021c9f640 1 Processor -- start 2024-04-25T08:52:31.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.376+0000 7fa021c9f640 1 -- start start 2024-04-25T08:52:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.376+0000 7fa021c9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c140ac0 0x7fa01c140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.376+0000 7fa021c9f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa01c0cbfb0 0x7fa01c0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.376+0000 7fa021c9f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa01c066f00 con 0x7fa01c0b3170 2024-04-25T08:52:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.376+0000 7fa021c9f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01c0cc410 con 0x7fa01c0cbfb0 2024-04-25T08:52:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.376+0000 7fa021c9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c141540 con 0x7fa01c140ac0 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa01c0b3170 0x7fa01c15f220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37116/0 (socket says 172.21.15.37:37116) 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01bfff640 1 -- 172.21.15.37:0/881985901 learned_addr learned my addr 172.21.15.37:0/881985901 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01affd640 1 --2- 172.21.15.37:0/881985901 >> v2:172.21.15.203:3300/0 conn(0x7fa01c0cbfb0 0x7fa01c0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01b7fe640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c140ac0 0x7fa01c140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3308840177 0 0) 0x7fa01c066f00 con 0x7fa01c0b3170 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa004003530 con 0x7fa01c0b3170 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3097813300 0 0) 0x7fa004003530 con 0x7fa01c0b3170 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c140ac0 msgr2=0x7fa01c140ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c140ac0 0x7fa01c140ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 >> v2:172.21.15.203:3300/0 conn(0x7fa01c0cbfb0 msgr2=0x7fa01c0b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 --2- 172.21.15.37:0/881985901 >> v2:172.21.15.203:3300/0 conn(0x7fa01c0cbfb0 0x7fa01c0b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c141820 con 0x7fa01c0b3170 2024-04-25T08:52:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fa014001a00 con 0x7fa01c0b3170 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 >> v1:172.21.15.37:6789/0 conn(0x7fa01c0b3170 legacy=0x7fa01c15f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c140ac0 0x7fa0040039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa004003f40 0x7fa004004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 0x7fa0040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c0cc410 con 0x7fa004004d50 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c141540 con 0x7fa004003f40 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00400b9d0 con 0x7fa01c140ac0 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01bfff640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 0x7fa0040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01b7fe640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c140ac0 0x7fa0040039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01affd640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa004003f40 0x7fa004004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01bfff640 1 -- 172.21.15.37:0/881985901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c140ac0 msgr2=0x7fa0040039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01bfff640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c140ac0 0x7fa0040039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01bfff640 1 -- 172.21.15.37:0/881985901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa004003f40 msgr2=0x7fa004004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01bfff640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa004003f40 0x7fa004004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01bfff640 1 -- 172.21.15.37:0/881985901 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa004003530 con 0x7fa004004d50 2024-04-25T08:52:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01bfff640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 0x7fa0040051d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa01400a920 tx=0x7fa01401d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc87788651d97f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01400a4c0 con 0x7fa004004d50 2024-04-25T08:52:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0140072f0 con 0x7fa004004d50 2024-04-25T08:52:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.380+0000 7fa01a7fc640 1 -- 172.21.15.37:0/881985901 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa014007720 con 0x7fa004004d50 2024-04-25T08:52:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 -- 172.21.15.37:0/881985901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 msgr2=0x7fa0040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 --2- 172.21.15.37:0/881985901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 0x7fa0040051d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa01400a920 tx=0x7fa01401d3e0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01bfff640 1 -- 172.21.15.37:0/881985901 reap_dead start 2024-04-25T08:52:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 -- 172.21.15.37:0/881985901 shutdown_connections 2024-04-25T08:52:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 -- 172.21.15.37:0/881985901 >> 172.21.15.37:0/881985901 conn(0x7fa01c0cc680 msgr2=0x7fa01c0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 -- 172.21.15.37:0/881985901 shutdown_connections 2024-04-25T08:52:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 -- 172.21.15.37:0/881985901 wait complete. 2024-04-25T08:52:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 Processor -- start 2024-04-25T08:52:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 -- start start 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 0x7fa01c13f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa01c153ec0 0x7fa01c1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c152100 0x7fa01c152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c06ac90 con 0x7fa004004d50 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c141de0 con 0x7fa01c153ec0 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa01c06c1c0 con 0x7fa01c152100 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 0x7fa01c13f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 0x7fa01c13f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39122/0 (socket says 172.21.15.37:39122) 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01b7fe640 1 -- 172.21.15.37:0/254595072 learned_addr learned my addr 172.21.15.37:0/254595072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01affd640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa01c153ec0 0x7fa01c1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01bfff640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c152100 0x7fa01c152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01b7fe640 1 -- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c152100 msgr2=0x7fa01c152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01b7fe640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c152100 0x7fa01c152560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01b7fe640 1 -- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa01c153ec0 msgr2=0x7fa01c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:31.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01b7fe640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa01c153ec0 0x7fa01c1534c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:31.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01b7fe640 1 -- 172.21.15.37:0/254595072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c1510a0 con 0x7fa004004d50 2024-04-25T08:52:31.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa01b7fe640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 0x7fa01c13f710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa00c011610 tx=0x7fa00c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:31.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa003fff640 1 -- 172.21.15.37:0/254595072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00f8f0 con 0x7fa004004d50 2024-04-25T08:52:31.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa003fff640 1 -- 172.21.15.37:0/254595072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c00fa90 con 0x7fa004004d50 2024-04-25T08:52:31.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 -- 172.21.15.37:0/254595072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa01c151340 con 0x7fa004004d50 2024-04-25T08:52:31.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa003fff640 1 -- 172.21.15.37:0/254595072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00fe40 con 0x7fa004004d50 2024-04-25T08:52:31.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.384+0000 7fa021c9f640 1 -- 172.21.15.37:0/254595072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01c151900 con 0x7fa004004d50 2024-04-25T08:52:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.388+0000 7fa021c9f640 1 -- 172.21.15.37:0/254595072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01c068340 con 0x7fa004004d50 2024-04-25T08:52:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.388+0000 7fa003fff640 1 -- 172.21.15.37:0/254595072 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa00c015460 con 0x7fa004004d50 2024-04-25T08:52:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.388+0000 7fa003fff640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa008066f60 0x7fa008069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.388+0000 7fa003fff640 1 -- 172.21.15.37:0/254595072 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa00c085a40 con 0x7fa004004d50 2024-04-25T08:52:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.388+0000 7fa01affd640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa008066f60 0x7fa008069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:31.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.388+0000 7fa01affd640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa008066f60 0x7fa008069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa01c140830 tx=0x7fa010008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:31.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.392+0000 7fa003fff640 1 -- 172.21.15.37:0/254595072 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa00c019030 con 0x7fa004004d50 2024-04-25T08:52:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:31.560+0000 7fa021c9f640 1 -- 172.21.15.37:0/254595072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fa01c072650 con 0x7fa004004d50 2024-04-25T08:52:32.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.212+0000 7fa003fff640 1 -- 172.21.15.37:0/254595072 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7fa00c085e20 con 0x7fa004004d50 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 -- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa008066f60 msgr2=0x7fa008069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa008066f60 0x7fa008069420 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa01c140830 tx=0x7fa010008040 comp rx=0 tx=0).stop 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 -- 172.21.15.37:0/254595072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 msgr2=0x7fa01c13f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 0x7fa01c13f710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa00c011610 tx=0x7fa00c011ad0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 -- 172.21.15.37:0/254595072 shutdown_connections 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa008066f60 0x7fa008069420 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa01c152100 0x7fa01c152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa01c153ec0 0x7fa01c1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 --2- 172.21.15.37:0/254595072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa004004d50 0x7fa01c13f710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 -- 172.21.15.37:0/254595072 >> 172.21.15.37:0/254595072 conn(0x7fa01c0cc680 msgr2=0x7fa01c06da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 -- 172.21.15.37:0/254595072 shutdown_connections 2024-04-25T08:52:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.216+0000 7fa021c9f640 1 -- 172.21.15.37:0/254595072 wait complete. 2024-04-25T08:52:32.235 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:52:32.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.376+0000 7fa323f77640 1 Processor -- start 2024-04-25T08:52:32.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.376+0000 7fa323f77640 1 -- start start 2024-04-25T08:52:32.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa323f77640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa31c15da80 0x7fa31c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa323f77640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa31c0d1cd0 0x7fa31c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa323f77640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31c06be60 con 0x7fa31c0d0620 2024-04-25T08:52:32.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa323f77640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c0d0a70 con 0x7fa31c0d1cd0 2024-04-25T08:52:32.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa323f77640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c06c440 con 0x7fa31c15da80 2024-04-25T08:52:32.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa322514640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa31c0d0620 0x7fa31c0ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37126/0 (socket says 172.21.15.37:37126) 2024-04-25T08:52:32.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa322514640 1 -- 172.21.15.37:0/576763419 learned_addr learned my addr 172.21.15.37:0/576763419 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:32.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa321512640 1 --2- 172.21.15.37:0/576763419 >> v2:172.21.15.37:3300/0 conn(0x7fa31c0d1cd0 0x7fa31c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa321d13640 1 --2- 172.21.15.37:0/576763419 >> v2:172.21.15.203:3300/0 conn(0x7fa31c15da80 0x7fa31c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa320d11640 1 -- 172.21.15.37:0/576763419 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656345083 0 0) 0x7fa31c06be60 con 0x7fa31c0d0620 2024-04-25T08:52:32.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa320d11640 1 -- 172.21.15.37:0/576763419 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa308003530 con 0x7fa31c0d0620 2024-04-25T08:52:32.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa320d11640 1 -- 172.21.15.37:0/576763419 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1270245113 0 0) 0x7fa308003530 con 0x7fa31c0d0620 2024-04-25T08:52:32.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa321512640 1 -- 172.21.15.37:0/576763419 >> v2:172.21.15.203:3300/0 conn(0x7fa31c15da80 msgr2=0x7fa31c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:32.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa321512640 1 --2- 172.21.15.37:0/576763419 >> v2:172.21.15.203:3300/0 conn(0x7fa31c15da80 0x7fa31c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa321512640 1 -- 172.21.15.37:0/576763419 >> v1:172.21.15.37:6789/0 conn(0x7fa31c0d0620 legacy=0x7fa31c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa321512640 1 -- 172.21.15.37:0/576763419 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c15de60 con 0x7fa31c0d1cd0 2024-04-25T08:52:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa321512640 1 --2- 172.21.15.37:0/576763419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa31c0d1cd0 0x7fa31c0d00e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0097a0 tx=0x7fa30c01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34db7fea99e8b2cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa320d11640 1 -- 172.21.15.37:0/576763419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c0b0070 con 0x7fa31c0d1cd0 2024-04-25T08:52:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.380+0000 7fa320d11640 1 -- 172.21.15.37:0/576763419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c0ab400 con 0x7fa31c0d1cd0 2024-04-25T08:52:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa320d11640 1 -- 172.21.15.37:0/576763419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318001680 con 0x7fa31c0d1cd0 2024-04-25T08:52:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 -- 172.21.15.37:0/576763419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa31c0d1cd0 msgr2=0x7fa31c0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 --2- 172.21.15.37:0/576763419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa31c0d1cd0 0x7fa31c0d00e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0097a0 tx=0x7fa30c01c360 comp rx=0 tx=0).stop 2024-04-25T08:52:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 -- 172.21.15.37:0/576763419 shutdown_connections 2024-04-25T08:52:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 --2- 172.21.15.37:0/576763419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa31c0d1cd0 0x7fa31c0d00e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 --2- 172.21.15.37:0/576763419 >> v2:172.21.15.203:3300/0 conn(0x7fa31c15da80 0x7fa31c0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 -- 172.21.15.37:0/576763419 >> 172.21.15.37:0/576763419 conn(0x7fa31c08b000 msgr2=0x7fa31c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 -- 172.21.15.37:0/576763419 shutdown_connections 2024-04-25T08:52:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 -- 172.21.15.37:0/576763419 wait complete. 2024-04-25T08:52:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 Processor -- start 2024-04-25T08:52:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 -- start start 2024-04-25T08:52:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa31c0d0620 0x7fa31c156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c0d1cd0 0x7fa31c0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa31c15da80 0x7fa31c0b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c16a730 con 0x7fa31c15da80 2024-04-25T08:52:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c16a5b0 con 0x7fa31c0d1cd0 2024-04-25T08:52:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa323f77640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c16a8b0 con 0x7fa31c0d0620 2024-04-25T08:52:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa322514640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa31c15da80 0x7fa31c0b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa322514640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa31c15da80 0x7fa31c0b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39136/0 (socket says 172.21.15.37:39136) 2024-04-25T08:52:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa322514640 1 -- 172.21.15.37:0/1143735422 learned_addr learned my addr 172.21.15.37:0/1143735422 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa321512640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c0d1cd0 0x7fa31c0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa321d13640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa31c0d0620 0x7fa31c156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa322514640 1 -- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa31c0d0620 msgr2=0x7fa31c156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa322514640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa31c0d0620 0x7fa31c156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa322514640 1 -- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c0d1cd0 msgr2=0x7fa31c0b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa322514640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c0d1cd0 0x7fa31c0b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa322514640 1 -- 172.21.15.37:0/1143735422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c0b8770 con 0x7fa31c15da80 2024-04-25T08:52:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa322514640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa31c15da80 0x7fa31c0b4160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa31800c750 tx=0x7fa31800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa3127fc640 1 -- 172.21.15.37:0/1143735422 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318005ae0 con 0x7fa31c15da80 2024-04-25T08:52:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa3127fc640 1 -- 172.21.15.37:0/1143735422 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa318005ca0 con 0x7fa31c15da80 2024-04-25T08:52:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.384+0000 7fa3127fc640 1 -- 172.21.15.37:0/1143735422 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3180072b0 con 0x7fa31c15da80 2024-04-25T08:52:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.388+0000 7fa323f77640 1 -- 172.21.15.37:0/1143735422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c0b8a50 con 0x7fa31c15da80 2024-04-25T08:52:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.388+0000 7fa323f77640 1 -- 172.21.15.37:0/1143735422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c0b9010 con 0x7fa31c15da80 2024-04-25T08:52:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.388+0000 7fa323f77640 1 -- 172.21.15.37:0/1143735422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c04efa0 con 0x7fa31c15da80 2024-04-25T08:52:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.388+0000 7fa3127fc640 1 -- 172.21.15.37:0/1143735422 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa318007450 con 0x7fa31c15da80 2024-04-25T08:52:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.388+0000 7fa3127fc640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa300066f60 0x7fa300069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.388+0000 7fa3127fc640 1 -- 172.21.15.37:0/1143735422 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6007+0+0 (secure 0 0 0) 0x7fa318084f60 con 0x7fa31c15da80 2024-04-25T08:52:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.388+0000 7fa321d13640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa300066f60 0x7fa300069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.388+0000 7fa321d13640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa300066f60 0x7fa300069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa314007380 tx=0x7fa31400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.392+0000 7fa3127fc640 1 -- 172.21.15.37:0/1143735422 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa318010300 con 0x7fa31c15da80 2024-04-25T08:52:32.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:32.552+0000 7fa323f77640 1 -- 172.21.15.37:0/1143735422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa31c0b5e60 con 0x7fa31c15da80 2024-04-25T08:52:34.252 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T08:52:34.260+0000 7f841fbe2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:52:34.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.260+0000 7fa3127fc640 1 -- 172.21.15.37:0/1143735422 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7fa318055ed0 con 0x7fa31c15da80 2024-04-25T08:52:34.255 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data_ec' created 2024-04-25T08:52:34.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 -- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa300066f60 msgr2=0x7fa300069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa300066f60 0x7fa300069420 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa314007380 tx=0x7fa31400b040 comp rx=0 tx=0).stop 2024-04-25T08:52:34.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 -- 172.21.15.37:0/1143735422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa31c15da80 msgr2=0x7fa31c0b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa31c15da80 0x7fa31c0b4160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa31800c750 tx=0x7fa31800cc10 comp rx=0 tx=0).stop 2024-04-25T08:52:34.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 -- 172.21.15.37:0/1143735422 shutdown_connections 2024-04-25T08:52:34.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa300066f60 0x7fa300069420 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa31c15da80 0x7fa31c0b4160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa31c0d1cd0 0x7fa31c0b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 --2- 172.21.15.37:0/1143735422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa31c0d0620 0x7fa31c156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 -- 172.21.15.37:0/1143735422 >> 172.21.15.37:0/1143735422 conn(0x7fa31c08b000 msgr2=0x7fa31c0b0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:34.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 -- 172.21.15.37:0/1143735422 shutdown_connections 2024-04-25T08:52:34.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.264+0000 7fa323f77640 1 -- 172.21.15.37:0/1143735422 wait complete. 2024-04-25T08:52:34.287 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51aef50640 1 Processor -- start 2024-04-25T08:52:34.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51aef50640 1 -- start start 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51aef50640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51a80d12c0 0x7f51a80d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51aef50640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51a80d21f0 0x7f51a806cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51aef50640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a815dd60 con 0x7f51a815d980 2024-04-25T08:52:34.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51aef50640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a806c810 con 0x7f51a80d12c0 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51aef50640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a80d45e0 con 0x7f51a80d21f0 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51a7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51a80d12c0 0x7f51a80d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51a7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51a80d12c0 0x7f51a80d1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39148/0 (socket says 172.21.15.37:39148) 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51a7fff640 1 -- 172.21.15.37:0/3340890074 learned_addr learned my addr 172.21.15.37:0/3340890074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51ad4ed640 1 --2- 172.21.15.37:0/3340890074 >> v2:172.21.15.203:3300/0 conn(0x7f51a80d21f0 0x7f51a806cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51a7fff640 1 -- 172.21.15.37:0/3340890074 >> v2:172.21.15.203:3300/0 conn(0x7f51a80d21f0 msgr2=0x7f51a806cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51a7fff640 1 --2- 172.21.15.37:0/3340890074 >> v2:172.21.15.203:3300/0 conn(0x7f51a80d21f0 0x7f51a806cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51a7fff640 1 -- 172.21.15.37:0/3340890074 >> v1:172.21.15.203:6789/0 conn(0x7f51a815d980 legacy=0x7f51a80d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:34.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.416+0000 7f51a7fff640 1 -- 172.21.15.37:0/3340890074 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a80b02c0 con 0x7f51a80d12c0 2024-04-25T08:52:34.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51a7fff640 1 --2- 172.21.15.37:0/3340890074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a80d12c0 0x7f51a80d1cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f51a808d8a0 tx=0x7f519801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdd36856ada6f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51a77fe640 1 -- 172.21.15.37:0/3340890074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51980b0070 con 0x7f51a80d12c0 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51a77fe640 1 -- 172.21.15.37:0/3340890074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51980ab600 con 0x7f51a80d12c0 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51a77fe640 1 -- 172.21.15.37:0/3340890074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51980ab9d0 con 0x7f51a80d12c0 2024-04-25T08:52:34.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51aef50640 1 -- 172.21.15.37:0/3340890074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a80d12c0 msgr2=0x7f51a80d1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51aef50640 1 --2- 172.21.15.37:0/3340890074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a80d12c0 0x7f51a80d1cb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f51a808d8a0 tx=0x7f519801c450 comp rx=0 tx=0).stop 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51aef50640 1 -- 172.21.15.37:0/3340890074 shutdown_connections 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51aef50640 1 --2- 172.21.15.37:0/3340890074 >> v2:172.21.15.203:3300/0 conn(0x7f51a80d21f0 0x7f51a806cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51aef50640 1 --2- 172.21.15.37:0/3340890074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a80d12c0 0x7f51a80d1cb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51aef50640 1 -- 172.21.15.37:0/3340890074 >> 172.21.15.37:0/3340890074 conn(0x7f51a808b000 msgr2=0x7f51a80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51aef50640 1 -- 172.21.15.37:0/3340890074 shutdown_connections 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51aef50640 1 -- 172.21.15.37:0/3340890074 wait complete. 2024-04-25T08:52:34.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51aef50640 1 Processor -- start 2024-04-25T08:52:34.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.420+0000 7f51aef50640 1 -- start start 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51aef50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a80d12c0 0x7f51a8158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51aef50640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51a80d21f0 0x7f51a815ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51aef50640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a815d980 0x7f51a815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51aef50640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a806db70 con 0x7f51a80d12c0 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51aef50640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a806d9f0 con 0x7f51a815d980 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51aef50640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a806dcf0 con 0x7f51a80d21f0 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51accec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a80d12c0 0x7f51a8158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51accec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a80d12c0 0x7f51a8158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39160/0 (socket says 172.21.15.37:39160) 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51accec640 1 -- 172.21.15.37:0/1012108744 learned_addr learned my addr 172.21.15.37:0/1012108744 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51ad4ed640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a815d980 0x7f51a815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51accec640 1 -- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51a80d21f0 msgr2=0x7f51a815ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51a7fff640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51a80d21f0 0x7f51a815ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51accec640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51a80d21f0 0x7f51a815ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51accec640 1 -- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a815d980 msgr2=0x7f51a815b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51accec640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a815d980 0x7f51a815b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51accec640 1 -- 172.21.15.37:0/1012108744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a80ca8f0 con 0x7f51a80d12c0 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51accec640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a80d12c0 0x7f51a8158970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f519000b440 tx=0x7f519000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51a57fa640 1 -- 172.21.15.37:0/1012108744 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190015020 con 0x7f51a80d12c0 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51a57fa640 1 -- 172.21.15.37:0/1012108744 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5190011890 con 0x7f51a80d12c0 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51aef50640 1 -- 172.21.15.37:0/1012108744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a80cabd0 con 0x7f51a80d12c0 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51a57fa640 1 -- 172.21.15.37:0/1012108744 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190011c80 con 0x7f51a80d12c0 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51aef50640 1 -- 172.21.15.37:0/1012108744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a80caeb0 con 0x7f51a80d12c0 2024-04-25T08:52:34.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51aef50640 1 -- 172.21.15.37:0/1012108744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a8067c30 con 0x7f51a80d12c0 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51a57fa640 1 -- 172.21.15.37:0/1012108744 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5190018020 con 0x7f51a80d12c0 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51a57fa640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5188066f60 0x7f5188069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51a57fa640 1 -- 172.21.15.37:0/1012108744 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f5190085240 con 0x7f51a80d12c0 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.424+0000 7f51a7fff640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5188066f60 0x7f5188069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:34.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.428+0000 7f51a7fff640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5188066f60 0x7f5188069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51980ad000 tx=0x7f5198008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:34.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.432+0000 7f51a57fa640 1 -- 172.21.15.37:0/1012108744 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f519001f030 con 0x7f51a80d12c0 2024-04-25T08:52:34.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:34.584+0000 7f51aef50640 1 -- 172.21.15.37:0/1012108744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f51a815d170 con 0x7f51a80d12c0 2024-04-25T08:52:35.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.276+0000 7f51a57fa640 1 -- 172.21.15.37:0/1012108744 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f519000d040 con 0x7f51a80d12c0 2024-04-25T08:52:35.270 INFO:teuthology.orchestra.run.smithi037.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T08:52:35.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 -- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5188066f60 msgr2=0x7f5188069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:35.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5188066f60 0x7f5188069420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f51980ad000 tx=0x7f5198008040 comp rx=0 tx=0).stop 2024-04-25T08:52:35.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 -- 172.21.15.37:0/1012108744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a80d12c0 msgr2=0x7f51a8158970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a80d12c0 0x7f51a8158970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f519000b440 tx=0x7f519000b900 comp rx=0 tx=0).stop 2024-04-25T08:52:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 -- 172.21.15.37:0/1012108744 shutdown_connections 2024-04-25T08:52:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5188066f60 0x7f5188069420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51a815d980 0x7f51a815b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51a80d21f0 0x7f51a815ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 --2- 172.21.15.37:0/1012108744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a80d12c0 0x7f51a8158970 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 -- 172.21.15.37:0/1012108744 >> 172.21.15.37:0/1012108744 conn(0x7f51a808b000 msgr2=0x7f51a80b6a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 -- 172.21.15.37:0/1012108744 shutdown_connections 2024-04-25T08:52:35.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.280+0000 7f51aef50640 1 -- 172.21.15.37:0/1012108744 wait complete. 2024-04-25T08:52:35.308 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T08:52:35.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.440+0000 7f3d7b070640 1 Processor -- start 2024-04-25T08:52:35.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.440+0000 7f3d7b070640 1 -- start start 2024-04-25T08:52:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d7b070640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d740cf720 0x7f3d740cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d7b070640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3d740b2c60 0x7f3d7406e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d7b070640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d740d0040 con 0x7f3d740af520 2024-04-25T08:52:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d7b070640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d740cdd70 con 0x7f3d740b2c60 2024-04-25T08:52:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d7b070640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d740afa20 con 0x7f3d740cf720 2024-04-25T08:52:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d7960d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3d740af520 0x7f3d7406e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37140/0 (socket says 172.21.15.37:37140) 2024-04-25T08:52:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d7960d640 1 -- 172.21.15.37:0/2391879205 learned_addr learned my addr 172.21.15.37:0/2391879205 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:35.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d73fff640 1 --2- 172.21.15.37:0/2391879205 >> v2:172.21.15.37:3300/0 conn(0x7f3d740b2c60 0x7f3d7406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d78e0c640 1 --2- 172.21.15.37:0/2391879205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d740cf720 0x7f3d740cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d737fe640 1 -- 172.21.15.37:0/2391879205 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298187356 0 0) 0x7f3d740d0040 con 0x7f3d740af520 2024-04-25T08:52:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d737fe640 1 -- 172.21.15.37:0/2391879205 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d64003530 con 0x7f3d740af520 2024-04-25T08:52:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d73fff640 1 -- 172.21.15.37:0/2391879205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d740cf720 msgr2=0x7f3d740cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d73fff640 1 --2- 172.21.15.37:0/2391879205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d740cf720 0x7f3d740cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d73fff640 1 -- 172.21.15.37:0/2391879205 >> v1:172.21.15.37:6789/0 conn(0x7f3d740af520 legacy=0x7f3d7406e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:35.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d73fff640 1 -- 172.21.15.37:0/2391879205 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740ccfb0 con 0x7f3d740b2c60 2024-04-25T08:52:35.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d73fff640 1 --2- 172.21.15.37:0/2391879205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d740b2c60 0x7f3d7406e170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d600097f0 tx=0x7f3d6001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fc3d2df6398bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:35.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d737fe640 1 -- 172.21.15.37:0/2391879205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d600ab600 con 0x7f3d740b2c60 2024-04-25T08:52:35.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d737fe640 1 -- 172.21.15.37:0/2391879205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d600b0070 con 0x7f3d740b2c60 2024-04-25T08:52:35.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.444+0000 7f3d737fe640 1 -- 172.21.15.37:0/2391879205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d600ab9c0 con 0x7f3d740b2c60 2024-04-25T08:52:35.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 -- 172.21.15.37:0/2391879205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d740b2c60 msgr2=0x7f3d7406e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:35.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 --2- 172.21.15.37:0/2391879205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d740b2c60 0x7f3d7406e170 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3d600097f0 tx=0x7f3d6001c3e0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 -- 172.21.15.37:0/2391879205 shutdown_connections 2024-04-25T08:52:35.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 --2- 172.21.15.37:0/2391879205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d740b2c60 0x7f3d7406e170 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 --2- 172.21.15.37:0/2391879205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d740cf720 0x7f3d740cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 -- 172.21.15.37:0/2391879205 >> 172.21.15.37:0/2391879205 conn(0x7f3d74099a70 msgr2=0x7f3d740b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:35.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 -- 172.21.15.37:0/2391879205 shutdown_connections 2024-04-25T08:52:35.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 -- 172.21.15.37:0/2391879205 wait complete. 2024-04-25T08:52:35.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 Processor -- start 2024-04-25T08:52:35.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 -- start start 2024-04-25T08:52:35.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d740af520 0x7f3d74149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d740b2c60 0x7f3d740d0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d740cf720 0x7f3d740b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d741611e0 con 0x7f3d740b2c60 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d740caac0 con 0x7f3d740af520 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7406edc0 con 0x7f3d740cf720 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d73fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d740b2c60 0x7f3d740d0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d73fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d740b2c60 0x7f3d740d0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39184/0 (socket says 172.21.15.37:39184) 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d73fff640 1 -- 172.21.15.37:0/2268575264 learned_addr learned my addr 172.21.15.37:0/2268575264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d73fff640 1 -- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d740cf720 msgr2=0x7f3d740b1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d73fff640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d740cf720 0x7f3d740b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d73fff640 1 -- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d740af520 msgr2=0x7f3d74149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d73fff640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d740af520 0x7f3d74149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d73fff640 1 -- 172.21.15.37:0/2268575264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740ae7b0 con 0x7f3d740b2c60 2024-04-25T08:52:35.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d73fff640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d740b2c60 0x7f3d740d0250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d600ad040 tx=0x7f3d6001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:35.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d717fa640 1 -- 172.21.15.37:0/2268575264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d600bf410 con 0x7f3d740b2c60 2024-04-25T08:52:35.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d717fa640 1 -- 172.21.15.37:0/2268575264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d600bf5f0 con 0x7f3d740b2c60 2024-04-25T08:52:35.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d717fa640 1 -- 172.21.15.37:0/2268575264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d600bf9e0 con 0x7f3d740b2c60 2024-04-25T08:52:35.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 -- 172.21.15.37:0/2268575264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d740aea90 con 0x7f3d740b2c60 2024-04-25T08:52:35.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.448+0000 7f3d7b070640 1 -- 172.21.15.37:0/2268575264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d740af050 con 0x7f3d740b2c60 2024-04-25T08:52:35.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.452+0000 7f3d7b070640 1 -- 172.21.15.37:0/2268575264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7404efa0 con 0x7f3d740b2c60 2024-04-25T08:52:35.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.452+0000 7f3d717fa640 1 -- 172.21.15.37:0/2268575264 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d600bd020 con 0x7f3d740b2c60 2024-04-25T08:52:35.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.452+0000 7f3d717fa640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d58066f60 0x7f3d58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:35.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.452+0000 7f3d717fa640 1 -- 172.21.15.37:0/2268575264 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f3d6011f2d0 con 0x7f3d740b2c60 2024-04-25T08:52:35.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.452+0000 7f3d78e0c640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d58066f60 0x7f3d58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:35.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.452+0000 7f3d78e0c640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d58066f60 0x7f3d58069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3d68006aa0 tx=0x7f3d68006a30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:35.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.456+0000 7f3d717fa640 1 -- 172.21.15.37:0/2268575264 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d600c6070 con 0x7f3d740b2c60 2024-04-25T08:52:35.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:35.652+0000 7f3d7b070640 1 -- 172.21.15.37:0/2268575264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f3d740b2630 con 0x7f3d740b2c60 2024-04-25T08:52:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.276+0000 7f3d717fa640 1 -- 172.21.15.37:0/2268575264 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f3d60016090 con 0x7f3d740b2c60 2024-04-25T08:52:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:added data pool 4 to fsmap 2024-04-25T08:52:36.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 -- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d58066f60 msgr2=0x7f3d58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d58066f60 0x7f3d58069420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3d68006aa0 tx=0x7f3d68006a30 comp rx=0 tx=0).stop 2024-04-25T08:52:36.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 -- 172.21.15.37:0/2268575264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d740b2c60 msgr2=0x7f3d740d0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d740b2c60 0x7f3d740d0250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3d600ad040 tx=0x7f3d6001ac20 comp rx=0 tx=0).stop 2024-04-25T08:52:36.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 -- 172.21.15.37:0/2268575264 shutdown_connections 2024-04-25T08:52:36.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d58066f60 0x7f3d58069420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d740cf720 0x7f3d740b1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d740b2c60 0x7f3d740d0250 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 --2- 172.21.15.37:0/2268575264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d740af520 0x7f3d74149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 -- 172.21.15.37:0/2268575264 >> 172.21.15.37:0/2268575264 conn(0x7f3d74099a70 msgr2=0x7f3d74155940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 -- 172.21.15.37:0/2268575264 shutdown_connections 2024-04-25T08:52:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.280+0000 7f3d7b070640 1 -- 172.21.15.37:0/2268575264 wait complete. 2024-04-25T08:52:36.295 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.432+0000 7f1733efb640 1 Processor -- start 2024-04-25T08:52:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.432+0000 7f1733efb640 1 -- start start 2024-04-25T08:52:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1733efb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f172c0afb60 0x7f172c06f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1733efb640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172c0701b0 con 0x7f172c0cc730 2024-04-25T08:52:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1733efb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f172c06fbf0 con 0x7f172c06ac90 2024-04-25T08:52:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1733efb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172c06fed0 con 0x7f172c0afb60 2024-04-25T08:52:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1731496640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f172c0cc730 0x7f172c0cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37156/0 (socket says 172.21.15.37:37156) 2024-04-25T08:52:36.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1731496640 1 -- 172.21.15.37:0/63831113 learned_addr learned my addr 172.21.15.37:0/63831113 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1731c97640 1 --2- 172.21.15.37:0/63831113 >> v2:172.21.15.203:3300/0 conn(0x7f172c0afb60 0x7f172c06f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306803862 0 0) 0x7f172c0701b0 con 0x7f172c0cc730 2024-04-25T08:52:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1718003530 con 0x7f172c0cc730 2024-04-25T08:52:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1758957451 0 0) 0x7f1718003530 con 0x7f172c0cc730 2024-04-25T08:52:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 >> v2:172.21.15.203:3300/0 conn(0x7f172c0afb60 msgr2=0x7f172c06f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 --2- 172.21.15.37:0/63831113 >> v2:172.21.15.203:3300/0 conn(0x7f172c0afb60 0x7f172c06f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 >> v1:172.21.15.203:6789/0 conn(0x7f172c06ac90 legacy=0x7f172c06b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c0cd040 con 0x7f172c0cc730 2024-04-25T08:52:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1720001680 con 0x7f172c0cc730 2024-04-25T08:52:36.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 >> v1:172.21.15.37:6789/0 conn(0x7f172c0cc730 legacy=0x7f172c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 0x7f1718003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1718004810 0x7f1718004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1718005320 0x7f171800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c06fed0 con 0x7f1718003a80 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171800c260 con 0x7f1718005320 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171800c460 con 0x7f1718004810 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1731c97640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 0x7f1718003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1732498640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1718005320 0x7f171800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1731c97640 1 -- 172.21.15.37:0/63831113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1718004810 msgr2=0x7f1718004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1731496640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1718004810 0x7f1718004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1731c97640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1718004810 0x7f1718004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1731c97640 1 -- 172.21.15.37:0/63831113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1718005320 msgr2=0x7f171800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1731c97640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1718005320 0x7f171800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1731c97640 1 -- 172.21.15.37:0/63831113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1718003530 con 0x7f1718003a80 2024-04-25T08:52:36.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1732498640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1718005320 0x7f171800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:36.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1731496640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1718004810 0x7f1718004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:36.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.436+0000 7f1732498640 1 -- 172.21.15.37:0/63831113 reap_dead start 2024-04-25T08:52:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1731c97640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 0x7f1718003ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1728009ac0 tx=0x7f172801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692d4d9415ccdce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728009690 con 0x7f1718003a80 2024-04-25T08:52:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17280a6510 con 0x7f1718003a80 2024-04-25T08:52:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1730c95640 1 -- 172.21.15.37:0/63831113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17280a68e0 con 0x7f1718003a80 2024-04-25T08:52:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 -- 172.21.15.37:0/63831113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 msgr2=0x7f1718003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 0x7f1718003ea0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1728009ac0 tx=0x7f172801ca20 comp rx=0 tx=0).stop 2024-04-25T08:52:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 -- 172.21.15.37:0/63831113 shutdown_connections 2024-04-25T08:52:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 --2- 172.21.15.37:0/63831113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 0x7f1718003ea0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 -- 172.21.15.37:0/63831113 >> 172.21.15.37:0/63831113 conn(0x7f172c099a70 msgr2=0x7f172c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 -- 172.21.15.37:0/63831113 shutdown_connections 2024-04-25T08:52:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 -- 172.21.15.37:0/63831113 wait complete. 2024-04-25T08:52:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 Processor -- start 2024-04-25T08:52:36.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 -- start start 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 0x7f172c14db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f172c14e060 0x7f172c15f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f172c15f840 0x7f172c15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c15baf0 con 0x7f1718003a80 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172c15b970 con 0x7f172c15f840 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1733efb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172c06b070 con 0x7f172c14e060 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1731c97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 0x7f172c14db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1731c97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 0x7f172c14db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39198/0 (socket says 172.21.15.37:39198) 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1731c97640 1 -- 172.21.15.37:0/1941292307 learned_addr learned my addr 172.21.15.37:0/1941292307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1731c97640 1 -- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f172c14e060 msgr2=0x7f172c15f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1731c97640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f172c14e060 0x7f172c15f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1731c97640 1 -- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f172c15f840 msgr2=0x7f172c15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1732498640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f172c15f840 0x7f172c15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1731c97640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f172c15f840 0x7f172c15fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.440+0000 7f1731c97640 1 -- 172.21.15.37:0/1941292307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c160d70 con 0x7f1718003a80 2024-04-25T08:52:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f1731c97640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 0x7f172c14db20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17280b0040 tx=0x7f17280197c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f171e7fc640 1 -- 172.21.15.37:0/1941292307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172800f080 con 0x7f1718003a80 2024-04-25T08:52:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f171e7fc640 1 -- 172.21.15.37:0/1941292307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17280be070 con 0x7f1718003a80 2024-04-25T08:52:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f1733efb640 1 -- 172.21.15.37:0/1941292307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172c161030 con 0x7f1718003a80 2024-04-25T08:52:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f171e7fc640 1 -- 172.21.15.37:0/1941292307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17280a68e0 con 0x7f1718003a80 2024-04-25T08:52:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f1733efb640 1 -- 172.21.15.37:0/1941292307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172c0b3650 con 0x7f1718003a80 2024-04-25T08:52:36.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f1733efb640 1 -- 172.21.15.37:0/1941292307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172c14e460 con 0x7f1718003a80 2024-04-25T08:52:36.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f171e7fc640 1 -- 172.21.15.37:0/1941292307 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17280a6a80 con 0x7f1718003a80 2024-04-25T08:52:36.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f171e7fc640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1710066f60 0x7f1710069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f171e7fc640 1 -- 172.21.15.37:0/1941292307 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1728120430 con 0x7f1718003a80 2024-04-25T08:52:36.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f1731496640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1710066f60 0x7f1710069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.444+0000 7f1731496640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1710066f60 0x7f1710069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1720000c10 tx=0x7f1720009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:36.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.448+0000 7f171e7fc640 1 -- 172.21.15.37:0/1941292307 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1728016760 con 0x7f1718003a80 2024-04-25T08:52:36.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.652+0000 7f1733efb640 1 -- 172.21.15.37:0/1941292307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f172c04efa0 con 0x7f1718003a80 2024-04-25T08:52:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.652+0000 7f171e7fc640 1 -- 172.21.15.37:0/1941292307 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f17280f1170 con 0x7f1718003a80 2024-04-25T08:52:36.647 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:36.648 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2902565757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2902565757},{"type":"v1","addr":"172.21.15.203:6835","nonce":2902565757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/822172299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":822172299},{"type":"v1","addr":"172.21.15.203:6837","nonce":822172299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3681855890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3681855890},{"type":"v1","addr":"172.21.15.37:6833","nonce":3681855890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:30.564153+0000","modified":"2024-04-25T08:52:35.660106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T08:52:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 -- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1710066f60 msgr2=0x7f1710069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1710066f60 0x7f1710069420 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1720000c10 tx=0x7f1720009040 comp rx=0 tx=0).stop 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 -- 172.21.15.37:0/1941292307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 msgr2=0x7f172c14db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 0x7f172c14db20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f17280b0040 tx=0x7f17280197c0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 -- 172.21.15.37:0/1941292307 shutdown_connections 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1710066f60 0x7f1710069420 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f172c15f840 0x7f172c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f172c14e060 0x7f172c15f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 --2- 172.21.15.37:0/1941292307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1718003a80 0x7f172c14db20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 -- 172.21.15.37:0/1941292307 >> 172.21.15.37:0/1941292307 conn(0x7f172c099a70 msgr2=0x7f172c153c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 -- 172.21.15.37:0/1941292307 shutdown_connections 2024-04-25T08:52:36.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.656+0000 7f1733efb640 1 -- 172.21.15.37:0/1941292307 wait complete. 2024-04-25T08:52:36.667 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:36.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b81059640 1 Processor -- start 2024-04-25T08:52:36.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b81059640 1 -- start start 2024-04-25T08:52:36.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b81059640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5b7c0b2c30 0x7f5b7c0b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b81059640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c0cbec0 con 0x7f5b7c09a7b0 2024-04-25T08:52:36.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b81059640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c0cc1a0 con 0x7f5b7c09a180 2024-04-25T08:52:36.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b81059640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7c0cc480 con 0x7f5b7c0b2c30 2024-04-25T08:52:36.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b7ad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5b7c0b2c30 0x7f5b7c0b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b7a59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5b7c09a7b0 0x7f5b7c099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37162/0 (socket says 172.21.15.37:37162) 2024-04-25T08:52:36.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b7a59c640 1 -- 172.21.15.37:0/2359273640 learned_addr learned my addr 172.21.15.37:0/2359273640 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:36.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b7ad9d640 1 -- 172.21.15.37:0/2359273640 >> v1:172.21.15.203:6789/0 conn(0x7f5b7c09a180 legacy=0x7f5b7c0ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:36.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b7ad9d640 1 -- 172.21.15.37:0/2359273640 >> v1:172.21.15.37:6789/0 conn(0x7f5b7c09a7b0 legacy=0x7f5b7c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b79d9b640 1 -- 172.21.15.37:0/2359273640 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3635426077 0 0) 0x7f5b7c0cbec0 con 0x7f5b7c09a7b0 2024-04-25T08:52:36.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.800+0000 7f5b7ad9d640 1 -- 172.21.15.37:0/2359273640 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c0cc760 con 0x7f5b7c0b2c30 2024-04-25T08:52:36.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b7ad9d640 1 --2- 172.21.15.37:0/2359273640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b7c0b2c30 0x7f5b7c0b0ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0097c0 tx=0x7f5b6c01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7bacfce347e744 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:36.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b79d9b640 1 -- 172.21.15.37:0/2359273640 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0ab610 con 0x7f5b7c0b2c30 2024-04-25T08:52:36.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b79d9b640 1 -- 172.21.15.37:0/2359273640 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6c0b0070 con 0x7f5b7c0b2c30 2024-04-25T08:52:36.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b79d9b640 1 -- 172.21.15.37:0/2359273640 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0ab9d0 con 0x7f5b7c0b2c30 2024-04-25T08:52:36.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b81059640 1 -- 172.21.15.37:0/2359273640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b7c0b2c30 msgr2=0x7f5b7c0b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b81059640 1 --2- 172.21.15.37:0/2359273640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b7c0b2c30 0x7f5b7c0b0ba0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0097c0 tx=0x7f5b6c01c440 comp rx=0 tx=0).stop 2024-04-25T08:52:36.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b81059640 1 -- 172.21.15.37:0/2359273640 shutdown_connections 2024-04-25T08:52:36.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b81059640 1 --2- 172.21.15.37:0/2359273640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b7c0b2c30 0x7f5b7c0b0ba0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b81059640 1 -- 172.21.15.37:0/2359273640 >> 172.21.15.37:0/2359273640 conn(0x7f5b7c0b3010 msgr2=0x7f5b7c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:36.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b81059640 1 -- 172.21.15.37:0/2359273640 shutdown_connections 2024-04-25T08:52:36.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b81059640 1 -- 172.21.15.37:0/2359273640 wait complete. 2024-04-25T08:52:36.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b81059640 1 Processor -- start 2024-04-25T08:52:36.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.804+0000 7f5b81059640 1 -- start start 2024-04-25T08:52:36.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b81059640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b7c09a180 0x7f5b7c0b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b81059640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b7c09a7b0 0x7f5b7c0b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b81059640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c0b2c30 0x7f5b7c0b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b81059640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c06c380 con 0x7f5b7c09a180 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b81059640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c06c200 con 0x7f5b7c09a7b0 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b81059640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c15d190 con 0x7f5b7c0b2c30 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b7c09a180 0x7f5b7c0b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b7c09a180 0x7f5b7c0b59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39210/0 (socket says 172.21.15.37:39210) 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7ad9d640 1 -- 172.21.15.37:0/3455755556 learned_addr learned my addr 172.21.15.37:0/3455755556 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7a59c640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b7c09a7b0 0x7f5b7c0b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7b59e640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c0b2c30 0x7f5b7c0b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7ad9d640 1 -- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c0b2c30 msgr2=0x7f5b7c0b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7ad9d640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c0b2c30 0x7f5b7c0b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7ad9d640 1 -- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b7c09a7b0 msgr2=0x7f5b7c0b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7ad9d640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b7c09a7b0 0x7f5b7c0b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7ad9d640 1 -- 172.21.15.37:0/3455755556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c0b6320 con 0x7f5b7c09a180 2024-04-25T08:52:36.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7b59e640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c0b2c30 0x7f5b7c0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:36.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7ad9d640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b7c09a180 0x7f5b7c0b59b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c019050 tx=0x7f5b6c01ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:36.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b637fe640 1 -- 172.21.15.37:0/3455755556 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0bf610 con 0x7f5b7c09a180 2024-04-25T08:52:36.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b637fe640 1 -- 172.21.15.37:0/3455755556 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6c0bf7d0 con 0x7f5b7c09a180 2024-04-25T08:52:36.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b81059640 1 -- 172.21.15.37:0/3455755556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c0b6600 con 0x7f5b7c09a180 2024-04-25T08:52:36.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b637fe640 1 -- 172.21.15.37:0/3455755556 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6c0bfbc0 con 0x7f5b7c09a180 2024-04-25T08:52:36.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b81059640 1 -- 172.21.15.37:0/3455755556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c0b6bc0 con 0x7f5b7c09a180 2024-04-25T08:52:36.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b81059640 1 -- 172.21.15.37:0/3455755556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7c04efa0 con 0x7f5b7c09a180 2024-04-25T08:52:36.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b637fe640 1 -- 172.21.15.37:0/3455755556 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b6c0bd020 con 0x7f5b7c09a180 2024-04-25T08:52:36.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b637fe640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b68066f90 0x7f5b68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:36.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b637fe640 1 -- 172.21.15.37:0/3455755556 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5b6c11ff20 con 0x7f5b7c09a180 2024-04-25T08:52:36.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.808+0000 7f5b7a59c640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b68066f90 0x7f5b68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:36.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.812+0000 7f5b7a59c640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b68066f90 0x7f5b68069450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b70007fd0 tx=0x7f5b70007ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:36.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.812+0000 7f5b637fe640 1 -- 172.21.15.37:0/3455755556 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b6c0a8050 con 0x7f5b7c09a180 2024-04-25T08:52:36.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.972+0000 7f5b81059640 1 -- 172.21.15.37:0/3455755556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b7c06ba80 con 0x7f5b7c09a180 2024-04-25T08:52:36.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b637fe640 1 -- 172.21.15.37:0/3455755556 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18491 (secure 0 0 0) 0x7f5b6c016090 con 0x7f5b7c09a180 2024-04-25T08:52:36.969 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:36.969 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:35.659644+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 -- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b68066f90 msgr2=0x7f5b68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b68066f90 0x7f5b68069450 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5b70007fd0 tx=0x7f5b70007ba0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 -- 172.21.15.37:0/3455755556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b7c09a180 msgr2=0x7f5b7c0b59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b7c09a180 0x7f5b7c0b59b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c019050 tx=0x7f5b6c01ac80 comp rx=0 tx=0).stop 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 -- 172.21.15.37:0/3455755556 shutdown_connections 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b68066f90 0x7f5b68069450 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b7c0b2c30 0x7f5b7c0b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b7c09a7b0 0x7f5b7c0b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 --2- 172.21.15.37:0/3455755556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b7c09a180 0x7f5b7c0b59b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:36.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 -- 172.21.15.37:0/3455755556 >> 172.21.15.37:0/3455755556 conn(0x7f5b7c0b3010 msgr2=0x7f5b7c0d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 -- 172.21.15.37:0/3455755556 shutdown_connections 2024-04-25T08:52:36.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:36.976+0000 7f5b81059640 1 -- 172.21.15.37:0/3455755556 wait complete. 2024-04-25T08:52:36.989 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:37.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3ca2a4e640 1 Processor -- start 2024-04-25T08:52:37.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3ca2a4e640 1 -- start start 2024-04-25T08:52:37.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3ca2a4e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3c9c0d1d70 0x7f3c9c0d21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3ca2a4e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9c06d180 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3ca2a4e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9c06d460 con 0x7f3c9c15db30 2024-04-25T08:52:37.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3ca2a4e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c9c0d1320 con 0x7f3c9c0d1d70 2024-04-25T08:52:37.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3ca0feb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3c9c0d1d70 0x7f3c9c0d21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3c9b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3c9c0d34a0 0x7f3c9c0d3910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37176/0 (socket says 172.21.15.37:37176) 2024-04-25T08:52:37.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3c9b7fe640 1 -- 172.21.15.37:0/3078466445 learned_addr learned my addr 172.21.15.37:0/3078466445 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:37.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3ca0feb640 1 -- 172.21.15.37:0/3078466445 >> v1:172.21.15.203:6789/0 conn(0x7f3c9c15db30 legacy=0x7f3c9c0ce360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:37.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3ca0feb640 1 -- 172.21.15.37:0/3078466445 >> v1:172.21.15.37:6789/0 conn(0x7f3c9c0d34a0 legacy=0x7f3c9c0d3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.120+0000 7f3ca0feb640 1 -- 172.21.15.37:0/3078466445 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c0d1570 con 0x7f3c9c0d1d70 2024-04-25T08:52:37.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca0feb640 1 --2- 172.21.15.37:0/3078466445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c0d1d70 0x7f3c9c0d21e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009d90 tx=0x7f3c9401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a19f94b1615362b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3c9affd640 1 -- 172.21.15.37:0/3078466445 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c940b2070 con 0x7f3c9c0d1d70 2024-04-25T08:52:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3c9affd640 1 -- 172.21.15.37:0/3078466445 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c940a5870 con 0x7f3c9c0d1d70 2024-04-25T08:52:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3c9affd640 1 -- 172.21.15.37:0/3078466445 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c940a5c40 con 0x7f3c9c0d1d70 2024-04-25T08:52:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3078466445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c0d1d70 msgr2=0x7f3c9c0d21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 --2- 172.21.15.37:0/3078466445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c0d1d70 0x7f3c9c0d21e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009d90 tx=0x7f3c9401c8a0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3078466445 shutdown_connections 2024-04-25T08:52:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 --2- 172.21.15.37:0/3078466445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c0d1d70 0x7f3c9c0d21e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3078466445 >> 172.21.15.37:0/3078466445 conn(0x7f3c9c09cad0 msgr2=0x7f3c9c08b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3078466445 shutdown_connections 2024-04-25T08:52:37.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3078466445 wait complete. 2024-04-25T08:52:37.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 Processor -- start 2024-04-25T08:52:37.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 -- start start 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c0d1d70 0x7f3c9c153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c0d34a0 0x7f3c9c0b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c9c15db30 0x7f3c9c0b8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c16b880 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c06ce90 con 0x7f3c9c0d1d70 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.124+0000 7f3ca2a4e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c16ba00 con 0x7f3c9c15db30 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c0d34a0 0x7f3c9c0b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c0d34a0 0x7f3c9c0b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39224/0 (socket says 172.21.15.37:39224) 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9b7fe640 1 -- 172.21.15.37:0/3809652248 learned_addr learned my addr 172.21.15.37:0/3809652248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9bfff640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c0d1d70 0x7f3c9c153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3ca0feb640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c9c15db30 0x7f3c9c0b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9b7fe640 1 -- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c9c15db30 msgr2=0x7f3c9c0b8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9b7fe640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c9c15db30 0x7f3c9c0b8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9b7fe640 1 -- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c0d1d70 msgr2=0x7f3c9c153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9b7fe640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c0d1d70 0x7f3c9c153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9b7fe640 1 -- 172.21.15.37:0/3809652248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c154440 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9bfff640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c0d1d70 0x7f3c9c153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3ca0feb640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c9c15db30 0x7f3c9c0b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9b7fe640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c0d34a0 0x7f3c9c0b6310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000c410 tx=0x7f3c9000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c98ff9640 1 -- 172.21.15.37:0/3809652248 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90017070 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c98ff9640 1 -- 172.21.15.37:0/3809652248 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c90005d10 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3809652248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9c0b8f90 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c98ff9640 1 -- 172.21.15.37:0/3809652248 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90007310 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3809652248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9c0cab50 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3809652248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c9c04efa0 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c98ff9640 1 -- 172.21.15.37:0/3809652248 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3c9001a070 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c98ff9640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3c88066f60 0x7f3c88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c98ff9640 1 -- 172.21.15.37:0/3809652248 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3c90084760 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.128+0000 7f3c9bfff640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3c88066f60 0x7f3c88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.132+0000 7f3c9bfff640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3c88066f60 0x7f3c88069420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0b24c0 tx=0x7f3c8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.132+0000 7f3c98ff9640 1 -- 172.21.15.37:0/3809652248 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3c90055430 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.304+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3809652248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c9c06bf90 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.304+0000 7f3c98ff9640 1 -- 172.21.15.37:0/3809652248 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f3c90054d30 con 0x7f3c9c0d34a0 2024-04-25T08:52:37.298 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:37.298 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:37.270949+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:37.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3c88066f60 msgr2=0x7f3c88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3c88066f60 0x7f3c88069420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3c9c0b24c0 tx=0x7f3c8c009040 comp rx=0 tx=0).stop 2024-04-25T08:52:37.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3809652248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c0d34a0 msgr2=0x7f3c9c0b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c0d34a0 0x7f3c9c0b6310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c9000c410 tx=0x7f3c9000c8d0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3809652248 shutdown_connections 2024-04-25T08:52:37.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3c88066f60 0x7f3c88069420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3c9c15db30 0x7f3c9c0b8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c0d34a0 0x7f3c9c0b6310 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 --2- 172.21.15.37:0/3809652248 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3c9c0d1d70 0x7f3c9c153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3809652248 >> 172.21.15.37:0/3809652248 conn(0x7f3c9c09cad0 msgr2=0x7f3c9c15b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:37.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3809652248 shutdown_connections 2024-04-25T08:52:37.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.308+0000 7f3ca2a4e640 1 -- 172.21.15.37:0/3809652248 wait complete. 2024-04-25T08:52:37.331 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:37.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadfdc2e640 1 Processor -- start 2024-04-25T08:52:37.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadfdc2e640 1 -- start start 2024-04-25T08:52:37.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadfdc2e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf8144ca0 0x7fadf806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadfdc2e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf80ccc70 con 0x7fadf80ce510 2024-04-25T08:52:37.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadfdc2e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf806dc40 con 0x7fadf806ec60 2024-04-25T08:52:37.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadfdc2e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80cc2b0 con 0x7fadf8144ca0 2024-04-25T08:52:37.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf77fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fadf80ce510 0x7fadf80ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37184/0 (socket says 172.21.15.37:37184) 2024-04-25T08:52:37.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf77fe640 1 -- 172.21.15.37:0/3598002917 learned_addr learned my addr 172.21.15.37:0/3598002917 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:37.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadeffff640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf8144ca0 0x7fadf806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1317733690 0 0) 0x7fadf80ccc70 con 0x7fadf80ce510 2024-04-25T08:52:37.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade4003530 con 0x7fadf80ce510 2024-04-25T08:52:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3731416726 0 0) 0x7fade4003530 con 0x7fadf80ce510 2024-04-25T08:52:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf8144ca0 msgr2=0x7fadf806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf8144ca0 0x7fadf806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 >> v1:172.21.15.203:6789/0 conn(0x7fadf806ec60 legacy=0x7fadf8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf806df20 con 0x7fadf80ce510 2024-04-25T08:52:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fade8001680 con 0x7fadf80ce510 2024-04-25T08:52:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 >> v1:172.21.15.37:6789/0 conn(0x7fadf80ce510 legacy=0x7fadf80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf8144ca0 0x7fade4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade4003fe0 0x7fade4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fade4004d50 0x7fade40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf80cc2b0 con 0x7fade4003fe0 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade400bbc0 con 0x7fade4004d50 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.464+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade400bdc0 con 0x7fadf8144ca0 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadeffff640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade4003fe0 0x7fade4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadf77fe640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf8144ca0 0x7fade4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadf7fff640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fade4004d50 0x7fade40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadeffff640 1 -- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf8144ca0 msgr2=0x7fade4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadeffff640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf8144ca0 0x7fade4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadeffff640 1 -- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fade4004d50 msgr2=0x7fade40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadeffff640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fade4004d50 0x7fade40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadeffff640 1 -- 172.21.15.37:0/3598002917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4003530 con 0x7fade4003fe0 2024-04-25T08:52:37.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadf77fe640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fadf8144ca0 0x7fade4003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:37.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadf7fff640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fade4004d50 0x7fade40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:37.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadeffff640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade4003fe0 0x7fade4004810 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fade0009ab0 tx=0x7fade001c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a24832ebdeb56d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade00b2070 con 0x7fade4003fe0 2024-04-25T08:52:37.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade00a5580 con 0x7fade4003fe0 2024-04-25T08:52:37.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadf6ffd640 1 -- 172.21.15.37:0/3598002917 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade00096b0 con 0x7fade4003fe0 2024-04-25T08:52:37.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadfdc2e640 1 -- 172.21.15.37:0/3598002917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade4003fe0 msgr2=0x7fade4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadfdc2e640 1 --2- 172.21.15.37:0/3598002917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fade4003fe0 0x7fade4004810 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fade0009ab0 tx=0x7fade001c990 comp rx=0 tx=0).stop 2024-04-25T08:52:37.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadf7fff640 1 -- 172.21.15.37:0/3598002917 reap_dead start 2024-04-25T08:52:37.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadfdc2e640 1 -- 172.21.15.37:0/3598002917 shutdown_connections 2024-04-25T08:52:37.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadfdc2e640 1 -- 172.21.15.37:0/3598002917 >> 172.21.15.37:0/3598002917 conn(0x7fadf80b11a0 msgr2=0x7fadf80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:37.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadfdc2e640 1 -- 172.21.15.37:0/3598002917 shutdown_connections 2024-04-25T08:52:37.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.468+0000 7fadfdc2e640 1 -- 172.21.15.37:0/3598002917 wait complete. 2024-04-25T08:52:37.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadfdc2e640 1 Processor -- start 2024-04-25T08:52:37.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadfdc2e640 1 -- start start 2024-04-25T08:52:37.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadfdc2e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fade4003fe0 0x7fadf8138a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadfdc2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadf8137b90 0x7fadf815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadfdc2e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf81f1a10 0x7fadf8139d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadfdc2e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf8142d90 con 0x7fadf81f1a10 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadfdc2e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf806e290 con 0x7fadf8137b90 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadfdc2e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf8142f10 con 0x7fade4003fe0 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf81f1a10 0x7fadf8139d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf81f1a10 0x7fadf8139d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39246/0 (socket says 172.21.15.37:39246) 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf7fff640 1 -- 172.21.15.37:0/2199827622 learned_addr learned my addr 172.21.15.37:0/2199827622 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf77fe640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fade4003fe0 0x7fadf8138a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadeffff640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadf8137b90 0x7fadf815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf7fff640 1 -- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fade4003fe0 msgr2=0x7fadf8138a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf7fff640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fade4003fe0 0x7fadf8138a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf7fff640 1 -- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadf8137b90 msgr2=0x7fadf815ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf7fff640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadf8137b90 0x7fadf815ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf7fff640 1 -- 172.21.15.37:0/2199827622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf813bf10 con 0x7fadf81f1a10 2024-04-25T08:52:37.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf7fff640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf81f1a10 0x7fadf8139d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fadf804f950 tx=0x7fadf000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf4ff9640 1 -- 172.21.15.37:0/2199827622 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0006d60 con 0x7fadf81f1a10 2024-04-25T08:52:37.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf4ff9640 1 -- 172.21.15.37:0/2199827622 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0003980 con 0x7fadf81f1a10 2024-04-25T08:52:37.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf4ff9640 1 -- 172.21.15.37:0/2199827622 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0003db0 con 0x7fadf81f1a10 2024-04-25T08:52:37.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadfdc2e640 1 -- 172.21.15.37:0/2199827622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf813c1f0 con 0x7fadf81f1a10 2024-04-25T08:52:37.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadfdc2e640 1 -- 172.21.15.37:0/2199827622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf813c7b0 con 0x7fadf81f1a10 2024-04-25T08:52:37.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadfdc2e640 1 -- 172.21.15.37:0/2199827622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf804efa0 con 0x7fadf81f1a10 2024-04-25T08:52:37.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.472+0000 7fadf4ff9640 1 -- 172.21.15.37:0/2199827622 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fadf001b020 con 0x7fadf81f1a10 2024-04-25T08:52:37.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.476+0000 7fadf4ff9640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadd8066f60 0x7fadd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.476+0000 7fadf4ff9640 1 -- 172.21.15.37:0/2199827622 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fadf0084e20 con 0x7fadf81f1a10 2024-04-25T08:52:37.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.476+0000 7fadf77fe640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadd8066f60 0x7fadd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.476+0000 7fadf77fe640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadd8066f60 0x7fadd8069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fade80017f0 tx=0x7fade8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.480+0000 7fadf4ff9640 1 -- 172.21.15.37:0/2199827622 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fadf0055b80 con 0x7fadf81f1a10 2024-04-25T08:52:37.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.676+0000 7fadfdc2e640 1 -- 172.21.15.37:0/2199827622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadf813c980 con 0x7fadf81f1a10 2024-04-25T08:52:37.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.676+0000 7fadf4ff9640 1 -- 172.21.15.37:0/2199827622 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4937 (secure 0 0 0) 0x7fadf0055480 con 0x7fadf81f1a10 2024-04-25T08:52:37.670 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:37.670 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2902565757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2902565757},{"type":"v1","addr":"172.21.15.203:6835","nonce":2902565757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/822172299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":822172299},{"type":"v1","addr":"172.21.15.203:6837","nonce":822172299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3681855890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3681855890},{"type":"v1","addr":"172.21.15.37:6833","nonce":3681855890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:30.564153+0000","modified":"2024-04-25T08:52:35.660106+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T08:52:37.670 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-25T08:52:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 -- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadd8066f60 msgr2=0x7fadd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadd8066f60 0x7fadd8069420 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fade80017f0 tx=0x7fade8009040 comp rx=0 tx=0).stop 2024-04-25T08:52:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 -- 172.21.15.37:0/2199827622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf81f1a10 msgr2=0x7fadf8139d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf81f1a10 0x7fadf8139d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fadf804f950 tx=0x7fadf000dc00 comp rx=0 tx=0).stop 2024-04-25T08:52:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 -- 172.21.15.37:0/2199827622 shutdown_connections 2024-04-25T08:52:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadd8066f60 0x7fadd8069420 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fadf81f1a10 0x7fadf8139d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fadf8137b90 0x7fadf815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 --2- 172.21.15.37:0/2199827622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fade4003fe0 0x7fadf8138a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 -- 172.21.15.37:0/2199827622 >> 172.21.15.37:0/2199827622 conn(0x7fadf80b11a0 msgr2=0x7fadf80c4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:37.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 -- 172.21.15.37:0/2199827622 shutdown_connections 2024-04-25T08:52:37.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.680+0000 7fadfdc2e640 1 -- 172.21.15.37:0/2199827622 wait complete. 2024-04-25T08:52:37.690 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:37.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39c17ac640 1 Processor -- start 2024-04-25T08:52:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39c17ac640 1 -- start start 2024-04-25T08:52:37.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39c17ac640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f39bc071030 0x7f39bc161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39c17ac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39bc071e90 0x7f39bc0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39c17ac640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39bc0b29e0 con 0x7f39bc071410 2024-04-25T08:52:37.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39c17ac640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39bc162680 con 0x7f39bc071030 2024-04-25T08:52:37.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39c17ac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc162960 con 0x7f39bc071e90 2024-04-25T08:52:37.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39baffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f39bc071030 0x7f39bc161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39baffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f39bc071030 0x7f39bc161fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39262/0 (socket says 172.21.15.37:39262) 2024-04-25T08:52:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39baffd640 1 -- 172.21.15.37:0/3836725989 learned_addr learned my addr 172.21.15.37:0/3836725989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39ba7fc640 1 --2- 172.21.15.37:0/3836725989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39bc071e90 0x7f39bc0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39baffd640 1 -- 172.21.15.37:0/3836725989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39bc071e90 msgr2=0x7f39bc0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39baffd640 1 --2- 172.21.15.37:0/3836725989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39bc071e90 0x7f39bc0b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39baffd640 1 -- 172.21.15.37:0/3836725989 >> v1:172.21.15.203:6789/0 conn(0x7f39bc071410 legacy=0x7f39bc165920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39baffd640 1 -- 172.21.15.37:0/3836725989 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc162c40 con 0x7f39bc071030 2024-04-25T08:52:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.816+0000 7f39ba7fc640 1 --2- 172.21.15.37:0/3836725989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39bc071e90 0x7f39bc0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:37.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.820+0000 7f39baffd640 1 --2- 172.21.15.37:0/3836725989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39bc071030 0x7f39bc161fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009870 tx=0x7f39ac01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9418fadf979b468f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.820+0000 7f39b9ffb640 1 -- 172.21.15.37:0/3836725989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0b1070 con 0x7f39bc071030 2024-04-25T08:52:37.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.820+0000 7f39b9ffb640 1 -- 172.21.15.37:0/3836725989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac0a8050 con 0x7f39bc071030 2024-04-25T08:52:37.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.820+0000 7f39b9ffb640 1 -- 172.21.15.37:0/3836725989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0ac860 con 0x7f39bc071030 2024-04-25T08:52:37.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 -- 172.21.15.37:0/3836725989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39bc071030 msgr2=0x7f39bc161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 --2- 172.21.15.37:0/3836725989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39bc071030 0x7f39bc161fb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009870 tx=0x7f39ac01c4d0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 -- 172.21.15.37:0/3836725989 shutdown_connections 2024-04-25T08:52:37.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 --2- 172.21.15.37:0/3836725989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39bc071e90 0x7f39bc0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 --2- 172.21.15.37:0/3836725989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39bc071030 0x7f39bc161fb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 -- 172.21.15.37:0/3836725989 >> 172.21.15.37:0/3836725989 conn(0x7f39bc0b11a0 msgr2=0x7f39bc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:37.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 -- 172.21.15.37:0/3836725989 shutdown_connections 2024-04-25T08:52:37.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 -- 172.21.15.37:0/3836725989 wait complete. 2024-04-25T08:52:37.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 Processor -- start 2024-04-25T08:52:37.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 -- start start 2024-04-25T08:52:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39bc071030 0x7f39bc14a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39bc071410 0x7f39bc150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39bc071e90 0x7f39bc154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc16f050 con 0x7f39bc071e90 2024-04-25T08:52:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc16eed0 con 0x7f39bc071030 2024-04-25T08:52:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc16f1d0 con 0x7f39bc071410 2024-04-25T08:52:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39bb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39bc071e90 0x7f39bc154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39bb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39bc071e90 0x7f39bc154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39266/0 (socket says 172.21.15.37:39266) 2024-04-25T08:52:37.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39bb7fe640 1 -- 172.21.15.37:0/3872267899 learned_addr learned my addr 172.21.15.37:0/3872267899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:37.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39ba7fc640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39bc071410 0x7f39bc150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39baffd640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39bc071030 0x7f39bc14a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39bb7fe640 1 -- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39bc071410 msgr2=0x7f39bc150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39bb7fe640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39bc071410 0x7f39bc150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39bb7fe640 1 -- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39bc071030 msgr2=0x7f39bc14a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39bb7fe640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39bc071030 0x7f39bc14a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39bb7fe640 1 -- 172.21.15.37:0/3872267899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc156d00 con 0x7f39bc071e90 2024-04-25T08:52:37.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39bb7fe640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39bc071e90 0x7f39bc154f30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f39b400c390 tx=0x7f39b400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39b37fe640 1 -- 172.21.15.37:0/3872267899 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b40052d0 con 0x7f39bc071e90 2024-04-25T08:52:37.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39b37fe640 1 -- 172.21.15.37:0/3872267899 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39b4005470 con 0x7f39bc071e90 2024-04-25T08:52:37.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 -- 172.21.15.37:0/3872267899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc156fa0 con 0x7f39bc071e90 2024-04-25T08:52:37.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39b37fe640 1 -- 172.21.15.37:0/3872267899 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39b4007240 con 0x7f39bc071e90 2024-04-25T08:52:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.824+0000 7f39c17ac640 1 -- 172.21.15.37:0/3872267899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc157560 con 0x7f39bc071e90 2024-04-25T08:52:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.828+0000 7f39c17ac640 1 -- 172.21.15.37:0/3872267899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc04efa0 con 0x7f39bc071e90 2024-04-25T08:52:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.828+0000 7f39b37fe640 1 -- 172.21.15.37:0/3872267899 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f39b40073e0 con 0x7f39bc071e90 2024-04-25T08:52:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.828+0000 7f39b37fe640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f399c066f60 0x7f399c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.828+0000 7f39b37fe640 1 -- 172.21.15.37:0/3872267899 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f39b40848b0 con 0x7f39bc071e90 2024-04-25T08:52:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.828+0000 7f39baffd640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f399c066f60 0x7f399c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.828+0000 7f39baffd640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f399c066f60 0x7f399c069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0ae000 tx=0x7f39ac0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:37.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.832+0000 7f39b37fe640 1 -- 172.21.15.37:0/3872267899 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f39b40554f0 con 0x7f39bc071e90 2024-04-25T08:52:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.992+0000 7f39c17ac640 1 -- 172.21.15.37:0/3872267899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39bc161770 con 0x7f39bc071e90 2024-04-25T08:52:37.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.992+0000 7f39b37fe640 1 -- 172.21.15.37:0/3872267899 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f39b4082e20 con 0x7f39bc071e90 2024-04-25T08:52:37.986 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:37.986 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":26,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:37.270949+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:37.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.992+0000 7f39c17ac640 1 -- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f399c066f60 msgr2=0x7f399c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.992+0000 7f39c17ac640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f399c066f60 0x7f399c069420 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0ae000 tx=0x7f39ac0a7000 comp rx=0 tx=0).stop 2024-04-25T08:52:37.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.992+0000 7f39c17ac640 1 -- 172.21.15.37:0/3872267899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39bc071e90 msgr2=0x7f39bc154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:37.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.992+0000 7f39c17ac640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39bc071e90 0x7f39bc154f30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f39b400c390 tx=0x7f39b400c850 comp rx=0 tx=0).stop 2024-04-25T08:52:37.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.996+0000 7f39c17ac640 1 -- 172.21.15.37:0/3872267899 shutdown_connections 2024-04-25T08:52:37.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.996+0000 7f39c17ac640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f399c066f60 0x7f399c069420 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.996+0000 7f39c17ac640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f39bc071e90 0x7f39bc154f30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.996+0000 7f39c17ac640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f39bc071410 0x7f39bc150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.996+0000 7f39c17ac640 1 --2- 172.21.15.37:0/3872267899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f39bc071030 0x7f39bc14a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.996+0000 7f39c17ac640 1 -- 172.21.15.37:0/3872267899 >> 172.21.15.37:0/3872267899 conn(0x7f39bc0b11a0 msgr2=0x7f39bc164160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.996+0000 7f39c17ac640 1 -- 172.21.15.37:0/3872267899 shutdown_connections 2024-04-25T08:52:37.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:37.996+0000 7f39c17ac640 1 -- 172.21.15.37:0/3872267899 wait complete. 2024-04-25T08:52:38.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:52:30.564153+0000', 'modified': '2024-04-25T08:52:35.660106+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4251}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4251': {'gid': 4251, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/1246598982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 1246598982}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1246598982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T08:52:38.410 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:52:38.410 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:52:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.552+0000 7f7285f42640 1 Processor -- start 2024-04-25T08:52:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.552+0000 7f7285f42640 1 -- start start 2024-04-25T08:52:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.552+0000 7f7285f42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72801451f0 0x7f7280142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.552+0000 7f7285f42640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72800ccc70 con 0x7f72800ce510 2024-04-25T08:52:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.552+0000 7f7285f42640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728006dc40 con 0x7f7280144ca0 2024-04-25T08:52:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.552+0000 7f7285f42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72800cc2b0 con 0x7f72801451f0 2024-04-25T08:52:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.552+0000 7f727f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f72800ce510 0x7f72800ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37200/0 (socket says 172.21.15.37:37200) 2024-04-25T08:52:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.552+0000 7f727f7fe640 1 -- 172.21.15.37:0/3827757365 learned_addr learned my addr 172.21.15.37:0/3827757365 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.552+0000 7f727ffff640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72801451f0 0x7f7280142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.552+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836371042 0 0) 0x7f72800ccc70 con 0x7f72800ce510 2024-04-25T08:52:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.552+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f726c003530 con 0x7f72800ce510 2024-04-25T08:52:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 178799058 0 0) 0x7f726c003530 con 0x7f72800ce510 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72801451f0 msgr2=0x7f7280142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72801451f0 0x7f7280142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 >> v1:172.21.15.203:6789/0 conn(0x7f7280144ca0 legacy=0x7f728006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728006df20 con 0x7f72800ce510 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f7270001680 con 0x7f72800ce510 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 >> v1:172.21.15.37:6789/0 conn(0x7f72800ce510 legacy=0x7f72800ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f726c003a60 0x7f726c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726c004810 0x7f726c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72801451f0 0x7f726c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72800cc2b0 con 0x7f726c004810 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c00bc50 con 0x7f726c003a60 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c00be50 con 0x7f72801451f0 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7276ffd640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726c004810 0x7f726c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727f7fe640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f726c003a60 0x7f726c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727ffff640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72801451f0 0x7f726c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7276ffd640 1 -- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72801451f0 msgr2=0x7f726c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7276ffd640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72801451f0 0x7f726c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7276ffd640 1 -- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f726c003a60 msgr2=0x7f726c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7276ffd640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f726c003a60 0x7f726c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7276ffd640 1 -- 172.21.15.37:0/3827757365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c003530 con 0x7f726c004810 2024-04-25T08:52:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727ffff640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72801451f0 0x7f726c005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7276ffd640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726c004810 0x7f726c004c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7268001750 tx=0x7f726801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13dc97339f4c6d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72680013a0 con 0x7f726c004810 2024-04-25T08:52:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72680052f0 con 0x7f726c004810 2024-04-25T08:52:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727effd640 1 -- 172.21.15.37:0/3827757365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268003840 con 0x7f726c004810 2024-04-25T08:52:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7285f42640 1 -- 172.21.15.37:0/3827757365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726c004810 msgr2=0x7f726c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7285f42640 1 --2- 172.21.15.37:0/3827757365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f726c004810 0x7f726c004c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7268001750 tx=0x7f726801c3e0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f727ffff640 1 -- 172.21.15.37:0/3827757365 reap_dead start 2024-04-25T08:52:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7285f42640 1 -- 172.21.15.37:0/3827757365 shutdown_connections 2024-04-25T08:52:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7285f42640 1 -- 172.21.15.37:0/3827757365 >> 172.21.15.37:0/3827757365 conn(0x7f72800b11a0 msgr2=0x7f72800b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7285f42640 1 -- 172.21.15.37:0/3827757365 shutdown_connections 2024-04-25T08:52:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.556+0000 7f7285f42640 1 -- 172.21.15.37:0/3827757365 wait complete. 2024-04-25T08:52:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7285f42640 1 Processor -- start 2024-04-25T08:52:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7285f42640 1 -- start start 2024-04-25T08:52:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7285f42640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f726c004810 0x7f72801f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7285f42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72801e5190 0x7f72801e5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7285f42640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7280131ca0 0x7f7280132120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7285f42640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72800cc900 con 0x7f72801e5190 2024-04-25T08:52:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7285f42640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728006e290 con 0x7f7280131ca0 2024-04-25T08:52:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7285f42640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7280142680 con 0x7f726c004810 2024-04-25T08:52:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7276ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72801e5190 0x7f72801e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7276ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72801e5190 0x7f72801e5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39282/0 (socket says 172.21.15.37:39282) 2024-04-25T08:52:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7276ffd640 1 -- 172.21.15.37:0/1045313206 learned_addr learned my addr 172.21.15.37:0/1045313206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f727ffff640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7280131ca0 0x7f7280132120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f727f7fe640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f726c004810 0x7f72801f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7276ffd640 1 -- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f726c004810 msgr2=0x7f72801f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7276ffd640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f726c004810 0x7f72801f41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7276ffd640 1 -- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7280131ca0 msgr2=0x7f7280132120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7276ffd640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7280131ca0 0x7f7280132120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7276ffd640 1 -- 172.21.15.37:0/1045313206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7280132b40 con 0x7f72801e5190 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7276ffd640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72801e5190 0x7f72801e5610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7268005040 tx=0x7f726c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f727cff9640 1 -- 172.21.15.37:0/1045313206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268007a40 con 0x7f72801e5190 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f727cff9640 1 -- 172.21.15.37:0/1045313206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72680bb070 con 0x7f72801e5190 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f727cff9640 1 -- 172.21.15.37:0/1045313206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268006a00 con 0x7f72801e5190 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7285f42640 1 -- 172.21.15.37:0/1045313206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7280132e20 con 0x7f72801e5190 2024-04-25T08:52:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.560+0000 7f7285f42640 1 -- 172.21.15.37:0/1045313206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72801333e0 con 0x7f72801e5190 2024-04-25T08:52:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f727cff9640 1 -- 172.21.15.37:0/1045313206 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72680b4040 con 0x7f72801e5190 2024-04-25T08:52:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f727cff9640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7260066f90 0x7f7260069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f727cff9640 1 -- 172.21.15.37:0/1045313206 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f726811f830 con 0x7f72801e5190 2024-04-25T08:52:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f727f7fe640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7260066f90 0x7f7260069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f72890af000 1 Processor -- start 2024-04-25T08:52:38.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f72890af000 1 -- start start 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f727f7fe640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7260066f90 0x7f7260069450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7270001060 tx=0x7f7270009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f72890af000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa1920 0x563a0caa1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f72890af000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa2240 0x563a0ca579f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f72890af000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0ca57f30 0x563a0ca58310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f72890af000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c93f5f0 con 0x563a0caa2240 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f72890af000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c8a6da0 con 0x563a0ca57f30 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f72890af000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a0c82f150 con 0x563a0caa1920 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f7276ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa2240 0x563a0ca579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f7276ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa2240 0x563a0ca579f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39294/0 (socket says 172.21.15.37:39294) 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f7276ffd640 1 -- 172.21.15.37:0/835899624 learned_addr learned my addr 172.21.15.37:0/835899624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f727ffff640 1 --2- 172.21.15.37:0/835899624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0ca57f30 0x563a0ca58310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f727f7fe640 1 --2- 172.21.15.37:0/835899624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa1920 0x563a0caa1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f7276ffd640 1 -- 172.21.15.37:0/835899624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa1920 msgr2=0x563a0caa1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f7276ffd640 1 --2- 172.21.15.37:0/835899624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa1920 0x563a0caa1d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f7276ffd640 1 -- 172.21.15.37:0/835899624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0ca57f30 msgr2=0x563a0ca58310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f7276ffd640 1 --2- 172.21.15.37:0/835899624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0ca57f30 0x563a0ca58310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f7276ffd640 1 -- 172.21.15.37:0/835899624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a0c848ca0 con 0x563a0caa2240 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f727ffff640 1 --2- 172.21.15.37:0/835899624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0ca57f30 0x563a0ca58310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f7276ffd640 1 --2- 172.21.15.37:0/835899624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa2240 0x563a0ca579f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f72680f7fe0 tx=0x7f72680aa460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15e649932c6b610 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f72757fa640 1 -- 172.21.15.37:0/835899624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72680f3f50 con 0x563a0caa2240 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f72757fa640 1 -- 172.21.15.37:0/835899624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72680f8510 con 0x563a0caa2240 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.564+0000 7f72757fa640 1 -- 172.21.15.37:0/835899624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268006db0 con 0x563a0caa2240 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- 172.21.15.37:0/835899624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa2240 msgr2=0x563a0ca579f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 --2- 172.21.15.37:0/835899624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa2240 0x563a0ca579f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f72680f7fe0 tx=0x7f72680aa460 comp rx=0 tx=0).stop 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- 172.21.15.37:0/835899624 shutdown_connections 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 --2- 172.21.15.37:0/835899624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0ca57f30 0x563a0ca58310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 --2- 172.21.15.37:0/835899624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa2240 0x563a0ca579f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 --2- 172.21.15.37:0/835899624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa1920 0x563a0caa1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- 172.21.15.37:0/835899624 >> 172.21.15.37:0/835899624 conn(0x563a0ca5e2a0 msgr2=0x563a0ca562d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- 172.21.15.37:0/835899624 shutdown_connections 2024-04-25T08:52:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- 172.21.15.37:0/835899624 wait complete. 2024-04-25T08:52:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- start start 2024-04-25T08:52:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 Processor -- start 2024-04-25T08:52:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0ca57f30 0x563a0ca5be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0ca5c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0caa2240 0x563a0ca5c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c861990 con 0x563a0caa1920 2024-04-25T08:52:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c8a2e40 con 0x563a0caa2240 2024-04-25T08:52:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a0c832900 con 0x563a0ca57f30 2024-04-25T08:52:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 10 client.0 _ll_register_callbacks cb 0x563a0c95a1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f7276ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0ca5c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f7276ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0ca5c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39310/0 (socket says 172.21.15.37:39310) 2024-04-25T08:52:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f7276ffd640 1 -- 172.21.15.37:0/389972715 learned_addr learned my addr 172.21.15.37:0/389972715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f727ffff640 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0caa2240 0x563a0ca5c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f727f7fe640 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0ca57f30 0x563a0ca5be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f7276ffd640 1 -- 172.21.15.37:0/389972715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0ca57f30 msgr2=0x563a0ca5be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f7276ffd640 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0ca57f30 0x563a0ca5be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f7276ffd640 1 -- 172.21.15.37:0/389972715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0caa2240 msgr2=0x563a0ca5c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f7276ffd640 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0caa2240 0x563a0ca5c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f7276ffd640 1 -- 172.21.15.37:0/389972715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a0c85f630 con 0x563a0caa1920 2024-04-25T08:52:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f727ffff640 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0caa2240 0x563a0ca5c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f727f7fe640 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0ca57f30 0x563a0ca5be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:38.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f7276ffd640 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0ca5c3d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f72680efd90 tx=0x7f726801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4178286069661580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f725e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f725e7fc640 1 -- 172.21.15.37:0/389972715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72680f3f50 con 0x563a0caa1920 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f725e7fc640 1 -- 172.21.15.37:0/389972715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f72680ee100 con 0x563a0caa1920 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- 172.21.15.37:0/389972715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72680f8510 con 0x563a0caa1920 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f725e7fc640 1 -- 172.21.15.37:0/389972715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72680ec470 con 0x563a0caa1920 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 10 client.4407 fetch_fsmap learned FSMap version 7 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- 172.21.15.37:0/389972715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x563a0c82f3a0 con 0x563a0caa1920 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f725e7fc640 1 -- 172.21.15.37:0/389972715 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72680f8510 con 0x563a0caa1920 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f725e7fc640 1 -- 172.21.15.37:0/389972715 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f72680052f0 con 0x563a0caa1920 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 10 client.4407 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 20 client.4407 populate_metadata read hostname 'smithi037' 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x563a0cab0bc0 0x563a0cab2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 4 client.4407 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:52:38.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f72890af000 1 -- 172.21.15.37:0/389972715 --> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a0c842a50 con 0x563a0cab0bc0 2024-04-25T08:52:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f727f7fe640 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x563a0cab0bc0 0x563a0cab2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f727f7fe640 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x563a0cab0bc0 0x563a0cab2fb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=843cc089501815c9 server_cookie=ffe5f478a4966e9a in_seq=0 out_seq=0 2024-04-25T08:52:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.568+0000 7f725e7fc640 10 client.4407 ms_handle_connect on v2:172.21.15.37:6834/1246598982 2024-04-25T08:52:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f725e7fc640 1 -- 172.21.15.37:0/389972715 <== mds.0 v2:172.21.15.37:6834/1246598982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a0c842a50 con 0x563a0cab0bc0 2024-04-25T08:52:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f725e7fc640 10 client.4407 handle_command_reply: tid=0 2024-04-25T08:52:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 1 client.4407 shutdown 2024-04-25T08:52:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 1 -- 172.21.15.37:0/389972715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 msgr2=0x563a0ca5c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0ca5c3d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f72680efd90 tx=0x7f726801c3e0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 1 -- 172.21.15.37:0/389972715 shutdown_connections 2024-04-25T08:52:38.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0caa2240 0x563a0ca5c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0ca5c3d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x563a0cab0bc0 0x563a0cab2fb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 1 --2- 172.21.15.37:0/389972715 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0ca57f30 0x563a0ca5be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f725e7fc640 0 client.4407 ms_handle_reset on v2:172.21.15.37:6834/1246598982 2024-04-25T08:52:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 1 -- 172.21.15.37:0/389972715 >> 172.21.15.37:0/389972715 conn(0x563a0ca5e2a0 msgr2=0x563a0ca569e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 1 -- 172.21.15.37:0/389972715 shutdown_connections 2024-04-25T08:52:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 1 -- 172.21.15.37:0/389972715 wait complete. 2024-04-25T08:52:38.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.572+0000 7f72890af000 20 client.4407 trim_cache size 0 max 16384 2024-04-25T08:52:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 Processor -- start 2024-04-25T08:52:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 -- start start 2024-04-25T08:52:38.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0ca57f30 0x563a0cafce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa1920 0x563a0cafd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0caa2240 0x563a0cafd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7280200b20 con 0x563a0ca57f30 2024-04-25T08:52:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72800b3890 con 0x563a0caa2240 2024-04-25T08:52:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7280200520 con 0x563a0caa1920 2024-04-25T08:52:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f727f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0ca57f30 0x563a0cafce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f727f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0ca57f30 0x563a0cafce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39314/0 (socket says 172.21.15.37:39314) 2024-04-25T08:52:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f727f7fe640 1 -- 172.21.15.37:0/2596826458 learned_addr learned my addr 172.21.15.37:0/2596826458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f727f7fe640 1 -- 172.21.15.37:0/2596826458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa1920 msgr2=0x563a0cafd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f727f7fe640 1 --2- 172.21.15.37:0/2596826458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa1920 0x563a0cafd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f727f7fe640 1 -- 172.21.15.37:0/2596826458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0caa2240 msgr2=0x563a0cafd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f727f7fe640 1 --2- 172.21.15.37:0/2596826458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0caa2240 0x563a0cafd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f727f7fe640 1 -- 172.21.15.37:0/2596826458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a0caa2070 con 0x563a0ca57f30 2024-04-25T08:52:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f727f7fe640 1 --2- 172.21.15.37:0/2596826458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0ca57f30 0x563a0cafce30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x563a0ca630f0 tx=0x7f727000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad71c18d593203 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f7274ff9640 1 -- 172.21.15.37:0/2596826458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727001a040 con 0x563a0ca57f30 2024-04-25T08:52:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f7274ff9640 1 -- 172.21.15.37:0/2596826458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7270025690 con 0x563a0ca57f30 2024-04-25T08:52:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f7274ff9640 1 -- 172.21.15.37:0/2596826458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7270021400 con 0x563a0ca57f30 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 -- 172.21.15.37:0/2596826458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0ca57f30 msgr2=0x563a0cafce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 --2- 172.21.15.37:0/2596826458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0ca57f30 0x563a0cafce30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x563a0ca630f0 tx=0x7f727000c000 comp rx=0 tx=0).stop 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 -- 172.21.15.37:0/2596826458 shutdown_connections 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 --2- 172.21.15.37:0/2596826458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0caa2240 0x563a0cafd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 --2- 172.21.15.37:0/2596826458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa1920 0x563a0cafd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 --2- 172.21.15.37:0/2596826458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0ca57f30 0x563a0cafce30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 -- 172.21.15.37:0/2596826458 >> 172.21.15.37:0/2596826458 conn(0x563a0ca5e2a0 msgr2=0x563a0ca571f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 -- 172.21.15.37:0/2596826458 shutdown_connections 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 -- 172.21.15.37:0/2596826458 wait complete. 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 -- start start 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.588+0000 7f72890af000 1 Processor -- start 2024-04-25T08:52:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0ca57f30 0x563a0ca5c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0cae8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa2240 0x563a0cae9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c8ab3d0 con 0x563a0caa1920 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a0c962d50 con 0x563a0ca57f30 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a0c841980 con 0x563a0caa2240 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 10 client.0 _ll_register_callbacks cb 0x7f72680ec0a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7276ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0cae8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7276ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0cae8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39324/0 (socket says 172.21.15.37:39324) 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7276ffd640 1 -- 172.21.15.37:0/2310649302 learned_addr learned my addr 172.21.15.37:0/2310649302 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f727ffff640 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa2240 0x563a0cae9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f727f7fe640 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0ca57f30 0x563a0ca5c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7276ffd640 1 -- 172.21.15.37:0/2310649302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa2240 msgr2=0x563a0cae9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7276ffd640 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa2240 0x563a0cae9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7276ffd640 1 -- 172.21.15.37:0/2310649302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0ca57f30 msgr2=0x563a0ca5c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7276ffd640 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0ca57f30 0x563a0ca5c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7276ffd640 1 -- 172.21.15.37:0/2310649302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a0c928600 con 0x563a0caa1920 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7276ffd640 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0cae8f00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72680f8290 tx=0x7f726811d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e4caed35faa4f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f725e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f725e7fc640 1 -- 172.21.15.37:0/2310649302 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72680f8080 con 0x563a0caa1920 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f725e7fc640 1 -- 172.21.15.37:0/2310649302 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f72680ef250 con 0x563a0caa1920 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 -- 172.21.15.37:0/2310649302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7270025690 con 0x563a0caa1920 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f725e7fc640 1 -- 172.21.15.37:0/2310649302 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7268114e60 con 0x563a0caa1920 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 10 client.4413 fetch_fsmap learned FSMap version 7 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 -- 172.21.15.37:0/2310649302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x563a0c83b9b0 con 0x563a0caa1920 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f725e7fc640 1 -- 172.21.15.37:0/2310649302 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7270025690 con 0x563a0caa1920 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f725e7fc640 1 -- 172.21.15.37:0/2310649302 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f7268111070 con 0x563a0caa1920 2024-04-25T08:52:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 10 client.4413 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 10 client.4413 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 20 client.4413 populate_metadata read hostname 'smithi037' 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x563a0cae9980 0x563a0cb01aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 4 client.4413 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 -- 172.21.15.37:0/2310649302 --> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563a0c8b42e0 con 0x563a0cae9980 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f727f7fe640 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x563a0cae9980 0x563a0cb01aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f725e7fc640 10 client.4413 ms_handle_connect on v2:172.21.15.37:6834/1246598982 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f727f7fe640 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x563a0cae9980 0x563a0cb01aa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9571bdc8e7011bf0 server_cookie=e1fa68e67613be6a in_seq=0 out_seq=0 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f725e7fc640 1 -- 172.21.15.37:0/2310649302 <== mds.0 v2:172.21.15.37:6834/1246598982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x563a0c8b42e0 con 0x563a0cae9980 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f725e7fc640 10 client.4413 handle_command_reply: tid=0 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 client.4413 shutdown 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 -- 172.21.15.37:0/2310649302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 msgr2=0x563a0cae8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0cae8f00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72680f8290 tx=0x7f726811d8f0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 -- 172.21.15.37:0/2310649302 shutdown_connections 2024-04-25T08:52:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x563a0cae9980 0x563a0cb01aa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563a0caa2240 0x563a0cae9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563a0caa1920 0x563a0cae8f00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 --2- 172.21.15.37:0/2310649302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563a0ca57f30 0x563a0ca5c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 -- 172.21.15.37:0/2310649302 >> 172.21.15.37:0/2310649302 conn(0x563a0ca5e2a0 msgr2=0x563a0ca8d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f725e7fc640 0 client.4413 ms_handle_reset on v2:172.21.15.37:6834/1246598982 2024-04-25T08:52:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 -- 172.21.15.37:0/2310649302 shutdown_connections 2024-04-25T08:52:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 1 -- 172.21.15.37:0/2310649302 wait complete. 2024-04-25T08:52:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f72890af000 20 client.4413 trim_cache size 0 max 16384 2024-04-25T08:52:38.593 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:52:38.593 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T08:52:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T08:52:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4251, 2024-04-25T08:52:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T08:52:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T08:52:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T08:52:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 7.3821254830000003, 2024-04-25T08:52:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 7, 2024-04-25T08:52:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 27, 2024-04-25T08:52:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 25, 2024-04-25T08:52:38.594 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 14.363721755 2024-04-25T08:52:38.594 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:52:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 -- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7260066f90 msgr2=0x7f7260069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7260066f90 0x7f7260069450 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7270001060 tx=0x7f7270009040 comp rx=0 tx=0).stop 2024-04-25T08:52:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 -- 172.21.15.37:0/1045313206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72801e5190 msgr2=0x7f72801e5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72801e5190 0x7f72801e5610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7268005040 tx=0x7f726c005840 comp rx=0 tx=0).stop 2024-04-25T08:52:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 -- 172.21.15.37:0/1045313206 shutdown_connections 2024-04-25T08:52:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7260066f90 0x7f7260069450 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7280131ca0 0x7f7280132120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72801e5190 0x7f72801e5610 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 --2- 172.21.15.37:0/1045313206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f726c004810 0x7f72801f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 -- 172.21.15.37:0/1045313206 >> 172.21.15.37:0/1045313206 conn(0x7f72800b11a0 msgr2=0x7f7280141d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 -- 172.21.15.37:0/1045313206 shutdown_connections 2024-04-25T08:52:38.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.592+0000 7f7285f42640 1 -- 172.21.15.37:0/1045313206 wait complete. 2024-04-25T08:52:38.602 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:52:38.602 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:52:38.603 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:52:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4b134f640 1 Processor -- start 2024-04-25T08:52:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4b134f640 1 -- start start 2024-04-25T08:52:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4b134f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc4ac06d7a0 0x7fc4ac15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4b134f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4ac15ae30 con 0x7fc4ac0b3200 2024-04-25T08:52:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4b134f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4ac15b110 con 0x7fc4ac0b2e20 2024-04-25T08:52:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4b134f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ac0b2990 con 0x7fc4ac06d7a0 2024-04-25T08:52:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4ab7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc4ac06d7a0 0x7fc4ac15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4aa7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc4ac0b3200 0x7fc4ac159ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37206/0 (socket says 172.21.15.37:37206) 2024-04-25T08:52:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4aa7fc640 1 -- 172.21.15.37:0/645619823 learned_addr learned my addr 172.21.15.37:0/645619823 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4ab7fe640 1 -- 172.21.15.37:0/645619823 >> v1:172.21.15.203:6789/0 conn(0x7fc4ac0b2e20 legacy=0x7fc4ac06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4ab7fe640 1 -- 172.21.15.37:0/645619823 >> v1:172.21.15.37:6789/0 conn(0x7fc4ac0b3200 legacy=0x7fc4ac159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4a9ffb640 1 -- 172.21.15.37:0/645619823 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641950053 0 0) 0x7fc4ac15ae30 con 0x7fc4ac0b3200 2024-04-25T08:52:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4ab7fe640 1 -- 172.21.15.37:0/645619823 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac0b2620 con 0x7fc4ac06d7a0 2024-04-25T08:52:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.720+0000 7fc4ab7fe640 1 --2- 172.21.15.37:0/645619823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac06d7a0 0x7fc4ac15a5b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40097c0 tx=0x7fc4a401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0cc73cf9a606134 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4a9ffb640 1 -- 172.21.15.37:0/645619823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40b0070 con 0x7fc4ac06d7a0 2024-04-25T08:52:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4a9ffb640 1 -- 172.21.15.37:0/645619823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a40ab600 con 0x7fc4ac06d7a0 2024-04-25T08:52:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4a9ffb640 1 -- 172.21.15.37:0/645619823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40ab9d0 con 0x7fc4ac06d7a0 2024-04-25T08:52:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 -- 172.21.15.37:0/645619823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac06d7a0 msgr2=0x7fc4ac15a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 --2- 172.21.15.37:0/645619823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac06d7a0 0x7fc4ac15a5b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40097c0 tx=0x7fc4a401c460 comp rx=0 tx=0).stop 2024-04-25T08:52:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 -- 172.21.15.37:0/645619823 shutdown_connections 2024-04-25T08:52:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 --2- 172.21.15.37:0/645619823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac06d7a0 0x7fc4ac15a5b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 -- 172.21.15.37:0/645619823 >> 172.21.15.37:0/645619823 conn(0x7fc4ac0b11a0 msgr2=0x7fc4ac0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 -- 172.21.15.37:0/645619823 shutdown_connections 2024-04-25T08:52:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 -- 172.21.15.37:0/645619823 wait complete. 2024-04-25T08:52:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 Processor -- start 2024-04-25T08:52:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 -- start start 2024-04-25T08:52:38.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ac06d7a0 0x7fc4ac151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ac0b2e20 0x7fc4ac155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac0b3200 0x7fc4ac154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac1686e0 con 0x7fc4ac0b3200 2024-04-25T08:52:38.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac072250 con 0x7fc4ac06d7a0 2024-04-25T08:52:38.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac168860 con 0x7fc4ac0b2e20 2024-04-25T08:52:38.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4ab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac0b3200 0x7fc4ac154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4ab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac0b3200 0x7fc4ac154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39350/0 (socket says 172.21.15.37:39350) 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4ab7fe640 1 -- 172.21.15.37:0/3999054107 learned_addr learned my addr 172.21.15.37:0/3999054107 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4ab7fe640 1 -- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ac0b2e20 msgr2=0x7fc4ac155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4ab7fe640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ac0b2e20 0x7fc4ac155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4ab7fe640 1 -- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ac06d7a0 msgr2=0x7fc4ac151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4ab7fe640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ac06d7a0 0x7fc4ac151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4ab7fe640 1 -- 172.21.15.37:0/3999054107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac067e40 con 0x7fc4ac0b3200 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4ab7fe640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac0b3200 0x7fc4ac154580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40097c0 tx=0x7fc4a401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4a37fe640 1 -- 172.21.15.37:0/3999054107 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40b0050 con 0x7fc4ac0b3200 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4a37fe640 1 -- 172.21.15.37:0/3999054107 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a40bc070 con 0x7fc4ac0b3200 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 -- 172.21.15.37:0/3999054107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ac154ac0 con 0x7fc4ac0b3200 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4a37fe640 1 -- 172.21.15.37:0/3999054107 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40b5830 con 0x7fc4ac0b3200 2024-04-25T08:52:38.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.724+0000 7fc4b134f640 1 -- 172.21.15.37:0/3999054107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ac155040 con 0x7fc4ac0b3200 2024-04-25T08:52:38.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.728+0000 7fc4b134f640 1 -- 172.21.15.37:0/3999054107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ac04efa0 con 0x7fc4ac0b3200 2024-04-25T08:52:38.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.728+0000 7fc4a37fe640 1 -- 172.21.15.37:0/3999054107 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4a40b59d0 con 0x7fc4ac0b3200 2024-04-25T08:52:38.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.728+0000 7fc4a37fe640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc48c066f60 0x7fc48c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:38.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.728+0000 7fc4a37fe640 1 -- 172.21.15.37:0/3999054107 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc4a411fb30 con 0x7fc4ac0b3200 2024-04-25T08:52:38.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.728+0000 7fc4aaffd640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc48c066f60 0x7fc48c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:38.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.728+0000 7fc4aaffd640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc48c066f60 0x7fc48c069420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac150a60 tx=0x7fc49c0079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:38.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.732+0000 7fc4a37fe640 1 -- 172.21.15.37:0/3999054107 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4a4016050 con 0x7fc4ac0b3200 2024-04-25T08:52:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.920+0000 7fc4b134f640 1 -- 172.21.15.37:0/3999054107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc4ac0affa0 con 0x7fc4ac0b3200 2024-04-25T08:52:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.920+0000 7fc4a37fe640 1 -- 172.21.15.37:0/3999054107 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc4a40f0890 con 0x7fc4ac0b3200 2024-04-25T08:52:38.914 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:52:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 -- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc48c066f60 msgr2=0x7fc48c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc48c066f60 0x7fc48c069420 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac150a60 tx=0x7fc49c0079c0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 -- 172.21.15.37:0/3999054107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac0b3200 msgr2=0x7fc4ac154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac0b3200 0x7fc4ac154580 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40097c0 tx=0x7fc4a401aca0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 -- 172.21.15.37:0/3999054107 shutdown_connections 2024-04-25T08:52:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc48c066f60 0x7fc48c069420 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac0b3200 0x7fc4ac154580 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ac0b2e20 0x7fc4ac155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 --2- 172.21.15.37:0/3999054107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ac06d7a0 0x7fc4ac151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 -- 172.21.15.37:0/3999054107 >> 172.21.15.37:0/3999054107 conn(0x7fc4ac0b11a0 msgr2=0x7fc4ac165240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 -- 172.21.15.37:0/3999054107 shutdown_connections 2024-04-25T08:52:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:38.924+0000 7fc4b134f640 1 -- 172.21.15.37:0/3999054107 wait complete. 2024-04-25T08:52:38.931 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T08:52:38.931 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:52:38.931 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T08:52:38.931 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T08:52:38.931 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:52:38.932 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T08:52:38.932 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:52:38.932 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:52:38.932 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T08:52:38.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:38.932 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T08:52:38.938 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:52:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:52:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:52:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:52:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:38.938 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:38.938 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T08:52:38.938 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T08:52:38.939 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:52:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T08:52:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:52:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42482sec preferred_lft 42482sec 2024-04-25T08:52:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T08:52:38.939 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:38.939 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:52:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T08:52:38.940 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:52:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:52:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:52:38.940 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:52:38.940 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:52:38.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:38.941 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:52:38.941 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:52:38.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:52:38.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-04-25T08:52:38.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-04-25T08:52:38.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:52:38.941 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:52:39.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.015 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:52:39.068 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:52:39.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.069 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T08:52:39.139 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T08:52:39.140 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:52:39.140 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:52:39.140 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:52:39.140 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:52:39.140 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:52:39.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.141 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:52:39.141 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:52:39.141 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:52:39.141 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:52:39.142 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:52:39.142 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:52:39.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.217 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:52:39.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.265 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-25T08:52:39.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.319 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:52:39.320 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:52:39.320 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:52:39.320 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T08:52:39.320 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:52:39.386 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T08:52:39.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.390 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:52:39.390 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:52:39.390 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T08:52:39.390 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:52:39.390 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T08:52:39.390 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T08:52:39.390 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T08:52:39.390 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:52:39.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.488 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:52:39.488 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:52:39.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T08:52:39.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:52:39.488 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:52:39.508 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T08:52:39.508 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:52:39.508 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:52:39.556 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T08:52:39.556 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:52:39.556 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:52:39.603 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:52:39.663 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:52:39.709 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:52:39.710 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:52:39.711 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:52:39.712 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:52:39.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.713 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:52:39.760 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:52:39.761 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T08:52:39.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:52:39.824 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:52:39.876 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:52:39.876 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:52:39.876 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:52:39.876 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:52:39.876 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:52:39.876 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:52:39.876 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:52:39.876 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:52:39.876 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:52:39.877 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:52:39.878 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:52:39.878 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:52:39.878 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:52:39.878 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:52:39.878 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:52:39.878 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:52:39.878 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:52:39.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:39.879 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:52:39.895 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:2024-04-25T08:52:39.900+0000 7f32e2dcd580 -1 init, newargv = 0x563789261a40 newargc=15 2024-04-25T08:52:39.895 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[34305]: starting ceph client 2024-04-25T08:52:39.902 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[34305]: starting fuse 2024-04-25T08:52:41.001 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:52:41.017 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:52:41.067 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:52:41.067 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:52:41.067 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:52:41.067 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:52:41.067 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:52:41.067 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:52:41.067 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:52:41.067 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:52:41.067 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:52:41.067 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:52:41.068 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:52:41.069 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:52:41.069 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:52:41.069 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:52:41.069 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:52:41.069 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:52:41.069 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:52:41.069 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.admin 2024-04-25T08:52:41.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:41.070 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:52:41.116 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:52:41.117 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:52:41.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:52:41.117 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T08:52:41.117 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T08:52:41.117 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:52:41.117 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T08:52:41.117 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:52:41.117 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T08:52:41.117 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T08:52:41.117 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T08:52:41.117 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T08:52:41.117 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.admin")) 2024-04-25T08:52:41.118 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:52:41.245 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.admin.34305.asok 2024-04-25T08:52:41.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:52:41.258 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.34305.asok 2024-04-25T08:52:41.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:41.258 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.34305.asok status 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "admin", 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "34305", 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T08:52:41.417 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4431, 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 4431 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1994306030 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:52:41.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1994306030 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.4431 192.168.0.1:0/1994306030", 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/1994306030", 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 7, 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 27, 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T08:52:41.419 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:52:41.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:41.431 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T08:52:41.431 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:52:41.437 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T08:52:41.437 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:52:41.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:41.438 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:52:41.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:41.492 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T08:52:41.812 INFO:teuthology.orchestra.run.smithi037.stdout:# file: . 2024-04-25T08:52:41.812 INFO:teuthology.orchestra.run.smithi037.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T08:52:41.812 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:41.813 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:52:41.813 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:41.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:41.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4ee3152640 1 Processor -- start 2024-04-25T08:52:41.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4ee3152640 1 -- start start 2024-04-25T08:52:41.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4ee3152640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4edc0ce510 0x7f4edc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:41.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4ee3152640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4edc0cee30 con 0x7f4edc144ca0 2024-04-25T08:52:41.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4ee3152640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4edc0ccc70 con 0x7f4edc1451f0 2024-04-25T08:52:41.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4ee3152640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc0cc2b0 con 0x7f4edc0ce510 2024-04-25T08:52:41.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4edbfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4edc144ca0 0x7f4edc06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47500/0 (socket says 172.21.15.37:47500) 2024-04-25T08:52:41.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4edbfff640 1 -- 172.21.15.37:0/2380094468 learned_addr learned my addr 172.21.15.37:0/2380094468 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:41.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4ee0eee640 1 --2- 172.21.15.37:0/2380094468 >> v2:172.21.15.37:3300/0 conn(0x7f4edc0ce510 0x7f4edc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:41.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4edb7fe640 1 -- 172.21.15.37:0/2380094468 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4177837405 0 0) 0x7f4edc0cee30 con 0x7f4edc144ca0 2024-04-25T08:52:41.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4edb7fe640 1 -- 172.21.15.37:0/2380094468 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec4003530 con 0x7f4edc144ca0 2024-04-25T08:52:41.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4ee0eee640 1 -- 172.21.15.37:0/2380094468 >> v1:172.21.15.203:6789/0 conn(0x7f4edc1451f0 legacy=0x7f4edc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:41.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4ee0eee640 1 -- 172.21.15.37:0/2380094468 >> v1:172.21.15.37:6789/0 conn(0x7f4edc144ca0 legacy=0x7f4edc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:41.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.004+0000 7f4ee0eee640 1 -- 172.21.15.37:0/2380094468 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc06dc40 con 0x7f4edc0ce510 2024-04-25T08:52:42.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee0eee640 1 --2- 172.21.15.37:0/2380094468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc0ce510 0x7f4edc0ce8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0097c0 tx=0x7f4ecc01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a587c735d432acf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4edb7fe640 1 -- 172.21.15.37:0/2380094468 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0b5070 con 0x7f4edc0ce510 2024-04-25T08:52:42.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4edb7fe640 1 -- 172.21.15.37:0/2380094468 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc0a8950 con 0x7f4edc0ce510 2024-04-25T08:52:42.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4edb7fe640 1 -- 172.21.15.37:0/2380094468 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0a8d20 con 0x7f4edc0ce510 2024-04-25T08:52:42.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 -- 172.21.15.37:0/2380094468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc0ce510 msgr2=0x7f4edc0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 --2- 172.21.15.37:0/2380094468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc0ce510 0x7f4edc0ce8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0097c0 tx=0x7f4ecc01ec50 comp rx=0 tx=0).stop 2024-04-25T08:52:42.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 -- 172.21.15.37:0/2380094468 shutdown_connections 2024-04-25T08:52:42.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 --2- 172.21.15.37:0/2380094468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc0ce510 0x7f4edc0ce8f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 -- 172.21.15.37:0/2380094468 >> 172.21.15.37:0/2380094468 conn(0x7f4edc0b11a0 msgr2=0x7f4edc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:42.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 -- 172.21.15.37:0/2380094468 shutdown_connections 2024-04-25T08:52:42.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 -- 172.21.15.37:0/2380094468 wait complete. 2024-04-25T08:52:42.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 Processor -- start 2024-04-25T08:52:42.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 -- start start 2024-04-25T08:52:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc0ce510 0x7f4edc15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4edc144ca0 0x7f4edc1e55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4edc1451f0 0x7f4edc1e5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06ee30 con 0x7f4edc0ce510 2024-04-25T08:52:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06e290 con 0x7f4edc144ca0 2024-04-25T08:52:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0cc900 con 0x7f4edc1451f0 2024-04-25T08:52:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee0eee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc0ce510 0x7f4edc15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee0eee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc0ce510 0x7f4edc15ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39482/0 (socket says 172.21.15.37:39482) 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee0eee640 1 -- 172.21.15.37:0/2176853431 learned_addr learned my addr 172.21.15.37:0/2176853431 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4edbfff640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4edc144ca0 0x7f4edc1e55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee0eee640 1 -- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4edc1451f0 msgr2=0x7f4edc1e5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee16ef640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4edc1451f0 0x7f4edc1e5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee0eee640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4edc1451f0 0x7f4edc1e5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee0eee640 1 -- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4edc144ca0 msgr2=0x7f4edc1e55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee0eee640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4edc144ca0 0x7f4edc1e55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee0eee640 1 -- 172.21.15.37:0/2176853431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc1e2500 con 0x7f4edc0ce510 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4edbfff640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4edc144ca0 0x7f4edc1e55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee0eee640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc0ce510 0x7f4edc15ae40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0b3040 tx=0x7f4ecc01b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ed97fa640 1 -- 172.21.15.37:0/2176853431 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0b5040 con 0x7f4edc0ce510 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ed97fa640 1 -- 172.21.15.37:0/2176853431 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ecc0bf070 con 0x7f4edc0ce510 2024-04-25T08:52:42.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.008+0000 7f4ee3152640 1 -- 172.21.15.37:0/2176853431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc1e27e0 con 0x7f4edc0ce510 2024-04-25T08:52:42.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.012+0000 7f4ed97fa640 1 -- 172.21.15.37:0/2176853431 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc0b0900 con 0x7f4edc0ce510 2024-04-25T08:52:42.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.012+0000 7f4ee3152640 1 -- 172.21.15.37:0/2176853431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc1e3000 con 0x7f4edc0ce510 2024-04-25T08:52:42.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.012+0000 7f4ee3152640 1 -- 172.21.15.37:0/2176853431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc04efa0 con 0x7f4edc0ce510 2024-04-25T08:52:42.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.012+0000 7f4ed97fa640 1 -- 172.21.15.37:0/2176853431 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ecc0b0aa0 con 0x7f4edc0ce510 2024-04-25T08:52:42.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.012+0000 7f4ed97fa640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ec8066f60 0x7f4ec8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.012+0000 7f4ed97fa640 1 -- 172.21.15.37:0/2176853431 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4ecc123250 con 0x7f4edc0ce510 2024-04-25T08:52:42.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.012+0000 7f4edbfff640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ec8066f60 0x7f4ec8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.012+0000 7f4edbfff640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ec8066f60 0x7f4ec8069420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0005890 tx=0x7f4ed0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.016+0000 7f4ed97fa640 1 -- 172.21.15.37:0/2176853431 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ecc0187b0 con 0x7f4edc0ce510 2024-04-25T08:52:42.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.180+0000 7f4ee3152640 1 -- 172.21.15.37:0/2176853431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4edc0b1820 con 0x7f4edc0ce510 2024-04-25T08:52:42.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ed97fa640 1 -- 172.21.15.37:0/2176853431 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f4ecc0f3f90 con 0x7f4edc0ce510 2024-04-25T08:52:42.177 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:42.177 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:38.282648+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:42.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 -- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ec8066f60 msgr2=0x7f4ec8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ec8066f60 0x7f4ec8069420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0005890 tx=0x7f4ed0009040 comp rx=0 tx=0).stop 2024-04-25T08:52:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 -- 172.21.15.37:0/2176853431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc0ce510 msgr2=0x7f4edc15ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc0ce510 0x7f4edc15ae40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc0b3040 tx=0x7f4ecc01b9f0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 -- 172.21.15.37:0/2176853431 shutdown_connections 2024-04-25T08:52:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ec8066f60 0x7f4ec8069420 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4edc1451f0 0x7f4edc1e5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4edc144ca0 0x7f4edc1e55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 --2- 172.21.15.37:0/2176853431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4edc0ce510 0x7f4edc15ae40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 -- 172.21.15.37:0/2176853431 >> 172.21.15.37:0/2176853431 conn(0x7f4edc0b11a0 msgr2=0x7f4edc0b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 -- 172.21.15.37:0/2176853431 shutdown_connections 2024-04-25T08:52:42.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.184+0000 7f4ee3152640 1 -- 172.21.15.37:0/2176853431 wait complete. 2024-04-25T08:52:42.196 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:52:42.196 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:52:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6349262640 1 Processor -- start 2024-04-25T08:52:42.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6349262640 1 -- start start 2024-04-25T08:52:42.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6349262640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f634415b920 0x7f6344099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6349262640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f63440b0440 0x7f6344099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6349262640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f634409a950 con 0x7f63440aeb80 2024-04-25T08:52:42.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6349262640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f634409ac30 con 0x7f634415b920 2024-04-25T08:52:42.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6349262640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63440ae380 con 0x7f63440b0440 2024-04-25T08:52:42.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6342ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f634415b920 0x7f6344099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f63437fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f63440aeb80 0x7f63440aef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47506/0 (socket says 172.21.15.37:47506) 2024-04-25T08:52:42.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f63437fe640 1 -- 172.21.15.37:0/2950617877 learned_addr learned my addr 172.21.15.37:0/2950617877 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:42.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6342ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f634415b920 0x7f6344099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39494/0 (socket says 172.21.15.37:39494) 2024-04-25T08:52:42.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f63427fc640 1 --2- 172.21.15.37:0/2950617877 >> v2:172.21.15.203:3300/0 conn(0x7f63440b0440 0x7f6344099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6342ffd640 1 -- 172.21.15.37:0/2950617877 >> v2:172.21.15.203:3300/0 conn(0x7f63440b0440 msgr2=0x7f6344099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:42.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6342ffd640 1 --2- 172.21.15.37:0/2950617877 >> v2:172.21.15.203:3300/0 conn(0x7f63440b0440 0x7f6344099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6342ffd640 1 -- 172.21.15.37:0/2950617877 >> v1:172.21.15.37:6789/0 conn(0x7f63440aeb80 legacy=0x7f63440aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f6342ffd640 1 -- 172.21.15.37:0/2950617877 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63440ae660 con 0x7f634415b920 2024-04-25T08:52:42.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.336+0000 7f63427fc640 1 --2- 172.21.15.37:0/2950617877 >> v2:172.21.15.203:3300/0 conn(0x7f63440b0440 0x7f6344099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6342ffd640 1 --2- 172.21.15.37:0/2950617877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f634415b920 0x7f6344099a70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f63440b3d40 tx=0x7f633801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3ede152bd1a0ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6341ffb640 1 -- 172.21.15.37:0/2950617877 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63380b1070 con 0x7f634415b920 2024-04-25T08:52:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6341ffb640 1 -- 172.21.15.37:0/2950617877 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63380ac5f0 con 0x7f634415b920 2024-04-25T08:52:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6341ffb640 1 -- 172.21.15.37:0/2950617877 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63440afeb0 con 0x7f634415b920 2024-04-25T08:52:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 -- 172.21.15.37:0/2950617877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f634415b920 msgr2=0x7f6344099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 --2- 172.21.15.37:0/2950617877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f634415b920 0x7f6344099a70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f63440b3d40 tx=0x7f633801c480 comp rx=0 tx=0).stop 2024-04-25T08:52:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 -- 172.21.15.37:0/2950617877 shutdown_connections 2024-04-25T08:52:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 --2- 172.21.15.37:0/2950617877 >> v2:172.21.15.203:3300/0 conn(0x7f63440b0440 0x7f6344099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 --2- 172.21.15.37:0/2950617877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f634415b920 0x7f6344099a70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 -- 172.21.15.37:0/2950617877 >> 172.21.15.37:0/2950617877 conn(0x7f63440b3010 msgr2=0x7f63440b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 -- 172.21.15.37:0/2950617877 shutdown_connections 2024-04-25T08:52:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 -- 172.21.15.37:0/2950617877 wait complete. 2024-04-25T08:52:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 Processor -- start 2024-04-25T08:52:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 -- start start 2024-04-25T08:52:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63440aeb80 0x7f6344153b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63440b0440 0x7f63441feb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f634415b920 0x7f63440b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6344168850 con 0x7f634415b920 2024-04-25T08:52:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63441686d0 con 0x7f63440b0440 2024-04-25T08:52:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6349262640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63441689d0 con 0x7f63440aeb80 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f63437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f634415b920 0x7f63440b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f63437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f634415b920 0x7f63440b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39508/0 (socket says 172.21.15.37:39508) 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f63437fe640 1 -- 172.21.15.37:0/2916699320 learned_addr learned my addr 172.21.15.37:0/2916699320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f6342ffd640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63440aeb80 0x7f6344153b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f63437fe640 1 -- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63440aeb80 msgr2=0x7f6344153b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f63437fe640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63440aeb80 0x7f6344153b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f63427fc640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63440b0440 0x7f63441feb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f63437fe640 1 -- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63440b0440 msgr2=0x7f63441feb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f63437fe640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63440b0440 0x7f63441feb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f63437fe640 1 -- 172.21.15.37:0/2916699320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63441ff060 con 0x7f634415b920 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.340+0000 7f63427fc640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63440b0440 0x7f63441feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f63437fe640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f634415b920 0x7f63440b22a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f633c00ca80 tx=0x7f633c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f632b7fe640 1 -- 172.21.15.37:0/2916699320 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c017070 con 0x7f634415b920 2024-04-25T08:52:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f632b7fe640 1 -- 172.21.15.37:0/2916699320 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633c00ac30 con 0x7f634415b920 2024-04-25T08:52:42.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f6349262640 1 -- 172.21.15.37:0/2916699320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634415ec60 con 0x7f634415b920 2024-04-25T08:52:42.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f632b7fe640 1 -- 172.21.15.37:0/2916699320 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f633c012600 con 0x7f634415b920 2024-04-25T08:52:42.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f6349262640 1 -- 172.21.15.37:0/2916699320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634415cc60 con 0x7f634415b920 2024-04-25T08:52:42.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f6349262640 1 -- 172.21.15.37:0/2916699320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634404efa0 con 0x7f634415b920 2024-04-25T08:52:42.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f632b7fe640 1 -- 172.21.15.37:0/2916699320 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f633c01a070 con 0x7f634415b920 2024-04-25T08:52:42.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f632b7fe640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6330066f60 0x7f6330069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f632b7fe640 1 -- 172.21.15.37:0/2916699320 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f633c084f60 con 0x7f634415b920 2024-04-25T08:52:42.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f6342ffd640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6330066f60 0x7f6330069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.344+0000 7f6342ffd640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6330066f60 0x7f6330069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f63380ae000 tx=0x7f63380a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.348+0000 7f632b7fe640 1 -- 172.21.15.37:0/2916699320 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f63441ff060 con 0x7f634415b920 2024-04-25T08:52:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.512+0000 7f6349262640 1 -- 172.21.15.37:0/2916699320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6344073d60 con 0x7f634415b920 2024-04-25T08:52:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.512+0000 7f632b7fe640 1 -- 172.21.15.37:0/2916699320 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f633c00ae20 con 0x7f634415b920 2024-04-25T08:52:42.506 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:52:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 -- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6330066f60 msgr2=0x7f6330069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6330066f60 0x7f6330069420 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f63380ae000 tx=0x7f63380a7000 comp rx=0 tx=0).stop 2024-04-25T08:52:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 -- 172.21.15.37:0/2916699320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f634415b920 msgr2=0x7f63440b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f634415b920 0x7f63440b22a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f633c00ca80 tx=0x7f633c00cf40 comp rx=0 tx=0).stop 2024-04-25T08:52:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 -- 172.21.15.37:0/2916699320 shutdown_connections 2024-04-25T08:52:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6330066f60 0x7f6330069420 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f634415b920 0x7f63440b22a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63440b0440 0x7f63441feb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 --2- 172.21.15.37:0/2916699320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63440aeb80 0x7f6344153b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 -- 172.21.15.37:0/2916699320 >> 172.21.15.37:0/2916699320 conn(0x7f63440b3010 msgr2=0x7f6344099a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 -- 172.21.15.37:0/2916699320 shutdown_connections 2024-04-25T08:52:42.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.516+0000 7f6349262640 1 -- 172.21.15.37:0/2916699320 wait complete. 2024-04-25T08:52:42.525 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:52:42.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b4742640 1 Processor -- start 2024-04-25T08:52:42.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b4742640 1 -- start start 2024-04-25T08:52:42.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b4742640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb0ac15ecb0 0x7fb0ac0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b4742640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0ac0c9ef0 0x7fb0ac0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b4742640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ac0ca4c0 con 0x7fb0ac0cf590 2024-04-25T08:52:42.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b4742640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ac0ca780 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b4742640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac15f0f0 con 0x7fb0ac0c9ef0 2024-04-25T08:52:42.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b1cdd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb0ac15ecb0 0x7fb0ac0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b1cdd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb0ac15ecb0 0x7fb0ac0c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39510/0 (socket says 172.21.15.37:39510) 2024-04-25T08:52:42.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b1cdd640 1 -- 172.21.15.37:0/1042076118 learned_addr learned my addr 172.21.15.37:0/1042076118 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:42.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b2cdf640 1 --2- 172.21.15.37:0/1042076118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0ac0c9ef0 0x7fb0ac0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b1cdd640 1 -- 172.21.15.37:0/1042076118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0ac0c9ef0 msgr2=0x7fb0ac0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:42.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b1cdd640 1 --2- 172.21.15.37:0/1042076118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0ac0c9ef0 0x7fb0ac0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b1cdd640 1 -- 172.21.15.37:0/1042076118 >> v1:172.21.15.203:6789/0 conn(0x7fb0ac0cf590 legacy=0x7fb0ac0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.664+0000 7fb0b1cdd640 1 -- 172.21.15.37:0/1042076118 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac0cd690 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b1cdd640 1 --2- 172.21.15.37:0/1042076118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15ecb0 0x7fb0ac0c99b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009870 tx=0x7fb09c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f51ef01fb11c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b14dc640 1 -- 172.21.15.37:0/1042076118 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0a7070 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b14dc640 1 -- 172.21.15.37:0/1042076118 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb09c0b0070 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b14dc640 1 -- 172.21.15.37:0/1042076118 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c0ab890 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b4742640 1 -- 172.21.15.37:0/1042076118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15ecb0 msgr2=0x7fb0ac0c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b4742640 1 --2- 172.21.15.37:0/1042076118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15ecb0 0x7fb0ac0c99b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb09c009870 tx=0x7fb09c01c560 comp rx=0 tx=0).stop 2024-04-25T08:52:42.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b4742640 1 -- 172.21.15.37:0/1042076118 shutdown_connections 2024-04-25T08:52:42.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b4742640 1 --2- 172.21.15.37:0/1042076118 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0ac0c9ef0 0x7fb0ac0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b4742640 1 --2- 172.21.15.37:0/1042076118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15ecb0 0x7fb0ac0c99b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b4742640 1 -- 172.21.15.37:0/1042076118 >> 172.21.15.37:0/1042076118 conn(0x7fb0ac06ae60 msgr2=0x7fb0ac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:42.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b4742640 1 -- 172.21.15.37:0/1042076118 shutdown_connections 2024-04-25T08:52:42.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.668+0000 7fb0b4742640 1 -- 172.21.15.37:0/1042076118 wait complete. 2024-04-25T08:52:42.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b4742640 1 Processor -- start 2024-04-25T08:52:42.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b4742640 1 -- start start 2024-04-25T08:52:42.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b4742640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0ac0c9ef0 0x7fb0ac0c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b4742640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0ac0cf590 0x7fb0ac15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b4742640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15ecb0 0x7fb0ac071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b4742640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac1676f0 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b4742640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac0cdce0 con 0x7fb0ac0cf590 2024-04-25T08:52:42.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b4742640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac167870 con 0x7fb0ac0c9ef0 2024-04-25T08:52:42.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b2cdf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15ecb0 0x7fb0ac071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b2cdf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15ecb0 0x7fb0ac071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39524/0 (socket says 172.21.15.37:39524) 2024-04-25T08:52:42.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b2cdf640 1 -- 172.21.15.37:0/3368433563 learned_addr learned my addr 172.21.15.37:0/3368433563 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:42.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b24de640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0ac0c9ef0 0x7fb0ac0c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b1cdd640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0ac0cf590 0x7fb0ac15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b2cdf640 1 -- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0ac0c9ef0 msgr2=0x7fb0ac0c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b2cdf640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0ac0c9ef0 0x7fb0ac0c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b2cdf640 1 -- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0ac0cf590 msgr2=0x7fb0ac15abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b2cdf640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0ac0cf590 0x7fb0ac15abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b2cdf640 1 -- 172.21.15.37:0/3368433563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac0c5c10 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b2cdf640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15ecb0 0x7fb0ac071f10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4007ea0 tx=0x7fb0a4007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0a2ffd640 1 -- 172.21.15.37:0/3368433563 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4014620 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0a2ffd640 1 -- 172.21.15.37:0/3368433563 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a4019070 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b4742640 1 -- 172.21.15.37:0/3368433563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ac0c2f30 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0a2ffd640 1 -- 172.21.15.37:0/3368433563 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4014a20 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.672+0000 7fb0b4742640 1 -- 172.21.15.37:0/3368433563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ac0c34f0 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.676+0000 7fb0b4742640 1 -- 172.21.15.37:0/3368433563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ac04efa0 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.676+0000 7fb0a2ffd640 1 -- 172.21.15.37:0/3368433563 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0a4014bc0 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.676+0000 7fb0a2ffd640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb090066f90 0x7fb090069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.676+0000 7fb0a2ffd640 1 -- 172.21.15.37:0/3368433563 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb0a4085c70 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.676+0000 7fb0b24de640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb090066f90 0x7fb090069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.676+0000 7fb0b24de640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb090066f90 0x7fb090069450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8001650 tx=0x7fb0a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.680+0000 7fb0a2ffd640 1 -- 172.21.15.37:0/3368433563 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0a40568c0 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 -- 172.21.15.37:0/3368433563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb0ac157390 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0a2ffd640 1 -- 172.21.15.37:0/3368433563 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb0a40561c0 con 0x7fb0ac15ecb0 2024-04-25T08:52:42.833 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:52:42.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 -- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb090066f90 msgr2=0x7fb090069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb090066f90 0x7fb090069450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8001650 tx=0x7fb0a8009040 comp rx=0 tx=0).stop 2024-04-25T08:52:42.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 -- 172.21.15.37:0/3368433563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15ecb0 msgr2=0x7fb0ac071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15ecb0 0x7fb0ac071f10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4007ea0 tx=0x7fb0a4007f90 comp rx=0 tx=0).stop 2024-04-25T08:52:42.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 -- 172.21.15.37:0/3368433563 shutdown_connections 2024-04-25T08:52:42.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb090066f90 0x7fb090069450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0ac15ecb0 0x7fb0ac071f10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0ac0cf590 0x7fb0ac15abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 --2- 172.21.15.37:0/3368433563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0ac0c9ef0 0x7fb0ac0c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 -- 172.21.15.37:0/3368433563 >> 172.21.15.37:0/3368433563 conn(0x7fb0ac06ae60 msgr2=0x7fb0ac06e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:42.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 -- 172.21.15.37:0/3368433563 shutdown_connections 2024-04-25T08:52:42.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.840+0000 7fb0b4742640 1 -- 172.21.15.37:0/3368433563 wait complete. 2024-04-25T08:52:42.852 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:52:42.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f758cd89640 1 Processor -- start 2024-04-25T08:52:42.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f758cd89640 1 -- start start 2024-04-25T08:52:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f758cd89640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f75880cf590 0x7f75880cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f758cd89640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7588072000 0x7f7588072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f758cd89640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75880cbd20 con 0x7f758815b6b0 2024-04-25T08:52:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f758cd89640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758815ba90 con 0x7f75880cf590 2024-04-25T08:52:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f758cd89640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75880cc000 con 0x7f7588072000 2024-04-25T08:52:42.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f7585d9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f75880cf590 0x7f75880cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f7586d9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7588072000 0x7f7588072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f7585d9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f75880cf590 0x7f75880cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:57300/0 (socket says 172.21.15.37:57300) 2024-04-25T08:52:42.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f758659c640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f758815b6b0 0x7f75880ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:47906/0 (socket says 172.21.15.37:47906) 2024-04-25T08:52:42.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f7585d9b640 1 -- 172.21.15.37:0/2335225961 learned_addr learned my addr 172.21.15.37:0/2335225961 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f7586d9d640 1 -- 172.21.15.37:0/2335225961 >> v2:172.21.15.203:3300/0 conn(0x7f75880cf590 msgr2=0x7f75880cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f7586d9d640 1 --2- 172.21.15.37:0/2335225961 >> v2:172.21.15.203:3300/0 conn(0x7f75880cf590 0x7f75880cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f7586d9d640 1 -- 172.21.15.37:0/2335225961 >> v1:172.21.15.203:6789/0 conn(0x7f758815b6b0 legacy=0x7f75880ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f7586d9d640 1 -- 172.21.15.37:0/2335225961 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75880cc2e0 con 0x7f7588072000 2024-04-25T08:52:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.980+0000 7f7585d9b640 1 --2- 172.21.15.37:0/2335225961 >> v2:172.21.15.203:3300/0 conn(0x7f75880cf590 0x7f75880cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f7586d9d640 1 --2- 172.21.15.37:0/2335225961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7588072000 0x7f7588072470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7580009d90 tx=0x7f758001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76f5ada89ae32e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758559a640 1 -- 172.21.15.37:0/2335225961 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800b2070 con 0x7f7588072000 2024-04-25T08:52:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758559a640 1 -- 172.21.15.37:0/2335225961 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75800a5920 con 0x7f7588072000 2024-04-25T08:52:42.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758559a640 1 -- 172.21.15.37:0/2335225961 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800a5cf0 con 0x7f7588072000 2024-04-25T08:52:42.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 -- 172.21.15.37:0/2335225961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7588072000 msgr2=0x7f7588072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 --2- 172.21.15.37:0/2335225961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7588072000 0x7f7588072470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7580009d90 tx=0x7f758001c970 comp rx=0 tx=0).stop 2024-04-25T08:52:42.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 -- 172.21.15.37:0/2335225961 shutdown_connections 2024-04-25T08:52:42.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 --2- 172.21.15.37:0/2335225961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7588072000 0x7f7588072470 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 --2- 172.21.15.37:0/2335225961 >> v2:172.21.15.203:3300/0 conn(0x7f75880cf590 0x7f75880cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 -- 172.21.15.37:0/2335225961 >> 172.21.15.37:0/2335225961 conn(0x7f758806ae60 msgr2=0x7f758806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:42.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 -- 172.21.15.37:0/2335225961 shutdown_connections 2024-04-25T08:52:42.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 -- 172.21.15.37:0/2335225961 wait complete. 2024-04-25T08:52:42.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 Processor -- start 2024-04-25T08:52:42.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 -- start start 2024-04-25T08:52:42.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7588072000 0x7f7588150ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75880cf590 0x7f75881529c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f758815b6b0 0x7f75880985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75880caf30 con 0x7f758815b6b0 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75880cadb0 con 0x7f7588072000 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f758cd89640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75881682a0 con 0x7f75880cf590 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.984+0000 7f7586d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f758815b6b0 0x7f75880985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7586d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f758815b6b0 0x7f75880985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39532/0 (socket says 172.21.15.37:39532) 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7586d9d640 1 -- 172.21.15.37:0/843521134 learned_addr learned my addr 172.21.15.37:0/843521134 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f758659c640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7588072000 0x7f7588150ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7585d9b640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75880cf590 0x7f75881529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7586d9d640 1 -- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75880cf590 msgr2=0x7f75881529c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7586d9d640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75880cf590 0x7f75881529c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7586d9d640 1 -- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7588072000 msgr2=0x7f7588150ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7586d9d640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7588072000 0x7f7588150ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:42.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7586d9d640 1 -- 172.21.15.37:0/843521134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7588151530 con 0x7f758815b6b0 2024-04-25T08:52:42.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f758659c640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7588072000 0x7f7588150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:42.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7586d9d640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f758815b6b0 0x7f75880985b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f75800b0040 tx=0x7f7580019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7576ffd640 1 -- 172.21.15.37:0/843521134 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800b2050 con 0x7f758815b6b0 2024-04-25T08:52:42.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7576ffd640 1 -- 172.21.15.37:0/843521134 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75800be070 con 0x7f758815b6b0 2024-04-25T08:52:42.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7576ffd640 1 -- 172.21.15.37:0/843521134 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800ada90 con 0x7f758815b6b0 2024-04-25T08:52:42.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f758cd89640 1 -- 172.21.15.37:0/843521134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75881517f0 con 0x7f758815b6b0 2024-04-25T08:52:42.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f758cd89640 1 -- 172.21.15.37:0/843521134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7588151db0 con 0x7f758815b6b0 2024-04-25T08:52:42.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f758cd89640 1 -- 172.21.15.37:0/843521134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758804efa0 con 0x7f758815b6b0 2024-04-25T08:52:42.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7576ffd640 1 -- 172.21.15.37:0/843521134 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75880731c0 con 0x7f758815b6b0 2024-04-25T08:52:42.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7576ffd640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7568066f90 0x7f7568069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:42.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f7576ffd640 1 -- 172.21.15.37:0/843521134 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7580120200 con 0x7f758815b6b0 2024-04-25T08:52:42.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f758659c640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7568066f90 0x7f7568069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:42.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.988+0000 7f758659c640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7568066f90 0x7f7568069450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7578000f70 tx=0x7f7578007d70 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:42.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:42.992+0000 7f7576ffd640 1 -- 172.21.15.37:0/843521134 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7580016560 con 0x7f758815b6b0 2024-04-25T08:52:43.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.148+0000 7f758cd89640 1 -- 172.21.15.37:0/843521134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7588158a70 con 0x7f758815b6b0 2024-04-25T08:52:43.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.148+0000 7f7576ffd640 1 -- 172.21.15.37:0/843521134 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f75800f0f60 con 0x7f758815b6b0 2024-04-25T08:52:43.143 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:52:43.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 -- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7568066f90 msgr2=0x7f7568069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7568066f90 0x7f7568069450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7578000f70 tx=0x7f7578007d70 comp rx=0 tx=0).stop 2024-04-25T08:52:43.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 -- 172.21.15.37:0/843521134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f758815b6b0 msgr2=0x7f75880985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f758815b6b0 0x7f75880985b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f75800b0040 tx=0x7f7580019710 comp rx=0 tx=0).stop 2024-04-25T08:52:43.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 -- 172.21.15.37:0/843521134 shutdown_connections 2024-04-25T08:52:43.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7568066f90 0x7f7568069450 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f758815b6b0 0x7f75880985b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75880cf590 0x7f75881529c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 --2- 172.21.15.37:0/843521134 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7588072000 0x7f7588150ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 -- 172.21.15.37:0/843521134 >> 172.21.15.37:0/843521134 conn(0x7f758806ae60 msgr2=0x7f7588157af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:43.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 -- 172.21.15.37:0/843521134 shutdown_connections 2024-04-25T08:52:43.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.152+0000 7f758cd89640 1 -- 172.21.15.37:0/843521134 wait complete. 2024-04-25T08:52:43.160 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:52:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e84a38640 1 Processor -- start 2024-04-25T08:52:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e84a38640 1 -- start start 2024-04-25T08:52:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e84a38640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0e8009a180 0x7f0e800cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e84a38640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0e800afb90 0x7f0e800cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e84a38640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8009a950 con 0x7f0e8015b920 2024-04-25T08:52:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e84a38640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8009ac30 con 0x7f0e8009a180 2024-04-25T08:52:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e84a38640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e800ae380 con 0x7f0e800afb90 2024-04-25T08:52:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e7dd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0e8009a180 0x7f0e800cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e7ed9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0e800afb90 0x7f0e800cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e7dd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0e8009a180 0x7f0e800cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39544/0 (socket says 172.21.15.37:39544) 2024-04-25T08:52:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e7dd9b640 1 -- 172.21.15.37:0/3536769692 learned_addr learned my addr 172.21.15.37:0/3536769692 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e7dd9b640 1 -- 172.21.15.37:0/3536769692 >> v2:172.21.15.203:3300/0 conn(0x7f0e800afb90 msgr2=0x7f0e800cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e7dd9b640 1 --2- 172.21.15.37:0/3536769692 >> v2:172.21.15.203:3300/0 conn(0x7f0e800afb90 0x7f0e800cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e7dd9b640 1 -- 172.21.15.37:0/3536769692 >> v1:172.21.15.203:6789/0 conn(0x7f0e8015b920 legacy=0x7f0e80099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.288+0000 7f0e7dd9b640 1 -- 172.21.15.37:0/3536769692 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e800ae660 con 0x7f0e8009a180 2024-04-25T08:52:43.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e7dd9b640 1 --2- 172.21.15.37:0/3536769692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e8009a180 0x7f0e800cc130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e800b7550 tx=0x7f0e6801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ab6b16055af854 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e7d59a640 1 -- 172.21.15.37:0/3536769692 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e680ab430 con 0x7f0e8009a180 2024-04-25T08:52:43.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e7d59a640 1 -- 172.21.15.37:0/3536769692 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e680b0070 con 0x7f0e8009a180 2024-04-25T08:52:43.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e7d59a640 1 -- 172.21.15.37:0/3536769692 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e680ab7f0 con 0x7f0e8009a180 2024-04-25T08:52:43.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e84a38640 1 -- 172.21.15.37:0/3536769692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e8009a180 msgr2=0x7f0e800cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e84a38640 1 --2- 172.21.15.37:0/3536769692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e8009a180 0x7f0e800cc130 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0e800b7550 tx=0x7f0e6801c520 comp rx=0 tx=0).stop 2024-04-25T08:52:43.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e84a38640 1 -- 172.21.15.37:0/3536769692 shutdown_connections 2024-04-25T08:52:43.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e84a38640 1 --2- 172.21.15.37:0/3536769692 >> v2:172.21.15.203:3300/0 conn(0x7f0e800afb90 0x7f0e800cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e84a38640 1 --2- 172.21.15.37:0/3536769692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e8009a180 0x7f0e800cc130 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e84a38640 1 -- 172.21.15.37:0/3536769692 >> 172.21.15.37:0/3536769692 conn(0x7f0e800b3010 msgr2=0x7f0e800b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:43.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e84a38640 1 -- 172.21.15.37:0/3536769692 shutdown_connections 2024-04-25T08:52:43.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e84a38640 1 -- 172.21.15.37:0/3536769692 wait complete. 2024-04-25T08:52:43.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e84a38640 1 Processor -- start 2024-04-25T08:52:43.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.292+0000 7f0e84a38640 1 -- start start 2024-04-25T08:52:43.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e84a38640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e8009a180 0x7f0e80151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e84a38640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e800afb90 0x7f0e801f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e84a38640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e8015b920 0x7f0e8006f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e84a38640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e801684c0 con 0x7f0e8009a180 2024-04-25T08:52:43.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e84a38640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e800cd5d0 con 0x7f0e800afb90 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e84a38640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e80168640 con 0x7f0e8015b920 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e8009a180 0x7f0e80151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e8009a180 0x7f0e80151750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39552/0 (socket says 172.21.15.37:39552) 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7e59c640 1 -- 172.21.15.37:0/2452593372 learned_addr learned my addr 172.21.15.37:0/2452593372 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7dd9b640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e800afb90 0x7f0e801f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7ed9d640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e8015b920 0x7f0e8006f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7e59c640 1 -- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e8015b920 msgr2=0x7f0e8006f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7e59c640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e8015b920 0x7f0e8006f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7e59c640 1 -- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e800afb90 msgr2=0x7f0e801f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7e59c640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e800afb90 0x7f0e801f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7e59c640 1 -- 172.21.15.37:0/2452593372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e801f5380 con 0x7f0e8009a180 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7e59c640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e8009a180 0x7f0e80151750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c820 tx=0x7f0e7800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e76ffd640 1 -- 172.21.15.37:0/2452593372 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78003b80 con 0x7f0e8009a180 2024-04-25T08:52:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e76ffd640 1 -- 172.21.15.37:0/2452593372 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e78003d20 con 0x7f0e8009a180 2024-04-25T08:52:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e84a38640 1 -- 172.21.15.37:0/2452593372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e801f5660 con 0x7f0e8009a180 2024-04-25T08:52:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e76ffd640 1 -- 172.21.15.37:0/2452593372 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7800aa90 con 0x7f0e8009a180 2024-04-25T08:52:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e84a38640 1 -- 172.21.15.37:0/2452593372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e801f4f20 con 0x7f0e8009a180 2024-04-25T08:52:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e84a38640 1 -- 172.21.15.37:0/2452593372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e8004efa0 con 0x7f0e8009a180 2024-04-25T08:52:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e76ffd640 1 -- 172.21.15.37:0/2452593372 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e7800ac30 con 0x7f0e8009a180 2024-04-25T08:52:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e76ffd640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 0x7f0e60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e76ffd640 1 -- 172.21.15.37:0/2452593372 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0e78085410 con 0x7f0e8009a180 2024-04-25T08:52:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.296+0000 7f0e7dd9b640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 0x7f0e60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.300+0000 7f0e7dd9b640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 0x7f0e60069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0e680af000 tx=0x7f0e680a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.300+0000 7f0e76ffd640 1 -- 172.21.15.37:0/2452593372 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e780560e0 con 0x7f0e8009a180 2024-04-25T08:52:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.460+0000 7f0e84a38640 1 -- 172.21.15.37:0/2452593372 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0e80159310 con 0x7f0e8009a180 2024-04-25T08:52:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.460+0000 7f0e76ffd640 1 -- 172.21.15.37:0/2452593372 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0e780559e0 con 0x7f0e8009a180 2024-04-25T08:52:43.455 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:52:43.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 -- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 msgr2=0x7f0e60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 0x7f0e60069420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0e680af000 tx=0x7f0e680a7000 comp rx=0 tx=0).stop 2024-04-25T08:52:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 -- 172.21.15.37:0/2452593372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e8009a180 msgr2=0x7f0e80151750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e8009a180 0x7f0e80151750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0e7800c820 tx=0x7f0e7800cce0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 -- 172.21.15.37:0/2452593372 shutdown_connections 2024-04-25T08:52:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 0x7f0e60069420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e8015b920 0x7f0e8006f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e800afb90 0x7f0e801f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 --2- 172.21.15.37:0/2452593372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e8009a180 0x7f0e80151750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 -- 172.21.15.37:0/2452593372 >> 172.21.15.37:0/2452593372 conn(0x7f0e800b3010 msgr2=0x7f0e800b0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 -- 172.21.15.37:0/2452593372 shutdown_connections 2024-04-25T08:52:43.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.464+0000 7f0e84a38640 1 -- 172.21.15.37:0/2452593372 wait complete. 2024-04-25T08:52:43.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:52:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd303eb7640 1 Processor -- start 2024-04-25T08:52:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd303eb7640 1 -- start start 2024-04-25T08:52:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd303eb7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd303eb7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd2fc1451f0 0x7fd2fc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd303eb7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2fc0cee30 con 0x7fd2fc144ca0 2024-04-25T08:52:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd303eb7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2fc0cc2b0 con 0x7fd2fc1451f0 2024-04-25T08:52:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd303eb7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc06e290 con 0x7fd2fc0ce510 2024-04-25T08:52:43.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd302454640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd2fc1451f0 0x7fd2fc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd302454640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd2fc1451f0 0x7fd2fc142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39556/0 (socket says 172.21.15.37:39556) 2024-04-25T08:52:43.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd302454640 1 -- 172.21.15.37:0/2279705101 learned_addr learned my addr 172.21.15.37:0/2279705101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:43.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd301c53640 1 --2- 172.21.15.37:0/2279705101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd302454640 1 -- 172.21.15.37:0/2279705101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2fc0ce510 msgr2=0x7fd2fc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:43.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd302454640 1 --2- 172.21.15.37:0/2279705101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd302454640 1 -- 172.21.15.37:0/2279705101 >> v1:172.21.15.203:6789/0 conn(0x7fd2fc144ca0 legacy=0x7fd2fc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:43.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.596+0000 7fd302454640 1 -- 172.21.15.37:0/2279705101 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc0ccc70 con 0x7fd2fc1451f0 2024-04-25T08:52:43.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd302454640 1 --2- 172.21.15.37:0/2279705101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2fc1451f0 0x7fd2fc142680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009870 tx=0x7fd2f801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dba9711f3403e20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd300c51640 1 -- 172.21.15.37:0/2279705101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80b5070 con 0x7fd2fc1451f0 2024-04-25T08:52:43.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd300c51640 1 -- 172.21.15.37:0/2279705101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f80a89d0 con 0x7fd2fc1451f0 2024-04-25T08:52:43.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd300c51640 1 -- 172.21.15.37:0/2279705101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80a8da0 con 0x7fd2fc1451f0 2024-04-25T08:52:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 -- 172.21.15.37:0/2279705101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2fc1451f0 msgr2=0x7fd2fc142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 --2- 172.21.15.37:0/2279705101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2fc1451f0 0x7fd2fc142680 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009870 tx=0x7fd2f801ed10 comp rx=0 tx=0).stop 2024-04-25T08:52:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 -- 172.21.15.37:0/2279705101 shutdown_connections 2024-04-25T08:52:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 --2- 172.21.15.37:0/2279705101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2fc1451f0 0x7fd2fc142680 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 --2- 172.21.15.37:0/2279705101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 -- 172.21.15.37:0/2279705101 >> 172.21.15.37:0/2279705101 conn(0x7fd2fc0b11a0 msgr2=0x7fd2fc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 -- 172.21.15.37:0/2279705101 shutdown_connections 2024-04-25T08:52:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 -- 172.21.15.37:0/2279705101 wait complete. 2024-04-25T08:52:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 Processor -- start 2024-04-25T08:52:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 -- start start 2024-04-25T08:52:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc15b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2fc144ca0 0x7fd2fc1e6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2fc1451f0 0x7fd2fc1e7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc06ee30 con 0x7fd2fc1451f0 2024-04-25T08:52:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc06df20 con 0x7fd2fc144ca0 2024-04-25T08:52:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd303eb7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc0cc900 con 0x7fd2fc0ce510 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd302454640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2fc1451f0 0x7fd2fc1e7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd302454640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2fc1451f0 0x7fd2fc1e7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39564/0 (socket says 172.21.15.37:39564) 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd302454640 1 -- 172.21.15.37:0/132131478 learned_addr learned my addr 172.21.15.37:0/132131478 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd301c53640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc15b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd301452640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2fc144ca0 0x7fd2fc1e6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd302454640 1 -- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2fc0ce510 msgr2=0x7fd2fc15b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd302454640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc15b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd302454640 1 -- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2fc144ca0 msgr2=0x7fd2fc1e6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd302454640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2fc144ca0 0x7fd2fc1e6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd302454640 1 -- 172.21.15.37:0/132131478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc1e21a0 con 0x7fd2fc1451f0 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd301452640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2fc144ca0 0x7fd2fc1e6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd302454640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2fc1451f0 0x7fd2fc1e7360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009870 tx=0x7fd2f801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.600+0000 7fd2f27fc640 1 -- 172.21.15.37:0/132131478 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80c0070 con 0x7fd2fc1451f0 2024-04-25T08:52:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.604+0000 7fd2f27fc640 1 -- 172.21.15.37:0/132131478 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f80b0800 con 0x7fd2fc1451f0 2024-04-25T08:52:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.604+0000 7fd303eb7640 1 -- 172.21.15.37:0/132131478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc1e2480 con 0x7fd2fc1451f0 2024-04-25T08:52:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.604+0000 7fd2f27fc640 1 -- 172.21.15.37:0/132131478 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f80b5040 con 0x7fd2fc1451f0 2024-04-25T08:52:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.604+0000 7fd303eb7640 1 -- 172.21.15.37:0/132131478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc1e2a40 con 0x7fd2fc1451f0 2024-04-25T08:52:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.604+0000 7fd303eb7640 1 -- 172.21.15.37:0/132131478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc04efa0 con 0x7fd2fc1451f0 2024-04-25T08:52:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.604+0000 7fd2f27fc640 1 -- 172.21.15.37:0/132131478 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2f80c3070 con 0x7fd2fc1451f0 2024-04-25T08:52:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.604+0000 7fd2f27fc640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2e0066f60 0x7fd2e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.604+0000 7fd2f27fc640 1 -- 172.21.15.37:0/132131478 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd2f8122110 con 0x7fd2fc1451f0 2024-04-25T08:52:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.604+0000 7fd301c53640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2e0066f60 0x7fd2e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.604+0000 7fd301c53640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2e0066f60 0x7fd2e0069420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4003fd0 tx=0x7fd2f40039a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.608+0000 7fd2f27fc640 1 -- 172.21.15.37:0/132131478 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2f8018870 con 0x7fd2fc1451f0 2024-04-25T08:52:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.772+0000 7fd303eb7640 1 -- 172.21.15.37:0/132131478 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd2fc0b1cc0 con 0x7fd2fc1451f0 2024-04-25T08:52:43.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.772+0000 7fd2f27fc640 1 -- 172.21.15.37:0/132131478 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd2f80f2e70 con 0x7fd2fc1451f0 2024-04-25T08:52:43.766 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:52:43.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 -- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2e0066f60 msgr2=0x7fd2e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2e0066f60 0x7fd2e0069420 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4003fd0 tx=0x7fd2f40039a0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 -- 172.21.15.37:0/132131478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2fc1451f0 msgr2=0x7fd2fc1e7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2fc1451f0 0x7fd2fc1e7360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd2f8009870 tx=0x7fd2f801bab0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 -- 172.21.15.37:0/132131478 shutdown_connections 2024-04-25T08:52:43.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2e0066f60 0x7fd2e0069420 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2fc1451f0 0x7fd2fc1e7360 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2fc144ca0 0x7fd2fc1e6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 --2- 172.21.15.37:0/132131478 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2fc0ce510 0x7fd2fc15b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 -- 172.21.15.37:0/132131478 >> 172.21.15.37:0/132131478 conn(0x7fd2fc0b11a0 msgr2=0x7fd2fc0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:43.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 -- 172.21.15.37:0/132131478 shutdown_connections 2024-04-25T08:52:43.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.776+0000 7fd303eb7640 1 -- 172.21.15.37:0/132131478 wait complete. 2024-04-25T08:52:43.783 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:52:43.783 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:52:43.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:43.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6afd872640 1 Processor -- start 2024-04-25T08:52:43.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6afd872640 1 -- start start 2024-04-25T08:52:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6afd872640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6af80cc820 0x7f6af80b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6afd872640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6af80b1910 0x7f6af8151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6afd872640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af8160310 con 0x7f6af80b0b00 2024-04-25T08:52:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6afd872640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af815cfa0 con 0x7f6af80b1910 2024-04-25T08:52:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6afd872640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af815cdf0 con 0x7f6af80cc820 2024-04-25T08:52:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6af6ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6af80b0b00 0x7f6af80cc110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47512/0 (socket says 172.21.15.37:47512) 2024-04-25T08:52:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6af6ffd640 1 -- 172.21.15.37:0/1974384117 learned_addr learned my addr 172.21.15.37:0/1974384117 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6af77fe640 1 --2- 172.21.15.37:0/1974384117 >> v2:172.21.15.37:3300/0 conn(0x7f6af80b1910 0x7f6af8151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6aeffff640 1 --2- 172.21.15.37:0/1974384117 >> v2:172.21.15.203:3300/0 conn(0x7f6af80cc820 0x7f6af80b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6af67fc640 1 -- 172.21.15.37:0/1974384117 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372280684 0 0) 0x7f6af8160310 con 0x7f6af80b0b00 2024-04-25T08:52:43.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6af67fc640 1 -- 172.21.15.37:0/1974384117 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae4003530 con 0x7f6af80b0b00 2024-04-25T08:52:43.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6af77fe640 1 -- 172.21.15.37:0/1974384117 >> v2:172.21.15.203:3300/0 conn(0x7f6af80cc820 msgr2=0x7f6af80b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:43.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6af77fe640 1 --2- 172.21.15.37:0/1974384117 >> v2:172.21.15.203:3300/0 conn(0x7f6af80cc820 0x7f6af80b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6af77fe640 1 -- 172.21.15.37:0/1974384117 >> v1:172.21.15.37:6789/0 conn(0x7f6af80b0b00 legacy=0x7f6af80cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.956+0000 7f6af77fe640 1 -- 172.21.15.37:0/1974384117 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af815f550 con 0x7f6af80b1910 2024-04-25T08:52:43.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6af77fe640 1 --2- 172.21.15.37:0/1974384117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6af80b1910 0x7f6af8151fd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6af80ca880 tx=0x7f6af001c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b359eecd5d34f510 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6af67fc640 1 -- 172.21.15.37:0/1974384117 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af00a6620 con 0x7f6af80b1910 2024-04-25T08:52:43.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6af67fc640 1 -- 172.21.15.37:0/1974384117 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af00a67c0 con 0x7f6af80b1910 2024-04-25T08:52:43.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6af67fc640 1 -- 172.21.15.37:0/1974384117 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af00a6b90 con 0x7f6af80b1910 2024-04-25T08:52:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 -- 172.21.15.37:0/1974384117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6af80b1910 msgr2=0x7f6af8151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 --2- 172.21.15.37:0/1974384117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6af80b1910 0x7f6af8151fd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6af80ca880 tx=0x7f6af001c750 comp rx=0 tx=0).stop 2024-04-25T08:52:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 -- 172.21.15.37:0/1974384117 shutdown_connections 2024-04-25T08:52:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 --2- 172.21.15.37:0/1974384117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6af80b1910 0x7f6af8151fd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 --2- 172.21.15.37:0/1974384117 >> v2:172.21.15.203:3300/0 conn(0x7f6af80cc820 0x7f6af80b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 -- 172.21.15.37:0/1974384117 >> 172.21.15.37:0/1974384117 conn(0x7f6af80b1f10 msgr2=0x7f6af80b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 -- 172.21.15.37:0/1974384117 shutdown_connections 2024-04-25T08:52:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 -- 172.21.15.37:0/1974384117 wait complete. 2024-04-25T08:52:43.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 Processor -- start 2024-04-25T08:52:43.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 -- start start 2024-04-25T08:52:43.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af80b0b00 0x7f6af81514e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6af80b1910 0x7f6af80adc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6af8151d10 0x7f6af80b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80b1060 con 0x7f6af80b1910 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80b0ee0 con 0x7f6af8151d10 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af815bde0 con 0x7f6af80b0b00 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6aeffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6af80b1910 0x7f6af80adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6aeffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6af80b1910 0x7f6af80adc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39572/0 (socket says 172.21.15.37:39572) 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6aeffff640 1 -- 172.21.15.37:0/1279310161 learned_addr learned my addr 172.21.15.37:0/1279310161 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6aeffff640 1 -- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af80b0b00 msgr2=0x7f6af81514e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6aeffff640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af80b0b00 0x7f6af81514e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6aeffff640 1 -- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6af8151d10 msgr2=0x7f6af80b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6aeffff640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6af8151d10 0x7f6af80b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6aeffff640 1 -- 172.21.15.37:0/1279310161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af80af610 con 0x7f6af80b1910 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6aeffff640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6af80b1910 0x7f6af80adc00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000d5f0 tx=0x7f6ae000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6af4ff9640 1 -- 172.21.15.37:0/1279310161 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00125f0 con 0x7f6af80b1910 2024-04-25T08:52:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6af4ff9640 1 -- 172.21.15.37:0/1279310161 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae0019070 con 0x7f6af80b1910 2024-04-25T08:52:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 -- 172.21.15.37:0/1279310161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af80ae570 con 0x7f6af80b1910 2024-04-25T08:52:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6af4ff9640 1 -- 172.21.15.37:0/1279310161 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00129b0 con 0x7f6af80b1910 2024-04-25T08:52:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.960+0000 7f6afd872640 1 -- 172.21.15.37:0/1279310161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af80aeb30 con 0x7f6af80b1910 2024-04-25T08:52:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.964+0000 7f6afd872640 1 -- 172.21.15.37:0/1279310161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af804efa0 con 0x7f6af80b1910 2024-04-25T08:52:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.964+0000 7f6af4ff9640 1 -- 172.21.15.37:0/1279310161 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ae0012b50 con 0x7f6af80b1910 2024-04-25T08:52:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.964+0000 7f6af4ff9640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6ad8066f60 0x7f6ad8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.964+0000 7f6af4ff9640 1 -- 172.21.15.37:0/1279310161 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6ae00857e0 con 0x7f6af80b1910 2024-04-25T08:52:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.964+0000 7f6af6ffd640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6ad8066f60 0x7f6ad8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.964+0000 7f6af6ffd640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6ad8066f60 0x7f6ad8069420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8007fd0 tx=0x7f6ae8007900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:43.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:43.968+0000 7f6af4ff9640 1 -- 172.21.15.37:0/1279310161 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6af80af610 con 0x7f6af80b1910 2024-04-25T08:52:44.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.124+0000 7f6afd872640 1 -- 172.21.15.37:0/1279310161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6af806f120 con 0x7f6af80b1910 2024-04-25T08:52:44.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6af4ff9640 1 -- 172.21.15.37:0/1279310161 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f6ae0056420 con 0x7f6af80b1910 2024-04-25T08:52:44.120 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:44.120 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:38.282648+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:44.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 -- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6ad8066f60 msgr2=0x7f6ad8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6ad8066f60 0x7f6ad8069420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ae8007fd0 tx=0x7f6ae8007900 comp rx=0 tx=0).stop 2024-04-25T08:52:44.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 -- 172.21.15.37:0/1279310161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6af80b1910 msgr2=0x7f6af80adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6af80b1910 0x7f6af80adc00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000d5f0 tx=0x7f6ae000dab0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 -- 172.21.15.37:0/1279310161 shutdown_connections 2024-04-25T08:52:44.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6ad8066f60 0x7f6ad8069420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6af8151d10 0x7f6af80b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6af80b1910 0x7f6af80adc00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 --2- 172.21.15.37:0/1279310161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6af80b0b00 0x7f6af81514e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 -- 172.21.15.37:0/1279310161 >> 172.21.15.37:0/1279310161 conn(0x7f6af80b1f10 msgr2=0x7f6af806f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:44.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 -- 172.21.15.37:0/1279310161 shutdown_connections 2024-04-25T08:52:44.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.128+0000 7f6afd872640 1 -- 172.21.15.37:0/1279310161 wait complete. 2024-04-25T08:52:44.138 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:52:44.139 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:52:44.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7de4ec640 1 Processor -- start 2024-04-25T08:52:44.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7de4ec640 1 -- start start 2024-04-25T08:52:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7de4ec640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa7d80cc130 0x7fa7d8166650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7de4ec640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d815f360 con 0x7fa7d806ac90 2024-04-25T08:52:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7de4ec640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d815f640 con 0x7fa7d815ca90 2024-04-25T08:52:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7de4ec640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d815ff70 con 0x7fa7d80cc130 2024-04-25T08:52:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7dca89640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa7d80cc130 0x7fa7d8166650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7d7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa7d806ac90 0x7fa7d806b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47528/0 (socket says 172.21.15.37:47528) 2024-04-25T08:52:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7d7fff640 1 -- 172.21.15.37:0/3324510869 learned_addr learned my addr 172.21.15.37:0/3324510869 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7dca89640 1 -- 172.21.15.37:0/3324510869 >> v1:172.21.15.203:6789/0 conn(0x7fa7d815ca90 legacy=0x7fa7d80d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7dca89640 1 -- 172.21.15.37:0/3324510869 >> v1:172.21.15.37:6789/0 conn(0x7fa7d806ac90 legacy=0x7fa7d806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7d6ffd640 1 -- 172.21.15.37:0/3324510869 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266560743 0 0) 0x7fa7d815f360 con 0x7fa7d806ac90 2024-04-25T08:52:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.272+0000 7fa7dca89640 1 -- 172.21.15.37:0/3324510869 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d815f920 con 0x7fa7d80cc130 2024-04-25T08:52:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7dca89640 1 --2- 172.21.15.37:0/3324510869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d80cc130 0x7fa7d8166650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00097c0 tx=0x7fa7d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303603fc93a8113d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7d6ffd640 1 -- 172.21.15.37:0/3324510869 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00b0070 con 0x7fa7d80cc130 2024-04-25T08:52:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7d6ffd640 1 -- 172.21.15.37:0/3324510869 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d00ab600 con 0x7fa7d80cc130 2024-04-25T08:52:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7d6ffd640 1 -- 172.21.15.37:0/3324510869 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00ab9d0 con 0x7fa7d80cc130 2024-04-25T08:52:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 -- 172.21.15.37:0/3324510869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d80cc130 msgr2=0x7fa7d8166650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 --2- 172.21.15.37:0/3324510869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d80cc130 0x7fa7d8166650 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00097c0 tx=0x7fa7d001c460 comp rx=0 tx=0).stop 2024-04-25T08:52:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 -- 172.21.15.37:0/3324510869 shutdown_connections 2024-04-25T08:52:44.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 --2- 172.21.15.37:0/3324510869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d80cc130 0x7fa7d8166650 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 -- 172.21.15.37:0/3324510869 >> 172.21.15.37:0/3324510869 conn(0x7fa7d8099a70 msgr2=0x7fa7d80d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:44.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 -- 172.21.15.37:0/3324510869 shutdown_connections 2024-04-25T08:52:44.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 -- 172.21.15.37:0/3324510869 wait complete. 2024-04-25T08:52:44.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 Processor -- start 2024-04-25T08:52:44.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 -- start start 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d806ac90 0x7fa7d80c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7d80cc130 0x7fa7d81507f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d815ca90 0x7fa7d80b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d8169940 con 0x7fa7d815ca90 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d81697c0 con 0x7fa7d806ac90 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7de4ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d8169ac0 con 0x7fa7d80cc130 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7dca89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d815ca90 0x7fa7d80b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7dca89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d815ca90 0x7fa7d80b3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39588/0 (socket says 172.21.15.37:39588) 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.276+0000 7fa7dca89640 1 -- 172.21.15.37:0/1491404521 learned_addr learned my addr 172.21.15.37:0/1491404521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7d7fff640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d806ac90 0x7fa7d80c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7d77fe640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7d80cc130 0x7fa7d81507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7dca89640 1 -- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7d80cc130 msgr2=0x7fa7d81507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7dca89640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7d80cc130 0x7fa7d81507f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7dca89640 1 -- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d806ac90 msgr2=0x7fa7d80c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7dca89640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d806ac90 0x7fa7d80c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7dca89640 1 -- 172.21.15.37:0/1491404521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d80c9d30 con 0x7fa7d815ca90 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7d77fe640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7d80cc130 0x7fa7d81507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7dca89640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d815ca90 0x7fa7d80b3340 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00ae040 tx=0x7fa7d0019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7d4ff9640 1 -- 172.21.15.37:0/1491404521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00b0040 con 0x7fa7d815ca90 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7d4ff9640 1 -- 172.21.15.37:0/1491404521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d00b9070 con 0x7fa7d815ca90 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7de4ec640 1 -- 172.21.15.37:0/1491404521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d80ca010 con 0x7fa7d815ca90 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7d4ff9640 1 -- 172.21.15.37:0/1491404521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00b4630 con 0x7fa7d815ca90 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7de4ec640 1 -- 172.21.15.37:0/1491404521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d80b4f30 con 0x7fa7d815ca90 2024-04-25T08:52:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7de4ec640 1 -- 172.21.15.37:0/1491404521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d804efa0 con 0x7fa7d815ca90 2024-04-25T08:52:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7d4ff9640 1 -- 172.21.15.37:0/1491404521 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7d00b47d0 con 0x7fa7d815ca90 2024-04-25T08:52:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7d4ff9640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c4066f60 0x7fa7c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7d4ff9640 1 -- 172.21.15.37:0/1491404521 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa7d011f100 con 0x7fa7d815ca90 2024-04-25T08:52:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7d7fff640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c4066f60 0x7fa7c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.280+0000 7fa7d7fff640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c4066f60 0x7fa7c4069420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8007fd0 tx=0x7fa7c8007ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.284+0000 7fa7d4ff9640 1 -- 172.21.15.37:0/1491404521 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7d0016050 con 0x7fa7d815ca90 2024-04-25T08:52:44.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 -- 172.21.15.37:0/1491404521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7d80b5ad0 con 0x7fa7d815ca90 2024-04-25T08:52:44.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7d4ff9640 1 -- 172.21.15.37:0/1491404521 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7d00efe60 con 0x7fa7d815ca90 2024-04-25T08:52:44.437 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:52:44.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 -- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c4066f60 msgr2=0x7fa7c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c4066f60 0x7fa7c4069420 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8007fd0 tx=0x7fa7c8007ee0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 -- 172.21.15.37:0/1491404521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d815ca90 msgr2=0x7fa7d80b3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d815ca90 0x7fa7d80b3340 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00ae040 tx=0x7fa7d0019200 comp rx=0 tx=0).stop 2024-04-25T08:52:44.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 -- 172.21.15.37:0/1491404521 shutdown_connections 2024-04-25T08:52:44.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c4066f60 0x7fa7c4069420 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d815ca90 0x7fa7d80b3340 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7d80cc130 0x7fa7d81507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 --2- 172.21.15.37:0/1491404521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d806ac90 0x7fa7d80c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 -- 172.21.15.37:0/1491404521 >> 172.21.15.37:0/1491404521 conn(0x7fa7d8099a70 msgr2=0x7fa7d815cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:44.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 -- 172.21.15.37:0/1491404521 shutdown_connections 2024-04-25T08:52:44.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.444+0000 7fa7de4ec640 1 -- 172.21.15.37:0/1491404521 wait complete. 2024-04-25T08:52:44.456 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:52:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.584+0000 7fc3e262c640 1 Processor -- start 2024-04-25T08:52:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.584+0000 7fc3e262c640 1 -- start start 2024-04-25T08:52:44.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.584+0000 7fc3e262c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc3dc0cf840 0x7fc3dc0cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.584+0000 7fc3e262c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3dc06b710 con 0x7fc3dc15d180 2024-04-25T08:52:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.584+0000 7fc3e262c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3dc1617b0 con 0x7fc3dc06fc50 2024-04-25T08:52:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.584+0000 7fc3e262c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3dc161560 con 0x7fc3dc0cf840 2024-04-25T08:52:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.584+0000 7fc3db7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc3dc0cf840 0x7fc3dc0cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.584+0000 7fc3db7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc3dc0cf840 0x7fc3dc0cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39596/0 (socket says 172.21.15.37:39596) 2024-04-25T08:52:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.584+0000 7fc3db7fe640 1 -- 172.21.15.37:0/1027275556 learned_addr learned my addr 172.21.15.37:0/1027275556 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.584+0000 7fc3e0bc9640 1 --1- 172.21.15.37:0/1027275556 >> v1:172.21.15.37:6789/0 conn(0x7fc3dc15d180 0x7fc3dc15d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:47534/0 (socket says 172.21.15.37:47534) 2024-04-25T08:52:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3db7fe640 1 -- 172.21.15.37:0/1027275556 >> v1:172.21.15.203:6789/0 conn(0x7fc3dc06fc50 legacy=0x7fc3dc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3db7fe640 1 -- 172.21.15.37:0/1027275556 >> v1:172.21.15.37:6789/0 conn(0x7fc3dc15d180 legacy=0x7fc3dc15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3daffd640 1 -- 172.21.15.37:0/1027275556 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996460103 0 0) 0x7fc3dc06b710 con 0x7fc3dc15d180 2024-04-25T08:52:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3db7fe640 1 -- 172.21.15.37:0/1027275556 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc161a90 con 0x7fc3dc0cf840 2024-04-25T08:52:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3db7fe640 1 --2- 172.21.15.37:0/1027275556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3dc0cf840 0x7fc3dc0cfcd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009870 tx=0x7fc3d001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe230c6b65bc722 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3daffd640 1 -- 172.21.15.37:0/1027275556 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d00ab430 con 0x7fc3dc0cf840 2024-04-25T08:52:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3daffd640 1 -- 172.21.15.37:0/1027275556 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d00b0070 con 0x7fc3dc0cf840 2024-04-25T08:52:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3daffd640 1 -- 172.21.15.37:0/1027275556 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d00ab7f0 con 0x7fc3dc0cf840 2024-04-25T08:52:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3e262c640 1 -- 172.21.15.37:0/1027275556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3dc0cf840 msgr2=0x7fc3dc0cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3e262c640 1 --2- 172.21.15.37:0/1027275556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3dc0cf840 0x7fc3dc0cfcd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009870 tx=0x7fc3d001c520 comp rx=0 tx=0).stop 2024-04-25T08:52:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3e262c640 1 -- 172.21.15.37:0/1027275556 shutdown_connections 2024-04-25T08:52:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3e262c640 1 --2- 172.21.15.37:0/1027275556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3dc0cf840 0x7fc3dc0cfcd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3e262c640 1 -- 172.21.15.37:0/1027275556 >> 172.21.15.37:0/1027275556 conn(0x7fc3dc099a70 msgr2=0x7fc3dc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3e262c640 1 -- 172.21.15.37:0/1027275556 shutdown_connections 2024-04-25T08:52:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.588+0000 7fc3e262c640 1 -- 172.21.15.37:0/1027275556 wait complete. 2024-04-25T08:52:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e262c640 1 Processor -- start 2024-04-25T08:52:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e262c640 1 -- start start 2024-04-25T08:52:44.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e262c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3dc06fc50 0x7fc3dc15d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e262c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3dc0cf840 0x7fc3dc0b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e262c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3dc15d180 0x7fc3dc0b5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e262c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc070030 con 0x7fc3dc0cf840 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e262c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc0d0e30 con 0x7fc3dc06fc50 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e262c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc0701b0 con 0x7fc3dc15d180 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3dc0cf840 0x7fc3dc0b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3dc0cf840 0x7fc3dc0b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39604/0 (socket says 172.21.15.37:39604) 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3db7fe640 1 -- 172.21.15.37:0/4216049976 learned_addr learned my addr 172.21.15.37:0/4216049976 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e0bc9640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3dc15d180 0x7fc3dc0b5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3db7fe640 1 -- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3dc15d180 msgr2=0x7fc3dc0b5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3dbfff640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3dc06fc50 0x7fc3dc15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3db7fe640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3dc15d180 0x7fc3dc0b5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3db7fe640 1 -- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3dc06fc50 msgr2=0x7fc3dc15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3db7fe640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3dc06fc50 0x7fc3dc15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3db7fe640 1 -- 172.21.15.37:0/4216049976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc0b5dc0 con 0x7fc3dc0cf840 2024-04-25T08:52:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3db7fe640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3dc0cf840 0x7fc3dc0b2640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00ad040 tx=0x7fc3d001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3d8ff9640 1 -- 172.21.15.37:0/4216049976 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0004020 con 0x7fc3dc0cf840 2024-04-25T08:52:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3d8ff9640 1 -- 172.21.15.37:0/4216049976 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d00041c0 con 0x7fc3dc0cf840 2024-04-25T08:52:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e262c640 1 -- 172.21.15.37:0/4216049976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3dc0afda0 con 0x7fc3dc0cf840 2024-04-25T08:52:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3d8ff9640 1 -- 172.21.15.37:0/4216049976 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0003620 con 0x7fc3dc0cf840 2024-04-25T08:52:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e262c640 1 -- 172.21.15.37:0/4216049976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3dc0b0360 con 0x7fc3dc0cf840 2024-04-25T08:52:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3e262c640 1 -- 172.21.15.37:0/4216049976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3dc0afb90 con 0x7fc3dc0cf840 2024-04-25T08:52:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3d8ff9640 1 -- 172.21.15.37:0/4216049976 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3d00037c0 con 0x7fc3dc0cf840 2024-04-25T08:52:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.592+0000 7fc3d8ff9640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3c8066f60 0x7fc3c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.596+0000 7fc3d8ff9640 1 -- 172.21.15.37:0/4216049976 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc3d011edf0 con 0x7fc3dc0cf840 2024-04-25T08:52:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.596+0000 7fc3dbfff640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3c8066f60 0x7fc3c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.596+0000 7fc3dbfff640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3c8066f60 0x7fc3c8069420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc1592f0 tx=0x7fc3cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.600+0000 7fc3d8ff9640 1 -- 172.21.15.37:0/4216049976 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3d00c4060 con 0x7fc3dc0cf840 2024-04-25T08:52:44.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 -- 172.21.15.37:0/4216049976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc3dc04efa0 con 0x7fc3dc0cf840 2024-04-25T08:52:44.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3d8ff9640 1 -- 172.21.15.37:0/4216049976 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc3d0016110 con 0x7fc3dc0cf840 2024-04-25T08:52:44.745 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 -- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3c8066f60 msgr2=0x7fc3c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3c8066f60 0x7fc3c8069420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc1592f0 tx=0x7fc3cc008040 comp rx=0 tx=0).stop 2024-04-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 -- 172.21.15.37:0/4216049976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3dc0cf840 msgr2=0x7fc3dc0b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3dc0cf840 0x7fc3dc0b2640 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc3d00ad040 tx=0x7fc3d001ad60 comp rx=0 tx=0).stop 2024-04-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 -- 172.21.15.37:0/4216049976 shutdown_connections 2024-04-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3c8066f60 0x7fc3c8069420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3dc15d180 0x7fc3dc0b5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3dc0cf840 0x7fc3dc0b2640 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 --2- 172.21.15.37:0/4216049976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3dc06fc50 0x7fc3dc15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 -- 172.21.15.37:0/4216049976 >> 172.21.15.37:0/4216049976 conn(0x7fc3dc099a70 msgr2=0x7fc3dc0cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:44.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 -- 172.21.15.37:0/4216049976 shutdown_connections 2024-04-25T08:52:44.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.752+0000 7fc3e262c640 1 -- 172.21.15.37:0/4216049976 wait complete. 2024-04-25T08:52:44.762 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:52:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b5b7a7640 1 Processor -- start 2024-04-25T08:52:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b5b7a7640 1 -- start start 2024-04-25T08:52:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b5b7a7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0b540b3cf0 0x7f0b540cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b5b7a7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0b5415e6c0 0x7f0b540caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b5b7a7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b540cbf10 con 0x7f0b540cfad0 2024-04-25T08:52:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b5b7a7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b540cff40 con 0x7f0b540b3cf0 2024-04-25T08:52:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b5b7a7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5415eae0 con 0x7f0b5415e6c0 2024-04-25T08:52:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b59543640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0b540b3cf0 0x7f0b540cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b59543640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0b540b3cf0 0x7f0b540cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39620/0 (socket says 172.21.15.37:39620) 2024-04-25T08:52:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b59543640 1 -- 172.21.15.37:0/4094675781 learned_addr learned my addr 172.21.15.37:0/4094675781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b59d44640 1 --2- 172.21.15.37:0/4094675781 >> v2:172.21.15.203:3300/0 conn(0x7f0b5415e6c0 0x7f0b540caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b4bfff640 1 -- 172.21.15.37:0/4094675781 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029868870 0 0) 0x7f0b540cbf10 con 0x7f0b540cfad0 2024-04-25T08:52:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b59543640 1 -- 172.21.15.37:0/4094675781 >> v2:172.21.15.203:3300/0 conn(0x7f0b5415e6c0 msgr2=0x7f0b540caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b59543640 1 --2- 172.21.15.37:0/4094675781 >> v2:172.21.15.203:3300/0 conn(0x7f0b5415e6c0 0x7f0b540caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b59543640 1 -- 172.21.15.37:0/4094675781 >> v1:172.21.15.37:6789/0 conn(0x7f0b540cfad0 legacy=0x7f0b5415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.900+0000 7f0b59543640 1 -- 172.21.15.37:0/4094675781 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540cc1f0 con 0x7f0b540b3cf0 2024-04-25T08:52:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b59543640 1 --2- 172.21.15.37:0/4094675781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b540b3cf0 0x7f0b540cf590 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0b540d01a0 tx=0x7f0b5001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944371b5deb1643c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b4bfff640 1 -- 172.21.15.37:0/4094675781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b500a56c0 con 0x7f0b540b3cf0 2024-04-25T08:52:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b4bfff640 1 -- 172.21.15.37:0/4094675781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b500a5860 con 0x7f0b540b3cf0 2024-04-25T08:52:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b4bfff640 1 -- 172.21.15.37:0/4094675781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b500a5c30 con 0x7f0b540b3cf0 2024-04-25T08:52:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/4094675781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b540b3cf0 msgr2=0x7f0b540cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b5b7a7640 1 --2- 172.21.15.37:0/4094675781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b540b3cf0 0x7f0b540cf590 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0b540d01a0 tx=0x7f0b5001c9f0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/4094675781 shutdown_connections 2024-04-25T08:52:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b5b7a7640 1 --2- 172.21.15.37:0/4094675781 >> v2:172.21.15.203:3300/0 conn(0x7f0b5415e6c0 0x7f0b540caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b5b7a7640 1 --2- 172.21.15.37:0/4094675781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b540b3cf0 0x7f0b540cf590 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/4094675781 >> 172.21.15.37:0/4094675781 conn(0x7f0b5406ae60 msgr2=0x7f0b5406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/4094675781 shutdown_connections 2024-04-25T08:52:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/4094675781 wait complete. 2024-04-25T08:52:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b5b7a7640 1 Processor -- start 2024-04-25T08:52:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.904+0000 7f0b5b7a7640 1 -- start start 2024-04-25T08:52:44.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b5b7a7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b540b3cf0 0x7f0b541577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b5b7a7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b540cfad0 0x7f0b540aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b5b7a7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b5415e6c0 0x7f0b541552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b5b7a7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540b4250 con 0x7f0b540cfad0 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b5b7a7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540b40d0 con 0x7f0b5415e6c0 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b5b7a7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540cb840 con 0x7f0b540b3cf0 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b58d42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b540cfad0 0x7f0b540aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b58d42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b540cfad0 0x7f0b540aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39628/0 (socket says 172.21.15.37:39628) 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b58d42640 1 -- 172.21.15.37:0/1496837247 learned_addr learned my addr 172.21.15.37:0/1496837247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b59d44640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b5415e6c0 0x7f0b541552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b59543640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b540b3cf0 0x7f0b541577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b58d42640 1 -- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b540b3cf0 msgr2=0x7f0b541577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b58d42640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b540b3cf0 0x7f0b541577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b58d42640 1 -- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b5415e6c0 msgr2=0x7f0b541552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b58d42640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b5415e6c0 0x7f0b541552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b58d42640 1 -- 172.21.15.37:0/1496837247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b54160810 con 0x7f0b540cfad0 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b59543640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b540b3cf0 0x7f0b541577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b58d42640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b540cfad0 0x7f0b540aee00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400c4d0 tx=0x7f0b4400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b49ffb640 1 -- 172.21.15.37:0/1496837247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44017070 con 0x7f0b540cfad0 2024-04-25T08:52:44.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b49ffb640 1 -- 172.21.15.37:0/1496837247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44005480 con 0x7f0b540cfad0 2024-04-25T08:52:44.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/1496837247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b54157d90 con 0x7f0b540cfad0 2024-04-25T08:52:44.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b49ffb640 1 -- 172.21.15.37:0/1496837247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44003840 con 0x7f0b540cfad0 2024-04-25T08:52:44.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/1496837247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b540b1b60 con 0x7f0b540cfad0 2024-04-25T08:52:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/1496837247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5404efa0 con 0x7f0b540cfad0 2024-04-25T08:52:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b49ffb640 1 -- 172.21.15.37:0/1496837247 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b4401a070 con 0x7f0b540cfad0 2024-04-25T08:52:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b49ffb640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b38066f60 0x7f0b38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.908+0000 7f0b49ffb640 1 -- 172.21.15.37:0/1496837247 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0b44084740 con 0x7f0b540cfad0 2024-04-25T08:52:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.912+0000 7f0b59543640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b38066f60 0x7f0b38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.912+0000 7f0b59543640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b38066f60 0x7f0b38069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0b500ae000 tx=0x7f0b500a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:44.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:44.912+0000 7f0b49ffb640 1 -- 172.21.15.37:0/1496837247 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b440554d0 con 0x7f0b540cfad0 2024-04-25T08:52:45.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/1496837247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b540b3060 con 0x7f0b540cfad0 2024-04-25T08:52:45.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b49ffb640 1 -- 172.21.15.37:0/1496837247 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b44054dd0 con 0x7f0b540cfad0 2024-04-25T08:52:45.070 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:52:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b38066f60 msgr2=0x7f0b38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b5b7a7640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b38066f60 0x7f0b38069420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0b500ae000 tx=0x7f0b500a7070 comp rx=0 tx=0).stop 2024-04-25T08:52:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/1496837247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b540cfad0 msgr2=0x7f0b540aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b5b7a7640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b540cfad0 0x7f0b540aee00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400c4d0 tx=0x7f0b4400c990 comp rx=0 tx=0).stop 2024-04-25T08:52:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/1496837247 shutdown_connections 2024-04-25T08:52:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b5b7a7640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b38066f60 0x7f0b38069420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b5b7a7640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b5415e6c0 0x7f0b541552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b5b7a7640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b540cfad0 0x7f0b540aee00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b5b7a7640 1 --2- 172.21.15.37:0/1496837247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b540b3cf0 0x7f0b541577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.076+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/1496837247 >> 172.21.15.37:0/1496837247 conn(0x7f0b5406ae60 msgr2=0x7f0b540ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:45.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.080+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/1496837247 shutdown_connections 2024-04-25T08:52:45.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.080+0000 7f0b5b7a7640 1 -- 172.21.15.37:0/1496837247 wait complete. 2024-04-25T08:52:45.090 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:52:45.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b1b5ac640 1 Processor -- start 2024-04-25T08:52:45.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b1b5ac640 1 -- start start 2024-04-25T08:52:45.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b1b5ac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140cc820 0x7f9b140b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b1b5ac640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9b140b1910 0x7f9b14151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b1b5ac640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b14160310 con 0x7f9b140b0b00 2024-04-25T08:52:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b1b5ac640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b1415cfa0 con 0x7f9b140b1910 2024-04-25T08:52:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b1b5ac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1415cdf0 con 0x7f9b140cc820 2024-04-25T08:52:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b19b49640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9b140b1910 0x7f9b14151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b19348640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9b140b0b00 0x7f9b140cc110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:47946/0 (socket says 172.21.15.37:47946) 2024-04-25T08:52:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b19348640 1 -- 172.21.15.37:0/3735812698 learned_addr learned my addr 172.21.15.37:0/3735812698 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b18b47640 1 --2- 172.21.15.37:0/3735812698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140cc820 0x7f9b140b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b19b49640 1 -- 172.21.15.37:0/3735812698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140cc820 msgr2=0x7f9b140b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b19b49640 1 --2- 172.21.15.37:0/3735812698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140cc820 0x7f9b140b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b19b49640 1 -- 172.21.15.37:0/3735812698 >> v1:172.21.15.203:6789/0 conn(0x7f9b140b0b00 legacy=0x7f9b140cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b19b49640 1 -- 172.21.15.37:0/3735812698 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b1415f550 con 0x7f9b140b1910 2024-04-25T08:52:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.228+0000 7f9b18b47640 1 --2- 172.21.15.37:0/3735812698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140cc820 0x7f9b140b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b19b49640 1 --2- 172.21.15.37:0/3735812698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b140b1910 0x7f9b14151fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b100097c0 tx=0x7f9b1001c2c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a526e374a3a9f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b0bfff640 1 -- 172.21.15.37:0/3735812698 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b100b0070 con 0x7f9b140b1910 2024-04-25T08:52:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b0bfff640 1 -- 172.21.15.37:0/3735812698 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b100ab600 con 0x7f9b140b1910 2024-04-25T08:52:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b0bfff640 1 -- 172.21.15.37:0/3735812698 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b100ab9d0 con 0x7f9b140b1910 2024-04-25T08:52:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/3735812698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b140b1910 msgr2=0x7f9b14151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b1b5ac640 1 --2- 172.21.15.37:0/3735812698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b140b1910 0x7f9b14151fd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9b100097c0 tx=0x7f9b1001c2c0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/3735812698 shutdown_connections 2024-04-25T08:52:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b1b5ac640 1 --2- 172.21.15.37:0/3735812698 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b140b1910 0x7f9b14151fd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b1b5ac640 1 --2- 172.21.15.37:0/3735812698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140cc820 0x7f9b140b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/3735812698 >> 172.21.15.37:0/3735812698 conn(0x7f9b140b1f10 msgr2=0x7f9b140b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:45.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/3735812698 shutdown_connections 2024-04-25T08:52:45.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/3735812698 wait complete. 2024-04-25T08:52:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b1b5ac640 1 Processor -- start 2024-04-25T08:52:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.232+0000 7f9b1b5ac640 1 -- start start 2024-04-25T08:52:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b1b5ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b140b0b00 0x7f9b1414e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b1b5ac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140b1910 0x7f9b140c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b1b5ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b1415a480 0x7f9b1415a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b1b5ac640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b140b1060 con 0x7f9b1415a480 2024-04-25T08:52:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b1b5ac640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b140b0ee0 con 0x7f9b140b0b00 2024-04-25T08:52:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b1b5ac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1415bde0 con 0x7f9b140b1910 2024-04-25T08:52:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b19b49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b1415a480 0x7f9b1415a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b19b49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b1415a480 0x7f9b1415a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39634/0 (socket says 172.21.15.37:39634) 2024-04-25T08:52:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b19b49640 1 -- 172.21.15.37:0/4291760095 learned_addr learned my addr 172.21.15.37:0/4291760095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b18b47640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140b1910 0x7f9b140c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b19348640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b140b0b00 0x7f9b1414e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b19b49640 1 -- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140b1910 msgr2=0x7f9b140c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b19b49640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140b1910 0x7f9b140c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b19b49640 1 -- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b140b0b00 msgr2=0x7f9b1414e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b19b49640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b140b0b00 0x7f9b1414e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b19b49640 1 -- 172.21.15.37:0/4291760095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b14156640 con 0x7f9b1415a480 2024-04-25T08:52:45.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b18b47640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140b1910 0x7f9b140c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:45.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b19b49640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b1415a480 0x7f9b1415a930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9b10019ef0 tx=0x7f9b10019060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:45.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b09ffb640 1 -- 172.21.15.37:0/4291760095 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b100b0040 con 0x7f9b1415a480 2024-04-25T08:52:45.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b09ffb640 1 -- 172.21.15.37:0/4291760095 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b100b9070 con 0x7f9b1415a480 2024-04-25T08:52:45.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b09ffb640 1 -- 172.21.15.37:0/4291760095 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10006e20 con 0x7f9b1415a480 2024-04-25T08:52:45.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/4291760095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b14156920 con 0x7f9b1415a480 2024-04-25T08:52:45.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/4291760095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b14156ee0 con 0x7f9b1415a480 2024-04-25T08:52:45.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/4291760095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1404efa0 con 0x7f9b1415a480 2024-04-25T08:52:45.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b09ffb640 1 -- 172.21.15.37:0/4291760095 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b100b4650 con 0x7f9b1415a480 2024-04-25T08:52:45.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b09ffb640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9af8066f60 0x7f9af8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b09ffb640 1 -- 172.21.15.37:0/4291760095 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9b1011f010 con 0x7f9b1415a480 2024-04-25T08:52:45.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.236+0000 7f9b19348640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9af8066f60 0x7f9af8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.240+0000 7f9b19348640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9af8066f60 0x7f9af8069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005c10 tx=0x7f9b0c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:45.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.240+0000 7f9b09ffb640 1 -- 172.21.15.37:0/4291760095 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b14152d20 con 0x7f9b1415a480 2024-04-25T08:52:45.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.404+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/4291760095 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9b140b6380 con 0x7f9b1415a480 2024-04-25T08:52:45.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.404+0000 7f9b09ffb640 1 -- 172.21.15.37:0/4291760095 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9b100bb310 con 0x7f9b1415a480 2024-04-25T08:52:45.399 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:52:45.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9af8066f60 msgr2=0x7f9af8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9af8066f60 0x7f9af8069420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c005c10 tx=0x7f9b0c009040 comp rx=0 tx=0).stop 2024-04-25T08:52:45.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/4291760095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b1415a480 msgr2=0x7f9b1415a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b1415a480 0x7f9b1415a930 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9b10019ef0 tx=0x7f9b10019060 comp rx=0 tx=0).stop 2024-04-25T08:52:45.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/4291760095 shutdown_connections 2024-04-25T08:52:45.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9af8066f60 0x7f9af8069420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b1415a480 0x7f9b1415a930 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b140b1910 0x7f9b140c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 --2- 172.21.15.37:0/4291760095 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b140b0b00 0x7f9b1414e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/4291760095 >> 172.21.15.37:0/4291760095 conn(0x7f9b140b1f10 msgr2=0x7f9b1415b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:45.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/4291760095 shutdown_connections 2024-04-25T08:52:45.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.408+0000 7f9b1b5ac640 1 -- 172.21.15.37:0/4291760095 wait complete. 2024-04-25T08:52:45.419 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:52:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc525f8b640 1 Processor -- start 2024-04-25T08:52:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc525f8b640 1 -- start start 2024-04-25T08:52:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc525f8b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5200b0a30 0x7fc5200cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc525f8b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc52006f4a0 0x7fc520099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc525f8b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5200b0e10 con 0x7fc520099fb0 2024-04-25T08:52:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc525f8b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5200d0d90 con 0x7fc52006f4a0 2024-04-25T08:52:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc525f8b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5200d0180 con 0x7fc5200b0a30 2024-04-25T08:52:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc51effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc52006f4a0 0x7fc520099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc51effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc52006f4a0 0x7fc520099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39642/0 (socket says 172.21.15.37:39642) 2024-04-25T08:52:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc51effd640 1 -- 172.21.15.37:0/1426401334 learned_addr learned my addr 172.21.15.37:0/1426401334 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc51f7fe640 1 --2- 172.21.15.37:0/1426401334 >> v2:172.21.15.203:3300/0 conn(0x7fc5200b0a30 0x7fc5200cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc51effd640 1 -- 172.21.15.37:0/1426401334 >> v2:172.21.15.203:3300/0 conn(0x7fc5200b0a30 msgr2=0x7fc5200cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:45.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc51effd640 1 --2- 172.21.15.37:0/1426401334 >> v2:172.21.15.203:3300/0 conn(0x7fc5200b0a30 0x7fc5200cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc51effd640 1 -- 172.21.15.37:0/1426401334 >> v1:172.21.15.203:6789/0 conn(0x7fc520099fb0 legacy=0x7fc52015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:45.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc51effd640 1 -- 172.21.15.37:0/1426401334 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5200d0460 con 0x7fc52006f4a0 2024-04-25T08:52:45.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.548+0000 7fc51f7fe640 1 --2- 172.21.15.37:0/1426401334 >> v2:172.21.15.203:3300/0 conn(0x7fc5200b0a30 0x7fc5200cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:45.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc51effd640 1 --2- 172.21.15.37:0/1426401334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc52006f4a0 0x7fc520099a70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc520072ac0 tx=0x7fc51401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57aad2b1c15c65c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:45.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc51e7fc640 1 -- 172.21.15.37:0/1426401334 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5140b1070 con 0x7fc52006f4a0 2024-04-25T08:52:45.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc51e7fc640 1 -- 172.21.15.37:0/1426401334 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5140ac630 con 0x7fc52006f4a0 2024-04-25T08:52:45.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc51e7fc640 1 -- 172.21.15.37:0/1426401334 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5140a8050 con 0x7fc52006f4a0 2024-04-25T08:52:45.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc525f8b640 1 -- 172.21.15.37:0/1426401334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc52006f4a0 msgr2=0x7fc520099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc525f8b640 1 --2- 172.21.15.37:0/1426401334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc52006f4a0 0x7fc520099a70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc520072ac0 tx=0x7fc51401c4e0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc525f8b640 1 -- 172.21.15.37:0/1426401334 shutdown_connections 2024-04-25T08:52:45.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc525f8b640 1 --2- 172.21.15.37:0/1426401334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc52006f4a0 0x7fc520099a70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc525f8b640 1 --2- 172.21.15.37:0/1426401334 >> v2:172.21.15.203:3300/0 conn(0x7fc5200b0a30 0x7fc5200cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc525f8b640 1 -- 172.21.15.37:0/1426401334 >> 172.21.15.37:0/1426401334 conn(0x7fc5200b30b0 msgr2=0x7fc5200b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:45.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc525f8b640 1 -- 172.21.15.37:0/1426401334 shutdown_connections 2024-04-25T08:52:45.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc525f8b640 1 -- 172.21.15.37:0/1426401334 wait complete. 2024-04-25T08:52:45.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc525f8b640 1 Processor -- start 2024-04-25T08:52:45.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.552+0000 7fc525f8b640 1 -- start start 2024-04-25T08:52:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc525f8b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc52006f4a0 0x7fc52015cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc525f8b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc520099fb0 0x7fc5200b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc525f8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5200b0a30 0x7fc52006fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc525f8b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52015ded0 con 0x7fc5200b0a30 2024-04-25T08:52:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc525f8b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52015dd50 con 0x7fc52006f4a0 2024-04-25T08:52:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc525f8b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5200cc420 con 0x7fc520099fb0 2024-04-25T08:52:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5200b0a30 0x7fc52006fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5200b0a30 0x7fc52006fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39646/0 (socket says 172.21.15.37:39646) 2024-04-25T08:52:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51ffff640 1 -- 172.21.15.37:0/2625117021 learned_addr learned my addr 172.21.15.37:0/2625117021 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:45.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51f7fe640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc52006f4a0 0x7fc52015cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51effd640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc520099fb0 0x7fc5200b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51ffff640 1 -- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc520099fb0 msgr2=0x7fc5200b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51ffff640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc520099fb0 0x7fc5200b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51ffff640 1 -- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc52006f4a0 msgr2=0x7fc52015cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51ffff640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc52006f4a0 0x7fc52015cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51ffff640 1 -- 172.21.15.37:0/2625117021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc52015d550 con 0x7fc5200b0a30 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51f7fe640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc52006f4a0 0x7fc52015cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51effd640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc520099fb0 0x7fc5200b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51ffff640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5200b0a30 0x7fc52006fca0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc51800c780 tx=0x7fc51800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc507fff640 1 -- 172.21.15.37:0/2625117021 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc518004b70 con 0x7fc5200b0a30 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc507fff640 1 -- 172.21.15.37:0/2625117021 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc518004d10 con 0x7fc5200b0a30 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc525f8b640 1 -- 172.21.15.37:0/2625117021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5200c9280 con 0x7fc5200b0a30 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc507fff640 1 -- 172.21.15.37:0/2625117021 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51800aa90 con 0x7fc5200b0a30 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc525f8b640 1 -- 172.21.15.37:0/2625117021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5200c9840 con 0x7fc5200b0a30 2024-04-25T08:52:45.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc525f8b640 1 -- 172.21.15.37:0/2625117021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52004efa0 con 0x7fc5200b0a30 2024-04-25T08:52:45.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc507fff640 1 -- 172.21.15.37:0/2625117021 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc51800ac30 con 0x7fc5200b0a30 2024-04-25T08:52:45.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc507fff640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc50c066f60 0x7fc50c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:45.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc507fff640 1 -- 172.21.15.37:0/2625117021 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc518085630 con 0x7fc5200b0a30 2024-04-25T08:52:45.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.556+0000 7fc51f7fe640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc50c066f60 0x7fc50c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:45.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.560+0000 7fc51f7fe640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc50c066f60 0x7fc50c069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5200b6f70 tx=0x7fc510008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:45.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.560+0000 7fc507fff640 1 -- 172.21.15.37:0/2625117021 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc518020070 con 0x7fc5200b0a30 2024-04-25T08:52:45.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 -- 172.21.15.37:0/2625117021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc5200b3ee0 con 0x7fc5200b0a30 2024-04-25T08:52:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc507fff640 1 -- 172.21.15.37:0/2625117021 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc518056300 con 0x7fc5200b0a30 2024-04-25T08:52:45.729 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:52:45.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 -- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc50c066f60 msgr2=0x7fc50c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc50c066f60 0x7fc50c069420 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc5200b6f70 tx=0x7fc510008040 comp rx=0 tx=0).stop 2024-04-25T08:52:45.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 -- 172.21.15.37:0/2625117021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5200b0a30 msgr2=0x7fc52006fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:45.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5200b0a30 0x7fc52006fca0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc51800c780 tx=0x7fc51800cc40 comp rx=0 tx=0).stop 2024-04-25T08:52:45.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 -- 172.21.15.37:0/2625117021 shutdown_connections 2024-04-25T08:52:45.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc50c066f60 0x7fc50c069420 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5200b0a30 0x7fc52006fca0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc520099fb0 0x7fc5200b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 --2- 172.21.15.37:0/2625117021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc52006f4a0 0x7fc52015cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:45.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 -- 172.21.15.37:0/2625117021 >> 172.21.15.37:0/2625117021 conn(0x7fc5200b30b0 msgr2=0x7fc5200b19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:45.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 -- 172.21.15.37:0/2625117021 shutdown_connections 2024-04-25T08:52:45.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:45.736+0000 7fc525f8b640 1 -- 172.21.15.37:0/2625117021 wait complete. 2024-04-25T08:52:45.747 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T08:52:45.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:45.748 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T08:52:45.774 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi037.stderr:ceph-fuse[34305]: fuse finished with error 0 and tester_r 0 2024-04-25T08:52:45.775 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:52:51.876 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:52:51.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:52:51.877 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T08:52:51.893 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:52.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f4841ceb640 1 Processor -- start 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f4841ceb640 1 -- start start 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f4841ceb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c0ce510 0x7f483c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f4841ceb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f483c1451f0 0x7f483c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f4841ceb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483c0cee30 con 0x7f483c144ca0 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f4841ceb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f483c0cc2b0 con 0x7f483c1451f0 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f4841ceb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483c06e290 con 0x7f483c0ce510 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f483b7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c0ce510 0x7f483c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f483b7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c0ce510 0x7f483c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:38692/0 (socket says 172.21.15.37:38692) 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f483bfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f483c1451f0 0x7f483c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f483b7fe640 1 -- 172.21.15.37:0/1739636473 learned_addr learned my addr 172.21.15.37:0/1739636473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:52.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f483affd640 1 --1- 172.21.15.37:0/1739636473 >> v1:172.21.15.203:6789/0 conn(0x7f483c144ca0 0x7f483c06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:36024/0 (socket says 172.21.15.37:36024) 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f483b7fe640 1 -- 172.21.15.37:0/1739636473 >> v2:172.21.15.203:3300/0 conn(0x7f483c1451f0 msgr2=0x7f483c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f483b7fe640 1 --2- 172.21.15.37:0/1739636473 >> v2:172.21.15.203:3300/0 conn(0x7f483c1451f0 0x7f483c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f483b7fe640 1 -- 172.21.15.37:0/1739636473 >> v1:172.21.15.203:6789/0 conn(0x7f483c144ca0 legacy=0x7f483c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.076+0000 7f483b7fe640 1 -- 172.21.15.37:0/1739636473 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c0ccc70 con 0x7f483c0ce510 2024-04-25T08:52:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f483b7fe640 1 --2- 172.21.15.37:0/1739636473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c0ce510 0x7f483c0ce8f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f482c009870 tx=0x7f482c01ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b746a1329eea3948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f483a7fc640 1 -- 172.21.15.37:0/1739636473 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c0b5070 con 0x7f483c0ce510 2024-04-25T08:52:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f483a7fc640 1 -- 172.21.15.37:0/1739636473 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c0a8a50 con 0x7f483c0ce510 2024-04-25T08:52:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f483a7fc640 1 -- 172.21.15.37:0/1739636473 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c0a8e20 con 0x7f483c0ce510 2024-04-25T08:52:52.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f4841ceb640 1 -- 172.21.15.37:0/1739636473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c0ce510 msgr2=0x7f483c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f4841ceb640 1 --2- 172.21.15.37:0/1739636473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c0ce510 0x7f483c0ce8f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f482c009870 tx=0x7f482c01ed50 comp rx=0 tx=0).stop 2024-04-25T08:52:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f4841ceb640 1 -- 172.21.15.37:0/1739636473 shutdown_connections 2024-04-25T08:52:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f4841ceb640 1 --2- 172.21.15.37:0/1739636473 >> v2:172.21.15.203:3300/0 conn(0x7f483c1451f0 0x7f483c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f4841ceb640 1 --2- 172.21.15.37:0/1739636473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c0ce510 0x7f483c0ce8f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f4841ceb640 1 -- 172.21.15.37:0/1739636473 >> 172.21.15.37:0/1739636473 conn(0x7f483c0b11a0 msgr2=0x7f483c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f4841ceb640 1 -- 172.21.15.37:0/1739636473 shutdown_connections 2024-04-25T08:52:52.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.080+0000 7f4841ceb640 1 -- 172.21.15.37:0/1739636473 wait complete. 2024-04-25T08:52:52.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4841ceb640 1 Processor -- start 2024-04-25T08:52:52.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4841ceb640 1 -- start start 2024-04-25T08:52:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4841ceb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f483c0ce510 0x7f483c0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4841ceb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f483c144ca0 0x7f483c1f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4841ceb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c1451f0 0x7f483c13f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4841ceb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c06ee30 con 0x7f483c144ca0 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4841ceb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c06df20 con 0x7f483c0ce510 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4841ceb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483c0cc900 con 0x7f483c1451f0 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f483affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f483c144ca0 0x7f483c1f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f483affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f483c144ca0 0x7f483c1f37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46214/0 (socket says 172.21.15.37:46214) 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f483affd640 1 -- 172.21.15.37:0/750151242 learned_addr learned my addr 172.21.15.37:0/750151242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f483bfff640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c1451f0 0x7f483c13f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f483b7fe640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f483c0ce510 0x7f483c0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f483affd640 1 -- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c1451f0 msgr2=0x7f483c13f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f483affd640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c1451f0 0x7f483c13f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f483affd640 1 -- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f483c0ce510 msgr2=0x7f483c0ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f483affd640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f483c0ce510 0x7f483c0ca580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f483affd640 1 -- 172.21.15.37:0/750151242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c0b25a0 con 0x7f483c144ca0 2024-04-25T08:52:52.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f483affd640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f483c144ca0 0x7f483c1f37a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f483000c430 tx=0x7f483000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4823fff640 1 -- 172.21.15.37:0/750151242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300074e0 con 0x7f483c144ca0 2024-04-25T08:52:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4823fff640 1 -- 172.21.15.37:0/750151242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4830007680 con 0x7f483c144ca0 2024-04-25T08:52:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4841ceb640 1 -- 172.21.15.37:0/750151242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483c1f40b0 con 0x7f483c144ca0 2024-04-25T08:52:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4823fff640 1 -- 172.21.15.37:0/750151242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4830004a30 con 0x7f483c144ca0 2024-04-25T08:52:52.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.084+0000 7f4841ceb640 1 -- 172.21.15.37:0/750151242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483c1f4670 con 0x7f483c144ca0 2024-04-25T08:52:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.088+0000 7f4841ceb640 1 -- 172.21.15.37:0/750151242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483c04efa0 con 0x7f483c144ca0 2024-04-25T08:52:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.088+0000 7f4823fff640 1 -- 172.21.15.37:0/750151242 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f483001a020 con 0x7f483c144ca0 2024-04-25T08:52:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.088+0000 7f4823fff640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4828066f60 0x7f4828069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.088+0000 7f4823fff640 1 -- 172.21.15.37:0/750151242 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4830084b10 con 0x7f483c144ca0 2024-04-25T08:52:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.088+0000 7f483b7fe640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4828066f60 0x7f4828069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.088+0000 7f483b7fe640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4828066f60 0x7f4828069420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f482c0b2000 tx=0x7f482c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.096+0000 7f4823fff640 1 -- 172.21.15.37:0/750151242 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4830055870 con 0x7f483c144ca0 2024-04-25T08:52:52.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.256+0000 7f4841ceb640 1 -- 172.21.15.37:0/750151242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f483c0b1cc0 con 0x7f483c144ca0 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4823fff640 1 -- 172.21.15.37:0/750151242 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f4830055170 con 0x7f483c144ca0 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:38.282648+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 -- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4828066f60 msgr2=0x7f4828069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4828066f60 0x7f4828069420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f482c0b2000 tx=0x7f482c008040 comp rx=0 tx=0).stop 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 -- 172.21.15.37:0/750151242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f483c144ca0 msgr2=0x7f483c1f37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f483c144ca0 0x7f483c1f37a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f483000c430 tx=0x7f483000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 -- 172.21.15.37:0/750151242 shutdown_connections 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4828066f60 0x7f4828069420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f483c1451f0 0x7f483c13f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f483c144ca0 0x7f483c1f37a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 --2- 172.21.15.37:0/750151242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f483c0ce510 0x7f483c0ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 -- 172.21.15.37:0/750151242 >> 172.21.15.37:0/750151242 conn(0x7f483c0b11a0 msgr2=0x7f483c0cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:52.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 -- 172.21.15.37:0/750151242 shutdown_connections 2024-04-25T08:52:52.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.260+0000 7f4841ceb640 1 -- 172.21.15.37:0/750151242 wait complete. 2024-04-25T08:52:52.271 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:52.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4af55b7640 1 Processor -- start 2024-04-25T08:52:52.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4af55b7640 1 -- start start 2024-04-25T08:52:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4af55b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 0x7f4af009aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4af55b7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4af00b28b0 0x7f4af00b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4af55b7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af0099cc0 con 0x7f4af00b19f0 2024-04-25T08:52:52.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4af55b7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af0099a70 con 0x7f4af00b28b0 2024-04-25T08:52:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4af55b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af0099fa0 con 0x7f4af00b24d0 2024-04-25T08:52:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4aeeffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 0x7f4af009aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4aeeffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 0x7f4af009aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:38698/0 (socket says 172.21.15.37:38698) 2024-04-25T08:52:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4aeeffd640 1 -- 172.21.15.37:0/472923708 learned_addr learned my addr 172.21.15.37:0/472923708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4aee7fc640 1 --2- 172.21.15.37:0/472923708 >> v2:172.21.15.203:3300/0 conn(0x7f4af00b28b0 0x7f4af00b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4aeeffd640 1 -- 172.21.15.37:0/472923708 >> v2:172.21.15.203:3300/0 conn(0x7f4af00b28b0 msgr2=0x7f4af00b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:52.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4aeeffd640 1 --2- 172.21.15.37:0/472923708 >> v2:172.21.15.203:3300/0 conn(0x7f4af00b28b0 0x7f4af00b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4aeeffd640 1 -- 172.21.15.37:0/472923708 >> v1:172.21.15.203:6789/0 conn(0x7f4af00b19f0 legacy=0x7f4af00cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4aeeffd640 1 -- 172.21.15.37:0/472923708 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af009a280 con 0x7f4af00b24d0 2024-04-25T08:52:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.400+0000 7f4aee7fc640 1 --2- 172.21.15.37:0/472923708 >> v2:172.21.15.203:3300/0 conn(0x7f4af00b28b0 0x7f4af00b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:52.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4aeeffd640 1 --2- 172.21.15.37:0/472923708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 0x7f4af009aa80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009870 tx=0x7f4ae001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93f4091c25365966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4aedffb640 1 -- 172.21.15.37:0/472923708 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae00b0070 con 0x7f4af00b24d0 2024-04-25T08:52:52.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4aedffb640 1 -- 172.21.15.37:0/472923708 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae0004230 con 0x7f4af00b24d0 2024-04-25T08:52:52.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4aedffb640 1 -- 172.21.15.37:0/472923708 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae0003620 con 0x7f4af00b24d0 2024-04-25T08:52:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4af55b7640 1 -- 172.21.15.37:0/472923708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 msgr2=0x7f4af009aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4af55b7640 1 --2- 172.21.15.37:0/472923708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 0x7f4af009aa80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4ae0009870 tx=0x7f4ae001c410 comp rx=0 tx=0).stop 2024-04-25T08:52:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4af55b7640 1 -- 172.21.15.37:0/472923708 shutdown_connections 2024-04-25T08:52:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4af55b7640 1 --2- 172.21.15.37:0/472923708 >> v2:172.21.15.203:3300/0 conn(0x7f4af00b28b0 0x7f4af00b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4af55b7640 1 --2- 172.21.15.37:0/472923708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 0x7f4af009aa80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4af55b7640 1 -- 172.21.15.37:0/472923708 >> 172.21.15.37:0/472923708 conn(0x7f4af00b3020 msgr2=0x7f4af00b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4af55b7640 1 -- 172.21.15.37:0/472923708 shutdown_connections 2024-04-25T08:52:52.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4af55b7640 1 -- 172.21.15.37:0/472923708 wait complete. 2024-04-25T08:52:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4af55b7640 1 Processor -- start 2024-04-25T08:52:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.404+0000 7f4af55b7640 1 -- start start 2024-04-25T08:52:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4af55b7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4af00b19f0 0x7f4af00b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4af55b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 0x7f4af0158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4af55b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af00b28b0 0x7f4af0159110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4af55b7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af015e020 con 0x7f4af00b28b0 2024-04-25T08:52:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4af55b7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af015dea0 con 0x7f4af00b19f0 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4af55b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af016a770 con 0x7f4af00b24d0 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4aef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af00b28b0 0x7f4af0159110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4aef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af00b28b0 0x7f4af0159110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46230/0 (socket says 172.21.15.37:46230) 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4aef7fe640 1 -- 172.21.15.37:0/2054201068 learned_addr learned my addr 172.21.15.37:0/2054201068 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4aee7fc640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 0x7f4af0158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4aeeffd640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4af00b19f0 0x7f4af00b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4aef7fe640 1 -- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 msgr2=0x7f4af0158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4aef7fe640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 0x7f4af0158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4aef7fe640 1 -- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4af00b19f0 msgr2=0x7f4af00b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4aef7fe640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4af00b19f0 0x7f4af00b43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4aef7fe640 1 -- 172.21.15.37:0/2054201068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af0159890 con 0x7f4af00b28b0 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4aef7fe640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af00b28b0 0x7f4af0159110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800c730 tx=0x7f4ae800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4ad77fe640 1 -- 172.21.15.37:0/2054201068 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae8003b80 con 0x7f4af00b28b0 2024-04-25T08:52:52.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4ad77fe640 1 -- 172.21.15.37:0/2054201068 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae8003d20 con 0x7f4af00b28b0 2024-04-25T08:52:52.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4af55b7640 1 -- 172.21.15.37:0/2054201068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af0159b70 con 0x7f4af00b28b0 2024-04-25T08:52:52.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4ad77fe640 1 -- 172.21.15.37:0/2054201068 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae800aa30 con 0x7f4af00b28b0 2024-04-25T08:52:52.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4af55b7640 1 -- 172.21.15.37:0/2054201068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af015a130 con 0x7f4af00b28b0 2024-04-25T08:52:52.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4af55b7640 1 -- 172.21.15.37:0/2054201068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af004efa0 con 0x7f4af00b28b0 2024-04-25T08:52:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4ad77fe640 1 -- 172.21.15.37:0/2054201068 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ae800abd0 con 0x7f4af00b28b0 2024-04-25T08:52:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4ad77fe640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4adc066f60 0x7f4adc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.408+0000 7f4ad77fe640 1 -- 172.21.15.37:0/2054201068 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4ae8085140 con 0x7f4af00b28b0 2024-04-25T08:52:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.412+0000 7f4aeeffd640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4adc066f60 0x7f4adc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.412+0000 7f4aeeffd640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4adc066f60 0x7f4adc069420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00ad000 tx=0x7f4ae0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.416+0000 7f4ad77fe640 1 -- 172.21.15.37:0/2054201068 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ae8055ea0 con 0x7f4af00b28b0 2024-04-25T08:52:52.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.576+0000 7f4af55b7640 1 -- 172.21.15.37:0/2054201068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4af00c4e10 con 0x7f4af00b28b0 2024-04-25T08:52:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.576+0000 7f4ad77fe640 1 -- 172.21.15.37:0/2054201068 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f4ae80557a0 con 0x7f4af00b28b0 2024-04-25T08:52:52.570 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:52.571 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:38.282648+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 -- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4adc066f60 msgr2=0x7f4adc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4adc066f60 0x7f4adc069420 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4ae00ad000 tx=0x7f4ae0008040 comp rx=0 tx=0).stop 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 -- 172.21.15.37:0/2054201068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af00b28b0 msgr2=0x7f4af0159110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af00b28b0 0x7f4af0159110 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ae800c730 tx=0x7f4ae800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 -- 172.21.15.37:0/2054201068 shutdown_connections 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4adc066f60 0x7f4adc069420 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4af00b28b0 0x7f4af0159110 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4af00b24d0 0x7f4af0158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 --2- 172.21.15.37:0/2054201068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4af00b19f0 0x7f4af00b43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 -- 172.21.15.37:0/2054201068 >> 172.21.15.37:0/2054201068 conn(0x7f4af00b3020 msgr2=0x7f4af01543b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 -- 172.21.15.37:0/2054201068 shutdown_connections 2024-04-25T08:52:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.580+0000 7f4af55b7640 1 -- 172.21.15.37:0/2054201068 wait complete. 2024-04-25T08:52:52.590 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:52:52.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.716+0000 7fc597fe5640 1 Processor -- start 2024-04-25T08:52:52.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.716+0000 7fc597fe5640 1 -- start start 2024-04-25T08:52:52.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc597fe5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc59006c3b0 0x7fc59006ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc597fe5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc590066bb0 0x7fc5900b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc597fe5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59015ab30 con 0x7fc59015a750 2024-04-25T08:52:52.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc597fe5640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc590067100 con 0x7fc590066bb0 2024-04-25T08:52:52.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc597fe5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5900cf560 con 0x7fc59006c3b0 2024-04-25T08:52:52.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc596582640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc590066bb0 0x7fc5900b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc596582640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc590066bb0 0x7fc5900b2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46240/0 (socket says 172.21.15.37:46240) 2024-04-25T08:52:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc596582640 1 -- 172.21.15.37:0/491031155 learned_addr learned my addr 172.21.15.37:0/491031155 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc595580640 1 --2- 172.21.15.37:0/491031155 >> v2:172.21.15.203:3300/0 conn(0x7fc59006c3b0 0x7fc59006ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc596582640 1 -- 172.21.15.37:0/491031155 >> v2:172.21.15.203:3300/0 conn(0x7fc59006c3b0 msgr2=0x7fc59006ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc596582640 1 --2- 172.21.15.37:0/491031155 >> v2:172.21.15.203:3300/0 conn(0x7fc59006c3b0 0x7fc59006ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc596582640 1 -- 172.21.15.37:0/491031155 >> v1:172.21.15.203:6789/0 conn(0x7fc59015a750 legacy=0x7fc59006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc596582640 1 -- 172.21.15.37:0/491031155 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5900cf840 con 0x7fc590066bb0 2024-04-25T08:52:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc596582640 1 --2- 172.21.15.37:0/491031155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc590066bb0 0x7fc5900b2c30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc5900b5ef0 tx=0x7fc58801c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83666d36658085ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc594d7f640 1 -- 172.21.15.37:0/491031155 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880b1070 con 0x7fc590066bb0 2024-04-25T08:52:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc594d7f640 1 -- 172.21.15.37:0/491031155 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5880ac470 con 0x7fc590066bb0 2024-04-25T08:52:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.720+0000 7fc594d7f640 1 -- 172.21.15.37:0/491031155 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880ac840 con 0x7fc590066bb0 2024-04-25T08:52:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 -- 172.21.15.37:0/491031155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc590066bb0 msgr2=0x7fc5900b2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 --2- 172.21.15.37:0/491031155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc590066bb0 0x7fc5900b2c30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc5900b5ef0 tx=0x7fc58801c8d0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 -- 172.21.15.37:0/491031155 shutdown_connections 2024-04-25T08:52:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 --2- 172.21.15.37:0/491031155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc590066bb0 0x7fc5900b2c30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 --2- 172.21.15.37:0/491031155 >> v2:172.21.15.203:3300/0 conn(0x7fc59006c3b0 0x7fc59006ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 -- 172.21.15.37:0/491031155 >> 172.21.15.37:0/491031155 conn(0x7fc590071810 msgr2=0x7fc590071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 -- 172.21.15.37:0/491031155 shutdown_connections 2024-04-25T08:52:52.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 -- 172.21.15.37:0/491031155 wait complete. 2024-04-25T08:52:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 Processor -- start 2024-04-25T08:52:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 -- start start 2024-04-25T08:52:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc590066bb0 0x7fc59014e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc59006c3b0 0x7fc59013c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc59015a750 0x7fc590156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5901677b0 con 0x7fc59015a750 2024-04-25T08:52:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5900ced80 con 0x7fc59006c3b0 2024-04-25T08:52:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc590167930 con 0x7fc590066bb0 2024-04-25T08:52:52.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc596582640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc59015a750 0x7fc590156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc596582640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc59015a750 0x7fc590156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46248/0 (socket says 172.21.15.37:46248) 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc596582640 1 -- 172.21.15.37:0/603714354 learned_addr learned my addr 172.21.15.37:0/603714354 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc595580640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc59006c3b0 0x7fc59013c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc595d81640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc590066bb0 0x7fc59014e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc596582640 1 -- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc590066bb0 msgr2=0x7fc59014e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc596582640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc590066bb0 0x7fc59014e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc596582640 1 -- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc59006c3b0 msgr2=0x7fc59013c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc596582640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc59006c3b0 0x7fc59013c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc596582640 1 -- 172.21.15.37:0/603714354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc590157320 con 0x7fc59015a750 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc596582640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc59015a750 0x7fc590156de0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc5880af040 tx=0x7fc588019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc5867fc640 1 -- 172.21.15.37:0/603714354 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880bb020 con 0x7fc59015a750 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc5867fc640 1 -- 172.21.15.37:0/603714354 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5880090d0 con 0x7fc59015a750 2024-04-25T08:52:52.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 -- 172.21.15.37:0/603714354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59014ec90 con 0x7fc59015a750 2024-04-25T08:52:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc5867fc640 1 -- 172.21.15.37:0/603714354 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880b1040 con 0x7fc59015a750 2024-04-25T08:52:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.724+0000 7fc597fe5640 1 -- 172.21.15.37:0/603714354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59014f250 con 0x7fc59015a750 2024-04-25T08:52:52.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.728+0000 7fc597fe5640 1 -- 172.21.15.37:0/603714354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59004efa0 con 0x7fc59015a750 2024-04-25T08:52:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.728+0000 7fc5867fc640 1 -- 172.21.15.37:0/603714354 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5880be020 con 0x7fc59015a750 2024-04-25T08:52:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.728+0000 7fc5867fc640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 0x7fc574069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.728+0000 7fc5867fc640 1 -- 172.21.15.37:0/603714354 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc58811f5c0 con 0x7fc59015a750 2024-04-25T08:52:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.728+0000 7fc595d81640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 0x7fc574069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:52.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.728+0000 7fc595d81640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 0x7fc574069420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc59015d6d0 tx=0x7fc58c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:52.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.732+0000 7fc5867fc640 1 -- 172.21.15.37:0/603714354 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc5880c5070 con 0x7fc59015a750 2024-04-25T08:52:52.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:52.920+0000 7fc597fe5640 1 -- 172.21.15.37:0/603714354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc5900cc790 con 0x7fc59015a750 2024-04-25T08:52:53.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.348+0000 7fc5867fc640 1 -- 172.21.15.37:0/603714354 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc590157320 con 0x7fc59015a750 2024-04-25T08:52:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 -- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 msgr2=0x7fc574069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 0x7fc574069420 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc59015d6d0 tx=0x7fc58c009040 comp rx=0 tx=0).stop 2024-04-25T08:52:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 -- 172.21.15.37:0/603714354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc59015a750 msgr2=0x7fc590156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc59015a750 0x7fc590156de0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc5880af040 tx=0x7fc588019670 comp rx=0 tx=0).stop 2024-04-25T08:52:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 -- 172.21.15.37:0/603714354 shutdown_connections 2024-04-25T08:52:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 0x7fc574069420 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc59015a750 0x7fc590156de0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc59006c3b0 0x7fc59013c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 --2- 172.21.15.37:0/603714354 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc590066bb0 0x7fc59014e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 -- 172.21.15.37:0/603714354 >> 172.21.15.37:0/603714354 conn(0x7fc590071810 msgr2=0x7fc59015d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 -- 172.21.15.37:0/603714354 shutdown_connections 2024-04-25T08:52:53.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.352+0000 7fc597fe5640 1 -- 172.21.15.37:0/603714354 wait complete. 2024-04-25T08:52:53.369 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T08:52:53.369 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba14f71640 1 Processor -- start 2024-04-25T08:52:53.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba14f71640 1 -- start start 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba14f71640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fba100b0a30 0x7fba100cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba14f71640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba1006f4a0 0x7fba10099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba14f71640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fba10099fb0 0x7fba1015d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba14f71640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba100d0d90 con 0x7fba100b0a30 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba14f71640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba100b0e10 con 0x7fba10099fb0 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba14f71640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba100d0180 con 0x7fba1006f4a0 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba0e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fba100b0a30 0x7fba100cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba0e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fba100b0a30 0x7fba100cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46264/0 (socket says 172.21.15.37:46264) 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba0e59c640 1 -- 172.21.15.37:0/882246990 learned_addr learned my addr 172.21.15.37:0/882246990 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba0e59c640 1 -- 172.21.15.37:0/882246990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba1006f4a0 msgr2=0x7fba10099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba0ed9d640 1 --2- 172.21.15.37:0/882246990 >> v2:172.21.15.203:3300/0 conn(0x7fba10099fb0 0x7fba1015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba05d9b640 1 --2- 172.21.15.37:0/882246990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba1006f4a0 0x7fba10099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba0e59c640 1 --2- 172.21.15.37:0/882246990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba1006f4a0 0x7fba10099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba0e59c640 1 -- 172.21.15.37:0/882246990 >> v2:172.21.15.203:3300/0 conn(0x7fba10099fb0 msgr2=0x7fba1015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba0e59c640 1 --2- 172.21.15.37:0/882246990 >> v2:172.21.15.203:3300/0 conn(0x7fba10099fb0 0x7fba1015d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba0e59c640 1 -- 172.21.15.37:0/882246990 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba100d0460 con 0x7fba100b0a30 2024-04-25T08:52:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba0ed9d640 1 --2- 172.21.15.37:0/882246990 >> v2:172.21.15.203:3300/0 conn(0x7fba10099fb0 0x7fba1015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.512+0000 7fba05d9b640 1 --2- 172.21.15.37:0/882246990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba1006f4a0 0x7fba10099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0e59c640 1 --2- 172.21.15.37:0/882246990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba100b0a30 0x7fba100cf890 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fba10072850 tx=0x7fba0001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cbbe83cd62ab489 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0dd9b640 1 -- 172.21.15.37:0/882246990 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba000b1070 con 0x7fba100b0a30 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0dd9b640 1 -- 172.21.15.37:0/882246990 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba000ac630 con 0x7fba100b0a30 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0dd9b640 1 -- 172.21.15.37:0/882246990 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba000a8050 con 0x7fba100b0a30 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 -- 172.21.15.37:0/882246990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba100b0a30 msgr2=0x7fba100cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 --2- 172.21.15.37:0/882246990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba100b0a30 0x7fba100cf890 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fba10072850 tx=0x7fba0001c520 comp rx=0 tx=0).stop 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 -- 172.21.15.37:0/882246990 shutdown_connections 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 --2- 172.21.15.37:0/882246990 >> v2:172.21.15.203:3300/0 conn(0x7fba10099fb0 0x7fba1015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 --2- 172.21.15.37:0/882246990 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba1006f4a0 0x7fba10099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 --2- 172.21.15.37:0/882246990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba100b0a30 0x7fba100cf890 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 -- 172.21.15.37:0/882246990 >> 172.21.15.37:0/882246990 conn(0x7fba100b30b0 msgr2=0x7fba100b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 -- 172.21.15.37:0/882246990 shutdown_connections 2024-04-25T08:52:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 -- 172.21.15.37:0/882246990 wait complete. 2024-04-25T08:52:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 Processor -- start 2024-04-25T08:52:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 -- start start 2024-04-25T08:52:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba1006f4a0 0x7fba100b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba10099fb0 0x7fba1015e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba100b0a30 0x7fba1015e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba10161790 con 0x7fba100b0a30 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba10161610 con 0x7fba10099fb0 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba14f71640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba10161910 con 0x7fba1006f4a0 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba100b0a30 0x7fba1015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba100b0a30 0x7fba1015e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46276/0 (socket says 172.21.15.37:46276) 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0ed9d640 1 -- 172.21.15.37:0/3246924860 learned_addr learned my addr 172.21.15.37:0/3246924860 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba05d9b640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba10099fb0 0x7fba1015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0e59c640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba1006f4a0 0x7fba100b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0ed9d640 1 -- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba1006f4a0 msgr2=0x7fba100b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0ed9d640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba1006f4a0 0x7fba100b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0ed9d640 1 -- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba10099fb0 msgr2=0x7fba1015e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0ed9d640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba10099fb0 0x7fba1015e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0ed9d640 1 -- 172.21.15.37:0/3246924860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba1015ecf0 con 0x7fba100b0a30 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba0ed9d640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba100b0a30 0x7fba1015e650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fba0800cf20 tx=0x7fba0800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba077fe640 1 -- 172.21.15.37:0/3246924860 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0801a070 con 0x7fba100b0a30 2024-04-25T08:52:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.516+0000 7fba077fe640 1 -- 172.21.15.37:0/3246924860 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0800d840 con 0x7fba100b0a30 2024-04-25T08:52:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.520+0000 7fba14f71640 1 -- 172.21.15.37:0/3246924860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba1015efd0 con 0x7fba100b0a30 2024-04-25T08:52:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.520+0000 7fba077fe640 1 -- 172.21.15.37:0/3246924860 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0800dc30 con 0x7fba100b0a30 2024-04-25T08:52:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.520+0000 7fba14f71640 1 -- 172.21.15.37:0/3246924860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba1015f590 con 0x7fba100b0a30 2024-04-25T08:52:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.520+0000 7fba14f71640 1 -- 172.21.15.37:0/3246924860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1004efa0 con 0x7fba100b0a30 2024-04-25T08:52:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.520+0000 7fba077fe640 1 -- 172.21.15.37:0/3246924860 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba0801d070 con 0x7fba100b0a30 2024-04-25T08:52:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.520+0000 7fba077fe640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9f0066f60 0x7fb9f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.520+0000 7fba077fe640 1 -- 172.21.15.37:0/3246924860 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fba08085bf0 con 0x7fba100b0a30 2024-04-25T08:52:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.520+0000 7fba0e59c640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9f0066f60 0x7fb9f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.520+0000 7fba0e59c640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9f0066f60 0x7fb9f0069420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fba000ae000 tx=0x7fba00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.524+0000 7fba077fe640 1 -- 172.21.15.37:0/3246924860 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba080568b0 con 0x7fba100b0a30 2024-04-25T08:52:53.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:53.720+0000 7fba14f71640 1 -- 172.21.15.37:0/3246924860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fba1006fa80 con 0x7fba100b0a30 2024-04-25T08:52:54.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.352+0000 7fba077fe640 1 -- 172.21.15.37:0/3246924860 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7fba080561b0 con 0x7fba100b0a30 2024-04-25T08:52:54.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 -- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9f0066f60 msgr2=0x7fb9f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9f0066f60 0x7fb9f0069420 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fba000ae000 tx=0x7fba00008040 comp rx=0 tx=0).stop 2024-04-25T08:52:54.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 -- 172.21.15.37:0/3246924860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba100b0a30 msgr2=0x7fba1015e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba100b0a30 0x7fba1015e650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fba0800cf20 tx=0x7fba0800a7d0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 -- 172.21.15.37:0/3246924860 shutdown_connections 2024-04-25T08:52:54.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9f0066f60 0x7fb9f0069420 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba100b0a30 0x7fba1015e650 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba10099fb0 0x7fba1015e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 --2- 172.21.15.37:0/3246924860 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba1006f4a0 0x7fba100b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 -- 172.21.15.37:0/3246924860 >> 172.21.15.37:0/3246924860 conn(0x7fba100b30b0 msgr2=0x7fba100af200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 -- 172.21.15.37:0/3246924860 shutdown_connections 2024-04-25T08:52:54.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.356+0000 7fba14f71640 1 -- 172.21.15.37:0/3246924860 wait complete. 2024-04-25T08:52:54.372 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:52:54.372 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:52:54.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb154c1c640 1 Processor -- start 2024-04-25T08:52:54.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb154c1c640 1 -- start start 2024-04-25T08:52:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb154c1c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1500b3f50 0x7fb15015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb154c1c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 0x7fb1500b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb154c1c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1500b2090 con 0x7fb15015a700 2024-04-25T08:52:54.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb154c1c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb15015f200 con 0x7fb1500b3f50 2024-04-25T08:52:54.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb154c1c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1500b43e0 con 0x7fb1500b24a0 2024-04-25T08:52:54.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb14ed9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 0x7fb1500b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb14ed9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 0x7fb1500b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:38762/0 (socket says 172.21.15.37:38762) 2024-04-25T08:52:54.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb14dd9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb1500b3f50 0x7fb15015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb14ed9d640 1 -- 172.21.15.37:0/3533825906 learned_addr learned my addr 172.21.15.37:0/3533825906 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:54.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb14ed9d640 1 -- 172.21.15.37:0/3533825906 >> v2:172.21.15.203:3300/0 conn(0x7fb1500b3f50 msgr2=0x7fb15015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb14ed9d640 1 --2- 172.21.15.37:0/3533825906 >> v2:172.21.15.203:3300/0 conn(0x7fb1500b3f50 0x7fb15015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb14ed9d640 1 -- 172.21.15.37:0/3533825906 >> v1:172.21.15.203:6789/0 conn(0x7fb15015a700 legacy=0x7fb1500cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.508+0000 7fb14ed9d640 1 -- 172.21.15.37:0/3533825906 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1500b46c0 con 0x7fb1500b24a0 2024-04-25T08:52:54.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb14ed9d640 1 --2- 172.21.15.37:0/3533825906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 0x7fb1500b2930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb14800d150 tx=0x7fb14801eb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad7b6cb2ce4fc3e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb14d59a640 1 -- 172.21.15.37:0/3533825906 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14800b2d0 con 0x7fb1500b24a0 2024-04-25T08:52:54.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb14d59a640 1 -- 172.21.15.37:0/3533825906 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14800b470 con 0x7fb1500b24a0 2024-04-25T08:52:54.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb14d59a640 1 -- 172.21.15.37:0/3533825906 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1480a7af0 con 0x7fb1500b24a0 2024-04-25T08:52:54.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 -- 172.21.15.37:0/3533825906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 msgr2=0x7fb1500b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 --2- 172.21.15.37:0/3533825906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 0x7fb1500b2930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb14800d150 tx=0x7fb14801eb10 comp rx=0 tx=0).stop 2024-04-25T08:52:54.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 -- 172.21.15.37:0/3533825906 shutdown_connections 2024-04-25T08:52:54.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 --2- 172.21.15.37:0/3533825906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 0x7fb1500b2930 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 --2- 172.21.15.37:0/3533825906 >> v2:172.21.15.203:3300/0 conn(0x7fb1500b3f50 0x7fb15015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 -- 172.21.15.37:0/3533825906 >> 172.21.15.37:0/3533825906 conn(0x7fb1500b11a0 msgr2=0x7fb1500b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 -- 172.21.15.37:0/3533825906 shutdown_connections 2024-04-25T08:52:54.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 -- 172.21.15.37:0/3533825906 wait complete. 2024-04-25T08:52:54.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 Processor -- start 2024-04-25T08:52:54.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 -- start start 2024-04-25T08:52:54.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 0x7fb1501534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb15015a700 0x7fb1501f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1501f5d50 0x7fb150164880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1500cd310 con 0x7fb15015a700 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1500ce600 con 0x7fb1501f5d50 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb154c1c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1500cd490 con 0x7fb1500b24a0 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb14dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb15015a700 0x7fb1501f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb14dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb15015a700 0x7fb1501f5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46290/0 (socket says 172.21.15.37:46290) 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.512+0000 7fb14dd9b640 1 -- 172.21.15.37:0/1797173840 learned_addr learned my addr 172.21.15.37:0/1797173840 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb14ed9d640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1501f5d50 0x7fb150164880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb14e59c640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 0x7fb1501534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb14dd9b640 1 -- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 msgr2=0x7fb1501534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb14dd9b640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 0x7fb1501534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb14dd9b640 1 -- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1501f5d50 msgr2=0x7fb150164880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb14dd9b640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1501f5d50 0x7fb150164880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb14dd9b640 1 -- 172.21.15.37:0/1797173840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb150158940 con 0x7fb15015a700 2024-04-25T08:52:54.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb14e59c640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 0x7fb1501534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:54.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb14dd9b640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb15015a700 0x7fb1501f5810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb13800e930 tx=0x7fb13800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb146ffd640 1 -- 172.21.15.37:0/1797173840 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138017070 con 0x7fb15015a700 2024-04-25T08:52:54.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb146ffd640 1 -- 172.21.15.37:0/1797173840 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13800ca00 con 0x7fb15015a700 2024-04-25T08:52:54.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb154c1c640 1 -- 172.21.15.37:0/1797173840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb150158c20 con 0x7fb15015a700 2024-04-25T08:52:54.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb146ffd640 1 -- 172.21.15.37:0/1797173840 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13800cdb0 con 0x7fb15015a700 2024-04-25T08:52:54.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb154c1c640 1 -- 172.21.15.37:0/1797173840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1501591e0 con 0x7fb15015a700 2024-04-25T08:52:54.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb154c1c640 1 -- 172.21.15.37:0/1797173840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15004efa0 con 0x7fb15015a700 2024-04-25T08:52:54.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb146ffd640 1 -- 172.21.15.37:0/1797173840 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb13801a070 con 0x7fb15015a700 2024-04-25T08:52:54.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb146ffd640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb130066f60 0x7fb130069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb146ffd640 1 -- 172.21.15.37:0/1797173840 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb138085810 con 0x7fb15015a700 2024-04-25T08:52:54.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb14e59c640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb130066f60 0x7fb130069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.516+0000 7fb14e59c640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb130066f60 0x7fb130069420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb140000f70 tx=0x7fb140009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.520+0000 7fb146ffd640 1 -- 172.21.15.37:0/1797173840 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb138056500 con 0x7fb15015a700 2024-04-25T08:52:54.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.716+0000 7fb154c1c640 1 -- 172.21.15.37:0/1797173840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb15006f260 con 0x7fb15015a700 2024-04-25T08:52:54.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.716+0000 7fb146ffd640 1 -- 172.21.15.37:0/1797173840 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7fb138055e00 con 0x7fb15015a700 2024-04-25T08:52:54.710 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:54.710 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2902565757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2902565757},{"type":"v1","addr":"172.21.15.203:6835","nonce":2902565757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/822172299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":822172299},{"type":"v1","addr":"172.21.15.203:6837","nonce":822172299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3681855890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3681855890},{"type":"v1","addr":"172.21.15.37:6833","nonce":3681855890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:30.564153+0000","modified":"2024-04-25T08:52:53.726659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:52:54.711 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-25T08:52:54.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 -- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb130066f60 msgr2=0x7fb130069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb130066f60 0x7fb130069420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb140000f70 tx=0x7fb140009040 comp rx=0 tx=0).stop 2024-04-25T08:52:54.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 -- 172.21.15.37:0/1797173840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb15015a700 msgr2=0x7fb1501f5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb15015a700 0x7fb1501f5810 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb13800e930 tx=0x7fb13800edf0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 -- 172.21.15.37:0/1797173840 shutdown_connections 2024-04-25T08:52:54.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb130066f60 0x7fb130069420 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1501f5d50 0x7fb150164880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb15015a700 0x7fb1501f5810 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 --2- 172.21.15.37:0/1797173840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1500b24a0 0x7fb1501534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 -- 172.21.15.37:0/1797173840 >> 172.21.15.37:0/1797173840 conn(0x7fb1500b11a0 msgr2=0x7fb1500af260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 -- 172.21.15.37:0/1797173840 shutdown_connections 2024-04-25T08:52:54.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.720+0000 7fb154c1c640 1 -- 172.21.15.37:0/1797173840 wait complete. 2024-04-25T08:52:54.728 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:54.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.856+0000 7fe39f307640 1 Processor -- start 2024-04-25T08:52:54.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.856+0000 7fe39f307640 1 -- start start 2024-04-25T08:52:54.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39f307640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe39815a700 0x7fe39815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39f307640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3980b24a0 0x7fe3980b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39f307640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3980b2090 con 0x7fe3980b3f50 2024-04-25T08:52:54.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39f307640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe39815f200 con 0x7fe39815a700 2024-04-25T08:52:54.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39f307640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980b43e0 con 0x7fe3980b24a0 2024-04-25T08:52:54.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39d0a3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe39815a700 0x7fe39815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39d8a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3980b24a0 0x7fe3980b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39c8a2640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe3980b3f50 0x7fe3980cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:36058/0 (socket says 172.21.15.37:36058) 2024-04-25T08:52:54.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39c8a2640 1 -- 172.21.15.37:0/9655340 learned_addr learned my addr 172.21.15.37:0/9655340 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:54.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39d8a4640 1 -- 172.21.15.37:0/9655340 >> v2:172.21.15.203:3300/0 conn(0x7fe39815a700 msgr2=0x7fe39815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:54.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39d8a4640 1 --2- 172.21.15.37:0/9655340 >> v2:172.21.15.203:3300/0 conn(0x7fe39815a700 0x7fe39815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39d8a4640 1 -- 172.21.15.37:0/9655340 >> v1:172.21.15.203:6789/0 conn(0x7fe3980b3f50 legacy=0x7fe3980cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.860+0000 7fe39d8a4640 1 -- 172.21.15.37:0/9655340 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3980b46c0 con 0x7fe3980b24a0 2024-04-25T08:52:54.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39d8a4640 1 --2- 172.21.15.37:0/9655340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3980b24a0 0x7fe3980b2930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe390009d90 tx=0x7fe39001c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6147b03acde88c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe38ffff640 1 -- 172.21.15.37:0/9655340 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3900b2070 con 0x7fe3980b24a0 2024-04-25T08:52:54.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe38ffff640 1 -- 172.21.15.37:0/9655340 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3900a5800 con 0x7fe3980b24a0 2024-04-25T08:52:54.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe38ffff640 1 -- 172.21.15.37:0/9655340 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3900a5bd0 con 0x7fe3980b24a0 2024-04-25T08:52:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 -- 172.21.15.37:0/9655340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3980b24a0 msgr2=0x7fe3980b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 --2- 172.21.15.37:0/9655340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3980b24a0 0x7fe3980b2930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe390009d90 tx=0x7fe39001c850 comp rx=0 tx=0).stop 2024-04-25T08:52:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 -- 172.21.15.37:0/9655340 shutdown_connections 2024-04-25T08:52:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 --2- 172.21.15.37:0/9655340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3980b24a0 0x7fe3980b2930 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 --2- 172.21.15.37:0/9655340 >> v2:172.21.15.203:3300/0 conn(0x7fe39815a700 0x7fe39815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 -- 172.21.15.37:0/9655340 >> 172.21.15.37:0/9655340 conn(0x7fe3980b11a0 msgr2=0x7fe3980b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 -- 172.21.15.37:0/9655340 shutdown_connections 2024-04-25T08:52:54.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 -- 172.21.15.37:0/9655340 wait complete. 2024-04-25T08:52:54.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 Processor -- start 2024-04-25T08:52:54.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 -- start start 2024-04-25T08:52:54.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3980b24a0 0x7fe39814aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39815a700 0x7fe3980af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe39814e220 0x7fe39814e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980cd310 con 0x7fe39815a700 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980ce600 con 0x7fe3980b24a0 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39f307640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3980cd490 con 0x7fe39814e220 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39c8a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39815a700 0x7fe3980af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39c8a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39815a700 0x7fe3980af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46298/0 (socket says 172.21.15.37:46298) 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39c8a2640 1 -- 172.21.15.37:0/3284570438 learned_addr learned my addr 172.21.15.37:0/3284570438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39d0a3640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3980b24a0 0x7fe39814aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.864+0000 7fe39d8a4640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe39814e220 0x7fe39814e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39c8a2640 1 -- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe39814e220 msgr2=0x7fe39814e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39c8a2640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe39814e220 0x7fe39814e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39c8a2640 1 -- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3980b24a0 msgr2=0x7fe39814aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39c8a2640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3980b24a0 0x7fe39814aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:54.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39c8a2640 1 -- 172.21.15.37:0/3284570438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe398167bf0 con 0x7fe39815a700 2024-04-25T08:52:54.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39d8a4640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe39814e220 0x7fe39814e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:54.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39d0a3640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3980b24a0 0x7fe39814aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:54.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39c8a2640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39815a700 0x7fe3980af270 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c420 tx=0x7fe38800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe38dffb640 1 -- 172.21.15.37:0/3284570438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe388017070 con 0x7fe39815a700 2024-04-25T08:52:54.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe38dffb640 1 -- 172.21.15.37:0/3284570438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe388005cf0 con 0x7fe39815a700 2024-04-25T08:52:54.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39f307640 1 -- 172.21.15.37:0/3284570438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe398167ed0 con 0x7fe39815a700 2024-04-25T08:52:54.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe38dffb640 1 -- 172.21.15.37:0/3284570438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3880072f0 con 0x7fe39815a700 2024-04-25T08:52:54.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39f307640 1 -- 172.21.15.37:0/3284570438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe398168490 con 0x7fe39815a700 2024-04-25T08:52:54.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39f307640 1 -- 172.21.15.37:0/3284570438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe39804efa0 con 0x7fe39815a700 2024-04-25T08:52:54.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe38dffb640 1 -- 172.21.15.37:0/3284570438 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe38801a070 con 0x7fe39815a700 2024-04-25T08:52:54.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe38dffb640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe37c066f90 0x7fe37c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:54.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe38dffb640 1 -- 172.21.15.37:0/3284570438 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe3880847a0 con 0x7fe39815a700 2024-04-25T08:52:54.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.868+0000 7fe39d0a3640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe37c066f90 0x7fe37c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:54.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.872+0000 7fe39d0a3640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe37c066f90 0x7fe37c069450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe398152270 tx=0x7fe394008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:54.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:54.872+0000 7fe38dffb640 1 -- 172.21.15.37:0/3284570438 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe3880553d0 con 0x7fe39815a700 2024-04-25T08:52:54.871 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:52:54.872 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:52:55.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.028+0000 7fe39f307640 1 -- 172.21.15.37:0/3284570438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe39806f260 con 0x7fe39815a700 2024-04-25T08:52:55.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe38dffb640 1 -- 172.21.15.37:0/3284570438 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18478 (secure 0 0 0) 0x7fe388054cd0 con 0x7fe39815a700 2024-04-25T08:52:55.024 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:55.024 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:38.282648+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:55.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 -- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe37c066f90 msgr2=0x7fe37c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe37c066f90 0x7fe37c069450 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe398152270 tx=0x7fe394008040 comp rx=0 tx=0).stop 2024-04-25T08:52:55.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 -- 172.21.15.37:0/3284570438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39815a700 msgr2=0x7fe3980af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39815a700 0x7fe3980af270 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe38800c420 tx=0x7fe38800c8e0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 -- 172.21.15.37:0/3284570438 shutdown_connections 2024-04-25T08:52:55.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe37c066f90 0x7fe37c069450 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe39814e220 0x7fe39814e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe39815a700 0x7fe3980af270 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 --2- 172.21.15.37:0/3284570438 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3980b24a0 0x7fe39814aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 -- 172.21.15.37:0/3284570438 >> 172.21.15.37:0/3284570438 conn(0x7fe3980b11a0 msgr2=0x7fe3980cdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:55.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 -- 172.21.15.37:0/3284570438 shutdown_connections 2024-04-25T08:52:55.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.032+0000 7fe39f307640 1 -- 172.21.15.37:0/3284570438 wait complete. 2024-04-25T08:52:55.042 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:52:55.042 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:52:55.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9ddca40640 1 Processor -- start 2024-04-25T08:52:55.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9ddca40640 1 -- start start 2024-04-25T08:52:55.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9ddca40640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9dd815dd50 0x7f9dd806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9ddca40640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd806c1e0 0x7f9dd806dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9ddca40640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dd815e2e0 con 0x7f9dd80af390 2024-04-25T08:52:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9ddca40640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dd80b2f10 con 0x7f9dd815dd50 2024-04-25T08:52:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9ddca40640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd806c630 con 0x7f9dd806c1e0 2024-04-25T08:52:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9dd659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9dd80af390 0x7f9dd8066cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59832/0 (socket says 172.21.15.37:59832) 2024-04-25T08:52:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9dd5d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9dd815dd50 0x7f9dd806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9dd659c640 1 -- 172.21.15.37:0/1593273674 learned_addr learned my addr 172.21.15.37:0/1593273674 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9dd6d9d640 1 --2- 172.21.15.37:0/1593273674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd806c1e0 0x7f9dd806dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9dd5d9b640 1 -- 172.21.15.37:0/1593273674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd806c1e0 msgr2=0x7f9dd806dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9dd5d9b640 1 --2- 172.21.15.37:0/1593273674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd806c1e0 0x7f9dd806dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9dd5d9b640 1 -- 172.21.15.37:0/1593273674 >> v1:172.21.15.37:6789/0 conn(0x7f9dd80af390 legacy=0x7f9dd8066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9dd5d9b640 1 -- 172.21.15.37:0/1593273674 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd815e5c0 con 0x7f9dd815dd50 2024-04-25T08:52:55.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9dd559a640 1 -- 172.21.15.37:0/1593273674 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4200570068 0 0) 0x7f9dd815e2e0 con 0x7f9dd80af390 2024-04-25T08:52:55.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.164+0000 7f9dd5d9b640 1 --2- 172.21.15.37:0/1593273674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd815dd50 0x7f9dd806bca0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0097c0 tx=0x7f9dcc01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7aa90ab3f7fcb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:55.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd559a640 1 -- 172.21.15.37:0/1593273674 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0b0070 con 0x7f9dd815dd50 2024-04-25T08:52:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd559a640 1 -- 172.21.15.37:0/1593273674 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dcc0ab600 con 0x7f9dd815dd50 2024-04-25T08:52:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd559a640 1 -- 172.21.15.37:0/1593273674 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dcc0ab9d0 con 0x7f9dd815dd50 2024-04-25T08:52:55.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 -- 172.21.15.37:0/1593273674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd815dd50 msgr2=0x7f9dd806bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 --2- 172.21.15.37:0/1593273674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd815dd50 0x7f9dd806bca0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9dcc0097c0 tx=0x7f9dcc01c380 comp rx=0 tx=0).stop 2024-04-25T08:52:55.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 -- 172.21.15.37:0/1593273674 shutdown_connections 2024-04-25T08:52:55.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 --2- 172.21.15.37:0/1593273674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd806c1e0 0x7f9dd806dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 --2- 172.21.15.37:0/1593273674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd815dd50 0x7f9dd806bca0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 -- 172.21.15.37:0/1593273674 >> 172.21.15.37:0/1593273674 conn(0x7f9dd80717f0 msgr2=0x7f9dd8071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:55.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 -- 172.21.15.37:0/1593273674 shutdown_connections 2024-04-25T08:52:55.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 -- 172.21.15.37:0/1593273674 wait complete. 2024-04-25T08:52:55.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 Processor -- start 2024-04-25T08:52:55.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 -- start start 2024-04-25T08:52:55.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd806c1e0 0x7f9dd814cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd80af390 0x7f9dd8155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd815dd50 0x7f9dd814d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd806ee00 con 0x7f9dd815dd50 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd806ec80 con 0x7f9dd806c1e0 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd806ef80 con 0x7f9dd80af390 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd815dd50 0x7f9dd814d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd815dd50 0x7f9dd814d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46316/0 (socket says 172.21.15.37:46316) 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd6d9d640 1 -- 172.21.15.37:0/593615356 learned_addr learned my addr 172.21.15.37:0/593615356 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd6d9d640 1 -- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd80af390 msgr2=0x7f9dd8155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd659c640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd806c1e0 0x7f9dd814cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd6d9d640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd80af390 0x7f9dd8155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd6d9d640 1 -- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd806c1e0 msgr2=0x7f9dd814cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd6d9d640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd806c1e0 0x7f9dd814cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd6d9d640 1 -- 172.21.15.37:0/593615356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd8160640 con 0x7f9dd815dd50 2024-04-25T08:52:55.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dd6d9d640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd815dd50 0x7f9dd814d2d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0007e90 tx=0x7f9dd000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:55.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dc6ffd640 1 -- 172.21.15.37:0/593615356 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0014420 con 0x7f9dd815dd50 2024-04-25T08:52:55.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dc6ffd640 1 -- 172.21.15.37:0/593615356 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dd0019070 con 0x7f9dd815dd50 2024-04-25T08:52:55.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 -- 172.21.15.37:0/593615356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd8160920 con 0x7f9dd815dd50 2024-04-25T08:52:55.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9dc6ffd640 1 -- 172.21.15.37:0/593615356 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dd0014800 con 0x7f9dd815dd50 2024-04-25T08:52:55.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.168+0000 7f9ddca40640 1 -- 172.21.15.37:0/593615356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd814f2d0 con 0x7f9dd815dd50 2024-04-25T08:52:55.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.172+0000 7f9ddca40640 1 -- 172.21.15.37:0/593615356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd804efa0 con 0x7f9dd815dd50 2024-04-25T08:52:55.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.172+0000 7f9dc6ffd640 1 -- 172.21.15.37:0/593615356 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9dd00149a0 con 0x7f9dd815dd50 2024-04-25T08:52:55.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.172+0000 7f9dc6ffd640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9db8066f60 0x7f9db8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.172+0000 7f9dc6ffd640 1 -- 172.21.15.37:0/593615356 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9dd0085cf0 con 0x7f9dd815dd50 2024-04-25T08:52:55.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.172+0000 7f9dd659c640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9db8066f60 0x7f9db8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.172+0000 7f9dd659c640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9db8066f60 0x7f9db8069420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8000f70 tx=0x7f9dc8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:55.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.176+0000 7f9dc6ffd640 1 -- 172.21.15.37:0/593615356 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9dd0056920 con 0x7f9dd815dd50 2024-04-25T08:52:55.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.328+0000 7f9ddca40640 1 -- 172.21.15.37:0/593615356 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9dd81603f0 con 0x7f9db8066f60 2024-04-25T08:52:55.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.356+0000 7f9dc6ffd640 1 -- 172.21.15.37:0/593615356 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f9dd81603f0 con 0x7f9db8066f60 2024-04-25T08:52:55.352 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:55.354 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:52:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.356+0000 7f9ddca40640 1 -- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9db8066f60 msgr2=0x7f9db8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.356+0000 7f9ddca40640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9db8066f60 0x7f9db8069420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8000f70 tx=0x7f9dc8009040 comp rx=0 tx=0).stop 2024-04-25T08:52:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.356+0000 7f9ddca40640 1 -- 172.21.15.37:0/593615356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd815dd50 msgr2=0x7f9dd814d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.356+0000 7f9ddca40640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd815dd50 0x7f9dd814d2d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9dd0007e90 tx=0x7f9dd000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.360+0000 7f9ddca40640 1 -- 172.21.15.37:0/593615356 shutdown_connections 2024-04-25T08:52:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.360+0000 7f9ddca40640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9db8066f60 0x7f9db8069420 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.360+0000 7f9ddca40640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dd815dd50 0x7f9dd814d2d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.360+0000 7f9ddca40640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9dd80af390 0x7f9dd8155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.360+0000 7f9ddca40640 1 --2- 172.21.15.37:0/593615356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9dd806c1e0 0x7f9dd814cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.360+0000 7f9ddca40640 1 -- 172.21.15.37:0/593615356 >> 172.21.15.37:0/593615356 conn(0x7f9dd80717f0 msgr2=0x7f9dd8066cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.360+0000 7f9ddca40640 1 -- 172.21.15.37:0/593615356 shutdown_connections 2024-04-25T08:52:55.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.360+0000 7f9ddca40640 1 -- 172.21.15.37:0/593615356 wait complete. 2024-04-25T08:52:55.370 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T08:52:55.126529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219668,"kb_used_data":3496,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755052,"statfs":{"total":773094113280,"available":772869173248,"internally_reserved":0,"allocated":3579904,"data_stored":2140384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004976"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836749+0000","last_change":"2024-04-25T08:52:30.238555+0000","last_active":"2024-04-25T08:52:39.836749+0000","last_peered":"2024-04-25T08:52:39.836749+0000","last_clean":"2024-04-25T08:52:39.836749+0000","last_became_active":"2024-04-25T08:52:30.238074+0000","last_became_peered":"2024-04-25T08:52:30.238074+0000","last_unstale":"2024-04-25T08:52:39.836749+0000","last_undegraded":"2024-04-25T08:52:39.836749+0000","last_fullsized":"2024-04-25T08:52:39.836749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:52.505423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834324+0000","last_change":"2024-04-25T08:52:31.237632+0000","last_active":"2024-04-25T08:52:39.834324+0000","last_peered":"2024-04-25T08:52:39.834324+0000","last_clean":"2024-04-25T08:52:39.834324+0000","last_became_active":"2024-04-25T08:52:31.237356+0000","last_became_peered":"2024-04-25T08:52:31.237356+0000","last_unstale":"2024-04-25T08:52:39.834324+0000","last_undegraded":"2024-04-25T08:52:39.834324+0000","last_fullsized":"2024-04-25T08:52:39.834324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833357+0000","last_change":"2024-04-25T08:52:36.008156+0000","last_active":"2024-04-25T08:52:39.833357+0000","last_peered":"2024-04-25T08:52:39.833357+0000","last_clean":"2024-04-25T08:52:39.833357+0000","last_became_active":"2024-04-25T08:52:36.006941+0000","last_became_peered":"2024-04-25T08:52:36.006941+0000","last_unstale":"2024-04-25T08:52:39.833357+0000","last_undegraded":"2024-04-25T08:52:39.833357+0000","last_fullsized":"2024-04-25T08:52:39.833357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:00.158176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834859+0000","last_change":"2024-04-25T08:52:30.225034+0000","last_active":"2024-04-25T08:52:39.834859+0000","last_peered":"2024-04-25T08:52:39.834859+0000","last_clean":"2024-04-25T08:52:39.834859+0000","last_became_active":"2024-04-25T08:52:30.224727+0000","last_became_peered":"2024-04-25T08:52:30.224727+0000","last_unstale":"2024-04-25T08:52:39.834859+0000","last_undegraded":"2024-04-25T08:52:39.834859+0000","last_fullsized":"2024-04-25T08:52:39.834859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836491+0000","last_change":"2024-04-25T08:52:31.256496+0000","last_active":"2024-04-25T08:52:39.836491+0000","last_peered":"2024-04-25T08:52:39.836491+0000","last_clean":"2024-04-25T08:52:39.836491+0000","last_became_active":"2024-04-25T08:52:31.255998+0000","last_became_peered":"2024-04-25T08:52:31.255998+0000","last_unstale":"2024-04-25T08:52:39.836491+0000","last_undegraded":"2024-04-25T08:52:39.836491+0000","last_fullsized":"2024-04-25T08:52:39.836491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299627+0000","last_change":"2024-04-25T08:52:35.324509+0000","last_active":"2024-04-25T08:52:38.299627+0000","last_peered":"2024-04-25T08:52:38.299627+0000","last_clean":"2024-04-25T08:52:38.299627+0000","last_became_active":"2024-04-25T08:52:35.323398+0000","last_became_peered":"2024-04-25T08:52:35.323398+0000","last_unstale":"2024-04-25T08:52:38.299627+0000","last_undegraded":"2024-04-25T08:52:38.299627+0000","last_fullsized":"2024-04-25T08:52:38.299627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:56.088772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307843+0000","last_change":"2024-04-25T08:52:30.248641+0000","last_active":"2024-04-25T08:52:39.307843+0000","last_peered":"2024-04-25T08:52:39.307843+0000","last_clean":"2024-04-25T08:52:39.307843+0000","last_became_active":"2024-04-25T08:52:30.245183+0000","last_became_peered":"2024-04-25T08:52:30.245183+0000","last_unstale":"2024-04-25T08:52:39.307843+0000","last_undegraded":"2024-04-25T08:52:39.307843+0000","last_fullsized":"2024-04-25T08:52:39.307843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:46.012895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644062+0000","last_change":"2024-04-25T08:52:31.272777+0000","last_active":"2024-04-25T08:52:39.644062+0000","last_peered":"2024-04-25T08:52:39.644062+0000","last_clean":"2024-04-25T08:52:39.644062+0000","last_became_active":"2024-04-25T08:52:31.271932+0000","last_became_peered":"2024-04-25T08:52:31.271932+0000","last_unstale":"2024-04-25T08:52:39.644062+0000","last_undegraded":"2024-04-25T08:52:39.644062+0000","last_fullsized":"2024-04-25T08:52:39.644062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300302+0000","last_change":"2024-04-25T08:52:35.330473+0000","last_active":"2024-04-25T08:52:38.300302+0000","last_peered":"2024-04-25T08:52:38.300302+0000","last_clean":"2024-04-25T08:52:38.300302+0000","last_became_active":"2024-04-25T08:52:35.328741+0000","last_became_peered":"2024-04-25T08:52:35.328741+0000","last_unstale":"2024-04-25T08:52:38.300302+0000","last_undegraded":"2024-04-25T08:52:38.300302+0000","last_fullsized":"2024-04-25T08:52:38.300302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:14.472581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307492+0000","last_change":"2024-04-25T08:52:30.245422+0000","last_active":"2024-04-25T08:52:39.307492+0000","last_peered":"2024-04-25T08:52:39.307492+0000","last_clean":"2024-04-25T08:52:39.307492+0000","last_became_active":"2024-04-25T08:52:30.244911+0000","last_became_peered":"2024-04-25T08:52:30.244911+0000","last_unstale":"2024-04-25T08:52:39.307492+0000","last_undegraded":"2024-04-25T08:52:39.307492+0000","last_fullsized":"2024-04-25T08:52:39.307492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644128+0000","last_change":"2024-04-25T08:52:31.274339+0000","last_active":"2024-04-25T08:52:39.644128+0000","last_peered":"2024-04-25T08:52:39.644128+0000","last_clean":"2024-04-25T08:52:39.644128+0000","last_became_active":"2024-04-25T08:52:31.273607+0000","last_became_peered":"2024-04-25T08:52:31.273607+0000","last_unstale":"2024-04-25T08:52:39.644128+0000","last_undegraded":"2024-04-25T08:52:39.644128+0000","last_fullsized":"2024-04-25T08:52:39.644128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:56.409407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299563+0000","last_change":"2024-04-25T08:52:35.316303+0000","last_active":"2024-04-25T08:52:38.299563+0000","last_peered":"2024-04-25T08:52:38.299563+0000","last_clean":"2024-04-25T08:52:38.299563+0000","last_became_active":"2024-04-25T08:52:35.315156+0000","last_became_peered":"2024-04-25T08:52:35.315156+0000","last_unstale":"2024-04-25T08:52:38.299563+0000","last_undegraded":"2024-04-25T08:52:38.299563+0000","last_fullsized":"2024-04-25T08:52:38.299563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:35.695987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300591+0000","last_change":"2024-04-25T08:52:31.269860+0000","last_active":"2024-04-25T08:52:38.300591+0000","last_peered":"2024-04-25T08:52:38.300591+0000","last_clean":"2024-04-25T08:52:38.300591+0000","last_became_active":"2024-04-25T08:52:31.268731+0000","last_became_peered":"2024-04-25T08:52:31.268731+0000","last_unstale":"2024-04-25T08:52:38.300591+0000","last_undegraded":"2024-04-25T08:52:38.300591+0000","last_fullsized":"2024-04-25T08:52:38.300591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307114+0000","last_change":"2024-04-25T08:52:30.246021+0000","last_active":"2024-04-25T08:52:39.307114+0000","last_peered":"2024-04-25T08:52:39.307114+0000","last_clean":"2024-04-25T08:52:39.307114+0000","last_became_active":"2024-04-25T08:52:30.245132+0000","last_became_peered":"2024-04-25T08:52:30.245132+0000","last_unstale":"2024-04-25T08:52:39.307114+0000","last_undegraded":"2024-04-25T08:52:39.307114+0000","last_fullsized":"2024-04-25T08:52:39.307114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:11.290953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301681+0000","last_change":"2024-04-25T08:52:35.323528+0000","last_active":"2024-04-25T08:52:38.301681+0000","last_peered":"2024-04-25T08:52:38.301681+0000","last_clean":"2024-04-25T08:52:38.301681+0000","last_became_active":"2024-04-25T08:52:35.320911+0000","last_became_peered":"2024-04-25T08:52:35.320911+0000","last_unstale":"2024-04-25T08:52:38.301681+0000","last_undegraded":"2024-04-25T08:52:38.301681+0000","last_fullsized":"2024-04-25T08:52:38.301681+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835879+0000","last_change":"2024-04-25T08:52:31.257885+0000","last_active":"2024-04-25T08:52:39.835879+0000","last_peered":"2024-04-25T08:52:39.835879+0000","last_clean":"2024-04-25T08:52:39.835879+0000","last_became_active":"2024-04-25T08:52:31.257583+0000","last_became_peered":"2024-04-25T08:52:31.257583+0000","last_unstale":"2024-04-25T08:52:39.835879+0000","last_undegraded":"2024-04-25T08:52:39.835879+0000","last_fullsized":"2024-04-25T08:52:39.835879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834075+0000","last_change":"2024-04-25T08:52:30.213612+0000","last_active":"2024-04-25T08:52:39.834075+0000","last_peered":"2024-04-25T08:52:39.834075+0000","last_clean":"2024-04-25T08:52:39.834075+0000","last_became_active":"2024-04-25T08:52:30.212962+0000","last_became_peered":"2024-04-25T08:52:30.212962+0000","last_unstale":"2024-04-25T08:52:39.834075+0000","last_undegraded":"2024-04-25T08:52:39.834075+0000","last_fullsized":"2024-04-25T08:52:39.834075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300390+0000","last_change":"2024-04-25T08:52:35.324597+0000","last_active":"2024-04-25T08:52:38.300390+0000","last_peered":"2024-04-25T08:52:38.300390+0000","last_clean":"2024-04-25T08:52:38.300390+0000","last_became_active":"2024-04-25T08:52:35.323534+0000","last_became_peered":"2024-04-25T08:52:35.323534+0000","last_unstale":"2024-04-25T08:52:38.300390+0000","last_undegraded":"2024-04-25T08:52:38.300390+0000","last_fullsized":"2024-04-25T08:52:38.300390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:24.182927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834456+0000","last_change":"2024-04-25T08:52:31.231520+0000","last_active":"2024-04-25T08:52:39.834456+0000","last_peered":"2024-04-25T08:52:39.834456+0000","last_clean":"2024-04-25T08:52:39.834456+0000","last_became_active":"2024-04-25T08:52:31.231009+0000","last_became_peered":"2024-04-25T08:52:31.231009+0000","last_unstale":"2024-04-25T08:52:39.834456+0000","last_undegraded":"2024-04-25T08:52:39.834456+0000","last_fullsized":"2024-04-25T08:52:39.834456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834462+0000","last_change":"2024-04-25T08:52:30.231226+0000","last_active":"2024-04-25T08:52:39.834462+0000","last_peered":"2024-04-25T08:52:39.834462+0000","last_clean":"2024-04-25T08:52:39.834462+0000","last_became_active":"2024-04-25T08:52:30.230790+0000","last_became_peered":"2024-04-25T08:52:30.230790+0000","last_unstale":"2024-04-25T08:52:39.834462+0000","last_undegraded":"2024-04-25T08:52:39.834462+0000","last_fullsized":"2024-04-25T08:52:39.834462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:11.290953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834915+0000","last_change":"2024-04-25T08:52:36.004737+0000","last_active":"2024-04-25T08:52:39.834915+0000","last_peered":"2024-04-25T08:52:39.834915+0000","last_clean":"2024-04-25T08:52:39.834915+0000","last_became_active":"2024-04-25T08:52:36.003639+0000","last_became_peered":"2024-04-25T08:52:36.003639+0000","last_unstale":"2024-04-25T08:52:39.834915+0000","last_undegraded":"2024-04-25T08:52:39.834915+0000","last_fullsized":"2024-04-25T08:52:39.834915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301223+0000","last_change":"2024-04-25T08:52:31.260426+0000","last_active":"2024-04-25T08:52:38.301223+0000","last_peered":"2024-04-25T08:52:38.301223+0000","last_clean":"2024-04-25T08:52:38.301223+0000","last_became_active":"2024-04-25T08:52:31.259594+0000","last_became_peered":"2024-04-25T08:52:31.259594+0000","last_unstale":"2024-04-25T08:52:38.301223+0000","last_undegraded":"2024-04-25T08:52:38.301223+0000","last_fullsized":"2024-04-25T08:52:38.301223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:45.962867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306464+0000","last_change":"2024-04-25T08:52:30.240896+0000","last_active":"2024-04-25T08:52:39.306464+0000","last_peered":"2024-04-25T08:52:39.306464+0000","last_clean":"2024-04-25T08:52:39.306464+0000","last_became_active":"2024-04-25T08:52:30.240290+0000","last_became_peered":"2024-04-25T08:52:30.240290+0000","last_unstale":"2024-04-25T08:52:39.306464+0000","last_undegraded":"2024-04-25T08:52:39.306464+0000","last_fullsized":"2024-04-25T08:52:39.306464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300735+0000","last_change":"2024-04-25T08:52:35.328692+0000","last_active":"2024-04-25T08:52:38.300735+0000","last_peered":"2024-04-25T08:52:38.300735+0000","last_clean":"2024-04-25T08:52:38.300735+0000","last_became_active":"2024-04-25T08:52:35.323899+0000","last_became_peered":"2024-04-25T08:52:35.323899+0000","last_unstale":"2024-04-25T08:52:38.300735+0000","last_undegraded":"2024-04-25T08:52:38.300735+0000","last_fullsized":"2024-04-25T08:52:38.300735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836941+0000","last_change":"2024-04-25T08:52:31.253016+0000","last_active":"2024-04-25T08:52:39.836941+0000","last_peered":"2024-04-25T08:52:39.836941+0000","last_clean":"2024-04-25T08:52:39.836941+0000","last_became_active":"2024-04-25T08:52:31.252369+0000","last_became_peered":"2024-04-25T08:52:31.252369+0000","last_unstale":"2024-04-25T08:52:39.836941+0000","last_undegraded":"2024-04-25T08:52:39.836941+0000","last_fullsized":"2024-04-25T08:52:39.836941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303292+0000","last_change":"2024-04-25T08:52:30.241679+0000","last_active":"2024-04-25T08:52:38.303292+0000","last_peered":"2024-04-25T08:52:38.303292+0000","last_clean":"2024-04-25T08:52:38.303292+0000","last_became_active":"2024-04-25T08:52:30.241063+0000","last_became_peered":"2024-04-25T08:52:30.241063+0000","last_unstale":"2024-04-25T08:52:38.303292+0000","last_undegraded":"2024-04-25T08:52:38.303292+0000","last_fullsized":"2024-04-25T08:52:38.303292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300540+0000","last_change":"2024-04-25T08:52:36.007015+0000","last_active":"2024-04-25T08:52:38.300540+0000","last_peered":"2024-04-25T08:52:38.300540+0000","last_clean":"2024-04-25T08:52:38.300540+0000","last_became_active":"2024-04-25T08:52:36.005832+0000","last_became_peered":"2024-04-25T08:52:36.005832+0000","last_unstale":"2024-04-25T08:52:38.300540+0000","last_undegraded":"2024-04-25T08:52:38.300540+0000","last_fullsized":"2024-04-25T08:52:38.300540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:19.422348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834979+0000","last_change":"2024-04-25T08:52:31.282175+0000","last_active":"2024-04-25T08:52:39.834979+0000","last_peered":"2024-04-25T08:52:39.834979+0000","last_clean":"2024-04-25T08:52:39.834979+0000","last_became_active":"2024-04-25T08:52:31.281955+0000","last_became_peered":"2024-04-25T08:52:31.281955+0000","last_unstale":"2024-04-25T08:52:39.834979+0000","last_undegraded":"2024-04-25T08:52:39.834979+0000","last_fullsized":"2024-04-25T08:52:39.834979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834733+0000","last_change":"2024-04-25T08:52:30.215220+0000","last_active":"2024-04-25T08:52:39.834733+0000","last_peered":"2024-04-25T08:52:39.834733+0000","last_clean":"2024-04-25T08:52:39.834733+0000","last_became_active":"2024-04-25T08:52:30.214773+0000","last_became_peered":"2024-04-25T08:52:30.214773+0000","last_unstale":"2024-04-25T08:52:39.834733+0000","last_undegraded":"2024-04-25T08:52:39.834733+0000","last_fullsized":"2024-04-25T08:52:39.834733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833949+0000","last_change":"2024-04-25T08:52:36.003489+0000","last_active":"2024-04-25T08:52:39.833949+0000","last_peered":"2024-04-25T08:52:39.833949+0000","last_clean":"2024-04-25T08:52:39.833949+0000","last_became_active":"2024-04-25T08:52:36.002295+0000","last_became_peered":"2024-04-25T08:52:36.002295+0000","last_unstale":"2024-04-25T08:52:39.833949+0000","last_undegraded":"2024-04-25T08:52:39.833949+0000","last_fullsized":"2024-04-25T08:52:39.833949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:47.463029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833050+0000","last_change":"2024-04-25T08:52:31.226718+0000","last_active":"2024-04-25T08:52:39.833050+0000","last_peered":"2024-04-25T08:52:39.833050+0000","last_clean":"2024-04-25T08:52:39.833050+0000","last_became_active":"2024-04-25T08:52:31.226442+0000","last_became_peered":"2024-04-25T08:52:31.226442+0000","last_unstale":"2024-04-25T08:52:39.833050+0000","last_undegraded":"2024-04-25T08:52:39.833050+0000","last_fullsized":"2024-04-25T08:52:39.833050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307955+0000","last_change":"2024-04-25T08:52:30.249239+0000","last_active":"2024-04-25T08:52:39.307955+0000","last_peered":"2024-04-25T08:52:39.307955+0000","last_clean":"2024-04-25T08:52:39.307955+0000","last_became_active":"2024-04-25T08:52:30.248612+0000","last_became_peered":"2024-04-25T08:52:30.248612+0000","last_unstale":"2024-04-25T08:52:39.307955+0000","last_undegraded":"2024-04-25T08:52:39.307955+0000","last_fullsized":"2024-04-25T08:52:39.307955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:34.198893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836268+0000","last_change":"2024-04-25T08:52:36.001292+0000","last_active":"2024-04-25T08:52:39.836268+0000","last_peered":"2024-04-25T08:52:39.836268+0000","last_clean":"2024-04-25T08:52:39.836268+0000","last_became_active":"2024-04-25T08:52:36.000113+0000","last_became_peered":"2024-04-25T08:52:36.000113+0000","last_unstale":"2024-04-25T08:52:39.836268+0000","last_undegraded":"2024-04-25T08:52:39.836268+0000","last_fullsized":"2024-04-25T08:52:39.836268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645594+0000","last_change":"2024-04-25T08:52:31.247003+0000","last_active":"2024-04-25T08:52:39.645594+0000","last_peered":"2024-04-25T08:52:39.645594+0000","last_clean":"2024-04-25T08:52:39.645594+0000","last_became_active":"2024-04-25T08:52:31.246200+0000","last_became_peered":"2024-04-25T08:52:31.246200+0000","last_unstale":"2024-04-25T08:52:39.645594+0000","last_undegraded":"2024-04-25T08:52:39.645594+0000","last_fullsized":"2024-04-25T08:52:39.645594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:52.007683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835389+0000","last_change":"2024-04-25T08:52:30.215091+0000","last_active":"2024-04-25T08:52:39.835389+0000","last_peered":"2024-04-25T08:52:39.835389+0000","last_clean":"2024-04-25T08:52:39.835389+0000","last_became_active":"2024-04-25T08:52:30.214664+0000","last_became_peered":"2024-04-25T08:52:30.214664+0000","last_unstale":"2024-04-25T08:52:39.835389+0000","last_undegraded":"2024-04-25T08:52:39.835389+0000","last_fullsized":"2024-04-25T08:52:39.835389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833696+0000","last_change":"2024-04-25T08:52:35.313874+0000","last_active":"2024-04-25T08:52:39.833696+0000","last_peered":"2024-04-25T08:52:39.833696+0000","last_clean":"2024-04-25T08:52:39.833696+0000","last_became_active":"2024-04-25T08:52:35.313429+0000","last_became_peered":"2024-04-25T08:52:35.313429+0000","last_unstale":"2024-04-25T08:52:39.833696+0000","last_undegraded":"2024-04-25T08:52:39.833696+0000","last_fullsized":"2024-04-25T08:52:39.833696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:59.165277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300917+0000","last_change":"2024-04-25T08:52:31.257574+0000","last_active":"2024-04-25T08:52:38.300917+0000","last_peered":"2024-04-25T08:52:38.300917+0000","last_clean":"2024-04-25T08:52:38.300917+0000","last_became_active":"2024-04-25T08:52:31.257040+0000","last_became_peered":"2024-04-25T08:52:31.257040+0000","last_unstale":"2024-04-25T08:52:38.300917+0000","last_undegraded":"2024-04-25T08:52:38.300917+0000","last_fullsized":"2024-04-25T08:52:38.300917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:52.221264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645682+0000","last_change":"2024-04-25T08:52:30.225801+0000","last_active":"2024-04-25T08:52:39.645682+0000","last_peered":"2024-04-25T08:52:39.645682+0000","last_clean":"2024-04-25T08:52:39.645682+0000","last_became_active":"2024-04-25T08:52:30.224857+0000","last_became_peered":"2024-04-25T08:52:30.224857+0000","last_unstale":"2024-04-25T08:52:39.645682+0000","last_undegraded":"2024-04-25T08:52:39.645682+0000","last_fullsized":"2024-04-25T08:52:39.645682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833379+0000","last_change":"2024-04-25T08:52:35.309762+0000","last_active":"2024-04-25T08:52:39.833379+0000","last_peered":"2024-04-25T08:52:39.833379+0000","last_clean":"2024-04-25T08:52:39.833379+0000","last_became_active":"2024-04-25T08:52:35.309315+0000","last_became_peered":"2024-04-25T08:52:35.309315+0000","last_unstale":"2024-04-25T08:52:39.833379+0000","last_undegraded":"2024-04-25T08:52:39.833379+0000","last_fullsized":"2024-04-25T08:52:39.833379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.832782+0000","last_change":"2024-04-25T08:52:31.232199+0000","last_active":"2024-04-25T08:52:39.832782+0000","last_peered":"2024-04-25T08:52:39.832782+0000","last_clean":"2024-04-25T08:52:39.832782+0000","last_became_active":"2024-04-25T08:52:31.231928+0000","last_became_peered":"2024-04-25T08:52:31.231928+0000","last_unstale":"2024-04-25T08:52:39.832782+0000","last_undegraded":"2024-04-25T08:52:39.832782+0000","last_fullsized":"2024-04-25T08:52:39.832782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299166+0000","last_change":"2024-04-25T08:52:30.218615+0000","last_active":"2024-04-25T08:52:38.299166+0000","last_peered":"2024-04-25T08:52:38.299166+0000","last_clean":"2024-04-25T08:52:38.299166+0000","last_became_active":"2024-04-25T08:52:30.213378+0000","last_became_peered":"2024-04-25T08:52:30.213378+0000","last_unstale":"2024-04-25T08:52:38.299166+0000","last_undegraded":"2024-04-25T08:52:38.299166+0000","last_fullsized":"2024-04-25T08:52:38.299166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834271+0000","last_change":"2024-04-25T08:52:35.311156+0000","last_active":"2024-04-25T08:52:39.834271+0000","last_peered":"2024-04-25T08:52:39.834271+0000","last_clean":"2024-04-25T08:52:39.834271+0000","last_became_active":"2024-04-25T08:52:35.310700+0000","last_became_peered":"2024-04-25T08:52:35.310700+0000","last_unstale":"2024-04-25T08:52:39.834271+0000","last_undegraded":"2024-04-25T08:52:39.834271+0000","last_fullsized":"2024-04-25T08:52:39.834271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.832874+0000","last_change":"2024-04-25T08:52:31.260514+0000","last_active":"2024-04-25T08:52:39.832874+0000","last_peered":"2024-04-25T08:52:39.832874+0000","last_clean":"2024-04-25T08:52:39.832874+0000","last_became_active":"2024-04-25T08:52:31.260019+0000","last_became_peered":"2024-04-25T08:52:31.260019+0000","last_unstale":"2024-04-25T08:52:39.832874+0000","last_undegraded":"2024-04-25T08:52:39.832874+0000","last_fullsized":"2024-04-25T08:52:39.832874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.602114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306466+0000","last_change":"2024-04-25T08:52:30.245223+0000","last_active":"2024-04-25T08:52:39.306466+0000","last_peered":"2024-04-25T08:52:39.306466+0000","last_clean":"2024-04-25T08:52:39.306466+0000","last_became_active":"2024-04-25T08:52:30.244782+0000","last_became_peered":"2024-04-25T08:52:30.244782+0000","last_unstale":"2024-04-25T08:52:39.306466+0000","last_undegraded":"2024-04-25T08:52:39.306466+0000","last_fullsized":"2024-04-25T08:52:39.306466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836419+0000","last_change":"2024-04-25T08:52:36.002202+0000","last_active":"2024-04-25T08:52:39.836419+0000","last_peered":"2024-04-25T08:52:39.836419+0000","last_clean":"2024-04-25T08:52:39.836419+0000","last_became_active":"2024-04-25T08:52:36.000348+0000","last_became_peered":"2024-04-25T08:52:36.000348+0000","last_unstale":"2024-04-25T08:52:39.836419+0000","last_undegraded":"2024-04-25T08:52:39.836419+0000","last_fullsized":"2024-04-25T08:52:39.836419+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:12.373033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834845+0000","last_change":"2024-04-25T08:52:31.257777+0000","last_active":"2024-04-25T08:52:39.834845+0000","last_peered":"2024-04-25T08:52:39.834845+0000","last_clean":"2024-04-25T08:52:39.834845+0000","last_became_active":"2024-04-25T08:52:31.257295+0000","last_became_peered":"2024-04-25T08:52:31.257295+0000","last_unstale":"2024-04-25T08:52:39.834845+0000","last_undegraded":"2024-04-25T08:52:39.834845+0000","last_fullsized":"2024-04-25T08:52:39.834845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645136+0000","last_change":"2024-04-25T08:52:30.222595+0000","last_active":"2024-04-25T08:52:39.645136+0000","last_peered":"2024-04-25T08:52:39.645136+0000","last_clean":"2024-04-25T08:52:39.645136+0000","last_became_active":"2024-04-25T08:52:30.217788+0000","last_became_peered":"2024-04-25T08:52:30.217788+0000","last_unstale":"2024-04-25T08:52:39.645136+0000","last_undegraded":"2024-04-25T08:52:39.645136+0000","last_fullsized":"2024-04-25T08:52:39.645136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:52.505423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306756+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:52:39.306756+0000","last_peered":"2024-04-25T08:52:39.306756+0000","last_clean":"2024-04-25T08:52:39.306756+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:52:39.306756+0000","last_undegraded":"2024-04-25T08:52:39.306756+0000","last_fullsized":"2024-04-25T08:52:39.306756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.302216+0000","last_change":"2024-04-25T08:52:36.004910+0000","last_active":"2024-04-25T08:52:38.302216+0000","last_peered":"2024-04-25T08:52:38.302216+0000","last_clean":"2024-04-25T08:52:38.302216+0000","last_became_active":"2024-04-25T08:52:36.003878+0000","last_became_peered":"2024-04-25T08:52:36.003878+0000","last_unstale":"2024-04-25T08:52:38.302216+0000","last_undegraded":"2024-04-25T08:52:38.302216+0000","last_fullsized":"2024-04-25T08:52:38.302216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:12.106880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307624+0000","last_change":"2024-04-25T08:52:31.286252+0000","last_active":"2024-04-25T08:52:39.307624+0000","last_peered":"2024-04-25T08:52:39.307624+0000","last_clean":"2024-04-25T08:52:39.307624+0000","last_became_active":"2024-04-25T08:52:31.282622+0000","last_became_peered":"2024-04-25T08:52:31.282622+0000","last_unstale":"2024-04-25T08:52:39.307624+0000","last_undegraded":"2024-04-25T08:52:39.307624+0000","last_fullsized":"2024-04-25T08:52:39.307624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299865+0000","last_change":"2024-04-25T08:52:30.226514+0000","last_active":"2024-04-25T08:52:38.299865+0000","last_peered":"2024-04-25T08:52:38.299865+0000","last_clean":"2024-04-25T08:52:38.299865+0000","last_became_active":"2024-04-25T08:52:30.226243+0000","last_became_peered":"2024-04-25T08:52:30.226243+0000","last_unstale":"2024-04-25T08:52:38.299865+0000","last_undegraded":"2024-04-25T08:52:38.299865+0000","last_fullsized":"2024-04-25T08:52:38.299865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300450+0000","last_change":"2024-04-25T08:52:36.003056+0000","last_active":"2024-04-25T08:52:38.300450+0000","last_peered":"2024-04-25T08:52:38.300450+0000","last_clean":"2024-04-25T08:52:38.300450+0000","last_became_active":"2024-04-25T08:52:36.000882+0000","last_became_peered":"2024-04-25T08:52:36.000882+0000","last_unstale":"2024-04-25T08:52:38.300450+0000","last_undegraded":"2024-04-25T08:52:38.300450+0000","last_fullsized":"2024-04-25T08:52:38.300450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:56.088772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645689+0000","last_change":"2024-04-25T08:52:31.244866+0000","last_active":"2024-04-25T08:52:39.645689+0000","last_peered":"2024-04-25T08:52:39.645689+0000","last_clean":"2024-04-25T08:52:39.645689+0000","last_became_active":"2024-04-25T08:52:31.244249+0000","last_became_peered":"2024-04-25T08:52:31.244249+0000","last_unstale":"2024-04-25T08:52:39.645689+0000","last_undegraded":"2024-04-25T08:52:39.645689+0000","last_fullsized":"2024-04-25T08:52:39.645689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835210+0000","last_change":"2024-04-25T08:52:30.228380+0000","last_active":"2024-04-25T08:52:39.835210+0000","last_peered":"2024-04-25T08:52:39.835210+0000","last_clean":"2024-04-25T08:52:39.835210+0000","last_became_active":"2024-04-25T08:52:30.227732+0000","last_became_peered":"2024-04-25T08:52:30.227732+0000","last_unstale":"2024-04-25T08:52:39.835210+0000","last_undegraded":"2024-04-25T08:52:39.835210+0000","last_fullsized":"2024-04-25T08:52:39.835210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:46.012895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836520+0000","last_change":"2024-04-25T08:52:35.316462+0000","last_active":"2024-04-25T08:52:39.836520+0000","last_peered":"2024-04-25T08:52:39.836520+0000","last_clean":"2024-04-25T08:52:39.836520+0000","last_became_active":"2024-04-25T08:52:35.315760+0000","last_became_peered":"2024-04-25T08:52:35.315760+0000","last_unstale":"2024-04-25T08:52:39.836520+0000","last_undegraded":"2024-04-25T08:52:39.836520+0000","last_fullsized":"2024-04-25T08:52:39.836520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:49.973872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305761+0000","last_change":"2024-04-25T08:52:31.289534+0000","last_active":"2024-04-25T08:52:39.305761+0000","last_peered":"2024-04-25T08:52:39.305761+0000","last_clean":"2024-04-25T08:52:39.305761+0000","last_became_active":"2024-04-25T08:52:31.288580+0000","last_became_peered":"2024-04-25T08:52:31.288580+0000","last_unstale":"2024-04-25T08:52:39.305761+0000","last_undegraded":"2024-04-25T08:52:39.305761+0000","last_fullsized":"2024-04-25T08:52:39.305761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:10.391492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299950+0000","last_change":"2024-04-25T08:52:30.215006+0000","last_active":"2024-04-25T08:52:38.299950+0000","last_peered":"2024-04-25T08:52:38.299950+0000","last_clean":"2024-04-25T08:52:38.299950+0000","last_became_active":"2024-04-25T08:52:30.213814+0000","last_became_peered":"2024-04-25T08:52:30.213814+0000","last_unstale":"2024-04-25T08:52:38.299950+0000","last_undegraded":"2024-04-25T08:52:38.299950+0000","last_fullsized":"2024-04-25T08:52:38.299950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644315+0000","last_change":"2024-04-25T08:52:36.005410+0000","last_active":"2024-04-25T08:52:39.644315+0000","last_peered":"2024-04-25T08:52:39.644315+0000","last_clean":"2024-04-25T08:52:39.644315+0000","last_became_active":"2024-04-25T08:52:36.004309+0000","last_became_peered":"2024-04-25T08:52:36.004309+0000","last_unstale":"2024-04-25T08:52:39.644315+0000","last_undegraded":"2024-04-25T08:52:39.644315+0000","last_fullsized":"2024-04-25T08:52:39.644315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305320+0000","last_change":"2024-04-25T08:52:31.285984+0000","last_active":"2024-04-25T08:52:39.305320+0000","last_peered":"2024-04-25T08:52:39.305320+0000","last_clean":"2024-04-25T08:52:39.305320+0000","last_became_active":"2024-04-25T08:52:31.281988+0000","last_became_peered":"2024-04-25T08:52:31.281988+0000","last_unstale":"2024-04-25T08:52:39.305320+0000","last_undegraded":"2024-04-25T08:52:39.305320+0000","last_fullsized":"2024-04-25T08:52:39.305320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:56.409407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301053+0000","last_change":"2024-04-25T08:52:30.215598+0000","last_active":"2024-04-25T08:52:38.301053+0000","last_peered":"2024-04-25T08:52:38.301053+0000","last_clean":"2024-04-25T08:52:38.301053+0000","last_became_active":"2024-04-25T08:52:30.214765+0000","last_became_peered":"2024-04-25T08:52:30.214765+0000","last_unstale":"2024-04-25T08:52:38.301053+0000","last_undegraded":"2024-04-25T08:52:38.301053+0000","last_fullsized":"2024-04-25T08:52:38.301053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:36.565944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300087+0000","last_change":"2024-04-25T08:52:35.328574+0000","last_active":"2024-04-25T08:52:38.300087+0000","last_peered":"2024-04-25T08:52:38.300087+0000","last_clean":"2024-04-25T08:52:38.300087+0000","last_became_active":"2024-04-25T08:52:35.323891+0000","last_became_peered":"2024-04-25T08:52:35.323891+0000","last_unstale":"2024-04-25T08:52:38.300087+0000","last_undegraded":"2024-04-25T08:52:38.300087+0000","last_fullsized":"2024-04-25T08:52:38.300087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:12.106880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305125+0000","last_change":"2024-04-25T08:52:31.278037+0000","last_active":"2024-04-25T08:52:39.305125+0000","last_peered":"2024-04-25T08:52:39.305125+0000","last_clean":"2024-04-25T08:52:39.305125+0000","last_became_active":"2024-04-25T08:52:31.277117+0000","last_became_peered":"2024-04-25T08:52:31.277117+0000","last_unstale":"2024-04-25T08:52:39.305125+0000","last_undegraded":"2024-04-25T08:52:39.305125+0000","last_fullsized":"2024-04-25T08:52:39.305125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298797+0000","last_change":"2024-04-25T08:52:30.221811+0000","last_active":"2024-04-25T08:52:38.298797+0000","last_peered":"2024-04-25T08:52:38.298797+0000","last_clean":"2024-04-25T08:52:38.298797+0000","last_became_active":"2024-04-25T08:52:30.220494+0000","last_became_peered":"2024-04-25T08:52:30.220494+0000","last_unstale":"2024-04-25T08:52:38.298797+0000","last_undegraded":"2024-04-25T08:52:38.298797+0000","last_fullsized":"2024-04-25T08:52:38.298797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836281+0000","last_change":"2024-04-25T08:52:35.317037+0000","last_active":"2024-04-25T08:52:39.836281+0000","last_peered":"2024-04-25T08:52:39.836281+0000","last_clean":"2024-04-25T08:52:39.836281+0000","last_became_active":"2024-04-25T08:52:35.316491+0000","last_became_peered":"2024-04-25T08:52:35.316491+0000","last_unstale":"2024-04-25T08:52:39.836281+0000","last_undegraded":"2024-04-25T08:52:39.836281+0000","last_fullsized":"2024-04-25T08:52:39.836281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299015+0000","last_change":"2024-04-25T08:52:31.256415+0000","last_active":"2024-04-25T08:52:38.299015+0000","last_peered":"2024-04-25T08:52:38.299015+0000","last_clean":"2024-04-25T08:52:38.299015+0000","last_became_active":"2024-04-25T08:52:31.254747+0000","last_became_peered":"2024-04-25T08:52:31.254747+0000","last_unstale":"2024-04-25T08:52:38.299015+0000","last_undegraded":"2024-04-25T08:52:38.299015+0000","last_fullsized":"2024-04-25T08:52:38.299015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:24.567213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299991+0000","last_change":"2024-04-25T08:52:30.223725+0000","last_active":"2024-04-25T08:52:38.299991+0000","last_peered":"2024-04-25T08:52:38.299991+0000","last_clean":"2024-04-25T08:52:38.299991+0000","last_became_active":"2024-04-25T08:52:30.222820+0000","last_became_peered":"2024-04-25T08:52:30.222820+0000","last_unstale":"2024-04-25T08:52:38.299991+0000","last_undegraded":"2024-04-25T08:52:38.299991+0000","last_fullsized":"2024-04-25T08:52:38.299991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834851+0000","last_change":"2024-04-25T08:52:36.003180+0000","last_active":"2024-04-25T08:52:39.834851+0000","last_peered":"2024-04-25T08:52:39.834851+0000","last_clean":"2024-04-25T08:52:39.834851+0000","last_became_active":"2024-04-25T08:52:36.002030+0000","last_became_peered":"2024-04-25T08:52:36.002030+0000","last_unstale":"2024-04-25T08:52:39.834851+0000","last_undegraded":"2024-04-25T08:52:39.834851+0000","last_fullsized":"2024-04-25T08:52:39.834851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:58.721596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305176+0000","last_change":"2024-04-25T08:52:31.278626+0000","last_active":"2024-04-25T08:52:39.305176+0000","last_peered":"2024-04-25T08:52:39.305176+0000","last_clean":"2024-04-25T08:52:39.305176+0000","last_became_active":"2024-04-25T08:52:31.278027+0000","last_became_peered":"2024-04-25T08:52:31.278027+0000","last_unstale":"2024-04-25T08:52:39.305176+0000","last_undegraded":"2024-04-25T08:52:39.305176+0000","last_fullsized":"2024-04-25T08:52:39.305176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:15.341259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645485+0000","last_change":"2024-04-25T08:52:30.233250+0000","last_active":"2024-04-25T08:52:39.645485+0000","last_peered":"2024-04-25T08:52:39.645485+0000","last_clean":"2024-04-25T08:52:39.645485+0000","last_became_active":"2024-04-25T08:52:30.229813+0000","last_became_peered":"2024-04-25T08:52:30.229813+0000","last_unstale":"2024-04-25T08:52:39.645485+0000","last_undegraded":"2024-04-25T08:52:39.645485+0000","last_fullsized":"2024-04-25T08:52:39.645485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835406+0000","last_change":"2024-04-25T08:52:36.001662+0000","last_active":"2024-04-25T08:52:39.835406+0000","last_peered":"2024-04-25T08:52:39.835406+0000","last_clean":"2024-04-25T08:52:39.835406+0000","last_became_active":"2024-04-25T08:52:36.000433+0000","last_became_peered":"2024-04-25T08:52:36.000433+0000","last_unstale":"2024-04-25T08:52:39.835406+0000","last_undegraded":"2024-04-25T08:52:39.835406+0000","last_fullsized":"2024-04-25T08:52:39.835406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:13.578788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836042+0000","last_change":"2024-04-25T08:52:31.278698+0000","last_active":"2024-04-25T08:52:39.836042+0000","last_peered":"2024-04-25T08:52:39.836042+0000","last_clean":"2024-04-25T08:52:39.836042+0000","last_became_active":"2024-04-25T08:52:31.278388+0000","last_became_peered":"2024-04-25T08:52:31.278388+0000","last_unstale":"2024-04-25T08:52:39.836042+0000","last_undegraded":"2024-04-25T08:52:39.836042+0000","last_fullsized":"2024-04-25T08:52:39.836042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:24.567213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836775+0000","last_change":"2024-04-25T08:52:30.197089+0000","last_active":"2024-04-25T08:52:39.836775+0000","last_peered":"2024-04-25T08:52:39.836775+0000","last_clean":"2024-04-25T08:52:39.836775+0000","last_became_active":"2024-04-25T08:52:30.196592+0000","last_became_peered":"2024-04-25T08:52:30.196592+0000","last_unstale":"2024-04-25T08:52:39.836775+0000","last_undegraded":"2024-04-25T08:52:39.836775+0000","last_fullsized":"2024-04-25T08:52:39.836775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645712+0000","last_change":"2024-04-25T08:52:36.005556+0000","last_active":"2024-04-25T08:52:39.645712+0000","last_peered":"2024-04-25T08:52:39.645712+0000","last_clean":"2024-04-25T08:52:39.645712+0000","last_became_active":"2024-04-25T08:52:36.004511+0000","last_became_peered":"2024-04-25T08:52:36.004511+0000","last_unstale":"2024-04-25T08:52:39.645712+0000","last_undegraded":"2024-04-25T08:52:39.645712+0000","last_fullsized":"2024-04-25T08:52:39.645712+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307456+0000","last_change":"2024-04-25T08:52:31.286270+0000","last_active":"2024-04-25T08:52:39.307456+0000","last_peered":"2024-04-25T08:52:39.307456+0000","last_clean":"2024-04-25T08:52:39.307456+0000","last_became_active":"2024-04-25T08:52:31.281689+0000","last_became_peered":"2024-04-25T08:52:31.281689+0000","last_unstale":"2024-04-25T08:52:39.307456+0000","last_undegraded":"2024-04-25T08:52:39.307456+0000","last_fullsized":"2024-04-25T08:52:39.307456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:41.689859+0000","last_change":"2024-04-25T08:52:30.215322+0000","last_active":"2024-04-25T08:52:41.689859+0000","last_peered":"2024-04-25T08:52:41.689859+0000","last_clean":"2024-04-25T08:52:41.689859+0000","last_became_active":"2024-04-25T08:52:30.214531+0000","last_became_peered":"2024-04-25T08:52:30.214531+0000","last_unstale":"2024-04-25T08:52:41.689859+0000","last_undegraded":"2024-04-25T08:52:41.689859+0000","last_fullsized":"2024-04-25T08:52:41.689859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833881+0000","last_change":"2024-04-25T08:52:35.307400+0000","last_active":"2024-04-25T08:52:39.833881+0000","last_peered":"2024-04-25T08:52:39.833881+0000","last_clean":"2024-04-25T08:52:39.833881+0000","last_became_active":"2024-04-25T08:52:35.304579+0000","last_became_peered":"2024-04-25T08:52:35.304579+0000","last_unstale":"2024-04-25T08:52:39.833881+0000","last_undegraded":"2024-04-25T08:52:39.833881+0000","last_fullsized":"2024-04-25T08:52:39.833881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:24.182927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301767+0000","last_change":"2024-04-25T08:52:31.262376+0000","last_active":"2024-04-25T08:52:38.301767+0000","last_peered":"2024-04-25T08:52:38.301767+0000","last_clean":"2024-04-25T08:52:38.301767+0000","last_became_active":"2024-04-25T08:52:31.261642+0000","last_became_peered":"2024-04-25T08:52:31.261642+0000","last_unstale":"2024-04-25T08:52:38.301767+0000","last_undegraded":"2024-04-25T08:52:38.301767+0000","last_fullsized":"2024-04-25T08:52:38.301767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:12.300628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833373+0000","last_change":"2024-04-25T08:52:30.219030+0000","last_active":"2024-04-25T08:52:39.833373+0000","last_peered":"2024-04-25T08:52:39.833373+0000","last_clean":"2024-04-25T08:52:39.833373+0000","last_became_active":"2024-04-25T08:52:30.218788+0000","last_became_peered":"2024-04-25T08:52:30.218788+0000","last_unstale":"2024-04-25T08:52:39.833373+0000","last_undegraded":"2024-04-25T08:52:39.833373+0000","last_fullsized":"2024-04-25T08:52:39.833373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:34.198893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300753+0000","last_change":"2024-04-25T08:52:35.331866+0000","last_active":"2024-04-25T08:52:38.300753+0000","last_peered":"2024-04-25T08:52:38.300753+0000","last_clean":"2024-04-25T08:52:38.300753+0000","last_became_active":"2024-04-25T08:52:35.330775+0000","last_became_peered":"2024-04-25T08:52:35.330775+0000","last_unstale":"2024-04-25T08:52:38.300753+0000","last_undegraded":"2024-04-25T08:52:38.300753+0000","last_fullsized":"2024-04-25T08:52:38.300753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:59.165277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836632+0000","last_change":"2024-04-25T08:52:31.256646+0000","last_active":"2024-04-25T08:52:39.836632+0000","last_peered":"2024-04-25T08:52:39.836632+0000","last_clean":"2024-04-25T08:52:39.836632+0000","last_became_active":"2024-04-25T08:52:31.256362+0000","last_became_peered":"2024-04-25T08:52:31.256362+0000","last_unstale":"2024-04-25T08:52:39.836632+0000","last_undegraded":"2024-04-25T08:52:39.836632+0000","last_fullsized":"2024-04-25T08:52:39.836632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645734+0000","last_change":"2024-04-25T08:52:30.217888+0000","last_active":"2024-04-25T08:52:39.645734+0000","last_peered":"2024-04-25T08:52:39.645734+0000","last_clean":"2024-04-25T08:52:39.645734+0000","last_became_active":"2024-04-25T08:52:30.217100+0000","last_became_peered":"2024-04-25T08:52:30.217100+0000","last_unstale":"2024-04-25T08:52:39.645734+0000","last_undegraded":"2024-04-25T08:52:39.645734+0000","last_fullsized":"2024-04-25T08:52:39.645734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834254+0000","last_change":"2024-04-25T08:52:35.313671+0000","last_active":"2024-04-25T08:52:39.834254+0000","last_peered":"2024-04-25T08:52:39.834254+0000","last_clean":"2024-04-25T08:52:39.834254+0000","last_became_active":"2024-04-25T08:52:35.312785+0000","last_became_peered":"2024-04-25T08:52:35.312785+0000","last_unstale":"2024-04-25T08:52:39.834254+0000","last_undegraded":"2024-04-25T08:52:39.834254+0000","last_fullsized":"2024-04-25T08:52:39.834254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:23.828143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645723+0000","last_change":"2024-04-25T08:52:31.273223+0000","last_active":"2024-04-25T08:52:39.645723+0000","last_peered":"2024-04-25T08:52:39.645723+0000","last_clean":"2024-04-25T08:52:39.645723+0000","last_became_active":"2024-04-25T08:52:31.272627+0000","last_became_peered":"2024-04-25T08:52:31.272627+0000","last_unstale":"2024-04-25T08:52:39.645723+0000","last_undegraded":"2024-04-25T08:52:39.645723+0000","last_fullsized":"2024-04-25T08:52:39.645723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303022+0000","last_change":"2024-04-25T08:52:30.229619+0000","last_active":"2024-04-25T08:52:38.303022+0000","last_peered":"2024-04-25T08:52:38.303022+0000","last_clean":"2024-04-25T08:52:38.303022+0000","last_became_active":"2024-04-25T08:52:30.228770+0000","last_became_peered":"2024-04-25T08:52:30.228770+0000","last_unstale":"2024-04-25T08:52:38.303022+0000","last_undegraded":"2024-04-25T08:52:38.303022+0000","last_fullsized":"2024-04-25T08:52:38.303022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833502+0000","last_change":"2024-04-25T08:52:36.008479+0000","last_active":"2024-04-25T08:52:39.833502+0000","last_peered":"2024-04-25T08:52:39.833502+0000","last_clean":"2024-04-25T08:52:39.833502+0000","last_became_active":"2024-04-25T08:52:36.007404+0000","last_became_peered":"2024-04-25T08:52:36.007404+0000","last_unstale":"2024-04-25T08:52:39.833502+0000","last_undegraded":"2024-04-25T08:52:39.833502+0000","last_fullsized":"2024-04-25T08:52:39.833502+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:54.631138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836260+0000","last_change":"2024-04-25T08:52:36.004959+0000","last_active":"2024-04-25T08:52:39.836260+0000","last_peered":"2024-04-25T08:52:39.836260+0000","last_clean":"2024-04-25T08:52:39.836260+0000","last_became_active":"2024-04-25T08:52:36.003575+0000","last_became_peered":"2024-04-25T08:52:36.003575+0000","last_unstale":"2024-04-25T08:52:39.836260+0000","last_undegraded":"2024-04-25T08:52:39.836260+0000","last_fullsized":"2024-04-25T08:52:39.836260+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834357+0000","last_change":"2024-04-25T08:52:31.231230+0000","last_active":"2024-04-25T08:52:39.834357+0000","last_peered":"2024-04-25T08:52:39.834357+0000","last_clean":"2024-04-25T08:52:39.834357+0000","last_became_active":"2024-04-25T08:52:31.230951+0000","last_became_peered":"2024-04-25T08:52:31.230951+0000","last_unstale":"2024-04-25T08:52:39.834357+0000","last_undegraded":"2024-04-25T08:52:39.834357+0000","last_fullsized":"2024-04-25T08:52:39.834357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644784+0000","last_change":"2024-04-25T08:52:30.233534+0000","last_active":"2024-04-25T08:52:39.644784+0000","last_peered":"2024-04-25T08:52:39.644784+0000","last_clean":"2024-04-25T08:52:39.644784+0000","last_became_active":"2024-04-25T08:52:30.232832+0000","last_became_peered":"2024-04-25T08:52:30.232832+0000","last_unstale":"2024-04-25T08:52:39.644784+0000","last_undegraded":"2024-04-25T08:52:39.644784+0000","last_fullsized":"2024-04-25T08:52:39.644784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303102+0000","last_change":"2024-04-25T08:52:35.317543+0000","last_active":"2024-04-25T08:52:38.303102+0000","last_peered":"2024-04-25T08:52:38.303102+0000","last_clean":"2024-04-25T08:52:38.303102+0000","last_became_active":"2024-04-25T08:52:35.316474+0000","last_became_peered":"2024-04-25T08:52:35.316474+0000","last_unstale":"2024-04-25T08:52:38.303102+0000","last_undegraded":"2024-04-25T08:52:38.303102+0000","last_fullsized":"2024-04-25T08:52:38.303102+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:35.695987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834357+0000","last_change":"2024-04-25T08:52:31.257812+0000","last_active":"2024-04-25T08:52:39.834357+0000","last_peered":"2024-04-25T08:52:39.834357+0000","last_clean":"2024-04-25T08:52:39.834357+0000","last_became_active":"2024-04-25T08:52:31.257528+0000","last_became_peered":"2024-04-25T08:52:31.257528+0000","last_unstale":"2024-04-25T08:52:39.834357+0000","last_undegraded":"2024-04-25T08:52:39.834357+0000","last_fullsized":"2024-04-25T08:52:39.834357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835503+0000","last_change":"2024-04-25T08:52:30.217142+0000","last_active":"2024-04-25T08:52:39.835503+0000","last_peered":"2024-04-25T08:52:39.835503+0000","last_clean":"2024-04-25T08:52:39.835503+0000","last_became_active":"2024-04-25T08:52:30.216729+0000","last_became_peered":"2024-04-25T08:52:30.216729+0000","last_unstale":"2024-04-25T08:52:39.835503+0000","last_undegraded":"2024-04-25T08:52:39.835503+0000","last_fullsized":"2024-04-25T08:52:39.835503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300819+0000","last_change":"2024-04-25T08:52:30.228704+0000","last_active":"2024-04-25T08:52:38.300819+0000","last_peered":"2024-04-25T08:52:38.300819+0000","last_clean":"2024-04-25T08:52:38.300819+0000","last_became_active":"2024-04-25T08:52:30.227861+0000","last_became_peered":"2024-04-25T08:52:30.227861+0000","last_unstale":"2024-04-25T08:52:38.300819+0000","last_undegraded":"2024-04-25T08:52:38.300819+0000","last_fullsized":"2024-04-25T08:52:38.300819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835140+0000","last_change":"2024-04-25T08:52:31.258702+0000","last_active":"2024-04-25T08:52:39.835140+0000","last_peered":"2024-04-25T08:52:39.835140+0000","last_clean":"2024-04-25T08:52:39.835140+0000","last_became_active":"2024-04-25T08:52:31.258349+0000","last_became_peered":"2024-04-25T08:52:31.258349+0000","last_unstale":"2024-04-25T08:52:39.835140+0000","last_undegraded":"2024-04-25T08:52:39.835140+0000","last_fullsized":"2024-04-25T08:52:39.835140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299697+0000","last_change":"2024-04-25T08:52:35.313396+0000","last_active":"2024-04-25T08:52:38.299697+0000","last_peered":"2024-04-25T08:52:38.299697+0000","last_clean":"2024-04-25T08:52:38.299697+0000","last_became_active":"2024-04-25T08:52:35.312845+0000","last_became_peered":"2024-04-25T08:52:35.312845+0000","last_unstale":"2024-04-25T08:52:38.299697+0000","last_undegraded":"2024-04-25T08:52:38.299697+0000","last_fullsized":"2024-04-25T08:52:38.299697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.648302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836001+0000","last_change":"2024-04-25T08:52:30.239169+0000","last_active":"2024-04-25T08:52:39.836001+0000","last_peered":"2024-04-25T08:52:39.836001+0000","last_clean":"2024-04-25T08:52:39.836001+0000","last_became_active":"2024-04-25T08:52:30.238820+0000","last_became_peered":"2024-04-25T08:52:30.238820+0000","last_unstale":"2024-04-25T08:52:39.836001+0000","last_undegraded":"2024-04-25T08:52:39.836001+0000","last_fullsized":"2024-04-25T08:52:39.836001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300334+0000","last_change":"2024-04-25T08:52:31.243105+0000","last_active":"2024-04-25T08:52:38.300334+0000","last_peered":"2024-04-25T08:52:38.300334+0000","last_clean":"2024-04-25T08:52:38.300334+0000","last_became_active":"2024-04-25T08:52:31.242326+0000","last_became_peered":"2024-04-25T08:52:31.242326+0000","last_unstale":"2024-04-25T08:52:38.300334+0000","last_undegraded":"2024-04-25T08:52:38.300334+0000","last_fullsized":"2024-04-25T08:52:38.300334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:04.149068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834664+0000","last_change":"2024-04-25T08:52:35.313390+0000","last_active":"2024-04-25T08:52:39.834664+0000","last_peered":"2024-04-25T08:52:39.834664+0000","last_clean":"2024-04-25T08:52:39.834664+0000","last_became_active":"2024-04-25T08:52:35.312750+0000","last_became_peered":"2024-04-25T08:52:35.312750+0000","last_unstale":"2024-04-25T08:52:39.834664+0000","last_undegraded":"2024-04-25T08:52:39.834664+0000","last_fullsized":"2024-04-25T08:52:39.834664+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:49.016809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298785+0000","last_change":"2024-04-25T08:52:31.257097+0000","last_active":"2024-04-25T08:52:38.298785+0000","last_peered":"2024-04-25T08:52:38.298785+0000","last_clean":"2024-04-25T08:52:38.298785+0000","last_became_active":"2024-04-25T08:52:31.255697+0000","last_became_peered":"2024-04-25T08:52:31.255697+0000","last_unstale":"2024-04-25T08:52:38.298785+0000","last_undegraded":"2024-04-25T08:52:38.298785+0000","last_fullsized":"2024-04-25T08:52:38.298785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298858+0000","last_change":"2024-04-25T08:52:31.237994+0000","last_active":"2024-04-25T08:52:38.298858+0000","last_peered":"2024-04-25T08:52:38.298858+0000","last_clean":"2024-04-25T08:52:38.298858+0000","last_became_active":"2024-04-25T08:52:31.237190+0000","last_became_peered":"2024-04-25T08:52:31.237190+0000","last_unstale":"2024-04-25T08:52:38.298858+0000","last_undegraded":"2024-04-25T08:52:38.298858+0000","last_fullsized":"2024-04-25T08:52:38.298858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:35.208568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645427+0000","last_change":"2024-04-25T08:52:31.248873+0000","last_active":"2024-04-25T08:52:39.645427+0000","last_peered":"2024-04-25T08:52:39.645427+0000","last_clean":"2024-04-25T08:52:39.645427+0000","last_became_active":"2024-04-25T08:52:31.248359+0000","last_became_peered":"2024-04-25T08:52:31.248359+0000","last_unstale":"2024-04-25T08:52:39.645427+0000","last_undegraded":"2024-04-25T08:52:39.645427+0000","last_fullsized":"2024-04-25T08:52:39.645427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836722+0000","last_change":"2024-04-25T08:52:31.256155+0000","last_active":"2024-04-25T08:52:39.836722+0000","last_peered":"2024-04-25T08:52:39.836722+0000","last_clean":"2024-04-25T08:52:39.836722+0000","last_became_active":"2024-04-25T08:52:31.255884+0000","last_became_peered":"2024-04-25T08:52:31.255884+0000","last_unstale":"2024-04-25T08:52:39.836722+0000","last_undegraded":"2024-04-25T08:52:39.836722+0000","last_fullsized":"2024-04-25T08:52:39.836722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834472+0000","last_change":"2024-04-25T08:52:31.266984+0000","last_active":"2024-04-25T08:52:39.834472+0000","last_peered":"2024-04-25T08:52:39.834472+0000","last_clean":"2024-04-25T08:52:39.834472+0000","last_became_active":"2024-04-25T08:52:31.266697+0000","last_became_peered":"2024-04-25T08:52:31.266697+0000","last_unstale":"2024-04-25T08:52:39.834472+0000","last_undegraded":"2024-04-25T08:52:39.834472+0000","last_fullsized":"2024-04-25T08:52:39.834472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833201+0000","last_change":"2024-04-25T08:52:31.234570+0000","last_active":"2024-04-25T08:52:39.833201+0000","last_peered":"2024-04-25T08:52:39.833201+0000","last_clean":"2024-04-25T08:52:39.833201+0000","last_became_active":"2024-04-25T08:52:31.234049+0000","last_became_peered":"2024-04-25T08:52:31.234049+0000","last_unstale":"2024-04-25T08:52:39.833201+0000","last_undegraded":"2024-04-25T08:52:39.833201+0000","last_fullsized":"2024-04-25T08:52:39.833201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303100+0000","last_change":"2024-04-25T08:52:31.271083+0000","last_active":"2024-04-25T08:52:38.303100+0000","last_peered":"2024-04-25T08:52:38.303100+0000","last_clean":"2024-04-25T08:52:38.303100+0000","last_became_active":"2024-04-25T08:52:31.268755+0000","last_became_peered":"2024-04-25T08:52:31.268755+0000","last_unstale":"2024-04-25T08:52:38.303100+0000","last_undegraded":"2024-04-25T08:52:38.303100+0000","last_fullsized":"2024-04-25T08:52:38.303100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834579+0000","last_change":"2024-04-25T08:52:31.265576+0000","last_active":"2024-04-25T08:52:39.834579+0000","last_peered":"2024-04-25T08:52:39.834579+0000","last_clean":"2024-04-25T08:52:39.834579+0000","last_became_active":"2024-04-25T08:52:31.265273+0000","last_became_peered":"2024-04-25T08:52:31.265273+0000","last_unstale":"2024-04-25T08:52:39.834579+0000","last_undegraded":"2024-04-25T08:52:39.834579+0000","last_fullsized":"2024-04-25T08:52:39.834579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644626+0000","last_change":"2024-04-25T08:52:31.247316+0000","last_active":"2024-04-25T08:52:39.644626+0000","last_peered":"2024-04-25T08:52:39.644626+0000","last_clean":"2024-04-25T08:52:39.644626+0000","last_became_active":"2024-04-25T08:52:31.246547+0000","last_became_peered":"2024-04-25T08:52:31.246547+0000","last_unstale":"2024-04-25T08:52:39.644626+0000","last_undegraded":"2024-04-25T08:52:39.644626+0000","last_fullsized":"2024-04-25T08:52:39.644626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833082+0000","last_change":"2024-04-25T08:52:31.265084+0000","last_active":"2024-04-25T08:52:39.833082+0000","last_peered":"2024-04-25T08:52:39.833082+0000","last_clean":"2024-04-25T08:52:39.833082+0000","last_became_active":"2024-04-25T08:52:31.264883+0000","last_became_peered":"2024-04-25T08:52:31.264883+0000","last_unstale":"2024-04-25T08:52:39.833082+0000","last_undegraded":"2024-04-25T08:52:39.833082+0000","last_fullsized":"2024-04-25T08:52:39.833082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645197+0000","last_change":"2024-04-25T08:52:31.280090+0000","last_active":"2024-04-25T08:52:39.645197+0000","last_peered":"2024-04-25T08:52:39.645197+0000","last_clean":"2024-04-25T08:52:39.645197+0000","last_became_active":"2024-04-25T08:52:31.279453+0000","last_became_peered":"2024-04-25T08:52:31.279453+0000","last_unstale":"2024-04-25T08:52:39.645197+0000","last_undegraded":"2024-04-25T08:52:39.645197+0000","last_fullsized":"2024-04-25T08:52:39.645197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836409+0000","last_change":"2024-04-25T08:52:31.271765+0000","last_active":"2024-04-25T08:52:39.836409+0000","last_peered":"2024-04-25T08:52:39.836409+0000","last_clean":"2024-04-25T08:52:39.836409+0000","last_became_active":"2024-04-25T08:52:31.271438+0000","last_became_peered":"2024-04-25T08:52:31.271438+0000","last_unstale":"2024-04-25T08:52:39.836409+0000","last_undegraded":"2024-04-25T08:52:39.836409+0000","last_fullsized":"2024-04-25T08:52:39.836409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301809+0000","last_change":"2024-04-25T08:52:31.270307+0000","last_active":"2024-04-25T08:52:38.301809+0000","last_peered":"2024-04-25T08:52:38.301809+0000","last_clean":"2024-04-25T08:52:38.301809+0000","last_became_active":"2024-04-25T08:52:31.269440+0000","last_became_peered":"2024-04-25T08:52:31.269440+0000","last_unstale":"2024-04-25T08:52:38.301809+0000","last_undegraded":"2024-04-25T08:52:38.301809+0000","last_fullsized":"2024-04-25T08:52:38.301809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305298+0000","last_change":"2024-04-25T08:52:31.281491+0000","last_active":"2024-04-25T08:52:39.305298+0000","last_peered":"2024-04-25T08:52:39.305298+0000","last_clean":"2024-04-25T08:52:39.305298+0000","last_became_active":"2024-04-25T08:52:31.276333+0000","last_became_peered":"2024-04-25T08:52:31.276333+0000","last_unstale":"2024-04-25T08:52:39.305298+0000","last_undegraded":"2024-04-25T08:52:39.305298+0000","last_fullsized":"2024-04-25T08:52:39.305298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:31.614898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299491+0000","last_change":"2024-04-25T08:52:31.237856+0000","last_active":"2024-04-25T08:52:38.299491+0000","last_peered":"2024-04-25T08:52:38.299491+0000","last_clean":"2024-04-25T08:52:38.299491+0000","last_became_active":"2024-04-25T08:52:31.237582+0000","last_became_peered":"2024-04-25T08:52:31.237582+0000","last_unstale":"2024-04-25T08:52:38.299491+0000","last_undegraded":"2024-04-25T08:52:38.299491+0000","last_fullsized":"2024-04-25T08:52:38.299491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:05.271538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.646089+0000","last_change":"2024-04-25T08:52:31.283655+0000","last_active":"2024-04-25T08:52:39.646089+0000","last_peered":"2024-04-25T08:52:39.646089+0000","last_clean":"2024-04-25T08:52:39.646089+0000","last_became_active":"2024-04-25T08:52:31.282969+0000","last_became_peered":"2024-04-25T08:52:31.282969+0000","last_unstale":"2024-04-25T08:52:39.646089+0000","last_undegraded":"2024-04-25T08:52:39.646089+0000","last_fullsized":"2024-04-25T08:52:39.646089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:08.025791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644525+0000","last_change":"2024-04-25T08:52:31.256907+0000","last_active":"2024-04-25T08:52:39.644525+0000","last_peered":"2024-04-25T08:52:39.644525+0000","last_clean":"2024-04-25T08:52:39.644525+0000","last_became_active":"2024-04-25T08:52:31.255494+0000","last_became_peered":"2024-04-25T08:52:31.255494+0000","last_unstale":"2024-04-25T08:52:39.644525+0000","last_undegraded":"2024-04-25T08:52:39.644525+0000","last_fullsized":"2024-04-25T08:52:39.644525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834186+0000","last_change":"2024-04-25T08:52:31.270805+0000","last_active":"2024-04-25T08:52:39.834186+0000","last_peered":"2024-04-25T08:52:39.834186+0000","last_clean":"2024-04-25T08:52:39.834186+0000","last_became_active":"2024-04-25T08:52:31.270594+0000","last_became_peered":"2024-04-25T08:52:31.270594+0000","last_unstale":"2024-04-25T08:52:39.834186+0000","last_undegraded":"2024-04-25T08:52:39.834186+0000","last_fullsized":"2024-04-25T08:52:39.834186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836035+0000","last_change":"2024-04-25T08:52:31.252734+0000","last_active":"2024-04-25T08:52:39.836035+0000","last_peered":"2024-04-25T08:52:39.836035+0000","last_clean":"2024-04-25T08:52:39.836035+0000","last_became_active":"2024-04-25T08:52:31.252476+0000","last_became_peered":"2024-04-25T08:52:31.252476+0000","last_unstale":"2024-04-25T08:52:39.836035+0000","last_undegraded":"2024-04-25T08:52:39.836035+0000","last_fullsized":"2024-04-25T08:52:39.836035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833928+0000","last_change":"2024-04-25T08:52:31.242347+0000","last_active":"2024-04-25T08:52:39.833928+0000","last_peered":"2024-04-25T08:52:39.833928+0000","last_clean":"2024-04-25T08:52:39.833928+0000","last_became_active":"2024-04-25T08:52:31.242082+0000","last_became_peered":"2024-04-25T08:52:31.242082+0000","last_unstale":"2024-04-25T08:52:39.833928+0000","last_undegraded":"2024-04-25T08:52:39.833928+0000","last_fullsized":"2024-04-25T08:52:39.833928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301519+0000","last_change":"2024-04-25T08:52:31.258568+0000","last_active":"2024-04-25T08:52:38.301519+0000","last_peered":"2024-04-25T08:52:38.301519+0000","last_clean":"2024-04-25T08:52:38.301519+0000","last_became_active":"2024-04-25T08:52:31.257438+0000","last_became_peered":"2024-04-25T08:52:31.257438+0000","last_unstale":"2024-04-25T08:52:38.301519+0000","last_undegraded":"2024-04-25T08:52:38.301519+0000","last_fullsized":"2024-04-25T08:52:38.301519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:47.022570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.302377+0000","last_change":"2024-04-25T08:52:31.254993+0000","last_active":"2024-04-25T08:52:38.302377+0000","last_peered":"2024-04-25T08:52:38.302377+0000","last_clean":"2024-04-25T08:52:38.302377+0000","last_became_active":"2024-04-25T08:52:31.254072+0000","last_became_peered":"2024-04-25T08:52:31.254072+0000","last_unstale":"2024-04-25T08:52:38.302377+0000","last_undegraded":"2024-04-25T08:52:38.302377+0000","last_fullsized":"2024-04-25T08:52:38.302377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303017+0000","last_change":"2024-04-25T08:52:31.247059+0000","last_active":"2024-04-25T08:52:38.303017+0000","last_peered":"2024-04-25T08:52:38.303017+0000","last_clean":"2024-04-25T08:52:38.303017+0000","last_became_active":"2024-04-25T08:52:31.246364+0000","last_became_peered":"2024-04-25T08:52:31.246364+0000","last_unstale":"2024-04-25T08:52:38.303017+0000","last_undegraded":"2024-04-25T08:52:38.303017+0000","last_fullsized":"2024-04-25T08:52:38.303017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.308187+0000","last_change":"2024-04-25T08:52:31.289253+0000","last_active":"2024-04-25T08:52:39.308187+0000","last_peered":"2024-04-25T08:52:39.308187+0000","last_clean":"2024-04-25T08:52:39.308187+0000","last_became_active":"2024-04-25T08:52:31.288175+0000","last_became_peered":"2024-04-25T08:52:31.288175+0000","last_unstale":"2024-04-25T08:52:39.308187+0000","last_undegraded":"2024-04-25T08:52:39.308187+0000","last_fullsized":"2024-04-25T08:52:39.308187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:08.025791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834144+0000","last_change":"2024-04-25T08:52:31.256539+0000","last_active":"2024-04-25T08:52:39.834144+0000","last_peered":"2024-04-25T08:52:39.834144+0000","last_clean":"2024-04-25T08:52:39.834144+0000","last_became_active":"2024-04-25T08:52:31.255921+0000","last_became_peered":"2024-04-25T08:52:31.255921+0000","last_unstale":"2024-04-25T08:52:39.834144+0000","last_undegraded":"2024-04-25T08:52:39.834144+0000","last_fullsized":"2024-04-25T08:52:39.834144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306615+0000","last_change":"2024-04-25T08:52:31.277410+0000","last_active":"2024-04-25T08:52:39.306615+0000","last_peered":"2024-04-25T08:52:39.306615+0000","last_clean":"2024-04-25T08:52:39.306615+0000","last_became_active":"2024-04-25T08:52:31.276941+0000","last_became_peered":"2024-04-25T08:52:31.276941+0000","last_unstale":"2024-04-25T08:52:39.306615+0000","last_undegraded":"2024-04-25T08:52:39.306615+0000","last_fullsized":"2024-04-25T08:52:39.306615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645097+0000","last_change":"2024-04-25T08:52:31.256171+0000","last_active":"2024-04-25T08:52:39.645097+0000","last_peered":"2024-04-25T08:52:39.645097+0000","last_clean":"2024-04-25T08:52:39.645097+0000","last_became_active":"2024-04-25T08:52:31.255417+0000","last_became_peered":"2024-04-25T08:52:31.255417+0000","last_unstale":"2024-04-25T08:52:39.645097+0000","last_undegraded":"2024-04-25T08:52:39.645097+0000","last_fullsized":"2024-04-25T08:52:39.645097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305186+0000","last_change":"2024-04-25T08:52:31.269978+0000","last_active":"2024-04-25T08:52:39.305186+0000","last_peered":"2024-04-25T08:52:39.305186+0000","last_clean":"2024-04-25T08:52:39.305186+0000","last_became_active":"2024-04-25T08:52:31.269108+0000","last_became_peered":"2024-04-25T08:52:31.269108+0000","last_unstale":"2024-04-25T08:52:39.305186+0000","last_undegraded":"2024-04-25T08:52:39.305186+0000","last_fullsized":"2024-04-25T08:52:39.305186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645458+0000","last_change":"2024-04-25T08:52:31.248569+0000","last_active":"2024-04-25T08:52:39.645458+0000","last_peered":"2024-04-25T08:52:39.645458+0000","last_clean":"2024-04-25T08:52:39.645458+0000","last_became_active":"2024-04-25T08:52:31.247655+0000","last_became_peered":"2024-04-25T08:52:31.247655+0000","last_unstale":"2024-04-25T08:52:39.645458+0000","last_undegraded":"2024-04-25T08:52:39.645458+0000","last_fullsized":"2024-04-25T08:52:39.645458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833573+0000","last_change":"2024-04-25T08:52:31.252449+0000","last_active":"2024-04-25T08:52:39.833573+0000","last_peered":"2024-04-25T08:52:39.833573+0000","last_clean":"2024-04-25T08:52:39.833573+0000","last_became_active":"2024-04-25T08:52:31.252212+0000","last_became_peered":"2024-04-25T08:52:31.252212+0000","last_unstale":"2024-04-25T08:52:39.833573+0000","last_undegraded":"2024-04-25T08:52:39.833573+0000","last_fullsized":"2024-04-25T08:52:39.833573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833700+0000","last_change":"2024-04-25T08:52:31.254735+0000","last_active":"2024-04-25T08:52:39.833700+0000","last_peered":"2024-04-25T08:52:39.833700+0000","last_clean":"2024-04-25T08:52:39.833700+0000","last_became_active":"2024-04-25T08:52:31.254523+0000","last_became_peered":"2024-04-25T08:52:31.254523+0000","last_unstale":"2024-04-25T08:52:39.833700+0000","last_undegraded":"2024-04-25T08:52:39.833700+0000","last_fullsized":"2024-04-25T08:52:39.833700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305895+0000","last_change":"2024-04-25T08:52:31.288927+0000","last_active":"2024-04-25T08:52:39.305895+0000","last_peered":"2024-04-25T08:52:39.305895+0000","last_clean":"2024-04-25T08:52:39.305895+0000","last_became_active":"2024-04-25T08:52:31.287790+0000","last_became_peered":"2024-04-25T08:52:31.287790+0000","last_unstale":"2024-04-25T08:52:39.305895+0000","last_undegraded":"2024-04-25T08:52:39.305895+0000","last_fullsized":"2024-04-25T08:52:39.305895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:52.007683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":709746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":122638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":118886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":118922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":122638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":118796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":118956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":897024,"data_stored":709802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:52:55.379 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:52:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09afcda640 1 Processor -- start 2024-04-25T08:52:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09afcda640 1 -- start start 2024-04-25T08:52:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09afcda640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09a80b0a30 0x7f09a80cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09afcda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a809a180 0x7f09a815d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09afcda640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a80b0e10 con 0x7f09a806f4a0 2024-04-25T08:52:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09afcda640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a80d0d90 con 0x7f09a80b0a30 2024-04-25T08:52:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09afcda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80d0180 con 0x7f09a809a180 2024-04-25T08:52:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09ada76640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09a80b0a30 0x7f09a80cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09ada76640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f09a80b0a30 0x7f09a80cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46320/0 (socket says 172.21.15.37:46320) 2024-04-25T08:52:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09ada76640 1 -- 172.21.15.37:0/3280687837 learned_addr learned my addr 172.21.15.37:0/3280687837 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09ae277640 1 --2- 172.21.15.37:0/3280687837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a809a180 0x7f09a815d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09ada76640 1 -- 172.21.15.37:0/3280687837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a809a180 msgr2=0x7f09a815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09ada76640 1 --2- 172.21.15.37:0/3280687837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a809a180 0x7f09a815d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09ada76640 1 -- 172.21.15.37:0/3280687837 >> v1:172.21.15.203:6789/0 conn(0x7f09a806f4a0 legacy=0x7f09a8099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:55.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.540+0000 7f09ada76640 1 -- 172.21.15.37:0/3280687837 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a80d0460 con 0x7f09a80b0a30 2024-04-25T08:52:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09ada76640 1 --2- 172.21.15.37:0/3280687837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a80b0a30 0x7f09a80cf890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0998009870 tx=0x7f099801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c017a04d9efe4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09aca74640 1 -- 172.21.15.37:0/3280687837 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980aa610 con 0x7f09a80b0a30 2024-04-25T08:52:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09aca74640 1 -- 172.21.15.37:0/3280687837 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09980af070 con 0x7f09a80b0a30 2024-04-25T08:52:55.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09aca74640 1 -- 172.21.15.37:0/3280687837 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980aa9d0 con 0x7f09a80b0a30 2024-04-25T08:52:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 -- 172.21.15.37:0/3280687837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a80b0a30 msgr2=0x7f09a80cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 --2- 172.21.15.37:0/3280687837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a80b0a30 0x7f09a80cf890 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0998009870 tx=0x7f099801c4f0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 -- 172.21.15.37:0/3280687837 shutdown_connections 2024-04-25T08:52:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 --2- 172.21.15.37:0/3280687837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a809a180 0x7f09a815d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 --2- 172.21.15.37:0/3280687837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a80b0a30 0x7f09a80cf890 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 -- 172.21.15.37:0/3280687837 >> 172.21.15.37:0/3280687837 conn(0x7f09a80b30b0 msgr2=0x7f09a80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:55.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 -- 172.21.15.37:0/3280687837 shutdown_connections 2024-04-25T08:52:55.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 -- 172.21.15.37:0/3280687837 wait complete. 2024-04-25T08:52:55.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 Processor -- start 2024-04-25T08:52:55.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 -- start start 2024-04-25T08:52:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a806f4a0 0x7f09a80c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a809a180 0x7f09a81543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a80b0a30 0x7f09a8154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80cbd90 con 0x7f09a806f4a0 2024-04-25T08:52:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80cbc10 con 0x7f09a80b0a30 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09afcda640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a80cbf10 con 0x7f09a809a180 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09ada76640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a806f4a0 0x7f09a80c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09ada76640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a806f4a0 0x7f09a80c88f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46322/0 (socket says 172.21.15.37:46322) 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09ada76640 1 -- 172.21.15.37:0/2164720425 learned_addr learned my addr 172.21.15.37:0/2164720425 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09ae277640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a80b0a30 0x7f09a8154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09ad275640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a809a180 0x7f09a81543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09ada76640 1 -- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a809a180 msgr2=0x7f09a81543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09ada76640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a809a180 0x7f09a81543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09ada76640 1 -- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a80b0a30 msgr2=0x7f09a8154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09ada76640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a80b0a30 0x7f09a8154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.544+0000 7f09ada76640 1 -- 172.21.15.37:0/2164720425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a80c5bb0 con 0x7f09a806f4a0 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f09ada76640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a806f4a0 0x7f09a80c88f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f09980ac040 tx=0x7f099801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f099e7fc640 1 -- 172.21.15.37:0/2164720425 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980bd5f0 con 0x7f09a806f4a0 2024-04-25T08:52:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f099e7fc640 1 -- 172.21.15.37:0/2164720425 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09980bd7b0 con 0x7f09a806f4a0 2024-04-25T08:52:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f099e7fc640 1 -- 172.21.15.37:0/2164720425 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980bdba0 con 0x7f09a806f4a0 2024-04-25T08:52:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f09afcda640 1 -- 172.21.15.37:0/2164720425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a80c5e90 con 0x7f09a806f4a0 2024-04-25T08:52:55.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f09afcda640 1 -- 172.21.15.37:0/2164720425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a80c8e30 con 0x7f09a806f4a0 2024-04-25T08:52:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f09afcda640 1 -- 172.21.15.37:0/2164720425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a804efa0 con 0x7f09a806f4a0 2024-04-25T08:52:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f099e7fc640 1 -- 172.21.15.37:0/2164720425 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09980b33f0 con 0x7f09a806f4a0 2024-04-25T08:52:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f099e7fc640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f098c066f60 0x7f098c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f099e7fc640 1 -- 172.21.15.37:0/2164720425 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0998120000 con 0x7f09a806f4a0 2024-04-25T08:52:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f09ad275640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f098c066f60 0x7f098c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.548+0000 7f09ad275640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f098c066f60 0x7f098c069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f09a80b69a0 tx=0x7f09a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:55.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.552+0000 7f099e7fc640 1 -- 172.21.15.37:0/2164720425 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09980c3060 con 0x7f09a806f4a0 2024-04-25T08:52:55.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.704+0000 7f09afcda640 1 -- 172.21.15.37:0/2164720425 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09a80b3ee0 con 0x7f098c066f60 2024-04-25T08:52:55.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.728+0000 7f099e7fc640 1 -- 172.21.15.37:0/2164720425 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332534 (secure 0 0 0) 0x7f09a80b3ee0 con 0x7f098c066f60 2024-04-25T08:52:55.723 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 -- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f098c066f60 msgr2=0x7f098c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f098c066f60 0x7f098c069420 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f09a80b69a0 tx=0x7f09a4009040 comp rx=0 tx=0).stop 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 -- 172.21.15.37:0/2164720425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a806f4a0 msgr2=0x7f09a80c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a806f4a0 0x7f09a80c88f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f09980ac040 tx=0x7f099801ad30 comp rx=0 tx=0).stop 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 -- 172.21.15.37:0/2164720425 shutdown_connections 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f098c066f60 0x7f098c069420 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09a80b0a30 0x7f09a8154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09a809a180 0x7f09a81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 --2- 172.21.15.37:0/2164720425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a806f4a0 0x7f09a80c88f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 -- 172.21.15.37:0/2164720425 >> 172.21.15.37:0/2164720425 conn(0x7f09a80b30b0 msgr2=0x7f09a8073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 -- 172.21.15.37:0/2164720425 shutdown_connections 2024-04-25T08:52:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.732+0000 7f09afcda640 1 -- 172.21.15.37:0/2164720425 wait complete. 2024-04-25T08:52:55.740 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-25T08:52:55.126529+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219668,"kb_used_data":3496,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755052,"statfs":{"total":773094113280,"available":772869173248,"internally_reserved":0,"allocated":3579904,"data_stored":2140384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004976"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836749+0000","last_change":"2024-04-25T08:52:30.238555+0000","last_active":"2024-04-25T08:52:39.836749+0000","last_peered":"2024-04-25T08:52:39.836749+0000","last_clean":"2024-04-25T08:52:39.836749+0000","last_became_active":"2024-04-25T08:52:30.238074+0000","last_became_peered":"2024-04-25T08:52:30.238074+0000","last_unstale":"2024-04-25T08:52:39.836749+0000","last_undegraded":"2024-04-25T08:52:39.836749+0000","last_fullsized":"2024-04-25T08:52:39.836749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:52.505423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834324+0000","last_change":"2024-04-25T08:52:31.237632+0000","last_active":"2024-04-25T08:52:39.834324+0000","last_peered":"2024-04-25T08:52:39.834324+0000","last_clean":"2024-04-25T08:52:39.834324+0000","last_became_active":"2024-04-25T08:52:31.237356+0000","last_became_peered":"2024-04-25T08:52:31.237356+0000","last_unstale":"2024-04-25T08:52:39.834324+0000","last_undegraded":"2024-04-25T08:52:39.834324+0000","last_fullsized":"2024-04-25T08:52:39.834324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833357+0000","last_change":"2024-04-25T08:52:36.008156+0000","last_active":"2024-04-25T08:52:39.833357+0000","last_peered":"2024-04-25T08:52:39.833357+0000","last_clean":"2024-04-25T08:52:39.833357+0000","last_became_active":"2024-04-25T08:52:36.006941+0000","last_became_peered":"2024-04-25T08:52:36.006941+0000","last_unstale":"2024-04-25T08:52:39.833357+0000","last_undegraded":"2024-04-25T08:52:39.833357+0000","last_fullsized":"2024-04-25T08:52:39.833357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:00.158176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834859+0000","last_change":"2024-04-25T08:52:30.225034+0000","last_active":"2024-04-25T08:52:39.834859+0000","last_peered":"2024-04-25T08:52:39.834859+0000","last_clean":"2024-04-25T08:52:39.834859+0000","last_became_active":"2024-04-25T08:52:30.224727+0000","last_became_peered":"2024-04-25T08:52:30.224727+0000","last_unstale":"2024-04-25T08:52:39.834859+0000","last_undegraded":"2024-04-25T08:52:39.834859+0000","last_fullsized":"2024-04-25T08:52:39.834859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836491+0000","last_change":"2024-04-25T08:52:31.256496+0000","last_active":"2024-04-25T08:52:39.836491+0000","last_peered":"2024-04-25T08:52:39.836491+0000","last_clean":"2024-04-25T08:52:39.836491+0000","last_became_active":"2024-04-25T08:52:31.255998+0000","last_became_peered":"2024-04-25T08:52:31.255998+0000","last_unstale":"2024-04-25T08:52:39.836491+0000","last_undegraded":"2024-04-25T08:52:39.836491+0000","last_fullsized":"2024-04-25T08:52:39.836491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299627+0000","last_change":"2024-04-25T08:52:35.324509+0000","last_active":"2024-04-25T08:52:38.299627+0000","last_peered":"2024-04-25T08:52:38.299627+0000","last_clean":"2024-04-25T08:52:38.299627+0000","last_became_active":"2024-04-25T08:52:35.323398+0000","last_became_peered":"2024-04-25T08:52:35.323398+0000","last_unstale":"2024-04-25T08:52:38.299627+0000","last_undegraded":"2024-04-25T08:52:38.299627+0000","last_fullsized":"2024-04-25T08:52:38.299627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:56.088772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307843+0000","last_change":"2024-04-25T08:52:30.248641+0000","last_active":"2024-04-25T08:52:39.307843+0000","last_peered":"2024-04-25T08:52:39.307843+0000","last_clean":"2024-04-25T08:52:39.307843+0000","last_became_active":"2024-04-25T08:52:30.245183+0000","last_became_peered":"2024-04-25T08:52:30.245183+0000","last_unstale":"2024-04-25T08:52:39.307843+0000","last_undegraded":"2024-04-25T08:52:39.307843+0000","last_fullsized":"2024-04-25T08:52:39.307843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:46.012895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644062+0000","last_change":"2024-04-25T08:52:31.272777+0000","last_active":"2024-04-25T08:52:39.644062+0000","last_peered":"2024-04-25T08:52:39.644062+0000","last_clean":"2024-04-25T08:52:39.644062+0000","last_became_active":"2024-04-25T08:52:31.271932+0000","last_became_peered":"2024-04-25T08:52:31.271932+0000","last_unstale":"2024-04-25T08:52:39.644062+0000","last_undegraded":"2024-04-25T08:52:39.644062+0000","last_fullsized":"2024-04-25T08:52:39.644062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300302+0000","last_change":"2024-04-25T08:52:35.330473+0000","last_active":"2024-04-25T08:52:38.300302+0000","last_peered":"2024-04-25T08:52:38.300302+0000","last_clean":"2024-04-25T08:52:38.300302+0000","last_became_active":"2024-04-25T08:52:35.328741+0000","last_became_peered":"2024-04-25T08:52:35.328741+0000","last_unstale":"2024-04-25T08:52:38.300302+0000","last_undegraded":"2024-04-25T08:52:38.300302+0000","last_fullsized":"2024-04-25T08:52:38.300302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:14.472581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307492+0000","last_change":"2024-04-25T08:52:30.245422+0000","last_active":"2024-04-25T08:52:39.307492+0000","last_peered":"2024-04-25T08:52:39.307492+0000","last_clean":"2024-04-25T08:52:39.307492+0000","last_became_active":"2024-04-25T08:52:30.244911+0000","last_became_peered":"2024-04-25T08:52:30.244911+0000","last_unstale":"2024-04-25T08:52:39.307492+0000","last_undegraded":"2024-04-25T08:52:39.307492+0000","last_fullsized":"2024-04-25T08:52:39.307492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644128+0000","last_change":"2024-04-25T08:52:31.274339+0000","last_active":"2024-04-25T08:52:39.644128+0000","last_peered":"2024-04-25T08:52:39.644128+0000","last_clean":"2024-04-25T08:52:39.644128+0000","last_became_active":"2024-04-25T08:52:31.273607+0000","last_became_peered":"2024-04-25T08:52:31.273607+0000","last_unstale":"2024-04-25T08:52:39.644128+0000","last_undegraded":"2024-04-25T08:52:39.644128+0000","last_fullsized":"2024-04-25T08:52:39.644128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:56.409407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299563+0000","last_change":"2024-04-25T08:52:35.316303+0000","last_active":"2024-04-25T08:52:38.299563+0000","last_peered":"2024-04-25T08:52:38.299563+0000","last_clean":"2024-04-25T08:52:38.299563+0000","last_became_active":"2024-04-25T08:52:35.315156+0000","last_became_peered":"2024-04-25T08:52:35.315156+0000","last_unstale":"2024-04-25T08:52:38.299563+0000","last_undegraded":"2024-04-25T08:52:38.299563+0000","last_fullsized":"2024-04-25T08:52:38.299563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:35.695987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300591+0000","last_change":"2024-04-25T08:52:31.269860+0000","last_active":"2024-04-25T08:52:38.300591+0000","last_peered":"2024-04-25T08:52:38.300591+0000","last_clean":"2024-04-25T08:52:38.300591+0000","last_became_active":"2024-04-25T08:52:31.268731+0000","last_became_peered":"2024-04-25T08:52:31.268731+0000","last_unstale":"2024-04-25T08:52:38.300591+0000","last_undegraded":"2024-04-25T08:52:38.300591+0000","last_fullsized":"2024-04-25T08:52:38.300591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307114+0000","last_change":"2024-04-25T08:52:30.246021+0000","last_active":"2024-04-25T08:52:39.307114+0000","last_peered":"2024-04-25T08:52:39.307114+0000","last_clean":"2024-04-25T08:52:39.307114+0000","last_became_active":"2024-04-25T08:52:30.245132+0000","last_became_peered":"2024-04-25T08:52:30.245132+0000","last_unstale":"2024-04-25T08:52:39.307114+0000","last_undegraded":"2024-04-25T08:52:39.307114+0000","last_fullsized":"2024-04-25T08:52:39.307114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:11.290953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301681+0000","last_change":"2024-04-25T08:52:35.323528+0000","last_active":"2024-04-25T08:52:38.301681+0000","last_peered":"2024-04-25T08:52:38.301681+0000","last_clean":"2024-04-25T08:52:38.301681+0000","last_became_active":"2024-04-25T08:52:35.320911+0000","last_became_peered":"2024-04-25T08:52:35.320911+0000","last_unstale":"2024-04-25T08:52:38.301681+0000","last_undegraded":"2024-04-25T08:52:38.301681+0000","last_fullsized":"2024-04-25T08:52:38.301681+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835879+0000","last_change":"2024-04-25T08:52:31.257885+0000","last_active":"2024-04-25T08:52:39.835879+0000","last_peered":"2024-04-25T08:52:39.835879+0000","last_clean":"2024-04-25T08:52:39.835879+0000","last_became_active":"2024-04-25T08:52:31.257583+0000","last_became_peered":"2024-04-25T08:52:31.257583+0000","last_unstale":"2024-04-25T08:52:39.835879+0000","last_undegraded":"2024-04-25T08:52:39.835879+0000","last_fullsized":"2024-04-25T08:52:39.835879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834075+0000","last_change":"2024-04-25T08:52:30.213612+0000","last_active":"2024-04-25T08:52:39.834075+0000","last_peered":"2024-04-25T08:52:39.834075+0000","last_clean":"2024-04-25T08:52:39.834075+0000","last_became_active":"2024-04-25T08:52:30.212962+0000","last_became_peered":"2024-04-25T08:52:30.212962+0000","last_unstale":"2024-04-25T08:52:39.834075+0000","last_undegraded":"2024-04-25T08:52:39.834075+0000","last_fullsized":"2024-04-25T08:52:39.834075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300390+0000","last_change":"2024-04-25T08:52:35.324597+0000","last_active":"2024-04-25T08:52:38.300390+0000","last_peered":"2024-04-25T08:52:38.300390+0000","last_clean":"2024-04-25T08:52:38.300390+0000","last_became_active":"2024-04-25T08:52:35.323534+0000","last_became_peered":"2024-04-25T08:52:35.323534+0000","last_unstale":"2024-04-25T08:52:38.300390+0000","last_undegraded":"2024-04-25T08:52:38.300390+0000","last_fullsized":"2024-04-25T08:52:38.300390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:24.182927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834456+0000","last_change":"2024-04-25T08:52:31.231520+0000","last_active":"2024-04-25T08:52:39.834456+0000","last_peered":"2024-04-25T08:52:39.834456+0000","last_clean":"2024-04-25T08:52:39.834456+0000","last_became_active":"2024-04-25T08:52:31.231009+0000","last_became_peered":"2024-04-25T08:52:31.231009+0000","last_unstale":"2024-04-25T08:52:39.834456+0000","last_undegraded":"2024-04-25T08:52:39.834456+0000","last_fullsized":"2024-04-25T08:52:39.834456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834462+0000","last_change":"2024-04-25T08:52:30.231226+0000","last_active":"2024-04-25T08:52:39.834462+0000","last_peered":"2024-04-25T08:52:39.834462+0000","last_clean":"2024-04-25T08:52:39.834462+0000","last_became_active":"2024-04-25T08:52:30.230790+0000","last_became_peered":"2024-04-25T08:52:30.230790+0000","last_unstale":"2024-04-25T08:52:39.834462+0000","last_undegraded":"2024-04-25T08:52:39.834462+0000","last_fullsized":"2024-04-25T08:52:39.834462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:11.290953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834915+0000","last_change":"2024-04-25T08:52:36.004737+0000","last_active":"2024-04-25T08:52:39.834915+0000","last_peered":"2024-04-25T08:52:39.834915+0000","last_clean":"2024-04-25T08:52:39.834915+0000","last_became_active":"2024-04-25T08:52:36.003639+0000","last_became_peered":"2024-04-25T08:52:36.003639+0000","last_unstale":"2024-04-25T08:52:39.834915+0000","last_undegraded":"2024-04-25T08:52:39.834915+0000","last_fullsized":"2024-04-25T08:52:39.834915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301223+0000","last_change":"2024-04-25T08:52:31.260426+0000","last_active":"2024-04-25T08:52:38.301223+0000","last_peered":"2024-04-25T08:52:38.301223+0000","last_clean":"2024-04-25T08:52:38.301223+0000","last_became_active":"2024-04-25T08:52:31.259594+0000","last_became_peered":"2024-04-25T08:52:31.259594+0000","last_unstale":"2024-04-25T08:52:38.301223+0000","last_undegraded":"2024-04-25T08:52:38.301223+0000","last_fullsized":"2024-04-25T08:52:38.301223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:45.962867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306464+0000","last_change":"2024-04-25T08:52:30.240896+0000","last_active":"2024-04-25T08:52:39.306464+0000","last_peered":"2024-04-25T08:52:39.306464+0000","last_clean":"2024-04-25T08:52:39.306464+0000","last_became_active":"2024-04-25T08:52:30.240290+0000","last_became_peered":"2024-04-25T08:52:30.240290+0000","last_unstale":"2024-04-25T08:52:39.306464+0000","last_undegraded":"2024-04-25T08:52:39.306464+0000","last_fullsized":"2024-04-25T08:52:39.306464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300735+0000","last_change":"2024-04-25T08:52:35.328692+0000","last_active":"2024-04-25T08:52:38.300735+0000","last_peered":"2024-04-25T08:52:38.300735+0000","last_clean":"2024-04-25T08:52:38.300735+0000","last_became_active":"2024-04-25T08:52:35.323899+0000","last_became_peered":"2024-04-25T08:52:35.323899+0000","last_unstale":"2024-04-25T08:52:38.300735+0000","last_undegraded":"2024-04-25T08:52:38.300735+0000","last_fullsized":"2024-04-25T08:52:38.300735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836941+0000","last_change":"2024-04-25T08:52:31.253016+0000","last_active":"2024-04-25T08:52:39.836941+0000","last_peered":"2024-04-25T08:52:39.836941+0000","last_clean":"2024-04-25T08:52:39.836941+0000","last_became_active":"2024-04-25T08:52:31.252369+0000","last_became_peered":"2024-04-25T08:52:31.252369+0000","last_unstale":"2024-04-25T08:52:39.836941+0000","last_undegraded":"2024-04-25T08:52:39.836941+0000","last_fullsized":"2024-04-25T08:52:39.836941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303292+0000","last_change":"2024-04-25T08:52:30.241679+0000","last_active":"2024-04-25T08:52:38.303292+0000","last_peered":"2024-04-25T08:52:38.303292+0000","last_clean":"2024-04-25T08:52:38.303292+0000","last_became_active":"2024-04-25T08:52:30.241063+0000","last_became_peered":"2024-04-25T08:52:30.241063+0000","last_unstale":"2024-04-25T08:52:38.303292+0000","last_undegraded":"2024-04-25T08:52:38.303292+0000","last_fullsized":"2024-04-25T08:52:38.303292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300540+0000","last_change":"2024-04-25T08:52:36.007015+0000","last_active":"2024-04-25T08:52:38.300540+0000","last_peered":"2024-04-25T08:52:38.300540+0000","last_clean":"2024-04-25T08:52:38.300540+0000","last_became_active":"2024-04-25T08:52:36.005832+0000","last_became_peered":"2024-04-25T08:52:36.005832+0000","last_unstale":"2024-04-25T08:52:38.300540+0000","last_undegraded":"2024-04-25T08:52:38.300540+0000","last_fullsized":"2024-04-25T08:52:38.300540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:19.422348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834979+0000","last_change":"2024-04-25T08:52:31.282175+0000","last_active":"2024-04-25T08:52:39.834979+0000","last_peered":"2024-04-25T08:52:39.834979+0000","last_clean":"2024-04-25T08:52:39.834979+0000","last_became_active":"2024-04-25T08:52:31.281955+0000","last_became_peered":"2024-04-25T08:52:31.281955+0000","last_unstale":"2024-04-25T08:52:39.834979+0000","last_undegraded":"2024-04-25T08:52:39.834979+0000","last_fullsized":"2024-04-25T08:52:39.834979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834733+0000","last_change":"2024-04-25T08:52:30.215220+0000","last_active":"2024-04-25T08:52:39.834733+0000","last_peered":"2024-04-25T08:52:39.834733+0000","last_clean":"2024-04-25T08:52:39.834733+0000","last_became_active":"2024-04-25T08:52:30.214773+0000","last_became_peered":"2024-04-25T08:52:30.214773+0000","last_unstale":"2024-04-25T08:52:39.834733+0000","last_undegraded":"2024-04-25T08:52:39.834733+0000","last_fullsized":"2024-04-25T08:52:39.834733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833949+0000","last_change":"2024-04-25T08:52:36.003489+0000","last_active":"2024-04-25T08:52:39.833949+0000","last_peered":"2024-04-25T08:52:39.833949+0000","last_clean":"2024-04-25T08:52:39.833949+0000","last_became_active":"2024-04-25T08:52:36.002295+0000","last_became_peered":"2024-04-25T08:52:36.002295+0000","last_unstale":"2024-04-25T08:52:39.833949+0000","last_undegraded":"2024-04-25T08:52:39.833949+0000","last_fullsized":"2024-04-25T08:52:39.833949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:47.463029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833050+0000","last_change":"2024-04-25T08:52:31.226718+0000","last_active":"2024-04-25T08:52:39.833050+0000","last_peered":"2024-04-25T08:52:39.833050+0000","last_clean":"2024-04-25T08:52:39.833050+0000","last_became_active":"2024-04-25T08:52:31.226442+0000","last_became_peered":"2024-04-25T08:52:31.226442+0000","last_unstale":"2024-04-25T08:52:39.833050+0000","last_undegraded":"2024-04-25T08:52:39.833050+0000","last_fullsized":"2024-04-25T08:52:39.833050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307955+0000","last_change":"2024-04-25T08:52:30.249239+0000","last_active":"2024-04-25T08:52:39.307955+0000","last_peered":"2024-04-25T08:52:39.307955+0000","last_clean":"2024-04-25T08:52:39.307955+0000","last_became_active":"2024-04-25T08:52:30.248612+0000","last_became_peered":"2024-04-25T08:52:30.248612+0000","last_unstale":"2024-04-25T08:52:39.307955+0000","last_undegraded":"2024-04-25T08:52:39.307955+0000","last_fullsized":"2024-04-25T08:52:39.307955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:34.198893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836268+0000","last_change":"2024-04-25T08:52:36.001292+0000","last_active":"2024-04-25T08:52:39.836268+0000","last_peered":"2024-04-25T08:52:39.836268+0000","last_clean":"2024-04-25T08:52:39.836268+0000","last_became_active":"2024-04-25T08:52:36.000113+0000","last_became_peered":"2024-04-25T08:52:36.000113+0000","last_unstale":"2024-04-25T08:52:39.836268+0000","last_undegraded":"2024-04-25T08:52:39.836268+0000","last_fullsized":"2024-04-25T08:52:39.836268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645594+0000","last_change":"2024-04-25T08:52:31.247003+0000","last_active":"2024-04-25T08:52:39.645594+0000","last_peered":"2024-04-25T08:52:39.645594+0000","last_clean":"2024-04-25T08:52:39.645594+0000","last_became_active":"2024-04-25T08:52:31.246200+0000","last_became_peered":"2024-04-25T08:52:31.246200+0000","last_unstale":"2024-04-25T08:52:39.645594+0000","last_undegraded":"2024-04-25T08:52:39.645594+0000","last_fullsized":"2024-04-25T08:52:39.645594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:52.007683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835389+0000","last_change":"2024-04-25T08:52:30.215091+0000","last_active":"2024-04-25T08:52:39.835389+0000","last_peered":"2024-04-25T08:52:39.835389+0000","last_clean":"2024-04-25T08:52:39.835389+0000","last_became_active":"2024-04-25T08:52:30.214664+0000","last_became_peered":"2024-04-25T08:52:30.214664+0000","last_unstale":"2024-04-25T08:52:39.835389+0000","last_undegraded":"2024-04-25T08:52:39.835389+0000","last_fullsized":"2024-04-25T08:52:39.835389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833696+0000","last_change":"2024-04-25T08:52:35.313874+0000","last_active":"2024-04-25T08:52:39.833696+0000","last_peered":"2024-04-25T08:52:39.833696+0000","last_clean":"2024-04-25T08:52:39.833696+0000","last_became_active":"2024-04-25T08:52:35.313429+0000","last_became_peered":"2024-04-25T08:52:35.313429+0000","last_unstale":"2024-04-25T08:52:39.833696+0000","last_undegraded":"2024-04-25T08:52:39.833696+0000","last_fullsized":"2024-04-25T08:52:39.833696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:59.165277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300917+0000","last_change":"2024-04-25T08:52:31.257574+0000","last_active":"2024-04-25T08:52:38.300917+0000","last_peered":"2024-04-25T08:52:38.300917+0000","last_clean":"2024-04-25T08:52:38.300917+0000","last_became_active":"2024-04-25T08:52:31.257040+0000","last_became_peered":"2024-04-25T08:52:31.257040+0000","last_unstale":"2024-04-25T08:52:38.300917+0000","last_undegraded":"2024-04-25T08:52:38.300917+0000","last_fullsized":"2024-04-25T08:52:38.300917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:52.221264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645682+0000","last_change":"2024-04-25T08:52:30.225801+0000","last_active":"2024-04-25T08:52:39.645682+0000","last_peered":"2024-04-25T08:52:39.645682+0000","last_clean":"2024-04-25T08:52:39.645682+0000","last_became_active":"2024-04-25T08:52:30.224857+0000","last_became_peered":"2024-04-25T08:52:30.224857+0000","last_unstale":"2024-04-25T08:52:39.645682+0000","last_undegraded":"2024-04-25T08:52:39.645682+0000","last_fullsized":"2024-04-25T08:52:39.645682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833379+0000","last_change":"2024-04-25T08:52:35.309762+0000","last_active":"2024-04-25T08:52:39.833379+0000","last_peered":"2024-04-25T08:52:39.833379+0000","last_clean":"2024-04-25T08:52:39.833379+0000","last_became_active":"2024-04-25T08:52:35.309315+0000","last_became_peered":"2024-04-25T08:52:35.309315+0000","last_unstale":"2024-04-25T08:52:39.833379+0000","last_undegraded":"2024-04-25T08:52:39.833379+0000","last_fullsized":"2024-04-25T08:52:39.833379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.832782+0000","last_change":"2024-04-25T08:52:31.232199+0000","last_active":"2024-04-25T08:52:39.832782+0000","last_peered":"2024-04-25T08:52:39.832782+0000","last_clean":"2024-04-25T08:52:39.832782+0000","last_became_active":"2024-04-25T08:52:31.231928+0000","last_became_peered":"2024-04-25T08:52:31.231928+0000","last_unstale":"2024-04-25T08:52:39.832782+0000","last_undegraded":"2024-04-25T08:52:39.832782+0000","last_fullsized":"2024-04-25T08:52:39.832782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299166+0000","last_change":"2024-04-25T08:52:30.218615+0000","last_active":"2024-04-25T08:52:38.299166+0000","last_peered":"2024-04-25T08:52:38.299166+0000","last_clean":"2024-04-25T08:52:38.299166+0000","last_became_active":"2024-04-25T08:52:30.213378+0000","last_became_peered":"2024-04-25T08:52:30.213378+0000","last_unstale":"2024-04-25T08:52:38.299166+0000","last_undegraded":"2024-04-25T08:52:38.299166+0000","last_fullsized":"2024-04-25T08:52:38.299166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834271+0000","last_change":"2024-04-25T08:52:35.311156+0000","last_active":"2024-04-25T08:52:39.834271+0000","last_peered":"2024-04-25T08:52:39.834271+0000","last_clean":"2024-04-25T08:52:39.834271+0000","last_became_active":"2024-04-25T08:52:35.310700+0000","last_became_peered":"2024-04-25T08:52:35.310700+0000","last_unstale":"2024-04-25T08:52:39.834271+0000","last_undegraded":"2024-04-25T08:52:39.834271+0000","last_fullsized":"2024-04-25T08:52:39.834271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.832874+0000","last_change":"2024-04-25T08:52:31.260514+0000","last_active":"2024-04-25T08:52:39.832874+0000","last_peered":"2024-04-25T08:52:39.832874+0000","last_clean":"2024-04-25T08:52:39.832874+0000","last_became_active":"2024-04-25T08:52:31.260019+0000","last_became_peered":"2024-04-25T08:52:31.260019+0000","last_unstale":"2024-04-25T08:52:39.832874+0000","last_undegraded":"2024-04-25T08:52:39.832874+0000","last_fullsized":"2024-04-25T08:52:39.832874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.602114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306466+0000","last_change":"2024-04-25T08:52:30.245223+0000","last_active":"2024-04-25T08:52:39.306466+0000","last_peered":"2024-04-25T08:52:39.306466+0000","last_clean":"2024-04-25T08:52:39.306466+0000","last_became_active":"2024-04-25T08:52:30.244782+0000","last_became_peered":"2024-04-25T08:52:30.244782+0000","last_unstale":"2024-04-25T08:52:39.306466+0000","last_undegraded":"2024-04-25T08:52:39.306466+0000","last_fullsized":"2024-04-25T08:52:39.306466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836419+0000","last_change":"2024-04-25T08:52:36.002202+0000","last_active":"2024-04-25T08:52:39.836419+0000","last_peered":"2024-04-25T08:52:39.836419+0000","last_clean":"2024-04-25T08:52:39.836419+0000","last_became_active":"2024-04-25T08:52:36.000348+0000","last_became_peered":"2024-04-25T08:52:36.000348+0000","last_unstale":"2024-04-25T08:52:39.836419+0000","last_undegraded":"2024-04-25T08:52:39.836419+0000","last_fullsized":"2024-04-25T08:52:39.836419+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:12.373033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834845+0000","last_change":"2024-04-25T08:52:31.257777+0000","last_active":"2024-04-25T08:52:39.834845+0000","last_peered":"2024-04-25T08:52:39.834845+0000","last_clean":"2024-04-25T08:52:39.834845+0000","last_became_active":"2024-04-25T08:52:31.257295+0000","last_became_peered":"2024-04-25T08:52:31.257295+0000","last_unstale":"2024-04-25T08:52:39.834845+0000","last_undegraded":"2024-04-25T08:52:39.834845+0000","last_fullsized":"2024-04-25T08:52:39.834845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645136+0000","last_change":"2024-04-25T08:52:30.222595+0000","last_active":"2024-04-25T08:52:39.645136+0000","last_peered":"2024-04-25T08:52:39.645136+0000","last_clean":"2024-04-25T08:52:39.645136+0000","last_became_active":"2024-04-25T08:52:30.217788+0000","last_became_peered":"2024-04-25T08:52:30.217788+0000","last_unstale":"2024-04-25T08:52:39.645136+0000","last_undegraded":"2024-04-25T08:52:39.645136+0000","last_fullsized":"2024-04-25T08:52:39.645136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:52.505423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306756+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:52:39.306756+0000","last_peered":"2024-04-25T08:52:39.306756+0000","last_clean":"2024-04-25T08:52:39.306756+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:52:39.306756+0000","last_undegraded":"2024-04-25T08:52:39.306756+0000","last_fullsized":"2024-04-25T08:52:39.306756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.302216+0000","last_change":"2024-04-25T08:52:36.004910+0000","last_active":"2024-04-25T08:52:38.302216+0000","last_peered":"2024-04-25T08:52:38.302216+0000","last_clean":"2024-04-25T08:52:38.302216+0000","last_became_active":"2024-04-25T08:52:36.003878+0000","last_became_peered":"2024-04-25T08:52:36.003878+0000","last_unstale":"2024-04-25T08:52:38.302216+0000","last_undegraded":"2024-04-25T08:52:38.302216+0000","last_fullsized":"2024-04-25T08:52:38.302216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:12.106880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307624+0000","last_change":"2024-04-25T08:52:31.286252+0000","last_active":"2024-04-25T08:52:39.307624+0000","last_peered":"2024-04-25T08:52:39.307624+0000","last_clean":"2024-04-25T08:52:39.307624+0000","last_became_active":"2024-04-25T08:52:31.282622+0000","last_became_peered":"2024-04-25T08:52:31.282622+0000","last_unstale":"2024-04-25T08:52:39.307624+0000","last_undegraded":"2024-04-25T08:52:39.307624+0000","last_fullsized":"2024-04-25T08:52:39.307624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299865+0000","last_change":"2024-04-25T08:52:30.226514+0000","last_active":"2024-04-25T08:52:38.299865+0000","last_peered":"2024-04-25T08:52:38.299865+0000","last_clean":"2024-04-25T08:52:38.299865+0000","last_became_active":"2024-04-25T08:52:30.226243+0000","last_became_peered":"2024-04-25T08:52:30.226243+0000","last_unstale":"2024-04-25T08:52:38.299865+0000","last_undegraded":"2024-04-25T08:52:38.299865+0000","last_fullsized":"2024-04-25T08:52:38.299865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300450+0000","last_change":"2024-04-25T08:52:36.003056+0000","last_active":"2024-04-25T08:52:38.300450+0000","last_peered":"2024-04-25T08:52:38.300450+0000","last_clean":"2024-04-25T08:52:38.300450+0000","last_became_active":"2024-04-25T08:52:36.000882+0000","last_became_peered":"2024-04-25T08:52:36.000882+0000","last_unstale":"2024-04-25T08:52:38.300450+0000","last_undegraded":"2024-04-25T08:52:38.300450+0000","last_fullsized":"2024-04-25T08:52:38.300450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:56.088772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645689+0000","last_change":"2024-04-25T08:52:31.244866+0000","last_active":"2024-04-25T08:52:39.645689+0000","last_peered":"2024-04-25T08:52:39.645689+0000","last_clean":"2024-04-25T08:52:39.645689+0000","last_became_active":"2024-04-25T08:52:31.244249+0000","last_became_peered":"2024-04-25T08:52:31.244249+0000","last_unstale":"2024-04-25T08:52:39.645689+0000","last_undegraded":"2024-04-25T08:52:39.645689+0000","last_fullsized":"2024-04-25T08:52:39.645689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835210+0000","last_change":"2024-04-25T08:52:30.228380+0000","last_active":"2024-04-25T08:52:39.835210+0000","last_peered":"2024-04-25T08:52:39.835210+0000","last_clean":"2024-04-25T08:52:39.835210+0000","last_became_active":"2024-04-25T08:52:30.227732+0000","last_became_peered":"2024-04-25T08:52:30.227732+0000","last_unstale":"2024-04-25T08:52:39.835210+0000","last_undegraded":"2024-04-25T08:52:39.835210+0000","last_fullsized":"2024-04-25T08:52:39.835210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:46.012895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836520+0000","last_change":"2024-04-25T08:52:35.316462+0000","last_active":"2024-04-25T08:52:39.836520+0000","last_peered":"2024-04-25T08:52:39.836520+0000","last_clean":"2024-04-25T08:52:39.836520+0000","last_became_active":"2024-04-25T08:52:35.315760+0000","last_became_peered":"2024-04-25T08:52:35.315760+0000","last_unstale":"2024-04-25T08:52:39.836520+0000","last_undegraded":"2024-04-25T08:52:39.836520+0000","last_fullsized":"2024-04-25T08:52:39.836520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:49.973872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305761+0000","last_change":"2024-04-25T08:52:31.289534+0000","last_active":"2024-04-25T08:52:39.305761+0000","last_peered":"2024-04-25T08:52:39.305761+0000","last_clean":"2024-04-25T08:52:39.305761+0000","last_became_active":"2024-04-25T08:52:31.288580+0000","last_became_peered":"2024-04-25T08:52:31.288580+0000","last_unstale":"2024-04-25T08:52:39.305761+0000","last_undegraded":"2024-04-25T08:52:39.305761+0000","last_fullsized":"2024-04-25T08:52:39.305761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:10.391492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299950+0000","last_change":"2024-04-25T08:52:30.215006+0000","last_active":"2024-04-25T08:52:38.299950+0000","last_peered":"2024-04-25T08:52:38.299950+0000","last_clean":"2024-04-25T08:52:38.299950+0000","last_became_active":"2024-04-25T08:52:30.213814+0000","last_became_peered":"2024-04-25T08:52:30.213814+0000","last_unstale":"2024-04-25T08:52:38.299950+0000","last_undegraded":"2024-04-25T08:52:38.299950+0000","last_fullsized":"2024-04-25T08:52:38.299950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644315+0000","last_change":"2024-04-25T08:52:36.005410+0000","last_active":"2024-04-25T08:52:39.644315+0000","last_peered":"2024-04-25T08:52:39.644315+0000","last_clean":"2024-04-25T08:52:39.644315+0000","last_became_active":"2024-04-25T08:52:36.004309+0000","last_became_peered":"2024-04-25T08:52:36.004309+0000","last_unstale":"2024-04-25T08:52:39.644315+0000","last_undegraded":"2024-04-25T08:52:39.644315+0000","last_fullsized":"2024-04-25T08:52:39.644315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305320+0000","last_change":"2024-04-25T08:52:31.285984+0000","last_active":"2024-04-25T08:52:39.305320+0000","last_peered":"2024-04-25T08:52:39.305320+0000","last_clean":"2024-04-25T08:52:39.305320+0000","last_became_active":"2024-04-25T08:52:31.281988+0000","last_became_peered":"2024-04-25T08:52:31.281988+0000","last_unstale":"2024-04-25T08:52:39.305320+0000","last_undegraded":"2024-04-25T08:52:39.305320+0000","last_fullsized":"2024-04-25T08:52:39.305320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:56.409407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301053+0000","last_change":"2024-04-25T08:52:30.215598+0000","last_active":"2024-04-25T08:52:38.301053+0000","last_peered":"2024-04-25T08:52:38.301053+0000","last_clean":"2024-04-25T08:52:38.301053+0000","last_became_active":"2024-04-25T08:52:30.214765+0000","last_became_peered":"2024-04-25T08:52:30.214765+0000","last_unstale":"2024-04-25T08:52:38.301053+0000","last_undegraded":"2024-04-25T08:52:38.301053+0000","last_fullsized":"2024-04-25T08:52:38.301053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:36.565944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300087+0000","last_change":"2024-04-25T08:52:35.328574+0000","last_active":"2024-04-25T08:52:38.300087+0000","last_peered":"2024-04-25T08:52:38.300087+0000","last_clean":"2024-04-25T08:52:38.300087+0000","last_became_active":"2024-04-25T08:52:35.323891+0000","last_became_peered":"2024-04-25T08:52:35.323891+0000","last_unstale":"2024-04-25T08:52:38.300087+0000","last_undegraded":"2024-04-25T08:52:38.300087+0000","last_fullsized":"2024-04-25T08:52:38.300087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:12.106880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305125+0000","last_change":"2024-04-25T08:52:31.278037+0000","last_active":"2024-04-25T08:52:39.305125+0000","last_peered":"2024-04-25T08:52:39.305125+0000","last_clean":"2024-04-25T08:52:39.305125+0000","last_became_active":"2024-04-25T08:52:31.277117+0000","last_became_peered":"2024-04-25T08:52:31.277117+0000","last_unstale":"2024-04-25T08:52:39.305125+0000","last_undegraded":"2024-04-25T08:52:39.305125+0000","last_fullsized":"2024-04-25T08:52:39.305125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298797+0000","last_change":"2024-04-25T08:52:30.221811+0000","last_active":"2024-04-25T08:52:38.298797+0000","last_peered":"2024-04-25T08:52:38.298797+0000","last_clean":"2024-04-25T08:52:38.298797+0000","last_became_active":"2024-04-25T08:52:30.220494+0000","last_became_peered":"2024-04-25T08:52:30.220494+0000","last_unstale":"2024-04-25T08:52:38.298797+0000","last_undegraded":"2024-04-25T08:52:38.298797+0000","last_fullsized":"2024-04-25T08:52:38.298797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836281+0000","last_change":"2024-04-25T08:52:35.317037+0000","last_active":"2024-04-25T08:52:39.836281+0000","last_peered":"2024-04-25T08:52:39.836281+0000","last_clean":"2024-04-25T08:52:39.836281+0000","last_became_active":"2024-04-25T08:52:35.316491+0000","last_became_peered":"2024-04-25T08:52:35.316491+0000","last_unstale":"2024-04-25T08:52:39.836281+0000","last_undegraded":"2024-04-25T08:52:39.836281+0000","last_fullsized":"2024-04-25T08:52:39.836281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299015+0000","last_change":"2024-04-25T08:52:31.256415+0000","last_active":"2024-04-25T08:52:38.299015+0000","last_peered":"2024-04-25T08:52:38.299015+0000","last_clean":"2024-04-25T08:52:38.299015+0000","last_became_active":"2024-04-25T08:52:31.254747+0000","last_became_peered":"2024-04-25T08:52:31.254747+0000","last_unstale":"2024-04-25T08:52:38.299015+0000","last_undegraded":"2024-04-25T08:52:38.299015+0000","last_fullsized":"2024-04-25T08:52:38.299015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:24.567213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299991+0000","last_change":"2024-04-25T08:52:30.223725+0000","last_active":"2024-04-25T08:52:38.299991+0000","last_peered":"2024-04-25T08:52:38.299991+0000","last_clean":"2024-04-25T08:52:38.299991+0000","last_became_active":"2024-04-25T08:52:30.222820+0000","last_became_peered":"2024-04-25T08:52:30.222820+0000","last_unstale":"2024-04-25T08:52:38.299991+0000","last_undegraded":"2024-04-25T08:52:38.299991+0000","last_fullsized":"2024-04-25T08:52:38.299991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834851+0000","last_change":"2024-04-25T08:52:36.003180+0000","last_active":"2024-04-25T08:52:39.834851+0000","last_peered":"2024-04-25T08:52:39.834851+0000","last_clean":"2024-04-25T08:52:39.834851+0000","last_became_active":"2024-04-25T08:52:36.002030+0000","last_became_peered":"2024-04-25T08:52:36.002030+0000","last_unstale":"2024-04-25T08:52:39.834851+0000","last_undegraded":"2024-04-25T08:52:39.834851+0000","last_fullsized":"2024-04-25T08:52:39.834851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:58.721596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305176+0000","last_change":"2024-04-25T08:52:31.278626+0000","last_active":"2024-04-25T08:52:39.305176+0000","last_peered":"2024-04-25T08:52:39.305176+0000","last_clean":"2024-04-25T08:52:39.305176+0000","last_became_active":"2024-04-25T08:52:31.278027+0000","last_became_peered":"2024-04-25T08:52:31.278027+0000","last_unstale":"2024-04-25T08:52:39.305176+0000","last_undegraded":"2024-04-25T08:52:39.305176+0000","last_fullsized":"2024-04-25T08:52:39.305176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:15.341259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645485+0000","last_change":"2024-04-25T08:52:30.233250+0000","last_active":"2024-04-25T08:52:39.645485+0000","last_peered":"2024-04-25T08:52:39.645485+0000","last_clean":"2024-04-25T08:52:39.645485+0000","last_became_active":"2024-04-25T08:52:30.229813+0000","last_became_peered":"2024-04-25T08:52:30.229813+0000","last_unstale":"2024-04-25T08:52:39.645485+0000","last_undegraded":"2024-04-25T08:52:39.645485+0000","last_fullsized":"2024-04-25T08:52:39.645485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835406+0000","last_change":"2024-04-25T08:52:36.001662+0000","last_active":"2024-04-25T08:52:39.835406+0000","last_peered":"2024-04-25T08:52:39.835406+0000","last_clean":"2024-04-25T08:52:39.835406+0000","last_became_active":"2024-04-25T08:52:36.000433+0000","last_became_peered":"2024-04-25T08:52:36.000433+0000","last_unstale":"2024-04-25T08:52:39.835406+0000","last_undegraded":"2024-04-25T08:52:39.835406+0000","last_fullsized":"2024-04-25T08:52:39.835406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:13.578788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836042+0000","last_change":"2024-04-25T08:52:31.278698+0000","last_active":"2024-04-25T08:52:39.836042+0000","last_peered":"2024-04-25T08:52:39.836042+0000","last_clean":"2024-04-25T08:52:39.836042+0000","last_became_active":"2024-04-25T08:52:31.278388+0000","last_became_peered":"2024-04-25T08:52:31.278388+0000","last_unstale":"2024-04-25T08:52:39.836042+0000","last_undegraded":"2024-04-25T08:52:39.836042+0000","last_fullsized":"2024-04-25T08:52:39.836042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:24.567213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836775+0000","last_change":"2024-04-25T08:52:30.197089+0000","last_active":"2024-04-25T08:52:39.836775+0000","last_peered":"2024-04-25T08:52:39.836775+0000","last_clean":"2024-04-25T08:52:39.836775+0000","last_became_active":"2024-04-25T08:52:30.196592+0000","last_became_peered":"2024-04-25T08:52:30.196592+0000","last_unstale":"2024-04-25T08:52:39.836775+0000","last_undegraded":"2024-04-25T08:52:39.836775+0000","last_fullsized":"2024-04-25T08:52:39.836775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645712+0000","last_change":"2024-04-25T08:52:36.005556+0000","last_active":"2024-04-25T08:52:39.645712+0000","last_peered":"2024-04-25T08:52:39.645712+0000","last_clean":"2024-04-25T08:52:39.645712+0000","last_became_active":"2024-04-25T08:52:36.004511+0000","last_became_peered":"2024-04-25T08:52:36.004511+0000","last_unstale":"2024-04-25T08:52:39.645712+0000","last_undegraded":"2024-04-25T08:52:39.645712+0000","last_fullsized":"2024-04-25T08:52:39.645712+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307456+0000","last_change":"2024-04-25T08:52:31.286270+0000","last_active":"2024-04-25T08:52:39.307456+0000","last_peered":"2024-04-25T08:52:39.307456+0000","last_clean":"2024-04-25T08:52:39.307456+0000","last_became_active":"2024-04-25T08:52:31.281689+0000","last_became_peered":"2024-04-25T08:52:31.281689+0000","last_unstale":"2024-04-25T08:52:39.307456+0000","last_undegraded":"2024-04-25T08:52:39.307456+0000","last_fullsized":"2024-04-25T08:52:39.307456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:41.689859+0000","last_change":"2024-04-25T08:52:30.215322+0000","last_active":"2024-04-25T08:52:41.689859+0000","last_peered":"2024-04-25T08:52:41.689859+0000","last_clean":"2024-04-25T08:52:41.689859+0000","last_became_active":"2024-04-25T08:52:30.214531+0000","last_became_peered":"2024-04-25T08:52:30.214531+0000","last_unstale":"2024-04-25T08:52:41.689859+0000","last_undegraded":"2024-04-25T08:52:41.689859+0000","last_fullsized":"2024-04-25T08:52:41.689859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833881+0000","last_change":"2024-04-25T08:52:35.307400+0000","last_active":"2024-04-25T08:52:39.833881+0000","last_peered":"2024-04-25T08:52:39.833881+0000","last_clean":"2024-04-25T08:52:39.833881+0000","last_became_active":"2024-04-25T08:52:35.304579+0000","last_became_peered":"2024-04-25T08:52:35.304579+0000","last_unstale":"2024-04-25T08:52:39.833881+0000","last_undegraded":"2024-04-25T08:52:39.833881+0000","last_fullsized":"2024-04-25T08:52:39.833881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:24.182927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301767+0000","last_change":"2024-04-25T08:52:31.262376+0000","last_active":"2024-04-25T08:52:38.301767+0000","last_peered":"2024-04-25T08:52:38.301767+0000","last_clean":"2024-04-25T08:52:38.301767+0000","last_became_active":"2024-04-25T08:52:31.261642+0000","last_became_peered":"2024-04-25T08:52:31.261642+0000","last_unstale":"2024-04-25T08:52:38.301767+0000","last_undegraded":"2024-04-25T08:52:38.301767+0000","last_fullsized":"2024-04-25T08:52:38.301767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:12.300628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833373+0000","last_change":"2024-04-25T08:52:30.219030+0000","last_active":"2024-04-25T08:52:39.833373+0000","last_peered":"2024-04-25T08:52:39.833373+0000","last_clean":"2024-04-25T08:52:39.833373+0000","last_became_active":"2024-04-25T08:52:30.218788+0000","last_became_peered":"2024-04-25T08:52:30.218788+0000","last_unstale":"2024-04-25T08:52:39.833373+0000","last_undegraded":"2024-04-25T08:52:39.833373+0000","last_fullsized":"2024-04-25T08:52:39.833373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:34.198893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300753+0000","last_change":"2024-04-25T08:52:35.331866+0000","last_active":"2024-04-25T08:52:38.300753+0000","last_peered":"2024-04-25T08:52:38.300753+0000","last_clean":"2024-04-25T08:52:38.300753+0000","last_became_active":"2024-04-25T08:52:35.330775+0000","last_became_peered":"2024-04-25T08:52:35.330775+0000","last_unstale":"2024-04-25T08:52:38.300753+0000","last_undegraded":"2024-04-25T08:52:38.300753+0000","last_fullsized":"2024-04-25T08:52:38.300753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:59.165277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836632+0000","last_change":"2024-04-25T08:52:31.256646+0000","last_active":"2024-04-25T08:52:39.836632+0000","last_peered":"2024-04-25T08:52:39.836632+0000","last_clean":"2024-04-25T08:52:39.836632+0000","last_became_active":"2024-04-25T08:52:31.256362+0000","last_became_peered":"2024-04-25T08:52:31.256362+0000","last_unstale":"2024-04-25T08:52:39.836632+0000","last_undegraded":"2024-04-25T08:52:39.836632+0000","last_fullsized":"2024-04-25T08:52:39.836632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645734+0000","last_change":"2024-04-25T08:52:30.217888+0000","last_active":"2024-04-25T08:52:39.645734+0000","last_peered":"2024-04-25T08:52:39.645734+0000","last_clean":"2024-04-25T08:52:39.645734+0000","last_became_active":"2024-04-25T08:52:30.217100+0000","last_became_peered":"2024-04-25T08:52:30.217100+0000","last_unstale":"2024-04-25T08:52:39.645734+0000","last_undegraded":"2024-04-25T08:52:39.645734+0000","last_fullsized":"2024-04-25T08:52:39.645734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834254+0000","last_change":"2024-04-25T08:52:35.313671+0000","last_active":"2024-04-25T08:52:39.834254+0000","last_peered":"2024-04-25T08:52:39.834254+0000","last_clean":"2024-04-25T08:52:39.834254+0000","last_became_active":"2024-04-25T08:52:35.312785+0000","last_became_peered":"2024-04-25T08:52:35.312785+0000","last_unstale":"2024-04-25T08:52:39.834254+0000","last_undegraded":"2024-04-25T08:52:39.834254+0000","last_fullsized":"2024-04-25T08:52:39.834254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:23.828143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645723+0000","last_change":"2024-04-25T08:52:31.273223+0000","last_active":"2024-04-25T08:52:39.645723+0000","last_peered":"2024-04-25T08:52:39.645723+0000","last_clean":"2024-04-25T08:52:39.645723+0000","last_became_active":"2024-04-25T08:52:31.272627+0000","last_became_peered":"2024-04-25T08:52:31.272627+0000","last_unstale":"2024-04-25T08:52:39.645723+0000","last_undegraded":"2024-04-25T08:52:39.645723+0000","last_fullsized":"2024-04-25T08:52:39.645723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303022+0000","last_change":"2024-04-25T08:52:30.229619+0000","last_active":"2024-04-25T08:52:38.303022+0000","last_peered":"2024-04-25T08:52:38.303022+0000","last_clean":"2024-04-25T08:52:38.303022+0000","last_became_active":"2024-04-25T08:52:30.228770+0000","last_became_peered":"2024-04-25T08:52:30.228770+0000","last_unstale":"2024-04-25T08:52:38.303022+0000","last_undegraded":"2024-04-25T08:52:38.303022+0000","last_fullsized":"2024-04-25T08:52:38.303022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833502+0000","last_change":"2024-04-25T08:52:36.008479+0000","last_active":"2024-04-25T08:52:39.833502+0000","last_peered":"2024-04-25T08:52:39.833502+0000","last_clean":"2024-04-25T08:52:39.833502+0000","last_became_active":"2024-04-25T08:52:36.007404+0000","last_became_peered":"2024-04-25T08:52:36.007404+0000","last_unstale":"2024-04-25T08:52:39.833502+0000","last_undegraded":"2024-04-25T08:52:39.833502+0000","last_fullsized":"2024-04-25T08:52:39.833502+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:54.631138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836260+0000","last_change":"2024-04-25T08:52:36.004959+0000","last_active":"2024-04-25T08:52:39.836260+0000","last_peered":"2024-04-25T08:52:39.836260+0000","last_clean":"2024-04-25T08:52:39.836260+0000","last_became_active":"2024-04-25T08:52:36.003575+0000","last_became_peered":"2024-04-25T08:52:36.003575+0000","last_unstale":"2024-04-25T08:52:39.836260+0000","last_undegraded":"2024-04-25T08:52:39.836260+0000","last_fullsized":"2024-04-25T08:52:39.836260+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834357+0000","last_change":"2024-04-25T08:52:31.231230+0000","last_active":"2024-04-25T08:52:39.834357+0000","last_peered":"2024-04-25T08:52:39.834357+0000","last_clean":"2024-04-25T08:52:39.834357+0000","last_became_active":"2024-04-25T08:52:31.230951+0000","last_became_peered":"2024-04-25T08:52:31.230951+0000","last_unstale":"2024-04-25T08:52:39.834357+0000","last_undegraded":"2024-04-25T08:52:39.834357+0000","last_fullsized":"2024-04-25T08:52:39.834357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644784+0000","last_change":"2024-04-25T08:52:30.233534+0000","last_active":"2024-04-25T08:52:39.644784+0000","last_peered":"2024-04-25T08:52:39.644784+0000","last_clean":"2024-04-25T08:52:39.644784+0000","last_became_active":"2024-04-25T08:52:30.232832+0000","last_became_peered":"2024-04-25T08:52:30.232832+0000","last_unstale":"2024-04-25T08:52:39.644784+0000","last_undegraded":"2024-04-25T08:52:39.644784+0000","last_fullsized":"2024-04-25T08:52:39.644784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303102+0000","last_change":"2024-04-25T08:52:35.317543+0000","last_active":"2024-04-25T08:52:38.303102+0000","last_peered":"2024-04-25T08:52:38.303102+0000","last_clean":"2024-04-25T08:52:38.303102+0000","last_became_active":"2024-04-25T08:52:35.316474+0000","last_became_peered":"2024-04-25T08:52:35.316474+0000","last_unstale":"2024-04-25T08:52:38.303102+0000","last_undegraded":"2024-04-25T08:52:38.303102+0000","last_fullsized":"2024-04-25T08:52:38.303102+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:35.695987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834357+0000","last_change":"2024-04-25T08:52:31.257812+0000","last_active":"2024-04-25T08:52:39.834357+0000","last_peered":"2024-04-25T08:52:39.834357+0000","last_clean":"2024-04-25T08:52:39.834357+0000","last_became_active":"2024-04-25T08:52:31.257528+0000","last_became_peered":"2024-04-25T08:52:31.257528+0000","last_unstale":"2024-04-25T08:52:39.834357+0000","last_undegraded":"2024-04-25T08:52:39.834357+0000","last_fullsized":"2024-04-25T08:52:39.834357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835503+0000","last_change":"2024-04-25T08:52:30.217142+0000","last_active":"2024-04-25T08:52:39.835503+0000","last_peered":"2024-04-25T08:52:39.835503+0000","last_clean":"2024-04-25T08:52:39.835503+0000","last_became_active":"2024-04-25T08:52:30.216729+0000","last_became_peered":"2024-04-25T08:52:30.216729+0000","last_unstale":"2024-04-25T08:52:39.835503+0000","last_undegraded":"2024-04-25T08:52:39.835503+0000","last_fullsized":"2024-04-25T08:52:39.835503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300819+0000","last_change":"2024-04-25T08:52:30.228704+0000","last_active":"2024-04-25T08:52:38.300819+0000","last_peered":"2024-04-25T08:52:38.300819+0000","last_clean":"2024-04-25T08:52:38.300819+0000","last_became_active":"2024-04-25T08:52:30.227861+0000","last_became_peered":"2024-04-25T08:52:30.227861+0000","last_unstale":"2024-04-25T08:52:38.300819+0000","last_undegraded":"2024-04-25T08:52:38.300819+0000","last_fullsized":"2024-04-25T08:52:38.300819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835140+0000","last_change":"2024-04-25T08:52:31.258702+0000","last_active":"2024-04-25T08:52:39.835140+0000","last_peered":"2024-04-25T08:52:39.835140+0000","last_clean":"2024-04-25T08:52:39.835140+0000","last_became_active":"2024-04-25T08:52:31.258349+0000","last_became_peered":"2024-04-25T08:52:31.258349+0000","last_unstale":"2024-04-25T08:52:39.835140+0000","last_undegraded":"2024-04-25T08:52:39.835140+0000","last_fullsized":"2024-04-25T08:52:39.835140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299697+0000","last_change":"2024-04-25T08:52:35.313396+0000","last_active":"2024-04-25T08:52:38.299697+0000","last_peered":"2024-04-25T08:52:38.299697+0000","last_clean":"2024-04-25T08:52:38.299697+0000","last_became_active":"2024-04-25T08:52:35.312845+0000","last_became_peered":"2024-04-25T08:52:35.312845+0000","last_unstale":"2024-04-25T08:52:38.299697+0000","last_undegraded":"2024-04-25T08:52:38.299697+0000","last_fullsized":"2024-04-25T08:52:38.299697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.648302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836001+0000","last_change":"2024-04-25T08:52:30.239169+0000","last_active":"2024-04-25T08:52:39.836001+0000","last_peered":"2024-04-25T08:52:39.836001+0000","last_clean":"2024-04-25T08:52:39.836001+0000","last_became_active":"2024-04-25T08:52:30.238820+0000","last_became_peered":"2024-04-25T08:52:30.238820+0000","last_unstale":"2024-04-25T08:52:39.836001+0000","last_undegraded":"2024-04-25T08:52:39.836001+0000","last_fullsized":"2024-04-25T08:52:39.836001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300334+0000","last_change":"2024-04-25T08:52:31.243105+0000","last_active":"2024-04-25T08:52:38.300334+0000","last_peered":"2024-04-25T08:52:38.300334+0000","last_clean":"2024-04-25T08:52:38.300334+0000","last_became_active":"2024-04-25T08:52:31.242326+0000","last_became_peered":"2024-04-25T08:52:31.242326+0000","last_unstale":"2024-04-25T08:52:38.300334+0000","last_undegraded":"2024-04-25T08:52:38.300334+0000","last_fullsized":"2024-04-25T08:52:38.300334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:04.149068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834664+0000","last_change":"2024-04-25T08:52:35.313390+0000","last_active":"2024-04-25T08:52:39.834664+0000","last_peered":"2024-04-25T08:52:39.834664+0000","last_clean":"2024-04-25T08:52:39.834664+0000","last_became_active":"2024-04-25T08:52:35.312750+0000","last_became_peered":"2024-04-25T08:52:35.312750+0000","last_unstale":"2024-04-25T08:52:39.834664+0000","last_undegraded":"2024-04-25T08:52:39.834664+0000","last_fullsized":"2024-04-25T08:52:39.834664+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:49.016809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298785+0000","last_change":"2024-04-25T08:52:31.257097+0000","last_active":"2024-04-25T08:52:38.298785+0000","last_peered":"2024-04-25T08:52:38.298785+0000","last_clean":"2024-04-25T08:52:38.298785+0000","last_became_active":"2024-04-25T08:52:31.255697+0000","last_became_peered":"2024-04-25T08:52:31.255697+0000","last_unstale":"2024-04-25T08:52:38.298785+0000","last_undegraded":"2024-04-25T08:52:38.298785+0000","last_fullsized":"2024-04-25T08:52:38.298785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298858+0000","last_change":"2024-04-25T08:52:31.237994+0000","last_active":"2024-04-25T08:52:38.298858+0000","last_peered":"2024-04-25T08:52:38.298858+0000","last_clean":"2024-04-25T08:52:38.298858+0000","last_became_active":"2024-04-25T08:52:31.237190+0000","last_became_peered":"2024-04-25T08:52:31.237190+0000","last_unstale":"2024-04-25T08:52:38.298858+0000","last_undegraded":"2024-04-25T08:52:38.298858+0000","last_fullsized":"2024-04-25T08:52:38.298858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:35.208568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645427+0000","last_change":"2024-04-25T08:52:31.248873+0000","last_active":"2024-04-25T08:52:39.645427+0000","last_peered":"2024-04-25T08:52:39.645427+0000","last_clean":"2024-04-25T08:52:39.645427+0000","last_became_active":"2024-04-25T08:52:31.248359+0000","last_became_peered":"2024-04-25T08:52:31.248359+0000","last_unstale":"2024-04-25T08:52:39.645427+0000","last_undegraded":"2024-04-25T08:52:39.645427+0000","last_fullsized":"2024-04-25T08:52:39.645427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836722+0000","last_change":"2024-04-25T08:52:31.256155+0000","last_active":"2024-04-25T08:52:39.836722+0000","last_peered":"2024-04-25T08:52:39.836722+0000","last_clean":"2024-04-25T08:52:39.836722+0000","last_became_active":"2024-04-25T08:52:31.255884+0000","last_became_peered":"2024-04-25T08:52:31.255884+0000","last_unstale":"2024-04-25T08:52:39.836722+0000","last_undegraded":"2024-04-25T08:52:39.836722+0000","last_fullsized":"2024-04-25T08:52:39.836722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834472+0000","last_change":"2024-04-25T08:52:31.266984+0000","last_active":"2024-04-25T08:52:39.834472+0000","last_peered":"2024-04-25T08:52:39.834472+0000","last_clean":"2024-04-25T08:52:39.834472+0000","last_became_active":"2024-04-25T08:52:31.266697+0000","last_became_peered":"2024-04-25T08:52:31.266697+0000","last_unstale":"2024-04-25T08:52:39.834472+0000","last_undegraded":"2024-04-25T08:52:39.834472+0000","last_fullsized":"2024-04-25T08:52:39.834472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833201+0000","last_change":"2024-04-25T08:52:31.234570+0000","last_active":"2024-04-25T08:52:39.833201+0000","last_peered":"2024-04-25T08:52:39.833201+0000","last_clean":"2024-04-25T08:52:39.833201+0000","last_became_active":"2024-04-25T08:52:31.234049+0000","last_became_peered":"2024-04-25T08:52:31.234049+0000","last_unstale":"2024-04-25T08:52:39.833201+0000","last_undegraded":"2024-04-25T08:52:39.833201+0000","last_fullsized":"2024-04-25T08:52:39.833201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303100+0000","last_change":"2024-04-25T08:52:31.271083+0000","last_active":"2024-04-25T08:52:38.303100+0000","last_peered":"2024-04-25T08:52:38.303100+0000","last_clean":"2024-04-25T08:52:38.303100+0000","last_became_active":"2024-04-25T08:52:31.268755+0000","last_became_peered":"2024-04-25T08:52:31.268755+0000","last_unstale":"2024-04-25T08:52:38.303100+0000","last_undegraded":"2024-04-25T08:52:38.303100+0000","last_fullsized":"2024-04-25T08:52:38.303100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834579+0000","last_change":"2024-04-25T08:52:31.265576+0000","last_active":"2024-04-25T08:52:39.834579+0000","last_peered":"2024-04-25T08:52:39.834579+0000","last_clean":"2024-04-25T08:52:39.834579+0000","last_became_active":"2024-04-25T08:52:31.265273+0000","last_became_peered":"2024-04-25T08:52:31.265273+0000","last_unstale":"2024-04-25T08:52:39.834579+0000","last_undegraded":"2024-04-25T08:52:39.834579+0000","last_fullsized":"2024-04-25T08:52:39.834579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644626+0000","last_change":"2024-04-25T08:52:31.247316+0000","last_active":"2024-04-25T08:52:39.644626+0000","last_peered":"2024-04-25T08:52:39.644626+0000","last_clean":"2024-04-25T08:52:39.644626+0000","last_became_active":"2024-04-25T08:52:31.246547+0000","last_became_peered":"2024-04-25T08:52:31.246547+0000","last_unstale":"2024-04-25T08:52:39.644626+0000","last_undegraded":"2024-04-25T08:52:39.644626+0000","last_fullsized":"2024-04-25T08:52:39.644626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833082+0000","last_change":"2024-04-25T08:52:31.265084+0000","last_active":"2024-04-25T08:52:39.833082+0000","last_peered":"2024-04-25T08:52:39.833082+0000","last_clean":"2024-04-25T08:52:39.833082+0000","last_became_active":"2024-04-25T08:52:31.264883+0000","last_became_peered":"2024-04-25T08:52:31.264883+0000","last_unstale":"2024-04-25T08:52:39.833082+0000","last_undegraded":"2024-04-25T08:52:39.833082+0000","last_fullsized":"2024-04-25T08:52:39.833082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645197+0000","last_change":"2024-04-25T08:52:31.280090+0000","last_active":"2024-04-25T08:52:39.645197+0000","last_peered":"2024-04-25T08:52:39.645197+0000","last_clean":"2024-04-25T08:52:39.645197+0000","last_became_active":"2024-04-25T08:52:31.279453+0000","last_became_peered":"2024-04-25T08:52:31.279453+0000","last_unstale":"2024-04-25T08:52:39.645197+0000","last_undegraded":"2024-04-25T08:52:39.645197+0000","last_fullsized":"2024-04-25T08:52:39.645197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836409+0000","last_change":"2024-04-25T08:52:31.271765+0000","last_active":"2024-04-25T08:52:39.836409+0000","last_peered":"2024-04-25T08:52:39.836409+0000","last_clean":"2024-04-25T08:52:39.836409+0000","last_became_active":"2024-04-25T08:52:31.271438+0000","last_became_peered":"2024-04-25T08:52:31.271438+0000","last_unstale":"2024-04-25T08:52:39.836409+0000","last_undegraded":"2024-04-25T08:52:39.836409+0000","last_fullsized":"2024-04-25T08:52:39.836409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301809+0000","last_change":"2024-04-25T08:52:31.270307+0000","last_active":"2024-04-25T08:52:38.301809+0000","last_peered":"2024-04-25T08:52:38.301809+0000","last_clean":"2024-04-25T08:52:38.301809+0000","last_became_active":"2024-04-25T08:52:31.269440+0000","last_became_peered":"2024-04-25T08:52:31.269440+0000","last_unstale":"2024-04-25T08:52:38.301809+0000","last_undegraded":"2024-04-25T08:52:38.301809+0000","last_fullsized":"2024-04-25T08:52:38.301809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305298+0000","last_change":"2024-04-25T08:52:31.281491+0000","last_active":"2024-04-25T08:52:39.305298+0000","last_peered":"2024-04-25T08:52:39.305298+0000","last_clean":"2024-04-25T08:52:39.305298+0000","last_became_active":"2024-04-25T08:52:31.276333+0000","last_became_peered":"2024-04-25T08:52:31.276333+0000","last_unstale":"2024-04-25T08:52:39.305298+0000","last_undegraded":"2024-04-25T08:52:39.305298+0000","last_fullsized":"2024-04-25T08:52:39.305298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:31.614898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299491+0000","last_change":"2024-04-25T08:52:31.237856+0000","last_active":"2024-04-25T08:52:38.299491+0000","last_peered":"2024-04-25T08:52:38.299491+0000","last_clean":"2024-04-25T08:52:38.299491+0000","last_became_active":"2024-04-25T08:52:31.237582+0000","last_became_peered":"2024-04-25T08:52:31.237582+0000","last_unstale":"2024-04-25T08:52:38.299491+0000","last_undegraded":"2024-04-25T08:52:38.299491+0000","last_fullsized":"2024-04-25T08:52:38.299491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:05.271538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.646089+0000","last_change":"2024-04-25T08:52:31.283655+0000","last_active":"2024-04-25T08:52:39.646089+0000","last_peered":"2024-04-25T08:52:39.646089+0000","last_clean":"2024-04-25T08:52:39.646089+0000","last_became_active":"2024-04-25T08:52:31.282969+0000","last_became_peered":"2024-04-25T08:52:31.282969+0000","last_unstale":"2024-04-25T08:52:39.646089+0000","last_undegraded":"2024-04-25T08:52:39.646089+0000","last_fullsized":"2024-04-25T08:52:39.646089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:08.025791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644525+0000","last_change":"2024-04-25T08:52:31.256907+0000","last_active":"2024-04-25T08:52:39.644525+0000","last_peered":"2024-04-25T08:52:39.644525+0000","last_clean":"2024-04-25T08:52:39.644525+0000","last_became_active":"2024-04-25T08:52:31.255494+0000","last_became_peered":"2024-04-25T08:52:31.255494+0000","last_unstale":"2024-04-25T08:52:39.644525+0000","last_undegraded":"2024-04-25T08:52:39.644525+0000","last_fullsized":"2024-04-25T08:52:39.644525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834186+0000","last_change":"2024-04-25T08:52:31.270805+0000","last_active":"2024-04-25T08:52:39.834186+0000","last_peered":"2024-04-25T08:52:39.834186+0000","last_clean":"2024-04-25T08:52:39.834186+0000","last_became_active":"2024-04-25T08:52:31.270594+0000","last_became_peered":"2024-04-25T08:52:31.270594+0000","last_unstale":"2024-04-25T08:52:39.834186+0000","last_undegraded":"2024-04-25T08:52:39.834186+0000","last_fullsized":"2024-04-25T08:52:39.834186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836035+0000","last_change":"2024-04-25T08:52:31.252734+0000","last_active":"2024-04-25T08:52:39.836035+0000","last_peered":"2024-04-25T08:52:39.836035+0000","last_clean":"2024-04-25T08:52:39.836035+0000","last_became_active":"2024-04-25T08:52:31.252476+0000","last_became_peered":"2024-04-25T08:52:31.252476+0000","last_unstale":"2024-04-25T08:52:39.836035+0000","last_undegraded":"2024-04-25T08:52:39.836035+0000","last_fullsized":"2024-04-25T08:52:39.836035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833928+0000","last_change":"2024-04-25T08:52:31.242347+0000","last_active":"2024-04-25T08:52:39.833928+0000","last_peered":"2024-04-25T08:52:39.833928+0000","last_clean":"2024-04-25T08:52:39.833928+0000","last_became_active":"2024-04-25T08:52:31.242082+0000","last_became_peered":"2024-04-25T08:52:31.242082+0000","last_unstale":"2024-04-25T08:52:39.833928+0000","last_undegraded":"2024-04-25T08:52:39.833928+0000","last_fullsized":"2024-04-25T08:52:39.833928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301519+0000","last_change":"2024-04-25T08:52:31.258568+0000","last_active":"2024-04-25T08:52:38.301519+0000","last_peered":"2024-04-25T08:52:38.301519+0000","last_clean":"2024-04-25T08:52:38.301519+0000","last_became_active":"2024-04-25T08:52:31.257438+0000","last_became_peered":"2024-04-25T08:52:31.257438+0000","last_unstale":"2024-04-25T08:52:38.301519+0000","last_undegraded":"2024-04-25T08:52:38.301519+0000","last_fullsized":"2024-04-25T08:52:38.301519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:47.022570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.302377+0000","last_change":"2024-04-25T08:52:31.254993+0000","last_active":"2024-04-25T08:52:38.302377+0000","last_peered":"2024-04-25T08:52:38.302377+0000","last_clean":"2024-04-25T08:52:38.302377+0000","last_became_active":"2024-04-25T08:52:31.254072+0000","last_became_peered":"2024-04-25T08:52:31.254072+0000","last_unstale":"2024-04-25T08:52:38.302377+0000","last_undegraded":"2024-04-25T08:52:38.302377+0000","last_fullsized":"2024-04-25T08:52:38.302377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303017+0000","last_change":"2024-04-25T08:52:31.247059+0000","last_active":"2024-04-25T08:52:38.303017+0000","last_peered":"2024-04-25T08:52:38.303017+0000","last_clean":"2024-04-25T08:52:38.303017+0000","last_became_active":"2024-04-25T08:52:31.246364+0000","last_became_peered":"2024-04-25T08:52:31.246364+0000","last_unstale":"2024-04-25T08:52:38.303017+0000","last_undegraded":"2024-04-25T08:52:38.303017+0000","last_fullsized":"2024-04-25T08:52:38.303017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.308187+0000","last_change":"2024-04-25T08:52:31.289253+0000","last_active":"2024-04-25T08:52:39.308187+0000","last_peered":"2024-04-25T08:52:39.308187+0000","last_clean":"2024-04-25T08:52:39.308187+0000","last_became_active":"2024-04-25T08:52:31.288175+0000","last_became_peered":"2024-04-25T08:52:31.288175+0000","last_unstale":"2024-04-25T08:52:39.308187+0000","last_undegraded":"2024-04-25T08:52:39.308187+0000","last_fullsized":"2024-04-25T08:52:39.308187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:08.025791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834144+0000","last_change":"2024-04-25T08:52:31.256539+0000","last_active":"2024-04-25T08:52:39.834144+0000","last_peered":"2024-04-25T08:52:39.834144+0000","last_clean":"2024-04-25T08:52:39.834144+0000","last_became_active":"2024-04-25T08:52:31.255921+0000","last_became_peered":"2024-04-25T08:52:31.255921+0000","last_unstale":"2024-04-25T08:52:39.834144+0000","last_undegraded":"2024-04-25T08:52:39.834144+0000","last_fullsized":"2024-04-25T08:52:39.834144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306615+0000","last_change":"2024-04-25T08:52:31.277410+0000","last_active":"2024-04-25T08:52:39.306615+0000","last_peered":"2024-04-25T08:52:39.306615+0000","last_clean":"2024-04-25T08:52:39.306615+0000","last_became_active":"2024-04-25T08:52:31.276941+0000","last_became_peered":"2024-04-25T08:52:31.276941+0000","last_unstale":"2024-04-25T08:52:39.306615+0000","last_undegraded":"2024-04-25T08:52:39.306615+0000","last_fullsized":"2024-04-25T08:52:39.306615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645097+0000","last_change":"2024-04-25T08:52:31.256171+0000","last_active":"2024-04-25T08:52:39.645097+0000","last_peered":"2024-04-25T08:52:39.645097+0000","last_clean":"2024-04-25T08:52:39.645097+0000","last_became_active":"2024-04-25T08:52:31.255417+0000","last_became_peered":"2024-04-25T08:52:31.255417+0000","last_unstale":"2024-04-25T08:52:39.645097+0000","last_undegraded":"2024-04-25T08:52:39.645097+0000","last_fullsized":"2024-04-25T08:52:39.645097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305186+0000","last_change":"2024-04-25T08:52:31.269978+0000","last_active":"2024-04-25T08:52:39.305186+0000","last_peered":"2024-04-25T08:52:39.305186+0000","last_clean":"2024-04-25T08:52:39.305186+0000","last_became_active":"2024-04-25T08:52:31.269108+0000","last_became_peered":"2024-04-25T08:52:31.269108+0000","last_unstale":"2024-04-25T08:52:39.305186+0000","last_undegraded":"2024-04-25T08:52:39.305186+0000","last_fullsized":"2024-04-25T08:52:39.305186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645458+0000","last_change":"2024-04-25T08:52:31.248569+0000","last_active":"2024-04-25T08:52:39.645458+0000","last_peered":"2024-04-25T08:52:39.645458+0000","last_clean":"2024-04-25T08:52:39.645458+0000","last_became_active":"2024-04-25T08:52:31.247655+0000","last_became_peered":"2024-04-25T08:52:31.247655+0000","last_unstale":"2024-04-25T08:52:39.645458+0000","last_undegraded":"2024-04-25T08:52:39.645458+0000","last_fullsized":"2024-04-25T08:52:39.645458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833573+0000","last_change":"2024-04-25T08:52:31.252449+0000","last_active":"2024-04-25T08:52:39.833573+0000","last_peered":"2024-04-25T08:52:39.833573+0000","last_clean":"2024-04-25T08:52:39.833573+0000","last_became_active":"2024-04-25T08:52:31.252212+0000","last_became_peered":"2024-04-25T08:52:31.252212+0000","last_unstale":"2024-04-25T08:52:39.833573+0000","last_undegraded":"2024-04-25T08:52:39.833573+0000","last_fullsized":"2024-04-25T08:52:39.833573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833700+0000","last_change":"2024-04-25T08:52:31.254735+0000","last_active":"2024-04-25T08:52:39.833700+0000","last_peered":"2024-04-25T08:52:39.833700+0000","last_clean":"2024-04-25T08:52:39.833700+0000","last_became_active":"2024-04-25T08:52:31.254523+0000","last_became_peered":"2024-04-25T08:52:31.254523+0000","last_unstale":"2024-04-25T08:52:39.833700+0000","last_undegraded":"2024-04-25T08:52:39.833700+0000","last_fullsized":"2024-04-25T08:52:39.833700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305895+0000","last_change":"2024-04-25T08:52:31.288927+0000","last_active":"2024-04-25T08:52:39.305895+0000","last_peered":"2024-04-25T08:52:39.305895+0000","last_clean":"2024-04-25T08:52:39.305895+0000","last_became_active":"2024-04-25T08:52:31.287790+0000","last_became_peered":"2024-04-25T08:52:31.287790+0000","last_unstale":"2024-04-25T08:52:39.305895+0000","last_undegraded":"2024-04-25T08:52:39.305895+0000","last_fullsized":"2024-04-25T08:52:39.305895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:52.007683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":709746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":122638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":118886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":118922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":122638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":118796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":118956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":897024,"data_stored":709802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:52:55.745 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:52:55.746 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T08:52:55.746 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T08:52:55.746 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T08:52:55.746 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T08:52:55.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe4320c640 1 Processor -- start 2024-04-25T08:52:55.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe4320c640 1 -- start start 2024-04-25T08:52:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe4320c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efe3c15b950 0x7efe3c09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe4320c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c15bd30 con 0x7efe3c0cfb80 2024-04-25T08:52:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe4320c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c0cc0d0 con 0x7efe3c0cf420 2024-04-25T08:52:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe4320c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c09ab10 con 0x7efe3c15b950 2024-04-25T08:52:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe40fa8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efe3c15b950 0x7efe3c09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe417a9640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7efe3c0cfb80 0x7efe3c0cff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59844/0 (socket says 172.21.15.37:59844) 2024-04-25T08:52:55.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe417a9640 1 -- 172.21.15.37:0/779191183 learned_addr learned my addr 172.21.15.37:0/779191183 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:55.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe40fa8640 1 -- 172.21.15.37:0/779191183 >> v1:172.21.15.203:6789/0 conn(0x7efe3c0cf420 legacy=0x7efe3c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:55.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe40fa8640 1 -- 172.21.15.37:0/779191183 >> v1:172.21.15.37:6789/0 conn(0x7efe3c0cfb80 legacy=0x7efe3c0cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe40fa8640 1 -- 172.21.15.37:0/779191183 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c0cc370 con 0x7efe3c15b950 2024-04-25T08:52:55.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.916+0000 7efe40fa8640 1 --2- 172.21.15.37:0/779191183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c15b950 0x7efe3c09a480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097c0 tx=0x7efe2c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c840174aae7538ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:55.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe3b7fe640 1 -- 172.21.15.37:0/779191183 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0b0070 con 0x7efe3c15b950 2024-04-25T08:52:55.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe3b7fe640 1 -- 172.21.15.37:0/779191183 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c004230 con 0x7efe3c15b950 2024-04-25T08:52:55.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe3b7fe640 1 -- 172.21.15.37:0/779191183 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c003620 con 0x7efe3c15b950 2024-04-25T08:52:55.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 -- 172.21.15.37:0/779191183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c15b950 msgr2=0x7efe3c09a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 --2- 172.21.15.37:0/779191183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c15b950 0x7efe3c09a480 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0097c0 tx=0x7efe2c01c460 comp rx=0 tx=0).stop 2024-04-25T08:52:55.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 -- 172.21.15.37:0/779191183 shutdown_connections 2024-04-25T08:52:55.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 --2- 172.21.15.37:0/779191183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c15b950 0x7efe3c09a480 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 -- 172.21.15.37:0/779191183 >> 172.21.15.37:0/779191183 conn(0x7efe3c0b30b0 msgr2=0x7efe3c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:55.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 -- 172.21.15.37:0/779191183 shutdown_connections 2024-04-25T08:52:55.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 -- 172.21.15.37:0/779191183 wait complete. 2024-04-25T08:52:55.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 Processor -- start 2024-04-25T08:52:55.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 -- start start 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c0cf420 0x7efe3c152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0cfb80 0x7efe3c0b6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe3c15b950 0x7efe3c070560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c06fc20 con 0x7efe3c0cf420 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c06faa0 con 0x7efe3c0cfb80 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c169780 con 0x7efe3c15b950 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe40fa8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c0cf420 0x7efe3c152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe40fa8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c0cf420 0x7efe3c152930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46338/0 (socket says 172.21.15.37:46338) 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe40fa8640 1 -- 172.21.15.37:0/2371288996 learned_addr learned my addr 172.21.15.37:0/2371288996 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe3bfff640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0cfb80 0x7efe3c0b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe40fa8640 1 -- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe3c15b950 msgr2=0x7efe3c070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe417a9640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe3c15b950 0x7efe3c070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe40fa8640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe3c15b950 0x7efe3c070560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe40fa8640 1 -- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0cfb80 msgr2=0x7efe3c0b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe40fa8640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0cfb80 0x7efe3c0b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe40fa8640 1 -- 172.21.15.37:0/2371288996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c0b56f0 con 0x7efe3c0cf420 2024-04-25T08:52:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe3bfff640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0cfb80 0x7efe3c0b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe417a9640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe3c15b950 0x7efe3c070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe40fa8640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c0cf420 0x7efe3c152930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0ae040 tx=0x7efe2c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe397fa640 1 -- 172.21.15.37:0/2371288996 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0b9020 con 0x7efe3c0cf420 2024-04-25T08:52:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe397fa640 1 -- 172.21.15.37:0/2371288996 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c0043e0 con 0x7efe3c0cf420 2024-04-25T08:52:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe4320c640 1 -- 172.21.15.37:0/2371288996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c0b59d0 con 0x7efe3c0cf420 2024-04-25T08:52:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.920+0000 7efe397fa640 1 -- 172.21.15.37:0/2371288996 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0b0040 con 0x7efe3c0cf420 2024-04-25T08:52:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.924+0000 7efe4320c640 1 -- 172.21.15.37:0/2371288996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c0b5f90 con 0x7efe3c0cf420 2024-04-25T08:52:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.924+0000 7efe4320c640 1 -- 172.21.15.37:0/2371288996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c04efa0 con 0x7efe3c0cf420 2024-04-25T08:52:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.924+0000 7efe397fa640 1 -- 172.21.15.37:0/2371288996 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe2c0bc020 con 0x7efe3c0cf420 2024-04-25T08:52:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.924+0000 7efe397fa640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe28066f60 0x7efe28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.924+0000 7efe397fa640 1 -- 172.21.15.37:0/2371288996 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efe2c11f3d0 con 0x7efe3c0cf420 2024-04-25T08:52:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.924+0000 7efe3bfff640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe28066f60 0x7efe28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:55.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.924+0000 7efe3bfff640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe28066f60 0x7efe28069420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0cd810 tx=0x7efe30008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:55.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:55.928+0000 7efe397fa640 1 -- 172.21.15.37:0/2371288996 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe2c0c3030 con 0x7efe3c0cf420 2024-04-25T08:52:56.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.128+0000 7efe4320c640 1 -- 172.21.15.37:0/2371288996 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7efe3c0d13a0 con 0x7efe3c0cf420 2024-04-25T08:52:56.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.136+0000 7efe397fa640 1 -- 172.21.15.37:0/2371288996 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165739 (secure 0 0 0) 0x7efe2c016050 con 0x7efe3c0cf420 2024-04-25T08:52:56.132 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:56.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 -- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe28066f60 msgr2=0x7efe28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe28066f60 0x7efe28069420 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0cd810 tx=0x7efe30008040 comp rx=0 tx=0).stop 2024-04-25T08:52:56.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 -- 172.21.15.37:0/2371288996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c0cf420 msgr2=0x7efe3c152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c0cf420 0x7efe3c152930 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efe2c0ae040 tx=0x7efe2c019200 comp rx=0 tx=0).stop 2024-04-25T08:52:56.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 -- 172.21.15.37:0/2371288996 shutdown_connections 2024-04-25T08:52:56.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe28066f60 0x7efe28069420 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe3c15b950 0x7efe3c070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c0cfb80 0x7efe3c0b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 --2- 172.21.15.37:0/2371288996 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c0cf420 0x7efe3c152930 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 -- 172.21.15.37:0/2371288996 >> 172.21.15.37:0/2371288996 conn(0x7efe3c0b30b0 msgr2=0x7efe3c0ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:56.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 -- 172.21.15.37:0/2371288996 shutdown_connections 2024-04-25T08:52:56.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.140+0000 7efe4320c640 1 -- 172.21.15.37:0/2371288996 wait complete. 2024-04-25T08:52:56.152 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6832","nonce":3562039738},{"type":"v1","addr":"172.21.15.203:6833","nonce":3562039738}]},"active_addr":"172.21.15.203:6833/3562039738","active_change":"2024-04-25T08:52:11.103871+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":956554480}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":4185726171}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":3690485917}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.203:0","nonce":2433196309}]}]} 2024-04-25T08:52:56.155 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T08:52:56.155 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T08:52:56.155 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:56.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.280+0000 7f60bc280640 1 Processor -- start 2024-04-25T08:52:56.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.280+0000 7f60bc280640 1 -- start start 2024-04-25T08:52:56.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60bc280640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f60b40d07f0 0x7f60b40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60bc280640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f60b40d1cd0 0x7f60b406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60bc280640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b406be60 con 0x7f60b415da80 2024-04-25T08:52:56.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60bc280640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b406c440 con 0x7f60b40d1cd0 2024-04-25T08:52:56.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60bc280640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b415de60 con 0x7f60b40d07f0 2024-04-25T08:52:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60ba81d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f60b40d1cd0 0x7f60b406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60ba81d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f60b40d1cd0 0x7f60b406c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46352/0 (socket says 172.21.15.37:46352) 2024-04-25T08:52:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60ba81d640 1 -- 172.21.15.37:0/1023155125 learned_addr learned my addr 172.21.15.37:0/1023155125 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60b981b640 1 --2- 172.21.15.37:0/1023155125 >> v2:172.21.15.203:3300/0 conn(0x7f60b40d07f0 0x7f60b40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60ba81d640 1 -- 172.21.15.37:0/1023155125 >> v2:172.21.15.203:3300/0 conn(0x7f60b40d07f0 msgr2=0x7f60b40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60ba81d640 1 --2- 172.21.15.37:0/1023155125 >> v2:172.21.15.203:3300/0 conn(0x7f60b40d07f0 0x7f60b40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60ba81d640 1 -- 172.21.15.37:0/1023155125 >> v1:172.21.15.203:6789/0 conn(0x7f60b415da80 legacy=0x7f60b40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60ba81d640 1 -- 172.21.15.37:0/1023155125 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b40d2450 con 0x7f60b40d1cd0 2024-04-25T08:52:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60ba81d640 1 --2- 172.21.15.37:0/1023155125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b40d1cd0 0x7f60b406c8d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f60b408d880 tx=0x7f60ac01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671ca42cf64c2e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60b901a640 1 -- 172.21.15.37:0/1023155125 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac0aa670 con 0x7f60b40d1cd0 2024-04-25T08:52:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60b901a640 1 -- 172.21.15.37:0/1023155125 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac0af070 con 0x7f60b40d1cd0 2024-04-25T08:52:56.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.284+0000 7f60b901a640 1 -- 172.21.15.37:0/1023155125 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac0aaa30 con 0x7f60b40d1cd0 2024-04-25T08:52:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 -- 172.21.15.37:0/1023155125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b40d1cd0 msgr2=0x7f60b406c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 --2- 172.21.15.37:0/1023155125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b40d1cd0 0x7f60b406c8d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f60b408d880 tx=0x7f60ac01c4f0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 -- 172.21.15.37:0/1023155125 shutdown_connections 2024-04-25T08:52:56.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 --2- 172.21.15.37:0/1023155125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b40d1cd0 0x7f60b406c8d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 --2- 172.21.15.37:0/1023155125 >> v2:172.21.15.203:3300/0 conn(0x7f60b40d07f0 0x7f60b40d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 -- 172.21.15.37:0/1023155125 >> 172.21.15.37:0/1023155125 conn(0x7f60b408b000 msgr2=0x7f60b40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:56.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 -- 172.21.15.37:0/1023155125 shutdown_connections 2024-04-25T08:52:56.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 -- 172.21.15.37:0/1023155125 wait complete. 2024-04-25T08:52:56.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 Processor -- start 2024-04-25T08:52:56.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 -- start start 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60b40d07f0 0x7f60b4156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b40d1cd0 0x7f60b415b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b415da80 0x7f60b415bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b406d7a0 con 0x7f60b415da80 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b406d620 con 0x7f60b40d1cd0 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60bc280640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b406d920 con 0x7f60b40d07f0 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60ba81d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b415da80 0x7f60b415bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60ba81d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b415da80 0x7f60b415bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46364/0 (socket says 172.21.15.37:46364) 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60ba81d640 1 -- 172.21.15.37:0/1174769470 learned_addr learned my addr 172.21.15.37:0/1174769470 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60b981b640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b40d1cd0 0x7f60b415b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60ba01c640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60b40d07f0 0x7f60b4156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60ba81d640 1 -- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60b40d07f0 msgr2=0x7f60b4156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60ba81d640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60b40d07f0 0x7f60b4156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60ba81d640 1 -- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b40d1cd0 msgr2=0x7f60b415b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60ba81d640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b40d1cd0 0x7f60b415b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60ba81d640 1 -- 172.21.15.37:0/1174769470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b4160e60 con 0x7f60b415da80 2024-04-25T08:52:56.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60b981b640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b40d1cd0 0x7f60b415b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:56.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60ba81d640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b415da80 0x7f60b415bf10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f60ac019100 tx=0x7f60ac01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60b2ffd640 1 -- 172.21.15.37:0/1174769470 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac0bd5f0 con 0x7f60b415da80 2024-04-25T08:52:56.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60b2ffd640 1 -- 172.21.15.37:0/1174769470 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac0bd790 con 0x7f60b415da80 2024-04-25T08:52:56.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.288+0000 7f60b2ffd640 1 -- 172.21.15.37:0/1174769470 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac0bdb80 con 0x7f60b415da80 2024-04-25T08:52:56.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.292+0000 7f60bc280640 1 -- 172.21.15.37:0/1174769470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b415fdc0 con 0x7f60b415da80 2024-04-25T08:52:56.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.292+0000 7f60bc280640 1 -- 172.21.15.37:0/1174769470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b4160380 con 0x7f60b415da80 2024-04-25T08:52:56.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.292+0000 7f60bc280640 1 -- 172.21.15.37:0/1174769470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b404efa0 con 0x7f60b415da80 2024-04-25T08:52:56.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.292+0000 7f60b2ffd640 1 -- 172.21.15.37:0/1174769470 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60ac0b33f0 con 0x7f60b415da80 2024-04-25T08:52:56.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.292+0000 7f60b2ffd640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f60a0066f90 0x7f60a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.292+0000 7f60b2ffd640 1 -- 172.21.15.37:0/1174769470 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f60ac11fcc0 con 0x7f60b415da80 2024-04-25T08:52:56.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.292+0000 7f60ba01c640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f60a0066f90 0x7f60a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.292+0000 7f60ba01c640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f60a0066f90 0x7f60a0069450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f60a40012f0 tx=0x7f60a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.296+0000 7f60b2ffd640 1 -- 172.21.15.37:0/1174769470 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f60ac016110 con 0x7f60b415da80 2024-04-25T08:52:56.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.456+0000 7f60bc280640 1 -- 172.21.15.37:0/1174769470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60b40cdcb0 con 0x7f60b415da80 2024-04-25T08:52:56.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.460+0000 7f60b2ffd640 1 -- 172.21.15.37:0/1174769470 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f60ac0f0340 con 0x7f60b415da80 2024-04-25T08:52:56.454 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:56.454 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:38.282648+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:56.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 -- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f60a0066f90 msgr2=0x7f60a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f60a0066f90 0x7f60a0069450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f60a40012f0 tx=0x7f60a4009040 comp rx=0 tx=0).stop 2024-04-25T08:52:56.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 -- 172.21.15.37:0/1174769470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b415da80 msgr2=0x7f60b415bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b415da80 0x7f60b415bf10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f60ac019100 tx=0x7f60ac01ad30 comp rx=0 tx=0).stop 2024-04-25T08:52:56.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 -- 172.21.15.37:0/1174769470 shutdown_connections 2024-04-25T08:52:56.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f60a0066f90 0x7f60a0069450 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60b415da80 0x7f60b415bf10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60b40d1cd0 0x7f60b415b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 --2- 172.21.15.37:0/1174769470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f60b40d07f0 0x7f60b4156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 -- 172.21.15.37:0/1174769470 >> 172.21.15.37:0/1174769470 conn(0x7f60b408b000 msgr2=0x7f60b40b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:56.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 -- 172.21.15.37:0/1174769470 shutdown_connections 2024-04-25T08:52:56.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.464+0000 7f60bc280640 1 -- 172.21.15.37:0/1174769470 wait complete. 2024-04-25T08:52:56.472 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T08:52:56.472 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:56.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd699b4640 1 Processor -- start 2024-04-25T08:52:56.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd699b4640 1 -- start start 2024-04-25T08:52:56.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd699b4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd640d07f0 0x7fcd640d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd699b4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6406be60 con 0x7fcd6415da80 2024-04-25T08:52:56.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd699b4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd6415de60 con 0x7fcd640d1cd0 2024-04-25T08:52:56.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd699b4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6406c440 con 0x7fcd640d07f0 2024-04-25T08:52:56.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd62ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fcd6415da80 0x7fcd640d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59856/0 (socket says 172.21.15.37:59856) 2024-04-25T08:52:56.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd62ffd640 1 -- 172.21.15.37:0/1623442573 learned_addr learned my addr 172.21.15.37:0/1623442573 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd627fc640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd640d07f0 0x7fcd640d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984407267 0 0) 0x7fcd6406be60 con 0x7fcd6415da80 2024-04-25T08:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd50003530 con 0x7fcd6415da80 2024-04-25T08:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2696465823 0 0) 0x7fcd50003530 con 0x7fcd6415da80 2024-04-25T08:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd640d07f0 msgr2=0x7fcd640d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd640d07f0 0x7fcd640d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 >> v1:172.21.15.203:6789/0 conn(0x7fcd640d1cd0 legacy=0x7fcd6406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd640d2450 con 0x7fcd6415da80 2024-04-25T08:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fcd54001680 con 0x7fcd6415da80 2024-04-25T08:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 >> v1:172.21.15.37:6789/0 conn(0x7fcd6415da80 legacy=0x7fcd640d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd50003a60 0x7fcd50003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 0x7fcd50004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd640d07f0 0x7fcd500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6406c440 con 0x7fcd50004810 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5000bb00 con 0x7fcd50003a60 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5000bd00 con 0x7fcd640d07f0 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd627fc640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 0x7fcd50004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd637fe640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd640d07f0 0x7fcd500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd627fc640 1 -- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd640d07f0 msgr2=0x7fcd500051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd627fc640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd640d07f0 0x7fcd500051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd62ffd640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd50003a60 0x7fcd50003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd627fc640 1 -- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd50003a60 msgr2=0x7fcd50003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd627fc640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd50003a60 0x7fcd50003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd627fc640 1 -- 172.21.15.37:0/1623442573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50003530 con 0x7fcd50004810 2024-04-25T08:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.604+0000 7fcd62ffd640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd50003a60 0x7fcd50003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd627fc640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 0x7fcd50004c90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009ac0 tx=0x7fcd4c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75704e9deab35461 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c009690 con 0x7fcd50004810 2024-04-25T08:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c0059a0 con 0x7fcd50004810 2024-04-25T08:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd61ffb640 1 -- 172.21.15.37:0/1623442573 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd54001d50 con 0x7fcd50004810 2024-04-25T08:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd699b4640 1 -- 172.21.15.37:0/1623442573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 msgr2=0x7fcd50004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd699b4640 1 --2- 172.21.15.37:0/1623442573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 0x7fcd50004c90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c009ac0 tx=0x7fcd4c01ca50 comp rx=0 tx=0).stop 2024-04-25T08:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd637fe640 1 -- 172.21.15.37:0/1623442573 reap_dead start 2024-04-25T08:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd699b4640 1 -- 172.21.15.37:0/1623442573 shutdown_connections 2024-04-25T08:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd699b4640 1 -- 172.21.15.37:0/1623442573 >> 172.21.15.37:0/1623442573 conn(0x7fcd6408b000 msgr2=0x7fcd640d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd699b4640 1 -- 172.21.15.37:0/1623442573 shutdown_connections 2024-04-25T08:52:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd699b4640 1 -- 172.21.15.37:0/1623442573 wait complete. 2024-04-25T08:52:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd699b4640 1 Processor -- start 2024-04-25T08:52:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.608+0000 7fcd699b4640 1 -- start start 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd699b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 0x7fcd641f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd699b4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd64157de0 0x7fcd6415f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd699b4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd641f3280 0x7fcd640c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd699b4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd640d26f0 con 0x7fcd50004810 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd699b4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd640d2160 con 0x7fcd64157de0 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd699b4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd640d2870 con 0x7fcd641f3280 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd62ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 0x7fcd641f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd62ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 0x7fcd641f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46390/0 (socket says 172.21.15.37:46390) 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd62ffd640 1 -- 172.21.15.37:0/1987685323 learned_addr learned my addr 172.21.15.37:0/1987685323 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd637fe640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd641f3280 0x7fcd640c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd627fc640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd64157de0 0x7fcd6415f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd62ffd640 1 -- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd641f3280 msgr2=0x7fcd640c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd62ffd640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd641f3280 0x7fcd640c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd62ffd640 1 -- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd64157de0 msgr2=0x7fcd6415f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd62ffd640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd64157de0 0x7fcd6415f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd62ffd640 1 -- 172.21.15.37:0/1987685323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd64158260 con 0x7fcd50004810 2024-04-25T08:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd62ffd640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 0x7fcd641f67f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcd54010b40 tx=0x7fcd54005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd5bfff640 1 -- 172.21.15.37:0/1987685323 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5400e9b0 con 0x7fcd50004810 2024-04-25T08:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd5bfff640 1 -- 172.21.15.37:0/1987685323 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd54011040 con 0x7fcd50004810 2024-04-25T08:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd699b4640 1 -- 172.21.15.37:0/1987685323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd641f7670 con 0x7fcd50004810 2024-04-25T08:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd5bfff640 1 -- 172.21.15.37:0/1987685323 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5400ed50 con 0x7fcd50004810 2024-04-25T08:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd699b4640 1 -- 172.21.15.37:0/1987685323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd641f70b0 con 0x7fcd50004810 2024-04-25T08:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd699b4640 1 -- 172.21.15.37:0/1987685323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6404efa0 con 0x7fcd50004810 2024-04-25T08:52:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd5bfff640 1 -- 172.21.15.37:0/1987685323 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcd5401d020 con 0x7fcd50004810 2024-04-25T08:52:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd5bfff640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcd44066f60 0x7fcd44069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd5bfff640 1 -- 172.21.15.37:0/1987685323 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcd54088b30 con 0x7fcd50004810 2024-04-25T08:52:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.612+0000 7fcd627fc640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcd44066f60 0x7fcd44069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.616+0000 7fcd627fc640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcd44066f60 0x7fcd44069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0ae000 tx=0x7fcd4c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.616+0000 7fcd5bfff640 1 -- 172.21.15.37:0/1987685323 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcd64158260 con 0x7fcd50004810 2024-04-25T08:52:56.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.772+0000 7fcd699b4640 1 -- 172.21.15.37:0/1987685323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd64073b80 con 0x7fcd50004810 2024-04-25T08:52:56.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd5bfff640 1 -- 172.21.15.37:0/1987685323 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18478 (secure 0 0 0) 0x7fcd54059870 con 0x7fcd50004810 2024-04-25T08:52:56.769 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:56.769 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:38.282648+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:56.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 -- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcd44066f60 msgr2=0x7fcd44069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcd44066f60 0x7fcd44069420 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c0ae000 tx=0x7fcd4c0046d0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 -- 172.21.15.37:0/1987685323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 msgr2=0x7fcd641f67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 0x7fcd641f67f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcd54010b40 tx=0x7fcd54005700 comp rx=0 tx=0).stop 2024-04-25T08:52:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 -- 172.21.15.37:0/1987685323 shutdown_connections 2024-04-25T08:52:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcd44066f60 0x7fcd44069420 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd641f3280 0x7fcd640c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd64157de0 0x7fcd6415f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 --2- 172.21.15.37:0/1987685323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd50004810 0x7fcd641f67f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 -- 172.21.15.37:0/1987685323 >> 172.21.15.37:0/1987685323 conn(0x7fcd6408b000 msgr2=0x7fcd6415e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 -- 172.21.15.37:0/1987685323 shutdown_connections 2024-04-25T08:52:56.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.776+0000 7fcd699b4640 1 -- 172.21.15.37:0/1987685323 wait complete. 2024-04-25T08:52:56.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T08:52:56.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T08:52:56.788 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T08:52:56.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T08:52:56.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T08:52:56.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T08:52:56.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T08:52:56.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T08:52:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.972+0000 7f22e9ff9640 1 Processor -- start 2024-04-25T08:52:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.972+0000 7f22e9ff9640 1 -- start start 2024-04-25T08:52:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d7af83640 1 Processor -- start 2024-04-25T08:52:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d7af83640 1 -- start start 2024-04-25T08:52:56.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82dba8e640 1 Processor -- start 2024-04-25T08:52:56.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faac0e8e640 1 Processor -- start 2024-04-25T08:52:56.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e9ff9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22e4159a80 0x7f22e40aec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e9ff9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e40c5950 0x7f22e40cd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e9ff9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22e40acb10 con 0x7f22e40cd600 2024-04-25T08:52:56.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e9ff9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e40ace40 con 0x7f22e4159a80 2024-04-25T08:52:56.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e9ff9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40c5110 con 0x7f22e40c5950 2024-04-25T08:52:56.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d7af83640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7415a430 0x7f3d740c4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d7af83640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3d7415a810 0x7f3d7406e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d7af83640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d7406e6b0 0x7f3d740cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d7af83640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d740c45a0 con 0x7f3d7415a810 2024-04-25T08:52:56.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d7af83640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d7415ad80 con 0x7f3d7406e6b0 2024-04-25T08:52:56.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d7af83640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7415b060 con 0x7f3d7415a430 2024-04-25T08:52:56.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82dba8e640 1 -- start start 2024-04-25T08:52:56.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faac0e8e640 1 -- start start 2024-04-25T08:52:56.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e40c5950 0x7f22e40cd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e8ff7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22e4159a80 0x7f22e40aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e8ff7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22e4159a80 0x7f22e40aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:33920/0 (socket says 172.21.15.37:33920) 2024-04-25T08:52:56.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e8ff7640 1 -- 172.21.15.37:0/2973136350 learned_addr learned my addr 172.21.15.37:0/2973136350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:56.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d79780640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3d7415a810 0x7f3d7406e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d79780640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3d7415a810 0x7f3d7406e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46396/0 (socket says 172.21.15.37:46396) 2024-04-25T08:52:56.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d79780640 1 -- 172.21.15.37:0/1848394064 learned_addr learned my addr 172.21.15.37:0/1848394064 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:56.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d79f81640 1 --2- 172.21.15.37:0/1848394064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7415a430 0x7f3d740c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d79780640 1 -- 172.21.15.37:0/1848394064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7415a430 msgr2=0x7f3d740c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d79780640 1 --2- 172.21.15.37:0/1848394064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7415a430 0x7f3d740c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d79780640 1 -- 172.21.15.37:0/1848394064 >> v2:172.21.15.203:3300/0 conn(0x7f3d7406e6b0 msgr2=0x7f3d740cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d79780640 1 --2- 172.21.15.37:0/1848394064 >> v2:172.21.15.203:3300/0 conn(0x7f3d7406e6b0 0x7f3d740cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f3d79780640 1 -- 172.21.15.37:0/1848394064 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7415b340 con 0x7f3d7415a810 2024-04-25T08:52:56.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faac0e8e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0ce560 0x7faabc0ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faac0e8e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faabc163f50 0x7faabc164380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faac0e8e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faabc0ce2d0 con 0x7faabc0c6a50 2024-04-25T08:52:56.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faac0e8e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faabc0cef00 con 0x7faabc163f50 2024-04-25T08:52:56.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faac0e8e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0c6e30 con 0x7faabc0ce560 2024-04-25T08:52:56.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faaba59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7faabc0c6a50 0x7faabc0cdbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59866/0 (socket says 172.21.15.37:59866) 2024-04-25T08:52:56.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faaba59c640 1 -- 172.21.15.37:0/1969840906 learned_addr learned my addr 172.21.15.37:0/1969840906 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:56.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82dba8e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f82d40c2130 0x7f82d4166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82dba8e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f82d40cead0 0x7f82d40c2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82dba8e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d40c2a50 con 0x7f82d406e230 2024-04-25T08:52:56.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82dba8e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d41667d0 con 0x7f82d40c2130 2024-04-25T08:52:56.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82dba8e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d4165e00 con 0x7f82d40cead0 2024-04-25T08:52:56.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82d982a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f82d40c2130 0x7f82d4166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82d982a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f82d40c2130 0x7f82d4166290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46404/0 (socket says 172.21.15.37:46404) 2024-04-25T08:52:56.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82d982a640 1 -- 172.21.15.37:0/757062401 learned_addr learned my addr 172.21.15.37:0/757062401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:56.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711814711 0 0) 0x7faabc0ce2d0 con 0x7faabc0c6a50 2024-04-25T08:52:56.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82d9029640 1 --2- 172.21.15.37:0/757062401 >> v2:172.21.15.203:3300/0 conn(0x7f82d40cead0 0x7f82d40c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaa8003530 con 0x7faabc0c6a50 2024-04-25T08:52:56.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e8ff7640 1 -- 172.21.15.37:0/2973136350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e40c5950 msgr2=0x7f22e40cd0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e8ff7640 1 --2- 172.21.15.37:0/2973136350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e40c5950 0x7f22e40cd0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e8ff7640 1 -- 172.21.15.37:0/2973136350 >> v1:172.21.15.203:6789/0 conn(0x7f22e40cd600 legacy=0x7f22e40ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f22e8ff7640 1 -- 172.21.15.37:0/2973136350 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40c53f0 con 0x7f22e4159a80 2024-04-25T08:52:56.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1634768332 0 0) 0x7faaa8003530 con 0x7faabc0c6a50 2024-04-25T08:52:56.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0ce560 msgr2=0x7faabc0ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82d982a640 1 -- 172.21.15.37:0/757062401 >> v2:172.21.15.203:3300/0 conn(0x7f82d40cead0 msgr2=0x7f82d40c2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82d982a640 1 --2- 172.21.15.37:0/757062401 >> v2:172.21.15.203:3300/0 conn(0x7f82d40cead0 0x7f82d40c2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82d982a640 1 -- 172.21.15.37:0/757062401 >> v1:172.21.15.37:6789/0 conn(0x7f82d406e230 legacy=0x7f82d4162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7f82d982a640 1 -- 172.21.15.37:0/757062401 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d406eb80 con 0x7f82d40c2130 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab3fff640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0ce560 0x7faabc0ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0ce560 0x7faabc0ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 >> v2:172.21.15.203:3300/0 conn(0x7faabc163f50 msgr2=0x7faabc164380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faabad9d640 1 --2- 172.21.15.37:0/1969840906 >> v2:172.21.15.203:3300/0 conn(0x7faabc163f50 0x7faabc164380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 --2- 172.21.15.37:0/1969840906 >> v2:172.21.15.203:3300/0 conn(0x7faabc163f50 0x7faabc164380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc0d2630 con 0x7faabc0c6a50 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7faaac001680 con 0x7faabc0c6a50 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 >> v1:172.21.15.37:6789/0 conn(0x7faabc0c6a50 legacy=0x7faabc0cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0ce560 0x7faaa80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa80040d0 0x7faaa8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faaa8004da0 0x7faaa8005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0c6e30 con 0x7faaa80040d0 2024-04-25T08:52:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa800bcb0 con 0x7faaa8004da0 2024-04-25T08:52:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.976+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa800beb0 con 0x7faabc0ce560 2024-04-25T08:52:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7faabad9d640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faaa8004da0 0x7faaa8005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7faaba59c640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0ce560 0x7faaa80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7faab3fff640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa80040d0 0x7faaa8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4f620640 1 Processor -- start 2024-04-25T08:52:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4f620640 1 -- start start 2024-04-25T08:52:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe687288640 1 Processor -- start 2024-04-25T08:52:56.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7faabad9d640 1 -- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0ce560 msgr2=0x7faaa80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7faabad9d640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0ce560 0x7faaa80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7faabad9d640 1 -- 172.21.15.37:0/1969840906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa80040d0 msgr2=0x7faaa8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7faabad9d640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa80040d0 0x7faaa8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7faabad9d640 1 -- 172.21.15.37:0/1969840906 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc0d2630 con 0x7faaa8004da0 2024-04-25T08:52:56.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe687288640 1 -- start start 2024-04-25T08:52:56.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f99588e8640 1 Processor -- start 2024-04-25T08:52:56.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f99588e8640 1 -- start start 2024-04-25T08:52:56.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4f620640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efe48161d60 0x7efe480cc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4f620640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efe480c2840 0x7efe48071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f5643109640 1 Processor -- start 2024-04-25T08:52:56.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4f620640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4806e240 con 0x7efe480729e0 2024-04-25T08:52:56.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4f620640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4806e010 con 0x7efe480c2840 2024-04-25T08:52:56.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4f620640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe48162140 con 0x7efe48161d60 2024-04-25T08:52:56.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f22e8ff7640 1 --2- 172.21.15.37:0/2973136350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e4159a80 0x7f22e40aec30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009870 tx=0x7f22d8020f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a925812b15f9bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4d3bc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efe48161d60 0x7efe480cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4dbbd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efe480c2840 0x7efe48071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4d3bc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efe48161d60 0x7efe480cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:33970/0 (socket says 172.21.15.37:33970) 2024-04-25T08:52:56.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4d3bc640 1 -- 172.21.15.37:0/1245288741 learned_addr learned my addr 172.21.15.37:0/1245288741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:56.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4dbbd640 1 -- 172.21.15.37:0/1245288741 >> v2:172.21.15.203:3300/0 conn(0x7efe48161d60 msgr2=0x7efe480cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe687288640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6800cbc70 0x7fe68006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe685024640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe68015f750 0x7fe680162a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:36110/0 (socket says 172.21.15.37:36110) 2024-04-25T08:52:56.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe685024640 1 -- 172.21.15.37:0/3553165193 learned_addr learned my addr 172.21.15.37:0/3553165193 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:56.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f82d982a640 1 --2- 172.21.15.37:0/757062401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d40c2130 0x7f82d4166290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009e50 tx=0x7f82cc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cad0f16f8c0d95e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe685825640 1 --2- 172.21.15.37:0/3553165193 >> v2:172.21.15.37:3300/0 conn(0x7fe6800cbc70 0x7fe68006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4dbbd640 1 --2- 172.21.15.37:0/1245288741 >> v2:172.21.15.203:3300/0 conn(0x7efe48161d60 0x7efe480cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4dbbd640 1 -- 172.21.15.37:0/1245288741 >> v1:172.21.15.37:6789/0 conn(0x7efe480729e0 legacy=0x7efe480c2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4dbbd640 1 -- 172.21.15.37:0/1245288741 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4806e520 con 0x7efe480c2840 2024-04-25T08:52:56.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe687288640 1 -- 172.21.15.37:0/3553165193 --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6800cf060 con 0x7fe68006ddc0 2024-04-25T08:52:56.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe687288640 1 -- 172.21.15.37:0/3553165193 --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6800cf340 con 0x7fe68015f750 2024-04-25T08:52:56.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f5643109640 1 -- start start 2024-04-25T08:52:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe687288640 1 -- 172.21.15.37:0/3553165193 --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe68006d370 con 0x7fe6800cbc70 2024-04-25T08:52:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f22e37fe640 1 -- 172.21.15.37:0/2973136350 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80b7070 con 0x7f22e4159a80 2024-04-25T08:52:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f22e37fe640 1 -- 172.21.15.37:0/2973136350 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d80aac60 con 0x7f22e4159a80 2024-04-25T08:52:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f22e37fe640 1 -- 172.21.15.37:0/2973136350 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80b2660 con 0x7f22e4159a80 2024-04-25T08:52:56.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7efe4d3bc640 1 --2- 172.21.15.37:0/1245288741 >> v2:172.21.15.203:3300/0 conn(0x7efe48161d60 0x7efe480cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:56.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f82d8828640 1 -- 172.21.15.37:0/757062401 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0a56c0 con 0x7f82d40c2130 2024-04-25T08:52:56.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f82d8828640 1 -- 172.21.15.37:0/757062401 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc0a5860 con 0x7f82d40c2130 2024-04-25T08:52:56.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f82d8828640 1 -- 172.21.15.37:0/757062401 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc0a5c30 con 0x7f82d40c2130 2024-04-25T08:52:56.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f3d79780640 1 --2- 172.21.15.37:0/1848394064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d7415a810 0x7f3d7406e170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d740aeb00 tx=0x7f3d6401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40574761c2c2cf47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f99588e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f995415dc30 0x7f99540ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f99588e8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f99540ce890 0x7f99540cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f99588e8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995406dc70 con 0x7f995415d850 2024-04-25T08:52:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f99588e8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995406ed10 con 0x7f99540ce890 2024-04-25T08:52:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f99588e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995406df50 con 0x7f995415dc30 2024-04-25T08:52:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f995259c640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f995415d850 0x7f995406d370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:36126/0 (socket says 172.21.15.37:36126) 2024-04-25T08:52:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f995259c640 1 -- 172.21.15.37:0/1643541906 learned_addr learned my addr 172.21.15.37:0/1643541906 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f5643109640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15e170 0x7f563c0ae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f5643109640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f563c0cbf40 0x7f563c0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f5643109640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f563c0729a0 con 0x7f563c0b1810 2024-04-25T08:52:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f5643109640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f563c0cc8f0 con 0x7f563c0cbf40 2024-04-25T08:52:56.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f5643109640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0ae7d0 con 0x7f563c15e170 2024-04-25T08:52:56.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7f3d78f7f640 1 -- 172.21.15.37:0/1848394064 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d640b1070 con 0x7f3d7415a810 2024-04-25T08:52:56.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe685825640 1 -- 172.21.15.37:0/3553165193 >> v1:172.21.15.203:6789/0 conn(0x7fe68015f750 legacy=0x7fe680162a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe685825640 1 -- 172.21.15.37:0/3553165193 >> v1:172.21.15.37:6789/0 conn(0x7fe68006ddc0 legacy=0x7fe6800cb560 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:52:56.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.980+0000 7fe685825640 1 -- 172.21.15.37:0/3553165193 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68006d5c0 con 0x7fe6800cbc70 2024-04-25T08:52:56.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d78f7f640 1 -- 172.21.15.37:0/1848394064 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d640ac620 con 0x7f3d7415a810 2024-04-25T08:52:56.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f5640ea5640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f563c0b1810 0x7f563c15cdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:36130/0 (socket says 172.21.15.37:36130) 2024-04-25T08:52:56.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7faabad9d640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faaa8004da0 0x7faaa8005220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faab4007f60 tx=0x7faab401ce90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dafcc2df24cc2439 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f9952d9d640 1 --2- 172.21.15.37:0/1643541906 >> v2:172.21.15.203:3300/0 conn(0x7f99540ce890 0x7f99540cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d78f7f640 1 -- 172.21.15.37:0/1848394064 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d640b1070 con 0x7f3d7415a810 2024-04-25T08:52:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f5640ea5640 1 -- 172.21.15.37:0/1110959048 learned_addr learned my addr 172.21.15.37:0/1110959048 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f995159a640 1 -- 172.21.15.37:0/1643541906 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1356082873 0 0) 0x7f995406dc70 con 0x7f995415d850 2024-04-25T08:52:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2973136350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e4159a80 msgr2=0x7f22e40aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 --2- 172.21.15.37:0/2973136350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e4159a80 0x7f22e40aec30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f22d8009870 tx=0x7f22d8020f60 comp rx=0 tx=0).stop 2024-04-25T08:52:56.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f995159a640 1 -- 172.21.15.37:0/1643541906 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f993c003530 con 0x7f995415d850 2024-04-25T08:52:56.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2973136350 shutdown_connections 2024-04-25T08:52:56.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 --2- 172.21.15.37:0/2973136350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e40c5950 0x7f22e40cd0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 --2- 172.21.15.37:0/2973136350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e4159a80 0x7f22e40aec30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2973136350 >> 172.21.15.37:0/2973136350 conn(0x7f22e40b0df0 msgr2=0x7f22e40b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:56.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f82dba8e640 1 -- 172.21.15.37:0/757062401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d40c2130 msgr2=0x7f82d4166290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f82dba8e640 1 --2- 172.21.15.37:0/757062401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d40c2130 0x7f82d4166290 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f82cc009e50 tx=0x7f82cc01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2973136350 shutdown_connections 2024-04-25T08:52:56.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab40b2070 con 0x7faaa8004da0 2024-04-25T08:52:56.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab40a5d90 con 0x7faaa8004da0 2024-04-25T08:52:56.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7faab9d9b640 1 -- 172.21.15.37:0/1969840906 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4007b30 con 0x7faaa8004da0 2024-04-25T08:52:56.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563bfff640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15e170 0x7f563c0ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f56416a6640 1 --2- 172.21.15.37:0/1110959048 >> v2:172.21.15.203:3300/0 conn(0x7f563c0cbf40 0x7f563c0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018821470 0 0) 0x7f563c0729a0 con 0x7f563c0b1810 2024-04-25T08:52:56.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5624003530 con 0x7f563c0b1810 2024-04-25T08:52:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f9951d9b640 1 --2- 172.21.15.37:0/1643541906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f995415dc30 0x7f99540ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f9952d9d640 1 -- 172.21.15.37:0/1643541906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f995415dc30 msgr2=0x7f99540ce350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f9952d9d640 1 --2- 172.21.15.37:0/1643541906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f995415dc30 0x7f99540ce350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f9952d9d640 1 -- 172.21.15.37:0/1643541906 >> v1:172.21.15.203:6789/0 conn(0x7f995415d850 legacy=0x7f995406d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f9952d9d640 1 -- 172.21.15.37:0/1643541906 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995406e230 con 0x7f99540ce890 2024-04-25T08:52:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d7af83640 1 -- 172.21.15.37:0/1848394064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d7415a810 msgr2=0x7f3d7406e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d7af83640 1 --2- 172.21.15.37:0/1848394064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d7415a810 0x7f3d7406e170 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3d740aeb00 tx=0x7f3d6401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d7af83640 1 -- 172.21.15.37:0/1848394064 shutdown_connections 2024-04-25T08:52:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d7af83640 1 --2- 172.21.15.37:0/1848394064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d7415a810 0x7f3d7406e170 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d7af83640 1 --2- 172.21.15.37:0/1848394064 >> v2:172.21.15.203:3300/0 conn(0x7f3d7406e6b0 0x7f3d740cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d7af83640 1 --2- 172.21.15.37:0/1848394064 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7415a430 0x7f3d740c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d7af83640 1 -- 172.21.15.37:0/1848394064 >> 172.21.15.37:0/1848394064 conn(0x7f3d7406ac90 msgr2=0x7f3d7406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f9951d9b640 1 --2- 172.21.15.37:0/1643541906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f995415dc30 0x7f99540ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 <== mon.1 v1:172.21.15.203:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1567049877 0 0) 0x7f5624003530 con 0x7f563c0b1810 2024-04-25T08:52:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7faac0e8e640 1 -- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faaa8004da0 msgr2=0x7faaa8005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:56.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7faac0e8e640 1 --2- 172.21.15.37:0/1969840906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faaa8004da0 0x7faaa8005220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faab4007f60 tx=0x7faab401ce90 comp rx=0 tx=0).stop 2024-04-25T08:52:56.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7faabad9d640 1 -- 172.21.15.37:0/1969840906 reap_dead start 2024-04-25T08:52:56.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15e170 msgr2=0x7f563c0ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d7af83640 1 -- 172.21.15.37:0/1848394064 shutdown_connections 2024-04-25T08:52:56.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f82dba8e640 1 -- 172.21.15.37:0/757062401 shutdown_connections 2024-04-25T08:52:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f82dba8e640 1 --2- 172.21.15.37:0/757062401 >> v2:172.21.15.203:3300/0 conn(0x7f82d40cead0 0x7f82d40c2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f82dba8e640 1 --2- 172.21.15.37:0/757062401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d40c2130 0x7f82d4166290 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f82dba8e640 1 -- 172.21.15.37:0/757062401 >> 172.21.15.37:0/757062401 conn(0x7f82d40b0cf0 msgr2=0x7f82d40b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15e170 0x7f563c0ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 >> v2:172.21.15.203:3300/0 conn(0x7f563c0cbf40 msgr2=0x7f563c0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7faac0e8e640 1 -- 172.21.15.37:0/1969840906 shutdown_connections 2024-04-25T08:52:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7faac0e8e640 1 -- 172.21.15.37:0/1969840906 >> 172.21.15.37:0/1969840906 conn(0x7faabc08b000 msgr2=0x7faabc0d3500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 --2- 172.21.15.37:0/1110959048 >> v2:172.21.15.203:3300/0 conn(0x7f563c0cbf40 0x7f563c0cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 --> v1:172.21.15.203:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c06e240 con 0x7f563c0b1810 2024-04-25T08:52:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7efe4dbbd640 1 --2- 172.21.15.37:0/1245288741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe480c2840 0x7efe48071220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efe40009d90 tx=0x7efe4001c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed83d64a9ec7d6cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d7af83640 1 -- 172.21.15.37:0/1848394064 wait complete. 2024-04-25T08:52:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 <== mon.1 v1:172.21.15.203:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f5634001a00 con 0x7f563c0b1810 2024-04-25T08:52:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 >> v1:172.21.15.203:6789/0 conn(0x7f563c0b1810 legacy=0x7f563c15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56240039c0 0x7f5624003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7faac0e8e640 1 -- 172.21.15.37:0/1969840906 shutdown_connections 2024-04-25T08:52:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5624004810 0x7f5624004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7efe3ffff640 1 -- 172.21.15.37:0/1245288741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe400b2070 con 0x7efe480c2840 2024-04-25T08:52:57.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7efe3ffff640 1 -- 172.21.15.37:0/1245288741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe400a58f0 con 0x7efe480c2840 2024-04-25T08:52:57.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7efe3ffff640 1 -- 172.21.15.37:0/1245288741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe400a5cc0 con 0x7efe480c2840 2024-04-25T08:52:57.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15e170 0x7f56240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0cc8f0 con 0x7f5624004810 2024-04-25T08:52:57.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0ae7d0 con 0x7f56240039c0 2024-04-25T08:52:57.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562400b9d0 con 0x7f563c15e170 2024-04-25T08:52:57.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2973136350 wait complete. 2024-04-25T08:52:57.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 Processor -- start 2024-04-25T08:52:57.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f56416a6640 1 --2- 172.21.15.37:0/1110959048 >> v2:172.21.15.203:3300/0 conn(0x7f563c0cbf40 0x7f563c0cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:57.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 -- start start 2024-04-25T08:52:57.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7faac0e8e640 1 -- 172.21.15.37:0/1969840906 wait complete. 2024-04-25T08:52:57.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7fe685825640 1 --2- 172.21.15.37:0/3553165193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800cbc70 0x7fe68006e250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe68006d4f0 tx=0x7fe67801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fa0c477a91e260 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7efe4f620640 1 -- 172.21.15.37:0/1245288741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe480c2840 msgr2=0x7efe48071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7efe4f620640 1 --2- 172.21.15.37:0/1245288741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe480c2840 0x7efe48071220 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efe40009d90 tx=0x7efe4001c830 comp rx=0 tx=0).stop 2024-04-25T08:52:57.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7efe4f620640 1 -- 172.21.15.37:0/1245288741 shutdown_connections 2024-04-25T08:52:57.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7efe4f620640 1 --2- 172.21.15.37:0/1245288741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe480c2840 0x7efe48071220 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7efe4f620640 1 --2- 172.21.15.37:0/1245288741 >> v2:172.21.15.203:3300/0 conn(0x7efe48161d60 0x7efe480cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7efe4f620640 1 -- 172.21.15.37:0/1245288741 >> 172.21.15.37:0/1245288741 conn(0x7efe480b0cf0 msgr2=0x7efe480b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d7af83640 1 Processor -- start 2024-04-25T08:52:57.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f82dba8e640 1 -- 172.21.15.37:0/757062401 shutdown_connections 2024-04-25T08:52:57.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f82dba8e640 1 -- 172.21.15.37:0/757062401 wait complete. 2024-04-25T08:52:57.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e40c5950 0x7f22e41577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f22e9ff9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e40cd600 0x7f22e40b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f3d7af83640 1 -- start start 2024-04-25T08:52:57.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7fe677fff640 1 -- 172.21.15.37:0/3553165193 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6780b2070 con 0x7fe6800cbc70 2024-04-25T08:52:57.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7efe4f620640 1 -- 172.21.15.37:0/1245288741 shutdown_connections 2024-04-25T08:52:57.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e9ff9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e4159a80 0x7f22e4158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7fe677fff640 1 -- 172.21.15.37:0/3553165193 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6780a57d0 con 0x7fe6800cbc70 2024-04-25T08:52:57.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e9ff9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4159e60 con 0x7f22e40c5950 2024-04-25T08:52:57.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e9ff9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e40c56d0 con 0x7f22e40cd600 2024-04-25T08:52:57.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e9ff9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e4159fe0 con 0x7f22e4159a80 2024-04-25T08:52:57.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faac0e8e640 1 Processor -- start 2024-04-25T08:52:57.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faac0e8e640 1 -- start start 2024-04-25T08:52:57.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f5640ea5640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56240039c0 0x7f5624003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563bfff640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5624004810 0x7f5624004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563bfff640 1 -- 172.21.15.37:0/1110959048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15e170 msgr2=0x7f56240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:57.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563bfff640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15e170 0x7f56240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563bfff640 1 -- 172.21.15.37:0/1110959048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56240039c0 msgr2=0x7f5624003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:57.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563bfff640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56240039c0 0x7f5624003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f563bfff640 1 -- 172.21.15.37:0/1110959048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5624003530 con 0x7f5624004810 2024-04-25T08:52:57.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f5640ea5640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56240039c0 0x7f5624003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:57.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.984+0000 7f9952d9d640 1 --2- 172.21.15.37:0/1643541906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99540ce890 0x7f99540cc690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99440097c0 tx=0x7f994401c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=547f9f3ebb62a27c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f995159a640 1 -- 172.21.15.37:0/1643541906 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99440b0070 con 0x7f99540ce890 2024-04-25T08:52:57.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f995159a640 1 -- 172.21.15.37:0/1643541906 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99440ab600 con 0x7f99540ce890 2024-04-25T08:52:57.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f995159a640 1 -- 172.21.15.37:0/1643541906 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99440ab9b0 con 0x7f99540ce890 2024-04-25T08:52:57.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7fe677fff640 1 -- 172.21.15.37:0/3553165193 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6780b2070 con 0x7fe6800cbc70 2024-04-25T08:52:57.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e8ff7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e40c5950 0x7f22e41577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e8ff7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e40c5950 0x7f22e41577b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46460/0 (socket says 172.21.15.37:46460) 2024-04-25T08:52:57.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e8ff7640 1 -- 172.21.15.37:0/2872120327 learned_addr learned my addr 172.21.15.37:0/2872120327 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e8ff7640 1 -- 172.21.15.37:0/2872120327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e4159a80 msgr2=0x7f22e4158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e8ff7640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22e4159a80 0x7f22e4158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e8ff7640 1 -- 172.21.15.37:0/2872120327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e40cd600 msgr2=0x7f22e40b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e8ff7640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22e40cd600 0x7f22e40b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e8ff7640 1 -- 172.21.15.37:0/2872120327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e40cdb30 con 0x7f22e40c5950 2024-04-25T08:52:57.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e8ff7640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e40c5950 0x7f22e41577b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f22d801eb90 tx=0x7f22d801dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f99588e8640 1 -- 172.21.15.37:0/1643541906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99540ce890 msgr2=0x7f99540cc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f99588e8640 1 --2- 172.21.15.37:0/1643541906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99540ce890 0x7f99540cc690 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f99440097c0 tx=0x7f994401c4c0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7af83640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d7406e6b0 0x7f3d740af2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7af83640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7415a430 0x7f3d7409c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7af83640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7415f110 0x7f3d740ce240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7af83640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7415ee80 con 0x7f3d7406e6b0 2024-04-25T08:52:57.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7af83640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7415ed00 con 0x7f3d7415a430 2024-04-25T08:52:57.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7af83640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74164140 con 0x7f3d7415f110 2024-04-25T08:52:57.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faac0e8e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faaa8004da0 0x7faabc202c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faac0e8e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc16bed0 0x7faabc16c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faac0e8e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0b8540 0x7faabc0b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f82dba8e640 1 Processor -- start 2024-04-25T08:52:57.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7a782640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7415f110 0x7f3d740ce240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faaba59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faaa8004da0 0x7faabc202c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faaba59c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faaa8004da0 0x7faabc202c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:34012/0 (socket says 172.21.15.37:34012) 2024-04-25T08:52:57.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7efe4f620640 1 -- 172.21.15.37:0/1245288741 wait complete. 2024-04-25T08:52:57.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7a782640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7415f110 0x7f3d740ce240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:38988/0 (socket says 172.21.15.37:38988) 2024-04-25T08:52:57.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7a782640 1 -- 172.21.15.37:0/2158845777 learned_addr learned my addr 172.21.15.37:0/2158845777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faaba59c640 1 -- 172.21.15.37:0/347567888 learned_addr learned my addr 172.21.15.37:0/347567888 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0d23d0 con 0x7faabc16bed0 2024-04-25T08:52:57.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc0d2250 con 0x7faaa8004da0 2024-04-25T08:52:57.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d79f81640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d7406e6b0 0x7f3d740af2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7fe687288640 1 -- 172.21.15.37:0/3553165193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800cbc70 msgr2=0x7fe68006e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7fe687288640 1 --2- 172.21.15.37:0/3553165193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800cbc70 0x7fe68006e250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe68006d4f0 tx=0x7fe67801c910 comp rx=0 tx=0).stop 2024-04-25T08:52:57.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7fe687288640 1 -- 172.21.15.37:0/3553165193 shutdown_connections 2024-04-25T08:52:57.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7fe687288640 1 --2- 172.21.15.37:0/3553165193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6800cbc70 0x7fe68006e250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7fe687288640 1 -- 172.21.15.37:0/3553165193 >> 172.21.15.37:0/3553165193 conn(0x7fe68006f470 msgr2=0x7fe68006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d79780640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7415a430 0x7f3d7409c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faab3fff640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc16bed0 0x7faabc16c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7faabad9d640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0b8540 0x7faabc0b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7fe687288640 1 -- 172.21.15.37:0/3553165193 shutdown_connections 2024-04-25T08:52:57.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e17fa640 1 -- 172.21.15.37:0/2872120327 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80c0070 con 0x7f22e40c5950 2024-04-25T08:52:57.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e17fa640 1 -- 172.21.15.37:0/2872120327 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d80b3cf0 con 0x7f22e40c5950 2024-04-25T08:52:57.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2872120327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e40b50a0 con 0x7f22e40c5950 2024-04-25T08:52:57.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7a782640 1 -- 172.21.15.37:0/2158845777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7415a430 msgr2=0x7f3d7409c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7a782640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d7415a430 0x7f3d7409c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7a782640 1 -- 172.21.15.37:0/2158845777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d7406e6b0 msgr2=0x7f3d740af2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7a782640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d7406e6b0 0x7f3d740af2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7a782640 1 -- 172.21.15.37:0/2158845777 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d74140ee0 con 0x7f3d7415f110 2024-04-25T08:52:57.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f99588e8640 1 -- 172.21.15.37:0/1643541906 shutdown_connections 2024-04-25T08:52:57.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f99588e8640 1 --2- 172.21.15.37:0/1643541906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99540ce890 0x7f99540cc690 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f99588e8640 1 --2- 172.21.15.37:0/1643541906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f995415dc30 0x7f99540ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f99588e8640 1 -- 172.21.15.37:0/1643541906 >> 172.21.15.37:0/1643541906 conn(0x7f995406f470 msgr2=0x7f995406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2872120327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e4157cf0 con 0x7f22e40c5950 2024-04-25T08:52:57.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f22e17fa640 1 -- 172.21.15.37:0/2872120327 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80b7050 con 0x7f22e40c5950 2024-04-25T08:52:57.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f22ceffd640 1 -- 172.21.15.37:0/2872120327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f22bc001040 con 0x7f22e40c5950 2024-04-25T08:52:57.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f22e17fa640 1 -- 172.21.15.37:0/2872120327 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22d80c3070 con 0x7f22e40c5950 2024-04-25T08:52:57.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f22e17fa640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f22c8067030 0x7f22c80694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f22e17fa640 1 -- 172.21.15.37:0/2872120327 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f22d8122440 con 0x7f22e40c5950 2024-04-25T08:52:57.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f22e17fa640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x7f22c8070720 0x7f22c8072b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f99588e8640 1 -- 172.21.15.37:0/1643541906 shutdown_connections 2024-04-25T08:52:57.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f99588e8640 1 -- 172.21.15.37:0/1643541906 wait complete. 2024-04-25T08:52:57.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f99588e8640 1 Processor -- start 2024-04-25T08:52:57.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f99588e8640 1 -- start start 2024-04-25T08:52:57.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f99588e8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99540ce890 0x7f995415a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f99588e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f995415d850 0x7f9954153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f99588e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f995415dc30 0x7f99540b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f995259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99540ce890 0x7f995415a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f995259c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99540ce890 0x7f995415a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:34044/0 (socket says 172.21.15.37:34044) 2024-04-25T08:52:57.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7efe4f620640 1 Processor -- start 2024-04-25T08:52:57.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7efe4f620640 1 -- start start 2024-04-25T08:52:57.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4f620640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480729e0 0x7efe480c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4f620640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480c2840 0x7efe48154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4f620640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe48161d60 0x7efe48156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4f620640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe480720f0 con 0x7efe48161d60 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4f620640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe48071f70 con 0x7efe480c2840 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4f620640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe48072270 con 0x7efe480729e0 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4dbbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe48161d60 0x7efe48156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4dbbd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe48161d60 0x7efe48156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46496/0 (socket says 172.21.15.37:46496) 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4dbbd640 1 -- 172.21.15.37:0/1269432722 learned_addr learned my addr 172.21.15.37:0/1269432722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4cbbb640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480c2840 0x7efe48154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4d3bc640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480729e0 0x7efe480c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4dbbd640 1 -- 172.21.15.37:0/1269432722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480729e0 msgr2=0x7efe480c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4dbbd640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe480729e0 0x7efe480c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4dbbd640 1 -- 172.21.15.37:0/1269432722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480c2840 msgr2=0x7efe48154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4dbbd640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe480c2840 0x7efe48154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4dbbd640 1 -- 172.21.15.37:0/1269432722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe480c67a0 con 0x7efe48161d60 2024-04-25T08:52:57.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7a782640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7415f110 0x7f3d740ce240 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00b9a0 tx=0x7f3d6c00be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d6a7fc640 1 -- 172.21.15.37:0/2158845777 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c019070 con 0x7f3d7415f110 2024-04-25T08:52:57.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7af83640 1 -- 172.21.15.37:0/2158845777 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d74140360 con 0x7f3d7415f110 2024-04-25T08:52:57.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f3d7af83640 1 -- 172.21.15.37:0/2158845777 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d74140920 con 0x7f3d7415f110 2024-04-25T08:52:57.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f3d6a7fc640 1 -- 172.21.15.37:0/2158845777 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c00cbc0 con 0x7f3d7415f110 2024-04-25T08:52:57.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f3d6a7fc640 1 -- 172.21.15.37:0/2158845777 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c014440 con 0x7f3d7415f110 2024-04-25T08:52:57.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f3d7af83640 1 -- 172.21.15.37:0/2158845777 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d5c001040 con 0x7f3d7415f110 2024-04-25T08:52:57.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faabc16ca90 con 0x7faabc0b8540 2024-04-25T08:52:57.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7faab3fff640 1 -- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0b8540 msgr2=0x7faabc0b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7faab3fff640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faabc0b8540 0x7faabc0b89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7faab3fff640 1 -- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faaa8004da0 msgr2=0x7faabc202c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7faab3fff640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faaa8004da0 0x7faabc202c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7faab3fff640 1 -- 172.21.15.37:0/347567888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc0b8f00 con 0x7faabc16bed0 2024-04-25T08:52:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7faab3fff640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc16bed0 0x7faabc16c350 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faaa400f910 tx=0x7faaa400fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f82dba8e640 1 -- start start 2024-04-25T08:52:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f82dba8e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82d406e230 0x7f82d4159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f82dba8e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d40c2130 0x7f82d40712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f82dba8e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82d40cead0 0x7f82d4071800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f82dba8e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d416ed40 con 0x7f82d40c2130 2024-04-25T08:52:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f82dba8e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d416ebc0 con 0x7f82d406e230 2024-04-25T08:52:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f82dba8e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d416eec0 con 0x7f82d40cead0 2024-04-25T08:52:57.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f82d9029640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d40c2130 0x7f82d40712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f82d9029640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d40c2130 0x7f82d40712c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46484/0 (socket says 172.21.15.37:46484) 2024-04-25T08:52:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f82d9029640 1 -- 172.21.15.37:0/1923540866 learned_addr learned my addr 172.21.15.37:0/1923540866 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f82da02b640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82d40cead0 0x7f82d4071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f82d982a640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82d406e230 0x7f82d4159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f82d9029640 1 -- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82d40cead0 msgr2=0x7f82d4071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f82d9029640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82d40cead0 0x7f82d4071800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f82d9029640 1 -- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82d406e230 msgr2=0x7f82d4159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f82d9029640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82d406e230 0x7f82d4159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f82d9029640 1 -- 172.21.15.37:0/1923540866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d415b610 con 0x7f82d40c2130 2024-04-25T08:52:57.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f82d9029640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d40c2130 0x7f82d40712c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f82c400e930 tx=0x7f82c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f563bfff640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5624004810 0x7f5624004c90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f562c009ae0 tx=0x7f562c0252a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea2c71d26f4154b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0b9070 con 0x7f5624004810 2024-04-25T08:52:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562c0b4400 con 0x7f5624004810 2024-04-25T08:52:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7f563b7fe640 1 -- 172.21.15.37:0/1110959048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562c0096b0 con 0x7f5624004810 2024-04-25T08:52:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 -- 172.21.15.37:0/1110959048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5624004810 msgr2=0x7f5624004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 --2- 172.21.15.37:0/1110959048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5624004810 0x7f5624004c90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f562c009ae0 tx=0x7f562c0252a0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f56416a6640 1 -- 172.21.15.37:0/1110959048 reap_dead start 2024-04-25T08:52:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 -- 172.21.15.37:0/1110959048 shutdown_connections 2024-04-25T08:52:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 -- 172.21.15.37:0/1110959048 >> 172.21.15.37:0/1110959048 conn(0x7f563c0b0cf0 msgr2=0x7f563c0b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 -- 172.21.15.37:0/1110959048 shutdown_connections 2024-04-25T08:52:57.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 -- 172.21.15.37:0/1110959048 wait complete. 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7fe687288640 1 -- 172.21.15.37:0/3553165193 wait complete. 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.988+0000 7fe687288640 1 Processor -- start 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe687288640 1 -- start start 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe687288640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe68006ddc0 0x7fe68015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe687288640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe68015f750 0x7fe6800cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe687288640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe68015e300 0x7fe6800cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe687288640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68016fb70 con 0x7fe68015f750 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe687288640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6800cee40 con 0x7fe68006ddc0 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe687288640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68016fcf0 con 0x7fe68015e300 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe685024640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe68006ddc0 0x7fe68015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe685024640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe68006ddc0 0x7fe68015b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:34042/0 (socket says 172.21.15.37:34042) 2024-04-25T08:52:57.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe685024640 1 -- 172.21.15.37:0/958896829 learned_addr learned my addr 172.21.15.37:0/958896829 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe684823640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe68015f750 0x7fe6800cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe685825640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe68015e300 0x7fe6800cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe684823640 1 -- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe68015e300 msgr2=0x7fe6800cc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe684823640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe68015e300 0x7fe6800cc5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe684823640 1 -- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe68006ddc0 msgr2=0x7fe68015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe684823640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe68006ddc0 0x7fe68015b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe684823640 1 -- 172.21.15.37:0/958896829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68015d260 con 0x7fe68015f750 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe684823640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe68015f750 0x7fe6800cc060 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe67000b3f0 tx=0x7fe67000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f99588e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540cd5f0 con 0x7f995415dc30 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f99588e8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995415e010 con 0x7f99540ce890 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f99588e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99540cd770 con 0x7f995415d850 2024-04-25T08:52:57.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f995259c640 1 -- 172.21.15.37:0/2365196505 learned_addr learned my addr 172.21.15.37:0/2365196505 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4dbbd640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe48161d60 0x7efe48156220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe4001a460 tx=0x7efe400195d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 Processor -- start 2024-04-25T08:52:57.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f22e17fa640 1 -- 172.21.15.37:0/2872120327 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f22d80b1ec0 con 0x7f22c8070720 2024-04-25T08:52:57.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 -- start start 2024-04-25T08:52:57.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f9952d9d640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f995415dc30 0x7f99540b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f22e97f8640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x7f22c8070720 0x7f22c8072b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f22e3fff640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f22c8067030 0x7f22c80694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe3dffb640 1 -- 172.21.15.37:0/1269432722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe400bb020 con 0x7efe48161d60 2024-04-25T08:52:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f9951d9b640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f995415d850 0x7f9954153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f995259c640 1 -- 172.21.15.37:0/2365196505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f995415d850 msgr2=0x7f9954153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f995259c640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f995415d850 0x7f9954153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f995259c640 1 -- 172.21.15.37:0/2365196505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f995415dc30 msgr2=0x7f99540b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f995259c640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f995415dc30 0x7f99540b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f995259c640 1 -- 172.21.15.37:0/2365196505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99540ca500 con 0x7f99540ce890 2024-04-25T08:52:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4f620640 1 -- 172.21.15.37:0/1269432722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe480c6a80 con 0x7efe48161d60 2024-04-25T08:52:57.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7efe4f620640 1 -- 172.21.15.37:0/1269432722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe480c7040 con 0x7efe48161d60 2024-04-25T08:52:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f82ca7fc640 1 -- 172.21.15.37:0/1923540866 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4017070 con 0x7f82d40c2130 2024-04-25T08:52:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f82dba8e640 1 -- 172.21.15.37:0/1923540866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d415b8f0 con 0x7f82d40c2130 2024-04-25T08:52:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f82dba8e640 1 -- 172.21.15.37:0/1923540866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d415beb0 con 0x7f82d40c2130 2024-04-25T08:52:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe675ffb640 1 -- 172.21.15.37:0/958896829 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670015020 con 0x7fe68015f750 2024-04-25T08:52:57.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f22e97f8640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x7f22c8070720 0x7f22c8072b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15c490 0x7f563c0ad150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f563c0ad690 0x7f563c1f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f563c1f2fd0 0x7f563c0b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0c9b30 con 0x7f563c1f2fd0 2024-04-25T08:52:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0c9850 con 0x7f563c0ad690 2024-04-25T08:52:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f5643109640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563c0c9e10 con 0x7f563c15c490 2024-04-25T08:52:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe687288640 1 -- 172.21.15.37:0/958896829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68015d540 con 0x7fe68015f750 2024-04-25T08:52:57.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7fe687288640 1 -- 172.21.15.37:0/958896829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe68015db00 con 0x7fe68015f750 2024-04-25T08:52:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f995259c640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99540ce890 0x7f995415a250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f994c00cbc0 tx=0x7f994c00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f82ca7fc640 1 -- 172.21.15.37:0/1923540866 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c400ca00 con 0x7f82d40c2130 2024-04-25T08:52:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f82ca7fc640 1 -- 172.21.15.37:0/1923540866 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400cdb0 con 0x7f82d40c2130 2024-04-25T08:52:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5640ea5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15c490 0x7f563c0ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f82dba8e640 1 -- 172.21.15.37:0/1923540866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82d415b0b0 con 0x7f82d40c2130 2024-04-25T08:52:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5640ea5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15c490 0x7f563c0ad150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:39048/0 (socket says 172.21.15.37:39048) 2024-04-25T08:52:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5640ea5640 1 -- 172.21.15.37:0/1880971172 learned_addr learned my addr 172.21.15.37:0/1880971172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f563bfff640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f563c0ad690 0x7f563c1f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f9942ffd640 1 -- 172.21.15.37:0/2365196505 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c017070 con 0x7f99540ce890 2024-04-25T08:52:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f9942ffd640 1 -- 172.21.15.37:0/2365196505 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c00ad40 con 0x7f99540ce890 2024-04-25T08:52:57.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f99588e8640 1 -- 172.21.15.37:0/2365196505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99540ca7e0 con 0x7f99540ce890 2024-04-25T08:52:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.992+0000 7f3d6a7fc640 1 -- 172.21.15.37:0/2158845777 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d6c01c070 con 0x7f3d7415f110 2024-04-25T08:52:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f3d6a7fc640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d58066f60 0x7f3d58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f3d6a7fc640 1 -- 172.21.15.37:0/2158845777 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3d6c085800 con 0x7f3d7415f110 2024-04-25T08:52:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f3d6a7fc640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x7f3d580706e0 0x7f3d58072b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f3d6a7fc640 1 -- 172.21.15.37:0/2158845777 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d6c022ec0 con 0x7f3d580706e0 2024-04-25T08:52:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f3d6a7fc640 1 -- 172.21.15.37:0/2158845777 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d6c085bf0 con 0x7f3d7415f110 2024-04-25T08:52:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7efe4f620640 1 -- 172.21.15.37:0/1269432722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe4804ef30 con 0x7efe48161d60 2024-04-25T08:52:57.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7efe3dffb640 1 -- 172.21.15.37:0/1269432722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe400a5830 con 0x7efe48161d60 2024-04-25T08:52:57.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7efe3dffb640 1 -- 172.21.15.37:0/1269432722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe400b2040 con 0x7efe48161d60 2024-04-25T08:52:57.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7fe675ffb640 1 -- 172.21.15.37:0/958896829 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe670011830 con 0x7fe68015f750 2024-04-25T08:52:57.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7fe675ffb640 1 -- 172.21.15.37:0/958896829 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe670011be0 con 0x7fe68015f750 2024-04-25T08:52:57.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7faab2ffd640 1 -- 172.21.15.37:0/347567888 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa400dbb0 con 0x7faabc16bed0 2024-04-25T08:52:57.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faabc0b72e0 con 0x7faabc16bed0 2024-04-25T08:52:57.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc0b78a0 con 0x7faabc16bed0 2024-04-25T08:52:57.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f99588e8640 1 -- 172.21.15.37:0/2365196505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99540cada0 con 0x7f99540ce890 2024-04-25T08:52:57.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f99588e8640 1 -- 172.21.15.37:0/2365196505 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f995404ef30 con 0x7f99540ce890 2024-04-25T08:52:57.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f9942ffd640 1 -- 172.21.15.37:0/2365196505 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c012600 con 0x7f99540ce890 2024-04-25T08:52:57.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f3d79780640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x7f3d580706e0 0x7f3d58072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7faab2ffd640 1 -- 172.21.15.37:0/347567888 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa400dd50 con 0x7faabc16bed0 2024-04-25T08:52:57.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faabc04ef30 con 0x7faabc16bed0 2024-04-25T08:52:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5640ea5640 1 -- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f563c0ad690 msgr2=0x7f563c1f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5640ea5640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f563c0ad690 0x7f563c1f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5640ea5640 1 -- 172.21.15.37:0/1880971172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f563c1f2fd0 msgr2=0x7f563c0b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5640ea5640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f563c1f2fd0 0x7f563c0b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5640ea5640 1 -- 172.21.15.37:0/1880971172 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f563c1ef4a0 con 0x7f563c15c490 2024-04-25T08:52:57.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f3d79780640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x7f3d580706e0 0x7f3d58072b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f22e17fa640 1 -- 172.21.15.37:0/2872120327 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22d8129050 con 0x7f22e40c5950 2024-04-25T08:52:57.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f22e17fa640 1 -- 172.21.15.37:0/2872120327 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f22d80b1ec0 con 0x7f22c8070720 2024-04-25T08:52:57.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5640ea5640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15c490 0x7f563c0ad150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5634001b70 tx=0x7f56340073d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f3d79f81640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d58066f60 0x7f3d58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f22e3fff640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f22c8067030 0x7f22c80694f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f22e406e890 tx=0x7f22d000c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7faab2ffd640 1 -- 172.21.15.37:0/347567888 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4013680 con 0x7faabc16bed0 2024-04-25T08:52:57.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7fe687288640 1 -- 172.21.15.37:0/958896829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe660001040 con 0x7fe68015f750 2024-04-25T08:52:57.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f3d79f81640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d58066f60 0x7f3d58069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3d700079c0 tx=0x7f3d70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f56397fa640 1 -- 172.21.15.37:0/1880971172 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634007ae0 con 0x7f563c15c490 2024-04-25T08:52:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5643109640 1 -- 172.21.15.37:0/1880971172 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563c1f3780 con 0x7f563c15c490 2024-04-25T08:52:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5643109640 1 -- 172.21.15.37:0/1880971172 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563c1efa00 con 0x7f563c15c490 2024-04-25T08:52:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f5643109640 1 -- 172.21.15.37:0/1880971172 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f563c04ef30 con 0x7f563c15c490 2024-04-25T08:52:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f9942ffd640 1 -- 172.21.15.37:0/2365196505 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f994c01a070 con 0x7f99540ce890 2024-04-25T08:52:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f9942ffd640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9934066fb0 0x7f9934069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f9942ffd640 1 -- 172.21.15.37:0/2365196505 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f994c085950 con 0x7f99540ce890 2024-04-25T08:52:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f9942ffd640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x7f9934070730 0x7f9934072b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f9942ffd640 1 -- 172.21.15.37:0/2365196505 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f994c00de50 con 0x7f9934070730 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f9942ffd640 1 -- 172.21.15.37:0/2365196505 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f994c085d40 con 0x7f99540ce890 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f9952d9d640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x7f9934070730 0x7f9934072b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f9951d9b640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9934066fb0 0x7f9934069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f9952d9d640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x7f9934070730 0x7f9934072b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f9942ffd640 1 -- 172.21.15.37:0/2365196505 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f994c00de50 con 0x7f9934070730 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7efe3dffb640 1 -- 172.21.15.37:0/1269432722 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe400be020 con 0x7efe48161d60 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7efe3dffb640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe2c066f60 0x7efe2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7efe3dffb640 1 -- 172.21.15.37:0/1269432722 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7efe401202c0 con 0x7efe48161d60 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7efe3dffb640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x7efe2c070650 0x7efe2c072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7efe3dffb640 1 -- 172.21.15.37:0/1269432722 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe2c073160 con 0x7efe2c070650 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7efe3dffb640 1 -- 172.21.15.37:0/1269432722 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe40120750 con 0x7efe48161d60 2024-04-25T08:52:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7efe4cbbb640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x7efe2c070650 0x7efe2c072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7efe4d3bc640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe2c066f60 0x7efe2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7efe4cbbb640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x7efe2c070650 0x7efe2c072a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f3d6a7fc640 1 -- 172.21.15.37:0/2158845777 <== osd.3 v2:172.21.15.37:6800/4126478460 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3d6c022ec0 con 0x7f3d580706e0 2024-04-25T08:52:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7faab2ffd640 1 -- 172.21.15.37:0/347567888 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faaa4013820 con 0x7faabc16bed0 2024-04-25T08:52:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7faab2ffd640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faa9c067060 0x7faa9c069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7faab2ffd640 1 -- 172.21.15.37:0/347567888 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7faaa4086220 con 0x7faabc16bed0 2024-04-25T08:52:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7faab2ffd640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x7faa9c0707d0 0x7faa9c072c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7faaba59c640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faa9c067060 0x7faa9c069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7faab2ffd640 1 -- 172.21.15.37:0/347567888 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faaa4007e50 con 0x7faa9c0707d0 2024-04-25T08:52:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7faab2ffd640 1 -- 172.21.15.37:0/347567888 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaa4086610 con 0x7faabc16bed0 2024-04-25T08:52:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7faabad9d640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x7faa9c0707d0 0x7faa9c072c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7faaba59c640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faa9c067060 0x7faa9c069520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faaac007fd0 tx=0x7faaac007980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7faabad9d640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x7faa9c0707d0 0x7faa9c072c10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f82ca7fc640 1 -- 172.21.15.37:0/1923540866 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82c401a070 con 0x7f82d40c2130 2024-04-25T08:52:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f82ca7fc640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f82b8066f90 0x7f82b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f82ca7fc640 1 -- 172.21.15.37:0/1923540866 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f82c4085b40 con 0x7f82d40c2130 2024-04-25T08:52:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f82ca7fc640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x7f82b8070690 0x7f82b8072ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f82ca7fc640 1 -- 172.21.15.37:0/1923540866 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f82b80731a0 con 0x7f82b8070690 2024-04-25T08:52:57.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f82d982a640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f82b8066f90 0x7f82b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f82da02b640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x7f82b8070690 0x7f82b8072ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f82ca7fc640 1 -- 172.21.15.37:0/1923540866 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f82c4052460 con 0x7f82d40c2130 2024-04-25T08:52:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f82da02b640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x7f82b8070690 0x7f82b8072ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f82d982a640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f82b8066f90 0x7f82b8069450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0ae000 tx=0x7f82cc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:56.996+0000 7f56397fa640 1 -- 172.21.15.37:0/1880971172 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5634018070 con 0x7f563c15c490 2024-04-25T08:52:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f56397fa640 1 -- 172.21.15.37:0/1880971172 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634012780 con 0x7f563c15c490 2024-04-25T08:52:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7fe675ffb640 1 -- 172.21.15.37:0/958896829 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe670018020 con 0x7fe68015f750 2024-04-25T08:52:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7fe675ffb640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe664066f60 0x7fe664069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7fe675ffb640 1 -- 172.21.15.37:0/958896829 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe670085260 con 0x7fe68015f750 2024-04-25T08:52:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7fe675ffb640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x7fe664070650 0x7fe664072a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7fe675ffb640 1 -- 172.21.15.37:0/958896829 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe664073160 con 0x7fe664070650 2024-04-25T08:52:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7fe675ffb640 1 -- 172.21.15.37:0/958896829 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe67008c050 con 0x7fe68015f750 2024-04-25T08:52:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7fe685024640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe664066f60 0x7fe664069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7fe685825640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x7fe664070650 0x7fe664072a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7fe685825640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x7fe664070650 0x7fe664072a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7fe685024640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe664066f60 0x7fe664069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe67c005890 tx=0x7fe67c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f9951d9b640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9934066fb0 0x7f9934069470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9948006fd0 tx=0x7f9948008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f56397fa640 1 -- 172.21.15.37:0/1880971172 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5634012920 con 0x7f563c15c490 2024-04-25T08:52:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f56397fa640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5628066f60 0x7f5628069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f56397fa640 1 -- 172.21.15.37:0/1880971172 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5634086830 con 0x7f563c15c490 2024-04-25T08:52:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f56397fa640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x7f56280706e0 0x7f5628072b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f56397fa640 1 -- 172.21.15.37:0/1880971172 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f56280731f0 con 0x7f56280706e0 2024-04-25T08:52:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7f56397fa640 1 -- 172.21.15.37:0/1880971172 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5634086c20 con 0x7f563c15c490 2024-04-25T08:52:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.000+0000 7efe3dffb640 1 -- 172.21.15.37:0/1269432722 <== osd.2 v2:172.21.15.37:6808/332967247 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7efe2c073160 con 0x7efe2c070650 2024-04-25T08:52:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.004+0000 7efe4d3bc640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe2c066f60 0x7efe2c069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efe44006cb0 tx=0x7efe44009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.004+0000 7f563bfff640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5628066f60 0x7f5628069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.004+0000 7f56416a6640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x7f56280706e0 0x7f5628072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.004+0000 7f563bfff640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5628066f60 0x7f5628069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f562c0b6000 tx=0x7f562c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.004+0000 7f56416a6640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x7f56280706e0 0x7f5628072b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.004+0000 7faab2ffd640 1 -- 172.21.15.37:0/347567888 <== osd.4 v2:172.21.15.203:6802/714177927 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7faaa4007e50 con 0x7faa9c0707d0 2024-04-25T08:52:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.004+0000 7f82ca7fc640 1 -- 172.21.15.37:0/1923540866 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f82b80731a0 con 0x7f82b8070690 2024-04-25T08:52:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.004+0000 7f56397fa640 1 -- 172.21.15.37:0/1880971172 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f56280731f0 con 0x7f56280706e0 2024-04-25T08:52:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.020+0000 7fe675ffb640 1 -- 172.21.15.37:0/958896829 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fe664073160 con 0x7fe664070650 2024-04-25T08:52:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f5643109640 1 -- 172.21.15.37:0/1880971172 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f563c0ceed0 con 0x7f56280706e0 2024-04-25T08:52:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f56397fa640 1 -- 172.21.15.37:0/1880971172 <== osd.6 v2:172.21.15.203:6800/467003745 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f563c0ceed0 con 0x7f56280706e0 2024-04-25T08:52:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f5622ffd640 1 -- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x7f56280706e0 msgr2=0x7f5628072b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f5622ffd640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x7f56280706e0 0x7f5628072b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f5622ffd640 1 -- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5628066f60 msgr2=0x7f5628069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f5622ffd640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5628066f60 0x7f5628069420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f562c0b6000 tx=0x7f562c008040 comp rx=0 tx=0).stop 2024-04-25T08:52:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f5622ffd640 1 -- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15c490 msgr2=0x7f563c0ad150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f5622ffd640 1 --2- 172.21.15.37:0/1880971172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f563c15c490 0x7f563c0ad150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5634001b70 tx=0x7f56340073d0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f56416a6640 1 -- 172.21.15.37:0/1880971172 reap_dead start 2024-04-25T08:52:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f5622ffd640 1 -- 172.21.15.37:0/1880971172 shutdown_connections 2024-04-25T08:52:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f5622ffd640 1 -- 172.21.15.37:0/1880971172 >> 172.21.15.37:0/1880971172 conn(0x7f563c0b0cf0 msgr2=0x7f563c0ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f5622ffd640 1 -- 172.21.15.37:0/1880971172 shutdown_connections 2024-04-25T08:52:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.024+0000 7f5622ffd640 1 -- 172.21.15.37:0/1880971172 wait complete. 2024-04-25T08:52:57.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22ceffd640 1 -- 172.21.15.37:0/2872120327 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f22bc002de0 con 0x7f22c8070720 2024-04-25T08:52:57.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e17fa640 1 -- 172.21.15.37:0/2872120327 <== osd.0 v2:172.21.15.37:6824/1116141266 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f22bc002de0 con 0x7f22c8070720 2024-04-25T08:52:57.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f3d7af83640 1 -- 172.21.15.37:0/2158845777 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d5c002d60 con 0x7f3d580706e0 2024-04-25T08:52:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2872120327 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x7f22c8070720 msgr2=0x7f22c8072b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e9ff9640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x7f22c8070720 0x7f22c8072b60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2872120327 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f22c8067030 msgr2=0x7f22c80694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e9ff9640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f22c8067030 0x7f22c80694f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f22e406e890 tx=0x7f22d000c040 comp rx=0 tx=0).stop 2024-04-25T08:52:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2872120327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e40c5950 msgr2=0x7f22e41577b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e9ff9640 1 --2- 172.21.15.37:0/2872120327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22e40c5950 0x7f22e41577b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f22d801eb90 tx=0x7f22d801dd00 comp rx=0 tx=0).stop 2024-04-25T08:52:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e97f8640 1 -- 172.21.15.37:0/2872120327 reap_dead start 2024-04-25T08:52:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2872120327 shutdown_connections 2024-04-25T08:52:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2872120327 >> 172.21.15.37:0/2872120327 conn(0x7f22e40b0df0 msgr2=0x7f22e40b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2872120327 shutdown_connections 2024-04-25T08:52:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.028+0000 7f22e9ff9640 1 -- 172.21.15.37:0/2872120327 wait complete. 2024-04-25T08:52:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d6a7fc640 1 -- 172.21.15.37:0/2158845777 <== osd.3 v2:172.21.15.37:6800/4126478460 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3d5c002d60 con 0x7f3d580706e0 2024-04-25T08:52:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7efe4f620640 1 -- 172.21.15.37:0/1269432722 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efe480afd70 con 0x7efe2c070650 2024-04-25T08:52:57.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d7af83640 1 -- 172.21.15.37:0/2158845777 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x7f3d580706e0 msgr2=0x7f3d58072b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d7af83640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x7f3d580706e0 0x7f3d58072b20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d7af83640 1 -- 172.21.15.37:0/2158845777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d58066f60 msgr2=0x7f3d58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d7af83640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d58066f60 0x7f3d58069420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3d700079c0 tx=0x7f3d70008040 comp rx=0 tx=0).stop 2024-04-25T08:52:57.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d7af83640 1 -- 172.21.15.37:0/2158845777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7415f110 msgr2=0x7f3d740ce240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d7af83640 1 --2- 172.21.15.37:0/2158845777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d7415f110 0x7f3d740ce240 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00b9a0 tx=0x7f3d6c00be60 comp rx=0 tx=0).stop 2024-04-25T08:52:57.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d7a782640 1 -- 172.21.15.37:0/2158845777 reap_dead start 2024-04-25T08:52:57.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d7af83640 1 -- 172.21.15.37:0/2158845777 shutdown_connections 2024-04-25T08:52:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d7af83640 1 -- 172.21.15.37:0/2158845777 >> 172.21.15.37:0/2158845777 conn(0x7f3d7406ac90 msgr2=0x7f3d7415c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d7af83640 1 -- 172.21.15.37:0/2158845777 shutdown_connections 2024-04-25T08:52:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f3d7af83640 1 -- 172.21.15.37:0/2158845777 wait complete. 2024-04-25T08:52:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faabc0b5ed0 con 0x7faa9c0707d0 2024-04-25T08:52:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faab2ffd640 1 -- 172.21.15.37:0/347567888 <== osd.4 v2:172.21.15.203:6802/714177927 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faabc0b5ed0 con 0x7faa9c0707d0 2024-04-25T08:52:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x7faa9c0707d0 msgr2=0x7faa9c072c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faac0e8e640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x7faa9c0707d0 0x7faa9c072c10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faa9c067060 msgr2=0x7faa9c069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faac0e8e640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faa9c067060 0x7faa9c069520 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7faaac007fd0 tx=0x7faaac007980 comp rx=0 tx=0).stop 2024-04-25T08:52:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc16bed0 msgr2=0x7faabc16c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faac0e8e640 1 --2- 172.21.15.37:0/347567888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faabc16bed0 0x7faabc16c350 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7faaa400f910 tx=0x7faaa400fdd0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faabad9d640 1 -- 172.21.15.37:0/347567888 reap_dead start 2024-04-25T08:52:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 shutdown_connections 2024-04-25T08:52:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 >> 172.21.15.37:0/347567888 conn(0x7faabc08b000 msgr2=0x7faabc166ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 shutdown_connections 2024-04-25T08:52:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7faac0e8e640 1 -- 172.21.15.37:0/347567888 wait complete. 2024-04-25T08:52:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.032+0000 7f82dba8e640 1 -- 172.21.15.37:0/1923540866 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f82d415c620 con 0x7f82b8070690 2024-04-25T08:52:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7f99588e8640 1 -- 172.21.15.37:0/2365196505 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f99540b32e0 con 0x7f9934070730 2024-04-25T08:52:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe3dffb640 1 -- 172.21.15.37:0/1269432722 <== osd.2 v2:172.21.15.37:6808/332967247 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7efe480afd70 con 0x7efe2c070650 2024-04-25T08:52:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7f82ca7fc640 1 -- 172.21.15.37:0/1923540866 <== osd.5 v2:172.21.15.203:6808/213476662 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f82d415c620 con 0x7f82b8070690 2024-04-25T08:52:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7f9942ffd640 1 -- 172.21.15.37:0/2365196505 <== osd.1 v2:172.21.15.37:6816/2944303862 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f99540b32e0 con 0x7f9934070730 2024-04-25T08:52:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe4f620640 1 -- 172.21.15.37:0/1269432722 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x7efe2c070650 msgr2=0x7efe2c072a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe4f620640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x7efe2c070650 0x7efe2c072a90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe4f620640 1 -- 172.21.15.37:0/1269432722 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe2c066f60 msgr2=0x7efe2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe4f620640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe2c066f60 0x7efe2c069420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7efe44006cb0 tx=0x7efe44009240 comp rx=0 tx=0).stop 2024-04-25T08:52:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe4f620640 1 -- 172.21.15.37:0/1269432722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe48161d60 msgr2=0x7efe48156220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe4f620640 1 --2- 172.21.15.37:0/1269432722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe48161d60 0x7efe48156220 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efe4001a460 tx=0x7efe400195d0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe4dbbd640 1 -- 172.21.15.37:0/1269432722 reap_dead start 2024-04-25T08:52:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe4f620640 1 -- 172.21.15.37:0/1269432722 shutdown_connections 2024-04-25T08:52:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe4f620640 1 -- 172.21.15.37:0/1269432722 >> 172.21.15.37:0/1269432722 conn(0x7efe480b0cf0 msgr2=0x7efe48160c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe4f620640 1 -- 172.21.15.37:0/1269432722 shutdown_connections 2024-04-25T08:52:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.036+0000 7efe4f620640 1 -- 172.21.15.37:0/1269432722 wait complete. 2024-04-25T08:52:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f82dba8e640 1 -- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x7f82b8070690 msgr2=0x7f82b8072ad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f82dba8e640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x7f82b8070690 0x7f82b8072ad0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f82dba8e640 1 -- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f82b8066f90 msgr2=0x7f82b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f82dba8e640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f82b8066f90 0x7f82b8069450 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f82cc0ae000 tx=0x7f82cc0a7070 comp rx=0 tx=0).stop 2024-04-25T08:52:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f82dba8e640 1 -- 172.21.15.37:0/1923540866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d40c2130 msgr2=0x7f82d40712c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f82dba8e640 1 --2- 172.21.15.37:0/1923540866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82d40c2130 0x7f82d40712c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f82c400e930 tx=0x7f82c400edf0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f82da02b640 1 -- 172.21.15.37:0/1923540866 reap_dead start 2024-04-25T08:52:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f9940ff9640 1 -- 172.21.15.37:0/2365196505 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x7f9934070730 msgr2=0x7f9934072b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f9940ff9640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x7f9934070730 0x7f9934072b70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f9940ff9640 1 -- 172.21.15.37:0/2365196505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9934066fb0 msgr2=0x7f9934069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f9940ff9640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9934066fb0 0x7f9934069470 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9948006fd0 tx=0x7f9948008040 comp rx=0 tx=0).stop 2024-04-25T08:52:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f9940ff9640 1 -- 172.21.15.37:0/2365196505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99540ce890 msgr2=0x7f995415a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f9940ff9640 1 --2- 172.21.15.37:0/2365196505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99540ce890 0x7f995415a250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f994c00cbc0 tx=0x7f994c00a430 comp rx=0 tx=0).stop 2024-04-25T08:52:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f9952d9d640 1 -- 172.21.15.37:0/2365196505 reap_dead start 2024-04-25T08:52:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f82dba8e640 1 -- 172.21.15.37:0/1923540866 shutdown_connections 2024-04-25T08:52:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f82dba8e640 1 -- 172.21.15.37:0/1923540866 >> 172.21.15.37:0/1923540866 conn(0x7f82d40b0cf0 msgr2=0x7f82d40cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f9940ff9640 1 -- 172.21.15.37:0/2365196505 shutdown_connections 2024-04-25T08:52:57.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f9940ff9640 1 -- 172.21.15.37:0/2365196505 >> 172.21.15.37:0/2365196505 conn(0x7f995406f470 msgr2=0x7f995406d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f9940ff9640 1 -- 172.21.15.37:0/2365196505 shutdown_connections 2024-04-25T08:52:57.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.040+0000 7f9940ff9640 1 -- 172.21.15.37:0/2365196505 wait complete. 2024-04-25T08:52:57.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.044+0000 7f82dba8e640 1 -- 172.21.15.37:0/1923540866 shutdown_connections 2024-04-25T08:52:57.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.044+0000 7f82dba8e640 1 -- 172.21.15.37:0/1923540866 wait complete. 2024-04-25T08:52:57.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.044+0000 7fe687288640 1 -- 172.21.15.37:0/958896829 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe660002de0 con 0x7fe664070650 2024-04-25T08:52:57.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.044+0000 7fe675ffb640 1 -- 172.21.15.37:0/958896829 <== osd.7 v2:172.21.15.203:6824/2957998038 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe660002de0 con 0x7fe664070650 2024-04-25T08:52:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.048+0000 7fe687288640 1 -- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x7fe664070650 msgr2=0x7fe664072a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.048+0000 7fe687288640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x7fe664070650 0x7fe664072a90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.048+0000 7fe687288640 1 -- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe664066f60 msgr2=0x7fe664069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.048+0000 7fe687288640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe664066f60 0x7fe664069420 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe67c005890 tx=0x7fe67c009040 comp rx=0 tx=0).stop 2024-04-25T08:52:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.048+0000 7fe687288640 1 -- 172.21.15.37:0/958896829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe68015f750 msgr2=0x7fe6800cc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.048+0000 7fe687288640 1 --2- 172.21.15.37:0/958896829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe68015f750 0x7fe6800cc060 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe67000b3f0 tx=0x7fe67000b8b0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.048+0000 7fe685825640 1 -- 172.21.15.37:0/958896829 reap_dead start 2024-04-25T08:52:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.048+0000 7fe687288640 1 -- 172.21.15.37:0/958896829 shutdown_connections 2024-04-25T08:52:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.048+0000 7fe687288640 1 -- 172.21.15.37:0/958896829 >> 172.21.15.37:0/958896829 conn(0x7fe68006f470 msgr2=0x7fe6801578c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.048+0000 7fe687288640 1 -- 172.21.15.37:0/958896829 shutdown_connections 2024-04-25T08:52:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.048+0000 7fe687288640 1 -- 172.21.15.37:0/958896829 wait complete. 2024-04-25T08:52:57.070 INFO:teuthology.orchestra.run.smithi037.stdout:55834574860 2024-04-25T08:52:57.070 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T08:52:57.071 INFO:teuthology.orchestra.run.smithi037.stdout:55834574859 2024-04-25T08:52:57.071 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T08:52:57.072 INFO:teuthology.orchestra.run.smithi037.stdout:55834574858 2024-04-25T08:52:57.072 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T08:52:57.073 INFO:teuthology.orchestra.run.smithi037.stdout:55834574858 2024-04-25T08:52:57.073 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T08:52:57.074 INFO:teuthology.orchestra.run.smithi037.stdout:55834574859 2024-04-25T08:52:57.074 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T08:52:57.075 INFO:teuthology.orchestra.run.smithi037.stdout:55834574860 2024-04-25T08:52:57.075 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T08:52:57.076 INFO:teuthology.orchestra.run.smithi037.stdout:55834574858 2024-04-25T08:52:57.076 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T08:52:57.077 INFO:teuthology.orchestra.run.smithi037.stdout:55834574859 2024-04-25T08:52:57.077 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T08:52:57.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f90653d4640 1 Processor -- start 2024-04-25T08:52:57.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f90653d4640 1 -- start start 2024-04-25T08:52:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f0a0caa4640 1 Processor -- start 2024-04-25T08:52:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f0a0caa4640 1 -- start start 2024-04-25T08:52:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f90653d4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a4d0 0x7f90600cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f90653d4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90600caef0 con 0x7f906015a8b0 2024-04-25T08:52:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f90653d4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90600cb160 con 0x7f90600ceac0 2024-04-25T08:52:57.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f90653d4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90600cba90 con 0x7f906015a4d0 2024-04-25T08:52:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f1fa624b640 1 Processor -- start 2024-04-25T08:52:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f2f4f4ae640 1 Processor -- start 2024-04-25T08:52:57.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f2f4f4ae640 1 -- start start 2024-04-25T08:52:57.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f905f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f906015a8b0 0x7f90600ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59894/0 (socket says 172.21.15.37:59894) 2024-04-25T08:52:57.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f905f7fe640 1 -- 172.21.15.37:0/3585902937 learned_addr learned my addr 172.21.15.37:0/3585902937 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:57.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f905ffff640 1 --2- 172.21.15.37:0/3585902937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a4d0 0x7f90600cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f1fa624b640 1 -- start start 2024-04-25T08:52:57.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f905effd640 1 -- 172.21.15.37:0/3585902937 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3484305062 0 0) 0x7f90600caef0 con 0x7f906015a8b0 2024-04-25T08:52:57.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f905effd640 1 -- 172.21.15.37:0/3585902937 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9048003530 con 0x7f906015a8b0 2024-04-25T08:52:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f905ffff640 1 -- 172.21.15.37:0/3585902937 >> v1:172.21.15.203:6789/0 conn(0x7f90600ceac0 legacy=0x7f90600b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f905ffff640 1 -- 172.21.15.37:0/3585902937 >> v1:172.21.15.37:6789/0 conn(0x7f906015a8b0 legacy=0x7f90600ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7fed06571640 1 Processor -- start 2024-04-25T08:52:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f905ffff640 1 -- 172.21.15.37:0/3585902937 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90600cb440 con 0x7f906015a4d0 2024-04-25T08:52:57.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7fed06571640 1 -- start start 2024-04-25T08:52:57.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7fbce4392640 1 Processor -- start 2024-04-25T08:52:57.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f0a0caa4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0815f6d0 0x7f0a08162a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f0a0caa4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0a080cf260 0x7f0a080b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f0a0caa4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a08162f70 con 0x7f0a080b52c0 2024-04-25T08:52:57.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f0a0caa4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a080b5750 con 0x7f0a080cf260 2024-04-25T08:52:57.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f0a0caa4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080ce3e0 con 0x7f0a0815f6d0 2024-04-25T08:52:57.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f0a05d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0a080cf260 0x7f0a080b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7fbce4392640 1 -- start start 2024-04-25T08:52:57.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f0a05d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0a080cf260 0x7f0a080b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46526/0 (socket says 172.21.15.37:46526) 2024-04-25T08:52:57.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3110041806 learned_addr learned my addr 172.21.15.37:0/3110041806 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f390ce36640 1 Processor -- start 2024-04-25T08:52:57.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f390ce36640 1 -- start start 2024-04-25T08:52:57.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4f4ae640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f481557b0 0x7f2f4815b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4f4ae640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2f48155b90 0x7f2f4806dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4f4ae640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4806d850 con 0x7f2f4806fd10 2024-04-25T08:52:57.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4f4ae640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4806db50 con 0x7f2f48155b90 2024-04-25T08:52:57.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4f4ae640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4815c260 con 0x7f2f481557b0 2024-04-25T08:52:57.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3110041806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0815f6d0 msgr2=0x7f0a08162a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:52:57.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f0a05d9b640 1 --2- 172.21.15.37:0/3110041806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0815f6d0 0x7f0a08162a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3110041806 >> v1:172.21.15.37:6789/0 conn(0x7f0a080b52c0 legacy=0x7f0a081655a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:57.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3110041806 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a080cd7d0 con 0x7f0a080cf260 2024-04-25T08:52:57.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f1fa624b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1fa0071030 0x7f1fa0161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f1fa624b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fa0071410 0x7f1fa006d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f1fa624b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa0162680 con 0x7f1fa0071e90 2024-04-25T08:52:57.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f1fa624b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa0162960 con 0x7f1fa0071030 2024-04-25T08:52:57.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.292+0000 7f1fa624b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa0162c40 con 0x7f1fa0071410 2024-04-25T08:52:57.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f1f9f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1fa0071030 0x7f1fa0161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f1f9f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1fa0071030 0x7f1fa0161fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46538/0 (socket says 172.21.15.37:46538) 2024-04-25T08:52:57.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f1f9f7fe640 1 -- 172.21.15.37:0/1047889076 learned_addr learned my addr 172.21.15.37:0/1047889076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f1f9ffff640 1 --2- 172.21.15.37:0/1047889076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fa0071410 0x7f1fa006d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f1f9f7fe640 1 -- 172.21.15.37:0/1047889076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fa0071410 msgr2=0x7f1fa006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:57.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f1f9f7fe640 1 --2- 172.21.15.37:0/1047889076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fa0071410 0x7f1fa006d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f1f9f7fe640 1 -- 172.21.15.37:0/1047889076 >> v1:172.21.15.37:6789/0 conn(0x7f1fa0071e90 legacy=0x7f1fa00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f1f9f7fe640 1 -- 172.21.15.37:0/1047889076 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa0162f20 con 0x7f1fa0071030 2024-04-25T08:52:57.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390ce36640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3900037510 0x7f39000380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390ce36640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3900054950 0x7f3900049d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390ce36640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3900007bd0 con 0x7f3900054db0 2024-04-25T08:52:57.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390ce36640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39000e7c60 con 0x7f3900037510 2024-04-25T08:52:57.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390ce36640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39000522f0 con 0x7f3900054950 2024-04-25T08:52:57.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4ecad640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2f4806fd10 0x7f2f480b4e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59908/0 (socket says 172.21.15.37:59908) 2024-04-25T08:52:57.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4ecad640 1 -- 172.21.15.37:0/1555137727 learned_addr learned my addr 172.21.15.37:0/1555137727 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:57.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4dcab640 1 --2- 172.21.15.37:0/1555137727 >> v2:172.21.15.37:3300/0 conn(0x7f2f48155b90 0x7f2f4806dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4e4ac640 1 --2- 172.21.15.37:0/1555137727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f481557b0 0x7f2f4815b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4dcab640 1 -- 172.21.15.37:0/1555137727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f481557b0 msgr2=0x7f2f4815b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:57.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4dcab640 1 --2- 172.21.15.37:0/1555137727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f481557b0 0x7f2f4815b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4dcab640 1 -- 172.21.15.37:0/1555137727 >> v1:172.21.15.37:6789/0 conn(0x7f2f4806fd10 legacy=0x7f2f480b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fed06571640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fed000b0990 0x7fed000ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fed06571640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed000b1bb0 con 0x7fed000cc020 2024-04-25T08:52:57.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fed06571640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed000b1910 con 0x7fed00098a20 2024-04-25T08:52:57.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fed06571640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed000c99d0 con 0x7fed000b0990 2024-04-25T08:52:57.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fed0556f640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fed00098a20 0x7fed000cb910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:36148/0 (socket says 172.21.15.37:36148) 2024-04-25T08:52:57.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fed0556f640 1 -- 172.21.15.37:0/671596215 learned_addr learned my addr 172.21.15.37:0/671596215 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:57.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fbce4392640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbcd4047d70 0x7fbcd4037ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fbce4392640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd40543d0 con 0x7fbcd40e7700 2024-04-25T08:52:57.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fbce4392640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd40546b0 con 0x7fbcd40381e0 2024-04-25T08:52:57.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fbce4392640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd4006df0 con 0x7fbcd4047d70 2024-04-25T08:52:57.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f2f4dcab640 1 -- 172.21.15.37:0/1555137727 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4815c540 con 0x7f2f48155b90 2024-04-25T08:52:57.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fed05d70640 1 --2- 172.21.15.37:0/671596215 >> v2:172.21.15.37:3300/0 conn(0x7fed000b0990 0x7fed000ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3900037510 0x7f39000380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3900037510 0x7f39000380e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46554/0 (socket says 172.21.15.37:46554) 2024-04-25T08:52:57.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f3905d9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3900054950 0x7f3900049d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390659c640 1 -- 172.21.15.37:0/2691409593 learned_addr learned my addr 172.21.15.37:0/2691409593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fbce292f640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fbcd40381e0 0x7fbcd404f150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:36154/0 (socket says 172.21.15.37:36154) 2024-04-25T08:52:57.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fbce292f640 1 -- 172.21.15.37:0/2385739071 learned_addr learned my addr 172.21.15.37:0/2385739071 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:57.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f905ffff640 1 --2- 172.21.15.37:0/3585902937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a4d0 0x7f90600cd4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f90600cbc10 tx=0x7f905001ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b093097348599833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fed05d70640 1 -- 172.21.15.37:0/671596215 >> v1:172.21.15.203:6789/0 conn(0x7fed00098a20 legacy=0x7fed000cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fed05d70640 1 -- 172.21.15.37:0/671596215 >> v1:172.21.15.37:6789/0 conn(0x7fed000cc020 legacy=0x7fed000c9e60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:52:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fed05d70640 1 -- 172.21.15.37:0/671596215 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed000cd1f0 con 0x7fed000b0990 2024-04-25T08:52:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fbce192d640 1 --2- 172.21.15.37:0/2385739071 >> v2:172.21.15.37:3300/0 conn(0x7fbcd4047d70 0x7fbcd4037ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390659c640 1 -- 172.21.15.37:0/2691409593 >> v2:172.21.15.203:3300/0 conn(0x7f3900054950 msgr2=0x7f3900049d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:57.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390659c640 1 --2- 172.21.15.37:0/2691409593 >> v2:172.21.15.203:3300/0 conn(0x7f3900054950 0x7f3900049d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390659c640 1 -- 172.21.15.37:0/2691409593 >> v1:172.21.15.203:6789/0 conn(0x7f3900054db0 legacy=0x7f3900052780 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:52:57.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f390659c640 1 -- 172.21.15.37:0/2691409593 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900008430 con 0x7f3900037510 2024-04-25T08:52:57.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fbce192d640 1 -- 172.21.15.37:0/2385739071 >> v1:172.21.15.203:6789/0 conn(0x7fbcd40381e0 legacy=0x7fbcd404f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fbce192d640 1 -- 172.21.15.37:0/2385739071 >> v1:172.21.15.37:6789/0 conn(0x7fbcd40e7700 legacy=0x7fbcd4007280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:57.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7fbce192d640 1 -- 172.21.15.37:0/2385739071 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd4006a80 con 0x7fbcd4047d70 2024-04-25T08:52:57.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f905effd640 1 -- 172.21.15.37:0/3585902937 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90500b5070 con 0x7f906015a4d0 2024-04-25T08:52:57.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f905effd640 1 -- 172.21.15.37:0/3585902937 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90500a8970 con 0x7f906015a4d0 2024-04-25T08:52:57.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f90653d4640 1 -- 172.21.15.37:0/3585902937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a4d0 msgr2=0x7f90600cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f90653d4640 1 --2- 172.21.15.37:0/3585902937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a4d0 0x7f90600cd4f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f90600cbc10 tx=0x7f905001ec70 comp rx=0 tx=0).stop 2024-04-25T08:52:57.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f0a05d9b640 1 --2- 172.21.15.37:0/3110041806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080cf260 0x7f0a080b4d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009e50 tx=0x7f0a0001c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc1c28b169114fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f90653d4640 1 -- 172.21.15.37:0/3585902937 shutdown_connections 2024-04-25T08:52:57.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f90653d4640 1 --2- 172.21.15.37:0/3585902937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a4d0 0x7f90600cd4f0 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f90600cbc10 tx=0x7f905001ec70 comp rx=0 tx=0).stop 2024-04-25T08:52:57.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.296+0000 7f90653d4640 1 -- 172.21.15.37:0/3585902937 >> 172.21.15.37:0/3585902937 conn(0x7f90600b1d90 msgr2=0x7f90600b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1f9f7fe640 1 --2- 172.21.15.37:0/1047889076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fa0071030 0x7f1fa0161fb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009870 tx=0x7f1f9001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a5714b4ccf26c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0559a640 1 -- 172.21.15.37:0/3110041806 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000ac020 con 0x7f0a080cf260 2024-04-25T08:52:57.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0559a640 1 -- 172.21.15.37:0/3110041806 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a000a6460 con 0x7f0a080cf260 2024-04-25T08:52:57.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0559a640 1 -- 172.21.15.37:0/3110041806 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a000a6830 con 0x7f0a080cf260 2024-04-25T08:52:57.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f90653d4640 1 -- 172.21.15.37:0/3585902937 shutdown_connections 2024-04-25T08:52:57.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f90653d4640 1 -- 172.21.15.37:0/3585902937 wait complete. 2024-04-25T08:52:57.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4dcab640 1 --2- 172.21.15.37:0/1555137727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f48155b90 0x7f2f4806dfe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f380097c0 tx=0x7f2f3801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9724f9479c7c005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4d4aa640 1 -- 172.21.15.37:0/1555137727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380ab600 con 0x7f2f48155b90 2024-04-25T08:52:57.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4d4aa640 1 -- 172.21.15.37:0/1555137727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f380b0070 con 0x7f2f48155b90 2024-04-25T08:52:57.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4d4aa640 1 -- 172.21.15.37:0/1555137727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f380ab9c0 con 0x7f2f48155b90 2024-04-25T08:52:57.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1f9e7fc640 1 -- 172.21.15.37:0/1047889076 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f900b1070 con 0x7f1fa0071030 2024-04-25T08:52:57.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1f9e7fc640 1 -- 172.21.15.37:0/1047889076 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f900a8050 con 0x7f1fa0071030 2024-04-25T08:52:57.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1f9e7fc640 1 -- 172.21.15.37:0/1047889076 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f900ac860 con 0x7f1fa0071030 2024-04-25T08:52:57.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0caa4640 1 -- 172.21.15.37:0/3110041806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080cf260 msgr2=0x7f0a080b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0caa4640 1 --2- 172.21.15.37:0/3110041806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080cf260 0x7f0a080b4d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009e50 tx=0x7f0a0001c7a0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc82ad4b640 1 Processor -- start 2024-04-25T08:52:57.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc82ad4b640 1 -- start start 2024-04-25T08:52:57.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0caa4640 1 -- 172.21.15.37:0/3110041806 shutdown_connections 2024-04-25T08:52:57.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0caa4640 1 --2- 172.21.15.37:0/3110041806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080cf260 0x7f0a080b4d80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0caa4640 1 --2- 172.21.15.37:0/3110041806 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a0815f6d0 0x7f0a08162a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0caa4640 1 -- 172.21.15.37:0/3110041806 >> 172.21.15.37:0/3110041806 conn(0x7f0a08072550 msgr2=0x7f0a0806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0caa4640 1 -- 172.21.15.37:0/3110041806 shutdown_connections 2024-04-25T08:52:57.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fed05d70640 1 --2- 172.21.15.37:0/671596215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed000b0990 0x7fed000ca570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0097c0 tx=0x7fed000cb2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcf51dc28d2960e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1555137727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f48155b90 msgr2=0x7f2f4806dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4f4ae640 1 --2- 172.21.15.37:0/1555137727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f48155b90 0x7f2f4806dfe0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2f380097c0 tx=0x7f2f3801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1555137727 shutdown_connections 2024-04-25T08:52:57.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4f4ae640 1 --2- 172.21.15.37:0/1555137727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f48155b90 0x7f2f4806dfe0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4f4ae640 1 --2- 172.21.15.37:0/1555137727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f481557b0 0x7f2f4815b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1555137727 >> 172.21.15.37:0/1555137727 conn(0x7f2f48072590 msgr2=0x7f2f4806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fbce192d640 1 --2- 172.21.15.37:0/2385739071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd4047d70 0x7fbcd4037ca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80097c0 tx=0x7fbcd801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac8b94394d4cdca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1fa624b640 1 -- 172.21.15.37:0/1047889076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fa0071030 msgr2=0x7f1fa0161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1fa624b640 1 --2- 172.21.15.37:0/1047889076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fa0071030 0x7f1fa0161fb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009870 tx=0x7f1f9001c4d0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1555137727 shutdown_connections 2024-04-25T08:52:57.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1555137727 wait complete. 2024-04-25T08:52:57.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fecf7fff640 1 -- 172.21.15.37:0/671596215 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0b0070 con 0x7fed000b0990 2024-04-25T08:52:57.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fecf7fff640 1 -- 172.21.15.37:0/671596215 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecfc0ab600 con 0x7fed000b0990 2024-04-25T08:52:57.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fecf7fff640 1 -- 172.21.15.37:0/671596215 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc0ab9d0 con 0x7fed000b0990 2024-04-25T08:52:57.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1fa624b640 1 -- 172.21.15.37:0/1047889076 shutdown_connections 2024-04-25T08:52:57.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1fa624b640 1 --2- 172.21.15.37:0/1047889076 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fa0071410 0x7f1fa006d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1fa624b640 1 --2- 172.21.15.37:0/1047889076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fa0071030 0x7f1fa0161fb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1fa624b640 1 -- 172.21.15.37:0/1047889076 >> 172.21.15.37:0/1047889076 conn(0x7f1fa00b11a0 msgr2=0x7f1fa00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f90653d4640 1 Processor -- start 2024-04-25T08:52:57.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f90653d4640 1 -- start start 2024-04-25T08:52:57.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1fa624b640 1 -- 172.21.15.37:0/1047889076 shutdown_connections 2024-04-25T08:52:57.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc82ad4b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc82415d050 0x7fc8240cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc82ad4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8240ca160 0x7fc82406d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fed06571640 1 -- 172.21.15.37:0/671596215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed000b0990 msgr2=0x7fed000ca570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fed06571640 1 --2- 172.21.15.37:0/671596215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed000b0990 0x7fed000ca570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0097c0 tx=0x7fed000cb2c0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc82ad4b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8240ca5a0 con 0x7fc8240cbff0 2024-04-25T08:52:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc828ae7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc82415d050 0x7fc8240cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc828ae7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc82415d050 0x7fc8240cbab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:34082/0 (socket says 172.21.15.37:34082) 2024-04-25T08:52:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc828ae7640 1 -- 172.21.15.37:0/3391961633 learned_addr learned my addr 172.21.15.37:0/3391961633 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc82ad4b640 1 -- 172.21.15.37:0/3391961633 --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc82415d4d0 con 0x7fc82415d050 2024-04-25T08:52:57.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc82ad4b640 1 -- 172.21.15.37:0/3391961633 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82415d7b0 con 0x7fc8240ca160 2024-04-25T08:52:57.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0caa4640 1 -- 172.21.15.37:0/3110041806 wait complete. 2024-04-25T08:52:57.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0caa4640 1 Processor -- start 2024-04-25T08:52:57.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fbce112c640 1 -- 172.21.15.37:0/2385739071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80b5070 con 0x7fbcd4047d70 2024-04-25T08:52:57.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fbce112c640 1 -- 172.21.15.37:0/2385739071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd80a8950 con 0x7fbcd4047d70 2024-04-25T08:52:57.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fbce112c640 1 -- 172.21.15.37:0/2385739071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd80a8d20 con 0x7fbcd4047d70 2024-04-25T08:52:57.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fbce4392640 1 -- 172.21.15.37:0/2385739071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd4047d70 msgr2=0x7fbcd4037ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fbce4392640 1 --2- 172.21.15.37:0/2385739071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd4047d70 0x7fbcd4037ca0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80097c0 tx=0x7fbcd801ec50 comp rx=0 tx=0).stop 2024-04-25T08:52:57.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fbce4392640 1 -- 172.21.15.37:0/2385739071 shutdown_connections 2024-04-25T08:52:57.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fbce4392640 1 --2- 172.21.15.37:0/2385739071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd4047d70 0x7fbcd4037ca0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fbce4392640 1 -- 172.21.15.37:0/2385739071 >> 172.21.15.37:0/2385739071 conn(0x7fbcd4054b00 msgr2=0x7fbcd4054f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4f4ae640 1 Processor -- start 2024-04-25T08:52:57.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fbce4392640 1 -- 172.21.15.37:0/2385739071 shutdown_connections 2024-04-25T08:52:57.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc828ae7640 1 -- 172.21.15.37:0/3391961633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8240ca160 msgr2=0x7fc82406d230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:52:57.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc828ae7640 1 --2- 172.21.15.37:0/3391961633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8240ca160 0x7fc82406d230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc828ae7640 1 -- 172.21.15.37:0/3391961633 >> v1:172.21.15.203:6789/0 conn(0x7fc8240cbff0 legacy=0x7fc8240c9a50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:52:57.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fc828ae7640 1 -- 172.21.15.37:0/3391961633 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc82415da90 con 0x7fc82415d050 2024-04-25T08:52:57.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f2f4f4ae640 1 -- start start 2024-04-25T08:52:57.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f90653d4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90600ceac0 0x7f90601fbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f90653d4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a8b0 0x7f90600c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f1fa624b640 1 -- 172.21.15.37:0/1047889076 wait complete. 2024-04-25T08:52:57.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1fa624b640 1 Processor -- start 2024-04-25T08:52:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f90653d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90601f7b70 0x7f9060158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f90653d4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90600cdbb0 con 0x7f90601f7b70 2024-04-25T08:52:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f90653d4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90600cda30 con 0x7f90600ceac0 2024-04-25T08:52:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7f0a0caa4640 1 -- start start 2024-04-25T08:52:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0caa4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080b52c0 0x7f0a08151b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90600ceac0 0x7f90601fbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905f7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a8b0 0x7f90600c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905ffff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90600ceac0 0x7f90601fbba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:34086/0 (socket says 172.21.15.37:34086) 2024-04-25T08:52:57.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905ffff640 1 -- 172.21.15.37:0/1729121570 learned_addr learned my addr 172.21.15.37:0/1729121570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f90653d4640 1 -- 172.21.15.37:0/1729121570 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90600b29b0 con 0x7f906015a8b0 2024-04-25T08:52:57.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4f4ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4806fd10 0x7f2f4806aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4f4ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f481557b0 0x7f2f480b4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4f4ae640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48155b90 0x7f2f480b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4f4ae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480cc690 con 0x7f2f481557b0 2024-04-25T08:52:57.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4f4ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480cc510 con 0x7f2f4806fd10 2024-04-25T08:52:57.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4f4ae640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480b5530 con 0x7f2f48155b90 2024-04-25T08:52:57.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0caa4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a080cf260 0x7f0a0815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0caa4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0815f6d0 0x7f0a081641a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0caa4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080ce870 con 0x7f0a080b52c0 2024-04-25T08:52:57.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0caa4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0816e940 con 0x7f0a0815f6d0 2024-04-25T08:52:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4e4ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4806fd10 0x7f2f4806aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4ecad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48155b90 0x7f2f480b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0caa4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0815bd60 con 0x7f0a080cf260 2024-04-25T08:52:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4ecad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48155b90 0x7f2f480b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:39094/0 (socket says 172.21.15.37:39094) 2024-04-25T08:52:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4ecad640 1 -- 172.21.15.37:0/1527791601 learned_addr learned my addr 172.21.15.37:0/1527791601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f390659c640 1 --2- 172.21.15.37:0/2691409593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3900037510 0x7f39000380e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3908050520 tx=0x7f39080970f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f5309ddcc254cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a06d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0815f6d0 0x7f0a081641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a06d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0815f6d0 0x7f0a081641a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:34112/0 (socket says 172.21.15.37:34112) 2024-04-25T08:52:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a06d9d640 1 -- 172.21.15.37:0/2757752368 learned_addr learned my addr 172.21.15.37:0/2757752368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a05d9b640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a080cf260 0x7f0a0815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.300+0000 7fbce4392640 1 -- 172.21.15.37:0/2385739071 wait complete. 2024-04-25T08:52:57.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce4392640 1 Processor -- start 2024-04-25T08:52:57.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1fa624b640 1 -- start start 2024-04-25T08:52:57.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1fa624b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa0071030 0x7f1fa014eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1fa624b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fa0071410 0x7f1fa01589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4dcab640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f481557b0 0x7f2f480b4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0659c640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080b52c0 0x7f0a08151b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905f7fe640 1 -- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90600ceac0 msgr2=0x7f90601fbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905f7fe640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90600ceac0 0x7f90601fbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905f7fe640 1 -- 172.21.15.37:0/1729121570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90601f7b70 msgr2=0x7f9060158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905f7fe640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90601f7b70 0x7f9060158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905f7fe640 1 -- 172.21.15.37:0/1729121570 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90601fc0e0 con 0x7f906015a8b0 2024-04-25T08:52:57.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f390559a640 1 -- 172.21.15.37:0/2691409593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908127440 con 0x7f3900037510 2024-04-25T08:52:57.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f390559a640 1 -- 172.21.15.37:0/2691409593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39081275e0 con 0x7f3900037510 2024-04-25T08:52:57.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f390559a640 1 -- 172.21.15.37:0/2691409593 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908127a50 con 0x7f3900037510 2024-04-25T08:52:57.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905f7fe640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a8b0 0x7f90600c59d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9054001ec0 tx=0x7f905400c870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4ecad640 1 -- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4806fd10 msgr2=0x7f2f4806aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4ecad640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4806fd10 0x7f2f4806aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4ecad640 1 -- 172.21.15.37:0/1527791601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f481557b0 msgr2=0x7f2f480b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4ecad640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f481557b0 0x7f2f480b4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4ecad640 1 -- 172.21.15.37:0/1527791601 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4813ff50 con 0x7f2f48155b90 2024-04-25T08:52:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0659c640 1 -- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a080cf260 msgr2=0x7f0a0815ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0659c640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a080cf260 0x7f0a0815ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0659c640 1 -- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0815f6d0 msgr2=0x7f0a081641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0659c640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0815f6d0 0x7f0a081641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0659c640 1 -- 172.21.15.37:0/2757752368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0815b4c0 con 0x7f0a080b52c0 2024-04-25T08:52:57.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4ecad640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48155b90 0x7f2f480b4d90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000caa0 tx=0x7f2f4000cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a05d9b640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a080cf260 0x7f0a0815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:57.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a06d9d640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0815f6d0 0x7f0a081641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:52:57.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905cff9640 1 -- 172.21.15.37:0/1729121570 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90540074e0 con 0x7f906015a8b0 2024-04-25T08:52:57.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fc828ae7640 1 --2- 172.21.15.37:0/3391961633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc82415d050 0x7fc8240cbab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc8240ce2b0 tx=0x7fc81801d530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f73e924afcf67a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f90653d4640 1 -- 172.21.15.37:0/1729121570 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90600c73b0 con 0x7f906015a8b0 2024-04-25T08:52:57.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f90653d4640 1 -- 172.21.15.37:0/1729121570 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90600c7c50 con 0x7f906015a8b0 2024-04-25T08:52:57.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905cff9640 1 -- 172.21.15.37:0/1729121570 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90540047f0 con 0x7f906015a8b0 2024-04-25T08:52:57.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f905cff9640 1 -- 172.21.15.37:0/1729121570 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9054007680 con 0x7f906015a8b0 2024-04-25T08:52:57.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f390ce36640 1 -- 172.21.15.37:0/2691409593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3900037510 msgr2=0x7f39000380e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f390ce36640 1 --2- 172.21.15.37:0/2691409593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3900037510 0x7f39000380e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3908050520 tx=0x7f39080970f0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f3effd640 1 -- 172.21.15.37:0/1527791601 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40017070 con 0x7f2f48155b90 2024-04-25T08:52:57.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f3effd640 1 -- 172.21.15.37:0/1527791601 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4000ab40 con 0x7f2f48155b90 2024-04-25T08:52:57.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f3effd640 1 -- 172.21.15.37:0/1527791601 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40012400 con 0x7f2f48155b90 2024-04-25T08:52:57.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1f9f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa0071030 0x7f1fa014eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1f9f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa0071030 0x7f1fa014eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:34118/0 (socket says 172.21.15.37:34118) 2024-04-25T08:52:57.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1f9effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fa0071410 0x7f1fa01589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1f9effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fa0071410 0x7f1fa01589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46600/0 (socket says 172.21.15.37:46600) 2024-04-25T08:52:57.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1fa624b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fa0071e90 0x7f1fa015a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1527791601 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f48140230 con 0x7f2f48155b90 2024-04-25T08:52:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1527791601 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4806b430 con 0x7f2f48155b90 2024-04-25T08:52:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce4392640 1 -- start start 2024-04-25T08:52:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce4392640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcd40381e0 0x7fbcd40e48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce4392640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd4047d70 0x7fbcd40e9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce4392640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcd40e7700 0x7fbcd40d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce4392640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4038790 con 0x7fbcd4047d70 2024-04-25T08:52:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce4392640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4038610 con 0x7fbcd40381e0 2024-04-25T08:52:57.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce4392640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4038910 con 0x7fbcd40e7700 2024-04-25T08:52:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fed06571640 1 -- 172.21.15.37:0/671596215 shutdown_connections 2024-04-25T08:52:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fed06571640 1 --2- 172.21.15.37:0/671596215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed000b0990 0x7fed000ca570 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fed06571640 1 -- 172.21.15.37:0/671596215 >> 172.21.15.37:0/671596215 conn(0x7fed000b0df0 msgr2=0x7fed000b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce212e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcd40381e0 0x7fbcd40e48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce212e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcd40381e0 0x7fbcd40e48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:34130/0 (socket says 172.21.15.37:34130) 2024-04-25T08:52:57.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce212e640 1 -- 172.21.15.37:0/272723987 learned_addr learned my addr 172.21.15.37:0/272723987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1fa624b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa006e170 con 0x7f1fa0071410 2024-04-25T08:52:57.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1fa624b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa006dff0 con 0x7f1fa0071030 2024-04-25T08:52:57.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1fa624b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa006e2f0 con 0x7f1fa0071e90 2024-04-25T08:52:57.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f1f9f7fe640 1 -- 172.21.15.37:0/2303748466 learned_addr learned my addr 172.21.15.37:0/2303748466 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0659c640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080b52c0 0x7f0a08151b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09f800b410 tx=0x7f09f800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f09f6ffd640 1 -- 172.21.15.37:0/2757752368 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8015020 con 0x7f0a080b52c0 2024-04-25T08:52:57.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f09f6ffd640 1 -- 172.21.15.37:0/2757752368 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f8011830 con 0x7f0a080b52c0 2024-04-25T08:52:57.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f09f6ffd640 1 -- 172.21.15.37:0/2757752368 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8011c20 con 0x7f0a080b52c0 2024-04-25T08:52:57.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0caa4640 1 -- 172.21.15.37:0/2757752368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0815b7a0 con 0x7f0a080b52c0 2024-04-25T08:52:57.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7f0a0caa4640 1 -- 172.21.15.37:0/2757752368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0815b0d0 con 0x7f0a080b52c0 2024-04-25T08:52:57.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fc8237fe640 1 -- 172.21.15.37:0/3391961633 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8180a7040 con 0x7fc82415d050 2024-04-25T08:52:57.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fc8237fe640 1 -- 172.21.15.37:0/3391961633 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8180b0070 con 0x7fc82415d050 2024-04-25T08:52:57.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fc8237fe640 1 -- 172.21.15.37:0/3391961633 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818008e00 con 0x7fc82415d050 2024-04-25T08:52:57.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1f9effd640 1 -- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fa0071e90 msgr2=0x7f1fa015a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1f9effd640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fa0071e90 0x7f1fa015a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1f9effd640 1 -- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa0071030 msgr2=0x7f1fa014eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1f9effd640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa0071030 0x7f1fa014eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1f9effd640 1 -- 172.21.15.37:0/2303748466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa01510c0 con 0x7f1fa0071410 2024-04-25T08:52:57.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1527791601 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4804ef30 con 0x7f2f48155b90 2024-04-25T08:52:57.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fed06571640 1 -- 172.21.15.37:0/671596215 shutdown_connections 2024-04-25T08:52:57.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fed06571640 1 -- 172.21.15.37:0/671596215 wait complete. 2024-04-25T08:52:57.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.304+0000 7fbce292f640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcd40e7700 0x7fbcd40d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1f9effd640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fa0071410 0x7f1fa01589c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00b2bb0 tx=0x7f1f9400c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f0a0caa4640 1 -- 172.21.15.37:0/2757752368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ec0052a0 con 0x7f0a080b52c0 2024-04-25T08:52:57.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fc82ad4b640 1 -- 172.21.15.37:0/3391961633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc82415d050 msgr2=0x7fc8240cbab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fc82ad4b640 1 --2- 172.21.15.37:0/3391961633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc82415d050 0x7fc8240cbab0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc8240ce2b0 tx=0x7fc81801d530 comp rx=0 tx=0).stop 2024-04-25T08:52:57.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fbce212e640 1 -- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcd40e7700 msgr2=0x7fbcd40d2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fbce212e640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcd40e7700 0x7fbcd40d2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fbce212e640 1 -- 172.21.15.37:0/272723987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd4047d70 msgr2=0x7fbcd40e9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fbce212e640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd4047d70 0x7fbcd40e9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fbce212e640 1 -- 172.21.15.37:0/272723987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd4180bc0 con 0x7fbcd40381e0 2024-04-25T08:52:57.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1f87fff640 1 -- 172.21.15.37:0/2303748466 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94017070 con 0x7f1fa0071410 2024-04-25T08:52:57.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1f87fff640 1 -- 172.21.15.37:0/2303748466 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f940052d0 con 0x7f1fa0071410 2024-04-25T08:52:57.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1f87fff640 1 -- 172.21.15.37:0/2303748466 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94003840 con 0x7f1fa0071410 2024-04-25T08:52:57.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1fa624b640 1 -- 172.21.15.37:0/2303748466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa01513a0 con 0x7f1fa0071410 2024-04-25T08:52:57.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed06571640 1 Processor -- start 2024-04-25T08:52:57.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1fa624b640 1 -- 172.21.15.37:0/2303748466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa014f070 con 0x7f1fa0071410 2024-04-25T08:52:57.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fbce212e640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcd40381e0 0x7fbcd40e48f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00c480 tx=0x7fbccc00c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f09f6ffd640 1 -- 172.21.15.37:0/2757752368 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f09f8018020 con 0x7f0a080b52c0 2024-04-25T08:52:57.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f09f6ffd640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09e8067060 0x7f09e8069520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f09f6ffd640 1 -- 172.21.15.37:0/2757752368 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f09f8084ab0 con 0x7f0a080b52c0 2024-04-25T08:52:57.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f0a05d9b640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09e8067060 0x7f09e8069520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f90653d4640 1 -- 172.21.15.37:0/1729121570 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90400052a0 con 0x7f906015a8b0 2024-04-25T08:52:57.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f905cff9640 1 -- 172.21.15.37:0/1729121570 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9054011070 con 0x7f906015a8b0 2024-04-25T08:52:57.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f905cff9640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f904c066f60 0x7f904c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f905cff9640 1 -- 172.21.15.37:0/1729121570 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f9054084e60 con 0x7f906015a8b0 2024-04-25T08:52:57.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fbcd2ffd640 1 -- 172.21.15.37:0/272723987 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc012830 con 0x7fbcd40381e0 2024-04-25T08:52:57.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fbcd2ffd640 1 -- 172.21.15.37:0/272723987 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc017070 con 0x7fbcd40381e0 2024-04-25T08:52:57.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fbcd2ffd640 1 -- 172.21.15.37:0/272723987 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc012c50 con 0x7fbcd40381e0 2024-04-25T08:52:57.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed06571640 1 -- start start 2024-04-25T08:52:57.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f905ffff640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f904c066f60 0x7f904c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed06571640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed00098a20 0x7fed000c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed06571640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed000b0990 0x7fed0015d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed06571640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed000cc020 0x7fed00156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed06571640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0016b820 con 0x7fed00098a20 2024-04-25T08:52:57.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed06571640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed00098e00 con 0x7fed000cc020 2024-04-25T08:52:57.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed06571640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0016b9a0 con 0x7fed000b0990 2024-04-25T08:52:57.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fbce4392640 1 -- 172.21.15.37:0/272723987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd4180e60 con 0x7fbcd40381e0 2024-04-25T08:52:57.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1fa624b640 1 -- 172.21.15.37:0/2303748466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa004efa0 con 0x7f1fa0071410 2024-04-25T08:52:57.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fbce4392640 1 -- 172.21.15.37:0/272723987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd4181420 con 0x7fbcd40381e0 2024-04-25T08:52:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fc82ad4b640 1 -- 172.21.15.37:0/3391961633 shutdown_connections 2024-04-25T08:52:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fc82ad4b640 1 --2- 172.21.15.37:0/3391961633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8240ca160 0x7fc82406d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fc82ad4b640 1 --2- 172.21.15.37:0/3391961633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc82415d050 0x7fc8240cbab0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fc82ad4b640 1 -- 172.21.15.37:0/3391961633 >> 172.21.15.37:0/3391961633 conn(0x7fc8240b0cf0 msgr2=0x7fc8240b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed0556f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed00098a20 0x7fed000c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed0556f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed00098a20 0x7fed000c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46620/0 (socket says 172.21.15.37:46620) 2024-04-25T08:52:57.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed0556f640 1 -- 172.21.15.37:0/1855397078 learned_addr learned my addr 172.21.15.37:0/1855397078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed05d70640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed000cc020 0x7fed00156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f0a05d9b640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09e8067060 0x7f09e8069520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0a000ad000 tx=0x7f0a000b4040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fc82ad4b640 1 -- 172.21.15.37:0/3391961633 shutdown_connections 2024-04-25T08:52:57.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fc82ad4b640 1 -- 172.21.15.37:0/3391961633 wait complete. 2024-04-25T08:52:57.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1f87fff640 1 -- 172.21.15.37:0/2303748466 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f9401a070 con 0x7f1fa0071410 2024-04-25T08:52:57.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f1f87fff640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f8c067030 0x7f1f8c0694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f1f87fff640 1 -- 172.21.15.37:0/2303748466 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1f94084640 con 0x7f1fa0071410 2024-04-25T08:52:57.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 -- 172.21.15.37:0/2691409593 shutdown_connections 2024-04-25T08:52:57.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 --2- 172.21.15.37:0/2691409593 >> v2:172.21.15.203:3300/0 conn(0x7f3900054950 0x7f3900049d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 --2- 172.21.15.37:0/2691409593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3900037510 0x7f39000380e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 -- 172.21.15.37:0/2691409593 >> 172.21.15.37:0/2691409593 conn(0x7f3900020050 msgr2=0x7f3900056d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 -- 172.21.15.37:0/2691409593 shutdown_connections 2024-04-25T08:52:57.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f2f3effd640 1 -- 172.21.15.37:0/1527791601 <== mon.2 v2:172.21.15.203:3301/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f4001a070 con 0x7f2f48155b90 2024-04-25T08:52:57.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7f2f3effd640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f2c066f60 0x7f2f2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f2f3effd640 1 -- 172.21.15.37:0/1527791601 <== mon.2 v2:172.21.15.203:3301/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2f40085330 con 0x7f2f48155b90 2024-04-25T08:52:57.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.308+0000 7fed04d6e640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed000b0990 0x7fed0015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fed0556f640 1 -- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed000b0990 msgr2=0x7fed0015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fed0556f640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed000b0990 0x7fed0015d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fed0556f640 1 -- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed000cc020 msgr2=0x7fed00156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fed0556f640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed000cc020 0x7fed00156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fed0556f640 1 -- 172.21.15.37:0/1855397078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed000c8930 con 0x7fed00098a20 2024-04-25T08:52:57.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fed05d70640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed000cc020 0x7fed00156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:57.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fbce4392640 1 -- 172.21.15.37:0/272723987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc40052a0 con 0x7fbcd40381e0 2024-04-25T08:52:57.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 -- 172.21.15.37:0/2691409593 wait complete. 2024-04-25T08:52:57.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fed0556f640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed00098a20 0x7fed000c8350 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fecf000c400 tx=0x7fecf000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f905ffff640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f904c066f60 0x7f904c069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f905001d320 tx=0x7f9050008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f1f9f7fe640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f8c067030 0x7f1f8c0694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f2f4e4ac640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f2c066f60 0x7f2f2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc82ad4b640 1 Processor -- start 2024-04-25T08:52:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f1f9f7fe640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f8c067030 0x7f1f8c0694f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f90000c10 tx=0x7f1f90008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f2f4e4ac640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f2c066f60 0x7f2f2c069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009b60 tx=0x7f2f44009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc82ad4b640 1 -- start start 2024-04-25T08:52:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc82ad4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240ca160 0x7fc8241557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc82ad4b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8240cbff0 0x7fc824200720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc82ad4b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc82415d050 0x7fc8240b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc82ad4b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82406e100 con 0x7fc8240ca160 2024-04-25T08:52:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc82ad4b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82406df80 con 0x7fc8240cbff0 2024-04-25T08:52:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc82ad4b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82406e280 con 0x7fc82415d050 2024-04-25T08:52:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc823fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8240cbff0 0x7fc824200720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc823fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8240cbff0 0x7fc824200720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:34150/0 (socket says 172.21.15.37:34150) 2024-04-25T08:52:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc823fff640 1 -- 172.21.15.37:0/91319850 learned_addr learned my addr 172.21.15.37:0/91319850 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc8292e8640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc82415d050 0x7fc8240b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 Processor -- start 2024-04-25T08:52:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 -- start start 2024-04-25T08:52:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fbcd2ffd640 1 -- 172.21.15.37:0/272723987 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbccc0050d0 con 0x7fbcd40381e0 2024-04-25T08:52:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fbcd2ffd640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbcc0066f60 0x7fbcc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fbcd2ffd640 1 -- 172.21.15.37:0/272723987 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbccc0852b0 con 0x7fbcd40381e0 2024-04-25T08:52:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc823fff640 1 -- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc82415d050 msgr2=0x7fc8240b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc823fff640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc82415d050 0x7fc8240b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc823fff640 1 -- 172.21.15.37:0/91319850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240ca160 msgr2=0x7fc8241557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc823fff640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240ca160 0x7fc8241557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc823fff640 1 -- 172.21.15.37:0/91319850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8242034d0 con 0x7fc8240cbff0 2024-04-25T08:52:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fbce192d640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbcc0066f60 0x7fbcc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7fc823fff640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8240cbff0 0x7fc824200720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc81000b4d0 tx=0x7fc81000b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3900037510 0x7f390004fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3900054950 0x7f39000e6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3900054db0 0x7f3900035770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3900038620 con 0x7f3900054db0 2024-04-25T08:52:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390004a6a0 con 0x7f3900037510 2024-04-25T08:52:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390ce36640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3900057620 con 0x7f3900054950 2024-04-25T08:52:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f3906d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3900054db0 0x7f3900035770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f3906d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3900054db0 0x7f3900035770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46642/0 (socket says 172.21.15.37:46642) 2024-04-25T08:52:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f3906d9d640 1 -- 172.21.15.37:0/1310618681 learned_addr learned my addr 172.21.15.37:0/1310618681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f390659c640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3900037510 0x7f390004fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.312+0000 7f3905d9b640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3900054950 0x7f39000e6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.316+0000 7fbce192d640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbcc0066f60 0x7fbcc0069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80b2000 tx=0x7fbcd8004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.316+0000 7f3906d9d640 1 -- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3900054950 msgr2=0x7f39000e6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.316+0000 7f3906d9d640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3900054950 0x7f39000e6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.316+0000 7f3906d9d640 1 -- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3900037510 msgr2=0x7f390004fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.316+0000 7f3906d9d640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3900037510 0x7f390004fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.316+0000 7f3906d9d640 1 -- 172.21.15.37:0/1310618681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39000e70f0 con 0x7f3900054db0 2024-04-25T08:52:57.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.316+0000 7f390659c640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3900037510 0x7f390004fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:52:57.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.316+0000 7f3906d9d640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3900054db0 0x7f3900035770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00b410 tx=0x7f38fc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.316+0000 7f09f6ffd640 1 -- 172.21.15.37:0/2757752368 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f09f801f070 con 0x7f0a080b52c0 2024-04-25T08:52:57.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7fecf5ffb640 1 -- 172.21.15.37:0/1855397078 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0017070 con 0x7fed00098a20 2024-04-25T08:52:57.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7fed06571640 1 -- 172.21.15.37:0/1855397078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed000c8c10 con 0x7fed00098a20 2024-04-25T08:52:57.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7fecf5ffb640 1 -- 172.21.15.37:0/1855397078 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf0005cf0 con 0x7fed00098a20 2024-04-25T08:52:57.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7f1f87fff640 1 -- 172.21.15.37:0/2303748466 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f94005480 con 0x7f1fa0071410 2024-04-25T08:52:57.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7f38f6ffd640 1 -- 172.21.15.37:0/1310618681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc015020 con 0x7f3900054db0 2024-04-25T08:52:57.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7f390ce36640 1 -- 172.21.15.37:0/1310618681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390004f070 con 0x7f3900054db0 2024-04-25T08:52:57.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7fed06571640 1 -- 172.21.15.37:0/1855397078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0015cd10 con 0x7fed00098a20 2024-04-25T08:52:57.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7f390ce36640 1 -- 172.21.15.37:0/1310618681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390004f330 con 0x7f3900054db0 2024-04-25T08:52:57.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7f38f6ffd640 1 -- 172.21.15.37:0/1310618681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38fc011830 con 0x7f3900054db0 2024-04-25T08:52:57.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7f38f6ffd640 1 -- 172.21.15.37:0/1310618681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38fc011be0 con 0x7f3900054db0 2024-04-25T08:52:57.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7fecf5ffb640 1 -- 172.21.15.37:0/1855397078 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0021060 con 0x7fed00098a20 2024-04-25T08:52:57.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7fed06571640 1 -- 172.21.15.37:0/1855397078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed0015cf30 con 0x7fed00098a20 2024-04-25T08:52:57.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7fecf5ffb640 1 -- 172.21.15.37:0/1855397078 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fecf001a070 con 0x7fed00098a20 2024-04-25T08:52:57.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7f390ce36640 1 -- 172.21.15.37:0/1310618681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec0052a0 con 0x7f3900054db0 2024-04-25T08:52:57.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7fecf5ffb640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fece4066f60 0x7fece4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7fecf5ffb640 1 -- 172.21.15.37:0/1855397078 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fecf00847a0 con 0x7fed00098a20 2024-04-25T08:52:57.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7f38f6ffd640 1 -- 172.21.15.37:0/1310618681 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38fc018020 con 0x7f3900054db0 2024-04-25T08:52:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.320+0000 7fed04d6e640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fece4066f60 0x7fece4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.324+0000 7f38f6ffd640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f38e8066f60 0x7f38e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.324+0000 7f38f6ffd640 1 -- 172.21.15.37:0/1310618681 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f38fc0850f0 con 0x7f3900054db0 2024-04-25T08:52:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.324+0000 7f390659c640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f38e8066f60 0x7f38e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.324+0000 7fed04d6e640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fece4066f60 0x7fece4069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fed0006ed10 tx=0x7fecf800d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.324+0000 7f2f3effd640 1 -- 172.21.15.37:0/1527791601 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f40055a20 con 0x7f2f48155b90 2024-04-25T08:52:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.324+0000 7f390659c640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f38e8066f60 0x7f38e8069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f390000c150 tx=0x7f3908121040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.328+0000 7fecf5ffb640 1 -- 172.21.15.37:0/1855397078 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed0015cf30 con 0x7fed00098a20 2024-04-25T08:52:57.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.328+0000 7fbcd2ffd640 1 -- 172.21.15.37:0/272723987 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbccc0560f0 con 0x7fbcd40381e0 2024-04-25T08:52:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.332+0000 7fc8217fa640 1 -- 172.21.15.37:0/91319850 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810014070 con 0x7fc8240cbff0 2024-04-25T08:52:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.332+0000 7fc8217fa640 1 -- 172.21.15.37:0/91319850 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc810010890 con 0x7fc8240cbff0 2024-04-25T08:52:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.332+0000 7fc82ad4b640 1 -- 172.21.15.37:0/91319850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8242037b0 con 0x7fc8240cbff0 2024-04-25T08:52:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.332+0000 7fc8217fa640 1 -- 172.21.15.37:0/91319850 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810010c80 con 0x7fc8240cbff0 2024-04-25T08:52:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.332+0000 7fc82ad4b640 1 -- 172.21.15.37:0/91319850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc824200c60 con 0x7fc8240cbff0 2024-04-25T08:52:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.332+0000 7fc82ad4b640 1 -- 172.21.15.37:0/91319850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82404ef30 con 0x7fc8240cbff0 2024-04-25T08:52:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.332+0000 7fc8217fa640 1 -- 172.21.15.37:0/91319850 <== mon.1 v2:172.21.15.203:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc810017070 con 0x7fc8240cbff0 2024-04-25T08:52:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.332+0000 7fc8217fa640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc808066fb0 0x7fc808069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.332+0000 7fc8217fa640 1 -- 172.21.15.37:0/91319850 <== mon.1 v2:172.21.15.203:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc810085560 con 0x7fc8240cbff0 2024-04-25T08:52:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.332+0000 7fc828ae7640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc808066fb0 0x7fc808069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.336+0000 7fc828ae7640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc808066fb0 0x7fc808069470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc8180af000 tx=0x7fc818009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.336+0000 7f38f6ffd640 1 -- 172.21.15.37:0/1310618681 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38fc01f020 con 0x7f3900054db0 2024-04-25T08:52:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.336+0000 7f905cff9640 1 -- 172.21.15.37:0/1729121570 <== mon.2 v2:172.21.15.203:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90601fc0e0 con 0x7f906015a8b0 2024-04-25T08:52:57.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.344+0000 7fc8217fa640 1 -- 172.21.15.37:0/91319850 <== mon.1 v2:172.21.15.203:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc810055bc0 con 0x7fc8240cbff0 2024-04-25T08:52:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.636+0000 7f0a0caa4640 1 -- 172.21.15.37:0/2757752368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f09ec005590 con 0x7f0a080b52c0 2024-04-25T08:52:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.636+0000 7f09f6ffd640 1 -- 172.21.15.37:0/2757752368 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f09f8055110 con 0x7f0a080b52c0 2024-04-25T08:52:57.632 INFO:teuthology.orchestra.run.smithi037.stdout:55834574860 2024-04-25T08:52:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 -- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09e8067060 msgr2=0x7f09e8069520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09e8067060 0x7f09e8069520 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0a000ad000 tx=0x7f0a000b4040 comp rx=0 tx=0).stop 2024-04-25T08:52:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 -- 172.21.15.37:0/2757752368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080b52c0 msgr2=0x7f0a08151b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080b52c0 0x7f0a08151b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09f800b410 tx=0x7f09f800b8d0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 -- 172.21.15.37:0/2757752368 shutdown_connections 2024-04-25T08:52:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09e8067060 0x7f09e8069520 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a0815f6d0 0x7f0a081641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a080cf260 0x7f0a0815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 --2- 172.21.15.37:0/2757752368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080b52c0 0x7f0a08151b00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 -- 172.21.15.37:0/2757752368 >> 172.21.15.37:0/2757752368 conn(0x7f0a08072550 msgr2=0x7f0a080cbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 -- 172.21.15.37:0/2757752368 shutdown_connections 2024-04-25T08:52:57.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.640+0000 7f09f4ff9640 1 -- 172.21.15.37:0/2757752368 wait complete. 2024-04-25T08:52:57.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.648+0000 7f1fa624b640 1 -- 172.21.15.37:0/2303748466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1fa00cc190 con 0x7f1fa0071410 2024-04-25T08:52:57.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.648+0000 7f1f87fff640 1 -- 172.21.15.37:0/2303748466 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f9405b080 con 0x7f1fa0071410 2024-04-25T08:52:57.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.648+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1527791601 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2f4813fc90 con 0x7f2f48155b90 2024-04-25T08:52:57.641 INFO:teuthology.orchestra.run.smithi037.stdout:55834574858 2024-04-25T08:52:57.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.648+0000 7fed06571640 1 -- 172.21.15.37:0/1855397078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fed0004ef30 con 0x7fed00098a20 2024-04-25T08:52:57.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.648+0000 7f2f3effd640 1 -- 172.21.15.37:0/1527791601 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f40059ac0 con 0x7f2f48155b90 2024-04-25T08:52:57.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.648+0000 7fecf5ffb640 1 -- 172.21.15.37:0/1855397078 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fecf0054d00 con 0x7fed00098a20 2024-04-25T08:52:57.643 INFO:teuthology.orchestra.run.smithi037.stdout:55834574860 2024-04-25T08:52:57.643 INFO:teuthology.orchestra.run.smithi037.stdout:55834574858 2024-04-25T08:52:57.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 -- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f8c067030 msgr2=0x7f1f8c0694f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f8c067030 0x7f1f8c0694f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1f90000c10 tx=0x7f1f90008040 comp rx=0 tx=0).stop 2024-04-25T08:52:57.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 -- 172.21.15.37:0/2303748466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fa0071410 msgr2=0x7f1fa01589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fa0071410 0x7f1fa01589c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1fa00b2bb0 tx=0x7f1f9400c820 comp rx=0 tx=0).stop 2024-04-25T08:52:57.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 -- 172.21.15.37:0/2303748466 shutdown_connections 2024-04-25T08:52:57.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f8c067030 0x7f1f8c0694f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1fa0071e90 0x7f1fa015a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1fa0071410 0x7f1fa01589c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 --2- 172.21.15.37:0/2303748466 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1fa0071030 0x7f1fa014eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 -- 172.21.15.37:0/2303748466 >> 172.21.15.37:0/2303748466 conn(0x7f1fa00b11a0 msgr2=0x7f1fa00b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 -- 172.21.15.37:0/2303748466 shutdown_connections 2024-04-25T08:52:57.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f1fa624b640 1 -- 172.21.15.37:0/2303748466 wait complete. 2024-04-25T08:52:57.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f2c066f60 msgr2=0x7f2f2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f2c066f60 0x7f2f2c069420 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009b60 tx=0x7f2f44009190 comp rx=0 tx=0).stop 2024-04-25T08:52:57.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48155b90 msgr2=0x7f2f480b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48155b90 0x7f2f480b4d90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000caa0 tx=0x7f2f4000cf60 comp rx=0 tx=0).stop 2024-04-25T08:52:57.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1527791601 shutdown_connections 2024-04-25T08:52:57.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f2c066f60 0x7f2f2c069420 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48155b90 0x7f2f480b4d90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f481557b0 0x7f2f480b4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 --2- 172.21.15.37:0/1527791601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4806fd10 0x7f2f4806aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1527791601 >> 172.21.15.37:0/1527791601 conn(0x7f2f48072590 msgr2=0x7f2f480cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 -- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fece4066f60 msgr2=0x7fece4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fece4066f60 0x7fece4069420 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fed0006ed10 tx=0x7fecf800d040 comp rx=0 tx=0).stop 2024-04-25T08:52:57.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 -- 172.21.15.37:0/1855397078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed00098a20 msgr2=0x7fed000c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed00098a20 0x7fed000c8350 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fecf000c400 tx=0x7fecf000c8c0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1527791601 shutdown_connections 2024-04-25T08:52:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7f2f4f4ae640 1 -- 172.21.15.37:0/1527791601 wait complete. 2024-04-25T08:52:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 -- 172.21.15.37:0/1855397078 shutdown_connections 2024-04-25T08:52:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fece4066f60 0x7fece4069420 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed000cc020 0x7fed00156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed000b0990 0x7fed0015d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 --2- 172.21.15.37:0/1855397078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed00098a20 0x7fed000c8350 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 -- 172.21.15.37:0/1855397078 >> 172.21.15.37:0/1855397078 conn(0x7fed000b0df0 msgr2=0x7fed00159340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 -- 172.21.15.37:0/1855397078 shutdown_connections 2024-04-25T08:52:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fed06571640 1 -- 172.21.15.37:0/1855397078 wait complete. 2024-04-25T08:52:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fbce4392640 1 -- 172.21.15.37:0/272723987 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fbcc4005860 con 0x7fbcd40381e0 2024-04-25T08:52:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.652+0000 7fbcd2ffd640 1 -- 172.21.15.37:0/272723987 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbccc0559f0 con 0x7fbcd40381e0 2024-04-25T08:52:57.649 INFO:teuthology.orchestra.run.smithi037.stdout:55834574859 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.656+0000 7fbce4392640 1 -- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbcc0066f60 msgr2=0x7fbcc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.656+0000 7fbce4392640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbcc0066f60 0x7fbcc0069420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80b2000 tx=0x7fbcd8004740 comp rx=0 tx=0).stop 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.656+0000 7fbce4392640 1 -- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcd40381e0 msgr2=0x7fbcd40e48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.656+0000 7fbce4392640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcd40381e0 0x7fbcd40e48f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00c480 tx=0x7fbccc00c940 comp rx=0 tx=0).stop 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.656+0000 7fbce4392640 1 -- 172.21.15.37:0/272723987 shutdown_connections 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.656+0000 7fbce4392640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbcc0066f60 0x7fbcc0069420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.656+0000 7fbce4392640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbcd40e7700 0x7fbcd40d2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.660+0000 7fbce4392640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbcd4047d70 0x7fbcd40e9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.660+0000 7fbce4392640 1 --2- 172.21.15.37:0/272723987 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbcd40381e0 0x7fbcd40e48f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.660+0000 7fbce4392640 1 -- 172.21.15.37:0/272723987 >> 172.21.15.37:0/272723987 conn(0x7fbcd4054b00 msgr2=0x7fbcd404f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.660+0000 7fbce4392640 1 -- 172.21.15.37:0/272723987 shutdown_connections 2024-04-25T08:52:57.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.660+0000 7fbce4392640 1 -- 172.21.15.37:0/272723987 wait complete. 2024-04-25T08:52:57.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.668+0000 7f90653d4640 1 -- 172.21.15.37:0/1729121570 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9040005590 con 0x7f906015a8b0 2024-04-25T08:52:57.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.668+0000 7f390ce36640 1 -- 172.21.15.37:0/1310618681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f38ec005860 con 0x7f3900054db0 2024-04-25T08:52:57.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.668+0000 7f38f6ffd640 1 -- 172.21.15.37:0/1310618681 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f38fc0172c0 con 0x7f3900054db0 2024-04-25T08:52:57.663 INFO:teuthology.orchestra.run.smithi037.stdout:55834574858 2024-04-25T08:52:57.663 INFO:teuthology.orchestra.run.smithi037.stdout:55834574859 2024-04-25T08:52:57.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.668+0000 7f905cff9640 1 -- 172.21.15.37:0/1729121570 <== mon.2 v2:172.21.15.203:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9054055b20 con 0x7f906015a8b0 2024-04-25T08:52:57.664 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.0 2024-04-25T08:52:57.665 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:57.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7fc82ad4b640 1 -- 172.21.15.37:0/91319850 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc8240cec10 con 0x7fc8240cbff0 2024-04-25T08:52:57.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 -- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f904c066f60 msgr2=0x7f904c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f904c066f60 0x7f904c069420 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f905001d320 tx=0x7f9050008040 comp rx=0 tx=0).stop 2024-04-25T08:52:57.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 -- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a8b0 msgr2=0x7f90600c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a8b0 0x7f90600c59d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9054001ec0 tx=0x7f905400c870 comp rx=0 tx=0).stop 2024-04-25T08:52:57.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 -- 172.21.15.37:0/1729121570 shutdown_connections 2024-04-25T08:52:57.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f904c066f60 0x7f904c069420 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90601f7b70 0x7f9060158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f906015a8b0 0x7f90600c59d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 --2- 172.21.15.37:0/1729121570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90600ceac0 0x7f90601fbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 -- 172.21.15.37:0/1729121570 >> 172.21.15.37:0/1729121570 conn(0x7f90600b1d90 msgr2=0x7f906006df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 -- 172.21.15.37:0/1729121570 shutdown_connections 2024-04-25T08:52:57.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f90653d4640 1 -- 172.21.15.37:0/1729121570 wait complete. 2024-04-25T08:52:57.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7fc8217fa640 1 -- 172.21.15.37:0/91319850 <== mon.1 v2:172.21.15.203:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc810010e20 con 0x7fc8240cbff0 2024-04-25T08:52:57.668 INFO:teuthology.orchestra.run.smithi037.stdout:55834574859 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f390ce36640 1 -- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f38e8066f60 msgr2=0x7f38e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f390ce36640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f38e8066f60 0x7f38e8069420 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f390000c150 tx=0x7f3908121040 comp rx=0 tx=0).stop 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f390ce36640 1 -- 172.21.15.37:0/1310618681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3900054db0 msgr2=0x7f3900035770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f390ce36640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3900054db0 0x7f3900035770 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00b410 tx=0x7f38fc00b8d0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f390ce36640 1 -- 172.21.15.37:0/1310618681 shutdown_connections 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f390ce36640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f38e8066f60 0x7f38e8069420 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f390ce36640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3900054db0 0x7f3900035770 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f390ce36640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3900054950 0x7f39000e6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f390ce36640 1 --2- 172.21.15.37:0/1310618681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3900037510 0x7f390004fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f390ce36640 1 -- 172.21.15.37:0/1310618681 >> 172.21.15.37:0/1310618681 conn(0x7f3900020050 msgr2=0x7f390000ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.672+0000 7f390ce36640 1 -- 172.21.15.37:0/1310618681 shutdown_connections 2024-04-25T08:52:57.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7f390ce36640 1 -- 172.21.15.37:0/1310618681 wait complete. 2024-04-25T08:52:57.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 -- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc808066fb0 msgr2=0x7fc808069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc808066fb0 0x7fc808069470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc8180af000 tx=0x7fc818009040 comp rx=0 tx=0).stop 2024-04-25T08:52:57.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 -- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8240cbff0 msgr2=0x7fc824200720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8240cbff0 0x7fc824200720 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc81000b4d0 tx=0x7fc81000b990 comp rx=0 tx=0).stop 2024-04-25T08:52:57.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 -- 172.21.15.37:0/91319850 shutdown_connections 2024-04-25T08:52:57.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc808066fb0 0x7fc808069470 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc82415d050 0x7fc8240b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8240cbff0 0x7fc824200720 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 --2- 172.21.15.37:0/91319850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8240ca160 0x7fc8241557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 -- 172.21.15.37:0/91319850 >> 172.21.15.37:0/91319850 conn(0x7fc8240b0cf0 msgr2=0x7fc8241593f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 -- 172.21.15.37:0/91319850 shutdown_connections 2024-04-25T08:52:57.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.676+0000 7fc82ad4b640 1 -- 172.21.15.37:0/91319850 wait complete. 2024-04-25T08:52:57.680 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.2 2024-04-25T08:52:57.680 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:57.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.7 2024-04-25T08:52:57.681 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:57.681 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-04-25T08:52:57.681 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:57.685 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-04-25T08:52:57.685 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:57.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-25T08:52:57.693 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:57.699 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-25T08:52:57.700 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:57.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-04-25T08:52:57.700 DEBUG:teuthology.parallel:result is None 2024-04-25T08:52:57.701 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T08:52:57.701 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:52:57.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.856+0000 7f5204301640 1 Processor -- start 2024-04-25T08:52:57.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.856+0000 7f5204301640 1 -- start start 2024-04-25T08:52:57.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51fc144ca0 0x7f51fc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51fc06ec60 0x7f51fc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc0cc2b0 con 0x7f51fc06ec60 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc06e290 con 0x7f51fc144ca0 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0cee30 con 0x7f51fc0ce510 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520289e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51fc06ec60 0x7f51fc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520289e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51fc06ec60 0x7f51fc142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46654/0 (socket says 172.21.15.37:46654) 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520289e640 1 -- 172.21.15.37:0/2998256271 learned_addr learned my addr 172.21.15.37:0/2998256271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520209d640 1 --2- 172.21.15.37:0/2998256271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520189c640 1 --2- 172.21.15.37:0/2998256271 >> v2:172.21.15.203:3300/0 conn(0x7f51fc144ca0 0x7f51fc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520289e640 1 -- 172.21.15.37:0/2998256271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 msgr2=0x7f51fc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520289e640 1 --2- 172.21.15.37:0/2998256271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520289e640 1 -- 172.21.15.37:0/2998256271 >> v2:172.21.15.203:3300/0 conn(0x7f51fc144ca0 msgr2=0x7f51fc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520289e640 1 --2- 172.21.15.37:0/2998256271 >> v2:172.21.15.203:3300/0 conn(0x7f51fc144ca0 0x7f51fc06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520289e640 1 -- 172.21.15.37:0/2998256271 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc0ccc70 con 0x7f51fc06ec60 2024-04-25T08:52:57.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520289e640 1 --2- 172.21.15.37:0/2998256271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc06ec60 0x7f51fc142680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51fc06cfe0 tx=0x7f51f401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4302f6bfe5ad41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520109b640 1 -- 172.21.15.37:0/2998256271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40b4070 con 0x7f51fc06ec60 2024-04-25T08:52:57.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520109b640 1 -- 172.21.15.37:0/2998256271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f40af400 con 0x7f51fc06ec60 2024-04-25T08:52:57.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f520109b640 1 -- 172.21.15.37:0/2998256271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40af7d0 con 0x7f51fc06ec60 2024-04-25T08:52:57.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 -- 172.21.15.37:0/2998256271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc06ec60 msgr2=0x7f51fc142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 --2- 172.21.15.37:0/2998256271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc06ec60 0x7f51fc142680 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f51fc06cfe0 tx=0x7f51f401f1e0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 -- 172.21.15.37:0/2998256271 shutdown_connections 2024-04-25T08:52:57.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 --2- 172.21.15.37:0/2998256271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc06ec60 0x7f51fc142680 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 --2- 172.21.15.37:0/2998256271 >> v2:172.21.15.203:3300/0 conn(0x7f51fc144ca0 0x7f51fc06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 --2- 172.21.15.37:0/2998256271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 -- 172.21.15.37:0/2998256271 >> 172.21.15.37:0/2998256271 conn(0x7f51fc0b11a0 msgr2=0x7f51fc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:57.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 -- 172.21.15.37:0/2998256271 shutdown_connections 2024-04-25T08:52:57.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.860+0000 7f5204301640 1 -- 172.21.15.37:0/2998256271 wait complete. 2024-04-25T08:52:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f5204301640 1 Processor -- start 2024-04-25T08:52:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f5204301640 1 -- start start 2024-04-25T08:52:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f5204301640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51fc06ec60 0x7f51fc139be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f5204301640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc13b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f5204301640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc144ca0 0x7f51fc0adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f5204301640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0cc900 con 0x7f51fc144ca0 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f5204301640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc06df20 con 0x7f51fc06ec60 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f5204301640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc1451f0 con 0x7f51fc0ce510 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f520289e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc144ca0 0x7f51fc0adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f520289e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc144ca0 0x7f51fc0adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46658/0 (socket says 172.21.15.37:46658) 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f520289e640 1 -- 172.21.15.37:0/2493278380 learned_addr learned my addr 172.21.15.37:0/2493278380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f520189c640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc13b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f520209d640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51fc06ec60 0x7f51fc139be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f520289e640 1 -- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 msgr2=0x7f51fc13b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f520289e640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc13b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f520289e640 1 -- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51fc06ec60 msgr2=0x7f51fc139be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f520289e640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51fc06ec60 0x7f51fc139be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f520289e640 1 -- 172.21.15.37:0/2493278380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc0b0020 con 0x7f51fc144ca0 2024-04-25T08:52:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f520289e640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc144ca0 0x7f51fc0adeb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f51f40b2040 tx=0x7f51f401bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f51f2ffd640 1 -- 172.21.15.37:0/2493278380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40b4040 con 0x7f51fc144ca0 2024-04-25T08:52:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f51f2ffd640 1 -- 172.21.15.37:0/2493278380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f40c0070 con 0x7f51fc144ca0 2024-04-25T08:52:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f5204301640 1 -- 172.21.15.37:0/2493278380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc0b0300 con 0x7f51fc144ca0 2024-04-25T08:52:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f51f2ffd640 1 -- 172.21.15.37:0/2493278380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4009510 con 0x7f51fc144ca0 2024-04-25T08:52:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f5204301640 1 -- 172.21.15.37:0/2493278380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc0b08c0 con 0x7f51fc144ca0 2024-04-25T08:52:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f5204301640 1 -- 172.21.15.37:0/2493278380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc04efa0 con 0x7f51fc144ca0 2024-04-25T08:52:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.864+0000 7f51f2ffd640 1 -- 172.21.15.37:0/2493278380 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51f40c5460 con 0x7f51fc144ca0 2024-04-25T08:52:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.868+0000 7f51f2ffd640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e0066f60 0x7f51e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.868+0000 7f51f2ffd640 1 -- 172.21.15.37:0/2493278380 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f51f41235f0 con 0x7f51fc144ca0 2024-04-25T08:52:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.868+0000 7f520209d640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e0066f60 0x7f51e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.868+0000 7f520209d640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e0066f60 0x7f51e0069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009a70 tx=0x7f51f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:57.872+0000 7f51f2ffd640 1 -- 172.21.15.37:0/2493278380 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51f4018d40 con 0x7f51fc144ca0 2024-04-25T08:52:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.020+0000 7f5204301640 1 -- 172.21.15.37:0/2493278380 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51fc13a120 con 0x7f51e0066f60 2024-04-25T08:52:58.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.044+0000 7f51f2ffd640 1 -- 172.21.15.37:0/2493278380 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7f51fc13a120 con 0x7f51e0066f60 2024-04-25T08:52:58.039 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:52:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 -- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e0066f60 msgr2=0x7f51e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e0066f60 0x7f51e0069420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f51f8009a70 tx=0x7f51f8008040 comp rx=0 tx=0).stop 2024-04-25T08:52:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 -- 172.21.15.37:0/2493278380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc144ca0 msgr2=0x7f51fc0adeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc144ca0 0x7f51fc0adeb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f51f40b2040 tx=0x7f51f401bf80 comp rx=0 tx=0).stop 2024-04-25T08:52:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 -- 172.21.15.37:0/2493278380 shutdown_connections 2024-04-25T08:52:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e0066f60 0x7f51e0069420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc144ca0 0x7f51fc0adeb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc13b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 --2- 172.21.15.37:0/2493278380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51fc06ec60 0x7f51fc139be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 -- 172.21.15.37:0/2493278380 >> 172.21.15.37:0/2493278380 conn(0x7f51fc0b11a0 msgr2=0x7f51fc0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 -- 172.21.15.37:0/2493278380 shutdown_connections 2024-04-25T08:52:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.048+0000 7f5204301640 1 -- 172.21.15.37:0/2493278380 wait complete. 2024-04-25T08:52:58.056 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-25T08:52:57.127714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219668,"kb_used_data":3496,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755052,"statfs":{"total":773094113280,"available":772869173248,"internally_reserved":0,"allocated":3579904,"data_stored":2140384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005526"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836749+0000","last_change":"2024-04-25T08:52:30.238555+0000","last_active":"2024-04-25T08:52:39.836749+0000","last_peered":"2024-04-25T08:52:39.836749+0000","last_clean":"2024-04-25T08:52:39.836749+0000","last_became_active":"2024-04-25T08:52:30.238074+0000","last_became_peered":"2024-04-25T08:52:30.238074+0000","last_unstale":"2024-04-25T08:52:39.836749+0000","last_undegraded":"2024-04-25T08:52:39.836749+0000","last_fullsized":"2024-04-25T08:52:39.836749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:52.505423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834324+0000","last_change":"2024-04-25T08:52:31.237632+0000","last_active":"2024-04-25T08:52:39.834324+0000","last_peered":"2024-04-25T08:52:39.834324+0000","last_clean":"2024-04-25T08:52:39.834324+0000","last_became_active":"2024-04-25T08:52:31.237356+0000","last_became_peered":"2024-04-25T08:52:31.237356+0000","last_unstale":"2024-04-25T08:52:39.834324+0000","last_undegraded":"2024-04-25T08:52:39.834324+0000","last_fullsized":"2024-04-25T08:52:39.834324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833357+0000","last_change":"2024-04-25T08:52:36.008156+0000","last_active":"2024-04-25T08:52:39.833357+0000","last_peered":"2024-04-25T08:52:39.833357+0000","last_clean":"2024-04-25T08:52:39.833357+0000","last_became_active":"2024-04-25T08:52:36.006941+0000","last_became_peered":"2024-04-25T08:52:36.006941+0000","last_unstale":"2024-04-25T08:52:39.833357+0000","last_undegraded":"2024-04-25T08:52:39.833357+0000","last_fullsized":"2024-04-25T08:52:39.833357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:00.158176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834859+0000","last_change":"2024-04-25T08:52:30.225034+0000","last_active":"2024-04-25T08:52:39.834859+0000","last_peered":"2024-04-25T08:52:39.834859+0000","last_clean":"2024-04-25T08:52:39.834859+0000","last_became_active":"2024-04-25T08:52:30.224727+0000","last_became_peered":"2024-04-25T08:52:30.224727+0000","last_unstale":"2024-04-25T08:52:39.834859+0000","last_undegraded":"2024-04-25T08:52:39.834859+0000","last_fullsized":"2024-04-25T08:52:39.834859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836491+0000","last_change":"2024-04-25T08:52:31.256496+0000","last_active":"2024-04-25T08:52:39.836491+0000","last_peered":"2024-04-25T08:52:39.836491+0000","last_clean":"2024-04-25T08:52:39.836491+0000","last_became_active":"2024-04-25T08:52:31.255998+0000","last_became_peered":"2024-04-25T08:52:31.255998+0000","last_unstale":"2024-04-25T08:52:39.836491+0000","last_undegraded":"2024-04-25T08:52:39.836491+0000","last_fullsized":"2024-04-25T08:52:39.836491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299627+0000","last_change":"2024-04-25T08:52:35.324509+0000","last_active":"2024-04-25T08:52:38.299627+0000","last_peered":"2024-04-25T08:52:38.299627+0000","last_clean":"2024-04-25T08:52:38.299627+0000","last_became_active":"2024-04-25T08:52:35.323398+0000","last_became_peered":"2024-04-25T08:52:35.323398+0000","last_unstale":"2024-04-25T08:52:38.299627+0000","last_undegraded":"2024-04-25T08:52:38.299627+0000","last_fullsized":"2024-04-25T08:52:38.299627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:56.088772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307843+0000","last_change":"2024-04-25T08:52:30.248641+0000","last_active":"2024-04-25T08:52:39.307843+0000","last_peered":"2024-04-25T08:52:39.307843+0000","last_clean":"2024-04-25T08:52:39.307843+0000","last_became_active":"2024-04-25T08:52:30.245183+0000","last_became_peered":"2024-04-25T08:52:30.245183+0000","last_unstale":"2024-04-25T08:52:39.307843+0000","last_undegraded":"2024-04-25T08:52:39.307843+0000","last_fullsized":"2024-04-25T08:52:39.307843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:46.012895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644062+0000","last_change":"2024-04-25T08:52:31.272777+0000","last_active":"2024-04-25T08:52:39.644062+0000","last_peered":"2024-04-25T08:52:39.644062+0000","last_clean":"2024-04-25T08:52:39.644062+0000","last_became_active":"2024-04-25T08:52:31.271932+0000","last_became_peered":"2024-04-25T08:52:31.271932+0000","last_unstale":"2024-04-25T08:52:39.644062+0000","last_undegraded":"2024-04-25T08:52:39.644062+0000","last_fullsized":"2024-04-25T08:52:39.644062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300302+0000","last_change":"2024-04-25T08:52:35.330473+0000","last_active":"2024-04-25T08:52:38.300302+0000","last_peered":"2024-04-25T08:52:38.300302+0000","last_clean":"2024-04-25T08:52:38.300302+0000","last_became_active":"2024-04-25T08:52:35.328741+0000","last_became_peered":"2024-04-25T08:52:35.328741+0000","last_unstale":"2024-04-25T08:52:38.300302+0000","last_undegraded":"2024-04-25T08:52:38.300302+0000","last_fullsized":"2024-04-25T08:52:38.300302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:14.472581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307492+0000","last_change":"2024-04-25T08:52:30.245422+0000","last_active":"2024-04-25T08:52:39.307492+0000","last_peered":"2024-04-25T08:52:39.307492+0000","last_clean":"2024-04-25T08:52:39.307492+0000","last_became_active":"2024-04-25T08:52:30.244911+0000","last_became_peered":"2024-04-25T08:52:30.244911+0000","last_unstale":"2024-04-25T08:52:39.307492+0000","last_undegraded":"2024-04-25T08:52:39.307492+0000","last_fullsized":"2024-04-25T08:52:39.307492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644128+0000","last_change":"2024-04-25T08:52:31.274339+0000","last_active":"2024-04-25T08:52:39.644128+0000","last_peered":"2024-04-25T08:52:39.644128+0000","last_clean":"2024-04-25T08:52:39.644128+0000","last_became_active":"2024-04-25T08:52:31.273607+0000","last_became_peered":"2024-04-25T08:52:31.273607+0000","last_unstale":"2024-04-25T08:52:39.644128+0000","last_undegraded":"2024-04-25T08:52:39.644128+0000","last_fullsized":"2024-04-25T08:52:39.644128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:56.409407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299563+0000","last_change":"2024-04-25T08:52:35.316303+0000","last_active":"2024-04-25T08:52:38.299563+0000","last_peered":"2024-04-25T08:52:38.299563+0000","last_clean":"2024-04-25T08:52:38.299563+0000","last_became_active":"2024-04-25T08:52:35.315156+0000","last_became_peered":"2024-04-25T08:52:35.315156+0000","last_unstale":"2024-04-25T08:52:38.299563+0000","last_undegraded":"2024-04-25T08:52:38.299563+0000","last_fullsized":"2024-04-25T08:52:38.299563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:35.695987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300591+0000","last_change":"2024-04-25T08:52:31.269860+0000","last_active":"2024-04-25T08:52:38.300591+0000","last_peered":"2024-04-25T08:52:38.300591+0000","last_clean":"2024-04-25T08:52:38.300591+0000","last_became_active":"2024-04-25T08:52:31.268731+0000","last_became_peered":"2024-04-25T08:52:31.268731+0000","last_unstale":"2024-04-25T08:52:38.300591+0000","last_undegraded":"2024-04-25T08:52:38.300591+0000","last_fullsized":"2024-04-25T08:52:38.300591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307114+0000","last_change":"2024-04-25T08:52:30.246021+0000","last_active":"2024-04-25T08:52:39.307114+0000","last_peered":"2024-04-25T08:52:39.307114+0000","last_clean":"2024-04-25T08:52:39.307114+0000","last_became_active":"2024-04-25T08:52:30.245132+0000","last_became_peered":"2024-04-25T08:52:30.245132+0000","last_unstale":"2024-04-25T08:52:39.307114+0000","last_undegraded":"2024-04-25T08:52:39.307114+0000","last_fullsized":"2024-04-25T08:52:39.307114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:11.290953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301681+0000","last_change":"2024-04-25T08:52:35.323528+0000","last_active":"2024-04-25T08:52:38.301681+0000","last_peered":"2024-04-25T08:52:38.301681+0000","last_clean":"2024-04-25T08:52:38.301681+0000","last_became_active":"2024-04-25T08:52:35.320911+0000","last_became_peered":"2024-04-25T08:52:35.320911+0000","last_unstale":"2024-04-25T08:52:38.301681+0000","last_undegraded":"2024-04-25T08:52:38.301681+0000","last_fullsized":"2024-04-25T08:52:38.301681+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835879+0000","last_change":"2024-04-25T08:52:31.257885+0000","last_active":"2024-04-25T08:52:39.835879+0000","last_peered":"2024-04-25T08:52:39.835879+0000","last_clean":"2024-04-25T08:52:39.835879+0000","last_became_active":"2024-04-25T08:52:31.257583+0000","last_became_peered":"2024-04-25T08:52:31.257583+0000","last_unstale":"2024-04-25T08:52:39.835879+0000","last_undegraded":"2024-04-25T08:52:39.835879+0000","last_fullsized":"2024-04-25T08:52:39.835879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834075+0000","last_change":"2024-04-25T08:52:30.213612+0000","last_active":"2024-04-25T08:52:39.834075+0000","last_peered":"2024-04-25T08:52:39.834075+0000","last_clean":"2024-04-25T08:52:39.834075+0000","last_became_active":"2024-04-25T08:52:30.212962+0000","last_became_peered":"2024-04-25T08:52:30.212962+0000","last_unstale":"2024-04-25T08:52:39.834075+0000","last_undegraded":"2024-04-25T08:52:39.834075+0000","last_fullsized":"2024-04-25T08:52:39.834075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300390+0000","last_change":"2024-04-25T08:52:35.324597+0000","last_active":"2024-04-25T08:52:38.300390+0000","last_peered":"2024-04-25T08:52:38.300390+0000","last_clean":"2024-04-25T08:52:38.300390+0000","last_became_active":"2024-04-25T08:52:35.323534+0000","last_became_peered":"2024-04-25T08:52:35.323534+0000","last_unstale":"2024-04-25T08:52:38.300390+0000","last_undegraded":"2024-04-25T08:52:38.300390+0000","last_fullsized":"2024-04-25T08:52:38.300390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:24.182927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834456+0000","last_change":"2024-04-25T08:52:31.231520+0000","last_active":"2024-04-25T08:52:39.834456+0000","last_peered":"2024-04-25T08:52:39.834456+0000","last_clean":"2024-04-25T08:52:39.834456+0000","last_became_active":"2024-04-25T08:52:31.231009+0000","last_became_peered":"2024-04-25T08:52:31.231009+0000","last_unstale":"2024-04-25T08:52:39.834456+0000","last_undegraded":"2024-04-25T08:52:39.834456+0000","last_fullsized":"2024-04-25T08:52:39.834456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834462+0000","last_change":"2024-04-25T08:52:30.231226+0000","last_active":"2024-04-25T08:52:39.834462+0000","last_peered":"2024-04-25T08:52:39.834462+0000","last_clean":"2024-04-25T08:52:39.834462+0000","last_became_active":"2024-04-25T08:52:30.230790+0000","last_became_peered":"2024-04-25T08:52:30.230790+0000","last_unstale":"2024-04-25T08:52:39.834462+0000","last_undegraded":"2024-04-25T08:52:39.834462+0000","last_fullsized":"2024-04-25T08:52:39.834462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:11.290953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834915+0000","last_change":"2024-04-25T08:52:36.004737+0000","last_active":"2024-04-25T08:52:39.834915+0000","last_peered":"2024-04-25T08:52:39.834915+0000","last_clean":"2024-04-25T08:52:39.834915+0000","last_became_active":"2024-04-25T08:52:36.003639+0000","last_became_peered":"2024-04-25T08:52:36.003639+0000","last_unstale":"2024-04-25T08:52:39.834915+0000","last_undegraded":"2024-04-25T08:52:39.834915+0000","last_fullsized":"2024-04-25T08:52:39.834915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301223+0000","last_change":"2024-04-25T08:52:31.260426+0000","last_active":"2024-04-25T08:52:38.301223+0000","last_peered":"2024-04-25T08:52:38.301223+0000","last_clean":"2024-04-25T08:52:38.301223+0000","last_became_active":"2024-04-25T08:52:31.259594+0000","last_became_peered":"2024-04-25T08:52:31.259594+0000","last_unstale":"2024-04-25T08:52:38.301223+0000","last_undegraded":"2024-04-25T08:52:38.301223+0000","last_fullsized":"2024-04-25T08:52:38.301223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:45.962867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306464+0000","last_change":"2024-04-25T08:52:30.240896+0000","last_active":"2024-04-25T08:52:39.306464+0000","last_peered":"2024-04-25T08:52:39.306464+0000","last_clean":"2024-04-25T08:52:39.306464+0000","last_became_active":"2024-04-25T08:52:30.240290+0000","last_became_peered":"2024-04-25T08:52:30.240290+0000","last_unstale":"2024-04-25T08:52:39.306464+0000","last_undegraded":"2024-04-25T08:52:39.306464+0000","last_fullsized":"2024-04-25T08:52:39.306464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300735+0000","last_change":"2024-04-25T08:52:35.328692+0000","last_active":"2024-04-25T08:52:38.300735+0000","last_peered":"2024-04-25T08:52:38.300735+0000","last_clean":"2024-04-25T08:52:38.300735+0000","last_became_active":"2024-04-25T08:52:35.323899+0000","last_became_peered":"2024-04-25T08:52:35.323899+0000","last_unstale":"2024-04-25T08:52:38.300735+0000","last_undegraded":"2024-04-25T08:52:38.300735+0000","last_fullsized":"2024-04-25T08:52:38.300735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836941+0000","last_change":"2024-04-25T08:52:31.253016+0000","last_active":"2024-04-25T08:52:39.836941+0000","last_peered":"2024-04-25T08:52:39.836941+0000","last_clean":"2024-04-25T08:52:39.836941+0000","last_became_active":"2024-04-25T08:52:31.252369+0000","last_became_peered":"2024-04-25T08:52:31.252369+0000","last_unstale":"2024-04-25T08:52:39.836941+0000","last_undegraded":"2024-04-25T08:52:39.836941+0000","last_fullsized":"2024-04-25T08:52:39.836941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303292+0000","last_change":"2024-04-25T08:52:30.241679+0000","last_active":"2024-04-25T08:52:38.303292+0000","last_peered":"2024-04-25T08:52:38.303292+0000","last_clean":"2024-04-25T08:52:38.303292+0000","last_became_active":"2024-04-25T08:52:30.241063+0000","last_became_peered":"2024-04-25T08:52:30.241063+0000","last_unstale":"2024-04-25T08:52:38.303292+0000","last_undegraded":"2024-04-25T08:52:38.303292+0000","last_fullsized":"2024-04-25T08:52:38.303292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300540+0000","last_change":"2024-04-25T08:52:36.007015+0000","last_active":"2024-04-25T08:52:38.300540+0000","last_peered":"2024-04-25T08:52:38.300540+0000","last_clean":"2024-04-25T08:52:38.300540+0000","last_became_active":"2024-04-25T08:52:36.005832+0000","last_became_peered":"2024-04-25T08:52:36.005832+0000","last_unstale":"2024-04-25T08:52:38.300540+0000","last_undegraded":"2024-04-25T08:52:38.300540+0000","last_fullsized":"2024-04-25T08:52:38.300540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:19.422348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834979+0000","last_change":"2024-04-25T08:52:31.282175+0000","last_active":"2024-04-25T08:52:39.834979+0000","last_peered":"2024-04-25T08:52:39.834979+0000","last_clean":"2024-04-25T08:52:39.834979+0000","last_became_active":"2024-04-25T08:52:31.281955+0000","last_became_peered":"2024-04-25T08:52:31.281955+0000","last_unstale":"2024-04-25T08:52:39.834979+0000","last_undegraded":"2024-04-25T08:52:39.834979+0000","last_fullsized":"2024-04-25T08:52:39.834979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834733+0000","last_change":"2024-04-25T08:52:30.215220+0000","last_active":"2024-04-25T08:52:39.834733+0000","last_peered":"2024-04-25T08:52:39.834733+0000","last_clean":"2024-04-25T08:52:39.834733+0000","last_became_active":"2024-04-25T08:52:30.214773+0000","last_became_peered":"2024-04-25T08:52:30.214773+0000","last_unstale":"2024-04-25T08:52:39.834733+0000","last_undegraded":"2024-04-25T08:52:39.834733+0000","last_fullsized":"2024-04-25T08:52:39.834733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833949+0000","last_change":"2024-04-25T08:52:36.003489+0000","last_active":"2024-04-25T08:52:39.833949+0000","last_peered":"2024-04-25T08:52:39.833949+0000","last_clean":"2024-04-25T08:52:39.833949+0000","last_became_active":"2024-04-25T08:52:36.002295+0000","last_became_peered":"2024-04-25T08:52:36.002295+0000","last_unstale":"2024-04-25T08:52:39.833949+0000","last_undegraded":"2024-04-25T08:52:39.833949+0000","last_fullsized":"2024-04-25T08:52:39.833949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:47.463029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833050+0000","last_change":"2024-04-25T08:52:31.226718+0000","last_active":"2024-04-25T08:52:39.833050+0000","last_peered":"2024-04-25T08:52:39.833050+0000","last_clean":"2024-04-25T08:52:39.833050+0000","last_became_active":"2024-04-25T08:52:31.226442+0000","last_became_peered":"2024-04-25T08:52:31.226442+0000","last_unstale":"2024-04-25T08:52:39.833050+0000","last_undegraded":"2024-04-25T08:52:39.833050+0000","last_fullsized":"2024-04-25T08:52:39.833050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307955+0000","last_change":"2024-04-25T08:52:30.249239+0000","last_active":"2024-04-25T08:52:39.307955+0000","last_peered":"2024-04-25T08:52:39.307955+0000","last_clean":"2024-04-25T08:52:39.307955+0000","last_became_active":"2024-04-25T08:52:30.248612+0000","last_became_peered":"2024-04-25T08:52:30.248612+0000","last_unstale":"2024-04-25T08:52:39.307955+0000","last_undegraded":"2024-04-25T08:52:39.307955+0000","last_fullsized":"2024-04-25T08:52:39.307955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:34.198893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836268+0000","last_change":"2024-04-25T08:52:36.001292+0000","last_active":"2024-04-25T08:52:39.836268+0000","last_peered":"2024-04-25T08:52:39.836268+0000","last_clean":"2024-04-25T08:52:39.836268+0000","last_became_active":"2024-04-25T08:52:36.000113+0000","last_became_peered":"2024-04-25T08:52:36.000113+0000","last_unstale":"2024-04-25T08:52:39.836268+0000","last_undegraded":"2024-04-25T08:52:39.836268+0000","last_fullsized":"2024-04-25T08:52:39.836268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645594+0000","last_change":"2024-04-25T08:52:31.247003+0000","last_active":"2024-04-25T08:52:39.645594+0000","last_peered":"2024-04-25T08:52:39.645594+0000","last_clean":"2024-04-25T08:52:39.645594+0000","last_became_active":"2024-04-25T08:52:31.246200+0000","last_became_peered":"2024-04-25T08:52:31.246200+0000","last_unstale":"2024-04-25T08:52:39.645594+0000","last_undegraded":"2024-04-25T08:52:39.645594+0000","last_fullsized":"2024-04-25T08:52:39.645594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:52.007683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835389+0000","last_change":"2024-04-25T08:52:30.215091+0000","last_active":"2024-04-25T08:52:39.835389+0000","last_peered":"2024-04-25T08:52:39.835389+0000","last_clean":"2024-04-25T08:52:39.835389+0000","last_became_active":"2024-04-25T08:52:30.214664+0000","last_became_peered":"2024-04-25T08:52:30.214664+0000","last_unstale":"2024-04-25T08:52:39.835389+0000","last_undegraded":"2024-04-25T08:52:39.835389+0000","last_fullsized":"2024-04-25T08:52:39.835389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833696+0000","last_change":"2024-04-25T08:52:35.313874+0000","last_active":"2024-04-25T08:52:39.833696+0000","last_peered":"2024-04-25T08:52:39.833696+0000","last_clean":"2024-04-25T08:52:39.833696+0000","last_became_active":"2024-04-25T08:52:35.313429+0000","last_became_peered":"2024-04-25T08:52:35.313429+0000","last_unstale":"2024-04-25T08:52:39.833696+0000","last_undegraded":"2024-04-25T08:52:39.833696+0000","last_fullsized":"2024-04-25T08:52:39.833696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:59.165277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300917+0000","last_change":"2024-04-25T08:52:31.257574+0000","last_active":"2024-04-25T08:52:38.300917+0000","last_peered":"2024-04-25T08:52:38.300917+0000","last_clean":"2024-04-25T08:52:38.300917+0000","last_became_active":"2024-04-25T08:52:31.257040+0000","last_became_peered":"2024-04-25T08:52:31.257040+0000","last_unstale":"2024-04-25T08:52:38.300917+0000","last_undegraded":"2024-04-25T08:52:38.300917+0000","last_fullsized":"2024-04-25T08:52:38.300917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:52.221264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645682+0000","last_change":"2024-04-25T08:52:30.225801+0000","last_active":"2024-04-25T08:52:39.645682+0000","last_peered":"2024-04-25T08:52:39.645682+0000","last_clean":"2024-04-25T08:52:39.645682+0000","last_became_active":"2024-04-25T08:52:30.224857+0000","last_became_peered":"2024-04-25T08:52:30.224857+0000","last_unstale":"2024-04-25T08:52:39.645682+0000","last_undegraded":"2024-04-25T08:52:39.645682+0000","last_fullsized":"2024-04-25T08:52:39.645682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833379+0000","last_change":"2024-04-25T08:52:35.309762+0000","last_active":"2024-04-25T08:52:39.833379+0000","last_peered":"2024-04-25T08:52:39.833379+0000","last_clean":"2024-04-25T08:52:39.833379+0000","last_became_active":"2024-04-25T08:52:35.309315+0000","last_became_peered":"2024-04-25T08:52:35.309315+0000","last_unstale":"2024-04-25T08:52:39.833379+0000","last_undegraded":"2024-04-25T08:52:39.833379+0000","last_fullsized":"2024-04-25T08:52:39.833379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.832782+0000","last_change":"2024-04-25T08:52:31.232199+0000","last_active":"2024-04-25T08:52:39.832782+0000","last_peered":"2024-04-25T08:52:39.832782+0000","last_clean":"2024-04-25T08:52:39.832782+0000","last_became_active":"2024-04-25T08:52:31.231928+0000","last_became_peered":"2024-04-25T08:52:31.231928+0000","last_unstale":"2024-04-25T08:52:39.832782+0000","last_undegraded":"2024-04-25T08:52:39.832782+0000","last_fullsized":"2024-04-25T08:52:39.832782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299166+0000","last_change":"2024-04-25T08:52:30.218615+0000","last_active":"2024-04-25T08:52:38.299166+0000","last_peered":"2024-04-25T08:52:38.299166+0000","last_clean":"2024-04-25T08:52:38.299166+0000","last_became_active":"2024-04-25T08:52:30.213378+0000","last_became_peered":"2024-04-25T08:52:30.213378+0000","last_unstale":"2024-04-25T08:52:38.299166+0000","last_undegraded":"2024-04-25T08:52:38.299166+0000","last_fullsized":"2024-04-25T08:52:38.299166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834271+0000","last_change":"2024-04-25T08:52:35.311156+0000","last_active":"2024-04-25T08:52:39.834271+0000","last_peered":"2024-04-25T08:52:39.834271+0000","last_clean":"2024-04-25T08:52:39.834271+0000","last_became_active":"2024-04-25T08:52:35.310700+0000","last_became_peered":"2024-04-25T08:52:35.310700+0000","last_unstale":"2024-04-25T08:52:39.834271+0000","last_undegraded":"2024-04-25T08:52:39.834271+0000","last_fullsized":"2024-04-25T08:52:39.834271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.832874+0000","last_change":"2024-04-25T08:52:31.260514+0000","last_active":"2024-04-25T08:52:39.832874+0000","last_peered":"2024-04-25T08:52:39.832874+0000","last_clean":"2024-04-25T08:52:39.832874+0000","last_became_active":"2024-04-25T08:52:31.260019+0000","last_became_peered":"2024-04-25T08:52:31.260019+0000","last_unstale":"2024-04-25T08:52:39.832874+0000","last_undegraded":"2024-04-25T08:52:39.832874+0000","last_fullsized":"2024-04-25T08:52:39.832874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.602114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306466+0000","last_change":"2024-04-25T08:52:30.245223+0000","last_active":"2024-04-25T08:52:39.306466+0000","last_peered":"2024-04-25T08:52:39.306466+0000","last_clean":"2024-04-25T08:52:39.306466+0000","last_became_active":"2024-04-25T08:52:30.244782+0000","last_became_peered":"2024-04-25T08:52:30.244782+0000","last_unstale":"2024-04-25T08:52:39.306466+0000","last_undegraded":"2024-04-25T08:52:39.306466+0000","last_fullsized":"2024-04-25T08:52:39.306466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836419+0000","last_change":"2024-04-25T08:52:36.002202+0000","last_active":"2024-04-25T08:52:39.836419+0000","last_peered":"2024-04-25T08:52:39.836419+0000","last_clean":"2024-04-25T08:52:39.836419+0000","last_became_active":"2024-04-25T08:52:36.000348+0000","last_became_peered":"2024-04-25T08:52:36.000348+0000","last_unstale":"2024-04-25T08:52:39.836419+0000","last_undegraded":"2024-04-25T08:52:39.836419+0000","last_fullsized":"2024-04-25T08:52:39.836419+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:12.373033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834845+0000","last_change":"2024-04-25T08:52:31.257777+0000","last_active":"2024-04-25T08:52:39.834845+0000","last_peered":"2024-04-25T08:52:39.834845+0000","last_clean":"2024-04-25T08:52:39.834845+0000","last_became_active":"2024-04-25T08:52:31.257295+0000","last_became_peered":"2024-04-25T08:52:31.257295+0000","last_unstale":"2024-04-25T08:52:39.834845+0000","last_undegraded":"2024-04-25T08:52:39.834845+0000","last_fullsized":"2024-04-25T08:52:39.834845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645136+0000","last_change":"2024-04-25T08:52:30.222595+0000","last_active":"2024-04-25T08:52:39.645136+0000","last_peered":"2024-04-25T08:52:39.645136+0000","last_clean":"2024-04-25T08:52:39.645136+0000","last_became_active":"2024-04-25T08:52:30.217788+0000","last_became_peered":"2024-04-25T08:52:30.217788+0000","last_unstale":"2024-04-25T08:52:39.645136+0000","last_undegraded":"2024-04-25T08:52:39.645136+0000","last_fullsized":"2024-04-25T08:52:39.645136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:52.505423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306756+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:52:39.306756+0000","last_peered":"2024-04-25T08:52:39.306756+0000","last_clean":"2024-04-25T08:52:39.306756+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:52:39.306756+0000","last_undegraded":"2024-04-25T08:52:39.306756+0000","last_fullsized":"2024-04-25T08:52:39.306756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.302216+0000","last_change":"2024-04-25T08:52:36.004910+0000","last_active":"2024-04-25T08:52:38.302216+0000","last_peered":"2024-04-25T08:52:38.302216+0000","last_clean":"2024-04-25T08:52:38.302216+0000","last_became_active":"2024-04-25T08:52:36.003878+0000","last_became_peered":"2024-04-25T08:52:36.003878+0000","last_unstale":"2024-04-25T08:52:38.302216+0000","last_undegraded":"2024-04-25T08:52:38.302216+0000","last_fullsized":"2024-04-25T08:52:38.302216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:12.106880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307624+0000","last_change":"2024-04-25T08:52:31.286252+0000","last_active":"2024-04-25T08:52:39.307624+0000","last_peered":"2024-04-25T08:52:39.307624+0000","last_clean":"2024-04-25T08:52:39.307624+0000","last_became_active":"2024-04-25T08:52:31.282622+0000","last_became_peered":"2024-04-25T08:52:31.282622+0000","last_unstale":"2024-04-25T08:52:39.307624+0000","last_undegraded":"2024-04-25T08:52:39.307624+0000","last_fullsized":"2024-04-25T08:52:39.307624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299865+0000","last_change":"2024-04-25T08:52:30.226514+0000","last_active":"2024-04-25T08:52:38.299865+0000","last_peered":"2024-04-25T08:52:38.299865+0000","last_clean":"2024-04-25T08:52:38.299865+0000","last_became_active":"2024-04-25T08:52:30.226243+0000","last_became_peered":"2024-04-25T08:52:30.226243+0000","last_unstale":"2024-04-25T08:52:38.299865+0000","last_undegraded":"2024-04-25T08:52:38.299865+0000","last_fullsized":"2024-04-25T08:52:38.299865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300450+0000","last_change":"2024-04-25T08:52:36.003056+0000","last_active":"2024-04-25T08:52:38.300450+0000","last_peered":"2024-04-25T08:52:38.300450+0000","last_clean":"2024-04-25T08:52:38.300450+0000","last_became_active":"2024-04-25T08:52:36.000882+0000","last_became_peered":"2024-04-25T08:52:36.000882+0000","last_unstale":"2024-04-25T08:52:38.300450+0000","last_undegraded":"2024-04-25T08:52:38.300450+0000","last_fullsized":"2024-04-25T08:52:38.300450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:56.088772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645689+0000","last_change":"2024-04-25T08:52:31.244866+0000","last_active":"2024-04-25T08:52:39.645689+0000","last_peered":"2024-04-25T08:52:39.645689+0000","last_clean":"2024-04-25T08:52:39.645689+0000","last_became_active":"2024-04-25T08:52:31.244249+0000","last_became_peered":"2024-04-25T08:52:31.244249+0000","last_unstale":"2024-04-25T08:52:39.645689+0000","last_undegraded":"2024-04-25T08:52:39.645689+0000","last_fullsized":"2024-04-25T08:52:39.645689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835210+0000","last_change":"2024-04-25T08:52:30.228380+0000","last_active":"2024-04-25T08:52:39.835210+0000","last_peered":"2024-04-25T08:52:39.835210+0000","last_clean":"2024-04-25T08:52:39.835210+0000","last_became_active":"2024-04-25T08:52:30.227732+0000","last_became_peered":"2024-04-25T08:52:30.227732+0000","last_unstale":"2024-04-25T08:52:39.835210+0000","last_undegraded":"2024-04-25T08:52:39.835210+0000","last_fullsized":"2024-04-25T08:52:39.835210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:46.012895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836520+0000","last_change":"2024-04-25T08:52:35.316462+0000","last_active":"2024-04-25T08:52:39.836520+0000","last_peered":"2024-04-25T08:52:39.836520+0000","last_clean":"2024-04-25T08:52:39.836520+0000","last_became_active":"2024-04-25T08:52:35.315760+0000","last_became_peered":"2024-04-25T08:52:35.315760+0000","last_unstale":"2024-04-25T08:52:39.836520+0000","last_undegraded":"2024-04-25T08:52:39.836520+0000","last_fullsized":"2024-04-25T08:52:39.836520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:49.973872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305761+0000","last_change":"2024-04-25T08:52:31.289534+0000","last_active":"2024-04-25T08:52:39.305761+0000","last_peered":"2024-04-25T08:52:39.305761+0000","last_clean":"2024-04-25T08:52:39.305761+0000","last_became_active":"2024-04-25T08:52:31.288580+0000","last_became_peered":"2024-04-25T08:52:31.288580+0000","last_unstale":"2024-04-25T08:52:39.305761+0000","last_undegraded":"2024-04-25T08:52:39.305761+0000","last_fullsized":"2024-04-25T08:52:39.305761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:10.391492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299950+0000","last_change":"2024-04-25T08:52:30.215006+0000","last_active":"2024-04-25T08:52:38.299950+0000","last_peered":"2024-04-25T08:52:38.299950+0000","last_clean":"2024-04-25T08:52:38.299950+0000","last_became_active":"2024-04-25T08:52:30.213814+0000","last_became_peered":"2024-04-25T08:52:30.213814+0000","last_unstale":"2024-04-25T08:52:38.299950+0000","last_undegraded":"2024-04-25T08:52:38.299950+0000","last_fullsized":"2024-04-25T08:52:38.299950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644315+0000","last_change":"2024-04-25T08:52:36.005410+0000","last_active":"2024-04-25T08:52:39.644315+0000","last_peered":"2024-04-25T08:52:39.644315+0000","last_clean":"2024-04-25T08:52:39.644315+0000","last_became_active":"2024-04-25T08:52:36.004309+0000","last_became_peered":"2024-04-25T08:52:36.004309+0000","last_unstale":"2024-04-25T08:52:39.644315+0000","last_undegraded":"2024-04-25T08:52:39.644315+0000","last_fullsized":"2024-04-25T08:52:39.644315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305320+0000","last_change":"2024-04-25T08:52:31.285984+0000","last_active":"2024-04-25T08:52:39.305320+0000","last_peered":"2024-04-25T08:52:39.305320+0000","last_clean":"2024-04-25T08:52:39.305320+0000","last_became_active":"2024-04-25T08:52:31.281988+0000","last_became_peered":"2024-04-25T08:52:31.281988+0000","last_unstale":"2024-04-25T08:52:39.305320+0000","last_undegraded":"2024-04-25T08:52:39.305320+0000","last_fullsized":"2024-04-25T08:52:39.305320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:56.409407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301053+0000","last_change":"2024-04-25T08:52:30.215598+0000","last_active":"2024-04-25T08:52:38.301053+0000","last_peered":"2024-04-25T08:52:38.301053+0000","last_clean":"2024-04-25T08:52:38.301053+0000","last_became_active":"2024-04-25T08:52:30.214765+0000","last_became_peered":"2024-04-25T08:52:30.214765+0000","last_unstale":"2024-04-25T08:52:38.301053+0000","last_undegraded":"2024-04-25T08:52:38.301053+0000","last_fullsized":"2024-04-25T08:52:38.301053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:36.565944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300087+0000","last_change":"2024-04-25T08:52:35.328574+0000","last_active":"2024-04-25T08:52:38.300087+0000","last_peered":"2024-04-25T08:52:38.300087+0000","last_clean":"2024-04-25T08:52:38.300087+0000","last_became_active":"2024-04-25T08:52:35.323891+0000","last_became_peered":"2024-04-25T08:52:35.323891+0000","last_unstale":"2024-04-25T08:52:38.300087+0000","last_undegraded":"2024-04-25T08:52:38.300087+0000","last_fullsized":"2024-04-25T08:52:38.300087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:12.106880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305125+0000","last_change":"2024-04-25T08:52:31.278037+0000","last_active":"2024-04-25T08:52:39.305125+0000","last_peered":"2024-04-25T08:52:39.305125+0000","last_clean":"2024-04-25T08:52:39.305125+0000","last_became_active":"2024-04-25T08:52:31.277117+0000","last_became_peered":"2024-04-25T08:52:31.277117+0000","last_unstale":"2024-04-25T08:52:39.305125+0000","last_undegraded":"2024-04-25T08:52:39.305125+0000","last_fullsized":"2024-04-25T08:52:39.305125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298797+0000","last_change":"2024-04-25T08:52:30.221811+0000","last_active":"2024-04-25T08:52:38.298797+0000","last_peered":"2024-04-25T08:52:38.298797+0000","last_clean":"2024-04-25T08:52:38.298797+0000","last_became_active":"2024-04-25T08:52:30.220494+0000","last_became_peered":"2024-04-25T08:52:30.220494+0000","last_unstale":"2024-04-25T08:52:38.298797+0000","last_undegraded":"2024-04-25T08:52:38.298797+0000","last_fullsized":"2024-04-25T08:52:38.298797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836281+0000","last_change":"2024-04-25T08:52:35.317037+0000","last_active":"2024-04-25T08:52:39.836281+0000","last_peered":"2024-04-25T08:52:39.836281+0000","last_clean":"2024-04-25T08:52:39.836281+0000","last_became_active":"2024-04-25T08:52:35.316491+0000","last_became_peered":"2024-04-25T08:52:35.316491+0000","last_unstale":"2024-04-25T08:52:39.836281+0000","last_undegraded":"2024-04-25T08:52:39.836281+0000","last_fullsized":"2024-04-25T08:52:39.836281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299015+0000","last_change":"2024-04-25T08:52:31.256415+0000","last_active":"2024-04-25T08:52:38.299015+0000","last_peered":"2024-04-25T08:52:38.299015+0000","last_clean":"2024-04-25T08:52:38.299015+0000","last_became_active":"2024-04-25T08:52:31.254747+0000","last_became_peered":"2024-04-25T08:52:31.254747+0000","last_unstale":"2024-04-25T08:52:38.299015+0000","last_undegraded":"2024-04-25T08:52:38.299015+0000","last_fullsized":"2024-04-25T08:52:38.299015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:24.567213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299991+0000","last_change":"2024-04-25T08:52:30.223725+0000","last_active":"2024-04-25T08:52:38.299991+0000","last_peered":"2024-04-25T08:52:38.299991+0000","last_clean":"2024-04-25T08:52:38.299991+0000","last_became_active":"2024-04-25T08:52:30.222820+0000","last_became_peered":"2024-04-25T08:52:30.222820+0000","last_unstale":"2024-04-25T08:52:38.299991+0000","last_undegraded":"2024-04-25T08:52:38.299991+0000","last_fullsized":"2024-04-25T08:52:38.299991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834851+0000","last_change":"2024-04-25T08:52:36.003180+0000","last_active":"2024-04-25T08:52:39.834851+0000","last_peered":"2024-04-25T08:52:39.834851+0000","last_clean":"2024-04-25T08:52:39.834851+0000","last_became_active":"2024-04-25T08:52:36.002030+0000","last_became_peered":"2024-04-25T08:52:36.002030+0000","last_unstale":"2024-04-25T08:52:39.834851+0000","last_undegraded":"2024-04-25T08:52:39.834851+0000","last_fullsized":"2024-04-25T08:52:39.834851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:58.721596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305176+0000","last_change":"2024-04-25T08:52:31.278626+0000","last_active":"2024-04-25T08:52:39.305176+0000","last_peered":"2024-04-25T08:52:39.305176+0000","last_clean":"2024-04-25T08:52:39.305176+0000","last_became_active":"2024-04-25T08:52:31.278027+0000","last_became_peered":"2024-04-25T08:52:31.278027+0000","last_unstale":"2024-04-25T08:52:39.305176+0000","last_undegraded":"2024-04-25T08:52:39.305176+0000","last_fullsized":"2024-04-25T08:52:39.305176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:15.341259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645485+0000","last_change":"2024-04-25T08:52:30.233250+0000","last_active":"2024-04-25T08:52:39.645485+0000","last_peered":"2024-04-25T08:52:39.645485+0000","last_clean":"2024-04-25T08:52:39.645485+0000","last_became_active":"2024-04-25T08:52:30.229813+0000","last_became_peered":"2024-04-25T08:52:30.229813+0000","last_unstale":"2024-04-25T08:52:39.645485+0000","last_undegraded":"2024-04-25T08:52:39.645485+0000","last_fullsized":"2024-04-25T08:52:39.645485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835406+0000","last_change":"2024-04-25T08:52:36.001662+0000","last_active":"2024-04-25T08:52:39.835406+0000","last_peered":"2024-04-25T08:52:39.835406+0000","last_clean":"2024-04-25T08:52:39.835406+0000","last_became_active":"2024-04-25T08:52:36.000433+0000","last_became_peered":"2024-04-25T08:52:36.000433+0000","last_unstale":"2024-04-25T08:52:39.835406+0000","last_undegraded":"2024-04-25T08:52:39.835406+0000","last_fullsized":"2024-04-25T08:52:39.835406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:13.578788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836042+0000","last_change":"2024-04-25T08:52:31.278698+0000","last_active":"2024-04-25T08:52:39.836042+0000","last_peered":"2024-04-25T08:52:39.836042+0000","last_clean":"2024-04-25T08:52:39.836042+0000","last_became_active":"2024-04-25T08:52:31.278388+0000","last_became_peered":"2024-04-25T08:52:31.278388+0000","last_unstale":"2024-04-25T08:52:39.836042+0000","last_undegraded":"2024-04-25T08:52:39.836042+0000","last_fullsized":"2024-04-25T08:52:39.836042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:24.567213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836775+0000","last_change":"2024-04-25T08:52:30.197089+0000","last_active":"2024-04-25T08:52:39.836775+0000","last_peered":"2024-04-25T08:52:39.836775+0000","last_clean":"2024-04-25T08:52:39.836775+0000","last_became_active":"2024-04-25T08:52:30.196592+0000","last_became_peered":"2024-04-25T08:52:30.196592+0000","last_unstale":"2024-04-25T08:52:39.836775+0000","last_undegraded":"2024-04-25T08:52:39.836775+0000","last_fullsized":"2024-04-25T08:52:39.836775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645712+0000","last_change":"2024-04-25T08:52:36.005556+0000","last_active":"2024-04-25T08:52:39.645712+0000","last_peered":"2024-04-25T08:52:39.645712+0000","last_clean":"2024-04-25T08:52:39.645712+0000","last_became_active":"2024-04-25T08:52:36.004511+0000","last_became_peered":"2024-04-25T08:52:36.004511+0000","last_unstale":"2024-04-25T08:52:39.645712+0000","last_undegraded":"2024-04-25T08:52:39.645712+0000","last_fullsized":"2024-04-25T08:52:39.645712+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307456+0000","last_change":"2024-04-25T08:52:31.286270+0000","last_active":"2024-04-25T08:52:39.307456+0000","last_peered":"2024-04-25T08:52:39.307456+0000","last_clean":"2024-04-25T08:52:39.307456+0000","last_became_active":"2024-04-25T08:52:31.281689+0000","last_became_peered":"2024-04-25T08:52:31.281689+0000","last_unstale":"2024-04-25T08:52:39.307456+0000","last_undegraded":"2024-04-25T08:52:39.307456+0000","last_fullsized":"2024-04-25T08:52:39.307456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:41.689859+0000","last_change":"2024-04-25T08:52:30.215322+0000","last_active":"2024-04-25T08:52:41.689859+0000","last_peered":"2024-04-25T08:52:41.689859+0000","last_clean":"2024-04-25T08:52:41.689859+0000","last_became_active":"2024-04-25T08:52:30.214531+0000","last_became_peered":"2024-04-25T08:52:30.214531+0000","last_unstale":"2024-04-25T08:52:41.689859+0000","last_undegraded":"2024-04-25T08:52:41.689859+0000","last_fullsized":"2024-04-25T08:52:41.689859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833881+0000","last_change":"2024-04-25T08:52:35.307400+0000","last_active":"2024-04-25T08:52:39.833881+0000","last_peered":"2024-04-25T08:52:39.833881+0000","last_clean":"2024-04-25T08:52:39.833881+0000","last_became_active":"2024-04-25T08:52:35.304579+0000","last_became_peered":"2024-04-25T08:52:35.304579+0000","last_unstale":"2024-04-25T08:52:39.833881+0000","last_undegraded":"2024-04-25T08:52:39.833881+0000","last_fullsized":"2024-04-25T08:52:39.833881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:24.182927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301767+0000","last_change":"2024-04-25T08:52:31.262376+0000","last_active":"2024-04-25T08:52:38.301767+0000","last_peered":"2024-04-25T08:52:38.301767+0000","last_clean":"2024-04-25T08:52:38.301767+0000","last_became_active":"2024-04-25T08:52:31.261642+0000","last_became_peered":"2024-04-25T08:52:31.261642+0000","last_unstale":"2024-04-25T08:52:38.301767+0000","last_undegraded":"2024-04-25T08:52:38.301767+0000","last_fullsized":"2024-04-25T08:52:38.301767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:12.300628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833373+0000","last_change":"2024-04-25T08:52:30.219030+0000","last_active":"2024-04-25T08:52:39.833373+0000","last_peered":"2024-04-25T08:52:39.833373+0000","last_clean":"2024-04-25T08:52:39.833373+0000","last_became_active":"2024-04-25T08:52:30.218788+0000","last_became_peered":"2024-04-25T08:52:30.218788+0000","last_unstale":"2024-04-25T08:52:39.833373+0000","last_undegraded":"2024-04-25T08:52:39.833373+0000","last_fullsized":"2024-04-25T08:52:39.833373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:34.198893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300753+0000","last_change":"2024-04-25T08:52:35.331866+0000","last_active":"2024-04-25T08:52:38.300753+0000","last_peered":"2024-04-25T08:52:38.300753+0000","last_clean":"2024-04-25T08:52:38.300753+0000","last_became_active":"2024-04-25T08:52:35.330775+0000","last_became_peered":"2024-04-25T08:52:35.330775+0000","last_unstale":"2024-04-25T08:52:38.300753+0000","last_undegraded":"2024-04-25T08:52:38.300753+0000","last_fullsized":"2024-04-25T08:52:38.300753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:59.165277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836632+0000","last_change":"2024-04-25T08:52:31.256646+0000","last_active":"2024-04-25T08:52:39.836632+0000","last_peered":"2024-04-25T08:52:39.836632+0000","last_clean":"2024-04-25T08:52:39.836632+0000","last_became_active":"2024-04-25T08:52:31.256362+0000","last_became_peered":"2024-04-25T08:52:31.256362+0000","last_unstale":"2024-04-25T08:52:39.836632+0000","last_undegraded":"2024-04-25T08:52:39.836632+0000","last_fullsized":"2024-04-25T08:52:39.836632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645734+0000","last_change":"2024-04-25T08:52:30.217888+0000","last_active":"2024-04-25T08:52:39.645734+0000","last_peered":"2024-04-25T08:52:39.645734+0000","last_clean":"2024-04-25T08:52:39.645734+0000","last_became_active":"2024-04-25T08:52:30.217100+0000","last_became_peered":"2024-04-25T08:52:30.217100+0000","last_unstale":"2024-04-25T08:52:39.645734+0000","last_undegraded":"2024-04-25T08:52:39.645734+0000","last_fullsized":"2024-04-25T08:52:39.645734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834254+0000","last_change":"2024-04-25T08:52:35.313671+0000","last_active":"2024-04-25T08:52:39.834254+0000","last_peered":"2024-04-25T08:52:39.834254+0000","last_clean":"2024-04-25T08:52:39.834254+0000","last_became_active":"2024-04-25T08:52:35.312785+0000","last_became_peered":"2024-04-25T08:52:35.312785+0000","last_unstale":"2024-04-25T08:52:39.834254+0000","last_undegraded":"2024-04-25T08:52:39.834254+0000","last_fullsized":"2024-04-25T08:52:39.834254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:23.828143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645723+0000","last_change":"2024-04-25T08:52:31.273223+0000","last_active":"2024-04-25T08:52:39.645723+0000","last_peered":"2024-04-25T08:52:39.645723+0000","last_clean":"2024-04-25T08:52:39.645723+0000","last_became_active":"2024-04-25T08:52:31.272627+0000","last_became_peered":"2024-04-25T08:52:31.272627+0000","last_unstale":"2024-04-25T08:52:39.645723+0000","last_undegraded":"2024-04-25T08:52:39.645723+0000","last_fullsized":"2024-04-25T08:52:39.645723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303022+0000","last_change":"2024-04-25T08:52:30.229619+0000","last_active":"2024-04-25T08:52:38.303022+0000","last_peered":"2024-04-25T08:52:38.303022+0000","last_clean":"2024-04-25T08:52:38.303022+0000","last_became_active":"2024-04-25T08:52:30.228770+0000","last_became_peered":"2024-04-25T08:52:30.228770+0000","last_unstale":"2024-04-25T08:52:38.303022+0000","last_undegraded":"2024-04-25T08:52:38.303022+0000","last_fullsized":"2024-04-25T08:52:38.303022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833502+0000","last_change":"2024-04-25T08:52:36.008479+0000","last_active":"2024-04-25T08:52:39.833502+0000","last_peered":"2024-04-25T08:52:39.833502+0000","last_clean":"2024-04-25T08:52:39.833502+0000","last_became_active":"2024-04-25T08:52:36.007404+0000","last_became_peered":"2024-04-25T08:52:36.007404+0000","last_unstale":"2024-04-25T08:52:39.833502+0000","last_undegraded":"2024-04-25T08:52:39.833502+0000","last_fullsized":"2024-04-25T08:52:39.833502+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:54.631138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836260+0000","last_change":"2024-04-25T08:52:36.004959+0000","last_active":"2024-04-25T08:52:39.836260+0000","last_peered":"2024-04-25T08:52:39.836260+0000","last_clean":"2024-04-25T08:52:39.836260+0000","last_became_active":"2024-04-25T08:52:36.003575+0000","last_became_peered":"2024-04-25T08:52:36.003575+0000","last_unstale":"2024-04-25T08:52:39.836260+0000","last_undegraded":"2024-04-25T08:52:39.836260+0000","last_fullsized":"2024-04-25T08:52:39.836260+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834357+0000","last_change":"2024-04-25T08:52:31.231230+0000","last_active":"2024-04-25T08:52:39.834357+0000","last_peered":"2024-04-25T08:52:39.834357+0000","last_clean":"2024-04-25T08:52:39.834357+0000","last_became_active":"2024-04-25T08:52:31.230951+0000","last_became_peered":"2024-04-25T08:52:31.230951+0000","last_unstale":"2024-04-25T08:52:39.834357+0000","last_undegraded":"2024-04-25T08:52:39.834357+0000","last_fullsized":"2024-04-25T08:52:39.834357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644784+0000","last_change":"2024-04-25T08:52:30.233534+0000","last_active":"2024-04-25T08:52:39.644784+0000","last_peered":"2024-04-25T08:52:39.644784+0000","last_clean":"2024-04-25T08:52:39.644784+0000","last_became_active":"2024-04-25T08:52:30.232832+0000","last_became_peered":"2024-04-25T08:52:30.232832+0000","last_unstale":"2024-04-25T08:52:39.644784+0000","last_undegraded":"2024-04-25T08:52:39.644784+0000","last_fullsized":"2024-04-25T08:52:39.644784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303102+0000","last_change":"2024-04-25T08:52:35.317543+0000","last_active":"2024-04-25T08:52:38.303102+0000","last_peered":"2024-04-25T08:52:38.303102+0000","last_clean":"2024-04-25T08:52:38.303102+0000","last_became_active":"2024-04-25T08:52:35.316474+0000","last_became_peered":"2024-04-25T08:52:35.316474+0000","last_unstale":"2024-04-25T08:52:38.303102+0000","last_undegraded":"2024-04-25T08:52:38.303102+0000","last_fullsized":"2024-04-25T08:52:38.303102+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:35.695987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834357+0000","last_change":"2024-04-25T08:52:31.257812+0000","last_active":"2024-04-25T08:52:39.834357+0000","last_peered":"2024-04-25T08:52:39.834357+0000","last_clean":"2024-04-25T08:52:39.834357+0000","last_became_active":"2024-04-25T08:52:31.257528+0000","last_became_peered":"2024-04-25T08:52:31.257528+0000","last_unstale":"2024-04-25T08:52:39.834357+0000","last_undegraded":"2024-04-25T08:52:39.834357+0000","last_fullsized":"2024-04-25T08:52:39.834357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835503+0000","last_change":"2024-04-25T08:52:30.217142+0000","last_active":"2024-04-25T08:52:39.835503+0000","last_peered":"2024-04-25T08:52:39.835503+0000","last_clean":"2024-04-25T08:52:39.835503+0000","last_became_active":"2024-04-25T08:52:30.216729+0000","last_became_peered":"2024-04-25T08:52:30.216729+0000","last_unstale":"2024-04-25T08:52:39.835503+0000","last_undegraded":"2024-04-25T08:52:39.835503+0000","last_fullsized":"2024-04-25T08:52:39.835503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300819+0000","last_change":"2024-04-25T08:52:30.228704+0000","last_active":"2024-04-25T08:52:38.300819+0000","last_peered":"2024-04-25T08:52:38.300819+0000","last_clean":"2024-04-25T08:52:38.300819+0000","last_became_active":"2024-04-25T08:52:30.227861+0000","last_became_peered":"2024-04-25T08:52:30.227861+0000","last_unstale":"2024-04-25T08:52:38.300819+0000","last_undegraded":"2024-04-25T08:52:38.300819+0000","last_fullsized":"2024-04-25T08:52:38.300819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835140+0000","last_change":"2024-04-25T08:52:31.258702+0000","last_active":"2024-04-25T08:52:39.835140+0000","last_peered":"2024-04-25T08:52:39.835140+0000","last_clean":"2024-04-25T08:52:39.835140+0000","last_became_active":"2024-04-25T08:52:31.258349+0000","last_became_peered":"2024-04-25T08:52:31.258349+0000","last_unstale":"2024-04-25T08:52:39.835140+0000","last_undegraded":"2024-04-25T08:52:39.835140+0000","last_fullsized":"2024-04-25T08:52:39.835140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299697+0000","last_change":"2024-04-25T08:52:35.313396+0000","last_active":"2024-04-25T08:52:38.299697+0000","last_peered":"2024-04-25T08:52:38.299697+0000","last_clean":"2024-04-25T08:52:38.299697+0000","last_became_active":"2024-04-25T08:52:35.312845+0000","last_became_peered":"2024-04-25T08:52:35.312845+0000","last_unstale":"2024-04-25T08:52:38.299697+0000","last_undegraded":"2024-04-25T08:52:38.299697+0000","last_fullsized":"2024-04-25T08:52:38.299697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.648302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836001+0000","last_change":"2024-04-25T08:52:30.239169+0000","last_active":"2024-04-25T08:52:39.836001+0000","last_peered":"2024-04-25T08:52:39.836001+0000","last_clean":"2024-04-25T08:52:39.836001+0000","last_became_active":"2024-04-25T08:52:30.238820+0000","last_became_peered":"2024-04-25T08:52:30.238820+0000","last_unstale":"2024-04-25T08:52:39.836001+0000","last_undegraded":"2024-04-25T08:52:39.836001+0000","last_fullsized":"2024-04-25T08:52:39.836001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300334+0000","last_change":"2024-04-25T08:52:31.243105+0000","last_active":"2024-04-25T08:52:38.300334+0000","last_peered":"2024-04-25T08:52:38.300334+0000","last_clean":"2024-04-25T08:52:38.300334+0000","last_became_active":"2024-04-25T08:52:31.242326+0000","last_became_peered":"2024-04-25T08:52:31.242326+0000","last_unstale":"2024-04-25T08:52:38.300334+0000","last_undegraded":"2024-04-25T08:52:38.300334+0000","last_fullsized":"2024-04-25T08:52:38.300334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:04.149068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834664+0000","last_change":"2024-04-25T08:52:35.313390+0000","last_active":"2024-04-25T08:52:39.834664+0000","last_peered":"2024-04-25T08:52:39.834664+0000","last_clean":"2024-04-25T08:52:39.834664+0000","last_became_active":"2024-04-25T08:52:35.312750+0000","last_became_peered":"2024-04-25T08:52:35.312750+0000","last_unstale":"2024-04-25T08:52:39.834664+0000","last_undegraded":"2024-04-25T08:52:39.834664+0000","last_fullsized":"2024-04-25T08:52:39.834664+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:49.016809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298785+0000","last_change":"2024-04-25T08:52:31.257097+0000","last_active":"2024-04-25T08:52:38.298785+0000","last_peered":"2024-04-25T08:52:38.298785+0000","last_clean":"2024-04-25T08:52:38.298785+0000","last_became_active":"2024-04-25T08:52:31.255697+0000","last_became_peered":"2024-04-25T08:52:31.255697+0000","last_unstale":"2024-04-25T08:52:38.298785+0000","last_undegraded":"2024-04-25T08:52:38.298785+0000","last_fullsized":"2024-04-25T08:52:38.298785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298858+0000","last_change":"2024-04-25T08:52:31.237994+0000","last_active":"2024-04-25T08:52:38.298858+0000","last_peered":"2024-04-25T08:52:38.298858+0000","last_clean":"2024-04-25T08:52:38.298858+0000","last_became_active":"2024-04-25T08:52:31.237190+0000","last_became_peered":"2024-04-25T08:52:31.237190+0000","last_unstale":"2024-04-25T08:52:38.298858+0000","last_undegraded":"2024-04-25T08:52:38.298858+0000","last_fullsized":"2024-04-25T08:52:38.298858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:35.208568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645427+0000","last_change":"2024-04-25T08:52:31.248873+0000","last_active":"2024-04-25T08:52:39.645427+0000","last_peered":"2024-04-25T08:52:39.645427+0000","last_clean":"2024-04-25T08:52:39.645427+0000","last_became_active":"2024-04-25T08:52:31.248359+0000","last_became_peered":"2024-04-25T08:52:31.248359+0000","last_unstale":"2024-04-25T08:52:39.645427+0000","last_undegraded":"2024-04-25T08:52:39.645427+0000","last_fullsized":"2024-04-25T08:52:39.645427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836722+0000","last_change":"2024-04-25T08:52:31.256155+0000","last_active":"2024-04-25T08:52:39.836722+0000","last_peered":"2024-04-25T08:52:39.836722+0000","last_clean":"2024-04-25T08:52:39.836722+0000","last_became_active":"2024-04-25T08:52:31.255884+0000","last_became_peered":"2024-04-25T08:52:31.255884+0000","last_unstale":"2024-04-25T08:52:39.836722+0000","last_undegraded":"2024-04-25T08:52:39.836722+0000","last_fullsized":"2024-04-25T08:52:39.836722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834472+0000","last_change":"2024-04-25T08:52:31.266984+0000","last_active":"2024-04-25T08:52:39.834472+0000","last_peered":"2024-04-25T08:52:39.834472+0000","last_clean":"2024-04-25T08:52:39.834472+0000","last_became_active":"2024-04-25T08:52:31.266697+0000","last_became_peered":"2024-04-25T08:52:31.266697+0000","last_unstale":"2024-04-25T08:52:39.834472+0000","last_undegraded":"2024-04-25T08:52:39.834472+0000","last_fullsized":"2024-04-25T08:52:39.834472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833201+0000","last_change":"2024-04-25T08:52:31.234570+0000","last_active":"2024-04-25T08:52:39.833201+0000","last_peered":"2024-04-25T08:52:39.833201+0000","last_clean":"2024-04-25T08:52:39.833201+0000","last_became_active":"2024-04-25T08:52:31.234049+0000","last_became_peered":"2024-04-25T08:52:31.234049+0000","last_unstale":"2024-04-25T08:52:39.833201+0000","last_undegraded":"2024-04-25T08:52:39.833201+0000","last_fullsized":"2024-04-25T08:52:39.833201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303100+0000","last_change":"2024-04-25T08:52:31.271083+0000","last_active":"2024-04-25T08:52:38.303100+0000","last_peered":"2024-04-25T08:52:38.303100+0000","last_clean":"2024-04-25T08:52:38.303100+0000","last_became_active":"2024-04-25T08:52:31.268755+0000","last_became_peered":"2024-04-25T08:52:31.268755+0000","last_unstale":"2024-04-25T08:52:38.303100+0000","last_undegraded":"2024-04-25T08:52:38.303100+0000","last_fullsized":"2024-04-25T08:52:38.303100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834579+0000","last_change":"2024-04-25T08:52:31.265576+0000","last_active":"2024-04-25T08:52:39.834579+0000","last_peered":"2024-04-25T08:52:39.834579+0000","last_clean":"2024-04-25T08:52:39.834579+0000","last_became_active":"2024-04-25T08:52:31.265273+0000","last_became_peered":"2024-04-25T08:52:31.265273+0000","last_unstale":"2024-04-25T08:52:39.834579+0000","last_undegraded":"2024-04-25T08:52:39.834579+0000","last_fullsized":"2024-04-25T08:52:39.834579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644626+0000","last_change":"2024-04-25T08:52:31.247316+0000","last_active":"2024-04-25T08:52:39.644626+0000","last_peered":"2024-04-25T08:52:39.644626+0000","last_clean":"2024-04-25T08:52:39.644626+0000","last_became_active":"2024-04-25T08:52:31.246547+0000","last_became_peered":"2024-04-25T08:52:31.246547+0000","last_unstale":"2024-04-25T08:52:39.644626+0000","last_undegraded":"2024-04-25T08:52:39.644626+0000","last_fullsized":"2024-04-25T08:52:39.644626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833082+0000","last_change":"2024-04-25T08:52:31.265084+0000","last_active":"2024-04-25T08:52:39.833082+0000","last_peered":"2024-04-25T08:52:39.833082+0000","last_clean":"2024-04-25T08:52:39.833082+0000","last_became_active":"2024-04-25T08:52:31.264883+0000","last_became_peered":"2024-04-25T08:52:31.264883+0000","last_unstale":"2024-04-25T08:52:39.833082+0000","last_undegraded":"2024-04-25T08:52:39.833082+0000","last_fullsized":"2024-04-25T08:52:39.833082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645197+0000","last_change":"2024-04-25T08:52:31.280090+0000","last_active":"2024-04-25T08:52:39.645197+0000","last_peered":"2024-04-25T08:52:39.645197+0000","last_clean":"2024-04-25T08:52:39.645197+0000","last_became_active":"2024-04-25T08:52:31.279453+0000","last_became_peered":"2024-04-25T08:52:31.279453+0000","last_unstale":"2024-04-25T08:52:39.645197+0000","last_undegraded":"2024-04-25T08:52:39.645197+0000","last_fullsized":"2024-04-25T08:52:39.645197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836409+0000","last_change":"2024-04-25T08:52:31.271765+0000","last_active":"2024-04-25T08:52:39.836409+0000","last_peered":"2024-04-25T08:52:39.836409+0000","last_clean":"2024-04-25T08:52:39.836409+0000","last_became_active":"2024-04-25T08:52:31.271438+0000","last_became_peered":"2024-04-25T08:52:31.271438+0000","last_unstale":"2024-04-25T08:52:39.836409+0000","last_undegraded":"2024-04-25T08:52:39.836409+0000","last_fullsized":"2024-04-25T08:52:39.836409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301809+0000","last_change":"2024-04-25T08:52:31.270307+0000","last_active":"2024-04-25T08:52:38.301809+0000","last_peered":"2024-04-25T08:52:38.301809+0000","last_clean":"2024-04-25T08:52:38.301809+0000","last_became_active":"2024-04-25T08:52:31.269440+0000","last_became_peered":"2024-04-25T08:52:31.269440+0000","last_unstale":"2024-04-25T08:52:38.301809+0000","last_undegraded":"2024-04-25T08:52:38.301809+0000","last_fullsized":"2024-04-25T08:52:38.301809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305298+0000","last_change":"2024-04-25T08:52:31.281491+0000","last_active":"2024-04-25T08:52:39.305298+0000","last_peered":"2024-04-25T08:52:39.305298+0000","last_clean":"2024-04-25T08:52:39.305298+0000","last_became_active":"2024-04-25T08:52:31.276333+0000","last_became_peered":"2024-04-25T08:52:31.276333+0000","last_unstale":"2024-04-25T08:52:39.305298+0000","last_undegraded":"2024-04-25T08:52:39.305298+0000","last_fullsized":"2024-04-25T08:52:39.305298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:31.614898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299491+0000","last_change":"2024-04-25T08:52:31.237856+0000","last_active":"2024-04-25T08:52:38.299491+0000","last_peered":"2024-04-25T08:52:38.299491+0000","last_clean":"2024-04-25T08:52:38.299491+0000","last_became_active":"2024-04-25T08:52:31.237582+0000","last_became_peered":"2024-04-25T08:52:31.237582+0000","last_unstale":"2024-04-25T08:52:38.299491+0000","last_undegraded":"2024-04-25T08:52:38.299491+0000","last_fullsized":"2024-04-25T08:52:38.299491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:05.271538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.646089+0000","last_change":"2024-04-25T08:52:31.283655+0000","last_active":"2024-04-25T08:52:39.646089+0000","last_peered":"2024-04-25T08:52:39.646089+0000","last_clean":"2024-04-25T08:52:39.646089+0000","last_became_active":"2024-04-25T08:52:31.282969+0000","last_became_peered":"2024-04-25T08:52:31.282969+0000","last_unstale":"2024-04-25T08:52:39.646089+0000","last_undegraded":"2024-04-25T08:52:39.646089+0000","last_fullsized":"2024-04-25T08:52:39.646089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:08.025791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644525+0000","last_change":"2024-04-25T08:52:31.256907+0000","last_active":"2024-04-25T08:52:39.644525+0000","last_peered":"2024-04-25T08:52:39.644525+0000","last_clean":"2024-04-25T08:52:39.644525+0000","last_became_active":"2024-04-25T08:52:31.255494+0000","last_became_peered":"2024-04-25T08:52:31.255494+0000","last_unstale":"2024-04-25T08:52:39.644525+0000","last_undegraded":"2024-04-25T08:52:39.644525+0000","last_fullsized":"2024-04-25T08:52:39.644525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834186+0000","last_change":"2024-04-25T08:52:31.270805+0000","last_active":"2024-04-25T08:52:39.834186+0000","last_peered":"2024-04-25T08:52:39.834186+0000","last_clean":"2024-04-25T08:52:39.834186+0000","last_became_active":"2024-04-25T08:52:31.270594+0000","last_became_peered":"2024-04-25T08:52:31.270594+0000","last_unstale":"2024-04-25T08:52:39.834186+0000","last_undegraded":"2024-04-25T08:52:39.834186+0000","last_fullsized":"2024-04-25T08:52:39.834186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836035+0000","last_change":"2024-04-25T08:52:31.252734+0000","last_active":"2024-04-25T08:52:39.836035+0000","last_peered":"2024-04-25T08:52:39.836035+0000","last_clean":"2024-04-25T08:52:39.836035+0000","last_became_active":"2024-04-25T08:52:31.252476+0000","last_became_peered":"2024-04-25T08:52:31.252476+0000","last_unstale":"2024-04-25T08:52:39.836035+0000","last_undegraded":"2024-04-25T08:52:39.836035+0000","last_fullsized":"2024-04-25T08:52:39.836035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833928+0000","last_change":"2024-04-25T08:52:31.242347+0000","last_active":"2024-04-25T08:52:39.833928+0000","last_peered":"2024-04-25T08:52:39.833928+0000","last_clean":"2024-04-25T08:52:39.833928+0000","last_became_active":"2024-04-25T08:52:31.242082+0000","last_became_peered":"2024-04-25T08:52:31.242082+0000","last_unstale":"2024-04-25T08:52:39.833928+0000","last_undegraded":"2024-04-25T08:52:39.833928+0000","last_fullsized":"2024-04-25T08:52:39.833928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301519+0000","last_change":"2024-04-25T08:52:31.258568+0000","last_active":"2024-04-25T08:52:38.301519+0000","last_peered":"2024-04-25T08:52:38.301519+0000","last_clean":"2024-04-25T08:52:38.301519+0000","last_became_active":"2024-04-25T08:52:31.257438+0000","last_became_peered":"2024-04-25T08:52:31.257438+0000","last_unstale":"2024-04-25T08:52:38.301519+0000","last_undegraded":"2024-04-25T08:52:38.301519+0000","last_fullsized":"2024-04-25T08:52:38.301519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:47.022570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.302377+0000","last_change":"2024-04-25T08:52:31.254993+0000","last_active":"2024-04-25T08:52:38.302377+0000","last_peered":"2024-04-25T08:52:38.302377+0000","last_clean":"2024-04-25T08:52:38.302377+0000","last_became_active":"2024-04-25T08:52:31.254072+0000","last_became_peered":"2024-04-25T08:52:31.254072+0000","last_unstale":"2024-04-25T08:52:38.302377+0000","last_undegraded":"2024-04-25T08:52:38.302377+0000","last_fullsized":"2024-04-25T08:52:38.302377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303017+0000","last_change":"2024-04-25T08:52:31.247059+0000","last_active":"2024-04-25T08:52:38.303017+0000","last_peered":"2024-04-25T08:52:38.303017+0000","last_clean":"2024-04-25T08:52:38.303017+0000","last_became_active":"2024-04-25T08:52:31.246364+0000","last_became_peered":"2024-04-25T08:52:31.246364+0000","last_unstale":"2024-04-25T08:52:38.303017+0000","last_undegraded":"2024-04-25T08:52:38.303017+0000","last_fullsized":"2024-04-25T08:52:38.303017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.308187+0000","last_change":"2024-04-25T08:52:31.289253+0000","last_active":"2024-04-25T08:52:39.308187+0000","last_peered":"2024-04-25T08:52:39.308187+0000","last_clean":"2024-04-25T08:52:39.308187+0000","last_became_active":"2024-04-25T08:52:31.288175+0000","last_became_peered":"2024-04-25T08:52:31.288175+0000","last_unstale":"2024-04-25T08:52:39.308187+0000","last_undegraded":"2024-04-25T08:52:39.308187+0000","last_fullsized":"2024-04-25T08:52:39.308187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:08.025791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834144+0000","last_change":"2024-04-25T08:52:31.256539+0000","last_active":"2024-04-25T08:52:39.834144+0000","last_peered":"2024-04-25T08:52:39.834144+0000","last_clean":"2024-04-25T08:52:39.834144+0000","last_became_active":"2024-04-25T08:52:31.255921+0000","last_became_peered":"2024-04-25T08:52:31.255921+0000","last_unstale":"2024-04-25T08:52:39.834144+0000","last_undegraded":"2024-04-25T08:52:39.834144+0000","last_fullsized":"2024-04-25T08:52:39.834144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306615+0000","last_change":"2024-04-25T08:52:31.277410+0000","last_active":"2024-04-25T08:52:39.306615+0000","last_peered":"2024-04-25T08:52:39.306615+0000","last_clean":"2024-04-25T08:52:39.306615+0000","last_became_active":"2024-04-25T08:52:31.276941+0000","last_became_peered":"2024-04-25T08:52:31.276941+0000","last_unstale":"2024-04-25T08:52:39.306615+0000","last_undegraded":"2024-04-25T08:52:39.306615+0000","last_fullsized":"2024-04-25T08:52:39.306615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645097+0000","last_change":"2024-04-25T08:52:31.256171+0000","last_active":"2024-04-25T08:52:39.645097+0000","last_peered":"2024-04-25T08:52:39.645097+0000","last_clean":"2024-04-25T08:52:39.645097+0000","last_became_active":"2024-04-25T08:52:31.255417+0000","last_became_peered":"2024-04-25T08:52:31.255417+0000","last_unstale":"2024-04-25T08:52:39.645097+0000","last_undegraded":"2024-04-25T08:52:39.645097+0000","last_fullsized":"2024-04-25T08:52:39.645097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305186+0000","last_change":"2024-04-25T08:52:31.269978+0000","last_active":"2024-04-25T08:52:39.305186+0000","last_peered":"2024-04-25T08:52:39.305186+0000","last_clean":"2024-04-25T08:52:39.305186+0000","last_became_active":"2024-04-25T08:52:31.269108+0000","last_became_peered":"2024-04-25T08:52:31.269108+0000","last_unstale":"2024-04-25T08:52:39.305186+0000","last_undegraded":"2024-04-25T08:52:39.305186+0000","last_fullsized":"2024-04-25T08:52:39.305186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645458+0000","last_change":"2024-04-25T08:52:31.248569+0000","last_active":"2024-04-25T08:52:39.645458+0000","last_peered":"2024-04-25T08:52:39.645458+0000","last_clean":"2024-04-25T08:52:39.645458+0000","last_became_active":"2024-04-25T08:52:31.247655+0000","last_became_peered":"2024-04-25T08:52:31.247655+0000","last_unstale":"2024-04-25T08:52:39.645458+0000","last_undegraded":"2024-04-25T08:52:39.645458+0000","last_fullsized":"2024-04-25T08:52:39.645458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833573+0000","last_change":"2024-04-25T08:52:31.252449+0000","last_active":"2024-04-25T08:52:39.833573+0000","last_peered":"2024-04-25T08:52:39.833573+0000","last_clean":"2024-04-25T08:52:39.833573+0000","last_became_active":"2024-04-25T08:52:31.252212+0000","last_became_peered":"2024-04-25T08:52:31.252212+0000","last_unstale":"2024-04-25T08:52:39.833573+0000","last_undegraded":"2024-04-25T08:52:39.833573+0000","last_fullsized":"2024-04-25T08:52:39.833573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833700+0000","last_change":"2024-04-25T08:52:31.254735+0000","last_active":"2024-04-25T08:52:39.833700+0000","last_peered":"2024-04-25T08:52:39.833700+0000","last_clean":"2024-04-25T08:52:39.833700+0000","last_became_active":"2024-04-25T08:52:31.254523+0000","last_became_peered":"2024-04-25T08:52:31.254523+0000","last_unstale":"2024-04-25T08:52:39.833700+0000","last_undegraded":"2024-04-25T08:52:39.833700+0000","last_fullsized":"2024-04-25T08:52:39.833700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305895+0000","last_change":"2024-04-25T08:52:31.288927+0000","last_active":"2024-04-25T08:52:39.305895+0000","last_peered":"2024-04-25T08:52:39.305895+0000","last_clean":"2024-04-25T08:52:39.305895+0000","last_became_active":"2024-04-25T08:52:31.287790+0000","last_became_peered":"2024-04-25T08:52:31.287790+0000","last_unstale":"2024-04-25T08:52:39.305895+0000","last_undegraded":"2024-04-25T08:52:39.305895+0000","last_fullsized":"2024-04-25T08:52:39.305895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:52.007683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":709746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":122638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":118886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":118922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":122638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":118796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":118956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":897024,"data_stored":709802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:52:58.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:52:58.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.220+0000 7fdcebbab640 1 Processor -- start 2024-04-25T08:52:58.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.220+0000 7fdcebbab640 1 -- start start 2024-04-25T08:52:58.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcebbab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdce415ed40 0x7fdce40d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcebbab640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce40affa0 con 0x7fdce40c7020 2024-04-25T08:52:58.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcebbab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce415f120 con 0x7fdce4160080 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcebbab640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce40b0240 con 0x7fdce415ed40 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcea148640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdce40c7020 0x7fdce40caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59928/0 (socket says 172.21.15.37:59928) 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcea148640 1 -- 172.21.15.37:0/1243974244 learned_addr learned my addr 172.21.15.37:0/1243974244 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce9947640 1 --2- 172.21.15.37:0/1243974244 >> v2:172.21.15.203:3300/0 conn(0x7fdce415ed40 0x7fdce40d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3510970741 0 0) 0x7fdce40affa0 con 0x7fdce40c7020 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdccc003530 con 0x7fdce40c7020 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 129033365 0 0) 0x7fdccc003530 con 0x7fdce40c7020 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 >> v2:172.21.15.203:3300/0 conn(0x7fdce415ed40 msgr2=0x7fdce40d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 --2- 172.21.15.37:0/1243974244 >> v2:172.21.15.203:3300/0 conn(0x7fdce415ed40 0x7fdce40d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 >> v1:172.21.15.203:6789/0 conn(0x7fdce4160080 legacy=0x7fdce40c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce40b0520 con 0x7fdce40c7020 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fdcdc001680 con 0x7fdce40c7020 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 >> v1:172.21.15.37:6789/0 conn(0x7fdce40c7020 legacy=0x7fdce40caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 --2- 172.21.15.37:0/1243974244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdccc003a80 0x7fdccc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 --2- 172.21.15.37:0/1243974244 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdccc004810 0x7fdccc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 --2- 172.21.15.37:0/1243974244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc0052f0 0x7fdccc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40b0240 con 0x7fdccc0052f0 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc00c1c0 con 0x7fdccc003a80 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc00c3c0 con 0x7fdccc004810 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcea148640 1 --2- 172.21.15.37:0/1243974244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc0052f0 0x7fdccc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce9947640 1 --2- 172.21.15.37:0/1243974244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdccc003a80 0x7fdccc003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcea148640 1 -- 172.21.15.37:0/1243974244 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdccc004810 msgr2=0x7fdccc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdce9146640 1 --2- 172.21.15.37:0/1243974244 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdccc004810 0x7fdccc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcea148640 1 --2- 172.21.15.37:0/1243974244 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdccc004810 0x7fdccc004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcea148640 1 -- 172.21.15.37:0/1243974244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdccc003a80 msgr2=0x7fdccc003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcea148640 1 --2- 172.21.15.37:0/1243974244 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdccc003a80 0x7fdccc003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.224+0000 7fdcea148640 1 -- 172.21.15.37:0/1243974244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc003530 con 0x7fdccc0052f0 2024-04-25T08:52:58.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcea148640 1 -- 172.21.15.37:0/1243974244 reap_dead start 2024-04-25T08:52:58.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcea148640 1 --2- 172.21.15.37:0/1243974244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc0052f0 0x7fdccc00b9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00c190 tx=0x7fdcdc01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f60c32e75db958e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:58.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc007c70 con 0x7fdccc0052f0 2024-04-25T08:52:58.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc005b20 con 0x7fdccc0052f0 2024-04-25T08:52:58.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdce8945640 1 -- 172.21.15.37:0/1243974244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc007070 con 0x7fdccc0052f0 2024-04-25T08:52:58.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 -- 172.21.15.37:0/1243974244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc0052f0 msgr2=0x7fdccc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 --2- 172.21.15.37:0/1243974244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc0052f0 0x7fdccc00b9d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc00c190 tx=0x7fdcdc01f330 comp rx=0 tx=0).stop 2024-04-25T08:52:58.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 -- 172.21.15.37:0/1243974244 shutdown_connections 2024-04-25T08:52:58.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 --2- 172.21.15.37:0/1243974244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdccc0052f0 0x7fdccc00b9d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 -- 172.21.15.37:0/1243974244 >> 172.21.15.37:0/1243974244 conn(0x7fdce4099a70 msgr2=0x7fdce40d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:58.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 -- 172.21.15.37:0/1243974244 shutdown_connections 2024-04-25T08:52:58.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 -- 172.21.15.37:0/1243974244 wait complete. 2024-04-25T08:52:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 Processor -- start 2024-04-25T08:52:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 -- start start 2024-04-25T08:52:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdccc0052f0 0x7fdce415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce41f68d0 0x7fdce41f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce40b2470 0x7fdce40b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40c6910 con 0x7fdce41f68d0 2024-04-25T08:52:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40b0ae0 con 0x7fdce40b2470 2024-04-25T08:52:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdcebbab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40c6a90 con 0x7fdccc0052f0 2024-04-25T08:52:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdce9146640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce41f68d0 0x7fdce41f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdce9146640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce41f68d0 0x7fdce41f6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46674/0 (socket says 172.21.15.37:46674) 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdce9146640 1 -- 172.21.15.37:0/3472639309 learned_addr learned my addr 172.21.15.37:0/3472639309 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.228+0000 7fdce9947640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdccc0052f0 0x7fdce415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdcea148640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce40b2470 0x7fdce40b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce9146640 1 -- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdccc0052f0 msgr2=0x7fdce415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce9146640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdccc0052f0 0x7fdce415c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce9146640 1 -- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce40b2470 msgr2=0x7fdce40b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce9146640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce40b2470 0x7fdce40b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce9146640 1 -- 172.21.15.37:0/3472639309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce40cb4f0 con 0x7fdce41f68d0 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce9146640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce41f68d0 0x7fdce41f6d50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800d8d0 tx=0x7fdcd800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce27fc640 1 -- 172.21.15.37:0/3472639309 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8003db0 con 0x7fdce41f68d0 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce27fc640 1 -- 172.21.15.37:0/3472639309 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd8017070 con 0x7fdce41f68d0 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdcebbab640 1 -- 172.21.15.37:0/3472639309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce41f7440 con 0x7fdce41f68d0 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce27fc640 1 -- 172.21.15.37:0/3472639309 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd8005270 con 0x7fdce41f68d0 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdcebbab640 1 -- 172.21.15.37:0/3472639309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce40c7550 con 0x7fdce41f68d0 2024-04-25T08:52:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdcebbab640 1 -- 172.21.15.37:0/3472639309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce40b2e30 con 0x7fdce41f68d0 2024-04-25T08:52:58.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce27fc640 1 -- 172.21.15.37:0/3472639309 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdcd8005410 con 0x7fdce41f68d0 2024-04-25T08:52:58.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce27fc640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdcd0066f60 0x7fdcd0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce27fc640 1 -- 172.21.15.37:0/3472639309 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdcd80854e0 con 0x7fdce41f68d0 2024-04-25T08:52:58.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce9947640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdcd0066f60 0x7fdcd0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.232+0000 7fdce9947640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdcd0066f60 0x7fdcd0069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40099b0 tx=0x7fdcd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:58.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.236+0000 7fdce27fc640 1 -- 172.21.15.37:0/3472639309 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdcd80561a0 con 0x7fdce41f68d0 2024-04-25T08:52:58.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.388+0000 7fdcebbab640 1 -- 172.21.15.37:0/3472639309 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdce404ef80 con 0x7fdcd0066f60 2024-04-25T08:52:58.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.412+0000 7fdce27fc640 1 -- 172.21.15.37:0/3472639309 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332533 (secure 0 0 0) 0x7fdce404ef80 con 0x7fdcd0066f60 2024-04-25T08:52:58.409 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:58.412 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:52:58.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 -- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdcd0066f60 msgr2=0x7fdcd0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdcd0066f60 0x7fdcd0069420 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdcd40099b0 tx=0x7fdcd4008040 comp rx=0 tx=0).stop 2024-04-25T08:52:58.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 -- 172.21.15.37:0/3472639309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce41f68d0 msgr2=0x7fdce41f6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce41f68d0 0x7fdce41f6d50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800d8d0 tx=0x7fdcd800dd90 comp rx=0 tx=0).stop 2024-04-25T08:52:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 -- 172.21.15.37:0/3472639309 shutdown_connections 2024-04-25T08:52:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdcd0066f60 0x7fdcd0069420 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdce40b2470 0x7fdce40b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdce41f68d0 0x7fdce41f6d50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 --2- 172.21.15.37:0/3472639309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdccc0052f0 0x7fdce415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 -- 172.21.15.37:0/3472639309 >> 172.21.15.37:0/3472639309 conn(0x7fdce4099a70 msgr2=0x7fdce406e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 -- 172.21.15.37:0/3472639309 shutdown_connections 2024-04-25T08:52:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.416+0000 7fdcebbab640 1 -- 172.21.15.37:0/3472639309 wait complete. 2024-04-25T08:52:58.425 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-04-25T08:52:57.127714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219668,"kb_used_data":3496,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755052,"statfs":{"total":773094113280,"available":772869173248,"internally_reserved":0,"allocated":3579904,"data_stored":2140384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005526"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836749+0000","last_change":"2024-04-25T08:52:30.238555+0000","last_active":"2024-04-25T08:52:39.836749+0000","last_peered":"2024-04-25T08:52:39.836749+0000","last_clean":"2024-04-25T08:52:39.836749+0000","last_became_active":"2024-04-25T08:52:30.238074+0000","last_became_peered":"2024-04-25T08:52:30.238074+0000","last_unstale":"2024-04-25T08:52:39.836749+0000","last_undegraded":"2024-04-25T08:52:39.836749+0000","last_fullsized":"2024-04-25T08:52:39.836749+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:52.505423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834324+0000","last_change":"2024-04-25T08:52:31.237632+0000","last_active":"2024-04-25T08:52:39.834324+0000","last_peered":"2024-04-25T08:52:39.834324+0000","last_clean":"2024-04-25T08:52:39.834324+0000","last_became_active":"2024-04-25T08:52:31.237356+0000","last_became_peered":"2024-04-25T08:52:31.237356+0000","last_unstale":"2024-04-25T08:52:39.834324+0000","last_undegraded":"2024-04-25T08:52:39.834324+0000","last_fullsized":"2024-04-25T08:52:39.834324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833357+0000","last_change":"2024-04-25T08:52:36.008156+0000","last_active":"2024-04-25T08:52:39.833357+0000","last_peered":"2024-04-25T08:52:39.833357+0000","last_clean":"2024-04-25T08:52:39.833357+0000","last_became_active":"2024-04-25T08:52:36.006941+0000","last_became_peered":"2024-04-25T08:52:36.006941+0000","last_unstale":"2024-04-25T08:52:39.833357+0000","last_undegraded":"2024-04-25T08:52:39.833357+0000","last_fullsized":"2024-04-25T08:52:39.833357+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:00.158176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834859+0000","last_change":"2024-04-25T08:52:30.225034+0000","last_active":"2024-04-25T08:52:39.834859+0000","last_peered":"2024-04-25T08:52:39.834859+0000","last_clean":"2024-04-25T08:52:39.834859+0000","last_became_active":"2024-04-25T08:52:30.224727+0000","last_became_peered":"2024-04-25T08:52:30.224727+0000","last_unstale":"2024-04-25T08:52:39.834859+0000","last_undegraded":"2024-04-25T08:52:39.834859+0000","last_fullsized":"2024-04-25T08:52:39.834859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836491+0000","last_change":"2024-04-25T08:52:31.256496+0000","last_active":"2024-04-25T08:52:39.836491+0000","last_peered":"2024-04-25T08:52:39.836491+0000","last_clean":"2024-04-25T08:52:39.836491+0000","last_became_active":"2024-04-25T08:52:31.255998+0000","last_became_peered":"2024-04-25T08:52:31.255998+0000","last_unstale":"2024-04-25T08:52:39.836491+0000","last_undegraded":"2024-04-25T08:52:39.836491+0000","last_fullsized":"2024-04-25T08:52:39.836491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299627+0000","last_change":"2024-04-25T08:52:35.324509+0000","last_active":"2024-04-25T08:52:38.299627+0000","last_peered":"2024-04-25T08:52:38.299627+0000","last_clean":"2024-04-25T08:52:38.299627+0000","last_became_active":"2024-04-25T08:52:35.323398+0000","last_became_peered":"2024-04-25T08:52:35.323398+0000","last_unstale":"2024-04-25T08:52:38.299627+0000","last_undegraded":"2024-04-25T08:52:38.299627+0000","last_fullsized":"2024-04-25T08:52:38.299627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:56.088772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307843+0000","last_change":"2024-04-25T08:52:30.248641+0000","last_active":"2024-04-25T08:52:39.307843+0000","last_peered":"2024-04-25T08:52:39.307843+0000","last_clean":"2024-04-25T08:52:39.307843+0000","last_became_active":"2024-04-25T08:52:30.245183+0000","last_became_peered":"2024-04-25T08:52:30.245183+0000","last_unstale":"2024-04-25T08:52:39.307843+0000","last_undegraded":"2024-04-25T08:52:39.307843+0000","last_fullsized":"2024-04-25T08:52:39.307843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:46.012895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644062+0000","last_change":"2024-04-25T08:52:31.272777+0000","last_active":"2024-04-25T08:52:39.644062+0000","last_peered":"2024-04-25T08:52:39.644062+0000","last_clean":"2024-04-25T08:52:39.644062+0000","last_became_active":"2024-04-25T08:52:31.271932+0000","last_became_peered":"2024-04-25T08:52:31.271932+0000","last_unstale":"2024-04-25T08:52:39.644062+0000","last_undegraded":"2024-04-25T08:52:39.644062+0000","last_fullsized":"2024-04-25T08:52:39.644062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300302+0000","last_change":"2024-04-25T08:52:35.330473+0000","last_active":"2024-04-25T08:52:38.300302+0000","last_peered":"2024-04-25T08:52:38.300302+0000","last_clean":"2024-04-25T08:52:38.300302+0000","last_became_active":"2024-04-25T08:52:35.328741+0000","last_became_peered":"2024-04-25T08:52:35.328741+0000","last_unstale":"2024-04-25T08:52:38.300302+0000","last_undegraded":"2024-04-25T08:52:38.300302+0000","last_fullsized":"2024-04-25T08:52:38.300302+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:14.472581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307492+0000","last_change":"2024-04-25T08:52:30.245422+0000","last_active":"2024-04-25T08:52:39.307492+0000","last_peered":"2024-04-25T08:52:39.307492+0000","last_clean":"2024-04-25T08:52:39.307492+0000","last_became_active":"2024-04-25T08:52:30.244911+0000","last_became_peered":"2024-04-25T08:52:30.244911+0000","last_unstale":"2024-04-25T08:52:39.307492+0000","last_undegraded":"2024-04-25T08:52:39.307492+0000","last_fullsized":"2024-04-25T08:52:39.307492+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644128+0000","last_change":"2024-04-25T08:52:31.274339+0000","last_active":"2024-04-25T08:52:39.644128+0000","last_peered":"2024-04-25T08:52:39.644128+0000","last_clean":"2024-04-25T08:52:39.644128+0000","last_became_active":"2024-04-25T08:52:31.273607+0000","last_became_peered":"2024-04-25T08:52:31.273607+0000","last_unstale":"2024-04-25T08:52:39.644128+0000","last_undegraded":"2024-04-25T08:52:39.644128+0000","last_fullsized":"2024-04-25T08:52:39.644128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:56.409407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299563+0000","last_change":"2024-04-25T08:52:35.316303+0000","last_active":"2024-04-25T08:52:38.299563+0000","last_peered":"2024-04-25T08:52:38.299563+0000","last_clean":"2024-04-25T08:52:38.299563+0000","last_became_active":"2024-04-25T08:52:35.315156+0000","last_became_peered":"2024-04-25T08:52:35.315156+0000","last_unstale":"2024-04-25T08:52:38.299563+0000","last_undegraded":"2024-04-25T08:52:38.299563+0000","last_fullsized":"2024-04-25T08:52:38.299563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:35.695987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300591+0000","last_change":"2024-04-25T08:52:31.269860+0000","last_active":"2024-04-25T08:52:38.300591+0000","last_peered":"2024-04-25T08:52:38.300591+0000","last_clean":"2024-04-25T08:52:38.300591+0000","last_became_active":"2024-04-25T08:52:31.268731+0000","last_became_peered":"2024-04-25T08:52:31.268731+0000","last_unstale":"2024-04-25T08:52:38.300591+0000","last_undegraded":"2024-04-25T08:52:38.300591+0000","last_fullsized":"2024-04-25T08:52:38.300591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307114+0000","last_change":"2024-04-25T08:52:30.246021+0000","last_active":"2024-04-25T08:52:39.307114+0000","last_peered":"2024-04-25T08:52:39.307114+0000","last_clean":"2024-04-25T08:52:39.307114+0000","last_became_active":"2024-04-25T08:52:30.245132+0000","last_became_peered":"2024-04-25T08:52:30.245132+0000","last_unstale":"2024-04-25T08:52:39.307114+0000","last_undegraded":"2024-04-25T08:52:39.307114+0000","last_fullsized":"2024-04-25T08:52:39.307114+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:11.290953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301681+0000","last_change":"2024-04-25T08:52:35.323528+0000","last_active":"2024-04-25T08:52:38.301681+0000","last_peered":"2024-04-25T08:52:38.301681+0000","last_clean":"2024-04-25T08:52:38.301681+0000","last_became_active":"2024-04-25T08:52:35.320911+0000","last_became_peered":"2024-04-25T08:52:35.320911+0000","last_unstale":"2024-04-25T08:52:38.301681+0000","last_undegraded":"2024-04-25T08:52:38.301681+0000","last_fullsized":"2024-04-25T08:52:38.301681+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835879+0000","last_change":"2024-04-25T08:52:31.257885+0000","last_active":"2024-04-25T08:52:39.835879+0000","last_peered":"2024-04-25T08:52:39.835879+0000","last_clean":"2024-04-25T08:52:39.835879+0000","last_became_active":"2024-04-25T08:52:31.257583+0000","last_became_peered":"2024-04-25T08:52:31.257583+0000","last_unstale":"2024-04-25T08:52:39.835879+0000","last_undegraded":"2024-04-25T08:52:39.835879+0000","last_fullsized":"2024-04-25T08:52:39.835879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834075+0000","last_change":"2024-04-25T08:52:30.213612+0000","last_active":"2024-04-25T08:52:39.834075+0000","last_peered":"2024-04-25T08:52:39.834075+0000","last_clean":"2024-04-25T08:52:39.834075+0000","last_became_active":"2024-04-25T08:52:30.212962+0000","last_became_peered":"2024-04-25T08:52:30.212962+0000","last_unstale":"2024-04-25T08:52:39.834075+0000","last_undegraded":"2024-04-25T08:52:39.834075+0000","last_fullsized":"2024-04-25T08:52:39.834075+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300390+0000","last_change":"2024-04-25T08:52:35.324597+0000","last_active":"2024-04-25T08:52:38.300390+0000","last_peered":"2024-04-25T08:52:38.300390+0000","last_clean":"2024-04-25T08:52:38.300390+0000","last_became_active":"2024-04-25T08:52:35.323534+0000","last_became_peered":"2024-04-25T08:52:35.323534+0000","last_unstale":"2024-04-25T08:52:38.300390+0000","last_undegraded":"2024-04-25T08:52:38.300390+0000","last_fullsized":"2024-04-25T08:52:38.300390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:24.182927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834456+0000","last_change":"2024-04-25T08:52:31.231520+0000","last_active":"2024-04-25T08:52:39.834456+0000","last_peered":"2024-04-25T08:52:39.834456+0000","last_clean":"2024-04-25T08:52:39.834456+0000","last_became_active":"2024-04-25T08:52:31.231009+0000","last_became_peered":"2024-04-25T08:52:31.231009+0000","last_unstale":"2024-04-25T08:52:39.834456+0000","last_undegraded":"2024-04-25T08:52:39.834456+0000","last_fullsized":"2024-04-25T08:52:39.834456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834462+0000","last_change":"2024-04-25T08:52:30.231226+0000","last_active":"2024-04-25T08:52:39.834462+0000","last_peered":"2024-04-25T08:52:39.834462+0000","last_clean":"2024-04-25T08:52:39.834462+0000","last_became_active":"2024-04-25T08:52:30.230790+0000","last_became_peered":"2024-04-25T08:52:30.230790+0000","last_unstale":"2024-04-25T08:52:39.834462+0000","last_undegraded":"2024-04-25T08:52:39.834462+0000","last_fullsized":"2024-04-25T08:52:39.834462+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:11.290953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834915+0000","last_change":"2024-04-25T08:52:36.004737+0000","last_active":"2024-04-25T08:52:39.834915+0000","last_peered":"2024-04-25T08:52:39.834915+0000","last_clean":"2024-04-25T08:52:39.834915+0000","last_became_active":"2024-04-25T08:52:36.003639+0000","last_became_peered":"2024-04-25T08:52:36.003639+0000","last_unstale":"2024-04-25T08:52:39.834915+0000","last_undegraded":"2024-04-25T08:52:39.834915+0000","last_fullsized":"2024-04-25T08:52:39.834915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301223+0000","last_change":"2024-04-25T08:52:31.260426+0000","last_active":"2024-04-25T08:52:38.301223+0000","last_peered":"2024-04-25T08:52:38.301223+0000","last_clean":"2024-04-25T08:52:38.301223+0000","last_became_active":"2024-04-25T08:52:31.259594+0000","last_became_peered":"2024-04-25T08:52:31.259594+0000","last_unstale":"2024-04-25T08:52:38.301223+0000","last_undegraded":"2024-04-25T08:52:38.301223+0000","last_fullsized":"2024-04-25T08:52:38.301223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:45.962867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306464+0000","last_change":"2024-04-25T08:52:30.240896+0000","last_active":"2024-04-25T08:52:39.306464+0000","last_peered":"2024-04-25T08:52:39.306464+0000","last_clean":"2024-04-25T08:52:39.306464+0000","last_became_active":"2024-04-25T08:52:30.240290+0000","last_became_peered":"2024-04-25T08:52:30.240290+0000","last_unstale":"2024-04-25T08:52:39.306464+0000","last_undegraded":"2024-04-25T08:52:39.306464+0000","last_fullsized":"2024-04-25T08:52:39.306464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300735+0000","last_change":"2024-04-25T08:52:35.328692+0000","last_active":"2024-04-25T08:52:38.300735+0000","last_peered":"2024-04-25T08:52:38.300735+0000","last_clean":"2024-04-25T08:52:38.300735+0000","last_became_active":"2024-04-25T08:52:35.323899+0000","last_became_peered":"2024-04-25T08:52:35.323899+0000","last_unstale":"2024-04-25T08:52:38.300735+0000","last_undegraded":"2024-04-25T08:52:38.300735+0000","last_fullsized":"2024-04-25T08:52:38.300735+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836941+0000","last_change":"2024-04-25T08:52:31.253016+0000","last_active":"2024-04-25T08:52:39.836941+0000","last_peered":"2024-04-25T08:52:39.836941+0000","last_clean":"2024-04-25T08:52:39.836941+0000","last_became_active":"2024-04-25T08:52:31.252369+0000","last_became_peered":"2024-04-25T08:52:31.252369+0000","last_unstale":"2024-04-25T08:52:39.836941+0000","last_undegraded":"2024-04-25T08:52:39.836941+0000","last_fullsized":"2024-04-25T08:52:39.836941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303292+0000","last_change":"2024-04-25T08:52:30.241679+0000","last_active":"2024-04-25T08:52:38.303292+0000","last_peered":"2024-04-25T08:52:38.303292+0000","last_clean":"2024-04-25T08:52:38.303292+0000","last_became_active":"2024-04-25T08:52:30.241063+0000","last_became_peered":"2024-04-25T08:52:30.241063+0000","last_unstale":"2024-04-25T08:52:38.303292+0000","last_undegraded":"2024-04-25T08:52:38.303292+0000","last_fullsized":"2024-04-25T08:52:38.303292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300540+0000","last_change":"2024-04-25T08:52:36.007015+0000","last_active":"2024-04-25T08:52:38.300540+0000","last_peered":"2024-04-25T08:52:38.300540+0000","last_clean":"2024-04-25T08:52:38.300540+0000","last_became_active":"2024-04-25T08:52:36.005832+0000","last_became_peered":"2024-04-25T08:52:36.005832+0000","last_unstale":"2024-04-25T08:52:38.300540+0000","last_undegraded":"2024-04-25T08:52:38.300540+0000","last_fullsized":"2024-04-25T08:52:38.300540+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:19.422348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834979+0000","last_change":"2024-04-25T08:52:31.282175+0000","last_active":"2024-04-25T08:52:39.834979+0000","last_peered":"2024-04-25T08:52:39.834979+0000","last_clean":"2024-04-25T08:52:39.834979+0000","last_became_active":"2024-04-25T08:52:31.281955+0000","last_became_peered":"2024-04-25T08:52:31.281955+0000","last_unstale":"2024-04-25T08:52:39.834979+0000","last_undegraded":"2024-04-25T08:52:39.834979+0000","last_fullsized":"2024-04-25T08:52:39.834979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834733+0000","last_change":"2024-04-25T08:52:30.215220+0000","last_active":"2024-04-25T08:52:39.834733+0000","last_peered":"2024-04-25T08:52:39.834733+0000","last_clean":"2024-04-25T08:52:39.834733+0000","last_became_active":"2024-04-25T08:52:30.214773+0000","last_became_peered":"2024-04-25T08:52:30.214773+0000","last_unstale":"2024-04-25T08:52:39.834733+0000","last_undegraded":"2024-04-25T08:52:39.834733+0000","last_fullsized":"2024-04-25T08:52:39.834733+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833949+0000","last_change":"2024-04-25T08:52:36.003489+0000","last_active":"2024-04-25T08:52:39.833949+0000","last_peered":"2024-04-25T08:52:39.833949+0000","last_clean":"2024-04-25T08:52:39.833949+0000","last_became_active":"2024-04-25T08:52:36.002295+0000","last_became_peered":"2024-04-25T08:52:36.002295+0000","last_unstale":"2024-04-25T08:52:39.833949+0000","last_undegraded":"2024-04-25T08:52:39.833949+0000","last_fullsized":"2024-04-25T08:52:39.833949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:47.463029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833050+0000","last_change":"2024-04-25T08:52:31.226718+0000","last_active":"2024-04-25T08:52:39.833050+0000","last_peered":"2024-04-25T08:52:39.833050+0000","last_clean":"2024-04-25T08:52:39.833050+0000","last_became_active":"2024-04-25T08:52:31.226442+0000","last_became_peered":"2024-04-25T08:52:31.226442+0000","last_unstale":"2024-04-25T08:52:39.833050+0000","last_undegraded":"2024-04-25T08:52:39.833050+0000","last_fullsized":"2024-04-25T08:52:39.833050+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307955+0000","last_change":"2024-04-25T08:52:30.249239+0000","last_active":"2024-04-25T08:52:39.307955+0000","last_peered":"2024-04-25T08:52:39.307955+0000","last_clean":"2024-04-25T08:52:39.307955+0000","last_became_active":"2024-04-25T08:52:30.248612+0000","last_became_peered":"2024-04-25T08:52:30.248612+0000","last_unstale":"2024-04-25T08:52:39.307955+0000","last_undegraded":"2024-04-25T08:52:39.307955+0000","last_fullsized":"2024-04-25T08:52:39.307955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:34.198893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836268+0000","last_change":"2024-04-25T08:52:36.001292+0000","last_active":"2024-04-25T08:52:39.836268+0000","last_peered":"2024-04-25T08:52:39.836268+0000","last_clean":"2024-04-25T08:52:39.836268+0000","last_became_active":"2024-04-25T08:52:36.000113+0000","last_became_peered":"2024-04-25T08:52:36.000113+0000","last_unstale":"2024-04-25T08:52:39.836268+0000","last_undegraded":"2024-04-25T08:52:39.836268+0000","last_fullsized":"2024-04-25T08:52:39.836268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645594+0000","last_change":"2024-04-25T08:52:31.247003+0000","last_active":"2024-04-25T08:52:39.645594+0000","last_peered":"2024-04-25T08:52:39.645594+0000","last_clean":"2024-04-25T08:52:39.645594+0000","last_became_active":"2024-04-25T08:52:31.246200+0000","last_became_peered":"2024-04-25T08:52:31.246200+0000","last_unstale":"2024-04-25T08:52:39.645594+0000","last_undegraded":"2024-04-25T08:52:39.645594+0000","last_fullsized":"2024-04-25T08:52:39.645594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:52.007683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835389+0000","last_change":"2024-04-25T08:52:30.215091+0000","last_active":"2024-04-25T08:52:39.835389+0000","last_peered":"2024-04-25T08:52:39.835389+0000","last_clean":"2024-04-25T08:52:39.835389+0000","last_became_active":"2024-04-25T08:52:30.214664+0000","last_became_peered":"2024-04-25T08:52:30.214664+0000","last_unstale":"2024-04-25T08:52:39.835389+0000","last_undegraded":"2024-04-25T08:52:39.835389+0000","last_fullsized":"2024-04-25T08:52:39.835389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833696+0000","last_change":"2024-04-25T08:52:35.313874+0000","last_active":"2024-04-25T08:52:39.833696+0000","last_peered":"2024-04-25T08:52:39.833696+0000","last_clean":"2024-04-25T08:52:39.833696+0000","last_became_active":"2024-04-25T08:52:35.313429+0000","last_became_peered":"2024-04-25T08:52:35.313429+0000","last_unstale":"2024-04-25T08:52:39.833696+0000","last_undegraded":"2024-04-25T08:52:39.833696+0000","last_fullsized":"2024-04-25T08:52:39.833696+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:59.165277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300917+0000","last_change":"2024-04-25T08:52:31.257574+0000","last_active":"2024-04-25T08:52:38.300917+0000","last_peered":"2024-04-25T08:52:38.300917+0000","last_clean":"2024-04-25T08:52:38.300917+0000","last_became_active":"2024-04-25T08:52:31.257040+0000","last_became_peered":"2024-04-25T08:52:31.257040+0000","last_unstale":"2024-04-25T08:52:38.300917+0000","last_undegraded":"2024-04-25T08:52:38.300917+0000","last_fullsized":"2024-04-25T08:52:38.300917+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:52.221264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645682+0000","last_change":"2024-04-25T08:52:30.225801+0000","last_active":"2024-04-25T08:52:39.645682+0000","last_peered":"2024-04-25T08:52:39.645682+0000","last_clean":"2024-04-25T08:52:39.645682+0000","last_became_active":"2024-04-25T08:52:30.224857+0000","last_became_peered":"2024-04-25T08:52:30.224857+0000","last_unstale":"2024-04-25T08:52:39.645682+0000","last_undegraded":"2024-04-25T08:52:39.645682+0000","last_fullsized":"2024-04-25T08:52:39.645682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833379+0000","last_change":"2024-04-25T08:52:35.309762+0000","last_active":"2024-04-25T08:52:39.833379+0000","last_peered":"2024-04-25T08:52:39.833379+0000","last_clean":"2024-04-25T08:52:39.833379+0000","last_became_active":"2024-04-25T08:52:35.309315+0000","last_became_peered":"2024-04-25T08:52:35.309315+0000","last_unstale":"2024-04-25T08:52:39.833379+0000","last_undegraded":"2024-04-25T08:52:39.833379+0000","last_fullsized":"2024-04-25T08:52:39.833379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.832782+0000","last_change":"2024-04-25T08:52:31.232199+0000","last_active":"2024-04-25T08:52:39.832782+0000","last_peered":"2024-04-25T08:52:39.832782+0000","last_clean":"2024-04-25T08:52:39.832782+0000","last_became_active":"2024-04-25T08:52:31.231928+0000","last_became_peered":"2024-04-25T08:52:31.231928+0000","last_unstale":"2024-04-25T08:52:39.832782+0000","last_undegraded":"2024-04-25T08:52:39.832782+0000","last_fullsized":"2024-04-25T08:52:39.832782+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299166+0000","last_change":"2024-04-25T08:52:30.218615+0000","last_active":"2024-04-25T08:52:38.299166+0000","last_peered":"2024-04-25T08:52:38.299166+0000","last_clean":"2024-04-25T08:52:38.299166+0000","last_became_active":"2024-04-25T08:52:30.213378+0000","last_became_peered":"2024-04-25T08:52:30.213378+0000","last_unstale":"2024-04-25T08:52:38.299166+0000","last_undegraded":"2024-04-25T08:52:38.299166+0000","last_fullsized":"2024-04-25T08:52:38.299166+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834271+0000","last_change":"2024-04-25T08:52:35.311156+0000","last_active":"2024-04-25T08:52:39.834271+0000","last_peered":"2024-04-25T08:52:39.834271+0000","last_clean":"2024-04-25T08:52:39.834271+0000","last_became_active":"2024-04-25T08:52:35.310700+0000","last_became_peered":"2024-04-25T08:52:35.310700+0000","last_unstale":"2024-04-25T08:52:39.834271+0000","last_undegraded":"2024-04-25T08:52:39.834271+0000","last_fullsized":"2024-04-25T08:52:39.834271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.832874+0000","last_change":"2024-04-25T08:52:31.260514+0000","last_active":"2024-04-25T08:52:39.832874+0000","last_peered":"2024-04-25T08:52:39.832874+0000","last_clean":"2024-04-25T08:52:39.832874+0000","last_became_active":"2024-04-25T08:52:31.260019+0000","last_became_peered":"2024-04-25T08:52:31.260019+0000","last_unstale":"2024-04-25T08:52:39.832874+0000","last_undegraded":"2024-04-25T08:52:39.832874+0000","last_fullsized":"2024-04-25T08:52:39.832874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.602114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306466+0000","last_change":"2024-04-25T08:52:30.245223+0000","last_active":"2024-04-25T08:52:39.306466+0000","last_peered":"2024-04-25T08:52:39.306466+0000","last_clean":"2024-04-25T08:52:39.306466+0000","last_became_active":"2024-04-25T08:52:30.244782+0000","last_became_peered":"2024-04-25T08:52:30.244782+0000","last_unstale":"2024-04-25T08:52:39.306466+0000","last_undegraded":"2024-04-25T08:52:39.306466+0000","last_fullsized":"2024-04-25T08:52:39.306466+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836419+0000","last_change":"2024-04-25T08:52:36.002202+0000","last_active":"2024-04-25T08:52:39.836419+0000","last_peered":"2024-04-25T08:52:39.836419+0000","last_clean":"2024-04-25T08:52:39.836419+0000","last_became_active":"2024-04-25T08:52:36.000348+0000","last_became_peered":"2024-04-25T08:52:36.000348+0000","last_unstale":"2024-04-25T08:52:39.836419+0000","last_undegraded":"2024-04-25T08:52:39.836419+0000","last_fullsized":"2024-04-25T08:52:39.836419+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:12.373033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834845+0000","last_change":"2024-04-25T08:52:31.257777+0000","last_active":"2024-04-25T08:52:39.834845+0000","last_peered":"2024-04-25T08:52:39.834845+0000","last_clean":"2024-04-25T08:52:39.834845+0000","last_became_active":"2024-04-25T08:52:31.257295+0000","last_became_peered":"2024-04-25T08:52:31.257295+0000","last_unstale":"2024-04-25T08:52:39.834845+0000","last_undegraded":"2024-04-25T08:52:39.834845+0000","last_fullsized":"2024-04-25T08:52:39.834845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645136+0000","last_change":"2024-04-25T08:52:30.222595+0000","last_active":"2024-04-25T08:52:39.645136+0000","last_peered":"2024-04-25T08:52:39.645136+0000","last_clean":"2024-04-25T08:52:39.645136+0000","last_became_active":"2024-04-25T08:52:30.217788+0000","last_became_peered":"2024-04-25T08:52:30.217788+0000","last_unstale":"2024-04-25T08:52:39.645136+0000","last_undegraded":"2024-04-25T08:52:39.645136+0000","last_fullsized":"2024-04-25T08:52:39.645136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:52.505423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":182,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306756+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:52:39.306756+0000","last_peered":"2024-04-25T08:52:39.306756+0000","last_clean":"2024-04-25T08:52:39.306756+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:52:39.306756+0000","last_undegraded":"2024-04-25T08:52:39.306756+0000","last_fullsized":"2024-04-25T08:52:39.306756+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.302216+0000","last_change":"2024-04-25T08:52:36.004910+0000","last_active":"2024-04-25T08:52:38.302216+0000","last_peered":"2024-04-25T08:52:38.302216+0000","last_clean":"2024-04-25T08:52:38.302216+0000","last_became_active":"2024-04-25T08:52:36.003878+0000","last_became_peered":"2024-04-25T08:52:36.003878+0000","last_unstale":"2024-04-25T08:52:38.302216+0000","last_undegraded":"2024-04-25T08:52:38.302216+0000","last_fullsized":"2024-04-25T08:52:38.302216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:12.106880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307624+0000","last_change":"2024-04-25T08:52:31.286252+0000","last_active":"2024-04-25T08:52:39.307624+0000","last_peered":"2024-04-25T08:52:39.307624+0000","last_clean":"2024-04-25T08:52:39.307624+0000","last_became_active":"2024-04-25T08:52:31.282622+0000","last_became_peered":"2024-04-25T08:52:31.282622+0000","last_unstale":"2024-04-25T08:52:39.307624+0000","last_undegraded":"2024-04-25T08:52:39.307624+0000","last_fullsized":"2024-04-25T08:52:39.307624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299865+0000","last_change":"2024-04-25T08:52:30.226514+0000","last_active":"2024-04-25T08:52:38.299865+0000","last_peered":"2024-04-25T08:52:38.299865+0000","last_clean":"2024-04-25T08:52:38.299865+0000","last_became_active":"2024-04-25T08:52:30.226243+0000","last_became_peered":"2024-04-25T08:52:30.226243+0000","last_unstale":"2024-04-25T08:52:38.299865+0000","last_undegraded":"2024-04-25T08:52:38.299865+0000","last_fullsized":"2024-04-25T08:52:38.299865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300450+0000","last_change":"2024-04-25T08:52:36.003056+0000","last_active":"2024-04-25T08:52:38.300450+0000","last_peered":"2024-04-25T08:52:38.300450+0000","last_clean":"2024-04-25T08:52:38.300450+0000","last_became_active":"2024-04-25T08:52:36.000882+0000","last_became_peered":"2024-04-25T08:52:36.000882+0000","last_unstale":"2024-04-25T08:52:38.300450+0000","last_undegraded":"2024-04-25T08:52:38.300450+0000","last_fullsized":"2024-04-25T08:52:38.300450+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:56.088772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645689+0000","last_change":"2024-04-25T08:52:31.244866+0000","last_active":"2024-04-25T08:52:39.645689+0000","last_peered":"2024-04-25T08:52:39.645689+0000","last_clean":"2024-04-25T08:52:39.645689+0000","last_became_active":"2024-04-25T08:52:31.244249+0000","last_became_peered":"2024-04-25T08:52:31.244249+0000","last_unstale":"2024-04-25T08:52:39.645689+0000","last_undegraded":"2024-04-25T08:52:39.645689+0000","last_fullsized":"2024-04-25T08:52:39.645689+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:52.330848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835210+0000","last_change":"2024-04-25T08:52:30.228380+0000","last_active":"2024-04-25T08:52:39.835210+0000","last_peered":"2024-04-25T08:52:39.835210+0000","last_clean":"2024-04-25T08:52:39.835210+0000","last_became_active":"2024-04-25T08:52:30.227732+0000","last_became_peered":"2024-04-25T08:52:30.227732+0000","last_unstale":"2024-04-25T08:52:39.835210+0000","last_undegraded":"2024-04-25T08:52:39.835210+0000","last_fullsized":"2024-04-25T08:52:39.835210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:46.012895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836520+0000","last_change":"2024-04-25T08:52:35.316462+0000","last_active":"2024-04-25T08:52:39.836520+0000","last_peered":"2024-04-25T08:52:39.836520+0000","last_clean":"2024-04-25T08:52:39.836520+0000","last_became_active":"2024-04-25T08:52:35.315760+0000","last_became_peered":"2024-04-25T08:52:35.315760+0000","last_unstale":"2024-04-25T08:52:39.836520+0000","last_undegraded":"2024-04-25T08:52:39.836520+0000","last_fullsized":"2024-04-25T08:52:39.836520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:49.973872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305761+0000","last_change":"2024-04-25T08:52:31.289534+0000","last_active":"2024-04-25T08:52:39.305761+0000","last_peered":"2024-04-25T08:52:39.305761+0000","last_clean":"2024-04-25T08:52:39.305761+0000","last_became_active":"2024-04-25T08:52:31.288580+0000","last_became_peered":"2024-04-25T08:52:31.288580+0000","last_unstale":"2024-04-25T08:52:39.305761+0000","last_undegraded":"2024-04-25T08:52:39.305761+0000","last_fullsized":"2024-04-25T08:52:39.305761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:10.391492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299950+0000","last_change":"2024-04-25T08:52:30.215006+0000","last_active":"2024-04-25T08:52:38.299950+0000","last_peered":"2024-04-25T08:52:38.299950+0000","last_clean":"2024-04-25T08:52:38.299950+0000","last_became_active":"2024-04-25T08:52:30.213814+0000","last_became_peered":"2024-04-25T08:52:30.213814+0000","last_unstale":"2024-04-25T08:52:38.299950+0000","last_undegraded":"2024-04-25T08:52:38.299950+0000","last_fullsized":"2024-04-25T08:52:38.299950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644315+0000","last_change":"2024-04-25T08:52:36.005410+0000","last_active":"2024-04-25T08:52:39.644315+0000","last_peered":"2024-04-25T08:52:39.644315+0000","last_clean":"2024-04-25T08:52:39.644315+0000","last_became_active":"2024-04-25T08:52:36.004309+0000","last_became_peered":"2024-04-25T08:52:36.004309+0000","last_unstale":"2024-04-25T08:52:39.644315+0000","last_undegraded":"2024-04-25T08:52:39.644315+0000","last_fullsized":"2024-04-25T08:52:39.644315+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305320+0000","last_change":"2024-04-25T08:52:31.285984+0000","last_active":"2024-04-25T08:52:39.305320+0000","last_peered":"2024-04-25T08:52:39.305320+0000","last_clean":"2024-04-25T08:52:39.305320+0000","last_became_active":"2024-04-25T08:52:31.281988+0000","last_became_peered":"2024-04-25T08:52:31.281988+0000","last_unstale":"2024-04-25T08:52:39.305320+0000","last_undegraded":"2024-04-25T08:52:39.305320+0000","last_fullsized":"2024-04-25T08:52:39.305320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:56.409407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301053+0000","last_change":"2024-04-25T08:52:30.215598+0000","last_active":"2024-04-25T08:52:38.301053+0000","last_peered":"2024-04-25T08:52:38.301053+0000","last_clean":"2024-04-25T08:52:38.301053+0000","last_became_active":"2024-04-25T08:52:30.214765+0000","last_became_peered":"2024-04-25T08:52:30.214765+0000","last_unstale":"2024-04-25T08:52:38.301053+0000","last_undegraded":"2024-04-25T08:52:38.301053+0000","last_fullsized":"2024-04-25T08:52:38.301053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:36.565944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300087+0000","last_change":"2024-04-25T08:52:35.328574+0000","last_active":"2024-04-25T08:52:38.300087+0000","last_peered":"2024-04-25T08:52:38.300087+0000","last_clean":"2024-04-25T08:52:38.300087+0000","last_became_active":"2024-04-25T08:52:35.323891+0000","last_became_peered":"2024-04-25T08:52:35.323891+0000","last_unstale":"2024-04-25T08:52:38.300087+0000","last_undegraded":"2024-04-25T08:52:38.300087+0000","last_fullsized":"2024-04-25T08:52:38.300087+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:12.106880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305125+0000","last_change":"2024-04-25T08:52:31.278037+0000","last_active":"2024-04-25T08:52:39.305125+0000","last_peered":"2024-04-25T08:52:39.305125+0000","last_clean":"2024-04-25T08:52:39.305125+0000","last_became_active":"2024-04-25T08:52:31.277117+0000","last_became_peered":"2024-04-25T08:52:31.277117+0000","last_unstale":"2024-04-25T08:52:39.305125+0000","last_undegraded":"2024-04-25T08:52:39.305125+0000","last_fullsized":"2024-04-25T08:52:39.305125+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298797+0000","last_change":"2024-04-25T08:52:30.221811+0000","last_active":"2024-04-25T08:52:38.298797+0000","last_peered":"2024-04-25T08:52:38.298797+0000","last_clean":"2024-04-25T08:52:38.298797+0000","last_became_active":"2024-04-25T08:52:30.220494+0000","last_became_peered":"2024-04-25T08:52:30.220494+0000","last_unstale":"2024-04-25T08:52:38.298797+0000","last_undegraded":"2024-04-25T08:52:38.298797+0000","last_fullsized":"2024-04-25T08:52:38.298797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:03.139393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836281+0000","last_change":"2024-04-25T08:52:35.317037+0000","last_active":"2024-04-25T08:52:39.836281+0000","last_peered":"2024-04-25T08:52:39.836281+0000","last_clean":"2024-04-25T08:52:39.836281+0000","last_became_active":"2024-04-25T08:52:35.316491+0000","last_became_peered":"2024-04-25T08:52:35.316491+0000","last_unstale":"2024-04-25T08:52:39.836281+0000","last_undegraded":"2024-04-25T08:52:39.836281+0000","last_fullsized":"2024-04-25T08:52:39.836281+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:33.132823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299015+0000","last_change":"2024-04-25T08:52:31.256415+0000","last_active":"2024-04-25T08:52:38.299015+0000","last_peered":"2024-04-25T08:52:38.299015+0000","last_clean":"2024-04-25T08:52:38.299015+0000","last_became_active":"2024-04-25T08:52:31.254747+0000","last_became_peered":"2024-04-25T08:52:31.254747+0000","last_unstale":"2024-04-25T08:52:38.299015+0000","last_undegraded":"2024-04-25T08:52:38.299015+0000","last_fullsized":"2024-04-25T08:52:38.299015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:24.567213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299991+0000","last_change":"2024-04-25T08:52:30.223725+0000","last_active":"2024-04-25T08:52:38.299991+0000","last_peered":"2024-04-25T08:52:38.299991+0000","last_clean":"2024-04-25T08:52:38.299991+0000","last_became_active":"2024-04-25T08:52:30.222820+0000","last_became_peered":"2024-04-25T08:52:30.222820+0000","last_unstale":"2024-04-25T08:52:38.299991+0000","last_undegraded":"2024-04-25T08:52:38.299991+0000","last_fullsized":"2024-04-25T08:52:38.299991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834851+0000","last_change":"2024-04-25T08:52:36.003180+0000","last_active":"2024-04-25T08:52:39.834851+0000","last_peered":"2024-04-25T08:52:39.834851+0000","last_clean":"2024-04-25T08:52:39.834851+0000","last_became_active":"2024-04-25T08:52:36.002030+0000","last_became_peered":"2024-04-25T08:52:36.002030+0000","last_unstale":"2024-04-25T08:52:39.834851+0000","last_undegraded":"2024-04-25T08:52:39.834851+0000","last_fullsized":"2024-04-25T08:52:39.834851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:58.721596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305176+0000","last_change":"2024-04-25T08:52:31.278626+0000","last_active":"2024-04-25T08:52:39.305176+0000","last_peered":"2024-04-25T08:52:39.305176+0000","last_clean":"2024-04-25T08:52:39.305176+0000","last_became_active":"2024-04-25T08:52:31.278027+0000","last_became_peered":"2024-04-25T08:52:31.278027+0000","last_unstale":"2024-04-25T08:52:39.305176+0000","last_undegraded":"2024-04-25T08:52:39.305176+0000","last_fullsized":"2024-04-25T08:52:39.305176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:15.341259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645485+0000","last_change":"2024-04-25T08:52:30.233250+0000","last_active":"2024-04-25T08:52:39.645485+0000","last_peered":"2024-04-25T08:52:39.645485+0000","last_clean":"2024-04-25T08:52:39.645485+0000","last_became_active":"2024-04-25T08:52:30.229813+0000","last_became_peered":"2024-04-25T08:52:30.229813+0000","last_unstale":"2024-04-25T08:52:39.645485+0000","last_undegraded":"2024-04-25T08:52:39.645485+0000","last_fullsized":"2024-04-25T08:52:39.645485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835406+0000","last_change":"2024-04-25T08:52:36.001662+0000","last_active":"2024-04-25T08:52:39.835406+0000","last_peered":"2024-04-25T08:52:39.835406+0000","last_clean":"2024-04-25T08:52:39.835406+0000","last_became_active":"2024-04-25T08:52:36.000433+0000","last_became_peered":"2024-04-25T08:52:36.000433+0000","last_unstale":"2024-04-25T08:52:39.835406+0000","last_undegraded":"2024-04-25T08:52:39.835406+0000","last_fullsized":"2024-04-25T08:52:39.835406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:13.578788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836042+0000","last_change":"2024-04-25T08:52:31.278698+0000","last_active":"2024-04-25T08:52:39.836042+0000","last_peered":"2024-04-25T08:52:39.836042+0000","last_clean":"2024-04-25T08:52:39.836042+0000","last_became_active":"2024-04-25T08:52:31.278388+0000","last_became_peered":"2024-04-25T08:52:31.278388+0000","last_unstale":"2024-04-25T08:52:39.836042+0000","last_undegraded":"2024-04-25T08:52:39.836042+0000","last_fullsized":"2024-04-25T08:52:39.836042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:24.567213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836775+0000","last_change":"2024-04-25T08:52:30.197089+0000","last_active":"2024-04-25T08:52:39.836775+0000","last_peered":"2024-04-25T08:52:39.836775+0000","last_clean":"2024-04-25T08:52:39.836775+0000","last_became_active":"2024-04-25T08:52:30.196592+0000","last_became_peered":"2024-04-25T08:52:30.196592+0000","last_unstale":"2024-04-25T08:52:39.836775+0000","last_undegraded":"2024-04-25T08:52:39.836775+0000","last_fullsized":"2024-04-25T08:52:39.836775+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645712+0000","last_change":"2024-04-25T08:52:36.005556+0000","last_active":"2024-04-25T08:52:39.645712+0000","last_peered":"2024-04-25T08:52:39.645712+0000","last_clean":"2024-04-25T08:52:39.645712+0000","last_became_active":"2024-04-25T08:52:36.004511+0000","last_became_peered":"2024-04-25T08:52:36.004511+0000","last_unstale":"2024-04-25T08:52:39.645712+0000","last_undegraded":"2024-04-25T08:52:39.645712+0000","last_fullsized":"2024-04-25T08:52:39.645712+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:46.222649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.307456+0000","last_change":"2024-04-25T08:52:31.286270+0000","last_active":"2024-04-25T08:52:39.307456+0000","last_peered":"2024-04-25T08:52:39.307456+0000","last_clean":"2024-04-25T08:52:39.307456+0000","last_became_active":"2024-04-25T08:52:31.281689+0000","last_became_peered":"2024-04-25T08:52:31.281689+0000","last_unstale":"2024-04-25T08:52:39.307456+0000","last_undegraded":"2024-04-25T08:52:39.307456+0000","last_fullsized":"2024-04-25T08:52:39.307456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'9","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:41.689859+0000","last_change":"2024-04-25T08:52:30.215322+0000","last_active":"2024-04-25T08:52:41.689859+0000","last_peered":"2024-04-25T08:52:41.689859+0000","last_clean":"2024-04-25T08:52:41.689859+0000","last_became_active":"2024-04-25T08:52:30.214531+0000","last_became_peered":"2024-04-25T08:52:30.214531+0000","last_unstale":"2024-04-25T08:52:41.689859+0000","last_undegraded":"2024-04-25T08:52:41.689859+0000","last_fullsized":"2024-04-25T08:52:41.689859+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833881+0000","last_change":"2024-04-25T08:52:35.307400+0000","last_active":"2024-04-25T08:52:39.833881+0000","last_peered":"2024-04-25T08:52:39.833881+0000","last_clean":"2024-04-25T08:52:39.833881+0000","last_became_active":"2024-04-25T08:52:35.304579+0000","last_became_peered":"2024-04-25T08:52:35.304579+0000","last_unstale":"2024-04-25T08:52:39.833881+0000","last_undegraded":"2024-04-25T08:52:39.833881+0000","last_fullsized":"2024-04-25T08:52:39.833881+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:24.182927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301767+0000","last_change":"2024-04-25T08:52:31.262376+0000","last_active":"2024-04-25T08:52:38.301767+0000","last_peered":"2024-04-25T08:52:38.301767+0000","last_clean":"2024-04-25T08:52:38.301767+0000","last_became_active":"2024-04-25T08:52:31.261642+0000","last_became_peered":"2024-04-25T08:52:31.261642+0000","last_unstale":"2024-04-25T08:52:38.301767+0000","last_undegraded":"2024-04-25T08:52:38.301767+0000","last_fullsized":"2024-04-25T08:52:38.301767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:12.300628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833373+0000","last_change":"2024-04-25T08:52:30.219030+0000","last_active":"2024-04-25T08:52:39.833373+0000","last_peered":"2024-04-25T08:52:39.833373+0000","last_clean":"2024-04-25T08:52:39.833373+0000","last_became_active":"2024-04-25T08:52:30.218788+0000","last_became_peered":"2024-04-25T08:52:30.218788+0000","last_unstale":"2024-04-25T08:52:39.833373+0000","last_undegraded":"2024-04-25T08:52:39.833373+0000","last_fullsized":"2024-04-25T08:52:39.833373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:34.198893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300753+0000","last_change":"2024-04-25T08:52:35.331866+0000","last_active":"2024-04-25T08:52:38.300753+0000","last_peered":"2024-04-25T08:52:38.300753+0000","last_clean":"2024-04-25T08:52:38.300753+0000","last_became_active":"2024-04-25T08:52:35.330775+0000","last_became_peered":"2024-04-25T08:52:35.330775+0000","last_unstale":"2024-04-25T08:52:38.300753+0000","last_undegraded":"2024-04-25T08:52:38.300753+0000","last_fullsized":"2024-04-25T08:52:38.300753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:59.165277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836632+0000","last_change":"2024-04-25T08:52:31.256646+0000","last_active":"2024-04-25T08:52:39.836632+0000","last_peered":"2024-04-25T08:52:39.836632+0000","last_clean":"2024-04-25T08:52:39.836632+0000","last_became_active":"2024-04-25T08:52:31.256362+0000","last_became_peered":"2024-04-25T08:52:31.256362+0000","last_unstale":"2024-04-25T08:52:39.836632+0000","last_undegraded":"2024-04-25T08:52:39.836632+0000","last_fullsized":"2024-04-25T08:52:39.836632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645734+0000","last_change":"2024-04-25T08:52:30.217888+0000","last_active":"2024-04-25T08:52:39.645734+0000","last_peered":"2024-04-25T08:52:39.645734+0000","last_clean":"2024-04-25T08:52:39.645734+0000","last_became_active":"2024-04-25T08:52:30.217100+0000","last_became_peered":"2024-04-25T08:52:30.217100+0000","last_unstale":"2024-04-25T08:52:39.645734+0000","last_undegraded":"2024-04-25T08:52:39.645734+0000","last_fullsized":"2024-04-25T08:52:39.645734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:52.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834254+0000","last_change":"2024-04-25T08:52:35.313671+0000","last_active":"2024-04-25T08:52:39.834254+0000","last_peered":"2024-04-25T08:52:39.834254+0000","last_clean":"2024-04-25T08:52:39.834254+0000","last_became_active":"2024-04-25T08:52:35.312785+0000","last_became_peered":"2024-04-25T08:52:35.312785+0000","last_unstale":"2024-04-25T08:52:39.834254+0000","last_undegraded":"2024-04-25T08:52:39.834254+0000","last_fullsized":"2024-04-25T08:52:39.834254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:53:23.828143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645723+0000","last_change":"2024-04-25T08:52:31.273223+0000","last_active":"2024-04-25T08:52:39.645723+0000","last_peered":"2024-04-25T08:52:39.645723+0000","last_clean":"2024-04-25T08:52:39.645723+0000","last_became_active":"2024-04-25T08:52:31.272627+0000","last_became_peered":"2024-04-25T08:52:31.272627+0000","last_unstale":"2024-04-25T08:52:39.645723+0000","last_undegraded":"2024-04-25T08:52:39.645723+0000","last_fullsized":"2024-04-25T08:52:39.645723+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303022+0000","last_change":"2024-04-25T08:52:30.229619+0000","last_active":"2024-04-25T08:52:38.303022+0000","last_peered":"2024-04-25T08:52:38.303022+0000","last_clean":"2024-04-25T08:52:38.303022+0000","last_became_active":"2024-04-25T08:52:30.228770+0000","last_became_peered":"2024-04-25T08:52:30.228770+0000","last_unstale":"2024-04-25T08:52:38.303022+0000","last_undegraded":"2024-04-25T08:52:38.303022+0000","last_fullsized":"2024-04-25T08:52:38.303022+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:44.953192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833502+0000","last_change":"2024-04-25T08:52:36.008479+0000","last_active":"2024-04-25T08:52:39.833502+0000","last_peered":"2024-04-25T08:52:39.833502+0000","last_clean":"2024-04-25T08:52:39.833502+0000","last_became_active":"2024-04-25T08:52:36.007404+0000","last_became_peered":"2024-04-25T08:52:36.007404+0000","last_unstale":"2024-04-25T08:52:39.833502+0000","last_undegraded":"2024-04-25T08:52:39.833502+0000","last_fullsized":"2024-04-25T08:52:39.833502+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:54.631138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836260+0000","last_change":"2024-04-25T08:52:36.004959+0000","last_active":"2024-04-25T08:52:39.836260+0000","last_peered":"2024-04-25T08:52:39.836260+0000","last_clean":"2024-04-25T08:52:39.836260+0000","last_became_active":"2024-04-25T08:52:36.003575+0000","last_became_peered":"2024-04-25T08:52:36.003575+0000","last_unstale":"2024-04-25T08:52:39.836260+0000","last_undegraded":"2024-04-25T08:52:39.836260+0000","last_fullsized":"2024-04-25T08:52:39.836260+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:30.253907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834357+0000","last_change":"2024-04-25T08:52:31.231230+0000","last_active":"2024-04-25T08:52:39.834357+0000","last_peered":"2024-04-25T08:52:39.834357+0000","last_clean":"2024-04-25T08:52:39.834357+0000","last_became_active":"2024-04-25T08:52:31.230951+0000","last_became_peered":"2024-04-25T08:52:31.230951+0000","last_unstale":"2024-04-25T08:52:39.834357+0000","last_undegraded":"2024-04-25T08:52:39.834357+0000","last_fullsized":"2024-04-25T08:52:39.834357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644784+0000","last_change":"2024-04-25T08:52:30.233534+0000","last_active":"2024-04-25T08:52:39.644784+0000","last_peered":"2024-04-25T08:52:39.644784+0000","last_clean":"2024-04-25T08:52:39.644784+0000","last_became_active":"2024-04-25T08:52:30.232832+0000","last_became_peered":"2024-04-25T08:52:30.232832+0000","last_unstale":"2024-04-25T08:52:39.644784+0000","last_undegraded":"2024-04-25T08:52:39.644784+0000","last_fullsized":"2024-04-25T08:52:39.644784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303102+0000","last_change":"2024-04-25T08:52:35.317543+0000","last_active":"2024-04-25T08:52:38.303102+0000","last_peered":"2024-04-25T08:52:38.303102+0000","last_clean":"2024-04-25T08:52:38.303102+0000","last_became_active":"2024-04-25T08:52:35.316474+0000","last_became_peered":"2024-04-25T08:52:35.316474+0000","last_unstale":"2024-04-25T08:52:38.303102+0000","last_undegraded":"2024-04-25T08:52:38.303102+0000","last_fullsized":"2024-04-25T08:52:38.303102+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:35.695987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834357+0000","last_change":"2024-04-25T08:52:31.257812+0000","last_active":"2024-04-25T08:52:39.834357+0000","last_peered":"2024-04-25T08:52:39.834357+0000","last_clean":"2024-04-25T08:52:39.834357+0000","last_became_active":"2024-04-25T08:52:31.257528+0000","last_became_peered":"2024-04-25T08:52:31.257528+0000","last_unstale":"2024-04-25T08:52:39.834357+0000","last_undegraded":"2024-04-25T08:52:39.834357+0000","last_fullsized":"2024-04-25T08:52:39.834357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835503+0000","last_change":"2024-04-25T08:52:30.217142+0000","last_active":"2024-04-25T08:52:39.835503+0000","last_peered":"2024-04-25T08:52:39.835503+0000","last_clean":"2024-04-25T08:52:39.835503+0000","last_became_active":"2024-04-25T08:52:30.216729+0000","last_became_peered":"2024-04-25T08:52:30.216729+0000","last_unstale":"2024-04-25T08:52:39.835503+0000","last_undegraded":"2024-04-25T08:52:39.835503+0000","last_fullsized":"2024-04-25T08:52:39.835503+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:34:31.537548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300819+0000","last_change":"2024-04-25T08:52:30.228704+0000","last_active":"2024-04-25T08:52:38.300819+0000","last_peered":"2024-04-25T08:52:38.300819+0000","last_clean":"2024-04-25T08:52:38.300819+0000","last_became_active":"2024-04-25T08:52:30.227861+0000","last_became_peered":"2024-04-25T08:52:30.227861+0000","last_unstale":"2024-04-25T08:52:38.300819+0000","last_undegraded":"2024-04-25T08:52:38.300819+0000","last_fullsized":"2024-04-25T08:52:38.300819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:51.211589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.835140+0000","last_change":"2024-04-25T08:52:31.258702+0000","last_active":"2024-04-25T08:52:39.835140+0000","last_peered":"2024-04-25T08:52:39.835140+0000","last_clean":"2024-04-25T08:52:39.835140+0000","last_became_active":"2024-04-25T08:52:31.258349+0000","last_became_peered":"2024-04-25T08:52:31.258349+0000","last_unstale":"2024-04-25T08:52:39.835140+0000","last_undegraded":"2024-04-25T08:52:39.835140+0000","last_fullsized":"2024-04-25T08:52:39.835140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299697+0000","last_change":"2024-04-25T08:52:35.313396+0000","last_active":"2024-04-25T08:52:38.299697+0000","last_peered":"2024-04-25T08:52:38.299697+0000","last_clean":"2024-04-25T08:52:38.299697+0000","last_became_active":"2024-04-25T08:52:35.312845+0000","last_became_peered":"2024-04-25T08:52:35.312845+0000","last_unstale":"2024-04-25T08:52:38.299697+0000","last_undegraded":"2024-04-25T08:52:38.299697+0000","last_fullsized":"2024-04-25T08:52:38.299697+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:28.648302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836001+0000","last_change":"2024-04-25T08:52:30.239169+0000","last_active":"2024-04-25T08:52:39.836001+0000","last_peered":"2024-04-25T08:52:39.836001+0000","last_clean":"2024-04-25T08:52:39.836001+0000","last_became_active":"2024-04-25T08:52:30.238820+0000","last_became_peered":"2024-04-25T08:52:30.238820+0000","last_unstale":"2024-04-25T08:52:39.836001+0000","last_undegraded":"2024-04-25T08:52:39.836001+0000","last_fullsized":"2024-04-25T08:52:39.836001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:29.171847+0000","last_clean_scrub_stamp":"2024-04-25T08:52:29.171847+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:33.932446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.300334+0000","last_change":"2024-04-25T08:52:31.243105+0000","last_active":"2024-04-25T08:52:38.300334+0000","last_peered":"2024-04-25T08:52:38.300334+0000","last_clean":"2024-04-25T08:52:38.300334+0000","last_became_active":"2024-04-25T08:52:31.242326+0000","last_became_peered":"2024-04-25T08:52:31.242326+0000","last_unstale":"2024-04-25T08:52:38.300334+0000","last_undegraded":"2024-04-25T08:52:38.300334+0000","last_fullsized":"2024-04-25T08:52:38.300334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:04.149068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834664+0000","last_change":"2024-04-25T08:52:35.313390+0000","last_active":"2024-04-25T08:52:39.834664+0000","last_peered":"2024-04-25T08:52:39.834664+0000","last_clean":"2024-04-25T08:52:39.834664+0000","last_became_active":"2024-04-25T08:52:35.312750+0000","last_became_peered":"2024-04-25T08:52:35.312750+0000","last_unstale":"2024-04-25T08:52:39.834664+0000","last_undegraded":"2024-04-25T08:52:39.834664+0000","last_fullsized":"2024-04-25T08:52:39.834664+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:34.262611+0000","last_clean_scrub_stamp":"2024-04-25T08:52:34.262611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:49.016809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298785+0000","last_change":"2024-04-25T08:52:31.257097+0000","last_active":"2024-04-25T08:52:38.298785+0000","last_peered":"2024-04-25T08:52:38.298785+0000","last_clean":"2024-04-25T08:52:38.298785+0000","last_became_active":"2024-04-25T08:52:31.255697+0000","last_became_peered":"2024-04-25T08:52:31.255697+0000","last_unstale":"2024-04-25T08:52:38.298785+0000","last_undegraded":"2024-04-25T08:52:38.298785+0000","last_fullsized":"2024-04-25T08:52:38.298785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.298858+0000","last_change":"2024-04-25T08:52:31.237994+0000","last_active":"2024-04-25T08:52:38.298858+0000","last_peered":"2024-04-25T08:52:38.298858+0000","last_clean":"2024-04-25T08:52:38.298858+0000","last_became_active":"2024-04-25T08:52:31.237190+0000","last_became_peered":"2024-04-25T08:52:31.237190+0000","last_unstale":"2024-04-25T08:52:38.298858+0000","last_undegraded":"2024-04-25T08:52:38.298858+0000","last_fullsized":"2024-04-25T08:52:38.298858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:44:35.208568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645427+0000","last_change":"2024-04-25T08:52:31.248873+0000","last_active":"2024-04-25T08:52:39.645427+0000","last_peered":"2024-04-25T08:52:39.645427+0000","last_clean":"2024-04-25T08:52:39.645427+0000","last_became_active":"2024-04-25T08:52:31.248359+0000","last_became_peered":"2024-04-25T08:52:31.248359+0000","last_unstale":"2024-04-25T08:52:39.645427+0000","last_undegraded":"2024-04-25T08:52:39.645427+0000","last_fullsized":"2024-04-25T08:52:39.645427+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836722+0000","last_change":"2024-04-25T08:52:31.256155+0000","last_active":"2024-04-25T08:52:39.836722+0000","last_peered":"2024-04-25T08:52:39.836722+0000","last_clean":"2024-04-25T08:52:39.836722+0000","last_became_active":"2024-04-25T08:52:31.255884+0000","last_became_peered":"2024-04-25T08:52:31.255884+0000","last_unstale":"2024-04-25T08:52:39.836722+0000","last_undegraded":"2024-04-25T08:52:39.836722+0000","last_fullsized":"2024-04-25T08:52:39.836722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834472+0000","last_change":"2024-04-25T08:52:31.266984+0000","last_active":"2024-04-25T08:52:39.834472+0000","last_peered":"2024-04-25T08:52:39.834472+0000","last_clean":"2024-04-25T08:52:39.834472+0000","last_became_active":"2024-04-25T08:52:31.266697+0000","last_became_peered":"2024-04-25T08:52:31.266697+0000","last_unstale":"2024-04-25T08:52:39.834472+0000","last_undegraded":"2024-04-25T08:52:39.834472+0000","last_fullsized":"2024-04-25T08:52:39.834472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833201+0000","last_change":"2024-04-25T08:52:31.234570+0000","last_active":"2024-04-25T08:52:39.833201+0000","last_peered":"2024-04-25T08:52:39.833201+0000","last_clean":"2024-04-25T08:52:39.833201+0000","last_became_active":"2024-04-25T08:52:31.234049+0000","last_became_peered":"2024-04-25T08:52:31.234049+0000","last_unstale":"2024-04-25T08:52:39.833201+0000","last_undegraded":"2024-04-25T08:52:39.833201+0000","last_fullsized":"2024-04-25T08:52:39.833201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303100+0000","last_change":"2024-04-25T08:52:31.271083+0000","last_active":"2024-04-25T08:52:38.303100+0000","last_peered":"2024-04-25T08:52:38.303100+0000","last_clean":"2024-04-25T08:52:38.303100+0000","last_became_active":"2024-04-25T08:52:31.268755+0000","last_became_peered":"2024-04-25T08:52:31.268755+0000","last_unstale":"2024-04-25T08:52:38.303100+0000","last_undegraded":"2024-04-25T08:52:38.303100+0000","last_fullsized":"2024-04-25T08:52:38.303100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834579+0000","last_change":"2024-04-25T08:52:31.265576+0000","last_active":"2024-04-25T08:52:39.834579+0000","last_peered":"2024-04-25T08:52:39.834579+0000","last_clean":"2024-04-25T08:52:39.834579+0000","last_became_active":"2024-04-25T08:52:31.265273+0000","last_became_peered":"2024-04-25T08:52:31.265273+0000","last_unstale":"2024-04-25T08:52:39.834579+0000","last_undegraded":"2024-04-25T08:52:39.834579+0000","last_fullsized":"2024-04-25T08:52:39.834579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644626+0000","last_change":"2024-04-25T08:52:31.247316+0000","last_active":"2024-04-25T08:52:39.644626+0000","last_peered":"2024-04-25T08:52:39.644626+0000","last_clean":"2024-04-25T08:52:39.644626+0000","last_became_active":"2024-04-25T08:52:31.246547+0000","last_became_peered":"2024-04-25T08:52:31.246547+0000","last_unstale":"2024-04-25T08:52:39.644626+0000","last_undegraded":"2024-04-25T08:52:39.644626+0000","last_fullsized":"2024-04-25T08:52:39.644626+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:09.263949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833082+0000","last_change":"2024-04-25T08:52:31.265084+0000","last_active":"2024-04-25T08:52:39.833082+0000","last_peered":"2024-04-25T08:52:39.833082+0000","last_clean":"2024-04-25T08:52:39.833082+0000","last_became_active":"2024-04-25T08:52:31.264883+0000","last_became_peered":"2024-04-25T08:52:31.264883+0000","last_unstale":"2024-04-25T08:52:39.833082+0000","last_undegraded":"2024-04-25T08:52:39.833082+0000","last_fullsized":"2024-04-25T08:52:39.833082+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:45.331558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645197+0000","last_change":"2024-04-25T08:52:31.280090+0000","last_active":"2024-04-25T08:52:39.645197+0000","last_peered":"2024-04-25T08:52:39.645197+0000","last_clean":"2024-04-25T08:52:39.645197+0000","last_became_active":"2024-04-25T08:52:31.279453+0000","last_became_peered":"2024-04-25T08:52:31.279453+0000","last_unstale":"2024-04-25T08:52:39.645197+0000","last_undegraded":"2024-04-25T08:52:39.645197+0000","last_fullsized":"2024-04-25T08:52:39.645197+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836409+0000","last_change":"2024-04-25T08:52:31.271765+0000","last_active":"2024-04-25T08:52:39.836409+0000","last_peered":"2024-04-25T08:52:39.836409+0000","last_clean":"2024-04-25T08:52:39.836409+0000","last_became_active":"2024-04-25T08:52:31.271438+0000","last_became_peered":"2024-04-25T08:52:31.271438+0000","last_unstale":"2024-04-25T08:52:39.836409+0000","last_undegraded":"2024-04-25T08:52:39.836409+0000","last_fullsized":"2024-04-25T08:52:39.836409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301809+0000","last_change":"2024-04-25T08:52:31.270307+0000","last_active":"2024-04-25T08:52:38.301809+0000","last_peered":"2024-04-25T08:52:38.301809+0000","last_clean":"2024-04-25T08:52:38.301809+0000","last_became_active":"2024-04-25T08:52:31.269440+0000","last_became_peered":"2024-04-25T08:52:31.269440+0000","last_unstale":"2024-04-25T08:52:38.301809+0000","last_undegraded":"2024-04-25T08:52:38.301809+0000","last_fullsized":"2024-04-25T08:52:38.301809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:47.298275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305298+0000","last_change":"2024-04-25T08:52:31.281491+0000","last_active":"2024-04-25T08:52:39.305298+0000","last_peered":"2024-04-25T08:52:39.305298+0000","last_clean":"2024-04-25T08:52:39.305298+0000","last_became_active":"2024-04-25T08:52:31.276333+0000","last_became_peered":"2024-04-25T08:52:31.276333+0000","last_unstale":"2024-04-25T08:52:39.305298+0000","last_undegraded":"2024-04-25T08:52:39.305298+0000","last_fullsized":"2024-04-25T08:52:39.305298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:31.614898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.299491+0000","last_change":"2024-04-25T08:52:31.237856+0000","last_active":"2024-04-25T08:52:38.299491+0000","last_peered":"2024-04-25T08:52:38.299491+0000","last_clean":"2024-04-25T08:52:38.299491+0000","last_became_active":"2024-04-25T08:52:31.237582+0000","last_became_peered":"2024-04-25T08:52:31.237582+0000","last_unstale":"2024-04-25T08:52:38.299491+0000","last_undegraded":"2024-04-25T08:52:38.299491+0000","last_fullsized":"2024-04-25T08:52:38.299491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:05.271538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.646089+0000","last_change":"2024-04-25T08:52:31.283655+0000","last_active":"2024-04-25T08:52:39.646089+0000","last_peered":"2024-04-25T08:52:39.646089+0000","last_clean":"2024-04-25T08:52:39.646089+0000","last_became_active":"2024-04-25T08:52:31.282969+0000","last_became_peered":"2024-04-25T08:52:31.282969+0000","last_unstale":"2024-04-25T08:52:39.646089+0000","last_undegraded":"2024-04-25T08:52:39.646089+0000","last_fullsized":"2024-04-25T08:52:39.646089+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:08.025791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.644525+0000","last_change":"2024-04-25T08:52:31.256907+0000","last_active":"2024-04-25T08:52:39.644525+0000","last_peered":"2024-04-25T08:52:39.644525+0000","last_clean":"2024-04-25T08:52:39.644525+0000","last_became_active":"2024-04-25T08:52:31.255494+0000","last_became_peered":"2024-04-25T08:52:31.255494+0000","last_unstale":"2024-04-25T08:52:39.644525+0000","last_undegraded":"2024-04-25T08:52:39.644525+0000","last_fullsized":"2024-04-25T08:52:39.644525+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834186+0000","last_change":"2024-04-25T08:52:31.270805+0000","last_active":"2024-04-25T08:52:39.834186+0000","last_peered":"2024-04-25T08:52:39.834186+0000","last_clean":"2024-04-25T08:52:39.834186+0000","last_became_active":"2024-04-25T08:52:31.270594+0000","last_became_peered":"2024-04-25T08:52:31.270594+0000","last_unstale":"2024-04-25T08:52:39.834186+0000","last_undegraded":"2024-04-25T08:52:39.834186+0000","last_fullsized":"2024-04-25T08:52:39.834186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.836035+0000","last_change":"2024-04-25T08:52:31.252734+0000","last_active":"2024-04-25T08:52:39.836035+0000","last_peered":"2024-04-25T08:52:39.836035+0000","last_clean":"2024-04-25T08:52:39.836035+0000","last_became_active":"2024-04-25T08:52:31.252476+0000","last_became_peered":"2024-04-25T08:52:31.252476+0000","last_unstale":"2024-04-25T08:52:39.836035+0000","last_undegraded":"2024-04-25T08:52:39.836035+0000","last_fullsized":"2024-04-25T08:52:39.836035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833928+0000","last_change":"2024-04-25T08:52:31.242347+0000","last_active":"2024-04-25T08:52:39.833928+0000","last_peered":"2024-04-25T08:52:39.833928+0000","last_clean":"2024-04-25T08:52:39.833928+0000","last_became_active":"2024-04-25T08:52:31.242082+0000","last_became_peered":"2024-04-25T08:52:31.242082+0000","last_unstale":"2024-04-25T08:52:39.833928+0000","last_undegraded":"2024-04-25T08:52:39.833928+0000","last_fullsized":"2024-04-25T08:52:39.833928+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:45.410016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.301519+0000","last_change":"2024-04-25T08:52:31.258568+0000","last_active":"2024-04-25T08:52:38.301519+0000","last_peered":"2024-04-25T08:52:38.301519+0000","last_clean":"2024-04-25T08:52:38.301519+0000","last_became_active":"2024-04-25T08:52:31.257438+0000","last_became_peered":"2024-04-25T08:52:31.257438+0000","last_unstale":"2024-04-25T08:52:38.301519+0000","last_undegraded":"2024-04-25T08:52:38.301519+0000","last_fullsized":"2024-04-25T08:52:38.301519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:47.022570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.302377+0000","last_change":"2024-04-25T08:52:31.254993+0000","last_active":"2024-04-25T08:52:38.302377+0000","last_peered":"2024-04-25T08:52:38.302377+0000","last_clean":"2024-04-25T08:52:38.302377+0000","last_became_active":"2024-04-25T08:52:31.254072+0000","last_became_peered":"2024-04-25T08:52:31.254072+0000","last_unstale":"2024-04-25T08:52:38.302377+0000","last_undegraded":"2024-04-25T08:52:38.302377+0000","last_fullsized":"2024-04-25T08:52:38.302377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:38.303017+0000","last_change":"2024-04-25T08:52:31.247059+0000","last_active":"2024-04-25T08:52:38.303017+0000","last_peered":"2024-04-25T08:52:38.303017+0000","last_clean":"2024-04-25T08:52:38.303017+0000","last_became_active":"2024-04-25T08:52:31.246364+0000","last_became_peered":"2024-04-25T08:52:31.246364+0000","last_unstale":"2024-04-25T08:52:38.303017+0000","last_undegraded":"2024-04-25T08:52:38.303017+0000","last_fullsized":"2024-04-25T08:52:38.303017+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.308187+0000","last_change":"2024-04-25T08:52:31.289253+0000","last_active":"2024-04-25T08:52:39.308187+0000","last_peered":"2024-04-25T08:52:39.308187+0000","last_clean":"2024-04-25T08:52:39.308187+0000","last_became_active":"2024-04-25T08:52:31.288175+0000","last_became_peered":"2024-04-25T08:52:31.288175+0000","last_unstale":"2024-04-25T08:52:39.308187+0000","last_undegraded":"2024-04-25T08:52:39.308187+0000","last_fullsized":"2024-04-25T08:52:39.308187+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:08.025791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.834144+0000","last_change":"2024-04-25T08:52:31.256539+0000","last_active":"2024-04-25T08:52:39.834144+0000","last_peered":"2024-04-25T08:52:39.834144+0000","last_clean":"2024-04-25T08:52:39.834144+0000","last_became_active":"2024-04-25T08:52:31.255921+0000","last_became_peered":"2024-04-25T08:52:31.255921+0000","last_unstale":"2024-04-25T08:52:39.834144+0000","last_undegraded":"2024-04-25T08:52:39.834144+0000","last_fullsized":"2024-04-25T08:52:39.834144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:05.074954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.306615+0000","last_change":"2024-04-25T08:52:31.277410+0000","last_active":"2024-04-25T08:52:39.306615+0000","last_peered":"2024-04-25T08:52:39.306615+0000","last_clean":"2024-04-25T08:52:39.306615+0000","last_became_active":"2024-04-25T08:52:31.276941+0000","last_became_peered":"2024-04-25T08:52:31.276941+0000","last_unstale":"2024-04-25T08:52:39.306615+0000","last_undegraded":"2024-04-25T08:52:39.306615+0000","last_fullsized":"2024-04-25T08:52:39.306615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645097+0000","last_change":"2024-04-25T08:52:31.256171+0000","last_active":"2024-04-25T08:52:39.645097+0000","last_peered":"2024-04-25T08:52:39.645097+0000","last_clean":"2024-04-25T08:52:39.645097+0000","last_became_active":"2024-04-25T08:52:31.255417+0000","last_became_peered":"2024-04-25T08:52:31.255417+0000","last_unstale":"2024-04-25T08:52:39.645097+0000","last_undegraded":"2024-04-25T08:52:39.645097+0000","last_fullsized":"2024-04-25T08:52:39.645097+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:54.822584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305186+0000","last_change":"2024-04-25T08:52:31.269978+0000","last_active":"2024-04-25T08:52:39.305186+0000","last_peered":"2024-04-25T08:52:39.305186+0000","last_clean":"2024-04-25T08:52:39.305186+0000","last_became_active":"2024-04-25T08:52:31.269108+0000","last_became_peered":"2024-04-25T08:52:31.269108+0000","last_unstale":"2024-04-25T08:52:39.305186+0000","last_undegraded":"2024-04-25T08:52:39.305186+0000","last_fullsized":"2024-04-25T08:52:39.305186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:00.631693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.645458+0000","last_change":"2024-04-25T08:52:31.248569+0000","last_active":"2024-04-25T08:52:39.645458+0000","last_peered":"2024-04-25T08:52:39.645458+0000","last_clean":"2024-04-25T08:52:39.645458+0000","last_became_active":"2024-04-25T08:52:31.247655+0000","last_became_peered":"2024-04-25T08:52:31.247655+0000","last_unstale":"2024-04-25T08:52:39.645458+0000","last_undegraded":"2024-04-25T08:52:39.645458+0000","last_fullsized":"2024-04-25T08:52:39.645458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.601476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833573+0000","last_change":"2024-04-25T08:52:31.252449+0000","last_active":"2024-04-25T08:52:39.833573+0000","last_peered":"2024-04-25T08:52:39.833573+0000","last_clean":"2024-04-25T08:52:39.833573+0000","last_became_active":"2024-04-25T08:52:31.252212+0000","last_became_peered":"2024-04-25T08:52:31.252212+0000","last_unstale":"2024-04-25T08:52:39.833573+0000","last_undegraded":"2024-04-25T08:52:39.833573+0000","last_fullsized":"2024-04-25T08:52:39.833573+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:29.720623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.833700+0000","last_change":"2024-04-25T08:52:31.254735+0000","last_active":"2024-04-25T08:52:39.833700+0000","last_peered":"2024-04-25T08:52:39.833700+0000","last_clean":"2024-04-25T08:52:39.833700+0000","last_became_active":"2024-04-25T08:52:31.254523+0000","last_became_peered":"2024-04-25T08:52:31.254523+0000","last_unstale":"2024-04-25T08:52:39.833700+0000","last_undegraded":"2024-04-25T08:52:39.833700+0000","last_fullsized":"2024-04-25T08:52:39.833700+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:02.824606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-25T08:52:39.305895+0000","last_change":"2024-04-25T08:52:31.288927+0000","last_active":"2024-04-25T08:52:39.305895+0000","last_peered":"2024-04-25T08:52:39.305895+0000","last_clean":"2024-04-25T08:52:39.305895+0000","last_became_active":"2024-04-25T08:52:31.287790+0000","last_became_peered":"2024-04-25T08:52:31.287790+0000","last_unstale":"2024-04-25T08:52:39.305895+0000","last_undegraded":"2024-04-25T08:52:39.305895+0000","last_fullsized":"2024-04-25T08:52:39.305895+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:30.181522+0000","last_clean_scrub_stamp":"2024-04-25T08:52:30.181522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:52.007683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27888,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343952,"statfs":{"total":96636764160,"available":96608206848,"internally_reserved":0,"allocated":888832,"data_stored":709746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":122638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":294912,"data_stored":118886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":303104,"data_stored":118922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27316,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344524,"statfs":{"total":96636764160,"available":96608792576,"internally_reserved":0,"allocated":303104,"data_stored":122638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":118796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":299008,"data_stored":118956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574860,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27896,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343944,"statfs":{"total":96636764160,"available":96608198656,"internally_reserved":0,"allocated":897024,"data_stored":709802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:52:58.431 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T08:52:58.431 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T08:52:58.431 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T08:52:58.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:52:58.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff95dd24640 1 Processor -- start 2024-04-25T08:52:58.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff95dd24640 1 -- start start 2024-04-25T08:52:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff95dd24640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff95806cec0 0x7ff95806bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff95dd24640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff9580cd490 0x7ff9580cf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff95dd24640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff95809a980 con 0x7ff95815d9e0 2024-04-25T08:52:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff95dd24640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9580d1400 con 0x7ff9580cd490 2024-04-25T08:52:58.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff95dd24640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff95809a730 con 0x7ff95806cec0 2024-04-25T08:52:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff957fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff9580cd490 0x7ff9580cf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff9577fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff95815d9e0 0x7ff95806c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59930/0 (socket says 172.21.15.37:59930) 2024-04-25T08:52:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff9577fe640 1 -- 172.21.15.37:0/538924331 learned_addr learned my addr 172.21.15.37:0/538924331 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff956ffd640 1 --2- 172.21.15.37:0/538924331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff95806cec0 0x7ff95806bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff957fff640 1 -- 172.21.15.37:0/538924331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff95806cec0 msgr2=0x7ff95806bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff957fff640 1 --2- 172.21.15.37:0/538924331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff95806cec0 0x7ff95806bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff957fff640 1 -- 172.21.15.37:0/538924331 >> v1:172.21.15.37:6789/0 conn(0x7ff95815d9e0 legacy=0x7ff95806c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.596+0000 7ff957fff640 1 -- 172.21.15.37:0/538924331 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95809ac60 con 0x7ff9580cd490 2024-04-25T08:52:58.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff957fff640 1 --2- 172.21.15.37:0/538924331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9580cd490 0x7ff9580cf7d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff9400097c0 tx=0x7ff94001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e851c2f9001911 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:58.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff9567fc640 1 -- 172.21.15.37:0/538924331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400b0070 con 0x7ff9580cd490 2024-04-25T08:52:58.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff9567fc640 1 -- 172.21.15.37:0/538924331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9400ab600 con 0x7ff9580cd490 2024-04-25T08:52:58.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff9567fc640 1 -- 172.21.15.37:0/538924331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400ab9d0 con 0x7ff9580cd490 2024-04-25T08:52:58.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff95dd24640 1 -- 172.21.15.37:0/538924331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9580cd490 msgr2=0x7ff9580cf7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff95dd24640 1 --2- 172.21.15.37:0/538924331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9580cd490 0x7ff9580cf7d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff9400097c0 tx=0x7ff94001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff95dd24640 1 -- 172.21.15.37:0/538924331 shutdown_connections 2024-04-25T08:52:58.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff95dd24640 1 --2- 172.21.15.37:0/538924331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9580cd490 0x7ff9580cf7d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff95dd24640 1 --2- 172.21.15.37:0/538924331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff95806cec0 0x7ff95806bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff95dd24640 1 -- 172.21.15.37:0/538924331 >> 172.21.15.37:0/538924331 conn(0x7ff9580b3080 msgr2=0x7ff9580b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:58.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff95dd24640 1 -- 172.21.15.37:0/538924331 shutdown_connections 2024-04-25T08:52:58.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.600+0000 7ff95dd24640 1 -- 172.21.15.37:0/538924331 wait complete. 2024-04-25T08:52:58.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff95dd24640 1 Processor -- start 2024-04-25T08:52:58.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff95dd24640 1 -- start start 2024-04-25T08:52:58.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff95dd24640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95806cec0 0x7ff9580b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff95dd24640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9580cd490 0x7ff958151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff95dd24640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95815d9e0 0x7ff9580cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff95dd24640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9580d08b0 con 0x7ff95815d9e0 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff95dd24640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9580d0730 con 0x7ff95806cec0 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff95dd24640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9580d0a30 con 0x7ff9580cd490 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff957fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95815d9e0 0x7ff9580cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff957fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95815d9e0 0x7ff9580cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46696/0 (socket says 172.21.15.37:46696) 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff957fff640 1 -- 172.21.15.37:0/1727581980 learned_addr learned my addr 172.21.15.37:0/1727581980 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff957fff640 1 -- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9580cd490 msgr2=0x7ff958151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff957fff640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9580cd490 0x7ff958151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff957fff640 1 -- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95806cec0 msgr2=0x7ff9580b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff9577fe640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95806cec0 0x7ff9580b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff957fff640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95806cec0 0x7ff9580b41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff957fff640 1 -- 172.21.15.37:0/1727581980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9580c9c20 con 0x7ff95815d9e0 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff957fff640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95815d9e0 0x7ff9580cacc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff9400ae040 tx=0x7ff940019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:58.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff94ffff640 1 -- 172.21.15.37:0/1727581980 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400b8070 con 0x7ff95815d9e0 2024-04-25T08:52:58.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff94ffff640 1 -- 172.21.15.37:0/1727581980 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9400b4410 con 0x7ff95815d9e0 2024-04-25T08:52:58.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff95dd24640 1 -- 172.21.15.37:0/1727581980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9580c9f00 con 0x7ff95815d9e0 2024-04-25T08:52:58.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff94ffff640 1 -- 172.21.15.37:0/1727581980 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400b0040 con 0x7ff95815d9e0 2024-04-25T08:52:58.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff95dd24640 1 -- 172.21.15.37:0/1727581980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9580ca4c0 con 0x7ff95815d9e0 2024-04-25T08:52:58.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff95dd24640 1 -- 172.21.15.37:0/1727581980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff95804efa0 con 0x7ff95815d9e0 2024-04-25T08:52:58.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff94ffff640 1 -- 172.21.15.37:0/1727581980 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9400bb070 con 0x7ff95815d9e0 2024-04-25T08:52:58.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff94ffff640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff938066f60 0x7ff938069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff94ffff640 1 -- 172.21.15.37:0/1727581980 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff94011ef60 con 0x7ff95815d9e0 2024-04-25T08:52:58.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.604+0000 7ff9577fe640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff938066f60 0x7ff938069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.608+0000 7ff9577fe640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff938066f60 0x7ff938069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff9500012f0 tx=0x7ff950009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:58.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.608+0000 7ff94ffff640 1 -- 172.21.15.37:0/1727581980 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff940016090 con 0x7ff95815d9e0 2024-04-25T08:52:58.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.792+0000 7ff95dd24640 1 -- 172.21.15.37:0/1727581980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff9580b46f0 con 0x7ff95815d9e0 2024-04-25T08:52:58.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff94ffff640 1 -- 172.21.15.37:0/1727581980 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff9400efcc0 con 0x7ff95815d9e0 2024-04-25T08:52:58.788 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:58.788 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:52:58.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 -- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff938066f60 msgr2=0x7ff938069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff938066f60 0x7ff938069420 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff9500012f0 tx=0x7ff950009040 comp rx=0 tx=0).stop 2024-04-25T08:52:58.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 -- 172.21.15.37:0/1727581980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95815d9e0 msgr2=0x7ff9580cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95815d9e0 0x7ff9580cacc0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff9400ae040 tx=0x7ff940019240 comp rx=0 tx=0).stop 2024-04-25T08:52:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 -- 172.21.15.37:0/1727581980 shutdown_connections 2024-04-25T08:52:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff938066f60 0x7ff938069420 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff95815d9e0 0x7ff9580cacc0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9580cd490 0x7ff958151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 --2- 172.21.15.37:0/1727581980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff95806cec0 0x7ff9580b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 -- 172.21.15.37:0/1727581980 >> 172.21.15.37:0/1727581980 conn(0x7ff9580b3080 msgr2=0x7ff95806f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 -- 172.21.15.37:0/1727581980 shutdown_connections 2024-04-25T08:52:58.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.796+0000 7ff95dd24640 1 -- 172.21.15.37:0/1727581980 wait complete. 2024-04-25T08:52:58.806 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T08:52:58.807 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:52:58.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f199e4ee640 1 Processor -- start 2024-04-25T08:52:58.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f199e4ee640 1 -- start start 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f199e4ee640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1998144ca0 0x7f199806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f199e4ee640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19980ccc70 con 0x7f19980ce510 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f199e4ee640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f199806dc40 con 0x7f199806ec60 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f199e4ee640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19980cc2b0 con 0x7f1998144ca0 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f1997fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f19980ce510 0x7f19980ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:59944/0 (socket says 172.21.15.37:59944) 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f1997fff640 1 -- 172.21.15.37:0/1503284439 learned_addr learned my addr 172.21.15.37:0/1503284439 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f198ffff640 1 --2- 172.21.15.37:0/1503284439 >> v2:172.21.15.203:3300/0 conn(0x7f1998144ca0 0x7f199806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142948819 0 0) 0x7f19980ccc70 con 0x7f19980ce510 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1984003530 con 0x7f19980ce510 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3389273016 0 0) 0x7f1984003530 con 0x7f19980ce510 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 >> v2:172.21.15.203:3300/0 conn(0x7f1998144ca0 msgr2=0x7f199806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:58.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 --2- 172.21.15.37:0/1503284439 >> v2:172.21.15.203:3300/0 conn(0x7f1998144ca0 0x7f199806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 >> v1:172.21.15.203:6789/0 conn(0x7f199806ec60 legacy=0x7f1998142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199806df20 con 0x7f19980ce510 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1988001680 con 0x7f19980ce510 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 >> v1:172.21.15.37:6789/0 conn(0x7f19980ce510 legacy=0x7f19980ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1984003a30 0x7f1984003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1984004810 0x7f1984004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19840051d0 0x7f198400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19980cc2b0 con 0x7f1984003a30 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198400c0e0 con 0x7f19840051d0 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198400c2e0 con 0x7f1984004810 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f1997fff640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1984003a30 0x7f1984003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f198ffff640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1984004810 0x7f1984004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f199ca8b640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19840051d0 0x7f198400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f1997fff640 1 -- 172.21.15.37:0/1503284439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1984004810 msgr2=0x7f1984004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:58.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f1997fff640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1984004810 0x7f1984004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f1997fff640 1 -- 172.21.15.37:0/1503284439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19840051d0 msgr2=0x7f198400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:58.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f1997fff640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19840051d0 0x7f198400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f1997fff640 1 -- 172.21.15.37:0/1503284439 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1984003530 con 0x7f1984003a30 2024-04-25T08:52:58.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f199ca8b640 1 -- 172.21.15.37:0/1503284439 reap_dead start 2024-04-25T08:52:58.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.932+0000 7f198ffff640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1984004810 0x7f1984004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:58.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f1997fff640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1984003a30 0x7f1984003e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f198800c290 tx=0x7f198801f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae5f755940cfbb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:58.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988007d40 con 0x7f1984003a30 2024-04-25T08:52:58.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1988005b00 con 0x7f1984003a30 2024-04-25T08:52:58.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f19977fe640 1 -- 172.21.15.37:0/1503284439 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988007070 con 0x7f1984003a30 2024-04-25T08:52:58.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f199e4ee640 1 -- 172.21.15.37:0/1503284439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1984003a30 msgr2=0x7f1984003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f199e4ee640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1984003a30 0x7f1984003e50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f198800c290 tx=0x7f198801f450 comp rx=0 tx=0).stop 2024-04-25T08:52:58.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f199e4ee640 1 -- 172.21.15.37:0/1503284439 shutdown_connections 2024-04-25T08:52:58.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f199e4ee640 1 --2- 172.21.15.37:0/1503284439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1984003a30 0x7f1984003e50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f199e4ee640 1 -- 172.21.15.37:0/1503284439 >> 172.21.15.37:0/1503284439 conn(0x7f19980b11a0 msgr2=0x7f19980b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:58.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f199e4ee640 1 -- 172.21.15.37:0/1503284439 shutdown_connections 2024-04-25T08:52:58.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f199e4ee640 1 -- 172.21.15.37:0/1503284439 wait complete. 2024-04-25T08:52:58.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f199e4ee640 1 Processor -- start 2024-04-25T08:52:58.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.936+0000 7f199e4ee640 1 -- start start 2024-04-25T08:52:58.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199e4ee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1984003a30 0x7f19980c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199e4ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19981e7790 0x7f19980c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199e4ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19981e7c10 0x7f19980c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199e4ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19981451f0 con 0x7f19981e7c10 2024-04-25T08:52:58.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199e4ee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199806e290 con 0x7f1984003a30 2024-04-25T08:52:58.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199e4ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1998145370 con 0x7f19981e7790 2024-04-25T08:52:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199ca8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19981e7c10 0x7f19980c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199ca8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19981e7c10 0x7f19980c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46714/0 (socket says 172.21.15.37:46714) 2024-04-25T08:52:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199ca8b640 1 -- 172.21.15.37:0/2797957015 learned_addr learned my addr 172.21.15.37:0/2797957015 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f1997fff640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1984003a30 0x7f19980c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f198ffff640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19981e7790 0x7f19980c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199ca8b640 1 -- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19981e7790 msgr2=0x7f19980c7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199ca8b640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19981e7790 0x7f19980c7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199ca8b640 1 -- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1984003a30 msgr2=0x7f19980c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199ca8b640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1984003a30 0x7f19980c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199ca8b640 1 -- 172.21.15.37:0/2797957015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19980c8990 con 0x7f19981e7c10 2024-04-25T08:52:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199ca8b640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19981e7c10 0x7f19980c8450 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19980cd600 tx=0x7f199000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f19957fa640 1 -- 172.21.15.37:0/2797957015 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990019070 con 0x7f19981e7c10 2024-04-25T08:52:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f19957fa640 1 -- 172.21.15.37:0/2797957015 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1990006b60 con 0x7f19981e7c10 2024-04-25T08:52:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199e4ee640 1 -- 172.21.15.37:0/2797957015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19980c8c00 con 0x7f19981e7c10 2024-04-25T08:52:58.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f19957fa640 1 -- 172.21.15.37:0/2797957015 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990003980 con 0x7f19981e7c10 2024-04-25T08:52:58.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199e4ee640 1 -- 172.21.15.37:0/2797957015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19980c91c0 con 0x7f19981e7c10 2024-04-25T08:52:58.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f199e4ee640 1 -- 172.21.15.37:0/2797957015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199804efa0 con 0x7f19981e7c10 2024-04-25T08:52:58.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f19957fa640 1 -- 172.21.15.37:0/2797957015 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f199001c070 con 0x7f19981e7c10 2024-04-25T08:52:58.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f19957fa640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1978066f60 0x7f1978069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:58.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.940+0000 7f19957fa640 1 -- 172.21.15.37:0/2797957015 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f19900848d0 con 0x7f19981e7c10 2024-04-25T08:52:58.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.944+0000 7f1997fff640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1978066f60 0x7f1978069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:58.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.944+0000 7f1997fff640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1978066f60 0x7f1978069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19880ae000 tx=0x7f1988007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:58.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:58.944+0000 7f19957fa640 1 -- 172.21.15.37:0/2797957015 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1990054f50 con 0x7f19981e7c10 2024-04-25T08:52:59.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.100+0000 7f199e4ee640 1 -- 172.21.15.37:0/2797957015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19980b1cc0 con 0x7f19981e7c10 2024-04-25T08:52:59.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.104+0000 7f19957fa640 1 -- 172.21.15.37:0/2797957015 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f1990058ff0 con 0x7f19981e7c10 2024-04-25T08:52:59.098 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:52:59.098 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:38.282648+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:52:59.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 -- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1978066f60 msgr2=0x7f1978069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1978066f60 0x7f1978069420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f19880ae000 tx=0x7f1988007040 comp rx=0 tx=0).stop 2024-04-25T08:52:59.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 -- 172.21.15.37:0/2797957015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19981e7c10 msgr2=0x7f19980c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19981e7c10 0x7f19980c8450 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f19980cd600 tx=0x7f199000ec10 comp rx=0 tx=0).stop 2024-04-25T08:52:59.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 -- 172.21.15.37:0/2797957015 shutdown_connections 2024-04-25T08:52:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1978066f60 0x7f1978069420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19981e7c10 0x7f19980c8450 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19981e7790 0x7f19980c7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 --2- 172.21.15.37:0/2797957015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1984003a30 0x7f19980c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 -- 172.21.15.37:0/2797957015 >> 172.21.15.37:0/2797957015 conn(0x7f19980b11a0 msgr2=0x7f19980c7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 -- 172.21.15.37:0/2797957015 shutdown_connections 2024-04-25T08:52:59.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.108+0000 7f199e4ee640 1 -- 172.21.15.37:0/2797957015 wait complete. 2024-04-25T08:52:59.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:52:59.116 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:52:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f82688e3640 1 Processor -- start 2024-04-25T08:52:59.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f82688e3640 1 -- start start 2024-04-25T08:52:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f82688e3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f826415b6b0 0x7f82640ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f82688e3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826415ff50 con 0x7f8264073010 2024-04-25T08:52:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f82688e3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82640cbd20 con 0x7f82640cf590 2024-04-25T08:52:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f82688e3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826415ba90 con 0x7f826415b6b0 2024-04-25T08:52:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f826259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f826415b6b0 0x7f82640ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f826259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f826415b6b0 0x7f82640ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46728/0 (socket says 172.21.15.37:46728) 2024-04-25T08:52:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f826259c640 1 -- 172.21.15.37:0/2910409148 learned_addr learned my addr 172.21.15.37:0/2910409148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f826259c640 1 -- 172.21.15.37:0/2910409148 >> v1:172.21.15.203:6789/0 conn(0x7f82640cf590 legacy=0x7f826415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f826259c640 1 -- 172.21.15.37:0/2910409148 >> v1:172.21.15.37:6789/0 conn(0x7f8264073010 legacy=0x7f826415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.248+0000 7f826259c640 1 -- 172.21.15.37:0/2910409148 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640cc000 con 0x7f826415b6b0 2024-04-25T08:52:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f826259c640 1 --2- 172.21.15.37:0/2910409148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826415b6b0 0x7f82640ca3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8254009870 tx=0x7f825401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35953de3dbd04a2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f826159a640 1 -- 172.21.15.37:0/2910409148 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82540aa650 con 0x7f826415b6b0 2024-04-25T08:52:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f826159a640 1 -- 172.21.15.37:0/2910409148 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82540af070 con 0x7f826415b6b0 2024-04-25T08:52:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f826159a640 1 -- 172.21.15.37:0/2910409148 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82540aaa10 con 0x7f826415b6b0 2024-04-25T08:52:59.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 -- 172.21.15.37:0/2910409148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826415b6b0 msgr2=0x7f82640ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 --2- 172.21.15.37:0/2910409148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826415b6b0 0x7f82640ca3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8254009870 tx=0x7f825401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 -- 172.21.15.37:0/2910409148 shutdown_connections 2024-04-25T08:52:59.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 --2- 172.21.15.37:0/2910409148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826415b6b0 0x7f82640ca3c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 -- 172.21.15.37:0/2910409148 >> 172.21.15.37:0/2910409148 conn(0x7f826406ae60 msgr2=0x7f826406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:59.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 -- 172.21.15.37:0/2910409148 shutdown_connections 2024-04-25T08:52:59.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 -- 172.21.15.37:0/2910409148 wait complete. 2024-04-25T08:52:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 Processor -- start 2024-04-25T08:52:59.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 -- start start 2024-04-25T08:52:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8264073010 0x7f82640b31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82640cf590 0x7f82640c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826415b6b0 0x7f82640c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640cfe30 con 0x7f82640cf590 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82640cba30 con 0x7f826415b6b0 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f82688e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82640ca900 con 0x7f8264073010 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f8261d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82640cf590 0x7f82640c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f8261d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82640cf590 0x7f82640c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46736/0 (socket says 172.21.15.37:46736) 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f8261d9b640 1 -- 172.21.15.37:0/3612612103 learned_addr learned my addr 172.21.15.37:0/3612612103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f826259c640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8264073010 0x7f82640b31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.252+0000 7f8262d9d640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826415b6b0 0x7f82640c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8261d9b640 1 -- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8264073010 msgr2=0x7f82640b31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8261d9b640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8264073010 0x7f82640b31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8261d9b640 1 -- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826415b6b0 msgr2=0x7f82640c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8261d9b640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826415b6b0 0x7f82640c3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8261d9b640 1 -- 172.21.15.37:0/3612612103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640c2f40 con 0x7f82640cf590 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f826259c640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8264073010 0x7f82640b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8261d9b640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82640cf590 0x7f82640c9220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f825800c490 tx=0x7f8258001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8252ffd640 1 -- 172.21.15.37:0/3612612103 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258017070 con 0x7f82640cf590 2024-04-25T08:52:59.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8252ffd640 1 -- 172.21.15.37:0/3612612103 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8258003600 con 0x7f82640cf590 2024-04-25T08:52:59.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f82688e3640 1 -- 172.21.15.37:0/3612612103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82640c31e0 con 0x7f82640cf590 2024-04-25T08:52:59.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8252ffd640 1 -- 172.21.15.37:0/3612612103 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82580039f0 con 0x7f82640cf590 2024-04-25T08:52:59.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f82688e3640 1 -- 172.21.15.37:0/3612612103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82640c37a0 con 0x7f82640cf590 2024-04-25T08:52:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f82688e3640 1 -- 172.21.15.37:0/3612612103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826404efa0 con 0x7f82640cf590 2024-04-25T08:52:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8252ffd640 1 -- 172.21.15.37:0/3612612103 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f825801a070 con 0x7f82640cf590 2024-04-25T08:52:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8252ffd640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 0x7f8244069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f8252ffd640 1 -- 172.21.15.37:0/3612612103 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8258084700 con 0x7f82640cf590 2024-04-25T08:52:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f826259c640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 0x7f8244069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.256+0000 7f826259c640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 0x7f8244069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82540ae000 tx=0x7f8254008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.260+0000 7f8252ffd640 1 -- 172.21.15.37:0/3612612103 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8258055460 con 0x7f82640cf590 2024-04-25T08:52:59.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.428+0000 7f82688e3640 1 -- 172.21.15.37:0/3612612103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82640b3d20 con 0x7f82640cf590 2024-04-25T08:52:59.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.428+0000 7f8252ffd640 1 -- 172.21.15.37:0/3612612103 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8258054d60 con 0x7f82640cf590 2024-04-25T08:52:59.422 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:52:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 -- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 msgr2=0x7f8244069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 0x7f8244069420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f82540ae000 tx=0x7f8254008040 comp rx=0 tx=0).stop 2024-04-25T08:52:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 -- 172.21.15.37:0/3612612103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82640cf590 msgr2=0x7f82640c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82640cf590 0x7f82640c9220 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f825800c490 tx=0x7f8258001e80 comp rx=0 tx=0).stop 2024-04-25T08:52:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 -- 172.21.15.37:0/3612612103 shutdown_connections 2024-04-25T08:52:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 0x7f8244069420 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f826415b6b0 0x7f82640c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82640cf590 0x7f82640c9220 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 --2- 172.21.15.37:0/3612612103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8264073010 0x7f82640b31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 -- 172.21.15.37:0/3612612103 >> 172.21.15.37:0/3612612103 conn(0x7f826406ae60 msgr2=0x7f82640af240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:59.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 -- 172.21.15.37:0/3612612103 shutdown_connections 2024-04-25T08:52:59.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.432+0000 7f82688e3640 1 -- 172.21.15.37:0/3612612103 wait complete. 2024-04-25T08:52:59.440 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:52:59.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a15f2a640 1 Processor -- start 2024-04-25T08:52:59.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a15f2a640 1 -- start start 2024-04-25T08:52:59.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a15f2a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3a1015b6b0 0x7f3a100ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a15f2a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a100cf590 0x7f3a100cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a15f2a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a100cbd20 con 0x7f3a1015f130 2024-04-25T08:52:59.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a15f2a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1015ba90 con 0x7f3a1015b6b0 2024-04-25T08:52:59.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a15f2a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100cff30 con 0x7f3a100cf590 2024-04-25T08:52:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a0f7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3a1015b6b0 0x7f3a100ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a07fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a100cf590 0x7f3a100cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a0ffff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f3a1015f130 0x7f3a1015f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:55330/0 (socket says 172.21.15.37:55330) 2024-04-25T08:52:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a0f7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3a1015b6b0 0x7f3a100ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:43992/0 (socket says 172.21.15.37:43992) 2024-04-25T08:52:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a0ffff640 1 -- 172.21.15.37:0/3983570468 learned_addr learned my addr 172.21.15.37:0/3983570468 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a07fff640 1 -- 172.21.15.37:0/3983570468 >> v2:172.21.15.203:3300/0 conn(0x7f3a1015b6b0 msgr2=0x7f3a100ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a07fff640 1 --2- 172.21.15.37:0/3983570468 >> v2:172.21.15.203:3300/0 conn(0x7f3a1015b6b0 0x7f3a100ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a07fff640 1 -- 172.21.15.37:0/3983570468 >> v1:172.21.15.203:6789/0 conn(0x7f3a1015f130 legacy=0x7f3a1015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.576+0000 7f3a07fff640 1 -- 172.21.15.37:0/3983570468 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100cc000 con 0x7f3a100cf590 2024-04-25T08:52:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a07fff640 1 --2- 172.21.15.37:0/3983570468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a100cf590 0x7f3a100cf9f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f39f80097c0 tx=0x7f39f801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac7adaf8b9933371 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a0effd640 1 -- 172.21.15.37:0/3983570468 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80a7070 con 0x7f3a100cf590 2024-04-25T08:52:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a0effd640 1 -- 172.21.15.37:0/3983570468 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f80b0070 con 0x7f3a100cf590 2024-04-25T08:52:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a0effd640 1 -- 172.21.15.37:0/3983570468 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f80ab890 con 0x7f3a100cf590 2024-04-25T08:52:59.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 -- 172.21.15.37:0/3983570468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a100cf590 msgr2=0x7f3a100cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 --2- 172.21.15.37:0/3983570468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a100cf590 0x7f3a100cf9f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f39f80097c0 tx=0x7f39f801c470 comp rx=0 tx=0).stop 2024-04-25T08:52:59.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 -- 172.21.15.37:0/3983570468 shutdown_connections 2024-04-25T08:52:59.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 --2- 172.21.15.37:0/3983570468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a100cf590 0x7f3a100cf9f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 --2- 172.21.15.37:0/3983570468 >> v2:172.21.15.203:3300/0 conn(0x7f3a1015b6b0 0x7f3a100ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 -- 172.21.15.37:0/3983570468 >> 172.21.15.37:0/3983570468 conn(0x7f3a1006ae60 msgr2=0x7f3a1006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:59.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 -- 172.21.15.37:0/3983570468 shutdown_connections 2024-04-25T08:52:59.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 -- 172.21.15.37:0/3983570468 wait complete. 2024-04-25T08:52:59.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 Processor -- start 2024-04-25T08:52:59.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 -- start start 2024-04-25T08:52:59.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a100cf590 0x7f3a100ce950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a1015b6b0 0x7f3a10097bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a1015f130 0x7f3a10164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100ca900 con 0x7f3a1015f130 2024-04-25T08:52:59.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100cba30 con 0x7f3a1015b6b0 2024-04-25T08:52:59.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a15f2a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100cadb0 con 0x7f3a100cf590 2024-04-25T08:52:59.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a1015f130 0x7f3a10164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a1015f130 0x7f3a10164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42062/0 (socket says 172.21.15.37:42062) 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a0ffff640 1 -- 172.21.15.37:0/4112252363 learned_addr learned my addr 172.21.15.37:0/4112252363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a0f7fe640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a100cf590 0x7f3a100ce950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a07fff640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a1015b6b0 0x7f3a10097bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a0ffff640 1 -- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a100cf590 msgr2=0x7f3a100ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a0ffff640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a100cf590 0x7f3a100ce950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a0ffff640 1 -- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a1015b6b0 msgr2=0x7f3a10097bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.580+0000 7f3a0ffff640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a1015b6b0 0x7f3a10097bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a0ffff640 1 -- 172.21.15.37:0/4112252363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a100cdb90 con 0x7f3a1015f130 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a0ffff640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a1015f130 0x7f3a10164670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800ca80 tx=0x7f3a0800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a0cff9640 1 -- 172.21.15.37:0/4112252363 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08017070 con 0x7f3a1015f130 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a0cff9640 1 -- 172.21.15.37:0/4112252363 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a0800ac30 con 0x7f3a1015f130 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a15f2a640 1 -- 172.21.15.37:0/4112252363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a100cde70 con 0x7f3a1015f130 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a0cff9640 1 -- 172.21.15.37:0/4112252363 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08012400 con 0x7f3a1015f130 2024-04-25T08:52:59.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a15f2a640 1 -- 172.21.15.37:0/4112252363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a100cee90 con 0x7f3a1015f130 2024-04-25T08:52:59.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a15f2a640 1 -- 172.21.15.37:0/4112252363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a1004efa0 con 0x7f3a1015f130 2024-04-25T08:52:59.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a0cff9640 1 -- 172.21.15.37:0/4112252363 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a0801a070 con 0x7f3a1015f130 2024-04-25T08:52:59.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a0cff9640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f39f0066f60 0x7f39f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a0cff9640 1 -- 172.21.15.37:0/4112252363 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3a08084f30 con 0x7f3a1015f130 2024-04-25T08:52:59.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a0f7fe640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f39f0066f60 0x7f39f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.584+0000 7f3a0f7fe640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f39f0066f60 0x7f39f0069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009af0 tx=0x7f3a00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.588+0000 7f3a0cff9640 1 -- 172.21.15.37:0/4112252363 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a080555b0 con 0x7f3a1015f130 2024-04-25T08:52:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.744+0000 7f3a15f2a640 1 -- 172.21.15.37:0/4112252363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a100b2310 con 0x7f3a1015f130 2024-04-25T08:52:59.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.744+0000 7f3a0cff9640 1 -- 172.21.15.37:0/4112252363 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a08059650 con 0x7f3a1015f130 2024-04-25T08:52:59.739 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:52:59.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 -- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f39f0066f60 msgr2=0x7f39f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f39f0066f60 0x7f39f0069420 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009af0 tx=0x7f3a00008040 comp rx=0 tx=0).stop 2024-04-25T08:52:59.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 -- 172.21.15.37:0/4112252363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a1015f130 msgr2=0x7f3a10164670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a1015f130 0x7f3a10164670 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800ca80 tx=0x7f3a0800cf40 comp rx=0 tx=0).stop 2024-04-25T08:52:59.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 -- 172.21.15.37:0/4112252363 shutdown_connections 2024-04-25T08:52:59.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f39f0066f60 0x7f39f0069420 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a1015f130 0x7f3a10164670 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a1015b6b0 0x7f3a10097bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 --2- 172.21.15.37:0/4112252363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a100cf590 0x7f3a100ce950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 -- 172.21.15.37:0/4112252363 >> 172.21.15.37:0/4112252363 conn(0x7f3a1006ae60 msgr2=0x7f3a100ad820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:59.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 -- 172.21.15.37:0/4112252363 shutdown_connections 2024-04-25T08:52:59.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.748+0000 7f3a15f2a640 1 -- 172.21.15.37:0/4112252363 wait complete. 2024-04-25T08:52:59.757 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:52:59.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45b1e1d640 1 Processor -- start 2024-04-25T08:52:59.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45b1e1d640 1 -- start start 2024-04-25T08:52:59.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45b1e1d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac0cf590 0x7f45ac0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45b1e1d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac0cbd20 con 0x7f45ac072000 2024-04-25T08:52:59.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45b1e1d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac0cc000 con 0x7f45ac15b6b0 2024-04-25T08:52:59.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45b1e1d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac15ba90 con 0x7f45ac0cf590 2024-04-25T08:52:59.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45abfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f45ac072000 0x7f45ac072470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44598/0 (socket says 172.21.15.37:44598) 2024-04-25T08:52:59.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45abfff640 1 -- 172.21.15.37:0/2669976138 learned_addr learned my addr 172.21.15.37:0/2669976138 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:52:59.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aaffd640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac0cf590 0x7f45ac0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123637981 0 0) 0x7f45ac0cbd20 con 0x7f45ac072000 2024-04-25T08:52:59.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4594003530 con 0x7f45ac072000 2024-04-25T08:52:59.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 210386274 0 0) 0x7f4594003530 con 0x7f45ac072000 2024-04-25T08:52:59.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac0cf590 msgr2=0x7f45ac0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:59.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac0cf590 0x7f45ac0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 >> v1:172.21.15.203:6789/0 conn(0x7f45ac15b6b0 legacy=0x7f45ac0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac0cc2e0 con 0x7f45ac072000 2024-04-25T08:52:59.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f45a4001680 con 0x7f45ac072000 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 >> v1:172.21.15.37:6789/0 conn(0x7f45ac072000 legacy=0x7f45ac072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac0cf590 0x7f4594003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4594003fe0 0x7f4594004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4594004d50 0x7f45940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac15ba90 con 0x7f4594003fe0 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459400bb30 con 0x7f4594004d50 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459400bd30 con 0x7f45ac0cf590 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aaffd640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4594003fe0 0x7f4594004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45ab7fe640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac0cf590 0x7f4594003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45abfff640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4594004d50 0x7f45940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aaffd640 1 -- 172.21.15.37:0/2669976138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac0cf590 msgr2=0x7f4594003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aaffd640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac0cf590 0x7f4594003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aaffd640 1 -- 172.21.15.37:0/2669976138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4594004d50 msgr2=0x7f45940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:52:59.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aaffd640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4594004d50 0x7f45940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.884+0000 7f45aaffd640 1 -- 172.21.15.37:0/2669976138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4594003530 con 0x7f4594003fe0 2024-04-25T08:52:59.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45aaffd640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4594003fe0 0x7f4594004810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009ac0 tx=0x7f45a001c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba0f22e58742aef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a0009690 con 0x7f4594003fe0 2024-04-25T08:52:59.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a00a6480 con 0x7f4594003fe0 2024-04-25T08:52:59.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45aa7fc640 1 -- 172.21.15.37:0/2669976138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a00a6850 con 0x7f4594003fe0 2024-04-25T08:52:59.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 -- 172.21.15.37:0/2669976138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4594003fe0 msgr2=0x7f4594004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 --2- 172.21.15.37:0/2669976138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4594003fe0 0x7f4594004810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009ac0 tx=0x7f45a001c990 comp rx=0 tx=0).stop 2024-04-25T08:52:59.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45abfff640 1 -- 172.21.15.37:0/2669976138 reap_dead start 2024-04-25T08:52:59.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 -- 172.21.15.37:0/2669976138 shutdown_connections 2024-04-25T08:52:59.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 -- 172.21.15.37:0/2669976138 >> 172.21.15.37:0/2669976138 conn(0x7f45ac06ae60 msgr2=0x7f45ac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:52:59.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 -- 172.21.15.37:0/2669976138 shutdown_connections 2024-04-25T08:52:59.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 -- 172.21.15.37:0/2669976138 wait complete. 2024-04-25T08:52:59.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 Processor -- start 2024-04-25T08:52:59.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 -- start start 2024-04-25T08:52:59.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac098750 0x7f45ac0c58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45ac098b30 0x7f45ac0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac15bc10 0x7f45ac0b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac0cba30 con 0x7f45ac098750 2024-04-25T08:52:59.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac0cc8a0 con 0x7f45ac098b30 2024-04-25T08:52:59.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45b1e1d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac0cadb0 con 0x7f45ac15bc10 2024-04-25T08:52:59.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45ab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac098750 0x7f45ac0c58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45ab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac098750 0x7f45ac0c58a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42086/0 (socket says 172.21.15.37:42086) 2024-04-25T08:52:59.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.888+0000 7f45ab7fe640 1 -- 172.21.15.37:0/3736029991 learned_addr learned my addr 172.21.15.37:0/3736029991 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:52:59.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45abfff640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac15bc10 0x7f45ac0b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45aaffd640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45ac098b30 0x7f45ac0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45ab7fe640 1 -- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac15bc10 msgr2=0x7f45ac0b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45ab7fe640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac15bc10 0x7f45ac0b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45ab7fe640 1 -- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45ac098b30 msgr2=0x7f45ac0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:52:59.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45ab7fe640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45ac098b30 0x7f45ac0b0fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:52:59.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45ab7fe640 1 -- 172.21.15.37:0/3736029991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac15c1d0 con 0x7f45ac098750 2024-04-25T08:52:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45abfff640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac15bc10 0x7f45ac0b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:52:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45ab7fe640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac098750 0x7f45ac0c58a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e8d0 tx=0x7f459c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f4593fff640 1 -- 172.21.15.37:0/3736029991 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c003b90 con 0x7f45ac098750 2024-04-25T08:52:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f4593fff640 1 -- 172.21.15.37:0/3736029991 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459c018070 con 0x7f45ac098750 2024-04-25T08:52:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f4593fff640 1 -- 172.21.15.37:0/3736029991 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c0050d0 con 0x7f45ac098750 2024-04-25T08:52:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45b1e1d640 1 -- 172.21.15.37:0/3736029991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac15c4b0 con 0x7f45ac098750 2024-04-25T08:52:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45b1e1d640 1 -- 172.21.15.37:0/3736029991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac15ca70 con 0x7f45ac098750 2024-04-25T08:52:59.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45b1e1d640 1 -- 172.21.15.37:0/3736029991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ac04efa0 con 0x7f45ac098750 2024-04-25T08:52:59.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f4593fff640 1 -- 172.21.15.37:0/3736029991 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f459c005270 con 0x7f45ac098750 2024-04-25T08:52:59.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f4593fff640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4598066f90 0x7f4598069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:52:59.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f4593fff640 1 -- 172.21.15.37:0/3736029991 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f459c085470 con 0x7f45ac098750 2024-04-25T08:52:59.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45aaffd640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4598066f90 0x7f4598069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:52:59.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.892+0000 7f45aaffd640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4598066f90 0x7f4598069450 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f45a00ad000 tx=0x7f45a00a9040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:52:59.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:52:59.896+0000 7f4593fff640 1 -- 172.21.15.37:0/3736029991 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f459c01a310 con 0x7f45ac098750 2024-04-25T08:53:00.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.052+0000 7f45b1e1d640 1 -- 172.21.15.37:0/3736029991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45ac071900 con 0x7f45ac098750 2024-04-25T08:53:00.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.052+0000 7f4593fff640 1 -- 172.21.15.37:0/3736029991 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f459c056130 con 0x7f45ac098750 2024-04-25T08:53:00.047 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 -- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4598066f90 msgr2=0x7f4598069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4598066f90 0x7f4598069450 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f45a00ad000 tx=0x7f45a00a9040 comp rx=0 tx=0).stop 2024-04-25T08:53:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 -- 172.21.15.37:0/3736029991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac098750 msgr2=0x7f45ac0c58a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac098750 0x7f45ac0c58a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f459c00e8d0 tx=0x7f459c00ed90 comp rx=0 tx=0).stop 2024-04-25T08:53:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 -- 172.21.15.37:0/3736029991 shutdown_connections 2024-04-25T08:53:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4598066f90 0x7f4598069450 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45ac15bc10 0x7f45ac0b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45ac098b30 0x7f45ac0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 --2- 172.21.15.37:0/3736029991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45ac098750 0x7f45ac0c58a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 -- 172.21.15.37:0/3736029991 >> 172.21.15.37:0/3736029991 conn(0x7f45ac06ae60 msgr2=0x7f45ac072430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 -- 172.21.15.37:0/3736029991 shutdown_connections 2024-04-25T08:53:00.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.056+0000 7f45b1e1d640 1 -- 172.21.15.37:0/3736029991 wait complete. 2024-04-25T08:53:00.065 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:00.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303c569640 1 Processor -- start 2024-04-25T08:53:00.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303c569640 1 -- start start 2024-04-25T08:53:00.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303c569640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30341634c0 0x7f30341658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303c569640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30340ca4c0 con 0x7f303415ecb0 2024-04-25T08:53:00.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303c569640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30340cd690 con 0x7f30340cf590 2024-04-25T08:53:00.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303c569640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30340ca780 con 0x7f30341634c0 2024-04-25T08:53:00.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039b04640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f303415ecb0 0x7f30340c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44606/0 (socket says 172.21.15.37:44606) 2024-04-25T08:53:00.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039b04640 1 -- 172.21.15.37:0/79294336 learned_addr learned my addr 172.21.15.37:0/79294336 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3871237761 0 0) 0x7f30340ca4c0 con 0x7f303415ecb0 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303ab06640 1 --2- 172.21.15.37:0/79294336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30341634c0 0x7f30341658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3020003530 con 0x7f303415ecb0 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2921946919 0 0) 0x7f3020003530 con 0x7f303415ecb0 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30341634c0 msgr2=0x7f30341658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 --2- 172.21.15.37:0/79294336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30341634c0 0x7f30341658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 >> v1:172.21.15.203:6789/0 conn(0x7f30340cf590 legacy=0x7f30340cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30340cd970 con 0x7f303415ecb0 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3024001680 con 0x7f303415ecb0 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 >> v1:172.21.15.37:6789/0 conn(0x7f303415ecb0 legacy=0x7f30340c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 --2- 172.21.15.37:0/79294336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3020003a60 0x7f3020003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 --2- 172.21.15.37:0/79294336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3020004810 0x7f3020004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 --2- 172.21.15.37:0/79294336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30341634c0 0x7f30200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302000bad0 con 0x7f3020003a60 2024-04-25T08:53:00.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302000bc80 con 0x7f3020004810 2024-04-25T08:53:00.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302000be80 con 0x7f30341634c0 2024-04-25T08:53:00.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303a305640 1 --2- 172.21.15.37:0/79294336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3020003a60 0x7f3020003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303a305640 1 -- 172.21.15.37:0/79294336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30341634c0 msgr2=0x7f30200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303a305640 1 --2- 172.21.15.37:0/79294336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30341634c0 0x7f30200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303a305640 1 -- 172.21.15.37:0/79294336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3020004810 msgr2=0x7f3020004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303a305640 1 --2- 172.21.15.37:0/79294336 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3020004810 0x7f3020004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.188+0000 7f303a305640 1 -- 172.21.15.37:0/79294336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3020003530 con 0x7f3020003a60 2024-04-25T08:53:00.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303a305640 1 --2- 172.21.15.37:0/79294336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3020003a60 0x7f3020003e80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3030001780 tx=0x7f303001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ffdf5c21445470 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300013a0 con 0x7f3020003a60 2024-04-25T08:53:00.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30300052f0 con 0x7f3020003a60 2024-04-25T08:53:00.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039303640 1 -- 172.21.15.37:0/79294336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030003840 con 0x7f3020003a60 2024-04-25T08:53:00.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 -- 172.21.15.37:0/79294336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3020003a60 msgr2=0x7f3020003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 --2- 172.21.15.37:0/79294336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3020003a60 0x7f3020003e80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3030001780 tx=0x7f303001c420 comp rx=0 tx=0).stop 2024-04-25T08:53:00.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303ab06640 1 -- 172.21.15.37:0/79294336 reap_dead start 2024-04-25T08:53:00.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 -- 172.21.15.37:0/79294336 shutdown_connections 2024-04-25T08:53:00.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 -- 172.21.15.37:0/79294336 >> 172.21.15.37:0/79294336 conn(0x7f303406ae60 msgr2=0x7f303406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 -- 172.21.15.37:0/79294336 shutdown_connections 2024-04-25T08:53:00.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 -- 172.21.15.37:0/79294336 wait complete. 2024-04-25T08:53:00.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 Processor -- start 2024-04-25T08:53:00.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 -- start start 2024-04-25T08:53:00.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3020003a60 0x7f30341f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3034070ec0 0x7f30341f4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30340b2480 0x7f30340b2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340cdce0 con 0x7f3034070ec0 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340cd440 con 0x7f3020003a60 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30341662d0 con 0x7f30340b2480 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039b04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3034070ec0 0x7f30341f4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039b04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3034070ec0 0x7f30341f4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42100/0 (socket says 172.21.15.37:42100) 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039b04640 1 -- 172.21.15.37:0/3271679779 learned_addr learned my addr 172.21.15.37:0/3271679779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303ab06640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30340b2480 0x7f30340b2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303a305640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3020003a60 0x7f30341f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039b04640 1 -- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30340b2480 msgr2=0x7f30340b2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039b04640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30340b2480 0x7f30340b2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039b04640 1 -- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3020003a60 msgr2=0x7f30341f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039b04640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3020003a60 0x7f30341f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039b04640 1 -- 172.21.15.37:0/3271679779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3034071350 con 0x7f3034070ec0 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303ab06640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30340b2480 0x7f30340b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f3039b04640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3034070ec0 0x7f30341f4950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f302400c730 tx=0x7f302400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f302affd640 1 -- 172.21.15.37:0/3271679779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024018070 con 0x7f3034070ec0 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f302affd640 1 -- 172.21.15.37:0/3271679779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30240072d0 con 0x7f3034070ec0 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 -- 172.21.15.37:0/3271679779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3034071630 con 0x7f3034070ec0 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f302affd640 1 -- 172.21.15.37:0/3271679779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240047f0 con 0x7f3034070ec0 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.192+0000 7f303c569640 1 -- 172.21.15.37:0/3271679779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3034071bf0 con 0x7f3034070ec0 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.196+0000 7f303c569640 1 -- 172.21.15.37:0/3271679779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303404efa0 con 0x7f3034070ec0 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.196+0000 7f302affd640 1 -- 172.21.15.37:0/3271679779 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f302401b070 con 0x7f3034070ec0 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.196+0000 7f302affd640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3018066f60 0x7f3018069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.196+0000 7f302affd640 1 -- 172.21.15.37:0/3271679779 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3024084b60 con 0x7f3034070ec0 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.196+0000 7f303a305640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3018066f60 0x7f3018069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.196+0000 7f303a305640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3018066f60 0x7f3018069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3030001750 tx=0x7f30300011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.200+0000 7f302affd640 1 -- 172.21.15.37:0/3271679779 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3024055910 con 0x7f3034070ec0 2024-04-25T08:53:00.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.356+0000 7f303c569640 1 -- 172.21.15.37:0/3271679779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f30340c9b30 con 0x7f3034070ec0 2024-04-25T08:53:00.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.356+0000 7f302affd640 1 -- 172.21.15.37:0/3271679779 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3024055210 con 0x7f3034070ec0 2024-04-25T08:53:00.352 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 -- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3018066f60 msgr2=0x7f3018069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3018066f60 0x7f3018069420 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3030001750 tx=0x7f30300011a0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 -- 172.21.15.37:0/3271679779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3034070ec0 msgr2=0x7f30341f4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3034070ec0 0x7f30341f4950 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f302400c730 tx=0x7f302400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 -- 172.21.15.37:0/3271679779 shutdown_connections 2024-04-25T08:53:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3018066f60 0x7f3018069420 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f30340b2480 0x7f30340b2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3034070ec0 0x7f30341f4950 secure :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f302400c730 tx=0x7f302400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 --2- 172.21.15.37:0/3271679779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3020003a60 0x7f30341f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 -- 172.21.15.37:0/3271679779 >> 172.21.15.37:0/3271679779 conn(0x7f303406ae60 msgr2=0x7f30340cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 -- 172.21.15.37:0/3271679779 shutdown_connections 2024-04-25T08:53:00.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.360+0000 7f303c569640 1 -- 172.21.15.37:0/3271679779 wait complete. 2024-04-25T08:53:00.370 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f498e455640 1 Processor -- start 2024-04-25T08:53:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f498e455640 1 -- start start 2024-04-25T08:53:00.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f498e455640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f498815da10 0x7f4988071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f498e455640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f49880b24a0 0x7f4988071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f498e455640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49880adbd0 con 0x7f4988071e90 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f498e455640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49880ae860 con 0x7f49880b24a0 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f498e455640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498815e210 con 0x7f498815da10 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f498c9f2640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4988071e90 0x7f49881614f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44618/0 (socket says 172.21.15.37:44618) 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f498c9f2640 1 -- 172.21.15.37:0/1811210124 learned_addr learned my addr 172.21.15.37:0/1811210124 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f49877fe640 1 --2- 172.21.15.37:0/1811210124 >> v2:172.21.15.37:3300/0 conn(0x7f49880b24a0 0x7f4988071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f4987fff640 1 --2- 172.21.15.37:0/1811210124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f498815da10 0x7f4988071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f49877fe640 1 -- 172.21.15.37:0/1811210124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f498815da10 msgr2=0x7f4988071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f49877fe640 1 --2- 172.21.15.37:0/1811210124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f498815da10 0x7f4988071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f49877fe640 1 -- 172.21.15.37:0/1811210124 >> v1:172.21.15.37:6789/0 conn(0x7f4988071e90 legacy=0x7f49881614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f49877fe640 1 -- 172.21.15.37:0/1811210124 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498815e4f0 con 0x7f49880b24a0 2024-04-25T08:53:00.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.500+0000 7f4986ffd640 1 -- 172.21.15.37:0/1811210124 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795184831 0 0) 0x7f49880adbd0 con 0x7f4988071e90 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f49877fe640 1 --2- 172.21.15.37:0/1811210124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49880b24a0 0x7f4988071570 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49700097c0 tx=0x7f497001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c0f793dc78486d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f4986ffd640 1 -- 172.21.15.37:0/1811210124 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700b0070 con 0x7f49880b24a0 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f4986ffd640 1 -- 172.21.15.37:0/1811210124 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49700ab600 con 0x7f49880b24a0 2024-04-25T08:53:00.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f4986ffd640 1 -- 172.21.15.37:0/1811210124 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700ab9d0 con 0x7f49880b24a0 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 -- 172.21.15.37:0/1811210124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49880b24a0 msgr2=0x7f4988071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 --2- 172.21.15.37:0/1811210124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49880b24a0 0x7f4988071570 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f49700097c0 tx=0x7f497001c380 comp rx=0 tx=0).stop 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 -- 172.21.15.37:0/1811210124 shutdown_connections 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 --2- 172.21.15.37:0/1811210124 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49880b24a0 0x7f4988071570 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 --2- 172.21.15.37:0/1811210124 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f498815da10 0x7f4988071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 -- 172.21.15.37:0/1811210124 >> 172.21.15.37:0/1811210124 conn(0x7f49880b11a0 msgr2=0x7f49880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 -- 172.21.15.37:0/1811210124 shutdown_connections 2024-04-25T08:53:00.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 -- 172.21.15.37:0/1811210124 wait complete. 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 Processor -- start 2024-04-25T08:53:00.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 -- start start 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4988071e90 0x7f4988152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49880b24a0 0x7f498815a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f498815da10 0x7f4988152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498816aa10 con 0x7f498815da10 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498816a890 con 0x7f49880b24a0 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498816ab90 con 0x7f4988071e90 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498c9f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f498815da10 0x7f4988152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498c9f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f498815da10 0x7f4988152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42128/0 (socket says 172.21.15.37:42128) 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498c9f2640 1 -- 172.21.15.37:0/3285434101 learned_addr learned my addr 172.21.15.37:0/3285434101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f49877fe640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49880b24a0 0x7f498815a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f4987fff640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4988071e90 0x7f4988152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498c9f2640 1 -- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4988071e90 msgr2=0x7f4988152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498c9f2640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4988071e90 0x7f4988152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498c9f2640 1 -- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49880b24a0 msgr2=0x7f498815a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498c9f2640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49880b24a0 0x7f498815a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498c9f2640 1 -- 172.21.15.37:0/3285434101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498815d290 con 0x7f498815da10 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498c9f2640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f498815da10 0x7f4988152ce0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f498000c4b0 tx=0x7f498000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f4984ff9640 1 -- 172.21.15.37:0/3285434101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980018070 con 0x7f498815da10 2024-04-25T08:53:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f4984ff9640 1 -- 172.21.15.37:0/3285434101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49800074e0 con 0x7f498815da10 2024-04-25T08:53:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 -- 172.21.15.37:0/3285434101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4988158f30 con 0x7f498815da10 2024-04-25T08:53:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f4984ff9640 1 -- 172.21.15.37:0/3285434101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980004a30 con 0x7f498815da10 2024-04-25T08:53:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.504+0000 7f498e455640 1 -- 172.21.15.37:0/3285434101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49881594f0 con 0x7f498815da10 2024-04-25T08:53:00.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.508+0000 7f498e455640 1 -- 172.21.15.37:0/3285434101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4988159730 con 0x7f498815da10 2024-04-25T08:53:00.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.508+0000 7f4984ff9640 1 -- 172.21.15.37:0/3285434101 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f498001b070 con 0x7f498815da10 2024-04-25T08:53:00.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.508+0000 7f4984ff9640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4968066f60 0x7f4968069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.508+0000 7f4984ff9640 1 -- 172.21.15.37:0/3285434101 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f49800845c0 con 0x7f498815da10 2024-04-25T08:53:00.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.508+0000 7f4987fff640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4968066f60 0x7f4968069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.508+0000 7f4987fff640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4968066f60 0x7f4968069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4978008e50 tx=0x7f49780083a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.512+0000 7f4984ff9640 1 -- 172.21.15.37:0/3285434101 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4988159730 con 0x7f498815da10 2024-04-25T08:53:00.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 -- 172.21.15.37:0/3285434101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49880b1dc0 con 0x7f498815da10 2024-04-25T08:53:00.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f4984ff9640 1 -- 172.21.15.37:0/3285434101 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f4980055280 con 0x7f498815da10 2024-04-25T08:53:00.693 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:00.693 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2902565757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2902565757},{"type":"v1","addr":"172.21.15.203:6835","nonce":2902565757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/822172299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":822172299},{"type":"v1","addr":"172.21.15.203:6837","nonce":822172299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3681855890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3681855890},{"type":"v1","addr":"172.21.15.37:6833","nonce":3681855890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:30.564153+0000","modified":"2024-04-25T08:52:53.726659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:53:00.693 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 -- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4968066f60 msgr2=0x7f4968069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4968066f60 0x7f4968069420 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4978008e50 tx=0x7f49780083a0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 -- 172.21.15.37:0/3285434101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f498815da10 msgr2=0x7f4988152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f498815da10 0x7f4988152ce0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f498000c4b0 tx=0x7f498000c970 comp rx=0 tx=0).stop 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 -- 172.21.15.37:0/3285434101 shutdown_connections 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4968066f60 0x7f4968069420 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f498815da10 0x7f4988152ce0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49880b24a0 0x7f498815a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 --2- 172.21.15.37:0/3285434101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4988071e90 0x7f4988152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 -- 172.21.15.37:0/3285434101 >> 172.21.15.37:0/3285434101 conn(0x7f49880b11a0 msgr2=0x7f49880b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 -- 172.21.15.37:0/3285434101 shutdown_connections 2024-04-25T08:53:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.700+0000 7f498e455640 1 -- 172.21.15.37:0/3285434101 wait complete. 2024-04-25T08:53:00.711 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:00.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a4d960640 1 Processor -- start 2024-04-25T08:53:00.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a4d960640 1 -- start start 2024-04-25T08:53:00.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a4d960640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a480af280 0x7f5a4815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a4d960640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5a4806c3b0 0x7f5a4806df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a4d960640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a48066c30 con 0x7f5a480b2c30 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a4d960640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a4806daa0 con 0x7f5a4806c3b0 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a4d960640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480af660 con 0x7f5a480af280 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a467fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5a480b2c30 0x7f5a4806bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44630/0 (socket says 172.21.15.37:44630) 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a467fc640 1 -- 172.21.15.37:0/1419967049 learned_addr learned my addr 172.21.15.37:0/1419967049 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a46ffd640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a480af280 0x7f5a4815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003848791 0 0) 0x7f5a48066c30 con 0x7f5a480b2c30 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a30003530 con 0x7f5a480b2c30 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a477fe640 1 --2- 172.21.15.37:0/1419967049 >> v2:172.21.15.203:3300/0 conn(0x7f5a4806c3b0 0x7f5a4806df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4165224074 0 0) 0x7f5a30003530 con 0x7f5a480b2c30 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a480af280 msgr2=0x7f5a4815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a480af280 0x7f5a4815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 >> v2:172.21.15.203:3300/0 conn(0x7f5a4806c3b0 msgr2=0x7f5a4806df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 --2- 172.21.15.37:0/1419967049 >> v2:172.21.15.203:3300/0 conn(0x7f5a4806c3b0 0x7f5a4806df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4815e640 con 0x7f5a480b2c30 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f5a3c001680 con 0x7f5a480b2c30 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a477fe640 1 --2- 172.21.15.37:0/1419967049 >> v2:172.21.15.203:3300/0 conn(0x7f5a4806c3b0 0x7f5a4806df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 >> v1:172.21.15.37:6789/0 conn(0x7f5a480b2c30 legacy=0x7f5a4806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a480af280 0x7f5a30003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 0x7f5a30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a30004d50 0x7f5a300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480af660 con 0x7f5a30003fb0 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3000ba30 con 0x7f5a30004d50 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3000bc30 con 0x7f5a480af280 2024-04-25T08:53:00.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a467fc640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 0x7f5a30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a46ffd640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a480af280 0x7f5a30003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a477fe640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a30004d50 0x7f5a300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a467fc640 1 -- 172.21.15.37:0/1419967049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a480af280 msgr2=0x7f5a30003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a467fc640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a480af280 0x7f5a30003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a467fc640 1 -- 172.21.15.37:0/1419967049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a30004d50 msgr2=0x7f5a300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:00.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a467fc640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a30004d50 0x7f5a300051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.832+0000 7f5a467fc640 1 -- 172.21.15.37:0/1419967049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a30003530 con 0x7f5a30003fb0 2024-04-25T08:53:00.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a467fc640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 0x7f5a30004810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c001060 tx=0x7f5a3c01f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0842ed6ca30a377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c007d80 con 0x7f5a30003fb0 2024-04-25T08:53:00.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c005b00 con 0x7f5a30003fb0 2024-04-25T08:53:00.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a45ffb640 1 -- 172.21.15.37:0/1419967049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c007070 con 0x7f5a30003fb0 2024-04-25T08:53:00.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a4d960640 1 -- 172.21.15.37:0/1419967049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 msgr2=0x7f5a30004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a4d960640 1 --2- 172.21.15.37:0/1419967049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 0x7f5a30004810 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c001060 tx=0x7f5a3c01f410 comp rx=0 tx=0).stop 2024-04-25T08:53:00.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a477fe640 1 -- 172.21.15.37:0/1419967049 reap_dead start 2024-04-25T08:53:00.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a4d960640 1 -- 172.21.15.37:0/1419967049 shutdown_connections 2024-04-25T08:53:00.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a4d960640 1 -- 172.21.15.37:0/1419967049 >> 172.21.15.37:0/1419967049 conn(0x7f5a48071810 msgr2=0x7f5a48071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a4d960640 1 -- 172.21.15.37:0/1419967049 shutdown_connections 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a4d960640 1 -- 172.21.15.37:0/1419967049 wait complete. 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a4d960640 1 Processor -- start 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.836+0000 7f5a4d960640 1 -- start start 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a4d960640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 0x7f5a480c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a4d960640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a480c5a50 0x7f5a481f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a4d960640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a481f3970 0x7f5a481f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a4d960640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4815df10 con 0x7f5a30003fb0 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a4d960640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4815ec00 con 0x7f5a480c5a50 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a4d960640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4815e090 con 0x7f5a481f3970 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a46ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 0x7f5a480c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a46ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 0x7f5a480c5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42148/0 (socket says 172.21.15.37:42148) 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a46ffd640 1 -- 172.21.15.37:0/2693625240 learned_addr learned my addr 172.21.15.37:0/2693625240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:00.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a477fe640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a481f3970 0x7f5a481f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a46ffd640 1 -- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a481f3970 msgr2=0x7f5a481f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a46ffd640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a481f3970 0x7f5a481f3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a46ffd640 1 -- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a480c5a50 msgr2=0x7f5a481f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a467fc640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a480c5a50 0x7f5a481f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a46ffd640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a480c5a50 0x7f5a481f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a46ffd640 1 -- 172.21.15.37:0/2693625240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a48153e70 con 0x7f5a30003fb0 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a46ffd640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 0x7f5a480c5510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5a38010890 tx=0x7f5a38010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a2ffff640 1 -- 172.21.15.37:0/2693625240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800f040 con 0x7f5a30003fb0 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a2ffff640 1 -- 172.21.15.37:0/2693625240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3800eb60 con 0x7f5a30003fb0 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a2ffff640 1 -- 172.21.15.37:0/2693625240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3801c400 con 0x7f5a30003fb0 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a4d960640 1 -- 172.21.15.37:0/2693625240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a48156c30 con 0x7f5a30003fb0 2024-04-25T08:53:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a4d960640 1 -- 172.21.15.37:0/2693625240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a481571f0 con 0x7f5a30003fb0 2024-04-25T08:53:00.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a4d960640 1 -- 172.21.15.37:0/2693625240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4804efa0 con 0x7f5a30003fb0 2024-04-25T08:53:00.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a2ffff640 1 -- 172.21.15.37:0/2693625240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a3801c5a0 con 0x7f5a30003fb0 2024-04-25T08:53:00.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a2ffff640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a34066f90 0x7f5a34069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:00.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a2ffff640 1 -- 172.21.15.37:0/2693625240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5a38085970 con 0x7f5a30003fb0 2024-04-25T08:53:00.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.840+0000 7f5a467fc640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a34066f90 0x7f5a34069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:00.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.844+0000 7f5a467fc640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a34066f90 0x7f5a34069450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0ae000 tx=0x7f5a3c001a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:00.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:00.844+0000 7f5a2ffff640 1 -- 172.21.15.37:0/2693625240 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a38018030 con 0x7f5a30003fb0 2024-04-25T08:53:00.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.004+0000 7f5a4d960640 1 -- 172.21.15.37:0/2693625240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a480727c0 con 0x7f5a30003fb0 2024-04-25T08:53:00.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a2ffff640 1 -- 172.21.15.37:0/2693625240 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f5a38056630 con 0x7f5a30003fb0 2024-04-25T08:53:01.000 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:01.000 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:38.282648+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 -- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a34066f90 msgr2=0x7f5a34069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a34066f90 0x7f5a34069450 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0ae000 tx=0x7f5a3c001a50 comp rx=0 tx=0).stop 2024-04-25T08:53:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 -- 172.21.15.37:0/2693625240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 msgr2=0x7f5a480c5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 0x7f5a480c5510 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5a38010890 tx=0x7f5a38010d50 comp rx=0 tx=0).stop 2024-04-25T08:53:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 -- 172.21.15.37:0/2693625240 shutdown_connections 2024-04-25T08:53:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a34066f90 0x7f5a34069450 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a481f3970 0x7f5a481f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a480c5a50 0x7f5a481f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 --2- 172.21.15.37:0/2693625240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30003fb0 0x7f5a480c5510 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 -- 172.21.15.37:0/2693625240 >> 172.21.15.37:0/2693625240 conn(0x7f5a48071810 msgr2=0x7f5a481544f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:01.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 -- 172.21.15.37:0/2693625240 shutdown_connections 2024-04-25T08:53:01.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.008+0000 7f5a4d960640 1 -- 172.21.15.37:0/2693625240 wait complete. 2024-04-25T08:53:01.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:01.020 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:01.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.144+0000 7f0890436640 1 Processor -- start 2024-04-25T08:53:01.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.144+0000 7f0890436640 1 -- start start 2024-04-25T08:53:01.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.144+0000 7f0890436640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f088815b6b0 0x7f08880ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.144+0000 7f0890436640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0888073010 0x7f08880cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.144+0000 7f0890436640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08880cbd20 con 0x7f08880cf590 2024-04-25T08:53:01.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.144+0000 7f0890436640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088815ba90 con 0x7f088815b6b0 2024-04-25T08:53:01.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.144+0000 7f0890436640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08880cc000 con 0x7f0888073010 2024-04-25T08:53:01.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088e1d2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f088815b6b0 0x7f08880ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088d9d1640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f08880cf590 0x7f088815f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44644/0 (socket says 172.21.15.37:44644) 2024-04-25T08:53:01.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088d9d1640 1 -- 172.21.15.37:0/3404116089 learned_addr learned my addr 172.21.15.37:0/3404116089 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:01.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088e9d3640 1 --2- 172.21.15.37:0/3404116089 >> v2:172.21.15.203:3300/0 conn(0x7f0888073010 0x7f08880cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088e1d2640 1 -- 172.21.15.37:0/3404116089 >> v2:172.21.15.203:3300/0 conn(0x7f0888073010 msgr2=0x7f08880cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:01.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088e1d2640 1 --2- 172.21.15.37:0/3404116089 >> v2:172.21.15.203:3300/0 conn(0x7f0888073010 0x7f08880cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088e1d2640 1 -- 172.21.15.37:0/3404116089 >> v1:172.21.15.37:6789/0 conn(0x7f08880cf590 legacy=0x7f088815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088e1d2640 1 -- 172.21.15.37:0/3404116089 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880cc2e0 con 0x7f088815b6b0 2024-04-25T08:53:01.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088d1d0640 1 -- 172.21.15.37:0/3404116089 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903039393 0 0) 0x7f08880cbd20 con 0x7f08880cf590 2024-04-25T08:53:01.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088e9d3640 1 --2- 172.21.15.37:0/3404116089 >> v2:172.21.15.203:3300/0 conn(0x7f0888073010 0x7f08880cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:01.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088e1d2640 1 --2- 172.21.15.37:0/3404116089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f088815b6b0 0x7f08880ca3c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f088806f000 tx=0x7f088401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1126a9cf3e1297ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:01.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088d1d0640 1 -- 172.21.15.37:0/3404116089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840b0070 con 0x7f088815b6b0 2024-04-25T08:53:01.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088d1d0640 1 -- 172.21.15.37:0/3404116089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08840ab600 con 0x7f088815b6b0 2024-04-25T08:53:01.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f088d1d0640 1 -- 172.21.15.37:0/3404116089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08840ab9d0 con 0x7f088815b6b0 2024-04-25T08:53:01.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f0890436640 1 -- 172.21.15.37:0/3404116089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f088815b6b0 msgr2=0x7f08880ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f0890436640 1 --2- 172.21.15.37:0/3404116089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f088815b6b0 0x7f08880ca3c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f088806f000 tx=0x7f088401c480 comp rx=0 tx=0).stop 2024-04-25T08:53:01.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f0890436640 1 -- 172.21.15.37:0/3404116089 shutdown_connections 2024-04-25T08:53:01.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f0890436640 1 --2- 172.21.15.37:0/3404116089 >> v2:172.21.15.203:3300/0 conn(0x7f0888073010 0x7f08880cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f0890436640 1 --2- 172.21.15.37:0/3404116089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f088815b6b0 0x7f08880ca3c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f0890436640 1 -- 172.21.15.37:0/3404116089 >> 172.21.15.37:0/3404116089 conn(0x7f088806ae60 msgr2=0x7f088806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:01.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f0890436640 1 -- 172.21.15.37:0/3404116089 shutdown_connections 2024-04-25T08:53:01.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.148+0000 7f0890436640 1 -- 172.21.15.37:0/3404116089 wait complete. 2024-04-25T08:53:01.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f0890436640 1 Processor -- start 2024-04-25T08:53:01.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f0890436640 1 -- start start 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f0890436640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0888073010 0x7f0888098ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f0890436640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08880cf590 0x7f08880b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f0890436640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f088815b6b0 0x7f08880b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f0890436640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08880cadb0 con 0x7f088815b6b0 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f0890436640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08880ca900 con 0x7f08880cf590 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f0890436640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08880caf30 con 0x7f0888073010 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088e9d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f088815b6b0 0x7f08880b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088e9d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f088815b6b0 0x7f08880b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42166/0 (socket says 172.21.15.37:42166) 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088e9d3640 1 -- 172.21.15.37:0/1946657122 learned_addr learned my addr 172.21.15.37:0/1946657122 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088e1d2640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0888073010 0x7f0888098ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088d9d1640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08880cf590 0x7f08880b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088e9d3640 1 -- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0888073010 msgr2=0x7f0888098ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088e9d3640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0888073010 0x7f0888098ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088e9d3640 1 -- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08880cf590 msgr2=0x7f08880b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088e9d3640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08880cf590 0x7f08880b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088e9d3640 1 -- 172.21.15.37:0/1946657122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08880737e0 con 0x7f088815b6b0 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088e1d2640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0888073010 0x7f0888098ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088d9d1640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08880cf590 0x7f08880b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f088e9d3640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f088815b6b0 0x7f08880b3c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f088000cef0 tx=0x7f088000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f087effd640 1 -- 172.21.15.37:0/1946657122 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088001a070 con 0x7f088815b6b0 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f087effd640 1 -- 172.21.15.37:0/1946657122 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f088000d860 con 0x7f088815b6b0 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f0890436640 1 -- 172.21.15.37:0/1946657122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088815de80 con 0x7f088815b6b0 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f087effd640 1 -- 172.21.15.37:0/1946657122 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088000dc50 con 0x7f088815b6b0 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f0890436640 1 -- 172.21.15.37:0/1946657122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088815e960 con 0x7f088815b6b0 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f0890436640 1 -- 172.21.15.37:0/1946657122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088804efa0 con 0x7f088815b6b0 2024-04-25T08:53:01.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f087effd640 1 -- 172.21.15.37:0/1946657122 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f088001d070 con 0x7f088815b6b0 2024-04-25T08:53:01.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f087effd640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f086c066f60 0x7f086c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.152+0000 7f087effd640 1 -- 172.21.15.37:0/1946657122 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0880085cb0 con 0x7f088815b6b0 2024-04-25T08:53:01.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.156+0000 7f088e1d2640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f086c066f60 0x7f086c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.156+0000 7f088e1d2640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f086c066f60 0x7f086c069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f08840ad000 tx=0x7f0884008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:01.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.160+0000 7f087effd640 1 -- 172.21.15.37:0/1946657122 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f08800568f0 con 0x7f088815b6b0 2024-04-25T08:53:01.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.312+0000 7f0890436640 1 -- 172.21.15.37:0/1946657122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08880b0960 con 0x7f088815b6b0 2024-04-25T08:53:01.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.312+0000 7f087effd640 1 -- 172.21.15.37:0/1946657122 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08800561f0 con 0x7f088815b6b0 2024-04-25T08:53:01.305 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.312+0000 7f0890436640 1 -- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f086c066f60 msgr2=0x7f086c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.312+0000 7f0890436640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f086c066f60 0x7f086c069420 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f08840ad000 tx=0x7f0884008040 comp rx=0 tx=0).stop 2024-04-25T08:53:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.312+0000 7f0890436640 1 -- 172.21.15.37:0/1946657122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f088815b6b0 msgr2=0x7f08880b3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.312+0000 7f0890436640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f088815b6b0 0x7f08880b3c10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f088000cef0 tx=0x7f088000a710 comp rx=0 tx=0).stop 2024-04-25T08:53:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.312+0000 7f0890436640 1 -- 172.21.15.37:0/1946657122 shutdown_connections 2024-04-25T08:53:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.312+0000 7f0890436640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f086c066f60 0x7f086c069420 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.312+0000 7f0890436640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f088815b6b0 0x7f08880b3c10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.312+0000 7f0890436640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f08880cf590 0x7f08880b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.312+0000 7f0890436640 1 --2- 172.21.15.37:0/1946657122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0888073010 0x7f0888098ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.316+0000 7f0890436640 1 -- 172.21.15.37:0/1946657122 >> 172.21.15.37:0/1946657122 conn(0x7f088806ae60 msgr2=0x7f08880c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.316+0000 7f0890436640 1 -- 172.21.15.37:0/1946657122 shutdown_connections 2024-04-25T08:53:01.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.316+0000 7f0890436640 1 -- 172.21.15.37:0/1946657122 wait complete. 2024-04-25T08:53:01.323 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:01.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00eec15640 1 Processor -- start 2024-04-25T08:53:01.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00eec15640 1 -- start start 2024-04-25T08:53:01.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00eec15640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80cd350 0x7f00e815aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00eec15640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f00e80cf0e0 0x7f00e815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00eec15640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00e80ce320 con 0x7f00e815aac0 2024-04-25T08:53:01.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00eec15640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e80ce0d0 con 0x7f00e80cf0e0 2024-04-25T08:53:01.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00eec15640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e80ce600 con 0x7f00e80cd350 2024-04-25T08:53:01.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00ec9b1640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f00e815aac0 0x7f00e80ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44650/0 (socket says 172.21.15.37:44650) 2024-04-25T08:53:01.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00ec9b1640 1 -- 172.21.15.37:0/1085647837 learned_addr learned my addr 172.21.15.37:0/1085647837 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e7fff640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80cd350 0x7f00e815aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00ed1b2640 1 --2- 172.21.15.37:0/1085647837 >> v2:172.21.15.203:3300/0 conn(0x7f00e80cf0e0 0x7f00e815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2878537577 0 0) 0x7f00e80ce320 con 0x7f00e815aac0 2024-04-25T08:53:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00d0005740 con 0x7f00e815aac0 2024-04-25T08:53:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4198630568 0 0) 0x7f00d0005740 con 0x7f00e815aac0 2024-04-25T08:53:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80cd350 msgr2=0x7f00e815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80cd350 0x7f00e815aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 >> v2:172.21.15.203:3300/0 conn(0x7f00e80cf0e0 msgr2=0x7f00e815b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 --2- 172.21.15.37:0/1085647837 >> v2:172.21.15.203:3300/0 conn(0x7f00e80cf0e0 0x7f00e815b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e80ce8e0 con 0x7f00e815aac0 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f00d8001680 con 0x7f00e815aac0 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 >> v1:172.21.15.37:6789/0 conn(0x7f00e815aac0 legacy=0x7f00e80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80cd350 0x7f00d0005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d0006210 0x7f00d0006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d0006f60 0x7f00d00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e80ce0d0 con 0x7f00d0006f60 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e80ce600 con 0x7f00d0006210 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d000dc60 con 0x7f00e80cd350 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00ed1b2640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d0006f60 0x7f00d00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00ec9b1640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80cd350 0x7f00d0005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e7fff640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d0006210 0x7f00d0006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00ed1b2640 1 -- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80cd350 msgr2=0x7f00d0005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00ed1b2640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80cd350 0x7f00d0005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00ed1b2640 1 -- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d0006210 msgr2=0x7f00d0006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00ed1b2640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d0006210 0x7f00d0006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00ed1b2640 1 -- 172.21.15.37:0/1085647837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0005740 con 0x7f00d0006f60 2024-04-25T08:53:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00ec9b1640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80cd350 0x7f00d0005c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:01.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.452+0000 7f00e7fff640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00d0006210 0x7f00d0006a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:01.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00ed1b2640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d0006f60 0x7f00d00073e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009fc0 tx=0x7f00e001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b31723e29c15f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:01.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e00b2070 con 0x7f00d0006f60 2024-04-25T08:53:01.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e00a5cd0 con 0x7f00d0006f60 2024-04-25T08:53:01.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00e77fe640 1 -- 172.21.15.37:0/1085647837 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0009c10 con 0x7f00d0006f60 2024-04-25T08:53:01.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00eec15640 1 -- 172.21.15.37:0/1085647837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d0006f60 msgr2=0x7f00d00073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00eec15640 1 --2- 172.21.15.37:0/1085647837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00d0006f60 0x7f00d00073e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009fc0 tx=0x7f00e001ccc0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00ed1b2640 1 -- 172.21.15.37:0/1085647837 reap_dead start 2024-04-25T08:53:01.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00eec15640 1 -- 172.21.15.37:0/1085647837 shutdown_connections 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00eec15640 1 -- 172.21.15.37:0/1085647837 >> 172.21.15.37:0/1085647837 conn(0x7f00e8099a70 msgr2=0x7f00e80d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00eec15640 1 -- 172.21.15.37:0/1085647837 shutdown_connections 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00eec15640 1 -- 172.21.15.37:0/1085647837 wait complete. 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00eec15640 1 Processor -- start 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.456+0000 7f00eec15640 1 -- start start 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00eec15640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e8152f00 0x7f00e80b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00eec15640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e81532e0 0x7f00e80ae6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00eec15640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80aec10 0x7f00e81541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00eec15640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e815baa0 con 0x7f00e81532e0 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00eec15640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e815b920 con 0x7f00e8152f00 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00eec15640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e815aea0 con 0x7f00e80aec10 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e81532e0 0x7f00e80ae6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e81532e0 0x7f00e80ae6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42178/0 (socket says 172.21.15.37:42178) 2024-04-25T08:53:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e7fff640 1 -- 172.21.15.37:0/762501386 learned_addr learned my addr 172.21.15.37:0/762501386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00ec9b1640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e8152f00 0x7f00e80b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00ed1b2640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80aec10 0x7f00e81541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e7fff640 1 -- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80aec10 msgr2=0x7f00e81541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e7fff640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80aec10 0x7f00e81541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e7fff640 1 -- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e8152f00 msgr2=0x7f00e80b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e7fff640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e8152f00 0x7f00e80b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e7fff640 1 -- 172.21.15.37:0/762501386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e8153fa0 con 0x7f00e81532e0 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00ed1b2640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80aec10 0x7f00e81541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00ec9b1640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e8152f00 0x7f00e80b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e7fff640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e81532e0 0x7f00e80ae6d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f00dc010900 tx=0x7f00dc010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e57fa640 1 -- 172.21.15.37:0/762501386 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00f040 con 0x7f00e81532e0 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e57fa640 1 -- 172.21.15.37:0/762501386 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc00ec20 con 0x7f00e81532e0 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00eec15640 1 -- 172.21.15.37:0/762501386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e80b31c0 con 0x7f00e81532e0 2024-04-25T08:53:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e57fa640 1 -- 172.21.15.37:0/762501386 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc004020 con 0x7f00e81532e0 2024-04-25T08:53:01.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00eec15640 1 -- 172.21.15.37:0/762501386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e8154da0 con 0x7f00e81532e0 2024-04-25T08:53:01.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00eec15640 1 -- 172.21.15.37:0/762501386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e804efa0 con 0x7f00e81532e0 2024-04-25T08:53:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e57fa640 1 -- 172.21.15.37:0/762501386 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00dc0041c0 con 0x7f00e81532e0 2024-04-25T08:53:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e57fa640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00d4066f90 0x7f00d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00e57fa640 1 -- 172.21.15.37:0/762501386 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f00dc085780 con 0x7f00e81532e0 2024-04-25T08:53:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.460+0000 7f00ec9b1640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00d4066f90 0x7f00d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.464+0000 7f00ec9b1640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00d4066f90 0x7f00d4069450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f00d8001210 tx=0x7f00d8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.464+0000 7f00e57fa640 1 -- 172.21.15.37:0/762501386 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00e8153fa0 con 0x7f00e81532e0 2024-04-25T08:53:01.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.624+0000 7f00eec15640 1 -- 172.21.15.37:0/762501386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00e80ca3e0 con 0x7f00e81532e0 2024-04-25T08:53:01.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.624+0000 7f00e57fa640 1 -- 172.21.15.37:0/762501386 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00dc018030 con 0x7f00e81532e0 2024-04-25T08:53:01.619 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 -- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00d4066f90 msgr2=0x7f00d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00d4066f90 0x7f00d4069450 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f00d8001210 tx=0x7f00d8009040 comp rx=0 tx=0).stop 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 -- 172.21.15.37:0/762501386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e81532e0 msgr2=0x7f00e80ae6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e81532e0 0x7f00e80ae6d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f00dc010900 tx=0x7f00dc010dc0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 -- 172.21.15.37:0/762501386 shutdown_connections 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00d4066f90 0x7f00d4069450 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80aec10 0x7f00e81541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e81532e0 0x7f00e80ae6d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 --2- 172.21.15.37:0/762501386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e8152f00 0x7f00e80b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 -- 172.21.15.37:0/762501386 >> 172.21.15.37:0/762501386 conn(0x7f00e8099a70 msgr2=0x7f00e8155970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 -- 172.21.15.37:0/762501386 shutdown_connections 2024-04-25T08:53:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.628+0000 7f00eec15640 1 -- 172.21.15.37:0/762501386 wait complete. 2024-04-25T08:53:01.638 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:01.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2973831640 1 Processor -- start 2024-04-25T08:53:01.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2973831640 1 -- start start 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2973831640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f296c0cf590 0x7f296c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2973831640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f296c072000 0x7f296c072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2973831640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296c0cbd20 con 0x7f296c15b6b0 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2973831640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296c15ba90 con 0x7f296c0cf590 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2973831640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0cc000 con 0x7f296c072000 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2970dcc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f296c0cf590 0x7f296c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2970dcc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f296c0cf590 0x7f296c0cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42180/0 (socket says 172.21.15.37:42180) 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2970dcc640 1 -- 172.21.15.37:0/840936508 learned_addr learned my addr 172.21.15.37:0/840936508 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2971dce640 1 --2- 172.21.15.37:0/840936508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f296c072000 0x7f296c072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2970dcc640 1 -- 172.21.15.37:0/840936508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f296c072000 msgr2=0x7f296c072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2970dcc640 1 --2- 172.21.15.37:0/840936508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f296c072000 0x7f296c072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2970dcc640 1 -- 172.21.15.37:0/840936508 >> v1:172.21.15.203:6789/0 conn(0x7f296c15b6b0 legacy=0x7f296c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:01.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.768+0000 7f2970dcc640 1 -- 172.21.15.37:0/840936508 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0cc2e0 con 0x7f296c0cf590 2024-04-25T08:53:01.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2970dcc640 1 --2- 172.21.15.37:0/840936508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f296c0cf590 0x7f296c0cfa20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f295c009870 tx=0x7f295c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc8753165eb9a94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:01.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2963fff640 1 -- 172.21.15.37:0/840936508 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c0b1070 con 0x7f296c0cf590 2024-04-25T08:53:01.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2963fff640 1 -- 172.21.15.37:0/840936508 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c0ac630 con 0x7f296c0cf590 2024-04-25T08:53:01.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2963fff640 1 -- 172.21.15.37:0/840936508 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c0a8050 con 0x7f296c0cf590 2024-04-25T08:53:01.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 -- 172.21.15.37:0/840936508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f296c0cf590 msgr2=0x7f296c0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 --2- 172.21.15.37:0/840936508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f296c0cf590 0x7f296c0cfa20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f295c009870 tx=0x7f295c01c520 comp rx=0 tx=0).stop 2024-04-25T08:53:01.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 -- 172.21.15.37:0/840936508 shutdown_connections 2024-04-25T08:53:01.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 --2- 172.21.15.37:0/840936508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f296c072000 0x7f296c072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 --2- 172.21.15.37:0/840936508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f296c0cf590 0x7f296c0cfa20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 -- 172.21.15.37:0/840936508 >> 172.21.15.37:0/840936508 conn(0x7f296c06ae60 msgr2=0x7f296c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:01.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 -- 172.21.15.37:0/840936508 shutdown_connections 2024-04-25T08:53:01.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 -- 172.21.15.37:0/840936508 wait complete. 2024-04-25T08:53:01.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 Processor -- start 2024-04-25T08:53:01.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 -- start start 2024-04-25T08:53:01.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f296c072000 0x7f296c098920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f296c0cf590 0x7f296c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f296c15b6b0 0x7f296c155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0caf30 con 0x7f296c0cf590 2024-04-25T08:53:01.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0cadb0 con 0x7f296c072000 2024-04-25T08:53:01.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2973831640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c1682a0 con 0x7f296c15b6b0 2024-04-25T08:53:01.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2970dcc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f296c0cf590 0x7f296c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2970dcc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f296c0cf590 0x7f296c0ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42186/0 (socket says 172.21.15.37:42186) 2024-04-25T08:53:01.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2970dcc640 1 -- 172.21.15.37:0/1886616285 learned_addr learned my addr 172.21.15.37:0/1886616285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:01.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2971dce640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f296c15b6b0 0x7f296c155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f29715cd640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f296c072000 0x7f296c098920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2970dcc640 1 -- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f296c15b6b0 msgr2=0x7f296c155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2970dcc640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f296c15b6b0 0x7f296c155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2970dcc640 1 -- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f296c072000 msgr2=0x7f296c098920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2970dcc640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f296c072000 0x7f296c098920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.772+0000 7f2970dcc640 1 -- 172.21.15.37:0/1886616285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0ce420 con 0x7f296c0cf590 2024-04-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f2970dcc640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f296c0cf590 0x7f296c0ccf30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f295c01abd0 tx=0x7f295c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f2961ffb640 1 -- 172.21.15.37:0/1886616285 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c0b1040 con 0x7f296c0cf590 2024-04-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f2961ffb640 1 -- 172.21.15.37:0/1886616285 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295c0ba070 con 0x7f296c0cf590 2024-04-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f2973831640 1 -- 172.21.15.37:0/1886616285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c0cd8a0 con 0x7f296c0cf590 2024-04-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f2961ffb640 1 -- 172.21.15.37:0/1886616285 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295c0a8040 con 0x7f296c0cf590 2024-04-25T08:53:01.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f2973831640 1 -- 172.21.15.37:0/1886616285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c0cde60 con 0x7f296c0cf590 2024-04-25T08:53:01.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f2973831640 1 -- 172.21.15.37:0/1886616285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c0ae190 con 0x7f296c0cf590 2024-04-25T08:53:01.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f2961ffb640 1 -- 172.21.15.37:0/1886616285 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f296c0731c0 con 0x7f296c0cf590 2024-04-25T08:53:01.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f2961ffb640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2950066f60 0x7f2950069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:01.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f2961ffb640 1 -- 172.21.15.37:0/1886616285 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f295c11eda0 con 0x7f296c0cf590 2024-04-25T08:53:01.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f29715cd640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2950066f60 0x7f2950069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:01.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.776+0000 7f29715cd640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2950066f60 0x7f2950069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f296800b810 tx=0x7f2968008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:01.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.780+0000 7f2961ffb640 1 -- 172.21.15.37:0/1886616285 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f295c016110 con 0x7f296c0cf590 2024-04-25T08:53:01.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.940+0000 7f2973831640 1 -- 172.21.15.37:0/1886616285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f296c0b3210 con 0x7f296c0cf590 2024-04-25T08:53:01.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.940+0000 7f2961ffb640 1 -- 172.21.15.37:0/1886616285 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f295c0efb00 con 0x7f296c0cf590 2024-04-25T08:53:01.934 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.940+0000 7f2973831640 1 -- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2950066f60 msgr2=0x7f2950069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.940+0000 7f2973831640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2950066f60 0x7f2950069420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f296800b810 tx=0x7f2968008040 comp rx=0 tx=0).stop 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.940+0000 7f2973831640 1 -- 172.21.15.37:0/1886616285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f296c0cf590 msgr2=0x7f296c0ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.940+0000 7f2973831640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f296c0cf590 0x7f296c0ccf30 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f295c01abd0 tx=0x7f295c0192c0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.944+0000 7f2973831640 1 -- 172.21.15.37:0/1886616285 shutdown_connections 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.944+0000 7f2973831640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2950066f60 0x7f2950069420 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.944+0000 7f2973831640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f296c15b6b0 0x7f296c155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.944+0000 7f2973831640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f296c0cf590 0x7f296c0ccf30 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.944+0000 7f2973831640 1 --2- 172.21.15.37:0/1886616285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f296c072000 0x7f296c098920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.944+0000 7f2973831640 1 -- 172.21.15.37:0/1886616285 >> 172.21.15.37:0/1886616285 conn(0x7f296c06ae60 msgr2=0x7f296c0b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.944+0000 7f2973831640 1 -- 172.21.15.37:0/1886616285 shutdown_connections 2024-04-25T08:53:01.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:01.944+0000 7f2973831640 1 -- 172.21.15.37:0/1886616285 wait complete. 2024-04-25T08:53:01.953 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:02.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f1035d16640 1 Processor -- start 2024-04-25T08:53:02.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f1035d16640 1 -- start start 2024-04-25T08:53:02.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f1035d16640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f10300cf590 0x7f10300cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f1035d16640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1030072000 0x7f1030072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f1035d16640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10300cbd20 con 0x7f103015b6b0 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f1035d16640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103015ba90 con 0x7f10300cf590 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f1035d16640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10300cc000 con 0x7f1030072000 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f102effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f10300cf590 0x7f10300cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f102effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f10300cf590 0x7f10300cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42196/0 (socket says 172.21.15.37:42196) 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f102effd640 1 -- 172.21.15.37:0/620129735 learned_addr learned my addr 172.21.15.37:0/620129735 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f102ffff640 1 --2- 172.21.15.37:0/620129735 >> v2:172.21.15.203:3300/0 conn(0x7f1030072000 0x7f1030072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f102effd640 1 -- 172.21.15.37:0/620129735 >> v2:172.21.15.203:3300/0 conn(0x7f1030072000 msgr2=0x7f1030072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f102effd640 1 --2- 172.21.15.37:0/620129735 >> v2:172.21.15.203:3300/0 conn(0x7f1030072000 0x7f1030072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f102effd640 1 -- 172.21.15.37:0/620129735 >> v1:172.21.15.203:6789/0 conn(0x7f103015b6b0 legacy=0x7f10300ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f102effd640 1 -- 172.21.15.37:0/620129735 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10300cc2e0 con 0x7f10300cf590 2024-04-25T08:53:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.092+0000 7f102ffff640 1 --2- 172.21.15.37:0/620129735 >> v2:172.21.15.203:3300/0 conn(0x7f1030072000 0x7f1030072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:02.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f102effd640 1 --2- 172.21.15.37:0/620129735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10300cf590 0x7f10300cfa20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f103006f000 tx=0x7f101801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca933535e2cefb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:02.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f102e7fc640 1 -- 172.21.15.37:0/620129735 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180b1070 con 0x7f10300cf590 2024-04-25T08:53:02.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f102e7fc640 1 -- 172.21.15.37:0/620129735 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10180ac630 con 0x7f10300cf590 2024-04-25T08:53:02.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f102e7fc640 1 -- 172.21.15.37:0/620129735 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180a8050 con 0x7f10300cf590 2024-04-25T08:53:02.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f1035d16640 1 -- 172.21.15.37:0/620129735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10300cf590 msgr2=0x7f10300cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f1035d16640 1 --2- 172.21.15.37:0/620129735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10300cf590 0x7f10300cfa20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f103006f000 tx=0x7f101801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f1035d16640 1 -- 172.21.15.37:0/620129735 shutdown_connections 2024-04-25T08:53:02.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f1035d16640 1 --2- 172.21.15.37:0/620129735 >> v2:172.21.15.203:3300/0 conn(0x7f1030072000 0x7f1030072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f1035d16640 1 --2- 172.21.15.37:0/620129735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10300cf590 0x7f10300cfa20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f1035d16640 1 -- 172.21.15.37:0/620129735 >> 172.21.15.37:0/620129735 conn(0x7f103006ae60 msgr2=0x7f103006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:02.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f1035d16640 1 -- 172.21.15.37:0/620129735 shutdown_connections 2024-04-25T08:53:02.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f1035d16640 1 -- 172.21.15.37:0/620129735 wait complete. 2024-04-25T08:53:02.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.096+0000 7f1035d16640 1 Processor -- start 2024-04-25T08:53:02.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1035d16640 1 -- start start 2024-04-25T08:53:02.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1035d16640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1030072000 0x7f1030157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1035d16640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10300cf590 0x7f10301f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1035d16640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f103015b6b0 0x7f1030164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1035d16640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10300caf30 con 0x7f103015b6b0 2024-04-25T08:53:02.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1035d16640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10300cadb0 con 0x7f10300cf590 2024-04-25T08:53:02.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1035d16640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10301682a0 con 0x7f1030072000 2024-04-25T08:53:02.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f103015b6b0 0x7f1030164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f103015b6b0 0x7f1030164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42208/0 (socket says 172.21.15.37:42208) 2024-04-25T08:53:02.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102ffff640 1 -- 172.21.15.37:0/943537673 learned_addr learned my addr 172.21.15.37:0/943537673 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:02.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102effd640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10300cf590 0x7f10301f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102ffff640 1 -- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1030072000 msgr2=0x7f1030157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102f7fe640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1030072000 0x7f1030157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102ffff640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1030072000 0x7f1030157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102ffff640 1 -- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10300cf590 msgr2=0x7f10301f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102ffff640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10300cf590 0x7f10301f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102ffff640 1 -- 172.21.15.37:0/943537673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10301f7770 con 0x7f103015b6b0 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102effd640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10300cf590 0x7f10301f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102f7fe640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1030072000 0x7f1030157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f102ffff640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f103015b6b0 0x7f1030164670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f102000cef0 tx=0x7f102000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1027fff640 1 -- 172.21.15.37:0/943537673 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102001a070 con 0x7f103015b6b0 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1027fff640 1 -- 172.21.15.37:0/943537673 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102000d860 con 0x7f103015b6b0 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1035d16640 1 -- 172.21.15.37:0/943537673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1030159250 con 0x7f103015b6b0 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1027fff640 1 -- 172.21.15.37:0/943537673 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102000dc50 con 0x7f103015b6b0 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1035d16640 1 -- 172.21.15.37:0/943537673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1030156d30 con 0x7f103015b6b0 2024-04-25T08:53:02.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1035d16640 1 -- 172.21.15.37:0/943537673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1030156840 con 0x7f103015b6b0 2024-04-25T08:53:02.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.100+0000 7f1027fff640 1 -- 172.21.15.37:0/943537673 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f102001d070 con 0x7f103015b6b0 2024-04-25T08:53:02.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.104+0000 7f1027fff640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1010066f60 0x7f1010069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.104+0000 7f1027fff640 1 -- 172.21.15.37:0/943537673 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f1020085ee0 con 0x7f103015b6b0 2024-04-25T08:53:02.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.104+0000 7f102f7fe640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1010066f60 0x7f1010069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.104+0000 7f102f7fe640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1010066f60 0x7f1010069420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f10301557c0 tx=0x7f1028008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:02.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.104+0000 7f1027fff640 1 -- 172.21.15.37:0/943537673 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1020024020 con 0x7f103015b6b0 2024-04-25T08:53:02.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 -- 172.21.15.37:0/943537673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f103004efa0 con 0x7f103015b6b0 2024-04-25T08:53:02.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1027fff640 1 -- 172.21.15.37:0/943537673 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f10301f7770 con 0x7f103015b6b0 2024-04-25T08:53:02.261 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 -- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1010066f60 msgr2=0x7f1010069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1010066f60 0x7f1010069420 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f10301557c0 tx=0x7f1028008040 comp rx=0 tx=0).stop 2024-04-25T08:53:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 -- 172.21.15.37:0/943537673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f103015b6b0 msgr2=0x7f1030164670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f103015b6b0 0x7f1030164670 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f102000cef0 tx=0x7f102000a710 comp rx=0 tx=0).stop 2024-04-25T08:53:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 -- 172.21.15.37:0/943537673 shutdown_connections 2024-04-25T08:53:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1010066f60 0x7f1010069420 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f103015b6b0 0x7f1030164670 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10300cf590 0x7f10301f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 --2- 172.21.15.37:0/943537673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1030072000 0x7f1030157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 -- 172.21.15.37:0/943537673 >> 172.21.15.37:0/943537673 conn(0x7f103006ae60 msgr2=0x7f10300b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.268+0000 7f1035d16640 1 -- 172.21.15.37:0/943537673 shutdown_connections 2024-04-25T08:53:02.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.272+0000 7f1035d16640 1 -- 172.21.15.37:0/943537673 wait complete. 2024-04-25T08:53:02.281 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:02.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5dedbe1640 1 Processor -- start 2024-04-25T08:53:02.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5dedbe1640 1 -- start start 2024-04-25T08:53:02.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5dedbe1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5de815b920 0x7f5de8099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5dedbe1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de80b0440 0x7f5de8099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5dedbe1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de809a950 con 0x7f5de80aeb80 2024-04-25T08:53:02.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5dedbe1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de809ac30 con 0x7f5de815b920 2024-04-25T08:53:02.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5dedbe1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de80ae380 con 0x7f5de80b0440 2024-04-25T08:53:02.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5de77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5de815b920 0x7f5de8099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5de7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5de80aeb80 0x7f5de80aeff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44664/0 (socket says 172.21.15.37:44664) 2024-04-25T08:53:02.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5de7fff640 1 -- 172.21.15.37:0/949221706 learned_addr learned my addr 172.21.15.37:0/949221706 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:02.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5de6ffd640 1 --2- 172.21.15.37:0/949221706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de80b0440 0x7f5de8099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5de77fe640 1 -- 172.21.15.37:0/949221706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de80b0440 msgr2=0x7f5de8099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:02.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5de77fe640 1 --2- 172.21.15.37:0/949221706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de80b0440 0x7f5de8099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5de77fe640 1 -- 172.21.15.37:0/949221706 >> v1:172.21.15.37:6789/0 conn(0x7f5de80aeb80 legacy=0x7f5de80aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5de77fe640 1 -- 172.21.15.37:0/949221706 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de80ae660 con 0x7f5de815b920 2024-04-25T08:53:02.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5de67fc640 1 -- 172.21.15.37:0/949221706 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918568552 0 0) 0x7f5de809a950 con 0x7f5de80aeb80 2024-04-25T08:53:02.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.408+0000 7f5de6ffd640 1 --2- 172.21.15.37:0/949221706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de80b0440 0x7f5de8099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:02.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5de77fe640 1 --2- 172.21.15.37:0/949221706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de815b920 0x7f5de8099a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80097c0 tx=0x7f5dd801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=216248a8f197dfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:02.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5de67fc640 1 -- 172.21.15.37:0/949221706 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80b1070 con 0x7f5de815b920 2024-04-25T08:53:02.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5de67fc640 1 -- 172.21.15.37:0/949221706 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd80ac410 con 0x7f5de815b920 2024-04-25T08:53:02.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5de67fc640 1 -- 172.21.15.37:0/949221706 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80ac7e0 con 0x7f5de815b920 2024-04-25T08:53:02.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5dedbe1640 1 -- 172.21.15.37:0/949221706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de815b920 msgr2=0x7f5de8099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5dedbe1640 1 --2- 172.21.15.37:0/949221706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de815b920 0x7f5de8099a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80097c0 tx=0x7f5dd801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5dedbe1640 1 -- 172.21.15.37:0/949221706 shutdown_connections 2024-04-25T08:53:02.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5dedbe1640 1 --2- 172.21.15.37:0/949221706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de80b0440 0x7f5de8099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5dedbe1640 1 --2- 172.21.15.37:0/949221706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de815b920 0x7f5de8099a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5dedbe1640 1 -- 172.21.15.37:0/949221706 >> 172.21.15.37:0/949221706 conn(0x7f5de80b3010 msgr2=0x7f5de80b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:02.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5dedbe1640 1 -- 172.21.15.37:0/949221706 shutdown_connections 2024-04-25T08:53:02.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5dedbe1640 1 -- 172.21.15.37:0/949221706 wait complete. 2024-04-25T08:53:02.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5dedbe1640 1 Processor -- start 2024-04-25T08:53:02.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.412+0000 7f5dedbe1640 1 -- start start 2024-04-25T08:53:02.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dedbe1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de80aeb80 0x7f5de8152a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dedbe1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5de80b0440 0x7f5de806bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dedbe1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de815b920 0x7f5de806c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dedbe1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de8168850 con 0x7f5de80aeb80 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dedbe1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de81686d0 con 0x7f5de80b0440 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dedbe1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de81689d0 con 0x7f5de815b920 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5de77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de80aeb80 0x7f5de8152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5de77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de80aeb80 0x7f5de8152a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42222/0 (socket says 172.21.15.37:42222) 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5de77fe640 1 -- 172.21.15.37:0/3093656108 learned_addr learned my addr 172.21.15.37:0/3093656108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5de77fe640 1 -- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de815b920 msgr2=0x7f5de806c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5de77fe640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de815b920 0x7f5de806c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5de77fe640 1 -- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5de80b0440 msgr2=0x7f5de806bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5de77fe640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5de80b0440 0x7f5de806bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5de77fe640 1 -- 172.21.15.37:0/3093656108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de8165c70 con 0x7f5de80aeb80 2024-04-25T08:53:02.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5de77fe640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de80aeb80 0x7f5de8152a30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80af040 tx=0x7f5dd8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:02.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dcffff640 1 -- 172.21.15.37:0/3093656108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80b9070 con 0x7f5de80aeb80 2024-04-25T08:53:02.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dcffff640 1 -- 172.21.15.37:0/3093656108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd80ace40 con 0x7f5de80aeb80 2024-04-25T08:53:02.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dedbe1640 1 -- 172.21.15.37:0/3093656108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de8163e50 con 0x7f5de80aeb80 2024-04-25T08:53:02.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dcffff640 1 -- 172.21.15.37:0/3093656108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80b1040 con 0x7f5de80aeb80 2024-04-25T08:53:02.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dedbe1640 1 -- 172.21.15.37:0/3093656108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de8165ef0 con 0x7f5de80aeb80 2024-04-25T08:53:02.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dedbe1640 1 -- 172.21.15.37:0/3093656108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de804efa0 con 0x7f5de80aeb80 2024-04-25T08:53:02.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dcffff640 1 -- 172.21.15.37:0/3093656108 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5dd80bc070 con 0x7f5de80aeb80 2024-04-25T08:53:02.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dcffff640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5dd4066f60 0x7f5dd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5dcffff640 1 -- 172.21.15.37:0/3093656108 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f5dd811eeb0 con 0x7f5de80aeb80 2024-04-25T08:53:02.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.416+0000 7f5de6ffd640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5dd4066f60 0x7f5dd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.420+0000 7f5de6ffd640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5dd4066f60 0x7f5dd4069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc006e90 tx=0x7f5ddc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.420+0000 7f5dcffff640 1 -- 172.21.15.37:0/3093656108 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5de8165c70 con 0x7f5de80aeb80 2024-04-25T08:53:02.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.616+0000 7f5dedbe1640 1 -- 172.21.15.37:0/3093656108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5de80b2810 con 0x7f5de80aeb80 2024-04-25T08:53:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dcffff640 1 -- 172.21.15.37:0/3093656108 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f5dd8016090 con 0x7f5de80aeb80 2024-04-25T08:53:02.612 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:02.612 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2902565757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2902565757},{"type":"v1","addr":"172.21.15.203:6835","nonce":2902565757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/822172299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":822172299},{"type":"v1","addr":"172.21.15.203:6837","nonce":822172299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3681855890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3681855890},{"type":"v1","addr":"172.21.15.37:6833","nonce":3681855890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:30.564153+0000","modified":"2024-04-25T08:52:53.726659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:53:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-25T08:53:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 -- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5dd4066f60 msgr2=0x7f5dd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5dd4066f60 0x7f5dd4069420 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc006e90 tx=0x7f5ddc008040 comp rx=0 tx=0).stop 2024-04-25T08:53:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 -- 172.21.15.37:0/3093656108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de80aeb80 msgr2=0x7f5de8152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de80aeb80 0x7f5de8152a30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5dd80af040 tx=0x7f5dd8019240 comp rx=0 tx=0).stop 2024-04-25T08:53:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 -- 172.21.15.37:0/3093656108 shutdown_connections 2024-04-25T08:53:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5dd4066f60 0x7f5dd4069420 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5de815b920 0x7f5de806c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5de80b0440 0x7f5de806bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 --2- 172.21.15.37:0/3093656108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5de80aeb80 0x7f5de8152a30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 -- 172.21.15.37:0/3093656108 >> 172.21.15.37:0/3093656108 conn(0x7f5de80b3010 msgr2=0x7f5de80aefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 -- 172.21.15.37:0/3093656108 shutdown_connections 2024-04-25T08:53:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.620+0000 7f5dedbe1640 1 -- 172.21.15.37:0/3093656108 wait complete. 2024-04-25T08:53:02.633 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.756+0000 7f2c76cb8640 1 Processor -- start 2024-04-25T08:53:02.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.756+0000 7f2c76cb8640 1 -- start start 2024-04-25T08:53:02.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2c7015a700 0x7f2c7015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c700b24a0 0x7f2c700b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c700b2090 con 0x7f2c700b3f50 2024-04-25T08:53:02.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7015f200 con 0x7f2c7015a700 2024-04-25T08:53:02.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c700b43e0 con 0x7f2c700b24a0 2024-04-25T08:53:02.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c74a54640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2c7015a700 0x7f2c7015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c74a54640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2c7015a700 0x7f2c7015ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42238/0 (socket says 172.21.15.37:42238) 2024-04-25T08:53:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c74a54640 1 -- 172.21.15.37:0/3197757385 learned_addr learned my addr 172.21.15.37:0/3197757385 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c75255640 1 --2- 172.21.15.37:0/3197757385 >> v2:172.21.15.203:3300/0 conn(0x7f2c700b24a0 0x7f2c700b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c74a54640 1 -- 172.21.15.37:0/3197757385 >> v2:172.21.15.203:3300/0 conn(0x7f2c700b24a0 msgr2=0x7f2c700b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c74a54640 1 --2- 172.21.15.37:0/3197757385 >> v2:172.21.15.203:3300/0 conn(0x7f2c700b24a0 0x7f2c700b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c74a54640 1 -- 172.21.15.37:0/3197757385 >> v1:172.21.15.203:6789/0 conn(0x7f2c700b3f50 legacy=0x7f2c700cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c74a54640 1 -- 172.21.15.37:0/3197757385 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c700b46c0 con 0x7f2c7015a700 2024-04-25T08:53:02.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c75255640 1 --2- 172.21.15.37:0/3197757385 >> v2:172.21.15.203:3300/0 conn(0x7f2c700b24a0 0x7f2c700b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:02.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c74a54640 1 --2- 172.21.15.37:0/3197757385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7015a700 0x7f2c7015ecc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c7006bd20 tx=0x7f2c6001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95f9d76e33a2453 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:02.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c6f7fe640 1 -- 172.21.15.37:0/3197757385 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c600a5620 con 0x7f2c7015a700 2024-04-25T08:53:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c6f7fe640 1 -- 172.21.15.37:0/3197757385 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c600a57c0 con 0x7f2c7015a700 2024-04-25T08:53:02.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c6f7fe640 1 -- 172.21.15.37:0/3197757385 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c600a5b90 con 0x7f2c7015a700 2024-04-25T08:53:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 -- 172.21.15.37:0/3197757385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7015a700 msgr2=0x7f2c7015ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 --2- 172.21.15.37:0/3197757385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7015a700 0x7f2c7015ecc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2c7006bd20 tx=0x7f2c6001c870 comp rx=0 tx=0).stop 2024-04-25T08:53:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 -- 172.21.15.37:0/3197757385 shutdown_connections 2024-04-25T08:53:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 --2- 172.21.15.37:0/3197757385 >> v2:172.21.15.203:3300/0 conn(0x7f2c700b24a0 0x7f2c700b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 --2- 172.21.15.37:0/3197757385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c7015a700 0x7f2c7015ecc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 -- 172.21.15.37:0/3197757385 >> 172.21.15.37:0/3197757385 conn(0x7f2c700b11a0 msgr2=0x7f2c700b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 -- 172.21.15.37:0/3197757385 shutdown_connections 2024-04-25T08:53:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.760+0000 7f2c76cb8640 1 -- 172.21.15.37:0/3197757385 wait complete. 2024-04-25T08:53:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c76cb8640 1 Processor -- start 2024-04-25T08:53:02.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c76cb8640 1 -- start start 2024-04-25T08:53:02.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c76cb8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c700b24a0 0x7f2c70151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c76cb8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c7015a700 0x7f2c701509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c76cb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c700cb940 0x7f2c70150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c76cb8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c700cd310 con 0x7f2c700cb940 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c76cb8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c700ce600 con 0x7f2c700b24a0 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c76cb8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c700cd490 con 0x7f2c7015a700 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c75255640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c700cb940 0x7f2c70150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c75255640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c700cb940 0x7f2c70150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42242/0 (socket says 172.21.15.37:42242) 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c75255640 1 -- 172.21.15.37:0/2372511506 learned_addr learned my addr 172.21.15.37:0/2372511506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c74a54640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c700b24a0 0x7f2c70151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c6ffff640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c7015a700 0x7f2c701509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c75255640 1 -- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c7015a700 msgr2=0x7f2c701509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c75255640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c7015a700 0x7f2c701509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c75255640 1 -- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c700b24a0 msgr2=0x7f2c70151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c75255640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c700b24a0 0x7f2c70151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c75255640 1 -- 172.21.15.37:0/2372511506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7015d340 con 0x7f2c700cb940 2024-04-25T08:53:02.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c74a54640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c700b24a0 0x7f2c70151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:02.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c6ffff640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c7015a700 0x7f2c701509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:02.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c75255640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c700cb940 0x7f2c70150ef0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800c7f0 tx=0x7f2c6800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:02.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c6d7fa640 1 -- 172.21.15.37:0/2372511506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68015020 con 0x7f2c700cb940 2024-04-25T08:53:02.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c6d7fa640 1 -- 172.21.15.37:0/2372511506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c68004250 con 0x7f2c700cb940 2024-04-25T08:53:02.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c6d7fa640 1 -- 172.21.15.37:0/2372511506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68003620 con 0x7f2c700cb940 2024-04-25T08:53:02.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c76cb8640 1 -- 172.21.15.37:0/2372511506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c7015d620 con 0x7f2c700cb940 2024-04-25T08:53:02.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.764+0000 7f2c76cb8640 1 -- 172.21.15.37:0/2372511506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c70154630 con 0x7f2c700cb940 2024-04-25T08:53:02.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.768+0000 7f2c76cb8640 1 -- 172.21.15.37:0/2372511506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7004efa0 con 0x7f2c700cb940 2024-04-25T08:53:02.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.768+0000 7f2c6d7fa640 1 -- 172.21.15.37:0/2372511506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c68018020 con 0x7f2c700cb940 2024-04-25T08:53:02.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.768+0000 7f2c6d7fa640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c50066f60 0x7f2c50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:02.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.768+0000 7f2c6d7fa640 1 -- 172.21.15.37:0/2372511506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2c680853d0 con 0x7f2c700cb940 2024-04-25T08:53:02.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.768+0000 7f2c74a54640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c50066f60 0x7f2c50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:02.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.768+0000 7f2c74a54640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c50066f60 0x7f2c50069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2c600ae000 tx=0x7f2c600a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:02.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.772+0000 7f2c6d7fa640 1 -- 172.21.15.37:0/2372511506 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c6801f030 con 0x7f2c700cb940 2024-04-25T08:53:02.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.928+0000 7f2c76cb8640 1 -- 172.21.15.37:0/2372511506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c70154800 con 0x7f2c700cb940 2024-04-25T08:53:02.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c6d7fa640 1 -- 172.21.15.37:0/2372511506 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f2c68056000 con 0x7f2c700cb940 2024-04-25T08:53:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:02.925 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:52:38.282648+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:02.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 -- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c50066f60 msgr2=0x7f2c50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c50066f60 0x7f2c50069420 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2c600ae000 tx=0x7f2c600a7070 comp rx=0 tx=0).stop 2024-04-25T08:53:02.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 -- 172.21.15.37:0/2372511506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c700cb940 msgr2=0x7f2c70150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:02.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c700cb940 0x7f2c70150ef0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2c6800c7f0 tx=0x7f2c6800ccb0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 -- 172.21.15.37:0/2372511506 shutdown_connections 2024-04-25T08:53:02.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c50066f60 0x7f2c50069420 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c700cb940 0x7f2c70150ef0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c7015a700 0x7f2c701509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 --2- 172.21.15.37:0/2372511506 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c700b24a0 0x7f2c70151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:02.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 -- 172.21.15.37:0/2372511506 >> 172.21.15.37:0/2372511506 conn(0x7f2c700b11a0 msgr2=0x7f2c700cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:02.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 -- 172.21.15.37:0/2372511506 shutdown_connections 2024-04-25T08:53:02.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:02.932+0000 7f2c76cb8640 1 -- 172.21.15.37:0/2372511506 wait complete. 2024-04-25T08:53:02.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:03.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f89357af640 1 Processor -- start 2024-04-25T08:53:03.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f89357af640 1 -- start start 2024-04-25T08:53:03.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f89357af640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f893009a180 0x7f89300cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f89357af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89300afb90 0x7f89300cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f89357af640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893009a950 con 0x7f893015b920 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f89357af640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893009ac30 con 0x7f893009a180 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f89357af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89300ae380 con 0x7f89300afb90 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f892effd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f893015b920 0x7f8930099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44666/0 (socket says 172.21.15.37:44666) 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f892effd640 1 -- 172.21.15.37:0/1390813392 learned_addr learned my addr 172.21.15.37:0/1390813392 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f8927fff640 1 --2- 172.21.15.37:0/1390813392 >> v2:172.21.15.37:3300/0 conn(0x7f893009a180 0x7f89300cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f892e7fc640 1 -- 172.21.15.37:0/1390813392 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137111543 0 0) 0x7f893009a950 con 0x7f893015b920 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f892f7fe640 1 --2- 172.21.15.37:0/1390813392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89300afb90 0x7f89300cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f892e7fc640 1 -- 172.21.15.37:0/1390813392 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f891c003530 con 0x7f893015b920 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f8927fff640 1 -- 172.21.15.37:0/1390813392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89300afb90 msgr2=0x7f89300cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f8927fff640 1 --2- 172.21.15.37:0/1390813392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89300afb90 0x7f89300cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f8927fff640 1 -- 172.21.15.37:0/1390813392 >> v1:172.21.15.37:6789/0 conn(0x7f893015b920 legacy=0x7f8930099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f8927fff640 1 -- 172.21.15.37:0/1390813392 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89300ae660 con 0x7f893009a180 2024-04-25T08:53:03.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.068+0000 7f8927fff640 1 --2- 172.21.15.37:0/1390813392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f893009a180 0x7f89300cc130 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f89180097c0 tx=0x7f891801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88c98880ab51ded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892e7fc640 1 -- 172.21.15.37:0/1390813392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89180b0070 con 0x7f893009a180 2024-04-25T08:53:03.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892e7fc640 1 -- 172.21.15.37:0/1390813392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89180ab600 con 0x7f893009a180 2024-04-25T08:53:03.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892e7fc640 1 -- 172.21.15.37:0/1390813392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89180ab9b0 con 0x7f893009a180 2024-04-25T08:53:03.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 -- 172.21.15.37:0/1390813392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f893009a180 msgr2=0x7f89300cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 --2- 172.21.15.37:0/1390813392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f893009a180 0x7f89300cc130 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f89180097c0 tx=0x7f891801c410 comp rx=0 tx=0).stop 2024-04-25T08:53:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 -- 172.21.15.37:0/1390813392 shutdown_connections 2024-04-25T08:53:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 --2- 172.21.15.37:0/1390813392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89300afb90 0x7f89300cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 --2- 172.21.15.37:0/1390813392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f893009a180 0x7f89300cc130 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 -- 172.21.15.37:0/1390813392 >> 172.21.15.37:0/1390813392 conn(0x7f89300b3010 msgr2=0x7f89300b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 -- 172.21.15.37:0/1390813392 shutdown_connections 2024-04-25T08:53:03.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 -- 172.21.15.37:0/1390813392 wait complete. 2024-04-25T08:53:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 Processor -- start 2024-04-25T08:53:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 -- start start 2024-04-25T08:53:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f893009a180 0x7f8930157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89300afb90 0x7f89301500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f893015b920 0x7f89301635c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89301684c0 con 0x7f893015b920 2024-04-25T08:53:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89300cd5d0 con 0x7f893009a180 2024-04-25T08:53:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8930168640 con 0x7f89300afb90 2024-04-25T08:53:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f893015b920 0x7f89301635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f893015b920 0x7f89301635c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42258/0 (socket says 172.21.15.37:42258) 2024-04-25T08:53:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892f7fe640 1 -- 172.21.15.37:0/3477364077 learned_addr learned my addr 172.21.15.37:0/3477364077 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892f7fe640 1 -- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89300afb90 msgr2=0x7f89301500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892f7fe640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89300afb90 0x7f89301500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892effd640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f893009a180 0x7f8930157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892f7fe640 1 -- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f893009a180 msgr2=0x7f8930157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892f7fe640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f893009a180 0x7f8930157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892f7fe640 1 -- 172.21.15.37:0/3477364077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893006be40 con 0x7f893015b920 2024-04-25T08:53:03.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f892f7fe640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f893015b920 0x7f89301635c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8928007e60 tx=0x7f892800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89277fe640 1 -- 172.21.15.37:0/3477364077 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8928014420 con 0x7f893015b920 2024-04-25T08:53:03.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89277fe640 1 -- 172.21.15.37:0/3477364077 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8928019070 con 0x7f893015b920 2024-04-25T08:53:03.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 -- 172.21.15.37:0/3477364077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893006c120 con 0x7f893015b920 2024-04-25T08:53:03.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89277fe640 1 -- 172.21.15.37:0/3477364077 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8928014820 con 0x7f893015b920 2024-04-25T08:53:03.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.072+0000 7f89357af640 1 -- 172.21.15.37:0/3477364077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893006c6e0 con 0x7f893015b920 2024-04-25T08:53:03.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.076+0000 7f89357af640 1 -- 172.21.15.37:0/3477364077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893004efa0 con 0x7f893015b920 2024-04-25T08:53:03.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.076+0000 7f89277fe640 1 -- 172.21.15.37:0/3477364077 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89280149c0 con 0x7f893015b920 2024-04-25T08:53:03.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.076+0000 7f89277fe640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8910066f90 0x7f8910069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.076+0000 7f89277fe640 1 -- 172.21.15.37:0/3477364077 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8928085bd0 con 0x7f893015b920 2024-04-25T08:53:03.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.076+0000 7f892effd640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8910066f90 0x7f8910069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.076+0000 7f892effd640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8910066f90 0x7f8910069450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8920007fd0 tx=0x7f8920009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.080+0000 7f89277fe640 1 -- 172.21.15.37:0/3477364077 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8928056890 con 0x7f893015b920 2024-04-25T08:53:03.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.276+0000 7f89357af640 1 -- 172.21.15.37:0/3477364077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f893015ed80 con 0x7f893015b920 2024-04-25T08:53:03.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.276+0000 7f89277fe640 1 -- 172.21.15.37:0/3477364077 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f8928056190 con 0x7f893015b920 2024-04-25T08:53:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:03.271 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2902565757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2902565757},{"type":"v1","addr":"172.21.15.203:6835","nonce":2902565757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/822172299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":822172299},{"type":"v1","addr":"172.21.15.203:6837","nonce":822172299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3681855890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3681855890},{"type":"v1","addr":"172.21.15.37:6833","nonce":3681855890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:30.564153+0000","modified":"2024-04-25T08:52:53.726659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:53:03.272 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-25T08:53:03.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 -- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8910066f90 msgr2=0x7f8910069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8910066f90 0x7f8910069450 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8920007fd0 tx=0x7f8920009040 comp rx=0 tx=0).stop 2024-04-25T08:53:03.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 -- 172.21.15.37:0/3477364077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f893015b920 msgr2=0x7f89301635c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f893015b920 0x7f89301635c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8928007e60 tx=0x7f892800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 -- 172.21.15.37:0/3477364077 shutdown_connections 2024-04-25T08:53:03.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8910066f90 0x7f8910069450 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f893015b920 0x7f89301635c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89300afb90 0x7f89301500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 --2- 172.21.15.37:0/3477364077 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f893009a180 0x7f8930157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 -- 172.21.15.37:0/3477364077 >> 172.21.15.37:0/3477364077 conn(0x7f89300b3010 msgr2=0x7f89300b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 -- 172.21.15.37:0/3477364077 shutdown_connections 2024-04-25T08:53:03.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.280+0000 7f89357af640 1 -- 172.21.15.37:0/3477364077 wait complete. 2024-04-25T08:53:03.694 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:52:30.564153+0000', 'modified': '2024-04-25T08:52:53.726659+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4251}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4251': {'gid': 4251, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/1246598982', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 1246598982}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1246598982}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:53:03.694 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:53:03.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:53:03.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.836+0000 7f4244aa3640 1 Processor -- start 2024-04-25T08:53:03.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.836+0000 7f4244aa3640 1 -- start start 2024-04-25T08:53:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.836+0000 7f4244aa3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f424015da80 0x7f42400d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.836+0000 7f4244aa3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424006be60 con 0x7f42400d1cd0 2024-04-25T08:53:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.836+0000 7f4244aa3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424015de60 con 0x7f42400d07f0 2024-04-25T08:53:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.836+0000 7f4244aa3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f424006c440 con 0x7f424015da80 2024-04-25T08:53:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f423e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f424015da80 0x7f42400d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f423dd9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f42400d1cd0 0x7f42400d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44680/0 (socket says 172.21.15.37:44680) 2024-04-25T08:53:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f423dd9b640 1 -- 172.21.15.37:0/2999251779 learned_addr learned my addr 172.21.15.37:0/2999251779 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f423e59c640 1 -- 172.21.15.37:0/2999251779 >> v1:172.21.15.203:6789/0 conn(0x7f42400d07f0 legacy=0x7f424006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f423e59c640 1 -- 172.21.15.37:0/2999251779 >> v1:172.21.15.37:6789/0 conn(0x7f42400d1cd0 legacy=0x7f42400d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f423e59c640 1 -- 172.21.15.37:0/2999251779 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42400d2450 con 0x7f424015da80 2024-04-25T08:53:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f423e59c640 1 --2- 172.21.15.37:0/2999251779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f424015da80 0x7f42400d4230 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f424006c5c0 tx=0x7f423801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cac2e122571a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f423d59a640 1 -- 172.21.15.37:0/2999251779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380b2070 con 0x7f424015da80 2024-04-25T08:53:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f423d59a640 1 -- 172.21.15.37:0/2999251779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42380a5900 con 0x7f424015da80 2024-04-25T08:53:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f423d59a640 1 -- 172.21.15.37:0/2999251779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380a5cd0 con 0x7f424015da80 2024-04-25T08:53:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f4244aa3640 1 -- 172.21.15.37:0/2999251779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f424015da80 msgr2=0x7f42400d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f4244aa3640 1 --2- 172.21.15.37:0/2999251779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f424015da80 0x7f42400d4230 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f424006c5c0 tx=0x7f423801c8e0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f4244aa3640 1 -- 172.21.15.37:0/2999251779 shutdown_connections 2024-04-25T08:53:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f4244aa3640 1 --2- 172.21.15.37:0/2999251779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f424015da80 0x7f42400d4230 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f4244aa3640 1 -- 172.21.15.37:0/2999251779 >> 172.21.15.37:0/2999251779 conn(0x7f424008b000 msgr2=0x7f42400d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f4244aa3640 1 -- 172.21.15.37:0/2999251779 shutdown_connections 2024-04-25T08:53:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.840+0000 7f4244aa3640 1 -- 172.21.15.37:0/2999251779 wait complete. 2024-04-25T08:53:03.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f4244aa3640 1 Processor -- start 2024-04-25T08:53:03.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f4244aa3640 1 -- start start 2024-04-25T08:53:03.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f4244aa3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42400d07f0 0x7f42400b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f4244aa3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42400d1cd0 0x7f42400a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f4244aa3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f424015da80 0x7f42400cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f4244aa3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424006d970 con 0x7f42400d07f0 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f4244aa3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424006d7f0 con 0x7f42400d1cd0 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f4244aa3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424006daf0 con 0x7f424015da80 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42400d07f0 0x7f42400b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42400d07f0 0x7f42400b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42266/0 (socket says 172.21.15.37:42266) 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423e59c640 1 -- 172.21.15.37:0/2800556935 learned_addr learned my addr 172.21.15.37:0/2800556935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423dd9b640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42400d1cd0 0x7f42400a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423ed9d640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f424015da80 0x7f42400cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423e59c640 1 -- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f424015da80 msgr2=0x7f42400cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423e59c640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f424015da80 0x7f42400cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423e59c640 1 -- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42400d1cd0 msgr2=0x7f42400a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423e59c640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42400d1cd0 0x7f42400a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423e59c640 1 -- 172.21.15.37:0/2800556935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42401faa90 con 0x7f42400d07f0 2024-04-25T08:53:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423ed9d640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f424015da80 0x7f42400cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f423e59c640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42400d07f0 0x7f42400b2550 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f423801a510 tx=0x7f4238019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f422effd640 1 -- 172.21.15.37:0/2800556935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380b2040 con 0x7f42400d07f0 2024-04-25T08:53:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f422effd640 1 -- 172.21.15.37:0/2800556935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42380bf070 con 0x7f42400d07f0 2024-04-25T08:53:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f4244aa3640 1 -- 172.21.15.37:0/2800556935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42400a0100 con 0x7f42400d07f0 2024-04-25T08:53:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f422effd640 1 -- 172.21.15.37:0/2800556935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380ad920 con 0x7f42400d07f0 2024-04-25T08:53:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.844+0000 7f4244aa3640 1 -- 172.21.15.37:0/2800556935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42400a06c0 con 0x7f42400d07f0 2024-04-25T08:53:03.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f422effd640 1 -- 172.21.15.37:0/2800556935 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f42380adac0 con 0x7f42400d07f0 2024-04-25T08:53:03.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f422effd640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4220066f60 0x7f4220069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f422effd640 1 -- 172.21.15.37:0/2800556935 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f42381205e0 con 0x7f42400d07f0 2024-04-25T08:53:03.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423dd9b640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4220066f60 0x7f4220069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f4247c10000 1 Processor -- start 2024-04-25T08:53:03.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f4247c10000 1 -- start start 2024-04-25T08:53:03.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423dd9b640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4220066f60 0x7f4220069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f42400cb940 tx=0x7f4234009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f4247c10000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05dd920 0x55b5c05ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f4247c10000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c05939f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f4247c10000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c0593f30 0x55b5c0594310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f4247c10000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c047b5f0 con 0x55b5c05de240 2024-04-25T08:53:03.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f4247c10000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c03e2da0 con 0x55b5c0593f30 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f4247c10000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5c036b150 con 0x55b5c05dd920 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c05939f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c05939f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42274/0 (socket says 172.21.15.37:42274) 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423dd9b640 1 -- 172.21.15.37:0/4029306911 learned_addr learned my addr 172.21.15.37:0/4029306911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423ed9d640 1 --2- 172.21.15.37:0/4029306911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c0593f30 0x55b5c0594310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423e59c640 1 --2- 172.21.15.37:0/4029306911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05dd920 0x55b5c05ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423dd9b640 1 -- 172.21.15.37:0/4029306911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05dd920 msgr2=0x55b5c05ddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423dd9b640 1 --2- 172.21.15.37:0/4029306911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05dd920 0x55b5c05ddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423dd9b640 1 -- 172.21.15.37:0/4029306911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c0593f30 msgr2=0x55b5c0594310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423dd9b640 1 --2- 172.21.15.37:0/4029306911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c0593f30 0x55b5c0594310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423dd9b640 1 -- 172.21.15.37:0/4029306911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5c0384ca0 con 0x55b5c05de240 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423e59c640 1 --2- 172.21.15.37:0/4029306911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05dd920 0x55b5c05ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f423dd9b640 1 --2- 172.21.15.37:0/4029306911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c05939f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4234001de0 tx=0x7f4234011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e439feb6a61ec07e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f421ffff640 1 -- 172.21.15.37:0/4029306911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4234005d30 con 0x55b5c05de240 2024-04-25T08:53:03.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f421ffff640 1 -- 172.21.15.37:0/4029306911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423400e7b0 con 0x55b5c05de240 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.848+0000 7f421ffff640 1 -- 172.21.15.37:0/4029306911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423400eb60 con 0x55b5c05de240 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- 172.21.15.37:0/4029306911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 msgr2=0x55b5c05939f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 --2- 172.21.15.37:0/4029306911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c05939f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4234001de0 tx=0x7f4234011000 comp rx=0 tx=0).stop 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- 172.21.15.37:0/4029306911 shutdown_connections 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 --2- 172.21.15.37:0/4029306911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c0593f30 0x55b5c0594310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 --2- 172.21.15.37:0/4029306911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c05939f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 --2- 172.21.15.37:0/4029306911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05dd920 0x55b5c05ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- 172.21.15.37:0/4029306911 >> 172.21.15.37:0/4029306911 conn(0x55b5c059a2a0 msgr2=0x55b5c05922d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- 172.21.15.37:0/4029306911 shutdown_connections 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- 172.21.15.37:0/4029306911 wait complete. 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- start start 2024-04-25T08:53:03.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 Processor -- start 2024-04-25T08:53:03.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c05e4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05dd920 0x55b5c05e53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05de240 0x55b5c0597aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c039d990 con 0x55b5c0593f30 2024-04-25T08:53:03.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c03dee40 con 0x55b5c05dd920 2024-04-25T08:53:03.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5c036e900 con 0x55b5c05de240 2024-04-25T08:53:03.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 10 client.0 _ll_register_callbacks cb 0x55b5c04961c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c05e4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c05e4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42282/0 (socket says 172.21.15.37:42282) 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423e59c640 1 -- 172.21.15.37:0/3048550227 learned_addr learned my addr 172.21.15.37:0/3048550227 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423dd9b640 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05dd920 0x55b5c05e53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423e59c640 1 -- 172.21.15.37:0/3048550227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05de240 msgr2=0x55b5c0597aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423e59c640 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05de240 0x55b5c0597aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423e59c640 1 -- 172.21.15.37:0/3048550227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05dd920 msgr2=0x55b5c05e53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423e59c640 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05dd920 0x55b5c05e53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423e59c640 1 -- 172.21.15.37:0/3048550227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5c039b630 con 0x55b5c0593f30 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423dd9b640 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05dd920 0x55b5c05e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423e59c640 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c05e4e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f42380f8eb0 tx=0x7f42380a59c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd2fd2d2c8b80b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f421d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f421d7fa640 1 -- 172.21.15.37:0/3048550227 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380f4e20 con 0x55b5c0593f30 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f421d7fa640 1 -- 172.21.15.37:0/3048550227 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f42380eefd0 con 0x55b5c0593f30 2024-04-25T08:53:03.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- 172.21.15.37:0/3048550227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f423400e7b0 con 0x55b5c0593f30 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f421d7fa640 1 -- 172.21.15.37:0/3048550227 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42380f93e0 con 0x55b5c0593f30 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 10 client.4785 fetch_fsmap learned FSMap version 9 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- 172.21.15.37:0/3048550227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55b5c036b3a0 con 0x55b5c0593f30 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f421d7fa640 1 -- 172.21.15.37:0/3048550227 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423400e7b0 con 0x55b5c0593f30 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f421d7fa640 1 -- 172.21.15.37:0/3048550227 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f42380f8f50 con 0x55b5c0593f30 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 10 client.4785 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 10 client.4785 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 20 client.4785 populate_metadata read hostname 'smithi037' 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x55b5c0599160 0x55b5c05eeec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 4 client.4785 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f4247c10000 1 -- 172.21.15.37:0/3048550227 --> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5c037ea50 con 0x55b5c0599160 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423dd9b640 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x55b5c0599160 0x55b5c05eeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f421d7fa640 10 client.4785 ms_handle_connect on v2:172.21.15.37:6834/1246598982 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f423dd9b640 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x55b5c0599160 0x55b5c05eeec0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71263b81c58275e3 server_cookie=b1fa63e71382d986 in_seq=0 out_seq=0 2024-04-25T08:53:03.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f421d7fa640 1 -- 172.21.15.37:0/3048550227 <== mds.0 v2:172.21.15.37:6834/1246598982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5c037ea50 con 0x55b5c0599160 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.852+0000 7f421d7fa640 10 client.4785 handle_command_reply: tid=0 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 1 client.4785 shutdown 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 1 -- 172.21.15.37:0/3048550227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 msgr2=0x55b5c05e4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c05e4e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f42380f8eb0 tx=0x7f42380a59c0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 1 -- 172.21.15.37:0/3048550227 shutdown_connections 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x55b5c0599160 0x55b5c05eeec0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05de240 0x55b5c0597aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05dd920 0x55b5c05e53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 1 --2- 172.21.15.37:0/3048550227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c05e4e60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f421d7fa640 0 client.4785 ms_handle_reset on v2:172.21.15.37:6834/1246598982 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 1 -- 172.21.15.37:0/3048550227 >> 172.21.15.37:0/3048550227 conn(0x55b5c059a2a0 msgr2=0x55b5c05929e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 1 -- 172.21.15.37:0/3048550227 shutdown_connections 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 1 -- 172.21.15.37:0/3048550227 wait complete. 2024-04-25T08:53:03.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.856+0000 7f4247c10000 20 client.4785 trim_cache size 0 max 16384 2024-04-25T08:53:03.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.868+0000 7f4247c10000 1 Processor -- start 2024-04-25T08:53:03.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- start start 2024-04-25T08:53:03.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c0639370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05dd920 0x55b5c06398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05de240 0x55b5c0639df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4240202ea0 con 0x55b5c0593f30 2024-04-25T08:53:03.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42400b8cb0 con 0x55b5c05de240 2024-04-25T08:53:03.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4240202920 con 0x55b5c05dd920 2024-04-25T08:53:03.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c0639370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c0639370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42288/0 (socket says 172.21.15.37:42288) 2024-04-25T08:53:03.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423e59c640 1 -- 172.21.15.37:0/3349049743 learned_addr learned my addr 172.21.15.37:0/3349049743 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:03.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423e59c640 1 -- 172.21.15.37:0/3349049743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05dd920 msgr2=0x55b5c06398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423e59c640 1 --2- 172.21.15.37:0/3349049743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05dd920 0x55b5c06398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423e59c640 1 -- 172.21.15.37:0/3349049743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05de240 msgr2=0x55b5c0639df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423e59c640 1 --2- 172.21.15.37:0/3349049743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05de240 0x55b5c0639df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423e59c640 1 -- 172.21.15.37:0/3349049743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5c05de070 con 0x55b5c0593f30 2024-04-25T08:53:03.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423e59c640 1 --2- 172.21.15.37:0/3349049743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c0639370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42380eef80 tx=0x7f42380c5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e2ae67dcc97171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f421f7fe640 1 -- 172.21.15.37:0/3349049743 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380f0810 con 0x55b5c0593f30 2024-04-25T08:53:03.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f421f7fe640 1 -- 172.21.15.37:0/3349049743 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423811e4f0 con 0x55b5c0593f30 2024-04-25T08:53:03.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f421f7fe640 1 -- 172.21.15.37:0/3349049743 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42380f0630 con 0x55b5c0593f30 2024-04-25T08:53:03.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- 172.21.15.37:0/3349049743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 msgr2=0x55b5c0639370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 --2- 172.21.15.37:0/3349049743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c0639370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f42380eef80 tx=0x7f42380c5000 comp rx=0 tx=0).stop 2024-04-25T08:53:03.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- 172.21.15.37:0/3349049743 shutdown_connections 2024-04-25T08:53:03.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 --2- 172.21.15.37:0/3349049743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05de240 0x55b5c0639df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 --2- 172.21.15.37:0/3349049743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c05dd920 0x55b5c06398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 --2- 172.21.15.37:0/3349049743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c0593f30 0x55b5c0639370 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- 172.21.15.37:0/3349049743 >> 172.21.15.37:0/3349049743 conn(0x55b5c059a2a0 msgr2=0x55b5c0597220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- 172.21.15.37:0/3349049743 shutdown_connections 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- 172.21.15.37:0/3349049743 wait complete. 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- start start 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 Processor -- start 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c0593f30 0x55b5c063ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05dd920 0x55b5c063ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c063afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c03e73d0 con 0x55b5c05de240 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5c049ed50 con 0x55b5c05dd920 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5c037d980 con 0x55b5c0593f30 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f4247c10000 10 client.0 _ll_register_callbacks cb 0x7f42340013d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c063afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c063afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42290/0 (socket says 172.21.15.37:42290) 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423ed9d640 1 -- 172.21.15.37:0/1134373278 learned_addr learned my addr 172.21.15.37:0/1134373278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423dd9b640 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05dd920 0x55b5c063ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423e59c640 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c0593f30 0x55b5c063ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423ed9d640 1 -- 172.21.15.37:0/1134373278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c0593f30 msgr2=0x55b5c063ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423ed9d640 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c0593f30 0x55b5c063ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423ed9d640 1 -- 172.21.15.37:0/1134373278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05dd920 msgr2=0x55b5c063ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423ed9d640 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05dd920 0x55b5c063ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423ed9d640 1 -- 172.21.15.37:0/1134373278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5c0464600 con 0x55b5c05de240 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.872+0000 7f423e59c640 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c0593f30 0x55b5c063ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f423ed9d640 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c063afb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4230005480 tx=0x7f423000f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2787dee9f71f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f421d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f421d7fa640 1 -- 172.21.15.37:0/1134373278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4230017020 con 0x55b5c05de240 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f421d7fa640 1 -- 172.21.15.37:0/1134373278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4230003870 con 0x55b5c05de240 2024-04-25T08:53:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 -- 172.21.15.37:0/1134373278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f423811e4f0 con 0x55b5c05de240 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f421d7fa640 1 -- 172.21.15.37:0/1134373278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4230021050 con 0x55b5c05de240 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 10 client.4791 fetch_fsmap learned FSMap version 9 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 -- 172.21.15.37:0/1134373278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55b5c03779b0 con 0x55b5c05de240 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f421d7fa640 1 -- 172.21.15.37:0/1134373278 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423811e4f0 con 0x55b5c05de240 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f421d7fa640 1 -- 172.21.15.37:0/1134373278 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f4230024020 con 0x55b5c05de240 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 10 client.4791 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 10 client.4791 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 20 client.4791 populate_metadata read hostname 'smithi037' 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x55b5c063b4f0 0x55b5c063b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 4 client.4791 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 -- 172.21.15.37:0/1134373278 --> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b5c03f02e0 con 0x55b5c063b4f0 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f423e59c640 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x55b5c063b4f0 0x55b5c063b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f423e59c640 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x55b5c063b4f0 0x55b5c063b8d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e2a351b31618194 server_cookie=6bd8c57219b41fd2 in_seq=0 out_seq=0 2024-04-25T08:53:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f421d7fa640 10 client.4791 ms_handle_connect on v2:172.21.15.37:6834/1246598982 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f421d7fa640 1 -- 172.21.15.37:0/1134373278 <== mds.0 v2:172.21.15.37:6834/1246598982 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55b5c03f02e0 con 0x55b5c063b4f0 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f421d7fa640 10 client.4791 handle_command_reply: tid=0 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 client.4791 shutdown 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 -- 172.21.15.37:0/1134373278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 msgr2=0x55b5c063afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c063afb0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4230005480 tx=0x7f423000f5a0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 -- 172.21.15.37:0/1134373278 shutdown_connections 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b5c05de240 0x55b5c063afb0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b5c05dd920 0x55b5c063ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.37:6834/1246598982,v1:172.21.15.37:6835/1246598982] conn(0x55b5c063b4f0 0x55b5c063b8d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f421d7fa640 0 client.4791 ms_handle_reset on v2:172.21.15.37:6834/1246598982 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 --2- 172.21.15.37:0/1134373278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b5c0593f30 0x55b5c063ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 -- 172.21.15.37:0/1134373278 >> 172.21.15.37:0/1134373278 conn(0x55b5c059a2a0 msgr2=0x55b5c05e5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 -- 172.21.15.37:0/1134373278 shutdown_connections 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 1 -- 172.21.15.37:0/1134373278 wait complete. 2024-04-25T08:53:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4247c10000 20 client.4791 trim_cache size 0 max 16384 2024-04-25T08:53:03.876 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:53:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T08:53:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T08:53:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4251, 2024-04-25T08:53:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T08:53:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T08:53:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T08:53:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 32.665358165000001, 2024-04-25T08:53:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 9, 2024-04-25T08:53:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 27, 2024-04-25T08:53:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 27, 2024-04-25T08:53:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 39.646954491999999 2024-04-25T08:53:03.877 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 -- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4220066f60 msgr2=0x7f4220069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4220066f60 0x7f4220069420 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f42400cb940 tx=0x7f4234009040 comp rx=0 tx=0).stop 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 -- 172.21.15.37:0/2800556935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42400d07f0 msgr2=0x7f42400b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42400d07f0 0x7f42400b2550 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f423801a510 tx=0x7f4238019680 comp rx=0 tx=0).stop 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 -- 172.21.15.37:0/2800556935 shutdown_connections 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4220066f60 0x7f4220069420 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f424015da80 0x7f42400cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f42400d1cd0 0x7f42400a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 --2- 172.21.15.37:0/2800556935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42400d07f0 0x7f42400b2550 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 -- 172.21.15.37:0/2800556935 >> 172.21.15.37:0/2800556935 conn(0x7f424008b000 msgr2=0x7f42400b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 -- 172.21.15.37:0/2800556935 shutdown_connections 2024-04-25T08:53:03.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:03.876+0000 7f4244aa3640 1 -- 172.21.15.37:0/2800556935 wait complete. 2024-04-25T08:53:03.885 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T08:53:03.896 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T08:53:03.896 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T08:53:03.896 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T08:53:03.897 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:53:03.897 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:03.897 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T08:53:03.897 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:53:03.897 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:53:03.897 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T08:53:03.897 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:53:03.897 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:03.897 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T08:53:03.897 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:53:03.897 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:53:03.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:03.898 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:53:03.902 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T08:53:03.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:03.903 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T08:53:03.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:03.954 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-25T08:53:04.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.125 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-25T08:53:04.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.133 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-25T08:53:04.192 INFO:teuthology.orchestra.run.smithi203.stderr:Cannot find device "ceph-brx" 2024-04-25T08:53:04.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:53:04.193 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T08:53:04.194 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:53:04.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.328+0000 7f38ab865640 1 Processor -- start 2024-04-25T08:53:04.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.328+0000 7f38ab865640 1 -- start start 2024-04-25T08:53:04.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38ab865640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a40ce510 0x7f38a40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38ab865640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f38a4144ca0 0x7f38a406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38ab865640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a40cee30 con 0x7f38a406ec60 2024-04-25T08:53:04.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38ab865640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a40cc2b0 con 0x7f38a4144ca0 2024-04-25T08:53:04.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38ab865640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a406e290 con 0x7f38a40ce510 2024-04-25T08:53:04.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38a8e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f38a4144ca0 0x7f38a406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38a9601640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a40ce510 0x7f38a40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38a9601640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a40ce510 0x7f38a40ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:40854/0 (socket says 172.21.15.37:40854) 2024-04-25T08:53:04.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38a8e00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f38a4144ca0 0x7f38a406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:44262/0 (socket says 172.21.15.37:44262) 2024-04-25T08:53:04.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38a9601640 1 -- 172.21.15.37:0/1602064271 learned_addr learned my addr 172.21.15.37:0/1602064271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:04.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38a8e00640 1 -- 172.21.15.37:0/1602064271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a40ce510 msgr2=0x7f38a40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:04.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38a8e00640 1 --2- 172.21.15.37:0/1602064271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a40ce510 0x7f38a40ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38a8e00640 1 -- 172.21.15.37:0/1602064271 >> v1:172.21.15.203:6789/0 conn(0x7f38a406ec60 legacy=0x7f38a4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38a8e00640 1 -- 172.21.15.37:0/1602064271 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a40ccc70 con 0x7f38a4144ca0 2024-04-25T08:53:04.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.332+0000 7f38a9601640 1 --2- 172.21.15.37:0/1602064271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a40ce510 0x7f38a40ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:04.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38a8e00640 1 --2- 172.21.15.37:0/1602064271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38a4144ca0 0x7f38a406e720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f38940097c0 tx=0x7f389401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=298ec926143544e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f389bfff640 1 -- 172.21.15.37:0/1602064271 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38940b5070 con 0x7f38a4144ca0 2024-04-25T08:53:04.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f389bfff640 1 -- 172.21.15.37:0/1602064271 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38940a8990 con 0x7f38a4144ca0 2024-04-25T08:53:04.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f389bfff640 1 -- 172.21.15.37:0/1602064271 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38940a8d60 con 0x7f38a4144ca0 2024-04-25T08:53:04.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 -- 172.21.15.37:0/1602064271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38a4144ca0 msgr2=0x7f38a406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 --2- 172.21.15.37:0/1602064271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38a4144ca0 0x7f38a406e720 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f38940097c0 tx=0x7f389401ec90 comp rx=0 tx=0).stop 2024-04-25T08:53:04.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 -- 172.21.15.37:0/1602064271 shutdown_connections 2024-04-25T08:53:04.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 --2- 172.21.15.37:0/1602064271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38a4144ca0 0x7f38a406e720 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 --2- 172.21.15.37:0/1602064271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a40ce510 0x7f38a40ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 -- 172.21.15.37:0/1602064271 >> 172.21.15.37:0/1602064271 conn(0x7f38a40b11a0 msgr2=0x7f38a40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:04.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 -- 172.21.15.37:0/1602064271 shutdown_connections 2024-04-25T08:53:04.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 -- 172.21.15.37:0/1602064271 wait complete. 2024-04-25T08:53:04.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 Processor -- start 2024-04-25T08:53:04.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 -- start start 2024-04-25T08:53:04.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a406ec60 0x7f38a41e2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38a40ce510 0x7f38a41eec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a4144ca0 0x7f38a41f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a40cc900 con 0x7f38a4144ca0 2024-04-25T08:53:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a406df20 con 0x7f38a40ce510 2024-04-25T08:53:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.336+0000 7f38ab865640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4142d90 con 0x7f38a406ec60 2024-04-25T08:53:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9e02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a4144ca0 0x7f38a41f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9e02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a4144ca0 0x7f38a41f20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42296/0 (socket says 172.21.15.37:42296) 2024-04-25T08:53:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9e02640 1 -- 172.21.15.37:0/2383820261 learned_addr learned my addr 172.21.15.37:0/2383820261 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9601640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a406ec60 0x7f38a41e2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a8e00640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38a40ce510 0x7f38a41eec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9e02640 1 -- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a406ec60 msgr2=0x7f38a41e2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9e02640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a406ec60 0x7f38a41e2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9e02640 1 -- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38a40ce510 msgr2=0x7f38a41eec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9e02640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38a40ce510 0x7f38a41eec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9e02640 1 -- 172.21.15.37:0/2383820261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a41f2610 con 0x7f38a4144ca0 2024-04-25T08:53:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9601640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a406ec60 0x7f38a41e2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9e02640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a4144ca0 0x7f38a41f20d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c7a0 tx=0x7f389c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f3899ffb640 1 -- 172.21.15.37:0/2383820261 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c003b80 con 0x7f38a4144ca0 2024-04-25T08:53:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f3899ffb640 1 -- 172.21.15.37:0/2383820261 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c003d20 con 0x7f38a4144ca0 2024-04-25T08:53:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38ab865640 1 -- 172.21.15.37:0/2383820261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a41e2720 con 0x7f38a4144ca0 2024-04-25T08:53:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f3899ffb640 1 -- 172.21.15.37:0/2383820261 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00aa30 con 0x7f38a4144ca0 2024-04-25T08:53:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38ab865640 1 -- 172.21.15.37:0/2383820261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a41e2ce0 con 0x7f38a4144ca0 2024-04-25T08:53:04.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38ab865640 1 -- 172.21.15.37:0/2383820261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a404efa0 con 0x7f38a4144ca0 2024-04-25T08:53:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f3899ffb640 1 -- 172.21.15.37:0/2383820261 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f389c00abd0 con 0x7f38a4144ca0 2024-04-25T08:53:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f3899ffb640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3888066f60 0x7f3888069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f3899ffb640 1 -- 172.21.15.37:0/2383820261 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f389c085280 con 0x7f38a4144ca0 2024-04-25T08:53:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.340+0000 7f38a9601640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3888066f60 0x7f3888069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:04.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.344+0000 7f38a9601640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3888066f60 0x7f3888069420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38a4140920 tx=0x7f38a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:04.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.344+0000 7f3899ffb640 1 -- 172.21.15.37:0/2383820261 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f38a41f2610 con 0x7f38a4144ca0 2024-04-25T08:53:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 -- 172.21.15.37:0/2383820261 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f38a4142460 con 0x7f38a4144ca0 2024-04-25T08:53:04.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f3899ffb640 1 -- 172.21.15.37:0/2383820261 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f389c055f50 con 0x7f38a4144ca0 2024-04-25T08:53:04.537 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:53:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 -- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3888066f60 msgr2=0x7f3888069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3888066f60 0x7f3888069420 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f38a4140920 tx=0x7f38a0008040 comp rx=0 tx=0).stop 2024-04-25T08:53:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 -- 172.21.15.37:0/2383820261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a4144ca0 msgr2=0x7f38a41f20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a4144ca0 0x7f38a41f20d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f389c00c7a0 tx=0x7f389c00cc60 comp rx=0 tx=0).stop 2024-04-25T08:53:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 -- 172.21.15.37:0/2383820261 shutdown_connections 2024-04-25T08:53:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3888066f60 0x7f3888069420 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a4144ca0 0x7f38a41f20d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f38a40ce510 0x7f38a41eec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 --2- 172.21.15.37:0/2383820261 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38a406ec60 0x7f38a41e2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 -- 172.21.15.37:0/2383820261 >> 172.21.15.37:0/2383820261 conn(0x7f38a40b11a0 msgr2=0x7f38a40c4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 -- 172.21.15.37:0/2383820261 shutdown_connections 2024-04-25T08:53:04.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:04.544+0000 7f38ab865640 1 -- 172.21.15.37:0/2383820261 wait complete. 2024-04-25T08:53:04.556 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:53:04.556 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:53:04.556 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T08:53:04.556 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:53:04.556 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:04.556 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:53:04.556 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:53:04.556 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:53:04.556 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:53:04.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.556 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T08:53:04.562 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:53:04.563 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:53:04.563 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:53:04.563 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:04.563 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:53:04.563 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:04.563 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:04.563 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:04.563 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T08:53:04.563 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42456sec preferred_lft 42456sec 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:53:04.564 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:04.565 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:53:04.565 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:04.565 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:53:04.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.565 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:53:04.565 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:53:04.565 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:53:04.565 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-04-25T08:53:04.565 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-04-25T08:53:04.565 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:53:04.565 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:53:04.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.642 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:53:04.694 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:53:04.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.696 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T08:53:04.746 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T08:53:04.746 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:53:04.746 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:53:04.746 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:53:04.746 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:53:04.746 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:53:04.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.747 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:53:04.747 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:53:04.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:53:04.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:53:04.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:53:04.747 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:53:04.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.823 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:53:04.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.873 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-25T08:53:04.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.921 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:53:04.921 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:53:04.921 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:04.921 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T08:53:04.921 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:53:04.993 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T08:53:04.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:04.993 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:53:04.993 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:53:04.993 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T08:53:04.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:53:04.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T08:53:04.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T08:53:04.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T08:53:04.994 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:53:05.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:05.090 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:53:05.091 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:53:05.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T08:53:05.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:53:05.091 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:53:05.110 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:53:05.110 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:53:05.110 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:05.156 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:53:05.156 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:53:05.156 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:05.204 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:53:05.265 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:05.313 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:53:05.313 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:53:05.313 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:53:05.313 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:53:05.313 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:53:05.314 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:05.315 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:05.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:05.316 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:53:05.365 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:53:05.365 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:53:05.409 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:53:05.424 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:05.477 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:53:05.477 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:53:05.477 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:53:05.477 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:53:05.477 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:53:05.478 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:05.479 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:05.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:05.480 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:53:05.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T08:53:05.492+0000 7fc69dd98580 -1 init, newargv = 0x562b185f9d80 newargc=15 2024-04-25T08:53:05.485 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[36007]: starting ceph client 2024-04-25T08:53:05.492 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[36007]: starting fuse 2024-04-25T08:53:06.592 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:53:06.609 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:06.659 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:53:06.659 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:53:06.659 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:53:06.659 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:53:06.659 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:53:06.659 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:53:06.659 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:53:06.659 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:53:06.659 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:53:06.659 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:53:06.659 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:53:06.660 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:53:06.661 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:53:06.661 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:53:06.661 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:53:06.661 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:06.661 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:06.661 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:06.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:06.662 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:53:06.709 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:53:06.709 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T08:53:06.710 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T08:53:06.711 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:53:06.838 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.36007.asok 2024-04-25T08:53:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:06.851 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36007.asok 2024-04-25T08:53:06.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:06.851 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36007.asok status 2024-04-25T08:53:07.012 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:53:07.012 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T08:53:07.012 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:53:07.012 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:53:07.012 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T08:53:07.012 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T08:53:07.012 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "36007", 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4800, 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 4800 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:53:07.013 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4275332019 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4275332019 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.4800 192.168.0.1:0/4275332019", 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/4275332019", 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 9, 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 27, 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T08:53:07.014 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T08:53:07.015 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:53:07.026 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:53:07.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.164+0000 7fae10829640 1 Processor -- start 2024-04-25T08:53:07.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.164+0000 7fae10829640 1 -- start start 2024-04-25T08:53:07.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.164+0000 7fae10829640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fae0c0ce510 0x7fae0c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.164+0000 7fae10829640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae0c1451f0 0x7fae0c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.164+0000 7fae10829640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0c0cee30 con 0x7fae0c144ca0 2024-04-25T08:53:07.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.164+0000 7fae10829640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0c0cc2b0 con 0x7fae0c0ce510 2024-04-25T08:53:07.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.164+0000 7fae10829640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c06e290 con 0x7fae0c1451f0 2024-04-25T08:53:07.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0a59c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fae0c0ce510 0x7fae0c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0ad9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae0c1451f0 0x7fae0c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0a59c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fae0c0ce510 0x7fae0c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:44272/0 (socket says 172.21.15.37:44272) 2024-04-25T08:53:07.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0a59c640 1 -- 172.21.15.37:0/820355607 learned_addr learned my addr 172.21.15.37:0/820355607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:07.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae09d9b640 1 --1- 172.21.15.37:0/820355607 >> v1:172.21.15.203:6789/0 conn(0x7fae0c144ca0 0x7fae0c06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:55388/0 (socket says 172.21.15.37:55388) 2024-04-25T08:53:07.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0ad9d640 1 -- 172.21.15.37:0/820355607 >> v2:172.21.15.203:3300/0 conn(0x7fae0c0ce510 msgr2=0x7fae0c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:07.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0ad9d640 1 --2- 172.21.15.37:0/820355607 >> v2:172.21.15.203:3300/0 conn(0x7fae0c0ce510 0x7fae0c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0ad9d640 1 -- 172.21.15.37:0/820355607 >> v1:172.21.15.203:6789/0 conn(0x7fae0c144ca0 legacy=0x7fae0c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0ad9d640 1 -- 172.21.15.37:0/820355607 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c0ccc70 con 0x7fae0c1451f0 2024-04-25T08:53:07.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0ad9d640 1 --2- 172.21.15.37:0/820355607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae0c1451f0 0x7fae0c142680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fae04009d90 tx=0x7fae0401f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256ac3ebf31f0fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0959a640 1 -- 172.21.15.37:0/820355607 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040b4070 con 0x7fae0c1451f0 2024-04-25T08:53:07.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0959a640 1 -- 172.21.15.37:0/820355607 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae040af600 con 0x7fae0c1451f0 2024-04-25T08:53:07.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.168+0000 7fae0959a640 1 -- 172.21.15.37:0/820355607 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae040af9d0 con 0x7fae0c1451f0 2024-04-25T08:53:07.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 -- 172.21.15.37:0/820355607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae0c1451f0 msgr2=0x7fae0c142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 --2- 172.21.15.37:0/820355607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae0c1451f0 0x7fae0c142680 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fae04009d90 tx=0x7fae0401f150 comp rx=0 tx=0).stop 2024-04-25T08:53:07.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 -- 172.21.15.37:0/820355607 shutdown_connections 2024-04-25T08:53:07.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 --2- 172.21.15.37:0/820355607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae0c1451f0 0x7fae0c142680 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 --2- 172.21.15.37:0/820355607 >> v2:172.21.15.203:3300/0 conn(0x7fae0c0ce510 0x7fae0c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 -- 172.21.15.37:0/820355607 >> 172.21.15.37:0/820355607 conn(0x7fae0c0b11a0 msgr2=0x7fae0c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:07.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 -- 172.21.15.37:0/820355607 shutdown_connections 2024-04-25T08:53:07.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 -- 172.21.15.37:0/820355607 wait complete. 2024-04-25T08:53:07.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 Processor -- start 2024-04-25T08:53:07.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 -- start start 2024-04-25T08:53:07.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae0c0ce510 0x7fae0c1e5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c144ca0 0x7fae0c143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae0c1451f0 0x7fae0c1f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c06ee30 con 0x7fae0c144ca0 2024-04-25T08:53:07.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c06df20 con 0x7fae0c0ce510 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0c0cc900 con 0x7fae0c1451f0 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae09d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c144ca0 0x7fae0c143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae09d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c144ca0 0x7fae0c143f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42306/0 (socket says 172.21.15.37:42306) 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae09d9b640 1 -- 172.21.15.37:0/1941767508 learned_addr learned my addr 172.21.15.37:0/1941767508 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae0a59c640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae0c0ce510 0x7fae0c1e5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae0ad9d640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae0c1451f0 0x7fae0c1f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae09d9b640 1 -- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae0c1451f0 msgr2=0x7fae0c1f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae09d9b640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae0c1451f0 0x7fae0c1f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae09d9b640 1 -- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae0c0ce510 msgr2=0x7fae0c1e5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae09d9b640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae0c0ce510 0x7fae0c1e5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae09d9b640 1 -- 172.21.15.37:0/1941767508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0c1f57e0 con 0x7fae0c144ca0 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae09d9b640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c144ca0 0x7fae0c143f50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fae0000c460 tx=0x7fae0000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fadfaffd640 1 -- 172.21.15.37:0/1941767508 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae00018070 con 0x7fae0c144ca0 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fadfaffd640 1 -- 172.21.15.37:0/1941767508 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae00007480 con 0x7fae0c144ca0 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 -- 172.21.15.37:0/1941767508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0c1e3e10 con 0x7fae0c144ca0 2024-04-25T08:53:07.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fadfaffd640 1 -- 172.21.15.37:0/1941767508 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae000047f0 con 0x7fae0c144ca0 2024-04-25T08:53:07.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.172+0000 7fae10829640 1 -- 172.21.15.37:0/1941767508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0c1e43d0 con 0x7fae0c144ca0 2024-04-25T08:53:07.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.176+0000 7fae10829640 1 -- 172.21.15.37:0/1941767508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0c04efa0 con 0x7fae0c144ca0 2024-04-25T08:53:07.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.176+0000 7fadfaffd640 1 -- 172.21.15.37:0/1941767508 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae0001b070 con 0x7fae0c144ca0 2024-04-25T08:53:07.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.176+0000 7fadfaffd640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadec066f60 0x7fadec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:07.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.176+0000 7fadfaffd640 1 -- 172.21.15.37:0/1941767508 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fae00084720 con 0x7fae0c144ca0 2024-04-25T08:53:07.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.176+0000 7fae0a59c640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadec066f60 0x7fadec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:07.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.176+0000 7fae0a59c640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadec066f60 0x7fadec069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009ae0 tx=0x7fadfc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:07.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.180+0000 7fadfaffd640 1 -- 172.21.15.37:0/1941767508 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae00055460 con 0x7fae0c144ca0 2024-04-25T08:53:07.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 -- 172.21.15.37:0/1941767508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fae0c0b0f80 con 0x7fae0c144ca0 2024-04-25T08:53:07.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fadfaffd640 1 -- 172.21.15.37:0/1941767508 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fae00054d60 con 0x7fae0c144ca0 2024-04-25T08:53:07.372 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:53:07.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 -- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadec066f60 msgr2=0x7fadec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadec066f60 0x7fadec069420 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009ae0 tx=0x7fadfc008040 comp rx=0 tx=0).stop 2024-04-25T08:53:07.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 -- 172.21.15.37:0/1941767508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c144ca0 msgr2=0x7fae0c143f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:07.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c144ca0 0x7fae0c143f50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fae0000c460 tx=0x7fae0000c920 comp rx=0 tx=0).stop 2024-04-25T08:53:07.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 -- 172.21.15.37:0/1941767508 shutdown_connections 2024-04-25T08:53:07.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fadec066f60 0x7fadec069420 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae0c1451f0 0x7fae0c1f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae0c144ca0 0x7fae0c143f50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 --2- 172.21.15.37:0/1941767508 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae0c0ce510 0x7fae0c1e5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:07.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 -- 172.21.15.37:0/1941767508 >> 172.21.15.37:0/1941767508 conn(0x7fae0c0b11a0 msgr2=0x7fae0c0c9520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:07.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 -- 172.21.15.37:0/1941767508 shutdown_connections 2024-04-25T08:53:07.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:07.380+0000 7fae10829640 1 -- 172.21.15.37:0/1941767508 wait complete. 2024-04-25T08:53:07.389 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:53:07.389 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:53:07.389 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T08:53:07.389 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T08:53:07.389 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:53:07.390 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:53:07.390 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:53:07.390 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:53:07.390 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:53:07.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:07.390 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T08:53:07.396 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:53:07.396 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:53:07.396 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:53:07.396 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.396 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:53:07.396 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.396 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:07.396 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:07.396 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T08:53:07.396 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42453sec preferred_lft 42453sec 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:07.397 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:53:07.398 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.398 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:53:07.398 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:07.398 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:53:07.398 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.398 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:53:07.398 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.398 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:53:07.398 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:07.398 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:53:07.398 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:07.399 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:53:07.447 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:53:07.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:07.448 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T08:53:07.496 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T08:53:07.497 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:53:07.497 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:53:07.497 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:53:07.497 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:53:07.497 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:53:07.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:07.497 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:53:07.497 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:53:07.497 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:53:07.497 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:53:07.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:53:07.498 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:53:07.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:07.567 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:53:07.616 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:53:07.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:07.616 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-25T08:53:07.663 INFO:teuthology.orchestra.run.smithi037.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T08:53:07.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:07.664 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:53:07.664 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:53:07.664 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:53:07.664 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T08:53:07.664 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:53:07.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:07.735 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 7a:c6:4b:17:b8:54 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::78c6:4bff:fe17:b854/64 scope link 2024-04-25T08:53:07.789 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:07.791 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:53:07.845 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:53:07.845 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:53:07.845 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:53:07.845 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.845 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:53:07.845 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.845 INFO:teuthology.orchestra.run.smithi037.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:53:07.845 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 7a:c6:4b:17:b8:54 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:53:07.845 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:53:07.845 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.845 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::78c6:4bff:fe17:b854/64 scope link 2024-04-25T08:53:07.846 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:07.847 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T08:53:07.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:07.847 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:53:07.847 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:53:07.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T08:53:07.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:53:07.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T08:53:07.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T08:53:07.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T08:53:07.847 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:53:07.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:07.943 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:53:07.943 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:53:07.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.1 up 2024-04-25T08:53:07.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:53:07.943 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:53:07.962 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T08:53:07.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:53:07.962 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T08:53:08.011 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:53:08.012 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:53:08.012 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:53:08.059 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:53:08.114 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:08.161 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:53:08.161 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:53:08.161 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:53:08.161 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:53:08.161 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:53:08.162 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:08.163 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:53:08.164 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:53:08.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:08.165 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:53:08.213 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:53:08.213 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T08:53:08.213 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T08:53:08.257 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:53:08.272 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:08.320 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:53:08.320 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:53:08.320 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:53:08.320 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:53:08.321 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:08.322 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:53:08.323 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:53:08.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:08.324 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:53:08.339 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi037.stderr:2024-04-25T08:53:08.344+0000 7f76aaa71580 -1 init, newargv = 0x5640990d9fc0 newargc=15 2024-04-25T08:53:08.339 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi037.stderr:ceph-fuse[36148]: starting ceph client 2024-04-25T08:53:08.343 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:53:08.345 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi037.stderr:ceph-fuse[36148]: starting fuse 2024-04-25T08:53:09.444 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:53:09.461 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:09.510 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:53:09.510 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:53:09.510 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:53:09.510 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:53:09.510 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:53:09.511 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:09.512 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:53:09.513 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:53:09.513 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-25T08:53:09.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:09.513 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:53:09.556 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:53:09.556 INFO:teuthology.orchestra.run.smithi037.stdout:46 2024-04-25T08:53:09.557 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T08:53:09.557 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:53:09.557 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T08:53:09.557 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T08:53:09.557 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:53:09.557 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T08:53:09.557 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:53:09.557 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T08:53:09.557 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T08:53:09.557 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T08:53:09.557 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.2")) 2024-04-25T08:53:09.558 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:53:09.684 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.2.36148.asok 2024-04-25T08:53:09.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:09.697 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.36148.asok 2024-04-25T08:53:09.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:09.697 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.36148.asok status 2024-04-25T08:53:09.857 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:53:09.858 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T08:53:09.858 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:53:09.858 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:53:09.858 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "2", 2024-04-25T08:53:09.858 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T08:53:09.858 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T08:53:09.858 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "36148", 2024-04-25T08:53:09.858 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T08:53:09.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:09.858 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T08:53:09.858 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 4812, 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 4812 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2034022551 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:53:09.859 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:53:09.860 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2034022551 2024-04-25T08:53:09.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:09.860 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.4812 192.168.0.2:0/2034022551", 2024-04-25T08:53:09.860 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.2:0/2034022551", 2024-04-25T08:53:09.860 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T08:53:09.860 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 9, 2024-04-25T08:53:09.860 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 27, 2024-04-25T08:53:09.860 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:53:09.860 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T08:53:09.860 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T08:53:09.860 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:53:09.871 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:53:09.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.000+0000 7f6a549df640 1 Processor -- start 2024-04-25T08:53:09.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.000+0000 7f6a549df640 1 -- start start 2024-04-25T08:53:09.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a549df640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6a500b3f50 0x7f6a5015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:09.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a549df640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a500b2090 con 0x7f6a500b24a0 2024-04-25T08:53:09.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a549df640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a500b43e0 con 0x7f6a5015a700 2024-04-25T08:53:09.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a549df640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5015f200 con 0x7f6a500b3f50 2024-04-25T08:53:09.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4ed9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6a500b24a0 0x7f6a5006e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45602/0 (socket says 172.21.15.37:45602) 2024-04-25T08:53:09.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4ed9d640 1 -- 172.21.15.37:0/2815015682 learned_addr learned my addr 172.21.15.37:0/2815015682 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:09.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a47fff640 1 --2- 172.21.15.37:0/2815015682 >> v2:172.21.15.203:3300/0 conn(0x7f6a500b3f50 0x7f6a5015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:09.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1088428314 0 0) 0x7f6a500b2090 con 0x7f6a500b24a0 2024-04-25T08:53:09.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a3c003530 con 0x7f6a500b24a0 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3013649931 0 0) 0x7f6a3c003530 con 0x7f6a500b24a0 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 >> v2:172.21.15.203:3300/0 conn(0x7f6a500b3f50 msgr2=0x7f6a5015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 --2- 172.21.15.37:0/2815015682 >> v2:172.21.15.203:3300/0 conn(0x7f6a500b3f50 0x7f6a5015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 >> v1:172.21.15.203:6789/0 conn(0x7f6a5015a700 legacy=0x7f6a500cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a500b46c0 con 0x7f6a500b24a0 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f6a48001a20 con 0x7f6a500b24a0 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 >> v1:172.21.15.37:6789/0 conn(0x7f6a500b24a0 legacy=0x7f6a5006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 --2- 172.21.15.37:0/2815015682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c003a30 0x7f6a3c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 --2- 172.21.15.37:0/2815015682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a3c004810 0x7f6a3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 --2- 172.21.15.37:0/2815015682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 0x7f6a3c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5015f200 con 0x7f6a3c0052d0 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c00c210 con 0x7f6a3c003a30 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3c00c410 con 0x7f6a3c004810 2024-04-25T08:53:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4ed9d640 1 --2- 172.21.15.37:0/2815015682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 0x7f6a3c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4e59c640 1 --2- 172.21.15.37:0/2815015682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c003a30 0x7f6a3c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a47fff640 1 --2- 172.21.15.37:0/2815015682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a3c004810 0x7f6a3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4ed9d640 1 -- 172.21.15.37:0/2815015682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a3c004810 msgr2=0x7f6a3c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4ed9d640 1 --2- 172.21.15.37:0/2815015682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a3c004810 0x7f6a3c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4ed9d640 1 -- 172.21.15.37:0/2815015682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c003a30 msgr2=0x7f6a3c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4ed9d640 1 --2- 172.21.15.37:0/2815015682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a3c003a30 0x7f6a3c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.004+0000 7f6a4ed9d640 1 -- 172.21.15.37:0/2815015682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a3c003530 con 0x7f6a3c0052d0 2024-04-25T08:53:10.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a4ed9d640 1 -- 172.21.15.37:0/2815015682 reap_dead start 2024-04-25T08:53:10.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a4ed9d640 1 --2- 172.21.15.37:0/2815015682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 0x7f6a3c00b9d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800a830 tx=0x7f6a4801d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f26c3a797592a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:10.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4800a480 con 0x7f6a3c0052d0 2024-04-25T08:53:10.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a480ac640 con 0x7f6a3c0052d0 2024-04-25T08:53:10.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a4dd9b640 1 -- 172.21.15.37:0/2815015682 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a480aca30 con 0x7f6a3c0052d0 2024-04-25T08:53:10.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 -- 172.21.15.37:0/2815015682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 msgr2=0x7f6a3c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:10.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 --2- 172.21.15.37:0/2815015682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 0x7f6a3c00b9d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800a830 tx=0x7f6a4801d310 comp rx=0 tx=0).stop 2024-04-25T08:53:10.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 -- 172.21.15.37:0/2815015682 shutdown_connections 2024-04-25T08:53:10.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 --2- 172.21.15.37:0/2815015682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 0x7f6a3c00b9d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 -- 172.21.15.37:0/2815015682 >> 172.21.15.37:0/2815015682 conn(0x7f6a500b11a0 msgr2=0x7f6a500b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:10.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 -- 172.21.15.37:0/2815015682 shutdown_connections 2024-04-25T08:53:10.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 -- 172.21.15.37:0/2815015682 wait complete. 2024-04-25T08:53:10.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 Processor -- start 2024-04-25T08:53:10.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 -- start start 2024-04-25T08:53:10.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 0x7f6a500cb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:10.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a50148ef0 0x7f6a5014f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:10.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a5014fc40 0x7f6a5014cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:10.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500cd490 con 0x7f6a3c0052d0 2024-04-25T08:53:10.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500cd310 con 0x7f6a50148ef0 2024-04-25T08:53:10.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a549df640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5015ecc0 con 0x7f6a5014fc40 2024-04-25T08:53:10.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a4e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 0x7f6a500cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:10.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a4e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 0x7f6a500cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37690/0 (socket says 172.21.15.37:37690) 2024-04-25T08:53:10.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a4e59c640 1 -- 172.21.15.37:0/1773022582 learned_addr learned my addr 172.21.15.37:0/1773022582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:10.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a4ed9d640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a5014fc40 0x7f6a5014cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:10.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a47fff640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a50148ef0 0x7f6a5014f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:10.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a4e59c640 1 -- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a5014fc40 msgr2=0x7f6a5014cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:10.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.008+0000 7f6a4e59c640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a5014fc40 0x7f6a5014cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a4e59c640 1 -- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a50148ef0 msgr2=0x7f6a5014f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:10.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a4e59c640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a50148ef0 0x7f6a5014f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a4e59c640 1 -- 172.21.15.37:0/1773022582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a500cbe50 con 0x7f6a3c0052d0 2024-04-25T08:53:10.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a4e59c640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 0x7f6a500cb910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000f5f0 tx=0x7f6a4000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:10.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a46ffd640 1 -- 172.21.15.37:0/1773022582 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a400052e0 con 0x7f6a3c0052d0 2024-04-25T08:53:10.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a46ffd640 1 -- 172.21.15.37:0/1773022582 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a40018070 con 0x7f6a3c0052d0 2024-04-25T08:53:10.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a549df640 1 -- 172.21.15.37:0/1773022582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5014d9a0 con 0x7f6a3c0052d0 2024-04-25T08:53:10.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a46ffd640 1 -- 172.21.15.37:0/1773022582 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a40003820 con 0x7f6a3c0052d0 2024-04-25T08:53:10.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a549df640 1 -- 172.21.15.37:0/1773022582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5014df60 con 0x7f6a3c0052d0 2024-04-25T08:53:10.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a549df640 1 -- 172.21.15.37:0/1773022582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5004efa0 con 0x7f6a3c0052d0 2024-04-25T08:53:10.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a46ffd640 1 -- 172.21.15.37:0/1773022582 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a400039c0 con 0x7f6a3c0052d0 2024-04-25T08:53:10.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a46ffd640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a30066f60 0x7f6a30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:10.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a46ffd640 1 -- 172.21.15.37:0/1773022582 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6a40084fc0 con 0x7f6a3c0052d0 2024-04-25T08:53:10.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a47fff640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a30066f60 0x7f6a30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:10.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.012+0000 7f6a47fff640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a30066f60 0x7f6a30069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a5006eba0 tx=0x7f6a38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:10.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.016+0000 7f6a46ffd640 1 -- 172.21.15.37:0/1773022582 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a40055d20 con 0x7f6a3c0052d0 2024-04-25T08:53:10.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.212+0000 7f6a549df640 1 -- 172.21.15.37:0/1773022582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6a5006f260 con 0x7f6a3c0052d0 2024-04-25T08:53:10.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.212+0000 7f6a46ffd640 1 -- 172.21.15.37:0/1773022582 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f6a40005480 con 0x7f6a3c0052d0 2024-04-25T08:53:10.209 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:53:10.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 -- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a30066f60 msgr2=0x7f6a30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:10.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a30066f60 0x7f6a30069420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6a5006eba0 tx=0x7f6a38008040 comp rx=0 tx=0).stop 2024-04-25T08:53:10.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 -- 172.21.15.37:0/1773022582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 msgr2=0x7f6a500cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:10.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 0x7f6a500cb910 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6a4000f5f0 tx=0x7f6a4000fab0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 -- 172.21.15.37:0/1773022582 shutdown_connections 2024-04-25T08:53:10.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a30066f60 0x7f6a30069420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a5014fc40 0x7f6a5014cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a50148ef0 0x7f6a5014f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 --2- 172.21.15.37:0/1773022582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a3c0052d0 0x7f6a500cb910 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:10.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 -- 172.21.15.37:0/1773022582 >> 172.21.15.37:0/1773022582 conn(0x7f6a500b11a0 msgr2=0x7f6a5015e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:10.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 -- 172.21.15.37:0/1773022582 shutdown_connections 2024-04-25T08:53:10.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:10.216+0000 7f6a549df640 1 -- 172.21.15.37:0/1773022582 wait complete. 2024-04-25T08:53:10.226 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:53:10.226 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:53:10.226 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-25T08:53:10.226 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T08:53:10.226 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:10.226 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:53:10.226 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:53:10.226 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:53:10.226 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:53:10.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:10.226 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-25T08:53:10.234 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:53:10.234 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:53:10.234 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:53:10.234 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:10.234 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-25T08:53:10.234 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:10.234 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:10.234 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:10.234 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-25T08:53:10.234 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:10.234 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42465sec preferred_lft 42465sec 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 02:42:77:67:cf:bd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:53:10.235 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:10.236 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:53:10.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:10.236 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2024-04-25T08:53:10.236 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T08:53:10.236 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:53:10.236 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr flush dev ceph-brx 2024-04-25T08:53:10.236 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set ceph-brx up 2024-04-25T08:53:10.236 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:53:10.236 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T08:53:10.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:10.318 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:53:10.368 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-25T08:53:10.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:10.370 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-25T08:53:10.442 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-25T08:53:10.442 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:53:10.443 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T08:53:10.443 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:53:10.443 INFO:teuthology.orchestra.run.smithi203.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T08:53:10.443 INFO:teuthology.orchestra.run.smithi203.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:53:10.443 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:53:10.443 INFO:teuthology.orchestra.run.smithi203.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:53:10.443 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:53:10.443 INFO:teuthology.orchestra.run.smithi203.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:53:10.443 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:53:10.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:10.444 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2024-04-25T08:53:10.444 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T08:53:10.444 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:53:10.444 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:53:10.444 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:53:10.444 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T08:53:10.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:10.518 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-25T08:53:10.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:10.568 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2024-04-25T08:53:10.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:10.615 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2024-04-25T08:53:10.615 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T08:53:10.615 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:10.615 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T08:53:10.615 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T08:53:10.689 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T08:53:10.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:10.689 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2024-04-25T08:53:10.689 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T08:53:10.689 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T08:53:10.690 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:53:10.690 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T08:53:10.690 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T08:53:10.690 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T08:53:10.690 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T08:53:10.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:10.807 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2024-04-25T08:53:10.807 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T08:53:10.807 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.0 up 2024-04-25T08:53:10.807 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:53:10.807 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T08:53:10.826 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T08:53:10.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:53:10.826 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:10.871 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:53:10.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:53:10.871 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:10.919 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-04-25T08:53:10.975 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:11.024 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-04-25T08:53:11.024 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-04-25T08:53:11.024 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-04-25T08:53:11.024 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-04-25T08:53:11.024 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-04-25T08:53:11.024 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-04-25T08:53:11.024 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-04-25T08:53:11.024 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-04-25T08:53:11.024 INFO:teuthology.orchestra.run.smithi203.stdout:/run/lock 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/fs/nfsd 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/run/rpc_pipefs 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-04-25T08:53:11.025 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-04-25T08:53:11.026 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:53:11.026 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:53:11.026 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:53:11.026 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:53:11.026 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-04-25T08:53:11.026 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:11.026 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:11.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:11.027 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-04-25T08:53:11.076 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:53:11.076 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T08:53:11.120 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-04-25T08:53:11.134 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:11.184 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-04-25T08:53:11.184 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-04-25T08:53:11.184 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-04-25T08:53:11.184 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-04-25T08:53:11.184 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-04-25T08:53:11.184 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-04-25T08:53:11.184 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-04-25T08:53:11.184 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-04-25T08:53:11.184 INFO:teuthology.orchestra.run.smithi203.stdout:/run/lock 2024-04-25T08:53:11.184 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/fs/nfsd 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/run/rpc_pipefs 2024-04-25T08:53:11.185 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-04-25T08:53:11.186 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-04-25T08:53:11.186 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:53:11.186 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:53:11.186 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:53:11.186 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:53:11.186 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-04-25T08:53:11.186 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:11.186 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:11.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:11.187 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-04-25T08:53:11.201 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:2024-04-25T08:53:11.208+0000 7f4a91c9f580 -1 init, newargv = 0x560373c36f10 newargc=15 2024-04-25T08:53:11.201 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:ceph-fuse[33862]: starting ceph client 2024-04-25T08:53:11.218 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi203.stderr:ceph-fuse[33862]: starting fuse 2024-04-25T08:53:12.308 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-04-25T08:53:12.324 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:12.374 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-04-25T08:53:12.374 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-04-25T08:53:12.374 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-04-25T08:53:12.374 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-04-25T08:53:12.374 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-04-25T08:53:12.374 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-04-25T08:53:12.374 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-04-25T08:53:12.374 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/run/lock 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/fs/nfsd 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:12.375 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/run/rpc_pipefs 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:12.376 INFO:teuthology.orchestra.run.smithi203.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:12.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:12.377 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-04-25T08:53:12.423 INFO:teuthology.orchestra.run.smithi203.stdout:45 2024-04-25T08:53:12.424 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:53:12.424 DEBUG:teuthology.orchestra.run.smithi203:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:53:12.424 DEBUG:teuthology.orchestra.run.smithi203:> import glob 2024-04-25T08:53:12.424 DEBUG:teuthology.orchestra.run.smithi203:> import re 2024-04-25T08:53:12.424 DEBUG:teuthology.orchestra.run.smithi203:> import os 2024-04-25T08:53:12.424 DEBUG:teuthology.orchestra.run.smithi203:> import subprocess 2024-04-25T08:53:12.424 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T08:53:12.424 DEBUG:teuthology.orchestra.run.smithi203:> def _find_admin_socket(client_name): 2024-04-25T08:53:12.424 DEBUG:teuthology.orchestra.run.smithi203:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T08:53:12.424 DEBUG:teuthology.orchestra.run.smithi203:> files = glob.glob(asok_path) 2024-04-25T08:53:12.424 DEBUG:teuthology.orchestra.run.smithi203:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> # Given a non-glob path, it better be there 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> if "*" not in asok_path: 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> assert(len(files) == 1) 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> return files[0] 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> for f in files: 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> contents = proc_f.read() 2024-04-25T08:53:12.425 DEBUG:teuthology.orchestra.run.smithi203:> if mountpoint in contents: 2024-04-25T08:53:12.426 DEBUG:teuthology.orchestra.run.smithi203:> return f 2024-04-25T08:53:12.426 DEBUG:teuthology.orchestra.run.smithi203:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:53:12.426 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T08:53:12.426 DEBUG:teuthology.orchestra.run.smithi203:> print(_find_admin_socket("client.1")) 2024-04-25T08:53:12.426 DEBUG:teuthology.orchestra.run.smithi203:> ' 2024-04-25T08:53:12.549 INFO:teuthology.orchestra.run.smithi203.stdout:/var/run/ceph/ceph-client.1.33862.asok 2024-04-25T08:53:12.553 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:53:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:12.561 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.33862.asok 2024-04-25T08:53:12.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:12.562 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.33862.asok status 2024-04-25T08:53:12.708 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-04-25T08:53:12.708 INFO:teuthology.orchestra.run.smithi203.stdout: "metadata": { 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "entity_id": "1", 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "hostname": "smithi203", 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "pid": "33862", 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "root": "/" 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_count": 0, 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_pinned_count": 0, 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "id": 4505, 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "inst": { 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "name": { 2024-04-25T08:53:12.709 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "client", 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: "num": 4505 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 2856856581 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 2856856581 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T08:53:12.710 INFO:teuthology.orchestra.run.smithi203.stdout: "inst_str": "client.4505 192.168.0.1:0/2856856581", 2024-04-25T08:53:12.711 INFO:teuthology.orchestra.run.smithi203.stdout: "addr_str": "192.168.0.1:0/2856856581", 2024-04-25T08:53:12.711 INFO:teuthology.orchestra.run.smithi203.stdout: "inode_count": 1, 2024-04-25T08:53:12.711 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_epoch": 9, 2024-04-25T08:53:12.711 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch": 28, 2024-04-25T08:53:12.711 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:53:12.711 INFO:teuthology.orchestra.run.smithi203.stdout: "blocklisted": false, 2024-04-25T08:53:12.711 INFO:teuthology.orchestra.run.smithi203.stdout: "fs_name": "cephfs" 2024-04-25T08:53:12.711 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-04-25T08:53:12.722 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:53:12.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.860+0000 7f59bea44640 1 Processor -- start 2024-04-25T08:53:12.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59bea44640 1 -- start start 2024-04-25T08:53:12.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59bea44640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f59b815da80 0x7f59b80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59bea44640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f59b80d07f0 0x7f59b806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59bea44640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b806be60 con 0x7f59b80d1cd0 2024-04-25T08:53:12.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59bea44640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b806c440 con 0x7f59b815da80 2024-04-25T08:53:12.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59bea44640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b815de60 con 0x7f59b80d07f0 2024-04-25T08:53:12.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59b7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f59b815da80 0x7f59b80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59b77fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f59b80d1cd0 0x7f59b80d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45608/0 (socket says 172.21.15.37:45608) 2024-04-25T08:53:12.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59b77fe640 1 -- 172.21.15.37:0/4149090170 learned_addr learned my addr 172.21.15.37:0/4149090170 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:12.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59bcfe1640 1 --2- 172.21.15.37:0/4149090170 >> v2:172.21.15.203:3300/0 conn(0x7f59b80d07f0 0x7f59b806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59b7fff640 1 -- 172.21.15.37:0/4149090170 >> v2:172.21.15.203:3300/0 conn(0x7f59b80d07f0 msgr2=0x7f59b806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:12.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59b7fff640 1 --2- 172.21.15.37:0/4149090170 >> v2:172.21.15.203:3300/0 conn(0x7f59b80d07f0 0x7f59b806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59b7fff640 1 -- 172.21.15.37:0/4149090170 >> v1:172.21.15.37:6789/0 conn(0x7f59b80d1cd0 legacy=0x7f59b80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59b6ffd640 1 -- 172.21.15.37:0/4149090170 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1665590391 0 0) 0x7f59b806be60 con 0x7f59b80d1cd0 2024-04-25T08:53:12.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.864+0000 7f59b7fff640 1 -- 172.21.15.37:0/4149090170 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b80d2450 con 0x7f59b815da80 2024-04-25T08:53:12.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b7fff640 1 --2- 172.21.15.37:0/4149090170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b815da80 0x7f59b80d4230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f59a00097c0 tx=0x7f59a001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e48ca19a0762aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:12.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b6ffd640 1 -- 172.21.15.37:0/4149090170 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a00b1070 con 0x7f59b815da80 2024-04-25T08:53:12.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b6ffd640 1 -- 172.21.15.37:0/4149090170 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a00ac5f0 con 0x7f59b815da80 2024-04-25T08:53:12.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b6ffd640 1 -- 172.21.15.37:0/4149090170 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a00ac9c0 con 0x7f59b815da80 2024-04-25T08:53:12.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 -- 172.21.15.37:0/4149090170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b815da80 msgr2=0x7f59b80d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 --2- 172.21.15.37:0/4149090170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b815da80 0x7f59b80d4230 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f59a00097c0 tx=0x7f59a001c430 comp rx=0 tx=0).stop 2024-04-25T08:53:12.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 -- 172.21.15.37:0/4149090170 shutdown_connections 2024-04-25T08:53:12.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 --2- 172.21.15.37:0/4149090170 >> v2:172.21.15.203:3300/0 conn(0x7f59b80d07f0 0x7f59b806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 --2- 172.21.15.37:0/4149090170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b815da80 0x7f59b80d4230 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 -- 172.21.15.37:0/4149090170 >> 172.21.15.37:0/4149090170 conn(0x7f59b808b000 msgr2=0x7f59b80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:12.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 -- 172.21.15.37:0/4149090170 shutdown_connections 2024-04-25T08:53:12.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 -- 172.21.15.37:0/4149090170 wait complete. 2024-04-25T08:53:12.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 Processor -- start 2024-04-25T08:53:12.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 -- start start 2024-04-25T08:53:12.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b80d07f0 0x7f59b80c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b80d1cd0 0x7f59b815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b815da80 0x7f59b81569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b806d7a0 con 0x7f59b80d07f0 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b806d620 con 0x7f59b815da80 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59bea44640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b806d920 con 0x7f59b80d1cd0 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b80d07f0 0x7f59b80c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b80d07f0 0x7f59b80c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37704/0 (socket says 172.21.15.37:37704) 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b7fff640 1 -- 172.21.15.37:0/971262229 learned_addr learned my addr 172.21.15.37:0/971262229 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b7fff640 1 -- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b80d1cd0 msgr2=0x7f59b815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b7fff640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b80d1cd0 0x7f59b815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b7fff640 1 -- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b815da80 msgr2=0x7f59b81569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b7fff640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b815da80 0x7f59b81569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b7fff640 1 -- 172.21.15.37:0/971262229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b80c8da0 con 0x7f59b80d07f0 2024-04-25T08:53:12.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b7fff640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b80d07f0 0x7f59b80c8860 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f59a00af040 tx=0x7f59a00191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:12.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b4ff9640 1 -- 172.21.15.37:0/971262229 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a00bc070 con 0x7f59b80d07f0 2024-04-25T08:53:12.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.868+0000 7f59b4ff9640 1 -- 172.21.15.37:0/971262229 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a0004030 con 0x7f59b80d07f0 2024-04-25T08:53:12.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.872+0000 7f59bea44640 1 -- 172.21.15.37:0/971262229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b80c9040 con 0x7f59b80d07f0 2024-04-25T08:53:12.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.872+0000 7f59b4ff9640 1 -- 172.21.15.37:0/971262229 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a00b1040 con 0x7f59b80d07f0 2024-04-25T08:53:12.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.872+0000 7f59bea44640 1 -- 172.21.15.37:0/971262229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b80c9600 con 0x7f59b80d07f0 2024-04-25T08:53:12.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.872+0000 7f59bea44640 1 -- 172.21.15.37:0/971262229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b804efa0 con 0x7f59b80d07f0 2024-04-25T08:53:12.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.872+0000 7f59b4ff9640 1 -- 172.21.15.37:0/971262229 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59a00bf070 con 0x7f59b80d07f0 2024-04-25T08:53:12.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.872+0000 7f59b4ff9640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5998066f60 0x7f5998069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:12.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.872+0000 7f59b4ff9640 1 -- 172.21.15.37:0/971262229 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f59a011e5c0 con 0x7f59b80d07f0 2024-04-25T08:53:12.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.872+0000 7f59b77fe640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5998066f60 0x7f5998069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:12.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.872+0000 7f59b77fe640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5998066f60 0x7f5998069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f59a80010f0 tx=0x7f59a80077d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:12.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:12.876+0000 7f59b4ff9640 1 -- 172.21.15.37:0/971262229 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59a00a9070 con 0x7f59b80d07f0 2024-04-25T08:53:13.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 -- 172.21.15.37:0/971262229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f59b8158790 con 0x7f59b80d07f0 2024-04-25T08:53:13.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59b4ff9640 1 -- 172.21.15.37:0/971262229 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f59a0016050 con 0x7f59b80d07f0 2024-04-25T08:53:13.069 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T08:53:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 -- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5998066f60 msgr2=0x7f5998069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5998066f60 0x7f5998069420 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f59a80010f0 tx=0x7f59a80077d0 comp rx=0 tx=0).stop 2024-04-25T08:53:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 -- 172.21.15.37:0/971262229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b80d07f0 msgr2=0x7f59b80c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b80d07f0 0x7f59b80c8860 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f59a00af040 tx=0x7f59a00191d0 comp rx=0 tx=0).stop 2024-04-25T08:53:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 -- 172.21.15.37:0/971262229 shutdown_connections 2024-04-25T08:53:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5998066f60 0x7f5998069420 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f59b815da80 0x7f59b81569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59b80d1cd0 0x7f59b815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 --2- 172.21.15.37:0/971262229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59b80d07f0 0x7f59b80c8860 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 -- 172.21.15.37:0/971262229 >> 172.21.15.37:0/971262229 conn(0x7f59b808b000 msgr2=0x7f59b80ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:13.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 -- 172.21.15.37:0/971262229 shutdown_connections 2024-04-25T08:53:13.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:13.076+0000 7f59bea44640 1 -- 172.21.15.37:0/971262229 wait complete. 2024-04-25T08:53:13.088 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:53:13.088 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:53:13.088 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi203.front.sepia.ceph.com 2024-04-25T08:53:13.088 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T08:53:13.088 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:53:13.088 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:53:13.088 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:53:13.088 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:53:13.088 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:53:13.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.088 DEBUG:teuthology.orchestra.run.smithi203:> ip addr 2024-04-25T08:53:13.095 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:53:13.095 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:53:13.095 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:53:13.095 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:44 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f0 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:d9:19:45 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout: altname enp7s0f1 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:22 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0c:c4:7a:8f:4a:23 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.21.15.203/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft 42462sec preferred_lft 42462sec 2024-04-25T08:53:13.096 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::ec4:7aff:fe8f:4a23/64 scope link 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 02:42:77:67:cf:bd brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:53:13.097 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.098 DEBUG:teuthology.orchestra.run.smithi203:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:53:13.153 INFO:teuthology.orchestra.run.smithi203.stdout:1 2024-04-25T08:53:13.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.155 DEBUG:teuthology.orchestra.run.smithi203:> route 2024-04-25T08:53:13.208 INFO:teuthology.orchestra.run.smithi203.stdout:Kernel IP routing table 2024-04-25T08:53:13.208 INFO:teuthology.orchestra.run.smithi203.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:53:13.208 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T08:53:13.208 INFO:teuthology.orchestra.run.smithi203.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:53:13.208 INFO:teuthology.orchestra.run.smithi203.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T08:53:13.208 INFO:teuthology.orchestra.run.smithi203.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:53:13.208 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:53:13.208 INFO:teuthology.orchestra.run.smithi203.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:53:13.208 INFO:teuthology.orchestra.run.smithi203.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:53:13.208 INFO:teuthology.orchestra.run.smithi203.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T08:53:13.208 INFO:teuthology.orchestra.run.smithi203.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:53:13.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.209 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2024-04-25T08:53:13.209 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T08:53:13.209 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:53:13.209 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:53:13.209 DEBUG:teuthology.orchestra.run.smithi203:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:53:13.209 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T08:53:13.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.286 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-25T08:53:13.331 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T08:53:13.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.332 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list-id 2024-04-25T08:53:13.384 INFO:teuthology.orchestra.run.smithi203.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T08:53:13.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.384 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2024-04-25T08:53:13.384 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T08:53:13.384 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:53:13.384 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T08:53:13.384 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T08:53:13.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.454 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:53:13.508 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:53:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:53:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:53:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-25T08:53:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.508 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:53:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 8a:33:ff:02:72:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:53:13.508 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:53:13.509 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.509 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::8833:ffff:fe02:720f/64 scope link 2024-04-25T08:53:13.509 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.510 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:53:13.563 INFO:teuthology.orchestra.run.smithi203.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:53:13.563 INFO:teuthology.orchestra.run.smithi203.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:53:13.563 INFO:teuthology.orchestra.run.smithi203.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:53:13.563 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.563 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 ::1/128 scope host 2024-04-25T08:53:13.563 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.563 INFO:teuthology.orchestra.run.smithi203.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:53:13.564 INFO:teuthology.orchestra.run.smithi203.stdout: link/ether 8a:33:ff:02:72:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:53:13.564 INFO:teuthology.orchestra.run.smithi203.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:53:13.564 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.564 INFO:teuthology.orchestra.run.smithi203.stdout: inet6 fe80::8833:ffff:fe02:720f/64 scope link 2024-04-25T08:53:13.564 INFO:teuthology.orchestra.run.smithi203.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:53:13.565 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T08:53:13.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.566 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2024-04-25T08:53:13.566 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T08:53:13.566 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T08:53:13.566 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:53:13.566 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T08:53:13.566 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T08:53:13.567 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T08:53:13.567 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T08:53:13.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.663 DEBUG:teuthology.orchestra.run.smithi203:> (cd / && exec bash -c ' 2024-04-25T08:53:13.663 DEBUG:teuthology.orchestra.run.smithi203:> set -e 2024-04-25T08:53:13.663 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set brx.1 up 2024-04-25T08:53:13.663 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:53:13.663 DEBUG:teuthology.orchestra.run.smithi203:> ') 2024-04-25T08:53:13.683 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T08:53:13.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:53:13.684 DEBUG:teuthology.orchestra.run.smithi203:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T08:53:13.731 INFO:teuthology.orchestra.run.smithi203.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:53:13.731 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:53:13.731 DEBUG:teuthology.orchestra.run.smithi203:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:53:13.779 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-04-25T08:53:13.838 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:13.885 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-04-25T08:53:13.886 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-04-25T08:53:13.886 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-04-25T08:53:13.886 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-04-25T08:53:13.886 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-04-25T08:53:13.886 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-04-25T08:53:13.886 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-04-25T08:53:13.886 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/run/lock 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/fs/nfsd 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/run/rpc_pipefs 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-04-25T08:53:13.887 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-04-25T08:53:13.888 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:53:13.888 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:53:13.888 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:53:13.888 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:53:13.888 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-04-25T08:53:13.888 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:13.888 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:13.888 INFO:teuthology.orchestra.run.smithi203.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:13.888 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:53:13.888 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:53:13.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:13.889 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-04-25T08:53:13.939 INFO:teuthology.orchestra.run.smithi203.stdout:45 2024-04-25T08:53:13.939 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T08:53:13.940 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T08:53:13.984 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-04-25T08:53:13.998 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:14.046 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-04-25T08:53:14.046 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/run/lock 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-04-25T08:53:14.047 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/fs/nfsd 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/run/rpc_pipefs 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:14.048 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:14.049 INFO:teuthology.orchestra.run.smithi203.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:14.049 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:53:14.049 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:53:14.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:14.050 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-04-25T08:53:14.061 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi203.stderr:2024-04-25T08:53:14.068+0000 7f1268065580 -1 init, newargv = 0x55814fd96c00 newargc=15 2024-04-25T08:53:14.062 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi203.stderr:ceph-fuse[33978]: starting ceph client 2024-04-25T08:53:14.065 INFO:teuthology.orchestra.run.smithi203.stdout:45 2024-04-25T08:53:14.069 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi203.stderr:ceph-fuse[33978]: starting fuse 2024-04-25T08:53:15.167 DEBUG:teuthology.orchestra.run.smithi203:> sudo modprobe fuse 2024-04-25T08:53:15.182 DEBUG:teuthology.orchestra.run.smithi203:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:53:15.234 INFO:teuthology.orchestra.run.smithi203.stdout:/sys 2024-04-25T08:53:15.234 INFO:teuthology.orchestra.run.smithi203.stdout:/proc 2024-04-25T08:53:15.234 INFO:teuthology.orchestra.run.smithi203.stdout:/dev 2024-04-25T08:53:15.234 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/pts 2024-04-25T08:53:15.234 INFO:teuthology.orchestra.run.smithi203.stdout:/run 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/ 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/security 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/shm 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/run/lock 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/cgroup 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/pstore 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/bpf 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/hugepages 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/dev/mqueue 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/debug 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/tracing 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/proc/fs/nfsd 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/fs/fuse/connections 2024-04-25T08:53:15.235 INFO:teuthology.orchestra.run.smithi203.stdout:/sys/kernel/config 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/run/rpc_pipefs 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/run/user/1000 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:53:15.236 INFO:teuthology.orchestra.run.smithi203.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-25T08:53:15.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:15.237 DEBUG:teuthology.orchestra.run.smithi203:> ls /sys/fs/fuse/connections 2024-04-25T08:53:15.284 INFO:teuthology.orchestra.run.smithi203.stdout:45 2024-04-25T08:53:15.284 INFO:teuthology.orchestra.run.smithi203.stdout:46 2024-04-25T08:53:15.284 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T08:53:15.284 DEBUG:teuthology.orchestra.run.smithi203:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:53:15.284 DEBUG:teuthology.orchestra.run.smithi203:> import glob 2024-04-25T08:53:15.284 DEBUG:teuthology.orchestra.run.smithi203:> import re 2024-04-25T08:53:15.284 DEBUG:teuthology.orchestra.run.smithi203:> import os 2024-04-25T08:53:15.284 DEBUG:teuthology.orchestra.run.smithi203:> import subprocess 2024-04-25T08:53:15.284 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> def _find_admin_socket(client_name): 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> files = glob.glob(asok_path) 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> # Given a non-glob path, it better be there 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> if "*" not in asok_path: 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> assert(len(files) == 1) 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> return files[0] 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> for f in files: 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> contents = proc_f.read() 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> if mountpoint in contents: 2024-04-25T08:53:15.285 DEBUG:teuthology.orchestra.run.smithi203:> return f 2024-04-25T08:53:15.286 DEBUG:teuthology.orchestra.run.smithi203:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:53:15.286 DEBUG:teuthology.orchestra.run.smithi203:> 2024-04-25T08:53:15.286 DEBUG:teuthology.orchestra.run.smithi203:> print(_find_admin_socket("client.3")) 2024-04-25T08:53:15.286 DEBUG:teuthology.orchestra.run.smithi203:> ' 2024-04-25T08:53:15.407 INFO:teuthology.orchestra.run.smithi203.stdout:/var/run/ceph/ceph-client.3.33978.asok 2024-04-25T08:53:15.411 INFO:teuthology.orchestra.run.smithi203.stderr:2024-04-25T08:53:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:53:15.420 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.33978.asok 2024-04-25T08:53:15.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:15.420 DEBUG:teuthology.orchestra.run.smithi203:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.33978.asok status 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout:{ 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout: "metadata": { 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout: "entity_id": "3", 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout: "hostname": "smithi203", 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout: "pid": "33978", 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout: "root": "/" 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_count": 0, 2024-04-25T08:53:15.577 INFO:teuthology.orchestra.run.smithi203.stdout: "dentry_pinned_count": 0, 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: "id": 4669, 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: "inst": { 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: "name": { 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "client", 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: "num": 4669 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 3117754030 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: } 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": { 2024-04-25T08:53:15.578 INFO:teuthology.orchestra.run.smithi203.stdout: "type": "v1", 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout: "nonce": 3117754030 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout: }, 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout: "inst_str": "client.4669 192.168.0.2:0/3117754030", 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout: "addr_str": "192.168.0.2:0/3117754030", 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout: "inode_count": 1, 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout: "mds_epoch": 9, 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch": 29, 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout: "blocklisted": false, 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout: "fs_name": "cephfs" 2024-04-25T08:53:15.579 INFO:teuthology.orchestra.run.smithi203.stdout:} 2024-04-25T08:53:15.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:15.590 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T08:53:15.590 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:15.598 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T08:53:15.598 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:15.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:15.598 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:15.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:15.653 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T08:53:15.654 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T08:53:15.702 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T08:53:15.702 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T08:53:15.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:15.702 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:53:15.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:15.757 DEBUG:teuthology.orchestra.run.smithi203:> stat --file-system '--printf=%T 2024-04-25T08:53:15.757 DEBUG:teuthology.orchestra.run.smithi203:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:15.764 INFO:teuthology.orchestra.run.smithi203.stdout:fuseblk 2024-04-25T08:53:15.764 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:15.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:15.764 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:15.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:15.821 DEBUG:teuthology.orchestra.run.smithi203:> stat --file-system '--printf=%T 2024-04-25T08:53:15.821 DEBUG:teuthology.orchestra.run.smithi203:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T08:53:15.869 INFO:teuthology.orchestra.run.smithi203.stdout:fuseblk 2024-04-25T08:53:15.869 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T08:53:15.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:15.869 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:53:15.924 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T08:53:15.941 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:16.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b767fc640 1 Processor -- start 2024-04-25T08:53:16.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b767fc640 1 -- start start 2024-04-25T08:53:16.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b767fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3b70066e30 0x7f3b7006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b767fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3b700af370 0x7f3b700ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b767fc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7006ec30 con 0x7f3b7006bca0 2024-04-25T08:53:16.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b767fc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b700669a0 con 0x7f3b70066e30 2024-04-25T08:53:16.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b767fc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7006c5c0 con 0x7f3b700af370 2024-04-25T08:53:16.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b6f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3b70066e30 0x7f3b7006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b6f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3b70066e30 0x7f3b7006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37706/0 (socket says 172.21.15.37:37706) 2024-04-25T08:53:16.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b6f7fe640 1 -- 172.21.15.37:0/2907841803 learned_addr learned my addr 172.21.15.37:0/2907841803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:16.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b74d99640 1 --2- 172.21.15.37:0/2907841803 >> v2:172.21.15.203:3300/0 conn(0x7f3b700af370 0x7f3b700ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b6f7fe640 1 -- 172.21.15.37:0/2907841803 >> v2:172.21.15.203:3300/0 conn(0x7f3b700af370 msgr2=0x7f3b700ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:16.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b6f7fe640 1 --2- 172.21.15.37:0/2907841803 >> v2:172.21.15.203:3300/0 conn(0x7f3b700af370 0x7f3b700ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b6f7fe640 1 -- 172.21.15.37:0/2907841803 >> v1:172.21.15.203:6789/0 conn(0x7f3b7006bca0 legacy=0x7f3b700cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:16.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.084+0000 7f3b6f7fe640 1 -- 172.21.15.37:0/2907841803 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7006f230 con 0x7f3b70066e30 2024-04-25T08:53:16.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b6f7fe640 1 --2- 172.21.15.37:0/2907841803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b70066e30 0x7f3b7006c080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3b7006d160 tx=0x7f3b5801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5159c1ab720e181e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b6effd640 1 -- 172.21.15.37:0/2907841803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b580b1070 con 0x7f3b70066e30 2024-04-25T08:53:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b6effd640 1 -- 172.21.15.37:0/2907841803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b580ac410 con 0x7f3b70066e30 2024-04-25T08:53:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b6effd640 1 -- 172.21.15.37:0/2907841803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b580ac7e0 con 0x7f3b70066e30 2024-04-25T08:53:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b767fc640 1 -- 172.21.15.37:0/2907841803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b70066e30 msgr2=0x7f3b7006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b767fc640 1 --2- 172.21.15.37:0/2907841803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b70066e30 0x7f3b7006c080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3b7006d160 tx=0x7f3b5801c390 comp rx=0 tx=0).stop 2024-04-25T08:53:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b767fc640 1 -- 172.21.15.37:0/2907841803 shutdown_connections 2024-04-25T08:53:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b767fc640 1 --2- 172.21.15.37:0/2907841803 >> v2:172.21.15.203:3300/0 conn(0x7f3b700af370 0x7f3b700ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b767fc640 1 --2- 172.21.15.37:0/2907841803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b70066e30 0x7f3b7006c080 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b767fc640 1 -- 172.21.15.37:0/2907841803 >> 172.21.15.37:0/2907841803 conn(0x7f3b70071810 msgr2=0x7f3b70071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:16.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b767fc640 1 -- 172.21.15.37:0/2907841803 shutdown_connections 2024-04-25T08:53:16.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b767fc640 1 -- 172.21.15.37:0/2907841803 wait complete. 2024-04-25T08:53:16.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b767fc640 1 Processor -- start 2024-04-25T08:53:16.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b767fc640 1 -- start start 2024-04-25T08:53:16.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.088+0000 7f3b767fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b70066e30 0x7f3b7014b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b767fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7006bca0 0x7f3b70155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b767fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b700af370 0x7f3b70155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b767fc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b701687b0 con 0x7f3b7006bca0 2024-04-25T08:53:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b767fc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b70168630 con 0x7f3b700af370 2024-04-25T08:53:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b767fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b70168930 con 0x7f3b70066e30 2024-04-25T08:53:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7006bca0 0x7f3b70155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7006bca0 0x7f3b70155550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37722/0 (socket says 172.21.15.37:37722) 2024-04-25T08:53:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6f7fe640 1 -- 172.21.15.37:0/1907521310 learned_addr learned my addr 172.21.15.37:0/1907521310 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6f7fe640 1 -- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b70066e30 msgr2=0x7f3b7014b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b74d99640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b700af370 0x7f3b70155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6f7fe640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b70066e30 0x7f3b7014b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6f7fe640 1 -- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b700af370 msgr2=0x7f3b70155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6f7fe640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b700af370 0x7f3b70155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6f7fe640 1 -- 172.21.15.37:0/1907521310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7014bdd0 con 0x7f3b7006bca0 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6f7fe640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7006bca0 0x7f3b70155550 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3b5801aa40 tx=0x7f3b58019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6cff9640 1 -- 172.21.15.37:0/1907521310 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b580b1040 con 0x7f3b7006bca0 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6cff9640 1 -- 172.21.15.37:0/1907521310 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b580ba070 con 0x7f3b7006bca0 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b767fc640 1 -- 172.21.15.37:0/1907521310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7014c0b0 con 0x7f3b7006bca0 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6cff9640 1 -- 172.21.15.37:0/1907521310 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b580b5440 con 0x7f3b7006bca0 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b767fc640 1 -- 172.21.15.37:0/1907521310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b70156970 con 0x7f3b7006bca0 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b767fc640 1 -- 172.21.15.37:0/1907521310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7004efa0 con 0x7f3b7006bca0 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6cff9640 1 -- 172.21.15.37:0/1907521310 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3b580b55e0 con 0x7f3b7006bca0 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6cff9640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3b50066f60 0x7f3b50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6cff9640 1 -- 172.21.15.37:0/1907521310 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3b5811eec0 con 0x7f3b7006bca0 2024-04-25T08:53:16.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.092+0000 7f3b6ffff640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3b50066f60 0x7f3b50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.096+0000 7f3b6ffff640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3b50066f60 0x7f3b50069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009670 tx=0x7f3b68006bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.096+0000 7f3b6cff9640 1 -- 172.21.15.37:0/1907521310 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3b580160d0 con 0x7f3b7006bca0 2024-04-25T08:53:16.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.260+0000 7f3b767fc640 1 -- 172.21.15.37:0/1907521310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b70072400 con 0x7f3b7006bca0 2024-04-25T08:53:16.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b6cff9640 1 -- 172.21.15.37:0/1907521310 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f3b580efbf0 con 0x7f3b7006bca0 2024-04-25T08:53:16.258 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:16.258 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 -- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3b50066f60 msgr2=0x7f3b50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3b50066f60 0x7f3b50069420 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009670 tx=0x7f3b68006bd0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 -- 172.21.15.37:0/1907521310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7006bca0 msgr2=0x7f3b70155550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7006bca0 0x7f3b70155550 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3b5801aa40 tx=0x7f3b58019130 comp rx=0 tx=0).stop 2024-04-25T08:53:16.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 -- 172.21.15.37:0/1907521310 shutdown_connections 2024-04-25T08:53:16.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3b50066f60 0x7f3b50069420 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3b700af370 0x7f3b70155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b7006bca0 0x7f3b70155550 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 --2- 172.21.15.37:0/1907521310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3b70066e30 0x7f3b7014b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 -- 172.21.15.37:0/1907521310 >> 172.21.15.37:0/1907521310 conn(0x7f3b70071810 msgr2=0x7f3b70159020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:16.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 -- 172.21.15.37:0/1907521310 shutdown_connections 2024-04-25T08:53:16.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.264+0000 7f3b767fc640 1 -- 172.21.15.37:0/1907521310 wait complete. 2024-04-25T08:53:16.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:16.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:16.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b7dcf1640 1 Processor -- start 2024-04-25T08:53:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b7dcf1640 1 -- start start 2024-04-25T08:53:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b7dcf1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0b780cf0e0 0x7f0b7815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b7dcf1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780ce320 con 0x7f0b7815aac0 2024-04-25T08:53:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b7dcf1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780ce600 con 0x7f0b780cd350 2024-04-25T08:53:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b7dcf1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b780ce0d0 con 0x7f0b780cf0e0 2024-04-25T08:53:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b777fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0b7815aac0 0x7f0b780ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45622/0 (socket says 172.21.15.37:45622) 2024-04-25T08:53:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b777fe640 1 -- 172.21.15.37:0/1151178631 learned_addr learned my addr 172.21.15.37:0/1151178631 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b77fff640 1 --2- 172.21.15.37:0/1151178631 >> v2:172.21.15.37:3300/0 conn(0x7f0b780cf0e0 0x7f0b7815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b767fc640 1 -- 172.21.15.37:0/1151178631 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13995882 0 0) 0x7f0b780ce320 con 0x7f0b7815aac0 2024-04-25T08:53:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b767fc640 1 -- 172.21.15.37:0/1151178631 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b60003530 con 0x7f0b7815aac0 2024-04-25T08:53:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b77fff640 1 -- 172.21.15.37:0/1151178631 >> v1:172.21.15.203:6789/0 conn(0x7f0b780cd350 legacy=0x7f0b7815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b77fff640 1 -- 172.21.15.37:0/1151178631 >> v1:172.21.15.37:6789/0 conn(0x7f0b7815aac0 legacy=0x7f0b780ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.412+0000 7f0b77fff640 1 -- 172.21.15.37:0/1151178631 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780ce8e0 con 0x7f0b780cf0e0 2024-04-25T08:53:16.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b77fff640 1 --2- 172.21.15.37:0/1151178631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780cf0e0 0x7f0b7815b5b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009d90 tx=0x7f0b7001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=259eec4e32dce26f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b767fc640 1 -- 172.21.15.37:0/1151178631 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700b2070 con 0x7f0b780cf0e0 2024-04-25T08:53:16.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b767fc640 1 -- 172.21.15.37:0/1151178631 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b700a5730 con 0x7f0b780cf0e0 2024-04-25T08:53:16.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b767fc640 1 -- 172.21.15.37:0/1151178631 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700a5b00 con 0x7f0b780cf0e0 2024-04-25T08:53:16.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/1151178631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780cf0e0 msgr2=0x7f0b7815b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 --2- 172.21.15.37:0/1151178631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780cf0e0 0x7f0b7815b5b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009d90 tx=0x7f0b7001c970 comp rx=0 tx=0).stop 2024-04-25T08:53:16.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/1151178631 shutdown_connections 2024-04-25T08:53:16.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 --2- 172.21.15.37:0/1151178631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780cf0e0 0x7f0b7815b5b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/1151178631 >> 172.21.15.37:0/1151178631 conn(0x7f0b78099a70 msgr2=0x7f0b780d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:16.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/1151178631 shutdown_connections 2024-04-25T08:53:16.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/1151178631 wait complete. 2024-04-25T08:53:16.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 Processor -- start 2024-04-25T08:53:16.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 -- start start 2024-04-25T08:53:16.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b780cd350 0x7f0b78151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780cf0e0 0x7f0b780c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b7815aac0 0x7f0b780c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7806b830 con 0x7f0b780cf0e0 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7806b6b0 con 0x7f0b7815aac0 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.416+0000 7f0b7dcf1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7806b9b0 con 0x7f0b780cd350 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b76ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780cf0e0 0x7f0b780c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b76ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780cf0e0 0x7f0b780c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37732/0 (socket says 172.21.15.37:37732) 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b76ffd640 1 -- 172.21.15.37:0/2247797846 learned_addr learned my addr 172.21.15.37:0/2247797846 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b77fff640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b7815aac0 0x7f0b780c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b777fe640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b780cd350 0x7f0b78151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b76ffd640 1 -- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b780cd350 msgr2=0x7f0b78151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b76ffd640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b780cd350 0x7f0b78151cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b76ffd640 1 -- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b7815aac0 msgr2=0x7f0b780c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b76ffd640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b7815aac0 0x7f0b780c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b76ffd640 1 -- 172.21.15.37:0/2247797846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b78152200 con 0x7f0b780cf0e0 2024-04-25T08:53:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b77fff640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b7815aac0 0x7f0b780c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b76ffd640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780cf0e0 0x7f0b780c8620 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c420 tx=0x7f0b6800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b5ffff640 1 -- 172.21.15.37:0/2247797846 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b680052d0 con 0x7f0b780cf0e0 2024-04-25T08:53:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b5ffff640 1 -- 172.21.15.37:0/2247797846 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68005470 con 0x7f0b780cf0e0 2024-04-25T08:53:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b5ffff640 1 -- 172.21.15.37:0/2247797846 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68007260 con 0x7f0b780cf0e0 2024-04-25T08:53:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/2247797846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b78157330 con 0x7f0b780cf0e0 2024-04-25T08:53:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/2247797846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b781578f0 con 0x7f0b780cf0e0 2024-04-25T08:53:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/2247797846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7804efa0 con 0x7f0b780cf0e0 2024-04-25T08:53:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b5ffff640 1 -- 172.21.15.37:0/2247797846 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b68007400 con 0x7f0b780cf0e0 2024-04-25T08:53:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b5ffff640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b64066f90 0x7f0b64069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b5ffff640 1 -- 172.21.15.37:0/2247797846 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0b680848c0 con 0x7f0b780cf0e0 2024-04-25T08:53:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.420+0000 7f0b777fe640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b64066f90 0x7f0b64069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.424+0000 7f0b777fe640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b64066f90 0x7f0b64069450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0011f0 tx=0x7f0b6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.424+0000 7f0b5ffff640 1 -- 172.21.15.37:0/2247797846 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b78152200 con 0x7f0b780cf0e0 2024-04-25T08:53:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/2247797846 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b780b6480 con 0x7f0b780cf0e0 2024-04-25T08:53:16.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b5ffff640 1 -- 172.21.15.37:0/2247797846 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b68055540 con 0x7f0b780cf0e0 2024-04-25T08:53:16.577 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b64066f90 msgr2=0x7f0b64069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b64066f90 0x7f0b64069450 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c0011f0 tx=0x7f0b6c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/2247797846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780cf0e0 msgr2=0x7f0b780c8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780cf0e0 0x7f0b780c8620 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c420 tx=0x7f0b6800c8e0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/2247797846 shutdown_connections 2024-04-25T08:53:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b64066f90 0x7f0b64069450 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b7815aac0 0x7f0b780c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b780cf0e0 0x7f0b780c8620 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 --2- 172.21.15.37:0/2247797846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b780cd350 0x7f0b78151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/2247797846 >> 172.21.15.37:0/2247797846 conn(0x7f0b78099a70 msgr2=0x7f0b780ae3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/2247797846 shutdown_connections 2024-04-25T08:53:16.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.584+0000 7f0b7dcf1640 1 -- 172.21.15.37:0/2247797846 wait complete. 2024-04-25T08:53:16.595 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72dcb48640 1 Processor -- start 2024-04-25T08:53:16.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72dcb48640 1 -- start start 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72dcb48640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d815b920 0x7f72d8099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72dcb48640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f72d80b0440 0x7f72d8099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72dcb48640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f72d80aeb80 0x7f72d80aefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72dcb48640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d809a950 con 0x7f72d80b0440 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72dcb48640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d809ac30 con 0x7f72d80aeb80 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72dcb48640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80ae380 con 0x7f72d815b920 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72d5d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f72d80b0440 0x7f72d8099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72d5d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f72d80b0440 0x7f72d8099fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37736/0 (socket says 172.21.15.37:37736) 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72d5d9b640 1 -- 172.21.15.37:0/533973849 learned_addr learned my addr 172.21.15.37:0/533973849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72d659c640 1 --2- 172.21.15.37:0/533973849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d815b920 0x7f72d8099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72d6d9d640 1 --2- 172.21.15.37:0/533973849 >> v2:172.21.15.203:3300/0 conn(0x7f72d80aeb80 0x7f72d80aefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72d5d9b640 1 -- 172.21.15.37:0/533973849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d815b920 msgr2=0x7f72d8099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72d5d9b640 1 --2- 172.21.15.37:0/533973849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d815b920 0x7f72d8099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72d5d9b640 1 -- 172.21.15.37:0/533973849 >> v2:172.21.15.203:3300/0 conn(0x7f72d80aeb80 msgr2=0x7f72d80aefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72d5d9b640 1 --2- 172.21.15.37:0/533973849 >> v2:172.21.15.203:3300/0 conn(0x7f72d80aeb80 0x7f72d80aefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.716+0000 7f72d5d9b640 1 -- 172.21.15.37:0/533973849 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d80ae660 con 0x7f72d80b0440 2024-04-25T08:53:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72d5d9b640 1 --2- 172.21.15.37:0/533973849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72d80b0440 0x7f72d8099fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f72d80723e0 tx=0x7f72cc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e7bfd8762b004b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72d559a640 1 -- 172.21.15.37:0/533973849 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0ab5f0 con 0x7f72d80b0440 2024-04-25T08:53:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72d559a640 1 -- 172.21.15.37:0/533973849 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc0b0070 con 0x7f72d80b0440 2024-04-25T08:53:16.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72d559a640 1 -- 172.21.15.37:0/533973849 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0ab9b0 con 0x7f72d80b0440 2024-04-25T08:53:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72dcb48640 1 -- 172.21.15.37:0/533973849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72d80b0440 msgr2=0x7f72d8099fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72dcb48640 1 --2- 172.21.15.37:0/533973849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72d80b0440 0x7f72d8099fb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f72d80723e0 tx=0x7f72cc01c520 comp rx=0 tx=0).stop 2024-04-25T08:53:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72dcb48640 1 -- 172.21.15.37:0/533973849 shutdown_connections 2024-04-25T08:53:16.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72dcb48640 1 --2- 172.21.15.37:0/533973849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72d80b0440 0x7f72d8099fb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72dcb48640 1 --2- 172.21.15.37:0/533973849 >> v2:172.21.15.203:3300/0 conn(0x7f72d80aeb80 0x7f72d80aefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72dcb48640 1 --2- 172.21.15.37:0/533973849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d815b920 0x7f72d8099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72dcb48640 1 -- 172.21.15.37:0/533973849 >> 172.21.15.37:0/533973849 conn(0x7f72d80b3010 msgr2=0x7f72d80b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:16.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72dcb48640 1 -- 172.21.15.37:0/533973849 shutdown_connections 2024-04-25T08:53:16.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.720+0000 7f72dcb48640 1 -- 172.21.15.37:0/533973849 wait complete. 2024-04-25T08:53:16.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72dcb48640 1 Processor -- start 2024-04-25T08:53:16.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72dcb48640 1 -- start start 2024-04-25T08:53:16.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72dcb48640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72d80aeb80 0x7f72d806c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72dcb48640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d80b0440 0x7f72d8165ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72dcb48640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72d815b920 0x7f72d80b6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72dcb48640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80cccd0 con 0x7f72d815b920 2024-04-25T08:53:16.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72dcb48640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80ccb50 con 0x7f72d80aeb80 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72dcb48640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d80cce50 con 0x7f72d80b0440 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72d815b920 0x7f72d80b6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72d6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72d815b920 0x7f72d80b6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37740/0 (socket says 172.21.15.37:37740) 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72d6d9d640 1 -- 172.21.15.37:0/991625711 learned_addr learned my addr 172.21.15.37:0/991625711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72d659c640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72d80aeb80 0x7f72d806c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72d5d9b640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d80b0440 0x7f72d8165ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72d6d9d640 1 -- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d80b0440 msgr2=0x7f72d8165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72d6d9d640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d80b0440 0x7f72d8165ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72d6d9d640 1 -- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72d80aeb80 msgr2=0x7f72d806c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72d6d9d640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72d80aeb80 0x7f72d806c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72d6d9d640 1 -- 172.21.15.37:0/991625711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d806d680 con 0x7f72d815b920 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72d6d9d640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72d815b920 0x7f72d80b6390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f72c800e930 tx=0x7f72c800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72c6ffd640 1 -- 172.21.15.37:0/991625711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8017070 con 0x7f72d815b920 2024-04-25T08:53:16.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72c6ffd640 1 -- 172.21.15.37:0/991625711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c800c9a0 con 0x7f72d815b920 2024-04-25T08:53:16.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72dcb48640 1 -- 172.21.15.37:0/991625711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d806d960 con 0x7f72d815b920 2024-04-25T08:53:16.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72c6ffd640 1 -- 172.21.15.37:0/991625711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c800cd70 con 0x7f72d815b920 2024-04-25T08:53:16.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72dcb48640 1 -- 172.21.15.37:0/991625711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d81654e0 con 0x7f72d815b920 2024-04-25T08:53:16.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72dcb48640 1 -- 172.21.15.37:0/991625711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d804efa0 con 0x7f72d815b920 2024-04-25T08:53:16.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72c6ffd640 1 -- 172.21.15.37:0/991625711 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72c801a070 con 0x7f72d815b920 2024-04-25T08:53:16.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.724+0000 7f72c6ffd640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72b8066f60 0x7f72b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:16.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.728+0000 7f72c6ffd640 1 -- 172.21.15.37:0/991625711 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f72c80853d0 con 0x7f72d815b920 2024-04-25T08:53:16.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.728+0000 7f72d659c640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72b8066f60 0x7f72b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:16.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.728+0000 7f72d659c640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72b8066f60 0x7f72b8069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f72d0003fd0 tx=0x7f72d0003ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:16.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.732+0000 7f72c6ffd640 1 -- 172.21.15.37:0/991625711 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72c8056130 con 0x7f72d815b920 2024-04-25T08:53:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.888+0000 7f72dcb48640 1 -- 172.21.15.37:0/991625711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72d80b09e0 con 0x7f72d815b920 2024-04-25T08:53:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.888+0000 7f72c6ffd640 1 -- 172.21.15.37:0/991625711 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f72c8055a30 con 0x7f72d815b920 2024-04-25T08:53:16.883 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 -- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72b8066f60 msgr2=0x7f72b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72b8066f60 0x7f72b8069420 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f72d0003fd0 tx=0x7f72d0003ce0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 -- 172.21.15.37:0/991625711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72d815b920 msgr2=0x7f72d80b6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72d815b920 0x7f72d80b6390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f72c800e930 tx=0x7f72c800edf0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 -- 172.21.15.37:0/991625711 shutdown_connections 2024-04-25T08:53:16.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72b8066f60 0x7f72b8069420 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72d815b920 0x7f72d80b6390 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72d80b0440 0x7f72d8165ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 --2- 172.21.15.37:0/991625711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72d80aeb80 0x7f72d806c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:16.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 -- 172.21.15.37:0/991625711 >> 172.21.15.37:0/991625711 conn(0x7f72d80b3010 msgr2=0x7f72d815a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:16.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 -- 172.21.15.37:0/991625711 shutdown_connections 2024-04-25T08:53:16.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:16.892+0000 7f72dcb48640 1 -- 172.21.15.37:0/991625711 wait complete. 2024-04-25T08:53:16.902 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.024+0000 7fb2cf04d640 1 Processor -- start 2024-04-25T08:53:17.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.024+0000 7fb2cf04d640 1 -- start start 2024-04-25T08:53:17.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cf04d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb2c80cf590 0x7fb2c815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cf04d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb2c80ce170 0x7fb2c80ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cf04d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2c80ca4c0 con 0x7fb2c80cf970 2024-04-25T08:53:17.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cf04d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c80ca780 con 0x7fb2c80ce170 2024-04-25T08:53:17.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cf04d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2c80cd440 con 0x7fb2c80cf590 2024-04-25T08:53:17.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cd5ea640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb2c80ce170 0x7fb2c80ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cd5ea640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb2c80ce170 0x7fb2c80ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37748/0 (socket says 172.21.15.37:37748) 2024-04-25T08:53:17.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cd5ea640 1 -- 172.21.15.37:0/3513766626 learned_addr learned my addr 172.21.15.37:0/3513766626 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:17.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cd5ea640 1 -- 172.21.15.37:0/3513766626 >> v2:172.21.15.203:3300/0 conn(0x7fb2c80cf590 msgr2=0x7fb2c815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cd5ea640 1 --2- 172.21.15.37:0/3513766626 >> v2:172.21.15.203:3300/0 conn(0x7fb2c80cf590 0x7fb2c815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cd5ea640 1 -- 172.21.15.37:0/3513766626 >> v1:172.21.15.203:6789/0 conn(0x7fb2c80cf970 legacy=0x7fb2c80c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cd5ea640 1 -- 172.21.15.37:0/3513766626 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c80cd690 con 0x7fb2c80ce170 2024-04-25T08:53:17.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cd5ea640 1 --2- 172.21.15.37:0/3513766626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c80ce170 0x7fb2c80ce600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80d0430 tx=0x7fb2c001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819afa69c4f2af7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2c77fe640 1 -- 172.21.15.37:0/3513766626 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00a5680 con 0x7fb2c80ce170 2024-04-25T08:53:17.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2c77fe640 1 -- 172.21.15.37:0/3513766626 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c00a5820 con 0x7fb2c80ce170 2024-04-25T08:53:17.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2c77fe640 1 -- 172.21.15.37:0/3513766626 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00a5bf0 con 0x7fb2c80ce170 2024-04-25T08:53:17.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cf04d640 1 -- 172.21.15.37:0/3513766626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c80ce170 msgr2=0x7fb2c80ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cf04d640 1 --2- 172.21.15.37:0/3513766626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c80ce170 0x7fb2c80ce600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80d0430 tx=0x7fb2c001c9b0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cf04d640 1 -- 172.21.15.37:0/3513766626 shutdown_connections 2024-04-25T08:53:17.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cf04d640 1 --2- 172.21.15.37:0/3513766626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c80ce170 0x7fb2c80ce600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cf04d640 1 --2- 172.21.15.37:0/3513766626 >> v2:172.21.15.203:3300/0 conn(0x7fb2c80cf590 0x7fb2c815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.028+0000 7fb2cf04d640 1 -- 172.21.15.37:0/3513766626 >> 172.21.15.37:0/3513766626 conn(0x7fb2c806ae60 msgr2=0x7fb2c806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 -- 172.21.15.37:0/3513766626 shutdown_connections 2024-04-25T08:53:17.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 -- 172.21.15.37:0/3513766626 wait complete. 2024-04-25T08:53:17.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 Processor -- start 2024-04-25T08:53:17.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 -- start start 2024-04-25T08:53:17.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c80ce170 0x7fb2c81592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c80cf590 0x7fb2c815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cf970 0x7fb2c815bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80ca240 con 0x7fb2c80cf590 2024-04-25T08:53:17.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80ca0c0 con 0x7fb2c80cf970 2024-04-25T08:53:17.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c80ac170 con 0x7fb2c80ce170 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c80cf590 0x7fb2c815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c80cf590 0x7fb2c815b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37760/0 (socket says 172.21.15.37:37760) 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c7fff640 1 -- 172.21.15.37:0/574768767 learned_addr learned my addr 172.21.15.37:0/574768767 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cd5ea640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cf970 0x7fb2c815bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2ccde9640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c80ce170 0x7fb2c81592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c7fff640 1 -- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c80ce170 msgr2=0x7fb2c81592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c7fff640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c80ce170 0x7fb2c81592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c7fff640 1 -- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cf970 msgr2=0x7fb2c815bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c7fff640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cf970 0x7fb2c815bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c7fff640 1 -- 172.21.15.37:0/574768767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8067e40 con 0x7fb2c80cf590 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cd5ea640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cf970 0x7fb2c815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c7fff640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c80cf590 0x7fb2c815b570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c490 tx=0x7fb2bc001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c57fa640 1 -- 172.21.15.37:0/574768767 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc017070 con 0x7fb2c80cf590 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c57fa640 1 -- 172.21.15.37:0/574768767 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2bc003600 con 0x7fb2c80cf590 2024-04-25T08:53:17.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2c57fa640 1 -- 172.21.15.37:0/574768767 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2bc0039f0 con 0x7fb2c80cf590 2024-04-25T08:53:17.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 -- 172.21.15.37:0/574768767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c80c5580 con 0x7fb2c80cf590 2024-04-25T08:53:17.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.032+0000 7fb2cf04d640 1 -- 172.21.15.37:0/574768767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c80c5ac0 con 0x7fb2c80cf590 2024-04-25T08:53:17.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.036+0000 7fb2cf04d640 1 -- 172.21.15.37:0/574768767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c80c5d00 con 0x7fb2c80cf590 2024-04-25T08:53:17.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.036+0000 7fb2c57fa640 1 -- 172.21.15.37:0/574768767 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2bc003b90 con 0x7fb2c80cf590 2024-04-25T08:53:17.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.036+0000 7fb2c57fa640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b4066f60 0x7fb2b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.036+0000 7fb2c57fa640 1 -- 172.21.15.37:0/574768767 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb2bc084760 con 0x7fb2c80cf590 2024-04-25T08:53:17.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.036+0000 7fb2ccde9640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b4066f60 0x7fb2b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.036+0000 7fb2ccde9640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b4066f60 0x7fb2b4069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800a830 tx=0x7fb2b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.040+0000 7fb2c57fa640 1 -- 172.21.15.37:0/574768767 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2c80c5d00 con 0x7fb2c80cf590 2024-04-25T08:53:17.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.192+0000 7fb2cf04d640 1 -- 172.21.15.37:0/574768767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2c80b4170 con 0x7fb2c80cf590 2024-04-25T08:53:17.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.192+0000 7fb2c57fa640 1 -- 172.21.15.37:0/574768767 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2bc0552d0 con 0x7fb2c80cf590 2024-04-25T08:53:17.187 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 -- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b4066f60 msgr2=0x7fb2b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b4066f60 0x7fb2b4069420 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb2b800a830 tx=0x7fb2b8008040 comp rx=0 tx=0).stop 2024-04-25T08:53:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 -- 172.21.15.37:0/574768767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c80cf590 msgr2=0x7fb2c815b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c80cf590 0x7fb2c815b570 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00c490 tx=0x7fb2bc001e80 comp rx=0 tx=0).stop 2024-04-25T08:53:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 -- 172.21.15.37:0/574768767 shutdown_connections 2024-04-25T08:53:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b4066f60 0x7fb2b4069420 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c80cf970 0x7fb2c815bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c80cf590 0x7fb2c815b570 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 --2- 172.21.15.37:0/574768767 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c80ce170 0x7fb2c81592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 -- 172.21.15.37:0/574768767 >> 172.21.15.37:0/574768767 conn(0x7fb2c806ae60 msgr2=0x7fb2c80b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 -- 172.21.15.37:0/574768767 shutdown_connections 2024-04-25T08:53:17.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.196+0000 7fb2cf04d640 1 -- 172.21.15.37:0/574768767 wait complete. 2024-04-25T08:53:17.205 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:17.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.328+0000 7f7a65fe9640 1 Processor -- start 2024-04-25T08:53:17.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.328+0000 7f7a65fe9640 1 -- start start 2024-04-25T08:53:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a65fe9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a6015ecb0 0x7f7a600c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a65fe9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7a600c9ef0 0x7f7a600ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a65fe9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a600ca4c0 con 0x7f7a600cf590 2024-04-25T08:53:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a65fe9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a600ca780 con 0x7f7a600c9ef0 2024-04-25T08:53:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a65fe9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6015f0f0 con 0x7f7a6015ecb0 2024-04-25T08:53:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7a600cf590 0x7f7a600cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45634/0 (socket says 172.21.15.37:45634) 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5f7fe640 1 -- 172.21.15.37:0/1931425489 learned_addr learned my addr 172.21.15.37:0/1931425489 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5ffff640 1 --2- 172.21.15.37:0/1931425489 >> v2:172.21.15.203:3300/0 conn(0x7f7a600c9ef0 0x7f7a600ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5effd640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a6015ecb0 0x7f7a600c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015666381 0 0) 0x7f7a600ca4c0 con 0x7f7a600cf590 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a48005740 con 0x7f7a600cf590 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2819473554 0 0) 0x7f7a48005740 con 0x7f7a600cf590 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a6015ecb0 msgr2=0x7f7a600c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a6015ecb0 0x7f7a600c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 >> v2:172.21.15.203:3300/0 conn(0x7f7a600c9ef0 msgr2=0x7f7a600ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 --2- 172.21.15.37:0/1931425489 >> v2:172.21.15.203:3300/0 conn(0x7f7a600c9ef0 0x7f7a600ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a600cd690 con 0x7f7a600cf590 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f7a50001680 con 0x7f7a600cf590 2024-04-25T08:53:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 >> v1:172.21.15.37:6789/0 conn(0x7f7a600cf590 legacy=0x7f7a600cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a6015ecb0 0x7f7a48005c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a480061c0 0x7f7a48006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a48006f60 0x7f7a480073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6015f0f0 con 0x7f7a480061c0 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4800dbe0 con 0x7f7a48006f60 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4800dd90 con 0x7f7a6015ecb0 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5effd640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a480061c0 0x7f7a48006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5f7fe640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a6015ecb0 0x7f7a48005c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5effd640 1 -- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a6015ecb0 msgr2=0x7f7a48005c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5effd640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a6015ecb0 0x7f7a48005c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5effd640 1 -- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a48006f60 msgr2=0x7f7a480073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5ffff640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a48006f60 0x7f7a480073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5effd640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a48006f60 0x7f7a480073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5effd640 1 -- 172.21.15.37:0/1931425489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a48005740 con 0x7f7a480061c0 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5ffff640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a48006f60 0x7f7a480073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5effd640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a480061c0 0x7f7a48006a20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009ae0 tx=0x7f7a5401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf10a7dfe0bb8ba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a540b2070 con 0x7f7a480061c0 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.332+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a540a5550 con 0x7f7a480061c0 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5e7fc640 1 -- 172.21.15.37:0/1931425489 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a540096b0 con 0x7f7a480061c0 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 -- 172.21.15.37:0/1931425489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a480061c0 msgr2=0x7f7a48006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 --2- 172.21.15.37:0/1931425489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a480061c0 0x7f7a48006a20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7a54009ae0 tx=0x7f7a5401ca70 comp rx=0 tx=0).stop 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5ffff640 1 -- 172.21.15.37:0/1931425489 reap_dead start 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 -- 172.21.15.37:0/1931425489 shutdown_connections 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 -- 172.21.15.37:0/1931425489 >> 172.21.15.37:0/1931425489 conn(0x7f7a6006ae60 msgr2=0x7f7a6006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 -- 172.21.15.37:0/1931425489 shutdown_connections 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 -- 172.21.15.37:0/1931425489 wait complete. 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 Processor -- start 2024-04-25T08:53:17.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 -- start start 2024-04-25T08:53:17.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a60155520 0x7f7a601f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a601f2720 0x7f7a60155900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a60155e40 0x7f7a6006cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a600cdce0 con 0x7f7a601f2720 2024-04-25T08:53:17.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a600cd440 con 0x7f7a60155520 2024-04-25T08:53:17.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a600ac170 con 0x7f7a60155e40 2024-04-25T08:53:17.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a601f2720 0x7f7a60155900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a601f2720 0x7f7a60155900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37776/0 (socket says 172.21.15.37:37776) 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5effd640 1 -- 172.21.15.37:0/2910895585 learned_addr learned my addr 172.21.15.37:0/2910895585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5f7fe640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a60155520 0x7f7a601f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5effd640 1 -- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a60155e40 msgr2=0x7f7a6006cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5ffff640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a60155e40 0x7f7a6006cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5effd640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a60155e40 0x7f7a6006cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5effd640 1 -- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a60155520 msgr2=0x7f7a601f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5effd640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a60155520 0x7f7a601f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5effd640 1 -- 172.21.15.37:0/2910895585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6006e200 con 0x7f7a601f2720 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5f7fe640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a60155520 0x7f7a601f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a5effd640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a601f2720 0x7f7a60155900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a540b0040 tx=0x7f7a54019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a47fff640 1 -- 172.21.15.37:0/2910895585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a540b2040 con 0x7f7a601f2720 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a47fff640 1 -- 172.21.15.37:0/2910895585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a54004020 con 0x7f7a601f2720 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 -- 172.21.15.37:0/2910895585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6006e4e0 con 0x7f7a601f2720 2024-04-25T08:53:17.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a47fff640 1 -- 172.21.15.37:0/2910895585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a540096b0 con 0x7f7a601f2720 2024-04-25T08:53:17.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.336+0000 7f7a65fe9640 1 -- 172.21.15.37:0/2910895585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6006eaa0 con 0x7f7a601f2720 2024-04-25T08:53:17.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.340+0000 7f7a65fe9640 1 -- 172.21.15.37:0/2910895585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6004efa0 con 0x7f7a601f2720 2024-04-25T08:53:17.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.340+0000 7f7a47fff640 1 -- 172.21.15.37:0/2910895585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a540bf060 con 0x7f7a601f2720 2024-04-25T08:53:17.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.340+0000 7f7a47fff640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a4c066f90 0x7f7a4c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.340+0000 7f7a47fff640 1 -- 172.21.15.37:0/2910895585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7a5411ec70 con 0x7f7a601f2720 2024-04-25T08:53:17.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.340+0000 7f7a5f7fe640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a4c066f90 0x7f7a4c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.340+0000 7f7a5f7fe640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a4c066f90 0x7f7a4c069450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a50001060 tx=0x7f7a50000ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.344+0000 7f7a47fff640 1 -- 172.21.15.37:0/2910895585 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a54016780 con 0x7f7a601f2720 2024-04-25T08:53:17.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.500+0000 7f7a65fe9640 1 -- 172.21.15.37:0/2910895585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7a600cfa10 con 0x7f7a601f2720 2024-04-25T08:53:17.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.500+0000 7f7a47fff640 1 -- 172.21.15.37:0/2910895585 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7a540ef910 con 0x7f7a601f2720 2024-04-25T08:53:17.494 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:17.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 -- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a4c066f90 msgr2=0x7f7a4c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a4c066f90 0x7f7a4c069450 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f7a50001060 tx=0x7f7a50000ff0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 -- 172.21.15.37:0/2910895585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a601f2720 msgr2=0x7f7a60155900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a601f2720 0x7f7a60155900 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7a540b0040 tx=0x7f7a54019810 comp rx=0 tx=0).stop 2024-04-25T08:53:17.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 -- 172.21.15.37:0/2910895585 shutdown_connections 2024-04-25T08:53:17.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a4c066f90 0x7f7a4c069450 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a60155e40 0x7f7a6006cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a601f2720 0x7f7a60155900 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 --2- 172.21.15.37:0/2910895585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a60155520 0x7f7a601f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 -- 172.21.15.37:0/2910895585 >> 172.21.15.37:0/2910895585 conn(0x7f7a6006ae60 msgr2=0x7f7a600cbb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 -- 172.21.15.37:0/2910895585 shutdown_connections 2024-04-25T08:53:17.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.504+0000 7f7a65fe9640 1 -- 172.21.15.37:0/2910895585 wait complete. 2024-04-25T08:53:17.512 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:17.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.640+0000 7f46a279f640 1 Processor -- start 2024-04-25T08:53:17.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.640+0000 7f46a279f640 1 -- start start 2024-04-25T08:53:17.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.640+0000 7f46a279f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c0b2e20 0x7f469c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.640+0000 7f46a279f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469c15ae30 con 0x7f469c06d5d0 2024-04-25T08:53:17.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.640+0000 7f46a279f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469c15b110 con 0x7f469c0b3200 2024-04-25T08:53:17.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.640+0000 7f46a279f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0b2990 con 0x7f469c0b2e20 2024-04-25T08:53:17.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.640+0000 7f46a0d3c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f469c06d5d0 0x7f469c15a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45644/0 (socket says 172.21.15.37:45644) 2024-04-25T08:53:17.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.640+0000 7f46a0d3c640 1 -- 172.21.15.37:0/1570079313 learned_addr learned my addr 172.21.15.37:0/1570079313 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:17.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.640+0000 7f469bfff640 1 --2- 172.21.15.37:0/1570079313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c0b2e20 0x7f469c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.640+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123502631 0 0) 0x7f469c15ae30 con 0x7f469c06d5d0 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4688003530 con 0x7f469c06d5d0 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3507896672 0 0) 0x7f4688003530 con 0x7f469c06d5d0 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c0b2e20 msgr2=0x7f469c06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 --2- 172.21.15.37:0/1570079313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c0b2e20 0x7f469c06d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 >> v1:172.21.15.203:6789/0 conn(0x7f469c0b3200 legacy=0x7f469c159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c0b2620 con 0x7f469c06d5d0 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f4694001a00 con 0x7f469c06d5d0 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 >> v1:172.21.15.37:6789/0 conn(0x7f469c06d5d0 legacy=0x7f469c15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 --2- 172.21.15.37:0/1570079313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 0x7f4688003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 --2- 172.21.15.37:0/1570079313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c0b2e20 0x7f4688004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 --2- 172.21.15.37:0/1570079313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4688004d50 0x7f46880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c0b2990 con 0x7f4688003a60 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468800dd70 con 0x7f4688004d50 2024-04-25T08:53:17.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f468800df40 con 0x7f469c0b2e20 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469bfff640 1 --2- 172.21.15.37:0/1570079313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 0x7f4688003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469bfff640 1 -- 172.21.15.37:0/1570079313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c0b2e20 msgr2=0x7f4688004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469bfff640 1 --2- 172.21.15.37:0/1570079313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c0b2e20 0x7f4688004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469bfff640 1 -- 172.21.15.37:0/1570079313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4688004d50 msgr2=0x7f46880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f46a0d3c640 1 --2- 172.21.15.37:0/1570079313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4688004d50 0x7f46880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469bfff640 1 --2- 172.21.15.37:0/1570079313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4688004d50 0x7f46880051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469bfff640 1 -- 172.21.15.37:0/1570079313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4688003530 con 0x7f4688003a60 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469bfff640 1 --2- 172.21.15.37:0/1570079313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 0x7f4688003e80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4684009ae0 tx=0x7f468401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e2044e0f8982b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840b2070 con 0x7f4688003a60 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840a5550 con 0x7f4688003a60 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f469b7fe640 1 -- 172.21.15.37:0/1570079313 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840096b0 con 0x7f4688003a60 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f46a279f640 1 -- 172.21.15.37:0/1570079313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 msgr2=0x7f4688003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f46a279f640 1 --2- 172.21.15.37:0/1570079313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 0x7f4688003e80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4684009ae0 tx=0x7f468401ca70 comp rx=0 tx=0).stop 2024-04-25T08:53:17.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f46a0d3c640 1 -- 172.21.15.37:0/1570079313 reap_dead start 2024-04-25T08:53:17.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f46a279f640 1 -- 172.21.15.37:0/1570079313 shutdown_connections 2024-04-25T08:53:17.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f46a279f640 1 -- 172.21.15.37:0/1570079313 >> 172.21.15.37:0/1570079313 conn(0x7f469c0b11a0 msgr2=0x7f469c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f46a279f640 1 -- 172.21.15.37:0/1570079313 shutdown_connections 2024-04-25T08:53:17.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.644+0000 7f46a279f640 1 -- 172.21.15.37:0/1570079313 wait complete. 2024-04-25T08:53:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a279f640 1 Processor -- start 2024-04-25T08:53:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a279f640 1 -- start start 2024-04-25T08:53:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a279f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 0x7f469c14ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a279f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c1480b0 0x7f469c147830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a279f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f469c153720 0x7f469c153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a279f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c160300 con 0x7f4688003a60 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a279f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469c160180 con 0x7f469c153720 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a279f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469c160480 con 0x7f469c1480b0 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f469bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 0x7f469c14ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f469bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 0x7f469c14ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37792/0 (socket says 172.21.15.37:37792) 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f469bfff640 1 -- 172.21.15.37:0/1893831832 learned_addr learned my addr 172.21.15.37:0/1893831832 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f4693fff640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c1480b0 0x7f469c147830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a0d3c640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f469c153720 0x7f469c153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f469bfff640 1 -- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c1480b0 msgr2=0x7f469c147830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f469bfff640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c1480b0 0x7f469c147830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f469bfff640 1 -- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f469c153720 msgr2=0x7f469c153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f469bfff640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f469c153720 0x7f469c153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f469bfff640 1 -- 172.21.15.37:0/1893831832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c14b090 con 0x7f4688003a60 2024-04-25T08:53:17.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f469bfff640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 0x7f469c14ab50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f46840b0040 tx=0x7f4688005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46997fa640 1 -- 172.21.15.37:0/1893831832 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46840b2040 con 0x7f4688003a60 2024-04-25T08:53:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46997fa640 1 -- 172.21.15.37:0/1893831832 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46840a5d20 con 0x7f4688003a60 2024-04-25T08:53:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a279f640 1 -- 172.21.15.37:0/1893831832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469c148570 con 0x7f4688003a60 2024-04-25T08:53:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46997fa640 1 -- 172.21.15.37:0/1893831832 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4684004260 con 0x7f4688003a60 2024-04-25T08:53:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a279f640 1 -- 172.21.15.37:0/1893831832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469c148b30 con 0x7f4688003a60 2024-04-25T08:53:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46a279f640 1 -- 172.21.15.37:0/1893831832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f469c04efa0 con 0x7f4688003a60 2024-04-25T08:53:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.648+0000 7f46997fa640 1 -- 172.21.15.37:0/1893831832 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f46840c0060 con 0x7f4688003a60 2024-04-25T08:53:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.652+0000 7f46997fa640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f467c066f60 0x7f467c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.652+0000 7f46997fa640 1 -- 172.21.15.37:0/1893831832 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f468411ebf0 con 0x7f4688003a60 2024-04-25T08:53:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.652+0000 7f4693fff640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f467c066f60 0x7f467c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.652+0000 7f4693fff640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f467c066f60 0x7f467c069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f469c0ae460 tx=0x7f468c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.656+0000 7f46997fa640 1 -- 172.21.15.37:0/1893831832 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f469c14b090 con 0x7f4688003a60 2024-04-25T08:53:17.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.816+0000 7f46a279f640 1 -- 172.21.15.37:0/1893831832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f469c06e6b0 con 0x7f4688003a60 2024-04-25T08:53:17.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.816+0000 7f46997fa640 1 -- 172.21.15.37:0/1893831832 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f46840167c0 con 0x7f4688003a60 2024-04-25T08:53:17.811 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:17.811 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:17.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 -- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f467c066f60 msgr2=0x7f467c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f467c066f60 0x7f467c069420 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f469c0ae460 tx=0x7f468c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:17.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 -- 172.21.15.37:0/1893831832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 msgr2=0x7f469c14ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 0x7f469c14ab50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f46840b0040 tx=0x7f4688005840 comp rx=0 tx=0).stop 2024-04-25T08:53:17.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 -- 172.21.15.37:0/1893831832 shutdown_connections 2024-04-25T08:53:17.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f467c066f60 0x7f467c069420 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f469c153720 0x7f469c153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f469c1480b0 0x7f469c147830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 --2- 172.21.15.37:0/1893831832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4688003a60 0x7f469c14ab50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 -- 172.21.15.37:0/1893831832 >> 172.21.15.37:0/1893831832 conn(0x7f469c0b11a0 msgr2=0x7f469c0ae740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 -- 172.21.15.37:0/1893831832 shutdown_connections 2024-04-25T08:53:17.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.820+0000 7f46a279f640 1 -- 172.21.15.37:0/1893831832 wait complete. 2024-04-25T08:53:17.830 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:17.831 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7748399640 1 Processor -- start 2024-04-25T08:53:17.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7748399640 1 -- start start 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7748399640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f77400cf590 0x7f77400cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7748399640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7740072000 0x7f7740072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7748399640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77400cbd20 con 0x7f774015b6b0 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7748399640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774015ba90 con 0x7f77400cf590 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7748399640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77400cc000 con 0x7f7740072000 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7746135640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f774015b6b0 0x7f77400ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45654/0 (socket says 172.21.15.37:45654) 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7746135640 1 -- 172.21.15.37:0/1809386876 learned_addr learned my addr 172.21.15.37:0/1809386876 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745934640 1 --2- 172.21.15.37:0/1809386876 >> v2:172.21.15.203:3300/0 conn(0x7f77400cf590 0x7f77400cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7746936640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7740072000 0x7f7740072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2514037423 0 0) 0x7f77400cbd20 con 0x7f774015b6b0 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f772c003530 con 0x7f774015b6b0 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2653606747 0 0) 0x7f772c003530 con 0x7f774015b6b0 2024-04-25T08:53:17.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7740072000 msgr2=0x7f7740072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7740072000 0x7f7740072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 >> v2:172.21.15.203:3300/0 conn(0x7f77400cf590 msgr2=0x7f77400cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 --2- 172.21.15.37:0/1809386876 >> v2:172.21.15.203:3300/0 conn(0x7f77400cf590 0x7f77400cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77400cc2e0 con 0x7f774015b6b0 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f7734001680 con 0x7f774015b6b0 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 >> v1:172.21.15.37:6789/0 conn(0x7f774015b6b0 legacy=0x7f77400ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f772c003a30 0x7f772c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c004810 0x7f772c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7740072000 0x7f772c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774015ba90 con 0x7f772c004810 2024-04-25T08:53:17.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400cc000 con 0x7f772c003a30 2024-04-25T08:53:17.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f772c00b9d0 con 0x7f7740072000 2024-04-25T08:53:17.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745934640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c004810 0x7f772c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7746135640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f772c003a30 0x7f772c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7746936640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7740072000 0x7f772c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745934640 1 -- 172.21.15.37:0/1809386876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7740072000 msgr2=0x7f772c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745934640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7740072000 0x7f772c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745934640 1 -- 172.21.15.37:0/1809386876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f772c003a30 msgr2=0x7f772c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:17.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745934640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f772c003a30 0x7f772c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.956+0000 7f7745934640 1 -- 172.21.15.37:0/1809386876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772c003530 con 0x7f772c004810 2024-04-25T08:53:17.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7745934640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c004810 0x7f772c004c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f773c00e170 tx=0x7f773c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75b46fa75b8214d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c0b2070 con 0x7f772c004810 2024-04-25T08:53:17.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773c0a5bd0 con 0x7f772c004810 2024-04-25T08:53:17.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7745133640 1 -- 172.21.15.37:0/1809386876 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773c009c10 con 0x7f772c004810 2024-04-25T08:53:17.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7748399640 1 -- 172.21.15.37:0/1809386876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c004810 msgr2=0x7f772c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7748399640 1 --2- 172.21.15.37:0/1809386876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f772c004810 0x7f772c004c90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f773c00e170 tx=0x7f773c01cca0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7746936640 1 -- 172.21.15.37:0/1809386876 reap_dead start 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7748399640 1 -- 172.21.15.37:0/1809386876 shutdown_connections 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7748399640 1 -- 172.21.15.37:0/1809386876 >> 172.21.15.37:0/1809386876 conn(0x7f774006ae60 msgr2=0x7f774006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7748399640 1 -- 172.21.15.37:0/1809386876 shutdown_connections 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7748399640 1 -- 172.21.15.37:0/1809386876 wait complete. 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7748399640 1 Processor -- start 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.960+0000 7f7748399640 1 -- start start 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7748399640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7740156990 0x7f7740156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7748399640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77400c21f0 0x7f77400c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7748399640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77400ccf30 0x7f774006e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7748399640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400cba30 con 0x7f77400ccf30 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7748399640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400cc8a0 con 0x7f7740156990 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7748399640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77400cadb0 con 0x7f77400c21f0 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7746936640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77400ccf30 0x7f774006e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7746936640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77400ccf30 0x7f774006e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37814/0 (socket says 172.21.15.37:37814) 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7746936640 1 -- 172.21.15.37:0/319177638 learned_addr learned my addr 172.21.15.37:0/319177638 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7746135640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7740156990 0x7f7740156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7745934640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77400c21f0 0x7f77400c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7746936640 1 -- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77400c21f0 msgr2=0x7f77400c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7746936640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77400c21f0 0x7f77400c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7746936640 1 -- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7740156990 msgr2=0x7f7740156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7746936640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7740156990 0x7f7740156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7746936640 1 -- 172.21.15.37:0/319177638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77400cd430 con 0x7f77400ccf30 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7746135640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7740156990 0x7f7740156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7746936640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77400ccf30 0x7f774006e0d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77400734b0 tx=0x7f7738010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7732ffd640 1 -- 172.21.15.37:0/319177638 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773800f040 con 0x7f77400ccf30 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7732ffd640 1 -- 172.21.15.37:0/319177638 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773801b070 con 0x7f77400ccf30 2024-04-25T08:53:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7732ffd640 1 -- 172.21.15.37:0/319177638 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738010ea0 con 0x7f77400ccf30 2024-04-25T08:53:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7748399640 1 -- 172.21.15.37:0/319177638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77400cd710 con 0x7f77400ccf30 2024-04-25T08:53:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7748399640 1 -- 172.21.15.37:0/319177638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77400cdcd0 con 0x7f77400ccf30 2024-04-25T08:53:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7748399640 1 -- 172.21.15.37:0/319177638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f774004efa0 con 0x7f77400ccf30 2024-04-25T08:53:17.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7732ffd640 1 -- 172.21.15.37:0/319177638 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f773800e6a0 con 0x7f77400ccf30 2024-04-25T08:53:17.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7732ffd640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7724066f60 0x7f7724069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:17.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.964+0000 7f7732ffd640 1 -- 172.21.15.37:0/319177638 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7738084e60 con 0x7f77400ccf30 2024-04-25T08:53:17.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.968+0000 7f7746135640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7724066f60 0x7f7724069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:17.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.968+0000 7f7746135640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7724066f60 0x7f7724069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7734004850 tx=0x7f7734009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:17.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:17.972+0000 7f7732ffd640 1 -- 172.21.15.37:0/319177638 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7738025020 con 0x7f77400ccf30 2024-04-25T08:53:18.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 -- 172.21.15.37:0/319177638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f774006bfc0 con 0x7f77400ccf30 2024-04-25T08:53:18.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7732ffd640 1 -- 172.21.15.37:0/319177638 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7738055a70 con 0x7f77400ccf30 2024-04-25T08:53:18.120 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:18.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 -- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7724066f60 msgr2=0x7f7724069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7724066f60 0x7f7724069420 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7734004850 tx=0x7f7734009040 comp rx=0 tx=0).stop 2024-04-25T08:53:18.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 -- 172.21.15.37:0/319177638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77400ccf30 msgr2=0x7f774006e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77400ccf30 0x7f774006e0d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f77400734b0 tx=0x7f7738010b60 comp rx=0 tx=0).stop 2024-04-25T08:53:18.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 -- 172.21.15.37:0/319177638 shutdown_connections 2024-04-25T08:53:18.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7724066f60 0x7f7724069420 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77400ccf30 0x7f774006e0d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f77400c21f0 0x7f77400c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 --2- 172.21.15.37:0/319177638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7740156990 0x7f7740156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 -- 172.21.15.37:0/319177638 >> 172.21.15.37:0/319177638 conn(0x7f774006ae60 msgr2=0x7f77400af8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 -- 172.21.15.37:0/319177638 shutdown_connections 2024-04-25T08:53:18.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.128+0000 7f7748399640 1 -- 172.21.15.37:0/319177638 wait complete. 2024-04-25T08:53:18.138 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:18.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.268+0000 7f17cef48640 1 Processor -- start 2024-04-25T08:53:18.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.268+0000 7f17cef48640 1 -- start start 2024-04-25T08:53:18.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f17c815b6b0 0x7f17c80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f17c80cf590 0x7f17c80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17c815f130 0x7f17c815f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c815ba90 con 0x7f17c815b6b0 2024-04-25T08:53:18.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17c80cbd20 con 0x7f17c80cf590 2024-04-25T08:53:18.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80cc000 con 0x7f17c815f130 2024-04-25T08:53:18.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17ccce4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f17c815b6b0 0x7f17c80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17ccce4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f17c815b6b0 0x7f17c80ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37828/0 (socket says 172.21.15.37:37828) 2024-04-25T08:53:18.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17ccce4640 1 -- 172.21.15.37:0/3948460564 learned_addr learned my addr 172.21.15.37:0/3948460564 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cd4e5640 1 --2- 172.21.15.37:0/3948460564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17c815f130 0x7f17c815f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17c7fff640 1 --2- 172.21.15.37:0/3948460564 >> v2:172.21.15.203:3300/0 conn(0x7f17c80cf590 0x7f17c80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17ccce4640 1 -- 172.21.15.37:0/3948460564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17c815f130 msgr2=0x7f17c815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17ccce4640 1 --2- 172.21.15.37:0/3948460564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17c815f130 0x7f17c815f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17ccce4640 1 -- 172.21.15.37:0/3948460564 >> v2:172.21.15.203:3300/0 conn(0x7f17c80cf590 msgr2=0x7f17c80cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17ccce4640 1 --2- 172.21.15.37:0/3948460564 >> v2:172.21.15.203:3300/0 conn(0x7f17c80cf590 0x7f17c80cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17ccce4640 1 -- 172.21.15.37:0/3948460564 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c80cc2e0 con 0x7f17c815b6b0 2024-04-25T08:53:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17c7fff640 1 --2- 172.21.15.37:0/3948460564 >> v2:172.21.15.203:3300/0 conn(0x7f17c80cf590 0x7f17c80cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:18.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cd4e5640 1 --2- 172.21.15.37:0/3948460564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17c815f130 0x7f17c815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:18.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17ccce4640 1 --2- 172.21.15.37:0/3948460564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17c815b6b0 0x7f17c80ca3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f17c80b4430 tx=0x7f17b801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1bf699ef488230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17c77fe640 1 -- 172.21.15.37:0/3948460564 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b80b1070 con 0x7f17c815b6b0 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17c77fe640 1 -- 172.21.15.37:0/3948460564 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b80ac630 con 0x7f17c815b6b0 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17c77fe640 1 -- 172.21.15.37:0/3948460564 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b80a8050 con 0x7f17c815b6b0 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 -- 172.21.15.37:0/3948460564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17c815b6b0 msgr2=0x7f17c80ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 --2- 172.21.15.37:0/3948460564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17c815b6b0 0x7f17c80ca3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f17c80b4430 tx=0x7f17b801c520 comp rx=0 tx=0).stop 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 -- 172.21.15.37:0/3948460564 shutdown_connections 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 --2- 172.21.15.37:0/3948460564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17c815f130 0x7f17c815f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 --2- 172.21.15.37:0/3948460564 >> v2:172.21.15.203:3300/0 conn(0x7f17c80cf590 0x7f17c80cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 --2- 172.21.15.37:0/3948460564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17c815b6b0 0x7f17c80ca3c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 -- 172.21.15.37:0/3948460564 >> 172.21.15.37:0/3948460564 conn(0x7f17c806ae60 msgr2=0x7f17c806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 -- 172.21.15.37:0/3948460564 shutdown_connections 2024-04-25T08:53:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.272+0000 7f17cef48640 1 -- 172.21.15.37:0/3948460564 wait complete. 2024-04-25T08:53:18.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cef48640 1 Processor -- start 2024-04-25T08:53:18.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cef48640 1 -- start start 2024-04-25T08:53:18.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cef48640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17c80cf590 0x7f17c80c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cef48640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17c815b6b0 0x7f17c80b36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cef48640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17c815f130 0x7f17c80c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cef48640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80ca900 con 0x7f17c815f130 2024-04-25T08:53:18.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cef48640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c815fef0 con 0x7f17c80cf590 2024-04-25T08:53:18.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cef48640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80cadb0 con 0x7f17c815b6b0 2024-04-25T08:53:18.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cd4e5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17c815f130 0x7f17c80c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cd4e5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17c815f130 0x7f17c80c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37840/0 (socket says 172.21.15.37:37840) 2024-04-25T08:53:18.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cd4e5640 1 -- 172.21.15.37:0/3464280523 learned_addr learned my addr 172.21.15.37:0/3464280523 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17ccce4640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17c80cf590 0x7f17c80c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cd4e5640 1 -- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17c815b6b0 msgr2=0x7f17c80b36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17c7fff640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17c815b6b0 0x7f17c80b36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cd4e5640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17c815b6b0 0x7f17c80b36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cd4e5640 1 -- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17c80cf590 msgr2=0x7f17c80c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cd4e5640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17c80cf590 0x7f17c80c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cd4e5640 1 -- 172.21.15.37:0/3464280523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c80ae3a0 con 0x7f17c815f130 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cd4e5640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17c815f130 0x7f17c80c7480 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f17c000cd80 tx=0x7f17c000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17c57fa640 1 -- 172.21.15.37:0/3464280523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000f680 con 0x7f17c815f130 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17c57fa640 1 -- 172.21.15.37:0/3464280523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c000f840 con 0x7f17c815f130 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cef48640 1 -- 172.21.15.37:0/3464280523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c8158390 con 0x7f17c815f130 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17c57fa640 1 -- 172.21.15.37:0/3464280523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000fc30 con 0x7f17c815f130 2024-04-25T08:53:18.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cef48640 1 -- 172.21.15.37:0/3464280523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c80c5070 con 0x7f17c815f130 2024-04-25T08:53:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.276+0000 7f17cef48640 1 -- 172.21.15.37:0/3464280523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c804efa0 con 0x7f17c815f130 2024-04-25T08:53:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.280+0000 7f17c57fa640 1 -- 172.21.15.37:0/3464280523 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f17c00213f0 con 0x7f17c815f130 2024-04-25T08:53:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.280+0000 7f17c57fa640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f17a8066f90 0x7f17a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.280+0000 7f17c57fa640 1 -- 172.21.15.37:0/3464280523 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f17c0086190 con 0x7f17c815f130 2024-04-25T08:53:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.280+0000 7f17ccce4640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f17a8066f90 0x7f17a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.280+0000 7f17ccce4640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f17a8066f90 0x7f17a8069450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f17b80ae000 tx=0x7f17b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.284+0000 7f17c57fa640 1 -- 172.21.15.37:0/3464280523 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f17c0014030 con 0x7f17c815f130 2024-04-25T08:53:18.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.448+0000 7f17cef48640 1 -- 172.21.15.37:0/3464280523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f17c8072db0 con 0x7f17c815f130 2024-04-25T08:53:18.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.448+0000 7f17c57fa640 1 -- 172.21.15.37:0/3464280523 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f17c0056d00 con 0x7f17c815f130 2024-04-25T08:53:18.441 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:18.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.448+0000 7f17cef48640 1 -- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f17a8066f90 msgr2=0x7f17a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.448+0000 7f17cef48640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f17a8066f90 0x7f17a8069450 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f17b80ae000 tx=0x7f17b8008040 comp rx=0 tx=0).stop 2024-04-25T08:53:18.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.448+0000 7f17cef48640 1 -- 172.21.15.37:0/3464280523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17c815f130 msgr2=0x7f17c80c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.448+0000 7f17cef48640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17c815f130 0x7f17c80c7480 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f17c000cd80 tx=0x7f17c000a650 comp rx=0 tx=0).stop 2024-04-25T08:53:18.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.452+0000 7f17cef48640 1 -- 172.21.15.37:0/3464280523 shutdown_connections 2024-04-25T08:53:18.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.452+0000 7f17cef48640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f17a8066f90 0x7f17a8069450 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.452+0000 7f17cef48640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17c815f130 0x7f17c80c7480 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.452+0000 7f17cef48640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f17c815b6b0 0x7f17c80b36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.452+0000 7f17cef48640 1 --2- 172.21.15.37:0/3464280523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f17c80cf590 0x7f17c80c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.452+0000 7f17cef48640 1 -- 172.21.15.37:0/3464280523 >> 172.21.15.37:0/3464280523 conn(0x7f17c806ae60 msgr2=0x7f17c80b2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.452+0000 7f17cef48640 1 -- 172.21.15.37:0/3464280523 shutdown_connections 2024-04-25T08:53:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.452+0000 7f17cef48640 1 -- 172.21.15.37:0/3464280523 wait complete. 2024-04-25T08:53:18.461 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:18.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4e2774640 1 Processor -- start 2024-04-25T08:53:18.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4e2774640 1 -- start start 2024-04-25T08:53:18.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4e2774640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff4dc15da80 0x7ff4dc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4e2774640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff4dc0d07f0 0x7ff4dc06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4e2774640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4dc06be60 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4e2774640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4dc06c440 con 0x7ff4dc15da80 2024-04-25T08:53:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4e2774640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4dc15de60 con 0x7ff4dc0d07f0 2024-04-25T08:53:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4dbfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff4dc15da80 0x7ff4dc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4db7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff4dc0d1cd0 0x7ff4dc0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45658/0 (socket says 172.21.15.37:45658) 2024-04-25T08:53:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4db7fe640 1 -- 172.21.15.37:0/1188140218 learned_addr learned my addr 172.21.15.37:0/1188140218 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4e0d11640 1 --2- 172.21.15.37:0/1188140218 >> v2:172.21.15.203:3300/0 conn(0x7ff4dc0d07f0 0x7ff4dc06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4dbfff640 1 -- 172.21.15.37:0/1188140218 >> v2:172.21.15.203:3300/0 conn(0x7ff4dc0d07f0 msgr2=0x7ff4dc06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4dbfff640 1 --2- 172.21.15.37:0/1188140218 >> v2:172.21.15.203:3300/0 conn(0x7ff4dc0d07f0 0x7ff4dc06c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4dbfff640 1 -- 172.21.15.37:0/1188140218 >> v1:172.21.15.37:6789/0 conn(0x7ff4dc0d1cd0 legacy=0x7ff4dc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4daffd640 1 -- 172.21.15.37:0/1188140218 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398288920 0 0) 0x7ff4dc06be60 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4dbfff640 1 -- 172.21.15.37:0/1188140218 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc0d2450 con 0x7ff4dc15da80 2024-04-25T08:53:18.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.592+0000 7ff4e0d11640 1 --2- 172.21.15.37:0/1188140218 >> v2:172.21.15.203:3300/0 conn(0x7ff4dc0d07f0 0x7ff4dc06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:18.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4dbfff640 1 --2- 172.21.15.37:0/1188140218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4dc15da80 0x7ff4dc0d4230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0097c0 tx=0x7ff4cc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d254b78e269184d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4daffd640 1 -- 172.21.15.37:0/1188140218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0b1070 con 0x7ff4dc15da80 2024-04-25T08:53:18.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4daffd640 1 -- 172.21.15.37:0/1188140218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc0ac5f0 con 0x7ff4dc15da80 2024-04-25T08:53:18.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4daffd640 1 -- 172.21.15.37:0/1188140218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc0ac9c0 con 0x7ff4dc15da80 2024-04-25T08:53:18.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4e2774640 1 -- 172.21.15.37:0/1188140218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4dc15da80 msgr2=0x7ff4dc0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4e2774640 1 --2- 172.21.15.37:0/1188140218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4dc15da80 0x7ff4dc0d4230 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0097c0 tx=0x7ff4cc01c430 comp rx=0 tx=0).stop 2024-04-25T08:53:18.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4e2774640 1 -- 172.21.15.37:0/1188140218 shutdown_connections 2024-04-25T08:53:18.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4e2774640 1 --2- 172.21.15.37:0/1188140218 >> v2:172.21.15.203:3300/0 conn(0x7ff4dc0d07f0 0x7ff4dc06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4e2774640 1 --2- 172.21.15.37:0/1188140218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4dc15da80 0x7ff4dc0d4230 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4e2774640 1 -- 172.21.15.37:0/1188140218 >> 172.21.15.37:0/1188140218 conn(0x7ff4dc08b000 msgr2=0x7ff4dc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4e2774640 1 -- 172.21.15.37:0/1188140218 shutdown_connections 2024-04-25T08:53:18.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4e2774640 1 -- 172.21.15.37:0/1188140218 wait complete. 2024-04-25T08:53:18.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4e2774640 1 Processor -- start 2024-04-25T08:53:18.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.596+0000 7ff4e2774640 1 -- start start 2024-04-25T08:53:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4e2774640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4dc0d07f0 0x7ff4dc15bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4e2774640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4dc0d1cd0 0x7ff4dc0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4e2774640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4dc15da80 0x7ff4dc0cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4e2774640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc06d7a0 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4e2774640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc06d620 con 0x7ff4dc15da80 2024-04-25T08:53:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4e2774640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4dc06d920 con 0x7ff4dc0d07f0 2024-04-25T08:53:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4dc0d1cd0 0x7ff4dc0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4dc0d1cd0 0x7ff4dc0cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37854/0 (socket says 172.21.15.37:37854) 2024-04-25T08:53:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4db7fe640 1 -- 172.21.15.37:0/2754229777 learned_addr learned my addr 172.21.15.37:0/2754229777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4db7fe640 1 -- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4dc0d07f0 msgr2=0x7ff4dc15bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4db7fe640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4dc0d07f0 0x7ff4dc15bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4e0d11640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4dc15da80 0x7ff4dc0cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4db7fe640 1 -- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4dc15da80 msgr2=0x7ff4dc0cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4db7fe640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4dc15da80 0x7ff4dc0cf0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4db7fe640 1 -- 172.21.15.37:0/2754229777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4dc15c5b0 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4db7fe640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4dc0d1cd0 0x7ff4dc0cebb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400c730 tx=0x7ff4c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4d8ff9640 1 -- 172.21.15.37:0/2754229777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4007270 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4d8ff9640 1 -- 172.21.15.37:0/2754229777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c4007410 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4e2774640 1 -- 172.21.15.37:0/2754229777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4dc15e2b0 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4d8ff9640 1 -- 172.21.15.37:0/2754229777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c40057b0 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4e2774640 1 -- 172.21.15.37:0/2754229777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4dc15bb00 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4e2774640 1 -- 172.21.15.37:0/2754229777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4dc04efa0 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4d8ff9640 1 -- 172.21.15.37:0/2754229777 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4c4005950 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4d8ff9640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4bc066f90 0x7ff4bc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4d8ff9640 1 -- 172.21.15.37:0/2754229777 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff4c40850a0 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.600+0000 7ff4dbfff640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4bc066f90 0x7ff4bc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.604+0000 7ff4dbfff640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4bc066f90 0x7ff4bc069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0ae000 tx=0x7ff4cc004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.604+0000 7ff4d8ff9640 1 -- 172.21.15.37:0/2754229777 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4c4055cb0 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 -- 172.21.15.37:0/2754229777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4dc157130 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4d8ff9640 1 -- 172.21.15.37:0/2754229777 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4c40555b0 con 0x7ff4dc0d1cd0 2024-04-25T08:53:18.756 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:18.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 -- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4bc066f90 msgr2=0x7ff4bc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4bc066f90 0x7ff4bc069450 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc0ae000 tx=0x7ff4cc004740 comp rx=0 tx=0).stop 2024-04-25T08:53:18.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 -- 172.21.15.37:0/2754229777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4dc0d1cd0 msgr2=0x7ff4dc0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4dc0d1cd0 0x7ff4dc0cebb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff4c400c730 tx=0x7ff4c400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 -- 172.21.15.37:0/2754229777 shutdown_connections 2024-04-25T08:53:18.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4bc066f90 0x7ff4bc069450 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4dc15da80 0x7ff4dc0cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4dc0d1cd0 0x7ff4dc0cebb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 --2- 172.21.15.37:0/2754229777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4dc0d07f0 0x7ff4dc15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 -- 172.21.15.37:0/2754229777 >> 172.21.15.37:0/2754229777 conn(0x7ff4dc08b000 msgr2=0x7ff4dc15ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 -- 172.21.15.37:0/2754229777 shutdown_connections 2024-04-25T08:53:18.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.764+0000 7ff4e2774640 1 -- 172.21.15.37:0/2754229777 wait complete. 2024-04-25T08:53:18.774 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.896+0000 7f75fdb2d640 1 Processor -- start 2024-04-25T08:53:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.896+0000 7f75fdb2d640 1 -- start start 2024-04-25T08:53:18.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.896+0000 7f75fdb2d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75f815b420 0x7f75f80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.896+0000 7f75fdb2d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f75f815b800 0x7f75f80cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.896+0000 7f75fdb2d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f75f80cfad0 0x7f75f815f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.896+0000 7f75fdb2d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f80cff20 con 0x7f75f815b800 2024-04-25T08:53:18.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.896+0000 7f75fdb2d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f80cc930 con 0x7f75f80cfad0 2024-04-25T08:53:18.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.896+0000 7f75fdb2d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80cbd20 con 0x7f75f815b420 2024-04-25T08:53:18.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f75f815b800 0x7f75f80cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f75f815b800 0x7f75f80cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37870/0 (socket says 172.21.15.37:37870) 2024-04-25T08:53:18.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f6ffd640 1 -- 172.21.15.37:0/2677546169 learned_addr learned my addr 172.21.15.37:0/2677546169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:18.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f7fff640 1 --2- 172.21.15.37:0/2677546169 >> v2:172.21.15.203:3300/0 conn(0x7f75f80cfad0 0x7f75f815f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f77fe640 1 --2- 172.21.15.37:0/2677546169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75f815b420 0x7f75f80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f6ffd640 1 -- 172.21.15.37:0/2677546169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75f815b420 msgr2=0x7f75f80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f6ffd640 1 --2- 172.21.15.37:0/2677546169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75f815b420 0x7f75f80ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f6ffd640 1 -- 172.21.15.37:0/2677546169 >> v2:172.21.15.203:3300/0 conn(0x7f75f80cfad0 msgr2=0x7f75f815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:18.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f6ffd640 1 --2- 172.21.15.37:0/2677546169 >> v2:172.21.15.203:3300/0 conn(0x7f75f80cfad0 0x7f75f815f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f6ffd640 1 -- 172.21.15.37:0/2677546169 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f80cc000 con 0x7f75f815b800 2024-04-25T08:53:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f6ffd640 1 --2- 172.21.15.37:0/2677546169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75f815b800 0x7f75f80cf590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f75f80d0440 tx=0x7f75ec01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fb2f8ae3f3099 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f67fc640 1 -- 172.21.15.37:0/2677546169 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0b0070 con 0x7f75f815b800 2024-04-25T08:53:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f67fc640 1 -- 172.21.15.37:0/2677546169 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ec0ab600 con 0x7f75f815b800 2024-04-25T08:53:18.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75f67fc640 1 -- 172.21.15.37:0/2677546169 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ec0ab9d0 con 0x7f75f815b800 2024-04-25T08:53:18.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75fdb2d640 1 -- 172.21.15.37:0/2677546169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75f815b800 msgr2=0x7f75f80cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75fdb2d640 1 --2- 172.21.15.37:0/2677546169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75f815b800 0x7f75f80cf590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f75f80d0440 tx=0x7f75ec01c230 comp rx=0 tx=0).stop 2024-04-25T08:53:18.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75fdb2d640 1 -- 172.21.15.37:0/2677546169 shutdown_connections 2024-04-25T08:53:18.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75fdb2d640 1 --2- 172.21.15.37:0/2677546169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75f815b800 0x7f75f80cf590 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75fdb2d640 1 --2- 172.21.15.37:0/2677546169 >> v2:172.21.15.203:3300/0 conn(0x7f75f80cfad0 0x7f75f815f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75fdb2d640 1 --2- 172.21.15.37:0/2677546169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75f815b420 0x7f75f80ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75fdb2d640 1 -- 172.21.15.37:0/2677546169 >> 172.21.15.37:0/2677546169 conn(0x7f75f806ae60 msgr2=0x7f75f806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:18.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75fdb2d640 1 -- 172.21.15.37:0/2677546169 shutdown_connections 2024-04-25T08:53:18.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.900+0000 7f75fdb2d640 1 -- 172.21.15.37:0/2677546169 wait complete. 2024-04-25T08:53:18.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75fdb2d640 1 Processor -- start 2024-04-25T08:53:18.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75fdb2d640 1 -- start start 2024-04-25T08:53:18.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75fdb2d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75f80cfad0 0x7f75f80ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75fdb2d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75f815b420 0x7f75f8098a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75fdb2d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75f815b800 0x7f75f80cea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75fdb2d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80caa80 con 0x7f75f80cfad0 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75fdb2d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80ca900 con 0x7f75f815b800 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75fdb2d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f80cadb0 con 0x7f75f815b420 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75f80cfad0 0x7f75f80ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75f80cfad0 0x7f75f80ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37874/0 (socket says 172.21.15.37:37874) 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f77fe640 1 -- 172.21.15.37:0/302467446 learned_addr learned my addr 172.21.15.37:0/302467446 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f7fff640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75f815b800 0x7f75f80cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f6ffd640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75f815b420 0x7f75f8098a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f77fe640 1 -- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75f815b420 msgr2=0x7f75f8098a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f77fe640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75f815b420 0x7f75f8098a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f77fe640 1 -- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75f815b800 msgr2=0x7f75f80cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:18.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f77fe640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75f815b800 0x7f75f80cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f77fe640 1 -- 172.21.15.37:0/302467446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f8165b70 con 0x7f75f80cfad0 2024-04-25T08:53:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f6ffd640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75f815b420 0x7f75f8098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75f77fe640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75f80cfad0 0x7f75f80ccf30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f75e800e9b0 tx=0x7f75e800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75dffff640 1 -- 172.21.15.37:0/302467446 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e8017070 con 0x7f75f80cfad0 2024-04-25T08:53:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75dffff640 1 -- 172.21.15.37:0/302467446 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e800c9a0 con 0x7f75f80cfad0 2024-04-25T08:53:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75dffff640 1 -- 172.21.15.37:0/302467446 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e800cd70 con 0x7f75f80cfad0 2024-04-25T08:53:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75fdb2d640 1 -- 172.21.15.37:0/302467446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f8165290 con 0x7f75f80cfad0 2024-04-25T08:53:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75fdb2d640 1 -- 172.21.15.37:0/302467446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f80b3240 con 0x7f75f80cfad0 2024-04-25T08:53:18.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75fdb2d640 1 -- 172.21.15.37:0/302467446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f804efa0 con 0x7f75f80cfad0 2024-04-25T08:53:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.904+0000 7f75dffff640 1 -- 172.21.15.37:0/302467446 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75e801a070 con 0x7f75f80cfad0 2024-04-25T08:53:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.908+0000 7f75dffff640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75e4066f90 0x7f75e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.908+0000 7f75dffff640 1 -- 172.21.15.37:0/302467446 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f75e8085650 con 0x7f75f80cfad0 2024-04-25T08:53:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.908+0000 7f75f6ffd640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75e4066f90 0x7f75e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:18.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.908+0000 7f75f6ffd640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75e4066f90 0x7f75e4069450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0ad000 tx=0x7f75ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:18.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:18.912+0000 7f75dffff640 1 -- 172.21.15.37:0/302467446 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75e8021030 con 0x7f75f80cfad0 2024-04-25T08:53:19.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.068+0000 7f75fdb2d640 1 -- 172.21.15.37:0/302467446 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f75f80ce710 con 0x7f75f80cfad0 2024-04-25T08:53:19.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.068+0000 7f75dffff640 1 -- 172.21.15.37:0/302467446 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f75e80562f0 con 0x7f75f80cfad0 2024-04-25T08:53:19.061 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:19.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.068+0000 7f75fdb2d640 1 -- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75e4066f90 msgr2=0x7f75e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.068+0000 7f75fdb2d640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75e4066f90 0x7f75e4069450 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f75ec0ad000 tx=0x7f75ec008040 comp rx=0 tx=0).stop 2024-04-25T08:53:19.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.068+0000 7f75fdb2d640 1 -- 172.21.15.37:0/302467446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75f80cfad0 msgr2=0x7f75f80ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.068+0000 7f75fdb2d640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75f80cfad0 0x7f75f80ccf30 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f75e800e9b0 tx=0x7f75e800ee70 comp rx=0 tx=0).stop 2024-04-25T08:53:19.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.068+0000 7f75fdb2d640 1 -- 172.21.15.37:0/302467446 shutdown_connections 2024-04-25T08:53:19.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.068+0000 7f75fdb2d640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75e4066f90 0x7f75e4069450 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.068+0000 7f75fdb2d640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75f815b800 0x7f75f80cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.068+0000 7f75fdb2d640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75f815b420 0x7f75f8098a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.068+0000 7f75fdb2d640 1 --2- 172.21.15.37:0/302467446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75f80cfad0 0x7f75f80ccf30 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.072+0000 7f75fdb2d640 1 -- 172.21.15.37:0/302467446 >> 172.21.15.37:0/302467446 conn(0x7f75f806ae60 msgr2=0x7f75f80aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.072+0000 7f75fdb2d640 1 -- 172.21.15.37:0/302467446 shutdown_connections 2024-04-25T08:53:19.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.072+0000 7f75fdb2d640 1 -- 172.21.15.37:0/302467446 wait complete. 2024-04-25T08:53:19.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:19.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13cbdbb640 1 Processor -- start 2024-04-25T08:53:19.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13cbdbb640 1 -- start start 2024-04-25T08:53:19.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13cbdbb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 0x7f13c415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13cbdbb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13c406c3b0 0x7f13c406df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13cbdbb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c4066c30 con 0x7f13c40b2c30 2024-04-25T08:53:19.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13cbdbb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c406daa0 con 0x7f13c406c3b0 2024-04-25T08:53:19.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13cbdbb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40af660 con 0x7f13c40af280 2024-04-25T08:53:19.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13c9b57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 0x7f13c415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13c9b57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 0x7f13c415df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:51350/0 (socket says 172.21.15.37:51350) 2024-04-25T08:53:19.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13c9b57640 1 -- 172.21.15.37:0/3140218198 learned_addr learned my addr 172.21.15.37:0/3140218198 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:19.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13ca358640 1 --2- 172.21.15.37:0/3140218198 >> v2:172.21.15.203:3300/0 conn(0x7f13c406c3b0 0x7f13c406df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13c9b57640 1 -- 172.21.15.37:0/3140218198 >> v2:172.21.15.203:3300/0 conn(0x7f13c406c3b0 msgr2=0x7f13c406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:19.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13c9b57640 1 --2- 172.21.15.37:0/3140218198 >> v2:172.21.15.203:3300/0 conn(0x7f13c406c3b0 0x7f13c406df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13c9b57640 1 -- 172.21.15.37:0/3140218198 >> v1:172.21.15.203:6789/0 conn(0x7f13c40b2c30 legacy=0x7f13c406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.196+0000 7f13c9b57640 1 -- 172.21.15.37:0/3140218198 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c415e640 con 0x7f13c40af280 2024-04-25T08:53:19.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13c9b57640 1 --2- 172.21.15.37:0/3140218198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 0x7f13c415df10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009870 tx=0x7f13c001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b6acd0093c93041 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13c8b55640 1 -- 172.21.15.37:0/3140218198 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00b2070 con 0x7f13c40af280 2024-04-25T08:53:19.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13c8b55640 1 -- 172.21.15.37:0/3140218198 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c00a9070 con 0x7f13c40af280 2024-04-25T08:53:19.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13c8b55640 1 -- 172.21.15.37:0/3140218198 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00ad880 con 0x7f13c40af280 2024-04-25T08:53:19.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13cbdbb640 1 -- 172.21.15.37:0/3140218198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 msgr2=0x7f13c415df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13cbdbb640 1 --2- 172.21.15.37:0/3140218198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 0x7f13c415df10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009870 tx=0x7f13c001c560 comp rx=0 tx=0).stop 2024-04-25T08:53:19.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13cbdbb640 1 -- 172.21.15.37:0/3140218198 shutdown_connections 2024-04-25T08:53:19.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13cbdbb640 1 --2- 172.21.15.37:0/3140218198 >> v2:172.21.15.203:3300/0 conn(0x7f13c406c3b0 0x7f13c406df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13cbdbb640 1 --2- 172.21.15.37:0/3140218198 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 0x7f13c415df10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13cbdbb640 1 -- 172.21.15.37:0/3140218198 >> 172.21.15.37:0/3140218198 conn(0x7f13c4071810 msgr2=0x7f13c4071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13cbdbb640 1 -- 172.21.15.37:0/3140218198 shutdown_connections 2024-04-25T08:53:19.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13cbdbb640 1 -- 172.21.15.37:0/3140218198 wait complete. 2024-04-25T08:53:19.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13cbdbb640 1 Processor -- start 2024-04-25T08:53:19.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.200+0000 7f13cbdbb640 1 -- start start 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13cbdbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c406c3b0 0x7f13c40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13cbdbb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 0x7f13c40ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13cbdbb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13c40b2c30 0x7f13c414d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13cbdbb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c406f010 con 0x7f13c406c3b0 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13cbdbb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c406ee90 con 0x7f13c40b2c30 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13cbdbb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c406f190 con 0x7f13c40af280 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9b57640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c406c3b0 0x7f13c40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9b57640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c406c3b0 0x7f13c40cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37878/0 (socket says 172.21.15.37:37878) 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9b57640 1 -- 172.21.15.37:0/4237542286 learned_addr learned my addr 172.21.15.37:0/4237542286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13ca358640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13c40b2c30 0x7f13c414d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9356640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 0x7f13c40ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9b57640 1 -- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 msgr2=0x7f13c40ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9b57640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 0x7f13c40ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9b57640 1 -- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13c40b2c30 msgr2=0x7f13c414d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9b57640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13c40b2c30 0x7f13c414d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9b57640 1 -- 172.21.15.37:0/4237542286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c40adc80 con 0x7f13c406c3b0 2024-04-25T08:53:19.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9356640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 0x7f13c40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:19.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9b57640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c406c3b0 0x7f13c40cc8d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13c001a190 tx=0x7f13c001ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13ba7fc640 1 -- 172.21.15.37:0/4237542286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00b2040 con 0x7f13c406c3b0 2024-04-25T08:53:19.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13ba7fc640 1 -- 172.21.15.37:0/4237542286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c00adb80 con 0x7f13c406c3b0 2024-04-25T08:53:19.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13cbdbb640 1 -- 172.21.15.37:0/4237542286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c40adef0 con 0x7f13c406c3b0 2024-04-25T08:53:19.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13ba7fc640 1 -- 172.21.15.37:0/4237542286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c00b8420 con 0x7f13c406c3b0 2024-04-25T08:53:19.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13cbdbb640 1 -- 172.21.15.37:0/4237542286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c40ae4b0 con 0x7f13c406c3b0 2024-04-25T08:53:19.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13cbdbb640 1 -- 172.21.15.37:0/4237542286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c404efa0 con 0x7f13c406c3b0 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13ba7fc640 1 -- 172.21.15.37:0/4237542286 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13c00c0030 con 0x7f13c406c3b0 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13ba7fc640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a8066f60 0x7f13a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13ba7fc640 1 -- 172.21.15.37:0/4237542286 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f13c011ecd0 con 0x7f13c406c3b0 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.204+0000 7f13c9356640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a8066f60 0x7f13a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.208+0000 7f13c9356640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a8066f60 0x7f13a8069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f13c414f160 tx=0x7f13b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.208+0000 7f13ba7fc640 1 -- 172.21.15.37:0/4237542286 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13c00c7060 con 0x7f13c406c3b0 2024-04-25T08:53:19.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.368+0000 7f13cbdbb640 1 -- 172.21.15.37:0/4237542286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13c40736c0 con 0x7f13c406c3b0 2024-04-25T08:53:19.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.368+0000 7f13ba7fc640 1 -- 172.21.15.37:0/4237542286 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f13c0016150 con 0x7f13c406c3b0 2024-04-25T08:53:19.364 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:19.364 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 -- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a8066f60 msgr2=0x7f13a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a8066f60 0x7f13a8069420 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f13c414f160 tx=0x7f13b4009040 comp rx=0 tx=0).stop 2024-04-25T08:53:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 -- 172.21.15.37:0/4237542286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c406c3b0 msgr2=0x7f13c40cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c406c3b0 0x7f13c40cc8d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f13c001a190 tx=0x7f13c001ada0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 -- 172.21.15.37:0/4237542286 shutdown_connections 2024-04-25T08:53:19.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a8066f60 0x7f13a8069420 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13c40b2c30 0x7f13c414d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40af280 0x7f13c40ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 --2- 172.21.15.37:0/4237542286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c406c3b0 0x7f13c40cc8d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 -- 172.21.15.37:0/4237542286 >> 172.21.15.37:0/4237542286 conn(0x7f13c4071810 msgr2=0x7f13c40b10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 -- 172.21.15.37:0/4237542286 shutdown_connections 2024-04-25T08:53:19.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.372+0000 7f13cbdbb640 1 -- 172.21.15.37:0/4237542286 wait complete. 2024-04-25T08:53:19.383 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:19.385 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:19.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1abf1640 1 Processor -- start 2024-04-25T08:53:19.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1abf1640 1 -- start start 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1abf1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ecb0 0x7efe140c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1abf1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efe140c9ef0 0x7efe140ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1abf1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe140ca4c0 con 0x7efe140cf590 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1abf1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe140ca780 con 0x7efe140c9ef0 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1abf1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1415f0f0 con 0x7efe1415ecb0 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1898d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7efe140cf590 0x7efe140cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41718/0 (socket says 172.21.15.37:41718) 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1898d640 1 -- 172.21.15.37:0/4205311243 learned_addr learned my addr 172.21.15.37:0/4205311243 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1918e640 1 --2- 172.21.15.37:0/4205311243 >> v2:172.21.15.37:3300/0 conn(0x7efe140c9ef0 0x7efe140ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe137fe640 1 -- 172.21.15.37:0/4205311243 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777829290 0 0) 0x7efe140ca4c0 con 0x7efe140cf590 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe137fe640 1 -- 172.21.15.37:0/4205311243 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdfc005740 con 0x7efe140cf590 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe13fff640 1 --2- 172.21.15.37:0/4205311243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ecb0 0x7efe140c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1918e640 1 -- 172.21.15.37:0/4205311243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ecb0 msgr2=0x7efe140c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1918e640 1 --2- 172.21.15.37:0/4205311243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ecb0 0x7efe140c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1918e640 1 -- 172.21.15.37:0/4205311243 >> v1:172.21.15.37:6789/0 conn(0x7efe140cf590 legacy=0x7efe140cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe1918e640 1 -- 172.21.15.37:0/4205311243 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe140cd690 con 0x7efe140c9ef0 2024-04-25T08:53:19.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.520+0000 7efe13fff640 1 --2- 172.21.15.37:0/4205311243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ecb0 0x7efe140c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:19.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe1918e640 1 --2- 172.21.15.37:0/4205311243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe140c9ef0 0x7efe140ce170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efe040097f0 tx=0x7efe0401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54a9440116f683e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe137fe640 1 -- 172.21.15.37:0/4205311243 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe040b0070 con 0x7efe140c9ef0 2024-04-25T08:53:19.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe137fe640 1 -- 172.21.15.37:0/4205311243 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe040ab600 con 0x7efe140c9ef0 2024-04-25T08:53:19.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe137fe640 1 -- 172.21.15.37:0/4205311243 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe040ab9d0 con 0x7efe140c9ef0 2024-04-25T08:53:19.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe1abf1640 1 -- 172.21.15.37:0/4205311243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe140c9ef0 msgr2=0x7efe140ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe1abf1640 1 --2- 172.21.15.37:0/4205311243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe140c9ef0 0x7efe140ce170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efe040097f0 tx=0x7efe0401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe1abf1640 1 -- 172.21.15.37:0/4205311243 shutdown_connections 2024-04-25T08:53:19.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe1abf1640 1 --2- 172.21.15.37:0/4205311243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe140c9ef0 0x7efe140ce170 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe1abf1640 1 --2- 172.21.15.37:0/4205311243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ecb0 0x7efe140c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe1abf1640 1 -- 172.21.15.37:0/4205311243 >> 172.21.15.37:0/4205311243 conn(0x7efe1406ae60 msgr2=0x7efe1406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe1abf1640 1 -- 172.21.15.37:0/4205311243 shutdown_connections 2024-04-25T08:53:19.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe1abf1640 1 -- 172.21.15.37:0/4205311243 wait complete. 2024-04-25T08:53:19.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe1abf1640 1 Processor -- start 2024-04-25T08:53:19.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.524+0000 7efe1abf1640 1 -- start start 2024-04-25T08:53:19.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1abf1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe140c9ef0 0x7efe140bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1abf1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe140cf590 0x7efe140c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1abf1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ecb0 0x7efe1415d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1abf1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140cc170 con 0x7efe140c9ef0 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1abf1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe140cbff0 con 0x7efe140cf590 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1abf1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe140cc2f0 con 0x7efe1415ecb0 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1898d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe140c9ef0 0x7efe140bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1898d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe140c9ef0 0x7efe140bfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41270/0 (socket says 172.21.15.37:41270) 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1898d640 1 -- 172.21.15.37:0/3548862968 learned_addr learned my addr 172.21.15.37:0/3548862968 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe13fff640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe140cf590 0x7efe140c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1918e640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ecb0 0x7efe1415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1898d640 1 -- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ecb0 msgr2=0x7efe1415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1898d640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ecb0 0x7efe1415d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1898d640 1 -- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe140cf590 msgr2=0x7efe140c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1898d640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe140cf590 0x7efe140c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1898d640 1 -- 172.21.15.37:0/3548862968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe140c5850 con 0x7efe140c9ef0 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe13fff640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe140cf590 0x7efe140c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:19.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1898d640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe140c9ef0 0x7efe140bfea0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00cb70 tx=0x7efe0c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe117fa640 1 -- 172.21.15.37:0/3548862968 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c017070 con 0x7efe140c9ef0 2024-04-25T08:53:19.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe117fa640 1 -- 172.21.15.37:0/3548862968 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0c00ad40 con 0x7efe140c9ef0 2024-04-25T08:53:19.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1abf1640 1 -- 172.21.15.37:0/3548862968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe140c5290 con 0x7efe140c9ef0 2024-04-25T08:53:19.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe117fa640 1 -- 172.21.15.37:0/3548862968 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0c012620 con 0x7efe140c9ef0 2024-04-25T08:53:19.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1abf1640 1 -- 172.21.15.37:0/3548862968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe140c0880 con 0x7efe140c9ef0 2024-04-25T08:53:19.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe1abf1640 1 -- 172.21.15.37:0/3548862968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1404efa0 con 0x7efe140c9ef0 2024-04-25T08:53:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe117fa640 1 -- 172.21.15.37:0/3548862968 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe0c01a070 con 0x7efe140c9ef0 2024-04-25T08:53:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe117fa640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe00066f60 0x7efe00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe117fa640 1 -- 172.21.15.37:0/3548862968 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7efe0c0851f0 con 0x7efe140c9ef0 2024-04-25T08:53:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.528+0000 7efe13fff640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe00066f60 0x7efe00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.532+0000 7efe13fff640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe00066f60 0x7efe00069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe14156bd0 tx=0x7efe08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.532+0000 7efe117fa640 1 -- 172.21.15.37:0/3548862968 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe0c055e90 con 0x7efe140c9ef0 2024-04-25T08:53:19.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.688+0000 7efe1abf1640 1 -- 172.21.15.37:0/3548862968 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efe140ae980 con 0x7efe140c9ef0 2024-04-25T08:53:19.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.688+0000 7efe117fa640 1 -- 172.21.15.37:0/3548862968 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7efe0c055790 con 0x7efe140c9ef0 2024-04-25T08:53:19.681 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 -- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe00066f60 msgr2=0x7efe00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe00066f60 0x7efe00069420 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe14156bd0 tx=0x7efe08008040 comp rx=0 tx=0).stop 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 -- 172.21.15.37:0/3548862968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe140c9ef0 msgr2=0x7efe140bfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe140c9ef0 0x7efe140bfea0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efe0c00cb70 tx=0x7efe0c00a430 comp rx=0 tx=0).stop 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 -- 172.21.15.37:0/3548862968 shutdown_connections 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe00066f60 0x7efe00069420 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe1415ecb0 0x7efe1415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe140cf590 0x7efe140c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 --2- 172.21.15.37:0/3548862968 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe140c9ef0 0x7efe140bfea0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 -- 172.21.15.37:0/3548862968 >> 172.21.15.37:0/3548862968 conn(0x7efe1406ae60 msgr2=0x7efe14072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 -- 172.21.15.37:0/3548862968 shutdown_connections 2024-04-25T08:53:19.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.692+0000 7efe1abf1640 1 -- 172.21.15.37:0/3548862968 wait complete. 2024-04-25T08:53:19.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:19.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f11681de640 1 Processor -- start 2024-04-25T08:53:19.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f11681de640 1 -- start start 2024-04-25T08:53:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f11681de640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116015b6b0 0x7f11600ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f11681de640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116015ff50 con 0x7f11600cf590 2024-04-25T08:53:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f11681de640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11600cbd20 con 0x7f1160073010 2024-04-25T08:53:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f11681de640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116015ba90 con 0x7f116015b6b0 2024-04-25T08:53:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1165779640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f11600cf590 0x7f116015f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41730/0 (socket says 172.21.15.37:41730) 2024-04-25T08:53:19.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1165779640 1 -- 172.21.15.37:0/4040552196 learned_addr learned my addr 172.21.15.37:0/4040552196 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1165f7a640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116015b6b0 0x7f11600ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300478502 0 0) 0x7f116015ff50 con 0x7f11600cf590 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1148003530 con 0x7f11600cf590 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 874524083 0 0) 0x7f1148003530 con 0x7f11600cf590 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116015b6b0 msgr2=0x7f11600ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116015b6b0 0x7f11600ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 >> v1:172.21.15.203:6789/0 conn(0x7f1160073010 legacy=0x7f116015f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11600cc000 con 0x7f11600cf590 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1154001680 con 0x7f11600cf590 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 >> v1:172.21.15.37:6789/0 conn(0x7f11600cf590 legacy=0x7f116015f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1148003a60 0x7f1148003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116015b6b0 0x7f1148004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1148004d50 0x7f11480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116015ba90 con 0x7f1148004d50 2024-04-25T08:53:19.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114800bae0 con 0x7f1148003a60 2024-04-25T08:53:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114800bce0 con 0x7f116015b6b0 2024-04-25T08:53:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f116677b640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1148004d50 0x7f11480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1165f7a640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1148003a60 0x7f1148003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1165779640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116015b6b0 0x7f1148004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f116677b640 1 -- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116015b6b0 msgr2=0x7f1148004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f116677b640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116015b6b0 0x7f1148004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f116677b640 1 -- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1148003a60 msgr2=0x7f1148003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f116677b640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1148003a60 0x7f1148003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f116677b640 1 -- 172.21.15.37:0/4040552196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1148003530 con 0x7f1148004d50 2024-04-25T08:53:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1165f7a640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1148003a60 0x7f1148003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:19.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.832+0000 7f1165779640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116015b6b0 0x7f1148004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f116677b640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1148004d50 0x7f11480051d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1160073480 tx=0x7f115001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66af49fbe188c6c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11500013a0 con 0x7f1148004d50 2024-04-25T08:53:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11500052f0 con 0x7f1148004d50 2024-04-25T08:53:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f1164f78640 1 -- 172.21.15.37:0/4040552196 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150003840 con 0x7f1148004d50 2024-04-25T08:53:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f11681de640 1 -- 172.21.15.37:0/4040552196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1148004d50 msgr2=0x7f11480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f11681de640 1 --2- 172.21.15.37:0/4040552196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1148004d50 0x7f11480051d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f1160073480 tx=0x7f115001c3c0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f116677b640 1 -- 172.21.15.37:0/4040552196 reap_dead start 2024-04-25T08:53:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f11681de640 1 -- 172.21.15.37:0/4040552196 shutdown_connections 2024-04-25T08:53:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f11681de640 1 -- 172.21.15.37:0/4040552196 >> 172.21.15.37:0/4040552196 conn(0x7f116006ae60 msgr2=0x7f116006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f11681de640 1 -- 172.21.15.37:0/4040552196 shutdown_connections 2024-04-25T08:53:19.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f11681de640 1 -- 172.21.15.37:0/4040552196 wait complete. 2024-04-25T08:53:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f11681de640 1 Processor -- start 2024-04-25T08:53:19.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.836+0000 7f11681de640 1 -- start start 2024-04-25T08:53:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f11681de640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1148004d50 0x7f11600caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f11681de640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116015e390 0x7f1160154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f11681de640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116015e810 0x7f1160155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f11681de640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11600cc8a0 con 0x7f116015e390 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f11681de640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11600cc5c0 con 0x7f116015e810 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f11681de640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11600cba30 con 0x7f1148004d50 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165779640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116015e390 0x7f1160154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165779640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116015e390 0x7f1160154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41300/0 (socket says 172.21.15.37:41300) 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165779640 1 -- 172.21.15.37:0/2906024352 learned_addr learned my addr 172.21.15.37:0/2906024352 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f116677b640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116015e810 0x7f1160155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165f7a640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1148004d50 0x7f11600caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165779640 1 -- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1148004d50 msgr2=0x7f11600caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165779640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1148004d50 0x7f11600caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165779640 1 -- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116015e810 msgr2=0x7f1160155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165779640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116015e810 0x7f1160155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165779640 1 -- 172.21.15.37:0/2906024352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116015bf40 con 0x7f116015e390 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f116677b640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116015e810 0x7f1160155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165779640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116015e390 0x7f1160154e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1154000fc0 tx=0x7f115400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f115e7fc640 1 -- 172.21.15.37:0/2906024352 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154019070 con 0x7f116015e390 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f115e7fc640 1 -- 172.21.15.37:0/2906024352 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11540074e0 con 0x7f116015e390 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f11681de640 1 -- 172.21.15.37:0/2906024352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116015c220 con 0x7f116015e390 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f115e7fc640 1 -- 172.21.15.37:0/2906024352 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154004a10 con 0x7f116015e390 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f11681de640 1 -- 172.21.15.37:0/2906024352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11600cd210 con 0x7f116015e390 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f11681de640 1 -- 172.21.15.37:0/2906024352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116004efa0 con 0x7f116015e390 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f115e7fc640 1 -- 172.21.15.37:0/2906024352 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f115401c070 con 0x7f116015e390 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f115e7fc640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f114c066f90 0x7f114c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f115e7fc640 1 -- 172.21.15.37:0/2906024352 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1154084e20 con 0x7f116015e390 2024-04-25T08:53:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165f7a640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f114c066f90 0x7f114c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:19.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.840+0000 7f1165f7a640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f114c066f90 0x7f114c069450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1158007330 tx=0x7f115800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:19.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:19.844+0000 7f115e7fc640 1 -- 172.21.15.37:0/2906024352 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11540553e0 con 0x7f116015e390 2024-04-25T08:53:19.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.004+0000 7f11681de640 1 -- 172.21.15.37:0/2906024352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11600008e0 con 0x7f116015e390 2024-04-25T08:53:19.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.004+0000 7f115e7fc640 1 -- 172.21.15.37:0/2906024352 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1154059480 con 0x7f116015e390 2024-04-25T08:53:19.999 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 -- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f114c066f90 msgr2=0x7f114c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f114c066f90 0x7f114c069450 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1158007330 tx=0x7f115800b040 comp rx=0 tx=0).stop 2024-04-25T08:53:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 -- 172.21.15.37:0/2906024352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116015e390 msgr2=0x7f1160154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116015e390 0x7f1160154e10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1154000fc0 tx=0x7f115400dc90 comp rx=0 tx=0).stop 2024-04-25T08:53:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 -- 172.21.15.37:0/2906024352 shutdown_connections 2024-04-25T08:53:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f114c066f90 0x7f114c069450 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116015e810 0x7f1160155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116015e390 0x7f1160154e10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 --2- 172.21.15.37:0/2906024352 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1148004d50 0x7f11600caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 -- 172.21.15.37:0/2906024352 >> 172.21.15.37:0/2906024352 conn(0x7f116006ae60 msgr2=0x7f11600ad770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 -- 172.21.15.37:0/2906024352 shutdown_connections 2024-04-25T08:53:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.008+0000 7f11681de640 1 -- 172.21.15.37:0/2906024352 wait complete. 2024-04-25T08:53:20.019 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc8223640 1 Processor -- start 2024-04-25T08:53:20.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc8223640 1 -- start start 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc8223640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdfc015d9c0 0x7fdfc00d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc8223640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc00d3df0 0x7fdfc006c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc8223640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc00d1950 con 0x7fdfc00cf270 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc8223640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc00d2520 con 0x7fdfc015d9c0 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc8223640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc00d1bf0 con 0x7fdfc00d3df0 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc57be640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdfc00cf270 0x7fdfc00d36e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41734/0 (socket says 172.21.15.37:41734) 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc57be640 1 -- 172.21.15.37:0/1216752518 learned_addr learned my addr 172.21.15.37:0/1216752518 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc5fbf640 1 --2- 172.21.15.37:0/1216752518 >> v2:172.21.15.203:3300/0 conn(0x7fdfc015d9c0 0x7fdfc00d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc67c0640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc00d3df0 0x7fdfc006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291246857 0 0) 0x7fdfc00d1950 con 0x7fdfc00cf270 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfac003530 con 0x7fdfc00cf270 2024-04-25T08:53:20.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2092877110 0 0) 0x7fdfac003530 con 0x7fdfc00cf270 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc00d3df0 msgr2=0x7fdfc006c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc00d3df0 0x7fdfc006c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 >> v2:172.21.15.203:3300/0 conn(0x7fdfc015d9c0 msgr2=0x7fdfc00d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 --2- 172.21.15.37:0/1216752518 >> v2:172.21.15.203:3300/0 conn(0x7fdfc015d9c0 0x7fdfc00d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc00d1ed0 con 0x7fdfc00cf270 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fdfb4001820 con 0x7fdfc00cf270 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 >> v1:172.21.15.37:6789/0 conn(0x7fdfc00cf270 legacy=0x7fdfc00d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc00d3df0 0x7fdfac003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac003fb0 0x7fdfac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfac004d50 0x7fdfac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc00d2520 con 0x7fdfac003fb0 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc00d1bf0 con 0x7fdfac004d50 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfac00ba00 con 0x7fdfc00d3df0 2024-04-25T08:53:20.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc57be640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac003fb0 0x7fdfac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc5fbf640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc00d3df0 0x7fdfac003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.156+0000 7fdfc67c0640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfac004d50 0x7fdfac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc57be640 1 -- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc00d3df0 msgr2=0x7fdfac003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc57be640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc00d3df0 0x7fdfac003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc57be640 1 -- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfac004d50 msgr2=0x7fdfac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc57be640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfac004d50 0x7fdfac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc57be640 1 -- 172.21.15.37:0/1216752518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfac003530 con 0x7fdfac003fb0 2024-04-25T08:53:20.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc67c0640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfac004d50 0x7fdfac0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:20.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc57be640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac003fb0 0x7fdfac004810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400c210 tx=0x7fdfb401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4784f829b79791e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4007d80 con 0x7fdfac003fb0 2024-04-25T08:53:20.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb4007290 con 0x7fdfac003fb0 2024-04-25T08:53:20.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc4fbd640 1 -- 172.21.15.37:0/1216752518 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40057b0 con 0x7fdfac003fb0 2024-04-25T08:53:20.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc8223640 1 -- 172.21.15.37:0/1216752518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac003fb0 msgr2=0x7fdfac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc8223640 1 --2- 172.21.15.37:0/1216752518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac003fb0 0x7fdfac004810 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400c210 tx=0x7fdfb401f3f0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc67c0640 1 -- 172.21.15.37:0/1216752518 reap_dead start 2024-04-25T08:53:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc8223640 1 -- 172.21.15.37:0/1216752518 shutdown_connections 2024-04-25T08:53:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc8223640 1 -- 172.21.15.37:0/1216752518 >> 172.21.15.37:0/1216752518 conn(0x7fdfc009cad0 msgr2=0x7fdfc008b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc8223640 1 -- 172.21.15.37:0/1216752518 shutdown_connections 2024-04-25T08:53:20.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.160+0000 7fdfc8223640 1 -- 172.21.15.37:0/1216752518 wait complete. 2024-04-25T08:53:20.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc8223640 1 Processor -- start 2024-04-25T08:53:20.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc8223640 1 -- start start 2024-04-25T08:53:20.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc8223640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfac003fb0 0x7fdfc01e81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc8223640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc015ad20 0x7fdfc01e6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc8223640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc014a920 0x7fdfc01f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc8223640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc00cf670 con 0x7fdfc015ad20 2024-04-25T08:53:20.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc8223640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc015dda0 con 0x7fdfc014a920 2024-04-25T08:53:20.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc8223640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc00cf7f0 con 0x7fdfac003fb0 2024-04-25T08:53:20.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc57be640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc015ad20 0x7fdfc01e6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc57be640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc015ad20 0x7fdfc01e6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41306/0 (socket says 172.21.15.37:41306) 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc57be640 1 -- 172.21.15.37:0/3949125257 learned_addr learned my addr 172.21.15.37:0/3949125257 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc67c0640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc014a920 0x7fdfc01f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc5fbf640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfac003fb0 0x7fdfc01e81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc57be640 1 -- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfac003fb0 msgr2=0x7fdfc01e81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc57be640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfac003fb0 0x7fdfc01e81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc57be640 1 -- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc014a920 msgr2=0x7fdfc01f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc57be640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc014a920 0x7fdfc01f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc57be640 1 -- 172.21.15.37:0/3949125257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc01e7120 con 0x7fdfc015ad20 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc67c0640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc014a920 0x7fdfc01f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc57be640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc015ad20 0x7fdfc01e6a20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdfb401f220 tx=0x7fdfac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfb27fc640 1 -- 172.21.15.37:0/3949125257 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40070f0 con 0x7fdfc015ad20 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfb27fc640 1 -- 172.21.15.37:0/3949125257 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb40017e0 con 0x7fdfc015ad20 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc8223640 1 -- 172.21.15.37:0/3949125257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc01e7400 con 0x7fdfc015ad20 2024-04-25T08:53:20.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfb27fc640 1 -- 172.21.15.37:0/3949125257 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40059f0 con 0x7fdfc015ad20 2024-04-25T08:53:20.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc8223640 1 -- 172.21.15.37:0/3949125257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc01e79c0 con 0x7fdfc015ad20 2024-04-25T08:53:20.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.164+0000 7fdfc8223640 1 -- 172.21.15.37:0/3949125257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc004efa0 con 0x7fdfc015ad20 2024-04-25T08:53:20.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.168+0000 7fdfb27fc640 1 -- 172.21.15.37:0/3949125257 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdfb4005b90 con 0x7fdfc015ad20 2024-04-25T08:53:20.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.168+0000 7fdfb27fc640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfa4066f90 0x7fdfa4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.168+0000 7fdfb27fc640 1 -- 172.21.15.37:0/3949125257 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fdfb4122820 con 0x7fdfc015ad20 2024-04-25T08:53:20.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.168+0000 7fdfc5fbf640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfa4066f90 0x7fdfa4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.168+0000 7fdfc5fbf640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfa4066f90 0x7fdfa4069450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0099d0 tx=0x7fdfbc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.172+0000 7fdfb27fc640 1 -- 172.21.15.37:0/3949125257 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdfb4019100 con 0x7fdfc015ad20 2024-04-25T08:53:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.332+0000 7fdfc8223640 1 -- 172.21.15.37:0/3949125257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdfc00cb5a0 con 0x7fdfc015ad20 2024-04-25T08:53:20.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.332+0000 7fdfb27fc640 1 -- 172.21.15.37:0/3949125257 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdfb40f34c0 con 0x7fdfc015ad20 2024-04-25T08:53:20.325 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.332+0000 7fdfc8223640 1 -- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfa4066f90 msgr2=0x7fdfa4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.336+0000 7fdfc8223640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfa4066f90 0x7fdfa4069450 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0099d0 tx=0x7fdfbc008040 comp rx=0 tx=0).stop 2024-04-25T08:53:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.336+0000 7fdfc8223640 1 -- 172.21.15.37:0/3949125257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc015ad20 msgr2=0x7fdfc01e6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.336+0000 7fdfc8223640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc015ad20 0x7fdfc01e6a20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdfb401f220 tx=0x7fdfac005840 comp rx=0 tx=0).stop 2024-04-25T08:53:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.336+0000 7fdfc8223640 1 -- 172.21.15.37:0/3949125257 shutdown_connections 2024-04-25T08:53:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.336+0000 7fdfc8223640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfa4066f90 0x7fdfa4069450 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.336+0000 7fdfc8223640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc014a920 0x7fdfc01f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.336+0000 7fdfc8223640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc015ad20 0x7fdfc01e6a20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.336+0000 7fdfc8223640 1 --2- 172.21.15.37:0/3949125257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfac003fb0 0x7fdfc01e81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.336+0000 7fdfc8223640 1 -- 172.21.15.37:0/3949125257 >> 172.21.15.37:0/3949125257 conn(0x7fdfc009cad0 msgr2=0x7fdfc00cf9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.336+0000 7fdfc8223640 1 -- 172.21.15.37:0/3949125257 shutdown_connections 2024-04-25T08:53:20.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.336+0000 7fdfc8223640 1 -- 172.21.15.37:0/3949125257 wait complete. 2024-04-25T08:53:20.344 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:20.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f289a6b3640 1 Processor -- start 2024-04-25T08:53:20.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f289a6b3640 1 -- start start 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f289a6b3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28940b3cf0 0x7f28940cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f289a6b3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28940cbf10 con 0x7f289415e6c0 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f289a6b3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28940cc1f0 con 0x7f28940cfad0 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f289a6b3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28940cff10 con 0x7f28940b3cf0 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2898c50640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f289415e6c0 0x7f28940caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41744/0 (socket says 172.21.15.37:41744) 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2898c50640 1 -- 172.21.15.37:0/4248063748 learned_addr learned my addr 172.21.15.37:0/4248063748 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2893fff640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28940b3cf0 0x7f28940cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401732844 0 0) 0x7f28940cbf10 con 0x7f289415e6c0 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f287c003530 con 0x7f289415e6c0 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4242657426 0 0) 0x7f287c003530 con 0x7f289415e6c0 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28940b3cf0 msgr2=0x7f28940cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28940b3cf0 0x7f28940cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 >> v1:172.21.15.203:6789/0 conn(0x7f28940cfad0 legacy=0x7f289415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940cc4d0 con 0x7f289415e6c0 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f288c001a00 con 0x7f289415e6c0 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 >> v1:172.21.15.37:6789/0 conn(0x7f289415e6c0 legacy=0x7f28940caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f287c003a60 0x7f287c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28940b3cf0 0x7f287c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f287c004d50 0x7f287c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940cff10 con 0x7f287c003a60 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c00bb00 con 0x7f287c004d50 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c00bd00 con 0x7f28940b3cf0 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2893fff640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f287c003a60 0x7f287c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2898c50640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f287c004d50 0x7f287c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2893fff640 1 -- 172.21.15.37:0/4248063748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28940b3cf0 msgr2=0x7f287c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2893fff640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28940b3cf0 0x7f287c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2893fff640 1 -- 172.21.15.37:0/4248063748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f287c004d50 msgr2=0x7f287c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2893fff640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f287c004d50 0x7f287c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2893fff640 1 -- 172.21.15.37:0/4248063748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c003530 con 0x7f287c003a60 2024-04-25T08:53:20.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.468+0000 7f2898c50640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f287c004d50 0x7f287c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f2893fff640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f287c003a60 0x7f287c003ea0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2884009ae0 tx=0x7f288401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93355cc45be7c0d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28840096b0 con 0x7f287c003a60 2024-04-25T08:53:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28840a8430 con 0x7f287c003a60 2024-04-25T08:53:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f2892ffd640 1 -- 172.21.15.37:0/4248063748 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28840a8820 con 0x7f287c003a60 2024-04-25T08:53:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f289a6b3640 1 -- 172.21.15.37:0/4248063748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f287c003a60 msgr2=0x7f287c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f289a6b3640 1 --2- 172.21.15.37:0/4248063748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f287c003a60 0x7f287c003ea0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2884009ae0 tx=0x7f288401c9f0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f2898c50640 1 -- 172.21.15.37:0/4248063748 reap_dead start 2024-04-25T08:53:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f289a6b3640 1 -- 172.21.15.37:0/4248063748 shutdown_connections 2024-04-25T08:53:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f289a6b3640 1 -- 172.21.15.37:0/4248063748 >> 172.21.15.37:0/4248063748 conn(0x7f289406ae60 msgr2=0x7f289406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f289a6b3640 1 -- 172.21.15.37:0/4248063748 shutdown_connections 2024-04-25T08:53:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f289a6b3640 1 -- 172.21.15.37:0/4248063748 wait complete. 2024-04-25T08:53:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f289a6b3640 1 Processor -- start 2024-04-25T08:53:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.472+0000 7f289a6b3640 1 -- start start 2024-04-25T08:53:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f289a6b3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f287c003a60 0x7f289415c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f289a6b3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894157530 0x7f289415f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f289a6b3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2894098710 0x7f2894072120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f289a6b3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2894169710 con 0x7f2894157530 2024-04-25T08:53:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f289a6b3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289415ead0 con 0x7f287c003a60 2024-04-25T08:53:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f289a6b3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2894169890 con 0x7f2894098710 2024-04-25T08:53:20.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f28937fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894157530 0x7f289415f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f28937fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894157530 0x7f289415f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41328/0 (socket says 172.21.15.37:41328) 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f28937fe640 1 -- 172.21.15.37:0/1553367663 learned_addr learned my addr 172.21.15.37:0/1553367663 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f2893fff640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f287c003a60 0x7f289415c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f2898c50640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2894098710 0x7f2894072120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f28937fe640 1 -- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2894098710 msgr2=0x7f2894072120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f28937fe640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2894098710 0x7f2894072120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f28937fe640 1 -- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f287c003a60 msgr2=0x7f289415c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f28937fe640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f287c003a60 0x7f289415c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f28937fe640 1 -- 172.21.15.37:0/1553367663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2894157a10 con 0x7f2894157530 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f28937fe640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894157530 0x7f289415f640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f288800c500 tx=0x7f288800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f2890ff9640 1 -- 172.21.15.37:0/1553367663 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28880052d0 con 0x7f2894157530 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f2890ff9640 1 -- 172.21.15.37:0/1553367663 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2888005470 con 0x7f2894157530 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f289a6b3640 1 -- 172.21.15.37:0/1553367663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289415bbf0 con 0x7f2894157530 2024-04-25T08:53:20.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f2890ff9640 1 -- 172.21.15.37:0/1553367663 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888012870 con 0x7f2894157530 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f289a6b3640 1 -- 172.21.15.37:0/1553367663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289415cc10 con 0x7f2894157530 2024-04-25T08:53:20.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f289a6b3640 1 -- 172.21.15.37:0/1553367663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289404efa0 con 0x7f2894157530 2024-04-25T08:53:20.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f2890ff9640 1 -- 172.21.15.37:0/1553367663 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2888012a10 con 0x7f2894157530 2024-04-25T08:53:20.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f2890ff9640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2880066f90 0x7f2880069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f2890ff9640 1 -- 172.21.15.37:0/1553367663 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2888085060 con 0x7f2894157530 2024-04-25T08:53:20.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.476+0000 7f2893fff640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2880066f90 0x7f2880069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.480+0000 7f2893fff640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2880066f90 0x7f2880069450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28840af000 tx=0x7f28840b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.480+0000 7f2890ff9640 1 -- 172.21.15.37:0/1553367663 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28880182b0 con 0x7f2894157530 2024-04-25T08:53:20.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.640+0000 7f289a6b3640 1 -- 172.21.15.37:0/1553367663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f28940ceee0 con 0x7f2894157530 2024-04-25T08:53:20.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.640+0000 7f2890ff9640 1 -- 172.21.15.37:0/1553367663 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2888055d00 con 0x7f2894157530 2024-04-25T08:53:20.634 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.640+0000 7f289a6b3640 1 -- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2880066f90 msgr2=0x7f2880069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.640+0000 7f289a6b3640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2880066f90 0x7f2880069450 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f28840af000 tx=0x7f28840b2040 comp rx=0 tx=0).stop 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.640+0000 7f289a6b3640 1 -- 172.21.15.37:0/1553367663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894157530 msgr2=0x7f289415f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.640+0000 7f289a6b3640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894157530 0x7f289415f640 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f288800c500 tx=0x7f288800c9c0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.644+0000 7f289a6b3640 1 -- 172.21.15.37:0/1553367663 shutdown_connections 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.644+0000 7f289a6b3640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2880066f90 0x7f2880069450 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.644+0000 7f289a6b3640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2894098710 0x7f2894072120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.644+0000 7f289a6b3640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894157530 0x7f289415f640 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.644+0000 7f289a6b3640 1 --2- 172.21.15.37:0/1553367663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f287c003a60 0x7f289415c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.644+0000 7f289a6b3640 1 -- 172.21.15.37:0/1553367663 >> 172.21.15.37:0/1553367663 conn(0x7f289406ae60 msgr2=0x7f289415d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.644+0000 7f289a6b3640 1 -- 172.21.15.37:0/1553367663 shutdown_connections 2024-04-25T08:53:20.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.644+0000 7f289a6b3640 1 -- 172.21.15.37:0/1553367663 wait complete. 2024-04-25T08:53:20.652 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:20.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a206e0640 1 Processor -- start 2024-04-25T08:53:20.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a206e0640 1 -- start start 2024-04-25T08:53:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a206e0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1a180d07f0 0x7f1a180d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a206e0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1a180d1cd0 0x7f1a1806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a206e0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1806be60 con 0x7f1a1815da80 2024-04-25T08:53:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a206e0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1806c440 con 0x7f1a180d1cd0 2024-04-25T08:53:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a206e0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1815de60 con 0x7f1a180d07f0 2024-04-25T08:53:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a1e47c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1a1815da80 0x7f1a180d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41758/0 (socket says 172.21.15.37:41758) 2024-04-25T08:53:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a1e47c640 1 -- 172.21.15.37:0/4061471506 learned_addr learned my addr 172.21.15.37:0/4061471506 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:20.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a1ec7d640 1 --2- 172.21.15.37:0/4061471506 >> v2:172.21.15.37:3300/0 conn(0x7f1a180d1cd0 0x7f1a1806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a1dc7b640 1 --2- 172.21.15.37:0/4061471506 >> v2:172.21.15.203:3300/0 conn(0x7f1a180d07f0 0x7f1a180d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a1d47a640 1 -- 172.21.15.37:0/4061471506 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781560083 0 0) 0x7f1a1806be60 con 0x7f1a1815da80 2024-04-25T08:53:20.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a1d47a640 1 -- 172.21.15.37:0/4061471506 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a04003530 con 0x7f1a1815da80 2024-04-25T08:53:20.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a1ec7d640 1 -- 172.21.15.37:0/4061471506 >> v2:172.21.15.203:3300/0 conn(0x7f1a180d07f0 msgr2=0x7f1a180d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:20.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a1ec7d640 1 --2- 172.21.15.37:0/4061471506 >> v2:172.21.15.203:3300/0 conn(0x7f1a180d07f0 0x7f1a180d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a1ec7d640 1 -- 172.21.15.37:0/4061471506 >> v1:172.21.15.37:6789/0 conn(0x7f1a1815da80 legacy=0x7f1a180d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.772+0000 7f1a1ec7d640 1 -- 172.21.15.37:0/4061471506 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a180d2450 con 0x7f1a180d1cd0 2024-04-25T08:53:20.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1ec7d640 1 --2- 172.21.15.37:0/4061471506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a180d1cd0 0x7f1a1806c8d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1a1808d880 tx=0x7f1a1001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46916196cb33cb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1d47a640 1 -- 172.21.15.37:0/4061471506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a100b2070 con 0x7f1a180d1cd0 2024-04-25T08:53:20.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1d47a640 1 -- 172.21.15.37:0/4061471506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a100a59e0 con 0x7f1a180d1cd0 2024-04-25T08:53:20.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1d47a640 1 -- 172.21.15.37:0/4061471506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a100a5db0 con 0x7f1a180d1cd0 2024-04-25T08:53:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 -- 172.21.15.37:0/4061471506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a180d1cd0 msgr2=0x7f1a1806c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 --2- 172.21.15.37:0/4061471506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a180d1cd0 0x7f1a1806c8d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1a1808d880 tx=0x7f1a1001c900 comp rx=0 tx=0).stop 2024-04-25T08:53:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 -- 172.21.15.37:0/4061471506 shutdown_connections 2024-04-25T08:53:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 --2- 172.21.15.37:0/4061471506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a180d1cd0 0x7f1a1806c8d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 --2- 172.21.15.37:0/4061471506 >> v2:172.21.15.203:3300/0 conn(0x7f1a180d07f0 0x7f1a180d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 -- 172.21.15.37:0/4061471506 >> 172.21.15.37:0/4061471506 conn(0x7f1a1808b000 msgr2=0x7f1a180d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 -- 172.21.15.37:0/4061471506 shutdown_connections 2024-04-25T08:53:20.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 -- 172.21.15.37:0/4061471506 wait complete. 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 Processor -- start 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 -- start start 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a180d07f0 0x7f1a18154830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a180d1cd0 0x7f1a18154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a1815da80 0x7f1a181552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1806d7a0 con 0x7f1a180d1cd0 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1806d620 con 0x7f1a1815da80 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a206e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1806d920 con 0x7f1a180d07f0 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1dc7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a180d1cd0 0x7f1a18154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1dc7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a180d1cd0 0x7f1a18154d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41344/0 (socket says 172.21.15.37:41344) 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1dc7b640 1 -- 172.21.15.37:0/1034143851 learned_addr learned my addr 172.21.15.37:0/1034143851 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1dc7b640 1 -- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a180d07f0 msgr2=0x7f1a18154830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1e47c640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a180d07f0 0x7f1a18154830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1ec7d640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a1815da80 0x7f1a181552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1dc7b640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a180d07f0 0x7f1a18154830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1dc7b640 1 -- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a1815da80 msgr2=0x7f1a181552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1dc7b640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a1815da80 0x7f1a181552b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.776+0000 7f1a1dc7b640 1 -- 172.21.15.37:0/1034143851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a181605d0 con 0x7f1a180d1cd0 2024-04-25T08:53:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a1dc7b640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a180d1cd0 0x7f1a18154d70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800d800 tx=0x7f1a0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a0effd640 1 -- 172.21.15.37:0/1034143851 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a080059a0 con 0x7f1a180d1cd0 2024-04-25T08:53:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a0effd640 1 -- 172.21.15.37:0/1034143851 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a08005b40 con 0x7f1a180d1cd0 2024-04-25T08:53:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a206e0640 1 -- 172.21.15.37:0/1034143851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a18160890 con 0x7f1a180d1cd0 2024-04-25T08:53:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a0effd640 1 -- 172.21.15.37:0/1034143851 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a08005d30 con 0x7f1a180d1cd0 2024-04-25T08:53:20.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a206e0640 1 -- 172.21.15.37:0/1034143851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a18160e50 con 0x7f1a180d1cd0 2024-04-25T08:53:20.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a206e0640 1 -- 172.21.15.37:0/1034143851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1804efa0 con 0x7f1a180d1cd0 2024-04-25T08:53:20.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a0effd640 1 -- 172.21.15.37:0/1034143851 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a0801b020 con 0x7f1a180d1cd0 2024-04-25T08:53:20.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a0effd640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19fc066f90 0x7f19fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:20.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a0effd640 1 -- 172.21.15.37:0/1034143851 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1a08085280 con 0x7f1a180d1cd0 2024-04-25T08:53:20.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a1e47c640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19fc066f90 0x7f19fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:20.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.780+0000 7f1a1e47c640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19fc066f90 0x7f19fc069450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1a14007fd0 tx=0x7f1a14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:20.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.784+0000 7f1a0effd640 1 -- 172.21.15.37:0/1034143851 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a08055e80 con 0x7f1a180d1cd0 2024-04-25T08:53:20.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.944+0000 7f1a206e0640 1 -- 172.21.15.37:0/1034143851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a180d1200 con 0x7f1a180d1cd0 2024-04-25T08:53:20.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a0effd640 1 -- 172.21.15.37:0/1034143851 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f1a08055780 con 0x7f1a180d1cd0 2024-04-25T08:53:20.941 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:20.941 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 -- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19fc066f90 msgr2=0x7f19fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19fc066f90 0x7f19fc069450 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1a14007fd0 tx=0x7f1a14009040 comp rx=0 tx=0).stop 2024-04-25T08:53:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 -- 172.21.15.37:0/1034143851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a180d1cd0 msgr2=0x7f1a18154d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a180d1cd0 0x7f1a18154d70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1a0800d800 tx=0x7f1a0800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 -- 172.21.15.37:0/1034143851 shutdown_connections 2024-04-25T08:53:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19fc066f90 0x7f19fc069450 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a1815da80 0x7f1a181552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a180d1cd0 0x7f1a18154d70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 --2- 172.21.15.37:0/1034143851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a180d07f0 0x7f1a18154830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:20.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 -- 172.21.15.37:0/1034143851 >> 172.21.15.37:0/1034143851 conn(0x7f1a1808b000 msgr2=0x7f1a180ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:20.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 -- 172.21.15.37:0/1034143851 shutdown_connections 2024-04-25T08:53:20.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:20.948+0000 7f1a206e0640 1 -- 172.21.15.37:0/1034143851 wait complete. 2024-04-25T08:53:20.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:20.960 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefb0ef4640 1 Processor -- start 2024-04-25T08:53:21.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefb0ef4640 1 -- start start 2024-04-25T08:53:21.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefb0ef4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac0b3cf0 0x7fefac0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefb0ef4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefac0cbf10 con 0x7fefac15e6c0 2024-04-25T08:53:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefb0ef4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefac0cc1f0 con 0x7fefac0cfad0 2024-04-25T08:53:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefb0ef4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0cff10 con 0x7fefac0b3cf0 2024-04-25T08:53:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefaad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fefac15e6c0 0x7fefac0caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41770/0 (socket says 172.21.15.37:41770) 2024-04-25T08:53:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefaad9d640 1 -- 172.21.15.37:0/1727015912 learned_addr learned my addr 172.21.15.37:0/1727015912 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefaa59c640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac0b3cf0 0x7fefac0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616383548 0 0) 0x7fefac0cbf10 con 0x7fefac15e6c0 2024-04-25T08:53:21.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef94003530 con 0x7fefac15e6c0 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4065083757 0 0) 0x7fef94003530 con 0x7fefac15e6c0 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac0b3cf0 msgr2=0x7fefac0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac0b3cf0 0x7fefac0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 >> v1:172.21.15.203:6789/0 conn(0x7fefac0cfad0 legacy=0x7fefac15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0cc4d0 con 0x7fefac15e6c0 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fefa4001a00 con 0x7fefac15e6c0 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 >> v1:172.21.15.37:6789/0 conn(0x7fefac15e6c0 legacy=0x7fefac0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac0b3cf0 0x7fef94003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef94003fc0 0x7fef94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94004d50 0x7fef940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0cff10 con 0x7fef94004d50 2024-04-25T08:53:21.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9400bb10 con 0x7fef94003fc0 2024-04-25T08:53:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9400bd10 con 0x7fefac0b3cf0 2024-04-25T08:53:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefaad9d640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94004d50 0x7fef940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefaa59c640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac0b3cf0 0x7fef94003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa9d9b640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef94003fc0 0x7fef94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefaad9d640 1 -- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac0b3cf0 msgr2=0x7fef94003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefaad9d640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac0b3cf0 0x7fef94003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefaad9d640 1 -- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef94003fc0 msgr2=0x7fef94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefaad9d640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef94003fc0 0x7fef94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefaad9d640 1 -- 172.21.15.37:0/1727015912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef94003530 con 0x7fef94004d50 2024-04-25T08:53:21.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.100+0000 7fefa9d9b640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef94003fc0 0x7fef94004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefaad9d640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94004d50 0x7fef940051d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fefa400a790 tx=0x7fefa401fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e232c9705fc64488 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa400a3b0 con 0x7fef94004d50 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa40aa630 con 0x7fef94004d50 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefa959a640 1 -- 172.21.15.37:0/1727015912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa40aaa20 con 0x7fef94004d50 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefb0ef4640 1 -- 172.21.15.37:0/1727015912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94004d50 msgr2=0x7fef940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefb0ef4640 1 --2- 172.21.15.37:0/1727015912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef94004d50 0x7fef940051d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fefa400a790 tx=0x7fefa401fa00 comp rx=0 tx=0).stop 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefaad9d640 1 -- 172.21.15.37:0/1727015912 reap_dead start 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefb0ef4640 1 -- 172.21.15.37:0/1727015912 shutdown_connections 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefb0ef4640 1 -- 172.21.15.37:0/1727015912 >> 172.21.15.37:0/1727015912 conn(0x7fefac06ae60 msgr2=0x7fefac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefb0ef4640 1 -- 172.21.15.37:0/1727015912 shutdown_connections 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefb0ef4640 1 -- 172.21.15.37:0/1727015912 wait complete. 2024-04-25T08:53:21.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefb0ef4640 1 Processor -- start 2024-04-25T08:53:21.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.104+0000 7fefb0ef4640 1 -- start start 2024-04-25T08:53:21.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefb0ef4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef94004d50 0x7fefac15c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefb0ef4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac15aa40 0x7fefac0ce760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefb0ef4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac0c7f80 0x7fefac0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefb0ef4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0ac170 con 0x7fefac0c7f80 2024-04-25T08:53:21.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefb0ef4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefac15ead0 con 0x7fef94004d50 2024-04-25T08:53:21.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefb0ef4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefac0b40d0 con 0x7fefac15aa40 2024-04-25T08:53:21.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac0c7f80 0x7fefac0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac0c7f80 0x7fefac0c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41368/0 (socket says 172.21.15.37:41368) 2024-04-25T08:53:21.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaad9d640 1 -- 172.21.15.37:0/143142527 learned_addr learned my addr 172.21.15.37:0/143142527 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:21.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaa59c640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef94004d50 0x7fefac15c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaad9d640 1 -- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac15aa40 msgr2=0x7fefac0ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaad9d640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac15aa40 0x7fefac0ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaad9d640 1 -- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef94004d50 msgr2=0x7fefac15c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaad9d640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef94004d50 0x7fefac15c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaad9d640 1 -- 172.21.15.37:0/143142527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac15af00 con 0x7fefac0c7f80 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaa59c640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef94004d50 0x7fefac15c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaad9d640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac0c7f80 0x7fefac0c8430 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fefa400a790 tx=0x7fef94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fef9affd640 1 -- 172.21.15.37:0/143142527 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa40011d0 con 0x7fefac0c7f80 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fef9affd640 1 -- 172.21.15.37:0/143142527 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa40aa430 con 0x7fefac0c7f80 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefb0ef4640 1 -- 172.21.15.37:0/143142527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefac15b1e0 con 0x7fefac0c7f80 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fef9affd640 1 -- 172.21.15.37:0/143142527 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4001f10 con 0x7fefac0c7f80 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefb0ef4640 1 -- 172.21.15.37:0/143142527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefac15b7a0 con 0x7fefac0c7f80 2024-04-25T08:53:21.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefb0ef4640 1 -- 172.21.15.37:0/143142527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefac04efa0 con 0x7fefac0c7f80 2024-04-25T08:53:21.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fef9affd640 1 -- 172.21.15.37:0/143142527 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fefa40bb070 con 0x7fefac0c7f80 2024-04-25T08:53:21.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fef9affd640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fef8c066f90 0x7fef8c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fef9affd640 1 -- 172.21.15.37:0/143142527 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fefa4122250 con 0x7fefac0c7f80 2024-04-25T08:53:21.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.108+0000 7fefaa59c640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fef8c066f90 0x7fef8c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.112+0000 7fefaa59c640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fef8c066f90 0x7fef8c069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009af0 tx=0x7fef9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.112+0000 7fef9affd640 1 -- 172.21.15.37:0/143142527 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fefa40af020 con 0x7fefac0c7f80 2024-04-25T08:53:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.276+0000 7fefb0ef4640 1 -- 172.21.15.37:0/143142527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fefac0b3060 con 0x7fefac0c7f80 2024-04-25T08:53:21.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.276+0000 7fef9affd640 1 -- 172.21.15.37:0/143142527 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fefa4019710 con 0x7fefac0c7f80 2024-04-25T08:53:21.271 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 -- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fef8c066f90 msgr2=0x7fef8c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fef8c066f90 0x7fef8c069450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fef9c009af0 tx=0x7fef9c008040 comp rx=0 tx=0).stop 2024-04-25T08:53:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 -- 172.21.15.37:0/143142527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac0c7f80 msgr2=0x7fefac0c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac0c7f80 0x7fefac0c8430 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fefa400a790 tx=0x7fef94005840 comp rx=0 tx=0).stop 2024-04-25T08:53:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 -- 172.21.15.37:0/143142527 shutdown_connections 2024-04-25T08:53:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fef8c066f90 0x7fef8c069450 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fefac0c7f80 0x7fefac0c8430 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fefac15aa40 0x7fefac0ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 --2- 172.21.15.37:0/143142527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef94004d50 0x7fefac15c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 -- 172.21.15.37:0/143142527 >> 172.21.15.37:0/143142527 conn(0x7fefac06ae60 msgr2=0x7fefac06e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 -- 172.21.15.37:0/143142527 shutdown_connections 2024-04-25T08:53:21.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.280+0000 7fefb0ef4640 1 -- 172.21.15.37:0/143142527 wait complete. 2024-04-25T08:53:21.290 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:21.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efea05640 1 Processor -- start 2024-04-25T08:53:21.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efea05640 1 -- start start 2024-04-25T08:53:21.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efea05640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80cf590 0x7f3ef815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efea05640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef80ca4c0 con 0x7f3ef80ce170 2024-04-25T08:53:21.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efea05640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef80cd690 con 0x7f3ef80cf970 2024-04-25T08:53:21.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efea05640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef80ca780 con 0x7f3ef80cf590 2024-04-25T08:53:21.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efcfa2640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3ef80ce170 0x7f3ef80ce600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41786/0 (socket says 172.21.15.37:41786) 2024-04-25T08:53:21.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efcfa2640 1 -- 172.21.15.37:0/1426518343 learned_addr learned my addr 172.21.15.37:0/1426518343 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:21.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488447194 0 0) 0x7f3ef80ca4c0 con 0x7f3ef80ce170 2024-04-25T08:53:21.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef7fff640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80cf590 0x7f3ef815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ee0005740 con 0x7f3ef80ce170 2024-04-25T08:53:21.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 257542030 0 0) 0x7f3ee0005740 con 0x7f3ef80ce170 2024-04-25T08:53:21.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80cf590 msgr2=0x7f3ef815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:21.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80cf590 0x7f3ef815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 >> v1:172.21.15.203:6789/0 conn(0x7f3ef80cf970 legacy=0x7f3ef80c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef80cd970 con 0x7f3ef80ce170 2024-04-25T08:53:21.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3ef0001a00 con 0x7f3ef80ce170 2024-04-25T08:53:21.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 >> v1:172.21.15.37:6789/0 conn(0x7f3ef80ce170 legacy=0x7f3ef80ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80cf590 0x7f3ee0005c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ee00062a0 0x7f3ee0006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ee0006fb0 0x7f3ee0007430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee000ddc0 con 0x7f3ee0006fb0 2024-04-25T08:53:21.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee000df70 con 0x7f3ee00062a0 2024-04-25T08:53:21.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee000e170 con 0x7f3ef80cf590 2024-04-25T08:53:21.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efcfa2640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ee0006fb0 0x7f3ee0007430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef7fff640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80cf590 0x7f3ee0005c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3ef77fe640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ee00062a0 0x7f3ee0006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efcfa2640 1 -- 172.21.15.37:0/1426518343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80cf590 msgr2=0x7f3ee0005c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:21.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efcfa2640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef80cf590 0x7f3ee0005c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efcfa2640 1 -- 172.21.15.37:0/1426518343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ee00062a0 msgr2=0x7f3ee0006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:21.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efcfa2640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ee00062a0 0x7f3ee0006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.420+0000 7f3efcfa2640 1 -- 172.21.15.37:0/1426518343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0005740 con 0x7f3ee0006fb0 2024-04-25T08:53:21.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3efcfa2640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ee0006fb0 0x7f3ee0007430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000a860 tx=0x7f3ef001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a611d63121f35b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:21.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef000a480 con 0x7f3ee0006fb0 2024-04-25T08:53:21.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef00ad630 con 0x7f3ee0006fb0 2024-04-25T08:53:21.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3ef6ffd640 1 -- 172.21.15.37:0/1426518343 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00ada80 con 0x7f3ee0006fb0 2024-04-25T08:53:21.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3efea05640 1 -- 172.21.15.37:0/1426518343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ee0006fb0 msgr2=0x7f3ee0007430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3efea05640 1 --2- 172.21.15.37:0/1426518343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ee0006fb0 0x7f3ee0007430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000a860 tx=0x7f3ef001d380 comp rx=0 tx=0).stop 2024-04-25T08:53:21.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3efcfa2640 1 -- 172.21.15.37:0/1426518343 reap_dead start 2024-04-25T08:53:21.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3efea05640 1 -- 172.21.15.37:0/1426518343 shutdown_connections 2024-04-25T08:53:21.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3efea05640 1 -- 172.21.15.37:0/1426518343 >> 172.21.15.37:0/1426518343 conn(0x7f3ef806ae60 msgr2=0x7f3ef806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:21.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3efea05640 1 -- 172.21.15.37:0/1426518343 shutdown_connections 2024-04-25T08:53:21.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3efea05640 1 -- 172.21.15.37:0/1426518343 wait complete. 2024-04-25T08:53:21.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3efea05640 1 Processor -- start 2024-04-25T08:53:21.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.424+0000 7f3efea05640 1 -- start start 2024-04-25T08:53:21.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efea05640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ee0006fb0 0x7f3ef8070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efea05640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef8072540 0x7f3ef8153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efea05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef8157660 0x7f3ef81e4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efea05640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef80cdce0 con 0x7f3ef8157660 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efea05640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef80cd440 con 0x7f3ef8072540 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efea05640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef80ca0c0 con 0x7f3ee0006fb0 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efcfa2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef8157660 0x7f3ef81e4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efcfa2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef8157660 0x7f3ef81e4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41388/0 (socket says 172.21.15.37:41388) 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efcfa2640 1 -- 172.21.15.37:0/1339142255 learned_addr learned my addr 172.21.15.37:0/1339142255 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3ef77fe640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef8072540 0x7f3ef8153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efcfa2640 1 -- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ee0006fb0 msgr2=0x7f3ef8070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3ef7fff640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ee0006fb0 0x7f3ef8070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efcfa2640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ee0006fb0 0x7f3ef8070ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efcfa2640 1 -- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef8072540 msgr2=0x7f3ef8153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efcfa2640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef8072540 0x7f3ef8153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efcfa2640 1 -- 172.21.15.37:0/1339142255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8154310 con 0x7f3ef8157660 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3ef77fe640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef8072540 0x7f3ef8153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efcfa2640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef8157660 0x7f3ef81e4390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3ef001d1b0 tx=0x7f3ee0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:21.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3ef4ff9640 1 -- 172.21.15.37:0/1339142255 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00b2040 con 0x7f3ef8157660 2024-04-25T08:53:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3ef4ff9640 1 -- 172.21.15.37:0/1339142255 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef00011d0 con 0x7f3ef8157660 2024-04-25T08:53:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efea05640 1 -- 172.21.15.37:0/1339142255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef80714a0 con 0x7f3ef8157660 2024-04-25T08:53:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3ef4ff9640 1 -- 172.21.15.37:0/1339142255 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef000a480 con 0x7f3ef8157660 2024-04-25T08:53:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efea05640 1 -- 172.21.15.37:0/1339142255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef8071a60 con 0x7f3ef8157660 2024-04-25T08:53:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3efea05640 1 -- 172.21.15.37:0/1339142255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef804efa0 con 0x7f3ef8157660 2024-04-25T08:53:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3ef4ff9640 1 -- 172.21.15.37:0/1339142255 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ef00bf060 con 0x7f3ef8157660 2024-04-25T08:53:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3ef4ff9640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ee4066f90 0x7f3ee4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3ef4ff9640 1 -- 172.21.15.37:0/1339142255 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3ef011f560 con 0x7f3ef8157660 2024-04-25T08:53:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.428+0000 7f3ef7fff640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ee4066f90 0x7f3ee4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.432+0000 7f3ef7fff640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ee4066f90 0x7f3ee4069450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ef806f270 tx=0x7f3ee8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:21.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.432+0000 7f3ef4ff9640 1 -- 172.21.15.37:0/1339142255 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ef8154310 con 0x7f3ef8157660 2024-04-25T08:53:21.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.584+0000 7f3efea05640 1 -- 172.21.15.37:0/1339142255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ef8158a80 con 0x7f3ef8157660 2024-04-25T08:53:21.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.584+0000 7f3ef4ff9640 1 -- 172.21.15.37:0/1339142255 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ef0017090 con 0x7f3ef8157660 2024-04-25T08:53:21.579 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 -- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ee4066f90 msgr2=0x7f3ee4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ee4066f90 0x7f3ee4069450 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ef806f270 tx=0x7f3ee8008040 comp rx=0 tx=0).stop 2024-04-25T08:53:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 -- 172.21.15.37:0/1339142255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef8157660 msgr2=0x7f3ef81e4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef8157660 0x7f3ef81e4390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3ef001d1b0 tx=0x7f3ee0007a50 comp rx=0 tx=0).stop 2024-04-25T08:53:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 -- 172.21.15.37:0/1339142255 shutdown_connections 2024-04-25T08:53:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ee4066f90 0x7f3ee4069450 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef8157660 0x7f3ef81e4390 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef8072540 0x7f3ef8153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 --2- 172.21.15.37:0/1339142255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ee0006fb0 0x7f3ef8070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 -- 172.21.15.37:0/1339142255 >> 172.21.15.37:0/1339142255 conn(0x7f3ef806ae60 msgr2=0x7f3ef8157ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 -- 172.21.15.37:0/1339142255 shutdown_connections 2024-04-25T08:53:21.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.588+0000 7f3efea05640 1 -- 172.21.15.37:0/1339142255 wait complete. 2024-04-25T08:53:21.596 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:21.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.716+0000 7fc83fa96640 1 Processor -- start 2024-04-25T08:53:21.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.716+0000 7fc83fa96640 1 -- start start 2024-04-25T08:53:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83fa96640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc8380cf590 0x7fc83815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83fa96640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8380cf970 0x7fc8380c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83fa96640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8380ca4c0 con 0x7fc8380c9ef0 2024-04-25T08:53:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83fa96640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8380ca780 con 0x7fc8380cf590 2024-04-25T08:53:21.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83fa96640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380cd440 con 0x7fc8380cf970 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83e033640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc8380c9ef0 0x7fc8380b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41802/0 (socket says 172.21.15.37:41802) 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83e033640 1 -- 172.21.15.37:0/185678671 learned_addr learned my addr 172.21.15.37:0/185678671 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83d832640 1 --2- 172.21.15.37:0/185678671 >> v2:172.21.15.203:3300/0 conn(0x7fc8380cf590 0x7fc83815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83d031640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8380cf970 0x7fc8380c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647416799 0 0) 0x7fc8380ca4c0 con 0x7fc8380c9ef0 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc820003530 con 0x7fc8380c9ef0 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3445923061 0 0) 0x7fc820003530 con 0x7fc8380c9ef0 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8380cf970 msgr2=0x7fc8380c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8380cf970 0x7fc8380c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 >> v2:172.21.15.203:3300/0 conn(0x7fc8380cf590 msgr2=0x7fc83815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 --2- 172.21.15.37:0/185678671 >> v2:172.21.15.203:3300/0 conn(0x7fc8380cf590 0x7fc83815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8380cd690 con 0x7fc8380c9ef0 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fc828001680 con 0x7fc8380c9ef0 2024-04-25T08:53:21.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 >> v1:172.21.15.37:6789/0 conn(0x7fc8380c9ef0 legacy=0x7fc8380b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc820003a30 0x7fc820003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc820004810 0x7fc820004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8380cf970 0x7fc8200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380cd440 con 0x7fc820004810 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82000b9d0 con 0x7fc820003a30 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82000bbd0 con 0x7fc8380cf970 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83d031640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc820004810 0x7fc820004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83d832640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc820003a30 0x7fc820003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83d031640 1 -- 172.21.15.37:0/185678671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8380cf970 msgr2=0x7fc8200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83d031640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8380cf970 0x7fc8200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83d031640 1 -- 172.21.15.37:0/185678671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc820003a30 msgr2=0x7fc820003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83d031640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc820003a30 0x7fc820003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83d031640 1 -- 172.21.15.37:0/185678671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc820003530 con 0x7fc820004810 2024-04-25T08:53:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.720+0000 7fc83d832640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc820003a30 0x7fc820003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83d031640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc820004810 0x7fc820004c90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00ba30 tx=0x7fc82c01c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58bb6ce18f19cab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c00b600 con 0x7fc820004810 2024-04-25T08:53:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82c0b0070 con 0x7fc820004810 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83c830640 1 -- 172.21.15.37:0/185678671 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0ab650 con 0x7fc820004810 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 -- 172.21.15.37:0/185678671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc820004810 msgr2=0x7fc820004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 --2- 172.21.15.37:0/185678671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc820004810 0x7fc820004c90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc82c00ba30 tx=0x7fc82c01c6e0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83e033640 1 -- 172.21.15.37:0/185678671 reap_dead start 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 -- 172.21.15.37:0/185678671 shutdown_connections 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 -- 172.21.15.37:0/185678671 >> 172.21.15.37:0/185678671 conn(0x7fc83806ae60 msgr2=0x7fc83806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 -- 172.21.15.37:0/185678671 shutdown_connections 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 -- 172.21.15.37:0/185678671 wait complete. 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 Processor -- start 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 -- start start 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc820004810 0x7fc8381f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8380c5820 0x7fc83815d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc838071f10 0x7fc838072390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380cfdb0 con 0x7fc838071f10 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380cdce0 con 0x7fc8380c5820 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc83815c2a0 con 0x7fc820004810 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83e033640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc838071f10 0x7fc838072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83e033640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc838071f10 0x7fc838072390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41408/0 (socket says 172.21.15.37:41408) 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83e033640 1 -- 172.21.15.37:0/1137089559 learned_addr learned my addr 172.21.15.37:0/1137089559 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83d031640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8380c5820 0x7fc83815d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83d832640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc820004810 0x7fc8381f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83e033640 1 -- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc820004810 msgr2=0x7fc8381f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83e033640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc820004810 0x7fc8381f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83e033640 1 -- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8380c5820 msgr2=0x7fc83815d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83e033640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8380c5820 0x7fc83815d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83e033640 1 -- 172.21.15.37:0/1137089559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8380b3b90 con 0x7fc838071f10 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83d031640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8380c5820 0x7fc83815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83e033640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc838071f10 0x7fc838072390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc8380c6390 tx=0x7fc82800f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc8367fc640 1 -- 172.21.15.37:0/1137089559 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828007af0 con 0x7fc838071f10 2024-04-25T08:53:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc8367fc640 1 -- 172.21.15.37:0/1137089559 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc828007c90 con 0x7fc838071f10 2024-04-25T08:53:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 -- 172.21.15.37:0/1137089559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8380cae90 con 0x7fc838071f10 2024-04-25T08:53:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc8367fc640 1 -- 172.21.15.37:0/1137089559 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800a2b0 con 0x7fc838071f10 2024-04-25T08:53:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.724+0000 7fc83fa96640 1 -- 172.21.15.37:0/1137089559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc83815c860 con 0x7fc838071f10 2024-04-25T08:53:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.728+0000 7fc83fa96640 1 -- 172.21.15.37:0/1137089559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83804efa0 con 0x7fc838071f10 2024-04-25T08:53:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.728+0000 7fc8367fc640 1 -- 172.21.15.37:0/1137089559 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc82800a450 con 0x7fc838071f10 2024-04-25T08:53:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.728+0000 7fc8367fc640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc824066ed0 0x7fc824069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.728+0000 7fc8367fc640 1 -- 172.21.15.37:0/1137089559 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc828087980 con 0x7fc838071f10 2024-04-25T08:53:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.728+0000 7fc83d832640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc824066ed0 0x7fc824069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.728+0000 7fc83d832640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc824066ed0 0x7fc824069390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc830003bf0 tx=0x7fc83000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.732+0000 7fc8367fc640 1 -- 172.21.15.37:0/1137089559 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc82800a740 con 0x7fc838071f10 2024-04-25T08:53:21.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.892+0000 7fc83fa96640 1 -- 172.21.15.37:0/1137089559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8380b4170 con 0x7fc838071f10 2024-04-25T08:53:21.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.892+0000 7fc8367fc640 1 -- 172.21.15.37:0/1137089559 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc828058590 con 0x7fc838071f10 2024-04-25T08:53:21.886 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 -- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc824066ed0 msgr2=0x7fc824069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc824066ed0 0x7fc824069390 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc830003bf0 tx=0x7fc83000b040 comp rx=0 tx=0).stop 2024-04-25T08:53:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 -- 172.21.15.37:0/1137089559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc838071f10 msgr2=0x7fc838072390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc838071f10 0x7fc838072390 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc8380c6390 tx=0x7fc82800f4b0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 -- 172.21.15.37:0/1137089559 shutdown_connections 2024-04-25T08:53:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc824066ed0 0x7fc824069390 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc838071f10 0x7fc838072390 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8380c5820 0x7fc83815d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 --2- 172.21.15.37:0/1137089559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc820004810 0x7fc8381f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:21.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 -- 172.21.15.37:0/1137089559 >> 172.21.15.37:0/1137089559 conn(0x7fc83806ae60 msgr2=0x7fc83815e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 -- 172.21.15.37:0/1137089559 shutdown_connections 2024-04-25T08:53:21.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:21.896+0000 7fc83fa96640 1 -- 172.21.15.37:0/1137089559 wait complete. 2024-04-25T08:53:21.905 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:22.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26aa077640 1 Processor -- start 2024-04-25T08:53:22.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26aa077640 1 -- start start 2024-04-25T08:53:22.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26aa077640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f26a415d9c0 0x7f26a40d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26aa077640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f26a40cf270 0x7f26a40d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26aa077640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a40d1950 con 0x7f26a40d3c20 2024-04-25T08:53:22.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26aa077640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a40d2520 con 0x7f26a415d9c0 2024-04-25T08:53:22.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26aa077640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a40d1bf0 con 0x7f26a40cf270 2024-04-25T08:53:22.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26a37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f26a415d9c0 0x7f26a40d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26a37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f26a415d9c0 0x7f26a40d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41412/0 (socket says 172.21.15.37:41412) 2024-04-25T08:53:22.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26a37fe640 1 -- 172.21.15.37:0/3318629530 learned_addr learned my addr 172.21.15.37:0/3318629530 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:22.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26a2ffd640 1 --2- 172.21.15.37:0/3318629530 >> v2:172.21.15.203:3300/0 conn(0x7f26a40cf270 0x7f26a40d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26a37fe640 1 -- 172.21.15.37:0/3318629530 >> v2:172.21.15.203:3300/0 conn(0x7f26a40cf270 msgr2=0x7f26a40d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:22.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26a37fe640 1 --2- 172.21.15.37:0/3318629530 >> v2:172.21.15.203:3300/0 conn(0x7f26a40cf270 0x7f26a40d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26a37fe640 1 -- 172.21.15.37:0/3318629530 >> v1:172.21.15.203:6789/0 conn(0x7f26a40d3c20 legacy=0x7f26a40d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.028+0000 7f26a37fe640 1 -- 172.21.15.37:0/3318629530 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a40d1ed0 con 0x7f26a415d9c0 2024-04-25T08:53:22.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26a37fe640 1 --2- 172.21.15.37:0/3318629530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26a415d9c0 0x7f26a40d1320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f26a40a1610 tx=0x7f269401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94476d4569c8647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26a27fc640 1 -- 172.21.15.37:0/3318629530 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940b1070 con 0x7f26a415d9c0 2024-04-25T08:53:22.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26a27fc640 1 -- 172.21.15.37:0/3318629530 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26940ac410 con 0x7f26a415d9c0 2024-04-25T08:53:22.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26a27fc640 1 -- 172.21.15.37:0/3318629530 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940ac7e0 con 0x7f26a415d9c0 2024-04-25T08:53:22.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 -- 172.21.15.37:0/3318629530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26a415d9c0 msgr2=0x7f26a40d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 --2- 172.21.15.37:0/3318629530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26a415d9c0 0x7f26a40d1320 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f26a40a1610 tx=0x7f269401c520 comp rx=0 tx=0).stop 2024-04-25T08:53:22.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 -- 172.21.15.37:0/3318629530 shutdown_connections 2024-04-25T08:53:22.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 --2- 172.21.15.37:0/3318629530 >> v2:172.21.15.203:3300/0 conn(0x7f26a40cf270 0x7f26a40d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 --2- 172.21.15.37:0/3318629530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26a415d9c0 0x7f26a40d1320 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 -- 172.21.15.37:0/3318629530 >> 172.21.15.37:0/3318629530 conn(0x7f26a409cad0 msgr2=0x7f26a408b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:22.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 -- 172.21.15.37:0/3318629530 shutdown_connections 2024-04-25T08:53:22.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 -- 172.21.15.37:0/3318629530 wait complete. 2024-04-25T08:53:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 Processor -- start 2024-04-25T08:53:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 -- start start 2024-04-25T08:53:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a40cf270 0x7f26a409d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26a40d3c20 0x7f26a4158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26a415d9c0 0x7f26a40a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a416a720 con 0x7f26a415d9c0 2024-04-25T08:53:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a416a5a0 con 0x7f26a40cf270 2024-04-25T08:53:22.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26aa077640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a416a8a0 con 0x7f26a40d3c20 2024-04-25T08:53:22.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26a3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26a415d9c0 0x7f26a40a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26a3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26a415d9c0 0x7f26a40a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41420/0 (socket says 172.21.15.37:41420) 2024-04-25T08:53:22.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26a3fff640 1 -- 172.21.15.37:0/4139321664 learned_addr learned my addr 172.21.15.37:0/4139321664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:22.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26a37fe640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a40cf270 0x7f26a409d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26a3fff640 1 -- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26a40d3c20 msgr2=0x7f26a4158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.032+0000 7f26a2ffd640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26a40d3c20 0x7f26a4158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26a3fff640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26a40d3c20 0x7f26a4158970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26a3fff640 1 -- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a40cf270 msgr2=0x7f26a409d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26a3fff640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a40cf270 0x7f26a409d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26a3fff640 1 -- 172.21.15.37:0/4139321664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a40b4110 con 0x7f26a415d9c0 2024-04-25T08:53:22.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26a37fe640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a40cf270 0x7f26a409d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:22.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26a2ffd640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26a40d3c20 0x7f26a4158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:22.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26a3fff640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26a415d9c0 0x7f26a40a05c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f269c00c7a0 tx=0x7f269c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f268bfff640 1 -- 172.21.15.37:0/4139321664 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c003ba0 con 0x7f26a415d9c0 2024-04-25T08:53:22.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f268bfff640 1 -- 172.21.15.37:0/4139321664 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f269c003d40 con 0x7f26a415d9c0 2024-04-25T08:53:22.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26aa077640 1 -- 172.21.15.37:0/4139321664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a40b3060 con 0x7f26a415d9c0 2024-04-25T08:53:22.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f268bfff640 1 -- 172.21.15.37:0/4139321664 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269c00aa30 con 0x7f26a415d9c0 2024-04-25T08:53:22.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26aa077640 1 -- 172.21.15.37:0/4139321664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a40b3e30 con 0x7f26a415d9c0 2024-04-25T08:53:22.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26aa077640 1 -- 172.21.15.37:0/4139321664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a404efa0 con 0x7f26a415d9c0 2024-04-25T08:53:22.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f268bfff640 1 -- 172.21.15.37:0/4139321664 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f269c00abd0 con 0x7f26a415d9c0 2024-04-25T08:53:22.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f268bfff640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2690066f90 0x7f2690069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f268bfff640 1 -- 172.21.15.37:0/4139321664 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f269c085330 con 0x7f26a415d9c0 2024-04-25T08:53:22.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26a37fe640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2690066f90 0x7f2690069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.036+0000 7f26a37fe640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2690066f90 0x7f2690069450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f26940ae000 tx=0x7f26940a6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.040+0000 7f268bfff640 1 -- 172.21.15.37:0/4139321664 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f269c055f40 con 0x7f26a415d9c0 2024-04-25T08:53:22.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 -- 172.21.15.37:0/4139321664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f26a40b0cd0 con 0x7f26a415d9c0 2024-04-25T08:53:22.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f268bfff640 1 -- 172.21.15.37:0/4139321664 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f269c055840 con 0x7f26a415d9c0 2024-04-25T08:53:22.192 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 -- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2690066f90 msgr2=0x7f2690069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2690066f90 0x7f2690069450 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f26940ae000 tx=0x7f26940a6040 comp rx=0 tx=0).stop 2024-04-25T08:53:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 -- 172.21.15.37:0/4139321664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26a415d9c0 msgr2=0x7f26a40a05c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26a415d9c0 0x7f26a40a05c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f269c00c7a0 tx=0x7f269c00cc60 comp rx=0 tx=0).stop 2024-04-25T08:53:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 -- 172.21.15.37:0/4139321664 shutdown_connections 2024-04-25T08:53:22.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2690066f90 0x7f2690069450 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26a415d9c0 0x7f26a40a05c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f26a40d3c20 0x7f26a4158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 --2- 172.21.15.37:0/4139321664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26a40cf270 0x7f26a409d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 -- 172.21.15.37:0/4139321664 >> 172.21.15.37:0/4139321664 conn(0x7f26a409cad0 msgr2=0x7f26a40cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:22.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 -- 172.21.15.37:0/4139321664 shutdown_connections 2024-04-25T08:53:22.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.200+0000 7f26aa077640 1 -- 172.21.15.37:0/4139321664 wait complete. 2024-04-25T08:53:22.218 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:53:22.219 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:22.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.336+0000 7f7de2097640 1 Processor -- start 2024-04-25T08:53:22.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.336+0000 7f7de2097640 1 -- start start 2024-04-25T08:53:22.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7de2097640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ddc0ce510 0x7f7ddc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7de2097640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7ddc144ca0 0x7f7ddc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7de2097640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ddc06ec60 0x7f7ddc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7de2097640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc0cc2b0 con 0x7f7ddc144ca0 2024-04-25T08:53:22.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7de2097640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc06e290 con 0x7f7ddc06ec60 2024-04-25T08:53:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7de2097640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0cee30 con 0x7f7ddc0ce510 2024-04-25T08:53:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7ddaffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7ddc144ca0 0x7f7ddc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7ddaffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7ddc144ca0 0x7f7ddc06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41422/0 (socket says 172.21.15.37:41422) 2024-04-25T08:53:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7ddaffd640 1 -- 172.21.15.37:0/2718368610 learned_addr learned my addr 172.21.15.37:0/2718368610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7ddbfff640 1 --2- 172.21.15.37:0/2718368610 >> v2:172.21.15.203:3300/0 conn(0x7f7ddc06ec60 0x7f7ddc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7ddb7fe640 1 --2- 172.21.15.37:0/2718368610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ddc0ce510 0x7f7ddc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7ddaffd640 1 -- 172.21.15.37:0/2718368610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ddc0ce510 msgr2=0x7f7ddc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7ddaffd640 1 --2- 172.21.15.37:0/2718368610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ddc0ce510 0x7f7ddc0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7ddaffd640 1 -- 172.21.15.37:0/2718368610 >> v2:172.21.15.203:3300/0 conn(0x7f7ddc06ec60 msgr2=0x7f7ddc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7ddaffd640 1 --2- 172.21.15.37:0/2718368610 >> v2:172.21.15.203:3300/0 conn(0x7f7ddc06ec60 0x7f7ddc142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7ddaffd640 1 -- 172.21.15.37:0/2718368610 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0ccc70 con 0x7f7ddc144ca0 2024-04-25T08:53:22.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7ddaffd640 1 --2- 172.21.15.37:0/2718368610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ddc144ca0 0x7f7ddc06e720 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0cdbc0 tx=0x7f7dcc01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fef32b58219e826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7dda7fc640 1 -- 172.21.15.37:0/2718368610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0b5070 con 0x7f7ddc144ca0 2024-04-25T08:53:22.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7dda7fc640 1 -- 172.21.15.37:0/2718368610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc0a89d0 con 0x7f7ddc144ca0 2024-04-25T08:53:22.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.340+0000 7f7dda7fc640 1 -- 172.21.15.37:0/2718368610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc0a8da0 con 0x7f7ddc144ca0 2024-04-25T08:53:22.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 -- 172.21.15.37:0/2718368610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ddc144ca0 msgr2=0x7f7ddc06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 --2- 172.21.15.37:0/2718368610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ddc144ca0 0x7f7ddc06e720 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0cdbc0 tx=0x7f7dcc01ed10 comp rx=0 tx=0).stop 2024-04-25T08:53:22.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 -- 172.21.15.37:0/2718368610 shutdown_connections 2024-04-25T08:53:22.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 --2- 172.21.15.37:0/2718368610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ddc144ca0 0x7f7ddc06e720 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 --2- 172.21.15.37:0/2718368610 >> v2:172.21.15.203:3300/0 conn(0x7f7ddc06ec60 0x7f7ddc142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 --2- 172.21.15.37:0/2718368610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ddc0ce510 0x7f7ddc0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 -- 172.21.15.37:0/2718368610 >> 172.21.15.37:0/2718368610 conn(0x7f7ddc0b11a0 msgr2=0x7f7ddc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:22.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 -- 172.21.15.37:0/2718368610 shutdown_connections 2024-04-25T08:53:22.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 -- 172.21.15.37:0/2718368610 wait complete. 2024-04-25T08:53:22.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 Processor -- start 2024-04-25T08:53:22.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 -- start start 2024-04-25T08:53:22.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ddc06ec60 0x7f7ddc15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ddc0ce510 0x7f7ddc142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ddc144ca0 0x7f7ddc1e26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0cc900 con 0x7f7ddc144ca0 2024-04-25T08:53:22.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc06df20 con 0x7f7ddc06ec60 2024-04-25T08:53:22.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc1451f0 con 0x7f7ddc0ce510 2024-04-25T08:53:22.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ddc144ca0 0x7f7ddc1e26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ddc144ca0 0x7f7ddc1e26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41436/0 (socket says 172.21.15.37:41436) 2024-04-25T08:53:22.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddbfff640 1 -- 172.21.15.37:0/2870435648 learned_addr learned my addr 172.21.15.37:0/2870435648 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:22.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddb7fe640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ddc06ec60 0x7f7ddc15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddaffd640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ddc0ce510 0x7f7ddc142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddbfff640 1 -- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ddc0ce510 msgr2=0x7f7ddc142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddbfff640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ddc0ce510 0x7f7ddc142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddbfff640 1 -- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ddc06ec60 msgr2=0x7f7ddc15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddbfff640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ddc06ec60 0x7f7ddc15b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddbfff640 1 -- 172.21.15.37:0/2870435648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc1e0f10 con 0x7f7ddc144ca0 2024-04-25T08:53:22.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddaffd640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ddc0ce510 0x7f7ddc142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:22.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7ddbfff640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ddc144ca0 0x7f7ddc1e26b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4007e30 tx=0x7f7dd400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7dc3fff640 1 -- 172.21.15.37:0/2870435648 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4014420 con 0x7f7ddc144ca0 2024-04-25T08:53:22.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7dc3fff640 1 -- 172.21.15.37:0/2870435648 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd4019070 con 0x7f7ddc144ca0 2024-04-25T08:53:22.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7de2097640 1 -- 172.21.15.37:0/2870435648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc1e11f0 con 0x7f7ddc144ca0 2024-04-25T08:53:22.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.344+0000 7f7dc3fff640 1 -- 172.21.15.37:0/2870435648 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4014800 con 0x7f7ddc144ca0 2024-04-25T08:53:22.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.348+0000 7f7de2097640 1 -- 172.21.15.37:0/2870435648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc1e17b0 con 0x7f7ddc144ca0 2024-04-25T08:53:22.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.348+0000 7f7de2097640 1 -- 172.21.15.37:0/2870435648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ddc15b860 con 0x7f7ddc144ca0 2024-04-25T08:53:22.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.348+0000 7f7dc3fff640 1 -- 172.21.15.37:0/2870435648 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7dd40149a0 con 0x7f7ddc144ca0 2024-04-25T08:53:22.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.348+0000 7f7dc3fff640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc8066f90 0x7f7dc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.348+0000 7f7dc3fff640 1 -- 172.21.15.37:0/2870435648 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7dd4085bf0 con 0x7f7ddc144ca0 2024-04-25T08:53:22.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.348+0000 7f7ddb7fe640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc8066f90 0x7f7dc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.348+0000 7f7ddb7fe640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc8066f90 0x7f7dc8069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000b9d0 tx=0x7f7dd0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.352+0000 7f7dc3fff640 1 -- 172.21.15.37:0/2870435648 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7dd4056890 con 0x7f7ddc144ca0 2024-04-25T08:53:22.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.504+0000 7f7de2097640 1 -- 172.21.15.37:0/2870435648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ddc04efa0 con 0x7f7ddc144ca0 2024-04-25T08:53:22.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7dc3fff640 1 -- 172.21.15.37:0/2870435648 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f7dd4056190 con 0x7f7ddc144ca0 2024-04-25T08:53:22.501 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:22.501 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:22.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 -- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc8066f90 msgr2=0x7f7dc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc8066f90 0x7f7dc8069450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000b9d0 tx=0x7f7dd0008040 comp rx=0 tx=0).stop 2024-04-25T08:53:22.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 -- 172.21.15.37:0/2870435648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ddc144ca0 msgr2=0x7f7ddc1e26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ddc144ca0 0x7f7ddc1e26b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4007e30 tx=0x7f7dd400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 -- 172.21.15.37:0/2870435648 shutdown_connections 2024-04-25T08:53:22.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc8066f90 0x7f7dc8069450 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ddc144ca0 0x7f7ddc1e26b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ddc0ce510 0x7f7ddc142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 --2- 172.21.15.37:0/2870435648 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ddc06ec60 0x7f7ddc15b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 -- 172.21.15.37:0/2870435648 >> 172.21.15.37:0/2870435648 conn(0x7f7ddc0b11a0 msgr2=0x7f7ddc071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:22.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 -- 172.21.15.37:0/2870435648 shutdown_connections 2024-04-25T08:53:22.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.508+0000 7f7de2097640 1 -- 172.21.15.37:0/2870435648 wait complete. 2024-04-25T08:53:22.519 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:22.519 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.644+0000 7f14f007d640 1 Processor -- start 2024-04-25T08:53:22.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.644+0000 7f14f007d640 1 -- start start 2024-04-25T08:53:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14f007d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f14e815b6b0 0x7f14e80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14f007d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14e80cf590 0x7f14e80cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14f007d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e80cbd20 con 0x7f14e815f130 2024-04-25T08:53:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14f007d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e815ba90 con 0x7f14e815b6b0 2024-04-25T08:53:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14f007d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e80cff30 con 0x7f14e80cf590 2024-04-25T08:53:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ede19640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f14e815b6b0 0x7f14e80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ede19640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f14e815b6b0 0x7f14e80ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41442/0 (socket says 172.21.15.37:41442) 2024-04-25T08:53:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ede19640 1 -- 172.21.15.37:0/2406788273 learned_addr learned my addr 172.21.15.37:0/2406788273 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ed618640 1 --2- 172.21.15.37:0/2406788273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14e80cf590 0x7f14e80cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ede19640 1 -- 172.21.15.37:0/2406788273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14e80cf590 msgr2=0x7f14e80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ede19640 1 --2- 172.21.15.37:0/2406788273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14e80cf590 0x7f14e80cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ede19640 1 -- 172.21.15.37:0/2406788273 >> v1:172.21.15.203:6789/0 conn(0x7f14e815f130 legacy=0x7f14e815f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:22.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ede19640 1 -- 172.21.15.37:0/2406788273 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e80cc000 con 0x7f14e815b6b0 2024-04-25T08:53:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ede19640 1 --2- 172.21.15.37:0/2406788273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e815b6b0 0x7f14e80ca3c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009e50 tx=0x7f14d801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e06e780367aaad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ece17640 1 -- 172.21.15.37:0/2406788273 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d80a58d0 con 0x7f14e815b6b0 2024-04-25T08:53:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ece17640 1 -- 172.21.15.37:0/2406788273 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d80a5a70 con 0x7f14e815b6b0 2024-04-25T08:53:22.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.648+0000 7f14ece17640 1 -- 172.21.15.37:0/2406788273 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d80a5e40 con 0x7f14e815b6b0 2024-04-25T08:53:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 -- 172.21.15.37:0/2406788273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e815b6b0 msgr2=0x7f14e80ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 --2- 172.21.15.37:0/2406788273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e815b6b0 0x7f14e80ca3c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009e50 tx=0x7f14d801ca00 comp rx=0 tx=0).stop 2024-04-25T08:53:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 -- 172.21.15.37:0/2406788273 shutdown_connections 2024-04-25T08:53:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 --2- 172.21.15.37:0/2406788273 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14e80cf590 0x7f14e80cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 --2- 172.21.15.37:0/2406788273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e815b6b0 0x7f14e80ca3c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 -- 172.21.15.37:0/2406788273 >> 172.21.15.37:0/2406788273 conn(0x7f14e806ae60 msgr2=0x7f14e806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 -- 172.21.15.37:0/2406788273 shutdown_connections 2024-04-25T08:53:22.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 -- 172.21.15.37:0/2406788273 wait complete. 2024-04-25T08:53:22.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 Processor -- start 2024-04-25T08:53:22.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 -- start start 2024-04-25T08:53:22.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14e80cf590 0x7f14e80c5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14e815b6b0 0x7f14e8159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e815f130 0x7f14e8164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e80ca900 con 0x7f14e815f130 2024-04-25T08:53:22.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e80cba30 con 0x7f14e80cf590 2024-04-25T08:53:22.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e80cadb0 con 0x7f14e815b6b0 2024-04-25T08:53:22.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ee61a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e815f130 0x7f14e8164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ee61a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e815f130 0x7f14e8164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41446/0 (socket says 172.21.15.37:41446) 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ee61a640 1 -- 172.21.15.37:0/1963966954 learned_addr learned my addr 172.21.15.37:0/1963966954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ede19640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14e80cf590 0x7f14e80c5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ed618640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14e815b6b0 0x7f14e8159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ee61a640 1 -- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14e815b6b0 msgr2=0x7f14e8159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ee61a640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14e815b6b0 0x7f14e8159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ee61a640 1 -- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14e80cf590 msgr2=0x7f14e80c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ee61a640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14e80cf590 0x7f14e80c5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ee61a640 1 -- 172.21.15.37:0/1963966954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e80c4bd0 con 0x7f14e815f130 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ed618640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14e815b6b0 0x7f14e8159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14ee61a640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e815f130 0x7f14e8164670 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14e000c370 tx=0x7f14e000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14e67fc640 1 -- 172.21.15.37:0/1963966954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0005220 con 0x7f14e815f130 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14e67fc640 1 -- 172.21.15.37:0/1963966954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e00053c0 con 0x7f14e815f130 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 -- 172.21.15.37:0/1963966954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e815a590 con 0x7f14e815f130 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14e67fc640 1 -- 172.21.15.37:0/1963966954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e0007270 con 0x7f14e815f130 2024-04-25T08:53:22.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.652+0000 7f14f007d640 1 -- 172.21.15.37:0/1963966954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e80c4e70 con 0x7f14e815f130 2024-04-25T08:53:22.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.656+0000 7f14f007d640 1 -- 172.21.15.37:0/1963966954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e804efa0 con 0x7f14e815f130 2024-04-25T08:53:22.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.656+0000 7f14e67fc640 1 -- 172.21.15.37:0/1963966954 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14e0019020 con 0x7f14e815f130 2024-04-25T08:53:22.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.656+0000 7f14e67fc640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f14d4066f60 0x7f14d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.656+0000 7f14e67fc640 1 -- 172.21.15.37:0/1963966954 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f14e0084680 con 0x7f14e815f130 2024-04-25T08:53:22.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.656+0000 7f14ede19640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f14d4066f60 0x7f14d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.656+0000 7f14ede19640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f14d4066f60 0x7f14d4069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14d80ae000 tx=0x7f14d80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.660+0000 7f14e67fc640 1 -- 172.21.15.37:0/1963966954 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14e0055320 con 0x7f14e815f130 2024-04-25T08:53:22.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.812+0000 7f14f007d640 1 -- 172.21.15.37:0/1963966954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f14e80b0060 con 0x7f14e815f130 2024-04-25T08:53:22.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.812+0000 7f14e67fc640 1 -- 172.21.15.37:0/1963966954 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f14e0054c20 con 0x7f14e815f130 2024-04-25T08:53:22.807 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 -- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f14d4066f60 msgr2=0x7f14d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f14d4066f60 0x7f14d4069420 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f14d80ae000 tx=0x7f14d80a7070 comp rx=0 tx=0).stop 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 -- 172.21.15.37:0/1963966954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e815f130 msgr2=0x7f14e8164670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e815f130 0x7f14e8164670 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f14e000c370 tx=0x7f14e000c830 comp rx=0 tx=0).stop 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 -- 172.21.15.37:0/1963966954 shutdown_connections 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f14d4066f60 0x7f14d4069420 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14e815f130 0x7f14e8164670 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14e815b6b0 0x7f14e8159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 --2- 172.21.15.37:0/1963966954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14e80cf590 0x7f14e80c5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 -- 172.21.15.37:0/1963966954 >> 172.21.15.37:0/1963966954 conn(0x7f14e806ae60 msgr2=0x7f14e8165a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 -- 172.21.15.37:0/1963966954 shutdown_connections 2024-04-25T08:53:22.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.816+0000 7f14f007d640 1 -- 172.21.15.37:0/1963966954 wait complete. 2024-04-25T08:53:22.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.960+0000 7fe33f043640 1 Processor -- start 2024-04-25T08:53:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.960+0000 7fe33f043640 1 -- start start 2024-04-25T08:53:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33f043640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe33806c3b0 0x7fe338164d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33f043640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe338143b70 con 0x7fe338143790 2024-04-25T08:53:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33f043640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe33806e170 con 0x7fe33815da40 2024-04-25T08:53:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33f043640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33806e450 con 0x7fe33806c3b0 2024-04-25T08:53:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33cddf640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe338143790 0x7fe33806d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41804/0 (socket says 172.21.15.37:41804) 2024-04-25T08:53:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33cddf640 1 -- 172.21.15.37:0/1114682830 learned_addr learned my addr 172.21.15.37:0/1114682830 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33d5e0640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe33806c3b0 0x7fe338164d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513602739 0 0) 0x7fe338143b70 con 0x7fe338143790 2024-04-25T08:53:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe320003530 con 0x7fe338143790 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1043981259 0 0) 0x7fe320003530 con 0x7fe338143790 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe33806c3b0 msgr2=0x7fe338164d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe33806c3b0 0x7fe338164d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 >> v1:172.21.15.203:6789/0 conn(0x7fe33815da40 legacy=0x7fe33806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33806e730 con 0x7fe338143790 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fe328001680 con 0x7fe338143790 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 >> v1:172.21.15.37:6789/0 conn(0x7fe338143790 legacy=0x7fe33806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe320003a60 0x7fe320003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe33806c3b0 0x7fe320004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 0x7fe3200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33806e450 con 0x7fe320004d50 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32000ba80 con 0x7fe320003a60 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32000bc80 con 0x7fe33806c3b0 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33d5e0640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 0x7fe3200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33cddf640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe320003a60 0x7fe320003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe337fff640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe33806c3b0 0x7fe320004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33d5e0640 1 -- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe33806c3b0 msgr2=0x7fe320004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33d5e0640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe33806c3b0 0x7fe320004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33d5e0640 1 -- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe320003a60 msgr2=0x7fe320003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33d5e0640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe320003a60 0x7fe320003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33d5e0640 1 -- 172.21.15.37:0/1114682830 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe320003530 con 0x7fe320004d50 2024-04-25T08:53:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe337fff640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe33806c3b0 0x7fe320004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.964+0000 7fe33cddf640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe320003a60 0x7fe320003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33d5e0640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 0x7fe3200051d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe330009ae0 tx=0x7fe33001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40010b1688fb0041 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300b2070 con 0x7fe320004d50 2024-04-25T08:53:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3300a5550 con 0x7fe320004d50 2024-04-25T08:53:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe3377fe640 1 -- 172.21.15.37:0/1114682830 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3300096b0 con 0x7fe320004d50 2024-04-25T08:53:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 -- 172.21.15.37:0/1114682830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 msgr2=0x7fe3200051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 --2- 172.21.15.37:0/1114682830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 0x7fe3200051d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe330009ae0 tx=0x7fe33001ca70 comp rx=0 tx=0).stop 2024-04-25T08:53:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33d5e0640 1 -- 172.21.15.37:0/1114682830 reap_dead start 2024-04-25T08:53:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 -- 172.21.15.37:0/1114682830 shutdown_connections 2024-04-25T08:53:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 -- 172.21.15.37:0/1114682830 >> 172.21.15.37:0/1114682830 conn(0x7fe338071810 msgr2=0x7fe338071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:22.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 -- 172.21.15.37:0/1114682830 shutdown_connections 2024-04-25T08:53:22.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 -- 172.21.15.37:0/1114682830 wait complete. 2024-04-25T08:53:22.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 Processor -- start 2024-04-25T08:53:22.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 -- start start 2024-04-25T08:53:22.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 0x7fe33813d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3380cf5d0 0x7fe33813bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3381557e0 0x7fe338155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380b4380 con 0x7fe320004d50 2024-04-25T08:53:22.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380b4200 con 0x7fe3380cf5d0 2024-04-25T08:53:22.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33f043640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3380b4500 con 0x7fe3381557e0 2024-04-25T08:53:22.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33cddf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 0x7fe33813d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33cddf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 0x7fe33813d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41460/0 (socket says 172.21.15.37:41460) 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33cddf640 1 -- 172.21.15.37:0/802706881 learned_addr learned my addr 172.21.15.37:0/802706881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe337fff640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3380cf5d0 0x7fe33813bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33d5e0640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3381557e0 0x7fe338155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33cddf640 1 -- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3381557e0 msgr2=0x7fe338155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33cddf640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3381557e0 0x7fe338155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33cddf640 1 -- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3380cf5d0 msgr2=0x7fe33813bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33cddf640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3380cf5d0 0x7fe33813bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33cddf640 1 -- 172.21.15.37:0/802706881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33813dc00 con 0x7fe320004d50 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe33cddf640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 0x7fe33813d320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe3280058b0 tx=0x7fe32800e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.968+0000 7fe3357fa640 1 -- 172.21.15.37:0/802706881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe32800ebe0 con 0x7fe320004d50 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.972+0000 7fe3357fa640 1 -- 172.21.15.37:0/802706881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe328011040 con 0x7fe320004d50 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.972+0000 7fe33f043640 1 -- 172.21.15.37:0/802706881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe338156590 con 0x7fe320004d50 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.972+0000 7fe3357fa640 1 -- 172.21.15.37:0/802706881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe328015400 con 0x7fe320004d50 2024-04-25T08:53:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.972+0000 7fe33f043640 1 -- 172.21.15.37:0/802706881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe338156b50 con 0x7fe320004d50 2024-04-25T08:53:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.972+0000 7fe33f043640 1 -- 172.21.15.37:0/802706881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe33804efa0 con 0x7fe320004d50 2024-04-25T08:53:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.972+0000 7fe3357fa640 1 -- 172.21.15.37:0/802706881 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe32801d020 con 0x7fe320004d50 2024-04-25T08:53:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.972+0000 7fe3357fa640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe324066f90 0x7fe324069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.972+0000 7fe3357fa640 1 -- 172.21.15.37:0/802706881 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fe328088420 con 0x7fe320004d50 2024-04-25T08:53:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.972+0000 7fe337fff640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe324066f90 0x7fe324069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.972+0000 7fe337fff640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe324066f90 0x7fe324069450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe33813c840 tx=0x7fe32c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:22.976+0000 7fe3357fa640 1 -- 172.21.15.37:0/802706881 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe328059030 con 0x7fe320004d50 2024-04-25T08:53:23.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.132+0000 7fe33f043640 1 -- 172.21.15.37:0/802706881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe338072670 con 0x7fe320004d50 2024-04-25T08:53:23.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe3357fa640 1 -- 172.21.15.37:0/802706881 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe328058930 con 0x7fe320004d50 2024-04-25T08:53:23.128 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:23.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 -- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe324066f90 msgr2=0x7fe324069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe324066f90 0x7fe324069450 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe33813c840 tx=0x7fe32c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:23.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 -- 172.21.15.37:0/802706881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 msgr2=0x7fe33813d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 0x7fe33813d320 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe3280058b0 tx=0x7fe32800e4d0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 -- 172.21.15.37:0/802706881 shutdown_connections 2024-04-25T08:53:23.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe324066f90 0x7fe324069450 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3381557e0 0x7fe338155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe3380cf5d0 0x7fe33813bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 --2- 172.21.15.37:0/802706881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe320004d50 0x7fe33813d320 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 -- 172.21.15.37:0/802706881 >> 172.21.15.37:0/802706881 conn(0x7fe338071810 msgr2=0x7fe33815b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:23.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 -- 172.21.15.37:0/802706881 shutdown_connections 2024-04-25T08:53:23.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.136+0000 7fe33f043640 1 -- 172.21.15.37:0/802706881 wait complete. 2024-04-25T08:53:23.145 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:23.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.268+0000 7f4a3e51a640 1 Processor -- start 2024-04-25T08:53:23.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.268+0000 7f4a3e51a640 1 -- start start 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.268+0000 7f4a3e51a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a3815da80 0x7f4a380d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.268+0000 7f4a3e51a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4a380d1cd0 0x7f4a380d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.268+0000 7f4a3e51a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4a380d0620 0x7f4a380ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.268+0000 7f4a3e51a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3806c440 con 0x7f4a380d1cd0 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.268+0000 7f4a3e51a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a3806be60 con 0x7f4a380d0620 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.268+0000 7f4a3e51a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3815de60 con 0x7f4a3815da80 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.268+0000 7f4a377fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4a380d1cd0 0x7f4a380d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.268+0000 7f4a377fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4a380d1cd0 0x7f4a380d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41466/0 (socket says 172.21.15.37:41466) 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a377fe640 1 -- 172.21.15.37:0/453834846 learned_addr learned my addr 172.21.15.37:0/453834846 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a3cab7640 1 --2- 172.21.15.37:0/453834846 >> v2:172.21.15.203:3300/0 conn(0x7f4a380d0620 0x7f4a380ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a37fff640 1 --2- 172.21.15.37:0/453834846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a3815da80 0x7f4a380d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a377fe640 1 -- 172.21.15.37:0/453834846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a3815da80 msgr2=0x7f4a380d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:23.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a377fe640 1 --2- 172.21.15.37:0/453834846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a3815da80 0x7f4a380d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a377fe640 1 -- 172.21.15.37:0/453834846 >> v2:172.21.15.203:3300/0 conn(0x7f4a380d0620 msgr2=0x7f4a380ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:23.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a377fe640 1 --2- 172.21.15.37:0/453834846 >> v2:172.21.15.203:3300/0 conn(0x7f4a380d0620 0x7f4a380ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a377fe640 1 -- 172.21.15.37:0/453834846 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380d2450 con 0x7f4a380d1cd0 2024-04-25T08:53:23.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a377fe640 1 --2- 172.21.15.37:0/453834846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a380d1cd0 0x7f4a380d00e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a3806de60 tx=0x7f4a2c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ba0610b6e28a0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:23.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a36ffd640 1 -- 172.21.15.37:0/453834846 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0ab5f0 con 0x7f4a380d1cd0 2024-04-25T08:53:23.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a36ffd640 1 -- 172.21.15.37:0/453834846 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2c0b0070 con 0x7f4a380d1cd0 2024-04-25T08:53:23.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.272+0000 7f4a36ffd640 1 -- 172.21.15.37:0/453834846 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c0ab9b0 con 0x7f4a380d1cd0 2024-04-25T08:53:23.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 -- 172.21.15.37:0/453834846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a380d1cd0 msgr2=0x7f4a380d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 --2- 172.21.15.37:0/453834846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a380d1cd0 0x7f4a380d00e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f4a3806de60 tx=0x7f4a2c01c520 comp rx=0 tx=0).stop 2024-04-25T08:53:23.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 -- 172.21.15.37:0/453834846 shutdown_connections 2024-04-25T08:53:23.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 --2- 172.21.15.37:0/453834846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a380d1cd0 0x7f4a380d00e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 --2- 172.21.15.37:0/453834846 >> v2:172.21.15.203:3300/0 conn(0x7f4a380d0620 0x7f4a380ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 --2- 172.21.15.37:0/453834846 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a3815da80 0x7f4a380d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 -- 172.21.15.37:0/453834846 >> 172.21.15.37:0/453834846 conn(0x7f4a3808b000 msgr2=0x7f4a380d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:23.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 -- 172.21.15.37:0/453834846 shutdown_connections 2024-04-25T08:53:23.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 -- 172.21.15.37:0/453834846 wait complete. 2024-04-25T08:53:23.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 Processor -- start 2024-04-25T08:53:23.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 -- start start 2024-04-25T08:53:23.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a380d0620 0x7f4a380b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a380d1cd0 0x7f4a38157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3815da80 0x7f4a380b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3806ca30 con 0x7f4a3815da80 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3806c8b0 con 0x7f4a380d1cd0 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a3806cbb0 con 0x7f4a380d0620 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3cab7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3815da80 0x7f4a380b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3cab7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3815da80 0x7f4a380b74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41480/0 (socket says 172.21.15.37:41480) 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3cab7640 1 -- 172.21.15.37:0/170215529 learned_addr learned my addr 172.21.15.37:0/170215529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a37fff640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a380d0620 0x7f4a380b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a377fe640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a380d1cd0 0x7f4a38157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3cab7640 1 -- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a380d0620 msgr2=0x7f4a380b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3cab7640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a380d0620 0x7f4a380b8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3cab7640 1 -- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a380d1cd0 msgr2=0x7f4a38157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3cab7640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a380d1cd0 0x7f4a38157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3cab7640 1 -- 172.21.15.37:0/170215529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a380b9200 con 0x7f4a3815da80 2024-04-25T08:53:23.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a37fff640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a380d0620 0x7f4a380b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:23.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3cab7640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3815da80 0x7f4a380b74e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4a30007e90 tx=0x7f4a3000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:23.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a34ff9640 1 -- 172.21.15.37:0/170215529 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30014420 con 0x7f4a3815da80 2024-04-25T08:53:23.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a34ff9640 1 -- 172.21.15.37:0/170215529 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a30019070 con 0x7f4a3815da80 2024-04-25T08:53:23.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a34ff9640 1 -- 172.21.15.37:0/170215529 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30014820 con 0x7f4a3815da80 2024-04-25T08:53:23.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 -- 172.21.15.37:0/170215529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a381583e0 con 0x7f4a3815da80 2024-04-25T08:53:23.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.276+0000 7f4a3e51a640 1 -- 172.21.15.37:0/170215529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a38158c80 con 0x7f4a3815da80 2024-04-25T08:53:23.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.280+0000 7f4a3e51a640 1 -- 172.21.15.37:0/170215529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3804efa0 con 0x7f4a3815da80 2024-04-25T08:53:23.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.280+0000 7f4a34ff9640 1 -- 172.21.15.37:0/170215529 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a300149c0 con 0x7f4a3815da80 2024-04-25T08:53:23.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.280+0000 7f4a34ff9640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a24066f60 0x7f4a24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.280+0000 7f4a34ff9640 1 -- 172.21.15.37:0/170215529 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4a30085bd0 con 0x7f4a3815da80 2024-04-25T08:53:23.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.280+0000 7f4a37fff640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a24066f60 0x7f4a24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.280+0000 7f4a37fff640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a24066f60 0x7f4a24069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800b9f0 tx=0x7f4a28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:23.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.284+0000 7f4a34ff9640 1 -- 172.21.15.37:0/170215529 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a30056810 con 0x7f4a3815da80 2024-04-25T08:53:23.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 -- 172.21.15.37:0/170215529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4a3806ad40 con 0x7f4a3815da80 2024-04-25T08:53:23.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a34ff9640 1 -- 172.21.15.37:0/170215529 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4a30056110 con 0x7f4a3815da80 2024-04-25T08:53:23.432 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 -- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a24066f60 msgr2=0x7f4a24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a24066f60 0x7f4a24069420 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800b9f0 tx=0x7f4a28008040 comp rx=0 tx=0).stop 2024-04-25T08:53:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 -- 172.21.15.37:0/170215529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3815da80 msgr2=0x7f4a380b74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3815da80 0x7f4a380b74e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4a30007e90 tx=0x7f4a3000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 -- 172.21.15.37:0/170215529 shutdown_connections 2024-04-25T08:53:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a24066f60 0x7f4a24069420 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a3815da80 0x7f4a380b74e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a380d1cd0 0x7f4a38157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 --2- 172.21.15.37:0/170215529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a380d0620 0x7f4a380b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 -- 172.21.15.37:0/170215529 >> 172.21.15.37:0/170215529 conn(0x7f4a3808b000 msgr2=0x7f4a380b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 -- 172.21.15.37:0/170215529 shutdown_connections 2024-04-25T08:53:23.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.440+0000 7f4a3e51a640 1 -- 172.21.15.37:0/170215529 wait complete. 2024-04-25T08:53:23.450 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:23.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d41d69640 1 Processor -- start 2024-04-25T08:53:23.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d41d69640 1 -- start start 2024-04-25T08:53:23.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d41d69640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c0cfca0 0x7f6d3c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d41d69640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6d3c15e4f0 0x7f6d3c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d41d69640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d3c0cbf10 con 0x7f6d3c0b3cf0 2024-04-25T08:53:23.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d41d69640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3c0cc1f0 con 0x7f6d3c15e4f0 2024-04-25T08:53:23.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d41d69640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c0cc4d0 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d3bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6d3c15e4f0 0x7f6d3c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d3bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6d3c15e4f0 0x7f6d3c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41496/0 (socket says 172.21.15.37:41496) 2024-04-25T08:53:23.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d3bfff640 1 -- 172.21.15.37:0/3556361074 learned_addr learned my addr 172.21.15.37:0/3556361074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:23.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d3affd640 1 --2- 172.21.15.37:0/3556361074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c0cfca0 0x7f6d3c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d3bfff640 1 -- 172.21.15.37:0/3556361074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c0cfca0 msgr2=0x7f6d3c15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:23.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d3bfff640 1 --2- 172.21.15.37:0/3556361074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c0cfca0 0x7f6d3c15dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d3bfff640 1 -- 172.21.15.37:0/3556361074 >> v1:172.21.15.203:6789/0 conn(0x7f6d3c0b3cf0 legacy=0x7f6d3c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:23.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d3bfff640 1 -- 172.21.15.37:0/3556361074 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c0cc7b0 con 0x7f6d3c15e4f0 2024-04-25T08:53:23.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.576+0000 7f6d3affd640 1 --2- 172.21.15.37:0/3556361074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c0cfca0 0x7f6d3c15dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d3bfff640 1 --2- 172.21.15.37:0/3556361074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d3c15e4f0 0x7f6d3c0caaf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009870 tx=0x7f6d2c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e768672d50fd9982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d3a7fc640 1 -- 172.21.15.37:0/3556361074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0b0070 con 0x7f6d3c15e4f0 2024-04-25T08:53:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d3a7fc640 1 -- 172.21.15.37:0/3556361074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2c0ab600 con 0x7f6d3c15e4f0 2024-04-25T08:53:23.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d3a7fc640 1 -- 172.21.15.37:0/3556361074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2c0ab9d0 con 0x7f6d3c15e4f0 2024-04-25T08:53:23.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 -- 172.21.15.37:0/3556361074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d3c15e4f0 msgr2=0x7f6d3c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 --2- 172.21.15.37:0/3556361074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d3c15e4f0 0x7f6d3c0caaf0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c009870 tx=0x7f6d2c01c520 comp rx=0 tx=0).stop 2024-04-25T08:53:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 -- 172.21.15.37:0/3556361074 shutdown_connections 2024-04-25T08:53:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 --2- 172.21.15.37:0/3556361074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d3c15e4f0 0x7f6d3c0caaf0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 --2- 172.21.15.37:0/3556361074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c0cfca0 0x7f6d3c15dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 -- 172.21.15.37:0/3556361074 >> 172.21.15.37:0/3556361074 conn(0x7f6d3c06ae60 msgr2=0x7f6d3c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 -- 172.21.15.37:0/3556361074 shutdown_connections 2024-04-25T08:53:23.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 -- 172.21.15.37:0/3556361074 wait complete. 2024-04-25T08:53:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 Processor -- start 2024-04-25T08:53:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 -- start start 2024-04-25T08:53:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3c0b3cf0 0x7f6d3c1534c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d3c0cfca0 0x7f6d3c15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c15e4f0 0x7f6d3c159d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c0cb9c0 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c0cb840 con 0x7f6d3c0b3cf0 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d41d69640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c0cbb40 con 0x7f6d3c15e4f0 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d3c0cfca0 0x7f6d3c15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d3c0cfca0 0x7f6d3c15b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41498/0 (socket says 172.21.15.37:41498) 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.580+0000 7f6d3affd640 1 -- 172.21.15.37:0/1873480091 learned_addr learned my addr 172.21.15.37:0/1873480091 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3bfff640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c15e4f0 0x7f6d3c159d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3b7fe640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3c0b3cf0 0x7f6d3c1534c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3affd640 1 -- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c15e4f0 msgr2=0x7f6d3c159d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3affd640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c15e4f0 0x7f6d3c159d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3affd640 1 -- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3c0b3cf0 msgr2=0x7f6d3c1534c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3affd640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3c0b3cf0 0x7f6d3c1534c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3affd640 1 -- 172.21.15.37:0/1873480091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c15bd30 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3bfff640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c15e4f0 0x7f6d3c159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3b7fe640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3c0b3cf0 0x7f6d3c1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:23.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3affd640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d3c0cfca0 0x7f6d3c15b430 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400cf90 tx=0x7f6d2400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:23.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d33fff640 1 -- 172.21.15.37:0/1873480091 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2401a070 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d33fff640 1 -- 172.21.15.37:0/1873480091 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2400d840 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d41d69640 1 -- 172.21.15.37:0/1873480091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d3c15c010 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d33fff640 1 -- 172.21.15.37:0/1873480091 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2400dc10 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d41d69640 1 -- 172.21.15.37:0/1873480091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3c1597e0 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d41d69640 1 -- 172.21.15.37:0/1873480091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3c1599e0 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d33fff640 1 -- 172.21.15.37:0/1873480091 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d2401d070 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d33fff640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d1c066f60 0x7f6d1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d33fff640 1 -- 172.21.15.37:0/1873480091 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6d24085cb0 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3b7fe640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d1c066f60 0x7f6d1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.584+0000 7f6d3b7fe640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d1c066f60 0x7f6d1c069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c15e1e0 tx=0x7f6d34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:23.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.588+0000 7f6d33fff640 1 -- 172.21.15.37:0/1873480091 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d3c1599e0 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.748+0000 7f6d41d69640 1 -- 172.21.15.37:0/1873480091 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6d3c04efa0 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.748+0000 7f6d33fff640 1 -- 172.21.15.37:0/1873480091 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6d240561d0 con 0x7f6d3c0cfca0 2024-04-25T08:53:23.741 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:23.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.748+0000 7f6d41d69640 1 -- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d1c066f60 msgr2=0x7f6d1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.748+0000 7f6d41d69640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d1c066f60 0x7f6d1c069420 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c15e1e0 tx=0x7f6d34009040 comp rx=0 tx=0).stop 2024-04-25T08:53:23.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.748+0000 7f6d41d69640 1 -- 172.21.15.37:0/1873480091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d3c0cfca0 msgr2=0x7f6d3c15b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.748+0000 7f6d41d69640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d3c0cfca0 0x7f6d3c15b430 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400cf90 tx=0x7f6d2400a7d0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.752+0000 7f6d41d69640 1 -- 172.21.15.37:0/1873480091 shutdown_connections 2024-04-25T08:53:23.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.752+0000 7f6d41d69640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d1c066f60 0x7f6d1c069420 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.752+0000 7f6d41d69640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d3c15e4f0 0x7f6d3c159d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.752+0000 7f6d41d69640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d3c0cfca0 0x7f6d3c15b430 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.752+0000 7f6d41d69640 1 --2- 172.21.15.37:0/1873480091 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d3c0b3cf0 0x7f6d3c1534c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.752+0000 7f6d41d69640 1 -- 172.21.15.37:0/1873480091 >> 172.21.15.37:0/1873480091 conn(0x7f6d3c06ae60 msgr2=0x7f6d3c0cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:23.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.752+0000 7f6d41d69640 1 -- 172.21.15.37:0/1873480091 shutdown_connections 2024-04-25T08:53:23.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.752+0000 7f6d41d69640 1 -- 172.21.15.37:0/1873480091 wait complete. 2024-04-25T08:53:23.761 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T08:53:23.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d7bf49640 1 Processor -- start 2024-04-25T08:53:23.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d7bf49640 1 -- start start 2024-04-25T08:53:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d7bf49640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2d740d07f0 0x7f2d740d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d7bf49640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2d740d1cd0 0x7f2d7406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d7bf49640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7406be60 con 0x7f2d7415da80 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d7bf49640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7406c440 con 0x7f2d740d07f0 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d7bf49640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7415de60 con 0x7f2d740d1cd0 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d794e4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2d740d07f0 0x7f2d740d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d794e4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2d740d07f0 0x7f2d740d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41506/0 (socket says 172.21.15.37:41506) 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d794e4640 1 -- 172.21.15.37:0/3830061297 learned_addr learned my addr 172.21.15.37:0/3830061297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d7a4e6640 1 --2- 172.21.15.37:0/3830061297 >> v2:172.21.15.203:3300/0 conn(0x7f2d740d1cd0 0x7f2d7406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d794e4640 1 -- 172.21.15.37:0/3830061297 >> v2:172.21.15.203:3300/0 conn(0x7f2d740d1cd0 msgr2=0x7f2d7406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d794e4640 1 --2- 172.21.15.37:0/3830061297 >> v2:172.21.15.203:3300/0 conn(0x7f2d740d1cd0 0x7f2d7406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d794e4640 1 -- 172.21.15.37:0/3830061297 >> v1:172.21.15.203:6789/0 conn(0x7f2d7415da80 legacy=0x7f2d740d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d794e4640 1 -- 172.21.15.37:0/3830061297 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d740d2450 con 0x7f2d740d07f0 2024-04-25T08:53:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.884+0000 7f2d7a4e6640 1 --2- 172.21.15.37:0/3830061297 >> v2:172.21.15.203:3300/0 conn(0x7f2d740d1cd0 0x7f2d7406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:23.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d794e4640 1 --2- 172.21.15.37:0/3830061297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d740d07f0 0x7f2d740d4230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2d7406de30 tx=0x7f2d6401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e716f7108d149b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:23.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d78ce3640 1 -- 172.21.15.37:0/3830061297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640b1070 con 0x7f2d740d07f0 2024-04-25T08:53:23.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d78ce3640 1 -- 172.21.15.37:0/3830061297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d640a8050 con 0x7f2d740d07f0 2024-04-25T08:53:23.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d78ce3640 1 -- 172.21.15.37:0/3830061297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640ac860 con 0x7f2d740d07f0 2024-04-25T08:53:23.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 -- 172.21.15.37:0/3830061297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d740d07f0 msgr2=0x7f2d740d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 --2- 172.21.15.37:0/3830061297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d740d07f0 0x7f2d740d4230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2d7406de30 tx=0x7f2d6401c4b0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 -- 172.21.15.37:0/3830061297 shutdown_connections 2024-04-25T08:53:23.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 --2- 172.21.15.37:0/3830061297 >> v2:172.21.15.203:3300/0 conn(0x7f2d740d1cd0 0x7f2d7406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 --2- 172.21.15.37:0/3830061297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d740d07f0 0x7f2d740d4230 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 -- 172.21.15.37:0/3830061297 >> 172.21.15.37:0/3830061297 conn(0x7f2d7408b000 msgr2=0x7f2d740d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:23.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 -- 172.21.15.37:0/3830061297 shutdown_connections 2024-04-25T08:53:23.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 -- 172.21.15.37:0/3830061297 wait complete. 2024-04-25T08:53:23.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 Processor -- start 2024-04-25T08:53:23.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 -- start start 2024-04-25T08:53:23.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d740d07f0 0x7f2d74154a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d740d1cd0 0x7f2d74156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d7415da80 0x7f2d740caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7406d7a0 con 0x7f2d740d1cd0 2024-04-25T08:53:23.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7406d620 con 0x7f2d7415da80 2024-04-25T08:53:23.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7406d920 con 0x7f2d740d07f0 2024-04-25T08:53:23.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d794e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d740d1cd0 0x7f2d74156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d794e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d740d1cd0 0x7f2d74156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41516/0 (socket says 172.21.15.37:41516) 2024-04-25T08:53:23.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d794e4640 1 -- 172.21.15.37:0/2843248089 learned_addr learned my addr 172.21.15.37:0/2843248089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d794e4640 1 -- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d740d07f0 msgr2=0x7f2d74154a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d794e4640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d740d07f0 0x7f2d74154a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7a4e6640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d7415da80 0x7f2d740caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d794e4640 1 -- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d7415da80 msgr2=0x7f2d740caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d794e4640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d7415da80 0x7f2d740caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d794e4640 1 -- 172.21.15.37:0/2843248089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d740cd170 con 0x7f2d740d1cd0 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d794e4640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d740d1cd0 0x7f2d74156270 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2d6401a0e0 tx=0x7f2d64019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d6a7fc640 1 -- 172.21.15.37:0/2843248089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640bb070 con 0x7f2d740d1cd0 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d6a7fc640 1 -- 172.21.15.37:0/2843248089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d64003a20 con 0x7f2d740d1cd0 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 -- 172.21.15.37:0/2843248089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d740cd450 con 0x7f2d740d1cd0 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d6a7fc640 1 -- 172.21.15.37:0/2843248089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d640b1040 con 0x7f2d740d1cd0 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.888+0000 7f2d7bf49640 1 -- 172.21.15.37:0/2843248089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d740cba90 con 0x7f2d740d1cd0 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.892+0000 7f2d7bf49640 1 -- 172.21.15.37:0/2843248089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7404efa0 con 0x7f2d740d1cd0 2024-04-25T08:53:23.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.892+0000 7f2d6a7fc640 1 -- 172.21.15.37:0/2843248089 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d640be070 con 0x7f2d740d1cd0 2024-04-25T08:53:23.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.892+0000 7f2d6a7fc640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d58066f60 0x7f2d58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:23.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.892+0000 7f2d6a7fc640 1 -- 172.21.15.37:0/2843248089 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2d6411e6e0 con 0x7f2d740d1cd0 2024-04-25T08:53:23.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.892+0000 7f2d79ce5640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d58066f60 0x7f2d58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:23.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.892+0000 7f2d79ce5640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d58066f60 0x7f2d58069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009670 tx=0x7f2d70006bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:23.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:23.896+0000 7f2d6a7fc640 1 -- 172.21.15.37:0/2843248089 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d640160d0 con 0x7f2d740d1cd0 2024-04-25T08:53:24.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.112+0000 7f2d7bf49640 1 -- 172.21.15.37:0/2843248089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f2d74154fa0 con 0x7f2d740d1cd0 2024-04-25T08:53:24.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d6a7fc640 1 -- 172.21.15.37:0/2843248089 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v70) v1 ==== 145+0+0 (secure 0 0 0) 0x7f2d640ef380 con 0x7f2d740d1cd0 2024-04-25T08:53:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 -- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d58066f60 msgr2=0x7f2d58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:24.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d58066f60 0x7f2d58069420 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2d70009670 tx=0x7f2d70006bd0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 -- 172.21.15.37:0/2843248089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d740d1cd0 msgr2=0x7f2d74156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d740d1cd0 0x7f2d74156270 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2d6401a0e0 tx=0x7f2d64019250 comp rx=0 tx=0).stop 2024-04-25T08:53:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 -- 172.21.15.37:0/2843248089 shutdown_connections 2024-04-25T08:53:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d58066f60 0x7f2d58069420 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d7415da80 0x7f2d740caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d740d1cd0 0x7f2d74156270 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 --2- 172.21.15.37:0/2843248089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d740d07f0 0x7f2d74154a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 -- 172.21.15.37:0/2843248089 >> 172.21.15.37:0/2843248089 conn(0x7f2d7408b000 msgr2=0x7f2d740b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 -- 172.21.15.37:0/2843248089 shutdown_connections 2024-04-25T08:53:24.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.280+0000 7f2d7bf49640 1 -- 172.21.15.37:0/2843248089 wait complete. 2024-04-25T08:53:24.295 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:53:24.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4786929640 1 Processor -- start 2024-04-25T08:53:24.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4786929640 1 -- start start 2024-04-25T08:53:24.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4786929640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f478015b6b0 0x7f47800ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4786929640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4780073010 0x7f47800cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4786929640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47800cbd20 con 0x7f47800cf590 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4786929640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f478015ba90 con 0x7f4780073010 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4786929640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47800cc000 con 0x7f478015b6b0 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4784ec6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4780073010 0x7f47800cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4784ec6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4780073010 0x7f47800cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41524/0 (socket says 172.21.15.37:41524) 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4777fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f47800cf590 0x7f478015f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41806/0 (socket says 172.21.15.37:41806) 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4784ec6640 1 -- 172.21.15.37:0/874594361 learned_addr learned my addr 172.21.15.37:0/874594361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f477ffff640 1 --2- 172.21.15.37:0/874594361 >> v2:172.21.15.203:3300/0 conn(0x7f478015b6b0 0x7f47800ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4784ec6640 1 -- 172.21.15.37:0/874594361 >> v2:172.21.15.203:3300/0 conn(0x7f478015b6b0 msgr2=0x7f47800ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4784ec6640 1 --2- 172.21.15.37:0/874594361 >> v2:172.21.15.203:3300/0 conn(0x7f478015b6b0 0x7f47800ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4784ec6640 1 -- 172.21.15.37:0/874594361 >> v1:172.21.15.37:6789/0 conn(0x7f47800cf590 legacy=0x7f478015f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:24.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.436+0000 7f4784ec6640 1 -- 172.21.15.37:0/874594361 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47800cc2e0 con 0x7f4780073010 2024-04-25T08:53:24.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4784ec6640 1 --2- 172.21.15.37:0/874594361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780073010 0x7f47800cfa20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f47800d0180 tx=0x7f477801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d9aad18b57fccd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:24.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f477f7fe640 1 -- 172.21.15.37:0/874594361 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47780a56c0 con 0x7f4780073010 2024-04-25T08:53:24.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f477f7fe640 1 -- 172.21.15.37:0/874594361 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47780a5860 con 0x7f4780073010 2024-04-25T08:53:24.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f477f7fe640 1 -- 172.21.15.37:0/874594361 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47780a5c30 con 0x7f4780073010 2024-04-25T08:53:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 -- 172.21.15.37:0/874594361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780073010 msgr2=0x7f47800cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 --2- 172.21.15.37:0/874594361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780073010 0x7f47800cfa20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f47800d0180 tx=0x7f477801c9f0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 -- 172.21.15.37:0/874594361 shutdown_connections 2024-04-25T08:53:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 --2- 172.21.15.37:0/874594361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780073010 0x7f47800cfa20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 --2- 172.21.15.37:0/874594361 >> v2:172.21.15.203:3300/0 conn(0x7f478015b6b0 0x7f47800ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 -- 172.21.15.37:0/874594361 >> 172.21.15.37:0/874594361 conn(0x7f478006ae60 msgr2=0x7f478006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 -- 172.21.15.37:0/874594361 shutdown_connections 2024-04-25T08:53:24.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 -- 172.21.15.37:0/874594361 wait complete. 2024-04-25T08:53:24.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 Processor -- start 2024-04-25T08:53:24.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 -- start start 2024-04-25T08:53:24.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780073010 0x7f47800b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47800cf590 0x7f47800ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478015b6b0 0x7f4780164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47800cadb0 con 0x7f4780073010 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47800ca900 con 0x7f478015b6b0 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.440+0000 7f4786929640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47800caf30 con 0x7f47800cf590 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780073010 0x7f47800b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780073010 0x7f47800b3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41538/0 (socket says 172.21.15.37:41538) 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477ffff640 1 -- 172.21.15.37:0/1656410622 learned_addr learned my addr 172.21.15.37:0/1656410622 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f4784ec6640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478015b6b0 0x7f4780164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f4777fff640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47800cf590 0x7f47800ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477ffff640 1 -- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47800cf590 msgr2=0x7f47800ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477ffff640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47800cf590 0x7f47800ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477ffff640 1 -- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478015b6b0 msgr2=0x7f4780164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477ffff640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478015b6b0 0x7f4780164670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477ffff640 1 -- 172.21.15.37:0/1656410622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47800cece0 con 0x7f4780073010 2024-04-25T08:53:24.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f4777fff640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47800cf590 0x7f47800ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f4784ec6640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478015b6b0 0x7f4780164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477ffff640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780073010 0x7f47800b3d30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f477000d5f0 tx=0x7f477000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477d7fa640 1 -- 172.21.15.37:0/1656410622 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47700125f0 con 0x7f4780073010 2024-04-25T08:53:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477d7fa640 1 -- 172.21.15.37:0/1656410622 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4770019070 con 0x7f4780073010 2024-04-25T08:53:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f4786929640 1 -- 172.21.15.37:0/1656410622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47800af940 con 0x7f4780073010 2024-04-25T08:53:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477d7fa640 1 -- 172.21.15.37:0/1656410622 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47700129b0 con 0x7f4780073010 2024-04-25T08:53:24.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f4786929640 1 -- 172.21.15.37:0/1656410622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47800aff00 con 0x7f4780073010 2024-04-25T08:53:24.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f4786929640 1 -- 172.21.15.37:0/1656410622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478004ef80 con 0x7f4780073010 2024-04-25T08:53:24.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477d7fa640 1 -- 172.21.15.37:0/1656410622 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4770012b50 con 0x7f4780073010 2024-04-25T08:53:24.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477d7fa640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4760066f60 0x7f4760069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f477d7fa640 1 -- 172.21.15.37:0/1656410622 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f47700861d0 con 0x7f4780073010 2024-04-25T08:53:24.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.444+0000 7f4777fff640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4760066f60 0x7f4760069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.448+0000 7f4777fff640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4760066f60 0x7f4760069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4768007fd0 tx=0x7f4768009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:24.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.448+0000 7f477d7fa640 1 -- 172.21.15.37:0/1656410622 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f477001c360 con 0x7f4780073010 2024-04-25T08:53:24.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.612+0000 7f4786929640 1 -- 172.21.15.37:0/1656410622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4780165e80 con 0x7f4780073010 2024-04-25T08:53:24.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.616+0000 7f477d7fa640 1 -- 172.21.15.37:0/1656410622 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4770056ec0 con 0x7f4780073010 2024-04-25T08:53:24.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 -- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4760066f60 msgr2=0x7f4760069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4760066f60 0x7f4760069420 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4768007fd0 tx=0x7f4768009040 comp rx=0 tx=0).stop 2024-04-25T08:53:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 -- 172.21.15.37:0/1656410622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780073010 msgr2=0x7f47800b3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780073010 0x7f47800b3d30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f477000d5f0 tx=0x7f477000dab0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 -- 172.21.15.37:0/1656410622 shutdown_connections 2024-04-25T08:53:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4760066f60 0x7f4760069420 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f478015b6b0 0x7f4780164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47800cf590 0x7f47800ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 --2- 172.21.15.37:0/1656410622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4780073010 0x7f47800b3d30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 -- 172.21.15.37:0/1656410622 >> 172.21.15.37:0/1656410622 conn(0x7f478006ae60 msgr2=0x7f4780156470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 -- 172.21.15.37:0/1656410622 shutdown_connections 2024-04-25T08:53:24.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.620+0000 7f4786929640 1 -- 172.21.15.37:0/1656410622 wait complete. 2024-04-25T08:53:24.630 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:53:24.630 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:53:24.685 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:53:24.745 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:53:24.745 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:53:24.745 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:24.751 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:53:24.751 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:24.801 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:24.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.980+0000 7fd08e1fc640 1 Processor -- start 2024-04-25T08:53:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd08e1fc640 1 -- start start 2024-04-25T08:53:24.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd08e1fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd08806ac90 0x7fd08806b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd08e1fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd08815ca90 0x7fd0880d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd08e1fc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd08806b5b0 con 0x7fd0880cc130 2024-04-25T08:53:24.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd08e1fc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd08815ff70 con 0x7fd08815ca90 2024-04-25T08:53:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd08e1fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0880d0300 con 0x7fd08806ac90 2024-04-25T08:53:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd086ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd08815ca90 0x7fd0880d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd086ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd08815ca90 0x7fd0880d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41550/0 (socket says 172.21.15.37:41550) 2024-04-25T08:53:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd086ffd640 1 -- 172.21.15.37:0/528261551 learned_addr learned my addr 172.21.15.37:0/528261551 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd0877fe640 1 --2- 172.21.15.37:0/528261551 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd08806ac90 0x7fd08806b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd086ffd640 1 -- 172.21.15.37:0/528261551 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd08806ac90 msgr2=0x7fd08806b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd086ffd640 1 --2- 172.21.15.37:0/528261551 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd08806ac90 0x7fd08806b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd086ffd640 1 -- 172.21.15.37:0/528261551 >> v1:172.21.15.203:6789/0 conn(0x7fd0880cc130 legacy=0x7fd0880ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.984+0000 7fd086ffd640 1 -- 172.21.15.37:0/528261551 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08815f360 con 0x7fd08815ca90 2024-04-25T08:53:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd086ffd640 1 --2- 172.21.15.37:0/528261551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08815ca90 0x7fd0880d0c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009870 tx=0x7fd07c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ca23bd1eb8c307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0867fc640 1 -- 172.21.15.37:0/528261551 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0aa610 con 0x7fd08815ca90 2024-04-25T08:53:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0867fc640 1 -- 172.21.15.37:0/528261551 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c0af070 con 0x7fd08815ca90 2024-04-25T08:53:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0867fc640 1 -- 172.21.15.37:0/528261551 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0aa9d0 con 0x7fd08815ca90 2024-04-25T08:53:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 -- 172.21.15.37:0/528261551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08815ca90 msgr2=0x7fd0880d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 --2- 172.21.15.37:0/528261551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08815ca90 0x7fd0880d0c10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009870 tx=0x7fd07c01c4f0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 -- 172.21.15.37:0/528261551 shutdown_connections 2024-04-25T08:53:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 --2- 172.21.15.37:0/528261551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08815ca90 0x7fd0880d0c10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 --2- 172.21.15.37:0/528261551 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd08806ac90 0x7fd08806b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 -- 172.21.15.37:0/528261551 >> 172.21.15.37:0/528261551 conn(0x7fd088099a70 msgr2=0x7fd0880d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 -- 172.21.15.37:0/528261551 shutdown_connections 2024-04-25T08:53:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 -- 172.21.15.37:0/528261551 wait complete. 2024-04-25T08:53:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 Processor -- start 2024-04-25T08:53:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 -- start start 2024-04-25T08:53:24.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08806ac90 0x7fd0880b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0880cc130 0x7fd08806e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd08815ca90 0x7fd08814d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd088159750 con 0x7fd08806ac90 2024-04-25T08:53:24.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0881595d0 con 0x7fd08815ca90 2024-04-25T08:53:24.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd088168660 con 0x7fd0880cc130 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08806ac90 0x7fd0880b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08806ac90 0x7fd0880b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41556/0 (socket says 172.21.15.37:41556) 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0877fe640 1 -- 172.21.15.37:0/1912441049 learned_addr learned my addr 172.21.15.37:0/1912441049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd086ffd640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0880cc130 0x7fd08806e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd087fff640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd08815ca90 0x7fd08814d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0877fe640 1 -- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0880cc130 msgr2=0x7fd08806e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0877fe640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0880cc130 0x7fd08806e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0877fe640 1 -- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd08815ca90 msgr2=0x7fd08814d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0877fe640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd08815ca90 0x7fd08814d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0877fe640 1 -- 172.21.15.37:0/1912441049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08806e680 con 0x7fd08806ac90 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd0877fe640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08806ac90 0x7fd0880b28b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd07800d800 tx=0x7fd07800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd06ffff640 1 -- 172.21.15.37:0/1912441049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07800b6c0 con 0x7fd08806ac90 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd06ffff640 1 -- 172.21.15.37:0/1912441049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd078019070 con 0x7fd08806ac90 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 -- 172.21.15.37:0/1912441049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08806e900 con 0x7fd08806ac90 2024-04-25T08:53:24.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd06ffff640 1 -- 172.21.15.37:0/1912441049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07800bae0 con 0x7fd08806ac90 2024-04-25T08:53:24.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.988+0000 7fd08e1fc640 1 -- 172.21.15.37:0/1912441049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08806eec0 con 0x7fd08806ac90 2024-04-25T08:53:24.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.992+0000 7fd08e1fc640 1 -- 172.21.15.37:0/1912441049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08804efa0 con 0x7fd08806ac90 2024-04-25T08:53:24.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.992+0000 7fd06ffff640 1 -- 172.21.15.37:0/1912441049 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd07800bc80 con 0x7fd08806ac90 2024-04-25T08:53:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.992+0000 7fd06ffff640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd074066f60 0x7fd074069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.992+0000 7fd06ffff640 1 -- 172.21.15.37:0/1912441049 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd0780862a0 con 0x7fd08806ac90 2024-04-25T08:53:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.992+0000 7fd086ffd640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd074066f60 0x7fd074069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:24.986 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.992+0000 7fd086ffd640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd074066f60 0x7fd074069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0ae000 tx=0x7fd07c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:24.988 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:24.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:24.996+0000 7fd06ffff640 1 -- 172.21.15.37:0/1912441049 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd08806e680 con 0x7fd08806ac90 2024-04-25T08:53:25.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.156+0000 7fd08e1fc640 1 -- 172.21.15.37:0/1912441049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd08806b7c0 con 0x7fd08806ac90 2024-04-25T08:53:25.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.160+0000 7fd06ffff640 1 -- 172.21.15.37:0/1912441049 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7fd078057020 con 0x7fd08806ac90 2024-04-25T08:53:25.154 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:25.154 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:25.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.160+0000 7fd08e1fc640 1 -- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd074066f60 msgr2=0x7fd074069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.160+0000 7fd08e1fc640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd074066f60 0x7fd074069420 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0ae000 tx=0x7fd07c0a7000 comp rx=0 tx=0).stop 2024-04-25T08:53:25.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.164+0000 7fd08e1fc640 1 -- 172.21.15.37:0/1912441049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08806ac90 msgr2=0x7fd0880b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.164+0000 7fd08e1fc640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08806ac90 0x7fd0880b28b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd07800d800 tx=0x7fd07800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.164+0000 7fd08e1fc640 1 -- 172.21.15.37:0/1912441049 shutdown_connections 2024-04-25T08:53:25.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.164+0000 7fd08e1fc640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd074066f60 0x7fd074069420 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.164+0000 7fd08e1fc640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd08815ca90 0x7fd08814d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.164+0000 7fd08e1fc640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0880cc130 0x7fd08806e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.164+0000 7fd08e1fc640 1 --2- 172.21.15.37:0/1912441049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08806ac90 0x7fd0880b28b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.164+0000 7fd08e1fc640 1 -- 172.21.15.37:0/1912441049 >> 172.21.15.37:0/1912441049 conn(0x7fd088099a70 msgr2=0x7fd08815be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:25.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.164+0000 7fd08e1fc640 1 -- 172.21.15.37:0/1912441049 shutdown_connections 2024-04-25T08:53:25.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.164+0000 7fd08e1fc640 1 -- 172.21.15.37:0/1912441049 wait complete. 2024-04-25T08:53:25.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:25.173 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:25.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053f255640 1 Processor -- start 2024-04-25T08:53:25.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053f255640 1 -- start start 2024-04-25T08:53:25.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053f255640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f05380cf590 0x7f05380cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053f255640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05380cbd20 con 0x7f053815b6b0 2024-04-25T08:53:25.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053f255640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05380cc000 con 0x7f0538072000 2024-04-25T08:53:25.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053f255640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f053815ba90 con 0x7f05380cf590 2024-04-25T08:53:25.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053cff1640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f053815b6b0 0x7f05380ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41816/0 (socket says 172.21.15.37:41816) 2024-04-25T08:53:25.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053cff1640 1 -- 172.21.15.37:0/2641140831 learned_addr learned my addr 172.21.15.37:0/2641140831 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:25.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259735966 0 0) 0x7f05380cbd20 con 0x7f053815b6b0 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f051c003530 con 0x7f053815b6b0 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f0537fff640 1 --2- 172.21.15.37:0/2641140831 >> v2:172.21.15.203:3300/0 conn(0x7f05380cf590 0x7f05380cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 199675769 0 0) 0x7f051c003530 con 0x7f053815b6b0 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 >> v2:172.21.15.203:3300/0 conn(0x7f05380cf590 msgr2=0x7f05380cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 --2- 172.21.15.37:0/2641140831 >> v2:172.21.15.203:3300/0 conn(0x7f05380cf590 0x7f05380cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 >> v1:172.21.15.203:6789/0 conn(0x7f0538072000 legacy=0x7f0538072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05380cc2e0 con 0x7f053815b6b0 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0528001680 con 0x7f053815b6b0 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 >> v1:172.21.15.37:6789/0 conn(0x7f053815b6b0 legacy=0x7f05380ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f051c003a80 0x7f051c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f051c004810 0x7f051c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f051c005220 0x7f051c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053815ba90 con 0x7f051c005220 2024-04-25T08:53:25.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051c00c150 con 0x7f051c004810 2024-04-25T08:53:25.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f051c00c350 con 0x7f051c003a80 2024-04-25T08:53:25.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053d7f2640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f051c005220 0x7f051c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f0537fff640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f051c004810 0x7f051c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053cff1640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f051c003a80 0x7f051c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053d7f2640 1 -- 172.21.15.37:0/2641140831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f051c003a80 msgr2=0x7f051c003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:25.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053d7f2640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f051c003a80 0x7f051c003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053d7f2640 1 -- 172.21.15.37:0/2641140831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f051c004810 msgr2=0x7f051c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:25.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053d7f2640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f051c004810 0x7f051c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f053d7f2640 1 -- 172.21.15.37:0/2641140831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c003530 con 0x7f051c005220 2024-04-25T08:53:25.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.304+0000 7f0537fff640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f051c004810 0x7f051c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:25.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f053d7f2640 1 -- 172.21.15.37:0/2641140831 reap_dead start 2024-04-25T08:53:25.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f053d7f2640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f051c005220 0x7f051c00b9d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0530001e70 tx=0x7f053001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f9ff93400eccd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:25.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0530001a90 con 0x7f051c005220 2024-04-25T08:53:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0530003bc0 con 0x7f051c005220 2024-04-25T08:53:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f05377fe640 1 -- 172.21.15.37:0/2641140831 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0530005020 con 0x7f051c005220 2024-04-25T08:53:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f053f255640 1 -- 172.21.15.37:0/2641140831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f051c005220 msgr2=0x7f051c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f053f255640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f051c005220 0x7f051c00b9d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0530001e70 tx=0x7f053001c780 comp rx=0 tx=0).stop 2024-04-25T08:53:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f053f255640 1 -- 172.21.15.37:0/2641140831 shutdown_connections 2024-04-25T08:53:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f053f255640 1 --2- 172.21.15.37:0/2641140831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f051c005220 0x7f051c00b9d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f053f255640 1 -- 172.21.15.37:0/2641140831 >> 172.21.15.37:0/2641140831 conn(0x7f053806ae60 msgr2=0x7f053806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f053f255640 1 -- 172.21.15.37:0/2641140831 shutdown_connections 2024-04-25T08:53:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f053f255640 1 -- 172.21.15.37:0/2641140831 wait complete. 2024-04-25T08:53:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f053f255640 1 Processor -- start 2024-04-25T08:53:25.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.308+0000 7f053f255640 1 -- start start 2024-04-25T08:53:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f053f255640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f051c005220 0x7f05380ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f053f255640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381f5200 0x7f0538098880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f053f255640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05381546e0 0x7f0538154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f053f255640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05380cba30 con 0x7f05381f5200 2024-04-25T08:53:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f053f255640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05380cc8a0 con 0x7f05381546e0 2024-04-25T08:53:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f053f255640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05380cadb0 con 0x7f051c005220 2024-04-25T08:53:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f0537fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381f5200 0x7f0538098880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f0537fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381f5200 0x7f0538098880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41578/0 (socket says 172.21.15.37:41578) 2024-04-25T08:53:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f0537fff640 1 -- 172.21.15.37:0/4277225420 learned_addr learned my addr 172.21.15.37:0/4277225420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f053d7f2640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05381546e0 0x7f0538154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f053cff1640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f051c005220 0x7f05380ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f0537fff640 1 -- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f051c005220 msgr2=0x7f05380ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f0537fff640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f051c005220 0x7f05380ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f0537fff640 1 -- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05381546e0 msgr2=0x7f0538154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f0537fff640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05381546e0 0x7f0538154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f0537fff640 1 -- 172.21.15.37:0/4277225420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0538158e90 con 0x7f05381f5200 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f0537fff640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381f5200 0x7f0538098880 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0524009160 tx=0x7f0524011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f05357fa640 1 -- 172.21.15.37:0/4277225420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524004220 con 0x7f05381f5200 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f05357fa640 1 -- 172.21.15.37:0/4277225420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05240043c0 con 0x7f05381f5200 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f05357fa640 1 -- 172.21.15.37:0/4277225420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524015890 con 0x7f05381f5200 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f053f255640 1 -- 172.21.15.37:0/4277225420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05381f5680 con 0x7f05381f5200 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f053f255640 1 -- 172.21.15.37:0/4277225420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05380acb30 con 0x7f05381f5200 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f053f255640 1 -- 172.21.15.37:0/4277225420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053804efa0 con 0x7f05381f5200 2024-04-25T08:53:25.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f05357fa640 1 -- 172.21.15.37:0/4277225420 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0524015a30 con 0x7f05381f5200 2024-04-25T08:53:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f05357fa640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0520066f90 0x7f0520069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.312+0000 7f05357fa640 1 -- 172.21.15.37:0/4277225420 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0524085d00 con 0x7f05381f5200 2024-04-25T08:53:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.316+0000 7f053cff1640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0520066f90 0x7f0520069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.316+0000 7f053cff1640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0520066f90 0x7f0520069450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0528001ec0 tx=0x7f0528000f40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:25.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.316+0000 7f05357fa640 1 -- 172.21.15.37:0/4277225420 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0524019030 con 0x7f05381f5200 2024-04-25T08:53:25.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.476+0000 7f053f255640 1 -- 172.21.15.37:0/4277225420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f05380c5c40 con 0x7f05381f5200 2024-04-25T08:53:25.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f05357fa640 1 -- 172.21.15.37:0/4277225420 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0524056a80 con 0x7f05381f5200 2024-04-25T08:53:25.472 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 -- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0520066f90 msgr2=0x7f0520069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0520066f90 0x7f0520069450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0528001ec0 tx=0x7f0528000f40 comp rx=0 tx=0).stop 2024-04-25T08:53:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 -- 172.21.15.37:0/4277225420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381f5200 msgr2=0x7f0538098880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381f5200 0x7f0538098880 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0524009160 tx=0x7f0524011a60 comp rx=0 tx=0).stop 2024-04-25T08:53:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 -- 172.21.15.37:0/4277225420 shutdown_connections 2024-04-25T08:53:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0520066f90 0x7f0520069450 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f05381546e0 0x7f0538154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381f5200 0x7f0538098880 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 --2- 172.21.15.37:0/4277225420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f051c005220 0x7f05380ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 -- 172.21.15.37:0/4277225420 >> 172.21.15.37:0/4277225420 conn(0x7f053806ae60 msgr2=0x7f05381581e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 -- 172.21.15.37:0/4277225420 shutdown_connections 2024-04-25T08:53:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.480+0000 7f053f255640 1 -- 172.21.15.37:0/4277225420 wait complete. 2024-04-25T08:53:25.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:25.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a4afa5640 1 Processor -- start 2024-04-25T08:53:25.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a4afa5640 1 -- start start 2024-04-25T08:53:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a4afa5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a4406ac90 0x7f5a4406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a4afa5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4406b5b0 con 0x7f5a440cc130 2024-04-25T08:53:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a4afa5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a4415f360 con 0x7f5a4415ca90 2024-04-25T08:53:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a4afa5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4415ff70 con 0x7f5a4406ac90 2024-04-25T08:53:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a49542640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5a440cc130 0x7f5a440cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41828/0 (socket says 172.21.15.37:41828) 2024-04-25T08:53:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a49542640 1 -- 172.21.15.37:0/334775837 learned_addr learned my addr 172.21.15.37:0/334775837 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a48d41640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a4406ac90 0x7f5a4406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98901258 0 0) 0x7f5a4406b5b0 con 0x7f5a440cc130 2024-04-25T08:53:25.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a2c003530 con 0x7f5a440cc130 2024-04-25T08:53:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1762949117 0 0) 0x7f5a2c003530 con 0x7f5a440cc130 2024-04-25T08:53:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a4406ac90 msgr2=0x7f5a4406b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a4406ac90 0x7f5a4406b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 >> v1:172.21.15.203:6789/0 conn(0x7f5a4415ca90 legacy=0x7f5a440d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4415f640 con 0x7f5a440cc130 2024-04-25T08:53:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f5a3c001680 con 0x7f5a440cc130 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 >> v1:172.21.15.37:6789/0 conn(0x7f5a440cc130 legacy=0x7f5a440cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a2c003a60 0x7f5a2c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a4406ac90 0x7f5a2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a2c004d50 0x7f5a2c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c00bab0 con 0x7f5a2c004d50 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c00bc60 con 0x7f5a2c003a60 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c00be60 con 0x7f5a4406ac90 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a49542640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a2c004d50 0x7f5a2c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a43fff640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a4406ac90 0x7f5a2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a48d41640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a2c003a60 0x7f5a2c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a49542640 1 -- 172.21.15.37:0/334775837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a4406ac90 msgr2=0x7f5a2c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a49542640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a4406ac90 0x7f5a2c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a49542640 1 -- 172.21.15.37:0/334775837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a2c003a60 msgr2=0x7f5a2c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a49542640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a2c003a60 0x7f5a2c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.620+0000 7f5a49542640 1 -- 172.21.15.37:0/334775837 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c003530 con 0x7f5a2c004d50 2024-04-25T08:53:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a49542640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a2c004d50 0x7f5a2c0051d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5a44072050 tx=0x7f5a3c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c508a63c6584f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c007d40 con 0x7f5a2c004d50 2024-04-25T08:53:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3c0a9070 con 0x7f5a2c004d50 2024-04-25T08:53:25.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a437fe640 1 -- 172.21.15.37:0/334775837 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c005d00 con 0x7f5a2c004d50 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a4afa5640 1 -- 172.21.15.37:0/334775837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a2c004d50 msgr2=0x7f5a2c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a4afa5640 1 --2- 172.21.15.37:0/334775837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a2c004d50 0x7f5a2c0051d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5a44072050 tx=0x7f5a3c01cc60 comp rx=0 tx=0).stop 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a49542640 1 -- 172.21.15.37:0/334775837 reap_dead start 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a4afa5640 1 -- 172.21.15.37:0/334775837 shutdown_connections 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a4afa5640 1 -- 172.21.15.37:0/334775837 >> 172.21.15.37:0/334775837 conn(0x7f5a44099a70 msgr2=0x7f5a440d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a4afa5640 1 -- 172.21.15.37:0/334775837 shutdown_connections 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a4afa5640 1 -- 172.21.15.37:0/334775837 wait complete. 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a4afa5640 1 Processor -- start 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.624+0000 7f5a4afa5640 1 -- start start 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a4afa5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a440b18d0 0x7f5a440b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a4afa5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a440b3620 0x7f5a440c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a4afa5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a440b1cb0 0x7f5a440ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a4afa5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a440d1320 con 0x7f5a440b3620 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a4afa5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4415fc00 con 0x7f5a440b1cb0 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a4afa5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a440b3ba0 con 0x7f5a440b18d0 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a43fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a440b3620 0x7f5a440c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a43fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a440b3620 0x7f5a440c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41594/0 (socket says 172.21.15.37:41594) 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a43fff640 1 -- 172.21.15.37:0/1080537100 learned_addr learned my addr 172.21.15.37:0/1080537100 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a43fff640 1 -- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a440b18d0 msgr2=0x7f5a440b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a43fff640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a440b18d0 0x7f5a440b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a49542640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a440b1cb0 0x7f5a440ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a43fff640 1 -- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a440b1cb0 msgr2=0x7f5a440ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a43fff640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a440b1cb0 0x7f5a440ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a43fff640 1 -- 172.21.15.37:0/1080537100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a440b3e80 con 0x7f5a440b3620 2024-04-25T08:53:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a49542640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a440b1cb0 0x7f5a440ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a43fff640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a440b3620 0x7f5a440c9f90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800e500 tx=0x7f5a3800e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a417fa640 1 -- 172.21.15.37:0/1080537100 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a380053a0 con 0x7f5a440b3620 2024-04-25T08:53:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a417fa640 1 -- 172.21.15.37:0/1080537100 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a38018070 con 0x7f5a440b3620 2024-04-25T08:53:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a4afa5640 1 -- 172.21.15.37:0/1080537100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a440b4160 con 0x7f5a440b3620 2024-04-25T08:53:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a417fa640 1 -- 172.21.15.37:0/1080537100 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a38003820 con 0x7f5a440b3620 2024-04-25T08:53:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a4afa5640 1 -- 172.21.15.37:0/1080537100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a440b4720 con 0x7f5a440b3620 2024-04-25T08:53:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a4afa5640 1 -- 172.21.15.37:0/1080537100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4404efa0 con 0x7f5a440b3620 2024-04-25T08:53:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a417fa640 1 -- 172.21.15.37:0/1080537100 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a380039c0 con 0x7f5a440b3620 2024-04-25T08:53:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a417fa640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a30066f90 0x7f5a30069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a417fa640 1 -- 172.21.15.37:0/1080537100 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5a380850b0 con 0x7f5a440b3620 2024-04-25T08:53:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a48d41640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a30066f90 0x7f5a30069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.628+0000 7f5a48d41640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a30066f90 0x7f5a30069450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009fd0 tx=0x7f5a34009750 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.632+0000 7f5a417fa640 1 -- 172.21.15.37:0/1080537100 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a38055cd0 con 0x7f5a440b3620 2024-04-25T08:53:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.784+0000 7f5a4afa5640 1 -- 172.21.15.37:0/1080537100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5a4406b7c0 con 0x7f5a440b3620 2024-04-25T08:53:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.784+0000 7f5a417fa640 1 -- 172.21.15.37:0/1080537100 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5a380555d0 con 0x7f5a440b3620 2024-04-25T08:53:25.779 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 -- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a30066f90 msgr2=0x7f5a30069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a30066f90 0x7f5a30069450 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009fd0 tx=0x7f5a34009750 comp rx=0 tx=0).stop 2024-04-25T08:53:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 -- 172.21.15.37:0/1080537100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a440b3620 msgr2=0x7f5a440c9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a440b3620 0x7f5a440c9f90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800e500 tx=0x7f5a3800e9c0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 -- 172.21.15.37:0/1080537100 shutdown_connections 2024-04-25T08:53:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a30066f90 0x7f5a30069450 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a440b1cb0 0x7f5a440ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a440b3620 0x7f5a440c9f90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 --2- 172.21.15.37:0/1080537100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a440b18d0 0x7f5a440b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 -- 172.21.15.37:0/1080537100 >> 172.21.15.37:0/1080537100 conn(0x7f5a44099a70 msgr2=0x7f5a440ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 -- 172.21.15.37:0/1080537100 shutdown_connections 2024-04-25T08:53:25.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.788+0000 7f5a4afa5640 1 -- 172.21.15.37:0/1080537100 wait complete. 2024-04-25T08:53:25.796 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:25.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f114e3bb640 1 Processor -- start 2024-04-25T08:53:25.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f114e3bb640 1 -- start start 2024-04-25T08:53:25.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f114e3bb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f114815da80 0x7f11480d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f114e3bb640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114806be60 con 0x7f11480d1cd0 2024-04-25T08:53:25.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f114e3bb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114815de60 con 0x7f11480d07f0 2024-04-25T08:53:25.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f114e3bb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114806c440 con 0x7f114815da80 2024-04-25T08:53:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f11477fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f11480d1cd0 0x7f11480d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41842/0 (socket says 172.21.15.37:41842) 2024-04-25T08:53:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f11477fe640 1 -- 172.21.15.37:0/2525349065 learned_addr learned my addr 172.21.15.37:0/2525349065 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1147fff640 1 --2- 172.21.15.37:0/2525349065 >> v2:172.21.15.203:3300/0 conn(0x7f114815da80 0x7f11480d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745864342 0 0) 0x7f114806be60 con 0x7f11480d1cd0 2024-04-25T08:53:25.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1134003530 con 0x7f11480d1cd0 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3487809125 0 0) 0x7f1134003530 con 0x7f11480d1cd0 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 >> v2:172.21.15.203:3300/0 conn(0x7f114815da80 msgr2=0x7f11480d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 --2- 172.21.15.37:0/2525349065 >> v2:172.21.15.203:3300/0 conn(0x7f114815da80 0x7f11480d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 >> v1:172.21.15.203:6789/0 conn(0x7f11480d07f0 legacy=0x7f114806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11480d2450 con 0x7f11480d1cd0 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1130001800 con 0x7f11480d1cd0 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 >> v1:172.21.15.37:6789/0 conn(0x7f11480d1cd0 legacy=0x7f11480d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134003a30 0x7f1134003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134004810 0x7f1134004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11340051d0 0x7f113400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114806c440 con 0x7f1134003a30 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113400c100 con 0x7f1134004810 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113400c300 con 0x7f11340051d0 2024-04-25T08:53:25.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1147fff640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134003a30 0x7f1134003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1147fff640 1 -- 172.21.15.37:0/2525349065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11340051d0 msgr2=0x7f113400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f114c958640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11340051d0 0x7f113400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f11477fe640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134004810 0x7f1134004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1147fff640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11340051d0 0x7f113400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1147fff640 1 -- 172.21.15.37:0/2525349065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134004810 msgr2=0x7f1134004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1147fff640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134004810 0x7f1134004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f1147fff640 1 -- 172.21.15.37:0/2525349065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1134003530 con 0x7f1134003a30 2024-04-25T08:53:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f114c958640 1 -- 172.21.15.37:0/2525349065 reap_dead start 2024-04-25T08:53:25.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.924+0000 7f11477fe640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134004810 0x7f1134004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:25.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f1147fff640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134003a30 0x7f1134003e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1138009f70 tx=0x7f113801cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d23360510d3e4a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:25.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11380b2070 con 0x7f1134003a30 2024-04-25T08:53:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11380a5d50 con 0x7f1134003a30 2024-04-25T08:53:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f1146ffd640 1 -- 172.21.15.37:0/2525349065 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1138009bf0 con 0x7f1134003a30 2024-04-25T08:53:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f114e3bb640 1 -- 172.21.15.37:0/2525349065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134003a30 msgr2=0x7f1134003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f114e3bb640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134003a30 0x7f1134003e70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1138009f70 tx=0x7f113801cbf0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f114e3bb640 1 -- 172.21.15.37:0/2525349065 shutdown_connections 2024-04-25T08:53:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f114e3bb640 1 --2- 172.21.15.37:0/2525349065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134003a30 0x7f1134003e70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f114e3bb640 1 -- 172.21.15.37:0/2525349065 >> 172.21.15.37:0/2525349065 conn(0x7f114808b000 msgr2=0x7f11480d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f114e3bb640 1 -- 172.21.15.37:0/2525349065 shutdown_connections 2024-04-25T08:53:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f114e3bb640 1 -- 172.21.15.37:0/2525349065 wait complete. 2024-04-25T08:53:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f114e3bb640 1 Processor -- start 2024-04-25T08:53:25.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.928+0000 7f114e3bb640 1 -- start start 2024-04-25T08:53:25.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f114e3bb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134003a30 0x7f11480c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f114e3bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11480b6310 0x7f11480b6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f114e3bb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114815fb50 0x7f114815ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f114e3bb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11480d26f0 con 0x7f11480b6310 2024-04-25T08:53:25.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f114e3bb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11480d4640 con 0x7f1134003a30 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f114e3bb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11480d2870 con 0x7f114815fb50 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f11477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11480b6310 0x7f11480b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f11477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11480b6310 0x7f11480b6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41606/0 (socket says 172.21.15.37:41606) 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f11477fe640 1 -- 172.21.15.37:0/1770741477 learned_addr learned my addr 172.21.15.37:0/1770741477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f1147fff640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134003a30 0x7f11480c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f114c958640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114815fb50 0x7f114815ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f11477fe640 1 -- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114815fb50 msgr2=0x7f114815ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f11477fe640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114815fb50 0x7f114815ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f11477fe640 1 -- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134003a30 msgr2=0x7f11480c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f11477fe640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134003a30 0x7f11480c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f11477fe640 1 -- 172.21.15.37:0/1770741477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11480cafc0 con 0x7f11480b6310 2024-04-25T08:53:25.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f1147fff640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134003a30 0x7f11480c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f11477fe640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11480b6310 0x7f11480b6790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f113000d770 tx=0x7f113000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f1144ff9640 1 -- 172.21.15.37:0/1770741477 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130007480 con 0x7f11480b6310 2024-04-25T08:53:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f1144ff9640 1 -- 172.21.15.37:0/1770741477 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1130017070 con 0x7f11480b6310 2024-04-25T08:53:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f114e3bb640 1 -- 172.21.15.37:0/1770741477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1148160960 con 0x7f11480b6310 2024-04-25T08:53:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f1144ff9640 1 -- 172.21.15.37:0/1770741477 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11300059b0 con 0x7f11480b6310 2024-04-25T08:53:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f114e3bb640 1 -- 172.21.15.37:0/1770741477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11480cb560 con 0x7f11480b6310 2024-04-25T08:53:25.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f114e3bb640 1 -- 172.21.15.37:0/1770741477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114804efa0 con 0x7f11480b6310 2024-04-25T08:53:25.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f1144ff9640 1 -- 172.21.15.37:0/1770741477 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1130005b50 con 0x7f11480b6310 2024-04-25T08:53:25.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f1144ff9640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1128066f90 0x7f1128069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:25.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f1144ff9640 1 -- 172.21.15.37:0/1770741477 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1130084fe0 con 0x7f11480b6310 2024-04-25T08:53:25.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.932+0000 7f1147fff640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1128066f90 0x7f1128069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:25.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.936+0000 7f1147fff640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1128066f90 0x7f1128069450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f11380af000 tx=0x7f11380a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:25.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:25.936+0000 7f1144ff9640 1 -- 172.21.15.37:0/1770741477 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f11300102d0 con 0x7f11480b6310 2024-04-25T08:53:26.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.096+0000 7f114e3bb640 1 -- 172.21.15.37:0/1770741477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11480c6c80 con 0x7f11480b6310 2024-04-25T08:53:26.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.100+0000 7f1144ff9640 1 -- 172.21.15.37:0/1770741477 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1130073030 con 0x7f11480b6310 2024-04-25T08:53:26.091 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 -- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1128066f90 msgr2=0x7f1128069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1128066f90 0x7f1128069450 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f11380af000 tx=0x7f11380a7040 comp rx=0 tx=0).stop 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 -- 172.21.15.37:0/1770741477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11480b6310 msgr2=0x7f11480b6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11480b6310 0x7f11480b6790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f113000d770 tx=0x7f113000dc30 comp rx=0 tx=0).stop 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 -- 172.21.15.37:0/1770741477 shutdown_connections 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1128066f90 0x7f1128069450 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114815fb50 0x7f114815ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11480b6310 0x7f11480b6790 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 --2- 172.21.15.37:0/1770741477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134003a30 0x7f11480c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 -- 172.21.15.37:0/1770741477 >> 172.21.15.37:0/1770741477 conn(0x7f114808b000 msgr2=0x7f11480b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 -- 172.21.15.37:0/1770741477 shutdown_connections 2024-04-25T08:53:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.104+0000 7f114e3bb640 1 -- 172.21.15.37:0/1770741477 wait complete. 2024-04-25T08:53:26.112 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f6555b18640 1 Processor -- start 2024-04-25T08:53:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f6555b18640 1 -- start start 2024-04-25T08:53:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f6555b18640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f655006fc50 0x7f655006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f6555b18640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655006b1d0 0x7f65500cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f6555b18640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f65500cfd80 0x7f655015d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f6555b18640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6550161560 con 0x7f655006fc50 2024-04-25T08:53:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f6555b18640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655006b710 con 0x7f65500cfd80 2024-04-25T08:53:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f6555b18640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65501617b0 con 0x7f655006b1d0 2024-04-25T08:53:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f654f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f655006fc50 0x7f655006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f654f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f655006fc50 0x7f655006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41608/0 (socket says 172.21.15.37:41608) 2024-04-25T08:53:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f654f7fe640 1 -- 172.21.15.37:0/4013886866 learned_addr learned my addr 172.21.15.37:0/4013886866 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f654ffff640 1 --2- 172.21.15.37:0/4013886866 >> v2:172.21.15.203:3300/0 conn(0x7f65500cfd80 0x7f655015d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f6547fff640 1 --2- 172.21.15.37:0/4013886866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655006b1d0 0x7f65500cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f654f7fe640 1 -- 172.21.15.37:0/4013886866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655006b1d0 msgr2=0x7f65500cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f654f7fe640 1 --2- 172.21.15.37:0/4013886866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655006b1d0 0x7f65500cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f654f7fe640 1 -- 172.21.15.37:0/4013886866 >> v2:172.21.15.203:3300/0 conn(0x7f65500cfd80 msgr2=0x7f655015d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f654f7fe640 1 --2- 172.21.15.37:0/4013886866 >> v2:172.21.15.203:3300/0 conn(0x7f65500cfd80 0x7f655015d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.236+0000 7f654f7fe640 1 -- 172.21.15.37:0/4013886866 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6550161a90 con 0x7f655006fc50 2024-04-25T08:53:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f654f7fe640 1 --2- 172.21.15.37:0/4013886866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f655006fc50 0x7f655006ac90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f65500b6430 tx=0x7f654001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b31ed7da8f6e071 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f654effd640 1 -- 172.21.15.37:0/4013886866 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65400aa430 con 0x7f655006fc50 2024-04-25T08:53:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f654effd640 1 -- 172.21.15.37:0/4013886866 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65400af070 con 0x7f655006fc50 2024-04-25T08:53:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f654effd640 1 -- 172.21.15.37:0/4013886866 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65400aa7f0 con 0x7f655006fc50 2024-04-25T08:53:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 -- 172.21.15.37:0/4013886866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f655006fc50 msgr2=0x7f655006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 --2- 172.21.15.37:0/4013886866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f655006fc50 0x7f655006ac90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f65500b6430 tx=0x7f654001c4f0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 -- 172.21.15.37:0/4013886866 shutdown_connections 2024-04-25T08:53:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 --2- 172.21.15.37:0/4013886866 >> v2:172.21.15.203:3300/0 conn(0x7f65500cfd80 0x7f655015d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 --2- 172.21.15.37:0/4013886866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655006b1d0 0x7f65500cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 --2- 172.21.15.37:0/4013886866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f655006fc50 0x7f655006ac90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 -- 172.21.15.37:0/4013886866 >> 172.21.15.37:0/4013886866 conn(0x7f6550099a70 msgr2=0x7f65500d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 -- 172.21.15.37:0/4013886866 shutdown_connections 2024-04-25T08:53:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 -- 172.21.15.37:0/4013886866 wait complete. 2024-04-25T08:53:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 Processor -- start 2024-04-25T08:53:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 -- start start 2024-04-25T08:53:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f655006b1d0 0x7f65500b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655006fc50 0x7f65500b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65500cfd80 0x7f65500c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65500701b0 con 0x7f65500cfd80 2024-04-25T08:53:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6550070030 con 0x7f655006b1d0 2024-04-25T08:53:26.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f6555b18640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6550162290 con 0x7f655006fc50 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f654ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65500cfd80 0x7f65500c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.240+0000 7f654ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65500cfd80 0x7f65500c97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41622/0 (socket says 172.21.15.37:41622) 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654ffff640 1 -- 172.21.15.37:0/1322680011 learned_addr learned my addr 172.21.15.37:0/1322680011 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654f7fe640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f655006b1d0 0x7f65500b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654ffff640 1 -- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655006fc50 msgr2=0x7f65500b5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654ffff640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655006fc50 0x7f65500b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654ffff640 1 -- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f655006b1d0 msgr2=0x7f65500b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654ffff640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f655006b1d0 0x7f65500b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654ffff640 1 -- 172.21.15.37:0/1322680011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65500b3950 con 0x7f65500cfd80 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654f7fe640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f655006b1d0 0x7f65500b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654ffff640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65500cfd80 0x7f65500c97a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6548007e90 tx=0x7f654800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:26.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654cff9640 1 -- 172.21.15.37:0/1322680011 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548014420 con 0x7f65500cfd80 2024-04-25T08:53:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654cff9640 1 -- 172.21.15.37:0/1322680011 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6548019070 con 0x7f65500cfd80 2024-04-25T08:53:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f6555b18640 1 -- 172.21.15.37:0/1322680011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65500b3c30 con 0x7f65500cfd80 2024-04-25T08:53:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654cff9640 1 -- 172.21.15.37:0/1322680011 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548014800 con 0x7f65500cfd80 2024-04-25T08:53:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f6555b18640 1 -- 172.21.15.37:0/1322680011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65500b41f0 con 0x7f65500cfd80 2024-04-25T08:53:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f6555b18640 1 -- 172.21.15.37:0/1322680011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f655004efa0 con 0x7f65500cfd80 2024-04-25T08:53:26.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654cff9640 1 -- 172.21.15.37:0/1322680011 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65480149a0 con 0x7f65500cfd80 2024-04-25T08:53:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654cff9640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6530066f90 0x7f6530069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654cff9640 1 -- 172.21.15.37:0/1322680011 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6548085c70 con 0x7f65500cfd80 2024-04-25T08:53:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654f7fe640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6530066f90 0x7f6530069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.244+0000 7f654f7fe640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6530066f90 0x7f6530069450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f65400ae000 tx=0x7f6540008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:26.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.248+0000 7f654cff9640 1 -- 172.21.15.37:0/1322680011 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6548056800 con 0x7f65500cfd80 2024-04-25T08:53:26.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.408+0000 7f6555b18640 1 -- 172.21.15.37:0/1322680011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6550155160 con 0x7f65500cfd80 2024-04-25T08:53:26.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.408+0000 7f654cff9640 1 -- 172.21.15.37:0/1322680011 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6548056100 con 0x7f65500cfd80 2024-04-25T08:53:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:26.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 -- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6530066f90 msgr2=0x7f6530069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6530066f90 0x7f6530069450 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f65400ae000 tx=0x7f6540008040 comp rx=0 tx=0).stop 2024-04-25T08:53:26.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 -- 172.21.15.37:0/1322680011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65500cfd80 msgr2=0x7f65500c97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65500cfd80 0x7f65500c97a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6548007e90 tx=0x7f654800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 -- 172.21.15.37:0/1322680011 shutdown_connections 2024-04-25T08:53:26.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6530066f90 0x7f6530069450 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65500cfd80 0x7f65500c97a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f655006fc50 0x7f65500b5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 --2- 172.21.15.37:0/1322680011 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f655006b1d0 0x7f65500b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 -- 172.21.15.37:0/1322680011 >> 172.21.15.37:0/1322680011 conn(0x7f6550099a70 msgr2=0x7f6550155c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:26.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 -- 172.21.15.37:0/1322680011 shutdown_connections 2024-04-25T08:53:26.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.412+0000 7f6555b18640 1 -- 172.21.15.37:0/1322680011 wait complete. 2024-04-25T08:53:26.422 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:53:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.544+0000 7f3f53d2e640 1 Processor -- start 2024-04-25T08:53:26.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.544+0000 7f3f53d2e640 1 -- start start 2024-04-25T08:53:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.544+0000 7f3f53d2e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 0x7f3f4c15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.544+0000 7f3f53d2e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f4c0718c0 0x7f3f4c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.544+0000 7f3f53d2e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4c15ae30 con 0x7f3f4c159ea0 2024-04-25T08:53:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.544+0000 7f3f53d2e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4c0b2990 con 0x7f3f4c0718c0 2024-04-25T08:53:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.544+0000 7f3f53d2e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c0b3530 con 0x7f3f4c15a280 2024-04-25T08:53:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.544+0000 7f3f512c9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 0x7f3f4c15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.544+0000 7f3f522cb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f4c0718c0 0x7f3f4c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.544+0000 7f3f512c9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 0x7f3f4c15a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:39688/0 (socket says 172.21.15.37:39688) 2024-04-25T08:53:26.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.544+0000 7f3f512c9640 1 -- 172.21.15.37:0/864269099 learned_addr learned my addr 172.21.15.37:0/864269099 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f512c9640 1 -- 172.21.15.37:0/864269099 >> v2:172.21.15.203:3300/0 conn(0x7f3f4c0718c0 msgr2=0x7f3f4c071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f512c9640 1 --2- 172.21.15.37:0/864269099 >> v2:172.21.15.203:3300/0 conn(0x7f3f4c0718c0 0x7f3f4c071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f512c9640 1 -- 172.21.15.37:0/864269099 >> v1:172.21.15.203:6789/0 conn(0x7f3f4c159ea0 legacy=0x7f3f4c0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f512c9640 1 -- 172.21.15.37:0/864269099 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c15b110 con 0x7f3f4c15a280 2024-04-25T08:53:26.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f522cb640 1 --2- 172.21.15.37:0/864269099 >> v2:172.21.15.203:3300/0 conn(0x7f3f4c0718c0 0x7f3f4c071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:26.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f512c9640 1 --2- 172.21.15.37:0/864269099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 0x7f3f4c15a700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009870 tx=0x7f3f3c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fec9ebd8fa2e863 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:26.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f50ac8640 1 -- 172.21.15.37:0/864269099 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0ab5f0 con 0x7f3f4c15a280 2024-04-25T08:53:26.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f50ac8640 1 -- 172.21.15.37:0/864269099 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c0b0070 con 0x7f3f4c15a280 2024-04-25T08:53:26.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f50ac8640 1 -- 172.21.15.37:0/864269099 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0ab9b0 con 0x7f3f4c15a280 2024-04-25T08:53:26.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f53d2e640 1 -- 172.21.15.37:0/864269099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 msgr2=0x7f3f4c15a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f53d2e640 1 --2- 172.21.15.37:0/864269099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 0x7f3f4c15a700 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c009870 tx=0x7f3f3c01c560 comp rx=0 tx=0).stop 2024-04-25T08:53:26.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f53d2e640 1 -- 172.21.15.37:0/864269099 shutdown_connections 2024-04-25T08:53:26.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f53d2e640 1 --2- 172.21.15.37:0/864269099 >> v2:172.21.15.203:3300/0 conn(0x7f3f4c0718c0 0x7f3f4c071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f53d2e640 1 --2- 172.21.15.37:0/864269099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 0x7f3f4c15a700 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f53d2e640 1 -- 172.21.15.37:0/864269099 >> 172.21.15.37:0/864269099 conn(0x7f3f4c0b11a0 msgr2=0x7f3f4c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:26.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f53d2e640 1 -- 172.21.15.37:0/864269099 shutdown_connections 2024-04-25T08:53:26.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.548+0000 7f3f53d2e640 1 -- 172.21.15.37:0/864269099 wait complete. 2024-04-25T08:53:26.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f53d2e640 1 Processor -- start 2024-04-25T08:53:26.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f53d2e640 1 -- start start 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f53d2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c0718c0 0x7f3f4c14ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f53d2e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c159ea0 0x7f3f4c153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f53d2e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 0x7f3f4c14ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f53d2e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c06d420 con 0x7f3f4c159ea0 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f53d2e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c06d2a0 con 0x7f3f4c0718c0 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f53d2e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4c06d5a0 con 0x7f3f4c15a280 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f512c9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c159ea0 0x7f3f4c153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f512c9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c159ea0 0x7f3f4c153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41626/0 (socket says 172.21.15.37:41626) 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f512c9640 1 -- 172.21.15.37:0/3416857790 learned_addr learned my addr 172.21.15.37:0/3416857790 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f522cb640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 0x7f3f4c14ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f51aca640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c0718c0 0x7f3f4c14ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f512c9640 1 -- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 msgr2=0x7f3f4c14ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f512c9640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 0x7f3f4c14ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f512c9640 1 -- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c0718c0 msgr2=0x7f3f4c14ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f512c9640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c0718c0 0x7f3f4c14ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f512c9640 1 -- 172.21.15.37:0/3416857790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c14bf50 con 0x7f3f4c159ea0 2024-04-25T08:53:26.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f51aca640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c0718c0 0x7f3f4c14ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:26.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f512c9640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c159ea0 0x7f3f4c153570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0ad040 tx=0x7f3f3c019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:26.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f427fc640 1 -- 172.21.15.37:0/3416857790 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0c0480 con 0x7f3f4c159ea0 2024-04-25T08:53:26.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f427fc640 1 -- 172.21.15.37:0/3416857790 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c0c0640 con 0x7f3f4c159ea0 2024-04-25T08:53:26.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f53d2e640 1 -- 172.21.15.37:0/3416857790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4c153f50 con 0x7f3f4c159ea0 2024-04-25T08:53:26.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f427fc640 1 -- 172.21.15.37:0/3416857790 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0c0a30 con 0x7f3f4c159ea0 2024-04-25T08:53:26.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f53d2e640 1 -- 172.21.15.37:0/3416857790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4c154510 con 0x7f3f4c159ea0 2024-04-25T08:53:26.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.552+0000 7f3f53d2e640 1 -- 172.21.15.37:0/3416857790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4c04efa0 con 0x7f3f4c159ea0 2024-04-25T08:53:26.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.556+0000 7f3f427fc640 1 -- 172.21.15.37:0/3416857790 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f3c0c0bd0 con 0x7f3f4c159ea0 2024-04-25T08:53:26.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.556+0000 7f3f427fc640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f30066f60 0x7f3f30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.556+0000 7f3f427fc640 1 -- 172.21.15.37:0/3416857790 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3f3c11fe40 con 0x7f3f4c159ea0 2024-04-25T08:53:26.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.556+0000 7f3f51aca640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f30066f60 0x7f3f30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.556+0000 7f3f51aca640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f30066f60 0x7f3f30069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f480016f0 tx=0x7f3f48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:26.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.560+0000 7f3f427fc640 1 -- 172.21.15.37:0/3416857790 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f3c016150 con 0x7f3f4c159ea0 2024-04-25T08:53:26.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 -- 172.21.15.37:0/3416857790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3f4c06e6b0 con 0x7f3f4c159ea0 2024-04-25T08:53:26.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f427fc640 1 -- 172.21.15.37:0/3416857790 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3f3c0f0520 con 0x7f3f4c159ea0 2024-04-25T08:53:26.720 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:53:26.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 -- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f30066f60 msgr2=0x7f3f30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f30066f60 0x7f3f30069420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3f480016f0 tx=0x7f3f48009040 comp rx=0 tx=0).stop 2024-04-25T08:53:26.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 -- 172.21.15.37:0/3416857790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c159ea0 msgr2=0x7f3f4c153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c159ea0 0x7f3f4c153570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0ad040 tx=0x7f3f3c019300 comp rx=0 tx=0).stop 2024-04-25T08:53:26.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 -- 172.21.15.37:0/3416857790 shutdown_connections 2024-04-25T08:53:26.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f30066f60 0x7f3f30069420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f4c15a280 0x7f3f4c14ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4c159ea0 0x7f3f4c153570 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 --2- 172.21.15.37:0/3416857790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4c0718c0 0x7f3f4c14ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 -- 172.21.15.37:0/3416857790 >> 172.21.15.37:0/3416857790 conn(0x7f3f4c0b11a0 msgr2=0x7f3f4c0b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:26.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 -- 172.21.15.37:0/3416857790 shutdown_connections 2024-04-25T08:53:26.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.728+0000 7f3f53d2e640 1 -- 172.21.15.37:0/3416857790 wait complete. 2024-04-25T08:53:26.737 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:53:26.738 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:26.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:26.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f1150c86640 1 Processor -- start 2024-04-25T08:53:26.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f1150c86640 1 -- start start 2024-04-25T08:53:26.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f1150c86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c144ca0 0x7f114c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f1150c86640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f114c06ec60 0x7f114c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f1150c86640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114c0ccc70 con 0x7f114c0ce510 2024-04-25T08:53:26.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f1150c86640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f114c0cc2b0 con 0x7f114c06ec60 2024-04-25T08:53:26.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f1150c86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c06e290 con 0x7f114c144ca0 2024-04-25T08:53:26.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114a59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f114c0ce510 0x7f114c0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41848/0 (socket says 172.21.15.37:41848) 2024-04-25T08:53:26.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114a59c640 1 -- 172.21.15.37:0/1635377728 learned_addr learned my addr 172.21.15.37:0/1635377728 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114ad9d640 1 --2- 172.21.15.37:0/1635377728 >> v2:172.21.15.203:3300/0 conn(0x7f114c06ec60 0x7f114c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f1149d9b640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c144ca0 0x7f114c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919180385 0 0) 0x7f114c0ccc70 con 0x7f114c0ce510 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1134003530 con 0x7f114c0ce510 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1425121916 0 0) 0x7f1134003530 con 0x7f114c0ce510 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c144ca0 msgr2=0x7f114c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c144ca0 0x7f114c06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 >> v2:172.21.15.203:3300/0 conn(0x7f114c06ec60 msgr2=0x7f114c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 --2- 172.21.15.37:0/1635377728 >> v2:172.21.15.203:3300/0 conn(0x7f114c06ec60 0x7f114c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c06dc40 con 0x7f114c0ce510 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1140001680 con 0x7f114c0ce510 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 >> v1:172.21.15.37:6789/0 conn(0x7f114c0ce510 legacy=0x7f114c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134003a30 0x7f1134003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c144ca0 0x7f1134004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 0x7f11340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0cc2b0 con 0x7f1134004d50 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c06e290 con 0x7f1134003a30 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113400ba50 con 0x7f114c144ca0 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114ad9d640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 0x7f11340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f1149d9b640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c144ca0 0x7f1134004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114a59c640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134003a30 0x7f1134003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114ad9d640 1 -- 172.21.15.37:0/1635377728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c144ca0 msgr2=0x7f1134004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114ad9d640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c144ca0 0x7f1134004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114ad9d640 1 -- 172.21.15.37:0/1635377728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134003a30 msgr2=0x7f1134003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114ad9d640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1134003a30 0x7f1134003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.920+0000 7f114ad9d640 1 -- 172.21.15.37:0/1635377728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1134003530 con 0x7f1134004d50 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f114ad9d640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 0x7f11340051d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f114c06f0c0 tx=0x7f114401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893788fd9529aa12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11440b2070 con 0x7f1134004d50 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11440a5c70 con 0x7f1134004d50 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f114959a640 1 -- 172.21.15.37:0/1635377728 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144009c10 con 0x7f1134004d50 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f1150c86640 1 -- 172.21.15.37:0/1635377728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 msgr2=0x7f11340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f1150c86640 1 --2- 172.21.15.37:0/1635377728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 0x7f11340051d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f114c06f0c0 tx=0x7f114401cc60 comp rx=0 tx=0).stop 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f114ad9d640 1 -- 172.21.15.37:0/1635377728 reap_dead start 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f1150c86640 1 -- 172.21.15.37:0/1635377728 shutdown_connections 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f1150c86640 1 -- 172.21.15.37:0/1635377728 >> 172.21.15.37:0/1635377728 conn(0x7f114c0b11a0 msgr2=0x7f114c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f1150c86640 1 -- 172.21.15.37:0/1635377728 shutdown_connections 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f1150c86640 1 -- 172.21.15.37:0/1635377728 wait complete. 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f1150c86640 1 Processor -- start 2024-04-25T08:53:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.924+0000 7f1150c86640 1 -- start start 2024-04-25T08:53:26.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f1150c86640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 0x7f114c15b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f1150c86640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c0c9030 0x7f114c0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f1150c86640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f114c0c4df0 0x7f114c0c5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f1150c86640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c1451f0 con 0x7f1134004d50 2024-04-25T08:53:26.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f1150c86640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c0cc900 con 0x7f114c0c4df0 2024-04-25T08:53:26.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f1150c86640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f114c145370 con 0x7f114c0c9030 2024-04-25T08:53:26.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f114a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 0x7f114c15b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f114a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 0x7f114c15b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41650/0 (socket says 172.21.15.37:41650) 2024-04-25T08:53:26.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f114a59c640 1 -- 172.21.15.37:0/3312256194 learned_addr learned my addr 172.21.15.37:0/3312256194 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:26.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f114ad9d640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f114c0c4df0 0x7f114c0c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f1149d9b640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c0c9030 0x7f114c0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f114a59c640 1 -- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c0c9030 msgr2=0x7f114c0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f114a59c640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c0c9030 0x7f114c0c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f114a59c640 1 -- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f114c0c4df0 msgr2=0x7f114c0c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f114a59c640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f114c0c4df0 0x7f114c0c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f114a59c640 1 -- 172.21.15.37:0/3312256194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114c0c99d0 con 0x7f1134004d50 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f114ad9d640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f114c0c4df0 0x7f114c0c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f114a59c640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 0x7f114c15b070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f114000e800 tx=0x7f114000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f113affd640 1 -- 172.21.15.37:0/3312256194 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114000c6c0 con 0x7f1134004d50 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f113affd640 1 -- 172.21.15.37:0/3312256194 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1140018070 con 0x7f1134004d50 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f113affd640 1 -- 172.21.15.37:0/3312256194 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114000caa0 con 0x7f1134004d50 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f1150c86640 1 -- 172.21.15.37:0/3312256194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c0c5b50 con 0x7f1134004d50 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f1150c86640 1 -- 172.21.15.37:0/3312256194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c0c6b50 con 0x7f1134004d50 2024-04-25T08:53:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f1150c86640 1 -- 172.21.15.37:0/3312256194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c04efa0 con 0x7f1134004d50 2024-04-25T08:53:26.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f113affd640 1 -- 172.21.15.37:0/3312256194 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f114000cc40 con 0x7f1134004d50 2024-04-25T08:53:26.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f113affd640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f112c066f90 0x7f112c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:26.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f113affd640 1 -- 172.21.15.37:0/3312256194 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1140085200 con 0x7f1134004d50 2024-04-25T08:53:26.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.928+0000 7f1149d9b640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f112c066f90 0x7f112c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:26.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.932+0000 7f1149d9b640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f112c066f90 0x7f112c069450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f114c0ae7a0 tx=0x7f113c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:26.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:26.932+0000 7f113affd640 1 -- 172.21.15.37:0/3312256194 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f114c0c99d0 con 0x7f1134004d50 2024-04-25T08:53:27.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.096+0000 7f1150c86640 1 -- 172.21.15.37:0/3312256194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f114c0b1cc0 con 0x7f1134004d50 2024-04-25T08:53:27.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.096+0000 7f113affd640 1 -- 172.21.15.37:0/3312256194 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f1140055ea0 con 0x7f1134004d50 2024-04-25T08:53:27.090 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:27.090 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:27.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 -- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f112c066f90 msgr2=0x7f112c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f112c066f90 0x7f112c069450 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f114c0ae7a0 tx=0x7f113c008040 comp rx=0 tx=0).stop 2024-04-25T08:53:27.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 -- 172.21.15.37:0/3312256194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 msgr2=0x7f114c15b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 0x7f114c15b070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f114000e800 tx=0x7f114000ecc0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 -- 172.21.15.37:0/3312256194 shutdown_connections 2024-04-25T08:53:27.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f112c066f90 0x7f112c069450 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f114c0c4df0 0x7f114c0c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f114c0c9030 0x7f114c0c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 --2- 172.21.15.37:0/3312256194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1134004d50 0x7f114c15b070 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 -- 172.21.15.37:0/3312256194 >> 172.21.15.37:0/3312256194 conn(0x7f114c0b11a0 msgr2=0x7f114c0a87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:27.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 -- 172.21.15.37:0/3312256194 shutdown_connections 2024-04-25T08:53:27.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.100+0000 7f1150c86640 1 -- 172.21.15.37:0/3312256194 wait complete. 2024-04-25T08:53:27.110 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:27.111 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f9764959640 1 Processor -- start 2024-04-25T08:53:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f9764959640 1 -- start start 2024-04-25T08:53:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f9764959640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f976015da80 0x7f97600d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f9764959640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976006be60 con 0x7f97600d07f0 2024-04-25T08:53:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f9764959640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976015de60 con 0x7f97600d1cd0 2024-04-25T08:53:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f9764959640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976006c440 con 0x7f976015da80 2024-04-25T08:53:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975ed9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f97600d07f0 0x7f976006c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41860/0 (socket says 172.21.15.37:41860) 2024-04-25T08:53:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975ed9d640 1 -- 172.21.15.37:0/2607616789 learned_addr learned my addr 172.21.15.37:0/2607616789 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975e59c640 1 --2- 172.21.15.37:0/2607616789 >> v2:172.21.15.203:3300/0 conn(0x7f976015da80 0x7f97600d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502445566 0 0) 0x7f976006be60 con 0x7f97600d07f0 2024-04-25T08:53:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9748003530 con 0x7f97600d07f0 2024-04-25T08:53:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1149330829 0 0) 0x7f9748003530 con 0x7f97600d07f0 2024-04-25T08:53:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 >> v2:172.21.15.203:3300/0 conn(0x7f976015da80 msgr2=0x7f97600d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 --2- 172.21.15.37:0/2607616789 >> v2:172.21.15.203:3300/0 conn(0x7f976015da80 0x7f97600d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 >> v1:172.21.15.203:6789/0 conn(0x7f97600d1cd0 legacy=0x7f97600d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97600d2450 con 0x7f97600d07f0 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f9758001680 con 0x7f97600d07f0 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 >> v1:172.21.15.37:6789/0 conn(0x7f97600d07f0 legacy=0x7f976006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 --2- 172.21.15.37:0/2607616789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9748003a30 0x7f9748003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 --2- 172.21.15.37:0/2607616789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9748004810 0x7f9748004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 --2- 172.21.15.37:0/2607616789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97480051d0 0x7f974800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976006c440 con 0x7f97480051d0 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974800c100 con 0x7f9748003a30 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974800c300 con 0x7f9748004810 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975ed9d640 1 --2- 172.21.15.37:0/2607616789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97480051d0 0x7f974800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975dd9b640 1 --2- 172.21.15.37:0/2607616789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9748004810 0x7f9748004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975e59c640 1 --2- 172.21.15.37:0/2607616789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9748003a30 0x7f9748003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975ed9d640 1 -- 172.21.15.37:0/2607616789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9748004810 msgr2=0x7f9748004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975ed9d640 1 --2- 172.21.15.37:0/2607616789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9748004810 0x7f9748004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975ed9d640 1 -- 172.21.15.37:0/2607616789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9748003a30 msgr2=0x7f9748003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975ed9d640 1 --2- 172.21.15.37:0/2607616789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9748003a30 0x7f9748003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.244+0000 7f975ed9d640 1 -- 172.21.15.37:0/2607616789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9748003530 con 0x7f97480051d0 2024-04-25T08:53:27.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f975ed9d640 1 -- 172.21.15.37:0/2607616789 reap_dead start 2024-04-25T08:53:27.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f975ed9d640 1 --2- 172.21.15.37:0/2607616789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97480051d0 0x7f974800b9d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f975800c210 tx=0x7f975801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602650da667497e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:27.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758007d20 con 0x7f97480051d0 2024-04-25T08:53:27.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97580a9070 con 0x7f97480051d0 2024-04-25T08:53:27.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f975d59a640 1 -- 172.21.15.37:0/2607616789 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97580a5da0 con 0x7f97480051d0 2024-04-25T08:53:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 -- 172.21.15.37:0/2607616789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97480051d0 msgr2=0x7f974800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 --2- 172.21.15.37:0/2607616789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97480051d0 0x7f974800b9d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f975800c210 tx=0x7f975801cc60 comp rx=0 tx=0).stop 2024-04-25T08:53:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 -- 172.21.15.37:0/2607616789 shutdown_connections 2024-04-25T08:53:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 --2- 172.21.15.37:0/2607616789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97480051d0 0x7f974800b9d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 -- 172.21.15.37:0/2607616789 >> 172.21.15.37:0/2607616789 conn(0x7f976008b000 msgr2=0x7f97600d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 -- 172.21.15.37:0/2607616789 shutdown_connections 2024-04-25T08:53:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 -- 172.21.15.37:0/2607616789 wait complete. 2024-04-25T08:53:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 Processor -- start 2024-04-25T08:53:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 -- start start 2024-04-25T08:53:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97480051d0 0x7f9760151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97600ca7c0 0x7f97600b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f976006d140 0x7f976006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97600d22a0 con 0x7f97600ca7c0 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97600d2120 con 0x7f97480051d0 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.248+0000 7f9764959640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97600d26f0 con 0x7f976006d140 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97600ca7c0 0x7f97600b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97600ca7c0 0x7f97600b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41662/0 (socket says 172.21.15.37:41662) 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975dd9b640 1 -- 172.21.15.37:0/3751024742 learned_addr learned my addr 172.21.15.37:0/3751024742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975e59c640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97480051d0 0x7f9760151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975ed9d640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f976006d140 0x7f976006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975dd9b640 1 -- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f976006d140 msgr2=0x7f976006d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975dd9b640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f976006d140 0x7f976006d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975dd9b640 1 -- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97480051d0 msgr2=0x7f9760151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975dd9b640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97480051d0 0x7f9760151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975dd9b640 1 -- 172.21.15.37:0/3751024742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97600b7390 con 0x7f97600ca7c0 2024-04-25T08:53:27.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975ed9d640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f976006d140 0x7f976006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:27.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975e59c640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97480051d0 0x7f9760151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:27.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975dd9b640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97600ca7c0 0x7f97600b6e50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f975400e820 tx=0x7f975400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:27.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f974effd640 1 -- 172.21.15.37:0/3751024742 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754003be0 con 0x7f97600ca7c0 2024-04-25T08:53:27.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f974effd640 1 -- 172.21.15.37:0/3751024742 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9754018070 con 0x7f97600ca7c0 2024-04-25T08:53:27.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f9764959640 1 -- 172.21.15.37:0/3751024742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97600cad80 con 0x7f97600ca7c0 2024-04-25T08:53:27.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f974effd640 1 -- 172.21.15.37:0/3751024742 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97540050d0 con 0x7f97600ca7c0 2024-04-25T08:53:27.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f9764959640 1 -- 172.21.15.37:0/3751024742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97600cb340 con 0x7f97600ca7c0 2024-04-25T08:53:27.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f9764959640 1 -- 172.21.15.37:0/3751024742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976004efa0 con 0x7f97600ca7c0 2024-04-25T08:53:27.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f974effd640 1 -- 172.21.15.37:0/3751024742 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9754005270 con 0x7f97600ca7c0 2024-04-25T08:53:27.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f974effd640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9740066f60 0x7f9740069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f974effd640 1 -- 172.21.15.37:0/3751024742 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f97540853f0 con 0x7f97600ca7c0 2024-04-25T08:53:27.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.252+0000 7f975e59c640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9740066f60 0x7f9740069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.256+0000 7f975e59c640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9740066f60 0x7f9740069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9750009a70 tx=0x7f9750008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:27.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.256+0000 7f974effd640 1 -- 172.21.15.37:0/3751024742 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f975401a310 con 0x7f97600ca7c0 2024-04-25T08:53:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 -- 172.21.15.37:0/3751024742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f97600b0460 con 0x7f97600ca7c0 2024-04-25T08:53:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f974effd640 1 -- 172.21.15.37:0/3751024742 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9754056000 con 0x7f97600ca7c0 2024-04-25T08:53:27.413 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 -- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9740066f60 msgr2=0x7f9740069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9740066f60 0x7f9740069420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9750009a70 tx=0x7f9750008040 comp rx=0 tx=0).stop 2024-04-25T08:53:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 -- 172.21.15.37:0/3751024742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97600ca7c0 msgr2=0x7f97600b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97600ca7c0 0x7f97600b6e50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f975400e820 tx=0x7f975400ece0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 -- 172.21.15.37:0/3751024742 shutdown_connections 2024-04-25T08:53:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9740066f60 0x7f9740069420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f976006d140 0x7f976006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97600ca7c0 0x7f97600b6e50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 --2- 172.21.15.37:0/3751024742 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97480051d0 0x7f9760151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 -- 172.21.15.37:0/3751024742 >> 172.21.15.37:0/3751024742 conn(0x7f976008b000 msgr2=0x7f97600b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 -- 172.21.15.37:0/3751024742 shutdown_connections 2024-04-25T08:53:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.420+0000 7f9764959640 1 -- 172.21.15.37:0/3751024742 wait complete. 2024-04-25T08:53:27.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:27.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.560+0000 7f976ae7b640 1 Processor -- start 2024-04-25T08:53:27.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f976ae7b640 1 -- start start 2024-04-25T08:53:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f976ae7b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9764159fb0 0x7f97640cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f976ae7b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97640cb8c0 0x7f97640cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f976ae7b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97640cc2b0 con 0x7f9764159bd0 2024-04-25T08:53:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f976ae7b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9764072780 con 0x7f97640cb8c0 2024-04-25T08:53:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f976ae7b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9764071b70 con 0x7f9764159fb0 2024-04-25T08:53:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f9769418640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97640cb8c0 0x7f97640cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f9769418640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97640cb8c0 0x7f97640cbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41668/0 (socket says 172.21.15.37:41668) 2024-04-25T08:53:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f9769418640 1 -- 172.21.15.37:0/1065628876 learned_addr learned my addr 172.21.15.37:0/1065628876 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f9763fff640 1 --2- 172.21.15.37:0/1065628876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9764159fb0 0x7f97640cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f9769418640 1 -- 172.21.15.37:0/1065628876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9764159fb0 msgr2=0x7f97640cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f9769418640 1 --2- 172.21.15.37:0/1065628876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9764159fb0 0x7f97640cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f9769418640 1 -- 172.21.15.37:0/1065628876 >> v1:172.21.15.203:6789/0 conn(0x7f9764159bd0 legacy=0x7f97640c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.564+0000 7f9769418640 1 -- 172.21.15.37:0/1065628876 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9764071e50 con 0x7f97640cb8c0 2024-04-25T08:53:27.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f9769418640 1 --2- 172.21.15.37:0/1065628876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97640cb8c0 0x7f97640cbd70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f975c009e80 tx=0x7f975c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2c04b821269404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:27.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f97637fe640 1 -- 172.21.15.37:0/1065628876 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c0a58a0 con 0x7f97640cb8c0 2024-04-25T08:53:27.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f97637fe640 1 -- 172.21.15.37:0/1065628876 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c0a5a40 con 0x7f97640cb8c0 2024-04-25T08:53:27.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f97637fe640 1 -- 172.21.15.37:0/1065628876 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c0a5e10 con 0x7f97640cb8c0 2024-04-25T08:53:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 -- 172.21.15.37:0/1065628876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97640cb8c0 msgr2=0x7f97640cbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 --2- 172.21.15.37:0/1065628876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97640cb8c0 0x7f97640cbd70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f975c009e80 tx=0x7f975c01c9e0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 -- 172.21.15.37:0/1065628876 shutdown_connections 2024-04-25T08:53:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 --2- 172.21.15.37:0/1065628876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97640cb8c0 0x7f97640cbd70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 --2- 172.21.15.37:0/1065628876 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9764159fb0 0x7f97640cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 -- 172.21.15.37:0/1065628876 >> 172.21.15.37:0/1065628876 conn(0x7f976406ae60 msgr2=0x7f976406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:27.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 -- 172.21.15.37:0/1065628876 shutdown_connections 2024-04-25T08:53:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 -- 172.21.15.37:0/1065628876 wait complete. 2024-04-25T08:53:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 Processor -- start 2024-04-25T08:53:27.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 -- start start 2024-04-25T08:53:27.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97640cb8c0 0x7f97640c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764159bd0 0x7f97640c8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9764159fb0 0x7f976415b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97640cf710 con 0x7f9764159bd0 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97640cf590 con 0x7f97640cb8c0 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.568+0000 7f976ae7b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97640cf890 con 0x7f9764159fb0 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9763fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764159bd0 0x7f97640c8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9763fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764159bd0 0x7f97640c8d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41670/0 (socket says 172.21.15.37:41670) 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9763fff640 1 -- 172.21.15.37:0/1458357160 learned_addr learned my addr 172.21.15.37:0/1458357160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9769418640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9764159fb0 0x7f976415b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9768c17640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97640cb8c0 0x7f97640c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9763fff640 1 -- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9764159fb0 msgr2=0x7f976415b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9763fff640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9764159fb0 0x7f976415b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9763fff640 1 -- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97640cb8c0 msgr2=0x7f97640c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9763fff640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97640cb8c0 0x7f97640c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9763fff640 1 -- 172.21.15.37:0/1458357160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97640c6f70 con 0x7f9764159bd0 2024-04-25T08:53:27.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9763fff640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764159bd0 0x7f97640c8d80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f975800d7d0 tx=0x7f975800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:27.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f97617fa640 1 -- 172.21.15.37:0/1458357160 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758004020 con 0x7f9764159bd0 2024-04-25T08:53:27.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f97617fa640 1 -- 172.21.15.37:0/1458357160 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97580041c0 con 0x7f9764159bd0 2024-04-25T08:53:27.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f976ae7b640 1 -- 172.21.15.37:0/1458357160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97640c7250 con 0x7f9764159bd0 2024-04-25T08:53:27.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f97617fa640 1 -- 172.21.15.37:0/1458357160 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97580043b0 con 0x7f9764159bd0 2024-04-25T08:53:27.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f976ae7b640 1 -- 172.21.15.37:0/1458357160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97640c7810 con 0x7f9764159bd0 2024-04-25T08:53:27.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f976ae7b640 1 -- 172.21.15.37:0/1458357160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976404efa0 con 0x7f9764159bd0 2024-04-25T08:53:27.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f97617fa640 1 -- 172.21.15.37:0/1458357160 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f975801b020 con 0x7f9764159bd0 2024-04-25T08:53:27.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f97617fa640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9750066f60 0x7f9750069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f97617fa640 1 -- 172.21.15.37:0/1458357160 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9758085280 con 0x7f9764159bd0 2024-04-25T08:53:27.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.572+0000 7f9768c17640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9750066f60 0x7f9750069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.576+0000 7f9768c17640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9750066f60 0x7f9750069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9754000f70 tx=0x7f9754009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:27.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.576+0000 7f97617fa640 1 -- 172.21.15.37:0/1458357160 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9758055e80 con 0x7f9764159bd0 2024-04-25T08:53:27.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.736+0000 7f976ae7b640 1 -- 172.21.15.37:0/1458357160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f976406ebb0 con 0x7f9764159bd0 2024-04-25T08:53:27.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.736+0000 7f97617fa640 1 -- 172.21.15.37:0/1458357160 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9758055780 con 0x7f9764159bd0 2024-04-25T08:53:27.731 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 -- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9750066f60 msgr2=0x7f9750069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9750066f60 0x7f9750069420 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9754000f70 tx=0x7f9754009040 comp rx=0 tx=0).stop 2024-04-25T08:53:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 -- 172.21.15.37:0/1458357160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764159bd0 msgr2=0x7f97640c8d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764159bd0 0x7f97640c8d80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f975800d7d0 tx=0x7f975800dc90 comp rx=0 tx=0).stop 2024-04-25T08:53:27.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 -- 172.21.15.37:0/1458357160 shutdown_connections 2024-04-25T08:53:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9750066f60 0x7f9750069420 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9764159fb0 0x7f976415b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9764159bd0 0x7f97640c8d80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 --2- 172.21.15.37:0/1458357160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97640cb8c0 0x7f97640c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 -- 172.21.15.37:0/1458357160 >> 172.21.15.37:0/1458357160 conn(0x7f976406ae60 msgr2=0x7f97640b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 -- 172.21.15.37:0/1458357160 shutdown_connections 2024-04-25T08:53:27.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.740+0000 7f976ae7b640 1 -- 172.21.15.37:0/1458357160 wait complete. 2024-04-25T08:53:27.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc09e9ae640 1 Processor -- start 2024-04-25T08:53:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc09e9ae640 1 -- start start 2024-04-25T08:53:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc09e9ae640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc09815b420 0x7fc0980ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc09e9ae640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980cfca0 0x7fc098072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc09e9ae640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0980cbd20 con 0x7fc09815b800 2024-04-25T08:53:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc09e9ae640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0980cc930 con 0x7fc09815b420 2024-04-25T08:53:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc09e9ae640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980cc000 con 0x7fc0980cfca0 2024-04-25T08:53:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc097fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc09815b420 0x7fc0980ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc097fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc09815b420 0x7fc0980ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41680/0 (socket says 172.21.15.37:41680) 2024-04-25T08:53:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc097fff640 1 -- 172.21.15.37:0/2099226468 learned_addr learned my addr 172.21.15.37:0/2099226468 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc09cf4b640 1 --2- 172.21.15.37:0/2099226468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980cfca0 0x7fc098072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc097fff640 1 -- 172.21.15.37:0/2099226468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980cfca0 msgr2=0x7fc098072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc097fff640 1 --2- 172.21.15.37:0/2099226468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980cfca0 0x7fc098072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc097fff640 1 -- 172.21.15.37:0/2099226468 >> v1:172.21.15.203:6789/0 conn(0x7fc09815b800 legacy=0x7fc0980cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.888+0000 7fc097fff640 1 -- 172.21.15.37:0/2099226468 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980cc2e0 con 0x7fc09815b420 2024-04-25T08:53:27.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc097fff640 1 --2- 172.21.15.37:0/2099226468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc09815b420 0x7fc0980ca3c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc088009870 tx=0x7fc08801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=101ac3de59f7cbfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:27.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc096ffd640 1 -- 172.21.15.37:0/2099226468 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880aa430 con 0x7fc09815b420 2024-04-25T08:53:27.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc096ffd640 1 -- 172.21.15.37:0/2099226468 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0880af070 con 0x7fc09815b420 2024-04-25T08:53:27.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc096ffd640 1 -- 172.21.15.37:0/2099226468 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880aa7f0 con 0x7fc09815b420 2024-04-25T08:53:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 -- 172.21.15.37:0/2099226468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc09815b420 msgr2=0x7fc0980ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 --2- 172.21.15.37:0/2099226468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc09815b420 0x7fc0980ca3c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fc088009870 tx=0x7fc08801c530 comp rx=0 tx=0).stop 2024-04-25T08:53:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 -- 172.21.15.37:0/2099226468 shutdown_connections 2024-04-25T08:53:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 --2- 172.21.15.37:0/2099226468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0980cfca0 0x7fc098072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 --2- 172.21.15.37:0/2099226468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc09815b420 0x7fc0980ca3c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 -- 172.21.15.37:0/2099226468 >> 172.21.15.37:0/2099226468 conn(0x7fc09806ae60 msgr2=0x7fc09806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 -- 172.21.15.37:0/2099226468 shutdown_connections 2024-04-25T08:53:27.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 -- 172.21.15.37:0/2099226468 wait complete. 2024-04-25T08:53:27.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 Processor -- start 2024-04-25T08:53:27.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 -- start start 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980cfca0 0x7fc09815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc09815b420 0x7fc0980c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc09815b800 0x7fc09815bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980caf30 con 0x7fc0980cfca0 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980cadb0 con 0x7fc09815b800 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc09e9ae640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc098072d50 con 0x7fc09815b420 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc097fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980cfca0 0x7fc09815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc097fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980cfca0 0x7fc09815e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41696/0 (socket says 172.21.15.37:41696) 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc097fff640 1 -- 172.21.15.37:0/3058541462 learned_addr learned my addr 172.21.15.37:0/3058541462 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc097fff640 1 -- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc09815b420 msgr2=0x7fc0980c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc097fff640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc09815b420 0x7fc0980c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc097fff640 1 -- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc09815b800 msgr2=0x7fc09815bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc097fff640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc09815b800 0x7fc09815bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.892+0000 7fc097fff640 1 -- 172.21.15.37:0/3058541462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc09815c6d0 con 0x7fc0980cfca0 2024-04-25T08:53:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc097fff640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980cfca0 0x7fc09815e0b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc0880ac040 tx=0x7fc08801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc094ff9640 1 -- 172.21.15.37:0/3058541462 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880bd5f0 con 0x7fc0980cfca0 2024-04-25T08:53:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc094ff9640 1 -- 172.21.15.37:0/3058541462 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0880bd7b0 con 0x7fc0980cfca0 2024-04-25T08:53:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc09e9ae640 1 -- 172.21.15.37:0/3058541462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0980c7690 con 0x7fc0980cfca0 2024-04-25T08:53:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc094ff9640 1 -- 172.21.15.37:0/3058541462 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0880bdba0 con 0x7fc0980cfca0 2024-04-25T08:53:27.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc09e9ae640 1 -- 172.21.15.37:0/3058541462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0980b01b0 con 0x7fc0980cfca0 2024-04-25T08:53:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc09e9ae640 1 -- 172.21.15.37:0/3058541462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09804efa0 con 0x7fc0980cfca0 2024-04-25T08:53:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc094ff9640 1 -- 172.21.15.37:0/3058541462 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0880b33f0 con 0x7fc0980cfca0 2024-04-25T08:53:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc094ff9640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc084066f90 0x7fc084069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc094ff9640 1 -- 172.21.15.37:0/3058541462 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc088120060 con 0x7fc0980cfca0 2024-04-25T08:53:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc0977fe640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc084066f90 0x7fc084069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:27.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.896+0000 7fc0977fe640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc084066f90 0x7fc084069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc08c000f70 tx=0x7fc08c0079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:27.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:27.900+0000 7fc094ff9640 1 -- 172.21.15.37:0/3058541462 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc0880c3060 con 0x7fc0980cfca0 2024-04-25T08:53:28.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.060+0000 7fc09e9ae640 1 -- 172.21.15.37:0/3058541462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc09806d3a0 con 0x7fc0980cfca0 2024-04-25T08:53:28.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.060+0000 7fc094ff9640 1 -- 172.21.15.37:0/3058541462 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc088016150 con 0x7fc0980cfca0 2024-04-25T08:53:28.055 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:28.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 -- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc084066f90 msgr2=0x7fc084069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc084066f90 0x7fc084069450 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc08c000f70 tx=0x7fc08c0079e0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 -- 172.21.15.37:0/3058541462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980cfca0 msgr2=0x7fc09815e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980cfca0 0x7fc09815e0b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc0880ac040 tx=0x7fc08801ad70 comp rx=0 tx=0).stop 2024-04-25T08:53:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 -- 172.21.15.37:0/3058541462 shutdown_connections 2024-04-25T08:53:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc084066f90 0x7fc084069450 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc09815b800 0x7fc09815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc09815b420 0x7fc0980c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 --2- 172.21.15.37:0/3058541462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0980cfca0 0x7fc09815e0b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 -- 172.21.15.37:0/3058541462 >> 172.21.15.37:0/3058541462 conn(0x7fc09806ae60 msgr2=0x7fc0980c9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 -- 172.21.15.37:0/3058541462 shutdown_connections 2024-04-25T08:53:28.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.064+0000 7fc09e9ae640 1 -- 172.21.15.37:0/3058541462 wait complete. 2024-04-25T08:53:28.074 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:28.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.212+0000 7f596ac3f640 1 Processor -- start 2024-04-25T08:53:28.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.212+0000 7f596ac3f640 1 -- start start 2024-04-25T08:53:28.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f596ac3f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f596415d220 0x7f59640c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f596ac3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640cbe90 0x7f596415d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f596ac3f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596415e710 con 0x7f596415de60 2024-04-25T08:53:28.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f596ac3f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5964072a20 con 0x7f596415d220 2024-04-25T08:53:28.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f596ac3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5964070fc0 con 0x7f59640cbe90 2024-04-25T08:53:28.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f59689db640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f596415d220 0x7f59640c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f5963fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640cbe90 0x7f596415d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f59691dc640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f596415de60 0x7f59640b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:44976/0 (socket says 172.21.15.37:44976) 2024-04-25T08:53:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f5963fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640cbe90 0x7f596415d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:39810/0 (socket says 172.21.15.37:39810) 2024-04-25T08:53:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f59691dc640 1 -- 172.21.15.37:0/2624514819 learned_addr learned my addr 172.21.15.37:0/2624514819 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f59689db640 1 -- 172.21.15.37:0/2624514819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640cbe90 msgr2=0x7f596415d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f59689db640 1 --2- 172.21.15.37:0/2624514819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640cbe90 0x7f596415d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f59689db640 1 -- 172.21.15.37:0/2624514819 >> v1:172.21.15.203:6789/0 conn(0x7f596415de60 legacy=0x7f59640b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f59689db640 1 -- 172.21.15.37:0/2624514819 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640c9ef0 con 0x7f596415d220 2024-04-25T08:53:28.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f59689db640 1 --2- 172.21.15.37:0/2624514819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596415d220 0x7f59640c99b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f59540097c0 tx=0x7f595401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d4140d159095750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:28.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f59637fe640 1 -- 172.21.15.37:0/2624514819 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540b0070 con 0x7f596415d220 2024-04-25T08:53:28.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f59637fe640 1 -- 172.21.15.37:0/2624514819 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59540ab600 con 0x7f596415d220 2024-04-25T08:53:28.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.216+0000 7f59637fe640 1 -- 172.21.15.37:0/2624514819 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59540ab9d0 con 0x7f596415d220 2024-04-25T08:53:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 -- 172.21.15.37:0/2624514819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596415d220 msgr2=0x7f59640c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 --2- 172.21.15.37:0/2624514819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596415d220 0x7f59640c99b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f59540097c0 tx=0x7f595401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 -- 172.21.15.37:0/2624514819 shutdown_connections 2024-04-25T08:53:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 --2- 172.21.15.37:0/2624514819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640cbe90 0x7f596415d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 --2- 172.21.15.37:0/2624514819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596415d220 0x7f59640c99b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 -- 172.21.15.37:0/2624514819 >> 172.21.15.37:0/2624514819 conn(0x7f596406ae60 msgr2=0x7f596406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 -- 172.21.15.37:0/2624514819 shutdown_connections 2024-04-25T08:53:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 -- 172.21.15.37:0/2624514819 wait complete. 2024-04-25T08:53:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 Processor -- start 2024-04-25T08:53:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 -- start start 2024-04-25T08:53:28.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640cbe90 0x7f5964153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596415d220 0x7f59640c2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f596415de60 0x7f59640c3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59640ccf30 con 0x7f596415de60 2024-04-25T08:53:28.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596415d600 con 0x7f596415d220 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59640cd0b0 con 0x7f59640cbe90 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59691dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f596415de60 0x7f59640c3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59691dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f596415de60 0x7f59640c3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41712/0 (socket says 172.21.15.37:41712) 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59691dc640 1 -- 172.21.15.37:0/3994493740 learned_addr learned my addr 172.21.15.37:0/3994493740 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f5963fff640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596415d220 0x7f59640c2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59689db640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640cbe90 0x7f5964153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59691dc640 1 -- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640cbe90 msgr2=0x7f5964153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59691dc640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640cbe90 0x7f5964153b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59691dc640 1 -- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596415d220 msgr2=0x7f59640c2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59691dc640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596415d220 0x7f59640c2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59691dc640 1 -- 172.21.15.37:0/3994493740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596415b750 con 0x7f596415de60 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59691dc640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f596415de60 0x7f59640c3250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f595c00cbf0 tx=0x7f595c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59617fa640 1 -- 172.21.15.37:0/3994493740 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c017070 con 0x7f596415de60 2024-04-25T08:53:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59617fa640 1 -- 172.21.15.37:0/3994493740 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c00ad40 con 0x7f596415de60 2024-04-25T08:53:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 -- 172.21.15.37:0/3994493740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59640b30f0 con 0x7f596415de60 2024-04-25T08:53:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f59617fa640 1 -- 172.21.15.37:0/3994493740 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c012600 con 0x7f596415de60 2024-04-25T08:53:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.220+0000 7f596ac3f640 1 -- 172.21.15.37:0/3994493740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5964155c60 con 0x7f596415de60 2024-04-25T08:53:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.224+0000 7f596ac3f640 1 -- 172.21.15.37:0/3994493740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596404efa0 con 0x7f596415de60 2024-04-25T08:53:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.224+0000 7f59617fa640 1 -- 172.21.15.37:0/3994493740 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f595c01a070 con 0x7f596415de60 2024-04-25T08:53:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.224+0000 7f59617fa640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5944066f60 0x7f5944069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.224+0000 7f59617fa640 1 -- 172.21.15.37:0/3994493740 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f595c084fe0 con 0x7f596415de60 2024-04-25T08:53:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.224+0000 7f59689db640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5944066f60 0x7f5944069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.224+0000 7f59689db640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5944066f60 0x7f5944069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59540ad000 tx=0x7f59540a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.228+0000 7f59617fa640 1 -- 172.21.15.37:0/3994493740 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f595c055630 con 0x7f596415de60 2024-04-25T08:53:28.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.388+0000 7f596ac3f640 1 -- 172.21.15.37:0/3994493740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f59640b38f0 con 0x7f596415de60 2024-04-25T08:53:28.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.388+0000 7f59617fa640 1 -- 172.21.15.37:0/3994493740 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f595c0596d0 con 0x7f596415de60 2024-04-25T08:53:28.382 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 -- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5944066f60 msgr2=0x7f5944069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5944066f60 0x7f5944069420 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f59540ad000 tx=0x7f59540a6000 comp rx=0 tx=0).stop 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 -- 172.21.15.37:0/3994493740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f596415de60 msgr2=0x7f59640c3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f596415de60 0x7f59640c3250 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f595c00cbf0 tx=0x7f595c00a430 comp rx=0 tx=0).stop 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 -- 172.21.15.37:0/3994493740 shutdown_connections 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5944066f60 0x7f5944069420 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f596415de60 0x7f59640c3250 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f596415d220 0x7f59640c2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 --2- 172.21.15.37:0/3994493740 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f59640cbe90 0x7f5964153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 -- 172.21.15.37:0/3994493740 >> 172.21.15.37:0/3994493740 conn(0x7f596406ae60 msgr2=0x7f59640ad690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 -- 172.21.15.37:0/3994493740 shutdown_connections 2024-04-25T08:53:28.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.392+0000 7f596ac3f640 1 -- 172.21.15.37:0/3994493740 wait complete. 2024-04-25T08:53:28.402 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:53:28.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b86e13640 1 Processor -- start 2024-04-25T08:53:28.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b86e13640 1 -- start start 2024-04-25T08:53:28.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b86e13640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5b800ce510 0x7f5b800ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b86e13640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b80144ca0 0x7f5b8006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b86e13640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b800cee30 con 0x7f5b8006ec60 2024-04-25T08:53:28.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b86e13640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b800cc2b0 con 0x7f5b800ce510 2024-04-25T08:53:28.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b86e13640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8006e290 con 0x7f5b80144ca0 2024-04-25T08:53:28.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b84baf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5b800ce510 0x7f5b800ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b84baf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5b800ce510 0x7f5b800ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41714/0 (socket says 172.21.15.37:41714) 2024-04-25T08:53:28.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b84baf640 1 -- 172.21.15.37:0/3356671697 learned_addr learned my addr 172.21.15.37:0/3356671697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:28.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b7ffff640 1 --2- 172.21.15.37:0/3356671697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b80144ca0 0x7f5b8006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b84baf640 1 -- 172.21.15.37:0/3356671697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b80144ca0 msgr2=0x7f5b8006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:28.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b84baf640 1 --2- 172.21.15.37:0/3356671697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b80144ca0 0x7f5b8006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b84baf640 1 -- 172.21.15.37:0/3356671697 >> v1:172.21.15.203:6789/0 conn(0x7f5b8006ec60 legacy=0x7f5b80142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:28.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b84baf640 1 -- 172.21.15.37:0/3356671697 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b800ccc70 con 0x7f5b800ce510 2024-04-25T08:53:28.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.528+0000 7f5b7ffff640 1 --2- 172.21.15.37:0/3356671697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b80144ca0 0x7f5b8006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b84baf640 1 --2- 172.21.15.37:0/3356671697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b800ce510 0x7f5b800ce8f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009870 tx=0x7f5b7001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ccaf6436ebd8fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b7f7fe640 1 -- 172.21.15.37:0/3356671697 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b700b5070 con 0x7f5b800ce510 2024-04-25T08:53:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b7f7fe640 1 -- 172.21.15.37:0/3356671697 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b700a89d0 con 0x7f5b800ce510 2024-04-25T08:53:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b7f7fe640 1 -- 172.21.15.37:0/3356671697 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b700a8da0 con 0x7f5b800ce510 2024-04-25T08:53:28.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b86e13640 1 -- 172.21.15.37:0/3356671697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b800ce510 msgr2=0x7f5b800ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b86e13640 1 --2- 172.21.15.37:0/3356671697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b800ce510 0x7f5b800ce8f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009870 tx=0x7f5b7001ed10 comp rx=0 tx=0).stop 2024-04-25T08:53:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b86e13640 1 -- 172.21.15.37:0/3356671697 shutdown_connections 2024-04-25T08:53:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b86e13640 1 --2- 172.21.15.37:0/3356671697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b80144ca0 0x7f5b8006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b86e13640 1 --2- 172.21.15.37:0/3356671697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b800ce510 0x7f5b800ce8f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b86e13640 1 -- 172.21.15.37:0/3356671697 >> 172.21.15.37:0/3356671697 conn(0x7f5b800b11a0 msgr2=0x7f5b800b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b86e13640 1 -- 172.21.15.37:0/3356671697 shutdown_connections 2024-04-25T08:53:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.532+0000 7f5b86e13640 1 -- 172.21.15.37:0/3356671697 wait complete. 2024-04-25T08:53:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b86e13640 1 Processor -- start 2024-04-25T08:53:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b86e13640 1 -- start start 2024-04-25T08:53:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b86e13640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b8006ec60 0x7f5b800cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b86e13640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b800ce510 0x7f5b800c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b86e13640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b80144ca0 0x7f5b8013cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b86e13640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b800cc900 con 0x7f5b80144ca0 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b86e13640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8006df20 con 0x7f5b8006ec60 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b86e13640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b80142d90 con 0x7f5b800ce510 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b853b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b80144ca0 0x7f5b8013cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b853b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b80144ca0 0x7f5b8013cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41716/0 (socket says 172.21.15.37:41716) 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b853b0640 1 -- 172.21.15.37:0/2201060774 learned_addr learned my addr 172.21.15.37:0/2201060774 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b7ffff640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b800ce510 0x7f5b800c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b84baf640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b8006ec60 0x7f5b800cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b853b0640 1 -- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b800ce510 msgr2=0x7f5b800c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b853b0640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b800ce510 0x7f5b800c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b853b0640 1 -- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b8006ec60 msgr2=0x7f5b800cb140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b853b0640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b8006ec60 0x7f5b800cb140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b853b0640 1 -- 172.21.15.37:0/2201060774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b801f23d0 con 0x7f5b80144ca0 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b7ffff640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b800ce510 0x7f5b800c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b84baf640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b8006ec60 0x7f5b800cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:28.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b853b0640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b80144ca0 0x7f5b8013cb30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c780 tx=0x7f5b7800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:28.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b7d7fa640 1 -- 172.21.15.37:0/2201060774 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78016020 con 0x7f5b80144ca0 2024-04-25T08:53:28.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b7d7fa640 1 -- 172.21.15.37:0/2201060774 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7800aa90 con 0x7f5b80144ca0 2024-04-25T08:53:28.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b7d7fa640 1 -- 172.21.15.37:0/2201060774 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800ce30 con 0x7f5b80144ca0 2024-04-25T08:53:28.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b86e13640 1 -- 172.21.15.37:0/2201060774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b800c7a70 con 0x7f5b80144ca0 2024-04-25T08:53:28.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b86e13640 1 -- 172.21.15.37:0/2201060774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b801f2690 con 0x7f5b80144ca0 2024-04-25T08:53:28.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.536+0000 7f5b86e13640 1 -- 172.21.15.37:0/2201060774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8004efa0 con 0x7f5b80144ca0 2024-04-25T08:53:28.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.540+0000 7f5b7d7fa640 1 -- 172.21.15.37:0/2201060774 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b78019020 con 0x7f5b80144ca0 2024-04-25T08:53:28.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.540+0000 7f5b7d7fa640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b6c066f60 0x7f5b6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.540+0000 7f5b7d7fa640 1 -- 172.21.15.37:0/2201060774 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5b78085390 con 0x7f5b80144ca0 2024-04-25T08:53:28.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.540+0000 7f5b84baf640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b6c066f60 0x7f5b6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.540+0000 7f5b84baf640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b6c066f60 0x7f5b6c069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b700b2000 tx=0x7f5b700aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:28.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.544+0000 7f5b7d7fa640 1 -- 172.21.15.37:0/2201060774 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5b78020030 con 0x7f5b80144ca0 2024-04-25T08:53:28.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.716+0000 7f5b86e13640 1 -- 172.21.15.37:0/2201060774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5b800b1cc0 con 0x7f5b80144ca0 2024-04-25T08:53:28.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.716+0000 7f5b7d7fa640 1 -- 172.21.15.37:0/2201060774 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5b78055950 con 0x7f5b80144ca0 2024-04-25T08:53:28.710 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:53:28.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 -- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b6c066f60 msgr2=0x7f5b6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b6c066f60 0x7f5b6c069420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5b700b2000 tx=0x7f5b700aa070 comp rx=0 tx=0).stop 2024-04-25T08:53:28.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 -- 172.21.15.37:0/2201060774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b80144ca0 msgr2=0x7f5b8013cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b80144ca0 0x7f5b8013cb30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f5b7800c780 tx=0x7f5b7800cc40 comp rx=0 tx=0).stop 2024-04-25T08:53:28.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 -- 172.21.15.37:0/2201060774 shutdown_connections 2024-04-25T08:53:28.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b6c066f60 0x7f5b6c069420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b80144ca0 0x7f5b8013cb30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5b800ce510 0x7f5b800c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 --2- 172.21.15.37:0/2201060774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5b8006ec60 0x7f5b800cb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 -- 172.21.15.37:0/2201060774 >> 172.21.15.37:0/2201060774 conn(0x7f5b800b11a0 msgr2=0x7f5b80143dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:28.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 -- 172.21.15.37:0/2201060774 shutdown_connections 2024-04-25T08:53:28.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.720+0000 7f5b86e13640 1 -- 172.21.15.37:0/2201060774 wait complete. 2024-04-25T08:53:28.728 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T08:53:28.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:28.728 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:53:28.739 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:28.739 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:53:28.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:28.739 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:53:28.788 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:53:28.788 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:28.796 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:53:28.796 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:28.845 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:28.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c7921a640 1 Processor -- start 2024-04-25T08:53:28.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c7921a640 1 -- start start 2024-04-25T08:53:28.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c7921a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5c74144ca0 0x7f5c7406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c7921a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c7406ec60 0x7f5c74142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c7921a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c740ccc70 con 0x7f5c740ce510 2024-04-25T08:53:28.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c7921a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c740cc2b0 con 0x7f5c74144ca0 2024-04-25T08:53:28.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c7921a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7406e290 con 0x7f5c7406ec60 2024-04-25T08:53:28.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c6bfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5c74144ca0 0x7f5c7406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c7359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c7406ec60 0x7f5c74142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c6bfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5c74144ca0 0x7f5c7406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:57218/0 (socket says 172.21.15.37:57218) 2024-04-25T08:53:28.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c6bfff640 1 -- 172.21.15.37:0/2575253903 learned_addr learned my addr 172.21.15.37:0/2575253903 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:28.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c6bfff640 1 -- 172.21.15.37:0/2575253903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c7406ec60 msgr2=0x7f5c74142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:28.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c6bfff640 1 --2- 172.21.15.37:0/2575253903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c7406ec60 0x7f5c74142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c6bfff640 1 -- 172.21.15.37:0/2575253903 >> v1:172.21.15.203:6789/0 conn(0x7f5c740ce510 legacy=0x7f5c740ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.976+0000 7f5c6bfff640 1 -- 172.21.15.37:0/2575253903 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7406dc40 con 0x7f5c74144ca0 2024-04-25T08:53:28.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c6bfff640 1 --2- 172.21.15.37:0/2575253903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c74144ca0 0x7f5c7406e720 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009870 tx=0x7f5c5c01ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3831a141a3de2005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:28.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7259c640 1 -- 172.21.15.37:0/2575253903 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0b5070 con 0x7f5c74144ca0 2024-04-25T08:53:28.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7259c640 1 -- 172.21.15.37:0/2575253903 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c0a8a50 con 0x7f5c74144ca0 2024-04-25T08:53:28.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7259c640 1 -- 172.21.15.37:0/2575253903 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0a8e20 con 0x7f5c74144ca0 2024-04-25T08:53:28.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 -- 172.21.15.37:0/2575253903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c74144ca0 msgr2=0x7f5c7406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 --2- 172.21.15.37:0/2575253903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c74144ca0 0x7f5c7406e720 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009870 tx=0x7f5c5c01ed50 comp rx=0 tx=0).stop 2024-04-25T08:53:28.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 -- 172.21.15.37:0/2575253903 shutdown_connections 2024-04-25T08:53:28.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 --2- 172.21.15.37:0/2575253903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c7406ec60 0x7f5c74142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 --2- 172.21.15.37:0/2575253903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c74144ca0 0x7f5c7406e720 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 -- 172.21.15.37:0/2575253903 >> 172.21.15.37:0/2575253903 conn(0x7f5c740b11a0 msgr2=0x7f5c740b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:28.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 -- 172.21.15.37:0/2575253903 shutdown_connections 2024-04-25T08:53:28.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 -- 172.21.15.37:0/2575253903 wait complete. 2024-04-25T08:53:28.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 Processor -- start 2024-04-25T08:53:28.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 -- start start 2024-04-25T08:53:28.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c7406ec60 0x7f5c7415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c740ce510 0x7f5c740c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c74144ca0 0x7f5c740ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c741451f0 con 0x7f5c7406ec60 2024-04-25T08:53:28.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c740cc900 con 0x7f5c74144ca0 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c7921a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c74145370 con 0x7f5c740ce510 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.980+0000 7f5c72d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c7406ec60 0x7f5c7415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c72d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c7406ec60 0x7f5c7415b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41728/0 (socket says 172.21.15.37:41728) 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c72d9d640 1 -- 172.21.15.37:0/3958920205 learned_addr learned my addr 172.21.15.37:0/3958920205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c7359e640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c74144ca0 0x7f5c740ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c6bfff640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c740ce510 0x7f5c740c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c72d9d640 1 -- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c740ce510 msgr2=0x7f5c740c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c72d9d640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c740ce510 0x7f5c740c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c72d9d640 1 -- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c74144ca0 msgr2=0x7f5c740ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c72d9d640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c74144ca0 0x7f5c740ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c72d9d640 1 -- 172.21.15.37:0/3958920205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c740c92d0 con 0x7f5c7406ec60 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c7359e640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c74144ca0 0x7f5c740ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c72d9d640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c7406ec60 0x7f5c7415b4c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400c420 tx=0x7f5c6400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:28.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c6b7fe640 1 -- 172.21.15.37:0/3958920205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640052d0 con 0x7f5c7406ec60 2024-04-25T08:53:28.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c6b7fe640 1 -- 172.21.15.37:0/3958920205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c64005470 con 0x7f5c7406ec60 2024-04-25T08:53:28.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c6b7fe640 1 -- 172.21.15.37:0/3958920205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64007260 con 0x7f5c7406ec60 2024-04-25T08:53:28.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c7921a640 1 -- 172.21.15.37:0/3958920205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c740c95b0 con 0x7f5c7406ec60 2024-04-25T08:53:28.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c7921a640 1 -- 172.21.15.37:0/3958920205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c740c9b70 con 0x7f5c7406ec60 2024-04-25T08:53:28.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c7921a640 1 -- 172.21.15.37:0/3958920205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7404efa0 con 0x7f5c7406ec60 2024-04-25T08:53:28.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c6b7fe640 1 -- 172.21.15.37:0/3958920205 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5c64007400 con 0x7f5c7406ec60 2024-04-25T08:53:28.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c6b7fe640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5c54066f60 0x7f5c54069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:28.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c6b7fe640 1 -- 172.21.15.37:0/3958920205 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5c64084950 con 0x7f5c7406ec60 2024-04-25T08:53:28.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c6bfff640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5c54066f60 0x7f5c54069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:28.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.984+0000 7f5c6bfff640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5c54066f60 0x7f5c54069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0b2000 tx=0x7f5c5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:28.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:28.988+0000 7f5c6b7fe640 1 -- 172.21.15.37:0/3958920205 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5c64054e70 con 0x7f5c7406ec60 2024-04-25T08:53:29.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.144+0000 7f5c7921a640 1 -- 172.21.15.37:0/3958920205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c740cee20 con 0x7f5c7406ec60 2024-04-25T08:53:29.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c6b7fe640 1 -- 172.21.15.37:0/3958920205 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f5c64058f10 con 0x7f5c7406ec60 2024-04-25T08:53:29.141 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:29.141 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:29.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 -- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5c54066f60 msgr2=0x7f5c54069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5c54066f60 0x7f5c54069420 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0b2000 tx=0x7f5c5c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:29.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 -- 172.21.15.37:0/3958920205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c7406ec60 msgr2=0x7f5c7415b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c7406ec60 0x7f5c7415b4c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400c420 tx=0x7f5c6400c8e0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 -- 172.21.15.37:0/3958920205 shutdown_connections 2024-04-25T08:53:29.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5c54066f60 0x7f5c54069420 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5c74144ca0 0x7f5c740ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5c740ce510 0x7f5c740c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 --2- 172.21.15.37:0/3958920205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c7406ec60 0x7f5c7415b4c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 -- 172.21.15.37:0/3958920205 >> 172.21.15.37:0/3958920205 conn(0x7f5c740b11a0 msgr2=0x7f5c740c6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:29.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 -- 172.21.15.37:0/3958920205 shutdown_connections 2024-04-25T08:53:29.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.148+0000 7f5c7921a640 1 -- 172.21.15.37:0/3958920205 wait complete. 2024-04-25T08:53:29.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:29.159 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:29.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac44fc8640 1 Processor -- start 2024-04-25T08:53:29.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac44fc8640 1 -- start start 2024-04-25T08:53:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac44fc8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fac4015b6b0 0x7fac400ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac44fc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac400cf590 0x7fac400cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac44fc8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac400cbd20 con 0x7fac4015f130 2024-04-25T08:53:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac44fc8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4015ba90 con 0x7fac4015b6b0 2024-04-25T08:53:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac44fc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac400cff30 con 0x7fac400cf590 2024-04-25T08:53:29.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac3e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fac4015b6b0 0x7fac400ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac3ed9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fac4015f130 0x7fac4015f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41874/0 (socket says 172.21.15.37:41874) 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac3ed9d640 1 -- 172.21.15.37:0/3813756240 learned_addr learned my addr 172.21.15.37:0/3813756240 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac3dd9b640 1 --2- 172.21.15.37:0/3813756240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac400cf590 0x7fac400cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac3e59c640 1 -- 172.21.15.37:0/3813756240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac400cf590 msgr2=0x7fac400cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac3e59c640 1 --2- 172.21.15.37:0/3813756240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac400cf590 0x7fac400cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac3e59c640 1 -- 172.21.15.37:0/3813756240 >> v1:172.21.15.37:6789/0 conn(0x7fac4015f130 legacy=0x7fac4015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac3d59a640 1 -- 172.21.15.37:0/3813756240 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885677314 0 0) 0x7fac400cbd20 con 0x7fac4015f130 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac3e59c640 1 -- 172.21.15.37:0/3813756240 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac400cc000 con 0x7fac4015b6b0 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.284+0000 7fac3dd9b640 1 --2- 172.21.15.37:0/3813756240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac400cf590 0x7fac400cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac3e59c640 1 --2- 172.21.15.37:0/3813756240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4015b6b0 0x7fac400ca3c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fac300097c0 tx=0x7fac3001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c04b232608db544 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac3d59a640 1 -- 172.21.15.37:0/3813756240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac300a7070 con 0x7fac4015b6b0 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac3d59a640 1 -- 172.21.15.37:0/3813756240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac300b0070 con 0x7fac4015b6b0 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac3d59a640 1 -- 172.21.15.37:0/3813756240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac300ab890 con 0x7fac4015b6b0 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 -- 172.21.15.37:0/3813756240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4015b6b0 msgr2=0x7fac400ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 --2- 172.21.15.37:0/3813756240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4015b6b0 0x7fac400ca3c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fac300097c0 tx=0x7fac3001c470 comp rx=0 tx=0).stop 2024-04-25T08:53:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 -- 172.21.15.37:0/3813756240 shutdown_connections 2024-04-25T08:53:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 --2- 172.21.15.37:0/3813756240 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac400cf590 0x7fac400cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 --2- 172.21.15.37:0/3813756240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4015b6b0 0x7fac400ca3c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 -- 172.21.15.37:0/3813756240 >> 172.21.15.37:0/3813756240 conn(0x7fac4006ae60 msgr2=0x7fac4006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 -- 172.21.15.37:0/3813756240 shutdown_connections 2024-04-25T08:53:29.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 -- 172.21.15.37:0/3813756240 wait complete. 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 Processor -- start 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 -- start start 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac400cf590 0x7fac400c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4015b6b0 0x7fac4015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4015f130 0x7fac400ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac400ca900 con 0x7fac400cf590 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac400cba30 con 0x7fac4015b6b0 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac44fc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac400cadb0 con 0x7fac4015f130 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac3e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac400cf590 0x7fac400c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac3e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac400cf590 0x7fac400c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41744/0 (socket says 172.21.15.37:41744) 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.288+0000 7fac3e59c640 1 -- 172.21.15.37:0/497577332 learned_addr learned my addr 172.21.15.37:0/497577332 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac3ed9d640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4015f130 0x7fac400ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac3dd9b640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4015b6b0 0x7fac4015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac3e59c640 1 -- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4015f130 msgr2=0x7fac400ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac3e59c640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4015f130 0x7fac400ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac3e59c640 1 -- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4015b6b0 msgr2=0x7fac4015bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac3e59c640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4015b6b0 0x7fac4015bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac3e59c640 1 -- 172.21.15.37:0/497577332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac40164b70 con 0x7fac400cf590 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac3dd9b640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4015b6b0 0x7fac4015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac3e59c640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac400cf590 0x7fac400c8b50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fac300097c0 tx=0x7fac3001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac36ffd640 1 -- 172.21.15.37:0/497577332 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30004220 con 0x7fac400cf590 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac36ffd640 1 -- 172.21.15.37:0/497577332 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac300043c0 con 0x7fac400cf590 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac36ffd640 1 -- 172.21.15.37:0/497577332 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac30003820 con 0x7fac400cf590 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac44fc8640 1 -- 172.21.15.37:0/497577332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac40164e50 con 0x7fac400cf590 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac44fc8640 1 -- 172.21.15.37:0/497577332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac400ce070 con 0x7fac400cf590 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac44fc8640 1 -- 172.21.15.37:0/497577332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4004efa0 con 0x7fac400cf590 2024-04-25T08:53:29.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac36ffd640 1 -- 172.21.15.37:0/497577332 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac300039c0 con 0x7fac400cf590 2024-04-25T08:53:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac36ffd640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac20066f60 0x7fac20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac36ffd640 1 -- 172.21.15.37:0/497577332 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fac30120420 con 0x7fac400cf590 2024-04-25T08:53:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.292+0000 7fac3dd9b640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac20066f60 0x7fac20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.296+0000 7fac3dd9b640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac20066f60 0x7fac20069420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac40071b00 tx=0x7fac28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:29.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.296+0000 7fac36ffd640 1 -- 172.21.15.37:0/497577332 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac30016090 con 0x7fac400cf590 2024-04-25T08:53:29.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.456+0000 7fac44fc8640 1 -- 172.21.15.37:0/497577332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac400c9690 con 0x7fac400cf590 2024-04-25T08:53:29.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.456+0000 7fac36ffd640 1 -- 172.21.15.37:0/497577332 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac300f1010 con 0x7fac400cf590 2024-04-25T08:53:29.451 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 -- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac20066f60 msgr2=0x7fac20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac20066f60 0x7fac20069420 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fac40071b00 tx=0x7fac28008040 comp rx=0 tx=0).stop 2024-04-25T08:53:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 -- 172.21.15.37:0/497577332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac400cf590 msgr2=0x7fac400c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac400cf590 0x7fac400c8b50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fac300097c0 tx=0x7fac3001acb0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 -- 172.21.15.37:0/497577332 shutdown_connections 2024-04-25T08:53:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac20066f60 0x7fac20069420 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4015f130 0x7fac400ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac4015b6b0 0x7fac4015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 --2- 172.21.15.37:0/497577332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac400cf590 0x7fac400c8b50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 -- 172.21.15.37:0/497577332 >> 172.21.15.37:0/497577332 conn(0x7fac4006ae60 msgr2=0x7fac400c9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 -- 172.21.15.37:0/497577332 shutdown_connections 2024-04-25T08:53:29.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.460+0000 7fac44fc8640 1 -- 172.21.15.37:0/497577332 wait complete. 2024-04-25T08:53:29.469 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:29.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f6526666640 1 Processor -- start 2024-04-25T08:53:29.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f6526666640 1 -- start start 2024-04-25T08:53:29.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f6526666640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652015b6b0 0x7f65200ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f6526666640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f65200cf590 0x7f65200cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f6526666640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65200cbd20 con 0x7f652015f130 2024-04-25T08:53:29.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f6526666640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652015ba90 con 0x7f65200cf590 2024-04-25T08:53:29.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f6526666640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65200cff30 con 0x7f652015b6b0 2024-04-25T08:53:29.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f651f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f65200cf590 0x7f65200cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f651f7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f65200cf590 0x7f65200cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47576/0 (socket says 172.21.15.37:47576) 2024-04-25T08:53:29.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f651f7fe640 1 -- 172.21.15.37:0/92683449 learned_addr learned my addr 172.21.15.37:0/92683449 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:29.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f651ffff640 1 --2- 172.21.15.37:0/92683449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652015b6b0 0x7f65200ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f651f7fe640 1 -- 172.21.15.37:0/92683449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652015b6b0 msgr2=0x7f65200ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:29.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f651f7fe640 1 --2- 172.21.15.37:0/92683449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652015b6b0 0x7f65200ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f651f7fe640 1 -- 172.21.15.37:0/92683449 >> v1:172.21.15.203:6789/0 conn(0x7f652015f130 legacy=0x7f652015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.592+0000 7f651f7fe640 1 -- 172.21.15.37:0/92683449 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200cc000 con 0x7f65200cf590 2024-04-25T08:53:29.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651f7fe640 1 --2- 172.21.15.37:0/92683449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65200cf590 0x7f65200cf9f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6514009870 tx=0x7f651401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8535a611757f49be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:29.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651effd640 1 -- 172.21.15.37:0/92683449 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140a95f0 con 0x7f65200cf590 2024-04-25T08:53:29.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651effd640 1 -- 172.21.15.37:0/92683449 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65140a9790 con 0x7f65200cf590 2024-04-25T08:53:29.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651effd640 1 -- 172.21.15.37:0/92683449 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140a9b60 con 0x7f65200cf590 2024-04-25T08:53:29.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 -- 172.21.15.37:0/92683449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65200cf590 msgr2=0x7f65200cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 --2- 172.21.15.37:0/92683449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65200cf590 0x7f65200cf9f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6514009870 tx=0x7f651401c530 comp rx=0 tx=0).stop 2024-04-25T08:53:29.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 -- 172.21.15.37:0/92683449 shutdown_connections 2024-04-25T08:53:29.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 --2- 172.21.15.37:0/92683449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65200cf590 0x7f65200cf9f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 --2- 172.21.15.37:0/92683449 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652015b6b0 0x7f65200ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 -- 172.21.15.37:0/92683449 >> 172.21.15.37:0/92683449 conn(0x7f652006ae60 msgr2=0x7f652006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:29.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 -- 172.21.15.37:0/92683449 shutdown_connections 2024-04-25T08:53:29.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 -- 172.21.15.37:0/92683449 wait complete. 2024-04-25T08:53:29.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 Processor -- start 2024-04-25T08:53:29.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 -- start start 2024-04-25T08:53:29.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65200cf590 0x7f652006d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f652015b6b0 0x7f6520097d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652015f130 0x7f6520098240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65200ca900 con 0x7f652015b6b0 2024-04-25T08:53:29.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65200cba30 con 0x7f65200cf590 2024-04-25T08:53:29.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f6526666640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65200cadb0 con 0x7f652015f130 2024-04-25T08:53:29.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f652015b6b0 0x7f6520097d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f652015b6b0 0x7f6520097d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47578/0 (socket says 172.21.15.37:47578) 2024-04-25T08:53:29.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651f7fe640 1 -- 172.21.15.37:0/1691656007 learned_addr learned my addr 172.21.15.37:0/1691656007 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:29.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651f7fe640 1 -- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652015f130 msgr2=0x7f6520098240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651f7fe640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652015f130 0x7f6520098240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651f7fe640 1 -- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65200cf590 msgr2=0x7f652006d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651f7fe640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65200cf590 0x7f652006d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.596+0000 7f651f7fe640 1 -- 172.21.15.37:0/1691656007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200af590 con 0x7f652015b6b0 2024-04-25T08:53:29.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f651f7fe640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f652015b6b0 0x7f6520097d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f65140ab040 tx=0x7f651401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:29.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f651cff9640 1 -- 172.21.15.37:0/1691656007 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140b1800 con 0x7f652015b6b0 2024-04-25T08:53:29.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f651cff9640 1 -- 172.21.15.37:0/1691656007 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65140b19a0 con 0x7f652015b6b0 2024-04-25T08:53:29.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f6526666640 1 -- 172.21.15.37:0/1691656007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65200adf10 con 0x7f652015b6b0 2024-04-25T08:53:29.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f651cff9640 1 -- 172.21.15.37:0/1691656007 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65140b1d90 con 0x7f652015b6b0 2024-04-25T08:53:29.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f6526666640 1 -- 172.21.15.37:0/1691656007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65200cd2b0 con 0x7f652015b6b0 2024-04-25T08:53:29.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f6526666640 1 -- 172.21.15.37:0/1691656007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f652004efa0 con 0x7f652015b6b0 2024-04-25T08:53:29.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f651cff9640 1 -- 172.21.15.37:0/1691656007 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65140a9650 con 0x7f652015b6b0 2024-04-25T08:53:29.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f651cff9640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f60 0x7f650c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f651cff9640 1 -- 172.21.15.37:0/1691656007 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f651411ffe0 con 0x7f652015b6b0 2024-04-25T08:53:29.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f651ffff640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f60 0x7f650c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.600+0000 7f651ffff640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f60 0x7f650c069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65100098f0 tx=0x7f6510008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:29.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.604+0000 7f651cff9640 1 -- 172.21.15.37:0/1691656007 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65140ad020 con 0x7f652015b6b0 2024-04-25T08:53:29.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.760+0000 7f6526666640 1 -- 172.21.15.37:0/1691656007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f652006eae0 con 0x7f652015b6b0 2024-04-25T08:53:29.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.760+0000 7f651cff9640 1 -- 172.21.15.37:0/1691656007 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6514016150 con 0x7f652015b6b0 2024-04-25T08:53:29.755 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:29.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 -- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f60 msgr2=0x7f650c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f60 0x7f650c069420 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f65100098f0 tx=0x7f6510008040 comp rx=0 tx=0).stop 2024-04-25T08:53:29.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 -- 172.21.15.37:0/1691656007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f652015b6b0 msgr2=0x7f6520097d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f652015b6b0 0x7f6520097d00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f65140ab040 tx=0x7f651401ad70 comp rx=0 tx=0).stop 2024-04-25T08:53:29.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 -- 172.21.15.37:0/1691656007 shutdown_connections 2024-04-25T08:53:29.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f60 0x7f650c069420 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652015f130 0x7f6520098240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f652015b6b0 0x7f6520097d00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 --2- 172.21.15.37:0/1691656007 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65200cf590 0x7f652006d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 -- 172.21.15.37:0/1691656007 >> 172.21.15.37:0/1691656007 conn(0x7f652006ae60 msgr2=0x7f65200b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 -- 172.21.15.37:0/1691656007 shutdown_connections 2024-04-25T08:53:29.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.764+0000 7f6526666640 1 -- 172.21.15.37:0/1691656007 wait complete. 2024-04-25T08:53:29.773 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:29.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.896+0000 7f5a4bfd0640 1 Processor -- start 2024-04-25T08:53:29.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.896+0000 7f5a4bfd0640 1 -- start start 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4bfd0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5a440cfca0 0x7f5a4415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4bfd0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a440cbf10 con 0x7f5a440b3cf0 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4bfd0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a440cc1f0 con 0x7f5a4415e4f0 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4bfd0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a440cc4d0 con 0x7f5a440cfca0 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a49d6c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5a440b3cf0 0x7f5a440cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52450/0 (socket says 172.21.15.37:52450) 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a49d6c640 1 -- 172.21.15.37:0/3733459580 learned_addr learned my addr 172.21.15.37:0/3733459580 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4956b640 1 --2- 172.21.15.37:0/3733459580 >> v2:172.21.15.203:3300/0 conn(0x7f5a440cfca0 0x7f5a4415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 994770813 0 0) 0x7f5a440cbf10 con 0x7f5a440b3cf0 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a30003530 con 0x7f5a440b3cf0 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 373559490 0 0) 0x7f5a30003530 con 0x7f5a440b3cf0 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 >> v2:172.21.15.203:3300/0 conn(0x7f5a440cfca0 msgr2=0x7f5a4415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:29.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 --2- 172.21.15.37:0/3733459580 >> v2:172.21.15.203:3300/0 conn(0x7f5a440cfca0 0x7f5a4415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 >> v1:172.21.15.203:6789/0 conn(0x7f5a4415e4f0 legacy=0x7f5a440caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a440cc7b0 con 0x7f5a440b3cf0 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f5a40001680 con 0x7f5a440b3cf0 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4956b640 1 --2- 172.21.15.37:0/3733459580 >> v2:172.21.15.203:3300/0 conn(0x7f5a440cfca0 0x7f5a4415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 >> v1:172.21.15.37:6789/0 conn(0x7f5a440b3cf0 legacy=0x7f5a440cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 --2- 172.21.15.37:0/3733459580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a30003a30 0x7f5a30003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 --2- 172.21.15.37:0/3733459580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 0x7f5a30004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 --2- 172.21.15.37:0/3733459580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a30005240 0x7f5a3000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a440cc4d0 con 0x7f5a30004810 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3000c1a0 con 0x7f5a30003a30 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3000c3a0 con 0x7f5a30005240 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4956b640 1 --2- 172.21.15.37:0/3733459580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 0x7f5a30004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a49d6c640 1 --2- 172.21.15.37:0/3733459580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a30003a30 0x7f5a30003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4956b640 1 -- 172.21.15.37:0/3733459580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a30005240 msgr2=0x7f5a3000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:29.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4a56d640 1 --2- 172.21.15.37:0/3733459580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a30005240 0x7f5a3000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4956b640 1 --2- 172.21.15.37:0/3733459580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a30005240 0x7f5a3000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4956b640 1 -- 172.21.15.37:0/3733459580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a30003a30 msgr2=0x7f5a30003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:29.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4956b640 1 --2- 172.21.15.37:0/3733459580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a30003a30 0x7f5a30003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4956b640 1 -- 172.21.15.37:0/3733459580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a30003530 con 0x7f5a30004810 2024-04-25T08:53:29.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4a56d640 1 -- 172.21.15.37:0/3733459580 reap_dead start 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a4956b640 1 --2- 172.21.15.37:0/3733459580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 0x7f5a30004c90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a44070f70 tx=0x7f5a3401c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505709d6c3333628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3400f040 con 0x7f5a30004810 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a340a9680 con 0x7f5a30004810 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.900+0000 7f5a48d6a640 1 -- 172.21.15.37:0/3733459580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a340a9a50 con 0x7f5a30004810 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3733459580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 msgr2=0x7f5a30004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 --2- 172.21.15.37:0/3733459580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 0x7f5a30004c90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5a44070f70 tx=0x7f5a3401c570 comp rx=0 tx=0).stop 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3733459580 shutdown_connections 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 --2- 172.21.15.37:0/3733459580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 0x7f5a30004c90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3733459580 >> 172.21.15.37:0/3733459580 conn(0x7f5a4406ae60 msgr2=0x7f5a4406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3733459580 shutdown_connections 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3733459580 wait complete. 2024-04-25T08:53:29.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 Processor -- start 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- start start 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 0x7f5a44160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a440c9530 0x7f5a44157570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a440c5040 0x7f5a440aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4415eae0 con 0x7f5a30004810 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4415e960 con 0x7f5a440c5040 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a440b40d0 con 0x7f5a440c9530 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a49d6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 0x7f5a44160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a49d6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 0x7f5a44160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47594/0 (socket says 172.21.15.37:47594) 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a49d6c640 1 -- 172.21.15.37:0/3347489311 learned_addr learned my addr 172.21.15.37:0/3347489311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4956b640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a440c9530 0x7f5a44157570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a49d6c640 1 -- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a440c9530 msgr2=0x7f5a44157570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a49d6c640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a440c9530 0x7f5a44157570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a49d6c640 1 -- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a440c5040 msgr2=0x7f5a440aede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a49d6c640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a440c5040 0x7f5a440aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a49d6c640 1 -- 172.21.15.37:0/3347489311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a440c6240 con 0x7f5a30004810 2024-04-25T08:53:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4956b640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a440c9530 0x7f5a44157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a49d6c640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 0x7f5a44160900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5a40010c00 tx=0x7f5a4000e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a3a7fc640 1 -- 172.21.15.37:0/3347489311 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4001b070 con 0x7f5a30004810 2024-04-25T08:53:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a3a7fc640 1 -- 172.21.15.37:0/3347489311 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a40011040 con 0x7f5a30004810 2024-04-25T08:53:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3347489311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a440c8240 con 0x7f5a30004810 2024-04-25T08:53:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a3a7fc640 1 -- 172.21.15.37:0/3347489311 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400143a0 con 0x7f5a30004810 2024-04-25T08:53:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3347489311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a440c8800 con 0x7f5a30004810 2024-04-25T08:53:29.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.904+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3347489311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4404efa0 con 0x7f5a30004810 2024-04-25T08:53:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.908+0000 7f5a3a7fc640 1 -- 172.21.15.37:0/3347489311 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a4001e070 con 0x7f5a30004810 2024-04-25T08:53:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.908+0000 7f5a3a7fc640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a28066f60 0x7f5a28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.908+0000 7f5a3a7fc640 1 -- 172.21.15.37:0/3347489311 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5a400880c0 con 0x7f5a30004810 2024-04-25T08:53:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.908+0000 7f5a4956b640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a28066f60 0x7f5a28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:29.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.908+0000 7f5a4956b640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a28066f60 0x7f5a28069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a340ae000 tx=0x7f5a340046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:29.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:29.912+0000 7f5a3a7fc640 1 -- 172.21.15.37:0/3347489311 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a40058c30 con 0x7f5a30004810 2024-04-25T08:53:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.064+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3347489311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a440c5e10 con 0x7f5a30004810 2024-04-25T08:53:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.064+0000 7f5a3a7fc640 1 -- 172.21.15.37:0/3347489311 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a40058530 con 0x7f5a30004810 2024-04-25T08:53:30.057 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a28066f60 msgr2=0x7f5a28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a28066f60 0x7f5a28069420 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5a340ae000 tx=0x7f5a340046d0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3347489311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 msgr2=0x7f5a44160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 0x7f5a44160900 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5a40010c00 tx=0x7f5a4000e4d0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3347489311 shutdown_connections 2024-04-25T08:53:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a28066f60 0x7f5a28069420 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a440c5040 0x7f5a440aede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a440c9530 0x7f5a44157570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 --2- 172.21.15.37:0/3347489311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a30004810 0x7f5a44160900 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3347489311 >> 172.21.15.37:0/3347489311 conn(0x7f5a4406ae60 msgr2=0x7f5a440ac380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:30.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3347489311 shutdown_connections 2024-04-25T08:53:30.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.068+0000 7f5a4bfd0640 1 -- 172.21.15.37:0/3347489311 wait complete. 2024-04-25T08:53:30.075 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:30.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.204+0000 7f304f268640 1 Processor -- start 2024-04-25T08:53:30.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.204+0000 7f304f268640 1 -- start start 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304f268640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f304815b800 0x7f30480cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304f268640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30480cbd20 con 0x7f30480cfad0 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304f268640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30480cc000 con 0x7f304815b420 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304f268640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30480cff40 con 0x7f304815b800 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304d805640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f30480cfad0 0x7f3048072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52454/0 (socket says 172.21.15.37:52454) 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304d805640 1 -- 172.21.15.37:0/2641068395 learned_addr learned my addr 172.21.15.37:0/2641068395 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304c803640 1 --2- 172.21.15.37:0/2641068395 >> v2:172.21.15.203:3300/0 conn(0x7f304815b800 0x7f30480cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2335272652 0 0) 0x7f30480cbd20 con 0x7f30480cfad0 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3034003530 con 0x7f30480cfad0 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3789285155 0 0) 0x7f3034003530 con 0x7f30480cfad0 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 >> v2:172.21.15.203:3300/0 conn(0x7f304815b800 msgr2=0x7f30480cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:30.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 --2- 172.21.15.37:0/2641068395 >> v2:172.21.15.203:3300/0 conn(0x7f304815b800 0x7f30480cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 >> v1:172.21.15.203:6789/0 conn(0x7f304815b420 legacy=0x7f30480ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30480cc2e0 con 0x7f30480cfad0 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3040001a00 con 0x7f30480cfad0 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 >> v1:172.21.15.37:6789/0 conn(0x7f30480cfad0 legacy=0x7f3048072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 --2- 172.21.15.37:0/2641068395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3034003a30 0x7f3034003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 --2- 172.21.15.37:0/2641068395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3034004810 0x7f3034004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 --2- 172.21.15.37:0/2641068395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30340051d0 0x7f303400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30480cff40 con 0x7f30340051d0 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303400c100 con 0x7f3034004810 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303400c300 con 0x7f3034003a30 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304d805640 1 --2- 172.21.15.37:0/2641068395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30340051d0 0x7f303400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304d004640 1 --2- 172.21.15.37:0/2641068395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3034003a30 0x7f3034003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304d805640 1 -- 172.21.15.37:0/2641068395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3034003a30 msgr2=0x7f3034003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304d805640 1 --2- 172.21.15.37:0/2641068395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3034003a30 0x7f3034003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304d805640 1 -- 172.21.15.37:0/2641068395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3034004810 msgr2=0x7f3034004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:30.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304d805640 1 --2- 172.21.15.37:0/2641068395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3034004810 0x7f3034004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.208+0000 7f304d805640 1 -- 172.21.15.37:0/2641068395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3034003530 con 0x7f30340051d0 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304d805640 1 -- 172.21.15.37:0/2641068395 reap_dead start 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304d805640 1 --2- 172.21.15.37:0/2641068395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30340051d0 0x7f303400b9d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f304000a840 tx=0x7f304001d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4729e368817b854d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304000a460 con 0x7f30340051d0 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30400ac640 con 0x7f30340051d0 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f303ffff640 1 -- 172.21.15.37:0/2641068395 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30400aca30 con 0x7f30340051d0 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 -- 172.21.15.37:0/2641068395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30340051d0 msgr2=0x7f303400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 --2- 172.21.15.37:0/2641068395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30340051d0 0x7f303400b9d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f304000a840 tx=0x7f304001d360 comp rx=0 tx=0).stop 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 -- 172.21.15.37:0/2641068395 shutdown_connections 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 --2- 172.21.15.37:0/2641068395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30340051d0 0x7f303400b9d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 -- 172.21.15.37:0/2641068395 >> 172.21.15.37:0/2641068395 conn(0x7f304806ae60 msgr2=0x7f304806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 -- 172.21.15.37:0/2641068395 shutdown_connections 2024-04-25T08:53:30.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 -- 172.21.15.37:0/2641068395 wait complete. 2024-04-25T08:53:30.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 Processor -- start 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 -- start start 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30340051d0 0x7f3048157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f304815c3d0 0x7f3048156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3048157d10 0x7f30481581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30480cb9f0 con 0x7f3048157d10 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30480cc930 con 0x7f30340051d0 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304f268640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30480cbb70 con 0x7f304815c3d0 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304d805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3048157d10 0x7f30481581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304d805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3048157d10 0x7f30481581a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47598/0 (socket says 172.21.15.37:47598) 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304d805640 1 -- 172.21.15.37:0/2644488126 learned_addr learned my addr 172.21.15.37:0/2644488126 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304c803640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f304815c3d0 0x7f3048156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304d805640 1 -- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f304815c3d0 msgr2=0x7f3048156240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304d805640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f304815c3d0 0x7f3048156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304d805640 1 -- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30340051d0 msgr2=0x7f3048157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304d805640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30340051d0 0x7f3048157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304d805640 1 -- 172.21.15.37:0/2644488126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304815c890 con 0x7f3048157d10 2024-04-25T08:53:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f304d805640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3048157d10 0x7f30481581a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f30400012f0 tx=0x7f3034005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f303dffb640 1 -- 172.21.15.37:0/2644488126 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30400019c0 con 0x7f3048157d10 2024-04-25T08:53:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f303dffb640 1 -- 172.21.15.37:0/2644488126 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3040001b60 con 0x7f3048157d10 2024-04-25T08:53:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.212+0000 7f303dffb640 1 -- 172.21.15.37:0/2644488126 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30400ac9f0 con 0x7f3048157d10 2024-04-25T08:53:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.216+0000 7f304f268640 1 -- 172.21.15.37:0/2644488126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304815e2a0 con 0x7f3048157d10 2024-04-25T08:53:30.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.216+0000 7f304f268640 1 -- 172.21.15.37:0/2644488126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3048157280 con 0x7f3048157d10 2024-04-25T08:53:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.216+0000 7f304f268640 1 -- 172.21.15.37:0/2644488126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304804efa0 con 0x7f3048157d10 2024-04-25T08:53:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.216+0000 7f303dffb640 1 -- 172.21.15.37:0/2644488126 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f30400070c0 con 0x7f3048157d10 2024-04-25T08:53:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.216+0000 7f303dffb640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f302c066f90 0x7f302c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.216+0000 7f303dffb640 1 -- 172.21.15.37:0/2644488126 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f304011ea70 con 0x7f3048157d10 2024-04-25T08:53:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.216+0000 7f304d004640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f302c066f90 0x7f302c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.216+0000 7f304d004640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f302c066f90 0x7f302c069450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3044001500 tx=0x7f3044009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:30.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.220+0000 7f303dffb640 1 -- 172.21.15.37:0/2644488126 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f30400bd020 con 0x7f3048157d10 2024-04-25T08:53:30.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.376+0000 7f304f268640 1 -- 172.21.15.37:0/2644488126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f304815e070 con 0x7f3048157d10 2024-04-25T08:53:30.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.376+0000 7f303dffb640 1 -- 172.21.15.37:0/2644488126 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3040017070 con 0x7f3048157d10 2024-04-25T08:53:30.371 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:30.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 -- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f302c066f90 msgr2=0x7f302c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f302c066f90 0x7f302c069450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3044001500 tx=0x7f3044009040 comp rx=0 tx=0).stop 2024-04-25T08:53:30.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 -- 172.21.15.37:0/2644488126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3048157d10 msgr2=0x7f30481581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3048157d10 0x7f30481581a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f30400012f0 tx=0x7f3034005840 comp rx=0 tx=0).stop 2024-04-25T08:53:30.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 -- 172.21.15.37:0/2644488126 shutdown_connections 2024-04-25T08:53:30.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f302c066f90 0x7f302c069450 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3048157d10 0x7f30481581a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f304815c3d0 0x7f3048156240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 --2- 172.21.15.37:0/2644488126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f30340051d0 0x7f3048157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 -- 172.21.15.37:0/2644488126 >> 172.21.15.37:0/2644488126 conn(0x7f304806ae60 msgr2=0x7f304806cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:30.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 -- 172.21.15.37:0/2644488126 shutdown_connections 2024-04-25T08:53:30.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.380+0000 7f304f268640 1 -- 172.21.15.37:0/2644488126 wait complete. 2024-04-25T08:53:30.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:53:30.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.520+0000 7f7d8e2ef640 1 Processor -- start 2024-04-25T08:53:30.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.520+0000 7f7d8e2ef640 1 -- start start 2024-04-25T08:53:30.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.520+0000 7f7d8e2ef640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7d8815a750 0x7f7d8806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.520+0000 7f7d8e2ef640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7d88066bb0 0x7f7d8806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.520+0000 7f7d8e2ef640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d8806c1e0 0x7f7d8815e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.520+0000 7f7d8e2ef640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d88067100 con 0x7f7d8815a750 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.520+0000 7f7d8e2ef640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d8815ab30 con 0x7f7d88066bb0 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.520+0000 7f7d8e2ef640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d880cf560 con 0x7f7d8806c1e0 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.520+0000 7f7d87fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7d8815a750 0x7f7d8806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d87fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7d8815a750 0x7f7d8806ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47602/0 (socket says 172.21.15.37:47602) 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d87fff640 1 -- 172.21.15.37:0/1678876652 learned_addr learned my addr 172.21.15.37:0/1678876652 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d877fe640 1 --2- 172.21.15.37:0/1678876652 >> v2:172.21.15.203:3300/0 conn(0x7f7d88066bb0 0x7f7d8806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d8c88c640 1 --2- 172.21.15.37:0/1678876652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d8806c1e0 0x7f7d8815e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d87fff640 1 -- 172.21.15.37:0/1678876652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d8806c1e0 msgr2=0x7f7d8815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d87fff640 1 --2- 172.21.15.37:0/1678876652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d8806c1e0 0x7f7d8815e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d87fff640 1 -- 172.21.15.37:0/1678876652 >> v2:172.21.15.203:3300/0 conn(0x7f7d88066bb0 msgr2=0x7f7d8806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d87fff640 1 --2- 172.21.15.37:0/1678876652 >> v2:172.21.15.203:3300/0 conn(0x7f7d88066bb0 0x7f7d8806bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d87fff640 1 -- 172.21.15.37:0/1678876652 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d880cf840 con 0x7f7d8815a750 2024-04-25T08:53:30.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d87fff640 1 --2- 172.21.15.37:0/1678876652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d8815a750 0x7f7d8806ee60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7d880b5ef0 tx=0x7f7d7801c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2cdfb9db77a7fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:30.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d86ffd640 1 -- 172.21.15.37:0/1678876652 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780b1070 con 0x7f7d8815a750 2024-04-25T08:53:30.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d86ffd640 1 -- 172.21.15.37:0/1678876652 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d780ac440 con 0x7f7d8815a750 2024-04-25T08:53:30.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d86ffd640 1 -- 172.21.15.37:0/1678876652 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780ac810 con 0x7f7d8815a750 2024-04-25T08:53:30.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/1678876652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d8815a750 msgr2=0x7f7d8806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d8e2ef640 1 --2- 172.21.15.37:0/1678876652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d8815a750 0x7f7d8806ee60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7d880b5ef0 tx=0x7f7d7801c140 comp rx=0 tx=0).stop 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/1678876652 shutdown_connections 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d8e2ef640 1 --2- 172.21.15.37:0/1678876652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d8806c1e0 0x7f7d8815e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d8e2ef640 1 --2- 172.21.15.37:0/1678876652 >> v2:172.21.15.203:3300/0 conn(0x7f7d88066bb0 0x7f7d8806bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d8e2ef640 1 --2- 172.21.15.37:0/1678876652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d8815a750 0x7f7d8806ee60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/1678876652 >> 172.21.15.37:0/1678876652 conn(0x7f7d88071810 msgr2=0x7f7d88071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/1678876652 shutdown_connections 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.524+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/1678876652 wait complete. 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8e2ef640 1 Processor -- start 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8e2ef640 1 -- start start 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8e2ef640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d88066bb0 0x7f7d8815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8e2ef640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8806c1e0 0x7f7d88148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8e2ef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d8815a750 0x7f7d88149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8e2ef640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8806dc40 con 0x7f7d8815a750 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8e2ef640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8806dac0 con 0x7f7d8806c1e0 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8e2ef640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8806ddc0 con 0x7f7d88066bb0 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8c88c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d8815a750 0x7f7d88149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8c88c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d8815a750 0x7f7d88149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47614/0 (socket says 172.21.15.37:47614) 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8c88c640 1 -- 172.21.15.37:0/8859353 learned_addr learned my addr 172.21.15.37:0/8859353 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d877fe640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8806c1e0 0x7f7d88148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d87fff640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d88066bb0 0x7f7d8815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8c88c640 1 -- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d88066bb0 msgr2=0x7f7d8815bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8c88c640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d88066bb0 0x7f7d8815bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8c88c640 1 -- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8806c1e0 msgr2=0x7f7d88148f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8c88c640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8806c1e0 0x7f7d88148f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8c88c640 1 -- 172.21.15.37:0/8859353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8815c290 con 0x7f7d8815a750 2024-04-25T08:53:30.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d877fe640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8806c1e0 0x7f7d88148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d87fff640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d88066bb0 0x7f7d8815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8c88c640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d8815a750 0x7f7d88149450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009dd0 tx=0x7f7d8000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d84ff9640 1 -- 172.21.15.37:0/8859353 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80018020 con 0x7f7d8815a750 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d84ff9640 1 -- 172.21.15.37:0/8859353 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d80014480 con 0x7f7d8815a750 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/8859353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d8815c530 con 0x7f7d8815a750 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d84ff9640 1 -- 172.21.15.37:0/8859353 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80014870 con 0x7f7d8815a750 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/8859353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8815caf0 con 0x7f7d8815a750 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.528+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/8859353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d8804efa0 con 0x7f7d8815a750 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.532+0000 7f7d84ff9640 1 -- 172.21.15.37:0/8859353 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d8001b020 con 0x7f7d8815a750 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.532+0000 7f7d84ff9640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d74066f60 0x7f7d74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.532+0000 7f7d84ff9640 1 -- 172.21.15.37:0/8859353 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7d80085ed0 con 0x7f7d8815a750 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.532+0000 7f7d87fff640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d74066f60 0x7f7d74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.532+0000 7f7d87fff640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d74066f60 0x7f7d74069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d780ae000 tx=0x7f7d78008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:30.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.536+0000 7f7d84ff9640 1 -- 172.21.15.37:0/8859353 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d80022070 con 0x7f7d8815a750 2024-04-25T08:53:30.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.704+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/8859353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7d880cd580 con 0x7f7d8815a750 2024-04-25T08:53:30.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.704+0000 7f7d84ff9640 1 -- 172.21.15.37:0/8859353 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7d80010040 con 0x7f7d8815a750 2024-04-25T08:53:30.698 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:53:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d74066f60 msgr2=0x7f7d74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d74066f60 0x7f7d74069420 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7d780ae000 tx=0x7f7d78008040 comp rx=0 tx=0).stop 2024-04-25T08:53:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/8859353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d8815a750 msgr2=0x7f7d88149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d8815a750 0x7f7d88149450 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009dd0 tx=0x7f7d8000c490 comp rx=0 tx=0).stop 2024-04-25T08:53:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/8859353 shutdown_connections 2024-04-25T08:53:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d74066f60 0x7f7d74069420 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d8815a750 0x7f7d88149450 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d8806c1e0 0x7f7d88148f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 --2- 172.21.15.37:0/8859353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d88066bb0 0x7f7d8815bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/8859353 >> 172.21.15.37:0/8859353 conn(0x7f7d88071810 msgr2=0x7f7d880b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:30.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/8859353 shutdown_connections 2024-04-25T08:53:30.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.708+0000 7f7d8e2ef640 1 -- 172.21.15.37:0/8859353 wait complete. 2024-04-25T08:53:30.715 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:53:30.715 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:30.722 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:30.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f5159439640 1 Processor -- start 2024-04-25T08:53:30.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f5159439640 1 -- start start 2024-04-25T08:53:30.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f5159439640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f515415a700 0x7f515415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f5159439640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540b3f50 0x7f51540cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f5159439640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51540b2090 con 0x7f51540ce430 2024-04-25T08:53:30.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f5159439640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f515415f200 con 0x7f515415a700 2024-04-25T08:53:30.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f5159439640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51540b43e0 con 0x7f51540b3f50 2024-04-25T08:53:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f514bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540b3f50 0x7f51540cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f5152ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f515415a700 0x7f515415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f514bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540b3f50 0x7f51540cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:54846/0 (socket says 172.21.15.37:54846) 2024-04-25T08:53:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f514bfff640 1 -- 172.21.15.37:0/1499913787 learned_addr learned my addr 172.21.15.37:0/1499913787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f51537fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f51540ce430 0x7f51540b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:55714/0 (socket says 172.21.15.37:55714) 2024-04-25T08:53:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f51527fc640 1 -- 172.21.15.37:0/1499913787 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2740035882 0 0) 0x7f51540b2090 con 0x7f51540ce430 2024-04-25T08:53:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f514bfff640 1 -- 172.21.15.37:0/1499913787 >> v2:172.21.15.203:3300/0 conn(0x7f515415a700 msgr2=0x7f515415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f514bfff640 1 --2- 172.21.15.37:0/1499913787 >> v2:172.21.15.203:3300/0 conn(0x7f515415a700 0x7f515415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f514bfff640 1 -- 172.21.15.37:0/1499913787 >> v1:172.21.15.203:6789/0 conn(0x7f51540ce430 legacy=0x7f51540b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.840+0000 7f514bfff640 1 -- 172.21.15.37:0/1499913787 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51540b46c0 con 0x7f51540b3f50 2024-04-25T08:53:30.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f514bfff640 1 --2- 172.21.15.37:0/1499913787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540b3f50 0x7f51540cdef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f513c009870 tx=0x7f513c01e620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=563a03991c9db445 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f51527fc640 1 -- 172.21.15.37:0/1499913787 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c0aa650 con 0x7f51540b3f50 2024-04-25T08:53:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f51527fc640 1 -- 172.21.15.37:0/1499913787 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513c0aa7f0 con 0x7f51540b3f50 2024-04-25T08:53:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f51527fc640 1 -- 172.21.15.37:0/1499913787 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513c0aabc0 con 0x7f51540b3f50 2024-04-25T08:53:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f5159439640 1 -- 172.21.15.37:0/1499913787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540b3f50 msgr2=0x7f51540cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f5159439640 1 --2- 172.21.15.37:0/1499913787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540b3f50 0x7f51540cdef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f513c009870 tx=0x7f513c01e620 comp rx=0 tx=0).stop 2024-04-25T08:53:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f5159439640 1 -- 172.21.15.37:0/1499913787 shutdown_connections 2024-04-25T08:53:30.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f5159439640 1 --2- 172.21.15.37:0/1499913787 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540b3f50 0x7f51540cdef0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f5159439640 1 --2- 172.21.15.37:0/1499913787 >> v2:172.21.15.203:3300/0 conn(0x7f515415a700 0x7f515415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f5159439640 1 -- 172.21.15.37:0/1499913787 >> 172.21.15.37:0/1499913787 conn(0x7f51540b11a0 msgr2=0x7f51540b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:30.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f5159439640 1 -- 172.21.15.37:0/1499913787 shutdown_connections 2024-04-25T08:53:30.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f5159439640 1 -- 172.21.15.37:0/1499913787 wait complete. 2024-04-25T08:53:30.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f5159439640 1 Processor -- start 2024-04-25T08:53:30.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.844+0000 7f5159439640 1 -- start start 2024-04-25T08:53:30.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5159439640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540ce430 0x7f5154159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5159439640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f515415a700 0x7f515414d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5159439640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515414fbb0 0x7f51541f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5159439640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51540cd490 con 0x7f515414fbb0 2024-04-25T08:53:30.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5159439640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51540cd310 con 0x7f515415a700 2024-04-25T08:53:30.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5159439640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51540b2bb0 con 0x7f51540ce430 2024-04-25T08:53:30.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f51537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515414fbb0 0x7f51541f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f51537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515414fbb0 0x7f51541f6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47626/0 (socket says 172.21.15.37:47626) 2024-04-25T08:53:30.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f51537fe640 1 -- 172.21.15.37:0/3248886847 learned_addr learned my addr 172.21.15.37:0/3248886847 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:30.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f514bfff640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f515415a700 0x7f515414d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5152ffd640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540ce430 0x7f5154159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f51537fe640 1 -- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540ce430 msgr2=0x7f5154159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f51537fe640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540ce430 0x7f5154159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f51537fe640 1 -- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f515415a700 msgr2=0x7f515414d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f51537fe640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f515415a700 0x7f515414d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f51537fe640 1 -- 172.21.15.37:0/3248886847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5154150090 con 0x7f515414fbb0 2024-04-25T08:53:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5152ffd640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540ce430 0x7f5154159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f514bfff640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f515415a700 0x7f515414d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f51537fe640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515414fbb0 0x7f51541f6780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c7a0 tx=0x7f514c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f514b7fe640 1 -- 172.21.15.37:0/3248886847 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c018070 con 0x7f515414fbb0 2024-04-25T08:53:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f514b7fe640 1 -- 172.21.15.37:0/3248886847 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514c00a640 con 0x7f515414fbb0 2024-04-25T08:53:30.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5159439640 1 -- 172.21.15.37:0/3248886847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51541503d0 con 0x7f515414fbb0 2024-04-25T08:53:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f514b7fe640 1 -- 172.21.15.37:0/3248886847 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514c00aab0 con 0x7f515414fbb0 2024-04-25T08:53:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5159439640 1 -- 172.21.15.37:0/3248886847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5154150c10 con 0x7f515414fbb0 2024-04-25T08:53:30.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5159439640 1 -- 172.21.15.37:0/3248886847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f515404efa0 con 0x7f515414fbb0 2024-04-25T08:53:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f514b7fe640 1 -- 172.21.15.37:0/3248886847 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f514c00ac50 con 0x7f515414fbb0 2024-04-25T08:53:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f514b7fe640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f60 0x7f5134069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f514b7fe640 1 -- 172.21.15.37:0/3248886847 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f514c084f70 con 0x7f515414fbb0 2024-04-25T08:53:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.848+0000 7f5152ffd640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f60 0x7f5134069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:30.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.852+0000 7f5152ffd640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f60 0x7f5134069420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5154164900 tx=0x7f5144008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:30.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:30.852+0000 7f514b7fe640 1 -- 172.21.15.37:0/3248886847 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f514c021020 con 0x7f515414fbb0 2024-04-25T08:53:31.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.012+0000 7f5159439640 1 -- 172.21.15.37:0/3248886847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5154157e70 con 0x7f515414fbb0 2024-04-25T08:53:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f514b7fe640 1 -- 172.21.15.37:0/3248886847 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f5154157e70 con 0x7f515414fbb0 2024-04-25T08:53:31.009 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:31.009 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 -- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f60 msgr2=0x7f5134069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f60 0x7f5134069420 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5154164900 tx=0x7f5144008040 comp rx=0 tx=0).stop 2024-04-25T08:53:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 -- 172.21.15.37:0/3248886847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515414fbb0 msgr2=0x7f51541f6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515414fbb0 0x7f51541f6780 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f514c00c7a0 tx=0x7f514c00cc60 comp rx=0 tx=0).stop 2024-04-25T08:53:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 -- 172.21.15.37:0/3248886847 shutdown_connections 2024-04-25T08:53:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f60 0x7f5134069420 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f515414fbb0 0x7f51541f6780 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f515415a700 0x7f515414d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 --2- 172.21.15.37:0/3248886847 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51540ce430 0x7f5154159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 -- 172.21.15.37:0/3248886847 >> 172.21.15.37:0/3248886847 conn(0x7f51540b11a0 msgr2=0x7f51540afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:31.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 -- 172.21.15.37:0/3248886847 shutdown_connections 2024-04-25T08:53:31.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.016+0000 7f5159439640 1 -- 172.21.15.37:0/3248886847 wait complete. 2024-04-25T08:53:31.029 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:31.029 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.156+0000 7f9ab20c1640 1 Processor -- start 2024-04-25T08:53:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9ab20c1640 1 -- start start 2024-04-25T08:53:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9ab20c1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac1634c0 0x7f9aac1658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9ab20c1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aac0ca4c0 con 0x7f9aac0cf590 2024-04-25T08:53:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9ab20c1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aac0cd690 con 0x7f9aac15ecb0 2024-04-25T08:53:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9ab20c1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0ca780 con 0x7f9aac1634c0 2024-04-25T08:53:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aab7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9aac0cf590 0x7f9aac0cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52468/0 (socket says 172.21.15.37:52468) 2024-04-25T08:53:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aab7fe640 1 -- 172.21.15.37:0/1297682728 learned_addr learned my addr 172.21.15.37:0/1297682728 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aabfff640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac1634c0 0x7f9aac1658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159604300 0 0) 0x7f9aac0ca4c0 con 0x7f9aac0cf590 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a94003530 con 0x7f9aac0cf590 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1327698954 0 0) 0x7f9a94003530 con 0x7f9aac0cf590 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac1634c0 msgr2=0x7f9aac1658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac1634c0 0x7f9aac1658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 >> v1:172.21.15.203:6789/0 conn(0x7f9aac15ecb0 legacy=0x7f9aac0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac0cd970 con 0x7f9aac0cf590 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f9a9c001a00 con 0x7f9aac0cf590 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 >> v1:172.21.15.37:6789/0 conn(0x7f9aac0cf590 legacy=0x7f9aac0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a94003a60 0x7f9a94003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94004810 0x7f9a94004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac1634c0 0x7f9a940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9400dce0 con 0x7f9a94004810 2024-04-25T08:53:31.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9400de90 con 0x7f9a94003a60 2024-04-25T08:53:31.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9400e090 con 0x7f9aac1634c0 2024-04-25T08:53:31.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaaffd640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94004810 0x7f9a94004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aabfff640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac1634c0 0x7f9a940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aab7fe640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a94003a60 0x7f9a94003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaaffd640 1 -- 172.21.15.37:0/1297682728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac1634c0 msgr2=0x7f9a940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:31.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaaffd640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac1634c0 0x7f9a940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaaffd640 1 -- 172.21.15.37:0/1297682728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a94003a60 msgr2=0x7f9a94003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:31.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaaffd640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a94003a60 0x7f9a94003e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.160+0000 7f9aaaffd640 1 -- 172.21.15.37:0/1297682728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a94003530 con 0x7f9a94004810 2024-04-25T08:53:31.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9aaaffd640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94004810 0x7f9a94004c90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0001780 tx=0x7f9aa001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e8671b6bf33660 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:31.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00013a0 con 0x7f9a94004810 2024-04-25T08:53:31.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa00052f0 con 0x7f9a94004810 2024-04-25T08:53:31.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9aaa7fc640 1 -- 172.21.15.37:0/1297682728 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0003840 con 0x7f9a94004810 2024-04-25T08:53:31.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 -- 172.21.15.37:0/1297682728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94004810 msgr2=0x7f9a94004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 --2- 172.21.15.37:0/1297682728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a94004810 0x7f9a94004c90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0001780 tx=0x7f9aa001c420 comp rx=0 tx=0).stop 2024-04-25T08:53:31.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9aabfff640 1 -- 172.21.15.37:0/1297682728 reap_dead start 2024-04-25T08:53:31.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 -- 172.21.15.37:0/1297682728 shutdown_connections 2024-04-25T08:53:31.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 -- 172.21.15.37:0/1297682728 >> 172.21.15.37:0/1297682728 conn(0x7f9aac06ae60 msgr2=0x7f9aac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:31.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 -- 172.21.15.37:0/1297682728 shutdown_connections 2024-04-25T08:53:31.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 -- 172.21.15.37:0/1297682728 wait complete. 2024-04-25T08:53:31.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 Processor -- start 2024-04-25T08:53:31.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 -- start start 2024-04-25T08:53:31.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a94004810 0x7f9aac155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9aac156240 0x7f9aac0ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac15b610 0x7f9aac0ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0cd440 con 0x7f9aac156240 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac15f0f0 con 0x7f9a94004810 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.164+0000 7f9ab20c1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0cdce0 con 0x7f9aac15b610 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aaaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9aac156240 0x7f9aac0ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aaaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9aac156240 0x7f9aac0ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47650/0 (socket says 172.21.15.37:47650) 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aaaffd640 1 -- 172.21.15.37:0/2238835145 learned_addr learned my addr 172.21.15.37:0/2238835145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aab7fe640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a94004810 0x7f9aac155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aabfff640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac15b610 0x7f9aac0ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aaaffd640 1 -- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac15b610 msgr2=0x7f9aac0ac6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aaaffd640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac15b610 0x7f9aac0ac6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aaaffd640 1 -- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a94004810 msgr2=0x7f9aac155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aaaffd640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a94004810 0x7f9aac155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aaaffd640 1 -- 172.21.15.37:0/2238835145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac156770 con 0x7f9aac156240 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aab7fe640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a94004810 0x7f9aac155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:31.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aaaffd640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9aac156240 0x7f9aac0ac170 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0005040 tx=0x7f9a94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:31.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9a93fff640 1 -- 172.21.15.37:0/2238835145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0007a40 con 0x7f9aac156240 2024-04-25T08:53:31.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9a93fff640 1 -- 172.21.15.37:0/2238835145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa00bb070 con 0x7f9aac156240 2024-04-25T08:53:31.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9ab20c1640 1 -- 172.21.15.37:0/2238835145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aac06d640 con 0x7f9aac156240 2024-04-25T08:53:31.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9a93fff640 1 -- 172.21.15.37:0/2238835145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa00013a0 con 0x7f9aac156240 2024-04-25T08:53:31.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9ab20c1640 1 -- 172.21.15.37:0/2238835145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aac06dc00 con 0x7f9aac156240 2024-04-25T08:53:31.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9ab20c1640 1 -- 172.21.15.37:0/2238835145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aac04efa0 con 0x7f9aac156240 2024-04-25T08:53:31.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9a93fff640 1 -- 172.21.15.37:0/2238835145 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9aa00b4040 con 0x7f9aac156240 2024-04-25T08:53:31.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9a93fff640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a98066f60 0x7f9a98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9a93fff640 1 -- 172.21.15.37:0/2238835145 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9aa011f7f0 con 0x7f9aac156240 2024-04-25T08:53:31.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aab7fe640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a98066f60 0x7f9a98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.168+0000 7f9aab7fe640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a98066f60 0x7f9a98069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c001340 tx=0x7f9a9c001920 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:31.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.172+0000 7f9a93fff640 1 -- 172.21.15.37:0/2238835145 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9aa0016130 con 0x7f9aac156240 2024-04-25T08:53:31.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.332+0000 7f9ab20c1640 1 -- 172.21.15.37:0/2238835145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9aac0b4170 con 0x7f9aac156240 2024-04-25T08:53:31.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.332+0000 7f9a93fff640 1 -- 172.21.15.37:0/2238835145 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9aa00f0400 con 0x7f9aac156240 2024-04-25T08:53:31.327 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 -- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a98066f60 msgr2=0x7f9a98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a98066f60 0x7f9a98069420 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c001340 tx=0x7f9a9c001920 comp rx=0 tx=0).stop 2024-04-25T08:53:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 -- 172.21.15.37:0/2238835145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9aac156240 msgr2=0x7f9aac0ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9aac156240 0x7f9aac0ac170 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0005040 tx=0x7f9a94005840 comp rx=0 tx=0).stop 2024-04-25T08:53:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 -- 172.21.15.37:0/2238835145 shutdown_connections 2024-04-25T08:53:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a98066f60 0x7f9a98069420 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9aac15b610 0x7f9aac0ac6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9aac156240 0x7f9aac0ac170 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 --2- 172.21.15.37:0/2238835145 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a94004810 0x7f9aac155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 -- 172.21.15.37:0/2238835145 >> 172.21.15.37:0/2238835145 conn(0x7f9aac06ae60 msgr2=0x7f9aac0c9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:31.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 -- 172.21.15.37:0/2238835145 shutdown_connections 2024-04-25T08:53:31.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.336+0000 7f9ab20c1640 1 -- 172.21.15.37:0/2238835145 wait complete. 2024-04-25T08:53:31.345 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:31.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.476+0000 7fbfeea1a640 1 Processor -- start 2024-04-25T08:53:31.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.476+0000 7fbfeea1a640 1 -- start start 2024-04-25T08:53:31.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.476+0000 7fbfeea1a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe80cfb80 0x7fbfe80cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.476+0000 7fbfeea1a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe815bd30 con 0x7fbfe809a480 2024-04-25T08:53:31.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.476+0000 7fbfeea1a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe80cc0d0 con 0x7fbfe815b950 2024-04-25T08:53:31.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.476+0000 7fbfeea1a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80d0510 con 0x7fbfe80cfb80 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.476+0000 7fbfe77fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbfe809a480 0x7fbfe80cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52478/0 (socket says 172.21.15.37:52478) 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.476+0000 7fbfe77fe640 1 -- 172.21.15.37:0/3555083873 learned_addr learned my addr 172.21.15.37:0/3555083873 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.476+0000 7fbfecfb7640 1 --2- 172.21.15.37:0/3555083873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe80cfb80 0x7fbfe80cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.476+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3370560820 0 0) 0x7fbfe815bd30 con 0x7fbfe809a480 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfd4003530 con 0x7fbfe809a480 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3879414818 0 0) 0x7fbfd4003530 con 0x7fbfe809a480 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe80cfb80 msgr2=0x7fbfe80cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 --2- 172.21.15.37:0/3555083873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe80cfb80 0x7fbfe80cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 >> v1:172.21.15.203:6789/0 conn(0x7fbfe815b950 legacy=0x7fbfe80cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe809ab10 con 0x7fbfe809a480 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fbfd8001a00 con 0x7fbfe809a480 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 >> v1:172.21.15.37:6789/0 conn(0x7fbfe809a480 legacy=0x7fbfe80cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 --2- 172.21.15.37:0/3555083873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe80cfb80 0x7fbfd4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 --2- 172.21.15.37:0/3555083873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4003fe0 0x7fbfd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 --2- 172.21.15.37:0/3555083873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfd4004d50 0x7fbfd40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80d0510 con 0x7fbfd4003fe0 2024-04-25T08:53:31.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd400bb30 con 0x7fbfd4004d50 2024-04-25T08:53:31.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd400bd30 con 0x7fbfe80cfb80 2024-04-25T08:53:31.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe77fe640 1 --2- 172.21.15.37:0/3555083873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4003fe0 0x7fbfd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfecfb7640 1 --2- 172.21.15.37:0/3555083873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfd4004d50 0x7fbfd40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe77fe640 1 -- 172.21.15.37:0/3555083873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe80cfb80 msgr2=0x7fbfd4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:31.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe77fe640 1 --2- 172.21.15.37:0/3555083873 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfe80cfb80 0x7fbfd4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe77fe640 1 -- 172.21.15.37:0/3555083873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfd4004d50 msgr2=0x7fbfd40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:31.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe77fe640 1 --2- 172.21.15.37:0/3555083873 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfd4004d50 0x7fbfd40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe77fe640 1 -- 172.21.15.37:0/3555083873 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd4003530 con 0x7fbfd4003fe0 2024-04-25T08:53:31.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe77fe640 1 --2- 172.21.15.37:0/3555083873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4003fe0 0x7fbfd4004810 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800a850 tx=0x7fbfd801fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db23b16c9c1f81c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:31.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd800a4a0 con 0x7fbfd4003fe0 2024-04-25T08:53:31.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfd80aa690 con 0x7fbfd4003fe0 2024-04-25T08:53:31.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfe6ffd640 1 -- 172.21.15.37:0/3555083873 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd80aaac0 con 0x7fbfd4003fe0 2024-04-25T08:53:31.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3555083873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4003fe0 msgr2=0x7fbfd4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfeea1a640 1 --2- 172.21.15.37:0/3555083873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4003fe0 0x7fbfd4004810 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800a850 tx=0x7fbfd801fa40 comp rx=0 tx=0).stop 2024-04-25T08:53:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfecfb7640 1 -- 172.21.15.37:0/3555083873 reap_dead start 2024-04-25T08:53:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3555083873 shutdown_connections 2024-04-25T08:53:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3555083873 >> 172.21.15.37:0/3555083873 conn(0x7fbfe80b30b0 msgr2=0x7fbfe80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3555083873 shutdown_connections 2024-04-25T08:53:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.480+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3555083873 wait complete. 2024-04-25T08:53:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfeea1a640 1 Processor -- start 2024-04-25T08:53:31.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfeea1a640 1 -- start start 2024-04-25T08:53:31.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfeea1a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfd4003fe0 0x7fbfe8148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfeea1a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe80c6140 0x7fbfe80c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfeea1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe80c6580 0x7fbfe80c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfeea1a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80cc930 con 0x7fbfe80c6580 2024-04-25T08:53:31.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfeea1a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80cc650 con 0x7fbfe80c6140 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfeea1a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe80cffd0 con 0x7fbfd4003fe0 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfecfb7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe80c6580 0x7fbfe80c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfecfb7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe80c6580 0x7fbfe80c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47676/0 (socket says 172.21.15.37:47676) 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfecfb7640 1 -- 172.21.15.37:0/3768618314 learned_addr learned my addr 172.21.15.37:0/3768618314 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfe7fff640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfd4003fe0 0x7fbfe8148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfe77fe640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe80c6140 0x7fbfe80c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfecfb7640 1 -- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfd4003fe0 msgr2=0x7fbfe8148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfecfb7640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfd4003fe0 0x7fbfe8148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfecfb7640 1 -- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe80c6140 msgr2=0x7fbfe80c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfecfb7640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe80c6140 0x7fbfe80c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfecfb7640 1 -- 172.21.15.37:0/3768618314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe8148bd0 con 0x7fbfe80c6580 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfecfb7640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe80c6580 0x7fbfe80c8bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8072430 tx=0x7fbfe0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfe4ff9640 1 -- 172.21.15.37:0/3768618314 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0015930 con 0x7fbfe80c6580 2024-04-25T08:53:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfe4ff9640 1 -- 172.21.15.37:0/3768618314 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe001c070 con 0x7fbfe80c6580 2024-04-25T08:53:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3768618314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe80c6b20 con 0x7fbfe80c6580 2024-04-25T08:53:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfe4ff9640 1 -- 172.21.15.37:0/3768618314 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe0015d10 con 0x7fbfe80c6580 2024-04-25T08:53:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3768618314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe80c70e0 con 0x7fbfe80c6580 2024-04-25T08:53:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.484+0000 7fbfe4ff9640 1 -- 172.21.15.37:0/3768618314 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbfe0020410 con 0x7fbfe80c6580 2024-04-25T08:53:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.488+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3768618314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe804efa0 con 0x7fbfe80c6580 2024-04-25T08:53:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.488+0000 7fbfe4ff9640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbfc8066f90 0x7fbfc8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.488+0000 7fbfe4ff9640 1 -- 172.21.15.37:0/3768618314 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fbfe00851e0 con 0x7fbfe80c6580 2024-04-25T08:53:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.488+0000 7fbfe7fff640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbfc8066f90 0x7fbfc8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.488+0000 7fbfe7fff640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbfc8066f90 0x7fbfc8069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0001170 tx=0x7fbfd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.492+0000 7fbfe4ff9640 1 -- 172.21.15.37:0/3768618314 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbfe0025020 con 0x7fbfe80c6580 2024-04-25T08:53:31.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.648+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3768618314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbfe80b3ec0 con 0x7fbfe80c6580 2024-04-25T08:53:31.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfe4ff9640 1 -- 172.21.15.37:0/3768618314 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbfe0055d40 con 0x7fbfe80c6580 2024-04-25T08:53:31.644 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:31.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbfc8066f90 msgr2=0x7fbfc8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbfc8066f90 0x7fbfc8069450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0001170 tx=0x7fbfd0009040 comp rx=0 tx=0).stop 2024-04-25T08:53:31.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3768618314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe80c6580 msgr2=0x7fbfe80c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe80c6580 0x7fbfe80c8bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fbfe8072430 tx=0x7fbfe0011920 comp rx=0 tx=0).stop 2024-04-25T08:53:31.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3768618314 shutdown_connections 2024-04-25T08:53:31.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbfc8066f90 0x7fbfc8069450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfe80c6580 0x7fbfe80c8bd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbfe80c6140 0x7fbfe80c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 --2- 172.21.15.37:0/3768618314 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbfd4003fe0 0x7fbfe8148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3768618314 >> 172.21.15.37:0/3768618314 conn(0x7fbfe80b30b0 msgr2=0x7fbfe80b23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:31.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3768618314 shutdown_connections 2024-04-25T08:53:31.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.652+0000 7fbfeea1a640 1 -- 172.21.15.37:0/3768618314 wait complete. 2024-04-25T08:53:31.662 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.784+0000 7ff131ef6640 1 Processor -- start 2024-04-25T08:53:31.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.784+0000 7ff131ef6640 1 -- start start 2024-04-25T08:53:31.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff131ef6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c0b2980 0x7ff12c15c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff131ef6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff12c06ac90 0x7ff12c06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff131ef6640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12c0cfbb0 con 0x7ff12c0cc950 2024-04-25T08:53:31.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff131ef6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12c06b600 con 0x7ff12c06ac90 2024-04-25T08:53:31.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff131ef6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c0cfe60 con 0x7ff12c0b2980 2024-04-25T08:53:31.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff12c0cc950 0x7ff12c0cf4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52492/0 (socket says 172.21.15.37:52492) 2024-04-25T08:53:31.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12b7fe640 1 -- 172.21.15.37:0/1665405459 learned_addr learned my addr 172.21.15.37:0/1665405459 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:31.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12bfff640 1 --2- 172.21.15.37:0/1665405459 >> v2:172.21.15.203:3300/0 conn(0x7ff12c06ac90 0x7ff12c06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff122ffd640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c0b2980 0x7ff12c15c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843986200 0 0) 0x7ff12c0cfbb0 con 0x7ff12c0cc950 2024-04-25T08:53:31.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff118003530 con 0x7ff12c0cc950 2024-04-25T08:53:31.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3212977251 0 0) 0x7ff118003530 con 0x7ff12c0cc950 2024-04-25T08:53:31.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c0b2980 msgr2=0x7ff12c15c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c0b2980 0x7ff12c15c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 >> v2:172.21.15.203:3300/0 conn(0x7ff12c06ac90 msgr2=0x7ff12c06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 --2- 172.21.15.37:0/1665405459 >> v2:172.21.15.203:3300/0 conn(0x7ff12c06ac90 0x7ff12c06b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c0d0140 con 0x7ff12c0cc950 2024-04-25T08:53:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7ff11c001680 con 0x7ff12c0cc950 2024-04-25T08:53:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 >> v1:172.21.15.37:6789/0 conn(0x7ff12c0cc950 legacy=0x7ff12c0cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff118003a30 0x7ff118003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 0x7ff118004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c0b2980 0x7ff1180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c06b600 con 0x7ff118004810 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c0cfe60 con 0x7ff118003a30 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11800bb00 con 0x7ff12c0b2980 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff122ffd640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 0x7ff118004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12b7fe640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff118003a30 0x7ff118003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12bfff640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c0b2980 0x7ff1180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff122ffd640 1 -- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c0b2980 msgr2=0x7ff1180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff122ffd640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c0b2980 0x7ff1180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff122ffd640 1 -- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff118003a30 msgr2=0x7ff118003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff122ffd640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff118003a30 0x7ff118003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff122ffd640 1 -- 172.21.15.37:0/1665405459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118003530 con 0x7ff118004810 2024-04-25T08:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12bfff640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c0b2980 0x7ff1180051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:31.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff122ffd640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 0x7ff118004c90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff114009ab0 tx=0x7ff11401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca51851432ad13cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:31.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1140b2070 con 0x7ff118004810 2024-04-25T08:53:31.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1140a5620 con 0x7ff118004810 2024-04-25T08:53:31.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.788+0000 7ff12affd640 1 -- 172.21.15.37:0/1665405459 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1140096b0 con 0x7ff118004810 2024-04-25T08:53:31.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 -- 172.21.15.37:0/1665405459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 msgr2=0x7ff118004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 --2- 172.21.15.37:0/1665405459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 0x7ff118004c90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff114009ab0 tx=0x7ff11401ca30 comp rx=0 tx=0).stop 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12bfff640 1 -- 172.21.15.37:0/1665405459 reap_dead start 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 -- 172.21.15.37:0/1665405459 shutdown_connections 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 -- 172.21.15.37:0/1665405459 >> 172.21.15.37:0/1665405459 conn(0x7ff12c099a70 msgr2=0x7ff12c0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 -- 172.21.15.37:0/1665405459 shutdown_connections 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 -- 172.21.15.37:0/1665405459 wait complete. 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 Processor -- start 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 -- start start 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 0x7ff12c1f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c15e8a0 0x7ff12c1595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c15ed00 0x7ff12c159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c15f9e0 con 0x7ff118004810 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c15bc00 con 0x7ff12c15e8a0 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c15fb60 con 0x7ff12c15ed00 2024-04-25T08:53:31.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 0x7ff12c1f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 0x7ff12c1f9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47686/0 (socket says 172.21.15.37:47686) 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12b7fe640 1 -- 172.21.15.37:0/2651934611 learned_addr learned my addr 172.21.15.37:0/2651934611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff122ffd640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c15e8a0 0x7ff12c1595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12bfff640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c15ed00 0x7ff12c159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12b7fe640 1 -- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c15ed00 msgr2=0x7ff12c159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12b7fe640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c15ed00 0x7ff12c159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12b7fe640 1 -- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c15e8a0 msgr2=0x7ff12c1595d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12b7fe640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c15e8a0 0x7ff12c1595d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12b7fe640 1 -- 172.21.15.37:0/2651934611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c15d800 con 0x7ff118004810 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff122ffd640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c15e8a0 0x7ff12c1595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12bfff640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c15ed00 0x7ff12c159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff12b7fe640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 0x7ff12c1f9ee0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e7e0 tx=0x7ff11c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff128ff9640 1 -- 172.21.15.37:0/2651934611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00c660 con 0x7ff118004810 2024-04-25T08:53:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff128ff9640 1 -- 172.21.15.37:0/2651934611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11c018070 con 0x7ff118004810 2024-04-25T08:53:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff128ff9640 1 -- 172.21.15.37:0/2651934611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00ca60 con 0x7ff118004810 2024-04-25T08:53:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 -- 172.21.15.37:0/2651934611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c15dae0 con 0x7ff118004810 2024-04-25T08:53:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.792+0000 7ff131ef6640 1 -- 172.21.15.37:0/2651934611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c15e0a0 con 0x7ff118004810 2024-04-25T08:53:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.796+0000 7ff131ef6640 1 -- 172.21.15.37:0/2651934611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c04efa0 con 0x7ff118004810 2024-04-25T08:53:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.796+0000 7ff128ff9640 1 -- 172.21.15.37:0/2651934611 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff11c00cc00 con 0x7ff118004810 2024-04-25T08:53:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.796+0000 7ff128ff9640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff10c066f90 0x7ff10c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.796+0000 7ff128ff9640 1 -- 172.21.15.37:0/2651934611 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff11c084d40 con 0x7ff118004810 2024-04-25T08:53:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.796+0000 7ff122ffd640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff10c066f90 0x7ff10c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.796+0000 7ff122ffd640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff10c066f90 0x7ff10c069450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff1140af000 tx=0x7ff1140046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.800+0000 7ff128ff9640 1 -- 172.21.15.37:0/2651934611 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff11c055950 con 0x7ff118004810 2024-04-25T08:53:31.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.952+0000 7ff131ef6640 1 -- 172.21.15.37:0/2651934611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff12c0680d0 con 0x7ff118004810 2024-04-25T08:53:31.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.952+0000 7ff128ff9640 1 -- 172.21.15.37:0/2651934611 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff11c055250 con 0x7ff118004810 2024-04-25T08:53:31.947 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:31.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 -- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff10c066f90 msgr2=0x7ff10c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff10c066f90 0x7ff10c069450 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ff1140af000 tx=0x7ff1140046d0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 -- 172.21.15.37:0/2651934611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 msgr2=0x7ff12c1f9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:31.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 0x7ff12c1f9ee0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e7e0 tx=0x7ff11c00eca0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 -- 172.21.15.37:0/2651934611 shutdown_connections 2024-04-25T08:53:31.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff10c066f90 0x7ff10c069450 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff12c15ed00 0x7ff12c159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff12c15e8a0 0x7ff12c1595d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 --2- 172.21.15.37:0/2651934611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff118004810 0x7ff12c1f9ee0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:31.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 -- 172.21.15.37:0/2651934611 >> 172.21.15.37:0/2651934611 conn(0x7ff12c099a70 msgr2=0x7ff12c0cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:31.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 -- 172.21.15.37:0/2651934611 shutdown_connections 2024-04-25T08:53:31.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:31.956+0000 7ff131ef6640 1 -- 172.21.15.37:0/2651934611 wait complete. 2024-04-25T08:53:31.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:32.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d9b88640 1 Processor -- start 2024-04-25T08:53:32.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d9b88640 1 -- start start 2024-04-25T08:53:32.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d9b88640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd9d40cf590 0x7fd9d40cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d9b88640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd9d4072000 0x7fd9d4072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d9b88640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d40cbd20 con 0x7fd9d415b6b0 2024-04-25T08:53:32.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d9b88640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d415ba90 con 0x7fd9d40cf590 2024-04-25T08:53:32.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d9b88640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d40cc000 con 0x7fd9d4072000 2024-04-25T08:53:32.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d37fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd9d415b6b0 0x7fd9d40ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52506/0 (socket says 172.21.15.37:52506) 2024-04-25T08:53:32.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d37fe640 1 -- 172.21.15.37:0/3948806800 learned_addr learned my addr 172.21.15.37:0/3948806800 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:32.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d2ffd640 1 --2- 172.21.15.37:0/3948806800 >> v2:172.21.15.37:3300/0 conn(0x7fd9d40cf590 0x7fd9d40cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d3fff640 1 --2- 172.21.15.37:0/3948806800 >> v2:172.21.15.203:3300/0 conn(0x7fd9d4072000 0x7fd9d4072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d27fc640 1 -- 172.21.15.37:0/3948806800 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803013497 0 0) 0x7fd9d40cbd20 con 0x7fd9d415b6b0 2024-04-25T08:53:32.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d27fc640 1 -- 172.21.15.37:0/3948806800 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9bc003530 con 0x7fd9d415b6b0 2024-04-25T08:53:32.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d2ffd640 1 -- 172.21.15.37:0/3948806800 >> v2:172.21.15.203:3300/0 conn(0x7fd9d4072000 msgr2=0x7fd9d4072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:32.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d2ffd640 1 --2- 172.21.15.37:0/3948806800 >> v2:172.21.15.203:3300/0 conn(0x7fd9d4072000 0x7fd9d4072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d2ffd640 1 -- 172.21.15.37:0/3948806800 >> v1:172.21.15.37:6789/0 conn(0x7fd9d415b6b0 legacy=0x7fd9d40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.092+0000 7fd9d2ffd640 1 -- 172.21.15.37:0/3948806800 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40cc2e0 con 0x7fd9d40cf590 2024-04-25T08:53:32.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d2ffd640 1 --2- 172.21.15.37:0/3948806800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d40cf590 0x7fd9d40cfa20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd9d406f000 tx=0x7fd9c801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefcd18b7451b582 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:32.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d27fc640 1 -- 172.21.15.37:0/3948806800 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80b0070 con 0x7fd9d40cf590 2024-04-25T08:53:32.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d27fc640 1 -- 172.21.15.37:0/3948806800 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c80ab600 con 0x7fd9d40cf590 2024-04-25T08:53:32.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d27fc640 1 -- 172.21.15.37:0/3948806800 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80ab9d0 con 0x7fd9d40cf590 2024-04-25T08:53:32.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 -- 172.21.15.37:0/3948806800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d40cf590 msgr2=0x7fd9d40cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 --2- 172.21.15.37:0/3948806800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d40cf590 0x7fd9d40cfa20 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fd9d406f000 tx=0x7fd9c801c480 comp rx=0 tx=0).stop 2024-04-25T08:53:32.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 -- 172.21.15.37:0/3948806800 shutdown_connections 2024-04-25T08:53:32.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 --2- 172.21.15.37:0/3948806800 >> v2:172.21.15.203:3300/0 conn(0x7fd9d4072000 0x7fd9d4072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 --2- 172.21.15.37:0/3948806800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d40cf590 0x7fd9d40cfa20 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 -- 172.21.15.37:0/3948806800 >> 172.21.15.37:0/3948806800 conn(0x7fd9d406ae60 msgr2=0x7fd9d406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:32.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 -- 172.21.15.37:0/3948806800 shutdown_connections 2024-04-25T08:53:32.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 -- 172.21.15.37:0/3948806800 wait complete. 2024-04-25T08:53:32.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 Processor -- start 2024-04-25T08:53:32.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 -- start start 2024-04-25T08:53:32.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9d4072000 0x7fd9d4098a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d40cf590 0x7fd9d41508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415b6b0 0x7fd9d415a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40caf30 con 0x7fd9d415b6b0 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d40cadb0 con 0x7fd9d40cf590 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d41682a0 con 0x7fd9d4072000 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415b6b0 0x7fd9d415a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415b6b0 0x7fd9d415a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47704/0 (socket says 172.21.15.37:47704) 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d3fff640 1 -- 172.21.15.37:0/810711264 learned_addr learned my addr 172.21.15.37:0/810711264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d2ffd640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d40cf590 0x7fd9d41508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d3fff640 1 -- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9d4072000 msgr2=0x7fd9d4098a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d37fe640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9d4072000 0x7fd9d4098a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d3fff640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9d4072000 0x7fd9d4098a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d3fff640 1 -- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d40cf590 msgr2=0x7fd9d41508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d3fff640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d40cf590 0x7fd9d41508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d3fff640 1 -- 172.21.15.37:0/810711264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d40975b0 con 0x7fd9d415b6b0 2024-04-25T08:53:32.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d2ffd640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d40cf590 0x7fd9d41508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:32.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d37fe640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9d4072000 0x7fd9d4098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:32.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d3fff640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415b6b0 0x7fd9d415a070 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009dd0 tx=0x7fd9cc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:32.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9bbfff640 1 -- 172.21.15.37:0/810711264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc018020 con 0x7fd9d415b6b0 2024-04-25T08:53:32.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9bbfff640 1 -- 172.21.15.37:0/810711264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9cc014480 con 0x7fd9d415b6b0 2024-04-25T08:53:32.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 -- 172.21.15.37:0/810711264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d4097890 con 0x7fd9d415b6b0 2024-04-25T08:53:32.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9bbfff640 1 -- 172.21.15.37:0/810711264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc014870 con 0x7fd9d415b6b0 2024-04-25T08:53:32.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.096+0000 7fd9d9b88640 1 -- 172.21.15.37:0/810711264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d4097e50 con 0x7fd9d415b6b0 2024-04-25T08:53:32.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.100+0000 7fd9d9b88640 1 -- 172.21.15.37:0/810711264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d404efa0 con 0x7fd9d415b6b0 2024-04-25T08:53:32.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.100+0000 7fd9bbfff640 1 -- 172.21.15.37:0/810711264 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9cc01b020 con 0x7fd9d415b6b0 2024-04-25T08:53:32.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.100+0000 7fd9bbfff640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9c0066f60 0x7fd9c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.100+0000 7fd9bbfff640 1 -- 172.21.15.37:0/810711264 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd9cc085fe0 con 0x7fd9d415b6b0 2024-04-25T08:53:32.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.100+0000 7fd9d37fe640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9c0066f60 0x7fd9c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.100+0000 7fd9d37fe640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9c0066f60 0x7fd9c0069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4005890 tx=0x7fd9c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:32.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.104+0000 7fd9bbfff640 1 -- 172.21.15.37:0/810711264 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9cc022070 con 0x7fd9d415b6b0 2024-04-25T08:53:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.256+0000 7fd9d9b88640 1 -- 172.21.15.37:0/810711264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fd9d40724d0 con 0x7fd9d415b6b0 2024-04-25T08:53:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.256+0000 7fd9bbfff640 1 -- 172.21.15.37:0/810711264 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd9cc010040 con 0x7fd9d415b6b0 2024-04-25T08:53:32.251 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:32.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 -- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9c0066f60 msgr2=0x7fd9c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9c0066f60 0x7fd9c0069420 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4005890 tx=0x7fd9c4009040 comp rx=0 tx=0).stop 2024-04-25T08:53:32.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 -- 172.21.15.37:0/810711264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415b6b0 msgr2=0x7fd9d415a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415b6b0 0x7fd9d415a070 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc009dd0 tx=0x7fd9cc00c490 comp rx=0 tx=0).stop 2024-04-25T08:53:32.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 -- 172.21.15.37:0/810711264 shutdown_connections 2024-04-25T08:53:32.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9c0066f60 0x7fd9c0069420 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d415b6b0 0x7fd9d415a070 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d40cf590 0x7fd9d41508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 --2- 172.21.15.37:0/810711264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9d4072000 0x7fd9d4098a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 -- 172.21.15.37:0/810711264 >> 172.21.15.37:0/810711264 conn(0x7fd9d406ae60 msgr2=0x7fd9d41558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:32.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 -- 172.21.15.37:0/810711264 shutdown_connections 2024-04-25T08:53:32.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.260+0000 7fd9d9b88640 1 -- 172.21.15.37:0/810711264 wait complete. 2024-04-25T08:53:32.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e3473c640 1 Processor -- start 2024-04-25T08:53:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e3473c640 1 -- start start 2024-04-25T08:53:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e3473c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c0c6a10 0x7f6e2c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e3473c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c0d31a0 con 0x7f6e2c0c6df0 2024-04-25T08:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e3473c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e2c0d3480 con 0x7f6e2c0d2a90 2024-04-25T08:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e3473c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0d1610 con 0x7f6e2c0c6a10 2024-04-25T08:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e31cd7640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6e2c0c6df0 0x7f6e2c0d1aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52520/0 (socket says 172.21.15.37:52520) 2024-04-25T08:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e31cd7640 1 -- 172.21.15.37:0/3188196781 learned_addr learned my addr 172.21.15.37:0/3188196781 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e324d8640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c0c6a10 0x7f6e2c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35091653 0 0) 0x7f6e2c0d31a0 con 0x7f6e2c0c6df0 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e18003530 con 0x7f6e2c0c6df0 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 454488840 0 0) 0x7f6e18003530 con 0x7f6e2c0c6df0 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c0c6a10 msgr2=0x7f6e2c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c0c6a10 0x7f6e2c0cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 >> v1:172.21.15.203:6789/0 conn(0x7f6e2c0d2a90 legacy=0x7f6e2c0b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c0d2560 con 0x7f6e2c0c6df0 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f6e1c001680 con 0x7f6e2c0c6df0 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 >> v1:172.21.15.37:6789/0 conn(0x7f6e2c0c6df0 legacy=0x7f6e2c0d1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c0c6a10 0x7f6e18003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e18003fe0 0x7f6e18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 0x7f6e180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1800bb00 con 0x7f6e18004d50 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1800bcb0 con 0x7f6e18003fe0 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1800beb0 con 0x7f6e2c0c6a10 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e32cd9640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 0x7f6e180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e324d8640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c0c6a10 0x7f6e18003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e31cd7640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e18003fe0 0x7f6e18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e32cd9640 1 -- 172.21.15.37:0/3188196781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c0c6a10 msgr2=0x7f6e18003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e32cd9640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c0c6a10 0x7f6e18003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e32cd9640 1 -- 172.21.15.37:0/3188196781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e18003fe0 msgr2=0x7f6e18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e32cd9640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e18003fe0 0x7f6e18004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.400+0000 7f6e32cd9640 1 -- 172.21.15.37:0/3188196781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e18003530 con 0x7f6e18004d50 2024-04-25T08:53:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.404+0000 7f6e32cd9640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 0x7f6e180051d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6e28001780 tx=0x7f6e2801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc39dc2ea46c286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.404+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e280013a0 con 0x7f6e18004d50 2024-04-25T08:53:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.404+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e280a9070 con 0x7f6e18004d50 2024-04-25T08:53:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.404+0000 7f6e314d6640 1 -- 172.21.15.37:0/3188196781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e28003600 con 0x7f6e18004d50 2024-04-25T08:53:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.404+0000 7f6e3473c640 1 -- 172.21.15.37:0/3188196781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 msgr2=0x7f6e180051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.404+0000 7f6e3473c640 1 --2- 172.21.15.37:0/3188196781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 0x7f6e180051d0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6e28001780 tx=0x7f6e2801c420 comp rx=0 tx=0).stop 2024-04-25T08:53:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.404+0000 7f6e32cd9640 1 -- 172.21.15.37:0/3188196781 reap_dead start 2024-04-25T08:53:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.404+0000 7f6e3473c640 1 -- 172.21.15.37:0/3188196781 shutdown_connections 2024-04-25T08:53:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.404+0000 7f6e3473c640 1 -- 172.21.15.37:0/3188196781 >> 172.21.15.37:0/3188196781 conn(0x7f6e2c08b000 msgr2=0x7f6e2c0d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.404+0000 7f6e3473c640 1 -- 172.21.15.37:0/3188196781 shutdown_connections 2024-04-25T08:53:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.404+0000 7f6e3473c640 1 -- 172.21.15.37:0/3188196781 wait complete. 2024-04-25T08:53:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e3473c640 1 Processor -- start 2024-04-25T08:53:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e3473c640 1 -- start start 2024-04-25T08:53:32.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e3473c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 0x7f6e2c1646d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e3473c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c157fe0 0x7f6e2c164c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e3473c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e2c0b7070 0x7f6e2c0b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e3473c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0d1330 con 0x7f6e18004d50 2024-04-25T08:53:32.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e3473c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0d23c0 con 0x7f6e2c0b7070 2024-04-25T08:53:32.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e3473c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2c0b5980 con 0x7f6e2c157fe0 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e324d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 0x7f6e2c1646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e324d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 0x7f6e2c1646d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47718/0 (socket says 172.21.15.37:47718) 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e324d8640 1 -- 172.21.15.37:0/3568810358 learned_addr learned my addr 172.21.15.37:0/3568810358 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e31cd7640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c157fe0 0x7f6e2c164c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e32cd9640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e2c0b7070 0x7f6e2c0b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e324d8640 1 -- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c157fe0 msgr2=0x7f6e2c164c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e324d8640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c157fe0 0x7f6e2c164c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e324d8640 1 -- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e2c0b7070 msgr2=0x7f6e2c0b77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e324d8640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e2c0b7070 0x7f6e2c0b77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e324d8640 1 -- 172.21.15.37:0/3568810358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e2c0b8770 con 0x7f6e18004d50 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e324d8640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 0x7f6e2c1646d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e24007f30 tx=0x7f6e24011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e22ffd640 1 -- 172.21.15.37:0/3568810358 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2401a020 con 0x7f6e18004d50 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e22ffd640 1 -- 172.21.15.37:0/3568810358 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e24016470 con 0x7f6e18004d50 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e22ffd640 1 -- 172.21.15.37:0/3568810358 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24016840 con 0x7f6e18004d50 2024-04-25T08:53:32.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e3473c640 1 -- 172.21.15.37:0/3568810358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2c0b8490 con 0x7f6e18004d50 2024-04-25T08:53:32.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e3473c640 1 -- 172.21.15.37:0/3568810358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2c0b6870 con 0x7f6e18004d50 2024-04-25T08:53:32.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.408+0000 7f6e3473c640 1 -- 172.21.15.37:0/3568810358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e2c04efa0 con 0x7f6e18004d50 2024-04-25T08:53:32.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.412+0000 7f6e22ffd640 1 -- 172.21.15.37:0/3568810358 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6e2401d020 con 0x7f6e18004d50 2024-04-25T08:53:32.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.412+0000 7f6e22ffd640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6e10066f90 0x7f6e10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.412+0000 7f6e22ffd640 1 -- 172.21.15.37:0/3568810358 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6e240861d0 con 0x7f6e18004d50 2024-04-25T08:53:32.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.412+0000 7f6e31cd7640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6e10066f90 0x7f6e10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.412+0000 7f6e31cd7640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6e10066f90 0x7f6e10069450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c001060 tx=0x7f6e1c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:32.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.416+0000 7f6e22ffd640 1 -- 172.21.15.37:0/3568810358 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6e24024030 con 0x7f6e18004d50 2024-04-25T08:53:32.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 -- 172.21.15.37:0/3568810358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6e2c0b5f50 con 0x7f6e18004d50 2024-04-25T08:53:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e22ffd640 1 -- 172.21.15.37:0/3568810358 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6e24056e70 con 0x7f6e18004d50 2024-04-25T08:53:32.584 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:53:32.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 -- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6e10066f90 msgr2=0x7f6e10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6e10066f90 0x7f6e10069450 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c001060 tx=0x7f6e1c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 -- 172.21.15.37:0/3568810358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 msgr2=0x7f6e2c1646d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 0x7f6e2c1646d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f6e24007f30 tx=0x7f6e24011660 comp rx=0 tx=0).stop 2024-04-25T08:53:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 -- 172.21.15.37:0/3568810358 shutdown_connections 2024-04-25T08:53:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6e10066f90 0x7f6e10069450 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6e2c0b7070 0x7f6e2c0b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6e2c157fe0 0x7f6e2c164c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 --2- 172.21.15.37:0/3568810358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6e18004d50 0x7f6e2c1646d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 -- 172.21.15.37:0/3568810358 >> 172.21.15.37:0/3568810358 conn(0x7f6e2c08b000 msgr2=0x7f6e2c160d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 -- 172.21.15.37:0/3568810358 shutdown_connections 2024-04-25T08:53:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.592+0000 7f6e3473c640 1 -- 172.21.15.37:0/3568810358 wait complete. 2024-04-25T08:53:32.604 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi203.front.sepia.ceph.com... 2024-04-25T08:53:32.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:32.604 DEBUG:teuthology.orchestra.run.smithi203:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T08:53:32.633 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:32.633 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:53:32.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:32.634 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:53:32.684 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:53:32.684 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:32.690 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:53:32.690 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:32.736 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:32.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.912+0000 7f5aeaca8640 1 Processor -- start 2024-04-25T08:53:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.912+0000 7f5aeaca8640 1 -- start start 2024-04-25T08:53:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5aeaca8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5ae40b3f50 0x7f5ae415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5aeaca8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae40b2090 con 0x7f5ae40b24a0 2024-04-25T08:53:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5aeaca8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae40b43e0 con 0x7f5ae415a700 2024-04-25T08:53:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5aeaca8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae415f200 con 0x7f5ae40b3f50 2024-04-25T08:53:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5ae3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5ae40b3f50 0x7f5ae415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5ae3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5ae40b3f50 0x7f5ae415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47732/0 (socket says 172.21.15.37:47732) 2024-04-25T08:53:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5ae3fff640 1 -- 172.21.15.37:0/3032695620 learned_addr learned my addr 172.21.15.37:0/3032695620 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:32.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5ae3fff640 1 -- 172.21.15.37:0/3032695620 >> v1:172.21.15.203:6789/0 conn(0x7f5ae415a700 legacy=0x7f5ae40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5ae3fff640 1 -- 172.21.15.37:0/3032695620 >> v1:172.21.15.37:6789/0 conn(0x7f5ae40b24a0 legacy=0x7f5ae406e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5ae3fff640 1 -- 172.21.15.37:0/3032695620 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae40b46c0 con 0x7f5ae40b3f50 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5ae3fff640 1 --2- 172.21.15.37:0/3032695620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae40b3f50 0x7f5ae415ecc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009870 tx=0x7f5ad001e5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=386dac266ecc33cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5ae37fe640 1 -- 172.21.15.37:0/3032695620 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00ac650 con 0x7f5ae40b3f50 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5ae37fe640 1 -- 172.21.15.37:0/3032695620 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad00b1070 con 0x7f5ae40b3f50 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.916+0000 7f5ae37fe640 1 -- 172.21.15.37:0/3032695620 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad00aca10 con 0x7f5ae40b3f50 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 -- 172.21.15.37:0/3032695620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae40b3f50 msgr2=0x7f5ae415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 --2- 172.21.15.37:0/3032695620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae40b3f50 0x7f5ae415ecc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009870 tx=0x7f5ad001e5e0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 -- 172.21.15.37:0/3032695620 shutdown_connections 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 --2- 172.21.15.37:0/3032695620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae40b3f50 0x7f5ae415ecc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 -- 172.21.15.37:0/3032695620 >> 172.21.15.37:0/3032695620 conn(0x7f5ae40b11a0 msgr2=0x7f5ae40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 -- 172.21.15.37:0/3032695620 shutdown_connections 2024-04-25T08:53:32.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 -- 172.21.15.37:0/3032695620 wait complete. 2024-04-25T08:53:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 Processor -- start 2024-04-25T08:53:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 -- start start 2024-04-25T08:53:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae40b24a0 0x7f5ae40add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae415a700 0x7f5ae413d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae40ae250 0x7f5ae413de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae40cd310 con 0x7f5ae40ae250 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae40ce600 con 0x7f5ae40b24a0 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae40cd490 con 0x7f5ae415a700 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae9245640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae40ae250 0x7f5ae413de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae9245640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae40ae250 0x7f5ae413de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47744/0 (socket says 172.21.15.37:47744) 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae9245640 1 -- 172.21.15.37:0/2980443527 learned_addr learned my addr 172.21.15.37:0/2980443527 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae8a44640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae40b24a0 0x7f5ae40add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae9245640 1 -- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae415a700 msgr2=0x7f5ae413d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae9245640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae415a700 0x7f5ae413d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae9245640 1 -- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae40b24a0 msgr2=0x7f5ae40add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae9245640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae40b24a0 0x7f5ae40add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae9245640 1 -- 172.21.15.37:0/2980443527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae40cbbd0 con 0x7f5ae40ae250 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae8a44640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae40b24a0 0x7f5ae40add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae9245640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae40ae250 0x7f5ae413de30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00c730 tx=0x7f5adc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:32.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae17fa640 1 -- 172.21.15.37:0/2980443527 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc003ba0 con 0x7f5ae40ae250 2024-04-25T08:53:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae17fa640 1 -- 172.21.15.37:0/2980443527 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc003d40 con 0x7f5ae40ae250 2024-04-25T08:53:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 -- 172.21.15.37:0/2980443527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae40aca10 con 0x7f5ae40ae250 2024-04-25T08:53:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5ae17fa640 1 -- 172.21.15.37:0/2980443527 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc005310 con 0x7f5ae40ae250 2024-04-25T08:53:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.920+0000 7f5aeaca8640 1 -- 172.21.15.37:0/2980443527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae40acf70 con 0x7f5ae40ae250 2024-04-25T08:53:32.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.924+0000 7f5aeaca8640 1 -- 172.21.15.37:0/2980443527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae404efa0 con 0x7f5ae40ae250 2024-04-25T08:53:32.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.924+0000 7f5ae17fa640 1 -- 172.21.15.37:0/2980443527 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5adc0056c0 con 0x7f5ae40ae250 2024-04-25T08:53:32.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.924+0000 7f5ae17fa640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ac8066f60 0x7f5ac8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:32.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.924+0000 7f5ae17fa640 1 -- 172.21.15.37:0/2980443527 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5adc085460 con 0x7f5ae40ae250 2024-04-25T08:53:32.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.924+0000 7f5ae8a44640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ac8066f60 0x7f5ac8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:32.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.924+0000 7f5ae8a44640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ac8066f60 0x7f5ac8069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ae415a2d0 tx=0x7f5ad8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:32.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:32.928+0000 7f5ae17fa640 1 -- 172.21.15.37:0/2980443527 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ae40aca10 con 0x7f5ae40ae250 2024-04-25T08:53:33.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.080+0000 7f5aeaca8640 1 -- 172.21.15.37:0/2980443527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ae40a8780 con 0x7f5ae40ae250 2024-04-25T08:53:33.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.084+0000 7f5ae17fa640 1 -- 172.21.15.37:0/2980443527 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f5adc056100 con 0x7f5ae40ae250 2024-04-25T08:53:33.078 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:33.078 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:33.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.084+0000 7f5aeaca8640 1 -- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ac8066f60 msgr2=0x7f5ac8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.084+0000 7f5aeaca8640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ac8066f60 0x7f5ac8069420 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ae415a2d0 tx=0x7f5ad8009040 comp rx=0 tx=0).stop 2024-04-25T08:53:33.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.084+0000 7f5aeaca8640 1 -- 172.21.15.37:0/2980443527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae40ae250 msgr2=0x7f5ae413de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.084+0000 7f5aeaca8640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae40ae250 0x7f5ae413de30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5adc00c730 tx=0x7f5adc00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.084+0000 7f5aeaca8640 1 -- 172.21.15.37:0/2980443527 shutdown_connections 2024-04-25T08:53:33.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.084+0000 7f5aeaca8640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ac8066f60 0x7f5ac8069420 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.084+0000 7f5aeaca8640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae40ae250 0x7f5ae413de30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.084+0000 7f5aeaca8640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae415a700 0x7f5ae413d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.084+0000 7f5aeaca8640 1 --2- 172.21.15.37:0/2980443527 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae40b24a0 0x7f5ae40add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.084+0000 7f5aeaca8640 1 -- 172.21.15.37:0/2980443527 >> 172.21.15.37:0/2980443527 conn(0x7f5ae40b11a0 msgr2=0x7f5ae415dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:33.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.088+0000 7f5aeaca8640 1 -- 172.21.15.37:0/2980443527 shutdown_connections 2024-04-25T08:53:33.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.088+0000 7f5aeaca8640 1 -- 172.21.15.37:0/2980443527 wait complete. 2024-04-25T08:53:33.095 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:33.096 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:33.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f5215d49640 1 Processor -- start 2024-04-25T08:53:33.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f5215d49640 1 -- start start 2024-04-25T08:53:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f5215d49640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5210161c80 0x7f52100b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f5215d49640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f52100c9f70 0x7f52100c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f5215d49640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f52100c2870 0x7f5210165770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f5215d49640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5210162060 con 0x7f52100c9f70 2024-04-25T08:53:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f5215d49640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52100ca480 con 0x7f52100c2870 2024-04-25T08:53:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f5215d49640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52100ca740 con 0x7f5210161c80 2024-04-25T08:53:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f520effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f52100c9f70 0x7f52100c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f520effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f52100c9f70 0x7f52100c2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47752/0 (socket says 172.21.15.37:47752) 2024-04-25T08:53:33.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f520effd640 1 -- 172.21.15.37:0/1906369650 learned_addr learned my addr 172.21.15.37:0/1906369650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:33.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f520ffff640 1 --2- 172.21.15.37:0/1906369650 >> v2:172.21.15.203:3300/0 conn(0x7f52100c2870 0x7f5210165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f520f7fe640 1 --2- 172.21.15.37:0/1906369650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5210161c80 0x7f52100b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f520effd640 1 -- 172.21.15.37:0/1906369650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5210161c80 msgr2=0x7f52100b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:33.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f520effd640 1 --2- 172.21.15.37:0/1906369650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5210161c80 0x7f52100b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f520effd640 1 -- 172.21.15.37:0/1906369650 >> v2:172.21.15.203:3300/0 conn(0x7f52100c2870 msgr2=0x7f5210165770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:33.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f520effd640 1 --2- 172.21.15.37:0/1906369650 >> v2:172.21.15.203:3300/0 conn(0x7f52100c2870 0x7f5210165770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f520effd640 1 -- 172.21.15.37:0/1906369650 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52100caa20 con 0x7f52100c9f70 2024-04-25T08:53:33.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.216+0000 7f520ffff640 1 --2- 172.21.15.37:0/1906369650 >> v2:172.21.15.203:3300/0 conn(0x7f52100c2870 0x7f5210165770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:33.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520effd640 1 --2- 172.21.15.37:0/1906369650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100c9f70 0x7f52100c2330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5210073480 tx=0x7f51f801c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ba0bab71c0db75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520e7fc640 1 -- 172.21.15.37:0/1906369650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f80b0070 con 0x7f52100c9f70 2024-04-25T08:53:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520e7fc640 1 -- 172.21.15.37:0/1906369650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f80ab600 con 0x7f52100c9f70 2024-04-25T08:53:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520e7fc640 1 -- 172.21.15.37:0/1906369650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f80ab9d0 con 0x7f52100c9f70 2024-04-25T08:53:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 -- 172.21.15.37:0/1906369650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100c9f70 msgr2=0x7f52100c2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 --2- 172.21.15.37:0/1906369650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100c9f70 0x7f52100c2330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5210073480 tx=0x7f51f801c1d0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 -- 172.21.15.37:0/1906369650 shutdown_connections 2024-04-25T08:53:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 --2- 172.21.15.37:0/1906369650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100c9f70 0x7f52100c2330 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 --2- 172.21.15.37:0/1906369650 >> v2:172.21.15.203:3300/0 conn(0x7f52100c2870 0x7f5210165770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 --2- 172.21.15.37:0/1906369650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5210161c80 0x7f52100b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 -- 172.21.15.37:0/1906369650 >> 172.21.15.37:0/1906369650 conn(0x7f521006ae60 msgr2=0x7f521006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 -- 172.21.15.37:0/1906369650 shutdown_connections 2024-04-25T08:53:33.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 -- 172.21.15.37:0/1906369650 wait complete. 2024-04-25T08:53:33.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 Processor -- start 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 -- start start 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100c2870 0x7f5210159690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52100c9f70 0x7f521015a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5210161c80 0x7f521006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100ccf30 con 0x7f52100c2870 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52100b3ba0 con 0x7f5210161c80 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f5215d49640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52100cd0b0 con 0x7f52100c9f70 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100c2870 0x7f5210159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100c2870 0x7f5210159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47760/0 (socket says 172.21.15.37:47760) 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520f7fe640 1 -- 172.21.15.37:0/261990442 learned_addr learned my addr 172.21.15.37:0/261990442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520effd640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52100c9f70 0x7f521015a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520ffff640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5210161c80 0x7f521006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520f7fe640 1 -- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52100c9f70 msgr2=0x7f521015a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520f7fe640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52100c9f70 0x7f521015a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520f7fe640 1 -- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5210161c80 msgr2=0x7f521006d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520f7fe640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5210161c80 0x7f521006d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520f7fe640 1 -- 172.21.15.37:0/261990442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521015d610 con 0x7f52100c2870 2024-04-25T08:53:33.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.220+0000 7f520f7fe640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100c2870 0x7f5210159690 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f520800ccd0 tx=0x7f520800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:33.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.224+0000 7f5207fff640 1 -- 172.21.15.37:0/261990442 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52080039a0 con 0x7f52100c2870 2024-04-25T08:53:33.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.224+0000 7f5207fff640 1 -- 172.21.15.37:0/261990442 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5208003b40 con 0x7f52100c2870 2024-04-25T08:53:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.224+0000 7f5207fff640 1 -- 172.21.15.37:0/261990442 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208004020 con 0x7f52100c2870 2024-04-25T08:53:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.224+0000 7f5215d49640 1 -- 172.21.15.37:0/261990442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5210071fb0 con 0x7f52100c2870 2024-04-25T08:53:33.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.224+0000 7f5215d49640 1 -- 172.21.15.37:0/261990442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521015d880 con 0x7f52100c2870 2024-04-25T08:53:33.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.224+0000 7f5215d49640 1 -- 172.21.15.37:0/261990442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521004efa0 con 0x7f52100c2870 2024-04-25T08:53:33.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.224+0000 7f5207fff640 1 -- 172.21.15.37:0/261990442 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52080041c0 con 0x7f52100c2870 2024-04-25T08:53:33.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.224+0000 7f5207fff640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51f0066f60 0x7f51f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.224+0000 7f5207fff640 1 -- 172.21.15.37:0/261990442 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5208085830 con 0x7f52100c2870 2024-04-25T08:53:33.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.224+0000 7f520effd640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51f0066f60 0x7f51f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.224+0000 7f520effd640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51f0066f60 0x7f51f0069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f51f80ad000 tx=0x7f51f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:33.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.228+0000 7f5207fff640 1 -- 172.21.15.37:0/261990442 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5208014030 con 0x7f52100c2870 2024-04-25T08:53:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.392+0000 7f5215d49640 1 -- 172.21.15.37:0/261990442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52100ae180 con 0x7f52100c2870 2024-04-25T08:53:33.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.392+0000 7f5207fff640 1 -- 172.21.15.37:0/261990442 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5208055df0 con 0x7f52100c2870 2024-04-25T08:53:33.387 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 -- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51f0066f60 msgr2=0x7f51f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51f0066f60 0x7f51f0069420 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f51f80ad000 tx=0x7f51f8008040 comp rx=0 tx=0).stop 2024-04-25T08:53:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 -- 172.21.15.37:0/261990442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100c2870 msgr2=0x7f5210159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100c2870 0x7f5210159690 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f520800ccd0 tx=0x7f520800a5c0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 -- 172.21.15.37:0/261990442 shutdown_connections 2024-04-25T08:53:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51f0066f60 0x7f51f0069420 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5210161c80 0x7f521006d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52100c9f70 0x7f521015a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 --2- 172.21.15.37:0/261990442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52100c2870 0x7f5210159690 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 -- 172.21.15.37:0/261990442 >> 172.21.15.37:0/261990442 conn(0x7f521006ae60 msgr2=0x7f521006b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 -- 172.21.15.37:0/261990442 shutdown_connections 2024-04-25T08:53:33.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.396+0000 7f5215d49640 1 -- 172.21.15.37:0/261990442 wait complete. 2024-04-25T08:53:33.406 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f452b9a4640 1 Processor -- start 2024-04-25T08:53:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f452b9a4640 1 -- start start 2024-04-25T08:53:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f452b9a4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f452406ac90 0x7f452406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f452b9a4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f452415ca90 0x7f45240d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f452b9a4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452406b5b0 con 0x7f45240cc130 2024-04-25T08:53:33.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f452b9a4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452415ff70 con 0x7f452406ac90 2024-04-25T08:53:33.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f452b9a4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45240d0300 con 0x7f452415ca90 2024-04-25T08:53:33.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f4529740640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f452406ac90 0x7f452406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f4529740640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f452406ac90 0x7f452406b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47766/0 (socket says 172.21.15.37:47766) 2024-04-25T08:53:33.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f4529740640 1 -- 172.21.15.37:0/1284988062 learned_addr learned my addr 172.21.15.37:0/1284988062 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:33.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f4528f3f640 1 --2- 172.21.15.37:0/1284988062 >> v2:172.21.15.203:3300/0 conn(0x7f452415ca90 0x7f45240d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f4529740640 1 -- 172.21.15.37:0/1284988062 >> v2:172.21.15.203:3300/0 conn(0x7f452415ca90 msgr2=0x7f45240d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:33.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f4529740640 1 --2- 172.21.15.37:0/1284988062 >> v2:172.21.15.203:3300/0 conn(0x7f452415ca90 0x7f45240d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f4529740640 1 -- 172.21.15.37:0/1284988062 >> v1:172.21.15.37:6789/0 conn(0x7f45240cc130 legacy=0x7f45240ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:33.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.540+0000 7f4529740640 1 -- 172.21.15.37:0/1284988062 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452415f360 con 0x7f452406ac90 2024-04-25T08:53:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f4529740640 1 --2- 172.21.15.37:0/1284988062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452406ac90 0x7f452406b070 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f452406dbb0 tx=0x7f451c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b552854ba41e7bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f4523fff640 1 -- 172.21.15.37:0/1284988062 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c0aa650 con 0x7f452406ac90 2024-04-25T08:53:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f4523fff640 1 -- 172.21.15.37:0/1284988062 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c0af070 con 0x7f452406ac90 2024-04-25T08:53:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f4523fff640 1 -- 172.21.15.37:0/1284988062 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c0aaa10 con 0x7f452406ac90 2024-04-25T08:53:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 -- 172.21.15.37:0/1284988062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452406ac90 msgr2=0x7f452406b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 --2- 172.21.15.37:0/1284988062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452406ac90 0x7f452406b070 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f452406dbb0 tx=0x7f451c01c4f0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 -- 172.21.15.37:0/1284988062 shutdown_connections 2024-04-25T08:53:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 --2- 172.21.15.37:0/1284988062 >> v2:172.21.15.203:3300/0 conn(0x7f452415ca90 0x7f45240d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 --2- 172.21.15.37:0/1284988062 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452406ac90 0x7f452406b070 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 -- 172.21.15.37:0/1284988062 >> 172.21.15.37:0/1284988062 conn(0x7f4524099a70 msgr2=0x7f45240d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 -- 172.21.15.37:0/1284988062 shutdown_connections 2024-04-25T08:53:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 -- 172.21.15.37:0/1284988062 wait complete. 2024-04-25T08:53:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 Processor -- start 2024-04-25T08:53:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 -- start start 2024-04-25T08:53:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452406ac90 0x7f45240b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.544+0000 7f452b9a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45240cc130 0x7f452415d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f452b9a4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f452415ca90 0x7f452415d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f452b9a4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4524159750 con 0x7f452406ac90 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f452b9a4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45241595d0 con 0x7f452415ca90 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f452b9a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4524168660 con 0x7f45240cc130 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4529740640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452406ac90 0x7f45240b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4529740640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452406ac90 0x7f45240b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47768/0 (socket says 172.21.15.37:47768) 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4529740640 1 -- 172.21.15.37:0/2824035654 learned_addr learned my addr 172.21.15.37:0/2824035654 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4529740640 1 -- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45240cc130 msgr2=0x7f452415d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4529740640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45240cc130 0x7f452415d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4529740640 1 -- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f452415ca90 msgr2=0x7f452415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4529f41640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f452415ca90 0x7f452415d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4529740640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f452415ca90 0x7f452415d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4529740640 1 -- 172.21.15.37:0/2824035654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45240b2d70 con 0x7f452406ac90 2024-04-25T08:53:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4529f41640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f452415ca90 0x7f452415d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4529740640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452406ac90 0x7f45240b26e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f451c0ac040 tx=0x7f451c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4521ffb640 1 -- 172.21.15.37:0/2824035654 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c0bd5f0 con 0x7f452406ac90 2024-04-25T08:53:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4521ffb640 1 -- 172.21.15.37:0/2824035654 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c0a6040 con 0x7f452406ac90 2024-04-25T08:53:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f452b9a4640 1 -- 172.21.15.37:0/2824035654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45240c8430 con 0x7f452406ac90 2024-04-25T08:53:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4521ffb640 1 -- 172.21.15.37:0/2824035654 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c0bd9b0 con 0x7f452406ac90 2024-04-25T08:53:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f452b9a4640 1 -- 172.21.15.37:0/2824035654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45240c89f0 con 0x7f452406ac90 2024-04-25T08:53:33.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f452b9a4640 1 -- 172.21.15.37:0/2824035654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452404efa0 con 0x7f452406ac90 2024-04-25T08:53:33.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4521ffb640 1 -- 172.21.15.37:0/2824035654 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f451c0b33f0 con 0x7f452406ac90 2024-04-25T08:53:33.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4521ffb640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4510066f60 0x7f4510069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4521ffb640 1 -- 172.21.15.37:0/2824035654 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f451c11fb40 con 0x7f452406ac90 2024-04-25T08:53:33.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4528f3f640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4510066f60 0x7f4510069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.548+0000 7f4528f3f640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4510066f60 0x7f4510069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f45180070b0 tx=0x7f4518007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:33.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.552+0000 7f4521ffb640 1 -- 172.21.15.37:0/2824035654 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f451c0c3060 con 0x7f452406ac90 2024-04-25T08:53:33.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.712+0000 7f452b9a4640 1 -- 172.21.15.37:0/2824035654 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f452406b7c0 con 0x7f452406ac90 2024-04-25T08:53:33.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f4521ffb640 1 -- 172.21.15.37:0/2824035654 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f451c016110 con 0x7f452406ac90 2024-04-25T08:53:33.708 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 -- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4510066f60 msgr2=0x7f4510069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4510066f60 0x7f4510069420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f45180070b0 tx=0x7f4518007040 comp rx=0 tx=0).stop 2024-04-25T08:53:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 -- 172.21.15.37:0/2824035654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452406ac90 msgr2=0x7f45240b26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452406ac90 0x7f45240b26e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f451c0ac040 tx=0x7f451c01ad30 comp rx=0 tx=0).stop 2024-04-25T08:53:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 -- 172.21.15.37:0/2824035654 shutdown_connections 2024-04-25T08:53:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4510066f60 0x7f4510069420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f452415ca90 0x7f452415d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45240cc130 0x7f452415d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 --2- 172.21.15.37:0/2824035654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f452406ac90 0x7f45240b26e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 -- 172.21.15.37:0/2824035654 >> 172.21.15.37:0/2824035654 conn(0x7f4524099a70 msgr2=0x7f45241525e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 -- 172.21.15.37:0/2824035654 shutdown_connections 2024-04-25T08:53:33.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.716+0000 7f452b9a4640 1 -- 172.21.15.37:0/2824035654 wait complete. 2024-04-25T08:53:33.727 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15ea2f3640 1 Processor -- start 2024-04-25T08:53:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15ea2f3640 1 -- start start 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15ea2f3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15e40cf590 0x7f15e40cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15ea2f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e4072000 0x7f15e4072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15ea2f3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e40cbd20 con 0x7f15e415b6b0 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15ea2f3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e415ba90 con 0x7f15e40cf590 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15ea2f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40cc000 con 0x7f15e4072000 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15e3fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f15e415b6b0 0x7f15e40ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52540/0 (socket says 172.21.15.37:52540) 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15e3fff640 1 -- 172.21.15.37:0/2117760794 learned_addr learned my addr 172.21.15.37:0/2117760794 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15e37fe640 1 --2- 172.21.15.37:0/2117760794 >> v2:172.21.15.37:3300/0 conn(0x7f15e40cf590 0x7f15e40cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15e8890640 1 --2- 172.21.15.37:0/2117760794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e4072000 0x7f15e4072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15e2ffd640 1 -- 172.21.15.37:0/2117760794 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2306118209 0 0) 0x7f15e40cbd20 con 0x7f15e415b6b0 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15e2ffd640 1 -- 172.21.15.37:0/2117760794 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15cc003530 con 0x7f15e415b6b0 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15e37fe640 1 -- 172.21.15.37:0/2117760794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e4072000 msgr2=0x7f15e4072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15e37fe640 1 --2- 172.21.15.37:0/2117760794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e4072000 0x7f15e4072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15e37fe640 1 -- 172.21.15.37:0/2117760794 >> v1:172.21.15.37:6789/0 conn(0x7f15e415b6b0 legacy=0x7f15e40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.864+0000 7f15e37fe640 1 -- 172.21.15.37:0/2117760794 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e40cc2e0 con 0x7f15e40cf590 2024-04-25T08:53:33.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15e37fe640 1 --2- 172.21.15.37:0/2117760794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e40cf590 0x7f15e40cfa20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f15d80097c0 tx=0x7f15d801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da34c2a21303107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:33.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15e2ffd640 1 -- 172.21.15.37:0/2117760794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80b2070 con 0x7f15e40cf590 2024-04-25T08:53:33.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15e2ffd640 1 -- 172.21.15.37:0/2117760794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d80ad640 con 0x7f15e40cf590 2024-04-25T08:53:33.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15e2ffd640 1 -- 172.21.15.37:0/2117760794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80ada10 con 0x7f15e40cf590 2024-04-25T08:53:33.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 -- 172.21.15.37:0/2117760794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e40cf590 msgr2=0x7f15e40cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 --2- 172.21.15.37:0/2117760794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e40cf590 0x7f15e40cfa20 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f15d80097c0 tx=0x7f15d801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 -- 172.21.15.37:0/2117760794 shutdown_connections 2024-04-25T08:53:33.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 --2- 172.21.15.37:0/2117760794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e4072000 0x7f15e4072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 --2- 172.21.15.37:0/2117760794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e40cf590 0x7f15e40cfa20 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 -- 172.21.15.37:0/2117760794 >> 172.21.15.37:0/2117760794 conn(0x7f15e406ae60 msgr2=0x7f15e406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:33.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 -- 172.21.15.37:0/2117760794 shutdown_connections 2024-04-25T08:53:33.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 -- 172.21.15.37:0/2117760794 wait complete. 2024-04-25T08:53:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 Processor -- start 2024-04-25T08:53:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 -- start start 2024-04-25T08:53:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15e4072000 0x7f15e4098510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e40cf590 0x7f15e40c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e415b6b0 0x7f15e4151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40caf30 con 0x7f15e40cf590 2024-04-25T08:53:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40cadb0 con 0x7f15e4072000 2024-04-25T08:53:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15ea2f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e41682a0 con 0x7f15e415b6b0 2024-04-25T08:53:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15e37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e40cf590 0x7f15e40c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15e37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e40cf590 0x7f15e40c1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47790/0 (socket says 172.21.15.37:47790) 2024-04-25T08:53:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15e37fe640 1 -- 172.21.15.37:0/3898892778 learned_addr learned my addr 172.21.15.37:0/3898892778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15e8890640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e415b6b0 0x7f15e4151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.868+0000 7f15e3fff640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15e4072000 0x7f15e4098510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e37fe640 1 -- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e415b6b0 msgr2=0x7f15e4151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e37fe640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e415b6b0 0x7f15e4151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e37fe640 1 -- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15e4072000 msgr2=0x7f15e4098510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e37fe640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15e4072000 0x7f15e4098510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e37fe640 1 -- 172.21.15.37:0/3898892778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e41504c0 con 0x7f15e40cf590 2024-04-25T08:53:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e37fe640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e40cf590 0x7f15e40c1dc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f15d80097c0 tx=0x7f15d8019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e0ff9640 1 -- 172.21.15.37:0/3898892778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80b2040 con 0x7f15e40cf590 2024-04-25T08:53:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e0ff9640 1 -- 172.21.15.37:0/3898892778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d80bc070 con 0x7f15e40cf590 2024-04-25T08:53:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e0ff9640 1 -- 172.21.15.37:0/3898892778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80b6630 con 0x7f15e40cf590 2024-04-25T08:53:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15ea2f3640 1 -- 172.21.15.37:0/3898892778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e4097c70 con 0x7f15e40cf590 2024-04-25T08:53:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15ea2f3640 1 -- 172.21.15.37:0/3898892778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e4098230 con 0x7f15e40cf590 2024-04-25T08:53:33.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15ea2f3640 1 -- 172.21.15.37:0/3898892778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e4150690 con 0x7f15e40cf590 2024-04-25T08:53:33.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e0ff9640 1 -- 172.21.15.37:0/3898892778 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15e40731c0 con 0x7f15e40cf590 2024-04-25T08:53:33.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e0ff9640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15d0066f90 0x7f15d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:33.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e0ff9640 1 -- 172.21.15.37:0/3898892778 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f15d811eab0 con 0x7f15e40cf590 2024-04-25T08:53:33.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e3fff640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15d0066f90 0x7f15d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:33.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.872+0000 7f15e3fff640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15d0066f90 0x7f15d0069450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f15d4005890 tx=0x7f15d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:33.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:33.876+0000 7f15e0ff9640 1 -- 172.21.15.37:0/3898892778 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f15d80c5020 con 0x7f15e40cf590 2024-04-25T08:53:34.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.032+0000 7f15ea2f3640 1 -- 172.21.15.37:0/3898892778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f15e404efa0 con 0x7f15e40cf590 2024-04-25T08:53:34.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.032+0000 7f15e0ff9640 1 -- 172.21.15.37:0/3898892778 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f15d8016090 con 0x7f15e40cf590 2024-04-25T08:53:34.026 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 -- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15d0066f90 msgr2=0x7f15d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15d0066f90 0x7f15d0069450 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f15d4005890 tx=0x7f15d4009040 comp rx=0 tx=0).stop 2024-04-25T08:53:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 -- 172.21.15.37:0/3898892778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e40cf590 msgr2=0x7f15e40c1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e40cf590 0x7f15e40c1dc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f15d80097c0 tx=0x7f15d8019240 comp rx=0 tx=0).stop 2024-04-25T08:53:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 -- 172.21.15.37:0/3898892778 shutdown_connections 2024-04-25T08:53:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15d0066f90 0x7f15d0069450 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15e415b6b0 0x7f15e4151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e40cf590 0x7f15e40c1dc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 --2- 172.21.15.37:0/3898892778 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15e4072000 0x7f15e4098510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 -- 172.21.15.37:0/3898892778 >> 172.21.15.37:0/3898892778 conn(0x7f15e406ae60 msgr2=0x7f15e40b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:34.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 -- 172.21.15.37:0/3898892778 shutdown_connections 2024-04-25T08:53:34.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.036+0000 7f15ea2f3640 1 -- 172.21.15.37:0/3898892778 wait complete. 2024-04-25T08:53:34.045 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:34.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.168+0000 7f19fc0be640 1 Processor -- start 2024-04-25T08:53:34.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.168+0000 7f19fc0be640 1 -- start start 2024-04-25T08:53:34.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fc0be640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f19f415b6b0 0x7f19f40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fc0be640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f415ff50 con 0x7f19f4073010 2024-04-25T08:53:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fc0be640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f40cbd20 con 0x7f19f40cf590 2024-04-25T08:53:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fc0be640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f415ba90 con 0x7f19f415b6b0 2024-04-25T08:53:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fa65b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f19f4073010 0x7f19f415f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52546/0 (socket says 172.21.15.37:52546) 2024-04-25T08:53:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fa65b640 1 -- 172.21.15.37:0/3551038736 learned_addr learned my addr 172.21.15.37:0/3551038736 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f9e5a640 1 --2- 172.21.15.37:0/3551038736 >> v2:172.21.15.203:3300/0 conn(0x7f19f415b6b0 0x7f19f40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2757877242 0 0) 0x7f19f415ff50 con 0x7f19f4073010 2024-04-25T08:53:34.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e0003530 con 0x7f19f4073010 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2050250182 0 0) 0x7f19e0003530 con 0x7f19f4073010 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 >> v2:172.21.15.203:3300/0 conn(0x7f19f415b6b0 msgr2=0x7f19f40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 --2- 172.21.15.37:0/3551038736 >> v2:172.21.15.203:3300/0 conn(0x7f19f415b6b0 0x7f19f40ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 >> v1:172.21.15.203:6789/0 conn(0x7f19f40cf590 legacy=0x7f19f415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40cc000 con 0x7f19f4073010 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f19ec001680 con 0x7f19f4073010 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 >> v1:172.21.15.37:6789/0 conn(0x7f19f4073010 legacy=0x7f19f415f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e0003a30 0x7f19e0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19e0004810 0x7f19e0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19e00051d0 0x7f19e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f415ba90 con 0x7f19e00051d0 2024-04-25T08:53:34.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e000c100 con 0x7f19e0003a30 2024-04-25T08:53:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e000c300 con 0x7f19e0004810 2024-04-25T08:53:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fa65b640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19e00051d0 0x7f19e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fa65b640 1 -- 172.21.15.37:0/3551038736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19e0004810 msgr2=0x7f19e0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f9e5a640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e0003a30 0x7f19e0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f9659640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19e0004810 0x7f19e0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fa65b640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19e0004810 0x7f19e0004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fa65b640 1 -- 172.21.15.37:0/3551038736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e0003a30 msgr2=0x7f19e0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fa65b640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e0003a30 0x7f19e0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19fa65b640 1 -- 172.21.15.37:0/3551038736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0003530 con 0x7f19e00051d0 2024-04-25T08:53:34.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.172+0000 7f19f9e5a640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19e0003a30 0x7f19e0003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:34.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fa65b640 1 -- 172.21.15.37:0/3551038736 reap_dead start 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fa65b640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19e00051d0 0x7f19e000b9d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00c210 tx=0x7f19ec01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28134569f217fb0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec007d40 con 0x7f19e00051d0 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec0a9070 con 0x7f19e00051d0 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f8e58640 1 -- 172.21.15.37:0/3551038736 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec0a5e00 con 0x7f19e00051d0 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 -- 172.21.15.37:0/3551038736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19e00051d0 msgr2=0x7f19e000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19e00051d0 0x7f19e000b9d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00c210 tx=0x7f19ec01cca0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 -- 172.21.15.37:0/3551038736 shutdown_connections 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 --2- 172.21.15.37:0/3551038736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19e00051d0 0x7f19e000b9d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 -- 172.21.15.37:0/3551038736 >> 172.21.15.37:0/3551038736 conn(0x7f19f406ae60 msgr2=0x7f19f406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 -- 172.21.15.37:0/3551038736 shutdown_connections 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 -- 172.21.15.37:0/3551038736 wait complete. 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 Processor -- start 2024-04-25T08:53:34.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 -- start start 2024-04-25T08:53:34.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19e00051d0 0x7f19f41f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f4155290 0x7f19f40c5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19f415e060 0x7f19f415e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40cc8a0 con 0x7f19f4155290 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40cc5c0 con 0x7f19f415e060 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19fc0be640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40cba30 con 0x7f19e00051d0 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f9659640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f4155290 0x7f19f40c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f9659640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f4155290 0x7f19f40c5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47800/0 (socket says 172.21.15.37:47800) 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f9659640 1 -- 172.21.15.37:0/2094886564 learned_addr learned my addr 172.21.15.37:0/2094886564 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f9659640 1 -- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19e00051d0 msgr2=0x7f19f41f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f9659640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19e00051d0 0x7f19f41f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f9659640 1 -- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19f415e060 msgr2=0x7f19f415e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f9659640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19f415e060 0x7f19f415e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f9659640 1 -- 172.21.15.37:0/2094886564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f415c1d0 con 0x7f19f4155290 2024-04-25T08:53:34.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19f9659640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f4155290 0x7f19f40c5790 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f19e400d540 tx=0x7f19e400da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:34.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19ea7fc640 1 -- 172.21.15.37:0/2094886564 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4005480 con 0x7f19f4155290 2024-04-25T08:53:34.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19ea7fc640 1 -- 172.21.15.37:0/2094886564 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e4017070 con 0x7f19f4155290 2024-04-25T08:53:34.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.176+0000 7f19ea7fc640 1 -- 172.21.15.37:0/2094886564 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e4003820 con 0x7f19f4155290 2024-04-25T08:53:34.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.180+0000 7f19fc0be640 1 -- 172.21.15.37:0/2094886564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f40c3ee0 con 0x7f19f4155290 2024-04-25T08:53:34.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.180+0000 7f19fc0be640 1 -- 172.21.15.37:0/2094886564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f415c800 con 0x7f19f4155290 2024-04-25T08:53:34.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.180+0000 7f19fc0be640 1 -- 172.21.15.37:0/2094886564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f404efa0 con 0x7f19f4155290 2024-04-25T08:53:34.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.180+0000 7f19ea7fc640 1 -- 172.21.15.37:0/2094886564 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19e40039c0 con 0x7f19f4155290 2024-04-25T08:53:34.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.180+0000 7f19ea7fc640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19d8066f90 0x7f19d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.180+0000 7f19ea7fc640 1 -- 172.21.15.37:0/2094886564 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f19e4085240 con 0x7f19f4155290 2024-04-25T08:53:34.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.180+0000 7f19f9e5a640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19d8066f90 0x7f19d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.180+0000 7f19f9e5a640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19d8066f90 0x7f19d8069450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f19f0007460 tx=0x7f19f0009750 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.184+0000 7f19ea7fc640 1 -- 172.21.15.37:0/2094886564 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19f415c1d0 con 0x7f19f4155290 2024-04-25T08:53:34.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.340+0000 7f19fc0be640 1 -- 172.21.15.37:0/2094886564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f19f40c5cd0 con 0x7f19f4155290 2024-04-25T08:53:34.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.340+0000 7f19ea7fc640 1 -- 172.21.15.37:0/2094886564 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f19e400f2f0 con 0x7f19f4155290 2024-04-25T08:53:34.334 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 -- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19d8066f90 msgr2=0x7f19d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19d8066f90 0x7f19d8069450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f19f0007460 tx=0x7f19f0009750 comp rx=0 tx=0).stop 2024-04-25T08:53:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 -- 172.21.15.37:0/2094886564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f4155290 msgr2=0x7f19f40c5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f4155290 0x7f19f40c5790 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f19e400d540 tx=0x7f19e400da00 comp rx=0 tx=0).stop 2024-04-25T08:53:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 -- 172.21.15.37:0/2094886564 shutdown_connections 2024-04-25T08:53:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19d8066f90 0x7f19d8069450 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19f415e060 0x7f19f415e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19f4155290 0x7f19f40c5790 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 --2- 172.21.15.37:0/2094886564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19e00051d0 0x7f19f41f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 -- 172.21.15.37:0/2094886564 >> 172.21.15.37:0/2094886564 conn(0x7f19f406ae60 msgr2=0x7f19f40b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:34.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 -- 172.21.15.37:0/2094886564 shutdown_connections 2024-04-25T08:53:34.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.344+0000 7f19fc0be640 1 -- 172.21.15.37:0/2094886564 wait complete. 2024-04-25T08:53:34.351 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:53:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.480+0000 7f5a10fcf640 1 Processor -- start 2024-04-25T08:53:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.480+0000 7f5a10fcf640 1 -- start start 2024-04-25T08:53:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a10fcf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5a0c09a480 0x7f5a0c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a10fcf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5a0c0cfb80 0x7f5a0c0cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a10fcf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a0c0d04d0 con 0x7f5a0c15b950 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a10fcf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c15bd30 con 0x7f5a0c09a480 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a10fcf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0c0cc0d0 con 0x7f5a0c0cfb80 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a03fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5a0c09a480 0x7f5a0c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a03fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5a0c09a480 0x7f5a0c0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47806/0 (socket says 172.21.15.37:47806) 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a03fff640 1 -- 172.21.15.37:0/228438725 learned_addr learned my addr 172.21.15.37:0/228438725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a0ad9d640 1 --2- 172.21.15.37:0/228438725 >> v2:172.21.15.203:3300/0 conn(0x7f5a0c0cfb80 0x7f5a0c0cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a03fff640 1 -- 172.21.15.37:0/228438725 >> v2:172.21.15.203:3300/0 conn(0x7f5a0c0cfb80 msgr2=0x7f5a0c0cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a03fff640 1 --2- 172.21.15.37:0/228438725 >> v2:172.21.15.203:3300/0 conn(0x7f5a0c0cfb80 0x7f5a0c0cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a03fff640 1 -- 172.21.15.37:0/228438725 >> v1:172.21.15.203:6789/0 conn(0x7f5a0c15b950 legacy=0x7f5a0c0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a03fff640 1 -- 172.21.15.37:0/228438725 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c09ab10 con 0x7f5a0c09a480 2024-04-25T08:53:34.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a03fff640 1 --2- 172.21.15.37:0/228438725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a0c09a480 0x7f5a0c0cce50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0b3bd0 tx=0x7f59f401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4eea3df0adbe33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:34.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a09d9b640 1 -- 172.21.15.37:0/228438725 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f40b0070 con 0x7f5a0c09a480 2024-04-25T08:53:34.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a09d9b640 1 -- 172.21.15.37:0/228438725 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f40ab600 con 0x7f5a0c09a480 2024-04-25T08:53:34.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.484+0000 7f5a09d9b640 1 -- 172.21.15.37:0/228438725 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f40ab9d0 con 0x7f5a0c09a480 2024-04-25T08:53:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 -- 172.21.15.37:0/228438725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a0c09a480 msgr2=0x7f5a0c0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 --2- 172.21.15.37:0/228438725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a0c09a480 0x7f5a0c0cce50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0b3bd0 tx=0x7f59f401c300 comp rx=0 tx=0).stop 2024-04-25T08:53:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 -- 172.21.15.37:0/228438725 shutdown_connections 2024-04-25T08:53:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 --2- 172.21.15.37:0/228438725 >> v2:172.21.15.203:3300/0 conn(0x7f5a0c0cfb80 0x7f5a0c0cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 --2- 172.21.15.37:0/228438725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a0c09a480 0x7f5a0c0cce50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 -- 172.21.15.37:0/228438725 >> 172.21.15.37:0/228438725 conn(0x7f5a0c0b30b0 msgr2=0x7f5a0c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 -- 172.21.15.37:0/228438725 shutdown_connections 2024-04-25T08:53:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 -- 172.21.15.37:0/228438725 wait complete. 2024-04-25T08:53:34.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 Processor -- start 2024-04-25T08:53:34.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 -- start start 2024-04-25T08:53:34.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a0c09a480 0x7f5a0c151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a0c0cfb80 0x7f5a0c15a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a0c15b950 0x7f5a0c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c0cd390 con 0x7f5a0c09a480 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c06faa0 con 0x7f5a0c0cfb80 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0c158af0 con 0x7f5a0c15b950 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a0a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a0c09a480 0x7f5a0c151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a0a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a0c09a480 0x7f5a0c151f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47814/0 (socket says 172.21.15.37:47814) 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a0a59c640 1 -- 172.21.15.37:0/1251077605 learned_addr learned my addr 172.21.15.37:0/1251077605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a03fff640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a0c0cfb80 0x7f5a0c15a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a0a59c640 1 -- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a0c15b950 msgr2=0x7f5a0c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a0a59c640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a0c15b950 0x7f5a0c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a0a59c640 1 -- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a0c0cfb80 msgr2=0x7f5a0c15a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a0a59c640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a0c0cfb80 0x7f5a0c15a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a0a59c640 1 -- 172.21.15.37:0/1251077605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0c158dd0 con 0x7f5a0c09a480 2024-04-25T08:53:34.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a0a59c640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a0c09a480 0x7f5a0c151f10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b410 tx=0x7f59fc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:34.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a02ffd640 1 -- 172.21.15.37:0/1251077605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc015030 con 0x7f5a0c09a480 2024-04-25T08:53:34.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a02ffd640 1 -- 172.21.15.37:0/1251077605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc004020 con 0x7f5a0c09a480 2024-04-25T08:53:34.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 -- 172.21.15.37:0/1251077605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0c1590b0 con 0x7f5a0c09a480 2024-04-25T08:53:34.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a02ffd640 1 -- 172.21.15.37:0/1251077605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc004450 con 0x7f5a0c09a480 2024-04-25T08:53:34.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.488+0000 7f5a10fcf640 1 -- 172.21.15.37:0/1251077605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0c0b5800 con 0x7f5a0c09a480 2024-04-25T08:53:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.492+0000 7f5a10fcf640 1 -- 172.21.15.37:0/1251077605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a0c04efa0 con 0x7f5a0c09a480 2024-04-25T08:53:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.492+0000 7f5a02ffd640 1 -- 172.21.15.37:0/1251077605 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f59fc018070 con 0x7f5a0c09a480 2024-04-25T08:53:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.492+0000 7f5a02ffd640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f59ec066f60 0x7f59ec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.492+0000 7f5a02ffd640 1 -- 172.21.15.37:0/1251077605 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f59fc084fe0 con 0x7f5a0c09a480 2024-04-25T08:53:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.492+0000 7f5a03fff640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f59ec066f60 0x7f59ec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.492+0000 7f5a03fff640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f59ec066f60 0x7f59ec069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f59f40ad000 tx=0x7f59f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:34.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.496+0000 7f5a02ffd640 1 -- 172.21.15.37:0/1251077605 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f59fc055bf0 con 0x7f5a0c09a480 2024-04-25T08:53:34.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.668+0000 7f5a10fcf640 1 -- 172.21.15.37:0/1251077605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5a0c159560 con 0x7f5a0c09a480 2024-04-25T08:53:34.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.668+0000 7f5a02ffd640 1 -- 172.21.15.37:0/1251077605 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f59fc0554f0 con 0x7f5a0c09a480 2024-04-25T08:53:34.662 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:53:34.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 -- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f59ec066f60 msgr2=0x7f59ec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f59ec066f60 0x7f59ec069420 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f59f40ad000 tx=0x7f59f4008040 comp rx=0 tx=0).stop 2024-04-25T08:53:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 -- 172.21.15.37:0/1251077605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a0c09a480 msgr2=0x7f5a0c151f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a0c09a480 0x7f5a0c151f10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00b410 tx=0x7f59fc00b8d0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 -- 172.21.15.37:0/1251077605 shutdown_connections 2024-04-25T08:53:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f59ec066f60 0x7f59ec069420 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a0c15b950 0x7f5a0c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a0c0cfb80 0x7f5a0c15a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 --2- 172.21.15.37:0/1251077605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a0c09a480 0x7f5a0c151f10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 -- 172.21.15.37:0/1251077605 >> 172.21.15.37:0/1251077605 conn(0x7f5a0c0b30b0 msgr2=0x7f5a0c1542a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 -- 172.21.15.37:0/1251077605 shutdown_connections 2024-04-25T08:53:34.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.672+0000 7f5a10fcf640 1 -- 172.21.15.37:0/1251077605 wait complete. 2024-04-25T08:53:34.680 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:53:34.681 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:34.687 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:34.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2716482640 1 Processor -- start 2024-04-25T08:53:34.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2716482640 1 -- start start 2024-04-25T08:53:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2716482640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2710144ca0 0x7f271006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2716482640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f271006ec60 0x7f2710142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2716482640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27100ccc70 con 0x7f27100ce510 2024-04-25T08:53:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2716482640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27100cc2b0 con 0x7f271006ec60 2024-04-25T08:53:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2716482640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f271006e290 con 0x7f2710144ca0 2024-04-25T08:53:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f270ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f27100ce510 0x7f27100ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52558/0 (socket says 172.21.15.37:52558) 2024-04-25T08:53:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f270ffff640 1 -- 172.21.15.37:0/2911876218 learned_addr learned my addr 172.21.15.37:0/2911876218 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2714a1f640 1 --2- 172.21.15.37:0/2911876218 >> v2:172.21.15.37:3300/0 conn(0x7f271006ec60 0x7f2710142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f270effd640 1 -- 172.21.15.37:0/2911876218 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272828150 0 0) 0x7f27100ccc70 con 0x7f27100ce510 2024-04-25T08:53:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f270f7fe640 1 --2- 172.21.15.37:0/2911876218 >> v2:172.21.15.203:3300/0 conn(0x7f2710144ca0 0x7f271006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f270effd640 1 -- 172.21.15.37:0/2911876218 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26f8003530 con 0x7f27100ce510 2024-04-25T08:53:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2714a1f640 1 -- 172.21.15.37:0/2911876218 >> v2:172.21.15.203:3300/0 conn(0x7f2710144ca0 msgr2=0x7f271006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2714a1f640 1 --2- 172.21.15.37:0/2911876218 >> v2:172.21.15.203:3300/0 conn(0x7f2710144ca0 0x7f271006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2714a1f640 1 -- 172.21.15.37:0/2911876218 >> v1:172.21.15.37:6789/0 conn(0x7f27100ce510 legacy=0x7f27100ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f270effd640 1 -- 172.21.15.37:0/2911876218 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1270080996 0 0) 0x7f26f8003530 con 0x7f27100ce510 2024-04-25T08:53:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2714a1f640 1 -- 172.21.15.37:0/2911876218 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271006dc40 con 0x7f271006ec60 2024-04-25T08:53:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f270f7fe640 1 --2- 172.21.15.37:0/2911876218 >> v2:172.21.15.203:3300/0 conn(0x7f2710144ca0 0x7f271006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:34.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.856+0000 7f2714a1f640 1 --2- 172.21.15.37:0/2911876218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271006ec60 0x7f2710142680 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2710070fe0 tx=0x7f270001eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafc4e03091b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:34.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270effd640 1 -- 172.21.15.37:0/2911876218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27000b5070 con 0x7f271006ec60 2024-04-25T08:53:34.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270effd640 1 -- 172.21.15.37:0/2911876218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27000a8610 con 0x7f271006ec60 2024-04-25T08:53:34.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270effd640 1 -- 172.21.15.37:0/2911876218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708001a00 con 0x7f271006ec60 2024-04-25T08:53:34.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 -- 172.21.15.37:0/2911876218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271006ec60 msgr2=0x7f2710142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 --2- 172.21.15.37:0/2911876218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271006ec60 0x7f2710142680 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2710070fe0 tx=0x7f270001eb90 comp rx=0 tx=0).stop 2024-04-25T08:53:34.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 -- 172.21.15.37:0/2911876218 shutdown_connections 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 --2- 172.21.15.37:0/2911876218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271006ec60 0x7f2710142680 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 --2- 172.21.15.37:0/2911876218 >> v2:172.21.15.203:3300/0 conn(0x7f2710144ca0 0x7f271006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 -- 172.21.15.37:0/2911876218 >> 172.21.15.37:0/2911876218 conn(0x7f27100b11a0 msgr2=0x7f27100b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 -- 172.21.15.37:0/2911876218 shutdown_connections 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 -- 172.21.15.37:0/2911876218 wait complete. 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 Processor -- start 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 -- start start 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271006ec60 0x7f271015b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27100ce510 0x7f27100ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2710144ca0 0x7f27100ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27101451f0 con 0x7f271006ec60 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27100cc900 con 0x7f27100ce510 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2710145370 con 0x7f2710144ca0 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271006ec60 0x7f271015b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271006ec60 0x7f271015b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47826/0 (socket says 172.21.15.37:47826) 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270ffff640 1 -- 172.21.15.37:0/3494232599 learned_addr learned my addr 172.21.15.37:0/3494232599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270f7fe640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27100ce510 0x7f27100ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2714a1f640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2710144ca0 0x7f27100ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270ffff640 1 -- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2710144ca0 msgr2=0x7f27100ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270ffff640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2710144ca0 0x7f27100ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270ffff640 1 -- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27100ce510 msgr2=0x7f27100ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270ffff640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27100ce510 0x7f27100ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270ffff640 1 -- 172.21.15.37:0/3494232599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27100aa590 con 0x7f271006ec60 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270f7fe640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27100ce510 0x7f27100ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270ffff640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271006ec60 0x7f271015b1e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2708001920 tx=0x7f270800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270cff9640 1 -- 172.21.15.37:0/3494232599 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2708012400 con 0x7f271006ec60 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270cff9640 1 -- 172.21.15.37:0/3494232599 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27080125c0 con 0x7f271006ec60 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 -- 172.21.15.37:0/3494232599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27100aa870 con 0x7f271006ec60 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f270cff9640 1 -- 172.21.15.37:0/3494232599 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27080129b0 con 0x7f271006ec60 2024-04-25T08:53:34.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.860+0000 7f2716482640 1 -- 172.21.15.37:0/3494232599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27100aae30 con 0x7f271006ec60 2024-04-25T08:53:34.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.864+0000 7f2716482640 1 -- 172.21.15.37:0/3494232599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f271004efa0 con 0x7f271006ec60 2024-04-25T08:53:34.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.864+0000 7f270cff9640 1 -- 172.21.15.37:0/3494232599 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2708012b50 con 0x7f271006ec60 2024-04-25T08:53:34.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.864+0000 7f270cff9640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f26fc066f90 0x7f26fc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:34.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.864+0000 7f270cff9640 1 -- 172.21.15.37:0/3494232599 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2708085c30 con 0x7f271006ec60 2024-04-25T08:53:34.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.864+0000 7f270f7fe640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f26fc066f90 0x7f26fc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:34.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.864+0000 7f270f7fe640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f26fc066f90 0x7f26fc069450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f271013b670 tx=0x7f2704008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:34.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:34.868+0000 7f270cff9640 1 -- 172.21.15.37:0/3494232599 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27080568d0 con 0x7f271006ec60 2024-04-25T08:53:35.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.024+0000 7f2716482640 1 -- 172.21.15.37:0/3494232599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27100ccd70 con 0x7f271006ec60 2024-04-25T08:53:35.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.028+0000 7f270cff9640 1 -- 172.21.15.37:0/3494232599 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f27080561d0 con 0x7f271006ec60 2024-04-25T08:53:35.022 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:35.022 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 -- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f26fc066f90 msgr2=0x7f26fc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f26fc066f90 0x7f26fc069450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f271013b670 tx=0x7f2704008040 comp rx=0 tx=0).stop 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 -- 172.21.15.37:0/3494232599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271006ec60 msgr2=0x7f271015b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271006ec60 0x7f271015b1e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2708001920 tx=0x7f270800a430 comp rx=0 tx=0).stop 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 -- 172.21.15.37:0/3494232599 shutdown_connections 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f26fc066f90 0x7f26fc069450 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2710144ca0 0x7f27100ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27100ce510 0x7f27100ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 --2- 172.21.15.37:0/3494232599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f271006ec60 0x7f271015b1e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 -- 172.21.15.37:0/3494232599 >> 172.21.15.37:0/3494232599 conn(0x7f27100b11a0 msgr2=0x7f2710141800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 -- 172.21.15.37:0/3494232599 shutdown_connections 2024-04-25T08:53:35.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.032+0000 7f2716482640 1 -- 172.21.15.37:0/3494232599 wait complete. 2024-04-25T08:53:35.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:35.041 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:35.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbbdde5640 1 Processor -- start 2024-04-25T08:53:35.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbbdde5640 1 -- start start 2024-04-25T08:53:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbbdde5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcbb815b420 0x7fcbb80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbbdde5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcbb80cfca0 0x7fcbb8072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbbdde5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb80cbd20 con 0x7fcbb815b800 2024-04-25T08:53:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbbdde5640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb80cc930 con 0x7fcbb815b420 2024-04-25T08:53:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbbdde5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb80cc000 con 0x7fcbb80cfca0 2024-04-25T08:53:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbb77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcbb815b420 0x7fcbb80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbb77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcbb815b420 0x7fcbb80ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47832/0 (socket says 172.21.15.37:47832) 2024-04-25T08:53:35.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbb77fe640 1 -- 172.21.15.37:0/1712149004 learned_addr learned my addr 172.21.15.37:0/1712149004 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:35.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbb7fff640 1 --2- 172.21.15.37:0/1712149004 >> v2:172.21.15.203:3300/0 conn(0x7fcbb80cfca0 0x7fcbb8072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbb77fe640 1 -- 172.21.15.37:0/1712149004 >> v2:172.21.15.203:3300/0 conn(0x7fcbb80cfca0 msgr2=0x7fcbb8072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:35.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbb77fe640 1 --2- 172.21.15.37:0/1712149004 >> v2:172.21.15.203:3300/0 conn(0x7fcbb80cfca0 0x7fcbb8072000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbb77fe640 1 -- 172.21.15.37:0/1712149004 >> v1:172.21.15.203:6789/0 conn(0x7fcbb815b800 legacy=0x7fcbb80cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:35.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbb77fe640 1 -- 172.21.15.37:0/1712149004 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb80cc2e0 con 0x7fcbb815b420 2024-04-25T08:53:35.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbb7fff640 1 --2- 172.21.15.37:0/1712149004 >> v2:172.21.15.203:3300/0 conn(0x7fcbb80cfca0 0x7fcbb8072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:35.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.176+0000 7fcbb77fe640 1 --2- 172.21.15.37:0/1712149004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb815b420 0x7fcbb80ca3c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80712a0 tx=0x7fcba801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a82375aabffd0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:35.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb67fc640 1 -- 172.21.15.37:0/1712149004 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80a57e0 con 0x7fcbb815b420 2024-04-25T08:53:35.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb67fc640 1 -- 172.21.15.37:0/1712149004 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba80a5980 con 0x7fcbb815b420 2024-04-25T08:53:35.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb67fc640 1 -- 172.21.15.37:0/1712149004 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80a5d50 con 0x7fcbb815b420 2024-04-25T08:53:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 -- 172.21.15.37:0/1712149004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb815b420 msgr2=0x7fcbb80ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 --2- 172.21.15.37:0/1712149004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb815b420 0x7fcbb80ca3c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcbb80712a0 tx=0x7fcba801c980 comp rx=0 tx=0).stop 2024-04-25T08:53:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 -- 172.21.15.37:0/1712149004 shutdown_connections 2024-04-25T08:53:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 --2- 172.21.15.37:0/1712149004 >> v2:172.21.15.203:3300/0 conn(0x7fcbb80cfca0 0x7fcbb8072000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 --2- 172.21.15.37:0/1712149004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb815b420 0x7fcbb80ca3c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 -- 172.21.15.37:0/1712149004 >> 172.21.15.37:0/1712149004 conn(0x7fcbb806ae60 msgr2=0x7fcbb806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 -- 172.21.15.37:0/1712149004 shutdown_connections 2024-04-25T08:53:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 -- 172.21.15.37:0/1712149004 wait complete. 2024-04-25T08:53:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 Processor -- start 2024-04-25T08:53:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 -- start start 2024-04-25T08:53:35.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb80cfca0 0x7fcbb80c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbb815b420 0x7fcbb80c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbb815b800 0x7fcbb80c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb80caf30 con 0x7fcbb80cfca0 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb80cadb0 con 0x7fcbb815b800 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb8072d50 con 0x7fcbb815b420 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb80cfca0 0x7fcbb80c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb80cfca0 0x7fcbb80c2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47836/0 (socket says 172.21.15.37:47836) 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb77fe640 1 -- 172.21.15.37:0/75803229 learned_addr learned my addr 172.21.15.37:0/75803229 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb7fff640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbb815b800 0x7fcbb80c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb6ffd640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbb815b420 0x7fcbb80c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb77fe640 1 -- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbb815b420 msgr2=0x7fcbb80c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb77fe640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbb815b420 0x7fcbb80c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb77fe640 1 -- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbb815b800 msgr2=0x7fcbb80c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb77fe640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbb815b800 0x7fcbb80c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb77fe640 1 -- 172.21.15.37:0/75803229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb80ad020 con 0x7fcbb80cfca0 2024-04-25T08:53:35.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbb77fe640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb80cfca0 0x7fcbb80c2c50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcba80af040 tx=0x7fcba8019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:35.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbaffff640 1 -- 172.21.15.37:0/75803229 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80bc020 con 0x7fcbb80cfca0 2024-04-25T08:53:35.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbaffff640 1 -- 172.21.15.37:0/75803229 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba80b88a0 con 0x7fcbb80cfca0 2024-04-25T08:53:35.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.180+0000 7fcbbdde5640 1 -- 172.21.15.37:0/75803229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb80acd40 con 0x7fcbb80cfca0 2024-04-25T08:53:35.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.184+0000 7fcbaffff640 1 -- 172.21.15.37:0/75803229 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80b8c90 con 0x7fcbb80cfca0 2024-04-25T08:53:35.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.184+0000 7fcbbdde5640 1 -- 172.21.15.37:0/75803229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb8153da0 con 0x7fcbb80cfca0 2024-04-25T08:53:35.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.184+0000 7fcbbdde5640 1 -- 172.21.15.37:0/75803229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb804efa0 con 0x7fcbb80cfca0 2024-04-25T08:53:35.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.184+0000 7fcbaffff640 1 -- 172.21.15.37:0/75803229 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcba80bf020 con 0x7fcbb80cfca0 2024-04-25T08:53:35.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.184+0000 7fcbaffff640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcb98066f60 0x7fcb98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.184+0000 7fcbaffff640 1 -- 172.21.15.37:0/75803229 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcba8120690 con 0x7fcbb80cfca0 2024-04-25T08:53:35.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.184+0000 7fcbb6ffd640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcb98066f60 0x7fcb98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.184+0000 7fcbb6ffd640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcb98066f60 0x7fcb98069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcbb806db00 tx=0x7fcba0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:35.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.188+0000 7fcbaffff640 1 -- 172.21.15.37:0/75803229 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcba80c6030 con 0x7fcbb80cfca0 2024-04-25T08:53:35.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 -- 172.21.15.37:0/75803229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcbb80c4f80 con 0x7fcbb80cfca0 2024-04-25T08:53:35.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbaffff640 1 -- 172.21.15.37:0/75803229 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcba80165a0 con 0x7fcbb80cfca0 2024-04-25T08:53:35.340 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:35.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 -- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcb98066f60 msgr2=0x7fcb98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcb98066f60 0x7fcb98069420 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcbb806db00 tx=0x7fcba0008040 comp rx=0 tx=0).stop 2024-04-25T08:53:35.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 -- 172.21.15.37:0/75803229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb80cfca0 msgr2=0x7fcbb80c2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb80cfca0 0x7fcbb80c2c50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcba80af040 tx=0x7fcba8019720 comp rx=0 tx=0).stop 2024-04-25T08:53:35.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 -- 172.21.15.37:0/75803229 shutdown_connections 2024-04-25T08:53:35.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcb98066f60 0x7fcb98069420 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbb815b800 0x7fcbb80c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbb815b420 0x7fcbb80c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 --2- 172.21.15.37:0/75803229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbb80cfca0 0x7fcbb80c2c50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 -- 172.21.15.37:0/75803229 >> 172.21.15.37:0/75803229 conn(0x7fcbb806ae60 msgr2=0x7fcbb806df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:35.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 -- 172.21.15.37:0/75803229 shutdown_connections 2024-04-25T08:53:35.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.348+0000 7fcbbdde5640 1 -- 172.21.15.37:0/75803229 wait complete. 2024-04-25T08:53:35.359 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14fd43640 1 Processor -- start 2024-04-25T08:53:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14fd43640 1 -- start start 2024-04-25T08:53:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14fd43640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd1480cf590 0x7fd1480cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14fd43640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1480cbd20 con 0x7fd148072000 2024-04-25T08:53:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14fd43640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1480cc000 con 0x7fd14815b6b0 2024-04-25T08:53:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14fd43640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14815ba90 con 0x7fd1480cf590 2024-04-25T08:53:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14e2e0640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd148072000 0x7fd148072430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52564/0 (socket says 172.21.15.37:52564) 2024-04-25T08:53:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14e2e0640 1 -- 172.21.15.37:0/1861018344 learned_addr learned my addr 172.21.15.37:0/1861018344 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:35.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14d2de640 1 --2- 172.21.15.37:0/1861018344 >> v2:172.21.15.203:3300/0 conn(0x7fd1480cf590 0x7fd1480cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380771701 0 0) 0x7fd1480cbd20 con 0x7fd148072000 2024-04-25T08:53:35.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd134003530 con 0x7fd148072000 2024-04-25T08:53:35.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3483313052 0 0) 0x7fd134003530 con 0x7fd148072000 2024-04-25T08:53:35.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 >> v2:172.21.15.203:3300/0 conn(0x7fd1480cf590 msgr2=0x7fd1480cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:35.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 --2- 172.21.15.37:0/1861018344 >> v2:172.21.15.203:3300/0 conn(0x7fd1480cf590 0x7fd1480cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 >> v1:172.21.15.203:6789/0 conn(0x7fd14815b6b0 legacy=0x7fd1480ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480cc2e0 con 0x7fd148072000 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fd140001a00 con 0x7fd148072000 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 >> v1:172.21.15.37:6789/0 conn(0x7fd148072000 legacy=0x7fd148072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 --2- 172.21.15.37:0/1861018344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd134003a30 0x7fd134003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 --2- 172.21.15.37:0/1861018344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd134004810 0x7fd134004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 --2- 172.21.15.37:0/1861018344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1340051d0 0x7fd13400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14815ba90 con 0x7fd134003a30 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13400c100 con 0x7fd1340051d0 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13400c300 con 0x7fd134004810 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14dadf640 1 --2- 172.21.15.37:0/1861018344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd134003a30 0x7fd134003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14e2e0640 1 --2- 172.21.15.37:0/1861018344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1340051d0 0x7fd13400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14d2de640 1 --2- 172.21.15.37:0/1861018344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd134004810 0x7fd134004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14dadf640 1 -- 172.21.15.37:0/1861018344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd134004810 msgr2=0x7fd134004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14dadf640 1 --2- 172.21.15.37:0/1861018344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd134004810 0x7fd134004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14dadf640 1 -- 172.21.15.37:0/1861018344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1340051d0 msgr2=0x7fd13400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14dadf640 1 --2- 172.21.15.37:0/1861018344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd1340051d0 0x7fd13400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14dadf640 1 -- 172.21.15.37:0/1861018344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd134003530 con 0x7fd134003a30 2024-04-25T08:53:35.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.484+0000 7fd14e2e0640 1 -- 172.21.15.37:0/1861018344 reap_dead start 2024-04-25T08:53:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14dadf640 1 --2- 172.21.15.37:0/1861018344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd134003a30 0x7fd134003e50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd144001780 tx=0x7fd14401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a310b53cafab5ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1440013a0 con 0x7fd134003a30 2024-04-25T08:53:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1440052f0 con 0x7fd134003a30 2024-04-25T08:53:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14cadd640 1 -- 172.21.15.37:0/1861018344 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144003840 con 0x7fd134003a30 2024-04-25T08:53:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14fd43640 1 -- 172.21.15.37:0/1861018344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd134003a30 msgr2=0x7fd134003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14fd43640 1 --2- 172.21.15.37:0/1861018344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd134003a30 0x7fd134003e50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd144001780 tx=0x7fd14401c3d0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14fd43640 1 -- 172.21.15.37:0/1861018344 shutdown_connections 2024-04-25T08:53:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14fd43640 1 --2- 172.21.15.37:0/1861018344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd134003a30 0x7fd134003e50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14fd43640 1 -- 172.21.15.37:0/1861018344 >> 172.21.15.37:0/1861018344 conn(0x7fd14806ae60 msgr2=0x7fd14806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14fd43640 1 -- 172.21.15.37:0/1861018344 shutdown_connections 2024-04-25T08:53:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14fd43640 1 -- 172.21.15.37:0/1861018344 wait complete. 2024-04-25T08:53:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.488+0000 7fd14fd43640 1 Processor -- start 2024-04-25T08:53:35.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14fd43640 1 -- start start 2024-04-25T08:53:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14fd43640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd134003a30 0x7fd1480c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14fd43640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd14815a0e0 0x7fd1480c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14fd43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd148098880 0x7fd1480c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14fd43640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480cff00 con 0x7fd148098880 2024-04-25T08:53:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14fd43640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480cfd80 con 0x7fd14815a0e0 2024-04-25T08:53:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14fd43640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480cadb0 con 0x7fd134003a30 2024-04-25T08:53:35.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14e2e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd148098880 0x7fd1480c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14e2e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd148098880 0x7fd1480c4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47850/0 (socket says 172.21.15.37:47850) 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14e2e0640 1 -- 172.21.15.37:0/239111302 learned_addr learned my addr 172.21.15.37:0/239111302 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14d2de640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd14815a0e0 0x7fd1480c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14e2e0640 1 -- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd134003a30 msgr2=0x7fd1480c2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14e2e0640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd134003a30 0x7fd1480c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14e2e0640 1 -- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd14815a0e0 msgr2=0x7fd1480c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14e2e0640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd14815a0e0 0x7fd1480c4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14e2e0640 1 -- 172.21.15.37:0/239111302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14806ee00 con 0x7fd148098880 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14e2e0640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd148098880 0x7fd1480c4bd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd1480c2de0 tx=0x7fd140007ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd13e7fc640 1 -- 172.21.15.37:0/239111302 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14000ea20 con 0x7fd148098880 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd13e7fc640 1 -- 172.21.15.37:0/239111302 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd140018070 con 0x7fd148098880 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd13e7fc640 1 -- 172.21.15.37:0/239111302 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14000ee00 con 0x7fd148098880 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14fd43640 1 -- 172.21.15.37:0/239111302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1480c33b0 con 0x7fd148098880 2024-04-25T08:53:35.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14fd43640 1 -- 172.21.15.37:0/239111302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1480c3930 con 0x7fd148098880 2024-04-25T08:53:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd14fd43640 1 -- 172.21.15.37:0/239111302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14815c3b0 con 0x7fd148098880 2024-04-25T08:53:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd13e7fc640 1 -- 172.21.15.37:0/239111302 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd14001d460 con 0x7fd148098880 2024-04-25T08:53:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd13e7fc640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd12c066f90 0x7fd12c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.492+0000 7fd13e7fc640 1 -- 172.21.15.37:0/239111302 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd1400856a0 con 0x7fd148098880 2024-04-25T08:53:35.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.496+0000 7fd14dadf640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd12c066f90 0x7fd12c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.496+0000 7fd14dadf640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd12c066f90 0x7fd12c069450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd144001750 tx=0x7fd144001130 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:35.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.500+0000 7fd13e7fc640 1 -- 172.21.15.37:0/239111302 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd14815c3b0 con 0x7fd148098880 2024-04-25T08:53:35.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.656+0000 7fd14fd43640 1 -- 172.21.15.37:0/239111302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd14804efa0 con 0x7fd148098880 2024-04-25T08:53:35.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd13e7fc640 1 -- 172.21.15.37:0/239111302 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd1400562b0 con 0x7fd148098880 2024-04-25T08:53:35.652 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 -- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd12c066f90 msgr2=0x7fd12c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd12c066f90 0x7fd12c069450 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd144001750 tx=0x7fd144001130 comp rx=0 tx=0).stop 2024-04-25T08:53:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 -- 172.21.15.37:0/239111302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd148098880 msgr2=0x7fd1480c4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd148098880 0x7fd1480c4bd0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd1480c2de0 tx=0x7fd140007ba0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 -- 172.21.15.37:0/239111302 shutdown_connections 2024-04-25T08:53:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd12c066f90 0x7fd12c069450 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd148098880 0x7fd1480c4bd0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd14815a0e0 0x7fd1480c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 --2- 172.21.15.37:0/239111302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd134003a30 0x7fd1480c2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 -- 172.21.15.37:0/239111302 >> 172.21.15.37:0/239111302 conn(0x7fd14806ae60 msgr2=0x7fd1480aef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 -- 172.21.15.37:0/239111302 shutdown_connections 2024-04-25T08:53:35.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.660+0000 7fd14fd43640 1 -- 172.21.15.37:0/239111302 wait complete. 2024-04-25T08:53:35.671 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e202a6640 1 Processor -- start 2024-04-25T08:53:35.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e202a6640 1 -- start start 2024-04-25T08:53:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e202a6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3e180cf590 0x7f3e180cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e202a6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3e18072000 0x7f3e18072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e202a6640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e180cbd20 con 0x7f3e1815b6b0 2024-04-25T08:53:35.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e202a6640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e1815ba90 con 0x7f3e18072000 2024-04-25T08:53:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e202a6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e180cc000 con 0x7f3e180cf590 2024-04-25T08:53:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e1e843640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3e18072000 0x7f3e18072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e1e042640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3e1815b6b0 0x7f3e180ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52572/0 (socket says 172.21.15.37:52572) 2024-04-25T08:53:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e1e042640 1 -- 172.21.15.37:0/1148708564 learned_addr learned my addr 172.21.15.37:0/1148708564 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e1d841640 1 --2- 172.21.15.37:0/1148708564 >> v2:172.21.15.203:3300/0 conn(0x7f3e180cf590 0x7f3e180cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e1e843640 1 -- 172.21.15.37:0/1148708564 >> v2:172.21.15.203:3300/0 conn(0x7f3e180cf590 msgr2=0x7f3e180cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e1e843640 1 --2- 172.21.15.37:0/1148708564 >> v2:172.21.15.203:3300/0 conn(0x7f3e180cf590 0x7f3e180cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e1e843640 1 -- 172.21.15.37:0/1148708564 >> v1:172.21.15.37:6789/0 conn(0x7f3e1815b6b0 legacy=0x7f3e180ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.804+0000 7f3e1e843640 1 -- 172.21.15.37:0/1148708564 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e180cc2e0 con 0x7f3e18072000 2024-04-25T08:53:35.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e1e843640 1 --2- 172.21.15.37:0/1148708564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18072000 0x7f3e18072430 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3e180b4430 tx=0x7f3e1001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed50e1d27cf3c3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:35.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e1d040640 1 -- 172.21.15.37:0/1148708564 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e100b2070 con 0x7f3e18072000 2024-04-25T08:53:35.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e1d040640 1 -- 172.21.15.37:0/1148708564 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e100a58e0 con 0x7f3e18072000 2024-04-25T08:53:35.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e1d040640 1 -- 172.21.15.37:0/1148708564 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e100a5cb0 con 0x7f3e18072000 2024-04-25T08:53:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 -- 172.21.15.37:0/1148708564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18072000 msgr2=0x7f3e18072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 --2- 172.21.15.37:0/1148708564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18072000 0x7f3e18072430 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3e180b4430 tx=0x7f3e1001c930 comp rx=0 tx=0).stop 2024-04-25T08:53:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 -- 172.21.15.37:0/1148708564 shutdown_connections 2024-04-25T08:53:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 --2- 172.21.15.37:0/1148708564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18072000 0x7f3e18072430 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 --2- 172.21.15.37:0/1148708564 >> v2:172.21.15.203:3300/0 conn(0x7f3e180cf590 0x7f3e180cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 -- 172.21.15.37:0/1148708564 >> 172.21.15.37:0/1148708564 conn(0x7f3e1806ae60 msgr2=0x7f3e1806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 -- 172.21.15.37:0/1148708564 shutdown_connections 2024-04-25T08:53:35.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 -- 172.21.15.37:0/1148708564 wait complete. 2024-04-25T08:53:35.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 Processor -- start 2024-04-25T08:53:35.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 -- start start 2024-04-25T08:53:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18072000 0x7f3e18159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e180cf590 0x7f3e1806e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e1815b6b0 0x7f3e1815c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180caf30 con 0x7f3e18072000 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180cadb0 con 0x7f3e1815b6b0 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e202a6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e181682a0 con 0x7f3e180cf590 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e1e042640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18072000 0x7f3e18159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e1e042640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18072000 0x7f3e18159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47868/0 (socket says 172.21.15.37:47868) 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e1e042640 1 -- 172.21.15.37:0/361733985 learned_addr learned my addr 172.21.15.37:0/361733985 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e1e843640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e1815b6b0 0x7f3e1815c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e1d841640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e180cf590 0x7f3e1806e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e1e042640 1 -- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e180cf590 msgr2=0x7f3e1806e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.808+0000 7f3e1e042640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e180cf590 0x7f3e1806e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e1e042640 1 -- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e1815b6b0 msgr2=0x7f3e1815c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e1e042640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e1815b6b0 0x7f3e1815c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e1e042640 1 -- 172.21.15.37:0/361733985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e18072530 con 0x7f3e18072000 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e1e042640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18072000 0x7f3e18159de0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400c3a0 tx=0x7f3e1400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e0effd640 1 -- 172.21.15.37:0/361733985 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e14017070 con 0x7f3e18072000 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e0effd640 1 -- 172.21.15.37:0/361733985 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e14005cf0 con 0x7f3e18072000 2024-04-25T08:53:35.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e202a6640 1 -- 172.21.15.37:0/361733985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1815ffb0 con 0x7f3e18072000 2024-04-25T08:53:35.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e0effd640 1 -- 172.21.15.37:0/361733985 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e140072f0 con 0x7f3e18072000 2024-04-25T08:53:35.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e202a6640 1 -- 172.21.15.37:0/361733985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1815a5c0 con 0x7f3e18072000 2024-04-25T08:53:35.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e202a6640 1 -- 172.21.15.37:0/361733985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1804efa0 con 0x7f3e18072000 2024-04-25T08:53:35.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e0effd640 1 -- 172.21.15.37:0/361733985 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3e1401a070 con 0x7f3e18072000 2024-04-25T08:53:35.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e0effd640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3dfc066f60 0x7f3dfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:35.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e0effd640 1 -- 172.21.15.37:0/361733985 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f3e140847f0 con 0x7f3e18072000 2024-04-25T08:53:35.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e1d841640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3dfc066f60 0x7f3dfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:35.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.812+0000 7f3e1d841640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3dfc066f60 0x7f3dfc069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009790 tx=0x7f3e08008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:35.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.816+0000 7f3e0effd640 1 -- 172.21.15.37:0/361733985 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3e140553e0 con 0x7f3e18072000 2024-04-25T08:53:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.976+0000 7f3e202a6640 1 -- 172.21.15.37:0/361733985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e18073730 con 0x7f3e18072000 2024-04-25T08:53:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.976+0000 7f3e0effd640 1 -- 172.21.15.37:0/361733985 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e14054ce0 con 0x7f3e18072000 2024-04-25T08:53:35.969 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 -- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3dfc066f60 msgr2=0x7f3dfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3dfc066f60 0x7f3dfc069420 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009790 tx=0x7f3e08008040 comp rx=0 tx=0).stop 2024-04-25T08:53:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 -- 172.21.15.37:0/361733985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18072000 msgr2=0x7f3e18159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18072000 0x7f3e18159de0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400c3a0 tx=0x7f3e1400c860 comp rx=0 tx=0).stop 2024-04-25T08:53:35.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 -- 172.21.15.37:0/361733985 shutdown_connections 2024-04-25T08:53:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3dfc066f60 0x7f3dfc069420 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3e1815b6b0 0x7f3e1815c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3e180cf590 0x7f3e1806e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 --2- 172.21.15.37:0/361733985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18072000 0x7f3e18159de0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 -- 172.21.15.37:0/361733985 >> 172.21.15.37:0/361733985 conn(0x7f3e1806ae60 msgr2=0x7f3e181562c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 -- 172.21.15.37:0/361733985 shutdown_connections 2024-04-25T08:53:35.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:35.980+0000 7f3e202a6640 1 -- 172.21.15.37:0/361733985 wait complete. 2024-04-25T08:53:35.989 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:36.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.116+0000 7f4f7377a640 1 Processor -- start 2024-04-25T08:53:36.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.116+0000 7f4f7377a640 1 -- start start 2024-04-25T08:53:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f7377a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0af280 0x7f4f6c15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f7377a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4f6c06c3b0 0x7f4f6c06df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f7377a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6c066c30 con 0x7f4f6c0b2c30 2024-04-25T08:53:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f7377a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6c06daa0 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f7377a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c0af660 con 0x7f4f6c0af280 2024-04-25T08:53:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f71516640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0af280 0x7f4f6c15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f71d17640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4f6c06c3b0 0x7f4f6c06df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f70d15640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4f6c0b2c30 0x7f4f6c06bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:55800/0 (socket says 172.21.15.37:55800) 2024-04-25T08:53:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f70d15640 1 -- 172.21.15.37:0/617270721 learned_addr learned my addr 172.21.15.37:0/617270721 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f71516640 1 -- 172.21.15.37:0/617270721 >> v2:172.21.15.203:3300/0 conn(0x7f4f6c06c3b0 msgr2=0x7f4f6c06df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f71516640 1 --2- 172.21.15.37:0/617270721 >> v2:172.21.15.203:3300/0 conn(0x7f4f6c06c3b0 0x7f4f6c06df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f71516640 1 -- 172.21.15.37:0/617270721 >> v1:172.21.15.203:6789/0 conn(0x7f4f6c0b2c30 legacy=0x7f4f6c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f71516640 1 -- 172.21.15.37:0/617270721 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c15e640 con 0x7f4f6c0af280 2024-04-25T08:53:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.120+0000 7f4f71516640 1 --2- 172.21.15.37:0/617270721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0af280 0x7f4f6c15df10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4f680097c0 tx=0x7f4f6801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baa773a6a498afed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f63fff640 1 -- 172.21.15.37:0/617270721 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f680b0070 con 0x7f4f6c0af280 2024-04-25T08:53:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f63fff640 1 -- 172.21.15.37:0/617270721 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f680ab600 con 0x7f4f6c0af280 2024-04-25T08:53:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f63fff640 1 -- 172.21.15.37:0/617270721 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f680ab9d0 con 0x7f4f6c0af280 2024-04-25T08:53:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 -- 172.21.15.37:0/617270721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0af280 msgr2=0x7f4f6c15df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 --2- 172.21.15.37:0/617270721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0af280 0x7f4f6c15df10 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4f680097c0 tx=0x7f4f6801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 -- 172.21.15.37:0/617270721 shutdown_connections 2024-04-25T08:53:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 --2- 172.21.15.37:0/617270721 >> v2:172.21.15.203:3300/0 conn(0x7f4f6c06c3b0 0x7f4f6c06df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 --2- 172.21.15.37:0/617270721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0af280 0x7f4f6c15df10 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 -- 172.21.15.37:0/617270721 >> 172.21.15.37:0/617270721 conn(0x7f4f6c071810 msgr2=0x7f4f6c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 -- 172.21.15.37:0/617270721 shutdown_connections 2024-04-25T08:53:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 -- 172.21.15.37:0/617270721 wait complete. 2024-04-25T08:53:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 Processor -- start 2024-04-25T08:53:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 -- start start 2024-04-25T08:53:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f6c06c3b0 0x7f4f6c14c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c0af280 0x7f4f6c14d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0b2c30 0x7f4f6c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c06f010 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c06ee90 con 0x7f4f6c0af280 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f7377a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c06f190 con 0x7f4f6c0b2c30 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f71516640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f6c06c3b0 0x7f4f6c14c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f71516640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f6c06c3b0 0x7f4f6c14c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47882/0 (socket says 172.21.15.37:47882) 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.124+0000 7f4f71516640 1 -- 172.21.15.37:0/1002256906 learned_addr learned my addr 172.21.15.37:0/1002256906 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f70d15640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c0af280 0x7f4f6c14d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f71d17640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0b2c30 0x7f4f6c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f71516640 1 -- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0b2c30 msgr2=0x7f4f6c15c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f71516640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0b2c30 0x7f4f6c15c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f71516640 1 -- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c0af280 msgr2=0x7f4f6c14d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f71516640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c0af280 0x7f4f6c14d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f71516640 1 -- 172.21.15.37:0/1002256906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c14df20 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f71d17640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0b2c30 0x7f4f6c15c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:36.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f71516640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f6c06c3b0 0x7f4f6c14c420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4f680ae040 tx=0x7f4f6801ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:36.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f61ffb640 1 -- 172.21.15.37:0/1002256906 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f680b0050 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f7377a640 1 -- 172.21.15.37:0/1002256906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c14e200 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f61ffb640 1 -- 172.21.15.37:0/1002256906 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f680b1040 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f61ffb640 1 -- 172.21.15.37:0/1002256906 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f680b5610 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f7377a640 1 -- 172.21.15.37:0/1002256906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c14e7c0 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f7377a640 1 -- 172.21.15.37:0/1002256906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c04efa0 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f61ffb640 1 -- 172.21.15.37:0/1002256906 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f680be030 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f61ffb640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f50066f90 0x7f4f50069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f61ffb640 1 -- 172.21.15.37:0/1002256906 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4f6811f900 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.128+0000 7f4f70d15640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f50066f90 0x7f4f50069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.132+0000 7f4f70d15640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f50066f90 0x7f4f50069450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c14f470 tx=0x7f4f5c007e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:36.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.136+0000 7f4f61ffb640 1 -- 172.21.15.37:0/1002256906 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f680c5060 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 -- 172.21.15.37:0/1002256906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4f6c0cf830 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f61ffb640 1 -- 172.21.15.37:0/1002256906 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4f68016090 con 0x7f4f6c06c3b0 2024-04-25T08:53:36.285 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:36.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 -- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f50066f90 msgr2=0x7f4f50069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f50066f90 0x7f4f50069450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c14f470 tx=0x7f4f5c007e90 comp rx=0 tx=0).stop 2024-04-25T08:53:36.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 -- 172.21.15.37:0/1002256906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f6c06c3b0 msgr2=0x7f4f6c14c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f6c06c3b0 0x7f4f6c14c420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4f680ae040 tx=0x7f4f6801ace0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 -- 172.21.15.37:0/1002256906 shutdown_connections 2024-04-25T08:53:36.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f50066f90 0x7f4f50069450 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f6c0b2c30 0x7f4f6c15c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f6c0af280 0x7f4f6c14d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 --2- 172.21.15.37:0/1002256906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f6c06c3b0 0x7f4f6c14c420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 -- 172.21.15.37:0/1002256906 >> 172.21.15.37:0/1002256906 conn(0x7f4f6c071810 msgr2=0x7f4f6c0af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:36.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 -- 172.21.15.37:0/1002256906 shutdown_connections 2024-04-25T08:53:36.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.292+0000 7f4f7377a640 1 -- 172.21.15.37:0/1002256906 wait complete. 2024-04-25T08:53:36.303 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:53:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f838e4e5640 1 Processor -- start 2024-04-25T08:53:36.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f838e4e5640 1 -- start start 2024-04-25T08:53:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f838e4e5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f838815a700 0x7f838815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f838e4e5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83880b3f50 0x7f83880cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f838e4e5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83880b2090 con 0x7f83880ce430 2024-04-25T08:53:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f838e4e5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838815f200 con 0x7f83880b3f50 2024-04-25T08:53:36.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f838e4e5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880b43e0 con 0x7f838815a700 2024-04-25T08:53:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f83877fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83880b3f50 0x7f83880cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f8387fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f838815a700 0x7f838815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f83877fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f83880b3f50 0x7f83880cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:55208/0 (socket says 172.21.15.37:55208) 2024-04-25T08:53:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f83877fe640 1 -- 172.21.15.37:0/2208400102 learned_addr learned my addr 172.21.15.37:0/2208400102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f838ca82640 1 --1- 172.21.15.37:0/2208400102 >> v1:172.21.15.203:6789/0 conn(0x7f83880ce430 0x7f83880b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:55810/0 (socket says 172.21.15.37:55810) 2024-04-25T08:53:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f8387fff640 1 -- 172.21.15.37:0/2208400102 >> v2:172.21.15.203:3300/0 conn(0x7f83880b3f50 msgr2=0x7f83880cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f8387fff640 1 --2- 172.21.15.37:0/2208400102 >> v2:172.21.15.203:3300/0 conn(0x7f83880b3f50 0x7f83880cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f8387fff640 1 -- 172.21.15.37:0/2208400102 >> v1:172.21.15.203:6789/0 conn(0x7f83880ce430 legacy=0x7f83880b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f8387fff640 1 -- 172.21.15.37:0/2208400102 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83880b46c0 con 0x7f838815a700 2024-04-25T08:53:36.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.424+0000 7f83877fe640 1 --2- 172.21.15.37:0/2208400102 >> v2:172.21.15.203:3300/0 conn(0x7f83880b3f50 0x7f83880cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:36.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f8387fff640 1 --2- 172.21.15.37:0/2208400102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f838815a700 0x7f838815ecc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f83780097c0 tx=0x7f837801c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba1950babfa3d323 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f8386ffd640 1 -- 172.21.15.37:0/2208400102 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83780b0070 con 0x7f838815a700 2024-04-25T08:53:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f8386ffd640 1 -- 172.21.15.37:0/2208400102 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8378004230 con 0x7f838815a700 2024-04-25T08:53:36.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f8386ffd640 1 -- 172.21.15.37:0/2208400102 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378003620 con 0x7f838815a700 2024-04-25T08:53:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 -- 172.21.15.37:0/2208400102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f838815a700 msgr2=0x7f838815ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 --2- 172.21.15.37:0/2208400102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f838815a700 0x7f838815ecc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f83780097c0 tx=0x7f837801c350 comp rx=0 tx=0).stop 2024-04-25T08:53:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 -- 172.21.15.37:0/2208400102 shutdown_connections 2024-04-25T08:53:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 --2- 172.21.15.37:0/2208400102 >> v2:172.21.15.203:3300/0 conn(0x7f83880b3f50 0x7f83880cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 --2- 172.21.15.37:0/2208400102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f838815a700 0x7f838815ecc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 -- 172.21.15.37:0/2208400102 >> 172.21.15.37:0/2208400102 conn(0x7f83880b11a0 msgr2=0x7f83880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 -- 172.21.15.37:0/2208400102 shutdown_connections 2024-04-25T08:53:36.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 -- 172.21.15.37:0/2208400102 wait complete. 2024-04-25T08:53:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 Processor -- start 2024-04-25T08:53:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 -- start start 2024-04-25T08:53:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83880ce430 0x7f83880b0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f838815a700 0x7f8388153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f838814bdc0 0x7f838813fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83880cd490 con 0x7f838815a700 2024-04-25T08:53:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83880cd310 con 0x7f838814bdc0 2024-04-25T08:53:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.428+0000 7f838e4e5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83880b2bb0 con 0x7f83880ce430 2024-04-25T08:53:36.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f83877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f838815a700 0x7f8388153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f83877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f838815a700 0x7f8388153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47898/0 (socket says 172.21.15.37:47898) 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f83877fe640 1 -- 172.21.15.37:0/61804005 learned_addr learned my addr 172.21.15.37:0/61804005 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f8387fff640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83880ce430 0x7f83880b0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f838ca82640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f838814bdc0 0x7f838813fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f83877fe640 1 -- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83880ce430 msgr2=0x7f83880b0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f83877fe640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83880ce430 0x7f83880b0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f83877fe640 1 -- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f838814bdc0 msgr2=0x7f838813fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f83877fe640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f838814bdc0 0x7f838813fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f83877fe640 1 -- 172.21.15.37:0/61804005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8388151e10 con 0x7f838815a700 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f838ca82640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f838814bdc0 0x7f838813fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f83877fe640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f838815a700 0x7f8388153410 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f837c00e930 tx=0x7f837c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f8384ff9640 1 -- 172.21.15.37:0/61804005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c017070 con 0x7f838815a700 2024-04-25T08:53:36.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f838e4e5640 1 -- 172.21.15.37:0/61804005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83880b0fb0 con 0x7f838815a700 2024-04-25T08:53:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f8384ff9640 1 -- 172.21.15.37:0/61804005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c00ca00 con 0x7f838815a700 2024-04-25T08:53:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f8384ff9640 1 -- 172.21.15.37:0/61804005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c00cdd0 con 0x7f838815a700 2024-04-25T08:53:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f838e4e5640 1 -- 172.21.15.37:0/61804005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838815ad80 con 0x7f838815a700 2024-04-25T08:53:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f838e4e5640 1 -- 172.21.15.37:0/61804005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838804efa0 con 0x7f838815a700 2024-04-25T08:53:36.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f8384ff9640 1 -- 172.21.15.37:0/61804005 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f837c01a070 con 0x7f838815a700 2024-04-25T08:53:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f8384ff9640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8374066f60 0x7f8374069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f8384ff9640 1 -- 172.21.15.37:0/61804005 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f837c0852c0 con 0x7f838815a700 2024-04-25T08:53:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f8387fff640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8374066f60 0x7f8374069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.432+0000 7f8387fff640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8374066f60 0x7f8374069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f83780ad000 tx=0x7f8378008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:36.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.436+0000 7f8384ff9640 1 -- 172.21.15.37:0/61804005 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f837c055ee0 con 0x7f838815a700 2024-04-25T08:53:36.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.604+0000 7f838e4e5640 1 -- 172.21.15.37:0/61804005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f83880b1940 con 0x7f838815a700 2024-04-25T08:53:36.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.604+0000 7f8384ff9640 1 -- 172.21.15.37:0/61804005 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f837c0557e0 con 0x7f838815a700 2024-04-25T08:53:36.599 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:53:36.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 -- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8374066f60 msgr2=0x7f8374069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8374066f60 0x7f8374069420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f83780ad000 tx=0x7f8378008040 comp rx=0 tx=0).stop 2024-04-25T08:53:36.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 -- 172.21.15.37:0/61804005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f838815a700 msgr2=0x7f8388153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f838815a700 0x7f8388153410 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f837c00e930 tx=0x7f837c00edf0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 -- 172.21.15.37:0/61804005 shutdown_connections 2024-04-25T08:53:36.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8374066f60 0x7f8374069420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f838814bdc0 0x7f838813fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f838815a700 0x7f8388153410 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 --2- 172.21.15.37:0/61804005 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f83880ce430 0x7f83880b0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 -- 172.21.15.37:0/61804005 >> 172.21.15.37:0/61804005 conn(0x7f83880b11a0 msgr2=0x7f838806d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:36.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 -- 172.21.15.37:0/61804005 shutdown_connections 2024-04-25T08:53:36.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.608+0000 7f838e4e5640 1 -- 172.21.15.37:0/61804005 wait complete. 2024-04-25T08:53:36.616 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T08:53:36.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:36.617 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T08:53:36.629 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:36.630 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:53:36.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:36.630 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:53:36.677 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:53:36.677 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:36.685 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:53:36.685 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:36.732 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:36.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b256ad640 1 Processor -- start 2024-04-25T08:53:36.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b256ad640 1 -- start start 2024-04-25T08:53:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b256ad640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9b20144ca0 0x7f9b2006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b256ad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9b2006ec60 0x7f9b20142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b256ad640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b200ccc70 con 0x7f9b200ce510 2024-04-25T08:53:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b256ad640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b200cc2b0 con 0x7f9b20144ca0 2024-04-25T08:53:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b256ad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b2006e290 con 0x7f9b2006ec60 2024-04-25T08:53:36.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b17fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9b20144ca0 0x7f9b2006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b17fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9b20144ca0 0x7f9b2006e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47902/0 (socket says 172.21.15.37:47902) 2024-04-25T08:53:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b17fff640 1 -- 172.21.15.37:0/3937356322 learned_addr learned my addr 172.21.15.37:0/3937356322 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b1f7fe640 1 --2- 172.21.15.37:0/3937356322 >> v2:172.21.15.203:3300/0 conn(0x7f9b2006ec60 0x7f9b20142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b17fff640 1 -- 172.21.15.37:0/3937356322 >> v2:172.21.15.203:3300/0 conn(0x7f9b2006ec60 msgr2=0x7f9b20142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b17fff640 1 --2- 172.21.15.37:0/3937356322 >> v2:172.21.15.203:3300/0 conn(0x7f9b2006ec60 0x7f9b20142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b17fff640 1 -- 172.21.15.37:0/3937356322 >> v1:172.21.15.203:6789/0 conn(0x7f9b200ce510 legacy=0x7f9b200ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b17fff640 1 -- 172.21.15.37:0/3937356322 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2006dc40 con 0x7f9b20144ca0 2024-04-25T08:53:36.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.864+0000 7f9b1f7fe640 1 --2- 172.21.15.37:0/3937356322 >> v2:172.21.15.203:3300/0 conn(0x7f9b2006ec60 0x7f9b20142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b17fff640 1 --2- 172.21.15.37:0/3937356322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b20144ca0 0x7f9b2006e720 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9b200b48f0 tx=0x7f9b0801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3e5aa05c66b38b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b1e7fc640 1 -- 172.21.15.37:0/3937356322 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b080b5070 con 0x7f9b20144ca0 2024-04-25T08:53:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b1e7fc640 1 -- 172.21.15.37:0/3937356322 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b080a8a10 con 0x7f9b20144ca0 2024-04-25T08:53:36.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b1e7fc640 1 -- 172.21.15.37:0/3937356322 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b080a8de0 con 0x7f9b20144ca0 2024-04-25T08:53:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 -- 172.21.15.37:0/3937356322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b20144ca0 msgr2=0x7f9b2006e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 --2- 172.21.15.37:0/3937356322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b20144ca0 0x7f9b2006e720 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f9b200b48f0 tx=0x7f9b0801ed10 comp rx=0 tx=0).stop 2024-04-25T08:53:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 -- 172.21.15.37:0/3937356322 shutdown_connections 2024-04-25T08:53:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 --2- 172.21.15.37:0/3937356322 >> v2:172.21.15.203:3300/0 conn(0x7f9b2006ec60 0x7f9b20142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 --2- 172.21.15.37:0/3937356322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b20144ca0 0x7f9b2006e720 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 -- 172.21.15.37:0/3937356322 >> 172.21.15.37:0/3937356322 conn(0x7f9b200b11a0 msgr2=0x7f9b200b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 -- 172.21.15.37:0/3937356322 shutdown_connections 2024-04-25T08:53:36.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 -- 172.21.15.37:0/3937356322 wait complete. 2024-04-25T08:53:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 Processor -- start 2024-04-25T08:53:36.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 -- start start 2024-04-25T08:53:36.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b2006ec60 0x7f9b200c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b200ce510 0x7f9b200ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b20144ca0 0x7f9b2015b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b201451f0 con 0x7f9b200ce510 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200cc900 con 0x7f9b20144ca0 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b256ad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b20145370 con 0x7f9b2006ec60 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b200ce510 0x7f9b200ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b200ce510 0x7f9b200ca9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47904/0 (socket says 172.21.15.37:47904) 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b17fff640 1 -- 172.21.15.37:0/4278267915 learned_addr learned my addr 172.21.15.37:0/4278267915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b1f7fe640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b20144ca0 0x7f9b2015b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b1effd640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b2006ec60 0x7f9b200c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b17fff640 1 -- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b2006ec60 msgr2=0x7f9b200c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b17fff640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b2006ec60 0x7f9b200c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b17fff640 1 -- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b20144ca0 msgr2=0x7f9b2015b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b17fff640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b20144ca0 0x7f9b2015b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b17fff640 1 -- 172.21.15.37:0/4278267915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b200cb140 con 0x7f9b200ce510 2024-04-25T08:53:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.868+0000 7f9b1f7fe640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b20144ca0 0x7f9b2015b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b17fff640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b200ce510 0x7f9b200ca9d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9b08000c10 tx=0x7f9b0801bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b177fe640 1 -- 172.21.15.37:0/4278267915 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b080b5070 con 0x7f9b200ce510 2024-04-25T08:53:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b177fe640 1 -- 172.21.15.37:0/4278267915 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b080b0a50 con 0x7f9b200ce510 2024-04-25T08:53:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b256ad640 1 -- 172.21.15.37:0/4278267915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b200cb420 con 0x7f9b200ce510 2024-04-25T08:53:36.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b177fe640 1 -- 172.21.15.37:0/4278267915 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b080b0e20 con 0x7f9b200ce510 2024-04-25T08:53:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b256ad640 1 -- 172.21.15.37:0/4278267915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b200c5820 con 0x7f9b200ce510 2024-04-25T08:53:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b256ad640 1 -- 172.21.15.37:0/4278267915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2004efa0 con 0x7f9b200ce510 2024-04-25T08:53:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b177fe640 1 -- 172.21.15.37:0/4278267915 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b080c2060 con 0x7f9b200ce510 2024-04-25T08:53:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b177fe640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b00066f60 0x7f9b00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b177fe640 1 -- 172.21.15.37:0/4278267915 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9b081225b0 con 0x7f9b200ce510 2024-04-25T08:53:36.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b1effd640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b00066f60 0x7f9b00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.872+0000 7f9b1effd640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b00066f60 0x7f9b00069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9b10001650 tx=0x7f9b10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:36.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:36.876+0000 7f9b177fe640 1 -- 172.21.15.37:0/4278267915 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b08018870 con 0x7f9b200ce510 2024-04-25T08:53:37.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.036+0000 7f9b256ad640 1 -- 172.21.15.37:0/4278267915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b200b1cc0 con 0x7f9b200ce510 2024-04-25T08:53:37.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.040+0000 7f9b177fe640 1 -- 172.21.15.37:0/4278267915 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f9b080f31c0 con 0x7f9b200ce510 2024-04-25T08:53:37.034 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:37.034 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:37.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 -- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b00066f60 msgr2=0x7f9b00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b00066f60 0x7f9b00069420 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9b10001650 tx=0x7f9b10009040 comp rx=0 tx=0).stop 2024-04-25T08:53:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 -- 172.21.15.37:0/4278267915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b200ce510 msgr2=0x7f9b200ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b200ce510 0x7f9b200ca9d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9b08000c10 tx=0x7f9b0801bab0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 -- 172.21.15.37:0/4278267915 shutdown_connections 2024-04-25T08:53:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b00066f60 0x7f9b00069420 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b20144ca0 0x7f9b2015b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b200ce510 0x7f9b200ca9d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 --2- 172.21.15.37:0/4278267915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b2006ec60 0x7f9b200c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 -- 172.21.15.37:0/4278267915 >> 172.21.15.37:0/4278267915 conn(0x7f9b200b11a0 msgr2=0x7f9b2013fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:37.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 -- 172.21.15.37:0/4278267915 shutdown_connections 2024-04-25T08:53:37.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.044+0000 7f9b256ad640 1 -- 172.21.15.37:0/4278267915 wait complete. 2024-04-25T08:53:37.054 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:37.054 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8d02e18640 1 Processor -- start 2024-04-25T08:53:37.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8d02e18640 1 -- start start 2024-04-25T08:53:37.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8d02e18640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8cfc159fb0 0x7f8cfc0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8d02e18640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cfc0cb8c0 0x7f8cfc0cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8d02e18640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc0cc2b0 con 0x7f8cfc159bd0 2024-04-25T08:53:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8d02e18640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc072780 con 0x7f8cfc159fb0 2024-04-25T08:53:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8d02e18640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc071b70 con 0x7f8cfc0cb8c0 2024-04-25T08:53:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8cf3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8cfc159fb0 0x7f8cfc0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8cf3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8cfc159fb0 0x7f8cfc0cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47910/0 (socket says 172.21.15.37:47910) 2024-04-25T08:53:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8cf3fff640 1 -- 172.21.15.37:0/2834834084 learned_addr learned my addr 172.21.15.37:0/2834834084 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8d013b5640 1 --2- 172.21.15.37:0/2834834084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cfc0cb8c0 0x7f8cfc0cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8cf3fff640 1 -- 172.21.15.37:0/2834834084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cfc0cb8c0 msgr2=0x7f8cfc0cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8cf3fff640 1 --2- 172.21.15.37:0/2834834084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cfc0cb8c0 0x7f8cfc0cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8cf3fff640 1 -- 172.21.15.37:0/2834834084 >> v1:172.21.15.203:6789/0 conn(0x7f8cfc159bd0 legacy=0x7f8cfc0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.184+0000 7f8cf3fff640 1 -- 172.21.15.37:0/2834834084 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc071e50 con 0x7f8cfc159fb0 2024-04-25T08:53:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8cf3fff640 1 --2- 172.21.15.37:0/2834834084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cfc159fb0 0x7f8cfc0cb380 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8cec006fd0 tx=0x7f8cec01e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f66eec8dff29506 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8cf37fe640 1 -- 172.21.15.37:0/2834834084 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0ad070 con 0x7f8cfc159fb0 2024-04-25T08:53:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8cf37fe640 1 -- 172.21.15.37:0/2834834084 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec004230 con 0x7f8cfc159fb0 2024-04-25T08:53:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8cf37fe640 1 -- 172.21.15.37:0/2834834084 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec0a7810 con 0x7f8cfc159fb0 2024-04-25T08:53:37.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 -- 172.21.15.37:0/2834834084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cfc159fb0 msgr2=0x7f8cfc0cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 --2- 172.21.15.37:0/2834834084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cfc159fb0 0x7f8cfc0cb380 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8cec006fd0 tx=0x7f8cec01e9c0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 -- 172.21.15.37:0/2834834084 shutdown_connections 2024-04-25T08:53:37.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 --2- 172.21.15.37:0/2834834084 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cfc0cb8c0 0x7f8cfc0cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 --2- 172.21.15.37:0/2834834084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cfc159fb0 0x7f8cfc0cb380 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 -- 172.21.15.37:0/2834834084 >> 172.21.15.37:0/2834834084 conn(0x7f8cfc06ae60 msgr2=0x7f8cfc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:37.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 -- 172.21.15.37:0/2834834084 shutdown_connections 2024-04-25T08:53:37.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 -- 172.21.15.37:0/2834834084 wait complete. 2024-04-25T08:53:37.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 Processor -- start 2024-04-25T08:53:37.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 -- start start 2024-04-25T08:53:37.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cfc0cb8c0 0x7f8cfc0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cfc159bd0 0x7f8cfc15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cfc159fb0 0x7f8cfc06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc165af0 con 0x7f8cfc159fb0 2024-04-25T08:53:37.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc165970 con 0x7f8cfc0cb8c0 2024-04-25T08:53:37.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d02e18640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc165c70 con 0x7f8cfc159bd0 2024-04-25T08:53:37.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d013b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cfc159fb0 0x7f8cfc06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d013b5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cfc159fb0 0x7f8cfc06e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47912/0 (socket says 172.21.15.37:47912) 2024-04-25T08:53:37.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d013b5640 1 -- 172.21.15.37:0/3981925750 learned_addr learned my addr 172.21.15.37:0/3981925750 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:37.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d00bb4640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cfc0cb8c0 0x7f8cfc0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8cf3fff640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cfc159bd0 0x7f8cfc15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d013b5640 1 -- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cfc159bd0 msgr2=0x7f8cfc15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d013b5640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cfc159bd0 0x7f8cfc15cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d013b5640 1 -- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cfc0cb8c0 msgr2=0x7f8cfc0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d013b5640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cfc0cb8c0 0x7f8cfc0c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d013b5640 1 -- 172.21.15.37:0/3981925750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc0c6fa0 con 0x7f8cfc159fb0 2024-04-25T08:53:37.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.188+0000 7f8d013b5640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cfc159fb0 0x7f8cfc06e0d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4007e60 tx=0x7f8cf400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:37.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.192+0000 7f8cf17fa640 1 -- 172.21.15.37:0/3981925750 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4014420 con 0x7f8cfc159fb0 2024-04-25T08:53:37.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.192+0000 7f8cf17fa640 1 -- 172.21.15.37:0/3981925750 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf4019070 con 0x7f8cfc159fb0 2024-04-25T08:53:37.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.192+0000 7f8d02e18640 1 -- 172.21.15.37:0/3981925750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc0c7280 con 0x7f8cfc159fb0 2024-04-25T08:53:37.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.192+0000 7f8cf17fa640 1 -- 172.21.15.37:0/3981925750 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4014820 con 0x7f8cfc159fb0 2024-04-25T08:53:37.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.192+0000 7f8d02e18640 1 -- 172.21.15.37:0/3981925750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc0c7840 con 0x7f8cfc159fb0 2024-04-25T08:53:37.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.192+0000 7f8d02e18640 1 -- 172.21.15.37:0/3981925750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc04efa0 con 0x7f8cfc159fb0 2024-04-25T08:53:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.192+0000 7f8cf17fa640 1 -- 172.21.15.37:0/3981925750 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8cf40149c0 con 0x7f8cfc159fb0 2024-04-25T08:53:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.192+0000 7f8cf17fa640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ce8066f90 0x7f8ce8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.192+0000 7f8cf17fa640 1 -- 172.21.15.37:0/3981925750 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8cf4085c80 con 0x7f8cfc159fb0 2024-04-25T08:53:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.192+0000 7f8d00bb4640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ce8066f90 0x7f8ce8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.192+0000 7f8d00bb4640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ce8066f90 0x7f8ce8069450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc163930 tx=0x7f8cf8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.196+0000 7f8cf17fa640 1 -- 172.21.15.37:0/3981925750 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8cf4056890 con 0x7f8cfc159fb0 2024-04-25T08:53:37.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 -- 172.21.15.37:0/3981925750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8cfc0c9ea0 con 0x7f8cfc159fb0 2024-04-25T08:53:37.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8cf17fa640 1 -- 172.21.15.37:0/3981925750 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8cf4056190 con 0x7f8cfc159fb0 2024-04-25T08:53:37.349 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 -- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ce8066f90 msgr2=0x7f8ce8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ce8066f90 0x7f8ce8069450 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc163930 tx=0x7f8cf8009040 comp rx=0 tx=0).stop 2024-04-25T08:53:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 -- 172.21.15.37:0/3981925750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cfc159fb0 msgr2=0x7f8cfc06e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cfc159fb0 0x7f8cfc06e0d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4007e60 tx=0x7f8cf400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 -- 172.21.15.37:0/3981925750 shutdown_connections 2024-04-25T08:53:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ce8066f90 0x7f8ce8069450 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cfc159fb0 0x7f8cfc06e0d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8cfc159bd0 0x7f8cfc15cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 --2- 172.21.15.37:0/3981925750 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8cfc0cb8c0 0x7f8cfc0c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 -- 172.21.15.37:0/3981925750 >> 172.21.15.37:0/3981925750 conn(0x7f8cfc06ae60 msgr2=0x7f8cfc159100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 -- 172.21.15.37:0/3981925750 shutdown_connections 2024-04-25T08:53:37.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.356+0000 7f8d02e18640 1 -- 172.21.15.37:0/3981925750 wait complete. 2024-04-25T08:53:37.366 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f42622b2640 1 Processor -- start 2024-04-25T08:53:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f42622b2640 1 -- start start 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f42622b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c0b1ae0 0x7f425c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f42622b2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c160310 con 0x7f425c0cc820 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f42622b2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c15f550 con 0x7f425c0b0b00 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f42622b2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c15cfa0 con 0x7f425c0b1ae0 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f425c0cc820 0x7f425c0b13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52578/0 (socket says 172.21.15.37:52578) 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425b7fe640 1 -- 172.21.15.37:0/1931406832 learned_addr learned my addr 172.21.15.37:0/1931406832 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370985005 0 0) 0x7f425c160310 con 0x7f425c0cc820 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4244003530 con 0x7f425c0cc820 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f426084f640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c0b1ae0 0x7f425c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 909078324 0 0) 0x7f4244003530 con 0x7f425c0cc820 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c0b1ae0 msgr2=0x7f425c151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c0b1ae0 0x7f425c151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 >> v1:172.21.15.203:6789/0 conn(0x7f425c0b0b00 legacy=0x7f425c0cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c15f830 con 0x7f425c0cc820 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f4250001680 con 0x7f425c0cc820 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 >> v1:172.21.15.37:6789/0 conn(0x7f425c0cc820 legacy=0x7f425c0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4244003a60 0x7f4244003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c0b1ae0 0x7f4244004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4244004da0 0x7f4244005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424400bb60 con 0x7f4244004da0 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424400bd10 con 0x7f4244003a60 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f424400bf10 con 0x7f425c0b1ae0 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f426084f640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4244004da0 0x7f4244005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425b7fe640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c0b1ae0 0x7f4244004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425bfff640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4244003a60 0x7f4244003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f426084f640 1 -- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c0b1ae0 msgr2=0x7f4244004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f426084f640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c0b1ae0 0x7f4244004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f426084f640 1 -- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4244003a60 msgr2=0x7f4244003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f426084f640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4244003a60 0x7f4244003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f426084f640 1 -- 172.21.15.37:0/1931406832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4244003530 con 0x7f4244004da0 2024-04-25T08:53:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.500+0000 7f425bfff640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4244003a60 0x7f4244003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f426084f640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4244004da0 0x7f4244005220 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f425c1599d0 tx=0x7f425401c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7dbaeaf300a495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425400b610 con 0x7f4244004da0 2024-04-25T08:53:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42540b0070 con 0x7f4244004da0 2024-04-25T08:53:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f425affd640 1 -- 172.21.15.37:0/1931406832 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42540ab890 con 0x7f4244004da0 2024-04-25T08:53:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 -- 172.21.15.37:0/1931406832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4244004da0 msgr2=0x7f4244005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 --2- 172.21.15.37:0/1931406832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4244004da0 0x7f4244005220 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f425c1599d0 tx=0x7f425401c640 comp rx=0 tx=0).stop 2024-04-25T08:53:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f426084f640 1 -- 172.21.15.37:0/1931406832 reap_dead start 2024-04-25T08:53:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 -- 172.21.15.37:0/1931406832 shutdown_connections 2024-04-25T08:53:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 -- 172.21.15.37:0/1931406832 >> 172.21.15.37:0/1931406832 conn(0x7f425c0b1f10 msgr2=0x7f425c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 -- 172.21.15.37:0/1931406832 shutdown_connections 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 -- 172.21.15.37:0/1931406832 wait complete. 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 Processor -- start 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 -- start start 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c14d960 0x7f425c15a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c0c7830 0x7f425c15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f425c1f3650 0x7f425c0c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c15fe80 con 0x7f425c0c7830 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c06b950 con 0x7f425c1f3650 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f42622b2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0b0ee0 con 0x7f425c14d960 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f425b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c0c7830 0x7f425c15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f425b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c0c7830 0x7f425c15adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47930/0 (socket says 172.21.15.37:47930) 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.504+0000 7f425b7fe640 1 -- 172.21.15.37:0/106121089 learned_addr learned my addr 172.21.15.37:0/106121089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f425bfff640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c14d960 0x7f425c15a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f426084f640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f425c1f3650 0x7f425c0c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f425b7fe640 1 -- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c14d960 msgr2=0x7f425c15a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f425b7fe640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c14d960 0x7f425c15a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f425b7fe640 1 -- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f425c1f3650 msgr2=0x7f425c0c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f425b7fe640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f425c1f3650 0x7f425c0c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f425b7fe640 1 -- 172.21.15.37:0/106121089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0c7da0 con 0x7f425c0c7830 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f426084f640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f425c1f3650 0x7f425c0c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f425b7fe640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c0c7830 0x7f425c15adf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f425000eaa0 tx=0x7f425000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f4258ff9640 1 -- 172.21.15.37:0/106121089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425000c8a0 con 0x7f425c0c7830 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f4258ff9640 1 -- 172.21.15.37:0/106121089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4250018070 con 0x7f425c0c7830 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f42622b2640 1 -- 172.21.15.37:0/106121089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c0c8080 con 0x7f425c0c7830 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f4258ff9640 1 -- 172.21.15.37:0/106121089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425000cc80 con 0x7f425c0c7830 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f42622b2640 1 -- 172.21.15.37:0/106121089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c0ce3e0 con 0x7f425c0c7830 2024-04-25T08:53:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f42622b2640 1 -- 172.21.15.37:0/106121089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c0ce5b0 con 0x7f425c0c7830 2024-04-25T08:53:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f4258ff9640 1 -- 172.21.15.37:0/106121089 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f425001d460 con 0x7f425c0c7830 2024-04-25T08:53:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f4258ff9640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4248066f90 0x7f4248069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f4258ff9640 1 -- 172.21.15.37:0/106121089 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4250085270 con 0x7f425c0c7830 2024-04-25T08:53:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f425bfff640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4248066f90 0x7f4248069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.508+0000 7f425bfff640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4248066f90 0x7f4248069450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f424c007fd0 tx=0x7f424c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:37.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.512+0000 7f4258ff9640 1 -- 172.21.15.37:0/106121089 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f425000ce20 con 0x7f425c0c7830 2024-04-25T08:53:37.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.664+0000 7f42622b2640 1 -- 172.21.15.37:0/106121089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f425c0b6380 con 0x7f425c0c7830 2024-04-25T08:53:37.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.664+0000 7f4258ff9640 1 -- 172.21.15.37:0/106121089 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4250055f10 con 0x7f425c0c7830 2024-04-25T08:53:37.659 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:37.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 -- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4248066f90 msgr2=0x7f4248069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4248066f90 0x7f4248069450 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f424c007fd0 tx=0x7f424c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:37.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 -- 172.21.15.37:0/106121089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c0c7830 msgr2=0x7f425c15adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c0c7830 0x7f425c15adf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f425000eaa0 tx=0x7f425000ef60 comp rx=0 tx=0).stop 2024-04-25T08:53:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 -- 172.21.15.37:0/106121089 shutdown_connections 2024-04-25T08:53:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4248066f90 0x7f4248069450 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f425c1f3650 0x7f425c0c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f425c0c7830 0x7f425c15adf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 --2- 172.21.15.37:0/106121089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f425c14d960 0x7f425c15a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 -- 172.21.15.37:0/106121089 >> 172.21.15.37:0/106121089 conn(0x7f425c0b1f10 msgr2=0x7f425c06e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 -- 172.21.15.37:0/106121089 shutdown_connections 2024-04-25T08:53:37.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.668+0000 7f42622b2640 1 -- 172.21.15.37:0/106121089 wait complete. 2024-04-25T08:53:37.677 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d27030640 1 Processor -- start 2024-04-25T08:53:37.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d27030640 1 -- start start 2024-04-25T08:53:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d27030640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d200cfca0 0x7f2d20166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d27030640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d200cbd20 con 0x7f2d2015b420 2024-04-25T08:53:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d27030640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d200cc000 con 0x7f2d2015b800 2024-04-25T08:53:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d27030640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200cc930 con 0x7f2d200cfca0 2024-04-25T08:53:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d24dcc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2d2015b420 0x7f2d200ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52592/0 (socket says 172.21.15.37:52592) 2024-04-25T08:53:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d24dcc640 1 -- 172.21.15.37:0/1854039660 learned_addr learned my addr 172.21.15.37:0/1854039660 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d255cd640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d200cfca0 0x7f2d20166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1623745819 0 0) 0x7f2d200cbd20 con 0x7f2d2015b420 2024-04-25T08:53:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d10003530 con 0x7f2d2015b420 2024-04-25T08:53:37.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1798470187 0 0) 0x7f2d10003530 con 0x7f2d2015b420 2024-04-25T08:53:37.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d200cfca0 msgr2=0x7f2d20166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:37.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d200cfca0 0x7f2d20166290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 >> v1:172.21.15.203:6789/0 conn(0x7f2d2015b800 legacy=0x7f2d200cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d200cc2e0 con 0x7f2d2015b420 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f2d14001680 con 0x7f2d2015b420 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 >> v1:172.21.15.37:6789/0 conn(0x7f2d2015b420 legacy=0x7f2d200ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10003a60 0x7f2d10003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d10004810 0x7f2d10004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d200cfca0 0x7f2d100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d200cc930 con 0x7f2d10003a60 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1000bab0 con 0x7f2d10004810 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1000bcb0 con 0x7f2d200cfca0 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d24dcc640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10003a60 0x7f2d10003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d255cd640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d200cfca0 0x7f2d100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1ffff640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d10004810 0x7f2d10004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d24dcc640 1 -- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d200cfca0 msgr2=0x7f2d100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d24dcc640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d200cfca0 0x7f2d100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d24dcc640 1 -- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d10004810 msgr2=0x7f2d10004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:37.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d24dcc640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d10004810 0x7f2d10004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d24dcc640 1 -- 172.21.15.37:0/1854039660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d10003530 con 0x7f2d10003a60 2024-04-25T08:53:37.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d255cd640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d200cfca0 0x7f2d100051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:37.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.812+0000 7f2d1ffff640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d10004810 0x7f2d10004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d24dcc640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10003a60 0x7f2d10003e60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c210 tx=0x7f2d1401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f8732026baf00d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14007d20 con 0x7f2d10003a60 2024-04-25T08:53:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d140a9020 con 0x7f2d10003a60 2024-04-25T08:53:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1854039660 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14005b00 con 0x7f2d10003a60 2024-04-25T08:53:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 -- 172.21.15.37:0/1854039660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10003a60 msgr2=0x7f2d10003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 --2- 172.21.15.37:0/1854039660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d10003a60 0x7f2d10003e60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c210 tx=0x7f2d1401cc40 comp rx=0 tx=0).stop 2024-04-25T08:53:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d255cd640 1 -- 172.21.15.37:0/1854039660 reap_dead start 2024-04-25T08:53:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 -- 172.21.15.37:0/1854039660 shutdown_connections 2024-04-25T08:53:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 -- 172.21.15.37:0/1854039660 >> 172.21.15.37:0/1854039660 conn(0x7f2d2006ae60 msgr2=0x7f2d2006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 -- 172.21.15.37:0/1854039660 shutdown_connections 2024-04-25T08:53:37.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 -- 172.21.15.37:0/1854039660 wait complete. 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 Processor -- start 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 -- start start 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10003a60 0x7f2d20156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d201624a0 0x7f2d2015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d201629f0 0x7f2d2015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d20166c20 con 0x7f2d201629f0 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d20166aa0 con 0x7f2d201624a0 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.816+0000 7f2d27030640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d20166da0 con 0x7f2d10003a60 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d255cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d201629f0 0x7f2d2015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d255cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d201629f0 0x7f2d2015a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47956/0 (socket says 172.21.15.37:47956) 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d255cd640 1 -- 172.21.15.37:0/2107598096 learned_addr learned my addr 172.21.15.37:0/2107598096 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d1ffff640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d201624a0 0x7f2d2015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d24dcc640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10003a60 0x7f2d20156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d255cd640 1 -- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10003a60 msgr2=0x7f2d20156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d255cd640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10003a60 0x7f2d20156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d255cd640 1 -- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d201624a0 msgr2=0x7f2d2015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d255cd640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d201624a0 0x7f2d2015a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d255cd640 1 -- 172.21.15.37:0/2107598096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2015c7d0 con 0x7f2d201629f0 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d1ffff640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d201624a0 0x7f2d2015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d255cd640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d201629f0 0x7f2d2015a5c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009ed0 tx=0x7f2d18010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d1d7fa640 1 -- 172.21.15.37:0/2107598096 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18010e90 con 0x7f2d201629f0 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d1d7fa640 1 -- 172.21.15.37:0/2107598096 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d180039a0 con 0x7f2d201629f0 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d27030640 1 -- 172.21.15.37:0/2107598096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2015cab0 con 0x7f2d201629f0 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d1d7fa640 1 -- 172.21.15.37:0/2107598096 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d18003d90 con 0x7f2d201629f0 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d27030640 1 -- 172.21.15.37:0/2107598096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2015d070 con 0x7f2d201629f0 2024-04-25T08:53:37.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d27030640 1 -- 172.21.15.37:0/2107598096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2004efa0 con 0x7f2d201629f0 2024-04-25T08:53:37.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d1d7fa640 1 -- 172.21.15.37:0/2107598096 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d1801e020 con 0x7f2d201629f0 2024-04-25T08:53:37.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d1d7fa640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d04067100 0x7f2d040695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:37.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d1d7fa640 1 -- 172.21.15.37:0/2107598096 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2d18085a50 con 0x7f2d201629f0 2024-04-25T08:53:37.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d24dcc640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d04067100 0x7f2d040695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:37.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.820+0000 7f2d24dcc640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d04067100 0x7f2d040695c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2d140aa000 tx=0x7f2d200cc1d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:37.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.824+0000 7f2d1d7fa640 1 -- 172.21.15.37:0/2107598096 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d18056660 con 0x7f2d201629f0 2024-04-25T08:53:37.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.988+0000 7f2d27030640 1 -- 172.21.15.37:0/2107598096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2d200c60a0 con 0x7f2d201629f0 2024-04-25T08:53:37.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.988+0000 7f2d1d7fa640 1 -- 172.21.15.37:0/2107598096 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2d18055f60 con 0x7f2d201629f0 2024-04-25T08:53:37.981 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.988+0000 7f2d27030640 1 -- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d04067100 msgr2=0x7f2d040695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.988+0000 7f2d27030640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d04067100 0x7f2d040695c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2d140aa000 tx=0x7f2d200cc1d0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.992+0000 7f2d27030640 1 -- 172.21.15.37:0/2107598096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d201629f0 msgr2=0x7f2d2015a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.992+0000 7f2d27030640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d201629f0 0x7f2d2015a5c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009ed0 tx=0x7f2d18010590 comp rx=0 tx=0).stop 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.992+0000 7f2d27030640 1 -- 172.21.15.37:0/2107598096 shutdown_connections 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.992+0000 7f2d27030640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d04067100 0x7f2d040695c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.992+0000 7f2d27030640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d201629f0 0x7f2d2015a5c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.992+0000 7f2d27030640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d201624a0 0x7f2d2015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.992+0000 7f2d27030640 1 --2- 172.21.15.37:0/2107598096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d10003a60 0x7f2d20156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.992+0000 7f2d27030640 1 -- 172.21.15.37:0/2107598096 >> 172.21.15.37:0/2107598096 conn(0x7f2d2006ae60 msgr2=0x7f2d2015f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.992+0000 7f2d27030640 1 -- 172.21.15.37:0/2107598096 shutdown_connections 2024-04-25T08:53:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:37.992+0000 7f2d27030640 1 -- 172.21.15.37:0/2107598096 wait complete. 2024-04-25T08:53:38.001 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f42100ae640 1 Processor -- start 2024-04-25T08:53:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f42100ae640 1 -- start start 2024-04-25T08:53:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f42100ae640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42081634c0 0x7f42081658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f42100ae640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42080ca4c0 con 0x7f420815ecb0 2024-04-25T08:53:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f42100ae640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42080cd690 con 0x7f42080cf590 2024-04-25T08:53:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f42100ae640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42080ca780 con 0x7f42081634c0 2024-04-25T08:53:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420d649640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f420815ecb0 0x7f42080c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52594/0 (socket says 172.21.15.37:52594) 2024-04-25T08:53:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420d649640 1 -- 172.21.15.37:0/3929977303 learned_addr learned my addr 172.21.15.37:0/3929977303 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420e64b640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42081634c0 0x7f42081658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009365472 0 0) 0x7f42080ca4c0 con 0x7f420815ecb0 2024-04-25T08:53:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41f4003530 con 0x7f420815ecb0 2024-04-25T08:53:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2377619998 0 0) 0x7f41f4003530 con 0x7f420815ecb0 2024-04-25T08:53:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42081634c0 msgr2=0x7f42081658b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42081634c0 0x7f42081658b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 >> v1:172.21.15.203:6789/0 conn(0x7f42080cf590 legacy=0x7f42080cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42080cd970 con 0x7f420815ecb0 2024-04-25T08:53:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f41f8001680 con 0x7f420815ecb0 2024-04-25T08:53:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 >> v1:172.21.15.37:6789/0 conn(0x7f420815ecb0 legacy=0x7f42080c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41f4003a60 0x7f41f4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42081634c0 0x7f41f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41f4004d50 0x7f41f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f400bb60 con 0x7f41f4003a60 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f400bce0 con 0x7f41f4004d50 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f400bee0 con 0x7f42081634c0 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420de4a640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41f4003a60 0x7f41f4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420d649640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42081634c0 0x7f41f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420de4a640 1 -- 172.21.15.37:0/3929977303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42081634c0 msgr2=0x7f41f4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420e64b640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41f4004d50 0x7f41f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420de4a640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f42081634c0 0x7f41f4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420de4a640 1 -- 172.21.15.37:0/3929977303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41f4004d50 msgr2=0x7f41f40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420de4a640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41f4004d50 0x7f41f40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.132+0000 7f420de4a640 1 -- 172.21.15.37:0/3929977303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f4003530 con 0x7f41f4003a60 2024-04-25T08:53:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.136+0000 7f420de4a640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41f4003a60 0x7f41f4003e80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4204001750 tx=0x7f420401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3fcab6292c05be8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.136+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42040013a0 con 0x7f41f4003a60 2024-04-25T08:53:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.136+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42040052f0 con 0x7f41f4003a60 2024-04-25T08:53:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.136+0000 7f420ce48640 1 -- 172.21.15.37:0/3929977303 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4204003840 con 0x7f41f4003a60 2024-04-25T08:53:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.136+0000 7f42100ae640 1 -- 172.21.15.37:0/3929977303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41f4003a60 msgr2=0x7f41f4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.136+0000 7f42100ae640 1 --2- 172.21.15.37:0/3929977303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41f4003a60 0x7f41f4003e80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4204001750 tx=0x7f420401c3e0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.136+0000 7f420e64b640 1 -- 172.21.15.37:0/3929977303 reap_dead start 2024-04-25T08:53:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.136+0000 7f42100ae640 1 -- 172.21.15.37:0/3929977303 shutdown_connections 2024-04-25T08:53:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.136+0000 7f42100ae640 1 -- 172.21.15.37:0/3929977303 >> 172.21.15.37:0/3929977303 conn(0x7f420806ae60 msgr2=0x7f420806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.136+0000 7f42100ae640 1 -- 172.21.15.37:0/3929977303 shutdown_connections 2024-04-25T08:53:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.136+0000 7f42100ae640 1 -- 172.21.15.37:0/3929977303 wait complete. 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f42100ae640 1 Processor -- start 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f42100ae640 1 -- start start 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f42100ae640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41f4003a60 0x7f4208072060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f42100ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4208154ce0 0x7f4208154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f42100ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42080af050 0x7f42080af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f42100ae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42080cdce0 con 0x7f42080af050 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f42100ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42080cd440 con 0x7f4208154ce0 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f42100ae640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42081662d0 con 0x7f41f4003a60 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420e64b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42080af050 0x7f42080af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420e64b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42080af050 0x7f42080af4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47980/0 (socket says 172.21.15.37:47980) 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420e64b640 1 -- 172.21.15.37:0/2324579828 learned_addr learned my addr 172.21.15.37:0/2324579828 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420d649640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4208154ce0 0x7f4208154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420de4a640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41f4003a60 0x7f4208072060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420e64b640 1 -- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41f4003a60 msgr2=0x7f4208072060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420e64b640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41f4003a60 0x7f4208072060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420e64b640 1 -- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4208154ce0 msgr2=0x7f4208154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420e64b640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4208154ce0 0x7f4208154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420e64b640 1 -- 172.21.15.37:0/2324579828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4208073310 con 0x7f42080af050 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420d649640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4208154ce0 0x7f4208154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420de4a640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41f4003a60 0x7f4208072060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f420e64b640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42080af050 0x7f42080af4d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4200009fd0 tx=0x7f420000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f41fe7fc640 1 -- 172.21.15.37:0/2324579828 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420001c070 con 0x7f42080af050 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f41fe7fc640 1 -- 172.21.15.37:0/2324579828 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200017400 con 0x7f42080af050 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f42100ae640 1 -- 172.21.15.37:0/2324579828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420815ac70 con 0x7f42080af050 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f41fe7fc640 1 -- 172.21.15.37:0/2324579828 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000177f0 con 0x7f42080af050 2024-04-25T08:53:38.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f42100ae640 1 -- 172.21.15.37:0/2324579828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420815b230 con 0x7f42080af050 2024-04-25T08:53:38.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f42100ae640 1 -- 172.21.15.37:0/2324579828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420804efa0 con 0x7f42080af050 2024-04-25T08:53:38.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f41fe7fc640 1 -- 172.21.15.37:0/2324579828 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f420001f070 con 0x7f42080af050 2024-04-25T08:53:38.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.140+0000 7f41fe7fc640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41ec066f90 0x7f41ec069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.144+0000 7f41fe7fc640 1 -- 172.21.15.37:0/2324579828 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f42000854f0 con 0x7f42080af050 2024-04-25T08:53:38.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.144+0000 7f420de4a640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41ec066f90 0x7f41ec069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.144+0000 7f420de4a640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41ec066f90 0x7f41ec069450 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f42040ac000 tx=0x7f42040011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.148+0000 7f41fe7fc640 1 -- 172.21.15.37:0/2324579828 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4200012070 con 0x7f42080af050 2024-04-25T08:53:38.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 -- 172.21.15.37:0/2324579828 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f42080b4170 con 0x7f42080af050 2024-04-25T08:53:38.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f41fe7fc640 1 -- 172.21.15.37:0/2324579828 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4200056100 con 0x7f42080af050 2024-04-25T08:53:38.301 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:38.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 -- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41ec066f90 msgr2=0x7f41ec069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41ec066f90 0x7f41ec069450 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f42040ac000 tx=0x7f42040011a0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 -- 172.21.15.37:0/2324579828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42080af050 msgr2=0x7f42080af4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42080af050 0x7f42080af4d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4200009fd0 tx=0x7f420000f4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 -- 172.21.15.37:0/2324579828 shutdown_connections 2024-04-25T08:53:38.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41ec066f90 0x7f41ec069450 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f42080af050 0x7f42080af4d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4208154ce0 0x7f4208154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 --2- 172.21.15.37:0/2324579828 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41f4003a60 0x7f4208072060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 -- 172.21.15.37:0/2324579828 >> 172.21.15.37:0/2324579828 conn(0x7f420806ae60 msgr2=0x7f420815e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:38.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 -- 172.21.15.37:0/2324579828 shutdown_connections 2024-04-25T08:53:38.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.308+0000 7f42100ae640 1 -- 172.21.15.37:0/2324579828 wait complete. 2024-04-25T08:53:38.320 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:53:38.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4bf17ba640 1 Processor -- start 2024-04-25T08:53:38.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4bf17ba640 1 -- start start 2024-04-25T08:53:38.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4bf17ba640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bec0d07f0 0x7f4bec0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4bf17ba640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4bec0d1cd0 0x7f4bec06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4bf17ba640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bec06be60 con 0x7f4bec15da80 2024-04-25T08:53:38.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4bf17ba640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bec06c440 con 0x7f4bec0d1cd0 2024-04-25T08:53:38.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4bf17ba640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec15de60 con 0x7f4bec0d07f0 2024-04-25T08:53:38.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4beb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4bec0d1cd0 0x7f4bec06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4beb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4bec0d1cd0 0x7f4bec06c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47986/0 (socket says 172.21.15.37:47986) 2024-04-25T08:53:38.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4beb7fe640 1 -- 172.21.15.37:0/1561092619 learned_addr learned my addr 172.21.15.37:0/1561092619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:38.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4bea7fc640 1 --2- 172.21.15.37:0/1561092619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bec0d07f0 0x7f4bec0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4beb7fe640 1 -- 172.21.15.37:0/1561092619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bec0d07f0 msgr2=0x7f4bec0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:38.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4beb7fe640 1 --2- 172.21.15.37:0/1561092619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bec0d07f0 0x7f4bec0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4beb7fe640 1 -- 172.21.15.37:0/1561092619 >> v1:172.21.15.203:6789/0 conn(0x7f4bec15da80 legacy=0x7f4bec0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:38.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.448+0000 7f4beb7fe640 1 -- 172.21.15.37:0/1561092619 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec0d2450 con 0x7f4bec0d1cd0 2024-04-25T08:53:38.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4beb7fe640 1 --2- 172.21.15.37:0/1561092619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0d1cd0 0x7f4bec06c8d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009e50 tx=0x7f4be401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9c5a9a3191bdc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:38.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4be9ffb640 1 -- 172.21.15.37:0/1561092619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40a5820 con 0x7f4bec0d1cd0 2024-04-25T08:53:38.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4be9ffb640 1 -- 172.21.15.37:0/1561092619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be40a59c0 con 0x7f4bec0d1cd0 2024-04-25T08:53:38.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4be9ffb640 1 -- 172.21.15.37:0/1561092619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be40a5d90 con 0x7f4bec0d1cd0 2024-04-25T08:53:38.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 -- 172.21.15.37:0/1561092619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0d1cd0 msgr2=0x7f4bec06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 --2- 172.21.15.37:0/1561092619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0d1cd0 0x7f4bec06c8d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4be4009e50 tx=0x7f4be401c9c0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 -- 172.21.15.37:0/1561092619 shutdown_connections 2024-04-25T08:53:38.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 --2- 172.21.15.37:0/1561092619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0d1cd0 0x7f4bec06c8d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 --2- 172.21.15.37:0/1561092619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bec0d07f0 0x7f4bec0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 -- 172.21.15.37:0/1561092619 >> 172.21.15.37:0/1561092619 conn(0x7f4bec08b000 msgr2=0x7f4bec0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:38.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 -- 172.21.15.37:0/1561092619 shutdown_connections 2024-04-25T08:53:38.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 -- 172.21.15.37:0/1561092619 wait complete. 2024-04-25T08:53:38.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 Processor -- start 2024-04-25T08:53:38.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 -- start start 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0d07f0 0x7f4bec15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bec0d1cd0 0x7f4bec154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bec15da80 0x7f4bec155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec06d7a0 con 0x7f4bec0d07f0 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec06d620 con 0x7f4bec15da80 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bf17ba640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bec06d920 con 0x7f4bec0d1cd0 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4beaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0d07f0 0x7f4bec15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4beaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0d07f0 0x7f4bec15fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47988/0 (socket says 172.21.15.37:47988) 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4beaffd640 1 -- 172.21.15.37:0/3340057236 learned_addr learned my addr 172.21.15.37:0/3340057236 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4beb7fe640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bec15da80 0x7f4bec155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4bea7fc640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bec0d1cd0 0x7f4bec154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4beaffd640 1 -- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bec0d1cd0 msgr2=0x7f4bec154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4beaffd640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bec0d1cd0 0x7f4bec154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4beaffd640 1 -- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bec15da80 msgr2=0x7f4bec155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4beaffd640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bec15da80 0x7f4bec155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.452+0000 7f4beaffd640 1 -- 172.21.15.37:0/3340057236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bec0b6440 con 0x7f4bec0d07f0 2024-04-25T08:53:38.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4beaffd640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0d07f0 0x7f4bec15fe60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00b410 tx=0x7f4bdc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:38.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4bd37fe640 1 -- 172.21.15.37:0/3340057236 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc015030 con 0x7f4bec0d07f0 2024-04-25T08:53:38.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4bd37fe640 1 -- 172.21.15.37:0/3340057236 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc004020 con 0x7f4bec0d07f0 2024-04-25T08:53:38.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4bf17ba640 1 -- 172.21.15.37:0/3340057236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec0b6720 con 0x7f4bec0d07f0 2024-04-25T08:53:38.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4bd37fe640 1 -- 172.21.15.37:0/3340057236 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc004450 con 0x7f4bec0d07f0 2024-04-25T08:53:38.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4bf17ba640 1 -- 172.21.15.37:0/3340057236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bec0b6ce0 con 0x7f4bec0d07f0 2024-04-25T08:53:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4bf17ba640 1 -- 172.21.15.37:0/3340057236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bec0b6ee0 con 0x7f4bec0d07f0 2024-04-25T08:53:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4bd37fe640 1 -- 172.21.15.37:0/3340057236 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4bdc018070 con 0x7f4bec0d07f0 2024-04-25T08:53:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4bd37fe640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4bd8066f60 0x7f4bd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4bd37fe640 1 -- 172.21.15.37:0/3340057236 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4bdc084f50 con 0x7f4bec0d07f0 2024-04-25T08:53:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4bea7fc640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4bd8066f60 0x7f4bd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.456+0000 7f4bea7fc640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4bd8066f60 0x7f4bd8069420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0ce120 tx=0x7f4be0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.460+0000 7f4bd37fe640 1 -- 172.21.15.37:0/3340057236 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4bec0b6ee0 con 0x7f4bec0d07f0 2024-04-25T08:53:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.632+0000 7f4bf17ba640 1 -- 172.21.15.37:0/3340057236 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4bec04efa0 con 0x7f4bec0d07f0 2024-04-25T08:53:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.632+0000 7f4bd37fe640 1 -- 172.21.15.37:0/3340057236 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4bdc055bf0 con 0x7f4bec0d07f0 2024-04-25T08:53:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 -- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4bd8066f60 msgr2=0x7f4bd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4bd8066f60 0x7f4bd8069420 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0ce120 tx=0x7f4be0008040 comp rx=0 tx=0).stop 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 -- 172.21.15.37:0/3340057236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0d07f0 msgr2=0x7f4bec15fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0d07f0 0x7f4bec15fe60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00b410 tx=0x7f4bdc00b8d0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 -- 172.21.15.37:0/3340057236 shutdown_connections 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4bd8066f60 0x7f4bd8069420 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bec15da80 0x7f4bec155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bec0d1cd0 0x7f4bec154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 --2- 172.21.15.37:0/3340057236 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bec0d07f0 0x7f4bec15fe60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 -- 172.21.15.37:0/3340057236 >> 172.21.15.37:0/3340057236 conn(0x7f4bec08b000 msgr2=0x7f4bec0b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 -- 172.21.15.37:0/3340057236 shutdown_connections 2024-04-25T08:53:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.636+0000 7f4bf17ba640 1 -- 172.21.15.37:0/3340057236 wait complete. 2024-04-25T08:53:38.645 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:53:38.645 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:53:38.652 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70be1af640 1 Processor -- start 2024-04-25T08:53:38.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70be1af640 1 -- start start 2024-04-25T08:53:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70be1af640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f70b8071740 0x7f70b80acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70be1af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b80b24a0 0x7f70b80ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70be1af640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b80ae860 con 0x7f70b815da10 2024-04-25T08:53:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70be1af640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b815e210 con 0x7f70b8071740 2024-04-25T08:53:38.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70be1af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b815e4f0 con 0x7f70b80b24a0 2024-04-25T08:53:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70b6ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f70b8071740 0x7f70b80acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b80b24a0 0x7f70b80ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70b77fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f70b815da10 0x7f70b8071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:55882/0 (socket says 172.21.15.37:55882) 2024-04-25T08:53:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70b77fe640 1 -- 172.21.15.37:0/169852497 learned_addr learned my addr 172.21.15.37:0/169852497 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70b7fff640 1 -- 172.21.15.37:0/169852497 >> v2:172.21.15.203:3300/0 conn(0x7f70b8071740 msgr2=0x7f70b80acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70b7fff640 1 --2- 172.21.15.37:0/169852497 >> v2:172.21.15.203:3300/0 conn(0x7f70b8071740 0x7f70b80acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70b7fff640 1 -- 172.21.15.37:0/169852497 >> v1:172.21.15.203:6789/0 conn(0x7f70b815da10 legacy=0x7f70b8071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70b7fff640 1 -- 172.21.15.37:0/169852497 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b80ae4c0 con 0x7f70b80b24a0 2024-04-25T08:53:38.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.784+0000 7f70b6ffd640 1 --2- 172.21.15.37:0/169852497 >> v2:172.21.15.203:3300/0 conn(0x7f70b8071740 0x7f70b80acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:38.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70b7fff640 1 --2- 172.21.15.37:0/169852497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b80b24a0 0x7f70b80ad100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009d90 tx=0x7f70b001c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6282ea2677ab0fec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70b67fc640 1 -- 172.21.15.37:0/169852497 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b00b2070 con 0x7f70b80b24a0 2024-04-25T08:53:38.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70b67fc640 1 -- 172.21.15.37:0/169852497 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70b00a5710 con 0x7f70b80b24a0 2024-04-25T08:53:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70b67fc640 1 -- 172.21.15.37:0/169852497 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70b00a5ae0 con 0x7f70b80b24a0 2024-04-25T08:53:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70be1af640 1 -- 172.21.15.37:0/169852497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b80b24a0 msgr2=0x7f70b80ad100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70be1af640 1 --2- 172.21.15.37:0/169852497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b80b24a0 0x7f70b80ad100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f70b0009d90 tx=0x7f70b001c850 comp rx=0 tx=0).stop 2024-04-25T08:53:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70be1af640 1 -- 172.21.15.37:0/169852497 shutdown_connections 2024-04-25T08:53:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70be1af640 1 --2- 172.21.15.37:0/169852497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b80b24a0 0x7f70b80ad100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70be1af640 1 --2- 172.21.15.37:0/169852497 >> v2:172.21.15.203:3300/0 conn(0x7f70b8071740 0x7f70b80acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70be1af640 1 -- 172.21.15.37:0/169852497 >> 172.21.15.37:0/169852497 conn(0x7f70b80b11a0 msgr2=0x7f70b80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70be1af640 1 -- 172.21.15.37:0/169852497 shutdown_connections 2024-04-25T08:53:38.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70be1af640 1 -- 172.21.15.37:0/169852497 wait complete. 2024-04-25T08:53:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70be1af640 1 Processor -- start 2024-04-25T08:53:38.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.788+0000 7f70be1af640 1 -- start start 2024-04-25T08:53:38.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70be1af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b8071740 0x7f70b8150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70be1af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b80b24a0 0x7f70b815b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70be1af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b815da10 0x7f70b815fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70be1af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b815ef00 con 0x7f70b8071740 2024-04-25T08:53:38.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70be1af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b80adc60 con 0x7f70b815da10 2024-04-25T08:53:38.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70be1af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b816b0a0 con 0x7f70b80b24a0 2024-04-25T08:53:38.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b8071740 0x7f70b8150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b7fff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b815da10 0x7f70b815fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b8071740 0x7f70b8150130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47996/0 (socket says 172.21.15.37:47996) 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b77fe640 1 -- 172.21.15.37:0/2279514215 learned_addr learned my addr 172.21.15.37:0/2279514215 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b6ffd640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b80b24a0 0x7f70b815b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b77fe640 1 -- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b80b24a0 msgr2=0x7f70b815b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b77fe640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b80b24a0 0x7f70b815b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b77fe640 1 -- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b815da10 msgr2=0x7f70b815fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b77fe640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b815da10 0x7f70b815fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b77fe640 1 -- 172.21.15.37:0/2279514215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b815bad0 con 0x7f70b8071740 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b77fe640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b8071740 0x7f70b8150130 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00c370 tx=0x7f70ac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f709ffff640 1 -- 172.21.15.37:0/2279514215 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac017070 con 0x7f70b8071740 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f709ffff640 1 -- 172.21.15.37:0/2279514215 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ac005cf0 con 0x7f70b8071740 2024-04-25T08:53:38.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70be1af640 1 -- 172.21.15.37:0/2279514215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b815bdb0 con 0x7f70b8071740 2024-04-25T08:53:38.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f709ffff640 1 -- 172.21.15.37:0/2279514215 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac0072f0 con 0x7f70b8071740 2024-04-25T08:53:38.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70be1af640 1 -- 172.21.15.37:0/2279514215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b815c370 con 0x7f70b8071740 2024-04-25T08:53:38.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70be1af640 1 -- 172.21.15.37:0/2279514215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b804efa0 con 0x7f70b8071740 2024-04-25T08:53:38.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f709ffff640 1 -- 172.21.15.37:0/2279514215 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f70ac01a070 con 0x7f70b8071740 2024-04-25T08:53:38.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f709ffff640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70a4066f60 0x7f70a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:38.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f709ffff640 1 -- 172.21.15.37:0/2279514215 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f70ac084740 con 0x7f70b8071740 2024-04-25T08:53:38.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.792+0000 7f70b6ffd640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70a4066f60 0x7f70a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:38.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.796+0000 7f70b6ffd640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70a4066f60 0x7f70a4069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009580 tx=0x7f70a8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:38.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.796+0000 7f709ffff640 1 -- 172.21.15.37:0/2279514215 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f70ac054d00 con 0x7f70b8071740 2024-04-25T08:53:38.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.956+0000 7f70be1af640 1 -- 172.21.15.37:0/2279514215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70b80b1dc0 con 0x7f70b8071740 2024-04-25T08:53:38.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f709ffff640 1 -- 172.21.15.37:0/2279514215 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f70ac058da0 con 0x7f70b8071740 2024-04-25T08:53:38.952 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:38.953 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 -- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70a4066f60 msgr2=0x7f70a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70a4066f60 0x7f70a4069420 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f70a8009580 tx=0x7f70a8009190 comp rx=0 tx=0).stop 2024-04-25T08:53:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 -- 172.21.15.37:0/2279514215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b8071740 msgr2=0x7f70b8150130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b8071740 0x7f70b8150130 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f70ac00c370 tx=0x7f70ac00c830 comp rx=0 tx=0).stop 2024-04-25T08:53:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 -- 172.21.15.37:0/2279514215 shutdown_connections 2024-04-25T08:53:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70a4066f60 0x7f70a4069420 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70b815da10 0x7f70b815fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70b80b24a0 0x7f70b815b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 --2- 172.21.15.37:0/2279514215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70b8071740 0x7f70b8150130 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:38.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 -- 172.21.15.37:0/2279514215 >> 172.21.15.37:0/2279514215 conn(0x7f70b80b11a0 msgr2=0x7f70b8154830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:38.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 -- 172.21.15.37:0/2279514215 shutdown_connections 2024-04-25T08:53:38.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:38.960+0000 7f70be1af640 1 -- 172.21.15.37:0/2279514215 wait complete. 2024-04-25T08:53:38.971 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:38.971 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:39.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c74b0e640 1 Processor -- start 2024-04-25T08:53:39.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c74b0e640 1 -- start start 2024-04-25T08:53:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c74b0e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8c700cba90 0x7f8c700cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c74b0e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c70071b70 con 0x7f8c70159fb0 2024-04-25T08:53:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c74b0e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c70071e50 con 0x7f8c70159bd0 2024-04-25T08:53:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c74b0e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c70072780 con 0x7f8c700cba90 2024-04-25T08:53:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c65d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8c70159fb0 0x7f8c700cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52596/0 (socket says 172.21.15.37:52596) 2024-04-25T08:53:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c65d9b640 1 -- 172.21.15.37:0/1584310928 learned_addr learned my addr 172.21.15.37:0/1584310928 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c6ed9d640 1 --2- 172.21.15.37:0/1584310928 >> v2:172.21.15.37:3300/0 conn(0x7f8c700cba90 0x7f8c700cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c6dd9b640 1 -- 172.21.15.37:0/1584310928 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2244263472 0 0) 0x7f8c70071b70 con 0x7f8c70159fb0 2024-04-25T08:53:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c6dd9b640 1 -- 172.21.15.37:0/1584310928 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c5c003530 con 0x7f8c70159fb0 2024-04-25T08:53:39.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c6ed9d640 1 -- 172.21.15.37:0/1584310928 >> v1:172.21.15.203:6789/0 conn(0x7f8c70159bd0 legacy=0x7f8c700c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:39.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c6ed9d640 1 -- 172.21.15.37:0/1584310928 >> v1:172.21.15.37:6789/0 conn(0x7f8c70159fb0 legacy=0x7f8c700cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.096+0000 7f8c6ed9d640 1 -- 172.21.15.37:0/1584310928 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c70072130 con 0x7f8c700cba90 2024-04-25T08:53:39.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c6ed9d640 1 --2- 172.21.15.37:0/1584310928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c700cba90 0x7f8c700cbf40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009d90 tx=0x7f8c6801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82daa4e135951dce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:39.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c6dd9b640 1 -- 172.21.15.37:0/1584310928 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c680b2070 con 0x7f8c700cba90 2024-04-25T08:53:39.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c6dd9b640 1 -- 172.21.15.37:0/1584310928 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c680a59a0 con 0x7f8c700cba90 2024-04-25T08:53:39.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c6dd9b640 1 -- 172.21.15.37:0/1584310928 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c680a5d70 con 0x7f8c700cba90 2024-04-25T08:53:39.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c74b0e640 1 -- 172.21.15.37:0/1584310928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c700cba90 msgr2=0x7f8c700cbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c74b0e640 1 --2- 172.21.15.37:0/1584310928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c700cba90 0x7f8c700cbf40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009d90 tx=0x7f8c6801c940 comp rx=0 tx=0).stop 2024-04-25T08:53:39.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c74b0e640 1 -- 172.21.15.37:0/1584310928 shutdown_connections 2024-04-25T08:53:39.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c74b0e640 1 --2- 172.21.15.37:0/1584310928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c700cba90 0x7f8c700cbf40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c74b0e640 1 -- 172.21.15.37:0/1584310928 >> 172.21.15.37:0/1584310928 conn(0x7f8c7006ae60 msgr2=0x7f8c7006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c74b0e640 1 -- 172.21.15.37:0/1584310928 shutdown_connections 2024-04-25T08:53:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c74b0e640 1 -- 172.21.15.37:0/1584310928 wait complete. 2024-04-25T08:53:39.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c74b0e640 1 Processor -- start 2024-04-25T08:53:39.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.100+0000 7f8c74b0e640 1 -- start start 2024-04-25T08:53:39.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c74b0e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c700cba90 0x7f8c70161e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c74b0e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c70159bd0 0x7f8c70157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c74b0e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c70159fb0 0x7f8c70158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c74b0e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70167e30 con 0x7f8c70159bd0 2024-04-25T08:53:39.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c74b0e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70167cb0 con 0x7f8c700cba90 2024-04-25T08:53:39.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c74b0e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c70167fb0 con 0x7f8c70159fb0 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c65d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c70159bd0 0x7f8c70157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c65d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c70159bd0 0x7f8c70157250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48010/0 (socket says 172.21.15.37:48010) 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c65d9b640 1 -- 172.21.15.37:0/3596863656 learned_addr learned my addr 172.21.15.37:0/3596863656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c6e59c640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c700cba90 0x7f8c70161e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c65d9b640 1 -- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c70159fb0 msgr2=0x7f8c70158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c65d9b640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c70159fb0 0x7f8c70158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c65d9b640 1 -- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c700cba90 msgr2=0x7f8c70161e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c65d9b640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c700cba90 0x7f8c70161e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c65d9b640 1 -- 172.21.15.37:0/3596863656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c700c6430 con 0x7f8c70159bd0 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c6e59c640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c700cba90 0x7f8c70161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c65d9b640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c70159bd0 0x7f8c70157250 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8c58001ec0 tx=0x7f8c5800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c677fe640 1 -- 172.21.15.37:0/3596863656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58017070 con 0x7f8c70159bd0 2024-04-25T08:53:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c677fe640 1 -- 172.21.15.37:0/3596863656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58005d50 con 0x7f8c70159bd0 2024-04-25T08:53:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c74b0e640 1 -- 172.21.15.37:0/3596863656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7015b5a0 con 0x7f8c70159bd0 2024-04-25T08:53:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c677fe640 1 -- 172.21.15.37:0/3596863656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58007270 con 0x7f8c70159bd0 2024-04-25T08:53:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c74b0e640 1 -- 172.21.15.37:0/3596863656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7015bb60 con 0x7f8c70159bd0 2024-04-25T08:53:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c74b0e640 1 -- 172.21.15.37:0/3596863656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c7004efa0 con 0x7f8c70159bd0 2024-04-25T08:53:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c677fe640 1 -- 172.21.15.37:0/3596863656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c5801a070 con 0x7f8c70159bd0 2024-04-25T08:53:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c677fe640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c50066f60 0x7f8c50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c677fe640 1 -- 172.21.15.37:0/3596863656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8c58084650 con 0x7f8c70159bd0 2024-04-25T08:53:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.104+0000 7f8c6e59c640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c50066f60 0x7f8c50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.108+0000 7f8c6e59c640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c50066f60 0x7f8c50069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c600016f0 tx=0x7f8c60009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.108+0000 7f8c677fe640 1 -- 172.21.15.37:0/3596863656 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c58054d60 con 0x7f8c70159bd0 2024-04-25T08:53:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.264+0000 7f8c74b0e640 1 -- 172.21.15.37:0/3596863656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8c7015d380 con 0x7f8c70159bd0 2024-04-25T08:53:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.264+0000 7f8c677fe640 1 -- 172.21.15.37:0/3596863656 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8c58058e00 con 0x7f8c70159bd0 2024-04-25T08:53:39.258 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 -- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c50066f60 msgr2=0x7f8c50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c50066f60 0x7f8c50069420 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8c600016f0 tx=0x7f8c60009040 comp rx=0 tx=0).stop 2024-04-25T08:53:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 -- 172.21.15.37:0/3596863656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c70159bd0 msgr2=0x7f8c70157250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c70159bd0 0x7f8c70157250 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8c58001ec0 tx=0x7f8c5800c960 comp rx=0 tx=0).stop 2024-04-25T08:53:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 -- 172.21.15.37:0/3596863656 shutdown_connections 2024-04-25T08:53:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c50066f60 0x7f8c50069420 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c70159fb0 0x7f8c70158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c70159bd0 0x7f8c70157250 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 --2- 172.21.15.37:0/3596863656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c700cba90 0x7f8c70161e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 -- 172.21.15.37:0/3596863656 >> 172.21.15.37:0/3596863656 conn(0x7f8c7006ae60 msgr2=0x7f8c700b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:39.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 -- 172.21.15.37:0/3596863656 shutdown_connections 2024-04-25T08:53:39.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.268+0000 7f8c74b0e640 1 -- 172.21.15.37:0/3596863656 wait complete. 2024-04-25T08:53:39.275 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:39.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d7d245640 1 Processor -- start 2024-04-25T08:53:39.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d7d245640 1 -- start start 2024-04-25T08:53:39.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d7d245640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2d780b3cf0 0x7f2d780cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d7d245640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e6c0 0x7f2d780caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d7d245640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d780cbf10 con 0x7f2d780cfad0 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d7d245640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d780cff40 con 0x7f2d780b3cf0 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d7d245640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7815eae0 con 0x7f2d7815e6c0 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d76d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2d780b3cf0 0x7f2d780cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d76d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2d780b3cf0 0x7f2d780cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51670/0 (socket says 172.21.15.37:51670) 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d76d9d640 1 -- 172.21.15.37:0/2915221101 learned_addr learned my addr 172.21.15.37:0/2915221101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d7759e640 1 --2- 172.21.15.37:0/2915221101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e6c0 0x7f2d780caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d76d9d640 1 -- 172.21.15.37:0/2915221101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e6c0 msgr2=0x7f2d780caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d76d9d640 1 --2- 172.21.15.37:0/2915221101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e6c0 0x7f2d780caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d76d9d640 1 -- 172.21.15.37:0/2915221101 >> v1:172.21.15.203:6789/0 conn(0x7f2d780cfad0 legacy=0x7f2d7815dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:39.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d76d9d640 1 -- 172.21.15.37:0/2915221101 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d780cc1f0 con 0x7f2d780b3cf0 2024-04-25T08:53:39.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.408+0000 7f2d7759e640 1 --2- 172.21.15.37:0/2915221101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e6c0 0x7f2d780caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:39.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d76d9d640 1 --2- 172.21.15.37:0/2915221101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d780b3cf0 0x7f2d780cf590 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009870 tx=0x7f2d6801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1158eac183ff424 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:39.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d75d9b640 1 -- 172.21.15.37:0/2915221101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680b0070 con 0x7f2d780b3cf0 2024-04-25T08:53:39.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d75d9b640 1 -- 172.21.15.37:0/2915221101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d680ab600 con 0x7f2d780b3cf0 2024-04-25T08:53:39.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d75d9b640 1 -- 172.21.15.37:0/2915221101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680ab9d0 con 0x7f2d780b3cf0 2024-04-25T08:53:39.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 -- 172.21.15.37:0/2915221101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d780b3cf0 msgr2=0x7f2d780cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 --2- 172.21.15.37:0/2915221101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d780b3cf0 0x7f2d780cf590 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009870 tx=0x7f2d6801c520 comp rx=0 tx=0).stop 2024-04-25T08:53:39.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 -- 172.21.15.37:0/2915221101 shutdown_connections 2024-04-25T08:53:39.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 --2- 172.21.15.37:0/2915221101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d7815e6c0 0x7f2d780caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 --2- 172.21.15.37:0/2915221101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d780b3cf0 0x7f2d780cf590 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 -- 172.21.15.37:0/2915221101 >> 172.21.15.37:0/2915221101 conn(0x7f2d7806ae60 msgr2=0x7f2d7806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:39.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 -- 172.21.15.37:0/2915221101 shutdown_connections 2024-04-25T08:53:39.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 -- 172.21.15.37:0/2915221101 wait complete. 2024-04-25T08:53:39.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 Processor -- start 2024-04-25T08:53:39.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 -- start start 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d780b3cf0 0x7f2d78155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d780cfad0 0x7f2d780c3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d7815e6c0 0x7f2d78156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780b4250 con 0x7f2d780b3cf0 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780b40d0 con 0x7f2d7815e6c0 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7d245640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780cb840 con 0x7f2d780cfad0 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d76d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d780b3cf0 0x7f2d78155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d76d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d780b3cf0 0x7f2d78155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51674/0 (socket says 172.21.15.37:51674) 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d76d9d640 1 -- 172.21.15.37:0/4202163088 learned_addr learned my addr 172.21.15.37:0/4202163088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7759e640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d7815e6c0 0x7f2d78156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.412+0000 7f2d7659c640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d780cfad0 0x7f2d780c3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d76d9d640 1 -- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d780cfad0 msgr2=0x7f2d780c3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d76d9d640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d780cfad0 0x7f2d780c3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d76d9d640 1 -- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d7815e6c0 msgr2=0x7f2d78156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d76d9d640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d7815e6c0 0x7f2d78156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d76d9d640 1 -- 172.21.15.37:0/4202163088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78159de0 con 0x7f2d780b3cf0 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d76d9d640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d780b3cf0 0x7f2d78155100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2d680ae040 tx=0x7f2d680192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d5f7fe640 1 -- 172.21.15.37:0/4202163088 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680b0040 con 0x7f2d780b3cf0 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d5f7fe640 1 -- 172.21.15.37:0/4202163088 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d680b9070 con 0x7f2d780b3cf0 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d7d245640 1 -- 172.21.15.37:0/4202163088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7815a0c0 con 0x7f2d780b3cf0 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d5f7fe640 1 -- 172.21.15.37:0/4202163088 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d680b4660 con 0x7f2d780b3cf0 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d7d245640 1 -- 172.21.15.37:0/4202163088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7815a680 con 0x7f2d780b3cf0 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d7d245640 1 -- 172.21.15.37:0/4202163088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7804efa0 con 0x7f2d780b3cf0 2024-04-25T08:53:39.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d5f7fe640 1 -- 172.21.15.37:0/4202163088 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2d680b4800 con 0x7f2d780b3cf0 2024-04-25T08:53:39.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d5f7fe640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d64066f60 0x7f2d64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d5f7fe640 1 -- 172.21.15.37:0/4202163088 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2d6811f1e0 con 0x7f2d780b3cf0 2024-04-25T08:53:39.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d7659c640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d64066f60 0x7f2d64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.416+0000 7f2d7659c640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d64066f60 0x7f2d64069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d780aed20 tx=0x7f2d6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:39.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.420+0000 7f2d5f7fe640 1 -- 172.21.15.37:0/4202163088 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2d68016110 con 0x7f2d780b3cf0 2024-04-25T08:53:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.580+0000 7f2d7d245640 1 -- 172.21.15.37:0/4202163088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2d780b3060 con 0x7f2d780b3cf0 2024-04-25T08:53:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d5f7fe640 1 -- 172.21.15.37:0/4202163088 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2d680efd50 con 0x7f2d780b3cf0 2024-04-25T08:53:39.576 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:39.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 -- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d64066f60 msgr2=0x7f2d64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d64066f60 0x7f2d64069420 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d780aed20 tx=0x7f2d6c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:39.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 -- 172.21.15.37:0/4202163088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d780b3cf0 msgr2=0x7f2d78155100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d780b3cf0 0x7f2d78155100 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2d680ae040 tx=0x7f2d680192c0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 -- 172.21.15.37:0/4202163088 shutdown_connections 2024-04-25T08:53:39.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2d64066f60 0x7f2d64069420 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2d7815e6c0 0x7f2d78156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2d780cfad0 0x7f2d780c3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 --2- 172.21.15.37:0/4202163088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d780b3cf0 0x7f2d78155100 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 -- 172.21.15.37:0/4202163088 >> 172.21.15.37:0/4202163088 conn(0x7f2d7806ae60 msgr2=0x7f2d781604d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 -- 172.21.15.37:0/4202163088 shutdown_connections 2024-04-25T08:53:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.584+0000 7f2d7d245640 1 -- 172.21.15.37:0/4202163088 wait complete. 2024-04-25T08:53:39.595 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:39.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085ce2e640 1 Processor -- start 2024-04-25T08:53:39.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085ce2e640 1 -- start start 2024-04-25T08:53:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085ce2e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085815d9c0 0x7f08580d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085ce2e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f08580cf270 0x7f08580d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085ce2e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08580d1950 con 0x7f08580d3c20 2024-04-25T08:53:39.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085ce2e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08580d2520 con 0x7f08580cf270 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085ce2e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08580d1bf0 con 0x7f085815d9c0 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f0856d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f08580d3c20 0x7f08580d0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33558/0 (socket says 172.21.15.37:33558) 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f0856d9d640 1 -- 172.21.15.37:0/1739716503 learned_addr learned my addr 172.21.15.37:0/1739716503 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f0855d9b640 1 --2- 172.21.15.37:0/1739716503 >> v2:172.21.15.203:3300/0 conn(0x7f08580cf270 0x7f08580d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085659c640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085815d9c0 0x7f08580d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1882832952 0 0) 0x7f08580d1950 con 0x7f08580d3c20 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0840003530 con 0x7f08580d3c20 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3548601829 0 0) 0x7f0840003530 con 0x7f08580d3c20 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085815d9c0 msgr2=0x7f08580d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085815d9c0 0x7f08580d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 >> v2:172.21.15.203:3300/0 conn(0x7f08580cf270 msgr2=0x7f08580d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 --2- 172.21.15.37:0/1739716503 >> v2:172.21.15.203:3300/0 conn(0x7f08580cf270 0x7f08580d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08580d1ed0 con 0x7f08580d3c20 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0850001a00 con 0x7f08580d3c20 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 >> v1:172.21.15.37:6789/0 conn(0x7f08580d3c20 legacy=0x7f08580d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0840003a30 0x7f0840003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085815d9c0 0x7f0840004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840004d50 0x7f08400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08580d2520 con 0x7f0840003a30 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08580d1bf0 con 0x7f0840004d50 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084000b9d0 con 0x7f085815d9c0 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085659c640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0840003a30 0x7f0840003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085659c640 1 -- 172.21.15.37:0/1739716503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085815d9c0 msgr2=0x7f0840004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f0856d9d640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840004d50 0x7f08400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085659c640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f085815d9c0 0x7f0840004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085659c640 1 -- 172.21.15.37:0/1739716503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840004d50 msgr2=0x7f08400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085659c640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840004d50 0x7f08400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f085659c640 1 -- 172.21.15.37:0/1739716503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0840003530 con 0x7f0840003a30 2024-04-25T08:53:39.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.728+0000 7f0856d9d640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840004d50 0x7f08400051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085659c640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0840003a30 0x7f0840003e50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0848009ae0 tx=0x7f084801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45227f9f23488e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08480b2070 con 0x7f0840003a30 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08480a5510 con 0x7f0840003a30 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085559a640 1 -- 172.21.15.37:0/1739716503 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08480096b0 con 0x7f0840003a30 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085ce2e640 1 -- 172.21.15.37:0/1739716503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0840003a30 msgr2=0x7f0840003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085ce2e640 1 --2- 172.21.15.37:0/1739716503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0840003a30 0x7f0840003e50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0848009ae0 tx=0x7f084801cab0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f0856d9d640 1 -- 172.21.15.37:0/1739716503 reap_dead start 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085ce2e640 1 -- 172.21.15.37:0/1739716503 shutdown_connections 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085ce2e640 1 -- 172.21.15.37:0/1739716503 >> 172.21.15.37:0/1739716503 conn(0x7f085809cad0 msgr2=0x7f085808b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085ce2e640 1 -- 172.21.15.37:0/1739716503 shutdown_connections 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085ce2e640 1 -- 172.21.15.37:0/1739716503 wait complete. 2024-04-25T08:53:39.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085ce2e640 1 Processor -- start 2024-04-25T08:53:39.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.732+0000 7f085ce2e640 1 -- start start 2024-04-25T08:53:39.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f085ce2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840003a30 0x7f08580c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f085ce2e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0858160200 0x7f0858160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f085ce2e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0858159e80 0x7f085815a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f085ce2e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0858168710 con 0x7f0858160200 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f085ce2e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0858168590 con 0x7f0840003a30 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f085ce2e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0858168890 con 0x7f0858159e80 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0855d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0858160200 0x7f0858160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0855d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0858160200 0x7f0858160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51694/0 (socket says 172.21.15.37:51694) 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0855d9b640 1 -- 172.21.15.37:0/2574525760 learned_addr learned my addr 172.21.15.37:0/2574525760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f085659c640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840003a30 0x7f08580c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0856d9d640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0858159e80 0x7f085815a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0855d9b640 1 -- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0858159e80 msgr2=0x7f085815a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0855d9b640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0858159e80 0x7f085815a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0855d9b640 1 -- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840003a30 msgr2=0x7f08580c7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0855d9b640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840003a30 0x7f08580c7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0855d9b640 1 -- 172.21.15.37:0/2574525760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08580c78f0 con 0x7f0858160200 2024-04-25T08:53:39.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0856d9d640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0858159e80 0x7f085815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:39.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0855d9b640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0858160200 0x7f0858160680 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f084c00e890 tx=0x7f084c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:39.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0846ffd640 1 -- 172.21.15.37:0/2574525760 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c017070 con 0x7f0858160200 2024-04-25T08:53:39.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0846ffd640 1 -- 172.21.15.37:0/2574525760 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c00c940 con 0x7f0858160200 2024-04-25T08:53:39.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f085ce2e640 1 -- 172.21.15.37:0/2574525760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08580c7bd0 con 0x7f0858160200 2024-04-25T08:53:39.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0846ffd640 1 -- 172.21.15.37:0/2574525760 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c00cd30 con 0x7f0858160200 2024-04-25T08:53:39.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f085ce2e640 1 -- 172.21.15.37:0/2574525760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08580c8190 con 0x7f0858160200 2024-04-25T08:53:39.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f085ce2e640 1 -- 172.21.15.37:0/2574525760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0858160bc0 con 0x7f0858160200 2024-04-25T08:53:39.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0846ffd640 1 -- 172.21.15.37:0/2574525760 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f084c01a070 con 0x7f0858160200 2024-04-25T08:53:39.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0846ffd640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0838066f90 0x7f0838069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:39.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f0846ffd640 1 -- 172.21.15.37:0/2574525760 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f084c085640 con 0x7f0858160200 2024-04-25T08:53:39.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.736+0000 7f085659c640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0838066f90 0x7f0838069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:39.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.740+0000 7f085659c640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0838066f90 0x7f0838069450 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f08480af000 tx=0x7f08480046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:39.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.740+0000 7f0846ffd640 1 -- 172.21.15.37:0/2574525760 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f084c056250 con 0x7f0858160200 2024-04-25T08:53:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.904+0000 7f085ce2e640 1 -- 172.21.15.37:0/2574525760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f085804efa0 con 0x7f0858160200 2024-04-25T08:53:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.904+0000 7f0846ffd640 1 -- 172.21.15.37:0/2574525760 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7f084c055b50 con 0x7f0858160200 2024-04-25T08:53:39.897 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 -- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0838066f90 msgr2=0x7f0838069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0838066f90 0x7f0838069450 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f08480af000 tx=0x7f08480046d0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 -- 172.21.15.37:0/2574525760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0858160200 msgr2=0x7f0858160680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0858160200 0x7f0858160680 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f084c00e890 tx=0x7f084c00ed50 comp rx=0 tx=0).stop 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 -- 172.21.15.37:0/2574525760 shutdown_connections 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0838066f90 0x7f0838069450 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0858159e80 0x7f085815a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0858160200 0x7f0858160680 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 --2- 172.21.15.37:0/2574525760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0840003a30 0x7f08580c7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 -- 172.21.15.37:0/2574525760 >> 172.21.15.37:0/2574525760 conn(0x7f085809cad0 msgr2=0x7f08580cb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 -- 172.21.15.37:0/2574525760 shutdown_connections 2024-04-25T08:53:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:39.908+0000 7f085ce2e640 1 -- 172.21.15.37:0/2574525760 wait complete. 2024-04-25T08:53:39.917 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:40.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9bd996640 1 Processor -- start 2024-04-25T08:53:40.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9bd996640 1 -- start start 2024-04-25T08:53:40.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9bd996640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff9b815b950 0x7ff9b809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9bd996640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b815bd30 con 0x7ff9b80cf420 2024-04-25T08:53:40.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9bd996640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b80cc0d0 con 0x7ff9b80cfb80 2024-04-25T08:53:40.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9bd996640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b809ab10 con 0x7ff9b815b950 2024-04-25T08:53:40.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9affff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff9b80cf420 0x7ff9b80cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33570/0 (socket says 172.21.15.37:33570) 2024-04-25T08:53:40.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9affff640 1 -- 172.21.15.37:0/2740147922 learned_addr learned my addr 172.21.15.37:0/2740147922 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:40.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b6ffd640 1 --2- 172.21.15.37:0/2740147922 >> v2:172.21.15.203:3300/0 conn(0x7ff9b815b950 0x7ff9b809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570854983 0 0) 0x7ff9b815bd30 con 0x7ff9b80cf420 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9a4003530 con 0x7ff9b80cf420 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3350867050 0 0) 0x7ff9a4003530 con 0x7ff9b80cf420 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 >> v2:172.21.15.203:3300/0 conn(0x7ff9b815b950 msgr2=0x7ff9b809a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 --2- 172.21.15.37:0/2740147922 >> v2:172.21.15.203:3300/0 conn(0x7ff9b815b950 0x7ff9b809a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 >> v1:172.21.15.203:6789/0 conn(0x7ff9b80cfb80 legacy=0x7ff9b80cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80cc370 con 0x7ff9b80cf420 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7ff9a0001680 con 0x7ff9b80cf420 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 >> v1:172.21.15.37:6789/0 conn(0x7ff9b80cf420 legacy=0x7ff9b80cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a4003a30 0x7ff9a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a4004810 0x7ff9a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9a40051d0 0x7ff9a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b809ab10 con 0x7ff9a4004810 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a400c050 con 0x7ff9a4003a30 2024-04-25T08:53:40.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a400c250 con 0x7ff9a40051d0 2024-04-25T08:53:40.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9affff640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a4004810 0x7ff9a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b6ffd640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a4003a30 0x7ff9a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b77fe640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9a40051d0 0x7ff9a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9affff640 1 -- 172.21.15.37:0/2740147922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9a40051d0 msgr2=0x7ff9a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:40.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9affff640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9a40051d0 0x7ff9a400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9affff640 1 -- 172.21.15.37:0/2740147922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a4003a30 msgr2=0x7ff9a4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:40.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9affff640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a4003a30 0x7ff9a4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9affff640 1 -- 172.21.15.37:0/2740147922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a4003530 con 0x7ff9a4004810 2024-04-25T08:53:40.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b77fe640 1 -- 172.21.15.37:0/2740147922 reap_dead start 2024-04-25T08:53:40.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.060+0000 7ff9b6ffd640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a4003a30 0x7ff9a4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9affff640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a4004810 0x7ff9a4004c90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c190 tx=0x7ff9a001cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a684d6f21c66aff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0007cd0 con 0x7ff9a4004810 2024-04-25T08:53:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a00a9070 con 0x7ff9a4004810 2024-04-25T08:53:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9b67fc640 1 -- 172.21.15.37:0/2740147922 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0005d60 con 0x7ff9a4004810 2024-04-25T08:53:40.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9bd996640 1 -- 172.21.15.37:0/2740147922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a4004810 msgr2=0x7ff9a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9bd996640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a4004810 0x7ff9a4004c90 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000c190 tx=0x7ff9a001cbf0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9bd996640 1 -- 172.21.15.37:0/2740147922 shutdown_connections 2024-04-25T08:53:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9bd996640 1 --2- 172.21.15.37:0/2740147922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a4004810 0x7ff9a4004c90 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9bd996640 1 -- 172.21.15.37:0/2740147922 >> 172.21.15.37:0/2740147922 conn(0x7ff9b80b30b0 msgr2=0x7ff9b80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9bd996640 1 -- 172.21.15.37:0/2740147922 shutdown_connections 2024-04-25T08:53:40.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9bd996640 1 -- 172.21.15.37:0/2740147922 wait complete. 2024-04-25T08:53:40.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9bd996640 1 Processor -- start 2024-04-25T08:53:40.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.064+0000 7ff9bd996640 1 -- start start 2024-04-25T08:53:40.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9bd996640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a4004810 0x7ff9b80c9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9bd996640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b815a290 0x7ff9b80c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9bd996640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9b80caa70 0x7ff9b80caef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9bd996640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80d04f0 con 0x7ff9b815a290 2024-04-25T08:53:40.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9bd996640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80d0370 con 0x7ff9a4004810 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9bd996640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b806f290 con 0x7ff9b80caa70 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b815a290 0x7ff9b80c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9affff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b815a290 0x7ff9b80c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51710/0 (socket says 172.21.15.37:51710) 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9affff640 1 -- 172.21.15.37:0/1999798714 learned_addr learned my addr 172.21.15.37:0/1999798714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9b77fe640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9b80caa70 0x7ff9b80caef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9b6ffd640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a4004810 0x7ff9b80c9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9affff640 1 -- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9b80caa70 msgr2=0x7ff9b80caef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9affff640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9b80caa70 0x7ff9b80caef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9affff640 1 -- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a4004810 msgr2=0x7ff9b80c9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9affff640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a4004810 0x7ff9b80c9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9affff640 1 -- 172.21.15.37:0/1999798714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b815ab10 con 0x7ff9b815a290 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9b6ffd640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a4004810 0x7ff9b80c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:40.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9affff640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b815a290 0x7ff9b80c42a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0001900 tx=0x7ff9a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:40.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9b4ff9640 1 -- 172.21.15.37:0/1999798714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0007cd0 con 0x7ff9b815a290 2024-04-25T08:53:40.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9b4ff9640 1 -- 172.21.15.37:0/1999798714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a0004e50 con 0x7ff9b815a290 2024-04-25T08:53:40.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9b4ff9640 1 -- 172.21.15.37:0/1999798714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0001660 con 0x7ff9b815a290 2024-04-25T08:53:40.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9bd996640 1 -- 172.21.15.37:0/1999798714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b815caf0 con 0x7ff9b815a290 2024-04-25T08:53:40.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9bd996640 1 -- 172.21.15.37:0/1999798714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b80c8970 con 0x7ff9b815a290 2024-04-25T08:53:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9bd996640 1 -- 172.21.15.37:0/1999798714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b804efa0 con 0x7ff9b815a290 2024-04-25T08:53:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9b4ff9640 1 -- 172.21.15.37:0/1999798714 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9a00a57e0 con 0x7ff9b815a290 2024-04-25T08:53:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.068+0000 7ff9b4ff9640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff998066f60 0x7ff998069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.072+0000 7ff9b4ff9640 1 -- 172.21.15.37:0/1999798714 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff9a011e7e0 con 0x7ff9b815a290 2024-04-25T08:53:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.072+0000 7ff9b6ffd640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff998066f60 0x7ff998069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.072+0000 7ff9b6ffd640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff998066f60 0x7ff998069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8050240 tx=0x7ff9a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:40.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.076+0000 7ff9b4ff9640 1 -- 172.21.15.37:0/1999798714 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9b815ab10 con 0x7ff9b815a290 2024-04-25T08:53:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.232+0000 7ff9bd996640 1 -- 172.21.15.37:0/1999798714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff9b80d13a0 con 0x7ff9b815a290 2024-04-25T08:53:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9b4ff9640 1 -- 172.21.15.37:0/1999798714 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff9a0016930 con 0x7ff9b815a290 2024-04-25T08:53:40.227 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 -- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff998066f60 msgr2=0x7ff998069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff998066f60 0x7ff998069420 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9b8050240 tx=0x7ff9a8008040 comp rx=0 tx=0).stop 2024-04-25T08:53:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 -- 172.21.15.37:0/1999798714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b815a290 msgr2=0x7ff9b80c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b815a290 0x7ff9b80c42a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0001900 tx=0x7ff9a4005840 comp rx=0 tx=0).stop 2024-04-25T08:53:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 -- 172.21.15.37:0/1999798714 shutdown_connections 2024-04-25T08:53:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff998066f60 0x7ff998069420 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9b80caa70 0x7ff9b80caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b815a290 0x7ff9b80c42a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 --2- 172.21.15.37:0/1999798714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9a4004810 0x7ff9b80c9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 -- 172.21.15.37:0/1999798714 >> 172.21.15.37:0/1999798714 conn(0x7ff9b80b30b0 msgr2=0x7ff9b80b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 -- 172.21.15.37:0/1999798714 shutdown_connections 2024-04-25T08:53:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.236+0000 7ff9bd996640 1 -- 172.21.15.37:0/1999798714 wait complete. 2024-04-25T08:53:40.247 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:53:40.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89e1cc3640 1 Processor -- start 2024-04-25T08:53:40.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89e1cc3640 1 -- start start 2024-04-25T08:53:40.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89e1cc3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f89dc144ca0 0x7f89dc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89e1cc3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89dc0ccc70 con 0x7f89dc0ce510 2024-04-25T08:53:40.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89e1cc3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89dc06dc40 con 0x7f89dc06ec60 2024-04-25T08:53:40.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89e1cc3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc0cc2b0 con 0x7f89dc144ca0 2024-04-25T08:53:40.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89db7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f89dc0ce510 0x7f89dc0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33574/0 (socket says 172.21.15.37:33574) 2024-04-25T08:53:40.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89db7fe640 1 -- 172.21.15.37:0/1156808782 learned_addr learned my addr 172.21.15.37:0/1156808782 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:40.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89daffd640 1 --2- 172.21.15.37:0/1156808782 >> v2:172.21.15.37:3300/0 conn(0x7f89dc144ca0 0x7f89dc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89da7fc640 1 -- 172.21.15.37:0/1156808782 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626486379 0 0) 0x7f89dc0ccc70 con 0x7f89dc0ce510 2024-04-25T08:53:40.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89da7fc640 1 -- 172.21.15.37:0/1156808782 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89c8003530 con 0x7f89dc0ce510 2024-04-25T08:53:40.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89daffd640 1 -- 172.21.15.37:0/1156808782 >> v1:172.21.15.203:6789/0 conn(0x7f89dc06ec60 legacy=0x7f89dc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:40.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89daffd640 1 -- 172.21.15.37:0/1156808782 >> v1:172.21.15.37:6789/0 conn(0x7f89dc0ce510 legacy=0x7f89dc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.376+0000 7f89daffd640 1 -- 172.21.15.37:0/1156808782 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc06df20 con 0x7f89dc144ca0 2024-04-25T08:53:40.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89daffd640 1 --2- 172.21.15.37:0/1156808782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89dc144ca0 0x7f89dc06e720 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0097c0 tx=0x7f89cc01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c2a30cfbb692e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:40.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89da7fc640 1 -- 172.21.15.37:0/1156808782 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc0b5070 con 0x7f89dc144ca0 2024-04-25T08:53:40.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89da7fc640 1 -- 172.21.15.37:0/1156808782 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc0a8950 con 0x7f89dc144ca0 2024-04-25T08:53:40.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89da7fc640 1 -- 172.21.15.37:0/1156808782 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc0a8d20 con 0x7f89dc144ca0 2024-04-25T08:53:40.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 -- 172.21.15.37:0/1156808782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89dc144ca0 msgr2=0x7f89dc06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 --2- 172.21.15.37:0/1156808782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89dc144ca0 0x7f89dc06e720 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0097c0 tx=0x7f89cc01ec50 comp rx=0 tx=0).stop 2024-04-25T08:53:40.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 -- 172.21.15.37:0/1156808782 shutdown_connections 2024-04-25T08:53:40.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 --2- 172.21.15.37:0/1156808782 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89dc144ca0 0x7f89dc06e720 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 -- 172.21.15.37:0/1156808782 >> 172.21.15.37:0/1156808782 conn(0x7f89dc0b11a0 msgr2=0x7f89dc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:40.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 -- 172.21.15.37:0/1156808782 shutdown_connections 2024-04-25T08:53:40.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 -- 172.21.15.37:0/1156808782 wait complete. 2024-04-25T08:53:40.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 Processor -- start 2024-04-25T08:53:40.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 -- start start 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89dc06ec60 0x7f89dc15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89dc0ce510 0x7f89dc134b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89dc144ca0 0x7f89dc0abbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc142d90 con 0x7f89dc144ca0 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc06e290 con 0x7f89dc06ec60 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89e1cc3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc142f10 con 0x7f89dc0ce510 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89dc144ca0 0x7f89dc0abbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89dc144ca0 0x7f89dc0abbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51734/0 (socket says 172.21.15.37:51734) 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89dbfff640 1 -- 172.21.15.37:0/18828607 learned_addr learned my addr 172.21.15.37:0/18828607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89daffd640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89dc0ce510 0x7f89dc134b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89dbfff640 1 -- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89dc0ce510 msgr2=0x7f89dc134b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89dbfff640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89dc0ce510 0x7f89dc134b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89dbfff640 1 -- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89dc06ec60 msgr2=0x7f89dc15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89dbfff640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89dc06ec60 0x7f89dc15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89dbfff640 1 -- 172.21.15.37:0/18828607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc135370 con 0x7f89dc144ca0 2024-04-25T08:53:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.380+0000 7f89dbfff640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89dc144ca0 0x7f89dc0abbd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f89d400c7b0 tx=0x7f89d400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.384+0000 7f89d3fff640 1 -- 172.21.15.37:0/18828607 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4004b70 con 0x7f89dc144ca0 2024-04-25T08:53:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.384+0000 7f89d3fff640 1 -- 172.21.15.37:0/18828607 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d4004d10 con 0x7f89dc144ca0 2024-04-25T08:53:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.384+0000 7f89e1cc3640 1 -- 172.21.15.37:0/18828607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89dc135650 con 0x7f89dc144ca0 2024-04-25T08:53:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.384+0000 7f89d3fff640 1 -- 172.21.15.37:0/18828607 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d400aa90 con 0x7f89dc144ca0 2024-04-25T08:53:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.384+0000 7f89e1cc3640 1 -- 172.21.15.37:0/18828607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89dc135c10 con 0x7f89dc144ca0 2024-04-25T08:53:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.384+0000 7f89d3fff640 1 -- 172.21.15.37:0/18828607 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f89d400ac30 con 0x7f89dc144ca0 2024-04-25T08:53:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.384+0000 7f89e1cc3640 1 -- 172.21.15.37:0/18828607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89dc04efa0 con 0x7f89dc144ca0 2024-04-25T08:53:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.384+0000 7f89d3fff640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f89bc066f60 0x7f89bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.384+0000 7f89d3fff640 1 -- 172.21.15.37:0/18828607 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f89d4085670 con 0x7f89dc144ca0 2024-04-25T08:53:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.384+0000 7f89db7fe640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f89bc066f60 0x7f89bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.384+0000 7f89db7fe640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f89bc066f60 0x7f89bc069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f89c4007fd0 tx=0x7f89c4007900 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.388+0000 7f89d3fff640 1 -- 172.21.15.37:0/18828607 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f89d4020070 con 0x7f89dc144ca0 2024-04-25T08:53:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 -- 172.21.15.37:0/18828607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f89dc0b1cc0 con 0x7f89dc144ca0 2024-04-25T08:53:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89d3fff640 1 -- 172.21.15.37:0/18828607 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) v1 ==== 80+0+0 (secure 0 0 0) 0x7f89d4056310 con 0x7f89dc144ca0 2024-04-25T08:53:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:53:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 -- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f89bc066f60 msgr2=0x7f89bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f89bc066f60 0x7f89bc069420 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f89c4007fd0 tx=0x7f89c4007900 comp rx=0 tx=0).stop 2024-04-25T08:53:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 -- 172.21.15.37:0/18828607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89dc144ca0 msgr2=0x7f89dc0abbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89dc144ca0 0x7f89dc0abbd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f89d400c7b0 tx=0x7f89d400cc70 comp rx=0 tx=0).stop 2024-04-25T08:53:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 -- 172.21.15.37:0/18828607 shutdown_connections 2024-04-25T08:53:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f89bc066f60 0x7f89bc069420 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f89dc144ca0 0x7f89dc0abbd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f89dc0ce510 0x7f89dc134b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 --2- 172.21.15.37:0/18828607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89dc06ec60 0x7f89dc15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 -- 172.21.15.37:0/18828607 >> 172.21.15.37:0/18828607 conn(0x7f89dc0b11a0 msgr2=0x7f89dc0b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 -- 172.21.15.37:0/18828607 shutdown_connections 2024-04-25T08:53:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.560+0000 7f89e1cc3640 1 -- 172.21.15.37:0/18828607 wait complete. 2024-04-25T08:53:40.571 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi203.front.sepia.ceph.com... 2024-04-25T08:53:40.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:40.571 DEBUG:teuthology.orchestra.run.smithi203:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T08:53:40.586 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:40.586 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:53:40.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:53:40.586 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:53:40.633 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:53:40.633 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:40.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79eb69e640 1 Processor -- start 2024-04-25T08:53:40.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79eb69e640 1 -- start start 2024-04-25T08:53:40.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79eb69e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f79e40b3f50 0x7f79e415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79eb69e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e40b2090 con 0x7f79e415a700 2024-04-25T08:53:40.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79eb69e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e40b43e0 con 0x7f79e40b24a0 2024-04-25T08:53:40.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79eb69e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e415f200 con 0x7f79e40b3f50 2024-04-25T08:53:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e943a640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f79e415a700 0x7f79e40cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33578/0 (socket says 172.21.15.37:33578) 2024-04-25T08:53:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e943a640 1 -- 172.21.15.37:0/4224814908 learned_addr learned my addr 172.21.15.37:0/4224814908 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e8c39640 1 --2- 172.21.15.37:0/4224814908 >> v2:172.21.15.203:3300/0 conn(0x7f79e40b3f50 0x7f79e415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760690377 0 0) 0x7f79e40b2090 con 0x7f79e415a700 2024-04-25T08:53:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d0003530 con 0x7f79e415a700 2024-04-25T08:53:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1006132544 0 0) 0x7f79d0003530 con 0x7f79e415a700 2024-04-25T08:53:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 >> v2:172.21.15.203:3300/0 conn(0x7f79e40b3f50 msgr2=0x7f79e415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 --2- 172.21.15.37:0/4224814908 >> v2:172.21.15.203:3300/0 conn(0x7f79e40b3f50 0x7f79e415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 >> v1:172.21.15.203:6789/0 conn(0x7f79e40b24a0 legacy=0x7f79e406e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e40b46c0 con 0x7f79e415a700 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f79e00016a0 con 0x7f79e415a700 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 >> v1:172.21.15.37:6789/0 conn(0x7f79e415a700 legacy=0x7f79e40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79d0003a30 0x7f79d0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d0004810 0x7f79d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d00052d0 0x7f79d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e415f200 con 0x7f79d0003a30 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d000c210 con 0x7f79d0004810 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d000c410 con 0x7f79d00052d0 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e943a640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79d0003a30 0x7f79d0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e8c39640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d0004810 0x7f79d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e943a640 1 -- 172.21.15.37:0/4224814908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d00052d0 msgr2=0x7f79d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e9c3b640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d00052d0 0x7f79d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e943a640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d00052d0 0x7f79d000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e943a640 1 -- 172.21.15.37:0/4224814908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d0004810 msgr2=0x7f79d0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:40.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e943a640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d0004810 0x7f79d0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e943a640 1 -- 172.21.15.37:0/4224814908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0003530 con 0x7f79d0003a30 2024-04-25T08:53:40.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e8c39640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79d0004810 0x7f79d0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:40.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e9c3b640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d00052d0 0x7f79d000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:40.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.764+0000 7f79e9c3b640 1 -- 172.21.15.37:0/4224814908 reap_dead start 2024-04-25T08:53:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79e943a640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79d0003a30 0x7f79d0003e50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f79e000c210 tx=0x7f79e001cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7530ee3bee0690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0007d20 con 0x7f79d0003a30 2024-04-25T08:53:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e00a9020 con 0x7f79d0003a30 2024-04-25T08:53:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79dbfff640 1 -- 172.21.15.37:0/4224814908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0005ae0 con 0x7f79d0003a30 2024-04-25T08:53:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79eb69e640 1 -- 172.21.15.37:0/4224814908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79d0003a30 msgr2=0x7f79d0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79eb69e640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79d0003a30 0x7f79d0003e50 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f79e000c210 tx=0x7f79e001cc10 comp rx=0 tx=0).stop 2024-04-25T08:53:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79eb69e640 1 -- 172.21.15.37:0/4224814908 shutdown_connections 2024-04-25T08:53:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79eb69e640 1 --2- 172.21.15.37:0/4224814908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79d0003a30 0x7f79d0003e50 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79eb69e640 1 -- 172.21.15.37:0/4224814908 >> 172.21.15.37:0/4224814908 conn(0x7f79e40b11a0 msgr2=0x7f79e40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79eb69e640 1 -- 172.21.15.37:0/4224814908 shutdown_connections 2024-04-25T08:53:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79eb69e640 1 -- 172.21.15.37:0/4224814908 wait complete. 2024-04-25T08:53:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79eb69e640 1 Processor -- start 2024-04-25T08:53:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.768+0000 7f79eb69e640 1 -- start start 2024-04-25T08:53:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79eb69e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d0003a30 0x7f79e4154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79eb69e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79e4156900 0x7f79e41572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79eb69e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e41552f0 0x7f79e4157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79eb69e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40b2c90 con 0x7f79e41552f0 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79eb69e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40ce700 con 0x7f79e4156900 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79eb69e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e40cd2d0 con 0x7f79d0003a30 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e9c3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e41552f0 0x7f79e4157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e9c3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e41552f0 0x7f79e4157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51750/0 (socket says 172.21.15.37:51750) 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e9c3b640 1 -- 172.21.15.37:0/3236051050 learned_addr learned my addr 172.21.15.37:0/3236051050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e8c39640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79e4156900 0x7f79e41572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e943a640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d0003a30 0x7f79e4154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e9c3b640 1 -- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d0003a30 msgr2=0x7f79e4154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e9c3b640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d0003a30 0x7f79e4154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e9c3b640 1 -- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79e4156900 msgr2=0x7f79e41572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e9c3b640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79e4156900 0x7f79e41572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e9c3b640 1 -- 172.21.15.37:0/3236051050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e4155860 con 0x7f79e41552f0 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e9c3b640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e41552f0 0x7f79e4157800 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f79e40aef60 tx=0x7f79dc00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:40.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79d9ffb640 1 -- 172.21.15.37:0/3236051050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc018070 con 0x7f79e41552f0 2024-04-25T08:53:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79d9ffb640 1 -- 172.21.15.37:0/3236051050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc003db0 con 0x7f79e41552f0 2024-04-25T08:53:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79eb69e640 1 -- 172.21.15.37:0/3236051050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e4155b40 con 0x7f79e41552f0 2024-04-25T08:53:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79d9ffb640 1 -- 172.21.15.37:0/3236051050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc005330 con 0x7f79e41552f0 2024-04-25T08:53:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79eb69e640 1 -- 172.21.15.37:0/3236051050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e4156100 con 0x7f79e41552f0 2024-04-25T08:53:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79eb69e640 1 -- 172.21.15.37:0/3236051050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e404efa0 con 0x7f79e41552f0 2024-04-25T08:53:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79d9ffb640 1 -- 172.21.15.37:0/3236051050 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79dc0054d0 con 0x7f79e41552f0 2024-04-25T08:53:40.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79d9ffb640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f79c8066f90 0x7f79c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:40.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79d9ffb640 1 -- 172.21.15.37:0/3236051050 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f79dc0848d0 con 0x7f79e41552f0 2024-04-25T08:53:40.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.772+0000 7f79e943a640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f79c8066f90 0x7f79c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:40.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.776+0000 7f79e943a640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f79c8066f90 0x7f79c8069450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f79e00aa000 tx=0x7f79e40b45b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:40.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.776+0000 7f79d9ffb640 1 -- 172.21.15.37:0/3236051050 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79dc055570 con 0x7f79e41552f0 2024-04-25T08:53:40.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.972+0000 7f79eb69e640 1 -- 172.21.15.37:0/3236051050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79e406f260 con 0x7f79e41552f0 2024-04-25T08:53:40.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79d9ffb640 1 -- 172.21.15.37:0/3236051050 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f79dc082e20 con 0x7f79e41552f0 2024-04-25T08:53:40.968 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:40.968 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2902565757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2902565757},{"type":"v1","addr":"172.21.15.203:6835","nonce":2902565757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/822172299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":822172299},{"type":"v1","addr":"172.21.15.203:6837","nonce":822172299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3681855890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3681855890},{"type":"v1","addr":"172.21.15.37:6833","nonce":3681855890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:30.564153+0000","modified":"2024-04-25T08:52:53.726659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:53:40.968 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-25T08:53:40.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 -- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f79c8066f90 msgr2=0x7f79c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f79c8066f90 0x7f79c8069450 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f79e00aa000 tx=0x7f79e40b45b0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 -- 172.21.15.37:0/3236051050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e41552f0 msgr2=0x7f79e4157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:40.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e41552f0 0x7f79e4157800 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f79e40aef60 tx=0x7f79dc00d9f0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 -- 172.21.15.37:0/3236051050 shutdown_connections 2024-04-25T08:53:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f79c8066f90 0x7f79c8069450 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79e41552f0 0x7f79e4157800 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79e4156900 0x7f79e41572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 --2- 172.21.15.37:0/3236051050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79d0003a30 0x7f79e4154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 -- 172.21.15.37:0/3236051050 >> 172.21.15.37:0/3236051050 conn(0x7f79e40b11a0 msgr2=0x7f79e40b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 -- 172.21.15.37:0/3236051050 shutdown_connections 2024-04-25T08:53:40.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:40.976+0000 7f79eb69e640 1 -- 172.21.15.37:0/3236051050 wait complete. 2024-04-25T08:53:40.988 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.120+0000 7f026058e640 1 Processor -- start 2024-04-25T08:53:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.120+0000 7f026058e640 1 -- start start 2024-04-25T08:53:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f026058e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02580ce510 0x7f02580ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f026058e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0258144ca0 0x7f025806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f026058e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02580cee30 con 0x7f025806ec60 2024-04-25T08:53:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f026058e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02580cc2b0 con 0x7f0258144ca0 2024-04-25T08:53:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f026058e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025806e290 con 0x7f02580ce510 2024-04-25T08:53:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f025eb2b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f025806ec60 0x7f0258142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33590/0 (socket says 172.21.15.37:33590) 2024-04-25T08:53:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f025eb2b640 1 -- 172.21.15.37:0/3828160891 learned_addr learned my addr 172.21.15.37:0/3828160891 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f025e32a640 1 --2- 172.21.15.37:0/3828160891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02580ce510 0x7f02580ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f025db29640 1 --2- 172.21.15.37:0/3828160891 >> v2:172.21.15.203:3300/0 conn(0x7f0258144ca0 0x7f025806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f025d328640 1 -- 172.21.15.37:0/3828160891 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509906790 0 0) 0x7f02580cee30 con 0x7f025806ec60 2024-04-25T08:53:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f025d328640 1 -- 172.21.15.37:0/3828160891 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0244003530 con 0x7f025806ec60 2024-04-25T08:53:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f025db29640 1 -- 172.21.15.37:0/3828160891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02580ce510 msgr2=0x7f02580ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f025db29640 1 --2- 172.21.15.37:0/3828160891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02580ce510 0x7f02580ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f025db29640 1 -- 172.21.15.37:0/3828160891 >> v1:172.21.15.37:6789/0 conn(0x7f025806ec60 legacy=0x7f0258142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f025db29640 1 -- 172.21.15.37:0/3828160891 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02580ccc70 con 0x7f0258144ca0 2024-04-25T08:53:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.124+0000 7f025e32a640 1 --2- 172.21.15.37:0/3828160891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02580ce510 0x7f02580ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:41.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f025db29640 1 --2- 172.21.15.37:0/3828160891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0258144ca0 0x7f025806e720 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f02480097c0 tx=0x7f024801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5783c2b338b982c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f025d328640 1 -- 172.21.15.37:0/3828160891 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480b5070 con 0x7f0258144ca0 2024-04-25T08:53:41.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f025d328640 1 -- 172.21.15.37:0/3828160891 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02480a8990 con 0x7f0258144ca0 2024-04-25T08:53:41.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f025d328640 1 -- 172.21.15.37:0/3828160891 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480a8d40 con 0x7f0258144ca0 2024-04-25T08:53:41.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 -- 172.21.15.37:0/3828160891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0258144ca0 msgr2=0x7f025806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 --2- 172.21.15.37:0/3828160891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0258144ca0 0x7f025806e720 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f02480097c0 tx=0x7f024801ec90 comp rx=0 tx=0).stop 2024-04-25T08:53:41.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 -- 172.21.15.37:0/3828160891 shutdown_connections 2024-04-25T08:53:41.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 --2- 172.21.15.37:0/3828160891 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0258144ca0 0x7f025806e720 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 --2- 172.21.15.37:0/3828160891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02580ce510 0x7f02580ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 -- 172.21.15.37:0/3828160891 >> 172.21.15.37:0/3828160891 conn(0x7f02580b11a0 msgr2=0x7f02580b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:41.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 -- 172.21.15.37:0/3828160891 shutdown_connections 2024-04-25T08:53:41.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 -- 172.21.15.37:0/3828160891 wait complete. 2024-04-25T08:53:41.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 Processor -- start 2024-04-25T08:53:41.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 -- start start 2024-04-25T08:53:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f025806ec60 0x7f02580b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02580ce510 0x7f02580b25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0258144ca0 0x7f02581f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02580cc900 con 0x7f02580ce510 2024-04-25T08:53:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025806df20 con 0x7f0258144ca0 2024-04-25T08:53:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f026058e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0258142d90 con 0x7f025806ec60 2024-04-25T08:53:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f025db29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02580ce510 0x7f02580b25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f025db29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02580ce510 0x7f02580b25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51752/0 (socket says 172.21.15.37:51752) 2024-04-25T08:53:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.128+0000 7f025db29640 1 -- 172.21.15.37:0/2510930703 learned_addr learned my addr 172.21.15.37:0/2510930703 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f025e32a640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f025806ec60 0x7f02580b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f025eb2b640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0258144ca0 0x7f02581f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f025db29640 1 -- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f025806ec60 msgr2=0x7f02580b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f025db29640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f025806ec60 0x7f02580b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f025db29640 1 -- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0258144ca0 msgr2=0x7f02581f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f025db29640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0258144ca0 0x7f02581f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f025db29640 1 -- 172.21.15.37:0/2510930703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02580b3020 con 0x7f02580ce510 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f025e32a640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f025806ec60 0x7f02580b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f025db29640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02580ce510 0x7f02580b25a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f02480b3040 tx=0x7f024801ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f024effd640 1 -- 172.21.15.37:0/2510930703 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480bf070 con 0x7f02580ce510 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f024effd640 1 -- 172.21.15.37:0/2510930703 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0248004240 con 0x7f02580ce510 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f026058e640 1 -- 172.21.15.37:0/2510930703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02580b3300 con 0x7f02580ce510 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f024effd640 1 -- 172.21.15.37:0/2510930703 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02480b5040 con 0x7f02580ce510 2024-04-25T08:53:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f026058e640 1 -- 172.21.15.37:0/2510930703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02580b38c0 con 0x7f02580ce510 2024-04-25T08:53:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f026058e640 1 -- 172.21.15.37:0/2510930703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025804efa0 con 0x7f02580ce510 2024-04-25T08:53:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f024effd640 1 -- 172.21.15.37:0/2510930703 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02480c2070 con 0x7f02580ce510 2024-04-25T08:53:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f024effd640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f023c066f60 0x7f023c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f024effd640 1 -- 172.21.15.37:0/2510930703 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0248121d00 con 0x7f02580ce510 2024-04-25T08:53:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f025e32a640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f023c066f60 0x7f023c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.132+0000 7f025e32a640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f023c066f60 0x7f023c069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0250007520 tx=0x7f025000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.136+0000 7f024effd640 1 -- 172.21.15.37:0/2510930703 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f02480187f0 con 0x7f02580ce510 2024-04-25T08:53:41.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.304+0000 7f026058e640 1 -- 172.21.15.37:0/2510930703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02580b1cc0 con 0x7f02580ce510 2024-04-25T08:53:41.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.304+0000 7f024effd640 1 -- 172.21.15.37:0/2510930703 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f02480f2910 con 0x7f02580ce510 2024-04-25T08:53:41.299 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:41.299 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:41.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 -- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f023c066f60 msgr2=0x7f023c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f023c066f60 0x7f023c069420 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0250007520 tx=0x7f025000b040 comp rx=0 tx=0).stop 2024-04-25T08:53:41.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 -- 172.21.15.37:0/2510930703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02580ce510 msgr2=0x7f02580b25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02580ce510 0x7f02580b25a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f02480b3040 tx=0x7f024801ba30 comp rx=0 tx=0).stop 2024-04-25T08:53:41.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 -- 172.21.15.37:0/2510930703 shutdown_connections 2024-04-25T08:53:41.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f023c066f60 0x7f023c069420 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0258144ca0 0x7f02581f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02580ce510 0x7f02580b25a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 --2- 172.21.15.37:0/2510930703 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f025806ec60 0x7f02580b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 -- 172.21.15.37:0/2510930703 >> 172.21.15.37:0/2510930703 conn(0x7f02580b11a0 msgr2=0x7f025815b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:41.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 -- 172.21.15.37:0/2510930703 shutdown_connections 2024-04-25T08:53:41.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.308+0000 7f026058e640 1 -- 172.21.15.37:0/2510930703 wait complete. 2024-04-25T08:53:41.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:52:28.778700+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:52:29.556234+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T08:52:33.308816+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '26', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T08:53:41.319 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb52476640 1 Processor -- start 2024-04-25T08:53:41.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb52476640 1 -- start start 2024-04-25T08:53:41.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb52476640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feb4c0cfca0 0x7feb4c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb52476640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4c0cbf10 con 0x7feb4c15e4f0 2024-04-25T08:53:41.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb52476640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4c0cc1f0 con 0x7feb4c0b3cf0 2024-04-25T08:53:41.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb52476640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4c0cc4d0 con 0x7feb4c0cfca0 2024-04-25T08:53:41.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb50a13640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7feb4c15e4f0 0x7feb4c0caaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33606/0 (socket says 172.21.15.37:33606) 2024-04-25T08:53:41.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb50a13640 1 -- 172.21.15.37:0/946386811 learned_addr learned my addr 172.21.15.37:0/946386811 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:41.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb4b7fe640 1 --2- 172.21.15.37:0/946386811 >> v2:172.21.15.37:3300/0 conn(0x7feb4c0cfca0 0x7feb4c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb4affd640 1 -- 172.21.15.37:0/946386811 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097789221 0 0) 0x7feb4c0cbf10 con 0x7feb4c15e4f0 2024-04-25T08:53:41.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb4affd640 1 -- 172.21.15.37:0/946386811 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb34003530 con 0x7feb4c15e4f0 2024-04-25T08:53:41.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb4b7fe640 1 -- 172.21.15.37:0/946386811 >> v1:172.21.15.203:6789/0 conn(0x7feb4c0b3cf0 legacy=0x7feb4c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb4b7fe640 1 -- 172.21.15.37:0/946386811 >> v1:172.21.15.37:6789/0 conn(0x7feb4c15e4f0 legacy=0x7feb4c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.452+0000 7feb4b7fe640 1 -- 172.21.15.37:0/946386811 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c0cc7b0 con 0x7feb4c0cfca0 2024-04-25T08:53:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb4b7fe640 1 --2- 172.21.15.37:0/946386811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb4c0cfca0 0x7feb4c15dfb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7feb400097c0 tx=0x7feb4001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93eb93c84ebab4bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb4affd640 1 -- 172.21.15.37:0/946386811 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb400b0070 con 0x7feb4c0cfca0 2024-04-25T08:53:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb4affd640 1 -- 172.21.15.37:0/946386811 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb400ab600 con 0x7feb4c0cfca0 2024-04-25T08:53:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb4affd640 1 -- 172.21.15.37:0/946386811 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb400ab9d0 con 0x7feb4c0cfca0 2024-04-25T08:53:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 -- 172.21.15.37:0/946386811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb4c0cfca0 msgr2=0x7feb4c15dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 --2- 172.21.15.37:0/946386811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb4c0cfca0 0x7feb4c15dfb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7feb400097c0 tx=0x7feb4001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 -- 172.21.15.37:0/946386811 shutdown_connections 2024-04-25T08:53:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 --2- 172.21.15.37:0/946386811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb4c0cfca0 0x7feb4c15dfb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 -- 172.21.15.37:0/946386811 >> 172.21.15.37:0/946386811 conn(0x7feb4c06ae60 msgr2=0x7feb4c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 -- 172.21.15.37:0/946386811 shutdown_connections 2024-04-25T08:53:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 -- 172.21.15.37:0/946386811 wait complete. 2024-04-25T08:53:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 Processor -- start 2024-04-25T08:53:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 -- start start 2024-04-25T08:53:41.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb4c0b3cf0 0x7feb4c1513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb4c0cfca0 0x7feb4c155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb4c15e4f0 0x7feb4c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c16b8a0 con 0x7feb4c0cfca0 2024-04-25T08:53:41.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c16b720 con 0x7feb4c0b3cf0 2024-04-25T08:53:41.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.456+0000 7feb52476640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4c16ba20 con 0x7feb4c15e4f0 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb4b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb4c0cfca0 0x7feb4c155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb4b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb4c0cfca0 0x7feb4c155b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51772/0 (socket says 172.21.15.37:51772) 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb4b7fe640 1 -- 172.21.15.37:0/2056318171 learned_addr learned my addr 172.21.15.37:0/2056318171 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb4bfff640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb4c0b3cf0 0x7feb4c1513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb50a13640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb4c15e4f0 0x7feb4c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb4b7fe640 1 -- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb4c15e4f0 msgr2=0x7feb4c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb4b7fe640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb4c15e4f0 0x7feb4c153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb4b7fe640 1 -- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb4c0b3cf0 msgr2=0x7feb4c1513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb4b7fe640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb4c0b3cf0 0x7feb4c1513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb4b7fe640 1 -- 172.21.15.37:0/2056318171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c1562d0 con 0x7feb4c0cfca0 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb4b7fe640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb4c0cfca0 0x7feb4c155b60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feb400097c0 tx=0x7feb40019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb48ff9640 1 -- 172.21.15.37:0/2056318171 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb400ba070 con 0x7feb4c0cfca0 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb48ff9640 1 -- 172.21.15.37:0/2056318171 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb400b43f0 con 0x7feb4c0cfca0 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb52476640 1 -- 172.21.15.37:0/2056318171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4c1565b0 con 0x7feb4c0cfca0 2024-04-25T08:53:41.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb48ff9640 1 -- 172.21.15.37:0/2056318171 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb400b0040 con 0x7feb4c0cfca0 2024-04-25T08:53:41.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb52476640 1 -- 172.21.15.37:0/2056318171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4c14e240 con 0x7feb4c0cfca0 2024-04-25T08:53:41.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb52476640 1 -- 172.21.15.37:0/2056318171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4c04efa0 con 0x7feb4c0cfca0 2024-04-25T08:53:41.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb48ff9640 1 -- 172.21.15.37:0/2056318171 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feb400bd070 con 0x7feb4c0cfca0 2024-04-25T08:53:41.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb48ff9640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feb38066f60 0x7feb38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb48ff9640 1 -- 172.21.15.37:0/2056318171 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7feb4011e400 con 0x7feb4c0cfca0 2024-04-25T08:53:41.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.460+0000 7feb4bfff640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feb38066f60 0x7feb38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.464+0000 7feb4bfff640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feb38066f60 0x7feb38069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb3c001810 tx=0x7feb3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.464+0000 7feb48ff9640 1 -- 172.21.15.37:0/2056318171 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feb40016090 con 0x7feb4c0cfca0 2024-04-25T08:53:41.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.628+0000 7feb52476640 1 -- 172.21.15.37:0/2056318171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feb4c0b3060 con 0x7feb4c0cfca0 2024-04-25T08:53:41.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb48ff9640 1 -- 172.21.15.37:0/2056318171 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7feb400eeb30 con 0x7feb4c0cfca0 2024-04-25T08:53:41.624 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 -- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feb38066f60 msgr2=0x7feb38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feb38066f60 0x7feb38069420 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb3c001810 tx=0x7feb3c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 -- 172.21.15.37:0/2056318171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb4c0cfca0 msgr2=0x7feb4c155b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb4c0cfca0 0x7feb4c155b60 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7feb400097c0 tx=0x7feb40019240 comp rx=0 tx=0).stop 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 -- 172.21.15.37:0/2056318171 shutdown_connections 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feb38066f60 0x7feb38069420 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feb4c15e4f0 0x7feb4c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb4c0cfca0 0x7feb4c155b60 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 --2- 172.21.15.37:0/2056318171 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feb4c0b3cf0 0x7feb4c1513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 -- 172.21.15.37:0/2056318171 >> 172.21.15.37:0/2056318171 conn(0x7feb4c06ae60 msgr2=0x7feb4c1606f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 -- 172.21.15.37:0/2056318171 shutdown_connections 2024-04-25T08:53:41.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.632+0000 7feb52476640 1 -- 172.21.15.37:0/2056318171 wait complete. 2024-04-25T08:53:41.644 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:53:41.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c27dbc640 1 Processor -- start 2024-04-25T08:53:41.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c27dbc640 1 -- start start 2024-04-25T08:53:41.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c27dbc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1c200b3cf0 0x7f1c200cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c27dbc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1c200cfad0 0x7f1c2015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c27dbc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c200cbf10 con 0x7f1c2015e4f0 2024-04-25T08:53:41.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c27dbc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c200cff40 con 0x7f1c200cfad0 2024-04-25T08:53:41.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c27dbc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c200cc1f0 con 0x7f1c200b3cf0 2024-04-25T08:53:41.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c25357640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1c200cfad0 0x7f1c2015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c25357640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1c200cfad0 0x7f1c2015dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51782/0 (socket says 172.21.15.37:51782) 2024-04-25T08:53:41.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c25357640 1 -- 172.21.15.37:0/1220051111 learned_addr learned my addr 172.21.15.37:0/1220051111 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:41.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c25b58640 1 --2- 172.21.15.37:0/1220051111 >> v2:172.21.15.203:3300/0 conn(0x7f1c200b3cf0 0x7f1c200cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c25357640 1 -- 172.21.15.37:0/1220051111 >> v2:172.21.15.203:3300/0 conn(0x7f1c200b3cf0 msgr2=0x7f1c200cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:41.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c25357640 1 --2- 172.21.15.37:0/1220051111 >> v2:172.21.15.203:3300/0 conn(0x7f1c200b3cf0 0x7f1c200cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c25357640 1 -- 172.21.15.37:0/1220051111 >> v1:172.21.15.203:6789/0 conn(0x7f1c2015e4f0 legacy=0x7f1c20161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:41.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.780+0000 7f1c25357640 1 -- 172.21.15.37:0/1220051111 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c200cc4d0 con 0x7f1c200cfad0 2024-04-25T08:53:41.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c25357640 1 --2- 172.21.15.37:0/1220051111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c200cfad0 0x7f1c2015dfb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1c200cbd40 tx=0x7f1c1401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443def8b6b7cabc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c24b56640 1 -- 172.21.15.37:0/1220051111 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c140a5590 con 0x7f1c200cfad0 2024-04-25T08:53:41.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c24b56640 1 -- 172.21.15.37:0/1220051111 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c140a5730 con 0x7f1c200cfad0 2024-04-25T08:53:41.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c24b56640 1 -- 172.21.15.37:0/1220051111 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c140a5b00 con 0x7f1c200cfad0 2024-04-25T08:53:41.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 -- 172.21.15.37:0/1220051111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c200cfad0 msgr2=0x7f1c2015dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 --2- 172.21.15.37:0/1220051111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c200cfad0 0x7f1c2015dfb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1c200cbd40 tx=0x7f1c1401c9b0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 -- 172.21.15.37:0/1220051111 shutdown_connections 2024-04-25T08:53:41.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 --2- 172.21.15.37:0/1220051111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c200cfad0 0x7f1c2015dfb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 --2- 172.21.15.37:0/1220051111 >> v2:172.21.15.203:3300/0 conn(0x7f1c200b3cf0 0x7f1c200cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 -- 172.21.15.37:0/1220051111 >> 172.21.15.37:0/1220051111 conn(0x7f1c2006ae60 msgr2=0x7f1c2006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:41.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 -- 172.21.15.37:0/1220051111 shutdown_connections 2024-04-25T08:53:41.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 -- 172.21.15.37:0/1220051111 wait complete. 2024-04-25T08:53:41.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 Processor -- start 2024-04-25T08:53:41.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 -- start start 2024-04-25T08:53:41.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c200b3cf0 0x7f1c200aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c200cfad0 0x7f1c20154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c2015e4f0 0x7f1c2015d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200b4250 con 0x7f1c200cfad0 2024-04-25T08:53:41.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c200b40d0 con 0x7f1c200b3cf0 2024-04-25T08:53:41.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c27dbc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2015e910 con 0x7f1c2015e4f0 2024-04-25T08:53:41.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c25357640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c200cfad0 0x7f1c20154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c25357640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c200cfad0 0x7f1c20154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51792/0 (socket says 172.21.15.37:51792) 2024-04-25T08:53:41.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.784+0000 7f1c25357640 1 -- 172.21.15.37:0/618327790 learned_addr learned my addr 172.21.15.37:0/618327790 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:41.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c25b58640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c200b3cf0 0x7f1c200aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c26359640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c2015e4f0 0x7f1c2015d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c25357640 1 -- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c2015e4f0 msgr2=0x7f1c2015d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c25357640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c2015e4f0 0x7f1c2015d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c25357640 1 -- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c200b3cf0 msgr2=0x7f1c200aee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c25357640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c200b3cf0 0x7f1c200aee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c25357640 1 -- 172.21.15.37:0/618327790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c20154960 con 0x7f1c200cfad0 2024-04-25T08:53:41.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c26359640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c2015e4f0 0x7f1c2015d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:41.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c25357640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c200cfad0 0x7f1c20154220 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1c140af040 tx=0x7f1c14019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c127fc640 1 -- 172.21.15.37:0/618327790 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14009510 con 0x7f1c200cfad0 2024-04-25T08:53:41.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c127fc640 1 -- 172.21.15.37:0/618327790 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c140be070 con 0x7f1c200cfad0 2024-04-25T08:53:41.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c27dbc640 1 -- 172.21.15.37:0/618327790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2015b5f0 con 0x7f1c200cfad0 2024-04-25T08:53:41.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c127fc640 1 -- 172.21.15.37:0/618327790 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c140b88d0 con 0x7f1c200cfad0 2024-04-25T08:53:41.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c27dbc640 1 -- 172.21.15.37:0/618327790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2015d600 con 0x7f1c200cfad0 2024-04-25T08:53:41.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c27dbc640 1 -- 172.21.15.37:0/618327790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c20154760 con 0x7f1c200cfad0 2024-04-25T08:53:41.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c127fc640 1 -- 172.21.15.37:0/618327790 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c140b8a70 con 0x7f1c200cfad0 2024-04-25T08:53:41.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c127fc640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c04066f60 0x7f1c04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:41.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c127fc640 1 -- 172.21.15.37:0/618327790 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1c141204a0 con 0x7f1c200cfad0 2024-04-25T08:53:41.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c25b58640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c04066f60 0x7f1c04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:41.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.788+0000 7f1c25b58640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c04066f60 0x7f1c04069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c2015a0b0 tx=0x7f1c1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:41.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.792+0000 7f1c127fc640 1 -- 172.21.15.37:0/618327790 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c140165a0 con 0x7f1c200cfad0 2024-04-25T08:53:41.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.948+0000 7f1c27dbc640 1 -- 172.21.15.37:0/618327790 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c2004efa0 con 0x7f1c200cfad0 2024-04-25T08:53:41.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.948+0000 7f1c127fc640 1 -- 172.21.15.37:0/618327790 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c140f11f0 con 0x7f1c200cfad0 2024-04-25T08:53:41.942 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 -- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c04066f60 msgr2=0x7f1c04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c04066f60 0x7f1c04069420 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1c2015a0b0 tx=0x7f1c1c008040 comp rx=0 tx=0).stop 2024-04-25T08:53:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 -- 172.21.15.37:0/618327790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c200cfad0 msgr2=0x7f1c20154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c200cfad0 0x7f1c20154220 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1c140af040 tx=0x7f1c14019750 comp rx=0 tx=0).stop 2024-04-25T08:53:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 -- 172.21.15.37:0/618327790 shutdown_connections 2024-04-25T08:53:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c04066f60 0x7f1c04069420 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c2015e4f0 0x7f1c2015d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c200cfad0 0x7f1c20154220 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 --2- 172.21.15.37:0/618327790 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c200b3cf0 0x7f1c200aee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 -- 172.21.15.37:0/618327790 >> 172.21.15.37:0/618327790 conn(0x7f1c2006ae60 msgr2=0x7f1c20156c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:41.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 -- 172.21.15.37:0/618327790 shutdown_connections 2024-04-25T08:53:41.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:41.952+0000 7f1c27dbc640 1 -- 172.21.15.37:0/618327790 wait complete. 2024-04-25T08:53:41.961 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:53:42.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd8722e3640 1 Processor -- start 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd8722e3640 1 -- start start 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd8722e3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd86c0b5270 0x7fd86c0b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd8722e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86c0b5650 0x7fd86c0d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd8722e3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86c06bff0 con 0x7fd86c0d4440 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd8722e3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd86c06cc00 con 0x7fd86c0b5270 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd8722e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c06c2d0 con 0x7fd86c0b5650 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd86bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd86c0b5270 0x7fd86c0b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd870880640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd86c0d4440 0x7fd86c06d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33618/0 (socket says 172.21.15.37:33618) 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd870880640 1 -- 172.21.15.37:0/3330750560 learned_addr learned my addr 172.21.15.37:0/3330750560 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd86b7fe640 1 --2- 172.21.15.37:0/3330750560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86c0b5650 0x7fd86c0d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd86bfff640 1 -- 172.21.15.37:0/3330750560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86c0b5650 msgr2=0x7fd86c0d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd86bfff640 1 --2- 172.21.15.37:0/3330750560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86c0b5650 0x7fd86c0d3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd86bfff640 1 -- 172.21.15.37:0/3330750560 >> v1:172.21.15.37:6789/0 conn(0x7fd86c0d4440 legacy=0x7fd86c06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd86bfff640 1 -- 172.21.15.37:0/3330750560 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c06c5b0 con 0x7fd86c0b5270 2024-04-25T08:53:42.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.092+0000 7fd86b7fe640 1 --2- 172.21.15.37:0/3330750560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86c0b5650 0x7fd86c0d3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:42.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd86bfff640 1 --2- 172.21.15.37:0/3330750560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd86c0b5270 0x7fd86c0b0380 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0097a0 tx=0x7fd85c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5235e187d5fd243d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:42.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd86affd640 1 -- 172.21.15.37:0/3330750560 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0aa600 con 0x7fd86c0b5270 2024-04-25T08:53:42.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd86affd640 1 -- 172.21.15.37:0/3330750560 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c0af070 con 0x7fd86c0b5270 2024-04-25T08:53:42.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd86affd640 1 -- 172.21.15.37:0/3330750560 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0aa9c0 con 0x7fd86c0b5270 2024-04-25T08:53:42.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 -- 172.21.15.37:0/3330750560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd86c0b5270 msgr2=0x7fd86c0b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 --2- 172.21.15.37:0/3330750560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd86c0b5270 0x7fd86c0b0380 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0097a0 tx=0x7fd85c01c480 comp rx=0 tx=0).stop 2024-04-25T08:53:42.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 -- 172.21.15.37:0/3330750560 shutdown_connections 2024-04-25T08:53:42.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 --2- 172.21.15.37:0/3330750560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86c0b5650 0x7fd86c0d3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 --2- 172.21.15.37:0/3330750560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd86c0b5270 0x7fd86c0b0380 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 -- 172.21.15.37:0/3330750560 >> 172.21.15.37:0/3330750560 conn(0x7fd86c08b000 msgr2=0x7fd86c0d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:42.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 -- 172.21.15.37:0/3330750560 shutdown_connections 2024-04-25T08:53:42.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 -- 172.21.15.37:0/3330750560 wait complete. 2024-04-25T08:53:42.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 Processor -- start 2024-04-25T08:53:42.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 -- start start 2024-04-25T08:53:42.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd86c0b5270 0x7fd86c14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd86c0b5650 0x7fd86c148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86c0d4440 0x7fd86c148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c15dfa0 con 0x7fd86c0b5650 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c15de20 con 0x7fd86c0b5270 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.096+0000 7fd8722e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd86c16a720 con 0x7fd86c0d4440 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd86c0b5650 0x7fd86c148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd86c0b5650 0x7fd86c148a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51810/0 (socket says 172.21.15.37:51810) 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86b7fe640 1 -- 172.21.15.37:0/3759921963 learned_addr learned my addr 172.21.15.37:0/3759921963 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86bfff640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd86c0b5270 0x7fd86c14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86b7fe640 1 -- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86c0d4440 msgr2=0x7fd86c148f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86b7fe640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86c0d4440 0x7fd86c148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86b7fe640 1 -- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd86c0b5270 msgr2=0x7fd86c14a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86b7fe640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd86c0b5270 0x7fd86c14a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86b7fe640 1 -- 172.21.15.37:0/3759921963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86c14c5d0 con 0x7fd86c0b5650 2024-04-25T08:53:42.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86bfff640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd86c0b5270 0x7fd86c14a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:42.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86b7fe640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd86c0b5650 0x7fd86c148a00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd86000c820 tx=0x7fd86000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:42.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd868ff9640 1 -- 172.21.15.37:0/3759921963 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860005ba0 con 0x7fd86c0b5650 2024-04-25T08:53:42.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd868ff9640 1 -- 172.21.15.37:0/3759921963 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd860005d40 con 0x7fd86c0b5650 2024-04-25T08:53:42.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd8722e3640 1 -- 172.21.15.37:0/3759921963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd86c0b33a0 con 0x7fd86c0b5650 2024-04-25T08:53:42.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd868ff9640 1 -- 172.21.15.37:0/3759921963 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860007a50 con 0x7fd86c0b5650 2024-04-25T08:53:42.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd8722e3640 1 -- 172.21.15.37:0/3759921963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd86c0b3960 con 0x7fd86c0b5650 2024-04-25T08:53:42.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd8722e3640 1 -- 172.21.15.37:0/3759921963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86c04efa0 con 0x7fd86c0b5650 2024-04-25T08:53:42.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd868ff9640 1 -- 172.21.15.37:0/3759921963 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd860007bf0 con 0x7fd86c0b5650 2024-04-25T08:53:42.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd868ff9640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd858066f60 0x7fd858069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd868ff9640 1 -- 172.21.15.37:0/3759921963 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd8600854a0 con 0x7fd86c0b5650 2024-04-25T08:53:42.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.100+0000 7fd86bfff640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd858066f60 0x7fd858069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.104+0000 7fd86bfff640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd858066f60 0x7fd858069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0ae000 tx=0x7fd85c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:42.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.104+0000 7fd868ff9640 1 -- 172.21.15.37:0/3759921963 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd86c14c5d0 con 0x7fd86c0b5650 2024-04-25T08:53:42.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.264+0000 7fd8722e3640 1 -- 172.21.15.37:0/3759921963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd86c0c7070 con 0x7fd86c0b5650 2024-04-25T08:53:42.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd868ff9640 1 -- 172.21.15.37:0/3759921963 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd860015020 con 0x7fd86c0b5650 2024-04-25T08:53:42.260 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:53:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 -- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd858066f60 msgr2=0x7fd858069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd858066f60 0x7fd858069420 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd85c0ae000 tx=0x7fd85c008040 comp rx=0 tx=0).stop 2024-04-25T08:53:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 -- 172.21.15.37:0/3759921963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd86c0b5650 msgr2=0x7fd86c148a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd86c0b5650 0x7fd86c148a00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd86000c820 tx=0x7fd86000cc20 comp rx=0 tx=0).stop 2024-04-25T08:53:42.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 -- 172.21.15.37:0/3759921963 shutdown_connections 2024-04-25T08:53:42.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd858066f60 0x7fd858069420 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd86c0d4440 0x7fd86c148f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd86c0b5650 0x7fd86c148a00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 --2- 172.21.15.37:0/3759921963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd86c0b5270 0x7fd86c14a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 -- 172.21.15.37:0/3759921963 >> 172.21.15.37:0/3759921963 conn(0x7fd86c08b000 msgr2=0x7fd86c0b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:42.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 -- 172.21.15.37:0/3759921963 shutdown_connections 2024-04-25T08:53:42.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.268+0000 7fd8722e3640 1 -- 172.21.15.37:0/3759921963 wait complete. 2024-04-25T08:53:42.278 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T08:53:42.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64de367640 1 Processor -- start 2024-04-25T08:53:42.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64de367640 1 -- start start 2024-04-25T08:53:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64de367640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f64d80cf590 0x7f64d80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64de367640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f64d8072000 0x7f64d8072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64de367640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d80cbd20 con 0x7f64d815b6b0 2024-04-25T08:53:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64de367640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d815ba90 con 0x7f64d80cf590 2024-04-25T08:53:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64de367640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d80cc000 con 0x7f64d8072000 2024-04-25T08:53:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64d77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f64d80cf590 0x7f64d80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64d7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f64d815b6b0 0x7f64d80ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33620/0 (socket says 172.21.15.37:33620) 2024-04-25T08:53:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64d7fff640 1 -- 172.21.15.37:0/3256812533 learned_addr learned my addr 172.21.15.37:0/3256812533 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64dc904640 1 --2- 172.21.15.37:0/3256812533 >> v2:172.21.15.203:3300/0 conn(0x7f64d8072000 0x7f64d8072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64d6ffd640 1 -- 172.21.15.37:0/3256812533 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481144374 0 0) 0x7f64d80cbd20 con 0x7f64d815b6b0 2024-04-25T08:53:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64d77fe640 1 -- 172.21.15.37:0/3256812533 >> v2:172.21.15.203:3300/0 conn(0x7f64d8072000 msgr2=0x7f64d8072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64d77fe640 1 --2- 172.21.15.37:0/3256812533 >> v2:172.21.15.203:3300/0 conn(0x7f64d8072000 0x7f64d8072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64d77fe640 1 -- 172.21.15.37:0/3256812533 >> v1:172.21.15.37:6789/0 conn(0x7f64d815b6b0 legacy=0x7f64d80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.416+0000 7f64d77fe640 1 -- 172.21.15.37:0/3256812533 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d80cc2e0 con 0x7f64d80cf590 2024-04-25T08:53:42.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64d77fe640 1 --2- 172.21.15.37:0/3256812533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d80cf590 0x7f64d80cfa20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f64d806f000 tx=0x7f64cc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330821a66992a546 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:42.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64d6ffd640 1 -- 172.21.15.37:0/3256812533 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0b0070 con 0x7f64d80cf590 2024-04-25T08:53:42.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64d6ffd640 1 -- 172.21.15.37:0/3256812533 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc0ab600 con 0x7f64d80cf590 2024-04-25T08:53:42.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64d6ffd640 1 -- 172.21.15.37:0/3256812533 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc0ab9d0 con 0x7f64d80cf590 2024-04-25T08:53:42.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 -- 172.21.15.37:0/3256812533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d80cf590 msgr2=0x7f64d80cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 --2- 172.21.15.37:0/3256812533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d80cf590 0x7f64d80cfa20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f64d806f000 tx=0x7f64cc01c480 comp rx=0 tx=0).stop 2024-04-25T08:53:42.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 -- 172.21.15.37:0/3256812533 shutdown_connections 2024-04-25T08:53:42.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 --2- 172.21.15.37:0/3256812533 >> v2:172.21.15.203:3300/0 conn(0x7f64d8072000 0x7f64d8072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 --2- 172.21.15.37:0/3256812533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d80cf590 0x7f64d80cfa20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 -- 172.21.15.37:0/3256812533 >> 172.21.15.37:0/3256812533 conn(0x7f64d806ae60 msgr2=0x7f64d806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:42.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 -- 172.21.15.37:0/3256812533 shutdown_connections 2024-04-25T08:53:42.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 -- 172.21.15.37:0/3256812533 wait complete. 2024-04-25T08:53:42.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 Processor -- start 2024-04-25T08:53:42.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 -- start start 2024-04-25T08:53:42.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8072000 0x7f64d8153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64d80cf590 0x7f64d8154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64d815b6b0 0x7f64d8155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80caf30 con 0x7f64d8072000 2024-04-25T08:53:42.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80cadb0 con 0x7f64d815b6b0 2024-04-25T08:53:42.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64de367640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d81682a0 con 0x7f64d80cf590 2024-04-25T08:53:42.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8072000 0x7f64d8153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8072000 0x7f64d8153660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51820/0 (socket says 172.21.15.37:51820) 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.420+0000 7f64d7fff640 1 -- 172.21.15.37:0/4261102874 learned_addr learned my addr 172.21.15.37:0/4261102874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d77fe640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64d80cf590 0x7f64d8154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64dc904640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64d815b6b0 0x7f64d8155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d7fff640 1 -- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64d80cf590 msgr2=0x7f64d8154e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d7fff640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64d80cf590 0x7f64d8154e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d7fff640 1 -- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64d815b6b0 msgr2=0x7f64d8155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d7fff640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64d815b6b0 0x7f64d8155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d7fff640 1 -- 172.21.15.37:0/4261102874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d815aa90 con 0x7f64d8072000 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d7fff640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8072000 0x7f64d8153660 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f64c800c480 tx=0x7f64c800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d4ff9640 1 -- 172.21.15.37:0/4261102874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8018070 con 0x7f64d8072000 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d4ff9640 1 -- 172.21.15.37:0/4261102874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8007500 con 0x7f64d8072000 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64de367640 1 -- 172.21.15.37:0/4261102874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d8159ed0 con 0x7f64d8072000 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d4ff9640 1 -- 172.21.15.37:0/4261102874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8004a30 con 0x7f64d8072000 2024-04-25T08:53:42.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64de367640 1 -- 172.21.15.37:0/4261102874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d815a5d0 con 0x7f64d8072000 2024-04-25T08:53:42.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64de367640 1 -- 172.21.15.37:0/4261102874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d804efa0 con 0x7f64d8072000 2024-04-25T08:53:42.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d4ff9640 1 -- 172.21.15.37:0/4261102874 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64c801b070 con 0x7f64d8072000 2024-04-25T08:53:42.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d4ff9640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64c4066f60 0x7f64c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d4ff9640 1 -- 172.21.15.37:0/4261102874 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f64c80849f0 con 0x7f64d8072000 2024-04-25T08:53:42.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d77fe640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64c4066f60 0x7f64c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.424+0000 7f64d77fe640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64c4066f60 0x7f64c4069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0ad000 tx=0x7f64cc0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:42.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.428+0000 7f64d4ff9640 1 -- 172.21.15.37:0/4261102874 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64c8055770 con 0x7f64d8072000 2024-04-25T08:53:42.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.584+0000 7f64de367640 1 -- 172.21.15.37:0/4261102874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f64d81571c0 con 0x7f64d8072000 2024-04-25T08:53:42.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.584+0000 7f64d4ff9640 1 -- 172.21.15.37:0/4261102874 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) v1 ==== 103+0+11 (secure 0 0 0) 0x7f64c8055070 con 0x7f64d8072000 2024-04-25T08:53:42.578 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 -- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64c4066f60 msgr2=0x7f64c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64c4066f60 0x7f64c4069420 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0ad000 tx=0x7f64cc0a6000 comp rx=0 tx=0).stop 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 -- 172.21.15.37:0/4261102874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8072000 msgr2=0x7f64d8153660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8072000 0x7f64d8153660 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f64c800c480 tx=0x7f64c800c940 comp rx=0 tx=0).stop 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 -- 172.21.15.37:0/4261102874 shutdown_connections 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64c4066f60 0x7f64c4069420 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f64d815b6b0 0x7f64d8155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f64d80cf590 0x7f64d8154e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 --2- 172.21.15.37:0/4261102874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8072000 0x7f64d8153660 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 -- 172.21.15.37:0/4261102874 >> 172.21.15.37:0/4261102874 conn(0x7f64d806ae60 msgr2=0x7f64d80b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 -- 172.21.15.37:0/4261102874 shutdown_connections 2024-04-25T08:53:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.588+0000 7f64de367640 1 -- 172.21.15.37:0/4261102874 wait complete. 2024-04-25T08:53:42.595 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:42.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.712+0000 7f8c04b20640 1 Processor -- start 2024-04-25T08:53:42.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.712+0000 7f8c04b20640 1 -- start start 2024-04-25T08:53:42.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8c04b20640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8c0015a280 0x7f8c0015a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8c04b20640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0015ae30 con 0x7f8c000718c0 2024-04-25T08:53:42.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8c04b20640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0015b110 con 0x7f8c00159ea0 2024-04-25T08:53:42.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8c04b20640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c000b2990 con 0x7f8c0015a280 2024-04-25T08:53:42.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8bfed9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8c000718c0 0x7f8c0006d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33636/0 (socket says 172.21.15.37:33636) 2024-04-25T08:53:42.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8bfed9d640 1 -- 172.21.15.37:0/3281437980 learned_addr learned my addr 172.21.15.37:0/3281437980 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:42.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8bf7fff640 1 --2- 172.21.15.37:0/3281437980 >> v2:172.21.15.37:3300/0 conn(0x7f8c0015a280 0x7f8c0015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8bf7fff640 1 -- 172.21.15.37:0/3281437980 >> v1:172.21.15.203:6789/0 conn(0x7f8c00159ea0 legacy=0x7f8c000b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:42.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8bf7fff640 1 -- 172.21.15.37:0/3281437980 >> v1:172.21.15.37:6789/0 conn(0x7f8c000718c0 legacy=0x7f8c0006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8bf7fff640 1 -- 172.21.15.37:0/3281437980 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000b2620 con 0x7f8c0015a280 2024-04-25T08:53:42.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8bfdd9b640 1 -- 172.21.15.37:0/3281437980 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289918725 0 0) 0x7f8c0015ae30 con 0x7f8c000718c0 2024-04-25T08:53:42.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8bf7fff640 1 --2- 172.21.15.37:0/3281437980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c0015a280 0x7f8c0015a700 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8be80097c0 tx=0x7f8be801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c887fe4d8e8ef2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:42.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8bfdd9b640 1 -- 172.21.15.37:0/3281437980 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be80b0070 con 0x7f8c0015a280 2024-04-25T08:53:42.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8bfdd9b640 1 -- 172.21.15.37:0/3281437980 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be80ab600 con 0x7f8c0015a280 2024-04-25T08:53:42.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8bfdd9b640 1 -- 172.21.15.37:0/3281437980 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be80ab9d0 con 0x7f8c0015a280 2024-04-25T08:53:42.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8c04b20640 1 -- 172.21.15.37:0/3281437980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c0015a280 msgr2=0x7f8c0015a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8c04b20640 1 --2- 172.21.15.37:0/3281437980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c0015a280 0x7f8c0015a700 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8be80097c0 tx=0x7f8be801c460 comp rx=0 tx=0).stop 2024-04-25T08:53:42.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8c04b20640 1 -- 172.21.15.37:0/3281437980 shutdown_connections 2024-04-25T08:53:42.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8c04b20640 1 --2- 172.21.15.37:0/3281437980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c0015a280 0x7f8c0015a700 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8c04b20640 1 -- 172.21.15.37:0/3281437980 >> 172.21.15.37:0/3281437980 conn(0x7f8c000b11a0 msgr2=0x7f8c000b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:42.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8c04b20640 1 -- 172.21.15.37:0/3281437980 shutdown_connections 2024-04-25T08:53:42.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.716+0000 7f8c04b20640 1 -- 172.21.15.37:0/3281437980 wait complete. 2024-04-25T08:53:42.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8c04b20640 1 Processor -- start 2024-04-25T08:53:42.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8c04b20640 1 -- start start 2024-04-25T08:53:42.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8c04b20640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c000718c0 0x7f8c0013b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8c04b20640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c00159ea0 0x7f8c001f7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8c04b20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c0015a280 0x7f8c001502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8c04b20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c001683e0 con 0x7f8c0015a280 2024-04-25T08:53:42.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8c04b20640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c000b3530 con 0x7f8c000718c0 2024-04-25T08:53:42.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8c04b20640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00168560 con 0x7f8c00159ea0 2024-04-25T08:53:42.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bfed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c0015a280 0x7f8c001502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bfed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c0015a280 0x7f8c001502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51826/0 (socket says 172.21.15.37:51826) 2024-04-25T08:53:42.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bfed9d640 1 -- 172.21.15.37:0/4024605626 learned_addr learned my addr 172.21.15.37:0/4024605626 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:42.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bfe59c640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c000718c0 0x7f8c0013b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bf7fff640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c00159ea0 0x7f8c001f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bfed9d640 1 -- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c00159ea0 msgr2=0x7f8c001f7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bfed9d640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c00159ea0 0x7f8c001f7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bfed9d640 1 -- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c000718c0 msgr2=0x7f8c0013b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bfed9d640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c000718c0 0x7f8c0013b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bfed9d640 1 -- 172.21.15.37:0/4024605626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c00154070 con 0x7f8c0015a280 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bf7fff640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c00159ea0 0x7f8c001f7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bfed9d640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c0015a280 0x7f8c001502a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800c7d0 tx=0x7f8bf800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bf6ffd640 1 -- 172.21.15.37:0/4024605626 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8018070 con 0x7f8c0015a280 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bf6ffd640 1 -- 172.21.15.37:0/4024605626 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf800a640 con 0x7f8c0015a280 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8c04b20640 1 -- 172.21.15.37:0/4024605626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0013c0c0 con 0x7f8c0015a280 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8bf6ffd640 1 -- 172.21.15.37:0/4024605626 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800aa70 con 0x7f8c0015a280 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8c04b20640 1 -- 172.21.15.37:0/4024605626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0013c680 con 0x7f8c0015a280 2024-04-25T08:53:42.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.720+0000 7f8c04b20640 1 -- 172.21.15.37:0/4024605626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c0004efa0 con 0x7f8c0015a280 2024-04-25T08:53:42.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.724+0000 7f8bf6ffd640 1 -- 172.21.15.37:0/4024605626 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8bf801b070 con 0x7f8c0015a280 2024-04-25T08:53:42.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.724+0000 7f8bf6ffd640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8be0066f60 0x7f8be0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:42.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.724+0000 7f8bf6ffd640 1 -- 172.21.15.37:0/4024605626 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8bf80849c0 con 0x7f8c0015a280 2024-04-25T08:53:42.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.724+0000 7f8bfe59c640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8be0066f60 0x7f8be0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:42.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.724+0000 7f8bfe59c640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8be0066f60 0x7f8be0069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0004e30 tx=0x7f8bf0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:42.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.728+0000 7f8bf6ffd640 1 -- 172.21.15.37:0/4024605626 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8bf8086e20 con 0x7f8c0015a280 2024-04-25T08:53:42.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.916+0000 7f8c04b20640 1 -- 172.21.15.37:0/4024605626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c000cf1a0 con 0x7f8c0015a280 2024-04-25T08:53:42.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.916+0000 7f8bf6ffd640 1 -- 172.21.15.37:0/4024605626 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f8bf8055660 con 0x7f8c0015a280 2024-04-25T08:53:42.911 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:42.911 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2902565757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2902565757},{"type":"v1","addr":"172.21.15.203:6835","nonce":2902565757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/822172299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":822172299},{"type":"v1","addr":"172.21.15.203:6837","nonce":822172299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3681855890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3681855890},{"type":"v1","addr":"172.21.15.37:6833","nonce":3681855890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:30.564153+0000","modified":"2024-04-25T08:52:53.726659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:53:42.911 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 -- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8be0066f60 msgr2=0x7f8be0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8be0066f60 0x7f8be0069420 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0004e30 tx=0x7f8bf0009040 comp rx=0 tx=0).stop 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 -- 172.21.15.37:0/4024605626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c0015a280 msgr2=0x7f8c001502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c0015a280 0x7f8c001502a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800c7d0 tx=0x7f8bf800cc90 comp rx=0 tx=0).stop 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 -- 172.21.15.37:0/4024605626 shutdown_connections 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8be0066f60 0x7f8be0069420 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c0015a280 0x7f8c001502a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c00159ea0 0x7f8c001f7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 --2- 172.21.15.37:0/4024605626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c000718c0 0x7f8c0013b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 -- 172.21.15.37:0/4024605626 >> 172.21.15.37:0/4024605626 conn(0x7f8c000b11a0 msgr2=0x7f8c00073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 -- 172.21.15.37:0/4024605626 shutdown_connections 2024-04-25T08:53:42.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:42.920+0000 7f8c04b20640 1 -- 172.21.15.37:0/4024605626 wait complete. 2024-04-25T08:53:42.928 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f139696a640 1 Processor -- start 2024-04-25T08:53:43.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f139696a640 1 -- start start 2024-04-25T08:53:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f139696a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f139015a700 0x7f139015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f139696a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f13900b24a0 0x7f13900b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f139696a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13900b2090 con 0x7f13900b3f50 2024-04-25T08:53:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f139696a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f139015f200 con 0x7f13900b24a0 2024-04-25T08:53:43.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f139696a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13900b43e0 con 0x7f139015a700 2024-04-25T08:53:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f1394f07640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f13900b24a0 0x7f13900b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f1394f07640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f13900b24a0 0x7f13900b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51840/0 (socket says 172.21.15.37:51840) 2024-04-25T08:53:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f1394f07640 1 -- 172.21.15.37:0/477261918 learned_addr learned my addr 172.21.15.37:0/477261918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f138ffff640 1 --2- 172.21.15.37:0/477261918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f139015a700 0x7f139015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f1394f07640 1 -- 172.21.15.37:0/477261918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f139015a700 msgr2=0x7f139015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f1394f07640 1 --2- 172.21.15.37:0/477261918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f139015a700 0x7f139015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f1394f07640 1 -- 172.21.15.37:0/477261918 >> v1:172.21.15.203:6789/0 conn(0x7f13900b3f50 legacy=0x7f13900cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:43.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f1394f07640 1 -- 172.21.15.37:0/477261918 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13900b46c0 con 0x7f13900b24a0 2024-04-25T08:53:43.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.048+0000 7f1394f07640 1 --2- 172.21.15.37:0/477261918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13900b24a0 0x7f13900b2930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f138800d150 tx=0x7f138801ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a0c5760785c10d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:43.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138effd640 1 -- 172.21.15.37:0/477261918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138800b2d0 con 0x7f13900b24a0 2024-04-25T08:53:43.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138effd640 1 -- 172.21.15.37:0/477261918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138800b470 con 0x7f13900b24a0 2024-04-25T08:53:43.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138effd640 1 -- 172.21.15.37:0/477261918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13880a7a70 con 0x7f13900b24a0 2024-04-25T08:53:43.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 -- 172.21.15.37:0/477261918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13900b24a0 msgr2=0x7f13900b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 --2- 172.21.15.37:0/477261918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13900b24a0 0x7f13900b2930 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f138800d150 tx=0x7f138801ead0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 -- 172.21.15.37:0/477261918 shutdown_connections 2024-04-25T08:53:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 --2- 172.21.15.37:0/477261918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13900b24a0 0x7f13900b2930 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 --2- 172.21.15.37:0/477261918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f139015a700 0x7f139015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 -- 172.21.15.37:0/477261918 >> 172.21.15.37:0/477261918 conn(0x7f13900b11a0 msgr2=0x7f13900b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 -- 172.21.15.37:0/477261918 shutdown_connections 2024-04-25T08:53:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 -- 172.21.15.37:0/477261918 wait complete. 2024-04-25T08:53:43.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 Processor -- start 2024-04-25T08:53:43.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 -- start start 2024-04-25T08:53:43.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13900b24a0 0x7f1390164d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f139015a700 0x7f13900b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f139013f910 0x7f1390153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13900cd310 con 0x7f139015a700 2024-04-25T08:53:43.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13900ce600 con 0x7f13900b24a0 2024-04-25T08:53:43.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f139696a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13900cd490 con 0x7f139013f910 2024-04-25T08:53:43.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f139015a700 0x7f13900b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f139015a700 0x7f13900b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51850/0 (socket says 172.21.15.37:51850) 2024-04-25T08:53:43.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138f7fe640 1 -- 172.21.15.37:0/3840082151 learned_addr learned my addr 172.21.15.37:0/3840082151 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:43.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138ffff640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13900b24a0 0x7f1390164d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f1394f07640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f139013f910 0x7f1390153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138f7fe640 1 -- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f139013f910 msgr2=0x7f1390153a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138f7fe640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f139013f910 0x7f1390153a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138f7fe640 1 -- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13900b24a0 msgr2=0x7f1390164d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138f7fe640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13900b24a0 0x7f1390164d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138f7fe640 1 -- 172.21.15.37:0/3840082151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139013fda0 con 0x7f139015a700 2024-04-25T08:53:43.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.052+0000 7f138ffff640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13900b24a0 0x7f1390164d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:43.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f1394f07640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f139013f910 0x7f1390153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f138f7fe640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f139015a700 0x7f13900b08c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f138400c450 tx=0x7f138400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f138cff9640 1 -- 172.21.15.37:0/3840082151 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384012830 con 0x7f139015a700 2024-04-25T08:53:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f138cff9640 1 -- 172.21.15.37:0/3840082151 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1384017070 con 0x7f139015a700 2024-04-25T08:53:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f139696a640 1 -- 172.21.15.37:0/3840082151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1390140080 con 0x7f139015a700 2024-04-25T08:53:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f138cff9640 1 -- 172.21.15.37:0/3840082151 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384012c30 con 0x7f139015a700 2024-04-25T08:53:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f139696a640 1 -- 172.21.15.37:0/3840082151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1390140640 con 0x7f139015a700 2024-04-25T08:53:43.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f139696a640 1 -- 172.21.15.37:0/3840082151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f139004efa0 con 0x7f139015a700 2024-04-25T08:53:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f138cff9640 1 -- 172.21.15.37:0/3840082151 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f138401c460 con 0x7f139015a700 2024-04-25T08:53:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f138cff9640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f137c066f60 0x7f137c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f138cff9640 1 -- 172.21.15.37:0/3840082151 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1384085200 con 0x7f139015a700 2024-04-25T08:53:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f138ffff640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f137c066f60 0x7f137c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.056+0000 7f138ffff640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f137c066f60 0x7f137c069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1380009840 tx=0x7f1380008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:43.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.060+0000 7f138cff9640 1 -- 172.21.15.37:0/3840082151 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1384055e10 con 0x7f139015a700 2024-04-25T08:53:43.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.220+0000 7f139696a640 1 -- 172.21.15.37:0/3840082151 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f139006f260 con 0x7f139015a700 2024-04-25T08:53:43.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.224+0000 7f138cff9640 1 -- 172.21.15.37:0/3840082151 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f1384055710 con 0x7f139015a700 2024-04-25T08:53:43.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:43.217 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:43.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.224+0000 7f139696a640 1 -- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f137c066f60 msgr2=0x7f137c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.224+0000 7f139696a640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f137c066f60 0x7f137c069420 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1380009840 tx=0x7f1380008040 comp rx=0 tx=0).stop 2024-04-25T08:53:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.224+0000 7f139696a640 1 -- 172.21.15.37:0/3840082151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f139015a700 msgr2=0x7f13900b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.224+0000 7f139696a640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f139015a700 0x7f13900b08c0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f138400c450 tx=0x7f138400c910 comp rx=0 tx=0).stop 2024-04-25T08:53:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.224+0000 7f139696a640 1 -- 172.21.15.37:0/3840082151 shutdown_connections 2024-04-25T08:53:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.224+0000 7f139696a640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f137c066f60 0x7f137c069420 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.224+0000 7f139696a640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f139013f910 0x7f1390153a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.224+0000 7f139696a640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f139015a700 0x7f13900b08c0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.224+0000 7f139696a640 1 --2- 172.21.15.37:0/3840082151 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13900b24a0 0x7f1390164d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.224+0000 7f139696a640 1 -- 172.21.15.37:0/3840082151 >> 172.21.15.37:0/3840082151 conn(0x7f13900b11a0 msgr2=0x7f1390071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.228+0000 7f139696a640 1 -- 172.21.15.37:0/3840082151 shutdown_connections 2024-04-25T08:53:43.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.228+0000 7f139696a640 1 -- 172.21.15.37:0/3840082151 wait complete. 2024-04-25T08:53:43.236 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:53:43.237 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:43.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f962a8c6640 1 Processor -- start 2024-04-25T08:53:43.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f962a8c6640 1 -- start start 2024-04-25T08:53:43.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f962a8c6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f962406b440 0x7f96240b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f962a8c6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f962406b820 0x7f96240b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f962a8c6640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96240afb20 con 0x7f96240b18f0 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f962a8c6640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96240b06c0 con 0x7f962406b440 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f962a8c6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96240afd90 con 0x7f962406b820 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f9628e63640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f96240b18f0 0x7f96240cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33646/0 (socket says 172.21.15.37:33646) 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f9628e63640 1 -- 172.21.15.37:0/38402614 learned_addr learned my addr 172.21.15.37:0/38402614 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f9623fff640 1 --2- 172.21.15.37:0/38402614 >> v2:172.21.15.37:3300/0 conn(0x7f962406b440 0x7f96240b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f96237fe640 1 --2- 172.21.15.37:0/38402614 >> v2:172.21.15.203:3300/0 conn(0x7f962406b820 0x7f96240b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f9622ffd640 1 -- 172.21.15.37:0/38402614 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871601083 0 0) 0x7f96240afb20 con 0x7f96240b18f0 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f9622ffd640 1 -- 172.21.15.37:0/38402614 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f960c003530 con 0x7f96240b18f0 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f9623fff640 1 -- 172.21.15.37:0/38402614 >> v2:172.21.15.203:3300/0 conn(0x7f962406b820 msgr2=0x7f96240b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f9623fff640 1 --2- 172.21.15.37:0/38402614 >> v2:172.21.15.203:3300/0 conn(0x7f962406b820 0x7f96240b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f9623fff640 1 -- 172.21.15.37:0/38402614 >> v1:172.21.15.37:6789/0 conn(0x7f96240b18f0 legacy=0x7f96240cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.368+0000 7f9623fff640 1 -- 172.21.15.37:0/38402614 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96240b1d40 con 0x7f962406b440 2024-04-25T08:53:43.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f9623fff640 1 --2- 172.21.15.37:0/38402614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962406b440 0x7f96240b0b50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9624161530 tx=0x7f961401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65bf702fdfc06cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:43.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f9622ffd640 1 -- 172.21.15.37:0/38402614 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96140b0070 con 0x7f962406b440 2024-04-25T08:53:43.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f9622ffd640 1 -- 172.21.15.37:0/38402614 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96140ab600 con 0x7f962406b440 2024-04-25T08:53:43.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f9622ffd640 1 -- 172.21.15.37:0/38402614 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96140ab9d0 con 0x7f962406b440 2024-04-25T08:53:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 -- 172.21.15.37:0/38402614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962406b440 msgr2=0x7f96240b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 --2- 172.21.15.37:0/38402614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962406b440 0x7f96240b0b50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9624161530 tx=0x7f961401c480 comp rx=0 tx=0).stop 2024-04-25T08:53:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 -- 172.21.15.37:0/38402614 shutdown_connections 2024-04-25T08:53:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 --2- 172.21.15.37:0/38402614 >> v2:172.21.15.203:3300/0 conn(0x7f962406b820 0x7f96240b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 --2- 172.21.15.37:0/38402614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962406b440 0x7f96240b0b50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 -- 172.21.15.37:0/38402614 >> 172.21.15.37:0/38402614 conn(0x7f96240b1f10 msgr2=0x7f96240b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:43.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 -- 172.21.15.37:0/38402614 shutdown_connections 2024-04-25T08:53:43.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 -- 172.21.15.37:0/38402614 wait complete. 2024-04-25T08:53:43.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 Processor -- start 2024-04-25T08:53:43.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 -- start start 2024-04-25T08:53:43.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f962406b440 0x7f9624138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962406b820 0x7f962406e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96240b18f0 0x7f962406e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96240cd130 con 0x7f962406b820 2024-04-25T08:53:43.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96240ccfb0 con 0x7f962406b440 2024-04-25T08:53:43.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f962a8c6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962415d080 con 0x7f96240b18f0 2024-04-25T08:53:43.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f96237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962406b820 0x7f962406e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f96237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962406b820 0x7f962406e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51878/0 (socket says 172.21.15.37:51878) 2024-04-25T08:53:43.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f96237fe640 1 -- 172.21.15.37:0/2176240291 learned_addr learned my addr 172.21.15.37:0/2176240291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:43.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f9623fff640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f962406b440 0x7f9624138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f9628e63640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96240b18f0 0x7f962406e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f96237fe640 1 -- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96240b18f0 msgr2=0x7f962406e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f96237fe640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96240b18f0 0x7f962406e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f96237fe640 1 -- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f962406b440 msgr2=0x7f9624138860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f96237fe640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f962406b440 0x7f9624138860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f96237fe640 1 -- 172.21.15.37:0/2176240291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962413c170 con 0x7f962406b820 2024-04-25T08:53:43.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f9628e63640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96240b18f0 0x7f962406e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:43.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.372+0000 7f96237fe640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962406b820 0x7f962406e0b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f961800d7d0 tx=0x7f961800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.376+0000 7f9620ff9640 1 -- 172.21.15.37:0/2176240291 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961800b6c0 con 0x7f962406b820 2024-04-25T08:53:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.376+0000 7f9620ff9640 1 -- 172.21.15.37:0/2176240291 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9618019070 con 0x7f962406b820 2024-04-25T08:53:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.376+0000 7f962a8c6640 1 -- 172.21.15.37:0/2176240291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9624138da0 con 0x7f962406b820 2024-04-25T08:53:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.376+0000 7f9620ff9640 1 -- 172.21.15.37:0/2176240291 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961800baa0 con 0x7f962406b820 2024-04-25T08:53:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.376+0000 7f962a8c6640 1 -- 172.21.15.37:0/2176240291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9624151ec0 con 0x7f962406b820 2024-04-25T08:53:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.376+0000 7f962a8c6640 1 -- 172.21.15.37:0/2176240291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962404efa0 con 0x7f962406b820 2024-04-25T08:53:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.376+0000 7f9620ff9640 1 -- 172.21.15.37:0/2176240291 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f961800bc40 con 0x7f962406b820 2024-04-25T08:53:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.376+0000 7f9620ff9640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9610066f60 0x7f9610069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.376+0000 7f9620ff9640 1 -- 172.21.15.37:0/2176240291 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f96180858c0 con 0x7f962406b820 2024-04-25T08:53:43.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.376+0000 7f9623fff640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9610066f60 0x7f9610069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.380+0000 7f9623fff640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9610066f60 0x7f9610069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f96140ad000 tx=0x7f9614008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:43.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.380+0000 7f9620ff9640 1 -- 172.21.15.37:0/2176240291 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f96180564d0 con 0x7f962406b820 2024-04-25T08:53:43.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.576+0000 7f962a8c6640 1 -- 172.21.15.37:0/2176240291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96240cc160 con 0x7f962406b820 2024-04-25T08:53:43.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.576+0000 7f9620ff9640 1 -- 172.21.15.37:0/2176240291 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f9618055dd0 con 0x7f962406b820 2024-04-25T08:53:43.571 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:43.571 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2902565757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2902565757},{"type":"v1","addr":"172.21.15.203:6835","nonce":2902565757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/822172299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":822172299},{"type":"v1","addr":"172.21.15.203:6837","nonce":822172299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3681855890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3681855890},{"type":"v1","addr":"172.21.15.37:6833","nonce":3681855890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:30.564153+0000","modified":"2024-04-25T08:52:53.726659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:53:43.571 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-25T08:53:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 -- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9610066f60 msgr2=0x7f9610069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9610066f60 0x7f9610069420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f96140ad000 tx=0x7f9614008040 comp rx=0 tx=0).stop 2024-04-25T08:53:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 -- 172.21.15.37:0/2176240291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962406b820 msgr2=0x7f962406e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962406b820 0x7f962406e0b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f961800d7d0 tx=0x7f961800dc90 comp rx=0 tx=0).stop 2024-04-25T08:53:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 -- 172.21.15.37:0/2176240291 shutdown_connections 2024-04-25T08:53:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9610066f60 0x7f9610069420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96240b18f0 0x7f962406e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f962406b820 0x7f962406e0b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 --2- 172.21.15.37:0/2176240291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f962406b440 0x7f9624138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 -- 172.21.15.37:0/2176240291 >> 172.21.15.37:0/2176240291 conn(0x7f96240b1f10 msgr2=0x7f9624158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 -- 172.21.15.37:0/2176240291 shutdown_connections 2024-04-25T08:53:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.580+0000 7f962a8c6640 1 -- 172.21.15.37:0/2176240291 wait complete. 2024-04-25T08:53:43.591 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.720+0000 7f95fcab8640 1 Processor -- start 2024-04-25T08:53:43.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.720+0000 7f95fcab8640 1 -- start start 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95fcab8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f95f8144ca0 0x7f95f806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95fcab8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f806ec60 0x7f95f8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95fcab8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f80ccc70 con 0x7f95f80ce510 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95fcab8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f80cc2b0 con 0x7f95f8144ca0 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95fcab8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f806e290 con 0x7f95f806ec60 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f95f80ce510 0x7f95f80ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33654/0 (socket says 172.21.15.37:33654) 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f659c640 1 -- 172.21.15.37:0/1895902583 learned_addr learned my addr 172.21.15.37:0/1895902583 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f5d9b640 1 --2- 172.21.15.37:0/1895902583 >> v2:172.21.15.37:3300/0 conn(0x7f95f8144ca0 0x7f95f806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f559a640 1 -- 172.21.15.37:0/1895902583 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825508543 0 0) 0x7f95f80ccc70 con 0x7f95f80ce510 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f6d9d640 1 --2- 172.21.15.37:0/1895902583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f806ec60 0x7f95f8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f559a640 1 -- 172.21.15.37:0/1895902583 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95e4003530 con 0x7f95f80ce510 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f5d9b640 1 -- 172.21.15.37:0/1895902583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f806ec60 msgr2=0x7f95f8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:43.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f5d9b640 1 --2- 172.21.15.37:0/1895902583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f806ec60 0x7f95f8142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f5d9b640 1 -- 172.21.15.37:0/1895902583 >> v1:172.21.15.37:6789/0 conn(0x7f95f80ce510 legacy=0x7f95f80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f559a640 1 -- 172.21.15.37:0/1895902583 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4026322104 0 0) 0x7f95e4003530 con 0x7f95f80ce510 2024-04-25T08:53:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f5d9b640 1 -- 172.21.15.37:0/1895902583 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f806dc40 con 0x7f95f8144ca0 2024-04-25T08:53:43.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f6d9d640 1 --2- 172.21.15.37:0/1895902583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f806ec60 0x7f95f8142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:43.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f5d9b640 1 --2- 172.21.15.37:0/1895902583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f8144ca0 0x7f95f806e720 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f95e00097c0 tx=0x7f95e001ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4698f385acd5041a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:43.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f559a640 1 -- 172.21.15.37:0/1895902583 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e00b5070 con 0x7f95f8144ca0 2024-04-25T08:53:43.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f559a640 1 -- 172.21.15.37:0/1895902583 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e00a85f0 con 0x7f95f8144ca0 2024-04-25T08:53:43.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.724+0000 7f95f559a640 1 -- 172.21.15.37:0/1895902583 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0001a00 con 0x7f95f8144ca0 2024-04-25T08:53:43.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 -- 172.21.15.37:0/1895902583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f8144ca0 msgr2=0x7f95f806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 --2- 172.21.15.37:0/1895902583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f8144ca0 0x7f95f806e720 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f95e00097c0 tx=0x7f95e001ebb0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 -- 172.21.15.37:0/1895902583 shutdown_connections 2024-04-25T08:53:43.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 --2- 172.21.15.37:0/1895902583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f806ec60 0x7f95f8142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 --2- 172.21.15.37:0/1895902583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f8144ca0 0x7f95f806e720 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 -- 172.21.15.37:0/1895902583 >> 172.21.15.37:0/1895902583 conn(0x7f95f80b11a0 msgr2=0x7f95f80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:43.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 -- 172.21.15.37:0/1895902583 shutdown_connections 2024-04-25T08:53:43.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 -- 172.21.15.37:0/1895902583 wait complete. 2024-04-25T08:53:43.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 Processor -- start 2024-04-25T08:53:43.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 -- start start 2024-04-25T08:53:43.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95f806ec60 0x7f95f80b08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f80ce510 0x7f95f80a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f8144ca0 0x7f95f80a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f81451f0 con 0x7f95f80ce510 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80cc900 con 0x7f95f806ec60 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f8145370 con 0x7f95f8144ca0 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95f5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f80ce510 0x7f95f80a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95f5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f80ce510 0x7f95f80a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51894/0 (socket says 172.21.15.37:51894) 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95f5d9b640 1 -- 172.21.15.37:0/1072960614 learned_addr learned my addr 172.21.15.37:0/1072960614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95f659c640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95f806ec60 0x7f95f80b08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95f5d9b640 1 -- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f8144ca0 msgr2=0x7f95f80a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95f5d9b640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f8144ca0 0x7f95f80a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95f5d9b640 1 -- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95f806ec60 msgr2=0x7f95f80b08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95f5d9b640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95f806ec60 0x7f95f80b08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95f5d9b640 1 -- 172.21.15.37:0/1072960614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f81e1790 con 0x7f95f80ce510 2024-04-25T08:53:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95f5d9b640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f80ce510 0x7f95f80a84d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f95e0000c10 tx=0x7f95e001b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:43.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95eeffd640 1 -- 172.21.15.37:0/1072960614 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e00b5040 con 0x7f95f80ce510 2024-04-25T08:53:43.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95eeffd640 1 -- 172.21.15.37:0/1072960614 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e00b0440 con 0x7f95f80ce510 2024-04-25T08:53:43.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 -- 172.21.15.37:0/1072960614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f81e1a30 con 0x7f95f80ce510 2024-04-25T08:53:43.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95eeffd640 1 -- 172.21.15.37:0/1072960614 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e0006e20 con 0x7f95f80ce510 2024-04-25T08:53:43.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.728+0000 7f95fcab8640 1 -- 172.21.15.37:0/1072960614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f81e1ff0 con 0x7f95f80ce510 2024-04-25T08:53:43.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.732+0000 7f95fcab8640 1 -- 172.21.15.37:0/1072960614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f804efa0 con 0x7f95f80ce510 2024-04-25T08:53:43.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.732+0000 7f95eeffd640 1 -- 172.21.15.37:0/1072960614 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95e00c2030 con 0x7f95f80ce510 2024-04-25T08:53:43.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.732+0000 7f95eeffd640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95d8066f60 0x7f95d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:43.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.732+0000 7f95eeffd640 1 -- 172.21.15.37:0/1072960614 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f95e0121fb0 con 0x7f95f80ce510 2024-04-25T08:53:43.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.732+0000 7f95f659c640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95d8066f60 0x7f95d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:43.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.732+0000 7f95f659c640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95d8066f60 0x7f95d8069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95f0002240 tx=0x7f95f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:43.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.736+0000 7f95eeffd640 1 -- 172.21.15.37:0/1072960614 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95e00c9020 con 0x7f95f80ce510 2024-04-25T08:53:43.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 -- 172.21.15.37:0/1072960614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f95f80ce8f0 con 0x7f95f80ce510 2024-04-25T08:53:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95eeffd640 1 -- 172.21.15.37:0/1072960614 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4937 (secure 0 0 0) 0x7f95e0018710 con 0x7f95f80ce510 2024-04-25T08:53:43.921 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:43.921 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/2902565757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2902565757},{"type":"v1","addr":"172.21.15.203:6835","nonce":2902565757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/822172299","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":822172299},{"type":"v1","addr":"172.21.15.203:6837","nonce":822172299}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4248,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3681855890","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3681855890},{"type":"v1","addr":"172.21.15.37:6833","nonce":3681855890}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:52:30.564153+0000","modified":"2024-04-25T08:52:53.726659+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1246598982","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1246598982},{"type":"v1","addr":"172.21.15.37:6835","nonce":1246598982}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:53:43.921 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-25T08:53:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 -- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95d8066f60 msgr2=0x7f95d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95d8066f60 0x7f95d8069420 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f95f0002240 tx=0x7f95f0009040 comp rx=0 tx=0).stop 2024-04-25T08:53:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 -- 172.21.15.37:0/1072960614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f80ce510 msgr2=0x7f95f80a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f80ce510 0x7f95f80a84d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f95e0000c10 tx=0x7f95e001b950 comp rx=0 tx=0).stop 2024-04-25T08:53:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 -- 172.21.15.37:0/1072960614 shutdown_connections 2024-04-25T08:53:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95d8066f60 0x7f95d8069420 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95f8144ca0 0x7f95f80a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95f80ce510 0x7f95f80a84d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 --2- 172.21.15.37:0/1072960614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95f806ec60 0x7f95f80b08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 -- 172.21.15.37:0/1072960614 >> 172.21.15.37:0/1072960614 conn(0x7f95f80b11a0 msgr2=0x7f95f80c62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 -- 172.21.15.37:0/1072960614 shutdown_connections 2024-04-25T08:53:43.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:43.928+0000 7f95fcab8640 1 -- 172.21.15.37:0/1072960614 wait complete. 2024-04-25T08:53:43.939 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:44.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f3ec4640 1 Processor -- start 2024-04-25T08:53:44.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f3ec4640 1 -- start start 2024-04-25T08:53:44.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f3ec4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f04ec15da80 0x7f04ec0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f3ec4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04ec0d1cd0 0x7f04ec0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f3ec4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04ec06be60 con 0x7f04ec0d0620 2024-04-25T08:53:44.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f3ec4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04ec06c440 con 0x7f04ec15da80 2024-04-25T08:53:44.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f3ec4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec15de60 con 0x7f04ec0d1cd0 2024-04-25T08:53:44.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f1c60640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f04ec15da80 0x7f04ec0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f1c60640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f04ec15da80 0x7f04ec0d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51896/0 (socket says 172.21.15.37:51896) 2024-04-25T08:53:44.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f1c60640 1 -- 172.21.15.37:0/2719747419 learned_addr learned my addr 172.21.15.37:0/2719747419 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:44.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f1c60640 1 -- 172.21.15.37:0/2719747419 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04ec0d1cd0 msgr2=0x7f04ec0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:44.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f1c60640 1 --2- 172.21.15.37:0/2719747419 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04ec0d1cd0 0x7f04ec0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f1c60640 1 -- 172.21.15.37:0/2719747419 >> v1:172.21.15.203:6789/0 conn(0x7f04ec0d0620 legacy=0x7f04ec0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:44.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.060+0000 7f04f1c60640 1 -- 172.21.15.37:0/2719747419 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ec0d2450 con 0x7f04ec15da80 2024-04-25T08:53:44.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f1c60640 1 --2- 172.21.15.37:0/2719747419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04ec15da80 0x7f04ec0d4230 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009870 tx=0x7f04e801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27396532356081fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:44.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f0c5e640 1 -- 172.21.15.37:0/2719747419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e80ab5f0 con 0x7f04ec15da80 2024-04-25T08:53:44.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f0c5e640 1 -- 172.21.15.37:0/2719747419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e80b0070 con 0x7f04ec15da80 2024-04-25T08:53:44.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f0c5e640 1 -- 172.21.15.37:0/2719747419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e80ab9b0 con 0x7f04ec15da80 2024-04-25T08:53:44.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 -- 172.21.15.37:0/2719747419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04ec15da80 msgr2=0x7f04ec0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 --2- 172.21.15.37:0/2719747419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04ec15da80 0x7f04ec0d4230 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f04e8009870 tx=0x7f04e801c520 comp rx=0 tx=0).stop 2024-04-25T08:53:44.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 -- 172.21.15.37:0/2719747419 shutdown_connections 2024-04-25T08:53:44.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 --2- 172.21.15.37:0/2719747419 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04ec0d1cd0 0x7f04ec0d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 --2- 172.21.15.37:0/2719747419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04ec15da80 0x7f04ec0d4230 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 -- 172.21.15.37:0/2719747419 >> 172.21.15.37:0/2719747419 conn(0x7f04ec08b000 msgr2=0x7f04ec0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:44.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 -- 172.21.15.37:0/2719747419 shutdown_connections 2024-04-25T08:53:44.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 -- 172.21.15.37:0/2719747419 wait complete. 2024-04-25T08:53:44.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 Processor -- start 2024-04-25T08:53:44.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 -- start start 2024-04-25T08:53:44.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04ec0d0620 0x7f04ec154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04ec0d1cd0 0x7f04ec0b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04ec15da80 0x7f04ec15c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec16a730 con 0x7f04ec15da80 2024-04-25T08:53:44.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec16a5b0 con 0x7f04ec0d1cd0 2024-04-25T08:53:44.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04ec16a8b0 con 0x7f04ec0d0620 2024-04-25T08:53:44.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f2461640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04ec15da80 0x7f04ec15c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f2461640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04ec15da80 0x7f04ec15c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51898/0 (socket says 172.21.15.37:51898) 2024-04-25T08:53:44.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f2461640 1 -- 172.21.15.37:0/3721417106 learned_addr learned my addr 172.21.15.37:0/3721417106 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:44.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f2461640 1 -- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04ec0d0620 msgr2=0x7f04ec154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f2461640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04ec0d0620 0x7f04ec154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f145f640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04ec0d1cd0 0x7f04ec0b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f2461640 1 -- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04ec0d1cd0 msgr2=0x7f04ec0b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f2461640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04ec0d1cd0 0x7f04ec0b3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f2461640 1 -- 172.21.15.37:0/3721417106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04ec15f620 con 0x7f04ec15da80 2024-04-25T08:53:44.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f2461640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04ec15da80 0x7f04ec15c790 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f04e400c7b0 tx=0x7f04e400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:44.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04e27fc640 1 -- 172.21.15.37:0/3721417106 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e4017070 con 0x7f04ec15da80 2024-04-25T08:53:44.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04e27fc640 1 -- 172.21.15.37:0/3721417106 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04e4003db0 con 0x7f04ec15da80 2024-04-25T08:53:44.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04f3ec4640 1 -- 172.21.15.37:0/3721417106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04ec15f900 con 0x7f04ec15da80 2024-04-25T08:53:44.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.064+0000 7f04e27fc640 1 -- 172.21.15.37:0/3721417106 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04e40052b0 con 0x7f04ec15da80 2024-04-25T08:53:44.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.068+0000 7f04f3ec4640 1 -- 172.21.15.37:0/3721417106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04ec15fec0 con 0x7f04ec15da80 2024-04-25T08:53:44.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.068+0000 7f04f3ec4640 1 -- 172.21.15.37:0/3721417106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04ec04efa0 con 0x7f04ec15da80 2024-04-25T08:53:44.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.068+0000 7f04e27fc640 1 -- 172.21.15.37:0/3721417106 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04e401a070 con 0x7f04ec15da80 2024-04-25T08:53:44.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.068+0000 7f04e27fc640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f04d0066f60 0x7f04d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.068+0000 7f04e27fc640 1 -- 172.21.15.37:0/3721417106 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f04e4084a70 con 0x7f04ec15da80 2024-04-25T08:53:44.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.068+0000 7f04f1c60640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f04d0066f60 0x7f04d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.068+0000 7f04f1c60640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f04d0066f60 0x7f04d0069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f04e80af000 tx=0x7f04e8004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:44.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.072+0000 7f04e27fc640 1 -- 172.21.15.37:0/3721417106 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f04e4055680 con 0x7f04ec15da80 2024-04-25T08:53:44.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.236+0000 7f04f3ec4640 1 -- 172.21.15.37:0/3721417106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04ec0b02c0 con 0x7f04ec15da80 2024-04-25T08:53:44.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.236+0000 7f04e27fc640 1 -- 172.21.15.37:0/3721417106 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+18807 (secure 0 0 0) 0x7f04e4054f80 con 0x7f04ec15da80 2024-04-25T08:53:44.231 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:44.231 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:12.202641+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:52:28.778700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:52:29.556234+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T08:52:33.308816+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.13","mappings":[{"from":7,"to":2}]},{"pgid":"3.19","mappings":[{"from":3,"to":2}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.0","mappings":[{"from":1,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 -- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f04d0066f60 msgr2=0x7f04d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f04d0066f60 0x7f04d0069420 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f04e80af000 tx=0x7f04e8004740 comp rx=0 tx=0).stop 2024-04-25T08:53:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 -- 172.21.15.37:0/3721417106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04ec15da80 msgr2=0x7f04ec15c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04ec15da80 0x7f04ec15c790 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f04e400c7b0 tx=0x7f04e400cc70 comp rx=0 tx=0).stop 2024-04-25T08:53:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 -- 172.21.15.37:0/3721417106 shutdown_connections 2024-04-25T08:53:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f04d0066f60 0x7f04d0069420 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f04ec15da80 0x7f04ec15c790 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04ec0d1cd0 0x7f04ec0b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 --2- 172.21.15.37:0/3721417106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04ec0d0620 0x7f04ec154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 -- 172.21.15.37:0/3721417106 >> 172.21.15.37:0/3721417106 conn(0x7f04ec08b000 msgr2=0x7f04ec0ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 -- 172.21.15.37:0/3721417106 shutdown_connections 2024-04-25T08:53:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.240+0000 7f04f3ec4640 1 -- 172.21.15.37:0/3721417106 wait complete. 2024-04-25T08:53:44.251 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:53:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6f2568640 1 Processor -- start 2024-04-25T08:53:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6f2568640 1 -- start start 2024-04-25T08:53:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6f2568640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6ec15aac0 0x7fb6ec0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6f2568640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb6ec0cf0e0 0x7fb6ec0cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6f2568640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb6ec15aea0 0x7fb6ec15b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6f2568640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ec0ce0d0 con 0x7fb6ec0cf0e0 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6f2568640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ec0ce320 con 0x7fb6ec15aea0 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6f2568640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec0ce600 con 0x7fb6ec15aac0 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6eb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb6ec0cf0e0 0x7fb6ec0cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6eb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb6ec0cf0e0 0x7fb6ec0cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51904/0 (socket says 172.21.15.37:51904) 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6eb7fe640 1 -- 172.21.15.37:0/1656734560 learned_addr learned my addr 172.21.15.37:0/1656734560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6ebfff640 1 --2- 172.21.15.37:0/1656734560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6ec15aac0 0x7fb6ec0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6f0b05640 1 --2- 172.21.15.37:0/1656734560 >> v2:172.21.15.203:3300/0 conn(0x7fb6ec15aea0 0x7fb6ec15b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6eb7fe640 1 -- 172.21.15.37:0/1656734560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6ec15aac0 msgr2=0x7fb6ec0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6eb7fe640 1 --2- 172.21.15.37:0/1656734560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6ec15aac0 0x7fb6ec0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6eb7fe640 1 -- 172.21.15.37:0/1656734560 >> v2:172.21.15.203:3300/0 conn(0x7fb6ec15aea0 msgr2=0x7fb6ec15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6eb7fe640 1 --2- 172.21.15.37:0/1656734560 >> v2:172.21.15.203:3300/0 conn(0x7fb6ec15aea0 0x7fb6ec15b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.376+0000 7fb6eb7fe640 1 -- 172.21.15.37:0/1656734560 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec0ce8e0 con 0x7fb6ec0cf0e0 2024-04-25T08:53:44.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6eb7fe640 1 --2- 172.21.15.37:0/1656734560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6ec0cf0e0 0x7fb6ec0cd180 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec15e1b0 tx=0x7fb6d401ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0e403efb024379 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:44.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6eaffd640 1 -- 172.21.15.37:0/1656734560 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d400b2d0 con 0x7fb6ec0cf0e0 2024-04-25T08:53:44.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6eaffd640 1 -- 172.21.15.37:0/1656734560 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d400b470 con 0x7fb6ec0cf0e0 2024-04-25T08:53:44.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6eaffd640 1 -- 172.21.15.37:0/1656734560 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d40a7bb0 con 0x7fb6ec0cf0e0 2024-04-25T08:53:44.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6f2568640 1 -- 172.21.15.37:0/1656734560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6ec0cf0e0 msgr2=0x7fb6ec0cd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6f2568640 1 --2- 172.21.15.37:0/1656734560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6ec0cf0e0 0x7fb6ec0cd180 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec15e1b0 tx=0x7fb6d401ebf0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6f2568640 1 -- 172.21.15.37:0/1656734560 shutdown_connections 2024-04-25T08:53:44.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6f2568640 1 --2- 172.21.15.37:0/1656734560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6ec0cf0e0 0x7fb6ec0cd180 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6f2568640 1 --2- 172.21.15.37:0/1656734560 >> v2:172.21.15.203:3300/0 conn(0x7fb6ec15aea0 0x7fb6ec15b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6f2568640 1 --2- 172.21.15.37:0/1656734560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6ec15aac0 0x7fb6ec0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6f2568640 1 -- 172.21.15.37:0/1656734560 >> 172.21.15.37:0/1656734560 conn(0x7fb6ec099a70 msgr2=0x7fb6ec0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:44.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6f2568640 1 -- 172.21.15.37:0/1656734560 shutdown_connections 2024-04-25T08:53:44.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.380+0000 7fb6f2568640 1 -- 172.21.15.37:0/1656734560 wait complete. 2024-04-25T08:53:44.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f2568640 1 Processor -- start 2024-04-25T08:53:44.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f2568640 1 -- start start 2024-04-25T08:53:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f2568640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6ec0cf0e0 0x7fb6ec152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f2568640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6ec15aac0 0x7fb6ec14fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f2568640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6ec15aea0 0x7fb6ec15be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f2568640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec15ee10 con 0x7fb6ec15aea0 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f2568640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec15ec90 con 0x7fb6ec15aac0 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f2568640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ec15ef90 con 0x7fb6ec0cf0e0 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f0b05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6ec15aea0 0x7fb6ec15be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f0b05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6ec15aea0 0x7fb6ec15be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51916/0 (socket says 172.21.15.37:51916) 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f0b05640 1 -- 172.21.15.37:0/3101811184 learned_addr learned my addr 172.21.15.37:0/3101811184 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6ebfff640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6ec0cf0e0 0x7fb6ec152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6eb7fe640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6ec15aac0 0x7fb6ec14fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f0b05640 1 -- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6ec0cf0e0 msgr2=0x7fb6ec152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f0b05640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6ec0cf0e0 0x7fb6ec152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f0b05640 1 -- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6ec15aac0 msgr2=0x7fb6ec14fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f0b05640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6ec15aac0 0x7fb6ec14fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f0b05640 1 -- 172.21.15.37:0/3101811184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ec15c960 con 0x7fb6ec15aea0 2024-04-25T08:53:44.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f0b05640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6ec15aea0 0x7fb6ec15be60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00d800 tx=0x7fb6dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:44.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6e8ff9640 1 -- 172.21.15.37:0/3101811184 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00b6c0 con 0x7fb6ec15aea0 2024-04-25T08:53:44.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6e8ff9640 1 -- 172.21.15.37:0/3101811184 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc019070 con 0x7fb6ec15aea0 2024-04-25T08:53:44.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f2568640 1 -- 172.21.15.37:0/3101811184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ec1515a0 con 0x7fb6ec15aea0 2024-04-25T08:53:44.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6e8ff9640 1 -- 172.21.15.37:0/3101811184 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc00baa0 con 0x7fb6ec15aea0 2024-04-25T08:53:44.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f2568640 1 -- 172.21.15.37:0/3101811184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ec0ca2f0 con 0x7fb6ec15aea0 2024-04-25T08:53:44.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.384+0000 7fb6f2568640 1 -- 172.21.15.37:0/3101811184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ec04efa0 con 0x7fb6ec15aea0 2024-04-25T08:53:44.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.388+0000 7fb6e8ff9640 1 -- 172.21.15.37:0/3101811184 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6dc00bc40 con 0x7fb6ec15aea0 2024-04-25T08:53:44.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.388+0000 7fb6e8ff9640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6cc066f60 0x7fb6cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.388+0000 7fb6e8ff9640 1 -- 172.21.15.37:0/3101811184 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb6dc085770 con 0x7fb6ec15aea0 2024-04-25T08:53:44.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.388+0000 7fb6ebfff640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6cc066f60 0x7fb6cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.388+0000 7fb6ebfff640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6cc066f60 0x7fb6cc069420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400b9d0 tx=0x7fb6e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:44.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.392+0000 7fb6e8ff9640 1 -- 172.21.15.37:0/3101811184 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6dc056410 con 0x7fb6ec15aea0 2024-04-25T08:53:44.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.580+0000 7fb6f2568640 1 -- 172.21.15.37:0/3101811184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb6ec0b6480 con 0x7fb6ec15aea0 2024-04-25T08:53:44.574 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T08:53:44.580+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:53:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.584+0000 7fb6e8ff9640 1 -- 172.21.15.37:0/3101811184 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb6dc055d10 con 0x7fb6ec15aea0 2024-04-25T08:53:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:53:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 -- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6cc066f60 msgr2=0x7fb6cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6cc066f60 0x7fb6cc069420 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400b9d0 tx=0x7fb6e4008040 comp rx=0 tx=0).stop 2024-04-25T08:53:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 -- 172.21.15.37:0/3101811184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6ec15aea0 msgr2=0x7fb6ec15be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6ec15aea0 0x7fb6ec15be60 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc00d800 tx=0x7fb6dc00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 -- 172.21.15.37:0/3101811184 shutdown_connections 2024-04-25T08:53:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6cc066f60 0x7fb6cc069420 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb6ec15aea0 0x7fb6ec15be60 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb6ec15aac0 0x7fb6ec14fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 --2- 172.21.15.37:0/3101811184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb6ec0cf0e0 0x7fb6ec152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 -- 172.21.15.37:0/3101811184 >> 172.21.15.37:0/3101811184 conn(0x7fb6ec099a70 msgr2=0x7fb6ec152e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 -- 172.21.15.37:0/3101811184 shutdown_connections 2024-04-25T08:53:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.588+0000 7fb6f2568640 1 -- 172.21.15.37:0/3101811184 wait complete. 2024-04-25T08:53:44.586 INFO:tasks.ceph.mds.c.smithi037.stderr: -4845> 2024-04-25T08:52:34.260+0000 7f841fbe2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:53:44.598 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:53:44.625 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T08:53:44.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff40b812640 1 Processor -- start 2024-04-25T08:53:44.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff40b812640 1 -- start start 2024-04-25T08:53:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff40b812640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff40415aac0 0x7ff4040ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff40b812640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4040cf0e0 0x7ff4040cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff40b812640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4040ce320 con 0x7ff40415aea0 2024-04-25T08:53:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff40b812640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4040ce0d0 con 0x7ff40415aac0 2024-04-25T08:53:44.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff40b812640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040ce600 con 0x7ff4040cf0e0 2024-04-25T08:53:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff4095ae640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff40415aac0 0x7ff4040ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff409daf640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff40415aea0 0x7ff40415b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33670/0 (socket says 172.21.15.37:33670) 2024-04-25T08:53:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff409daf640 1 -- 172.21.15.37:0/2972644604 learned_addr learned my addr 172.21.15.37:0/2972644604 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff408dad640 1 --2- 172.21.15.37:0/2972644604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4040cf0e0 0x7ff4040cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff4095ae640 1 -- 172.21.15.37:0/2972644604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4040cf0e0 msgr2=0x7ff4040cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff4095ae640 1 --2- 172.21.15.37:0/2972644604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4040cf0e0 0x7ff4040cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff4095ae640 1 -- 172.21.15.37:0/2972644604 >> v1:172.21.15.37:6789/0 conn(0x7ff40415aea0 legacy=0x7ff40415b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff4095ae640 1 -- 172.21.15.37:0/2972644604 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4040ce8e0 con 0x7ff40415aac0 2024-04-25T08:53:44.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.720+0000 7ff3fbfff640 1 -- 172.21.15.37:0/2972644604 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3141579464 0 0) 0x7ff4040ce320 con 0x7ff40415aea0 2024-04-25T08:53:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff4095ae640 1 --2- 172.21.15.37:0/2972644604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40415aac0 0x7ff4040ccc40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff4000097c0 tx=0x7ff40001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a05297289ed4fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff3fbfff640 1 -- 172.21.15.37:0/2972644604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4000b1070 con 0x7ff40415aac0 2024-04-25T08:53:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff3fbfff640 1 -- 172.21.15.37:0/2972644604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4000ac410 con 0x7ff40415aac0 2024-04-25T08:53:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff3fbfff640 1 -- 172.21.15.37:0/2972644604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4000ac7e0 con 0x7ff40415aac0 2024-04-25T08:53:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 -- 172.21.15.37:0/2972644604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40415aac0 msgr2=0x7ff4040ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 --2- 172.21.15.37:0/2972644604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40415aac0 0x7ff4040ccc40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff4000097c0 tx=0x7ff40001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 -- 172.21.15.37:0/2972644604 shutdown_connections 2024-04-25T08:53:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 --2- 172.21.15.37:0/2972644604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4040cf0e0 0x7ff4040cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 --2- 172.21.15.37:0/2972644604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40415aac0 0x7ff4040ccc40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 -- 172.21.15.37:0/2972644604 >> 172.21.15.37:0/2972644604 conn(0x7ff404099a70 msgr2=0x7ff4040d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 -- 172.21.15.37:0/2972644604 shutdown_connections 2024-04-25T08:53:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 -- 172.21.15.37:0/2972644604 wait complete. 2024-04-25T08:53:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 Processor -- start 2024-04-25T08:53:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 -- start start 2024-04-25T08:53:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4040cf0e0 0x7ff404151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff40415aac0 0x7ff4040c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40415aea0 0x7ff404152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040cbbb0 con 0x7ff40415aea0 2024-04-25T08:53:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040cba30 con 0x7ff40415aac0 2024-04-25T08:53:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040cbd30 con 0x7ff4040cf0e0 2024-04-25T08:53:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff409daf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40415aea0 0x7ff404152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff409daf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40415aea0 0x7ff404152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51948/0 (socket says 172.21.15.37:51948) 2024-04-25T08:53:44.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff409daf640 1 -- 172.21.15.37:0/2573270501 learned_addr learned my addr 172.21.15.37:0/2573270501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff409daf640 1 -- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4040cf0e0 msgr2=0x7ff404151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff409daf640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4040cf0e0 0x7ff404151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff409daf640 1 -- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff40415aac0 msgr2=0x7ff4040c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff409daf640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff40415aac0 0x7ff4040c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff409daf640 1 -- 172.21.15.37:0/2573270501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4041521f0 con 0x7ff40415aea0 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff409daf640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40415aea0 0x7ff404152a10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00c7d0 tx=0x7ff3fc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff3f9ffb640 1 -- 172.21.15.37:0/2573270501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc018070 con 0x7ff40415aea0 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff3f9ffb640 1 -- 172.21.15.37:0/2573270501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3fc00a640 con 0x7ff40415aea0 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 -- 172.21.15.37:0/2573270501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4040c4fa0 con 0x7ff40415aea0 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff3f9ffb640 1 -- 172.21.15.37:0/2573270501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc00aa70 con 0x7ff40415aea0 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.724+0000 7ff40b812640 1 -- 172.21.15.37:0/2573270501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4040c5560 con 0x7ff40415aea0 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.728+0000 7ff40b812640 1 -- 172.21.15.37:0/2573270501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40404efa0 con 0x7ff40415aea0 2024-04-25T08:53:44.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.728+0000 7ff3f9ffb640 1 -- 172.21.15.37:0/2573270501 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3fc01b070 con 0x7ff40415aea0 2024-04-25T08:53:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.728+0000 7ff3f9ffb640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3e8066f60 0x7ff3e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:44.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.728+0000 7ff3f9ffb640 1 -- 172.21.15.37:0/2573270501 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6821+0+0 (secure 0 0 0) 0x7ff3fc084a50 con 0x7ff40415aea0 2024-04-25T08:53:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.728+0000 7ff4095ae640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3e8066f60 0x7ff3e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:44.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.728+0000 7ff4095ae640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3e8066f60 0x7ff3e8069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4000ae000 tx=0x7ff400008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:44.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.732+0000 7ff3f9ffb640 1 -- 172.21.15.37:0/2573270501 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3fc055600 con 0x7ff40415aea0 2024-04-25T08:53:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:44.920+0000 7ff40b812640 1 -- 172.21.15.37:0/2573270501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff4040cdc90 con 0x7ff40415aea0 2024-04-25T08:53:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff3f9ffb640 1 -- 172.21.15.37:0/2573270501 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff3fc054f00 con 0x7ff40415aea0 2024-04-25T08:53:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 -- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3e8066f60 msgr2=0x7ff3e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3e8066f60 0x7ff3e8069420 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4000ae000 tx=0x7ff400008040 comp rx=0 tx=0).stop 2024-04-25T08:53:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 -- 172.21.15.37:0/2573270501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40415aea0 msgr2=0x7ff404152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40415aea0 0x7ff404152a10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc00c7d0 tx=0x7ff3fc00cc90 comp rx=0 tx=0).stop 2024-04-25T08:53:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 -- 172.21.15.37:0/2573270501 shutdown_connections 2024-04-25T08:53:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3e8066f60 0x7ff3e8069420 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff40415aea0 0x7ff404152a10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff40415aac0 0x7ff4040c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 --2- 172.21.15.37:0/2573270501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4040cf0e0 0x7ff404151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 -- 172.21.15.37:0/2573270501 >> 172.21.15.37:0/2573270501 conn(0x7ff404099a70 msgr2=0x7ff4040aeac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:45.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 -- 172.21.15.37:0/2573270501 shutdown_connections 2024-04-25T08:53:45.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.588+0000 7ff40b812640 1 -- 172.21.15.37:0/2573270501 wait complete. 2024-04-25T08:53:45.604 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:53:45.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.788+0000 7f4022181640 1 Processor -- start 2024-04-25T08:53:45.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.788+0000 7f4022181640 1 -- start start 2024-04-25T08:53:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f4022181640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f401c0cfee0 0x7f401c0b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f4022181640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401c0cc7f0 con 0x7f401c15d980 2024-04-25T08:53:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f4022181640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401c0b1ba0 con 0x7f401c15dd60 2024-04-25T08:53:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f4022181640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c0cb1f0 con 0x7f401c0cfee0 2024-04-25T08:53:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f401c15d980 0x7f401c0caae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33684/0 (socket says 172.21.15.37:33684) 2024-04-25T08:53:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401b7fe640 1 -- 172.21.15.37:0/1418430498 learned_addr learned my addr 172.21.15.37:0/1418430498 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401bfff640 1 --2- 172.21.15.37:0/1418430498 >> v2:172.21.15.37:3300/0 conn(0x7f401c0cfee0 0x7f401c0b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401a7fc640 1 -- 172.21.15.37:0/1418430498 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423414435 0 0) 0x7f401c0cc7f0 con 0x7f401c15d980 2024-04-25T08:53:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401a7fc640 1 -- 172.21.15.37:0/1418430498 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4004003530 con 0x7f401c15d980 2024-04-25T08:53:45.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401bfff640 1 -- 172.21.15.37:0/1418430498 >> v1:172.21.15.203:6789/0 conn(0x7f401c15dd60 legacy=0x7f401c0cf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:45.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401bfff640 1 -- 172.21.15.37:0/1418430498 >> v1:172.21.15.37:6789/0 conn(0x7f401c15d980 legacy=0x7f401c0caae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:45.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401bfff640 1 -- 172.21.15.37:0/1418430498 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0ccab0 con 0x7f401c0cfee0 2024-04-25T08:53:45.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401bfff640 1 --2- 172.21.15.37:0/1418430498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cfee0 0x7f401c0b1510 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4014009d90 tx=0x7f401401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6247ba881d31305e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:45.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401a7fc640 1 -- 172.21.15.37:0/1418430498 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140b2070 con 0x7f401c0cfee0 2024-04-25T08:53:45.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401a7fc640 1 -- 172.21.15.37:0/1418430498 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40140a5900 con 0x7f401c0cfee0 2024-04-25T08:53:45.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f401a7fc640 1 -- 172.21.15.37:0/1418430498 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140a5cd0 con 0x7f401c0cfee0 2024-04-25T08:53:45.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f4022181640 1 -- 172.21.15.37:0/1418430498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cfee0 msgr2=0x7f401c0b1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:45.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f4022181640 1 --2- 172.21.15.37:0/1418430498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cfee0 0x7f401c0b1510 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f4014009d90 tx=0x7f401401c930 comp rx=0 tx=0).stop 2024-04-25T08:53:45.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f4022181640 1 -- 172.21.15.37:0/1418430498 shutdown_connections 2024-04-25T08:53:45.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f4022181640 1 --2- 172.21.15.37:0/1418430498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cfee0 0x7f401c0b1510 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:45.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f4022181640 1 -- 172.21.15.37:0/1418430498 >> 172.21.15.37:0/1418430498 conn(0x7f401c0b1fa0 msgr2=0x7f401c0b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:45.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f4022181640 1 -- 172.21.15.37:0/1418430498 shutdown_connections 2024-04-25T08:53:45.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.792+0000 7f4022181640 1 -- 172.21.15.37:0/1418430498 wait complete. 2024-04-25T08:53:45.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4022181640 1 Processor -- start 2024-04-25T08:53:45.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4022181640 1 -- start start 2024-04-25T08:53:45.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4022181640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cfee0 0x7f401c151090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:45.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4022181640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f401c15d980 0x7f401c0c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:45.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4022181640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f401c15dd60 0x7f401c0c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:45.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4022181640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c16b9b0 con 0x7f401c0cfee0 2024-04-25T08:53:45.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4022181640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0cdd10 con 0x7f401c15dd60 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4022181640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c16bb30 con 0x7f401c15d980 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f401b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cfee0 0x7f401c151090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f401b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cfee0 0x7f401c151090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51972/0 (socket says 172.21.15.37:51972) 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f401b7fe640 1 -- 172.21.15.37:0/1400381677 learned_addr learned my addr 172.21.15.37:0/1400381677 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f401affd640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f401c15d980 0x7f401c0c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f401b7fe640 1 -- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f401c15d980 msgr2=0x7f401c0c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f401b7fe640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f401c15d980 0x7f401c0c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f401b7fe640 1 -- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f401c15dd60 msgr2=0x7f401c0c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f401b7fe640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f401c15dd60 0x7f401c0c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f401b7fe640 1 -- 172.21.15.37:0/1400381677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0af4a0 con 0x7f401c0cfee0 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f401b7fe640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cfee0 0x7f401c151090 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f400c00c480 tx=0x7f400c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4003fff640 1 -- 172.21.15.37:0/1400381677 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c017070 con 0x7f401c0cfee0 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4003fff640 1 -- 172.21.15.37:0/1400381677 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c005cf0 con 0x7f401c0cfee0 2024-04-25T08:53:45.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4022181640 1 -- 172.21.15.37:0/1400381677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c15c5f0 con 0x7f401c0cfee0 2024-04-25T08:53:45.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4003fff640 1 -- 172.21.15.37:0/1400381677 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c007290 con 0x7f401c0cfee0 2024-04-25T08:53:45.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4022181640 1 -- 172.21.15.37:0/1400381677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c15eb80 con 0x7f401c0cfee0 2024-04-25T08:53:45.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.796+0000 7f4022181640 1 -- 172.21.15.37:0/1400381677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c068340 con 0x7f401c0cfee0 2024-04-25T08:53:45.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.800+0000 7f4003fff640 1 -- 172.21.15.37:0/1400381677 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f400c01a070 con 0x7f401c0cfee0 2024-04-25T08:53:45.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.800+0000 7f4003fff640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 0x7f4008069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:45.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.800+0000 7f4003fff640 1 -- 172.21.15.37:0/1400381677 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f400c084900 con 0x7f401c0cfee0 2024-04-25T08:53:45.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.800+0000 7f401affd640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 0x7f4008069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:45.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.800+0000 7f401affd640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 0x7f4008069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f401c1f9ff0 tx=0x7f4010008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:45.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.804+0000 7f4003fff640 1 -- 172.21.15.37:0/1400381677 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f400c0554a0 con 0x7f401c0cfee0 2024-04-25T08:53:45.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:45.964+0000 7f4022181640 1 -- 172.21.15.37:0/1400381677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f401c0c70d0 con 0x7f401c0cfee0 2024-04-25T08:53:46.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.596+0000 7f4003fff640 1 -- 172.21.15.37:0/1400381677 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) v1 ==== 181+0+0 (secure 0 0 0) 0x7f400c054da0 con 0x7f401c0cfee0 2024-04-25T08:53:46.590 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:53:46.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 -- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 msgr2=0x7f4008069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:46.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 0x7f4008069420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f401c1f9ff0 tx=0x7f4010008040 comp rx=0 tx=0).stop 2024-04-25T08:53:46.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 -- 172.21.15.37:0/1400381677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cfee0 msgr2=0x7f401c151090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:46.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cfee0 0x7f401c151090 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f400c00c480 tx=0x7f400c00c940 comp rx=0 tx=0).stop 2024-04-25T08:53:46.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 -- 172.21.15.37:0/1400381677 shutdown_connections 2024-04-25T08:53:46.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 0x7f4008069420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:46.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f401c15dd60 0x7f401c0c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:46.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f401c15d980 0x7f401c0c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:46.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 --2- 172.21.15.37:0/1400381677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0cfee0 0x7f401c151090 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:46.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 -- 172.21.15.37:0/1400381677 >> 172.21.15.37:0/1400381677 conn(0x7f401c0b1fa0 msgr2=0x7f401c0ad280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:46.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 -- 172.21.15.37:0/1400381677 shutdown_connections 2024-04-25T08:53:46.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.600+0000 7f4022181640 1 -- 172.21.15.37:0/1400381677 wait complete. 2024-04-25T08:53:46.616 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:53:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff994d9f640 1 Processor -- start 2024-04-25T08:53:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff994d9f640 1 -- start start 2024-04-25T08:53:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff994d9f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff99015b7e0 0x7ff9900cbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff994d9f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff99006ac90 0x7ff9900cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff994d9f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9900cc500 con 0x7ff9900cf5c0 2024-04-25T08:53:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff994d9f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff99015bbc0 con 0x7ff99015b7e0 2024-04-25T08:53:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff994d9f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9900cc7e0 con 0x7ff99006ac90 2024-04-25T08:53:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff98e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff99015b7e0 0x7ff9900cbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff98e59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff99015b7e0 0x7ff9900cbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51974/0 (socket says 172.21.15.37:51974) 2024-04-25T08:53:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff98e59c640 1 -- 172.21.15.37:0/1589489513 learned_addr learned my addr 172.21.15.37:0/1589489513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff98dd9b640 1 --2- 172.21.15.37:0/1589489513 >> v2:172.21.15.203:3300/0 conn(0x7ff99006ac90 0x7ff9900cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff98e59c640 1 -- 172.21.15.37:0/1589489513 >> v2:172.21.15.203:3300/0 conn(0x7ff99006ac90 msgr2=0x7ff9900cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff98e59c640 1 --2- 172.21.15.37:0/1589489513 >> v2:172.21.15.203:3300/0 conn(0x7ff99006ac90 0x7ff9900cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff98e59c640 1 -- 172.21.15.37:0/1589489513 >> v1:172.21.15.203:6789/0 conn(0x7ff9900cf5c0 legacy=0x7ff9900ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.752+0000 7ff98e59c640 1 -- 172.21.15.37:0/1589489513 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9900ccac0 con 0x7ff99015b7e0 2024-04-25T08:53:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff98e59c640 1 --2- 172.21.15.37:0/1589489513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99015b7e0 0x7ff9900cbe20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff990067950 tx=0x7ff98801c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fba92d545136d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff98d59a640 1 -- 172.21.15.37:0/1589489513 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880a57b0 con 0x7ff99015b7e0 2024-04-25T08:53:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff98d59a640 1 -- 172.21.15.37:0/1589489513 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9880a5950 con 0x7ff99015b7e0 2024-04-25T08:53:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff98d59a640 1 -- 172.21.15.37:0/1589489513 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9880a5d20 con 0x7ff99015b7e0 2024-04-25T08:53:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff994d9f640 1 -- 172.21.15.37:0/1589489513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99015b7e0 msgr2=0x7ff9900cbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff994d9f640 1 --2- 172.21.15.37:0/1589489513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99015b7e0 0x7ff9900cbe20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ff990067950 tx=0x7ff98801c8f0 comp rx=0 tx=0).stop 2024-04-25T08:53:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff994d9f640 1 -- 172.21.15.37:0/1589489513 shutdown_connections 2024-04-25T08:53:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff994d9f640 1 --2- 172.21.15.37:0/1589489513 >> v2:172.21.15.203:3300/0 conn(0x7ff99006ac90 0x7ff9900cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff994d9f640 1 --2- 172.21.15.37:0/1589489513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99015b7e0 0x7ff9900cbe20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff994d9f640 1 -- 172.21.15.37:0/1589489513 >> 172.21.15.37:0/1589489513 conn(0x7ff9900b1f80 msgr2=0x7ff9900b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff994d9f640 1 -- 172.21.15.37:0/1589489513 shutdown_connections 2024-04-25T08:53:46.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff994d9f640 1 -- 172.21.15.37:0/1589489513 wait complete. 2024-04-25T08:53:46.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff994d9f640 1 Processor -- start 2024-04-25T08:53:46.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.756+0000 7ff994d9f640 1 -- start start 2024-04-25T08:53:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff994d9f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99006ac90 0x7ff9900c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff994d9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9900cf5c0 0x7ff99013d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff994d9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99015b7e0 0x7ff9900c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff994d9f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900cdae0 con 0x7ff99015b7e0 2024-04-25T08:53:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff994d9f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900cfe00 con 0x7ff99006ac90 2024-04-25T08:53:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff994d9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9900cdc60 con 0x7ff9900cf5c0 2024-04-25T08:53:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99015b7e0 0x7ff9900c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99015b7e0 0x7ff9900c74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51982/0 (socket says 172.21.15.37:51982) 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98ed9d640 1 -- 172.21.15.37:0/2329549055 learned_addr learned my addr 172.21.15.37:0/2329549055 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98e59c640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99006ac90 0x7ff9900c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98dd9b640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9900cf5c0 0x7ff99013d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98ed9d640 1 -- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9900cf5c0 msgr2=0x7ff99013d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98ed9d640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9900cf5c0 0x7ff99013d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98ed9d640 1 -- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99006ac90 msgr2=0x7ff9900c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98ed9d640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99006ac90 0x7ff9900c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98ed9d640 1 -- 172.21.15.37:0/2329549055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99014c4b0 con 0x7ff99015b7e0 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98e59c640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99006ac90 0x7ff9900c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff98ed9d640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99015b7e0 0x7ff9900c74c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff98000c3f0 tx=0x7ff98000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff986ffd640 1 -- 172.21.15.37:0/2329549055 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980017070 con 0x7ff99015b7e0 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff986ffd640 1 -- 172.21.15.37:0/2329549055 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9800052d0 con 0x7ff99015b7e0 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff994d9f640 1 -- 172.21.15.37:0/2329549055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff99014c790 con 0x7ff99015b7e0 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff986ffd640 1 -- 172.21.15.37:0/2329549055 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff980003840 con 0x7ff99015b7e0 2024-04-25T08:53:46.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff994d9f640 1 -- 172.21.15.37:0/2329549055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9900c9520 con 0x7ff99015b7e0 2024-04-25T08:53:46.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff994d9f640 1 -- 172.21.15.37:0/2329549055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff990068340 con 0x7ff99015b7e0 2024-04-25T08:53:46.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff986ffd640 1 -- 172.21.15.37:0/2329549055 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff98001a070 con 0x7ff99015b7e0 2024-04-25T08:53:46.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff986ffd640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff970066f60 0x7ff970069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:46.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.760+0000 7ff986ffd640 1 -- 172.21.15.37:0/2329549055 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 6383+0+0 (secure 0 0 0) 0x7ff980084740 con 0x7ff99015b7e0 2024-04-25T08:53:46.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.764+0000 7ff98e59c640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff970066f60 0x7ff970069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:46.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.764+0000 7ff98e59c640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff970066f60 0x7ff970069420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9880ae000 tx=0x7ff9880a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:46.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.768+0000 7ff986ffd640 1 -- 172.21.15.37:0/2329549055 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff980005480 con 0x7ff99015b7e0 2024-04-25T08:53:46.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:46.928+0000 7ff994d9f640 1 -- 172.21.15.37:0/2329549055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff9900ad730 con 0x7ff99015b7e0 2024-04-25T08:53:47.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.604+0000 7ff986ffd640 1 -- 172.21.15.37:0/2329549055 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff980055410 con 0x7ff99015b7e0 2024-04-25T08:53:47.598 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T08:53:47.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 -- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff970066f60 msgr2=0x7ff970069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff970066f60 0x7ff970069420 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff9880ae000 tx=0x7ff9880a7070 comp rx=0 tx=0).stop 2024-04-25T08:53:47.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 -- 172.21.15.37:0/2329549055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99015b7e0 msgr2=0x7ff9900c74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99015b7e0 0x7ff9900c74c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff98000c3f0 tx=0x7ff98000c8b0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 -- 172.21.15.37:0/2329549055 shutdown_connections 2024-04-25T08:53:47.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff970066f60 0x7ff970069420 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99015b7e0 0x7ff9900c74c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff9900cf5c0 0x7ff99013d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 --2- 172.21.15.37:0/2329549055 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99006ac90 0x7ff9900c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 -- 172.21.15.37:0/2329549055 >> 172.21.15.37:0/2329549055 conn(0x7ff9900b1f80 msgr2=0x7ff990152950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:47.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 -- 172.21.15.37:0/2329549055 shutdown_connections 2024-04-25T08:53:47.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.608+0000 7ff994d9f640 1 -- 172.21.15.37:0/2329549055 wait complete. 2024-04-25T08:53:47.625 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T08:53:47.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.756+0000 7f9578d63640 1 Processor -- start 2024-04-25T08:53:47.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.756+0000 7f9578d63640 1 -- start start 2024-04-25T08:53:47.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f9578d63640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f957409a810 0x7f95740ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f9578d63640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95740d0d00 con 0x7f957415dd20 2024-04-25T08:53:47.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f9578d63640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95740d0740 con 0x7f9574099a70 2024-04-25T08:53:47.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f9578d63640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95740cc030 con 0x7f957409a810 2024-04-25T08:53:47.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f9572d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f957409a810 0x7f95740ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:47.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f9571d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f957415dd20 0x7f95740cc4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33696/0 (socket says 172.21.15.37:33696) 2024-04-25T08:53:47.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f9571d9b640 1 -- 172.21.15.37:0/275682523 learned_addr learned my addr 172.21.15.37:0/275682523 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:47.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f9572d9d640 1 -- 172.21.15.37:0/275682523 >> v1:172.21.15.203:6789/0 conn(0x7f9574099a70 legacy=0x7f9574099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:47.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f9572d9d640 1 -- 172.21.15.37:0/275682523 >> v1:172.21.15.37:6789/0 conn(0x7f957415dd20 legacy=0x7f95740cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f957159a640 1 -- 172.21.15.37:0/275682523 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250778264 0 0) 0x7f9568001120 con 0x7f957415dd20 2024-04-25T08:53:47.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f9572d9d640 1 -- 172.21.15.37:0/275682523 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95740d0a20 con 0x7f957409a810 2024-04-25T08:53:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f9572d9d640 1 --2- 172.21.15.37:0/275682523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957409a810 0x7f95740ccbd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f956c009d90 tx=0x7f956c01c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72ad31b5c6be81e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f957159a640 1 -- 172.21.15.37:0/275682523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0b2070 con 0x7f957409a810 2024-04-25T08:53:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f957159a640 1 -- 172.21.15.37:0/275682523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c0a5990 con 0x7f957409a810 2024-04-25T08:53:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.760+0000 7f957159a640 1 -- 172.21.15.37:0/275682523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0a5d60 con 0x7f957409a810 2024-04-25T08:53:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 -- 172.21.15.37:0/275682523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957409a810 msgr2=0x7f95740ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 --2- 172.21.15.37:0/275682523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957409a810 0x7f95740ccbd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f956c009d90 tx=0x7f956c01c8b0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 -- 172.21.15.37:0/275682523 shutdown_connections 2024-04-25T08:53:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 --2- 172.21.15.37:0/275682523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957409a810 0x7f95740ccbd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 -- 172.21.15.37:0/275682523 >> 172.21.15.37:0/275682523 conn(0x7f95740b3030 msgr2=0x7f95740b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 -- 172.21.15.37:0/275682523 shutdown_connections 2024-04-25T08:53:47.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 -- 172.21.15.37:0/275682523 wait complete. 2024-04-25T08:53:47.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 Processor -- start 2024-04-25T08:53:47.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 -- start start 2024-04-25T08:53:47.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9574099a70 0x7f95740c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957409a810 0x7f9574152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f957415dd20 0x7f95740ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95740d04a0 con 0x7f957409a810 2024-04-25T08:53:47.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95740d0320 con 0x7f9574099a70 2024-04-25T08:53:47.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957416b660 con 0x7f957415dd20 2024-04-25T08:53:47.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9571d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957409a810 0x7f9574152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:47.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9571d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957409a810 0x7f9574152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52000/0 (socket says 172.21.15.37:52000) 2024-04-25T08:53:47.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9571d9b640 1 -- 172.21.15.37:0/2273973529 learned_addr learned my addr 172.21.15.37:0/2273973529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:47.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f957259c640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9574099a70 0x7f95740c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:47.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9571d9b640 1 -- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f957415dd20 msgr2=0x7f95740ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9571d9b640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f957415dd20 0x7f95740ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9571d9b640 1 -- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9574099a70 msgr2=0x7f95740c5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9571d9b640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9574099a70 0x7f95740c5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9571d9b640 1 -- 172.21.15.37:0/2273973529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95740c5650 con 0x7f957409a810 2024-04-25T08:53:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f957259c640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9574099a70 0x7f95740c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9571d9b640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957409a810 0x7f9574152990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f956800c7b0 tx=0x7f956800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9562ffd640 1 -- 172.21.15.37:0/2273973529 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9568018070 con 0x7f957409a810 2024-04-25T08:53:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9562ffd640 1 -- 172.21.15.37:0/2273973529 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95680072d0 con 0x7f957409a810 2024-04-25T08:53:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 -- 172.21.15.37:0/2273973529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95740c5930 con 0x7f957409a810 2024-04-25T08:53:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9562ffd640 1 -- 172.21.15.37:0/2273973529 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95680047f0 con 0x7f957409a810 2024-04-25T08:53:47.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.764+0000 7f9578d63640 1 -- 172.21.15.37:0/2273973529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95740c5ef0 con 0x7f957409a810 2024-04-25T08:53:47.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.768+0000 7f9578d63640 1 -- 172.21.15.37:0/2273973529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9574068340 con 0x7f957409a810 2024-04-25T08:53:47.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.768+0000 7f9562ffd640 1 -- 172.21.15.37:0/2273973529 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f956801b070 con 0x7f957409a810 2024-04-25T08:53:47.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.768+0000 7f9562ffd640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9554066f90 0x7f9554069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.768+0000 7f9562ffd640 1 -- 172.21.15.37:0/2273973529 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f95680843b0 con 0x7f957409a810 2024-04-25T08:53:47.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.768+0000 7f957259c640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9554066f90 0x7f9554069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:47.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.768+0000 7f957259c640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9554066f90 0x7f9554069450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9564000f70 tx=0x7f9564009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:47.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.772+0000 7f9562ffd640 1 -- 172.21.15.37:0/2273973529 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f95740d0d00 con 0x7f957409a810 2024-04-25T08:53:47.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:47.936+0000 7f9578d63640 1 -- 172.21.15.37:0/2273973529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f95740b3620 con 0x7f957409a810 2024-04-25T08:53:48.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.612+0000 7f9562ffd640 1 -- 172.21.15.37:0/2273973529 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) v1 ==== 178+0+0 (secure 0 0 0) 0x7f9568055330 con 0x7f957409a810 2024-04-25T08:53:48.607 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data_ec' removed 2024-04-25T08:53:48.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.616+0000 7f9578d63640 1 -- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9554066f90 msgr2=0x7f9554069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.616+0000 7f9578d63640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9554066f90 0x7f9554069450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9564000f70 tx=0x7f9564009040 comp rx=0 tx=0).stop 2024-04-25T08:53:48.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.616+0000 7f9578d63640 1 -- 172.21.15.37:0/2273973529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957409a810 msgr2=0x7f9574152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.616+0000 7f9578d63640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957409a810 0x7f9574152990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f956800c7b0 tx=0x7f956800cc70 comp rx=0 tx=0).stop 2024-04-25T08:53:48.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.620+0000 7f9578d63640 1 -- 172.21.15.37:0/2273973529 shutdown_connections 2024-04-25T08:53:48.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.620+0000 7f9578d63640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9554066f90 0x7f9554069450 secure :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9564000f70 tx=0x7f9564009040 comp rx=0 tx=0).stop 2024-04-25T08:53:48.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.620+0000 7f9578d63640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f957415dd20 0x7f95740ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.620+0000 7f9578d63640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957409a810 0x7f9574152990 secure :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f956800c7b0 tx=0x7f956800cc70 comp rx=0 tx=0).stop 2024-04-25T08:53:48.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.620+0000 7f9578d63640 1 --2- 172.21.15.37:0/2273973529 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9574099a70 0x7f95740c5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.620+0000 7f9578d63640 1 -- 172.21.15.37:0/2273973529 >> 172.21.15.37:0/2273973529 conn(0x7f95740b3030 msgr2=0x7f9574153310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:48.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.620+0000 7f9578d63640 1 -- 172.21.15.37:0/2273973529 shutdown_connections 2024-04-25T08:53:48.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.620+0000 7f9578d63640 1 -- 172.21.15.37:0/2273973529 wait complete. 2024-04-25T08:53:48.636 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:53:48.637 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:53:48.637 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:53:48.637 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:48.637 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:53:48.637 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:53:48.637 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:53:48.637 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:48.638 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:53:48.638 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:53:48.638 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:53:48.638 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:48.638 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:53:48.638 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:53:48.638 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:53:48.639 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:53:48.652 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:53:48.652 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:53:48.653 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:53:48.653 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:53:48.654 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:53:48.654 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:53:48.656 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:53:48.657 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:53:48.697 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:53:48.697 DEBUG:teuthology.parallel:result is None 2024-04-25T08:53:48.698 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:53:48.698 DEBUG:teuthology.parallel:result is None 2024-04-25T08:53:48.700 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:53:48.701 DEBUG:teuthology.parallel:result is None 2024-04-25T08:53:48.702 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:53:48.703 DEBUG:teuthology.parallel:result is None 2024-04-25T08:53:48.703 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:53:48.767 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T08:53:48.767 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T08:53:48.772 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-25T08:53:48.772 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T08:53:48.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3ce5a0640 1 Processor -- start 2024-04-25T08:53:48.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3ce5a0640 1 -- start start 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3ce5a0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff3c80d07f0 0x7ff3c80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3ce5a0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff3c80d1cd0 0x7ff3c806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3ce5a0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3c806be60 con 0x7ff3c815da80 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3ce5a0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c806c440 con 0x7ff3c80d07f0 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3ce5a0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3c815de60 con 0x7ff3c80d1cd0 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3c7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff3c815da80 0x7ff3c80d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:33722/0 (socket says 172.21.15.37:33722) 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3c7fff640 1 -- 172.21.15.37:0/3222032810 learned_addr learned my addr 172.21.15.37:0/3222032810 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3c77fe640 1 --2- 172.21.15.37:0/3222032810 >> v2:172.21.15.37:3300/0 conn(0x7ff3c80d07f0 0x7ff3c80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3ccb3d640 1 --2- 172.21.15.37:0/3222032810 >> v2:172.21.15.203:3300/0 conn(0x7ff3c80d1cd0 0x7ff3c806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3c6ffd640 1 -- 172.21.15.37:0/3222032810 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942732022 0 0) 0x7ff3c806be60 con 0x7ff3c815da80 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3c6ffd640 1 -- 172.21.15.37:0/3222032810 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b0003530 con 0x7ff3c815da80 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3c77fe640 1 -- 172.21.15.37:0/3222032810 >> v2:172.21.15.203:3300/0 conn(0x7ff3c80d1cd0 msgr2=0x7ff3c806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:48.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3c77fe640 1 --2- 172.21.15.37:0/3222032810 >> v2:172.21.15.203:3300/0 conn(0x7ff3c80d1cd0 0x7ff3c806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3c77fe640 1 -- 172.21.15.37:0/3222032810 >> v1:172.21.15.37:6789/0 conn(0x7ff3c815da80 legacy=0x7ff3c80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.836+0000 7ff3c77fe640 1 -- 172.21.15.37:0/3222032810 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c80d2450 con 0x7ff3c80d07f0 2024-04-25T08:53:48.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3c77fe640 1 --2- 172.21.15.37:0/3222032810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80d07f0 0x7ff3c80d4230 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff3c806de30 tx=0x7ff3bc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d8b85fff022dc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3c6ffd640 1 -- 172.21.15.37:0/3222032810 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0b2070 con 0x7ff3c80d07f0 2024-04-25T08:53:48.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3c6ffd640 1 -- 172.21.15.37:0/3222032810 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc0a9070 con 0x7ff3c80d07f0 2024-04-25T08:53:48.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3c6ffd640 1 -- 172.21.15.37:0/3222032810 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0ad880 con 0x7ff3c80d07f0 2024-04-25T08:53:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/3222032810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80d07f0 msgr2=0x7ff3c80d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3ce5a0640 1 --2- 172.21.15.37:0/3222032810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80d07f0 0x7ff3c80d4230 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff3c806de30 tx=0x7ff3bc01c430 comp rx=0 tx=0).stop 2024-04-25T08:53:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/3222032810 shutdown_connections 2024-04-25T08:53:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3ce5a0640 1 --2- 172.21.15.37:0/3222032810 >> v2:172.21.15.203:3300/0 conn(0x7ff3c80d1cd0 0x7ff3c806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3ce5a0640 1 --2- 172.21.15.37:0/3222032810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80d07f0 0x7ff3c80d4230 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/3222032810 >> 172.21.15.37:0/3222032810 conn(0x7ff3c808b000 msgr2=0x7ff3c80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/3222032810 shutdown_connections 2024-04-25T08:53:48.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/3222032810 wait complete. 2024-04-25T08:53:48.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3ce5a0640 1 Processor -- start 2024-04-25T08:53:48.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.840+0000 7ff3ce5a0640 1 -- start start 2024-04-25T08:53:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3ce5a0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3c80d07f0 0x7ff3c80ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3ce5a0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80d1cd0 0x7ff3c80b74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3ce5a0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3c815da80 0x7ff3c8157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3ce5a0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c806d7a0 con 0x7ff3c80d1cd0 2024-04-25T08:53:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3ce5a0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c806d620 con 0x7ff3c815da80 2024-04-25T08:53:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3ce5a0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3c806d920 con 0x7ff3c80d07f0 2024-04-25T08:53:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80d1cd0 0x7ff3c80b74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80d1cd0 0x7ff3c80b74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52036/0 (socket says 172.21.15.37:52036) 2024-04-25T08:53:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c77fe640 1 -- 172.21.15.37:0/502040935 learned_addr learned my addr 172.21.15.37:0/502040935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3ccb3d640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3c815da80 0x7ff3c8157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c7fff640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3c80d07f0 0x7ff3c80ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c77fe640 1 -- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3c80d07f0 msgr2=0x7ff3c80ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c77fe640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3c80d07f0 0x7ff3c80ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c77fe640 1 -- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3c815da80 msgr2=0x7ff3c8157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c77fe640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3c815da80 0x7ff3c8157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c77fe640 1 -- 172.21.15.37:0/502040935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c80cb4d0 con 0x7ff3c80d1cd0 2024-04-25T08:53:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c77fe640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80d1cd0 0x7ff3c80b74b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc019040 tx=0x7ff3bc0191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c4ff9640 1 -- 172.21.15.37:0/502040935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0bc070 con 0x7ff3c80d1cd0 2024-04-25T08:53:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c4ff9640 1 -- 172.21.15.37:0/502040935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc0adcb0 con 0x7ff3c80d1cd0 2024-04-25T08:53:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/502040935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c80cb7b0 con 0x7ff3c80d1cd0 2024-04-25T08:53:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c4ff9640 1 -- 172.21.15.37:0/502040935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc0b2040 con 0x7ff3c80d1cd0 2024-04-25T08:53:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/502040935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3c81580a0 con 0x7ff3c80d1cd0 2024-04-25T08:53:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/502040935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3c804efa0 con 0x7ff3c80d1cd0 2024-04-25T08:53:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c4ff9640 1 -- 172.21.15.37:0/502040935 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff3bc0bf070 con 0x7ff3c80d1cd0 2024-04-25T08:53:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c4ff9640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3b4066f60 0x7ff3b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c4ff9640 1 -- 172.21.15.37:0/502040935 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff3bc11dd50 con 0x7ff3c80d1cd0 2024-04-25T08:53:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.844+0000 7ff3c7fff640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3b4066f60 0x7ff3b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.848+0000 7ff3c7fff640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3b4066f60 0x7ff3b4069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8007fd0 tx=0x7ff3b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:48.848+0000 7ff3c4ff9640 1 -- 172.21.15.37:0/502040935 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff3bc121030 con 0x7ff3c80d1cd0 2024-04-25T08:53:49.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.028+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/502040935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff3c80cfed0 con 0x7ff3c80d1cd0 2024-04-25T08:53:49.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.632+0000 7ff3c4ff9640 1 -- 172.21.15.37:0/502040935 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff3bc016050 con 0x7ff3c80d1cd0 2024-04-25T08:53:49.625 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-04-25T08:53:49.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3b4066f60 msgr2=0x7ff3b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3b4066f60 0x7ff3b4069420 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8007fd0 tx=0x7ff3b8009040 comp rx=0 tx=0).stop 2024-04-25T08:53:49.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/502040935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80d1cd0 msgr2=0x7ff3c80b74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80d1cd0 0x7ff3c80b74b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc019040 tx=0x7ff3bc0191d0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/502040935 shutdown_connections 2024-04-25T08:53:49.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff3b4066f60 0x7ff3b4069420 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff3c815da80 0x7ff3c8157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3c80d1cd0 0x7ff3c80b74b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 --2- 172.21.15.37:0/502040935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff3c80d07f0 0x7ff3c80ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/502040935 >> 172.21.15.37:0/502040935 conn(0x7ff3c808b000 msgr2=0x7ff3c80b3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:49.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/502040935 shutdown_connections 2024-04-25T08:53:49.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.636+0000 7ff3ce5a0640 1 -- 172.21.15.37:0/502040935 wait complete. 2024-04-25T08:53:49.651 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:53:49.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.780+0000 7fd781205640 1 Processor -- start 2024-04-25T08:53:50.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.780+0000 7fd781205640 1 -- start start 2024-04-25T08:53:50.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd781205640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd77c0ce510 0x7fd77c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd781205640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd77c144ca0 0x7fd77c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd781205640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd77c06ec60 0x7fd77c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd781205640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c0cc2b0 con 0x7fd77c144ca0 2024-04-25T08:53:50.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd781205640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd77c06e290 con 0x7fd77c0ce510 2024-04-25T08:53:50.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd781205640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0cee30 con 0x7fd77c06ec60 2024-04-25T08:53:50.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd77a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd77c144ca0 0x7fd77c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd77a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd77c144ca0 0x7fd77c06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58346/0 (socket says 172.21.15.37:58346) 2024-04-25T08:53:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd77a59c640 1 -- 172.21.15.37:0/3704393216 learned_addr learned my addr 172.21.15.37:0/3704393216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd77ad9d640 1 --2- 172.21.15.37:0/3704393216 >> v2:172.21.15.203:3300/0 conn(0x7fd77c0ce510 0x7fd77c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd77b59e640 1 --2- 172.21.15.37:0/3704393216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd77c06ec60 0x7fd77c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd77a59c640 1 -- 172.21.15.37:0/3704393216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd77c06ec60 msgr2=0x7fd77c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd77a59c640 1 --2- 172.21.15.37:0/3704393216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd77c06ec60 0x7fd77c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd77a59c640 1 -- 172.21.15.37:0/3704393216 >> v2:172.21.15.203:3300/0 conn(0x7fd77c0ce510 msgr2=0x7fd77c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd77a59c640 1 --2- 172.21.15.37:0/3704393216 >> v2:172.21.15.203:3300/0 conn(0x7fd77c0ce510 0x7fd77c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd77a59c640 1 -- 172.21.15.37:0/3704393216 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c0ccc70 con 0x7fd77c144ca0 2024-04-25T08:53:50.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd77a59c640 1 --2- 172.21.15.37:0/3704393216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd77c144ca0 0x7fd77c06e720 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd77c144c50 tx=0x7fd76401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953c79db15d8aeed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd779d9b640 1 -- 172.21.15.37:0/3704393216 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7640b5070 con 0x7fd77c144ca0 2024-04-25T08:53:50.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd779d9b640 1 -- 172.21.15.37:0/3704393216 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7640a89d0 con 0x7fd77c144ca0 2024-04-25T08:53:50.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.784+0000 7fd779d9b640 1 -- 172.21.15.37:0/3704393216 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7640a8da0 con 0x7fd77c144ca0 2024-04-25T08:53:50.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 -- 172.21.15.37:0/3704393216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd77c144ca0 msgr2=0x7fd77c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 --2- 172.21.15.37:0/3704393216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd77c144ca0 0x7fd77c06e720 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd77c144c50 tx=0x7fd76401ed10 comp rx=0 tx=0).stop 2024-04-25T08:53:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 -- 172.21.15.37:0/3704393216 shutdown_connections 2024-04-25T08:53:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 --2- 172.21.15.37:0/3704393216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd77c144ca0 0x7fd77c06e720 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 --2- 172.21.15.37:0/3704393216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd77c06ec60 0x7fd77c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 --2- 172.21.15.37:0/3704393216 >> v2:172.21.15.203:3300/0 conn(0x7fd77c0ce510 0x7fd77c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 -- 172.21.15.37:0/3704393216 >> 172.21.15.37:0/3704393216 conn(0x7fd77c0b11a0 msgr2=0x7fd77c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:50.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 -- 172.21.15.37:0/3704393216 shutdown_connections 2024-04-25T08:53:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 -- 172.21.15.37:0/3704393216 wait complete. 2024-04-25T08:53:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 Processor -- start 2024-04-25T08:53:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 -- start start 2024-04-25T08:53:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd77c06ec60 0x7fd77c0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd77c0ce510 0x7fd77c1e5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd77c144ca0 0x7fd77c132ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c0cc900 con 0x7fd77c0ce510 2024-04-25T08:53:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c06df20 con 0x7fd77c06ec60 2024-04-25T08:53:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c1451f0 con 0x7fd77c144ca0 2024-04-25T08:53:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd77c0ce510 0x7fd77c1e5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd77c0ce510 0x7fd77c1e5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58360/0 (socket says 172.21.15.37:58360) 2024-04-25T08:53:50.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77a59c640 1 -- 172.21.15.37:0/1253044272 learned_addr learned my addr 172.21.15.37:0/1253044272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:50.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77ad9d640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd77c06ec60 0x7fd77c0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77b59e640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd77c144ca0 0x7fd77c132ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77a59c640 1 -- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd77c144ca0 msgr2=0x7fd77c132ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77a59c640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd77c144ca0 0x7fd77c132ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77a59c640 1 -- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd77c06ec60 msgr2=0x7fd77c0a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77a59c640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd77c06ec60 0x7fd77c0a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77a59c640 1 -- 172.21.15.37:0/1253044272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c130f50 con 0x7fd77c0ce510 2024-04-25T08:53:50.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77ad9d640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd77c06ec60 0x7fd77c0a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:50.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd77a59c640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd77c0ce510 0x7fd77c1e5100 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd76401d3c0 tx=0x7fd76401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd7737fe640 1 -- 172.21.15.37:0/1253044272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7640b5070 con 0x7fd77c0ce510 2024-04-25T08:53:50.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd7737fe640 1 -- 172.21.15.37:0/1253044272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7640b0a50 con 0x7fd77c0ce510 2024-04-25T08:53:50.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 -- 172.21.15.37:0/1253044272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd77c131230 con 0x7fd77c0ce510 2024-04-25T08:53:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd7737fe640 1 -- 172.21.15.37:0/1253044272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7640b0e20 con 0x7fd77c0ce510 2024-04-25T08:53:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.788+0000 7fd781205640 1 -- 172.21.15.37:0/1253044272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c1317f0 con 0x7fd77c0ce510 2024-04-25T08:53:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.792+0000 7fd781205640 1 -- 172.21.15.37:0/1253044272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd77c04efa0 con 0x7fd77c0ce510 2024-04-25T08:53:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.792+0000 7fd7737fe640 1 -- 172.21.15.37:0/1253044272 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7640c2060 con 0x7fd77c0ce510 2024-04-25T08:53:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.792+0000 7fd7737fe640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd75c066f60 0x7fd75c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.792+0000 7fd7737fe640 1 -- 172.21.15.37:0/1253044272 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd764122260 con 0x7fd77c0ce510 2024-04-25T08:53:50.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.792+0000 7fd77ad9d640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd75c066f60 0x7fd75c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.792+0000 7fd77ad9d640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd75c066f60 0x7fd75c069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd774003fd0 tx=0x7fd7740039e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:49.796+0000 7fd7737fe640 1 -- 172.21.15.37:0/1253044272 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd764018870 con 0x7fd77c0ce510 2024-04-25T08:53:50.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 -- 172.21.15.37:0/1253044272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd77c0cb590 con 0x7fd77c0ce510 2024-04-25T08:53:50.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd7737fe640 1 -- 172.21.15.37:0/1253044272 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fd7640f3410 con 0x7fd77c0ce510 2024-04-25T08:53:50.040 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:50.040 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:53:50.040 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-04-25T08:53:50.040 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.041 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-04-25T08:53:50.041 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpm2pLfABAAJbn2dGKdl8SHS8pApIWC1Q==", 2024-04-25T08:53:50.041 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.041 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:53:50.041 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:53:50.041 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:53:50.042 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.042 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.042 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-04-25T08:53:50.043 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmqB+jDxAAmRnBzN3vqO8HIf/cd3QoJQ==", 2024-04-25T08:53:50.043 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.043 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:53:50.043 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:53:50.043 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:53:50.043 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.044 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.044 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.044 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-04-25T08:53:50.044 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmiXRqCRAACtY8cFxwqOT4WNzj9WmeKA==", 2024-04-25T08:53:50.044 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.045 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:53:50.045 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:53:50.045 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:53:50.045 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.045 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.046 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.046 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-04-25T08:53:50.046 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmX3ARGBAAW/ZUr85NsKtd8HzxE+Yefw==", 2024-04-25T08:53:50.046 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.046 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:53:50.046 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:53:50.047 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:53:50.047 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.047 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.047 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.047 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-04-25T08:53:50.048 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCSGSpmwhMiJhAAbmQz36LujHeYXaKIOdLLSA==", 2024-04-25T08:53:50.048 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.048 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:53:50.048 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:53:50.048 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.048 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.049 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.049 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-04-25T08:53:50.049 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCVGSpm/OLPKxAA9ZmrrWxvt4QDj2lHATnzSg==", 2024-04-25T08:53:50.049 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.049 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:53:50.050 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:53:50.050 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.050 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.050 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.050 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-04-25T08:53:50.051 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCYGSpmq7tEMxAA/wjwgkrtnTkrMoC6G+YIlQ==", 2024-04-25T08:53:50.051 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.051 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:53:50.051 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:53:50.051 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.051 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.052 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.052 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-04-25T08:53:50.052 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCbGSpmReB1ORAAkXje/r11I3ArZHErIvijgw==", 2024-04-25T08:53:50.052 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.052 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:53:50.053 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:53:50.053 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.053 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.053 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.053 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-04-25T08:53:50.054 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQChGSpm4XfLORAAGj+R1Nk/EGmvEdpt8kQJlg==", 2024-04-25T08:53:50.054 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.054 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:53:50.054 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:53:50.054 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.054 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.055 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.055 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-04-25T08:53:50.055 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQClGSpmd5f1ARAAKF3XG//v3Uqx4bckcD4JMw==", 2024-04-25T08:53:50.055 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.055 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:53:50.056 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:53:50.056 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.056 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.057 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-04-25T08:53:50.057 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCoGSpmi9OECBAAJBskMWGCNlAaej9xUc6g8w==", 2024-04-25T08:53:50.057 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.057 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:53:50.057 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:53:50.057 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.058 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.058 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-04-25T08:53:50.058 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCrGSpmunQiDhAA3UZo2vzT44HEYUBUia75NA==", 2024-04-25T08:53:50.058 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA==", 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmu8UoJhAAyutD5hzKLZmr57WuAWScuQ==", 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmGVVdIxAAY7/VOuqF5OoomqqVxE1Ifw==", 2024-04-25T08:53:50.060 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmjFNDKxAAmo6DB9W17TZaGlvkdqkSuQ==", 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm+mGgJRAA9oTRTlZpGBdCZ0/bj580Bg==", 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-mds", 2024-04-25T08:53:50.062 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC2GSpmxqN3ExAA6odK8kAU329ikgqjKUWuzQ==", 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC2GSpmRb53ExAAQC1YgZ+m1wuAogSUCCeefg==", 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-osd", 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC2GSpmb9R3ExAApuhbYzzEpdYrIdqJbZV52w==", 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC2GSpmY+p3ExAAqToxGaXy06C9EXZ6SiNgtg==", 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC2GSpmEQF4ExAAOxv5weWr8hJaRH9WhVBecg==", 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.064 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQC2GSpmKxh4ExAAGyIbORV6fjRjU5HrsiCRbg==", 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpmLChTORAAo6fs/9PDJ5YqqcbfHfPO9Q==", 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:50.065 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-04-25T08:53:50.066 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm3NLHNhAAeqSA658L6P9OcrSO5JVzLg==", 2024-04-25T08:53:50.066 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:53:50.066 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:53:50.066 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T08:53:50.066 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:53:50.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.066 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:50.066 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T08:53:50.066 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:53:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 -- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd75c066f60 msgr2=0x7fd75c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd75c066f60 0x7fd75c069420 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd774003fd0 tx=0x7fd7740039e0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 -- 172.21.15.37:0/1253044272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd77c0ce510 msgr2=0x7fd77c1e5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd77c0ce510 0x7fd77c1e5100 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd76401d3c0 tx=0x7fd76401bab0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 -- 172.21.15.37:0/1253044272 shutdown_connections 2024-04-25T08:53:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd75c066f60 0x7fd75c069420 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd77c144ca0 0x7fd77c132ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd77c0ce510 0x7fd77c1e5100 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 --2- 172.21.15.37:0/1253044272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd77c06ec60 0x7fd77c0a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 -- 172.21.15.37:0/1253044272 >> 172.21.15.37:0/1253044272 conn(0x7fd77c0b11a0 msgr2=0x7fd77c0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 -- 172.21.15.37:0/1253044272 shutdown_connections 2024-04-25T08:53:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.016+0000 7fd781205640 1 -- 172.21.15.37:0/1253044272 wait complete. 2024-04-25T08:53:50.068 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T08:53:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.196+0000 7fe4ba048640 1 Processor -- start 2024-04-25T08:53:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.196+0000 7fe4ba048640 1 -- start start 2024-04-25T08:53:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.196+0000 7fe4ba048640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe4b40b2c30 0x7fe4b40b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4ba048640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe4b409a7b0 0x7fe4b4099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4ba048640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4b4099fb0 0x7fe4b40ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4ba048640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b40cbec0 con 0x7fe4b40b2c30 2024-04-25T08:53:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4ba048640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b40cc1a0 con 0x7fe4b409a7b0 2024-04-25T08:53:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4ba048640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40cc480 con 0x7fe4b4099fb0 2024-04-25T08:53:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe4b40b2c30 0x7fe4b40b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b2ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe4b409a7b0 0x7fe4b4099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe4b40b2c30 0x7fe4b40b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58376/0 (socket says 172.21.15.37:58376) 2024-04-25T08:53:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b37fe640 1 -- 172.21.15.37:0/3448892210 learned_addr learned my addr 172.21.15.37:0/3448892210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b3fff640 1 --2- 172.21.15.37:0/3448892210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4b4099fb0 0x7fe4b40ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b37fe640 1 -- 172.21.15.37:0/3448892210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4b4099fb0 msgr2=0x7fe4b40ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b37fe640 1 --2- 172.21.15.37:0/3448892210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4b4099fb0 0x7fe4b40ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b37fe640 1 -- 172.21.15.37:0/3448892210 >> v2:172.21.15.203:3300/0 conn(0x7fe4b409a7b0 msgr2=0x7fe4b4099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b37fe640 1 --2- 172.21.15.37:0/3448892210 >> v2:172.21.15.203:3300/0 conn(0x7fe4b409a7b0 0x7fe4b4099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b37fe640 1 -- 172.21.15.37:0/3448892210 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b40cc760 con 0x7fe4b40b2c30 2024-04-25T08:53:50.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b37fe640 1 --2- 172.21.15.37:0/3448892210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b40b2c30 0x7fe4b40b0ba0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4072190 tx=0x7fe4a401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2d961164fbc5d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b27fc640 1 -- 172.21.15.37:0/3448892210 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40ab3f0 con 0x7fe4b40b2c30 2024-04-25T08:53:50.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b27fc640 1 -- 172.21.15.37:0/3448892210 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a40b0070 con 0x7fe4b40b2c30 2024-04-25T08:53:50.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4b27fc640 1 -- 172.21.15.37:0/3448892210 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40ab7b0 con 0x7fe4b40b2c30 2024-04-25T08:53:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4ba048640 1 -- 172.21.15.37:0/3448892210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b40b2c30 msgr2=0x7fe4b40b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.200+0000 7fe4ba048640 1 --2- 172.21.15.37:0/3448892210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b40b2c30 0x7fe4b40b0ba0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4072190 tx=0x7fe4a401c520 comp rx=0 tx=0).stop 2024-04-25T08:53:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 -- 172.21.15.37:0/3448892210 shutdown_connections 2024-04-25T08:53:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 --2- 172.21.15.37:0/3448892210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4b4099fb0 0x7fe4b40ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 --2- 172.21.15.37:0/3448892210 >> v2:172.21.15.203:3300/0 conn(0x7fe4b409a7b0 0x7fe4b4099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 --2- 172.21.15.37:0/3448892210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b40b2c30 0x7fe4b40b0ba0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 -- 172.21.15.37:0/3448892210 >> 172.21.15.37:0/3448892210 conn(0x7fe4b40b3010 msgr2=0x7fe4b40b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 -- 172.21.15.37:0/3448892210 shutdown_connections 2024-04-25T08:53:50.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 -- 172.21.15.37:0/3448892210 wait complete. 2024-04-25T08:53:50.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 Processor -- start 2024-04-25T08:53:50.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 -- start start 2024-04-25T08:53:50.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b4099fb0 0x7fe4b40cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4b409a7b0 0x7fe4b40c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4b40b2c30 0x7fe4b41537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b406c380 con 0x7fe4b4099fb0 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b406c200 con 0x7fe4b409a7b0 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4ba048640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b415d190 con 0x7fe4b40b2c30 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b4099fb0 0x7fe4b40cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b4099fb0 0x7fe4b40cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58382/0 (socket says 172.21.15.37:58382) 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4b37fe640 1 -- 172.21.15.37:0/989655257 learned_addr learned my addr 172.21.15.37:0/989655257 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4b2ffd640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4b409a7b0 0x7fe4b40c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.204+0000 7fe4b3fff640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4b40b2c30 0x7fe4b41537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe4b37fe640 1 -- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4b40b2c30 msgr2=0x7fe4b41537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe4b37fe640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4b40b2c30 0x7fe4b41537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe4b37fe640 1 -- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4b409a7b0 msgr2=0x7fe4b40c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe4b37fe640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4b409a7b0 0x7fe4b40c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe4b37fe640 1 -- 172.21.15.37:0/989655257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4068790 con 0x7fe4b4099fb0 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe4b3fff640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4b40b2c30 0x7fe4b41537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:50.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe4b37fe640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b4099fb0 0x7fe4b40cb170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40ad040 tx=0x7fe4a401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe49bfff640 1 -- 172.21.15.37:0/989655257 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40be3f0 con 0x7fe4b4099fb0 2024-04-25T08:53:50.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe49bfff640 1 -- 172.21.15.37:0/989655257 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a40be590 con 0x7fe4b4099fb0 2024-04-25T08:53:50.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe49bfff640 1 -- 172.21.15.37:0/989655257 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a40be960 con 0x7fe4b4099fb0 2024-04-25T08:53:50.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe4ba048640 1 -- 172.21.15.37:0/989655257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b40c8c80 con 0x7fe4b4099fb0 2024-04-25T08:53:50.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe4ba048640 1 -- 172.21.15.37:0/989655257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b40ca970 con 0x7fe4b4099fb0 2024-04-25T08:53:50.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe4ba048640 1 -- 172.21.15.37:0/989655257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b404efa0 con 0x7fe4b4099fb0 2024-04-25T08:53:50.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe49bfff640 1 -- 172.21.15.37:0/989655257 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4a40b43f0 con 0x7fe4b4099fb0 2024-04-25T08:53:50.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe49bfff640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4a0066f60 0x7fe4a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe49bfff640 1 -- 172.21.15.37:0/989655257 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe4a411ea10 con 0x7fe4b4099fb0 2024-04-25T08:53:50.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.208+0000 7fe4b2ffd640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4a0066f60 0x7fe4a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.212+0000 7fe4b2ffd640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4a0066f60 0x7fe4a0069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4157300 tx=0x7fe4a8009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.212+0000 7fe49bfff640 1 -- 172.21.15.37:0/989655257 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe4a4016110 con 0x7fe4b4099fb0 2024-04-25T08:53:50.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.432+0000 7fe4ba048640 1 -- 172.21.15.37:0/989655257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe4b40b2460 con 0x7fe4b4099fb0 2024-04-25T08:53:50.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.436+0000 7fe49bfff640 1 -- 172.21.15.37:0/989655257 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe4a40ef6b0 con 0x7fe4b4099fb0 2024-04-25T08:53:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.436+0000 7fe4ba048640 1 -- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4a0066f60 msgr2=0x7fe4a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.436+0000 7fe4ba048640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4a0066f60 0x7fe4a0069420 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4157300 tx=0x7fe4a8009190 comp rx=0 tx=0).stop 2024-04-25T08:53:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.436+0000 7fe4ba048640 1 -- 172.21.15.37:0/989655257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b4099fb0 msgr2=0x7fe4b40cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.436+0000 7fe4ba048640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b4099fb0 0x7fe4b40cb170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40ad040 tx=0x7fe4a401ad60 comp rx=0 tx=0).stop 2024-04-25T08:53:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.436+0000 7fe4ba048640 1 -- 172.21.15.37:0/989655257 shutdown_connections 2024-04-25T08:53:50.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.436+0000 7fe4ba048640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4a0066f60 0x7fe4a0069420 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.436+0000 7fe4ba048640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4b40b2c30 0x7fe4b41537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.436+0000 7fe4ba048640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4b409a7b0 0x7fe4b40c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.436+0000 7fe4ba048640 1 --2- 172.21.15.37:0/989655257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4b4099fb0 0x7fe4b40cb170 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.436+0000 7fe4ba048640 1 -- 172.21.15.37:0/989655257 >> 172.21.15.37:0/989655257 conn(0x7fe4b40b3010 msgr2=0x7fe4b40c69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.440+0000 7fe4ba048640 1 -- 172.21.15.37:0/989655257 shutdown_connections 2024-04-25T08:53:50.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.440+0000 7fe4ba048640 1 -- 172.21.15.37:0/989655257 wait complete. 2024-04-25T08:53:50.448 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T08:53:50.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f1915d1b640 1 Processor -- start 2024-04-25T08:53:50.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f1915d1b640 1 -- start start 2024-04-25T08:53:50.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f1915d1b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19100af280 0x7f191015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f1915d1b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1910066c30 con 0x7f191006c3b0 2024-04-25T08:53:50.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f1915d1b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19100af660 con 0x7f19100b2c30 2024-04-25T08:53:50.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f1915d1b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191006daa0 con 0x7f19100af280 2024-04-25T08:53:50.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f190ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f191006c3b0 0x7f191006df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57560/0 (socket says 172.21.15.37:57560) 2024-04-25T08:53:50.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f190ffff640 1 -- 172.21.15.37:0/1268477328 learned_addr learned my addr 172.21.15.37:0/1268477328 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:50.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f190f7fe640 1 --2- 172.21.15.37:0/1268477328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19100af280 0x7f191015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f190e7fc640 1 -- 172.21.15.37:0/1268477328 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323652934 0 0) 0x7f1910066c30 con 0x7f191006c3b0 2024-04-25T08:53:50.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f190e7fc640 1 -- 172.21.15.37:0/1268477328 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18f8003530 con 0x7f191006c3b0 2024-04-25T08:53:50.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f190e7fc640 1 -- 172.21.15.37:0/1268477328 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096777635 0 0) 0x7f19100af660 con 0x7f19100b2c30 2024-04-25T08:53:50.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f190e7fc640 1 -- 172.21.15.37:0/1268477328 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1910066c30 con 0x7f19100b2c30 2024-04-25T08:53:50.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f190f7fe640 1 -- 172.21.15.37:0/1268477328 >> v1:172.21.15.203:6789/0 conn(0x7f19100b2c30 legacy=0x7f191006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f190f7fe640 1 -- 172.21.15.37:0/1268477328 >> v1:172.21.15.37:6789/0 conn(0x7f191006c3b0 legacy=0x7f191006df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.568+0000 7f190f7fe640 1 -- 172.21.15.37:0/1268477328 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191015e640 con 0x7f19100af280 2024-04-25T08:53:50.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f190f7fe640 1 --2- 172.21.15.37:0/1268477328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19100af280 0x7f191015df10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f19000097f0 tx=0x7f190001c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df48e47cd805931 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f190e7fc640 1 -- 172.21.15.37:0/1268477328 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000b0070 con 0x7f19100af280 2024-04-25T08:53:50.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f190e7fc640 1 -- 172.21.15.37:0/1268477328 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19000ab600 con 0x7f19100af280 2024-04-25T08:53:50.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f190e7fc640 1 -- 172.21.15.37:0/1268477328 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000ab9d0 con 0x7f19100af280 2024-04-25T08:53:50.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f1915d1b640 1 -- 172.21.15.37:0/1268477328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19100af280 msgr2=0x7f191015df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f1915d1b640 1 --2- 172.21.15.37:0/1268477328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19100af280 0x7f191015df10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f19000097f0 tx=0x7f190001c4e0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f1915d1b640 1 -- 172.21.15.37:0/1268477328 shutdown_connections 2024-04-25T08:53:50.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f1915d1b640 1 --2- 172.21.15.37:0/1268477328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19100af280 0x7f191015df10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f1915d1b640 1 -- 172.21.15.37:0/1268477328 >> 172.21.15.37:0/1268477328 conn(0x7f1910071810 msgr2=0x7f1910071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:50.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f1915d1b640 1 -- 172.21.15.37:0/1268477328 shutdown_connections 2024-04-25T08:53:50.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f1915d1b640 1 -- 172.21.15.37:0/1268477328 wait complete. 2024-04-25T08:53:50.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f1915d1b640 1 Processor -- start 2024-04-25T08:53:50.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.572+0000 7f1915d1b640 1 -- start start 2024-04-25T08:53:50.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f1915d1b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f191006c3b0 0x7f1910154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f1915d1b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19100af280 0x7f1910157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f1915d1b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19100b2c30 0x7f191014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f1915d1b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191006f1e0 con 0x7f191006c3b0 2024-04-25T08:53:50.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f1915d1b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191006f060 con 0x7f19100b2c30 2024-04-25T08:53:50.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f1915d1b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191016b8b0 con 0x7f19100af280 2024-04-25T08:53:50.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f191006c3b0 0x7f1910154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f191006c3b0 0x7f1910154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58388/0 (socket says 172.21.15.37:58388) 2024-04-25T08:53:50.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190f7fe640 1 -- 172.21.15.37:0/1688899280 learned_addr learned my addr 172.21.15.37:0/1688899280 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190effd640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19100af280 0x7f1910157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190ffff640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19100b2c30 0x7f191014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190f7fe640 1 -- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19100af280 msgr2=0x7f1910157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190f7fe640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19100af280 0x7f1910157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190f7fe640 1 -- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19100b2c30 msgr2=0x7f191014a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190f7fe640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19100b2c30 0x7f191014a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190f7fe640 1 -- 172.21.15.37:0/1688899280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1910154d10 con 0x7f191006c3b0 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190f7fe640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f191006c3b0 0x7f1910154730 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f19000190f0 tx=0x7f190001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f18f7fff640 1 -- 172.21.15.37:0/1688899280 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000b0050 con 0x7f191006c3b0 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f18f7fff640 1 -- 172.21.15.37:0/1688899280 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19000bc070 con 0x7f191006c3b0 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f1915d1b640 1 -- 172.21.15.37:0/1688899280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1910153440 con 0x7f191006c3b0 2024-04-25T08:53:50.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f18f7fff640 1 -- 172.21.15.37:0/1688899280 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900006e20 con 0x7f191006c3b0 2024-04-25T08:53:50.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f1915d1b640 1 -- 172.21.15.37:0/1688899280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19101539c0 con 0x7f191006c3b0 2024-04-25T08:53:50.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f1915d1b640 1 -- 172.21.15.37:0/1688899280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191004efa0 con 0x7f191006c3b0 2024-04-25T08:53:50.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f18f7fff640 1 -- 172.21.15.37:0/1688899280 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19000b5870 con 0x7f191006c3b0 2024-04-25T08:53:50.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f18f7fff640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18fc066f60 0x7f18fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f18f7fff640 1 -- 172.21.15.37:0/1688899280 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f190011ffe0 con 0x7f191006c3b0 2024-04-25T08:53:50.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190effd640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18fc066f60 0x7f18fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.576+0000 7f190effd640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18fc066f60 0x7f18fc069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1904007fd0 tx=0x7f1904009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.580+0000 7f18f7fff640 1 -- 172.21.15.37:0/1688899280 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19000160d0 con 0x7f191006c3b0 2024-04-25T08:53:50.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.792+0000 7f1915d1b640 1 -- 172.21.15.37:0/1688899280 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f191015be10 con 0x7f191006c3b0 2024-04-25T08:53:50.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.796+0000 7f18f7fff640 1 -- 172.21.15.37:0/1688899280 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f19000f11e0 con 0x7f191006c3b0 2024-04-25T08:53:50.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 -- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18fc066f60 msgr2=0x7f18fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18fc066f60 0x7f18fc069420 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1904007fd0 tx=0x7f1904009040 comp rx=0 tx=0).stop 2024-04-25T08:53:50.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 -- 172.21.15.37:0/1688899280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f191006c3b0 msgr2=0x7f1910154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f191006c3b0 0x7f1910154730 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f19000190f0 tx=0x7f190001ad20 comp rx=0 tx=0).stop 2024-04-25T08:53:50.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 -- 172.21.15.37:0/1688899280 shutdown_connections 2024-04-25T08:53:50.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18fc066f60 0x7f18fc069420 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19100b2c30 0x7f191014a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19100af280 0x7f1910157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 --2- 172.21.15.37:0/1688899280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f191006c3b0 0x7f1910154730 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 -- 172.21.15.37:0/1688899280 >> 172.21.15.37:0/1688899280 conn(0x7f1910071810 msgr2=0x7f19100b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:50.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 -- 172.21.15.37:0/1688899280 shutdown_connections 2024-04-25T08:53:50.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.800+0000 7f1915d1b640 1 -- 172.21.15.37:0/1688899280 wait complete. 2024-04-25T08:53:50.808 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T08:53:50.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.924+0000 7eff802d5640 1 Processor -- start 2024-04-25T08:53:50.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.924+0000 7eff802d5640 1 -- start start 2024-04-25T08:53:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7eff7815b950 0x7eff7809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff780cf420 0x7eff780cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7815bd30 con 0x7eff780cfb80 2024-04-25T08:53:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff780cc0d0 con 0x7eff7815b950 2024-04-25T08:53:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7809ab10 con 0x7eff780cf420 2024-04-25T08:53:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7e071640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7eff7815b950 0x7eff7809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7e872640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7eff780cfb80 0x7eff7806f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57564/0 (socket says 172.21.15.37:57564) 2024-04-25T08:53:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7e872640 1 -- 172.21.15.37:0/293050194 learned_addr learned my addr 172.21.15.37:0/293050194 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:50.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7e071640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7eff7815b950 0x7eff7809a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58398/0 (socket says 172.21.15.37:58398) 2024-04-25T08:53:50.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7d870640 1 --2- 172.21.15.37:0/293050194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff780cf420 0x7eff780cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7e071640 1 -- 172.21.15.37:0/293050194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff780cf420 msgr2=0x7eff780cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:50.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7e071640 1 --2- 172.21.15.37:0/293050194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff780cf420 0x7eff780cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7e071640 1 -- 172.21.15.37:0/293050194 >> v1:172.21.15.37:6789/0 conn(0x7eff780cfb80 legacy=0x7eff7806f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7e071640 1 -- 172.21.15.37:0/293050194 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff780cc370 con 0x7eff7815b950 2024-04-25T08:53:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7e071640 1 --2- 172.21.15.37:0/293050194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7815b950 0x7eff7809a480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7eff740097c0 tx=0x7eff7401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91f0f23a7bd3b9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7d06f640 1 -- 172.21.15.37:0/293050194 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff740b2070 con 0x7eff7815b950 2024-04-25T08:53:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7d06f640 1 -- 172.21.15.37:0/293050194 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff740ad680 con 0x7eff7815b950 2024-04-25T08:53:50.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff7d06f640 1 -- 172.21.15.37:0/293050194 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff740ada50 con 0x7eff7815b950 2024-04-25T08:53:50.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 -- 172.21.15.37:0/293050194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7815b950 msgr2=0x7eff7809a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 --2- 172.21.15.37:0/293050194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7815b950 0x7eff7809a480 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7eff740097c0 tx=0x7eff7401c470 comp rx=0 tx=0).stop 2024-04-25T08:53:50.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 -- 172.21.15.37:0/293050194 shutdown_connections 2024-04-25T08:53:50.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 --2- 172.21.15.37:0/293050194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff780cf420 0x7eff780cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 --2- 172.21.15.37:0/293050194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff7815b950 0x7eff7809a480 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 -- 172.21.15.37:0/293050194 >> 172.21.15.37:0/293050194 conn(0x7eff780b30b0 msgr2=0x7eff780b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:50.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 -- 172.21.15.37:0/293050194 shutdown_connections 2024-04-25T08:53:50.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.928+0000 7eff802d5640 1 -- 172.21.15.37:0/293050194 wait complete. 2024-04-25T08:53:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff802d5640 1 Processor -- start 2024-04-25T08:53:50.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff802d5640 1 -- start start 2024-04-25T08:53:50.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff802d5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff780cf420 0x7eff78154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff802d5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff780cfb80 0x7eff78154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff802d5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7815b950 0x7eff7815a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff802d5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff780cd390 con 0x7eff780cfb80 2024-04-25T08:53:50.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff802d5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff780d03a0 con 0x7eff780cf420 2024-04-25T08:53:50.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff802d5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff78153b90 con 0x7eff7815b950 2024-04-25T08:53:50.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff7d870640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff780cfb80 0x7eff78154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff7d870640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff780cfb80 0x7eff78154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58404/0 (socket says 172.21.15.37:58404) 2024-04-25T08:53:50.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff7d870640 1 -- 172.21.15.37:0/3603933866 learned_addr learned my addr 172.21.15.37:0/3603933866 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff7e071640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff780cf420 0x7eff78154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff7e872640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7815b950 0x7eff7815a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff7d870640 1 -- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7815b950 msgr2=0x7eff7815a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff7d870640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7815b950 0x7eff7815a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff7d870640 1 -- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff780cf420 msgr2=0x7eff78154920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff7d870640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff780cf420 0x7eff78154920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff7d870640 1 -- 172.21.15.37:0/3603933866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff780c73b0 con 0x7eff780cfb80 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff7d870640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff780cfb80 0x7eff78154e60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7eff6800d5f0 tx=0x7eff6800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff6effd640 1 -- 172.21.15.37:0/3603933866 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680125f0 con 0x7eff780cfb80 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff6effd640 1 -- 172.21.15.37:0/3603933866 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff68019070 con 0x7eff780cfb80 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff802d5640 1 -- 172.21.15.37:0/3603933866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff780c7690 con 0x7eff780cfb80 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff6effd640 1 -- 172.21.15.37:0/3603933866 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff680129b0 con 0x7eff780cfb80 2024-04-25T08:53:50.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff802d5640 1 -- 172.21.15.37:0/3603933866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff780c7c50 con 0x7eff780cfb80 2024-04-25T08:53:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.932+0000 7eff802d5640 1 -- 172.21.15.37:0/3603933866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7804efa0 con 0x7eff780cfb80 2024-04-25T08:53:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.936+0000 7eff6effd640 1 -- 172.21.15.37:0/3603933866 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff68012b50 con 0x7eff780cfb80 2024-04-25T08:53:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.936+0000 7eff6effd640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7eff5c066f60 0x7eff5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.936+0000 7eff6effd640 1 -- 172.21.15.37:0/3603933866 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7eff68085280 con 0x7eff780cfb80 2024-04-25T08:53:50.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.936+0000 7eff7e071640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7eff5c066f60 0x7eff5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.936+0000 7eff7e071640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7eff5c066f60 0x7eff5c069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7eff740af000 tx=0x7eff740a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:50.940+0000 7eff6effd640 1 -- 172.21.15.37:0/3603933866 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff68056450 con 0x7eff780cfb80 2024-04-25T08:53:51.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.156+0000 7eff802d5640 1 -- 172.21.15.37:0/3603933866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7eff780d13a0 con 0x7eff780cfb80 2024-04-25T08:53:51.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff6effd640 1 -- 172.21.15.37:0/3603933866 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7eff68055d50 con 0x7eff780cfb80 2024-04-25T08:53:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 -- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7eff5c066f60 msgr2=0x7eff5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7eff5c066f60 0x7eff5c069420 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7eff740af000 tx=0x7eff740a7000 comp rx=0 tx=0).stop 2024-04-25T08:53:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 -- 172.21.15.37:0/3603933866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff780cfb80 msgr2=0x7eff78154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff780cfb80 0x7eff78154e60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7eff6800d5f0 tx=0x7eff6800dab0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 -- 172.21.15.37:0/3603933866 shutdown_connections 2024-04-25T08:53:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7eff5c066f60 0x7eff5c069420 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff7815b950 0x7eff7815a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff780cfb80 0x7eff78154e60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 --2- 172.21.15.37:0/3603933866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff780cf420 0x7eff78154920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 -- 172.21.15.37:0/3603933866 >> 172.21.15.37:0/3603933866 conn(0x7eff780b30b0 msgr2=0x7eff7806ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:51.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 -- 172.21.15.37:0/3603933866 shutdown_connections 2024-04-25T08:53:51.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.160+0000 7eff802d5640 1 -- 172.21.15.37:0/3603933866 wait complete. 2024-04-25T08:53:51.171 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T08:53:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.288+0000 7f02e23a3640 1 Processor -- start 2024-04-25T08:53:51.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.288+0000 7f02e23a3640 1 -- start start 2024-04-25T08:53:51.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02e23a3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f02dc0b2a70 0x7f02dc0b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02e23a3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f02dc15f120 0x7f02dc0cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02e23a3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02dc0b1000 con 0x7f02dc15ed40 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02e23a3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02dc0b1610 con 0x7f02dc0b2a70 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02e23a3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02dc0d0500 con 0x7f02dc15f120 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02db7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f02dc0b2a70 0x7f02dc0b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02dbfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f02dc15ed40 0x7f02dc0b2360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57572/0 (socket says 172.21.15.37:57572) 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02dbfff640 1 -- 172.21.15.37:0/896232035 learned_addr learned my addr 172.21.15.37:0/896232035 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02e0940640 1 --2- 172.21.15.37:0/896232035 >> v2:172.21.15.203:3300/0 conn(0x7f02dc15f120 0x7f02dc0cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02db7fe640 1 -- 172.21.15.37:0/896232035 >> v2:172.21.15.203:3300/0 conn(0x7f02dc15f120 msgr2=0x7f02dc0cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02db7fe640 1 --2- 172.21.15.37:0/896232035 >> v2:172.21.15.203:3300/0 conn(0x7f02dc15f120 0x7f02dc0cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02db7fe640 1 -- 172.21.15.37:0/896232035 >> v1:172.21.15.37:6789/0 conn(0x7f02dc15ed40 legacy=0x7f02dc0b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02db7fe640 1 -- 172.21.15.37:0/896232035 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02dc0d0220 con 0x7f02dc0b2a70 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02daffd640 1 -- 172.21.15.37:0/896232035 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796164924 0 0) 0x7f02dc0b1000 con 0x7f02dc15ed40 2024-04-25T08:53:51.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.292+0000 7f02e0940640 1 --2- 172.21.15.37:0/896232035 >> v2:172.21.15.203:3300/0 conn(0x7f02dc15f120 0x7f02dc0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02db7fe640 1 --2- 172.21.15.37:0/896232035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02dc0b2a70 0x7f02dc0b1aa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f02dc073450 tx=0x7f02d001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2ce8e74c2c9594 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02daffd640 1 -- 172.21.15.37:0/896232035 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d00a7070 con 0x7f02dc0b2a70 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02daffd640 1 -- 172.21.15.37:0/896232035 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d00b0070 con 0x7f02dc0b2a70 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02daffd640 1 -- 172.21.15.37:0/896232035 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d00ab890 con 0x7f02dc0b2a70 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 -- 172.21.15.37:0/896232035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02dc0b2a70 msgr2=0x7f02dc0b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 --2- 172.21.15.37:0/896232035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02dc0b2a70 0x7f02dc0b1aa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f02dc073450 tx=0x7f02d001c450 comp rx=0 tx=0).stop 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 -- 172.21.15.37:0/896232035 shutdown_connections 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 --2- 172.21.15.37:0/896232035 >> v2:172.21.15.203:3300/0 conn(0x7f02dc15f120 0x7f02dc0cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 --2- 172.21.15.37:0/896232035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02dc0b2a70 0x7f02dc0b1aa0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 -- 172.21.15.37:0/896232035 >> 172.21.15.37:0/896232035 conn(0x7f02dc0b3020 msgr2=0x7f02dc0b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 -- 172.21.15.37:0/896232035 shutdown_connections 2024-04-25T08:53:51.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 -- 172.21.15.37:0/896232035 wait complete. 2024-04-25T08:53:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 Processor -- start 2024-04-25T08:53:51.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 -- start start 2024-04-25T08:53:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02dc0b2a70 0x7f02dc0c40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02dc15ed40 0x7f02dc143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02dc15f120 0x7f02dc0c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc168780 con 0x7f02dc15f120 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc168600 con 0x7f02dc0b2a70 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e23a3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc168900 con 0x7f02dc15ed40 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e0940640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02dc15f120 0x7f02dc0c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e0940640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02dc15f120 0x7f02dc0c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58418/0 (socket says 172.21.15.37:58418) 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02e0940640 1 -- 172.21.15.37:0/2172463346 learned_addr learned my addr 172.21.15.37:0/2172463346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02dbfff640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02dc0b2a70 0x7f02dc0c40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.296+0000 7f02db7fe640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02dc15ed40 0x7f02dc143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02e0940640 1 -- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02dc15ed40 msgr2=0x7f02dc143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02e0940640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02dc15ed40 0x7f02dc143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02e0940640 1 -- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02dc0b2a70 msgr2=0x7f02dc0c40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02e0940640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02dc0b2a70 0x7f02dc0c40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02e0940640 1 -- 172.21.15.37:0/2172463346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02dc1441c0 con 0x7f02dc15f120 2024-04-25T08:53:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02dbfff640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02dc0b2a70 0x7f02dc0c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02db7fe640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02dc15ed40 0x7f02dc143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02e0940640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02dc15f120 0x7f02dc0c7ae0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f02d400cd50 tx=0x7f02d400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02d8ff9640 1 -- 172.21.15.37:0/2172463346 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d400f680 con 0x7f02dc15f120 2024-04-25T08:53:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02d8ff9640 1 -- 172.21.15.37:0/2172463346 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d400f840 con 0x7f02dc15f120 2024-04-25T08:53:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02e23a3640 1 -- 172.21.15.37:0/2172463346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02dc0c6a40 con 0x7f02dc15f120 2024-04-25T08:53:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02d8ff9640 1 -- 172.21.15.37:0/2172463346 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d400fc30 con 0x7f02dc15f120 2024-04-25T08:53:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02e23a3640 1 -- 172.21.15.37:0/2172463346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02dc0c7000 con 0x7f02dc15f120 2024-04-25T08:53:51.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02e23a3640 1 -- 172.21.15.37:0/2172463346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02dc04efa0 con 0x7f02dc15f120 2024-04-25T08:53:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02d8ff9640 1 -- 172.21.15.37:0/2172463346 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02d40213f0 con 0x7f02dc15f120 2024-04-25T08:53:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02d8ff9640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f90 0x7f02c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02d8ff9640 1 -- 172.21.15.37:0/2172463346 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f02d4085b20 con 0x7f02dc15f120 2024-04-25T08:53:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02dbfff640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f90 0x7f02c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.300+0000 7f02dbfff640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f90 0x7f02c8069450 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f02cc000f70 tx=0x7f02cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:51.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.304+0000 7f02d8ff9640 1 -- 172.21.15.37:0/2172463346 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f02d4014030 con 0x7f02dc15f120 2024-04-25T08:53:51.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.512+0000 7f02e23a3640 1 -- 172.21.15.37:0/2172463346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f02dc0c4c70 con 0x7f02dc15f120 2024-04-25T08:53:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.516+0000 7f02d8ff9640 1 -- 172.21.15.37:0/2172463346 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f02d4056d80 con 0x7f02dc15f120 2024-04-25T08:53:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 -- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f90 msgr2=0x7f02c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f90 0x7f02c8069450 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f02cc000f70 tx=0x7f02cc009040 comp rx=0 tx=0).stop 2024-04-25T08:53:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 -- 172.21.15.37:0/2172463346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02dc15f120 msgr2=0x7f02dc0c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02dc15f120 0x7f02dc0c7ae0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f02d400cd50 tx=0x7f02d400a650 comp rx=0 tx=0).stop 2024-04-25T08:53:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 -- 172.21.15.37:0/2172463346 shutdown_connections 2024-04-25T08:53:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f90 0x7f02c8069450 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02dc15f120 0x7f02dc0c7ae0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02dc15ed40 0x7f02dc143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 --2- 172.21.15.37:0/2172463346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02dc0b2a70 0x7f02dc0c40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 -- 172.21.15.37:0/2172463346 >> 172.21.15.37:0/2172463346 conn(0x7f02dc0b3020 msgr2=0x7f02dc0ae310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 -- 172.21.15.37:0/2172463346 shutdown_connections 2024-04-25T08:53:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.520+0000 7f02e23a3640 1 -- 172.21.15.37:0/2172463346 wait complete. 2024-04-25T08:53:51.529 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T08:53:51.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb72d901640 1 Processor -- start 2024-04-25T08:53:51.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb72d901640 1 -- start start 2024-04-25T08:53:51.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb72d901640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb728144ca0 0x7fb72806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb72d901640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb72806ec60 0x7fb728142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb72d901640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7280ccc70 con 0x7fb7280ce510 2024-04-25T08:53:51.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb72d901640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7280cc2b0 con 0x7fb728144ca0 2024-04-25T08:53:51.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb72d901640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb72806e290 con 0x7fb72806ec60 2024-04-25T08:53:51.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb7267fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb728144ca0 0x7fb72806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb726ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb7280ce510 0x7fb7280ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57574/0 (socket says 172.21.15.37:57574) 2024-04-25T08:53:51.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb726ffd640 1 -- 172.21.15.37:0/1427136960 learned_addr learned my addr 172.21.15.37:0/1427136960 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:51.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb7277fe640 1 --2- 172.21.15.37:0/1427136960 >> v2:172.21.15.203:3300/0 conn(0x7fb72806ec60 0x7fb728142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb725ffb640 1 -- 172.21.15.37:0/1427136960 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3097995664 0 0) 0x7fb7280ccc70 con 0x7fb7280ce510 2024-04-25T08:53:51.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb725ffb640 1 -- 172.21.15.37:0/1427136960 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb710003530 con 0x7fb7280ce510 2024-04-25T08:53:51.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb7267fc640 1 -- 172.21.15.37:0/1427136960 >> v2:172.21.15.203:3300/0 conn(0x7fb72806ec60 msgr2=0x7fb728142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:51.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb7267fc640 1 --2- 172.21.15.37:0/1427136960 >> v2:172.21.15.203:3300/0 conn(0x7fb72806ec60 0x7fb728142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb7267fc640 1 -- 172.21.15.37:0/1427136960 >> v1:172.21.15.37:6789/0 conn(0x7fb7280ce510 legacy=0x7fb7280ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb7267fc640 1 -- 172.21.15.37:0/1427136960 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72806dc40 con 0x7fb728144ca0 2024-04-25T08:53:51.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.660+0000 7fb7277fe640 1 --2- 172.21.15.37:0/1427136960 >> v2:172.21.15.203:3300/0 conn(0x7fb72806ec60 0x7fb728142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:51.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb7267fc640 1 --2- 172.21.15.37:0/1427136960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb728144ca0 0x7fb72806e720 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb7280b48f0 tx=0x7fb71c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdee631b03a2da87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb725ffb640 1 -- 172.21.15.37:0/1427136960 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0b5070 con 0x7fb728144ca0 2024-04-25T08:53:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb725ffb640 1 -- 172.21.15.37:0/1427136960 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c0a8970 con 0x7fb728144ca0 2024-04-25T08:53:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb725ffb640 1 -- 172.21.15.37:0/1427136960 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0a8d40 con 0x7fb728144ca0 2024-04-25T08:53:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb72d901640 1 -- 172.21.15.37:0/1427136960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb728144ca0 msgr2=0x7fb72806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb72d901640 1 --2- 172.21.15.37:0/1427136960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb728144ca0 0x7fb72806e720 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb7280b48f0 tx=0x7fb71c01ec70 comp rx=0 tx=0).stop 2024-04-25T08:53:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb72d901640 1 -- 172.21.15.37:0/1427136960 shutdown_connections 2024-04-25T08:53:51.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb72d901640 1 --2- 172.21.15.37:0/1427136960 >> v2:172.21.15.203:3300/0 conn(0x7fb72806ec60 0x7fb728142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb72d901640 1 --2- 172.21.15.37:0/1427136960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb728144ca0 0x7fb72806e720 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb72d901640 1 -- 172.21.15.37:0/1427136960 >> 172.21.15.37:0/1427136960 conn(0x7fb7280b11a0 msgr2=0x7fb7280b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb72d901640 1 -- 172.21.15.37:0/1427136960 shutdown_connections 2024-04-25T08:53:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.664+0000 7fb72d901640 1 -- 172.21.15.37:0/1427136960 wait complete. 2024-04-25T08:53:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb72d901640 1 Processor -- start 2024-04-25T08:53:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb72d901640 1 -- start start 2024-04-25T08:53:51.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb72d901640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb72806ec60 0x7fb7280c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb72d901640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7280ce510 0x7fb7281f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb72d901640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb728144ca0 0x7fb7281eb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb72d901640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7281451f0 con 0x7fb72806ec60 2024-04-25T08:53:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb72d901640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7280cc900 con 0x7fb7280ce510 2024-04-25T08:53:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb72d901640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb728145370 con 0x7fb728144ca0 2024-04-25T08:53:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb726ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb72806ec60 0x7fb7280c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb726ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb72806ec60 0x7fb7280c4e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58438/0 (socket says 172.21.15.37:58438) 2024-04-25T08:53:51.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb726ffd640 1 -- 172.21.15.37:0/1501549852 learned_addr learned my addr 172.21.15.37:0/1501549852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:51.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb7267fc640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7280ce510 0x7fb7281f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb7277fe640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb728144ca0 0x7fb7281eb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb726ffd640 1 -- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb728144ca0 msgr2=0x7fb7281eb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb726ffd640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb728144ca0 0x7fb7281eb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb726ffd640 1 -- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7280ce510 msgr2=0x7fb7281f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb726ffd640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7280ce510 0x7fb7281f1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb726ffd640 1 -- 172.21.15.37:0/1501549852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7281eb7c0 con 0x7fb72806ec60 2024-04-25T08:53:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb7267fc640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7280ce510 0x7fb7281f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb726ffd640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb72806ec60 0x7fb7280c4e20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb71800c450 tx=0x7fb71800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb70ffff640 1 -- 172.21.15.37:0/1501549852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718017070 con 0x7fb72806ec60 2024-04-25T08:53:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb70ffff640 1 -- 172.21.15.37:0/1501549852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb718005cf0 con 0x7fb72806ec60 2024-04-25T08:53:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb70ffff640 1 -- 172.21.15.37:0/1501549852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb718007290 con 0x7fb72806ec60 2024-04-25T08:53:51.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb72d901640 1 -- 172.21.15.37:0/1501549852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7280c5520 con 0x7fb72806ec60 2024-04-25T08:53:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb72d901640 1 -- 172.21.15.37:0/1501549852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7280c5ae0 con 0x7fb72806ec60 2024-04-25T08:53:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.668+0000 7fb72d901640 1 -- 172.21.15.37:0/1501549852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb72804efa0 con 0x7fb72806ec60 2024-04-25T08:53:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.672+0000 7fb70ffff640 1 -- 172.21.15.37:0/1501549852 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb71801a070 con 0x7fb72806ec60 2024-04-25T08:53:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.672+0000 7fb70ffff640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb714066f60 0x7fb714069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.672+0000 7fb70ffff640 1 -- 172.21.15.37:0/1501549852 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb7180841f0 con 0x7fb72806ec60 2024-04-25T08:53:51.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.672+0000 7fb7267fc640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb714066f60 0x7fb714069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:51.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.672+0000 7fb7267fc640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb714066f60 0x7fb714069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0b2000 tx=0x7fb71c0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:51.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.676+0000 7fb70ffff640 1 -- 172.21.15.37:0/1501549852 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb718055450 con 0x7fb72806ec60 2024-04-25T08:53:51.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.896+0000 7fb72d901640 1 -- 172.21.15.37:0/1501549852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fb7280aeeb0 con 0x7fb72806ec60 2024-04-25T08:53:51.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.900+0000 7fb70ffff640 1 -- 172.21.15.37:0/1501549852 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb718054d50 con 0x7fb72806ec60 2024-04-25T08:53:51.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 -- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb714066f60 msgr2=0x7fb714069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb714066f60 0x7fb714069420 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0b2000 tx=0x7fb71c0aa070 comp rx=0 tx=0).stop 2024-04-25T08:53:51.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 -- 172.21.15.37:0/1501549852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb72806ec60 msgr2=0x7fb7280c4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:51.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb72806ec60 0x7fb7280c4e20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb71800c450 tx=0x7fb71800c910 comp rx=0 tx=0).stop 2024-04-25T08:53:51.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 -- 172.21.15.37:0/1501549852 shutdown_connections 2024-04-25T08:53:51.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb714066f60 0x7fb714069420 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb728144ca0 0x7fb7281eb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7280ce510 0x7fb7281f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 --2- 172.21.15.37:0/1501549852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb72806ec60 0x7fb7280c4e20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:51.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 -- 172.21.15.37:0/1501549852 >> 172.21.15.37:0/1501549852 conn(0x7fb7280b11a0 msgr2=0x7fb728071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:51.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 -- 172.21.15.37:0/1501549852 shutdown_connections 2024-04-25T08:53:51.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:51.904+0000 7fb72d901640 1 -- 172.21.15.37:0/1501549852 wait complete. 2024-04-25T08:53:51.914 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T08:53:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7bdff640 1 Processor -- start 2024-04-25T08:53:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7bdff640 1 -- start start 2024-04-25T08:53:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7bdff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2e7406c3b0 0x7f2e74164d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7bdff640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e74143b70 con 0x7f2e74143790 2024-04-25T08:53:52.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7bdff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e7406e170 con 0x7f2e7415da40 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7bdff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e7406e450 con 0x7f2e7406c3b0 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e79b9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2e74143790 0x7f2e7406d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57584/0 (socket says 172.21.15.37:57584) 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e79b9b640 1 -- 172.21.15.37:0/1491787673 learned_addr learned my addr 172.21.15.37:0/1491787673 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7a39c640 1 --2- 172.21.15.37:0/1491787673 >> v2:172.21.15.203:3300/0 conn(0x7f2e7406c3b0 0x7f2e74164d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3476095467 0 0) 0x7f2e74143b70 con 0x7f2e74143790 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e60003530 con 0x7f2e74143790 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 818700539 0 0) 0x7f2e60003530 con 0x7f2e74143790 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 >> v2:172.21.15.203:3300/0 conn(0x7f2e7406c3b0 msgr2=0x7f2e74164d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 --2- 172.21.15.37:0/1491787673 >> v2:172.21.15.203:3300/0 conn(0x7f2e7406c3b0 0x7f2e74164d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 >> v1:172.21.15.203:6789/0 conn(0x7f2e7415da40 legacy=0x7f2e7406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7406e730 con 0x7f2e74143790 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f2e70001680 con 0x7f2e74143790 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 >> v1:172.21.15.37:6789/0 conn(0x7f2e74143790 legacy=0x7f2e7406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e60003a30 0x7f2e60003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e60004810 0x7f2e60004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 0x7f2e6000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7406e450 con 0x7f2e600051d0 2024-04-25T08:53:52.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6000c100 con 0x7f2e60004810 2024-04-25T08:53:52.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e6000c300 con 0x7f2e60003a30 2024-04-25T08:53:52.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7a39c640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 0x7f2e6000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7939a640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e60004810 0x7f2e60004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e79b9b640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e60003a30 0x7f2e60003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7a39c640 1 -- 172.21.15.37:0/1491787673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e60003a30 msgr2=0x7f2e60003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:52.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7a39c640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e60003a30 0x7f2e60003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7a39c640 1 -- 172.21.15.37:0/1491787673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e60004810 msgr2=0x7f2e60004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:52.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7a39c640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e60004810 0x7f2e60004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7a39c640 1 -- 172.21.15.37:0/1491787673 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e60003530 con 0x7f2e600051d0 2024-04-25T08:53:52.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e79b9b640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e60003a30 0x7f2e60003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:52.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.048+0000 7f2e7939a640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e60004810 0x7f2e60004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:52.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e7a39c640 1 -- 172.21.15.37:0/1491787673 reap_dead start 2024-04-25T08:53:52.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e7a39c640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 0x7f2e6000b9d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009f90 tx=0x7f2e6c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d861289243dc204a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:52.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c009c10 con 0x7f2e600051d0 2024-04-25T08:53:52.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6c0a5c90 con 0x7f2e600051d0 2024-04-25T08:53:52.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e78b99640 1 -- 172.21.15.37:0/1491787673 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c003be0 con 0x7f2e600051d0 2024-04-25T08:53:52.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1491787673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 msgr2=0x7f2e6000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e7bdff640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 0x7f2e6000b9d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c009f90 tx=0x7f2e6c01cc80 comp rx=0 tx=0).stop 2024-04-25T08:53:52.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1491787673 shutdown_connections 2024-04-25T08:53:52.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e7bdff640 1 --2- 172.21.15.37:0/1491787673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 0x7f2e6000b9d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1491787673 >> 172.21.15.37:0/1491787673 conn(0x7f2e74071810 msgr2=0x7f2e74071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:52.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1491787673 shutdown_connections 2024-04-25T08:53:52.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1491787673 wait complete. 2024-04-25T08:53:52.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e7bdff640 1 Processor -- start 2024-04-25T08:53:52.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.052+0000 7f2e7bdff640 1 -- start start 2024-04-25T08:53:52.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7bdff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 0x7f2e741595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7bdff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e7413e4b0 0x7f2e7413cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7bdff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e7413acc0 0x7f2e740af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7bdff640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7406be20 con 0x7f2e600051d0 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7bdff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7406bca0 con 0x7f2e7413acc0 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7bdff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e7406bfa0 con 0x7f2e7413e4b0 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e79b9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 0x7f2e741595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e79b9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 0x7f2e741595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58454/0 (socket says 172.21.15.37:58454) 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e79b9b640 1 -- 172.21.15.37:0/1913401798 learned_addr learned my addr 172.21.15.37:0/1913401798 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7939a640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e7413e4b0 0x7f2e7413cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7a39c640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e7413acc0 0x7f2e740af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e79b9b640 1 -- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e7413e4b0 msgr2=0x7f2e7413cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e79b9b640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e7413e4b0 0x7f2e7413cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e79b9b640 1 -- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e7413acc0 msgr2=0x7f2e740af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e79b9b640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e7413acc0 0x7f2e740af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e79b9b640 1 -- 172.21.15.37:0/1913401798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7413b150 con 0x7f2e600051d0 2024-04-25T08:53:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7939a640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e7413e4b0 0x7f2e7413cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e79b9b640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 0x7f2e741595f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000d830 tx=0x7f2e7000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e6a7fc640 1 -- 172.21.15.37:0/1913401798 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70019070 con 0x7f2e600051d0 2024-04-25T08:53:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e6a7fc640 1 -- 172.21.15.37:0/1913401798 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e700074e0 con 0x7f2e600051d0 2024-04-25T08:53:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1913401798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e741e2df0 con 0x7f2e600051d0 2024-04-25T08:53:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e6a7fc640 1 -- 172.21.15.37:0/1913401798 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e70004a10 con 0x7f2e600051d0 2024-04-25T08:53:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1913401798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e7413d6f0 con 0x7f2e600051d0 2024-04-25T08:53:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1913401798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e74072670 con 0x7f2e600051d0 2024-04-25T08:53:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e6a7fc640 1 -- 172.21.15.37:0/1913401798 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e7001c070 con 0x7f2e600051d0 2024-04-25T08:53:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e6a7fc640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e58066f60 0x7f2e58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e6a7fc640 1 -- 172.21.15.37:0/1913401798 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2e700848d0 con 0x7f2e600051d0 2024-04-25T08:53:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.056+0000 7f2e7939a640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e58066f60 0x7f2e58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.060+0000 7f2e7939a640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e58066f60 0x7f2e58069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2e7413e8b0 tx=0x7f2e64009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.060+0000 7f2e6a7fc640 1 -- 172.21.15.37:0/1913401798 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e70055b60 con 0x7f2e600051d0 2024-04-25T08:53:52.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.280+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1913401798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f2e7415a7a0 con 0x7f2e600051d0 2024-04-25T08:53:52.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e6a7fc640 1 -- 172.21.15.37:0/1913401798 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f2e70055460 con 0x7f2e600051d0 2024-04-25T08:53:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e58066f60 msgr2=0x7f2e58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e58066f60 0x7f2e58069420 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2e7413e8b0 tx=0x7f2e64009040 comp rx=0 tx=0).stop 2024-04-25T08:53:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1913401798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 msgr2=0x7f2e741595f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 0x7f2e741595f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2e7000d830 tx=0x7f2e7000dcf0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1913401798 shutdown_connections 2024-04-25T08:53:52.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e58066f60 0x7f2e58069420 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e7413acc0 0x7f2e740af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e7413e4b0 0x7f2e7413cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 --2- 172.21.15.37:0/1913401798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e600051d0 0x7f2e741595f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1913401798 >> 172.21.15.37:0/1913401798 conn(0x7f2e74071810 msgr2=0x7f2e7415acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:52.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1913401798 shutdown_connections 2024-04-25T08:53:52.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.284+0000 7f2e7bdff640 1 -- 172.21.15.37:0/1913401798 wait complete. 2024-04-25T08:53:52.297 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:52.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.428+0000 7f826af0d640 1 Processor -- start 2024-04-25T08:53:52.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.428+0000 7f826af0d640 1 -- start start 2024-04-25T08:53:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f826af0d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f826415da80 0x7f82640d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f826af0d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f82640d07f0 0x7f826406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f826af0d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826406be60 con 0x7f82640d1cd0 2024-04-25T08:53:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f826af0d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826406c440 con 0x7f82640d07f0 2024-04-25T08:53:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f826af0d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826415de60 con 0x7f826415da80 2024-04-25T08:53:52.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f8268ca9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f826415da80 0x7f82640d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f82694aa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f82640d07f0 0x7f826406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f8268ca9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f826415da80 0x7f82640d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:54646/0 (socket says 172.21.15.37:54646) 2024-04-25T08:53:52.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f8268ca9640 1 -- 172.21.15.37:0/1702524892 learned_addr learned my addr 172.21.15.37:0/1702524892 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:52.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f8263fff640 1 --1- 172.21.15.37:0/1702524892 >> v1:172.21.15.203:6789/0 conn(0x7f82640d1cd0 0x7f82640d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:58094/0 (socket says 172.21.15.37:58094) 2024-04-25T08:53:52.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f8268ca9640 1 -- 172.21.15.37:0/1702524892 >> v2:172.21.15.203:3300/0 conn(0x7f82640d07f0 msgr2=0x7f826406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:52.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f8268ca9640 1 --2- 172.21.15.37:0/1702524892 >> v2:172.21.15.203:3300/0 conn(0x7f82640d07f0 0x7f826406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f8268ca9640 1 -- 172.21.15.37:0/1702524892 >> v1:172.21.15.203:6789/0 conn(0x7f82640d1cd0 legacy=0x7f82640d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f8268ca9640 1 -- 172.21.15.37:0/1702524892 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640d2450 con 0x7f826415da80 2024-04-25T08:53:52.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f82694aa640 1 --2- 172.21.15.37:0/1702524892 >> v2:172.21.15.203:3300/0 conn(0x7f82640d07f0 0x7f826406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.432+0000 7f8268ca9640 1 --2- 172.21.15.37:0/1702524892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f826415da80 0x7f82640d4230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8254009e50 tx=0x7f825401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0e5b2aaccd63b23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f82637fe640 1 -- 172.21.15.37:0/1702524892 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82540a5800 con 0x7f826415da80 2024-04-25T08:53:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f82637fe640 1 -- 172.21.15.37:0/1702524892 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82540a59a0 con 0x7f826415da80 2024-04-25T08:53:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f82637fe640 1 -- 172.21.15.37:0/1702524892 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82540a5d70 con 0x7f826415da80 2024-04-25T08:53:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 -- 172.21.15.37:0/1702524892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f826415da80 msgr2=0x7f82640d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 --2- 172.21.15.37:0/1702524892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f826415da80 0x7f82640d4230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8254009e50 tx=0x7f825401ca00 comp rx=0 tx=0).stop 2024-04-25T08:53:52.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 -- 172.21.15.37:0/1702524892 shutdown_connections 2024-04-25T08:53:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 --2- 172.21.15.37:0/1702524892 >> v2:172.21.15.203:3300/0 conn(0x7f82640d07f0 0x7f826406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 --2- 172.21.15.37:0/1702524892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f826415da80 0x7f82640d4230 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 -- 172.21.15.37:0/1702524892 >> 172.21.15.37:0/1702524892 conn(0x7f826408b000 msgr2=0x7f82640d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 -- 172.21.15.37:0/1702524892 shutdown_connections 2024-04-25T08:53:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 -- 172.21.15.37:0/1702524892 wait complete. 2024-04-25T08:53:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 Processor -- start 2024-04-25T08:53:52.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 -- start start 2024-04-25T08:53:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82640d07f0 0x7f8264157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82640d1cd0 0x7f82640b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826415da80 0x7f82640b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826406d7a0 con 0x7f826415da80 2024-04-25T08:53:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826406d620 con 0x7f82640d1cd0 2024-04-25T08:53:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f826af0d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826406d920 con 0x7f82640d07f0 2024-04-25T08:53:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f82694aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826415da80 0x7f82640b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f82694aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826415da80 0x7f82640b7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58458/0 (socket says 172.21.15.37:58458) 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f82694aa640 1 -- 172.21.15.37:0/66770036 learned_addr learned my addr 172.21.15.37:0/66770036 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.436+0000 7f8263fff640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82640d1cd0 0x7f82640b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f8268ca9640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82640d07f0 0x7f8264157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82694aa640 1 -- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82640d07f0 msgr2=0x7f8264157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82694aa640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82640d07f0 0x7f8264157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82694aa640 1 -- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82640d1cd0 msgr2=0x7f82640b6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82694aa640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82640d1cd0 0x7f82640b6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82694aa640 1 -- 172.21.15.37:0/66770036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82640b77c0 con 0x7f826415da80 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82694aa640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826415da80 0x7f82640b7140 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f824c00e920 tx=0x7f824c00ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82617fa640 1 -- 172.21.15.37:0/66770036 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c017070 con 0x7f826415da80 2024-04-25T08:53:52.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82617fa640 1 -- 172.21.15.37:0/66770036 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824c00ca00 con 0x7f826415da80 2024-04-25T08:53:52.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f826af0d640 1 -- 172.21.15.37:0/66770036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82640b7aa0 con 0x7f826415da80 2024-04-25T08:53:52.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82617fa640 1 -- 172.21.15.37:0/66770036 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824c00cdd0 con 0x7f826415da80 2024-04-25T08:53:52.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f826af0d640 1 -- 172.21.15.37:0/66770036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82640b8340 con 0x7f826415da80 2024-04-25T08:53:52.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f826af0d640 1 -- 172.21.15.37:0/66770036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826404efa0 con 0x7f826415da80 2024-04-25T08:53:52.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82617fa640 1 -- 172.21.15.37:0/66770036 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f824c01a070 con 0x7f826415da80 2024-04-25T08:53:52.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82617fa640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 0x7f8244069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f82617fa640 1 -- 172.21.15.37:0/66770036 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f824c0850d0 con 0x7f826415da80 2024-04-25T08:53:52.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f8268ca9640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 0x7f8244069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.440+0000 7f8268ca9640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 0x7f8244069420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f82540ae000 tx=0x7f82540a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:52.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.444+0000 7f82617fa640 1 -- 172.21.15.37:0/66770036 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f824c055ce0 con 0x7f826415da80 2024-04-25T08:53:52.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.604+0000 7f826af0d640 1 -- 172.21.15.37:0/66770036 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82640b7c70 con 0x7f826415da80 2024-04-25T08:53:52.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.604+0000 7f82617fa640 1 -- 172.21.15.37:0/66770036 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+13128 (secure 0 0 0) 0x7f824c081360 con 0x7f826415da80 2024-04-25T08:53:52.599 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:52.599 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":37,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:49.650861+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 -- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 msgr2=0x7f8244069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 0x7f8244069420 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f82540ae000 tx=0x7f82540a7070 comp rx=0 tx=0).stop 2024-04-25T08:53:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 -- 172.21.15.37:0/66770036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826415da80 msgr2=0x7f82640b7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826415da80 0x7f82640b7140 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f824c00e920 tx=0x7f824c00ede0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 -- 172.21.15.37:0/66770036 shutdown_connections 2024-04-25T08:53:52.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8244066f60 0x7f8244069420 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f826415da80 0x7f82640b7140 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82640d1cd0 0x7f82640b6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 --2- 172.21.15.37:0/66770036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f82640d07f0 0x7f8264157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 -- 172.21.15.37:0/66770036 >> 172.21.15.37:0/66770036 conn(0x7f826408b000 msgr2=0x7f826415eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:52.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 -- 172.21.15.37:0/66770036 shutdown_connections 2024-04-25T08:53:52.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.608+0000 7f826af0d640 1 -- 172.21.15.37:0/66770036 wait complete. 2024-04-25T08:53:52.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:53:52.619 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:53:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.752+0000 7fcaef3b8640 1 Processor -- start 2024-04-25T08:53:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.752+0000 7fcaef3b8640 1 -- start start 2024-04-25T08:53:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaef3b8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcae815b6b0 0x7fcae80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaef3b8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcae8073010 0x7fcae80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaef3b8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcae80cbd20 con 0x7fcae80cf590 2024-04-25T08:53:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaef3b8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae815ba90 con 0x7fcae8073010 2024-04-25T08:53:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaef3b8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcae80cc000 con 0x7fcae815b6b0 2024-04-25T08:53:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaed955640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcae8073010 0x7fcae80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaed955640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcae8073010 0x7fcae80cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58472/0 (socket says 172.21.15.37:58472) 2024-04-25T08:53:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaed955640 1 -- 172.21.15.37:0/2399686798 learned_addr learned my addr 172.21.15.37:0/2399686798 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaed154640 1 --2- 172.21.15.37:0/2399686798 >> v2:172.21.15.203:3300/0 conn(0x7fcae815b6b0 0x7fcae80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaed955640 1 -- 172.21.15.37:0/2399686798 >> v2:172.21.15.203:3300/0 conn(0x7fcae815b6b0 msgr2=0x7fcae80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:52.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaed955640 1 --2- 172.21.15.37:0/2399686798 >> v2:172.21.15.203:3300/0 conn(0x7fcae815b6b0 0x7fcae80ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaed955640 1 -- 172.21.15.37:0/2399686798 >> v1:172.21.15.203:6789/0 conn(0x7fcae80cf590 legacy=0x7fcae815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaed955640 1 -- 172.21.15.37:0/2399686798 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae80cc2e0 con 0x7fcae8073010 2024-04-25T08:53:52.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.756+0000 7fcaed955640 1 --2- 172.21.15.37:0/2399686798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcae8073010 0x7fcae80cfa20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcae80d0180 tx=0x7fcae401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ed5d7fa5769ffd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:52.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcadffff640 1 -- 172.21.15.37:0/2399686798 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae40b0070 con 0x7fcae8073010 2024-04-25T08:53:52.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcadffff640 1 -- 172.21.15.37:0/2399686798 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae40ab600 con 0x7fcae8073010 2024-04-25T08:53:52.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcadffff640 1 -- 172.21.15.37:0/2399686798 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae40ab9d0 con 0x7fcae8073010 2024-04-25T08:53:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 -- 172.21.15.37:0/2399686798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcae8073010 msgr2=0x7fcae80cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 --2- 172.21.15.37:0/2399686798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcae8073010 0x7fcae80cfa20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fcae80d0180 tx=0x7fcae401c520 comp rx=0 tx=0).stop 2024-04-25T08:53:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 -- 172.21.15.37:0/2399686798 shutdown_connections 2024-04-25T08:53:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 --2- 172.21.15.37:0/2399686798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcae8073010 0x7fcae80cfa20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 --2- 172.21.15.37:0/2399686798 >> v2:172.21.15.203:3300/0 conn(0x7fcae815b6b0 0x7fcae80ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 -- 172.21.15.37:0/2399686798 >> 172.21.15.37:0/2399686798 conn(0x7fcae806ae60 msgr2=0x7fcae806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 -- 172.21.15.37:0/2399686798 shutdown_connections 2024-04-25T08:53:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 -- 172.21.15.37:0/2399686798 wait complete. 2024-04-25T08:53:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 Processor -- start 2024-04-25T08:53:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 -- start start 2024-04-25T08:53:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae8073010 0x7fcae80c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae80cf590 0x7fcae80ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcae815b6b0 0x7fcae80ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80cadb0 con 0x7fcae815b6b0 2024-04-25T08:53:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80ca900 con 0x7fcae80cf590 2024-04-25T08:53:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaef3b8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcae80caf30 con 0x7fcae8073010 2024-04-25T08:53:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaed955640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcae815b6b0 0x7fcae80ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaed955640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcae815b6b0 0x7fcae80ac6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58486/0 (socket says 172.21.15.37:58486) 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaed955640 1 -- 172.21.15.37:0/1270200300 learned_addr learned my addr 172.21.15.37:0/1270200300 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaec953640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae80cf590 0x7fcae80ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.760+0000 7fcaed154640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae8073010 0x7fcae80c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaed955640 1 -- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae8073010 msgr2=0x7fcae80c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaed955640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae8073010 0x7fcae80c2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaed955640 1 -- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae80cf590 msgr2=0x7fcae80ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaed955640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae80cf590 0x7fcae80ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaed955640 1 -- 172.21.15.37:0/1270200300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae815c0f0 con 0x7fcae815b6b0 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaed955640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcae815b6b0 0x7fcae80ac6b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcae40ae040 tx=0x7fcae40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaddffb640 1 -- 172.21.15.37:0/1270200300 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae40bb070 con 0x7fcae815b6b0 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaddffb640 1 -- 172.21.15.37:0/1270200300 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae40b6430 con 0x7fcae815b6b0 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaddffb640 1 -- 172.21.15.37:0/1270200300 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae40b0040 con 0x7fcae815b6b0 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaef3b8640 1 -- 172.21.15.37:0/1270200300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae80adc30 con 0x7fcae815b6b0 2024-04-25T08:53:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaef3b8640 1 -- 172.21.15.37:0/1270200300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae80ae1f0 con 0x7fcae815b6b0 2024-04-25T08:53:52.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaef3b8640 1 -- 172.21.15.37:0/1270200300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae80ae3c0 con 0x7fcae815b6b0 2024-04-25T08:53:52.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaddffb640 1 -- 172.21.15.37:0/1270200300 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcae40be070 con 0x7fcae815b6b0 2024-04-25T08:53:52.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaddffb640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcacc066f90 0x7fcacc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:52.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaddffb640 1 -- 172.21.15.37:0/1270200300 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcae411e6a0 con 0x7fcae815b6b0 2024-04-25T08:53:52.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaed154640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcacc066f90 0x7fcacc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:52.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.764+0000 7fcaed154640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcacc066f90 0x7fcacc069450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcae80c2920 tx=0x7fcae000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:52.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.768+0000 7fcaddffb640 1 -- 172.21.15.37:0/1270200300 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcae4016110 con 0x7fcae815b6b0 2024-04-25T08:53:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 -- 172.21.15.37:0/1270200300 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcae804efa0 con 0x7fcae815b6b0 2024-04-25T08:53:52.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaddffb640 1 -- 172.21.15.37:0/1270200300 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcae411e030 con 0x7fcae815b6b0 2024-04-25T08:53:52.924 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 -- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcacc066f90 msgr2=0x7fcacc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcacc066f90 0x7fcacc069450 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcae80c2920 tx=0x7fcae000b040 comp rx=0 tx=0).stop 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 -- 172.21.15.37:0/1270200300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcae815b6b0 msgr2=0x7fcae80ac6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcae815b6b0 0x7fcae80ac6b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fcae40ae040 tx=0x7fcae40192c0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 -- 172.21.15.37:0/1270200300 shutdown_connections 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcacc066f90 0x7fcacc069450 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcae815b6b0 0x7fcae80ac6b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcae80cf590 0x7fcae80ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 --2- 172.21.15.37:0/1270200300 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcae8073010 0x7fcae80c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:52.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 -- 172.21.15.37:0/1270200300 >> 172.21.15.37:0/1270200300 conn(0x7fcae806ae60 msgr2=0x7fcae80b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:52.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 -- 172.21.15.37:0/1270200300 shutdown_connections 2024-04-25T08:53:52.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:52.932+0000 7fcaef3b8640 1 -- 172.21.15.37:0/1270200300 wait complete. 2024-04-25T08:53:52.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:53:53.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.076+0000 7f21425fa640 1 Processor -- start 2024-04-25T08:53:53.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.076+0000 7f21425fa640 1 -- start start 2024-04-25T08:53:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f21425fa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f213c0d07f0 0x7f213c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f21425fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f213c0d1cd0 0x7f213c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f21425fa640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f213c06be60 con 0x7f213c15da80 2024-04-25T08:53:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f21425fa640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f213c06c440 con 0x7f213c0d07f0 2024-04-25T08:53:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f21425fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c15de60 con 0x7f213c0d1cd0 2024-04-25T08:53:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f213b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f213c0d07f0 0x7f213c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f213b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f213c0d07f0 0x7f213c0d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58498/0 (socket says 172.21.15.37:58498) 2024-04-25T08:53:53.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f213b7fe640 1 -- 172.21.15.37:0/3138392242 learned_addr learned my addr 172.21.15.37:0/3138392242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:53.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f2140b97640 1 --2- 172.21.15.37:0/3138392242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f213c0d1cd0 0x7f213c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f213b7fe640 1 -- 172.21.15.37:0/3138392242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f213c0d1cd0 msgr2=0x7f213c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:53.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f213b7fe640 1 --2- 172.21.15.37:0/3138392242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f213c0d1cd0 0x7f213c06c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f213b7fe640 1 -- 172.21.15.37:0/3138392242 >> v1:172.21.15.203:6789/0 conn(0x7f213c15da80 legacy=0x7f213c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f213b7fe640 1 -- 172.21.15.37:0/3138392242 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c0d2450 con 0x7f213c0d07f0 2024-04-25T08:53:53.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f213b7fe640 1 --2- 172.21.15.37:0/3138392242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c0d07f0 0x7f213c0d4230 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2130006fd0 tx=0x7f213001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4772783f3aa04f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:53.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f213affd640 1 -- 172.21.15.37:0/3138392242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21300a7070 con 0x7f213c0d07f0 2024-04-25T08:53:53.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f213affd640 1 -- 172.21.15.37:0/3138392242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21300b0070 con 0x7f213c0d07f0 2024-04-25T08:53:53.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.080+0000 7f213affd640 1 -- 172.21.15.37:0/3138392242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21300ab890 con 0x7f213c0d07f0 2024-04-25T08:53:53.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 -- 172.21.15.37:0/3138392242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c0d07f0 msgr2=0x7f213c0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 --2- 172.21.15.37:0/3138392242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c0d07f0 0x7f213c0d4230 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2130006fd0 tx=0x7f213001c510 comp rx=0 tx=0).stop 2024-04-25T08:53:53.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 -- 172.21.15.37:0/3138392242 shutdown_connections 2024-04-25T08:53:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 --2- 172.21.15.37:0/3138392242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f213c0d1cd0 0x7f213c06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 --2- 172.21.15.37:0/3138392242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c0d07f0 0x7f213c0d4230 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 -- 172.21.15.37:0/3138392242 >> 172.21.15.37:0/3138392242 conn(0x7f213c08b000 msgr2=0x7f213c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 -- 172.21.15.37:0/3138392242 shutdown_connections 2024-04-25T08:53:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 -- 172.21.15.37:0/3138392242 wait complete. 2024-04-25T08:53:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 Processor -- start 2024-04-25T08:53:53.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 -- start start 2024-04-25T08:53:53.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f213c0d07f0 0x7f213c15c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c0d1cd0 0x7f213c154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f213c15da80 0x7f213c15e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c06d7a0 con 0x7f213c0d1cd0 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c06d620 con 0x7f213c0d07f0 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f213c06d920 con 0x7f213c15da80 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f213b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c0d1cd0 0x7f213c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f213b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c0d1cd0 0x7f213c154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58500/0 (socket says 172.21.15.37:58500) 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f213b7fe640 1 -- 172.21.15.37:0/347566450 learned_addr learned my addr 172.21.15.37:0/347566450 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f213bfff640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f213c0d07f0 0x7f213c15c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f2140b97640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f213c15da80 0x7f213c15e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f213b7fe640 1 -- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f213c15da80 msgr2=0x7f213c15e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f213b7fe640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f213c15da80 0x7f213c15e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f213b7fe640 1 -- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f213c0d07f0 msgr2=0x7f213c15c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f213b7fe640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f213c0d07f0 0x7f213c15c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f213b7fe640 1 -- 172.21.15.37:0/347566450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f213c156f20 con 0x7f213c0d1cd0 2024-04-25T08:53:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f213b7fe640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c0d1cd0 0x7f213c154060 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f213001abc0 tx=0x7f21300192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f2138ff9640 1 -- 172.21.15.37:0/347566450 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2130003c30 con 0x7f213c0d1cd0 2024-04-25T08:53:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f2138ff9640 1 -- 172.21.15.37:0/347566450 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2130003dd0 con 0x7f213c0d1cd0 2024-04-25T08:53:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 -- 172.21.15.37:0/347566450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c157200 con 0x7f213c0d1cd0 2024-04-25T08:53:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f2138ff9640 1 -- 172.21.15.37:0/347566450 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21300ab690 con 0x7f213c0d1cd0 2024-04-25T08:53:53.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.084+0000 7f21425fa640 1 -- 172.21.15.37:0/347566450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c0caaa0 con 0x7f213c0d1cd0 2024-04-25T08:53:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.088+0000 7f21425fa640 1 -- 172.21.15.37:0/347566450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213c04efa0 con 0x7f213c0d1cd0 2024-04-25T08:53:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.088+0000 7f2138ff9640 1 -- 172.21.15.37:0/347566450 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f21300ab830 con 0x7f213c0d1cd0 2024-04-25T08:53:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.088+0000 7f2138ff9640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2128066f60 0x7f2128069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.088+0000 7f2138ff9640 1 -- 172.21.15.37:0/347566450 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f213011f660 con 0x7f213c0d1cd0 2024-04-25T08:53:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.088+0000 7f213bfff640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2128066f60 0x7f2128069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.088+0000 7f213bfff640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2128066f60 0x7f2128069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f213c0b2da0 tx=0x7f212c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:53.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.092+0000 7f2138ff9640 1 -- 172.21.15.37:0/347566450 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2130016130 con 0x7f213c0d1cd0 2024-04-25T08:53:53.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.252+0000 7f21425fa640 1 -- 172.21.15.37:0/347566450 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f213c15a6a0 con 0x7f213c0d1cd0 2024-04-25T08:53:53.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f2138ff9640 1 -- 172.21.15.37:0/347566450 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v37) v1 ==== 84+0+65 (secure 0 0 0) 0x7f21300f0270 con 0x7f213c0d1cd0 2024-04-25T08:53:53.247 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:53.247 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T08:53:53.248 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:53:53.248 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T08:53:53.248 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T08:53:53.248 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:53:53.248 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T08:53:53.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 -- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2128066f60 msgr2=0x7f2128069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2128066f60 0x7f2128069420 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f213c0b2da0 tx=0x7f212c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:53.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 -- 172.21.15.37:0/347566450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c0d1cd0 msgr2=0x7f213c154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c0d1cd0 0x7f213c154060 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f213001abc0 tx=0x7f21300192b0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 -- 172.21.15.37:0/347566450 shutdown_connections 2024-04-25T08:53:53.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2128066f60 0x7f2128069420 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f213c15da80 0x7f213c15e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f213c0d1cd0 0x7f213c154060 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 --2- 172.21.15.37:0/347566450 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f213c0d07f0 0x7f213c15c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 -- 172.21.15.37:0/347566450 >> 172.21.15.37:0/347566450 conn(0x7f213c08b000 msgr2=0x7f213c0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:53.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 -- 172.21.15.37:0/347566450 shutdown_connections 2024-04-25T08:53:53.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.256+0000 7f21425fa640 1 -- 172.21.15.37:0/347566450 wait complete. 2024-04-25T08:53:53.266 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:53:53.267 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:53:53.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa5713ee640 1 Processor -- start 2024-04-25T08:53:53.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa5713ee640 1 -- start start 2024-04-25T08:53:53.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa5713ee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa56c0adc70 0x7fa56c06e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa5713ee640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa56c06e9f0 0x7fa56c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa5713ee640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa56c0b28c0 con 0x7fa56c0cfa50 2024-04-25T08:53:53.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa5713ee640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c0cfec0 con 0x7fa56c0adc70 2024-04-25T08:53:53.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa5713ee640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c15bb20 con 0x7fa56c06e9f0 2024-04-25T08:53:53.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa56affd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa56c0adc70 0x7fa56c06e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa56b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa56c0cfa50 0x7fa56c15f250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57592/0 (socket says 172.21.15.37:57592) 2024-04-25T08:53:53.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa56b7fe640 1 -- 172.21.15.37:0/3087213442 learned_addr learned my addr 172.21.15.37:0/3087213442 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:53.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa5627fc640 1 --2- 172.21.15.37:0/3087213442 >> v2:172.21.15.203:3300/0 conn(0x7fa56c06e9f0 0x7fa56c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa56a7fc640 1 -- 172.21.15.37:0/3087213442 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486277361 0 0) 0x7fa56c0b28c0 con 0x7fa56c0cfa50 2024-04-25T08:53:53.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.396+0000 7fa56a7fc640 1 -- 172.21.15.37:0/3087213442 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa558003530 con 0x7fa56c0cfa50 2024-04-25T08:53:53.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa56affd640 1 -- 172.21.15.37:0/3087213442 >> v2:172.21.15.203:3300/0 conn(0x7fa56c06e9f0 msgr2=0x7fa56c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:53.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa56affd640 1 --2- 172.21.15.37:0/3087213442 >> v2:172.21.15.203:3300/0 conn(0x7fa56c06e9f0 0x7fa56c0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa56affd640 1 -- 172.21.15.37:0/3087213442 >> v1:172.21.15.37:6789/0 conn(0x7fa56c0cfa50 legacy=0x7fa56c15f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa56affd640 1 -- 172.21.15.37:0/3087213442 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c0b2b60 con 0x7fa56c0adc70 2024-04-25T08:53:53.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa56affd640 1 --2- 172.21.15.37:0/3087213442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c0adc70 0x7fa56c06e4b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa56c06f460 tx=0x7fa55c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12781cd8cd7bb653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:53.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa56a7fc640 1 -- 172.21.15.37:0/3087213442 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0b0070 con 0x7fa56c0adc70 2024-04-25T08:53:53.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa56a7fc640 1 -- 172.21.15.37:0/3087213442 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c0ab600 con 0x7fa56c0adc70 2024-04-25T08:53:53.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa56a7fc640 1 -- 172.21.15.37:0/3087213442 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0ab9d0 con 0x7fa56c0adc70 2024-04-25T08:53:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa5713ee640 1 -- 172.21.15.37:0/3087213442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c0adc70 msgr2=0x7fa56c06e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa5713ee640 1 --2- 172.21.15.37:0/3087213442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c0adc70 0x7fa56c06e4b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa56c06f460 tx=0x7fa55c01c480 comp rx=0 tx=0).stop 2024-04-25T08:53:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa5713ee640 1 -- 172.21.15.37:0/3087213442 shutdown_connections 2024-04-25T08:53:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa5713ee640 1 --2- 172.21.15.37:0/3087213442 >> v2:172.21.15.203:3300/0 conn(0x7fa56c06e9f0 0x7fa56c0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa5713ee640 1 --2- 172.21.15.37:0/3087213442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c0adc70 0x7fa56c06e4b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa5713ee640 1 -- 172.21.15.37:0/3087213442 >> 172.21.15.37:0/3087213442 conn(0x7fa56c06ae60 msgr2=0x7fa56c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:53.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa5713ee640 1 -- 172.21.15.37:0/3087213442 shutdown_connections 2024-04-25T08:53:53.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.400+0000 7fa5713ee640 1 -- 172.21.15.37:0/3087213442 wait complete. 2024-04-25T08:53:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5713ee640 1 Processor -- start 2024-04-25T08:53:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5713ee640 1 -- start start 2024-04-25T08:53:53.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5713ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c06e9f0 0x7fa56c15b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5713ee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa56c0adc70 0x7fa56c0c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5713ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa56c0cfa50 0x7fa56c06dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5713ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0b36e0 con 0x7fa56c06e9f0 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5713ee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0b3c00 con 0x7fa56c0adc70 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5713ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0b1c70 con 0x7fa56c0cfa50 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa56affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c06e9f0 0x7fa56c15b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa56affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c06e9f0 0x7fa56c15b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58512/0 (socket says 172.21.15.37:58512) 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa56affd640 1 -- 172.21.15.37:0/3135291033 learned_addr learned my addr 172.21.15.37:0/3135291033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa56b7fe640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa56c0cfa50 0x7fa56c06dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5627fc640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa56c0adc70 0x7fa56c0c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa56affd640 1 -- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa56c0cfa50 msgr2=0x7fa56c06dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa56affd640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa56c0cfa50 0x7fa56c06dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa56affd640 1 -- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa56c0adc70 msgr2=0x7fa56c0c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa56affd640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa56c0adc70 0x7fa56c0c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa56affd640 1 -- 172.21.15.37:0/3135291033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c15be40 con 0x7fa56c06e9f0 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa56b7fe640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa56c0cfa50 0x7fa56c06dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5627fc640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa56c0adc70 0x7fa56c0c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa56affd640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c06e9f0 0x7fa56c15b480 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa55c01a0b0 tx=0x7fa55c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa563fff640 1 -- 172.21.15.37:0/3135291033 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0b0050 con 0x7fa56c06e9f0 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa563fff640 1 -- 172.21.15.37:0/3135291033 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c0bc070 con 0x7fa56c06e9f0 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5713ee640 1 -- 172.21.15.37:0/3135291033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa56c15c120 con 0x7fa56c06e9f0 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa563fff640 1 -- 172.21.15.37:0/3135291033 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0b5830 con 0x7fa56c06e9f0 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5713ee640 1 -- 172.21.15.37:0/3135291033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c0aee00 con 0x7fa56c06e9f0 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa5713ee640 1 -- 172.21.15.37:0/3135291033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa56c067c30 con 0x7fa56c06e9f0 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.404+0000 7fa563fff640 1 -- 172.21.15.37:0/3135291033 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa55c0b59d0 con 0x7fa56c06e9f0 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.408+0000 7fa563fff640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa54c066f60 0x7fa54c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.408+0000 7fa563fff640 1 -- 172.21.15.37:0/3135291033 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa55c11f770 con 0x7fa56c06e9f0 2024-04-25T08:53:53.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.408+0000 7fa5627fc640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa54c066f60 0x7fa54c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.408+0000 7fa5627fc640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa54c066f60 0x7fa54c069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0cf690 tx=0x7fa554008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:53.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.412+0000 7fa563fff640 1 -- 172.21.15.37:0/3135291033 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa55c016070 con 0x7fa56c06e9f0 2024-04-25T08:53:53.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.572+0000 7fa5713ee640 1 -- 172.21.15.37:0/3135291033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa56c06d760 con 0x7fa56c06e9f0 2024-04-25T08:53:53.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.812+0000 7fa563fff640 1 -- 172.21.15.37:0/3135291033 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v38) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa55c0f08a0 con 0x7fa56c06e9f0 2024-04-25T08:53:53.805 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T08:53:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.816+0000 7fa5713ee640 1 -- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa54c066f60 msgr2=0x7fa54c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.816+0000 7fa5713ee640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa54c066f60 0x7fa54c069420 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0cf690 tx=0x7fa554008040 comp rx=0 tx=0).stop 2024-04-25T08:53:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.816+0000 7fa5713ee640 1 -- 172.21.15.37:0/3135291033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c06e9f0 msgr2=0x7fa56c15b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.816+0000 7fa5713ee640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c06e9f0 0x7fa56c15b480 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa55c01a0b0 tx=0x7fa55c01acc0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.816+0000 7fa5713ee640 1 -- 172.21.15.37:0/3135291033 shutdown_connections 2024-04-25T08:53:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.816+0000 7fa5713ee640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa54c066f60 0x7fa54c069420 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.816+0000 7fa5713ee640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa56c0cfa50 0x7fa56c06dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.816+0000 7fa5713ee640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa56c0adc70 0x7fa56c0c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.816+0000 7fa5713ee640 1 --2- 172.21.15.37:0/3135291033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c06e9f0 0x7fa56c15b480 secure :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa55c01a0b0 tx=0x7fa55c01acc0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.816+0000 7fa5713ee640 1 -- 172.21.15.37:0/3135291033 >> 172.21.15.37:0/3135291033 conn(0x7fa56c06ae60 msgr2=0x7fa56c157ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.820+0000 7fa5713ee640 1 -- 172.21.15.37:0/3135291033 shutdown_connections 2024-04-25T08:53:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.820+0000 7fa5713ee640 1 -- 172.21.15.37:0/3135291033 wait complete. 2024-04-25T08:53:53.835 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:53:53.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.976+0000 7f1a24209640 1 Processor -- start 2024-04-25T08:53:53.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.976+0000 7f1a24209640 1 -- start start 2024-04-25T08:53:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.976+0000 7f1a24209640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1a1c0cf590 0x7f1a1c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.976+0000 7f1a24209640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 0x7f1a1c072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.976+0000 7f1a24209640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1c0cbd20 con 0x7f1a1c15b6b0 2024-04-25T08:53:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.976+0000 7f1a24209640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1c15ba90 con 0x7f1a1c0cf590 2024-04-25T08:53:53.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.976+0000 7f1a24209640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0cc000 con 0x7f1a1c072000 2024-04-25T08:53:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.976+0000 7f1a227a6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 0x7f1a1c072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.976+0000 7f1a227a6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 0x7f1a1c072470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:54688/0 (socket says 172.21.15.37:54688) 2024-04-25T08:53:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.976+0000 7f1a227a6640 1 -- 172.21.15.37:0/3305765172 learned_addr learned my addr 172.21.15.37:0/3305765172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.976+0000 7f1a217a4640 1 --2- 172.21.15.37:0/3305765172 >> v2:172.21.15.203:3300/0 conn(0x7f1a1c0cf590 0x7f1a1c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a227a6640 1 -- 172.21.15.37:0/3305765172 >> v2:172.21.15.203:3300/0 conn(0x7f1a1c0cf590 msgr2=0x7f1a1c0cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:53.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a227a6640 1 --2- 172.21.15.37:0/3305765172 >> v2:172.21.15.203:3300/0 conn(0x7f1a1c0cf590 0x7f1a1c0cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a227a6640 1 -- 172.21.15.37:0/3305765172 >> v1:172.21.15.203:6789/0 conn(0x7f1a1c15b6b0 legacy=0x7f1a1c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a227a6640 1 -- 172.21.15.37:0/3305765172 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c0cc2e0 con 0x7f1a1c072000 2024-04-25T08:53:53.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a227a6640 1 --2- 172.21.15.37:0/3305765172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 0x7f1a1c072470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009e50 tx=0x7f1a1401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72b045cb5aad0d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:53.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a20fa3640 1 -- 172.21.15.37:0/3305765172 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a140a57f0 con 0x7f1a1c072000 2024-04-25T08:53:53.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a20fa3640 1 -- 172.21.15.37:0/3305765172 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a140a5990 con 0x7f1a1c072000 2024-04-25T08:53:53.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a20fa3640 1 -- 172.21.15.37:0/3305765172 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a140a5d60 con 0x7f1a1c072000 2024-04-25T08:53:53.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a24209640 1 -- 172.21.15.37:0/3305765172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 msgr2=0x7f1a1c072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a24209640 1 --2- 172.21.15.37:0/3305765172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 0x7f1a1c072470 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1a14009e50 tx=0x7f1a1401ca30 comp rx=0 tx=0).stop 2024-04-25T08:53:53.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a24209640 1 -- 172.21.15.37:0/3305765172 shutdown_connections 2024-04-25T08:53:53.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a24209640 1 --2- 172.21.15.37:0/3305765172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 0x7f1a1c072470 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a24209640 1 --2- 172.21.15.37:0/3305765172 >> v2:172.21.15.203:3300/0 conn(0x7f1a1c0cf590 0x7f1a1c0cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a24209640 1 -- 172.21.15.37:0/3305765172 >> 172.21.15.37:0/3305765172 conn(0x7f1a1c06ae60 msgr2=0x7f1a1c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:53.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a24209640 1 -- 172.21.15.37:0/3305765172 shutdown_connections 2024-04-25T08:53:53.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.980+0000 7f1a24209640 1 -- 172.21.15.37:0/3305765172 wait complete. 2024-04-25T08:53:53.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a24209640 1 Processor -- start 2024-04-25T08:53:53.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a24209640 1 -- start start 2024-04-25T08:53:53.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a24209640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 0x7f1a1c0b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a24209640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a1c0cf590 0x7f1a1c098ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a24209640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a1c15b6b0 0x7f1a1c153390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a24209640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0caf30 con 0x7f1a1c0cf590 2024-04-25T08:53:53.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a24209640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0cadb0 con 0x7f1a1c15b6b0 2024-04-25T08:53:53.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a24209640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c1682a0 con 0x7f1a1c072000 2024-04-25T08:53:53.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a217a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a1c0cf590 0x7f1a1c098ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a217a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a1c0cf590 0x7f1a1c098ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58522/0 (socket says 172.21.15.37:58522) 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a217a4640 1 -- 172.21.15.37:0/1194214812 learned_addr learned my addr 172.21.15.37:0/1194214812 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a227a6640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a1c15b6b0 0x7f1a1c153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a21fa5640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 0x7f1a1c0b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a217a4640 1 -- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 msgr2=0x7f1a1c0b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a217a4640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 0x7f1a1c0b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a217a4640 1 -- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a1c15b6b0 msgr2=0x7f1a1c153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a217a4640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a1c15b6b0 0x7f1a1c153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a217a4640 1 -- 172.21.15.37:0/1194214812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c15a380 con 0x7f1a1c0cf590 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a217a4640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a1c0cf590 0x7f1a1c098ab0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00e930 tx=0x7f1a0c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a127fc640 1 -- 172.21.15.37:0/1194214812 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c017070 con 0x7f1a1c0cf590 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a127fc640 1 -- 172.21.15.37:0/1194214812 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c00ca00 con 0x7f1a1c0cf590 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a24209640 1 -- 172.21.15.37:0/1194214812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1c0b1610 con 0x7f1a1c0cf590 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a127fc640 1 -- 172.21.15.37:0/1194214812 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c00cdb0 con 0x7f1a1c0cf590 2024-04-25T08:53:53.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.984+0000 7f1a24209640 1 -- 172.21.15.37:0/1194214812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1c154050 con 0x7f1a1c0cf590 2024-04-25T08:53:53.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.988+0000 7f1a24209640 1 -- 172.21.15.37:0/1194214812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1c04efa0 con 0x7f1a1c0cf590 2024-04-25T08:53:53.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.988+0000 7f1a127fc640 1 -- 172.21.15.37:0/1194214812 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a0c01a070 con 0x7f1a1c0cf590 2024-04-25T08:53:53.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.988+0000 7f1a127fc640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a00066f60 0x7f1a00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:53.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.988+0000 7f1a127fc640 1 -- 172.21.15.37:0/1194214812 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f1a0c0857a0 con 0x7f1a1c0cf590 2024-04-25T08:53:53.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.988+0000 7f1a21fa5640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a00066f60 0x7f1a00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:53.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.988+0000 7f1a21fa5640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a00066f60 0x7f1a00069420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1a18001770 tx=0x7f1a18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:53.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:53.992+0000 7f1a127fc640 1 -- 172.21.15.37:0/1194214812 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a0c056260 con 0x7f1a1c0cf590 2024-04-25T08:53:54.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.152+0000 7f1a24209640 1 -- 172.21.15.37:0/1194214812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1a1c0ae180 con 0x7f1a1c0cf590 2024-04-25T08:53:54.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.816+0000 7f1a127fc640 1 -- 172.21.15.37:0/1194214812 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v39) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1a0c05a300 con 0x7f1a1c0cf590 2024-04-25T08:53:54.810 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T08:53:54.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 -- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a00066f60 msgr2=0x7f1a00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:54.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a00066f60 0x7f1a00069420 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1a18001770 tx=0x7f1a18009040 comp rx=0 tx=0).stop 2024-04-25T08:53:54.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 -- 172.21.15.37:0/1194214812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a1c0cf590 msgr2=0x7f1a1c098ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:54.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a1c0cf590 0x7f1a1c098ab0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00e930 tx=0x7f1a0c00edf0 comp rx=0 tx=0).stop 2024-04-25T08:53:54.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 -- 172.21.15.37:0/1194214812 shutdown_connections 2024-04-25T08:53:54.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a00066f60 0x7f1a00069420 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:54.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a1c15b6b0 0x7f1a1c153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:54.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a1c0cf590 0x7f1a1c098ab0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:54.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 --2- 172.21.15.37:0/1194214812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a1c072000 0x7f1a1c0b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:54.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 -- 172.21.15.37:0/1194214812 >> 172.21.15.37:0/1194214812 conn(0x7f1a1c06ae60 msgr2=0x7f1a1c0b2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:54.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 -- 172.21.15.37:0/1194214812 shutdown_connections 2024-04-25T08:53:54.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.820+0000 7f1a24209640 1 -- 172.21.15.37:0/1194214812 wait complete. 2024-04-25T08:53:54.846 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:53:54.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.980+0000 7fb73a92b640 1 Processor -- start 2024-04-25T08:53:54.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.980+0000 7fb73a92b640 1 -- start start 2024-04-25T08:53:54.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb73a92b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb73406c3b0 0x7fb73406ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:54.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb73a92b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb734066bb0 0x7fb7340b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:54.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb73a92b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73415ab30 con 0x7fb73415a750 2024-04-25T08:53:54.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb73a92b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb734067100 con 0x7fb73406c3b0 2024-04-25T08:53:54.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb73a92b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340cf560 con 0x7fb734066bb0 2024-04-25T08:53:54.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb7337fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb73406c3b0 0x7fb73406ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:54.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb7337fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb73406c3b0 0x7fb73406ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58526/0 (socket says 172.21.15.37:58526) 2024-04-25T08:53:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb7337fe640 1 -- 172.21.15.37:0/1765214816 learned_addr learned my addr 172.21.15.37:0/1765214816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb738ec8640 1 --2- 172.21.15.37:0/1765214816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb734066bb0 0x7fb7340b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb7337fe640 1 -- 172.21.15.37:0/1765214816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb734066bb0 msgr2=0x7fb7340b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb7337fe640 1 --2- 172.21.15.37:0/1765214816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb734066bb0 0x7fb7340b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb7337fe640 1 -- 172.21.15.37:0/1765214816 >> v1:172.21.15.203:6789/0 conn(0x7fb73415a750 legacy=0x7fb73406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb7337fe640 1 -- 172.21.15.37:0/1765214816 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7340cf840 con 0x7fb73406c3b0 2024-04-25T08:53:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb7337fe640 1 --2- 172.21.15.37:0/1765214816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb73406c3b0 0x7fb73406ee60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb728009870 tx=0x7fb72801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb8dda65add9cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb732ffd640 1 -- 172.21.15.37:0/1765214816 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7280b1070 con 0x7fb73406c3b0 2024-04-25T08:53:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb732ffd640 1 -- 172.21.15.37:0/1765214816 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7280a4e20 con 0x7fb73406c3b0 2024-04-25T08:53:54.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb732ffd640 1 -- 172.21.15.37:0/1765214816 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7280ac660 con 0x7fb73406c3b0 2024-04-25T08:53:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb73a92b640 1 -- 172.21.15.37:0/1765214816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb73406c3b0 msgr2=0x7fb73406ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb73a92b640 1 --2- 172.21.15.37:0/1765214816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb73406c3b0 0x7fb73406ee60 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb728009870 tx=0x7fb72801c440 comp rx=0 tx=0).stop 2024-04-25T08:53:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb73a92b640 1 -- 172.21.15.37:0/1765214816 shutdown_connections 2024-04-25T08:53:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb73a92b640 1 --2- 172.21.15.37:0/1765214816 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb734066bb0 0x7fb7340b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb73a92b640 1 --2- 172.21.15.37:0/1765214816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb73406c3b0 0x7fb73406ee60 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.984+0000 7fb73a92b640 1 -- 172.21.15.37:0/1765214816 >> 172.21.15.37:0/1765214816 conn(0x7fb734071810 msgr2=0x7fb734071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 -- 172.21.15.37:0/1765214816 shutdown_connections 2024-04-25T08:53:54.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 -- 172.21.15.37:0/1765214816 wait complete. 2024-04-25T08:53:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 Processor -- start 2024-04-25T08:53:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 -- start start 2024-04-25T08:53:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb734066bb0 0x7fb73414f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb73406c3b0 0x7fb7341578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb73415a750 0x7fb734157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7341677b0 con 0x7fb73415a750 2024-04-25T08:53:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340ced80 con 0x7fb734066bb0 2024-04-25T08:53:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb734167930 con 0x7fb73406c3b0 2024-04-25T08:53:54.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb738ec8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb73415a750 0x7fb734157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb738ec8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb73415a750 0x7fb734157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58536/0 (socket says 172.21.15.37:58536) 2024-04-25T08:53:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb738ec8640 1 -- 172.21.15.37:0/2643709294 learned_addr learned my addr 172.21.15.37:0/2643709294 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb733fff640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb734066bb0 0x7fb73414f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb7337fe640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb73406c3b0 0x7fb7341578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb738ec8640 1 -- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb73406c3b0 msgr2=0x7fb7341578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb738ec8640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb73406c3b0 0x7fb7341578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:54.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb738ec8640 1 -- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb734066bb0 msgr2=0x7fb73414f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:54.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb738ec8640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb734066bb0 0x7fb73414f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:54.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb738ec8640 1 -- 172.21.15.37:0/2643709294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73415bf60 con 0x7fb73415a750 2024-04-25T08:53:54.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb733fff640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb734066bb0 0x7fb73414f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:54.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb738ec8640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb73415a750 0x7fb734157e20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb72400e820 tx=0x7fb72400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:54.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb730ff9640 1 -- 172.21.15.37:0/2643709294 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb724017070 con 0x7fb73415a750 2024-04-25T08:53:54.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb730ff9640 1 -- 172.21.15.37:0/2643709294 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72400c940 con 0x7fb73415a750 2024-04-25T08:53:54.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 -- 172.21.15.37:0/2643709294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73415c240 con 0x7fb73415a750 2024-04-25T08:53:54.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb730ff9640 1 -- 172.21.15.37:0/2643709294 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72400cd10 con 0x7fb73415a750 2024-04-25T08:53:54.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 -- 172.21.15.37:0/2643709294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73415c800 con 0x7fb73415a750 2024-04-25T08:53:54.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.988+0000 7fb73a92b640 1 -- 172.21.15.37:0/2643709294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73404efa0 con 0x7fb73415a750 2024-04-25T08:53:54.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.992+0000 7fb730ff9640 1 -- 172.21.15.37:0/2643709294 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb72401a070 con 0x7fb73415a750 2024-04-25T08:53:54.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.992+0000 7fb730ff9640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb720066f90 0x7fb720069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:54.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.992+0000 7fb730ff9640 1 -- 172.21.15.37:0/2643709294 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fb7240852f0 con 0x7fb73415a750 2024-04-25T08:53:54.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.992+0000 7fb733fff640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb720066f90 0x7fb720069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:54.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.992+0000 7fb733fff640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb720066f90 0x7fb720069450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb72c001e10 tx=0x7fb72c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:54.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:54.996+0000 7fb730ff9640 1 -- 172.21.15.37:0/2643709294 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb724056260 con 0x7fb73415a750 2024-04-25T08:53:55.026 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:55.029 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:55.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.208+0000 7fb73a92b640 1 -- 172.21.15.37:0/2643709294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb7340727c0 con 0x7fb73415a750 2024-04-25T08:53:55.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.844+0000 7fb730ff9640 1 -- 172.21.15.37:0/2643709294 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb724055b60 con 0x7fb73415a750 2024-04-25T08:53:55.839 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T08:53:55.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.848+0000 7fb73a92b640 1 -- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb720066f90 msgr2=0x7fb720069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:55.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.848+0000 7fb73a92b640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb720066f90 0x7fb720069450 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb72c001e10 tx=0x7fb72c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:55.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.848+0000 7fb73a92b640 1 -- 172.21.15.37:0/2643709294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb73415a750 msgr2=0x7fb734157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.848+0000 7fb73a92b640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb73415a750 0x7fb734157e20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb72400e820 tx=0x7fb72400ece0 comp rx=0 tx=0).stop 2024-04-25T08:53:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.848+0000 7fb73a92b640 1 -- 172.21.15.37:0/2643709294 shutdown_connections 2024-04-25T08:53:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.848+0000 7fb73a92b640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb720066f90 0x7fb720069450 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.848+0000 7fb73a92b640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb73415a750 0x7fb734157e20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.848+0000 7fb73a92b640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb73406c3b0 0x7fb7341578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.848+0000 7fb73a92b640 1 --2- 172.21.15.37:0/2643709294 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb734066bb0 0x7fb73414f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:55.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.848+0000 7fb73a92b640 1 -- 172.21.15.37:0/2643709294 >> 172.21.15.37:0/2643709294 conn(0x7fb734071810 msgr2=0x7fb73415cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:55.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.852+0000 7fb73a92b640 1 -- 172.21.15.37:0/2643709294 shutdown_connections 2024-04-25T08:53:55.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:55.852+0000 7fb73a92b640 1 -- 172.21.15.37:0/2643709294 wait complete. 2024-04-25T08:53:55.885 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e6750640 1 Processor -- start 2024-04-25T08:53:56.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e6750640 1 -- start start 2024-04-25T08:53:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e6750640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67e015a700 0x7f67e015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e6750640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f67e00b24a0 0x7f67e00b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e6750640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e00b2090 con 0x7f67e00b3f50 2024-04-25T08:53:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e6750640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e015f200 con 0x7f67e00b24a0 2024-04-25T08:53:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e6750640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00b43e0 con 0x7f67e015a700 2024-04-25T08:53:56.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e4ced640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f67e00b24a0 0x7f67e00b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67d7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f67e00b3f50 0x7f67e00cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57608/0 (socket says 172.21.15.37:57608) 2024-04-25T08:53:56.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67d7fff640 1 -- 172.21.15.37:0/782193 learned_addr learned my addr 172.21.15.37:0/782193 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:56.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e4ced640 1 --2- 172.21.15.37:0/782193 >> v2:172.21.15.37:3300/0 conn(0x7f67e00b24a0 0x7f67e00b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58542/0 (socket says 172.21.15.37:58542) 2024-04-25T08:53:56.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67dffff640 1 --2- 172.21.15.37:0/782193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67e015a700 0x7f67e015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e4ced640 1 -- 172.21.15.37:0/782193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67e015a700 msgr2=0x7f67e015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:56.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e4ced640 1 --2- 172.21.15.37:0/782193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67e015a700 0x7f67e015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e4ced640 1 -- 172.21.15.37:0/782193 >> v1:172.21.15.37:6789/0 conn(0x7f67e00b3f50 legacy=0x7f67e00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e4ced640 1 -- 172.21.15.37:0/782193 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e00b46c0 con 0x7f67e00b24a0 2024-04-25T08:53:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.016+0000 7f67e4ced640 1 --2- 172.21.15.37:0/782193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67e00b24a0 0x7f67e00b2930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009d90 tx=0x7f67d801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de1c689cbdcb441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67df7fe640 1 -- 172.21.15.37:0/782193 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80b2070 con 0x7f67e00b24a0 2024-04-25T08:53:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67df7fe640 1 -- 172.21.15.37:0/782193 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d80a5800 con 0x7f67e00b24a0 2024-04-25T08:53:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67df7fe640 1 -- 172.21.15.37:0/782193 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80a5bd0 con 0x7f67e00b24a0 2024-04-25T08:53:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 -- 172.21.15.37:0/782193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67e00b24a0 msgr2=0x7f67e00b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 --2- 172.21.15.37:0/782193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67e00b24a0 0x7f67e00b2930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009d90 tx=0x7f67d801c850 comp rx=0 tx=0).stop 2024-04-25T08:53:56.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 -- 172.21.15.37:0/782193 shutdown_connections 2024-04-25T08:53:56.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 --2- 172.21.15.37:0/782193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67e00b24a0 0x7f67e00b2930 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 --2- 172.21.15.37:0/782193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67e015a700 0x7f67e015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 -- 172.21.15.37:0/782193 >> 172.21.15.37:0/782193 conn(0x7f67e00b11a0 msgr2=0x7f67e00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:56.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 -- 172.21.15.37:0/782193 shutdown_connections 2024-04-25T08:53:56.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 -- 172.21.15.37:0/782193 wait complete. 2024-04-25T08:53:56.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 Processor -- start 2024-04-25T08:53:56.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 -- start start 2024-04-25T08:53:56.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67e00b24a0 0x7f67e0153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e015a700 0x7f67e01f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67e0153540 0x7f67e0164b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00cd310 con 0x7f67e00b24a0 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00ce600 con 0x7f67e015a700 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e6750640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e00cd490 con 0x7f67e0153540 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67e00b24a0 0x7f67e0153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67e00b24a0 0x7f67e0153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58544/0 (socket says 172.21.15.37:58544) 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67dffff640 1 -- 172.21.15.37:0/1369515775 learned_addr learned my addr 172.21.15.37:0/1369515775 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67d7fff640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e015a700 0x7f67e01f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67e4ced640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67e0153540 0x7f67e0164b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67dffff640 1 -- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67e0153540 msgr2=0x7f67e0164b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67dffff640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67e0153540 0x7f67e0164b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67dffff640 1 -- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e015a700 msgr2=0x7f67e01f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67dffff640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e015a700 0x7f67e01f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67dffff640 1 -- 172.21.15.37:0/1369515775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e0160620 con 0x7f67e00b24a0 2024-04-25T08:53:56.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.020+0000 7f67dffff640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67e00b24a0 0x7f67e0153d60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f67d000d800 tx=0x7f67d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:56.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.024+0000 7f67dd7fa640 1 -- 172.21.15.37:0/1369515775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d000b6c0 con 0x7f67e00b24a0 2024-04-25T08:53:56.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.024+0000 7f67dd7fa640 1 -- 172.21.15.37:0/1369515775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d0019070 con 0x7f67e00b24a0 2024-04-25T08:53:56.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.024+0000 7f67e6750640 1 -- 172.21.15.37:0/1369515775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e0160900 con 0x7f67e00b24a0 2024-04-25T08:53:56.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.024+0000 7f67dd7fa640 1 -- 172.21.15.37:0/1369515775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d000ba60 con 0x7f67e00b24a0 2024-04-25T08:53:56.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.024+0000 7f67e6750640 1 -- 172.21.15.37:0/1369515775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e0160ec0 con 0x7f67e00b24a0 2024-04-25T08:53:56.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.024+0000 7f67e6750640 1 -- 172.21.15.37:0/1369515775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e004efa0 con 0x7f67e00b24a0 2024-04-25T08:53:56.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.024+0000 7f67dd7fa640 1 -- 172.21.15.37:0/1369515775 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f67d000bc00 con 0x7f67e00b24a0 2024-04-25T08:53:56.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.024+0000 7f67dd7fa640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f67c0066f60 0x7f67c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.024+0000 7f67dd7fa640 1 -- 172.21.15.37:0/1369515775 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67d00854c0 con 0x7f67e00b24a0 2024-04-25T08:53:56.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.024+0000 7f67d7fff640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f67c0066f60 0x7f67c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.024+0000 7f67d7fff640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f67c0066f60 0x7f67c0069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f67e0162390 tx=0x7f67c8007e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:56.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.028+0000 7f67dd7fa640 1 -- 172.21.15.37:0/1369515775 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f67d0055d80 con 0x7f67e00b24a0 2024-04-25T08:53:56.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.192+0000 7f67e6750640 1 -- 172.21.15.37:0/1369515775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67e006f260 con 0x7f67e00b24a0 2024-04-25T08:53:56.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.192+0000 7f67dd7fa640 1 -- 172.21.15.37:0/1369515775 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16919 (secure 0 0 0) 0x7f67d005c0d0 con 0x7f67e00b24a0 2024-04-25T08:53:56.186 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:56.186 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":40,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:55.211735+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:53:53.619419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:53:54.197204+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 -- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f67c0066f60 msgr2=0x7f67c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f67c0066f60 0x7f67c0069420 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f67e0162390 tx=0x7f67c8007e90 comp rx=0 tx=0).stop 2024-04-25T08:53:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 -- 172.21.15.37:0/1369515775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67e00b24a0 msgr2=0x7f67e0153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67e00b24a0 0x7f67e0153d60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f67d000d800 tx=0x7f67d000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 -- 172.21.15.37:0/1369515775 shutdown_connections 2024-04-25T08:53:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f67c0066f60 0x7f67c0069420 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f67e0153540 0x7f67e0164b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f67e015a700 0x7f67e01f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 --2- 172.21.15.37:0/1369515775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67e00b24a0 0x7f67e0153d60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 -- 172.21.15.37:0/1369515775 >> 172.21.15.37:0/1369515775 conn(0x7f67e00b11a0 msgr2=0x7f67e0071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 -- 172.21.15.37:0/1369515775 shutdown_connections 2024-04-25T08:53:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.196+0000 7f67e6750640 1 -- 172.21.15.37:0/1369515775 wait complete. 2024-04-25T08:53:56.206 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:56.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.336+0000 7f03ef938640 1 Processor -- start 2024-04-25T08:53:56.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.336+0000 7f03ef938640 1 -- start start 2024-04-25T08:53:56.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ef938640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03e806b440 0x7f03e80b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ef938640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f03e80b1ac0 0x7f03e80cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ef938640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e80afb20 con 0x7f03e806b820 2024-04-25T08:53:56.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ef938640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e80b06c0 con 0x7f03e80b1ac0 2024-04-25T08:53:56.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ef938640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80afd90 con 0x7f03e806b440 2024-04-25T08:53:56.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03eded5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f03e80b1ac0 0x7f03e80cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03eced3640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f03e806b820 0x7f03e80b13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57620/0 (socket says 172.21.15.37:57620) 2024-04-25T08:53:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03eced3640 1 -- 172.21.15.37:0/2945782566 learned_addr learned my addr 172.21.15.37:0/2945782566 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ed6d4640 1 --2- 172.21.15.37:0/2945782566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03e806b440 0x7f03e80b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03eded5640 1 -- 172.21.15.37:0/2945782566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03e806b440 msgr2=0x7f03e80b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03eded5640 1 --2- 172.21.15.37:0/2945782566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03e806b440 0x7f03e80b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03eded5640 1 -- 172.21.15.37:0/2945782566 >> v1:172.21.15.37:6789/0 conn(0x7f03e806b820 legacy=0x7f03e80b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03dffff640 1 -- 172.21.15.37:0/2945782566 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417264096 0 0) 0x7f03e80afb20 con 0x7f03e806b820 2024-04-25T08:53:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03eded5640 1 -- 172.21.15.37:0/2945782566 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80b0070 con 0x7f03e80b1ac0 2024-04-25T08:53:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03eded5640 1 --2- 172.21.15.37:0/2945782566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e80b1ac0 0x7f03e80cf650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009d90 tx=0x7f03e001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73be1709bae8529 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03dffff640 1 -- 172.21.15.37:0/2945782566 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00b2070 con 0x7f03e80b1ac0 2024-04-25T08:53:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03dffff640 1 -- 172.21.15.37:0/2945782566 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e00a5940 con 0x7f03e80b1ac0 2024-04-25T08:53:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03dffff640 1 -- 172.21.15.37:0/2945782566 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00a5d10 con 0x7f03e80b1ac0 2024-04-25T08:53:56.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ef938640 1 -- 172.21.15.37:0/2945782566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e80b1ac0 msgr2=0x7f03e80cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ef938640 1 --2- 172.21.15.37:0/2945782566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e80b1ac0 0x7f03e80cf650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009d90 tx=0x7f03e001c970 comp rx=0 tx=0).stop 2024-04-25T08:53:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ef938640 1 -- 172.21.15.37:0/2945782566 shutdown_connections 2024-04-25T08:53:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ef938640 1 --2- 172.21.15.37:0/2945782566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e80b1ac0 0x7f03e80cf650 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ef938640 1 --2- 172.21.15.37:0/2945782566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03e806b440 0x7f03e80b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.340+0000 7f03ef938640 1 -- 172.21.15.37:0/2945782566 >> 172.21.15.37:0/2945782566 conn(0x7f03e80b1f10 msgr2=0x7f03e80b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 -- 172.21.15.37:0/2945782566 shutdown_connections 2024-04-25T08:53:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 -- 172.21.15.37:0/2945782566 wait complete. 2024-04-25T08:53:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 Processor -- start 2024-04-25T08:53:56.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 -- start start 2024-04-25T08:53:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03e806b440 0x7f03e81584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03e806b820 0x7f03e80b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e80b1ac0 0x7f03e806f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80cd170 con 0x7f03e80b1ac0 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80ccff0 con 0x7f03e806b440 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e815d080 con 0x7f03e806b820 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03eded5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e80b1ac0 0x7f03e806f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03eded5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e80b1ac0 0x7f03e806f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58574/0 (socket says 172.21.15.37:58574) 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03eded5640 1 -- 172.21.15.37:0/736347479 learned_addr learned my addr 172.21.15.37:0/736347479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ed6d4640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03e806b440 0x7f03e81584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03eced3640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03e806b820 0x7f03e80b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03eded5640 1 -- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03e806b820 msgr2=0x7f03e80b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03eded5640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03e806b820 0x7f03e80b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03eded5640 1 -- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03e806b440 msgr2=0x7f03e81584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03eded5640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03e806b440 0x7f03e81584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03eded5640 1 -- 172.21.15.37:0/736347479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e8158a00 con 0x7f03e80b1ac0 2024-04-25T08:53:56.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ed6d4640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03e806b440 0x7f03e81584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:56.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03eded5640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e80b1ac0 0x7f03e806f100 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009d90 tx=0x7f03e0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:56.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ddffb640 1 -- 172.21.15.37:0/736347479 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00bd020 con 0x7f03e80b1ac0 2024-04-25T08:53:56.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ddffb640 1 -- 172.21.15.37:0/736347479 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e00b3040 con 0x7f03e80b1ac0 2024-04-25T08:53:56.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 -- 172.21.15.37:0/736347479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e80b5de0 con 0x7f03e80b1ac0 2024-04-25T08:53:56.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ddffb640 1 -- 172.21.15.37:0/736347479 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00b2050 con 0x7f03e80b1ac0 2024-04-25T08:53:56.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.344+0000 7f03ef938640 1 -- 172.21.15.37:0/736347479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e80b3550 con 0x7f03e80b1ac0 2024-04-25T08:53:56.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.348+0000 7f03ef938640 1 -- 172.21.15.37:0/736347479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e804efa0 con 0x7f03e80b1ac0 2024-04-25T08:53:56.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.348+0000 7f03ddffb640 1 -- 172.21.15.37:0/736347479 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f03e00ad650 con 0x7f03e80b1ac0 2024-04-25T08:53:56.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.348+0000 7f03ddffb640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03cc066f90 0x7f03cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.348+0000 7f03ddffb640 1 -- 172.21.15.37:0/736347479 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03e01205b0 con 0x7f03e80b1ac0 2024-04-25T08:53:56.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.348+0000 7f03ed6d4640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03cc066f90 0x7f03cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.348+0000 7f03ed6d4640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03cc066f90 0x7f03cc069450 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f03e4007a70 tx=0x7f03e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.352+0000 7f03ddffb640 1 -- 172.21.15.37:0/736347479 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f03e00c7060 con 0x7f03e80b1ac0 2024-04-25T08:53:56.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.512+0000 7f03ef938640 1 -- 172.21.15.37:0/736347479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03e80b1390 con 0x7f03e80b1ac0 2024-04-25T08:53:56.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.516+0000 7f03ddffb640 1 -- 172.21.15.37:0/736347479 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+16919 (secure 0 0 0) 0x7f03e0016560 con 0x7f03e80b1ac0 2024-04-25T08:53:56.509 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:56.509 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":40,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:55.211735+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:53:53.619419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:53:54.197204+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:56.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 -- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03cc066f90 msgr2=0x7f03cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03cc066f90 0x7f03cc069450 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f03e4007a70 tx=0x7f03e4008040 comp rx=0 tx=0).stop 2024-04-25T08:53:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 -- 172.21.15.37:0/736347479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e80b1ac0 msgr2=0x7f03e806f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e80b1ac0 0x7f03e806f100 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009d90 tx=0x7f03e0019710 comp rx=0 tx=0).stop 2024-04-25T08:53:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 -- 172.21.15.37:0/736347479 shutdown_connections 2024-04-25T08:53:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03cc066f90 0x7f03cc069450 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03e80b1ac0 0x7f03e806f100 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f03e806b820 0x7f03e80b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 --2- 172.21.15.37:0/736347479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f03e806b440 0x7f03e81584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 -- 172.21.15.37:0/736347479 >> 172.21.15.37:0/736347479 conn(0x7f03e80b1f10 msgr2=0x7f03e81613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 -- 172.21.15.37:0/736347479 shutdown_connections 2024-04-25T08:53:56.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.520+0000 7f03ef938640 1 -- 172.21.15.37:0/736347479 wait complete. 2024-04-25T08:53:56.529 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:53:56.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.664+0000 7fbbe5f8e640 1 Processor -- start 2024-04-25T08:53:56.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.664+0000 7fbbe5f8e640 1 -- start start 2024-04-25T08:53:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbe5f8e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe015b6b0 0x7fbbe00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbe5f8e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbbe00cf590 0x7fbbe00cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbe5f8e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe00cbd20 con 0x7fbbe015f130 2024-04-25T08:53:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbe5f8e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe015ba90 con 0x7fbbe00cf590 2024-04-25T08:53:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbe5f8e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00cff30 con 0x7fbbe015b6b0 2024-04-25T08:53:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbdffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbbe015f130 0x7fbbe015f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57628/0 (socket says 172.21.15.37:57628) 2024-04-25T08:53:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbdffff640 1 -- 172.21.15.37:0/2428558471 learned_addr learned my addr 172.21.15.37:0/2428558471 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:56.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbdeffd640 1 --2- 172.21.15.37:0/2428558471 >> v2:172.21.15.203:3300/0 conn(0x7fbbe00cf590 0x7fbbe00cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbdf7fe640 1 --2- 172.21.15.37:0/2428558471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe015b6b0 0x7fbbe00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbde7fc640 1 -- 172.21.15.37:0/2428558471 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2075620350 0 0) 0x7fbbe00cbd20 con 0x7fbbe015f130 2024-04-25T08:53:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbde7fc640 1 -- 172.21.15.37:0/2428558471 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbc8003530 con 0x7fbbe015f130 2024-04-25T08:53:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbdeffd640 1 -- 172.21.15.37:0/2428558471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe015b6b0 msgr2=0x7fbbe00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbdeffd640 1 --2- 172.21.15.37:0/2428558471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe015b6b0 0x7fbbe00ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbdeffd640 1 -- 172.21.15.37:0/2428558471 >> v1:172.21.15.37:6789/0 conn(0x7fbbe015f130 legacy=0x7fbbe015f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.668+0000 7fbbdeffd640 1 -- 172.21.15.37:0/2428558471 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe00cc000 con 0x7fbbe00cf590 2024-04-25T08:53:56.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbdeffd640 1 --2- 172.21.15.37:0/2428558471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe00cf590 0x7fbbe00cf9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009d90 tx=0x7fbbd401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e65cb92809982bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:56.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbde7fc640 1 -- 172.21.15.37:0/2428558471 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40b2070 con 0x7fbbe00cf590 2024-04-25T08:53:56.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbde7fc640 1 -- 172.21.15.37:0/2428558471 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd40a5a00 con 0x7fbbe00cf590 2024-04-25T08:53:56.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbde7fc640 1 -- 172.21.15.37:0/2428558471 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40a5db0 con 0x7fbbe00cf590 2024-04-25T08:53:56.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/2428558471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe00cf590 msgr2=0x7fbbe00cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 --2- 172.21.15.37:0/2428558471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe00cf590 0x7fbbe00cf9f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4009d90 tx=0x7fbbd401c940 comp rx=0 tx=0).stop 2024-04-25T08:53:56.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/2428558471 shutdown_connections 2024-04-25T08:53:56.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 --2- 172.21.15.37:0/2428558471 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe00cf590 0x7fbbe00cf9f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 --2- 172.21.15.37:0/2428558471 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe015b6b0 0x7fbbe00ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/2428558471 >> 172.21.15.37:0/2428558471 conn(0x7fbbe006ae60 msgr2=0x7fbbe006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:56.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/2428558471 shutdown_connections 2024-04-25T08:53:56.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/2428558471 wait complete. 2024-04-25T08:53:56.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 Processor -- start 2024-04-25T08:53:56.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 -- start start 2024-04-25T08:53:56.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe015b6b0 0x7fbbe00ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbe015f130 0x7fbbe00989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe00ad750 0x7fbbe01537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00ca900 con 0x7fbbe015f130 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00cba30 con 0x7fbbe015b6b0 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbe5f8e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe00cadb0 con 0x7fbbe00ad750 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbe015f130 0x7fbbe00989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbdeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbe015f130 0x7fbbe00989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58580/0 (socket says 172.21.15.37:58580) 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbdeffd640 1 -- 172.21.15.37:0/1990740899 learned_addr learned my addr 172.21.15.37:0/1990740899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbdffff640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe00ad750 0x7fbbe01537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.672+0000 7fbbdf7fe640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe015b6b0 0x7fbbe00ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbdeffd640 1 -- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe00ad750 msgr2=0x7fbbe01537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbdeffd640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe00ad750 0x7fbbe01537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbdeffd640 1 -- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe015b6b0 msgr2=0x7fbbe00ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:56.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbdeffd640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe015b6b0 0x7fbbe00ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:56.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbdeffd640 1 -- 172.21.15.37:0/1990740899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe0098ef0 con 0x7fbbe015f130 2024-04-25T08:53:56.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbdffff640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe00ad750 0x7fbbe01537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:56.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbdeffd640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbe015f130 0x7fbbe00989b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4000f90 tx=0x7fbbd40196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:56.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbc7fff640 1 -- 172.21.15.37:0/1990740899 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd40b2050 con 0x7fbbe015f130 2024-04-25T08:53:56.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbc7fff640 1 -- 172.21.15.37:0/1990740899 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd40be070 con 0x7fbbe015f130 2024-04-25T08:53:56.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/1990740899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe00adfa0 con 0x7fbbe015f130 2024-04-25T08:53:56.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbc7fff640 1 -- 172.21.15.37:0/1990740899 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4004040 con 0x7fbbe015f130 2024-04-25T08:53:56.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/1990740899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe00ae560 con 0x7fbbe015f130 2024-04-25T08:53:56.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/1990740899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe004efa0 con 0x7fbbe015f130 2024-04-25T08:53:56.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbc7fff640 1 -- 172.21.15.37:0/1990740899 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbbd40041e0 con 0x7fbbe015f130 2024-04-25T08:53:56.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbc7fff640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbbcc066f60 0x7fbbcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:56.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbc7fff640 1 -- 172.21.15.37:0/1990740899 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbbd41203b0 con 0x7fbbe015f130 2024-04-25T08:53:56.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.676+0000 7fbbdf7fe640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbbcc066f60 0x7fbbcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:56.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.680+0000 7fbbdf7fe640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbbcc066f60 0x7fbbcc069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbbe006d1b0 tx=0x7fbbd0009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:56.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.680+0000 7fbbc7fff640 1 -- 172.21.15.37:0/1990740899 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbbd4016560 con 0x7fbbe015f130 2024-04-25T08:53:56.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:56.880+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/1990740899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbbe015f740 con 0x7fbbe015f130 2024-04-25T08:53:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.872+0000 7fbbc7fff640 1 -- 172.21.15.37:0/1990740899 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbbd40f1190 con 0x7fbbe015f130 2024-04-25T08:53:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbbcc066f60 msgr2=0x7fbbcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbbcc066f60 0x7fbbcc069420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbbe006d1b0 tx=0x7fbbd0009190 comp rx=0 tx=0).stop 2024-04-25T08:53:57.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/1990740899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbe015f130 msgr2=0x7fbbe00989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:57.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbe015f130 0x7fbbe00989b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4000f90 tx=0x7fbbd40196e0 comp rx=0 tx=0).stop 2024-04-25T08:53:57.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/1990740899 shutdown_connections 2024-04-25T08:53:57.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbbcc066f60 0x7fbbcc069420 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:57.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbbe00ad750 0x7fbbe01537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:57.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbbe015f130 0x7fbbe00989b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:57.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 --2- 172.21.15.37:0/1990740899 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbbe015b6b0 0x7fbbe00ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:57.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/1990740899 >> 172.21.15.37:0/1990740899 conn(0x7fbbe006ae60 msgr2=0x7fbbe00af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:57.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/1990740899 shutdown_connections 2024-04-25T08:53:57.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:57.876+0000 7fbbe5f8e640 1 -- 172.21.15.37:0/1990740899 wait complete. 2024-04-25T08:53:57.891 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:53:57.891 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:53:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf60266640 1 Processor -- start 2024-04-25T08:53:58.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf60266640 1 -- start start 2024-04-25T08:53:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf60266640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5815da80 0x7faf580d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf60266640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5806be60 con 0x7faf580d07f0 2024-04-25T08:53:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf60266640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf5815de60 con 0x7faf580d1cd0 2024-04-25T08:53:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf60266640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5806c440 con 0x7faf5815da80 2024-04-25T08:53:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5e803640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7faf580d07f0 0x7faf5806c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57630/0 (socket says 172.21.15.37:57630) 2024-04-25T08:53:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5e803640 1 -- 172.21.15.37:0/2432104457 learned_addr learned my addr 172.21.15.37:0/2432104457 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5e002640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5815da80 0x7faf580d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:58.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820242891 0 0) 0x7faf5806be60 con 0x7faf580d07f0 2024-04-25T08:53:58.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf44003530 con 0x7faf580d07f0 2024-04-25T08:53:58.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2897498988 0 0) 0x7faf44003530 con 0x7faf580d07f0 2024-04-25T08:53:58.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5815da80 msgr2=0x7faf580d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:58.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5815da80 0x7faf580d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 >> v1:172.21.15.203:6789/0 conn(0x7faf580d1cd0 legacy=0x7faf580d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:58.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580d2450 con 0x7faf580d07f0 2024-04-25T08:53:58.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7faf50001a00 con 0x7faf580d07f0 2024-04-25T08:53:58.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 >> v1:172.21.15.37:6789/0 conn(0x7faf580d07f0 legacy=0x7faf5806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf44003a60 0x7faf44003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf44004810 0x7faf44004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5815da80 0x7faf44005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5806c440 con 0x7faf44003a60 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4400bb60 con 0x7faf44004810 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4400bd30 con 0x7faf5815da80 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5e002640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf44003a60 0x7faf44003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5e803640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5815da80 0x7faf44005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d801640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf44004810 0x7faf44004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5e002640 1 -- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5815da80 msgr2=0x7faf44005280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5e002640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5815da80 0x7faf44005280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5e002640 1 -- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf44004810 msgr2=0x7faf44004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5e002640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf44004810 0x7faf44004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5e002640 1 -- 172.21.15.37:0/2432104457 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf44003530 con 0x7faf44003a60 2024-04-25T08:53:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5d801640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf44004810 0x7faf44004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.032+0000 7faf5e803640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf5815da80 0x7faf44005280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.036+0000 7faf5e002640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf44003a60 0x7faf44003ea0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faf54009ae0 tx=0x7faf5401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8291deee6343e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.036+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540b2070 con 0x7faf44003a60 2024-04-25T08:53:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.036+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf540a5520 con 0x7faf44003a60 2024-04-25T08:53:58.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.036+0000 7faf5d000640 1 -- 172.21.15.37:0/2432104457 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540096b0 con 0x7faf44003a60 2024-04-25T08:53:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.036+0000 7faf60266640 1 -- 172.21.15.37:0/2432104457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf44003a60 msgr2=0x7faf44003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.036+0000 7faf60266640 1 --2- 172.21.15.37:0/2432104457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf44003a60 0x7faf44003ea0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7faf54009ae0 tx=0x7faf5401ca40 comp rx=0 tx=0).stop 2024-04-25T08:53:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.036+0000 7faf5e803640 1 -- 172.21.15.37:0/2432104457 reap_dead start 2024-04-25T08:53:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.036+0000 7faf60266640 1 -- 172.21.15.37:0/2432104457 shutdown_connections 2024-04-25T08:53:58.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.036+0000 7faf60266640 1 -- 172.21.15.37:0/2432104457 >> 172.21.15.37:0/2432104457 conn(0x7faf5808b000 msgr2=0x7faf580d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.036+0000 7faf60266640 1 -- 172.21.15.37:0/2432104457 shutdown_connections 2024-04-25T08:53:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.036+0000 7faf60266640 1 -- 172.21.15.37:0/2432104457 wait complete. 2024-04-25T08:53:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf60266640 1 Processor -- start 2024-04-25T08:53:58.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf60266640 1 -- start start 2024-04-25T08:53:58.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf60266640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf5814fcd0 0x7faf581f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:58.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf60266640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf580cca50 0x7faf580cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:58.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf60266640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf580b6eb0 0x7faf580b7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf60266640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5806d970 con 0x7faf5814fcd0 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf60266640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5806d7f0 con 0x7faf580b6eb0 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf60266640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5806daf0 con 0x7faf580cca50 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5e002640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf5814fcd0 0x7faf581f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5e002640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf5814fcd0 0x7faf581f7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58592/0 (socket says 172.21.15.37:58592) 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5e002640 1 -- 172.21.15.37:0/1666871386 learned_addr learned my addr 172.21.15.37:0/1666871386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5d801640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf580cca50 0x7faf580cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5e803640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf580b6eb0 0x7faf580b7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5e002640 1 -- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf580cca50 msgr2=0x7faf580cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5e002640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf580cca50 0x7faf580cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5e002640 1 -- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf580b6eb0 msgr2=0x7faf580b7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5e002640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf580b6eb0 0x7faf580b7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5e002640 1 -- 172.21.15.37:0/1666871386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580ca8f0 con 0x7faf5814fcd0 2024-04-25T08:53:58.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5e803640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf580b6eb0 0x7faf580b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:58.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5d801640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf580cca50 0x7faf580cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:58.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf5e002640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf5814fcd0 0x7faf581f7570 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faf540b0040 tx=0x7faf540197e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:58.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf4e7fc640 1 -- 172.21.15.37:0/1666871386 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540bc020 con 0x7faf5814fcd0 2024-04-25T08:53:58.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf4e7fc640 1 -- 172.21.15.37:0/1666871386 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf540a5cd0 con 0x7faf5814fcd0 2024-04-25T08:53:58.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf60266640 1 -- 172.21.15.37:0/1666871386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf580cabd0 con 0x7faf5814fcd0 2024-04-25T08:53:58.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf4e7fc640 1 -- 172.21.15.37:0/1666871386 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf540b2040 con 0x7faf5814fcd0 2024-04-25T08:53:58.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf60266640 1 -- 172.21.15.37:0/1666871386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf580cb190 con 0x7faf5814fcd0 2024-04-25T08:53:58.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf60266640 1 -- 172.21.15.37:0/1666871386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5804efa0 con 0x7faf5814fcd0 2024-04-25T08:53:58.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.040+0000 7faf4e7fc640 1 -- 172.21.15.37:0/1666871386 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf540bf070 con 0x7faf5814fcd0 2024-04-25T08:53:58.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.044+0000 7faf4e7fc640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf3c066f90 0x7faf3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:58.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.044+0000 7faf4e7fc640 1 -- 172.21.15.37:0/1666871386 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faf5411e9a0 con 0x7faf5814fcd0 2024-04-25T08:53:58.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.044+0000 7faf5d801640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf3c066f90 0x7faf3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:58.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.044+0000 7faf5d801640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf3c066f90 0x7faf3c069450 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faf580ca6a0 tx=0x7faf48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:58.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.048+0000 7faf4e7fc640 1 -- 172.21.15.37:0/1666871386 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf54016780 con 0x7faf5814fcd0 2024-04-25T08:53:58.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.240+0000 7faf60266640 1 -- 172.21.15.37:0/1666871386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7faf580b08e0 con 0x7faf5814fcd0 2024-04-25T08:53:58.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.888+0000 7faf4e7fc640 1 -- 172.21.15.37:0/1666871386 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7faf540ef810 con 0x7faf5814fcd0 2024-04-25T08:53:58.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 -- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf3c066f90 msgr2=0x7faf3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:58.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf3c066f90 0x7faf3c069450 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faf580ca6a0 tx=0x7faf48009040 comp rx=0 tx=0).stop 2024-04-25T08:53:58.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 -- 172.21.15.37:0/1666871386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf5814fcd0 msgr2=0x7faf581f7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:58.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf5814fcd0 0x7faf581f7570 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7faf540b0040 tx=0x7faf540197e0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 -- 172.21.15.37:0/1666871386 shutdown_connections 2024-04-25T08:53:58.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf3c066f90 0x7faf3c069450 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf580b6eb0 0x7faf580b7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf580cca50 0x7faf580cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 --2- 172.21.15.37:0/1666871386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf5814fcd0 0x7faf581f7570 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 -- 172.21.15.37:0/1666871386 >> 172.21.15.37:0/1666871386 conn(0x7faf5808b000 msgr2=0x7faf580ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:58.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 -- 172.21.15.37:0/1666871386 shutdown_connections 2024-04-25T08:53:58.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:58.892+0000 7faf60266640 1 -- 172.21.15.37:0/1666871386 wait complete. 2024-04-25T08:53:58.903 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:53:58.903 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:53:59.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd54e3d0640 1 Processor -- start 2024-04-25T08:53:59.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd54e3d0640 1 -- start start 2024-04-25T08:53:59.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd54e3d0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd5480b1ae0 0x7fd548151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd54e3d0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd548160310 con 0x7fd5480cc820 2024-04-25T08:53:59.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd54e3d0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd54815f550 con 0x7fd5480b0b00 2024-04-25T08:53:59.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd54e3d0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd54815cfa0 con 0x7fd5480b1ae0 2024-04-25T08:53:59.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd5477fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd5480cc820 0x7fd5480b13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:57636/0 (socket says 172.21.15.37:57636) 2024-04-25T08:53:59.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd5477fe640 1 -- 172.21.15.37:0/2407327937 learned_addr learned my addr 172.21.15.37:0/2407327937 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:59.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd54c96d640 1 --2- 172.21.15.37:0/2407327937 >> v2:172.21.15.203:3300/0 conn(0x7fd5480b1ae0 0x7fd548151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095805186 0 0) 0x7fd548160310 con 0x7fd5480cc820 2024-04-25T08:53:59.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd530003530 con 0x7fd5480cc820 2024-04-25T08:53:59.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1317332526 0 0) 0x7fd530003530 con 0x7fd5480cc820 2024-04-25T08:53:59.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 >> v2:172.21.15.203:3300/0 conn(0x7fd5480b1ae0 msgr2=0x7fd548151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:59.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 --2- 172.21.15.37:0/2407327937 >> v2:172.21.15.203:3300/0 conn(0x7fd5480b1ae0 0x7fd548151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 >> v1:172.21.15.203:6789/0 conn(0x7fd5480b0b00 legacy=0x7fd5480cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd54815f830 con 0x7fd5480cc820 2024-04-25T08:53:59.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fd53c001680 con 0x7fd5480cc820 2024-04-25T08:53:59.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 >> v1:172.21.15.37:6789/0 conn(0x7fd5480cc820 legacy=0x7fd5480b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 --2- 172.21.15.37:0/2407327937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd530003a30 0x7fd530003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 --2- 172.21.15.37:0/2407327937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd530004810 0x7fd530004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 --2- 172.21.15.37:0/2407327937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5300051d0 0x7fd53000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53000c0d0 con 0x7fd530004810 2024-04-25T08:53:59.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53000c280 con 0x7fd5300051d0 2024-04-25T08:53:59.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd53000c480 con 0x7fd530003a30 2024-04-25T08:53:59.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd5477fe640 1 --2- 172.21.15.37:0/2407327937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd530004810 0x7fd530004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd547fff640 1 --2- 172.21.15.37:0/2407327937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd530003a30 0x7fd530003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd54c96d640 1 --2- 172.21.15.37:0/2407327937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5300051d0 0x7fd53000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd5477fe640 1 -- 172.21.15.37:0/2407327937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd530003a30 msgr2=0x7fd530003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:59.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd5477fe640 1 --2- 172.21.15.37:0/2407327937 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd530003a30 0x7fd530003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd5477fe640 1 -- 172.21.15.37:0/2407327937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5300051d0 msgr2=0x7fd53000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:59.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd5477fe640 1 --2- 172.21.15.37:0/2407327937 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd5300051d0 0x7fd53000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd5477fe640 1 -- 172.21.15.37:0/2407327937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd530003530 con 0x7fd530004810 2024-04-25T08:53:59.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.040+0000 7fd54c96d640 1 -- 172.21.15.37:0/2407327937 reap_dead start 2024-04-25T08:53:59.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd5477fe640 1 --2- 172.21.15.37:0/2407327937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd530004810 0x7fd530004c90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00c210 tx=0x7fd53c01f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be430205497efc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:59.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c007d40 con 0x7fd530004810 2024-04-25T08:53:59.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c005b00 con 0x7fd530004810 2024-04-25T08:53:59.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd546ffd640 1 -- 172.21.15.37:0/2407327937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c007070 con 0x7fd530004810 2024-04-25T08:53:59.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd54e3d0640 1 -- 172.21.15.37:0/2407327937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd530004810 msgr2=0x7fd530004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd54e3d0640 1 --2- 172.21.15.37:0/2407327937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd530004810 0x7fd530004c90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fd53c00c210 tx=0x7fd53c01f3b0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd54e3d0640 1 -- 172.21.15.37:0/2407327937 shutdown_connections 2024-04-25T08:53:59.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd54e3d0640 1 --2- 172.21.15.37:0/2407327937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd530004810 0x7fd530004c90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd54e3d0640 1 -- 172.21.15.37:0/2407327937 >> 172.21.15.37:0/2407327937 conn(0x7fd5480b1f10 msgr2=0x7fd5480b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:59.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd54e3d0640 1 -- 172.21.15.37:0/2407327937 shutdown_connections 2024-04-25T08:53:59.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd54e3d0640 1 -- 172.21.15.37:0/2407327937 wait complete. 2024-04-25T08:53:59.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd54e3d0640 1 Processor -- start 2024-04-25T08:53:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd54e3d0640 1 -- start start 2024-04-25T08:53:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.044+0000 7fd54e3d0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd530004810 0x7fd5481f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd54e3d0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5480c38d0 0x7fd5481453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd54e3d0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd54806f100 0x7fd54806f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd54e3d0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54806b950 con 0x7fd5480c38d0 2024-04-25T08:53:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd54e3d0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54815fe80 con 0x7fd530004810 2024-04-25T08:53:59.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd54e3d0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5480b0ee0 con 0x7fd54806f100 2024-04-25T08:53:59.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5480c38d0 0x7fd5481453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd5477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5480c38d0 0x7fd5481453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58600/0 (socket says 172.21.15.37:58600) 2024-04-25T08:53:59.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd5477fe640 1 -- 172.21.15.37:0/1905924259 learned_addr learned my addr 172.21.15.37:0/1905924259 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:59.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd547fff640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd530004810 0x7fd5481f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd54c96d640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd54806f100 0x7fd54806f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd5477fe640 1 -- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd54806f100 msgr2=0x7fd54806f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd5477fe640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd54806f100 0x7fd54806f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd5477fe640 1 -- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd530004810 msgr2=0x7fd5481f5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd5477fe640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd530004810 0x7fd5481f5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd5477fe640 1 -- 172.21.15.37:0/1905924259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd548145b60 con 0x7fd5480c38d0 2024-04-25T08:53:59.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd547fff640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd530004810 0x7fd5481f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:59.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd5477fe640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5480c38d0 0x7fd5481453c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd53c001980 tx=0x7fd530005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:59.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd544ff9640 1 -- 172.21.15.37:0/1905924259 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c001680 con 0x7fd5480c38d0 2024-04-25T08:53:59.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd544ff9640 1 -- 172.21.15.37:0/1905924259 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53c0b6070 con 0x7fd5480c38d0 2024-04-25T08:53:59.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd544ff9640 1 -- 172.21.15.37:0/1905924259 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53c004a10 con 0x7fd5480c38d0 2024-04-25T08:53:59.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd54e3d0640 1 -- 172.21.15.37:0/1905924259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd548145e40 con 0x7fd5480c38d0 2024-04-25T08:53:59.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd54e3d0640 1 -- 172.21.15.37:0/1905924259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54815d7d0 con 0x7fd5480c38d0 2024-04-25T08:53:59.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd54e3d0640 1 -- 172.21.15.37:0/1905924259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54806fac0 con 0x7fd5480c38d0 2024-04-25T08:53:59.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd544ff9640 1 -- 172.21.15.37:0/1905924259 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd53c0b8020 con 0x7fd5480c38d0 2024-04-25T08:53:59.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd544ff9640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd534066f60 0x7fd534069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd544ff9640 1 -- 172.21.15.37:0/1905924259 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd53c122540 con 0x7fd5480c38d0 2024-04-25T08:53:59.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.048+0000 7fd547fff640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd534066f60 0x7fd534069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.052+0000 7fd547fff640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd534066f60 0x7fd534069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd538000c10 tx=0x7fd538009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:59.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.052+0000 7fd544ff9640 1 -- 172.21.15.37:0/1905924259 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd53c0190c0 con 0x7fd5480c38d0 2024-04-25T08:53:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.248+0000 7fd54e3d0640 1 -- 172.21.15.37:0/1905924259 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5480b56d0 con 0x7fd5480c38d0 2024-04-25T08:53:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.252+0000 7fd544ff9640 1 -- 172.21.15.37:0/1905924259 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fd53c0f33c0 con 0x7fd5480c38d0 2024-04-25T08:53:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:59.244 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3245776986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3245776986},{"type":"v1","addr":"172.21.15.203:6836","nonce":3245776986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5056,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/841808653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":841808653},{"type":"v1","addr":"172.21.15.37:6835","nonce":841808653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1908736119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1908736119},{"type":"v1","addr":"172.21.15.203:6837","nonce":1908736119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:53:55.212059+0000","modified":"2024-04-25T08:53:58.247733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035847732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035847732},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035847732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:53:59.244 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-04-25T08:53:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 -- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd534066f60 msgr2=0x7fd534069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd534066f60 0x7fd534069420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd538000c10 tx=0x7fd538009040 comp rx=0 tx=0).stop 2024-04-25T08:53:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 -- 172.21.15.37:0/1905924259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5480c38d0 msgr2=0x7fd5481453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5480c38d0 0x7fd5481453c0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd53c001980 tx=0x7fd530005840 comp rx=0 tx=0).stop 2024-04-25T08:53:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 -- 172.21.15.37:0/1905924259 shutdown_connections 2024-04-25T08:53:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd534066f60 0x7fd534069420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd54806f100 0x7fd54806f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5480c38d0 0x7fd5481453c0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 --2- 172.21.15.37:0/1905924259 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd530004810 0x7fd5481f5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 -- 172.21.15.37:0/1905924259 >> 172.21.15.37:0/1905924259 conn(0x7fd5480b1f10 msgr2=0x7fd5480ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 -- 172.21.15.37:0/1905924259 shutdown_connections 2024-04-25T08:53:59.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.256+0000 7fd54e3d0640 1 -- 172.21.15.37:0/1905924259 wait complete. 2024-04-25T08:53:59.264 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.392+0000 7f6a823fc640 1 Processor -- start 2024-04-25T08:53:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.392+0000 7f6a823fc640 1 -- start start 2024-04-25T08:53:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a823fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a7c0b0a30 0x7f6a7c0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a823fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6a7c09a180 0x7f6a7c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a823fc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0b0e10 con 0x7f6a7c06f4a0 2024-04-25T08:53:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a823fc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a7c0d0d90 con 0x7f6a7c09a180 2024-04-25T08:53:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a823fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0d0180 con 0x7f6a7c0b0a30 2024-04-25T08:53:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6a7c06f4a0 0x7f6a7c099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41858/0 (socket says 172.21.15.37:41858) 2024-04-25T08:53:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7b7fe640 1 -- 172.21.15.37:0/1618169277 learned_addr learned my addr 172.21.15.37:0/1618169277 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:53:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7bfff640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a7c0b0a30 0x7f6a7c0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a80999640 1 --2- 172.21.15.37:0/1618169277 >> v2:172.21.15.203:3300/0 conn(0x7f6a7c09a180 0x7f6a7c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713935595 0 0) 0x7f6a7c0b0e10 con 0x7f6a7c06f4a0 2024-04-25T08:53:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a64003530 con 0x7f6a7c06f4a0 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001778037 0 0) 0x7f6a64003530 con 0x7f6a7c06f4a0 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a7c0b0a30 msgr2=0x7f6a7c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a7c0b0a30 0x7f6a7c0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 >> v2:172.21.15.203:3300/0 conn(0x7f6a7c09a180 msgr2=0x7f6a7c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 --2- 172.21.15.37:0/1618169277 >> v2:172.21.15.203:3300/0 conn(0x7f6a7c09a180 0x7f6a7c15d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c0d0460 con 0x7f6a7c06f4a0 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f6a70001a00 con 0x7f6a7c06f4a0 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 >> v1:172.21.15.37:6789/0 conn(0x7f6a7c06f4a0 legacy=0x7f6a7c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a64003a30 0x7f6a64003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a64004810 0x7f6a64004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a7c0b0a30 0x7f6a640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0d0d90 con 0x7f6a64003a30 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0d0180 con 0x7f6a64004810 2024-04-25T08:53:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6400b9d0 con 0x7f6a7c0b0a30 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7bfff640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a64003a30 0x7f6a64003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a80999640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a7c0b0a30 0x7f6a640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7b7fe640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a64004810 0x7f6a64004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7bfff640 1 -- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a7c0b0a30 msgr2=0x7f6a640051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7bfff640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a7c0b0a30 0x7f6a640051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7bfff640 1 -- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a64004810 msgr2=0x7f6a64004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7bfff640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a64004810 0x7f6a64004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7bfff640 1 -- 172.21.15.37:0/1618169277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a64003530 con 0x7f6a64003a30 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.396+0000 7f6a7b7fe640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a64004810 0x7f6a64004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7bfff640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a64003a30 0x7f6a64003e50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009b10 tx=0x7f6a6c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=986a435fef3401a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0096b0 con 0x7f6a64003a30 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c0a6500 con 0x7f6a64003a30 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7affd640 1 -- 172.21.15.37:0/1618169277 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c0a68f0 con 0x7f6a64003a30 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 -- 172.21.15.37:0/1618169277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a64003a30 msgr2=0x7f6a64003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 --2- 172.21.15.37:0/1618169277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a64003a30 0x7f6a64003e50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c009b10 tx=0x7f6a6c01ca00 comp rx=0 tx=0).stop 2024-04-25T08:53:59.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a80999640 1 -- 172.21.15.37:0/1618169277 reap_dead start 2024-04-25T08:53:59.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 -- 172.21.15.37:0/1618169277 shutdown_connections 2024-04-25T08:53:59.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 -- 172.21.15.37:0/1618169277 >> 172.21.15.37:0/1618169277 conn(0x7f6a7c0b30b0 msgr2=0x7f6a7c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:59.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 -- 172.21.15.37:0/1618169277 shutdown_connections 2024-04-25T08:53:59.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 -- 172.21.15.37:0/1618169277 wait complete. 2024-04-25T08:53:59.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 Processor -- start 2024-04-25T08:53:59.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 -- start start 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a64003a30 0x7f6a7c1faa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7c1f7ac0 0x7f6a7c1f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a7c1f68b0 0x7f6a7c159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c15db00 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c15d980 con 0x7f6a7c1f68b0 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a823fc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c15dc80 con 0x7f6a64003a30 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7c1f7ac0 0x7f6a7c1f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7c1f7ac0 0x7f6a7c1f97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37974/0 (socket says 172.21.15.37:37974) 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7b7fe640 1 -- 172.21.15.37:0/4109845403 learned_addr learned my addr 172.21.15.37:0/4109845403 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7bfff640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a64003a30 0x7f6a7c1faa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a80999640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a7c1f68b0 0x7f6a7c159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7b7fe640 1 -- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a64003a30 msgr2=0x7f6a7c1faa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7b7fe640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a64003a30 0x7f6a7c1faa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7b7fe640 1 -- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a7c1f68b0 msgr2=0x7f6a7c159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7b7fe640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a7c1f68b0 0x7f6a7c159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7b7fe640 1 -- 172.21.15.37:0/4109845403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c1f7f40 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a7bfff640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a64003a30 0x7f6a7c1faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.400+0000 7f6a80999640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a7c1f68b0 0x7f6a7c159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a7b7fe640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7c1f7ac0 0x7f6a7c1f97f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6a70001340 tx=0x7f6a7000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a78ff9640 1 -- 172.21.15.37:0/4109845403 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a70019070 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a78ff9640 1 -- 172.21.15.37:0/4109845403 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a70004d70 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a823fc640 1 -- 172.21.15.37:0/4109845403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c1f7150 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a78ff9640 1 -- 172.21.15.37:0/4109845403 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a700072d0 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a823fc640 1 -- 172.21.15.37:0/4109845403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c1f7710 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a823fc640 1 -- 172.21.15.37:0/4109845403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c04efa0 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a78ff9640 1 -- 172.21.15.37:0/4109845403 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6a7001c070 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a78ff9640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a68066f60 0x7f6a68069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a78ff9640 1 -- 172.21.15.37:0/4109845403 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a70085380 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a7bfff640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a68066f60 0x7f6a68069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.404+0000 7f6a7bfff640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a68066f60 0x7f6a68069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0af000 tx=0x7f6a6c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.408+0000 7f6a78ff9640 1 -- 172.21.15.37:0/4109845403 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6a7c1f7f40 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.572+0000 7f6a823fc640 1 -- 172.21.15.37:0/4109845403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a7c0b3ee0 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.572+0000 7f6a78ff9640 1 -- 172.21.15.37:0/4109845403 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16906 (secure 0 0 0) 0x7f6a70056290 con 0x7f6a7c1f7ac0 2024-04-25T08:53:59.567 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:59.567 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:56.863329+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:53:53.619419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:53:54.197204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:59.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 -- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a68066f60 msgr2=0x7f6a68069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a68066f60 0x7f6a68069420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c0af000 tx=0x7f6a6c0046d0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 -- 172.21.15.37:0/4109845403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7c1f7ac0 msgr2=0x7f6a7c1f97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7c1f7ac0 0x7f6a7c1f97f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6a70001340 tx=0x7f6a7000b4a0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 -- 172.21.15.37:0/4109845403 shutdown_connections 2024-04-25T08:53:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6a68066f60 0x7f6a68069420 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6a7c1f68b0 0x7f6a7c159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a7c1f7ac0 0x7f6a7c1f97f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 --2- 172.21.15.37:0/4109845403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6a64003a30 0x7f6a7c1faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 -- 172.21.15.37:0/4109845403 >> 172.21.15.37:0/4109845403 conn(0x7f6a7c0b30b0 msgr2=0x7f6a7c15cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 -- 172.21.15.37:0/4109845403 shutdown_connections 2024-04-25T08:53:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.576+0000 7f6a823fc640 1 -- 172.21.15.37:0/4109845403 wait complete. 2024-04-25T08:53:59.586 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:53:59.587 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:53:59.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.716+0000 7f9222211640 1 Processor -- start 2024-04-25T08:53:59.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f9222211640 1 -- start start 2024-04-25T08:53:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f9222211640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f921c0ce510 0x7f921c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f9222211640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f921c144ca0 0x7f921c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f9222211640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f921c0cee30 con 0x7f921c06ec60 2024-04-25T08:53:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f9222211640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f921c0cc2b0 con 0x7f921c0ce510 2024-04-25T08:53:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f9222211640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921c06e290 con 0x7f921c144ca0 2024-04-25T08:53:59.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f921b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f921c0ce510 0x7f921c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f921b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f921c0ce510 0x7f921c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37986/0 (socket says 172.21.15.37:37986) 2024-04-25T08:53:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f921b7fe640 1 -- 172.21.15.37:0/486210784 learned_addr learned my addr 172.21.15.37:0/486210784 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f9212ffd640 1 --2- 172.21.15.37:0/486210784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f921c144ca0 0x7f921c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f921b7fe640 1 -- 172.21.15.37:0/486210784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f921c144ca0 msgr2=0x7f921c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f921b7fe640 1 --2- 172.21.15.37:0/486210784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f921c144ca0 0x7f921c06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f921b7fe640 1 -- 172.21.15.37:0/486210784 >> v1:172.21.15.203:6789/0 conn(0x7f921c06ec60 legacy=0x7f921c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.720+0000 7f921b7fe640 1 -- 172.21.15.37:0/486210784 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c0ccc70 con 0x7f921c0ce510 2024-04-25T08:53:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f921b7fe640 1 --2- 172.21.15.37:0/486210784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921c0ce510 0x7f921c0ce8f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f920c009870 tx=0x7f920c01ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb80cdbbbcc20f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f921affd640 1 -- 172.21.15.37:0/486210784 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0b5070 con 0x7f921c0ce510 2024-04-25T08:53:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f921affd640 1 -- 172.21.15.37:0/486210784 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920c0a8a50 con 0x7f921c0ce510 2024-04-25T08:53:59.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f921affd640 1 -- 172.21.15.37:0/486210784 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0a8e20 con 0x7f921c0ce510 2024-04-25T08:53:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 -- 172.21.15.37:0/486210784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921c0ce510 msgr2=0x7f921c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 --2- 172.21.15.37:0/486210784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921c0ce510 0x7f921c0ce8f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f920c009870 tx=0x7f920c01ed50 comp rx=0 tx=0).stop 2024-04-25T08:53:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 -- 172.21.15.37:0/486210784 shutdown_connections 2024-04-25T08:53:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 --2- 172.21.15.37:0/486210784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f921c144ca0 0x7f921c06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 --2- 172.21.15.37:0/486210784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921c0ce510 0x7f921c0ce8f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 -- 172.21.15.37:0/486210784 >> 172.21.15.37:0/486210784 conn(0x7f921c0b11a0 msgr2=0x7f921c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 -- 172.21.15.37:0/486210784 shutdown_connections 2024-04-25T08:53:59.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 -- 172.21.15.37:0/486210784 wait complete. 2024-04-25T08:53:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 Processor -- start 2024-04-25T08:53:59.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 -- start start 2024-04-25T08:53:59.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921c06ec60 0x7f921c13ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f921c0ce510 0x7f921c0caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f921c144ca0 0x7f921c0caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c0cc900 con 0x7f921c06ec60 2024-04-25T08:53:59.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f921c06df20 con 0x7f921c0ce510 2024-04-25T08:53:59.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f9222211640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921c142d90 con 0x7f921c144ca0 2024-04-25T08:53:59.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.724+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921c06ec60 0x7f921c13ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f921b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921c06ec60 0x7f921c13ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38002/0 (socket says 172.21.15.37:38002) 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f921b7fe640 1 -- 172.21.15.37:0/270906049 learned_addr learned my addr 172.21.15.37:0/270906049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f921bfff640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f921c144ca0 0x7f921c0caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f9212ffd640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f921c0ce510 0x7f921c0caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f921b7fe640 1 -- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f921c144ca0 msgr2=0x7f921c0caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f921b7fe640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f921c144ca0 0x7f921c0caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f921b7fe640 1 -- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f921c0ce510 msgr2=0x7f921c0caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f921b7fe640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f921c0ce510 0x7f921c0caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f921b7fe640 1 -- 172.21.15.37:0/270906049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c1e0d80 con 0x7f921c06ec60 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f921b7fe640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921c06ec60 0x7f921c13ce70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f920c0b3040 tx=0x7f920c01baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f9218ff9640 1 -- 172.21.15.37:0/270906049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0c0070 con 0x7f921c06ec60 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f9218ff9640 1 -- 172.21.15.37:0/270906049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f920c0bb440 con 0x7f921c06ec60 2024-04-25T08:53:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f9222211640 1 -- 172.21.15.37:0/270906049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f921c1e1020 con 0x7f921c06ec60 2024-04-25T08:53:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f9218ff9640 1 -- 172.21.15.37:0/270906049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f920c0b5070 con 0x7f921c06ec60 2024-04-25T08:53:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f9222211640 1 -- 172.21.15.37:0/270906049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f921c1e15e0 con 0x7f921c06ec60 2024-04-25T08:53:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f9222211640 1 -- 172.21.15.37:0/270906049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f921c04efa0 con 0x7f921c06ec60 2024-04-25T08:53:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f9218ff9640 1 -- 172.21.15.37:0/270906049 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f920c0c3070 con 0x7f921c06ec60 2024-04-25T08:53:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f9218ff9640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91fc066f60 0x7f91fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f9218ff9640 1 -- 172.21.15.37:0/270906049 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f920c122760 con 0x7f921c06ec60 2024-04-25T08:53:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.728+0000 7f9212ffd640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91fc066f60 0x7f91fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.732+0000 7f9212ffd640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91fc066f60 0x7f91fc069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f921c0cbcd0 tx=0x7f9204008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.732+0000 7f9218ff9640 1 -- 172.21.15.37:0/270906049 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f920c0188b0 con 0x7f921c06ec60 2024-04-25T08:53:59.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.888+0000 7f9222211640 1 -- 172.21.15.37:0/270906049 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f921c1443b0 con 0x7f91fc066f60 2024-04-25T08:53:59.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.908+0000 7f9218ff9640 1 -- 172.21.15.37:0/270906049 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251962 (secure 0 0 0) 0x7f921c1443b0 con 0x7f91fc066f60 2024-04-25T08:53:59.903 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:53:59.905 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:53:59.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 -- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91fc066f60 msgr2=0x7f91fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91fc066f60 0x7f91fc069420 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f921c0cbcd0 tx=0x7f9204008040 comp rx=0 tx=0).stop 2024-04-25T08:53:59.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 -- 172.21.15.37:0/270906049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921c06ec60 msgr2=0x7f921c13ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:59.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921c06ec60 0x7f921c13ce70 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f920c0b3040 tx=0x7f920c01baf0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 -- 172.21.15.37:0/270906049 shutdown_connections 2024-04-25T08:53:59.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91fc066f60 0x7f91fc069420 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f921c144ca0 0x7f921c0caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f921c0ce510 0x7f921c0caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 --2- 172.21.15.37:0/270906049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f921c06ec60 0x7f921c13ce70 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:59.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 -- 172.21.15.37:0/270906049 >> 172.21.15.37:0/270906049 conn(0x7f921c0b11a0 msgr2=0x7f921c0718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:59.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 -- 172.21.15.37:0/270906049 shutdown_connections 2024-04-25T08:53:59.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:53:59.912+0000 7f9222211640 1 -- 172.21.15.37:0/270906049 wait complete. 2024-04-25T08:53:59.920 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-04-25T08:53:59.152083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229448,"kb_used_data":4912,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745272,"statfs":{"total":773094113280,"available":772859158528,"internally_reserved":0,"allocated":5029888,"data_stored":2840780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.334439"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872415+0000","last_change":"2024-04-25T08:53:55.877266+0000","last_active":"2024-04-25T08:53:56.872415+0000","last_peered":"2024-04-25T08:53:56.872415+0000","last_clean":"2024-04-25T08:53:56.872415+0000","last_became_active":"2024-04-25T08:53:55.876917+0000","last_became_peered":"2024-04-25T08:53:55.876917+0000","last_unstale":"2024-04-25T08:53:56.872415+0000","last_undegraded":"2024-04-25T08:53:56.872415+0000","last_fullsized":"2024-04-25T08:53:56.872415+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.932428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871794+0000","last_change":"2024-04-25T08:53:55.869176+0000","last_active":"2024-04-25T08:53:56.871794+0000","last_peered":"2024-04-25T08:53:56.871794+0000","last_clean":"2024-04-25T08:53:56.871794+0000","last_became_active":"2024-04-25T08:53:55.868932+0000","last_became_peered":"2024-04-25T08:53:55.868932+0000","last_unstale":"2024-04-25T08:53:56.871794+0000","last_undegraded":"2024-04-25T08:53:56.871794+0000","last_fullsized":"2024-04-25T08:53:56.871794+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:42.119355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.927587+0000","last_change":"2024-04-25T08:53:55.924062+0000","last_active":"2024-04-25T08:53:55.927587+0000","last_peered":"2024-04-25T08:53:55.927587+0000","last_clean":"2024-04-25T08:53:55.927587+0000","last_became_active":"2024-04-25T08:53:55.923643+0000","last_became_peered":"2024-04-25T08:53:55.923643+0000","last_unstale":"2024-04-25T08:53:55.927587+0000","last_undegraded":"2024-04-25T08:53:55.927587+0000","last_fullsized":"2024-04-25T08:53:55.927587+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:54.520630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405636+0000","last_change":"2024-04-25T08:53:55.868687+0000","last_active":"2024-04-25T08:53:57.405636+0000","last_peered":"2024-04-25T08:53:57.405636+0000","last_clean":"2024-04-25T08:53:57.405636+0000","last_became_active":"2024-04-25T08:53:55.867467+0000","last_became_peered":"2024-04-25T08:53:55.867467+0000","last_unstale":"2024-04-25T08:53:57.405636+0000","last_undegraded":"2024-04-25T08:53:57.405636+0000","last_fullsized":"2024-04-25T08:53:57.405636+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:19.142992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405571+0000","last_change":"2024-04-25T08:53:55.886467+0000","last_active":"2024-04-25T08:53:57.405571+0000","last_peered":"2024-04-25T08:53:57.405571+0000","last_clean":"2024-04-25T08:53:57.405571+0000","last_became_active":"2024-04-25T08:53:55.886194+0000","last_became_peered":"2024-04-25T08:53:55.886194+0000","last_unstale":"2024-04-25T08:53:57.405571+0000","last_undegraded":"2024-04-25T08:53:57.405571+0000","last_fullsized":"2024-04-25T08:53:57.405571+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:57.316851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.442922+0000","last_change":"2024-04-25T08:53:55.874974+0000","last_active":"2024-04-25T08:53:57.442922+0000","last_peered":"2024-04-25T08:53:57.442922+0000","last_clean":"2024-04-25T08:53:57.442922+0000","last_became_active":"2024-04-25T08:53:55.874628+0000","last_became_peered":"2024-04-25T08:53:55.874628+0000","last_unstale":"2024-04-25T08:53:57.442922+0000","last_undegraded":"2024-04-25T08:53:57.442922+0000","last_fullsized":"2024-04-25T08:53:57.442922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:18.983179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759521+0000","last_change":"2024-04-25T08:53:55.914301+0000","last_active":"2024-04-25T08:53:57.759521+0000","last_peered":"2024-04-25T08:53:57.759521+0000","last_clean":"2024-04-25T08:53:57.759521+0000","last_became_active":"2024-04-25T08:53:55.913295+0000","last_became_peered":"2024-04-25T08:53:55.913295+0000","last_unstale":"2024-04-25T08:53:57.759521+0000","last_undegraded":"2024-04-25T08:53:57.759521+0000","last_fullsized":"2024-04-25T08:53:57.759521+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:34.677841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871146+0000","last_change":"2024-04-25T08:53:55.923705+0000","last_active":"2024-04-25T08:53:56.871146+0000","last_peered":"2024-04-25T08:53:56.871146+0000","last_clean":"2024-04-25T08:53:56.871146+0000","last_became_active":"2024-04-25T08:53:55.923349+0000","last_became_peered":"2024-04-25T08:53:55.923349+0000","last_unstale":"2024-04-25T08:53:56.871146+0000","last_undegraded":"2024-04-25T08:53:56.871146+0000","last_fullsized":"2024-04-25T08:53:56.871146+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:48.556932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.925250+0000","last_change":"2024-04-25T08:53:55.923872+0000","last_active":"2024-04-25T08:53:55.925250+0000","last_peered":"2024-04-25T08:53:55.925250+0000","last_clean":"2024-04-25T08:53:55.925250+0000","last_became_active":"2024-04-25T08:53:55.923553+0000","last_became_peered":"2024-04-25T08:53:55.923553+0000","last_unstale":"2024-04-25T08:53:55.925250+0000","last_undegraded":"2024-04-25T08:53:55.925250+0000","last_fullsized":"2024-04-25T08:53:55.925250+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:04.958660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760114+0000","last_change":"2024-04-25T08:53:55.907329+0000","last_active":"2024-04-25T08:53:57.760114+0000","last_peered":"2024-04-25T08:53:57.760114+0000","last_clean":"2024-04-25T08:53:57.760114+0000","last_became_active":"2024-04-25T08:53:55.900764+0000","last_became_peered":"2024-04-25T08:53:55.900764+0000","last_unstale":"2024-04-25T08:53:57.760114+0000","last_undegraded":"2024-04-25T08:53:57.760114+0000","last_fullsized":"2024-04-25T08:53:57.760114+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:54.232369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760512+0000","last_change":"2024-04-25T08:53:55.913130+0000","last_active":"2024-04-25T08:53:57.760512+0000","last_peered":"2024-04-25T08:53:57.760512+0000","last_clean":"2024-04-25T08:53:57.760512+0000","last_became_active":"2024-04-25T08:53:55.908809+0000","last_became_peered":"2024-04-25T08:53:55.908809+0000","last_unstale":"2024-04-25T08:53:57.760512+0000","last_undegraded":"2024-04-25T08:53:57.760512+0000","last_fullsized":"2024-04-25T08:53:57.760512+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:48.556932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.876342+0000","last_change":"2024-04-25T08:53:55.868505+0000","last_active":"2024-04-25T08:53:55.876342+0000","last_peered":"2024-04-25T08:53:55.876342+0000","last_clean":"2024-04-25T08:53:55.876342+0000","last_became_active":"2024-04-25T08:53:55.868179+0000","last_became_peered":"2024-04-25T08:53:55.868179+0000","last_unstale":"2024-04-25T08:53:55.876342+0000","last_undegraded":"2024-04-25T08:53:55.876342+0000","last_fullsized":"2024-04-25T08:53:55.876342+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:06.183084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759675+0000","last_change":"2024-04-25T08:53:55.908755+0000","last_active":"2024-04-25T08:53:57.759675+0000","last_peered":"2024-04-25T08:53:57.759675+0000","last_clean":"2024-04-25T08:53:57.759675+0000","last_became_active":"2024-04-25T08:53:55.908077+0000","last_became_peered":"2024-04-25T08:53:55.908077+0000","last_unstale":"2024-04-25T08:53:57.759675+0000","last_undegraded":"2024-04-25T08:53:57.759675+0000","last_fullsized":"2024-04-25T08:53:57.759675+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:21.902900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.870454+0000","last_change":"2024-04-25T08:53:55.917143+0000","last_active":"2024-04-25T08:53:56.870454+0000","last_peered":"2024-04-25T08:53:56.870454+0000","last_clean":"2024-04-25T08:53:56.870454+0000","last_became_active":"2024-04-25T08:53:55.916546+0000","last_became_peered":"2024-04-25T08:53:55.916546+0000","last_unstale":"2024-04-25T08:53:56.870454+0000","last_undegraded":"2024-04-25T08:53:56.870454+0000","last_fullsized":"2024-04-25T08:53:56.870454+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:33.836787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759609+0000","last_change":"2024-04-25T08:53:55.914127+0000","last_active":"2024-04-25T08:53:57.759609+0000","last_peered":"2024-04-25T08:53:57.759609+0000","last_clean":"2024-04-25T08:53:57.759609+0000","last_became_active":"2024-04-25T08:53:55.913063+0000","last_became_peered":"2024-04-25T08:53:55.913063+0000","last_unstale":"2024-04-25T08:53:57.759609+0000","last_undegraded":"2024-04-25T08:53:57.759609+0000","last_fullsized":"2024-04-25T08:53:57.759609+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:06.399307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.338998+0000","last_change":"2024-04-25T08:53:55.920744+0000","last_active":"2024-04-25T08:53:58.338998+0000","last_peered":"2024-04-25T08:53:58.338998+0000","last_clean":"2024-04-25T08:53:58.338998+0000","last_became_active":"2024-04-25T08:53:55.920154+0000","last_became_peered":"2024-04-25T08:53:55.920154+0000","last_unstale":"2024-04-25T08:53:58.338998+0000","last_undegraded":"2024-04-25T08:53:58.338998+0000","last_fullsized":"2024-04-25T08:53:58.338998+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:54.520630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339640+0000","last_change":"2024-04-25T08:53:55.917871+0000","last_active":"2024-04-25T08:53:58.339640+0000","last_peered":"2024-04-25T08:53:58.339640+0000","last_clean":"2024-04-25T08:53:58.339640+0000","last_became_active":"2024-04-25T08:53:55.917278+0000","last_became_peered":"2024-04-25T08:53:55.917278+0000","last_unstale":"2024-04-25T08:53:58.339640+0000","last_undegraded":"2024-04-25T08:53:58.339640+0000","last_fullsized":"2024-04-25T08:53:58.339640+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:18.983179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872876+0000","last_change":"2024-04-25T08:53:55.912964+0000","last_active":"2024-04-25T08:53:56.872876+0000","last_peered":"2024-04-25T08:53:56.872876+0000","last_clean":"2024-04-25T08:53:56.872876+0000","last_became_active":"2024-04-25T08:53:55.912342+0000","last_became_peered":"2024-04-25T08:53:55.912342+0000","last_unstale":"2024-04-25T08:53:56.872876+0000","last_undegraded":"2024-04-25T08:53:56.872876+0000","last_fullsized":"2024-04-25T08:53:56.872876+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:28.625065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405642+0000","last_change":"2024-04-25T08:53:55.868880+0000","last_active":"2024-04-25T08:53:57.405642+0000","last_peered":"2024-04-25T08:53:57.405642+0000","last_clean":"2024-04-25T08:53:57.405642+0000","last_became_active":"2024-04-25T08:53:55.868633+0000","last_became_peered":"2024-04-25T08:53:55.868633+0000","last_unstale":"2024-04-25T08:53:57.405642+0000","last_undegraded":"2024-04-25T08:53:57.405642+0000","last_fullsized":"2024-04-25T08:53:57.405642+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:47.707589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.870994+0000","last_change":"2024-04-25T08:53:55.913228+0000","last_active":"2024-04-25T08:53:56.870994+0000","last_peered":"2024-04-25T08:53:56.870994+0000","last_clean":"2024-04-25T08:53:56.870994+0000","last_became_active":"2024-04-25T08:53:55.912697+0000","last_became_peered":"2024-04-25T08:53:55.912697+0000","last_unstale":"2024-04-25T08:53:56.870994+0000","last_undegraded":"2024-04-25T08:53:56.870994+0000","last_fullsized":"2024-04-25T08:53:56.870994+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:13.647839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759793+0000","last_change":"2024-04-25T08:53:55.908433+0000","last_active":"2024-04-25T08:53:57.759793+0000","last_peered":"2024-04-25T08:53:57.759793+0000","last_clean":"2024-04-25T08:53:57.759793+0000","last_became_active":"2024-04-25T08:53:55.907886+0000","last_became_peered":"2024-04-25T08:53:55.907886+0000","last_unstale":"2024-04-25T08:53:57.759793+0000","last_undegraded":"2024-04-25T08:53:57.759793+0000","last_fullsized":"2024-04-25T08:53:57.759793+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:01.341183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871440+0000","last_change":"2024-04-25T08:53:55.871600+0000","last_active":"2024-04-25T08:53:56.871440+0000","last_peered":"2024-04-25T08:53:56.871440+0000","last_clean":"2024-04-25T08:53:56.871440+0000","last_became_active":"2024-04-25T08:53:55.871060+0000","last_became_peered":"2024-04-25T08:53:55.871060+0000","last_unstale":"2024-04-25T08:53:56.871440+0000","last_undegraded":"2024-04-25T08:53:56.871440+0000","last_fullsized":"2024-04-25T08:53:56.871440+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.163826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339160+0000","last_change":"2024-04-25T08:53:55.886801+0000","last_active":"2024-04-25T08:53:58.339160+0000","last_peered":"2024-04-25T08:53:58.339160+0000","last_clean":"2024-04-25T08:53:58.339160+0000","last_became_active":"2024-04-25T08:53:55.885970+0000","last_became_peered":"2024-04-25T08:53:55.885970+0000","last_unstale":"2024-04-25T08:53:58.339160+0000","last_undegraded":"2024-04-25T08:53:58.339160+0000","last_fullsized":"2024-04-25T08:53:58.339160+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:54.524193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405170+0000","last_change":"2024-04-25T08:53:55.892289+0000","last_active":"2024-04-25T08:53:57.405170+0000","last_peered":"2024-04-25T08:53:57.405170+0000","last_clean":"2024-04-25T08:53:57.405170+0000","last_became_active":"2024-04-25T08:53:55.891999+0000","last_became_peered":"2024-04-25T08:53:55.891999+0000","last_unstale":"2024-04-25T08:53:57.405170+0000","last_undegraded":"2024-04-25T08:53:57.405170+0000","last_fullsized":"2024-04-25T08:53:57.405170+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:21.479704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.873337+0000","last_change":"2024-04-25T08:53:55.897159+0000","last_active":"2024-04-25T08:53:56.873337+0000","last_peered":"2024-04-25T08:53:56.873337+0000","last_clean":"2024-04-25T08:53:56.873337+0000","last_became_active":"2024-04-25T08:53:55.896451+0000","last_became_peered":"2024-04-25T08:53:55.896451+0000","last_unstale":"2024-04-25T08:53:56.873337+0000","last_undegraded":"2024-04-25T08:53:56.873337+0000","last_fullsized":"2024-04-25T08:53:56.873337+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:23.170169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.443599+0000","last_change":"2024-04-25T08:53:55.902092+0000","last_active":"2024-04-25T08:53:57.443599+0000","last_peered":"2024-04-25T08:53:57.443599+0000","last_clean":"2024-04-25T08:53:57.443599+0000","last_became_active":"2024-04-25T08:53:55.901811+0000","last_became_peered":"2024-04-25T08:53:55.901811+0000","last_unstale":"2024-04-25T08:53:57.443599+0000","last_undegraded":"2024-04-25T08:53:57.443599+0000","last_fullsized":"2024-04-25T08:53:57.443599+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:42.981286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872430+0000","last_change":"2024-04-25T08:53:55.902629+0000","last_active":"2024-04-25T08:53:56.872430+0000","last_peered":"2024-04-25T08:53:56.872430+0000","last_clean":"2024-04-25T08:53:56.872430+0000","last_became_active":"2024-04-25T08:53:55.881384+0000","last_became_peered":"2024-04-25T08:53:55.881384+0000","last_unstale":"2024-04-25T08:53:56.872430+0000","last_undegraded":"2024-04-25T08:53:56.872430+0000","last_fullsized":"2024-04-25T08:53:56.872430+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:51.758262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760358+0000","last_change":"2024-04-25T08:53:55.908769+0000","last_active":"2024-04-25T08:53:57.760358+0000","last_peered":"2024-04-25T08:53:57.760358+0000","last_clean":"2024-04-25T08:53:57.760358+0000","last_became_active":"2024-04-25T08:53:55.907335+0000","last_became_peered":"2024-04-25T08:53:55.907335+0000","last_unstale":"2024-04-25T08:53:57.760358+0000","last_undegraded":"2024-04-25T08:53:57.760358+0000","last_fullsized":"2024-04-25T08:53:57.760358+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:09.318991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872227+0000","last_change":"2024-04-25T08:53:55.911461+0000","last_active":"2024-04-25T08:53:56.872227+0000","last_peered":"2024-04-25T08:53:56.872227+0000","last_clean":"2024-04-25T08:53:56.872227+0000","last_became_active":"2024-04-25T08:53:55.903300+0000","last_became_peered":"2024-04-25T08:53:55.903300+0000","last_unstale":"2024-04-25T08:53:56.872227+0000","last_undegraded":"2024-04-25T08:53:56.872227+0000","last_fullsized":"2024-04-25T08:53:56.872227+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:06.183084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871184+0000","last_change":"2024-04-25T08:53:55.903796+0000","last_active":"2024-04-25T08:53:56.871184+0000","last_peered":"2024-04-25T08:53:56.871184+0000","last_clean":"2024-04-25T08:53:56.871184+0000","last_became_active":"2024-04-25T08:53:55.903049+0000","last_became_peered":"2024-04-25T08:53:55.903049+0000","last_unstale":"2024-04-25T08:53:56.871184+0000","last_undegraded":"2024-04-25T08:53:56.871184+0000","last_fullsized":"2024-04-25T08:53:56.871184+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:12.913824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339332+0000","last_change":"2024-04-25T08:53:55.886832+0000","last_active":"2024-04-25T08:53:58.339332+0000","last_peered":"2024-04-25T08:53:58.339332+0000","last_clean":"2024-04-25T08:53:58.339332+0000","last_became_active":"2024-04-25T08:53:55.886068+0000","last_became_peered":"2024-04-25T08:53:55.886068+0000","last_unstale":"2024-04-25T08:53:58.339332+0000","last_undegraded":"2024-04-25T08:53:58.339332+0000","last_fullsized":"2024-04-25T08:53:58.339332+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:33.836787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.442739+0000","last_change":"2024-04-25T08:53:55.869089+0000","last_active":"2024-04-25T08:53:57.442739+0000","last_peered":"2024-04-25T08:53:57.442739+0000","last_clean":"2024-04-25T08:53:57.442739+0000","last_became_active":"2024-04-25T08:53:55.868734+0000","last_became_peered":"2024-04-25T08:53:55.868734+0000","last_unstale":"2024-04-25T08:53:57.442739+0000","last_undegraded":"2024-04-25T08:53:57.442739+0000","last_fullsized":"2024-04-25T08:53:57.442739+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:33.836787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339756+0000","last_change":"2024-04-25T08:53:55.910330+0000","last_active":"2024-04-25T08:53:58.339756+0000","last_peered":"2024-04-25T08:53:58.339756+0000","last_clean":"2024-04-25T08:53:58.339756+0000","last_became_active":"2024-04-25T08:53:55.909744+0000","last_became_peered":"2024-04-25T08:53:55.909744+0000","last_unstale":"2024-04-25T08:53:58.339756+0000","last_undegraded":"2024-04-25T08:53:58.339756+0000","last_fullsized":"2024-04-25T08:53:58.339756+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:28.381438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.873418+0000","last_change":"2024-04-25T08:53:54.851744+0000","last_active":"2024-04-25T08:53:56.873418+0000","last_peered":"2024-04-25T08:53:56.873418+0000","last_clean":"2024-04-25T08:53:56.873418+0000","last_became_active":"2024-04-25T08:53:54.851056+0000","last_became_peered":"2024-04-25T08:53:54.851056+0000","last_unstale":"2024-04-25T08:53:56.873418+0000","last_undegraded":"2024-04-25T08:53:56.873418+0000","last_fullsized":"2024-04-25T08:53:56.873418+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:46.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.443498+0000","last_change":"2024-04-25T08:53:55.924008+0000","last_active":"2024-04-25T08:53:57.443498+0000","last_peered":"2024-04-25T08:53:57.443498+0000","last_clean":"2024-04-25T08:53:57.443498+0000","last_became_active":"2024-04-25T08:53:55.923214+0000","last_became_peered":"2024-04-25T08:53:55.923214+0000","last_unstale":"2024-04-25T08:53:57.443498+0000","last_undegraded":"2024-04-25T08:53:57.443498+0000","last_fullsized":"2024-04-25T08:53:57.443498+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.163826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.887626+0000","last_change":"2024-04-25T08:53:54.862894+0000","last_active":"2024-04-25T08:53:55.887626+0000","last_peered":"2024-04-25T08:53:55.887626+0000","last_clean":"2024-04-25T08:53:55.887626+0000","last_became_active":"2024-04-25T08:53:54.862250+0000","last_became_peered":"2024-04-25T08:53:54.862250+0000","last_unstale":"2024-04-25T08:53:55.887626+0000","last_undegraded":"2024-04-25T08:53:55.887626+0000","last_fullsized":"2024-04-25T08:53:55.887626+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.214318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871380+0000","last_change":"2024-04-25T08:53:55.912519+0000","last_active":"2024-04-25T08:53:56.871380+0000","last_peered":"2024-04-25T08:53:56.871380+0000","last_clean":"2024-04-25T08:53:56.871380+0000","last_became_active":"2024-04-25T08:53:55.911965+0000","last_became_peered":"2024-04-25T08:53:55.911965+0000","last_unstale":"2024-04-25T08:53:56.871380+0000","last_undegraded":"2024-04-25T08:53:56.871380+0000","last_fullsized":"2024-04-25T08:53:56.871380+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:28.381438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.443756+0000","last_change":"2024-04-25T08:53:54.858661+0000","last_active":"2024-04-25T08:53:57.443756+0000","last_peered":"2024-04-25T08:53:57.443756+0000","last_clean":"2024-04-25T08:53:57.443756+0000","last_became_active":"2024-04-25T08:53:54.858376+0000","last_became_peered":"2024-04-25T08:53:54.858376+0000","last_unstale":"2024-04-25T08:53:57.443756+0000","last_undegraded":"2024-04-25T08:53:57.443756+0000","last_fullsized":"2024-04-25T08:53:57.443756+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:05.174377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872319+0000","last_change":"2024-04-25T08:53:55.913009+0000","last_active":"2024-04-25T08:53:56.872319+0000","last_peered":"2024-04-25T08:53:56.872319+0000","last_clean":"2024-04-25T08:53:56.872319+0000","last_became_active":"2024-04-25T08:53:55.912537+0000","last_became_peered":"2024-04-25T08:53:55.912537+0000","last_unstale":"2024-04-25T08:53:56.872319+0000","last_undegraded":"2024-04-25T08:53:56.872319+0000","last_fullsized":"2024-04-25T08:53:56.872319+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:53.486036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.873170+0000","last_change":"2024-04-25T08:53:54.847858+0000","last_active":"2024-04-25T08:53:56.873170+0000","last_peered":"2024-04-25T08:53:56.873170+0000","last_clean":"2024-04-25T08:53:56.873170+0000","last_became_active":"2024-04-25T08:53:54.847100+0000","last_became_peered":"2024-04-25T08:53:54.847100+0000","last_unstale":"2024-04-25T08:53:56.873170+0000","last_undegraded":"2024-04-25T08:53:56.873170+0000","last_fullsized":"2024-04-25T08:53:56.873170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.214318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.876419+0000","last_change":"2024-04-25T08:53:55.865056+0000","last_active":"2024-04-25T08:53:55.876419+0000","last_peered":"2024-04-25T08:53:55.876419+0000","last_clean":"2024-04-25T08:53:55.876419+0000","last_became_active":"2024-04-25T08:53:55.864765+0000","last_became_peered":"2024-04-25T08:53:55.864765+0000","last_unstale":"2024-04-25T08:53:55.876419+0000","last_undegraded":"2024-04-25T08:53:55.876419+0000","last_fullsized":"2024-04-25T08:53:55.876419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:12.913824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760437+0000","last_change":"2024-04-25T08:53:54.862961+0000","last_active":"2024-04-25T08:53:57.760437+0000","last_peered":"2024-04-25T08:53:57.760437+0000","last_clean":"2024-04-25T08:53:57.760437+0000","last_became_active":"2024-04-25T08:53:54.862258+0000","last_became_peered":"2024-04-25T08:53:54.862258+0000","last_unstale":"2024-04-25T08:53:57.760437+0000","last_undegraded":"2024-04-25T08:53:57.760437+0000","last_fullsized":"2024-04-25T08:53:57.760437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:05.174377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759939+0000","last_change":"2024-04-25T08:53:55.915599+0000","last_active":"2024-04-25T08:53:57.759939+0000","last_peered":"2024-04-25T08:53:57.759939+0000","last_clean":"2024-04-25T08:53:57.759939+0000","last_became_active":"2024-04-25T08:53:55.913728+0000","last_became_peered":"2024-04-25T08:53:55.913728+0000","last_unstale":"2024-04-25T08:53:57.759939+0000","last_undegraded":"2024-04-25T08:53:57.759939+0000","last_fullsized":"2024-04-25T08:53:57.759939+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:26.688821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339470+0000","last_change":"2024-04-25T08:53:54.867415+0000","last_active":"2024-04-25T08:53:58.339470+0000","last_peered":"2024-04-25T08:53:58.339470+0000","last_clean":"2024-04-25T08:53:58.339470+0000","last_became_active":"2024-04-25T08:53:54.864476+0000","last_became_peered":"2024-04-25T08:53:54.864476+0000","last_unstale":"2024-04-25T08:53:58.339470+0000","last_undegraded":"2024-04-25T08:53:58.339470+0000","last_fullsized":"2024-04-25T08:53:58.339470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:18.134284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871971+0000","last_change":"2024-04-25T08:53:55.902059+0000","last_active":"2024-04-25T08:53:56.871971+0000","last_peered":"2024-04-25T08:53:56.871971+0000","last_clean":"2024-04-25T08:53:56.871971+0000","last_became_active":"2024-04-25T08:53:55.901602+0000","last_became_peered":"2024-04-25T08:53:55.901602+0000","last_unstale":"2024-04-25T08:53:56.871971+0000","last_undegraded":"2024-04-25T08:53:56.871971+0000","last_fullsized":"2024-04-25T08:53:56.871971+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:42.981286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.443269+0000","last_change":"2024-04-25T08:53:54.873269+0000","last_active":"2024-04-25T08:53:57.443269+0000","last_peered":"2024-04-25T08:53:57.443269+0000","last_clean":"2024-04-25T08:53:57.443269+0000","last_became_active":"2024-04-25T08:53:54.873022+0000","last_became_peered":"2024-04-25T08:53:54.873022+0000","last_unstale":"2024-04-25T08:53:57.443269+0000","last_undegraded":"2024-04-25T08:53:57.443269+0000","last_fullsized":"2024-04-25T08:53:57.443269+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.214318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871535+0000","last_change":"2024-04-25T08:53:55.866850+0000","last_active":"2024-04-25T08:53:56.871535+0000","last_peered":"2024-04-25T08:53:56.871535+0000","last_clean":"2024-04-25T08:53:56.871535+0000","last_became_active":"2024-04-25T08:53:55.866568+0000","last_became_peered":"2024-04-25T08:53:55.866568+0000","last_unstale":"2024-04-25T08:53:56.871535+0000","last_undegraded":"2024-04-25T08:53:56.871535+0000","last_fullsized":"2024-04-25T08:53:56.871535+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:32.159132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.870948+0000","last_change":"2024-04-25T08:53:54.846517+0000","last_active":"2024-04-25T08:53:56.870948+0000","last_peered":"2024-04-25T08:53:56.870948+0000","last_clean":"2024-04-25T08:53:56.870948+0000","last_became_active":"2024-04-25T08:53:54.842958+0000","last_became_peered":"2024-04-25T08:53:54.842958+0000","last_unstale":"2024-04-25T08:53:56.870948+0000","last_undegraded":"2024-04-25T08:53:56.870948+0000","last_fullsized":"2024-04-25T08:53:56.870948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:11.905116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405595+0000","last_change":"2024-04-25T08:53:55.886972+0000","last_active":"2024-04-25T08:53:57.405595+0000","last_peered":"2024-04-25T08:53:57.405595+0000","last_clean":"2024-04-25T08:53:57.405595+0000","last_became_active":"2024-04-25T08:53:55.886216+0000","last_became_peered":"2024-04-25T08:53:55.886216+0000","last_unstale":"2024-04-25T08:53:57.405595+0000","last_undegraded":"2024-04-25T08:53:57.405595+0000","last_fullsized":"2024-04-25T08:53:57.405595+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:59.602303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872584+0000","last_change":"2024-04-25T08:53:54.842673+0000","last_active":"2024-04-25T08:53:56.872584+0000","last_peered":"2024-04-25T08:53:56.872584+0000","last_clean":"2024-04-25T08:53:56.872584+0000","last_became_active":"2024-04-25T08:53:54.842401+0000","last_became_peered":"2024-04-25T08:53:54.842401+0000","last_unstale":"2024-04-25T08:53:56.872584+0000","last_undegraded":"2024-04-25T08:53:56.872584+0000","last_fullsized":"2024-04-25T08:53:56.872584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:29.155118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339828+0000","last_change":"2024-04-25T08:53:55.913304+0000","last_active":"2024-04-25T08:53:58.339828+0000","last_peered":"2024-04-25T08:53:58.339828+0000","last_clean":"2024-04-25T08:53:58.339828+0000","last_became_active":"2024-04-25T08:53:55.912555+0000","last_became_peered":"2024-04-25T08:53:55.912555+0000","last_unstale":"2024-04-25T08:53:58.339828+0000","last_undegraded":"2024-04-25T08:53:58.339828+0000","last_fullsized":"2024-04-25T08:53:58.339828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:42.981286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760483+0000","last_change":"2024-04-25T08:53:54.871214+0000","last_active":"2024-04-25T08:53:57.760483+0000","last_peered":"2024-04-25T08:53:57.760483+0000","last_clean":"2024-04-25T08:53:57.760483+0000","last_became_active":"2024-04-25T08:53:54.864020+0000","last_became_peered":"2024-04-25T08:53:54.864020+0000","last_unstale":"2024-04-25T08:53:57.760483+0000","last_undegraded":"2024-04-25T08:53:57.760483+0000","last_fullsized":"2024-04-25T08:53:57.760483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:12.639132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.892513+0000","last_change":"2024-04-25T08:53:55.888098+0000","last_active":"2024-04-25T08:53:55.892513+0000","last_peered":"2024-04-25T08:53:55.892513+0000","last_clean":"2024-04-25T08:53:55.892513+0000","last_became_active":"2024-04-25T08:53:55.887802+0000","last_became_peered":"2024-04-25T08:53:55.887802+0000","last_unstale":"2024-04-25T08:53:55.892513+0000","last_undegraded":"2024-04-25T08:53:55.892513+0000","last_fullsized":"2024-04-25T08:53:55.892513+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:28.625065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760666+0000","last_change":"2024-04-25T08:53:54.873385+0000","last_active":"2024-04-25T08:53:57.760666+0000","last_peered":"2024-04-25T08:53:57.760666+0000","last_clean":"2024-04-25T08:53:57.760666+0000","last_became_active":"2024-04-25T08:53:54.872382+0000","last_became_peered":"2024-04-25T08:53:54.872382+0000","last_unstale":"2024-04-25T08:53:57.760666+0000","last_undegraded":"2024-04-25T08:53:57.760666+0000","last_fullsized":"2024-04-25T08:53:57.760666+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:27.616358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.870922+0000","last_change":"2024-04-25T08:53:55.873437+0000","last_active":"2024-04-25T08:53:56.870922+0000","last_peered":"2024-04-25T08:53:56.870922+0000","last_clean":"2024-04-25T08:53:56.870922+0000","last_became_active":"2024-04-25T08:53:55.872710+0000","last_became_peered":"2024-04-25T08:53:55.872710+0000","last_unstale":"2024-04-25T08:53:56.870922+0000","last_undegraded":"2024-04-25T08:53:56.870922+0000","last_fullsized":"2024-04-25T08:53:56.870922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:18.983179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872503+0000","last_change":"2024-04-25T08:53:54.865425+0000","last_active":"2024-04-25T08:53:56.872503+0000","last_peered":"2024-04-25T08:53:56.872503+0000","last_clean":"2024-04-25T08:53:56.872503+0000","last_became_active":"2024-04-25T08:53:54.865084+0000","last_became_peered":"2024-04-25T08:53:54.865084+0000","last_unstale":"2024-04-25T08:53:56.872503+0000","last_undegraded":"2024-04-25T08:53:56.872503+0000","last_fullsized":"2024-04-25T08:53:56.872503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:17.974471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.927613+0000","last_change":"2024-04-25T08:53:55.924625+0000","last_active":"2024-04-25T08:53:55.927613+0000","last_peered":"2024-04-25T08:53:55.927613+0000","last_clean":"2024-04-25T08:53:55.927613+0000","last_became_active":"2024-04-25T08:53:55.923408+0000","last_became_peered":"2024-04-25T08:53:55.923408+0000","last_unstale":"2024-04-25T08:53:55.927613+0000","last_undegraded":"2024-04-25T08:53:55.927613+0000","last_fullsized":"2024-04-25T08:53:55.927613+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:43.538676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872440+0000","last_change":"2024-04-25T08:53:54.841968+0000","last_active":"2024-04-25T08:53:56.872440+0000","last_peered":"2024-04-25T08:53:56.872440+0000","last_clean":"2024-04-25T08:53:56.872440+0000","last_became_active":"2024-04-25T08:53:54.841614+0000","last_became_peered":"2024-04-25T08:53:54.841614+0000","last_unstale":"2024-04-25T08:53:56.872440+0000","last_undegraded":"2024-04-25T08:53:56.872440+0000","last_fullsized":"2024-04-25T08:53:56.872440+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:41.972578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871305+0000","last_change":"2024-04-25T08:53:55.863560+0000","last_active":"2024-04-25T08:53:56.871305+0000","last_peered":"2024-04-25T08:53:56.871305+0000","last_clean":"2024-04-25T08:53:56.871305+0000","last_became_active":"2024-04-25T08:53:55.863214+0000","last_became_peered":"2024-04-25T08:53:55.863214+0000","last_unstale":"2024-04-25T08:53:56.871305+0000","last_undegraded":"2024-04-25T08:53:56.871305+0000","last_fullsized":"2024-04-25T08:53:56.871305+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:42.712711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.443216+0000","last_change":"2024-04-25T08:53:54.875234+0000","last_active":"2024-04-25T08:53:57.443216+0000","last_peered":"2024-04-25T08:53:57.443216+0000","last_clean":"2024-04-25T08:53:57.443216+0000","last_became_active":"2024-04-25T08:53:54.875031+0000","last_became_peered":"2024-04-25T08:53:54.875031+0000","last_unstale":"2024-04-25T08:53:57.443216+0000","last_undegraded":"2024-04-25T08:53:57.443216+0000","last_fullsized":"2024-04-25T08:53:57.443216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:12.639132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339119+0000","last_change":"2024-04-25T08:53:55.886593+0000","last_active":"2024-04-25T08:53:58.339119+0000","last_peered":"2024-04-25T08:53:58.339119+0000","last_clean":"2024-04-25T08:53:58.339119+0000","last_became_active":"2024-04-25T08:53:55.885815+0000","last_became_peered":"2024-04-25T08:53:55.885815+0000","last_unstale":"2024-04-25T08:53:58.339119+0000","last_undegraded":"2024-04-25T08:53:58.339119+0000","last_fullsized":"2024-04-25T08:53:58.339119+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:43.538676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872623+0000","last_change":"2024-04-25T08:53:54.849856+0000","last_active":"2024-04-25T08:53:56.872623+0000","last_peered":"2024-04-25T08:53:56.872623+0000","last_clean":"2024-04-25T08:53:56.872623+0000","last_became_active":"2024-04-25T08:53:54.849102+0000","last_became_peered":"2024-04-25T08:53:54.849102+0000","last_unstale":"2024-04-25T08:53:56.872623+0000","last_undegraded":"2024-04-25T08:53:56.872623+0000","last_fullsized":"2024-04-25T08:53:56.872623+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:56.308143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871668+0000","last_change":"2024-04-25T08:53:55.864268+0000","last_active":"2024-04-25T08:53:56.871668+0000","last_peered":"2024-04-25T08:53:56.871668+0000","last_clean":"2024-04-25T08:53:56.871668+0000","last_became_active":"2024-04-25T08:53:55.864013+0000","last_became_peered":"2024-04-25T08:53:55.864013+0000","last_unstale":"2024-04-25T08:53:56.871668+0000","last_undegraded":"2024-04-25T08:53:56.871668+0000","last_fullsized":"2024-04-25T08:53:56.871668+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:07.784180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339064+0000","last_change":"2024-04-25T08:53:54.864793+0000","last_active":"2024-04-25T08:53:58.339064+0000","last_peered":"2024-04-25T08:53:58.339064+0000","last_clean":"2024-04-25T08:53:58.339064+0000","last_became_active":"2024-04-25T08:53:54.864140+0000","last_became_peered":"2024-04-25T08:53:54.864140+0000","last_unstale":"2024-04-25T08:53:58.339064+0000","last_undegraded":"2024-04-25T08:53:58.339064+0000","last_fullsized":"2024-04-25T08:53:58.339064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:48.132257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872744+0000","last_change":"2024-04-25T08:53:55.913076+0000","last_active":"2024-04-25T08:53:56.872744+0000","last_peered":"2024-04-25T08:53:56.872744+0000","last_clean":"2024-04-25T08:53:56.872744+0000","last_became_active":"2024-04-25T08:53:55.912794+0000","last_became_peered":"2024-04-25T08:53:55.912794+0000","last_unstale":"2024-04-25T08:53:56.872744+0000","last_undegraded":"2024-04-25T08:53:56.872744+0000","last_fullsized":"2024-04-25T08:53:56.872744+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:59.690305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760509+0000","last_change":"2024-04-25T08:53:54.861175+0000","last_active":"2024-04-25T08:53:57.760509+0000","last_peered":"2024-04-25T08:53:57.760509+0000","last_clean":"2024-04-25T08:53:57.760509+0000","last_became_active":"2024-04-25T08:53:54.860530+0000","last_became_peered":"2024-04-25T08:53:54.860530+0000","last_unstale":"2024-04-25T08:53:57.760509+0000","last_undegraded":"2024-04-25T08:53:57.760509+0000","last_fullsized":"2024-04-25T08:53:57.760509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:11.905116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760575+0000","last_change":"2024-04-25T08:53:55.919306+0000","last_active":"2024-04-25T08:53:57.760575+0000","last_peered":"2024-04-25T08:53:57.760575+0000","last_clean":"2024-04-25T08:53:57.760575+0000","last_became_active":"2024-04-25T08:53:55.908770+0000","last_became_peered":"2024-04-25T08:53:55.908770+0000","last_unstale":"2024-04-25T08:53:57.760575+0000","last_undegraded":"2024-04-25T08:53:57.760575+0000","last_fullsized":"2024-04-25T08:53:57.760575+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:42.712711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871653+0000","last_change":"2024-04-25T08:53:54.867127+0000","last_active":"2024-04-25T08:53:56.871653+0000","last_peered":"2024-04-25T08:53:56.871653+0000","last_clean":"2024-04-25T08:53:56.871653+0000","last_became_active":"2024-04-25T08:53:54.866337+0000","last_became_peered":"2024-04-25T08:53:54.866337+0000","last_unstale":"2024-04-25T08:53:56.871653+0000","last_undegraded":"2024-04-25T08:53:56.871653+0000","last_fullsized":"2024-04-25T08:53:56.871653+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:05.174377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872275+0000","last_change":"2024-04-25T08:53:55.874774+0000","last_active":"2024-04-25T08:53:56.872275+0000","last_peered":"2024-04-25T08:53:56.872275+0000","last_clean":"2024-04-25T08:53:56.872275+0000","last_became_active":"2024-04-25T08:53:55.872554+0000","last_became_peered":"2024-04-25T08:53:55.872554+0000","last_unstale":"2024-04-25T08:53:56.872275+0000","last_undegraded":"2024-04-25T08:53:56.872275+0000","last_fullsized":"2024-04-25T08:53:56.872275+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:47.567171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405568+0000","last_change":"2024-04-25T08:53:54.862001+0000","last_active":"2024-04-25T08:53:57.405568+0000","last_peered":"2024-04-25T08:53:57.405568+0000","last_clean":"2024-04-25T08:53:57.405568+0000","last_became_active":"2024-04-25T08:53:54.861772+0000","last_became_peered":"2024-04-25T08:53:54.861772+0000","last_unstale":"2024-04-25T08:53:57.405568+0000","last_undegraded":"2024-04-25T08:53:57.405568+0000","last_fullsized":"2024-04-25T08:53:57.405568+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:33.123852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405957+0000","last_change":"2024-04-25T08:53:55.874265+0000","last_active":"2024-04-25T08:53:57.405957+0000","last_peered":"2024-04-25T08:53:57.405957+0000","last_clean":"2024-04-25T08:53:57.405957+0000","last_became_active":"2024-04-25T08:53:55.873901+0000","last_became_peered":"2024-04-25T08:53:55.873901+0000","last_unstale":"2024-04-25T08:53:57.405957+0000","last_undegraded":"2024-04-25T08:53:57.405957+0000","last_fullsized":"2024-04-25T08:53:57.405957+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:37.446552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.442928+0000","last_change":"2024-04-25T08:53:54.855073+0000","last_active":"2024-04-25T08:53:57.442928+0000","last_peered":"2024-04-25T08:53:57.442928+0000","last_clean":"2024-04-25T08:53:57.442928+0000","last_became_active":"2024-04-25T08:53:54.854784+0000","last_became_peered":"2024-04-25T08:53:54.854784+0000","last_unstale":"2024-04-25T08:53:57.442928+0000","last_undegraded":"2024-04-25T08:53:57.442928+0000","last_fullsized":"2024-04-25T08:53:57.442928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:11.905116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339445+0000","last_change":"2024-04-25T08:53:55.892170+0000","last_active":"2024-04-25T08:53:58.339445+0000","last_peered":"2024-04-25T08:53:58.339445+0000","last_clean":"2024-04-25T08:53:58.339445+0000","last_became_active":"2024-04-25T08:53:55.891404+0000","last_became_peered":"2024-04-25T08:53:55.891404+0000","last_unstale":"2024-04-25T08:53:58.339445+0000","last_undegraded":"2024-04-25T08:53:58.339445+0000","last_fullsized":"2024-04-25T08:53:58.339445+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:58.292874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405185+0000","last_change":"2024-04-25T08:53:54.846561+0000","last_active":"2024-04-25T08:53:57.405185+0000","last_peered":"2024-04-25T08:53:57.405185+0000","last_clean":"2024-04-25T08:53:57.405185+0000","last_became_active":"2024-04-25T08:53:54.845972+0000","last_became_peered":"2024-04-25T08:53:54.845972+0000","last_unstale":"2024-04-25T08:53:57.405185+0000","last_undegraded":"2024-04-25T08:53:57.405185+0000","last_fullsized":"2024-04-25T08:53:57.405185+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:08.561873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.892547+0000","last_change":"2024-04-25T08:53:55.888416+0000","last_active":"2024-04-25T08:53:55.892547+0000","last_peered":"2024-04-25T08:53:55.892547+0000","last_clean":"2024-04-25T08:53:55.892547+0000","last_became_active":"2024-04-25T08:53:55.888086+0000","last_became_peered":"2024-04-25T08:53:55.888086+0000","last_unstale":"2024-04-25T08:53:55.892547+0000","last_undegraded":"2024-04-25T08:53:55.892547+0000","last_fullsized":"2024-04-25T08:53:55.892547+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:23.170169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405193+0000","last_change":"2024-04-25T08:53:54.859141+0000","last_active":"2024-04-25T08:53:57.405193+0000","last_peered":"2024-04-25T08:53:57.405193+0000","last_clean":"2024-04-25T08:53:57.405193+0000","last_became_active":"2024-04-25T08:53:54.858807+0000","last_became_peered":"2024-04-25T08:53:54.858807+0000","last_unstale":"2024-04-25T08:53:57.405193+0000","last_undegraded":"2024-04-25T08:53:57.405193+0000","last_fullsized":"2024-04-25T08:53:57.405193+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:43.373208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759811+0000","last_change":"2024-04-25T08:53:55.920669+0000","last_active":"2024-04-25T08:53:57.759811+0000","last_peered":"2024-04-25T08:53:57.759811+0000","last_clean":"2024-04-25T08:53:57.759811+0000","last_became_active":"2024-04-25T08:53:55.919969+0000","last_became_peered":"2024-04-25T08:53:55.919969+0000","last_unstale":"2024-04-25T08:53:57.759811+0000","last_undegraded":"2024-04-25T08:53:57.759811+0000","last_fullsized":"2024-04-25T08:53:57.759811+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:08.899755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339834+0000","last_change":"2024-04-25T08:53:54.869769+0000","last_active":"2024-04-25T08:53:58.339834+0000","last_peered":"2024-04-25T08:53:58.339834+0000","last_clean":"2024-04-25T08:53:58.339834+0000","last_became_active":"2024-04-25T08:53:54.868883+0000","last_became_peered":"2024-04-25T08:53:54.868883+0000","last_unstale":"2024-04-25T08:53:58.339834+0000","last_undegraded":"2024-04-25T08:53:58.339834+0000","last_fullsized":"2024-04-25T08:53:58.339834+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:36.437845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871870+0000","last_change":"2024-04-25T08:53:55.896505+0000","last_active":"2024-04-25T08:53:56.871870+0000","last_peered":"2024-04-25T08:53:56.871870+0000","last_clean":"2024-04-25T08:53:56.871870+0000","last_became_active":"2024-04-25T08:53:55.896187+0000","last_became_peered":"2024-04-25T08:53:55.896187+0000","last_unstale":"2024-04-25T08:53:56.871870+0000","last_undegraded":"2024-04-25T08:53:56.871870+0000","last_fullsized":"2024-04-25T08:53:56.871870+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:50.990569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405060+0000","last_change":"2024-04-25T08:53:54.847178+0000","last_active":"2024-04-25T08:53:57.405060+0000","last_peered":"2024-04-25T08:53:57.405060+0000","last_clean":"2024-04-25T08:53:57.405060+0000","last_became_active":"2024-04-25T08:53:54.846201+0000","last_became_peered":"2024-04-25T08:53:54.846201+0000","last_unstale":"2024-04-25T08:53:57.405060+0000","last_undegraded":"2024-04-25T08:53:57.405060+0000","last_fullsized":"2024-04-25T08:53:57.405060+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.176202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.442967+0000","last_change":"2024-04-25T08:53:55.909242+0000","last_active":"2024-04-25T08:53:57.442967+0000","last_peered":"2024-04-25T08:53:57.442967+0000","last_clean":"2024-04-25T08:53:57.442967+0000","last_became_active":"2024-04-25T08:53:55.909006+0000","last_became_peered":"2024-04-25T08:53:55.909006+0000","last_unstale":"2024-04-25T08:53:57.442967+0000","last_undegraded":"2024-04-25T08:53:57.442967+0000","last_fullsized":"2024-04-25T08:53:57.442967+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:28.381438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760765+0000","last_change":"2024-04-25T08:53:54.873765+0000","last_active":"2024-04-25T08:53:57.760765+0000","last_peered":"2024-04-25T08:53:57.760765+0000","last_clean":"2024-04-25T08:53:57.760765+0000","last_became_active":"2024-04-25T08:53:54.872783+0000","last_became_peered":"2024-04-25T08:53:54.872783+0000","last_unstale":"2024-04-25T08:53:57.760765+0000","last_undegraded":"2024-04-25T08:53:57.760765+0000","last_fullsized":"2024-04-25T08:53:57.760765+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:50.749554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339780+0000","last_change":"2024-04-25T08:53:55.904712+0000","last_active":"2024-04-25T08:53:58.339780+0000","last_peered":"2024-04-25T08:53:58.339780+0000","last_clean":"2024-04-25T08:53:58.339780+0000","last_became_active":"2024-04-25T08:53:55.904084+0000","last_became_peered":"2024-04-25T08:53:55.904084+0000","last_unstale":"2024-04-25T08:53:58.339780+0000","last_undegraded":"2024-04-25T08:53:58.339780+0000","last_fullsized":"2024-04-25T08:53:58.339780+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.163826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405823+0000","last_change":"2024-04-25T08:53:54.866358+0000","last_active":"2024-04-25T08:53:57.405823+0000","last_peered":"2024-04-25T08:53:57.405823+0000","last_clean":"2024-04-25T08:53:57.405823+0000","last_became_active":"2024-04-25T08:53:54.866094+0000","last_became_peered":"2024-04-25T08:53:54.866094+0000","last_unstale":"2024-04-25T08:53:57.405823+0000","last_undegraded":"2024-04-25T08:53:57.405823+0000","last_fullsized":"2024-04-25T08:53:57.405823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.266660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872250+0000","last_change":"2024-04-25T08:53:55.896939+0000","last_active":"2024-04-25T08:53:56.872250+0000","last_peered":"2024-04-25T08:53:56.872250+0000","last_clean":"2024-04-25T08:53:56.872250+0000","last_became_active":"2024-04-25T08:53:55.896625+0000","last_became_peered":"2024-04-25T08:53:55.896625+0000","last_unstale":"2024-04-25T08:53:56.872250+0000","last_undegraded":"2024-04-25T08:53:56.872250+0000","last_fullsized":"2024-04-25T08:53:56.872250+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:25.881602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.861626+0000","last_change":"2024-04-25T08:53:54.842737+0000","last_active":"2024-04-25T08:53:55.861626+0000","last_peered":"2024-04-25T08:53:55.861626+0000","last_clean":"2024-04-25T08:53:55.861626+0000","last_became_active":"2024-04-25T08:53:54.842345+0000","last_became_peered":"2024-04-25T08:53:54.842345+0000","last_unstale":"2024-04-25T08:53:55.861626+0000","last_undegraded":"2024-04-25T08:53:55.861626+0000","last_fullsized":"2024-04-25T08:53:55.861626+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:12.639132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.878849+0000","last_change":"2024-04-25T08:53:55.873870+0000","last_active":"2024-04-25T08:53:55.878849+0000","last_peered":"2024-04-25T08:53:55.878849+0000","last_clean":"2024-04-25T08:53:55.878849+0000","last_became_active":"2024-04-25T08:53:55.873628+0000","last_became_peered":"2024-04-25T08:53:55.873628+0000","last_unstale":"2024-04-25T08:53:55.878849+0000","last_undegraded":"2024-04-25T08:53:55.878849+0000","last_fullsized":"2024-04-25T08:53:55.878849+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:51.758262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.858878+0000","last_change":"2024-04-25T08:53:54.844627+0000","last_active":"2024-04-25T08:53:55.858878+0000","last_peered":"2024-04-25T08:53:55.858878+0000","last_clean":"2024-04-25T08:53:55.858878+0000","last_became_active":"2024-04-25T08:53:54.844100+0000","last_became_peered":"2024-04-25T08:53:54.844100+0000","last_unstale":"2024-04-25T08:53:55.858878+0000","last_undegraded":"2024-04-25T08:53:55.858878+0000","last_fullsized":"2024-04-25T08:53:55.858878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:13.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.442907+0000","last_change":"2024-04-25T08:53:55.883542+0000","last_active":"2024-04-25T08:53:57.442907+0000","last_peered":"2024-04-25T08:53:57.442907+0000","last_clean":"2024-04-25T08:53:57.442907+0000","last_became_active":"2024-04-25T08:53:55.883298+0000","last_became_peered":"2024-04-25T08:53:55.883298+0000","last_unstale":"2024-04-25T08:53:57.442907+0000","last_undegraded":"2024-04-25T08:53:57.442907+0000","last_fullsized":"2024-04-25T08:53:57.442907+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:58.292874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871474+0000","last_change":"2024-04-25T08:53:54.833794+0000","last_active":"2024-04-25T08:53:56.871474+0000","last_peered":"2024-04-25T08:53:56.871474+0000","last_clean":"2024-04-25T08:53:56.871474+0000","last_became_active":"2024-04-25T08:53:54.833089+0000","last_became_peered":"2024-04-25T08:53:54.833089+0000","last_unstale":"2024-04-25T08:53:56.871474+0000","last_undegraded":"2024-04-25T08:53:56.871474+0000","last_fullsized":"2024-04-25T08:53:56.871474+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:58.593596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.870864+0000","last_change":"2024-04-25T08:53:55.886925+0000","last_active":"2024-04-25T08:53:56.870864+0000","last_peered":"2024-04-25T08:53:56.870864+0000","last_clean":"2024-04-25T08:53:56.870864+0000","last_became_active":"2024-04-25T08:53:55.886032+0000","last_became_peered":"2024-04-25T08:53:55.886032+0000","last_unstale":"2024-04-25T08:53:56.870864+0000","last_undegraded":"2024-04-25T08:53:56.870864+0000","last_fullsized":"2024-04-25T08:53:56.870864+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:58.292874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872562+0000","last_change":"2024-04-25T08:53:54.847271+0000","last_active":"2024-04-25T08:53:56.872562+0000","last_peered":"2024-04-25T08:53:56.872562+0000","last_clean":"2024-04-25T08:53:56.872562+0000","last_became_active":"2024-04-25T08:53:54.846542+0000","last_became_peered":"2024-04-25T08:53:54.846542+0000","last_unstale":"2024-04-25T08:53:56.872562+0000","last_undegraded":"2024-04-25T08:53:56.872562+0000","last_fullsized":"2024-04-25T08:53:56.872562+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:12.639132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339876+0000","last_change":"2024-04-25T08:53:55.902565+0000","last_active":"2024-04-25T08:53:58.339876+0000","last_peered":"2024-04-25T08:53:58.339876+0000","last_clean":"2024-04-25T08:53:58.339876+0000","last_became_active":"2024-04-25T08:53:55.901728+0000","last_became_peered":"2024-04-25T08:53:55.901728+0000","last_unstale":"2024-04-25T08:53:58.339876+0000","last_undegraded":"2024-04-25T08:53:58.339876+0000","last_fullsized":"2024-04-25T08:53:58.339876+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:47.567171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871078+0000","last_change":"2024-04-25T08:53:54.842570+0000","last_active":"2024-04-25T08:53:56.871078+0000","last_peered":"2024-04-25T08:53:56.871078+0000","last_clean":"2024-04-25T08:53:56.871078+0000","last_became_active":"2024-04-25T08:53:54.841650+0000","last_became_peered":"2024-04-25T08:53:54.841650+0000","last_unstale":"2024-04-25T08:53:56.871078+0000","last_undegraded":"2024-04-25T08:53:56.871078+0000","last_fullsized":"2024-04-25T08:53:56.871078+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:27.372730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760252+0000","last_change":"2024-04-25T08:53:55.919955+0000","last_active":"2024-04-25T08:53:57.760252+0000","last_peered":"2024-04-25T08:53:57.760252+0000","last_clean":"2024-04-25T08:53:57.760252+0000","last_became_active":"2024-04-25T08:53:55.919277+0000","last_became_peered":"2024-04-25T08:53:55.919277+0000","last_unstale":"2024-04-25T08:53:57.760252+0000","last_undegraded":"2024-04-25T08:53:57.760252+0000","last_fullsized":"2024-04-25T08:53:57.760252+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.226385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339985+0000","last_change":"2024-04-25T08:53:54.871341+0000","last_active":"2024-04-25T08:53:58.339985+0000","last_peered":"2024-04-25T08:53:58.339985+0000","last_clean":"2024-04-25T08:53:58.339985+0000","last_became_active":"2024-04-25T08:53:54.870324+0000","last_became_peered":"2024-04-25T08:53:54.870324+0000","last_unstale":"2024-04-25T08:53:58.339985+0000","last_undegraded":"2024-04-25T08:53:58.339985+0000","last_fullsized":"2024-04-25T08:53:58.339985+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:46.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339536+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:53:58.339536+0000","last_peered":"2024-04-25T08:53:58.339536+0000","last_clean":"2024-04-25T08:53:58.339536+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:53:58.339536+0000","last_undegraded":"2024-04-25T08:53:58.339536+0000","last_fullsized":"2024-04-25T08:53:58.339536+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1073152,"data_stored":793934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":209158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":208688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":209180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":491520,"data_stored":210222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":208646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":208666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":792286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:53:59.925 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:54:00.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f8135270640 1 Processor -- start 2024-04-25T08:54:00.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f8135270640 1 -- start start 2024-04-25T08:54:00.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f8135270640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f813015da80 0x7f81300d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f8135270640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81300d1cd0 0x7f81300d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f8135270640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813006be60 con 0x7f81300d0620 2024-04-25T08:54:00.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f8135270640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813006c440 con 0x7f813015da80 2024-04-25T08:54:00.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f8135270640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813015de60 con 0x7f81300d1cd0 2024-04-25T08:54:00.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f81300d0620 0x7f81300ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41872/0 (socket says 172.21.15.37:41872) 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812f7fe640 1 -- 172.21.15.37:0/2264380339 learned_addr learned my addr 172.21.15.37:0/2264380339 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812e7fc640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81300d1cd0 0x7f81300d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812effd640 1 --2- 172.21.15.37:0/2264380339 >> v2:172.21.15.203:3300/0 conn(0x7f813015da80 0x7f81300d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541207906 0 0) 0x7f813006be60 con 0x7f81300d0620 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8118003530 con 0x7f81300d0620 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2610780598 0 0) 0x7f8118003530 con 0x7f81300d0620 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81300d1cd0 msgr2=0x7f81300d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81300d1cd0 0x7f81300d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 >> v2:172.21.15.203:3300/0 conn(0x7f813015da80 msgr2=0x7f81300d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 --2- 172.21.15.37:0/2264380339 >> v2:172.21.15.203:3300/0 conn(0x7f813015da80 0x7f81300d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81300d2450 con 0x7f81300d0620 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f8128001680 con 0x7f81300d0620 2024-04-25T08:54:00.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 >> v1:172.21.15.37:6789/0 conn(0x7f81300d0620 legacy=0x7f81300ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81300d1cd0 0x7f8118003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8118003fb0 0x7f8118004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8118004d50 0x7f81180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813006c440 con 0x7f8118004d50 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813015de60 con 0x7f8118003fb0 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811800ba00 con 0x7f81300d1cd0 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812f7fe640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8118004d50 0x7f81180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812effd640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81300d1cd0 0x7f8118003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812e7fc640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8118003fb0 0x7f8118004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812f7fe640 1 -- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81300d1cd0 msgr2=0x7f8118003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812f7fe640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81300d1cd0 0x7f8118003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812f7fe640 1 -- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8118003fb0 msgr2=0x7f8118004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:00.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812f7fe640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8118003fb0 0x7f8118004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812f7fe640 1 -- 172.21.15.37:0/2264380339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8118003530 con 0x7f8118004d50 2024-04-25T08:54:00.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.096+0000 7f812effd640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f81300d1cd0 0x7f8118003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f812f7fe640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8118004d50 0x7f81180051d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f812800c2c0 tx=0x7f812801cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14f0a1373044b53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8128007d60 con 0x7f8118004d50 2024-04-25T08:54:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81280a9020 con 0x7f8118004d50 2024-04-25T08:54:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f812dffb640 1 -- 172.21.15.37:0/2264380339 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81280a5bb0 con 0x7f8118004d50 2024-04-25T08:54:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f8135270640 1 -- 172.21.15.37:0/2264380339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8118004d50 msgr2=0x7f81180051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f8135270640 1 --2- 172.21.15.37:0/2264380339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8118004d50 0x7f81180051d0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f812800c2c0 tx=0x7f812801cd40 comp rx=0 tx=0).stop 2024-04-25T08:54:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f812f7fe640 1 -- 172.21.15.37:0/2264380339 reap_dead start 2024-04-25T08:54:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f8135270640 1 -- 172.21.15.37:0/2264380339 shutdown_connections 2024-04-25T08:54:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f8135270640 1 -- 172.21.15.37:0/2264380339 >> 172.21.15.37:0/2264380339 conn(0x7f813008b000 msgr2=0x7f81300d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f8135270640 1 -- 172.21.15.37:0/2264380339 shutdown_connections 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f8135270640 1 -- 172.21.15.37:0/2264380339 wait complete. 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f8135270640 1 Processor -- start 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.100+0000 7f8135270640 1 -- start start 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f8135270640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8118004d50 0x7f81300c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f8135270640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81301f6490 0x7f81301f6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f8135270640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81300cb810 0x7f81300cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f8135270640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81300d26f0 con 0x7f81301f6490 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f8135270640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81300d20f0 con 0x7f81300cb810 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f8135270640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81300d2870 con 0x7f8118004d50 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81301f6490 0x7f81301f6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81301f6490 0x7f81301f6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38028/0 (socket says 172.21.15.37:38028) 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812e7fc640 1 -- 172.21.15.37:0/190184666 learned_addr learned my addr 172.21.15.37:0/190184666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812effd640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8118004d50 0x7f81300c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812f7fe640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81300cb810 0x7f81300cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812e7fc640 1 -- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8118004d50 msgr2=0x7f81300c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812e7fc640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8118004d50 0x7f81300c9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812e7fc640 1 -- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81300cb810 msgr2=0x7f81300cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812e7fc640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81300cb810 0x7f81300cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812e7fc640 1 -- 172.21.15.37:0/190184666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813006dcc0 con 0x7f81301f6490 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812effd640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8118004d50 0x7f81300c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812e7fc640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81301f6490 0x7f81301f6910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8124010860 tx=0x7f8124010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f81177fe640 1 -- 172.21.15.37:0/190184666 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400f040 con 0x7f81301f6490 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f81177fe640 1 -- 172.21.15.37:0/190184666 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8124005ba0 con 0x7f81301f6490 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f8135270640 1 -- 172.21.15.37:0/190184666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81300cc3f0 con 0x7f81301f6490 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f81177fe640 1 -- 172.21.15.37:0/190184666 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f812400e960 con 0x7f81301f6490 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f8135270640 1 -- 172.21.15.37:0/190184666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81300cc9b0 con 0x7f81301f6490 2024-04-25T08:54:00.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f8135270640 1 -- 172.21.15.37:0/190184666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813004efa0 con 0x7f81301f6490 2024-04-25T08:54:00.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f81177fe640 1 -- 172.21.15.37:0/190184666 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f812400eb00 con 0x7f81301f6490 2024-04-25T08:54:00.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f81177fe640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f811c066f60 0x7f811c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f81177fe640 1 -- 172.21.15.37:0/190184666 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8124085540 con 0x7f81301f6490 2024-04-25T08:54:00.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.104+0000 7f812effd640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f811c066f60 0x7f811c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.108+0000 7f812effd640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f811c066f60 0x7f811c069420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8120007520 tx=0x7f812000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:00.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.108+0000 7f81177fe640 1 -- 172.21.15.37:0/190184666 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8124018030 con 0x7f81301f6490 2024-04-25T08:54:00.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.260+0000 7f8135270640 1 -- 172.21.15.37:0/190184666 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f813006ad40 con 0x7f811c066f60 2024-04-25T08:54:00.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.280+0000 7f81177fe640 1 -- 172.21.15.37:0/190184666 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251962 (secure 0 0 0) 0x7f813006ad40 con 0x7f811c066f60 2024-04-25T08:54:00.275 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:00.277 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:54:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 -- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f811c066f60 msgr2=0x7f811c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f811c066f60 0x7f811c069420 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f8120007520 tx=0x7f812000b040 comp rx=0 tx=0).stop 2024-04-25T08:54:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 -- 172.21.15.37:0/190184666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81301f6490 msgr2=0x7f81301f6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81301f6490 0x7f81301f6910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8124010860 tx=0x7f8124010d20 comp rx=0 tx=0).stop 2024-04-25T08:54:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 -- 172.21.15.37:0/190184666 shutdown_connections 2024-04-25T08:54:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f811c066f60 0x7f811c069420 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81300cb810 0x7f81300cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f81301f6490 0x7f81301f6910 secure :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f8124010860 tx=0x7f8124010d20 comp rx=0 tx=0).stop 2024-04-25T08:54:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 --2- 172.21.15.37:0/190184666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8118004d50 0x7f81300c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 -- 172.21.15.37:0/190184666 >> 172.21.15.37:0/190184666 conn(0x7f813008b000 msgr2=0x7f81300b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 -- 172.21.15.37:0/190184666 shutdown_connections 2024-04-25T08:54:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.284+0000 7f8135270640 1 -- 172.21.15.37:0/190184666 wait complete. 2024-04-25T08:54:00.292 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-04-25T08:53:59.152083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229448,"kb_used_data":4912,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745272,"statfs":{"total":773094113280,"available":772859158528,"internally_reserved":0,"allocated":5029888,"data_stored":2840780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":52,"apply_latency_ms":52,"commit_latency_ns":52000000,"apply_latency_ns":52000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.334439"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872415+0000","last_change":"2024-04-25T08:53:55.877266+0000","last_active":"2024-04-25T08:53:56.872415+0000","last_peered":"2024-04-25T08:53:56.872415+0000","last_clean":"2024-04-25T08:53:56.872415+0000","last_became_active":"2024-04-25T08:53:55.876917+0000","last_became_peered":"2024-04-25T08:53:55.876917+0000","last_unstale":"2024-04-25T08:53:56.872415+0000","last_undegraded":"2024-04-25T08:53:56.872415+0000","last_fullsized":"2024-04-25T08:53:56.872415+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.932428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871794+0000","last_change":"2024-04-25T08:53:55.869176+0000","last_active":"2024-04-25T08:53:56.871794+0000","last_peered":"2024-04-25T08:53:56.871794+0000","last_clean":"2024-04-25T08:53:56.871794+0000","last_became_active":"2024-04-25T08:53:55.868932+0000","last_became_peered":"2024-04-25T08:53:55.868932+0000","last_unstale":"2024-04-25T08:53:56.871794+0000","last_undegraded":"2024-04-25T08:53:56.871794+0000","last_fullsized":"2024-04-25T08:53:56.871794+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:54:42.119355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.927587+0000","last_change":"2024-04-25T08:53:55.924062+0000","last_active":"2024-04-25T08:53:55.927587+0000","last_peered":"2024-04-25T08:53:55.927587+0000","last_clean":"2024-04-25T08:53:55.927587+0000","last_became_active":"2024-04-25T08:53:55.923643+0000","last_became_peered":"2024-04-25T08:53:55.923643+0000","last_unstale":"2024-04-25T08:53:55.927587+0000","last_undegraded":"2024-04-25T08:53:55.927587+0000","last_fullsized":"2024-04-25T08:53:55.927587+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:54.520630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405636+0000","last_change":"2024-04-25T08:53:55.868687+0000","last_active":"2024-04-25T08:53:57.405636+0000","last_peered":"2024-04-25T08:53:57.405636+0000","last_clean":"2024-04-25T08:53:57.405636+0000","last_became_active":"2024-04-25T08:53:55.867467+0000","last_became_peered":"2024-04-25T08:53:55.867467+0000","last_unstale":"2024-04-25T08:53:57.405636+0000","last_undegraded":"2024-04-25T08:53:57.405636+0000","last_fullsized":"2024-04-25T08:53:57.405636+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:19.142992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405571+0000","last_change":"2024-04-25T08:53:55.886467+0000","last_active":"2024-04-25T08:53:57.405571+0000","last_peered":"2024-04-25T08:53:57.405571+0000","last_clean":"2024-04-25T08:53:57.405571+0000","last_became_active":"2024-04-25T08:53:55.886194+0000","last_became_peered":"2024-04-25T08:53:55.886194+0000","last_unstale":"2024-04-25T08:53:57.405571+0000","last_undegraded":"2024-04-25T08:53:57.405571+0000","last_fullsized":"2024-04-25T08:53:57.405571+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:57.316851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.442922+0000","last_change":"2024-04-25T08:53:55.874974+0000","last_active":"2024-04-25T08:53:57.442922+0000","last_peered":"2024-04-25T08:53:57.442922+0000","last_clean":"2024-04-25T08:53:57.442922+0000","last_became_active":"2024-04-25T08:53:55.874628+0000","last_became_peered":"2024-04-25T08:53:55.874628+0000","last_unstale":"2024-04-25T08:53:57.442922+0000","last_undegraded":"2024-04-25T08:53:57.442922+0000","last_fullsized":"2024-04-25T08:53:57.442922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:18.983179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759521+0000","last_change":"2024-04-25T08:53:55.914301+0000","last_active":"2024-04-25T08:53:57.759521+0000","last_peered":"2024-04-25T08:53:57.759521+0000","last_clean":"2024-04-25T08:53:57.759521+0000","last_became_active":"2024-04-25T08:53:55.913295+0000","last_became_peered":"2024-04-25T08:53:55.913295+0000","last_unstale":"2024-04-25T08:53:57.759521+0000","last_undegraded":"2024-04-25T08:53:57.759521+0000","last_fullsized":"2024-04-25T08:53:57.759521+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:34.677841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871146+0000","last_change":"2024-04-25T08:53:55.923705+0000","last_active":"2024-04-25T08:53:56.871146+0000","last_peered":"2024-04-25T08:53:56.871146+0000","last_clean":"2024-04-25T08:53:56.871146+0000","last_became_active":"2024-04-25T08:53:55.923349+0000","last_became_peered":"2024-04-25T08:53:55.923349+0000","last_unstale":"2024-04-25T08:53:56.871146+0000","last_undegraded":"2024-04-25T08:53:56.871146+0000","last_fullsized":"2024-04-25T08:53:56.871146+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:48.556932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.925250+0000","last_change":"2024-04-25T08:53:55.923872+0000","last_active":"2024-04-25T08:53:55.925250+0000","last_peered":"2024-04-25T08:53:55.925250+0000","last_clean":"2024-04-25T08:53:55.925250+0000","last_became_active":"2024-04-25T08:53:55.923553+0000","last_became_peered":"2024-04-25T08:53:55.923553+0000","last_unstale":"2024-04-25T08:53:55.925250+0000","last_undegraded":"2024-04-25T08:53:55.925250+0000","last_fullsized":"2024-04-25T08:53:55.925250+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:04.958660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760114+0000","last_change":"2024-04-25T08:53:55.907329+0000","last_active":"2024-04-25T08:53:57.760114+0000","last_peered":"2024-04-25T08:53:57.760114+0000","last_clean":"2024-04-25T08:53:57.760114+0000","last_became_active":"2024-04-25T08:53:55.900764+0000","last_became_peered":"2024-04-25T08:53:55.900764+0000","last_unstale":"2024-04-25T08:53:57.760114+0000","last_undegraded":"2024-04-25T08:53:57.760114+0000","last_fullsized":"2024-04-25T08:53:57.760114+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:54.232369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760512+0000","last_change":"2024-04-25T08:53:55.913130+0000","last_active":"2024-04-25T08:53:57.760512+0000","last_peered":"2024-04-25T08:53:57.760512+0000","last_clean":"2024-04-25T08:53:57.760512+0000","last_became_active":"2024-04-25T08:53:55.908809+0000","last_became_peered":"2024-04-25T08:53:55.908809+0000","last_unstale":"2024-04-25T08:53:57.760512+0000","last_undegraded":"2024-04-25T08:53:57.760512+0000","last_fullsized":"2024-04-25T08:53:57.760512+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:48.556932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.876342+0000","last_change":"2024-04-25T08:53:55.868505+0000","last_active":"2024-04-25T08:53:55.876342+0000","last_peered":"2024-04-25T08:53:55.876342+0000","last_clean":"2024-04-25T08:53:55.876342+0000","last_became_active":"2024-04-25T08:53:55.868179+0000","last_became_peered":"2024-04-25T08:53:55.868179+0000","last_unstale":"2024-04-25T08:53:55.876342+0000","last_undegraded":"2024-04-25T08:53:55.876342+0000","last_fullsized":"2024-04-25T08:53:55.876342+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:06.183084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759675+0000","last_change":"2024-04-25T08:53:55.908755+0000","last_active":"2024-04-25T08:53:57.759675+0000","last_peered":"2024-04-25T08:53:57.759675+0000","last_clean":"2024-04-25T08:53:57.759675+0000","last_became_active":"2024-04-25T08:53:55.908077+0000","last_became_peered":"2024-04-25T08:53:55.908077+0000","last_unstale":"2024-04-25T08:53:57.759675+0000","last_undegraded":"2024-04-25T08:53:57.759675+0000","last_fullsized":"2024-04-25T08:53:57.759675+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:21.902900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.870454+0000","last_change":"2024-04-25T08:53:55.917143+0000","last_active":"2024-04-25T08:53:56.870454+0000","last_peered":"2024-04-25T08:53:56.870454+0000","last_clean":"2024-04-25T08:53:56.870454+0000","last_became_active":"2024-04-25T08:53:55.916546+0000","last_became_peered":"2024-04-25T08:53:55.916546+0000","last_unstale":"2024-04-25T08:53:56.870454+0000","last_undegraded":"2024-04-25T08:53:56.870454+0000","last_fullsized":"2024-04-25T08:53:56.870454+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:33.836787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759609+0000","last_change":"2024-04-25T08:53:55.914127+0000","last_active":"2024-04-25T08:53:57.759609+0000","last_peered":"2024-04-25T08:53:57.759609+0000","last_clean":"2024-04-25T08:53:57.759609+0000","last_became_active":"2024-04-25T08:53:55.913063+0000","last_became_peered":"2024-04-25T08:53:55.913063+0000","last_unstale":"2024-04-25T08:53:57.759609+0000","last_undegraded":"2024-04-25T08:53:57.759609+0000","last_fullsized":"2024-04-25T08:53:57.759609+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:06.399307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.338998+0000","last_change":"2024-04-25T08:53:55.920744+0000","last_active":"2024-04-25T08:53:58.338998+0000","last_peered":"2024-04-25T08:53:58.338998+0000","last_clean":"2024-04-25T08:53:58.338998+0000","last_became_active":"2024-04-25T08:53:55.920154+0000","last_became_peered":"2024-04-25T08:53:55.920154+0000","last_unstale":"2024-04-25T08:53:58.338998+0000","last_undegraded":"2024-04-25T08:53:58.338998+0000","last_fullsized":"2024-04-25T08:53:58.338998+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:54.520630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339640+0000","last_change":"2024-04-25T08:53:55.917871+0000","last_active":"2024-04-25T08:53:58.339640+0000","last_peered":"2024-04-25T08:53:58.339640+0000","last_clean":"2024-04-25T08:53:58.339640+0000","last_became_active":"2024-04-25T08:53:55.917278+0000","last_became_peered":"2024-04-25T08:53:55.917278+0000","last_unstale":"2024-04-25T08:53:58.339640+0000","last_undegraded":"2024-04-25T08:53:58.339640+0000","last_fullsized":"2024-04-25T08:53:58.339640+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:18.983179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872876+0000","last_change":"2024-04-25T08:53:55.912964+0000","last_active":"2024-04-25T08:53:56.872876+0000","last_peered":"2024-04-25T08:53:56.872876+0000","last_clean":"2024-04-25T08:53:56.872876+0000","last_became_active":"2024-04-25T08:53:55.912342+0000","last_became_peered":"2024-04-25T08:53:55.912342+0000","last_unstale":"2024-04-25T08:53:56.872876+0000","last_undegraded":"2024-04-25T08:53:56.872876+0000","last_fullsized":"2024-04-25T08:53:56.872876+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:28.625065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405642+0000","last_change":"2024-04-25T08:53:55.868880+0000","last_active":"2024-04-25T08:53:57.405642+0000","last_peered":"2024-04-25T08:53:57.405642+0000","last_clean":"2024-04-25T08:53:57.405642+0000","last_became_active":"2024-04-25T08:53:55.868633+0000","last_became_peered":"2024-04-25T08:53:55.868633+0000","last_unstale":"2024-04-25T08:53:57.405642+0000","last_undegraded":"2024-04-25T08:53:57.405642+0000","last_fullsized":"2024-04-25T08:53:57.405642+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:47.707589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.870994+0000","last_change":"2024-04-25T08:53:55.913228+0000","last_active":"2024-04-25T08:53:56.870994+0000","last_peered":"2024-04-25T08:53:56.870994+0000","last_clean":"2024-04-25T08:53:56.870994+0000","last_became_active":"2024-04-25T08:53:55.912697+0000","last_became_peered":"2024-04-25T08:53:55.912697+0000","last_unstale":"2024-04-25T08:53:56.870994+0000","last_undegraded":"2024-04-25T08:53:56.870994+0000","last_fullsized":"2024-04-25T08:53:56.870994+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:13.647839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759793+0000","last_change":"2024-04-25T08:53:55.908433+0000","last_active":"2024-04-25T08:53:57.759793+0000","last_peered":"2024-04-25T08:53:57.759793+0000","last_clean":"2024-04-25T08:53:57.759793+0000","last_became_active":"2024-04-25T08:53:55.907886+0000","last_became_peered":"2024-04-25T08:53:55.907886+0000","last_unstale":"2024-04-25T08:53:57.759793+0000","last_undegraded":"2024-04-25T08:53:57.759793+0000","last_fullsized":"2024-04-25T08:53:57.759793+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:01.341183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871440+0000","last_change":"2024-04-25T08:53:55.871600+0000","last_active":"2024-04-25T08:53:56.871440+0000","last_peered":"2024-04-25T08:53:56.871440+0000","last_clean":"2024-04-25T08:53:56.871440+0000","last_became_active":"2024-04-25T08:53:55.871060+0000","last_became_peered":"2024-04-25T08:53:55.871060+0000","last_unstale":"2024-04-25T08:53:56.871440+0000","last_undegraded":"2024-04-25T08:53:56.871440+0000","last_fullsized":"2024-04-25T08:53:56.871440+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.163826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339160+0000","last_change":"2024-04-25T08:53:55.886801+0000","last_active":"2024-04-25T08:53:58.339160+0000","last_peered":"2024-04-25T08:53:58.339160+0000","last_clean":"2024-04-25T08:53:58.339160+0000","last_became_active":"2024-04-25T08:53:55.885970+0000","last_became_peered":"2024-04-25T08:53:55.885970+0000","last_unstale":"2024-04-25T08:53:58.339160+0000","last_undegraded":"2024-04-25T08:53:58.339160+0000","last_fullsized":"2024-04-25T08:53:58.339160+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:51:54.524193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405170+0000","last_change":"2024-04-25T08:53:55.892289+0000","last_active":"2024-04-25T08:53:57.405170+0000","last_peered":"2024-04-25T08:53:57.405170+0000","last_clean":"2024-04-25T08:53:57.405170+0000","last_became_active":"2024-04-25T08:53:55.891999+0000","last_became_peered":"2024-04-25T08:53:55.891999+0000","last_unstale":"2024-04-25T08:53:57.405170+0000","last_undegraded":"2024-04-25T08:53:57.405170+0000","last_fullsized":"2024-04-25T08:53:57.405170+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:21.479704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.873337+0000","last_change":"2024-04-25T08:53:55.897159+0000","last_active":"2024-04-25T08:53:56.873337+0000","last_peered":"2024-04-25T08:53:56.873337+0000","last_clean":"2024-04-25T08:53:56.873337+0000","last_became_active":"2024-04-25T08:53:55.896451+0000","last_became_peered":"2024-04-25T08:53:55.896451+0000","last_unstale":"2024-04-25T08:53:56.873337+0000","last_undegraded":"2024-04-25T08:53:56.873337+0000","last_fullsized":"2024-04-25T08:53:56.873337+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:23.170169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.443599+0000","last_change":"2024-04-25T08:53:55.902092+0000","last_active":"2024-04-25T08:53:57.443599+0000","last_peered":"2024-04-25T08:53:57.443599+0000","last_clean":"2024-04-25T08:53:57.443599+0000","last_became_active":"2024-04-25T08:53:55.901811+0000","last_became_peered":"2024-04-25T08:53:55.901811+0000","last_unstale":"2024-04-25T08:53:57.443599+0000","last_undegraded":"2024-04-25T08:53:57.443599+0000","last_fullsized":"2024-04-25T08:53:57.443599+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:42.981286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872430+0000","last_change":"2024-04-25T08:53:55.902629+0000","last_active":"2024-04-25T08:53:56.872430+0000","last_peered":"2024-04-25T08:53:56.872430+0000","last_clean":"2024-04-25T08:53:56.872430+0000","last_became_active":"2024-04-25T08:53:55.881384+0000","last_became_peered":"2024-04-25T08:53:55.881384+0000","last_unstale":"2024-04-25T08:53:56.872430+0000","last_undegraded":"2024-04-25T08:53:56.872430+0000","last_fullsized":"2024-04-25T08:53:56.872430+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:51.758262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760358+0000","last_change":"2024-04-25T08:53:55.908769+0000","last_active":"2024-04-25T08:53:57.760358+0000","last_peered":"2024-04-25T08:53:57.760358+0000","last_clean":"2024-04-25T08:53:57.760358+0000","last_became_active":"2024-04-25T08:53:55.907335+0000","last_became_peered":"2024-04-25T08:53:55.907335+0000","last_unstale":"2024-04-25T08:53:57.760358+0000","last_undegraded":"2024-04-25T08:53:57.760358+0000","last_fullsized":"2024-04-25T08:53:57.760358+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:09.318991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872227+0000","last_change":"2024-04-25T08:53:55.911461+0000","last_active":"2024-04-25T08:53:56.872227+0000","last_peered":"2024-04-25T08:53:56.872227+0000","last_clean":"2024-04-25T08:53:56.872227+0000","last_became_active":"2024-04-25T08:53:55.903300+0000","last_became_peered":"2024-04-25T08:53:55.903300+0000","last_unstale":"2024-04-25T08:53:56.872227+0000","last_undegraded":"2024-04-25T08:53:56.872227+0000","last_fullsized":"2024-04-25T08:53:56.872227+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:06.183084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871184+0000","last_change":"2024-04-25T08:53:55.903796+0000","last_active":"2024-04-25T08:53:56.871184+0000","last_peered":"2024-04-25T08:53:56.871184+0000","last_clean":"2024-04-25T08:53:56.871184+0000","last_became_active":"2024-04-25T08:53:55.903049+0000","last_became_peered":"2024-04-25T08:53:55.903049+0000","last_unstale":"2024-04-25T08:53:56.871184+0000","last_undegraded":"2024-04-25T08:53:56.871184+0000","last_fullsized":"2024-04-25T08:53:56.871184+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:12.913824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339332+0000","last_change":"2024-04-25T08:53:55.886832+0000","last_active":"2024-04-25T08:53:58.339332+0000","last_peered":"2024-04-25T08:53:58.339332+0000","last_clean":"2024-04-25T08:53:58.339332+0000","last_became_active":"2024-04-25T08:53:55.886068+0000","last_became_peered":"2024-04-25T08:53:55.886068+0000","last_unstale":"2024-04-25T08:53:58.339332+0000","last_undegraded":"2024-04-25T08:53:58.339332+0000","last_fullsized":"2024-04-25T08:53:58.339332+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:33.836787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.442739+0000","last_change":"2024-04-25T08:53:55.869089+0000","last_active":"2024-04-25T08:53:57.442739+0000","last_peered":"2024-04-25T08:53:57.442739+0000","last_clean":"2024-04-25T08:53:57.442739+0000","last_became_active":"2024-04-25T08:53:55.868734+0000","last_became_peered":"2024-04-25T08:53:55.868734+0000","last_unstale":"2024-04-25T08:53:57.442739+0000","last_undegraded":"2024-04-25T08:53:57.442739+0000","last_fullsized":"2024-04-25T08:53:57.442739+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:33.836787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339756+0000","last_change":"2024-04-25T08:53:55.910330+0000","last_active":"2024-04-25T08:53:58.339756+0000","last_peered":"2024-04-25T08:53:58.339756+0000","last_clean":"2024-04-25T08:53:58.339756+0000","last_became_active":"2024-04-25T08:53:55.909744+0000","last_became_peered":"2024-04-25T08:53:55.909744+0000","last_unstale":"2024-04-25T08:53:58.339756+0000","last_undegraded":"2024-04-25T08:53:58.339756+0000","last_fullsized":"2024-04-25T08:53:58.339756+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:28.381438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"40'4","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.873418+0000","last_change":"2024-04-25T08:53:54.851744+0000","last_active":"2024-04-25T08:53:56.873418+0000","last_peered":"2024-04-25T08:53:56.873418+0000","last_clean":"2024-04-25T08:53:56.873418+0000","last_became_active":"2024-04-25T08:53:54.851056+0000","last_became_peered":"2024-04-25T08:53:54.851056+0000","last_unstale":"2024-04-25T08:53:56.873418+0000","last_undegraded":"2024-04-25T08:53:56.873418+0000","last_fullsized":"2024-04-25T08:53:56.873418+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:46.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.443498+0000","last_change":"2024-04-25T08:53:55.924008+0000","last_active":"2024-04-25T08:53:57.443498+0000","last_peered":"2024-04-25T08:53:57.443498+0000","last_clean":"2024-04-25T08:53:57.443498+0000","last_became_active":"2024-04-25T08:53:55.923214+0000","last_became_peered":"2024-04-25T08:53:55.923214+0000","last_unstale":"2024-04-25T08:53:57.443498+0000","last_undegraded":"2024-04-25T08:53:57.443498+0000","last_fullsized":"2024-04-25T08:53:57.443498+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.163826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"40'1","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.887626+0000","last_change":"2024-04-25T08:53:54.862894+0000","last_active":"2024-04-25T08:53:55.887626+0000","last_peered":"2024-04-25T08:53:55.887626+0000","last_clean":"2024-04-25T08:53:55.887626+0000","last_became_active":"2024-04-25T08:53:54.862250+0000","last_became_peered":"2024-04-25T08:53:54.862250+0000","last_unstale":"2024-04-25T08:53:55.887626+0000","last_undegraded":"2024-04-25T08:53:55.887626+0000","last_fullsized":"2024-04-25T08:53:55.887626+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.214318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871380+0000","last_change":"2024-04-25T08:53:55.912519+0000","last_active":"2024-04-25T08:53:56.871380+0000","last_peered":"2024-04-25T08:53:56.871380+0000","last_clean":"2024-04-25T08:53:56.871380+0000","last_became_active":"2024-04-25T08:53:55.911965+0000","last_became_peered":"2024-04-25T08:53:55.911965+0000","last_unstale":"2024-04-25T08:53:56.871380+0000","last_undegraded":"2024-04-25T08:53:56.871380+0000","last_fullsized":"2024-04-25T08:53:56.871380+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:28.381438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.443756+0000","last_change":"2024-04-25T08:53:54.858661+0000","last_active":"2024-04-25T08:53:57.443756+0000","last_peered":"2024-04-25T08:53:57.443756+0000","last_clean":"2024-04-25T08:53:57.443756+0000","last_became_active":"2024-04-25T08:53:54.858376+0000","last_became_peered":"2024-04-25T08:53:54.858376+0000","last_unstale":"2024-04-25T08:53:57.443756+0000","last_undegraded":"2024-04-25T08:53:57.443756+0000","last_fullsized":"2024-04-25T08:53:57.443756+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:05.174377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872319+0000","last_change":"2024-04-25T08:53:55.913009+0000","last_active":"2024-04-25T08:53:56.872319+0000","last_peered":"2024-04-25T08:53:56.872319+0000","last_clean":"2024-04-25T08:53:56.872319+0000","last_became_active":"2024-04-25T08:53:55.912537+0000","last_became_peered":"2024-04-25T08:53:55.912537+0000","last_unstale":"2024-04-25T08:53:56.872319+0000","last_undegraded":"2024-04-25T08:53:56.872319+0000","last_fullsized":"2024-04-25T08:53:56.872319+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:53.486036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.873170+0000","last_change":"2024-04-25T08:53:54.847858+0000","last_active":"2024-04-25T08:53:56.873170+0000","last_peered":"2024-04-25T08:53:56.873170+0000","last_clean":"2024-04-25T08:53:56.873170+0000","last_became_active":"2024-04-25T08:53:54.847100+0000","last_became_peered":"2024-04-25T08:53:54.847100+0000","last_unstale":"2024-04-25T08:53:56.873170+0000","last_undegraded":"2024-04-25T08:53:56.873170+0000","last_fullsized":"2024-04-25T08:53:56.873170+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.214318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.876419+0000","last_change":"2024-04-25T08:53:55.865056+0000","last_active":"2024-04-25T08:53:55.876419+0000","last_peered":"2024-04-25T08:53:55.876419+0000","last_clean":"2024-04-25T08:53:55.876419+0000","last_became_active":"2024-04-25T08:53:55.864765+0000","last_became_peered":"2024-04-25T08:53:55.864765+0000","last_unstale":"2024-04-25T08:53:55.876419+0000","last_undegraded":"2024-04-25T08:53:55.876419+0000","last_fullsized":"2024-04-25T08:53:55.876419+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:12.913824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760437+0000","last_change":"2024-04-25T08:53:54.862961+0000","last_active":"2024-04-25T08:53:57.760437+0000","last_peered":"2024-04-25T08:53:57.760437+0000","last_clean":"2024-04-25T08:53:57.760437+0000","last_became_active":"2024-04-25T08:53:54.862258+0000","last_became_peered":"2024-04-25T08:53:54.862258+0000","last_unstale":"2024-04-25T08:53:57.760437+0000","last_undegraded":"2024-04-25T08:53:57.760437+0000","last_fullsized":"2024-04-25T08:53:57.760437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:05.174377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759939+0000","last_change":"2024-04-25T08:53:55.915599+0000","last_active":"2024-04-25T08:53:57.759939+0000","last_peered":"2024-04-25T08:53:57.759939+0000","last_clean":"2024-04-25T08:53:57.759939+0000","last_became_active":"2024-04-25T08:53:55.913728+0000","last_became_peered":"2024-04-25T08:53:55.913728+0000","last_unstale":"2024-04-25T08:53:57.759939+0000","last_undegraded":"2024-04-25T08:53:57.759939+0000","last_fullsized":"2024-04-25T08:53:57.759939+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:26.688821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339470+0000","last_change":"2024-04-25T08:53:54.867415+0000","last_active":"2024-04-25T08:53:58.339470+0000","last_peered":"2024-04-25T08:53:58.339470+0000","last_clean":"2024-04-25T08:53:58.339470+0000","last_became_active":"2024-04-25T08:53:54.864476+0000","last_became_peered":"2024-04-25T08:53:54.864476+0000","last_unstale":"2024-04-25T08:53:58.339470+0000","last_undegraded":"2024-04-25T08:53:58.339470+0000","last_fullsized":"2024-04-25T08:53:58.339470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:18.134284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871971+0000","last_change":"2024-04-25T08:53:55.902059+0000","last_active":"2024-04-25T08:53:56.871971+0000","last_peered":"2024-04-25T08:53:56.871971+0000","last_clean":"2024-04-25T08:53:56.871971+0000","last_became_active":"2024-04-25T08:53:55.901602+0000","last_became_peered":"2024-04-25T08:53:55.901602+0000","last_unstale":"2024-04-25T08:53:56.871971+0000","last_undegraded":"2024-04-25T08:53:56.871971+0000","last_fullsized":"2024-04-25T08:53:56.871971+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:42.981286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.443269+0000","last_change":"2024-04-25T08:53:54.873269+0000","last_active":"2024-04-25T08:53:57.443269+0000","last_peered":"2024-04-25T08:53:57.443269+0000","last_clean":"2024-04-25T08:53:57.443269+0000","last_became_active":"2024-04-25T08:53:54.873022+0000","last_became_peered":"2024-04-25T08:53:54.873022+0000","last_unstale":"2024-04-25T08:53:57.443269+0000","last_undegraded":"2024-04-25T08:53:57.443269+0000","last_fullsized":"2024-04-25T08:53:57.443269+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.214318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871535+0000","last_change":"2024-04-25T08:53:55.866850+0000","last_active":"2024-04-25T08:53:56.871535+0000","last_peered":"2024-04-25T08:53:56.871535+0000","last_clean":"2024-04-25T08:53:56.871535+0000","last_became_active":"2024-04-25T08:53:55.866568+0000","last_became_peered":"2024-04-25T08:53:55.866568+0000","last_unstale":"2024-04-25T08:53:56.871535+0000","last_undegraded":"2024-04-25T08:53:56.871535+0000","last_fullsized":"2024-04-25T08:53:56.871535+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:32.159132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.870948+0000","last_change":"2024-04-25T08:53:54.846517+0000","last_active":"2024-04-25T08:53:56.870948+0000","last_peered":"2024-04-25T08:53:56.870948+0000","last_clean":"2024-04-25T08:53:56.870948+0000","last_became_active":"2024-04-25T08:53:54.842958+0000","last_became_peered":"2024-04-25T08:53:54.842958+0000","last_unstale":"2024-04-25T08:53:56.870948+0000","last_undegraded":"2024-04-25T08:53:56.870948+0000","last_fullsized":"2024-04-25T08:53:56.870948+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:11.905116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405595+0000","last_change":"2024-04-25T08:53:55.886972+0000","last_active":"2024-04-25T08:53:57.405595+0000","last_peered":"2024-04-25T08:53:57.405595+0000","last_clean":"2024-04-25T08:53:57.405595+0000","last_became_active":"2024-04-25T08:53:55.886216+0000","last_became_peered":"2024-04-25T08:53:55.886216+0000","last_unstale":"2024-04-25T08:53:57.405595+0000","last_undegraded":"2024-04-25T08:53:57.405595+0000","last_fullsized":"2024-04-25T08:53:57.405595+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:59.602303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872584+0000","last_change":"2024-04-25T08:53:54.842673+0000","last_active":"2024-04-25T08:53:56.872584+0000","last_peered":"2024-04-25T08:53:56.872584+0000","last_clean":"2024-04-25T08:53:56.872584+0000","last_became_active":"2024-04-25T08:53:54.842401+0000","last_became_peered":"2024-04-25T08:53:54.842401+0000","last_unstale":"2024-04-25T08:53:56.872584+0000","last_undegraded":"2024-04-25T08:53:56.872584+0000","last_fullsized":"2024-04-25T08:53:56.872584+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:29.155118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339828+0000","last_change":"2024-04-25T08:53:55.913304+0000","last_active":"2024-04-25T08:53:58.339828+0000","last_peered":"2024-04-25T08:53:58.339828+0000","last_clean":"2024-04-25T08:53:58.339828+0000","last_became_active":"2024-04-25T08:53:55.912555+0000","last_became_peered":"2024-04-25T08:53:55.912555+0000","last_unstale":"2024-04-25T08:53:58.339828+0000","last_undegraded":"2024-04-25T08:53:58.339828+0000","last_fullsized":"2024-04-25T08:53:58.339828+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:42.981286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760483+0000","last_change":"2024-04-25T08:53:54.871214+0000","last_active":"2024-04-25T08:53:57.760483+0000","last_peered":"2024-04-25T08:53:57.760483+0000","last_clean":"2024-04-25T08:53:57.760483+0000","last_became_active":"2024-04-25T08:53:54.864020+0000","last_became_peered":"2024-04-25T08:53:54.864020+0000","last_unstale":"2024-04-25T08:53:57.760483+0000","last_undegraded":"2024-04-25T08:53:57.760483+0000","last_fullsized":"2024-04-25T08:53:57.760483+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:12.639132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.892513+0000","last_change":"2024-04-25T08:53:55.888098+0000","last_active":"2024-04-25T08:53:55.892513+0000","last_peered":"2024-04-25T08:53:55.892513+0000","last_clean":"2024-04-25T08:53:55.892513+0000","last_became_active":"2024-04-25T08:53:55.887802+0000","last_became_peered":"2024-04-25T08:53:55.887802+0000","last_unstale":"2024-04-25T08:53:55.892513+0000","last_undegraded":"2024-04-25T08:53:55.892513+0000","last_fullsized":"2024-04-25T08:53:55.892513+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:28.625065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"40'3","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760666+0000","last_change":"2024-04-25T08:53:54.873385+0000","last_active":"2024-04-25T08:53:57.760666+0000","last_peered":"2024-04-25T08:53:57.760666+0000","last_clean":"2024-04-25T08:53:57.760666+0000","last_became_active":"2024-04-25T08:53:54.872382+0000","last_became_peered":"2024-04-25T08:53:54.872382+0000","last_unstale":"2024-04-25T08:53:57.760666+0000","last_undegraded":"2024-04-25T08:53:57.760666+0000","last_fullsized":"2024-04-25T08:53:57.760666+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:27.616358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.870922+0000","last_change":"2024-04-25T08:53:55.873437+0000","last_active":"2024-04-25T08:53:56.870922+0000","last_peered":"2024-04-25T08:53:56.870922+0000","last_clean":"2024-04-25T08:53:56.870922+0000","last_became_active":"2024-04-25T08:53:55.872710+0000","last_became_peered":"2024-04-25T08:53:55.872710+0000","last_unstale":"2024-04-25T08:53:56.870922+0000","last_undegraded":"2024-04-25T08:53:56.870922+0000","last_fullsized":"2024-04-25T08:53:56.870922+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:18.983179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"40'6","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872503+0000","last_change":"2024-04-25T08:53:54.865425+0000","last_active":"2024-04-25T08:53:56.872503+0000","last_peered":"2024-04-25T08:53:56.872503+0000","last_clean":"2024-04-25T08:53:56.872503+0000","last_became_active":"2024-04-25T08:53:54.865084+0000","last_became_peered":"2024-04-25T08:53:54.865084+0000","last_unstale":"2024-04-25T08:53:56.872503+0000","last_undegraded":"2024-04-25T08:53:56.872503+0000","last_fullsized":"2024-04-25T08:53:56.872503+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:17.974471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.927613+0000","last_change":"2024-04-25T08:53:55.924625+0000","last_active":"2024-04-25T08:53:55.927613+0000","last_peered":"2024-04-25T08:53:55.927613+0000","last_clean":"2024-04-25T08:53:55.927613+0000","last_became_active":"2024-04-25T08:53:55.923408+0000","last_became_peered":"2024-04-25T08:53:55.923408+0000","last_unstale":"2024-04-25T08:53:55.927613+0000","last_undegraded":"2024-04-25T08:53:55.927613+0000","last_fullsized":"2024-04-25T08:53:55.927613+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:43.538676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872440+0000","last_change":"2024-04-25T08:53:54.841968+0000","last_active":"2024-04-25T08:53:56.872440+0000","last_peered":"2024-04-25T08:53:56.872440+0000","last_clean":"2024-04-25T08:53:56.872440+0000","last_became_active":"2024-04-25T08:53:54.841614+0000","last_became_peered":"2024-04-25T08:53:54.841614+0000","last_unstale":"2024-04-25T08:53:56.872440+0000","last_undegraded":"2024-04-25T08:53:56.872440+0000","last_fullsized":"2024-04-25T08:53:56.872440+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:41.972578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871305+0000","last_change":"2024-04-25T08:53:55.863560+0000","last_active":"2024-04-25T08:53:56.871305+0000","last_peered":"2024-04-25T08:53:56.871305+0000","last_clean":"2024-04-25T08:53:56.871305+0000","last_became_active":"2024-04-25T08:53:55.863214+0000","last_became_peered":"2024-04-25T08:53:55.863214+0000","last_unstale":"2024-04-25T08:53:56.871305+0000","last_undegraded":"2024-04-25T08:53:56.871305+0000","last_fullsized":"2024-04-25T08:53:56.871305+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:42.712711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.443216+0000","last_change":"2024-04-25T08:53:54.875234+0000","last_active":"2024-04-25T08:53:57.443216+0000","last_peered":"2024-04-25T08:53:57.443216+0000","last_clean":"2024-04-25T08:53:57.443216+0000","last_became_active":"2024-04-25T08:53:54.875031+0000","last_became_peered":"2024-04-25T08:53:54.875031+0000","last_unstale":"2024-04-25T08:53:57.443216+0000","last_undegraded":"2024-04-25T08:53:57.443216+0000","last_fullsized":"2024-04-25T08:53:57.443216+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:12.639132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339119+0000","last_change":"2024-04-25T08:53:55.886593+0000","last_active":"2024-04-25T08:53:58.339119+0000","last_peered":"2024-04-25T08:53:58.339119+0000","last_clean":"2024-04-25T08:53:58.339119+0000","last_became_active":"2024-04-25T08:53:55.885815+0000","last_became_peered":"2024-04-25T08:53:55.885815+0000","last_unstale":"2024-04-25T08:53:58.339119+0000","last_undegraded":"2024-04-25T08:53:58.339119+0000","last_fullsized":"2024-04-25T08:53:58.339119+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:43.538676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872623+0000","last_change":"2024-04-25T08:53:54.849856+0000","last_active":"2024-04-25T08:53:56.872623+0000","last_peered":"2024-04-25T08:53:56.872623+0000","last_clean":"2024-04-25T08:53:56.872623+0000","last_became_active":"2024-04-25T08:53:54.849102+0000","last_became_peered":"2024-04-25T08:53:54.849102+0000","last_unstale":"2024-04-25T08:53:56.872623+0000","last_undegraded":"2024-04-25T08:53:56.872623+0000","last_fullsized":"2024-04-25T08:53:56.872623+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:56.308143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871668+0000","last_change":"2024-04-25T08:53:55.864268+0000","last_active":"2024-04-25T08:53:56.871668+0000","last_peered":"2024-04-25T08:53:56.871668+0000","last_clean":"2024-04-25T08:53:56.871668+0000","last_became_active":"2024-04-25T08:53:55.864013+0000","last_became_peered":"2024-04-25T08:53:55.864013+0000","last_unstale":"2024-04-25T08:53:56.871668+0000","last_undegraded":"2024-04-25T08:53:56.871668+0000","last_fullsized":"2024-04-25T08:53:56.871668+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:07.784180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"40'8","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339064+0000","last_change":"2024-04-25T08:53:54.864793+0000","last_active":"2024-04-25T08:53:58.339064+0000","last_peered":"2024-04-25T08:53:58.339064+0000","last_clean":"2024-04-25T08:53:58.339064+0000","last_became_active":"2024-04-25T08:53:54.864140+0000","last_became_peered":"2024-04-25T08:53:54.864140+0000","last_unstale":"2024-04-25T08:53:58.339064+0000","last_undegraded":"2024-04-25T08:53:58.339064+0000","last_fullsized":"2024-04-25T08:53:58.339064+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:48.132257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872744+0000","last_change":"2024-04-25T08:53:55.913076+0000","last_active":"2024-04-25T08:53:56.872744+0000","last_peered":"2024-04-25T08:53:56.872744+0000","last_clean":"2024-04-25T08:53:56.872744+0000","last_became_active":"2024-04-25T08:53:55.912794+0000","last_became_peered":"2024-04-25T08:53:55.912794+0000","last_unstale":"2024-04-25T08:53:56.872744+0000","last_undegraded":"2024-04-25T08:53:56.872744+0000","last_fullsized":"2024-04-25T08:53:56.872744+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:59.690305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760509+0000","last_change":"2024-04-25T08:53:54.861175+0000","last_active":"2024-04-25T08:53:57.760509+0000","last_peered":"2024-04-25T08:53:57.760509+0000","last_clean":"2024-04-25T08:53:57.760509+0000","last_became_active":"2024-04-25T08:53:54.860530+0000","last_became_peered":"2024-04-25T08:53:54.860530+0000","last_unstale":"2024-04-25T08:53:57.760509+0000","last_undegraded":"2024-04-25T08:53:57.760509+0000","last_fullsized":"2024-04-25T08:53:57.760509+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:11.905116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760575+0000","last_change":"2024-04-25T08:53:55.919306+0000","last_active":"2024-04-25T08:53:57.760575+0000","last_peered":"2024-04-25T08:53:57.760575+0000","last_clean":"2024-04-25T08:53:57.760575+0000","last_became_active":"2024-04-25T08:53:55.908770+0000","last_became_peered":"2024-04-25T08:53:55.908770+0000","last_unstale":"2024-04-25T08:53:57.760575+0000","last_undegraded":"2024-04-25T08:53:57.760575+0000","last_fullsized":"2024-04-25T08:53:57.760575+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:42.712711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871653+0000","last_change":"2024-04-25T08:53:54.867127+0000","last_active":"2024-04-25T08:53:56.871653+0000","last_peered":"2024-04-25T08:53:56.871653+0000","last_clean":"2024-04-25T08:53:56.871653+0000","last_became_active":"2024-04-25T08:53:54.866337+0000","last_became_peered":"2024-04-25T08:53:54.866337+0000","last_unstale":"2024-04-25T08:53:56.871653+0000","last_undegraded":"2024-04-25T08:53:56.871653+0000","last_fullsized":"2024-04-25T08:53:56.871653+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:05.174377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872275+0000","last_change":"2024-04-25T08:53:55.874774+0000","last_active":"2024-04-25T08:53:56.872275+0000","last_peered":"2024-04-25T08:53:56.872275+0000","last_clean":"2024-04-25T08:53:56.872275+0000","last_became_active":"2024-04-25T08:53:55.872554+0000","last_became_peered":"2024-04-25T08:53:55.872554+0000","last_unstale":"2024-04-25T08:53:56.872275+0000","last_undegraded":"2024-04-25T08:53:56.872275+0000","last_fullsized":"2024-04-25T08:53:56.872275+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:47.567171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405568+0000","last_change":"2024-04-25T08:53:54.862001+0000","last_active":"2024-04-25T08:53:57.405568+0000","last_peered":"2024-04-25T08:53:57.405568+0000","last_clean":"2024-04-25T08:53:57.405568+0000","last_became_active":"2024-04-25T08:53:54.861772+0000","last_became_peered":"2024-04-25T08:53:54.861772+0000","last_unstale":"2024-04-25T08:53:57.405568+0000","last_undegraded":"2024-04-25T08:53:57.405568+0000","last_fullsized":"2024-04-25T08:53:57.405568+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:33.123852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405957+0000","last_change":"2024-04-25T08:53:55.874265+0000","last_active":"2024-04-25T08:53:57.405957+0000","last_peered":"2024-04-25T08:53:57.405957+0000","last_clean":"2024-04-25T08:53:57.405957+0000","last_became_active":"2024-04-25T08:53:55.873901+0000","last_became_peered":"2024-04-25T08:53:55.873901+0000","last_unstale":"2024-04-25T08:53:57.405957+0000","last_undegraded":"2024-04-25T08:53:57.405957+0000","last_fullsized":"2024-04-25T08:53:57.405957+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:37.446552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.442928+0000","last_change":"2024-04-25T08:53:54.855073+0000","last_active":"2024-04-25T08:53:57.442928+0000","last_peered":"2024-04-25T08:53:57.442928+0000","last_clean":"2024-04-25T08:53:57.442928+0000","last_became_active":"2024-04-25T08:53:54.854784+0000","last_became_peered":"2024-04-25T08:53:54.854784+0000","last_unstale":"2024-04-25T08:53:57.442928+0000","last_undegraded":"2024-04-25T08:53:57.442928+0000","last_fullsized":"2024-04-25T08:53:57.442928+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:11.905116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339445+0000","last_change":"2024-04-25T08:53:55.892170+0000","last_active":"2024-04-25T08:53:58.339445+0000","last_peered":"2024-04-25T08:53:58.339445+0000","last_clean":"2024-04-25T08:53:58.339445+0000","last_became_active":"2024-04-25T08:53:55.891404+0000","last_became_peered":"2024-04-25T08:53:55.891404+0000","last_unstale":"2024-04-25T08:53:58.339445+0000","last_undegraded":"2024-04-25T08:53:58.339445+0000","last_fullsized":"2024-04-25T08:53:58.339445+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:58.292874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405185+0000","last_change":"2024-04-25T08:53:54.846561+0000","last_active":"2024-04-25T08:53:57.405185+0000","last_peered":"2024-04-25T08:53:57.405185+0000","last_clean":"2024-04-25T08:53:57.405185+0000","last_became_active":"2024-04-25T08:53:54.845972+0000","last_became_peered":"2024-04-25T08:53:54.845972+0000","last_unstale":"2024-04-25T08:53:57.405185+0000","last_undegraded":"2024-04-25T08:53:57.405185+0000","last_fullsized":"2024-04-25T08:53:57.405185+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:08.561873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.892547+0000","last_change":"2024-04-25T08:53:55.888416+0000","last_active":"2024-04-25T08:53:55.892547+0000","last_peered":"2024-04-25T08:53:55.892547+0000","last_clean":"2024-04-25T08:53:55.892547+0000","last_became_active":"2024-04-25T08:53:55.888086+0000","last_became_peered":"2024-04-25T08:53:55.888086+0000","last_unstale":"2024-04-25T08:53:55.892547+0000","last_undegraded":"2024-04-25T08:53:55.892547+0000","last_fullsized":"2024-04-25T08:53:55.892547+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:23.170169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405193+0000","last_change":"2024-04-25T08:53:54.859141+0000","last_active":"2024-04-25T08:53:57.405193+0000","last_peered":"2024-04-25T08:53:57.405193+0000","last_clean":"2024-04-25T08:53:57.405193+0000","last_became_active":"2024-04-25T08:53:54.858807+0000","last_became_peered":"2024-04-25T08:53:54.858807+0000","last_unstale":"2024-04-25T08:53:57.405193+0000","last_undegraded":"2024-04-25T08:53:57.405193+0000","last_fullsized":"2024-04-25T08:53:57.405193+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:54:43.373208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.759811+0000","last_change":"2024-04-25T08:53:55.920669+0000","last_active":"2024-04-25T08:53:57.759811+0000","last_peered":"2024-04-25T08:53:57.759811+0000","last_clean":"2024-04-25T08:53:57.759811+0000","last_became_active":"2024-04-25T08:53:55.919969+0000","last_became_peered":"2024-04-25T08:53:55.919969+0000","last_unstale":"2024-04-25T08:53:57.759811+0000","last_undegraded":"2024-04-25T08:53:57.759811+0000","last_fullsized":"2024-04-25T08:53:57.759811+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:08.899755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339834+0000","last_change":"2024-04-25T08:53:54.869769+0000","last_active":"2024-04-25T08:53:58.339834+0000","last_peered":"2024-04-25T08:53:58.339834+0000","last_clean":"2024-04-25T08:53:58.339834+0000","last_became_active":"2024-04-25T08:53:54.868883+0000","last_became_peered":"2024-04-25T08:53:54.868883+0000","last_unstale":"2024-04-25T08:53:58.339834+0000","last_undegraded":"2024-04-25T08:53:58.339834+0000","last_fullsized":"2024-04-25T08:53:58.339834+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:38:36.437845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871870+0000","last_change":"2024-04-25T08:53:55.896505+0000","last_active":"2024-04-25T08:53:56.871870+0000","last_peered":"2024-04-25T08:53:56.871870+0000","last_clean":"2024-04-25T08:53:56.871870+0000","last_became_active":"2024-04-25T08:53:55.896187+0000","last_became_peered":"2024-04-25T08:53:55.896187+0000","last_unstale":"2024-04-25T08:53:56.871870+0000","last_undegraded":"2024-04-25T08:53:56.871870+0000","last_fullsized":"2024-04-25T08:53:56.871870+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:50.990569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405060+0000","last_change":"2024-04-25T08:53:54.847178+0000","last_active":"2024-04-25T08:53:57.405060+0000","last_peered":"2024-04-25T08:53:57.405060+0000","last_clean":"2024-04-25T08:53:57.405060+0000","last_became_active":"2024-04-25T08:53:54.846201+0000","last_became_peered":"2024-04-25T08:53:54.846201+0000","last_unstale":"2024-04-25T08:53:57.405060+0000","last_undegraded":"2024-04-25T08:53:57.405060+0000","last_fullsized":"2024-04-25T08:53:57.405060+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:14.176202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.442967+0000","last_change":"2024-04-25T08:53:55.909242+0000","last_active":"2024-04-25T08:53:57.442967+0000","last_peered":"2024-04-25T08:53:57.442967+0000","last_clean":"2024-04-25T08:53:57.442967+0000","last_became_active":"2024-04-25T08:53:55.909006+0000","last_became_peered":"2024-04-25T08:53:55.909006+0000","last_unstale":"2024-04-25T08:53:57.442967+0000","last_undegraded":"2024-04-25T08:53:57.442967+0000","last_fullsized":"2024-04-25T08:53:57.442967+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:28.381438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"40'2","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760765+0000","last_change":"2024-04-25T08:53:54.873765+0000","last_active":"2024-04-25T08:53:57.760765+0000","last_peered":"2024-04-25T08:53:57.760765+0000","last_clean":"2024-04-25T08:53:57.760765+0000","last_became_active":"2024-04-25T08:53:54.872783+0000","last_became_peered":"2024-04-25T08:53:54.872783+0000","last_unstale":"2024-04-25T08:53:57.760765+0000","last_undegraded":"2024-04-25T08:53:57.760765+0000","last_fullsized":"2024-04-25T08:53:57.760765+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:50.749554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339780+0000","last_change":"2024-04-25T08:53:55.904712+0000","last_active":"2024-04-25T08:53:58.339780+0000","last_peered":"2024-04-25T08:53:58.339780+0000","last_clean":"2024-04-25T08:53:58.339780+0000","last_became_active":"2024-04-25T08:53:55.904084+0000","last_became_peered":"2024-04-25T08:53:55.904084+0000","last_unstale":"2024-04-25T08:53:58.339780+0000","last_undegraded":"2024-04-25T08:53:58.339780+0000","last_fullsized":"2024-04-25T08:53:58.339780+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:51:30.163826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"40'1","reported_seq":15,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.405823+0000","last_change":"2024-04-25T08:53:54.866358+0000","last_active":"2024-04-25T08:53:57.405823+0000","last_peered":"2024-04-25T08:53:57.405823+0000","last_clean":"2024-04-25T08:53:57.405823+0000","last_became_active":"2024-04-25T08:53:54.866094+0000","last_became_peered":"2024-04-25T08:53:54.866094+0000","last_unstale":"2024-04-25T08:53:57.405823+0000","last_undegraded":"2024-04-25T08:53:57.405823+0000","last_fullsized":"2024-04-25T08:53:57.405823+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:18.266660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872250+0000","last_change":"2024-04-25T08:53:55.896939+0000","last_active":"2024-04-25T08:53:56.872250+0000","last_peered":"2024-04-25T08:53:56.872250+0000","last_clean":"2024-04-25T08:53:56.872250+0000","last_became_active":"2024-04-25T08:53:55.896625+0000","last_became_peered":"2024-04-25T08:53:55.896625+0000","last_unstale":"2024-04-25T08:53:56.872250+0000","last_undegraded":"2024-04-25T08:53:56.872250+0000","last_fullsized":"2024-04-25T08:53:56.872250+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:25.881602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.861626+0000","last_change":"2024-04-25T08:53:54.842737+0000","last_active":"2024-04-25T08:53:55.861626+0000","last_peered":"2024-04-25T08:53:55.861626+0000","last_clean":"2024-04-25T08:53:55.861626+0000","last_became_active":"2024-04-25T08:53:54.842345+0000","last_became_peered":"2024-04-25T08:53:54.842345+0000","last_unstale":"2024-04-25T08:53:55.861626+0000","last_undegraded":"2024-04-25T08:53:55.861626+0000","last_fullsized":"2024-04-25T08:53:55.861626+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:12.639132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.878849+0000","last_change":"2024-04-25T08:53:55.873870+0000","last_active":"2024-04-25T08:53:55.878849+0000","last_peered":"2024-04-25T08:53:55.878849+0000","last_clean":"2024-04-25T08:53:55.878849+0000","last_became_active":"2024-04-25T08:53:55.873628+0000","last_became_peered":"2024-04-25T08:53:55.873628+0000","last_unstale":"2024-04-25T08:53:55.878849+0000","last_undegraded":"2024-04-25T08:53:55.878849+0000","last_fullsized":"2024-04-25T08:53:55.878849+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:51.758262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-25T08:53:55.858878+0000","last_change":"2024-04-25T08:53:54.844627+0000","last_active":"2024-04-25T08:53:55.858878+0000","last_peered":"2024-04-25T08:53:55.858878+0000","last_clean":"2024-04-25T08:53:55.858878+0000","last_became_active":"2024-04-25T08:53:54.844100+0000","last_became_peered":"2024-04-25T08:53:54.844100+0000","last_unstale":"2024-04-25T08:53:55.858878+0000","last_undegraded":"2024-04-25T08:53:55.858878+0000","last_fullsized":"2024-04-25T08:53:55.858878+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:13.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.442907+0000","last_change":"2024-04-25T08:53:55.883542+0000","last_active":"2024-04-25T08:53:57.442907+0000","last_peered":"2024-04-25T08:53:57.442907+0000","last_clean":"2024-04-25T08:53:57.442907+0000","last_became_active":"2024-04-25T08:53:55.883298+0000","last_became_peered":"2024-04-25T08:53:55.883298+0000","last_unstale":"2024-04-25T08:53:57.442907+0000","last_undegraded":"2024-04-25T08:53:57.442907+0000","last_fullsized":"2024-04-25T08:53:57.442907+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:58.292874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871474+0000","last_change":"2024-04-25T08:53:54.833794+0000","last_active":"2024-04-25T08:53:56.871474+0000","last_peered":"2024-04-25T08:53:56.871474+0000","last_clean":"2024-04-25T08:53:56.871474+0000","last_became_active":"2024-04-25T08:53:54.833089+0000","last_became_peered":"2024-04-25T08:53:54.833089+0000","last_unstale":"2024-04-25T08:53:56.871474+0000","last_undegraded":"2024-04-25T08:53:56.871474+0000","last_fullsized":"2024-04-25T08:53:56.871474+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:58.593596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.870864+0000","last_change":"2024-04-25T08:53:55.886925+0000","last_active":"2024-04-25T08:53:56.870864+0000","last_peered":"2024-04-25T08:53:56.870864+0000","last_clean":"2024-04-25T08:53:56.870864+0000","last_became_active":"2024-04-25T08:53:55.886032+0000","last_became_peered":"2024-04-25T08:53:55.886032+0000","last_unstale":"2024-04-25T08:53:56.870864+0000","last_undegraded":"2024-04-25T08:53:56.870864+0000","last_fullsized":"2024-04-25T08:53:56.870864+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:58.292874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.872562+0000","last_change":"2024-04-25T08:53:54.847271+0000","last_active":"2024-04-25T08:53:56.872562+0000","last_peered":"2024-04-25T08:53:56.872562+0000","last_clean":"2024-04-25T08:53:56.872562+0000","last_became_active":"2024-04-25T08:53:54.846542+0000","last_became_peered":"2024-04-25T08:53:54.846542+0000","last_unstale":"2024-04-25T08:53:56.872562+0000","last_undegraded":"2024-04-25T08:53:56.872562+0000","last_fullsized":"2024-04-25T08:53:56.872562+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:12.639132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339876+0000","last_change":"2024-04-25T08:53:55.902565+0000","last_active":"2024-04-25T08:53:58.339876+0000","last_peered":"2024-04-25T08:53:58.339876+0000","last_clean":"2024-04-25T08:53:58.339876+0000","last_became_active":"2024-04-25T08:53:55.901728+0000","last_became_peered":"2024-04-25T08:53:55.901728+0000","last_unstale":"2024-04-25T08:53:58.339876+0000","last_undegraded":"2024-04-25T08:53:58.339876+0000","last_fullsized":"2024-04-25T08:53:58.339876+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:47.567171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:56.871078+0000","last_change":"2024-04-25T08:53:54.842570+0000","last_active":"2024-04-25T08:53:56.871078+0000","last_peered":"2024-04-25T08:53:56.871078+0000","last_clean":"2024-04-25T08:53:56.871078+0000","last_became_active":"2024-04-25T08:53:54.841650+0000","last_became_peered":"2024-04-25T08:53:54.841650+0000","last_unstale":"2024-04-25T08:53:56.871078+0000","last_undegraded":"2024-04-25T08:53:56.871078+0000","last_fullsized":"2024-04-25T08:53:56.871078+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:58:27.372730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:57.760252+0000","last_change":"2024-04-25T08:53:55.919955+0000","last_active":"2024-04-25T08:53:57.760252+0000","last_peered":"2024-04-25T08:53:57.760252+0000","last_clean":"2024-04-25T08:53:57.760252+0000","last_became_active":"2024-04-25T08:53:55.919277+0000","last_became_peered":"2024-04-25T08:53:55.919277+0000","last_unstale":"2024-04-25T08:53:57.760252+0000","last_undegraded":"2024-04-25T08:53:57.760252+0000","last_fullsized":"2024-04-25T08:53:57.760252+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:54.816382+0000","last_clean_scrub_stamp":"2024-04-25T08:53:54.816382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.226385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339985+0000","last_change":"2024-04-25T08:53:54.871341+0000","last_active":"2024-04-25T08:53:58.339985+0000","last_peered":"2024-04-25T08:53:58.339985+0000","last_clean":"2024-04-25T08:53:58.339985+0000","last_became_active":"2024-04-25T08:53:54.870324+0000","last_became_peered":"2024-04-25T08:53:54.870324+0000","last_unstale":"2024-04-25T08:53:58.339985+0000","last_undegraded":"2024-04-25T08:53:58.339985+0000","last_fullsized":"2024-04-25T08:53:58.339985+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:53.807675+0000","last_clean_scrub_stamp":"2024-04-25T08:53:53.807675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:46.698881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":208,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-25T08:53:58.339536+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:53:58.339536+0000","last_peered":"2024-04-25T08:53:58.339536+0000","last_clean":"2024-04-25T08:53:58.339536+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:53:58.339536+0000","last_undegraded":"2024-04-25T08:53:58.339536+0000","last_fullsized":"2024-04-25T08:53:58.339536+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1073152,"data_stored":793934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":209158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":208688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":209180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":480,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":491520,"data_stored":210222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":208646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":468,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":479232,"data_stored":208666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1060864,"data_stored":792286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:54:00.296 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:54:00.296 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:54:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f17710da640 1 Processor -- start 2024-04-25T08:54:00.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f17710da640 1 -- start start 2024-04-25T08:54:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f17710da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f176c0b3cf0 0x7f176c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f17710da640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f176c0cfad0 0x7f176c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f17710da640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f176c0cbf10 con 0x7f176c15e4f0 2024-04-25T08:54:00.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f17710da640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f176c0cff10 con 0x7f176c0cfad0 2024-04-25T08:54:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f17710da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0cc1f0 con 0x7f176c0b3cf0 2024-04-25T08:54:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f176a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f176c0cfad0 0x7f176c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f176a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f176c0cfad0 0x7f176c15dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38040/0 (socket says 172.21.15.37:38040) 2024-04-25T08:54:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f176a59c640 1 -- 172.21.15.37:0/4080421971 learned_addr learned my addr 172.21.15.37:0/4080421971 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f176ad9d640 1 --2- 172.21.15.37:0/4080421971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f176c0b3cf0 0x7f176c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f176a59c640 1 -- 172.21.15.37:0/4080421971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f176c0b3cf0 msgr2=0x7f176c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f176a59c640 1 --2- 172.21.15.37:0/4080421971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f176c0b3cf0 0x7f176c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f176a59c640 1 -- 172.21.15.37:0/4080421971 >> v1:172.21.15.203:6789/0 conn(0x7f176c15e4f0 legacy=0x7f176c161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.464+0000 7f176a59c640 1 -- 172.21.15.37:0/4080421971 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c0cc4d0 con 0x7f176c0cfad0 2024-04-25T08:54:00.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f176a59c640 1 --2- 172.21.15.37:0/4080421971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176c0cfad0 0x7f176c15dfb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1760009870 tx=0x7f176001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e0833c4a1152ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:00.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f1769d9b640 1 -- 172.21.15.37:0/4080421971 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17600b0070 con 0x7f176c0cfad0 2024-04-25T08:54:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f1769d9b640 1 -- 172.21.15.37:0/4080421971 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17600ab600 con 0x7f176c0cfad0 2024-04-25T08:54:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f1769d9b640 1 -- 172.21.15.37:0/4080421971 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17600ab9d0 con 0x7f176c0cfad0 2024-04-25T08:54:00.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f17710da640 1 -- 172.21.15.37:0/4080421971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176c0cfad0 msgr2=0x7f176c15dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f17710da640 1 --2- 172.21.15.37:0/4080421971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176c0cfad0 0x7f176c15dfb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f1760009870 tx=0x7f176001c560 comp rx=0 tx=0).stop 2024-04-25T08:54:00.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f17710da640 1 -- 172.21.15.37:0/4080421971 shutdown_connections 2024-04-25T08:54:00.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f17710da640 1 --2- 172.21.15.37:0/4080421971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176c0cfad0 0x7f176c15dfb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f17710da640 1 --2- 172.21.15.37:0/4080421971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f176c0b3cf0 0x7f176c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f17710da640 1 -- 172.21.15.37:0/4080421971 >> 172.21.15.37:0/4080421971 conn(0x7f176c06ae60 msgr2=0x7f176c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:00.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f17710da640 1 -- 172.21.15.37:0/4080421971 shutdown_connections 2024-04-25T08:54:00.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f17710da640 1 -- 172.21.15.37:0/4080421971 wait complete. 2024-04-25T08:54:00.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f17710da640 1 Processor -- start 2024-04-25T08:54:00.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.468+0000 7f17710da640 1 -- start start 2024-04-25T08:54:00.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17710da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f176c0b3cf0 0x7f176c154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17710da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f176c0cfad0 0x7f176c156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17710da640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176c15e4f0 0x7f176c0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17710da640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0b4250 con 0x7f176c15e4f0 2024-04-25T08:54:00.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17710da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f176c0b40d0 con 0x7f176c0b3cf0 2024-04-25T08:54:00.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17710da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f176c15e900 con 0x7f176c0cfad0 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176c15e4f0 0x7f176c0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176c15e4f0 0x7f176c0cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38052/0 (socket says 172.21.15.37:38052) 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176b59e640 1 -- 172.21.15.37:0/46994813 learned_addr learned my addr 172.21.15.37:0/46994813 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176a59c640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f176c0cfad0 0x7f176c156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176ad9d640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f176c0b3cf0 0x7f176c154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176b59e640 1 -- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f176c0cfad0 msgr2=0x7f176c156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176b59e640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f176c0cfad0 0x7f176c156190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176b59e640 1 -- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f176c0b3cf0 msgr2=0x7f176c154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176b59e640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f176c0b3cf0 0x7f176c154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176b59e640 1 -- 172.21.15.37:0/46994813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f176c15a740 con 0x7f176c15e4f0 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176a59c640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f176c0cfad0 0x7f176c156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f176b59e640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176c15e4f0 0x7f176c0cdf60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f176400c730 tx=0x7f176400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:00.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17537fe640 1 -- 172.21.15.37:0/46994813 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764003b80 con 0x7f176c15e4f0 2024-04-25T08:54:00.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17537fe640 1 -- 172.21.15.37:0/46994813 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1764003d20 con 0x7f176c15e4f0 2024-04-25T08:54:00.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17537fe640 1 -- 172.21.15.37:0/46994813 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176400aa30 con 0x7f176c15e4f0 2024-04-25T08:54:00.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17710da640 1 -- 172.21.15.37:0/46994813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f176c15aa20 con 0x7f176c15e4f0 2024-04-25T08:54:00.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17710da640 1 -- 172.21.15.37:0/46994813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f176c15afe0 con 0x7f176c15e4f0 2024-04-25T08:54:00.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17710da640 1 -- 172.21.15.37:0/46994813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f176c04efa0 con 0x7f176c15e4f0 2024-04-25T08:54:00.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17537fe640 1 -- 172.21.15.37:0/46994813 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f176400abd0 con 0x7f176c15e4f0 2024-04-25T08:54:00.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.472+0000 7f17537fe640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1758066f60 0x7f1758069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.476+0000 7f17537fe640 1 -- 172.21.15.37:0/46994813 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1764084fd0 con 0x7f176c15e4f0 2024-04-25T08:54:00.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.476+0000 7f176ad9d640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1758066f60 0x7f1758069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.476+0000 7f176ad9d640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1758066f60 0x7f1758069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f176c0acbe0 tx=0x7f175c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.480+0000 7f17537fe640 1 -- 172.21.15.37:0/46994813 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1764055ee0 con 0x7f176c15e4f0 2024-04-25T08:54:00.637 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T08:54:00.644+0000 7f897e871640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:54:00.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.696+0000 7f17710da640 1 -- 172.21.15.37:0/46994813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f176c0b3060 con 0x7f176c15e4f0 2024-04-25T08:54:00.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.700+0000 7f17537fe640 1 -- 172.21.15.37:0/46994813 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f17640557e0 con 0x7f176c15e4f0 2024-04-25T08:54:00.694 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T08:54:00.695 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA== 2024-04-25T08:54:00.695 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-04-25T08:54:00.695 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-04-25T08:54:00.695 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:54:00.695 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 -- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1758066f60 msgr2=0x7f1758069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1758066f60 0x7f1758069420 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f176c0acbe0 tx=0x7f175c008040 comp rx=0 tx=0).stop 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 -- 172.21.15.37:0/46994813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176c15e4f0 msgr2=0x7f176c0cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176c15e4f0 0x7f176c0cdf60 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f176400c730 tx=0x7f176400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 -- 172.21.15.37:0/46994813 shutdown_connections 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1758066f60 0x7f1758069420 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f176c15e4f0 0x7f176c0cdf60 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f176c0cfad0 0x7f176c156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 --2- 172.21.15.37:0/46994813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f176c0b3cf0 0x7f176c154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 -- 172.21.15.37:0/46994813 >> 172.21.15.37:0/46994813 conn(0x7f176c06ae60 msgr2=0x7f176c0ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 -- 172.21.15.37:0/46994813 shutdown_connections 2024-04-25T08:54:00.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.704+0000 7f17710da640 1 -- 172.21.15.37:0/46994813 wait complete. 2024-04-25T08:54:00.713 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:00.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc26c09640 1 Processor -- start 2024-04-25T08:54:00.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc26c09640 1 -- start start 2024-04-25T08:54:00.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc26c09640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbc200ce510 0x7fbc200ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc26c09640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc201451f0 0x7fbc20142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc26c09640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc200cee30 con 0x7fbc20144ca0 2024-04-25T08:54:00.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc26c09640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc200cc2b0 con 0x7fbc200ce510 2024-04-25T08:54:00.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc26c09640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2006e290 con 0x7fbc201451f0 2024-04-25T08:54:00.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc249a5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbc200ce510 0x7fbc200ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc249a5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbc200ce510 0x7fbc200ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38064/0 (socket says 172.21.15.37:38064) 2024-04-25T08:54:00.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc249a5640 1 -- 172.21.15.37:0/2630116330 learned_addr learned my addr 172.21.15.37:0/2630116330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:00.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc251a6640 1 --2- 172.21.15.37:0/2630116330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc201451f0 0x7fbc20142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc249a5640 1 -- 172.21.15.37:0/2630116330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc201451f0 msgr2=0x7fbc20142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:00.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc249a5640 1 --2- 172.21.15.37:0/2630116330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc201451f0 0x7fbc20142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc249a5640 1 -- 172.21.15.37:0/2630116330 >> v1:172.21.15.203:6789/0 conn(0x7fbc20144ca0 legacy=0x7fbc2006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:00.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.844+0000 7fbc249a5640 1 -- 172.21.15.37:0/2630116330 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc200ccc70 con 0x7fbc200ce510 2024-04-25T08:54:00.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc249a5640 1 --2- 172.21.15.37:0/2630116330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc200ce510 0x7fbc200ce8f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009870 tx=0x7fbc1001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aff18df5779f9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:00.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc1f7fe640 1 -- 172.21.15.37:0/2630116330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc100b5070 con 0x7fbc200ce510 2024-04-25T08:54:00.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc1f7fe640 1 -- 172.21.15.37:0/2630116330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc100a89d0 con 0x7fbc200ce510 2024-04-25T08:54:00.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc1f7fe640 1 -- 172.21.15.37:0/2630116330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc100a8da0 con 0x7fbc200ce510 2024-04-25T08:54:00.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 -- 172.21.15.37:0/2630116330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc200ce510 msgr2=0x7fbc200ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 --2- 172.21.15.37:0/2630116330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc200ce510 0x7fbc200ce8f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009870 tx=0x7fbc1001ed10 comp rx=0 tx=0).stop 2024-04-25T08:54:00.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 -- 172.21.15.37:0/2630116330 shutdown_connections 2024-04-25T08:54:00.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 --2- 172.21.15.37:0/2630116330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc201451f0 0x7fbc20142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 --2- 172.21.15.37:0/2630116330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc200ce510 0x7fbc200ce8f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 -- 172.21.15.37:0/2630116330 >> 172.21.15.37:0/2630116330 conn(0x7fbc200b11a0 msgr2=0x7fbc200b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:00.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 -- 172.21.15.37:0/2630116330 shutdown_connections 2024-04-25T08:54:00.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 -- 172.21.15.37:0/2630116330 wait complete. 2024-04-25T08:54:00.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 Processor -- start 2024-04-25T08:54:00.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 -- start start 2024-04-25T08:54:00.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc200ce510 0x7fbc200c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc20144ca0 0x7fbc20140a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc201451f0 0x7fbc20140f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2006ee30 con 0x7fbc201451f0 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2006df20 con 0x7fbc20144ca0 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc26c09640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc200cc900 con 0x7fbc200ce510 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc251a6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc201451f0 0x7fbc20140f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc251a6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc201451f0 0x7fbc20140f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38072/0 (socket says 172.21.15.37:38072) 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc251a6640 1 -- 172.21.15.37:0/824666187 learned_addr learned my addr 172.21.15.37:0/824666187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc249a5640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc200ce510 0x7fbc200c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc1ffff640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc20144ca0 0x7fbc20140a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc251a6640 1 -- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc200ce510 msgr2=0x7fbc200c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc251a6640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc200ce510 0x7fbc200c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc251a6640 1 -- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc20144ca0 msgr2=0x7fbc20140a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:00.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc251a6640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc20144ca0 0x7fbc20140a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:00.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc251a6640 1 -- 172.21.15.37:0/824666187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc200ca750 con 0x7fbc201451f0 2024-04-25T08:54:00.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.848+0000 7fbc251a6640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc201451f0 0x7fbc20140f90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbc18007e60 tx=0x7fbc1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:00.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.852+0000 7fbc1d7fa640 1 -- 172.21.15.37:0/824666187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18014420 con 0x7fbc201451f0 2024-04-25T08:54:00.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.852+0000 7fbc1d7fa640 1 -- 172.21.15.37:0/824666187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc18019070 con 0x7fbc201451f0 2024-04-25T08:54:00.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.852+0000 7fbc26c09640 1 -- 172.21.15.37:0/824666187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2013ff70 con 0x7fbc201451f0 2024-04-25T08:54:00.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.852+0000 7fbc1d7fa640 1 -- 172.21.15.37:0/824666187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc18014820 con 0x7fbc201451f0 2024-04-25T08:54:00.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.852+0000 7fbc26c09640 1 -- 172.21.15.37:0/824666187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc20140530 con 0x7fbc201451f0 2024-04-25T08:54:00.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.852+0000 7fbc26c09640 1 -- 172.21.15.37:0/824666187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2013fd90 con 0x7fbc201451f0 2024-04-25T08:54:00.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.852+0000 7fbc1d7fa640 1 -- 172.21.15.37:0/824666187 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc180149c0 con 0x7fbc201451f0 2024-04-25T08:54:00.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.852+0000 7fbc1d7fa640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc0c066f60 0x7fbc0c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:00.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.852+0000 7fbc1d7fa640 1 -- 172.21.15.37:0/824666187 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc18085a60 con 0x7fbc201451f0 2024-04-25T08:54:00.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.852+0000 7fbc249a5640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc0c066f60 0x7fbc0c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:00.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.852+0000 7fbc249a5640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc0c066f60 0x7fbc0c069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbc100b2000 tx=0x7fbc10008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:00.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:00.856+0000 7fbc1d7fa640 1 -- 172.21.15.37:0/824666187 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbc2013fd90 con 0x7fbc201451f0 2024-04-25T08:54:01.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.052+0000 7fbc26c09640 1 -- 172.21.15.37:0/824666187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc2004efa0 con 0x7fbc201451f0 2024-04-25T08:54:01.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.052+0000 7fbc1d7fa640 1 -- 172.21.15.37:0/824666187 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fbc1805c070 con 0x7fbc201451f0 2024-04-25T08:54:01.045 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:01.045 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3245776986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3245776986},{"type":"v1","addr":"172.21.15.203:6836","nonce":3245776986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5056,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/841808653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":841808653},{"type":"v1","addr":"172.21.15.37:6835","nonce":841808653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1908736119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1908736119},{"type":"v1","addr":"172.21.15.203:6837","nonce":1908736119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:53:55.212059+0000","modified":"2024-04-25T08:53:58.247733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035847732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035847732},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035847732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:54:01.046 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-04-25T08:54:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 -- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc0c066f60 msgr2=0x7fbc0c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc0c066f60 0x7fbc0c069420 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbc100b2000 tx=0x7fbc10008040 comp rx=0 tx=0).stop 2024-04-25T08:54:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 -- 172.21.15.37:0/824666187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc201451f0 msgr2=0x7fbc20140f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc201451f0 0x7fbc20140f90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbc18007e60 tx=0x7fbc1800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 -- 172.21.15.37:0/824666187 shutdown_connections 2024-04-25T08:54:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc0c066f60 0x7fbc0c069420 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc201451f0 0x7fbc20140f90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc20144ca0 0x7fbc20140a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 --2- 172.21.15.37:0/824666187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc200ce510 0x7fbc200c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 -- 172.21.15.37:0/824666187 >> 172.21.15.37:0/824666187 conn(0x7fbc200b11a0 msgr2=0x7fbc200b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:01.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 -- 172.21.15.37:0/824666187 shutdown_connections 2024-04-25T08:54:01.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.056+0000 7fbc26c09640 1 -- 172.21.15.37:0/824666187 wait complete. 2024-04-25T08:54:01.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:53:55.212059+0000', 'modified': '2024-04-25T08:53:58.247733+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5445}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5445': {'gid': 5445, 'name': 'a', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6834/3035847732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6832', 'nonce': 3035847732}, {'type': 'v1', 'addr': '172.21.15.37:6834', 'nonce': 3035847732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:54:01.468 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:54:01.468 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:54:01.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2f68b0640 1 Processor -- start 2024-04-25T08:54:01.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2f68b0640 1 -- start start 2024-04-25T08:54:01.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2f68b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d07f0 0x7fd2f00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2f68b0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f006be60 con 0x7fd2f015da80 2024-04-25T08:54:01.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2f68b0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f015de60 con 0x7fd2f00d1cd0 2024-04-25T08:54:01.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2f68b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f006c440 con 0x7fd2f00d07f0 2024-04-25T08:54:01.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2effff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd2f015da80 0x7fd2f00d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41884/0 (socket says 172.21.15.37:41884) 2024-04-25T08:54:01.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2effff640 1 -- 172.21.15.37:0/4108866334 learned_addr learned my addr 172.21.15.37:0/4108866334 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:01.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2ef7fe640 1 --2- 172.21.15.37:0/4108866334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d07f0 0x7fd2f00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2eeffd640 1 -- 172.21.15.37:0/4108866334 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 973529406 0 0) 0x7fd2f006be60 con 0x7fd2f015da80 2024-04-25T08:54:01.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2eeffd640 1 -- 172.21.15.37:0/4108866334 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2dc003530 con 0x7fd2f015da80 2024-04-25T08:54:01.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2eeffd640 1 -- 172.21.15.37:0/4108866334 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2293325044 0 0) 0x7fd2f015de60 con 0x7fd2f00d1cd0 2024-04-25T08:54:01.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.612+0000 7fd2eeffd640 1 -- 172.21.15.37:0/4108866334 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f006be60 con 0x7fd2f00d1cd0 2024-04-25T08:54:01.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2ef7fe640 1 -- 172.21.15.37:0/4108866334 >> v1:172.21.15.203:6789/0 conn(0x7fd2f00d1cd0 legacy=0x7fd2f006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2ef7fe640 1 -- 172.21.15.37:0/4108866334 >> v1:172.21.15.37:6789/0 conn(0x7fd2f015da80 legacy=0x7fd2f00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2eeffd640 1 -- 172.21.15.37:0/4108866334 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1866052157 0 0) 0x7fd2dc003530 con 0x7fd2f015da80 2024-04-25T08:54:01.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2ef7fe640 1 -- 172.21.15.37:0/4108866334 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f00d2450 con 0x7fd2f00d07f0 2024-04-25T08:54:01.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2ef7fe640 1 --2- 172.21.15.37:0/4108866334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d07f0 0x7fd2f00d4230 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80097d0 tx=0x7fd2d801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90a3171543f70c9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:01.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2eeffd640 1 -- 172.21.15.37:0/4108866334 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80b0070 con 0x7fd2f00d07f0 2024-04-25T08:54:01.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2eeffd640 1 -- 172.21.15.37:0/4108866334 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d80ab400 con 0x7fd2f00d07f0 2024-04-25T08:54:01.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2eeffd640 1 -- 172.21.15.37:0/4108866334 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e0001680 con 0x7fd2f00d07f0 2024-04-25T08:54:01.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2f68b0640 1 -- 172.21.15.37:0/4108866334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d07f0 msgr2=0x7fd2f00d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2f68b0640 1 --2- 172.21.15.37:0/4108866334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d07f0 0x7fd2f00d4230 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80097d0 tx=0x7fd2d801c3b0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2f68b0640 1 -- 172.21.15.37:0/4108866334 shutdown_connections 2024-04-25T08:54:01.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2f68b0640 1 --2- 172.21.15.37:0/4108866334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d07f0 0x7fd2f00d4230 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2f68b0640 1 -- 172.21.15.37:0/4108866334 >> 172.21.15.37:0/4108866334 conn(0x7fd2f008b000 msgr2=0x7fd2f00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:01.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2f68b0640 1 -- 172.21.15.37:0/4108866334 shutdown_connections 2024-04-25T08:54:01.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.616+0000 7fd2f68b0640 1 -- 172.21.15.37:0/4108866334 wait complete. 2024-04-25T08:54:01.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f68b0640 1 Processor -- start 2024-04-25T08:54:01.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f68b0640 1 -- start start 2024-04-25T08:54:01.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f68b0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2f00d07f0 0x7fd2f014e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f68b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d1cd0 0x7fd2f014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f68b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2f015da80 0x7fd2f00cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f68b0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f006d970 con 0x7fd2f015da80 2024-04-25T08:54:01.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f68b0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f006d7f0 con 0x7fd2f00d07f0 2024-04-25T08:54:01.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f68b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f006daf0 con 0x7fd2f00d1cd0 2024-04-25T08:54:01.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f4e4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2f015da80 0x7fd2f00cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f4e4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2f015da80 0x7fd2f00cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38074/0 (socket says 172.21.15.37:38074) 2024-04-25T08:54:01.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/2742328915 learned_addr learned my addr 172.21.15.37:0/2742328915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:01.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2ef7fe640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d1cd0 0x7fd2f014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2effff640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2f00d07f0 0x7fd2f014e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d1cd0 msgr2=0x7fd2f014eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d1cd0 0x7fd2f014eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2f00d07f0 msgr2=0x7fd2f014e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2f00d07f0 0x7fd2f014e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/2742328915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f00cd020 con 0x7fd2f015da80 2024-04-25T08:54:01.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2ef7fe640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d1cd0 0x7fd2f014eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:01.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2f015da80 0x7fd2f00cc010 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c790 tx=0x7fd2e800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:01.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2ecff9640 1 -- 172.21.15.37:0/2742328915 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8003b80 con 0x7fd2f015da80 2024-04-25T08:54:01.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2ecff9640 1 -- 172.21.15.37:0/2742328915 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8003d20 con 0x7fd2f015da80 2024-04-25T08:54:01.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f68b0640 1 -- 172.21.15.37:0/2742328915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f00cd300 con 0x7fd2f015da80 2024-04-25T08:54:01.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2ecff9640 1 -- 172.21.15.37:0/2742328915 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e800a9f0 con 0x7fd2f015da80 2024-04-25T08:54:01.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.620+0000 7fd2f68b0640 1 -- 172.21.15.37:0/2742328915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f015e340 con 0x7fd2f015da80 2024-04-25T08:54:01.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2ecff9640 1 -- 172.21.15.37:0/2742328915 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2e800ab90 con 0x7fd2f015da80 2024-04-25T08:54:01.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2ecff9640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2d0066f90 0x7fd2d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2ecff9640 1 -- 172.21.15.37:0/2742328915 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2e8085090 con 0x7fd2f015da80 2024-04-25T08:54:01.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2effff640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2d0066f90 0x7fd2d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f9a1d000 1 Processor -- start 2024-04-25T08:54:01.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f9a1d000 1 -- start start 2024-04-25T08:54:01.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2effff640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2d0066f90 0x7fd2d0069450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00011f0 tx=0x7fd2e0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:01.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd455920 0x55fefd455d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd456240 0x55fefd40b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd40bf30 0x55fefd40c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefd2f35f0 con 0x55fefd40bf30 2024-04-25T08:54:01.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefd25ada0 con 0x55fefd455920 2024-04-25T08:54:01.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fefd1e3150 con 0x55fefd456240 2024-04-25T08:54:01.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f4e4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd40bf30 0x55fefd40c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f4e4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd40bf30 0x55fefd40c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38084/0 (socket says 172.21.15.37:38084) 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/790320307 learned_addr learned my addr 172.21.15.37:0/790320307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2ef7fe640 1 --2- 172.21.15.37:0/790320307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd456240 0x55fefd40b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2effff640 1 --2- 172.21.15.37:0/790320307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd455920 0x55fefd455d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/790320307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd456240 msgr2=0x55fefd40b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/790320307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd456240 0x55fefd40b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/790320307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd455920 msgr2=0x55fefd455d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/790320307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd455920 0x55fefd455d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/790320307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fefd1fcca0 con 0x55fefd40bf30 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/790320307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd40bf30 0x55fefd40c310 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8055830 tx=0x7fd2e80552c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789ae98f224db5f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2e5ffb640 1 -- 172.21.15.37:0/790320307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e805dfb0 con 0x55fefd40bf30 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2e5ffb640 1 -- 172.21.15.37:0/790320307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8019de0 con 0x55fefd40bf30 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.624+0000 7fd2e5ffb640 1 -- 172.21.15.37:0/790320307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8051f10 con 0x55fefd40bf30 2024-04-25T08:54:01.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/790320307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd40bf30 msgr2=0x55fefd40c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/790320307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd40bf30 0x55fefd40c310 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8055830 tx=0x7fd2e80552c0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/790320307 shutdown_connections 2024-04-25T08:54:01.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/790320307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd40bf30 0x55fefd40c310 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/790320307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd456240 0x55fefd40b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/790320307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd455920 0x55fefd455d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/790320307 >> 172.21.15.37:0/790320307 conn(0x55fefd4122a0 msgr2=0x55fefd40a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:01.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/790320307 shutdown_connections 2024-04-25T08:54:01.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/790320307 wait complete. 2024-04-25T08:54:01.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- start start 2024-04-25T08:54:01.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 Processor -- start 2024-04-25T08:54:01.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd40bf30 0x55fefd45cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd45d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd456240 0x55fefd40faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefd215990 con 0x55fefd455920 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefd256e40 con 0x55fefd40bf30 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fefd1e6900 con 0x55fefd456240 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 10 client.0 _ll_register_callbacks cb 0x55fefd30e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd45d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd45d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38092/0 (socket says 172.21.15.37:38092) 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ef7fe640 1 -- 172.21.15.37:0/4120538627 learned_addr learned my addr 172.21.15.37:0/4120538627 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2effff640 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd40bf30 0x55fefd45cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd456240 0x55fefd40faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ef7fe640 1 -- 172.21.15.37:0/4120538627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd456240 msgr2=0x55fefd40faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ef7fe640 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd456240 0x55fefd40faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ef7fe640 1 -- 172.21.15.37:0/4120538627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd40bf30 msgr2=0x55fefd45cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ef7fe640 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd40bf30 0x55fefd45cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ef7fe640 1 -- 172.21.15.37:0/4120538627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fefd213630 con 0x55fefd455920 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ef7fe640 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd45d2a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd2d801c3f0 tx=0x7fd2d80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8756c16e04118a2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ceffd640 1 -- 172.21.15.37:0/4120538627 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80ba030 con 0x55fefd455920 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ceffd640 1 -- 172.21.15.37:0/4120538627 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2d8003620 con 0x55fefd455920 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/4120538627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2e8019de0 con 0x55fefd455920 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ceffd640 1 -- 172.21.15.37:0/4120538627 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2d80c6050 con 0x55fefd455920 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 10 client.5625 fetch_fsmap learned FSMap version 17 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/4120538627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55fefd1e33a0 con 0x55fefd455920 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ceffd640 1 -- 172.21.15.37:0/4120538627 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8019de0 con 0x55fefd455920 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ceffd640 1 -- 172.21.15.37:0/4120538627 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd2d80c9020 con 0x55fefd455920 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 10 client.5625 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 10 client.5625 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 20 client.5625 populate_metadata read hostname 'smithi037' 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.37:6832/3035847732,v1:172.21.15.37:6834/3035847732] conn(0x55fefd411160 0x55fefd466dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 4 client.5625 mds_command: new command op to 5445 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:54:01.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/4120538627 --> [v2:172.21.15.37:6832/3035847732,v1:172.21.15.37:6834/3035847732] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fefd1f6a50 con 0x55fefd411160 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2effff640 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.37:6832/3035847732,v1:172.21.15.37:6834/3035847732] conn(0x55fefd411160 0x55fefd466dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2effff640 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.37:6832/3035847732,v1:172.21.15.37:6834/3035847732] conn(0x55fefd411160 0x55fefd466dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9915644e32afa12 server_cookie=bc3529681b913ff4 in_seq=0 out_seq=0 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.628+0000 7fd2ceffd640 10 client.5625 ms_handle_connect on v2:172.21.15.37:6832/3035847732 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2ceffd640 1 -- 172.21.15.37:0/4120538627 <== mds.0 v2:172.21.15.37:6832/3035847732 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fefd1f6a50 con 0x55fefd411160 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2ceffd640 10 client.5625 handle_command_reply: tid=0 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 1 client.5625 shutdown 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/4120538627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 msgr2=0x55fefd45d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd45d2a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd2d801c3f0 tx=0x7fd2d80046d0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/4120538627 shutdown_connections 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.37:6832/3035847732,v1:172.21.15.37:6834/3035847732] conn(0x55fefd411160 0x55fefd466dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd456240 0x55fefd40faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd45d2a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/4120538627 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd40bf30 0x55fefd45cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2ceffd640 0 client.5625 ms_handle_reset on v2:172.21.15.37:6832/3035847732 2024-04-25T08:54:01.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/4120538627 >> 172.21.15.37:0/4120538627 conn(0x55fefd4122a0 msgr2=0x55fefd40a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:01.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/4120538627 shutdown_connections 2024-04-25T08:54:01.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/4120538627 wait complete. 2024-04-25T08:54:01.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.632+0000 7fd2f9a1d000 20 client.5625 trim_cache size 0 max 16384 2024-04-25T08:54:01.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 Processor -- start 2024-04-25T08:54:01.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 -- start start 2024-04-25T08:54:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd40bf30 0x55fefd40f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd455920 0x55fefd40f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd456240 0x55fefd45f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f0201ec0 con 0x55fefd456240 2024-04-25T08:54:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f015f5d0 con 0x55fefd455920 2024-04-25T08:54:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f0201a10 con 0x55fefd40bf30 2024-04-25T08:54:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f4e4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd456240 0x55fefd45f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f4e4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd456240 0x55fefd45f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38098/0 (socket says 172.21.15.37:38098) 2024-04-25T08:54:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/3922534074 learned_addr learned my addr 172.21.15.37:0/3922534074 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/3922534074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd40bf30 msgr2=0x55fefd40f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:01.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/3922534074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd40bf30 0x55fefd40f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/3922534074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd455920 msgr2=0x55fefd40f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/3922534074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd455920 0x55fefd40f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f4e4d640 1 -- 172.21.15.37:0/3922534074 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fefd456070 con 0x55fefd456240 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/3922534074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd456240 0x55fefd45f990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8001590 tx=0x7fd2e8005070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e963c45384513c2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2e57fa640 1 -- 172.21.15.37:0/3922534074 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8072070 con 0x55fefd456240 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2e57fa640 1 -- 172.21.15.37:0/3922534074 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e805dfb0 con 0x55fefd456240 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2e57fa640 1 -- 172.21.15.37:0/3922534074 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8053ba0 con 0x55fefd456240 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/3922534074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd456240 msgr2=0x55fefd45f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/3922534074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd456240 0x55fefd45f990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8001590 tx=0x7fd2e8005070 comp rx=0 tx=0).stop 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/3922534074 shutdown_connections 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/3922534074 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd456240 0x55fefd45f990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/3922534074 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd455920 0x55fefd40f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/3922534074 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd40bf30 0x55fefd40f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/3922534074 >> 172.21.15.37:0/3922534074 conn(0x55fefd4122a0 msgr2=0x55fefd442170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:01.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/3922534074 shutdown_connections 2024-04-25T08:54:01.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/3922534074 wait complete. 2024-04-25T08:54:01.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 -- start start 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.648+0000 7fd2f9a1d000 1 Processor -- start 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd40bf30 0x55fefd4626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd462c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd456240 0x55fefd470840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefd25f3d0 con 0x55fefd455920 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fefd316d50 con 0x55fefd456240 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fefd1f5980 con 0x55fefd40bf30 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 10 client.0 _ll_register_callbacks cb 0x7fd2e800af00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd462c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ef7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd462c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38110/0 (socket says 172.21.15.37:38110) 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ef7fe640 1 -- 172.21.15.37:0/2002333034 learned_addr learned my addr 172.21.15.37:0/2002333034 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2effff640 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd40bf30 0x55fefd4626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd456240 0x55fefd470840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ef7fe640 1 -- 172.21.15.37:0/2002333034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd40bf30 msgr2=0x55fefd4626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ef7fe640 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd40bf30 0x55fefd4626c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ef7fe640 1 -- 172.21.15.37:0/2002333034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd456240 msgr2=0x55fefd470840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ef7fe640 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd456240 0x55fefd470840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ef7fe640 1 -- 172.21.15.37:0/2002333034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fefd2dc600 con 0x55fefd455920 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f4e4d640 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd456240 0x55fefd470840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ef7fe640 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd462c00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd2d801cc50 tx=0x7fd2d8019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccd0159daf7370d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ceffd640 1 -- 172.21.15.37:0/2002333034 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d80ba040 con 0x55fefd455920 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ceffd640 1 -- 172.21.15.37:0/2002333034 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd2d80c0c50 con 0x55fefd455920 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/2002333034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd2e805dfb0 con 0x55fefd455920 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ceffd640 1 -- 172.21.15.37:0/2002333034 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2d80b0040 con 0x55fefd455920 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 10 client.5631 fetch_fsmap learned FSMap version 17 2024-04-25T08:54:01.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/2002333034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55fefd1ef9b0 con 0x55fefd455920 2024-04-25T08:54:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ceffd640 1 -- 172.21.15.37:0/2002333034 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e805dfb0 con 0x55fefd455920 2024-04-25T08:54:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ceffd640 1 -- 172.21.15.37:0/2002333034 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd2d80c9030 con 0x55fefd455920 2024-04-25T08:54:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 10 client.5631 fetch_fsmap finished waiting for FSMap version 17 2024-04-25T08:54:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 10 client.5631 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:54:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 20 client.5631 populate_metadata read hostname 'smithi037' 2024-04-25T08:54:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.37:6832/3035847732,v1:172.21.15.37:6834/3035847732] conn(0x55fefd49d1b0 0x55fefd470d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 4 client.5631 mds_command: new command op to 5445 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:54:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/2002333034 --> [v2:172.21.15.37:6832/3035847732,v1:172.21.15.37:6834/3035847732] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fefd2682e0 con 0x55fefd49d1b0 2024-04-25T08:54:01.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2effff640 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.37:6832/3035847732,v1:172.21.15.37:6834/3035847732] conn(0x55fefd49d1b0 0x55fefd470d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2effff640 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.37:6832/3035847732,v1:172.21.15.37:6834/3035847732] conn(0x55fefd49d1b0 0x55fefd470d80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707a8359b4a27ffd server_cookie=77a570c4be892aeb in_seq=0 out_seq=0 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ceffd640 10 client.5631 ms_handle_connect on v2:172.21.15.37:6832/3035847732 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ceffd640 1 -- 172.21.15.37:0/2002333034 <== mds.0 v2:172.21.15.37:6832/3035847732 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55fefd2682e0 con 0x55fefd49d1b0 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ceffd640 10 client.5631 handle_command_reply: tid=0 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 client.5631 shutdown 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/2002333034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 msgr2=0x55fefd462c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd462c00 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fd2d801cc50 tx=0x7fd2d8019150 comp rx=0 tx=0).stop 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/2002333034 shutdown_connections 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.37:6832/3035847732,v1:172.21.15.37:6834/3035847732] conn(0x55fefd49d1b0 0x55fefd470d80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55fefd456240 0x55fefd470840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fefd455920 0x55fefd462c00 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 --2- 172.21.15.37:0/2002333034 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55fefd40bf30 0x55fefd4626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/2002333034 >> 172.21.15.37:0/2002333034 conn(0x55fefd4122a0 msgr2=0x55fefd40e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:01.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2ceffd640 0 client.5631 ms_handle_reset on v2:172.21.15.37:6832/3035847732 2024-04-25T08:54:01.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/2002333034 shutdown_connections 2024-04-25T08:54:01.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 1 -- 172.21.15.37:0/2002333034 wait complete. 2024-04-25T08:54:01.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f9a1d000 20 client.5631 trim_cache size 0 max 16384 2024-04-25T08:54:01.652 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:54:01.652 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T08:54:01.652 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T08:54:01.652 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5445, 2024-04-25T08:54:01.652 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T08:54:01.652 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T08:54:01.653 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T08:54:01.653 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 5.8062398499999999, 2024-04-25T08:54:01.653 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 17, 2024-04-25T08:54:01.653 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 41, 2024-04-25T08:54:01.653 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 41, 2024-04-25T08:54:01.653 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 12.876673124 2024-04-25T08:54:01.653 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:54:01.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 -- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2d0066f90 msgr2=0x7fd2d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2d0066f90 0x7fd2d0069450 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00011f0 tx=0x7fd2e0009040 comp rx=0 tx=0).stop 2024-04-25T08:54:01.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 -- 172.21.15.37:0/2742328915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2f015da80 msgr2=0x7fd2f00cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2f015da80 0x7fd2f00cc010 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800c790 tx=0x7fd2e800cc50 comp rx=0 tx=0).stop 2024-04-25T08:54:01.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 -- 172.21.15.37:0/2742328915 shutdown_connections 2024-04-25T08:54:01.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd2d0066f90 0x7fd2d0069450 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2f015da80 0x7fd2f00cc010 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2f00d1cd0 0x7fd2f014eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 --2- 172.21.15.37:0/2742328915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2f00d07f0 0x7fd2f014e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 -- 172.21.15.37:0/2742328915 >> 172.21.15.37:0/2742328915 conn(0x7fd2f008b000 msgr2=0x7fd2f00b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:01.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 -- 172.21.15.37:0/2742328915 shutdown_connections 2024-04-25T08:54:01.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.652+0000 7fd2f68b0640 1 -- 172.21.15.37:0/2742328915 wait complete. 2024-04-25T08:54:01.663 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:54:01.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd08070d640 1 Processor -- start 2024-04-25T08:54:01.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd08070d640 1 -- start start 2024-04-25T08:54:01.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd08070d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd07815b920 0x7fd078099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd08070d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd078099fb0 0x7fd0780cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd08070d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07809a3c0 con 0x7fd0780b0440 2024-04-25T08:54:01.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd08070d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0780cd680 con 0x7fd07815b920 2024-04-25T08:54:01.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd08070d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07809a950 con 0x7fd078099fb0 2024-04-25T08:54:01.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd07dca8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd0780b0440 0x7fd0780aeb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41892/0 (socket says 172.21.15.37:41892) 2024-04-25T08:54:01.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd07dca8640 1 -- 172.21.15.37:0/3911883436 learned_addr learned my addr 172.21.15.37:0/3911883436 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:01.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd07e4a9640 1 --2- 172.21.15.37:0/3911883436 >> v2:172.21.15.203:3300/0 conn(0x7fd07815b920 0x7fd078099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd07ecaa640 1 --2- 172.21.15.37:0/3911883436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd078099fb0 0x7fd0780cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd07d4a7640 1 -- 172.21.15.37:0/3911883436 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992638162 0 0) 0x7fd068001120 con 0x7fd0780b0440 2024-04-25T08:54:01.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd07d4a7640 1 -- 172.21.15.37:0/3911883436 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd064003530 con 0x7fd0780b0440 2024-04-25T08:54:01.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd07e4a9640 1 -- 172.21.15.37:0/3911883436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd078099fb0 msgr2=0x7fd0780cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:01.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd07e4a9640 1 --2- 172.21.15.37:0/3911883436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd078099fb0 0x7fd0780cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd07e4a9640 1 -- 172.21.15.37:0/3911883436 >> v1:172.21.15.37:6789/0 conn(0x7fd0780b0440 legacy=0x7fd0780aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.780+0000 7fd07e4a9640 1 -- 172.21.15.37:0/3911883436 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07809ac30 con 0x7fd07815b920 2024-04-25T08:54:01.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07e4a9640 1 --2- 172.21.15.37:0/3911883436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd07815b920 0x7fd078099a70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd0740097c0 tx=0x7fd07401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e4f72478a735890 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:01.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07d4a7640 1 -- 172.21.15.37:0/3911883436 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0740b1070 con 0x7fd07815b920 2024-04-25T08:54:01.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07d4a7640 1 -- 172.21.15.37:0/3911883436 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0740ac410 con 0x7fd07815b920 2024-04-25T08:54:01.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07d4a7640 1 -- 172.21.15.37:0/3911883436 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0740ac7e0 con 0x7fd07815b920 2024-04-25T08:54:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 -- 172.21.15.37:0/3911883436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd07815b920 msgr2=0x7fd078099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 --2- 172.21.15.37:0/3911883436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd07815b920 0x7fd078099a70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd0740097c0 tx=0x7fd07401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 -- 172.21.15.37:0/3911883436 shutdown_connections 2024-04-25T08:54:01.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 --2- 172.21.15.37:0/3911883436 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd078099fb0 0x7fd0780cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 --2- 172.21.15.37:0/3911883436 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd07815b920 0x7fd078099a70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 -- 172.21.15.37:0/3911883436 >> 172.21.15.37:0/3911883436 conn(0x7fd0780b3010 msgr2=0x7fd0780b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:01.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 -- 172.21.15.37:0/3911883436 shutdown_connections 2024-04-25T08:54:01.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 -- 172.21.15.37:0/3911883436 wait complete. 2024-04-25T08:54:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 Processor -- start 2024-04-25T08:54:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 -- start start 2024-04-25T08:54:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd078099fb0 0x7fd0780c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0780b0440 0x7fd0781518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd07815b920 0x7fd07815a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780afc20 con 0x7fd078099fb0 2024-04-25T08:54:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780afaa0 con 0x7fd0780b0440 2024-04-25T08:54:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd08070d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780afda0 con 0x7fd07815b920 2024-04-25T08:54:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07e4a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd078099fb0 0x7fd0780c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07e4a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd078099fb0 0x7fd0780c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38116/0 (socket says 172.21.15.37:38116) 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07e4a9640 1 -- 172.21.15.37:0/3300645866 learned_addr learned my addr 172.21.15.37:0/3300645866 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07e4a9640 1 -- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd07815b920 msgr2=0x7fd07815a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07e4a9640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd07815b920 0x7fd07815a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07e4a9640 1 -- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0780b0440 msgr2=0x7fd0781518a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07e4a9640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0780b0440 0x7fd0781518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.784+0000 7fd07e4a9640 1 -- 172.21.15.37:0/3300645866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd078152210 con 0x7fd078099fb0 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd07e4a9640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd078099fb0 0x7fd0780c7090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd07401a0d0 tx=0x7fd074019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd06effd640 1 -- 172.21.15.37:0/3300645866 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0740b1040 con 0x7fd078099fb0 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd06effd640 1 -- 172.21.15.37:0/3300645866 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0740bb070 con 0x7fd078099fb0 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd08070d640 1 -- 172.21.15.37:0/3300645866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0781524f0 con 0x7fd078099fb0 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd06effd640 1 -- 172.21.15.37:0/3300645866 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0740b5410 con 0x7fd078099fb0 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd08070d640 1 -- 172.21.15.37:0/3300645866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd078152ab0 con 0x7fd078099fb0 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd08070d640 1 -- 172.21.15.37:0/3300645866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07804efa0 con 0x7fd078099fb0 2024-04-25T08:54:01.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd06effd640 1 -- 172.21.15.37:0/3300645866 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0740b55b0 con 0x7fd078099fb0 2024-04-25T08:54:01.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd06effd640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd05c066f60 0x7fd05c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:01.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd06effd640 1 -- 172.21.15.37:0/3300645866 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd07411eb40 con 0x7fd078099fb0 2024-04-25T08:54:01.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd07dca8640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd05c066f60 0x7fd05c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:01.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.788+0000 7fd07dca8640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd05c066f60 0x7fd05c069420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd068005890 tx=0x7fd068001160 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:01.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.792+0000 7fd06effd640 1 -- 172.21.15.37:0/3300645866 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd074016090 con 0x7fd078099fb0 2024-04-25T08:54:01.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.980+0000 7fd08070d640 1 -- 172.21.15.37:0/3300645866 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd0780c79f0 con 0x7fd078099fb0 2024-04-25T08:54:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.980+0000 7fd06effd640 1 -- 172.21.15.37:0/3300645866 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd0740ef940 con 0x7fd078099fb0 2024-04-25T08:54:01.974 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:54:01.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 -- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd05c066f60 msgr2=0x7fd05c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd05c066f60 0x7fd05c069420 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd068005890 tx=0x7fd068001160 comp rx=0 tx=0).stop 2024-04-25T08:54:01.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 -- 172.21.15.37:0/3300645866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd078099fb0 msgr2=0x7fd0780c7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:01.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd078099fb0 0x7fd0780c7090 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd07401a0d0 tx=0x7fd074019240 comp rx=0 tx=0).stop 2024-04-25T08:54:01.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 -- 172.21.15.37:0/3300645866 shutdown_connections 2024-04-25T08:54:01.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd05c066f60 0x7fd05c069420 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd07815b920 0x7fd07815a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0780b0440 0x7fd0781518a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 --2- 172.21.15.37:0/3300645866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd078099fb0 0x7fd0780c7090 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:01.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 -- 172.21.15.37:0/3300645866 >> 172.21.15.37:0/3300645866 conn(0x7fd0780b3010 msgr2=0x7fd0780737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:01.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 -- 172.21.15.37:0/3300645866 shutdown_connections 2024-04-25T08:54:01.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:01.984+0000 7fd08070d640 1 -- 172.21.15.37:0/3300645866 wait complete. 2024-04-25T08:54:01.991 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:54:01.991 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:54:01.991 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T08:54:01.991 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:54:01.991 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:01.991 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:54:01.992 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:54:01.992 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:54:01.992 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:54:01.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:01.992 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T08:54:01.998 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:54:01.998 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:54:01.998 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:54:01.998 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:01.998 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:54:01.998 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:01.998 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:54:01.998 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:01.998 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T08:54:01.998 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:54:01.998 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42399sec preferred_lft 42399sec 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:54:01.999 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:02.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:02.001 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:54:02.049 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:54:02.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:02.051 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T08:54:02.101 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T08:54:02.101 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:54:02.101 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:54:02.101 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:54:02.101 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:54:02.101 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:54:02.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:02.101 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:54:02.101 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:54:02.102 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:54:02.102 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:54:02.102 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:54:02.102 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:54:02.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:02.180 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:54:02.228 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:54:02.228 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:54:02.228 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:54:02.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:02.228 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T08:54:02.284 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:54:02.284 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:02.284 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:02.332 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:54:02.332 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:02.332 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:02.379 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:54:02.437 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:02.487 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:54:02.488 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:54:02.489 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:02.490 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:02.490 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:02.490 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:02.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:02.491 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:54:02.540 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:54:02.540 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:54:02.585 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:54:02.600 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:02.652 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:54:02.652 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:54:02.652 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:54:02.652 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:54:02.652 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:54:02.652 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:54:02.652 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:54:02.653 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:02.654 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:02.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:02.655 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:54:02.668 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T08:54:02.676+0000 7f746b260580 -1 init, newargv = 0x55ebecc26b60 newargc=15 2024-04-25T08:54:02.669 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[39457]: starting ceph client 2024-04-25T08:54:02.675 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[39457]: starting fuse 2024-04-25T08:54:03.774 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:54:03.790 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:03.838 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:54:03.838 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:54:03.838 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:54:03.838 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:03.839 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:03.840 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:03.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:03.841 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:54:03.888 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:54:03.888 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:03.889 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T08:54:03.890 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:54:04.016 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.39457.asok 2024-04-25T08:54:04.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:04.027 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.39457.asok 2024-04-25T08:54:04.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:04.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.39457.asok status 2024-04-25T08:54:04.193 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:54:04.193 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T08:54:04.193 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:54:04.193 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:54:04.193 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T08:54:04.193 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T08:54:04.193 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:54:04.193 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "39457", 2024-04-25T08:54:04.193 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T08:54:04.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5646, 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 5646 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3532270066 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:04.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3532270066 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.5646 192.168.0.1:0/3532270066", 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/3532270066", 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 17, 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 41, 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T08:54:04.195 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:54:04.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:04.207 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T08:54:04.207 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:04.213 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T08:54:04.213 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:04.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:04.213 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:04.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:04.267 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-04-25T08:54:04.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:04.320 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-04-25T08:54:04.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:04.372 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-04-25T08:54:04.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:04.429 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-04-25T08:54:04.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:04.485 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-04-25T08:54:04.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:04.537 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T08:54:04.606 INFO:teuthology.orchestra.run.smithi037.stdout:./ 2024-04-25T08:54:04.606 INFO:teuthology.orchestra.run.smithi037.stdout:./rfile 2024-04-25T08:54:04.606 INFO:teuthology.orchestra.run.smithi037.stdout:./parentdir 2024-04-25T08:54:04.606 INFO:teuthology.orchestra.run.smithi037.stdout:./parentdir/pfile 2024-04-25T08:54:04.606 INFO:teuthology.orchestra.run.smithi037.stdout:./parentdir/childdir 2024-04-25T08:54:04.606 INFO:teuthology.orchestra.run.smithi037.stdout:./parentdir/childdir/cfile 2024-04-25T08:54:04.607 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T08:54:04.607 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:54:04.607 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T08:54:04.607 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:04.607 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T08:54:04.607 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:54:04.718 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:54:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:04.728 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T08:54:04.728 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:54:04.728 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T08:54:04.728 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:04.728 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-04-25T08:54:04.728 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:54:04.843 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627778 2024-04-25T08:54:04.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:04.853 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T08:54:04.853 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:54:04.853 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T08:54:04.853 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:04.853 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-04-25T08:54:04.853 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:54:04.963 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627776 2024-04-25T08:54:04.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:04.974 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T08:54:04.974 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:54:04.974 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T08:54:04.974 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:04.974 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-04-25T08:54:04.974 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:54:05.078 INFO:teuthology.orchestra.run.smithi037.stdout:1099511628279 2024-04-25T08:54:05.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:05.089 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T08:54:05.089 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:54:05.089 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T08:54:05.089 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:05.089 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-04-25T08:54:05.089 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:54:05.199 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627777 2024-04-25T08:54:05.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:05.210 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T08:54:05.210 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:54:05.210 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T08:54:05.210 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:05.210 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-04-25T08:54:05.210 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:54:05.317 INFO:teuthology.orchestra.run.smithi037.stdout:1099511628280 2024-04-25T08:54:05.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:05.327 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:54:05.327 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:05.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:05.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.552+0000 7f4e711c6640 1 Processor -- start 2024-04-25T08:54:05.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.552+0000 7f4e711c6640 1 -- start start 2024-04-25T08:54:05.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e711c6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e6c0ce510 0x7f4e6c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e711c6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4e6c1451f0 0x7f4e6c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e711c6640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c0cee30 con 0x7f4e6c144ca0 2024-04-25T08:54:05.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e711c6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c0cc2b0 con 0x7f4e6c1451f0 2024-04-25T08:54:05.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e711c6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c06e290 con 0x7f4e6c0ce510 2024-04-25T08:54:05.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e6a59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4e6c144ca0 0x7f4e6c06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41902/0 (socket says 172.21.15.37:41902) 2024-04-25T08:54:05.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e6a59c640 1 -- 172.21.15.37:0/2303983886 learned_addr learned my addr 172.21.15.37:0/2303983886 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:05.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e6ad9d640 1 --2- 172.21.15.37:0/2303983886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e6c0ce510 0x7f4e6c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e6b59e640 1 --2- 172.21.15.37:0/2303983886 >> v2:172.21.15.203:3300/0 conn(0x7f4e6c1451f0 0x7f4e6c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552241315 0 0) 0x7f4e6c0cee30 con 0x7f4e6c144ca0 2024-04-25T08:54:05.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e54003530 con 0x7f4e6c144ca0 2024-04-25T08:54:05.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 630626101 0 0) 0x7f4e54003530 con 0x7f4e6c144ca0 2024-04-25T08:54:05.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e6c0ce510 msgr2=0x7f4e6c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:05.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 --2- 172.21.15.37:0/2303983886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e6c0ce510 0x7f4e6c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 >> v2:172.21.15.203:3300/0 conn(0x7f4e6c1451f0 msgr2=0x7f4e6c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:05.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 --2- 172.21.15.37:0/2303983886 >> v2:172.21.15.203:3300/0 conn(0x7f4e6c1451f0 0x7f4e6c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0ccc70 con 0x7f4e6c144ca0 2024-04-25T08:54:05.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f4e60001680 con 0x7f4e6c144ca0 2024-04-25T08:54:05.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 >> v1:172.21.15.37:6789/0 conn(0x7f4e6c144ca0 legacy=0x7f4e6c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 --2- 172.21.15.37:0/2303983886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e54003a30 0x7f4e54003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 --2- 172.21.15.37:0/2303983886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e54004810 0x7f4e54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 --2- 172.21.15.37:0/2303983886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e6c0ce510 0x7f4e540051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0cc2b0 con 0x7f4e54003a30 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c06e290 con 0x7f4e54004810 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5400ba20 con 0x7f4e6c0ce510 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e6ad9d640 1 --2- 172.21.15.37:0/2303983886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e54003a30 0x7f4e54003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e6a59c640 1 --2- 172.21.15.37:0/2303983886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e54004810 0x7f4e54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e6ad9d640 1 -- 172.21.15.37:0/2303983886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e6c0ce510 msgr2=0x7f4e540051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e6ad9d640 1 --2- 172.21.15.37:0/2303983886 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e6c0ce510 0x7f4e540051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e6ad9d640 1 -- 172.21.15.37:0/2303983886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e54004810 msgr2=0x7f4e54004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:05.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e6ad9d640 1 --2- 172.21.15.37:0/2303983886 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e54004810 0x7f4e54004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.556+0000 7f4e6ad9d640 1 -- 172.21.15.37:0/2303983886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e54003530 con 0x7f4e54003a30 2024-04-25T08:54:05.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6ad9d640 1 --2- 172.21.15.37:0/2303983886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e54003a30 0x7f4e54003e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009ae0 tx=0x7f4e5c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c77f3549c31346d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0096b0 con 0x7f4e54003a30 2024-04-25T08:54:05.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c0a64c0 con 0x7f4e54003a30 2024-04-25T08:54:05.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e69d9b640 1 -- 172.21.15.37:0/2303983886 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c0a68b0 con 0x7f4e54003a30 2024-04-25T08:54:05.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 -- 172.21.15.37:0/2303983886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e54003a30 msgr2=0x7f4e54003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 --2- 172.21.15.37:0/2303983886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e54003a30 0x7f4e54003e50 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c009ae0 tx=0x7f4e5c01cab0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6b59e640 1 -- 172.21.15.37:0/2303983886 reap_dead start 2024-04-25T08:54:05.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 -- 172.21.15.37:0/2303983886 shutdown_connections 2024-04-25T08:54:05.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 -- 172.21.15.37:0/2303983886 >> 172.21.15.37:0/2303983886 conn(0x7f4e6c0b11a0 msgr2=0x7f4e6c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 -- 172.21.15.37:0/2303983886 shutdown_connections 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 -- 172.21.15.37:0/2303983886 wait complete. 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 Processor -- start 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 -- start start 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e54003a30 0x7f4e6c0b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c0ad410 0x7f4e6c13dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e6c0af2d0 0x7f4e6c0af750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c06ee30 con 0x7f4e6c0ad410 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c06df20 con 0x7f4e6c0af2d0 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e711c6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c0cc900 con 0x7f4e54003a30 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c0ad410 0x7f4e6c13dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c0ad410 0x7f4e6c13dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38140/0 (socket says 172.21.15.37:38140) 2024-04-25T08:54:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6a59c640 1 -- 172.21.15.37:0/270442524 learned_addr learned my addr 172.21.15.37:0/270442524 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6ad9d640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e54003a30 0x7f4e6c0b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6b59e640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e6c0af2d0 0x7f4e6c0af750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6a59c640 1 -- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e54003a30 msgr2=0x7f4e6c0b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6a59c640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e54003a30 0x7f4e6c0b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6a59c640 1 -- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e6c0af2d0 msgr2=0x7f4e6c0af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6a59c640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e6c0af2d0 0x7f4e6c0af750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.560+0000 7f4e6a59c640 1 -- 172.21.15.37:0/270442524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0adb60 con 0x7f4e6c0ad410 2024-04-25T08:54:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.564+0000 7f4e6a59c640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c0ad410 0x7f4e6c13dbd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4e600117c0 tx=0x7f4e60011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.564+0000 7f4e537fe640 1 -- 172.21.15.37:0/270442524 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e60010040 con 0x7f4e6c0ad410 2024-04-25T08:54:05.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.564+0000 7f4e537fe640 1 -- 172.21.15.37:0/270442524 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6001a070 con 0x7f4e6c0ad410 2024-04-25T08:54:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.564+0000 7f4e537fe640 1 -- 172.21.15.37:0/270442524 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6000fbb0 con 0x7f4e6c0ad410 2024-04-25T08:54:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.564+0000 7f4e711c6640 1 -- 172.21.15.37:0/270442524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c0ade40 con 0x7f4e6c0ad410 2024-04-25T08:54:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.564+0000 7f4e711c6640 1 -- 172.21.15.37:0/270442524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c0ae400 con 0x7f4e6c0ad410 2024-04-25T08:54:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.564+0000 7f4e711c6640 1 -- 172.21.15.37:0/270442524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c04efa0 con 0x7f4e6c0ad410 2024-04-25T08:54:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.564+0000 7f4e537fe640 1 -- 172.21.15.37:0/270442524 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4e6001f460 con 0x7f4e6c0ad410 2024-04-25T08:54:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.564+0000 7f4e537fe640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4e58066f90 0x7f4e58069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.564+0000 7f4e537fe640 1 -- 172.21.15.37:0/270442524 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4e600881d0 con 0x7f4e6c0ad410 2024-04-25T08:54:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.564+0000 7f4e6ad9d640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4e58066f90 0x7f4e58069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.568+0000 7f4e6ad9d640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4e58066f90 0x7f4e58069450 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0af000 tx=0x7f4e5c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.568+0000 7f4e537fe640 1 -- 172.21.15.37:0/270442524 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4e600590e0 con 0x7f4e6c0ad410 2024-04-25T08:54:05.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.728+0000 7f4e711c6640 1 -- 172.21.15.37:0/270442524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e6c0b1cc0 con 0x7f4e6c0ad410 2024-04-25T08:54:05.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e537fe640 1 -- 172.21.15.37:0/270442524 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16906 (secure 0 0 0) 0x7f4e600589e0 con 0x7f4e6c0ad410 2024-04-25T08:54:05.725 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:05.725 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:56.863329+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:53:53.619419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:53:54.197204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 -- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4e58066f90 msgr2=0x7f4e58069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4e58066f90 0x7f4e58069450 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c0af000 tx=0x7f4e5c0046d0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 -- 172.21.15.37:0/270442524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c0ad410 msgr2=0x7f4e6c13dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c0ad410 0x7f4e6c13dbd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4e600117c0 tx=0x7f4e60011bc0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 -- 172.21.15.37:0/270442524 shutdown_connections 2024-04-25T08:54:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4e58066f90 0x7f4e58069450 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4e6c0af2d0 0x7f4e6c0af750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4e6c0ad410 0x7f4e6c13dbd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 --2- 172.21.15.37:0/270442524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4e54003a30 0x7f4e6c0b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 -- 172.21.15.37:0/270442524 >> 172.21.15.37:0/270442524 conn(0x7f4e6c0b11a0 msgr2=0x7f4e6c0c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 -- 172.21.15.37:0/270442524 shutdown_connections 2024-04-25T08:54:05.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.732+0000 7f4e711c6640 1 -- 172.21.15.37:0/270442524 wait complete. 2024-04-25T08:54:05.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:53:53.619419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:53:54.197204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:05.745 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:05.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.872+0000 7f0cb467d640 1 Processor -- start 2024-04-25T08:54:05.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.872+0000 7f0cb467d640 1 -- start start 2024-04-25T08:54:05.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb467d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0cac15b6b0 0x7f0cac0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb467d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac0cf590 0x7f0cac0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb467d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cac0cbd20 con 0x7f0cac15f130 2024-04-25T08:54:05.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb467d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cac15ba90 con 0x7f0cac15b6b0 2024-04-25T08:54:05.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb467d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac0cff30 con 0x7f0cac0cf590 2024-04-25T08:54:05.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb1c18640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac0cf590 0x7f0cac0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb2419640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0cac15b6b0 0x7f0cac0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb2c1a640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0cac15f130 0x7f0cac15f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:49854/0 (socket says 172.21.15.37:49854) 2024-04-25T08:54:05.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb1c18640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac0cf590 0x7f0cac0cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:57632/0 (socket says 172.21.15.37:57632) 2024-04-25T08:54:05.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb2c1a640 1 -- 172.21.15.37:0/75985782 learned_addr learned my addr 172.21.15.37:0/75985782 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:05.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb2419640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0cac15b6b0 0x7f0cac0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:36348/0 (socket says 172.21.15.37:36348) 2024-04-25T08:54:05.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb1c18640 1 -- 172.21.15.37:0/75985782 >> v2:172.21.15.203:3300/0 conn(0x7f0cac15b6b0 msgr2=0x7f0cac0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:05.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb1c18640 1 --2- 172.21.15.37:0/75985782 >> v2:172.21.15.203:3300/0 conn(0x7f0cac15b6b0 0x7f0cac0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb1c18640 1 -- 172.21.15.37:0/75985782 >> v1:172.21.15.203:6789/0 conn(0x7f0cac15f130 legacy=0x7f0cac15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.876+0000 7f0cb1c18640 1 -- 172.21.15.37:0/75985782 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac0cc000 con 0x7f0cac0cf590 2024-04-25T08:54:05.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb1c18640 1 --2- 172.21.15.37:0/75985782 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac0cf590 0x7f0cac0cf9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00097c0 tx=0x7f0ca001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4314526b4a0ddf55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb1417640 1 -- 172.21.15.37:0/75985782 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00a7070 con 0x7f0cac0cf590 2024-04-25T08:54:05.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb1417640 1 -- 172.21.15.37:0/75985782 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca00b0070 con 0x7f0cac0cf590 2024-04-25T08:54:05.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb1417640 1 -- 172.21.15.37:0/75985782 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00ab890 con 0x7f0cac0cf590 2024-04-25T08:54:05.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 -- 172.21.15.37:0/75985782 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac0cf590 msgr2=0x7f0cac0cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 --2- 172.21.15.37:0/75985782 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac0cf590 0x7f0cac0cf9f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00097c0 tx=0x7f0ca001c470 comp rx=0 tx=0).stop 2024-04-25T08:54:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 -- 172.21.15.37:0/75985782 shutdown_connections 2024-04-25T08:54:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 --2- 172.21.15.37:0/75985782 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac0cf590 0x7f0cac0cf9f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 --2- 172.21.15.37:0/75985782 >> v2:172.21.15.203:3300/0 conn(0x7f0cac15b6b0 0x7f0cac0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 -- 172.21.15.37:0/75985782 >> 172.21.15.37:0/75985782 conn(0x7f0cac06ae60 msgr2=0x7f0cac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 -- 172.21.15.37:0/75985782 shutdown_connections 2024-04-25T08:54:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 -- 172.21.15.37:0/75985782 wait complete. 2024-04-25T08:54:05.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 Processor -- start 2024-04-25T08:54:05.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 -- start start 2024-04-25T08:54:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cac0cf590 0x7f0cac097e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac15b6b0 0x7f0cac1525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac15f130 0x7f0cac0ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac0ca900 con 0x7f0cac15b6b0 2024-04-25T08:54:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac0cba30 con 0x7f0cac0cf590 2024-04-25T08:54:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb467d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cac0cadb0 con 0x7f0cac15f130 2024-04-25T08:54:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb1c18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac15b6b0 0x7f0cac1525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb1c18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac15b6b0 0x7f0cac1525a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38142/0 (socket says 172.21.15.37:38142) 2024-04-25T08:54:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb1c18640 1 -- 172.21.15.37:0/3199543741 learned_addr learned my addr 172.21.15.37:0/3199543741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:05.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb2419640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cac0cf590 0x7f0cac097e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.880+0000 7f0cb2c1a640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac15f130 0x7f0cac0ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0cb1c18640 1 -- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac15f130 msgr2=0x7f0cac0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0cb1c18640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac15f130 0x7f0cac0ad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0cb1c18640 1 -- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cac0cf590 msgr2=0x7f0cac097e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0cb1c18640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cac0cf590 0x7f0cac097e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0cb1c18640 1 -- 172.21.15.37:0/3199543741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cac152ae0 con 0x7f0cac15b6b0 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0cb1c18640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac15b6b0 0x7f0cac1525a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00ad040 tx=0x7f0ca001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0caaffd640 1 -- 172.21.15.37:0/3199543741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00bf610 con 0x7f0cac15b6b0 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0caaffd640 1 -- 172.21.15.37:0/3199543741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ca00bf7f0 con 0x7f0cac15b6b0 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0cb467d640 1 -- 172.21.15.37:0/3199543741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cac155dd0 con 0x7f0cac15b6b0 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0caaffd640 1 -- 172.21.15.37:0/3199543741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca00bfbe0 con 0x7f0cac15b6b0 2024-04-25T08:54:05.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0cb467d640 1 -- 172.21.15.37:0/3199543741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cac156070 con 0x7f0cac15b6b0 2024-04-25T08:54:05.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0cb467d640 1 -- 172.21.15.37:0/3199543741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cac04efa0 con 0x7f0cac15b6b0 2024-04-25T08:54:05.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0caaffd640 1 -- 172.21.15.37:0/3199543741 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ca00bd020 con 0x7f0cac15b6b0 2024-04-25T08:54:05.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0caaffd640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c98066f90 0x7f0c98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0caaffd640 1 -- 172.21.15.37:0/3199543741 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ca011f850 con 0x7f0cac15b6b0 2024-04-25T08:54:05.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0cb2419640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c98066f90 0x7f0c98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.884+0000 7f0cb2419640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c98066f90 0x7f0c98069450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0cac1f7e00 tx=0x7f0ca400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:05.888+0000 7f0caaffd640 1 -- 172.21.15.37:0/3199543741 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ca0016090 con 0x7f0cac15b6b0 2024-04-25T08:54:06.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.048+0000 7f0cb467d640 1 -- 172.21.15.37:0/3199543741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0cac0cb300 con 0x7f0cac15b6b0 2024-04-25T08:54:06.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.048+0000 7f0caaffd640 1 -- 172.21.15.37:0/3199543741 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0ca00f01a0 con 0x7f0cac15b6b0 2024-04-25T08:54:06.043 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:54:06.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 -- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c98066f90 msgr2=0x7f0c98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c98066f90 0x7f0c98069450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0cac1f7e00 tx=0x7f0ca400b040 comp rx=0 tx=0).stop 2024-04-25T08:54:06.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 -- 172.21.15.37:0/3199543741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac15b6b0 msgr2=0x7f0cac1525a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac15b6b0 0x7f0cac1525a0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00ad040 tx=0x7f0ca001acb0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 -- 172.21.15.37:0/3199543741 shutdown_connections 2024-04-25T08:54:06.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c98066f90 0x7f0c98069450 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0cac15f130 0x7f0cac0ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac15b6b0 0x7f0cac1525a0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 --2- 172.21.15.37:0/3199543741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0cac0cf590 0x7f0cac097e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 -- 172.21.15.37:0/3199543741 >> 172.21.15.37:0/3199543741 conn(0x7f0cac06ae60 msgr2=0x7f0cac0b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:06.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 -- 172.21.15.37:0/3199543741 shutdown_connections 2024-04-25T08:54:06.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.052+0000 7f0cb467d640 1 -- 172.21.15.37:0/3199543741 wait complete. 2024-04-25T08:54:06.062 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:06.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf3cb7b640 1 Processor -- start 2024-04-25T08:54:06.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf3cb7b640 1 -- start start 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf3cb7b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbf3815b6b0 0x7fbf380ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf3cb7b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbf38073010 0x7fbf380cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf3cb7b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf380cbd20 con 0x7fbf380cf590 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf3cb7b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3815ba90 con 0x7fbf38073010 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf3cb7b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf380cc000 con 0x7fbf3815b6b0 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf36d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbf38073010 0x7fbf380cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf36d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbf38073010 0x7fbf380cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38146/0 (socket says 172.21.15.37:38146) 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf36d9d640 1 -- 172.21.15.37:0/3037843910 learned_addr learned my addr 172.21.15.37:0/3037843910 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf3659c640 1 --2- 172.21.15.37:0/3037843910 >> v2:172.21.15.203:3300/0 conn(0x7fbf3815b6b0 0x7fbf380ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf36d9d640 1 -- 172.21.15.37:0/3037843910 >> v2:172.21.15.203:3300/0 conn(0x7fbf3815b6b0 msgr2=0x7fbf380ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf36d9d640 1 --2- 172.21.15.37:0/3037843910 >> v2:172.21.15.203:3300/0 conn(0x7fbf3815b6b0 0x7fbf380ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf36d9d640 1 -- 172.21.15.37:0/3037843910 >> v1:172.21.15.203:6789/0 conn(0x7fbf380cf590 legacy=0x7fbf3815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf36d9d640 1 -- 172.21.15.37:0/3037843910 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf380cc2e0 con 0x7fbf38073010 2024-04-25T08:54:06.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.200+0000 7fbf3659c640 1 --2- 172.21.15.37:0/3037843910 >> v2:172.21.15.203:3300/0 conn(0x7fbf3815b6b0 0x7fbf380ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:06.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf36d9d640 1 --2- 172.21.15.37:0/3037843910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf38073010 0x7fbf380cfa20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbf380d0180 tx=0x7fbf3001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c8841cbd916ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:06.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3559a640 1 -- 172.21.15.37:0/3037843910 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300a56c0 con 0x7fbf38073010 2024-04-25T08:54:06.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3559a640 1 -- 172.21.15.37:0/3037843910 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf300a5860 con 0x7fbf38073010 2024-04-25T08:54:06.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3559a640 1 -- 172.21.15.37:0/3037843910 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300a5c30 con 0x7fbf38073010 2024-04-25T08:54:06.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3037843910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf38073010 msgr2=0x7fbf380cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 --2- 172.21.15.37:0/3037843910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf38073010 0x7fbf380cfa20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fbf380d0180 tx=0x7fbf3001c9f0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3037843910 shutdown_connections 2024-04-25T08:54:06.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 --2- 172.21.15.37:0/3037843910 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf38073010 0x7fbf380cfa20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 --2- 172.21.15.37:0/3037843910 >> v2:172.21.15.203:3300/0 conn(0x7fbf3815b6b0 0x7fbf380ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3037843910 >> 172.21.15.37:0/3037843910 conn(0x7fbf3806ae60 msgr2=0x7fbf3806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:06.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3037843910 shutdown_connections 2024-04-25T08:54:06.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3037843910 wait complete. 2024-04-25T08:54:06.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 Processor -- start 2024-04-25T08:54:06.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 -- start start 2024-04-25T08:54:06.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf38073010 0x7fbf38098880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.204+0000 7fbf3cb7b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf380cf590 0x7fbf381542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf3cb7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf3815b6b0 0x7fbf38153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf3cb7b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380cadb0 con 0x7fbf3815b6b0 2024-04-25T08:54:06.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf3cb7b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380ca900 con 0x7fbf380cf590 2024-04-25T08:54:06.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf3cb7b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf380caf30 con 0x7fbf38073010 2024-04-25T08:54:06.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf36d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf3815b6b0 0x7fbf38153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf36d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf3815b6b0 0x7fbf38153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38158/0 (socket says 172.21.15.37:38158) 2024-04-25T08:54:06.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf36d9d640 1 -- 172.21.15.37:0/3994615067 learned_addr learned my addr 172.21.15.37:0/3994615067 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:06.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf35d9b640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf380cf590 0x7fbf381542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf3659c640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf38073010 0x7fbf38098880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf36d9d640 1 -- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf38073010 msgr2=0x7fbf38098880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf36d9d640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf38073010 0x7fbf38098880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf36d9d640 1 -- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf380cf590 msgr2=0x7fbf381542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf36d9d640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf380cf590 0x7fbf381542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf36d9d640 1 -- 172.21.15.37:0/3994615067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf380ca570 con 0x7fbf3815b6b0 2024-04-25T08:54:06.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf35d9b640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf380cf590 0x7fbf381542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:06.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf36d9d640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf3815b6b0 0x7fbf38153750 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbf30019600 tx=0x7fbf30019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:06.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf26ffd640 1 -- 172.21.15.37:0/3994615067 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300bc020 con 0x7fbf3815b6b0 2024-04-25T08:54:06.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf26ffd640 1 -- 172.21.15.37:0/3994615067 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf300b8670 con 0x7fbf3815b6b0 2024-04-25T08:54:06.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3994615067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf380ac670 con 0x7fbf3815b6b0 2024-04-25T08:54:06.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf26ffd640 1 -- 172.21.15.37:0/3994615067 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf30009510 con 0x7fbf3815b6b0 2024-04-25T08:54:06.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3994615067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf380acc30 con 0x7fbf3815b6b0 2024-04-25T08:54:06.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3994615067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3804ef80 con 0x7fbf3815b6b0 2024-04-25T08:54:06.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.208+0000 7fbf26ffd640 1 -- 172.21.15.37:0/3994615067 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbf300bf020 con 0x7fbf3815b6b0 2024-04-25T08:54:06.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.212+0000 7fbf26ffd640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbf18066fb0 0x7fbf18069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.212+0000 7fbf26ffd640 1 -- 172.21.15.37:0/3994615067 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbf301200f0 con 0x7fbf3815b6b0 2024-04-25T08:54:06.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.212+0000 7fbf3659c640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbf18066fb0 0x7fbf18069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.212+0000 7fbf3659c640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbf18066fb0 0x7fbf18069470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800b930 tx=0x7fbf28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:06.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.216+0000 7fbf26ffd640 1 -- 172.21.15.37:0/3994615067 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbf300c6030 con 0x7fbf3815b6b0 2024-04-25T08:54:06.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.376+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3994615067 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf380b3550 con 0x7fbf3815b6b0 2024-04-25T08:54:06.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.376+0000 7fbf26ffd640 1 -- 172.21.15.37:0/3994615067 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf300165e0 con 0x7fbf3815b6b0 2024-04-25T08:54:06.369 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:54:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbf18066fb0 msgr2=0x7fbf18069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbf18066fb0 0x7fbf18069470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800b930 tx=0x7fbf28008040 comp rx=0 tx=0).stop 2024-04-25T08:54:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3994615067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf3815b6b0 msgr2=0x7fbf38153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf3815b6b0 0x7fbf38153750 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fbf30019600 tx=0x7fbf30019790 comp rx=0 tx=0).stop 2024-04-25T08:54:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3994615067 shutdown_connections 2024-04-25T08:54:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbf18066fb0 0x7fbf18069470 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf3815b6b0 0x7fbf38153750 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbf380cf590 0x7fbf381542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 --2- 172.21.15.37:0/3994615067 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbf38073010 0x7fbf38098880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3994615067 >> 172.21.15.37:0/3994615067 conn(0x7fbf3806ae60 msgr2=0x7fbf380b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:06.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3994615067 shutdown_connections 2024-04-25T08:54:06.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.380+0000 7fbf3cb7b640 1 -- 172.21.15.37:0/3994615067 wait complete. 2024-04-25T08:54:06.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc487316640 1 Processor -- start 2024-04-25T08:54:06.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc487316640 1 -- start start 2024-04-25T08:54:06.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc487316640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc48009a180 0x7fc4800b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc487316640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4800cbec0 con 0x7fc48009a7b0 2024-04-25T08:54:06.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc487316640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4800cc1a0 con 0x7fc4800b2c30 2024-04-25T08:54:06.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc487316640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4800cc480 con 0x7fc48009a180 2024-04-25T08:54:06.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc4848b1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc48009a180 0x7fc4800b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc4858b3640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc48009a7b0 0x7fc4800ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41904/0 (socket says 172.21.15.37:41904) 2024-04-25T08:54:06.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc4858b3640 1 -- 172.21.15.37:0/807576484 learned_addr learned my addr 172.21.15.37:0/807576484 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:06.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc4848b1640 1 -- 172.21.15.37:0/807576484 >> v1:172.21.15.203:6789/0 conn(0x7fc4800b2c30 legacy=0x7fc480099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc4848b1640 1 -- 172.21.15.37:0/807576484 >> v1:172.21.15.37:6789/0 conn(0x7fc48009a7b0 legacy=0x7fc4800ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.520+0000 7fc4848b1640 1 -- 172.21.15.37:0/807576484 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800cc760 con 0x7fc48009a180 2024-04-25T08:54:06.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc4848b1640 1 --2- 172.21.15.37:0/807576484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc48009a180 0x7fc4800b0ba0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc4700097c0 tx=0x7fc47001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f004ff8470367223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:06.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc477fff640 1 -- 172.21.15.37:0/807576484 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4700ab610 con 0x7fc48009a180 2024-04-25T08:54:06.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc477fff640 1 -- 172.21.15.37:0/807576484 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4700b0070 con 0x7fc48009a180 2024-04-25T08:54:06.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc477fff640 1 -- 172.21.15.37:0/807576484 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4700ab9d0 con 0x7fc48009a180 2024-04-25T08:54:06.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc487316640 1 -- 172.21.15.37:0/807576484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc48009a180 msgr2=0x7fc4800b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc487316640 1 --2- 172.21.15.37:0/807576484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc48009a180 0x7fc4800b0ba0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc4700097c0 tx=0x7fc47001c440 comp rx=0 tx=0).stop 2024-04-25T08:54:06.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc487316640 1 -- 172.21.15.37:0/807576484 shutdown_connections 2024-04-25T08:54:06.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc487316640 1 --2- 172.21.15.37:0/807576484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc48009a180 0x7fc4800b0ba0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc487316640 1 -- 172.21.15.37:0/807576484 >> 172.21.15.37:0/807576484 conn(0x7fc4800b3010 msgr2=0x7fc4800b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:06.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc487316640 1 -- 172.21.15.37:0/807576484 shutdown_connections 2024-04-25T08:54:06.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc487316640 1 -- 172.21.15.37:0/807576484 wait complete. 2024-04-25T08:54:06.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc487316640 1 Processor -- start 2024-04-25T08:54:06.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.524+0000 7fc487316640 1 -- start start 2024-04-25T08:54:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc487316640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc48009a180 0x7fc48014f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc487316640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc48009a7b0 0x7fc4800ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc487316640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4800b2c30 0x7fc4800ce900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc487316640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48006c380 con 0x7fc48009a180 2024-04-25T08:54:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc487316640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48006c200 con 0x7fc48009a7b0 2024-04-25T08:54:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc487316640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48015d190 con 0x7fc4800b2c30 2024-04-25T08:54:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc4850b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc48009a180 0x7fc48014f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc4850b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc48009a180 0x7fc48014f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38190/0 (socket says 172.21.15.37:38190) 2024-04-25T08:54:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc4850b2640 1 -- 172.21.15.37:0/3509732973 learned_addr learned my addr 172.21.15.37:0/3509732973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc4848b1640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc48009a7b0 0x7fc4800ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc4850b2640 1 -- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4800b2c30 msgr2=0x7fc4800ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc4850b2640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4800b2c30 0x7fc4800ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc4850b2640 1 -- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc48009a7b0 msgr2=0x7fc4800ce3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc4850b2640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc48009a7b0 0x7fc4800ce3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc4850b2640 1 -- 172.21.15.37:0/3509732973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800c5140 con 0x7fc48009a180 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc4848b1640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc48009a7b0 0x7fc4800ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc4850b2640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc48009a180 0x7fc48014f420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00c3a0 tx=0x7fc47c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc475ffb640 1 -- 172.21.15.37:0/3509732973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0052d0 con 0x7fc48009a180 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc475ffb640 1 -- 172.21.15.37:0/3509732973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c005470 con 0x7fc48009a180 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc475ffb640 1 -- 172.21.15.37:0/3509732973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c007260 con 0x7fc48009a180 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc487316640 1 -- 172.21.15.37:0/3509732973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48014f960 con 0x7fc48009a180 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc487316640 1 -- 172.21.15.37:0/3509732973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4800c56c0 con 0x7fc48009a180 2024-04-25T08:54:06.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc487316640 1 -- 172.21.15.37:0/3509732973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48004efa0 con 0x7fc48009a180 2024-04-25T08:54:06.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc475ffb640 1 -- 172.21.15.37:0/3509732973 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc47c007400 con 0x7fc48009a180 2024-04-25T08:54:06.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc475ffb640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc464066f60 0x7fc464069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.528+0000 7fc475ffb640 1 -- 172.21.15.37:0/3509732973 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc47c084660 con 0x7fc48009a180 2024-04-25T08:54:06.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.532+0000 7fc4848b1640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc464066f60 0x7fc464069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.532+0000 7fc4848b1640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc464066f60 0x7fc464069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4700af000 tx=0x7fc470008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:06.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.536+0000 7fc475ffb640 1 -- 172.21.15.37:0/3509732973 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc47c0554d0 con 0x7fc48009a180 2024-04-25T08:54:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.696+0000 7fc487316640 1 -- 172.21.15.37:0/3509732973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc48006ba80 con 0x7fc48009a180 2024-04-25T08:54:06.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc475ffb640 1 -- 172.21.15.37:0/3509732973 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc47c054dd0 con 0x7fc48009a180 2024-04-25T08:54:06.692 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:54:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 -- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc464066f60 msgr2=0x7fc464069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc464066f60 0x7fc464069420 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc4700af000 tx=0x7fc470008040 comp rx=0 tx=0).stop 2024-04-25T08:54:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 -- 172.21.15.37:0/3509732973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc48009a180 msgr2=0x7fc48014f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc48009a180 0x7fc48014f420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00c3a0 tx=0x7fc47c00c860 comp rx=0 tx=0).stop 2024-04-25T08:54:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 -- 172.21.15.37:0/3509732973 shutdown_connections 2024-04-25T08:54:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc464066f60 0x7fc464069420 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4800b2c30 0x7fc4800ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc48009a7b0 0x7fc4800ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 --2- 172.21.15.37:0/3509732973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc48009a180 0x7fc48014f420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 -- 172.21.15.37:0/3509732973 >> 172.21.15.37:0/3509732973 conn(0x7fc4800b3010 msgr2=0x7fc4800b0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 -- 172.21.15.37:0/3509732973 shutdown_connections 2024-04-25T08:54:06.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.700+0000 7fc487316640 1 -- 172.21.15.37:0/3509732973 wait complete. 2024-04-25T08:54:06.711 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:54:06.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a9c1c640 1 Processor -- start 2024-04-25T08:54:06.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a9c1c640 1 -- start start 2024-04-25T08:54:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a9c1c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a40ce510 0x7f74a40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a9c1c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f74a41451f0 0x7f74a4142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a9c1c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a40cee30 con 0x7f74a4144ca0 2024-04-25T08:54:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a9c1c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74a40cc2b0 con 0x7f74a41451f0 2024-04-25T08:54:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a9c1c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a406e290 con 0x7f74a40ce510 2024-04-25T08:54:06.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f74a41451f0 0x7f74a4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f74a41451f0 0x7f74a4142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38192/0 (socket says 172.21.15.37:38192) 2024-04-25T08:54:06.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a3fff640 1 -- 172.21.15.37:0/3269481966 learned_addr learned my addr 172.21.15.37:0/3269481966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:06.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a37fe640 1 --2- 172.21.15.37:0/3269481966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a40ce510 0x7f74a40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a3fff640 1 -- 172.21.15.37:0/3269481966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a40ce510 msgr2=0x7f74a40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:06.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a3fff640 1 --2- 172.21.15.37:0/3269481966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a40ce510 0x7f74a40ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a3fff640 1 -- 172.21.15.37:0/3269481966 >> v1:172.21.15.203:6789/0 conn(0x7f74a4144ca0 legacy=0x7f74a406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.840+0000 7f74a3fff640 1 -- 172.21.15.37:0/3269481966 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a40ccc70 con 0x7f74a41451f0 2024-04-25T08:54:06.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a3fff640 1 --2- 172.21.15.37:0/3269481966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a41451f0 0x7f74a4142680 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f749c009e50 tx=0x7f749c01f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a420a74ba70eb3de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:06.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a27fc640 1 -- 172.21.15.37:0/3269481966 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c0b4070 con 0x7f74a41451f0 2024-04-25T08:54:06.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a27fc640 1 -- 172.21.15.37:0/3269481966 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c0af400 con 0x7f74a41451f0 2024-04-25T08:54:06.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a27fc640 1 -- 172.21.15.37:0/3269481966 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c0af7d0 con 0x7f74a41451f0 2024-04-25T08:54:06.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a9c1c640 1 -- 172.21.15.37:0/3269481966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a41451f0 msgr2=0x7f74a4142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a9c1c640 1 --2- 172.21.15.37:0/3269481966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a41451f0 0x7f74a4142680 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f749c009e50 tx=0x7f749c01f210 comp rx=0 tx=0).stop 2024-04-25T08:54:06.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a9c1c640 1 -- 172.21.15.37:0/3269481966 shutdown_connections 2024-04-25T08:54:06.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a9c1c640 1 --2- 172.21.15.37:0/3269481966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a41451f0 0x7f74a4142680 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a9c1c640 1 --2- 172.21.15.37:0/3269481966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a40ce510 0x7f74a40ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a9c1c640 1 -- 172.21.15.37:0/3269481966 >> 172.21.15.37:0/3269481966 conn(0x7f74a40b11a0 msgr2=0x7f74a40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:06.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a9c1c640 1 -- 172.21.15.37:0/3269481966 shutdown_connections 2024-04-25T08:54:06.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.844+0000 7f74a9c1c640 1 -- 172.21.15.37:0/3269481966 wait complete. 2024-04-25T08:54:06.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a9c1c640 1 Processor -- start 2024-04-25T08:54:06.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a9c1c640 1 -- start start 2024-04-25T08:54:06.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a9c1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a40ce510 0x7f74a415b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a9c1c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a4144ca0 0x7f74a4141fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a9c1c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a41451f0 0x7f74a41424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a9c1c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a406ee30 con 0x7f74a40ce510 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a9c1c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a406df20 con 0x7f74a41451f0 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a9c1c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a40cc900 con 0x7f74a4144ca0 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a40ce510 0x7f74a415b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a40ce510 0x7f74a415b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38208/0 (socket says 172.21.15.37:38208) 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a37fe640 1 -- 172.21.15.37:0/2187474366 learned_addr learned my addr 172.21.15.37:0/2187474366 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a3fff640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a41451f0 0x7f74a41424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a37fe640 1 -- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a4144ca0 msgr2=0x7f74a4141fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a2ffd640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a4144ca0 0x7f74a4141fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a37fe640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a4144ca0 0x7f74a4141fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a37fe640 1 -- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a41451f0 msgr2=0x7f74a41424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a37fe640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a41451f0 0x7f74a41424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a37fe640 1 -- 172.21.15.37:0/2187474366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a41e1c90 con 0x7f74a40ce510 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a2ffd640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a4144ca0 0x7f74a4141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:06.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a37fe640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a40ce510 0x7f74a415b630 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f749400e960 tx=0x7f749400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:06.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f748bfff640 1 -- 172.21.15.37:0/2187474366 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7494017070 con 0x7f74a40ce510 2024-04-25T08:54:06.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f748bfff640 1 -- 172.21.15.37:0/2187474366 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749400ca00 con 0x7f74a40ce510 2024-04-25T08:54:06.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a9c1c640 1 -- 172.21.15.37:0/2187474366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a41e1f70 con 0x7f74a40ce510 2024-04-25T08:54:06.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f748bfff640 1 -- 172.21.15.37:0/2187474366 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749400cdb0 con 0x7f74a40ce510 2024-04-25T08:54:06.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a9c1c640 1 -- 172.21.15.37:0/2187474366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a41e2530 con 0x7f74a40ce510 2024-04-25T08:54:06.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f74a9c1c640 1 -- 172.21.15.37:0/2187474366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a404efa0 con 0x7f74a40ce510 2024-04-25T08:54:06.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.848+0000 7f748bfff640 1 -- 172.21.15.37:0/2187474366 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f749401a070 con 0x7f74a40ce510 2024-04-25T08:54:06.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.852+0000 7f748bfff640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7490066f90 0x7f7490069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:06.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.852+0000 7f748bfff640 1 -- 172.21.15.37:0/2187474366 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7494085600 con 0x7f74a40ce510 2024-04-25T08:54:06.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.852+0000 7f74a2ffd640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7490066f90 0x7f7490069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:06.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.852+0000 7f74a2ffd640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7490066f90 0x7f7490069450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f74a40cd270 tx=0x7f7498009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:06.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:06.856+0000 7f748bfff640 1 -- 172.21.15.37:0/2187474366 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7494056400 con 0x7f74a40ce510 2024-04-25T08:54:07.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 -- 172.21.15.37:0/2187474366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f74a40b0e40 con 0x7f74a40ce510 2024-04-25T08:54:07.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f748bfff640 1 -- 172.21.15.37:0/2187474366 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7494055d00 con 0x7f74a40ce510 2024-04-25T08:54:07.017 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:54:07.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 -- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7490066f90 msgr2=0x7f7490069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7490066f90 0x7f7490069450 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f74a40cd270 tx=0x7f7498009040 comp rx=0 tx=0).stop 2024-04-25T08:54:07.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 -- 172.21.15.37:0/2187474366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a40ce510 msgr2=0x7f74a415b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a40ce510 0x7f74a415b630 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f749400e960 tx=0x7f749400ee20 comp rx=0 tx=0).stop 2024-04-25T08:54:07.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 -- 172.21.15.37:0/2187474366 shutdown_connections 2024-04-25T08:54:07.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7490066f90 0x7f7490069450 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74a41451f0 0x7f74a41424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74a4144ca0 0x7f74a4141fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 --2- 172.21.15.37:0/2187474366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74a40ce510 0x7f74a415b630 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 -- 172.21.15.37:0/2187474366 >> 172.21.15.37:0/2187474366 conn(0x7f74a40b11a0 msgr2=0x7f74a4071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:07.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 -- 172.21.15.37:0/2187474366 shutdown_connections 2024-04-25T08:54:07.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.024+0000 7f74a9c1c640 1 -- 172.21.15.37:0/2187474366 wait complete. 2024-04-25T08:54:07.034 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:54:07.034 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:07.040 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc52216f640 1 Processor -- start 2024-04-25T08:54:07.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc52216f640 1 -- start start 2024-04-25T08:54:07.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc52216f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc51c06f610 0x7fc51c0cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc52216f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc51c06ac90 0x7fc51c06b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc52216f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc51c0701b0 con 0x7fc51c0afb60 2024-04-25T08:54:07.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc52216f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc51c06b640 con 0x7fc51c06ac90 2024-04-25T08:54:07.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc52216f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c06fbf0 con 0x7fc51c06f610 2024-04-25T08:54:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc51b7fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fc51c0afb60 0x7fc51c0cc730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:49890/0 (socket says 172.21.15.37:49890) 2024-04-25T08:54:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc51b7fe640 1 -- 172.21.15.37:0/2629863854 learned_addr learned my addr 172.21.15.37:0/2629863854 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc51bfff640 1 --2- 172.21.15.37:0/2629863854 >> v2:172.21.15.203:3300/0 conn(0x7fc51c06ac90 0x7fc51c06b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc51affd640 1 --2- 172.21.15.37:0/2629863854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc51c06f610 0x7fc51c0cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc51bfff640 1 -- 172.21.15.37:0/2629863854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc51c06f610 msgr2=0x7fc51c0cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:07.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc51bfff640 1 --2- 172.21.15.37:0/2629863854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc51c06f610 0x7fc51c0cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc51bfff640 1 -- 172.21.15.37:0/2629863854 >> v1:172.21.15.203:6789/0 conn(0x7fc51c0afb60 legacy=0x7fc51c0cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.212+0000 7fc51bfff640 1 -- 172.21.15.37:0/2629863854 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c06fed0 con 0x7fc51c06ac90 2024-04-25T08:54:07.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.216+0000 7fc51bfff640 1 --2- 172.21.15.37:0/2629863854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc51c06ac90 0x7fc51c06b100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0097c0 tx=0x7fc50c01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3c2d991ebac03e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc51a7fc640 1 -- 172.21.15.37:0/2629863854 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0b0070 con 0x7fc51c06ac90 2024-04-25T08:54:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc51a7fc640 1 -- 172.21.15.37:0/2629863854 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c0ab600 con 0x7fc51c06ac90 2024-04-25T08:54:07.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc51a7fc640 1 -- 172.21.15.37:0/2629863854 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0ab9d0 con 0x7fc51c06ac90 2024-04-25T08:54:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc52216f640 1 -- 172.21.15.37:0/2629863854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc51c06ac90 msgr2=0x7fc51c06b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc52216f640 1 --2- 172.21.15.37:0/2629863854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc51c06ac90 0x7fc51c06b100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0097c0 tx=0x7fc50c01c3e0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc52216f640 1 -- 172.21.15.37:0/2629863854 shutdown_connections 2024-04-25T08:54:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc52216f640 1 --2- 172.21.15.37:0/2629863854 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc51c06ac90 0x7fc51c06b100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc52216f640 1 --2- 172.21.15.37:0/2629863854 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc51c06f610 0x7fc51c0cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc52216f640 1 -- 172.21.15.37:0/2629863854 >> 172.21.15.37:0/2629863854 conn(0x7fc51c099a70 msgr2=0x7fc51c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc52216f640 1 -- 172.21.15.37:0/2629863854 shutdown_connections 2024-04-25T08:54:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc52216f640 1 -- 172.21.15.37:0/2629863854 wait complete. 2024-04-25T08:54:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc52216f640 1 Processor -- start 2024-04-25T08:54:07.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.220+0000 7fc52216f640 1 -- start start 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc52216f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc51c06ac90 0x7fc51c14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc52216f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc51c06f610 0x7fc51c157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc52216f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51c0afb60 0x7fc51c1540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc52216f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c15e250 con 0x7fc51c0afb60 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc52216f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c15e0d0 con 0x7fc51c06f610 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc52216f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c15e3d0 con 0x7fc51c06ac90 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51c0afb60 0x7fc51c1540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51c0afb60 0x7fc51c1540c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38222/0 (socket says 172.21.15.37:38222) 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51bfff640 1 -- 172.21.15.37:0/851825146 learned_addr learned my addr 172.21.15.37:0/851825146 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51affd640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc51c06f610 0x7fc51c157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51b7fe640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc51c06ac90 0x7fc51c14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51bfff640 1 -- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc51c06ac90 msgr2=0x7fc51c14ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51bfff640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc51c06ac90 0x7fc51c14ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51bfff640 1 -- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc51c06f610 msgr2=0x7fc51c157260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51bfff640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc51c06f610 0x7fc51c157260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51bfff640 1 -- 172.21.15.37:0/851825146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c1564e0 con 0x7fc51c0afb60 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51b7fe640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc51c06ac90 0x7fc51c14ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51bfff640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51c0afb60 0x7fc51c1540c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0ae040 tx=0x7fc50c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc513fff640 1 -- 172.21.15.37:0/851825146 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0b8020 con 0x7fc51c0afb60 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc513fff640 1 -- 172.21.15.37:0/851825146 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50c0b4410 con 0x7fc51c0afb60 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc52216f640 1 -- 172.21.15.37:0/851825146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c156780 con 0x7fc51c0afb60 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc513fff640 1 -- 172.21.15.37:0/851825146 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0b0040 con 0x7fc51c0afb60 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc52216f640 1 -- 172.21.15.37:0/851825146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c1577a0 con 0x7fc51c0afb60 2024-04-25T08:54:07.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc52216f640 1 -- 172.21.15.37:0/851825146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51c04efa0 con 0x7fc51c0afb60 2024-04-25T08:54:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc513fff640 1 -- 172.21.15.37:0/851825146 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc50c0bb020 con 0x7fc51c0afb60 2024-04-25T08:54:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc513fff640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4fc066f60 0x7fc4fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc513fff640 1 -- 172.21.15.37:0/851825146 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc50c11ecb0 con 0x7fc51c0afb60 2024-04-25T08:54:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.224+0000 7fc51b7fe640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4fc066f60 0x7fc4fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.228+0000 7fc51b7fe640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4fc066f60 0x7fc4fc069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc51c154070 tx=0x7fc514009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:07.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.228+0000 7fc513fff640 1 -- 172.21.15.37:0/851825146 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc50c0c2030 con 0x7fc51c0afb60 2024-04-25T08:54:07.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.388+0000 7fc52216f640 1 -- 172.21.15.37:0/851825146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc51c0cdb60 con 0x7fc51c0afb60 2024-04-25T08:54:07.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.392+0000 7fc513fff640 1 -- 172.21.15.37:0/851825146 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+16906 (secure 0 0 0) 0x7fc50c016090 con 0x7fc51c0afb60 2024-04-25T08:54:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:07.386 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:53:56.863329+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:53:53.619419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:53:54.197204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 -- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4fc066f60 msgr2=0x7fc4fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4fc066f60 0x7fc4fc069420 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc51c154070 tx=0x7fc514009040 comp rx=0 tx=0).stop 2024-04-25T08:54:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 -- 172.21.15.37:0/851825146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51c0afb60 msgr2=0x7fc51c1540c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51c0afb60 0x7fc51c1540c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0ae040 tx=0x7fc50c019180 comp rx=0 tx=0).stop 2024-04-25T08:54:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 -- 172.21.15.37:0/851825146 shutdown_connections 2024-04-25T08:54:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4fc066f60 0x7fc4fc069420 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc51c0afb60 0x7fc51c1540c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc51c06f610 0x7fc51c157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 --2- 172.21.15.37:0/851825146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc51c06ac90 0x7fc51c14ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 -- 172.21.15.37:0/851825146 >> 172.21.15.37:0/851825146 conn(0x7fc51c099a70 msgr2=0x7fc51c0ae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 -- 172.21.15.37:0/851825146 shutdown_connections 2024-04-25T08:54:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.396+0000 7fc52216f640 1 -- 172.21.15.37:0/851825146 wait complete. 2024-04-25T08:54:07.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:53:53.619419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:53:54.197204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:07.407 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:07.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0aa90e640 1 Processor -- start 2024-04-25T08:54:07.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0aa90e640 1 -- start start 2024-04-25T08:54:07.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0aa90e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe0a4161c80 0x7fe0a40b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0aa90e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40c9f70 0x7fe0a40c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0aa90e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe0a40c2870 0x7fe0a4165770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0aa90e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a4162060 con 0x7fe0a40c2870 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0aa90e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0a40ca480 con 0x7fe0a4161c80 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0aa90e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a40ca740 con 0x7fe0a40c9f70 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0a8eab640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe0a40c2870 0x7fe0a4165770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0a8eab640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe0a40c2870 0x7fe0a4165770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38234/0 (socket says 172.21.15.37:38234) 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0a8eab640 1 -- 172.21.15.37:0/1689527987 learned_addr learned my addr 172.21.15.37:0/1689527987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0a37fe640 1 --2- 172.21.15.37:0/1689527987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40c9f70 0x7fe0a40c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0a3fff640 1 --2- 172.21.15.37:0/1689527987 >> v2:172.21.15.203:3300/0 conn(0x7fe0a4161c80 0x7fe0a40b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0a8eab640 1 -- 172.21.15.37:0/1689527987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40c9f70 msgr2=0x7fe0a40c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:07.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0a8eab640 1 --2- 172.21.15.37:0/1689527987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40c9f70 0x7fe0a40c2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0a8eab640 1 -- 172.21.15.37:0/1689527987 >> v2:172.21.15.203:3300/0 conn(0x7fe0a4161c80 msgr2=0x7fe0a40b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:07.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0a8eab640 1 --2- 172.21.15.37:0/1689527987 >> v2:172.21.15.203:3300/0 conn(0x7fe0a4161c80 0x7fe0a40b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0a8eab640 1 -- 172.21.15.37:0/1689527987 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a40caa20 con 0x7fe0a40c2870 2024-04-25T08:54:07.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.544+0000 7fe0a37fe640 1 --2- 172.21.15.37:0/1689527987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40c9f70 0x7fe0a40c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:07.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0a8eab640 1 --2- 172.21.15.37:0/1689527987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40c2870 0x7fe0a4165770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40cf790 tx=0x7fe09401c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0078ad35e3b6f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:07.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0a2ffd640 1 -- 172.21.15.37:0/1689527987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940b0070 con 0x7fe0a40c2870 2024-04-25T08:54:07.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0a2ffd640 1 -- 172.21.15.37:0/1689527987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0940ab600 con 0x7fe0a40c2870 2024-04-25T08:54:07.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0a2ffd640 1 -- 172.21.15.37:0/1689527987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940ab9d0 con 0x7fe0a40c2870 2024-04-25T08:54:07.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 -- 172.21.15.37:0/1689527987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40c2870 msgr2=0x7fe0a4165770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 --2- 172.21.15.37:0/1689527987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40c2870 0x7fe0a4165770 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe0a40cf790 tx=0x7fe09401c1d0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 -- 172.21.15.37:0/1689527987 shutdown_connections 2024-04-25T08:54:07.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 --2- 172.21.15.37:0/1689527987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a40c2870 0x7fe0a4165770 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 --2- 172.21.15.37:0/1689527987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40c9f70 0x7fe0a40c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 --2- 172.21.15.37:0/1689527987 >> v2:172.21.15.203:3300/0 conn(0x7fe0a4161c80 0x7fe0a40b3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 -- 172.21.15.37:0/1689527987 >> 172.21.15.37:0/1689527987 conn(0x7fe0a406ae60 msgr2=0x7fe0a406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:07.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 -- 172.21.15.37:0/1689527987 shutdown_connections 2024-04-25T08:54:07.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 -- 172.21.15.37:0/1689527987 wait complete. 2024-04-25T08:54:07.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 Processor -- start 2024-04-25T08:54:07.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 -- start start 2024-04-25T08:54:07.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40c2870 0x7fe0a40afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a40c9f70 0x7fe0a4098570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a4161c80 0x7fe0a4098ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a40ccf30 con 0x7fe0a4161c80 2024-04-25T08:54:07.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a40b3ba0 con 0x7fe0a40c9f70 2024-04-25T08:54:07.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0aa90e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0a40cd0b0 con 0x7fe0a40c2870 2024-04-25T08:54:07.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0a8eab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a4161c80 0x7fe0a4098ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0a8eab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a4161c80 0x7fe0a4098ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38240/0 (socket says 172.21.15.37:38240) 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.548+0000 7fe0a8eab640 1 -- 172.21.15.37:0/3094586176 learned_addr learned my addr 172.21.15.37:0/3094586176 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a3fff640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40c2870 0x7fe0a40afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a37fe640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a40c9f70 0x7fe0a4098570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a8eab640 1 -- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40c2870 msgr2=0x7fe0a40afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a8eab640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40c2870 0x7fe0a40afc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a8eab640 1 -- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a40c9f70 msgr2=0x7fe0a4098570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a8eab640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a40c9f70 0x7fe0a4098570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a8eab640 1 -- 172.21.15.37:0/3094586176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a40ad880 con 0x7fe0a4161c80 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a8eab640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a4161c80 0x7fe0a4098ab0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe0940ae040 tx=0x7fe094018f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a0ff9640 1 -- 172.21.15.37:0/3094586176 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940b0040 con 0x7fe0a4161c80 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a0ff9640 1 -- 172.21.15.37:0/3094586176 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0940b9070 con 0x7fe0a4161c80 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a0ff9640 1 -- 172.21.15.37:0/3094586176 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0940b4670 con 0x7fe0a4161c80 2024-04-25T08:54:07.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0aa90e640 1 -- 172.21.15.37:0/3094586176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0a40adb60 con 0x7fe0a4161c80 2024-04-25T08:54:07.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0aa90e640 1 -- 172.21.15.37:0/3094586176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0a40b0150 con 0x7fe0a4161c80 2024-04-25T08:54:07.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0aa90e640 1 -- 172.21.15.37:0/3094586176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0a404efa0 con 0x7fe0a4161c80 2024-04-25T08:54:07.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a0ff9640 1 -- 172.21.15.37:0/3094586176 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0940a4d10 con 0x7fe0a4161c80 2024-04-25T08:54:07.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a0ff9640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe084066f60 0x7fe084069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a0ff9640 1 -- 172.21.15.37:0/3094586176 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe09411ebd0 con 0x7fe0a4161c80 2024-04-25T08:54:07.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a3fff640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe084066f60 0x7fe084069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.552+0000 7fe0a3fff640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe084066f60 0x7fe084069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe09c003fd0 tx=0x7fe09c0039a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:07.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.556+0000 7fe0a0ff9640 1 -- 172.21.15.37:0/3094586176 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe094016110 con 0x7fe0a4161c80 2024-04-25T08:54:07.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.712+0000 7fe0aa90e640 1 -- 172.21.15.37:0/3094586176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0a40af1b0 con 0x7fe0a4161c80 2024-04-25T08:54:07.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0a0ff9640 1 -- 172.21.15.37:0/3094586176 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0940efae0 con 0x7fe0a4161c80 2024-04-25T08:54:07.707 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:54:07.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 -- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe084066f60 msgr2=0x7fe084069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe084066f60 0x7fe084069420 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe09c003fd0 tx=0x7fe09c0039a0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 -- 172.21.15.37:0/3094586176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a4161c80 msgr2=0x7fe0a4098ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a4161c80 0x7fe0a4098ab0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe0940ae040 tx=0x7fe094018f70 comp rx=0 tx=0).stop 2024-04-25T08:54:07.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 -- 172.21.15.37:0/3094586176 shutdown_connections 2024-04-25T08:54:07.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe084066f60 0x7fe084069420 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0a4161c80 0x7fe0a4098ab0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0a40c9f70 0x7fe0a4098570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 --2- 172.21.15.37:0/3094586176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0a40c2870 0x7fe0a40afc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 -- 172.21.15.37:0/3094586176 >> 172.21.15.37:0/3094586176 conn(0x7fe0a406ae60 msgr2=0x7fe0a406e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:07.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 -- 172.21.15.37:0/3094586176 shutdown_connections 2024-04-25T08:54:07.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.716+0000 7fe0aa90e640 1 -- 172.21.15.37:0/3094586176 wait complete. 2024-04-25T08:54:07.724 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:07.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe18ae84640 1 Processor -- start 2024-04-25T08:54:07.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe18ae84640 1 -- start start 2024-04-25T08:54:07.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe18ae84640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe18415b920 0x7fe184099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe18ae84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1840b0440 0x7fe184099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe18ae84640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18409a950 con 0x7fe1840aeb80 2024-04-25T08:54:07.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe18ae84640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18409ac30 con 0x7fe18415b920 2024-04-25T08:54:07.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe18ae84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1840ae380 con 0x7fe1840b0440 2024-04-25T08:54:07.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe189421640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe1840aeb80 0x7fe1840aeff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41916/0 (socket says 172.21.15.37:41916) 2024-04-25T08:54:07.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe189421640 1 -- 172.21.15.37:0/1767149392 learned_addr learned my addr 172.21.15.37:0/1767149392 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:07.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe188c20640 1 --2- 172.21.15.37:0/1767149392 >> v2:172.21.15.37:3300/0 conn(0x7fe18415b920 0x7fe184099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe183fff640 1 --2- 172.21.15.37:0/1767149392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1840b0440 0x7fe184099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe1837fe640 1 -- 172.21.15.37:0/1767149392 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901950114 0 0) 0x7fe18409a950 con 0x7fe1840aeb80 2024-04-25T08:54:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe1837fe640 1 -- 172.21.15.37:0/1767149392 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe170003530 con 0x7fe1840aeb80 2024-04-25T08:54:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe188c20640 1 -- 172.21.15.37:0/1767149392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1840b0440 msgr2=0x7fe184099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe188c20640 1 --2- 172.21.15.37:0/1767149392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1840b0440 0x7fe184099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe188c20640 1 -- 172.21.15.37:0/1767149392 >> v1:172.21.15.37:6789/0 conn(0x7fe1840aeb80 legacy=0x7fe1840aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.860+0000 7fe188c20640 1 -- 172.21.15.37:0/1767149392 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1840ae660 con 0x7fe18415b920 2024-04-25T08:54:07.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe188c20640 1 --2- 172.21.15.37:0/1767149392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18415b920 0x7fe184099a70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe1740097c0 tx=0x7fe17401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e769d4760cac05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:07.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe1837fe640 1 -- 172.21.15.37:0/1767149392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1740b1070 con 0x7fe18415b920 2024-04-25T08:54:07.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe1837fe640 1 -- 172.21.15.37:0/1767149392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1740ac410 con 0x7fe18415b920 2024-04-25T08:54:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe1837fe640 1 -- 172.21.15.37:0/1767149392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1740ac7e0 con 0x7fe18415b920 2024-04-25T08:54:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe18ae84640 1 -- 172.21.15.37:0/1767149392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18415b920 msgr2=0x7fe184099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe18ae84640 1 --2- 172.21.15.37:0/1767149392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18415b920 0x7fe184099a70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe1740097c0 tx=0x7fe17401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe18ae84640 1 -- 172.21.15.37:0/1767149392 shutdown_connections 2024-04-25T08:54:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe18ae84640 1 --2- 172.21.15.37:0/1767149392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1840b0440 0x7fe184099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe18ae84640 1 --2- 172.21.15.37:0/1767149392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe18415b920 0x7fe184099a70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe18ae84640 1 -- 172.21.15.37:0/1767149392 >> 172.21.15.37:0/1767149392 conn(0x7fe1840b3010 msgr2=0x7fe1840b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe18ae84640 1 -- 172.21.15.37:0/1767149392 shutdown_connections 2024-04-25T08:54:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe18ae84640 1 -- 172.21.15.37:0/1767149392 wait complete. 2024-04-25T08:54:07.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe18ae84640 1 Processor -- start 2024-04-25T08:54:07.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.864+0000 7fe18ae84640 1 -- start start 2024-04-25T08:54:07.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe18ae84640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1840aeb80 0x7fe184157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe18ae84640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1840b0440 0x7fe18415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe18ae84640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe18415b920 0x7fe18415e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe18ae84640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe184168850 con 0x7fe1840b0440 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe18ae84640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1841686d0 con 0x7fe1840aeb80 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe18ae84640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1841689d0 con 0x7fe18415b920 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe183fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1840b0440 0x7fe18415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe183fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1840b0440 0x7fe18415e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38270/0 (socket says 172.21.15.37:38270) 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe183fff640 1 -- 172.21.15.37:0/3096888547 learned_addr learned my addr 172.21.15.37:0/3096888547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe188c20640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1840aeb80 0x7fe184157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe183fff640 1 -- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe18415b920 msgr2=0x7fe18415e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe183fff640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe18415b920 0x7fe18415e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe183fff640 1 -- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1840aeb80 msgr2=0x7fe184157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe183fff640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1840aeb80 0x7fe184157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:07.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe183fff640 1 -- 172.21.15.37:0/3096888547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1840b11c0 con 0x7fe1840b0440 2024-04-25T08:54:07.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe183fff640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1840b0440 0x7fe18415e140 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00e870 tx=0x7fe16c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:07.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe1817fa640 1 -- 172.21.15.37:0/3096888547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c017070 con 0x7fe1840b0440 2024-04-25T08:54:07.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe1817fa640 1 -- 172.21.15.37:0/3096888547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe16c00c940 con 0x7fe1840b0440 2024-04-25T08:54:07.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe18ae84640 1 -- 172.21.15.37:0/3096888547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1840b6480 con 0x7fe1840b0440 2024-04-25T08:54:07.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe1817fa640 1 -- 172.21.15.37:0/3096888547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16c00ccf0 con 0x7fe1840b0440 2024-04-25T08:54:07.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe18ae84640 1 -- 172.21.15.37:0/3096888547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1840b6a40 con 0x7fe1840b0440 2024-04-25T08:54:07.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe18ae84640 1 -- 172.21.15.37:0/3096888547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18404ef80 con 0x7fe1840b0440 2024-04-25T08:54:07.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe1817fa640 1 -- 172.21.15.37:0/3096888547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe16c01a070 con 0x7fe1840b0440 2024-04-25T08:54:07.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe1817fa640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe164066f60 0x7fe164069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:07.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe1817fa640 1 -- 172.21.15.37:0/3096888547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe16c085060 con 0x7fe1840b0440 2024-04-25T08:54:07.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.868+0000 7fe188c20640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe164066f60 0x7fe164069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:07.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.872+0000 7fe188c20640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe164066f60 0x7fe164069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1740ae000 tx=0x7fe1740a6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:07.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:07.872+0000 7fe1817fa640 1 -- 172.21.15.37:0/3096888547 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe1840b11c0 con 0x7fe1840b0440 2024-04-25T08:54:08.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.032+0000 7fe18ae84640 1 -- 172.21.15.37:0/3096888547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1840b25f0 con 0x7fe1840b0440 2024-04-25T08:54:08.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.032+0000 7fe1817fa640 1 -- 172.21.15.37:0/3096888547 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v41) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe16c055ed0 con 0x7fe1840b0440 2024-04-25T08:54:08.025 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:54:08.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 -- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe164066f60 msgr2=0x7fe164069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe164066f60 0x7fe164069420 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe1740ae000 tx=0x7fe1740a6040 comp rx=0 tx=0).stop 2024-04-25T08:54:08.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 -- 172.21.15.37:0/3096888547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1840b0440 msgr2=0x7fe18415e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1840b0440 0x7fe18415e140 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe16c00e870 tx=0x7fe16c00ec70 comp rx=0 tx=0).stop 2024-04-25T08:54:08.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 -- 172.21.15.37:0/3096888547 shutdown_connections 2024-04-25T08:54:08.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe164066f60 0x7fe164069420 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe18415b920 0x7fe18415e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1840b0440 0x7fe18415e140 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 --2- 172.21.15.37:0/3096888547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe1840aeb80 0x7fe184157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 -- 172.21.15.37:0/3096888547 >> 172.21.15.37:0/3096888547 conn(0x7fe1840b3010 msgr2=0x7fe1840b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:08.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 -- 172.21.15.37:0/3096888547 shutdown_connections 2024-04-25T08:54:08.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.036+0000 7fe18ae84640 1 -- 172.21.15.37:0/3096888547 wait complete. 2024-04-25T08:54:08.043 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:08.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.164+0000 7fb097a23640 1 Processor -- start 2024-04-25T08:54:08.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb097a23640 1 -- start start 2024-04-25T08:54:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb097a23640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb090161c80 0x7fb0900b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb097a23640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900c9f70 0x7fb0900c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb097a23640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb090162060 con 0x7fb0900c2870 2024-04-25T08:54:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb097a23640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0900ca480 con 0x7fb090161c80 2024-04-25T08:54:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb097a23640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900ca740 con 0x7fb0900c9f70 2024-04-25T08:54:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb095fc0640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb0900c2870 0x7fb090165770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41930/0 (socket says 172.21.15.37:41930) 2024-04-25T08:54:08.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb095fc0640 1 -- 172.21.15.37:0/230896184 learned_addr learned my addr 172.21.15.37:0/230896184 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb094fbe640 1 --2- 172.21.15.37:0/230896184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900c9f70 0x7fb0900c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb0957bf640 1 --2- 172.21.15.37:0/230896184 >> v2:172.21.15.203:3300/0 conn(0x7fb090161c80 0x7fb0900b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553590106 0 0) 0x7fb090162060 con 0x7fb0900c2870 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb07c003530 con 0x7fb0900c2870 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3880824654 0 0) 0x7fb07c003530 con 0x7fb0900c2870 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900c9f70 msgr2=0x7fb0900c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 --2- 172.21.15.37:0/230896184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900c9f70 0x7fb0900c2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 >> v2:172.21.15.203:3300/0 conn(0x7fb090161c80 msgr2=0x7fb0900b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 --2- 172.21.15.37:0/230896184 >> v2:172.21.15.203:3300/0 conn(0x7fb090161c80 0x7fb0900b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0900caa20 con 0x7fb0900c2870 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fb088001a00 con 0x7fb0900c2870 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 >> v1:172.21.15.37:6789/0 conn(0x7fb0900c2870 legacy=0x7fb090165770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 --2- 172.21.15.37:0/230896184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb07c003a30 0x7fb07c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 --2- 172.21.15.37:0/230896184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900c9f70 0x7fb07c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 --2- 172.21.15.37:0/230896184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb07c004d50 0x7fb07c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900ca480 con 0x7fb07c003a30 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900ca740 con 0x7fb07c004d50 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07c00ba20 con 0x7fb0900c9f70 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb0957bf640 1 --2- 172.21.15.37:0/230896184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb07c003a30 0x7fb07c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb094fbe640 1 --2- 172.21.15.37:0/230896184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900c9f70 0x7fb07c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb0957bf640 1 -- 172.21.15.37:0/230896184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900c9f70 msgr2=0x7fb07c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb0957bf640 1 --2- 172.21.15.37:0/230896184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0900c9f70 0x7fb07c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb0957bf640 1 -- 172.21.15.37:0/230896184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb07c004d50 msgr2=0x7fb07c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb0957bf640 1 --2- 172.21.15.37:0/230896184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb07c004d50 0x7fb07c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.168+0000 7fb0957bf640 1 -- 172.21.15.37:0/230896184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb07c003530 con 0x7fb07c003a30 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb0957bf640 1 --2- 172.21.15.37:0/230896184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb07c003a30 0x7fb07c003e50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009ae0 tx=0x7fb08c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730801a8d92bb417 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0b2070 con 0x7fb07c003a30 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c0a5570 con 0x7fb07c003a30 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb087fff640 1 -- 172.21.15.37:0/230896184 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0096b0 con 0x7fb07c003a30 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 -- 172.21.15.37:0/230896184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb07c003a30 msgr2=0x7fb07c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 --2- 172.21.15.37:0/230896184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb07c003a30 0x7fb07c003e50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009ae0 tx=0x7fb08c01cab0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb095fc0640 1 -- 172.21.15.37:0/230896184 reap_dead start 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 -- 172.21.15.37:0/230896184 shutdown_connections 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 -- 172.21.15.37:0/230896184 >> 172.21.15.37:0/230896184 conn(0x7fb09006ae60 msgr2=0x7fb09006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 -- 172.21.15.37:0/230896184 shutdown_connections 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 -- 172.21.15.37:0/230896184 wait complete. 2024-04-25T08:54:08.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 Processor -- start 2024-04-25T08:54:08.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 -- start start 2024-04-25T08:54:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb07c003a30 0x7fb09015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0901602f0 0x7fb090167730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb090167c70 0x7fb0900c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb09016c890 con 0x7fb090167c70 2024-04-25T08:54:08.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0900b3ba0 con 0x7fb07c003a30 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb097a23640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09016ca10 con 0x7fb0901602f0 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb095fc0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb090167c70 0x7fb0900c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb095fc0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb090167c70 0x7fb0900c67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38280/0 (socket says 172.21.15.37:38280) 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb095fc0640 1 -- 172.21.15.37:0/2475442682 learned_addr learned my addr 172.21.15.37:0/2475442682 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb0957bf640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb07c003a30 0x7fb09015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb094fbe640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0901602f0 0x7fb090167730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb095fc0640 1 -- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0901602f0 msgr2=0x7fb090167730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb095fc0640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0901602f0 0x7fb090167730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb095fc0640 1 -- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb07c003a30 msgr2=0x7fb09015dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb095fc0640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb07c003a30 0x7fb09015dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb095fc0640 1 -- 172.21.15.37:0/2475442682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0901fe790 con 0x7fb090167c70 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb094fbe640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0901602f0 0x7fb090167730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb095fc0640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb090167c70 0x7fb0900c67d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb0900c9940 tx=0x7fb08800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.172+0000 7fb085ffb640 1 -- 172.21.15.37:0/2475442682 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800a5e0 con 0x7fb090167c70 2024-04-25T08:54:08.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.176+0000 7fb085ffb640 1 -- 172.21.15.37:0/2475442682 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08800a7a0 con 0x7fb090167c70 2024-04-25T08:54:08.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.176+0000 7fb097a23640 1 -- 172.21.15.37:0/2475442682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0901fea70 con 0x7fb090167c70 2024-04-25T08:54:08.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.176+0000 7fb085ffb640 1 -- 172.21.15.37:0/2475442682 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800ab90 con 0x7fb090167c70 2024-04-25T08:54:08.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.176+0000 7fb097a23640 1 -- 172.21.15.37:0/2475442682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0901ff030 con 0x7fb090167c70 2024-04-25T08:54:08.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.176+0000 7fb097a23640 1 -- 172.21.15.37:0/2475442682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09004efa0 con 0x7fb090167c70 2024-04-25T08:54:08.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.176+0000 7fb085ffb640 1 -- 172.21.15.37:0/2475442682 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb08800ad30 con 0x7fb090167c70 2024-04-25T08:54:08.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.176+0000 7fb085ffb640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.176+0000 7fb085ffb640 1 -- 172.21.15.37:0/2475442682 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb0880849b0 con 0x7fb090167c70 2024-04-25T08:54:08.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.176+0000 7fb0957bf640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.176+0000 7fb0957bf640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb074066f60 0x7fb074069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0af000 tx=0x7fb08c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.180+0000 7fb085ffb640 1 -- 172.21.15.37:0/2475442682 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb0880557a0 con 0x7fb090167c70 2024-04-25T08:54:08.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.332+0000 7fb097a23640 1 -- 172.21.15.37:0/2475442682 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0901654c0 con 0x7fb090167c70 2024-04-25T08:54:08.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb085ffb640 1 -- 172.21.15.37:0/2475442682 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v41) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0880550a0 con 0x7fb090167c70 2024-04-25T08:54:08.328 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:54:08.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 -- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb074066f60 msgr2=0x7fb074069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb074066f60 0x7fb074069420 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb08c0af000 tx=0x7fb08c0046d0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 -- 172.21.15.37:0/2475442682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb090167c70 msgr2=0x7fb0900c67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb090167c70 0x7fb0900c67d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fb0900c9940 tx=0x7fb08800ccd0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 -- 172.21.15.37:0/2475442682 shutdown_connections 2024-04-25T08:54:08.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb074066f60 0x7fb074069420 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb090167c70 0x7fb0900c67d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0901602f0 0x7fb090167730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 --2- 172.21.15.37:0/2475442682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb07c003a30 0x7fb09015dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 -- 172.21.15.37:0/2475442682 >> 172.21.15.37:0/2475442682 conn(0x7fb09006ae60 msgr2=0x7fb090160e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 -- 172.21.15.37:0/2475442682 shutdown_connections 2024-04-25T08:54:08.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.336+0000 7fb097a23640 1 -- 172.21.15.37:0/2475442682 wait complete. 2024-04-25T08:54:08.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:54:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.476+0000 7f882b953640 1 Processor -- start 2024-04-25T08:54:08.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.476+0000 7f882b953640 1 -- start start 2024-04-25T08:54:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.476+0000 7f882b953640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f882415ecc0 0x7f882406e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.476+0000 7f882b953640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88240b2090 con 0x7f88240b3f50 2024-04-25T08:54:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.476+0000 7f882b953640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88240b43e0 con 0x7f882415a700 2024-04-25T08:54:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.476+0000 7f882b953640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88240b46c0 con 0x7f882415ecc0 2024-04-25T08:54:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.476+0000 7f8828eee640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f88240b3f50 0x7f88240b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41944/0 (socket says 172.21.15.37:41944) 2024-04-25T08:54:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.476+0000 7f8828eee640 1 -- 172.21.15.37:0/2239682204 learned_addr learned my addr 172.21.15.37:0/2239682204 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8829ef0640 1 --2- 172.21.15.37:0/2239682204 >> v2:172.21.15.203:3300/0 conn(0x7f882415ecc0 0x7f882406e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919506540 0 0) 0x7f88240b2090 con 0x7f88240b3f50 2024-04-25T08:54:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8810005740 con 0x7f88240b3f50 2024-04-25T08:54:08.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3413612257 0 0) 0x7f8810005740 con 0x7f88240b3f50 2024-04-25T08:54:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 >> v2:172.21.15.203:3300/0 conn(0x7f882415ecc0 msgr2=0x7f882406e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 --2- 172.21.15.37:0/2239682204 >> v2:172.21.15.203:3300/0 conn(0x7f882415ecc0 0x7f882406e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 >> v1:172.21.15.203:6789/0 conn(0x7f882415a700 legacy=0x7f88240cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88240cd6f0 con 0x7f88240b3f50 2024-04-25T08:54:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f88200016a0 con 0x7f88240b3f50 2024-04-25T08:54:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 >> v1:172.21.15.37:6789/0 conn(0x7f88240b3f50 legacy=0x7f88240b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8810005c40 0x7f8810006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8810006a20 0x7f8810006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88100074e0 0x7f881000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88240b46c0 con 0x7f88100074e0 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881000e420 con 0x7f8810006a20 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f881000e620 con 0x7f8810005c40 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8829ef0640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88100074e0 0x7f881000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f88296ef640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8810005c40 0x7f8810006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8828eee640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8810006a20 0x7f8810006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8829ef0640 1 -- 172.21.15.37:0/2239682204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8810005c40 msgr2=0x7f8810006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8829ef0640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8810005c40 0x7f8810006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8829ef0640 1 -- 172.21.15.37:0/2239682204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8810006a20 msgr2=0x7f8810006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8829ef0640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8810006a20 0x7f8810006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8829ef0640 1 -- 172.21.15.37:0/2239682204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8810005740 con 0x7f88100074e0 2024-04-25T08:54:08.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8828eee640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8810006a20 0x7f8810006ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:08.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f88296ef640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8810005c40 0x7f8810006060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:08.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8829ef0640 1 -- 172.21.15.37:0/2239682204 reap_dead start 2024-04-25T08:54:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8829ef0640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88100074e0 0x7f881000dbe0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f881c009fc0 tx=0x7f881c01cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5acb261c35888c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c009c10 con 0x7f88100074e0 2024-04-25T08:54:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881c003ba0 con 0x7f88100074e0 2024-04-25T08:54:08.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.480+0000 7f8817fff640 1 -- 172.21.15.37:0/2239682204 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c0a5890 con 0x7f88100074e0 2024-04-25T08:54:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 -- 172.21.15.37:0/2239682204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88100074e0 msgr2=0x7f881000dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88100074e0 0x7f881000dbe0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f881c009fc0 tx=0x7f881c01cc90 comp rx=0 tx=0).stop 2024-04-25T08:54:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 -- 172.21.15.37:0/2239682204 shutdown_connections 2024-04-25T08:54:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 --2- 172.21.15.37:0/2239682204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88100074e0 0x7f881000dbe0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 -- 172.21.15.37:0/2239682204 >> 172.21.15.37:0/2239682204 conn(0x7f88240b11a0 msgr2=0x7f88240b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 -- 172.21.15.37:0/2239682204 shutdown_connections 2024-04-25T08:54:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 -- 172.21.15.37:0/2239682204 wait complete. 2024-04-25T08:54:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 Processor -- start 2024-04-25T08:54:08.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 -- start start 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88100074e0 0x7f882415acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88241417d0 0x7f88241447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f882415e6c0 0x7f8824158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88240cd490 con 0x7f88241417d0 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88240cd310 con 0x7f882415e6c0 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882406e1a0 con 0x7f88100074e0 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8828eee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88241417d0 0x7f88241447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8828eee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88241417d0 0x7f88241447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38296/0 (socket says 172.21.15.37:38296) 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8828eee640 1 -- 172.21.15.37:0/4097348774 learned_addr learned my addr 172.21.15.37:0/4097348774 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f88296ef640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88100074e0 0x7f882415acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8829ef0640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f882415e6c0 0x7f8824158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8828eee640 1 -- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88100074e0 msgr2=0x7f882415acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8828eee640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88100074e0 0x7f882415acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8828eee640 1 -- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f882415e6c0 msgr2=0x7f8824158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8828eee640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f882415e6c0 0x7f8824158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8828eee640 1 -- 172.21.15.37:0/4097348774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8824144d30 con 0x7f88241417d0 2024-04-25T08:54:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8828eee640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88241417d0 0x7f88241447f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8820000fe0 tx=0x7f882000db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8815ffb640 1 -- 172.21.15.37:0/4097348774 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820019070 con 0x7f88241417d0 2024-04-25T08:54:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8815ffb640 1 -- 172.21.15.37:0/4097348774 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88200072d0 con 0x7f88241417d0 2024-04-25T08:54:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 -- 172.21.15.37:0/4097348774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8824141f40 con 0x7f88241417d0 2024-04-25T08:54:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f8815ffb640 1 -- 172.21.15.37:0/4097348774 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820007680 con 0x7f88241417d0 2024-04-25T08:54:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.484+0000 7f882b953640 1 -- 172.21.15.37:0/4097348774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8824142500 con 0x7f88241417d0 2024-04-25T08:54:08.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.488+0000 7f882b953640 1 -- 172.21.15.37:0/4097348774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882404efa0 con 0x7f88241417d0 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.488+0000 7f8815ffb640 1 -- 172.21.15.37:0/4097348774 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f882001c070 con 0x7f88241417d0 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.488+0000 7f8815ffb640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8808066f60 0x7f8808069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.488+0000 7f8815ffb640 1 -- 172.21.15.37:0/4097348774 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8820084e70 con 0x7f88241417d0 2024-04-25T08:54:08.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.488+0000 7f88296ef640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8808066f60 0x7f8808069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.488+0000 7f88296ef640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8808066f60 0x7f8808069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8818007fd0 tx=0x7f8818009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.492+0000 7f8815ffb640 1 -- 172.21.15.37:0/4097348774 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8820023020 con 0x7f88241417d0 2024-04-25T08:54:08.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.656+0000 7f882b953640 1 -- 172.21.15.37:0/4097348774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f882406f260 con 0x7f88241417d0 2024-04-25T08:54:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f8815ffb640 1 -- 172.21.15.37:0/4097348774 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8820055dd0 con 0x7f88241417d0 2024-04-25T08:54:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:54:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 -- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8808066f60 msgr2=0x7f8808069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8808066f60 0x7f8808069420 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8818007fd0 tx=0x7f8818009040 comp rx=0 tx=0).stop 2024-04-25T08:54:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 -- 172.21.15.37:0/4097348774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88241417d0 msgr2=0x7f88241447f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88241417d0 0x7f88241447f0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8820000fe0 tx=0x7f882000db50 comp rx=0 tx=0).stop 2024-04-25T08:54:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 -- 172.21.15.37:0/4097348774 shutdown_connections 2024-04-25T08:54:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8808066f60 0x7f8808069420 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f882415e6c0 0x7f8824158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88241417d0 0x7f88241447f0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 --2- 172.21.15.37:0/4097348774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88100074e0 0x7f882415acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 -- 172.21.15.37:0/4097348774 >> 172.21.15.37:0/4097348774 conn(0x7f88240b11a0 msgr2=0x7f88240cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 -- 172.21.15.37:0/4097348774 shutdown_connections 2024-04-25T08:54:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:08.660+0000 7f882b953640 1 -- 172.21.15.37:0/4097348774 wait complete. 2024-04-25T08:54:08.669 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T08:54:08.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:08.669 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:08.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[39457]: fuse finished with error 0 and tester_r 0 2024-04-25T08:54:08.682 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:54:14.783 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:54:14.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:14.783 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:54:14.791 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:14.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb87d937640 1 Processor -- start 2024-04-25T08:54:14.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb87d937640 1 -- start start 2024-04-25T08:54:14.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb87d937640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb87815da80 0x7fb8780d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:14.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb87d937640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87806be60 con 0x7fb8780d07f0 2024-04-25T08:54:14.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb87d937640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb87815de60 con 0x7fb8780d1cd0 2024-04-25T08:54:14.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb87d937640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb87806c440 con 0x7fb87815da80 2024-04-25T08:54:14.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb8777fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb8780d07f0 0x7fb87806c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45806/0 (socket says 172.21.15.37:45806) 2024-04-25T08:54:14.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb8777fe640 1 -- 172.21.15.37:0/1590258672 learned_addr learned my addr 172.21.15.37:0/1590258672 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:14.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb876ffd640 1 --2- 172.21.15.37:0/1590258672 >> v2:172.21.15.37:3300/0 conn(0x7fb87815da80 0x7fb8780d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:14.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb876ffd640 1 -- 172.21.15.37:0/1590258672 >> v1:172.21.15.203:6789/0 conn(0x7fb8780d1cd0 legacy=0x7fb8780d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb876ffd640 1 -- 172.21.15.37:0/1590258672 >> v1:172.21.15.37:6789/0 conn(0x7fb8780d07f0 legacy=0x7fb87806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb875ffb640 1 -- 172.21.15.37:0/1590258672 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862809599 0 0) 0x7fb87806be60 con 0x7fb8780d07f0 2024-04-25T08:54:14.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.979+0000 7fb876ffd640 1 -- 172.21.15.37:0/1590258672 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8780d2450 con 0x7fb87815da80 2024-04-25T08:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb876ffd640 1 --2- 172.21.15.37:0/1590258672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87815da80 0x7fb8780d4230 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb87806c5c0 tx=0x7fb86801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b65570de6febcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb875ffb640 1 -- 172.21.15.37:0/1590258672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8680b1070 con 0x7fb87815da80 2024-04-25T08:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb875ffb640 1 -- 172.21.15.37:0/1590258672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8680ac410 con 0x7fb87815da80 2024-04-25T08:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb875ffb640 1 -- 172.21.15.37:0/1590258672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8680ac7e0 con 0x7fb87815da80 2024-04-25T08:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 -- 172.21.15.37:0/1590258672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87815da80 msgr2=0x7fb8780d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 --2- 172.21.15.37:0/1590258672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87815da80 0x7fb8780d4230 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb87806c5c0 tx=0x7fb86801c450 comp rx=0 tx=0).stop 2024-04-25T08:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 -- 172.21.15.37:0/1590258672 shutdown_connections 2024-04-25T08:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 --2- 172.21.15.37:0/1590258672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87815da80 0x7fb8780d4230 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 -- 172.21.15.37:0/1590258672 >> 172.21.15.37:0/1590258672 conn(0x7fb87808b000 msgr2=0x7fb8780d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 -- 172.21.15.37:0/1590258672 shutdown_connections 2024-04-25T08:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 -- 172.21.15.37:0/1590258672 wait complete. 2024-04-25T08:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 Processor -- start 2024-04-25T08:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 -- start start 2024-04-25T08:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8780d07f0 0x7fb8780c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8780d1cd0 0x7fb87815ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb87815da80 0x7fb878160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87806d970 con 0x7fb8780d1cd0 2024-04-25T08:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb87806d7f0 con 0x7fb87815da80 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb87d937640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87806daf0 con 0x7fb8780d07f0 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb8767fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8780d1cd0 0x7fb87815ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb8767fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8780d1cd0 0x7fb87815ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39702/0 (socket says 172.21.15.37:39702) 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb8767fc640 1 -- 172.21.15.37:0/1495451204 learned_addr learned my addr 172.21.15.37:0/1495451204 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb876ffd640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8780d07f0 0x7fb8780c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb8777fe640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb87815da80 0x7fb878160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb8767fc640 1 -- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8780d07f0 msgr2=0x7fb8780c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb8767fc640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8780d07f0 0x7fb8780c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb8767fc640 1 -- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb87815da80 msgr2=0x7fb878160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb8767fc640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb87815da80 0x7fb878160540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb8767fc640 1 -- 172.21.15.37:0/1495451204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb87815f4c0 con 0x7fb8780d1cd0 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.983+0000 7fb8767fc640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8780d1cd0 0x7fb87815ea70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00c480 tx=0x7fb86c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.987+0000 7fb85ffff640 1 -- 172.21.15.37:0/1495451204 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0052d0 con 0x7fb8780d1cd0 2024-04-25T08:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.987+0000 7fb85ffff640 1 -- 172.21.15.37:0/1495451204 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86c005470 con 0x7fb8780d1cd0 2024-04-25T08:54:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.987+0000 7fb87d937640 1 -- 172.21.15.37:0/1495451204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb87815f7a0 con 0x7fb8780d1cd0 2024-04-25T08:54:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.987+0000 7fb85ffff640 1 -- 172.21.15.37:0/1495451204 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86c003890 con 0x7fb8780d1cd0 2024-04-25T08:54:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.987+0000 7fb87d937640 1 -- 172.21.15.37:0/1495451204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8780ca4f0 con 0x7fb8780d1cd0 2024-04-25T08:54:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.987+0000 7fb87d937640 1 -- 172.21.15.37:0/1495451204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb878152d30 con 0x7fb8780d1cd0 2024-04-25T08:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.987+0000 7fb85ffff640 1 -- 172.21.15.37:0/1495451204 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb86c003a30 con 0x7fb8780d1cd0 2024-04-25T08:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.987+0000 7fb85ffff640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb864066f90 0x7fb864069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.987+0000 7fb85ffff640 1 -- 172.21.15.37:0/1495451204 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb86c084e40 con 0x7fb8780d1cd0 2024-04-25T08:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.987+0000 7fb876ffd640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb864066f90 0x7fb864069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.987+0000 7fb876ffd640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb864066f90 0x7fb864069450 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb8680ae000 tx=0x7fb868008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:14.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:14.991+0000 7fb85ffff640 1 -- 172.21.15.37:0/1495451204 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb86c055d30 con 0x7fb8780d1cd0 2024-04-25T08:54:15.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.191+0000 7fb87d937640 1 -- 172.21.15.37:0/1495451204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb878073cb0 con 0x7fb8780d1cd0 2024-04-25T08:54:15.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.191+0000 7fb85ffff640 1 -- 172.21.15.37:0/1495451204 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fb86c055630 con 0x7fb8780d1cd0 2024-04-25T08:54:15.186 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:15.186 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3245776986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3245776986},{"type":"v1","addr":"172.21.15.203:6836","nonce":3245776986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5056,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/841808653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":841808653},{"type":"v1","addr":"172.21.15.37:6835","nonce":841808653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1908736119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1908736119},{"type":"v1","addr":"172.21.15.203:6837","nonce":1908736119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:53:55.212059+0000","modified":"2024-04-25T08:53:58.247733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035847732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035847732},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035847732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:54:15.186 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-04-25T08:54:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 -- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb864066f90 msgr2=0x7fb864069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb864066f90 0x7fb864069450 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb8680ae000 tx=0x7fb868008040 comp rx=0 tx=0).stop 2024-04-25T08:54:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 -- 172.21.15.37:0/1495451204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8780d1cd0 msgr2=0x7fb87815ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8780d1cd0 0x7fb87815ea70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb86c00c480 tx=0x7fb86c00c940 comp rx=0 tx=0).stop 2024-04-25T08:54:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 -- 172.21.15.37:0/1495451204 shutdown_connections 2024-04-25T08:54:15.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb864066f90 0x7fb864069450 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb87815da80 0x7fb878160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8780d1cd0 0x7fb87815ea70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 --2- 172.21.15.37:0/1495451204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb8780d07f0 0x7fb8780c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 -- 172.21.15.37:0/1495451204 >> 172.21.15.37:0/1495451204 conn(0x7fb87808b000 msgr2=0x7fb8780b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 -- 172.21.15.37:0/1495451204 shutdown_connections 2024-04-25T08:54:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.195+0000 7fb87d937640 1 -- 172.21.15.37:0/1495451204 wait complete. 2024-04-25T08:54:15.205 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T08:54:15.357 INFO:teuthology.orchestra.run.smithi037.stdout:{"message":"","return_code":0} 2024-04-25T08:54:15.358 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:54:15.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:15.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b4a60d640 1 Processor -- start 2024-04-25T08:54:15.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b4a60d640 1 -- start start 2024-04-25T08:54:15.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b4a60d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b4415da80 0x7f0b440d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b4a60d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4406be60 con 0x7f0b440d1cd0 2024-04-25T08:54:15.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b4a60d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4415de60 con 0x7f0b440d07f0 2024-04-25T08:54:15.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b4a60d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4406c440 con 0x7f0b4415da80 2024-04-25T08:54:15.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b437fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0b440d1cd0 0x7f0b440d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45820/0 (socket says 172.21.15.37:45820) 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b437fe640 1 -- 172.21.15.37:0/912332322 learned_addr learned my addr 172.21.15.37:0/912332322 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b43fff640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b4415da80 0x7f0b440d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070027333 0 0) 0x7f0b4406be60 con 0x7f0b440d1cd0 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b30003530 con 0x7f0b440d1cd0 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1821136786 0 0) 0x7f0b30003530 con 0x7f0b440d1cd0 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b4415da80 msgr2=0x7f0b440d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b4415da80 0x7f0b440d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 >> v1:172.21.15.203:6789/0 conn(0x7f0b440d07f0 legacy=0x7f0b4406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b440d2450 con 0x7f0b440d1cd0 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0b2c001b80 con 0x7f0b440d1cd0 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 >> v1:172.21.15.37:6789/0 conn(0x7f0b440d1cd0 legacy=0x7f0b440d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b30003a60 0x7f0b30003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b4415da80 0x7f0b30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b30004d50 0x7f0b300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4406c440 con 0x7f0b30004d50 2024-04-25T08:54:15.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3000bb00 con 0x7f0b30003a60 2024-04-25T08:54:15.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3000bd00 con 0x7f0b4415da80 2024-04-25T08:54:15.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b48baa640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b30004d50 0x7f0b300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b43fff640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b30003a60 0x7f0b30003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b437fe640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b4415da80 0x7f0b30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b48baa640 1 -- 172.21.15.37:0/912332322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b4415da80 msgr2=0x7f0b30004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:15.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b48baa640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b4415da80 0x7f0b30004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b48baa640 1 -- 172.21.15.37:0/912332322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b30003a60 msgr2=0x7f0b30003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:15.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b48baa640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b30003a60 0x7f0b30003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.487+0000 7f0b48baa640 1 -- 172.21.15.37:0/912332322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b30003530 con 0x7f0b30004d50 2024-04-25T08:54:15.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b48baa640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b30004d50 0x7f0b300051d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c001750 tx=0x7f0b3c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c31f72117e9ade3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0013a0 con 0x7f0b30004d50 2024-04-25T08:54:15.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c0052f0 con 0x7f0b30004d50 2024-04-25T08:54:15.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b42ffd640 1 -- 172.21.15.37:0/912332322 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c003840 con 0x7f0b30004d50 2024-04-25T08:54:15.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b4a60d640 1 -- 172.21.15.37:0/912332322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b30004d50 msgr2=0x7f0b300051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b4a60d640 1 --2- 172.21.15.37:0/912332322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b30004d50 0x7f0b300051d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c001750 tx=0x7f0b3c01c3e0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b48baa640 1 -- 172.21.15.37:0/912332322 reap_dead start 2024-04-25T08:54:15.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b4a60d640 1 -- 172.21.15.37:0/912332322 shutdown_connections 2024-04-25T08:54:15.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b4a60d640 1 -- 172.21.15.37:0/912332322 >> 172.21.15.37:0/912332322 conn(0x7f0b4408b000 msgr2=0x7f0b440d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:15.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b4a60d640 1 -- 172.21.15.37:0/912332322 shutdown_connections 2024-04-25T08:54:15.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b4a60d640 1 -- 172.21.15.37:0/912332322 wait complete. 2024-04-25T08:54:15.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b4a60d640 1 Processor -- start 2024-04-25T08:54:15.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b4a60d640 1 -- start start 2024-04-25T08:54:15.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b4a60d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b30004d50 0x7f0b440b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b4a60d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b440b4f80 0x7f0b4415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.491+0000 7f0b4a60d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b4415bcf0 0x7f0b4415c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b4a60d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b440d26f0 con 0x7f0b4415bcf0 2024-04-25T08:54:15.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b4a60d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b440d20f0 con 0x7f0b440b4f80 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b4a60d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b440d2870 con 0x7f0b30004d50 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b48baa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b4415bcf0 0x7f0b4415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b48baa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b4415bcf0 0x7f0b4415c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39716/0 (socket says 172.21.15.37:39716) 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b48baa640 1 -- 172.21.15.37:0/769115859 learned_addr learned my addr 172.21.15.37:0/769115859 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b43fff640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b30004d50 0x7f0b440b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b437fe640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b440b4f80 0x7f0b4415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b48baa640 1 -- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b30004d50 msgr2=0x7f0b440b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b48baa640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b30004d50 0x7f0b440b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b48baa640 1 -- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b440b4f80 msgr2=0x7f0b4415afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b48baa640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b440b4f80 0x7f0b4415afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b48baa640 1 -- 172.21.15.37:0/769115859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b440b6d20 con 0x7f0b4415bcf0 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b48baa640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b4415bcf0 0x7f0b4415c170 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c01c8c0 tx=0x7f0b30005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b40ff9640 1 -- 172.21.15.37:0/769115859 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c007a40 con 0x7f0b4415bcf0 2024-04-25T08:54:15.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b40ff9640 1 -- 172.21.15.37:0/769115859 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c0bb070 con 0x7f0b4415bcf0 2024-04-25T08:54:15.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b4a60d640 1 -- 172.21.15.37:0/769115859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b440b7000 con 0x7f0b4415bcf0 2024-04-25T08:54:15.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b40ff9640 1 -- 172.21.15.37:0/769115859 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c0054a0 con 0x7f0b4415bcf0 2024-04-25T08:54:15.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b4a60d640 1 -- 172.21.15.37:0/769115859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b440b75c0 con 0x7f0b4415bcf0 2024-04-25T08:54:15.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b4a60d640 1 -- 172.21.15.37:0/769115859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4404efa0 con 0x7f0b4415bcf0 2024-04-25T08:54:15.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b40ff9640 1 -- 172.21.15.37:0/769115859 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b3c0b4040 con 0x7f0b4415bcf0 2024-04-25T08:54:15.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b40ff9640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b24066f60 0x7f0b24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b40ff9640 1 -- 172.21.15.37:0/769115859 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0b3c11f100 con 0x7f0b4415bcf0 2024-04-25T08:54:15.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.495+0000 7f0b43fff640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b24066f60 0x7f0b24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.499+0000 7f0b43fff640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b24066f60 0x7f0b24069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009b20 tx=0x7f0b34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.499+0000 7f0b40ff9640 1 -- 172.21.15.37:0/769115859 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b3c0160f0 con 0x7f0b4415bcf0 2024-04-25T08:54:15.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.695+0000 7f0b4a60d640 1 -- 172.21.15.37:0/769115859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b440ca210 con 0x7f0b4415bcf0 2024-04-25T08:54:15.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.695+0000 7f0b40ff9640 1 -- 172.21.15.37:0/769115859 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f0b3c0efff0 con 0x7f0b4415bcf0 2024-04-25T08:54:15.690 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:15.690 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3245776986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3245776986},{"type":"v1","addr":"172.21.15.203:6836","nonce":3245776986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5056,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/841808653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":841808653},{"type":"v1","addr":"172.21.15.37:6835","nonce":841808653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1908736119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1908736119},{"type":"v1","addr":"172.21.15.203:6837","nonce":1908736119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:53:55.212059+0000","modified":"2024-04-25T08:53:58.247733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035847732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035847732},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035847732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:54:15.690 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 -- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b24066f60 msgr2=0x7f0b24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b24066f60 0x7f0b24069420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0b34009b20 tx=0x7f0b34008040 comp rx=0 tx=0).stop 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 -- 172.21.15.37:0/769115859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b4415bcf0 msgr2=0x7f0b4415c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b4415bcf0 0x7f0b4415c170 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c01c8c0 tx=0x7f0b30005840 comp rx=0 tx=0).stop 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 -- 172.21.15.37:0/769115859 shutdown_connections 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b24066f60 0x7f0b24069420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b4415bcf0 0x7f0b4415c170 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b440b4f80 0x7f0b4415afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 --2- 172.21.15.37:0/769115859 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b30004d50 0x7f0b440b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 -- 172.21.15.37:0/769115859 >> 172.21.15.37:0/769115859 conn(0x7f0b4408b000 msgr2=0x7f0b4415e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 -- 172.21.15.37:0/769115859 shutdown_connections 2024-04-25T08:54:15.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:15.699+0000 7f0b4a60d640 1 -- 172.21.15.37:0/769115859 wait complete. 2024-04-25T08:54:15.710 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parentdir mytag 2024-04-25T08:54:15.853 INFO:teuthology.orchestra.run.smithi037.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-04-25T08:54:15.853 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-04-25T08:54:25.224 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:54:25.227 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:54:25.855 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-04-25T08:54:25.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.903+0000 7f5656116080 1 Processor -- start 2024-04-25T08:54:25.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.903+0000 7f5656116080 1 -- start start 2024-04-25T08:54:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56400022b0 0x55bc9bdd7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bddfba0 0x55bc9be5b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bc9bf05500 0x55bc9bf05910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bd3e9f0 con 0x55bc9bddfba0 2024-04-25T08:54:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bd42950 con 0x7f56400022b0 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bde1260 con 0x55bc9bf05500 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bddfba0 0x55bc9be5b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bddfba0 0x55bc9be5b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51968/0 (socket says 172.21.15.37:51968) 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 -- 172.21.15.37:0/1129679038 learned_addr learned my addr 172.21.15.37:0/1129679038 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653c34640 1 --2- 172.21.15.37:0/1129679038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56400022b0 0x55bc9bdd7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5654435640 1 --2- 172.21.15.37:0/1129679038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bc9bf05500 0x55bc9bf05910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 -- 172.21.15.37:0/1129679038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bc9bf05500 msgr2=0x55bc9bf05910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 --2- 172.21.15.37:0/1129679038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bc9bf05500 0x55bc9bf05910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 -- 172.21.15.37:0/1129679038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56400022b0 msgr2=0x55bc9bdd7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 --2- 172.21.15.37:0/1129679038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56400022b0 0x55bc9bdd7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 -- 172.21.15.37:0/1129679038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc9bd9b540 con 0x55bc9bddfba0 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653c34640 1 --2- 172.21.15.37:0/1129679038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56400022b0 0x55bc9bdd7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 --2- 172.21.15.37:0/1129679038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bddfba0 0x55bc9be5b6a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f563c009740 tx=0x7f563c005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf733c28af50fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5652431640 1 -- 172.21.15.37:0/1129679038 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c008bf0 con 0x55bc9bddfba0 2024-04-25T08:54:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5652431640 1 -- 172.21.15.37:0/1129679038 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c008d90 con 0x55bc9bddfba0 2024-04-25T08:54:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5652431640 1 -- 172.21.15.37:0/1129679038 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c0156a0 con 0x55bc9bddfba0 2024-04-25T08:54:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- 172.21.15.37:0/1129679038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bddfba0 msgr2=0x55bc9be5b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 --2- 172.21.15.37:0/1129679038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bddfba0 0x55bc9be5b6a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f563c009740 tx=0x7f563c005a10 comp rx=0 tx=0).stop 2024-04-25T08:54:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- 172.21.15.37:0/1129679038 shutdown_connections 2024-04-25T08:54:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 --2- 172.21.15.37:0/1129679038 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bc9bf05500 0x55bc9bf05910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 --2- 172.21.15.37:0/1129679038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bddfba0 0x55bc9be5b6a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 --2- 172.21.15.37:0/1129679038 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56400022b0 0x55bc9bdd7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- 172.21.15.37:0/1129679038 >> 172.21.15.37:0/1129679038 conn(0x55bc9bdc2ab0 msgr2=0x55bc9bdc5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- 172.21.15.37:0/1129679038 shutdown_connections 2024-04-25T08:54:25.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- 172.21.15.37:0/1129679038 wait complete. 2024-04-25T08:54:25.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 Processor -- start 2024-04-25T08:54:25.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- start start 2024-04-25T08:54:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bc9bddfba0 0x55bc9bd9afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bf05500 0x55bc9bdd93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56400022b0 0x55bc9bf07b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9be76730 con 0x55bc9bf05500 2024-04-25T08:54:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc9bd9b7e0 con 0x7f56400022b0 2024-04-25T08:54:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5656116080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bc9be769d0 con 0x55bc9bddfba0 2024-04-25T08:54:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bf05500 0x55bc9bdd93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bf05500 0x55bc9bdd93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51970/0 (socket says 172.21.15.37:51970) 2024-04-25T08:54:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.907+0000 7f5653433640 1 -- 172.21.15.37:0/1421840331 learned_addr learned my addr 172.21.15.37:0/1421840331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:25.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5653c34640 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bc9bddfba0 0x55bc9bd9afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5654435640 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56400022b0 0x55bc9bf07b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5653433640 1 -- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bc9bddfba0 msgr2=0x55bc9bd9afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5653433640 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bc9bddfba0 0x55bc9bd9afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5653433640 1 -- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56400022b0 msgr2=0x55bc9bf07b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5653433640 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56400022b0 0x55bc9bf07b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5653433640 1 -- 172.21.15.37:0/1421840331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc9bd9ac20 con 0x55bc9bf05500 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5654435640 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56400022b0 0x55bc9bf07b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5653433640 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bf05500 0x55bc9bdd93d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f563c005a50 tx=0x7f563c002fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f565142f640 1 -- 172.21.15.37:0/1421840331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c01f820 con 0x55bc9bf05500 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f565142f640 1 -- 172.21.15.37:0/1421840331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c01f9c0 con 0x55bc9bf05500 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5656116080 1 -- 172.21.15.37:0/1421840331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bc9bdd9a00 con 0x55bc9bf05500 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f565142f640 1 -- 172.21.15.37:0/1421840331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c01fdb0 con 0x55bc9bf05500 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5656116080 1 -- 172.21.15.37:0/1421840331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc9bdd9f40 con 0x55bc9bf05500 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f565142f640 1 -- 172.21.15.37:0/1421840331 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f563c013030 con 0x55bc9bf05500 2024-04-25T08:54:25.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f565142f640 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5634065970 0x7f5634067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f565142f640 1 -- 172.21.15.37:0/1421840331 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f563c086800 con 0x55bc9bf05500 2024-04-25T08:54:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5656116080 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55bc9bf080d0 0x55bc9bf114d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5656116080 1 -- 172.21.15.37:0/1421840331 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v9 -- 0x55bc9bddd000 con 0x55bc9bf080d0 2024-04-25T08:54:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5653c34640 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5634065970 0x7f5634067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.911+0000 7f5654435640 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55bc9bf080d0 0x55bc9bf114d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5653c34640 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5634065970 0x7f5634067e30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56440aa000 tx=0x55bc9bde0fa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5654435640 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55bc9bf080d0 0x55bc9bf114d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5654435640 1 -- 172.21.15.37:0/1421840331 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f564c011430 con 0x55bc9bf080d0 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5656116080 1 -- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55bc9bf080d0 msgr2=0x55bc9bf114d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5656116080 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55bc9bf080d0 0x55bc9bf114d0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5656116080 1 -- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5634065970 msgr2=0x7f5634067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5656116080 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5634065970 0x7f5634067e30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56440aa000 tx=0x55bc9bde0fa0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5656116080 1 -- 172.21.15.37:0/1421840331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bf05500 msgr2=0x55bc9bdd93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5656116080 1 --2- 172.21.15.37:0/1421840331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bc9bf05500 0x55bc9bdd93d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f563c005a50 tx=0x7f563c002fc0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5654435640 1 -- 172.21.15.37:0/1421840331 reap_dead start 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5656116080 1 -- 172.21.15.37:0/1421840331 shutdown_connections 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5656116080 1 -- 172.21.15.37:0/1421840331 >> 172.21.15.37:0/1421840331 conn(0x55bc9bdc2ab0 msgr2=0x55bc9bdd7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5656116080 1 -- 172.21.15.37:0/1421840331 shutdown_connections 2024-04-25T08:54:25.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:25.915+0000 7f5656116080 1 -- 172.21.15.37:0/1421840331 wait complete. 2024-04-25T08:54:25.911 INFO:teuthology.orchestra.run.smithi037.stdout:mytag 2024-04-25T08:54:25.911 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-04-25T08:54:25.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 Processor -- start 2024-04-25T08:54:25.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 -- start start 2024-04-25T08:54:26.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0022b0 0x55c5ec30d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5ec323c40 0x55c5ec4485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5ec324020 0x55c5ec448b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ec2829f0 con 0x7f4eec0022b0 2024-04-25T08:54:26.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ec286950 con 0x55c5ec324020 2024-04-25T08:54:26.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5ec3b3810 con 0x55c5ec323c40 2024-04-25T08:54:26.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0438a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0022b0 0x55c5ec30d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0438a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0022b0 0x55c5ec30d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51992/0 (socket says 172.21.15.37:51992) 2024-04-25T08:54:26.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0438a640 1 -- 172.21.15.37:0/1192493827 learned_addr learned my addr 172.21.15.37:0/1192493827 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:26.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f04b8b640 1 --2- 172.21.15.37:0/1192493827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5ec324020 0x55c5ec448b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f03b89640 1 --2- 172.21.15.37:0/1192493827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5ec323c40 0x55c5ec4485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0438a640 1 -- 172.21.15.37:0/1192493827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5ec323c40 msgr2=0x55c5ec4485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:26.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0438a640 1 --2- 172.21.15.37:0/1192493827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5ec323c40 0x55c5ec4485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0438a640 1 -- 172.21.15.37:0/1192493827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5ec324020 msgr2=0x55c5ec448b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0438a640 1 --2- 172.21.15.37:0/1192493827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5ec324020 0x55c5ec448b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0438a640 1 -- 172.21.15.37:0/1192493827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ec322680 con 0x7f4eec0022b0 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f03b89640 1 --2- 172.21.15.37:0/1192493827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5ec323c40 0x55c5ec4485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0438a640 1 --2- 172.21.15.37:0/1192493827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0022b0 0x55c5ec30d9a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0ab040 tx=0x55c5ec3292e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a048f920ac96567f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f02b87640 1 -- 172.21.15.37:0/1192493827 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc0ad070 con 0x7f4eec0022b0 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f02b87640 1 -- 172.21.15.37:0/1192493827 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4efc0ae040 con 0x7f4eec0022b0 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f02b87640 1 -- 172.21.15.37:0/1192493827 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc003460 con 0x7f4eec0022b0 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 -- 172.21.15.37:0/1192493827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0022b0 msgr2=0x55c5ec30d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 --2- 172.21.15.37:0/1192493827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0022b0 0x55c5ec30d9a0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0ab040 tx=0x55c5ec3292e0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 -- 172.21.15.37:0/1192493827 shutdown_connections 2024-04-25T08:54:26.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 --2- 172.21.15.37:0/1192493827 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5ec324020 0x55c5ec448b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 --2- 172.21.15.37:0/1192493827 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5ec323c40 0x55c5ec4485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 --2- 172.21.15.37:0/1192493827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec0022b0 0x55c5ec30d9a0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 -- 172.21.15.37:0/1192493827 >> 172.21.15.37:0/1192493827 conn(0x55c5ec2dc0f0 msgr2=0x55c5ec30d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 -- 172.21.15.37:0/1192493827 shutdown_connections 2024-04-25T08:54:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.003+0000 7f4f0686c080 1 -- 172.21.15.37:0/1192493827 wait complete. 2024-04-25T08:54:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0686c080 1 Processor -- start 2024-04-25T08:54:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0686c080 1 -- start start 2024-04-25T08:54:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0686c080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5ec323c40 0x55c5ec3ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0686c080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5ec324020 0x55c5ec3ab780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0686c080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec0022b0 0x55c5ec3abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0686c080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ec322bc0 con 0x55c5ec324020 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0686c080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5ec322920 con 0x7f4eec0022b0 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0686c080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5ec322e60 con 0x55c5ec323c40 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f03b89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5ec324020 0x55c5ec3ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f03b89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5ec324020 0x55c5ec3ab780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52002/0 (socket says 172.21.15.37:52002) 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f03b89640 1 -- 172.21.15.37:0/1787331925 learned_addr learned my addr 172.21.15.37:0/1787331925 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f04b8b640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec0022b0 0x55c5ec3abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0438a640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5ec323c40 0x55c5ec3ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f03b89640 1 -- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5ec323c40 msgr2=0x55c5ec3ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f03b89640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5ec323c40 0x55c5ec3ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f03b89640 1 -- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec0022b0 msgr2=0x55c5ec3abcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f03b89640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec0022b0 0x55c5ec3abcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f03b89640 1 -- 172.21.15.37:0/1787331925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5ec3b20a0 con 0x55c5ec324020 2024-04-25T08:54:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0438a640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5ec323c40 0x55c5ec3ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f04b8b640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec0022b0 0x55c5ec3abcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f03b89640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5ec324020 0x55c5ec3ab780 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8007730 tx=0x7f4ef8007bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f01b85640 1 -- 172.21.15.37:0/1787331925 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef801b070 con 0x55c5ec324020 2024-04-25T08:54:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f01b85640 1 -- 172.21.15.37:0/1787331925 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef8006d20 con 0x55c5ec324020 2024-04-25T08:54:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f01b85640 1 -- 172.21.15.37:0/1787331925 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80166b0 con 0x55c5ec324020 2024-04-25T08:54:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0686c080 1 -- 172.21.15.37:0/1787331925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c5ec3b2340 con 0x55c5ec324020 2024-04-25T08:54:26.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.007+0000 7f4f0686c080 1 -- 172.21.15.37:0/1787331925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5ec43fd70 con 0x55c5ec324020 2024-04-25T08:54:26.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f01b85640 1 -- 172.21.15.37:0/1787331925 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ef801e070 con 0x55c5ec324020 2024-04-25T08:54:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f01b85640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ef0065970 0x7f4ef0067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f01b85640 1 -- 172.21.15.37:0/1787331925 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4ef8084620 con 0x55c5ec324020 2024-04-25T08:54:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55c5ec3adb90 0x55c5ec440210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 -- 172.21.15.37:0/1787331925 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:1 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v9 -- 0x55c5ec440750 con 0x55c5ec3adb90 2024-04-25T08:54:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0438a640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ef0065970 0x7f4ef0067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f04b8b640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55c5ec3adb90 0x55c5ec440210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0438a640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ef0065970 0x7f4ef0067e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0b4000 tx=0x7f4efc006000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f04b8b640 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55c5ec3adb90 0x55c5ec440210 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f04b8b640 1 -- 172.21.15.37:0/1787331925 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f4ef40037f0 con 0x55c5ec3adb90 2024-04-25T08:54:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 -- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55c5ec3adb90 msgr2=0x55c5ec440210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55c5ec3adb90 0x55c5ec440210 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 -- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ef0065970 msgr2=0x7f4ef0067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ef0065970 0x7f4ef0067e30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4efc0b4000 tx=0x7f4efc006000 comp rx=0 tx=0).stop 2024-04-25T08:54:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 -- 172.21.15.37:0/1787331925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5ec324020 msgr2=0x55c5ec3ab780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 --2- 172.21.15.37:0/1787331925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5ec324020 0x55c5ec3ab780 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8007730 tx=0x7f4ef8007bf0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f04b8b640 1 -- 172.21.15.37:0/1787331925 reap_dead start 2024-04-25T08:54:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 -- 172.21.15.37:0/1787331925 shutdown_connections 2024-04-25T08:54:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 -- 172.21.15.37:0/1787331925 >> 172.21.15.37:0/1787331925 conn(0x55c5ec2dc0f0 msgr2=0x55c5ec325cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 -- 172.21.15.37:0/1787331925 shutdown_connections 2024-04-25T08:54:26.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.011+0000 7f4f0686c080 1 -- 172.21.15.37:0/1787331925 wait complete. 2024-04-25T08:54:26.012 INFO:teuthology.orchestra.run.smithi037.stdout:mytag 2024-04-25T08:54:26.012 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-25T08:54:26.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.099+0000 7fe3ca42c080 1 Processor -- start 2024-04-25T08:54:26.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.099+0000 7fe3ca42c080 1 -- start start 2024-04-25T08:54:26.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.099+0000 7fe3ca42c080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0003640 0x5629d1b49f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.099+0000 7fe3ca42c080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629d1c1d450 0x5629d1b4d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5629d1c09780 0x5629d1b8b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d1b7ab50 con 0x7fe3b0003640 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d1af19f0 con 0x5629d1c1d450 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629d1b7adf0 con 0x5629d1c09780 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7f4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0003640 0x5629d1b49f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7f4a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0003640 0x5629d1b49f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52026/0 (socket says 172.21.15.37:52026) 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7f4a640 1 -- 172.21.15.37:0/331927101 learned_addr learned my addr 172.21.15.37:0/331927101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7749640 1 --2- 172.21.15.37:0/331927101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629d1c1d450 0x5629d1b4d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c874b640 1 --2- 172.21.15.37:0/331927101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5629d1c09780 0x5629d1b8b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7f4a640 1 -- 172.21.15.37:0/331927101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5629d1c09780 msgr2=0x5629d1b8b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7f4a640 1 --2- 172.21.15.37:0/331927101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5629d1c09780 0x5629d1b8b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7f4a640 1 -- 172.21.15.37:0/331927101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629d1c1d450 msgr2=0x5629d1b4d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:26.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7f4a640 1 --2- 172.21.15.37:0/331927101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629d1c1d450 0x5629d1b4d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7f4a640 1 -- 172.21.15.37:0/331927101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629d1b4cb80 con 0x7fe3b0003640 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7f4a640 1 --2- 172.21.15.37:0/331927101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0003640 0x5629d1b49f70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009840 tx=0x7fe3b8005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa718c2bff90dbbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c6747640 1 -- 172.21.15.37:0/331927101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8008e00 con 0x7fe3b0003640 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c6747640 1 -- 172.21.15.37:0/331927101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b801a070 con 0x7fe3b0003640 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c6747640 1 -- 172.21.15.37:0/331927101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8015660 con 0x7fe3b0003640 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- 172.21.15.37:0/331927101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0003640 msgr2=0x5629d1b49f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 --2- 172.21.15.37:0/331927101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0003640 0x5629d1b49f70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009840 tx=0x7fe3b8005a10 comp rx=0 tx=0).stop 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- 172.21.15.37:0/331927101 shutdown_connections 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 --2- 172.21.15.37:0/331927101 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5629d1c09780 0x5629d1b8b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 --2- 172.21.15.37:0/331927101 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629d1c1d450 0x5629d1b4d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 --2- 172.21.15.37:0/331927101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe3b0003640 0x5629d1b49f70 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- 172.21.15.37:0/331927101 >> 172.21.15.37:0/331927101 conn(0x5629d1b75ab0 msgr2=0x5629d1b794c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- 172.21.15.37:0/331927101 shutdown_connections 2024-04-25T08:54:26.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- 172.21.15.37:0/331927101 wait complete. 2024-04-25T08:54:26.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 Processor -- start 2024-04-25T08:54:26.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- start start 2024-04-25T08:54:26.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629d1c09780 0x5629d1cb9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629d1c1d450 0x5629d1c048c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b0003640 0x5629d1c04e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d1b4ce20 con 0x5629d1c1d450 2024-04-25T08:54:26.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629d1b4a4b0 con 0x5629d1c09780 2024-04-25T08:54:26.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3ca42c080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5629d1b4d0c0 con 0x7fe3b0003640 2024-04-25T08:54:26.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7749640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629d1c1d450 0x5629d1c048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7749640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629d1c1d450 0x5629d1c048c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52042/0 (socket says 172.21.15.37:52042) 2024-04-25T08:54:26.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7749640 1 -- 172.21.15.37:0/3355950872 learned_addr learned my addr 172.21.15.37:0/3355950872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7f4a640 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629d1c09780 0x5629d1cb9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c874b640 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b0003640 0x5629d1c04e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7749640 1 -- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b0003640 msgr2=0x5629d1c04e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7749640 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b0003640 0x5629d1c04e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7749640 1 -- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629d1c09780 msgr2=0x5629d1cb9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7749640 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5629d1c09780 0x5629d1cb9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7749640 1 -- 172.21.15.37:0/3355950872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629d1c05340 con 0x5629d1c1d450 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c874b640 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe3b0003640 0x5629d1c04e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.103+0000 7fe3c7749640 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629d1c1d450 0x5629d1c048c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc007760 tx=0x7fe3bc007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3c5745640 1 -- 172.21.15.37:0/3355950872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc01b070 con 0x5629d1c1d450 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3c5745640 1 -- 172.21.15.37:0/3355950872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3bc006d80 con 0x5629d1c1d450 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3c5745640 1 -- 172.21.15.37:0/3355950872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc0166c0 con 0x5629d1c1d450 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3ca42c080 1 -- 172.21.15.37:0/3355950872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5629d1c055d0 con 0x5629d1c1d450 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3ca42c080 1 -- 172.21.15.37:0/3355950872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5629d1c05b10 con 0x5629d1c1d450 2024-04-25T08:54:26.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3c5745640 1 -- 172.21.15.37:0/3355950872 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe3bc01e070 con 0x5629d1c1d450 2024-04-25T08:54:26.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3c5745640 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe3b4065990 0x7fe3b4067e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3c5745640 1 -- 172.21.15.37:0/3355950872 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fe3bc084360 con 0x5629d1c1d450 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3ca42c080 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x5629d1b6e150 0x5629d1b8fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3ca42c080 1 -- 172.21.15.37:0/3355950872 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:1 6.36 6:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v9 -- 0x5629d1b90250 con 0x5629d1b6e150 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3c7f4a640 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe3b4065990 0x7fe3b4067e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3c874b640 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x5629d1b6e150 0x5629d1b8fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.107+0000 7fe3c7f4a640 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe3b4065990 0x7fe3b4067e50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8017040 tx=0x7fe3b8014000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3c874b640 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x5629d1b6e150 0x5629d1b8fd10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3c874b640 1 -- 172.21.15.37:0/3355950872 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fe3c00017c0 con 0x5629d1b6e150 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3ca42c080 1 -- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x5629d1b6e150 msgr2=0x5629d1b8fd10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3ca42c080 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x5629d1b6e150 0x5629d1b8fd10 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3ca42c080 1 -- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe3b4065990 msgr2=0x7fe3b4067e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3ca42c080 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe3b4065990 0x7fe3b4067e50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8017040 tx=0x7fe3b8014000 comp rx=0 tx=0).stop 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3ca42c080 1 -- 172.21.15.37:0/3355950872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629d1c1d450 msgr2=0x5629d1c048c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3ca42c080 1 --2- 172.21.15.37:0/3355950872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5629d1c1d450 0x5629d1c048c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc007760 tx=0x7fe3bc007c20 comp rx=0 tx=0).stop 2024-04-25T08:54:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3c874b640 1 -- 172.21.15.37:0/3355950872 reap_dead start 2024-04-25T08:54:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3ca42c080 1 -- 172.21.15.37:0/3355950872 shutdown_connections 2024-04-25T08:54:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3ca42c080 1 -- 172.21.15.37:0/3355950872 >> 172.21.15.37:0/3355950872 conn(0x5629d1b75ab0 msgr2=0x5629d1b492c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3ca42c080 1 -- 172.21.15.37:0/3355950872 shutdown_connections 2024-04-25T08:54:26.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.111+0000 7fe3ca42c080 1 -- 172.21.15.37:0/3355950872 wait complete. 2024-04-25T08:54:26.107 INFO:teuthology.orchestra.run.smithi037.stdout:mytag 2024-04-25T08:54:26.107 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-25T08:54:26.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.195+0000 7f4659f21080 1 Processor -- start 2024-04-25T08:54:26.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.195+0000 7f4659f21080 1 -- start start 2024-04-25T08:54:26.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4644003100 0x557be5d04b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557be5dd4ed0 0x557be5dd4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557be5d4ac80 0x557be5dd4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be5dd9a50 con 0x557be5dd4ed0 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be5ca99f0 con 0x7f4644003100 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557be5dd9bd0 con 0x557be5d4ac80 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465723e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557be5dd4ed0 0x557be5dd4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465723e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557be5dd4ed0 0x557be5dd4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52056/0 (socket says 172.21.15.37:52056) 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465723e640 1 -- 172.21.15.37:0/3857507003 learned_addr learned my addr 172.21.15.37:0/3857507003 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4658240640 1 --2- 172.21.15.37:0/3857507003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557be5d4ac80 0x557be5dd4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4657a3f640 1 --2- 172.21.15.37:0/3857507003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4644003100 0x557be5d04b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465723e640 1 -- 172.21.15.37:0/3857507003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557be5d4ac80 msgr2=0x557be5dd4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465723e640 1 --2- 172.21.15.37:0/3857507003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557be5d4ac80 0x557be5dd4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465723e640 1 -- 172.21.15.37:0/3857507003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4644003100 msgr2=0x557be5d04b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465723e640 1 --2- 172.21.15.37:0/3857507003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4644003100 0x557be5d04b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465723e640 1 -- 172.21.15.37:0/3857507003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557be5d4b190 con 0x557be5dd4ed0 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465723e640 1 --2- 172.21.15.37:0/3857507003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557be5dd4ed0 0x557be5dd4010 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f464001c200 tx=0x557be5d434e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4476d09d971a32da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465623c640 1 -- 172.21.15.37:0/3857507003 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4640001700 con 0x557be5dd4ed0 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465623c640 1 -- 172.21.15.37:0/3857507003 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46400bb070 con 0x557be5dd4ed0 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f465623c640 1 -- 172.21.15.37:0/3857507003 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46400a49f0 con 0x557be5dd4ed0 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- 172.21.15.37:0/3857507003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557be5dd4ed0 msgr2=0x557be5dd4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 --2- 172.21.15.37:0/3857507003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557be5dd4ed0 0x557be5dd4010 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f464001c200 tx=0x557be5d434e0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- 172.21.15.37:0/3857507003 shutdown_connections 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 --2- 172.21.15.37:0/3857507003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557be5d4ac80 0x557be5dd4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 --2- 172.21.15.37:0/3857507003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557be5dd4ed0 0x557be5dd4010 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 --2- 172.21.15.37:0/3857507003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4644003100 0x557be5d04b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- 172.21.15.37:0/3857507003 >> 172.21.15.37:0/3857507003 conn(0x557be5d2dab0 msgr2=0x557be5ddc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- 172.21.15.37:0/3857507003 shutdown_connections 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- 172.21.15.37:0/3857507003 wait complete. 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 Processor -- start 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- start start 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557be5d4ac80 0x557be5dd1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557be5dd4ed0 0x557be5dd0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4644003100 0x557be5d25060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be5d05fe0 con 0x7f4644003100 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557be5d05d40 con 0x557be5d4ac80 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.199+0000 7f4659f21080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557be5d06280 con 0x557be5dd4ed0 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4658240640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4644003100 0x557be5d25060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4658240640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4644003100 0x557be5d25060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52072/0 (socket says 172.21.15.37:52072) 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4658240640 1 -- 172.21.15.37:0/1678745634 learned_addr learned my addr 172.21.15.37:0/1678745634 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f465723e640 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557be5dd4ed0 0x557be5dd0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4657a3f640 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557be5d4ac80 0x557be5dd1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4658240640 1 -- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557be5dd4ed0 msgr2=0x557be5dd0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4658240640 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557be5dd4ed0 0x557be5dd0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4658240640 1 -- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557be5d4ac80 msgr2=0x557be5dd1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4658240640 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557be5d4ac80 0x557be5dd1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4658240640 1 -- 172.21.15.37:0/1678745634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557be5d06a80 con 0x7f4644003100 2024-04-25T08:54:26.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4658240640 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4644003100 0x557be5d25060 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4650012570 tx=0x7f4650012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f465523a640 1 -- 172.21.15.37:0/1678745634 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650018070 con 0x7f4644003100 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f465523a640 1 -- 172.21.15.37:0/1678745634 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46500147b0 con 0x7f4644003100 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4659f21080 1 -- 172.21.15.37:0/1678745634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557be5dd1f40 con 0x7f4644003100 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f465523a640 1 -- 172.21.15.37:0/1678745634 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650014b60 con 0x7f4644003100 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4659f21080 1 -- 172.21.15.37:0/1678745634 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557be5dd2480 con 0x7f4644003100 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f465523a640 1 -- 172.21.15.37:0/1678745634 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4650011040 con 0x7f4644003100 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f465523a640 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4638065940 0x7f4638067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f465523a640 1 -- 172.21.15.37:0/1678745634 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f4650084730 con 0x7f4644003100 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4659f21080 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x557be5e73e20 0x557be5e74200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4657a3f640 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4638065940 0x7f4638067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f4659f21080 1 -- 172.21.15.37:0/1678745634 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:1 6.19 6:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v9 -- 0x557be5e74740 con 0x557be5e73e20 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.203+0000 7f465723e640 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x557be5e73e20 0x557be5e74200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4657a3f640 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4638065940 0x7f4638067e00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4648005fd0 tx=0x7f464800c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f465723e640 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x557be5e73e20 0x557be5e74200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f465723e640 1 -- 172.21.15.37:0/1678745634 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f4640008030 con 0x557be5e73e20 2024-04-25T08:54:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4659f21080 1 -- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x557be5e73e20 msgr2=0x557be5e74200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4659f21080 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x557be5e73e20 0x557be5e74200 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4659f21080 1 -- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4638065940 msgr2=0x7f4638067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4659f21080 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4638065940 0x7f4638067e00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4648005fd0 tx=0x7f464800c040 comp rx=0 tx=0).stop 2024-04-25T08:54:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4659f21080 1 -- 172.21.15.37:0/1678745634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4644003100 msgr2=0x557be5d25060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4659f21080 1 --2- 172.21.15.37:0/1678745634 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4644003100 0x557be5d25060 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4650012570 tx=0x7f4650012a30 comp rx=0 tx=0).stop 2024-04-25T08:54:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4658240640 1 -- 172.21.15.37:0/1678745634 reap_dead start 2024-04-25T08:54:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4659f21080 1 -- 172.21.15.37:0/1678745634 shutdown_connections 2024-04-25T08:54:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4659f21080 1 -- 172.21.15.37:0/1678745634 >> 172.21.15.37:0/1678745634 conn(0x557be5d2dab0 msgr2=0x557be5d012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4659f21080 1 -- 172.21.15.37:0/1678745634 shutdown_connections 2024-04-25T08:54:26.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.207+0000 7f4659f21080 1 -- 172.21.15.37:0/1678745634 wait complete. 2024-04-25T08:54:26.206 INFO:teuthology.orchestra.run.smithi037.stdout:mytag 2024-04-25T08:54:26.206 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T08:54:26.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.291+0000 7fc221cf5080 1 Processor -- start 2024-04-25T08:54:26.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.291+0000 7fc221cf5080 1 -- start start 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581fdf79110 0x5581fdf7c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5581fdf794f0 0x5581fdf942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fe0222a0 0x5581fdf94810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fe024880 con 0x5581fe0222a0 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fdef39f0 con 0x5581fdf79110 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581fe024a00 con 0x5581fdf794f0 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc220014640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fe0222a0 0x5581fdf94810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc220014640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fe0222a0 0x5581fdf94810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52080/0 (socket says 172.21.15.37:52080) 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc220014640 1 -- 172.21.15.37:0/4233175113 learned_addr learned my addr 172.21.15.37:0/4233175113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc21f012640 1 --2- 172.21.15.37:0/4233175113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5581fdf794f0 0x5581fdf942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc21f813640 1 --2- 172.21.15.37:0/4233175113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581fdf79110 0x5581fdf7c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc220014640 1 -- 172.21.15.37:0/4233175113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5581fdf794f0 msgr2=0x5581fdf942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:26.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc220014640 1 --2- 172.21.15.37:0/4233175113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5581fdf794f0 0x5581fdf942d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc220014640 1 -- 172.21.15.37:0/4233175113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581fdf79110 msgr2=0x5581fdf7c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:26.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc220014640 1 --2- 172.21.15.37:0/4233175113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581fdf79110 0x5581fdf7c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc220014640 1 -- 172.21.15.37:0/4233175113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581fe0220b0 con 0x5581fe0222a0 2024-04-25T08:54:26.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc21f813640 1 --2- 172.21.15.37:0/4233175113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581fdf79110 0x5581fdf7c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:26.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc220014640 1 --2- 172.21.15.37:0/4233175113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fe0222a0 0x5581fdf94810 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc2180098a0 tx=0x7fc218005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca79f600f02d271 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc21e010640 1 -- 172.21.15.37:0/4233175113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218019070 con 0x5581fe0222a0 2024-04-25T08:54:26.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc21e010640 1 -- 172.21.15.37:0/4233175113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc218008da0 con 0x5581fe0222a0 2024-04-25T08:54:26.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc21e010640 1 -- 172.21.15.37:0/4233175113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180146a0 con 0x5581fe0222a0 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- 172.21.15.37:0/4233175113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fe0222a0 msgr2=0x5581fdf94810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 --2- 172.21.15.37:0/4233175113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fe0222a0 0x5581fdf94810 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fc2180098a0 tx=0x7fc218005ad0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- 172.21.15.37:0/4233175113 shutdown_connections 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 --2- 172.21.15.37:0/4233175113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fe0222a0 0x5581fdf94810 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 --2- 172.21.15.37:0/4233175113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5581fdf794f0 0x5581fdf942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 --2- 172.21.15.37:0/4233175113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581fdf79110 0x5581fdf7c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- 172.21.15.37:0/4233175113 >> 172.21.15.37:0/4233175113 conn(0x5581fdf77a80 msgr2=0x5581fdf4c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- 172.21.15.37:0/4233175113 shutdown_connections 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- 172.21.15.37:0/4233175113 wait complete. 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 Processor -- start 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- start start 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fdf79110 0x5581fdf503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5581fdf794f0 0x5581fdf909d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581fe0222a0 0x5581fe012330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fe021430 con 0x5581fdf79110 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581fe021190 con 0x5581fe0222a0 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.295+0000 7fc221cf5080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581fe0216d0 con 0x5581fdf794f0 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f813640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fdf79110 0x5581fdf503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f813640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fdf79110 0x5581fdf503e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52086/0 (socket says 172.21.15.37:52086) 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f813640 1 -- 172.21.15.37:0/3944187785 learned_addr learned my addr 172.21.15.37:0/3944187785 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f012640 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5581fdf794f0 0x5581fdf909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc220014640 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581fe0222a0 0x5581fe012330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f813640 1 -- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5581fdf794f0 msgr2=0x5581fdf909d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f813640 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5581fdf794f0 0x5581fdf909d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f813640 1 -- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581fe0222a0 msgr2=0x5581fe012330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f813640 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5581fe0222a0 0x5581fe012330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f813640 1 -- 172.21.15.37:0/3944187785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581fdf91120 con 0x5581fdf79110 2024-04-25T08:54:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f012640 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5581fdf794f0 0x5581fdf909d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f813640 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fdf79110 0x5581fdf503e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc20800cef0 tx=0x7fc20800a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21d00e640 1 -- 172.21.15.37:0/3944187785 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800f430 con 0x5581fdf79110 2024-04-25T08:54:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21d00e640 1 -- 172.21.15.37:0/3944187785 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208014020 con 0x5581fdf79110 2024-04-25T08:54:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc221cf5080 1 -- 172.21.15.37:0/3944187785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5581fdf913c0 con 0x5581fdf79110 2024-04-25T08:54:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21d00e640 1 -- 172.21.15.37:0/3944187785 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800f7d0 con 0x5581fdf79110 2024-04-25T08:54:26.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc221cf5080 1 -- 172.21.15.37:0/3944187785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581fdf91900 con 0x5581fdf79110 2024-04-25T08:54:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21d00e640 1 -- 172.21.15.37:0/3944187785 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc20800f970 con 0x5581fdf79110 2024-04-25T08:54:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21d00e640 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc200065970 0x7fc200067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21d00e640 1 -- 172.21.15.37:0/3944187785 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc208084370 con 0x5581fdf79110 2024-04-25T08:54:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc221cf5080 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5581fe012870 0x5581fe0bd000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc221cf5080 1 -- 172.21.15.37:0/3944187785 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:1 6.12 6:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) v9 -- 0x5581fe0bd540 con 0x5581fe012870 2024-04-25T08:54:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc21f012640 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc200065970 0x7fc200067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.299+0000 7fc220014640 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5581fe012870 0x5581fe0bd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc21f012640 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc200065970 0x7fc200067e30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc2100ad000 tx=0x7fc2100a5070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc220014640 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5581fe012870 0x5581fe0bd000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc220014640 1 -- 172.21.15.37:0/3944187785 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7fc218015490 con 0x5581fe012870 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc221cf5080 1 -- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5581fe012870 msgr2=0x5581fe0bd000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc221cf5080 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5581fe012870 0x5581fe0bd000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc221cf5080 1 -- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc200065970 msgr2=0x7fc200067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc221cf5080 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc200065970 0x7fc200067e30 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc2100ad000 tx=0x7fc2100a5070 comp rx=0 tx=0).stop 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc221cf5080 1 -- 172.21.15.37:0/3944187785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fdf79110 msgr2=0x5581fdf503e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc221cf5080 1 --2- 172.21.15.37:0/3944187785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581fdf79110 0x5581fdf503e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fc20800cef0 tx=0x7fc20800a600 comp rx=0 tx=0).stop 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc220014640 1 -- 172.21.15.37:0/3944187785 reap_dead start 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc221cf5080 1 -- 172.21.15.37:0/3944187785 shutdown_connections 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc221cf5080 1 -- 172.21.15.37:0/3944187785 >> 172.21.15.37:0/3944187785 conn(0x5581fdf77a80 msgr2=0x5581fdf93750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc221cf5080 1 -- 172.21.15.37:0/3944187785 shutdown_connections 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.303+0000 7fc221cf5080 1 -- 172.21.15.37:0/3944187785 wait complete. 2024-04-25T08:54:26.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:54:26.300 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:54:26.300 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:54:26.389 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:54:26.446 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:54:26.446 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:54:26.446 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:26.452 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:54:26.452 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:54:26.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:26.452 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:54:26.499 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T08:54:26.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:54:26.501 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:54:26.501 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:26.507 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:54:26.507 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:54:26.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:26.507 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:54:26.556 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:54:26.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:54:26.557 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:54:26.557 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:26.563 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:54:26.563 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:54:26.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:26.564 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:54:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:54:26.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:54:26.614 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:54:26.614 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:26.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:54:26.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:54:26.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:26.620 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:54:26.667 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:54:26.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:54:26.669 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.811+0000 7f0e18520640 1 Processor -- start 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.811+0000 7f0e18520640 1 -- start start 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.811+0000 7f0e18520640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0e1015b950 0x7f0e1009a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.811+0000 7f0e18520640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0e100cf420 0x7f0e100cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.811+0000 7f0e18520640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1015bd30 con 0x7f0e100cfb80 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.811+0000 7f0e18520640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e100cc0d0 con 0x7f0e100cf420 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.811+0000 7f0e18520640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1009ab10 con 0x7f0e1015b950 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.811+0000 7f0e15abb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0e100cf420 0x7f0e100cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.811+0000 7f0e16abd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0e100cfb80 0x7f0e100cff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52554/0 (socket says 172.21.15.37:52554) 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.811+0000 7f0e16abd640 1 -- 172.21.15.37:0/1846535884 learned_addr learned my addr 172.21.15.37:0/1846535884 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e162bc640 1 --2- 172.21.15.37:0/1846535884 >> v2:172.21.15.203:3300/0 conn(0x7f0e1015b950 0x7f0e1009a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e15abb640 1 -- 172.21.15.37:0/1846535884 >> v2:172.21.15.203:3300/0 conn(0x7f0e1015b950 msgr2=0x7f0e1009a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:26.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e15abb640 1 --2- 172.21.15.37:0/1846535884 >> v2:172.21.15.203:3300/0 conn(0x7f0e1015b950 0x7f0e1009a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e15abb640 1 -- 172.21.15.37:0/1846535884 >> v1:172.21.15.37:6789/0 conn(0x7f0e100cfb80 legacy=0x7f0e100cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e15abb640 1 -- 172.21.15.37:0/1846535884 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e100cc370 con 0x7f0e100cf420 2024-04-25T08:54:26.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e162bc640 1 --2- 172.21.15.37:0/1846535884 >> v2:172.21.15.203:3300/0 conn(0x7f0e1015b950 0x7f0e1009a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:26.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e15abb640 1 --2- 172.21.15.37:0/1846535884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e100cf420 0x7f0e100cce50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0e10072480 tx=0x7f0e0401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f89d01b238e4f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e152ba640 1 -- 172.21.15.37:0/1846535884 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040b0070 con 0x7f0e100cf420 2024-04-25T08:54:26.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e152ba640 1 -- 172.21.15.37:0/1846535884 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e040ab600 con 0x7f0e100cf420 2024-04-25T08:54:26.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e152ba640 1 -- 172.21.15.37:0/1846535884 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e040ab9d0 con 0x7f0e100cf420 2024-04-25T08:54:26.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e18520640 1 -- 172.21.15.37:0/1846535884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e100cf420 msgr2=0x7f0e100cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e18520640 1 --2- 172.21.15.37:0/1846535884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e100cf420 0x7f0e100cce50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0e10072480 tx=0x7f0e0401c310 comp rx=0 tx=0).stop 2024-04-25T08:54:26.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e18520640 1 -- 172.21.15.37:0/1846535884 shutdown_connections 2024-04-25T08:54:26.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e18520640 1 --2- 172.21.15.37:0/1846535884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e100cf420 0x7f0e100cce50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e18520640 1 --2- 172.21.15.37:0/1846535884 >> v2:172.21.15.203:3300/0 conn(0x7f0e1015b950 0x7f0e1009a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e18520640 1 -- 172.21.15.37:0/1846535884 >> 172.21.15.37:0/1846535884 conn(0x7f0e100b30b0 msgr2=0x7f0e100b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:26.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e18520640 1 -- 172.21.15.37:0/1846535884 shutdown_connections 2024-04-25T08:54:26.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.815+0000 7f0e18520640 1 -- 172.21.15.37:0/1846535884 wait complete. 2024-04-25T08:54:26.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e18520640 1 Processor -- start 2024-04-25T08:54:26.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e18520640 1 -- start start 2024-04-25T08:54:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e18520640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e100cf420 0x7f0e100af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e18520640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100cfb80 0x7f0e100af880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e18520640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e1015b950 0x7f0e1006d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e18520640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100cf8b0 con 0x7f0e100cf420 2024-04-25T08:54:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e18520640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100cc930 con 0x7f0e100cfb80 2024-04-25T08:54:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e18520640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e100cd390 con 0x7f0e1015b950 2024-04-25T08:54:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e162bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e100cf420 0x7f0e100af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e162bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e100cf420 0x7f0e100af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52106/0 (socket says 172.21.15.37:52106) 2024-04-25T08:54:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e162bc640 1 -- 172.21.15.37:0/135935420 learned_addr learned my addr 172.21.15.37:0/135935420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:26.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e15abb640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100cfb80 0x7f0e100af880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e16abd640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e1015b950 0x7f0e1006d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e162bc640 1 -- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e1015b950 msgr2=0x7f0e1006d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e162bc640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e1015b950 0x7f0e1006d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e162bc640 1 -- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100cfb80 msgr2=0x7f0e100af880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:26.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e162bc640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100cfb80 0x7f0e100af880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:26.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e162bc640 1 -- 172.21.15.37:0/135935420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e100b4330 con 0x7f0e100cf420 2024-04-25T08:54:26.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e16abd640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e1015b950 0x7f0e1006d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:26.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e15abb640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100cfb80 0x7f0e100af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:26.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e162bc640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e100cf420 0x7f0e100af340 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800cf60 tx=0x7f0e0800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e0effd640 1 -- 172.21.15.37:0/135935420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0801a070 con 0x7f0e100cf420 2024-04-25T08:54:26.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e0effd640 1 -- 172.21.15.37:0/135935420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0800d840 con 0x7f0e100cf420 2024-04-25T08:54:26.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e18520640 1 -- 172.21.15.37:0/135935420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e100b4610 con 0x7f0e100cf420 2024-04-25T08:54:26.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e0effd640 1 -- 172.21.15.37:0/135935420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0800dc10 con 0x7f0e100cf420 2024-04-25T08:54:26.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e18520640 1 -- 172.21.15.37:0/135935420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e100b4bd0 con 0x7f0e100cf420 2024-04-25T08:54:26.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.819+0000 7f0e18520640 1 -- 172.21.15.37:0/135935420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1004efa0 con 0x7f0e100cf420 2024-04-25T08:54:26.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.823+0000 7f0e0effd640 1 -- 172.21.15.37:0/135935420 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e0801d070 con 0x7f0e100cf420 2024-04-25T08:54:26.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.823+0000 7f0e0effd640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dfc066f60 0x7f0dfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:26.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.823+0000 7f0e0effd640 1 -- 172.21.15.37:0/135935420 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0e08085930 con 0x7f0e100cf420 2024-04-25T08:54:26.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.823+0000 7f0e15abb640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dfc066f60 0x7f0dfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:26.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.823+0000 7f0e15abb640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dfc066f60 0x7f0dfc069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0e040ad000 tx=0x7f0e04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:26.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:26.827+0000 7f0e0effd640 1 -- 172.21.15.37:0/135935420 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e080561d0 con 0x7f0e100cf420 2024-04-25T08:54:27.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.019+0000 7f0e18520640 1 -- 172.21.15.37:0/135935420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e100d13a0 con 0x7f0e100cf420 2024-04-25T08:54:27.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.019+0000 7f0e0effd640 1 -- 172.21.15.37:0/135935420 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f0e0805a270 con 0x7f0e100cf420 2024-04-25T08:54:27.014 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:27.014 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3245776986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3245776986},{"type":"v1","addr":"172.21.15.203:6836","nonce":3245776986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5056,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/841808653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":841808653},{"type":"v1","addr":"172.21.15.37:6835","nonce":841808653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1908736119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1908736119},{"type":"v1","addr":"172.21.15.203:6837","nonce":1908736119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:53:55.212059+0000","modified":"2024-04-25T08:53:58.247733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035847732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035847732},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035847732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:54:27.014 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-04-25T08:54:27.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 -- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dfc066f60 msgr2=0x7f0dfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dfc066f60 0x7f0dfc069420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0e040ad000 tx=0x7f0e04008040 comp rx=0 tx=0).stop 2024-04-25T08:54:27.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 -- 172.21.15.37:0/135935420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e100cf420 msgr2=0x7f0e100af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e100cf420 0x7f0e100af340 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800cf60 tx=0x7f0e0800cf90 comp rx=0 tx=0).stop 2024-04-25T08:54:27.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 -- 172.21.15.37:0/135935420 shutdown_connections 2024-04-25T08:54:27.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dfc066f60 0x7f0dfc069420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e1015b950 0x7f0e1006d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e100cfb80 0x7f0e100af880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 --2- 172.21.15.37:0/135935420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e100cf420 0x7f0e100af340 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 -- 172.21.15.37:0/135935420 >> 172.21.15.37:0/135935420 conn(0x7f0e100b30b0 msgr2=0x7f0e100ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:27.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 -- 172.21.15.37:0/135935420 shutdown_connections 2024-04-25T08:54:27.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.023+0000 7f0e18520640 1 -- 172.21.15.37:0/135935420 wait complete. 2024-04-25T08:54:27.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:27.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fc1c03640 1 Processor -- start 2024-04-25T08:54:27.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fc1c03640 1 -- start start 2024-04-25T08:54:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fc1c03640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc144ca0 0x7f6fbc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fc1c03640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc0ccc70 con 0x7f6fbc06ec60 2024-04-25T08:54:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fc1c03640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc06dc40 con 0x7f6fbc0ce510 2024-04-25T08:54:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fc1c03640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0cc2b0 con 0x7f6fbc144ca0 2024-04-25T08:54:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fbbfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6fbc06ec60 0x7f6fbc142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52560/0 (socket says 172.21.15.37:52560) 2024-04-25T08:54:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fbbfff640 1 -- 172.21.15.37:0/2373535194 learned_addr learned my addr 172.21.15.37:0/2373535194 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fbaffd640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc144ca0 0x7f6fbc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 616409856 0 0) 0x7f6fbc0ccc70 con 0x7f6fbc06ec60 2024-04-25T08:54:27.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fa4003530 con 0x7f6fbc06ec60 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881846438 0 0) 0x7f6fbc06dc40 con 0x7f6fbc0ce510 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fbc0ccc70 con 0x7f6fbc0ce510 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3706438122 0 0) 0x7f6fa4003530 con 0x7f6fbc06ec60 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc144ca0 msgr2=0x7f6fbc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fba7fc640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc144ca0 0x7f6fbc06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 >> v1:172.21.15.203:6789/0 conn(0x7f6fbc0ce510 legacy=0x7f6fbc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc06df20 con 0x7f6fbc06ec60 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f6fb4001a00 con 0x7f6fbc06ec60 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.163+0000 7f6fbaffd640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc144ca0 0x7f6fbc06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 >> v1:172.21.15.37:6789/0 conn(0x7f6fbc06ec60 legacy=0x7f6fbc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fba7fc640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fa4003bf0 0x7f6fa4003ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fba7fc640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc144ca0 0x7f6fa4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fba7fc640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa4004d50 0x7f6fa40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0cc2b0 con 0x7f6fa4003bf0 2024-04-25T08:54:27.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa400ba80 con 0x7f6fa4004d50 2024-04-25T08:54:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa400bc80 con 0x7f6fbc144ca0 2024-04-25T08:54:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fbb7fe640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fa4003bf0 0x7f6fa4003ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fbaffd640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc144ca0 0x7f6fa4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fbbfff640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa4004d50 0x7f6fa40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fbb7fe640 1 -- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc144ca0 msgr2=0x7f6fa4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fbb7fe640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fbc144ca0 0x7f6fa4004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fbb7fe640 1 -- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa4004d50 msgr2=0x7f6fa40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fbb7fe640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fa4004d50 0x7f6fa40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fbb7fe640 1 -- 172.21.15.37:0/2373535194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa4003530 con 0x7f6fa4003bf0 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fbb7fe640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fa4003bf0 0x7f6fa4003ff0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6fac007fd0 tx=0x7f6fac01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65fafffb6b8d620d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac001520 con 0x7f6fa4003bf0 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac0052f0 con 0x7f6fa4003bf0 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fba7fc640 1 -- 172.21.15.37:0/2373535194 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0a8930 con 0x7f6fa4003bf0 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2373535194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fa4003bf0 msgr2=0x7f6fa4003ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fc1c03640 1 --2- 172.21.15.37:0/2373535194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fa4003bf0 0x7f6fa4003ff0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6fac007fd0 tx=0x7f6fac01ecd0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fbbfff640 1 -- 172.21.15.37:0/2373535194 reap_dead start 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2373535194 shutdown_connections 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2373535194 >> 172.21.15.37:0/2373535194 conn(0x7f6fbc0b11a0 msgr2=0x7f6fbc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2373535194 shutdown_connections 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.167+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2373535194 wait complete. 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fc1c03640 1 Processor -- start 2024-04-25T08:54:27.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fc1c03640 1 -- start start 2024-04-25T08:54:27.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fc1c03640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fa4003bf0 0x7f6fbc15ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fc1c03640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fbc1e2bf0 0x7f6fbc1e3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fc1c03640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc1e3790 0x7f6fbc1e0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fc1c03640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc142d90 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fc1c03640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc06e290 con 0x7f6fbc1e3790 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fc1c03640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc142f10 con 0x7f6fa4003bf0 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fbc1e2bf0 0x7f6fbc1e3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fbc1e2bf0 0x7f6fbc1e3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52118/0 (socket says 172.21.15.37:52118) 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbaffd640 1 -- 172.21.15.37:0/2171822603 learned_addr learned my addr 172.21.15.37:0/2171822603 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbb7fe640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fa4003bf0 0x7f6fbc15ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbbfff640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc1e3790 0x7f6fbc1e0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbaffd640 1 -- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fa4003bf0 msgr2=0x7f6fbc15ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbaffd640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fa4003bf0 0x7f6fbc15ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbaffd640 1 -- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc1e3790 msgr2=0x7f6fbc1e0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbaffd640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc1e3790 0x7f6fbc1e0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbaffd640 1 -- 172.21.15.37:0/2171822603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc1e1b50 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbbfff640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc1e3790 0x7f6fbc1e0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:27.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fbaffd640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fbc1e2bf0 0x7f6fbc1e3250 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0010890 tx=0x7f6fb0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fa3fff640 1 -- 172.21.15.37:0/2171822603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb000f040 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fa3fff640 1 -- 172.21.15.37:0/2171822603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fb000eb60 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2171822603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc1e1e30 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fa3fff640 1 -- 172.21.15.37:0/2171822603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0010e80 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2171822603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc1e23f0 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2171822603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc04efa0 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.171+0000 7f6fa3fff640 1 -- 172.21.15.37:0/2171822603 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6fb001c400 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.175+0000 7f6fa3fff640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fa8066f60 0x7f6fa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.175+0000 7f6fa3fff640 1 -- 172.21.15.37:0/2171822603 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6fb0085780 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.175+0000 7f6fbb7fe640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fa8066f60 0x7f6fa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.175+0000 7f6fbb7fe640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fa8066f60 0x7f6fa8069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6fac007fd0 tx=0x7f6fac0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.179+0000 7f6fa3fff640 1 -- 172.21.15.37:0/2171822603 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6fb0018030 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.331+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2171822603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fbc1e3b70 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fa3fff640 1 -- 172.21.15.37:0/2171822603 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16955 (secure 0 0 0) 0x7f6fb00565e0 con 0x7f6fbc1e2bf0 2024-04-25T08:54:27.328 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:27.328 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:54:12.248632+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:53:53.619419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:53:54.197204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:27.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fa8066f60 msgr2=0x7f6fa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fa8066f60 0x7f6fa8069420 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6fac007fd0 tx=0x7f6fac0aa070 comp rx=0 tx=0).stop 2024-04-25T08:54:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2171822603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fbc1e2bf0 msgr2=0x7f6fbc1e3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fbc1e2bf0 0x7f6fbc1e3250 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6fb0010890 tx=0x7f6fb0010d50 comp rx=0 tx=0).stop 2024-04-25T08:54:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2171822603 shutdown_connections 2024-04-25T08:54:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fa8066f60 0x7f6fa8069420 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6fbc1e3790 0x7f6fbc1e0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6fbc1e2bf0 0x7f6fbc1e3250 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 --2- 172.21.15.37:0/2171822603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6fa4003bf0 0x7f6fbc15ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2171822603 >> 172.21.15.37:0/2171822603 conn(0x7f6fbc0b11a0 msgr2=0x7f6fbc0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2171822603 shutdown_connections 2024-04-25T08:54:27.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.335+0000 7f6fc1c03640 1 -- 172.21.15.37:0/2171822603 wait complete. 2024-04-25T08:54:27.346 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:53:53.619419+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:53:54.197204+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:27.347 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.483+0000 7fd9d4e73640 1 Processor -- start 2024-04-25T08:54:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.483+0000 7fd9d4e73640 1 -- start start 2024-04-25T08:54:27.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.483+0000 7fd9d4e73640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd9d015b770 0x7fd9d006d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.483+0000 7fd9d4e73640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d00713f0 con 0x7fd9d00cf1c0 2024-04-25T08:54:27.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.483+0000 7fd9d4e73640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d00716d0 con 0x7fd9d00cc8a0 2024-04-25T08:54:27.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.483+0000 7fd9d4e73640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9d00719b0 con 0x7fd9d015b770 2024-04-25T08:54:27.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cdd9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd9d00cf1c0 0x7fd9d00cc190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52576/0 (socket says 172.21.15.37:52576) 2024-04-25T08:54:27.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cdd9b640 1 -- 172.21.15.37:0/853814727 learned_addr learned my addr 172.21.15.37:0/853814727 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:27.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9ce59c640 1 --2- 172.21.15.37:0/853814727 >> v2:172.21.15.203:3300/0 conn(0x7fd9d015b770 0x7fd9d006d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3378059391 0 0) 0x7fd9d00713f0 con 0x7fd9d00cf1c0 2024-04-25T08:54:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9bc003530 con 0x7fd9d00cf1c0 2024-04-25T08:54:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4193413530 0 0) 0x7fd9bc003530 con 0x7fd9d00cf1c0 2024-04-25T08:54:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 >> v2:172.21.15.203:3300/0 conn(0x7fd9d015b770 msgr2=0x7fd9d006d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 --2- 172.21.15.37:0/853814727 >> v2:172.21.15.203:3300/0 conn(0x7fd9d015b770 0x7fd9d006d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 >> v1:172.21.15.203:6789/0 conn(0x7fd9d00cc8a0 legacy=0x7fd9d015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d0071c90 con 0x7fd9d00cf1c0 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fd9c0001680 con 0x7fd9d00cf1c0 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 >> v1:172.21.15.37:6789/0 conn(0x7fd9d00cf1c0 legacy=0x7fd9d00cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 --2- 172.21.15.37:0/853814727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 0x7fd9bc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 --2- 172.21.15.37:0/853814727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9bc004810 0x7fd9bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 --2- 172.21.15.37:0/853814727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9bc0051d0 0x7fd9bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d00719b0 con 0x7fd9bc003a30 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc00c0e0 con 0x7fd9bc0051d0 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc00c2e0 con 0x7fd9bc004810 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9ce59c640 1 --2- 172.21.15.37:0/853814727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 0x7fd9bc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9ced9d640 1 --2- 172.21.15.37:0/853814727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9bc0051d0 0x7fd9bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cdd9b640 1 --2- 172.21.15.37:0/853814727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9bc004810 0x7fd9bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9ce59c640 1 -- 172.21.15.37:0/853814727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9bc004810 msgr2=0x7fd9bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9ce59c640 1 --2- 172.21.15.37:0/853814727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9bc004810 0x7fd9bc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9ce59c640 1 -- 172.21.15.37:0/853814727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9bc0051d0 msgr2=0x7fd9bc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:27.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9ce59c640 1 --2- 172.21.15.37:0/853814727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9bc0051d0 0x7fd9bc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9ce59c640 1 -- 172.21.15.37:0/853814727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc003530 con 0x7fd9bc003a30 2024-04-25T08:54:27.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9ced9d640 1 -- 172.21.15.37:0/853814727 reap_dead start 2024-04-25T08:54:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9ce59c640 1 --2- 172.21.15.37:0/853814727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 0x7fd9bc003e50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009ae0 tx=0x7fd9b801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c2a9dad3339402 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80b2070 con 0x7fd9bc003a30 2024-04-25T08:54:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b80a5550 con 0x7fd9bc003a30 2024-04-25T08:54:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.487+0000 7fd9cd59a640 1 -- 172.21.15.37:0/853814727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80096b0 con 0x7fd9bc003a30 2024-04-25T08:54:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 -- 172.21.15.37:0/853814727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 msgr2=0x7fd9bc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 --2- 172.21.15.37:0/853814727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 0x7fd9bc003e50 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009ae0 tx=0x7fd9b801ca70 comp rx=0 tx=0).stop 2024-04-25T08:54:27.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 -- 172.21.15.37:0/853814727 shutdown_connections 2024-04-25T08:54:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 --2- 172.21.15.37:0/853814727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 0x7fd9bc003e50 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 -- 172.21.15.37:0/853814727 >> 172.21.15.37:0/853814727 conn(0x7fd9d00b14b0 msgr2=0x7fd9d00b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 -- 172.21.15.37:0/853814727 shutdown_connections 2024-04-25T08:54:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 -- 172.21.15.37:0/853814727 wait complete. 2024-04-25T08:54:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 Processor -- start 2024-04-25T08:54:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 -- start start 2024-04-25T08:54:27.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 0x7fd9d0155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d0156290 0x7fd9d0156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9d00abc70 0x7fd9d00ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d00cc310 con 0x7fd9bc003a30 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d00cc190 con 0x7fd9d0156290 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d00cc490 con 0x7fd9d00abc70 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 0x7fd9d0155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 0x7fd9d0155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52124/0 (socket says 172.21.15.37:52124) 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9ce59c640 1 -- 172.21.15.37:0/3485115504 learned_addr learned my addr 172.21.15.37:0/3485115504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9cdd9b640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d0156290 0x7fd9d0156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9ced9d640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9d00abc70 0x7fd9d00ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9ce59c640 1 -- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9d00abc70 msgr2=0x7fd9d00ac0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9ce59c640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9d00abc70 0x7fd9d00ac0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9ce59c640 1 -- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d0156290 msgr2=0x7fd9d0156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9ce59c640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d0156290 0x7fd9d0156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9ce59c640 1 -- 172.21.15.37:0/3485115504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d00a84d0 con 0x7fd9bc003a30 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9ce59c640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 0x7fd9d0155ad0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80b0040 tx=0x7fd9b8019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9c6ffd640 1 -- 172.21.15.37:0/3485115504 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b80b2040 con 0x7fd9bc003a30 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9c6ffd640 1 -- 172.21.15.37:0/3485115504 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b8004020 con 0x7fd9bc003a30 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 -- 172.21.15.37:0/3485115504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d00a87b0 con 0x7fd9bc003a30 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9c6ffd640 1 -- 172.21.15.37:0/3485115504 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8004410 con 0x7fd9bc003a30 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.491+0000 7fd9d4e73640 1 -- 172.21.15.37:0/3485115504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d00a8d70 con 0x7fd9bc003a30 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.495+0000 7fd9d4e73640 1 -- 172.21.15.37:0/3485115504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d004efa0 con 0x7fd9bc003a30 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.495+0000 7fd9c6ffd640 1 -- 172.21.15.37:0/3485115504 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd9b8003620 con 0x7fd9bc003a30 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.495+0000 7fd9c6ffd640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9b0066f90 0x7fd9b0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.495+0000 7fd9c6ffd640 1 -- 172.21.15.37:0/3485115504 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fd9b811e8b0 con 0x7fd9bc003a30 2024-04-25T08:54:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.495+0000 7fd9cdd9b640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9b0066f90 0x7fd9b0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.495+0000 7fd9cdd9b640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9b0066f90 0x7fd9b0069450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd9d01585f0 tx=0x7fd9c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.499+0000 7fd9c6ffd640 1 -- 172.21.15.37:0/3485115504 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd9b8016780 con 0x7fd9bc003a30 2024-04-25T08:54:27.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 -- 172.21.15.37:0/3485115504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9d00cec50 con 0x7fd9bc003a30 2024-04-25T08:54:27.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9c6ffd640 1 -- 172.21.15.37:0/3485115504 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9b80ef150 con 0x7fd9bc003a30 2024-04-25T08:54:27.660 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:54:27.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 -- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9b0066f90 msgr2=0x7fd9b0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9b0066f90 0x7fd9b0069450 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd9d01585f0 tx=0x7fd9c0009040 comp rx=0 tx=0).stop 2024-04-25T08:54:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 -- 172.21.15.37:0/3485115504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 msgr2=0x7fd9d0155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 0x7fd9d0155ad0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80b0040 tx=0x7fd9b8019810 comp rx=0 tx=0).stop 2024-04-25T08:54:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 -- 172.21.15.37:0/3485115504 shutdown_connections 2024-04-25T08:54:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9b0066f90 0x7fd9b0069450 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd9d00abc70 0x7fd9d00ac0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd9d0156290 0x7fd9d0156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 --2- 172.21.15.37:0/3485115504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9bc003a30 0x7fd9d0155ad0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 -- 172.21.15.37:0/3485115504 >> 172.21.15.37:0/3485115504 conn(0x7fd9d00b14b0 msgr2=0x7fd9d00cd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 -- 172.21.15.37:0/3485115504 shutdown_connections 2024-04-25T08:54:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.667+0000 7fd9d4e73640 1 -- 172.21.15.37:0/3485115504 wait complete. 2024-04-25T08:54:27.678 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.803+0000 7fc28ee07640 1 Processor -- start 2024-04-25T08:54:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.803+0000 7fc28ee07640 1 -- start start 2024-04-25T08:54:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28ee07640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc288159bd0 0x7fc2880c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28ee07640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc288159fb0 0x7fc2880b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28ee07640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc2880b26f0 0x7fc2880b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28ee07640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc288072780 con 0x7fc288159bd0 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28ee07640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2880c9ef0 con 0x7fc2880b26f0 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28ee07640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc288071b70 con 0x7fc288159fb0 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28cba3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc288159bd0 0x7fc2880c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28cba3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc288159bd0 0x7fc2880c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52130/0 (socket says 172.21.15.37:52130) 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28cba3640 1 -- 172.21.15.37:0/2547340123 learned_addr learned my addr 172.21.15.37:0/2547340123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28d3a4640 1 --2- 172.21.15.37:0/2547340123 >> v2:172.21.15.203:3300/0 conn(0x7fc2880b26f0 0x7fc2880b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc27ffff640 1 --2- 172.21.15.37:0/2547340123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc288159fb0 0x7fc2880b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28cba3640 1 -- 172.21.15.37:0/2547340123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc288159fb0 msgr2=0x7fc2880b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28cba3640 1 --2- 172.21.15.37:0/2547340123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc288159fb0 0x7fc2880b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28cba3640 1 -- 172.21.15.37:0/2547340123 >> v2:172.21.15.203:3300/0 conn(0x7fc2880b26f0 msgr2=0x7fc2880b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28cba3640 1 --2- 172.21.15.37:0/2547340123 >> v2:172.21.15.203:3300/0 conn(0x7fc2880b26f0 0x7fc2880b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28cba3640 1 -- 172.21.15.37:0/2547340123 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc288071e50 con 0x7fc288159bd0 2024-04-25T08:54:27.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.807+0000 7fc28cba3640 1 --2- 172.21.15.37:0/2547340123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc288159bd0 0x7fc2880c99b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc2880cc9e0 tx=0x7fc28401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ee893e692fb12c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27f7fe640 1 -- 172.21.15.37:0/2547340123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2840ab430 con 0x7fc288159bd0 2024-04-25T08:54:27.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27f7fe640 1 -- 172.21.15.37:0/2547340123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2840b0070 con 0x7fc288159bd0 2024-04-25T08:54:27.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27f7fe640 1 -- 172.21.15.37:0/2547340123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2840ab7f0 con 0x7fc288159bd0 2024-04-25T08:54:27.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 -- 172.21.15.37:0/2547340123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc288159bd0 msgr2=0x7fc2880c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 --2- 172.21.15.37:0/2547340123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc288159bd0 0x7fc2880c99b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc2880cc9e0 tx=0x7fc28401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 -- 172.21.15.37:0/2547340123 shutdown_connections 2024-04-25T08:54:27.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 --2- 172.21.15.37:0/2547340123 >> v2:172.21.15.203:3300/0 conn(0x7fc2880b26f0 0x7fc2880b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 --2- 172.21.15.37:0/2547340123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc288159fb0 0x7fc2880b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 --2- 172.21.15.37:0/2547340123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc288159bd0 0x7fc2880c99b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 -- 172.21.15.37:0/2547340123 >> 172.21.15.37:0/2547340123 conn(0x7fc28806ae60 msgr2=0x7fc28806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:27.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 -- 172.21.15.37:0/2547340123 shutdown_connections 2024-04-25T08:54:27.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 -- 172.21.15.37:0/2547340123 wait complete. 2024-04-25T08:54:27.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 Processor -- start 2024-04-25T08:54:27.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 -- start start 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2880b26f0 0x7fc288158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc288159bd0 0x7fc28808d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc288159fb0 0x7fc2880c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288165af0 con 0x7fc288159bd0 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc288165970 con 0x7fc2880b26f0 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc288165c70 con 0x7fc288159fb0 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc288159bd0 0x7fc28808d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc288159bd0 0x7fc28808d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52136/0 (socket says 172.21.15.37:52136) 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27ffff640 1 -- 172.21.15.37:0/1817025944 learned_addr learned my addr 172.21.15.37:0/1817025944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28cba3640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2880b26f0 0x7fc288158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28d3a4640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc288159fb0 0x7fc2880c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27ffff640 1 -- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc288159fb0 msgr2=0x7fc2880c15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27ffff640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc288159fb0 0x7fc2880c15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27ffff640 1 -- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2880b26f0 msgr2=0x7fc288158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27ffff640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2880b26f0 0x7fc288158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27ffff640 1 -- 172.21.15.37:0/1817025944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28808e030 con 0x7fc288159bd0 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28d3a4640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc288159fb0 0x7fc2880c15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27ffff640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc288159bd0 0x7fc28808d5b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc27800d800 tx=0x7fc27800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27d7fa640 1 -- 172.21.15.37:0/1817025944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc278004020 con 0x7fc288159bd0 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27d7fa640 1 -- 172.21.15.37:0/1817025944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2780041c0 con 0x7fc288159bd0 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 -- 172.21.15.37:0/1817025944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28806f0f0 con 0x7fc288159bd0 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc27d7fa640 1 -- 172.21.15.37:0/1817025944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2780043b0 con 0x7fc288159bd0 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.811+0000 7fc28ee07640 1 -- 172.21.15.37:0/1817025944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28808dcb0 con 0x7fc288159bd0 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.815+0000 7fc28ee07640 1 -- 172.21.15.37:0/1817025944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28804efa0 con 0x7fc288159bd0 2024-04-25T08:54:27.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.815+0000 7fc27d7fa640 1 -- 172.21.15.37:0/1817025944 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc27801b020 con 0x7fc288159bd0 2024-04-25T08:54:27.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.815+0000 7fc27d7fa640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc274066f90 0x7fc274069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.815+0000 7fc27d7fa640 1 -- 172.21.15.37:0/1817025944 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc278084ff0 con 0x7fc288159bd0 2024-04-25T08:54:27.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.815+0000 7fc28cba3640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc274066f90 0x7fc274069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.815+0000 7fc28cba3640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc274066f90 0x7fc274069450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc2840af000 tx=0x7fc2840a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.819+0000 7fc27d7fa640 1 -- 172.21.15.37:0/1817025944 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc278055db0 con 0x7fc288159bd0 2024-04-25T08:54:27.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 -- 172.21.15.37:0/1817025944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc28806f2c0 con 0x7fc288159bd0 2024-04-25T08:54:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc27d7fa640 1 -- 172.21.15.37:0/1817025944 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2780556b0 con 0x7fc288159bd0 2024-04-25T08:54:27.977 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:54:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 -- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc274066f90 msgr2=0x7fc274069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc274066f90 0x7fc274069450 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc2840af000 tx=0x7fc2840a7000 comp rx=0 tx=0).stop 2024-04-25T08:54:27.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 -- 172.21.15.37:0/1817025944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc288159bd0 msgr2=0x7fc28808d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc288159bd0 0x7fc28808d5b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc27800d800 tx=0x7fc27800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 -- 172.21.15.37:0/1817025944 shutdown_connections 2024-04-25T08:54:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc274066f90 0x7fc274069450 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc288159fb0 0x7fc2880c15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc288159bd0 0x7fc28808d5b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 --2- 172.21.15.37:0/1817025944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2880b26f0 0x7fc288158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 -- 172.21.15.37:0/1817025944 >> 172.21.15.37:0/1817025944 conn(0x7fc28806ae60 msgr2=0x7fc2880b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 -- 172.21.15.37:0/1817025944 shutdown_connections 2024-04-25T08:54:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:27.983+0000 7fc28ee07640 1 -- 172.21.15.37:0/1817025944 wait complete. 2024-04-25T08:54:27.996 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:28.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27f2c57640 1 Processor -- start 2024-04-25T08:54:28.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27f2c57640 1 -- start start 2024-04-25T08:54:28.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27f2c57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27ec15b950 0x7f27ec09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27f2c57640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f27ec0cf420 0x7f27ec0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27f2c57640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f27ec0cfb80 0x7f27ec0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27f2c57640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ec15bd30 con 0x7f27ec0cf420 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27f2c57640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ec0cc0d0 con 0x7f27ec0cfb80 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27f2c57640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec09ab10 con 0x7f27ec15b950 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27ebfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f27ec0cf420 0x7f27ec0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27ebfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f27ec0cf420 0x7f27ec0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52146/0 (socket says 172.21.15.37:52146) 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27ebfff640 1 -- 172.21.15.37:0/1274252607 learned_addr learned my addr 172.21.15.37:0/1274252607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27f09f3640 1 --2- 172.21.15.37:0/1274252607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27ec15b950 0x7f27ec09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27f11f4640 1 --2- 172.21.15.37:0/1274252607 >> v2:172.21.15.203:3300/0 conn(0x7f27ec0cfb80 0x7f27ec0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27ebfff640 1 -- 172.21.15.37:0/1274252607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27ec15b950 msgr2=0x7f27ec09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27ebfff640 1 --2- 172.21.15.37:0/1274252607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27ec15b950 0x7f27ec09a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27ebfff640 1 -- 172.21.15.37:0/1274252607 >> v2:172.21.15.203:3300/0 conn(0x7f27ec0cfb80 msgr2=0x7f27ec0d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:28.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27ebfff640 1 --2- 172.21.15.37:0/1274252607 >> v2:172.21.15.203:3300/0 conn(0x7f27ec0cfb80 0x7f27ec0d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27ebfff640 1 -- 172.21.15.37:0/1274252607 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec0cc370 con 0x7f27ec0cf420 2024-04-25T08:54:28.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.123+0000 7f27f09f3640 1 --2- 172.21.15.37:0/1274252607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27ec15b950 0x7f27ec09a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27ebfff640 1 --2- 172.21.15.37:0/1274252607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ec0cf420 0x7f27ec0cce50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0b3bd0 tx=0x7f27e001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9833099d1cae860b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27eb7fe640 1 -- 172.21.15.37:0/1274252607 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00b1070 con 0x7f27ec0cf420 2024-04-25T08:54:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27eb7fe640 1 -- 172.21.15.37:0/1274252607 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e00ac630 con 0x7f27ec0cf420 2024-04-25T08:54:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27eb7fe640 1 -- 172.21.15.37:0/1274252607 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00a8050 con 0x7f27ec0cf420 2024-04-25T08:54:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 -- 172.21.15.37:0/1274252607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ec0cf420 msgr2=0x7f27ec0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 --2- 172.21.15.37:0/1274252607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ec0cf420 0x7f27ec0cce50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0b3bd0 tx=0x7f27e001c520 comp rx=0 tx=0).stop 2024-04-25T08:54:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 -- 172.21.15.37:0/1274252607 shutdown_connections 2024-04-25T08:54:28.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 --2- 172.21.15.37:0/1274252607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ec0cf420 0x7f27ec0cce50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 --2- 172.21.15.37:0/1274252607 >> v2:172.21.15.203:3300/0 conn(0x7f27ec0cfb80 0x7f27ec0d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 --2- 172.21.15.37:0/1274252607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27ec15b950 0x7f27ec09a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 -- 172.21.15.37:0/1274252607 >> 172.21.15.37:0/1274252607 conn(0x7f27ec0b30b0 msgr2=0x7f27ec0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 -- 172.21.15.37:0/1274252607 shutdown_connections 2024-04-25T08:54:28.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 -- 172.21.15.37:0/1274252607 wait complete. 2024-04-25T08:54:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 Processor -- start 2024-04-25T08:54:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 -- start start 2024-04-25T08:54:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27ec0cf420 0x7f27ec151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27ec0cfb80 0x7f27ec1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ec15b950 0x7f27ec1568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec0cd390 con 0x7f27ec15b950 2024-04-25T08:54:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec06faa0 con 0x7f27ec0cf420 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec157100 con 0x7f27ec0cfb80 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f11f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ec15b950 0x7f27ec1568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f11f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ec15b950 0x7f27ec1568f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52160/0 (socket says 172.21.15.37:52160) 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f11f4640 1 -- 172.21.15.37:0/2869478285 learned_addr learned my addr 172.21.15.37:0/2869478285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f09f3640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27ec0cf420 0x7f27ec151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27ebfff640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27ec0cfb80 0x7f27ec1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f11f4640 1 -- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27ec0cfb80 msgr2=0x7f27ec1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f11f4640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27ec0cfb80 0x7f27ec1563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f11f4640 1 -- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27ec0cf420 msgr2=0x7f27ec151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f11f4640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27ec0cf420 0x7f27ec151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f11f4640 1 -- 172.21.15.37:0/2869478285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec0c6700 con 0x7f27ec15b950 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f11f4640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ec15b950 0x7f27ec1568f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f27e400cd70 tx=0x7f27e400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27e97fa640 1 -- 172.21.15.37:0/2869478285 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4003ba0 con 0x7f27ec15b950 2024-04-25T08:54:28.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27e97fa640 1 -- 172.21.15.37:0/2869478285 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e4003d40 con 0x7f27ec15b950 2024-04-25T08:54:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27f2c57640 1 -- 172.21.15.37:0/2869478285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ec154d10 con 0x7f27ec15b950 2024-04-25T08:54:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.127+0000 7f27e97fa640 1 -- 172.21.15.37:0/2869478285 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4004240 con 0x7f27ec15b950 2024-04-25T08:54:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.131+0000 7f27f2c57640 1 -- 172.21.15.37:0/2869478285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ec1552d0 con 0x7f27ec15b950 2024-04-25T08:54:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.131+0000 7f27f2c57640 1 -- 172.21.15.37:0/2869478285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ec0c68d0 con 0x7f27ec15b950 2024-04-25T08:54:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.131+0000 7f27e97fa640 1 -- 172.21.15.37:0/2869478285 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27e40043e0 con 0x7f27ec15b950 2024-04-25T08:54:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.131+0000 7f27e97fa640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27d8066f60 0x7f27d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.131+0000 7f27e97fa640 1 -- 172.21.15.37:0/2869478285 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f27e4085cd0 con 0x7f27ec15b950 2024-04-25T08:54:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.131+0000 7f27f09f3640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27d8066f60 0x7f27d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.131+0000 7f27f09f3640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27d8066f60 0x7f27d8069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006fd0 tx=0x7f27dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:28.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.135+0000 7f27e97fa640 1 -- 172.21.15.37:0/2869478285 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27e401d340 con 0x7f27ec15b950 2024-04-25T08:54:28.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.287+0000 7f27f2c57640 1 -- 172.21.15.37:0/2869478285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27ec04efa0 con 0x7f27ec15b950 2024-04-25T08:54:28.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.287+0000 7f27e97fa640 1 -- 172.21.15.37:0/2869478285 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27e4056b30 con 0x7f27ec15b950 2024-04-25T08:54:28.282 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:54:28.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 -- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27d8066f60 msgr2=0x7f27d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27d8066f60 0x7f27d8069420 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f27dc006fd0 tx=0x7f27dc008040 comp rx=0 tx=0).stop 2024-04-25T08:54:28.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 -- 172.21.15.37:0/2869478285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ec15b950 msgr2=0x7f27ec1568f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ec15b950 0x7f27ec1568f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f27e400cd70 tx=0x7f27e400a660 comp rx=0 tx=0).stop 2024-04-25T08:54:28.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 -- 172.21.15.37:0/2869478285 shutdown_connections 2024-04-25T08:54:28.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27d8066f60 0x7f27d8069420 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27ec15b950 0x7f27ec1568f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27ec0cfb80 0x7f27ec1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 --2- 172.21.15.37:0/2869478285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27ec0cf420 0x7f27ec151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 -- 172.21.15.37:0/2869478285 >> 172.21.15.37:0/2869478285 conn(0x7f27ec0b30b0 msgr2=0x7f27ec15a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:28.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 -- 172.21.15.37:0/2869478285 shutdown_connections 2024-04-25T08:54:28.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.291+0000 7f27f2c57640 1 -- 172.21.15.37:0/2869478285 wait complete. 2024-04-25T08:54:28.300 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe4459c640 1 Processor -- start 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe4459c640 1 -- start start 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe4459c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efe3c143790 0x7efe3c06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe4459c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c143b70 con 0x7efe3c06c3b0 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe4459c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe3c06e170 con 0x7efe3c15da40 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe4459c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe3c06e450 con 0x7efe3c143790 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe42338640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efe3c143790 0x7efe3c06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe42b39640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7efe3c06c3b0 0x7efe3c0b31f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52588/0 (socket says 172.21.15.37:52588) 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe42b39640 1 -- 172.21.15.37:0/2782840258 learned_addr learned my addr 172.21.15.37:0/2782840258 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe42338640 1 -- 172.21.15.37:0/2782840258 >> v1:172.21.15.203:6789/0 conn(0x7efe3c15da40 legacy=0x7efe3c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe42338640 1 -- 172.21.15.37:0/2782840258 >> v1:172.21.15.37:6789/0 conn(0x7efe3c06c3b0 legacy=0x7efe3c0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.427+0000 7efe42338640 1 -- 172.21.15.37:0/2782840258 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c06e730 con 0x7efe3c143790 2024-04-25T08:54:28.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe42338640 1 --2- 172.21.15.37:0/2782840258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c143790 0x7efe3c06d870 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7efe3800a960 tx=0x7efe3801d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47de5a6b0e8c343f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:28.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41336640 1 -- 172.21.15.37:0/2782840258 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3800b040 con 0x7efe3c143790 2024-04-25T08:54:28.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41336640 1 -- 172.21.15.37:0/2782840258 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe380a76d0 con 0x7efe3c143790 2024-04-25T08:54:28.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41336640 1 -- 172.21.15.37:0/2782840258 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe380a7aa0 con 0x7efe3c143790 2024-04-25T08:54:28.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 -- 172.21.15.37:0/2782840258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c143790 msgr2=0x7efe3c06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 --2- 172.21.15.37:0/2782840258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c143790 0x7efe3c06d870 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7efe3800a960 tx=0x7efe3801d870 comp rx=0 tx=0).stop 2024-04-25T08:54:28.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 -- 172.21.15.37:0/2782840258 shutdown_connections 2024-04-25T08:54:28.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 --2- 172.21.15.37:0/2782840258 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c143790 0x7efe3c06d870 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 -- 172.21.15.37:0/2782840258 >> 172.21.15.37:0/2782840258 conn(0x7efe3c071810 msgr2=0x7efe3c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:28.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 -- 172.21.15.37:0/2782840258 shutdown_connections 2024-04-25T08:54:28.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 -- 172.21.15.37:0/2782840258 wait complete. 2024-04-25T08:54:28.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 Processor -- start 2024-04-25T08:54:28.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 -- start start 2024-04-25T08:54:28.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe3c06c3b0 0x7efe3c140700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c143790 0x7efe3c13b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c15da40 0x7efe3c13ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0b3c90 con 0x7efe3c143790 2024-04-25T08:54:28.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0b3b10 con 0x7efe3c15da40 2024-04-25T08:54:28.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe4459c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3c0b3e10 con 0x7efe3c06c3b0 2024-04-25T08:54:28.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41b37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c143790 0x7efe3c13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41b37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c143790 0x7efe3c13b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52180/0 (socket says 172.21.15.37:52180) 2024-04-25T08:54:28.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41b37640 1 -- 172.21.15.37:0/541571218 learned_addr learned my addr 172.21.15.37:0/541571218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:28.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe42338640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe3c06c3b0 0x7efe3c140700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41b37640 1 -- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe3c06c3b0 msgr2=0x7efe3c140700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe42b39640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c15da40 0x7efe3c13ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41b37640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe3c06c3b0 0x7efe3c140700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41b37640 1 -- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c15da40 msgr2=0x7efe3c13ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41b37640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c15da40 0x7efe3c13ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41b37640 1 -- 172.21.15.37:0/541571218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c140ce0 con 0x7efe3c143790 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe42b39640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c15da40 0x7efe3c13ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.431+0000 7efe41b37640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c143790 0x7efe3c13b530 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c370 tx=0x7efe2c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.435+0000 7efe32ffd640 1 -- 172.21.15.37:0/541571218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c0052d0 con 0x7efe3c143790 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.435+0000 7efe32ffd640 1 -- 172.21.15.37:0/541571218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2c005470 con 0x7efe3c143790 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.435+0000 7efe4459c640 1 -- 172.21.15.37:0/541571218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe3c139bb0 con 0x7efe3c143790 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.435+0000 7efe32ffd640 1 -- 172.21.15.37:0/541571218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2c007260 con 0x7efe3c143790 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.435+0000 7efe4459c640 1 -- 172.21.15.37:0/541571218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe3c13a170 con 0x7efe3c143790 2024-04-25T08:54:28.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.435+0000 7efe4459c640 1 -- 172.21.15.37:0/541571218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe3c04efa0 con 0x7efe3c143790 2024-04-25T08:54:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.435+0000 7efe32ffd640 1 -- 172.21.15.37:0/541571218 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe2c007400 con 0x7efe3c143790 2024-04-25T08:54:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.435+0000 7efe32ffd640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe20066f90 0x7efe20069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.435+0000 7efe32ffd640 1 -- 172.21.15.37:0/541571218 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7efe2c084690 con 0x7efe3c143790 2024-04-25T08:54:28.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.435+0000 7efe42338640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe20066f90 0x7efe20069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.435+0000 7efe42338640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe20066f90 0x7efe20069450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efe380b0000 tx=0x7efe3800c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:28.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.439+0000 7efe32ffd640 1 -- 172.21.15.37:0/541571218 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe2c055450 con 0x7efe3c143790 2024-04-25T08:54:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.627+0000 7efe4459c640 1 -- 172.21.15.37:0/541571218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe3c15bea0 con 0x7efe3c143790 2024-04-25T08:54:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.627+0000 7efe32ffd640 1 -- 172.21.15.37:0/541571218 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7efe2c054d50 con 0x7efe3c143790 2024-04-25T08:54:28.622 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:28.622 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3245776986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3245776986},{"type":"v1","addr":"172.21.15.203:6836","nonce":3245776986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5056,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/841808653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":841808653},{"type":"v1","addr":"172.21.15.37:6835","nonce":841808653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1908736119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1908736119},{"type":"v1","addr":"172.21.15.203:6837","nonce":1908736119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:53:55.212059+0000","modified":"2024-04-25T08:53:58.247733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035847732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035847732},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035847732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:54:28.622 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-04-25T08:54:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 -- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe20066f90 msgr2=0x7efe20069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe20066f90 0x7efe20069450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efe380b0000 tx=0x7efe3800c040 comp rx=0 tx=0).stop 2024-04-25T08:54:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 -- 172.21.15.37:0/541571218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c143790 msgr2=0x7efe3c13b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c143790 0x7efe3c13b530 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c370 tx=0x7efe2c00c830 comp rx=0 tx=0).stop 2024-04-25T08:54:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 -- 172.21.15.37:0/541571218 shutdown_connections 2024-04-25T08:54:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe20066f90 0x7efe20069450 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe3c15da40 0x7efe3c13ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe3c143790 0x7efe3c13b530 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 --2- 172.21.15.37:0/541571218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe3c06c3b0 0x7efe3c140700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 -- 172.21.15.37:0/541571218 >> 172.21.15.37:0/541571218 conn(0x7efe3c071810 msgr2=0x7efe3c15a660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 -- 172.21.15.37:0/541571218 shutdown_connections 2024-04-25T08:54:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.631+0000 7efe4459c640 1 -- 172.21.15.37:0/541571218 wait complete. 2024-04-25T08:54:28.639 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29ee0a6640 1 Processor -- start 2024-04-25T08:54:28.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29ee0a6640 1 -- start start 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29ee0a6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f29e80ce510 0x7f29e80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29ee0a6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e8144ca0 0x7f29e806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29ee0a6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f29e806ec60 0x7f29e8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29ee0a6640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e80cc2b0 con 0x7f29e806ec60 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29ee0a6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e806e290 con 0x7f29e80ce510 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29ee0a6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80cee30 con 0x7f29e8144ca0 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29e7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f29e806ec60 0x7f29e8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29e7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f29e806ec60 0x7f29e8142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52182/0 (socket says 172.21.15.37:52182) 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29e7fff640 1 -- 172.21.15.37:0/2843699365 learned_addr learned my addr 172.21.15.37:0/2843699365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29e6ffd640 1 --2- 172.21.15.37:0/2843699365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e8144ca0 0x7f29e806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29e77fe640 1 --2- 172.21.15.37:0/2843699365 >> v2:172.21.15.203:3300/0 conn(0x7f29e80ce510 0x7f29e80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29e7fff640 1 -- 172.21.15.37:0/2843699365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e8144ca0 msgr2=0x7f29e806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:28.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29e7fff640 1 --2- 172.21.15.37:0/2843699365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e8144ca0 0x7f29e806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29e7fff640 1 -- 172.21.15.37:0/2843699365 >> v2:172.21.15.203:3300/0 conn(0x7f29e80ce510 msgr2=0x7f29e80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:28.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29e7fff640 1 --2- 172.21.15.37:0/2843699365 >> v2:172.21.15.203:3300/0 conn(0x7f29e80ce510 0x7f29e80ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29e7fff640 1 -- 172.21.15.37:0/2843699365 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80ccc70 con 0x7f29e806ec60 2024-04-25T08:54:28.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.763+0000 7f29e77fe640 1 --2- 172.21.15.37:0/2843699365 >> v2:172.21.15.203:3300/0 conn(0x7f29e80ce510 0x7f29e80ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29e7fff640 1 --2- 172.21.15.37:0/2843699365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29e806ec60 0x7f29e8142680 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f29e8144c50 tx=0x7f29e001f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d3d92b2b6124f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29e67fc640 1 -- 172.21.15.37:0/2843699365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e00b4070 con 0x7f29e806ec60 2024-04-25T08:54:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29e67fc640 1 -- 172.21.15.37:0/2843699365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e00af400 con 0x7f29e806ec60 2024-04-25T08:54:28.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29e67fc640 1 -- 172.21.15.37:0/2843699365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e00af7d0 con 0x7f29e806ec60 2024-04-25T08:54:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29ee0a6640 1 -- 172.21.15.37:0/2843699365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29e806ec60 msgr2=0x7f29e8142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29ee0a6640 1 --2- 172.21.15.37:0/2843699365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29e806ec60 0x7f29e8142680 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f29e8144c50 tx=0x7f29e001f1e0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29ee0a6640 1 -- 172.21.15.37:0/2843699365 shutdown_connections 2024-04-25T08:54:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29ee0a6640 1 --2- 172.21.15.37:0/2843699365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29e806ec60 0x7f29e8142680 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29ee0a6640 1 --2- 172.21.15.37:0/2843699365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e8144ca0 0x7f29e806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29ee0a6640 1 --2- 172.21.15.37:0/2843699365 >> v2:172.21.15.203:3300/0 conn(0x7f29e80ce510 0x7f29e80ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29ee0a6640 1 -- 172.21.15.37:0/2843699365 >> 172.21.15.37:0/2843699365 conn(0x7f29e80b11a0 msgr2=0x7f29e80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29ee0a6640 1 -- 172.21.15.37:0/2843699365 shutdown_connections 2024-04-25T08:54:28.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29ee0a6640 1 -- 172.21.15.37:0/2843699365 wait complete. 2024-04-25T08:54:28.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29ee0a6640 1 Processor -- start 2024-04-25T08:54:28.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.767+0000 7f29ee0a6640 1 -- start start 2024-04-25T08:54:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29ee0a6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29e806ec60 0x7f29e81f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29ee0a6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e80ce510 0x7f29e80a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29ee0a6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e8144ca0 0x7f29e80a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29ee0a6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e80cc900 con 0x7f29e806ec60 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29ee0a6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e806df20 con 0x7f29e80ce510 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29ee0a6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e81451f0 con 0x7f29e8144ca0 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29e806ec60 0x7f29e81f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29e806ec60 0x7f29e81f2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52184/0 (socket says 172.21.15.37:52184) 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e77fe640 1 -- 172.21.15.37:0/1920312323 learned_addr learned my addr 172.21.15.37:0/1920312323 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e7fff640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e8144ca0 0x7f29e80a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e6ffd640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e80ce510 0x7f29e80a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e77fe640 1 -- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e8144ca0 msgr2=0x7f29e80a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e77fe640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e8144ca0 0x7f29e80a8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e77fe640 1 -- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e80ce510 msgr2=0x7f29e80a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e77fe640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e80ce510 0x7f29e80a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e77fe640 1 -- 172.21.15.37:0/1920312323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80b3f10 con 0x7f29e806ec60 2024-04-25T08:54:28.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e77fe640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29e806ec60 0x7f29e81f2c30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c7c0 tx=0x7f29d800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29dffff640 1 -- 172.21.15.37:0/1920312323 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8015070 con 0x7f29e806ec60 2024-04-25T08:54:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29dffff640 1 -- 172.21.15.37:0/1920312323 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d8007a30 con 0x7f29e806ec60 2024-04-25T08:54:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29ee0a6640 1 -- 172.21.15.37:0/1920312323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e80b3c30 con 0x7f29e806ec60 2024-04-25T08:54:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29dffff640 1 -- 172.21.15.37:0/1920312323 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8007e00 con 0x7f29e806ec60 2024-04-25T08:54:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29ee0a6640 1 -- 172.21.15.37:0/1920312323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e8133430 con 0x7f29e806ec60 2024-04-25T08:54:28.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29ee0a6640 1 -- 172.21.15.37:0/1920312323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e804efa0 con 0x7f29e806ec60 2024-04-25T08:54:28.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29dffff640 1 -- 172.21.15.37:0/1920312323 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f29d8018070 con 0x7f29e806ec60 2024-04-25T08:54:28.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29dffff640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f29c8066f60 0x7f29c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29dffff640 1 -- 172.21.15.37:0/1920312323 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f29d80850e0 con 0x7f29e806ec60 2024-04-25T08:54:28.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.771+0000 7f29e6ffd640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f29c8066f60 0x7f29c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.775+0000 7f29e6ffd640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f29c8066f60 0x7f29c8069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f29e80b42e0 tx=0x7f29d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:28.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.775+0000 7f29dffff640 1 -- 172.21.15.37:0/1920312323 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f29d8055f40 con 0x7f29e806ec60 2024-04-25T08:54:28.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.935+0000 7f29ee0a6640 1 -- 172.21.15.37:0/1920312323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29e80b1cc0 con 0x7f29e806ec60 2024-04-25T08:54:28.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29dffff640 1 -- 172.21.15.37:0/1920312323 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16955 (secure 0 0 0) 0x7f29d8055840 con 0x7f29e806ec60 2024-04-25T08:54:28.933 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:28.933 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:54:12.248632+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:53:53.619419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:53:54.197204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 -- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f29c8066f60 msgr2=0x7f29c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f29c8066f60 0x7f29c8069420 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f29e80b42e0 tx=0x7f29d0008040 comp rx=0 tx=0).stop 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 -- 172.21.15.37:0/1920312323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29e806ec60 msgr2=0x7f29e81f2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29e806ec60 0x7f29e81f2c30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f29d800c7c0 tx=0x7f29d800cc80 comp rx=0 tx=0).stop 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 -- 172.21.15.37:0/1920312323 shutdown_connections 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f29c8066f60 0x7f29c8069420 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f29e8144ca0 0x7f29e80a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29e80ce510 0x7f29e80a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 --2- 172.21.15.37:0/1920312323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29e806ec60 0x7f29e81f2c30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 -- 172.21.15.37:0/1920312323 >> 172.21.15.37:0/1920312323 conn(0x7f29e80b11a0 msgr2=0x7f29e80b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 -- 172.21.15.37:0/1920312323 shutdown_connections 2024-04-25T08:54:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:28.939+0000 7f29ee0a6640 1 -- 172.21.15.37:0/1920312323 wait complete. 2024-04-25T08:54:28.951 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:54:28.951 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf8a6d640 1 Processor -- start 2024-04-25T08:54:29.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf8a6d640 1 -- start start 2024-04-25T08:54:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf8a6d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feaf406bca0 0x7feaf40cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf8a6d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feaf4066e30 0x7feaf406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf8a6d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf406ec30 con 0x7feaf40af370 2024-04-25T08:54:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf8a6d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf40669a0 con 0x7feaf406bca0 2024-04-25T08:54:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf8a6d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf40cd5f0 con 0x7feaf4066e30 2024-04-25T08:54:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feaf406bca0 0x7feaf40cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feaf406bca0 0x7feaf40cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52194/0 (socket says 172.21.15.37:52194) 2024-04-25T08:54:29.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf259c640 1 -- 172.21.15.37:0/4208658728 learned_addr learned my addr 172.21.15.37:0/4208658728 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:29.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaebfff640 1 --2- 172.21.15.37:0/4208658728 >> v2:172.21.15.203:3300/0 conn(0x7feaf4066e30 0x7feaf406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf259c640 1 -- 172.21.15.37:0/4208658728 >> v2:172.21.15.203:3300/0 conn(0x7feaf4066e30 msgr2=0x7feaf406c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:29.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf259c640 1 --2- 172.21.15.37:0/4208658728 >> v2:172.21.15.203:3300/0 conn(0x7feaf4066e30 0x7feaf406c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf259c640 1 -- 172.21.15.37:0/4208658728 >> v1:172.21.15.37:6789/0 conn(0x7feaf40af370 legacy=0x7feaf406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf259c640 1 -- 172.21.15.37:0/4208658728 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf406c5c0 con 0x7feaf406bca0 2024-04-25T08:54:29.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.071+0000 7feaf1d9b640 1 -- 172.21.15.37:0/4208658728 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584345672 0 0) 0x7feaf406ec30 con 0x7feaf40af370 2024-04-25T08:54:29.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf259c640 1 --2- 172.21.15.37:0/4208658728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf406bca0 0x7feaf40cd0b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7feaf40731a0 tx=0x7feae401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62868e6d55c89fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:29.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf1d9b640 1 -- 172.21.15.37:0/4208658728 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae40b0070 con 0x7feaf406bca0 2024-04-25T08:54:29.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf1d9b640 1 -- 172.21.15.37:0/4208658728 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae40ab600 con 0x7feaf406bca0 2024-04-25T08:54:29.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf1d9b640 1 -- 172.21.15.37:0/4208658728 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae40ab9d0 con 0x7feaf406bca0 2024-04-25T08:54:29.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 -- 172.21.15.37:0/4208658728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf406bca0 msgr2=0x7feaf40cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 --2- 172.21.15.37:0/4208658728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf406bca0 0x7feaf40cd0b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7feaf40731a0 tx=0x7feae401c3d0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 -- 172.21.15.37:0/4208658728 shutdown_connections 2024-04-25T08:54:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 --2- 172.21.15.37:0/4208658728 >> v2:172.21.15.203:3300/0 conn(0x7feaf4066e30 0x7feaf406c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 --2- 172.21.15.37:0/4208658728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf406bca0 0x7feaf40cd0b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 -- 172.21.15.37:0/4208658728 >> 172.21.15.37:0/4208658728 conn(0x7feaf4071810 msgr2=0x7feaf4071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:29.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 -- 172.21.15.37:0/4208658728 shutdown_connections 2024-04-25T08:54:29.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 -- 172.21.15.37:0/4208658728 wait complete. 2024-04-25T08:54:29.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 Processor -- start 2024-04-25T08:54:29.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 -- start start 2024-04-25T08:54:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf4066e30 0x7feaf4146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf406bca0 0x7feaf41578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf40af370 0x7feaf4156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf415cc80 con 0x7feaf406bca0 2024-04-25T08:54:29.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf415cb00 con 0x7feaf4066e30 2024-04-25T08:54:29.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaf8a6d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf4168630 con 0x7feaf40af370 2024-04-25T08:54:29.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf406bca0 0x7feaf41578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf406bca0 0x7feaf41578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52208/0 (socket says 172.21.15.37:52208) 2024-04-25T08:54:29.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.075+0000 7feaebfff640 1 -- 172.21.15.37:0/240764798 learned_addr learned my addr 172.21.15.37:0/240764798 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:29.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaf2d9d640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf40af370 0x7feaf4156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaf259c640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf4066e30 0x7feaf4146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaebfff640 1 -- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf40af370 msgr2=0x7feaf4156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaebfff640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf40af370 0x7feaf4156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaebfff640 1 -- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf4066e30 msgr2=0x7feaf4146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaebfff640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf4066e30 0x7feaf4146610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaebfff640 1 -- 172.21.15.37:0/240764798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf414c630 con 0x7feaf406bca0 2024-04-25T08:54:29.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaf2d9d640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf40af370 0x7feaf4156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:29.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaebfff640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf406bca0 0x7feaf41578f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7feadc00d620 tx=0x7feadc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:29.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaeaffd640 1 -- 172.21.15.37:0/240764798 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00b620 con 0x7feaf406bca0 2024-04-25T08:54:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaeaffd640 1 -- 172.21.15.37:0/240764798 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc019070 con 0x7feaf406bca0 2024-04-25T08:54:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaf8a6d640 1 -- 172.21.15.37:0/240764798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf414c910 con 0x7feaf406bca0 2024-04-25T08:54:29.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaeaffd640 1 -- 172.21.15.37:0/240764798 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00ba20 con 0x7feaf406bca0 2024-04-25T08:54:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaf8a6d640 1 -- 172.21.15.37:0/240764798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf4156970 con 0x7feaf406bca0 2024-04-25T08:54:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaf8a6d640 1 -- 172.21.15.37:0/240764798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf4156480 con 0x7feaf406bca0 2024-04-25T08:54:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaeaffd640 1 -- 172.21.15.37:0/240764798 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feadc00bbc0 con 0x7feaf406bca0 2024-04-25T08:54:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaeaffd640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead4066f60 0x7fead4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaeaffd640 1 -- 172.21.15.37:0/240764798 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7feadc085660 con 0x7feaf406bca0 2024-04-25T08:54:29.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaf259c640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead4066f60 0x7fead4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.079+0000 7feaf259c640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead4066f60 0x7fead4069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feae40ad000 tx=0x7feae4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:29.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.083+0000 7feaeaffd640 1 -- 172.21.15.37:0/240764798 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feadc0564b0 con 0x7feaf406bca0 2024-04-25T08:54:29.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 -- 172.21.15.37:0/240764798 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feaf404efa0 con 0x7feaf406bca0 2024-04-25T08:54:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaeaffd640 1 -- 172.21.15.37:0/240764798 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7feadc05ae20 con 0x7feaf406bca0 2024-04-25T08:54:29.265 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:29.265 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3245776986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3245776986},{"type":"v1","addr":"172.21.15.203:6836","nonce":3245776986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5056,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/841808653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":841808653},{"type":"v1","addr":"172.21.15.37:6835","nonce":841808653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1908736119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1908736119},{"type":"v1","addr":"172.21.15.203:6837","nonce":1908736119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:53:55.212059+0000","modified":"2024-04-25T08:53:58.247733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035847732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035847732},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035847732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:54:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-04-25T08:54:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 -- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead4066f60 msgr2=0x7fead4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead4066f60 0x7fead4069420 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7feae40ad000 tx=0x7feae4008040 comp rx=0 tx=0).stop 2024-04-25T08:54:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 -- 172.21.15.37:0/240764798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf406bca0 msgr2=0x7feaf41578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf406bca0 0x7feaf41578f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7feadc00d620 tx=0x7feadc00dae0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 -- 172.21.15.37:0/240764798 shutdown_connections 2024-04-25T08:54:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead4066f60 0x7fead4069420 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf40af370 0x7feaf4156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf406bca0 0x7feaf41578f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 --2- 172.21.15.37:0/240764798 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf4066e30 0x7feaf4146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 -- 172.21.15.37:0/240764798 >> 172.21.15.37:0/240764798 conn(0x7feaf4071810 msgr2=0x7feaf40cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:29.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 -- 172.21.15.37:0/240764798 shutdown_connections 2024-04-25T08:54:29.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.271+0000 7feaf8a6d640 1 -- 172.21.15.37:0/240764798 wait complete. 2024-04-25T08:54:29.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:29.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f692a2cc640 1 Processor -- start 2024-04-25T08:54:29.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f692a2cc640 1 -- start start 2024-04-25T08:54:29.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f692a2cc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f692415da80 0x7f69240d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f692a2cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d1cd0 0x7f69240d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f692a2cc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692406be60 con 0x7f69240d0620 2024-04-25T08:54:29.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f692a2cc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692406c440 con 0x7f692415da80 2024-04-25T08:54:29.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f692a2cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692415de60 con 0x7f69240d1cd0 2024-04-25T08:54:29.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6928869640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f69240d0620 0x7f69240ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36170/0 (socket says 172.21.15.37:36170) 2024-04-25T08:54:29.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6928869640 1 -- 172.21.15.37:0/2251429558 learned_addr learned my addr 172.21.15.37:0/2251429558 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:29.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f69237fe640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d1cd0 0x7f69240d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6923fff640 1 --2- 172.21.15.37:0/2251429558 >> v2:172.21.15.203:3300/0 conn(0x7f692415da80 0x7f69240d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369903189 0 0) 0x7f692406be60 con 0x7f69240d0620 2024-04-25T08:54:29.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6910003530 con 0x7f69240d0620 2024-04-25T08:54:29.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2310755567 0 0) 0x7f6910003530 con 0x7f69240d0620 2024-04-25T08:54:29.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d1cd0 msgr2=0x7f69240d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:29.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d1cd0 0x7f69240d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 >> v2:172.21.15.203:3300/0 conn(0x7f692415da80 msgr2=0x7f69240d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:29.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 --2- 172.21.15.37:0/2251429558 >> v2:172.21.15.203:3300/0 conn(0x7f692415da80 0x7f69240d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69240d2450 con 0x7f69240d0620 2024-04-25T08:54:29.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f691c001a00 con 0x7f69240d0620 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 >> v1:172.21.15.37:6789/0 conn(0x7f69240d0620 legacy=0x7f69240ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d1cd0 0x7f6910003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6910003fb0 0x7f6910004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910004d50 0x7f69100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692406c440 con 0x7f6910004d50 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692415de60 con 0x7f6910003fb0 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691000ba00 con 0x7f69240d1cd0 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6928869640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910004d50 0x7f69100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6923fff640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d1cd0 0x7f6910003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f69237fe640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6910003fb0 0x7f6910004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6928869640 1 -- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d1cd0 msgr2=0x7f6910003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6928869640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f69240d1cd0 0x7f6910003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6928869640 1 -- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6910003fb0 msgr2=0x7f6910004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6928869640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6910003fb0 0x7f6910004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f6928869640 1 -- 172.21.15.37:0/2251429558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6910003530 con 0x7f6910004d50 2024-04-25T08:54:29.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.411+0000 7f69237fe640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6910003fb0 0x7f6910004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:29.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f6928869640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910004d50 0x7f69100051d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f691c00a870 tx=0x7f691c01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4578ba63dc48191e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:29.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c00a4c0 con 0x7f6910004d50 2024-04-25T08:54:29.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c0072d0 con 0x7f6910004d50 2024-04-25T08:54:29.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f6922ffd640 1 -- 172.21.15.37:0/2251429558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c0076c0 con 0x7f6910004d50 2024-04-25T08:54:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f692a2cc640 1 -- 172.21.15.37:0/2251429558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910004d50 msgr2=0x7f69100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f692a2cc640 1 --2- 172.21.15.37:0/2251429558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6910004d50 0x7f69100051d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f691c00a870 tx=0x7f691c01d3a0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f6928869640 1 -- 172.21.15.37:0/2251429558 reap_dead start 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f692a2cc640 1 -- 172.21.15.37:0/2251429558 shutdown_connections 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f692a2cc640 1 -- 172.21.15.37:0/2251429558 >> 172.21.15.37:0/2251429558 conn(0x7f692408b000 msgr2=0x7f69240d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f692a2cc640 1 -- 172.21.15.37:0/2251429558 shutdown_connections 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f692a2cc640 1 -- 172.21.15.37:0/2251429558 wait complete. 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f692a2cc640 1 Processor -- start 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.415+0000 7f692a2cc640 1 -- start start 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f692a2cc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6910004d50 0x7f69241f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f692a2cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69240cc560 0x7f692415ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f692a2cc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69240b51c0 0x7f69240b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f692a2cc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69240d26f0 con 0x7f69240cc560 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f692a2cc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69240d20f0 con 0x7f69240b51c0 2024-04-25T08:54:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f692a2cc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69240d2870 con 0x7f6910004d50 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f69237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69240cc560 0x7f692415ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f69237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69240cc560 0x7f692415ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59688/0 (socket says 172.21.15.37:59688) 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f69237fe640 1 -- 172.21.15.37:0/2181753262 learned_addr learned my addr 172.21.15.37:0/2181753262 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f6928869640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69240b51c0 0x7f69240b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f6923fff640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6910004d50 0x7f69241f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f69237fe640 1 -- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6910004d50 msgr2=0x7f69241f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f69237fe640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6910004d50 0x7f69241f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f69237fe640 1 -- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69240b51c0 msgr2=0x7f69240b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f69237fe640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69240b51c0 0x7f69240b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f69237fe640 1 -- 172.21.15.37:0/2181753262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692415b2c0 con 0x7f69240cc560 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f6923fff640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6910004d50 0x7f69241f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f69237fe640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69240cc560 0x7f692415ad80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f690c0108c0 tx=0x7f690c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f6920ff9640 1 -- 172.21.15.37:0/2181753262 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c00f040 con 0x7f69240cc560 2024-04-25T08:54:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f6920ff9640 1 -- 172.21.15.37:0/2181753262 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c00eb00 con 0x7f69240cc560 2024-04-25T08:54:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f692a2cc640 1 -- 172.21.15.37:0/2181753262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69241f39b0 con 0x7f69240cc560 2024-04-25T08:54:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f6920ff9640 1 -- 172.21.15.37:0/2181753262 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c01c400 con 0x7f69240cc560 2024-04-25T08:54:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f692a2cc640 1 -- 172.21.15.37:0/2181753262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69241f3f70 con 0x7f69240cc560 2024-04-25T08:54:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f692a2cc640 1 -- 172.21.15.37:0/2181753262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692404efa0 con 0x7f69240cc560 2024-04-25T08:54:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f6920ff9640 1 -- 172.21.15.37:0/2181753262 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f690c01c5a0 con 0x7f69240cc560 2024-04-25T08:54:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f6920ff9640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6904066f60 0x7f6904069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f6920ff9640 1 -- 172.21.15.37:0/2181753262 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f690c0856f0 con 0x7f69240cc560 2024-04-25T08:54:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f6923fff640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6904066f60 0x7f6904069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.419+0000 7f6923fff640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6904066f60 0x7f6904069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f69241f7a30 tx=0x7f6914008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:29.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.423+0000 7f6920ff9640 1 -- 172.21.15.37:0/2181753262 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f690c018030 con 0x7f69240cc560 2024-04-25T08:54:29.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.611+0000 7f692a2cc640 1 -- 172.21.15.37:0/2181753262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69240b1a80 con 0x7f69240cc560 2024-04-25T08:54:29.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.611+0000 7f6920ff9640 1 -- 172.21.15.37:0/2181753262 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f690c0565e0 con 0x7f69240cc560 2024-04-25T08:54:29.606 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:29.606 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3245776986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3245776986},{"type":"v1","addr":"172.21.15.203:6836","nonce":3245776986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5056,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/841808653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":841808653},{"type":"v1","addr":"172.21.15.37:6835","nonce":841808653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1908736119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1908736119},{"type":"v1","addr":"172.21.15.203:6837","nonce":1908736119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:53:55.212059+0000","modified":"2024-04-25T08:53:58.247733+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5445},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5445":{"gid":5445,"name":"a","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6834/3035847732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3035847732},{"type":"v1","addr":"172.21.15.37:6834","nonce":3035847732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:54:29.606 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 17 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 -- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6904066f60 msgr2=0x7f6904069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6904066f60 0x7f6904069420 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f69241f7a30 tx=0x7f6914008040 comp rx=0 tx=0).stop 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 -- 172.21.15.37:0/2181753262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69240cc560 msgr2=0x7f692415ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69240cc560 0x7f692415ad80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f690c0108c0 tx=0x7f690c010d80 comp rx=0 tx=0).stop 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 -- 172.21.15.37:0/2181753262 shutdown_connections 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6904066f60 0x7f6904069420 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f69240b51c0 0x7f69240b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69240cc560 0x7f692415ad80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 --2- 172.21.15.37:0/2181753262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6910004d50 0x7f69241f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 -- 172.21.15.37:0/2181753262 >> 172.21.15.37:0/2181753262 conn(0x7f692408b000 msgr2=0x7f692406d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 -- 172.21.15.37:0/2181753262 shutdown_connections 2024-04-25T08:54:29.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.615+0000 7f692a2cc640 1 -- 172.21.15.37:0/2181753262 wait complete. 2024-04-25T08:54:29.624 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:29.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8cd1f640 1 Processor -- start 2024-04-25T08:54:29.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8cd1f640 1 -- start start 2024-04-25T08:54:29.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8cd1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b880b3f50 0x7f0b8815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8cd1f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0b880b24a0 0x7f0b880b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8cd1f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b880b2090 con 0x7f0b8815a700 2024-04-25T08:54:29.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8cd1f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8815f200 con 0x7f0b880b24a0 2024-04-25T08:54:29.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8cd1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b880b43e0 con 0x7f0b880b3f50 2024-04-25T08:54:29.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0b8815a700 0x7f0b880cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36176/0 (socket says 172.21.15.37:36176) 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8659c640 1 -- 172.21.15.37:0/857871602 learned_addr learned my addr 172.21.15.37:0/857871602 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b86d9d640 1 --2- 172.21.15.37:0/857871602 >> v2:172.21.15.203:3300/0 conn(0x7f0b880b24a0 0x7f0b880b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b85d9b640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b880b3f50 0x7f0b8815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113340548 0 0) 0x7f0b880b2090 con 0x7f0b8815a700 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b70003530 con 0x7f0b8815a700 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 342375386 0 0) 0x7f0b70003530 con 0x7f0b8815a700 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b880b3f50 msgr2=0x7f0b8815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b880b3f50 0x7f0b8815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 >> v2:172.21.15.203:3300/0 conn(0x7f0b880b24a0 msgr2=0x7f0b880b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 --2- 172.21.15.37:0/857871602 >> v2:172.21.15.203:3300/0 conn(0x7f0b880b24a0 0x7f0b880b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b880b46c0 con 0x7f0b8815a700 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0b78001680 con 0x7f0b8815a700 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 >> v1:172.21.15.37:6789/0 conn(0x7f0b8815a700 legacy=0x7f0b880cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b70003a30 0x7f0b70003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b70004810 0x7f0b70004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b70005190 0x7f0b7000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8815f200 con 0x7f0b70003a30 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b880b43e0 con 0x7f0b70005190 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7000bf10 con 0x7f0b70004810 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8659c640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b70003a30 0x7f0b70003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b86d9d640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b70005190 0x7f0b7000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b85d9b640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b70004810 0x7f0b70004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8659c640 1 -- 172.21.15.37:0/857871602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b70004810 msgr2=0x7f0b70004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8659c640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b70004810 0x7f0b70004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8659c640 1 -- 172.21.15.37:0/857871602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b70005190 msgr2=0x7f0b7000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8659c640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b70005190 0x7f0b7000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.743+0000 7f0b8659c640 1 -- 172.21.15.37:0/857871602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b70003530 con 0x7f0b70003a30 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8659c640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b70003a30 0x7f0b70003e50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800c210 tx=0x7f0b7801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a985e42b0478b082 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78007d60 con 0x7f0b70003a30 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b780a9020 con 0x7f0b70003a30 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8559a640 1 -- 172.21.15.37:0/857871602 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b78005b20 con 0x7f0b70003a30 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/857871602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b70003a30 msgr2=0x7f0b70003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 --2- 172.21.15.37:0/857871602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b70003a30 0x7f0b70003e50 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800c210 tx=0x7f0b7801cca0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b86d9d640 1 -- 172.21.15.37:0/857871602 reap_dead start 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/857871602 shutdown_connections 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/857871602 >> 172.21.15.37:0/857871602 conn(0x7f0b880b11a0 msgr2=0x7f0b880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/857871602 shutdown_connections 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/857871602 wait complete. 2024-04-25T08:54:29.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 Processor -- start 2024-04-25T08:54:29.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 -- start start 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b70003a30 0x7f0b881485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b881600b0 0x7f0b88153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b88150cd0 0x7f0b88151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b880cd310 con 0x7f0b88150cd0 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b880ce600 con 0x7f0b881600b0 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8cd1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b880cd490 con 0x7f0b70003a30 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b86d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b88150cd0 0x7f0b88151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b86d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b88150cd0 0x7f0b88151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59718/0 (socket says 172.21.15.37:59718) 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b86d9d640 1 -- 172.21.15.37:0/3981423618 learned_addr learned my addr 172.21.15.37:0/3981423618 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b85d9b640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b881600b0 0x7f0b88153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.747+0000 7f0b8659c640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b70003a30 0x7f0b881485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b86d9d640 1 -- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b70003a30 msgr2=0x7f0b881485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b86d9d640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b70003a30 0x7f0b881485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b86d9d640 1 -- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b881600b0 msgr2=0x7f0b88153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b86d9d640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b881600b0 0x7f0b88153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b86d9d640 1 -- 172.21.15.37:0/3981423618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b88161390 con 0x7f0b88150cd0 2024-04-25T08:54:29.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b85d9b640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b881600b0 0x7f0b88153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:29.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b86d9d640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b88150cd0 0x7f0b88151150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0b880aef60 tx=0x7f0b800115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:29.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b76ffd640 1 -- 172.21.15.37:0/3981423618 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b80011e10 con 0x7f0b88150cd0 2024-04-25T08:54:29.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b76ffd640 1 -- 172.21.15.37:0/3981423618 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8001b070 con 0x7f0b88150cd0 2024-04-25T08:54:29.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/3981423618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8814e8f0 con 0x7f0b88150cd0 2024-04-25T08:54:29.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b76ffd640 1 -- 172.21.15.37:0/3981423618 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b800039a0 con 0x7f0b88150cd0 2024-04-25T08:54:29.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/3981423618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8814eeb0 con 0x7f0b88150cd0 2024-04-25T08:54:29.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/3981423618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8804efa0 con 0x7f0b88150cd0 2024-04-25T08:54:29.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b76ffd640 1 -- 172.21.15.37:0/3981423618 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0b80003b60 con 0x7f0b88150cd0 2024-04-25T08:54:29.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b76ffd640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b68066f90 0x7f0b68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b76ffd640 1 -- 172.21.15.37:0/3981423618 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0b800866b0 con 0x7f0b88150cd0 2024-04-25T08:54:29.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b8659c640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b68066f90 0x7f0b68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.751+0000 7f0b8659c640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b68066f90 0x7f0b68069450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0b780aa000 tx=0x7f0b78000ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:29.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.755+0000 7f0b76ffd640 1 -- 172.21.15.37:0/3981423618 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0b800575f0 con 0x7f0b88150cd0 2024-04-25T08:54:29.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.915+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/3981423618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b8806f260 con 0x7f0b88150cd0 2024-04-25T08:54:29.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.915+0000 7f0b76ffd640 1 -- 172.21.15.37:0/3981423618 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+16955 (secure 0 0 0) 0x7f0b80056ef0 con 0x7f0b88150cd0 2024-04-25T08:54:29.911 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:29.911 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:54:12.248632+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:53:53.619419+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T08:53:54.197204+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.1a","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b68066f90 msgr2=0x7f0b68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b68066f90 0x7f0b68069450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0b780aa000 tx=0x7f0b78000ff0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/3981423618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b88150cd0 msgr2=0x7f0b88151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b88150cd0 0x7f0b88151150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0b880aef60 tx=0x7f0b800115a0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/3981423618 shutdown_connections 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0b68066f90 0x7f0b68069450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0b88150cd0 0x7f0b88151150 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0b881600b0 0x7f0b88153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 --2- 172.21.15.37:0/3981423618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0b70003a30 0x7f0b881485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/3981423618 >> 172.21.15.37:0/3981423618 conn(0x7f0b880b11a0 msgr2=0x7f0b88163bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/3981423618 shutdown_connections 2024-04-25T08:54:29.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:29.919+0000 7f0b8cd1f640 1 -- 172.21.15.37:0/3981423618 wait complete. 2024-04-25T08:54:29.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:54:30.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f5744922640 1 Processor -- start 2024-04-25T08:54:30.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f5744922640 1 -- start start 2024-04-25T08:54:30.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f5744922640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f574015ecc0 0x7f574006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f5744922640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57400b2090 con 0x7f574015a700 2024-04-25T08:54:30.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f5744922640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57400b43e0 con 0x7f57400b3f50 2024-04-25T08:54:30.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f5744922640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57400b46c0 con 0x7f574015ecc0 2024-04-25T08:54:30.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f573e59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f574015a700 0x7f57400cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36186/0 (socket says 172.21.15.37:36186) 2024-04-25T08:54:30.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f573e59c640 1 -- 172.21.15.37:0/3275955960 learned_addr learned my addr 172.21.15.37:0/3275955960 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:30.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f573ed9d640 1 --2- 172.21.15.37:0/3275955960 >> v2:172.21.15.203:3300/0 conn(0x7f574015ecc0 0x7f574006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124948462 0 0) 0x7f57400b2090 con 0x7f574015a700 2024-04-25T08:54:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5728003530 con 0x7f574015a700 2024-04-25T08:54:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2717829830 0 0) 0x7f5728003530 con 0x7f574015a700 2024-04-25T08:54:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 >> v2:172.21.15.203:3300/0 conn(0x7f574015ecc0 msgr2=0x7f574006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:30.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f573d59a640 1 --2- 172.21.15.37:0/3275955960 >> v2:172.21.15.203:3300/0 conn(0x7f574015ecc0 0x7f574006e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 >> v1:172.21.15.203:6789/0 conn(0x7f57400b3f50 legacy=0x7f57400b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57400cd6f0 con 0x7f574015a700 2024-04-25T08:54:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.059+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f57300016a0 con 0x7f574015a700 2024-04-25T08:54:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 >> v1:172.21.15.37:6789/0 conn(0x7f574015a700 legacy=0x7f57400cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573d59a640 1 --2- 172.21.15.37:0/3275955960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5728003a30 0x7f5728003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573d59a640 1 --2- 172.21.15.37:0/3275955960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5728004810 0x7f5728004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573d59a640 1 --2- 172.21.15.37:0/3275955960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 0x7f572800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57400b46c0 con 0x7f57280052d0 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572800c210 con 0x7f5728003a30 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572800c410 con 0x7f5728004810 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573ed9d640 1 --2- 172.21.15.37:0/3275955960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 0x7f572800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573e59c640 1 --2- 172.21.15.37:0/3275955960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5728003a30 0x7f5728003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573dd9b640 1 --2- 172.21.15.37:0/3275955960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5728004810 0x7f5728004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573ed9d640 1 -- 172.21.15.37:0/3275955960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5728004810 msgr2=0x7f5728004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573ed9d640 1 --2- 172.21.15.37:0/3275955960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5728004810 0x7f5728004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573ed9d640 1 -- 172.21.15.37:0/3275955960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5728003a30 msgr2=0x7f5728003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573ed9d640 1 --2- 172.21.15.37:0/3275955960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5728003a30 0x7f5728003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573ed9d640 1 -- 172.21.15.37:0/3275955960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5728003530 con 0x7f57280052d0 2024-04-25T08:54:30.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573ed9d640 1 -- 172.21.15.37:0/3275955960 reap_dead start 2024-04-25T08:54:30.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573ed9d640 1 --2- 172.21.15.37:0/3275955960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 0x7f572800b9d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5738009fc0 tx=0x7f573801cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee01d0a23a4cfb19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:30.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738009c10 con 0x7f57280052d0 2024-04-25T08:54:30.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5738003ba0 con 0x7f57280052d0 2024-04-25T08:54:30.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f573d59a640 1 -- 172.21.15.37:0/3275955960 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57380a5990 con 0x7f57280052d0 2024-04-25T08:54:30.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f5744922640 1 -- 172.21.15.37:0/3275955960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 msgr2=0x7f572800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f5744922640 1 --2- 172.21.15.37:0/3275955960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 0x7f572800b9d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f5738009fc0 tx=0x7f573801cc90 comp rx=0 tx=0).stop 2024-04-25T08:54:30.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f5744922640 1 -- 172.21.15.37:0/3275955960 shutdown_connections 2024-04-25T08:54:30.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f5744922640 1 --2- 172.21.15.37:0/3275955960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 0x7f572800b9d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f5744922640 1 -- 172.21.15.37:0/3275955960 >> 172.21.15.37:0/3275955960 conn(0x7f57400b11a0 msgr2=0x7f57400b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f5744922640 1 -- 172.21.15.37:0/3275955960 shutdown_connections 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.063+0000 7f5744922640 1 -- 172.21.15.37:0/3275955960 wait complete. 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f5744922640 1 Processor -- start 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f5744922640 1 -- start start 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f5744922640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 0x7f574014dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f5744922640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f574014e3e0 0x7f5740150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f5744922640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5740150dd0 0x7f574013ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f5744922640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57400cd490 con 0x7f57280052d0 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f5744922640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57400cd310 con 0x7f5740150dd0 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f5744922640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574006e1a0 con 0x7f574014e3e0 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f573e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 0x7f574014dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f573e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 0x7f574014dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59744/0 (socket says 172.21.15.37:59744) 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f573e59c640 1 -- 172.21.15.37:0/828334732 learned_addr learned my addr 172.21.15.37:0/828334732 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:30.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f573ed9d640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5740150dd0 0x7f574013ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f573dd9b640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f574014e3e0 0x7f5740150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f573e59c640 1 -- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f574014e3e0 msgr2=0x7f5740150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f573e59c640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f574014e3e0 0x7f5740150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f573e59c640 1 -- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5740150dd0 msgr2=0x7f574013ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f573e59c640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5740150dd0 0x7f574013ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f573e59c640 1 -- 172.21.15.37:0/828334732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57400aca10 con 0x7f57280052d0 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f573e59c640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 0x7f574014dea0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5730001000 tx=0x7f573000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f572effd640 1 -- 172.21.15.37:0/828334732 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573000c6c0 con 0x7f57280052d0 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f572effd640 1 -- 172.21.15.37:0/828334732 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5730018070 con 0x7f57280052d0 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f5744922640 1 -- 172.21.15.37:0/828334732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57400accf0 con 0x7f57280052d0 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f572effd640 1 -- 172.21.15.37:0/828334732 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573000cac0 con 0x7f57280052d0 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f5744922640 1 -- 172.21.15.37:0/828334732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57400ad2b0 con 0x7f57280052d0 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f5744922640 1 -- 172.21.15.37:0/828334732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574004efa0 con 0x7f57280052d0 2024-04-25T08:54:30.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f572effd640 1 -- 172.21.15.37:0/828334732 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f573000cc60 con 0x7f57280052d0 2024-04-25T08:54:30.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.067+0000 7f572effd640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5720066f60 0x7f5720069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.071+0000 7f572effd640 1 -- 172.21.15.37:0/828334732 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f5730084df0 con 0x7f57280052d0 2024-04-25T08:54:30.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.071+0000 7f573dd9b640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5720066f60 0x7f5720069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.071+0000 7f573dd9b640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5720066f60 0x7f5720069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f574013cad0 tx=0x7f5734009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:30.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.075+0000 7f572effd640 1 -- 172.21.15.37:0/828334732 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5730055ce0 con 0x7f57280052d0 2024-04-25T08:54:30.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.263+0000 7f5744922640 1 -- 172.21.15.37:0/828334732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f574006f260 con 0x7f57280052d0 2024-04-25T08:54:30.256 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T08:54:30.263+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:54:30.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f572effd640 1 -- 172.21.15.37:0/828334732 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f57300555e0 con 0x7f57280052d0 2024-04-25T08:54:30.259 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:54:30.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 -- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5720066f60 msgr2=0x7f5720069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5720066f60 0x7f5720069420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f574013cad0 tx=0x7f5734009040 comp rx=0 tx=0).stop 2024-04-25T08:54:30.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 -- 172.21.15.37:0/828334732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 msgr2=0x7f574014dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 0x7f574014dea0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5730001000 tx=0x7f573000ec10 comp rx=0 tx=0).stop 2024-04-25T08:54:30.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 -- 172.21.15.37:0/828334732 shutdown_connections 2024-04-25T08:54:30.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5720066f60 0x7f5720069420 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5740150dd0 0x7f574013ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f574014e3e0 0x7f5740150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 --2- 172.21.15.37:0/828334732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f57280052d0 0x7f574014dea0 secure :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5730001000 tx=0x7f573000ec10 comp rx=0 tx=0).stop 2024-04-25T08:54:30.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 -- 172.21.15.37:0/828334732 >> 172.21.15.37:0/828334732 conn(0x7f57400b11a0 msgr2=0x7f57400cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:30.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 -- 172.21.15.37:0/828334732 shutdown_connections 2024-04-25T08:54:30.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.267+0000 7f5744922640 1 -- 172.21.15.37:0/828334732 wait complete. 2024-04-25T08:54:30.269 INFO:tasks.ceph.mds.a.smithi037.stderr: -5886> 2024-04-25T08:54:00.644+0000 7f897e871640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:54:30.285 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:54:30.311 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T08:54:30.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a99377640 1 Processor -- start 2024-04-25T08:54:30.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a99377640 1 -- start start 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a99377640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3a940b3cf0 0x7f3a940cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a99377640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3a940cfad0 0x7f3a9415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a99377640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a940cbf10 con 0x7f3a9415e4f0 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a99377640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a940cff40 con 0x7f3a940b3cf0 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a99377640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a940cc1f0 con 0x7f3a940cfad0 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a92ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3a940b3cf0 0x7f3a940cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a92ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3a940b3cf0 0x7f3a940cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59772/0 (socket says 172.21.15.37:59772) 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a92ffd640 1 -- 172.21.15.37:0/3789308205 learned_addr learned my addr 172.21.15.37:0/3789308205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a937fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3a9415e4f0 0x7f3a94161370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36196/0 (socket says 172.21.15.37:36196) 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a8a7fc640 1 --2- 172.21.15.37:0/3789308205 >> v2:172.21.15.203:3300/0 conn(0x7f3a940cfad0 0x7f3a9415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a92ffd640 1 -- 172.21.15.37:0/3789308205 >> v2:172.21.15.203:3300/0 conn(0x7f3a940cfad0 msgr2=0x7f3a9415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a92ffd640 1 --2- 172.21.15.37:0/3789308205 >> v2:172.21.15.203:3300/0 conn(0x7f3a940cfad0 0x7f3a9415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a92ffd640 1 -- 172.21.15.37:0/3789308205 >> v1:172.21.15.37:6789/0 conn(0x7f3a9415e4f0 legacy=0x7f3a94161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a927fc640 1 -- 172.21.15.37:0/3789308205 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275263164 0 0) 0x7f3a940cbf10 con 0x7f3a9415e4f0 2024-04-25T08:54:30.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a92ffd640 1 -- 172.21.15.37:0/3789308205 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a940cc4d0 con 0x7f3a940b3cf0 2024-04-25T08:54:30.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.407+0000 7f3a8a7fc640 1 --2- 172.21.15.37:0/3789308205 >> v2:172.21.15.203:3300/0 conn(0x7f3a940cfad0 0x7f3a9415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:30.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a92ffd640 1 --2- 172.21.15.37:0/3789308205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a940b3cf0 0x7f3a940cf590 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3a940aedb0 tx=0x7f3a8401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec75fc80ba6c1b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:30.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a927fc640 1 -- 172.21.15.37:0/3789308205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a840b0070 con 0x7f3a940b3cf0 2024-04-25T08:54:30.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a927fc640 1 -- 172.21.15.37:0/3789308205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a840ab600 con 0x7f3a940b3cf0 2024-04-25T08:54:30.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a927fc640 1 -- 172.21.15.37:0/3789308205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a840ab9d0 con 0x7f3a940b3cf0 2024-04-25T08:54:30.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a99377640 1 -- 172.21.15.37:0/3789308205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a940b3cf0 msgr2=0x7f3a940cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a99377640 1 --2- 172.21.15.37:0/3789308205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a940b3cf0 0x7f3a940cf590 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3a940aedb0 tx=0x7f3a8401c520 comp rx=0 tx=0).stop 2024-04-25T08:54:30.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a99377640 1 -- 172.21.15.37:0/3789308205 shutdown_connections 2024-04-25T08:54:30.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a99377640 1 --2- 172.21.15.37:0/3789308205 >> v2:172.21.15.203:3300/0 conn(0x7f3a940cfad0 0x7f3a9415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a99377640 1 --2- 172.21.15.37:0/3789308205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a940b3cf0 0x7f3a940cf590 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a99377640 1 -- 172.21.15.37:0/3789308205 >> 172.21.15.37:0/3789308205 conn(0x7f3a9406ae60 msgr2=0x7f3a9406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:30.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a99377640 1 -- 172.21.15.37:0/3789308205 shutdown_connections 2024-04-25T08:54:30.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a99377640 1 -- 172.21.15.37:0/3789308205 wait complete. 2024-04-25T08:54:30.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a99377640 1 Processor -- start 2024-04-25T08:54:30.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.411+0000 7f3a99377640 1 -- start start 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a99377640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a940b3cf0 0x7f3a9415fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a99377640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a940cfad0 0x7f3a940989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a99377640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9415e4f0 0x7f3a94158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a99377640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940b4250 con 0x7f3a940b3cf0 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a99377640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a940b40d0 con 0x7f3a9415e4f0 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a99377640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9415e8d0 con 0x7f3a940cfad0 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a92ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a940b3cf0 0x7f3a9415fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a92ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a940b3cf0 0x7f3a9415fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59776/0 (socket says 172.21.15.37:59776) 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a92ffd640 1 -- 172.21.15.37:0/1292089555 learned_addr learned my addr 172.21.15.37:0/1292089555 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a8a7fc640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a940cfad0 0x7f3a940989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a92ffd640 1 -- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a940cfad0 msgr2=0x7f3a940989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a92ffd640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a940cfad0 0x7f3a940989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a92ffd640 1 -- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9415e4f0 msgr2=0x7f3a94158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a92ffd640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9415e4f0 0x7f3a94158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a92ffd640 1 -- 172.21.15.37:0/1292089555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a94098ef0 con 0x7f3a940b3cf0 2024-04-25T08:54:30.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a8a7fc640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a940cfad0 0x7f3a940989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:30.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a92ffd640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a940b3cf0 0x7f3a9415fc70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3a8401a150 tx=0x7f3a8401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:30.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a8bfff640 1 -- 172.21.15.37:0/1292089555 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a840bd070 con 0x7f3a940b3cf0 2024-04-25T08:54:30.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a8bfff640 1 -- 172.21.15.37:0/1292089555 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a84004030 con 0x7f3a940b3cf0 2024-04-25T08:54:30.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a99377640 1 -- 172.21.15.37:0/1292089555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a94073190 con 0x7f3a940b3cf0 2024-04-25T08:54:30.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a8bfff640 1 -- 172.21.15.37:0/1292089555 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a840b0040 con 0x7f3a940b3cf0 2024-04-25T08:54:30.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a99377640 1 -- 172.21.15.37:0/1292089555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9415c770 con 0x7f3a940b3cf0 2024-04-25T08:54:30.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a99377640 1 -- 172.21.15.37:0/1292089555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9404efa0 con 0x7f3a940b3cf0 2024-04-25T08:54:30.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a8bfff640 1 -- 172.21.15.37:0/1292089555 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a840b3070 con 0x7f3a940b3cf0 2024-04-25T08:54:30.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a8bfff640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a74066f90 0x7f3a74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a8bfff640 1 -- 172.21.15.37:0/1292089555 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f3a8411e710 con 0x7f3a940b3cf0 2024-04-25T08:54:30.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.415+0000 7f3a8a7fc640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a74066f90 0x7f3a74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.419+0000 7f3a8a7fc640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a74066f90 0x7f3a74069450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3a94160ea0 tx=0x7f3a7c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:30.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.419+0000 7f3a8bfff640 1 -- 172.21.15.37:0/1292089555 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a84016110 con 0x7f3a940b3cf0 2024-04-25T08:54:30.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:30.607+0000 7f3a99377640 1 -- 172.21.15.37:0/1292089555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3a940b3060 con 0x7f3a940b3cf0 2024-04-25T08:54:31.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.331+0000 7f3a8bfff640 1 -- 172.21.15.37:0/1292089555 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3a840ef480 con 0x7f3a940b3cf0 2024-04-25T08:54:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 -- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a74066f90 msgr2=0x7f3a74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a74066f90 0x7f3a74069450 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3a94160ea0 tx=0x7f3a7c008040 comp rx=0 tx=0).stop 2024-04-25T08:54:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 -- 172.21.15.37:0/1292089555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a940b3cf0 msgr2=0x7f3a9415fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a940b3cf0 0x7f3a9415fc70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3a8401a150 tx=0x7f3a8401ad60 comp rx=0 tx=0).stop 2024-04-25T08:54:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 -- 172.21.15.37:0/1292089555 shutdown_connections 2024-04-25T08:54:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a74066f90 0x7f3a74069450 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a9415e4f0 0x7f3a94158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a940cfad0 0x7f3a940989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 --2- 172.21.15.37:0/1292089555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a940b3cf0 0x7f3a9415fc70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 -- 172.21.15.37:0/1292089555 >> 172.21.15.37:0/1292089555 conn(0x7f3a9406ae60 msgr2=0x7f3a940ac380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:31.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 -- 172.21.15.37:0/1292089555 shutdown_connections 2024-04-25T08:54:31.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.335+0000 7f3a99377640 1 -- 172.21.15.37:0/1292089555 wait complete. 2024-04-25T08:54:31.355 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:54:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.539+0000 7f489a144640 1 Processor -- start 2024-04-25T08:54:31.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.539+0000 7f489a144640 1 -- start start 2024-04-25T08:54:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f489a144640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f48940adc70 0x7f489406e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:31.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f489a144640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48940b28c0 con 0x7f489406e9f0 2024-04-25T08:54:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f489a144640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489415bb20 con 0x7f48940cfc20 2024-04-25T08:54:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f489a144640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48940b2b60 con 0x7f48940adc70 2024-04-25T08:54:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4892ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f489406e9f0 0x7f48940cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36208/0 (socket says 172.21.15.37:36208) 2024-04-25T08:54:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4892ffd640 1 -- 172.21.15.37:0/3383802200 learned_addr learned my addr 172.21.15.37:0/3383802200 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48937fe640 1 --2- 172.21.15.37:0/3383802200 >> v2:172.21.15.203:3300/0 conn(0x7f48940adc70 0x7f489406e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 781705091 0 0) 0x7f48940b28c0 con 0x7f489406e9f0 2024-04-25T08:54:31.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f487c003530 con 0x7f489406e9f0 2024-04-25T08:54:31.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1478486408 0 0) 0x7f487c003530 con 0x7f489406e9f0 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 >> v2:172.21.15.203:3300/0 conn(0x7f48940adc70 msgr2=0x7f489406e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 --2- 172.21.15.37:0/3383802200 >> v2:172.21.15.203:3300/0 conn(0x7f48940adc70 0x7f489406e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 >> v1:172.21.15.203:6789/0 conn(0x7f48940cfc20 legacy=0x7f48940ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48940b2e40 con 0x7f489406e9f0 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f48880016a0 con 0x7f489406e9f0 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 >> v1:172.21.15.37:6789/0 conn(0x7f489406e9f0 legacy=0x7f48940cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 --2- 172.21.15.37:0/3383802200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f487c0039c0 0x7f487c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 --2- 172.21.15.37:0/3383802200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f487c004810 0x7f487c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 --2- 172.21.15.37:0/3383802200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 0x7f487c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b2b60 con 0x7f487c0051d0 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c00c070 con 0x7f487c0039c0 2024-04-25T08:54:31.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c00c270 con 0x7f487c004810 2024-04-25T08:54:31.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4893fff640 1 --2- 172.21.15.37:0/3383802200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 0x7f487c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:31.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4892ffd640 1 --2- 172.21.15.37:0/3383802200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f487c004810 0x7f487c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:31.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4893fff640 1 -- 172.21.15.37:0/3383802200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f487c004810 msgr2=0x7f487c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:31.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4893fff640 1 --2- 172.21.15.37:0/3383802200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f487c004810 0x7f487c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4893fff640 1 -- 172.21.15.37:0/3383802200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f487c0039c0 msgr2=0x7f487c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:31.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4893fff640 1 --2- 172.21.15.37:0/3383802200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f487c0039c0 0x7f487c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4893fff640 1 -- 172.21.15.37:0/3383802200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c003530 con 0x7f487c0051d0 2024-04-25T08:54:31.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4892ffd640 1 --2- 172.21.15.37:0/3383802200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f487c004810 0x7f487c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4893fff640 1 -- 172.21.15.37:0/3383802200 reap_dead start 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f4893fff640 1 --2- 172.21.15.37:0/3383802200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 0x7f487c00b9d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f488c001e70 tx=0x7f488c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01646e78cbc5b5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c001a90 con 0x7f487c0051d0 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c003ba0 con 0x7f487c0051d0 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.543+0000 7f48927fc640 1 -- 172.21.15.37:0/3383802200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c005020 con 0x7f487c0051d0 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 -- 172.21.15.37:0/3383802200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 msgr2=0x7f487c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 --2- 172.21.15.37:0/3383802200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 0x7f487c00b9d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f488c001e70 tx=0x7f488c01c780 comp rx=0 tx=0).stop 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 -- 172.21.15.37:0/3383802200 shutdown_connections 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 --2- 172.21.15.37:0/3383802200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 0x7f487c00b9d0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 -- 172.21.15.37:0/3383802200 >> 172.21.15.37:0/3383802200 conn(0x7f489406ae60 msgr2=0x7f489406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 -- 172.21.15.37:0/3383802200 shutdown_connections 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 -- 172.21.15.37:0/3383802200 wait complete. 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 Processor -- start 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 -- start start 2024-04-25T08:54:31.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 0x7f48940b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:31.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48940cec70 0x7f48940b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48940b0e30 0x7f48940ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b36e0 con 0x7f487c0051d0 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b3400 con 0x7f48940b0e30 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48940b3c00 con 0x7f48940cec70 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f48937fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 0x7f48940b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f48937fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 0x7f48940b1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59788/0 (socket says 172.21.15.37:59788) 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f48937fe640 1 -- 172.21.15.37:0/2285033465 learned_addr learned my addr 172.21.15.37:0/2285033465 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f4892ffd640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48940cec70 0x7f48940b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f4893fff640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48940b0e30 0x7f48940ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f48937fe640 1 -- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48940cec70 msgr2=0x7f48940b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f48937fe640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48940cec70 0x7f48940b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f48937fe640 1 -- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48940b0e30 msgr2=0x7f48940ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f48937fe640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48940b0e30 0x7f48940ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f48937fe640 1 -- 172.21.15.37:0/2285033465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48941f6e80 con 0x7f487c0051d0 2024-04-25T08:54:31.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f48937fe640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 0x7f48940b1720 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f488400e7c0 tx=0x7f488400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:31.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f487bfff640 1 -- 172.21.15.37:0/2285033465 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884017070 con 0x7f487c0051d0 2024-04-25T08:54:31.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f487bfff640 1 -- 172.21.15.37:0/2285033465 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400ee70 con 0x7f487c0051d0 2024-04-25T08:54:31.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 -- 172.21.15.37:0/2285033465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48941f7160 con 0x7f487c0051d0 2024-04-25T08:54:31.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f487bfff640 1 -- 172.21.15.37:0/2285033465 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400cb40 con 0x7f487c0051d0 2024-04-25T08:54:31.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.547+0000 7f489a144640 1 -- 172.21.15.37:0/2285033465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48941f7720 con 0x7f487c0051d0 2024-04-25T08:54:31.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.551+0000 7f489a144640 1 -- 172.21.15.37:0/2285033465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4894067c30 con 0x7f487c0051d0 2024-04-25T08:54:31.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.551+0000 7f487bfff640 1 -- 172.21.15.37:0/2285033465 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f488401a070 con 0x7f487c0051d0 2024-04-25T08:54:31.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.551+0000 7f487bfff640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4880066f60 0x7f4880069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:31.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.551+0000 7f487bfff640 1 -- 172.21.15.37:0/2285033465 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f48840853d0 con 0x7f487c0051d0 2024-04-25T08:54:31.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.551+0000 7f4892ffd640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4880066f60 0x7f4880069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:31.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.551+0000 7f4892ffd640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4880066f60 0x7f4880069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f48940c8ff0 tx=0x7f4888009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:31.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.555+0000 7f487bfff640 1 -- 172.21.15.37:0/2285033465 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4884056270 con 0x7f487c0051d0 2024-04-25T08:54:31.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:31.715+0000 7f489a144640 1 -- 172.21.15.37:0/2285033465 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4894071360 con 0x7f487c0051d0 2024-04-25T08:54:32.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.351+0000 7f487bfff640 1 -- 172.21.15.37:0/2285033465 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4884055b70 con 0x7f487c0051d0 2024-04-25T08:54:32.347 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:54:32.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.355+0000 7f489a144640 1 -- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4880066f60 msgr2=0x7f4880069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:32.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.355+0000 7f489a144640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4880066f60 0x7f4880069420 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f48940c8ff0 tx=0x7f4888009040 comp rx=0 tx=0).stop 2024-04-25T08:54:32.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.355+0000 7f489a144640 1 -- 172.21.15.37:0/2285033465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 msgr2=0x7f48940b1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:32.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.355+0000 7f489a144640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 0x7f48940b1720 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f488400e7c0 tx=0x7f488400ec80 comp rx=0 tx=0).stop 2024-04-25T08:54:32.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.359+0000 7f489a144640 1 -- 172.21.15.37:0/2285033465 shutdown_connections 2024-04-25T08:54:32.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.359+0000 7f489a144640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4880066f60 0x7f4880069420 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.359+0000 7f489a144640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f48940b0e30 0x7f48940ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.359+0000 7f489a144640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48940cec70 0x7f48940b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.359+0000 7f489a144640 1 --2- 172.21.15.37:0/2285033465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f487c0051d0 0x7f48940b1720 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.359+0000 7f489a144640 1 -- 172.21.15.37:0/2285033465 >> 172.21.15.37:0/2285033465 conn(0x7f489406ae60 msgr2=0x7f48940ae050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:32.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.359+0000 7f489a144640 1 -- 172.21.15.37:0/2285033465 shutdown_connections 2024-04-25T08:54:32.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.359+0000 7f489a144640 1 -- 172.21.15.37:0/2285033465 wait complete. 2024-04-25T08:54:32.374 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:54:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4034486640 1 Processor -- start 2024-04-25T08:54:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4034486640 1 -- start start 2024-04-25T08:54:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4034486640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c0d12c0 0x7f402c0d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4034486640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402c15dd60 con 0x7f402c15d980 2024-04-25T08:54:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4034486640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402c0d45e0 con 0x7f402c0d21f0 2024-04-25T08:54:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4034486640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c06c810 con 0x7f402c0d12c0 2024-04-25T08:54:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4032222640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f402c15d980 0x7f402c0d3e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36218/0 (socket says 172.21.15.37:36218) 2024-04-25T08:54:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4032222640 1 -- 172.21.15.37:0/255928288 learned_addr learned my addr 172.21.15.37:0/255928288 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031a21640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c0d12c0 0x7f402c0d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516022473 0 0) 0x7f402c15dd60 con 0x7f402c15d980 2024-04-25T08:54:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4014003530 con 0x7f402c15d980 2024-04-25T08:54:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 558868144 0 0) 0x7f4014003530 con 0x7f402c15d980 2024-04-25T08:54:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c0d12c0 msgr2=0x7f402c0d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c0d12c0 0x7f402c0d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 >> v1:172.21.15.203:6789/0 conn(0x7f402c0d21f0 legacy=0x7f402c06cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c0b02c0 con 0x7f402c15d980 2024-04-25T08:54:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f4024001a00 con 0x7f402c15d980 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 >> v1:172.21.15.37:6789/0 conn(0x7f402c15d980 legacy=0x7f402c0d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 0x7f4014003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4014004810 0x7f4014004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c0d12c0 0x7f4014005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c06c810 con 0x7f40140039f0 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401400bb00 con 0x7f4014004810 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401400bd00 con 0x7f402c0d12c0 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4032222640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 0x7f4014003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4031a21640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4014004810 0x7f4014004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4032a23640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c0d12c0 0x7f4014005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4032222640 1 -- 172.21.15.37:0/255928288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c0d12c0 msgr2=0x7f4014005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4032222640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c0d12c0 0x7f4014005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4032222640 1 -- 172.21.15.37:0/255928288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4014004810 msgr2=0x7f4014004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4032222640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4014004810 0x7f4014004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.523+0000 7f4032222640 1 -- 172.21.15.37:0/255928288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4014003530 con 0x7f40140039f0 2024-04-25T08:54:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.527+0000 7f4032222640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 0x7f4014003e10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f402400a860 tx=0x7f402401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff033ba85e9c299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.527+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402400a480 con 0x7f40140039f0 2024-04-25T08:54:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.527+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40240ad630 con 0x7f40140039f0 2024-04-25T08:54:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.527+0000 7f4031220640 1 -- 172.21.15.37:0/255928288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240ada60 con 0x7f40140039f0 2024-04-25T08:54:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.527+0000 7f4034486640 1 -- 172.21.15.37:0/255928288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 msgr2=0x7f4014003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.527+0000 7f4034486640 1 --2- 172.21.15.37:0/255928288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 0x7f4014003e10 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f402400a860 tx=0x7f402401d380 comp rx=0 tx=0).stop 2024-04-25T08:54:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.527+0000 7f4032a23640 1 -- 172.21.15.37:0/255928288 reap_dead start 2024-04-25T08:54:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.527+0000 7f4034486640 1 -- 172.21.15.37:0/255928288 shutdown_connections 2024-04-25T08:54:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.527+0000 7f4034486640 1 -- 172.21.15.37:0/255928288 >> 172.21.15.37:0/255928288 conn(0x7f402c08b000 msgr2=0x7f402c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.527+0000 7f4034486640 1 -- 172.21.15.37:0/255928288 shutdown_connections 2024-04-25T08:54:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.527+0000 7f4034486640 1 -- 172.21.15.37:0/255928288 wait complete. 2024-04-25T08:54:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4034486640 1 Processor -- start 2024-04-25T08:54:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4034486640 1 -- start start 2024-04-25T08:54:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4034486640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 0x7f402c0c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4034486640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f402c1676d0 0x7f402c0b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4034486640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c167b50 0x7f402c0b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4034486640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c1694b0 con 0x7f40140039f0 2024-04-25T08:54:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4034486640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c06c4a0 con 0x7f402c1676d0 2024-04-25T08:54:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4034486640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c169630 con 0x7f402c167b50 2024-04-25T08:54:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4032222640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 0x7f402c0c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4032222640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 0x7f402c0c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59800/0 (socket says 172.21.15.37:59800) 2024-04-25T08:54:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4032222640 1 -- 172.21.15.37:0/2103312751 learned_addr learned my addr 172.21.15.37:0/2103312751 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4032a23640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c167b50 0x7f402c0b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4031a21640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f402c1676d0 0x7f402c0b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4032222640 1 -- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c167b50 msgr2=0x7f402c0b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4032222640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c167b50 0x7f402c0b8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4032222640 1 -- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f402c1676d0 msgr2=0x7f402c0b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4032222640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f402c1676d0 0x7f402c0b8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4032222640 1 -- 172.21.15.37:0/2103312751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c0cd3e0 con 0x7f40140039f0 2024-04-25T08:54:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4032222640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 0x7f402c0c5ea0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f402401d1b0 tx=0x7f4014005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f402affd640 1 -- 172.21.15.37:0/2103312751 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240ad430 con 0x7f40140039f0 2024-04-25T08:54:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f402affd640 1 -- 172.21.15.37:0/2103312751 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40240bd070 con 0x7f40140039f0 2024-04-25T08:54:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4034486640 1 -- 172.21.15.37:0/2103312751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c0cb090 con 0x7f40140039f0 2024-04-25T08:54:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f402affd640 1 -- 172.21.15.37:0/2103312751 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40240017c0 con 0x7f40140039f0 2024-04-25T08:54:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4034486640 1 -- 172.21.15.37:0/2103312751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c0cb650 con 0x7f40140039f0 2024-04-25T08:54:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4034486640 1 -- 172.21.15.37:0/2103312751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402c067c30 con 0x7f40140039f0 2024-04-25T08:54:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f402affd640 1 -- 172.21.15.37:0/2103312751 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40240b5070 con 0x7f40140039f0 2024-04-25T08:54:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f402affd640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4018066f60 0x7f4018069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f402affd640 1 -- 172.21.15.37:0/2103312751 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5778+0+0 (secure 0 0 0) 0x7f402411f7b0 con 0x7f40140039f0 2024-04-25T08:54:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.531+0000 7f4031a21640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4018066f60 0x7f4018069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.535+0000 7f4031a21640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4018066f60 0x7f4018069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f402c15d380 tx=0x7f4020008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.539+0000 7f402affd640 1 -- 172.21.15.37:0/2103312751 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4024017090 con 0x7f40140039f0 2024-04-25T08:54:32.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:32.699+0000 7f4034486640 1 -- 172.21.15.37:0/2103312751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f402c15e4a0 con 0x7f40140039f0 2024-04-25T08:54:33.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.355+0000 7f402affd640 1 -- 172.21.15.37:0/2103312751 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 169+0+0 (secure 0 0 0) 0x7f40240f0920 con 0x7f40140039f0 2024-04-25T08:54:33.350 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T08:54:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.359+0000 7f4034486640 1 -- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4018066f60 msgr2=0x7f4018069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.359+0000 7f4034486640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4018066f60 0x7f4018069420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f402c15d380 tx=0x7f4020008040 comp rx=0 tx=0).stop 2024-04-25T08:54:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.359+0000 7f4034486640 1 -- 172.21.15.37:0/2103312751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 msgr2=0x7f402c0c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.359+0000 7f4034486640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 0x7f402c0c5ea0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f402401d1b0 tx=0x7f4014005840 comp rx=0 tx=0).stop 2024-04-25T08:54:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.363+0000 7f4034486640 1 -- 172.21.15.37:0/2103312751 shutdown_connections 2024-04-25T08:54:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.363+0000 7f4034486640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4018066f60 0x7f4018069420 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.363+0000 7f4034486640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f402c167b50 0x7f402c0b8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.363+0000 7f4034486640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f402c1676d0 0x7f402c0b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.363+0000 7f4034486640 1 --2- 172.21.15.37:0/2103312751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40140039f0 0x7f402c0c5ea0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.363+0000 7f4034486640 1 -- 172.21.15.37:0/2103312751 >> 172.21.15.37:0/2103312751 conn(0x7f402c08b000 msgr2=0x7f402c1607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.363+0000 7f4034486640 1 -- 172.21.15.37:0/2103312751 shutdown_connections 2024-04-25T08:54:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.363+0000 7f4034486640 1 -- 172.21.15.37:0/2103312751 wait complete. 2024-04-25T08:54:33.377 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:54:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.503+0000 7f45d7f00640 1 Processor -- start 2024-04-25T08:54:33.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.503+0000 7f45d7f00640 1 -- start start 2024-04-25T08:54:33.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d7f00640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f45d015b950 0x7f45d009a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d7f00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f45d00cfb80 0x7f45d00cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d7f00640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d00d04d0 con 0x7f45d00cf420 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d7f00640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d015bd30 con 0x7f45d015b950 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d7f00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d00cc0d0 con 0x7f45d00cfb80 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d5c9c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f45d015b950 0x7f45d009a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d549b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f45d00cf420 0x7f45d00cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36230/0 (socket says 172.21.15.37:36230) 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d549b640 1 -- 172.21.15.37:0/3179029925 learned_addr learned my addr 172.21.15.37:0/3179029925 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d649d640 1 --2- 172.21.15.37:0/3179029925 >> v2:172.21.15.203:3300/0 conn(0x7f45d00cfb80 0x7f45d00cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d4c9a640 1 -- 172.21.15.37:0/3179029925 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500508518 0 0) 0x7f45c0001120 con 0x7f45d00cf420 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d4c9a640 1 -- 172.21.15.37:0/3179029925 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45bc003530 con 0x7f45d00cf420 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d5c9c640 1 -- 172.21.15.37:0/3179029925 >> v2:172.21.15.203:3300/0 conn(0x7f45d00cfb80 msgr2=0x7f45d00cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d5c9c640 1 --2- 172.21.15.37:0/3179029925 >> v2:172.21.15.203:3300/0 conn(0x7f45d00cfb80 0x7f45d00cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d5c9c640 1 -- 172.21.15.37:0/3179029925 >> v1:172.21.15.37:6789/0 conn(0x7f45d00cf420 legacy=0x7f45d00cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d5c9c640 1 -- 172.21.15.37:0/3179029925 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d009ab10 con 0x7f45d015b950 2024-04-25T08:54:33.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d5c9c640 1 --2- 172.21.15.37:0/3179029925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d015b950 0x7f45d009a480 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f45d00b3bd0 tx=0x7f45cc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5b4b44c9eb3168 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d4c9a640 1 -- 172.21.15.37:0/3179029925 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0b0070 con 0x7f45d015b950 2024-04-25T08:54:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d4c9a640 1 -- 172.21.15.37:0/3179029925 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45cc0ab600 con 0x7f45d015b950 2024-04-25T08:54:33.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.507+0000 7f45d4c9a640 1 -- 172.21.15.37:0/3179029925 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45cc0ab9d0 con 0x7f45d015b950 2024-04-25T08:54:33.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 -- 172.21.15.37:0/3179029925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d015b950 msgr2=0x7f45d009a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 --2- 172.21.15.37:0/3179029925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d015b950 0x7f45d009a480 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f45d00b3bd0 tx=0x7f45cc01c480 comp rx=0 tx=0).stop 2024-04-25T08:54:33.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 -- 172.21.15.37:0/3179029925 shutdown_connections 2024-04-25T08:54:33.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 --2- 172.21.15.37:0/3179029925 >> v2:172.21.15.203:3300/0 conn(0x7f45d00cfb80 0x7f45d00cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 --2- 172.21.15.37:0/3179029925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d015b950 0x7f45d009a480 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 -- 172.21.15.37:0/3179029925 >> 172.21.15.37:0/3179029925 conn(0x7f45d00b30b0 msgr2=0x7f45d00b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:33.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 -- 172.21.15.37:0/3179029925 shutdown_connections 2024-04-25T08:54:33.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 -- 172.21.15.37:0/3179029925 wait complete. 2024-04-25T08:54:33.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 Processor -- start 2024-04-25T08:54:33.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 -- start start 2024-04-25T08:54:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00cf420 0x7f45d0159190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d00cfb80 0x7f45d0142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d015b950 0x7f45d0156520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00cf9b0 con 0x7f45d00cfb80 2024-04-25T08:54:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d00cf830 con 0x7f45d015b950 2024-04-25T08:54:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d006faa0 con 0x7f45d00cf420 2024-04-25T08:54:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d549b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d00cfb80 0x7f45d0142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d549b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d00cfb80 0x7f45d0142c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59820/0 (socket says 172.21.15.37:59820) 2024-04-25T08:54:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d549b640 1 -- 172.21.15.37:0/1556616474 learned_addr learned my addr 172.21.15.37:0/1556616474 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d5c9c640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00cf420 0x7f45d0159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d649d640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d015b950 0x7f45d0156520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d549b640 1 -- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00cf420 msgr2=0x7f45d0159190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d549b640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00cf420 0x7f45d0159190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d549b640 1 -- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d015b950 msgr2=0x7f45d0156520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d549b640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d015b950 0x7f45d0156520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d549b640 1 -- 172.21.15.37:0/1556616474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d0143b70 con 0x7f45d00cfb80 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d5c9c640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00cf420 0x7f45d0159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d549b640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d00cfb80 0x7f45d0142c40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f45c000c730 tx=0x7f45c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45c67fc640 1 -- 172.21.15.37:0/1556616474 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0017070 con 0x7f45d00cfb80 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45c67fc640 1 -- 172.21.15.37:0/1556616474 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c0005ae0 con 0x7f45d00cfb80 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 -- 172.21.15.37:0/1556616474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d01435b0 con 0x7f45d00cfb80 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45c67fc640 1 -- 172.21.15.37:0/1556616474 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c0007e30 con 0x7f45d00cfb80 2024-04-25T08:54:33.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.511+0000 7f45d7f00640 1 -- 172.21.15.37:0/1556616474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d0140ce0 con 0x7f45d00cfb80 2024-04-25T08:54:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.515+0000 7f45d7f00640 1 -- 172.21.15.37:0/1556616474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d004efa0 con 0x7f45d00cfb80 2024-04-25T08:54:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.515+0000 7f45c67fc640 1 -- 172.21.15.37:0/1556616474 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45c001a070 con 0x7f45d00cfb80 2024-04-25T08:54:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.515+0000 7f45c67fc640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45b4066f60 0x7f45b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.515+0000 7f45c67fc640 1 -- 172.21.15.37:0/1556616474 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f45c00841b0 con 0x7f45d00cfb80 2024-04-25T08:54:33.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.515+0000 7f45d5c9c640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45b4066f60 0x7f45b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.515+0000 7f45d5c9c640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45b4066f60 0x7f45b4069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0ad000 tx=0x7f45cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.519+0000 7f45c67fc640 1 -- 172.21.15.37:0/1556616474 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45d00d04d0 con 0x7f45d00cfb80 2024-04-25T08:54:33.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.675+0000 7f45d7f00640 1 -- 172.21.15.37:0/1556616474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f45d00d13a0 con 0x7f45d00cfb80 2024-04-25T08:54:33.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.679+0000 7f45c67fc640 1 -- 172.21.15.37:0/1556616474 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f45c00553c0 con 0x7f45d00cfb80 2024-04-25T08:54:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 -- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45b4066f60 msgr2=0x7f45b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45b4066f60 0x7f45b4069420 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f45cc0ad000 tx=0x7f45cc008040 comp rx=0 tx=0).stop 2024-04-25T08:54:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 -- 172.21.15.37:0/1556616474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d00cfb80 msgr2=0x7f45d0142c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d00cfb80 0x7f45d0142c40 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f45c000c730 tx=0x7f45c000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 -- 172.21.15.37:0/1556616474 shutdown_connections 2024-04-25T08:54:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45b4066f60 0x7f45b4069420 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45d015b950 0x7f45d0156520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45d00cfb80 0x7f45d0142c40 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 --2- 172.21.15.37:0/1556616474 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45d00cf420 0x7f45d0159190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 -- 172.21.15.37:0/1556616474 >> 172.21.15.37:0/1556616474 conn(0x7f45d00b30b0 msgr2=0x7f45d00b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:33.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 -- 172.21.15.37:0/1556616474 shutdown_connections 2024-04-25T08:54:33.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.683+0000 7f45d7f00640 1 -- 172.21.15.37:0/1556616474 wait complete. 2024-04-25T08:54:33.693 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-04-25T08:54:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.819+0000 7f7d9feb0640 1 Processor -- start 2024-04-25T08:54:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.819+0000 7f7d9feb0640 1 -- start start 2024-04-25T08:54:33.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9feb0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7d980b3f50 0x7f7d9815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9feb0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d980b24a0 0x7f7d980b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9feb0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d980b2090 con 0x7f7d9815a700 2024-04-25T08:54:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9feb0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9815f200 con 0x7f7d980b3f50 2024-04-25T08:54:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9feb0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980b43e0 con 0x7f7d980b24a0 2024-04-25T08:54:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9dc4c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7d9815a700 0x7f7d980cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36244/0 (socket says 172.21.15.37:36244) 2024-04-25T08:54:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9dc4c640 1 -- 172.21.15.37:0/375901332 learned_addr learned my addr 172.21.15.37:0/375901332 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9e44d640 1 --2- 172.21.15.37:0/375901332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d980b24a0 0x7f7d980b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9d44b640 1 --2- 172.21.15.37:0/375901332 >> v2:172.21.15.203:3300/0 conn(0x7f7d980b3f50 0x7f7d9815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262102465 0 0) 0x7f7d980b2090 con 0x7f7d9815a700 2024-04-25T08:54:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d80003530 con 0x7f7d9815a700 2024-04-25T08:54:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1359313488 0 0) 0x7f7d80003530 con 0x7f7d9815a700 2024-04-25T08:54:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d980b24a0 msgr2=0x7f7d980b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 --2- 172.21.15.37:0/375901332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d980b24a0 0x7f7d980b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 >> v2:172.21.15.203:3300/0 conn(0x7f7d980b3f50 msgr2=0x7f7d9815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 --2- 172.21.15.37:0/375901332 >> v2:172.21.15.203:3300/0 conn(0x7f7d980b3f50 0x7f7d9815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d980b46c0 con 0x7f7d9815a700 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f7d90001a20 con 0x7f7d9815a700 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 >> v1:172.21.15.37:6789/0 conn(0x7f7d9815a700 legacy=0x7f7d980cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 --2- 172.21.15.37:0/375901332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d80003a30 0x7f7d80003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 --2- 172.21.15.37:0/375901332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d80004810 0x7f7d80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 --2- 172.21.15.37:0/375901332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d980b24a0 0x7f7d800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9815f200 con 0x7f7d80003a30 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980b43e0 con 0x7f7d80004810 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8000dce0 con 0x7f7d980b24a0 2024-04-25T08:54:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9dc4c640 1 --2- 172.21.15.37:0/375901332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d80003a30 0x7f7d80003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9dc4c640 1 -- 172.21.15.37:0/375901332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d980b24a0 msgr2=0x7f7d800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9dc4c640 1 --2- 172.21.15.37:0/375901332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d980b24a0 0x7f7d800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9dc4c640 1 -- 172.21.15.37:0/375901332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d80004810 msgr2=0x7f7d80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9d44b640 1 --2- 172.21.15.37:0/375901332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d80004810 0x7f7d80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9dc4c640 1 --2- 172.21.15.37:0/375901332 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d80004810 0x7f7d80004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.823+0000 7f7d9dc4c640 1 -- 172.21.15.37:0/375901332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d80003530 con 0x7f7d80003a30 2024-04-25T08:54:33.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9dc4c640 1 --2- 172.21.15.37:0/375901332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d80003a30 0x7f7d80003e50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7d980b2260 tx=0x7f7d9001d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8097eee8eeadd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9000a500 con 0x7f7d80003a30 2024-04-25T08:54:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d900ad630 con 0x7f7d80003a30 2024-04-25T08:54:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9cc4a640 1 -- 172.21.15.37:0/375901332 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d900ada20 con 0x7f7d80003a30 2024-04-25T08:54:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 -- 172.21.15.37:0/375901332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d80003a30 msgr2=0x7f7d80003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 --2- 172.21.15.37:0/375901332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d80003a30 0x7f7d80003e50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7d980b2260 tx=0x7f7d9001d350 comp rx=0 tx=0).stop 2024-04-25T08:54:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9e44d640 1 -- 172.21.15.37:0/375901332 reap_dead start 2024-04-25T08:54:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 -- 172.21.15.37:0/375901332 shutdown_connections 2024-04-25T08:54:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 -- 172.21.15.37:0/375901332 >> 172.21.15.37:0/375901332 conn(0x7f7d980b11a0 msgr2=0x7f7d980b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 -- 172.21.15.37:0/375901332 shutdown_connections 2024-04-25T08:54:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 -- 172.21.15.37:0/375901332 wait complete. 2024-04-25T08:54:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 Processor -- start 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 -- start start 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d80003a30 0x7f7d9815d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d9814df80 0x7f7d9815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d981f6cb0 0x7f7d981f7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980ce600 con 0x7f7d981f6cb0 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d9815aae0 con 0x7f7d80003a30 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9feb0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980cd310 con 0x7f7d9814df80 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9e44d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d981f6cb0 0x7f7d981f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9e44d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d981f6cb0 0x7f7d981f7130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59844/0 (socket says 172.21.15.37:59844) 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9e44d640 1 -- 172.21.15.37:0/1123883278 learned_addr learned my addr 172.21.15.37:0/1123883278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9d44b640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d9814df80 0x7f7d9815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9dc4c640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d80003a30 0x7f7d9815d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9e44d640 1 -- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d9814df80 msgr2=0x7f7d9815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9e44d640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d9814df80 0x7f7d9815afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9e44d640 1 -- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d80003a30 msgr2=0x7f7d9815d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9e44d640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d80003a30 0x7f7d9815d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9e44d640 1 -- 172.21.15.37:0/1123883278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9815b810 con 0x7f7d981f6cb0 2024-04-25T08:54:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d9e44d640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d981f6cb0 0x7f7d981f7130 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7d9806cfe0 tx=0x7f7d8800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.827+0000 7f7d967fc640 1 -- 172.21.15.37:0/1123883278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d88017070 con 0x7f7d981f6cb0 2024-04-25T08:54:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.831+0000 7f7d967fc640 1 -- 172.21.15.37:0/1123883278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8800c680 con 0x7f7d981f6cb0 2024-04-25T08:54:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.831+0000 7f7d9feb0640 1 -- 172.21.15.37:0/1123883278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d98152120 con 0x7f7d981f6cb0 2024-04-25T08:54:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.831+0000 7f7d967fc640 1 -- 172.21.15.37:0/1123883278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800ca50 con 0x7f7d981f6cb0 2024-04-25T08:54:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.831+0000 7f7d9feb0640 1 -- 172.21.15.37:0/1123883278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d981526e0 con 0x7f7d981f6cb0 2024-04-25T08:54:33.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.831+0000 7f7d9feb0640 1 -- 172.21.15.37:0/1123883278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d981528e0 con 0x7f7d981f6cb0 2024-04-25T08:54:33.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.831+0000 7f7d967fc640 1 -- 172.21.15.37:0/1123883278 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d8801a070 con 0x7f7d981f6cb0 2024-04-25T08:54:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.831+0000 7f7d967fc640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d84066f90 0x7f7d84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.831+0000 7f7d967fc640 1 -- 172.21.15.37:0/1123883278 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7d88084d80 con 0x7f7d981f6cb0 2024-04-25T08:54:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.831+0000 7f7d9dc4c640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d84066f90 0x7f7d84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.831+0000 7f7d9dc4c640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d84066f90 0x7f7d84069450 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d900ad000 tx=0x7f7d90001e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:33.835+0000 7f7d967fc640 1 -- 172.21.15.37:0/1123883278 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d981528e0 con 0x7f7d981f6cb0 2024-04-25T08:54:34.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.051+0000 7f7d9feb0640 1 -- 172.21.15.37:0/1123883278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f7d9804efa0 con 0x7f7d981f6cb0 2024-04-25T08:54:34.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.371+0000 7f7d967fc640 1 -- 172.21.15.37:0/1123883278 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v135) v1 ==== 142+0+0 (secure 0 0 0) 0x7f7d88055ef0 con 0x7f7d981f6cb0 2024-04-25T08:54:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 -- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d84066f90 msgr2=0x7f7d84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d84066f90 0x7f7d84069450 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7d900ad000 tx=0x7f7d90001e60 comp rx=0 tx=0).stop 2024-04-25T08:54:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 -- 172.21.15.37:0/1123883278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d981f6cb0 msgr2=0x7f7d981f7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d981f6cb0 0x7f7d981f7130 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7d9806cfe0 tx=0x7f7d8800ec00 comp rx=0 tx=0).stop 2024-04-25T08:54:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 -- 172.21.15.37:0/1123883278 shutdown_connections 2024-04-25T08:54:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d84066f90 0x7f7d84069450 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d981f6cb0 0x7f7d981f7130 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d9814df80 0x7f7d9815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 --2- 172.21.15.37:0/1123883278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d80003a30 0x7f7d9815d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 -- 172.21.15.37:0/1123883278 >> 172.21.15.37:0/1123883278 conn(0x7f7d980b11a0 msgr2=0x7f7d980b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 -- 172.21.15.37:0/1123883278 shutdown_connections 2024-04-25T08:54:34.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.375+0000 7f7d9feb0640 1 -- 172.21.15.37:0/1123883278 wait complete. 2024-04-25T08:54:34.388 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T08:54:34.389 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-04-25T08:54:34.389 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:54:34.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:34.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5b12af640 1 Processor -- start 2024-04-25T08:54:34.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5b12af640 1 -- start start 2024-04-25T08:54:34.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5b12af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac071740 0x7fa5ac0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5b12af640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5ac0b24a0 0x7fa5ac0ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5b12af640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5ac0ae860 con 0x7fa5ac15da10 2024-04-25T08:54:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5b12af640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5ac15e210 con 0x7fa5ac0b24a0 2024-04-25T08:54:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5b12af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac15e4f0 con 0x7fa5ac071740 2024-04-25T08:54:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5aaffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa5ac15da10 0x7fa5ac071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36258/0 (socket says 172.21.15.37:36258) 2024-04-25T08:54:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5aaffd640 1 -- 172.21.15.37:0/2792743312 learned_addr learned my addr 172.21.15.37:0/2792743312 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5aa7fc640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac071740 0x7fa5ac0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5ab7fe640 1 --2- 172.21.15.37:0/2792743312 >> v2:172.21.15.203:3300/0 conn(0x7fa5ac0b24a0 0x7fa5ac0ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694832953 0 0) 0x7fa5ac0ae860 con 0x7fa5ac15da10 2024-04-25T08:54:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa594003530 con 0x7fa5ac15da10 2024-04-25T08:54:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2360246953 0 0) 0x7fa594003530 con 0x7fa5ac15da10 2024-04-25T08:54:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac071740 msgr2=0x7fa5ac0acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac071740 0x7fa5ac0acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 >> v2:172.21.15.203:3300/0 conn(0x7fa5ac0b24a0 msgr2=0x7fa5ac0ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 --2- 172.21.15.37:0/2792743312 >> v2:172.21.15.203:3300/0 conn(0x7fa5ac0b24a0 0x7fa5ac0ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac0ae4c0 con 0x7fa5ac15da10 2024-04-25T08:54:34.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fa59c001680 con 0x7fa5ac15da10 2024-04-25T08:54:34.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 >> v1:172.21.15.37:6789/0 conn(0x7fa5ac15da10 legacy=0x7fa5ac071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa594003a30 0x7fa594003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac071740 0x7fa594004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594004d50 0x7fa5940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac15e210 con 0x7fa594003a30 2024-04-25T08:54:34.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac15e4f0 con 0x7fa594004d50 2024-04-25T08:54:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59400bb00 con 0x7fa5ac071740 2024-04-25T08:54:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5aaffd640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa594003a30 0x7fa594003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5ab7fe640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594004d50 0x7fa5940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5aa7fc640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac071740 0x7fa594004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5aaffd640 1 -- 172.21.15.37:0/2792743312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac071740 msgr2=0x7fa594004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5aaffd640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac071740 0x7fa594004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5aaffd640 1 -- 172.21.15.37:0/2792743312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594004d50 msgr2=0x7fa5940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5aaffd640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594004d50 0x7fa5940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.523+0000 7fa5aaffd640 1 -- 172.21.15.37:0/2792743312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa594003530 con 0x7fa594003a30 2024-04-25T08:54:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.527+0000 7fa5aaffd640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa594003a30 0x7fa594003e50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa59c001060 tx=0x7fa59c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fe66e288305252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.527+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c007d80 con 0x7fa594003a30 2024-04-25T08:54:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.527+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c0a9020 con 0x7fa594003a30 2024-04-25T08:54:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.527+0000 7fa5a9ffb640 1 -- 172.21.15.37:0/2792743312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c005b00 con 0x7fa594003a30 2024-04-25T08:54:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.527+0000 7fa5b12af640 1 -- 172.21.15.37:0/2792743312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa594003a30 msgr2=0x7fa594003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.527+0000 7fa5b12af640 1 --2- 172.21.15.37:0/2792743312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa594003a30 0x7fa594003e50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa59c001060 tx=0x7fa59c01cca0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.527+0000 7fa5ab7fe640 1 -- 172.21.15.37:0/2792743312 reap_dead start 2024-04-25T08:54:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.527+0000 7fa5b12af640 1 -- 172.21.15.37:0/2792743312 shutdown_connections 2024-04-25T08:54:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.527+0000 7fa5b12af640 1 -- 172.21.15.37:0/2792743312 >> 172.21.15.37:0/2792743312 conn(0x7fa5ac0b11a0 msgr2=0x7fa5ac0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.527+0000 7fa5b12af640 1 -- 172.21.15.37:0/2792743312 shutdown_connections 2024-04-25T08:54:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.527+0000 7fa5b12af640 1 -- 172.21.15.37:0/2792743312 wait complete. 2024-04-25T08:54:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5b12af640 1 Processor -- start 2024-04-25T08:54:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5b12af640 1 -- start start 2024-04-25T08:54:34.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5b12af640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594003a30 0x7fa5ac1588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5b12af640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac1630e0 0x7fa5ac14ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5b12af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac06da30 0x7fa5ac06deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5b12af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0ad9d0 con 0x7fa5ac06da30 2024-04-25T08:54:34.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5b12af640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0ad850 con 0x7fa594003a30 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5b12af640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5ac0adb50 con 0x7fa5ac1630e0 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5ab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac06da30 0x7fa5ac06deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5ab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac06da30 0x7fa5ac06deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59848/0 (socket says 172.21.15.37:59848) 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5ab7fe640 1 -- 172.21.15.37:0/4256525307 learned_addr learned my addr 172.21.15.37:0/4256525307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5aaffd640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594003a30 0x7fa5ac1588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5aa7fc640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac1630e0 0x7fa5ac14ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5ab7fe640 1 -- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac1630e0 msgr2=0x7fa5ac14ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5ab7fe640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac1630e0 0x7fa5ac14ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5ab7fe640 1 -- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594003a30 msgr2=0x7fa5ac1588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5ab7fe640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594003a30 0x7fa5ac1588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5ab7fe640 1 -- 172.21.15.37:0/4256525307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac158df0 con 0x7fa5ac06da30 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5aaffd640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594003a30 0x7fa5ac1588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5ab7fe640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac06da30 0x7fa5ac06deb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0b43e0 tx=0x7fa5a40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5937fe640 1 -- 172.21.15.37:0/4256525307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a4011e40 con 0x7fa5ac06da30 2024-04-25T08:54:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5937fe640 1 -- 172.21.15.37:0/4256525307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a401b070 con 0x7fa5ac06da30 2024-04-25T08:54:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5b12af640 1 -- 172.21.15.37:0/4256525307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5ac159360 con 0x7fa5ac06da30 2024-04-25T08:54:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5937fe640 1 -- 172.21.15.37:0/4256525307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40039a0 con 0x7fa5ac06da30 2024-04-25T08:54:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5b12af640 1 -- 172.21.15.37:0/4256525307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5ac159920 con 0x7fa5ac06da30 2024-04-25T08:54:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.531+0000 7fa5b12af640 1 -- 172.21.15.37:0/4256525307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5ac04efa0 con 0x7fa5ac06da30 2024-04-25T08:54:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.535+0000 7fa5937fe640 1 -- 172.21.15.37:0/4256525307 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5a4003b40 con 0x7fa5ac06da30 2024-04-25T08:54:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.535+0000 7fa5937fe640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa598066f90 0x7fa598069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.535+0000 7fa5937fe640 1 -- 172.21.15.37:0/4256525307 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa5a40856e0 con 0x7fa5ac06da30 2024-04-25T08:54:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.535+0000 7fa5aaffd640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa598066f90 0x7fa598069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.535+0000 7fa5aaffd640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa598066f90 0x7fa598069450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa59c001060 tx=0x7fa59c001a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.539+0000 7fa5937fe640 1 -- 172.21.15.37:0/4256525307 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5a4025020 con 0x7fa5ac06da30 2024-04-25T08:54:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.699+0000 7fa5b12af640 1 -- 172.21.15.37:0/4256525307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5ac0b1dc0 con 0x7fa5ac06da30 2024-04-25T08:54:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5937fe640 1 -- 172.21.15.37:0/4256525307 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+13257 (secure 0 0 0) 0x7fa5a4056860 con 0x7fa5ac06da30 2024-04-25T08:54:34.696 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:34.696 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":46,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:54:33.358756+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6834/3035847732":"2024-04-26T08:54:30.268514+0000","172.21.15.37:6832/3035847732":"2024-04-26T08:54:30.268514+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 -- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa598066f90 msgr2=0x7fa598069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa598066f90 0x7fa598069450 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa59c001060 tx=0x7fa59c001a50 comp rx=0 tx=0).stop 2024-04-25T08:54:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 -- 172.21.15.37:0/4256525307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac06da30 msgr2=0x7fa5ac06deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac06da30 0x7fa5ac06deb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac0b43e0 tx=0x7fa5a40115a0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 -- 172.21.15.37:0/4256525307 shutdown_connections 2024-04-25T08:54:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa598066f90 0x7fa598069450 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5ac06da30 0x7fa5ac06deb0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5ac1630e0 0x7fa5ac14ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 --2- 172.21.15.37:0/4256525307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594003a30 0x7fa5ac1588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 -- 172.21.15.37:0/4256525307 >> 172.21.15.37:0/4256525307 conn(0x7fa5ac0b11a0 msgr2=0x7fa5ac167ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 -- 172.21.15.37:0/4256525307 shutdown_connections 2024-04-25T08:54:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.703+0000 7fa5b12af640 1 -- 172.21.15.37:0/4256525307 wait complete. 2024-04-25T08:54:34.715 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:34.716 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:34.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.847+0000 7f73c0e24640 1 Processor -- start 2024-04-25T08:54:34.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.847+0000 7f73c0e24640 1 -- start start 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73c0e24640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc06ac90 0x7f73bc06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73c0e24640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc06b5b0 con 0x7f73bc0cc130 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73c0e24640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73bc15f360 con 0x7f73bc15ca90 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73c0e24640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc15ff70 con 0x7f73bc06ac90 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73bad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f73bc0cc130 0x7f73bc0cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36262/0 (socket says 172.21.15.37:36262) 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73bad9d640 1 -- 172.21.15.37:0/1557011772 learned_addr learned my addr 172.21.15.37:0/1557011772 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73ba59c640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc06ac90 0x7f73bc06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018349072 0 0) 0x7f73bc06b5b0 con 0x7f73bc0cc130 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73a8003530 con 0x7f73bc0cc130 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2186112505 0 0) 0x7f73a8003530 con 0x7f73bc0cc130 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc06ac90 msgr2=0x7f73bc06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc06ac90 0x7f73bc06b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 >> v1:172.21.15.203:6789/0 conn(0x7f73bc15ca90 legacy=0x7f73bc0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc15f640 con 0x7f73bc0cc130 2024-04-25T08:54:34.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f73b4001a00 con 0x7f73bc0cc130 2024-04-25T08:54:34.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 >> v1:172.21.15.37:6789/0 conn(0x7f73bc0cc130 legacy=0x7f73bc0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc06ac90 0x7f73a8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73a8003fe0 0x7f73a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73a8004d50 0x7f73a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a800bb00 con 0x7f73a8004d50 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a800bcb0 con 0x7f73a8003fe0 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73a800beb0 con 0x7f73bc06ac90 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73bad9d640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73a8004d50 0x7f73a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73ba59c640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc06ac90 0x7f73a8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73b3fff640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73a8003fe0 0x7f73a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73bad9d640 1 -- 172.21.15.37:0/1557011772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc06ac90 msgr2=0x7f73a8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73bad9d640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc06ac90 0x7f73a8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73bad9d640 1 -- 172.21.15.37:0/1557011772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73a8003fe0 msgr2=0x7f73a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73bad9d640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73a8003fe0 0x7f73a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.851+0000 7f73bad9d640 1 -- 172.21.15.37:0/1557011772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73a8003530 con 0x7f73a8004d50 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73bad9d640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73a8004d50 0x7f73a80051d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0cc560 tx=0x7f73b401d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5c21c2adec0d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b400a4c0 con 0x7f73a8004d50 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73b40ad650 con 0x7f73a8004d50 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b9d9b640 1 -- 172.21.15.37:0/1557011772 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40ada80 con 0x7f73a8004d50 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 -- 172.21.15.37:0/1557011772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73a8004d50 msgr2=0x7f73a80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 --2- 172.21.15.37:0/1557011772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73a8004d50 0x7f73a80051d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f73bc0cc560 tx=0x7f73b401d3c0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73bad9d640 1 -- 172.21.15.37:0/1557011772 reap_dead start 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 -- 172.21.15.37:0/1557011772 shutdown_connections 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 -- 172.21.15.37:0/1557011772 >> 172.21.15.37:0/1557011772 conn(0x7f73bc099a70 msgr2=0x7f73bc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 -- 172.21.15.37:0/1557011772 shutdown_connections 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 -- 172.21.15.37:0/1557011772 wait complete. 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 Processor -- start 2024-04-25T08:54:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 -- start start 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73bc0c5c00 0x7f73bc0c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73bc06f740 0x7f73bc06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc0b1670 0x7f73bc0b1af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0d1320 con 0x7f73bc06f740 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc15fc00 con 0x7f73bc0c5c00 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73c0e24640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73bc0b5010 con 0x7f73bc0b1670 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73bc06f740 0x7f73bc06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73bc06f740 0x7f73bc06fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59868/0 (socket says 172.21.15.37:59868) 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b3fff640 1 -- 172.21.15.37:0/2908299234 learned_addr learned my addr 172.21.15.37:0/2908299234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73bad9d640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc0b1670 0x7f73bc0b1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b3fff640 1 -- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc0b1670 msgr2=0x7f73bc0b1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b3fff640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc0b1670 0x7f73bc0b1af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b3fff640 1 -- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73bc0c5c00 msgr2=0x7f73bc0c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73ba59c640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73bc0c5c00 0x7f73bc0c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b3fff640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73bc0c5c00 0x7f73bc0c4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b3fff640 1 -- 172.21.15.37:0/2908299234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc0b52f0 con 0x7f73bc06f740 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.855+0000 7f73b3fff640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73bc06f740 0x7f73bc06fbc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f73a400e550 tx=0x7f73a400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.859+0000 7f73b2ffd640 1 -- 172.21.15.37:0/2908299234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a40052e0 con 0x7f73bc06f740 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.859+0000 7f73b2ffd640 1 -- 172.21.15.37:0/2908299234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a4018070 con 0x7f73bc06f740 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.859+0000 7f73c0e24640 1 -- 172.21.15.37:0/2908299234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73bc0b55d0 con 0x7f73bc06f740 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.859+0000 7f73b2ffd640 1 -- 172.21.15.37:0/2908299234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a4003820 con 0x7f73bc06f740 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.859+0000 7f73c0e24640 1 -- 172.21.15.37:0/2908299234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73bc0c5fe0 con 0x7f73bc06f740 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.859+0000 7f73c0e24640 1 -- 172.21.15.37:0/2908299234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73bc0b2030 con 0x7f73bc06f740 2024-04-25T08:54:34.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.859+0000 7f73b2ffd640 1 -- 172.21.15.37:0/2908299234 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f73a40039c0 con 0x7f73bc06f740 2024-04-25T08:54:34.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.859+0000 7f73b2ffd640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f739c066f60 0x7f739c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.859+0000 7f73b2ffd640 1 -- 172.21.15.37:0/2908299234 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f73a4085660 con 0x7f73bc06f740 2024-04-25T08:54:34.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.859+0000 7f73ba59c640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f739c066f60 0x7f739c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.859+0000 7f73ba59c640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f739c066f60 0x7f739c069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009970 tx=0x7f73ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:34.863+0000 7f73b2ffd640 1 -- 172.21.15.37:0/2908299234 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73a4056850 con 0x7f73bc06f740 2024-04-25T08:54:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.023+0000 7f73c0e24640 1 -- 172.21.15.37:0/2908299234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73bc0b59b0 con 0x7f73bc06f740 2024-04-25T08:54:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.023+0000 7f73b2ffd640 1 -- 172.21.15.37:0/2908299234 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73bc0b59b0 con 0x7f73bc06f740 2024-04-25T08:54:35.016 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:54:35.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.023+0000 7f73c0e24640 1 -- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f739c066f60 msgr2=0x7f739c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.023+0000 7f73c0e24640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f739c066f60 0x7f739c069420 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f73ac009970 tx=0x7f73ac008040 comp rx=0 tx=0).stop 2024-04-25T08:54:35.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.023+0000 7f73c0e24640 1 -- 172.21.15.37:0/2908299234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73bc06f740 msgr2=0x7f73bc06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.023+0000 7f73c0e24640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73bc06f740 0x7f73bc06fbc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f73a400e550 tx=0x7f73a400ea10 comp rx=0 tx=0).stop 2024-04-25T08:54:35.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.027+0000 7f73c0e24640 1 -- 172.21.15.37:0/2908299234 shutdown_connections 2024-04-25T08:54:35.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.027+0000 7f73c0e24640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f739c066f60 0x7f739c069420 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.027+0000 7f73c0e24640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73bc0b1670 0x7f73bc0b1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.027+0000 7f73c0e24640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f73bc06f740 0x7f73bc06fbc0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.027+0000 7f73c0e24640 1 --2- 172.21.15.37:0/2908299234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73bc0c5c00 0x7f73bc0c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.027+0000 7f73c0e24640 1 -- 172.21.15.37:0/2908299234 >> 172.21.15.37:0/2908299234 conn(0x7f73bc099a70 msgr2=0x7f73bc0ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:35.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.027+0000 7f73c0e24640 1 -- 172.21.15.37:0/2908299234 shutdown_connections 2024-04-25T08:54:35.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.027+0000 7f73c0e24640 1 -- 172.21.15.37:0/2908299234 wait complete. 2024-04-25T08:54:35.037 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T08:54:35.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.167+0000 7f7a5a45b640 1 Processor -- start 2024-04-25T08:54:35.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a5a45b640 1 -- start start 2024-04-25T08:54:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a5a45b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a540ce510 0x7f7a540ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a5a45b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a540cee30 con 0x7f7a54144ca0 2024-04-25T08:54:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a5a45b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a540ccc70 con 0x7f7a541451f0 2024-04-25T08:54:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a5a45b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540cc2b0 con 0x7f7a540ce510 2024-04-25T08:54:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a537fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7a54144ca0 0x7f7a5406e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36272/0 (socket says 172.21.15.37:36272) 2024-04-25T08:54:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a537fe640 1 -- 172.21.15.37:0/2207812120 learned_addr learned my addr 172.21.15.37:0/2207812120 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a53fff640 1 --2- 172.21.15.37:0/2207812120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a540ce510 0x7f7a540ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a52ffd640 1 -- 172.21.15.37:0/2207812120 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249276016 0 0) 0x7f7a540cee30 con 0x7f7a54144ca0 2024-04-25T08:54:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a52ffd640 1 -- 172.21.15.37:0/2207812120 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a3c003530 con 0x7f7a54144ca0 2024-04-25T08:54:35.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a53fff640 1 -- 172.21.15.37:0/2207812120 >> v1:172.21.15.203:6789/0 conn(0x7f7a541451f0 legacy=0x7f7a54142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a53fff640 1 -- 172.21.15.37:0/2207812120 >> v1:172.21.15.37:6789/0 conn(0x7f7a54144ca0 legacy=0x7f7a5406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a52ffd640 1 -- 172.21.15.37:0/2207812120 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2446684133 0 0) 0x7f7a3c003530 con 0x7f7a54144ca0 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.171+0000 7f7a53fff640 1 -- 172.21.15.37:0/2207812120 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5406dc40 con 0x7f7a540ce510 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a53fff640 1 --2- 172.21.15.37:0/2207812120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a540ce510 0x7f7a540ce8f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009dc0 tx=0x7f7a4c01f0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3362063283825a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a52ffd640 1 -- 172.21.15.37:0/2207812120 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0b5070 con 0x7f7a540ce510 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a52ffd640 1 -- 172.21.15.37:0/2207812120 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c0a8c30 con 0x7f7a540ce510 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a52ffd640 1 -- 172.21.15.37:0/2207812120 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a48001680 con 0x7f7a540ce510 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2207812120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a540ce510 msgr2=0x7f7a540ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 --2- 172.21.15.37:0/2207812120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a540ce510 0x7f7a540ce8f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009dc0 tx=0x7f7a4c01f0a0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2207812120 shutdown_connections 2024-04-25T08:54:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 --2- 172.21.15.37:0/2207812120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a540ce510 0x7f7a540ce8f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2207812120 >> 172.21.15.37:0/2207812120 conn(0x7f7a540b11a0 msgr2=0x7f7a540b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2207812120 shutdown_connections 2024-04-25T08:54:35.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2207812120 wait complete. 2024-04-25T08:54:35.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 Processor -- start 2024-04-25T08:54:35.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 -- start start 2024-04-25T08:54:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a540ce510 0x7f7a541f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a54144ca0 0x7f7a540b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a541451f0 0x7f7a540a8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5406ee30 con 0x7f7a540ce510 2024-04-25T08:54:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5406e290 con 0x7f7a541451f0 2024-04-25T08:54:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a5a45b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540cc900 con 0x7f7a54144ca0 2024-04-25T08:54:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a53fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a540ce510 0x7f7a541f7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a53fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a540ce510 0x7f7a541f7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59876/0 (socket says 172.21.15.37:59876) 2024-04-25T08:54:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a53fff640 1 -- 172.21.15.37:0/2124853702 learned_addr learned my addr 172.21.15.37:0/2124853702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:35.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a537fe640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a54144ca0 0x7f7a540b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.175+0000 7f7a589f8640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a541451f0 0x7f7a540a8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a53fff640 1 -- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a54144ca0 msgr2=0x7f7a540b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a53fff640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a54144ca0 0x7f7a540b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a53fff640 1 -- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a541451f0 msgr2=0x7f7a540a8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a53fff640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a541451f0 0x7f7a540a8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a53fff640 1 -- 172.21.15.37:0/2124853702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a540b3700 con 0x7f7a540ce510 2024-04-25T08:54:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a53fff640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a540ce510 0x7f7a541f7a70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c01ccd0 tx=0x7f7a4c01be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a50ff9640 1 -- 172.21.15.37:0/2124853702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0bf030 con 0x7f7a540ce510 2024-04-25T08:54:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a50ff9640 1 -- 172.21.15.37:0/2124853702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c004040 con 0x7f7a540ce510 2024-04-25T08:54:35.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a50ff9640 1 -- 172.21.15.37:0/2124853702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c0b5040 con 0x7f7a540ce510 2024-04-25T08:54:35.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2124853702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a540b39e0 con 0x7f7a540ce510 2024-04-25T08:54:35.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2124853702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a540a8c70 con 0x7f7a540ce510 2024-04-25T08:54:35.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2124853702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a5404efa0 con 0x7f7a540ce510 2024-04-25T08:54:35.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a50ff9640 1 -- 172.21.15.37:0/2124853702 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7a4c0c2070 con 0x7f7a540ce510 2024-04-25T08:54:35.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a50ff9640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a40066f60 0x7f7a40069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a50ff9640 1 -- 172.21.15.37:0/2124853702 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7a4c122c10 con 0x7f7a540ce510 2024-04-25T08:54:35.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a537fe640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a40066f60 0x7f7a40069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.179+0000 7f7a537fe640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a40066f60 0x7f7a40069420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7a480017f0 tx=0x7f7a480015e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.183+0000 7f7a50ff9640 1 -- 172.21.15.37:0/2124853702 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7a4c018c00 con 0x7f7a540ce510 2024-04-25T08:54:35.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:35.411+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2124853702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f7a540b1cc0 con 0x7f7a540ce510 2024-04-25T08:54:36.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.383+0000 7f7a50ff9640 1 -- 172.21.15.37:0/2124853702 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v137) v1 ==== 152+0+0 (secure 0 0 0) 0x7f7a4c0f3ec0 con 0x7f7a540ce510 2024-04-25T08:54:36.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a40066f60 msgr2=0x7f7a40069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a40066f60 0x7f7a40069420 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7a480017f0 tx=0x7f7a480015e0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2124853702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a540ce510 msgr2=0x7f7a541f7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a540ce510 0x7f7a541f7a70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c01ccd0 tx=0x7f7a4c01be40 comp rx=0 tx=0).stop 2024-04-25T08:54:36.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2124853702 shutdown_connections 2024-04-25T08:54:36.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7a40066f60 0x7f7a40069420 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7a541451f0 0x7f7a540a8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7a54144ca0 0x7f7a540b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 --2- 172.21.15.37:0/2124853702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a540ce510 0x7f7a541f7a70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2124853702 >> 172.21.15.37:0/2124853702 conn(0x7f7a540b11a0 msgr2=0x7f7a54142680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2124853702 shutdown_connections 2024-04-25T08:54:36.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.387+0000 7f7a5a45b640 1 -- 172.21.15.37:0/2124853702 wait complete. 2024-04-25T08:54:36.405 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:54:36.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b27a1640 1 Processor -- start 2024-04-25T08:54:36.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b27a1640 1 -- start start 2024-04-25T08:54:36.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b27a1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc4ac15ecb0 0x7fc4ac0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b27a1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc4ac0c9ef0 0x7fc4ac0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b27a1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4ac0ca4c0 con 0x7fc4ac0cf590 2024-04-25T08:54:36.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b27a1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ac0ca780 con 0x7fc4ac0c9ef0 2024-04-25T08:54:36.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b27a1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4ac15f0f0 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b0d3e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc4ac0c9ef0 0x7fc4ac0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b0d3e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc4ac0c9ef0 0x7fc4ac0ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59890/0 (socket says 172.21.15.37:59890) 2024-04-25T08:54:36.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b0d3e640 1 -- 172.21.15.37:0/1866391876 learned_addr learned my addr 172.21.15.37:0/1866391876 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:36.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4ab7fe640 1 --2- 172.21.15.37:0/1866391876 >> v2:172.21.15.203:3300/0 conn(0x7fc4ac15ecb0 0x7fc4ac0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b0d3e640 1 -- 172.21.15.37:0/1866391876 >> v2:172.21.15.203:3300/0 conn(0x7fc4ac15ecb0 msgr2=0x7fc4ac0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:36.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b0d3e640 1 --2- 172.21.15.37:0/1866391876 >> v2:172.21.15.203:3300/0 conn(0x7fc4ac15ecb0 0x7fc4ac0c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b0d3e640 1 -- 172.21.15.37:0/1866391876 >> v1:172.21.15.203:6789/0 conn(0x7fc4ac0cf590 legacy=0x7fc4ac0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:36.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.547+0000 7fc4b0d3e640 1 -- 172.21.15.37:0/1866391876 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac0cd690 con 0x7fc4ac0c9ef0 2024-04-25T08:54:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4b0d3e640 1 --2- 172.21.15.37:0/1866391876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac0c9ef0 0x7fc4ac0ce170 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0d00e0 tx=0x7fc4a401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5734cb86b96a4a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4aaffd640 1 -- 172.21.15.37:0/1866391876 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40a5670 con 0x7fc4ac0c9ef0 2024-04-25T08:54:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4aaffd640 1 -- 172.21.15.37:0/1866391876 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a40a5810 con 0x7fc4ac0c9ef0 2024-04-25T08:54:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4aaffd640 1 -- 172.21.15.37:0/1866391876 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40a5be0 con 0x7fc4ac0c9ef0 2024-04-25T08:54:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4b27a1640 1 -- 172.21.15.37:0/1866391876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac0c9ef0 msgr2=0x7fc4ac0ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4b27a1640 1 --2- 172.21.15.37:0/1866391876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac0c9ef0 0x7fc4ac0ce170 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0d00e0 tx=0x7fc4a401c9b0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4b27a1640 1 -- 172.21.15.37:0/1866391876 shutdown_connections 2024-04-25T08:54:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4b27a1640 1 --2- 172.21.15.37:0/1866391876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac0c9ef0 0x7fc4ac0ce170 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4b27a1640 1 --2- 172.21.15.37:0/1866391876 >> v2:172.21.15.203:3300/0 conn(0x7fc4ac15ecb0 0x7fc4ac0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4b27a1640 1 -- 172.21.15.37:0/1866391876 >> 172.21.15.37:0/1866391876 conn(0x7fc4ac06ae60 msgr2=0x7fc4ac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:36.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4b27a1640 1 -- 172.21.15.37:0/1866391876 shutdown_connections 2024-04-25T08:54:36.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.551+0000 7fc4b27a1640 1 -- 172.21.15.37:0/1866391876 wait complete. 2024-04-25T08:54:36.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b27a1640 1 Processor -- start 2024-04-25T08:54:36.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b27a1640 1 -- start start 2024-04-25T08:54:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b27a1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ac0c9ef0 0x7fc4ac0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b27a1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ac0cf590 0x7fc4ac155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b27a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac15ecb0 0x7fc4ac1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b27a1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac167870 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b27a1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac1676f0 con 0x7fc4ac0c9ef0 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b27a1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac1679f0 con 0x7fc4ac0cf590 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b0d3e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac15ecb0 0x7fc4ac1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b0d3e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac15ecb0 0x7fc4ac1559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59892/0 (socket says 172.21.15.37:59892) 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b0d3e640 1 -- 172.21.15.37:0/14440976 learned_addr learned my addr 172.21.15.37:0/14440976 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4ab7fe640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ac0cf590 0x7fc4ac155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b0d3e640 1 -- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ac0cf590 msgr2=0x7fc4ac155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4abfff640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ac0c9ef0 0x7fc4ac0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b0d3e640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ac0cf590 0x7fc4ac155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b0d3e640 1 -- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ac0c9ef0 msgr2=0x7fc4ac0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b0d3e640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ac0c9ef0 0x7fc4ac0b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b0d3e640 1 -- 172.21.15.37:0/14440976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac0c6430 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b0d3e640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac15ecb0 0x7fc4ac1559d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40af040 tx=0x7fc4a4019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4a8ff9640 1 -- 172.21.15.37:0/14440976 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a40bc020 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4a8ff9640 1 -- 172.21.15.37:0/14440976 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a40b8670 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b27a1640 1 -- 172.21.15.37:0/14440976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ac15d7e0 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4a8ff9640 1 -- 172.21.15.37:0/14440976 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4009510 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b27a1640 1 -- 172.21.15.37:0/14440976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ac15ba40 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4b27a1640 1 -- 172.21.15.37:0/14440976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4ac04efa0 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.555+0000 7fc4a8ff9640 1 -- 172.21.15.37:0/14440976 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4a40bf020 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.559+0000 7fc4a8ff9640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc498066f60 0x7fc498069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.559+0000 7fc4a8ff9640 1 -- 172.21.15.37:0/14440976 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc4a411ff10 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.559+0000 7fc4abfff640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc498066f60 0x7fc498069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.559+0000 7fc4abfff640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc498066f60 0x7fc498069420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00a750 tx=0x7fc49c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.563+0000 7fc4a8ff9640 1 -- 172.21.15.37:0/14440976 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4a40c6030 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.723+0000 7fc4b27a1640 1 -- 172.21.15.37:0/14440976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc4ac0b0f10 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.727+0000 7fc4a8ff9640 1 -- 172.21.15.37:0/14440976 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc4a40165a0 con 0x7fc4ac15ecb0 2024-04-25T08:54:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 -- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc498066f60 msgr2=0x7fc498069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc498066f60 0x7fc498069420 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00a750 tx=0x7fc49c008040 comp rx=0 tx=0).stop 2024-04-25T08:54:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 -- 172.21.15.37:0/14440976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac15ecb0 msgr2=0x7fc4ac1559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac15ecb0 0x7fc4ac1559d0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc4a40af040 tx=0x7fc4a4019750 comp rx=0 tx=0).stop 2024-04-25T08:54:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 -- 172.21.15.37:0/14440976 shutdown_connections 2024-04-25T08:54:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc498066f60 0x7fc498069420 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4ac15ecb0 0x7fc4ac1559d0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4ac0cf590 0x7fc4ac155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 --2- 172.21.15.37:0/14440976 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4ac0c9ef0 0x7fc4ac0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 -- 172.21.15.37:0/14440976 >> 172.21.15.37:0/14440976 conn(0x7fc4ac06ae60 msgr2=0x7fc4ac06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 -- 172.21.15.37:0/14440976 shutdown_connections 2024-04-25T08:54:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:36.731+0000 7fc4b27a1640 1 -- 172.21.15.37:0/14440976 wait complete. 2024-04-25T08:54:36.742 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:54:36.743 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:54:36.792 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:54:36.857 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:54:36.857 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:54:36.858 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:36.864 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:54:36.864 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:54:36.864 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:36.869 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:54:36.870 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:54:36.870 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:36.912 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:54:36.912 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:54:36.912 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:36.918 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:54:36.918 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:54:36.918 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:37.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f244a24b640 1 Processor -- start 2024-04-25T08:54:37.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f244a24b640 1 -- start start 2024-04-25T08:54:37.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f244a24b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244415a700 0x7f244415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f244a24b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f24440b3f50 0x7f24440cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f244a24b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24440b2090 con 0x7f24440ce430 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f244a24b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244415f200 con 0x7f24440b3f50 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f244a24b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440b43e0 con 0x7f244415a700 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f2442ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f24440b3f50 0x7f24440cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f2443fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f24440ce430 0x7f24440b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36274/0 (socket says 172.21.15.37:36274) 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f2443fff640 1 -- 172.21.15.37:0/674422353 learned_addr learned my addr 172.21.15.37:0/674422353 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f24437fe640 1 --2- 172.21.15.37:0/674422353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244415a700 0x7f244415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f2442ffd640 1 -- 172.21.15.37:0/674422353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244415a700 msgr2=0x7f244415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f2442ffd640 1 --2- 172.21.15.37:0/674422353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244415a700 0x7f244415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f2442ffd640 1 -- 172.21.15.37:0/674422353 >> v1:172.21.15.37:6789/0 conn(0x7f24440ce430 legacy=0x7f24440b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f24427fc640 1 -- 172.21.15.37:0/674422353 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626930986 0 0) 0x7f24440b2090 con 0x7f24440ce430 2024-04-25T08:54:37.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.095+0000 7f2442ffd640 1 -- 172.21.15.37:0/674422353 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24440b46c0 con 0x7f24440b3f50 2024-04-25T08:54:37.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f2442ffd640 1 --2- 172.21.15.37:0/674422353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24440b3f50 0x7f24440cdef0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f242c00ba60 tx=0x7f242c01ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b2e3fcf0641be2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f24427fc640 1 -- 172.21.15.37:0/674422353 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0b4070 con 0x7f24440b3f50 2024-04-25T08:54:37.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f24427fc640 1 -- 172.21.15.37:0/674422353 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c0a7af0 con 0x7f24440b3f50 2024-04-25T08:54:37.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f24427fc640 1 -- 172.21.15.37:0/674422353 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0af440 con 0x7f24440b3f50 2024-04-25T08:54:37.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 -- 172.21.15.37:0/674422353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24440b3f50 msgr2=0x7f24440cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 --2- 172.21.15.37:0/674422353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24440b3f50 0x7f24440cdef0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f242c00ba60 tx=0x7f242c01ea20 comp rx=0 tx=0).stop 2024-04-25T08:54:37.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 -- 172.21.15.37:0/674422353 shutdown_connections 2024-04-25T08:54:37.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 --2- 172.21.15.37:0/674422353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f24440b3f50 0x7f24440cdef0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 --2- 172.21.15.37:0/674422353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244415a700 0x7f244415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 -- 172.21.15.37:0/674422353 >> 172.21.15.37:0/674422353 conn(0x7f24440b11a0 msgr2=0x7f24440b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:37.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 -- 172.21.15.37:0/674422353 shutdown_connections 2024-04-25T08:54:37.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 -- 172.21.15.37:0/674422353 wait complete. 2024-04-25T08:54:37.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 Processor -- start 2024-04-25T08:54:37.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 -- start start 2024-04-25T08:54:37.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24440ce430 0x7f24441f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f244415a700 0x7f24441f9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244414dac0 0x7f24440ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440cd490 con 0x7f244415a700 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440cd310 con 0x7f24440ce430 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f244a24b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24440b2bb0 con 0x7f244414dac0 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f2442ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f244415a700 0x7f24441f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f2442ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f244415a700 0x7f24441f9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59904/0 (socket says 172.21.15.37:59904) 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f2442ffd640 1 -- 172.21.15.37:0/3818320880 learned_addr learned my addr 172.21.15.37:0/3818320880 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f24437fe640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24440ce430 0x7f24441f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f2443fff640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244414dac0 0x7f24440ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f2442ffd640 1 -- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244414dac0 msgr2=0x7f24440ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f2442ffd640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244414dac0 0x7f24440ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f2442ffd640 1 -- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24440ce430 msgr2=0x7f24441f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f2442ffd640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24440ce430 0x7f24441f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f2442ffd640 1 -- 172.21.15.37:0/3818320880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24441554d0 con 0x7f244415a700 2024-04-25T08:54:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f2443fff640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244414dac0 0x7f24440ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.099+0000 7f24437fe640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24440ce430 0x7f24441f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f2442ffd640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f244415a700 0x7f24441f9970 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f242c0b2040 tx=0x7f242c01b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f243bfff640 1 -- 172.21.15.37:0/3818320880 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0b4040 con 0x7f244415a700 2024-04-25T08:54:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f243bfff640 1 -- 172.21.15.37:0/3818320880 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c0afc20 con 0x7f244415a700 2024-04-25T08:54:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f244a24b640 1 -- 172.21.15.37:0/3818320880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24440aded0 con 0x7f244415a700 2024-04-25T08:54:37.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f243bfff640 1 -- 172.21.15.37:0/3818320880 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c006e20 con 0x7f244415a700 2024-04-25T08:54:37.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f244a24b640 1 -- 172.21.15.37:0/3818320880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244415b880 con 0x7f244415a700 2024-04-25T08:54:37.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f244a24b640 1 -- 172.21.15.37:0/3818320880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f244404efa0 con 0x7f244415a700 2024-04-25T08:54:37.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f243bfff640 1 -- 172.21.15.37:0/3818320880 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f242c0c1060 con 0x7f244415a700 2024-04-25T08:54:37.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f243bfff640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2424066f90 0x7f2424069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f243bfff640 1 -- 172.21.15.37:0/3818320880 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f242c11f610 con 0x7f244415a700 2024-04-25T08:54:37.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f24437fe640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2424066f90 0x7f2424069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.103+0000 7f24437fe640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2424066f90 0x7f2424069450 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2434007330 tx=0x7f243400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.107+0000 7f243bfff640 1 -- 172.21.15.37:0/3818320880 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f242c018760 con 0x7f244415a700 2024-04-25T08:54:37.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.299+0000 7f244a24b640 1 -- 172.21.15.37:0/3818320880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24441504b0 con 0x7f244415a700 2024-04-25T08:54:37.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f243bfff640 1 -- 172.21.15.37:0/3818320880 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f242c0f0900 con 0x7f244415a700 2024-04-25T08:54:37.295 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:37.295 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3245776986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3245776986},{"type":"v1","addr":"172.21.15.203:6836","nonce":3245776986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5056,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/841808653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6833","nonce":841808653},{"type":"v1","addr":"172.21.15.37:6835","nonce":841808653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/1908736119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":1908736119},{"type":"v1","addr":"172.21.15.203:6837","nonce":1908736119}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5847,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6834/1422376480","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1422376480},{"type":"v1","addr":"172.21.15.37:6834","nonce":1422376480}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2024-04-25T08:54:37.295 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-25T08:54:37.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 -- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2424066f90 msgr2=0x7f2424069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2424066f90 0x7f2424069450 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2434007330 tx=0x7f243400b040 comp rx=0 tx=0).stop 2024-04-25T08:54:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 -- 172.21.15.37:0/3818320880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f244415a700 msgr2=0x7f24441f9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f244415a700 0x7f24441f9970 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f242c0b2040 tx=0x7f242c01b7c0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 -- 172.21.15.37:0/3818320880 shutdown_connections 2024-04-25T08:54:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2424066f90 0x7f2424069450 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f244414dac0 0x7f24440ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f244415a700 0x7f24441f9970 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 --2- 172.21.15.37:0/3818320880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f24440ce430 0x7f24441f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 -- 172.21.15.37:0/3818320880 >> 172.21.15.37:0/3818320880 conn(0x7f24440b11a0 msgr2=0x7f24440af010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 -- 172.21.15.37:0/3818320880 shutdown_connections 2024-04-25T08:54:37.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.303+0000 7f244a24b640 1 -- 172.21.15.37:0/3818320880 wait complete. 2024-04-25T08:54:37.315 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:54:37.316 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:54:37.316 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:54:37.316 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:54:37.316 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:54:37.316 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:54:37.316 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:54:37.316 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:54:37.317 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:54:37.317 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:54:37.317 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:54:37.317 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:54:37.317 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:54:37.317 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:54:37.317 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:54:37.317 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:54:37.331 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:54:37.331 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:54:37.332 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:54:37.332 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:54:37.333 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:54:37.333 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:54:37.334 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:54:37.334 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:54:37.377 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:54:37.377 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:37.378 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:54:37.378 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:37.380 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:54:37.380 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:37.381 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:54:37.381 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:37.381 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:54:37.440 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T08:54:37.441 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T08:54:37.445 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T08:54:37.448 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-25T08:54:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.499+0000 7f3a93afb640 1 Processor -- start 2024-04-25T08:54:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.499+0000 7f3a93afb640 1 -- start start 2024-04-25T08:54:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3a8c15da80 0x7f3a8c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3a8c0d07f0 0x7f3a8c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a8c06be60 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8c06c440 con 0x7f3a8c0d07f0 2024-04-25T08:54:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a8c15de60 con 0x7f3a8c15da80 2024-04-25T08:54:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a92098640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3a8c0d07f0 0x7f3a8c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a91096640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3a8c0d1cd0 0x7f3a8c0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36294/0 (socket says 172.21.15.37:36294) 2024-04-25T08:54:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a91096640 1 -- 172.21.15.37:0/998411964 learned_addr learned my addr 172.21.15.37:0/998411964 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a91897640 1 --2- 172.21.15.37:0/998411964 >> v2:172.21.15.203:3300/0 conn(0x7f3a8c15da80 0x7f3a8c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a92098640 1 -- 172.21.15.37:0/998411964 >> v2:172.21.15.203:3300/0 conn(0x7f3a8c15da80 msgr2=0x7f3a8c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a92098640 1 --2- 172.21.15.37:0/998411964 >> v2:172.21.15.203:3300/0 conn(0x7f3a8c15da80 0x7f3a8c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a92098640 1 -- 172.21.15.37:0/998411964 >> v1:172.21.15.37:6789/0 conn(0x7f3a8c0d1cd0 legacy=0x7f3a8c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a92098640 1 -- 172.21.15.37:0/998411964 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c0d2450 con 0x7f3a8c0d07f0 2024-04-25T08:54:37.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a92098640 1 --2- 172.21.15.37:0/998411964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a8c0d07f0 0x7f3a8c06c8d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3a840097c0 tx=0x7f3a8401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d1d5aa8500b90a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a90895640 1 -- 172.21.15.37:0/998411964 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a840b1070 con 0x7f3a8c0d07f0 2024-04-25T08:54:37.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a90895640 1 -- 172.21.15.37:0/998411964 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a840ac5f0 con 0x7f3a8c0d07f0 2024-04-25T08:54:37.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a90895640 1 -- 172.21.15.37:0/998411964 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a840ac9c0 con 0x7f3a8c0d07f0 2024-04-25T08:54:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 -- 172.21.15.37:0/998411964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a8c0d07f0 msgr2=0x7f3a8c06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 --2- 172.21.15.37:0/998411964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a8c0d07f0 0x7f3a8c06c8d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3a840097c0 tx=0x7f3a8401c430 comp rx=0 tx=0).stop 2024-04-25T08:54:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 -- 172.21.15.37:0/998411964 shutdown_connections 2024-04-25T08:54:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 --2- 172.21.15.37:0/998411964 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a8c0d07f0 0x7f3a8c06c8d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 --2- 172.21.15.37:0/998411964 >> v2:172.21.15.203:3300/0 conn(0x7f3a8c15da80 0x7f3a8c0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 -- 172.21.15.37:0/998411964 >> 172.21.15.37:0/998411964 conn(0x7f3a8c08b000 msgr2=0x7f3a8c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 -- 172.21.15.37:0/998411964 shutdown_connections 2024-04-25T08:54:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.503+0000 7f3a93afb640 1 -- 172.21.15.37:0/998411964 wait complete. 2024-04-25T08:54:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a93afb640 1 Processor -- start 2024-04-25T08:54:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a93afb640 1 -- start start 2024-04-25T08:54:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a93afb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a8c0d07f0 0x7f3a8c159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a93afb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a8c0d1cd0 0x7f3a8c0b8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a93afb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c15da80 0x7f3a8c0ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a93afb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c06d7a0 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a93afb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c06d620 con 0x7f3a8c15da80 2024-04-25T08:54:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a93afb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c06d920 con 0x7f3a8c0d07f0 2024-04-25T08:54:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a91096640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a8c0d1cd0 0x7f3a8c0b8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a91096640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a8c0d1cd0 0x7f3a8c0b8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59956/0 (socket says 172.21.15.37:59956) 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a91096640 1 -- 172.21.15.37:0/2594082909 learned_addr learned my addr 172.21.15.37:0/2594082909 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a92098640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c15da80 0x7f3a8c0ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a91897640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a8c0d07f0 0x7f3a8c159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a91096640 1 -- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a8c0d07f0 msgr2=0x7f3a8c159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a91096640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a8c0d07f0 0x7f3a8c159f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a91096640 1 -- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c15da80 msgr2=0x7f3a8c0ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a91096640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c15da80 0x7f3a8c0ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a91096640 1 -- 172.21.15.37:0/2594082909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c15e170 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a92098640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c15da80 0x7f3a8c0ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a91096640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a8c0d1cd0 0x7f3a8c0b8cd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00c400 tx=0x7f3a7c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a827fc640 1 -- 172.21.15.37:0/2594082909 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c017070 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a827fc640 1 -- 172.21.15.37:0/2594082909 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7c0074e0 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a93afb640 1 -- 172.21.15.37:0/2594082909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8c15e450 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a827fc640 1 -- 172.21.15.37:0/2594082909 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0059d0 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a93afb640 1 -- 172.21.15.37:0/2594082909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c15ea10 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.507+0000 7f3a93afb640 1 -- 172.21.15.37:0/2594082909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8c04efa0 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.511+0000 7f3a827fc640 1 -- 172.21.15.37:0/2594082909 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a7c01a070 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.511+0000 7f3a827fc640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a70066f90 0x7f3a70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.511+0000 7f3a827fc640 1 -- 172.21.15.37:0/2594082909 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3a7c084220 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.511+0000 7f3a91897640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a70066f90 0x7f3a70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.511+0000 7f3a91897640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a70066f90 0x7f3a70069450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0b8ac0 tx=0x7f3a8800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.515+0000 7f3a827fc640 1 -- 172.21.15.37:0/2594082909 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a7c055430 con 0x7f3a8c0d1cd0 2024-04-25T08:54:37.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:37.683+0000 7f3a93afb640 1 -- 172.21.15.37:0/2594082909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3a8c0b20f0 con 0x7f3a8c0d1cd0 2024-04-25T08:54:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.755+0000 7f3a827fc640 1 -- 172.21.15.37:0/2594082909 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v51) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3a7c054d30 con 0x7f3a8c0d1cd0 2024-04-25T08:54:38.748 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-04-25T08:54:38.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 -- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a70066f90 msgr2=0x7f3a70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a70066f90 0x7f3a70069450 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0b8ac0 tx=0x7f3a8800b040 comp rx=0 tx=0).stop 2024-04-25T08:54:38.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 -- 172.21.15.37:0/2594082909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a8c0d1cd0 msgr2=0x7f3a8c0b8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a8c0d1cd0 0x7f3a8c0b8cd0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00c400 tx=0x7f3a7c00c8c0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 -- 172.21.15.37:0/2594082909 shutdown_connections 2024-04-25T08:54:38.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a70066f90 0x7f3a70069450 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a8c15da80 0x7f3a8c0ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a8c0d1cd0 0x7f3a8c0b8cd0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 --2- 172.21.15.37:0/2594082909 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a8c0d07f0 0x7f3a8c159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 -- 172.21.15.37:0/2594082909 >> 172.21.15.37:0/2594082909 conn(0x7f3a8c08b000 msgr2=0x7f3a8c0b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:38.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 -- 172.21.15.37:0/2594082909 shutdown_connections 2024-04-25T08:54:38.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.759+0000 7f3a93afb640 1 -- 172.21.15.37:0/2594082909 wait complete. 2024-04-25T08:54:38.775 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:54:38.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.899+0000 7fc64f73a640 1 Processor -- start 2024-04-25T08:54:38.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.899+0000 7fc64f73a640 1 -- start start 2024-04-25T08:54:38.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64f73a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6480b3f50 0x7fc64815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64f73a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc6480b24a0 0x7fc6480b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64f73a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6480b2090 con 0x7fc64815a700 2024-04-25T08:54:38.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64f73a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64815f200 con 0x7fc6480b24a0 2024-04-25T08:54:38.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64f73a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480b43e0 con 0x7fc6480b3f50 2024-04-25T08:54:38.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64ccd5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6480b3f50 0x7fc64815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64dcd7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc6480b24a0 0x7fc6480b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64dcd7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc6480b24a0 0x7fc6480b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:41040/0 (socket says 172.21.15.37:41040) 2024-04-25T08:54:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64ccd5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6480b3f50 0x7fc64815ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:52296/0 (socket says 172.21.15.37:52296) 2024-04-25T08:54:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64d4d6640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fc64815a700 0x7fc6480cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:57980/0 (socket says 172.21.15.37:57980) 2024-04-25T08:54:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64dcd7640 1 -- 172.21.15.37:0/3280732794 learned_addr learned my addr 172.21.15.37:0/3280732794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64ccd5640 1 -- 172.21.15.37:0/3280732794 >> v2:172.21.15.203:3300/0 conn(0x7fc6480b24a0 msgr2=0x7fc6480b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64ccd5640 1 --2- 172.21.15.37:0/3280732794 >> v2:172.21.15.203:3300/0 conn(0x7fc6480b24a0 0x7fc6480b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64ccd5640 1 -- 172.21.15.37:0/3280732794 >> v1:172.21.15.203:6789/0 conn(0x7fc64815a700 legacy=0x7fc6480cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64ccd5640 1 -- 172.21.15.37:0/3280732794 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6480b46c0 con 0x7fc6480b3f50 2024-04-25T08:54:38.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64dcd7640 1 --2- 172.21.15.37:0/3280732794 >> v2:172.21.15.203:3300/0 conn(0x7fc6480b24a0 0x7fc6480b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:38.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc64ccd5640 1 --2- 172.21.15.37:0/3280732794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6480b3f50 0x7fc64815ecc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6380097c0 tx=0x7fc63801c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19319e40ac535d49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc63ffff640 1 -- 172.21.15.37:0/3280732794 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6380b0070 con 0x7fc6480b3f50 2024-04-25T08:54:38.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc63ffff640 1 -- 172.21.15.37:0/3280732794 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6380ab600 con 0x7fc6480b3f50 2024-04-25T08:54:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.903+0000 7fc63ffff640 1 -- 172.21.15.37:0/3280732794 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6380ab9d0 con 0x7fc6480b3f50 2024-04-25T08:54:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 -- 172.21.15.37:0/3280732794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6480b3f50 msgr2=0x7fc64815ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 --2- 172.21.15.37:0/3280732794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6480b3f50 0x7fc64815ecc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc6380097c0 tx=0x7fc63801c380 comp rx=0 tx=0).stop 2024-04-25T08:54:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 -- 172.21.15.37:0/3280732794 shutdown_connections 2024-04-25T08:54:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 --2- 172.21.15.37:0/3280732794 >> v2:172.21.15.203:3300/0 conn(0x7fc6480b24a0 0x7fc6480b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 --2- 172.21.15.37:0/3280732794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6480b3f50 0x7fc64815ecc0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 -- 172.21.15.37:0/3280732794 >> 172.21.15.37:0/3280732794 conn(0x7fc6480b11a0 msgr2=0x7fc6480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 -- 172.21.15.37:0/3280732794 shutdown_connections 2024-04-25T08:54:38.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 -- 172.21.15.37:0/3280732794 wait complete. 2024-04-25T08:54:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 Processor -- start 2024-04-25T08:54:38.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 -- start start 2024-04-25T08:54:38.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6480b24a0 0x7fc6481520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64815a700 0x7fc6480aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc648156f70 0x7fc6480aa720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480cd310 con 0x7fc64815a700 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480ce600 con 0x7fc6480b24a0 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480cd490 con 0x7fc648156f70 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64ccd5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64815a700 0x7fc6480aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64ccd5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64815a700 0x7fc6480aa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59970/0 (socket says 172.21.15.37:59970) 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64ccd5640 1 -- 172.21.15.37:0/16815461 learned_addr learned my addr 172.21.15.37:0/16815461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64d4d6640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6480b24a0 0x7fc6481520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64dcd7640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc648156f70 0x7fc6480aa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64ccd5640 1 -- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc648156f70 msgr2=0x7fc6480aa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64ccd5640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc648156f70 0x7fc6480aa720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64ccd5640 1 -- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6480b24a0 msgr2=0x7fc6481520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64ccd5640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6480b24a0 0x7fc6481520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64ccd5640 1 -- 172.21.15.37:0/16815461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc648157530 con 0x7fc64815a700 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64ccd5640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64815a700 0x7fc6480aa1e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc638018f90 tx=0x7fc638019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc63dffb640 1 -- 172.21.15.37:0/16815461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6380b8020 con 0x7fc64815a700 2024-04-25T08:54:38.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc63dffb640 1 -- 172.21.15.37:0/16815461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6380b4600 con 0x7fc64815a700 2024-04-25T08:54:38.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 -- 172.21.15.37:0/16815461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6480a9130 con 0x7fc64815a700 2024-04-25T08:54:38.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc63dffb640 1 -- 172.21.15.37:0/16815461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6380b0040 con 0x7fc64815a700 2024-04-25T08:54:38.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.907+0000 7fc64f73a640 1 -- 172.21.15.37:0/16815461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc648153100 con 0x7fc64815a700 2024-04-25T08:54:38.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.911+0000 7fc64f73a640 1 -- 172.21.15.37:0/16815461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6480a9740 con 0x7fc64815a700 2024-04-25T08:54:38.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.911+0000 7fc63dffb640 1 -- 172.21.15.37:0/16815461 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc6380bb020 con 0x7fc64815a700 2024-04-25T08:54:38.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.911+0000 7fc63dffb640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc62c066f60 0x7fc62c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.911+0000 7fc63dffb640 1 -- 172.21.15.37:0/16815461 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc63811f4d0 con 0x7fc64815a700 2024-04-25T08:54:38.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.911+0000 7fc64d4d6640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc62c066f60 0x7fc62c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.911+0000 7fc64d4d6640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc62c066f60 0x7fc62c069420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc644000f70 tx=0x7fc644007c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:38.915+0000 7fc63dffb640 1 -- 172.21.15.37:0/16815461 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc638016090 con 0x7fc64815a700 2024-04-25T08:54:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.131+0000 7fc64f73a640 1 -- 172.21.15.37:0/16815461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc64804efa0 con 0x7fc64815a700 2024-04-25T08:54:39.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.131+0000 7fc63dffb640 1 -- 172.21.15.37:0/16815461 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc6380f0790 con 0x7fc64815a700 2024-04-25T08:54:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:39.125 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:54:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-04-25T08:54:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-04-25T08:54:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpm2pLfABAAJbn2dGKdl8SHS8pApIWC1Q==", 2024-04-25T08:54:39.125 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmqB+jDxAAmRnBzN3vqO8HIf/cd3QoJQ==", 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:54:39.126 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmiXRqCRAACtY8cFxwqOT4WNzj9WmeKA==", 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmX3ARGBAAW/ZUr85NsKtd8HzxE+Yefw==", 2024-04-25T08:54:39.127 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCSGSpmwhMiJhAAbmQz36LujHeYXaKIOdLLSA==", 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCVGSpm/OLPKxAA9ZmrrWxvt4QDj2lHATnzSg==", 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCYGSpmq7tEMxAA/wjwgkrtnTkrMoC6G+YIlQ==", 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:54:39.129 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:54:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-04-25T08:54:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCbGSpmReB1ORAAkXje/r11I3ArZHErIvijgw==", 2024-04-25T08:54:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:54:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:54:39.130 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQChGSpm4XfLORAAGj+R1Nk/EGmvEdpt8kQJlg==", 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-04-25T08:54:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQClGSpmd5f1ARAAKF3XG//v3Uqx4bckcD4JMw==", 2024-04-25T08:54:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:54:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:54:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-04-25T08:54:39.132 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCoGSpmi9OECBAAJBskMWGCNlAaej9xUc6g8w==", 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCrGSpmunQiDhAA3UZo2vzT44HEYUBUia75NA==", 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA==", 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmu8UoJhAAyutD5hzKLZmr57WuAWScuQ==", 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.134 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmGVVdIxAAY7/VOuqF5OoomqqVxE1Ifw==", 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmjFNDKxAAmo6DB9W17TZaGlvkdqkSuQ==", 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm+mGgJRAA9oTRTlZpGBdCZ0/bj580Bg==", 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:54:39.136 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpmLChTORAAo6fs/9PDJ5YqqcbfHfPO9Q==", 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-04-25T08:54:39.137 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm3NLHNhAAeqSA658L6P9OcrSO5JVzLg==", 2024-04-25T08:54:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:54:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:54:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T08:54:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:54:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:39.138 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T08:54:39.138 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:54:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.131+0000 7fc64f73a640 1 -- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc62c066f60 msgr2=0x7fc62c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.131+0000 7fc64f73a640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc62c066f60 0x7fc62c069420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc644000f70 tx=0x7fc644007c90 comp rx=0 tx=0).stop 2024-04-25T08:54:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.135+0000 7fc64f73a640 1 -- 172.21.15.37:0/16815461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64815a700 msgr2=0x7fc6480aa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.135+0000 7fc64f73a640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64815a700 0x7fc6480aa1e0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc638018f90 tx=0x7fc638019120 comp rx=0 tx=0).stop 2024-04-25T08:54:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.135+0000 7fc64f73a640 1 -- 172.21.15.37:0/16815461 shutdown_connections 2024-04-25T08:54:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.135+0000 7fc64f73a640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc62c066f60 0x7fc62c069420 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.135+0000 7fc64f73a640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc648156f70 0x7fc6480aa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.135+0000 7fc64f73a640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc64815a700 0x7fc6480aa1e0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.135+0000 7fc64f73a640 1 --2- 172.21.15.37:0/16815461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc6480b24a0 0x7fc6481520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.135+0000 7fc64f73a640 1 -- 172.21.15.37:0/16815461 >> 172.21.15.37:0/16815461 conn(0x7fc6480b11a0 msgr2=0x7fc6480b0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.135+0000 7fc64f73a640 1 -- 172.21.15.37:0/16815461 shutdown_connections 2024-04-25T08:54:39.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.135+0000 7fc64f73a640 1 -- 172.21.15.37:0/16815461 wait complete. 2024-04-25T08:54:39.143 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0b00ac640 1 Processor -- start 2024-04-25T08:54:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0b00ac640 1 -- start start 2024-04-25T08:54:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0b00ac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a8144ca0 0x7fa0a806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0b00ac640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa0a806ec60 0x7fa0a8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0b00ac640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a80ccc70 con 0x7fa0a80ce510 2024-04-25T08:54:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0b00ac640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a80cc2b0 con 0x7fa0a806ec60 2024-04-25T08:54:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0b00ac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a806e290 con 0x7fa0a8144ca0 2024-04-25T08:54:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0ae649640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa0a806ec60 0x7fa0a8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0ae649640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa0a806ec60 0x7fa0a8142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59978/0 (socket says 172.21.15.37:59978) 2024-04-25T08:54:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0ae649640 1 -- 172.21.15.37:0/2648907107 learned_addr learned my addr 172.21.15.37:0/2648907107 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0ad647640 1 --2- 172.21.15.37:0/2648907107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a8144ca0 0x7fa0a806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0ae649640 1 -- 172.21.15.37:0/2648907107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a8144ca0 msgr2=0x7fa0a806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0ae649640 1 --2- 172.21.15.37:0/2648907107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a8144ca0 0x7fa0a806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0ae649640 1 -- 172.21.15.37:0/2648907107 >> v1:172.21.15.203:6789/0 conn(0x7fa0a80ce510 legacy=0x7fa0a80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.271+0000 7fa0ae649640 1 -- 172.21.15.37:0/2648907107 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a806dc40 con 0x7fa0a806ec60 2024-04-25T08:54:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ae649640 1 --2- 172.21.15.37:0/2648907107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a806ec60 0x7fa0a8142680 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009e50 tx=0x7fa0a001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a670ef7be73465d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ace46640 1 -- 172.21.15.37:0/2648907107 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00b4070 con 0x7fa0a806ec60 2024-04-25T08:54:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ace46640 1 -- 172.21.15.37:0/2648907107 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a00af400 con 0x7fa0a806ec60 2024-04-25T08:54:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ace46640 1 -- 172.21.15.37:0/2648907107 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a00af7d0 con 0x7fa0a806ec60 2024-04-25T08:54:39.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2648907107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a806ec60 msgr2=0x7fa0a8142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 --2- 172.21.15.37:0/2648907107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a806ec60 0x7fa0a8142680 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009e50 tx=0x7fa0a001f210 comp rx=0 tx=0).stop 2024-04-25T08:54:39.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2648907107 shutdown_connections 2024-04-25T08:54:39.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 --2- 172.21.15.37:0/2648907107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a806ec60 0x7fa0a8142680 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 --2- 172.21.15.37:0/2648907107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a8144ca0 0x7fa0a806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2648907107 >> 172.21.15.37:0/2648907107 conn(0x7fa0a80b11a0 msgr2=0x7fa0a80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:39.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2648907107 shutdown_connections 2024-04-25T08:54:39.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2648907107 wait complete. 2024-04-25T08:54:39.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 Processor -- start 2024-04-25T08:54:39.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 -- start start 2024-04-25T08:54:39.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a806ec60 0x7fa0a81e5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a80ce510 0x7fa0a81e6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a8144ca0 0x7fa0a8130b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81451f0 con 0x7fa0a80ce510 2024-04-25T08:54:39.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a80cc900 con 0x7fa0a806ec60 2024-04-25T08:54:39.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0b00ac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a8145370 con 0x7fa0a8144ca0 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ad647640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a80ce510 0x7fa0a81e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ad647640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a80ce510 0x7fa0a81e6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59992/0 (socket says 172.21.15.37:59992) 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ad647640 1 -- 172.21.15.37:0/2060617138 learned_addr learned my addr 172.21.15.37:0/2060617138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ae649640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a8144ca0 0x7fa0a8130b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ade48640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a806ec60 0x7fa0a81e5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ad647640 1 -- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a8144ca0 msgr2=0x7fa0a8130b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ad647640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a8144ca0 0x7fa0a8130b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ad647640 1 -- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a806ec60 msgr2=0x7fa0a81e5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ad647640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a806ec60 0x7fa0a81e5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ad647640 1 -- 172.21.15.37:0/2060617138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a8142470 con 0x7fa0a80ce510 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.275+0000 7fa0ade48640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a806ec60 0x7fa0a81e5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa0ad647640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a80ce510 0x7fa0a81e6070 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa09800d5c0 tx=0x7fa09800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa09e7fc640 1 -- 172.21.15.37:0/2060617138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0980125f0 con 0x7fa0a80ce510 2024-04-25T08:54:39.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa09e7fc640 1 -- 172.21.15.37:0/2060617138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa098019070 con 0x7fa0a80ce510 2024-04-25T08:54:39.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2060617138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a81e2260 con 0x7fa0a80ce510 2024-04-25T08:54:39.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa09e7fc640 1 -- 172.21.15.37:0/2060617138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0980129d0 con 0x7fa0a80ce510 2024-04-25T08:54:39.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2060617138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a812eb20 con 0x7fa0a80ce510 2024-04-25T08:54:39.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2060617138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a804efa0 con 0x7fa0a80ce510 2024-04-25T08:54:39.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa09e7fc640 1 -- 172.21.15.37:0/2060617138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa098012b70 con 0x7fa0a80ce510 2024-04-25T08:54:39.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa09e7fc640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa08c066f60 0x7fa08c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa09e7fc640 1 -- 172.21.15.37:0/2060617138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa098085bc0 con 0x7fa0a80ce510 2024-04-25T08:54:39.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa0ade48640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa08c066f60 0x7fa08c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.279+0000 7fa0ade48640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa08c066f60 0x7fa08c069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4001710 tx=0x7fa0a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.283+0000 7fa09e7fc640 1 -- 172.21.15.37:0/2060617138 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa098056f00 con 0x7fa0a80ce510 2024-04-25T08:54:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.439+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2060617138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0a80b1cc0 con 0x7fa0a80ce510 2024-04-25T08:54:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.439+0000 7fa09e7fc640 1 -- 172.21.15.37:0/2060617138 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v51) v1 ==== 74+0+13128 (secure 0 0 0) 0x7fa098056800 con 0x7fa0a80ce510 2024-04-25T08:54:39.434 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:39.434 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":51,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:54:38.756823+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:39.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa08c066f60 msgr2=0x7fa08c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa08c066f60 0x7fa08c069420 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4001710 tx=0x7fa0a4009040 comp rx=0 tx=0).stop 2024-04-25T08:54:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2060617138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a80ce510 msgr2=0x7fa0a81e6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a80ce510 0x7fa0a81e6070 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa09800d5c0 tx=0x7fa09800da80 comp rx=0 tx=0).stop 2024-04-25T08:54:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2060617138 shutdown_connections 2024-04-25T08:54:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa08c066f60 0x7fa08c069420 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0a8144ca0 0x7fa0a8130b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a80ce510 0x7fa0a81e6070 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 --2- 172.21.15.37:0/2060617138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0a806ec60 0x7fa0a81e5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2060617138 >> 172.21.15.37:0/2060617138 conn(0x7fa0a80b11a0 msgr2=0x7fa0a80b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2060617138 shutdown_connections 2024-04-25T08:54:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.443+0000 7fa0b00ac640 1 -- 172.21.15.37:0/2060617138 wait complete. 2024-04-25T08:54:39.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:39.453 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:39.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2ad004640 1 Processor -- start 2024-04-25T08:54:39.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2ad004640 1 -- start start 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2ad004640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb2a80cf590 0x7fb2a80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2ad004640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb2a8072000 0x7fb2a8072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2ad004640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a80cbd20 con 0x7fb2a815b6b0 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2ad004640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a815ba90 con 0x7fb2a8072000 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2ad004640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a80cc000 con 0x7fb2a80cf590 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2a759e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb2a8072000 0x7fb2a8072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2a759e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb2a8072000 0x7fb2a8072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55200/0 (socket says 172.21.15.37:55200) 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2a759e640 1 -- 172.21.15.37:0/3528671907 learned_addr learned my addr 172.21.15.37:0/3528671907 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2a759e640 1 -- 172.21.15.37:0/3528671907 >> v2:172.21.15.203:3300/0 conn(0x7fb2a80cf590 msgr2=0x7fb2a80cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2a759e640 1 --2- 172.21.15.37:0/3528671907 >> v2:172.21.15.203:3300/0 conn(0x7fb2a80cf590 0x7fb2a80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2a759e640 1 -- 172.21.15.37:0/3528671907 >> v1:172.21.15.203:6789/0 conn(0x7fb2a815b6b0 legacy=0x7fb2a80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:39.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2a759e640 1 -- 172.21.15.37:0/3528671907 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80cc2e0 con 0x7fb2a8072000 2024-04-25T08:54:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.579+0000 7fb2a759e640 1 --2- 172.21.15.37:0/3528671907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a8072000 0x7fb2a8072430 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80b4430 tx=0x7fb2a001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775099ccbc021ee4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a5d9b640 1 -- 172.21.15.37:0/3528671907 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a00a5760 con 0x7fb2a8072000 2024-04-25T08:54:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a5d9b640 1 -- 172.21.15.37:0/3528671907 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a00a5900 con 0x7fb2a8072000 2024-04-25T08:54:39.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a5d9b640 1 -- 172.21.15.37:0/3528671907 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a00a5cd0 con 0x7fb2a8072000 2024-04-25T08:54:39.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 -- 172.21.15.37:0/3528671907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a8072000 msgr2=0x7fb2a8072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 --2- 172.21.15.37:0/3528671907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a8072000 0x7fb2a8072430 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80b4430 tx=0x7fb2a001c9b0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 -- 172.21.15.37:0/3528671907 shutdown_connections 2024-04-25T08:54:39.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 --2- 172.21.15.37:0/3528671907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a8072000 0x7fb2a8072430 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 --2- 172.21.15.37:0/3528671907 >> v2:172.21.15.203:3300/0 conn(0x7fb2a80cf590 0x7fb2a80cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 -- 172.21.15.37:0/3528671907 >> 172.21.15.37:0/3528671907 conn(0x7fb2a806ae60 msgr2=0x7fb2a806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:39.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 -- 172.21.15.37:0/3528671907 shutdown_connections 2024-04-25T08:54:39.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 -- 172.21.15.37:0/3528671907 wait complete. 2024-04-25T08:54:39.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 Processor -- start 2024-04-25T08:54:39.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 -- start start 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2a8072000 0x7fb2a80c20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2a80cf590 0x7fb2a8098080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a815b6b0 0x7fb2a806e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80caf30 con 0x7fb2a815b6b0 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a80cadb0 con 0x7fb2a8072000 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a81682a0 con 0x7fb2a80cf590 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a815b6b0 0x7fb2a806e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a815b6b0 0x7fb2a806e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55216/0 (socket says 172.21.15.37:55216) 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a759e640 1 -- 172.21.15.37:0/1927664602 learned_addr learned my addr 172.21.15.37:0/1927664602 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a759e640 1 -- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2a80cf590 msgr2=0x7fb2a8098080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a759e640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2a80cf590 0x7fb2a8098080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a759e640 1 -- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2a8072000 msgr2=0x7fb2a80c20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a759e640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2a8072000 0x7fb2a80c20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a759e640 1 -- 172.21.15.37:0/1927664602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80985c0 con 0x7fb2a815b6b0 2024-04-25T08:54:39.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2a759e640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a815b6b0 0x7fb2a806e0d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00af040 tx=0x7fb2a0019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb28f7fe640 1 -- 172.21.15.37:0/1927664602 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a00bc020 con 0x7fb2a815b6b0 2024-04-25T08:54:39.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb28f7fe640 1 -- 172.21.15.37:0/1927664602 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a00b8670 con 0x7fb2a815b6b0 2024-04-25T08:54:39.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 -- 172.21.15.37:0/1927664602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a80c1320 con 0x7fb2a815b6b0 2024-04-25T08:54:39.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb28f7fe640 1 -- 172.21.15.37:0/1927664602 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a00b8a60 con 0x7fb2a815b6b0 2024-04-25T08:54:39.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.583+0000 7fb2ad004640 1 -- 172.21.15.37:0/1927664602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a80c18e0 con 0x7fb2a815b6b0 2024-04-25T08:54:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.587+0000 7fb2ad004640 1 -- 172.21.15.37:0/1927664602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a804efa0 con 0x7fb2a815b6b0 2024-04-25T08:54:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.587+0000 7fb28f7fe640 1 -- 172.21.15.37:0/1927664602 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2a00bf020 con 0x7fb2a815b6b0 2024-04-25T08:54:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.587+0000 7fb28f7fe640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb294066f90 0x7fb294069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.587+0000 7fb28f7fe640 1 -- 172.21.15.37:0/1927664602 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb2a011f950 con 0x7fb2a815b6b0 2024-04-25T08:54:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.587+0000 7fb2a6d9d640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb294066f90 0x7fb294069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.587+0000 7fb2a6d9d640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb294066f90 0x7fb294069450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb298009650 tx=0x7fb298006bb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.591+0000 7fb28f7fe640 1 -- 172.21.15.37:0/1927664602 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2a00c7020 con 0x7fb2a815b6b0 2024-04-25T08:54:39.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.751+0000 7fb2ad004640 1 -- 172.21.15.37:0/1927664602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2a80b3a50 con 0x7fb2a815b6b0 2024-04-25T08:54:39.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.751+0000 7fb28f7fe640 1 -- 172.21.15.37:0/1927664602 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v51) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb2a00165a0 con 0x7fb2a815b6b0 2024-04-25T08:54:39.746 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 -- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb294066f90 msgr2=0x7fb294069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb294066f90 0x7fb294069450 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb298009650 tx=0x7fb298006bb0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 -- 172.21.15.37:0/1927664602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a815b6b0 msgr2=0x7fb2a806e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a815b6b0 0x7fb2a806e0d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00af040 tx=0x7fb2a0019750 comp rx=0 tx=0).stop 2024-04-25T08:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 -- 172.21.15.37:0/1927664602 shutdown_connections 2024-04-25T08:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb294066f90 0x7fb294069450 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2a815b6b0 0x7fb2a806e0d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2a80cf590 0x7fb2a8098080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 --2- 172.21.15.37:0/1927664602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2a8072000 0x7fb2a80c20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 -- 172.21.15.37:0/1927664602 >> 172.21.15.37:0/1927664602 conn(0x7fb2a806ae60 msgr2=0x7fb2a80b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 -- 172.21.15.37:0/1927664602 shutdown_connections 2024-04-25T08:54:39.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.755+0000 7fb2ad004640 1 -- 172.21.15.37:0/1927664602 wait complete. 2024-04-25T08:54:39.766 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:54:39.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfcb24c640 1 Processor -- start 2024-04-25T08:54:39.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfcb24c640 1 -- start start 2024-04-25T08:54:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfcb24c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc40b5270 0x7fdfc40b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfcb24c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdfc406d090 0x7fdfc406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfcb24c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc406bff0 con 0x7fdfc40b5650 2024-04-25T08:54:39.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfcb24c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc406cc00 con 0x7fdfc406d090 2024-04-25T08:54:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfcb24c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc406c2d0 con 0x7fdfc40b5270 2024-04-25T08:54:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfc3fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdfc40b5650 0x7fdfc40d3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:39236/0 (socket says 172.21.15.37:39236) 2024-04-25T08:54:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfc3fff640 1 -- 172.21.15.37:0/2314076099 learned_addr learned my addr 172.21.15.37:0/2314076099 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfc97e9640 1 --2- 172.21.15.37:0/2314076099 >> v2:172.21.15.37:3300/0 conn(0x7fdfc406d090 0x7fdfc406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfc8fe8640 1 --2- 172.21.15.37:0/2314076099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc40b5270 0x7fdfc40b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfc37fe640 1 -- 172.21.15.37:0/2314076099 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161326891 0 0) 0x7fdfc406bff0 con 0x7fdfc40b5650 2024-04-25T08:54:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfc37fe640 1 -- 172.21.15.37:0/2314076099 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfac003530 con 0x7fdfc40b5650 2024-04-25T08:54:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfc97e9640 1 -- 172.21.15.37:0/2314076099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc40b5270 msgr2=0x7fdfc40b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfc97e9640 1 --2- 172.21.15.37:0/2314076099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc40b5270 0x7fdfc40b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfc97e9640 1 -- 172.21.15.37:0/2314076099 >> v1:172.21.15.37:6789/0 conn(0x7fdfc40b5650 legacy=0x7fdfc40d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.899+0000 7fdfc97e9640 1 -- 172.21.15.37:0/2314076099 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc406c5b0 con 0x7fdfc406d090 2024-04-25T08:54:39.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfc97e9640 1 --2- 172.21.15.37:0/2314076099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc406d090 0x7fdfc406d4f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40097c0 tx=0x7fdfb401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9ffed239a5c16a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfc37fe640 1 -- 172.21.15.37:0/2314076099 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40b1070 con 0x7fdfc406d090 2024-04-25T08:54:39.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfc37fe640 1 -- 172.21.15.37:0/2314076099 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb40ac410 con 0x7fdfc406d090 2024-04-25T08:54:39.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfc37fe640 1 -- 172.21.15.37:0/2314076099 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40ac7e0 con 0x7fdfc406d090 2024-04-25T08:54:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 -- 172.21.15.37:0/2314076099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc406d090 msgr2=0x7fdfc406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 --2- 172.21.15.37:0/2314076099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc406d090 0x7fdfc406d4f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fdfb40097c0 tx=0x7fdfb401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 -- 172.21.15.37:0/2314076099 shutdown_connections 2024-04-25T08:54:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 --2- 172.21.15.37:0/2314076099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc406d090 0x7fdfc406d4f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 --2- 172.21.15.37:0/2314076099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc40b5270 0x7fdfc40b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 -- 172.21.15.37:0/2314076099 >> 172.21.15.37:0/2314076099 conn(0x7fdfc408b000 msgr2=0x7fdfc40d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 -- 172.21.15.37:0/2314076099 shutdown_connections 2024-04-25T08:54:39.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 -- 172.21.15.37:0/2314076099 wait complete. 2024-04-25T08:54:39.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 Processor -- start 2024-04-25T08:54:39.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 -- start start 2024-04-25T08:54:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc406d090 0x7fdfc40c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.903+0000 7fdfcb24c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc40b5270 0x7fdfc40ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfcb24c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc40b5650 0x7fdfc41428e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfcb24c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40b5bb0 con 0x7fdfc406d090 2024-04-25T08:54:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfcb24c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40b5a30 con 0x7fdfc40b5650 2024-04-25T08:54:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfcb24c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc415de20 con 0x7fdfc40b5270 2024-04-25T08:54:39.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc8fe8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc406d090 0x7fdfc40c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc8fe8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc406d090 0x7fdfc40c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55230/0 (socket says 172.21.15.37:55230) 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc8fe8640 1 -- 172.21.15.37:0/122539857 learned_addr learned my addr 172.21.15.37:0/122539857 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc3fff640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc40b5270 0x7fdfc40ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc97e9640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc40b5650 0x7fdfc41428e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc8fe8640 1 -- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc40b5270 msgr2=0x7fdfc40ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc8fe8640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc40b5270 0x7fdfc40ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc8fe8640 1 -- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc40b5650 msgr2=0x7fdfc41428e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc8fe8640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc40b5650 0x7fdfc41428e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc8fe8640 1 -- 172.21.15.37:0/122539857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc40c9d10 con 0x7fdfc406d090 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc8fe8640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc406d090 0x7fdfc40c7290 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009fc0 tx=0x7fdfbc00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc17fa640 1 -- 172.21.15.37:0/122539857 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc014620 con 0x7fdfc406d090 2024-04-25T08:54:39.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc17fa640 1 -- 172.21.15.37:0/122539857 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfbc019070 con 0x7fdfc406d090 2024-04-25T08:54:39.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfcb24c640 1 -- 172.21.15.37:0/122539857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc40c9ff0 con 0x7fdfc406d090 2024-04-25T08:54:39.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc17fa640 1 -- 172.21.15.37:0/122539857 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfbc014a00 con 0x7fdfc406d090 2024-04-25T08:54:39.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfcb24c640 1 -- 172.21.15.37:0/122539857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc40c77d0 con 0x7fdfc406d090 2024-04-25T08:54:39.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfcb24c640 1 -- 172.21.15.37:0/122539857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc415ae70 con 0x7fdfc406d090 2024-04-25T08:54:39.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc17fa640 1 -- 172.21.15.37:0/122539857 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdfbc014ba0 con 0x7fdfc406d090 2024-04-25T08:54:39.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc17fa640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfb0066f60 0x7fdfb0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc17fa640 1 -- 172.21.15.37:0/122539857 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fdfbc085880 con 0x7fdfc406d090 2024-04-25T08:54:39.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.907+0000 7fdfc3fff640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfb0066f60 0x7fdfb0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.911+0000 7fdfc3fff640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfb0066f60 0x7fdfb0069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4157680 tx=0x7fdfb8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:39.911+0000 7fdfc17fa640 1 -- 172.21.15.37:0/122539857 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdfbc0569b0 con 0x7fdfc406d090 2024-04-25T08:54:40.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.071+0000 7fdfcb24c640 1 -- 172.21.15.37:0/122539857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdfc404efa0 con 0x7fdfc406d090 2024-04-25T08:54:40.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfc17fa640 1 -- 172.21.15.37:0/122539857 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v51) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdfbc0562b0 con 0x7fdfc406d090 2024-04-25T08:54:40.067 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:40.068 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T08:54:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T08:54:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T08:54:40.068 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:40.068 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T08:54:40.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 -- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfb0066f60 msgr2=0x7fdfb0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfb0066f60 0x7fdfb0069420 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4157680 tx=0x7fdfb8009040 comp rx=0 tx=0).stop 2024-04-25T08:54:40.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 -- 172.21.15.37:0/122539857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc406d090 msgr2=0x7fdfc40c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc406d090 0x7fdfc40c7290 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc009fc0 tx=0x7fdfbc00e570 comp rx=0 tx=0).stop 2024-04-25T08:54:40.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 -- 172.21.15.37:0/122539857 shutdown_connections 2024-04-25T08:54:40.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdfb0066f60 0x7fdfb0069420 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdfc40b5650 0x7fdfc41428e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdfc40b5270 0x7fdfc40ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 --2- 172.21.15.37:0/122539857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfc406d090 0x7fdfc40c7290 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 -- 172.21.15.37:0/122539857 >> 172.21.15.37:0/122539857 conn(0x7fdfc408b000 msgr2=0x7fdfc4156650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:40.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 -- 172.21.15.37:0/122539857 shutdown_connections 2024-04-25T08:54:40.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.075+0000 7fdfcb24c640 1 -- 172.21.15.37:0/122539857 wait complete. 2024-04-25T08:54:40.087 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:54:40.087 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:54:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.223+0000 7f9bdceba640 1 Processor -- start 2024-04-25T08:54:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.223+0000 7f9bdceba640 1 -- start start 2024-04-25T08:54:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.223+0000 7f9bdceba640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9bd815d980 0x7f9bd80d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 0x7f9bd806cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd815dd60 con 0x7f9bd80d12c0 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd806c810 con 0x7f9bd815d980 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd80d4340 con 0x7f9bd80d23c0 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9bd815d980 0x7f9bd80d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9bd815d980 0x7f9bd80d3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55246/0 (socket says 172.21.15.37:55246) 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd659c640 1 -- 172.21.15.37:0/1388656536 learned_addr learned my addr 172.21.15.37:0/1388656536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd6d9d640 1 --2- 172.21.15.37:0/1388656536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 0x7f9bd806cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd659c640 1 -- 172.21.15.37:0/1388656536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 msgr2=0x7f9bd806cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd659c640 1 --2- 172.21.15.37:0/1388656536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 0x7f9bd806cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd659c640 1 -- 172.21.15.37:0/1388656536 >> v1:172.21.15.203:6789/0 conn(0x7f9bd80d12c0 legacy=0x7f9bd80d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd659c640 1 -- 172.21.15.37:0/1388656536 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd80d45e0 con 0x7f9bd815d980 2024-04-25T08:54:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd6d9d640 1 --2- 172.21.15.37:0/1388656536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 0x7f9bd806cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd659c640 1 --2- 172.21.15.37:0/1388656536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd815d980 0x7f9bd80d3e00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9bd806c990 tx=0x7f9bc801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d129b7b8d30265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd559a640 1 -- 172.21.15.37:0/1388656536 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80b0070 con 0x7f9bd815d980 2024-04-25T08:54:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd559a640 1 -- 172.21.15.37:0/1388656536 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc80ab600 con 0x7f9bd815d980 2024-04-25T08:54:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bd559a640 1 -- 172.21.15.37:0/1388656536 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80ab9d0 con 0x7f9bd815d980 2024-04-25T08:54:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 -- 172.21.15.37:0/1388656536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd815d980 msgr2=0x7f9bd80d3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 --2- 172.21.15.37:0/1388656536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd815d980 0x7f9bd80d3e00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9bd806c990 tx=0x7f9bc801c470 comp rx=0 tx=0).stop 2024-04-25T08:54:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 -- 172.21.15.37:0/1388656536 shutdown_connections 2024-04-25T08:54:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 --2- 172.21.15.37:0/1388656536 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 0x7f9bd806cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 --2- 172.21.15.37:0/1388656536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd815d980 0x7f9bd80d3e00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 -- 172.21.15.37:0/1388656536 >> 172.21.15.37:0/1388656536 conn(0x7f9bd808b000 msgr2=0x7f9bd80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 -- 172.21.15.37:0/1388656536 shutdown_connections 2024-04-25T08:54:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.227+0000 7f9bdceba640 1 -- 172.21.15.37:0/1388656536 wait complete. 2024-04-25T08:54:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bdceba640 1 Processor -- start 2024-04-25T08:54:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bdceba640 1 -- start start 2024-04-25T08:54:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bdceba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd80d12c0 0x7f9bd80cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bdceba640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 0x7f9bd80c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bdceba640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bd815d980 0x7f9bd81596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bdceba640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd806d9f0 con 0x7f9bd80d12c0 2024-04-25T08:54:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bdceba640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd806c4a0 con 0x7f9bd815d980 2024-04-25T08:54:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bdceba640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd806db70 con 0x7f9bd80d23c0 2024-04-25T08:54:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd80d12c0 0x7f9bd80cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd80d12c0 0x7f9bd80cb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55248/0 (socket says 172.21.15.37:55248) 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd659c640 1 -- 172.21.15.37:0/1699321243 learned_addr learned my addr 172.21.15.37:0/1699321243 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd6d9d640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bd815d980 0x7f9bd81596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd5d9b640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 0x7f9bd80c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd659c640 1 -- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 msgr2=0x7f9bd80c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd659c640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 0x7f9bd80c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd659c640 1 -- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bd815d980 msgr2=0x7f9bd81596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd659c640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bd815d980 0x7f9bd81596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd659c640 1 -- 172.21.15.37:0/1699321243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd8158bb0 con 0x7f9bd80d12c0 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd5d9b640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 0x7f9bd80c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd6d9d640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bd815d980 0x7f9bd81596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bd659c640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd80d12c0 0x7f9bd80cb810 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9bc801cd10 tx=0x7f9bc801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bceffd640 1 -- 172.21.15.37:0/1699321243 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80ba020 con 0x7f9bd80d12c0 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bceffd640 1 -- 172.21.15.37:0/1699321243 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc80b6700 con 0x7f9bd80d12c0 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bdceba640 1 -- 172.21.15.37:0/1699321243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd8159be0 con 0x7f9bd80d12c0 2024-04-25T08:54:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bceffd640 1 -- 172.21.15.37:0/1699321243 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80b0040 con 0x7f9bd80d12c0 2024-04-25T08:54:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.231+0000 7f9bdceba640 1 -- 172.21.15.37:0/1699321243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd80cc0d0 con 0x7f9bd80d12c0 2024-04-25T08:54:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.235+0000 7f9bdceba640 1 -- 172.21.15.37:0/1699321243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd8067c30 con 0x7f9bd80d12c0 2024-04-25T08:54:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.235+0000 7f9bceffd640 1 -- 172.21.15.37:0/1699321243 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bc80bd020 con 0x7f9bd80d12c0 2024-04-25T08:54:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.235+0000 7f9bceffd640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bb8066f60 0x7f9bb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.235+0000 7f9bceffd640 1 -- 172.21.15.37:0/1699321243 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9bc811f480 con 0x7f9bd80d12c0 2024-04-25T08:54:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.235+0000 7f9bd5d9b640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bb8066f60 0x7f9bb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.235+0000 7f9bd5d9b640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bb8066f60 0x7f9bb8069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9bd809d610 tx=0x7f9bc00073a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.239+0000 7f9bceffd640 1 -- 172.21.15.37:0/1699321243 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bc80c4030 con 0x7f9bd80d12c0 2024-04-25T08:54:40.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:40.403+0000 7f9bdceba640 1 -- 172.21.15.37:0/1699321243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9bd80b9350 con 0x7f9bd80d12c0 2024-04-25T08:54:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.407+0000 7f9bceffd640 1 -- 172.21.15.37:0/1699321243 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v52) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9bd8159be0 con 0x7f9bd80d12c0 2024-04-25T08:54:41.401 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T08:54:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 -- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bb8066f60 msgr2=0x7f9bb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bb8066f60 0x7f9bb8069420 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9bd809d610 tx=0x7f9bc00073a0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 -- 172.21.15.37:0/1699321243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd80d12c0 msgr2=0x7f9bd80cb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd80d12c0 0x7f9bd80cb810 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9bc801cd10 tx=0x7f9bc801acb0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 -- 172.21.15.37:0/1699321243 shutdown_connections 2024-04-25T08:54:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bb8066f60 0x7f9bb8069420 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9bd815d980 0x7f9bd81596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9bd80d23c0 0x7f9bd80c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 --2- 172.21.15.37:0/1699321243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9bd80d12c0 0x7f9bd80cb810 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 -- 172.21.15.37:0/1699321243 >> 172.21.15.37:0/1699321243 conn(0x7f9bd808b000 msgr2=0x7f9bd80b7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:41.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 -- 172.21.15.37:0/1699321243 shutdown_connections 2024-04-25T08:54:41.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.411+0000 7f9bdceba640 1 -- 172.21.15.37:0/1699321243 wait complete. 2024-04-25T08:54:41.435 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:54:41.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.575+0000 7f1226b70640 1 Processor -- start 2024-04-25T08:54:41.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.575+0000 7f1226b70640 1 -- start start 2024-04-25T08:54:41.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f1226b70640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f122015da80 0x7f12200d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f1226b70640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f12200d1cd0 0x7f12200d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f1226b70640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f122006be60 con 0x7f12200d0620 2024-04-25T08:54:41.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f1226b70640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12200d0a70 con 0x7f122015da80 2024-04-25T08:54:41.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f1226b70640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f122006c440 con 0x7f12200d1cd0 2024-04-25T08:54:41.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f122490c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f122015da80 0x7f12200d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f122490c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f122015da80 0x7f12200d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55258/0 (socket says 172.21.15.37:55258) 2024-04-25T08:54:41.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f122490c640 1 -- 172.21.15.37:0/942240609 learned_addr learned my addr 172.21.15.37:0/942240609 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:41.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f121ffff640 1 --2- 172.21.15.37:0/942240609 >> v2:172.21.15.203:3300/0 conn(0x7f12200d1cd0 0x7f12200d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f122490c640 1 -- 172.21.15.37:0/942240609 >> v2:172.21.15.203:3300/0 conn(0x7f12200d1cd0 msgr2=0x7f12200d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:41.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f122490c640 1 --2- 172.21.15.37:0/942240609 >> v2:172.21.15.203:3300/0 conn(0x7f12200d1cd0 0x7f12200d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f122490c640 1 -- 172.21.15.37:0/942240609 >> v1:172.21.15.203:6789/0 conn(0x7f12200d0620 legacy=0x7f12200ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:41.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f122490c640 1 -- 172.21.15.37:0/942240609 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122015de60 con 0x7f122015da80 2024-04-25T08:54:41.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f122490c640 1 --2- 172.21.15.37:0/942240609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122015da80 0x7f12200d4230 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1214009870 tx=0x7f121401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ca3f8af8fbe90f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f121f7fe640 1 -- 172.21.15.37:0/942240609 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140b0070 con 0x7f122015da80 2024-04-25T08:54:41.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f121f7fe640 1 -- 172.21.15.37:0/942240609 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12140ab600 con 0x7f122015da80 2024-04-25T08:54:41.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.579+0000 7f121f7fe640 1 -- 172.21.15.37:0/942240609 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140ab9d0 con 0x7f122015da80 2024-04-25T08:54:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 -- 172.21.15.37:0/942240609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122015da80 msgr2=0x7f12200d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 --2- 172.21.15.37:0/942240609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122015da80 0x7f12200d4230 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1214009870 tx=0x7f121401c4e0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 -- 172.21.15.37:0/942240609 shutdown_connections 2024-04-25T08:54:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 --2- 172.21.15.37:0/942240609 >> v2:172.21.15.203:3300/0 conn(0x7f12200d1cd0 0x7f12200d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 --2- 172.21.15.37:0/942240609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f122015da80 0x7f12200d4230 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 -- 172.21.15.37:0/942240609 >> 172.21.15.37:0/942240609 conn(0x7f122008b000 msgr2=0x7f12200d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 -- 172.21.15.37:0/942240609 shutdown_connections 2024-04-25T08:54:41.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 -- 172.21.15.37:0/942240609 wait complete. 2024-04-25T08:54:41.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 Processor -- start 2024-04-25T08:54:41.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 -- start start 2024-04-25T08:54:41.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d0620 0x7f1220160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12200d1cd0 0x7f122015c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f122015da80 0x7f122015cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122016a730 con 0x7f12200d0620 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122016a5b0 con 0x7f12200d1cd0 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f1226b70640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f122016a8b0 con 0x7f122015da80 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f122490c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d0620 0x7f1220160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f122490c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d0620 0x7f1220160240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55268/0 (socket says 172.21.15.37:55268) 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f122490c640 1 -- 172.21.15.37:0/1079746992 learned_addr learned my addr 172.21.15.37:0/1079746992 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f121ffff640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12200d1cd0 0x7f122015c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f122510d640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f122015da80 0x7f122015cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f122490c640 1 -- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f122015da80 msgr2=0x7f122015cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f122490c640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f122015da80 0x7f122015cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f122490c640 1 -- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12200d1cd0 msgr2=0x7f122015c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f122490c640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12200d1cd0 0x7f122015c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f122490c640 1 -- 172.21.15.37:0/1079746992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1220151f60 con 0x7f12200d0620 2024-04-25T08:54:41.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f122490c640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d0620 0x7f1220160240 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f12140ae040 tx=0x7f1214019280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f121d7fa640 1 -- 172.21.15.37:0/1079746992 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140b0040 con 0x7f12200d0620 2024-04-25T08:54:41.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f121d7fa640 1 -- 172.21.15.37:0/1079746992 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12140b9070 con 0x7f12200d0620 2024-04-25T08:54:41.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.583+0000 7f121d7fa640 1 -- 172.21.15.37:0/1079746992 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140b4660 con 0x7f12200d0620 2024-04-25T08:54:41.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.587+0000 7f1226b70640 1 -- 172.21.15.37:0/1079746992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12201516c0 con 0x7f12200d0620 2024-04-25T08:54:41.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.587+0000 7f1226b70640 1 -- 172.21.15.37:0/1079746992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1220151c80 con 0x7f12200d0620 2024-04-25T08:54:41.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.587+0000 7f1226b70640 1 -- 172.21.15.37:0/1079746992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122004efa0 con 0x7f12200d0620 2024-04-25T08:54:41.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.587+0000 7f121d7fa640 1 -- 172.21.15.37:0/1079746992 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f12140b4800 con 0x7f12200d0620 2024-04-25T08:54:41.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.587+0000 7f121d7fa640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1204066f60 0x7f1204069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.587+0000 7f121d7fa640 1 -- 172.21.15.37:0/1079746992 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f121411eb10 con 0x7f12200d0620 2024-04-25T08:54:41.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.587+0000 7f121ffff640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1204066f60 0x7f1204069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.587+0000 7f121ffff640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1204066f60 0x7f1204069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f12200a1550 tx=0x7f120c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.591+0000 7f121d7fa640 1 -- 172.21.15.37:0/1079746992 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f12140160d0 con 0x7f12200d0620 2024-04-25T08:54:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:41.763+0000 7f1226b70640 1 -- 172.21.15.37:0/1079746992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f12200cdcb0 con 0x7f12200d0620 2024-04-25T08:54:42.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.411+0000 7f121d7fa640 1 -- 172.21.15.37:0/1079746992 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v53) v1 ==== 170+0+0 (secure 0 0 0) 0x7f12140efc00 con 0x7f12200d0620 2024-04-25T08:54:42.406 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T08:54:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 -- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1204066f60 msgr2=0x7f1204069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1204066f60 0x7f1204069420 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f12200a1550 tx=0x7f120c008040 comp rx=0 tx=0).stop 2024-04-25T08:54:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 -- 172.21.15.37:0/1079746992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d0620 msgr2=0x7f1220160240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d0620 0x7f1220160240 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f12140ae040 tx=0x7f1214019280 comp rx=0 tx=0).stop 2024-04-25T08:54:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 -- 172.21.15.37:0/1079746992 shutdown_connections 2024-04-25T08:54:42.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1204066f60 0x7f1204069420 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f122015da80 0x7f122015cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f12200d1cd0 0x7f122015c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 --2- 172.21.15.37:0/1079746992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12200d0620 0x7f1220160240 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 -- 172.21.15.37:0/1079746992 >> 172.21.15.37:0/1079746992 conn(0x7f122008b000 msgr2=0x7f12200b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 -- 172.21.15.37:0/1079746992 shutdown_connections 2024-04-25T08:54:42.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.415+0000 7f1226b70640 1 -- 172.21.15.37:0/1079746992 wait complete. 2024-04-25T08:54:42.440 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:54:42.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd1207e5640 1 Processor -- start 2024-04-25T08:54:42.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd1207e5640 1 -- start start 2024-04-25T08:54:42.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd1207e5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd11815aac0 0x7fd1180ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd1207e5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1180cf0e0 0x7fd1180cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd1207e5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1180ce320 con 0x7fd11815aea0 2024-04-25T08:54:42.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd1207e5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1180ce0d0 con 0x7fd11815aac0 2024-04-25T08:54:42.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd1207e5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180ce600 con 0x7fd1180cf0e0 2024-04-25T08:54:42.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd11dd80640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1180cf0e0 0x7fd1180cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd11e581640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd11815aac0 0x7fd1180ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd11dd80640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1180cf0e0 0x7fd1180cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:39292/0 (socket says 172.21.15.37:39292) 2024-04-25T08:54:42.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd11dd80640 1 -- 172.21.15.37:0/1759679805 learned_addr learned my addr 172.21.15.37:0/1759679805 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:42.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd11e581640 1 -- 172.21.15.37:0/1759679805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1180cf0e0 msgr2=0x7fd1180cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:42.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd11e581640 1 --2- 172.21.15.37:0/1759679805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1180cf0e0 0x7fd1180cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd11e581640 1 -- 172.21.15.37:0/1759679805 >> v1:172.21.15.203:6789/0 conn(0x7fd11815aea0 legacy=0x7fd11815b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd11e581640 1 -- 172.21.15.37:0/1759679805 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1180ce8e0 con 0x7fd11815aac0 2024-04-25T08:54:42.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.575+0000 7fd11dd80640 1 --2- 172.21.15.37:0/1759679805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1180cf0e0 0x7fd1180cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:42.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd11e581640 1 --2- 172.21.15.37:0/1759679805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd11815aac0 0x7fd1180ccc40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd1140097c0 tx=0x7fd11401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=539a01b87967452f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd11d57f640 1 -- 172.21.15.37:0/1759679805 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1140b1070 con 0x7fd11815aac0 2024-04-25T08:54:42.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd11d57f640 1 -- 172.21.15.37:0/1759679805 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1140ac410 con 0x7fd11815aac0 2024-04-25T08:54:42.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd11d57f640 1 -- 172.21.15.37:0/1759679805 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1140ac7e0 con 0x7fd11815aac0 2024-04-25T08:54:42.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd1207e5640 1 -- 172.21.15.37:0/1759679805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd11815aac0 msgr2=0x7fd1180ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd1207e5640 1 --2- 172.21.15.37:0/1759679805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd11815aac0 0x7fd1180ccc40 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd1140097c0 tx=0x7fd11401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd1207e5640 1 -- 172.21.15.37:0/1759679805 shutdown_connections 2024-04-25T08:54:42.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd1207e5640 1 --2- 172.21.15.37:0/1759679805 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd1180cf0e0 0x7fd1180cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd1207e5640 1 --2- 172.21.15.37:0/1759679805 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd11815aac0 0x7fd1180ccc40 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd1207e5640 1 -- 172.21.15.37:0/1759679805 >> 172.21.15.37:0/1759679805 conn(0x7fd118099a70 msgr2=0x7fd1180d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:42.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd1207e5640 1 -- 172.21.15.37:0/1759679805 shutdown_connections 2024-04-25T08:54:42.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.579+0000 7fd1207e5640 1 -- 172.21.15.37:0/1759679805 wait complete. 2024-04-25T08:54:42.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd1207e5640 1 Processor -- start 2024-04-25T08:54:42.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd1207e5640 1 -- start start 2024-04-25T08:54:42.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd1207e5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1180cf0e0 0x7fd1180b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd1207e5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd11815aac0 0x7fd1180b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd1207e5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd11815aea0 0x7fd1181515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd1207e5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180cbbf0 con 0x7fd1180cf0e0 2024-04-25T08:54:42.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd1207e5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180cba70 con 0x7fd11815aac0 2024-04-25T08:54:42.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd1207e5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1180cbd70 con 0x7fd11815aea0 2024-04-25T08:54:42.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd11e581640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1180cf0e0 0x7fd1180b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd11e581640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1180cf0e0 0x7fd1180b4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55284/0 (socket says 172.21.15.37:55284) 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd11e581640 1 -- 172.21.15.37:0/3614297462 learned_addr learned my addr 172.21.15.37:0/3614297462 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd11ed82640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd11815aea0 0x7fd1181515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd11dd80640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd11815aac0 0x7fd1180b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd11e581640 1 -- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd11815aea0 msgr2=0x7fd1181515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd11e581640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd11815aea0 0x7fd1181515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd11e581640 1 -- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd11815aac0 msgr2=0x7fd1180b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd11e581640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd11815aac0 0x7fd1180b5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd11e581640 1 -- 172.21.15.37:0/3614297462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd118158c90 con 0x7fd1180cf0e0 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd11e581640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1180cf0e0 0x7fd1180b4630 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd11401a0d0 tx=0x7fd114019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd10effd640 1 -- 172.21.15.37:0/3614297462 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1140b1040 con 0x7fd1180cf0e0 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd10effd640 1 -- 172.21.15.37:0/3614297462 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1140bb070 con 0x7fd1180cf0e0 2024-04-25T08:54:42.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd1207e5640 1 -- 172.21.15.37:0/3614297462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1180b6270 con 0x7fd1180cf0e0 2024-04-25T08:54:42.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd10effd640 1 -- 172.21.15.37:0/3614297462 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1140b5410 con 0x7fd1180cf0e0 2024-04-25T08:54:42.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd1207e5640 1 -- 172.21.15.37:0/3614297462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1180b4f60 con 0x7fd1180cf0e0 2024-04-25T08:54:42.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.583+0000 7fd1207e5640 1 -- 172.21.15.37:0/3614297462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11804efa0 con 0x7fd1180cf0e0 2024-04-25T08:54:42.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.587+0000 7fd10effd640 1 -- 172.21.15.37:0/3614297462 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd1140b55b0 con 0x7fd1180cf0e0 2024-04-25T08:54:42.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.587+0000 7fd10effd640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0fc066f60 0x7fd0fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.587+0000 7fd10effd640 1 -- 172.21.15.37:0/3614297462 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd11411f690 con 0x7fd1180cf0e0 2024-04-25T08:54:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.587+0000 7fd11dd80640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0fc066f60 0x7fd0fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.587+0000 7fd11dd80640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0fc066f60 0x7fd0fc069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd118151140 tx=0x7fd108008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.591+0000 7fd10effd640 1 -- 172.21.15.37:0/3614297462 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd114016090 con 0x7fd1180cf0e0 2024-04-25T08:54:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:42.791+0000 7fd1207e5640 1 -- 172.21.15.37:0/3614297462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd11806e7e0 con 0x7fd1180cf0e0 2024-04-25T08:54:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.475+0000 7fd10effd640 1 -- 172.21.15.37:0/3614297462 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd1140f0650 con 0x7fd1180cf0e0 2024-04-25T08:54:43.471 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T08:54:43.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 -- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0fc066f60 msgr2=0x7fd0fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0fc066f60 0x7fd0fc069420 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd118151140 tx=0x7fd108008040 comp rx=0 tx=0).stop 2024-04-25T08:54:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 -- 172.21.15.37:0/3614297462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1180cf0e0 msgr2=0x7fd1180b4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1180cf0e0 0x7fd1180b4630 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd11401a0d0 tx=0x7fd114019240 comp rx=0 tx=0).stop 2024-04-25T08:54:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 -- 172.21.15.37:0/3614297462 shutdown_connections 2024-04-25T08:54:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0fc066f60 0x7fd0fc069420 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd11815aea0 0x7fd1181515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd11815aac0 0x7fd1180b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 --2- 172.21.15.37:0/3614297462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd1180cf0e0 0x7fd1180b4630 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 -- 172.21.15.37:0/3614297462 >> 172.21.15.37:0/3614297462 conn(0x7fd118099a70 msgr2=0x7fd118157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 -- 172.21.15.37:0/3614297462 shutdown_connections 2024-04-25T08:54:43.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.491+0000 7fd1207e5640 1 -- 172.21.15.37:0/3614297462 wait complete. 2024-04-25T08:54:43.513 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:43.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.635+0000 7ffa38bdd640 1 Processor -- start 2024-04-25T08:54:43.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.635+0000 7ffa38bdd640 1 -- start start 2024-04-25T08:54:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa38bdd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffa3415ecc0 0x7ffa3406e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa38bdd640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa340b2090 con 0x7ffa340b3f50 2024-04-25T08:54:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa38bdd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa340b43e0 con 0x7ffa3415a700 2024-04-25T08:54:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa38bdd640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa340b46c0 con 0x7ffa3415ecc0 2024-04-25T08:54:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa32d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffa3415ecc0 0x7ffa3406e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa31d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ffa340b3f50 0x7ffa340b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:39250/0 (socket says 172.21.15.37:39250) 2024-04-25T08:54:43.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa31d9b640 1 -- 172.21.15.37:0/1911100409 learned_addr learned my addr 172.21.15.37:0/1911100409 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa3159a640 1 -- 172.21.15.37:0/1911100409 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292852036 0 0) 0x7ffa340b2090 con 0x7ffa340b3f50 2024-04-25T08:54:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa3159a640 1 -- 172.21.15.37:0/1911100409 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa1c003530 con 0x7ffa340b3f50 2024-04-25T08:54:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa32d9d640 1 -- 172.21.15.37:0/1911100409 >> v1:172.21.15.203:6789/0 conn(0x7ffa3415a700 legacy=0x7ffa340cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa32d9d640 1 -- 172.21.15.37:0/1911100409 >> v1:172.21.15.37:6789/0 conn(0x7ffa340b3f50 legacy=0x7ffa340b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa32d9d640 1 -- 172.21.15.37:0/1911100409 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa340cd6f0 con 0x7ffa3415ecc0 2024-04-25T08:54:43.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa32d9d640 1 --2- 172.21.15.37:0/1911100409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3415ecc0 0x7ffa3406e1a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009d90 tx=0x7ffa2c01c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95f808a546480d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.639+0000 7ffa3159a640 1 -- 172.21.15.37:0/1911100409 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c0b2070 con 0x7ffa3415ecc0 2024-04-25T08:54:43.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa3159a640 1 -- 172.21.15.37:0/1911100409 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2c0a57f0 con 0x7ffa3415ecc0 2024-04-25T08:54:43.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa3159a640 1 -- 172.21.15.37:0/1911100409 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c0a5bc0 con 0x7ffa3415ecc0 2024-04-25T08:54:43.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 -- 172.21.15.37:0/1911100409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3415ecc0 msgr2=0x7ffa3406e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 --2- 172.21.15.37:0/1911100409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3415ecc0 0x7ffa3406e1a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009d90 tx=0x7ffa2c01c820 comp rx=0 tx=0).stop 2024-04-25T08:54:43.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 -- 172.21.15.37:0/1911100409 shutdown_connections 2024-04-25T08:54:43.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 --2- 172.21.15.37:0/1911100409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3415ecc0 0x7ffa3406e1a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 -- 172.21.15.37:0/1911100409 >> 172.21.15.37:0/1911100409 conn(0x7ffa340b11a0 msgr2=0x7ffa340b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:43.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 -- 172.21.15.37:0/1911100409 shutdown_connections 2024-04-25T08:54:43.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 -- 172.21.15.37:0/1911100409 wait complete. 2024-04-25T08:54:43.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 Processor -- start 2024-04-25T08:54:43.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 -- start start 2024-04-25T08:54:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa3415a700 0x7ffa3414f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3415ecc0 0x7ffa3406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa34151570 0x7ffa3414a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340cd310 con 0x7ffa3415ecc0 2024-04-25T08:54:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340ce600 con 0x7ffa34151570 2024-04-25T08:54:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa38bdd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340cd490 con 0x7ffa3415a700 2024-04-25T08:54:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa31d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3415ecc0 0x7ffa3406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa31d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3415ecc0 0x7ffa3406d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55298/0 (socket says 172.21.15.37:55298) 2024-04-25T08:54:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa31d9b640 1 -- 172.21.15.37:0/48723858 learned_addr learned my addr 172.21.15.37:0/48723858 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa32d9d640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa34151570 0x7ffa3414a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa3259c640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa3415a700 0x7ffa3414f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa31d9b640 1 -- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa3415a700 msgr2=0x7ffa3414f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa31d9b640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa3415a700 0x7ffa3414f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa31d9b640 1 -- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa34151570 msgr2=0x7ffa3414a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa31d9b640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa34151570 0x7ffa3414a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa31d9b640 1 -- 172.21.15.37:0/48723858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3414ea50 con 0x7ffa3415ecc0 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa32d9d640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa34151570 0x7ffa3414a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa3259c640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa3415a700 0x7ffa3414f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa31d9b640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3415ecc0 0x7ffa3406d5a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400c420 tx=0x7ffa2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa22ffd640 1 -- 172.21.15.37:0/48723858 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa24017070 con 0x7ffa3415ecc0 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa22ffd640 1 -- 172.21.15.37:0/48723858 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa24005cf0 con 0x7ffa3415ecc0 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.643+0000 7ffa22ffd640 1 -- 172.21.15.37:0/48723858 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa240072d0 con 0x7ffa3415ecc0 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.647+0000 7ffa38bdd640 1 -- 172.21.15.37:0/48723858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa3414ed30 con 0x7ffa3415ecc0 2024-04-25T08:54:43.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.647+0000 7ffa38bdd640 1 -- 172.21.15.37:0/48723858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa3414f2f0 con 0x7ffa3415ecc0 2024-04-25T08:54:43.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.647+0000 7ffa38bdd640 1 -- 172.21.15.37:0/48723858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3404efa0 con 0x7ffa3415ecc0 2024-04-25T08:54:43.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.647+0000 7ffa22ffd640 1 -- 172.21.15.37:0/48723858 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffa24007470 con 0x7ffa3415ecc0 2024-04-25T08:54:43.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.647+0000 7ffa22ffd640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffa14066f90 0x7ffa14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.647+0000 7ffa22ffd640 1 -- 172.21.15.37:0/48723858 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffa24084430 con 0x7ffa3415ecc0 2024-04-25T08:54:43.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.647+0000 7ffa3259c640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffa14066f90 0x7ffa14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.647+0000 7ffa3259c640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffa14066f90 0x7ffa14069450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffa280016f0 tx=0x7ffa28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.651+0000 7ffa22ffd640 1 -- 172.21.15.37:0/48723858 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffa24054d20 con 0x7ffa3415ecc0 2024-04-25T08:54:43.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.811+0000 7ffa38bdd640 1 -- 172.21.15.37:0/48723858 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa340aecc0 con 0x7ffa3415ecc0 2024-04-25T08:54:43.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.815+0000 7ffa22ffd640 1 -- 172.21.15.37:0/48723858 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16919 (secure 0 0 0) 0x7ffa24058dc0 con 0x7ffa3415ecc0 2024-04-25T08:54:43.809 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:43.809 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":54,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:54:42.799570+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:40.447670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:41.812203+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:43.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.815+0000 7ffa38bdd640 1 -- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffa14066f90 msgr2=0x7ffa14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.815+0000 7ffa38bdd640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffa14066f90 0x7ffa14069450 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ffa280016f0 tx=0x7ffa28009040 comp rx=0 tx=0).stop 2024-04-25T08:54:43.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.815+0000 7ffa38bdd640 1 -- 172.21.15.37:0/48723858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3415ecc0 msgr2=0x7ffa3406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.815+0000 7ffa38bdd640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3415ecc0 0x7ffa3406d5a0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400c420 tx=0x7ffa2400c8e0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.815+0000 7ffa38bdd640 1 -- 172.21.15.37:0/48723858 shutdown_connections 2024-04-25T08:54:43.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.815+0000 7ffa38bdd640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffa14066f90 0x7ffa14069450 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.815+0000 7ffa38bdd640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffa34151570 0x7ffa3414a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.815+0000 7ffa38bdd640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa3415ecc0 0x7ffa3406d5a0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.815+0000 7ffa38bdd640 1 --2- 172.21.15.37:0/48723858 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffa3415a700 0x7ffa3414f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.815+0000 7ffa38bdd640 1 -- 172.21.15.37:0/48723858 >> 172.21.15.37:0/48723858 conn(0x7ffa340b11a0 msgr2=0x7ffa340b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:43.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.819+0000 7ffa38bdd640 1 -- 172.21.15.37:0/48723858 shutdown_connections 2024-04-25T08:54:43.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.819+0000 7ffa38bdd640 1 -- 172.21.15.37:0/48723858 wait complete. 2024-04-25T08:54:43.828 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:43.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.951+0000 7f000fe68640 1 Processor -- start 2024-04-25T08:54:43.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.951+0000 7f000fe68640 1 -- start start 2024-04-25T08:54:43.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000fe68640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f000815da80 0x7f00080d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000fe68640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f00080d07f0 0x7f000806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000fe68640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f000806be60 con 0x7f00080d1cd0 2024-04-25T08:54:43.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000fe68640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000806c440 con 0x7f000815da80 2024-04-25T08:54:43.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000fe68640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f000815de60 con 0x7f00080d07f0 2024-04-25T08:54:43.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000d403640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f00080d1cd0 0x7f00080d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:39254/0 (socket says 172.21.15.37:39254) 2024-04-25T08:54:43.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000d403640 1 -- 172.21.15.37:0/1552507298 learned_addr learned my addr 172.21.15.37:0/1552507298 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:43.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000dc04640 1 --2- 172.21.15.37:0/1552507298 >> v2:172.21.15.37:3300/0 conn(0x7f000815da80 0x7f00080d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000e405640 1 --2- 172.21.15.37:0/1552507298 >> v2:172.21.15.203:3300/0 conn(0x7f00080d07f0 0x7f000806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000cc02640 1 -- 172.21.15.37:0/1552507298 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021779497 0 0) 0x7f000806be60 con 0x7f00080d1cd0 2024-04-25T08:54:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000cc02640 1 -- 172.21.15.37:0/1552507298 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff4003530 con 0x7f00080d1cd0 2024-04-25T08:54:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000dc04640 1 -- 172.21.15.37:0/1552507298 >> v2:172.21.15.203:3300/0 conn(0x7f00080d07f0 msgr2=0x7f000806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000dc04640 1 --2- 172.21.15.37:0/1552507298 >> v2:172.21.15.203:3300/0 conn(0x7f00080d07f0 0x7f000806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000dc04640 1 -- 172.21.15.37:0/1552507298 >> v1:172.21.15.37:6789/0 conn(0x7f00080d1cd0 legacy=0x7f00080d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.955+0000 7f000dc04640 1 -- 172.21.15.37:0/1552507298 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00080d2450 con 0x7f000815da80 2024-04-25T08:54:43.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000dc04640 1 --2- 172.21.15.37:0/1552507298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f000815da80 0x7f00080d4230 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f00040097c0 tx=0x7f000401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7d9e7a9a6ad989 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000cc02640 1 -- 172.21.15.37:0/1552507298 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00040b1070 con 0x7f000815da80 2024-04-25T08:54:43.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000cc02640 1 -- 172.21.15.37:0/1552507298 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00040ac5f0 con 0x7f000815da80 2024-04-25T08:54:43.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000cc02640 1 -- 172.21.15.37:0/1552507298 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00040ac9c0 con 0x7f000815da80 2024-04-25T08:54:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 -- 172.21.15.37:0/1552507298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f000815da80 msgr2=0x7f00080d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 --2- 172.21.15.37:0/1552507298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f000815da80 0x7f00080d4230 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f00040097c0 tx=0x7f000401c430 comp rx=0 tx=0).stop 2024-04-25T08:54:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 -- 172.21.15.37:0/1552507298 shutdown_connections 2024-04-25T08:54:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 --2- 172.21.15.37:0/1552507298 >> v2:172.21.15.203:3300/0 conn(0x7f00080d07f0 0x7f000806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 --2- 172.21.15.37:0/1552507298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f000815da80 0x7f00080d4230 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 -- 172.21.15.37:0/1552507298 >> 172.21.15.37:0/1552507298 conn(0x7f000808b000 msgr2=0x7f00080d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:43.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 -- 172.21.15.37:0/1552507298 shutdown_connections 2024-04-25T08:54:43.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 -- 172.21.15.37:0/1552507298 wait complete. 2024-04-25T08:54:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 Processor -- start 2024-04-25T08:54:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 -- start start 2024-04-25T08:54:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00080d07f0 0x7f00080b8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.959+0000 7f000fe68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00080d1cd0 0x7f00080b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000fe68640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f000815da80 0x7f00080cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000fe68640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000806d7a0 con 0x7f00080d1cd0 2024-04-25T08:54:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000fe68640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f000806d620 con 0x7f000815da80 2024-04-25T08:54:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000fe68640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f000806d920 con 0x7f00080d07f0 2024-04-25T08:54:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000d403640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00080d1cd0 0x7f00080b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000d403640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00080d1cd0 0x7f00080b6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55326/0 (socket says 172.21.15.37:55326) 2024-04-25T08:54:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000d403640 1 -- 172.21.15.37:0/3873570714 learned_addr learned my addr 172.21.15.37:0/3873570714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000dc04640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00080d07f0 0x7f00080b8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000e405640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f000815da80 0x7f00080cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000d403640 1 -- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00080d07f0 msgr2=0x7f00080b8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000d403640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00080d07f0 0x7f00080b8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000d403640 1 -- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f000815da80 msgr2=0x7f00080cb8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000d403640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f000815da80 0x7f00080cb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000d403640 1 -- 172.21.15.37:0/3873570714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00080cb130 con 0x7f00080d1cd0 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000dc04640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00080d07f0 0x7f00080b8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000d403640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00080d1cd0 0x7f00080b6730 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efffc00c840 tx=0x7efffc00cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7efffa7fc640 1 -- 172.21.15.37:0/3873570714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc017070 con 0x7f00080d1cd0 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7efffa7fc640 1 -- 172.21.15.37:0/3873570714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efffc00a640 con 0x7f00080d1cd0 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000fe68640 1 -- 172.21.15.37:0/3873570714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00080cd390 con 0x7f00080d1cd0 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7efffa7fc640 1 -- 172.21.15.37:0/3873570714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efffc00aa30 con 0x7f00080d1cd0 2024-04-25T08:54:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000fe68640 1 -- 172.21.15.37:0/3873570714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00081593a0 con 0x7f00080d1cd0 2024-04-25T08:54:43.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000fe68640 1 -- 172.21.15.37:0/3873570714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f000804efa0 con 0x7f00080d1cd0 2024-04-25T08:54:43.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7efffa7fc640 1 -- 172.21.15.37:0/3873570714 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efffc01a070 con 0x7f00080d1cd0 2024-04-25T08:54:43.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7efffa7fc640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7effec066f60 0x7effec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7efffa7fc640 1 -- 172.21.15.37:0/3873570714 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efffc0848b0 con 0x7f00080d1cd0 2024-04-25T08:54:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.963+0000 7f000dc04640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7effec066f60 0x7effec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.967+0000 7f000dc04640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7effec066f60 0x7effec069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f00040ae000 tx=0x7f00040a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:43.967+0000 7efffa7fc640 1 -- 172.21.15.37:0/3873570714 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efffc055730 con 0x7f00080d1cd0 2024-04-25T08:54:44.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.127+0000 7f000fe68640 1 -- 172.21.15.37:0/3873570714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00080d0180 con 0x7f00080d1cd0 2024-04-25T08:54:44.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7efffa7fc640 1 -- 172.21.15.37:0/3873570714 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16919 (secure 0 0 0) 0x7efffc055030 con 0x7f00080d1cd0 2024-04-25T08:54:44.125 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:44.125 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":54,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:54:42.799570+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:40.447670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:41.812203+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:44.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 -- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7effec066f60 msgr2=0x7effec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7effec066f60 0x7effec069420 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f00040ae000 tx=0x7f00040a7000 comp rx=0 tx=0).stop 2024-04-25T08:54:44.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 -- 172.21.15.37:0/3873570714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00080d1cd0 msgr2=0x7f00080b6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00080d1cd0 0x7f00080b6730 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7efffc00c840 tx=0x7efffc00cd00 comp rx=0 tx=0).stop 2024-04-25T08:54:44.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 -- 172.21.15.37:0/3873570714 shutdown_connections 2024-04-25T08:54:44.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7effec066f60 0x7effec069420 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f000815da80 0x7f00080cb8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00080d1cd0 0x7f00080b6730 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 --2- 172.21.15.37:0/3873570714 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00080d07f0 0x7f00080b8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 -- 172.21.15.37:0/3873570714 >> 172.21.15.37:0/3873570714 conn(0x7f000808b000 msgr2=0x7f00080b0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:44.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 -- 172.21.15.37:0/3873570714 shutdown_connections 2024-04-25T08:54:44.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.131+0000 7f000fe68640 1 -- 172.21.15.37:0/3873570714 wait complete. 2024-04-25T08:54:44.144 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:54:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499fb02640 1 Processor -- start 2024-04-25T08:54:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499fb02640 1 -- start start 2024-04-25T08:54:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499fb02640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f499815da80 0x7f49980d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499fb02640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499806be60 con 0x7f49980d1cd0 2024-04-25T08:54:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499fb02640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f499815de60 con 0x7f49980d07f0 2024-04-25T08:54:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499fb02640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f499806c440 con 0x7f499815da80 2024-04-25T08:54:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499d89e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f499815da80 0x7f49980d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499d09d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f49980d1cd0 0x7f49980d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:39256/0 (socket says 172.21.15.37:39256) 2024-04-25T08:54:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499d09d640 1 -- 172.21.15.37:0/557678323 learned_addr learned my addr 172.21.15.37:0/557678323 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499d89e640 1 -- 172.21.15.37:0/557678323 >> v1:172.21.15.203:6789/0 conn(0x7f49980d07f0 legacy=0x7f499806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499d89e640 1 -- 172.21.15.37:0/557678323 >> v1:172.21.15.37:6789/0 conn(0x7f49980d1cd0 legacy=0x7f49980d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.275+0000 7f499d89e640 1 -- 172.21.15.37:0/557678323 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49980d2450 con 0x7f499815da80 2024-04-25T08:54:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499d89e640 1 --2- 172.21.15.37:0/557678323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f499815da80 0x7f49980d4230 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f499806c5c0 tx=0x7f499401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cded8d021794e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499c89c640 1 -- 172.21.15.37:0/557678323 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49940b0070 con 0x7f499815da80 2024-04-25T08:54:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499c89c640 1 -- 172.21.15.37:0/557678323 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49940ab600 con 0x7f499815da80 2024-04-25T08:54:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499c89c640 1 -- 172.21.15.37:0/557678323 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49940ab9d0 con 0x7f499815da80 2024-04-25T08:54:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 -- 172.21.15.37:0/557678323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f499815da80 msgr2=0x7f49980d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 --2- 172.21.15.37:0/557678323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f499815da80 0x7f49980d4230 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f499806c5c0 tx=0x7f499401c310 comp rx=0 tx=0).stop 2024-04-25T08:54:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 -- 172.21.15.37:0/557678323 shutdown_connections 2024-04-25T08:54:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 --2- 172.21.15.37:0/557678323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f499815da80 0x7f49980d4230 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 -- 172.21.15.37:0/557678323 >> 172.21.15.37:0/557678323 conn(0x7f499808b000 msgr2=0x7f49980d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 -- 172.21.15.37:0/557678323 shutdown_connections 2024-04-25T08:54:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 -- 172.21.15.37:0/557678323 wait complete. 2024-04-25T08:54:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 Processor -- start 2024-04-25T08:54:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 -- start start 2024-04-25T08:54:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49980d07f0 0x7f49981f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49980d1cd0 0x7f49980b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f499815da80 0x7f49980b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49980d26f0 con 0x7f499815da80 2024-04-25T08:54:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49980d2120 con 0x7f49980d1cd0 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499fb02640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49980d2870 con 0x7f49980d07f0 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.279+0000 7f499e09f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f499815da80 0x7f49980b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499e09f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f499815da80 0x7f49980b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55330/0 (socket says 172.21.15.37:55330) 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499e09f640 1 -- 172.21.15.37:0/1393548919 learned_addr learned my addr 172.21.15.37:0/1393548919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499d09d640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49980d1cd0 0x7f49980b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499d89e640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49980d07f0 0x7f49981f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499e09f640 1 -- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49980d07f0 msgr2=0x7f49981f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499e09f640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49980d07f0 0x7f49981f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499e09f640 1 -- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49980d1cd0 msgr2=0x7f49980b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499e09f640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49980d1cd0 0x7f49980b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499e09f640 1 -- 172.21.15.37:0/1393548919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49980b6c80 con 0x7f499815da80 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499d09d640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49980d1cd0 0x7f49980b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499e09f640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f499815da80 0x7f49980b1670 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f499000c770 tx=0x7f499000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f498e7fc640 1 -- 172.21.15.37:0/1393548919 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990003b80 con 0x7f499815da80 2024-04-25T08:54:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f498e7fc640 1 -- 172.21.15.37:0/1393548919 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4990003d20 con 0x7f499815da80 2024-04-25T08:54:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499fb02640 1 -- 172.21.15.37:0/1393548919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49980b6f60 con 0x7f499815da80 2024-04-25T08:54:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f498e7fc640 1 -- 172.21.15.37:0/1393548919 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499000aa90 con 0x7f499815da80 2024-04-25T08:54:44.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499fb02640 1 -- 172.21.15.37:0/1393548919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49980b7bf0 con 0x7f499815da80 2024-04-25T08:54:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499fb02640 1 -- 172.21.15.37:0/1393548919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f499804efa0 con 0x7f499815da80 2024-04-25T08:54:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f498e7fc640 1 -- 172.21.15.37:0/1393548919 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f499000ac30 con 0x7f499815da80 2024-04-25T08:54:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f498e7fc640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f497c066f90 0x7f497c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f498e7fc640 1 -- 172.21.15.37:0/1393548919 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4990084fa0 con 0x7f499815da80 2024-04-25T08:54:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499d89e640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f497c066f90 0x7f497c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.283+0000 7f499d89e640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f497c066f90 0x7f497c069450 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f49940ad000 tx=0x7f4994008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.287+0000 7f498e7fc640 1 -- 172.21.15.37:0/1393548919 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4990055eb0 con 0x7f499815da80 2024-04-25T08:54:44.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:44.483+0000 7f499fb02640 1 -- 172.21.15.37:0/1393548919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f49980b29c0 con 0x7f499815da80 2024-04-25T08:54:45.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.483+0000 7f498e7fc640 1 -- 172.21.15.37:0/1393548919 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f49900557b0 con 0x7f499815da80 2024-04-25T08:54:45.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 -- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f497c066f90 msgr2=0x7f497c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f497c066f90 0x7f497c069450 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f49940ad000 tx=0x7f4994008040 comp rx=0 tx=0).stop 2024-04-25T08:54:45.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 -- 172.21.15.37:0/1393548919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f499815da80 msgr2=0x7f49980b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f499815da80 0x7f49980b1670 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f499000c770 tx=0x7f499000cc30 comp rx=0 tx=0).stop 2024-04-25T08:54:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 -- 172.21.15.37:0/1393548919 shutdown_connections 2024-04-25T08:54:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f497c066f90 0x7f497c069450 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f499815da80 0x7f49980b1670 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49980d1cd0 0x7f49980b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 --2- 172.21.15.37:0/1393548919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49980d07f0 0x7f49981f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 -- 172.21.15.37:0/1393548919 >> 172.21.15.37:0/1393548919 conn(0x7f499808b000 msgr2=0x7f4998073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 -- 172.21.15.37:0/1393548919 shutdown_connections 2024-04-25T08:54:45.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.487+0000 7f499fb02640 1 -- 172.21.15.37:0/1393548919 wait complete. 2024-04-25T08:54:45.502 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:54:45.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:54:45.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbbcd44640 1 Processor -- start 2024-04-25T08:54:45.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbbcd44640 1 -- start start 2024-04-25T08:54:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbbcd44640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffbb80cf590 0x7ffbb815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbbcd44640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ffbb80cf970 0x7ffbb80c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbbcd44640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb80ca4c0 con 0x7ffbb80c9ef0 2024-04-25T08:54:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbbcd44640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80ca780 con 0x7ffbb80cf590 2024-04-25T08:54:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbbcd44640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80cd440 con 0x7ffbb80cf970 2024-04-25T08:54:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbb659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffbb80cf590 0x7ffbb815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbb659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffbb80cf590 0x7ffbb815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55332/0 (socket says 172.21.15.37:55332) 2024-04-25T08:54:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbb659c640 1 -- 172.21.15.37:0/1769999819 learned_addr learned my addr 172.21.15.37:0/1769999819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbadd9b640 1 --2- 172.21.15.37:0/1769999819 >> v2:172.21.15.203:3300/0 conn(0x7ffbb80cf970 0x7ffbb80c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbb659c640 1 -- 172.21.15.37:0/1769999819 >> v2:172.21.15.203:3300/0 conn(0x7ffbb80cf970 msgr2=0x7ffbb80c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbb659c640 1 --2- 172.21.15.37:0/1769999819 >> v2:172.21.15.203:3300/0 conn(0x7ffbb80cf970 0x7ffbb80c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbb659c640 1 -- 172.21.15.37:0/1769999819 >> v1:172.21.15.203:6789/0 conn(0x7ffbb80c9ef0 legacy=0x7ffbb80b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:45.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbb659c640 1 -- 172.21.15.37:0/1769999819 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb80cd690 con 0x7ffbb80cf590 2024-04-25T08:54:45.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.639+0000 7ffbadd9b640 1 --2- 172.21.15.37:0/1769999819 >> v2:172.21.15.203:3300/0 conn(0x7ffbb80cf970 0x7ffbb80c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:45.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbb659c640 1 --2- 172.21.15.37:0/1769999819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbb80cf590 0x7ffbb815ecb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80d00e0 tx=0x7ffba801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19288817835bb8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbb5d9b640 1 -- 172.21.15.37:0/1769999819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba80b1070 con 0x7ffbb80cf590 2024-04-25T08:54:45.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbb5d9b640 1 -- 172.21.15.37:0/1769999819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba80ac640 con 0x7ffbb80cf590 2024-04-25T08:54:45.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbb5d9b640 1 -- 172.21.15.37:0/1769999819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba80aca10 con 0x7ffbb80cf590 2024-04-25T08:54:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbbcd44640 1 -- 172.21.15.37:0/1769999819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbb80cf590 msgr2=0x7ffbb815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbbcd44640 1 --2- 172.21.15.37:0/1769999819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbb80cf590 0x7ffbb815ecb0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80d00e0 tx=0x7ffba801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbbcd44640 1 -- 172.21.15.37:0/1769999819 shutdown_connections 2024-04-25T08:54:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbbcd44640 1 --2- 172.21.15.37:0/1769999819 >> v2:172.21.15.203:3300/0 conn(0x7ffbb80cf970 0x7ffbb80c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbbcd44640 1 --2- 172.21.15.37:0/1769999819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbb80cf590 0x7ffbb815ecb0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbbcd44640 1 -- 172.21.15.37:0/1769999819 >> 172.21.15.37:0/1769999819 conn(0x7ffbb806ae60 msgr2=0x7ffbb806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbbcd44640 1 -- 172.21.15.37:0/1769999819 shutdown_connections 2024-04-25T08:54:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbbcd44640 1 -- 172.21.15.37:0/1769999819 wait complete. 2024-04-25T08:54:45.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbbcd44640 1 Processor -- start 2024-04-25T08:54:45.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.643+0000 7ffbbcd44640 1 -- start start 2024-04-25T08:54:45.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbbcd44640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80c9ef0 0x7ffbb81636a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbbcd44640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb80cf590 0x7ffbb81f9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbbcd44640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbb80cf970 0x7ffbb80724b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbbcd44640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80cbf60 con 0x7ffbb80cf970 2024-04-25T08:54:45.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbbcd44640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80cbde0 con 0x7ffbb80cf590 2024-04-25T08:54:45.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbbcd44640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80cc0e0 con 0x7ffbb80c9ef0 2024-04-25T08:54:45.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbb80cf970 0x7ffbb80724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbb80cf970 0x7ffbb80724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55348/0 (socket says 172.21.15.37:55348) 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb6d9d640 1 -- 172.21.15.37:0/2548130358 learned_addr learned my addr 172.21.15.37:0/2548130358 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbadd9b640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb80cf590 0x7ffbb81f9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb659c640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80c9ef0 0x7ffbb81636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb6d9d640 1 -- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80c9ef0 msgr2=0x7ffbb81636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb6d9d640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80c9ef0 0x7ffbb81636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb6d9d640 1 -- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb80cf590 msgr2=0x7ffbb81f9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb6d9d640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb80cf590 0x7ffbb81f9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb6d9d640 1 -- 172.21.15.37:0/2548130358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb8098a40 con 0x7ffbb80cf970 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb659c640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80c9ef0 0x7ffbb81636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbadd9b640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb80cf590 0x7ffbb81f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb6d9d640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbb80cf970 0x7ffbb80724b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000c780 tx=0x7ffbb000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbaf7fe640 1 -- 172.21.15.37:0/2548130358 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0004b70 con 0x7ffbb80cf970 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbaf7fe640 1 -- 172.21.15.37:0/2548130358 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb0004d10 con 0x7ffbb80cf970 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbbcd44640 1 -- 172.21.15.37:0/2548130358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb8098d20 con 0x7ffbb80cf970 2024-04-25T08:54:45.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbaf7fe640 1 -- 172.21.15.37:0/2548130358 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000aa90 con 0x7ffbb80cf970 2024-04-25T08:54:45.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbbcd44640 1 -- 172.21.15.37:0/2548130358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb80c4550 con 0x7ffbb80cf970 2024-04-25T08:54:45.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbbcd44640 1 -- 172.21.15.37:0/2548130358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb804efa0 con 0x7ffbb80cf970 2024-04-25T08:54:45.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbaf7fe640 1 -- 172.21.15.37:0/2548130358 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffbb000ac30 con 0x7ffbb80cf970 2024-04-25T08:54:45.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbaf7fe640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb98066f90 0x7ffb98069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbaf7fe640 1 -- 172.21.15.37:0/2548130358 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffbb0085380 con 0x7ffbb80cf970 2024-04-25T08:54:45.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.647+0000 7ffbb659c640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb98066f90 0x7ffb98069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.651+0000 7ffbb659c640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb98066f90 0x7ffb98069450 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffba80ae000 tx=0x7ffba80a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.651+0000 7ffbaf7fe640 1 -- 172.21.15.37:0/2548130358 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffbb0020070 con 0x7ffbb80cf970 2024-04-25T08:54:45.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:45.843+0000 7ffbbcd44640 1 -- 172.21.15.37:0/2548130358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ffbb80b4170 con 0x7ffbb80cf970 2024-04-25T08:54:46.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.491+0000 7ffbaf7fe640 1 -- 172.21.15.37:0/2548130358 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffbb0056290 con 0x7ffbb80cf970 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 -- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb98066f90 msgr2=0x7ffb98069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb98066f90 0x7ffb98069450 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffba80ae000 tx=0x7ffba80a6070 comp rx=0 tx=0).stop 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 -- 172.21.15.37:0/2548130358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbb80cf970 msgr2=0x7ffbb80724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbb80cf970 0x7ffbb80724b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7ffbb000c780 tx=0x7ffbb000cc40 comp rx=0 tx=0).stop 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 -- 172.21.15.37:0/2548130358 shutdown_connections 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb98066f90 0x7ffb98069450 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbb80cf970 0x7ffbb80724b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb80cf590 0x7ffbb81f9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 --2- 172.21.15.37:0/2548130358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80c9ef0 0x7ffbb81636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 -- 172.21.15.37:0/2548130358 >> 172.21.15.37:0/2548130358 conn(0x7ffbb806ae60 msgr2=0x7ffbb80ad6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:46.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 -- 172.21.15.37:0/2548130358 shutdown_connections 2024-04-25T08:54:46.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.499+0000 7ffbbcd44640 1 -- 172.21.15.37:0/2548130358 wait complete. 2024-04-25T08:54:46.516 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:54:46.516 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.651+0000 7f3cedf21640 1 Processor -- start 2024-04-25T08:54:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.651+0000 7f3cedf21640 1 -- start start 2024-04-25T08:54:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3cedf21640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3ce80ce510 0x7f3ce80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3cedf21640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce81451f0 0x7f3ce8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3cedf21640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce80cee30 con 0x7f3ce8144ca0 2024-04-25T08:54:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3cedf21640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce80cc2b0 con 0x7f3ce80ce510 2024-04-25T08:54:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3cedf21640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce806e290 con 0x7f3ce81451f0 2024-04-25T08:54:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3ce80ce510 0x7f3ce80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3ce80ce510 0x7f3ce80ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55356/0 (socket says 172.21.15.37:55356) 2024-04-25T08:54:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce77fe640 1 -- 172.21.15.37:0/562416759 learned_addr learned my addr 172.21.15.37:0/562416759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce7fff640 1 --2- 172.21.15.37:0/562416759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce81451f0 0x7f3ce8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce77fe640 1 -- 172.21.15.37:0/562416759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce81451f0 msgr2=0x7f3ce8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce77fe640 1 --2- 172.21.15.37:0/562416759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce81451f0 0x7f3ce8142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce77fe640 1 -- 172.21.15.37:0/562416759 >> v1:172.21.15.203:6789/0 conn(0x7f3ce8144ca0 legacy=0x7f3ce806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce77fe640 1 -- 172.21.15.37:0/562416759 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce80ccc70 con 0x7f3ce80ce510 2024-04-25T08:54:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce77fe640 1 --2- 172.21.15.37:0/562416759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ce80ce510 0x7f3ce80ce8f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009870 tx=0x7f3cd801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c169943e6f63eacd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:46.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce67fc640 1 -- 172.21.15.37:0/562416759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd80b5070 con 0x7f3ce80ce510 2024-04-25T08:54:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce67fc640 1 -- 172.21.15.37:0/562416759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd80a8a50 con 0x7f3ce80ce510 2024-04-25T08:54:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.655+0000 7f3ce67fc640 1 -- 172.21.15.37:0/562416759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd80a8e20 con 0x7f3ce80ce510 2024-04-25T08:54:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 -- 172.21.15.37:0/562416759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ce80ce510 msgr2=0x7f3ce80ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 --2- 172.21.15.37:0/562416759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ce80ce510 0x7f3ce80ce8f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8009870 tx=0x7f3cd801ed50 comp rx=0 tx=0).stop 2024-04-25T08:54:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 -- 172.21.15.37:0/562416759 shutdown_connections 2024-04-25T08:54:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 --2- 172.21.15.37:0/562416759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce81451f0 0x7f3ce8142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 --2- 172.21.15.37:0/562416759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ce80ce510 0x7f3ce80ce8f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 -- 172.21.15.37:0/562416759 >> 172.21.15.37:0/562416759 conn(0x7f3ce80b11a0 msgr2=0x7f3ce80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 -- 172.21.15.37:0/562416759 shutdown_connections 2024-04-25T08:54:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 -- 172.21.15.37:0/562416759 wait complete. 2024-04-25T08:54:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 Processor -- start 2024-04-25T08:54:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 -- start start 2024-04-25T08:54:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce80ce510 0x7f3ce8136150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce8144ca0 0x7f3ce81e6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ce81451f0 0x7f3ce81e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce806ee30 con 0x7f3ce81451f0 2024-04-25T08:54:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce806df20 con 0x7f3ce8144ca0 2024-04-25T08:54:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce80cc900 con 0x7f3ce80ce510 2024-04-25T08:54:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3ce7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ce81451f0 0x7f3ce81e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3ce7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ce81451f0 0x7f3ce81e7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55364/0 (socket says 172.21.15.37:55364) 2024-04-25T08:54:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3ce7fff640 1 -- 172.21.15.37:0/172706669 learned_addr learned my addr 172.21.15.37:0/172706669 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3ce77fe640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce80ce510 0x7f3ce8136150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3ce6ffd640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce8144ca0 0x7f3ce81e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3ce7fff640 1 -- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce80ce510 msgr2=0x7f3ce8136150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3ce7fff640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce80ce510 0x7f3ce8136150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3ce7fff640 1 -- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce8144ca0 msgr2=0x7f3ce81e6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3ce7fff640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce8144ca0 0x7f3ce81e6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3ce7fff640 1 -- 172.21.15.37:0/172706669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce81e6970 con 0x7f3ce81451f0 2024-04-25T08:54:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3ce7fff640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ce81451f0 0x7f3ce81e7e30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000d590 tx=0x7f3ce000da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cdffff640 1 -- 172.21.15.37:0/172706669 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce00125f0 con 0x7f3ce81451f0 2024-04-25T08:54:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cdffff640 1 -- 172.21.15.37:0/172706669 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce0019070 con 0x7f3ce81451f0 2024-04-25T08:54:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cdffff640 1 -- 172.21.15.37:0/172706669 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce00129f0 con 0x7f3ce81451f0 2024-04-25T08:54:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 -- 172.21.15.37:0/172706669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce81e6c50 con 0x7f3ce81451f0 2024-04-25T08:54:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.659+0000 7f3cedf21640 1 -- 172.21.15.37:0/172706669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce81e7210 con 0x7f3ce81451f0 2024-04-25T08:54:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.663+0000 7f3cedf21640 1 -- 172.21.15.37:0/172706669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce804efa0 con 0x7f3ce81451f0 2024-04-25T08:54:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.663+0000 7f3cdffff640 1 -- 172.21.15.37:0/172706669 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ce0012b90 con 0x7f3ce81451f0 2024-04-25T08:54:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.663+0000 7f3cdffff640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cc8066f60 0x7f3cc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.663+0000 7f3cdffff640 1 -- 172.21.15.37:0/172706669 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ce00855b0 con 0x7f3ce81451f0 2024-04-25T08:54:46.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.663+0000 7f3ce77fe640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cc8066f60 0x7f3cc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.663+0000 7f3ce77fe640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cc8066f60 0x7f3cc8069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80b2000 tx=0x7f3cd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:46.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.667+0000 7f3cdffff640 1 -- 172.21.15.37:0/172706669 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ce0056430 con 0x7f3ce81451f0 2024-04-25T08:54:46.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.851+0000 7f3cedf21640 1 -- 172.21.15.37:0/172706669 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ce80b1cc0 con 0x7f3ce81451f0 2024-04-25T08:54:46.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cdffff640 1 -- 172.21.15.37:0/172706669 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f3ce0055d30 con 0x7f3ce81451f0 2024-04-25T08:54:46.848 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:46.848 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2963121871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2963121871},{"type":"v1","addr":"172.21.15.203:6837","nonce":2963121871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1490204181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1490204181},{"type":"v1","addr":"172.21.15.203:6835","nonce":1490204181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2218454043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2218454043},{"type":"v1","addr":"172.21.15.37:6833","nonce":2218454043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:42.800130+0000","modified":"2024-04-25T08:54:45.849035+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/3892786810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3892786810},{"type":"v1","addr":"172.21.15.37:6835","nonce":3892786810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:54:46.848 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-25T08:54:46.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 -- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cc8066f60 msgr2=0x7f3cc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cc8066f60 0x7f3cc8069420 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80b2000 tx=0x7f3cd8008040 comp rx=0 tx=0).stop 2024-04-25T08:54:46.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 -- 172.21.15.37:0/172706669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ce81451f0 msgr2=0x7f3ce81e7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ce81451f0 0x7f3ce81e7e30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000d590 tx=0x7f3ce000da50 comp rx=0 tx=0).stop 2024-04-25T08:54:46.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 -- 172.21.15.37:0/172706669 shutdown_connections 2024-04-25T08:54:46.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cc8066f60 0x7f3cc8069420 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ce81451f0 0x7f3ce81e7e30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ce8144ca0 0x7f3ce81e6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 --2- 172.21.15.37:0/172706669 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ce80ce510 0x7f3ce8136150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 -- 172.21.15.37:0/172706669 >> 172.21.15.37:0/172706669 conn(0x7f3ce80b11a0 msgr2=0x7f3ce80c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 -- 172.21.15.37:0/172706669 shutdown_connections 2024-04-25T08:54:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.855+0000 7f3cedf21640 1 -- 172.21.15.37:0/172706669 wait complete. 2024-04-25T08:54:46.865 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:46.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff77ca640 1 Processor -- start 2024-04-25T08:54:46.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff77ca640 1 -- start start 2024-04-25T08:54:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff77ca640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff0099a70 0x7f3ff00cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff77ca640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff00d0050 con 0x7f3ff00cbea0 2024-04-25T08:54:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff77ca640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff00d0330 con 0x7f3ff0099e50 2024-04-25T08:54:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff77ca640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00cfe20 con 0x7f3ff0099a70 2024-04-25T08:54:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff5d67640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3ff00cbea0 0x7f3ff00cc330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:39262/0 (socket says 172.21.15.37:39262) 2024-04-25T08:54:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff5d67640 1 -- 172.21.15.37:0/3114532382 learned_addr learned my addr 172.21.15.37:0/3114532382 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff5566640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff0099a70 0x7f3ff00cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876723036 0 0) 0x7f3ff00d0050 con 0x7f3ff00cbea0 2024-04-25T08:54:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fdc003530 con 0x7f3ff00cbea0 2024-04-25T08:54:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2624187267 0 0) 0x7f3fdc003530 con 0x7f3ff00cbea0 2024-04-25T08:54:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff0099a70 msgr2=0x7f3ff00cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff0099a70 0x7f3ff00cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 >> v1:172.21.15.203:6789/0 conn(0x7f3ff0099e50 legacy=0x7f3ff006c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00d0610 con 0x7f3ff00cbea0 2024-04-25T08:54:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3fe8001a00 con 0x7f3ff00cbea0 2024-04-25T08:54:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 >> v1:172.21.15.37:6789/0 conn(0x7f3ff00cbea0 legacy=0x7f3ff00cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fdc003a60 0x7f3fdc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff0099a70 0x7f3fdc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fdc004d50 0x7f3fdc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc00bb80 con 0x7f3fdc003a60 2024-04-25T08:54:46.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc00bd30 con 0x7f3fdc004d50 2024-04-25T08:54:46.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fdc00bf30 con 0x7f3ff0099a70 2024-04-25T08:54:46.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff5566640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fdc003a60 0x7f3fdc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff4d65640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff0099a70 0x7f3fdc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff5d67640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fdc004d50 0x7f3fdc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff5566640 1 -- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff0099a70 msgr2=0x7f3fdc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:46.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff5566640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff0099a70 0x7f3fdc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff5566640 1 -- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fdc004d50 msgr2=0x7f3fdc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:46.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff5566640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3fdc004d50 0x7f3fdc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff5566640 1 -- 172.21.15.37:0/3114532382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fdc003530 con 0x7f3fdc003a60 2024-04-25T08:54:46.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:46.999+0000 7f3ff4d65640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff0099a70 0x7f3fdc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:46.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff5566640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fdc003a60 0x7f3fdc003e80 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009ae0 tx=0x7f3fec01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3ed8ddcf246d4d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:46.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0096b0 con 0x7f3fdc003a60 2024-04-25T08:54:46.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fec0a6450 con 0x7f3fdc003a60 2024-04-25T08:54:46.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3fe7fff640 1 -- 172.21.15.37:0/3114532382 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0a6840 con 0x7f3fdc003a60 2024-04-25T08:54:46.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff77ca640 1 -- 172.21.15.37:0/3114532382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fdc003a60 msgr2=0x7f3fdc003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff77ca640 1 --2- 172.21.15.37:0/3114532382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3fdc003a60 0x7f3fdc003e80 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3fec009ae0 tx=0x7f3fec01ca70 comp rx=0 tx=0).stop 2024-04-25T08:54:46.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff5d67640 1 -- 172.21.15.37:0/3114532382 reap_dead start 2024-04-25T08:54:46.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff77ca640 1 -- 172.21.15.37:0/3114532382 shutdown_connections 2024-04-25T08:54:46.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff77ca640 1 -- 172.21.15.37:0/3114532382 >> 172.21.15.37:0/3114532382 conn(0x7f3ff00b3100 msgr2=0x7f3ff00b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:46.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff77ca640 1 -- 172.21.15.37:0/3114532382 shutdown_connections 2024-04-25T08:54:46.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff77ca640 1 -- 172.21.15.37:0/3114532382 wait complete. 2024-04-25T08:54:47.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff77ca640 1 Processor -- start 2024-04-25T08:54:47.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff77ca640 1 -- start start 2024-04-25T08:54:47.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff77ca640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fdc003a60 0x7f3ff015c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff77ca640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff00c65f0 0x7f3ff00c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.003+0000 7f3ff77ca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015f810 0x7f3ff015fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff77ca640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff009a3b0 con 0x7f3ff015f810 2024-04-25T08:54:47.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff77ca640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff009a230 con 0x7f3ff00c65f0 2024-04-25T08:54:47.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff77ca640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff0169850 con 0x7f3fdc003a60 2024-04-25T08:54:47.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff5d67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015f810 0x7f3ff015fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff5d67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015f810 0x7f3ff015fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55380/0 (socket says 172.21.15.37:55380) 2024-04-25T08:54:47.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff5d67640 1 -- 172.21.15.37:0/1548675747 learned_addr learned my addr 172.21.15.37:0/1548675747 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:47.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff5566640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fdc003a60 0x7f3ff015c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff4d65640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff00c65f0 0x7f3ff00c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff5d67640 1 -- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fdc003a60 msgr2=0x7f3ff015c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff5d67640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fdc003a60 0x7f3ff015c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff5d67640 1 -- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff00c65f0 msgr2=0x7f3ff00c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff5d67640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff00c65f0 0x7f3ff00c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff5d67640 1 -- 172.21.15.37:0/1548675747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff01f6f00 con 0x7f3ff015f810 2024-04-25T08:54:47.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff5d67640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015f810 0x7f3ff015fc90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0072830 tx=0x7f3fe8007ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3fe5ffb640 1 -- 172.21.15.37:0/1548675747 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800ea20 con 0x7f3ff015f810 2024-04-25T08:54:47.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3fe5ffb640 1 -- 172.21.15.37:0/1548675747 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe8018070 con 0x7f3ff015f810 2024-04-25T08:54:47.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff77ca640 1 -- 172.21.15.37:0/1548675747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff01f71e0 con 0x7f3ff015f810 2024-04-25T08:54:47.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3fe5ffb640 1 -- 172.21.15.37:0/1548675747 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800ee20 con 0x7f3ff015f810 2024-04-25T08:54:47.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff77ca640 1 -- 172.21.15.37:0/1548675747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff01f77a0 con 0x7f3ff015f810 2024-04-25T08:54:47.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff77ca640 1 -- 172.21.15.37:0/1548675747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff004efa0 con 0x7f3ff015f810 2024-04-25T08:54:47.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3fe5ffb640 1 -- 172.21.15.37:0/1548675747 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3fe801d460 con 0x7f3ff015f810 2024-04-25T08:54:47.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3fe5ffb640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fd4066f60 0x7f3fd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3fe5ffb640 1 -- 172.21.15.37:0/1548675747 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3fe8085ea0 con 0x7f3ff015f810 2024-04-25T08:54:47.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.007+0000 7f3ff5566640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fd4066f60 0x7f3fd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.011+0000 7f3ff5566640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fd4066f60 0x7f3fd4069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0ad000 tx=0x7f3fec0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.011+0000 7f3fe5ffb640 1 -- 172.21.15.37:0/1548675747 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3fe8056d90 con 0x7f3ff015f810 2024-04-25T08:54:47.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.167+0000 7f3ff77ca640 1 -- 172.21.15.37:0/1548675747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ff00b3cc0 con 0x7f3ff015f810 2024-04-25T08:54:47.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.167+0000 7f3fe5ffb640 1 -- 172.21.15.37:0/1548675747 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16906 (secure 0 0 0) 0x7f3fe8056690 con 0x7f3ff015f810 2024-04-25T08:54:47.163 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:47.163 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":55,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:54:44.445489+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:40.447670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:41.812203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 -- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fd4066f60 msgr2=0x7f3fd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fd4066f60 0x7f3fd4069420 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3fec0ad000 tx=0x7f3fec0046d0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 -- 172.21.15.37:0/1548675747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015f810 msgr2=0x7f3ff015fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015f810 0x7f3ff015fc90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0072830 tx=0x7f3fe8007ba0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 -- 172.21.15.37:0/1548675747 shutdown_connections 2024-04-25T08:54:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fd4066f60 0x7f3fd4069420 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015f810 0x7f3ff015fc90 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff00c65f0 0x7f3ff00c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 --2- 172.21.15.37:0/1548675747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3fdc003a60 0x7f3ff015c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 -- 172.21.15.37:0/1548675747 >> 172.21.15.37:0/1548675747 conn(0x7f3ff00b3100 msgr2=0x7f3ff0073700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 -- 172.21.15.37:0/1548675747 shutdown_connections 2024-04-25T08:54:47.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.171+0000 7f3ff77ca640 1 -- 172.21.15.37:0/1548675747 wait complete. 2024-04-25T08:54:47.181 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:54:47.182 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:54:47.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.299+0000 7fc65afc7640 1 Processor -- start 2024-04-25T08:54:47.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.299+0000 7fc65afc7640 1 -- start start 2024-04-25T08:54:47.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc65afc7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc65415da80 0x7fc6540d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc65afc7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65406be60 con 0x7fc6540d1cd0 2024-04-25T08:54:47.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc65afc7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65415de60 con 0x7fc6540d07f0 2024-04-25T08:54:47.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc65afc7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65406c440 con 0x7fc65415da80 2024-04-25T08:54:47.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc653fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc6540d1cd0 0x7fc6540d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:39266/0 (socket says 172.21.15.37:39266) 2024-04-25T08:54:47.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc653fff640 1 -- 172.21.15.37:0/3694790107 learned_addr learned my addr 172.21.15.37:0/3694790107 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:47.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc658d63640 1 --2- 172.21.15.37:0/3694790107 >> v2:172.21.15.203:3300/0 conn(0x7fc65415da80 0x7fc6540d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724678186 0 0) 0x7fc65406be60 con 0x7fc6540d1cd0 2024-04-25T08:54:47.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc63c003530 con 0x7fc6540d1cd0 2024-04-25T08:54:47.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3706989423 0 0) 0x7fc63c003530 con 0x7fc6540d1cd0 2024-04-25T08:54:47.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 >> v2:172.21.15.203:3300/0 conn(0x7fc65415da80 msgr2=0x7fc6540d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 --2- 172.21.15.37:0/3694790107 >> v2:172.21.15.203:3300/0 conn(0x7fc65415da80 0x7fc6540d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 >> v1:172.21.15.203:6789/0 conn(0x7fc6540d07f0 legacy=0x7fc65406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540d2450 con 0x7fc6540d1cd0 2024-04-25T08:54:47.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fc648001800 con 0x7fc6540d1cd0 2024-04-25T08:54:47.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 >> v1:172.21.15.37:6789/0 conn(0x7fc6540d1cd0 legacy=0x7fc6540d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 --2- 172.21.15.37:0/3694790107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc63c003a30 0x7fc63c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 --2- 172.21.15.37:0/3694790107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c004810 0x7fc63c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 --2- 172.21.15.37:0/3694790107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c0051d0 0x7fc63c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65406c440 con 0x7fc63c0051d0 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c00c100 con 0x7fc63c004810 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c00c300 con 0x7fc63c003a30 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc659564640 1 --2- 172.21.15.37:0/3694790107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c0051d0 0x7fc63c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc658d63640 1 --2- 172.21.15.37:0/3694790107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc63c003a30 0x7fc63c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc653fff640 1 --2- 172.21.15.37:0/3694790107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c004810 0x7fc63c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc659564640 1 -- 172.21.15.37:0/3694790107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc63c003a30 msgr2=0x7fc63c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc659564640 1 --2- 172.21.15.37:0/3694790107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc63c003a30 0x7fc63c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc659564640 1 -- 172.21.15.37:0/3694790107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c004810 msgr2=0x7fc63c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc659564640 1 --2- 172.21.15.37:0/3694790107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c004810 0x7fc63c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.303+0000 7fc659564640 1 -- 172.21.15.37:0/3694790107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c003530 con 0x7fc63c0051d0 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc659564640 1 -- 172.21.15.37:0/3694790107 reap_dead start 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc659564640 1 --2- 172.21.15.37:0/3694790107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c0051d0 0x7fc63c00b9d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc64c001e40 tx=0x7fc64c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae428c646d429c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c001a90 con 0x7fc63c0051d0 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c003b80 con 0x7fc63c0051d0 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc6537fe640 1 -- 172.21.15.37:0/3694790107 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c005020 con 0x7fc63c0051d0 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc65afc7640 1 -- 172.21.15.37:0/3694790107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c0051d0 msgr2=0x7fc63c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc65afc7640 1 --2- 172.21.15.37:0/3694790107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c0051d0 0x7fc63c00b9d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc64c001e40 tx=0x7fc64c01c760 comp rx=0 tx=0).stop 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc65afc7640 1 -- 172.21.15.37:0/3694790107 shutdown_connections 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc65afc7640 1 --2- 172.21.15.37:0/3694790107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c0051d0 0x7fc63c00b9d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc65afc7640 1 -- 172.21.15.37:0/3694790107 >> 172.21.15.37:0/3694790107 conn(0x7fc65408b000 msgr2=0x7fc6540d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc65afc7640 1 -- 172.21.15.37:0/3694790107 shutdown_connections 2024-04-25T08:54:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc65afc7640 1 -- 172.21.15.37:0/3694790107 wait complete. 2024-04-25T08:54:47.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc65afc7640 1 Processor -- start 2024-04-25T08:54:47.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.307+0000 7fc65afc7640 1 -- start start 2024-04-25T08:54:47.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc65afc7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c0051d0 0x7fc65415ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc65afc7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541542f0 0x7fc6540b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc65afc7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540b6240 0x7fc6540b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc65afc7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540d22a0 con 0x7fc6541542f0 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc65afc7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540d2120 con 0x7fc63c0051d0 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc65afc7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540d26f0 con 0x7fc6540b6240 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc653fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541542f0 0x7fc6540b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc653fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541542f0 0x7fc6540b8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55400/0 (socket says 172.21.15.37:55400) 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc653fff640 1 -- 172.21.15.37:0/2095815896 learned_addr learned my addr 172.21.15.37:0/2095815896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc659564640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540b6240 0x7fc6540b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc658d63640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c0051d0 0x7fc65415ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc653fff640 1 -- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540b6240 msgr2=0x7fc6540b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc653fff640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540b6240 0x7fc6540b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc653fff640 1 -- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c0051d0 msgr2=0x7fc65415ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc653fff640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c0051d0 0x7fc65415ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc653fff640 1 -- 172.21.15.37:0/2095815896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65415f1c0 con 0x7fc6541542f0 2024-04-25T08:54:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc658d63640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c0051d0 0x7fc65415ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc653fff640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541542f0 0x7fc6540b8c40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc64800f890 tx=0x7fc64800fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc6517fa640 1 -- 172.21.15.37:0/2095815896 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6480072e0 con 0x7fc6541542f0 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc6517fa640 1 -- 172.21.15.37:0/2095815896 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc648018070 con 0x7fc6541542f0 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc65afc7640 1 -- 172.21.15.37:0/2095815896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65415f4a0 con 0x7fc6541542f0 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc6517fa640 1 -- 172.21.15.37:0/2095815896 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6480057b0 con 0x7fc6541542f0 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc65afc7640 1 -- 172.21.15.37:0/2095815896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6540b66c0 con 0x7fc6541542f0 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc65afc7640 1 -- 172.21.15.37:0/2095815896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65404efa0 con 0x7fc6541542f0 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc6517fa640 1 -- 172.21.15.37:0/2095815896 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc648005950 con 0x7fc6541542f0 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc6517fa640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 0x7fc640069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc6517fa640 1 -- 172.21.15.37:0/2095815896 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc648085290 con 0x7fc6541542f0 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.311+0000 7fc658d63640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 0x7fc640069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.315+0000 7fc658d63640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 0x7fc640069420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc644009a60 tx=0x7fc644008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.315+0000 7fc6517fa640 1 -- 172.21.15.37:0/2095815896 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc648056070 con 0x7fc6541542f0 2024-04-25T08:54:47.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.475+0000 7fc65afc7640 1 -- 172.21.15.37:0/2095815896 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6540cf720 con 0x7fc640066f60 2024-04-25T08:54:47.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.491+0000 7fc6517fa640 1 -- 172.21.15.37:0/2095815896 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246641 (secure 0 0 0) 0x7fc6540cf720 con 0x7fc640066f60 2024-04-25T08:54:47.489 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:47.490 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:54:47.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 -- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 msgr2=0x7fc640069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 0x7fc640069420 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc644009a60 tx=0x7fc644008040 comp rx=0 tx=0).stop 2024-04-25T08:54:47.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 -- 172.21.15.37:0/2095815896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541542f0 msgr2=0x7fc6540b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541542f0 0x7fc6540b8c40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc64800f890 tx=0x7fc64800fd50 comp rx=0 tx=0).stop 2024-04-25T08:54:47.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 -- 172.21.15.37:0/2095815896 shutdown_connections 2024-04-25T08:54:47.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 0x7fc640069420 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc6540b6240 0x7fc6540b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc6541542f0 0x7fc6540b8c40 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 --2- 172.21.15.37:0/2095815896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c0051d0 0x7fc65415ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 -- 172.21.15.37:0/2095815896 >> 172.21.15.37:0/2095815896 conn(0x7fc65408b000 msgr2=0x7fc6540b2550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:47.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 -- 172.21.15.37:0/2095815896 shutdown_connections 2024-04-25T08:54:47.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.499+0000 7fc65afc7640 1 -- 172.21.15.37:0/2095815896 wait complete. 2024-04-25T08:54:47.509 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-04-25T08:54:47.168024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":6088,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":6234112,"data_stored":3426400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.753404"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.450878+0000","last_change":"2024-04-25T08:54:42.450878+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.450878+0000","last_undegraded":"2024-04-25T08:54:42.450878+0000","last_fullsized":"2024-04-25T08:54:42.450878+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443054+0000","last_change":"2024-04-25T08:54:42.443054+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443054+0000","last_undegraded":"2024-04-25T08:54:42.443054+0000","last_fullsized":"2024-04-25T08:54:42.443054+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444301+0000","last_change":"2024-04-25T08:54:42.444301+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444301+0000","last_undegraded":"2024-04-25T08:54:42.444301+0000","last_fullsized":"2024-04-25T08:54:42.444301+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.434500+0000","last_change":"2024-04-25T08:54:42.434500+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.434500+0000","last_undegraded":"2024-04-25T08:54:42.434500+0000","last_fullsized":"2024-04-25T08:54:42.434500+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.427901+0000","last_change":"2024-04-25T08:54:42.427901+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.427901+0000","last_undegraded":"2024-04-25T08:54:42.427901+0000","last_fullsized":"2024-04-25T08:54:42.427901+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.449668+0000","last_change":"2024-04-25T08:54:42.449668+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.449668+0000","last_undegraded":"2024-04-25T08:54:42.449668+0000","last_fullsized":"2024-04-25T08:54:42.449668+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444216+0000","last_change":"2024-04-25T08:54:42.444216+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444216+0000","last_undegraded":"2024-04-25T08:54:42.444216+0000","last_fullsized":"2024-04-25T08:54:42.444216+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.445284+0000","last_change":"2024-04-25T08:54:42.445284+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.445284+0000","last_undegraded":"2024-04-25T08:54:42.445284+0000","last_fullsized":"2024-04-25T08:54:42.445284+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.445459+0000","last_change":"2024-04-25T08:54:42.445459+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.445459+0000","last_undegraded":"2024-04-25T08:54:42.445459+0000","last_fullsized":"2024-04-25T08:54:42.445459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.449723+0000","last_change":"2024-04-25T08:54:42.449723+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.449723+0000","last_undegraded":"2024-04-25T08:54:42.449723+0000","last_fullsized":"2024-04-25T08:54:42.449723+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.437742+0000","last_change":"2024-04-25T08:54:42.437742+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.437742+0000","last_undegraded":"2024-04-25T08:54:42.437742+0000","last_fullsized":"2024-04-25T08:54:42.437742+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446682+0000","last_change":"2024-04-25T08:54:42.446682+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446682+0000","last_undegraded":"2024-04-25T08:54:42.446682+0000","last_fullsized":"2024-04-25T08:54:42.446682+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.454164+0000","last_change":"2024-04-25T08:54:42.454164+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.454164+0000","last_undegraded":"2024-04-25T08:54:42.454164+0000","last_fullsized":"2024-04-25T08:54:42.454164+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447877+0000","last_change":"2024-04-25T08:54:42.447877+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447877+0000","last_undegraded":"2024-04-25T08:54:42.447877+0000","last_fullsized":"2024-04-25T08:54:42.447877+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436375+0000","last_change":"2024-04-25T08:54:42.436375+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436375+0000","last_undegraded":"2024-04-25T08:54:42.436375+0000","last_fullsized":"2024-04-25T08:54:42.436375+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.440722+0000","last_change":"2024-04-25T08:54:42.440722+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.440722+0000","last_undegraded":"2024-04-25T08:54:42.440722+0000","last_fullsized":"2024-04-25T08:54:42.440722+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.457634+0000","last_change":"2024-04-25T08:54:42.457634+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.457634+0000","last_undegraded":"2024-04-25T08:54:42.457634+0000","last_fullsized":"2024-04-25T08:54:42.457634+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435705+0000","last_change":"2024-04-25T08:54:42.435705+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435705+0000","last_undegraded":"2024-04-25T08:54:42.435705+0000","last_fullsized":"2024-04-25T08:54:42.435705+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443799+0000","last_change":"2024-04-25T08:54:42.443799+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443799+0000","last_undegraded":"2024-04-25T08:54:42.443799+0000","last_fullsized":"2024-04-25T08:54:42.443799+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.426676+0000","last_change":"2024-04-25T08:54:42.426676+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.426676+0000","last_undegraded":"2024-04-25T08:54:42.426676+0000","last_fullsized":"2024-04-25T08:54:42.426676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.424865+0000","last_change":"2024-04-25T08:54:42.424865+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.424865+0000","last_undegraded":"2024-04-25T08:54:42.424865+0000","last_fullsized":"2024-04-25T08:54:42.424865+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446467+0000","last_change":"2024-04-25T08:54:42.446467+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446467+0000","last_undegraded":"2024-04-25T08:54:42.446467+0000","last_fullsized":"2024-04-25T08:54:42.446467+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436604+0000","last_change":"2024-04-25T08:54:42.436604+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436604+0000","last_undegraded":"2024-04-25T08:54:42.436604+0000","last_fullsized":"2024-04-25T08:54:42.436604+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443913+0000","last_change":"2024-04-25T08:54:42.443913+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443913+0000","last_undegraded":"2024-04-25T08:54:42.443913+0000","last_fullsized":"2024-04-25T08:54:42.443913+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447326+0000","last_change":"2024-04-25T08:54:42.447326+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447326+0000","last_undegraded":"2024-04-25T08:54:42.447326+0000","last_fullsized":"2024-04-25T08:54:42.447326+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.448405+0000","last_change":"2024-04-25T08:54:42.448405+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.448405+0000","last_undegraded":"2024-04-25T08:54:42.448405+0000","last_fullsized":"2024-04-25T08:54:42.448405+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443630+0000","last_change":"2024-04-25T08:54:42.443630+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443630+0000","last_undegraded":"2024-04-25T08:54:42.443630+0000","last_fullsized":"2024-04-25T08:54:42.443630+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.425583+0000","last_change":"2024-04-25T08:54:42.425583+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.425583+0000","last_undegraded":"2024-04-25T08:54:42.425583+0000","last_fullsized":"2024-04-25T08:54:42.425583+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442282+0000","last_change":"2024-04-25T08:54:42.442282+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442282+0000","last_undegraded":"2024-04-25T08:54:42.442282+0000","last_fullsized":"2024-04-25T08:54:42.442282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.441022+0000","last_change":"2024-04-25T08:54:42.441022+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.441022+0000","last_undegraded":"2024-04-25T08:54:42.441022+0000","last_fullsized":"2024-04-25T08:54:42.441022+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442094+0000","last_change":"2024-04-25T08:54:42.442094+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442094+0000","last_undegraded":"2024-04-25T08:54:42.442094+0000","last_fullsized":"2024-04-25T08:54:42.442094+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.432912+0000","last_change":"2024-04-25T08:54:42.432912+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.432912+0000","last_undegraded":"2024-04-25T08:54:42.432912+0000","last_fullsized":"2024-04-25T08:54:42.432912+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435703+0000","last_change":"2024-04-25T08:54:42.435703+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435703+0000","last_undegraded":"2024-04-25T08:54:42.435703+0000","last_fullsized":"2024-04-25T08:54:42.435703+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.454729+0000","last_change":"2024-04-25T08:54:42.435894+0000","last_active":"2024-04-25T08:54:42.454729+0000","last_peered":"2024-04-25T08:54:42.454729+0000","last_clean":"2024-04-25T08:54:42.454729+0000","last_became_active":"2024-04-25T08:54:42.435139+0000","last_became_peered":"2024-04-25T08:54:42.435139+0000","last_unstale":"2024-04-25T08:54:42.454729+0000","last_undegraded":"2024-04-25T08:54:42.454729+0000","last_fullsized":"2024-04-25T08:54:42.454729+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:55.487618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431034+0000","last_change":"2024-04-25T08:54:42.431034+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431034+0000","last_undegraded":"2024-04-25T08:54:42.431034+0000","last_fullsized":"2024-04-25T08:54:42.431034+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.448887+0000","last_change":"2024-04-25T08:54:42.434433+0000","last_active":"2024-04-25T08:54:42.448887+0000","last_peered":"2024-04-25T08:54:42.448887+0000","last_clean":"2024-04-25T08:54:42.448887+0000","last_became_active":"2024-04-25T08:54:42.434128+0000","last_became_peered":"2024-04-25T08:54:42.434128+0000","last_unstale":"2024-04-25T08:54:42.448887+0000","last_undegraded":"2024-04-25T08:54:42.448887+0000","last_fullsized":"2024-04-25T08:54:42.448887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:28.707217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.423925+0000","last_change":"2024-04-25T08:54:42.423925+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.423925+0000","last_undegraded":"2024-04-25T08:54:42.423925+0000","last_fullsized":"2024-04-25T08:54:42.423925+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.476536+0000","last_change":"2024-04-25T08:54:42.455613+0000","last_active":"2024-04-25T08:54:42.476536+0000","last_peered":"2024-04-25T08:54:42.476536+0000","last_clean":"2024-04-25T08:54:42.476536+0000","last_became_active":"2024-04-25T08:54:42.450954+0000","last_became_peered":"2024-04-25T08:54:42.450954+0000","last_unstale":"2024-04-25T08:54:42.476536+0000","last_undegraded":"2024-04-25T08:54:42.476536+0000","last_fullsized":"2024-04-25T08:54:42.476536+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439956+0000","last_change":"2024-04-25T08:54:42.439956+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439956+0000","last_undegraded":"2024-04-25T08:54:42.439956+0000","last_fullsized":"2024-04-25T08:54:42.439956+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.464425+0000","last_change":"2024-04-25T08:54:42.461742+0000","last_active":"2024-04-25T08:54:42.464425+0000","last_peered":"2024-04-25T08:54:42.464425+0000","last_clean":"2024-04-25T08:54:42.464425+0000","last_became_active":"2024-04-25T08:54:42.460925+0000","last_became_peered":"2024-04-25T08:54:42.460925+0000","last_unstale":"2024-04-25T08:54:42.464425+0000","last_undegraded":"2024-04-25T08:54:42.464425+0000","last_fullsized":"2024-04-25T08:54:42.464425+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:25.002800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.429055+0000","last_change":"2024-04-25T08:54:42.429055+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.429055+0000","last_undegraded":"2024-04-25T08:54:42.429055+0000","last_fullsized":"2024-04-25T08:54:42.429055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.474003+0000","last_change":"2024-04-25T08:54:42.468263+0000","last_active":"2024-04-25T08:54:42.474003+0000","last_peered":"2024-04-25T08:54:42.474003+0000","last_clean":"2024-04-25T08:54:42.474003+0000","last_became_active":"2024-04-25T08:54:42.467914+0000","last_became_peered":"2024-04-25T08:54:42.467914+0000","last_unstale":"2024-04-25T08:54:42.474003+0000","last_undegraded":"2024-04-25T08:54:42.474003+0000","last_fullsized":"2024-04-25T08:54:42.474003+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:08.427686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439634+0000","last_change":"2024-04-25T08:54:42.439634+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439634+0000","last_undegraded":"2024-04-25T08:54:42.439634+0000","last_fullsized":"2024-04-25T08:54:42.439634+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.443413+0000","last_change":"2024-04-25T08:54:42.439471+0000","last_active":"2024-04-25T08:54:42.443413+0000","last_peered":"2024-04-25T08:54:42.443413+0000","last_clean":"2024-04-25T08:54:42.443413+0000","last_became_active":"2024-04-25T08:54:42.438730+0000","last_became_peered":"2024-04-25T08:54:42.438730+0000","last_unstale":"2024-04-25T08:54:42.443413+0000","last_undegraded":"2024-04-25T08:54:42.443413+0000","last_fullsized":"2024-04-25T08:54:42.443413+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.440613+0000","last_change":"2024-04-25T08:54:42.440613+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.440613+0000","last_undegraded":"2024-04-25T08:54:42.440613+0000","last_fullsized":"2024-04-25T08:54:42.440613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456706+0000","last_change":"2024-04-25T08:54:42.450407+0000","last_active":"2024-04-25T08:54:42.456706+0000","last_peered":"2024-04-25T08:54:42.456706+0000","last_clean":"2024-04-25T08:54:42.456706+0000","last_became_active":"2024-04-25T08:54:42.448040+0000","last_became_peered":"2024-04-25T08:54:42.448040+0000","last_unstale":"2024-04-25T08:54:42.456706+0000","last_undegraded":"2024-04-25T08:54:42.456706+0000","last_fullsized":"2024-04-25T08:54:42.456706+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:55.906854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436423+0000","last_change":"2024-04-25T08:54:42.436423+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436423+0000","last_undegraded":"2024-04-25T08:54:42.436423+0000","last_fullsized":"2024-04-25T08:54:42.436423+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471348+0000","last_change":"2024-04-25T08:54:42.466835+0000","last_active":"2024-04-25T08:54:42.471348+0000","last_peered":"2024-04-25T08:54:42.471348+0000","last_clean":"2024-04-25T08:54:42.471348+0000","last_became_active":"2024-04-25T08:54:42.466542+0000","last_became_peered":"2024-04-25T08:54:42.466542+0000","last_unstale":"2024-04-25T08:54:42.471348+0000","last_undegraded":"2024-04-25T08:54:42.471348+0000","last_fullsized":"2024-04-25T08:54:42.471348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:46.278167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.430403+0000","last_change":"2024-04-25T08:54:42.430403+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.430403+0000","last_undegraded":"2024-04-25T08:54:42.430403+0000","last_fullsized":"2024-04-25T08:54:42.430403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.461066+0000","last_change":"2024-04-25T08:54:42.458215+0000","last_active":"2024-04-25T08:54:42.461066+0000","last_peered":"2024-04-25T08:54:42.461066+0000","last_clean":"2024-04-25T08:54:42.461066+0000","last_became_active":"2024-04-25T08:54:42.457493+0000","last_became_peered":"2024-04-25T08:54:42.457493+0000","last_unstale":"2024-04-25T08:54:42.461066+0000","last_undegraded":"2024-04-25T08:54:42.461066+0000","last_fullsized":"2024-04-25T08:54:42.461066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435429+0000","last_change":"2024-04-25T08:54:42.435429+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435429+0000","last_undegraded":"2024-04-25T08:54:42.435429+0000","last_fullsized":"2024-04-25T08:54:42.435429+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453284+0000","last_change":"2024-04-25T08:54:42.442179+0000","last_active":"2024-04-25T08:54:42.453284+0000","last_peered":"2024-04-25T08:54:42.453284+0000","last_clean":"2024-04-25T08:54:42.453284+0000","last_became_active":"2024-04-25T08:54:42.441094+0000","last_became_peered":"2024-04-25T08:54:42.441094+0000","last_unstale":"2024-04-25T08:54:42.453284+0000","last_undegraded":"2024-04-25T08:54:42.453284+0000","last_fullsized":"2024-04-25T08:54:42.453284+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442670+0000","last_change":"2024-04-25T08:54:42.442670+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442670+0000","last_undegraded":"2024-04-25T08:54:42.442670+0000","last_fullsized":"2024-04-25T08:54:42.442670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.474953+0000","last_change":"2024-04-25T08:54:42.474076+0000","last_active":"2024-04-25T08:54:42.474953+0000","last_peered":"2024-04-25T08:54:42.474953+0000","last_clean":"2024-04-25T08:54:42.474953+0000","last_became_active":"2024-04-25T08:54:42.473614+0000","last_became_peered":"2024-04-25T08:54:42.473614+0000","last_unstale":"2024-04-25T08:54:42.474953+0000","last_undegraded":"2024-04-25T08:54:42.474953+0000","last_fullsized":"2024-04-25T08:54:42.474953+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.426642+0000","last_change":"2024-04-25T08:54:42.426642+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.426642+0000","last_undegraded":"2024-04-25T08:54:42.426642+0000","last_fullsized":"2024-04-25T08:54:42.426642+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.470808+0000","last_change":"2024-04-25T08:54:42.461363+0000","last_active":"2024-04-25T08:54:42.470808+0000","last_peered":"2024-04-25T08:54:42.470808+0000","last_clean":"2024-04-25T08:54:42.470808+0000","last_became_active":"2024-04-25T08:54:42.460631+0000","last_became_peered":"2024-04-25T08:54:42.460631+0000","last_unstale":"2024-04-25T08:54:42.470808+0000","last_undegraded":"2024-04-25T08:54:42.470808+0000","last_fullsized":"2024-04-25T08:54:42.470808+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:51.318624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.433752+0000","last_change":"2024-04-25T08:54:42.433752+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.433752+0000","last_undegraded":"2024-04-25T08:54:42.433752+0000","last_fullsized":"2024-04-25T08:54:42.433752+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.465848+0000","last_change":"2024-04-25T08:54:42.456578+0000","last_active":"2024-04-25T08:54:42.465848+0000","last_peered":"2024-04-25T08:54:42.465848+0000","last_clean":"2024-04-25T08:54:42.465848+0000","last_became_active":"2024-04-25T08:54:42.456167+0000","last_became_peered":"2024-04-25T08:54:42.456167+0000","last_unstale":"2024-04-25T08:54:42.465848+0000","last_undegraded":"2024-04-25T08:54:42.465848+0000","last_fullsized":"2024-04-25T08:54:42.465848+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442253+0000","last_change":"2024-04-25T08:54:42.442253+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442253+0000","last_undegraded":"2024-04-25T08:54:42.442253+0000","last_fullsized":"2024-04-25T08:54:42.442253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456810+0000","last_change":"2024-04-25T08:54:42.451116+0000","last_active":"2024-04-25T08:54:42.456810+0000","last_peered":"2024-04-25T08:54:42.456810+0000","last_clean":"2024-04-25T08:54:42.456810+0000","last_became_active":"2024-04-25T08:54:42.447149+0000","last_became_peered":"2024-04-25T08:54:42.447149+0000","last_unstale":"2024-04-25T08:54:42.456810+0000","last_undegraded":"2024-04-25T08:54:42.456810+0000","last_fullsized":"2024-04-25T08:54:42.456810+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:35.144794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447349+0000","last_change":"2024-04-25T08:54:42.447349+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447349+0000","last_undegraded":"2024-04-25T08:54:42.447349+0000","last_fullsized":"2024-04-25T08:54:42.447349+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.460990+0000","last_change":"2024-04-25T08:54:42.458531+0000","last_active":"2024-04-25T08:54:42.460990+0000","last_peered":"2024-04-25T08:54:42.460990+0000","last_clean":"2024-04-25T08:54:42.460990+0000","last_became_active":"2024-04-25T08:54:42.457748+0000","last_became_peered":"2024-04-25T08:54:42.457748+0000","last_unstale":"2024-04-25T08:54:42.460990+0000","last_undegraded":"2024-04-25T08:54:42.460990+0000","last_fullsized":"2024-04-25T08:54:42.460990+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446203+0000","last_change":"2024-04-25T08:54:42.446203+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446203+0000","last_undegraded":"2024-04-25T08:54:42.446203+0000","last_fullsized":"2024-04-25T08:54:42.446203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.473524+0000","last_change":"2024-04-25T08:54:42.466236+0000","last_active":"2024-04-25T08:54:42.473524+0000","last_peered":"2024-04-25T08:54:42.473524+0000","last_clean":"2024-04-25T08:54:42.473524+0000","last_became_active":"2024-04-25T08:54:42.465507+0000","last_became_peered":"2024-04-25T08:54:42.465507+0000","last_unstale":"2024-04-25T08:54:42.473524+0000","last_undegraded":"2024-04-25T08:54:42.473524+0000","last_fullsized":"2024-04-25T08:54:42.473524+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:22.360154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439111+0000","last_change":"2024-04-25T08:54:42.439111+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439111+0000","last_undegraded":"2024-04-25T08:54:42.439111+0000","last_fullsized":"2024-04-25T08:54:42.439111+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.473955+0000","last_change":"2024-04-25T08:54:42.469518+0000","last_active":"2024-04-25T08:54:42.473955+0000","last_peered":"2024-04-25T08:54:42.473955+0000","last_clean":"2024-04-25T08:54:42.473955+0000","last_became_active":"2024-04-25T08:54:42.469147+0000","last_became_peered":"2024-04-25T08:54:42.469147+0000","last_unstale":"2024-04-25T08:54:42.473955+0000","last_undegraded":"2024-04-25T08:54:42.473955+0000","last_fullsized":"2024-04-25T08:54:42.473955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:44.662568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.437803+0000","last_change":"2024-04-25T08:54:42.437803+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.437803+0000","last_undegraded":"2024-04-25T08:54:42.437803+0000","last_fullsized":"2024-04-25T08:54:42.437803+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453781+0000","last_change":"2024-04-25T08:54:42.450583+0000","last_active":"2024-04-25T08:54:42.453781+0000","last_peered":"2024-04-25T08:54:42.453781+0000","last_clean":"2024-04-25T08:54:42.453781+0000","last_became_active":"2024-04-25T08:54:42.450316+0000","last_became_peered":"2024-04-25T08:54:42.450316+0000","last_unstale":"2024-04-25T08:54:42.453781+0000","last_undegraded":"2024-04-25T08:54:42.453781+0000","last_fullsized":"2024-04-25T08:54:42.453781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:41.112056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.428105+0000","last_change":"2024-04-25T08:54:42.428105+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.428105+0000","last_undegraded":"2024-04-25T08:54:42.428105+0000","last_fullsized":"2024-04-25T08:54:42.428105+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.457250+0000","last_change":"2024-04-25T08:54:42.444423+0000","last_active":"2024-04-25T08:54:42.457250+0000","last_peered":"2024-04-25T08:54:42.457250+0000","last_clean":"2024-04-25T08:54:42.457250+0000","last_became_active":"2024-04-25T08:54:42.439289+0000","last_became_peered":"2024-04-25T08:54:42.439289+0000","last_unstale":"2024-04-25T08:54:42.457250+0000","last_undegraded":"2024-04-25T08:54:42.457250+0000","last_fullsized":"2024-04-25T08:54:42.457250+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:40.820232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439073+0000","last_change":"2024-04-25T08:54:42.439073+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439073+0000","last_undegraded":"2024-04-25T08:54:42.439073+0000","last_fullsized":"2024-04-25T08:54:42.439073+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.454629+0000","last_change":"2024-04-25T08:54:42.436282+0000","last_active":"2024-04-25T08:54:42.454629+0000","last_peered":"2024-04-25T08:54:42.454629+0000","last_clean":"2024-04-25T08:54:42.454629+0000","last_became_active":"2024-04-25T08:54:42.435494+0000","last_became_peered":"2024-04-25T08:54:42.435494+0000","last_unstale":"2024-04-25T08:54:42.454629+0000","last_undegraded":"2024-04-25T08:54:42.454629+0000","last_fullsized":"2024-04-25T08:54:42.454629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:13.276684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.432676+0000","last_change":"2024-04-25T08:54:42.432676+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.432676+0000","last_undegraded":"2024-04-25T08:54:42.432676+0000","last_fullsized":"2024-04-25T08:54:42.432676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.476553+0000","last_change":"2024-04-25T08:54:42.456188+0000","last_active":"2024-04-25T08:54:42.476553+0000","last_peered":"2024-04-25T08:54:42.476553+0000","last_clean":"2024-04-25T08:54:42.476553+0000","last_became_active":"2024-04-25T08:54:42.455422+0000","last_became_peered":"2024-04-25T08:54:42.455422+0000","last_unstale":"2024-04-25T08:54:42.476553+0000","last_undegraded":"2024-04-25T08:54:42.476553+0000","last_fullsized":"2024-04-25T08:54:42.476553+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.427057+0000","last_change":"2024-04-25T08:54:42.427057+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.427057+0000","last_undegraded":"2024-04-25T08:54:42.427057+0000","last_fullsized":"2024-04-25T08:54:42.427057+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471615+0000","last_change":"2024-04-25T08:54:42.463454+0000","last_active":"2024-04-25T08:54:42.471615+0000","last_peered":"2024-04-25T08:54:42.471615+0000","last_clean":"2024-04-25T08:54:42.471615+0000","last_became_active":"2024-04-25T08:54:42.461962+0000","last_became_peered":"2024-04-25T08:54:42.461962+0000","last_unstale":"2024-04-25T08:54:42.471615+0000","last_undegraded":"2024-04-25T08:54:42.471615+0000","last_fullsized":"2024-04-25T08:54:42.471615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:08.285003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.441892+0000","last_change":"2024-04-25T08:54:42.441892+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.441892+0000","last_undegraded":"2024-04-25T08:54:42.441892+0000","last_fullsized":"2024-04-25T08:54:42.441892+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.465722+0000","last_change":"2024-04-25T08:54:42.456453+0000","last_active":"2024-04-25T08:54:42.465722+0000","last_peered":"2024-04-25T08:54:42.465722+0000","last_clean":"2024-04-25T08:54:42.465722+0000","last_became_active":"2024-04-25T08:54:42.456124+0000","last_became_peered":"2024-04-25T08:54:42.456124+0000","last_unstale":"2024-04-25T08:54:42.465722+0000","last_undegraded":"2024-04-25T08:54:42.465722+0000","last_fullsized":"2024-04-25T08:54:42.465722+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439639+0000","last_change":"2024-04-25T08:54:42.439639+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439639+0000","last_undegraded":"2024-04-25T08:54:42.439639+0000","last_fullsized":"2024-04-25T08:54:42.439639+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453850+0000","last_change":"2024-04-25T08:54:42.448496+0000","last_active":"2024-04-25T08:54:42.453850+0000","last_peered":"2024-04-25T08:54:42.453850+0000","last_clean":"2024-04-25T08:54:42.453850+0000","last_became_active":"2024-04-25T08:54:42.447727+0000","last_became_peered":"2024-04-25T08:54:42.447727+0000","last_unstale":"2024-04-25T08:54:42.453850+0000","last_undegraded":"2024-04-25T08:54:42.453850+0000","last_fullsized":"2024-04-25T08:54:42.453850+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:30.126539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431858+0000","last_change":"2024-04-25T08:54:42.431858+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431858+0000","last_undegraded":"2024-04-25T08:54:42.431858+0000","last_fullsized":"2024-04-25T08:54:42.431858+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456883+0000","last_change":"2024-04-25T08:54:42.449480+0000","last_active":"2024-04-25T08:54:42.456883+0000","last_peered":"2024-04-25T08:54:42.456883+0000","last_clean":"2024-04-25T08:54:42.456883+0000","last_became_active":"2024-04-25T08:54:42.448749+0000","last_became_peered":"2024-04-25T08:54:42.448749+0000","last_unstale":"2024-04-25T08:54:42.456883+0000","last_undegraded":"2024-04-25T08:54:42.456883+0000","last_fullsized":"2024-04-25T08:54:42.456883+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:19.814248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439551+0000","last_change":"2024-04-25T08:54:42.439551+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439551+0000","last_undegraded":"2024-04-25T08:54:42.439551+0000","last_fullsized":"2024-04-25T08:54:42.439551+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.459614+0000","last_change":"2024-04-25T08:54:42.455018+0000","last_active":"2024-04-25T08:54:42.459614+0000","last_peered":"2024-04-25T08:54:42.459614+0000","last_clean":"2024-04-25T08:54:42.459614+0000","last_became_active":"2024-04-25T08:54:42.454140+0000","last_became_peered":"2024-04-25T08:54:42.454140+0000","last_unstale":"2024-04-25T08:54:42.459614+0000","last_undegraded":"2024-04-25T08:54:42.459614+0000","last_fullsized":"2024-04-25T08:54:42.459614+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:28.707217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.438653+0000","last_change":"2024-04-25T08:54:42.438653+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.438653+0000","last_undegraded":"2024-04-25T08:54:42.438653+0000","last_fullsized":"2024-04-25T08:54:42.438653+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.466103+0000","last_change":"2024-04-25T08:54:42.464859+0000","last_active":"2024-04-25T08:54:42.466103+0000","last_peered":"2024-04-25T08:54:42.466103+0000","last_clean":"2024-04-25T08:54:42.466103+0000","last_became_active":"2024-04-25T08:54:42.457143+0000","last_became_peered":"2024-04-25T08:54:42.457143+0000","last_unstale":"2024-04-25T08:54:42.466103+0000","last_undegraded":"2024-04-25T08:54:42.466103+0000","last_fullsized":"2024-04-25T08:54:42.466103+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444706+0000","last_change":"2024-04-25T08:54:42.444706+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444706+0000","last_undegraded":"2024-04-25T08:54:42.444706+0000","last_fullsized":"2024-04-25T08:54:42.444706+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471541+0000","last_change":"2024-04-25T08:54:42.463152+0000","last_active":"2024-04-25T08:54:42.471541+0000","last_peered":"2024-04-25T08:54:42.471541+0000","last_clean":"2024-04-25T08:54:42.471541+0000","last_became_active":"2024-04-25T08:54:42.462413+0000","last_became_peered":"2024-04-25T08:54:42.462413+0000","last_unstale":"2024-04-25T08:54:42.471541+0000","last_undegraded":"2024-04-25T08:54:42.471541+0000","last_fullsized":"2024-04-25T08:54:42.471541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:15.835374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431106+0000","last_change":"2024-04-25T08:54:42.431106+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431106+0000","last_undegraded":"2024-04-25T08:54:42.431106+0000","last_fullsized":"2024-04-25T08:54:42.431106+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471662+0000","last_change":"2024-04-25T08:54:42.463691+0000","last_active":"2024-04-25T08:54:42.471662+0000","last_peered":"2024-04-25T08:54:42.471662+0000","last_clean":"2024-04-25T08:54:42.471662+0000","last_became_active":"2024-04-25T08:54:42.462922+0000","last_became_peered":"2024-04-25T08:54:42.462922+0000","last_unstale":"2024-04-25T08:54:42.471662+0000","last_undegraded":"2024-04-25T08:54:42.471662+0000","last_fullsized":"2024-04-25T08:54:42.471662+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:18.405142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442356+0000","last_change":"2024-04-25T08:54:42.442356+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442356+0000","last_undegraded":"2024-04-25T08:54:42.442356+0000","last_fullsized":"2024-04-25T08:54:42.442356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.452975+0000","last_change":"2024-04-25T08:54:42.441517+0000","last_active":"2024-04-25T08:54:42.452975+0000","last_peered":"2024-04-25T08:54:42.452975+0000","last_clean":"2024-04-25T08:54:42.452975+0000","last_became_active":"2024-04-25T08:54:42.439663+0000","last_became_peered":"2024-04-25T08:54:42.439663+0000","last_unstale":"2024-04-25T08:54:42.452975+0000","last_undegraded":"2024-04-25T08:54:42.452975+0000","last_fullsized":"2024-04-25T08:54:42.452975+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439988+0000","last_change":"2024-04-25T08:54:42.439988+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439988+0000","last_undegraded":"2024-04-25T08:54:42.439988+0000","last_fullsized":"2024-04-25T08:54:42.439988+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.459448+0000","last_change":"2024-04-25T08:54:42.456403+0000","last_active":"2024-04-25T08:54:42.459448+0000","last_peered":"2024-04-25T08:54:42.459448+0000","last_clean":"2024-04-25T08:54:42.459448+0000","last_became_active":"2024-04-25T08:54:42.455569+0000","last_became_peered":"2024-04-25T08:54:42.455569+0000","last_unstale":"2024-04-25T08:54:42.459448+0000","last_undegraded":"2024-04-25T08:54:42.459448+0000","last_fullsized":"2024-04-25T08:54:42.459448+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:44.370399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.423932+0000","last_change":"2024-04-25T08:54:42.423932+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.423932+0000","last_undegraded":"2024-04-25T08:54:42.423932+0000","last_fullsized":"2024-04-25T08:54:42.423932+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471131+0000","last_change":"2024-04-25T08:54:42.466348+0000","last_active":"2024-04-25T08:54:42.471131+0000","last_peered":"2024-04-25T08:54:42.471131+0000","last_clean":"2024-04-25T08:54:42.471131+0000","last_became_active":"2024-04-25T08:54:42.456331+0000","last_became_peered":"2024-04-25T08:54:42.456331+0000","last_unstale":"2024-04-25T08:54:42.471131+0000","last_undegraded":"2024-04-25T08:54:42.471131+0000","last_fullsized":"2024-04-25T08:54:42.471131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:34.520291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":231,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.445808+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:54:42.445808+0000","last_peered":"2024-04-25T08:54:42.445808+0000","last_clean":"2024-04-25T08:54:42.445808+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:54:42.445808+0000","last_undegraded":"2024-04-25T08:54:42.445808+0000","last_fullsized":"2024-04-25T08:54:42.445808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1224704,"data_stored":871076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31796,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1224704,"data_stored":871076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:54:47.514 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:54:47.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.683+0000 7f6593121640 1 Processor -- start 2024-04-25T08:54:47.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.683+0000 7f6593121640 1 -- start start 2024-04-25T08:54:47.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f6593121640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f658c0b0a30 0x7f658c0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f6593121640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658c0b0e10 con 0x7f658c06f4a0 2024-04-25T08:54:47.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f6593121640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f658c0d0180 con 0x7f658c09a180 2024-04-25T08:54:47.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f6593121640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658c0d0d90 con 0x7f658c0b0a30 2024-04-25T08:54:47.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f658c06f4a0 0x7f658c099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:39274/0 (socket says 172.21.15.37:39274) 2024-04-25T08:54:47.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658bfff640 1 -- 172.21.15.37:0/2799740109 learned_addr learned my addr 172.21.15.37:0/2799740109 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:47.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112501076 0 0) 0x7f658c0b0e10 con 0x7f658c06f4a0 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f6590ebd640 1 --2- 172.21.15.37:0/2799740109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f658c0b0a30 0x7f658c0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f657c003530 con 0x7f658c06f4a0 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4110489718 0 0) 0x7f657c003530 con 0x7f658c06f4a0 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f658c0b0a30 msgr2=0x7f658c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 --2- 172.21.15.37:0/2799740109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f658c0b0a30 0x7f658c0cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 >> v1:172.21.15.203:6789/0 conn(0x7f658c09a180 legacy=0x7f658c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c0d0460 con 0x7f658c06f4a0 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f6580001680 con 0x7f658c06f4a0 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 >> v1:172.21.15.37:6789/0 conn(0x7f658c06f4a0 legacy=0x7f658c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 --2- 172.21.15.37:0/2799740109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657c003a60 0x7f657c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 --2- 172.21.15.37:0/2799740109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f658c0b0a30 0x7f657c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 --2- 172.21.15.37:0/2799740109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f657c004d50 0x7f657c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c0d0d90 con 0x7f657c003a60 2024-04-25T08:54:47.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657c00bae0 con 0x7f657c004d50 2024-04-25T08:54:47.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657c00bce0 con 0x7f658c0b0a30 2024-04-25T08:54:47.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f6590ebd640 1 --2- 172.21.15.37:0/2799740109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657c003a60 0x7f657c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f65916be640 1 --2- 172.21.15.37:0/2799740109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f657c004d50 0x7f657c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f6590ebd640 1 -- 172.21.15.37:0/2799740109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f658c0b0a30 msgr2=0x7f657c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f6590ebd640 1 --2- 172.21.15.37:0/2799740109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f658c0b0a30 0x7f657c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f6590ebd640 1 -- 172.21.15.37:0/2799740109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f657c004d50 msgr2=0x7f657c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f6590ebd640 1 --2- 172.21.15.37:0/2799740109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f657c004d50 0x7f657c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.687+0000 7f6590ebd640 1 -- 172.21.15.37:0/2799740109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657c003530 con 0x7f657c003a60 2024-04-25T08:54:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6590ebd640 1 --2- 172.21.15.37:0/2799740109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657c003a60 0x7f657c003e80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6578009ab0 tx=0x7f657801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f73eef5eae1e82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65780b2070 con 0x7f657c003a60 2024-04-25T08:54:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65780a5530 con 0x7f657c003a60 2024-04-25T08:54:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f658b7fe640 1 -- 172.21.15.37:0/2799740109 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65780096b0 con 0x7f657c003a60 2024-04-25T08:54:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 -- 172.21.15.37:0/2799740109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657c003a60 msgr2=0x7f657c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 --2- 172.21.15.37:0/2799740109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f657c003a60 0x7f657c003e80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6578009ab0 tx=0x7f657801c9a0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f65916be640 1 -- 172.21.15.37:0/2799740109 reap_dead start 2024-04-25T08:54:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 -- 172.21.15.37:0/2799740109 shutdown_connections 2024-04-25T08:54:47.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 -- 172.21.15.37:0/2799740109 >> 172.21.15.37:0/2799740109 conn(0x7f658c0b30b0 msgr2=0x7f658c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:47.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 -- 172.21.15.37:0/2799740109 shutdown_connections 2024-04-25T08:54:47.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 -- 172.21.15.37:0/2799740109 wait complete. 2024-04-25T08:54:47.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 Processor -- start 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 -- start start 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f657c003a60 0x7f658c0c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f658c155d20 0x7f658c06bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c06c2e0 0x7f658c0c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c15dd50 con 0x7f658c06c2e0 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c0cce30 con 0x7f658c155d20 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6593121640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f658c15ded0 con 0x7f657c003a60 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f65916be640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c06c2e0 0x7f658c0c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f65916be640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c06c2e0 0x7f658c0c6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55426/0 (socket says 172.21.15.37:55426) 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f65916be640 1 -- 172.21.15.37:0/1839290717 learned_addr learned my addr 172.21.15.37:0/1839290717 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f6590ebd640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f657c003a60 0x7f658c0c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.691+0000 7f658bfff640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f658c155d20 0x7f658c06bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65916be640 1 -- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f657c003a60 msgr2=0x7f658c0c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65916be640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f657c003a60 0x7f658c0c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65916be640 1 -- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f658c155d20 msgr2=0x7f658c06bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65916be640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f658c155d20 0x7f658c06bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65916be640 1 -- 172.21.15.37:0/1839290717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c15ac40 con 0x7f658c06c2e0 2024-04-25T08:54:47.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65916be640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c06c2e0 0x7f658c0c6f40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f658c0cf050 tx=0x7f658400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65897fa640 1 -- 172.21.15.37:0/1839290717 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584003980 con 0x7f658c06c2e0 2024-04-25T08:54:47.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65897fa640 1 -- 172.21.15.37:0/1839290717 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6584018070 con 0x7f658c06c2e0 2024-04-25T08:54:47.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65897fa640 1 -- 172.21.15.37:0/1839290717 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584003d60 con 0x7f658c06c2e0 2024-04-25T08:54:47.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f6593121640 1 -- 172.21.15.37:0/1839290717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658c15af20 con 0x7f658c06c2e0 2024-04-25T08:54:47.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f6593121640 1 -- 172.21.15.37:0/1839290717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658c15b4e0 con 0x7f658c06c2e0 2024-04-25T08:54:47.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f6593121640 1 -- 172.21.15.37:0/1839290717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658c06c760 con 0x7f658c06c2e0 2024-04-25T08:54:47.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65897fa640 1 -- 172.21.15.37:0/1839290717 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65840050d0 con 0x7f658c06c2e0 2024-04-25T08:54:47.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65897fa640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6570066f60 0x7f6570069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f65897fa640 1 -- 172.21.15.37:0/1839290717 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6584084ee0 con 0x7f658c06c2e0 2024-04-25T08:54:47.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f6590ebd640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6570066f60 0x7f6570069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.695+0000 7f6590ebd640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6570066f60 0x7f6570069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f65780af000 tx=0x7f65780046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.699+0000 7f65897fa640 1 -- 172.21.15.37:0/1839290717 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6584055d60 con 0x7f658c06c2e0 2024-04-25T08:54:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.855+0000 7f6593121640 1 -- 172.21.15.37:0/1839290717 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f658c15a9f0 con 0x7f6570066f60 2024-04-25T08:54:47.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.875+0000 7f65897fa640 1 -- 172.21.15.37:0/1839290717 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246641 (secure 0 0 0) 0x7f65780f8390 con 0x7f6570066f60 2024-04-25T08:54:47.870 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:47.871 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:54:47.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 -- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6570066f60 msgr2=0x7f6570069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6570066f60 0x7f6570069420 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f65780af000 tx=0x7f65780046d0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 -- 172.21.15.37:0/1839290717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c06c2e0 msgr2=0x7f658c0c6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c06c2e0 0x7f658c0c6f40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f658c0cf050 tx=0x7f658400ec10 comp rx=0 tx=0).stop 2024-04-25T08:54:47.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 -- 172.21.15.37:0/1839290717 shutdown_connections 2024-04-25T08:54:47.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6570066f60 0x7f6570069420 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f658c06c2e0 0x7f658c0c6f40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f658c155d20 0x7f658c06bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 --2- 172.21.15.37:0/1839290717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f657c003a60 0x7f658c0c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 -- 172.21.15.37:0/1839290717 >> 172.21.15.37:0/1839290717 conn(0x7f658c0b30b0 msgr2=0x7f658c0ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:47.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 -- 172.21.15.37:0/1839290717 shutdown_connections 2024-04-25T08:54:47.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:47.879+0000 7f6593121640 1 -- 172.21.15.37:0/1839290717 wait complete. 2024-04-25T08:54:47.889 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-04-25T08:54:47.168024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":6088,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":6234112,"data_stored":3426400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.753404"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.450878+0000","last_change":"2024-04-25T08:54:42.450878+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.450878+0000","last_undegraded":"2024-04-25T08:54:42.450878+0000","last_fullsized":"2024-04-25T08:54:42.450878+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443054+0000","last_change":"2024-04-25T08:54:42.443054+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443054+0000","last_undegraded":"2024-04-25T08:54:42.443054+0000","last_fullsized":"2024-04-25T08:54:42.443054+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444301+0000","last_change":"2024-04-25T08:54:42.444301+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444301+0000","last_undegraded":"2024-04-25T08:54:42.444301+0000","last_fullsized":"2024-04-25T08:54:42.444301+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.434500+0000","last_change":"2024-04-25T08:54:42.434500+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.434500+0000","last_undegraded":"2024-04-25T08:54:42.434500+0000","last_fullsized":"2024-04-25T08:54:42.434500+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.427901+0000","last_change":"2024-04-25T08:54:42.427901+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.427901+0000","last_undegraded":"2024-04-25T08:54:42.427901+0000","last_fullsized":"2024-04-25T08:54:42.427901+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.449668+0000","last_change":"2024-04-25T08:54:42.449668+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.449668+0000","last_undegraded":"2024-04-25T08:54:42.449668+0000","last_fullsized":"2024-04-25T08:54:42.449668+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444216+0000","last_change":"2024-04-25T08:54:42.444216+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444216+0000","last_undegraded":"2024-04-25T08:54:42.444216+0000","last_fullsized":"2024-04-25T08:54:42.444216+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.445284+0000","last_change":"2024-04-25T08:54:42.445284+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.445284+0000","last_undegraded":"2024-04-25T08:54:42.445284+0000","last_fullsized":"2024-04-25T08:54:42.445284+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.445459+0000","last_change":"2024-04-25T08:54:42.445459+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.445459+0000","last_undegraded":"2024-04-25T08:54:42.445459+0000","last_fullsized":"2024-04-25T08:54:42.445459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.449723+0000","last_change":"2024-04-25T08:54:42.449723+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.449723+0000","last_undegraded":"2024-04-25T08:54:42.449723+0000","last_fullsized":"2024-04-25T08:54:42.449723+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.437742+0000","last_change":"2024-04-25T08:54:42.437742+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.437742+0000","last_undegraded":"2024-04-25T08:54:42.437742+0000","last_fullsized":"2024-04-25T08:54:42.437742+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446682+0000","last_change":"2024-04-25T08:54:42.446682+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446682+0000","last_undegraded":"2024-04-25T08:54:42.446682+0000","last_fullsized":"2024-04-25T08:54:42.446682+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.454164+0000","last_change":"2024-04-25T08:54:42.454164+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.454164+0000","last_undegraded":"2024-04-25T08:54:42.454164+0000","last_fullsized":"2024-04-25T08:54:42.454164+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447877+0000","last_change":"2024-04-25T08:54:42.447877+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447877+0000","last_undegraded":"2024-04-25T08:54:42.447877+0000","last_fullsized":"2024-04-25T08:54:42.447877+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436375+0000","last_change":"2024-04-25T08:54:42.436375+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436375+0000","last_undegraded":"2024-04-25T08:54:42.436375+0000","last_fullsized":"2024-04-25T08:54:42.436375+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.440722+0000","last_change":"2024-04-25T08:54:42.440722+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.440722+0000","last_undegraded":"2024-04-25T08:54:42.440722+0000","last_fullsized":"2024-04-25T08:54:42.440722+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.457634+0000","last_change":"2024-04-25T08:54:42.457634+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.457634+0000","last_undegraded":"2024-04-25T08:54:42.457634+0000","last_fullsized":"2024-04-25T08:54:42.457634+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435705+0000","last_change":"2024-04-25T08:54:42.435705+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435705+0000","last_undegraded":"2024-04-25T08:54:42.435705+0000","last_fullsized":"2024-04-25T08:54:42.435705+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443799+0000","last_change":"2024-04-25T08:54:42.443799+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443799+0000","last_undegraded":"2024-04-25T08:54:42.443799+0000","last_fullsized":"2024-04-25T08:54:42.443799+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.426676+0000","last_change":"2024-04-25T08:54:42.426676+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.426676+0000","last_undegraded":"2024-04-25T08:54:42.426676+0000","last_fullsized":"2024-04-25T08:54:42.426676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.424865+0000","last_change":"2024-04-25T08:54:42.424865+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.424865+0000","last_undegraded":"2024-04-25T08:54:42.424865+0000","last_fullsized":"2024-04-25T08:54:42.424865+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446467+0000","last_change":"2024-04-25T08:54:42.446467+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446467+0000","last_undegraded":"2024-04-25T08:54:42.446467+0000","last_fullsized":"2024-04-25T08:54:42.446467+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436604+0000","last_change":"2024-04-25T08:54:42.436604+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436604+0000","last_undegraded":"2024-04-25T08:54:42.436604+0000","last_fullsized":"2024-04-25T08:54:42.436604+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443913+0000","last_change":"2024-04-25T08:54:42.443913+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443913+0000","last_undegraded":"2024-04-25T08:54:42.443913+0000","last_fullsized":"2024-04-25T08:54:42.443913+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447326+0000","last_change":"2024-04-25T08:54:42.447326+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447326+0000","last_undegraded":"2024-04-25T08:54:42.447326+0000","last_fullsized":"2024-04-25T08:54:42.447326+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.448405+0000","last_change":"2024-04-25T08:54:42.448405+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.448405+0000","last_undegraded":"2024-04-25T08:54:42.448405+0000","last_fullsized":"2024-04-25T08:54:42.448405+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443630+0000","last_change":"2024-04-25T08:54:42.443630+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443630+0000","last_undegraded":"2024-04-25T08:54:42.443630+0000","last_fullsized":"2024-04-25T08:54:42.443630+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.425583+0000","last_change":"2024-04-25T08:54:42.425583+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.425583+0000","last_undegraded":"2024-04-25T08:54:42.425583+0000","last_fullsized":"2024-04-25T08:54:42.425583+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442282+0000","last_change":"2024-04-25T08:54:42.442282+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442282+0000","last_undegraded":"2024-04-25T08:54:42.442282+0000","last_fullsized":"2024-04-25T08:54:42.442282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.441022+0000","last_change":"2024-04-25T08:54:42.441022+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.441022+0000","last_undegraded":"2024-04-25T08:54:42.441022+0000","last_fullsized":"2024-04-25T08:54:42.441022+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442094+0000","last_change":"2024-04-25T08:54:42.442094+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442094+0000","last_undegraded":"2024-04-25T08:54:42.442094+0000","last_fullsized":"2024-04-25T08:54:42.442094+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.432912+0000","last_change":"2024-04-25T08:54:42.432912+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.432912+0000","last_undegraded":"2024-04-25T08:54:42.432912+0000","last_fullsized":"2024-04-25T08:54:42.432912+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435703+0000","last_change":"2024-04-25T08:54:42.435703+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435703+0000","last_undegraded":"2024-04-25T08:54:42.435703+0000","last_fullsized":"2024-04-25T08:54:42.435703+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.454729+0000","last_change":"2024-04-25T08:54:42.435894+0000","last_active":"2024-04-25T08:54:42.454729+0000","last_peered":"2024-04-25T08:54:42.454729+0000","last_clean":"2024-04-25T08:54:42.454729+0000","last_became_active":"2024-04-25T08:54:42.435139+0000","last_became_peered":"2024-04-25T08:54:42.435139+0000","last_unstale":"2024-04-25T08:54:42.454729+0000","last_undegraded":"2024-04-25T08:54:42.454729+0000","last_fullsized":"2024-04-25T08:54:42.454729+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:55.487618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431034+0000","last_change":"2024-04-25T08:54:42.431034+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431034+0000","last_undegraded":"2024-04-25T08:54:42.431034+0000","last_fullsized":"2024-04-25T08:54:42.431034+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.448887+0000","last_change":"2024-04-25T08:54:42.434433+0000","last_active":"2024-04-25T08:54:42.448887+0000","last_peered":"2024-04-25T08:54:42.448887+0000","last_clean":"2024-04-25T08:54:42.448887+0000","last_became_active":"2024-04-25T08:54:42.434128+0000","last_became_peered":"2024-04-25T08:54:42.434128+0000","last_unstale":"2024-04-25T08:54:42.448887+0000","last_undegraded":"2024-04-25T08:54:42.448887+0000","last_fullsized":"2024-04-25T08:54:42.448887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:28.707217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.423925+0000","last_change":"2024-04-25T08:54:42.423925+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.423925+0000","last_undegraded":"2024-04-25T08:54:42.423925+0000","last_fullsized":"2024-04-25T08:54:42.423925+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.476536+0000","last_change":"2024-04-25T08:54:42.455613+0000","last_active":"2024-04-25T08:54:42.476536+0000","last_peered":"2024-04-25T08:54:42.476536+0000","last_clean":"2024-04-25T08:54:42.476536+0000","last_became_active":"2024-04-25T08:54:42.450954+0000","last_became_peered":"2024-04-25T08:54:42.450954+0000","last_unstale":"2024-04-25T08:54:42.476536+0000","last_undegraded":"2024-04-25T08:54:42.476536+0000","last_fullsized":"2024-04-25T08:54:42.476536+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439956+0000","last_change":"2024-04-25T08:54:42.439956+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439956+0000","last_undegraded":"2024-04-25T08:54:42.439956+0000","last_fullsized":"2024-04-25T08:54:42.439956+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.464425+0000","last_change":"2024-04-25T08:54:42.461742+0000","last_active":"2024-04-25T08:54:42.464425+0000","last_peered":"2024-04-25T08:54:42.464425+0000","last_clean":"2024-04-25T08:54:42.464425+0000","last_became_active":"2024-04-25T08:54:42.460925+0000","last_became_peered":"2024-04-25T08:54:42.460925+0000","last_unstale":"2024-04-25T08:54:42.464425+0000","last_undegraded":"2024-04-25T08:54:42.464425+0000","last_fullsized":"2024-04-25T08:54:42.464425+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:25.002800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.429055+0000","last_change":"2024-04-25T08:54:42.429055+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.429055+0000","last_undegraded":"2024-04-25T08:54:42.429055+0000","last_fullsized":"2024-04-25T08:54:42.429055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.474003+0000","last_change":"2024-04-25T08:54:42.468263+0000","last_active":"2024-04-25T08:54:42.474003+0000","last_peered":"2024-04-25T08:54:42.474003+0000","last_clean":"2024-04-25T08:54:42.474003+0000","last_became_active":"2024-04-25T08:54:42.467914+0000","last_became_peered":"2024-04-25T08:54:42.467914+0000","last_unstale":"2024-04-25T08:54:42.474003+0000","last_undegraded":"2024-04-25T08:54:42.474003+0000","last_fullsized":"2024-04-25T08:54:42.474003+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:08.427686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439634+0000","last_change":"2024-04-25T08:54:42.439634+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439634+0000","last_undegraded":"2024-04-25T08:54:42.439634+0000","last_fullsized":"2024-04-25T08:54:42.439634+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.443413+0000","last_change":"2024-04-25T08:54:42.439471+0000","last_active":"2024-04-25T08:54:42.443413+0000","last_peered":"2024-04-25T08:54:42.443413+0000","last_clean":"2024-04-25T08:54:42.443413+0000","last_became_active":"2024-04-25T08:54:42.438730+0000","last_became_peered":"2024-04-25T08:54:42.438730+0000","last_unstale":"2024-04-25T08:54:42.443413+0000","last_undegraded":"2024-04-25T08:54:42.443413+0000","last_fullsized":"2024-04-25T08:54:42.443413+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.440613+0000","last_change":"2024-04-25T08:54:42.440613+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.440613+0000","last_undegraded":"2024-04-25T08:54:42.440613+0000","last_fullsized":"2024-04-25T08:54:42.440613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456706+0000","last_change":"2024-04-25T08:54:42.450407+0000","last_active":"2024-04-25T08:54:42.456706+0000","last_peered":"2024-04-25T08:54:42.456706+0000","last_clean":"2024-04-25T08:54:42.456706+0000","last_became_active":"2024-04-25T08:54:42.448040+0000","last_became_peered":"2024-04-25T08:54:42.448040+0000","last_unstale":"2024-04-25T08:54:42.456706+0000","last_undegraded":"2024-04-25T08:54:42.456706+0000","last_fullsized":"2024-04-25T08:54:42.456706+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:55.906854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436423+0000","last_change":"2024-04-25T08:54:42.436423+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436423+0000","last_undegraded":"2024-04-25T08:54:42.436423+0000","last_fullsized":"2024-04-25T08:54:42.436423+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471348+0000","last_change":"2024-04-25T08:54:42.466835+0000","last_active":"2024-04-25T08:54:42.471348+0000","last_peered":"2024-04-25T08:54:42.471348+0000","last_clean":"2024-04-25T08:54:42.471348+0000","last_became_active":"2024-04-25T08:54:42.466542+0000","last_became_peered":"2024-04-25T08:54:42.466542+0000","last_unstale":"2024-04-25T08:54:42.471348+0000","last_undegraded":"2024-04-25T08:54:42.471348+0000","last_fullsized":"2024-04-25T08:54:42.471348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:46.278167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.430403+0000","last_change":"2024-04-25T08:54:42.430403+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.430403+0000","last_undegraded":"2024-04-25T08:54:42.430403+0000","last_fullsized":"2024-04-25T08:54:42.430403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.461066+0000","last_change":"2024-04-25T08:54:42.458215+0000","last_active":"2024-04-25T08:54:42.461066+0000","last_peered":"2024-04-25T08:54:42.461066+0000","last_clean":"2024-04-25T08:54:42.461066+0000","last_became_active":"2024-04-25T08:54:42.457493+0000","last_became_peered":"2024-04-25T08:54:42.457493+0000","last_unstale":"2024-04-25T08:54:42.461066+0000","last_undegraded":"2024-04-25T08:54:42.461066+0000","last_fullsized":"2024-04-25T08:54:42.461066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435429+0000","last_change":"2024-04-25T08:54:42.435429+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435429+0000","last_undegraded":"2024-04-25T08:54:42.435429+0000","last_fullsized":"2024-04-25T08:54:42.435429+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453284+0000","last_change":"2024-04-25T08:54:42.442179+0000","last_active":"2024-04-25T08:54:42.453284+0000","last_peered":"2024-04-25T08:54:42.453284+0000","last_clean":"2024-04-25T08:54:42.453284+0000","last_became_active":"2024-04-25T08:54:42.441094+0000","last_became_peered":"2024-04-25T08:54:42.441094+0000","last_unstale":"2024-04-25T08:54:42.453284+0000","last_undegraded":"2024-04-25T08:54:42.453284+0000","last_fullsized":"2024-04-25T08:54:42.453284+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442670+0000","last_change":"2024-04-25T08:54:42.442670+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442670+0000","last_undegraded":"2024-04-25T08:54:42.442670+0000","last_fullsized":"2024-04-25T08:54:42.442670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.474953+0000","last_change":"2024-04-25T08:54:42.474076+0000","last_active":"2024-04-25T08:54:42.474953+0000","last_peered":"2024-04-25T08:54:42.474953+0000","last_clean":"2024-04-25T08:54:42.474953+0000","last_became_active":"2024-04-25T08:54:42.473614+0000","last_became_peered":"2024-04-25T08:54:42.473614+0000","last_unstale":"2024-04-25T08:54:42.474953+0000","last_undegraded":"2024-04-25T08:54:42.474953+0000","last_fullsized":"2024-04-25T08:54:42.474953+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.426642+0000","last_change":"2024-04-25T08:54:42.426642+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.426642+0000","last_undegraded":"2024-04-25T08:54:42.426642+0000","last_fullsized":"2024-04-25T08:54:42.426642+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.470808+0000","last_change":"2024-04-25T08:54:42.461363+0000","last_active":"2024-04-25T08:54:42.470808+0000","last_peered":"2024-04-25T08:54:42.470808+0000","last_clean":"2024-04-25T08:54:42.470808+0000","last_became_active":"2024-04-25T08:54:42.460631+0000","last_became_peered":"2024-04-25T08:54:42.460631+0000","last_unstale":"2024-04-25T08:54:42.470808+0000","last_undegraded":"2024-04-25T08:54:42.470808+0000","last_fullsized":"2024-04-25T08:54:42.470808+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:51.318624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.433752+0000","last_change":"2024-04-25T08:54:42.433752+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.433752+0000","last_undegraded":"2024-04-25T08:54:42.433752+0000","last_fullsized":"2024-04-25T08:54:42.433752+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.465848+0000","last_change":"2024-04-25T08:54:42.456578+0000","last_active":"2024-04-25T08:54:42.465848+0000","last_peered":"2024-04-25T08:54:42.465848+0000","last_clean":"2024-04-25T08:54:42.465848+0000","last_became_active":"2024-04-25T08:54:42.456167+0000","last_became_peered":"2024-04-25T08:54:42.456167+0000","last_unstale":"2024-04-25T08:54:42.465848+0000","last_undegraded":"2024-04-25T08:54:42.465848+0000","last_fullsized":"2024-04-25T08:54:42.465848+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442253+0000","last_change":"2024-04-25T08:54:42.442253+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442253+0000","last_undegraded":"2024-04-25T08:54:42.442253+0000","last_fullsized":"2024-04-25T08:54:42.442253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456810+0000","last_change":"2024-04-25T08:54:42.451116+0000","last_active":"2024-04-25T08:54:42.456810+0000","last_peered":"2024-04-25T08:54:42.456810+0000","last_clean":"2024-04-25T08:54:42.456810+0000","last_became_active":"2024-04-25T08:54:42.447149+0000","last_became_peered":"2024-04-25T08:54:42.447149+0000","last_unstale":"2024-04-25T08:54:42.456810+0000","last_undegraded":"2024-04-25T08:54:42.456810+0000","last_fullsized":"2024-04-25T08:54:42.456810+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:35.144794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447349+0000","last_change":"2024-04-25T08:54:42.447349+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447349+0000","last_undegraded":"2024-04-25T08:54:42.447349+0000","last_fullsized":"2024-04-25T08:54:42.447349+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.460990+0000","last_change":"2024-04-25T08:54:42.458531+0000","last_active":"2024-04-25T08:54:42.460990+0000","last_peered":"2024-04-25T08:54:42.460990+0000","last_clean":"2024-04-25T08:54:42.460990+0000","last_became_active":"2024-04-25T08:54:42.457748+0000","last_became_peered":"2024-04-25T08:54:42.457748+0000","last_unstale":"2024-04-25T08:54:42.460990+0000","last_undegraded":"2024-04-25T08:54:42.460990+0000","last_fullsized":"2024-04-25T08:54:42.460990+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446203+0000","last_change":"2024-04-25T08:54:42.446203+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446203+0000","last_undegraded":"2024-04-25T08:54:42.446203+0000","last_fullsized":"2024-04-25T08:54:42.446203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.473524+0000","last_change":"2024-04-25T08:54:42.466236+0000","last_active":"2024-04-25T08:54:42.473524+0000","last_peered":"2024-04-25T08:54:42.473524+0000","last_clean":"2024-04-25T08:54:42.473524+0000","last_became_active":"2024-04-25T08:54:42.465507+0000","last_became_peered":"2024-04-25T08:54:42.465507+0000","last_unstale":"2024-04-25T08:54:42.473524+0000","last_undegraded":"2024-04-25T08:54:42.473524+0000","last_fullsized":"2024-04-25T08:54:42.473524+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:22.360154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439111+0000","last_change":"2024-04-25T08:54:42.439111+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439111+0000","last_undegraded":"2024-04-25T08:54:42.439111+0000","last_fullsized":"2024-04-25T08:54:42.439111+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.473955+0000","last_change":"2024-04-25T08:54:42.469518+0000","last_active":"2024-04-25T08:54:42.473955+0000","last_peered":"2024-04-25T08:54:42.473955+0000","last_clean":"2024-04-25T08:54:42.473955+0000","last_became_active":"2024-04-25T08:54:42.469147+0000","last_became_peered":"2024-04-25T08:54:42.469147+0000","last_unstale":"2024-04-25T08:54:42.473955+0000","last_undegraded":"2024-04-25T08:54:42.473955+0000","last_fullsized":"2024-04-25T08:54:42.473955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:44.662568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.437803+0000","last_change":"2024-04-25T08:54:42.437803+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.437803+0000","last_undegraded":"2024-04-25T08:54:42.437803+0000","last_fullsized":"2024-04-25T08:54:42.437803+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453781+0000","last_change":"2024-04-25T08:54:42.450583+0000","last_active":"2024-04-25T08:54:42.453781+0000","last_peered":"2024-04-25T08:54:42.453781+0000","last_clean":"2024-04-25T08:54:42.453781+0000","last_became_active":"2024-04-25T08:54:42.450316+0000","last_became_peered":"2024-04-25T08:54:42.450316+0000","last_unstale":"2024-04-25T08:54:42.453781+0000","last_undegraded":"2024-04-25T08:54:42.453781+0000","last_fullsized":"2024-04-25T08:54:42.453781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:41.112056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.428105+0000","last_change":"2024-04-25T08:54:42.428105+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.428105+0000","last_undegraded":"2024-04-25T08:54:42.428105+0000","last_fullsized":"2024-04-25T08:54:42.428105+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.457250+0000","last_change":"2024-04-25T08:54:42.444423+0000","last_active":"2024-04-25T08:54:42.457250+0000","last_peered":"2024-04-25T08:54:42.457250+0000","last_clean":"2024-04-25T08:54:42.457250+0000","last_became_active":"2024-04-25T08:54:42.439289+0000","last_became_peered":"2024-04-25T08:54:42.439289+0000","last_unstale":"2024-04-25T08:54:42.457250+0000","last_undegraded":"2024-04-25T08:54:42.457250+0000","last_fullsized":"2024-04-25T08:54:42.457250+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:40.820232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439073+0000","last_change":"2024-04-25T08:54:42.439073+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439073+0000","last_undegraded":"2024-04-25T08:54:42.439073+0000","last_fullsized":"2024-04-25T08:54:42.439073+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.454629+0000","last_change":"2024-04-25T08:54:42.436282+0000","last_active":"2024-04-25T08:54:42.454629+0000","last_peered":"2024-04-25T08:54:42.454629+0000","last_clean":"2024-04-25T08:54:42.454629+0000","last_became_active":"2024-04-25T08:54:42.435494+0000","last_became_peered":"2024-04-25T08:54:42.435494+0000","last_unstale":"2024-04-25T08:54:42.454629+0000","last_undegraded":"2024-04-25T08:54:42.454629+0000","last_fullsized":"2024-04-25T08:54:42.454629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:13.276684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.432676+0000","last_change":"2024-04-25T08:54:42.432676+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.432676+0000","last_undegraded":"2024-04-25T08:54:42.432676+0000","last_fullsized":"2024-04-25T08:54:42.432676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.476553+0000","last_change":"2024-04-25T08:54:42.456188+0000","last_active":"2024-04-25T08:54:42.476553+0000","last_peered":"2024-04-25T08:54:42.476553+0000","last_clean":"2024-04-25T08:54:42.476553+0000","last_became_active":"2024-04-25T08:54:42.455422+0000","last_became_peered":"2024-04-25T08:54:42.455422+0000","last_unstale":"2024-04-25T08:54:42.476553+0000","last_undegraded":"2024-04-25T08:54:42.476553+0000","last_fullsized":"2024-04-25T08:54:42.476553+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.427057+0000","last_change":"2024-04-25T08:54:42.427057+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.427057+0000","last_undegraded":"2024-04-25T08:54:42.427057+0000","last_fullsized":"2024-04-25T08:54:42.427057+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471615+0000","last_change":"2024-04-25T08:54:42.463454+0000","last_active":"2024-04-25T08:54:42.471615+0000","last_peered":"2024-04-25T08:54:42.471615+0000","last_clean":"2024-04-25T08:54:42.471615+0000","last_became_active":"2024-04-25T08:54:42.461962+0000","last_became_peered":"2024-04-25T08:54:42.461962+0000","last_unstale":"2024-04-25T08:54:42.471615+0000","last_undegraded":"2024-04-25T08:54:42.471615+0000","last_fullsized":"2024-04-25T08:54:42.471615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:08.285003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.441892+0000","last_change":"2024-04-25T08:54:42.441892+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.441892+0000","last_undegraded":"2024-04-25T08:54:42.441892+0000","last_fullsized":"2024-04-25T08:54:42.441892+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.465722+0000","last_change":"2024-04-25T08:54:42.456453+0000","last_active":"2024-04-25T08:54:42.465722+0000","last_peered":"2024-04-25T08:54:42.465722+0000","last_clean":"2024-04-25T08:54:42.465722+0000","last_became_active":"2024-04-25T08:54:42.456124+0000","last_became_peered":"2024-04-25T08:54:42.456124+0000","last_unstale":"2024-04-25T08:54:42.465722+0000","last_undegraded":"2024-04-25T08:54:42.465722+0000","last_fullsized":"2024-04-25T08:54:42.465722+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439639+0000","last_change":"2024-04-25T08:54:42.439639+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439639+0000","last_undegraded":"2024-04-25T08:54:42.439639+0000","last_fullsized":"2024-04-25T08:54:42.439639+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453850+0000","last_change":"2024-04-25T08:54:42.448496+0000","last_active":"2024-04-25T08:54:42.453850+0000","last_peered":"2024-04-25T08:54:42.453850+0000","last_clean":"2024-04-25T08:54:42.453850+0000","last_became_active":"2024-04-25T08:54:42.447727+0000","last_became_peered":"2024-04-25T08:54:42.447727+0000","last_unstale":"2024-04-25T08:54:42.453850+0000","last_undegraded":"2024-04-25T08:54:42.453850+0000","last_fullsized":"2024-04-25T08:54:42.453850+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:30.126539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431858+0000","last_change":"2024-04-25T08:54:42.431858+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431858+0000","last_undegraded":"2024-04-25T08:54:42.431858+0000","last_fullsized":"2024-04-25T08:54:42.431858+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456883+0000","last_change":"2024-04-25T08:54:42.449480+0000","last_active":"2024-04-25T08:54:42.456883+0000","last_peered":"2024-04-25T08:54:42.456883+0000","last_clean":"2024-04-25T08:54:42.456883+0000","last_became_active":"2024-04-25T08:54:42.448749+0000","last_became_peered":"2024-04-25T08:54:42.448749+0000","last_unstale":"2024-04-25T08:54:42.456883+0000","last_undegraded":"2024-04-25T08:54:42.456883+0000","last_fullsized":"2024-04-25T08:54:42.456883+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:19.814248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439551+0000","last_change":"2024-04-25T08:54:42.439551+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439551+0000","last_undegraded":"2024-04-25T08:54:42.439551+0000","last_fullsized":"2024-04-25T08:54:42.439551+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.459614+0000","last_change":"2024-04-25T08:54:42.455018+0000","last_active":"2024-04-25T08:54:42.459614+0000","last_peered":"2024-04-25T08:54:42.459614+0000","last_clean":"2024-04-25T08:54:42.459614+0000","last_became_active":"2024-04-25T08:54:42.454140+0000","last_became_peered":"2024-04-25T08:54:42.454140+0000","last_unstale":"2024-04-25T08:54:42.459614+0000","last_undegraded":"2024-04-25T08:54:42.459614+0000","last_fullsized":"2024-04-25T08:54:42.459614+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:28.707217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.438653+0000","last_change":"2024-04-25T08:54:42.438653+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.438653+0000","last_undegraded":"2024-04-25T08:54:42.438653+0000","last_fullsized":"2024-04-25T08:54:42.438653+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.466103+0000","last_change":"2024-04-25T08:54:42.464859+0000","last_active":"2024-04-25T08:54:42.466103+0000","last_peered":"2024-04-25T08:54:42.466103+0000","last_clean":"2024-04-25T08:54:42.466103+0000","last_became_active":"2024-04-25T08:54:42.457143+0000","last_became_peered":"2024-04-25T08:54:42.457143+0000","last_unstale":"2024-04-25T08:54:42.466103+0000","last_undegraded":"2024-04-25T08:54:42.466103+0000","last_fullsized":"2024-04-25T08:54:42.466103+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444706+0000","last_change":"2024-04-25T08:54:42.444706+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444706+0000","last_undegraded":"2024-04-25T08:54:42.444706+0000","last_fullsized":"2024-04-25T08:54:42.444706+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471541+0000","last_change":"2024-04-25T08:54:42.463152+0000","last_active":"2024-04-25T08:54:42.471541+0000","last_peered":"2024-04-25T08:54:42.471541+0000","last_clean":"2024-04-25T08:54:42.471541+0000","last_became_active":"2024-04-25T08:54:42.462413+0000","last_became_peered":"2024-04-25T08:54:42.462413+0000","last_unstale":"2024-04-25T08:54:42.471541+0000","last_undegraded":"2024-04-25T08:54:42.471541+0000","last_fullsized":"2024-04-25T08:54:42.471541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:15.835374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431106+0000","last_change":"2024-04-25T08:54:42.431106+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431106+0000","last_undegraded":"2024-04-25T08:54:42.431106+0000","last_fullsized":"2024-04-25T08:54:42.431106+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471662+0000","last_change":"2024-04-25T08:54:42.463691+0000","last_active":"2024-04-25T08:54:42.471662+0000","last_peered":"2024-04-25T08:54:42.471662+0000","last_clean":"2024-04-25T08:54:42.471662+0000","last_became_active":"2024-04-25T08:54:42.462922+0000","last_became_peered":"2024-04-25T08:54:42.462922+0000","last_unstale":"2024-04-25T08:54:42.471662+0000","last_undegraded":"2024-04-25T08:54:42.471662+0000","last_fullsized":"2024-04-25T08:54:42.471662+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:18.405142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442356+0000","last_change":"2024-04-25T08:54:42.442356+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442356+0000","last_undegraded":"2024-04-25T08:54:42.442356+0000","last_fullsized":"2024-04-25T08:54:42.442356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.452975+0000","last_change":"2024-04-25T08:54:42.441517+0000","last_active":"2024-04-25T08:54:42.452975+0000","last_peered":"2024-04-25T08:54:42.452975+0000","last_clean":"2024-04-25T08:54:42.452975+0000","last_became_active":"2024-04-25T08:54:42.439663+0000","last_became_peered":"2024-04-25T08:54:42.439663+0000","last_unstale":"2024-04-25T08:54:42.452975+0000","last_undegraded":"2024-04-25T08:54:42.452975+0000","last_fullsized":"2024-04-25T08:54:42.452975+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439988+0000","last_change":"2024-04-25T08:54:42.439988+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439988+0000","last_undegraded":"2024-04-25T08:54:42.439988+0000","last_fullsized":"2024-04-25T08:54:42.439988+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.459448+0000","last_change":"2024-04-25T08:54:42.456403+0000","last_active":"2024-04-25T08:54:42.459448+0000","last_peered":"2024-04-25T08:54:42.459448+0000","last_clean":"2024-04-25T08:54:42.459448+0000","last_became_active":"2024-04-25T08:54:42.455569+0000","last_became_peered":"2024-04-25T08:54:42.455569+0000","last_unstale":"2024-04-25T08:54:42.459448+0000","last_undegraded":"2024-04-25T08:54:42.459448+0000","last_fullsized":"2024-04-25T08:54:42.459448+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:44.370399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.423932+0000","last_change":"2024-04-25T08:54:42.423932+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.423932+0000","last_undegraded":"2024-04-25T08:54:42.423932+0000","last_fullsized":"2024-04-25T08:54:42.423932+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471131+0000","last_change":"2024-04-25T08:54:42.466348+0000","last_active":"2024-04-25T08:54:42.471131+0000","last_peered":"2024-04-25T08:54:42.471131+0000","last_clean":"2024-04-25T08:54:42.471131+0000","last_became_active":"2024-04-25T08:54:42.456331+0000","last_became_peered":"2024-04-25T08:54:42.456331+0000","last_unstale":"2024-04-25T08:54:42.471131+0000","last_undegraded":"2024-04-25T08:54:42.471131+0000","last_fullsized":"2024-04-25T08:54:42.471131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:34.520291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":231,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.445808+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:54:42.445808+0000","last_peered":"2024-04-25T08:54:42.445808+0000","last_clean":"2024-04-25T08:54:42.445808+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:54:42.445808+0000","last_undegraded":"2024-04-25T08:54:42.445808+0000","last_fullsized":"2024-04-25T08:54:42.445808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1224704,"data_stored":871076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31796,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1224704,"data_stored":871076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:54:47.893 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:54:48.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.055+0000 7f812865f640 1 Processor -- start 2024-04-25T08:54:48.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.055+0000 7f812865f640 1 -- start start 2024-04-25T08:54:48.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.055+0000 7f812865f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8120071740 0x7f81200acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.055+0000 7f812865f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f81200b24a0 0x7f81200ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.055+0000 7f812865f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81200ae860 con 0x7f812015da10 2024-04-25T08:54:48.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.055+0000 7f812865f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812015e210 con 0x7f8120071740 2024-04-25T08:54:48.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.055+0000 7f812865f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f812015e4f0 con 0x7f81200b24a0 2024-04-25T08:54:48.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.055+0000 7f8125bfa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8120071740 0x7f81200acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f8125bfa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8120071740 0x7f81200acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55430/0 (socket says 172.21.15.37:55430) 2024-04-25T08:54:48.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f8125bfa640 1 -- 172.21.15.37:0/2381239884 learned_addr learned my addr 172.21.15.37:0/2381239884 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:48.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f81263fb640 1 --1- 172.21.15.37:0/2381239884 >> v1:172.21.15.37:6789/0 conn(0x7f812015da10 0x7f8120071030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:39282/0 (socket says 172.21.15.37:39282) 2024-04-25T08:54:48.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f8126bfc640 1 --2- 172.21.15.37:0/2381239884 >> v2:172.21.15.203:3300/0 conn(0x7f81200b24a0 0x7f81200ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f8125bfa640 1 -- 172.21.15.37:0/2381239884 >> v2:172.21.15.203:3300/0 conn(0x7f81200b24a0 msgr2=0x7f81200ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f8125bfa640 1 --2- 172.21.15.37:0/2381239884 >> v2:172.21.15.203:3300/0 conn(0x7f81200b24a0 0x7f81200ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f8125bfa640 1 -- 172.21.15.37:0/2381239884 >> v1:172.21.15.37:6789/0 conn(0x7f812015da10 legacy=0x7f8120071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f81253f9640 1 -- 172.21.15.37:0/2381239884 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2108747676 0 0) 0x7f81200ae860 con 0x7f812015da10 2024-04-25T08:54:48.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f8125bfa640 1 -- 172.21.15.37:0/2381239884 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81200ae4c0 con 0x7f8120071740 2024-04-25T08:54:48.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f8125bfa640 1 --2- 172.21.15.37:0/2381239884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8120071740 0x7f81200acbc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f812006cf80 tx=0x7f811001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245655e12ee79e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f81253f9640 1 -- 172.21.15.37:0/2381239884 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81100b0070 con 0x7f8120071740 2024-04-25T08:54:48.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f81253f9640 1 -- 172.21.15.37:0/2381239884 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81100ab600 con 0x7f8120071740 2024-04-25T08:54:48.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f81253f9640 1 -- 172.21.15.37:0/2381239884 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81100ab9d0 con 0x7f8120071740 2024-04-25T08:54:48.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f812865f640 1 -- 172.21.15.37:0/2381239884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8120071740 msgr2=0x7f81200acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f812865f640 1 --2- 172.21.15.37:0/2381239884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8120071740 0x7f81200acbc0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f812006cf80 tx=0x7f811001c400 comp rx=0 tx=0).stop 2024-04-25T08:54:48.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f812865f640 1 -- 172.21.15.37:0/2381239884 shutdown_connections 2024-04-25T08:54:48.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f812865f640 1 --2- 172.21.15.37:0/2381239884 >> v2:172.21.15.203:3300/0 conn(0x7f81200b24a0 0x7f81200ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f812865f640 1 --2- 172.21.15.37:0/2381239884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8120071740 0x7f81200acbc0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f812865f640 1 -- 172.21.15.37:0/2381239884 >> 172.21.15.37:0/2381239884 conn(0x7f81200b11a0 msgr2=0x7f81200b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f812865f640 1 -- 172.21.15.37:0/2381239884 shutdown_connections 2024-04-25T08:54:48.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.059+0000 7f812865f640 1 -- 172.21.15.37:0/2381239884 wait complete. 2024-04-25T08:54:48.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f812865f640 1 Processor -- start 2024-04-25T08:54:48.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f812865f640 1 -- start start 2024-04-25T08:54:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f812865f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8120071740 0x7f8120152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f812865f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81200b24a0 0x7f8120156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f812865f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812015da10 0x7f8120146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f812865f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f812015ef00 con 0x7f812015da10 2024-04-25T08:54:48.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f812865f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81200adc60 con 0x7f81200b24a0 2024-04-25T08:54:48.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f812865f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812016b0a0 con 0x7f8120071740 2024-04-25T08:54:48.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8126bfc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812015da10 0x7f8120146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8126bfc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812015da10 0x7f8120146210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55436/0 (socket says 172.21.15.37:55436) 2024-04-25T08:54:48.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8126bfc640 1 -- 172.21.15.37:0/515077421 learned_addr learned my addr 172.21.15.37:0/515077421 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:48.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f81263fb640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8120071740 0x7f8120152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8125bfa640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81200b24a0 0x7f8120156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8126bfc640 1 -- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8120071740 msgr2=0x7f8120152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8126bfc640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8120071740 0x7f8120152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8126bfc640 1 -- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81200b24a0 msgr2=0x7f8120156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8126bfc640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81200b24a0 0x7f8120156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8126bfc640 1 -- 172.21.15.37:0/515077421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8120153100 con 0x7f812015da10 2024-04-25T08:54:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8125bfa640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81200b24a0 0x7f8120156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8126bfc640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812015da10 0x7f8120146210 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8118007e40 tx=0x7f811800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8116ffd640 1 -- 172.21.15.37:0/515077421 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118014620 con 0x7f812015da10 2024-04-25T08:54:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8116ffd640 1 -- 172.21.15.37:0/515077421 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8118019070 con 0x7f812015da10 2024-04-25T08:54:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f812865f640 1 -- 172.21.15.37:0/515077421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81201533e0 con 0x7f812015da10 2024-04-25T08:54:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f8116ffd640 1 -- 172.21.15.37:0/515077421 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8118014a20 con 0x7f812015da10 2024-04-25T08:54:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f812865f640 1 -- 172.21.15.37:0/515077421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812015b390 con 0x7f812015da10 2024-04-25T08:54:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.063+0000 7f812865f640 1 -- 172.21.15.37:0/515077421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f812004efa0 con 0x7f812015da10 2024-04-25T08:54:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.067+0000 7f8116ffd640 1 -- 172.21.15.37:0/515077421 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8118014bc0 con 0x7f812015da10 2024-04-25T08:54:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.067+0000 7f8116ffd640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8104066f60 0x7f8104069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.067+0000 7f8116ffd640 1 -- 172.21.15.37:0/515077421 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8118085a10 con 0x7f812015da10 2024-04-25T08:54:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.067+0000 7f81263fb640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8104066f60 0x7f8104069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.067+0000 7f81263fb640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8104066f60 0x7f8104069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f811c000f70 tx=0x7f811c007b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.071+0000 7f8116ffd640 1 -- 172.21.15.37:0/515077421 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8118056920 con 0x7f812015da10 2024-04-25T08:54:48.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.227+0000 7f812865f640 1 -- 172.21.15.37:0/515077421 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8120168660 con 0x7f8104066f60 2024-04-25T08:54:48.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.243+0000 7f8116ffd640 1 -- 172.21.15.37:0/515077421 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246641 (secure 0 0 0) 0x7f8120168660 con 0x7f8104066f60 2024-04-25T08:54:48.240 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:48.241 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 -- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8104066f60 msgr2=0x7f8104069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8104066f60 0x7f8104069420 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f811c000f70 tx=0x7f811c007b80 comp rx=0 tx=0).stop 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 -- 172.21.15.37:0/515077421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812015da10 msgr2=0x7f8120146210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812015da10 0x7f8120146210 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8118007e40 tx=0x7f811800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 -- 172.21.15.37:0/515077421 shutdown_connections 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8104066f60 0x7f8104069420 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f812015da10 0x7f8120146210 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f81200b24a0 0x7f8120156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 --2- 172.21.15.37:0/515077421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8120071740 0x7f8120152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 -- 172.21.15.37:0/515077421 >> 172.21.15.37:0/515077421 conn(0x7f81200b11a0 msgr2=0x7f81201509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 -- 172.21.15.37:0/515077421 shutdown_connections 2024-04-25T08:54:48.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.247+0000 7f812865f640 1 -- 172.21.15.37:0/515077421 wait complete. 2024-04-25T08:54:48.259 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-04-25T08:54:47.168024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":6088,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":6234112,"data_stored":3426400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.753404"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.450878+0000","last_change":"2024-04-25T08:54:42.450878+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.450878+0000","last_undegraded":"2024-04-25T08:54:42.450878+0000","last_fullsized":"2024-04-25T08:54:42.450878+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443054+0000","last_change":"2024-04-25T08:54:42.443054+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443054+0000","last_undegraded":"2024-04-25T08:54:42.443054+0000","last_fullsized":"2024-04-25T08:54:42.443054+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444301+0000","last_change":"2024-04-25T08:54:42.444301+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444301+0000","last_undegraded":"2024-04-25T08:54:42.444301+0000","last_fullsized":"2024-04-25T08:54:42.444301+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.434500+0000","last_change":"2024-04-25T08:54:42.434500+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.434500+0000","last_undegraded":"2024-04-25T08:54:42.434500+0000","last_fullsized":"2024-04-25T08:54:42.434500+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.427901+0000","last_change":"2024-04-25T08:54:42.427901+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.427901+0000","last_undegraded":"2024-04-25T08:54:42.427901+0000","last_fullsized":"2024-04-25T08:54:42.427901+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.449668+0000","last_change":"2024-04-25T08:54:42.449668+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.449668+0000","last_undegraded":"2024-04-25T08:54:42.449668+0000","last_fullsized":"2024-04-25T08:54:42.449668+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444216+0000","last_change":"2024-04-25T08:54:42.444216+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444216+0000","last_undegraded":"2024-04-25T08:54:42.444216+0000","last_fullsized":"2024-04-25T08:54:42.444216+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.445284+0000","last_change":"2024-04-25T08:54:42.445284+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.445284+0000","last_undegraded":"2024-04-25T08:54:42.445284+0000","last_fullsized":"2024-04-25T08:54:42.445284+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.445459+0000","last_change":"2024-04-25T08:54:42.445459+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.445459+0000","last_undegraded":"2024-04-25T08:54:42.445459+0000","last_fullsized":"2024-04-25T08:54:42.445459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.449723+0000","last_change":"2024-04-25T08:54:42.449723+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.449723+0000","last_undegraded":"2024-04-25T08:54:42.449723+0000","last_fullsized":"2024-04-25T08:54:42.449723+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.437742+0000","last_change":"2024-04-25T08:54:42.437742+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.437742+0000","last_undegraded":"2024-04-25T08:54:42.437742+0000","last_fullsized":"2024-04-25T08:54:42.437742+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446682+0000","last_change":"2024-04-25T08:54:42.446682+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446682+0000","last_undegraded":"2024-04-25T08:54:42.446682+0000","last_fullsized":"2024-04-25T08:54:42.446682+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.454164+0000","last_change":"2024-04-25T08:54:42.454164+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.454164+0000","last_undegraded":"2024-04-25T08:54:42.454164+0000","last_fullsized":"2024-04-25T08:54:42.454164+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447877+0000","last_change":"2024-04-25T08:54:42.447877+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447877+0000","last_undegraded":"2024-04-25T08:54:42.447877+0000","last_fullsized":"2024-04-25T08:54:42.447877+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436375+0000","last_change":"2024-04-25T08:54:42.436375+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436375+0000","last_undegraded":"2024-04-25T08:54:42.436375+0000","last_fullsized":"2024-04-25T08:54:42.436375+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.440722+0000","last_change":"2024-04-25T08:54:42.440722+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.440722+0000","last_undegraded":"2024-04-25T08:54:42.440722+0000","last_fullsized":"2024-04-25T08:54:42.440722+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.457634+0000","last_change":"2024-04-25T08:54:42.457634+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.457634+0000","last_undegraded":"2024-04-25T08:54:42.457634+0000","last_fullsized":"2024-04-25T08:54:42.457634+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435705+0000","last_change":"2024-04-25T08:54:42.435705+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435705+0000","last_undegraded":"2024-04-25T08:54:42.435705+0000","last_fullsized":"2024-04-25T08:54:42.435705+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443799+0000","last_change":"2024-04-25T08:54:42.443799+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443799+0000","last_undegraded":"2024-04-25T08:54:42.443799+0000","last_fullsized":"2024-04-25T08:54:42.443799+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.426676+0000","last_change":"2024-04-25T08:54:42.426676+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.426676+0000","last_undegraded":"2024-04-25T08:54:42.426676+0000","last_fullsized":"2024-04-25T08:54:42.426676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.424865+0000","last_change":"2024-04-25T08:54:42.424865+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.424865+0000","last_undegraded":"2024-04-25T08:54:42.424865+0000","last_fullsized":"2024-04-25T08:54:42.424865+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446467+0000","last_change":"2024-04-25T08:54:42.446467+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446467+0000","last_undegraded":"2024-04-25T08:54:42.446467+0000","last_fullsized":"2024-04-25T08:54:42.446467+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436604+0000","last_change":"2024-04-25T08:54:42.436604+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436604+0000","last_undegraded":"2024-04-25T08:54:42.436604+0000","last_fullsized":"2024-04-25T08:54:42.436604+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443913+0000","last_change":"2024-04-25T08:54:42.443913+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443913+0000","last_undegraded":"2024-04-25T08:54:42.443913+0000","last_fullsized":"2024-04-25T08:54:42.443913+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447326+0000","last_change":"2024-04-25T08:54:42.447326+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447326+0000","last_undegraded":"2024-04-25T08:54:42.447326+0000","last_fullsized":"2024-04-25T08:54:42.447326+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.448405+0000","last_change":"2024-04-25T08:54:42.448405+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.448405+0000","last_undegraded":"2024-04-25T08:54:42.448405+0000","last_fullsized":"2024-04-25T08:54:42.448405+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443630+0000","last_change":"2024-04-25T08:54:42.443630+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443630+0000","last_undegraded":"2024-04-25T08:54:42.443630+0000","last_fullsized":"2024-04-25T08:54:42.443630+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.425583+0000","last_change":"2024-04-25T08:54:42.425583+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.425583+0000","last_undegraded":"2024-04-25T08:54:42.425583+0000","last_fullsized":"2024-04-25T08:54:42.425583+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442282+0000","last_change":"2024-04-25T08:54:42.442282+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442282+0000","last_undegraded":"2024-04-25T08:54:42.442282+0000","last_fullsized":"2024-04-25T08:54:42.442282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.441022+0000","last_change":"2024-04-25T08:54:42.441022+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.441022+0000","last_undegraded":"2024-04-25T08:54:42.441022+0000","last_fullsized":"2024-04-25T08:54:42.441022+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442094+0000","last_change":"2024-04-25T08:54:42.442094+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442094+0000","last_undegraded":"2024-04-25T08:54:42.442094+0000","last_fullsized":"2024-04-25T08:54:42.442094+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.432912+0000","last_change":"2024-04-25T08:54:42.432912+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.432912+0000","last_undegraded":"2024-04-25T08:54:42.432912+0000","last_fullsized":"2024-04-25T08:54:42.432912+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435703+0000","last_change":"2024-04-25T08:54:42.435703+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435703+0000","last_undegraded":"2024-04-25T08:54:42.435703+0000","last_fullsized":"2024-04-25T08:54:42.435703+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.454729+0000","last_change":"2024-04-25T08:54:42.435894+0000","last_active":"2024-04-25T08:54:42.454729+0000","last_peered":"2024-04-25T08:54:42.454729+0000","last_clean":"2024-04-25T08:54:42.454729+0000","last_became_active":"2024-04-25T08:54:42.435139+0000","last_became_peered":"2024-04-25T08:54:42.435139+0000","last_unstale":"2024-04-25T08:54:42.454729+0000","last_undegraded":"2024-04-25T08:54:42.454729+0000","last_fullsized":"2024-04-25T08:54:42.454729+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:55.487618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431034+0000","last_change":"2024-04-25T08:54:42.431034+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431034+0000","last_undegraded":"2024-04-25T08:54:42.431034+0000","last_fullsized":"2024-04-25T08:54:42.431034+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.448887+0000","last_change":"2024-04-25T08:54:42.434433+0000","last_active":"2024-04-25T08:54:42.448887+0000","last_peered":"2024-04-25T08:54:42.448887+0000","last_clean":"2024-04-25T08:54:42.448887+0000","last_became_active":"2024-04-25T08:54:42.434128+0000","last_became_peered":"2024-04-25T08:54:42.434128+0000","last_unstale":"2024-04-25T08:54:42.448887+0000","last_undegraded":"2024-04-25T08:54:42.448887+0000","last_fullsized":"2024-04-25T08:54:42.448887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:28.707217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.423925+0000","last_change":"2024-04-25T08:54:42.423925+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.423925+0000","last_undegraded":"2024-04-25T08:54:42.423925+0000","last_fullsized":"2024-04-25T08:54:42.423925+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.476536+0000","last_change":"2024-04-25T08:54:42.455613+0000","last_active":"2024-04-25T08:54:42.476536+0000","last_peered":"2024-04-25T08:54:42.476536+0000","last_clean":"2024-04-25T08:54:42.476536+0000","last_became_active":"2024-04-25T08:54:42.450954+0000","last_became_peered":"2024-04-25T08:54:42.450954+0000","last_unstale":"2024-04-25T08:54:42.476536+0000","last_undegraded":"2024-04-25T08:54:42.476536+0000","last_fullsized":"2024-04-25T08:54:42.476536+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439956+0000","last_change":"2024-04-25T08:54:42.439956+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439956+0000","last_undegraded":"2024-04-25T08:54:42.439956+0000","last_fullsized":"2024-04-25T08:54:42.439956+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.464425+0000","last_change":"2024-04-25T08:54:42.461742+0000","last_active":"2024-04-25T08:54:42.464425+0000","last_peered":"2024-04-25T08:54:42.464425+0000","last_clean":"2024-04-25T08:54:42.464425+0000","last_became_active":"2024-04-25T08:54:42.460925+0000","last_became_peered":"2024-04-25T08:54:42.460925+0000","last_unstale":"2024-04-25T08:54:42.464425+0000","last_undegraded":"2024-04-25T08:54:42.464425+0000","last_fullsized":"2024-04-25T08:54:42.464425+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:25.002800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.429055+0000","last_change":"2024-04-25T08:54:42.429055+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.429055+0000","last_undegraded":"2024-04-25T08:54:42.429055+0000","last_fullsized":"2024-04-25T08:54:42.429055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.474003+0000","last_change":"2024-04-25T08:54:42.468263+0000","last_active":"2024-04-25T08:54:42.474003+0000","last_peered":"2024-04-25T08:54:42.474003+0000","last_clean":"2024-04-25T08:54:42.474003+0000","last_became_active":"2024-04-25T08:54:42.467914+0000","last_became_peered":"2024-04-25T08:54:42.467914+0000","last_unstale":"2024-04-25T08:54:42.474003+0000","last_undegraded":"2024-04-25T08:54:42.474003+0000","last_fullsized":"2024-04-25T08:54:42.474003+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:08.427686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439634+0000","last_change":"2024-04-25T08:54:42.439634+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439634+0000","last_undegraded":"2024-04-25T08:54:42.439634+0000","last_fullsized":"2024-04-25T08:54:42.439634+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.443413+0000","last_change":"2024-04-25T08:54:42.439471+0000","last_active":"2024-04-25T08:54:42.443413+0000","last_peered":"2024-04-25T08:54:42.443413+0000","last_clean":"2024-04-25T08:54:42.443413+0000","last_became_active":"2024-04-25T08:54:42.438730+0000","last_became_peered":"2024-04-25T08:54:42.438730+0000","last_unstale":"2024-04-25T08:54:42.443413+0000","last_undegraded":"2024-04-25T08:54:42.443413+0000","last_fullsized":"2024-04-25T08:54:42.443413+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.440613+0000","last_change":"2024-04-25T08:54:42.440613+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.440613+0000","last_undegraded":"2024-04-25T08:54:42.440613+0000","last_fullsized":"2024-04-25T08:54:42.440613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456706+0000","last_change":"2024-04-25T08:54:42.450407+0000","last_active":"2024-04-25T08:54:42.456706+0000","last_peered":"2024-04-25T08:54:42.456706+0000","last_clean":"2024-04-25T08:54:42.456706+0000","last_became_active":"2024-04-25T08:54:42.448040+0000","last_became_peered":"2024-04-25T08:54:42.448040+0000","last_unstale":"2024-04-25T08:54:42.456706+0000","last_undegraded":"2024-04-25T08:54:42.456706+0000","last_fullsized":"2024-04-25T08:54:42.456706+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:55.906854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436423+0000","last_change":"2024-04-25T08:54:42.436423+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436423+0000","last_undegraded":"2024-04-25T08:54:42.436423+0000","last_fullsized":"2024-04-25T08:54:42.436423+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471348+0000","last_change":"2024-04-25T08:54:42.466835+0000","last_active":"2024-04-25T08:54:42.471348+0000","last_peered":"2024-04-25T08:54:42.471348+0000","last_clean":"2024-04-25T08:54:42.471348+0000","last_became_active":"2024-04-25T08:54:42.466542+0000","last_became_peered":"2024-04-25T08:54:42.466542+0000","last_unstale":"2024-04-25T08:54:42.471348+0000","last_undegraded":"2024-04-25T08:54:42.471348+0000","last_fullsized":"2024-04-25T08:54:42.471348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:46.278167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.430403+0000","last_change":"2024-04-25T08:54:42.430403+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.430403+0000","last_undegraded":"2024-04-25T08:54:42.430403+0000","last_fullsized":"2024-04-25T08:54:42.430403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.461066+0000","last_change":"2024-04-25T08:54:42.458215+0000","last_active":"2024-04-25T08:54:42.461066+0000","last_peered":"2024-04-25T08:54:42.461066+0000","last_clean":"2024-04-25T08:54:42.461066+0000","last_became_active":"2024-04-25T08:54:42.457493+0000","last_became_peered":"2024-04-25T08:54:42.457493+0000","last_unstale":"2024-04-25T08:54:42.461066+0000","last_undegraded":"2024-04-25T08:54:42.461066+0000","last_fullsized":"2024-04-25T08:54:42.461066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435429+0000","last_change":"2024-04-25T08:54:42.435429+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435429+0000","last_undegraded":"2024-04-25T08:54:42.435429+0000","last_fullsized":"2024-04-25T08:54:42.435429+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453284+0000","last_change":"2024-04-25T08:54:42.442179+0000","last_active":"2024-04-25T08:54:42.453284+0000","last_peered":"2024-04-25T08:54:42.453284+0000","last_clean":"2024-04-25T08:54:42.453284+0000","last_became_active":"2024-04-25T08:54:42.441094+0000","last_became_peered":"2024-04-25T08:54:42.441094+0000","last_unstale":"2024-04-25T08:54:42.453284+0000","last_undegraded":"2024-04-25T08:54:42.453284+0000","last_fullsized":"2024-04-25T08:54:42.453284+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442670+0000","last_change":"2024-04-25T08:54:42.442670+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442670+0000","last_undegraded":"2024-04-25T08:54:42.442670+0000","last_fullsized":"2024-04-25T08:54:42.442670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.474953+0000","last_change":"2024-04-25T08:54:42.474076+0000","last_active":"2024-04-25T08:54:42.474953+0000","last_peered":"2024-04-25T08:54:42.474953+0000","last_clean":"2024-04-25T08:54:42.474953+0000","last_became_active":"2024-04-25T08:54:42.473614+0000","last_became_peered":"2024-04-25T08:54:42.473614+0000","last_unstale":"2024-04-25T08:54:42.474953+0000","last_undegraded":"2024-04-25T08:54:42.474953+0000","last_fullsized":"2024-04-25T08:54:42.474953+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.426642+0000","last_change":"2024-04-25T08:54:42.426642+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.426642+0000","last_undegraded":"2024-04-25T08:54:42.426642+0000","last_fullsized":"2024-04-25T08:54:42.426642+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.470808+0000","last_change":"2024-04-25T08:54:42.461363+0000","last_active":"2024-04-25T08:54:42.470808+0000","last_peered":"2024-04-25T08:54:42.470808+0000","last_clean":"2024-04-25T08:54:42.470808+0000","last_became_active":"2024-04-25T08:54:42.460631+0000","last_became_peered":"2024-04-25T08:54:42.460631+0000","last_unstale":"2024-04-25T08:54:42.470808+0000","last_undegraded":"2024-04-25T08:54:42.470808+0000","last_fullsized":"2024-04-25T08:54:42.470808+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:51.318624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.433752+0000","last_change":"2024-04-25T08:54:42.433752+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.433752+0000","last_undegraded":"2024-04-25T08:54:42.433752+0000","last_fullsized":"2024-04-25T08:54:42.433752+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.465848+0000","last_change":"2024-04-25T08:54:42.456578+0000","last_active":"2024-04-25T08:54:42.465848+0000","last_peered":"2024-04-25T08:54:42.465848+0000","last_clean":"2024-04-25T08:54:42.465848+0000","last_became_active":"2024-04-25T08:54:42.456167+0000","last_became_peered":"2024-04-25T08:54:42.456167+0000","last_unstale":"2024-04-25T08:54:42.465848+0000","last_undegraded":"2024-04-25T08:54:42.465848+0000","last_fullsized":"2024-04-25T08:54:42.465848+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442253+0000","last_change":"2024-04-25T08:54:42.442253+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442253+0000","last_undegraded":"2024-04-25T08:54:42.442253+0000","last_fullsized":"2024-04-25T08:54:42.442253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456810+0000","last_change":"2024-04-25T08:54:42.451116+0000","last_active":"2024-04-25T08:54:42.456810+0000","last_peered":"2024-04-25T08:54:42.456810+0000","last_clean":"2024-04-25T08:54:42.456810+0000","last_became_active":"2024-04-25T08:54:42.447149+0000","last_became_peered":"2024-04-25T08:54:42.447149+0000","last_unstale":"2024-04-25T08:54:42.456810+0000","last_undegraded":"2024-04-25T08:54:42.456810+0000","last_fullsized":"2024-04-25T08:54:42.456810+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:35.144794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447349+0000","last_change":"2024-04-25T08:54:42.447349+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447349+0000","last_undegraded":"2024-04-25T08:54:42.447349+0000","last_fullsized":"2024-04-25T08:54:42.447349+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.460990+0000","last_change":"2024-04-25T08:54:42.458531+0000","last_active":"2024-04-25T08:54:42.460990+0000","last_peered":"2024-04-25T08:54:42.460990+0000","last_clean":"2024-04-25T08:54:42.460990+0000","last_became_active":"2024-04-25T08:54:42.457748+0000","last_became_peered":"2024-04-25T08:54:42.457748+0000","last_unstale":"2024-04-25T08:54:42.460990+0000","last_undegraded":"2024-04-25T08:54:42.460990+0000","last_fullsized":"2024-04-25T08:54:42.460990+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446203+0000","last_change":"2024-04-25T08:54:42.446203+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446203+0000","last_undegraded":"2024-04-25T08:54:42.446203+0000","last_fullsized":"2024-04-25T08:54:42.446203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.473524+0000","last_change":"2024-04-25T08:54:42.466236+0000","last_active":"2024-04-25T08:54:42.473524+0000","last_peered":"2024-04-25T08:54:42.473524+0000","last_clean":"2024-04-25T08:54:42.473524+0000","last_became_active":"2024-04-25T08:54:42.465507+0000","last_became_peered":"2024-04-25T08:54:42.465507+0000","last_unstale":"2024-04-25T08:54:42.473524+0000","last_undegraded":"2024-04-25T08:54:42.473524+0000","last_fullsized":"2024-04-25T08:54:42.473524+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:22.360154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439111+0000","last_change":"2024-04-25T08:54:42.439111+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439111+0000","last_undegraded":"2024-04-25T08:54:42.439111+0000","last_fullsized":"2024-04-25T08:54:42.439111+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.473955+0000","last_change":"2024-04-25T08:54:42.469518+0000","last_active":"2024-04-25T08:54:42.473955+0000","last_peered":"2024-04-25T08:54:42.473955+0000","last_clean":"2024-04-25T08:54:42.473955+0000","last_became_active":"2024-04-25T08:54:42.469147+0000","last_became_peered":"2024-04-25T08:54:42.469147+0000","last_unstale":"2024-04-25T08:54:42.473955+0000","last_undegraded":"2024-04-25T08:54:42.473955+0000","last_fullsized":"2024-04-25T08:54:42.473955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:44.662568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.437803+0000","last_change":"2024-04-25T08:54:42.437803+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.437803+0000","last_undegraded":"2024-04-25T08:54:42.437803+0000","last_fullsized":"2024-04-25T08:54:42.437803+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453781+0000","last_change":"2024-04-25T08:54:42.450583+0000","last_active":"2024-04-25T08:54:42.453781+0000","last_peered":"2024-04-25T08:54:42.453781+0000","last_clean":"2024-04-25T08:54:42.453781+0000","last_became_active":"2024-04-25T08:54:42.450316+0000","last_became_peered":"2024-04-25T08:54:42.450316+0000","last_unstale":"2024-04-25T08:54:42.453781+0000","last_undegraded":"2024-04-25T08:54:42.453781+0000","last_fullsized":"2024-04-25T08:54:42.453781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:41.112056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.428105+0000","last_change":"2024-04-25T08:54:42.428105+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.428105+0000","last_undegraded":"2024-04-25T08:54:42.428105+0000","last_fullsized":"2024-04-25T08:54:42.428105+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.457250+0000","last_change":"2024-04-25T08:54:42.444423+0000","last_active":"2024-04-25T08:54:42.457250+0000","last_peered":"2024-04-25T08:54:42.457250+0000","last_clean":"2024-04-25T08:54:42.457250+0000","last_became_active":"2024-04-25T08:54:42.439289+0000","last_became_peered":"2024-04-25T08:54:42.439289+0000","last_unstale":"2024-04-25T08:54:42.457250+0000","last_undegraded":"2024-04-25T08:54:42.457250+0000","last_fullsized":"2024-04-25T08:54:42.457250+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:40.820232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439073+0000","last_change":"2024-04-25T08:54:42.439073+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439073+0000","last_undegraded":"2024-04-25T08:54:42.439073+0000","last_fullsized":"2024-04-25T08:54:42.439073+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.454629+0000","last_change":"2024-04-25T08:54:42.436282+0000","last_active":"2024-04-25T08:54:42.454629+0000","last_peered":"2024-04-25T08:54:42.454629+0000","last_clean":"2024-04-25T08:54:42.454629+0000","last_became_active":"2024-04-25T08:54:42.435494+0000","last_became_peered":"2024-04-25T08:54:42.435494+0000","last_unstale":"2024-04-25T08:54:42.454629+0000","last_undegraded":"2024-04-25T08:54:42.454629+0000","last_fullsized":"2024-04-25T08:54:42.454629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:13.276684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.432676+0000","last_change":"2024-04-25T08:54:42.432676+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.432676+0000","last_undegraded":"2024-04-25T08:54:42.432676+0000","last_fullsized":"2024-04-25T08:54:42.432676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.476553+0000","last_change":"2024-04-25T08:54:42.456188+0000","last_active":"2024-04-25T08:54:42.476553+0000","last_peered":"2024-04-25T08:54:42.476553+0000","last_clean":"2024-04-25T08:54:42.476553+0000","last_became_active":"2024-04-25T08:54:42.455422+0000","last_became_peered":"2024-04-25T08:54:42.455422+0000","last_unstale":"2024-04-25T08:54:42.476553+0000","last_undegraded":"2024-04-25T08:54:42.476553+0000","last_fullsized":"2024-04-25T08:54:42.476553+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.427057+0000","last_change":"2024-04-25T08:54:42.427057+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.427057+0000","last_undegraded":"2024-04-25T08:54:42.427057+0000","last_fullsized":"2024-04-25T08:54:42.427057+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471615+0000","last_change":"2024-04-25T08:54:42.463454+0000","last_active":"2024-04-25T08:54:42.471615+0000","last_peered":"2024-04-25T08:54:42.471615+0000","last_clean":"2024-04-25T08:54:42.471615+0000","last_became_active":"2024-04-25T08:54:42.461962+0000","last_became_peered":"2024-04-25T08:54:42.461962+0000","last_unstale":"2024-04-25T08:54:42.471615+0000","last_undegraded":"2024-04-25T08:54:42.471615+0000","last_fullsized":"2024-04-25T08:54:42.471615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:08.285003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.441892+0000","last_change":"2024-04-25T08:54:42.441892+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.441892+0000","last_undegraded":"2024-04-25T08:54:42.441892+0000","last_fullsized":"2024-04-25T08:54:42.441892+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.465722+0000","last_change":"2024-04-25T08:54:42.456453+0000","last_active":"2024-04-25T08:54:42.465722+0000","last_peered":"2024-04-25T08:54:42.465722+0000","last_clean":"2024-04-25T08:54:42.465722+0000","last_became_active":"2024-04-25T08:54:42.456124+0000","last_became_peered":"2024-04-25T08:54:42.456124+0000","last_unstale":"2024-04-25T08:54:42.465722+0000","last_undegraded":"2024-04-25T08:54:42.465722+0000","last_fullsized":"2024-04-25T08:54:42.465722+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439639+0000","last_change":"2024-04-25T08:54:42.439639+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439639+0000","last_undegraded":"2024-04-25T08:54:42.439639+0000","last_fullsized":"2024-04-25T08:54:42.439639+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453850+0000","last_change":"2024-04-25T08:54:42.448496+0000","last_active":"2024-04-25T08:54:42.453850+0000","last_peered":"2024-04-25T08:54:42.453850+0000","last_clean":"2024-04-25T08:54:42.453850+0000","last_became_active":"2024-04-25T08:54:42.447727+0000","last_became_peered":"2024-04-25T08:54:42.447727+0000","last_unstale":"2024-04-25T08:54:42.453850+0000","last_undegraded":"2024-04-25T08:54:42.453850+0000","last_fullsized":"2024-04-25T08:54:42.453850+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:30.126539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431858+0000","last_change":"2024-04-25T08:54:42.431858+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431858+0000","last_undegraded":"2024-04-25T08:54:42.431858+0000","last_fullsized":"2024-04-25T08:54:42.431858+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456883+0000","last_change":"2024-04-25T08:54:42.449480+0000","last_active":"2024-04-25T08:54:42.456883+0000","last_peered":"2024-04-25T08:54:42.456883+0000","last_clean":"2024-04-25T08:54:42.456883+0000","last_became_active":"2024-04-25T08:54:42.448749+0000","last_became_peered":"2024-04-25T08:54:42.448749+0000","last_unstale":"2024-04-25T08:54:42.456883+0000","last_undegraded":"2024-04-25T08:54:42.456883+0000","last_fullsized":"2024-04-25T08:54:42.456883+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:19.814248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439551+0000","last_change":"2024-04-25T08:54:42.439551+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439551+0000","last_undegraded":"2024-04-25T08:54:42.439551+0000","last_fullsized":"2024-04-25T08:54:42.439551+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.459614+0000","last_change":"2024-04-25T08:54:42.455018+0000","last_active":"2024-04-25T08:54:42.459614+0000","last_peered":"2024-04-25T08:54:42.459614+0000","last_clean":"2024-04-25T08:54:42.459614+0000","last_became_active":"2024-04-25T08:54:42.454140+0000","last_became_peered":"2024-04-25T08:54:42.454140+0000","last_unstale":"2024-04-25T08:54:42.459614+0000","last_undegraded":"2024-04-25T08:54:42.459614+0000","last_fullsized":"2024-04-25T08:54:42.459614+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:28.707217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.438653+0000","last_change":"2024-04-25T08:54:42.438653+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.438653+0000","last_undegraded":"2024-04-25T08:54:42.438653+0000","last_fullsized":"2024-04-25T08:54:42.438653+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.466103+0000","last_change":"2024-04-25T08:54:42.464859+0000","last_active":"2024-04-25T08:54:42.466103+0000","last_peered":"2024-04-25T08:54:42.466103+0000","last_clean":"2024-04-25T08:54:42.466103+0000","last_became_active":"2024-04-25T08:54:42.457143+0000","last_became_peered":"2024-04-25T08:54:42.457143+0000","last_unstale":"2024-04-25T08:54:42.466103+0000","last_undegraded":"2024-04-25T08:54:42.466103+0000","last_fullsized":"2024-04-25T08:54:42.466103+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444706+0000","last_change":"2024-04-25T08:54:42.444706+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444706+0000","last_undegraded":"2024-04-25T08:54:42.444706+0000","last_fullsized":"2024-04-25T08:54:42.444706+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471541+0000","last_change":"2024-04-25T08:54:42.463152+0000","last_active":"2024-04-25T08:54:42.471541+0000","last_peered":"2024-04-25T08:54:42.471541+0000","last_clean":"2024-04-25T08:54:42.471541+0000","last_became_active":"2024-04-25T08:54:42.462413+0000","last_became_peered":"2024-04-25T08:54:42.462413+0000","last_unstale":"2024-04-25T08:54:42.471541+0000","last_undegraded":"2024-04-25T08:54:42.471541+0000","last_fullsized":"2024-04-25T08:54:42.471541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:15.835374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431106+0000","last_change":"2024-04-25T08:54:42.431106+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431106+0000","last_undegraded":"2024-04-25T08:54:42.431106+0000","last_fullsized":"2024-04-25T08:54:42.431106+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471662+0000","last_change":"2024-04-25T08:54:42.463691+0000","last_active":"2024-04-25T08:54:42.471662+0000","last_peered":"2024-04-25T08:54:42.471662+0000","last_clean":"2024-04-25T08:54:42.471662+0000","last_became_active":"2024-04-25T08:54:42.462922+0000","last_became_peered":"2024-04-25T08:54:42.462922+0000","last_unstale":"2024-04-25T08:54:42.471662+0000","last_undegraded":"2024-04-25T08:54:42.471662+0000","last_fullsized":"2024-04-25T08:54:42.471662+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:18.405142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442356+0000","last_change":"2024-04-25T08:54:42.442356+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442356+0000","last_undegraded":"2024-04-25T08:54:42.442356+0000","last_fullsized":"2024-04-25T08:54:42.442356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.452975+0000","last_change":"2024-04-25T08:54:42.441517+0000","last_active":"2024-04-25T08:54:42.452975+0000","last_peered":"2024-04-25T08:54:42.452975+0000","last_clean":"2024-04-25T08:54:42.452975+0000","last_became_active":"2024-04-25T08:54:42.439663+0000","last_became_peered":"2024-04-25T08:54:42.439663+0000","last_unstale":"2024-04-25T08:54:42.452975+0000","last_undegraded":"2024-04-25T08:54:42.452975+0000","last_fullsized":"2024-04-25T08:54:42.452975+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439988+0000","last_change":"2024-04-25T08:54:42.439988+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439988+0000","last_undegraded":"2024-04-25T08:54:42.439988+0000","last_fullsized":"2024-04-25T08:54:42.439988+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.459448+0000","last_change":"2024-04-25T08:54:42.456403+0000","last_active":"2024-04-25T08:54:42.459448+0000","last_peered":"2024-04-25T08:54:42.459448+0000","last_clean":"2024-04-25T08:54:42.459448+0000","last_became_active":"2024-04-25T08:54:42.455569+0000","last_became_peered":"2024-04-25T08:54:42.455569+0000","last_unstale":"2024-04-25T08:54:42.459448+0000","last_undegraded":"2024-04-25T08:54:42.459448+0000","last_fullsized":"2024-04-25T08:54:42.459448+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:44.370399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.423932+0000","last_change":"2024-04-25T08:54:42.423932+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.423932+0000","last_undegraded":"2024-04-25T08:54:42.423932+0000","last_fullsized":"2024-04-25T08:54:42.423932+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471131+0000","last_change":"2024-04-25T08:54:42.466348+0000","last_active":"2024-04-25T08:54:42.471131+0000","last_peered":"2024-04-25T08:54:42.471131+0000","last_clean":"2024-04-25T08:54:42.471131+0000","last_became_active":"2024-04-25T08:54:42.456331+0000","last_became_peered":"2024-04-25T08:54:42.456331+0000","last_unstale":"2024-04-25T08:54:42.471131+0000","last_undegraded":"2024-04-25T08:54:42.471131+0000","last_fullsized":"2024-04-25T08:54:42.471131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:34.520291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":231,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.445808+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:54:42.445808+0000","last_peered":"2024-04-25T08:54:42.445808+0000","last_clean":"2024-04-25T08:54:42.445808+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:54:42.445808+0000","last_undegraded":"2024-04-25T08:54:42.445808+0000","last_fullsized":"2024-04-25T08:54:42.445808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1224704,"data_stored":871076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31796,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1224704,"data_stored":871076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:54:48.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3f is not active+clean 2024-04-25T08:54:48.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.450878+0000', 'last_change': '2024-04-25T08:54:42.450878+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.450878+0000', 'last_undegraded': '2024-04-25T08:54:42.450878+0000', 'last_fullsized': '2024-04-25T08:54:42.450878+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:54:48.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3e is not active+clean 2024-04-25T08:54:48.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.443054+0000', 'last_change': '2024-04-25T08:54:42.443054+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.443054+0000', 'last_undegraded': '2024-04-25T08:54:42.443054+0000', 'last_fullsized': '2024-04-25T08:54:42.443054+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:54:48.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3d is not active+clean 2024-04-25T08:54:48.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.444301+0000', 'last_change': '2024-04-25T08:54:42.444301+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.444301+0000', 'last_undegraded': '2024-04-25T08:54:42.444301+0000', 'last_fullsized': '2024-04-25T08:54:42.444301+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3c is not active+clean 2024-04-25T08:54:48.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.434500+0000', 'last_change': '2024-04-25T08:54:42.434500+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.434500+0000', 'last_undegraded': '2024-04-25T08:54:42.434500+0000', 'last_fullsized': '2024-04-25T08:54:42.434500+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:54:48.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3b is not active+clean 2024-04-25T08:54:48.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.427901+0000', 'last_change': '2024-04-25T08:54:42.427901+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.427901+0000', 'last_undegraded': '2024-04-25T08:54:42.427901+0000', 'last_fullsized': '2024-04-25T08:54:42.427901+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:54:48.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3a is not active+clean 2024-04-25T08:54:48.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.449668+0000', 'last_change': '2024-04-25T08:54:42.449668+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.449668+0000', 'last_undegraded': '2024-04-25T08:54:42.449668+0000', 'last_fullsized': '2024-04-25T08:54:42.449668+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:54:48.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.39 is not active+clean 2024-04-25T08:54:48.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.444216+0000', 'last_change': '2024-04-25T08:54:42.444216+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.444216+0000', 'last_undegraded': '2024-04-25T08:54:42.444216+0000', 'last_fullsized': '2024-04-25T08:54:42.444216+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.38 is not active+clean 2024-04-25T08:54:48.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.445284+0000', 'last_change': '2024-04-25T08:54:42.445284+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.445284+0000', 'last_undegraded': '2024-04-25T08:54:42.445284+0000', 'last_fullsized': '2024-04-25T08:54:42.445284+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.37 is not active+clean 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.445459+0000', 'last_change': '2024-04-25T08:54:42.445459+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.445459+0000', 'last_undegraded': '2024-04-25T08:54:42.445459+0000', 'last_fullsized': '2024-04-25T08:54:42.445459+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.36 is not active+clean 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.449723+0000', 'last_change': '2024-04-25T08:54:42.449723+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.449723+0000', 'last_undegraded': '2024-04-25T08:54:42.449723+0000', 'last_fullsized': '2024-04-25T08:54:42.449723+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.35 is not active+clean 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.437742+0000', 'last_change': '2024-04-25T08:54:42.437742+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.437742+0000', 'last_undegraded': '2024-04-25T08:54:42.437742+0000', 'last_fullsized': '2024-04-25T08:54:42.437742+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.34 is not active+clean 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.446682+0000', 'last_change': '2024-04-25T08:54:42.446682+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.446682+0000', 'last_undegraded': '2024-04-25T08:54:42.446682+0000', 'last_fullsized': '2024-04-25T08:54:42.446682+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.33 is not active+clean 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.454164+0000', 'last_change': '2024-04-25T08:54:42.454164+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.454164+0000', 'last_undegraded': '2024-04-25T08:54:42.454164+0000', 'last_fullsized': '2024-04-25T08:54:42.454164+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.32 is not active+clean 2024-04-25T08:54:48.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.447877+0000', 'last_change': '2024-04-25T08:54:42.447877+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.447877+0000', 'last_undegraded': '2024-04-25T08:54:42.447877+0000', 'last_fullsized': '2024-04-25T08:54:42.447877+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.31 is not active+clean 2024-04-25T08:54:48.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.436375+0000', 'last_change': '2024-04-25T08:54:42.436375+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.436375+0000', 'last_undegraded': '2024-04-25T08:54:42.436375+0000', 'last_fullsized': '2024-04-25T08:54:42.436375+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:54:48.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.30 is not active+clean 2024-04-25T08:54:48.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.440722+0000', 'last_change': '2024-04-25T08:54:42.440722+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.440722+0000', 'last_undegraded': '2024-04-25T08:54:42.440722+0000', 'last_fullsized': '2024-04-25T08:54:42.440722+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:54:48.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2f is not active+clean 2024-04-25T08:54:48.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.457634+0000', 'last_change': '2024-04-25T08:54:42.457634+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.457634+0000', 'last_undegraded': '2024-04-25T08:54:42.457634+0000', 'last_fullsized': '2024-04-25T08:54:42.457634+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2e is not active+clean 2024-04-25T08:54:48.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.435705+0000', 'last_change': '2024-04-25T08:54:42.435705+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.435705+0000', 'last_undegraded': '2024-04-25T08:54:42.435705+0000', 'last_fullsized': '2024-04-25T08:54:42.435705+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:54:48.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2d is not active+clean 2024-04-25T08:54:48.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.443799+0000', 'last_change': '2024-04-25T08:54:42.443799+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.443799+0000', 'last_undegraded': '2024-04-25T08:54:42.443799+0000', 'last_fullsized': '2024-04-25T08:54:42.443799+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2c is not active+clean 2024-04-25T08:54:48.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.426676+0000', 'last_change': '2024-04-25T08:54:42.426676+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.426676+0000', 'last_undegraded': '2024-04-25T08:54:42.426676+0000', 'last_fullsized': '2024-04-25T08:54:42.426676+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:54:48.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2b is not active+clean 2024-04-25T08:54:48.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.424865+0000', 'last_change': '2024-04-25T08:54:42.424865+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.424865+0000', 'last_undegraded': '2024-04-25T08:54:42.424865+0000', 'last_fullsized': '2024-04-25T08:54:42.424865+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:54:48.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2a is not active+clean 2024-04-25T08:54:48.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.446467+0000', 'last_change': '2024-04-25T08:54:42.446467+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.446467+0000', 'last_undegraded': '2024-04-25T08:54:42.446467+0000', 'last_fullsized': '2024-04-25T08:54:42.446467+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:54:48.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.29 is not active+clean 2024-04-25T08:54:48.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.436604+0000', 'last_change': '2024-04-25T08:54:42.436604+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.436604+0000', 'last_undegraded': '2024-04-25T08:54:42.436604+0000', 'last_fullsized': '2024-04-25T08:54:42.436604+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:54:48.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.28 is not active+clean 2024-04-25T08:54:48.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.443913+0000', 'last_change': '2024-04-25T08:54:42.443913+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.443913+0000', 'last_undegraded': '2024-04-25T08:54:42.443913+0000', 'last_fullsized': '2024-04-25T08:54:42.443913+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.27 is not active+clean 2024-04-25T08:54:48.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.447326+0000', 'last_change': '2024-04-25T08:54:42.447326+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.447326+0000', 'last_undegraded': '2024-04-25T08:54:42.447326+0000', 'last_fullsized': '2024-04-25T08:54:42.447326+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.26 is not active+clean 2024-04-25T08:54:48.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.448405+0000', 'last_change': '2024-04-25T08:54:42.448405+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.448405+0000', 'last_undegraded': '2024-04-25T08:54:42.448405+0000', 'last_fullsized': '2024-04-25T08:54:42.448405+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:54:48.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.25 is not active+clean 2024-04-25T08:54:48.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.443630+0000', 'last_change': '2024-04-25T08:54:42.443630+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.443630+0000', 'last_undegraded': '2024-04-25T08:54:42.443630+0000', 'last_fullsized': '2024-04-25T08:54:42.443630+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:54:48.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.24 is not active+clean 2024-04-25T08:54:48.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.425583+0000', 'last_change': '2024-04-25T08:54:42.425583+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.425583+0000', 'last_undegraded': '2024-04-25T08:54:42.425583+0000', 'last_fullsized': '2024-04-25T08:54:42.425583+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:54:48.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.23 is not active+clean 2024-04-25T08:54:48.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.442282+0000', 'last_change': '2024-04-25T08:54:42.442282+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.442282+0000', 'last_undegraded': '2024-04-25T08:54:42.442282+0000', 'last_fullsized': '2024-04-25T08:54:42.442282+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.22 is not active+clean 2024-04-25T08:54:48.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.441022+0000', 'last_change': '2024-04-25T08:54:42.441022+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.441022+0000', 'last_undegraded': '2024-04-25T08:54:42.441022+0000', 'last_fullsized': '2024-04-25T08:54:42.441022+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:54:48.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.21 is not active+clean 2024-04-25T08:54:48.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.442094+0000', 'last_change': '2024-04-25T08:54:42.442094+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.442094+0000', 'last_undegraded': '2024-04-25T08:54:42.442094+0000', 'last_fullsized': '2024-04-25T08:54:42.442094+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.20 is not active+clean 2024-04-25T08:54:48.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.432912+0000', 'last_change': '2024-04-25T08:54:42.432912+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.432912+0000', 'last_undegraded': '2024-04-25T08:54:42.432912+0000', 'last_fullsized': '2024-04-25T08:54:42.432912+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:54:48.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-04-25T08:54:48.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.435703+0000', 'last_change': '2024-04-25T08:54:42.435703+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.435703+0000', 'last_undegraded': '2024-04-25T08:54:42.435703+0000', 'last_fullsized': '2024-04-25T08:54:42.435703+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:54:48.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-04-25T08:54:48.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.431034+0000', 'last_change': '2024-04-25T08:54:42.431034+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.431034+0000', 'last_undegraded': '2024-04-25T08:54:42.431034+0000', 'last_fullsized': '2024-04-25T08:54:42.431034+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:54:48.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-04-25T08:54:48.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.423925+0000', 'last_change': '2024-04-25T08:54:42.423925+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.423925+0000', 'last_undegraded': '2024-04-25T08:54:42.423925+0000', 'last_fullsized': '2024-04-25T08:54:42.423925+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:54:48.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-25T08:54:48.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.439956+0000', 'last_change': '2024-04-25T08:54:42.439956+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.439956+0000', 'last_undegraded': '2024-04-25T08:54:42.439956+0000', 'last_fullsized': '2024-04-25T08:54:42.439956+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:54:48.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-04-25T08:54:48.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.429055+0000', 'last_change': '2024-04-25T08:54:42.429055+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.429055+0000', 'last_undegraded': '2024-04-25T08:54:42.429055+0000', 'last_fullsized': '2024-04-25T08:54:42.429055+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:54:48.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-04-25T08:54:48.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.439634+0000', 'last_change': '2024-04-25T08:54:42.439634+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.439634+0000', 'last_undegraded': '2024-04-25T08:54:42.439634+0000', 'last_fullsized': '2024-04-25T08:54:42.439634+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:54:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-25T08:54:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.440613+0000', 'last_change': '2024-04-25T08:54:42.440613+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.440613+0000', 'last_undegraded': '2024-04-25T08:54:42.440613+0000', 'last_fullsized': '2024-04-25T08:54:42.440613+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:54:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-04-25T08:54:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.436423+0000', 'last_change': '2024-04-25T08:54:42.436423+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.436423+0000', 'last_undegraded': '2024-04-25T08:54:42.436423+0000', 'last_fullsized': '2024-04-25T08:54:42.436423+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:54:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-04-25T08:54:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.430403+0000', 'last_change': '2024-04-25T08:54:42.430403+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.430403+0000', 'last_undegraded': '2024-04-25T08:54:42.430403+0000', 'last_fullsized': '2024-04-25T08:54:42.430403+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:54:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-04-25T08:54:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.435429+0000', 'last_change': '2024-04-25T08:54:42.435429+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.435429+0000', 'last_undegraded': '2024-04-25T08:54:42.435429+0000', 'last_fullsized': '2024-04-25T08:54:42.435429+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:54:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-04-25T08:54:48.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.442670+0000', 'last_change': '2024-04-25T08:54:42.442670+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.442670+0000', 'last_undegraded': '2024-04-25T08:54:42.442670+0000', 'last_fullsized': '2024-04-25T08:54:42.442670+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.426642+0000', 'last_change': '2024-04-25T08:54:42.426642+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.426642+0000', 'last_undegraded': '2024-04-25T08:54:42.426642+0000', 'last_fullsized': '2024-04-25T08:54:42.426642+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.433752+0000', 'last_change': '2024-04-25T08:54:42.433752+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.433752+0000', 'last_undegraded': '2024-04-25T08:54:42.433752+0000', 'last_fullsized': '2024-04-25T08:54:42.433752+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.442253+0000', 'last_change': '2024-04-25T08:54:42.442253+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.442253+0000', 'last_undegraded': '2024-04-25T08:54:42.442253+0000', 'last_fullsized': '2024-04-25T08:54:42.442253+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.447349+0000', 'last_change': '2024-04-25T08:54:42.447349+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.447349+0000', 'last_undegraded': '2024-04-25T08:54:42.447349+0000', 'last_fullsized': '2024-04-25T08:54:42.447349+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.446203+0000', 'last_change': '2024-04-25T08:54:42.446203+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.446203+0000', 'last_undegraded': '2024-04-25T08:54:42.446203+0000', 'last_fullsized': '2024-04-25T08:54:42.446203+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-25T08:54:48.275 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.439111+0000', 'last_change': '2024-04-25T08:54:42.439111+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.439111+0000', 'last_undegraded': '2024-04-25T08:54:42.439111+0000', 'last_fullsized': '2024-04-25T08:54:42.439111+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.437803+0000', 'last_change': '2024-04-25T08:54:42.437803+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.437803+0000', 'last_undegraded': '2024-04-25T08:54:42.437803+0000', 'last_fullsized': '2024-04-25T08:54:42.437803+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.428105+0000', 'last_change': '2024-04-25T08:54:42.428105+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.428105+0000', 'last_undegraded': '2024-04-25T08:54:42.428105+0000', 'last_fullsized': '2024-04-25T08:54:42.428105+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.439073+0000', 'last_change': '2024-04-25T08:54:42.439073+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.439073+0000', 'last_undegraded': '2024-04-25T08:54:42.439073+0000', 'last_fullsized': '2024-04-25T08:54:42.439073+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.432676+0000', 'last_change': '2024-04-25T08:54:42.432676+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.432676+0000', 'last_undegraded': '2024-04-25T08:54:42.432676+0000', 'last_fullsized': '2024-04-25T08:54:42.432676+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.427057+0000', 'last_change': '2024-04-25T08:54:42.427057+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.427057+0000', 'last_undegraded': '2024-04-25T08:54:42.427057+0000', 'last_fullsized': '2024-04-25T08:54:42.427057+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-04-25T08:54:48.276 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.441892+0000', 'last_change': '2024-04-25T08:54:42.441892+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.441892+0000', 'last_undegraded': '2024-04-25T08:54:42.441892+0000', 'last_fullsized': '2024-04-25T08:54:42.441892+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.439639+0000', 'last_change': '2024-04-25T08:54:42.439639+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.439639+0000', 'last_undegraded': '2024-04-25T08:54:42.439639+0000', 'last_fullsized': '2024-04-25T08:54:42.439639+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.431858+0000', 'last_change': '2024-04-25T08:54:42.431858+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.431858+0000', 'last_undegraded': '2024-04-25T08:54:42.431858+0000', 'last_fullsized': '2024-04-25T08:54:42.431858+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.439551+0000', 'last_change': '2024-04-25T08:54:42.439551+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.439551+0000', 'last_undegraded': '2024-04-25T08:54:42.439551+0000', 'last_fullsized': '2024-04-25T08:54:42.439551+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.438653+0000', 'last_change': '2024-04-25T08:54:42.438653+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.438653+0000', 'last_undegraded': '2024-04-25T08:54:42.438653+0000', 'last_fullsized': '2024-04-25T08:54:42.438653+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.444706+0000', 'last_change': '2024-04-25T08:54:42.444706+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.444706+0000', 'last_undegraded': '2024-04-25T08:54:42.444706+0000', 'last_fullsized': '2024-04-25T08:54:42.444706+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-04-25T08:54:48.277 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.431106+0000', 'last_change': '2024-04-25T08:54:42.431106+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.431106+0000', 'last_undegraded': '2024-04-25T08:54:42.431106+0000', 'last_fullsized': '2024-04-25T08:54:42.431106+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:54:48.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-04-25T08:54:48.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.442356+0000', 'last_change': '2024-04-25T08:54:42.442356+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.442356+0000', 'last_undegraded': '2024-04-25T08:54:42.442356+0000', 'last_fullsized': '2024-04-25T08:54:42.442356+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:54:48.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-04-25T08:54:48.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.439988+0000', 'last_change': '2024-04-25T08:54:42.439988+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.439988+0000', 'last_undegraded': '2024-04-25T08:54:42.439988+0000', 'last_fullsized': '2024-04-25T08:54:42.439988+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:54:48.278 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-04-25T08:54:48.278 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 53, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:54:42.423932+0000', 'last_change': '2024-04-25T08:54:42.423932+0000', 'last_active': '2024-04-25T08:54:42.412663+0000', 'last_peered': '2024-04-25T08:54:42.412663+0000', 'last_clean': '2024-04-25T08:54:42.412663+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:54:42.423932+0000', 'last_undegraded': '2024-04-25T08:54:42.423932+0000', 'last_fullsized': '2024-04-25T08:54:42.423932+0000', 'mapping_epoch': 53, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 53, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'last_clean_scrub_stamp': '2024-04-25T08:54:42.412663+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:54:48.279 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:54:48.396 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T08:54:48.403+0000 7f838a683640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:54:48.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.439+0000 7fd85332e640 1 Processor -- start 2024-04-25T08:54:48.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.439+0000 7fd85332e640 1 -- start start 2024-04-25T08:54:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.439+0000 7fd85332e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd84c161160 0x7fd84c0d00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.439+0000 7fd85332e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd84c0d0600 0x7fd84c0cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.439+0000 7fd85332e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84c0d1d40 con 0x7fd84c0b2780 2024-04-25T08:54:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.439+0000 7fd85332e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd84c0d0a30 con 0x7fd84c161160 2024-04-25T08:54:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.439+0000 7fd85332e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd84c0d2020 con 0x7fd84c0d0600 2024-04-25T08:54:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.439+0000 7fd8510ca640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd84c161160 0x7fd84c0d00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.439+0000 7fd8508c9640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd84c0b2780 0x7fd84c0b3470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:39294/0 (socket says 172.21.15.37:39294) 2024-04-25T08:54:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.439+0000 7fd8508c9640 1 -- 172.21.15.37:0/3625855195 learned_addr learned my addr 172.21.15.37:0/3625855195 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd8518cb640 1 --2- 172.21.15.37:0/3625855195 >> v2:172.21.15.203:3300/0 conn(0x7fd84c0d0600 0x7fd84c0cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd8510ca640 1 -- 172.21.15.37:0/3625855195 >> v2:172.21.15.203:3300/0 conn(0x7fd84c0d0600 msgr2=0x7fd84c0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd8510ca640 1 --2- 172.21.15.37:0/3625855195 >> v2:172.21.15.203:3300/0 conn(0x7fd84c0d0600 0x7fd84c0cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd8510ca640 1 -- 172.21.15.37:0/3625855195 >> v1:172.21.15.37:6789/0 conn(0x7fd84c0b2780 legacy=0x7fd84c0b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd8510ca640 1 -- 172.21.15.37:0/3625855195 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c0d2300 con 0x7fd84c161160 2024-04-25T08:54:48.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd843fff640 1 -- 172.21.15.37:0/3625855195 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2923714157 0 0) 0x7fd84c0d1d40 con 0x7fd84c0b2780 2024-04-25T08:54:48.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd8510ca640 1 --2- 172.21.15.37:0/3625855195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd84c161160 0x7fd84c0d00c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0b4c00 tx=0x7fd84801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19fff6557f0b5189 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd843fff640 1 -- 172.21.15.37:0/3625855195 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480b1070 con 0x7fd84c161160 2024-04-25T08:54:48.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd843fff640 1 -- 172.21.15.37:0/3625855195 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8480ac600 con 0x7fd84c161160 2024-04-25T08:54:48.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd843fff640 1 -- 172.21.15.37:0/3625855195 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480ac9d0 con 0x7fd84c161160 2024-04-25T08:54:48.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd85332e640 1 -- 172.21.15.37:0/3625855195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd84c161160 msgr2=0x7fd84c0d00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd85332e640 1 --2- 172.21.15.37:0/3625855195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd84c161160 0x7fd84c0d00c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0b4c00 tx=0x7fd84801c3c0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd85332e640 1 -- 172.21.15.37:0/3625855195 shutdown_connections 2024-04-25T08:54:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd85332e640 1 --2- 172.21.15.37:0/3625855195 >> v2:172.21.15.203:3300/0 conn(0x7fd84c0d0600 0x7fd84c0cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd85332e640 1 --2- 172.21.15.37:0/3625855195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd84c161160 0x7fd84c0d00c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd85332e640 1 -- 172.21.15.37:0/3625855195 >> 172.21.15.37:0/3625855195 conn(0x7fd84c08b000 msgr2=0x7fd84c0d36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd85332e640 1 -- 172.21.15.37:0/3625855195 shutdown_connections 2024-04-25T08:54:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.443+0000 7fd85332e640 1 -- 172.21.15.37:0/3625855195 wait complete. 2024-04-25T08:54:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd85332e640 1 Processor -- start 2024-04-25T08:54:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd85332e640 1 -- start start 2024-04-25T08:54:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd85332e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd84c0b2780 0x7fd84c0b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd85332e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd84c0d0600 0x7fd84c158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd85332e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd84c161160 0x7fd84c0ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd85332e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0b4510 con 0x7fd84c0b2780 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd85332e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0b4390 con 0x7fd84c161160 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd85332e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84c0b4690 con 0x7fd84c0d0600 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8510ca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd84c0b2780 0x7fd84c0b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8510ca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd84c0b2780 0x7fd84c0b6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55452/0 (socket says 172.21.15.37:55452) 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8510ca640 1 -- 172.21.15.37:0/2651227962 learned_addr learned my addr 172.21.15.37:0/2651227962 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8510ca640 1 -- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd84c0d0600 msgr2=0x7fd84c158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8508c9640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd84c0d0600 0x7fd84c158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8518cb640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd84c161160 0x7fd84c0ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8510ca640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd84c0d0600 0x7fd84c158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8510ca640 1 -- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd84c161160 msgr2=0x7fd84c0ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8510ca640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd84c161160 0x7fd84c0ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8510ca640 1 -- 172.21.15.37:0/2651227962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c158d40 con 0x7fd84c0b2780 2024-04-25T08:54:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8518cb640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd84c161160 0x7fd84c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:48.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8508c9640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd84c0d0600 0x7fd84c158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:48.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd8510ca640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd84c0b2780 0x7fd84c0b6500 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd848018fd0 tx=0x7fd84801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd841ffb640 1 -- 172.21.15.37:0/2651227962 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480b1040 con 0x7fd84c0b2780 2024-04-25T08:54:48.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd841ffb640 1 -- 172.21.15.37:0/2651227962 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8480bc070 con 0x7fd84c0b2780 2024-04-25T08:54:48.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd85332e640 1 -- 172.21.15.37:0/2651227962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84c159020 con 0x7fd84c0b2780 2024-04-25T08:54:48.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd841ffb640 1 -- 172.21.15.37:0/2651227962 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480b7650 con 0x7fd84c0b2780 2024-04-25T08:54:48.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd85332e640 1 -- 172.21.15.37:0/2651227962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84c1595e0 con 0x7fd84c0b2780 2024-04-25T08:54:48.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd85332e640 1 -- 172.21.15.37:0/2651227962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84c04efa0 con 0x7fd84c0b2780 2024-04-25T08:54:48.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.447+0000 7fd841ffb640 1 -- 172.21.15.37:0/2651227962 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd8480b77f0 con 0x7fd84c0b2780 2024-04-25T08:54:48.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.451+0000 7fd841ffb640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd830066f60 0x7fd830069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.451+0000 7fd841ffb640 1 -- 172.21.15.37:0/2651227962 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd84811efe0 con 0x7fd84c0b2780 2024-04-25T08:54:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.451+0000 7fd8508c9640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd830066f60 0x7fd830069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.451+0000 7fd8508c9640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd830066f60 0x7fd830069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd83c000f70 tx=0x7fd83c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.455+0000 7fd841ffb640 1 -- 172.21.15.37:0/2651227962 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd8480c5020 con 0x7fd84c0b2780 2024-04-25T08:54:48.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.671+0000 7fd85332e640 1 -- 172.21.15.37:0/2651227962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd84c06ad40 con 0x7fd84c0b2780 2024-04-25T08:54:48.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.671+0000 7fd841ffb640 1 -- 172.21.15.37:0/2651227962 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1453 (secure 0 0 0) 0x7fd848016070 con 0x7fd84c0b2780 2024-04-25T08:54:48.665 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:48.666 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":162,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035137,"num_in_osds":8,"osd_in_since":1714035128,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":240381952,"bytes_avail":772853731328,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5937}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-04-25T08:54:31.163554+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:54:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 -- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd830066f60 msgr2=0x7fd830069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd830066f60 0x7fd830069420 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd83c000f70 tx=0x7fd83c009040 comp rx=0 tx=0).stop 2024-04-25T08:54:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 -- 172.21.15.37:0/2651227962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd84c0b2780 msgr2=0x7fd84c0b6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd84c0b2780 0x7fd84c0b6500 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd848018fd0 tx=0x7fd84801ac00 comp rx=0 tx=0).stop 2024-04-25T08:54:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 -- 172.21.15.37:0/2651227962 shutdown_connections 2024-04-25T08:54:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd830066f60 0x7fd830069420 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd84c161160 0x7fd84c0ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd84c0d0600 0x7fd84c158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 --2- 172.21.15.37:0/2651227962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd84c0b2780 0x7fd84c0b6500 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 -- 172.21.15.37:0/2651227962 >> 172.21.15.37:0/2651227962 conn(0x7fd84c08b000 msgr2=0x7fd84c06c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 -- 172.21.15.37:0/2651227962 shutdown_connections 2024-04-25T08:54:48.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.675+0000 7fd85332e640 1 -- 172.21.15.37:0/2651227962 wait complete. 2024-04-25T08:54:48.684 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:54:48.684 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:54:48.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7d5ad2640 1 Processor -- start 2024-04-25T08:54:48.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7d5ad2640 1 -- start start 2024-04-25T08:54:48.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7d5ad2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa7d015a700 0x7fa7d015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7d5ad2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa7d00b3f50 0x7fa7d00cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7d5ad2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d00b2090 con 0x7fa7d00ce430 2024-04-25T08:54:48.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7d5ad2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d015f200 con 0x7fa7d015a700 2024-04-25T08:54:48.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7d5ad2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d00b43e0 con 0x7fa7d00b3f50 2024-04-25T08:54:48.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7cf7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa7d015a700 0x7fa7d015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7cf7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa7d015a700 0x7fa7d015ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55462/0 (socket says 172.21.15.37:55462) 2024-04-25T08:54:48.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7cf7fe640 1 -- 172.21.15.37:0/2765525141 learned_addr learned my addr 172.21.15.37:0/2765525141 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:48.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7cf7fe640 1 -- 172.21.15.37:0/2765525141 >> v2:172.21.15.203:3300/0 conn(0x7fa7d00b3f50 msgr2=0x7fa7d00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7cf7fe640 1 --2- 172.21.15.37:0/2765525141 >> v2:172.21.15.203:3300/0 conn(0x7fa7d00b3f50 0x7fa7d00cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7cf7fe640 1 -- 172.21.15.37:0/2765525141 >> v1:172.21.15.203:6789/0 conn(0x7fa7d00ce430 legacy=0x7fa7d00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.803+0000 7fa7cf7fe640 1 -- 172.21.15.37:0/2765525141 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d00b46c0 con 0x7fa7d015a700 2024-04-25T08:54:48.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7cf7fe640 1 --2- 172.21.15.37:0/2765525141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d015a700 0x7fa7d015ecc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa7d006bd20 tx=0x7fa7c001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef2334a8245ba45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7ce7fc640 1 -- 172.21.15.37:0/2765525141 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00b2070 con 0x7fa7d015a700 2024-04-25T08:54:48.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7ce7fc640 1 -- 172.21.15.37:0/2765525141 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c00ad640 con 0x7fa7d015a700 2024-04-25T08:54:48.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7ce7fc640 1 -- 172.21.15.37:0/2765525141 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00ada10 con 0x7fa7d015a700 2024-04-25T08:54:48.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2765525141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d015a700 msgr2=0x7fa7d015ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7d5ad2640 1 --2- 172.21.15.37:0/2765525141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d015a700 0x7fa7d015ecc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fa7d006bd20 tx=0x7fa7c001c390 comp rx=0 tx=0).stop 2024-04-25T08:54:48.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2765525141 shutdown_connections 2024-04-25T08:54:48.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7d5ad2640 1 --2- 172.21.15.37:0/2765525141 >> v2:172.21.15.203:3300/0 conn(0x7fa7d00b3f50 0x7fa7d00cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7d5ad2640 1 --2- 172.21.15.37:0/2765525141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d015a700 0x7fa7d015ecc0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2765525141 >> 172.21.15.37:0/2765525141 conn(0x7fa7d00b11a0 msgr2=0x7fa7d00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2765525141 shutdown_connections 2024-04-25T08:54:48.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2765525141 wait complete. 2024-04-25T08:54:48.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7d5ad2640 1 Processor -- start 2024-04-25T08:54:48.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.807+0000 7fa7d5ad2640 1 -- start start 2024-04-25T08:54:48.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7d5ad2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d00ce430 0x7fa7d0154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7d5ad2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d015a700 0x7fa7d00a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7d5ad2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7d00ad7d0 0x7fa7d00a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7d5ad2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00cd490 con 0x7fa7d00ce430 2024-04-25T08:54:48.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7d5ad2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00cd310 con 0x7fa7d015a700 2024-04-25T08:54:48.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7d5ad2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d00b2bb0 con 0x7fa7d00ad7d0 2024-04-25T08:54:48.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d00ce430 0x7fa7d0154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d00ce430 0x7fa7d0154430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55474/0 (socket says 172.21.15.37:55474) 2024-04-25T08:54:48.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7cf7fe640 1 -- 172.21.15.37:0/2392478338 learned_addr learned my addr 172.21.15.37:0/2392478338 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:48.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7ceffd640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d015a700 0x7fa7d00a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7cf7fe640 1 -- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7d00ad7d0 msgr2=0x7fa7d00a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7cffff640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7d00ad7d0 0x7fa7d00a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7cf7fe640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7d00ad7d0 0x7fa7d00a8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7cf7fe640 1 -- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d015a700 msgr2=0x7fa7d00a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7cf7fe640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d015a700 0x7fa7d00a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7cf7fe640 1 -- 172.21.15.37:0/2392478338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d0154970 con 0x7fa7d00ce430 2024-04-25T08:54:48.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7cf7fe640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d00ce430 0x7fa7d0154430 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa7c001aa40 tx=0x7fa7c0019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7b7fff640 1 -- 172.21.15.37:0/2392478338 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00b2040 con 0x7fa7d00ce430 2024-04-25T08:54:48.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7b7fff640 1 -- 172.21.15.37:0/2392478338 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c00bb070 con 0x7fa7d00ce430 2024-04-25T08:54:48.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7b7fff640 1 -- 172.21.15.37:0/2392478338 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c00b68a0 con 0x7fa7d00ce430 2024-04-25T08:54:48.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2392478338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d00ac730 con 0x7fa7d00ce430 2024-04-25T08:54:48.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2392478338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d00accf0 con 0x7fa7d00ce430 2024-04-25T08:54:48.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2392478338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d004efa0 con 0x7fa7d00ce430 2024-04-25T08:54:48.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7b7fff640 1 -- 172.21.15.37:0/2392478338 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7c00b6a40 con 0x7fa7d00ce430 2024-04-25T08:54:48.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7b7fff640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7bc066f60 0x7fa7bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7b7fff640 1 -- 172.21.15.37:0/2392478338 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa7c011f4d0 con 0x7fa7d00ce430 2024-04-25T08:54:48.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.811+0000 7fa7ceffd640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7bc066f60 0x7fa7bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.815+0000 7fa7ceffd640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7bc066f60 0x7fa7bc069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00ceb50 tx=0x7fa7c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.815+0000 7fa7b7fff640 1 -- 172.21.15.37:0/2392478338 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7c00160d0 con 0x7fa7d00ce430 2024-04-25T08:54:48.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.971+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2392478338 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7d006f260 con 0x7fa7bc066f60 2024-04-25T08:54:48.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.987+0000 7fa7b7fff640 1 -- 172.21.15.37:0/2392478338 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246641 (secure 0 0 0) 0x7fa7d006f260 con 0x7fa7bc066f60 2024-04-25T08:54:48.985 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:48.986 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:54:48.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7bc066f60 msgr2=0x7fa7bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7bc066f60 0x7fa7bc069420 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00ceb50 tx=0x7fa7c4008040 comp rx=0 tx=0).stop 2024-04-25T08:54:48.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2392478338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d00ce430 msgr2=0x7fa7d0154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d00ce430 0x7fa7d0154430 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa7c001aa40 tx=0x7fa7c0019130 comp rx=0 tx=0).stop 2024-04-25T08:54:48.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2392478338 shutdown_connections 2024-04-25T08:54:48.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7bc066f60 0x7fa7bc069420 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7d00ad7d0 0x7fa7d00a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7d015a700 0x7fa7d00a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 --2- 172.21.15.37:0/2392478338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7d00ce430 0x7fa7d0154430 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2392478338 >> 172.21.15.37:0/2392478338 conn(0x7fa7d00b11a0 msgr2=0x7fa7d0164b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2392478338 shutdown_connections 2024-04-25T08:54:48.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:48.991+0000 7fa7d5ad2640 1 -- 172.21.15.37:0/2392478338 wait complete. 2024-04-25T08:54:49.003 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-04-25T08:54:47.168024+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234748,"kb_used_data":6088,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739972,"statfs":{"total":773094113280,"available":772853731328,"internally_reserved":0,"allocated":6234112,"data_stored":3426400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.753404"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.450878+0000","last_change":"2024-04-25T08:54:42.450878+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.450878+0000","last_undegraded":"2024-04-25T08:54:42.450878+0000","last_fullsized":"2024-04-25T08:54:42.450878+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443054+0000","last_change":"2024-04-25T08:54:42.443054+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443054+0000","last_undegraded":"2024-04-25T08:54:42.443054+0000","last_fullsized":"2024-04-25T08:54:42.443054+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444301+0000","last_change":"2024-04-25T08:54:42.444301+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444301+0000","last_undegraded":"2024-04-25T08:54:42.444301+0000","last_fullsized":"2024-04-25T08:54:42.444301+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.434500+0000","last_change":"2024-04-25T08:54:42.434500+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.434500+0000","last_undegraded":"2024-04-25T08:54:42.434500+0000","last_fullsized":"2024-04-25T08:54:42.434500+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.427901+0000","last_change":"2024-04-25T08:54:42.427901+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.427901+0000","last_undegraded":"2024-04-25T08:54:42.427901+0000","last_fullsized":"2024-04-25T08:54:42.427901+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.449668+0000","last_change":"2024-04-25T08:54:42.449668+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.449668+0000","last_undegraded":"2024-04-25T08:54:42.449668+0000","last_fullsized":"2024-04-25T08:54:42.449668+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444216+0000","last_change":"2024-04-25T08:54:42.444216+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444216+0000","last_undegraded":"2024-04-25T08:54:42.444216+0000","last_fullsized":"2024-04-25T08:54:42.444216+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.445284+0000","last_change":"2024-04-25T08:54:42.445284+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.445284+0000","last_undegraded":"2024-04-25T08:54:42.445284+0000","last_fullsized":"2024-04-25T08:54:42.445284+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.445459+0000","last_change":"2024-04-25T08:54:42.445459+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.445459+0000","last_undegraded":"2024-04-25T08:54:42.445459+0000","last_fullsized":"2024-04-25T08:54:42.445459+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.449723+0000","last_change":"2024-04-25T08:54:42.449723+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.449723+0000","last_undegraded":"2024-04-25T08:54:42.449723+0000","last_fullsized":"2024-04-25T08:54:42.449723+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.437742+0000","last_change":"2024-04-25T08:54:42.437742+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.437742+0000","last_undegraded":"2024-04-25T08:54:42.437742+0000","last_fullsized":"2024-04-25T08:54:42.437742+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446682+0000","last_change":"2024-04-25T08:54:42.446682+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446682+0000","last_undegraded":"2024-04-25T08:54:42.446682+0000","last_fullsized":"2024-04-25T08:54:42.446682+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.454164+0000","last_change":"2024-04-25T08:54:42.454164+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.454164+0000","last_undegraded":"2024-04-25T08:54:42.454164+0000","last_fullsized":"2024-04-25T08:54:42.454164+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447877+0000","last_change":"2024-04-25T08:54:42.447877+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447877+0000","last_undegraded":"2024-04-25T08:54:42.447877+0000","last_fullsized":"2024-04-25T08:54:42.447877+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436375+0000","last_change":"2024-04-25T08:54:42.436375+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436375+0000","last_undegraded":"2024-04-25T08:54:42.436375+0000","last_fullsized":"2024-04-25T08:54:42.436375+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.440722+0000","last_change":"2024-04-25T08:54:42.440722+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.440722+0000","last_undegraded":"2024-04-25T08:54:42.440722+0000","last_fullsized":"2024-04-25T08:54:42.440722+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.457634+0000","last_change":"2024-04-25T08:54:42.457634+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.457634+0000","last_undegraded":"2024-04-25T08:54:42.457634+0000","last_fullsized":"2024-04-25T08:54:42.457634+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435705+0000","last_change":"2024-04-25T08:54:42.435705+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435705+0000","last_undegraded":"2024-04-25T08:54:42.435705+0000","last_fullsized":"2024-04-25T08:54:42.435705+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443799+0000","last_change":"2024-04-25T08:54:42.443799+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443799+0000","last_undegraded":"2024-04-25T08:54:42.443799+0000","last_fullsized":"2024-04-25T08:54:42.443799+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.426676+0000","last_change":"2024-04-25T08:54:42.426676+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.426676+0000","last_undegraded":"2024-04-25T08:54:42.426676+0000","last_fullsized":"2024-04-25T08:54:42.426676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.424865+0000","last_change":"2024-04-25T08:54:42.424865+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.424865+0000","last_undegraded":"2024-04-25T08:54:42.424865+0000","last_fullsized":"2024-04-25T08:54:42.424865+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446467+0000","last_change":"2024-04-25T08:54:42.446467+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446467+0000","last_undegraded":"2024-04-25T08:54:42.446467+0000","last_fullsized":"2024-04-25T08:54:42.446467+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436604+0000","last_change":"2024-04-25T08:54:42.436604+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436604+0000","last_undegraded":"2024-04-25T08:54:42.436604+0000","last_fullsized":"2024-04-25T08:54:42.436604+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443913+0000","last_change":"2024-04-25T08:54:42.443913+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443913+0000","last_undegraded":"2024-04-25T08:54:42.443913+0000","last_fullsized":"2024-04-25T08:54:42.443913+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447326+0000","last_change":"2024-04-25T08:54:42.447326+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447326+0000","last_undegraded":"2024-04-25T08:54:42.447326+0000","last_fullsized":"2024-04-25T08:54:42.447326+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.448405+0000","last_change":"2024-04-25T08:54:42.448405+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.448405+0000","last_undegraded":"2024-04-25T08:54:42.448405+0000","last_fullsized":"2024-04-25T08:54:42.448405+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.443630+0000","last_change":"2024-04-25T08:54:42.443630+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.443630+0000","last_undegraded":"2024-04-25T08:54:42.443630+0000","last_fullsized":"2024-04-25T08:54:42.443630+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.425583+0000","last_change":"2024-04-25T08:54:42.425583+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.425583+0000","last_undegraded":"2024-04-25T08:54:42.425583+0000","last_fullsized":"2024-04-25T08:54:42.425583+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442282+0000","last_change":"2024-04-25T08:54:42.442282+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442282+0000","last_undegraded":"2024-04-25T08:54:42.442282+0000","last_fullsized":"2024-04-25T08:54:42.442282+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.441022+0000","last_change":"2024-04-25T08:54:42.441022+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.441022+0000","last_undegraded":"2024-04-25T08:54:42.441022+0000","last_fullsized":"2024-04-25T08:54:42.441022+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442094+0000","last_change":"2024-04-25T08:54:42.442094+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442094+0000","last_undegraded":"2024-04-25T08:54:42.442094+0000","last_fullsized":"2024-04-25T08:54:42.442094+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.432912+0000","last_change":"2024-04-25T08:54:42.432912+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.432912+0000","last_undegraded":"2024-04-25T08:54:42.432912+0000","last_fullsized":"2024-04-25T08:54:42.432912+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435703+0000","last_change":"2024-04-25T08:54:42.435703+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435703+0000","last_undegraded":"2024-04-25T08:54:42.435703+0000","last_fullsized":"2024-04-25T08:54:42.435703+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.454729+0000","last_change":"2024-04-25T08:54:42.435894+0000","last_active":"2024-04-25T08:54:42.454729+0000","last_peered":"2024-04-25T08:54:42.454729+0000","last_clean":"2024-04-25T08:54:42.454729+0000","last_became_active":"2024-04-25T08:54:42.435139+0000","last_became_peered":"2024-04-25T08:54:42.435139+0000","last_unstale":"2024-04-25T08:54:42.454729+0000","last_undegraded":"2024-04-25T08:54:42.454729+0000","last_fullsized":"2024-04-25T08:54:42.454729+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:55.487618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431034+0000","last_change":"2024-04-25T08:54:42.431034+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431034+0000","last_undegraded":"2024-04-25T08:54:42.431034+0000","last_fullsized":"2024-04-25T08:54:42.431034+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.448887+0000","last_change":"2024-04-25T08:54:42.434433+0000","last_active":"2024-04-25T08:54:42.448887+0000","last_peered":"2024-04-25T08:54:42.448887+0000","last_clean":"2024-04-25T08:54:42.448887+0000","last_became_active":"2024-04-25T08:54:42.434128+0000","last_became_peered":"2024-04-25T08:54:42.434128+0000","last_unstale":"2024-04-25T08:54:42.448887+0000","last_undegraded":"2024-04-25T08:54:42.448887+0000","last_fullsized":"2024-04-25T08:54:42.448887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:28.707217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.423925+0000","last_change":"2024-04-25T08:54:42.423925+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.423925+0000","last_undegraded":"2024-04-25T08:54:42.423925+0000","last_fullsized":"2024-04-25T08:54:42.423925+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.476536+0000","last_change":"2024-04-25T08:54:42.455613+0000","last_active":"2024-04-25T08:54:42.476536+0000","last_peered":"2024-04-25T08:54:42.476536+0000","last_clean":"2024-04-25T08:54:42.476536+0000","last_became_active":"2024-04-25T08:54:42.450954+0000","last_became_peered":"2024-04-25T08:54:42.450954+0000","last_unstale":"2024-04-25T08:54:42.476536+0000","last_undegraded":"2024-04-25T08:54:42.476536+0000","last_fullsized":"2024-04-25T08:54:42.476536+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439956+0000","last_change":"2024-04-25T08:54:42.439956+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439956+0000","last_undegraded":"2024-04-25T08:54:42.439956+0000","last_fullsized":"2024-04-25T08:54:42.439956+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.464425+0000","last_change":"2024-04-25T08:54:42.461742+0000","last_active":"2024-04-25T08:54:42.464425+0000","last_peered":"2024-04-25T08:54:42.464425+0000","last_clean":"2024-04-25T08:54:42.464425+0000","last_became_active":"2024-04-25T08:54:42.460925+0000","last_became_peered":"2024-04-25T08:54:42.460925+0000","last_unstale":"2024-04-25T08:54:42.464425+0000","last_undegraded":"2024-04-25T08:54:42.464425+0000","last_fullsized":"2024-04-25T08:54:42.464425+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:25.002800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.429055+0000","last_change":"2024-04-25T08:54:42.429055+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.429055+0000","last_undegraded":"2024-04-25T08:54:42.429055+0000","last_fullsized":"2024-04-25T08:54:42.429055+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.474003+0000","last_change":"2024-04-25T08:54:42.468263+0000","last_active":"2024-04-25T08:54:42.474003+0000","last_peered":"2024-04-25T08:54:42.474003+0000","last_clean":"2024-04-25T08:54:42.474003+0000","last_became_active":"2024-04-25T08:54:42.467914+0000","last_became_peered":"2024-04-25T08:54:42.467914+0000","last_unstale":"2024-04-25T08:54:42.474003+0000","last_undegraded":"2024-04-25T08:54:42.474003+0000","last_fullsized":"2024-04-25T08:54:42.474003+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:08.427686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439634+0000","last_change":"2024-04-25T08:54:42.439634+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439634+0000","last_undegraded":"2024-04-25T08:54:42.439634+0000","last_fullsized":"2024-04-25T08:54:42.439634+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.443413+0000","last_change":"2024-04-25T08:54:42.439471+0000","last_active":"2024-04-25T08:54:42.443413+0000","last_peered":"2024-04-25T08:54:42.443413+0000","last_clean":"2024-04-25T08:54:42.443413+0000","last_became_active":"2024-04-25T08:54:42.438730+0000","last_became_peered":"2024-04-25T08:54:42.438730+0000","last_unstale":"2024-04-25T08:54:42.443413+0000","last_undegraded":"2024-04-25T08:54:42.443413+0000","last_fullsized":"2024-04-25T08:54:42.443413+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.440613+0000","last_change":"2024-04-25T08:54:42.440613+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.440613+0000","last_undegraded":"2024-04-25T08:54:42.440613+0000","last_fullsized":"2024-04-25T08:54:42.440613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456706+0000","last_change":"2024-04-25T08:54:42.450407+0000","last_active":"2024-04-25T08:54:42.456706+0000","last_peered":"2024-04-25T08:54:42.456706+0000","last_clean":"2024-04-25T08:54:42.456706+0000","last_became_active":"2024-04-25T08:54:42.448040+0000","last_became_peered":"2024-04-25T08:54:42.448040+0000","last_unstale":"2024-04-25T08:54:42.456706+0000","last_undegraded":"2024-04-25T08:54:42.456706+0000","last_fullsized":"2024-04-25T08:54:42.456706+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:55.906854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.436423+0000","last_change":"2024-04-25T08:54:42.436423+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.436423+0000","last_undegraded":"2024-04-25T08:54:42.436423+0000","last_fullsized":"2024-04-25T08:54:42.436423+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471348+0000","last_change":"2024-04-25T08:54:42.466835+0000","last_active":"2024-04-25T08:54:42.471348+0000","last_peered":"2024-04-25T08:54:42.471348+0000","last_clean":"2024-04-25T08:54:42.471348+0000","last_became_active":"2024-04-25T08:54:42.466542+0000","last_became_peered":"2024-04-25T08:54:42.466542+0000","last_unstale":"2024-04-25T08:54:42.471348+0000","last_undegraded":"2024-04-25T08:54:42.471348+0000","last_fullsized":"2024-04-25T08:54:42.471348+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:46.278167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.430403+0000","last_change":"2024-04-25T08:54:42.430403+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.430403+0000","last_undegraded":"2024-04-25T08:54:42.430403+0000","last_fullsized":"2024-04-25T08:54:42.430403+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.461066+0000","last_change":"2024-04-25T08:54:42.458215+0000","last_active":"2024-04-25T08:54:42.461066+0000","last_peered":"2024-04-25T08:54:42.461066+0000","last_clean":"2024-04-25T08:54:42.461066+0000","last_became_active":"2024-04-25T08:54:42.457493+0000","last_became_peered":"2024-04-25T08:54:42.457493+0000","last_unstale":"2024-04-25T08:54:42.461066+0000","last_undegraded":"2024-04-25T08:54:42.461066+0000","last_fullsized":"2024-04-25T08:54:42.461066+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.435429+0000","last_change":"2024-04-25T08:54:42.435429+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.435429+0000","last_undegraded":"2024-04-25T08:54:42.435429+0000","last_fullsized":"2024-04-25T08:54:42.435429+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453284+0000","last_change":"2024-04-25T08:54:42.442179+0000","last_active":"2024-04-25T08:54:42.453284+0000","last_peered":"2024-04-25T08:54:42.453284+0000","last_clean":"2024-04-25T08:54:42.453284+0000","last_became_active":"2024-04-25T08:54:42.441094+0000","last_became_peered":"2024-04-25T08:54:42.441094+0000","last_unstale":"2024-04-25T08:54:42.453284+0000","last_undegraded":"2024-04-25T08:54:42.453284+0000","last_fullsized":"2024-04-25T08:54:42.453284+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442670+0000","last_change":"2024-04-25T08:54:42.442670+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442670+0000","last_undegraded":"2024-04-25T08:54:42.442670+0000","last_fullsized":"2024-04-25T08:54:42.442670+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.474953+0000","last_change":"2024-04-25T08:54:42.474076+0000","last_active":"2024-04-25T08:54:42.474953+0000","last_peered":"2024-04-25T08:54:42.474953+0000","last_clean":"2024-04-25T08:54:42.474953+0000","last_became_active":"2024-04-25T08:54:42.473614+0000","last_became_peered":"2024-04-25T08:54:42.473614+0000","last_unstale":"2024-04-25T08:54:42.474953+0000","last_undegraded":"2024-04-25T08:54:42.474953+0000","last_fullsized":"2024-04-25T08:54:42.474953+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.426642+0000","last_change":"2024-04-25T08:54:42.426642+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.426642+0000","last_undegraded":"2024-04-25T08:54:42.426642+0000","last_fullsized":"2024-04-25T08:54:42.426642+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.470808+0000","last_change":"2024-04-25T08:54:42.461363+0000","last_active":"2024-04-25T08:54:42.470808+0000","last_peered":"2024-04-25T08:54:42.470808+0000","last_clean":"2024-04-25T08:54:42.470808+0000","last_became_active":"2024-04-25T08:54:42.460631+0000","last_became_peered":"2024-04-25T08:54:42.460631+0000","last_unstale":"2024-04-25T08:54:42.470808+0000","last_undegraded":"2024-04-25T08:54:42.470808+0000","last_fullsized":"2024-04-25T08:54:42.470808+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:51.318624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.433752+0000","last_change":"2024-04-25T08:54:42.433752+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.433752+0000","last_undegraded":"2024-04-25T08:54:42.433752+0000","last_fullsized":"2024-04-25T08:54:42.433752+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.465848+0000","last_change":"2024-04-25T08:54:42.456578+0000","last_active":"2024-04-25T08:54:42.465848+0000","last_peered":"2024-04-25T08:54:42.465848+0000","last_clean":"2024-04-25T08:54:42.465848+0000","last_became_active":"2024-04-25T08:54:42.456167+0000","last_became_peered":"2024-04-25T08:54:42.456167+0000","last_unstale":"2024-04-25T08:54:42.465848+0000","last_undegraded":"2024-04-25T08:54:42.465848+0000","last_fullsized":"2024-04-25T08:54:42.465848+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442253+0000","last_change":"2024-04-25T08:54:42.442253+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442253+0000","last_undegraded":"2024-04-25T08:54:42.442253+0000","last_fullsized":"2024-04-25T08:54:42.442253+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456810+0000","last_change":"2024-04-25T08:54:42.451116+0000","last_active":"2024-04-25T08:54:42.456810+0000","last_peered":"2024-04-25T08:54:42.456810+0000","last_clean":"2024-04-25T08:54:42.456810+0000","last_became_active":"2024-04-25T08:54:42.447149+0000","last_became_peered":"2024-04-25T08:54:42.447149+0000","last_unstale":"2024-04-25T08:54:42.456810+0000","last_undegraded":"2024-04-25T08:54:42.456810+0000","last_fullsized":"2024-04-25T08:54:42.456810+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:35.144794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.447349+0000","last_change":"2024-04-25T08:54:42.447349+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.447349+0000","last_undegraded":"2024-04-25T08:54:42.447349+0000","last_fullsized":"2024-04-25T08:54:42.447349+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.460990+0000","last_change":"2024-04-25T08:54:42.458531+0000","last_active":"2024-04-25T08:54:42.460990+0000","last_peered":"2024-04-25T08:54:42.460990+0000","last_clean":"2024-04-25T08:54:42.460990+0000","last_became_active":"2024-04-25T08:54:42.457748+0000","last_became_peered":"2024-04-25T08:54:42.457748+0000","last_unstale":"2024-04-25T08:54:42.460990+0000","last_undegraded":"2024-04-25T08:54:42.460990+0000","last_fullsized":"2024-04-25T08:54:42.460990+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.446203+0000","last_change":"2024-04-25T08:54:42.446203+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.446203+0000","last_undegraded":"2024-04-25T08:54:42.446203+0000","last_fullsized":"2024-04-25T08:54:42.446203+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.473524+0000","last_change":"2024-04-25T08:54:42.466236+0000","last_active":"2024-04-25T08:54:42.473524+0000","last_peered":"2024-04-25T08:54:42.473524+0000","last_clean":"2024-04-25T08:54:42.473524+0000","last_became_active":"2024-04-25T08:54:42.465507+0000","last_became_peered":"2024-04-25T08:54:42.465507+0000","last_unstale":"2024-04-25T08:54:42.473524+0000","last_undegraded":"2024-04-25T08:54:42.473524+0000","last_fullsized":"2024-04-25T08:54:42.473524+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:22.360154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439111+0000","last_change":"2024-04-25T08:54:42.439111+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439111+0000","last_undegraded":"2024-04-25T08:54:42.439111+0000","last_fullsized":"2024-04-25T08:54:42.439111+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.473955+0000","last_change":"2024-04-25T08:54:42.469518+0000","last_active":"2024-04-25T08:54:42.473955+0000","last_peered":"2024-04-25T08:54:42.473955+0000","last_clean":"2024-04-25T08:54:42.473955+0000","last_became_active":"2024-04-25T08:54:42.469147+0000","last_became_peered":"2024-04-25T08:54:42.469147+0000","last_unstale":"2024-04-25T08:54:42.473955+0000","last_undegraded":"2024-04-25T08:54:42.473955+0000","last_fullsized":"2024-04-25T08:54:42.473955+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:44.662568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.437803+0000","last_change":"2024-04-25T08:54:42.437803+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.437803+0000","last_undegraded":"2024-04-25T08:54:42.437803+0000","last_fullsized":"2024-04-25T08:54:42.437803+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453781+0000","last_change":"2024-04-25T08:54:42.450583+0000","last_active":"2024-04-25T08:54:42.453781+0000","last_peered":"2024-04-25T08:54:42.453781+0000","last_clean":"2024-04-25T08:54:42.453781+0000","last_became_active":"2024-04-25T08:54:42.450316+0000","last_became_peered":"2024-04-25T08:54:42.450316+0000","last_unstale":"2024-04-25T08:54:42.453781+0000","last_undegraded":"2024-04-25T08:54:42.453781+0000","last_fullsized":"2024-04-25T08:54:42.453781+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:41.112056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.428105+0000","last_change":"2024-04-25T08:54:42.428105+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.428105+0000","last_undegraded":"2024-04-25T08:54:42.428105+0000","last_fullsized":"2024-04-25T08:54:42.428105+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.457250+0000","last_change":"2024-04-25T08:54:42.444423+0000","last_active":"2024-04-25T08:54:42.457250+0000","last_peered":"2024-04-25T08:54:42.457250+0000","last_clean":"2024-04-25T08:54:42.457250+0000","last_became_active":"2024-04-25T08:54:42.439289+0000","last_became_peered":"2024-04-25T08:54:42.439289+0000","last_unstale":"2024-04-25T08:54:42.457250+0000","last_undegraded":"2024-04-25T08:54:42.457250+0000","last_fullsized":"2024-04-25T08:54:42.457250+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:40.820232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439073+0000","last_change":"2024-04-25T08:54:42.439073+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439073+0000","last_undegraded":"2024-04-25T08:54:42.439073+0000","last_fullsized":"2024-04-25T08:54:42.439073+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.454629+0000","last_change":"2024-04-25T08:54:42.436282+0000","last_active":"2024-04-25T08:54:42.454629+0000","last_peered":"2024-04-25T08:54:42.454629+0000","last_clean":"2024-04-25T08:54:42.454629+0000","last_became_active":"2024-04-25T08:54:42.435494+0000","last_became_peered":"2024-04-25T08:54:42.435494+0000","last_unstale":"2024-04-25T08:54:42.454629+0000","last_undegraded":"2024-04-25T08:54:42.454629+0000","last_fullsized":"2024-04-25T08:54:42.454629+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:13.276684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.432676+0000","last_change":"2024-04-25T08:54:42.432676+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.432676+0000","last_undegraded":"2024-04-25T08:54:42.432676+0000","last_fullsized":"2024-04-25T08:54:42.432676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.476553+0000","last_change":"2024-04-25T08:54:42.456188+0000","last_active":"2024-04-25T08:54:42.476553+0000","last_peered":"2024-04-25T08:54:42.476553+0000","last_clean":"2024-04-25T08:54:42.476553+0000","last_became_active":"2024-04-25T08:54:42.455422+0000","last_became_peered":"2024-04-25T08:54:42.455422+0000","last_unstale":"2024-04-25T08:54:42.476553+0000","last_undegraded":"2024-04-25T08:54:42.476553+0000","last_fullsized":"2024-04-25T08:54:42.476553+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.427057+0000","last_change":"2024-04-25T08:54:42.427057+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.427057+0000","last_undegraded":"2024-04-25T08:54:42.427057+0000","last_fullsized":"2024-04-25T08:54:42.427057+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471615+0000","last_change":"2024-04-25T08:54:42.463454+0000","last_active":"2024-04-25T08:54:42.471615+0000","last_peered":"2024-04-25T08:54:42.471615+0000","last_clean":"2024-04-25T08:54:42.471615+0000","last_became_active":"2024-04-25T08:54:42.461962+0000","last_became_peered":"2024-04-25T08:54:42.461962+0000","last_unstale":"2024-04-25T08:54:42.471615+0000","last_undegraded":"2024-04-25T08:54:42.471615+0000","last_fullsized":"2024-04-25T08:54:42.471615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:08.285003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.441892+0000","last_change":"2024-04-25T08:54:42.441892+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.441892+0000","last_undegraded":"2024-04-25T08:54:42.441892+0000","last_fullsized":"2024-04-25T08:54:42.441892+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.465722+0000","last_change":"2024-04-25T08:54:42.456453+0000","last_active":"2024-04-25T08:54:42.465722+0000","last_peered":"2024-04-25T08:54:42.465722+0000","last_clean":"2024-04-25T08:54:42.465722+0000","last_became_active":"2024-04-25T08:54:42.456124+0000","last_became_peered":"2024-04-25T08:54:42.456124+0000","last_unstale":"2024-04-25T08:54:42.465722+0000","last_undegraded":"2024-04-25T08:54:42.465722+0000","last_fullsized":"2024-04-25T08:54:42.465722+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439639+0000","last_change":"2024-04-25T08:54:42.439639+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439639+0000","last_undegraded":"2024-04-25T08:54:42.439639+0000","last_fullsized":"2024-04-25T08:54:42.439639+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.453850+0000","last_change":"2024-04-25T08:54:42.448496+0000","last_active":"2024-04-25T08:54:42.453850+0000","last_peered":"2024-04-25T08:54:42.453850+0000","last_clean":"2024-04-25T08:54:42.453850+0000","last_became_active":"2024-04-25T08:54:42.447727+0000","last_became_peered":"2024-04-25T08:54:42.447727+0000","last_unstale":"2024-04-25T08:54:42.453850+0000","last_undegraded":"2024-04-25T08:54:42.453850+0000","last_fullsized":"2024-04-25T08:54:42.453850+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:30.126539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431858+0000","last_change":"2024-04-25T08:54:42.431858+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431858+0000","last_undegraded":"2024-04-25T08:54:42.431858+0000","last_fullsized":"2024-04-25T08:54:42.431858+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.456883+0000","last_change":"2024-04-25T08:54:42.449480+0000","last_active":"2024-04-25T08:54:42.456883+0000","last_peered":"2024-04-25T08:54:42.456883+0000","last_clean":"2024-04-25T08:54:42.456883+0000","last_became_active":"2024-04-25T08:54:42.448749+0000","last_became_peered":"2024-04-25T08:54:42.448749+0000","last_unstale":"2024-04-25T08:54:42.456883+0000","last_undegraded":"2024-04-25T08:54:42.456883+0000","last_fullsized":"2024-04-25T08:54:42.456883+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:19.814248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439551+0000","last_change":"2024-04-25T08:54:42.439551+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439551+0000","last_undegraded":"2024-04-25T08:54:42.439551+0000","last_fullsized":"2024-04-25T08:54:42.439551+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.459614+0000","last_change":"2024-04-25T08:54:42.455018+0000","last_active":"2024-04-25T08:54:42.459614+0000","last_peered":"2024-04-25T08:54:42.459614+0000","last_clean":"2024-04-25T08:54:42.459614+0000","last_became_active":"2024-04-25T08:54:42.454140+0000","last_became_peered":"2024-04-25T08:54:42.454140+0000","last_unstale":"2024-04-25T08:54:42.459614+0000","last_undegraded":"2024-04-25T08:54:42.459614+0000","last_fullsized":"2024-04-25T08:54:42.459614+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:28.707217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.438653+0000","last_change":"2024-04-25T08:54:42.438653+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.438653+0000","last_undegraded":"2024-04-25T08:54:42.438653+0000","last_fullsized":"2024-04-25T08:54:42.438653+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.466103+0000","last_change":"2024-04-25T08:54:42.464859+0000","last_active":"2024-04-25T08:54:42.466103+0000","last_peered":"2024-04-25T08:54:42.466103+0000","last_clean":"2024-04-25T08:54:42.466103+0000","last_became_active":"2024-04-25T08:54:42.457143+0000","last_became_peered":"2024-04-25T08:54:42.457143+0000","last_unstale":"2024-04-25T08:54:42.466103+0000","last_undegraded":"2024-04-25T08:54:42.466103+0000","last_fullsized":"2024-04-25T08:54:42.466103+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.444706+0000","last_change":"2024-04-25T08:54:42.444706+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.444706+0000","last_undegraded":"2024-04-25T08:54:42.444706+0000","last_fullsized":"2024-04-25T08:54:42.444706+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471541+0000","last_change":"2024-04-25T08:54:42.463152+0000","last_active":"2024-04-25T08:54:42.471541+0000","last_peered":"2024-04-25T08:54:42.471541+0000","last_clean":"2024-04-25T08:54:42.471541+0000","last_became_active":"2024-04-25T08:54:42.462413+0000","last_became_peered":"2024-04-25T08:54:42.462413+0000","last_unstale":"2024-04-25T08:54:42.471541+0000","last_undegraded":"2024-04-25T08:54:42.471541+0000","last_fullsized":"2024-04-25T08:54:42.471541+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:15.835374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.431106+0000","last_change":"2024-04-25T08:54:42.431106+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.431106+0000","last_undegraded":"2024-04-25T08:54:42.431106+0000","last_fullsized":"2024-04-25T08:54:42.431106+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471662+0000","last_change":"2024-04-25T08:54:42.463691+0000","last_active":"2024-04-25T08:54:42.471662+0000","last_peered":"2024-04-25T08:54:42.471662+0000","last_clean":"2024-04-25T08:54:42.471662+0000","last_became_active":"2024-04-25T08:54:42.462922+0000","last_became_peered":"2024-04-25T08:54:42.462922+0000","last_unstale":"2024-04-25T08:54:42.471662+0000","last_undegraded":"2024-04-25T08:54:42.471662+0000","last_fullsized":"2024-04-25T08:54:42.471662+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:18.405142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.442356+0000","last_change":"2024-04-25T08:54:42.442356+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.442356+0000","last_undegraded":"2024-04-25T08:54:42.442356+0000","last_fullsized":"2024-04-25T08:54:42.442356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.452975+0000","last_change":"2024-04-25T08:54:42.441517+0000","last_active":"2024-04-25T08:54:42.452975+0000","last_peered":"2024-04-25T08:54:42.452975+0000","last_clean":"2024-04-25T08:54:42.452975+0000","last_became_active":"2024-04-25T08:54:42.439663+0000","last_became_peered":"2024-04-25T08:54:42.439663+0000","last_unstale":"2024-04-25T08:54:42.452975+0000","last_undegraded":"2024-04-25T08:54:42.452975+0000","last_fullsized":"2024-04-25T08:54:42.452975+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.439988+0000","last_change":"2024-04-25T08:54:42.439988+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.439988+0000","last_undegraded":"2024-04-25T08:54:42.439988+0000","last_fullsized":"2024-04-25T08:54:42.439988+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.459448+0000","last_change":"2024-04-25T08:54:42.456403+0000","last_active":"2024-04-25T08:54:42.459448+0000","last_peered":"2024-04-25T08:54:42.459448+0000","last_clean":"2024-04-25T08:54:42.459448+0000","last_became_active":"2024-04-25T08:54:42.455569+0000","last_became_peered":"2024-04-25T08:54:42.455569+0000","last_unstale":"2024-04-25T08:54:42.459448+0000","last_undegraded":"2024-04-25T08:54:42.459448+0000","last_fullsized":"2024-04-25T08:54:42.459448+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:44.370399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":53,"state":"creating+peering","last_fresh":"2024-04-25T08:54:42.423932+0000","last_change":"2024-04-25T08:54:42.423932+0000","last_active":"2024-04-25T08:54:42.412663+0000","last_peered":"2024-04-25T08:54:42.412663+0000","last_clean":"2024-04-25T08:54:42.412663+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:42.423932+0000","last_undegraded":"2024-04-25T08:54:42.423932+0000","last_fullsized":"2024-04-25T08:54:42.423932+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.471131+0000","last_change":"2024-04-25T08:54:42.466348+0000","last_active":"2024-04-25T08:54:42.471131+0000","last_peered":"2024-04-25T08:54:42.471131+0000","last_clean":"2024-04-25T08:54:42.471131+0000","last_became_active":"2024-04-25T08:54:42.456331+0000","last_became_peered":"2024-04-25T08:54:42.456331+0000","last_unstale":"2024-04-25T08:54:42.471131+0000","last_undegraded":"2024-04-25T08:54:42.471131+0000","last_fullsized":"2024-04-25T08:54:42.471131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:34.520291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":231,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-04-25T08:54:42.445808+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:54:42.445808+0000","last_peered":"2024-04-25T08:54:42.445808+0000","last_clean":"2024-04-25T08:54:42.445808+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:54:42.445808+0000","last_undegraded":"2024-04-25T08:54:42.445808+0000","last_fullsized":"2024-04-25T08:54:42.445808+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1224704,"data_stored":871076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31796,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340044,"statfs":{"total":96636764160,"available":96604205056,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":280708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32376,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339464,"statfs":{"total":96636764160,"available":96603611136,"internally_reserved":0,"allocated":1224704,"data_stored":871076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:54:52.009 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:54:52.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.155+0000 7fe91bdcc640 1 Processor -- start 2024-04-25T08:54:52.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.155+0000 7fe91bdcc640 1 -- start start 2024-04-25T08:54:52.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.155+0000 7fe91bdcc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9140ce510 0x7fe9140ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.155+0000 7fe91bdcc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe914144ca0 0x7fe91406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.155+0000 7fe91bdcc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe91406ec60 0x7fe914142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.155+0000 7fe91bdcc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9140cc2b0 con 0x7fe914144ca0 2024-04-25T08:54:52.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.155+0000 7fe91bdcc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe91406e290 con 0x7fe91406ec60 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.155+0000 7fe91bdcc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140cee30 con 0x7fe9140ce510 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.155+0000 7fe919367640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe914144ca0 0x7fe91406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.155+0000 7fe919367640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe914144ca0 0x7fe91406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52342/0 (socket says 172.21.15.37:52342) 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.155+0000 7fe919367640 1 -- 172.21.15.37:0/2181260335 learned_addr learned my addr 172.21.15.37:0/2181260335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe919b68640 1 --2- 172.21.15.37:0/2181260335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9140ce510 0x7fe9140ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe91a369640 1 --2- 172.21.15.37:0/2181260335 >> v2:172.21.15.203:3300/0 conn(0x7fe91406ec60 0x7fe914142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe919367640 1 -- 172.21.15.37:0/2181260335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9140ce510 msgr2=0x7fe9140ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe919367640 1 --2- 172.21.15.37:0/2181260335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9140ce510 0x7fe9140ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe919367640 1 -- 172.21.15.37:0/2181260335 >> v2:172.21.15.203:3300/0 conn(0x7fe91406ec60 msgr2=0x7fe914142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe919367640 1 --2- 172.21.15.37:0/2181260335 >> v2:172.21.15.203:3300/0 conn(0x7fe91406ec60 0x7fe914142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe919367640 1 -- 172.21.15.37:0/2181260335 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9140ccc70 con 0x7fe914144ca0 2024-04-25T08:54:52.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe91a369640 1 --2- 172.21.15.37:0/2181260335 >> v2:172.21.15.203:3300/0 conn(0x7fe91406ec60 0x7fe914142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:52.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe919367640 1 --2- 172.21.15.37:0/2181260335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe914144ca0 0x7fe91406e720 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe9140cdbc0 tx=0x7fe90401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a722212ac6f0647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe918b66640 1 -- 172.21.15.37:0/2181260335 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9040b5070 con 0x7fe914144ca0 2024-04-25T08:54:52.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe918b66640 1 -- 172.21.15.37:0/2181260335 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9040a89d0 con 0x7fe914144ca0 2024-04-25T08:54:52.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe918b66640 1 -- 172.21.15.37:0/2181260335 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9040a8da0 con 0x7fe914144ca0 2024-04-25T08:54:52.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe91bdcc640 1 -- 172.21.15.37:0/2181260335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe914144ca0 msgr2=0x7fe91406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe91bdcc640 1 --2- 172.21.15.37:0/2181260335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe914144ca0 0x7fe91406e720 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe9140cdbc0 tx=0x7fe90401ed10 comp rx=0 tx=0).stop 2024-04-25T08:54:52.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe91bdcc640 1 -- 172.21.15.37:0/2181260335 shutdown_connections 2024-04-25T08:54:52.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe91bdcc640 1 --2- 172.21.15.37:0/2181260335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe914144ca0 0x7fe91406e720 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe91bdcc640 1 --2- 172.21.15.37:0/2181260335 >> v2:172.21.15.203:3300/0 conn(0x7fe91406ec60 0x7fe914142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe91bdcc640 1 --2- 172.21.15.37:0/2181260335 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9140ce510 0x7fe9140ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe91bdcc640 1 -- 172.21.15.37:0/2181260335 >> 172.21.15.37:0/2181260335 conn(0x7fe9140b11a0 msgr2=0x7fe9140b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe91bdcc640 1 -- 172.21.15.37:0/2181260335 shutdown_connections 2024-04-25T08:54:52.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.159+0000 7fe91bdcc640 1 -- 172.21.15.37:0/2181260335 wait complete. 2024-04-25T08:54:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91bdcc640 1 Processor -- start 2024-04-25T08:54:52.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91bdcc640 1 -- start start 2024-04-25T08:54:52.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91bdcc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91406ec60 0x7fe91415b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91bdcc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9140ce510 0x7fe9140c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91bdcc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe914144ca0 0x7fe9140c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91bdcc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9140cc900 con 0x7fe91406ec60 2024-04-25T08:54:52.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91bdcc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe91406df20 con 0x7fe9140ce510 2024-04-25T08:54:52.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91bdcc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9141451f0 con 0x7fe914144ca0 2024-04-25T08:54:52.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe919b68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91406ec60 0x7fe91415b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe919b68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91406ec60 0x7fe91415b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52352/0 (socket says 172.21.15.37:52352) 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe919b68640 1 -- 172.21.15.37:0/1900182187 learned_addr learned my addr 172.21.15.37:0/1900182187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe919367640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9140ce510 0x7fe9140c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91a369640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe914144ca0 0x7fe9140c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe919b68640 1 -- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe914144ca0 msgr2=0x7fe9140c6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe919b68640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe914144ca0 0x7fe9140c6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe919b68640 1 -- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9140ce510 msgr2=0x7fe9140c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe919b68640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9140ce510 0x7fe9140c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe919b68640 1 -- 172.21.15.37:0/1900182187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9141e3520 con 0x7fe91406ec60 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91a369640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe914144ca0 0x7fe9140c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe919b68640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91406ec60 0x7fe91415b0e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe91000d800 tx=0x7fe91000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe90a7fc640 1 -- 172.21.15.37:0/1900182187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91000b6c0 con 0x7fe91406ec60 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe90a7fc640 1 -- 172.21.15.37:0/1900182187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe910019070 con 0x7fe91406ec60 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91bdcc640 1 -- 172.21.15.37:0/1900182187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9141e3800 con 0x7fe91406ec60 2024-04-25T08:54:52.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe90a7fc640 1 -- 172.21.15.37:0/1900182187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91000bae0 con 0x7fe91406ec60 2024-04-25T08:54:52.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91bdcc640 1 -- 172.21.15.37:0/1900182187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9141e3dc0 con 0x7fe91406ec60 2024-04-25T08:54:52.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.163+0000 7fe91bdcc640 1 -- 172.21.15.37:0/1900182187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91404efa0 con 0x7fe91406ec60 2024-04-25T08:54:52.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.167+0000 7fe90a7fc640 1 -- 172.21.15.37:0/1900182187 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe91000bc80 con 0x7fe91406ec60 2024-04-25T08:54:52.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.167+0000 7fe90a7fc640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe8f8066f60 0x7fe8f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.167+0000 7fe90a7fc640 1 -- 172.21.15.37:0/1900182187 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9100856b0 con 0x7fe91406ec60 2024-04-25T08:54:52.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.167+0000 7fe919367640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe8f8066f60 0x7fe8f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.167+0000 7fe919367640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe8f8066f60 0x7fe8f8069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe9040b2000 tx=0x7fe904008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.171+0000 7fe90a7fc640 1 -- 172.21.15.37:0/1900182187 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe910056530 con 0x7fe91406ec60 2024-04-25T08:54:52.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.327+0000 7fe91bdcc640 1 -- 172.21.15.37:0/1900182187 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9140b1cc0 con 0x7fe8f8066f60 2024-04-25T08:54:52.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.343+0000 7fe90a7fc640 1 -- 172.21.15.37:0/1900182187 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251957 (secure 0 0 0) 0x7fe9140b1cc0 con 0x7fe8f8066f60 2024-04-25T08:54:52.340 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:52.341 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 -- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe8f8066f60 msgr2=0x7fe8f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe8f8066f60 0x7fe8f8069420 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fe9040b2000 tx=0x7fe904008040 comp rx=0 tx=0).stop 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 -- 172.21.15.37:0/1900182187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91406ec60 msgr2=0x7fe91415b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91406ec60 0x7fe91415b0e0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe91000d800 tx=0x7fe91000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 -- 172.21.15.37:0/1900182187 shutdown_connections 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe8f8066f60 0x7fe8f8069420 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe914144ca0 0x7fe9140c6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9140ce510 0x7fe9140c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 --2- 172.21.15.37:0/1900182187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe91406ec60 0x7fe91415b0e0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 -- 172.21.15.37:0/1900182187 >> 172.21.15.37:0/1900182187 conn(0x7fe9140b11a0 msgr2=0x7fe9140cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 -- 172.21.15.37:0/1900182187 shutdown_connections 2024-04-25T08:54:52.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.347+0000 7fe91bdcc640 1 -- 172.21.15.37:0/1900182187 wait complete. 2024-04-25T08:54:52.359 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-04-25T08:54:51.169599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235024,"kb_used_data":6344,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754739696,"statfs":{"total":773094113280,"available":772853448704,"internally_reserved":0,"allocated":6496256,"data_stored":3542092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":50,"apply_latency_ms":50,"commit_latency_ns":50000000,"apply_latency_ns":50000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002794"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.547019+0000","last_change":"2024-04-25T08:54:43.484428+0000","last_active":"2024-04-25T08:54:45.547019+0000","last_peered":"2024-04-25T08:54:45.547019+0000","last_clean":"2024-04-25T08:54:45.547019+0000","last_became_active":"2024-04-25T08:54:43.483938+0000","last_became_peered":"2024-04-25T08:54:43.483938+0000","last_unstale":"2024-04-25T08:54:45.547019+0000","last_undegraded":"2024-04-25T08:54:45.547019+0000","last_fullsized":"2024-04-25T08:54:45.547019+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:22.784539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.503458+0000","last_change":"2024-04-25T08:54:43.478746+0000","last_active":"2024-04-25T08:54:44.503458+0000","last_peered":"2024-04-25T08:54:44.503458+0000","last_clean":"2024-04-25T08:54:44.503458+0000","last_became_active":"2024-04-25T08:54:43.477585+0000","last_became_peered":"2024-04-25T08:54:43.477585+0000","last_unstale":"2024-04-25T08:54:44.503458+0000","last_undegraded":"2024-04-25T08:54:44.503458+0000","last_fullsized":"2024-04-25T08:54:44.503458+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:23.368572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499752+0000","last_change":"2024-04-25T08:54:43.503818+0000","last_active":"2024-04-25T08:54:44.499752+0000","last_peered":"2024-04-25T08:54:44.499752+0000","last_clean":"2024-04-25T08:54:44.499752+0000","last_became_active":"2024-04-25T08:54:43.503548+0000","last_became_peered":"2024-04-25T08:54:43.503548+0000","last_unstale":"2024-04-25T08:54:44.499752+0000","last_undegraded":"2024-04-25T08:54:44.499752+0000","last_fullsized":"2024-04-25T08:54:44.499752+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:10.010158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.503217+0000","last_change":"2024-04-25T08:54:43.488262+0000","last_active":"2024-04-25T08:54:44.503217+0000","last_peered":"2024-04-25T08:54:44.503217+0000","last_clean":"2024-04-25T08:54:44.503217+0000","last_became_active":"2024-04-25T08:54:43.487537+0000","last_became_peered":"2024-04-25T08:54:43.487537+0000","last_unstale":"2024-04-25T08:54:44.503217+0000","last_undegraded":"2024-04-25T08:54:44.503217+0000","last_fullsized":"2024-04-25T08:54:44.503217+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:19.413559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.755934+0000","last_change":"2024-04-25T08:54:43.475216+0000","last_active":"2024-04-25T08:54:44.755934+0000","last_peered":"2024-04-25T08:54:44.755934+0000","last_clean":"2024-04-25T08:54:44.755934+0000","last_became_active":"2024-04-25T08:54:43.474496+0000","last_became_peered":"2024-04-25T08:54:43.474496+0000","last_unstale":"2024-04-25T08:54:44.755934+0000","last_undegraded":"2024-04-25T08:54:44.755934+0000","last_fullsized":"2024-04-25T08:54:44.755934+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:36.982333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.546587+0000","last_change":"2024-04-25T08:54:43.466772+0000","last_active":"2024-04-25T08:54:45.546587+0000","last_peered":"2024-04-25T08:54:45.546587+0000","last_clean":"2024-04-25T08:54:45.546587+0000","last_became_active":"2024-04-25T08:54:43.464961+0000","last_became_peered":"2024-04-25T08:54:43.464961+0000","last_unstale":"2024-04-25T08:54:45.546587+0000","last_undegraded":"2024-04-25T08:54:45.546587+0000","last_fullsized":"2024-04-25T08:54:45.546587+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:09.293421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499091+0000","last_change":"2024-04-25T08:54:43.496454+0000","last_active":"2024-04-25T08:54:44.499091+0000","last_peered":"2024-04-25T08:54:44.499091+0000","last_clean":"2024-04-25T08:54:44.499091+0000","last_became_active":"2024-04-25T08:54:43.496165+0000","last_became_peered":"2024-04-25T08:54:43.496165+0000","last_unstale":"2024-04-25T08:54:44.499091+0000","last_undegraded":"2024-04-25T08:54:44.499091+0000","last_fullsized":"2024-04-25T08:54:44.499091+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:00.558105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456043+0000","last_change":"2024-04-25T08:54:43.497601+0000","last_active":"2024-04-25T08:54:44.456043+0000","last_peered":"2024-04-25T08:54:44.456043+0000","last_clean":"2024-04-25T08:54:44.456043+0000","last_became_active":"2024-04-25T08:54:43.497269+0000","last_became_peered":"2024-04-25T08:54:43.497269+0000","last_unstale":"2024-04-25T08:54:44.456043+0000","last_undegraded":"2024-04-25T08:54:44.456043+0000","last_fullsized":"2024-04-25T08:54:44.456043+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:10.010158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457345+0000","last_change":"2024-04-25T08:54:43.501707+0000","last_active":"2024-04-25T08:54:44.457345+0000","last_peered":"2024-04-25T08:54:44.457345+0000","last_clean":"2024-04-25T08:54:44.457345+0000","last_became_active":"2024-04-25T08:54:43.501029+0000","last_became_peered":"2024-04-25T08:54:43.501029+0000","last_unstale":"2024-04-25T08:54:44.457345+0000","last_undegraded":"2024-04-25T08:54:44.457345+0000","last_fullsized":"2024-04-25T08:54:44.457345+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:20.686701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.546920+0000","last_change":"2024-04-25T08:54:43.478145+0000","last_active":"2024-04-25T08:54:45.546920+0000","last_peered":"2024-04-25T08:54:45.546920+0000","last_clean":"2024-04-25T08:54:45.546920+0000","last_became_active":"2024-04-25T08:54:43.477420+0000","last_became_peered":"2024-04-25T08:54:43.477420+0000","last_unstale":"2024-04-25T08:54:45.546920+0000","last_undegraded":"2024-04-25T08:54:45.546920+0000","last_fullsized":"2024-04-25T08:54:45.546920+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:23.376794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.512057+0000","last_change":"2024-04-25T08:54:43.489655+0000","last_active":"2024-04-25T08:54:44.512057+0000","last_peered":"2024-04-25T08:54:44.512057+0000","last_clean":"2024-04-25T08:54:44.512057+0000","last_became_active":"2024-04-25T08:54:43.489075+0000","last_became_peered":"2024-04-25T08:54:43.489075+0000","last_unstale":"2024-04-25T08:54:44.512057+0000","last_undegraded":"2024-04-25T08:54:44.512057+0000","last_fullsized":"2024-04-25T08:54:44.512057+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:22.274122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456852+0000","last_change":"2024-04-25T08:54:43.486001+0000","last_active":"2024-04-25T08:54:44.456852+0000","last_peered":"2024-04-25T08:54:44.456852+0000","last_clean":"2024-04-25T08:54:44.456852+0000","last_became_active":"2024-04-25T08:54:43.482139+0000","last_became_peered":"2024-04-25T08:54:43.482139+0000","last_unstale":"2024-04-25T08:54:44.456852+0000","last_undegraded":"2024-04-25T08:54:44.456852+0000","last_fullsized":"2024-04-25T08:54:44.456852+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:05.779017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.546585+0000","last_change":"2024-04-25T08:54:43.462662+0000","last_active":"2024-04-25T08:54:45.546585+0000","last_peered":"2024-04-25T08:54:45.546585+0000","last_clean":"2024-04-25T08:54:45.546585+0000","last_became_active":"2024-04-25T08:54:43.461904+0000","last_became_peered":"2024-04-25T08:54:43.461904+0000","last_unstale":"2024-04-25T08:54:45.546585+0000","last_undegraded":"2024-04-25T08:54:45.546585+0000","last_fullsized":"2024-04-25T08:54:45.546585+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:16.843792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499278+0000","last_change":"2024-04-25T08:54:43.491708+0000","last_active":"2024-04-25T08:54:44.499278+0000","last_peered":"2024-04-25T08:54:44.499278+0000","last_clean":"2024-04-25T08:54:44.499278+0000","last_became_active":"2024-04-25T08:54:43.491407+0000","last_became_peered":"2024-04-25T08:54:43.491407+0000","last_unstale":"2024-04-25T08:54:44.499278+0000","last_undegraded":"2024-04-25T08:54:44.499278+0000","last_fullsized":"2024-04-25T08:54:44.499278+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:02.679172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.502848+0000","last_change":"2024-04-25T08:54:43.478049+0000","last_active":"2024-04-25T08:54:44.502848+0000","last_peered":"2024-04-25T08:54:44.502848+0000","last_clean":"2024-04-25T08:54:44.502848+0000","last_became_active":"2024-04-25T08:54:43.477325+0000","last_became_peered":"2024-04-25T08:54:43.477325+0000","last_unstale":"2024-04-25T08:54:44.502848+0000","last_undegraded":"2024-04-25T08:54:44.502848+0000","last_fullsized":"2024-04-25T08:54:44.502848+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:52.327042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.546356+0000","last_change":"2024-04-25T08:54:43.463112+0000","last_active":"2024-04-25T08:54:45.546356+0000","last_peered":"2024-04-25T08:54:45.546356+0000","last_clean":"2024-04-25T08:54:45.546356+0000","last_became_active":"2024-04-25T08:54:43.461999+0000","last_became_peered":"2024-04-25T08:54:43.461999+0000","last_unstale":"2024-04-25T08:54:45.546356+0000","last_undegraded":"2024-04-25T08:54:45.546356+0000","last_fullsized":"2024-04-25T08:54:45.546356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:57.710497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.500067+0000","last_change":"2024-04-25T08:54:43.494911+0000","last_active":"2024-04-25T08:54:44.500067+0000","last_peered":"2024-04-25T08:54:44.500067+0000","last_clean":"2024-04-25T08:54:44.500067+0000","last_became_active":"2024-04-25T08:54:43.491919+0000","last_became_peered":"2024-04-25T08:54:43.491919+0000","last_unstale":"2024-04-25T08:54:44.500067+0000","last_undegraded":"2024-04-25T08:54:44.500067+0000","last_fullsized":"2024-04-25T08:54:44.500067+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:05.779017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.503691+0000","last_change":"2024-04-25T08:54:43.476163+0000","last_active":"2024-04-25T08:54:44.503691+0000","last_peered":"2024-04-25T08:54:44.503691+0000","last_clean":"2024-04-25T08:54:44.503691+0000","last_became_active":"2024-04-25T08:54:43.469440+0000","last_became_peered":"2024-04-25T08:54:43.469440+0000","last_unstale":"2024-04-25T08:54:44.503691+0000","last_undegraded":"2024-04-25T08:54:44.503691+0000","last_fullsized":"2024-04-25T08:54:44.503691+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:51.285978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499793+0000","last_change":"2024-04-25T08:54:43.485631+0000","last_active":"2024-04-25T08:54:44.499793+0000","last_peered":"2024-04-25T08:54:44.499793+0000","last_clean":"2024-04-25T08:54:44.499793+0000","last_became_active":"2024-04-25T08:54:43.485304+0000","last_became_peered":"2024-04-25T08:54:43.485304+0000","last_unstale":"2024-04-25T08:54:44.499793+0000","last_undegraded":"2024-04-25T08:54:44.499793+0000","last_fullsized":"2024-04-25T08:54:44.499793+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:18.424887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.458849+0000","last_change":"2024-04-25T08:54:43.486049+0000","last_active":"2024-04-25T08:54:44.458849+0000","last_peered":"2024-04-25T08:54:44.458849+0000","last_clean":"2024-04-25T08:54:44.458849+0000","last_became_active":"2024-04-25T08:54:43.485336+0000","last_became_peered":"2024-04-25T08:54:43.485336+0000","last_unstale":"2024-04-25T08:54:44.458849+0000","last_undegraded":"2024-04-25T08:54:44.458849+0000","last_fullsized":"2024-04-25T08:54:44.458849+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:36.982333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.458521+0000","last_change":"2024-04-25T08:54:43.491488+0000","last_active":"2024-04-25T08:54:44.458521+0000","last_peered":"2024-04-25T08:54:44.458521+0000","last_clean":"2024-04-25T08:54:44.458521+0000","last_became_active":"2024-04-25T08:54:43.490757+0000","last_became_peered":"2024-04-25T08:54:43.490757+0000","last_unstale":"2024-04-25T08:54:44.458521+0000","last_undegraded":"2024-04-25T08:54:44.458521+0000","last_fullsized":"2024-04-25T08:54:44.458521+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:59.576273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456245+0000","last_change":"2024-04-25T08:54:43.488243+0000","last_active":"2024-04-25T08:54:44.456245+0000","last_peered":"2024-04-25T08:54:44.456245+0000","last_clean":"2024-04-25T08:54:44.456245+0000","last_became_active":"2024-04-25T08:54:43.487903+0000","last_became_peered":"2024-04-25T08:54:43.487903+0000","last_unstale":"2024-04-25T08:54:44.456245+0000","last_undegraded":"2024-04-25T08:54:44.456245+0000","last_fullsized":"2024-04-25T08:54:44.456245+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:53.608712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.511791+0000","last_change":"2024-04-25T08:54:43.489579+0000","last_active":"2024-04-25T08:54:44.511791+0000","last_peered":"2024-04-25T08:54:44.511791+0000","last_clean":"2024-04-25T08:54:44.511791+0000","last_became_active":"2024-04-25T08:54:43.489090+0000","last_became_peered":"2024-04-25T08:54:43.489090+0000","last_unstale":"2024-04-25T08:54:44.511791+0000","last_undegraded":"2024-04-25T08:54:44.511791+0000","last_fullsized":"2024-04-25T08:54:44.511791+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:35.163452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.498963+0000","last_change":"2024-04-25T08:54:43.501378+0000","last_active":"2024-04-25T08:54:44.498963+0000","last_peered":"2024-04-25T08:54:44.498963+0000","last_clean":"2024-04-25T08:54:44.498963+0000","last_became_active":"2024-04-25T08:54:43.501076+0000","last_became_peered":"2024-04-25T08:54:43.501076+0000","last_unstale":"2024-04-25T08:54:44.498963+0000","last_undegraded":"2024-04-25T08:54:44.498963+0000","last_fullsized":"2024-04-25T08:54:44.498963+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:36.986078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.500114+0000","last_change":"2024-04-25T08:54:43.492614+0000","last_active":"2024-04-25T08:54:44.500114+0000","last_peered":"2024-04-25T08:54:44.500114+0000","last_clean":"2024-04-25T08:54:44.500114+0000","last_became_active":"2024-04-25T08:54:43.491461+0000","last_became_peered":"2024-04-25T08:54:43.491461+0000","last_unstale":"2024-04-25T08:54:44.500114+0000","last_undegraded":"2024-04-25T08:54:44.500114+0000","last_fullsized":"2024-04-25T08:54:44.500114+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:10.772322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456676+0000","last_change":"2024-04-25T08:54:43.491768+0000","last_active":"2024-04-25T08:54:44.456676+0000","last_peered":"2024-04-25T08:54:44.456676+0000","last_clean":"2024-04-25T08:54:44.456676+0000","last_became_active":"2024-04-25T08:54:43.491502+0000","last_became_peered":"2024-04-25T08:54:43.491502+0000","last_unstale":"2024-04-25T08:54:44.456676+0000","last_undegraded":"2024-04-25T08:54:44.456676+0000","last_fullsized":"2024-04-25T08:54:44.456676+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:37.647093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.546916+0000","last_change":"2024-04-25T08:54:43.496338+0000","last_active":"2024-04-25T08:54:45.546916+0000","last_peered":"2024-04-25T08:54:45.546916+0000","last_clean":"2024-04-25T08:54:45.546916+0000","last_became_active":"2024-04-25T08:54:43.496002+0000","last_became_peered":"2024-04-25T08:54:43.496002+0000","last_unstale":"2024-04-25T08:54:45.546916+0000","last_undegraded":"2024-04-25T08:54:45.546916+0000","last_fullsized":"2024-04-25T08:54:45.546916+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:51.285978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.459143+0000","last_change":"2024-04-25T08:54:43.488758+0000","last_active":"2024-04-25T08:54:44.459143+0000","last_peered":"2024-04-25T08:54:44.459143+0000","last_clean":"2024-04-25T08:54:44.459143+0000","last_became_active":"2024-04-25T08:54:43.488023+0000","last_became_peered":"2024-04-25T08:54:43.488023+0000","last_unstale":"2024-04-25T08:54:44.459143+0000","last_undegraded":"2024-04-25T08:54:44.459143+0000","last_fullsized":"2024-04-25T08:54:44.459143+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:57.710497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499215+0000","last_change":"2024-04-25T08:54:43.483859+0000","last_active":"2024-04-25T08:54:44.499215+0000","last_peered":"2024-04-25T08:54:44.499215+0000","last_clean":"2024-04-25T08:54:44.499215+0000","last_became_active":"2024-04-25T08:54:43.477859+0000","last_became_peered":"2024-04-25T08:54:43.477859+0000","last_unstale":"2024-04-25T08:54:44.499215+0000","last_undegraded":"2024-04-25T08:54:44.499215+0000","last_fullsized":"2024-04-25T08:54:44.499215+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:32.421589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.503084+0000","last_change":"2024-04-25T08:54:43.476418+0000","last_active":"2024-04-25T08:54:44.503084+0000","last_peered":"2024-04-25T08:54:44.503084+0000","last_clean":"2024-04-25T08:54:44.503084+0000","last_became_active":"2024-04-25T08:54:43.470486+0000","last_became_peered":"2024-04-25T08:54:43.470486+0000","last_unstale":"2024-04-25T08:54:44.503084+0000","last_undegraded":"2024-04-25T08:54:44.503084+0000","last_fullsized":"2024-04-25T08:54:44.503084+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:16.843792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499101+0000","last_change":"2024-04-25T08:54:43.496799+0000","last_active":"2024-04-25T08:54:44.499101+0000","last_peered":"2024-04-25T08:54:44.499101+0000","last_clean":"2024-04-25T08:54:44.499101+0000","last_became_active":"2024-04-25T08:54:43.494788+0000","last_became_peered":"2024-04-25T08:54:43.494788+0000","last_unstale":"2024-04-25T08:54:44.499101+0000","last_undegraded":"2024-04-25T08:54:44.499101+0000","last_fullsized":"2024-04-25T08:54:44.499101+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:39.305543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456877+0000","last_change":"2024-04-25T08:54:43.496405+0000","last_active":"2024-04-25T08:54:44.456877+0000","last_peered":"2024-04-25T08:54:44.456877+0000","last_clean":"2024-04-25T08:54:44.456877+0000","last_became_active":"2024-04-25T08:54:43.495103+0000","last_became_peered":"2024-04-25T08:54:43.495103+0000","last_unstale":"2024-04-25T08:54:44.456877+0000","last_undegraded":"2024-04-25T08:54:44.456877+0000","last_fullsized":"2024-04-25T08:54:44.456877+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:51.285978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.502514+0000","last_change":"2024-04-25T08:54:43.486789+0000","last_active":"2024-04-25T08:54:44.502514+0000","last_peered":"2024-04-25T08:54:44.502514+0000","last_clean":"2024-04-25T08:54:44.502514+0000","last_became_active":"2024-04-25T08:54:43.477490+0000","last_became_peered":"2024-04-25T08:54:43.477490+0000","last_unstale":"2024-04-25T08:54:44.502514+0000","last_undegraded":"2024-04-25T08:54:44.502514+0000","last_fullsized":"2024-04-25T08:54:44.502514+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:57.710497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"54'4","reported_seq":18,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.459131+0000","last_change":"2024-04-25T08:54:42.435894+0000","last_active":"2024-04-25T08:54:44.459131+0000","last_peered":"2024-04-25T08:54:44.459131+0000","last_clean":"2024-04-25T08:54:44.459131+0000","last_became_active":"2024-04-25T08:54:42.435139+0000","last_became_peered":"2024-04-25T08:54:42.435139+0000","last_unstale":"2024-04-25T08:54:44.459131+0000","last_undegraded":"2024-04-25T08:54:44.459131+0000","last_fullsized":"2024-04-25T08:54:44.459131+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:54.372043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457137+0000","last_change":"2024-04-25T08:54:43.494512+0000","last_active":"2024-04-25T08:54:44.457137+0000","last_peered":"2024-04-25T08:54:44.457137+0000","last_clean":"2024-04-25T08:54:44.457137+0000","last_became_active":"2024-04-25T08:54:43.492275+0000","last_became_peered":"2024-04-25T08:54:43.492275+0000","last_unstale":"2024-04-25T08:54:44.457137+0000","last_undegraded":"2024-04-25T08:54:44.457137+0000","last_fullsized":"2024-04-25T08:54:44.457137+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:01.786776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.546828+0000","last_change":"2024-04-25T08:54:42.434433+0000","last_active":"2024-04-25T08:54:45.546828+0000","last_peered":"2024-04-25T08:54:45.546828+0000","last_clean":"2024-04-25T08:54:45.546828+0000","last_became_active":"2024-04-25T08:54:42.434128+0000","last_became_peered":"2024-04-25T08:54:42.434128+0000","last_unstale":"2024-04-25T08:54:45.546828+0000","last_undegraded":"2024-04-25T08:54:45.546828+0000","last_fullsized":"2024-04-25T08:54:45.546828+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:34.520291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457673+0000","last_change":"2024-04-25T08:54:43.480424+0000","last_active":"2024-04-25T08:54:44.457673+0000","last_peered":"2024-04-25T08:54:44.457673+0000","last_clean":"2024-04-25T08:54:44.457673+0000","last_became_active":"2024-04-25T08:54:43.479623+0000","last_became_peered":"2024-04-25T08:54:43.479623+0000","last_unstale":"2024-04-25T08:54:44.457673+0000","last_undegraded":"2024-04-25T08:54:44.457673+0000","last_fullsized":"2024-04-25T08:54:44.457673+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:01.634890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.547129+0000","last_change":"2024-04-25T08:54:42.455613+0000","last_active":"2024-04-25T08:54:45.547129+0000","last_peered":"2024-04-25T08:54:45.547129+0000","last_clean":"2024-04-25T08:54:45.547129+0000","last_became_active":"2024-04-25T08:54:42.450954+0000","last_became_peered":"2024-04-25T08:54:42.450954+0000","last_unstale":"2024-04-25T08:54:45.547129+0000","last_undegraded":"2024-04-25T08:54:45.547129+0000","last_fullsized":"2024-04-25T08:54:45.547129+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456174+0000","last_change":"2024-04-25T08:54:43.490202+0000","last_active":"2024-04-25T08:54:44.456174+0000","last_peered":"2024-04-25T08:54:44.456174+0000","last_clean":"2024-04-25T08:54:44.456174+0000","last_became_active":"2024-04-25T08:54:43.489917+0000","last_became_peered":"2024-04-25T08:54:43.489917+0000","last_unstale":"2024-04-25T08:54:44.456174+0000","last_undegraded":"2024-04-25T08:54:44.456174+0000","last_fullsized":"2024-04-25T08:54:44.456174+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:38.646066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.458231+0000","last_change":"2024-04-25T08:54:42.461742+0000","last_active":"2024-04-25T08:54:44.458231+0000","last_peered":"2024-04-25T08:54:44.458231+0000","last_clean":"2024-04-25T08:54:44.458231+0000","last_became_active":"2024-04-25T08:54:42.460925+0000","last_became_peered":"2024-04-25T08:54:42.460925+0000","last_unstale":"2024-04-25T08:54:44.458231+0000","last_undegraded":"2024-04-25T08:54:44.458231+0000","last_fullsized":"2024-04-25T08:54:44.458231+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:00.626472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.756498+0000","last_change":"2024-04-25T08:54:43.472816+0000","last_active":"2024-04-25T08:54:44.756498+0000","last_peered":"2024-04-25T08:54:44.756498+0000","last_clean":"2024-04-25T08:54:44.756498+0000","last_became_active":"2024-04-25T08:54:43.471121+0000","last_became_peered":"2024-04-25T08:54:43.471121+0000","last_unstale":"2024-04-25T08:54:44.756498+0000","last_undegraded":"2024-04-25T08:54:44.756498+0000","last_fullsized":"2024-04-25T08:54:44.756498+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:52.327042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499353+0000","last_change":"2024-04-25T08:54:42.468263+0000","last_active":"2024-04-25T08:54:44.499353+0000","last_peered":"2024-04-25T08:54:44.499353+0000","last_clean":"2024-04-25T08:54:44.499353+0000","last_became_active":"2024-04-25T08:54:42.467914+0000","last_became_peered":"2024-04-25T08:54:42.467914+0000","last_unstale":"2024-04-25T08:54:44.499353+0000","last_undegraded":"2024-04-25T08:54:44.499353+0000","last_fullsized":"2024-04-25T08:54:44.499353+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:44.370399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.503154+0000","last_change":"2024-04-25T08:54:43.476671+0000","last_active":"2024-04-25T08:54:44.503154+0000","last_peered":"2024-04-25T08:54:44.503154+0000","last_clean":"2024-04-25T08:54:44.503154+0000","last_became_active":"2024-04-25T08:54:43.475769+0000","last_became_peered":"2024-04-25T08:54:43.475769+0000","last_unstale":"2024-04-25T08:54:44.503154+0000","last_undegraded":"2024-04-25T08:54:44.503154+0000","last_fullsized":"2024-04-25T08:54:44.503154+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:09.293421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.755589+0000","last_change":"2024-04-25T08:54:42.439471+0000","last_active":"2024-04-25T08:54:44.755589+0000","last_peered":"2024-04-25T08:54:44.755589+0000","last_clean":"2024-04-25T08:54:44.755589+0000","last_became_active":"2024-04-25T08:54:42.438730+0000","last_became_peered":"2024-04-25T08:54:42.438730+0000","last_unstale":"2024-04-25T08:54:44.755589+0000","last_undegraded":"2024-04-25T08:54:44.755589+0000","last_fullsized":"2024-04-25T08:54:44.755589+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:08.427686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457189+0000","last_change":"2024-04-25T08:54:43.490846+0000","last_active":"2024-04-25T08:54:44.457189+0000","last_peered":"2024-04-25T08:54:44.457189+0000","last_clean":"2024-04-25T08:54:44.457189+0000","last_became_active":"2024-04-25T08:54:43.490564+0000","last_became_peered":"2024-04-25T08:54:43.490564+0000","last_unstale":"2024-04-25T08:54:44.457189+0000","last_undegraded":"2024-04-25T08:54:44.457189+0000","last_fullsized":"2024-04-25T08:54:44.457189+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:21.076830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457553+0000","last_change":"2024-04-25T08:54:42.450407+0000","last_active":"2024-04-25T08:54:44.457553+0000","last_peered":"2024-04-25T08:54:44.457553+0000","last_clean":"2024-04-25T08:54:44.457553+0000","last_became_active":"2024-04-25T08:54:42.448040+0000","last_became_peered":"2024-04-25T08:54:42.448040+0000","last_unstale":"2024-04-25T08:54:44.457553+0000","last_undegraded":"2024-04-25T08:54:44.457553+0000","last_fullsized":"2024-04-25T08:54:44.457553+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:40.073899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.503650+0000","last_change":"2024-04-25T08:54:43.501023+0000","last_active":"2024-04-25T08:54:44.503650+0000","last_peered":"2024-04-25T08:54:44.503650+0000","last_clean":"2024-04-25T08:54:44.503650+0000","last_became_active":"2024-04-25T08:54:43.500148+0000","last_became_peered":"2024-04-25T08:54:43.500148+0000","last_unstale":"2024-04-25T08:54:44.503650+0000","last_undegraded":"2024-04-25T08:54:44.503650+0000","last_fullsized":"2024-04-25T08:54:44.503650+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:23.376794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.498988+0000","last_change":"2024-04-25T08:54:42.466835+0000","last_active":"2024-04-25T08:54:44.498988+0000","last_peered":"2024-04-25T08:54:44.498988+0000","last_clean":"2024-04-25T08:54:44.498988+0000","last_became_active":"2024-04-25T08:54:42.466542+0000","last_became_peered":"2024-04-25T08:54:42.466542+0000","last_unstale":"2024-04-25T08:54:44.498988+0000","last_undegraded":"2024-04-25T08:54:44.498988+0000","last_fullsized":"2024-04-25T08:54:44.498988+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:58.567855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.755679+0000","last_change":"2024-04-25T08:54:43.495832+0000","last_active":"2024-04-25T08:54:44.755679+0000","last_peered":"2024-04-25T08:54:44.755679+0000","last_clean":"2024-04-25T08:54:44.755679+0000","last_became_active":"2024-04-25T08:54:43.495114+0000","last_became_peered":"2024-04-25T08:54:43.495114+0000","last_unstale":"2024-04-25T08:54:44.755679+0000","last_undegraded":"2024-04-25T08:54:44.755679+0000","last_fullsized":"2024-04-25T08:54:44.755679+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:09.545613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.503711+0000","last_change":"2024-04-25T08:54:42.458215+0000","last_active":"2024-04-25T08:54:44.503711+0000","last_peered":"2024-04-25T08:54:44.503711+0000","last_clean":"2024-04-25T08:54:44.503711+0000","last_became_active":"2024-04-25T08:54:42.457493+0000","last_became_peered":"2024-04-25T08:54:42.457493+0000","last_unstale":"2024-04-25T08:54:44.503711+0000","last_undegraded":"2024-04-25T08:54:44.503711+0000","last_fullsized":"2024-04-25T08:54:44.503711+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:46.278167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456735+0000","last_change":"2024-04-25T08:54:43.487550+0000","last_active":"2024-04-25T08:54:44.456735+0000","last_peered":"2024-04-25T08:54:44.456735+0000","last_clean":"2024-04-25T08:54:44.456735+0000","last_became_active":"2024-04-25T08:54:43.486713+0000","last_became_peered":"2024-04-25T08:54:43.486713+0000","last_unstale":"2024-04-25T08:54:44.456735+0000","last_undegraded":"2024-04-25T08:54:44.456735+0000","last_fullsized":"2024-04-25T08:54:44.456735+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:23.376794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.756239+0000","last_change":"2024-04-25T08:54:42.442179+0000","last_active":"2024-04-25T08:54:44.756239+0000","last_peered":"2024-04-25T08:54:44.756239+0000","last_clean":"2024-04-25T08:54:44.756239+0000","last_became_active":"2024-04-25T08:54:42.441094+0000","last_became_peered":"2024-04-25T08:54:42.441094+0000","last_unstale":"2024-04-25T08:54:44.756239+0000","last_undegraded":"2024-04-25T08:54:44.756239+0000","last_fullsized":"2024-04-25T08:54:44.756239+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:34.520291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499166+0000","last_change":"2024-04-25T08:54:43.474219+0000","last_active":"2024-04-25T08:54:44.499166+0000","last_peered":"2024-04-25T08:54:44.499166+0000","last_clean":"2024-04-25T08:54:44.499166+0000","last_became_active":"2024-04-25T08:54:43.473088+0000","last_became_peered":"2024-04-25T08:54:43.473088+0000","last_unstale":"2024-04-25T08:54:44.499166+0000","last_undegraded":"2024-04-25T08:54:44.499166+0000","last_fullsized":"2024-04-25T08:54:44.499166+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:35.655711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"54'3","reported_seq":17,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.503434+0000","last_change":"2024-04-25T08:54:42.474076+0000","last_active":"2024-04-25T08:54:44.503434+0000","last_peered":"2024-04-25T08:54:44.503434+0000","last_clean":"2024-04-25T08:54:44.503434+0000","last_became_active":"2024-04-25T08:54:42.473614+0000","last_became_peered":"2024-04-25T08:54:42.473614+0000","last_unstale":"2024-04-25T08:54:44.503434+0000","last_undegraded":"2024-04-25T08:54:44.503434+0000","last_fullsized":"2024-04-25T08:54:44.503434+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:08.427686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.756209+0000","last_change":"2024-04-25T08:54:43.489530+0000","last_active":"2024-04-25T08:54:44.756209+0000","last_peered":"2024-04-25T08:54:44.756209+0000","last_clean":"2024-04-25T08:54:44.756209+0000","last_became_active":"2024-04-25T08:54:43.488804+0000","last_became_peered":"2024-04-25T08:54:43.488804+0000","last_unstale":"2024-04-25T08:54:44.756209+0000","last_undegraded":"2024-04-25T08:54:44.756209+0000","last_fullsized":"2024-04-25T08:54:44.756209+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:57.710497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"54'6","reported_seq":20,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457371+0000","last_change":"2024-04-25T08:54:42.461363+0000","last_active":"2024-04-25T08:54:44.457371+0000","last_peered":"2024-04-25T08:54:44.457371+0000","last_clean":"2024-04-25T08:54:44.457371+0000","last_became_active":"2024-04-25T08:54:42.460631+0000","last_became_peered":"2024-04-25T08:54:42.460631+0000","last_unstale":"2024-04-25T08:54:44.457371+0000","last_undegraded":"2024-04-25T08:54:44.457371+0000","last_fullsized":"2024-04-25T08:54:44.457371+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:21.776121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456687+0000","last_change":"2024-04-25T08:54:43.481651+0000","last_active":"2024-04-25T08:54:44.456687+0000","last_peered":"2024-04-25T08:54:44.456687+0000","last_clean":"2024-04-25T08:54:44.456687+0000","last_became_active":"2024-04-25T08:54:43.480995+0000","last_became_peered":"2024-04-25T08:54:43.480995+0000","last_unstale":"2024-04-25T08:54:44.456687+0000","last_undegraded":"2024-04-25T08:54:44.456687+0000","last_fullsized":"2024-04-25T08:54:44.456687+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:23.368572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499330+0000","last_change":"2024-04-25T08:54:42.456578+0000","last_active":"2024-04-25T08:54:44.499330+0000","last_peered":"2024-04-25T08:54:44.499330+0000","last_clean":"2024-04-25T08:54:44.499330+0000","last_became_active":"2024-04-25T08:54:42.456167+0000","last_became_peered":"2024-04-25T08:54:42.456167+0000","last_unstale":"2024-04-25T08:54:44.499330+0000","last_undegraded":"2024-04-25T08:54:44.499330+0000","last_fullsized":"2024-04-25T08:54:44.499330+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:25.002800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499942+0000","last_change":"2024-04-25T08:54:43.490642+0000","last_active":"2024-04-25T08:54:44.499942+0000","last_peered":"2024-04-25T08:54:44.499942+0000","last_clean":"2024-04-25T08:54:44.499942+0000","last_became_active":"2024-04-25T08:54:43.488221+0000","last_became_peered":"2024-04-25T08:54:43.488221+0000","last_unstale":"2024-04-25T08:54:44.499942+0000","last_undegraded":"2024-04-25T08:54:44.499942+0000","last_fullsized":"2024-04-25T08:54:44.499942+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:04:37.647093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457911+0000","last_change":"2024-04-25T08:54:42.451116+0000","last_active":"2024-04-25T08:54:44.457911+0000","last_peered":"2024-04-25T08:54:44.457911+0000","last_clean":"2024-04-25T08:54:44.457911+0000","last_became_active":"2024-04-25T08:54:42.447149+0000","last_became_peered":"2024-04-25T08:54:42.447149+0000","last_unstale":"2024-04-25T08:54:44.457911+0000","last_undegraded":"2024-04-25T08:54:44.457911+0000","last_fullsized":"2024-04-25T08:54:44.457911+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:12.469464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457002+0000","last_change":"2024-04-25T08:54:43.493689+0000","last_active":"2024-04-25T08:54:44.457002+0000","last_peered":"2024-04-25T08:54:44.457002+0000","last_clean":"2024-04-25T08:54:44.457002+0000","last_became_active":"2024-04-25T08:54:43.493375+0000","last_became_peered":"2024-04-25T08:54:43.493375+0000","last_unstale":"2024-04-25T08:54:44.457002+0000","last_undegraded":"2024-04-25T08:54:44.457002+0000","last_fullsized":"2024-04-25T08:54:44.457002+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:32.216763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.502978+0000","last_change":"2024-04-25T08:54:42.458531+0000","last_active":"2024-04-25T08:54:44.502978+0000","last_peered":"2024-04-25T08:54:44.502978+0000","last_clean":"2024-04-25T08:54:44.502978+0000","last_became_active":"2024-04-25T08:54:42.457748+0000","last_became_peered":"2024-04-25T08:54:42.457748+0000","last_unstale":"2024-04-25T08:54:44.502978+0000","last_undegraded":"2024-04-25T08:54:44.502978+0000","last_fullsized":"2024-04-25T08:54:44.502978+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:34.520291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499963+0000","last_change":"2024-04-25T08:54:43.495177+0000","last_active":"2024-04-25T08:54:44.499963+0000","last_peered":"2024-04-25T08:54:44.499963+0000","last_clean":"2024-04-25T08:54:44.499963+0000","last_became_active":"2024-04-25T08:54:43.492311+0000","last_became_peered":"2024-04-25T08:54:43.492311+0000","last_unstale":"2024-04-25T08:54:44.499963+0000","last_undegraded":"2024-04-25T08:54:44.499963+0000","last_fullsized":"2024-04-25T08:54:44.499963+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:53.608712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"54'8","reported_seq":22,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456635+0000","last_change":"2024-04-25T08:54:42.466236+0000","last_active":"2024-04-25T08:54:44.456635+0000","last_peered":"2024-04-25T08:54:44.456635+0000","last_clean":"2024-04-25T08:54:44.456635+0000","last_became_active":"2024-04-25T08:54:42.465507+0000","last_became_peered":"2024-04-25T08:54:42.465507+0000","last_unstale":"2024-04-25T08:54:44.456635+0000","last_undegraded":"2024-04-25T08:54:44.456635+0000","last_fullsized":"2024-04-25T08:54:44.456635+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:48.674667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456101+0000","last_change":"2024-04-25T08:54:43.495335+0000","last_active":"2024-04-25T08:54:44.456101+0000","last_peered":"2024-04-25T08:54:44.456101+0000","last_clean":"2024-04-25T08:54:44.456101+0000","last_became_active":"2024-04-25T08:54:43.495017+0000","last_became_peered":"2024-04-25T08:54:43.495017+0000","last_unstale":"2024-04-25T08:54:44.456101+0000","last_undegraded":"2024-04-25T08:54:44.456101+0000","last_fullsized":"2024-04-25T08:54:44.456101+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:18.424887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.500145+0000","last_change":"2024-04-25T08:54:42.469518+0000","last_active":"2024-04-25T08:54:44.500145+0000","last_peered":"2024-04-25T08:54:44.500145+0000","last_clean":"2024-04-25T08:54:44.500145+0000","last_became_active":"2024-04-25T08:54:42.469147+0000","last_became_peered":"2024-04-25T08:54:42.469147+0000","last_unstale":"2024-04-25T08:54:44.500145+0000","last_undegraded":"2024-04-25T08:54:44.500145+0000","last_fullsized":"2024-04-25T08:54:44.500145+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:08.285003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.546385+0000","last_change":"2024-04-25T08:54:43.472179+0000","last_active":"2024-04-25T08:54:45.546385+0000","last_peered":"2024-04-25T08:54:45.546385+0000","last_clean":"2024-04-25T08:54:45.546385+0000","last_became_active":"2024-04-25T08:54:43.471399+0000","last_became_peered":"2024-04-25T08:54:43.471399+0000","last_unstale":"2024-04-25T08:54:45.546385+0000","last_undegraded":"2024-04-25T08:54:45.546385+0000","last_fullsized":"2024-04-25T08:54:45.546385+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:52.327042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.512093+0000","last_change":"2024-04-25T08:54:42.450583+0000","last_active":"2024-04-25T08:54:44.512093+0000","last_peered":"2024-04-25T08:54:44.512093+0000","last_clean":"2024-04-25T08:54:44.512093+0000","last_became_active":"2024-04-25T08:54:42.450316+0000","last_became_peered":"2024-04-25T08:54:42.450316+0000","last_unstale":"2024-04-25T08:54:44.512093+0000","last_undegraded":"2024-04-25T08:54:44.512093+0000","last_fullsized":"2024-04-25T08:54:44.512093+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:05.571041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.755741+0000","last_change":"2024-04-25T08:54:43.471630+0000","last_active":"2024-04-25T08:54:44.755741+0000","last_peered":"2024-04-25T08:54:44.755741+0000","last_clean":"2024-04-25T08:54:44.755741+0000","last_became_active":"2024-04-25T08:54:43.470871+0000","last_became_peered":"2024-04-25T08:54:43.470871+0000","last_unstale":"2024-04-25T08:54:44.755741+0000","last_undegraded":"2024-04-25T08:54:44.755741+0000","last_fullsized":"2024-04-25T08:54:44.755741+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:59.576273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"54'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.458189+0000","last_change":"2024-04-25T08:54:42.444423+0000","last_active":"2024-04-25T08:54:44.458189+0000","last_peered":"2024-04-25T08:54:44.458189+0000","last_clean":"2024-04-25T08:54:44.458189+0000","last_became_active":"2024-04-25T08:54:42.439289+0000","last_became_peered":"2024-04-25T08:54:42.439289+0000","last_unstale":"2024-04-25T08:54:44.458189+0000","last_undegraded":"2024-04-25T08:54:44.458189+0000","last_fullsized":"2024-04-25T08:54:44.458189+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:34.520291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.546588+0000","last_change":"2024-04-25T08:54:43.474312+0000","last_active":"2024-04-25T08:54:45.546588+0000","last_peered":"2024-04-25T08:54:45.546588+0000","last_clean":"2024-04-25T08:54:45.546588+0000","last_became_active":"2024-04-25T08:54:43.473574+0000","last_became_peered":"2024-04-25T08:54:43.473574+0000","last_unstale":"2024-04-25T08:54:45.546588+0000","last_undegraded":"2024-04-25T08:54:45.546588+0000","last_fullsized":"2024-04-25T08:54:45.546588+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:19.413559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.459149+0000","last_change":"2024-04-25T08:54:42.436282+0000","last_active":"2024-04-25T08:54:44.459149+0000","last_peered":"2024-04-25T08:54:44.459149+0000","last_clean":"2024-04-25T08:54:44.459149+0000","last_became_active":"2024-04-25T08:54:42.435494+0000","last_became_peered":"2024-04-25T08:54:42.435494+0000","last_unstale":"2024-04-25T08:54:44.459149+0000","last_undegraded":"2024-04-25T08:54:44.459149+0000","last_fullsized":"2024-04-25T08:54:44.459149+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:28.707217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.458251+0000","last_change":"2024-04-25T08:54:43.496137+0000","last_active":"2024-04-25T08:54:44.458251+0000","last_peered":"2024-04-25T08:54:44.458251+0000","last_clean":"2024-04-25T08:54:44.458251+0000","last_became_active":"2024-04-25T08:54:43.495139+0000","last_became_peered":"2024-04-25T08:54:43.495139+0000","last_unstale":"2024-04-25T08:54:44.458251+0000","last_undegraded":"2024-04-25T08:54:44.458251+0000","last_fullsized":"2024-04-25T08:54:44.458251+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:05.788587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.546726+0000","last_change":"2024-04-25T08:54:42.456188+0000","last_active":"2024-04-25T08:54:45.546726+0000","last_peered":"2024-04-25T08:54:45.546726+0000","last_clean":"2024-04-25T08:54:45.546726+0000","last_became_active":"2024-04-25T08:54:42.455422+0000","last_became_peered":"2024-04-25T08:54:42.455422+0000","last_unstale":"2024-04-25T08:54:45.546726+0000","last_undegraded":"2024-04-25T08:54:45.546726+0000","last_fullsized":"2024-04-25T08:54:45.546726+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:46.278167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457800+0000","last_change":"2024-04-25T08:54:43.485952+0000","last_active":"2024-04-25T08:54:44.457800+0000","last_peered":"2024-04-25T08:54:44.457800+0000","last_clean":"2024-04-25T08:54:44.457800+0000","last_became_active":"2024-04-25T08:54:43.485257+0000","last_became_peered":"2024-04-25T08:54:43.485257+0000","last_unstale":"2024-04-25T08:54:44.457800+0000","last_undegraded":"2024-04-25T08:54:44.457800+0000","last_fullsized":"2024-04-25T08:54:44.457800+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:09.545613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456528+0000","last_change":"2024-04-25T08:54:42.463454+0000","last_active":"2024-04-25T08:54:44.456528+0000","last_peered":"2024-04-25T08:54:44.456528+0000","last_clean":"2024-04-25T08:54:44.456528+0000","last_became_active":"2024-04-25T08:54:42.461962+0000","last_became_peered":"2024-04-25T08:54:42.461962+0000","last_unstale":"2024-04-25T08:54:44.456528+0000","last_undegraded":"2024-04-25T08:54:44.456528+0000","last_fullsized":"2024-04-25T08:54:44.456528+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:50.277560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499860+0000","last_change":"2024-04-25T08:54:43.475868+0000","last_active":"2024-04-25T08:54:44.499860+0000","last_peered":"2024-04-25T08:54:44.499860+0000","last_clean":"2024-04-25T08:54:44.499860+0000","last_became_active":"2024-04-25T08:54:43.475132+0000","last_became_peered":"2024-04-25T08:54:43.475132+0000","last_unstale":"2024-04-25T08:54:44.499860+0000","last_undegraded":"2024-04-25T08:54:44.499860+0000","last_fullsized":"2024-04-25T08:54:44.499860+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:38.646066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499172+0000","last_change":"2024-04-25T08:54:42.456453+0000","last_active":"2024-04-25T08:54:44.499172+0000","last_peered":"2024-04-25T08:54:44.499172+0000","last_clean":"2024-04-25T08:54:44.499172+0000","last_became_active":"2024-04-25T08:54:42.456124+0000","last_became_peered":"2024-04-25T08:54:42.456124+0000","last_unstale":"2024-04-25T08:54:44.499172+0000","last_undegraded":"2024-04-25T08:54:44.499172+0000","last_fullsized":"2024-04-25T08:54:44.499172+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:57.149980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.547191+0000","last_change":"2024-04-25T08:54:43.481559+0000","last_active":"2024-04-25T08:54:45.547191+0000","last_peered":"2024-04-25T08:54:45.547191+0000","last_clean":"2024-04-25T08:54:45.547191+0000","last_became_active":"2024-04-25T08:54:43.480671+0000","last_became_peered":"2024-04-25T08:54:43.480671+0000","last_unstale":"2024-04-25T08:54:45.547191+0000","last_undegraded":"2024-04-25T08:54:45.547191+0000","last_fullsized":"2024-04-25T08:54:45.547191+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:01.786776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.511595+0000","last_change":"2024-04-25T08:54:42.448496+0000","last_active":"2024-04-25T08:54:44.511595+0000","last_peered":"2024-04-25T08:54:44.511595+0000","last_clean":"2024-04-25T08:54:44.511595+0000","last_became_active":"2024-04-25T08:54:42.447727+0000","last_became_peered":"2024-04-25T08:54:42.447727+0000","last_unstale":"2024-04-25T08:54:44.511595+0000","last_undegraded":"2024-04-25T08:54:44.511595+0000","last_fullsized":"2024-04-25T08:54:44.511595+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:44.880737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457318+0000","last_change":"2024-04-25T08:54:43.487984+0000","last_active":"2024-04-25T08:54:44.457318+0000","last_peered":"2024-04-25T08:54:44.457318+0000","last_clean":"2024-04-25T08:54:44.457318+0000","last_became_active":"2024-04-25T08:54:43.487278+0000","last_became_peered":"2024-04-25T08:54:43.487278+0000","last_unstale":"2024-04-25T08:54:44.457318+0000","last_undegraded":"2024-04-25T08:54:44.457318+0000","last_fullsized":"2024-04-25T08:54:44.457318+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:56:22.784539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"54'2","reported_seq":16,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.459152+0000","last_change":"2024-04-25T08:54:42.449480+0000","last_active":"2024-04-25T08:54:44.459152+0000","last_peered":"2024-04-25T08:54:44.459152+0000","last_clean":"2024-04-25T08:54:44.459152+0000","last_became_active":"2024-04-25T08:54:42.448749+0000","last_became_peered":"2024-04-25T08:54:42.448749+0000","last_unstale":"2024-04-25T08:54:44.459152+0000","last_undegraded":"2024-04-25T08:54:44.459152+0000","last_fullsized":"2024-04-25T08:54:44.459152+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:37.578431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:45.546847+0000","last_change":"2024-04-25T08:54:43.468773+0000","last_active":"2024-04-25T08:54:45.546847+0000","last_peered":"2024-04-25T08:54:45.546847+0000","last_clean":"2024-04-25T08:54:45.546847+0000","last_became_active":"2024-04-25T08:54:43.466001+0000","last_became_peered":"2024-04-25T08:54:43.466001+0000","last_unstale":"2024-04-25T08:54:45.546847+0000","last_undegraded":"2024-04-25T08:54:45.546847+0000","last_fullsized":"2024-04-25T08:54:45.546847+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:23.368572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.503454+0000","last_change":"2024-04-25T08:54:42.455018+0000","last_active":"2024-04-25T08:54:44.503454+0000","last_peered":"2024-04-25T08:54:44.503454+0000","last_clean":"2024-04-25T08:54:44.503454+0000","last_became_active":"2024-04-25T08:54:42.454140+0000","last_became_peered":"2024-04-25T08:54:42.454140+0000","last_unstale":"2024-04-25T08:54:44.503454+0000","last_undegraded":"2024-04-25T08:54:44.503454+0000","last_fullsized":"2024-04-25T08:54:44.503454+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:44.662568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456895+0000","last_change":"2024-04-25T08:54:43.496327+0000","last_active":"2024-04-25T08:54:44.456895+0000","last_peered":"2024-04-25T08:54:44.456895+0000","last_clean":"2024-04-25T08:54:44.456895+0000","last_became_active":"2024-04-25T08:54:43.496039+0000","last_became_peered":"2024-04-25T08:54:43.496039+0000","last_unstale":"2024-04-25T08:54:44.456895+0000","last_undegraded":"2024-04-25T08:54:44.456895+0000","last_fullsized":"2024-04-25T08:54:44.456895+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:10.772322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499622+0000","last_change":"2024-04-25T08:54:42.464859+0000","last_active":"2024-04-25T08:54:44.499622+0000","last_peered":"2024-04-25T08:54:44.499622+0000","last_clean":"2024-04-25T08:54:44.499622+0000","last_became_active":"2024-04-25T08:54:42.457143+0000","last_became_peered":"2024-04-25T08:54:42.457143+0000","last_unstale":"2024-04-25T08:54:44.499622+0000","last_undegraded":"2024-04-25T08:54:44.499622+0000","last_fullsized":"2024-04-25T08:54:44.499622+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:00.626472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499269+0000","last_change":"2024-04-25T08:54:43.495888+0000","last_active":"2024-04-25T08:54:44.499269+0000","last_peered":"2024-04-25T08:54:44.499269+0000","last_clean":"2024-04-25T08:54:44.499269+0000","last_became_active":"2024-04-25T08:54:43.495571+0000","last_became_peered":"2024-04-25T08:54:43.495571+0000","last_unstale":"2024-04-25T08:54:44.499269+0000","last_undegraded":"2024-04-25T08:54:44.499269+0000","last_fullsized":"2024-04-25T08:54:44.499269+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:05.432008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456305+0000","last_change":"2024-04-25T08:54:42.463152+0000","last_active":"2024-04-25T08:54:44.456305+0000","last_peered":"2024-04-25T08:54:44.456305+0000","last_clean":"2024-04-25T08:54:44.456305+0000","last_became_active":"2024-04-25T08:54:42.462413+0000","last_became_peered":"2024-04-25T08:54:42.462413+0000","last_unstale":"2024-04-25T08:54:44.456305+0000","last_undegraded":"2024-04-25T08:54:44.456305+0000","last_fullsized":"2024-04-25T08:54:44.456305+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:04.780169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.458356+0000","last_change":"2024-04-25T08:54:43.494315+0000","last_active":"2024-04-25T08:54:44.458356+0000","last_peered":"2024-04-25T08:54:44.458356+0000","last_clean":"2024-04-25T08:54:44.458356+0000","last_became_active":"2024-04-25T08:54:43.493211+0000","last_became_peered":"2024-04-25T08:54:43.493211+0000","last_unstale":"2024-04-25T08:54:44.458356+0000","last_undegraded":"2024-04-25T08:54:44.458356+0000","last_fullsized":"2024-04-25T08:54:44.458356+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:16.525108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.456774+0000","last_change":"2024-04-25T08:54:42.463691+0000","last_active":"2024-04-25T08:54:44.456774+0000","last_peered":"2024-04-25T08:54:44.456774+0000","last_clean":"2024-04-25T08:54:44.456774+0000","last_became_active":"2024-04-25T08:54:42.462922+0000","last_became_peered":"2024-04-25T08:54:42.462922+0000","last_unstale":"2024-04-25T08:54:44.456774+0000","last_undegraded":"2024-04-25T08:54:44.456774+0000","last_fullsized":"2024-04-25T08:54:44.456774+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:00.778358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.499733+0000","last_change":"2024-04-25T08:54:43.491694+0000","last_active":"2024-04-25T08:54:44.499733+0000","last_peered":"2024-04-25T08:54:44.499733+0000","last_clean":"2024-04-25T08:54:44.499733+0000","last_became_active":"2024-04-25T08:54:43.491359+0000","last_became_peered":"2024-04-25T08:54:43.491359+0000","last_unstale":"2024-04-25T08:54:44.499733+0000","last_undegraded":"2024-04-25T08:54:44.499733+0000","last_fullsized":"2024-04-25T08:54:44.499733+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:35.679912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.755587+0000","last_change":"2024-04-25T08:54:42.441517+0000","last_active":"2024-04-25T08:54:44.755587+0000","last_peered":"2024-04-25T08:54:44.755587+0000","last_clean":"2024-04-25T08:54:44.755587+0000","last_became_active":"2024-04-25T08:54:42.439663+0000","last_became_peered":"2024-04-25T08:54:42.439663+0000","last_unstale":"2024-04-25T08:54:44.755587+0000","last_undegraded":"2024-04-25T08:54:44.755587+0000","last_fullsized":"2024-04-25T08:54:44.755587+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:46.278167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457549+0000","last_change":"2024-04-25T08:54:43.492399+0000","last_active":"2024-04-25T08:54:44.457549+0000","last_peered":"2024-04-25T08:54:44.457549+0000","last_clean":"2024-04-25T08:54:44.457549+0000","last_became_active":"2024-04-25T08:54:43.492120+0000","last_became_peered":"2024-04-25T08:54:43.492120+0000","last_unstale":"2024-04-25T08:54:44.457549+0000","last_undegraded":"2024-04-25T08:54:44.457549+0000","last_fullsized":"2024-04-25T08:54:44.457549+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:06.647912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457554+0000","last_change":"2024-04-25T08:54:42.456403+0000","last_active":"2024-04-25T08:54:44.457554+0000","last_peered":"2024-04-25T08:54:44.457554+0000","last_clean":"2024-04-25T08:54:44.457554+0000","last_became_active":"2024-04-25T08:54:42.455569+0000","last_became_peered":"2024-04-25T08:54:42.455569+0000","last_unstale":"2024-04-25T08:54:44.457554+0000","last_undegraded":"2024-04-25T08:54:44.457554+0000","last_fullsized":"2024-04-25T08:54:44.457554+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:08.537195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.457305+0000","last_change":"2024-04-25T08:54:43.484290+0000","last_active":"2024-04-25T08:54:44.457305+0000","last_peered":"2024-04-25T08:54:44.457305+0000","last_clean":"2024-04-25T08:54:44.457305+0000","last_became_active":"2024-04-25T08:54:43.483531+0000","last_became_peered":"2024-04-25T08:54:43.483531+0000","last_unstale":"2024-04-25T08:54:44.457305+0000","last_undegraded":"2024-04-25T08:54:44.457305+0000","last_fullsized":"2024-04-25T08:54:44.457305+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:42.412663+0000","last_clean_scrub_stamp":"2024-04-25T08:54:42.412663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:26.011217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"55'1","reported_seq":15,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.500050+0000","last_change":"2024-04-25T08:54:42.466348+0000","last_active":"2024-04-25T08:54:44.500050+0000","last_peered":"2024-04-25T08:54:44.500050+0000","last_clean":"2024-04-25T08:54:44.500050+0000","last_became_active":"2024-04-25T08:54:42.456331+0000","last_became_peered":"2024-04-25T08:54:42.456331+0000","last_unstale":"2024-04-25T08:54:44.500050+0000","last_undegraded":"2024-04-25T08:54:44.500050+0000","last_fullsized":"2024-04-25T08:54:44.500050+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:41.404245+0000","last_clean_scrub_stamp":"2024-04-25T08:54:41.404245+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:08.537195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":235,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:54:44.502537+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:54:44.502537+0000","last_peered":"2024-04-25T08:54:44.502537+0000","last_clean":"2024-04-25T08:54:44.502537+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:54:44.502537+0000","last_undegraded":"2024-04-25T08:54:44.502537+0000","last_fullsized":"2024-04-25T08:54:44.502537+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":885059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":295219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":294567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":659456,"data_stored":295149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":671744,"data_stored":296201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":667648,"data_stored":296179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":663552,"data_stored":294727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32408,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339432,"statfs":{"total":96636764160,"available":96603578368,"internally_reserved":0,"allocated":1257472,"data_stored":884991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:54:52.364 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:54:52.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:54:52.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.531+0000 7ffbbdc85640 1 Processor -- start 2024-04-25T08:54:52.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.531+0000 7ffbbdc85640 1 -- start start 2024-04-25T08:54:52.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.531+0000 7ffbbdc85640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80cf590 0x7ffbb815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.531+0000 7ffbbdc85640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb80ca4c0 con 0x7ffbb80cf970 2024-04-25T08:54:52.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.531+0000 7ffbbdc85640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb80cd690 con 0x7ffbb80ce170 2024-04-25T08:54:52.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.531+0000 7ffbbdc85640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80ca780 con 0x7ffbb80cf590 2024-04-25T08:54:52.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.531+0000 7ffbb6ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ffbb80cf970 0x7ffbb80c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44786/0 (socket says 172.21.15.37:44786) 2024-04-25T08:54:52.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.531+0000 7ffbb6ffd640 1 -- 172.21.15.37:0/511514887 learned_addr learned my addr 172.21.15.37:0/511514887 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.531+0000 7ffbb77fe640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80cf590 0x7ffbb815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404508077 0 0) 0x7ffbb80ca4c0 con 0x7ffbb80cf970 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba0003530 con 0x7ffbb80cf970 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1443490226 0 0) 0x7ffba0003530 con 0x7ffbb80cf970 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80cf590 msgr2=0x7ffbb815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80cf590 0x7ffbb815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 >> v1:172.21.15.203:6789/0 conn(0x7ffbb80ce170 legacy=0x7ffbb80ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb80cd970 con 0x7ffbb80cf970 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7ffbac001680 con 0x7ffbb80cf970 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 >> v1:172.21.15.37:6789/0 conn(0x7ffbb80cf970 legacy=0x7ffbb80c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffba0003a60 0x7ffba0003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80cf590 0x7ffba0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 0x7ffba00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba000bab0 con 0x7ffba0004d50 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba000de70 con 0x7ffba0003a60 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba000e070 con 0x7ffbb80cf590 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb7fff640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 0x7ffba00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb6ffd640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80cf590 0x7ffba0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb77fe640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffba0003a60 0x7ffba0003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb7fff640 1 -- 172.21.15.37:0/511514887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80cf590 msgr2=0x7ffba0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb7fff640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80cf590 0x7ffba0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb7fff640 1 -- 172.21.15.37:0/511514887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffba0003a60 msgr2=0x7ffba0003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb7fff640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffba0003a60 0x7ffba0003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb7fff640 1 -- 172.21.15.37:0/511514887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba0003530 con 0x7ffba0004d50 2024-04-25T08:54:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb7fff640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 0x7ffba00051d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0001e40 tx=0x7ffbb001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec54bc3bad05598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0001a90 con 0x7ffba0004d50 2024-04-25T08:54:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb0003b80 con 0x7ffba0004d50 2024-04-25T08:54:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb67fc640 1 -- 172.21.15.37:0/511514887 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0005080 con 0x7ffba0004d50 2024-04-25T08:54:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbbdc85640 1 -- 172.21.15.37:0/511514887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 msgr2=0x7ffba00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbbdc85640 1 --2- 172.21.15.37:0/511514887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 0x7ffba00051d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0001e40 tx=0x7ffbb001c760 comp rx=0 tx=0).stop 2024-04-25T08:54:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbb7fff640 1 -- 172.21.15.37:0/511514887 reap_dead start 2024-04-25T08:54:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbbdc85640 1 -- 172.21.15.37:0/511514887 shutdown_connections 2024-04-25T08:54:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbbdc85640 1 -- 172.21.15.37:0/511514887 >> 172.21.15.37:0/511514887 conn(0x7ffbb806ae60 msgr2=0x7ffbb806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.535+0000 7ffbbdc85640 1 -- 172.21.15.37:0/511514887 shutdown_connections 2024-04-25T08:54:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbbdc85640 1 -- 172.21.15.37:0/511514887 wait complete. 2024-04-25T08:54:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbbdc85640 1 Processor -- start 2024-04-25T08:54:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbbdc85640 1 -- start start 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbbdc85640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 0x7ffbb806d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbbdc85640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80ae770 0x7ffbb8098b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbbdc85640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb8157140 0x7ffbb81575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbbdc85640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80cdce0 con 0x7ffba0004d50 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbbdc85640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80cd440 con 0x7ffbb8157140 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbbdc85640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80ca0c0 con 0x7ffbb80ae770 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 0x7ffbb806d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 0x7ffbb806d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52378/0 (socket says 172.21.15.37:52378) 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb77fe640 1 -- 172.21.15.37:0/2526131892 learned_addr learned my addr 172.21.15.37:0/2526131892 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb7fff640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb8157140 0x7ffbb81575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb6ffd640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80ae770 0x7ffbb8098b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb77fe640 1 -- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80ae770 msgr2=0x7ffbb8098b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb77fe640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80ae770 0x7ffbb8098b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb77fe640 1 -- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb8157140 msgr2=0x7ffbb81575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb77fe640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb8157140 0x7ffbb81575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb77fe640 1 -- 172.21.15.37:0/2526131892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb815c560 con 0x7ffba0004d50 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb7fff640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb8157140 0x7ffbb81575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbb77fe640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 0x7ffbb806d160 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ffba8011640 tx=0x7ffba8011a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffb9ffff640 1 -- 172.21.15.37:0/2526131892 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8015870 con 0x7ffba0004d50 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffb9ffff640 1 -- 172.21.15.37:0/2526131892 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba8015a10 con 0x7ffba0004d50 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbbdc85640 1 -- 172.21.15.37:0/2526131892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb8158a80 con 0x7ffba0004d50 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffb9ffff640 1 -- 172.21.15.37:0/2526131892 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8015dc0 con 0x7ffba0004d50 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.539+0000 7ffbbdc85640 1 -- 172.21.15.37:0/2526131892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb81567d0 con 0x7ffba0004d50 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.543+0000 7ffbbdc85640 1 -- 172.21.15.37:0/2526131892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb804efa0 con 0x7ffba0004d50 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.543+0000 7ffb9ffff640 1 -- 172.21.15.37:0/2526131892 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffba800f460 con 0x7ffba0004d50 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.543+0000 7ffb9ffff640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffba4066f90 0x7ffba4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.543+0000 7ffb9ffff640 1 -- 172.21.15.37:0/2526131892 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffba8085b60 con 0x7ffba0004d50 2024-04-25T08:54:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.543+0000 7ffbb6ffd640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffba4066f90 0x7ffba4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.543+0000 7ffbb6ffd640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffba4066f90 0x7ffba4069450 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffbb815e720 tx=0x7ffbac007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.547+0000 7ffb9ffff640 1 -- 172.21.15.37:0/2526131892 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffba8019030 con 0x7ffba0004d50 2024-04-25T08:54:52.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.759+0000 7ffbbdc85640 1 -- 172.21.15.37:0/2526131892 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ffbb80b4170 con 0x7ffba0004d50 2024-04-25T08:54:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.763+0000 7ffb9ffff640 1 -- 172.21.15.37:0/2526131892 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7ffba8056950 con 0x7ffba0004d50 2024-04-25T08:54:52.757 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T08:54:52.757 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA== 2024-04-25T08:54:52.757 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-04-25T08:54:52.757 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-04-25T08:54:52.757 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:54:52.757 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-04-25T08:54:52.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.763+0000 7ffbbdc85640 1 -- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffba4066f90 msgr2=0x7ffba4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.763+0000 7ffbbdc85640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffba4066f90 0x7ffba4069450 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ffbb815e720 tx=0x7ffbac007040 comp rx=0 tx=0).stop 2024-04-25T08:54:52.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.763+0000 7ffbbdc85640 1 -- 172.21.15.37:0/2526131892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 msgr2=0x7ffbb806d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.763+0000 7ffbbdc85640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 0x7ffbb806d160 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ffba8011640 tx=0x7ffba8011a40 comp rx=0 tx=0).stop 2024-04-25T08:54:52.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.767+0000 7ffbbdc85640 1 -- 172.21.15.37:0/2526131892 shutdown_connections 2024-04-25T08:54:52.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.767+0000 7ffbbdc85640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffba4066f90 0x7ffba4069450 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.767+0000 7ffbbdc85640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffbb8157140 0x7ffbb81575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.767+0000 7ffbbdc85640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffbb80ae770 0x7ffbb8098b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.767+0000 7ffbbdc85640 1 --2- 172.21.15.37:0/2526131892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffba0004d50 0x7ffbb806d160 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.767+0000 7ffbbdc85640 1 -- 172.21.15.37:0/2526131892 >> 172.21.15.37:0/2526131892 conn(0x7ffbb806ae60 msgr2=0x7ffbb8159e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.767+0000 7ffbbdc85640 1 -- 172.21.15.37:0/2526131892 shutdown_connections 2024-04-25T08:54:52.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.767+0000 7ffbbdc85640 1 -- 172.21.15.37:0/2526131892 wait complete. 2024-04-25T08:54:52.775 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:52.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.891+0000 7f0461991640 1 Processor -- start 2024-04-25T08:54:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.891+0000 7f0461991640 1 -- start start 2024-04-25T08:54:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0461991640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f045c15da80 0x7f045c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0461991640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d1cd0 0x7f045c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0461991640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f045c06be60 con 0x7f045c0d0620 2024-04-25T08:54:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0461991640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f045c06c440 con 0x7f045c15da80 2024-04-25T08:54:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0461991640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c15de60 con 0x7f045c0d1cd0 2024-04-25T08:54:52.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0453fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d1cd0 0x7f045c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0453fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d1cd0 0x7f045c0d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:60584/0 (socket says 172.21.15.37:60584) 2024-04-25T08:54:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0453fff640 1 -- 172.21.15.37:0/1275556566 learned_addr learned my addr 172.21.15.37:0/1275556566 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f045affd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f045c15da80 0x7f045c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0453fff640 1 -- 172.21.15.37:0/1275556566 >> v2:172.21.15.203:3300/0 conn(0x7f045c15da80 msgr2=0x7f045c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0453fff640 1 --2- 172.21.15.37:0/1275556566 >> v2:172.21.15.203:3300/0 conn(0x7f045c15da80 0x7f045c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0453fff640 1 -- 172.21.15.37:0/1275556566 >> v1:172.21.15.203:6789/0 conn(0x7f045c0d0620 legacy=0x7f045c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0453fff640 1 -- 172.21.15.37:0/1275556566 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c0d2450 con 0x7f045c0d1cd0 2024-04-25T08:54:52.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.895+0000 7f0453fff640 1 --2- 172.21.15.37:0/1275556566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d1cd0 0x7f045c0d00e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0444006fd0 tx=0x7f044401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad5d2bc84a58bed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045a7fc640 1 -- 172.21.15.37:0/1275556566 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04440ab650 con 0x7f045c0d1cd0 2024-04-25T08:54:52.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045a7fc640 1 -- 172.21.15.37:0/1275556566 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04440b0070 con 0x7f045c0d1cd0 2024-04-25T08:54:52.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045a7fc640 1 -- 172.21.15.37:0/1275556566 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04440aba10 con 0x7f045c0d1cd0 2024-04-25T08:54:52.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 -- 172.21.15.37:0/1275556566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d1cd0 msgr2=0x7f045c0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 --2- 172.21.15.37:0/1275556566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d1cd0 0x7f045c0d00e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0444006fd0 tx=0x7f044401c540 comp rx=0 tx=0).stop 2024-04-25T08:54:52.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 -- 172.21.15.37:0/1275556566 shutdown_connections 2024-04-25T08:54:52.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 --2- 172.21.15.37:0/1275556566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d1cd0 0x7f045c0d00e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 --2- 172.21.15.37:0/1275556566 >> v2:172.21.15.203:3300/0 conn(0x7f045c15da80 0x7f045c0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 -- 172.21.15.37:0/1275556566 >> 172.21.15.37:0/1275556566 conn(0x7f045c08b000 msgr2=0x7f045c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 -- 172.21.15.37:0/1275556566 shutdown_connections 2024-04-25T08:54:52.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 -- 172.21.15.37:0/1275556566 wait complete. 2024-04-25T08:54:52.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 Processor -- start 2024-04-25T08:54:52.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 -- start start 2024-04-25T08:54:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d0620 0x7f045c0b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f045c0d1cd0 0x7f045c157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f045c15da80 0x7f045c158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c16a730 con 0x7f045c15da80 2024-04-25T08:54:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c16a5b0 con 0x7f045c0d1cd0 2024-04-25T08:54:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c16a8b0 con 0x7f045c0d0620 2024-04-25T08:54:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f045c15da80 0x7f045c158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f045c15da80 0x7f045c158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52382/0 (socket says 172.21.15.37:52382) 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045b7fe640 1 -- 172.21.15.37:0/2360773120 learned_addr learned my addr 172.21.15.37:0/2360773120 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045affd640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d0620 0x7f045c0b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0453fff640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f045c0d1cd0 0x7f045c157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045b7fe640 1 -- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d0620 msgr2=0x7f045c0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045b7fe640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d0620 0x7f045c0b58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045b7fe640 1 -- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f045c0d1cd0 msgr2=0x7f045c157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045b7fe640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f045c0d1cd0 0x7f045c157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045b7fe640 1 -- 172.21.15.37:0/2360773120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c151330 con 0x7f045c15da80 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045affd640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d0620 0x7f045c0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f045b7fe640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f045c15da80 0x7f045c158350 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f045400c740 tx=0x7f045400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0458ff9640 1 -- 172.21.15.37:0/2360773120 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0454016020 con 0x7f045c15da80 2024-04-25T08:54:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0458ff9640 1 -- 172.21.15.37:0/2360773120 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045400ce50 con 0x7f045c15da80 2024-04-25T08:54:52.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0458ff9640 1 -- 172.21.15.37:0/2360773120 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0454003980 con 0x7f045c15da80 2024-04-25T08:54:52.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 -- 172.21.15.37:0/2360773120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045c151610 con 0x7f045c15da80 2024-04-25T08:54:52.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.899+0000 7f0461991640 1 -- 172.21.15.37:0/2360773120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045c151bd0 con 0x7f045c15da80 2024-04-25T08:54:52.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.903+0000 7f0461991640 1 -- 172.21.15.37:0/2360773120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045c04efa0 con 0x7f045c15da80 2024-04-25T08:54:52.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.903+0000 7f0458ff9640 1 -- 172.21.15.37:0/2360773120 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0454003b20 con 0x7f045c15da80 2024-04-25T08:54:52.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.903+0000 7f0458ff9640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f043c066f60 0x7f043c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.903+0000 7f0458ff9640 1 -- 172.21.15.37:0/2360773120 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0454084de0 con 0x7f045c15da80 2024-04-25T08:54:52.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.903+0000 7f045affd640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f043c066f60 0x7f043c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.903+0000 7f045affd640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f043c066f60 0x7f043c069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f044c009b10 tx=0x7f044c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:52.907+0000 7f0458ff9640 1 -- 172.21.15.37:0/2360773120 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0454020020 con 0x7f045c15da80 2024-04-25T08:54:53.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 -- 172.21.15.37:0/2360773120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f045c0b1aa0 con 0x7f045c15da80 2024-04-25T08:54:53.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0458ff9640 1 -- 172.21.15.37:0/2360773120 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f0454019310 con 0x7f045c15da80 2024-04-25T08:54:53.096 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:53.096 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2963121871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2963121871},{"type":"v1","addr":"172.21.15.203:6837","nonce":2963121871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1490204181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1490204181},{"type":"v1","addr":"172.21.15.203:6835","nonce":1490204181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2218454043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2218454043},{"type":"v1","addr":"172.21.15.37:6833","nonce":2218454043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:42.800130+0000","modified":"2024-04-25T08:54:45.849035+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/3892786810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3892786810},{"type":"v1","addr":"172.21.15.37:6835","nonce":3892786810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:54:53.097 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-25T08:54:53.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 -- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f043c066f60 msgr2=0x7f043c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f043c066f60 0x7f043c069420 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f044c009b10 tx=0x7f044c008040 comp rx=0 tx=0).stop 2024-04-25T08:54:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 -- 172.21.15.37:0/2360773120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f045c15da80 msgr2=0x7f045c158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f045c15da80 0x7f045c158350 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f045400c740 tx=0x7f045400cc00 comp rx=0 tx=0).stop 2024-04-25T08:54:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 -- 172.21.15.37:0/2360773120 shutdown_connections 2024-04-25T08:54:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f043c066f60 0x7f043c069420 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f045c15da80 0x7f045c158350 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f045c0d1cd0 0x7f045c157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 --2- 172.21.15.37:0/2360773120 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f045c0d0620 0x7f045c0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 -- 172.21.15.37:0/2360773120 >> 172.21.15.37:0/2360773120 conn(0x7f045c08b000 msgr2=0x7f045c0d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 -- 172.21.15.37:0/2360773120 shutdown_connections 2024-04-25T08:54:53.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.103+0000 7f0461991640 1 -- 172.21.15.37:0/2360773120 wait complete. 2024-04-25T08:54:53.518 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:54:42.800130+0000', 'modified': '2024-04-25T08:54:45.849035+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5937}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5937': {'gid': 5937, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/3892786810', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 3892786810}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 3892786810}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:54:53.518 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:54:53.518 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:54:53.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.663+0000 7f7b95e54640 1 Processor -- start 2024-04-25T08:54:53.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.663+0000 7f7b95e54640 1 -- start start 2024-04-25T08:54:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b95e54640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7b9015da80 0x7f7b900d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b95e54640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b900d1cd0 0x7f7b900d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b95e54640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9006be60 con 0x7f7b900d0620 2024-04-25T08:54:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b95e54640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9006c440 con 0x7f7b9015da80 2024-04-25T08:54:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b95e54640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9015de60 con 0x7f7b900d1cd0 2024-04-25T08:54:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7b900d0620 0x7f7b900ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44788/0 (socket says 172.21.15.37:44788) 2024-04-25T08:54:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8ffff640 1 -- 172.21.15.37:0/826251224 learned_addr learned my addr 172.21.15.37:0/826251224 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:53.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/826251224 >> v2:172.21.15.203:3300/0 conn(0x7f7b9015da80 0x7f7b900d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b900d1cd0 0x7f7b900d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406707998 0 0) 0x7f7b9006be60 con 0x7f7b900d0620 2024-04-25T08:54:53.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b70003530 con 0x7f7b900d0620 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2672878619 0 0) 0x7f7b70003530 con 0x7f7b900d0620 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b900d1cd0 msgr2=0x7f7b900d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b900d1cd0 0x7f7b900d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 >> v2:172.21.15.203:3300/0 conn(0x7f7b9015da80 msgr2=0x7f7b900d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 --2- 172.21.15.37:0/826251224 >> v2:172.21.15.203:3300/0 conn(0x7f7b9015da80 0x7f7b900d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900d2450 con 0x7f7b900d0620 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f7b88001a00 con 0x7f7b900d0620 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 >> v1:172.21.15.37:6789/0 conn(0x7f7b900d0620 legacy=0x7f7b900ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b70003a30 0x7f7b70003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b900d1cd0 0x7f7b70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70004d50 0x7f7b700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9006c440 con 0x7f7b70004d50 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9015de60 con 0x7f7b70003a30 2024-04-25T08:54:53.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7000b9d0 con 0x7f7b900d1cd0 2024-04-25T08:54:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70004d50 0x7f7b700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b70003a30 0x7f7b70003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b900d1cd0 0x7f7b70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8ffff640 1 -- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b900d1cd0 msgr2=0x7f7b70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b900d1cd0 0x7f7b70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8ffff640 1 -- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b70003a30 msgr2=0x7f7b70003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b70003a30 0x7f7b70003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b8ffff640 1 -- 172.21.15.37:0/826251224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70003530 con 0x7f7b70004d50 2024-04-25T08:54:53.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.667+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b900d1cd0 0x7f7b70004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70004d50 0x7f7b700051d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800a870 tx=0x7f7b8801d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b79515f65b7beb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800a4c0 con 0x7f7b70004d50 2024-04-25T08:54:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b880072d0 con 0x7f7b70004d50 2024-04-25T08:54:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b8effd640 1 -- 172.21.15.37:0/826251224 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b880076c0 con 0x7f7b70004d50 2024-04-25T08:54:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b95e54640 1 -- 172.21.15.37:0/826251224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70004d50 msgr2=0x7f7b700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b95e54640 1 --2- 172.21.15.37:0/826251224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b70004d50 0x7f7b700051d0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800a870 tx=0x7f7b8801d3a0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b8ffff640 1 -- 172.21.15.37:0/826251224 reap_dead start 2024-04-25T08:54:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b95e54640 1 -- 172.21.15.37:0/826251224 shutdown_connections 2024-04-25T08:54:53.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b95e54640 1 -- 172.21.15.37:0/826251224 >> 172.21.15.37:0/826251224 conn(0x7f7b9008b000 msgr2=0x7f7b900d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b95e54640 1 -- 172.21.15.37:0/826251224 shutdown_connections 2024-04-25T08:54:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b95e54640 1 -- 172.21.15.37:0/826251224 wait complete. 2024-04-25T08:54:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b95e54640 1 Processor -- start 2024-04-25T08:54:53.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b95e54640 1 -- start start 2024-04-25T08:54:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b95e54640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b9014f4d0 0x7f7b900cb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.671+0000 7f7b95e54640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b900cce90 0x7f7b900b7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b95e54640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b900b8110 0x7f7b9014f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b95e54640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900d26f0 con 0x7f7b900cce90 2024-04-25T08:54:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b95e54640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900d20f0 con 0x7f7b900b8110 2024-04-25T08:54:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b95e54640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b900d2870 con 0x7f7b9014f4d0 2024-04-25T08:54:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b900cce90 0x7f7b900b7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b900cce90 0x7f7b900b7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52396/0 (socket says 172.21.15.37:52396) 2024-04-25T08:54:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b86ffd640 1 -- 172.21.15.37:0/3749691271 learned_addr learned my addr 172.21.15.37:0/3749691271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b900b8110 0x7f7b9014f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b9014f4d0 0x7f7b900cb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b86ffd640 1 -- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b9014f4d0 msgr2=0x7f7b900cb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b9014f4d0 0x7f7b900cb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b86ffd640 1 -- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b900b8110 msgr2=0x7f7b9014f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b900b8110 0x7f7b9014f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b86ffd640 1 -- 172.21.15.37:0/3749691271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b900cbdf0 con 0x7f7b900cce90 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b900b8110 0x7f7b9014f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b900cce90 0x7f7b900b7bd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7b78011680 tx=0x7f7b78011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b8cff9640 1 -- 172.21.15.37:0/3749691271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78019070 con 0x7f7b900cce90 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b8cff9640 1 -- 172.21.15.37:0/3749691271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b78004220 con 0x7f7b900cce90 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b8cff9640 1 -- 172.21.15.37:0/3749691271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b780059a0 con 0x7f7b900cce90 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b95e54640 1 -- 172.21.15.37:0/3749691271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b900cc0d0 con 0x7f7b900cce90 2024-04-25T08:54:53.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b95e54640 1 -- 172.21.15.37:0/3749691271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b900cc690 con 0x7f7b900cce90 2024-04-25T08:54:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b8cff9640 1 -- 172.21.15.37:0/3749691271 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7b78005b40 con 0x7f7b900cce90 2024-04-25T08:54:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b8cff9640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7b74066f60 0x7f7b74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b8cff9640 1 -- 172.21.15.37:0/3749691271 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b780856f0 con 0x7f7b900cce90 2024-04-25T08:54:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.675+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7b74066f60 0x7f7b74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 Processor -- start 2024-04-25T08:54:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 -- start start 2024-04-25T08:54:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7b74066f60 0x7f7b74069420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009b20 tx=0x7f7b80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 0x558569bc0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc1240 0x558569b769f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569b77310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558569a5e5f0 con 0x558569b76f30 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585699c5da0 con 0x558569bc0920 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55856994e150 con 0x558569bc1240 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569b77310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 0x558569bc0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc1240 0x558569b769f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569b77310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52410/0 (socket says 172.21.15.37:52410) 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8ffff640 1 -- 172.21.15.37:0/3993746580 learned_addr learned my addr 172.21.15.37:0/3993746580 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8f7fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 0x558569bc0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:57784/0 (socket says 172.21.15.37:57784) 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc1240 0x558569b769f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:60632/0 (socket says 172.21.15.37:60632) 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8ffff640 1 -- 172.21.15.37:0/3993746580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc1240 msgr2=0x558569b769f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/3993746580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc1240 0x558569b769f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8ffff640 1 -- 172.21.15.37:0/3993746580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 msgr2=0x558569bc0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/3993746580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 0x558569bc0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8ffff640 1 -- 172.21.15.37:0/3993746580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558569967ca0 con 0x558569b76f30 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/3993746580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569b77310 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800a870 tx=0x7f7b88001370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4794fd17eecb17e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b857fa640 1 -- 172.21.15.37:0/3993746580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b88002040 con 0x558569b76f30 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b857fa640 1 -- 172.21.15.37:0/3993746580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8800a4c0 con 0x558569b76f30 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b857fa640 1 -- 172.21.15.37:0/3993746580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b880011d0 con 0x558569b76f30 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 -- 172.21.15.37:0/3993746580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 msgr2=0x558569b77310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/3993746580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569b77310 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800a870 tx=0x7f7b88001370 comp rx=0 tx=0).stop 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 -- 172.21.15.37:0/3993746580 shutdown_connections 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/3993746580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569b77310 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/3993746580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc1240 0x558569b769f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/3993746580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 0x558569bc0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 -- 172.21.15.37:0/3993746580 >> 172.21.15.37:0/3993746580 conn(0x558569b7d2a0 msgr2=0x558569b752d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 -- 172.21.15.37:0/3993746580 shutdown_connections 2024-04-25T08:54:53.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 -- 172.21.15.37:0/3993746580 wait complete. 2024-04-25T08:54:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 -- start start 2024-04-25T08:54:53.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.679+0000 7f7b98fc1000 1 Processor -- start 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569bc7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 0x558569bc81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc1240 0x558569b7aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558569980990 con 0x558569b76f30 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585699c1e40 con 0x558569bc0920 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558569951900 con 0x558569bc1240 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 10 client.0 _ll_register_callbacks cb 0x558569a791c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b8f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569bc7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b8f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569bc7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52424/0 (socket says 172.21.15.37:52424) 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b8f7fe640 1 -- 172.21.15.37:0/2698370952 learned_addr learned my addr 172.21.15.37:0/2698370952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 0x558569bc81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc1240 0x558569b7aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b8f7fe640 1 -- 172.21.15.37:0/2698370952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc1240 msgr2=0x558569b7aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc1240 0x558569b7aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b8f7fe640 1 -- 172.21.15.37:0/2698370952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 msgr2=0x558569bc81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 0x558569bc81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b8f7fe640 1 -- 172.21.15.37:0/2698370952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55856997e630 con 0x558569b76f30 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 0x558569bc81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569bc7c80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009620 tx=0x7f7b80004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28dec4dccf226a75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/2698370952 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80023b50 con 0x558569b76f30 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/2698370952 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b80023cf0 con 0x558569b76f30 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 -- 172.21.15.37:0/2698370952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b8800a4c0 con 0x558569b76f30 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/2698370952 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b80036050 con 0x558569b76f30 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 10 client.6102 fetch_fsmap learned FSMap version 25 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 -- 172.21.15.37:0/2698370952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55856994e3a0 con 0x558569b76f30 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/2698370952 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8800a4c0 con 0x558569b76f30 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/2698370952 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7b80039020 con 0x558569b76f30 2024-04-25T08:54:53.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 10 client.6102 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 10 client.6102 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 20 client.6102 populate_metadata read hostname 'smithi037' 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x558569b7c160 0x558569bd1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 4 client.6102 mds_command: new command op to 5937 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 -- 172.21.15.37:0/2698370952 --> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558569961a50 con 0x558569b7c160 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x558569b7c160 0x558569bd1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x558569b7c160 0x558569bd1c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2da9bc858e89813a server_cookie=a8360fea49c8115f in_seq=0 out_seq=0 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b7e7fc640 10 client.6102 ms_handle_connect on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/2698370952 <== mds.0 v2:172.21.15.37:6834/3892786810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558569961a50 con 0x558569b7c160 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b7e7fc640 10 client.6102 handle_command_reply: tid=0 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 client.6102 shutdown 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 -- 172.21.15.37:0/2698370952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 msgr2=0x558569bc7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569bc7c80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009620 tx=0x7f7b80004740 comp rx=0 tx=0).stop 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 -- 172.21.15.37:0/2698370952 shutdown_connections 2024-04-25T08:54:53.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x558569b7c160 0x558569bd1c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc1240 0x558569b7aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc0920 0x558569bc81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/2698370952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569bc7c80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 -- 172.21.15.37:0/2698370952 >> 172.21.15.37:0/2698370952 conn(0x558569b7d2a0 msgr2=0x558569b759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b7e7fc640 0 client.6102 ms_handle_reset on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 -- 172.21.15.37:0/2698370952 shutdown_connections 2024-04-25T08:54:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.683+0000 7f7b98fc1000 1 -- 172.21.15.37:0/2698370952 wait complete. 2024-04-25T08:54:53.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.687+0000 7f7b98fc1000 20 client.6102 trim_cache size 0 max 16384 2024-04-25T08:54:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.699+0000 7f7b98fc1000 1 Processor -- start 2024-04-25T08:54:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.699+0000 7f7b98fc1000 1 -- start start 2024-04-25T08:54:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569c1b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc0920 0x558569c1ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 0x558569c1bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b901ffdc0 con 0x558569b76f30 2024-04-25T08:54:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b901514e0 con 0x558569bc1240 2024-04-25T08:54:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b901ff7c0 con 0x558569bc0920 2024-04-25T08:54:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569c1b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569c1b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52432/0 (socket says 172.21.15.37:52432) 2024-04-25T08:54:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8f7fe640 1 -- 172.21.15.37:0/116064212 learned_addr learned my addr 172.21.15.37:0/116064212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/116064212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 0x558569c1bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/116064212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc0920 0x558569c1ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8f7fe640 1 -- 172.21.15.37:0/116064212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc0920 msgr2=0x558569c1ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/116064212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc0920 0x558569c1ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8f7fe640 1 -- 172.21.15.37:0/116064212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 msgr2=0x558569c1bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/116064212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 0x558569c1bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8f7fe640 1 -- 172.21.15.37:0/116064212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558569a47600 con 0x558569b76f30 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/116064212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 0x558569c1bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/116064212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569c1b530 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7b80008010 tx=0x7f7b80026070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81d25764a9880142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b84ff9640 1 -- 172.21.15.37:0/116064212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80023d50 con 0x558569b76f30 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b84ff9640 1 -- 172.21.15.37:0/116064212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b80023790 con 0x558569b76f30 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b84ff9640 1 -- 172.21.15.37:0/116064212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80024820 con 0x558569b76f30 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- 172.21.15.37:0/116064212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 msgr2=0x558569c1b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/116064212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569c1b530 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7b80008010 tx=0x7f7b80026070 comp rx=0 tx=0).stop 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- 172.21.15.37:0/116064212 shutdown_connections 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/116064212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 0x558569c1bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/116064212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569bc0920 0x558569c1ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/116064212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569b76f30 0x558569c1b530 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- 172.21.15.37:0/116064212 >> 172.21.15.37:0/116064212 conn(0x558569b7d2a0 msgr2=0x558569bcfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- 172.21.15.37:0/116064212 shutdown_connections 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- 172.21.15.37:0/116064212 wait complete. 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- start start 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 Processor -- start 2024-04-25T08:54:53.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569b76f30 0x558569bd5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569bc0920 0x558569c06e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 0x558569c07370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585699ca3d0 con 0x558569bc0920 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558569a81d50 con 0x558569bc1240 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558569960980 con 0x558569b76f30 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b98fc1000 10 client.0 _ll_register_callbacks cb 0x7f7b70003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569bc0920 0x558569c06e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b86ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569bc0920 0x558569c06e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52448/0 (socket says 172.21.15.37:52448) 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b86ffd640 1 -- 172.21.15.37:0/562614882 learned_addr learned my addr 172.21.15.37:0/562614882 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 0x558569c07370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569b76f30 0x558569bd5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b86ffd640 1 -- 172.21.15.37:0/562614882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569b76f30 msgr2=0x558569bd5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569b76f30 0x558569bd5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b86ffd640 1 -- 172.21.15.37:0/562614882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 msgr2=0x558569c07370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 0x558569c07370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b86ffd640 1 -- 172.21.15.37:0/562614882 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55856995a9b0 con 0x558569bc0920 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b8ffff640 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 0x558569c07370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b86ffd640 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569bc0920 0x558569c06e30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7b78052230 tx=0x7f7b7805c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385a07d7dc0a898a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.703+0000 7f7b7e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/562614882 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b78073060 con 0x558569bc0920 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/562614882 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b7805c0b0 con 0x558569bc0920 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 -- 172.21.15.37:0/562614882 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7b80023790 con 0x558569bc0920 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/562614882 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b7807a050 con 0x558569bc0920 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 10 client.6108 fetch_fsmap learned FSMap version 25 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 -- 172.21.15.37:0/562614882 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55856996d0b0 con 0x558569bc0920 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/562614882 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b80023790 con 0x558569bc0920 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/562614882 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7b78052440 con 0x558569bc0920 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 10 client.6108 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 10 client.6108 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 20 client.6108 populate_metadata read hostname 'smithi037' 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x558569c1d230 0x558569c1d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 4 client.6108 mds_command: new command op to 5937 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 -- 172.21.15.37:0/562614882 --> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5585699d32e0 con 0x558569c1d230 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x558569c1d230 0x558569c1d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b8f7fe640 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x558569c1d230 0x558569c1d610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b7602bc938441ed server_cookie=8ac3551d91070ce8 in_seq=0 out_seq=0 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b7e7fc640 10 client.6108 ms_handle_connect on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b7e7fc640 1 -- 172.21.15.37:0/562614882 <== mds.0 v2:172.21.15.37:6834/3892786810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5585699d32e0 con 0x558569c1d230 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b7e7fc640 10 client.6108 handle_command_reply: tid=0 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 client.6108 shutdown 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 -- 172.21.15.37:0/562614882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569bc0920 msgr2=0x558569c06e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569bc0920 0x558569c06e30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f7b78052230 tx=0x7f7b7805c040 comp rx=0 tx=0).stop 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 -- 172.21.15.37:0/562614882 shutdown_connections 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x558569c1d230 0x558569c1d610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558569bc1240 0x558569c07370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b7e7fc640 0 client.6108 ms_handle_reset on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558569bc0920 0x558569c06e30 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 --2- 172.21.15.37:0/562614882 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558569b76f30 0x558569bd5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 -- 172.21.15.37:0/562614882 >> 172.21.15.37:0/562614882 conn(0x558569b7d2a0 msgr2=0x558569bbea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 -- 172.21.15.37:0/562614882 shutdown_connections 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 1 -- 172.21.15.37:0/562614882 wait complete. 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b98fc1000 20 client.6108 trim_cache size 0 max 16384 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T08:54:53.708 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 5937, 2024-04-25T08:54:53.708 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T08:54:53.708 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T08:54:53.708 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T08:54:53.708 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 10.228657763999999, 2024-04-25T08:54:53.708 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 25, 2024-04-25T08:54:53.708 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 55, 2024-04-25T08:54:53.708 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 55, 2024-04-25T08:54:53.708 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 16.256957404000001 2024-04-25T08:54:53.708 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:54:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 -- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7b74066f60 msgr2=0x7f7b74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7b74066f60 0x7f7b74069420 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009b20 tx=0x7f7b80008040 comp rx=0 tx=0).stop 2024-04-25T08:54:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 -- 172.21.15.37:0/3749691271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b900cce90 msgr2=0x7f7b900b7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b900cce90 0x7f7b900b7bd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7b78011680 tx=0x7f7b78011b40 comp rx=0 tx=0).stop 2024-04-25T08:54:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 -- 172.21.15.37:0/3749691271 shutdown_connections 2024-04-25T08:54:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7b74066f60 0x7f7b74069420 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b900b8110 0x7f7b9014f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b900cce90 0x7f7b900b7bd0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 --2- 172.21.15.37:0/3749691271 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b9014f4d0 0x7f7b900cb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 -- 172.21.15.37:0/3749691271 >> 172.21.15.37:0/3749691271 conn(0x7f7b9008b000 msgr2=0x7f7b900d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 -- 172.21.15.37:0/3749691271 shutdown_connections 2024-04-25T08:54:53.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.707+0000 7f7b95e54640 1 -- 172.21.15.37:0/3749691271 wait complete. 2024-04-25T08:54:53.718 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:54:53.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f748219a640 1 Processor -- start 2024-04-25T08:54:53.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f748219a640 1 -- start start 2024-04-25T08:54:53.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f748219a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f747c0d4230 0x7f747c0d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f748219a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747c06be60 con 0x7f747c15da80 2024-04-25T08:54:53.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f748219a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f747c0d4650 con 0x7f747c0d07f0 2024-04-25T08:54:53.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f748219a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f747c0d2210 con 0x7f747c0d4230 2024-04-25T08:54:53.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f747c15da80 0x7f747c0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44794/0 (socket says 172.21.15.37:44794) 2024-04-25T08:54:53.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747b7fe640 1 -- 172.21.15.37:0/618049235 learned_addr learned my addr 172.21.15.37:0/618049235 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:53.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747bfff640 1 --2- 172.21.15.37:0/618049235 >> v2:172.21.15.203:3300/0 conn(0x7f747c0d4230 0x7f747c0d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117718682 0 0) 0x7f747c06be60 con 0x7f747c15da80 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7468003530 con 0x7f747c15da80 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2164587083 0 0) 0x7f7468003530 con 0x7f747c15da80 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 >> v2:172.21.15.203:3300/0 conn(0x7f747c0d4230 msgr2=0x7f747c0d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 --2- 172.21.15.37:0/618049235 >> v2:172.21.15.203:3300/0 conn(0x7f747c0d4230 0x7f747c0d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 >> v1:172.21.15.203:6789/0 conn(0x7f747c0d07f0 legacy=0x7f747c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c15de60 con 0x7f747c15da80 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f746c001680 con 0x7f747c15da80 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 >> v1:172.21.15.37:6789/0 conn(0x7f747c15da80 legacy=0x7f747c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468003a30 0x7f7468003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7468004810 0x7f7468004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74680051d0 0x7f746800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c0d2210 con 0x7f7468004810 2024-04-25T08:54:53.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746800c100 con 0x7f7468003a30 2024-04-25T08:54:53.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746800c300 con 0x7f74680051d0 2024-04-25T08:54:53.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f7473fff640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7468004810 0x7f7468004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747b7fe640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468003a30 0x7f7468003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747bfff640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74680051d0 0x7f746800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f7473fff640 1 -- 172.21.15.37:0/618049235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74680051d0 msgr2=0x7f746800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f7473fff640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74680051d0 0x7f746800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f7473fff640 1 -- 172.21.15.37:0/618049235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468003a30 msgr2=0x7f7468003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f7473fff640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468003a30 0x7f7468003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f7473fff640 1 -- 172.21.15.37:0/618049235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7468003530 con 0x7f7468004810 2024-04-25T08:54:53.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747bfff640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74680051d0 0x7f746800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:53.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.843+0000 7f747bfff640 1 -- 172.21.15.37:0/618049235 reap_dead start 2024-04-25T08:54:53.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f7473fff640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7468004810 0x7f7468004c90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f74640017b0 tx=0x7f746401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13895c98f0521e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74640013d0 con 0x7f7468004810 2024-04-25T08:54:53.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74640052d0 con 0x7f7468004810 2024-04-25T08:54:53.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f747affd640 1 -- 172.21.15.37:0/618049235 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464003650 con 0x7f7468004810 2024-04-25T08:54:53.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 -- 172.21.15.37:0/618049235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7468004810 msgr2=0x7f7468004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7468004810 0x7f7468004c90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f74640017b0 tx=0x7f746401c3e0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 -- 172.21.15.37:0/618049235 shutdown_connections 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 --2- 172.21.15.37:0/618049235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7468004810 0x7f7468004c90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 -- 172.21.15.37:0/618049235 >> 172.21.15.37:0/618049235 conn(0x7f747c08b000 msgr2=0x7f747c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 -- 172.21.15.37:0/618049235 shutdown_connections 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 -- 172.21.15.37:0/618049235 wait complete. 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 Processor -- start 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 -- start start 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468004810 0x7f747c0cd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f747c14ea30 0x7f747c0c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f747c0c7490 0x7f747c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c0d2450 con 0x7f747c14ea30 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c06c440 con 0x7f7468004810 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f748219a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747c0d26f0 con 0x7f747c0c7490 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f7473fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f747c14ea30 0x7f747c0c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f7473fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f747c14ea30 0x7f747c0c6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52464/0 (socket says 172.21.15.37:52464) 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.847+0000 7f7473fff640 1 -- 172.21.15.37:0/2170008554 learned_addr learned my addr 172.21.15.37:0/2170008554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f747b7fe640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468004810 0x7f747c0cd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7473fff640 1 -- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f747c0c7490 msgr2=0x7f747c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f747bfff640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f747c0c7490 0x7f747c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7473fff640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f747c0c7490 0x7f747c0ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7473fff640 1 -- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468004810 msgr2=0x7f747c0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7473fff640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468004810 0x7f747c0cd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7473fff640 1 -- 172.21.15.37:0/2170008554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c0cc000 con 0x7f747c14ea30 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f747b7fe640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468004810 0x7f747c0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7473fff640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f747c14ea30 0x7f747c0c6f50 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7464005040 tx=0x7f7468005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7478ff9640 1 -- 172.21.15.37:0/2170008554 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464007040 con 0x7f747c14ea30 2024-04-25T08:54:53.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7478ff9640 1 -- 172.21.15.37:0/2170008554 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74640bb070 con 0x7f747c14ea30 2024-04-25T08:54:53.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f748219a640 1 -- 172.21.15.37:0/2170008554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747c0cc2e0 con 0x7f747c14ea30 2024-04-25T08:54:53.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7478ff9640 1 -- 172.21.15.37:0/2170008554 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7464007420 con 0x7f747c14ea30 2024-04-25T08:54:53.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f748219a640 1 -- 172.21.15.37:0/2170008554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747c0cc8a0 con 0x7f747c14ea30 2024-04-25T08:54:53.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f748219a640 1 -- 172.21.15.37:0/2170008554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747c04efa0 con 0x7f747c14ea30 2024-04-25T08:54:53.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7478ff9640 1 -- 172.21.15.37:0/2170008554 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7464003650 con 0x7f747c14ea30 2024-04-25T08:54:53.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7478ff9640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f745c066f90 0x7f745c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f7478ff9640 1 -- 172.21.15.37:0/2170008554 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f746411f690 con 0x7f747c14ea30 2024-04-25T08:54:53.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f747b7fe640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f745c066f90 0x7f745c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.851+0000 7f747b7fe640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f745c066f90 0x7f745c069450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f746c001210 tx=0x7f746c001120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:53.855+0000 7f7478ff9640 1 -- 172.21.15.37:0/2170008554 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f74640160f0 con 0x7f747c14ea30 2024-04-25T08:54:54.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 -- 172.21.15.37:0/2170008554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f747c0b1ec0 con 0x7f747c14ea30 2024-04-25T08:54:54.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f7478ff9640 1 -- 172.21.15.37:0/2170008554 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f74640f0510 con 0x7f747c14ea30 2024-04-25T08:54:54.045 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 -- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f745c066f90 msgr2=0x7f745c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f745c066f90 0x7f745c069450 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f746c001210 tx=0x7f746c001120 comp rx=0 tx=0).stop 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 -- 172.21.15.37:0/2170008554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f747c14ea30 msgr2=0x7f747c0c6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f747c14ea30 0x7f747c0c6f50 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f7464005040 tx=0x7f7468005840 comp rx=0 tx=0).stop 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 -- 172.21.15.37:0/2170008554 shutdown_connections 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f745c066f90 0x7f745c069450 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f747c0c7490 0x7f747c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f747c14ea30 0x7f747c0c6f50 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 --2- 172.21.15.37:0/2170008554 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7468004810 0x7f747c0cd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 -- 172.21.15.37:0/2170008554 >> 172.21.15.37:0/2170008554 conn(0x7f747c08b000 msgr2=0x7f747c0b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 -- 172.21.15.37:0/2170008554 shutdown_connections 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:54.051+0000 7f748219a640 1 -- 172.21.15.37:0/2170008554 wait complete. 2024-04-25T08:54:54.062 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:54:54.062 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:54:54.062 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T08:54:54.062 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:54:54.062 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:54.062 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:54:54.062 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:54:54.062 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:54:54.062 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:54:54.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:54.063 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T08:54:54.069 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:54:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:54:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:54:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:54:54.069 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:54.069 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42347sec preferred_lft 42347sec 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:54:54.070 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:54:54.071 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:54.072 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:54:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:54:54.072 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:54.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:54.072 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:54:54.122 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:54:54.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:54.125 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T08:54:54.173 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T08:54:54.173 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:54:54.173 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:54:54.173 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:54:54.173 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:54:54.173 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:54:54.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:54.174 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:54:54.174 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:54:54.174 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:54:54.174 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:54:54.174 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:54:54.174 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:54:54.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:54.248 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:54:54.296 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:54:54.296 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:54:54.296 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:54:54.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:54.296 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T08:54:54.356 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:54.356 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:54.404 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:54:54.404 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:54.404 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:54.452 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:54:54.510 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:54.558 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:54:54.558 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:54:54.558 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:54:54.558 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:54:54.558 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:54:54.558 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:54:54.559 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:54.560 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:54.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:54.561 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:54:54.608 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:54:54.608 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:54:54.653 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:54:54.667 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:54:54.716 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:54:54.717 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:54:54.718 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:54:54.718 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:54:54.718 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:54.718 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:54.718 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:54.718 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:54.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:54.721 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:54:54.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T08:54:54.743+0000 7f0126d54580 -1 init, newargv = 0x555de71fa1e0 newargc=15 2024-04-25T08:54:54.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[41298]: starting ceph client 2024-04-25T08:54:54.741 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[41298]: starting fuse 2024-04-25T08:54:55.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:54:55.271 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:54:55.843 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:54:55.860 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:55.912 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:54:55.913 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:54:55.914 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:54:55.915 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:54:55.915 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:54:55.915 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:55.915 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:55.915 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:55.915 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:55.915 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:55.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:55.916 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:54:55.964 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:54:55.965 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:55.965 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T08:54:55.966 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:54:56.091 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.41298.asok 2024-04-25T08:54:56.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:56.103 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41298.asok 2024-04-25T08:54:56.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:56.104 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41298.asok status 2024-04-25T08:54:56.265 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:54:56.265 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T08:54:56.265 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "41298", 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6120, 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T08:54:56.266 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 6120 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 359888246 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 359888246 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.6120 192.168.0.1:0/359888246", 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/359888246", 2024-04-25T08:54:56.267 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T08:54:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 25, 2024-04-25T08:54:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 55, 2024-04-25T08:54:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:54:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T08:54:56.268 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T08:54:56.268 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:54:56.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:56.279 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T08:54:56.279 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:56.286 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T08:54:56.286 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:56.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:56.287 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:56.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:56.376 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-04-25T08:54:56.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:56.399 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-04-25T08:54:56.453 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T08:54:56.454 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:54:56.454 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T08:54:56.454 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:54:56.454 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-04-25T08:54:56.454 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:54:56.560 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627777 2024-04-25T08:54:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:56.571 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f337b769640 1 Processor -- start 2024-04-25T08:54:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f337b769640 1 -- start start 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f337b769640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f337415a700 0x7f337415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f337b769640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f33740b24a0 0x7f33740b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f337b769640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33740b2090 con 0x7f33740b3f50 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f337b769640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f337415f200 con 0x7f33740b24a0 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f337b769640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33740b43e0 con 0x7f337415a700 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f3379d06640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f33740b24a0 0x7f33740b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f3379d06640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f33740b24a0 0x7f33740b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52476/0 (socket says 172.21.15.37:52476) 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f3379d06640 1 -- 172.21.15.37:0/3468157093 learned_addr learned my addr 172.21.15.37:0/3468157093 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f3379505640 1 --2- 172.21.15.37:0/3468157093 >> v2:172.21.15.203:3300/0 conn(0x7f337415a700 0x7f337415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f3379d06640 1 -- 172.21.15.37:0/3468157093 >> v2:172.21.15.203:3300/0 conn(0x7f337415a700 msgr2=0x7f337415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f3379d06640 1 --2- 172.21.15.37:0/3468157093 >> v2:172.21.15.203:3300/0 conn(0x7f337415a700 0x7f337415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f3379d06640 1 -- 172.21.15.37:0/3468157093 >> v1:172.21.15.203:6789/0 conn(0x7f33740b3f50 legacy=0x7f33740cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f3379d06640 1 -- 172.21.15.37:0/3468157093 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33740b46c0 con 0x7f33740b24a0 2024-04-25T08:54:56.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.747+0000 7f3379505640 1 --2- 172.21.15.37:0/3468157093 >> v2:172.21.15.203:3300/0 conn(0x7f337415a700 0x7f337415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f3379d06640 1 --2- 172.21.15.37:0/3468157093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33740b24a0 0x7f33740b28f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f337406c980 tx=0x7f336c01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20eceb3140d6b43c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f336bfff640 1 -- 172.21.15.37:0/3468157093 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c0a5620 con 0x7f33740b24a0 2024-04-25T08:54:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f336bfff640 1 -- 172.21.15.37:0/3468157093 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f336c0a57c0 con 0x7f33740b24a0 2024-04-25T08:54:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f336bfff640 1 -- 172.21.15.37:0/3468157093 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f336c0a5b90 con 0x7f33740b24a0 2024-04-25T08:54:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 -- 172.21.15.37:0/3468157093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33740b24a0 msgr2=0x7f33740b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 --2- 172.21.15.37:0/3468157093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33740b24a0 0x7f33740b28f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f337406c980 tx=0x7f336c01c870 comp rx=0 tx=0).stop 2024-04-25T08:54:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 -- 172.21.15.37:0/3468157093 shutdown_connections 2024-04-25T08:54:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 --2- 172.21.15.37:0/3468157093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33740b24a0 0x7f33740b28f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 --2- 172.21.15.37:0/3468157093 >> v2:172.21.15.203:3300/0 conn(0x7f337415a700 0x7f337415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 -- 172.21.15.37:0/3468157093 >> 172.21.15.37:0/3468157093 conn(0x7f33740b11a0 msgr2=0x7f33740b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 -- 172.21.15.37:0/3468157093 shutdown_connections 2024-04-25T08:54:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 -- 172.21.15.37:0/3468157093 wait complete. 2024-04-25T08:54:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 Processor -- start 2024-04-25T08:54:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 -- start start 2024-04-25T08:54:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33740b24a0 0x7f33740cfe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f337415a700 0x7f3374158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f337415f3c0 0x7f337415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33740cd310 con 0x7f33740b24a0 2024-04-25T08:54:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33740ce600 con 0x7f337415f3c0 2024-04-25T08:54:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f337b769640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33740cd490 con 0x7f337415a700 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f3379505640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33740b24a0 0x7f33740cfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f3379505640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33740b24a0 0x7f33740cfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52480/0 (socket says 172.21.15.37:52480) 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.751+0000 7f3379505640 1 -- 172.21.15.37:0/312722771 learned_addr learned my addr 172.21.15.37:0/312722771 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3378d04640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f337415a700 0x7f3374158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3379d06640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f337415f3c0 0x7f337415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3379505640 1 -- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f337415a700 msgr2=0x7f3374158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3379505640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f337415a700 0x7f3374158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3379505640 1 -- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f337415f3c0 msgr2=0x7f337415cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3379505640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f337415f3c0 0x7f337415cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3379505640 1 -- 172.21.15.37:0/312722771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33741515a0 con 0x7f33740b24a0 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3379d06640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f337415f3c0 0x7f337415cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3379505640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33740b24a0 0x7f33740cfe50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f337000c7c0 tx=0x7f337000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3369ffb640 1 -- 172.21.15.37:0/312722771 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3370007a30 con 0x7f33740b24a0 2024-04-25T08:54:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3369ffb640 1 -- 172.21.15.37:0/312722771 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3370007bd0 con 0x7f33740b24a0 2024-04-25T08:54:56.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f337b769640 1 -- 172.21.15.37:0/312722771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3374151880 con 0x7f33740b24a0 2024-04-25T08:54:56.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3369ffb640 1 -- 172.21.15.37:0/312722771 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33700113f0 con 0x7f33740b24a0 2024-04-25T08:54:56.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f337b769640 1 -- 172.21.15.37:0/312722771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337415f980 con 0x7f33740b24a0 2024-04-25T08:54:56.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f337b769640 1 -- 172.21.15.37:0/312722771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f337404efa0 con 0x7f33740b24a0 2024-04-25T08:54:56.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3369ffb640 1 -- 172.21.15.37:0/312722771 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3370011590 con 0x7f33740b24a0 2024-04-25T08:54:56.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3369ffb640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3358066f60 0x7f3358069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3369ffb640 1 -- 172.21.15.37:0/312722771 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3370085370 con 0x7f33740b24a0 2024-04-25T08:54:56.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3378d04640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3358066f60 0x7f3358069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.755+0000 7f3378d04640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3358066f60 0x7f3358069420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f33741f5a60 tx=0x7f3364009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.759+0000 7f3369ffb640 1 -- 172.21.15.37:0/312722771 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3370015020 con 0x7f33740b24a0 2024-04-25T08:54:56.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 -- 172.21.15.37:0/312722771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f337415a010 con 0x7f33740b24a0 2024-04-25T08:54:56.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f3369ffb640 1 -- 172.21.15.37:0/312722771 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f3370056280 con 0x7f33740b24a0 2024-04-25T08:54:56.949 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:56.949 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2963121871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2963121871},{"type":"v1","addr":"172.21.15.203:6837","nonce":2963121871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1490204181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1490204181},{"type":"v1","addr":"172.21.15.203:6835","nonce":1490204181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2218454043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2218454043},{"type":"v1","addr":"172.21.15.37:6833","nonce":2218454043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:42.800130+0000","modified":"2024-04-25T08:54:45.849035+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/3892786810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3892786810},{"type":"v1","addr":"172.21.15.37:6835","nonce":3892786810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:54:56.950 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-25T08:54:56.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 -- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3358066f60 msgr2=0x7f3358069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3358066f60 0x7f3358069420 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f33741f5a60 tx=0x7f3364009040 comp rx=0 tx=0).stop 2024-04-25T08:54:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 -- 172.21.15.37:0/312722771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33740b24a0 msgr2=0x7f33740cfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33740b24a0 0x7f33740cfe50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f337000c7c0 tx=0x7f337000cc80 comp rx=0 tx=0).stop 2024-04-25T08:54:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 -- 172.21.15.37:0/312722771 shutdown_connections 2024-04-25T08:54:56.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3358066f60 0x7f3358069420 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f337415f3c0 0x7f337415cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f337415a700 0x7f3374158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 --2- 172.21.15.37:0/312722771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33740b24a0 0x7f33740cfe50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 -- 172.21.15.37:0/312722771 >> 172.21.15.37:0/312722771 conn(0x7f33740b11a0 msgr2=0x7f33740b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 -- 172.21.15.37:0/312722771 shutdown_connections 2024-04-25T08:54:56.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:56.955+0000 7f337b769640 1 -- 172.21.15.37:0/312722771 wait complete. 2024-04-25T08:54:56.968 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T08:54:57.115 INFO:teuthology.orchestra.run.smithi037.stdout:{"message":"","return_code":0} 2024-04-25T08:54:57.115 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:54:57.115 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T08:54:57.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.159+0000 7fac5fa87080 1 Processor -- start 2024-04-25T08:54:57.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.159+0000 7fac5fa87080 1 -- start start 2024-04-25T08:54:57.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.159+0000 7fac5fa87080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c587048c40 0x55c5870d2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.159+0000 7fac5fa87080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5870d5210 0x55c5870d70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.159+0000 7fac5fa87080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d55f0 0x55c5870d7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.159+0000 7fac5fa87080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5870d6970 con 0x55c5870d55f0 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.159+0000 7fac5fa87080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c586fa69f0 con 0x55c587048c40 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.159+0000 7fac5fa87080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5870d6af0 con 0x55c5870d5210 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.159+0000 7fac5dda6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d55f0 0x55c5870d7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.159+0000 7fac5dda6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d55f0 0x55c5870d7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52498/0 (socket says 172.21.15.37:52498) 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.159+0000 7fac5dda6640 1 -- 172.21.15.37:0/51081650 learned_addr learned my addr 172.21.15.37:0/51081650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5d5a5640 1 --2- 172.21.15.37:0/51081650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c587048c40 0x55c5870d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5cda4640 1 --2- 172.21.15.37:0/51081650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5870d5210 0x55c5870d70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5dda6640 1 -- 172.21.15.37:0/51081650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5870d5210 msgr2=0x55c5870d70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5dda6640 1 --2- 172.21.15.37:0/51081650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5870d5210 0x55c5870d70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5dda6640 1 -- 172.21.15.37:0/51081650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c587048c40 msgr2=0x55c5870d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5dda6640 1 --2- 172.21.15.37:0/51081650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c587048c40 0x55c5870d2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5dda6640 1 -- 172.21.15.37:0/51081650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5870d4820 con 0x55c5870d55f0 2024-04-25T08:54:57.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5dda6640 1 --2- 172.21.15.37:0/51081650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d55f0 0x55c5870d7630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fac58001670 tx=0x7fac58001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9450161ada78987b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac4f7fe640 1 -- 172.21.15.37:0/51081650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac580062f0 con 0x55c5870d55f0 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac4f7fe640 1 -- 172.21.15.37:0/51081650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac58003660 con 0x55c5870d55f0 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac4f7fe640 1 -- 172.21.15.37:0/51081650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac58003a30 con 0x55c5870d55f0 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 -- 172.21.15.37:0/51081650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d55f0 msgr2=0x55c5870d7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 --2- 172.21.15.37:0/51081650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d55f0 0x55c5870d7630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fac58001670 tx=0x7fac58001b30 comp rx=0 tx=0).stop 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 -- 172.21.15.37:0/51081650 shutdown_connections 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 --2- 172.21.15.37:0/51081650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d55f0 0x55c5870d7630 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 --2- 172.21.15.37:0/51081650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c5870d5210 0x55c5870d70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 --2- 172.21.15.37:0/51081650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c587048c40 0x55c5870d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 -- 172.21.15.37:0/51081650 >> 172.21.15.37:0/51081650 conn(0x55c58702aab0 msgr2=0x55c5870031d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 -- 172.21.15.37:0/51081650 shutdown_connections 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 -- 172.21.15.37:0/51081650 wait complete. 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 Processor -- start 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 -- start start 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c587048c40 0x55c587040600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d5210 0x55c5870d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5870d55f0 0x55c5870c0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5870d4ac0 con 0x55c5870d5210 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5870d81a0 con 0x55c5870d55f0 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5870d8400 con 0x55c587048c40 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5cda4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d5210 0x55c5870d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5cda4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d5210 0x55c5870d15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52510/0 (socket says 172.21.15.37:52510) 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5cda4640 1 -- 172.21.15.37:0/60192697 learned_addr learned my addr 172.21.15.37:0/60192697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5d5a5640 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c587048c40 0x55c587040600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5dda6640 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5870d55f0 0x55c5870c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5cda4640 1 -- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c587048c40 msgr2=0x55c587040600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5cda4640 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c587048c40 0x55c587040600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5cda4640 1 -- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5870d55f0 msgr2=0x55c5870c0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5cda4640 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5870d55f0 0x55c5870c0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5cda4640 1 -- 172.21.15.37:0/60192697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5870ded30 con 0x55c5870d5210 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5dda6640 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c5870d55f0 0x55c5870c0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5cda4640 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d5210 0x55c5870d15d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fac540ad040 tx=0x7fac54018160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac4e7fc640 1 -- 172.21.15.37:0/60192697 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540bb070 con 0x55c5870d5210 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac4e7fc640 1 -- 172.21.15.37:0/60192697 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac540b5860 con 0x55c5870d5210 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 -- 172.21.15.37:0/60192697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c5870defd0 con 0x55c5870d5210 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac4e7fc640 1 -- 172.21.15.37:0/60192697 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540af040 con 0x55c5870d5210 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.163+0000 7fac5fa87080 1 -- 172.21.15.37:0/60192697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5870df510 con 0x55c5870d5210 2024-04-25T08:54:57.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac4e7fc640 1 -- 172.21.15.37:0/60192697 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac540be070 con 0x55c5870d5210 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac4e7fc640 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac40065970 0x7fac40067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac4e7fc640 1 -- 172.21.15.37:0/60192697 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac5411f4c0 con 0x55c5870d5210 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c58716fe40 0x55c587170220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 -- 172.21.15.37:0/60192697 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55) v9 -- 0x55c5870c0940 con 0x55c58716fe40 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5d5a5640 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac40065970 0x7fac40067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5dda6640 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c58716fe40 0x55c587170220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5dda6640 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c58716fe40 0x55c587170220 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5dda6640 1 -- 172.21.15.37:0/60192697 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fac5800f030 con 0x55c58716fe40 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5d5a5640 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac40065970 0x7fac40067e30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fac50005740 tx=0x7fac500056d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 -- 172.21.15.37:0/60192697 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c58716fe40 msgr2=0x55c587170220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c58716fe40 0x55c587170220 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 -- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac40065970 msgr2=0x7fac40067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac40065970 0x7fac40067e30 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fac50005740 tx=0x7fac500056d0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 -- 172.21.15.37:0/60192697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d5210 msgr2=0x55c5870d15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 --2- 172.21.15.37:0/60192697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5870d5210 0x55c5870d15d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fac540ad040 tx=0x7fac54018160 comp rx=0 tx=0).stop 2024-04-25T08:54:57.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5dda6640 1 -- 172.21.15.37:0/60192697 reap_dead start 2024-04-25T08:54:57.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 -- 172.21.15.37:0/60192697 shutdown_connections 2024-04-25T08:54:57.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 -- 172.21.15.37:0/60192697 >> 172.21.15.37:0/60192697 conn(0x55c58702aab0 msgr2=0x55c587002fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 -- 172.21.15.37:0/60192697 shutdown_connections 2024-04-25T08:54:57.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.167+0000 7fac5fa87080 1 -- 172.21.15.37:0/60192697 wait complete. 2024-04-25T08:54:57.167 INFO:teuthology.orchestra.run.smithi037.stdout:Yalphaparent_a 2024-04-25T08:54:57.167 DEBUG:teuthology.orchestra.run.smithi037:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T08:54:57.259 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:54:57.259 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627777, 2024-04-25T08:54:57.259 INFO:teuthology.orchestra.run.smithi037.stdout: "ancestors": [ 2024-04-25T08:54:57.259 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:57.259 INFO:teuthology.orchestra.run.smithi037.stdout: "dirino": 1099511627776, 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout: "dname": "alpha", 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 2 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout: "dirino": 1, 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout: "dname": "parent_a", 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 4 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout: "pool": 8, 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout: "old_pools": [] 2024-04-25T08:54:57.260 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:54:57.261 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:57.262 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-04-25T08:54:57.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.343+0000 7f261cdbd080 1 Processor -- start 2024-04-25T08:54:57.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.343+0000 7f261cdbd080 1 -- start start 2024-04-25T08:54:57.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c69fd62570 0x55c69fe377a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c69fdb0360 0x55c69fe37ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 0x55c69fd90a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69fd099f0 con 0x55c69fe3a1f0 2024-04-25T08:54:57.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69fd0d950 con 0x55c69fdb0360 2024-04-25T08:54:57.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c69fd95660 con 0x55c69fd62570 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261b0dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 0x55c69fd90a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261b0dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 0x55c69fd90a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52518/0 (socket says 172.21.15.37:52518) 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261b0dc640 1 -- 172.21.15.37:0/609122497 learned_addr learned my addr 172.21.15.37:0/609122497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261a8db640 1 --2- 172.21.15.37:0/609122497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c69fd62570 0x55c69fe377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261a0da640 1 --2- 172.21.15.37:0/609122497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c69fdb0360 0x55c69fe37ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261b0dc640 1 -- 172.21.15.37:0/609122497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c69fd62570 msgr2=0x55c69fe377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261b0dc640 1 --2- 172.21.15.37:0/609122497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c69fd62570 0x55c69fe377a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261b0dc640 1 -- 172.21.15.37:0/609122497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c69fdb0360 msgr2=0x55c69fe37ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261b0dc640 1 --2- 172.21.15.37:0/609122497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c69fdb0360 0x55c69fe37ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261b0dc640 1 -- 172.21.15.37:0/609122497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c69fda60c0 con 0x55c69fe3a1f0 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261b0dc640 1 --2- 172.21.15.37:0/609122497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 0x55c69fd90a40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f261400b220 tx=0x7f261400b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a4e55dc30f73c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f26190d8640 1 -- 172.21.15.37:0/609122497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261400a040 con 0x55c69fe3a1f0 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f26190d8640 1 -- 172.21.15.37:0/609122497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261400f620 con 0x55c69fe3a1f0 2024-04-25T08:54:57.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f26190d8640 1 -- 172.21.15.37:0/609122497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261400f9f0 con 0x55c69fe3a1f0 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- 172.21.15.37:0/609122497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 msgr2=0x55c69fd90a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 --2- 172.21.15.37:0/609122497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 0x55c69fd90a40 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f261400b220 tx=0x7f261400b6e0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- 172.21.15.37:0/609122497 shutdown_connections 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 --2- 172.21.15.37:0/609122497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 0x55c69fd90a40 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 --2- 172.21.15.37:0/609122497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c69fdb0360 0x55c69fe37ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 --2- 172.21.15.37:0/609122497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c69fd62570 0x55c69fe377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- 172.21.15.37:0/609122497 >> 172.21.15.37:0/609122497 conn(0x55c69fd68c90 msgr2=0x55c69fd93dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- 172.21.15.37:0/609122497 shutdown_connections 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- 172.21.15.37:0/609122497 wait complete. 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 Processor -- start 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- start start 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c69fd62570 0x55c69fdac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c69fdb0360 0x55c69fda9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 0x55c69fdaa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69fda6360 con 0x55c69fe3a1f0 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c69fda6600 con 0x55c69fd62570 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.347+0000 7f261cdbd080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c69fd62230 con 0x55c69fdb0360 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261b0dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 0x55c69fdaa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261b0dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 0x55c69fdaa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52526/0 (socket says 172.21.15.37:52526) 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261b0dc640 1 -- 172.21.15.37:0/1817464461 learned_addr learned my addr 172.21.15.37:0/1817464461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261a8db640 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c69fd62570 0x55c69fdac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261a0da640 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c69fdb0360 0x55c69fda9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261b0dc640 1 -- 172.21.15.37:0/1817464461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c69fdb0360 msgr2=0x55c69fda9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261b0dc640 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c69fdb0360 0x55c69fda9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261b0dc640 1 -- 172.21.15.37:0/1817464461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c69fd62570 msgr2=0x55c69fdac720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261b0dc640 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c69fd62570 0x55c69fdac720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261b0dc640 1 -- 172.21.15.37:0/1817464461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c69fe18060 con 0x55c69fe3a1f0 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261b0dc640 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 0x55c69fdaa430 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f261400bbe0 tx=0x7f2614000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f2603fff640 1 -- 172.21.15.37:0/1817464461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614019880 con 0x55c69fe3a1f0 2024-04-25T08:54:57.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f2603fff640 1 -- 172.21.15.37:0/1817464461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614019a20 con 0x55c69fe3a1f0 2024-04-25T08:54:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261cdbd080 1 -- 172.21.15.37:0/1817464461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c69fe2e080 con 0x55c69fe3a1f0 2024-04-25T08:54:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f2603fff640 1 -- 172.21.15.37:0/1817464461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614019e10 con 0x55c69fe3a1f0 2024-04-25T08:54:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261cdbd080 1 -- 172.21.15.37:0/1817464461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c69fe2e5c0 con 0x55c69fe3a1f0 2024-04-25T08:54:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f2603fff640 1 -- 172.21.15.37:0/1817464461 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2614017040 con 0x55c69fe3a1f0 2024-04-25T08:54:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f2603fff640 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2608065970 0x7f2608067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f2603fff640 1 -- 172.21.15.37:0/1817464461 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2614085e00 con 0x55c69fe3a1f0 2024-04-25T08:54:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261cdbd080 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c69fe13270 0x55c69fe2ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261cdbd080 1 -- 172.21.15.37:0/1817464461 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e55) v9 -- 0x55c69fe2efa0 con 0x55c69fe13270 2024-04-25T08:54:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261a8db640 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2608065970 0x7f2608067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261a0da640 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c69fe13270 0x55c69fe2ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.351+0000 7f261a0da640 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c69fe13270 0x55c69fe2ea60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261a8db640 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2608065970 0x7f2608067e30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f260c0ad000 tx=0x7f260c0a5070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261a0da640 1 -- 172.21.15.37:0/1817464461 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v55'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f261000e850 con 0x55c69fe13270 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261cdbd080 1 -- 172.21.15.37:0/1817464461 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c69fe13270 msgr2=0x55c69fe2ea60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261cdbd080 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c69fe13270 0x55c69fe2ea60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261cdbd080 1 -- 172.21.15.37:0/1817464461 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2608065970 msgr2=0x7f2608067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261cdbd080 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2608065970 0x7f2608067e30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f260c0ad000 tx=0x7f260c0a5070 comp rx=0 tx=0).stop 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261cdbd080 1 -- 172.21.15.37:0/1817464461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 msgr2=0x55c69fdaa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261cdbd080 1 --2- 172.21.15.37:0/1817464461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c69fe3a1f0 0x55c69fdaa430 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f261400bbe0 tx=0x7f2614000ee0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261b0dc640 1 -- 172.21.15.37:0/1817464461 reap_dead start 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261cdbd080 1 -- 172.21.15.37:0/1817464461 shutdown_connections 2024-04-25T08:54:57.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261cdbd080 1 -- 172.21.15.37:0/1817464461 >> 172.21.15.37:0/1817464461 conn(0x55c69fd68c90 msgr2=0x55c69fd93dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261cdbd080 1 -- 172.21.15.37:0/1817464461 shutdown_connections 2024-04-25T08:54:57.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.355+0000 7f261cdbd080 1 -- 172.21.15.37:0/1817464461 wait complete. 2024-04-25T08:54:57.353 DEBUG:teuthology.orchestra.run.smithi037:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T08:54:57.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:57.400 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-04-25T08:54:57.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0f1f9a640 1 Processor -- start 2024-04-25T08:54:57.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0f1f9a640 1 -- start start 2024-04-25T08:54:57.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0f1f9a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe0ec15b350 0x7fe0ec15ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0f1f9a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe0ec06c3b0 0x7fe0ec06d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0f1f9a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0ec0cc450 con 0x7fe0ec066bb0 2024-04-25T08:54:57.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0f1f9a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ec0b31b0 con 0x7fe0ec06c3b0 2024-04-25T08:54:57.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0f1f9a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0ec0b2ed0 con 0x7fe0ec15b350 2024-04-25T08:54:57.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0ebfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe0ec06c3b0 0x7fe0ec06d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0e3fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe0ec066bb0 0x7fe0ec06bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44810/0 (socket says 172.21.15.37:44810) 2024-04-25T08:54:57.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0e3fff640 1 -- 172.21.15.37:0/4094417827 learned_addr learned my addr 172.21.15.37:0/4094417827 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:57.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/4094417827 >> v2:172.21.15.203:3300/0 conn(0x7fe0ec15b350 0x7fe0ec15ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0ebfff640 1 -- 172.21.15.37:0/4094417827 >> v2:172.21.15.203:3300/0 conn(0x7fe0ec15b350 msgr2=0x7fe0ec15ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/4094417827 >> v2:172.21.15.203:3300/0 conn(0x7fe0ec15b350 0x7fe0ec15ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0ebfff640 1 -- 172.21.15.37:0/4094417827 >> v1:172.21.15.37:6789/0 conn(0x7fe0ec066bb0 legacy=0x7fe0ec06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.539+0000 7fe0ebfff640 1 -- 172.21.15.37:0/4094417827 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec15df40 con 0x7fe0ec06c3b0 2024-04-25T08:54:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/4094417827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0ec06c3b0 0x7fe0ec06d850 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec072430 tx=0x7fe0e401c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b3430fe1239f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0eaffd640 1 -- 172.21.15.37:0/4094417827 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40b2070 con 0x7fe0ec06c3b0 2024-04-25T08:54:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0eaffd640 1 -- 172.21.15.37:0/4094417827 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e40a5820 con 0x7fe0ec06c3b0 2024-04-25T08:54:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0eaffd640 1 -- 172.21.15.37:0/4094417827 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40a5bf0 con 0x7fe0ec06c3b0 2024-04-25T08:54:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/4094417827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0ec06c3b0 msgr2=0x7fe0ec06d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0f1f9a640 1 --2- 172.21.15.37:0/4094417827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0ec06c3b0 0x7fe0ec06d850 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe0ec072430 tx=0x7fe0e401c7c0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/4094417827 shutdown_connections 2024-04-25T08:54:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0f1f9a640 1 --2- 172.21.15.37:0/4094417827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0ec06c3b0 0x7fe0ec06d850 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0f1f9a640 1 --2- 172.21.15.37:0/4094417827 >> v2:172.21.15.203:3300/0 conn(0x7fe0ec15b350 0x7fe0ec15ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/4094417827 >> 172.21.15.37:0/4094417827 conn(0x7fe0ec0717f0 msgr2=0x7fe0ec071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/4094417827 shutdown_connections 2024-04-25T08:54:57.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/4094417827 wait complete. 2024-04-25T08:54:57.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0f1f9a640 1 Processor -- start 2024-04-25T08:54:57.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.543+0000 7fe0f1f9a640 1 -- start start 2024-04-25T08:54:57.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0f1f9a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0ec066bb0 0x7fe0ec1473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0f1f9a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0ec06c3b0 0x7fe0ec15fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0f1f9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0ec15b350 0x7fe0ec15ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0f1f9a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec0b2bc0 con 0x7fe0ec15b350 2024-04-25T08:54:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0f1f9a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec15e500 con 0x7fe0ec066bb0 2024-04-25T08:54:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0f1f9a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec15b730 con 0x7fe0ec06c3b0 2024-04-25T08:54:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0ec15b350 0x7fe0ec15ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0ec15b350 0x7fe0ec15ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52544/0 (socket says 172.21.15.37:52544) 2024-04-25T08:54:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0ebfff640 1 -- 172.21.15.37:0/176492161 learned_addr learned my addr 172.21.15.37:0/176492161 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0ec066bb0 0x7fe0ec1473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0e3fff640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0ec06c3b0 0x7fe0ec15fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0ebfff640 1 -- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0ec06c3b0 msgr2=0x7fe0ec15fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0ec06c3b0 0x7fe0ec15fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0ebfff640 1 -- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0ec066bb0 msgr2=0x7fe0ec1473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0ec066bb0 0x7fe0ec1473e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0ebfff640 1 -- 172.21.15.37:0/176492161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ec0ce6f0 con 0x7fe0ec15b350 2024-04-25T08:54:57.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0ec15b350 0x7fe0ec15ff80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40b0040 tx=0x7fe0e4019560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0e8ff9640 1 -- 172.21.15.37:0/176492161 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40b2050 con 0x7fe0ec15b350 2024-04-25T08:54:57.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0e8ff9640 1 -- 172.21.15.37:0/176492161 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e40be070 con 0x7fe0ec15b350 2024-04-25T08:54:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/176492161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ec0ce9d0 con 0x7fe0ec15b350 2024-04-25T08:54:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0e8ff9640 1 -- 172.21.15.37:0/176492161 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e4009450 con 0x7fe0ec15b350 2024-04-25T08:54:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/176492161 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ec0cef90 con 0x7fe0ec15b350 2024-04-25T08:54:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0e8ff9640 1 -- 172.21.15.37:0/176492161 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe0e40c3460 con 0x7fe0ec15b350 2024-04-25T08:54:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.547+0000 7fe0e8ff9640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe0cc066f60 0x7fe0cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0e8ff9640 1 -- 172.21.15.37:0/176492161 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0e4120860 con 0x7fe0ec15b350 2024-04-25T08:54:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe0cc066f60 0x7fe0cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 Processor -- start 2024-04-25T08:54:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 -- start start 2024-04-25T08:54:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe0cc066f60 0x7fe0cc069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00b9d0 tx=0x7fe0dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62fd5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62fd55b0 0x555a62f8ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62f8b290 0x555a62f8b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a62e71f40 con 0x555a62fd4c90 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a62dd9070 con 0x555a62fd55b0 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a62d611a0 con 0x555a62f8b290 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62fd5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62fd5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52558/0 (socket says 172.21.15.37:52558) 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0eb7fe640 1 -- 172.21.15.37:0/2571462655 learned_addr learned my addr 172.21.15.37:0/2571462655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/2571462655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62f8b290 0x555a62f8b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0e3fff640 1 --2- 172.21.15.37:0/2571462655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62fd55b0 0x555a62f8ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0eb7fe640 1 -- 172.21.15.37:0/2571462655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62f8b290 msgr2=0x555a62f8b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/2571462655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62f8b290 0x555a62f8b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0eb7fe640 1 -- 172.21.15.37:0/2571462655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62fd55b0 msgr2=0x555a62f8ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/2571462655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62fd55b0 0x555a62f8ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0eb7fe640 1 -- 172.21.15.37:0/2571462655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a62d5f510 con 0x555a62fd4c90 2024-04-25T08:54:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/2571462655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62fd5070 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00fa70 tx=0x7fe0dc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1361fd36bf23732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0e17fa640 1 -- 172.21.15.37:0/2571462655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc01b050 con 0x555a62fd4c90 2024-04-25T08:54:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0e17fa640 1 -- 172.21.15.37:0/2571462655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc025070 con 0x555a62fd4c90 2024-04-25T08:54:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0e17fa640 1 -- 172.21.15.37:0/2571462655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0dc014de0 con 0x555a62fd4c90 2024-04-25T08:54:57.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 -- 172.21.15.37:0/2571462655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 msgr2=0x555a62fd5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 --2- 172.21.15.37:0/2571462655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62fd5070 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00fa70 tx=0x7fe0dc012000 comp rx=0 tx=0).stop 2024-04-25T08:54:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 -- 172.21.15.37:0/2571462655 shutdown_connections 2024-04-25T08:54:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 --2- 172.21.15.37:0/2571462655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62f8b290 0x555a62f8b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 --2- 172.21.15.37:0/2571462655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62fd55b0 0x555a62f8ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 --2- 172.21.15.37:0/2571462655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62fd5070 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 -- 172.21.15.37:0/2571462655 >> 172.21.15.37:0/2571462655 conn(0x555a62f87240 msgr2=0x555a62f89630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 -- 172.21.15.37:0/2571462655 shutdown_connections 2024-04-25T08:54:57.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.551+0000 7fe0f5107000 1 -- 172.21.15.37:0/2571462655 wait complete. 2024-04-25T08:54:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 -- start start 2024-04-25T08:54:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 Processor -- start 2024-04-25T08:54:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62f8b290 0x555a62f95e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62fd4c90 0x555a62f96370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62f968b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a62d936c0 con 0x555a62fd55b0 2024-04-25T08:54:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a62dd5110 con 0x555a62fd4c90 2024-04-25T08:54:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a62e8ff20 con 0x555a62f8b290 2024-04-25T08:54:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 10 client.0 _ll_register_callbacks cb 0x555a62f76570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:54:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62f968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62f968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52574/0 (socket says 172.21.15.37:52574) 2024-04-25T08:54:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ebfff640 1 -- 172.21.15.37:0/4192902514 learned_addr learned my addr 172.21.15.37:0/4192902514 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0e3fff640 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62fd4c90 0x555a62f96370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62f8b290 0x555a62f95e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ebfff640 1 -- 172.21.15.37:0/4192902514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62f8b290 msgr2=0x555a62f95e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62f8b290 0x555a62f95e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ebfff640 1 -- 172.21.15.37:0/4192902514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62fd4c90 msgr2=0x555a62f96370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62fd4c90 0x555a62f96370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ebfff640 1 -- 172.21.15.37:0/4192902514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a62d613b0 con 0x555a62fd55b0 2024-04-25T08:54:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62f8b290 0x555a62f95e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62f968b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40983c0 tx=0x7fe0e40f1640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2130213138e65081 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:54:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4192902514 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40f13a0 con 0x555a62fd55b0 2024-04-25T08:54:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4192902514 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0e4119d00 con 0x555a62fd55b0 2024-04-25T08:54:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 -- 172.21.15.37:0/4192902514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0dc025070 con 0x555a62fd55b0 2024-04-25T08:54:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4192902514 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0e40f5090 con 0x555a62fd55b0 2024-04-25T08:54:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 10 client.6162 fetch_fsmap learned FSMap version 25 2024-04-25T08:54:57.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 -- 172.21.15.37:0/4192902514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x555a62e5ae00 con 0x555a62fd55b0 2024-04-25T08:54:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4192902514 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0dc025070 con 0x555a62fd55b0 2024-04-25T08:54:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4192902514 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0e40f91c0 con 0x555a62fd55b0 2024-04-25T08:54:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 10 client.6162 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:54:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 10 client.6162 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:54:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 20 client.6162 populate_metadata read hostname 'smithi037' 2024-04-25T08:54:57.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x555a62f97150 0x555a62fe6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 4 client.6162 mds_command: new command op to 5937 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:54:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 -- 172.21.15.37:0/4192902514 --> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a62d74d60 con 0x555a62f97150 2024-04-25T08:54:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x555a62f97150 0x555a62fe6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ca7fc640 10 client.6162 ms_handle_connect on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x555a62f97150 0x555a62fe6410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aea883eb383a2346 server_cookie=54003c5ae6f56011 in_seq=0 out_seq=0 2024-04-25T08:54:57.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4192902514 <== mds.0 v2:172.21.15.37:6834/3892786810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555a62d74d60 con 0x555a62f97150 2024-04-25T08:54:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0ca7fc640 10 client.6162 handle_command_reply: tid=0 2024-04-25T08:54:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 client.6162 shutdown 2024-04-25T08:54:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 -- 172.21.15.37:0/4192902514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 msgr2=0x555a62f968b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.555+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62f968b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40983c0 tx=0x7fe0e40f1640 comp rx=0 tx=0).stop 2024-04-25T08:54:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7fe0f5107000 1 -- 172.21.15.37:0/4192902514 shutdown_connections 2024-04-25T08:54:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x555a62f97150 0x555a62fe6410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62f968b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7fe0ca7fc640 0 client.6162 ms_handle_reset on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62fd4c90 0x555a62f96370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4192902514 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62f8b290 0x555a62f95e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7fe0f5107000 1 -- 172.21.15.37:0/4192902514 >> 172.21.15.37:0/4192902514 conn(0x555a62f87240 msgr2=0x555a62f89ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462b301640 1 Processor -- start 2024-04-25T08:54:57.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7fe0f5107000 1 -- 172.21.15.37:0/4192902514 shutdown_connections 2024-04-25T08:54:57.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462b301640 1 -- start start 2024-04-25T08:54:57.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7fe0f5107000 1 -- 172.21.15.37:0/4192902514 wait complete. 2024-04-25T08:54:57.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7fe0f5107000 20 client.6162 trim_cache size 0 max 16384 2024-04-25T08:54:57.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462b301640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f462415abc0 0x7f46240af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462b301640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46240ae210 0x7f46240b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462b301640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46240afbe0 con 0x7f462415a7e0 2024-04-25T08:54:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462b301640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46240aeed0 con 0x7f462415abc0 2024-04-25T08:54:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462b301640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46240ae660 con 0x7f46240ae210 2024-04-25T08:54:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462989e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46240ae210 0x7f46240b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462989e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46240ae210 0x7f46240b3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:60782/0 (socket says 172.21.15.37:60782) 2024-04-25T08:54:57.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462989e640 1 -- 172.21.15.37:0/3500303216 learned_addr learned my addr 172.21.15.37:0/3500303216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462889c640 1 --2- 172.21.15.37:0/3500303216 >> v2:172.21.15.203:3300/0 conn(0x7f462415abc0 0x7f46240af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f461bfff640 1 -- 172.21.15.37:0/3500303216 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2629966378 0 0) 0x7f46240afbe0 con 0x7f462415a7e0 2024-04-25T08:54:57.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462989e640 1 -- 172.21.15.37:0/3500303216 >> v2:172.21.15.203:3300/0 conn(0x7f462415abc0 msgr2=0x7f46240af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462989e640 1 --2- 172.21.15.37:0/3500303216 >> v2:172.21.15.203:3300/0 conn(0x7f462415abc0 0x7f46240af270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462989e640 1 -- 172.21.15.37:0/3500303216 >> v1:172.21.15.203:6789/0 conn(0x7f462415a7e0 legacy=0x7f46240adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.559+0000 7f462989e640 1 -- 172.21.15.37:0/3500303216 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f462406ff20 con 0x7f46240ae210 2024-04-25T08:54:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462989e640 1 --2- 172.21.15.37:0/3500303216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46240ae210 0x7f46240b3310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f461c009e50 tx=0x7f461c01ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9123724917f0d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f461bfff640 1 -- 172.21.15.37:0/3500303216 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0a58d0 con 0x7f46240ae210 2024-04-25T08:54:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f461bfff640 1 -- 172.21.15.37:0/3500303216 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c0a5a70 con 0x7f46240ae210 2024-04-25T08:54:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f461bfff640 1 -- 172.21.15.37:0/3500303216 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c0a5e40 con 0x7f46240ae210 2024-04-25T08:54:57.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 -- 172.21.15.37:0/3500303216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46240ae210 msgr2=0x7f46240b3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 --2- 172.21.15.37:0/3500303216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46240ae210 0x7f46240b3310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f461c009e50 tx=0x7f461c01ca00 comp rx=0 tx=0).stop 2024-04-25T08:54:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 -- 172.21.15.37:0/3500303216 shutdown_connections 2024-04-25T08:54:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 --2- 172.21.15.37:0/3500303216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46240ae210 0x7f46240b3310 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 --2- 172.21.15.37:0/3500303216 >> v2:172.21.15.203:3300/0 conn(0x7f462415abc0 0x7f46240af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 -- 172.21.15.37:0/3500303216 >> 172.21.15.37:0/3500303216 conn(0x7f46240b5830 msgr2=0x7f46240b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 -- 172.21.15.37:0/3500303216 shutdown_connections 2024-04-25T08:54:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 -- 172.21.15.37:0/3500303216 wait complete. 2024-04-25T08:54:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 Processor -- start 2024-04-25T08:54:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 -- start start 2024-04-25T08:54:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46240ae210 0x7f46240b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462415a7e0 0x7f462413cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f462415abc0 0x7f4624155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46240b4060 con 0x7f46240ae210 2024-04-25T08:54:57.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46240aebf0 con 0x7f462415a7e0 2024-04-25T08:54:57.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462b301640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46240b41e0 con 0x7f462415abc0 2024-04-25T08:54:57.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462909d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46240ae210 0x7f46240b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462909d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46240ae210 0x7f46240b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52582/0 (socket says 172.21.15.37:52582) 2024-04-25T08:54:57.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462909d640 1 -- 172.21.15.37:0/84127720 learned_addr learned my addr 172.21.15.37:0/84127720 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462889c640 1 --2- 172.21.15.37:0/84127720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462415a7e0 0x7f462413cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462989e640 1 --2- 172.21.15.37:0/84127720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f462415abc0 0x7f4624155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.563+0000 7f462909d640 1 -- 172.21.15.37:0/84127720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f462415abc0 msgr2=0x7f4624155fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f462909d640 1 --2- 172.21.15.37:0/84127720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f462415abc0 0x7f4624155fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f462909d640 1 -- 172.21.15.37:0/84127720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462415a7e0 msgr2=0x7f462413cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f462909d640 1 --2- 172.21.15.37:0/84127720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f462415a7e0 0x7f462413cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f462909d640 1 -- 172.21.15.37:0/84127720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240afe70 con 0x7f46240ae210 2024-04-25T08:54:57.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f462909d640 1 --2- 172.21.15.37:0/84127720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46240ae210 0x7f46240b0c30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f462000c4b0 tx=0x7f462000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f4619ffb640 1 -- 172.21.15.37:0/84127720 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4620005480 con 0x7f46240ae210 2024-04-25T08:54:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f462b301640 1 -- 172.21.15.37:0/84127720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46240b0150 con 0x7f46240ae210 2024-04-25T08:54:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f4619ffb640 1 -- 172.21.15.37:0/84127720 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4620003600 con 0x7f46240ae210 2024-04-25T08:54:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f4619ffb640 1 -- 172.21.15.37:0/84127720 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46200039f0 con 0x7f46240ae210 2024-04-25T08:54:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f462b301640 1 -- 172.21.15.37:0/84127720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46240b1170 con 0x7f46240ae210 2024-04-25T08:54:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f462b301640 1 -- 172.21.15.37:0/84127720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f4624068340 con 0x7f46240ae210 2024-04-25T08:54:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f4619ffb640 1 -- 172.21.15.37:0/84127720 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4620003b90 con 0x7f46240ae210 2024-04-25T08:54:57.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f4619ffb640 1 --2- 172.21.15.37:0/84127720 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4608066f90 0x7f4608069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f4619ffb640 1 -- 172.21.15.37:0/84127720 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f46200858d0 con 0x7f46240ae210 2024-04-25T08:54:57.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f4619ffb640 1 -- 172.21.15.37:0/84127720 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+501 (secure 0 0 0) 0x7f46200862d0 con 0x7f46240ae210 2024-04-25T08:54:57.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f462889c640 1 --2- 172.21.15.37:0/84127720 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4608066f90 0x7f4608069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f462b301640 1 -- 172.21.15.37:0/84127720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f46240b5e60 con 0x7f46240ae210 2024-04-25T08:54:57.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.567+0000 7f462889c640 1 --2- 172.21.15.37:0/84127720 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4608066f90 0x7f4608069450 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4624156cc0 tx=0x7f4614008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 Processor -- start 2024-04-25T08:54:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 -- start start 2024-04-25T08:54:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62f8b290 0x555a62f8a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62f8ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62fd55b0 0x555a62fe6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec200110 con 0x555a62fd4c90 2024-04-25T08:54:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec1ff310 con 0x555a62f8b290 2024-04-25T08:54:57.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec15c560 con 0x555a62fd55b0 2024-04-25T08:54:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62f8ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62f8ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52594/0 (socket says 172.21.15.37:52594) 2024-04-25T08:54:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e3fff640 1 -- 172.21.15.37:0/1125318954 learned_addr learned my addr 172.21.15.37:0/1125318954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/1125318954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62f8b290 0x555a62f8a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/1125318954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62fd55b0 0x555a62fe6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e3fff640 1 -- 172.21.15.37:0/1125318954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62fd55b0 msgr2=0x555a62fe6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e3fff640 1 --2- 172.21.15.37:0/1125318954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62fd55b0 0x555a62fe6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e3fff640 1 -- 172.21.15.37:0/1125318954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62f8b290 msgr2=0x555a62f8a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e3fff640 1 --2- 172.21.15.37:0/1125318954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62f8b290 0x555a62f8a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e3fff640 1 -- 172.21.15.37:0/1125318954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a62e30c50 con 0x555a62fd4c90 2024-04-25T08:54:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/1125318954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62f8b290 0x555a62f8a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e3fff640 1 --2- 172.21.15.37:0/1125318954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62f8ab80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4001290 tx=0x7fe0d4007d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e04aec1f2ca274c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e0ff9640 1 -- 172.21.15.37:0/1125318954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4009630 con 0x555a62fd4c90 2024-04-25T08:54:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e0ff9640 1 -- 172.21.15.37:0/1125318954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d4018480 con 0x555a62fd4c90 2024-04-25T08:54:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0e0ff9640 1 -- 172.21.15.37:0/1125318954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0d4018830 con 0x555a62fd4c90 2024-04-25T08:54:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 -- 172.21.15.37:0/1125318954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 msgr2=0x555a62f8ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 --2- 172.21.15.37:0/1125318954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62f8ab80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe0d4001290 tx=0x7fe0d4007d80 comp rx=0 tx=0).stop 2024-04-25T08:54:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 -- 172.21.15.37:0/1125318954 shutdown_connections 2024-04-25T08:54:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 --2- 172.21.15.37:0/1125318954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62fd55b0 0x555a62fe6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 --2- 172.21.15.37:0/1125318954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd4c90 0x555a62f8ab80 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 --2- 172.21.15.37:0/1125318954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62f8b290 0x555a62f8a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 -- 172.21.15.37:0/1125318954 >> 172.21.15.37:0/1125318954 conn(0x555a62f87240 msgr2=0x555a62f8fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 -- 172.21.15.37:0/1125318954 shutdown_connections 2024-04-25T08:54:57.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.575+0000 7fe0f5107000 1 -- 172.21.15.37:0/1125318954 wait complete. 2024-04-25T08:54:57.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- start start 2024-04-25T08:54:57.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 Processor -- start 2024-04-25T08:54:57.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62f8b290 0x555a63035050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62fd4c90 0x555a63035590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62fefe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a62f90b70 con 0x555a62fd55b0 2024-04-25T08:54:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ec15cb60 con 0x555a62f8b290 2024-04-25T08:54:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a62f94200 con 0x555a62fd4c90 2024-04-25T08:54:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 10 client.0 _ll_register_callbacks cb 0x7fe0dc00b620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:54:57.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62fefe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62fefe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52602/0 (socket says 172.21.15.37:52602) 2024-04-25T08:54:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ebfff640 1 -- 172.21.15.37:0/4265930292 learned_addr learned my addr 172.21.15.37:0/4265930292 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0e3fff640 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62fd4c90 0x555a63035590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ebfff640 1 -- 172.21.15.37:0/4265930292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62fd4c90 msgr2=0x555a63035590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62f8b290 0x555a63035050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62fd4c90 0x555a63035590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ebfff640 1 -- 172.21.15.37:0/4265930292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62f8b290 msgr2=0x555a63035050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62f8b290 0x555a63035050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ebfff640 1 -- 172.21.15.37:0/4265930292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a62da1120 con 0x555a62fd55b0 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ebfff640 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62fefe90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40f9120 tx=0x7fe0e411ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332d6a9b08aed999 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4265930292 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e40f9650 con 0x555a62fd55b0 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4265930292 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe0e40f0390 con 0x555a62fd55b0 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- 172.21.15.37:0/4265930292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0d4018480 con 0x555a62fd55b0 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4265930292 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0e41181a0 con 0x555a62fd55b0 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 10 client.6171 fetch_fsmap learned FSMap version 25 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- 172.21.15.37:0/4265930292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x555a62d80170 con 0x555a62fd55b0 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4265930292 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0d4018480 con 0x555a62fd55b0 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4265930292 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0e4009030 con 0x555a62fd55b0 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 10 client.6171 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 10 client.6171 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 20 client.6171 populate_metadata read hostname 'smithi037' 2024-04-25T08:54:57.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x555a62ff0820 0x555a62ff0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 4 client.6171 mds_command: new command op to 5937 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- 172.21.15.37:0/4265930292 --> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x555a62de65b0 con 0x555a62ff0820 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x555a62ff0820 0x555a62ff0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0eb7fe640 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x555a62ff0820 0x555a62ff0c00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f94e6d4ab29cca8b server_cookie=e2b55dd95776e668 in_seq=0 out_seq=0 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ca7fc640 10 client.6171 ms_handle_connect on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ca7fc640 1 -- 172.21.15.37:0/4265930292 <== mds.0 v2:172.21.15.37:6834/3892786810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555a62de65b0 con 0x555a62ff0820 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ca7fc640 10 client.6171 handle_command_reply: tid=0 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 client.6171 shutdown 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- 172.21.15.37:0/4265930292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 msgr2=0x555a62fefe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62fefe90 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40f9120 tx=0x7fe0e411ea30 comp rx=0 tx=0).stop 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- 172.21.15.37:0/4265930292 shutdown_connections 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x555a62ff0820 0x555a62ff0c00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555a62fd55b0 0x555a62fefe90 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555a62fd4c90 0x555a63035590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 --2- 172.21.15.37:0/4265930292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555a62f8b290 0x555a63035050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0ca7fc640 0 client.6171 ms_handle_reset on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- 172.21.15.37:0/4265930292 >> 172.21.15.37:0/4265930292 conn(0x555a62f87240 msgr2=0x555a62f89630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- 172.21.15.37:0/4265930292 shutdown_connections 2024-04-25T08:54:57.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.579+0000 7fe0f5107000 1 -- 172.21.15.37:0/4265930292 wait complete. 2024-04-25T08:54:57.607 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T08:54:57.583+0000 7f838667b640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6120=0-4194304@1} caps={6120=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6120 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x56373e4ab700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:54:57.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f5107000 20 client.6171 trim_cache size 0 max 16384 2024-04-25T08:54:57.608 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:54:57.608 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-04-25T08:54:57.608 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "ce0464eb-fac5-4856-b9ee-6433c89179f5", 2024-04-25T08:54:57.608 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-04-25T08:54:57.608 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:54:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe0cc066f60 msgr2=0x7fe0cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe0cc066f60 0x7fe0cc069420 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe0dc00b9d0 tx=0x7fe0dc008040 comp rx=0 tx=0).stop 2024-04-25T08:54:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/176492161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0ec15b350 msgr2=0x7fe0ec15ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0ec15b350 0x7fe0ec15ff80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe0e40b0040 tx=0x7fe0e4019560 comp rx=0 tx=0).stop 2024-04-25T08:54:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/176492161 shutdown_connections 2024-04-25T08:54:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe0cc066f60 0x7fe0cc069420 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0ec15b350 0x7fe0ec15ff80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0ec06c3b0 0x7fe0ec15fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 --2- 172.21.15.37:0/176492161 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0ec066bb0 0x7fe0ec1473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/176492161 >> 172.21.15.37:0/176492161 conn(0x7fe0ec0717f0 msgr2=0x7fe0ec161390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/176492161 shutdown_connections 2024-04-25T08:54:57.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.583+0000 7fe0f1f9a640 1 -- 172.21.15.37:0/176492161 wait complete. 2024-04-25T08:54:57.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:57.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:54:57.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.735+0000 7f65735b7640 1 Processor -- start 2024-04-25T08:54:57.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f65735b7640 1 -- start start 2024-04-25T08:54:57.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f65735b7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f656c0cfca0 0x7f656c166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f65735b7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c0cbd20 con 0x7f656c15b800 2024-04-25T08:54:57.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f65735b7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f656c0cc000 con 0x7f656c15b420 2024-04-25T08:54:57.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f65735b7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656c0cc930 con 0x7f656c0cfca0 2024-04-25T08:54:57.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f6570b52640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f656c15b800 0x7f656c0cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44812/0 (socket says 172.21.15.37:44812) 2024-04-25T08:54:57.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f6570b52640 1 -- 172.21.15.37:0/1479905576 learned_addr learned my addr 172.21.15.37:0/1479905576 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:54:57.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f6571b54640 1 --2- 172.21.15.37:0/1479905576 >> v2:172.21.15.37:3300/0 conn(0x7f656c0cfca0 0x7f656c166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f6563fff640 1 -- 172.21.15.37:0/1479905576 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441821479 0 0) 0x7f656c0cbd20 con 0x7f656c15b800 2024-04-25T08:54:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f6563fff640 1 -- 172.21.15.37:0/1479905576 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6558003530 con 0x7f656c15b800 2024-04-25T08:54:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f6571b54640 1 -- 172.21.15.37:0/1479905576 >> v1:172.21.15.203:6789/0 conn(0x7f656c15b420 legacy=0x7f656c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f6571b54640 1 -- 172.21.15.37:0/1479905576 >> v1:172.21.15.37:6789/0 conn(0x7f656c15b800 legacy=0x7f656c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.739+0000 7f6571b54640 1 -- 172.21.15.37:0/1479905576 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c0cc2e0 con 0x7f656c0cfca0 2024-04-25T08:54:57.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f6571b54640 1 --2- 172.21.15.37:0/1479905576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c0cfca0 0x7f656c166290 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f656c0ccab0 tx=0x7f656401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9a7c8272607cde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f6563fff640 1 -- 172.21.15.37:0/1479905576 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65640b2070 con 0x7f656c0cfca0 2024-04-25T08:54:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f6563fff640 1 -- 172.21.15.37:0/1479905576 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65640a58a0 con 0x7f656c0cfca0 2024-04-25T08:54:57.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f6563fff640 1 -- 172.21.15.37:0/1479905576 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65640a5c70 con 0x7f656c0cfca0 2024-04-25T08:54:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 -- 172.21.15.37:0/1479905576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c0cfca0 msgr2=0x7f656c166290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 --2- 172.21.15.37:0/1479905576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c0cfca0 0x7f656c166290 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f656c0ccab0 tx=0x7f656401c950 comp rx=0 tx=0).stop 2024-04-25T08:54:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 -- 172.21.15.37:0/1479905576 shutdown_connections 2024-04-25T08:54:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 --2- 172.21.15.37:0/1479905576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c0cfca0 0x7f656c166290 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 -- 172.21.15.37:0/1479905576 >> 172.21.15.37:0/1479905576 conn(0x7f656c06ae60 msgr2=0x7f656c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 -- 172.21.15.37:0/1479905576 shutdown_connections 2024-04-25T08:54:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 -- 172.21.15.37:0/1479905576 wait complete. 2024-04-25T08:54:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 Processor -- start 2024-04-25T08:54:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 -- start start 2024-04-25T08:54:57.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f656c0cfca0 0x7f656c098b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c15b420 0x7f656c06dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f656c15b800 0x7f656c0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c169170 con 0x7f656c15b420 2024-04-25T08:54:57.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c168ff0 con 0x7f656c0cfca0 2024-04-25T08:54:57.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f65735b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1692f0 con 0x7f656c15b800 2024-04-25T08:54:57.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f6570b52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c15b420 0x7f656c06dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f6570b52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c15b420 0x7f656c06dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52610/0 (socket says 172.21.15.37:52610) 2024-04-25T08:54:57.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.743+0000 7f6570b52640 1 -- 172.21.15.37:0/2754081158 learned_addr learned my addr 172.21.15.37:0/2754081158 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6571353640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f656c0cfca0 0x7f656c098b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6571b54640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f656c15b800 0x7f656c0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6570b52640 1 -- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f656c15b800 msgr2=0x7f656c0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6570b52640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f656c15b800 0x7f656c0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6570b52640 1 -- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f656c0cfca0 msgr2=0x7f656c098b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6570b52640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f656c0cfca0 0x7f656c098b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6570b52640 1 -- 172.21.15.37:0/2754081158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656c06cf50 con 0x7f656c15b420 2024-04-25T08:54:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6571353640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f656c0cfca0 0x7f656c098b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6570b52640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c15b420 0x7f656c06dcd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f655c00c450 tx=0x7f655c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6561ffb640 1 -- 172.21.15.37:0/2754081158 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c017070 con 0x7f656c15b420 2024-04-25T08:54:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6561ffb640 1 -- 172.21.15.37:0/2754081158 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c005cf0 con 0x7f656c15b420 2024-04-25T08:54:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f65735b7640 1 -- 172.21.15.37:0/2754081158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656c0cedf0 con 0x7f656c15b420 2024-04-25T08:54:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6561ffb640 1 -- 172.21.15.37:0/2754081158 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c007270 con 0x7f656c15b420 2024-04-25T08:54:57.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f65735b7640 1 -- 172.21.15.37:0/2754081158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c06d4d0 con 0x7f656c15b420 2024-04-25T08:54:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6561ffb640 1 -- 172.21.15.37:0/2754081158 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f655c01a070 con 0x7f656c15b420 2024-04-25T08:54:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6561ffb640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6550066f60 0x7f6550069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6561ffb640 1 -- 172.21.15.37:0/2754081158 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f655c084360 con 0x7f656c15b420 2024-04-25T08:54:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6571353640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6550066f60 0x7f6550069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6576724000 1 Processor -- start 2024-04-25T08:54:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6576724000 1 -- start start 2024-04-25T08:54:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.747+0000 7f6571353640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6550066f60 0x7f6550069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6568001710 tx=0x7f6568009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b04adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b04b2f0 0x56356b000b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b0010a0 0x56356b001480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356ae4dff0 con 0x56356b04a9d0 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356add6150 con 0x56356b0010a0 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56356ae4a090 con 0x56356b04b2f0 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6571353640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b04adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6571353640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b04adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52612/0 (socket says 172.21.15.37:52612) 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6571353640 1 -- 172.21.15.37:0/3626028312 learned_addr learned my addr 172.21.15.37:0/3626028312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6570b52640 1 --2- 172.21.15.37:0/3626028312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b04b2f0 0x56356b000b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6571b54640 1 --2- 172.21.15.37:0/3626028312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b0010a0 0x56356b001480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6571353640 1 -- 172.21.15.37:0/3626028312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b04b2f0 msgr2=0x56356b000b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6571353640 1 --2- 172.21.15.37:0/3626028312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b04b2f0 0x56356b000b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6571353640 1 -- 172.21.15.37:0/3626028312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b0010a0 msgr2=0x56356b001480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6571353640 1 --2- 172.21.15.37:0/3626028312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b0010a0 0x56356b001480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6571353640 1 -- 172.21.15.37:0/3626028312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56356afe40d0 con 0x56356b04a9d0 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6570b52640 1 --2- 172.21.15.37:0/3626028312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b04b2f0 0x56356b000b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6571353640 1 --2- 172.21.15.37:0/3626028312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b04adb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6568001540 tx=0x7f6568011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd0a3f45635d40d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6556ffd640 1 -- 172.21.15.37:0/3626028312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568003600 con 0x56356b04a9d0 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6556ffd640 1 -- 172.21.15.37:0/3626028312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65680037a0 con 0x56356b04a9d0 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6556ffd640 1 -- 172.21.15.37:0/3626028312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568003b70 con 0x56356b04a9d0 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- 172.21.15.37:0/3626028312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 msgr2=0x56356b04adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 --2- 172.21.15.37:0/3626028312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b04adb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6568001540 tx=0x7f6568011000 comp rx=0 tx=0).stop 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- 172.21.15.37:0/3626028312 shutdown_connections 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 --2- 172.21.15.37:0/3626028312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b0010a0 0x56356b001480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 --2- 172.21.15.37:0/3626028312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b04b2f0 0x56356b000b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 --2- 172.21.15.37:0/3626028312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b04adb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- 172.21.15.37:0/3626028312 >> 172.21.15.37:0/3626028312 conn(0x56356affd020 msgr2=0x56356afff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- 172.21.15.37:0/3626028312 shutdown_connections 2024-04-25T08:54:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- 172.21.15.37:0/3626028312 wait complete. 2024-04-25T08:54:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- start start 2024-04-25T08:54:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 Processor -- start 2024-04-25T08:54:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b0010a0 0x56356b02f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b02fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b04b2f0 0x56356b030270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356ae12250 con 0x56356b04a9d0 2024-04-25T08:54:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356ae08690 con 0x56356b04b2f0 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56356add6a40 con 0x56356b0010a0 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6576724000 10 client.0 _ll_register_callbacks cb 0x56356afebe90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6570b52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b02fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6570b52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b02fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52614/0 (socket says 172.21.15.37:52614) 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6570b52640 1 -- 172.21.15.37:0/3051615599 learned_addr learned my addr 172.21.15.37:0/3051615599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.751+0000 7f6571353640 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b0010a0 0x56356b02f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6571b54640 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b04b2f0 0x56356b030270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6570b52640 1 -- 172.21.15.37:0/3051615599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b0010a0 msgr2=0x56356b02f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6570b52640 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b0010a0 0x56356b02f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6570b52640 1 -- 172.21.15.37:0/3051615599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b04b2f0 msgr2=0x56356b030270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6570b52640 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b04b2f0 0x56356b030270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6570b52640 1 -- 172.21.15.37:0/3051615599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56356aefb1c0 con 0x56356b04a9d0 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6570b52640 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b02fd30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f655c0011f0 tx=0x7f655c058d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaecf5a0aa33e501 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f654bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f654bfff640 1 -- 172.21.15.37:0/3051615599 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c05cfd0 con 0x56356b04a9d0 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f654bfff640 1 -- 172.21.15.37:0/3051615599 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f655c059bc0 con 0x56356b04a9d0 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 -- 172.21.15.37:0/3051615599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f65680037a0 con 0x56356b04a9d0 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f654bfff640 1 -- 172.21.15.37:0/3051615599 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f655c059de0 con 0x56356b04a9d0 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 10 client.6186 fetch_fsmap learned FSMap version 25 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 -- 172.21.15.37:0/3051615599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x56356aef7170 con 0x56356b04a9d0 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f654bfff640 1 -- 172.21.15.37:0/3051615599 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65680037a0 con 0x56356b04a9d0 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f654bfff640 1 -- 172.21.15.37:0/3051615599 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f655c054050 con 0x56356b04a9d0 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 10 client.6186 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 10 client.6186 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 20 client.6186 populate_metadata read hostname 'smithi037' 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x56356b05ac80 0x56356b05d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 4 client.6186 mds_command: new command op to 5937 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 -- 172.21.15.37:0/3051615599 --> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56356ade9c60 con 0x56356b05ac80 2024-04-25T08:54:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6571353640 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x56356b05ac80 0x56356b05d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f654bfff640 10 client.6186 ms_handle_connect on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6571353640 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x56356b05ac80 0x56356b05d070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3619c348d26188e5 server_cookie=3c28e7715242bc51 in_seq=0 out_seq=0 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f654bfff640 1 -- 172.21.15.37:0/3051615599 <== mds.0 v2:172.21.15.37:6834/3892786810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56356ade9c60 con 0x56356b05ac80 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f654bfff640 10 client.6186 handle_command_reply: tid=0 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 client.6186 shutdown 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 -- 172.21.15.37:0/3051615599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 msgr2=0x56356b02fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b02fd30 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f655c0011f0 tx=0x7f655c058d60 comp rx=0 tx=0).stop 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 -- 172.21.15.37:0/3051615599 shutdown_connections 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x56356b05ac80 0x56356b05d070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b04b2f0 0x56356b030270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b02fd30 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f654bfff640 0 client.6186 ms_handle_reset on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 --2- 172.21.15.37:0/3051615599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b0010a0 0x56356b02f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 -- 172.21.15.37:0/3051615599 >> 172.21.15.37:0/3051615599 conn(0x56356affd020 msgr2=0x56356afffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 -- 172.21.15.37:0/3051615599 shutdown_connections 2024-04-25T08:54:57.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 1 -- 172.21.15.37:0/3051615599 wait complete. 2024-04-25T08:54:57.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.755+0000 7f6576724000 20 client.6186 trim_cache size 0 max 16384 2024-04-25T08:54:57.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 Processor -- start 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 -- start start 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b0010a0 0x56356b0a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b0a7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b04b2f0 0x56356b0a7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c203250 con 0x56356b04a9d0 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c2023d0 con 0x56356b04b2f0 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1fa4e0 con 0x56356b0010a0 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6570b52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b0a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6570b52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b0a7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52616/0 (socket says 172.21.15.37:52616) 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6570b52640 1 -- 172.21.15.37:0/3454328512 learned_addr learned my addr 172.21.15.37:0/3454328512 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6570b52640 1 -- 172.21.15.37:0/3454328512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b0010a0 msgr2=0x56356b0a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6570b52640 1 --2- 172.21.15.37:0/3454328512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b0010a0 0x56356b0a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6570b52640 1 -- 172.21.15.37:0/3454328512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b04b2f0 msgr2=0x56356b0a7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6570b52640 1 --2- 172.21.15.37:0/3454328512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b04b2f0 0x56356b0a7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6570b52640 1 -- 172.21.15.37:0/3454328512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56356ae94de0 con 0x56356b04a9d0 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6570b52640 1 --2- 172.21.15.37:0/3454328512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b0a7630 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f655c00ce10 tx=0x7f655c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa3b011886a8db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f65567fc640 1 -- 172.21.15.37:0/3454328512 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c07e020 con 0x56356b04a9d0 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f65567fc640 1 -- 172.21.15.37:0/3454328512 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c05d260 con 0x56356b04a9d0 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f65567fc640 1 -- 172.21.15.37:0/3454328512 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c082240 con 0x56356b04a9d0 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 -- 172.21.15.37:0/3454328512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 msgr2=0x56356b0a7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 --2- 172.21.15.37:0/3454328512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b0a7630 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f655c00ce10 tx=0x7f655c014000 comp rx=0 tx=0).stop 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 -- 172.21.15.37:0/3454328512 shutdown_connections 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 --2- 172.21.15.37:0/3454328512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b04b2f0 0x56356b0a7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 --2- 172.21.15.37:0/3454328512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04a9d0 0x56356b0a7630 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 --2- 172.21.15.37:0/3454328512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b0010a0 0x56356b0a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.771+0000 7f6576724000 1 -- 172.21.15.37:0/3454328512 >> 172.21.15.37:0/3454328512 conn(0x56356affd020 msgr2=0x56356b057fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- 172.21.15.37:0/3454328512 shutdown_connections 2024-04-25T08:54:57.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- 172.21.15.37:0/3454328512 wait complete. 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- start start 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 Processor -- start 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b0010a0 0x56356b05d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b04a9d0 0x56356b05d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04b2f0 0x56356b05de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56356ade8e00 con 0x56356b04b2f0 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c201dd0 con 0x56356b0010a0 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56356ae4d300 con 0x56356b04a9d0 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 10 client.0 _ll_register_callbacks cb 0x56356afcc040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571b54640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04b2f0 0x56356b05de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571b54640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04b2f0 0x56356b05de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52626/0 (socket says 172.21.15.37:52626) 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571b54640 1 -- 172.21.15.37:0/2169889933 learned_addr learned my addr 172.21.15.37:0/2169889933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6570b52640 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b04a9d0 0x56356b05d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571353640 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b0010a0 0x56356b05d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571b54640 1 -- 172.21.15.37:0/2169889933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b04a9d0 msgr2=0x56356b05d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571b54640 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b04a9d0 0x56356b05d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571b54640 1 -- 172.21.15.37:0/2169889933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b0010a0 msgr2=0x56356b05d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571b54640 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b0010a0 0x56356b05d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571b54640 1 -- 172.21.15.37:0/2169889933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56356adef8e0 con 0x56356b04b2f0 2024-04-25T08:54:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571353640 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b0010a0 0x56356b05d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571b54640 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04b2f0 0x56356b05de00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f65640b0040 tx=0x7f65640196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5b5978bd32dfcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f654bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:54:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f654bfff640 1 -- 172.21.15.37:0/2169889933 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65640b2040 con 0x56356b04b2f0 2024-04-25T08:54:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f654bfff640 1 -- 172.21.15.37:0/2169889933 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(55..55 src has 1..55) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f65640a5da0 con 0x56356b04b2f0 2024-04-25T08:54:57.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- 172.21.15.37:0/2169889933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f655c05d260 con 0x56356b04b2f0 2024-04-25T08:54:57.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f654bfff640 1 -- 172.21.15.37:0/2169889933 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6564009450 con 0x56356b04b2f0 2024-04-25T08:54:57.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 10 client.6192 fetch_fsmap learned FSMap version 25 2024-04-25T08:54:57.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- 172.21.15.37:0/2169889933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x56356aea78f0 con 0x56356b04b2f0 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f654bfff640 1 -- 172.21.15.37:0/2169889933 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c05d260 con 0x56356b04b2f0 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f654bfff640 1 -- 172.21.15.37:0/2169889933 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f65640c9050 con 0x56356b04b2f0 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 10 client.6192 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 10 client.6192 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 20 client.6192 populate_metadata read hostname 'smithi037' 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x56356b0b0f50 0x56356b0b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 4 client.6192 mds_command: new command op to 5937 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- 172.21.15.37:0/2169889933 --> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56356ae5b530 con 0x56356b0b0f50 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571353640 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x56356b0b0f50 0x56356b0b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f654bfff640 10 client.6192 ms_handle_connect on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6571353640 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x56356b0b0f50 0x56356b0b3340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd3360dd3ba9022 server_cookie=4a7b9f5a4a487103 in_seq=0 out_seq=0 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f654bfff640 1 -- 172.21.15.37:0/2169889933 <== mds.0 v2:172.21.15.37:6834/3892786810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x56356ae5b530 con 0x56356b0b0f50 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f654bfff640 10 client.6192 handle_command_reply: tid=0 2024-04-25T08:54:57.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 client.6192 shutdown 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- 172.21.15.37:0/2169889933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04b2f0 msgr2=0x56356b05de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04b2f0 0x56356b05de00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f65640b0040 tx=0x7f65640196f0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- 172.21.15.37:0/2169889933 shutdown_connections 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x56356b0b0f50 0x56356b0b3340 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56356b04b2f0 0x56356b05de00 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56356b04a9d0 0x56356b05d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f654bfff640 0 client.6192 ms_handle_reset on v2:172.21.15.37:6834/3892786810 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 --2- 172.21.15.37:0/2169889933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56356b0010a0 0x56356b05d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- 172.21.15.37:0/2169889933 >> 172.21.15.37:0/2169889933 conn(0x56356affd020 msgr2=0x56356b0490d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- 172.21.15.37:0/2169889933 shutdown_connections 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.775+0000 7f6576724000 1 -- 172.21.15.37:0/2169889933 wait complete. 2024-04-25T08:54:57.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f6576724000 20 client.6192 trim_cache size 0 max 16384 2024-04-25T08:54:57.779 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:54:57.779 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:54:57.779 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-04-25T08:54:57.779 INFO:teuthology.orchestra.run.smithi037.stdout: "ce0464eb-fac5-4856-b9ee-6433c89179f5": { 2024-04-25T08:54:57.779 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-04-25T08:54:57.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "ce0464eb-fac5-4856-b9ee-6433c89179f5", 2024-04-25T08:54:57.779 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,repair" 2024-04-25T08:54:57.779 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:57.779 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:54:57.780 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:54:57.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 -- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6550066f60 msgr2=0x7f6550069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6550066f60 0x7f6550069420 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6568001710 tx=0x7f6568009040 comp rx=0 tx=0).stop 2024-04-25T08:54:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 -- 172.21.15.37:0/2754081158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c15b420 msgr2=0x7f656c06dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c15b420 0x7f656c06dcd0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f655c00c450 tx=0x7f655c00c910 comp rx=0 tx=0).stop 2024-04-25T08:54:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 -- 172.21.15.37:0/2754081158 shutdown_connections 2024-04-25T08:54:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6550066f60 0x7f6550069420 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6568001710 tx=0x7f6568009040 comp rx=0 tx=0).stop 2024-04-25T08:54:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f656c15b800 0x7f656c0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f656c15b420 0x7f656c06dcd0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 --2- 172.21.15.37:0/2754081158 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f656c0cfca0 0x7f656c098b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 -- 172.21.15.37:0/2754081158 >> 172.21.15.37:0/2754081158 conn(0x7f656c06ae60 msgr2=0x7f656c15f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 -- 172.21.15.37:0/2754081158 shutdown_connections 2024-04-25T08:54:57.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:57.779+0000 7f65735b7640 1 -- 172.21.15.37:0/2754081158 wait complete. 2024-04-25T08:54:57.788 INFO:tasks.cephfs.filesystem:scrub status for tag:ce0464eb-fac5-4856-b9ee-6433c89179f5 - {'path': '/', 'tag': 'ce0464eb-fac5-4856-b9ee-6433c89179f5', 'options': 'recursive,repair'} 2024-04-25T08:54:59.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:54:59.247+0000 7f4619ffb640 1 -- 172.21.15.37:0/84127720 <== mon.0 v2:172.21.15.37:3300/0 7 ==== log(6 entries from seq 1 at 2024-04-25T08:54:57.586414+0000) v1 ==== 1078+0+0 (secure 0 0 0) 0x7f4620083d80 con 0x7f46240ae210 2024-04-25T08:54:59.243 INFO:teuthology.orchestra.run.smithi037.stdout: cluster: 2024-04-25T08:54:59.243 INFO:teuthology.orchestra.run.smithi037.stdout: id: 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T08:54:59.243 INFO:teuthology.orchestra.run.smithi037.stdout: health: HEALTH_OK 2024-04-25T08:54:59.243 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:59.243 INFO:teuthology.orchestra.run.smithi037.stdout: services: 2024-04-25T08:54:59.243 INFO:teuthology.orchestra.run.smithi037.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2024-04-25T08:54:59.243 INFO:teuthology.orchestra.run.smithi037.stdout: mgr: x(active, since 2m), standbys: y 2024-04-25T08:54:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T08:54:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-04-25T08:54:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: data: 2024-04-25T08:54:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: volumes: 1/1 healthy 2024-04-25T08:54:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: pools: 3 pools, 97 pgs 2024-04-25T08:54:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: objects: 24 objects, 579 KiB 2024-04-25T08:54:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: usage: 230 MiB used, 720 GiB / 720 GiB avail 2024-04-25T08:54:59.244 INFO:teuthology.orchestra.run.smithi037.stdout: pgs: 97 active+clean 2024-04-25T08:54:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: io: 2024-04-25T08:54:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: client: 1.2 KiB/s wr, 0 op/s rd, 3 op/s wr 2024-04-25T08:54:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:59.245 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:54:59.245 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T08:54:57.586414+0000 mds.a [INF] scrub queued for path: / 2024-04-25T08:54:59.245 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T08:54:57.586417+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T08:54:59.245 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T08:54:57.586424+0000 mds.a [INF] scrub summary: active paths [/] 2024-04-25T08:54:59.245 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T08:54:57.587360+0000 mds.a [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-04-25T08:54:59.246 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T08:54:57.587370+0000 mds.a [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-04-25T08:54:59.246 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T08:54:57.588268+0000 mds.a [INF] scrub summary: idle+waiting paths [/] 2024-04-25T08:55:03.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:03.267+0000 7f4619ffb640 1 -- 172.21.15.37:0/84127720 <== mon.0 v2:172.21.15.37:3300/0 8 ==== log(1 entries from seq 7 at 2024-04-25T08:55:02.458294+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f4620086000 con 0x7f46240ae210 2024-04-25T08:55:03.259 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T08:55:02.458294+0000 mds.a [INF] scrub summary: idle 2024-04-25T08:55:11.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:11.299+0000 7f4619ffb640 1 -- 172.21.15.37:0/84127720 <== mon.0 v2:172.21.15.37:3300/0 9 ==== log(4 entries from seq 117 at 2024-04-25T08:55:11.279524+0000) v1 ==== 1120+0+0 (secure 0 0 0) 0x7f4620054070 con 0x7f46240ae210 2024-04-25T08:55:12.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:12.299+0000 7f4619ffb640 1 -- 172.21.15.37:0/84127720 <== mon.0 v2:172.21.15.37:3300/0 10 ==== log(2 entries from seq 522 at 2024-04-25T08:55:11.286302+0000) v1 ==== 557+0+0 (secure 0 0 0) 0x7f4620074100 con 0x7f46240ae210 2024-04-25T08:55:25.543 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:55:25.547 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:55:27.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:27.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:55:27.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.935+0000 7f5ba9928640 1 Processor -- start 2024-04-25T08:55:27.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.935+0000 7f5ba9928640 1 -- start start 2024-04-25T08:55:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba9928640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5ba4159bd0 0x7f5ba40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba9928640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5ba40b28c0 0x7f5ba4163960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba9928640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ba40c9ef0 con 0x7f5ba4159fb0 2024-04-25T08:55:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba9928640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba4072780 con 0x7f5ba40b28c0 2024-04-25T08:55:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba9928640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ba4071b70 con 0x7f5ba4159bd0 2024-04-25T08:55:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5ba40b28c0 0x7f5ba4163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba27fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5ba4159fb0 0x7f5ba40b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:54152/0 (socket says 172.21.15.37:54152) 2024-04-25T08:55:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba27fc640 1 -- 172.21.15.37:0/4135584449 learned_addr learned my addr 172.21.15.37:0/4135584449 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:27.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/4135584449 >> v2:172.21.15.203:3300/0 conn(0x7f5ba4159bd0 0x7f5ba40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba37fe640 1 -- 172.21.15.37:0/4135584449 >> v2:172.21.15.203:3300/0 conn(0x7f5ba4159bd0 msgr2=0x7f5ba40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:27.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/4135584449 >> v2:172.21.15.203:3300/0 conn(0x7f5ba4159bd0 0x7f5ba40c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba37fe640 1 -- 172.21.15.37:0/4135584449 >> v1:172.21.15.37:6789/0 conn(0x7f5ba4159fb0 legacy=0x7f5ba40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:27.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba37fe640 1 -- 172.21.15.37:0/4135584449 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba4071e50 con 0x7f5ba40b28c0 2024-04-25T08:55:27.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/4135584449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ba40b28c0 0x7f5ba4163960 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40cf2c0 tx=0x7f5b9401d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c446f8b30df0ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba1ffb640 1 -- 172.21.15.37:0/4135584449 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940ab430 con 0x7f5ba40b28c0 2024-04-25T08:55:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba1ffb640 1 -- 172.21.15.37:0/4135584449 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b940b0070 con 0x7f5ba40b28c0 2024-04-25T08:55:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.939+0000 7f5ba1ffb640 1 -- 172.21.15.37:0/4135584449 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940ab7f0 con 0x7f5ba40b28c0 2024-04-25T08:55:27.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 -- 172.21.15.37:0/4135584449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ba40b28c0 msgr2=0x7f5ba4163960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 --2- 172.21.15.37:0/4135584449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ba40b28c0 0x7f5ba4163960 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f5ba40cf2c0 tx=0x7f5b9401d560 comp rx=0 tx=0).stop 2024-04-25T08:55:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 -- 172.21.15.37:0/4135584449 shutdown_connections 2024-04-25T08:55:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 --2- 172.21.15.37:0/4135584449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ba40b28c0 0x7f5ba4163960 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 --2- 172.21.15.37:0/4135584449 >> v2:172.21.15.203:3300/0 conn(0x7f5ba4159bd0 0x7f5ba40c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 -- 172.21.15.37:0/4135584449 >> 172.21.15.37:0/4135584449 conn(0x7f5ba406ae60 msgr2=0x7f5ba406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 -- 172.21.15.37:0/4135584449 shutdown_connections 2024-04-25T08:55:27.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 -- 172.21.15.37:0/4135584449 wait complete. 2024-04-25T08:55:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 Processor -- start 2024-04-25T08:55:27.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 -- start start 2024-04-25T08:55:27.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ba40b28c0 0x7f5ba414bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ba4159bd0 0x7f5ba409b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ba4159fb0 0x7f5ba409a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba4166030 con 0x7f5ba4159fb0 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba4165eb0 con 0x7f5ba40b28c0 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba41661b0 con 0x7f5ba4159bd0 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ba4159fb0 0x7f5ba409a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ba4159fb0 0x7f5ba409a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49088/0 (socket says 172.21.15.37:49088) 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba37fe640 1 -- 172.21.15.37:0/1540463316 learned_addr learned my addr 172.21.15.37:0/1540463316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ba40b28c0 0x7f5ba414bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba27fc640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ba4159bd0 0x7f5ba409b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba37fe640 1 -- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ba4159bd0 msgr2=0x7f5ba409b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ba4159bd0 0x7f5ba409b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba37fe640 1 -- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ba40b28c0 msgr2=0x7f5ba414bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:27.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ba40b28c0 0x7f5ba414bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba37fe640 1 -- 172.21.15.37:0/1540463316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ba4099390 con 0x7f5ba4159fb0 2024-04-25T08:55:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba27fc640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ba4159bd0 0x7f5ba409b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ba4159fb0 0x7f5ba409a110 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5b9401a170 tx=0x7f5b9401a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5b8bfff640 1 -- 172.21.15.37:0/1540463316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940bd680 con 0x7f5ba4159fb0 2024-04-25T08:55:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5b8bfff640 1 -- 172.21.15.37:0/1540463316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b940bd820 con 0x7f5ba4159fb0 2024-04-25T08:55:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 -- 172.21.15.37:0/1540463316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba4099630 con 0x7f5ba4159fb0 2024-04-25T08:55:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5b8bfff640 1 -- 172.21.15.37:0/1540463316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940bdbf0 con 0x7f5ba4159fb0 2024-04-25T08:55:27.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.943+0000 7f5ba9928640 1 -- 172.21.15.37:0/1540463316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba4099bf0 con 0x7f5ba4159fb0 2024-04-25T08:55:27.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5b8bfff640 1 -- 172.21.15.37:0/1540463316 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5b940c63f0 con 0x7f5ba4159fb0 2024-04-25T08:55:27.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5b8bfff640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b90066f60 0x7f5b90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5b8bfff640 1 -- 172.21.15.37:0/1540463316 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5b9411fb90 con 0x7f5ba4159fb0 2024-04-25T08:55:27.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b90066f60 0x7f5b90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5baca95000 1 Processor -- start 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5baca95000 1 -- start start 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b90066f60 0x7f5b90069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c003fd0 tx=0x7f5b9c0039a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5baca95000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d448fe30 0x5588d4490210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5baca95000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d4490750 0x5588d4443440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5baca95000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4443980 0x5588d4443d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5baca95000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d428eff0 con 0x5588d4443980 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5baca95000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d4217150 con 0x5588d4490750 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5baca95000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588d428b090 con 0x5588d448fe30 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4443980 0x5588d4443d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4443980 0x5588d4443d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49104/0 (socket says 172.21.15.37:49104) 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba37fe640 1 -- 172.21.15.37:0/3177747141 learned_addr learned my addr 172.21.15.37:0/3177747141 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/3177747141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d448fe30 0x5588d4490210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba27fc640 1 --2- 172.21.15.37:0/3177747141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d4490750 0x5588d4443440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba37fe640 1 -- 172.21.15.37:0/3177747141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d448fe30 msgr2=0x5588d4490210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/3177747141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d448fe30 0x5588d4490210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba37fe640 1 -- 172.21.15.37:0/3177747141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d4490750 msgr2=0x5588d4443440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/3177747141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d4490750 0x5588d4443440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.947+0000 7f5ba37fe640 1 -- 172.21.15.37:0/3177747141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588d44250d0 con 0x5588d4443980 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/3177747141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4443980 0x5588d4443d60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5b940f8550 tx=0x7f5b940efeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4851f9ecde0560 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5b897fa640 1 -- 172.21.15.37:0/3177747141 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940f0370 con 0x5588d4443980 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5b897fa640 1 -- 172.21.15.37:0/3177747141 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b940b0040 con 0x5588d4443980 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5b897fa640 1 -- 172.21.15.37:0/3177747141 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940f44c0 con 0x5588d4443980 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- 172.21.15.37:0/3177747141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4443980 msgr2=0x5588d4443d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 --2- 172.21.15.37:0/3177747141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4443980 0x5588d4443d60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f5b940f8550 tx=0x7f5b940efeb0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- 172.21.15.37:0/3177747141 shutdown_connections 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 --2- 172.21.15.37:0/3177747141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4443980 0x5588d4443d60 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 --2- 172.21.15.37:0/3177747141 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d4490750 0x5588d4443440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 --2- 172.21.15.37:0/3177747141 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d448fe30 0x5588d4490210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- 172.21.15.37:0/3177747141 >> 172.21.15.37:0/3177747141 conn(0x5588d44709c0 msgr2=0x5588d4472db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:27.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- 172.21.15.37:0/3177747141 shutdown_connections 2024-04-25T08:55:27.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- 172.21.15.37:0/3177747141 wait complete. 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- start start 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 Processor -- start 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d4443980 0x5588d4476120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d448fe30 0x5588d4476660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4490750 0x5588d4476ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d4253250 con 0x5588d4490750 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d4249690 con 0x5588d448fe30 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588d4217a40 con 0x5588d4443980 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 10 client.0 _ll_register_callbacks cb 0x5588d442ce90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4490750 0x5588d4476ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4490750 0x5588d4476ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49110/0 (socket says 172.21.15.37:49110) 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba37fe640 1 -- 172.21.15.37:0/3856920904 learned_addr learned my addr 172.21.15.37:0/3856920904 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d4443980 0x5588d4476120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba27fc640 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d448fe30 0x5588d4476660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba37fe640 1 -- 172.21.15.37:0/3856920904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d4443980 msgr2=0x5588d4476120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d4443980 0x5588d4476120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba37fe640 1 -- 172.21.15.37:0/3856920904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d448fe30 msgr2=0x5588d4476660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d448fe30 0x5588d4476660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba37fe640 1 -- 172.21.15.37:0/3856920904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588d433c1c0 con 0x5588d4490750 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4490750 0x5588d4476ba0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5b940f0070 tx=0x7f5b94008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b020d4b1b786f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5b827fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5b827fc640 1 -- 172.21.15.37:0/3856920904 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9411d070 con 0x5588d4490750 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5b827fc640 1 -- 172.21.15.37:0/3856920904 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5b940f4040 con 0x5588d4490750 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- 172.21.15.37:0/3856920904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b940b0040 con 0x5588d4490750 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5b827fc640 1 -- 172.21.15.37:0/3856920904 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b94115050 con 0x5588d4490750 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 10 client.6207 fetch_fsmap learned FSMap version 25 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- 172.21.15.37:0/3856920904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5588d4338170 con 0x5588d4490750 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5b827fc640 1 -- 172.21.15.37:0/3856920904 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b940b0040 con 0x5588d4490750 2024-04-25T08:55:27.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5b827fc640 1 -- 172.21.15.37:0/3856920904 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5b940aa070 con 0x5588d4490750 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 10 client.6207 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 10 client.6207 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 20 client.6207 populate_metadata read hostname 'smithi037' 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x5588d449fed0 0x5588d44a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 4 client.6207 mds_command: new command op to 5937 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5baca95000 1 -- 172.21.15.37:0/3856920904 --> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588d422ac60 con 0x5588d449fed0 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.951+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x5588d449fed0 0x5588d44a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5b827fc640 10 client.6207 ms_handle_connect on v2:172.21.15.37:6834/3892786810 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x5588d449fed0 0x5588d44a22c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55cd6fce6d9a5bcd server_cookie=94e77dcf004daf1e in_seq=0 out_seq=0 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5b827fc640 1 -- 172.21.15.37:0/3856920904 <== mds.0 v2:172.21.15.37:6834/3892786810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588d422ac60 con 0x5588d449fed0 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5b827fc640 10 client.6207 handle_command_reply: tid=0 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 1 client.6207 shutdown 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 1 -- 172.21.15.37:0/3856920904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4490750 msgr2=0x5588d4476ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4490750 0x5588d4476ba0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5b940f0070 tx=0x7f5b94008000 comp rx=0 tx=0).stop 2024-04-25T08:55:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 1 -- 172.21.15.37:0/3856920904 shutdown_connections 2024-04-25T08:55:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x5588d449fed0 0x5588d44a22c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d4490750 0x5588d4476ba0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d448fe30 0x5588d4476660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 1 --2- 172.21.15.37:0/3856920904 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d4443980 0x5588d4476120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5b827fc640 0 client.6207 ms_handle_reset on v2:172.21.15.37:6834/3892786810 2024-04-25T08:55:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 1 -- 172.21.15.37:0/3856920904 >> 172.21.15.37:0/3856920904 conn(0x5588d44709c0 msgr2=0x5588d4472db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 1 -- 172.21.15.37:0/3856920904 shutdown_connections 2024-04-25T08:55:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 1 -- 172.21.15.37:0/3856920904 wait complete. 2024-04-25T08:55:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.955+0000 7f5baca95000 20 client.6207 trim_cache size 0 max 16384 2024-04-25T08:55:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 Processor -- start 2024-04-25T08:55:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 -- start start 2024-04-25T08:55:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d44709c0 0x5588d4442b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d4443980 0x5588d4443080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d448fe30 0x5588d44a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba41fef70 con 0x5588d448fe30 2024-04-25T08:55:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40ad630 con 0x5588d44709c0 2024-04-25T08:55:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba41f51e0 con 0x5588d4443980 2024-04-25T08:55:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d448fe30 0x5588d44a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5ba37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d448fe30 0x5588d44a12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49124/0 (socket says 172.21.15.37:49124) 2024-04-25T08:55:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5ba37fe640 1 -- 172.21.15.37:0/77352957 learned_addr learned my addr 172.21.15.37:0/77352957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5ba37fe640 1 -- 172.21.15.37:0/77352957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d4443980 msgr2=0x5588d4443080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:27.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5ba27fc640 1 --2- 172.21.15.37:0/77352957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d4443980 0x5588d4443080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/77352957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d44709c0 0x5588d4442b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/77352957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d4443980 0x5588d4443080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5ba37fe640 1 -- 172.21.15.37:0/77352957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d44709c0 msgr2=0x5588d4442b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/77352957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d44709c0 0x5588d4442b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5ba37fe640 1 -- 172.21.15.37:0/77352957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588d42d5de0 con 0x5588d448fe30 2024-04-25T08:55:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/77352957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d448fe30 0x5588d44a12c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5b9401cfd0 tx=0x7f5b94119070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76977af5cbee517 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5b88ff9640 1 -- 172.21.15.37:0/77352957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940efcd0 con 0x5588d448fe30 2024-04-25T08:55:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5b88ff9640 1 -- 172.21.15.37:0/77352957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b940b0040 con 0x5588d448fe30 2024-04-25T08:55:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5b88ff9640 1 -- 172.21.15.37:0/77352957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b940f87f0 con 0x5588d448fe30 2024-04-25T08:55:27.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 -- 172.21.15.37:0/77352957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d448fe30 msgr2=0x5588d44a12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 --2- 172.21.15.37:0/77352957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d448fe30 0x5588d44a12c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5b9401cfd0 tx=0x7f5b94119070 comp rx=0 tx=0).stop 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 -- 172.21.15.37:0/77352957 shutdown_connections 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 --2- 172.21.15.37:0/77352957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d448fe30 0x5588d44a12c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 --2- 172.21.15.37:0/77352957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d4443980 0x5588d4443080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 --2- 172.21.15.37:0/77352957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d44709c0 0x5588d4442b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 -- 172.21.15.37:0/77352957 >> 172.21.15.37:0/77352957 conn(0x5588d4447930 msgr2=0x5588d4442430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 -- 172.21.15.37:0/77352957 shutdown_connections 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 -- 172.21.15.37:0/77352957 wait complete. 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 -- start start 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.971+0000 7f5baca95000 1 Processor -- start 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d4443980 0x5588d44f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d44709c0 0x5588d44f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d448fe30 0x5588d44abd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588d4229e00 con 0x5588d44709c0 2024-04-25T08:55:27.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba40c4700 con 0x5588d4443980 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588d428e300 con 0x5588d448fe30 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 10 client.0 _ll_register_callbacks cb 0x5588d440d040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d44709c0 0x5588d44f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d44709c0 0x5588d44f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49128/0 (socket says 172.21.15.37:49128) 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba27fc640 1 -- 172.21.15.37:0/3141965330 learned_addr learned my addr 172.21.15.37:0/3141965330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba37fe640 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d448fe30 0x5588d44abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d4443980 0x5588d44f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba27fc640 1 -- 172.21.15.37:0/3141965330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d448fe30 msgr2=0x5588d44abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba27fc640 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d448fe30 0x5588d44abd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba27fc640 1 -- 172.21.15.37:0/3141965330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d4443980 msgr2=0x5588d44f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba27fc640 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d4443980 0x5588d44f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba27fc640 1 -- 172.21.15.37:0/3141965330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588d42308e0 con 0x5588d44709c0 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d4443980 0x5588d44f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba27fc640 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d44709c0 0x5588d44f0c00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5b980076d0 tx=0x7f5b98011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcbdb72223a7fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:27.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5b827fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5b827fc640 1 -- 172.21.15.37:0/3141965330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98019020 con 0x5588d44709c0 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5b827fc640 1 -- 172.21.15.37:0/3141965330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5b98015470 con 0x5588d44709c0 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 -- 172.21.15.37:0/3141965330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b940b0040 con 0x5588d44709c0 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5b827fc640 1 -- 172.21.15.37:0/3141965330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b98023050 con 0x5588d44709c0 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 10 client.6213 fetch_fsmap learned FSMap version 25 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 -- 172.21.15.37:0/3141965330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x5588d42e88f0 con 0x5588d44709c0 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5b827fc640 1 -- 172.21.15.37:0/3141965330 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b940b0040 con 0x5588d44709c0 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5b827fc640 1 -- 172.21.15.37:0/3141965330 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5b98026020 con 0x5588d44709c0 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 10 client.6213 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 10 client.6213 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 20 client.6213 populate_metadata read hostname 'smithi037' 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x5588d4490750 0x5588d44ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 4 client.6213 mds_command: new command op to 5937 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:55:27.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 -- 172.21.15.37:0/3141965330 --> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5588d429c530 con 0x5588d4490750 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x5588d4490750 0x5588d44ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5b827fc640 10 client.6213 ms_handle_connect on v2:172.21.15.37:6834/3892786810 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba2ffd640 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x5588d4490750 0x5588d44ac9b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1249127cadbde66 server_cookie=12bfdfd61b018abe in_seq=0 out_seq=0 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5b827fc640 1 -- 172.21.15.37:0/3141965330 <== mds.0 v2:172.21.15.37:6834/3892786810 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5588d429c530 con 0x5588d4490750 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5b827fc640 10 client.6213 handle_command_reply: tid=0 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 client.6213 shutdown 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 -- 172.21.15.37:0/3141965330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d44709c0 msgr2=0x5588d44f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d44709c0 0x5588d44f0c00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5b980076d0 tx=0x7f5b98011620 comp rx=0 tx=0).stop 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 -- 172.21.15.37:0/3141965330 shutdown_connections 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.37:6834/3892786810,v1:172.21.15.37:6835/3892786810] conn(0x5588d4490750 0x5588d44ac9b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5588d448fe30 0x5588d44abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588d44709c0 0x5588d44f0c00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 --2- 172.21.15.37:0/3141965330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588d4443980 0x5588d44f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5b827fc640 0 client.6213 ms_handle_reset on v2:172.21.15.37:6834/3892786810 2024-04-25T08:55:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 -- 172.21.15.37:0/3141965330 >> 172.21.15.37:0/3141965330 conn(0x5588d4447930 msgr2=0x5588d4442cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:27.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 -- 172.21.15.37:0/3141965330 shutdown_connections 2024-04-25T08:55:27.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 1 -- 172.21.15.37:0/3141965330 wait complete. 2024-04-25T08:55:27.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5baca95000 20 client.6213 trim_cache size 0 max 16384 2024-04-25T08:55:27.971 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:55:27.971 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-04-25T08:55:27.971 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-04-25T08:55:27.971 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:55:27.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 -- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b90066f60 msgr2=0x7f5b90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:27.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b90066f60 0x7f5b90069420 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c003fd0 tx=0x7f5b9c0039a0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 -- 172.21.15.37:0/1540463316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ba4159fb0 msgr2=0x7f5ba409a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:27.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ba4159fb0 0x7f5ba409a110 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5b9401a170 tx=0x7f5b9401a300 comp rx=0 tx=0).stop 2024-04-25T08:55:27.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 -- 172.21.15.37:0/1540463316 shutdown_connections 2024-04-25T08:55:27.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5b90066f60 0x7f5b90069420 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ba4159fb0 0x7f5ba409a110 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ba4159bd0 0x7f5ba409b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 --2- 172.21.15.37:0/1540463316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ba40b28c0 0x7f5ba414bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:27.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 -- 172.21.15.37:0/1540463316 >> 172.21.15.37:0/1540463316 conn(0x7f5ba406ae60 msgr2=0x7f5ba40aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:27.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 -- 172.21.15.37:0/1540463316 shutdown_connections 2024-04-25T08:55:27.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:27.975+0000 7f5ba9928640 1 -- 172.21.15.37:0/1540463316 wait complete. 2024-04-25T08:55:27.982 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:55:28.083 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:28.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2789cdc640 1 Processor -- start 2024-04-25T08:55:28.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2789cdc640 1 -- start start 2024-04-25T08:55:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2789cdc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278409a180 0x7f27840b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2789cdc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f278409a7b0 0x7f27840ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2789cdc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27840cbec0 con 0x7f27840b2c30 2024-04-25T08:55:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2789cdc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27840cc1a0 con 0x7f278409a7b0 2024-04-25T08:55:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2789cdc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27840cc480 con 0x7f278409a180 2024-04-25T08:55:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27837fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f27840b2c30 0x7f2784099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:54168/0 (socket says 172.21.15.37:54168) 2024-04-25T08:55:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27837fe640 1 -- 172.21.15.37:0/1213588073 learned_addr learned my addr 172.21.15.37:0/1213588073 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2783fff640 1 --2- 172.21.15.37:0/1213588073 >> v2:172.21.15.203:3300/0 conn(0x7f278409a7b0 0x7f27840ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2782ffd640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278409a180 0x7f27840b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903134141 0 0) 0x7f27840cbec0 con 0x7f27840b2c30 2024-04-25T08:55:28.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f276c003530 con 0x7f27840b2c30 2024-04-25T08:55:28.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2993442069 0 0) 0x7f276c003530 con 0x7f27840b2c30 2024-04-25T08:55:28.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278409a180 msgr2=0x7f27840b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:28.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278409a180 0x7f27840b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 >> v2:172.21.15.203:3300/0 conn(0x7f278409a7b0 msgr2=0x7f27840ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 --2- 172.21.15.37:0/1213588073 >> v2:172.21.15.203:3300/0 conn(0x7f278409a7b0 0x7f27840ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840cc760 con 0x7f27840b2c30 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f2774001680 con 0x7f27840b2c30 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 >> v1:172.21.15.37:6789/0 conn(0x7f27840b2c30 legacy=0x7f2784099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276c003a30 0x7f276c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f276c004810 0x7f276c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278409a180 0x7f276c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27840cc1a0 con 0x7f276c004810 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27840cc480 con 0x7f276c003a30 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276c00b9d0 con 0x7f278409a180 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2782ffd640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f276c004810 0x7f276c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27837fe640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276c003a30 0x7f276c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2783fff640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278409a180 0x7f276c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2782ffd640 1 -- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278409a180 msgr2=0x7f276c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2782ffd640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f278409a180 0x7f276c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2782ffd640 1 -- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276c003a30 msgr2=0x7f276c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2782ffd640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276c003a30 0x7f276c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f2782ffd640 1 -- 172.21.15.37:0/1213588073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c003530 con 0x7f276c004810 2024-04-25T08:55:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.219+0000 7f27837fe640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276c003a30 0x7f276c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f2782ffd640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f276c004810 0x7f276c004c90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2778009af0 tx=0x7f277801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f233a68a74dc8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27780b2070 con 0x7f276c004810 2024-04-25T08:55:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27780a5540 con 0x7f276c004810 2024-04-25T08:55:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f27827fc640 1 -- 172.21.15.37:0/1213588073 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778009690 con 0x7f276c004810 2024-04-25T08:55:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f2789cdc640 1 -- 172.21.15.37:0/1213588073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f276c004810 msgr2=0x7f276c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f2789cdc640 1 --2- 172.21.15.37:0/1213588073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f276c004810 0x7f276c004c90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2778009af0 tx=0x7f277801ca30 comp rx=0 tx=0).stop 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f2783fff640 1 -- 172.21.15.37:0/1213588073 reap_dead start 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f2789cdc640 1 -- 172.21.15.37:0/1213588073 shutdown_connections 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f2789cdc640 1 -- 172.21.15.37:0/1213588073 >> 172.21.15.37:0/1213588073 conn(0x7f27840b3010 msgr2=0x7f27840b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f2789cdc640 1 -- 172.21.15.37:0/1213588073 shutdown_connections 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f2789cdc640 1 -- 172.21.15.37:0/1213588073 wait complete. 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f2789cdc640 1 Processor -- start 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.223+0000 7f2789cdc640 1 -- start start 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2789cdc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276c004810 0x7f27840c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2789cdc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278409ed20 0x7f278409d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2789cdc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27840ca9e0 0x7f27840cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2789cdc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27840cbb70 con 0x7f278409ed20 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2789cdc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278409ac40 con 0x7f276c004810 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2789cdc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278406c200 con 0x7f27840ca9e0 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2782ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278409ed20 0x7f278409d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2782ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278409ed20 0x7f278409d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49146/0 (socket says 172.21.15.37:49146) 2024-04-25T08:55:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2782ffd640 1 -- 172.21.15.37:0/2862037344 learned_addr learned my addr 172.21.15.37:0/2862037344 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2783fff640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27840ca9e0 0x7f27840cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f27837fe640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276c004810 0x7f27840c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2782ffd640 1 -- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27840ca9e0 msgr2=0x7f27840cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2782ffd640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27840ca9e0 0x7f27840cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2782ffd640 1 -- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276c004810 msgr2=0x7f27840c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2782ffd640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276c004810 0x7f27840c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2782ffd640 1 -- 172.21.15.37:0/2862037344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278409d720 con 0x7f278409ed20 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2783fff640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27840ca9e0 0x7f27840cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2782ffd640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278409ed20 0x7f278409d1e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2778009a10 tx=0x7f276c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f276bfff640 1 -- 172.21.15.37:0/2862037344 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27780b2040 con 0x7f278409ed20 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f276bfff640 1 -- 172.21.15.37:0/2862037344 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27780ad640 con 0x7f278409ed20 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2789cdc640 1 -- 172.21.15.37:0/2862037344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27840c95e0 con 0x7f278409ed20 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f276bfff640 1 -- 172.21.15.37:0/2862037344 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27780ada10 con 0x7f278409ed20 2024-04-25T08:55:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2789cdc640 1 -- 172.21.15.37:0/2862037344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27840c9ba0 con 0x7f278409ed20 2024-04-25T08:55:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f2789cdc640 1 -- 172.21.15.37:0/2862037344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278404efa0 con 0x7f278409ed20 2024-04-25T08:55:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f276bfff640 1 -- 172.21.15.37:0/2862037344 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27780b5070 con 0x7f278409ed20 2024-04-25T08:55:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f276bfff640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2770066f90 0x7f2770069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f276bfff640 1 -- 172.21.15.37:0/2862037344 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f277811f2b0 con 0x7f278409ed20 2024-04-25T08:55:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.227+0000 7f27837fe640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2770066f90 0x7f2770069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.231+0000 7f27837fe640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2770066f90 0x7f2770069450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2774001060 tx=0x7f2774009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.231+0000 7f276bfff640 1 -- 172.21.15.37:0/2862037344 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2778016780 con 0x7f278409ed20 2024-04-25T08:55:28.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.423+0000 7f2789cdc640 1 -- 172.21.15.37:0/2862037344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f278409b2f0 con 0x7f278409ed20 2024-04-25T08:55:28.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.423+0000 7f276bfff640 1 -- 172.21.15.37:0/2862037344 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f27780f0190 con 0x7f278409ed20 2024-04-25T08:55:28.414 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:28.414 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2963121871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2963121871},{"type":"v1","addr":"172.21.15.203:6837","nonce":2963121871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1490204181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1490204181},{"type":"v1","addr":"172.21.15.203:6835","nonce":1490204181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2218454043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2218454043},{"type":"v1","addr":"172.21.15.37:6833","nonce":2218454043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:42.800130+0000","modified":"2024-04-25T08:54:45.849035+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/3892786810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3892786810},{"type":"v1","addr":"172.21.15.37:6835","nonce":3892786810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:55:28.414 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-25T08:55:28.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 -- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2770066f90 msgr2=0x7f2770069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2770066f90 0x7f2770069450 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2774001060 tx=0x7f2774009040 comp rx=0 tx=0).stop 2024-04-25T08:55:28.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 -- 172.21.15.37:0/2862037344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278409ed20 msgr2=0x7f278409d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278409ed20 0x7f278409d1e0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2778009a10 tx=0x7f276c005840 comp rx=0 tx=0).stop 2024-04-25T08:55:28.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 -- 172.21.15.37:0/2862037344 shutdown_connections 2024-04-25T08:55:28.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2770066f90 0x7f2770069450 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27840ca9e0 0x7f27840cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f278409ed20 0x7f278409d1e0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 --2- 172.21.15.37:0/2862037344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f276c004810 0x7f27840c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 -- 172.21.15.37:0/2862037344 >> 172.21.15.37:0/2862037344 conn(0x7f27840b3010 msgr2=0x7f27840cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:28.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 -- 172.21.15.37:0/2862037344 shutdown_connections 2024-04-25T08:55:28.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.427+0000 7f2789cdc640 1 -- 172.21.15.37:0/2862037344 wait complete. 2024-04-25T08:55:28.431 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T08:55:28.567 INFO:teuthology.orchestra.run.smithi037.stdout:{"message":"","return_code":0} 2024-04-25T08:55:28.567 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:55:28.568 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T08:55:28.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.615+0000 7f4c8a40d080 1 Processor -- start 2024-04-25T08:55:28.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.615+0000 7f4c8a40d080 1 -- start start 2024-04-25T08:55:28.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.615+0000 7f4c8a40d080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b61ad3cd70 0x55b61acb16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.615+0000 7f4c8a40d080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b61ac0feb0 0x55b61ad3d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.615+0000 7f4c8a40d080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3d690 0x55b61ad38c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.615+0000 7f4c8a40d080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61ab859f0 con 0x55b61ad3d690 2024-04-25T08:55:28.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.615+0000 7f4c8a40d080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61ab89950 con 0x55b61ad3cd70 2024-04-25T08:55:28.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.615+0000 7f4c8a40d080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b61ac0edf0 con 0x55b61ac0feb0 2024-04-25T08:55:28.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.615+0000 7f4c8872c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3d690 0x55b61ad38c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.615+0000 7f4c8872c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3d690 0x55b61ad38c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49160/0 (socket says 172.21.15.37:49160) 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.615+0000 7f4c8872c640 1 -- 172.21.15.37:0/1484373175 learned_addr learned my addr 172.21.15.37:0/1484373175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c87f2b640 1 --2- 172.21.15.37:0/1484373175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b61ad3cd70 0x55b61acb16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8772a640 1 --2- 172.21.15.37:0/1484373175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b61ac0feb0 0x55b61ad3d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8872c640 1 -- 172.21.15.37:0/1484373175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b61ac0feb0 msgr2=0x55b61ad3d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8872c640 1 --2- 172.21.15.37:0/1484373175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b61ac0feb0 0x55b61ad3d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8872c640 1 -- 172.21.15.37:0/1484373175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b61ad3cd70 msgr2=0x55b61acb16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8872c640 1 --2- 172.21.15.37:0/1484373175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b61ad3cd70 0x55b61acb16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8872c640 1 -- 172.21.15.37:0/1484373175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b61acb1c10 con 0x55b61ad3d690 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8872c640 1 --2- 172.21.15.37:0/1484373175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3d690 0x55b61ad38c60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4c800ae040 tx=0x55b61ac29230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0ac1ea665a8082 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c86728640 1 -- 172.21.15.37:0/1484373175 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80001c60 con 0x55b61ad3d690 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c86728640 1 -- 172.21.15.37:0/1484373175 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80001640 con 0x55b61ad3d690 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c86728640 1 -- 172.21.15.37:0/1484373175 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c800a4c80 con 0x55b61ad3d690 2024-04-25T08:55:28.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1484373175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3d690 msgr2=0x55b61ad38c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 --2- 172.21.15.37:0/1484373175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3d690 0x55b61ad38c60 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4c800ae040 tx=0x55b61ac29230 comp rx=0 tx=0).stop 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1484373175 shutdown_connections 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 --2- 172.21.15.37:0/1484373175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3d690 0x55b61ad38c60 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 --2- 172.21.15.37:0/1484373175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b61ac0feb0 0x55b61ad3d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 --2- 172.21.15.37:0/1484373175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b61ad3cd70 0x55b61acb16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1484373175 >> 172.21.15.37:0/1484373175 conn(0x55b61ac09ab0 msgr2=0x55b61ac0d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1484373175 shutdown_connections 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1484373175 wait complete. 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 Processor -- start 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 -- start start 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3cd70 0x55b61ac242d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b61ad3d690 0x55b61ac1fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b61ac201b0 0x55b61ac211f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61abe2ae0 con 0x55b61ad3cd70 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b61ad3c8d0 con 0x55b61ac201b0 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b61ac25510 con 0x55b61ad3d690 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c87f2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3cd70 0x55b61ac242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c87f2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3cd70 0x55b61ac242d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49162/0 (socket says 172.21.15.37:49162) 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c87f2b640 1 -- 172.21.15.37:0/1331241237 learned_addr learned my addr 172.21.15.37:0/1331241237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8772a640 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b61ad3d690 0x55b61ac1fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8872c640 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b61ac201b0 0x55b61ac211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c87f2b640 1 -- 172.21.15.37:0/1331241237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b61ad3d690 msgr2=0x55b61ac1fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c87f2b640 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b61ad3d690 0x55b61ac1fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c87f2b640 1 -- 172.21.15.37:0/1331241237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b61ac201b0 msgr2=0x55b61ac211f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c87f2b640 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b61ac201b0 0x55b61ac211f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c87f2b640 1 -- 172.21.15.37:0/1331241237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b61ac24b70 con 0x55b61ad3cd70 2024-04-25T08:55:28.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c87f2b640 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3cd70 0x55b61ac242d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800e3c0 tx=0x7f4c7800e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:28.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c85726640 1 -- 172.21.15.37:0/1331241237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78014910 con 0x55b61ad3cd70 2024-04-25T08:55:28.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c85726640 1 -- 172.21.15.37:0/1331241237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c78014ab0 con 0x55b61ad3cd70 2024-04-25T08:55:28.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c85726640 1 -- 172.21.15.37:0/1331241237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78012400 con 0x55b61ad3cd70 2024-04-25T08:55:28.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1331241237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b61ac24e10 con 0x55b61ad3cd70 2024-04-25T08:55:28.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.619+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1331241237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b61ad39c30 con 0x55b61ad3cd70 2024-04-25T08:55:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c85726640 1 -- 172.21.15.37:0/1331241237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c780125a0 con 0x55b61ad3cd70 2024-04-25T08:55:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c85726640 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4c740659a0 0x7f4c74067e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c85726640 1 -- 172.21.15.37:0/1331241237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f4c7800e020 con 0x55b61ad3cd70 2024-04-25T08:55:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b61ad3dcd0 0x55b61ad3a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1331241237 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:1 8.7 8:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57) v9 -- 0x55b61ad3a8b0 con 0x55b61ad3dcd0 2024-04-25T08:55:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8772a640 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4c740659a0 0x7f4c74067e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8872c640 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b61ad3dcd0 0x55b61ad3a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8872c640 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b61ad3dcd0 0x55b61ad3a370 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8872c640 1 -- 172.21.15.37:0/1331241237 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f4c80008030 con 0x55b61ad3dcd0 2024-04-25T08:55:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1331241237 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b61ad3dcd0 msgr2=0x55b61ad3a370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b61ad3dcd0 0x55b61ad3a370 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1331241237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4c740659a0 msgr2=0x7f4c74067e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4c740659a0 0x7f4c74067e60 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c000f70 tx=0x7f4c7c0036f0 comp rx=0 tx=0).stop 2024-04-25T08:55:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1331241237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3cd70 msgr2=0x55b61ac242d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 --2- 172.21.15.37:0/1331241237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b61ad3cd70 0x55b61ac242d0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800e3c0 tx=0x7f4c7800e880 comp rx=0 tx=0).stop 2024-04-25T08:55:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8872c640 1 -- 172.21.15.37:0/1331241237 reap_dead start 2024-04-25T08:55:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1331241237 shutdown_connections 2024-04-25T08:55:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1331241237 >> 172.21.15.37:0/1331241237 conn(0x55b61ac09ab0 msgr2=0x55b61acaf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1331241237 shutdown_connections 2024-04-25T08:55:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:28.623+0000 7f4c8a40d080 1 -- 172.21.15.37:0/1331241237 wait complete. 2024-04-25T08:55:28.618 INFO:teuthology.orchestra.run.smithi037.stdout:Yalphaparent_a 2024-04-25T08:55:28.618 DEBUG:teuthology.orchestra.run.smithi037:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T08:55:28.703 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:55:28.703 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627777, 2024-04-25T08:55:28.703 INFO:teuthology.orchestra.run.smithi037.stdout: "ancestors": [ 2024-04-25T08:55:28.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:28.703 INFO:teuthology.orchestra.run.smithi037.stdout: "dirino": 1099511627776, 2024-04-25T08:55:28.703 INFO:teuthology.orchestra.run.smithi037.stdout: "dname": "alpha", 2024-04-25T08:55:28.703 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 4 2024-04-25T08:55:28.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:28.704 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:28.704 INFO:teuthology.orchestra.run.smithi037.stdout: "dirino": 1, 2024-04-25T08:55:28.704 INFO:teuthology.orchestra.run.smithi037.stdout: "dname": "parent_a", 2024-04-25T08:55:28.704 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 7 2024-04-25T08:55:28.704 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:28.704 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T08:55:28.704 INFO:teuthology.orchestra.run.smithi037.stdout: "pool": 8, 2024-04-25T08:55:28.704 INFO:teuthology.orchestra.run.smithi037.stdout: "old_pools": [] 2024-04-25T08:55:28.704 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:55:28.704 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:28.707 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:55:28.707 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:55:28.784 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:55:28.844 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:55:28.845 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:55:28.845 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:55:28.852 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.027+0000 7f1b4c487640 1 Processor -- start 2024-04-25T08:55:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.027+0000 7f1b4c487640 1 -- start start 2024-04-25T08:55:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.027+0000 7f1b4c487640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1b4415da10 0x7f1b44071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.027+0000 7f1b4c487640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b44162310 con 0x7f1b440b24a0 2024-04-25T08:55:29.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.027+0000 7f1b4c487640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b440adbd0 con 0x7f1b44071e90 2024-04-25T08:55:29.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.027+0000 7f1b4c487640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b440ae860 con 0x7f1b4415da10 2024-04-25T08:55:29.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.027+0000 7f1b49a22640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1b440b24a0 0x7f1b441614f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:54180/0 (socket says 172.21.15.37:54180) 2024-04-25T08:55:29.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.027+0000 7f1b4a223640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1b4415da10 0x7f1b44071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.027+0000 7f1b49a22640 1 -- 172.21.15.37:0/1834370469 learned_addr learned my addr 172.21.15.37:0/1834370469 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:29.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.031+0000 7f1b4a223640 1 -- 172.21.15.37:0/1834370469 >> v1:172.21.15.203:6789/0 conn(0x7f1b44071e90 legacy=0x7f1b44161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:29.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.031+0000 7f1b4a223640 1 -- 172.21.15.37:0/1834370469 >> v1:172.21.15.37:6789/0 conn(0x7f1b440b24a0 legacy=0x7f1b441614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.031+0000 7f1b4a223640 1 -- 172.21.15.37:0/1834370469 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4415e210 con 0x7f1b4415da10 2024-04-25T08:55:29.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.031+0000 7f1b49221640 1 -- 172.21.15.37:0/1834370469 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1955284472 0 0) 0x7f1b44162310 con 0x7f1b440b24a0 2024-04-25T08:55:29.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4a223640 1 --2- 172.21.15.37:0/1834370469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4415da10 0x7f1b44071030 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1b400097c0 tx=0x7f1b4001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d212eeb6e9a40ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b49221640 1 -- 172.21.15.37:0/1834370469 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b400b0070 con 0x7f1b4415da10 2024-04-25T08:55:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b49221640 1 -- 172.21.15.37:0/1834370469 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b400ab600 con 0x7f1b4415da10 2024-04-25T08:55:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b49221640 1 -- 172.21.15.37:0/1834370469 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b400ab9d0 con 0x7f1b4415da10 2024-04-25T08:55:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 -- 172.21.15.37:0/1834370469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4415da10 msgr2=0x7f1b44071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 --2- 172.21.15.37:0/1834370469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4415da10 0x7f1b44071030 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1b400097c0 tx=0x7f1b4001c340 comp rx=0 tx=0).stop 2024-04-25T08:55:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 -- 172.21.15.37:0/1834370469 shutdown_connections 2024-04-25T08:55:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 --2- 172.21.15.37:0/1834370469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4415da10 0x7f1b44071030 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 -- 172.21.15.37:0/1834370469 >> 172.21.15.37:0/1834370469 conn(0x7f1b440b11a0 msgr2=0x7f1b440b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 -- 172.21.15.37:0/1834370469 shutdown_connections 2024-04-25T08:55:29.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 -- 172.21.15.37:0/1834370469 wait complete. 2024-04-25T08:55:29.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 Processor -- start 2024-04-25T08:55:29.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 -- start start 2024-04-25T08:55:29.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b44071e90 0x7f1b441486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b440b24a0 0x7f1b4414eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4415da10 0x7f1b4415c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4415ee70 con 0x7f1b4415da10 2024-04-25T08:55:29.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4415ecf0 con 0x7f1b440b24a0 2024-04-25T08:55:29.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.035+0000 7f1b4c487640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4416c4c0 con 0x7f1b44071e90 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4aa24640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4415da10 0x7f1b4415c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4aa24640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4415da10 0x7f1b4415c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49186/0 (socket says 172.21.15.37:49186) 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4aa24640 1 -- 172.21.15.37:0/3742180453 learned_addr learned my addr 172.21.15.37:0/3742180453 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b49a22640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b440b24a0 0x7f1b4414eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4a223640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b44071e90 0x7f1b441486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4aa24640 1 -- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b44071e90 msgr2=0x7f1b441486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4aa24640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b44071e90 0x7f1b441486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4aa24640 1 -- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b440b24a0 msgr2=0x7f1b4414eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4aa24640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b440b24a0 0x7f1b4414eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4aa24640 1 -- 172.21.15.37:0/3742180453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4414c8e0 con 0x7f1b4415da10 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4aa24640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4415da10 0x7f1b4415c6c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00b970 tx=0x7f1b3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b3affd640 1 -- 172.21.15.37:0/3742180453 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c019070 con 0x7f1b4415da10 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b3affd640 1 -- 172.21.15.37:0/3742180453 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3c00cb40 con 0x7f1b4415da10 2024-04-25T08:55:29.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b3affd640 1 -- 172.21.15.37:0/3742180453 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b3c014440 con 0x7f1b4415da10 2024-04-25T08:55:29.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4c487640 1 -- 172.21.15.37:0/3742180453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4414cbc0 con 0x7f1b4415da10 2024-04-25T08:55:29.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4c487640 1 -- 172.21.15.37:0/3742180453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4414d180 con 0x7f1b4415da10 2024-04-25T08:55:29.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4c487640 1 -- 172.21.15.37:0/3742180453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4404efa0 con 0x7f1b4415da10 2024-04-25T08:55:29.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b3affd640 1 -- 172.21.15.37:0/3742180453 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b3c01c070 con 0x7f1b4415da10 2024-04-25T08:55:29.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b3affd640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b28066f60 0x7f1b28069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b3affd640 1 -- 172.21.15.37:0/3742180453 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1b3c084d10 con 0x7f1b4415da10 2024-04-25T08:55:29.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.039+0000 7f1b4a223640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b28066f60 0x7f1b28069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.043+0000 7f1b4a223640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b28066f60 0x7f1b28069420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1b400ad000 tx=0x7f1b400a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.043+0000 7f1b3affd640 1 -- 172.21.15.37:0/3742180453 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b3c0555a0 con 0x7f1b4415da10 2024-04-25T08:55:29.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.207+0000 7f1b4c487640 1 -- 172.21.15.37:0/3742180453 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b4406f9b0 con 0x7f1b4415da10 2024-04-25T08:55:29.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.207+0000 7f1b3affd640 1 -- 172.21.15.37:0/3742180453 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f1b3c059640 con 0x7f1b4415da10 2024-04-25T08:55:29.198 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:29.198 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":57,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:55:12.296393+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:40.447670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:41.812203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:29.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 -- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b28066f60 msgr2=0x7f1b28069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b28066f60 0x7f1b28069420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1b400ad000 tx=0x7f1b400a6000 comp rx=0 tx=0).stop 2024-04-25T08:55:29.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 -- 172.21.15.37:0/3742180453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4415da10 msgr2=0x7f1b4415c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4415da10 0x7f1b4415c6c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c00b970 tx=0x7f1b3c00be30 comp rx=0 tx=0).stop 2024-04-25T08:55:29.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 -- 172.21.15.37:0/3742180453 shutdown_connections 2024-04-25T08:55:29.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b28066f60 0x7f1b28069420 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b4415da10 0x7f1b4415c6c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b440b24a0 0x7f1b4414eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 --2- 172.21.15.37:0/3742180453 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b44071e90 0x7f1b441486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 -- 172.21.15.37:0/3742180453 >> 172.21.15.37:0/3742180453 conn(0x7f1b440b11a0 msgr2=0x7f1b4406e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:29.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 -- 172.21.15.37:0/3742180453 shutdown_connections 2024-04-25T08:55:29.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.211+0000 7f1b4c487640 1 -- 172.21.15.37:0/3742180453 wait complete. 2024-04-25T08:55:29.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:40.447670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:41.812203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:55:29.218 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:55:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65ceb50640 1 Processor -- start 2024-04-25T08:55:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65ceb50640 1 -- start start 2024-04-25T08:55:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65ceb50640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f65c806d810 0x7f65c806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65ceb50640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65c806fce0 0x7f65c815f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65ceb50640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c80cb580 con 0x7f65c815b990 2024-04-25T08:55:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65ceb50640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c806b1d0 con 0x7f65c806d810 2024-04-25T08:55:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65ceb50640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c80cb860 con 0x7f65c806fce0 2024-04-25T08:55:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65cc8ec640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f65c815b990 0x7f65c806f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:51542/0 (socket says 172.21.15.37:51542) 2024-04-25T08:55:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65cc8ec640 1 -- 172.21.15.37:0/3153609424 learned_addr learned my addr 172.21.15.37:0/3153609424 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65cd0ed640 1 --2- 172.21.15.37:0/3153609424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65c806fce0 0x7f65c815f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65c7fff640 1 --2- 172.21.15.37:0/3153609424 >> v2:172.21.15.203:3300/0 conn(0x7f65c806d810 0x7f65c806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65cd0ed640 1 -- 172.21.15.37:0/3153609424 >> v2:172.21.15.203:3300/0 conn(0x7f65c806d810 msgr2=0x7f65c806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65cd0ed640 1 --2- 172.21.15.37:0/3153609424 >> v2:172.21.15.203:3300/0 conn(0x7f65c806d810 0x7f65c806ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65cd0ed640 1 -- 172.21.15.37:0/3153609424 >> v1:172.21.15.203:6789/0 conn(0x7f65c815b990 legacy=0x7f65c806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.359+0000 7f65cd0ed640 1 -- 172.21.15.37:0/3153609424 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80cbeb0 con 0x7f65c806fce0 2024-04-25T08:55:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65cd0ed640 1 --2- 172.21.15.37:0/3153609424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65c806fce0 0x7f65c815f300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f65c00097c0 tx=0x7f65c001c150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f494650ea29f6eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65c77fe640 1 -- 172.21.15.37:0/3153609424 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c00b0070 con 0x7f65c806fce0 2024-04-25T08:55:29.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65c77fe640 1 -- 172.21.15.37:0/3153609424 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c00ab600 con 0x7f65c806fce0 2024-04-25T08:55:29.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65c77fe640 1 -- 172.21.15.37:0/3153609424 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c00ab9d0 con 0x7f65c806fce0 2024-04-25T08:55:29.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65ceb50640 1 -- 172.21.15.37:0/3153609424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65c806fce0 msgr2=0x7f65c815f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65ceb50640 1 --2- 172.21.15.37:0/3153609424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65c806fce0 0x7f65c815f300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f65c00097c0 tx=0x7f65c001c150 comp rx=0 tx=0).stop 2024-04-25T08:55:29.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65ceb50640 1 -- 172.21.15.37:0/3153609424 shutdown_connections 2024-04-25T08:55:29.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65ceb50640 1 --2- 172.21.15.37:0/3153609424 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65c806fce0 0x7f65c815f300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65ceb50640 1 --2- 172.21.15.37:0/3153609424 >> v2:172.21.15.203:3300/0 conn(0x7f65c806d810 0x7f65c806ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65ceb50640 1 -- 172.21.15.37:0/3153609424 >> 172.21.15.37:0/3153609424 conn(0x7f65c80cc670 msgr2=0x7f65c80cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:29.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65ceb50640 1 -- 172.21.15.37:0/3153609424 shutdown_connections 2024-04-25T08:55:29.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.363+0000 7f65ceb50640 1 -- 172.21.15.37:0/3153609424 wait complete. 2024-04-25T08:55:29.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65ceb50640 1 Processor -- start 2024-04-25T08:55:29.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65ceb50640 1 -- start start 2024-04-25T08:55:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65ceb50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65c806d810 0x7f65c815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65ceb50640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65c806fce0 0x7f65c80c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65ceb50640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65c815b990 0x7f65c80cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65ceb50640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c81687c0 con 0x7f65c806d810 2024-04-25T08:55:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65ceb50640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8168640 con 0x7f65c806fce0 2024-04-25T08:55:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65ceb50640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c8168940 con 0x7f65c815b990 2024-04-25T08:55:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65cc8ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65c806d810 0x7f65c815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65cc8ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65c806d810 0x7f65c815a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38854/0 (socket says 172.21.15.37:38854) 2024-04-25T08:55:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65cc8ec640 1 -- 172.21.15.37:0/1224910239 learned_addr learned my addr 172.21.15.37:0/1224910239 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65c7fff640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65c806fce0 0x7f65c80c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65cd0ed640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65c815b990 0x7f65c80cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65cc8ec640 1 -- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65c815b990 msgr2=0x7f65c80cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65cc8ec640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65c815b990 0x7f65c80cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65cc8ec640 1 -- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65c806fce0 msgr2=0x7f65c80c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65cc8ec640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65c806fce0 0x7f65c80c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65cc8ec640 1 -- 172.21.15.37:0/1224910239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80b3150 con 0x7f65c806d810 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65cc8ec640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65c806d810 0x7f65c815a6f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f65b800cbd0 tx=0x7f65b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65c57fa640 1 -- 172.21.15.37:0/1224910239 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8017070 con 0x7f65c806d810 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65c57fa640 1 -- 172.21.15.37:0/1224910239 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b800ad40 con 0x7f65c806d810 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65ceb50640 1 -- 172.21.15.37:0/1224910239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65c815c750 con 0x7f65c806d810 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65c57fa640 1 -- 172.21.15.37:0/1224910239 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8012600 con 0x7f65c806d810 2024-04-25T08:55:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65ceb50640 1 -- 172.21.15.37:0/1224910239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c80b1170 con 0x7f65c806d810 2024-04-25T08:55:29.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65ceb50640 1 -- 172.21.15.37:0/1224910239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c804ef30 con 0x7f65c806d810 2024-04-25T08:55:29.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65c57fa640 1 -- 172.21.15.37:0/1224910239 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65b801a070 con 0x7f65c806d810 2024-04-25T08:55:29.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.367+0000 7f65c57fa640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f65b4066f60 0x7f65b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.371+0000 7f65c57fa640 1 -- 172.21.15.37:0/1224910239 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f65b80850e0 con 0x7f65c806d810 2024-04-25T08:55:29.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.371+0000 7f65c7fff640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f65b4066f60 0x7f65b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.371+0000 7f65c7fff640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f65b4066f60 0x7f65b4069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f65c815b2a0 tx=0x7f65bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.375+0000 7f65c57fa640 1 -- 172.21.15.37:0/1224910239 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f65b80558d0 con 0x7f65c806d810 2024-04-25T08:55:29.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.531+0000 7f65ceb50640 1 -- 172.21.15.37:0/1224910239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65c80cd380 con 0x7f65c806d810 2024-04-25T08:55:29.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.531+0000 7f65c57fa640 1 -- 172.21.15.37:0/1224910239 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65b8059970 con 0x7f65c806d810 2024-04-25T08:55:29.522 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 -- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f65b4066f60 msgr2=0x7f65b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f65b4066f60 0x7f65b4069420 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f65c815b2a0 tx=0x7f65bc008040 comp rx=0 tx=0).stop 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 -- 172.21.15.37:0/1224910239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65c806d810 msgr2=0x7f65c815a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65c806d810 0x7f65c815a6f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f65b800cbd0 tx=0x7f65b800a430 comp rx=0 tx=0).stop 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 -- 172.21.15.37:0/1224910239 shutdown_connections 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f65b4066f60 0x7f65b4069420 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65c815b990 0x7f65c80cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65c806fce0 0x7f65c80c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 --2- 172.21.15.37:0/1224910239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65c806d810 0x7f65c815a6f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 -- 172.21.15.37:0/1224910239 >> 172.21.15.37:0/1224910239 conn(0x7f65c80cc670 msgr2=0x7f65c80b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 -- 172.21.15.37:0/1224910239 shutdown_connections 2024-04-25T08:55:29.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.535+0000 7f65ceb50640 1 -- 172.21.15.37:0/1224910239 wait complete. 2024-04-25T08:55:29.541 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:55:29.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb1900a640 1 Processor -- start 2024-04-25T08:55:29.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb1900a640 1 -- start start 2024-04-25T08:55:29.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb1900a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbb1415db30 0x7fbb140ce360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb1900a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbb140ce8a0 0x7fbb140d34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb1900a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb140d39e0 0x7fbb140d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb1900a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb140d1320 con 0x7fbb1415db30 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb1900a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb140d3e70 con 0x7fbb140ce8a0 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb1900a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1406d180 con 0x7fbb140d39e0 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb12d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbb1415db30 0x7fbb140ce360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb12d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbb1415db30 0x7fbb140ce360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38860/0 (socket says 172.21.15.37:38860) 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb12d9d640 1 -- 172.21.15.37:0/2782289150 learned_addr learned my addr 172.21.15.37:0/2782289150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb1359e640 1 --2- 172.21.15.37:0/2782289150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb140d39e0 0x7fbb140d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb1259c640 1 --2- 172.21.15.37:0/2782289150 >> v2:172.21.15.203:3300/0 conn(0x7fbb140ce8a0 0x7fbb140d34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb12d9d640 1 -- 172.21.15.37:0/2782289150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb140d39e0 msgr2=0x7fbb140d1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb12d9d640 1 --2- 172.21.15.37:0/2782289150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb140d39e0 0x7fbb140d1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb12d9d640 1 -- 172.21.15.37:0/2782289150 >> v2:172.21.15.203:3300/0 conn(0x7fbb140ce8a0 msgr2=0x7fbb140d34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:29.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb12d9d640 1 --2- 172.21.15.37:0/2782289150 >> v2:172.21.15.203:3300/0 conn(0x7fbb140ce8a0 0x7fbb140d34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.667+0000 7fbb12d9d640 1 -- 172.21.15.37:0/2782289150 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1406d460 con 0x7fbb1415db30 2024-04-25T08:55:29.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb12d9d640 1 --2- 172.21.15.37:0/2782289150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb1415db30 0x7fbb140ce360 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbb1406b8e0 tx=0x7fbb0401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef999f64df15ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb11d9b640 1 -- 172.21.15.37:0/2782289150 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040a7070 con 0x7fbb1415db30 2024-04-25T08:55:29.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb11d9b640 1 -- 172.21.15.37:0/2782289150 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb040b0070 con 0x7fbb1415db30 2024-04-25T08:55:29.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb11d9b640 1 -- 172.21.15.37:0/2782289150 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb040ab890 con 0x7fbb1415db30 2024-04-25T08:55:29.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 -- 172.21.15.37:0/2782289150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb1415db30 msgr2=0x7fbb140ce360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 --2- 172.21.15.37:0/2782289150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb1415db30 0x7fbb140ce360 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbb1406b8e0 tx=0x7fbb0401c490 comp rx=0 tx=0).stop 2024-04-25T08:55:29.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 -- 172.21.15.37:0/2782289150 shutdown_connections 2024-04-25T08:55:29.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 --2- 172.21.15.37:0/2782289150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb140d39e0 0x7fbb140d1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 --2- 172.21.15.37:0/2782289150 >> v2:172.21.15.203:3300/0 conn(0x7fbb140ce8a0 0x7fbb140d34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 --2- 172.21.15.37:0/2782289150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb1415db30 0x7fbb140ce360 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 -- 172.21.15.37:0/2782289150 >> 172.21.15.37:0/2782289150 conn(0x7fbb1409cad0 msgr2=0x7fbb1408b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:29.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 -- 172.21.15.37:0/2782289150 shutdown_connections 2024-04-25T08:55:29.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 -- 172.21.15.37:0/2782289150 wait complete. 2024-04-25T08:55:29.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 Processor -- start 2024-04-25T08:55:29.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 -- start start 2024-04-25T08:55:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb140ce8a0 0x7fbb1414e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb140d39e0 0x7fbb141494d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb1415db30 0x7fbb1414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb140d0140 con 0x7fbb140d39e0 2024-04-25T08:55:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1406ce50 con 0x7fbb140ce8a0 2024-04-25T08:55:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1900a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb140d02c0 con 0x7fbb1415db30 2024-04-25T08:55:29.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb140d39e0 0x7fbb141494d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb140d39e0 0x7fbb141494d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38874/0 (socket says 172.21.15.37:38874) 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1259c640 1 -- 172.21.15.37:0/245424070 learned_addr learned my addr 172.21.15.37:0/245424070 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1259c640 1 -- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb1415db30 msgr2=0x7fbb1414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1259c640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb1415db30 0x7fbb1414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1259c640 1 -- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb140ce8a0 msgr2=0x7fbb1414e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1259c640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb140ce8a0 0x7fbb1414e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1259c640 1 -- 172.21.15.37:0/245424070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb140b1a70 con 0x7fbb140d39e0 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.671+0000 7fbb1259c640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb140d39e0 0x7fbb141494d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800d800 tx=0x7fbb0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.675+0000 7fbafb7fe640 1 -- 172.21.15.37:0/245424070 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08018020 con 0x7fbb140d39e0 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.675+0000 7fbafb7fe640 1 -- 172.21.15.37:0/245424070 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb080096a0 con 0x7fbb140d39e0 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.675+0000 7fbb1900a640 1 -- 172.21.15.37:0/245424070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb140b1d50 con 0x7fbb140d39e0 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.675+0000 7fbafb7fe640 1 -- 172.21.15.37:0/245424070 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08009a50 con 0x7fbb140d39e0 2024-04-25T08:55:29.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.675+0000 7fbb1900a640 1 -- 172.21.15.37:0/245424070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb140b2310 con 0x7fbb140d39e0 2024-04-25T08:55:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.675+0000 7fbb1900a640 1 -- 172.21.15.37:0/245424070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1404efa0 con 0x7fbb140d39e0 2024-04-25T08:55:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.675+0000 7fbafb7fe640 1 -- 172.21.15.37:0/245424070 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbb0801b020 con 0x7fbb140d39e0 2024-04-25T08:55:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.675+0000 7fbafb7fe640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbb00066f60 0x7fbb00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.675+0000 7fbafb7fe640 1 -- 172.21.15.37:0/245424070 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbb080857d0 con 0x7fbb140d39e0 2024-04-25T08:55:29.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.675+0000 7fbb12d9d640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbb00066f60 0x7fbb00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.675+0000 7fbb12d9d640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbb00066f60 0x7fbb00069420 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb040af000 tx=0x7fbb04004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.679+0000 7fbafb7fe640 1 -- 172.21.15.37:0/245424070 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbb08022020 con 0x7fbb140d39e0 2024-04-25T08:55:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.839+0000 7fbb1900a640 1 -- 172.21.15.37:0/245424070 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb1406bf90 con 0x7fbb140d39e0 2024-04-25T08:55:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.839+0000 7fbafb7fe640 1 -- 172.21.15.37:0/245424070 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb08056620 con 0x7fbb140d39e0 2024-04-25T08:55:29.830 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:55:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 -- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbb00066f60 msgr2=0x7fbb00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbb00066f60 0x7fbb00069420 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbb040af000 tx=0x7fbb04004740 comp rx=0 tx=0).stop 2024-04-25T08:55:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 -- 172.21.15.37:0/245424070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb140d39e0 msgr2=0x7fbb141494d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb140d39e0 0x7fbb141494d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fbb0800d800 tx=0x7fbb0800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 -- 172.21.15.37:0/245424070 shutdown_connections 2024-04-25T08:55:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbb00066f60 0x7fbb00069420 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbb1415db30 0x7fbb1414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb140d39e0 0x7fbb141494d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 --2- 172.21.15.37:0/245424070 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbb140ce8a0 0x7fbb1414e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 -- 172.21.15.37:0/245424070 >> 172.21.15.37:0/245424070 conn(0x7fbb1409cad0 msgr2=0x7fbb140b4890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:29.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 -- 172.21.15.37:0/245424070 shutdown_connections 2024-04-25T08:55:29.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.843+0000 7fbb1900a640 1 -- 172.21.15.37:0/245424070 wait complete. 2024-04-25T08:55:29.848 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:55:29.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.987+0000 7f77026ff640 1 Processor -- start 2024-04-25T08:55:29.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.987+0000 7f77026ff640 1 -- start start 2024-04-25T08:55:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.987+0000 7f77026ff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f76fc15da80 0x7f76fc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.987+0000 7f77026ff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f76fc0d1cd0 0x7f76fc0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.987+0000 7f77026ff640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76fc06be60 con 0x7f76fc0d0620 2024-04-25T08:55:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.987+0000 7f77026ff640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76fc0d0a70 con 0x7f76fc0d1cd0 2024-04-25T08:55:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.987+0000 7f77026ff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76fc06c440 con 0x7f76fc15da80 2024-04-25T08:55:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.987+0000 7f7700c9c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f76fc0d0620 0x7f76fc0ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:43316/0 (socket says 172.21.15.37:43316) 2024-04-25T08:55:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f7700c9c640 1 -- 172.21.15.37:0/3954825140 learned_addr learned my addr 172.21.15.37:0/3954825140 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f76fb7fe640 1 --2- 172.21.15.37:0/3954825140 >> v2:172.21.15.37:3300/0 conn(0x7f76fc0d1cd0 0x7f76fc0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f76fbfff640 1 --2- 172.21.15.37:0/3954825140 >> v2:172.21.15.203:3300/0 conn(0x7f76fc15da80 0x7f76fc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f76fb7fe640 1 -- 172.21.15.37:0/3954825140 >> v2:172.21.15.203:3300/0 conn(0x7f76fc15da80 msgr2=0x7f76fc0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:29.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f76fb7fe640 1 --2- 172.21.15.37:0/3954825140 >> v2:172.21.15.203:3300/0 conn(0x7f76fc15da80 0x7f76fc0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f76fb7fe640 1 -- 172.21.15.37:0/3954825140 >> v1:172.21.15.37:6789/0 conn(0x7f76fc0d0620 legacy=0x7f76fc0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f76fb7fe640 1 -- 172.21.15.37:0/3954825140 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc15de60 con 0x7f76fc0d1cd0 2024-04-25T08:55:29.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f76fb7fe640 1 --2- 172.21.15.37:0/3954825140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76fc0d1cd0 0x7f76fc0d00e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f76f00097b0 tx=0x7f76f001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5d00ce9e4cec1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f76faffd640 1 -- 172.21.15.37:0/3954825140 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f00b0070 con 0x7f76fc0d1cd0 2024-04-25T08:55:29.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f76faffd640 1 -- 172.21.15.37:0/3954825140 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f00ab600 con 0x7f76fc0d1cd0 2024-04-25T08:55:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f76faffd640 1 -- 172.21.15.37:0/3954825140 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f00ab9d0 con 0x7f76fc0d1cd0 2024-04-25T08:55:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f77026ff640 1 -- 172.21.15.37:0/3954825140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76fc0d1cd0 msgr2=0x7f76fc0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f77026ff640 1 --2- 172.21.15.37:0/3954825140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76fc0d1cd0 0x7f76fc0d00e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f76f00097b0 tx=0x7f76f001c460 comp rx=0 tx=0).stop 2024-04-25T08:55:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f77026ff640 1 -- 172.21.15.37:0/3954825140 shutdown_connections 2024-04-25T08:55:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f77026ff640 1 --2- 172.21.15.37:0/3954825140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76fc0d1cd0 0x7f76fc0d00e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f77026ff640 1 --2- 172.21.15.37:0/3954825140 >> v2:172.21.15.203:3300/0 conn(0x7f76fc15da80 0x7f76fc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f77026ff640 1 -- 172.21.15.37:0/3954825140 >> 172.21.15.37:0/3954825140 conn(0x7f76fc08b000 msgr2=0x7f76fc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f77026ff640 1 -- 172.21.15.37:0/3954825140 shutdown_connections 2024-04-25T08:55:29.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.991+0000 7f77026ff640 1 -- 172.21.15.37:0/3954825140 wait complete. 2024-04-25T08:55:29.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f77026ff640 1 Processor -- start 2024-04-25T08:55:29.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f77026ff640 1 -- start start 2024-04-25T08:55:29.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f77026ff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76fc0d0620 0x7f76fc0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f77026ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76fc15da80 0x7f76fc0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f77026ff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76fc0cc080 0x7f76fc155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f77026ff640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc16a730 con 0x7f76fc15da80 2024-04-25T08:55:29.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f77026ff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc16a5b0 con 0x7f76fc0d0620 2024-04-25T08:55:29.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f77026ff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc16a8b0 con 0x7f76fc0cc080 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76fc15da80 0x7f76fc0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76fc15da80 0x7f76fc0b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38882/0 (socket says 172.21.15.37:38882) 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76fb7fe640 1 -- 172.21.15.37:0/4011287769 learned_addr learned my addr 172.21.15.37:0/4011287769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76fbfff640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76fc0d0620 0x7f76fc0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f7700c9c640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76fc0cc080 0x7f76fc155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76fb7fe640 1 -- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76fc0cc080 msgr2=0x7f76fc155930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76fb7fe640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76fc0cc080 0x7f76fc155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76fb7fe640 1 -- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76fc0d0620 msgr2=0x7f76fc0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76fb7fe640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76fc0d0620 0x7f76fc0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76fb7fe640 1 -- 172.21.15.37:0/4011287769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76fc0cc5f0 con 0x7f76fc15da80 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76fbfff640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76fc0d0620 0x7f76fc0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76fb7fe640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76fc15da80 0x7f76fc0b3690 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f76f00ae040 tx=0x7f76f0019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76f8ff9640 1 -- 172.21.15.37:0/4011287769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f00bb070 con 0x7f76fc15da80 2024-04-25T08:55:29.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76f8ff9640 1 -- 172.21.15.37:0/4011287769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f0004030 con 0x7f76fc15da80 2024-04-25T08:55:29.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f76f8ff9640 1 -- 172.21.15.37:0/4011287769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f00b0040 con 0x7f76fc15da80 2024-04-25T08:55:29.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f77026ff640 1 -- 172.21.15.37:0/4011287769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76fc0cc8d0 con 0x7f76fc15da80 2024-04-25T08:55:29.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.995+0000 7f77026ff640 1 -- 172.21.15.37:0/4011287769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76fc0cce90 con 0x7f76fc15da80 2024-04-25T08:55:29.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.999+0000 7f77026ff640 1 -- 172.21.15.37:0/4011287769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76fc04efa0 con 0x7f76fc15da80 2024-04-25T08:55:29.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.999+0000 7f76f8ff9640 1 -- 172.21.15.37:0/4011287769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76f00be070 con 0x7f76fc15da80 2024-04-25T08:55:29.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.999+0000 7f76f8ff9640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76e8066f90 0x7f76e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:29.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.999+0000 7f76f8ff9640 1 -- 172.21.15.37:0/4011287769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f76f011e340 con 0x7f76fc15da80 2024-04-25T08:55:29.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.999+0000 7f76fbfff640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76e8066f90 0x7f76e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:29.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:29.999+0000 7f76fbfff640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76e8066f90 0x7f76e8069450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0097a0 tx=0x7f76ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:29.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.003+0000 7f76f8ff9640 1 -- 172.21.15.37:0/4011287769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76f0016050 con 0x7f76fc15da80 2024-04-25T08:55:30.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.163+0000 7f77026ff640 1 -- 172.21.15.37:0/4011287769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f76fc0d4530 con 0x7f76fc15da80 2024-04-25T08:55:30.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.163+0000 7f76f8ff9640 1 -- 172.21.15.37:0/4011287769 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7f76f00ef0f0 con 0x7f76fc15da80 2024-04-25T08:55:30.154 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:55:30.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 -- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76e8066f90 msgr2=0x7f76e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:30.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76e8066f90 0x7f76e8069450 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0097a0 tx=0x7f76ec008040 comp rx=0 tx=0).stop 2024-04-25T08:55:30.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 -- 172.21.15.37:0/4011287769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76fc15da80 msgr2=0x7f76fc0b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:30.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76fc15da80 0x7f76fc0b3690 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f76f00ae040 tx=0x7f76f0019200 comp rx=0 tx=0).stop 2024-04-25T08:55:30.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 -- 172.21.15.37:0/4011287769 shutdown_connections 2024-04-25T08:55:30.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76e8066f90 0x7f76e8069450 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76fc0cc080 0x7f76fc155930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76fc15da80 0x7f76fc0b3690 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 --2- 172.21.15.37:0/4011287769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76fc0d0620 0x7f76fc0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 -- 172.21.15.37:0/4011287769 >> 172.21.15.37:0/4011287769 conn(0x7f76fc08b000 msgr2=0x7f76fc160890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:30.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 -- 172.21.15.37:0/4011287769 shutdown_connections 2024-04-25T08:55:30.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.167+0000 7f77026ff640 1 -- 172.21.15.37:0/4011287769 wait complete. 2024-04-25T08:55:30.174 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:55:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f9991a05640 1 Processor -- start 2024-04-25T08:55:30.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f9991a05640 1 -- start start 2024-04-25T08:55:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f9991a05640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f998c15a280 0x7f998c15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f9991a05640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f998c0718c0 0x7f998c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f9991a05640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f998c15ae30 con 0x7f998c159ea0 2024-04-25T08:55:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f9991a05640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998c0b2990 con 0x7f998c0718c0 2024-04-25T08:55:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f9991a05640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f998c0b3530 con 0x7f998c15a280 2024-04-25T08:55:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f998b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f998c0718c0 0x7f998c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f998b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f998c0718c0 0x7f998c071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38894/0 (socket says 172.21.15.37:38894) 2024-04-25T08:55:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f998b7fe640 1 -- 172.21.15.37:0/3587242842 learned_addr learned my addr 172.21.15.37:0/3587242842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f99827fc640 1 --2- 172.21.15.37:0/3587242842 >> v2:172.21.15.203:3300/0 conn(0x7f998c15a280 0x7f998c15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f998b7fe640 1 -- 172.21.15.37:0/3587242842 >> v2:172.21.15.203:3300/0 conn(0x7f998c15a280 msgr2=0x7f998c15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:30.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f998b7fe640 1 --2- 172.21.15.37:0/3587242842 >> v2:172.21.15.203:3300/0 conn(0x7f998c15a280 0x7f998c15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f998b7fe640 1 -- 172.21.15.37:0/3587242842 >> v1:172.21.15.203:6789/0 conn(0x7f998c159ea0 legacy=0x7f998c0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:30.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.307+0000 7f998b7fe640 1 -- 172.21.15.37:0/3587242842 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c15b110 con 0x7f998c0718c0 2024-04-25T08:55:30.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f998b7fe640 1 --2- 172.21.15.37:0/3587242842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998c0718c0 0x7f998c071d50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f998c0728e0 tx=0x7f998401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd0ac09065c0e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:30.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f998a7fc640 1 -- 172.21.15.37:0/3587242842 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840a5700 con 0x7f998c0718c0 2024-04-25T08:55:30.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f998a7fc640 1 -- 172.21.15.37:0/3587242842 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99840a58a0 con 0x7f998c0718c0 2024-04-25T08:55:30.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f998a7fc640 1 -- 172.21.15.37:0/3587242842 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99840a5c70 con 0x7f998c0718c0 2024-04-25T08:55:30.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f9991a05640 1 -- 172.21.15.37:0/3587242842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998c0718c0 msgr2=0x7f998c071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:30.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f9991a05640 1 --2- 172.21.15.37:0/3587242842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998c0718c0 0x7f998c071d50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f998c0728e0 tx=0x7f998401c9b0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f9991a05640 1 -- 172.21.15.37:0/3587242842 shutdown_connections 2024-04-25T08:55:30.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f9991a05640 1 --2- 172.21.15.37:0/3587242842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998c0718c0 0x7f998c071d50 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f9991a05640 1 --2- 172.21.15.37:0/3587242842 >> v2:172.21.15.203:3300/0 conn(0x7f998c15a280 0x7f998c15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f9991a05640 1 -- 172.21.15.37:0/3587242842 >> 172.21.15.37:0/3587242842 conn(0x7f998c0b11a0 msgr2=0x7f998c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:30.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f9991a05640 1 -- 172.21.15.37:0/3587242842 shutdown_connections 2024-04-25T08:55:30.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f9991a05640 1 -- 172.21.15.37:0/3587242842 wait complete. 2024-04-25T08:55:30.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f9991a05640 1 Processor -- start 2024-04-25T08:55:30.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.311+0000 7f9991a05640 1 -- start start 2024-04-25T08:55:30.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9991a05640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998c0718c0 0x7f998c14ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9991a05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998c159ea0 0x7f998c14f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9991a05640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f998c15a280 0x7f998c156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9991a05640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c06d420 con 0x7f998c159ea0 2024-04-25T08:55:30.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9991a05640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f998c06d2a0 con 0x7f998c15a280 2024-04-25T08:55:30.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9991a05640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c06d5a0 con 0x7f998c0718c0 2024-04-25T08:55:30.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f99827fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998c159ea0 0x7f998c14f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f99827fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998c159ea0 0x7f998c14f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38902/0 (socket says 172.21.15.37:38902) 2024-04-25T08:55:30.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f99827fc640 1 -- 172.21.15.37:0/1281576896 learned_addr learned my addr 172.21.15.37:0/1281576896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:30.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f998affd640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998c0718c0 0x7f998c14ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f998b7fe640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f998c15a280 0x7f998c156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f99827fc640 1 -- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998c0718c0 msgr2=0x7f998c14ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f99827fc640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998c0718c0 0x7f998c14ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f99827fc640 1 -- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f998c15a280 msgr2=0x7f998c156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f99827fc640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f998c15a280 0x7f998c156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f99827fc640 1 -- 172.21.15.37:0/1281576896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c154ec0 con 0x7f998c159ea0 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f99827fc640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998c159ea0 0x7f998c14f780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f997400d5f0 tx=0x7f997400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9988ff9640 1 -- 172.21.15.37:0/1281576896 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740125f0 con 0x7f998c159ea0 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9988ff9640 1 -- 172.21.15.37:0/1281576896 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9974019070 con 0x7f998c159ea0 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9991a05640 1 -- 172.21.15.37:0/1281576896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f998c1551a0 con 0x7f998c159ea0 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9988ff9640 1 -- 172.21.15.37:0/1281576896 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99740129b0 con 0x7f998c159ea0 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9991a05640 1 -- 172.21.15.37:0/1281576896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f998c14bf50 con 0x7f998c159ea0 2024-04-25T08:55:30.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9991a05640 1 -- 172.21.15.37:0/1281576896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f998c155c90 con 0x7f998c159ea0 2024-04-25T08:55:30.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9988ff9640 1 -- 172.21.15.37:0/1281576896 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9974012b50 con 0x7f998c159ea0 2024-04-25T08:55:30.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9988ff9640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f996c066f60 0x7f996c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f9988ff9640 1 -- 172.21.15.37:0/1281576896 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f9974085620 con 0x7f998c159ea0 2024-04-25T08:55:30.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.315+0000 7f998affd640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f996c066f60 0x7f996c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.319+0000 7f998affd640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f996c066f60 0x7f996c069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f997c004e30 tx=0x7f997c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:30.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.319+0000 7f9988ff9640 1 -- 172.21.15.37:0/1281576896 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9974056460 con 0x7f998c159ea0 2024-04-25T08:55:30.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.491+0000 7f9991a05640 1 -- 172.21.15.37:0/1281576896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f998c04efa0 con 0x7f998c159ea0 2024-04-25T08:55:30.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.491+0000 7f9988ff9640 1 -- 172.21.15.37:0/1281576896 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v57) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9974055d60 con 0x7f998c159ea0 2024-04-25T08:55:30.480 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.491+0000 7f9991a05640 1 -- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f996c066f60 msgr2=0x7f996c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.491+0000 7f9991a05640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f996c066f60 0x7f996c069420 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f997c004e30 tx=0x7f997c009040 comp rx=0 tx=0).stop 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.491+0000 7f9991a05640 1 -- 172.21.15.37:0/1281576896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998c159ea0 msgr2=0x7f998c14f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.491+0000 7f9991a05640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998c159ea0 0x7f998c14f780 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f997400d5f0 tx=0x7f997400dab0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.491+0000 7f9991a05640 1 -- 172.21.15.37:0/1281576896 shutdown_connections 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.491+0000 7f9991a05640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f996c066f60 0x7f996c069420 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.491+0000 7f9991a05640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f998c15a280 0x7f998c156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.495+0000 7f9991a05640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f998c159ea0 0x7f998c14f780 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.495+0000 7f9991a05640 1 --2- 172.21.15.37:0/1281576896 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f998c0718c0 0x7f998c14ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.495+0000 7f9991a05640 1 -- 172.21.15.37:0/1281576896 >> 172.21.15.37:0/1281576896 conn(0x7f998c0b11a0 msgr2=0x7f998c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.495+0000 7f9991a05640 1 -- 172.21.15.37:0/1281576896 shutdown_connections 2024-04-25T08:55:30.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.495+0000 7f9991a05640 1 -- 172.21.15.37:0/1281576896 wait complete. 2024-04-25T08:55:30.498 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T08:55:30.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:30.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:30.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[41298]: fuse finished with error 0 and tester_r 0 2024-04-25T08:55:30.511 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:55:30.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:30.511 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:55:30.573 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:55:30.573 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:55:30.581 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:55:30.581 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:55:30.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:30.582 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:55:30.627 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:55:30.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:55:30.628 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:55:30.629 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:55:30.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:55:30.633 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:55:30.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:30.633 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:55:30.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:55:30.679 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:55:30.680 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:55:30.680 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:55:30.686 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:55:30.686 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:55:30.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:30.686 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:55:30.735 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:55:30.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:55:30.737 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:30.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.871+0000 7f764b3e0640 1 Processor -- start 2024-04-25T08:55:30.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.871+0000 7f764b3e0640 1 -- start start 2024-04-25T08:55:30.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.871+0000 7f764b3e0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f764415da80 0x7f76440d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.871+0000 7f764b3e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76440d07f0 0x7f764406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.871+0000 7f764b3e0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764406be60 con 0x7f76440d1cd0 2024-04-25T08:55:30.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.871+0000 7f764b3e0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764406c440 con 0x7f764415da80 2024-04-25T08:55:30.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.871+0000 7f764b3e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764415de60 con 0x7f76440d07f0 2024-04-25T08:55:30.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.871+0000 7f764917c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f764415da80 0x7f76440d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.871+0000 7f764917c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f764415da80 0x7f76440d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38904/0 (socket says 172.21.15.37:38904) 2024-04-25T08:55:30.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.875+0000 7f764917c640 1 -- 172.21.15.37:0/3445538211 learned_addr learned my addr 172.21.15.37:0/3445538211 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:30.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.875+0000 7f764997d640 1 --2- 172.21.15.37:0/3445538211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76440d07f0 0x7f764406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.875+0000 7f764917c640 1 -- 172.21.15.37:0/3445538211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76440d07f0 msgr2=0x7f764406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:30.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.875+0000 7f764917c640 1 --2- 172.21.15.37:0/3445538211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76440d07f0 0x7f764406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.875+0000 7f764917c640 1 -- 172.21.15.37:0/3445538211 >> v1:172.21.15.203:6789/0 conn(0x7f76440d1cd0 legacy=0x7f76440d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:30.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.875+0000 7f764917c640 1 -- 172.21.15.37:0/3445538211 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76440d2450 con 0x7f764415da80 2024-04-25T08:55:30.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.875+0000 7f764917c640 1 --2- 172.21.15.37:0/3445538211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764415da80 0x7f76440d4230 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f76400098a0 tx=0x7f764001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf62869fbd619bc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:30.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.875+0000 7f763bfff640 1 -- 172.21.15.37:0/3445538211 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76400a7070 con 0x7f764415da80 2024-04-25T08:55:30.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.875+0000 7f763bfff640 1 -- 172.21.15.37:0/3445538211 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76400b0070 con 0x7f764415da80 2024-04-25T08:55:30.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.875+0000 7f763bfff640 1 -- 172.21.15.37:0/3445538211 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76400ab890 con 0x7f764415da80 2024-04-25T08:55:30.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 -- 172.21.15.37:0/3445538211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764415da80 msgr2=0x7f76440d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:30.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 --2- 172.21.15.37:0/3445538211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764415da80 0x7f76440d4230 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f76400098a0 tx=0x7f764001c510 comp rx=0 tx=0).stop 2024-04-25T08:55:30.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 -- 172.21.15.37:0/3445538211 shutdown_connections 2024-04-25T08:55:30.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 --2- 172.21.15.37:0/3445538211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76440d07f0 0x7f764406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 --2- 172.21.15.37:0/3445538211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764415da80 0x7f76440d4230 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 -- 172.21.15.37:0/3445538211 >> 172.21.15.37:0/3445538211 conn(0x7f764408b000 msgr2=0x7f76440d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:30.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 -- 172.21.15.37:0/3445538211 shutdown_connections 2024-04-25T08:55:30.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 -- 172.21.15.37:0/3445538211 wait complete. 2024-04-25T08:55:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 Processor -- start 2024-04-25T08:55:30.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 -- start start 2024-04-25T08:55:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76440d07f0 0x7f76440c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76440d1cd0 0x7f7644158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764415da80 0x7f7644155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764406d7a0 con 0x7f764415da80 2024-04-25T08:55:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764406d620 con 0x7f76440d1cd0 2024-04-25T08:55:30.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764406d920 con 0x7f76440d07f0 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764997d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764415da80 0x7f7644155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764997d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764415da80 0x7f7644155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38914/0 (socket says 172.21.15.37:38914) 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764997d640 1 -- 172.21.15.37:0/1698954656 learned_addr learned my addr 172.21.15.37:0/1698954656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764897b640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76440d1cd0 0x7f7644158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764917c640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76440d07f0 0x7f76440c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764997d640 1 -- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76440d07f0 msgr2=0x7f76440c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764997d640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76440d07f0 0x7f76440c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764997d640 1 -- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76440d1cd0 msgr2=0x7f7644158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764997d640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76440d1cd0 0x7f7644158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764997d640 1 -- 172.21.15.37:0/1698954656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764415b9e0 con 0x7f764415da80 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764917c640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76440d07f0 0x7f76440c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764997d640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764415da80 0x7f7644155400 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f763c007de0 tx=0x7f763c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f7639ffb640 1 -- 172.21.15.37:0/1698954656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c014620 con 0x7f764415da80 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f7639ffb640 1 -- 172.21.15.37:0/1698954656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c019070 con 0x7f764415da80 2024-04-25T08:55:30.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f7639ffb640 1 -- 172.21.15.37:0/1698954656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c014a20 con 0x7f764415da80 2024-04-25T08:55:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 -- 172.21.15.37:0/1698954656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764415bc50 con 0x7f764415da80 2024-04-25T08:55:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.879+0000 7f764b3e0640 1 -- 172.21.15.37:0/1698954656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764415cb40 con 0x7f764415da80 2024-04-25T08:55:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.883+0000 7f764b3e0640 1 -- 172.21.15.37:0/1698954656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764404efa0 con 0x7f764415da80 2024-04-25T08:55:30.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.883+0000 7f7639ffb640 1 -- 172.21.15.37:0/1698954656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f763c014bc0 con 0x7f764415da80 2024-04-25T08:55:30.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.883+0000 7f7639ffb640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7628066f60 0x7f7628069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:30.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.883+0000 7f7639ffb640 1 -- 172.21.15.37:0/1698954656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f763c0859d0 con 0x7f764415da80 2024-04-25T08:55:30.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.883+0000 7f764917c640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7628066f60 0x7f7628069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:30.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.883+0000 7f764917c640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7628066f60 0x7f7628069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f76400af000 tx=0x7f76400a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:30.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:30.887+0000 7f7639ffb640 1 -- 172.21.15.37:0/1698954656 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f763c0568b0 con 0x7f764415da80 2024-04-25T08:55:31.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 -- 172.21.15.37:0/1698954656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7644073d70 con 0x7f764415da80 2024-04-25T08:55:31.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f7639ffb640 1 -- 172.21.15.37:0/1698954656 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f763c0561b0 con 0x7f764415da80 2024-04-25T08:55:31.072 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:31.073 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2963121871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2963121871},{"type":"v1","addr":"172.21.15.203:6837","nonce":2963121871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1490204181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1490204181},{"type":"v1","addr":"172.21.15.203:6835","nonce":1490204181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2218454043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2218454043},{"type":"v1","addr":"172.21.15.37:6833","nonce":2218454043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:42.800130+0000","modified":"2024-04-25T08:54:45.849035+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/3892786810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3892786810},{"type":"v1","addr":"172.21.15.37:6835","nonce":3892786810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:55:31.073 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 -- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7628066f60 msgr2=0x7f7628069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7628066f60 0x7f7628069420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f76400af000 tx=0x7f76400a7000 comp rx=0 tx=0).stop 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 -- 172.21.15.37:0/1698954656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764415da80 msgr2=0x7f7644155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764415da80 0x7f7644155400 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f763c007de0 tx=0x7f763c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 -- 172.21.15.37:0/1698954656 shutdown_connections 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7628066f60 0x7f7628069420 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f764415da80 0x7f7644155400 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76440d1cd0 0x7f7644158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 --2- 172.21.15.37:0/1698954656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76440d07f0 0x7f76440c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 -- 172.21.15.37:0/1698954656 >> 172.21.15.37:0/1698954656 conn(0x7f764408b000 msgr2=0x7f76440b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 -- 172.21.15.37:0/1698954656 shutdown_connections 2024-04-25T08:55:31.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.083+0000 7f764b3e0640 1 -- 172.21.15.37:0/1698954656 wait complete. 2024-04-25T08:55:31.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:31.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f7340865640 1 Processor -- start 2024-04-25T08:55:31.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f7340865640 1 -- start start 2024-04-25T08:55:31.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f7340865640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c06b440 0x7f733c0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f7340865640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f733c06b820 0x7f733c0b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f7340865640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f733c0afb20 con 0x7f733c0b18f0 2024-04-25T08:55:31.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f7340865640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f733c0b06c0 con 0x7f733c06b820 2024-04-25T08:55:31.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f7340865640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0afd90 con 0x7f733c06b440 2024-04-25T08:55:31.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f733a59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c06b440 0x7f733c0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f733a59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c06b440 0x7f733c0b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:45378/0 (socket says 172.21.15.37:45378) 2024-04-25T08:55:31.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f733a59c640 1 -- 172.21.15.37:0/2200547447 learned_addr learned my addr 172.21.15.37:0/2200547447 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:31.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f7339d9b640 1 --2- 172.21.15.37:0/2200547447 >> v2:172.21.15.203:3300/0 conn(0x7f733c06b820 0x7f733c0b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f733a59c640 1 -- 172.21.15.37:0/2200547447 >> v2:172.21.15.203:3300/0 conn(0x7f733c06b820 msgr2=0x7f733c0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:31.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f733a59c640 1 --2- 172.21.15.37:0/2200547447 >> v2:172.21.15.203:3300/0 conn(0x7f733c06b820 0x7f733c0b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f733a59c640 1 -- 172.21.15.37:0/2200547447 >> v1:172.21.15.203:6789/0 conn(0x7f733c0b18f0 legacy=0x7f733c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f733a59c640 1 -- 172.21.15.37:0/2200547447 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c0b0070 con 0x7f733c06b440 2024-04-25T08:55:31.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.231+0000 7f7339d9b640 1 --2- 172.21.15.37:0/2200547447 >> v2:172.21.15.203:3300/0 conn(0x7f733c06b820 0x7f733c0b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:55:31.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f733a59c640 1 --2- 172.21.15.37:0/2200547447 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c06b440 0x7f733c0b0b50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f732c009870 tx=0x7f732c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b685c36535ae905 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:31.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f733959a640 1 -- 172.21.15.37:0/2200547447 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c0b2070 con 0x7f733c06b440 2024-04-25T08:55:31.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f733959a640 1 -- 172.21.15.37:0/2200547447 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c0a9070 con 0x7f733c06b440 2024-04-25T08:55:31.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f733959a640 1 -- 172.21.15.37:0/2200547447 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c0ad880 con 0x7f733c06b440 2024-04-25T08:55:31.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 -- 172.21.15.37:0/2200547447 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c06b440 msgr2=0x7f733c0b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 --2- 172.21.15.37:0/2200547447 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c06b440 0x7f733c0b0b50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f732c009870 tx=0x7f732c01c560 comp rx=0 tx=0).stop 2024-04-25T08:55:31.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 -- 172.21.15.37:0/2200547447 shutdown_connections 2024-04-25T08:55:31.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 --2- 172.21.15.37:0/2200547447 >> v2:172.21.15.203:3300/0 conn(0x7f733c06b820 0x7f733c0b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 --2- 172.21.15.37:0/2200547447 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c06b440 0x7f733c0b0b50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 -- 172.21.15.37:0/2200547447 >> 172.21.15.37:0/2200547447 conn(0x7f733c0b1f10 msgr2=0x7f733c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:31.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 -- 172.21.15.37:0/2200547447 shutdown_connections 2024-04-25T08:55:31.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 -- 172.21.15.37:0/2200547447 wait complete. 2024-04-25T08:55:31.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 Processor -- start 2024-04-25T08:55:31.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 -- start start 2024-04-25T08:55:31.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f733c06b440 0x7f733c1f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733c06b820 0x7f733c0c66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c0b18f0 0x7f733c0ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0cd170 con 0x7f733c06b820 2024-04-25T08:55:31.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0ccff0 con 0x7f733c06b440 2024-04-25T08:55:31.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.235+0000 7f7340865640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f733c0b1d20 con 0x7f733c0b18f0 2024-04-25T08:55:31.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7339d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733c06b820 0x7f733c0c66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7339d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733c06b820 0x7f733c0c66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38930/0 (socket says 172.21.15.37:38930) 2024-04-25T08:55:31.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7339d9b640 1 -- 172.21.15.37:0/348452580 learned_addr learned my addr 172.21.15.37:0/348452580 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:31.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f733ad9d640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c0b18f0 0x7f733c0ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f733a59c640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f733c06b440 0x7f733c1f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7339d9b640 1 -- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c0b18f0 msgr2=0x7f733c0ae920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7339d9b640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c0b18f0 0x7f733c0ae920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7339d9b640 1 -- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f733c06b440 msgr2=0x7f733c1f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7339d9b640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f733c06b440 0x7f733c1f9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7339d9b640 1 -- 172.21.15.37:0/348452580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f733c0c5610 con 0x7f733c06b820 2024-04-25T08:55:31.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7339d9b640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733c06b820 0x7f733c0c66b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f733000e9d0 tx=0x7f733000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:31.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f732affd640 1 -- 172.21.15.37:0/348452580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330017070 con 0x7f733c06b820 2024-04-25T08:55:31.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f732affd640 1 -- 172.21.15.37:0/348452580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f733000ca60 con 0x7f733c06b820 2024-04-25T08:55:31.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7340865640 1 -- 172.21.15.37:0/348452580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f733c0c58f0 con 0x7f733c06b820 2024-04-25T08:55:31.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f732affd640 1 -- 172.21.15.37:0/348452580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733000ce50 con 0x7f733c06b820 2024-04-25T08:55:31.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7340865640 1 -- 172.21.15.37:0/348452580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f733c0c5eb0 con 0x7f733c06b820 2024-04-25T08:55:31.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f7340865640 1 -- 172.21.15.37:0/348452580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f733c04efa0 con 0x7f733c06b820 2024-04-25T08:55:31.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f732affd640 1 -- 172.21.15.37:0/348452580 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f733001a070 con 0x7f733c06b820 2024-04-25T08:55:31.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f732affd640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f731c067100 0x7f731c0695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f732affd640 1 -- 172.21.15.37:0/348452580 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7330085130 con 0x7f733c06b820 2024-04-25T08:55:31.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.239+0000 7f733a59c640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f731c067100 0x7f731c0695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.243+0000 7f733a59c640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f731c067100 0x7f731c0695c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f732c0af000 tx=0x7f732c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:31.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.243+0000 7f732affd640 1 -- 172.21.15.37:0/348452580 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7330055930 con 0x7f733c06b820 2024-04-25T08:55:31.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.407+0000 7f7340865640 1 -- 172.21.15.37:0/348452580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f733c0b6380 con 0x7f733c06b820 2024-04-25T08:55:31.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f732affd640 1 -- 172.21.15.37:0/348452580 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f73300599d0 con 0x7f733c06b820 2024-04-25T08:55:31.400 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:31.400 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":57,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:55:12.296393+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:40.447670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:41.812203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 -- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f731c067100 msgr2=0x7f731c0695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f731c067100 0x7f731c0695c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f732c0af000 tx=0x7f732c0a7000 comp rx=0 tx=0).stop 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 -- 172.21.15.37:0/348452580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733c06b820 msgr2=0x7f733c0c66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733c06b820 0x7f733c0c66b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f733000e9d0 tx=0x7f733000ee90 comp rx=0 tx=0).stop 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 -- 172.21.15.37:0/348452580 shutdown_connections 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f731c067100 0x7f731c0695c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f733c0b18f0 0x7f733c0ae920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f733c06b820 0x7f733c0c66b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 --2- 172.21.15.37:0/348452580 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f733c06b440 0x7f733c1f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 -- 172.21.15.37:0/348452580 >> 172.21.15.37:0/348452580 conn(0x7f733c0b1f10 msgr2=0x7f733c0ad1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 -- 172.21.15.37:0/348452580 shutdown_connections 2024-04-25T08:55:31.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.411+0000 7f7340865640 1 -- 172.21.15.37:0/348452580 wait complete. 2024-04-25T08:55:31.420 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:40.447670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:41.812203+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:55:31.420 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:55:31.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.555+0000 7fac66d6c640 1 Processor -- start 2024-04-25T08:55:31.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.555+0000 7fac66d6c640 1 -- start start 2024-04-25T08:55:31.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac66d6c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac600b2ac0 0x7fac600cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac66d6c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fac6015e1e0 0x7fac6015e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac66d6c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac600ce070 con 0x7fac6015a600 2024-04-25T08:55:31.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac66d6c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6015eb90 con 0x7fac6015e1e0 2024-04-25T08:55:31.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac66d6c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac600ce350 con 0x7fac600b2ac0 2024-04-25T08:55:31.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac65309640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fac6015e1e0 0x7fac6015e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac64b08640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fac6015a600 0x7fac600cee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:43324/0 (socket says 172.21.15.37:43324) 2024-04-25T08:55:31.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac64b08640 1 -- 172.21.15.37:0/410324139 learned_addr learned my addr 172.21.15.37:0/410324139 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:31.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac57fff640 1 --2- 172.21.15.37:0/410324139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac600b2ac0 0x7fac600cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac65309640 1 -- 172.21.15.37:0/410324139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac600b2ac0 msgr2=0x7fac600cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:31.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac65309640 1 --2- 172.21.15.37:0/410324139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac600b2ac0 0x7fac600cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac65309640 1 -- 172.21.15.37:0/410324139 >> v1:172.21.15.37:6789/0 conn(0x7fac6015a600 legacy=0x7fac600cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac65309640 1 -- 172.21.15.37:0/410324139 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac600ce630 con 0x7fac6015e1e0 2024-04-25T08:55:31.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac65309640 1 --2- 172.21.15.37:0/410324139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac6015e1e0 0x7fac6015e650 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fac58009d90 tx=0x7fac5801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b63a8288a10970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:31.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac577fe640 1 -- 172.21.15.37:0/410324139 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac580b4070 con 0x7fac6015e1e0 2024-04-25T08:55:31.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac577fe640 1 -- 172.21.15.37:0/410324139 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac580af600 con 0x7fac6015e1e0 2024-04-25T08:55:31.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.559+0000 7fac577fe640 1 -- 172.21.15.37:0/410324139 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac580af9d0 con 0x7fac6015e1e0 2024-04-25T08:55:31.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 -- 172.21.15.37:0/410324139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac6015e1e0 msgr2=0x7fac6015e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 --2- 172.21.15.37:0/410324139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac6015e1e0 0x7fac6015e650 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fac58009d90 tx=0x7fac5801f150 comp rx=0 tx=0).stop 2024-04-25T08:55:31.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 -- 172.21.15.37:0/410324139 shutdown_connections 2024-04-25T08:55:31.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 --2- 172.21.15.37:0/410324139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac6015e1e0 0x7fac6015e650 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 --2- 172.21.15.37:0/410324139 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac600b2ac0 0x7fac600cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 -- 172.21.15.37:0/410324139 >> 172.21.15.37:0/410324139 conn(0x7fac600b1f10 msgr2=0x7fac600b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:31.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 -- 172.21.15.37:0/410324139 shutdown_connections 2024-04-25T08:55:31.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 -- 172.21.15.37:0/410324139 wait complete. 2024-04-25T08:55:31.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 Processor -- start 2024-04-25T08:55:31.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 -- start start 2024-04-25T08:55:31.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac600b2ac0 0x7fac6014e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac6015a600 0x7fac600c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac6015e1e0 0x7fac600c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac600cc1e0 con 0x7fac6015a600 2024-04-25T08:55:31.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac600cbef0 con 0x7fac600b2ac0 2024-04-25T08:55:31.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6015a9e0 con 0x7fac6015e1e0 2024-04-25T08:55:31.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac57fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac6015a600 0x7fac600c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac57fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac6015a600 0x7fac600c5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38958/0 (socket says 172.21.15.37:38958) 2024-04-25T08:55:31.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac57fff640 1 -- 172.21.15.37:0/1069209093 learned_addr learned my addr 172.21.15.37:0/1069209093 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:31.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac64b08640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac600b2ac0 0x7fac6014e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac65309640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac6015e1e0 0x7fac600c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac57fff640 1 -- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac6015e1e0 msgr2=0x7fac600c3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac57fff640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac6015e1e0 0x7fac600c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac57fff640 1 -- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac600b2ac0 msgr2=0x7fac6014e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac57fff640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac600b2ac0 0x7fac6014e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac57fff640 1 -- 172.21.15.37:0/1069209093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac600c4490 con 0x7fac6015a600 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac64b08640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac600b2ac0 0x7fac6014e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac57fff640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac6015a600 0x7fac600c5530 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fac5000d830 tx=0x7fac5000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac557fa640 1 -- 172.21.15.37:0/1069209093 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5000b6c0 con 0x7fac6015a600 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac557fa640 1 -- 172.21.15.37:0/1069209093 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac50019070 con 0x7fac6015a600 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 -- 172.21.15.37:0/1069209093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac600c4770 con 0x7fac6015a600 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac557fa640 1 -- 172.21.15.37:0/1069209093 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5000ba60 con 0x7fac6015a600 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.563+0000 7fac66d6c640 1 -- 172.21.15.37:0/1069209093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac600c4d30 con 0x7fac6015a600 2024-04-25T08:55:31.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.567+0000 7fac66d6c640 1 -- 172.21.15.37:0/1069209093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6004efa0 con 0x7fac6015a600 2024-04-25T08:55:31.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.567+0000 7fac557fa640 1 -- 172.21.15.37:0/1069209093 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac5001e460 con 0x7fac6015a600 2024-04-25T08:55:31.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.567+0000 7fac557fa640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac4c066f60 0x7fac4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.567+0000 7fac557fa640 1 -- 172.21.15.37:0/1069209093 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fac50085970 con 0x7fac6015a600 2024-04-25T08:55:31.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.567+0000 7fac64b08640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac4c066f60 0x7fac4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.567+0000 7fac64b08640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac4c066f60 0x7fac4c069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fac5c007fd0 tx=0x7fac5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:31.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.571+0000 7fac557fa640 1 -- 172.21.15.37:0/1069209093 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac50056720 con 0x7fac6015a600 2024-04-25T08:55:31.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 -- 172.21.15.37:0/1069209093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac60158ee0 con 0x7fac6015a600 2024-04-25T08:55:31.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac557fa640 1 -- 172.21.15.37:0/1069209093 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac50056020 con 0x7fac6015a600 2024-04-25T08:55:31.720 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 -- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac4c066f60 msgr2=0x7fac4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac4c066f60 0x7fac4c069420 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fac5c007fd0 tx=0x7fac5c009040 comp rx=0 tx=0).stop 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 -- 172.21.15.37:0/1069209093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac6015a600 msgr2=0x7fac600c5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac6015a600 0x7fac600c5530 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fac5000d830 tx=0x7fac5000dcf0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 -- 172.21.15.37:0/1069209093 shutdown_connections 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac4c066f60 0x7fac4c069420 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac6015e1e0 0x7fac600c3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac6015a600 0x7fac600c5530 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 --2- 172.21.15.37:0/1069209093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac600b2ac0 0x7fac6014e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 -- 172.21.15.37:0/1069209093 >> 172.21.15.37:0/1069209093 conn(0x7fac600b1f10 msgr2=0x7fac6015ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 -- 172.21.15.37:0/1069209093 shutdown_connections 2024-04-25T08:55:31.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.731+0000 7fac66d6c640 1 -- 172.21.15.37:0/1069209093 wait complete. 2024-04-25T08:55:31.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:55:31.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6be044640 1 Processor -- start 2024-04-25T08:55:31.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6be044640 1 -- start start 2024-04-25T08:55:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6be044640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6b815dcb0 0x7fe6b80b5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6be044640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6b80b57b0 0x7fe6b8167900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6be044640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b80d2720 con 0x7fe6b815d8d0 2024-04-25T08:55:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6be044640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b80d2290 con 0x7fe6b815dcb0 2024-04-25T08:55:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6be044640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b80d1680 con 0x7fe6b80b57b0 2024-04-25T08:55:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6b6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6b815dcb0 0x7fe6b80b5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6b77fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fe6b815d8d0 0x7fe6b80cee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:51586/0 (socket says 172.21.15.37:51586) 2024-04-25T08:55:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6b77fe640 1 -- 172.21.15.37:0/613983066 learned_addr learned my addr 172.21.15.37:0/613983066 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6b7fff640 1 --2- 172.21.15.37:0/613983066 >> v2:172.21.15.203:3300/0 conn(0x7fe6b80b57b0 0x7fe6b8167900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6b6ffd640 1 -- 172.21.15.37:0/613983066 >> v2:172.21.15.203:3300/0 conn(0x7fe6b80b57b0 msgr2=0x7fe6b8167900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:31.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6b6ffd640 1 --2- 172.21.15.37:0/613983066 >> v2:172.21.15.203:3300/0 conn(0x7fe6b80b57b0 0x7fe6b8167900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6b6ffd640 1 -- 172.21.15.37:0/613983066 >> v1:172.21.15.203:6789/0 conn(0x7fe6b815d8d0 legacy=0x7fe6b80cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.875+0000 7fe6b6ffd640 1 -- 172.21.15.37:0/613983066 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b80d1960 con 0x7fe6b815dcb0 2024-04-25T08:55:31.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6b6ffd640 1 --2- 172.21.15.37:0/613983066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6b815dcb0 0x7fe6b80b5270 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8162670 tx=0x7fe6ac01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be02d66a60aee26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:31.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6b67fc640 1 -- 172.21.15.37:0/613983066 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0b0070 con 0x7fe6b815dcb0 2024-04-25T08:55:31.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6b67fc640 1 -- 172.21.15.37:0/613983066 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac0ab600 con 0x7fe6b815dcb0 2024-04-25T08:55:31.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6b67fc640 1 -- 172.21.15.37:0/613983066 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac0ab9d0 con 0x7fe6b815dcb0 2024-04-25T08:55:31.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6be044640 1 -- 172.21.15.37:0/613983066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6b815dcb0 msgr2=0x7fe6b80b5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6be044640 1 --2- 172.21.15.37:0/613983066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6b815dcb0 0x7fe6b80b5270 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8162670 tx=0x7fe6ac01c480 comp rx=0 tx=0).stop 2024-04-25T08:55:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6be044640 1 -- 172.21.15.37:0/613983066 shutdown_connections 2024-04-25T08:55:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6be044640 1 --2- 172.21.15.37:0/613983066 >> v2:172.21.15.203:3300/0 conn(0x7fe6b80b57b0 0x7fe6b8167900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6be044640 1 --2- 172.21.15.37:0/613983066 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6b815dcb0 0x7fe6b80b5270 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6be044640 1 -- 172.21.15.37:0/613983066 >> 172.21.15.37:0/613983066 conn(0x7fe6b808b000 msgr2=0x7fe6b80d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6be044640 1 -- 172.21.15.37:0/613983066 shutdown_connections 2024-04-25T08:55:31.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6be044640 1 -- 172.21.15.37:0/613983066 wait complete. 2024-04-25T08:55:31.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6be044640 1 Processor -- start 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.879+0000 7fe6be044640 1 -- start start 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6be044640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6b80b57b0 0x7fe6b8159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6be044640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6b815d8d0 0x7fe6b8153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6be044640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b815dcb0 0x7fe6b8153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6be044640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b816a660 con 0x7fe6b80b57b0 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6be044640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b80cf590 con 0x7fe6b815d8d0 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6be044640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b816a7e0 con 0x7fe6b815dcb0 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6b80b57b0 0x7fe6b8159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6b80b57b0 0x7fe6b8159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38972/0 (socket says 172.21.15.37:38972) 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6b77fe640 1 -- 172.21.15.37:0/1490504277 learned_addr learned my addr 172.21.15.37:0/1490504277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6b77fe640 1 -- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b815dcb0 msgr2=0x7fe6b8153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6b6ffd640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6b815d8d0 0x7fe6b8153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6b77fe640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b815dcb0 0x7fe6b8153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6b77fe640 1 -- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6b815d8d0 msgr2=0x7fe6b8153830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:31.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6b77fe640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6b815d8d0 0x7fe6b8153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:31.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6b77fe640 1 -- 172.21.15.37:0/1490504277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b8159e50 con 0x7fe6b80b57b0 2024-04-25T08:55:31.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6b77fe640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6b80b57b0 0x7fe6b8159470 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800c450 tx=0x7fe6a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:31.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe69ffff640 1 -- 172.21.15.37:0/1490504277 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8017070 con 0x7fe6b80b57b0 2024-04-25T08:55:31.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe69ffff640 1 -- 172.21.15.37:0/1490504277 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a8005cf0 con 0x7fe6b80b57b0 2024-04-25T08:55:31.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe69ffff640 1 -- 172.21.15.37:0/1490504277 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8007290 con 0x7fe6b80b57b0 2024-04-25T08:55:31.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6be044640 1 -- 172.21.15.37:0/1490504277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b815a130 con 0x7fe6b80b57b0 2024-04-25T08:55:31.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6be044640 1 -- 172.21.15.37:0/1490504277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b80ca830 con 0x7fe6b80b57b0 2024-04-25T08:55:31.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6be044640 1 -- 172.21.15.37:0/1490504277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b804efa0 con 0x7fe6b80b57b0 2024-04-25T08:55:31.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe69ffff640 1 -- 172.21.15.37:0/1490504277 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6a801a070 con 0x7fe6b80b57b0 2024-04-25T08:55:31.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe69ffff640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6a4066f60 0x7fe6a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:31.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe69ffff640 1 -- 172.21.15.37:0/1490504277 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe6a8084450 con 0x7fe6b80b57b0 2024-04-25T08:55:31.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.883+0000 7fe6b6ffd640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6a4066f60 0x7fe6a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:31.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.887+0000 7fe6b6ffd640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6a4066f60 0x7fe6a4069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0ad000 tx=0x7fe6ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:31.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:31.887+0000 7fe69ffff640 1 -- 172.21.15.37:0/1490504277 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6a8055330 con 0x7fe6b80b57b0 2024-04-25T08:55:32.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.047+0000 7fe6be044640 1 -- 172.21.15.37:0/1490504277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6b80ca5f0 con 0x7fe6b80b57b0 2024-04-25T08:55:32.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.047+0000 7fe69ffff640 1 -- 172.21.15.37:0/1490504277 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v57) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6a8054c30 con 0x7fe6b80b57b0 2024-04-25T08:55:32.037 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:55:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 -- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6a4066f60 msgr2=0x7fe6a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6a4066f60 0x7fe6a4069420 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac0ad000 tx=0x7fe6ac008040 comp rx=0 tx=0).stop 2024-04-25T08:55:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 -- 172.21.15.37:0/1490504277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6b80b57b0 msgr2=0x7fe6b8159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6b80b57b0 0x7fe6b8159470 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe6a800c450 tx=0x7fe6a800c910 comp rx=0 tx=0).stop 2024-04-25T08:55:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 -- 172.21.15.37:0/1490504277 shutdown_connections 2024-04-25T08:55:32.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6a4066f60 0x7fe6a4069420 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6b815dcb0 0x7fe6b8153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6b815d8d0 0x7fe6b8153830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 --2- 172.21.15.37:0/1490504277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6b80b57b0 0x7fe6b8159470 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 -- 172.21.15.37:0/1490504277 >> 172.21.15.37:0/1490504277 conn(0x7fe6b808b000 msgr2=0x7fe6b80b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 -- 172.21.15.37:0/1490504277 shutdown_connections 2024-04-25T08:55:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.051+0000 7fe6be044640 1 -- 172.21.15.37:0/1490504277 wait complete. 2024-04-25T08:55:32.056 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:55:32.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.195+0000 7f84d4fca640 1 Processor -- start 2024-04-25T08:55:32.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.195+0000 7f84d4fca640 1 -- start start 2024-04-25T08:55:32.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84d4fca640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f84d015b6b0 0x7f84d00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84d4fca640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f84d00cf590 0x7f84d00cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84d4fca640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d00cbd20 con 0x7f84d015f130 2024-04-25T08:55:32.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84d4fca640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d015ba90 con 0x7f84d015b6b0 2024-04-25T08:55:32.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84d4fca640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d00cc000 con 0x7f84d00cf590 2024-04-25T08:55:32.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84ce59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f84d015b6b0 0x7f84d00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84ce59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f84d015b6b0 0x7f84d00ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38978/0 (socket says 172.21.15.37:38978) 2024-04-25T08:55:32.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84ce59c640 1 -- 172.21.15.37:0/3208178439 learned_addr learned my addr 172.21.15.37:0/3208178439 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:32.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84cdd9b640 1 --2- 172.21.15.37:0/3208178439 >> v2:172.21.15.203:3300/0 conn(0x7f84d00cf590 0x7f84d00cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84ce59c640 1 -- 172.21.15.37:0/3208178439 >> v2:172.21.15.203:3300/0 conn(0x7f84d00cf590 msgr2=0x7f84d00cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:32.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84ce59c640 1 --2- 172.21.15.37:0/3208178439 >> v2:172.21.15.203:3300/0 conn(0x7f84d00cf590 0x7f84d00cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84ce59c640 1 -- 172.21.15.37:0/3208178439 >> v1:172.21.15.203:6789/0 conn(0x7f84d015f130 legacy=0x7f84d015f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.199+0000 7f84ce59c640 1 -- 172.21.15.37:0/3208178439 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d00cc2e0 con 0x7f84d015b6b0 2024-04-25T08:55:32.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ce59c640 1 --2- 172.21.15.37:0/3208178439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d015b6b0 0x7f84d00ca3c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f84d00b4430 tx=0x7f84c001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc88b73ef572b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:32.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84cd59a640 1 -- 172.21.15.37:0/3208178439 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c00ab610 con 0x7f84d015b6b0 2024-04-25T08:55:32.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84cd59a640 1 -- 172.21.15.37:0/3208178439 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c00b0070 con 0x7f84d015b6b0 2024-04-25T08:55:32.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84cd59a640 1 -- 172.21.15.37:0/3208178439 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c00ab9d0 con 0x7f84d015b6b0 2024-04-25T08:55:32.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 -- 172.21.15.37:0/3208178439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d015b6b0 msgr2=0x7f84d00ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 --2- 172.21.15.37:0/3208178439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d015b6b0 0x7f84d00ca3c0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f84d00b4430 tx=0x7f84c001c520 comp rx=0 tx=0).stop 2024-04-25T08:55:32.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 -- 172.21.15.37:0/3208178439 shutdown_connections 2024-04-25T08:55:32.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 --2- 172.21.15.37:0/3208178439 >> v2:172.21.15.203:3300/0 conn(0x7f84d00cf590 0x7f84d00cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 --2- 172.21.15.37:0/3208178439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d015b6b0 0x7f84d00ca3c0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 -- 172.21.15.37:0/3208178439 >> 172.21.15.37:0/3208178439 conn(0x7f84d006ae60 msgr2=0x7f84d006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:32.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 -- 172.21.15.37:0/3208178439 shutdown_connections 2024-04-25T08:55:32.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 -- 172.21.15.37:0/3208178439 wait complete. 2024-04-25T08:55:32.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 Processor -- start 2024-04-25T08:55:32.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 -- start start 2024-04-25T08:55:32.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d00cf590 0x7f84d01548f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84d015b6b0 0x7f84d00c3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84d015f130 0x7f84d015d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00cadb0 con 0x7f84d00cf590 2024-04-25T08:55:32.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00ca900 con 0x7f84d015f130 2024-04-25T08:55:32.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84d4fca640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d00caf30 con 0x7f84d015b6b0 2024-04-25T08:55:32.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d00cf590 0x7f84d01548f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ce59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d00cf590 0x7f84d01548f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38980/0 (socket says 172.21.15.37:38980) 2024-04-25T08:55:32.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ce59c640 1 -- 172.21.15.37:0/1513277932 learned_addr learned my addr 172.21.15.37:0/1513277932 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:32.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84cdd9b640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84d015b6b0 0x7f84d00c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ced9d640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84d015f130 0x7f84d015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ce59c640 1 -- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84d015b6b0 msgr2=0x7f84d00c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ce59c640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84d015b6b0 0x7f84d00c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ce59c640 1 -- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84d015f130 msgr2=0x7f84d015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ce59c640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84d015f130 0x7f84d015d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ce59c640 1 -- 172.21.15.37:0/1513277932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d015e920 con 0x7f84d00cf590 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ced9d640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84d015f130 0x7f84d015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84cdd9b640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84d015b6b0 0x7f84d00c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84ce59c640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d00cf590 0x7f84d01548f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f84c0019130 tx=0x7f84c001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.203+0000 7f84c6ffd640 1 -- 172.21.15.37:0/1513277932 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c00be5f0 con 0x7f84d00cf590 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.207+0000 7f84c6ffd640 1 -- 172.21.15.37:0/1513277932 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c00be790 con 0x7f84d00cf590 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.207+0000 7f84d4fca640 1 -- 172.21.15.37:0/1513277932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d015ce30 con 0x7f84d00cf590 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.207+0000 7f84c6ffd640 1 -- 172.21.15.37:0/1513277932 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c00beb60 con 0x7f84d00cf590 2024-04-25T08:55:32.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.207+0000 7f84d4fca640 1 -- 172.21.15.37:0/1513277932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d01543c0 con 0x7f84d00cf590 2024-04-25T08:55:32.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.207+0000 7f84d4fca640 1 -- 172.21.15.37:0/1513277932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d004efa0 con 0x7f84d00cf590 2024-04-25T08:55:32.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.207+0000 7f84c6ffd640 1 -- 172.21.15.37:0/1513277932 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84c00b43f0 con 0x7f84d00cf590 2024-04-25T08:55:32.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.207+0000 7f84c6ffd640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84b0066f60 0x7f84b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.207+0000 7f84c6ffd640 1 -- 172.21.15.37:0/1513277932 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f84c011f900 con 0x7f84d00cf590 2024-04-25T08:55:32.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.207+0000 7f84cdd9b640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84b0066f60 0x7f84b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.207+0000 7f84cdd9b640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84b0066f60 0x7f84b0069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f84d00cad80 tx=0x7f84b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:32.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.211+0000 7f84c6ffd640 1 -- 172.21.15.37:0/1513277932 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84c00c4060 con 0x7f84d00cf590 2024-04-25T08:55:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.367+0000 7f84d4fca640 1 -- 172.21.15.37:0/1513277932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f84d00af0a0 con 0x7f84d00cf590 2024-04-25T08:55:32.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.371+0000 7f84c6ffd640 1 -- 172.21.15.37:0/1513277932 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v57) v1 ==== 100+0+11 (secure 0 0 0) 0x7f84c00a8050 con 0x7f84d00cf590 2024-04-25T08:55:32.361 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.371+0000 7f84d4fca640 1 -- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84b0066f60 msgr2=0x7f84b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.371+0000 7f84d4fca640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84b0066f60 0x7f84b0069420 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f84d00cad80 tx=0x7f84b8008040 comp rx=0 tx=0).stop 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.371+0000 7f84d4fca640 1 -- 172.21.15.37:0/1513277932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d00cf590 msgr2=0x7f84d01548f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.371+0000 7f84d4fca640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d00cf590 0x7f84d01548f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f84c0019130 tx=0x7f84c001ad60 comp rx=0 tx=0).stop 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.371+0000 7f84d4fca640 1 -- 172.21.15.37:0/1513277932 shutdown_connections 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.371+0000 7f84d4fca640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84b0066f60 0x7f84b0069420 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.371+0000 7f84d4fca640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84d015f130 0x7f84d015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.371+0000 7f84d4fca640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84d015b6b0 0x7f84d00c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.371+0000 7f84d4fca640 1 --2- 172.21.15.37:0/1513277932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84d00cf590 0x7f84d01548f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.371+0000 7f84d4fca640 1 -- 172.21.15.37:0/1513277932 >> 172.21.15.37:0/1513277932 conn(0x7f84d006ae60 msgr2=0x7f84d00b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.375+0000 7f84d4fca640 1 -- 172.21.15.37:0/1513277932 shutdown_connections 2024-04-25T08:55:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.375+0000 7f84d4fca640 1 -- 172.21.15.37:0/1513277932 wait complete. 2024-04-25T08:55:32.379 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:32.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.507+0000 7f9851d47640 1 Processor -- start 2024-04-25T08:55:32.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.507+0000 7f9851d47640 1 -- start start 2024-04-25T08:55:32.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.507+0000 7f9851d47640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f984c15a280 0x7f984c15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.507+0000 7f9851d47640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f984c0718c0 0x7f984c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.507+0000 7f9851d47640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f984c15ae30 con 0x7f984c159ea0 2024-04-25T08:55:32.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.507+0000 7f9851d47640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984c0b2990 con 0x7f984c15a280 2024-04-25T08:55:32.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.507+0000 7f9851d47640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984c0b3530 con 0x7f984c0718c0 2024-04-25T08:55:32.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.507+0000 7f984affd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f984c15a280 0x7f984c15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.507+0000 7f984affd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f984c15a280 0x7f984c15a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38992/0 (socket says 172.21.15.37:38992) 2024-04-25T08:55:32.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f984affd640 1 -- 172.21.15.37:0/3683819192 learned_addr learned my addr 172.21.15.37:0/3683819192 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:32.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f984bfff640 1 --2- 172.21.15.37:0/3683819192 >> v2:172.21.15.203:3300/0 conn(0x7f984c0718c0 0x7f984c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f984affd640 1 -- 172.21.15.37:0/3683819192 >> v2:172.21.15.203:3300/0 conn(0x7f984c0718c0 msgr2=0x7f984c071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:32.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f984affd640 1 --2- 172.21.15.37:0/3683819192 >> v2:172.21.15.203:3300/0 conn(0x7f984c0718c0 0x7f984c071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f984affd640 1 -- 172.21.15.37:0/3683819192 >> v1:172.21.15.203:6789/0 conn(0x7f984c159ea0 legacy=0x7f984c0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:32.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f984affd640 1 -- 172.21.15.37:0/3683819192 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c15b110 con 0x7f984c15a280 2024-04-25T08:55:32.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f984affd640 1 --2- 172.21.15.37:0/3683819192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c15a280 0x7f984c15a700 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f984c06cef0 tx=0x7f983c01e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a356fe2a3b37aae0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:32.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f984a7fc640 1 -- 172.21.15.37:0/3683819192 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0ab660 con 0x7f984c15a280 2024-04-25T08:55:32.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f984a7fc640 1 -- 172.21.15.37:0/3683819192 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c0ab800 con 0x7f984c15a280 2024-04-25T08:55:32.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f984a7fc640 1 -- 172.21.15.37:0/3683819192 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0abbd0 con 0x7f984c15a280 2024-04-25T08:55:32.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f9851d47640 1 -- 172.21.15.37:0/3683819192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c15a280 msgr2=0x7f984c15a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f9851d47640 1 --2- 172.21.15.37:0/3683819192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c15a280 0x7f984c15a700 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f984c06cef0 tx=0x7f983c01e700 comp rx=0 tx=0).stop 2024-04-25T08:55:32.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f9851d47640 1 -- 172.21.15.37:0/3683819192 shutdown_connections 2024-04-25T08:55:32.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f9851d47640 1 --2- 172.21.15.37:0/3683819192 >> v2:172.21.15.203:3300/0 conn(0x7f984c0718c0 0x7f984c071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f9851d47640 1 --2- 172.21.15.37:0/3683819192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c15a280 0x7f984c15a700 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f9851d47640 1 -- 172.21.15.37:0/3683819192 >> 172.21.15.37:0/3683819192 conn(0x7f984c0b11a0 msgr2=0x7f984c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:32.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f9851d47640 1 -- 172.21.15.37:0/3683819192 shutdown_connections 2024-04-25T08:55:32.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.511+0000 7f9851d47640 1 -- 172.21.15.37:0/3683819192 wait complete. 2024-04-25T08:55:32.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9851d47640 1 Processor -- start 2024-04-25T08:55:32.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9851d47640 1 -- start start 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9851d47640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c0718c0 0x7f984c139d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9851d47640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c159ea0 0x7f984c158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9851d47640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f984c15a280 0x7f984c158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9851d47640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c06d420 con 0x7f984c159ea0 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9851d47640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c06d2a0 con 0x7f984c15a280 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9851d47640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984c06d5a0 con 0x7f984c0718c0 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c159ea0 0x7f984c158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c159ea0 0x7f984c158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39000/0 (socket says 172.21.15.37:39000) 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984affd640 1 -- 172.21.15.37:0/3112708136 learned_addr learned my addr 172.21.15.37:0/3112708136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984b7fe640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c0718c0 0x7f984c139d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984bfff640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f984c15a280 0x7f984c158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984affd640 1 -- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c0718c0 msgr2=0x7f984c139d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984affd640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c0718c0 0x7f984c139d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984affd640 1 -- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f984c15a280 msgr2=0x7f984c158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984affd640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f984c15a280 0x7f984c158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984affd640 1 -- 172.21.15.37:0/3112708136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c1577d0 con 0x7f984c159ea0 2024-04-25T08:55:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984b7fe640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c0718c0 0x7f984c139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f984affd640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c159ea0 0x7f984c158220 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f983c01cdb0 tx=0x7f983c01b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9833fff640 1 -- 172.21.15.37:0/3112708136 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0b3800 con 0x7f984c159ea0 2024-04-25T08:55:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9833fff640 1 -- 172.21.15.37:0/3112708136 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c0b39a0 con 0x7f984c159ea0 2024-04-25T08:55:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9851d47640 1 -- 172.21.15.37:0/3112708136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c157ab0 con 0x7f984c159ea0 2024-04-25T08:55:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9833fff640 1 -- 172.21.15.37:0/3112708136 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c0b3d50 con 0x7f984c159ea0 2024-04-25T08:55:32.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9851d47640 1 -- 172.21.15.37:0/3112708136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c13a2a0 con 0x7f984c159ea0 2024-04-25T08:55:32.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.515+0000 7f9851d47640 1 -- 172.21.15.37:0/3112708136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984c157fd0 con 0x7f984c159ea0 2024-04-25T08:55:32.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.519+0000 7f9833fff640 1 -- 172.21.15.37:0/3112708136 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f983c0c63f0 con 0x7f984c159ea0 2024-04-25T08:55:32.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.519+0000 7f9833fff640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9838066f60 0x7f9838069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.519+0000 7f9833fff640 1 -- 172.21.15.37:0/3112708136 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f983c121e00 con 0x7f984c159ea0 2024-04-25T08:55:32.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.519+0000 7f984b7fe640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9838066f60 0x7f9838069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.519+0000 7f984b7fe640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9838066f60 0x7f9838069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9840003e40 tx=0x7f9840009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:32.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.523+0000 7f9833fff640 1 -- 172.21.15.37:0/3112708136 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f984c157fd0 con 0x7f984c159ea0 2024-04-25T08:55:32.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.715+0000 7f9851d47640 1 -- 172.21.15.37:0/3112708136 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f984c04efa0 con 0x7f984c159ea0 2024-04-25T08:55:32.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.715+0000 7f9833fff640 1 -- 172.21.15.37:0/3112708136 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f983c0af020 con 0x7f984c159ea0 2024-04-25T08:55:32.706 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:32.706 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2963121871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2963121871},{"type":"v1","addr":"172.21.15.203:6837","nonce":2963121871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1490204181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1490204181},{"type":"v1","addr":"172.21.15.203:6835","nonce":1490204181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2218454043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2218454043},{"type":"v1","addr":"172.21.15.37:6833","nonce":2218454043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:42.800130+0000","modified":"2024-04-25T08:54:45.849035+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/3892786810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3892786810},{"type":"v1","addr":"172.21.15.37:6835","nonce":3892786810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:55:32.706 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-25T08:55:32.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 -- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9838066f60 msgr2=0x7f9838069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9838066f60 0x7f9838069420 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9840003e40 tx=0x7f9840009040 comp rx=0 tx=0).stop 2024-04-25T08:55:32.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 -- 172.21.15.37:0/3112708136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c159ea0 msgr2=0x7f984c158220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c159ea0 0x7f984c158220 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f983c01cdb0 tx=0x7f983c01b4a0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 -- 172.21.15.37:0/3112708136 shutdown_connections 2024-04-25T08:55:32.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9838066f60 0x7f9838069420 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f984c15a280 0x7f984c158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984c159ea0 0x7f984c158220 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 --2- 172.21.15.37:0/3112708136 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f984c0718c0 0x7f984c139d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 -- 172.21.15.37:0/3112708136 >> 172.21.15.37:0/3112708136 conn(0x7f984c0b11a0 msgr2=0x7f984c15b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:32.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 -- 172.21.15.37:0/3112708136 shutdown_connections 2024-04-25T08:55:32.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.719+0000 7f9851d47640 1 -- 172.21.15.37:0/3112708136 wait complete. 2024-04-25T08:55:32.724 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:32.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00c4fcf640 1 Processor -- start 2024-04-25T08:55:32.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00c4fcf640 1 -- start start 2024-04-25T08:55:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00c4fcf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f00c0066e30 0x7f00c006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00c4fcf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c00af370 0x7f00c00ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00c4fcf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00c006ec30 con 0x7f00c006bca0 2024-04-25T08:55:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00c4fcf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00c00669a0 con 0x7f00c0066e30 2024-04-25T08:55:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00c4fcf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c006c5c0 con 0x7f00c00af370 2024-04-25T08:55:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00bdd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f00c0066e30 0x7f00c006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00bdd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f00c0066e30 0x7f00c006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39012/0 (socket says 172.21.15.37:39012) 2024-04-25T08:55:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00bdd9b640 1 -- 172.21.15.37:0/1826965702 learned_addr learned my addr 172.21.15.37:0/1826965702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:32.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00bed9d640 1 --2- 172.21.15.37:0/1826965702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c00af370 0x7f00c00ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00bdd9b640 1 -- 172.21.15.37:0/1826965702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c00af370 msgr2=0x7f00c00ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:32.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00bdd9b640 1 --2- 172.21.15.37:0/1826965702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c00af370 0x7f00c00ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00bdd9b640 1 -- 172.21.15.37:0/1826965702 >> v1:172.21.15.203:6789/0 conn(0x7f00c006bca0 legacy=0x7f00c00cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:32.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.847+0000 7f00bdd9b640 1 -- 172.21.15.37:0/1826965702 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c006f230 con 0x7f00c0066e30 2024-04-25T08:55:32.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00bdd9b640 1 --2- 172.21.15.37:0/1826965702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c0066e30 0x7f00c006c080 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009e50 tx=0x7f00a801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fcca34748b9b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:32.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00bd59a640 1 -- 172.21.15.37:0/1826965702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a80a5740 con 0x7f00c0066e30 2024-04-25T08:55:32.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00bd59a640 1 -- 172.21.15.37:0/1826965702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a80a58e0 con 0x7f00c0066e30 2024-04-25T08:55:32.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00bd59a640 1 -- 172.21.15.37:0/1826965702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a80a5cb0 con 0x7f00c0066e30 2024-04-25T08:55:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00c4fcf640 1 -- 172.21.15.37:0/1826965702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c0066e30 msgr2=0x7f00c006c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00c4fcf640 1 --2- 172.21.15.37:0/1826965702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c0066e30 0x7f00c006c080 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f00a8009e50 tx=0x7f00a801c8a0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00c4fcf640 1 -- 172.21.15.37:0/1826965702 shutdown_connections 2024-04-25T08:55:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00c4fcf640 1 --2- 172.21.15.37:0/1826965702 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c00af370 0x7f00c00ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00c4fcf640 1 --2- 172.21.15.37:0/1826965702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c0066e30 0x7f00c006c080 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00c4fcf640 1 -- 172.21.15.37:0/1826965702 >> 172.21.15.37:0/1826965702 conn(0x7f00c0071810 msgr2=0x7f00c0071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00c4fcf640 1 -- 172.21.15.37:0/1826965702 shutdown_connections 2024-04-25T08:55:32.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00c4fcf640 1 -- 172.21.15.37:0/1826965702 wait complete. 2024-04-25T08:55:32.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00c4fcf640 1 Processor -- start 2024-04-25T08:55:32.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.851+0000 7f00c4fcf640 1 -- start start 2024-04-25T08:55:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00c4fcf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c0066e30 0x7f00c0148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00c4fcf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c006bca0 0x7f00c0156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00c4fcf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00c00af370 0x7f00c0156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00c4fcf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c01687b0 con 0x7f00c0066e30 2024-04-25T08:55:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00c4fcf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c0168630 con 0x7f00c00af370 2024-04-25T08:55:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00c4fcf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00c0168930 con 0x7f00c006bca0 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00be59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c0066e30 0x7f00c0148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00be59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c0066e30 0x7f00c0148730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39024/0 (socket says 172.21.15.37:39024) 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00be59c640 1 -- 172.21.15.37:0/784589872 learned_addr learned my addr 172.21.15.37:0/784589872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00be59c640 1 -- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c006bca0 msgr2=0x7f00c0156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00bdd9b640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c006bca0 0x7f00c0156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00bed9d640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00c00af370 0x7f00c0156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00be59c640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c006bca0 0x7f00c0156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00be59c640 1 -- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00c00af370 msgr2=0x7f00c0156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00be59c640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00c00af370 0x7f00c0156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00be59c640 1 -- 172.21.15.37:0/784589872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c0149cf0 con 0x7f00c0066e30 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00be59c640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c0066e30 0x7f00c0148730 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f00b800c480 tx=0x7f00b800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00b6ffd640 1 -- 172.21.15.37:0/784589872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8012830 con 0x7f00c0066e30 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00b6ffd640 1 -- 172.21.15.37:0/784589872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b8017070 con 0x7f00c0066e30 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00b6ffd640 1 -- 172.21.15.37:0/784589872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b8012c30 con 0x7f00c0066e30 2024-04-25T08:55:32.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00c4fcf640 1 -- 172.21.15.37:0/784589872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c0149fd0 con 0x7f00c0066e30 2024-04-25T08:55:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00c4fcf640 1 -- 172.21.15.37:0/784589872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c014a590 con 0x7f00c0066e30 2024-04-25T08:55:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00c4fcf640 1 -- 172.21.15.37:0/784589872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c004efa0 con 0x7f00c0066e30 2024-04-25T08:55:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00b6ffd640 1 -- 172.21.15.37:0/784589872 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00b801c460 con 0x7f00c0066e30 2024-04-25T08:55:32.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00b6ffd640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00a0066f90 0x7f00a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00b6ffd640 1 -- 172.21.15.37:0/784589872 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f00b8084f70 con 0x7f00c0066e30 2024-04-25T08:55:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.855+0000 7f00bdd9b640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00a0066f90 0x7f00a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.859+0000 7f00bdd9b640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00a0066f90 0x7f00a0069450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f00a80ae000 tx=0x7f00a80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:32.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:32.863+0000 7f00b6ffd640 1 -- 172.21.15.37:0/784589872 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f00b8055e50 con 0x7f00c0066e30 2024-04-25T08:55:33.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.015+0000 7f00c4fcf640 1 -- 172.21.15.37:0/784589872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00c0072400 con 0x7f00c0066e30 2024-04-25T08:55:33.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00b6ffd640 1 -- 172.21.15.37:0/784589872 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f00b8055750 con 0x7f00c0066e30 2024-04-25T08:55:33.008 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:33.008 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":57,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:55:12.296393+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:40.447670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:41.812203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:33.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 -- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00a0066f90 msgr2=0x7f00a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00a0066f90 0x7f00a0069450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f00a80ae000 tx=0x7f00a80a7070 comp rx=0 tx=0).stop 2024-04-25T08:55:33.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 -- 172.21.15.37:0/784589872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c0066e30 msgr2=0x7f00c0148730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c0066e30 0x7f00c0148730 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f00b800c480 tx=0x7f00b800c940 comp rx=0 tx=0).stop 2024-04-25T08:55:33.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 -- 172.21.15.37:0/784589872 shutdown_connections 2024-04-25T08:55:33.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00a0066f90 0x7f00a0069450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00c00af370 0x7f00c0156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00c006bca0 0x7f00c0156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 --2- 172.21.15.37:0/784589872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00c0066e30 0x7f00c0148730 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 -- 172.21.15.37:0/784589872 >> 172.21.15.37:0/784589872 conn(0x7f00c0071810 msgr2=0x7f00c0155ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:33.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 -- 172.21.15.37:0/784589872 shutdown_connections 2024-04-25T08:55:33.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.019+0000 7f00c4fcf640 1 -- 172.21.15.37:0/784589872 wait complete. 2024-04-25T08:55:33.027 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:55:33.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:33.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d49529640 1 Processor -- start 2024-04-25T08:55:33.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d49529640 1 -- start start 2024-04-25T08:55:33.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d49529640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7d4415a700 0x7f7d4415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d49529640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7d440b24a0 0x7f7d440b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d49529640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d440b2090 con 0x7f7d440b3f50 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d49529640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4415f200 con 0x7f7d440b24a0 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d49529640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d440b43e0 con 0x7f7d4415a700 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d437fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7d440b24a0 0x7f7d440b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d427fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7d440b3f50 0x7f7d440cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:43326/0 (socket says 172.21.15.37:43326) 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d427fc640 1 -- 172.21.15.37:0/4115514209 learned_addr learned my addr 172.21.15.37:0/4115514209 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d42ffd640 1 --2- 172.21.15.37:0/4115514209 >> v2:172.21.15.203:3300/0 conn(0x7f7d4415a700 0x7f7d4415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d437fe640 1 -- 172.21.15.37:0/4115514209 >> v2:172.21.15.203:3300/0 conn(0x7f7d4415a700 msgr2=0x7f7d4415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d437fe640 1 --2- 172.21.15.37:0/4115514209 >> v2:172.21.15.203:3300/0 conn(0x7f7d4415a700 0x7f7d4415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d437fe640 1 -- 172.21.15.37:0/4115514209 >> v1:172.21.15.37:6789/0 conn(0x7f7d440b3f50 legacy=0x7f7d440cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d437fe640 1 -- 172.21.15.37:0/4115514209 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d440b46c0 con 0x7f7d440b24a0 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d41ffb640 1 -- 172.21.15.37:0/4115514209 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597878572 0 0) 0x7f7d440b2090 con 0x7f7d440b3f50 2024-04-25T08:55:33.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.151+0000 7f7d42ffd640 1 --2- 172.21.15.37:0/4115514209 >> v2:172.21.15.203:3300/0 conn(0x7f7d4415a700 0x7f7d4415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d437fe640 1 --2- 172.21.15.37:0/4115514209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d440b24a0 0x7f7d440b28f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7d4406bd20 tx=0x7f7d3c01c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d6408b68f9d81a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d41ffb640 1 -- 172.21.15.37:0/4115514209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c0b2070 con 0x7f7d440b24a0 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d41ffb640 1 -- 172.21.15.37:0/4115514209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3c0a57c0 con 0x7f7d440b24a0 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d41ffb640 1 -- 172.21.15.37:0/4115514209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c0a5b90 con 0x7f7d440b24a0 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 -- 172.21.15.37:0/4115514209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d440b24a0 msgr2=0x7f7d440b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 --2- 172.21.15.37:0/4115514209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d440b24a0 0x7f7d440b28f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f7d4406bd20 tx=0x7f7d3c01c810 comp rx=0 tx=0).stop 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 -- 172.21.15.37:0/4115514209 shutdown_connections 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 --2- 172.21.15.37:0/4115514209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d440b24a0 0x7f7d440b28f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 --2- 172.21.15.37:0/4115514209 >> v2:172.21.15.203:3300/0 conn(0x7f7d4415a700 0x7f7d4415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 -- 172.21.15.37:0/4115514209 >> 172.21.15.37:0/4115514209 conn(0x7f7d440b11a0 msgr2=0x7f7d440b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 -- 172.21.15.37:0/4115514209 shutdown_connections 2024-04-25T08:55:33.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 -- 172.21.15.37:0/4115514209 wait complete. 2024-04-25T08:55:33.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 Processor -- start 2024-04-25T08:55:33.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 -- start start 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d440b24a0 0x7f7d440abf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d4415a700 0x7f7d441553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d440a8ee0 0x7f7d440a9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d440cd310 con 0x7f7d440b24a0 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d440ce600 con 0x7f7d440a8ee0 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d440cd490 con 0x7f7d4415a700 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d42ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d440b24a0 0x7f7d440abf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d42ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d440b24a0 0x7f7d440abf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39036/0 (socket says 172.21.15.37:39036) 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d42ffd640 1 -- 172.21.15.37:0/2437786971 learned_addr learned my addr 172.21.15.37:0/2437786971 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d42ffd640 1 -- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d4415a700 msgr2=0x7f7d441553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d42ffd640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d4415a700 0x7f7d441553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d42ffd640 1 -- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d440a8ee0 msgr2=0x7f7d440a9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d42ffd640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d440a8ee0 0x7f7d440a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d42ffd640 1 -- 172.21.15.37:0/2437786971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d440ac490 con 0x7f7d440b24a0 2024-04-25T08:55:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d42ffd640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d440b24a0 0x7f7d440abf50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400c7c0 tx=0x7f7d3400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d3b7fe640 1 -- 172.21.15.37:0/2437786971 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d34007a30 con 0x7f7d440b24a0 2024-04-25T08:55:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d3b7fe640 1 -- 172.21.15.37:0/2437786971 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d34007bd0 con 0x7f7d440b24a0 2024-04-25T08:55:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 -- 172.21.15.37:0/2437786971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d440a99a0 con 0x7f7d440b24a0 2024-04-25T08:55:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d3b7fe640 1 -- 172.21.15.37:0/2437786971 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d340113f0 con 0x7f7d440b24a0 2024-04-25T08:55:33.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.155+0000 7f7d49529640 1 -- 172.21.15.37:0/2437786971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d4415b2b0 con 0x7f7d440b24a0 2024-04-25T08:55:33.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.159+0000 7f7d49529640 1 -- 172.21.15.37:0/2437786971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d4404efa0 con 0x7f7d440b24a0 2024-04-25T08:55:33.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.159+0000 7f7d3b7fe640 1 -- 172.21.15.37:0/2437786971 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7d34011590 con 0x7f7d440b24a0 2024-04-25T08:55:33.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.159+0000 7f7d3b7fe640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d24066f60 0x7f7d24069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.159+0000 7f7d3b7fe640 1 -- 172.21.15.37:0/2437786971 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7d340853c0 con 0x7f7d440b24a0 2024-04-25T08:55:33.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.159+0000 7f7d427fc640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d24066f60 0x7f7d24069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.159+0000 7f7d427fc640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d24066f60 0x7f7d24069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c007fd0 tx=0x7f7d2c007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:33.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.163+0000 7f7d3b7fe640 1 -- 172.21.15.37:0/2437786971 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7d34015020 con 0x7f7d440b24a0 2024-04-25T08:55:33.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.347+0000 7f7d49529640 1 -- 172.21.15.37:0/2437786971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d4406f260 con 0x7f7d440b24a0 2024-04-25T08:55:33.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d3b7fe640 1 -- 172.21.15.37:0/2437786971 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f7d340561f0 con 0x7f7d440b24a0 2024-04-25T08:55:33.339 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:33.339 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2963121871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2963121871},{"type":"v1","addr":"172.21.15.203:6837","nonce":2963121871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1490204181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1490204181},{"type":"v1","addr":"172.21.15.203:6835","nonce":1490204181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2218454043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2218454043},{"type":"v1","addr":"172.21.15.37:6833","nonce":2218454043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:42.800130+0000","modified":"2024-04-25T08:54:45.849035+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/3892786810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3892786810},{"type":"v1","addr":"172.21.15.37:6835","nonce":3892786810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:55:33.340 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 -- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d24066f60 msgr2=0x7f7d24069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d24066f60 0x7f7d24069420 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c007fd0 tx=0x7f7d2c007960 comp rx=0 tx=0).stop 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 -- 172.21.15.37:0/2437786971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d440b24a0 msgr2=0x7f7d440abf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d440b24a0 0x7f7d440abf50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400c7c0 tx=0x7f7d3400cc80 comp rx=0 tx=0).stop 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 -- 172.21.15.37:0/2437786971 shutdown_connections 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7d24066f60 0x7f7d24069420 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7d440a8ee0 0x7f7d440a9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7d4415a700 0x7f7d441553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 --2- 172.21.15.37:0/2437786971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d440b24a0 0x7f7d440abf50 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 -- 172.21.15.37:0/2437786971 >> 172.21.15.37:0/2437786971 conn(0x7f7d440b11a0 msgr2=0x7f7d44163990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 -- 172.21.15.37:0/2437786971 shutdown_connections 2024-04-25T08:55:33.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.351+0000 7f7d49529640 1 -- 172.21.15.37:0/2437786971 wait complete. 2024-04-25T08:55:33.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:33.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfab2ab640 1 Processor -- start 2024-04-25T08:55:33.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfab2ab640 1 -- start start 2024-04-25T08:55:33.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfab2ab640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcfa415a280 0x7fcfa415a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfab2ab640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa415ae30 con 0x7fcfa4159ea0 2024-04-25T08:55:33.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfab2ab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa415b110 con 0x7fcfa40718c0 2024-04-25T08:55:33.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfab2ab640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa40b2990 con 0x7fcfa415a280 2024-04-25T08:55:33.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfa8846640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcfa415a280 0x7fcfa415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfa9047640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fcfa4159ea0 0x7fcfa40b2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:43328/0 (socket says 172.21.15.37:43328) 2024-04-25T08:55:33.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfa9047640 1 -- 172.21.15.37:0/17461114 learned_addr learned my addr 172.21.15.37:0/17461114 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:33.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfa8846640 1 -- 172.21.15.37:0/17461114 >> v1:172.21.15.203:6789/0 conn(0x7fcfa40718c0 legacy=0x7fcfa406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:33.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfa8846640 1 -- 172.21.15.37:0/17461114 >> v1:172.21.15.37:6789/0 conn(0x7fcfa4159ea0 legacy=0x7fcfa40b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcf9bfff640 1 -- 172.21.15.37:0/17461114 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296392498 0 0) 0x7fcfa415ae30 con 0x7fcfa4159ea0 2024-04-25T08:55:33.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.487+0000 7fcfa8846640 1 -- 172.21.15.37:0/17461114 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa40b2620 con 0x7fcfa415a280 2024-04-25T08:55:33.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfa8846640 1 --2- 172.21.15.37:0/17461114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfa415a280 0x7fcfa415a700 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400a960 tx=0x7fcf9401d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c9b30838ee84b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:33.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcf9bfff640 1 -- 172.21.15.37:0/17461114 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf940b3070 con 0x7fcfa415a280 2024-04-25T08:55:33.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcf9bfff640 1 -- 172.21.15.37:0/17461114 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf940a6850 con 0x7fcfa415a280 2024-04-25T08:55:33.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcf9bfff640 1 -- 172.21.15.37:0/17461114 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf9400b040 con 0x7fcfa415a280 2024-04-25T08:55:33.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 -- 172.21.15.37:0/17461114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfa415a280 msgr2=0x7fcfa415a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 --2- 172.21.15.37:0/17461114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfa415a280 0x7fcfa415a700 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400a960 tx=0x7fcf9401d870 comp rx=0 tx=0).stop 2024-04-25T08:55:33.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 -- 172.21.15.37:0/17461114 shutdown_connections 2024-04-25T08:55:33.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 --2- 172.21.15.37:0/17461114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfa415a280 0x7fcfa415a700 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 -- 172.21.15.37:0/17461114 >> 172.21.15.37:0/17461114 conn(0x7fcfa40b11a0 msgr2=0x7fcfa40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:33.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 -- 172.21.15.37:0/17461114 shutdown_connections 2024-04-25T08:55:33.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 -- 172.21.15.37:0/17461114 wait complete. 2024-04-25T08:55:33.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 Processor -- start 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 -- start start 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa40718c0 0x7fcfa4149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfa4159ea0 0x7fcfa4148f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfa415a280 0x7fcfa414e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4167bd0 con 0x7fcfa4159ea0 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40b3530 con 0x7fcfa415a280 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4167d50 con 0x7fcfa40718c0 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfa8846640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfa4159ea0 0x7fcfa4148f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfa8846640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfa4159ea0 0x7fcfa4148f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39062/0 (socket says 172.21.15.37:39062) 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfa8846640 1 -- 172.21.15.37:0/292535112 learned_addr learned my addr 172.21.15.37:0/292535112 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfa8846640 1 -- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa40718c0 msgr2=0x7fcfa4149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfa8846640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa40718c0 0x7fcfa4149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfa8846640 1 -- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfa415a280 msgr2=0x7fcfa414e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfa8846640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfa415a280 0x7fcfa414e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfa8846640 1 -- 172.21.15.37:0/292535112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa4151550 con 0x7fcfa4159ea0 2024-04-25T08:55:33.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfa8846640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfa4159ea0 0x7fcfa4148f10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400a960 tx=0x7fcf9401a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:33.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcf99ffb640 1 -- 172.21.15.37:0/292535112 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf940b3040 con 0x7fcfa4159ea0 2024-04-25T08:55:33.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 -- 172.21.15.37:0/292535112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa4151830 con 0x7fcfa4159ea0 2024-04-25T08:55:33.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcf99ffb640 1 -- 172.21.15.37:0/292535112 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf940bd070 con 0x7fcfa4159ea0 2024-04-25T08:55:33.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcf99ffb640 1 -- 172.21.15.37:0/292535112 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf940aea30 con 0x7fcfa4159ea0 2024-04-25T08:55:33.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.491+0000 7fcfab2ab640 1 -- 172.21.15.37:0/292535112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa4149450 con 0x7fcfa4159ea0 2024-04-25T08:55:33.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.495+0000 7fcfab2ab640 1 -- 172.21.15.37:0/292535112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa404efa0 con 0x7fcfa4159ea0 2024-04-25T08:55:33.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.495+0000 7fcf99ffb640 1 -- 172.21.15.37:0/292535112 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf940aebd0 con 0x7fcfa4159ea0 2024-04-25T08:55:33.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.495+0000 7fcf99ffb640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf88066f60 0x7fcf88069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.495+0000 7fcf99ffb640 1 -- 172.21.15.37:0/292535112 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcf9411f780 con 0x7fcfa4159ea0 2024-04-25T08:55:33.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.495+0000 7fcfa9047640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf88066f60 0x7fcf88069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.495+0000 7fcfa9047640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf88066f60 0x7fcf88069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0000c10 tx=0x7fcfa0007910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:33.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.499+0000 7fcf99ffb640 1 -- 172.21.15.37:0/292535112 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcf94017460 con 0x7fcfa4159ea0 2024-04-25T08:55:33.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.683+0000 7fcfab2ab640 1 -- 172.21.15.37:0/292535112 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfa40b2280 con 0x7fcfa4159ea0 2024-04-25T08:55:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.683+0000 7fcf99ffb640 1 -- 172.21.15.37:0/292535112 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fcf940f0550 con 0x7fcfa4159ea0 2024-04-25T08:55:33.674 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:33.674 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2963121871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2963121871},{"type":"v1","addr":"172.21.15.203:6837","nonce":2963121871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1490204181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1490204181},{"type":"v1","addr":"172.21.15.203:6835","nonce":1490204181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2218454043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2218454043},{"type":"v1","addr":"172.21.15.37:6833","nonce":2218454043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:42.800130+0000","modified":"2024-04-25T08:54:45.849035+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5937},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5937":{"gid":5937,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/3892786810","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3892786810},{"type":"v1","addr":"172.21.15.37:6835","nonce":3892786810}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:55:33.674 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-04-25T08:55:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 -- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf88066f60 msgr2=0x7fcf88069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf88066f60 0x7fcf88069420 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0000c10 tx=0x7fcfa0007910 comp rx=0 tx=0).stop 2024-04-25T08:55:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 -- 172.21.15.37:0/292535112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfa4159ea0 msgr2=0x7fcfa4148f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfa4159ea0 0x7fcfa4148f10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcf9400a960 tx=0x7fcf9401a610 comp rx=0 tx=0).stop 2024-04-25T08:55:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 -- 172.21.15.37:0/292535112 shutdown_connections 2024-04-25T08:55:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf88066f60 0x7fcf88069420 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfa415a280 0x7fcfa414e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfa4159ea0 0x7fcfa4148f10 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 --2- 172.21.15.37:0/292535112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfa40718c0 0x7fcfa4149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 -- 172.21.15.37:0/292535112 >> 172.21.15.37:0/292535112 conn(0x7fcfa40b11a0 msgr2=0x7fcfa415b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 -- 172.21.15.37:0/292535112 shutdown_connections 2024-04-25T08:55:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.687+0000 7fcfab2ab640 1 -- 172.21.15.37:0/292535112 wait complete. 2024-04-25T08:55:33.692 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:33.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.819+0000 7f957d31e640 1 Processor -- start 2024-04-25T08:55:33.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.819+0000 7f957d31e640 1 -- start start 2024-04-25T08:55:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f957d31e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f957809a480 0x7f95780cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f957d31e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f95780cfb80 0x7f95780cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f957d31e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95780d04d0 con 0x7f957815b950 2024-04-25T08:55:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f957d31e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957815bd30 con 0x7f957809a480 2024-04-25T08:55:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f957d31e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95780cc0d0 con 0x7f95780cfb80 2024-04-25T08:55:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f95767fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f957809a480 0x7f95780cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f95767fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f957809a480 0x7f95780cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39070/0 (socket says 172.21.15.37:39070) 2024-04-25T08:55:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f95767fc640 1 -- 172.21.15.37:0/1768565919 learned_addr learned my addr 172.21.15.37:0/1768565919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f95777fe640 1 --2- 172.21.15.37:0/1768565919 >> v2:172.21.15.203:3300/0 conn(0x7f95780cfb80 0x7f95780cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f95767fc640 1 -- 172.21.15.37:0/1768565919 >> v2:172.21.15.203:3300/0 conn(0x7f95780cfb80 msgr2=0x7f95780cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f95767fc640 1 --2- 172.21.15.37:0/1768565919 >> v2:172.21.15.203:3300/0 conn(0x7f95780cfb80 0x7f95780cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f95767fc640 1 -- 172.21.15.37:0/1768565919 >> v1:172.21.15.203:6789/0 conn(0x7f957815b950 legacy=0x7f95780cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f95767fc640 1 -- 172.21.15.37:0/1768565919 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957809ab10 con 0x7f957809a480 2024-04-25T08:55:33.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f95767fc640 1 --2- 172.21.15.37:0/1768565919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957809a480 0x7f95780cce50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f95780b3bd0 tx=0x7f956c01c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850a7c00d27e6da7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:33.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f9575ffb640 1 -- 172.21.15.37:0/1768565919 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0b0070 con 0x7f957809a480 2024-04-25T08:55:33.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f9575ffb640 1 -- 172.21.15.37:0/1768565919 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c0ab600 con 0x7f957809a480 2024-04-25T08:55:33.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.823+0000 7f9575ffb640 1 -- 172.21.15.37:0/1768565919 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0ab9d0 con 0x7f957809a480 2024-04-25T08:55:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 -- 172.21.15.37:0/1768565919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957809a480 msgr2=0x7f95780cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 --2- 172.21.15.37:0/1768565919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957809a480 0x7f95780cce50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f95780b3bd0 tx=0x7f956c01c300 comp rx=0 tx=0).stop 2024-04-25T08:55:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 -- 172.21.15.37:0/1768565919 shutdown_connections 2024-04-25T08:55:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 --2- 172.21.15.37:0/1768565919 >> v2:172.21.15.203:3300/0 conn(0x7f95780cfb80 0x7f95780cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 --2- 172.21.15.37:0/1768565919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957809a480 0x7f95780cce50 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 -- 172.21.15.37:0/1768565919 >> 172.21.15.37:0/1768565919 conn(0x7f95780b30b0 msgr2=0x7f95780b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:33.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 -- 172.21.15.37:0/1768565919 shutdown_connections 2024-04-25T08:55:33.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 -- 172.21.15.37:0/1768565919 wait complete. 2024-04-25T08:55:33.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 Processor -- start 2024-04-25T08:55:33.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 -- start start 2024-04-25T08:55:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f957809a480 0x7f9578153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95780cfb80 0x7f957815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957815b950 0x7f95780af340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95780cd390 con 0x7f957815b950 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957806faa0 con 0x7f95780cfb80 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f957d31e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95780b4180 con 0x7f957809a480 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f95777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957815b950 0x7f95780af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f95777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957815b950 0x7f95780af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39082/0 (socket says 172.21.15.37:39082) 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f95777fe640 1 -- 172.21.15.37:0/3683085232 learned_addr learned my addr 172.21.15.37:0/3683085232 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f9576ffd640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f957809a480 0x7f9578153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.827+0000 7f95767fc640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95780cfb80 0x7f957815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f95777fe640 1 -- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f957809a480 msgr2=0x7f9578153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f95777fe640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f957809a480 0x7f9578153340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f95777fe640 1 -- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95780cfb80 msgr2=0x7f957815a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f95777fe640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95780cfb80 0x7f957815a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f95777fe640 1 -- 172.21.15.37:0/3683085232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95780b51e0 con 0x7f957815b950 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f95767fc640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95780cfb80 0x7f957815a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f9576ffd640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f957809a480 0x7f9578153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f95777fe640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957815b950 0x7f95780af340 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9570007e40 tx=0x7f957000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:33.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f955f7fe640 1 -- 172.21.15.37:0/3683085232 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570014620 con 0x7f957815b950 2024-04-25T08:55:33.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f955f7fe640 1 -- 172.21.15.37:0/3683085232 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9570019070 con 0x7f957815b950 2024-04-25T08:55:33.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f955f7fe640 1 -- 172.21.15.37:0/3683085232 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570014a20 con 0x7f957815b950 2024-04-25T08:55:33.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f957d31e640 1 -- 172.21.15.37:0/3683085232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95780b4420 con 0x7f957815b950 2024-04-25T08:55:33.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f957d31e640 1 -- 172.21.15.37:0/3683085232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95780b49e0 con 0x7f957815b950 2024-04-25T08:55:33.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f957d31e640 1 -- 172.21.15.37:0/3683085232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957804efa0 con 0x7f957815b950 2024-04-25T08:55:33.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f955f7fe640 1 -- 172.21.15.37:0/3683085232 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9570014bc0 con 0x7f957815b950 2024-04-25T08:55:33.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f955f7fe640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9564066f60 0x7f9564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:33.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f955f7fe640 1 -- 172.21.15.37:0/3683085232 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f9570085ab0 con 0x7f957815b950 2024-04-25T08:55:33.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f9576ffd640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9564066f60 0x7f9564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:33.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.831+0000 7f9576ffd640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9564066f60 0x7f9564069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f95680098b0 tx=0x7f9568008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:33.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:33.835+0000 7f955f7fe640 1 -- 172.21.15.37:0/3683085232 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9570056900 con 0x7f957815b950 2024-04-25T08:55:33.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.003+0000 7f957d31e640 1 -- 172.21.15.37:0/3683085232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95780d13a0 con 0x7f957815b950 2024-04-25T08:55:33.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.003+0000 7f955f7fe640 1 -- 172.21.15.37:0/3683085232 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16945 (secure 0 0 0) 0x7f9570056200 con 0x7f957815b950 2024-04-25T08:55:33.993 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:33.994 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":57,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:55:12.296393+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:40.447670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:41.812203+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1","mappings":[{"from":0,"to":6}]},{"pgid":"8.1f","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:33.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 -- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9564066f60 msgr2=0x7f9564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9564066f60 0x7f9564069420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f95680098b0 tx=0x7f9568008040 comp rx=0 tx=0).stop 2024-04-25T08:55:33.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 -- 172.21.15.37:0/3683085232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957815b950 msgr2=0x7f95780af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:33.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957815b950 0x7f95780af340 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9570007e40 tx=0x7f957000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 -- 172.21.15.37:0/3683085232 shutdown_connections 2024-04-25T08:55:33.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9564066f60 0x7f9564069420 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f957815b950 0x7f95780af340 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95780cfb80 0x7f957815a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 --2- 172.21.15.37:0/3683085232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f957809a480 0x7f9578153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:33.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 -- 172.21.15.37:0/3683085232 >> 172.21.15.37:0/3683085232 conn(0x7f95780b30b0 msgr2=0x7f95780b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:33.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 -- 172.21.15.37:0/3683085232 shutdown_connections 2024-04-25T08:55:33.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.007+0000 7f957d31e640 1 -- 172.21.15.37:0/3683085232 wait complete. 2024-04-25T08:55:34.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:55:34.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.151+0000 7efc2a5c1640 1 Processor -- start 2024-04-25T08:55:34.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.151+0000 7efc2a5c1640 1 -- start start 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.151+0000 7efc2a5c1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efc2415a700 0x7efc2415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.151+0000 7efc2a5c1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efc240b3f50 0x7efc240cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.151+0000 7efc2a5c1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc240b2090 con 0x7efc240ce430 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.151+0000 7efc2a5c1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2415f200 con 0x7efc2415a700 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.151+0000 7efc2a5c1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc240b43e0 con 0x7efc240b3f50 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.151+0000 7efc23fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efc2415a700 0x7efc2415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.151+0000 7efc23fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efc2415a700 0x7efc2415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39098/0 (socket says 172.21.15.37:39098) 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.151+0000 7efc23fff640 1 -- 172.21.15.37:0/4261129278 learned_addr learned my addr 172.21.15.37:0/4261129278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.151+0000 7efc237fe640 1 --2- 172.21.15.37:0/4261129278 >> v2:172.21.15.203:3300/0 conn(0x7efc240b3f50 0x7efc240cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc23fff640 1 -- 172.21.15.37:0/4261129278 >> v2:172.21.15.203:3300/0 conn(0x7efc240b3f50 msgr2=0x7efc240cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc23fff640 1 --2- 172.21.15.37:0/4261129278 >> v2:172.21.15.203:3300/0 conn(0x7efc240b3f50 0x7efc240cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc23fff640 1 -- 172.21.15.37:0/4261129278 >> v1:172.21.15.203:6789/0 conn(0x7efc240ce430 legacy=0x7efc240b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:34.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc23fff640 1 -- 172.21.15.37:0/4261129278 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc240b46c0 con 0x7efc2415a700 2024-04-25T08:55:34.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc23fff640 1 --2- 172.21.15.37:0/4261129278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc2415a700 0x7efc2415ecc0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efc2406bd20 tx=0x7efc1401c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd6e1e191b3f95f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:34.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc22ffd640 1 -- 172.21.15.37:0/4261129278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc140b0070 con 0x7efc2415a700 2024-04-25T08:55:34.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc22ffd640 1 -- 172.21.15.37:0/4261129278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc14004230 con 0x7efc2415a700 2024-04-25T08:55:34.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc22ffd640 1 -- 172.21.15.37:0/4261129278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc14003620 con 0x7efc2415a700 2024-04-25T08:55:34.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4261129278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc2415a700 msgr2=0x7efc2415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:34.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc2a5c1640 1 --2- 172.21.15.37:0/4261129278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc2415a700 0x7efc2415ecc0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7efc2406bd20 tx=0x7efc1401c350 comp rx=0 tx=0).stop 2024-04-25T08:55:34.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4261129278 shutdown_connections 2024-04-25T08:55:34.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc2a5c1640 1 --2- 172.21.15.37:0/4261129278 >> v2:172.21.15.203:3300/0 conn(0x7efc240b3f50 0x7efc240cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc2a5c1640 1 --2- 172.21.15.37:0/4261129278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc2415a700 0x7efc2415ecc0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4261129278 >> 172.21.15.37:0/4261129278 conn(0x7efc240b11a0 msgr2=0x7efc240b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:34.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4261129278 shutdown_connections 2024-04-25T08:55:34.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.155+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4261129278 wait complete. 2024-04-25T08:55:34.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc2a5c1640 1 Processor -- start 2024-04-25T08:55:34.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc2a5c1640 1 -- start start 2024-04-25T08:55:34.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc2a5c1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc240ce430 0x7efc241589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc2a5c1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc2415a700 0x7efc240af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc2a5c1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc241543c0 0x7efc241f7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc2a5c1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc240cd490 con 0x7efc2415a700 2024-04-25T08:55:34.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc2a5c1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc240cd310 con 0x7efc240ce430 2024-04-25T08:55:34.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc2a5c1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc240b2bb0 con 0x7efc241543c0 2024-04-25T08:55:34.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc2415a700 0x7efc240af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc2415a700 0x7efc240af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39112/0 (socket says 172.21.15.37:39112) 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc237fe640 1 -- 172.21.15.37:0/4237064122 learned_addr learned my addr 172.21.15.37:0/4237064122 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc23fff640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc240ce430 0x7efc241589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc28b5e640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc241543c0 0x7efc241f7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc237fe640 1 -- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc241543c0 msgr2=0x7efc241f7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc237fe640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc241543c0 0x7efc241f7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc237fe640 1 -- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc240ce430 msgr2=0x7efc241589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc237fe640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc240ce430 0x7efc241589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc237fe640 1 -- 172.21.15.37:0/4237064122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc241622d0 con 0x7efc2415a700 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc28b5e640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc241543c0 0x7efc241f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc23fff640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc240ce430 0x7efc241589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc237fe640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc2415a700 0x7efc240af270 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7efc1800d5f0 tx=0x7efc1800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc20ff9640 1 -- 172.21.15.37:0/4237064122 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180125f0 con 0x7efc2415a700 2024-04-25T08:55:34.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc20ff9640 1 -- 172.21.15.37:0/4237064122 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc18019070 con 0x7efc2415a700 2024-04-25T08:55:34.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4237064122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc24158ee0 con 0x7efc2415a700 2024-04-25T08:55:34.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc20ff9640 1 -- 172.21.15.37:0/4237064122 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc180129b0 con 0x7efc2415a700 2024-04-25T08:55:34.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4237064122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc24159460 con 0x7efc2415a700 2024-04-25T08:55:34.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4237064122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc2404efa0 con 0x7efc2415a700 2024-04-25T08:55:34.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.159+0000 7efc20ff9640 1 -- 172.21.15.37:0/4237064122 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efc18012b50 con 0x7efc2415a700 2024-04-25T08:55:34.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.163+0000 7efc20ff9640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc10066f60 0x7efc10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.163+0000 7efc20ff9640 1 -- 172.21.15.37:0/4237064122 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6159+0+0 (secure 0 0 0) 0x7efc180856f0 con 0x7efc2415a700 2024-04-25T08:55:34.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.163+0000 7efc23fff640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc10066f60 0x7efc10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.163+0000 7efc23fff640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc10066f60 0x7efc10069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efc140ad000 tx=0x7efc14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:34.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.167+0000 7efc20ff9640 1 -- 172.21.15.37:0/4237064122 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efc241622d0 con 0x7efc2415a700 2024-04-25T08:55:34.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.359+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4237064122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efc2406c030 con 0x7efc2415a700 2024-04-25T08:55:34.350 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T08:55:34.359+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:55:34.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.363+0000 7efc20ff9640 1 -- 172.21.15.37:0/4237064122 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7efc18011340 con 0x7efc2415a700 2024-04-25T08:55:34.352 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:55:34.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc10066f60 msgr2=0x7efc10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:34.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc10066f60 0x7efc10069420 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efc140ad000 tx=0x7efc14008040 comp rx=0 tx=0).stop 2024-04-25T08:55:34.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4237064122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc2415a700 msgr2=0x7efc240af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:34.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc2415a700 0x7efc240af270 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7efc1800d5f0 tx=0x7efc1800dab0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4237064122 shutdown_connections 2024-04-25T08:55:34.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc10066f60 0x7efc10069420 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efc241543c0 0x7efc241f7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc2415a700 0x7efc240af270 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 --2- 172.21.15.37:0/4237064122 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efc240ce430 0x7efc241589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4237064122 >> 172.21.15.37:0/4237064122 conn(0x7efc240b11a0 msgr2=0x7efc240cfc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:34.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4237064122 shutdown_connections 2024-04-25T08:55:34.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.367+0000 7efc2a5c1640 1 -- 172.21.15.37:0/4237064122 wait complete. 2024-04-25T08:55:34.362 INFO:tasks.ceph.mds.a.smithi037.stderr: -4958> 2024-04-25T08:54:48.403+0000 7f838a683640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:55:34.366 INFO:tasks.ceph.mds.a.smithi037.stderr: -3175> 2024-04-25T08:54:57.583+0000 7f838667b640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6120=0-4194304@1} caps={6120=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6120 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x56373e4ab700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:55:34.374 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:55:34.397 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T08:55:34.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab96ac2640 1 Processor -- start 2024-04-25T08:55:34.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab96ac2640 1 -- start start 2024-04-25T08:55:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab96ac2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fab9006f4a0 0x7fab90099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab96ac2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab90099fb0 0x7fab9015d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab96ac2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab900b0e10 con 0x7fab900b0a30 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab96ac2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab900d0d90 con 0x7fab9006f4a0 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab96ac2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab900d0180 con 0x7fab90099fb0 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab9485e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fab900b0a30 0x7fab900cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:43348/0 (socket says 172.21.15.37:43348) 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab9485e640 1 -- 172.21.15.37:0/413611602 learned_addr learned my addr 172.21.15.37:0/413611602 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab9505f640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab90099fb0 0x7fab9015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab8ffff640 1 --2- 172.21.15.37:0/413611602 >> v2:172.21.15.203:3300/0 conn(0x7fab9006f4a0 0x7fab90099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765459389 0 0) 0x7fab900b0e10 con 0x7fab900b0a30 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.499+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab78003530 con 0x7fab900b0a30 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4157049597 0 0) 0x7fab78003530 con 0x7fab900b0a30 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab90099fb0 msgr2=0x7fab9015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab90099fb0 0x7fab9015d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 >> v2:172.21.15.203:3300/0 conn(0x7fab9006f4a0 msgr2=0x7fab90099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 --2- 172.21.15.37:0/413611602 >> v2:172.21.15.203:3300/0 conn(0x7fab9006f4a0 0x7fab90099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900d0460 con 0x7fab900b0a30 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fab80001680 con 0x7fab900b0a30 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 >> v1:172.21.15.37:6789/0 conn(0x7fab900b0a30 legacy=0x7fab900cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab78003a30 0x7fab78003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab90099fb0 0x7fab78004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab78004d50 0x7fab780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab900d0d90 con 0x7fab78003a30 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab900d0180 con 0x7fab78004d50 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab7800b9d0 con 0x7fab90099fb0 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab9485e640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab78003a30 0x7fab78003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8ffff640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab90099fb0 0x7fab78004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab9485e640 1 -- 172.21.15.37:0/413611602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab90099fb0 msgr2=0x7fab78004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab9485e640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab90099fb0 0x7fab78004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab9485e640 1 -- 172.21.15.37:0/413611602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab78004d50 msgr2=0x7fab780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab9485e640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab78004d50 0x7fab780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab9485e640 1 -- 172.21.15.37:0/413611602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab78003530 con 0x7fab78003a30 2024-04-25T08:55:34.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8ffff640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab90099fb0 0x7fab78004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab9485e640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab78003a30 0x7fab78003e50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fab8000c240 tx=0x7fab8001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8dee76460da0af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab80007d60 con 0x7fab78003a30 2024-04-25T08:55:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab800a9020 con 0x7fab78003a30 2024-04-25T08:55:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab8f7fe640 1 -- 172.21.15.37:0/413611602 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab80005ae0 con 0x7fab78003a30 2024-04-25T08:55:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab96ac2640 1 -- 172.21.15.37:0/413611602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab78003a30 msgr2=0x7fab78003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:34.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab96ac2640 1 --2- 172.21.15.37:0/413611602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab78003a30 0x7fab78003e50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fab8000c240 tx=0x7fab8001ccc0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab9505f640 1 -- 172.21.15.37:0/413611602 reap_dead start 2024-04-25T08:55:34.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab96ac2640 1 -- 172.21.15.37:0/413611602 shutdown_connections 2024-04-25T08:55:34.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab96ac2640 1 -- 172.21.15.37:0/413611602 >> 172.21.15.37:0/413611602 conn(0x7fab900b30b0 msgr2=0x7fab900b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:34.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab96ac2640 1 -- 172.21.15.37:0/413611602 shutdown_connections 2024-04-25T08:55:34.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.503+0000 7fab96ac2640 1 -- 172.21.15.37:0/413611602 wait complete. 2024-04-25T08:55:34.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab96ac2640 1 Processor -- start 2024-04-25T08:55:34.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab96ac2640 1 -- start start 2024-04-25T08:55:34.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab96ac2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab78003a30 0x7fab90150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab96ac2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab900c6e00 0x7fab900b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab96ac2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab900b6c20 0x7fab90155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab96ac2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9015de10 con 0x7fab900b6c20 2024-04-25T08:55:34.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab96ac2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9015dc90 con 0x7fab900c6e00 2024-04-25T08:55:34.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab96ac2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab9015df90 con 0x7fab78003a30 2024-04-25T08:55:34.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab9505f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab900b6c20 0x7fab90155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab9505f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab900b6c20 0x7fab90155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39158/0 (socket says 172.21.15.37:39158) 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab9505f640 1 -- 172.21.15.37:0/4278573299 learned_addr learned my addr 172.21.15.37:0/4278573299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab8ffff640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab900c6e00 0x7fab900b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab9485e640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab78003a30 0x7fab90150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab9505f640 1 -- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab78003a30 msgr2=0x7fab90150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab9505f640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab78003a30 0x7fab90150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab9505f640 1 -- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab900c6e00 msgr2=0x7fab900b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab9505f640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab900c6e00 0x7fab900b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab9505f640 1 -- 172.21.15.37:0/4278573299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab900b5e60 con 0x7fab900b6c20 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab9485e640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab78003a30 0x7fab90150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab9505f640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab900b6c20 0x7fab90155de0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fab900679a0 tx=0x7fab8800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab8d7fa640 1 -- 172.21.15.37:0/4278573299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88018020 con 0x7fab900b6c20 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab8d7fa640 1 -- 172.21.15.37:0/4278573299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab8800bae0 con 0x7fab900b6c20 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab96ac2640 1 -- 172.21.15.37:0/4278573299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab900b6140 con 0x7fab900b6c20 2024-04-25T08:55:34.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab8d7fa640 1 -- 172.21.15.37:0/4278573299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab88014400 con 0x7fab900b6c20 2024-04-25T08:55:34.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab96ac2640 1 -- 172.21.15.37:0/4278573299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab900b6700 con 0x7fab900b6c20 2024-04-25T08:55:34.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab96ac2640 1 -- 172.21.15.37:0/4278573299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab900b6900 con 0x7fab900b6c20 2024-04-25T08:55:34.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab8d7fa640 1 -- 172.21.15.37:0/4278573299 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fab8801b020 con 0x7fab900b6c20 2024-04-25T08:55:34.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.507+0000 7fab8d7fa640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fab7c066f90 0x7fab7c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:34.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.511+0000 7fab8d7fa640 1 -- 172.21.15.37:0/4278573299 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fab880858c0 con 0x7fab900b6c20 2024-04-25T08:55:34.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.511+0000 7fab9485e640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fab7c066f90 0x7fab7c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:34.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.511+0000 7fab9485e640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fab7c066f90 0x7fab7c069450 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fab800aa000 tx=0x7fab80001ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:34.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.515+0000 7fab8d7fa640 1 -- 172.21.15.37:0/4278573299 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fab900b6900 con 0x7fab900b6c20 2024-04-25T08:55:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:34.699+0000 7fab96ac2640 1 -- 172.21.15.37:0/4278573299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fab9004efa0 con 0x7fab900b6c20 2024-04-25T08:55:35.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.379+0000 7fab8d7fa640 1 -- 172.21.15.37:0/4278573299 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fab88022030 con 0x7fab900b6c20 2024-04-25T08:55:35.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 -- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fab7c066f90 msgr2=0x7fab7c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:35.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fab7c066f90 0x7fab7c069450 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fab800aa000 tx=0x7fab80001ac0 comp rx=0 tx=0).stop 2024-04-25T08:55:35.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 -- 172.21.15.37:0/4278573299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab900b6c20 msgr2=0x7fab90155de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:35.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab900b6c20 0x7fab90155de0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fab900679a0 tx=0x7fab8800b480 comp rx=0 tx=0).stop 2024-04-25T08:55:35.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 -- 172.21.15.37:0/4278573299 shutdown_connections 2024-04-25T08:55:35.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fab7c066f90 0x7fab7c069450 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:35.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab900b6c20 0x7fab90155de0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:35.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fab900c6e00 0x7fab900b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:35.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 --2- 172.21.15.37:0/4278573299 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fab78003a30 0x7fab90150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:35.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 -- 172.21.15.37:0/4278573299 >> 172.21.15.37:0/4278573299 conn(0x7fab900b30b0 msgr2=0x7fab9006fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:35.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 -- 172.21.15.37:0/4278573299 shutdown_connections 2024-04-25T08:55:35.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.383+0000 7fab96ac2640 1 -- 172.21.15.37:0/4278573299 wait complete. 2024-04-25T08:55:35.396 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:55:35.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.582+0000 7f800f498640 1 Processor -- start 2024-04-25T08:55:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.582+0000 7f800f498640 1 -- start start 2024-04-25T08:55:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.582+0000 7f800f498640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f80080b36d0 0x7f80080b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.582+0000 7f800f498640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80080cd000 0x7f80080b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.582+0000 7f800f498640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80080b3bd0 con 0x7f8008159f20 2024-04-25T08:55:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.582+0000 7f800f498640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80080b3eb0 con 0x7f80080cd000 2024-04-25T08:55:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.582+0000 7f800f498640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80080b4190 con 0x7f80080b36d0 2024-04-25T08:55:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.586+0000 7f800da35640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80080cd000 0x7f80080b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:35.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.586+0000 7f800da35640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80080cd000 0x7f80080b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39168/0 (socket says 172.21.15.37:39168) 2024-04-25T08:55:35.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.586+0000 7f800da35640 1 -- 172.21.15.37:0/2090840578 learned_addr learned my addr 172.21.15.37:0/2090840578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:35.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.586+0000 7f800ca33640 1 --2- 172.21.15.37:0/2090840578 >> v2:172.21.15.203:3300/0 conn(0x7f80080b36d0 0x7f80080b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:35.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.586+0000 7f800da35640 1 -- 172.21.15.37:0/2090840578 >> v2:172.21.15.203:3300/0 conn(0x7f80080b36d0 msgr2=0x7f80080b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:35.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.586+0000 7f800da35640 1 --2- 172.21.15.37:0/2090840578 >> v2:172.21.15.203:3300/0 conn(0x7f80080b36d0 0x7f80080b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:35.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.586+0000 7f800da35640 1 -- 172.21.15.37:0/2090840578 >> v1:172.21.15.203:6789/0 conn(0x7f8008159f20 legacy=0x7f800806d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:35.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.586+0000 7f800da35640 1 -- 172.21.15.37:0/2090840578 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800806cf80 con 0x7f80080cd000 2024-04-25T08:55:35.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800da35640 1 --2- 172.21.15.37:0/2090840578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80080cd000 0x7f80080b26d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f80080b1f20 tx=0x7f800001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e134278ac9592e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:35.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f7ffbfff640 1 -- 172.21.15.37:0/2090840578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000a57e0 con 0x7f80080cd000 2024-04-25T08:55:35.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f7ffbfff640 1 -- 172.21.15.37:0/2090840578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80000a5980 con 0x7f80080cd000 2024-04-25T08:55:35.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f7ffbfff640 1 -- 172.21.15.37:0/2090840578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000a5d50 con 0x7f80080cd000 2024-04-25T08:55:35.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 -- 172.21.15.37:0/2090840578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80080cd000 msgr2=0x7f80080b26d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:35.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 --2- 172.21.15.37:0/2090840578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80080cd000 0x7f80080b26d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f80080b1f20 tx=0x7f800001c980 comp rx=0 tx=0).stop 2024-04-25T08:55:35.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 -- 172.21.15.37:0/2090840578 shutdown_connections 2024-04-25T08:55:35.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 --2- 172.21.15.37:0/2090840578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80080cd000 0x7f80080b26d0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:35.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 --2- 172.21.15.37:0/2090840578 >> v2:172.21.15.203:3300/0 conn(0x7f80080b36d0 0x7f80080b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:35.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 -- 172.21.15.37:0/2090840578 >> 172.21.15.37:0/2090840578 conn(0x7f800806ae60 msgr2=0x7f800806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:35.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 -- 172.21.15.37:0/2090840578 shutdown_connections 2024-04-25T08:55:35.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 -- 172.21.15.37:0/2090840578 wait complete. 2024-04-25T08:55:35.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 Processor -- start 2024-04-25T08:55:35.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 -- start start 2024-04-25T08:55:35.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80080b36d0 0x7f80080b0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:35.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80080cd000 0x7f80080c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:35.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8008159f20 0x7f80081532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:35.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8008166f50 con 0x7f8008159f20 2024-04-25T08:55:35.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8008166dd0 con 0x7f80080b36d0 2024-04-25T08:55:35.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800f498640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80081670d0 con 0x7f80080cd000 2024-04-25T08:55:35.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800da35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8008159f20 0x7f80081532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:35.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800da35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8008159f20 0x7f80081532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39180/0 (socket says 172.21.15.37:39180) 2024-04-25T08:55:35.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.590+0000 7f800da35640 1 -- 172.21.15.37:0/138528954 learned_addr learned my addr 172.21.15.37:0/138528954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:35.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800d234640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80080b36d0 0x7f80080b0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:35.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800ca33640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80080cd000 0x7f80080c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:35.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800da35640 1 -- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80080cd000 msgr2=0x7f80080c9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:35.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800da35640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80080cd000 0x7f80080c9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:35.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800da35640 1 -- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80080b36d0 msgr2=0x7f80080b0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:35.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800da35640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80080b36d0 0x7f80080b0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:35.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800da35640 1 -- 172.21.15.37:0/138528954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80080b0d70 con 0x7f8008159f20 2024-04-25T08:55:35.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800da35640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8008159f20 0x7f80081532a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f800001a5b0 tx=0x7f8000019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:35.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f7ff9ffb640 1 -- 172.21.15.37:0/138528954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000b8470 con 0x7f8008159f20 2024-04-25T08:55:35.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f7ff9ffb640 1 -- 172.21.15.37:0/138528954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80000b8650 con 0x7f8008159f20 2024-04-25T08:55:35.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800f498640 1 -- 172.21.15.37:0/138528954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80080c7c90 con 0x7f8008159f20 2024-04-25T08:55:35.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f7ff9ffb640 1 -- 172.21.15.37:0/138528954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80000b8a40 con 0x7f8008159f20 2024-04-25T08:55:35.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800f498640 1 -- 172.21.15.37:0/138528954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80080c8530 con 0x7f8008159f20 2024-04-25T08:55:35.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800f498640 1 -- 172.21.15.37:0/138528954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8008067c30 con 0x7f8008159f20 2024-04-25T08:55:35.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f7ff9ffb640 1 -- 172.21.15.37:0/138528954 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80000c0020 con 0x7f8008159f20 2024-04-25T08:55:35.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f7ff9ffb640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7fec066f60 0x7f7fec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:35.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f7ff9ffb640 1 -- 172.21.15.37:0/138528954 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f8000120150 con 0x7f8008159f20 2024-04-25T08:55:35.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800d234640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7fec066f60 0x7f7fec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:35.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.594+0000 7f800d234640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7fec066f60 0x7f7fec069420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8004004e30 tx=0x7f8004009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:35.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.598+0000 7f7ff9ffb640 1 -- 172.21.15.37:0/138528954 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f80080b0d70 con 0x7f8008159f20 2024-04-25T08:55:35.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:35.758+0000 7f800f498640 1 -- 172.21.15.37:0/138528954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f800806e150 con 0x7f8008159f20 2024-04-25T08:55:36.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.394+0000 7f7ff9ffb640 1 -- 172.21.15.37:0/138528954 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v59) v1 ==== 181+0+0 (secure 0 0 0) 0x7f80000165a0 con 0x7f8008159f20 2024-04-25T08:55:36.386 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:55:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.398+0000 7f800f498640 1 -- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7fec066f60 msgr2=0x7f7fec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.398+0000 7f800f498640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7fec066f60 0x7f7fec069420 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8004004e30 tx=0x7f8004009040 comp rx=0 tx=0).stop 2024-04-25T08:55:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.398+0000 7f800f498640 1 -- 172.21.15.37:0/138528954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8008159f20 msgr2=0x7f80081532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.398+0000 7f800f498640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8008159f20 0x7f80081532a0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f800001a5b0 tx=0x7f8000019720 comp rx=0 tx=0).stop 2024-04-25T08:55:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.398+0000 7f800f498640 1 -- 172.21.15.37:0/138528954 shutdown_connections 2024-04-25T08:55:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.398+0000 7f800f498640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7fec066f60 0x7f7fec069420 secure :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8004004e30 tx=0x7f8004009040 comp rx=0 tx=0).stop 2024-04-25T08:55:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.398+0000 7f800f498640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8008159f20 0x7f80081532a0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.398+0000 7f800f498640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80080cd000 0x7f80080c9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.398+0000 7f800f498640 1 --2- 172.21.15.37:0/138528954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80080b36d0 0x7f80080b0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:36.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.398+0000 7f800f498640 1 -- 172.21.15.37:0/138528954 >> 172.21.15.37:0/138528954 conn(0x7f800806ae60 msgr2=0x7f80080c4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:36.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.402+0000 7f800f498640 1 -- 172.21.15.37:0/138528954 shutdown_connections 2024-04-25T08:55:36.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.402+0000 7f800f498640 1 -- 172.21.15.37:0/138528954 wait complete. 2024-04-25T08:55:36.414 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:55:36.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d1ea59640 1 Processor -- start 2024-04-25T08:55:36.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d1ea59640 1 -- start start 2024-04-25T08:55:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d1ea59640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8d180cf980 0x7f8d180ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d1ea59640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d180b28c0 con 0x7f8d180adc70 2024-04-25T08:55:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d1ea59640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d1815bb20 con 0x7f8d180cf510 2024-04-25T08:55:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d1ea59640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d180b2b60 con 0x7f8d180cf980 2024-04-25T08:55:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d1cff6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8d180cf980 0x7f8d180ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:36.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d17fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8d180adc70 0x7f8d1806e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:43364/0 (socket says 172.21.15.37:43364) 2024-04-25T08:55:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d17fff640 1 -- 172.21.15.37:0/1448254286 learned_addr learned my addr 172.21.15.37:0/1448254286 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d1cff6640 1 -- 172.21.15.37:0/1448254286 >> v1:172.21.15.203:6789/0 conn(0x7f8d180cf510 legacy=0x7f8d180ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d1cff6640 1 -- 172.21.15.37:0/1448254286 >> v1:172.21.15.37:6789/0 conn(0x7f8d180adc70 legacy=0x7f8d1806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d1cff6640 1 -- 172.21.15.37:0/1448254286 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d180b2e40 con 0x7f8d180cf980 2024-04-25T08:55:36.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.554+0000 7f8d16ffd640 1 -- 172.21.15.37:0/1448254286 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011857999 0 0) 0x7f8d180b28c0 con 0x7f8d180adc70 2024-04-25T08:55:36.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1cff6640 1 --2- 172.21.15.37:0/1448254286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d180cf980 0x7f8d180ac880 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009d90 tx=0x7f8d1001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1b11f51a04405c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:36.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d16ffd640 1 -- 172.21.15.37:0/1448254286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d100b2070 con 0x7f8d180cf980 2024-04-25T08:55:36.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d16ffd640 1 -- 172.21.15.37:0/1448254286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d100a58e0 con 0x7f8d180cf980 2024-04-25T08:55:36.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d16ffd640 1 -- 172.21.15.37:0/1448254286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d100a5cb0 con 0x7f8d180cf980 2024-04-25T08:55:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 -- 172.21.15.37:0/1448254286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d180cf980 msgr2=0x7f8d180ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 --2- 172.21.15.37:0/1448254286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d180cf980 0x7f8d180ac880 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8d10009d90 tx=0x7f8d1001c930 comp rx=0 tx=0).stop 2024-04-25T08:55:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 -- 172.21.15.37:0/1448254286 shutdown_connections 2024-04-25T08:55:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 --2- 172.21.15.37:0/1448254286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d180cf980 0x7f8d180ac880 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 -- 172.21.15.37:0/1448254286 >> 172.21.15.37:0/1448254286 conn(0x7f8d1806ae60 msgr2=0x7f8d1806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 -- 172.21.15.37:0/1448254286 shutdown_connections 2024-04-25T08:55:36.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 -- 172.21.15.37:0/1448254286 wait complete. 2024-04-25T08:55:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 Processor -- start 2024-04-25T08:55:36.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 -- start start 2024-04-25T08:55:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d180adc70 0x7f8d18155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d180cf510 0x7f8d18155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d180cf980 0x7f8d181669d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d180b3c00 con 0x7f8d180cf510 2024-04-25T08:55:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d180cfdf0 con 0x7f8d180cf980 2024-04-25T08:55:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.558+0000 7f8d1ea59640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d180c6550 con 0x7f8d180adc70 2024-04-25T08:55:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d180cf510 0x7f8d18155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d180cf510 0x7f8d18155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39202/0 (socket says 172.21.15.37:39202) 2024-04-25T08:55:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d177fe640 1 -- 172.21.15.37:0/4041901727 learned_addr learned my addr 172.21.15.37:0/4041901727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d1cff6640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d180cf980 0x7f8d181669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:36.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d177fe640 1 -- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d180adc70 msgr2=0x7f8d18155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d177fe640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d180adc70 0x7f8d18155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d177fe640 1 -- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d180cf980 msgr2=0x7f8d181669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d177fe640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d180cf980 0x7f8d181669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d177fe640 1 -- 172.21.15.37:0/4041901727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d180c7120 con 0x7f8d180cf510 2024-04-25T08:55:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d1cff6640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d180cf980 0x7f8d181669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d177fe640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d180cf510 0x7f8d18155dd0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800c450 tx=0x7f8d0800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d14ff9640 1 -- 172.21.15.37:0/4041901727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08012830 con 0x7f8d180cf510 2024-04-25T08:55:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d14ff9640 1 -- 172.21.15.37:0/4041901727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d08017070 con 0x7f8d180cf510 2024-04-25T08:55:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d1ea59640 1 -- 172.21.15.37:0/4041901727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d180c67f0 con 0x7f8d180cf510 2024-04-25T08:55:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d14ff9640 1 -- 172.21.15.37:0/4041901727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d08012c30 con 0x7f8d180cf510 2024-04-25T08:55:36.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d1ea59640 1 -- 172.21.15.37:0/4041901727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d180c6db0 con 0x7f8d180cf510 2024-04-25T08:55:36.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d1ea59640 1 -- 172.21.15.37:0/4041901727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d18067c30 con 0x7f8d180cf510 2024-04-25T08:55:36.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d14ff9640 1 -- 172.21.15.37:0/4041901727 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d0801c460 con 0x7f8d180cf510 2024-04-25T08:55:36.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d14ff9640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d04066f60 0x7f8d04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:36.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d14ff9640 1 -- 172.21.15.37:0/4041901727 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5807+0+0 (secure 0 0 0) 0x7f8d080858a0 con 0x7f8d180cf510 2024-04-25T08:55:36.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.562+0000 7f8d17fff640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d04066f60 0x7f8d04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:36.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.566+0000 7f8d17fff640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d04066f60 0x7f8d04069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c007fd0 tx=0x7f8d0c007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.566+0000 7f8d14ff9640 1 -- 172.21.15.37:0/4041901727 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8d080569c0 con 0x7f8d180cf510 2024-04-25T08:55:36.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:36.726+0000 7f8d1ea59640 1 -- 172.21.15.37:0/4041901727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8d1806d760 con 0x7f8d180cf510 2024-04-25T08:55:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.422+0000 7f8d14ff9640 1 -- 172.21.15.37:0/4041901727 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v60) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8d080562c0 con 0x7f8d180cf510 2024-04-25T08:55:37.415 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T08:55:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 -- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d04066f60 msgr2=0x7f8d04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d04066f60 0x7f8d04069420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c007fd0 tx=0x7f8d0c007960 comp rx=0 tx=0).stop 2024-04-25T08:55:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 -- 172.21.15.37:0/4041901727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d180cf510 msgr2=0x7f8d18155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d180cf510 0x7f8d18155dd0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f8d0800c450 tx=0x7f8d0800c910 comp rx=0 tx=0).stop 2024-04-25T08:55:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 -- 172.21.15.37:0/4041901727 shutdown_connections 2024-04-25T08:55:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d04066f60 0x7f8d04069420 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d180cf980 0x7f8d181669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d180cf510 0x7f8d18155dd0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 --2- 172.21.15.37:0/4041901727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d180adc70 0x7f8d18155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 -- 172.21.15.37:0/4041901727 >> 172.21.15.37:0/4041901727 conn(0x7f8d1806ae60 msgr2=0x7f8d180ce0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 -- 172.21.15.37:0/4041901727 shutdown_connections 2024-04-25T08:55:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.430+0000 7f8d1ea59640 1 -- 172.21.15.37:0/4041901727 wait complete. 2024-04-25T08:55:37.444 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:55:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa951e28640 1 Processor -- start 2024-04-25T08:55:37.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa951e28640 1 -- start start 2024-04-25T08:55:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa951e28640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa94c0b3cf0 0x7fa94c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa951e28640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0cfad0 0x7fa94c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa951e28640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c0cbf10 con 0x7fa94c15e4f0 2024-04-25T08:55:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa951e28640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94c0cff40 con 0x7fa94c0b3cf0 2024-04-25T08:55:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa951e28640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0cc1f0 con 0x7fa94c0cfad0 2024-04-25T08:55:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa94c15e4f0 0x7fa94c161370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:43380/0 (socket says 172.21.15.37:43380) 2024-04-25T08:55:37.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94bfff640 1 -- 172.21.15.37:0/758277143 learned_addr learned my addr 172.21.15.37:0/758277143 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:37.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94b7fe640 1 --2- 172.21.15.37:0/758277143 >> v2:172.21.15.203:3300/0 conn(0x7fa94c0b3cf0 0x7fa94c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94affd640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0cfad0 0x7fa94c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580458944 0 0) 0x7fa94c0cbf10 con 0x7fa94c15e4f0 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa938003530 con 0x7fa94c15e4f0 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 629245245 0 0) 0x7fa938003530 con 0x7fa94c15e4f0 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0cfad0 msgr2=0x7fa94c15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0cfad0 0x7fa94c15dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 >> v2:172.21.15.203:3300/0 conn(0x7fa94c0b3cf0 msgr2=0x7fa94c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 --2- 172.21.15.37:0/758277143 >> v2:172.21.15.203:3300/0 conn(0x7fa94c0b3cf0 0x7fa94c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0cc4d0 con 0x7fa94c15e4f0 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fa944001a00 con 0x7fa94c15e4f0 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 >> v1:172.21.15.37:6789/0 conn(0x7fa94c15e4f0 legacy=0x7fa94c161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa938003a30 0x7fa938003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa938004810 0x7fa938004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0cfad0 0x7fa9380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0cff40 con 0x7fa938003a30 2024-04-25T08:55:37.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0cc1f0 con 0x7fa938004810 2024-04-25T08:55:37.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93800ba20 con 0x7fa94c0cfad0 2024-04-25T08:55:37.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94b7fe640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa938003a30 0x7fa938003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94bfff640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0cfad0 0x7fa9380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94affd640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa938004810 0x7fa938004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94b7fe640 1 -- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0cfad0 msgr2=0x7fa9380051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:37.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94b7fe640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0cfad0 0x7fa9380051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94b7fe640 1 -- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa938004810 msgr2=0x7fa938004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:37.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94b7fe640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa938004810 0x7fa938004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94b7fe640 1 -- 172.21.15.37:0/758277143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa938003530 con 0x7fa938003a30 2024-04-25T08:55:37.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.582+0000 7fa94bfff640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c0cfad0 0x7fa9380051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa94b7fe640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa938003a30 0x7fa938003e50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009b10 tx=0x7fa93c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662f2b8d2ced6ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0096b0 con 0x7fa938003a30 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c0a8410 con 0x7fa938003a30 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa94a7fc640 1 -- 172.21.15.37:0/758277143 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0a8800 con 0x7fa938003a30 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa951e28640 1 -- 172.21.15.37:0/758277143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa938003a30 msgr2=0x7fa938003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa951e28640 1 --2- 172.21.15.37:0/758277143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa938003a30 0x7fa938003e50 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009b10 tx=0x7fa93c01cab0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa94bfff640 1 -- 172.21.15.37:0/758277143 reap_dead start 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa951e28640 1 -- 172.21.15.37:0/758277143 shutdown_connections 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa951e28640 1 -- 172.21.15.37:0/758277143 >> 172.21.15.37:0/758277143 conn(0x7fa94c06ae60 msgr2=0x7fa94c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa951e28640 1 -- 172.21.15.37:0/758277143 shutdown_connections 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa951e28640 1 -- 172.21.15.37:0/758277143 wait complete. 2024-04-25T08:55:37.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa951e28640 1 Processor -- start 2024-04-25T08:55:37.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.586+0000 7fa951e28640 1 -- start start 2024-04-25T08:55:37.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa951e28640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa938003a30 0x7fa94c0c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa951e28640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c162380 0x7fa94c1627e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa951e28640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c1fa830 0x7fa94c1facb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa951e28640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0b4250 con 0x7fa94c1fa830 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa951e28640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0b40d0 con 0x7fa938003a30 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa951e28640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c15e900 con 0x7fa94c162380 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c1fa830 0x7fa94c1facb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c1fa830 0x7fa94c1facb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39214/0 (socket says 172.21.15.37:39214) 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94bfff640 1 -- 172.21.15.37:0/2467880454 learned_addr learned my addr 172.21.15.37:0/2467880454 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94affd640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c162380 0x7fa94c1627e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94b7fe640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa938003a30 0x7fa94c0c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94bfff640 1 -- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c162380 msgr2=0x7fa94c1627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94bfff640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c162380 0x7fa94c1627e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94bfff640 1 -- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa938003a30 msgr2=0x7fa94c0c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94bfff640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa938003a30 0x7fa94c0c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94bfff640 1 -- 172.21.15.37:0/2467880454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0c5bf0 con 0x7fa94c1fa830 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94bfff640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c1fa830 0x7fa94c1facb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0b2fc0 tx=0x7fa94400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa943fff640 1 -- 172.21.15.37:0/2467880454 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944004d70 con 0x7fa94c1fa830 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa943fff640 1 -- 172.21.15.37:0/2467880454 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94400bb80 con 0x7fa94c1fa830 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa943fff640 1 -- 172.21.15.37:0/2467880454 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944007070 con 0x7fa94c1fa830 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa951e28640 1 -- 172.21.15.37:0/2467880454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c1499b0 con 0x7fa94c1fa830 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa951e28640 1 -- 172.21.15.37:0/2467880454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c149f70 con 0x7fa94c1fa830 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa951e28640 1 -- 172.21.15.37:0/2467880454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c04efa0 con 0x7fa94c1fa830 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa943fff640 1 -- 172.21.15.37:0/2467880454 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa94401b020 con 0x7fa94c1fa830 2024-04-25T08:55:37.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa943fff640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa92c066f90 0x7fa92c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa943fff640 1 -- 172.21.15.37:0/2467880454 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa944084f00 con 0x7fa94c1fa830 2024-04-25T08:55:37.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.590+0000 7fa94b7fe640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa92c066f90 0x7fa92c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.594+0000 7fa94b7fe640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa92c066f90 0x7fa92c069450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0af000 tx=0x7fa93c0b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:37.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.594+0000 7fa943fff640 1 -- 172.21.15.37:0/2467880454 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa944056080 con 0x7fa94c1fa830 2024-04-25T08:55:37.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.758+0000 7fa951e28640 1 -- 172.21.15.37:0/2467880454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa94c0b3060 con 0x7fa94c1fa830 2024-04-25T08:55:37.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.762+0000 7fa943fff640 1 -- 172.21.15.37:0/2467880454 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa944055980 con 0x7fa94c1fa830 2024-04-25T08:55:37.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 -- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa92c066f90 msgr2=0x7fa92c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa92c066f90 0x7fa92c069450 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0af000 tx=0x7fa93c0b2040 comp rx=0 tx=0).stop 2024-04-25T08:55:37.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 -- 172.21.15.37:0/2467880454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c1fa830 msgr2=0x7fa94c1facb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c1fa830 0x7fa94c1facb0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa94c0b2fc0 tx=0x7fa94400b4a0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 -- 172.21.15.37:0/2467880454 shutdown_connections 2024-04-25T08:55:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa92c066f90 0x7fa92c069450 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa94c1fa830 0x7fa94c1facb0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa94c162380 0x7fa94c1627e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 --2- 172.21.15.37:0/2467880454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa938003a30 0x7fa94c0c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 -- 172.21.15.37:0/2467880454 >> 172.21.15.37:0/2467880454 conn(0x7fa94c06ae60 msgr2=0x7fa94c06e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 -- 172.21.15.37:0/2467880454 shutdown_connections 2024-04-25T08:55:37.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.766+0000 7fa951e28640 1 -- 172.21.15.37:0/2467880454 wait complete. 2024-04-25T08:55:37.782 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-04-25T08:55:37.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd07e0cf640 1 Processor -- start 2024-04-25T08:55:37.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd07e0cf640 1 -- start start 2024-04-25T08:55:37.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd07e0cf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd0780b2e20 0x7fd07806d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd07e0cf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0780b3200 0x7fd07806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd07e0cf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07815ae30 con 0x7fd078159ea0 2024-04-25T08:55:37.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd07e0cf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd07806db10 con 0x7fd0780b2e20 2024-04-25T08:55:37.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd07e0cf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780b2990 con 0x7fd0780b3200 2024-04-25T08:55:37.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd076ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0780b3200 0x7fd07806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd0777fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd0780b2e20 0x7fd07806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd076ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0780b3200 0x7fd07806d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:45590/0 (socket says 172.21.15.37:45590) 2024-04-25T08:55:37.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd076ffd640 1 -- 172.21.15.37:0/3778656915 learned_addr learned my addr 172.21.15.37:0/3778656915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:37.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd076ffd640 1 -- 172.21.15.37:0/3778656915 >> v2:172.21.15.203:3300/0 conn(0x7fd0780b2e20 msgr2=0x7fd07806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:37.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd076ffd640 1 --2- 172.21.15.37:0/3778656915 >> v2:172.21.15.203:3300/0 conn(0x7fd0780b2e20 0x7fd07806d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd076ffd640 1 -- 172.21.15.37:0/3778656915 >> v1:172.21.15.203:6789/0 conn(0x7fd078159ea0 legacy=0x7fd07815a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.914+0000 7fd076ffd640 1 -- 172.21.15.37:0/3778656915 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07815b110 con 0x7fd0780b3200 2024-04-25T08:55:37.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd076ffd640 1 --2- 172.21.15.37:0/3778656915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0780b3200 0x7fd07806d5d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd06800bb10 tx=0x7fd06801ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f74a4d9e814ef96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:37.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd0767fc640 1 -- 172.21.15.37:0/3778656915 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0680b4070 con 0x7fd0780b3200 2024-04-25T08:55:37.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd0767fc640 1 -- 172.21.15.37:0/3778656915 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0680a7b10 con 0x7fd0780b3200 2024-04-25T08:55:37.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd0767fc640 1 -- 172.21.15.37:0/3778656915 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0680af440 con 0x7fd0780b3200 2024-04-25T08:55:37.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 -- 172.21.15.37:0/3778656915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0780b3200 msgr2=0x7fd07806d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 --2- 172.21.15.37:0/3778656915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0780b3200 0x7fd07806d5d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd06800bb10 tx=0x7fd06801ec30 comp rx=0 tx=0).stop 2024-04-25T08:55:37.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 -- 172.21.15.37:0/3778656915 shutdown_connections 2024-04-25T08:55:37.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 --2- 172.21.15.37:0/3778656915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0780b3200 0x7fd07806d5d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 --2- 172.21.15.37:0/3778656915 >> v2:172.21.15.203:3300/0 conn(0x7fd0780b2e20 0x7fd07806d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 -- 172.21.15.37:0/3778656915 >> 172.21.15.37:0/3778656915 conn(0x7fd0780b11a0 msgr2=0x7fd0780b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:37.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 -- 172.21.15.37:0/3778656915 shutdown_connections 2024-04-25T08:55:37.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 -- 172.21.15.37:0/3778656915 wait complete. 2024-04-25T08:55:37.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 Processor -- start 2024-04-25T08:55:37.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 -- start start 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0780b2e20 0x7fd078151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0780b3200 0x7fd07815ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd078159ea0 0x7fd07813cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0780721c0 con 0x7fd0780b2e20 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd078072040 con 0x7fd0780b3200 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd07e0cf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd07815e170 con 0x7fd078159ea0 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.918+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0780b2e20 0x7fd078151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd0777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0780b2e20 0x7fd078151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39220/0 (socket says 172.21.15.37:39220) 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd0777fe640 1 -- 172.21.15.37:0/931638607 learned_addr learned my addr 172.21.15.37:0/931638607 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd077fff640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd078159ea0 0x7fd07813cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd076ffd640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0780b3200 0x7fd07815ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd0777fe640 1 -- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd078159ea0 msgr2=0x7fd07813cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd0777fe640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd078159ea0 0x7fd07813cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd0777fe640 1 -- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0780b3200 msgr2=0x7fd07815ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:37.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd0777fe640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0780b3200 0x7fd07815ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:37.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd0777fe640 1 -- 172.21.15.37:0/931638607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd07815cc00 con 0x7fd0780b2e20 2024-04-25T08:55:37.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd0777fe640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0780b2e20 0x7fd078151ab0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00e930 tx=0x7fd06c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:37.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd05ffff640 1 -- 172.21.15.37:0/931638607 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c017070 con 0x7fd0780b2e20 2024-04-25T08:55:37.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd05ffff640 1 -- 172.21.15.37:0/931638607 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06c00ca00 con 0x7fd0780b2e20 2024-04-25T08:55:37.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd07e0cf640 1 -- 172.21.15.37:0/931638607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07815cee0 con 0x7fd0780b2e20 2024-04-25T08:55:37.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd05ffff640 1 -- 172.21.15.37:0/931638607 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06c00cdb0 con 0x7fd0780b2e20 2024-04-25T08:55:37.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd07e0cf640 1 -- 172.21.15.37:0/931638607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0781546d0 con 0x7fd0780b2e20 2024-04-25T08:55:37.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd07e0cf640 1 -- 172.21.15.37:0/931638607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07815d0b0 con 0x7fd0780b2e20 2024-04-25T08:55:37.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd05ffff640 1 -- 172.21.15.37:0/931638607 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd06c01a070 con 0x7fd0780b2e20 2024-04-25T08:55:37.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd05ffff640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd064066f60 0x7fd064069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:37.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd05ffff640 1 -- 172.21.15.37:0/931638607 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd06c085290 con 0x7fd0780b2e20 2024-04-25T08:55:37.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.922+0000 7fd076ffd640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd064066f60 0x7fd064069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:37.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.926+0000 7fd076ffd640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd064066f60 0x7fd064069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd0680b1000 tx=0x7fd0680a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:37.926+0000 7fd05ffff640 1 -- 172.21.15.37:0/931638607 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd06c055e00 con 0x7fd0780b2e20 2024-04-25T08:55:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.150+0000 7fd07e0cf640 1 -- 172.21.15.37:0/931638607 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7fd07806e6b0 con 0x7fd0780b2e20 2024-04-25T08:55:38.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.426+0000 7fd05ffff640 1 -- 172.21.15.37:0/931638607 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v185) v1 ==== 149+0+0 (secure 0 0 0) 0x7fd06c059ea0 con 0x7fd0780b2e20 2024-04-25T08:55:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 -- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd064066f60 msgr2=0x7fd064069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd064066f60 0x7fd064069420 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd0680b1000 tx=0x7fd0680a9070 comp rx=0 tx=0).stop 2024-04-25T08:55:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 -- 172.21.15.37:0/931638607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0780b2e20 msgr2=0x7fd078151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0780b2e20 0x7fd078151ab0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00e930 tx=0x7fd06c00edf0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 -- 172.21.15.37:0/931638607 shutdown_connections 2024-04-25T08:55:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd064066f60 0x7fd064069420 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd078159ea0 0x7fd07813cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0780b3200 0x7fd07815ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 --2- 172.21.15.37:0/931638607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0780b2e20 0x7fd078151ab0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 -- 172.21.15.37:0/931638607 >> 172.21.15.37:0/931638607 conn(0x7fd0780b11a0 msgr2=0x7fd0780b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 -- 172.21.15.37:0/931638607 shutdown_connections 2024-04-25T08:55:38.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.430+0000 7fd07e0cf640 1 -- 172.21.15.37:0/931638607 wait complete. 2024-04-25T08:55:38.445 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-04-25T08:55:38.446 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T08:55:38.446 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:55:38.446 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.586+0000 7fc3d84f3640 1 Processor -- start 2024-04-25T08:55:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.586+0000 7fc3d84f3640 1 -- start start 2024-04-25T08:55:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d84f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d009a180 0x7fc3d015d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d84f3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d00b0e10 con 0x7fc3d006f4a0 2024-04-25T08:55:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d84f3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d00d0180 con 0x7fc3d00b0a30 2024-04-25T08:55:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d84f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d00d0d90 con 0x7fc3d009a180 2024-04-25T08:55:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d5a8e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc3d006f4a0 0x7fc3d0099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:43390/0 (socket says 172.21.15.37:43390) 2024-04-25T08:55:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d5a8e640 1 -- 172.21.15.37:0/592402916 learned_addr learned my addr 172.21.15.37:0/592402916 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d6a90640 1 --2- 172.21.15.37:0/592402916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d009a180 0x7fc3d015d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913233033 0 0) 0x7fc3d00b0e10 con 0x7fc3d006f4a0 2024-04-25T08:55:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3bc003530 con 0x7fc3d006f4a0 2024-04-25T08:55:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1861317114 0 0) 0x7fc3bc003530 con 0x7fc3d006f4a0 2024-04-25T08:55:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d009a180 msgr2=0x7fc3d015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 --2- 172.21.15.37:0/592402916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d009a180 0x7fc3d015d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 >> v1:172.21.15.203:6789/0 conn(0x7fc3d00b0a30 legacy=0x7fc3d00cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d00d0460 con 0x7fc3d006f4a0 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fc3c0001680 con 0x7fc3d006f4a0 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 >> v1:172.21.15.37:6789/0 conn(0x7fc3d006f4a0 legacy=0x7fc3d0099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 --2- 172.21.15.37:0/592402916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d009a180 0x7fc3bc003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 --2- 172.21.15.37:0/592402916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3bc003fe0 0x7fc3bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 --2- 172.21.15.37:0/592402916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 0x7fc3bc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d00d0d90 con 0x7fc3bc004d50 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc00bb30 con 0x7fc3bc003fe0 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc00bd30 con 0x7fc3d009a180 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d6a90640 1 --2- 172.21.15.37:0/592402916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 0x7fc3bc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d5a8e640 1 --2- 172.21.15.37:0/592402916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3bc003fe0 0x7fc3bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d6a90640 1 -- 172.21.15.37:0/592402916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d009a180 msgr2=0x7fc3bc003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d6a90640 1 --2- 172.21.15.37:0/592402916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d009a180 0x7fc3bc003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d6a90640 1 -- 172.21.15.37:0/592402916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3bc003fe0 msgr2=0x7fc3bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d6a90640 1 --2- 172.21.15.37:0/592402916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3bc003fe0 0x7fc3bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.590+0000 7fc3d6a90640 1 -- 172.21.15.37:0/592402916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc003530 con 0x7fc3bc004d50 2024-04-25T08:55:38.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d6a90640 1 --2- 172.21.15.37:0/592402916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 0x7fc3bc0051d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009fc0 tx=0x7fc3c801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143f3d8a36c5fff5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c80b2070 con 0x7fc3bc004d50 2024-04-25T08:55:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c80a5cd0 con 0x7fc3bc004d50 2024-04-25T08:55:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d528d640 1 -- 172.21.15.37:0/592402916 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8009c10 con 0x7fc3bc004d50 2024-04-25T08:55:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 -- 172.21.15.37:0/592402916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 msgr2=0x7fc3bc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 --2- 172.21.15.37:0/592402916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 0x7fc3bc0051d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009fc0 tx=0x7fc3c801ccc0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d6a90640 1 -- 172.21.15.37:0/592402916 reap_dead start 2024-04-25T08:55:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 -- 172.21.15.37:0/592402916 shutdown_connections 2024-04-25T08:55:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 -- 172.21.15.37:0/592402916 >> 172.21.15.37:0/592402916 conn(0x7fc3d00b30b0 msgr2=0x7fc3d00b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 -- 172.21.15.37:0/592402916 shutdown_connections 2024-04-25T08:55:38.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 -- 172.21.15.37:0/592402916 wait complete. 2024-04-25T08:55:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 Processor -- start 2024-04-25T08:55:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 -- start start 2024-04-25T08:55:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 0x7fc3d00b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d00c5d10 0x7fc3d00c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d01544f0 0x7fc3d0154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d00ccc20 con 0x7fc3bc004d50 2024-04-25T08:55:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d00d0a20 con 0x7fc3d00c5d10 2024-04-25T08:55:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d84f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d00ccda0 con 0x7fc3d01544f0 2024-04-25T08:55:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d628f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 0x7fc3d00b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d628f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 0x7fc3d00b6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39248/0 (socket says 172.21.15.37:39248) 2024-04-25T08:55:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d628f640 1 -- 172.21.15.37:0/1363614954 learned_addr learned my addr 172.21.15.37:0/1363614954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d5a8e640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d00c5d10 0x7fc3d00c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d6a90640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d01544f0 0x7fc3d0154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d628f640 1 -- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d01544f0 msgr2=0x7fc3d0154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d628f640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d01544f0 0x7fc3d0154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d628f640 1 -- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d00c5d10 msgr2=0x7fc3d00c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.594+0000 7fc3d628f640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d00c5d10 0x7fc3d00c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3d628f640 1 -- 172.21.15.37:0/1363614954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d0154eb0 con 0x7fc3bc004d50 2024-04-25T08:55:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3d628f640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 0x7fc3d00b6af0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00c560 tx=0x7fc3cc00ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3c6ffd640 1 -- 172.21.15.37:0/1363614954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc0052d0 con 0x7fc3bc004d50 2024-04-25T08:55:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3c6ffd640 1 -- 172.21.15.37:0/1363614954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3cc0054b0 con 0x7fc3bc004d50 2024-04-25T08:55:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3d84f3640 1 -- 172.21.15.37:0/1363614954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d00c67d0 con 0x7fc3bc004d50 2024-04-25T08:55:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3c6ffd640 1 -- 172.21.15.37:0/1363614954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3cc012870 con 0x7fc3bc004d50 2024-04-25T08:55:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3d84f3640 1 -- 172.21.15.37:0/1363614954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d00b4230 con 0x7fc3bc004d50 2024-04-25T08:55:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3d84f3640 1 -- 172.21.15.37:0/1363614954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d004efa0 con 0x7fc3bc004d50 2024-04-25T08:55:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3c6ffd640 1 -- 172.21.15.37:0/1363614954 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc3cc012a10 con 0x7fc3bc004d50 2024-04-25T08:55:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3c6ffd640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3b4066f60 0x7fc3b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3c6ffd640 1 -- 172.21.15.37:0/1363614954 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc3cc084a80 con 0x7fc3bc004d50 2024-04-25T08:55:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3d5a8e640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3b4066f60 0x7fc3b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.598+0000 7fc3d5a8e640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3b4066f60 0x7fc3b4069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc3d009f580 tx=0x7fc3c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.602+0000 7fc3c6ffd640 1 -- 172.21.15.37:0/1363614954 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc3cc086e20 con 0x7fc3bc004d50 2024-04-25T08:55:38.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.766+0000 7fc3d84f3640 1 -- 172.21.15.37:0/1363614954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3d00b3ee0 con 0x7fc3bc004d50 2024-04-25T08:55:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.766+0000 7fc3c6ffd640 1 -- 172.21.15.37:0/1363614954 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+13257 (secure 0 0 0) 0x7fc3cc055bf0 con 0x7fc3bc004d50 2024-04-25T08:55:38.757 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:38.757 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":60,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:55:37.416247+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/3892786810":"2024-04-26T08:55:34.362403+0000","172.21.15.37:6834/3892786810":"2024-04-26T08:55:34.362403+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 -- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3b4066f60 msgr2=0x7fc3b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3b4066f60 0x7fc3b4069420 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc3d009f580 tx=0x7fc3c0009040 comp rx=0 tx=0).stop 2024-04-25T08:55:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 -- 172.21.15.37:0/1363614954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 msgr2=0x7fc3d00b6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 0x7fc3d00b6af0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc00c560 tx=0x7fc3cc00ca20 comp rx=0 tx=0).stop 2024-04-25T08:55:38.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 -- 172.21.15.37:0/1363614954 shutdown_connections 2024-04-25T08:55:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc3b4066f60 0x7fc3b4069420 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc3d01544f0 0x7fc3d0154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc3d00c5d10 0x7fc3d00c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 --2- 172.21.15.37:0/1363614954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3bc004d50 0x7fc3d00b6af0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 -- 172.21.15.37:0/1363614954 >> 172.21.15.37:0/1363614954 conn(0x7fc3d00b30b0 msgr2=0x7fc3d00737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 -- 172.21.15.37:0/1363614954 shutdown_connections 2024-04-25T08:55:38.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.770+0000 7fc3d84f3640 1 -- 172.21.15.37:0/1363614954 wait complete. 2024-04-25T08:55:38.777 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:55:38.777 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:55:38.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ec423640 1 Processor -- start 2024-04-25T08:55:38.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ec423640 1 -- start start 2024-04-25T08:55:38.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ec423640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e406c3b0 0x7f84e415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ec423640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f84e40b2c30 0x7f84e406df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ec423640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e4066c30 con 0x7f84e40af280 2024-04-25T08:55:38.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ec423640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e406daa0 con 0x7f84e40b2c30 2024-04-25T08:55:38.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ec423640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e40af660 con 0x7f84e406c3b0 2024-04-25T08:55:38.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ea9c0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f84e40b2c30 0x7f84e406df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84e99be640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e406c3b0 0x7f84e415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ea9c0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f84e40b2c30 0x7f84e406df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:57546/0 (socket says 172.21.15.37:57546) 2024-04-25T08:55:38.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ea9c0640 1 -- 172.21.15.37:0/4124762239 learned_addr learned my addr 172.21.15.37:0/4124762239 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:38.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ea9c0640 1 -- 172.21.15.37:0/4124762239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e406c3b0 msgr2=0x7f84e415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:38.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ea9c0640 1 --2- 172.21.15.37:0/4124762239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e406c3b0 0x7f84e415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ea9c0640 1 -- 172.21.15.37:0/4124762239 >> v1:172.21.15.203:6789/0 conn(0x7f84e40af280 legacy=0x7f84e406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.918+0000 7f84ea9c0640 1 -- 172.21.15.37:0/4124762239 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e415e640 con 0x7f84e40b2c30 2024-04-25T08:55:38.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84ea9c0640 1 --2- 172.21.15.37:0/4124762239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e40b2c30 0x7f84e406df30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009e50 tx=0x7f84dc01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67470285b0d2bd45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84e91bd640 1 -- 172.21.15.37:0/4124762239 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0a5700 con 0x7f84e40b2c30 2024-04-25T08:55:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84e91bd640 1 -- 172.21.15.37:0/4124762239 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84dc0a58a0 con 0x7f84e40b2c30 2024-04-25T08:55:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84e91bd640 1 -- 172.21.15.37:0/4124762239 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84dc0a5c70 con 0x7f84e40b2c30 2024-04-25T08:55:38.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84ec423640 1 -- 172.21.15.37:0/4124762239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e40b2c30 msgr2=0x7f84e406df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84ec423640 1 --2- 172.21.15.37:0/4124762239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e40b2c30 0x7f84e406df30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009e50 tx=0x7f84dc01ca30 comp rx=0 tx=0).stop 2024-04-25T08:55:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84ec423640 1 -- 172.21.15.37:0/4124762239 shutdown_connections 2024-04-25T08:55:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84ec423640 1 --2- 172.21.15.37:0/4124762239 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e40b2c30 0x7f84e406df30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84ec423640 1 --2- 172.21.15.37:0/4124762239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e406c3b0 0x7f84e415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84ec423640 1 -- 172.21.15.37:0/4124762239 >> 172.21.15.37:0/4124762239 conn(0x7f84e4071810 msgr2=0x7f84e4071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84ec423640 1 -- 172.21.15.37:0/4124762239 shutdown_connections 2024-04-25T08:55:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.922+0000 7f84ec423640 1 -- 172.21.15.37:0/4124762239 wait complete. 2024-04-25T08:55:38.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ec423640 1 Processor -- start 2024-04-25T08:55:38.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ec423640 1 -- start start 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ec423640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e406c3b0 0x7f84e414ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ec423640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e40af280 0x7f84e415ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ec423640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e40b2c30 0x7f84e4153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ec423640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e406f010 con 0x7f84e40af280 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ec423640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e406ee90 con 0x7f84e406c3b0 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ec423640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e406f190 con 0x7f84e40b2c30 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84e99be640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e40af280 0x7f84e415ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84e99be640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e40af280 0x7f84e415ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39250/0 (socket says 172.21.15.37:39250) 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84e99be640 1 -- 172.21.15.37:0/1047577958 learned_addr learned my addr 172.21.15.37:0/1047577958 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ea1bf640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e406c3b0 0x7f84e414ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ea9c0640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e40b2c30 0x7f84e4153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84e99be640 1 -- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e40b2c30 msgr2=0x7f84e4153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84e99be640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e40b2c30 0x7f84e4153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84e99be640 1 -- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e406c3b0 msgr2=0x7f84e414ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84e99be640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e406c3b0 0x7f84e414ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84e99be640 1 -- 172.21.15.37:0/1047577958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e4162000 con 0x7f84e40af280 2024-04-25T08:55:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ea9c0640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e40b2c30 0x7f84e4153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84e99be640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e40af280 0x7f84e415ff80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f84e000e870 tx=0x7f84e000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84daffd640 1 -- 172.21.15.37:0/1047577958 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e0017070 con 0x7f84e40af280 2024-04-25T08:55:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84daffd640 1 -- 172.21.15.37:0/1047577958 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e000c940 con 0x7f84e40af280 2024-04-25T08:55:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ec423640 1 -- 172.21.15.37:0/1047577958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e41622e0 con 0x7f84e40af280 2024-04-25T08:55:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84daffd640 1 -- 172.21.15.37:0/1047577958 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e000ccf0 con 0x7f84e40af280 2024-04-25T08:55:38.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ec423640 1 -- 172.21.15.37:0/1047577958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e414b290 con 0x7f84e40af280 2024-04-25T08:55:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84ec423640 1 -- 172.21.15.37:0/1047577958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e404efa0 con 0x7f84e40af280 2024-04-25T08:55:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84daffd640 1 -- 172.21.15.37:0/1047577958 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f84e001a070 con 0x7f84e40af280 2024-04-25T08:55:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.926+0000 7f84daffd640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84c8066f60 0x7f84c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.930+0000 7f84daffd640 1 -- 172.21.15.37:0/1047577958 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f84e00850e0 con 0x7f84e40af280 2024-04-25T08:55:38.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.930+0000 7f84ea1bf640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84c8066f60 0x7f84c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:38.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.930+0000 7f84ea1bf640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84c8066f60 0x7f84c8069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f84d40015b0 tx=0x7f84d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:38.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:38.934+0000 7f84daffd640 1 -- 172.21.15.37:0/1047577958 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f84e0056260 con 0x7f84e40af280 2024-04-25T08:55:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.094+0000 7f84ec423640 1 -- 172.21.15.37:0/1047577958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84e40727c0 con 0x7f84e40af280 2024-04-25T08:55:39.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.094+0000 7f84daffd640 1 -- 172.21.15.37:0/1047577958 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f84e0055b60 con 0x7f84e40af280 2024-04-25T08:55:39.086 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:55:39.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 -- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84c8066f60 msgr2=0x7f84c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:39.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84c8066f60 0x7f84c8069420 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f84d40015b0 tx=0x7f84d4009040 comp rx=0 tx=0).stop 2024-04-25T08:55:39.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 -- 172.21.15.37:0/1047577958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e40af280 msgr2=0x7f84e415ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:39.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e40af280 0x7f84e415ff80 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f84e000e870 tx=0x7f84e000ec70 comp rx=0 tx=0).stop 2024-04-25T08:55:39.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 -- 172.21.15.37:0/1047577958 shutdown_connections 2024-04-25T08:55:39.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f84c8066f60 0x7f84c8069420 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:39.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f84e40b2c30 0x7f84e4153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:39.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f84e40af280 0x7f84e415ff80 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:39.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 --2- 172.21.15.37:0/1047577958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f84e406c3b0 0x7f84e414ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:39.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 -- 172.21.15.37:0/1047577958 >> 172.21.15.37:0/1047577958 conn(0x7f84e4071810 msgr2=0x7f84e40cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:39.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 -- 172.21.15.37:0/1047577958 shutdown_connections 2024-04-25T08:55:39.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.098+0000 7f84ec423640 1 -- 172.21.15.37:0/1047577958 wait complete. 2024-04-25T08:55:39.105 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T08:55:39.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.242+0000 7f58b89ef640 1 Processor -- start 2024-04-25T08:55:39.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.242+0000 7f58b89ef640 1 -- start start 2024-04-25T08:55:39.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b89ef640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f58b415a700 0x7f58b415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b89ef640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f58b40b3f50 0x7f58b40cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b89ef640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b40b2090 con 0x7f58b40ce430 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b89ef640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b415f200 con 0x7f58b415a700 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b89ef640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b40b43e0 con 0x7f58b40b3f50 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f58b415a700 0x7f58b415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f58b415a700 0x7f58b415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39252/0 (socket says 172.21.15.37:39252) 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b259c640 1 -- 172.21.15.37:0/169196751 learned_addr learned my addr 172.21.15.37:0/169196751 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b1d9b640 1 --2- 172.21.15.37:0/169196751 >> v2:172.21.15.203:3300/0 conn(0x7f58b40b3f50 0x7f58b40cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b259c640 1 -- 172.21.15.37:0/169196751 >> v2:172.21.15.203:3300/0 conn(0x7f58b40b3f50 msgr2=0x7f58b40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b259c640 1 --2- 172.21.15.37:0/169196751 >> v2:172.21.15.203:3300/0 conn(0x7f58b40b3f50 0x7f58b40cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b259c640 1 -- 172.21.15.37:0/169196751 >> v1:172.21.15.203:6789/0 conn(0x7f58b40ce430 legacy=0x7f58b40b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:39.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b259c640 1 -- 172.21.15.37:0/169196751 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40b46c0 con 0x7f58b415a700 2024-04-25T08:55:39.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b259c640 1 --2- 172.21.15.37:0/169196751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58b415a700 0x7f58b415ecc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f58b406bd20 tx=0x7f58a401ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a0b6703fd1b9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:39.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.246+0000 7f58b159a640 1 -- 172.21.15.37:0/169196751 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40b4070 con 0x7f58b415a700 2024-04-25T08:55:39.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b159a640 1 -- 172.21.15.37:0/169196751 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a40a7b10 con 0x7f58b415a700 2024-04-25T08:55:39.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b159a640 1 -- 172.21.15.37:0/169196751 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40af440 con 0x7f58b415a700 2024-04-25T08:55:39.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 -- 172.21.15.37:0/169196751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58b415a700 msgr2=0x7f58b415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:39.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 --2- 172.21.15.37:0/169196751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58b415a700 0x7f58b415ecc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f58b406bd20 tx=0x7f58a401ea40 comp rx=0 tx=0).stop 2024-04-25T08:55:39.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 -- 172.21.15.37:0/169196751 shutdown_connections 2024-04-25T08:55:39.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 --2- 172.21.15.37:0/169196751 >> v2:172.21.15.203:3300/0 conn(0x7f58b40b3f50 0x7f58b40cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:39.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 --2- 172.21.15.37:0/169196751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58b415a700 0x7f58b415ecc0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:39.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 -- 172.21.15.37:0/169196751 >> 172.21.15.37:0/169196751 conn(0x7f58b40b11a0 msgr2=0x7f58b40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:39.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 -- 172.21.15.37:0/169196751 shutdown_connections 2024-04-25T08:55:39.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 -- 172.21.15.37:0/169196751 wait complete. 2024-04-25T08:55:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 Processor -- start 2024-04-25T08:55:39.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 -- start start 2024-04-25T08:55:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58b40ce430 0x7f58b4150510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58b415a700 0x7f58b4155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58b4155e90 0x7f58b41581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40cd490 con 0x7f58b40ce430 2024-04-25T08:55:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40cd310 con 0x7f58b4155e90 2024-04-25T08:55:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b89ef640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40b2bb0 con 0x7f58b415a700 2024-04-25T08:55:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58b40ce430 0x7f58b4150510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58b40ce430 0x7f58b4150510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39254/0 (socket says 172.21.15.37:39254) 2024-04-25T08:55:39.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b259c640 1 -- 172.21.15.37:0/1665762697 learned_addr learned my addr 172.21.15.37:0/1665762697 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b2d9d640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58b4155e90 0x7f58b41581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b1d9b640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58b415a700 0x7f58b4155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b259c640 1 -- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58b415a700 msgr2=0x7f58b4155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b259c640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58b415a700 0x7f58b4155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b259c640 1 -- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58b4155e90 msgr2=0x7f58b41581d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b259c640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58b4155e90 0x7f58b41581d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b259c640 1 -- 172.21.15.37:0/1665762697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b4150a50 con 0x7f58b40ce430 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b1d9b640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58b415a700 0x7f58b4155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.250+0000 7f58b259c640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58b40ce430 0x7f58b4150510 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f58a401d0f0 tx=0x7f58a401b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.254+0000 7f58a2ffd640 1 -- 172.21.15.37:0/1665762697 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40be070 con 0x7f58b40ce430 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.254+0000 7f58a2ffd640 1 -- 172.21.15.37:0/1665762697 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a40afcc0 con 0x7f58b40ce430 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.254+0000 7f58b89ef640 1 -- 172.21.15.37:0/1665762697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b4158e50 con 0x7f58b40ce430 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.254+0000 7f58a2ffd640 1 -- 172.21.15.37:0/1665762697 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40b4040 con 0x7f58b40ce430 2024-04-25T08:55:39.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.254+0000 7f58b89ef640 1 -- 172.21.15.37:0/1665762697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b4159410 con 0x7f58b40ce430 2024-04-25T08:55:39.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.254+0000 7f58b89ef640 1 -- 172.21.15.37:0/1665762697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b41562a0 con 0x7f58b40ce430 2024-04-25T08:55:39.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.254+0000 7f58a2ffd640 1 -- 172.21.15.37:0/1665762697 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58a40c1070 con 0x7f58b40ce430 2024-04-25T08:55:39.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.254+0000 7f58a2ffd640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5894066f60 0x7f5894069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:39.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.254+0000 7f58a2ffd640 1 -- 172.21.15.37:0/1665762697 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f58a4122730 con 0x7f58b40ce430 2024-04-25T08:55:39.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.254+0000 7f58b1d9b640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5894066f60 0x7f5894069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.254+0000 7f58b1d9b640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5894066f60 0x7f5894069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f58b40cf630 tx=0x7f58a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:39.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.258+0000 7f58a2ffd640 1 -- 172.21.15.37:0/1665762697 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58b41562a0 con 0x7f58b40ce430 2024-04-25T08:55:39.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:39.478+0000 7f58b89ef640 1 -- 172.21.15.37:0/1665762697 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f58b406f260 con 0x7f58b40ce430 2024-04-25T08:55:40.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.434+0000 7f58a2ffd640 1 -- 172.21.15.37:0/1665762697 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v187) v1 ==== 172+0+0 (secure 0 0 0) 0x7f58a4018780 con 0x7f58b40ce430 2024-04-25T08:55:40.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 -- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5894066f60 msgr2=0x7f5894069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:40.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5894066f60 0x7f5894069420 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f58b40cf630 tx=0x7f58a8008040 comp rx=0 tx=0).stop 2024-04-25T08:55:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 -- 172.21.15.37:0/1665762697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58b40ce430 msgr2=0x7f58b4150510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58b40ce430 0x7f58b4150510 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f58a401d0f0 tx=0x7f58a401b7e0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 -- 172.21.15.37:0/1665762697 shutdown_connections 2024-04-25T08:55:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5894066f60 0x7f5894069420 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58b4155e90 0x7f58b41581d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58b415a700 0x7f58b4155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 --2- 172.21.15.37:0/1665762697 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58b40ce430 0x7f58b4150510 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 -- 172.21.15.37:0/1665762697 >> 172.21.15.37:0/1665762697 conn(0x7f58b40b11a0 msgr2=0x7f58b406e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 -- 172.21.15.37:0/1665762697 shutdown_connections 2024-04-25T08:55:40.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.438+0000 7f58b89ef640 1 -- 172.21.15.37:0/1665762697 wait complete. 2024-04-25T08:55:40.452 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:55:40.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.594+0000 7ff89f577640 1 Processor -- start 2024-04-25T08:55:40.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.594+0000 7ff89f577640 1 -- start start 2024-04-25T08:55:40.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89f577640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff8980cfca0 0x7ff89815dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:40.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89f577640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff89815e4f0 0x7ff8980caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:40.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89f577640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8980cbf10 con 0x7ff8980b3cf0 2024-04-25T08:55:40.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89f577640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8980cc1f0 con 0x7ff8980cfca0 2024-04-25T08:55:40.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89f577640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8980cc4d0 con 0x7ff89815e4f0 2024-04-25T08:55:40.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89cb12640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff8980cfca0 0x7ff89815dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:40.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89cb12640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff8980cfca0 0x7ff89815dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40504/0 (socket says 172.21.15.37:40504) 2024-04-25T08:55:40.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89cb12640 1 -- 172.21.15.37:0/2474057142 learned_addr learned my addr 172.21.15.37:0/2474057142 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:40.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89db14640 1 --2- 172.21.15.37:0/2474057142 >> v2:172.21.15.203:3300/0 conn(0x7ff89815e4f0 0x7ff8980caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:40.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89cb12640 1 -- 172.21.15.37:0/2474057142 >> v2:172.21.15.203:3300/0 conn(0x7ff89815e4f0 msgr2=0x7ff8980caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:40.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89cb12640 1 --2- 172.21.15.37:0/2474057142 >> v2:172.21.15.203:3300/0 conn(0x7ff89815e4f0 0x7ff8980caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89cb12640 1 -- 172.21.15.37:0/2474057142 >> v1:172.21.15.203:6789/0 conn(0x7ff8980b3cf0 legacy=0x7ff8980cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:40.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89cb12640 1 -- 172.21.15.37:0/2474057142 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8980cc7b0 con 0x7ff8980cfca0 2024-04-25T08:55:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff89cb12640 1 --2- 172.21.15.37:0/2474057142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8980cfca0 0x7ff89815dfb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff8980b2be0 tx=0x7ff88801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c2f7f16f3d8e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff88ffff640 1 -- 172.21.15.37:0/2474057142 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880b0070 con 0x7ff8980cfca0 2024-04-25T08:55:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff88ffff640 1 -- 172.21.15.37:0/2474057142 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8880ab600 con 0x7ff8980cfca0 2024-04-25T08:55:40.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.598+0000 7ff88ffff640 1 -- 172.21.15.37:0/2474057142 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880ab9d0 con 0x7ff8980cfca0 2024-04-25T08:55:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 -- 172.21.15.37:0/2474057142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8980cfca0 msgr2=0x7ff89815dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 --2- 172.21.15.37:0/2474057142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8980cfca0 0x7ff89815dfb0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ff8980b2be0 tx=0x7ff88801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 -- 172.21.15.37:0/2474057142 shutdown_connections 2024-04-25T08:55:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 --2- 172.21.15.37:0/2474057142 >> v2:172.21.15.203:3300/0 conn(0x7ff89815e4f0 0x7ff8980caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 --2- 172.21.15.37:0/2474057142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8980cfca0 0x7ff89815dfb0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 -- 172.21.15.37:0/2474057142 >> 172.21.15.37:0/2474057142 conn(0x7ff89806ae60 msgr2=0x7ff89806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 -- 172.21.15.37:0/2474057142 shutdown_connections 2024-04-25T08:55:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 -- 172.21.15.37:0/2474057142 wait complete. 2024-04-25T08:55:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 Processor -- start 2024-04-25T08:55:40.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 -- start start 2024-04-25T08:55:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8980b3cf0 0x7ff898097df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8980cfca0 0x7ff8980c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff89815e4f0 0x7ff898098570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980cb9c0 con 0x7ff8980cfca0 2024-04-25T08:55:40.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980cb840 con 0x7ff89815e4f0 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980cbb40 con 0x7ff8980b3cf0 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89cb12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8980cfca0 0x7ff8980c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89cb12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8980cfca0 0x7ff8980c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40514/0 (socket says 172.21.15.37:40514) 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89cb12640 1 -- 172.21.15.37:0/2907220098 learned_addr learned my addr 172.21.15.37:0/2907220098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89db14640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff89815e4f0 0x7ff898098570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89d313640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8980b3cf0 0x7ff898097df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89cb12640 1 -- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8980b3cf0 msgr2=0x7ff898097df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89cb12640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8980b3cf0 0x7ff898097df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89cb12640 1 -- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff89815e4f0 msgr2=0x7ff898098570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89cb12640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff89815e4f0 0x7ff898098570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89cb12640 1 -- 172.21.15.37:0/2907220098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8980c59e0 con 0x7ff8980cfca0 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89d313640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8980b3cf0 0x7ff898097df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89db14640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff89815e4f0 0x7ff898098570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89cb12640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8980cfca0 0x7ff8980c54a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ae040 tx=0x7ff88801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:40.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff88dffb640 1 -- 172.21.15.37:0/2907220098 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880b0040 con 0x7ff8980cfca0 2024-04-25T08:55:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 -- 172.21.15.37:0/2907220098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff898154500 con 0x7ff8980cfca0 2024-04-25T08:55:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff88dffb640 1 -- 172.21.15.37:0/2907220098 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8880bc070 con 0x7ff8980cfca0 2024-04-25T08:55:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff89f577640 1 -- 172.21.15.37:0/2907220098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8981547a0 con 0x7ff8980cfca0 2024-04-25T08:55:40.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.602+0000 7ff88dffb640 1 -- 172.21.15.37:0/2907220098 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8880b6880 con 0x7ff8980cfca0 2024-04-25T08:55:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.606+0000 7ff89f577640 1 -- 172.21.15.37:0/2907220098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff89804efa0 con 0x7ff8980cfca0 2024-04-25T08:55:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.606+0000 7ff88dffb640 1 -- 172.21.15.37:0/2907220098 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8880b6a20 con 0x7ff8980cfca0 2024-04-25T08:55:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.606+0000 7ff88dffb640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff87c066f60 0x7ff87c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.606+0000 7ff88dffb640 1 -- 172.21.15.37:0/2907220098 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff88811f3f0 con 0x7ff8980cfca0 2024-04-25T08:55:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.606+0000 7ff89d313640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff87c066f60 0x7ff87c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:40.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.606+0000 7ff89d313640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff87c066f60 0x7ff87c069420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff894003e40 tx=0x7ff894009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:40.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.610+0000 7ff88dffb640 1 -- 172.21.15.37:0/2907220098 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8880160d0 con 0x7ff8980cfca0 2024-04-25T08:55:40.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.770+0000 7ff89f577640 1 -- 172.21.15.37:0/2907220098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff8980cad90 con 0x7ff8980cfca0 2024-04-25T08:55:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.774+0000 7ff88dffb640 1 -- 172.21.15.37:0/2907220098 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff8880f0450 con 0x7ff8980cfca0 2024-04-25T08:55:40.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 -- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff87c066f60 msgr2=0x7ff87c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:40.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff87c066f60 0x7ff87c069420 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff894003e40 tx=0x7ff894009040 comp rx=0 tx=0).stop 2024-04-25T08:55:40.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 -- 172.21.15.37:0/2907220098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8980cfca0 msgr2=0x7ff8980c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:40.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8980cfca0 0x7ff8980c54a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ff8880ae040 tx=0x7ff88801ad20 comp rx=0 tx=0).stop 2024-04-25T08:55:40.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 -- 172.21.15.37:0/2907220098 shutdown_connections 2024-04-25T08:55:40.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff87c066f60 0x7ff87c069420 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff89815e4f0 0x7ff898098570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8980cfca0 0x7ff8980c54a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 --2- 172.21.15.37:0/2907220098 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8980b3cf0 0x7ff898097df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:40.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 -- 172.21.15.37:0/2907220098 >> 172.21.15.37:0/2907220098 conn(0x7ff89806ae60 msgr2=0x7ff89815c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:40.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 -- 172.21.15.37:0/2907220098 shutdown_connections 2024-04-25T08:55:40.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:40.778+0000 7ff89f577640 1 -- 172.21.15.37:0/2907220098 wait complete. 2024-04-25T08:55:40.786 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:55:40.787 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:55:40.828 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:55:40.872 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:55:40.873 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:55:40.873 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:55:40.879 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:55:40.879 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:55:40.879 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:55:40.920 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:55:40.920 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:55:40.920 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:55:40.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:55:40.928 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:55:40.928 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:55:40.967 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:55:40.968 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:55:40.968 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:41.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f62081640 1 Processor -- start 2024-04-25T08:55:41.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f62081640 1 -- start start 2024-04-25T08:55:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f62081640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0ce510 0x7f7f5c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f62081640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5c0cee30 con 0x7f7f5c144ca0 2024-04-25T08:55:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f62081640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f5c0ccc70 con 0x7f7f5c1451f0 2024-04-25T08:55:41.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f62081640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c0cc2b0 con 0x7f7f5c0ce510 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5affd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7f5c144ca0 0x7f7f5c06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58518/0 (socket says 172.21.15.37:58518) 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5affd640 1 -- 172.21.15.37:0/2442307504 learned_addr learned my addr 172.21.15.37:0/2442307504 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5b7fe640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0ce510 0x7f7f5c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 361112037 0 0) 0x7f7f5c0cee30 con 0x7f7f5c144ca0 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f44003530 con 0x7f7f5c144ca0 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 69927480 0 0) 0x7f7f44003530 con 0x7f7f5c144ca0 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0ce510 msgr2=0x7f7f5c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0ce510 0x7f7f5c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 >> v1:172.21.15.203:6789/0 conn(0x7f7f5c1451f0 legacy=0x7f7f5c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c06dc40 con 0x7f7f5c144ca0 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f7f50001680 con 0x7f7f5c144ca0 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 >> v1:172.21.15.37:6789/0 conn(0x7f7f5c144ca0 legacy=0x7f7f5c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f44003a60 0x7f7f44003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f44004810 0x7f7f44004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0ce510 0x7f7f440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c0cc2b0 con 0x7f7f44004810 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4400bb60 con 0x7f7f44003a60 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4400bd60 con 0x7f7f5c0ce510 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5affd640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f44004810 0x7f7f44004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5b7fe640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f44003a60 0x7f7f44003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5bfff640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0ce510 0x7f7f440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5affd640 1 -- 172.21.15.37:0/2442307504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0ce510 msgr2=0x7f7f440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5affd640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f5c0ce510 0x7f7f440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5affd640 1 -- 172.21.15.37:0/2442307504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f44003a60 msgr2=0x7f7f44003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5affd640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f44003a60 0x7f7f44003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.114+0000 7f7f5affd640 1 -- 172.21.15.37:0/2442307504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f44003530 con 0x7f7f44004810 2024-04-25T08:55:41.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f5affd640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f44004810 0x7f7f44004c90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c06f4c0 tx=0x7f7f5001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cde12271b91b1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:41.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50007d40 con 0x7f7f44004810 2024-04-25T08:55:41.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f500a9070 con 0x7f7f44004810 2024-04-25T08:55:41.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f5a7fc640 1 -- 172.21.15.37:0/2442307504 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50005d40 con 0x7f7f44004810 2024-04-25T08:55:41.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f62081640 1 -- 172.21.15.37:0/2442307504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f44004810 msgr2=0x7f7f44004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:41.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f62081640 1 --2- 172.21.15.37:0/2442307504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f44004810 0x7f7f44004c90 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c06f4c0 tx=0x7f7f5001cc60 comp rx=0 tx=0).stop 2024-04-25T08:55:41.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f5bfff640 1 -- 172.21.15.37:0/2442307504 reap_dead start 2024-04-25T08:55:41.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f62081640 1 -- 172.21.15.37:0/2442307504 shutdown_connections 2024-04-25T08:55:41.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f62081640 1 -- 172.21.15.37:0/2442307504 >> 172.21.15.37:0/2442307504 conn(0x7f7f5c0b11a0 msgr2=0x7f7f5c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:41.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f62081640 1 -- 172.21.15.37:0/2442307504 shutdown_connections 2024-04-25T08:55:41.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f62081640 1 -- 172.21.15.37:0/2442307504 wait complete. 2024-04-25T08:55:41.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f62081640 1 Processor -- start 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.118+0000 7f7f62081640 1 -- start start 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f62081640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f44004810 0x7f7f5c1f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f62081640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f5c1e5080 0x7f7f5c1e5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f62081640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5c0a84d0 0x7f7f5c0a8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f62081640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c06ee30 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f62081640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c06e290 con 0x7f7f5c1e5080 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f62081640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5c0cc900 con 0x7f7f44004810 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5c0a84d0 0x7f7f5c0a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5c0a84d0 0x7f7f5c0a8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40532/0 (socket says 172.21.15.37:40532) 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f5bfff640 1 -- 172.21.15.37:0/365136961 learned_addr learned my addr 172.21.15.37:0/365136961 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f5b7fe640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f44004810 0x7f7f5c1f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f5affd640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f5c1e5080 0x7f7f5c1e5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f5bfff640 1 -- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f44004810 msgr2=0x7f7f5c1f64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f5bfff640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f44004810 0x7f7f5c1f64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f5bfff640 1 -- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f5c1e5080 msgr2=0x7f7f5c1e5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f5bfff640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f5c1e5080 0x7f7f5c1e5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f5bfff640 1 -- 172.21.15.37:0/365136961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c0b3150 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f5bfff640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5c0a84d0 0x7f7f5c0a8950 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c145840 tx=0x7f7f5400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f43fff640 1 -- 172.21.15.37:0/365136961 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54003980 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f43fff640 1 -- 172.21.15.37:0/365136961 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f54018070 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f43fff640 1 -- 172.21.15.37:0/365136961 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f54003d60 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f62081640 1 -- 172.21.15.37:0/365136961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5c1f6a30 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f62081640 1 -- 172.21.15.37:0/365136961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5c0b36d0 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f62081640 1 -- 172.21.15.37:0/365136961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f5c1e5a40 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.122+0000 7f7f43fff640 1 -- 172.21.15.37:0/365136961 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7f540050d0 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.126+0000 7f7f43fff640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7f48066f90 0x7f7f48069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.126+0000 7f7f43fff640 1 -- 172.21.15.37:0/365136961 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7f54084c10 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.126+0000 7f7f5b7fe640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7f48066f90 0x7f7f48069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.126+0000 7f7f5b7fe640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7f48066f90 0x7f7f48069450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0ca640 tx=0x7f7f4c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:41.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.130+0000 7f7f43fff640 1 -- 172.21.15.37:0/365136961 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7f54055d90 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.326+0000 7f7f62081640 1 -- 172.21.15.37:0/365136961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f5c04efa0 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.326+0000 7f7f43fff640 1 -- 172.21.15.37:0/365136961 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f7f54088090 con 0x7f7f5c0a84d0 2024-04-25T08:55:41.317 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:41.317 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5018,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2963121871","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2963121871},{"type":"v1","addr":"172.21.15.203:6837","nonce":2963121871}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5425,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1490204181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1490204181},{"type":"v1","addr":"172.21.15.203:6835","nonce":1490204181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5934,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/2218454043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":2218454043},{"type":"v1","addr":"172.21.15.37:6833","nonce":2218454043}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":6339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1362511002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1362511002},{"type":"v1","addr":"172.21.15.37:6835","nonce":1362511002}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[]} 2024-04-25T08:55:41.317 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-04-25T08:55:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.326+0000 7f7f62081640 1 -- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7f48066f90 msgr2=0x7f7f48069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.326+0000 7f7f62081640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7f48066f90 0x7f7f48069450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c0ca640 tx=0x7f7f4c008040 comp rx=0 tx=0).stop 2024-04-25T08:55:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.326+0000 7f7f62081640 1 -- 172.21.15.37:0/365136961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5c0a84d0 msgr2=0x7f7f5c0a8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.326+0000 7f7f62081640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5c0a84d0 0x7f7f5c0a8950 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c145840 tx=0x7f7f5400ec10 comp rx=0 tx=0).stop 2024-04-25T08:55:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.330+0000 7f7f62081640 1 -- 172.21.15.37:0/365136961 shutdown_connections 2024-04-25T08:55:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.330+0000 7f7f62081640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7f48066f90 0x7f7f48069450 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.330+0000 7f7f62081640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f5c0a84d0 0x7f7f5c0a8950 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.330+0000 7f7f62081640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7f5c1e5080 0x7f7f5c1e5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.330+0000 7f7f62081640 1 --2- 172.21.15.37:0/365136961 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7f44004810 0x7f7f5c1f64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.330+0000 7f7f62081640 1 -- 172.21.15.37:0/365136961 >> 172.21.15.37:0/365136961 conn(0x7f7f5c0b11a0 msgr2=0x7f7f5c142830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.330+0000 7f7f62081640 1 -- 172.21.15.37:0/365136961 shutdown_connections 2024-04-25T08:55:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.330+0000 7f7f62081640 1 -- 172.21.15.37:0/365136961 wait complete. 2024-04-25T08:55:41.337 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:55:41.337 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:55:41.337 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:55:41.337 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:55:41.337 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:55:41.337 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:55:41.337 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:55:41.338 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:55:41.338 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:55:41.338 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:55:41.338 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:55:41.338 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:55:41.338 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:55:41.338 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:55:41.338 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:55:41.339 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:55:41.348 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:55:41.348 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:55:41.349 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:55:41.349 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:55:41.353 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:55:41.353 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:55:41.358 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:55:41.358 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:55:41.392 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:55:41.392 DEBUG:teuthology.parallel:result is None 2024-04-25T08:55:41.393 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:55:41.393 DEBUG:teuthology.parallel:result is None 2024-04-25T08:55:41.399 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:55:41.399 DEBUG:teuthology.parallel:result is None 2024-04-25T08:55:41.401 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:55:41.401 DEBUG:teuthology.parallel:result is None 2024-04-25T08:55:41.401 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:55:41.455 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T08:55:41.457 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T08:55:41.471 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T08:55:41.476 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-25T08:55:41.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7c1d91640 1 Processor -- start 2024-04-25T08:55:41.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7c1d91640 1 -- start start 2024-04-25T08:55:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7c1d91640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0c44f0 0x7ff7bc099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7c1d91640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff7bc0b3b30 0x7ff7bc0c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7c1d91640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7bc09a150 con 0x7ff7bc0cc0c0 2024-04-25T08:55:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7c1d91640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7bc0cbc30 con 0x7ff7bc0b3b30 2024-04-25T08:55:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7c1d91640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc09a430 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7baffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff7bc0b3b30 0x7ff7bc0c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7baffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff7bc0b3b30 0x7ff7bc0c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40568/0 (socket says 172.21.15.37:40568) 2024-04-25T08:55:41.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7baffd640 1 -- 172.21.15.37:0/1685581392 learned_addr learned my addr 172.21.15.37:0/1685581392 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7bb7fe640 1 --2- 172.21.15.37:0/1685581392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0c44f0 0x7ff7bc099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7baffd640 1 -- 172.21.15.37:0/1685581392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0c44f0 msgr2=0x7ff7bc099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7baffd640 1 --2- 172.21.15.37:0/1685581392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0c44f0 0x7ff7bc099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7baffd640 1 -- 172.21.15.37:0/1685581392 >> v1:172.21.15.203:6789/0 conn(0x7ff7bc0cc0c0 legacy=0x7ff7bc0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:41.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.542+0000 7ff7baffd640 1 -- 172.21.15.37:0/1685581392 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc09a710 con 0x7ff7bc0b3b30 2024-04-25T08:55:41.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7baffd640 1 --2- 172.21.15.37:0/1685581392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc0b3b30 0x7ff7bc0c48d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009e50 tx=0x7ff7b401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26855c8260e3ee63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:41.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7ba7fc640 1 -- 172.21.15.37:0/1685581392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40a5620 con 0x7ff7bc0b3b30 2024-04-25T08:55:41.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7ba7fc640 1 -- 172.21.15.37:0/1685581392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b40a57c0 con 0x7ff7bc0b3b30 2024-04-25T08:55:41.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7ba7fc640 1 -- 172.21.15.37:0/1685581392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40a5b90 con 0x7ff7bc0b3b30 2024-04-25T08:55:41.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 -- 172.21.15.37:0/1685581392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc0b3b30 msgr2=0x7ff7bc0c48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:41.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 --2- 172.21.15.37:0/1685581392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc0b3b30 0x7ff7bc0c48d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009e50 tx=0x7ff7b401c910 comp rx=0 tx=0).stop 2024-04-25T08:55:41.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 -- 172.21.15.37:0/1685581392 shutdown_connections 2024-04-25T08:55:41.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 --2- 172.21.15.37:0/1685581392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc0b3b30 0x7ff7bc0c48d0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 --2- 172.21.15.37:0/1685581392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0c44f0 0x7ff7bc099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 -- 172.21.15.37:0/1685581392 >> 172.21.15.37:0/1685581392 conn(0x7ff7bc0b3010 msgr2=0x7ff7bc0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 -- 172.21.15.37:0/1685581392 shutdown_connections 2024-04-25T08:55:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 -- 172.21.15.37:0/1685581392 wait complete. 2024-04-25T08:55:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 Processor -- start 2024-04-25T08:55:41.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 -- start start 2024-04-25T08:55:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7bc0b3b30 0x7ff7bc0c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc0c44f0 0x7ff7bc0b6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0cc0c0 0x7ff7bc0b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc16e9f0 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc16e870 con 0x7ff7bc0b3b30 2024-04-25T08:55:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7c1d91640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc16eb70 con 0x7ff7bc0cc0c0 2024-04-25T08:55:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7baffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc0c44f0 0x7ff7bc0b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7baffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc0c44f0 0x7ff7bc0b6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40576/0 (socket says 172.21.15.37:40576) 2024-04-25T08:55:41.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7baffd640 1 -- 172.21.15.37:0/498343727 learned_addr learned my addr 172.21.15.37:0/498343727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7bb7fe640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7bc0b3b30 0x7ff7bc0c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.546+0000 7ff7bbfff640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0cc0c0 0x7ff7bc0b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7baffd640 1 -- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0cc0c0 msgr2=0x7ff7bc0b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7baffd640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0cc0c0 0x7ff7bc0b4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7baffd640 1 -- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7bc0b3b30 msgr2=0x7ff7bc0c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7baffd640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7bc0b3b30 0x7ff7bc0c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7baffd640 1 -- 172.21.15.37:0/498343727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc0b6fc0 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7baffd640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc0c44f0 0x7ff7bc0b6830 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff7b401a540 tx=0x7ff7b40196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7a3fff640 1 -- 172.21.15.37:0/498343727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40bc020 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7a3fff640 1 -- 172.21.15.37:0/498343727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b40b8670 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7a3fff640 1 -- 172.21.15.37:0/498343727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4009510 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7c1d91640 1 -- 172.21.15.37:0/498343727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc0b4350 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7c1d91640 1 -- 172.21.15.37:0/498343727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc0b4910 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7c1d91640 1 -- 172.21.15.37:0/498343727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7bc0b4140 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7a3fff640 1 -- 172.21.15.37:0/498343727 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7b40bf020 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7a3fff640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7a8066f60 0x7ff7a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7a3fff640 1 -- 172.21.15.37:0/498343727 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff7b411ffd0 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7bb7fe640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7a8066f60 0x7ff7a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:41.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.550+0000 7ff7bb7fe640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7a8066f60 0x7ff7a8069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00079c0 tx=0x7ff7b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:41.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.554+0000 7ff7a3fff640 1 -- 172.21.15.37:0/498343727 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7b40c6030 con 0x7ff7bc0c44f0 2024-04-25T08:55:41.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:41.726+0000 7ff7c1d91640 1 -- 172.21.15.37:0/498343727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff7bc04efa0 con 0x7ff7bc0c44f0 2024-04-25T08:55:42.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.810+0000 7ff7a3fff640 1 -- 172.21.15.37:0/498343727 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v65) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff7b4016620 con 0x7ff7bc0c44f0 2024-04-25T08:55:42.800 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-04-25T08:55:42.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 -- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7a8066f60 msgr2=0x7ff7a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:42.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7a8066f60 0x7ff7a8069420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00079c0 tx=0x7ff7b0008040 comp rx=0 tx=0).stop 2024-04-25T08:55:42.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 -- 172.21.15.37:0/498343727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc0c44f0 msgr2=0x7ff7bc0b6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:42.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc0c44f0 0x7ff7bc0b6830 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff7b401a540 tx=0x7ff7b40196b0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 -- 172.21.15.37:0/498343727 shutdown_connections 2024-04-25T08:55:42.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7a8066f60 0x7ff7a8069420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7bc0cc0c0 0x7ff7bc0b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc0c44f0 0x7ff7bc0b6830 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 --2- 172.21.15.37:0/498343727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7bc0b3b30 0x7ff7bc0c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 -- 172.21.15.37:0/498343727 >> 172.21.15.37:0/498343727 conn(0x7ff7bc0b3010 msgr2=0x7ff7bc070650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:42.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 -- 172.21.15.37:0/498343727 shutdown_connections 2024-04-25T08:55:42.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.814+0000 7ff7c1d91640 1 -- 172.21.15.37:0/498343727 wait complete. 2024-04-25T08:55:42.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:55:42.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806ee74640 1 Processor -- start 2024-04-25T08:55:42.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806ee74640 1 -- start start 2024-04-25T08:55:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806ee74640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80680ce510 0x7f80680ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806ee74640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f80681451f0 0x7f8068142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806ee74640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80680cee30 con 0x7f8068144ca0 2024-04-25T08:55:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806ee74640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80680cc2b0 con 0x7f80681451f0 2024-04-25T08:55:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806ee74640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f806806e290 con 0x7f80680ce510 2024-04-25T08:55:42.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f8067fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8068144ca0 0x7f806806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58528/0 (socket says 172.21.15.37:58528) 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f8067fff640 1 -- 172.21.15.37:0/2492780836 learned_addr learned my addr 172.21.15.37:0/2492780836 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806cc10640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80680ce510 0x7f80680ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806d411640 1 --2- 172.21.15.37:0/2492780836 >> v2:172.21.15.203:3300/0 conn(0x7f80681451f0 0x7f8068142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394810724 0 0) 0x7f80680cee30 con 0x7f8068144ca0 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8050003530 con 0x7f8068144ca0 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3283518213 0 0) 0x7f8050003530 con 0x7f8068144ca0 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80680ce510 msgr2=0x7f80680ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80680ce510 0x7f80680ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 >> v2:172.21.15.203:3300/0 conn(0x7f80681451f0 msgr2=0x7f8068142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 --2- 172.21.15.37:0/2492780836 >> v2:172.21.15.203:3300/0 conn(0x7f80681451f0 0x7f8068142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80680ccc70 con 0x7f8068144ca0 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f805c001680 con 0x7f8068144ca0 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 >> v1:172.21.15.37:6789/0 conn(0x7f8068144ca0 legacy=0x7f806806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:42.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8050003a30 0x7f8050003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80680ce510 0x7f8050004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8050004d50 0x7f80500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80680cc2b0 con 0x7f8050004d50 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806806e290 con 0x7f8050003a30 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805000ba50 con 0x7f80680ce510 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806d411640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8050004d50 0x7f80500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f8067fff640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80680ce510 0x7f8050004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806cc10640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8050003a30 0x7f8050003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806d411640 1 -- 172.21.15.37:0/2492780836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80680ce510 msgr2=0x7f8050004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806d411640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80680ce510 0x7f8050004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806d411640 1 -- 172.21.15.37:0/2492780836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8050003a30 msgr2=0x7f8050003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806d411640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8050003a30 0x7f8050003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.958+0000 7f806d411640 1 -- 172.21.15.37:0/2492780836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8050003530 con 0x7f8050004d50 2024-04-25T08:55:42.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f806d411640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8050004d50 0x7f80500051d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8060009fc0 tx=0x7f806001ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=379a73a09bd14a3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80600b2070 con 0x7f8050004d50 2024-04-25T08:55:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80600a5cd0 con 0x7f8050004d50 2024-04-25T08:55:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f80677fe640 1 -- 172.21.15.37:0/2492780836 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060009c10 con 0x7f8050004d50 2024-04-25T08:55:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f806ee74640 1 -- 172.21.15.37:0/2492780836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8050004d50 msgr2=0x7f80500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f806ee74640 1 --2- 172.21.15.37:0/2492780836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8050004d50 0x7f80500051d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8060009fc0 tx=0x7f806001ccc0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f806d411640 1 -- 172.21.15.37:0/2492780836 reap_dead start 2024-04-25T08:55:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f806ee74640 1 -- 172.21.15.37:0/2492780836 shutdown_connections 2024-04-25T08:55:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f806ee74640 1 -- 172.21.15.37:0/2492780836 >> 172.21.15.37:0/2492780836 conn(0x7f80680b11a0 msgr2=0x7f80680b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f806ee74640 1 -- 172.21.15.37:0/2492780836 shutdown_connections 2024-04-25T08:55:42.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f806ee74640 1 -- 172.21.15.37:0/2492780836 wait complete. 2024-04-25T08:55:42.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f806ee74640 1 Processor -- start 2024-04-25T08:55:42.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.962+0000 7f806ee74640 1 -- start start 2024-04-25T08:55:42.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806ee74640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f806815b1c0 0x7f80681e4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:42.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806ee74640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80681e4db0 0x7f80681e68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:42.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806ee74640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80681e6e00 0x7f80681e0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:42.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806ee74640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806806ee30 con 0x7f80681e4db0 2024-04-25T08:55:42.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806ee74640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806806df20 con 0x7f806815b1c0 2024-04-25T08:55:42.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806ee74640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80680cc900 con 0x7f80681e6e00 2024-04-25T08:55:42.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f8067fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80681e4db0 0x7f80681e68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:42.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f8067fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80681e4db0 0x7f80681e68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40590/0 (socket says 172.21.15.37:40590) 2024-04-25T08:55:42.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f8067fff640 1 -- 172.21.15.37:0/2510214624 learned_addr learned my addr 172.21.15.37:0/2510214624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:42.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806d411640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80681e6e00 0x7f80681e0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:42.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806cc10640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f806815b1c0 0x7f80681e4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f8067fff640 1 -- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80681e6e00 msgr2=0x7f80681e0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f8067fff640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80681e6e00 0x7f80681e0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f8067fff640 1 -- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f806815b1c0 msgr2=0x7f80681e4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f8067fff640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f806815b1c0 0x7f80681e4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f8067fff640 1 -- 172.21.15.37:0/2510214624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80681e7640 con 0x7f80681e4db0 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806d411640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80681e6e00 0x7f80681e0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f8067fff640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80681e4db0 0x7f80681e68c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f805c00e870 tx=0x7f805c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f80657fa640 1 -- 172.21.15.37:0/2510214624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c00c870 con 0x7f80681e4db0 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f80657fa640 1 -- 172.21.15.37:0/2510214624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805c018070 con 0x7f80681e4db0 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806ee74640 1 -- 172.21.15.37:0/2510214624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80681e7920 con 0x7f80681e4db0 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f80657fa640 1 -- 172.21.15.37:0/2510214624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c00cc50 con 0x7f80681e4db0 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806ee74640 1 -- 172.21.15.37:0/2510214624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80681e7ee0 con 0x7f80681e4db0 2024-04-25T08:55:42.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f806ee74640 1 -- 172.21.15.37:0/2510214624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f806804efa0 con 0x7f80681e4db0 2024-04-25T08:55:42.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f80657fa640 1 -- 172.21.15.37:0/2510214624 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f805c01d460 con 0x7f80681e4db0 2024-04-25T08:55:42.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.966+0000 7f80657fa640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8054066f90 0x7f8054069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:42.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.970+0000 7f80657fa640 1 -- 172.21.15.37:0/2510214624 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f805c084ed0 con 0x7f80681e4db0 2024-04-25T08:55:42.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.970+0000 7f806cc10640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8054066f90 0x7f8054069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:42.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.970+0000 7f806cc10640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8054066f90 0x7f8054069450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f80580098f0 tx=0x7f8058008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:42.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:42.974+0000 7f80657fa640 1 -- 172.21.15.37:0/2510214624 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f805c056000 con 0x7f80681e4db0 2024-04-25T08:55:43.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.194+0000 7f806ee74640 1 -- 172.21.15.37:0/2510214624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f806815bac0 con 0x7f80681e4db0 2024-04-25T08:55:43.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.194+0000 7f80657fa640 1 -- 172.21.15.37:0/2510214624 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f805c055900 con 0x7f80681e4db0 2024-04-25T08:55:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:43.186 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:55:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-04-25T08:55:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-04-25T08:55:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpm2pLfABAAJbn2dGKdl8SHS8pApIWC1Q==", 2024-04-25T08:55:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmqB+jDxAAmRnBzN3vqO8HIf/cd3QoJQ==", 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmiXRqCRAACtY8cFxwqOT4WNzj9WmeKA==", 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmX3ARGBAAW/ZUr85NsKtd8HzxE+Yefw==", 2024-04-25T08:55:43.188 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCSGSpmwhMiJhAAbmQz36LujHeYXaKIOdLLSA==", 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.189 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCVGSpm/OLPKxAA9ZmrrWxvt4QDj2lHATnzSg==", 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCYGSpmq7tEMxAA/wjwgkrtnTkrMoC6G+YIlQ==", 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.190 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCbGSpmReB1ORAAkXje/r11I3ArZHErIvijgw==", 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.191 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQChGSpm4XfLORAAGj+R1Nk/EGmvEdpt8kQJlg==", 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQClGSpmd5f1ARAAKF3XG//v3Uqx4bckcD4JMw==", 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.192 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCoGSpmi9OECBAAJBskMWGCNlAaej9xUc6g8w==", 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCrGSpmunQiDhAA3UZo2vzT44HEYUBUia75NA==", 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA==", 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmu8UoJhAAyutD5hzKLZmr57WuAWScuQ==", 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.194 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmGVVdIxAAY7/VOuqF5OoomqqVxE1Ifw==", 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmjFNDKxAAmo6DB9W17TZaGlvkdqkSuQ==", 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm+mGgJRAA9oTRTlZpGBdCZ0/bj580Bg==", 2024-04-25T08:55:43.196 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpmLChTORAAo6fs/9PDJ5YqqcbfHfPO9Q==", 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:43.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:43.198 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-04-25T08:55:43.198 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm3NLHNhAAeqSA658L6P9OcrSO5JVzLg==", 2024-04-25T08:55:43.198 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:55:43.198 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:55:43.198 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T08:55:43.198 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:55:43.198 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.198 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:43.198 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T08:55:43.198 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:55:43.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 -- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8054066f90 msgr2=0x7f8054069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8054066f90 0x7f8054069450 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f80580098f0 tx=0x7f8058008040 comp rx=0 tx=0).stop 2024-04-25T08:55:43.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 -- 172.21.15.37:0/2510214624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80681e4db0 msgr2=0x7f80681e68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80681e4db0 0x7f80681e68c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f805c00e870 tx=0x7f805c00ed30 comp rx=0 tx=0).stop 2024-04-25T08:55:43.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 -- 172.21.15.37:0/2510214624 shutdown_connections 2024-04-25T08:55:43.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8054066f90 0x7f8054069450 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80681e6e00 0x7f80681e0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80681e4db0 0x7f80681e68c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 --2- 172.21.15.37:0/2510214624 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f806815b1c0 0x7f80681e4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 -- 172.21.15.37:0/2510214624 >> 172.21.15.37:0/2510214624 conn(0x7f80680b11a0 msgr2=0x7f80680c59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 -- 172.21.15.37:0/2510214624 shutdown_connections 2024-04-25T08:55:43.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.198+0000 7f806ee74640 1 -- 172.21.15.37:0/2510214624 wait complete. 2024-04-25T08:55:43.206 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:43.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.338+0000 7febb9fe2640 1 Processor -- start 2024-04-25T08:55:43.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.338+0000 7febb9fe2640 1 -- start start 2024-04-25T08:55:43.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb9fe2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7febb4144ca0 0x7febb406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb9fe2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7febb406ec60 0x7febb4142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb9fe2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb40ccc70 con 0x7febb40ce510 2024-04-25T08:55:43.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb9fe2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb40cc2b0 con 0x7febb406ec60 2024-04-25T08:55:43.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb9fe2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb406e290 con 0x7febb4144ca0 2024-04-25T08:55:43.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7febb406ec60 0x7febb4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7febb406ec60 0x7febb4142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40594/0 (socket says 172.21.15.37:40594) 2024-04-25T08:55:43.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb3fff640 1 -- 172.21.15.37:0/892389061 learned_addr learned my addr 172.21.15.37:0/892389061 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:43.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb2ffd640 1 --2- 172.21.15.37:0/892389061 >> v2:172.21.15.203:3300/0 conn(0x7febb4144ca0 0x7febb406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb3fff640 1 -- 172.21.15.37:0/892389061 >> v2:172.21.15.203:3300/0 conn(0x7febb4144ca0 msgr2=0x7febb406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:43.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb3fff640 1 --2- 172.21.15.37:0/892389061 >> v2:172.21.15.203:3300/0 conn(0x7febb4144ca0 0x7febb406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb3fff640 1 -- 172.21.15.37:0/892389061 >> v1:172.21.15.203:6789/0 conn(0x7febb40ce510 legacy=0x7febb40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb3fff640 1 -- 172.21.15.37:0/892389061 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb406dc40 con 0x7febb406ec60 2024-04-25T08:55:43.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.342+0000 7febb3fff640 1 --2- 172.21.15.37:0/892389061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb406ec60 0x7febb4142680 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7febb4070fe0 tx=0x7febac01f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479cc78b6d6f60e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:43.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb27fc640 1 -- 172.21.15.37:0/892389061 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac0b4070 con 0x7febb406ec60 2024-04-25T08:55:43.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb27fc640 1 -- 172.21.15.37:0/892389061 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febac0af400 con 0x7febb406ec60 2024-04-25T08:55:43.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb27fc640 1 -- 172.21.15.37:0/892389061 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac0af7d0 con 0x7febb406ec60 2024-04-25T08:55:43.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 -- 172.21.15.37:0/892389061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb406ec60 msgr2=0x7febb4142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 --2- 172.21.15.37:0/892389061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb406ec60 0x7febb4142680 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7febb4070fe0 tx=0x7febac01f1a0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 -- 172.21.15.37:0/892389061 shutdown_connections 2024-04-25T08:55:43.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 --2- 172.21.15.37:0/892389061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb406ec60 0x7febb4142680 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 --2- 172.21.15.37:0/892389061 >> v2:172.21.15.203:3300/0 conn(0x7febb4144ca0 0x7febb406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 -- 172.21.15.37:0/892389061 >> 172.21.15.37:0/892389061 conn(0x7febb40b11a0 msgr2=0x7febb40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:43.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 -- 172.21.15.37:0/892389061 shutdown_connections 2024-04-25T08:55:43.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 -- 172.21.15.37:0/892389061 wait complete. 2024-04-25T08:55:43.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 Processor -- start 2024-04-25T08:55:43.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 -- start start 2024-04-25T08:55:43.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb406ec60 0x7febb40ca960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40ce510 0x7febb41f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb4144ca0 0x7febb41f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb41451f0 con 0x7febb40ce510 2024-04-25T08:55:43.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb40cc900 con 0x7febb406ec60 2024-04-25T08:55:43.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb4145370 con 0x7febb4144ca0 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40ce510 0x7febb41f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40ce510 0x7febb41f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40598/0 (socket says 172.21.15.37:40598) 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb2ffd640 1 -- 172.21.15.37:0/3063592072 learned_addr learned my addr 172.21.15.37:0/3063592072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb3fff640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb4144ca0 0x7febb41f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb37fe640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb406ec60 0x7febb40ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb2ffd640 1 -- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb4144ca0 msgr2=0x7febb41f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb2ffd640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb4144ca0 0x7febb41f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb2ffd640 1 -- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb406ec60 msgr2=0x7febb40ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb2ffd640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb406ec60 0x7febb40ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb2ffd640 1 -- 172.21.15.37:0/3063592072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb40caea0 con 0x7febb40ce510 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb37fe640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb406ec60 0x7febb40ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:43.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb2ffd640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40ce510 0x7febb41f4ba0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7feba800d5d0 tx=0x7feba800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:43.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7feb9bfff640 1 -- 172.21.15.37:0/3063592072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8004240 con 0x7febb40ce510 2024-04-25T08:55:43.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7feb9bfff640 1 -- 172.21.15.37:0/3063592072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba80043e0 con 0x7febb40ce510 2024-04-25T08:55:43.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 -- 172.21.15.37:0/3063592072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb40c82e0 con 0x7febb40ce510 2024-04-25T08:55:43.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7feb9bfff640 1 -- 172.21.15.37:0/3063592072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8003840 con 0x7febb40ce510 2024-04-25T08:55:43.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.346+0000 7febb9fe2640 1 -- 172.21.15.37:0/3063592072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb40c88a0 con 0x7febb40ce510 2024-04-25T08:55:43.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.350+0000 7febb9fe2640 1 -- 172.21.15.37:0/3063592072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb404efa0 con 0x7febb40ce510 2024-04-25T08:55:43.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.350+0000 7feb9bfff640 1 -- 172.21.15.37:0/3063592072 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feba80039e0 con 0x7febb40ce510 2024-04-25T08:55:43.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.350+0000 7feb9bfff640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feba0066f90 0x7feba0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.350+0000 7feb9bfff640 1 -- 172.21.15.37:0/3063592072 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5249+0+0 (secure 0 0 0) 0x7feba80850d0 con 0x7febb40ce510 2024-04-25T08:55:43.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.350+0000 7febb37fe640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feba0066f90 0x7feba0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.350+0000 7febb37fe640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feba0066f90 0x7feba0069450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7febb40c70b0 tx=0x7feba4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:43.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.354+0000 7feb9bfff640 1 -- 172.21.15.37:0/3063592072 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feba801c360 con 0x7febb40ce510 2024-04-25T08:55:43.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.510+0000 7febb9fe2640 1 -- 172.21.15.37:0/3063592072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febb40ca740 con 0x7febb40ce510 2024-04-25T08:55:43.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.510+0000 7feb9bfff640 1 -- 172.21.15.37:0/3063592072 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+13128 (secure 0 0 0) 0x7feba80562a0 con 0x7febb40ce510 2024-04-25T08:55:43.503 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:43.503 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":65,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:55:42.809459+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:43.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 -- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feba0066f90 msgr2=0x7feba0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feba0066f90 0x7feba0069450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7febb40c70b0 tx=0x7feba4009040 comp rx=0 tx=0).stop 2024-04-25T08:55:43.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 -- 172.21.15.37:0/3063592072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40ce510 msgr2=0x7febb41f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40ce510 0x7febb41f4ba0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7feba800d5d0 tx=0x7feba800da90 comp rx=0 tx=0).stop 2024-04-25T08:55:43.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 -- 172.21.15.37:0/3063592072 shutdown_connections 2024-04-25T08:55:43.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feba0066f90 0x7feba0069450 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7febb4144ca0 0x7febb41f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febb40ce510 0x7febb41f4ba0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 --2- 172.21.15.37:0/3063592072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7febb406ec60 0x7febb40ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 -- 172.21.15.37:0/3063592072 >> 172.21.15.37:0/3063592072 conn(0x7febb40b11a0 msgr2=0x7febb4071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:43.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 -- 172.21.15.37:0/3063592072 shutdown_connections 2024-04-25T08:55:43.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.514+0000 7febb9fe2640 1 -- 172.21.15.37:0/3063592072 wait complete. 2024-04-25T08:55:43.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:55:43.520 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:55:43.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.650+0000 7f27786b0640 1 Processor -- start 2024-04-25T08:55:43.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.650+0000 7f27786b0640 1 -- start start 2024-04-25T08:55:43.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f27786b0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f277015b420 0x7f27700ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f27786b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f277015b800 0x7f27700cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f27786b0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f27700cfad0 0x7f277015f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f27786b0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27700cff20 con 0x7f277015b420 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f27786b0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27700cc930 con 0x7f27700cfad0 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f27786b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27700cbd20 con 0x7f277015b800 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277644c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f277015b420 0x7f27700ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277644c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f277015b420 0x7f27700ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40608/0 (socket says 172.21.15.37:40608) 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277644c640 1 -- 172.21.15.37:0/1447608677 learned_addr learned my addr 172.21.15.37:0/1447608677 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f2776c4d640 1 --2- 172.21.15.37:0/1447608677 >> v2:172.21.15.203:3300/0 conn(0x7f27700cfad0 0x7f277015f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f2775c4b640 1 --2- 172.21.15.37:0/1447608677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f277015b800 0x7f27700cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277644c640 1 -- 172.21.15.37:0/1447608677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f277015b800 msgr2=0x7f27700cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277644c640 1 --2- 172.21.15.37:0/1447608677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f277015b800 0x7f27700cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277644c640 1 -- 172.21.15.37:0/1447608677 >> v2:172.21.15.203:3300/0 conn(0x7f27700cfad0 msgr2=0x7f277015f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277644c640 1 --2- 172.21.15.37:0/1447608677 >> v2:172.21.15.203:3300/0 conn(0x7f27700cfad0 0x7f277015f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277644c640 1 -- 172.21.15.37:0/1447608677 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700cc000 con 0x7f277015b420 2024-04-25T08:55:43.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277644c640 1 --2- 172.21.15.37:0/1447608677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277015b420 0x7f27700ca3c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f27700d0440 tx=0x7f276c01c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c0fb8a02830700 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:43.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277544a640 1 -- 172.21.15.37:0/1447608677 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0a5580 con 0x7f277015b420 2024-04-25T08:55:43.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277544a640 1 -- 172.21.15.37:0/1447608677 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f276c0a5720 con 0x7f277015b420 2024-04-25T08:55:43.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.654+0000 7f277544a640 1 -- 172.21.15.37:0/1447608677 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276c0a5af0 con 0x7f277015b420 2024-04-25T08:55:43.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 -- 172.21.15.37:0/1447608677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277015b420 msgr2=0x7f27700ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 --2- 172.21.15.37:0/1447608677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277015b420 0x7f27700ca3c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f27700d0440 tx=0x7f276c01c700 comp rx=0 tx=0).stop 2024-04-25T08:55:43.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 -- 172.21.15.37:0/1447608677 shutdown_connections 2024-04-25T08:55:43.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 --2- 172.21.15.37:0/1447608677 >> v2:172.21.15.203:3300/0 conn(0x7f27700cfad0 0x7f277015f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 --2- 172.21.15.37:0/1447608677 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f277015b800 0x7f27700cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 --2- 172.21.15.37:0/1447608677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277015b420 0x7f27700ca3c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 -- 172.21.15.37:0/1447608677 >> 172.21.15.37:0/1447608677 conn(0x7f277006ae60 msgr2=0x7f277006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:43.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 -- 172.21.15.37:0/1447608677 shutdown_connections 2024-04-25T08:55:43.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 -- 172.21.15.37:0/1447608677 wait complete. 2024-04-25T08:55:43.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 Processor -- start 2024-04-25T08:55:43.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 -- start start 2024-04-25T08:55:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27700cfad0 0x7f277006d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f277015b420 0x7f2770159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277015b800 0x7f277015bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27700caa80 con 0x7f277015b800 2024-04-25T08:55:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27700ca900 con 0x7f27700cfad0 2024-04-25T08:55:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f27786b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27700cadb0 con 0x7f277015b420 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f2776c4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277015b800 0x7f277015bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f2776c4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277015b800 0x7f277015bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40618/0 (socket says 172.21.15.37:40618) 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f2776c4d640 1 -- 172.21.15.37:0/517237681 learned_addr learned my addr 172.21.15.37:0/517237681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.658+0000 7f2775c4b640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f277015b420 0x7f2770159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f277644c640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27700cfad0 0x7f277006d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2776c4d640 1 -- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f277015b420 msgr2=0x7f2770159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2776c4d640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f277015b420 0x7f2770159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2776c4d640 1 -- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27700cfad0 msgr2=0x7f277006d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2776c4d640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27700cfad0 0x7f277006d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2776c4d640 1 -- 172.21.15.37:0/517237681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27700c65d0 con 0x7f277015b800 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f277644c640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27700cfad0 0x7f277006d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2776c4d640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277015b800 0x7f277015bf80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f276800d800 tx=0x7f276800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2766ffd640 1 -- 172.21.15.37:0/517237681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276800b6c0 con 0x7f277015b800 2024-04-25T08:55:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2766ffd640 1 -- 172.21.15.37:0/517237681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2768019070 con 0x7f277015b800 2024-04-25T08:55:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f27786b0640 1 -- 172.21.15.37:0/517237681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27700c68b0 con 0x7f277015b800 2024-04-25T08:55:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2766ffd640 1 -- 172.21.15.37:0/517237681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276800baa0 con 0x7f277015b800 2024-04-25T08:55:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f27786b0640 1 -- 172.21.15.37:0/517237681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27700c6e70 con 0x7f277015b800 2024-04-25T08:55:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f27786b0640 1 -- 172.21.15.37:0/517237681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f277004efa0 con 0x7f277015b800 2024-04-25T08:55:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2766ffd640 1 -- 172.21.15.37:0/517237681 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f276800bc40 con 0x7f277015b800 2024-04-25T08:55:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2766ffd640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2754066f60 0x7f2754069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f2766ffd640 1 -- 172.21.15.37:0/517237681 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2768085250 con 0x7f277015b800 2024-04-25T08:55:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f277644c640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2754066f60 0x7f2754069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.662+0000 7f277644c640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2754066f60 0x7f2754069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f276c0ae000 tx=0x7f276c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.666+0000 7f2766ffd640 1 -- 172.21.15.37:0/517237681 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27680564b0 con 0x7f277015b800 2024-04-25T08:55:43.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.826+0000 7f27786b0640 1 -- 172.21.15.37:0/517237681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27700ca540 con 0x7f277015b800 2024-04-25T08:55:43.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.826+0000 7f2766ffd640 1 -- 172.21.15.37:0/517237681 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f276805ae20 con 0x7f277015b800 2024-04-25T08:55:43.818 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:55:43.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 -- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2754066f60 msgr2=0x7f2754069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2754066f60 0x7f2754069420 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f276c0ae000 tx=0x7f276c0a7070 comp rx=0 tx=0).stop 2024-04-25T08:55:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 -- 172.21.15.37:0/517237681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277015b800 msgr2=0x7f277015bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277015b800 0x7f277015bf80 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f276800d800 tx=0x7f276800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 -- 172.21.15.37:0/517237681 shutdown_connections 2024-04-25T08:55:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2754066f60 0x7f2754069420 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f277015b800 0x7f277015bf80 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f277015b420 0x7f2770159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 --2- 172.21.15.37:0/517237681 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27700cfad0 0x7f277006d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 -- 172.21.15.37:0/517237681 >> 172.21.15.37:0/517237681 conn(0x7f277006ae60 msgr2=0x7f27700b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 -- 172.21.15.37:0/517237681 shutdown_connections 2024-04-25T08:55:43.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.830+0000 7f27786b0640 1 -- 172.21.15.37:0/517237681 wait complete. 2024-04-25T08:55:43.837 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:55:43.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.962+0000 7ff1de1b7640 1 Processor -- start 2024-04-25T08:55:43.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.962+0000 7ff1de1b7640 1 -- start start 2024-04-25T08:55:43.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1de1b7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff1d81451f0 0x7ff1d8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1de1b7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d80ccc70 con 0x7ff1d80ce510 2024-04-25T08:55:43.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1de1b7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d806dc40 con 0x7ff1d8144ca0 2024-04-25T08:55:43.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1de1b7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d80cc2b0 con 0x7ff1d81451f0 2024-04-25T08:55:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff1d81451f0 0x7ff1d8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d77fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff1d80ce510 0x7ff1d80ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58534/0 (socket says 172.21.15.37:58534) 2024-04-25T08:55:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d77fe640 1 -- 172.21.15.37:0/515656071 learned_addr learned my addr 172.21.15.37:0/515656071 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d6ffd640 1 -- 172.21.15.37:0/515656071 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972285814 0 0) 0x7ff1d80ccc70 con 0x7ff1d80ce510 2024-04-25T08:55:43.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d6ffd640 1 -- 172.21.15.37:0/515656071 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c4003530 con 0x7ff1d80ce510 2024-04-25T08:55:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d7fff640 1 -- 172.21.15.37:0/515656071 >> v1:172.21.15.203:6789/0 conn(0x7ff1d8144ca0 legacy=0x7ff1d806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d7fff640 1 -- 172.21.15.37:0/515656071 >> v1:172.21.15.37:6789/0 conn(0x7ff1d80ce510 legacy=0x7ff1d80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d7fff640 1 -- 172.21.15.37:0/515656071 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d806df20 con 0x7ff1d81451f0 2024-04-25T08:55:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d7fff640 1 --2- 172.21.15.37:0/515656071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1d81451f0 0x7ff1d8142680 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009dc0 tx=0x7ff1d001f140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de1c4b5caec1d434 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d6ffd640 1 -- 172.21.15.37:0/515656071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00b4070 con 0x7ff1d81451f0 2024-04-25T08:55:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d6ffd640 1 -- 172.21.15.37:0/515656071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d00af600 con 0x7ff1d81451f0 2024-04-25T08:55:43.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.966+0000 7ff1d6ffd640 1 -- 172.21.15.37:0/515656071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00af9d0 con 0x7ff1d81451f0 2024-04-25T08:55:43.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 -- 172.21.15.37:0/515656071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1d81451f0 msgr2=0x7ff1d8142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 --2- 172.21.15.37:0/515656071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1d81451f0 0x7ff1d8142680 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009dc0 tx=0x7ff1d001f140 comp rx=0 tx=0).stop 2024-04-25T08:55:43.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 -- 172.21.15.37:0/515656071 shutdown_connections 2024-04-25T08:55:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 --2- 172.21.15.37:0/515656071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1d81451f0 0x7ff1d8142680 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 -- 172.21.15.37:0/515656071 >> 172.21.15.37:0/515656071 conn(0x7ff1d80b11a0 msgr2=0x7ff1d80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 -- 172.21.15.37:0/515656071 shutdown_connections 2024-04-25T08:55:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 -- 172.21.15.37:0/515656071 wait complete. 2024-04-25T08:55:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 Processor -- start 2024-04-25T08:55:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 -- start start 2024-04-25T08:55:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1d80ce510 0x7ff1d81f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1d8144ca0 0x7ff1d81e55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1d81451f0 0x7ff1d81e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80cc900 con 0x7ff1d80ce510 2024-04-25T08:55:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d806e290 con 0x7ff1d81451f0 2024-04-25T08:55:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d8142bc0 con 0x7ff1d8144ca0 2024-04-25T08:55:43.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1d80ce510 0x7ff1d81f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1d80ce510 0x7ff1d81f0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40624/0 (socket says 172.21.15.37:40624) 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d77fe640 1 -- 172.21.15.37:0/1963175027 learned_addr learned my addr 172.21.15.37:0/1963175027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1ceffd640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1d8144ca0 0x7ff1d81e55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d7fff640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1d81451f0 0x7ff1d81e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d77fe640 1 -- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1d8144ca0 msgr2=0x7ff1d81e55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d77fe640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1d8144ca0 0x7ff1d81e55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d77fe640 1 -- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1d81451f0 msgr2=0x7ff1d81e5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d77fe640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1d81451f0 0x7ff1d81e5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d77fe640 1 -- 172.21.15.37:0/1963175027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d81e24b0 con 0x7ff1d80ce510 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1ceffd640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1d8144ca0 0x7ff1d81e55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d7fff640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1d81451f0 0x7ff1d81e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d77fe640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1d80ce510 0x7ff1d81f0e70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800c500 tx=0x7ff1c800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d4ff9640 1 -- 172.21.15.37:0/1963175027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8017070 con 0x7ff1d80ce510 2024-04-25T08:55:43.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d4ff9640 1 -- 172.21.15.37:0/1963175027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c8005cf0 con 0x7ff1d80ce510 2024-04-25T08:55:43.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 -- 172.21.15.37:0/1963175027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d81e2790 con 0x7ff1d80ce510 2024-04-25T08:55:43.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1d4ff9640 1 -- 172.21.15.37:0/1963175027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c80072f0 con 0x7ff1d80ce510 2024-04-25T08:55:43.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.970+0000 7ff1de1b7640 1 -- 172.21.15.37:0/1963175027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d812ec10 con 0x7ff1d80ce510 2024-04-25T08:55:43.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.974+0000 7ff1de1b7640 1 -- 172.21.15.37:0/1963175027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d804efa0 con 0x7ff1d80ce510 2024-04-25T08:55:43.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.974+0000 7ff1d4ff9640 1 -- 172.21.15.37:0/1963175027 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff1c801a070 con 0x7ff1d80ce510 2024-04-25T08:55:43.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.974+0000 7ff1d4ff9640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff1b8066f60 0x7ff1b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:43.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.974+0000 7ff1d4ff9640 1 -- 172.21.15.37:0/1963175027 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff1c80841c0 con 0x7ff1d80ce510 2024-04-25T08:55:43.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.974+0000 7ff1ceffd640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff1b8066f60 0x7ff1b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:43.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.974+0000 7ff1ceffd640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff1b8066f60 0x7ff1b8069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8142a80 tx=0x7ff1c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:43.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:43.978+0000 7ff1d4ff9640 1 -- 172.21.15.37:0/1963175027 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff1c8054d40 con 0x7ff1d80ce510 2024-04-25T08:55:44.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 -- 172.21.15.37:0/1963175027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff1d80b1cc0 con 0x7ff1d80ce510 2024-04-25T08:55:44.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1d4ff9640 1 -- 172.21.15.37:0/1963175027 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff1c8058de0 con 0x7ff1d80ce510 2024-04-25T08:55:44.128 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:44.128 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T08:55:44.128 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:44.128 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T08:55:44.128 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T08:55:44.129 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:44.129 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T08:55:44.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 -- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff1b8066f60 msgr2=0x7ff1b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff1b8066f60 0x7ff1b8069420 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8142a80 tx=0x7ff1c0009040 comp rx=0 tx=0).stop 2024-04-25T08:55:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 -- 172.21.15.37:0/1963175027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1d80ce510 msgr2=0x7ff1d81f0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1d80ce510 0x7ff1d81f0e70 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800c500 tx=0x7ff1c800c9c0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 -- 172.21.15.37:0/1963175027 shutdown_connections 2024-04-25T08:55:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff1b8066f60 0x7ff1b8069420 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1d81451f0 0x7ff1d81e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff1d8144ca0 0x7ff1d81e55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 --2- 172.21.15.37:0/1963175027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1d80ce510 0x7ff1d81f0e70 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 -- 172.21.15.37:0/1963175027 >> 172.21.15.37:0/1963175027 conn(0x7ff1d80b11a0 msgr2=0x7ff1d80c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 -- 172.21.15.37:0/1963175027 shutdown_connections 2024-04-25T08:55:44.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.138+0000 7ff1de1b7640 1 -- 172.21.15.37:0/1963175027 wait complete. 2024-04-25T08:55:44.145 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:55:44.146 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:55:44.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa9380b9640 1 Processor -- start 2024-04-25T08:55:44.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa9380b9640 1 -- start start 2024-04-25T08:55:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa9380b9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa93015b8e0 0x7fa93006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa9380b9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa9300cd470 0x7fa9300cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa9380b9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9300c3d10 con 0x7fa9300ad280 2024-04-25T08:55:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa9380b9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa93015bcc0 con 0x7fa9300cd470 2024-04-25T08:55:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa9380b9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300cca20 con 0x7fa93015b8e0 2024-04-25T08:55:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa935654640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa9300cd470 0x7fa9300cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa935654640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa9300cd470 0x7fa9300cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40636/0 (socket says 172.21.15.37:40636) 2024-04-25T08:55:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa935654640 1 -- 172.21.15.37:0/3385649353 learned_addr learned my addr 172.21.15.37:0/3385649353 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa935e55640 1 --2- 172.21.15.37:0/3385649353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa93015b8e0 0x7fa93006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa935654640 1 -- 172.21.15.37:0/3385649353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa93015b8e0 msgr2=0x7fa93006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa935654640 1 --2- 172.21.15.37:0/3385649353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa93015b8e0 0x7fa93006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa935654640 1 -- 172.21.15.37:0/3385649353 >> v1:172.21.15.203:6789/0 conn(0x7fa9300ad280 legacy=0x7fa9300cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa935654640 1 -- 172.21.15.37:0/3385649353 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93006b1d0 con 0x7fa9300cd470 2024-04-25T08:55:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.274+0000 7fa935e55640 1 --2- 172.21.15.37:0/3385649353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa93015b8e0 0x7fa93006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa935654640 1 --2- 172.21.15.37:0/3385649353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9300cd470 0x7fa9300cd8d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa920009870 tx=0x7fa92001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532df9ce92203d1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa934e53640 1 -- 172.21.15.37:0/3385649353 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200ab620 con 0x7fa9300cd470 2024-04-25T08:55:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa934e53640 1 -- 172.21.15.37:0/3385649353 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9200b0070 con 0x7fa9300cd470 2024-04-25T08:55:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa934e53640 1 -- 172.21.15.37:0/3385649353 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9200ab9e0 con 0x7fa9300cd470 2024-04-25T08:55:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 -- 172.21.15.37:0/3385649353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9300cd470 msgr2=0x7fa9300cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 --2- 172.21.15.37:0/3385649353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9300cd470 0x7fa9300cd8d0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fa920009870 tx=0x7fa92001c460 comp rx=0 tx=0).stop 2024-04-25T08:55:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 -- 172.21.15.37:0/3385649353 shutdown_connections 2024-04-25T08:55:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 --2- 172.21.15.37:0/3385649353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9300cd470 0x7fa9300cd8d0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 --2- 172.21.15.37:0/3385649353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa93015b8e0 0x7fa93006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 -- 172.21.15.37:0/3385649353 >> 172.21.15.37:0/3385649353 conn(0x7fa9300b1fa0 msgr2=0x7fa9300b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 -- 172.21.15.37:0/3385649353 shutdown_connections 2024-04-25T08:55:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 -- 172.21.15.37:0/3385649353 wait complete. 2024-04-25T08:55:44.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 Processor -- start 2024-04-25T08:55:44.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 -- start start 2024-04-25T08:55:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9300ad280 0x7fa9300b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9300cd470 0x7fa93006f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa93015b8e0 0x7fa930154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300cf200 con 0x7fa9300ad280 2024-04-25T08:55:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300ccf50 con 0x7fa93015b8e0 2024-04-25T08:55:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa9380b9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9300cf380 con 0x7fa9300cd470 2024-04-25T08:55:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.278+0000 7fa935e55640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9300ad280 0x7fa9300b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa935e55640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9300ad280 0x7fa9300b3e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40650/0 (socket says 172.21.15.37:40650) 2024-04-25T08:55:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa935e55640 1 -- 172.21.15.37:0/4034887601 learned_addr learned my addr 172.21.15.37:0/4034887601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa935654640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9300cd470 0x7fa93006f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa936656640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa93015b8e0 0x7fa930154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa935e55640 1 -- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9300cd470 msgr2=0x7fa93006f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa935e55640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9300cd470 0x7fa93006f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa935e55640 1 -- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa93015b8e0 msgr2=0x7fa930154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa935e55640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa93015b8e0 0x7fa930154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa935e55640 1 -- 172.21.15.37:0/4034887601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9300b4380 con 0x7fa9300ad280 2024-04-25T08:55:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa935e55640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9300ad280 0x7fa9300b3e40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa92800ce00 tx=0x7fa92800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa9267fc640 1 -- 172.21.15.37:0/4034887601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92800f890 con 0x7fa9300ad280 2024-04-25T08:55:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa9267fc640 1 -- 172.21.15.37:0/4034887601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92800fa30 con 0x7fa9300ad280 2024-04-25T08:55:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa9380b9640 1 -- 172.21.15.37:0/4034887601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9300b5e00 con 0x7fa9300ad280 2024-04-25T08:55:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa9267fc640 1 -- 172.21.15.37:0/4034887601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92800fe40 con 0x7fa9300ad280 2024-04-25T08:55:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa9380b9640 1 -- 172.21.15.37:0/4034887601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9300b5320 con 0x7fa9300ad280 2024-04-25T08:55:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa9380b9640 1 -- 172.21.15.37:0/4034887601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa930067c30 con 0x7fa9300ad280 2024-04-25T08:55:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa9267fc640 1 -- 172.21.15.37:0/4034887601 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa928018400 con 0x7fa9300ad280 2024-04-25T08:55:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa9267fc640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa914066f60 0x7fa914069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa9267fc640 1 -- 172.21.15.37:0/4034887601 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa928085c30 con 0x7fa9300ad280 2024-04-25T08:55:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa935654640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa914066f60 0x7fa914069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.282+0000 7fa935654640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa914066f60 0x7fa914069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa9200af000 tx=0x7fa9200a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.286+0000 7fa9267fc640 1 -- 172.21.15.37:0/4034887601 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa928014030 con 0x7fa9300ad280 2024-04-25T08:55:44.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.450+0000 7fa9380b9640 1 -- 172.21.15.37:0/4034887601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa9300ad970 con 0x7fa9300ad280 2024-04-25T08:55:44.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.814+0000 7fa9267fc640 1 -- 172.21.15.37:0/4034887601 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa92800ae20 con 0x7fa9300ad280 2024-04-25T08:55:44.806 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T08:55:44.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 -- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa914066f60 msgr2=0x7fa914069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:44.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa914066f60 0x7fa914069420 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa9200af000 tx=0x7fa9200a7000 comp rx=0 tx=0).stop 2024-04-25T08:55:44.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 -- 172.21.15.37:0/4034887601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9300ad280 msgr2=0x7fa9300b3e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:44.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9300ad280 0x7fa9300b3e40 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa92800ce00 tx=0x7fa92800a650 comp rx=0 tx=0).stop 2024-04-25T08:55:44.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 -- 172.21.15.37:0/4034887601 shutdown_connections 2024-04-25T08:55:44.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa914066f60 0x7fa914069420 secure :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa9200af000 tx=0x7fa9200a7000 comp rx=0 tx=0).stop 2024-04-25T08:55:44.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa93015b8e0 0x7fa930154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9300cd470 0x7fa93006f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 --2- 172.21.15.37:0/4034887601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9300ad280 0x7fa9300b3e40 secure :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa92800ce00 tx=0x7fa92800a650 comp rx=0 tx=0).stop 2024-04-25T08:55:44.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 -- 172.21.15.37:0/4034887601 >> 172.21.15.37:0/4034887601 conn(0x7fa9300b1fa0 msgr2=0x7fa9300b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:44.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 -- 172.21.15.37:0/4034887601 shutdown_connections 2024-04-25T08:55:44.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.818+0000 7fa9380b9640 1 -- 172.21.15.37:0/4034887601 wait complete. 2024-04-25T08:55:44.835 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:55:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a4a9e4640 1 Processor -- start 2024-04-25T08:55:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a4a9e4640 1 -- start start 2024-04-25T08:55:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a4a9e4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415b6b0 0x7f1a440ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a4a9e4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1a44073010 0x7f1a440cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a4a9e4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a440cbd20 con 0x7f1a440cf590 2024-04-25T08:55:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a4a9e4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4415ba90 con 0x7f1a44073010 2024-04-25T08:55:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a4a9e4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440cff30 con 0x7f1a4415b6b0 2024-04-25T08:55:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a437fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1a440cf590 0x7f1a4415f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58536/0 (socket says 172.21.15.37:58536) 2024-04-25T08:55:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a437fe640 1 -- 172.21.15.37:0/4053569071 learned_addr learned my addr 172.21.15.37:0/4053569071 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a43fff640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415b6b0 0x7f1a440ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a48f81640 1 --2- 172.21.15.37:0/4053569071 >> v2:172.21.15.203:3300/0 conn(0x7f1a44073010 0x7f1a440cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332292807 0 0) 0x7f1a440cbd20 con 0x7f1a440cf590 2024-04-25T08:55:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.982+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a30003530 con 0x7f1a440cf590 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3041503713 0 0) 0x7f1a30003530 con 0x7f1a440cf590 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415b6b0 msgr2=0x7f1a440ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415b6b0 0x7f1a440ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 >> v2:172.21.15.203:3300/0 conn(0x7f1a44073010 msgr2=0x7f1a440cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 --2- 172.21.15.37:0/4053569071 >> v2:172.21.15.203:3300/0 conn(0x7f1a44073010 0x7f1a440cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a440cc000 con 0x7f1a440cf590 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1a34001680 con 0x7f1a440cf590 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 >> v1:172.21.15.37:6789/0 conn(0x7f1a440cf590 legacy=0x7f1a4415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30003a30 0x7f1a30003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415b6b0 0x7f1a30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30004d50 0x7f1a300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4415ba90 con 0x7f1a30004d50 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3000b9d0 con 0x7f1a30003a30 2024-04-25T08:55:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3000bb50 con 0x7f1a4415b6b0 2024-04-25T08:55:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a48f81640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30004d50 0x7f1a300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a43fff640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30003a30 0x7f1a30003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a437fe640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415b6b0 0x7f1a30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a48f81640 1 -- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415b6b0 msgr2=0x7f1a30004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a48f81640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415b6b0 0x7f1a30004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a48f81640 1 -- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30003a30 msgr2=0x7f1a30003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a48f81640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30003a30 0x7f1a30003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a48f81640 1 -- 172.21.15.37:0/4053569071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a30003530 con 0x7f1a30004d50 2024-04-25T08:55:44.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a43fff640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30003a30 0x7f1a30003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:44.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a48f81640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30004d50 0x7f1a300051d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009f90 tx=0x7f1a3c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2bfdd37d9ad6553 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:44.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0b2070 con 0x7f1a30004d50 2024-04-25T08:55:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c0a5c90 con 0x7f1a30004d50 2024-04-25T08:55:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a42ffd640 1 -- 172.21.15.37:0/4053569071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c009c10 con 0x7f1a30004d50 2024-04-25T08:55:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/4053569071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30004d50 msgr2=0x7f1a300051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a4a9e4640 1 --2- 172.21.15.37:0/4053569071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a30004d50 0x7f1a300051d0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009f90 tx=0x7f1a3c01cc80 comp rx=0 tx=0).stop 2024-04-25T08:55:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a48f81640 1 -- 172.21.15.37:0/4053569071 reap_dead start 2024-04-25T08:55:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/4053569071 shutdown_connections 2024-04-25T08:55:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/4053569071 >> 172.21.15.37:0/4053569071 conn(0x7f1a4406ae60 msgr2=0x7f1a4406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/4053569071 shutdown_connections 2024-04-25T08:55:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.986+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/4053569071 wait complete. 2024-04-25T08:55:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a4a9e4640 1 Processor -- start 2024-04-25T08:55:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a4a9e4640 1 -- start start 2024-04-25T08:55:44.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a4a9e4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30004d50 0x7f1a44152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a4a9e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a44152a90 0x7f1a4415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a4a9e4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415e7b0 0x7f1a4415c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a4a9e4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440cc8a0 con 0x7f1a44152a90 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a4a9e4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440cc5c0 con 0x7f1a30004d50 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a4a9e4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a440cba30 con 0x7f1a4415e7b0 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a44152a90 0x7f1a4415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a44152a90 0x7f1a4415bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40662/0 (socket says 172.21.15.37:40662) 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a437fe640 1 -- 172.21.15.37:0/1858674283 learned_addr learned my addr 172.21.15.37:0/1858674283 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a43fff640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30004d50 0x7f1a44152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a48f81640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415e7b0 0x7f1a4415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a437fe640 1 -- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415e7b0 msgr2=0x7f1a4415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a437fe640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415e7b0 0x7f1a4415c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a437fe640 1 -- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30004d50 msgr2=0x7f1a44152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a437fe640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30004d50 0x7f1a44152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a437fe640 1 -- 172.21.15.37:0/1858674283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a440ae0b0 con 0x7f1a44152a90 2024-04-25T08:55:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a48f81640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415e7b0 0x7f1a4415c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a43fff640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30004d50 0x7f1a44152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a437fe640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a44152a90 0x7f1a4415bc10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400e850 tx=0x7f1a3400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a40ff9640 1 -- 172.21.15.37:0/1858674283 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400c870 con 0x7f1a44152a90 2024-04-25T08:55:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a40ff9640 1 -- 172.21.15.37:0/1858674283 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a34018070 con 0x7f1a44152a90 2024-04-25T08:55:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/1858674283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a440ad530 con 0x7f1a44152a90 2024-04-25T08:55:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a40ff9640 1 -- 172.21.15.37:0/1858674283 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3400cc50 con 0x7f1a44152a90 2024-04-25T08:55:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/1858674283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a440adaf0 con 0x7f1a44152a90 2024-04-25T08:55:44.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.990+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/1858674283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4404efa0 con 0x7f1a44152a90 2024-04-25T08:55:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.994+0000 7f1a40ff9640 1 -- 172.21.15.37:0/1858674283 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1a34026430 con 0x7f1a44152a90 2024-04-25T08:55:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.994+0000 7f1a40ff9640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a24066f90 0x7f1a24069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.994+0000 7f1a40ff9640 1 -- 172.21.15.37:0/1858674283 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f1a34085680 con 0x7f1a44152a90 2024-04-25T08:55:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.994+0000 7f1a43fff640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a24066f90 0x7f1a24069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:44.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.994+0000 7f1a43fff640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a24066f90 0x7f1a24069450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1a440b2d80 tx=0x7f1a2c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:44.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:44.998+0000 7f1a40ff9640 1 -- 172.21.15.37:0/1858674283 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1a34056740 con 0x7f1a44152a90 2024-04-25T08:55:45.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.158+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/1858674283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1a440b3660 con 0x7f1a44152a90 2024-04-25T08:55:45.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.818+0000 7f1a40ff9640 1 -- 172.21.15.37:0/1858674283 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1a34056040 con 0x7f1a44152a90 2024-04-25T08:55:45.809 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T08:55:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a24066f90 msgr2=0x7f1a24069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a24066f90 0x7f1a24069450 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1a440b2d80 tx=0x7f1a2c008040 comp rx=0 tx=0).stop 2024-04-25T08:55:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/1858674283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a44152a90 msgr2=0x7f1a4415bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a44152a90 0x7f1a4415bc10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1a3400e850 tx=0x7f1a3400ed10 comp rx=0 tx=0).stop 2024-04-25T08:55:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/1858674283 shutdown_connections 2024-04-25T08:55:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1a24066f90 0x7f1a24069450 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1a4415e7b0 0x7f1a4415c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a44152a90 0x7f1a4415bc10 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 --2- 172.21.15.37:0/1858674283 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1a30004d50 0x7f1a44152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/1858674283 >> 172.21.15.37:0/1858674283 conn(0x7f1a4406ae60 msgr2=0x7f1a440af390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:45.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/1858674283 shutdown_connections 2024-04-25T08:55:45.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.822+0000 7f1a4a9e4640 1 -- 172.21.15.37:0/1858674283 wait complete. 2024-04-25T08:55:45.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:55:45.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2ded592640 1 Processor -- start 2024-04-25T08:55:45.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2ded592640 1 -- start start 2024-04-25T08:55:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2ded592640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2de815ecb0 0x7f2de80c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2ded592640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2de80c9ef0 0x7f2de80ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2ded592640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de80ca4c0 con 0x7f2de80cf590 2024-04-25T08:55:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2ded592640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de80ca780 con 0x7f2de80c9ef0 2024-04-25T08:55:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2ded592640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de815f0f0 con 0x7f2de815ecb0 2024-04-25T08:55:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2de77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2de80c9ef0 0x7f2de80ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2de77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2de80c9ef0 0x7f2de80ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40670/0 (socket says 172.21.15.37:40670) 2024-04-25T08:55:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2de77fe640 1 -- 172.21.15.37:0/2126267861 learned_addr learned my addr 172.21.15.37:0/2126267861 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2de67fc640 1 --2- 172.21.15.37:0/2126267861 >> v2:172.21.15.203:3300/0 conn(0x7f2de815ecb0 0x7f2de80c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2de77fe640 1 -- 172.21.15.37:0/2126267861 >> v2:172.21.15.203:3300/0 conn(0x7f2de815ecb0 msgr2=0x7f2de80c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2de77fe640 1 --2- 172.21.15.37:0/2126267861 >> v2:172.21.15.203:3300/0 conn(0x7f2de815ecb0 0x7f2de80c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2de77fe640 1 -- 172.21.15.37:0/2126267861 >> v1:172.21.15.203:6789/0 conn(0x7f2de80cf590 legacy=0x7f2de80cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.986+0000 7f2de77fe640 1 -- 172.21.15.37:0/2126267861 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de80cd690 con 0x7f2de80c9ef0 2024-04-25T08:55:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2de77fe640 1 --2- 172.21.15.37:0/2126267861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2de80c9ef0 0x7f2de80ce170 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2de80d00e0 tx=0x7f2de001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cff31c0184f9bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2de5ffb640 1 -- 172.21.15.37:0/2126267861 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de00a5670 con 0x7f2de80c9ef0 2024-04-25T08:55:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2de5ffb640 1 -- 172.21.15.37:0/2126267861 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de00a5810 con 0x7f2de80c9ef0 2024-04-25T08:55:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2de5ffb640 1 -- 172.21.15.37:0/2126267861 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de00a5be0 con 0x7f2de80c9ef0 2024-04-25T08:55:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 -- 172.21.15.37:0/2126267861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2de80c9ef0 msgr2=0x7f2de80ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 --2- 172.21.15.37:0/2126267861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2de80c9ef0 0x7f2de80ce170 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2de80d00e0 tx=0x7f2de001c9b0 comp rx=0 tx=0).stop 2024-04-25T08:55:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 -- 172.21.15.37:0/2126267861 shutdown_connections 2024-04-25T08:55:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 --2- 172.21.15.37:0/2126267861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2de80c9ef0 0x7f2de80ce170 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:45.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 --2- 172.21.15.37:0/2126267861 >> v2:172.21.15.203:3300/0 conn(0x7f2de815ecb0 0x7f2de80c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:45.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 -- 172.21.15.37:0/2126267861 >> 172.21.15.37:0/2126267861 conn(0x7f2de806ae60 msgr2=0x7f2de806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:45.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 -- 172.21.15.37:0/2126267861 shutdown_connections 2024-04-25T08:55:45.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 -- 172.21.15.37:0/2126267861 wait complete. 2024-04-25T08:55:45.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 Processor -- start 2024-04-25T08:55:45.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 -- start start 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2de80c9ef0 0x7f2de8153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2de80cf590 0x7f2de81f4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2de815ecb0 0x7f2de81f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de8167870 con 0x7f2de80cf590 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de81676f0 con 0x7f2de80c9ef0 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2ded592640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de81679f0 con 0x7f2de815ecb0 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2de67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2de80cf590 0x7f2de81f4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2de67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2de80cf590 0x7f2de81f4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40684/0 (socket says 172.21.15.37:40684) 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.990+0000 7f2de67fc640 1 -- 172.21.15.37:0/1921164375 learned_addr learned my addr 172.21.15.37:0/1921164375 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2de77fe640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2de815ecb0 0x7f2de81f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2de6ffd640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2de80c9ef0 0x7f2de8153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:45.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2de67fc640 1 -- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2de815ecb0 msgr2=0x7f2de81f4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2de67fc640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2de815ecb0 0x7f2de81f4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2de67fc640 1 -- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2de80c9ef0 msgr2=0x7f2de8153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2de67fc640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2de80c9ef0 0x7f2de8153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2de67fc640 1 -- 172.21.15.37:0/1921164375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de81f5ad0 con 0x7f2de80cf590 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2de6ffd640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2de80c9ef0 0x7f2de8153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2de67fc640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2de80cf590 0x7f2de81f4970 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000c7f0 tx=0x7f2dd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2ddf7fe640 1 -- 172.21.15.37:0/1921164375 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0015070 con 0x7f2de80cf590 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2ddf7fe640 1 -- 172.21.15.37:0/1921164375 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd0004240 con 0x7f2de80cf590 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2ded592640 1 -- 172.21.15.37:0/1921164375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de8153c70 con 0x7f2de80cf590 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2ddf7fe640 1 -- 172.21.15.37:0/1921164375 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0003620 con 0x7f2de80cf590 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2ded592640 1 -- 172.21.15.37:0/1921164375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de80ae750 con 0x7f2de80cf590 2024-04-25T08:55:45.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2ded592640 1 -- 172.21.15.37:0/1921164375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de804efa0 con 0x7f2de80cf590 2024-04-25T08:55:45.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2ddf7fe640 1 -- 172.21.15.37:0/1921164375 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2dd00039d0 con 0x7f2de80cf590 2024-04-25T08:55:45.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2ddf7fe640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2dc8066f60 0x7f2dc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:45.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2ddf7fe640 1 -- 172.21.15.37:0/1921164375 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f2dd0085510 con 0x7f2de80cf590 2024-04-25T08:55:45.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.994+0000 7f2de6ffd640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2dc8066f60 0x7f2dc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:45.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.998+0000 7f2de6ffd640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2dc8066f60 0x7f2dc8069420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800a830 tx=0x7f2dd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:45.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:45.998+0000 7f2ddf7fe640 1 -- 172.21.15.37:0/1921164375 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2dd0056560 con 0x7f2de80cf590 2024-04-25T08:55:46.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.194+0000 7f2ded592640 1 -- 172.21.15.37:0/1921164375 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f2de80b4170 con 0x7f2de80cf590 2024-04-25T08:55:46.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.842+0000 7f2ddf7fe640 1 -- 172.21.15.37:0/1921164375 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v29) v1 ==== 173+0+0 (secure 0 0 0) 0x7f2dd0055e60 con 0x7f2de80cf590 2024-04-25T08:55:46.832 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T08:55:46.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 -- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2dc8066f60 msgr2=0x7f2dc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:46.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2dc8066f60 0x7f2dc8069420 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2dd800a830 tx=0x7f2dd8008040 comp rx=0 tx=0).stop 2024-04-25T08:55:46.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 -- 172.21.15.37:0/1921164375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2de80cf590 msgr2=0x7f2de81f4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:46.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2de80cf590 0x7f2de81f4970 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f2dd000c7f0 tx=0x7f2dd000ccb0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 -- 172.21.15.37:0/1921164375 shutdown_connections 2024-04-25T08:55:46.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2dc8066f60 0x7f2dc8069420 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2de815ecb0 0x7f2de81f4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2de80cf590 0x7f2de81f4970 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 --2- 172.21.15.37:0/1921164375 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2de80c9ef0 0x7f2de8153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 -- 172.21.15.37:0/1921164375 >> 172.21.15.37:0/1921164375 conn(0x7f2de806ae60 msgr2=0x7f2de8155010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:46.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 -- 172.21.15.37:0/1921164375 shutdown_connections 2024-04-25T08:55:46.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.846+0000 7f2ded592640 1 -- 172.21.15.37:0/1921164375 wait complete. 2024-04-25T08:55:46.861 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:46.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.990+0000 7f80bf662640 1 Processor -- start 2024-04-25T08:55:46.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.990+0000 7f80bf662640 1 -- start start 2024-04-25T08:55:46.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bf662640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f80b80b0a30 0x7f80b80cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:46.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bf662640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b806f4a0 0x7f80b8099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:46.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bf662640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b80b0e10 con 0x7f80b8099fb0 2024-04-25T08:55:46.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bf662640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b80d0d90 con 0x7f80b80b0a30 2024-04-25T08:55:46.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bf662640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80d0180 con 0x7f80b806f4a0 2024-04-25T08:55:46.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bdbff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f80b8099fb0 0x7f80b815d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58550/0 (socket says 172.21.15.37:58550) 2024-04-25T08:55:46.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bdbff640 1 -- 172.21.15.37:0/2986245020 learned_addr learned my addr 172.21.15.37:0/2986245020 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:46.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bcbfd640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b806f4a0 0x7f80b8099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:46.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bd3fe640 1 --2- 172.21.15.37:0/2986245020 >> v2:172.21.15.203:3300/0 conn(0x7f80b80b0a30 0x7f80b80cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:46.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335049929 0 0) 0x7f80b80b0e10 con 0x7f80b8099fb0 2024-04-25T08:55:46.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80a4003530 con 0x7f80b8099fb0 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 773008231 0 0) 0x7f80a4003530 con 0x7f80b8099fb0 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b806f4a0 msgr2=0x7f80b8099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b806f4a0 0x7f80b8099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 >> v2:172.21.15.203:3300/0 conn(0x7f80b80b0a30 msgr2=0x7f80b80cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 --2- 172.21.15.37:0/2986245020 >> v2:172.21.15.203:3300/0 conn(0x7f80b80b0a30 0x7f80b80cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b80d0460 con 0x7f80b8099fb0 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f80b0001a00 con 0x7f80b8099fb0 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 >> v1:172.21.15.37:6789/0 conn(0x7f80b8099fb0 legacy=0x7f80b815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b806f4a0 0x7f80a4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a4003fb0 0x7f80a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a4004d50 0x7f80a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:46.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80d0d90 con 0x7f80a4004d50 2024-04-25T08:55:46.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80d0180 con 0x7f80a4003fb0 2024-04-25T08:55:46.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a400ba00 con 0x7f80b806f4a0 2024-04-25T08:55:46.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bdbff640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a4004d50 0x7f80a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:46.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bcbfd640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a4003fb0 0x7f80a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:46.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bd3fe640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b806f4a0 0x7f80a4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:46.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bdbff640 1 -- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b806f4a0 msgr2=0x7f80a4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:46.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bdbff640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b806f4a0 0x7f80a4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bdbff640 1 -- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a4003fb0 msgr2=0x7f80a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:46.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bdbff640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80a4003fb0 0x7f80a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bdbff640 1 -- 172.21.15.37:0/2986245020 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a4003530 con 0x7f80a4004d50 2024-04-25T08:55:46.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.994+0000 7f80bd3fe640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80b806f4a0 0x7f80a4003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:46.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bdbff640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a4004d50 0x7f80a40051d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f80b00013e0 tx=0x7f80b001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b2d6311d50cd60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:46.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b000a4e0 con 0x7f80a4004d50 2024-04-25T08:55:46.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b00072f0 con 0x7f80a4004d50 2024-04-25T08:55:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80affff640 1 -- 172.21.15.37:0/2986245020 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b00076e0 con 0x7f80a4004d50 2024-04-25T08:55:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 -- 172.21.15.37:0/2986245020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a4004d50 msgr2=0x7f80a40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 --2- 172.21.15.37:0/2986245020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80a4004d50 0x7f80a40051d0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f80b00013e0 tx=0x7f80b001d3a0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bdbff640 1 -- 172.21.15.37:0/2986245020 reap_dead start 2024-04-25T08:55:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 -- 172.21.15.37:0/2986245020 shutdown_connections 2024-04-25T08:55:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 -- 172.21.15.37:0/2986245020 >> 172.21.15.37:0/2986245020 conn(0x7f80b80b30b0 msgr2=0x7f80b80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 -- 172.21.15.37:0/2986245020 shutdown_connections 2024-04-25T08:55:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 -- 172.21.15.37:0/2986245020 wait complete. 2024-04-25T08:55:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 Processor -- start 2024-04-25T08:55:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 -- start start 2024-04-25T08:55:46.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80a4004d50 0x7f80b80b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b80c75e0 0x7f80b80c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b81f3a80 0x7f80b81f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80cfdd0 con 0x7f80b81f3a80 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b80d0a20 con 0x7f80b80c75e0 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bf662640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b81d15d0 con 0x7f80a4004d50 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bdbff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b81f3a80 0x7f80b81f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bdbff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b81f3a80 0x7f80b81f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40710/0 (socket says 172.21.15.37:40710) 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bdbff640 1 -- 172.21.15.37:0/1462024939 learned_addr learned my addr 172.21.15.37:0/1462024939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bd3fe640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80a4004d50 0x7f80b80b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:46.998+0000 7f80bcbfd640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b80c75e0 0x7f80b80c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80bdbff640 1 -- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80a4004d50 msgr2=0x7f80b80b1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:46.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80bdbff640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80a4004d50 0x7f80b80b1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80bdbff640 1 -- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b80c75e0 msgr2=0x7f80b80c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:46.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80bdbff640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b80c75e0 0x7f80b80c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:46.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80bdbff640 1 -- 172.21.15.37:0/1462024939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b81d1820 con 0x7f80b81f3a80 2024-04-25T08:55:46.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80bdbff640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b81f3a80 0x7f80b81f3f00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f80b0007040 tx=0x7f80a4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:46.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80adffb640 1 -- 172.21.15.37:0/1462024939 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b00b2040 con 0x7f80b81f3a80 2024-04-25T08:55:46.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80adffb640 1 -- 172.21.15.37:0/1462024939 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b000a4c0 con 0x7f80b81f3a80 2024-04-25T08:55:46.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80adffb640 1 -- 172.21.15.37:0/1462024939 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b0002050 con 0x7f80b81f3a80 2024-04-25T08:55:46.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80bf662640 1 -- 172.21.15.37:0/1462024939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b81d1b00 con 0x7f80b81f3a80 2024-04-25T08:55:46.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80bf662640 1 -- 172.21.15.37:0/1462024939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b80b2090 con 0x7f80b81f3a80 2024-04-25T08:55:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80bf662640 1 -- 172.21.15.37:0/1462024939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b804efa0 con 0x7f80b81f3a80 2024-04-25T08:55:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80adffb640 1 -- 172.21.15.37:0/1462024939 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80b00bd060 con 0x7f80b81f3a80 2024-04-25T08:55:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80adffb640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f809c066f90 0x7f809c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80adffb640 1 -- 172.21.15.37:0/1462024939 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80b011e9b0 con 0x7f80b81f3a80 2024-04-25T08:55:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80bd3fe640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f809c066f90 0x7f809c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:46.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.002+0000 7f80bd3fe640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f809c066f90 0x7f809c069450 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009b40 tx=0x7f80b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:46.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.006+0000 7f80adffb640 1 -- 172.21.15.37:0/1462024939 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f80b00170b0 con 0x7f80b81f3a80 2024-04-25T08:55:47.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.166+0000 7f80bf662640 1 -- 172.21.15.37:0/1462024939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80b806fa80 con 0x7f80b81f3a80 2024-04-25T08:55:47.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.170+0000 7f80adffb640 1 -- 172.21.15.37:0/1462024939 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16861 (secure 0 0 0) 0x7f80b00019c0 con 0x7f80b81f3a80 2024-04-25T08:55:47.161 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:47.161 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":68,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:55:46.199959+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:44.492629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:45.203787+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.170+0000 7f80bf662640 1 -- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f809c066f90 msgr2=0x7f809c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.170+0000 7f80bf662640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f809c066f90 0x7f809c069450 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f80b4009b40 tx=0x7f80b4008040 comp rx=0 tx=0).stop 2024-04-25T08:55:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.170+0000 7f80bf662640 1 -- 172.21.15.37:0/1462024939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b81f3a80 msgr2=0x7f80b81f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.170+0000 7f80bf662640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b81f3a80 0x7f80b81f3f00 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f80b0007040 tx=0x7f80a4005840 comp rx=0 tx=0).stop 2024-04-25T08:55:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.174+0000 7f80bf662640 1 -- 172.21.15.37:0/1462024939 shutdown_connections 2024-04-25T08:55:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.174+0000 7f80bf662640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f809c066f90 0x7f809c069450 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.174+0000 7f80bf662640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b81f3a80 0x7f80b81f3f00 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.174+0000 7f80bf662640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b80c75e0 0x7f80b80c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.174+0000 7f80bf662640 1 --2- 172.21.15.37:0/1462024939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80a4004d50 0x7f80b80b1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.174+0000 7f80bf662640 1 -- 172.21.15.37:0/1462024939 >> 172.21.15.37:0/1462024939 conn(0x7f80b80b30b0 msgr2=0x7f80b80703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.174+0000 7f80bf662640 1 -- 172.21.15.37:0/1462024939 shutdown_connections 2024-04-25T08:55:47.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.174+0000 7f80bf662640 1 -- 172.21.15.37:0/1462024939 wait complete. 2024-04-25T08:55:47.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:47.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.314+0000 7f5ea76e8640 1 Processor -- start 2024-04-25T08:55:47.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.314+0000 7f5ea76e8640 1 -- start start 2024-04-25T08:55:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea76e8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5ea015a700 0x7f5ea015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea76e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea00b3f50 0x7f5ea00cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea76e8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ea00b2090 con 0x7f5ea00ce430 2024-04-25T08:55:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea76e8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ea015f200 con 0x7f5ea015a700 2024-04-25T08:55:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea76e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00b43e0 con 0x7f5ea00b3f50 2024-04-25T08:55:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea5484640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5ea015a700 0x7f5ea015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea5484640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5ea015a700 0x7f5ea015ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:55912/0 (socket says 172.21.15.37:55912) 2024-04-25T08:55:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea5484640 1 -- 172.21.15.37:0/783665110 learned_addr learned my addr 172.21.15.37:0/783665110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea4c83640 1 --2- 172.21.15.37:0/783665110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea00b3f50 0x7f5ea00cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea5484640 1 -- 172.21.15.37:0/783665110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea00b3f50 msgr2=0x7f5ea00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea5484640 1 --2- 172.21.15.37:0/783665110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea00b3f50 0x7f5ea00cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea5484640 1 -- 172.21.15.37:0/783665110 >> v1:172.21.15.203:6789/0 conn(0x7f5ea00ce430 legacy=0x7f5ea00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.318+0000 7f5ea5484640 1 -- 172.21.15.37:0/783665110 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea00b46c0 con 0x7f5ea015a700 2024-04-25T08:55:47.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea5484640 1 --2- 172.21.15.37:0/783665110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea015a700 0x7f5ea015ecc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009870 tx=0x7f5e9c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2acde59fbc40eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:47.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5e93fff640 1 -- 172.21.15.37:0/783665110 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0ab610 con 0x7f5ea015a700 2024-04-25T08:55:47.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5e93fff640 1 -- 172.21.15.37:0/783665110 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c0b0070 con 0x7f5ea015a700 2024-04-25T08:55:47.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5e93fff640 1 -- 172.21.15.37:0/783665110 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0ab9d0 con 0x7f5ea015a700 2024-04-25T08:55:47.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 -- 172.21.15.37:0/783665110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea015a700 msgr2=0x7f5ea015ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 --2- 172.21.15.37:0/783665110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea015a700 0x7f5ea015ecc0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c009870 tx=0x7f5e9c01c410 comp rx=0 tx=0).stop 2024-04-25T08:55:47.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 -- 172.21.15.37:0/783665110 shutdown_connections 2024-04-25T08:55:47.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 --2- 172.21.15.37:0/783665110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea00b3f50 0x7f5ea00cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 --2- 172.21.15.37:0/783665110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea015a700 0x7f5ea015ecc0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 -- 172.21.15.37:0/783665110 >> 172.21.15.37:0/783665110 conn(0x7f5ea00b11a0 msgr2=0x7f5ea00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:47.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 -- 172.21.15.37:0/783665110 shutdown_connections 2024-04-25T08:55:47.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 -- 172.21.15.37:0/783665110 wait complete. 2024-04-25T08:55:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 Processor -- start 2024-04-25T08:55:47.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 -- start start 2024-04-25T08:55:47.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea00ce430 0x7f5ea013ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea015a700 0x7f5ea014cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea01681d0 0x7f5ea014e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00cd490 con 0x7f5ea00ce430 2024-04-25T08:55:47.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00cd310 con 0x7f5ea015a700 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea76e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ea00b2bb0 con 0x7f5ea01681d0 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea5484640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea00ce430 0x7f5ea013ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea5484640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea00ce430 0x7f5ea013ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40722/0 (socket says 172.21.15.37:40722) 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea5484640 1 -- 172.21.15.37:0/2483628287 learned_addr learned my addr 172.21.15.37:0/2483628287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea4c83640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea015a700 0x7f5ea014cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea5c85640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea01681d0 0x7f5ea014e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea5484640 1 -- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea01681d0 msgr2=0x7f5ea014e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea5484640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea01681d0 0x7f5ea014e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea5484640 1 -- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea015a700 msgr2=0x7f5ea014cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea5484640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea015a700 0x7f5ea014cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea5484640 1 -- 172.21.15.37:0/2483628287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea014d840 con 0x7f5ea00ce430 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea4c83640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea015a700 0x7f5ea014cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.322+0000 7f5ea5484640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea00ce430 0x7f5ea013ee90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0ad040 tx=0x7f5e9c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.326+0000 7f5e91ffb640 1 -- 172.21.15.37:0/2483628287 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0be5f0 con 0x7f5ea00ce430 2024-04-25T08:55:47.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.326+0000 7f5e91ffb640 1 -- 172.21.15.37:0/2483628287 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c0be7b0 con 0x7f5ea00ce430 2024-04-25T08:55:47.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.326+0000 7f5ea76e8640 1 -- 172.21.15.37:0/2483628287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ea014db20 con 0x7f5ea00ce430 2024-04-25T08:55:47.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.326+0000 7f5e91ffb640 1 -- 172.21.15.37:0/2483628287 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0beba0 con 0x7f5ea00ce430 2024-04-25T08:55:47.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.326+0000 7f5ea76e8640 1 -- 172.21.15.37:0/2483628287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ea014e0e0 con 0x7f5ea00ce430 2024-04-25T08:55:47.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.326+0000 7f5ea76e8640 1 -- 172.21.15.37:0/2483628287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ea004efa0 con 0x7f5ea00ce430 2024-04-25T08:55:47.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.326+0000 7f5e91ffb640 1 -- 172.21.15.37:0/2483628287 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e9c0c7440 con 0x7f5ea00ce430 2024-04-25T08:55:47.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.326+0000 7f5e91ffb640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e84066f60 0x7f5e84069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.326+0000 7f5e91ffb640 1 -- 172.21.15.37:0/2483628287 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e9c1204d0 con 0x7f5ea00ce430 2024-04-25T08:55:47.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.326+0000 7f5ea4c83640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e84066f60 0x7f5e84069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.326+0000 7f5ea4c83640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e84066f60 0x7f5e84069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0167780 tx=0x7f5e94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:47.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.330+0000 7f5e91ffb640 1 -- 172.21.15.37:0/2483628287 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e9c0cc020 con 0x7f5ea00ce430 2024-04-25T08:55:47.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.490+0000 7f5ea76e8640 1 -- 172.21.15.37:0/2483628287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ea006f260 con 0x7f5ea00ce430 2024-04-25T08:55:47.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.494+0000 7f5e91ffb640 1 -- 172.21.15.37:0/2483628287 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16861 (secure 0 0 0) 0x7f5e9c11d1f0 con 0x7f5ea00ce430 2024-04-25T08:55:47.485 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:47.485 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":68,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:55:46.199959+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:44.492629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:45.203787+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:47.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 -- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e84066f60 msgr2=0x7f5e84069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e84066f60 0x7f5e84069420 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0167780 tx=0x7f5e94008040 comp rx=0 tx=0).stop 2024-04-25T08:55:47.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 -- 172.21.15.37:0/2483628287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea00ce430 msgr2=0x7f5ea013ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea00ce430 0x7f5ea013ee90 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c0ad040 tx=0x7f5e9c01ac50 comp rx=0 tx=0).stop 2024-04-25T08:55:47.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 -- 172.21.15.37:0/2483628287 shutdown_connections 2024-04-25T08:55:47.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e84066f60 0x7f5e84069420 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ea01681d0 0x7f5ea014e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ea015a700 0x7f5ea014cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 --2- 172.21.15.37:0/2483628287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ea00ce430 0x7f5ea013ee90 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 -- 172.21.15.37:0/2483628287 >> 172.21.15.37:0/2483628287 conn(0x7f5ea00b11a0 msgr2=0x7f5ea006d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 -- 172.21.15.37:0/2483628287 shutdown_connections 2024-04-25T08:55:47.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.498+0000 7f5ea76e8640 1 -- 172.21.15.37:0/2483628287 wait complete. 2024-04-25T08:55:47.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:55:47.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f6669182640 1 Processor -- start 2024-04-25T08:55:47.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f6669182640 1 -- start start 2024-04-25T08:55:47.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f6669182640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66640b5270 0x7f66640b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f6669182640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f666406d090 0x7f666406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f6669182640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666406bff0 con 0x7f66640b5650 2024-04-25T08:55:47.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f6669182640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666406cc00 con 0x7f666406d090 2024-04-25T08:55:47.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f6669182640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666406c2d0 con 0x7f66640b5270 2024-04-25T08:55:47.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f666359e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f666406d090 0x7f666406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f666259c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f66640b5650 0x7f66640d3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58560/0 (socket says 172.21.15.37:58560) 2024-04-25T08:55:47.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f666259c640 1 -- 172.21.15.37:0/2851331891 learned_addr learned my addr 172.21.15.37:0/2851331891 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:47.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f6662d9d640 1 --2- 172.21.15.37:0/2851331891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66640b5270 0x7f66640b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f666359e640 1 -- 172.21.15.37:0/2851331891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66640b5270 msgr2=0x7f66640b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:47.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f666359e640 1 --2- 172.21.15.37:0/2851331891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66640b5270 0x7f66640b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f666359e640 1 -- 172.21.15.37:0/2851331891 >> v1:172.21.15.37:6789/0 conn(0x7f66640b5650 legacy=0x7f66640d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f666359e640 1 -- 172.21.15.37:0/2851331891 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666406c5b0 con 0x7f666406d090 2024-04-25T08:55:47.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.650+0000 7f6661d9b640 1 -- 172.21.15.37:0/2851331891 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582618878 0 0) 0x7f666406bff0 con 0x7f66640b5650 2024-04-25T08:55:47.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f666359e640 1 --2- 172.21.15.37:0/2851331891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f666406d090 0x7f666406d4f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f66540097c0 tx=0x7f665401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd988e75df025fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:47.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6661d9b640 1 -- 172.21.15.37:0/2851331891 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66540b1070 con 0x7f666406d090 2024-04-25T08:55:47.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6661d9b640 1 -- 172.21.15.37:0/2851331891 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66540ac410 con 0x7f666406d090 2024-04-25T08:55:47.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6661d9b640 1 -- 172.21.15.37:0/2851331891 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66540ac7e0 con 0x7f666406d090 2024-04-25T08:55:47.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 -- 172.21.15.37:0/2851331891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f666406d090 msgr2=0x7f666406d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 --2- 172.21.15.37:0/2851331891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f666406d090 0x7f666406d4f0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f66540097c0 tx=0x7f665401c4a0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 -- 172.21.15.37:0/2851331891 shutdown_connections 2024-04-25T08:55:47.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 --2- 172.21.15.37:0/2851331891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f666406d090 0x7f666406d4f0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 --2- 172.21.15.37:0/2851331891 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66640b5270 0x7f66640b0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 -- 172.21.15.37:0/2851331891 >> 172.21.15.37:0/2851331891 conn(0x7f666408b000 msgr2=0x7f66640d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:47.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 -- 172.21.15.37:0/2851331891 shutdown_connections 2024-04-25T08:55:47.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 -- 172.21.15.37:0/2851331891 wait complete. 2024-04-25T08:55:47.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 Processor -- start 2024-04-25T08:55:47.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 -- start start 2024-04-25T08:55:47.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f666406d090 0x7f66640b90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66640b5270 0x7f66640c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66640b5650 0x7f6664155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66640b5bd0 con 0x7f66640b5650 2024-04-25T08:55:47.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66640b5a50 con 0x7f666406d090 2024-04-25T08:55:47.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6669182640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666415de20 con 0x7f66640b5270 2024-04-25T08:55:47.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f666359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66640b5650 0x7f6664155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f666359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66640b5650 0x7f6664155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40734/0 (socket says 172.21.15.37:40734) 2024-04-25T08:55:47.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f666359e640 1 -- 172.21.15.37:0/680071547 learned_addr learned my addr 172.21.15.37:0/680071547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:47.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f666259c640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66640b5270 0x7f66640c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f6662d9d640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f666406d090 0x7f66640b90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f666359e640 1 -- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66640b5270 msgr2=0x7f66640c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f666359e640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66640b5270 0x7f66640c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f666359e640 1 -- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f666406d090 msgr2=0x7f66640b90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f666359e640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f666406d090 0x7f66640b90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.654+0000 7f666359e640 1 -- 172.21.15.37:0/680071547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66640b62f0 con 0x7f66640b5650 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f666359e640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66640b5650 0x7f6664155880 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f66540097c0 tx=0x7f6654019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f664b7fe640 1 -- 172.21.15.37:0/680071547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66540b1040 con 0x7f66640b5650 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f664b7fe640 1 -- 172.21.15.37:0/680071547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66540acd70 con 0x7f66640b5650 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f6669182640 1 -- 172.21.15.37:0/680071547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66640b65d0 con 0x7f66640b5650 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f664b7fe640 1 -- 172.21.15.37:0/680071547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6654006e20 con 0x7f66640b5650 2024-04-25T08:55:47.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f6669182640 1 -- 172.21.15.37:0/680071547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66640cbd50 con 0x7f66640b5650 2024-04-25T08:55:47.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f6669182640 1 -- 172.21.15.37:0/680071547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666404efa0 con 0x7f66640b5650 2024-04-25T08:55:47.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f664b7fe640 1 -- 172.21.15.37:0/680071547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6654004250 con 0x7f66640b5650 2024-04-25T08:55:47.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f664b7fe640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6650066f60 0x7f6650069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:47.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f664b7fe640 1 -- 172.21.15.37:0/680071547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f665411ec00 con 0x7f66640b5650 2024-04-25T08:55:47.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f6662d9d640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6650066f60 0x7f6650069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:47.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.658+0000 7f6662d9d640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6650066f60 0x7f6650069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f665c006fd0 tx=0x7f665c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:47.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.662+0000 7f664b7fe640 1 -- 172.21.15.37:0/680071547 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6654125080 con 0x7f66640b5650 2024-04-25T08:55:47.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:47.858+0000 7f6669182640 1 -- 172.21.15.37:0/680071547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f66640c7070 con 0x7f66640b5650 2024-04-25T08:55:48.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.862+0000 7f664b7fe640 1 -- 172.21.15.37:0/680071547 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f66540b9020 con 0x7f66640b5650 2024-04-25T08:55:48.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 -- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6650066f60 msgr2=0x7f6650069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:48.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6650066f60 0x7f6650069420 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f665c006fd0 tx=0x7f665c008040 comp rx=0 tx=0).stop 2024-04-25T08:55:48.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 -- 172.21.15.37:0/680071547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66640b5650 msgr2=0x7f6664155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:48.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66640b5650 0x7f6664155880 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f66540097c0 tx=0x7f6654019240 comp rx=0 tx=0).stop 2024-04-25T08:55:48.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 -- 172.21.15.37:0/680071547 shutdown_connections 2024-04-25T08:55:48.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6650066f60 0x7f6650069420 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:48.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66640b5650 0x7f6664155880 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:48.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f66640b5270 0x7f66640c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:48.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 --2- 172.21.15.37:0/680071547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f666406d090 0x7f66640b90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:48.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 -- 172.21.15.37:0/680071547 >> 172.21.15.37:0/680071547 conn(0x7f666408b000 msgr2=0x7f66640b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:48.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 -- 172.21.15.37:0/680071547 shutdown_connections 2024-04-25T08:55:48.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:48.866+0000 7f6669182640 1 -- 172.21.15.37:0/680071547 wait complete. 2024-04-25T08:55:48.876 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:55:48.876 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:55:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa93151640 1 Processor -- start 2024-04-25T08:55:49.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa93151640 1 -- start start 2024-04-25T08:55:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa93151640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faa8c15d180 0x7faa8c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa93151640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8c06b710 con 0x7faa8c06fc50 2024-04-25T08:55:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa93151640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa8c1617b0 con 0x7faa8c0cf840 2024-04-25T08:55:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa93151640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa8c161560 con 0x7faa8c15d180 2024-04-25T08:55:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa90eed640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7faa8c06fc50 0x7faa8c06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58576/0 (socket says 172.21.15.37:58576) 2024-04-25T08:55:49.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa90eed640 1 -- 172.21.15.37:0/1110423784 learned_addr learned my addr 172.21.15.37:0/1110423784 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723602867 0 0) 0x7faa8c06b710 con 0x7faa8c06fc50 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa74003530 con 0x7faa8c06fc50 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa916ee640 1 --2- 172.21.15.37:0/1110423784 >> v2:172.21.15.203:3300/0 conn(0x7faa8c15d180 0x7faa8c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2803833369 0 0) 0x7faa74003530 con 0x7faa8c06fc50 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 >> v2:172.21.15.203:3300/0 conn(0x7faa8c15d180 msgr2=0x7faa8c15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 --2- 172.21.15.37:0/1110423784 >> v2:172.21.15.203:3300/0 conn(0x7faa8c15d180 0x7faa8c15d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 >> v1:172.21.15.203:6789/0 conn(0x7faa8c0cf840 legacy=0x7faa8c0cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c161a90 con 0x7faa8c06fc50 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7faa7c001680 con 0x7faa8c06fc50 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 >> v1:172.21.15.37:6789/0 conn(0x7faa8c06fc50 legacy=0x7faa8c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa74003a30 0x7faa74003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74004810 0x7faa74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 0x7faa7400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:49.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c161560 con 0x7faa740051d0 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7400c170 con 0x7faa74004810 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7400c370 con 0x7faa74003a30 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa916ee640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 0x7faa7400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa90eed640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa74003a30 0x7faa74003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8bfff640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74004810 0x7faa74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa916ee640 1 -- 172.21.15.37:0/1110423784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa74003a30 msgr2=0x7faa74003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa916ee640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa74003a30 0x7faa74003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa916ee640 1 -- 172.21.15.37:0/1110423784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74004810 msgr2=0x7faa74004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa916ee640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74004810 0x7faa74004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa916ee640 1 -- 172.21.15.37:0/1110423784 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa74003530 con 0x7faa740051d0 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa90eed640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa74003a30 0x7faa74003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:49.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.018+0000 7faa8bfff640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa74004810 0x7faa74004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:49.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa916ee640 1 -- 172.21.15.37:0/1110423784 reap_dead start 2024-04-25T08:55:49.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa916ee640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 0x7faa7400b9d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7faa84009fc0 tx=0x7faa8401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d218691738eb046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:49.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84009c10 con 0x7faa740051d0 2024-04-25T08:55:49.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa840a5cd0 con 0x7faa740051d0 2024-04-25T08:55:49.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa8b7fe640 1 -- 172.21.15.37:0/1110423784 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa84003be0 con 0x7faa740051d0 2024-04-25T08:55:49.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa93151640 1 -- 172.21.15.37:0/1110423784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 msgr2=0x7faa7400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:49.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa93151640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 0x7faa7400b9d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7faa84009fc0 tx=0x7faa8401ccc0 comp rx=0 tx=0).stop 2024-04-25T08:55:49.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa93151640 1 -- 172.21.15.37:0/1110423784 shutdown_connections 2024-04-25T08:55:49.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa93151640 1 --2- 172.21.15.37:0/1110423784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 0x7faa7400b9d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:49.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa93151640 1 -- 172.21.15.37:0/1110423784 >> 172.21.15.37:0/1110423784 conn(0x7faa8c099a70 msgr2=0x7faa8c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:49.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa93151640 1 -- 172.21.15.37:0/1110423784 shutdown_connections 2024-04-25T08:55:49.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa93151640 1 -- 172.21.15.37:0/1110423784 wait complete. 2024-04-25T08:55:49.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.022+0000 7faa93151640 1 Processor -- start 2024-04-25T08:55:49.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa93151640 1 -- start start 2024-04-25T08:55:49.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa93151640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 0x7faa8c0c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:49.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa93151640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8c0c4f30 0x7faa8c15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa93151640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa8c15a9a0 0x7faa8c15bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa93151640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c070030 con 0x7faa740051d0 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa93151640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c0d0e30 con 0x7faa8c15a9a0 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa93151640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8c0701b0 con 0x7faa8c0c4f30 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa90eed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 0x7faa8c0c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa90eed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 0x7faa8c0c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40746/0 (socket says 172.21.15.37:40746) 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa90eed640 1 -- 172.21.15.37:0/3794472087 learned_addr learned my addr 172.21.15.37:0/3794472087 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa916ee640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa8c15a9a0 0x7faa8c15bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa8bfff640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8c0c4f30 0x7faa8c15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa90eed640 1 -- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8c0c4f30 msgr2=0x7faa8c15b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa90eed640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8c0c4f30 0x7faa8c15b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:49.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa90eed640 1 -- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa8c15a9a0 msgr2=0x7faa8c15bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa90eed640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa8c15a9a0 0x7faa8c15bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa90eed640 1 -- 172.21.15.37:0/3794472087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c0c5f40 con 0x7faa740051d0 2024-04-25T08:55:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa8bfff640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8c0c4f30 0x7faa8c15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa90eed640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 0x7faa8c0c49f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7faa7c000fe0 tx=0x7faa7c005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa897fa640 1 -- 172.21.15.37:0/3794472087 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c00e950 con 0x7faa740051d0 2024-04-25T08:55:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa897fa640 1 -- 172.21.15.37:0/3794472087 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c011040 con 0x7faa740051d0 2024-04-25T08:55:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa93151640 1 -- 172.21.15.37:0/3794472087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8c0c53c0 con 0x7faa740051d0 2024-04-25T08:55:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa897fa640 1 -- 172.21.15.37:0/3794472087 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c00ecf0 con 0x7faa740051d0 2024-04-25T08:55:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa93151640 1 -- 172.21.15.37:0/3794472087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8c0c5980 con 0x7faa740051d0 2024-04-25T08:55:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa93151640 1 -- 172.21.15.37:0/3794472087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8c04efa0 con 0x7faa740051d0 2024-04-25T08:55:49.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa897fa640 1 -- 172.21.15.37:0/3794472087 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faa7c01d020 con 0x7faa740051d0 2024-04-25T08:55:49.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa897fa640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faa78066f90 0x7faa78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:49.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.026+0000 7faa897fa640 1 -- 172.21.15.37:0/3794472087 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faa7c087e40 con 0x7faa740051d0 2024-04-25T08:55:49.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.030+0000 7faa8bfff640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faa78066f90 0x7faa78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:49.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.030+0000 7faa8bfff640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faa78066f90 0x7faa78069450 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7faa8c1f6560 tx=0x7faa80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:49.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.034+0000 7faa897fa640 1 -- 172.21.15.37:0/3794472087 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faa7c058670 con 0x7faa740051d0 2024-04-25T08:55:49.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.226+0000 7faa93151640 1 -- 172.21.15.37:0/3794472087 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7faa8c0b6480 con 0x7faa740051d0 2024-04-25T08:55:49.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.862+0000 7faa897fa640 1 -- 172.21.15.37:0/3794472087 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa7c05c710 con 0x7faa740051d0 2024-04-25T08:55:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 -- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faa78066f90 msgr2=0x7faa78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faa78066f90 0x7faa78069450 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7faa8c1f6560 tx=0x7faa80009040 comp rx=0 tx=0).stop 2024-04-25T08:55:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 -- 172.21.15.37:0/3794472087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 msgr2=0x7faa8c0c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 0x7faa8c0c49f0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7faa7c000fe0 tx=0x7faa7c005700 comp rx=0 tx=0).stop 2024-04-25T08:55:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 -- 172.21.15.37:0/3794472087 shutdown_connections 2024-04-25T08:55:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faa78066f90 0x7faa78069450 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faa8c15a9a0 0x7faa8c15bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:49.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faa8c0c4f30 0x7faa8c15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:49.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 --2- 172.21.15.37:0/3794472087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faa740051d0 0x7faa8c0c49f0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:49.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 -- 172.21.15.37:0/3794472087 >> 172.21.15.37:0/3794472087 conn(0x7faa8c099a70 msgr2=0x7faa8c0ad360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:49.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 -- 172.21.15.37:0/3794472087 shutdown_connections 2024-04-25T08:55:49.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:49.866+0000 7faa93151640 1 -- 172.21.15.37:0/3794472087 wait complete. 2024-04-25T08:55:49.881 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:55:49.881 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:50.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a55875640 1 Processor -- start 2024-04-25T08:55:50.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a55875640 1 -- start start 2024-04-25T08:55:50.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a55875640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a5015da80 0x7f4a500d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a55875640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4a500d1cd0 0x7f4a500d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a55875640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5006be60 con 0x7f4a500d0620 2024-04-25T08:55:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a55875640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5006c440 con 0x7f4a500d1cd0 2024-04-25T08:55:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a55875640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5015de60 con 0x7f4a5015da80 2024-04-25T08:55:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a4effd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a5015da80 0x7f4a500d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a467fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4a500d1cd0 0x7f4a500d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a4effd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a5015da80 0x7f4a500d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:37948/0 (socket says 172.21.15.37:37948) 2024-04-25T08:55:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a4f7fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f4a500d0620 0x7f4a500ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:54454/0 (socket says 172.21.15.37:54454) 2024-04-25T08:55:50.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a4effd640 1 -- 172.21.15.37:0/2531133358 learned_addr learned my addr 172.21.15.37:0/2531133358 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:50.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a467fc640 1 -- 172.21.15.37:0/2531133358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a5015da80 msgr2=0x7f4a500d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:50.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a467fc640 1 --2- 172.21.15.37:0/2531133358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a5015da80 0x7f4a500d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a467fc640 1 -- 172.21.15.37:0/2531133358 >> v1:172.21.15.203:6789/0 conn(0x7f4a500d0620 legacy=0x7f4a500ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a467fc640 1 -- 172.21.15.37:0/2531133358 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500d2450 con 0x7f4a500d1cd0 2024-04-25T08:55:50.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.014+0000 7f4a4effd640 1 --2- 172.21.15.37:0/2531133358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a5015da80 0x7f4a500d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:55:50.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a467fc640 1 --2- 172.21.15.37:0/2531133358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a500d1cd0 0x7f4a500d00e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4a380097a0 tx=0x7f4a3801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5637e619871d80e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:50.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a4e7fc640 1 -- 172.21.15.37:0/2531133358 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380b2070 con 0x7f4a500d1cd0 2024-04-25T08:55:50.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a4e7fc640 1 -- 172.21.15.37:0/2531133358 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a380ad640 con 0x7f4a500d1cd0 2024-04-25T08:55:50.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a4e7fc640 1 -- 172.21.15.37:0/2531133358 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380ada10 con 0x7f4a500d1cd0 2024-04-25T08:55:50.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a55875640 1 -- 172.21.15.37:0/2531133358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a500d1cd0 msgr2=0x7f4a500d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a55875640 1 --2- 172.21.15.37:0/2531133358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a500d1cd0 0x7f4a500d00e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4a380097a0 tx=0x7f4a3801c480 comp rx=0 tx=0).stop 2024-04-25T08:55:50.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a55875640 1 -- 172.21.15.37:0/2531133358 shutdown_connections 2024-04-25T08:55:50.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a55875640 1 --2- 172.21.15.37:0/2531133358 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a500d1cd0 0x7f4a500d00e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a55875640 1 --2- 172.21.15.37:0/2531133358 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a5015da80 0x7f4a500d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a55875640 1 -- 172.21.15.37:0/2531133358 >> 172.21.15.37:0/2531133358 conn(0x7f4a5008b000 msgr2=0x7f4a500d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:50.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a55875640 1 -- 172.21.15.37:0/2531133358 shutdown_connections 2024-04-25T08:55:50.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a55875640 1 -- 172.21.15.37:0/2531133358 wait complete. 2024-04-25T08:55:50.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a55875640 1 Processor -- start 2024-04-25T08:55:50.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.018+0000 7f4a55875640 1 -- start start 2024-04-25T08:55:50.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a55875640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a500d0620 0x7f4a5015fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a55875640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a500d1cd0 0x7f4a500b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a55875640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a5015da80 0x7f4a50154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a55875640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5016a730 con 0x7f4a5015da80 2024-04-25T08:55:50.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a55875640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5016a5b0 con 0x7f4a500d1cd0 2024-04-25T08:55:50.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a55875640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5016a8b0 con 0x7f4a500d0620 2024-04-25T08:55:50.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a5015da80 0x7f4a50154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a5015da80 0x7f4a50154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57110/0 (socket says 172.21.15.37:57110) 2024-04-25T08:55:50.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4f7fe640 1 -- 172.21.15.37:0/335236269 learned_addr learned my addr 172.21.15.37:0/335236269 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:50.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4effd640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a500d0620 0x7f4a5015fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a467fc640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a500d1cd0 0x7f4a500b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4f7fe640 1 -- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a500d0620 msgr2=0x7f4a5015fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4f7fe640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a500d0620 0x7f4a5015fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4f7fe640 1 -- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a500d1cd0 msgr2=0x7f4a500b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4f7fe640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a500d1cd0 0x7f4a500b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4f7fe640 1 -- 172.21.15.37:0/335236269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500cb750 con 0x7f4a5015da80 2024-04-25T08:55:50.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a467fc640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a500d1cd0 0x7f4a500b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:50.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4f7fe640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a5015da80 0x7f4a50154c50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800c7d0 tx=0x7f4a4800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:50.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4cff9640 1 -- 172.21.15.37:0/335236269 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a48017070 con 0x7f4a5015da80 2024-04-25T08:55:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a55875640 1 -- 172.21.15.37:0/335236269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a500cba30 con 0x7f4a5015da80 2024-04-25T08:55:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a55875640 1 -- 172.21.15.37:0/335236269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a500cbff0 con 0x7f4a5015da80 2024-04-25T08:55:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4cff9640 1 -- 172.21.15.37:0/335236269 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a4800a640 con 0x7f4a5015da80 2024-04-25T08:55:50.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4cff9640 1 -- 172.21.15.37:0/335236269 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4800a9f0 con 0x7f4a5015da80 2024-04-25T08:55:50.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a55875640 1 -- 172.21.15.37:0/335236269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a340052a0 con 0x7f4a5015da80 2024-04-25T08:55:50.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4cff9640 1 -- 172.21.15.37:0/335236269 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4a4801a070 con 0x7f4a5015da80 2024-04-25T08:55:50.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4cff9640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a30066f60 0x7f4a30069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.022+0000 7f4a4cff9640 1 -- 172.21.15.37:0/335236269 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a48084c40 con 0x7f4a5015da80 2024-04-25T08:55:50.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.026+0000 7f4a4effd640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a30066f60 0x7f4a30069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.026+0000 7f4a4effd640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a30066f60 0x7f4a30069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4a40006cb0 tx=0x7f4a40009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:50.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.026+0000 7f4a4cff9640 1 -- 172.21.15.37:0/335236269 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4a480554d0 con 0x7f4a5015da80 2024-04-25T08:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 -- 172.21.15.37:0/335236269 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a34005590 con 0x7f4a5015da80 2024-04-25T08:55:50.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a4cff9640 1 -- 172.21.15.37:0/335236269 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f4a48059570 con 0x7f4a5015da80 2024-04-25T08:55:50.208 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:50.208 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/486740526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":486740526},{"type":"v1","addr":"172.21.15.37:6833","nonce":486740526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/121484331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":121484331},{"type":"v1","addr":"172.21.15.203:6835","nonce":121484331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2501630566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2501630566},{"type":"v1","addr":"172.21.15.203:6837","nonce":2501630566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:55:46.200939+0000","modified":"2024-04-25T08:55:49.230920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6405":{"gid":6405,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/2189232625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2189232625},{"type":"v1","addr":"172.21.15.37:6835","nonce":2189232625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:55:50.208 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-04-25T08:55:50.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 -- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a30066f60 msgr2=0x7f4a30069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a30066f60 0x7f4a30069420 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f4a40006cb0 tx=0x7f4a40009240 comp rx=0 tx=0).stop 2024-04-25T08:55:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 -- 172.21.15.37:0/335236269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a5015da80 msgr2=0x7f4a50154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a5015da80 0x7f4a50154c50 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f4a4800c7d0 tx=0x7f4a4800cc90 comp rx=0 tx=0).stop 2024-04-25T08:55:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 -- 172.21.15.37:0/335236269 shutdown_connections 2024-04-25T08:55:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4a30066f60 0x7f4a30069420 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a5015da80 0x7f4a50154c50 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4a500d1cd0 0x7f4a500b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 --2- 172.21.15.37:0/335236269 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4a500d0620 0x7f4a5015fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 -- 172.21.15.37:0/335236269 >> 172.21.15.37:0/335236269 conn(0x7f4a5008b000 msgr2=0x7f4a501608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 -- 172.21.15.37:0/335236269 shutdown_connections 2024-04-25T08:55:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.218+0000 7f4a55875640 1 -- 172.21.15.37:0/335236269 wait complete. 2024-04-25T08:55:50.226 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1caaff0640 1 Processor -- start 2024-04-25T08:55:50.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1caaff0640 1 -- start start 2024-04-25T08:55:50.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1caaff0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1ca415b920 0x7f1ca4099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1caaff0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 0x7f1ca4099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1caaff0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca409a950 con 0x7f1ca40aeb80 2024-04-25T08:55:50.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1caaff0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca409ac30 con 0x7f1ca415b920 2024-04-25T08:55:50.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1caaff0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40ae380 con 0x7f1ca40b0440 2024-04-25T08:55:50.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 0x7f1ca4099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1ca8d8c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1ca415b920 0x7f1ca4099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1ca8d8c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1ca415b920 0x7f1ca4099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:59174/0 (socket says 172.21.15.37:59174) 2024-04-25T08:55:50.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1ca958d640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f1ca40aeb80 0x7f1ca40aeff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:54466/0 (socket says 172.21.15.37:54466) 2024-04-25T08:55:50.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1ca958d640 1 -- 172.21.15.37:0/161674592 learned_addr learned my addr 172.21.15.37:0/161674592 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:50.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1ca3fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 0x7f1ca4099fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:37966/0 (socket says 172.21.15.37:37966) 2024-04-25T08:55:50.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1ca3fff640 1 -- 172.21.15.37:0/161674592 >> v2:172.21.15.203:3300/0 conn(0x7f1ca415b920 msgr2=0x7f1ca4099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:50.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1ca3fff640 1 --2- 172.21.15.37:0/161674592 >> v2:172.21.15.203:3300/0 conn(0x7f1ca415b920 0x7f1ca4099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1ca3fff640 1 -- 172.21.15.37:0/161674592 >> v1:172.21.15.203:6789/0 conn(0x7f1ca40aeb80 legacy=0x7f1ca40aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1ca3fff640 1 -- 172.21.15.37:0/161674592 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40ae660 con 0x7f1ca40b0440 2024-04-25T08:55:50.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.366+0000 7f1ca8d8c640 1 --2- 172.21.15.37:0/161674592 >> v2:172.21.15.203:3300/0 conn(0x7f1ca415b920 0x7f1ca4099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:55:50.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1ca3fff640 1 --2- 172.21.15.37:0/161674592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 0x7f1ca4099fb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1c980097c0 tx=0x7f1c9801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25bdd6538d8d58a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:50.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1ca37fe640 1 -- 172.21.15.37:0/161674592 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c980b0070 con 0x7f1ca40b0440 2024-04-25T08:55:50.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1ca37fe640 1 -- 172.21.15.37:0/161674592 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c980ab600 con 0x7f1ca40b0440 2024-04-25T08:55:50.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1ca37fe640 1 -- 172.21.15.37:0/161674592 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c980ab9d0 con 0x7f1ca40b0440 2024-04-25T08:55:50.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1caaff0640 1 -- 172.21.15.37:0/161674592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 msgr2=0x7f1ca4099fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1caaff0640 1 --2- 172.21.15.37:0/161674592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 0x7f1ca4099fb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1c980097c0 tx=0x7f1c9801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1caaff0640 1 -- 172.21.15.37:0/161674592 shutdown_connections 2024-04-25T08:55:50.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1caaff0640 1 --2- 172.21.15.37:0/161674592 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 0x7f1ca4099fb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1caaff0640 1 --2- 172.21.15.37:0/161674592 >> v2:172.21.15.203:3300/0 conn(0x7f1ca415b920 0x7f1ca4099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1caaff0640 1 -- 172.21.15.37:0/161674592 >> 172.21.15.37:0/161674592 conn(0x7f1ca40b3010 msgr2=0x7f1ca40b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:50.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1caaff0640 1 -- 172.21.15.37:0/161674592 shutdown_connections 2024-04-25T08:55:50.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1caaff0640 1 -- 172.21.15.37:0/161674592 wait complete. 2024-04-25T08:55:50.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1caaff0640 1 Processor -- start 2024-04-25T08:55:50.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.370+0000 7f1caaff0640 1 -- start start 2024-04-25T08:55:50.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1caaff0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca40aeb80 0x7f1ca4150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1caaff0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 0x7f1ca40b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1caaff0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca415b920 0x7f1ca40b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1caaff0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4168850 con 0x7f1ca415b920 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1caaff0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca41686d0 con 0x7f1ca40aeb80 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1caaff0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca41689d0 con 0x7f1ca40b0440 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca958d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca415b920 0x7f1ca40b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca958d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca415b920 0x7f1ca40b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57116/0 (socket says 172.21.15.37:57116) 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca958d640 1 -- 172.21.15.37:0/1433874619 learned_addr learned my addr 172.21.15.37:0/1433874619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca8d8c640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca40aeb80 0x7f1ca4150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca3fff640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 0x7f1ca40b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca958d640 1 -- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 msgr2=0x7f1ca40b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca958d640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 0x7f1ca40b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca958d640 1 -- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca40aeb80 msgr2=0x7f1ca4150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca958d640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca40aeb80 0x7f1ca4150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca958d640 1 -- 172.21.15.37:0/1433874619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca415c180 con 0x7f1ca415b920 2024-04-25T08:55:50.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca8d8c640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca40aeb80 0x7f1ca4150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:50.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca958d640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca415b920 0x7f1ca40b1fd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400ca80 tx=0x7f1c9400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:50.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca17fa640 1 -- 172.21.15.37:0/1433874619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94017070 con 0x7f1ca415b920 2024-04-25T08:55:50.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca17fa640 1 -- 172.21.15.37:0/1433874619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9400ac30 con 0x7f1ca415b920 2024-04-25T08:55:50.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1caaff0640 1 -- 172.21.15.37:0/1433874619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca415c460 con 0x7f1ca415b920 2024-04-25T08:55:50.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca17fa640 1 -- 172.21.15.37:0/1433874619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94012400 con 0x7f1ca415b920 2024-04-25T08:55:50.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1caaff0640 1 -- 172.21.15.37:0/1433874619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca415ca20 con 0x7f1ca415b920 2024-04-25T08:55:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1caaff0640 1 -- 172.21.15.37:0/1433874619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca404efa0 con 0x7f1ca415b920 2024-04-25T08:55:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca17fa640 1 -- 172.21.15.37:0/1433874619 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c9401a070 con 0x7f1ca415b920 2024-04-25T08:55:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca17fa640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066fb0 0x7f1c90069470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.374+0000 7f1ca17fa640 1 -- 172.21.15.37:0/1433874619 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c94084d10 con 0x7f1ca415b920 2024-04-25T08:55:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.378+0000 7f1ca8d8c640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066fb0 0x7f1c90069470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.378+0000 7f1ca8d8c640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066fb0 0x7f1c90069470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4156940 tx=0x7f1c9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:50.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.382+0000 7f1ca17fa640 1 -- 172.21.15.37:0/1433874619 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c94055620 con 0x7f1ca415b920 2024-04-25T08:55:50.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.538+0000 7f1caaff0640 1 -- 172.21.15.37:0/1433874619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ca4099c80 con 0x7f1ca415b920 2024-04-25T08:55:50.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.538+0000 7f1ca17fa640 1 -- 172.21.15.37:0/1433874619 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+16848 (secure 0 0 0) 0x7f1c940596c0 con 0x7f1ca415b920 2024-04-25T08:55:50.529 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:50.530 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":69,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:55:47.500249+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:44.492629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:45.203787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 -- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066fb0 msgr2=0x7f1c90069470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066fb0 0x7f1c90069470 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1ca4156940 tx=0x7f1c9c008040 comp rx=0 tx=0).stop 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 -- 172.21.15.37:0/1433874619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca415b920 msgr2=0x7f1ca40b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca415b920 0x7f1ca40b1fd0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400ca80 tx=0x7f1c9400cf40 comp rx=0 tx=0).stop 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 -- 172.21.15.37:0/1433874619 shutdown_connections 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066fb0 0x7f1c90069470 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca415b920 0x7f1ca40b1fd0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40b0440 0x7f1ca40b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 --2- 172.21.15.37:0/1433874619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca40aeb80 0x7f1ca4150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 -- 172.21.15.37:0/1433874619 >> 172.21.15.37:0/1433874619 conn(0x7f1ca40b3010 msgr2=0x7f1ca4070340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 -- 172.21.15.37:0/1433874619 shutdown_connections 2024-04-25T08:55:50.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.542+0000 7f1caaff0640 1 -- 172.21.15.37:0/1433874619 wait complete. 2024-04-25T08:55:50.547 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:55:50.547 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:55:50.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.666+0000 7f836d33c640 1 Processor -- start 2024-04-25T08:55:50.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.666+0000 7f836d33c640 1 -- start start 2024-04-25T08:55:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f836d33c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f836815ed40 0x7f83680d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f836d33c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83680c7020 0x7f83680caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f836d33c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83680affa0 con 0x7f8368160080 2024-04-25T08:55:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f836d33c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836815f120 con 0x7f83680c7020 2024-04-25T08:55:50.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f836d33c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83680b0240 con 0x7f836815ed40 2024-04-25T08:55:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f83677fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83680c7020 0x7f83680caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f83677fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f83680c7020 0x7f83680caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57118/0 (socket says 172.21.15.37:57118) 2024-04-25T08:55:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f83677fe640 1 -- 172.21.15.37:0/4213674642 learned_addr learned my addr 172.21.15.37:0/4213674642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f8366ffd640 1 --2- 172.21.15.37:0/4213674642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f836815ed40 0x7f83680d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f83677fe640 1 -- 172.21.15.37:0/4213674642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f836815ed40 msgr2=0x7f83680d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f83677fe640 1 --2- 172.21.15.37:0/4213674642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f836815ed40 0x7f83680d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f83677fe640 1 -- 172.21.15.37:0/4213674642 >> v1:172.21.15.203:6789/0 conn(0x7f8368160080 legacy=0x7f83680c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:50.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f83677fe640 1 -- 172.21.15.37:0/4213674642 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83680b0520 con 0x7f83680c7020 2024-04-25T08:55:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f83677fe640 1 --2- 172.21.15.37:0/4213674642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83680c7020 0x7f83680caad0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8360009870 tx=0x7f836001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615e53aa6a15d942 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f8365ffb640 1 -- 172.21.15.37:0/4213674642 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83600ab620 con 0x7f83680c7020 2024-04-25T08:55:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f8365ffb640 1 -- 172.21.15.37:0/4213674642 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83600b0070 con 0x7f83680c7020 2024-04-25T08:55:50.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.670+0000 7f8365ffb640 1 -- 172.21.15.37:0/4213674642 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83600ab9e0 con 0x7f83680c7020 2024-04-25T08:55:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- 172.21.15.37:0/4213674642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83680c7020 msgr2=0x7f83680caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 --2- 172.21.15.37:0/4213674642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83680c7020 0x7f83680caad0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f8360009870 tx=0x7f836001c460 comp rx=0 tx=0).stop 2024-04-25T08:55:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- 172.21.15.37:0/4213674642 shutdown_connections 2024-04-25T08:55:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 --2- 172.21.15.37:0/4213674642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83680c7020 0x7f83680caad0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 --2- 172.21.15.37:0/4213674642 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f836815ed40 0x7f83680d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- 172.21.15.37:0/4213674642 >> 172.21.15.37:0/4213674642 conn(0x7f8368099a70 msgr2=0x7f83680d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- 172.21.15.37:0/4213674642 shutdown_connections 2024-04-25T08:55:50.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- 172.21.15.37:0/4213674642 wait complete. 2024-04-25T08:55:50.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 Processor -- start 2024-04-25T08:55:50.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- start start 2024-04-25T08:55:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83680c7020 0x7f83680cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f836815ed40 0x7f836814d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8368160080 0x7f836815d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83680cb9a0 con 0x7f83680c7020 2024-04-25T08:55:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83680cb820 con 0x7f836815ed40 2024-04-25T08:55:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83680cbb20 con 0x7f8368160080 2024-04-25T08:55:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f8366ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83680c7020 0x7f83680cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f8366ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83680c7020 0x7f83680cfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57130/0 (socket says 172.21.15.37:57130) 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f8366ffd640 1 -- 172.21.15.37:0/2634179304 learned_addr learned my addr 172.21.15.37:0/2634179304 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f83677fe640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8368160080 0x7f836815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f8366ffd640 1 -- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8368160080 msgr2=0x7f836815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f8366ffd640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8368160080 0x7f836815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f8366ffd640 1 -- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f836815ed40 msgr2=0x7f836814d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f83667fc640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f836815ed40 0x7f836814d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f8366ffd640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f836815ed40 0x7f836814d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f8366ffd640 1 -- 172.21.15.37:0/2634179304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83680cf800 con 0x7f83680c7020 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f83677fe640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8368160080 0x7f836815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f83667fc640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f836815ed40 0x7f836814d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f8366ffd640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83680c7020 0x7f83680cfca0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f835800c820 tx=0x7f835800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f834f7fe640 1 -- 172.21.15.37:0/2634179304 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358015070 con 0x7f83680c7020 2024-04-25T08:55:50.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f834f7fe640 1 -- 172.21.15.37:0/2634179304 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8358007a30 con 0x7f83680c7020 2024-04-25T08:55:50.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- 172.21.15.37:0/2634179304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836815bca0 con 0x7f83680c7020 2024-04-25T08:55:50.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f834f7fe640 1 -- 172.21.15.37:0/2634179304 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358007e20 con 0x7f83680c7020 2024-04-25T08:55:50.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- 172.21.15.37:0/2634179304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f836815c260 con 0x7f83680c7020 2024-04-25T08:55:50.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.674+0000 7f836d33c640 1 -- 172.21.15.37:0/2634179304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f836804efa0 con 0x7f83680c7020 2024-04-25T08:55:50.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.678+0000 7f834f7fe640 1 -- 172.21.15.37:0/2634179304 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8358018070 con 0x7f83680c7020 2024-04-25T08:55:50.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.678+0000 7f834f7fe640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8354066f60 0x7f8354069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:50.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.678+0000 7f834f7fe640 1 -- 172.21.15.37:0/2634179304 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8358085140 con 0x7f83680c7020 2024-04-25T08:55:50.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.678+0000 7f83667fc640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8354066f60 0x7f8354069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:50.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.678+0000 7f83667fc640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8354066f60 0x7f8354069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8368157fd0 tx=0x7f835c0073c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:50.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.682+0000 7f834f7fe640 1 -- 172.21.15.37:0/2634179304 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8358055fb0 con 0x7f83680c7020 2024-04-25T08:55:50.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.834+0000 7f836d33c640 1 -- 172.21.15.37:0/2634179304 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f83680af8b0 con 0x7f8354066f60 2024-04-25T08:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.854+0000 7f834f7fe640 1 -- 172.21.15.37:0/2634179304 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252016 (secure 0 0 0) 0x7f83680af8b0 con 0x7f8354066f60 2024-04-25T08:55:50.847 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:50.849 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 -- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8354066f60 msgr2=0x7f8354069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8354066f60 0x7f8354069420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8368157fd0 tx=0x7f835c0073c0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 -- 172.21.15.37:0/2634179304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83680c7020 msgr2=0x7f83680cfca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83680c7020 0x7f83680cfca0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f835800c820 tx=0x7f835800cce0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 -- 172.21.15.37:0/2634179304 shutdown_connections 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8354066f60 0x7f8354069420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8368160080 0x7f836815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f836815ed40 0x7f836814d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 --2- 172.21.15.37:0/2634179304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83680c7020 0x7f83680cfca0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 -- 172.21.15.37:0/2634179304 >> 172.21.15.37:0/2634179304 conn(0x7f8368099a70 msgr2=0x7f836806e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 -- 172.21.15.37:0/2634179304 shutdown_connections 2024-04-25T08:55:50.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:50.858+0000 7f836d33c640 1 -- 172.21.15.37:0/2634179304 wait complete. 2024-04-25T08:55:50.866 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-04-25T08:55:49.188917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248900,"kb_used_data":7740,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725820,"statfs":{"total":773094113280,"available":772839239680,"internally_reserved":0,"allocated":7925760,"data_stored":4224624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.369315"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929113+0000","last_change":"2024-04-25T08:55:46.534398+0000","last_active":"2024-04-25T08:55:47.929113+0000","last_peered":"2024-04-25T08:55:47.929113+0000","last_clean":"2024-04-25T08:55:47.929113+0000","last_became_active":"2024-04-25T08:55:46.533596+0000","last_became_peered":"2024-04-25T08:55:46.533596+0000","last_unstale":"2024-04-25T08:55:47.929113+0000","last_undegraded":"2024-04-25T08:55:47.929113+0000","last_fullsized":"2024-04-25T08:55:47.929113+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:20.962033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929523+0000","last_change":"2024-04-25T08:55:46.539652+0000","last_active":"2024-04-25T08:55:47.929523+0000","last_peered":"2024-04-25T08:55:47.929523+0000","last_clean":"2024-04-25T08:55:47.929523+0000","last_became_active":"2024-04-25T08:55:46.538929+0000","last_became_peered":"2024-04-25T08:55:46.538929+0000","last_unstale":"2024-04-25T08:55:47.929523+0000","last_undegraded":"2024-04-25T08:55:47.929523+0000","last_fullsized":"2024-04-25T08:55:47.929523+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:14.229282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926911+0000","last_change":"2024-04-25T08:55:46.560306+0000","last_active":"2024-04-25T08:55:47.926911+0000","last_peered":"2024-04-25T08:55:47.926911+0000","last_clean":"2024-04-25T08:55:47.926911+0000","last_became_active":"2024-04-25T08:55:46.559561+0000","last_became_peered":"2024-04-25T08:55:46.559561+0000","last_unstale":"2024-04-25T08:55:47.926911+0000","last_undegraded":"2024-04-25T08:55:47.926911+0000","last_fullsized":"2024-04-25T08:55:47.926911+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:20.962033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516627+0000","last_change":"2024-04-25T08:55:46.544064+0000","last_active":"2024-04-25T08:55:47.516627+0000","last_peered":"2024-04-25T08:55:47.516627+0000","last_clean":"2024-04-25T08:55:47.516627+0000","last_became_active":"2024-04-25T08:55:46.542748+0000","last_became_peered":"2024-04-25T08:55:46.542748+0000","last_unstale":"2024-04-25T08:55:47.516627+0000","last_undegraded":"2024-04-25T08:55:47.516627+0000","last_fullsized":"2024-04-25T08:55:47.516627+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:05.492858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515707+0000","last_change":"2024-04-25T08:55:46.518087+0000","last_active":"2024-04-25T08:55:47.515707+0000","last_peered":"2024-04-25T08:55:47.515707+0000","last_clean":"2024-04-25T08:55:47.515707+0000","last_became_active":"2024-04-25T08:55:46.517109+0000","last_became_peered":"2024-04-25T08:55:46.517109+0000","last_unstale":"2024-04-25T08:55:47.515707+0000","last_undegraded":"2024-04-25T08:55:47.515707+0000","last_fullsized":"2024-04-25T08:55:47.515707+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:51.700529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928007+0000","last_change":"2024-04-25T08:55:46.554289+0000","last_active":"2024-04-25T08:55:47.928007+0000","last_peered":"2024-04-25T08:55:47.928007+0000","last_clean":"2024-04-25T08:55:47.928007+0000","last_became_active":"2024-04-25T08:55:46.553819+0000","last_became_peered":"2024-04-25T08:55:46.553819+0000","last_unstale":"2024-04-25T08:55:47.928007+0000","last_undegraded":"2024-04-25T08:55:47.928007+0000","last_fullsized":"2024-04-25T08:55:47.928007+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:57.531045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.514954+0000","last_change":"2024-04-25T08:55:46.549592+0000","last_active":"2024-04-25T08:55:47.514954+0000","last_peered":"2024-04-25T08:55:47.514954+0000","last_clean":"2024-04-25T08:55:47.514954+0000","last_became_active":"2024-04-25T08:55:46.548865+0000","last_became_peered":"2024-04-25T08:55:46.548865+0000","last_unstale":"2024-04-25T08:55:47.514954+0000","last_undegraded":"2024-04-25T08:55:47.514954+0000","last_fullsized":"2024-04-25T08:55:47.514954+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:12.950330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926333+0000","last_change":"2024-04-25T08:55:46.563914+0000","last_active":"2024-04-25T08:55:47.926333+0000","last_peered":"2024-04-25T08:55:47.926333+0000","last_clean":"2024-04-25T08:55:47.926333+0000","last_became_active":"2024-04-25T08:55:46.563267+0000","last_became_peered":"2024-04-25T08:55:46.563267+0000","last_unstale":"2024-04-25T08:55:47.926333+0000","last_undegraded":"2024-04-25T08:55:47.926333+0000","last_fullsized":"2024-04-25T08:55:47.926333+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:20.962033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929016+0000","last_change":"2024-04-25T08:55:46.562700+0000","last_active":"2024-04-25T08:55:47.929016+0000","last_peered":"2024-04-25T08:55:47.929016+0000","last_clean":"2024-04-25T08:55:47.929016+0000","last_became_active":"2024-04-25T08:55:46.562094+0000","last_became_peered":"2024-04-25T08:55:46.562094+0000","last_unstale":"2024-04-25T08:55:47.929016+0000","last_undegraded":"2024-04-25T08:55:47.929016+0000","last_fullsized":"2024-04-25T08:55:47.929016+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:12.531432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928629+0000","last_change":"2024-04-25T08:55:46.557333+0000","last_active":"2024-04-25T08:55:47.928629+0000","last_peered":"2024-04-25T08:55:47.928629+0000","last_clean":"2024-04-25T08:55:47.928629+0000","last_became_active":"2024-04-25T08:55:46.556524+0000","last_became_peered":"2024-04-25T08:55:46.556524+0000","last_unstale":"2024-04-25T08:55:47.928629+0000","last_undegraded":"2024-04-25T08:55:47.928629+0000","last_fullsized":"2024-04-25T08:55:47.928629+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:10.462685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516084+0000","last_change":"2024-04-25T08:55:46.528417+0000","last_active":"2024-04-25T08:55:47.516084+0000","last_peered":"2024-04-25T08:55:47.516084+0000","last_clean":"2024-04-25T08:55:47.516084+0000","last_became_active":"2024-04-25T08:55:46.527702+0000","last_became_peered":"2024-04-25T08:55:46.527702+0000","last_unstale":"2024-04-25T08:55:47.516084+0000","last_undegraded":"2024-04-25T08:55:47.516084+0000","last_fullsized":"2024-04-25T08:55:47.516084+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:31.380413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927804+0000","last_change":"2024-04-25T08:55:46.551665+0000","last_active":"2024-04-25T08:55:47.927804+0000","last_peered":"2024-04-25T08:55:47.927804+0000","last_clean":"2024-04-25T08:55:47.927804+0000","last_became_active":"2024-04-25T08:55:46.550994+0000","last_became_peered":"2024-04-25T08:55:46.550994+0000","last_unstale":"2024-04-25T08:55:47.927804+0000","last_undegraded":"2024-04-25T08:55:47.927804+0000","last_fullsized":"2024-04-25T08:55:47.927804+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:43.066148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927612+0000","last_change":"2024-04-25T08:55:46.541749+0000","last_active":"2024-04-25T08:55:47.927612+0000","last_peered":"2024-04-25T08:55:47.927612+0000","last_clean":"2024-04-25T08:55:47.927612+0000","last_became_active":"2024-04-25T08:55:46.540950+0000","last_became_peered":"2024-04-25T08:55:46.540950+0000","last_unstale":"2024-04-25T08:55:47.927612+0000","last_undegraded":"2024-04-25T08:55:47.927612+0000","last_fullsized":"2024-04-25T08:55:47.927612+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:01.335363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927728+0000","last_change":"2024-04-25T08:55:46.558897+0000","last_active":"2024-04-25T08:55:47.927728+0000","last_peered":"2024-04-25T08:55:47.927728+0000","last_clean":"2024-04-25T08:55:47.927728+0000","last_became_active":"2024-04-25T08:55:46.557937+0000","last_became_peered":"2024-04-25T08:55:46.557937+0000","last_unstale":"2024-04-25T08:55:47.927728+0000","last_undegraded":"2024-04-25T08:55:47.927728+0000","last_fullsized":"2024-04-25T08:55:47.927728+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:43.066148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927113+0000","last_change":"2024-04-25T08:55:46.560374+0000","last_active":"2024-04-25T08:55:47.927113+0000","last_peered":"2024-04-25T08:55:47.927113+0000","last_clean":"2024-04-25T08:55:47.927113+0000","last_became_active":"2024-04-25T08:55:46.559664+0000","last_became_peered":"2024-04-25T08:55:46.559664+0000","last_unstale":"2024-04-25T08:55:47.927113+0000","last_undegraded":"2024-04-25T08:55:47.927113+0000","last_fullsized":"2024-04-25T08:55:47.927113+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:33.737868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.514926+0000","last_change":"2024-04-25T08:55:46.534310+0000","last_active":"2024-04-25T08:55:47.514926+0000","last_peered":"2024-04-25T08:55:47.514926+0000","last_clean":"2024-04-25T08:55:47.514926+0000","last_became_active":"2024-04-25T08:55:46.533609+0000","last_became_peered":"2024-04-25T08:55:46.533609+0000","last_unstale":"2024-04-25T08:55:47.514926+0000","last_undegraded":"2024-04-25T08:55:47.514926+0000","last_fullsized":"2024-04-25T08:55:47.514926+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:05.039606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929454+0000","last_change":"2024-04-25T08:55:46.545284+0000","last_active":"2024-04-25T08:55:47.929454+0000","last_peered":"2024-04-25T08:55:47.929454+0000","last_clean":"2024-04-25T08:55:47.929454+0000","last_became_active":"2024-04-25T08:55:46.544088+0000","last_became_peered":"2024-04-25T08:55:46.544088+0000","last_unstale":"2024-04-25T08:55:47.929454+0000","last_undegraded":"2024-04-25T08:55:47.929454+0000","last_fullsized":"2024-04-25T08:55:47.929454+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:33.737868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516471+0000","last_change":"2024-04-25T08:55:46.543469+0000","last_active":"2024-04-25T08:55:47.516471+0000","last_peered":"2024-04-25T08:55:47.516471+0000","last_clean":"2024-04-25T08:55:47.516471+0000","last_became_active":"2024-04-25T08:55:46.542572+0000","last_became_peered":"2024-04-25T08:55:46.542572+0000","last_unstale":"2024-04-25T08:55:47.516471+0000","last_undegraded":"2024-04-25T08:55:47.516471+0000","last_fullsized":"2024-04-25T08:55:47.516471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:51.700529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515631+0000","last_change":"2024-04-25T08:55:46.548884+0000","last_active":"2024-04-25T08:55:47.515631+0000","last_peered":"2024-04-25T08:55:47.515631+0000","last_clean":"2024-04-25T08:55:47.515631+0000","last_became_active":"2024-04-25T08:55:46.548256+0000","last_became_peered":"2024-04-25T08:55:46.548256+0000","last_unstale":"2024-04-25T08:55:47.515631+0000","last_undegraded":"2024-04-25T08:55:47.515631+0000","last_fullsized":"2024-04-25T08:55:47.515631+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:31.380413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927144+0000","last_change":"2024-04-25T08:55:46.531222+0000","last_active":"2024-04-25T08:55:47.927144+0000","last_peered":"2024-04-25T08:55:47.927144+0000","last_clean":"2024-04-25T08:55:47.927144+0000","last_became_active":"2024-04-25T08:55:46.530456+0000","last_became_peered":"2024-04-25T08:55:46.530456+0000","last_unstale":"2024-04-25T08:55:47.927144+0000","last_undegraded":"2024-04-25T08:55:47.927144+0000","last_fullsized":"2024-04-25T08:55:47.927144+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:02.769517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927848+0000","last_change":"2024-04-25T08:55:46.543282+0000","last_active":"2024-04-25T08:55:47.927848+0000","last_peered":"2024-04-25T08:55:47.927848+0000","last_clean":"2024-04-25T08:55:47.927848+0000","last_became_active":"2024-04-25T08:55:46.542598+0000","last_became_peered":"2024-04-25T08:55:46.542598+0000","last_unstale":"2024-04-25T08:55:47.927848+0000","last_undegraded":"2024-04-25T08:55:47.927848+0000","last_fullsized":"2024-04-25T08:55:47.927848+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:10.462685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926610+0000","last_change":"2024-04-25T08:55:46.552013+0000","last_active":"2024-04-25T08:55:47.926610+0000","last_peered":"2024-04-25T08:55:47.926610+0000","last_clean":"2024-04-25T08:55:47.926610+0000","last_became_active":"2024-04-25T08:55:46.551322+0000","last_became_peered":"2024-04-25T08:55:46.551322+0000","last_unstale":"2024-04-25T08:55:47.926610+0000","last_undegraded":"2024-04-25T08:55:47.926610+0000","last_fullsized":"2024-04-25T08:55:47.926610+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:43.094059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927001+0000","last_change":"2024-04-25T08:55:46.552853+0000","last_active":"2024-04-25T08:55:47.927001+0000","last_peered":"2024-04-25T08:55:47.927001+0000","last_clean":"2024-04-25T08:55:47.927001+0000","last_became_active":"2024-04-25T08:55:46.551266+0000","last_became_peered":"2024-04-25T08:55:46.551266+0000","last_unstale":"2024-04-25T08:55:47.927001+0000","last_undegraded":"2024-04-25T08:55:47.927001+0000","last_fullsized":"2024-04-25T08:55:47.927001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:46.112447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927693+0000","last_change":"2024-04-25T08:55:46.559771+0000","last_active":"2024-04-25T08:55:47.927693+0000","last_peered":"2024-04-25T08:55:47.927693+0000","last_clean":"2024-04-25T08:55:47.927693+0000","last_became_active":"2024-04-25T08:55:46.559211+0000","last_became_peered":"2024-04-25T08:55:46.559211+0000","last_unstale":"2024-04-25T08:55:47.927693+0000","last_undegraded":"2024-04-25T08:55:47.927693+0000","last_fullsized":"2024-04-25T08:55:47.927693+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:01.335363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927929+0000","last_change":"2024-04-25T08:55:46.548273+0000","last_active":"2024-04-25T08:55:47.927929+0000","last_peered":"2024-04-25T08:55:47.927929+0000","last_clean":"2024-04-25T08:55:47.927929+0000","last_became_active":"2024-04-25T08:55:46.547611+0000","last_became_peered":"2024-04-25T08:55:46.547611+0000","last_unstale":"2024-04-25T08:55:47.927929+0000","last_undegraded":"2024-04-25T08:55:47.927929+0000","last_fullsized":"2024-04-25T08:55:47.927929+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:12.531432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928077+0000","last_change":"2024-04-25T08:55:46.548416+0000","last_active":"2024-04-25T08:55:47.928077+0000","last_peered":"2024-04-25T08:55:47.928077+0000","last_clean":"2024-04-25T08:55:47.928077+0000","last_became_active":"2024-04-25T08:55:46.547828+0000","last_became_peered":"2024-04-25T08:55:46.547828+0000","last_unstale":"2024-04-25T08:55:47.928077+0000","last_undegraded":"2024-04-25T08:55:47.928077+0000","last_fullsized":"2024-04-25T08:55:47.928077+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:38.300578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516492+0000","last_change":"2024-04-25T08:55:46.516452+0000","last_active":"2024-04-25T08:55:47.516492+0000","last_peered":"2024-04-25T08:55:47.516492+0000","last_clean":"2024-04-25T08:55:47.516492+0000","last_became_active":"2024-04-25T08:55:46.515423+0000","last_became_peered":"2024-04-25T08:55:46.515423+0000","last_unstale":"2024-04-25T08:55:47.516492+0000","last_undegraded":"2024-04-25T08:55:47.516492+0000","last_fullsized":"2024-04-25T08:55:47.516492+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:33.341553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929504+0000","last_change":"2024-04-25T08:55:46.557531+0000","last_active":"2024-04-25T08:55:47.929504+0000","last_peered":"2024-04-25T08:55:47.929504+0000","last_clean":"2024-04-25T08:55:47.929504+0000","last_became_active":"2024-04-25T08:55:46.556752+0000","last_became_peered":"2024-04-25T08:55:46.556752+0000","last_unstale":"2024-04-25T08:55:47.929504+0000","last_undegraded":"2024-04-25T08:55:47.929504+0000","last_fullsized":"2024-04-25T08:55:47.929504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:07.053904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927239+0000","last_change":"2024-04-25T08:55:46.558405+0000","last_active":"2024-04-25T08:55:47.927239+0000","last_peered":"2024-04-25T08:55:47.927239+0000","last_clean":"2024-04-25T08:55:47.927239+0000","last_became_active":"2024-04-25T08:55:46.557106+0000","last_became_peered":"2024-04-25T08:55:46.557106+0000","last_unstale":"2024-04-25T08:55:47.927239+0000","last_undegraded":"2024-04-25T08:55:47.927239+0000","last_fullsized":"2024-04-25T08:55:47.927239+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:14.229282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926363+0000","last_change":"2024-04-25T08:55:46.551564+0000","last_active":"2024-04-25T08:55:47.926363+0000","last_peered":"2024-04-25T08:55:47.926363+0000","last_clean":"2024-04-25T08:55:47.926363+0000","last_became_active":"2024-04-25T08:55:46.550835+0000","last_became_peered":"2024-04-25T08:55:46.550835+0000","last_unstale":"2024-04-25T08:55:47.926363+0000","last_undegraded":"2024-04-25T08:55:47.926363+0000","last_fullsized":"2024-04-25T08:55:47.926363+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:33.737868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515067+0000","last_change":"2024-04-25T08:55:46.514783+0000","last_active":"2024-04-25T08:55:47.515067+0000","last_peered":"2024-04-25T08:55:47.515067+0000","last_clean":"2024-04-25T08:55:47.515067+0000","last_became_active":"2024-04-25T08:55:46.513951+0000","last_became_peered":"2024-04-25T08:55:46.513951+0000","last_unstale":"2024-04-25T08:55:47.515067+0000","last_undegraded":"2024-04-25T08:55:47.515067+0000","last_fullsized":"2024-04-25T08:55:47.515067+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:32.326436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926459+0000","last_change":"2024-04-25T08:55:46.550278+0000","last_active":"2024-04-25T08:55:47.926459+0000","last_peered":"2024-04-25T08:55:47.926459+0000","last_clean":"2024-04-25T08:55:47.926459+0000","last_became_active":"2024-04-25T08:55:46.549390+0000","last_became_peered":"2024-04-25T08:55:46.549390+0000","last_unstale":"2024-04-25T08:55:47.926459+0000","last_undegraded":"2024-04-25T08:55:47.926459+0000","last_fullsized":"2024-04-25T08:55:47.926459+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:34.440044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516302+0000","last_change":"2024-04-25T08:55:46.547099+0000","last_active":"2024-04-25T08:55:47.516302+0000","last_peered":"2024-04-25T08:55:47.516302+0000","last_clean":"2024-04-25T08:55:47.516302+0000","last_became_active":"2024-04-25T08:55:46.546395+0000","last_became_peered":"2024-04-25T08:55:46.546395+0000","last_unstale":"2024-04-25T08:55:47.516302+0000","last_undegraded":"2024-04-25T08:55:47.516302+0000","last_fullsized":"2024-04-25T08:55:47.516302+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:34.060227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929393+0000","last_change":"2024-04-25T08:55:45.867969+0000","last_active":"2024-04-25T08:55:47.929393+0000","last_peered":"2024-04-25T08:55:47.929393+0000","last_clean":"2024-04-25T08:55:47.929393+0000","last_became_active":"2024-04-25T08:55:45.867621+0000","last_became_peered":"2024-04-25T08:55:45.867621+0000","last_unstale":"2024-04-25T08:55:47.929393+0000","last_undegraded":"2024-04-25T08:55:47.929393+0000","last_fullsized":"2024-04-25T08:55:47.929393+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:50.359008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516954+0000","last_change":"2024-04-25T08:55:46.526885+0000","last_active":"2024-04-25T08:55:47.516954+0000","last_peered":"2024-04-25T08:55:47.516954+0000","last_clean":"2024-04-25T08:55:47.516954+0000","last_became_active":"2024-04-25T08:55:46.525688+0000","last_became_peered":"2024-04-25T08:55:46.525688+0000","last_unstale":"2024-04-25T08:55:47.516954+0000","last_undegraded":"2024-04-25T08:55:47.516954+0000","last_fullsized":"2024-04-25T08:55:47.516954+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:34.060227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929068+0000","last_change":"2024-04-25T08:55:45.861921+0000","last_active":"2024-04-25T08:55:47.929068+0000","last_peered":"2024-04-25T08:55:47.929068+0000","last_clean":"2024-04-25T08:55:47.929068+0000","last_became_active":"2024-04-25T08:55:45.861609+0000","last_became_peered":"2024-04-25T08:55:45.861609+0000","last_unstale":"2024-04-25T08:55:47.929068+0000","last_undegraded":"2024-04-25T08:55:47.929068+0000","last_fullsized":"2024-04-25T08:55:47.929068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.528135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928915+0000","last_change":"2024-04-25T08:55:46.549759+0000","last_active":"2024-04-25T08:55:47.928915+0000","last_peered":"2024-04-25T08:55:47.928915+0000","last_clean":"2024-04-25T08:55:47.928915+0000","last_became_active":"2024-04-25T08:55:46.549225+0000","last_became_peered":"2024-04-25T08:55:46.549225+0000","last_unstale":"2024-04-25T08:55:47.928915+0000","last_undegraded":"2024-04-25T08:55:47.928915+0000","last_fullsized":"2024-04-25T08:55:47.928915+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:43.094059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927802+0000","last_change":"2024-04-25T08:55:45.859351+0000","last_active":"2024-04-25T08:55:47.927802+0000","last_peered":"2024-04-25T08:55:47.927802+0000","last_clean":"2024-04-25T08:55:47.927802+0000","last_became_active":"2024-04-25T08:55:45.859081+0000","last_became_peered":"2024-04-25T08:55:45.859081+0000","last_unstale":"2024-04-25T08:55:47.927802+0000","last_undegraded":"2024-04-25T08:55:47.927802+0000","last_fullsized":"2024-04-25T08:55:47.927802+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:09.459388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929027+0000","last_change":"2024-04-25T08:55:46.557632+0000","last_active":"2024-04-25T08:55:47.929027+0000","last_peered":"2024-04-25T08:55:47.929027+0000","last_clean":"2024-04-25T08:55:47.929027+0000","last_became_active":"2024-04-25T08:55:46.549992+0000","last_became_peered":"2024-04-25T08:55:46.549992+0000","last_unstale":"2024-04-25T08:55:47.929027+0000","last_undegraded":"2024-04-25T08:55:47.929027+0000","last_fullsized":"2024-04-25T08:55:47.929027+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:38.300578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928321+0000","last_change":"2024-04-25T08:55:45.836302+0000","last_active":"2024-04-25T08:55:47.928321+0000","last_peered":"2024-04-25T08:55:47.928321+0000","last_clean":"2024-04-25T08:55:47.928321+0000","last_became_active":"2024-04-25T08:55:45.836023+0000","last_became_peered":"2024-04-25T08:55:45.836023+0000","last_unstale":"2024-04-25T08:55:47.928321+0000","last_undegraded":"2024-04-25T08:55:47.928321+0000","last_fullsized":"2024-04-25T08:55:47.928321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:32.825742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515183+0000","last_change":"2024-04-25T08:55:46.531579+0000","last_active":"2024-04-25T08:55:47.515183+0000","last_peered":"2024-04-25T08:55:47.515183+0000","last_clean":"2024-04-25T08:55:47.515183+0000","last_became_active":"2024-04-25T08:55:46.530758+0000","last_became_peered":"2024-04-25T08:55:46.530758+0000","last_unstale":"2024-04-25T08:55:47.515183+0000","last_undegraded":"2024-04-25T08:55:47.515183+0000","last_fullsized":"2024-04-25T08:55:47.515183+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:40.387050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929007+0000","last_change":"2024-04-25T08:55:45.839986+0000","last_active":"2024-04-25T08:55:47.929007+0000","last_peered":"2024-04-25T08:55:47.929007+0000","last_clean":"2024-04-25T08:55:47.929007+0000","last_became_active":"2024-04-25T08:55:45.839521+0000","last_became_peered":"2024-04-25T08:55:45.839521+0000","last_unstale":"2024-04-25T08:55:47.929007+0000","last_undegraded":"2024-04-25T08:55:47.929007+0000","last_fullsized":"2024-04-25T08:55:47.929007+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:27.101381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928039+0000","last_change":"2024-04-25T08:55:46.541211+0000","last_active":"2024-04-25T08:55:47.928039+0000","last_peered":"2024-04-25T08:55:47.928039+0000","last_clean":"2024-04-25T08:55:47.928039+0000","last_became_active":"2024-04-25T08:55:46.540617+0000","last_became_peered":"2024-04-25T08:55:46.540617+0000","last_unstale":"2024-04-25T08:55:47.928039+0000","last_undegraded":"2024-04-25T08:55:47.928039+0000","last_fullsized":"2024-04-25T08:55:47.928039+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:53.544366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926648+0000","last_change":"2024-04-25T08:55:45.838676+0000","last_active":"2024-04-25T08:55:47.926648+0000","last_peered":"2024-04-25T08:55:47.926648+0000","last_clean":"2024-04-25T08:55:47.926648+0000","last_became_active":"2024-04-25T08:55:45.836467+0000","last_became_peered":"2024-04-25T08:55:45.836467+0000","last_unstale":"2024-04-25T08:55:47.926648+0000","last_undegraded":"2024-04-25T08:55:47.926648+0000","last_fullsized":"2024-04-25T08:55:47.926648+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.297281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515469+0000","last_change":"2024-04-25T08:55:46.550655+0000","last_active":"2024-04-25T08:55:47.515469+0000","last_peered":"2024-04-25T08:55:47.515469+0000","last_clean":"2024-04-25T08:55:47.515469+0000","last_became_active":"2024-04-25T08:55:46.550062+0000","last_became_peered":"2024-04-25T08:55:46.550062+0000","last_unstale":"2024-04-25T08:55:47.515469+0000","last_undegraded":"2024-04-25T08:55:47.515469+0000","last_fullsized":"2024-04-25T08:55:47.515469+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:09.965629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926752+0000","last_change":"2024-04-25T08:55:45.849357+0000","last_active":"2024-04-25T08:55:47.926752+0000","last_peered":"2024-04-25T08:55:47.926752+0000","last_clean":"2024-04-25T08:55:47.926752+0000","last_became_active":"2024-04-25T08:55:45.848617+0000","last_became_peered":"2024-04-25T08:55:45.848617+0000","last_unstale":"2024-04-25T08:55:47.926752+0000","last_undegraded":"2024-04-25T08:55:47.926752+0000","last_fullsized":"2024-04-25T08:55:47.926752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:18.161812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926772+0000","last_change":"2024-04-25T08:55:46.541838+0000","last_active":"2024-04-25T08:55:47.926772+0000","last_peered":"2024-04-25T08:55:47.926772+0000","last_clean":"2024-04-25T08:55:47.926772+0000","last_became_active":"2024-04-25T08:55:46.541140+0000","last_became_peered":"2024-04-25T08:55:46.541140+0000","last_unstale":"2024-04-25T08:55:47.926772+0000","last_undegraded":"2024-04-25T08:55:47.926772+0000","last_fullsized":"2024-04-25T08:55:47.926772+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:10.430318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.514926+0000","last_change":"2024-04-25T08:55:45.862024+0000","last_active":"2024-04-25T08:55:47.514926+0000","last_peered":"2024-04-25T08:55:47.514926+0000","last_clean":"2024-04-25T08:55:47.514926+0000","last_became_active":"2024-04-25T08:55:45.860893+0000","last_became_peered":"2024-04-25T08:55:45.860893+0000","last_unstale":"2024-04-25T08:55:47.514926+0000","last_undegraded":"2024-04-25T08:55:47.514926+0000","last_fullsized":"2024-04-25T08:55:47.514926+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:20.826306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928601+0000","last_change":"2024-04-25T08:55:46.547693+0000","last_active":"2024-04-25T08:55:47.928601+0000","last_peered":"2024-04-25T08:55:47.928601+0000","last_clean":"2024-04-25T08:55:47.928601+0000","last_became_active":"2024-04-25T08:55:46.546817+0000","last_became_peered":"2024-04-25T08:55:46.546817+0000","last_unstale":"2024-04-25T08:55:47.928601+0000","last_undegraded":"2024-04-25T08:55:47.928601+0000","last_fullsized":"2024-04-25T08:55:47.928601+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:46.787415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929209+0000","last_change":"2024-04-25T08:55:45.867789+0000","last_active":"2024-04-25T08:55:47.929209+0000","last_peered":"2024-04-25T08:55:47.929209+0000","last_clean":"2024-04-25T08:55:47.929209+0000","last_became_active":"2024-04-25T08:55:45.866828+0000","last_became_peered":"2024-04-25T08:55:45.866828+0000","last_unstale":"2024-04-25T08:55:47.929209+0000","last_undegraded":"2024-04-25T08:55:47.929209+0000","last_fullsized":"2024-04-25T08:55:47.929209+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:00.089017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927395+0000","last_change":"2024-04-25T08:55:46.539943+0000","last_active":"2024-04-25T08:55:47.927395+0000","last_peered":"2024-04-25T08:55:47.927395+0000","last_clean":"2024-04-25T08:55:47.927395+0000","last_became_active":"2024-04-25T08:55:46.539123+0000","last_became_peered":"2024-04-25T08:55:46.539123+0000","last_unstale":"2024-04-25T08:55:47.927395+0000","last_undegraded":"2024-04-25T08:55:47.927395+0000","last_fullsized":"2024-04-25T08:55:47.927395+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:23.583910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.517215+0000","last_change":"2024-04-25T08:55:45.847982+0000","last_active":"2024-04-25T08:55:47.517215+0000","last_peered":"2024-04-25T08:55:47.517215+0000","last_clean":"2024-04-25T08:55:47.517215+0000","last_became_active":"2024-04-25T08:55:45.846245+0000","last_became_peered":"2024-04-25T08:55:45.846245+0000","last_unstale":"2024-04-25T08:55:47.517215+0000","last_undegraded":"2024-04-25T08:55:47.517215+0000","last_fullsized":"2024-04-25T08:55:47.517215+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:20.826306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.517374+0000","last_change":"2024-04-25T08:55:46.534069+0000","last_active":"2024-04-25T08:55:47.517374+0000","last_peered":"2024-04-25T08:55:47.517374+0000","last_clean":"2024-04-25T08:55:47.517374+0000","last_became_active":"2024-04-25T08:55:46.529946+0000","last_became_peered":"2024-04-25T08:55:46.529946+0000","last_unstale":"2024-04-25T08:55:47.517374+0000","last_undegraded":"2024-04-25T08:55:47.517374+0000","last_fullsized":"2024-04-25T08:55:47.517374+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:53.544366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928087+0000","last_change":"2024-04-25T08:55:45.860638+0000","last_active":"2024-04-25T08:55:47.928087+0000","last_peered":"2024-04-25T08:55:47.928087+0000","last_clean":"2024-04-25T08:55:47.928087+0000","last_became_active":"2024-04-25T08:55:45.860019+0000","last_became_peered":"2024-04-25T08:55:45.860019+0000","last_unstale":"2024-04-25T08:55:47.928087+0000","last_undegraded":"2024-04-25T08:55:47.928087+0000","last_fullsized":"2024-04-25T08:55:47.928087+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:31.323139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.517111+0000","last_change":"2024-04-25T08:55:46.537372+0000","last_active":"2024-04-25T08:55:47.517111+0000","last_peered":"2024-04-25T08:55:47.517111+0000","last_clean":"2024-04-25T08:55:47.517111+0000","last_became_active":"2024-04-25T08:55:46.531983+0000","last_became_peered":"2024-04-25T08:55:46.531983+0000","last_unstale":"2024-04-25T08:55:47.517111+0000","last_undegraded":"2024-04-25T08:55:47.517111+0000","last_fullsized":"2024-04-25T08:55:47.517111+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:52.573081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929019+0000","last_change":"2024-04-25T08:55:45.839267+0000","last_active":"2024-04-25T08:55:47.929019+0000","last_peered":"2024-04-25T08:55:47.929019+0000","last_clean":"2024-04-25T08:55:47.929019+0000","last_became_active":"2024-04-25T08:55:45.838591+0000","last_became_peered":"2024-04-25T08:55:45.838591+0000","last_unstale":"2024-04-25T08:55:47.929019+0000","last_undegraded":"2024-04-25T08:55:47.929019+0000","last_fullsized":"2024-04-25T08:55:47.929019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:00.089017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515025+0000","last_change":"2024-04-25T08:55:46.555086+0000","last_active":"2024-04-25T08:55:47.515025+0000","last_peered":"2024-04-25T08:55:47.515025+0000","last_clean":"2024-04-25T08:55:47.515025+0000","last_became_active":"2024-04-25T08:55:46.554275+0000","last_became_peered":"2024-04-25T08:55:46.554275+0000","last_unstale":"2024-04-25T08:55:47.515025+0000","last_undegraded":"2024-04-25T08:55:47.515025+0000","last_fullsized":"2024-04-25T08:55:47.515025+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:51.008518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927258+0000","last_change":"2024-04-25T08:55:45.867429+0000","last_active":"2024-04-25T08:55:47.927258+0000","last_peered":"2024-04-25T08:55:47.927258+0000","last_clean":"2024-04-25T08:55:47.927258+0000","last_became_active":"2024-04-25T08:55:45.866476+0000","last_became_peered":"2024-04-25T08:55:45.866476+0000","last_unstale":"2024-04-25T08:55:47.927258+0000","last_undegraded":"2024-04-25T08:55:47.927258+0000","last_fullsized":"2024-04-25T08:55:47.927258+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:50.359008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927100+0000","last_change":"2024-04-25T08:55:46.539859+0000","last_active":"2024-04-25T08:55:47.927100+0000","last_peered":"2024-04-25T08:55:47.927100+0000","last_clean":"2024-04-25T08:55:47.927100+0000","last_became_active":"2024-04-25T08:55:46.539211+0000","last_became_peered":"2024-04-25T08:55:46.539211+0000","last_unstale":"2024-04-25T08:55:47.927100+0000","last_undegraded":"2024-04-25T08:55:47.927100+0000","last_fullsized":"2024-04-25T08:55:47.927100+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:14.229282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927040+0000","last_change":"2024-04-25T08:55:45.867756+0000","last_active":"2024-04-25T08:55:47.927040+0000","last_peered":"2024-04-25T08:55:47.927040+0000","last_clean":"2024-04-25T08:55:47.927040+0000","last_became_active":"2024-04-25T08:55:45.860605+0000","last_became_peered":"2024-04-25T08:55:45.860605+0000","last_unstale":"2024-04-25T08:55:47.927040+0000","last_undegraded":"2024-04-25T08:55:47.927040+0000","last_fullsized":"2024-04-25T08:55:47.927040+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:50.359008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515858+0000","last_change":"2024-04-25T08:55:46.533293+0000","last_active":"2024-04-25T08:55:47.515858+0000","last_peered":"2024-04-25T08:55:47.515858+0000","last_clean":"2024-04-25T08:55:47.515858+0000","last_became_active":"2024-04-25T08:55:46.532553+0000","last_became_peered":"2024-04-25T08:55:46.532553+0000","last_unstale":"2024-04-25T08:55:47.515858+0000","last_undegraded":"2024-04-25T08:55:47.515858+0000","last_fullsized":"2024-04-25T08:55:47.515858+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:33.341553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928331+0000","last_change":"2024-04-25T08:55:45.867068+0000","last_active":"2024-04-25T08:55:47.928331+0000","last_peered":"2024-04-25T08:55:47.928331+0000","last_clean":"2024-04-25T08:55:47.928331+0000","last_became_active":"2024-04-25T08:55:45.866196+0000","last_became_peered":"2024-04-25T08:55:45.866196+0000","last_unstale":"2024-04-25T08:55:47.928331+0000","last_undegraded":"2024-04-25T08:55:47.928331+0000","last_fullsized":"2024-04-25T08:55:47.928331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:30.377116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928286+0000","last_change":"2024-04-25T08:55:46.549689+0000","last_active":"2024-04-25T08:55:47.928286+0000","last_peered":"2024-04-25T08:55:47.928286+0000","last_clean":"2024-04-25T08:55:47.928286+0000","last_became_active":"2024-04-25T08:55:46.548980+0000","last_became_peered":"2024-04-25T08:55:46.548980+0000","last_unstale":"2024-04-25T08:55:47.928286+0000","last_undegraded":"2024-04-25T08:55:47.928286+0000","last_fullsized":"2024-04-25T08:55:47.928286+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:10.718524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515303+0000","last_change":"2024-04-25T08:55:45.862676+0000","last_active":"2024-04-25T08:55:47.515303+0000","last_peered":"2024-04-25T08:55:47.515303+0000","last_clean":"2024-04-25T08:55:47.515303+0000","last_became_active":"2024-04-25T08:55:45.861928+0000","last_became_peered":"2024-04-25T08:55:45.861928+0000","last_unstale":"2024-04-25T08:55:47.515303+0000","last_undegraded":"2024-04-25T08:55:47.515303+0000","last_fullsized":"2024-04-25T08:55:47.515303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:41.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927138+0000","last_change":"2024-04-25T08:55:46.557584+0000","last_active":"2024-04-25T08:55:47.927138+0000","last_peered":"2024-04-25T08:55:47.927138+0000","last_clean":"2024-04-25T08:55:47.927138+0000","last_became_active":"2024-04-25T08:55:46.556690+0000","last_became_peered":"2024-04-25T08:55:46.556690+0000","last_unstale":"2024-04-25T08:55:47.927138+0000","last_undegraded":"2024-04-25T08:55:47.927138+0000","last_fullsized":"2024-04-25T08:55:47.927138+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:07.053904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927959+0000","last_change":"2024-04-25T08:55:45.860638+0000","last_active":"2024-04-25T08:55:47.927959+0000","last_peered":"2024-04-25T08:55:47.927959+0000","last_clean":"2024-04-25T08:55:47.927959+0000","last_became_active":"2024-04-25T08:55:45.860008+0000","last_became_peered":"2024-04-25T08:55:45.860008+0000","last_unstale":"2024-04-25T08:55:47.927959+0000","last_undegraded":"2024-04-25T08:55:47.927959+0000","last_fullsized":"2024-04-25T08:55:47.927959+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:50.005220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516701+0000","last_change":"2024-04-25T08:55:46.519731+0000","last_active":"2024-04-25T08:55:47.516701+0000","last_peered":"2024-04-25T08:55:47.516701+0000","last_clean":"2024-04-25T08:55:47.516701+0000","last_became_active":"2024-04-25T08:55:46.519055+0000","last_became_peered":"2024-04-25T08:55:46.519055+0000","last_unstale":"2024-04-25T08:55:47.516701+0000","last_undegraded":"2024-04-25T08:55:47.516701+0000","last_fullsized":"2024-04-25T08:55:47.516701+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:52.573081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928914+0000","last_change":"2024-04-25T08:55:45.867699+0000","last_active":"2024-04-25T08:55:47.928914+0000","last_peered":"2024-04-25T08:55:47.928914+0000","last_clean":"2024-04-25T08:55:47.928914+0000","last_became_active":"2024-04-25T08:55:45.866911+0000","last_became_peered":"2024-04-25T08:55:45.866911+0000","last_unstale":"2024-04-25T08:55:47.928914+0000","last_undegraded":"2024-04-25T08:55:47.928914+0000","last_fullsized":"2024-04-25T08:55:47.928914+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:50.005220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516942+0000","last_change":"2024-04-25T08:55:46.528858+0000","last_active":"2024-04-25T08:55:47.516942+0000","last_peered":"2024-04-25T08:55:47.516942+0000","last_clean":"2024-04-25T08:55:47.516942+0000","last_became_active":"2024-04-25T08:55:46.516198+0000","last_became_peered":"2024-04-25T08:55:46.516198+0000","last_unstale":"2024-04-25T08:55:47.516942+0000","last_undegraded":"2024-04-25T08:55:47.516942+0000","last_fullsized":"2024-04-25T08:55:47.516942+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:58.970972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928479+0000","last_change":"2024-04-25T08:55:45.866514+0000","last_active":"2024-04-25T08:55:47.928479+0000","last_peered":"2024-04-25T08:55:47.928479+0000","last_clean":"2024-04-25T08:55:47.928479+0000","last_became_active":"2024-04-25T08:55:45.864232+0000","last_became_peered":"2024-04-25T08:55:45.864232+0000","last_unstale":"2024-04-25T08:55:47.928479+0000","last_undegraded":"2024-04-25T08:55:47.928479+0000","last_fullsized":"2024-04-25T08:55:47.928479+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:08.962332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516162+0000","last_change":"2024-04-25T08:55:46.546467+0000","last_active":"2024-04-25T08:55:47.516162+0000","last_peered":"2024-04-25T08:55:47.516162+0000","last_clean":"2024-04-25T08:55:47.516162+0000","last_became_active":"2024-04-25T08:55:46.545650+0000","last_became_peered":"2024-04-25T08:55:46.545650+0000","last_unstale":"2024-04-25T08:55:47.516162+0000","last_undegraded":"2024-04-25T08:55:47.516162+0000","last_fullsized":"2024-04-25T08:55:47.516162+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:05.492858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927395+0000","last_change":"2024-04-25T08:55:45.844352+0000","last_active":"2024-04-25T08:55:47.927395+0000","last_peered":"2024-04-25T08:55:47.927395+0000","last_clean":"2024-04-25T08:55:47.927395+0000","last_became_active":"2024-04-25T08:55:45.843827+0000","last_became_peered":"2024-04-25T08:55:45.843827+0000","last_unstale":"2024-04-25T08:55:47.927395+0000","last_undegraded":"2024-04-25T08:55:47.927395+0000","last_fullsized":"2024-04-25T08:55:47.927395+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:51.336610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928702+0000","last_change":"2024-04-25T08:55:46.549081+0000","last_active":"2024-04-25T08:55:47.928702+0000","last_peered":"2024-04-25T08:55:47.928702+0000","last_clean":"2024-04-25T08:55:47.928702+0000","last_became_active":"2024-04-25T08:55:46.548273+0000","last_became_peered":"2024-04-25T08:55:46.548273+0000","last_unstale":"2024-04-25T08:55:47.928702+0000","last_undegraded":"2024-04-25T08:55:47.928702+0000","last_fullsized":"2024-04-25T08:55:47.928702+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:58.588294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927553+0000","last_change":"2024-04-25T08:55:45.849231+0000","last_active":"2024-04-25T08:55:47.927553+0000","last_peered":"2024-04-25T08:55:47.927553+0000","last_clean":"2024-04-25T08:55:47.927553+0000","last_became_active":"2024-04-25T08:55:45.848819+0000","last_became_peered":"2024-04-25T08:55:45.848819+0000","last_unstale":"2024-04-25T08:55:47.927553+0000","last_undegraded":"2024-04-25T08:55:47.927553+0000","last_fullsized":"2024-04-25T08:55:47.927553+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:42.062850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926403+0000","last_change":"2024-04-25T08:55:46.548727+0000","last_active":"2024-04-25T08:55:47.926403+0000","last_peered":"2024-04-25T08:55:47.926403+0000","last_clean":"2024-04-25T08:55:47.926403+0000","last_became_active":"2024-04-25T08:55:46.548000+0000","last_became_peered":"2024-04-25T08:55:46.548000+0000","last_unstale":"2024-04-25T08:55:47.926403+0000","last_undegraded":"2024-04-25T08:55:47.926403+0000","last_fullsized":"2024-04-25T08:55:47.926403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:07.053904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926617+0000","last_change":"2024-04-25T08:55:45.867156+0000","last_active":"2024-04-25T08:55:47.926617+0000","last_peered":"2024-04-25T08:55:47.926617+0000","last_clean":"2024-04-25T08:55:47.926617+0000","last_became_active":"2024-04-25T08:55:45.859785+0000","last_became_peered":"2024-04-25T08:55:45.859785+0000","last_unstale":"2024-04-25T08:55:47.926617+0000","last_undegraded":"2024-04-25T08:55:47.926617+0000","last_fullsized":"2024-04-25T08:55:47.926617+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:00.089017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516244+0000","last_change":"2024-04-25T08:55:46.544509+0000","last_active":"2024-04-25T08:55:47.516244+0000","last_peered":"2024-04-25T08:55:47.516244+0000","last_clean":"2024-04-25T08:55:47.516244+0000","last_became_active":"2024-04-25T08:55:46.535006+0000","last_became_peered":"2024-04-25T08:55:46.535006+0000","last_unstale":"2024-04-25T08:55:47.516244+0000","last_undegraded":"2024-04-25T08:55:47.516244+0000","last_fullsized":"2024-04-25T08:55:47.516244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:58.970972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928716+0000","last_change":"2024-04-25T08:55:45.865786+0000","last_active":"2024-04-25T08:55:47.928716+0000","last_peered":"2024-04-25T08:55:47.928716+0000","last_clean":"2024-04-25T08:55:47.928716+0000","last_became_active":"2024-04-25T08:55:45.865455+0000","last_became_peered":"2024-04-25T08:55:45.865455+0000","last_unstale":"2024-04-25T08:55:47.928716+0000","last_undegraded":"2024-04-25T08:55:47.928716+0000","last_fullsized":"2024-04-25T08:55:47.928716+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:00.332066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928216+0000","last_change":"2024-04-25T08:55:46.543496+0000","last_active":"2024-04-25T08:55:47.928216+0000","last_peered":"2024-04-25T08:55:47.928216+0000","last_clean":"2024-04-25T08:55:47.928216+0000","last_became_active":"2024-04-25T08:55:46.542710+0000","last_became_peered":"2024-04-25T08:55:46.542710+0000","last_unstale":"2024-04-25T08:55:47.928216+0000","last_undegraded":"2024-04-25T08:55:47.928216+0000","last_fullsized":"2024-04-25T08:55:47.928216+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:04.547168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928300+0000","last_change":"2024-04-25T08:55:45.875396+0000","last_active":"2024-04-25T08:55:47.928300+0000","last_peered":"2024-04-25T08:55:47.928300+0000","last_clean":"2024-04-25T08:55:47.928300+0000","last_became_active":"2024-04-25T08:55:45.874752+0000","last_became_peered":"2024-04-25T08:55:45.874752+0000","last_unstale":"2024-04-25T08:55:47.928300+0000","last_undegraded":"2024-04-25T08:55:47.928300+0000","last_fullsized":"2024-04-25T08:55:47.928300+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:31.323139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515293+0000","last_change":"2024-04-25T08:55:46.535009+0000","last_active":"2024-04-25T08:55:47.515293+0000","last_peered":"2024-04-25T08:55:47.515293+0000","last_clean":"2024-04-25T08:55:47.515293+0000","last_became_active":"2024-04-25T08:55:46.534192+0000","last_became_peered":"2024-04-25T08:55:46.534192+0000","last_unstale":"2024-04-25T08:55:47.515293+0000","last_undegraded":"2024-04-25T08:55:47.515293+0000","last_fullsized":"2024-04-25T08:55:47.515293+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:01.115213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928539+0000","last_change":"2024-04-25T08:55:45.869353+0000","last_active":"2024-04-25T08:55:47.928539+0000","last_peered":"2024-04-25T08:55:47.928539+0000","last_clean":"2024-04-25T08:55:47.928539+0000","last_became_active":"2024-04-25T08:55:45.868748+0000","last_became_peered":"2024-04-25T08:55:45.868748+0000","last_unstale":"2024-04-25T08:55:47.928539+0000","last_undegraded":"2024-04-25T08:55:47.928539+0000","last_fullsized":"2024-04-25T08:55:47.928539+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:08.962332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928402+0000","last_change":"2024-04-25T08:55:46.544745+0000","last_active":"2024-04-25T08:55:47.928402+0000","last_peered":"2024-04-25T08:55:47.928402+0000","last_clean":"2024-04-25T08:55:47.928402+0000","last_became_active":"2024-04-25T08:55:46.543987+0000","last_became_peered":"2024-04-25T08:55:46.543987+0000","last_unstale":"2024-04-25T08:55:47.928402+0000","last_undegraded":"2024-04-25T08:55:47.928402+0000","last_fullsized":"2024-04-25T08:55:47.928402+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:33.829039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928069+0000","last_change":"2024-04-25T08:55:45.859531+0000","last_active":"2024-04-25T08:55:47.928069+0000","last_peered":"2024-04-25T08:55:47.928069+0000","last_clean":"2024-04-25T08:55:47.928069+0000","last_became_active":"2024-04-25T08:55:45.859051+0000","last_became_peered":"2024-04-25T08:55:45.859051+0000","last_unstale":"2024-04-25T08:55:47.928069+0000","last_undegraded":"2024-04-25T08:55:47.928069+0000","last_fullsized":"2024-04-25T08:55:47.928069+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.297281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.517870+0000","last_change":"2024-04-25T08:55:46.533242+0000","last_active":"2024-04-25T08:55:47.517870+0000","last_peered":"2024-04-25T08:55:47.517870+0000","last_clean":"2024-04-25T08:55:47.517870+0000","last_became_active":"2024-04-25T08:55:46.532509+0000","last_became_peered":"2024-04-25T08:55:46.532509+0000","last_unstale":"2024-04-25T08:55:47.517870+0000","last_undegraded":"2024-04-25T08:55:47.517870+0000","last_fullsized":"2024-04-25T08:55:47.517870+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:10.718524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929280+0000","last_change":"2024-04-25T08:55:45.838955+0000","last_active":"2024-04-25T08:55:47.929280+0000","last_peered":"2024-04-25T08:55:47.929280+0000","last_clean":"2024-04-25T08:55:47.929280+0000","last_became_active":"2024-04-25T08:55:45.838141+0000","last_became_peered":"2024-04-25T08:55:45.838141+0000","last_unstale":"2024-04-25T08:55:47.929280+0000","last_undegraded":"2024-04-25T08:55:47.929280+0000","last_fullsized":"2024-04-25T08:55:47.929280+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:50.005220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515037+0000","last_change":"2024-04-25T08:55:46.534568+0000","last_active":"2024-04-25T08:55:47.515037+0000","last_peered":"2024-04-25T08:55:47.515037+0000","last_clean":"2024-04-25T08:55:47.515037+0000","last_became_active":"2024-04-25T08:55:46.533790+0000","last_became_peered":"2024-04-25T08:55:46.533790+0000","last_unstale":"2024-04-25T08:55:47.515037+0000","last_undegraded":"2024-04-25T08:55:47.515037+0000","last_fullsized":"2024-04-25T08:55:47.515037+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:32.326436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927612+0000","last_change":"2024-04-25T08:55:45.867890+0000","last_active":"2024-04-25T08:55:47.927612+0000","last_peered":"2024-04-25T08:55:47.927612+0000","last_clean":"2024-04-25T08:55:47.927612+0000","last_became_active":"2024-04-25T08:55:45.866992+0000","last_became_peered":"2024-04-25T08:55:45.866992+0000","last_unstale":"2024-04-25T08:55:47.927612+0000","last_undegraded":"2024-04-25T08:55:47.927612+0000","last_fullsized":"2024-04-25T08:55:47.927612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:50.697232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515471+0000","last_change":"2024-04-25T08:55:46.551638+0000","last_active":"2024-04-25T08:55:47.515471+0000","last_peered":"2024-04-25T08:55:47.515471+0000","last_clean":"2024-04-25T08:55:47.515471+0000","last_became_active":"2024-04-25T08:55:46.550894+0000","last_became_peered":"2024-04-25T08:55:46.550894+0000","last_unstale":"2024-04-25T08:55:47.515471+0000","last_undegraded":"2024-04-25T08:55:47.515471+0000","last_fullsized":"2024-04-25T08:55:47.515471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:09.965629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926921+0000","last_change":"2024-04-25T08:55:45.836855+0000","last_active":"2024-04-25T08:55:47.926921+0000","last_peered":"2024-04-25T08:55:47.926921+0000","last_clean":"2024-04-25T08:55:47.926921+0000","last_became_active":"2024-04-25T08:55:45.836441+0000","last_became_peered":"2024-04-25T08:55:45.836441+0000","last_unstale":"2024-04-25T08:55:47.926921+0000","last_undegraded":"2024-04-25T08:55:47.926921+0000","last_fullsized":"2024-04-25T08:55:47.926921+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:32.825742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928218+0000","last_change":"2024-04-25T08:55:46.541655+0000","last_active":"2024-04-25T08:55:47.928218+0000","last_peered":"2024-04-25T08:55:47.928218+0000","last_clean":"2024-04-25T08:55:47.928218+0000","last_became_active":"2024-04-25T08:55:46.540890+0000","last_became_peered":"2024-04-25T08:55:46.540890+0000","last_unstale":"2024-04-25T08:55:47.928218+0000","last_undegraded":"2024-04-25T08:55:47.928218+0000","last_fullsized":"2024-04-25T08:55:47.928218+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:30.953107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927015+0000","last_change":"2024-04-25T08:55:45.848906+0000","last_active":"2024-04-25T08:55:47.927015+0000","last_peered":"2024-04-25T08:55:47.927015+0000","last_clean":"2024-04-25T08:55:47.927015+0000","last_became_active":"2024-04-25T08:55:45.848091+0000","last_became_peered":"2024-04-25T08:55:45.848091+0000","last_unstale":"2024-04-25T08:55:47.927015+0000","last_undegraded":"2024-04-25T08:55:47.927015+0000","last_fullsized":"2024-04-25T08:55:47.927015+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:11.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515089+0000","last_change":"2024-04-25T08:55:46.530967+0000","last_active":"2024-04-25T08:55:47.515089+0000","last_peered":"2024-04-25T08:55:47.515089+0000","last_clean":"2024-04-25T08:55:47.515089+0000","last_became_active":"2024-04-25T08:55:46.530235+0000","last_became_peered":"2024-04-25T08:55:46.530235+0000","last_unstale":"2024-04-25T08:55:47.515089+0000","last_undegraded":"2024-04-25T08:55:47.515089+0000","last_fullsized":"2024-04-25T08:55:47.515089+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:02.980990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927575+0000","last_change":"2024-04-25T08:55:45.853704+0000","last_active":"2024-04-25T08:55:47.927575+0000","last_peered":"2024-04-25T08:55:47.927575+0000","last_clean":"2024-04-25T08:55:47.927575+0000","last_became_active":"2024-04-25T08:55:45.853416+0000","last_became_peered":"2024-04-25T08:55:45.853416+0000","last_unstale":"2024-04-25T08:55:47.927575+0000","last_undegraded":"2024-04-25T08:55:47.927575+0000","last_fullsized":"2024-04-25T08:55:47.927575+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:09.049331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.517533+0000","last_change":"2024-04-25T08:55:46.533639+0000","last_active":"2024-04-25T08:55:47.517533+0000","last_peered":"2024-04-25T08:55:47.517533+0000","last_clean":"2024-04-25T08:55:47.517533+0000","last_became_active":"2024-04-25T08:55:46.532911+0000","last_became_peered":"2024-04-25T08:55:46.532911+0000","last_unstale":"2024-04-25T08:55:47.517533+0000","last_undegraded":"2024-04-25T08:55:47.517533+0000","last_fullsized":"2024-04-25T08:55:47.517533+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:30.953107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926370+0000","last_change":"2024-04-25T08:55:45.839990+0000","last_active":"2024-04-25T08:55:47.926370+0000","last_peered":"2024-04-25T08:55:47.926370+0000","last_clean":"2024-04-25T08:55:47.926370+0000","last_became_active":"2024-04-25T08:55:45.839607+0000","last_became_peered":"2024-04-25T08:55:45.839607+0000","last_unstale":"2024-04-25T08:55:47.926370+0000","last_undegraded":"2024-04-25T08:55:47.926370+0000","last_fullsized":"2024-04-25T08:55:47.926370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:39.887774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":260,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927212+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:55:47.927212+0000","last_peered":"2024-04-25T08:55:47.927212+0000","last_clean":"2024-04-25T08:55:47.927212+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:55:47.927212+0000","last_undegraded":"2024-04-25T08:55:47.927212+0000","last_fullsized":"2024-04-25T08:55:47.927212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1445888,"data_stored":971864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":839680,"data_stored":380754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":839680,"data_stored":380742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":847872,"data_stored":382332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":380720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":380720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":851968,"data_stored":376404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1429504,"data_stored":971088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:55:50.871 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:55:51.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf47aa640 1 Processor -- start 2024-04-25T08:55:51.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf47aa640 1 -- start start 2024-04-25T08:55:51.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf47aa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efdec09a480 0x7efdec0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf47aa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efdec0cfb80 0x7efdec0cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf47aa640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdec0d04d0 con 0x7efdec15b950 2024-04-25T08:55:51.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf47aa640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdec15bd30 con 0x7efdec09a480 2024-04-25T08:55:51.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf47aa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdec0cc0d0 con 0x7efdec0cfb80 2024-04-25T08:55:51.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf1d45640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efdec09a480 0x7efdec0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf1d45640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efdec09a480 0x7efdec0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57136/0 (socket says 172.21.15.37:57136) 2024-04-25T08:55:51.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf1d45640 1 -- 172.21.15.37:0/3929592626 learned_addr learned my addr 172.21.15.37:0/3929592626 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:51.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf2d47640 1 --2- 172.21.15.37:0/3929592626 >> v2:172.21.15.203:3300/0 conn(0x7efdec0cfb80 0x7efdec0cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf1d45640 1 -- 172.21.15.37:0/3929592626 >> v2:172.21.15.203:3300/0 conn(0x7efdec0cfb80 msgr2=0x7efdec0cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:51.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf1d45640 1 --2- 172.21.15.37:0/3929592626 >> v2:172.21.15.203:3300/0 conn(0x7efdec0cfb80 0x7efdec0cff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf1d45640 1 -- 172.21.15.37:0/3929592626 >> v1:172.21.15.203:6789/0 conn(0x7efdec15b950 legacy=0x7efdec0cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:51.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.034+0000 7efdf1d45640 1 -- 172.21.15.37:0/3929592626 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec09ab10 con 0x7efdec09a480 2024-04-25T08:55:51.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1d45640 1 --2- 172.21.15.37:0/3929592626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdec09a480 0x7efdec0cce50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efdec0b3bd0 tx=0x7efde801c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c013165b4f958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:51.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1544640 1 -- 172.21.15.37:0/3929592626 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80b0070 con 0x7efdec09a480 2024-04-25T08:55:51.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1544640 1 -- 172.21.15.37:0/3929592626 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde80ab600 con 0x7efdec09a480 2024-04-25T08:55:51.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1544640 1 -- 172.21.15.37:0/3929592626 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80ab9d0 con 0x7efdec09a480 2024-04-25T08:55:51.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 -- 172.21.15.37:0/3929592626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdec09a480 msgr2=0x7efdec0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 --2- 172.21.15.37:0/3929592626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdec09a480 0x7efdec0cce50 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7efdec0b3bd0 tx=0x7efde801c300 comp rx=0 tx=0).stop 2024-04-25T08:55:51.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 -- 172.21.15.37:0/3929592626 shutdown_connections 2024-04-25T08:55:51.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 --2- 172.21.15.37:0/3929592626 >> v2:172.21.15.203:3300/0 conn(0x7efdec0cfb80 0x7efdec0cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 --2- 172.21.15.37:0/3929592626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdec09a480 0x7efdec0cce50 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 -- 172.21.15.37:0/3929592626 >> 172.21.15.37:0/3929592626 conn(0x7efdec0b30b0 msgr2=0x7efdec0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:51.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 -- 172.21.15.37:0/3929592626 shutdown_connections 2024-04-25T08:55:51.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 -- 172.21.15.37:0/3929592626 wait complete. 2024-04-25T08:55:51.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 Processor -- start 2024-04-25T08:55:51.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 -- start start 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdec09a480 0x7efdec0c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdec0cfb80 0x7efdec151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdec15b950 0x7efdec0b5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec0cd390 con 0x7efdec0cfb80 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec06faa0 con 0x7efdec09a480 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdec151e60 con 0x7efdec15b950 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1d45640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdec0cfb80 0x7efdec151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1d45640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdec0cfb80 0x7efdec151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57146/0 (socket says 172.21.15.37:57146) 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1d45640 1 -- 172.21.15.37:0/4240677726 learned_addr learned my addr 172.21.15.37:0/4240677726 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1d45640 1 -- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdec15b950 msgr2=0x7efdec0b5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1d45640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdec15b950 0x7efdec0b5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf2546640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdec09a480 0x7efdec0c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1d45640 1 -- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdec09a480 msgr2=0x7efdec0c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1d45640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdec09a480 0x7efdec0c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1d45640 1 -- 172.21.15.37:0/4240677726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdec0c7a50 con 0x7efdec0cfb80 2024-04-25T08:55:51.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf1d45640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdec0cfb80 0x7efdec151340 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7efde80ae040 tx=0x7efde80190a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:51.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efddeffd640 1 -- 172.21.15.37:0/4240677726 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80b9070 con 0x7efdec0cfb80 2024-04-25T08:55:51.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efddeffd640 1 -- 172.21.15.37:0/4240677726 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efde80b4600 con 0x7efdec0cfb80 2024-04-25T08:55:51.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.038+0000 7efdf47aa640 1 -- 172.21.15.37:0/4240677726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec0c7d30 con 0x7efdec0cfb80 2024-04-25T08:55:51.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.042+0000 7efddeffd640 1 -- 172.21.15.37:0/4240677726 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efde80b0040 con 0x7efdec0cfb80 2024-04-25T08:55:51.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.042+0000 7efdf47aa640 1 -- 172.21.15.37:0/4240677726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec0b5760 con 0x7efdec0cfb80 2024-04-25T08:55:51.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.042+0000 7efdf47aa640 1 -- 172.21.15.37:0/4240677726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec151fe0 con 0x7efdec0cfb80 2024-04-25T08:55:51.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.042+0000 7efddeffd640 1 -- 172.21.15.37:0/4240677726 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efde80bc070 con 0x7efdec0cfb80 2024-04-25T08:55:51.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.042+0000 7efddeffd640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efdd0066f90 0x7efdd0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.042+0000 7efddeffd640 1 -- 172.21.15.37:0/4240677726 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efde811e2a0 con 0x7efdec0cfb80 2024-04-25T08:55:51.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.042+0000 7efdf2546640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efdd0066f90 0x7efdd0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.042+0000 7efdf2546640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efdd0066f90 0x7efdd0069450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efdec157c30 tx=0x7efde0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:51.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.046+0000 7efddeffd640 1 -- 172.21.15.37:0/4240677726 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efdec0d04d0 con 0x7efdec0cfb80 2024-04-25T08:55:51.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.194+0000 7efdf47aa640 1 -- 172.21.15.37:0/4240677726 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efdec04efa0 con 0x7efdd0066f90 2024-04-25T08:55:51.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.214+0000 7efddeffd640 1 -- 172.21.15.37:0/4240677726 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252016 (secure 0 0 0) 0x7efdec04efa0 con 0x7efdd0066f90 2024-04-25T08:55:51.206 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:51.208 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:55:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 -- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efdd0066f90 msgr2=0x7efdd0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efdd0066f90 0x7efdd0069450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7efdec157c30 tx=0x7efde0008040 comp rx=0 tx=0).stop 2024-04-25T08:55:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 -- 172.21.15.37:0/4240677726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdec0cfb80 msgr2=0x7efdec151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdec0cfb80 0x7efdec151340 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7efde80ae040 tx=0x7efde80190a0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 -- 172.21.15.37:0/4240677726 shutdown_connections 2024-04-25T08:55:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efdd0066f90 0x7efdd0069450 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efdec15b950 0x7efdec0b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efdec0cfb80 0x7efdec151340 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 --2- 172.21.15.37:0/4240677726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efdec09a480 0x7efdec0c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 -- 172.21.15.37:0/4240677726 >> 172.21.15.37:0/4240677726 conn(0x7efdec0b30b0 msgr2=0x7efdec0b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:51.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 -- 172.21.15.37:0/4240677726 shutdown_connections 2024-04-25T08:55:51.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.218+0000 7efdf47aa640 1 -- 172.21.15.37:0/4240677726 wait complete. 2024-04-25T08:55:51.223 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-04-25T08:55:51.189745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":118,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":248900,"kb_used_data":7740,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725820,"statfs":{"total":773094113280,"available":772839239680,"internally_reserved":0,"allocated":7925760,"data_stored":4224624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002008"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929113+0000","last_change":"2024-04-25T08:55:46.534398+0000","last_active":"2024-04-25T08:55:47.929113+0000","last_peered":"2024-04-25T08:55:47.929113+0000","last_clean":"2024-04-25T08:55:47.929113+0000","last_became_active":"2024-04-25T08:55:46.533596+0000","last_became_peered":"2024-04-25T08:55:46.533596+0000","last_unstale":"2024-04-25T08:55:47.929113+0000","last_undegraded":"2024-04-25T08:55:47.929113+0000","last_fullsized":"2024-04-25T08:55:47.929113+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:20.962033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929523+0000","last_change":"2024-04-25T08:55:46.539652+0000","last_active":"2024-04-25T08:55:47.929523+0000","last_peered":"2024-04-25T08:55:47.929523+0000","last_clean":"2024-04-25T08:55:47.929523+0000","last_became_active":"2024-04-25T08:55:46.538929+0000","last_became_peered":"2024-04-25T08:55:46.538929+0000","last_unstale":"2024-04-25T08:55:47.929523+0000","last_undegraded":"2024-04-25T08:55:47.929523+0000","last_fullsized":"2024-04-25T08:55:47.929523+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:14.229282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926911+0000","last_change":"2024-04-25T08:55:46.560306+0000","last_active":"2024-04-25T08:55:47.926911+0000","last_peered":"2024-04-25T08:55:47.926911+0000","last_clean":"2024-04-25T08:55:47.926911+0000","last_became_active":"2024-04-25T08:55:46.559561+0000","last_became_peered":"2024-04-25T08:55:46.559561+0000","last_unstale":"2024-04-25T08:55:47.926911+0000","last_undegraded":"2024-04-25T08:55:47.926911+0000","last_fullsized":"2024-04-25T08:55:47.926911+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:20.962033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516627+0000","last_change":"2024-04-25T08:55:46.544064+0000","last_active":"2024-04-25T08:55:47.516627+0000","last_peered":"2024-04-25T08:55:47.516627+0000","last_clean":"2024-04-25T08:55:47.516627+0000","last_became_active":"2024-04-25T08:55:46.542748+0000","last_became_peered":"2024-04-25T08:55:46.542748+0000","last_unstale":"2024-04-25T08:55:47.516627+0000","last_undegraded":"2024-04-25T08:55:47.516627+0000","last_fullsized":"2024-04-25T08:55:47.516627+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:05.492858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515707+0000","last_change":"2024-04-25T08:55:46.518087+0000","last_active":"2024-04-25T08:55:47.515707+0000","last_peered":"2024-04-25T08:55:47.515707+0000","last_clean":"2024-04-25T08:55:47.515707+0000","last_became_active":"2024-04-25T08:55:46.517109+0000","last_became_peered":"2024-04-25T08:55:46.517109+0000","last_unstale":"2024-04-25T08:55:47.515707+0000","last_undegraded":"2024-04-25T08:55:47.515707+0000","last_fullsized":"2024-04-25T08:55:47.515707+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:51.700529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928007+0000","last_change":"2024-04-25T08:55:46.554289+0000","last_active":"2024-04-25T08:55:47.928007+0000","last_peered":"2024-04-25T08:55:47.928007+0000","last_clean":"2024-04-25T08:55:47.928007+0000","last_became_active":"2024-04-25T08:55:46.553819+0000","last_became_peered":"2024-04-25T08:55:46.553819+0000","last_unstale":"2024-04-25T08:55:47.928007+0000","last_undegraded":"2024-04-25T08:55:47.928007+0000","last_fullsized":"2024-04-25T08:55:47.928007+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:57.531045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.514954+0000","last_change":"2024-04-25T08:55:46.549592+0000","last_active":"2024-04-25T08:55:47.514954+0000","last_peered":"2024-04-25T08:55:47.514954+0000","last_clean":"2024-04-25T08:55:47.514954+0000","last_became_active":"2024-04-25T08:55:46.548865+0000","last_became_peered":"2024-04-25T08:55:46.548865+0000","last_unstale":"2024-04-25T08:55:47.514954+0000","last_undegraded":"2024-04-25T08:55:47.514954+0000","last_fullsized":"2024-04-25T08:55:47.514954+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:12.950330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926333+0000","last_change":"2024-04-25T08:55:46.563914+0000","last_active":"2024-04-25T08:55:47.926333+0000","last_peered":"2024-04-25T08:55:47.926333+0000","last_clean":"2024-04-25T08:55:47.926333+0000","last_became_active":"2024-04-25T08:55:46.563267+0000","last_became_peered":"2024-04-25T08:55:46.563267+0000","last_unstale":"2024-04-25T08:55:47.926333+0000","last_undegraded":"2024-04-25T08:55:47.926333+0000","last_fullsized":"2024-04-25T08:55:47.926333+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:20:20.962033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929016+0000","last_change":"2024-04-25T08:55:46.562700+0000","last_active":"2024-04-25T08:55:47.929016+0000","last_peered":"2024-04-25T08:55:47.929016+0000","last_clean":"2024-04-25T08:55:47.929016+0000","last_became_active":"2024-04-25T08:55:46.562094+0000","last_became_peered":"2024-04-25T08:55:46.562094+0000","last_unstale":"2024-04-25T08:55:47.929016+0000","last_undegraded":"2024-04-25T08:55:47.929016+0000","last_fullsized":"2024-04-25T08:55:47.929016+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:12.531432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928629+0000","last_change":"2024-04-25T08:55:46.557333+0000","last_active":"2024-04-25T08:55:47.928629+0000","last_peered":"2024-04-25T08:55:47.928629+0000","last_clean":"2024-04-25T08:55:47.928629+0000","last_became_active":"2024-04-25T08:55:46.556524+0000","last_became_peered":"2024-04-25T08:55:46.556524+0000","last_unstale":"2024-04-25T08:55:47.928629+0000","last_undegraded":"2024-04-25T08:55:47.928629+0000","last_fullsized":"2024-04-25T08:55:47.928629+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:10.462685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516084+0000","last_change":"2024-04-25T08:55:46.528417+0000","last_active":"2024-04-25T08:55:47.516084+0000","last_peered":"2024-04-25T08:55:47.516084+0000","last_clean":"2024-04-25T08:55:47.516084+0000","last_became_active":"2024-04-25T08:55:46.527702+0000","last_became_peered":"2024-04-25T08:55:46.527702+0000","last_unstale":"2024-04-25T08:55:47.516084+0000","last_undegraded":"2024-04-25T08:55:47.516084+0000","last_fullsized":"2024-04-25T08:55:47.516084+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:31.380413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927804+0000","last_change":"2024-04-25T08:55:46.551665+0000","last_active":"2024-04-25T08:55:47.927804+0000","last_peered":"2024-04-25T08:55:47.927804+0000","last_clean":"2024-04-25T08:55:47.927804+0000","last_became_active":"2024-04-25T08:55:46.550994+0000","last_became_peered":"2024-04-25T08:55:46.550994+0000","last_unstale":"2024-04-25T08:55:47.927804+0000","last_undegraded":"2024-04-25T08:55:47.927804+0000","last_fullsized":"2024-04-25T08:55:47.927804+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:43.066148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927612+0000","last_change":"2024-04-25T08:55:46.541749+0000","last_active":"2024-04-25T08:55:47.927612+0000","last_peered":"2024-04-25T08:55:47.927612+0000","last_clean":"2024-04-25T08:55:47.927612+0000","last_became_active":"2024-04-25T08:55:46.540950+0000","last_became_peered":"2024-04-25T08:55:46.540950+0000","last_unstale":"2024-04-25T08:55:47.927612+0000","last_undegraded":"2024-04-25T08:55:47.927612+0000","last_fullsized":"2024-04-25T08:55:47.927612+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:01.335363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927728+0000","last_change":"2024-04-25T08:55:46.558897+0000","last_active":"2024-04-25T08:55:47.927728+0000","last_peered":"2024-04-25T08:55:47.927728+0000","last_clean":"2024-04-25T08:55:47.927728+0000","last_became_active":"2024-04-25T08:55:46.557937+0000","last_became_peered":"2024-04-25T08:55:46.557937+0000","last_unstale":"2024-04-25T08:55:47.927728+0000","last_undegraded":"2024-04-25T08:55:47.927728+0000","last_fullsized":"2024-04-25T08:55:47.927728+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:43.066148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927113+0000","last_change":"2024-04-25T08:55:46.560374+0000","last_active":"2024-04-25T08:55:47.927113+0000","last_peered":"2024-04-25T08:55:47.927113+0000","last_clean":"2024-04-25T08:55:47.927113+0000","last_became_active":"2024-04-25T08:55:46.559664+0000","last_became_peered":"2024-04-25T08:55:46.559664+0000","last_unstale":"2024-04-25T08:55:47.927113+0000","last_undegraded":"2024-04-25T08:55:47.927113+0000","last_fullsized":"2024-04-25T08:55:47.927113+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:33.737868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.514926+0000","last_change":"2024-04-25T08:55:46.534310+0000","last_active":"2024-04-25T08:55:47.514926+0000","last_peered":"2024-04-25T08:55:47.514926+0000","last_clean":"2024-04-25T08:55:47.514926+0000","last_became_active":"2024-04-25T08:55:46.533609+0000","last_became_peered":"2024-04-25T08:55:46.533609+0000","last_unstale":"2024-04-25T08:55:47.514926+0000","last_undegraded":"2024-04-25T08:55:47.514926+0000","last_fullsized":"2024-04-25T08:55:47.514926+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:05.039606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929454+0000","last_change":"2024-04-25T08:55:46.545284+0000","last_active":"2024-04-25T08:55:47.929454+0000","last_peered":"2024-04-25T08:55:47.929454+0000","last_clean":"2024-04-25T08:55:47.929454+0000","last_became_active":"2024-04-25T08:55:46.544088+0000","last_became_peered":"2024-04-25T08:55:46.544088+0000","last_unstale":"2024-04-25T08:55:47.929454+0000","last_undegraded":"2024-04-25T08:55:47.929454+0000","last_fullsized":"2024-04-25T08:55:47.929454+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:33.737868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516471+0000","last_change":"2024-04-25T08:55:46.543469+0000","last_active":"2024-04-25T08:55:47.516471+0000","last_peered":"2024-04-25T08:55:47.516471+0000","last_clean":"2024-04-25T08:55:47.516471+0000","last_became_active":"2024-04-25T08:55:46.542572+0000","last_became_peered":"2024-04-25T08:55:46.542572+0000","last_unstale":"2024-04-25T08:55:47.516471+0000","last_undegraded":"2024-04-25T08:55:47.516471+0000","last_fullsized":"2024-04-25T08:55:47.516471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:51.700529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515631+0000","last_change":"2024-04-25T08:55:46.548884+0000","last_active":"2024-04-25T08:55:47.515631+0000","last_peered":"2024-04-25T08:55:47.515631+0000","last_clean":"2024-04-25T08:55:47.515631+0000","last_became_active":"2024-04-25T08:55:46.548256+0000","last_became_peered":"2024-04-25T08:55:46.548256+0000","last_unstale":"2024-04-25T08:55:47.515631+0000","last_undegraded":"2024-04-25T08:55:47.515631+0000","last_fullsized":"2024-04-25T08:55:47.515631+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:31.380413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927144+0000","last_change":"2024-04-25T08:55:46.531222+0000","last_active":"2024-04-25T08:55:47.927144+0000","last_peered":"2024-04-25T08:55:47.927144+0000","last_clean":"2024-04-25T08:55:47.927144+0000","last_became_active":"2024-04-25T08:55:46.530456+0000","last_became_peered":"2024-04-25T08:55:46.530456+0000","last_unstale":"2024-04-25T08:55:47.927144+0000","last_undegraded":"2024-04-25T08:55:47.927144+0000","last_fullsized":"2024-04-25T08:55:47.927144+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:02.769517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927848+0000","last_change":"2024-04-25T08:55:46.543282+0000","last_active":"2024-04-25T08:55:47.927848+0000","last_peered":"2024-04-25T08:55:47.927848+0000","last_clean":"2024-04-25T08:55:47.927848+0000","last_became_active":"2024-04-25T08:55:46.542598+0000","last_became_peered":"2024-04-25T08:55:46.542598+0000","last_unstale":"2024-04-25T08:55:47.927848+0000","last_undegraded":"2024-04-25T08:55:47.927848+0000","last_fullsized":"2024-04-25T08:55:47.927848+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:10.462685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926610+0000","last_change":"2024-04-25T08:55:46.552013+0000","last_active":"2024-04-25T08:55:47.926610+0000","last_peered":"2024-04-25T08:55:47.926610+0000","last_clean":"2024-04-25T08:55:47.926610+0000","last_became_active":"2024-04-25T08:55:46.551322+0000","last_became_peered":"2024-04-25T08:55:46.551322+0000","last_unstale":"2024-04-25T08:55:47.926610+0000","last_undegraded":"2024-04-25T08:55:47.926610+0000","last_fullsized":"2024-04-25T08:55:47.926610+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:43.094059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927001+0000","last_change":"2024-04-25T08:55:46.552853+0000","last_active":"2024-04-25T08:55:47.927001+0000","last_peered":"2024-04-25T08:55:47.927001+0000","last_clean":"2024-04-25T08:55:47.927001+0000","last_became_active":"2024-04-25T08:55:46.551266+0000","last_became_peered":"2024-04-25T08:55:46.551266+0000","last_unstale":"2024-04-25T08:55:47.927001+0000","last_undegraded":"2024-04-25T08:55:47.927001+0000","last_fullsized":"2024-04-25T08:55:47.927001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:22:46.112447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927693+0000","last_change":"2024-04-25T08:55:46.559771+0000","last_active":"2024-04-25T08:55:47.927693+0000","last_peered":"2024-04-25T08:55:47.927693+0000","last_clean":"2024-04-25T08:55:47.927693+0000","last_became_active":"2024-04-25T08:55:46.559211+0000","last_became_peered":"2024-04-25T08:55:46.559211+0000","last_unstale":"2024-04-25T08:55:47.927693+0000","last_undegraded":"2024-04-25T08:55:47.927693+0000","last_fullsized":"2024-04-25T08:55:47.927693+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:01.335363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927929+0000","last_change":"2024-04-25T08:55:46.548273+0000","last_active":"2024-04-25T08:55:47.927929+0000","last_peered":"2024-04-25T08:55:47.927929+0000","last_clean":"2024-04-25T08:55:47.927929+0000","last_became_active":"2024-04-25T08:55:46.547611+0000","last_became_peered":"2024-04-25T08:55:46.547611+0000","last_unstale":"2024-04-25T08:55:47.927929+0000","last_undegraded":"2024-04-25T08:55:47.927929+0000","last_fullsized":"2024-04-25T08:55:47.927929+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:12.531432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928077+0000","last_change":"2024-04-25T08:55:46.548416+0000","last_active":"2024-04-25T08:55:47.928077+0000","last_peered":"2024-04-25T08:55:47.928077+0000","last_clean":"2024-04-25T08:55:47.928077+0000","last_became_active":"2024-04-25T08:55:46.547828+0000","last_became_peered":"2024-04-25T08:55:46.547828+0000","last_unstale":"2024-04-25T08:55:47.928077+0000","last_undegraded":"2024-04-25T08:55:47.928077+0000","last_fullsized":"2024-04-25T08:55:47.928077+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:38.300578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516492+0000","last_change":"2024-04-25T08:55:46.516452+0000","last_active":"2024-04-25T08:55:47.516492+0000","last_peered":"2024-04-25T08:55:47.516492+0000","last_clean":"2024-04-25T08:55:47.516492+0000","last_became_active":"2024-04-25T08:55:46.515423+0000","last_became_peered":"2024-04-25T08:55:46.515423+0000","last_unstale":"2024-04-25T08:55:47.516492+0000","last_undegraded":"2024-04-25T08:55:47.516492+0000","last_fullsized":"2024-04-25T08:55:47.516492+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:33.341553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929504+0000","last_change":"2024-04-25T08:55:46.557531+0000","last_active":"2024-04-25T08:55:47.929504+0000","last_peered":"2024-04-25T08:55:47.929504+0000","last_clean":"2024-04-25T08:55:47.929504+0000","last_became_active":"2024-04-25T08:55:46.556752+0000","last_became_peered":"2024-04-25T08:55:46.556752+0000","last_unstale":"2024-04-25T08:55:47.929504+0000","last_undegraded":"2024-04-25T08:55:47.929504+0000","last_fullsized":"2024-04-25T08:55:47.929504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:07.053904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927239+0000","last_change":"2024-04-25T08:55:46.558405+0000","last_active":"2024-04-25T08:55:47.927239+0000","last_peered":"2024-04-25T08:55:47.927239+0000","last_clean":"2024-04-25T08:55:47.927239+0000","last_became_active":"2024-04-25T08:55:46.557106+0000","last_became_peered":"2024-04-25T08:55:46.557106+0000","last_unstale":"2024-04-25T08:55:47.927239+0000","last_undegraded":"2024-04-25T08:55:47.927239+0000","last_fullsized":"2024-04-25T08:55:47.927239+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:14.229282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926363+0000","last_change":"2024-04-25T08:55:46.551564+0000","last_active":"2024-04-25T08:55:47.926363+0000","last_peered":"2024-04-25T08:55:47.926363+0000","last_clean":"2024-04-25T08:55:47.926363+0000","last_became_active":"2024-04-25T08:55:46.550835+0000","last_became_peered":"2024-04-25T08:55:46.550835+0000","last_unstale":"2024-04-25T08:55:47.926363+0000","last_undegraded":"2024-04-25T08:55:47.926363+0000","last_fullsized":"2024-04-25T08:55:47.926363+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:33.737868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515067+0000","last_change":"2024-04-25T08:55:46.514783+0000","last_active":"2024-04-25T08:55:47.515067+0000","last_peered":"2024-04-25T08:55:47.515067+0000","last_clean":"2024-04-25T08:55:47.515067+0000","last_became_active":"2024-04-25T08:55:46.513951+0000","last_became_peered":"2024-04-25T08:55:46.513951+0000","last_unstale":"2024-04-25T08:55:47.515067+0000","last_undegraded":"2024-04-25T08:55:47.515067+0000","last_fullsized":"2024-04-25T08:55:47.515067+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:32.326436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926459+0000","last_change":"2024-04-25T08:55:46.550278+0000","last_active":"2024-04-25T08:55:47.926459+0000","last_peered":"2024-04-25T08:55:47.926459+0000","last_clean":"2024-04-25T08:55:47.926459+0000","last_became_active":"2024-04-25T08:55:46.549390+0000","last_became_peered":"2024-04-25T08:55:46.549390+0000","last_unstale":"2024-04-25T08:55:47.926459+0000","last_undegraded":"2024-04-25T08:55:47.926459+0000","last_fullsized":"2024-04-25T08:55:47.926459+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:34.440044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516302+0000","last_change":"2024-04-25T08:55:46.547099+0000","last_active":"2024-04-25T08:55:47.516302+0000","last_peered":"2024-04-25T08:55:47.516302+0000","last_clean":"2024-04-25T08:55:47.516302+0000","last_became_active":"2024-04-25T08:55:46.546395+0000","last_became_peered":"2024-04-25T08:55:46.546395+0000","last_unstale":"2024-04-25T08:55:47.516302+0000","last_undegraded":"2024-04-25T08:55:47.516302+0000","last_fullsized":"2024-04-25T08:55:47.516302+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:34.060227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"68'4","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929393+0000","last_change":"2024-04-25T08:55:45.867969+0000","last_active":"2024-04-25T08:55:47.929393+0000","last_peered":"2024-04-25T08:55:47.929393+0000","last_clean":"2024-04-25T08:55:47.929393+0000","last_became_active":"2024-04-25T08:55:45.867621+0000","last_became_peered":"2024-04-25T08:55:45.867621+0000","last_unstale":"2024-04-25T08:55:47.929393+0000","last_undegraded":"2024-04-25T08:55:47.929393+0000","last_fullsized":"2024-04-25T08:55:47.929393+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:50.359008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516954+0000","last_change":"2024-04-25T08:55:46.526885+0000","last_active":"2024-04-25T08:55:47.516954+0000","last_peered":"2024-04-25T08:55:47.516954+0000","last_clean":"2024-04-25T08:55:47.516954+0000","last_became_active":"2024-04-25T08:55:46.525688+0000","last_became_peered":"2024-04-25T08:55:46.525688+0000","last_unstale":"2024-04-25T08:55:47.516954+0000","last_undegraded":"2024-04-25T08:55:47.516954+0000","last_fullsized":"2024-04-25T08:55:47.516954+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:34.060227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929068+0000","last_change":"2024-04-25T08:55:45.861921+0000","last_active":"2024-04-25T08:55:47.929068+0000","last_peered":"2024-04-25T08:55:47.929068+0000","last_clean":"2024-04-25T08:55:47.929068+0000","last_became_active":"2024-04-25T08:55:45.861609+0000","last_became_peered":"2024-04-25T08:55:45.861609+0000","last_unstale":"2024-04-25T08:55:47.929068+0000","last_undegraded":"2024-04-25T08:55:47.929068+0000","last_fullsized":"2024-04-25T08:55:47.929068+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:11.528135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928915+0000","last_change":"2024-04-25T08:55:46.549759+0000","last_active":"2024-04-25T08:55:47.928915+0000","last_peered":"2024-04-25T08:55:47.928915+0000","last_clean":"2024-04-25T08:55:47.928915+0000","last_became_active":"2024-04-25T08:55:46.549225+0000","last_became_peered":"2024-04-25T08:55:46.549225+0000","last_unstale":"2024-04-25T08:55:47.928915+0000","last_undegraded":"2024-04-25T08:55:47.928915+0000","last_fullsized":"2024-04-25T08:55:47.928915+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:43.094059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927802+0000","last_change":"2024-04-25T08:55:45.859351+0000","last_active":"2024-04-25T08:55:47.927802+0000","last_peered":"2024-04-25T08:55:47.927802+0000","last_clean":"2024-04-25T08:55:47.927802+0000","last_became_active":"2024-04-25T08:55:45.859081+0000","last_became_peered":"2024-04-25T08:55:45.859081+0000","last_unstale":"2024-04-25T08:55:47.927802+0000","last_undegraded":"2024-04-25T08:55:47.927802+0000","last_fullsized":"2024-04-25T08:55:47.927802+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:09.459388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929027+0000","last_change":"2024-04-25T08:55:46.557632+0000","last_active":"2024-04-25T08:55:47.929027+0000","last_peered":"2024-04-25T08:55:47.929027+0000","last_clean":"2024-04-25T08:55:47.929027+0000","last_became_active":"2024-04-25T08:55:46.549992+0000","last_became_peered":"2024-04-25T08:55:46.549992+0000","last_unstale":"2024-04-25T08:55:47.929027+0000","last_undegraded":"2024-04-25T08:55:47.929027+0000","last_fullsized":"2024-04-25T08:55:47.929027+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:38.300578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928321+0000","last_change":"2024-04-25T08:55:45.836302+0000","last_active":"2024-04-25T08:55:47.928321+0000","last_peered":"2024-04-25T08:55:47.928321+0000","last_clean":"2024-04-25T08:55:47.928321+0000","last_became_active":"2024-04-25T08:55:45.836023+0000","last_became_peered":"2024-04-25T08:55:45.836023+0000","last_unstale":"2024-04-25T08:55:47.928321+0000","last_undegraded":"2024-04-25T08:55:47.928321+0000","last_fullsized":"2024-04-25T08:55:47.928321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:32.825742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515183+0000","last_change":"2024-04-25T08:55:46.531579+0000","last_active":"2024-04-25T08:55:47.515183+0000","last_peered":"2024-04-25T08:55:47.515183+0000","last_clean":"2024-04-25T08:55:47.515183+0000","last_became_active":"2024-04-25T08:55:46.530758+0000","last_became_peered":"2024-04-25T08:55:46.530758+0000","last_unstale":"2024-04-25T08:55:47.515183+0000","last_undegraded":"2024-04-25T08:55:47.515183+0000","last_fullsized":"2024-04-25T08:55:47.515183+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:40.387050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929007+0000","last_change":"2024-04-25T08:55:45.839986+0000","last_active":"2024-04-25T08:55:47.929007+0000","last_peered":"2024-04-25T08:55:47.929007+0000","last_clean":"2024-04-25T08:55:47.929007+0000","last_became_active":"2024-04-25T08:55:45.839521+0000","last_became_peered":"2024-04-25T08:55:45.839521+0000","last_unstale":"2024-04-25T08:55:47.929007+0000","last_undegraded":"2024-04-25T08:55:47.929007+0000","last_fullsized":"2024-04-25T08:55:47.929007+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:27.101381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928039+0000","last_change":"2024-04-25T08:55:46.541211+0000","last_active":"2024-04-25T08:55:47.928039+0000","last_peered":"2024-04-25T08:55:47.928039+0000","last_clean":"2024-04-25T08:55:47.928039+0000","last_became_active":"2024-04-25T08:55:46.540617+0000","last_became_peered":"2024-04-25T08:55:46.540617+0000","last_unstale":"2024-04-25T08:55:47.928039+0000","last_undegraded":"2024-04-25T08:55:47.928039+0000","last_fullsized":"2024-04-25T08:55:47.928039+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:53.544366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926648+0000","last_change":"2024-04-25T08:55:45.838676+0000","last_active":"2024-04-25T08:55:47.926648+0000","last_peered":"2024-04-25T08:55:47.926648+0000","last_clean":"2024-04-25T08:55:47.926648+0000","last_became_active":"2024-04-25T08:55:45.836467+0000","last_became_peered":"2024-04-25T08:55:45.836467+0000","last_unstale":"2024-04-25T08:55:47.926648+0000","last_undegraded":"2024-04-25T08:55:47.926648+0000","last_fullsized":"2024-04-25T08:55:47.926648+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.297281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515469+0000","last_change":"2024-04-25T08:55:46.550655+0000","last_active":"2024-04-25T08:55:47.515469+0000","last_peered":"2024-04-25T08:55:47.515469+0000","last_clean":"2024-04-25T08:55:47.515469+0000","last_became_active":"2024-04-25T08:55:46.550062+0000","last_became_peered":"2024-04-25T08:55:46.550062+0000","last_unstale":"2024-04-25T08:55:47.515469+0000","last_undegraded":"2024-04-25T08:55:47.515469+0000","last_fullsized":"2024-04-25T08:55:47.515469+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:09.965629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926752+0000","last_change":"2024-04-25T08:55:45.849357+0000","last_active":"2024-04-25T08:55:47.926752+0000","last_peered":"2024-04-25T08:55:47.926752+0000","last_clean":"2024-04-25T08:55:47.926752+0000","last_became_active":"2024-04-25T08:55:45.848617+0000","last_became_peered":"2024-04-25T08:55:45.848617+0000","last_unstale":"2024-04-25T08:55:47.926752+0000","last_undegraded":"2024-04-25T08:55:47.926752+0000","last_fullsized":"2024-04-25T08:55:47.926752+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:18.161812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926772+0000","last_change":"2024-04-25T08:55:46.541838+0000","last_active":"2024-04-25T08:55:47.926772+0000","last_peered":"2024-04-25T08:55:47.926772+0000","last_clean":"2024-04-25T08:55:47.926772+0000","last_became_active":"2024-04-25T08:55:46.541140+0000","last_became_peered":"2024-04-25T08:55:46.541140+0000","last_unstale":"2024-04-25T08:55:47.926772+0000","last_undegraded":"2024-04-25T08:55:47.926772+0000","last_fullsized":"2024-04-25T08:55:47.926772+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:10.430318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.514926+0000","last_change":"2024-04-25T08:55:45.862024+0000","last_active":"2024-04-25T08:55:47.514926+0000","last_peered":"2024-04-25T08:55:47.514926+0000","last_clean":"2024-04-25T08:55:47.514926+0000","last_became_active":"2024-04-25T08:55:45.860893+0000","last_became_peered":"2024-04-25T08:55:45.860893+0000","last_unstale":"2024-04-25T08:55:47.514926+0000","last_undegraded":"2024-04-25T08:55:47.514926+0000","last_fullsized":"2024-04-25T08:55:47.514926+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:20.826306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928601+0000","last_change":"2024-04-25T08:55:46.547693+0000","last_active":"2024-04-25T08:55:47.928601+0000","last_peered":"2024-04-25T08:55:47.928601+0000","last_clean":"2024-04-25T08:55:47.928601+0000","last_became_active":"2024-04-25T08:55:46.546817+0000","last_became_peered":"2024-04-25T08:55:46.546817+0000","last_unstale":"2024-04-25T08:55:47.928601+0000","last_undegraded":"2024-04-25T08:55:47.928601+0000","last_fullsized":"2024-04-25T08:55:47.928601+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:46.787415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929209+0000","last_change":"2024-04-25T08:55:45.867789+0000","last_active":"2024-04-25T08:55:47.929209+0000","last_peered":"2024-04-25T08:55:47.929209+0000","last_clean":"2024-04-25T08:55:47.929209+0000","last_became_active":"2024-04-25T08:55:45.866828+0000","last_became_peered":"2024-04-25T08:55:45.866828+0000","last_unstale":"2024-04-25T08:55:47.929209+0000","last_undegraded":"2024-04-25T08:55:47.929209+0000","last_fullsized":"2024-04-25T08:55:47.929209+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:00.089017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927395+0000","last_change":"2024-04-25T08:55:46.539943+0000","last_active":"2024-04-25T08:55:47.927395+0000","last_peered":"2024-04-25T08:55:47.927395+0000","last_clean":"2024-04-25T08:55:47.927395+0000","last_became_active":"2024-04-25T08:55:46.539123+0000","last_became_peered":"2024-04-25T08:55:46.539123+0000","last_unstale":"2024-04-25T08:55:47.927395+0000","last_undegraded":"2024-04-25T08:55:47.927395+0000","last_fullsized":"2024-04-25T08:55:47.927395+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:23.583910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.517215+0000","last_change":"2024-04-25T08:55:45.847982+0000","last_active":"2024-04-25T08:55:47.517215+0000","last_peered":"2024-04-25T08:55:47.517215+0000","last_clean":"2024-04-25T08:55:47.517215+0000","last_became_active":"2024-04-25T08:55:45.846245+0000","last_became_peered":"2024-04-25T08:55:45.846245+0000","last_unstale":"2024-04-25T08:55:47.517215+0000","last_undegraded":"2024-04-25T08:55:47.517215+0000","last_fullsized":"2024-04-25T08:55:47.517215+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:20.826306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.517374+0000","last_change":"2024-04-25T08:55:46.534069+0000","last_active":"2024-04-25T08:55:47.517374+0000","last_peered":"2024-04-25T08:55:47.517374+0000","last_clean":"2024-04-25T08:55:47.517374+0000","last_became_active":"2024-04-25T08:55:46.529946+0000","last_became_peered":"2024-04-25T08:55:46.529946+0000","last_unstale":"2024-04-25T08:55:47.517374+0000","last_undegraded":"2024-04-25T08:55:47.517374+0000","last_fullsized":"2024-04-25T08:55:47.517374+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:53.544366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"68'3","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928087+0000","last_change":"2024-04-25T08:55:45.860638+0000","last_active":"2024-04-25T08:55:47.928087+0000","last_peered":"2024-04-25T08:55:47.928087+0000","last_clean":"2024-04-25T08:55:47.928087+0000","last_became_active":"2024-04-25T08:55:45.860019+0000","last_became_peered":"2024-04-25T08:55:45.860019+0000","last_unstale":"2024-04-25T08:55:47.928087+0000","last_undegraded":"2024-04-25T08:55:47.928087+0000","last_fullsized":"2024-04-25T08:55:47.928087+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:31.323139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.517111+0000","last_change":"2024-04-25T08:55:46.537372+0000","last_active":"2024-04-25T08:55:47.517111+0000","last_peered":"2024-04-25T08:55:47.517111+0000","last_clean":"2024-04-25T08:55:47.517111+0000","last_became_active":"2024-04-25T08:55:46.531983+0000","last_became_peered":"2024-04-25T08:55:46.531983+0000","last_unstale":"2024-04-25T08:55:47.517111+0000","last_undegraded":"2024-04-25T08:55:47.517111+0000","last_fullsized":"2024-04-25T08:55:47.517111+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:52.573081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"68'6","reported_seq":20,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929019+0000","last_change":"2024-04-25T08:55:45.839267+0000","last_active":"2024-04-25T08:55:47.929019+0000","last_peered":"2024-04-25T08:55:47.929019+0000","last_clean":"2024-04-25T08:55:47.929019+0000","last_became_active":"2024-04-25T08:55:45.838591+0000","last_became_peered":"2024-04-25T08:55:45.838591+0000","last_unstale":"2024-04-25T08:55:47.929019+0000","last_undegraded":"2024-04-25T08:55:47.929019+0000","last_fullsized":"2024-04-25T08:55:47.929019+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:00.089017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515025+0000","last_change":"2024-04-25T08:55:46.555086+0000","last_active":"2024-04-25T08:55:47.515025+0000","last_peered":"2024-04-25T08:55:47.515025+0000","last_clean":"2024-04-25T08:55:47.515025+0000","last_became_active":"2024-04-25T08:55:46.554275+0000","last_became_peered":"2024-04-25T08:55:46.554275+0000","last_unstale":"2024-04-25T08:55:47.515025+0000","last_undegraded":"2024-04-25T08:55:47.515025+0000","last_fullsized":"2024-04-25T08:55:47.515025+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:51.008518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927258+0000","last_change":"2024-04-25T08:55:45.867429+0000","last_active":"2024-04-25T08:55:47.927258+0000","last_peered":"2024-04-25T08:55:47.927258+0000","last_clean":"2024-04-25T08:55:47.927258+0000","last_became_active":"2024-04-25T08:55:45.866476+0000","last_became_peered":"2024-04-25T08:55:45.866476+0000","last_unstale":"2024-04-25T08:55:47.927258+0000","last_undegraded":"2024-04-25T08:55:47.927258+0000","last_fullsized":"2024-04-25T08:55:47.927258+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:50.359008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927100+0000","last_change":"2024-04-25T08:55:46.539859+0000","last_active":"2024-04-25T08:55:47.927100+0000","last_peered":"2024-04-25T08:55:47.927100+0000","last_clean":"2024-04-25T08:55:47.927100+0000","last_became_active":"2024-04-25T08:55:46.539211+0000","last_became_peered":"2024-04-25T08:55:46.539211+0000","last_unstale":"2024-04-25T08:55:47.927100+0000","last_undegraded":"2024-04-25T08:55:47.927100+0000","last_fullsized":"2024-04-25T08:55:47.927100+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:14.229282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927040+0000","last_change":"2024-04-25T08:55:45.867756+0000","last_active":"2024-04-25T08:55:47.927040+0000","last_peered":"2024-04-25T08:55:47.927040+0000","last_clean":"2024-04-25T08:55:47.927040+0000","last_became_active":"2024-04-25T08:55:45.860605+0000","last_became_peered":"2024-04-25T08:55:45.860605+0000","last_unstale":"2024-04-25T08:55:47.927040+0000","last_undegraded":"2024-04-25T08:55:47.927040+0000","last_fullsized":"2024-04-25T08:55:47.927040+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:50.359008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515858+0000","last_change":"2024-04-25T08:55:46.533293+0000","last_active":"2024-04-25T08:55:47.515858+0000","last_peered":"2024-04-25T08:55:47.515858+0000","last_clean":"2024-04-25T08:55:47.515858+0000","last_became_active":"2024-04-25T08:55:46.532553+0000","last_became_peered":"2024-04-25T08:55:46.532553+0000","last_unstale":"2024-04-25T08:55:47.515858+0000","last_undegraded":"2024-04-25T08:55:47.515858+0000","last_fullsized":"2024-04-25T08:55:47.515858+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:33.341553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928331+0000","last_change":"2024-04-25T08:55:45.867068+0000","last_active":"2024-04-25T08:55:47.928331+0000","last_peered":"2024-04-25T08:55:47.928331+0000","last_clean":"2024-04-25T08:55:47.928331+0000","last_became_active":"2024-04-25T08:55:45.866196+0000","last_became_peered":"2024-04-25T08:55:45.866196+0000","last_unstale":"2024-04-25T08:55:47.928331+0000","last_undegraded":"2024-04-25T08:55:47.928331+0000","last_fullsized":"2024-04-25T08:55:47.928331+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:30.377116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928286+0000","last_change":"2024-04-25T08:55:46.549689+0000","last_active":"2024-04-25T08:55:47.928286+0000","last_peered":"2024-04-25T08:55:47.928286+0000","last_clean":"2024-04-25T08:55:47.928286+0000","last_became_active":"2024-04-25T08:55:46.548980+0000","last_became_peered":"2024-04-25T08:55:46.548980+0000","last_unstale":"2024-04-25T08:55:47.928286+0000","last_undegraded":"2024-04-25T08:55:47.928286+0000","last_fullsized":"2024-04-25T08:55:47.928286+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:10.718524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"68'8","reported_seq":22,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515303+0000","last_change":"2024-04-25T08:55:45.862676+0000","last_active":"2024-04-25T08:55:47.515303+0000","last_peered":"2024-04-25T08:55:47.515303+0000","last_clean":"2024-04-25T08:55:47.515303+0000","last_became_active":"2024-04-25T08:55:45.861928+0000","last_became_peered":"2024-04-25T08:55:45.861928+0000","last_unstale":"2024-04-25T08:55:47.515303+0000","last_undegraded":"2024-04-25T08:55:47.515303+0000","last_fullsized":"2024-04-25T08:55:47.515303+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:41.047486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927138+0000","last_change":"2024-04-25T08:55:46.557584+0000","last_active":"2024-04-25T08:55:47.927138+0000","last_peered":"2024-04-25T08:55:47.927138+0000","last_clean":"2024-04-25T08:55:47.927138+0000","last_became_active":"2024-04-25T08:55:46.556690+0000","last_became_peered":"2024-04-25T08:55:46.556690+0000","last_unstale":"2024-04-25T08:55:47.927138+0000","last_undegraded":"2024-04-25T08:55:47.927138+0000","last_fullsized":"2024-04-25T08:55:47.927138+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:07.053904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927959+0000","last_change":"2024-04-25T08:55:45.860638+0000","last_active":"2024-04-25T08:55:47.927959+0000","last_peered":"2024-04-25T08:55:47.927959+0000","last_clean":"2024-04-25T08:55:47.927959+0000","last_became_active":"2024-04-25T08:55:45.860008+0000","last_became_peered":"2024-04-25T08:55:45.860008+0000","last_unstale":"2024-04-25T08:55:47.927959+0000","last_undegraded":"2024-04-25T08:55:47.927959+0000","last_fullsized":"2024-04-25T08:55:47.927959+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:50.005220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516701+0000","last_change":"2024-04-25T08:55:46.519731+0000","last_active":"2024-04-25T08:55:47.516701+0000","last_peered":"2024-04-25T08:55:47.516701+0000","last_clean":"2024-04-25T08:55:47.516701+0000","last_became_active":"2024-04-25T08:55:46.519055+0000","last_became_peered":"2024-04-25T08:55:46.519055+0000","last_unstale":"2024-04-25T08:55:47.516701+0000","last_undegraded":"2024-04-25T08:55:47.516701+0000","last_fullsized":"2024-04-25T08:55:47.516701+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:52.573081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928914+0000","last_change":"2024-04-25T08:55:45.867699+0000","last_active":"2024-04-25T08:55:47.928914+0000","last_peered":"2024-04-25T08:55:47.928914+0000","last_clean":"2024-04-25T08:55:47.928914+0000","last_became_active":"2024-04-25T08:55:45.866911+0000","last_became_peered":"2024-04-25T08:55:45.866911+0000","last_unstale":"2024-04-25T08:55:47.928914+0000","last_undegraded":"2024-04-25T08:55:47.928914+0000","last_fullsized":"2024-04-25T08:55:47.928914+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:50.005220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516942+0000","last_change":"2024-04-25T08:55:46.528858+0000","last_active":"2024-04-25T08:55:47.516942+0000","last_peered":"2024-04-25T08:55:47.516942+0000","last_clean":"2024-04-25T08:55:47.516942+0000","last_became_active":"2024-04-25T08:55:46.516198+0000","last_became_peered":"2024-04-25T08:55:46.516198+0000","last_unstale":"2024-04-25T08:55:47.516942+0000","last_undegraded":"2024-04-25T08:55:47.516942+0000","last_fullsized":"2024-04-25T08:55:47.516942+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:58.970972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"68'1","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928479+0000","last_change":"2024-04-25T08:55:45.866514+0000","last_active":"2024-04-25T08:55:47.928479+0000","last_peered":"2024-04-25T08:55:47.928479+0000","last_clean":"2024-04-25T08:55:47.928479+0000","last_became_active":"2024-04-25T08:55:45.864232+0000","last_became_peered":"2024-04-25T08:55:45.864232+0000","last_unstale":"2024-04-25T08:55:47.928479+0000","last_undegraded":"2024-04-25T08:55:47.928479+0000","last_fullsized":"2024-04-25T08:55:47.928479+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:08.962332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516162+0000","last_change":"2024-04-25T08:55:46.546467+0000","last_active":"2024-04-25T08:55:47.516162+0000","last_peered":"2024-04-25T08:55:47.516162+0000","last_clean":"2024-04-25T08:55:47.516162+0000","last_became_active":"2024-04-25T08:55:46.545650+0000","last_became_peered":"2024-04-25T08:55:46.545650+0000","last_unstale":"2024-04-25T08:55:47.516162+0000","last_undegraded":"2024-04-25T08:55:47.516162+0000","last_fullsized":"2024-04-25T08:55:47.516162+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:05.492858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927395+0000","last_change":"2024-04-25T08:55:45.844352+0000","last_active":"2024-04-25T08:55:47.927395+0000","last_peered":"2024-04-25T08:55:47.927395+0000","last_clean":"2024-04-25T08:55:47.927395+0000","last_became_active":"2024-04-25T08:55:45.843827+0000","last_became_peered":"2024-04-25T08:55:45.843827+0000","last_unstale":"2024-04-25T08:55:47.927395+0000","last_undegraded":"2024-04-25T08:55:47.927395+0000","last_fullsized":"2024-04-25T08:55:47.927395+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:51.336610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928702+0000","last_change":"2024-04-25T08:55:46.549081+0000","last_active":"2024-04-25T08:55:47.928702+0000","last_peered":"2024-04-25T08:55:47.928702+0000","last_clean":"2024-04-25T08:55:47.928702+0000","last_became_active":"2024-04-25T08:55:46.548273+0000","last_became_peered":"2024-04-25T08:55:46.548273+0000","last_unstale":"2024-04-25T08:55:47.928702+0000","last_undegraded":"2024-04-25T08:55:47.928702+0000","last_fullsized":"2024-04-25T08:55:47.928702+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:58.588294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927553+0000","last_change":"2024-04-25T08:55:45.849231+0000","last_active":"2024-04-25T08:55:47.927553+0000","last_peered":"2024-04-25T08:55:47.927553+0000","last_clean":"2024-04-25T08:55:47.927553+0000","last_became_active":"2024-04-25T08:55:45.848819+0000","last_became_peered":"2024-04-25T08:55:45.848819+0000","last_unstale":"2024-04-25T08:55:47.927553+0000","last_undegraded":"2024-04-25T08:55:47.927553+0000","last_fullsized":"2024-04-25T08:55:47.927553+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:42.062850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926403+0000","last_change":"2024-04-25T08:55:46.548727+0000","last_active":"2024-04-25T08:55:47.926403+0000","last_peered":"2024-04-25T08:55:47.926403+0000","last_clean":"2024-04-25T08:55:47.926403+0000","last_became_active":"2024-04-25T08:55:46.548000+0000","last_became_peered":"2024-04-25T08:55:46.548000+0000","last_unstale":"2024-04-25T08:55:47.926403+0000","last_undegraded":"2024-04-25T08:55:47.926403+0000","last_fullsized":"2024-04-25T08:55:47.926403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:07.053904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926617+0000","last_change":"2024-04-25T08:55:45.867156+0000","last_active":"2024-04-25T08:55:47.926617+0000","last_peered":"2024-04-25T08:55:47.926617+0000","last_clean":"2024-04-25T08:55:47.926617+0000","last_became_active":"2024-04-25T08:55:45.859785+0000","last_became_peered":"2024-04-25T08:55:45.859785+0000","last_unstale":"2024-04-25T08:55:47.926617+0000","last_undegraded":"2024-04-25T08:55:47.926617+0000","last_fullsized":"2024-04-25T08:55:47.926617+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:00.089017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.516244+0000","last_change":"2024-04-25T08:55:46.544509+0000","last_active":"2024-04-25T08:55:47.516244+0000","last_peered":"2024-04-25T08:55:47.516244+0000","last_clean":"2024-04-25T08:55:47.516244+0000","last_became_active":"2024-04-25T08:55:46.535006+0000","last_became_peered":"2024-04-25T08:55:46.535006+0000","last_unstale":"2024-04-25T08:55:47.516244+0000","last_undegraded":"2024-04-25T08:55:47.516244+0000","last_fullsized":"2024-04-25T08:55:47.516244+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:58.970972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928716+0000","last_change":"2024-04-25T08:55:45.865786+0000","last_active":"2024-04-25T08:55:47.928716+0000","last_peered":"2024-04-25T08:55:47.928716+0000","last_clean":"2024-04-25T08:55:47.928716+0000","last_became_active":"2024-04-25T08:55:45.865455+0000","last_became_peered":"2024-04-25T08:55:45.865455+0000","last_unstale":"2024-04-25T08:55:47.928716+0000","last_undegraded":"2024-04-25T08:55:47.928716+0000","last_fullsized":"2024-04-25T08:55:47.928716+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:00.332066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928216+0000","last_change":"2024-04-25T08:55:46.543496+0000","last_active":"2024-04-25T08:55:47.928216+0000","last_peered":"2024-04-25T08:55:47.928216+0000","last_clean":"2024-04-25T08:55:47.928216+0000","last_became_active":"2024-04-25T08:55:46.542710+0000","last_became_peered":"2024-04-25T08:55:46.542710+0000","last_unstale":"2024-04-25T08:55:47.928216+0000","last_undegraded":"2024-04-25T08:55:47.928216+0000","last_fullsized":"2024-04-25T08:55:47.928216+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:04.547168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"68'2","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928300+0000","last_change":"2024-04-25T08:55:45.875396+0000","last_active":"2024-04-25T08:55:47.928300+0000","last_peered":"2024-04-25T08:55:47.928300+0000","last_clean":"2024-04-25T08:55:47.928300+0000","last_became_active":"2024-04-25T08:55:45.874752+0000","last_became_peered":"2024-04-25T08:55:45.874752+0000","last_unstale":"2024-04-25T08:55:47.928300+0000","last_undegraded":"2024-04-25T08:55:47.928300+0000","last_fullsized":"2024-04-25T08:55:47.928300+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:31.323139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515293+0000","last_change":"2024-04-25T08:55:46.535009+0000","last_active":"2024-04-25T08:55:47.515293+0000","last_peered":"2024-04-25T08:55:47.515293+0000","last_clean":"2024-04-25T08:55:47.515293+0000","last_became_active":"2024-04-25T08:55:46.534192+0000","last_became_peered":"2024-04-25T08:55:46.534192+0000","last_unstale":"2024-04-25T08:55:47.515293+0000","last_undegraded":"2024-04-25T08:55:47.515293+0000","last_fullsized":"2024-04-25T08:55:47.515293+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:01.115213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928539+0000","last_change":"2024-04-25T08:55:45.869353+0000","last_active":"2024-04-25T08:55:47.928539+0000","last_peered":"2024-04-25T08:55:47.928539+0000","last_clean":"2024-04-25T08:55:47.928539+0000","last_became_active":"2024-04-25T08:55:45.868748+0000","last_became_peered":"2024-04-25T08:55:45.868748+0000","last_unstale":"2024-04-25T08:55:47.928539+0000","last_undegraded":"2024-04-25T08:55:47.928539+0000","last_fullsized":"2024-04-25T08:55:47.928539+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:08.962332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928402+0000","last_change":"2024-04-25T08:55:46.544745+0000","last_active":"2024-04-25T08:55:47.928402+0000","last_peered":"2024-04-25T08:55:47.928402+0000","last_clean":"2024-04-25T08:55:47.928402+0000","last_became_active":"2024-04-25T08:55:46.543987+0000","last_became_peered":"2024-04-25T08:55:46.543987+0000","last_unstale":"2024-04-25T08:55:47.928402+0000","last_undegraded":"2024-04-25T08:55:47.928402+0000","last_fullsized":"2024-04-25T08:55:47.928402+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:33.829039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928069+0000","last_change":"2024-04-25T08:55:45.859531+0000","last_active":"2024-04-25T08:55:47.928069+0000","last_peered":"2024-04-25T08:55:47.928069+0000","last_clean":"2024-04-25T08:55:47.928069+0000","last_became_active":"2024-04-25T08:55:45.859051+0000","last_became_peered":"2024-04-25T08:55:45.859051+0000","last_unstale":"2024-04-25T08:55:47.928069+0000","last_undegraded":"2024-04-25T08:55:47.928069+0000","last_fullsized":"2024-04-25T08:55:47.928069+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:37.297281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.517870+0000","last_change":"2024-04-25T08:55:46.533242+0000","last_active":"2024-04-25T08:55:47.517870+0000","last_peered":"2024-04-25T08:55:47.517870+0000","last_clean":"2024-04-25T08:55:47.517870+0000","last_became_active":"2024-04-25T08:55:46.532509+0000","last_became_peered":"2024-04-25T08:55:46.532509+0000","last_unstale":"2024-04-25T08:55:47.517870+0000","last_undegraded":"2024-04-25T08:55:47.517870+0000","last_fullsized":"2024-04-25T08:55:47.517870+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:10.718524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.929280+0000","last_change":"2024-04-25T08:55:45.838955+0000","last_active":"2024-04-25T08:55:47.929280+0000","last_peered":"2024-04-25T08:55:47.929280+0000","last_clean":"2024-04-25T08:55:47.929280+0000","last_became_active":"2024-04-25T08:55:45.838141+0000","last_became_peered":"2024-04-25T08:55:45.838141+0000","last_unstale":"2024-04-25T08:55:47.929280+0000","last_undegraded":"2024-04-25T08:55:47.929280+0000","last_fullsized":"2024-04-25T08:55:47.929280+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:50.005220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515037+0000","last_change":"2024-04-25T08:55:46.534568+0000","last_active":"2024-04-25T08:55:47.515037+0000","last_peered":"2024-04-25T08:55:47.515037+0000","last_clean":"2024-04-25T08:55:47.515037+0000","last_became_active":"2024-04-25T08:55:46.533790+0000","last_became_peered":"2024-04-25T08:55:46.533790+0000","last_unstale":"2024-04-25T08:55:47.515037+0000","last_undegraded":"2024-04-25T08:55:47.515037+0000","last_fullsized":"2024-04-25T08:55:47.515037+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:32.326436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927612+0000","last_change":"2024-04-25T08:55:45.867890+0000","last_active":"2024-04-25T08:55:47.927612+0000","last_peered":"2024-04-25T08:55:47.927612+0000","last_clean":"2024-04-25T08:55:47.927612+0000","last_became_active":"2024-04-25T08:55:45.866992+0000","last_became_peered":"2024-04-25T08:55:45.866992+0000","last_unstale":"2024-04-25T08:55:47.927612+0000","last_undegraded":"2024-04-25T08:55:47.927612+0000","last_fullsized":"2024-04-25T08:55:47.927612+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:50.697232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515471+0000","last_change":"2024-04-25T08:55:46.551638+0000","last_active":"2024-04-25T08:55:47.515471+0000","last_peered":"2024-04-25T08:55:47.515471+0000","last_clean":"2024-04-25T08:55:47.515471+0000","last_became_active":"2024-04-25T08:55:46.550894+0000","last_became_peered":"2024-04-25T08:55:46.550894+0000","last_unstale":"2024-04-25T08:55:47.515471+0000","last_undegraded":"2024-04-25T08:55:47.515471+0000","last_fullsized":"2024-04-25T08:55:47.515471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:09.965629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926921+0000","last_change":"2024-04-25T08:55:45.836855+0000","last_active":"2024-04-25T08:55:47.926921+0000","last_peered":"2024-04-25T08:55:47.926921+0000","last_clean":"2024-04-25T08:55:47.926921+0000","last_became_active":"2024-04-25T08:55:45.836441+0000","last_became_peered":"2024-04-25T08:55:45.836441+0000","last_unstale":"2024-04-25T08:55:47.926921+0000","last_undegraded":"2024-04-25T08:55:47.926921+0000","last_fullsized":"2024-04-25T08:55:47.926921+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:32.825742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.928218+0000","last_change":"2024-04-25T08:55:46.541655+0000","last_active":"2024-04-25T08:55:47.928218+0000","last_peered":"2024-04-25T08:55:47.928218+0000","last_clean":"2024-04-25T08:55:47.928218+0000","last_became_active":"2024-04-25T08:55:46.540890+0000","last_became_peered":"2024-04-25T08:55:46.540890+0000","last_unstale":"2024-04-25T08:55:47.928218+0000","last_undegraded":"2024-04-25T08:55:47.928218+0000","last_fullsized":"2024-04-25T08:55:47.928218+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:30.953107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927015+0000","last_change":"2024-04-25T08:55:45.848906+0000","last_active":"2024-04-25T08:55:47.927015+0000","last_peered":"2024-04-25T08:55:47.927015+0000","last_clean":"2024-04-25T08:55:47.927015+0000","last_became_active":"2024-04-25T08:55:45.848091+0000","last_became_peered":"2024-04-25T08:55:45.848091+0000","last_unstale":"2024-04-25T08:55:47.927015+0000","last_undegraded":"2024-04-25T08:55:47.927015+0000","last_fullsized":"2024-04-25T08:55:47.927015+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:11.489889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.515089+0000","last_change":"2024-04-25T08:55:46.530967+0000","last_active":"2024-04-25T08:55:47.515089+0000","last_peered":"2024-04-25T08:55:47.515089+0000","last_clean":"2024-04-25T08:55:47.515089+0000","last_became_active":"2024-04-25T08:55:46.530235+0000","last_became_peered":"2024-04-25T08:55:46.530235+0000","last_unstale":"2024-04-25T08:55:47.515089+0000","last_undegraded":"2024-04-25T08:55:47.515089+0000","last_fullsized":"2024-04-25T08:55:47.515089+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:02.980990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927575+0000","last_change":"2024-04-25T08:55:45.853704+0000","last_active":"2024-04-25T08:55:47.927575+0000","last_peered":"2024-04-25T08:55:47.927575+0000","last_clean":"2024-04-25T08:55:47.927575+0000","last_became_active":"2024-04-25T08:55:45.853416+0000","last_became_peered":"2024-04-25T08:55:45.853416+0000","last_unstale":"2024-04-25T08:55:47.927575+0000","last_undegraded":"2024-04-25T08:55:47.927575+0000","last_fullsized":"2024-04-25T08:55:47.927575+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:09.049331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.517533+0000","last_change":"2024-04-25T08:55:46.533639+0000","last_active":"2024-04-25T08:55:47.517533+0000","last_peered":"2024-04-25T08:55:47.517533+0000","last_clean":"2024-04-25T08:55:47.517533+0000","last_became_active":"2024-04-25T08:55:46.532911+0000","last_became_peered":"2024-04-25T08:55:46.532911+0000","last_unstale":"2024-04-25T08:55:47.517533+0000","last_undegraded":"2024-04-25T08:55:47.517533+0000","last_fullsized":"2024-04-25T08:55:47.517533+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:45.817380+0000","last_clean_scrub_stamp":"2024-04-25T08:55:45.817380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:30.953107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.926370+0000","last_change":"2024-04-25T08:55:45.839990+0000","last_active":"2024-04-25T08:55:47.926370+0000","last_peered":"2024-04-25T08:55:47.926370+0000","last_clean":"2024-04-25T08:55:47.926370+0000","last_became_active":"2024-04-25T08:55:45.839607+0000","last_became_peered":"2024-04-25T08:55:45.839607+0000","last_unstale":"2024-04-25T08:55:47.926370+0000","last_undegraded":"2024-04-25T08:55:47.926370+0000","last_fullsized":"2024-04-25T08:55:47.926370+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:44.814082+0000","last_clean_scrub_stamp":"2024-04-25T08:55:44.814082+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:39.887774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":260,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-04-25T08:55:47.927212+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:55:47.927212+0000","last_peered":"2024-04-25T08:55:47.927212+0000","last_clean":"2024-04-25T08:55:47.927212+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:55:47.927212+0000","last_undegraded":"2024-04-25T08:55:47.927212+0000","last_fullsized":"2024-04-25T08:55:47.927212+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1445888,"data_stored":971864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":839680,"data_stored":380754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343992,"statfs":{"total":96636764160,"available":96608247808,"internally_reserved":0,"allocated":839680,"data_stored":380742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":847872,"data_stored":382332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":380720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":380720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":851968,"data_stored":376404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1429504,"data_stored":971088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:55:51.228 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:55:51.228 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:55:51.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5f6ee640 1 Processor -- start 2024-04-25T08:55:51.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5f6ee640 1 -- start start 2024-04-25T08:55:51.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5f6ee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fac5815da80 0x7fac580d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5f6ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac580d07f0 0x7fac5806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5f6ee640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5806be60 con 0x7fac580d1cd0 2024-04-25T08:55:51.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5f6ee640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac5806c440 con 0x7fac5815da80 2024-04-25T08:55:51.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5f6ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac5815de60 con 0x7fac580d07f0 2024-04-25T08:55:51.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5d48a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fac5815da80 0x7fac580d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5d48a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fac5815da80 0x7fac580d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57150/0 (socket says 172.21.15.37:57150) 2024-04-25T08:55:51.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5d48a640 1 -- 172.21.15.37:0/3901832652 learned_addr learned my addr 172.21.15.37:0/3901832652 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:51.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5d48a640 1 -- 172.21.15.37:0/3901832652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac580d07f0 msgr2=0x7fac5806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:51.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5d48a640 1 --2- 172.21.15.37:0/3901832652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac580d07f0 0x7fac5806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5d48a640 1 -- 172.21.15.37:0/3901832652 >> v1:172.21.15.203:6789/0 conn(0x7fac580d1cd0 legacy=0x7fac580d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:51.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.402+0000 7fac5d48a640 1 -- 172.21.15.37:0/3901832652 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac580d2450 con 0x7fac5815da80 2024-04-25T08:55:51.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac5d48a640 1 --2- 172.21.15.37:0/3901832652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac5815da80 0x7fac580d4230 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fac50009e50 tx=0x7fac5001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab029da0456d2cda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:51.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac4ffff640 1 -- 172.21.15.37:0/3901832652 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500a57c0 con 0x7fac5815da80 2024-04-25T08:55:51.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac4ffff640 1 -- 172.21.15.37:0/3901832652 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac500a5960 con 0x7fac5815da80 2024-04-25T08:55:51.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac4ffff640 1 -- 172.21.15.37:0/3901832652 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500a5d30 con 0x7fac5815da80 2024-04-25T08:55:51.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac5f6ee640 1 -- 172.21.15.37:0/3901832652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac5815da80 msgr2=0x7fac580d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac5f6ee640 1 --2- 172.21.15.37:0/3901832652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac5815da80 0x7fac580d4230 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fac50009e50 tx=0x7fac5001c9a0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac5f6ee640 1 -- 172.21.15.37:0/3901832652 shutdown_connections 2024-04-25T08:55:51.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac5f6ee640 1 --2- 172.21.15.37:0/3901832652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac580d07f0 0x7fac5806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac5f6ee640 1 --2- 172.21.15.37:0/3901832652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac5815da80 0x7fac580d4230 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac5f6ee640 1 -- 172.21.15.37:0/3901832652 >> 172.21.15.37:0/3901832652 conn(0x7fac5808b000 msgr2=0x7fac580d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:51.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac5f6ee640 1 -- 172.21.15.37:0/3901832652 shutdown_connections 2024-04-25T08:55:51.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac5f6ee640 1 -- 172.21.15.37:0/3901832652 wait complete. 2024-04-25T08:55:51.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac5f6ee640 1 Processor -- start 2024-04-25T08:55:51.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.406+0000 7fac5f6ee640 1 -- start start 2024-04-25T08:55:51.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5f6ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac580d07f0 0x7fac58154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5f6ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac580d1cd0 0x7fac58156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5f6ee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5815da80 0x7fac58157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5f6ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5806d7a0 con 0x7fac580d07f0 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5f6ee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5806d620 con 0x7fac5815da80 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5f6ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac5806d920 con 0x7fac580d1cd0 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5d48a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac580d07f0 0x7fac58154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5d48a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac580d07f0 0x7fac58154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57154/0 (socket says 172.21.15.37:57154) 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5d48a640 1 -- 172.21.15.37:0/307390108 learned_addr learned my addr 172.21.15.37:0/307390108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5cc89640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac580d1cd0 0x7fac58156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5d48a640 1 -- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac580d1cd0 msgr2=0x7fac58156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5d48a640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac580d1cd0 0x7fac58156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5d48a640 1 -- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5815da80 msgr2=0x7fac58157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5dc8b640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5815da80 0x7fac58157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5d48a640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5815da80 0x7fac58157170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5d48a640 1 -- 172.21.15.37:0/307390108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac58155570 con 0x7fac580d07f0 2024-04-25T08:55:51.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5cc89640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac580d1cd0 0x7fac58156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:51.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5dc8b640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5815da80 0x7fac58157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:51.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5d48a640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac580d07f0 0x7fac58154b00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fac5001b050 tx=0x7fac50019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:51.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac4dffb640 1 -- 172.21.15.37:0/307390108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500b8670 con 0x7fac580d07f0 2024-04-25T08:55:51.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac4dffb640 1 -- 172.21.15.37:0/307390108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac500be070 con 0x7fac580d07f0 2024-04-25T08:55:51.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac4dffb640 1 -- 172.21.15.37:0/307390108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500b8a50 con 0x7fac580d07f0 2024-04-25T08:55:51.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5f6ee640 1 -- 172.21.15.37:0/307390108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac58155850 con 0x7fac580d07f0 2024-04-25T08:55:51.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5f6ee640 1 -- 172.21.15.37:0/307390108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac58155e10 con 0x7fac580d07f0 2024-04-25T08:55:51.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac5f6ee640 1 -- 172.21.15.37:0/307390108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac5804efa0 con 0x7fac580d07f0 2024-04-25T08:55:51.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.410+0000 7fac4dffb640 1 -- 172.21.15.37:0/307390108 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac500b8bf0 con 0x7fac580d07f0 2024-04-25T08:55:51.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.414+0000 7fac4dffb640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac3c066f60 0x7fac3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.414+0000 7fac4dffb640 1 -- 172.21.15.37:0/307390108 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fac50120440 con 0x7fac580d07f0 2024-04-25T08:55:51.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.414+0000 7fac5cc89640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac3c066f60 0x7fac3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.414+0000 7fac5cc89640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac3c066f60 0x7fac3c069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac581f9cd0 tx=0x7fac48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:51.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.418+0000 7fac4dffb640 1 -- 172.21.15.37:0/307390108 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac500165c0 con 0x7fac580d07f0 2024-04-25T08:55:51.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.626+0000 7fac5f6ee640 1 -- 172.21.15.37:0/307390108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fac58073b20 con 0x7fac580d07f0 2024-04-25T08:55:51.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.630+0000 7fac4dffb640 1 -- 172.21.15.37:0/307390108 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7fac500f13e0 con 0x7fac580d07f0 2024-04-25T08:55:51.621 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T08:55:51.621 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA== 2024-04-25T08:55:51.621 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-04-25T08:55:51.621 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-04-25T08:55:51.621 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:55:51.621 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-04-25T08:55:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.630+0000 7fac5f6ee640 1 -- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac3c066f60 msgr2=0x7fac3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.630+0000 7fac5f6ee640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac3c066f60 0x7fac3c069420 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac581f9cd0 tx=0x7fac48009040 comp rx=0 tx=0).stop 2024-04-25T08:55:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.630+0000 7fac5f6ee640 1 -- 172.21.15.37:0/307390108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac580d07f0 msgr2=0x7fac58154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.630+0000 7fac5f6ee640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac580d07f0 0x7fac58154b00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fac5001b050 tx=0x7fac50019740 comp rx=0 tx=0).stop 2024-04-25T08:55:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.634+0000 7fac5f6ee640 1 -- 172.21.15.37:0/307390108 shutdown_connections 2024-04-25T08:55:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.634+0000 7fac5f6ee640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac3c066f60 0x7fac3c069420 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.634+0000 7fac5f6ee640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac5815da80 0x7fac58157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.634+0000 7fac5f6ee640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac580d1cd0 0x7fac58156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.634+0000 7fac5f6ee640 1 --2- 172.21.15.37:0/307390108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac580d07f0 0x7fac58154b00 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.634+0000 7fac5f6ee640 1 -- 172.21.15.37:0/307390108 >> 172.21.15.37:0/307390108 conn(0x7fac5808b000 msgr2=0x7fac580b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.634+0000 7fac5f6ee640 1 -- 172.21.15.37:0/307390108 shutdown_connections 2024-04-25T08:55:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.634+0000 7fac5f6ee640 1 -- 172.21.15.37:0/307390108 wait complete. 2024-04-25T08:55:51.638 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:51.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.770+0000 7feafcfd2640 1 Processor -- start 2024-04-25T08:55:51.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.770+0000 7feafcfd2640 1 -- start start 2024-04-25T08:55:51.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.770+0000 7feafcfd2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feaf80b5270 0x7feaf80b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.770+0000 7feafcfd2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feaf80b5650 0x7feaf80d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.770+0000 7feafcfd2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf806bff0 con 0x7feaf80d4440 2024-04-25T08:55:51.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.770+0000 7feafcfd2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf806cc00 con 0x7feaf80b5270 2024-04-25T08:55:51.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.770+0000 7feafcfd2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf806c2d0 con 0x7feaf80b5650 2024-04-25T08:55:51.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feaf80b5270 0x7feaf80b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7feaf80b5270 0x7feaf80b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57164/0 (socket says 172.21.15.37:57164) 2024-04-25T08:55:51.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf659c640 1 -- 172.21.15.37:0/3278757371 learned_addr learned my addr 172.21.15.37:0/3278757371 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:51.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf5d9b640 1 --2- 172.21.15.37:0/3278757371 >> v2:172.21.15.203:3300/0 conn(0x7feaf80b5650 0x7feaf80d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf659c640 1 -- 172.21.15.37:0/3278757371 >> v2:172.21.15.203:3300/0 conn(0x7feaf80b5650 msgr2=0x7feaf80d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:51.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf659c640 1 --2- 172.21.15.37:0/3278757371 >> v2:172.21.15.203:3300/0 conn(0x7feaf80b5650 0x7feaf80d3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf659c640 1 -- 172.21.15.37:0/3278757371 >> v1:172.21.15.203:6789/0 conn(0x7feaf80d4440 legacy=0x7feaf806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:51.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf659c640 1 -- 172.21.15.37:0/3278757371 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf806c5b0 con 0x7feaf80b5270 2024-04-25T08:55:51.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf659c640 1 --2- 172.21.15.37:0/3278757371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf80b5270 0x7feaf80b0380 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7feaf80cdce0 tx=0x7feae801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c532ffe81d7de0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:51.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf559a640 1 -- 172.21.15.37:0/3278757371 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae80aa660 con 0x7feaf80b5270 2024-04-25T08:55:51.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf559a640 1 -- 172.21.15.37:0/3278757371 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae80af070 con 0x7feaf80b5270 2024-04-25T08:55:51.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feaf559a640 1 -- 172.21.15.37:0/3278757371 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae80aaa20 con 0x7feaf80b5270 2024-04-25T08:55:51.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feafcfd2640 1 -- 172.21.15.37:0/3278757371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf80b5270 msgr2=0x7feaf80b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feafcfd2640 1 --2- 172.21.15.37:0/3278757371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf80b5270 0x7feaf80b0380 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7feaf80cdce0 tx=0x7feae801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feafcfd2640 1 -- 172.21.15.37:0/3278757371 shutdown_connections 2024-04-25T08:55:51.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feafcfd2640 1 --2- 172.21.15.37:0/3278757371 >> v2:172.21.15.203:3300/0 conn(0x7feaf80b5650 0x7feaf80d3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feafcfd2640 1 --2- 172.21.15.37:0/3278757371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf80b5270 0x7feaf80b0380 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feafcfd2640 1 -- 172.21.15.37:0/3278757371 >> 172.21.15.37:0/3278757371 conn(0x7feaf808b000 msgr2=0x7feaf80d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:51.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feafcfd2640 1 -- 172.21.15.37:0/3278757371 shutdown_connections 2024-04-25T08:55:51.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.774+0000 7feafcfd2640 1 -- 172.21.15.37:0/3278757371 wait complete. 2024-04-25T08:55:51.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feafcfd2640 1 Processor -- start 2024-04-25T08:55:51.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feafcfd2640 1 -- start start 2024-04-25T08:55:51.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feafcfd2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf80b5270 0x7feaf814d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feafcfd2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf80b5650 0x7feaf814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feafcfd2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf80d4440 0x7feaf8150220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feafcfd2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf815dfa0 con 0x7feaf80b5650 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feafcfd2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf815de20 con 0x7feaf80d4440 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feafcfd2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf816a720 con 0x7feaf80b5270 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf80b5650 0x7feaf814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf80b5650 0x7feaf814fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57166/0 (socket says 172.21.15.37:57166) 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf5d9b640 1 -- 172.21.15.37:0/3487749549 learned_addr learned my addr 172.21.15.37:0/3487749549 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf659c640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf80b5270 0x7feaf814d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf6d9d640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf80d4440 0x7feaf8150220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf5d9b640 1 -- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf80b5270 msgr2=0x7feaf814d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf5d9b640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf80b5270 0x7feaf814d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf5d9b640 1 -- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf80d4440 msgr2=0x7feaf8150220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf5d9b640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf80d4440 0x7feaf8150220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf5d9b640 1 -- 172.21.15.37:0/3487749549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf80cce40 con 0x7feaf80b5650 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf659c640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf80b5270 0x7feaf814d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:51.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feaf5d9b640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf80b5650 0x7feaf814fce0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7feaec00d800 tx=0x7feaec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feae6ffd640 1 -- 172.21.15.37:0/3487749549 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec0059a0 con 0x7feaf80b5650 2024-04-25T08:55:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feae6ffd640 1 -- 172.21.15.37:0/3487749549 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaec005b40 con 0x7feaf80b5650 2024-04-25T08:55:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feafcfd2640 1 -- 172.21.15.37:0/3487749549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf80cb400 con 0x7feaf80b5650 2024-04-25T08:55:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feae6ffd640 1 -- 172.21.15.37:0/3487749549 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaec005d30 con 0x7feaf80b5650 2024-04-25T08:55:51.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feafcfd2640 1 -- 172.21.15.37:0/3487749549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf80cd3a0 con 0x7feaf80b5650 2024-04-25T08:55:51.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feafcfd2640 1 -- 172.21.15.37:0/3487749549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf804efa0 con 0x7feaf80b5650 2024-04-25T08:55:51.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.778+0000 7feae6ffd640 1 -- 172.21.15.37:0/3487749549 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feaec01b020 con 0x7feaf80b5650 2024-04-25T08:55:51.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.782+0000 7feae6ffd640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead8066f60 0x7fead8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:51.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.782+0000 7feae6ffd640 1 -- 172.21.15.37:0/3487749549 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feaec085030 con 0x7feaf80b5650 2024-04-25T08:55:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.782+0000 7feaf659c640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead8066f60 0x7fead8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:51.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.782+0000 7feaf659c640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead8066f60 0x7fead8069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feae80ae000 tx=0x7feae80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:51.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.786+0000 7feae6ffd640 1 -- 172.21.15.37:0/3487749549 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feaf80cce40 con 0x7feaf80b5650 2024-04-25T08:55:51.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.978+0000 7feafcfd2640 1 -- 172.21.15.37:0/3487749549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feaf80cda90 con 0x7feaf80b5650 2024-04-25T08:55:51.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feae6ffd640 1 -- 172.21.15.37:0/3487749549 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4939 (secure 0 0 0) 0x7feaec055eb0 con 0x7feaf80b5650 2024-04-25T08:55:51.971 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:51.972 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/486740526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":486740526},{"type":"v1","addr":"172.21.15.37:6833","nonce":486740526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/121484331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":121484331},{"type":"v1","addr":"172.21.15.203:6835","nonce":121484331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2501630566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2501630566},{"type":"v1","addr":"172.21.15.203:6837","nonce":2501630566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:55:46.200939+0000","modified":"2024-04-25T08:55:49.230920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6405":{"gid":6405,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/2189232625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2189232625},{"type":"v1","addr":"172.21.15.37:6835","nonce":2189232625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:55:51.972 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-04-25T08:55:51.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 -- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead8066f60 msgr2=0x7fead8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead8066f60 0x7fead8069420 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7feae80ae000 tx=0x7feae80a7000 comp rx=0 tx=0).stop 2024-04-25T08:55:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 -- 172.21.15.37:0/3487749549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf80b5650 msgr2=0x7feaf814fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf80b5650 0x7feaf814fce0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7feaec00d800 tx=0x7feaec00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 -- 172.21.15.37:0/3487749549 shutdown_connections 2024-04-25T08:55:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fead8066f60 0x7fead8069420 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf80d4440 0x7feaf8150220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feaf80b5650 0x7feaf814fce0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 --2- 172.21.15.37:0/3487749549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf80b5270 0x7feaf814d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 -- 172.21.15.37:0/3487749549 >> 172.21.15.37:0/3487749549 conn(0x7feaf808b000 msgr2=0x7feaf80d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 -- 172.21.15.37:0/3487749549 shutdown_connections 2024-04-25T08:55:51.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:51.982+0000 7feafcfd2640 1 -- 172.21.15.37:0/3487749549 wait complete. 2024-04-25T08:55:52.392 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:55:46.200939+0000', 'modified': '2024-04-25T08:55:49.230920+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6405}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6405': {'gid': 6405, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/2189232625', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2189232625}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2189232625}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:55:52.392 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:55:52.393 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:55:52.449 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T08:55:52.458+0000 7fac41d13640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:55:52.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f804b8ae640 1 Processor -- start 2024-04-25T08:55:52.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f804b8ae640 1 -- start start 2024-04-25T08:55:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f804b8ae640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f804409a180 0x7f80440b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f804b8ae640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80440cbec0 con 0x7f804409a7b0 2024-04-25T08:55:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f804b8ae640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80440cc1a0 con 0x7f80440b2c30 2024-04-25T08:55:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f804b8ae640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80440cc480 con 0x7f804409a180 2024-04-25T08:55:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8049e4b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f804409a7b0 0x7f80440ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41948/0 (socket says 172.21.15.37:41948) 2024-04-25T08:55:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8049e4b640 1 -- 172.21.15.37:0/3677878469 learned_addr learned my addr 172.21.15.37:0/3677878469 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:52.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8048e49640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f804409a180 0x7f80440b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395114560 0 0) 0x7f80440cbec0 con 0x7f804409a7b0 2024-04-25T08:55:52.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f802c003530 con 0x7f804409a7b0 2024-04-25T08:55:52.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3685364455 0 0) 0x7f802c003530 con 0x7f804409a7b0 2024-04-25T08:55:52.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f804409a180 msgr2=0x7f80440b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f804409a180 0x7f80440b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 >> v1:172.21.15.203:6789/0 conn(0x7f80440b2c30 legacy=0x7f8044099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440cc760 con 0x7f804409a7b0 2024-04-25T08:55:52.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f8034001680 con 0x7f804409a7b0 2024-04-25T08:55:52.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 >> v1:172.21.15.37:6789/0 conn(0x7f804409a7b0 legacy=0x7f80440ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f802c003a60 0x7f802c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f802c004810 0x7f802c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f804409a180 0x7f802c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440cc480 con 0x7f802c004810 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c00ba80 con 0x7f802c003a60 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c00bc50 con 0x7f804409a180 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8048e49640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f802c004810 0x7f802c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f804964a640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f802c003a60 0x7f802c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8049e4b640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f804409a180 0x7f802c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8048e49640 1 -- 172.21.15.37:0/3677878469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f804409a180 msgr2=0x7f802c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8048e49640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f804409a180 0x7f802c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8048e49640 1 -- 172.21.15.37:0/3677878469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f802c003a60 msgr2=0x7f802c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8048e49640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f802c003a60 0x7f802c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.538+0000 7f8048e49640 1 -- 172.21.15.37:0/3677878469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c003530 con 0x7f802c004810 2024-04-25T08:55:52.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.542+0000 7f8048e49640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f802c004810 0x7f802c004c90 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8038009ae0 tx=0x7f803801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b101730ee85142a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.542+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380b2070 con 0x7f802c004810 2024-04-25T08:55:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.542+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80380a5600 con 0x7f802c004810 2024-04-25T08:55:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.542+0000 7f8043fff640 1 -- 172.21.15.37:0/3677878469 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380096b0 con 0x7f802c004810 2024-04-25T08:55:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.542+0000 7f804b8ae640 1 -- 172.21.15.37:0/3677878469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f802c004810 msgr2=0x7f802c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.542+0000 7f804b8ae640 1 --2- 172.21.15.37:0/3677878469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f802c004810 0x7f802c004c90 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f8038009ae0 tx=0x7f803801ca10 comp rx=0 tx=0).stop 2024-04-25T08:55:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.542+0000 7f8049e4b640 1 -- 172.21.15.37:0/3677878469 reap_dead start 2024-04-25T08:55:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.542+0000 7f804b8ae640 1 -- 172.21.15.37:0/3677878469 shutdown_connections 2024-04-25T08:55:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.542+0000 7f804b8ae640 1 -- 172.21.15.37:0/3677878469 >> 172.21.15.37:0/3677878469 conn(0x7f80440b3010 msgr2=0x7f80440b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.542+0000 7f804b8ae640 1 -- 172.21.15.37:0/3677878469 shutdown_connections 2024-04-25T08:55:52.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.542+0000 7f804b8ae640 1 -- 172.21.15.37:0/3677878469 wait complete. 2024-04-25T08:55:52.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f804b8ae640 1 Processor -- start 2024-04-25T08:55:52.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f804b8ae640 1 -- start start 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f804b8ae640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f802c004810 0x7f80441f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f804b8ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80441f41d0 0x7f80440d0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f804b8ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80440d08d0 0x7f80440c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f804b8ae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80440cbb70 con 0x7f80440d08d0 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f804b8ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804409ac40 con 0x7f80441f41d0 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f804b8ae640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804406c200 con 0x7f802c004810 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8049e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80440d08d0 0x7f80440c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f804964a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f802c004810 0x7f80441f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8049e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80440d08d0 0x7f80440c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57190/0 (socket says 172.21.15.37:57190) 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8049e4b640 1 -- 172.21.15.37:0/587730355 learned_addr learned my addr 172.21.15.37:0/587730355 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8048e49640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80441f41d0 0x7f80440d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8049e4b640 1 -- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f802c004810 msgr2=0x7f80441f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8049e4b640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f802c004810 0x7f80441f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8049e4b640 1 -- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80441f41d0 msgr2=0x7f80440d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8049e4b640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80441f41d0 0x7f80440d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8049e4b640 1 -- 172.21.15.37:0/587730355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80441f3430 con 0x7f80440d08d0 2024-04-25T08:55:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8049e4b640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80440d08d0 0x7f80440c8930 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8044072d30 tx=0x7f803400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8041ffb640 1 -- 172.21.15.37:0/587730355 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034019070 con 0x7f80440d08d0 2024-04-25T08:55:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8041ffb640 1 -- 172.21.15.37:0/587730355 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803400c660 con 0x7f80440d08d0 2024-04-25T08:55:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f804b8ae640 1 -- 172.21.15.37:0/587730355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80441f36f0 con 0x7f80440d08d0 2024-04-25T08:55:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8041ffb640 1 -- 172.21.15.37:0/587730355 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803400ca50 con 0x7f80440d08d0 2024-04-25T08:55:52.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f804b8ae640 1 -- 172.21.15.37:0/587730355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80441f3050 con 0x7f80440d08d0 2024-04-25T08:55:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8041ffb640 1 -- 172.21.15.37:0/587730355 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f803400cbf0 con 0x7f80440d08d0 2024-04-25T08:55:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.546+0000 7f8041ffb640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8030066f90 0x7f8030069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f8041ffb640 1 -- 172.21.15.37:0/587730355 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8034084590 con 0x7f80440d08d0 2024-04-25T08:55:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804964a640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8030066f90 0x7f8030069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 Processor -- start 2024-04-25T08:55:52.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 -- start start 2024-04-25T08:55:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804964a640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8030066f90 0x7f8030069450 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f803c0012f0 tx=0x7f803c000f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567255920 0x559567255d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559567256240 0x55956720b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55956720bf30 0x55956720c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595670f35f0 con 0x55956720bf30 2024-04-25T08:55:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55956705ada0 con 0x559567255920 2024-04-25T08:55:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559566fe3150 con 0x559567256240 2024-04-25T08:55:52.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f8049e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55956720bf30 0x55956720c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f8049e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55956720bf30 0x55956720c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57202/0 (socket says 172.21.15.37:57202) 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f8049e4b640 1 -- 172.21.15.37:0/2575121763 learned_addr learned my addr 172.21.15.37:0/2575121763 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804964a640 1 --2- 172.21.15.37:0/2575121763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567255920 0x559567255d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f8048e49640 1 --2- 172.21.15.37:0/2575121763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559567256240 0x55956720b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f8049e4b640 1 -- 172.21.15.37:0/2575121763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559567256240 msgr2=0x55956720b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f8049e4b640 1 --2- 172.21.15.37:0/2575121763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559567256240 0x55956720b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f8049e4b640 1 -- 172.21.15.37:0/2575121763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567255920 msgr2=0x559567255d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f8049e4b640 1 --2- 172.21.15.37:0/2575121763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567255920 0x559567255d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f8049e4b640 1 -- 172.21.15.37:0/2575121763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559566ffcca0 con 0x55956720bf30 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f8049e4b640 1 --2- 172.21.15.37:0/2575121763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55956720bf30 0x55956720c310 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f803405cf80 tx=0x7f8034054d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955297c66d8cd797 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f802affd640 1 -- 172.21.15.37:0/2575121763 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034058070 con 0x55956720bf30 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f802affd640 1 -- 172.21.15.37:0/2575121763 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034058ef0 con 0x55956720bf30 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f802affd640 1 -- 172.21.15.37:0/2575121763 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803405d4b0 con 0x55956720bf30 2024-04-25T08:55:52.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 -- 172.21.15.37:0/2575121763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55956720bf30 msgr2=0x55956720c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 --2- 172.21.15.37:0/2575121763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55956720bf30 0x55956720c310 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f803405cf80 tx=0x7f8034054d50 comp rx=0 tx=0).stop 2024-04-25T08:55:52.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 -- 172.21.15.37:0/2575121763 shutdown_connections 2024-04-25T08:55:52.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 --2- 172.21.15.37:0/2575121763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55956720bf30 0x55956720c310 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 --2- 172.21.15.37:0/2575121763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559567256240 0x55956720b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 --2- 172.21.15.37:0/2575121763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567255920 0x559567255d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 -- 172.21.15.37:0/2575121763 >> 172.21.15.37:0/2575121763 conn(0x5595672122a0 msgr2=0x55956720a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:52.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 -- 172.21.15.37:0/2575121763 shutdown_connections 2024-04-25T08:55:52.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 -- 172.21.15.37:0/2575121763 wait complete. 2024-04-25T08:55:52.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 -- start start 2024-04-25T08:55:52.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.550+0000 7f804ea1b000 1 Processor -- start 2024-04-25T08:55:52.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55956720bf30 0x55956725cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x55956725d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559567256240 0x55956720faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559567015990 con 0x559567255920 2024-04-25T08:55:52.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559567056e40 con 0x55956720bf30 2024-04-25T08:55:52.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559566fe6900 con 0x559567256240 2024-04-25T08:55:52.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 10 client.0 _ll_register_callbacks cb 0x55956710e1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:52.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8048e49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x55956725d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8048e49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x55956725d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57204/0 (socket says 172.21.15.37:57204) 2024-04-25T08:55:52.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8048e49640 1 -- 172.21.15.37:0/4174910302 learned_addr learned my addr 172.21.15.37:0/4174910302 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8049e4b640 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559567256240 0x55956720faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804964a640 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55956720bf30 0x55956725cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8048e49640 1 -- 172.21.15.37:0/4174910302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559567256240 msgr2=0x55956720faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8048e49640 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559567256240 0x55956720faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8048e49640 1 -- 172.21.15.37:0/4174910302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55956720bf30 msgr2=0x55956725cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8048e49640 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55956720bf30 0x55956725cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8048e49640 1 -- 172.21.15.37:0/4174910302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559567013630 con 0x559567255920 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8048e49640 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x55956725d1c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f80380b0040 tx=0x7f80380197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739f1e185866fb6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8023fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8023fff640 1 -- 172.21.15.37:0/4174910302 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380b2040 con 0x559567255920 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8023fff640 1 -- 172.21.15.37:0/4174910302 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80380a5bd0 con 0x559567255920 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 -- 172.21.15.37:0/4174910302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8034058ef0 con 0x559567255920 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8023fff640 1 -- 172.21.15.37:0/4174910302 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80380a5dd0 con 0x559567255920 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 10 client.6534 fetch_fsmap learned FSMap version 33 2024-04-25T08:55:52.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 -- 172.21.15.37:0/4174910302 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x559566fe33a0 con 0x559567255920 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8023fff640 1 -- 172.21.15.37:0/4174910302 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034058ef0 con 0x559567255920 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8023fff640 1 -- 172.21.15.37:0/4174910302 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8038004040 con 0x559567255920 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 10 client.6534 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 10 client.6534 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 20 client.6534 populate_metadata read hostname 'smithi037' 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559567216af0 0x5595672104b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 4 client.6534 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 -- 172.21.15.37:0/4174910302 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559566ff6a50 con 0x559567216af0 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804964a640 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559567216af0 0x5595672104b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804964a640 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559567216af0 0x5595672104b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def6cd733e7cf315 server_cookie=14db0a051d71cb7d in_seq=0 out_seq=0 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8023fff640 10 client.6534 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8023fff640 1 -- 172.21.15.37:0/4174910302 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559566ff6a50 con 0x559567216af0 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8023fff640 10 client.6534 handle_command_reply: tid=0 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 client.6534 shutdown 2024-04-25T08:55:52.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 -- 172.21.15.37:0/4174910302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 msgr2=0x55956725d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x55956725d1c0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f80380b0040 tx=0x7f80380197b0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 -- 172.21.15.37:0/4174910302 shutdown_connections 2024-04-25T08:55:52.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559567216af0 0x5595672104b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559567256240 0x55956720faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x55956725d1c0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 --2- 172.21.15.37:0/4174910302 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55956720bf30 0x55956725cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f8023fff640 0 client.6534 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:52.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 -- 172.21.15.37:0/4174910302 >> 172.21.15.37:0/4174910302 conn(0x5595672122a0 msgr2=0x55956720a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:52.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 -- 172.21.15.37:0/4174910302 shutdown_connections 2024-04-25T08:55:52.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 1 -- 172.21.15.37:0/4174910302 wait complete. 2024-04-25T08:55:52.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.554+0000 7f804ea1b000 20 client.6534 trim_cache size 0 max 16384 2024-04-25T08:55:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f804ea1b000 1 Processor -- start 2024-04-25T08:55:52.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f804ea1b000 1 -- start start 2024-04-25T08:55:52.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55956720bf30 0x55956720f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567255920 0x55956720f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567256240 0x55956725d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f804ea1b000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441ff8b0 con 0x559567256240 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f804ea1b000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441fecb0 con 0x559567255920 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f804ea1b000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80441ff2b0 con 0x55956720bf30 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f8049e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567256240 0x55956725d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f8049e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567256240 0x55956725d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57210/0 (socket says 172.21.15.37:57210) 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f8049e4b640 1 -- 172.21.15.37:0/3840534672 learned_addr learned my addr 172.21.15.37:0/3840534672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f8049e4b640 1 -- 172.21.15.37:0/3840534672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55956720bf30 msgr2=0x55956720f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f804964a640 1 --2- 172.21.15.37:0/3840534672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55956720bf30 0x55956720f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.570+0000 7f8048e49640 1 --2- 172.21.15.37:0/3840534672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567255920 0x55956720f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8049e4b640 1 --2- 172.21.15.37:0/3840534672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55956720bf30 0x55956720f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8049e4b640 1 -- 172.21.15.37:0/3840534672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567255920 msgr2=0x55956720f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8049e4b640 1 --2- 172.21.15.37:0/3840534672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567255920 0x55956720f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8049e4b640 1 -- 172.21.15.37:0/3840534672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559567263980 con 0x559567256240 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8048e49640 1 --2- 172.21.15.37:0/3840534672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567255920 0x55956720f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804964a640 1 --2- 172.21.15.37:0/3840534672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55956720bf30 0x55956720f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8049e4b640 1 --2- 172.21.15.37:0/3840534672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567256240 0x55956725d780 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f803400f010 tx=0x7f8034016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8536aa18fada4a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f802a7fc640 1 -- 172.21.15.37:0/3840534672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034058040 con 0x559567256240 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f802a7fc640 1 -- 172.21.15.37:0/3840534672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803405d4b0 con 0x559567256240 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f802a7fc640 1 -- 172.21.15.37:0/3840534672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803405b0b0 con 0x559567256240 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 -- 172.21.15.37:0/3840534672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567256240 msgr2=0x55956725d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 --2- 172.21.15.37:0/3840534672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567256240 0x55956725d780 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f803400f010 tx=0x7f8034016000 comp rx=0 tx=0).stop 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 -- 172.21.15.37:0/3840534672 shutdown_connections 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 --2- 172.21.15.37:0/3840534672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567256240 0x55956725d780 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 --2- 172.21.15.37:0/3840534672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567255920 0x55956720f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 --2- 172.21.15.37:0/3840534672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55956720bf30 0x55956720f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 -- 172.21.15.37:0/3840534672 >> 172.21.15.37:0/3840534672 conn(0x5595672122a0 msgr2=0x5595672113b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:52.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 -- 172.21.15.37:0/3840534672 shutdown_connections 2024-04-25T08:55:52.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 -- 172.21.15.37:0/3840534672 wait complete. 2024-04-25T08:55:52.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 -- start start 2024-04-25T08:55:52.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 Processor -- start 2024-04-25T08:55:52.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55956720bf30 0x559567260db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x5595672713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567256240 0x5595672718e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55956705f3d0 con 0x559567255920 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559567116d50 con 0x559567256240 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559566ff5980 con 0x55956720bf30 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 10 client.0 _ll_register_callbacks cb 0x7f80380b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8048e49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x5595672713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8048e49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x5595672713a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57214/0 (socket says 172.21.15.37:57214) 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8048e49640 1 -- 172.21.15.37:0/2609911982 learned_addr learned my addr 172.21.15.37:0/2609911982 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8049e4b640 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567256240 0x5595672718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804964a640 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55956720bf30 0x559567260db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8048e49640 1 -- 172.21.15.37:0/2609911982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55956720bf30 msgr2=0x559567260db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8048e49640 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55956720bf30 0x559567260db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8048e49640 1 -- 172.21.15.37:0/2609911982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567256240 msgr2=0x5595672718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8048e49640 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567256240 0x5595672718e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8048e49640 1 -- 172.21.15.37:0/2609911982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559567256070 con 0x559567255920 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8048e49640 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x5595672713a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8038006f90 tx=0x7f8038009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e582be4641abde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8023fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8023fff640 1 -- 172.21.15.37:0/2609911982 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380b2040 con 0x559567255920 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8023fff640 1 -- 172.21.15.37:0/2609911982 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8038005db0 con 0x559567255920 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 -- 172.21.15.37:0/2609911982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f803405d4b0 con 0x559567255920 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8023fff640 1 -- 172.21.15.37:0/2609911982 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80380d3050 con 0x559567255920 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 10 client.6540 fetch_fsmap learned FSMap version 33 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 1 -- 172.21.15.37:0/2609911982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5595670dc600 con 0x559567255920 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8023fff640 1 -- 172.21.15.37:0/2609911982 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803405d4b0 con 0x559567255920 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f8023fff640 1 -- 172.21.15.37:0/2609911982 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f80380d6020 con 0x559567255920 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.574+0000 7f804ea1b000 10 client.6540 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 10 client.6540 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:55:52.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 20 client.6540 populate_metadata read hostname 'smithi037' 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559567271e20 0x5595672b6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 4 client.6540 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 -- 172.21.15.37:0/2609911982 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5595670682e0 con 0x559567271e20 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804964a640 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559567271e20 0x5595672b6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804964a640 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559567271e20 0x5595672b6350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b30fb1f64a9d599e server_cookie=578efc5b7a79255c in_seq=0 out_seq=0 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f8023fff640 10 client.6540 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f8023fff640 1 -- 172.21.15.37:0/2609911982 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5595670682e0 con 0x559567271e20 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f8023fff640 10 client.6540 handle_command_reply: tid=0 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 client.6540 shutdown 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 -- 172.21.15.37:0/2609911982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 msgr2=0x5595672713a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x5595672713a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8038006f90 tx=0x7f8038009190 comp rx=0 tx=0).stop 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 -- 172.21.15.37:0/2609911982 shutdown_connections 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559567271e20 0x5595672b6350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559567256240 0x5595672718e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559567255920 0x5595672713a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 --2- 172.21.15.37:0/2609911982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55956720bf30 0x559567260db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f8023fff640 0 client.6540 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 -- 172.21.15.37:0/2609911982 >> 172.21.15.37:0/2609911982 conn(0x5595672122a0 msgr2=0x55956720e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 -- 172.21.15.37:0/2609911982 shutdown_connections 2024-04-25T08:55:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 1 -- 172.21.15.37:0/2609911982 wait complete. 2024-04-25T08:55:52.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804ea1b000 20 client.6540 trim_cache size 0 max 16384 2024-04-25T08:55:52.573 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:55:52.573 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T08:55:52.573 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T08:55:52.573 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6405, 2024-04-25T08:55:52.573 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T08:55:52.573 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T08:55:52.574 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T08:55:52.574 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 5.7344018080000003, 2024-04-25T08:55:52.574 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 33, 2024-04-25T08:55:52.574 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 69, 2024-04-25T08:55:52.574 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 69, 2024-04-25T08:55:52.574 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 11.107980825 2024-04-25T08:55:52.574 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 -- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8030066f90 msgr2=0x7f8030069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8030066f90 0x7f8030069450 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f803c0012f0 tx=0x7f803c000f50 comp rx=0 tx=0).stop 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 -- 172.21.15.37:0/587730355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80440d08d0 msgr2=0x7f80440c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80440d08d0 0x7f80440c8930 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f8044072d30 tx=0x7f803400eb90 comp rx=0 tx=0).stop 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 -- 172.21.15.37:0/587730355 shutdown_connections 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8030066f90 0x7f8030069450 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80440d08d0 0x7f80440c8930 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80441f41d0 0x7f80440d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 --2- 172.21.15.37:0/587730355 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f802c004810 0x7f80441f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 -- 172.21.15.37:0/587730355 >> 172.21.15.37:0/587730355 conn(0x7f80440b3010 msgr2=0x7f8044160c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 -- 172.21.15.37:0/587730355 shutdown_connections 2024-04-25T08:55:52.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.578+0000 7f804b8ae640 1 -- 172.21.15.37:0/587730355 wait complete. 2024-04-25T08:55:52.584 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:55:52.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.706+0000 7f58e70a4640 1 Processor -- start 2024-04-25T08:55:52.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.706+0000 7f58e70a4640 1 -- start start 2024-04-25T08:55:52.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f58e00ce510 0x7f58e00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e0144ca0 0x7f58e006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e00cee30 con 0x7f58e006ec60 2024-04-25T08:55:52.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e00cc2b0 con 0x7f58e00ce510 2024-04-25T08:55:52.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e006e290 con 0x7f58e0144ca0 2024-04-25T08:55:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e4e40640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f58e00ce510 0x7f58e00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e5641640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f58e006ec60 0x7f58e0142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41962/0 (socket says 172.21.15.37:41962) 2024-04-25T08:55:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e5641640 1 -- 172.21.15.37:0/111862647 learned_addr learned my addr 172.21.15.37:0/111862647 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58dffff640 1 --2- 172.21.15.37:0/111862647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e0144ca0 0x7f58e006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e4e40640 1 -- 172.21.15.37:0/111862647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e0144ca0 msgr2=0x7f58e006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e4e40640 1 --2- 172.21.15.37:0/111862647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e0144ca0 0x7f58e006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e4e40640 1 -- 172.21.15.37:0/111862647 >> v1:172.21.15.37:6789/0 conn(0x7f58e006ec60 legacy=0x7f58e0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e4e40640 1 -- 172.21.15.37:0/111862647 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e00ccc70 con 0x7f58e00ce510 2024-04-25T08:55:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58dffff640 1 --2- 172.21.15.37:0/111862647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e0144ca0 0x7f58e006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e4e40640 1 --2- 172.21.15.37:0/111862647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e00ce510 0x7f58e00ce8f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f58d00097c0 tx=0x7f58d001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90216c7cc42fb285 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58df7fe640 1 -- 172.21.15.37:0/111862647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d00b5070 con 0x7f58e00ce510 2024-04-25T08:55:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58df7fe640 1 -- 172.21.15.37:0/111862647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d00a8990 con 0x7f58e00ce510 2024-04-25T08:55:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58df7fe640 1 -- 172.21.15.37:0/111862647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d00a8d60 con 0x7f58e00ce510 2024-04-25T08:55:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 -- 172.21.15.37:0/111862647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e00ce510 msgr2=0x7f58e00ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 --2- 172.21.15.37:0/111862647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e00ce510 0x7f58e00ce8f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f58d00097c0 tx=0x7f58d001ec90 comp rx=0 tx=0).stop 2024-04-25T08:55:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 -- 172.21.15.37:0/111862647 shutdown_connections 2024-04-25T08:55:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 --2- 172.21.15.37:0/111862647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e0144ca0 0x7f58e006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 --2- 172.21.15.37:0/111862647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e00ce510 0x7f58e00ce8f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 -- 172.21.15.37:0/111862647 >> 172.21.15.37:0/111862647 conn(0x7f58e00b11a0 msgr2=0x7f58e00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 -- 172.21.15.37:0/111862647 shutdown_connections 2024-04-25T08:55:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.710+0000 7f58e70a4640 1 -- 172.21.15.37:0/111862647 wait complete. 2024-04-25T08:55:52.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e70a4640 1 Processor -- start 2024-04-25T08:55:52.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e70a4640 1 -- start start 2024-04-25T08:55:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e70a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e006ec60 0x7f58e01f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e70a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e00ce510 0x7f58e0143090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e70a4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e0144ca0 0x7f58e01435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e70a4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e00cc900 con 0x7f58e006ec60 2024-04-25T08:55:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e70a4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e006df20 con 0x7f58e0144ca0 2024-04-25T08:55:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e70a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e0142d90 con 0x7f58e00ce510 2024-04-25T08:55:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e4e40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e006ec60 0x7f58e01f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e4e40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e006ec60 0x7f58e01f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57238/0 (socket says 172.21.15.37:57238) 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e4e40640 1 -- 172.21.15.37:0/1968842268 learned_addr learned my addr 172.21.15.37:0/1968842268 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e5641640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e0144ca0 0x7f58e01435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58dffff640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e00ce510 0x7f58e0143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e4e40640 1 -- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e00ce510 msgr2=0x7f58e0143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e4e40640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e00ce510 0x7f58e0143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e4e40640 1 -- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e0144ca0 msgr2=0x7f58e01435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e4e40640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e0144ca0 0x7f58e01435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e4e40640 1 -- 172.21.15.37:0/1968842268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e014c9e0 con 0x7f58e006ec60 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58dffff640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e00ce510 0x7f58e0143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e4e40640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e006ec60 0x7f58e01f23c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f58d00b3040 tx=0x7f58d001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58dd7fa640 1 -- 172.21.15.37:0/1968842268 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d00bf070 con 0x7f58e006ec60 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58dd7fa640 1 -- 172.21.15.37:0/1968842268 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d00b0900 con 0x7f58e006ec60 2024-04-25T08:55:52.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58dd7fa640 1 -- 172.21.15.37:0/1968842268 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d00b5040 con 0x7f58e006ec60 2024-04-25T08:55:52.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e70a4640 1 -- 172.21.15.37:0/1968842268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e014cc80 con 0x7f58e006ec60 2024-04-25T08:55:52.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.714+0000 7f58e70a4640 1 -- 172.21.15.37:0/1968842268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e014d240 con 0x7f58e006ec60 2024-04-25T08:55:52.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.718+0000 7f58e70a4640 1 -- 172.21.15.37:0/1968842268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e004efa0 con 0x7f58e006ec60 2024-04-25T08:55:52.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.718+0000 7f58dd7fa640 1 -- 172.21.15.37:0/1968842268 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58d00b0cf0 con 0x7f58e006ec60 2024-04-25T08:55:52.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.718+0000 7f58dd7fa640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58cc066f60 0x7f58cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:52.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.718+0000 7f58dd7fa640 1 -- 172.21.15.37:0/1968842268 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58d0121cd0 con 0x7f58e006ec60 2024-04-25T08:55:52.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.718+0000 7f58dffff640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58cc066f60 0x7f58cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:52.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.718+0000 7f58dffff640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58cc066f60 0x7f58cc069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f58e0144ad0 tx=0x7f58d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:52.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.722+0000 7f58dd7fa640 1 -- 172.21.15.37:0/1968842268 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58d00187f0 con 0x7f58e006ec60 2024-04-25T08:55:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.914+0000 7f58e70a4640 1 -- 172.21.15.37:0/1968842268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f58e0144820 con 0x7f58e006ec60 2024-04-25T08:55:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.914+0000 7f58dd7fa640 1 -- 172.21.15.37:0/1968842268 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7f58d00f2ab0 con 0x7f58e006ec60 2024-04-25T08:55:52.907 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:55:52.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 -- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58cc066f60 msgr2=0x7f58cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58cc066f60 0x7f58cc069420 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f58e0144ad0 tx=0x7f58d4008040 comp rx=0 tx=0).stop 2024-04-25T08:55:52.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 -- 172.21.15.37:0/1968842268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e006ec60 msgr2=0x7f58e01f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:52.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e006ec60 0x7f58e01f23c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f58d00b3040 tx=0x7f58d001ba30 comp rx=0 tx=0).stop 2024-04-25T08:55:52.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 -- 172.21.15.37:0/1968842268 shutdown_connections 2024-04-25T08:55:52.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58cc066f60 0x7f58cc069420 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58e0144ca0 0x7f58e01435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58e00ce510 0x7f58e0143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 --2- 172.21.15.37:0/1968842268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58e006ec60 0x7f58e01f23c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:52.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 -- 172.21.15.37:0/1968842268 >> 172.21.15.37:0/1968842268 conn(0x7f58e00b11a0 msgr2=0x7f58e0143dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:52.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 -- 172.21.15.37:0/1968842268 shutdown_connections 2024-04-25T08:55:52.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:52.918+0000 7f58e70a4640 1 -- 172.21.15.37:0/1968842268 wait complete. 2024-04-25T08:55:52.924 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:55:52.925 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:55:52.925 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T08:55:52.925 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:55:52.925 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:52.925 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:55:52.925 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:55:52.925 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:55:52.925 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:55:52.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:52.926 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:55:52.932 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42288sec preferred_lft 42288sec 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:55:52.933 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:55:52.934 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:52.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:52.935 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:55:52.983 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:55:52.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:52.984 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T08:55:53.032 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T08:55:53.032 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:55:53.032 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:55:53.033 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:55:53.033 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:55:53.033 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:55:53.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:53.033 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:55:53.033 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:55:53.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:55:53.033 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:55:53.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:55:53.034 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:55:53.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:53.111 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:55:53.160 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:55:53.160 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:55:53.160 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:55:53.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:53.160 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T08:55:53.216 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:55:53.216 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:55:53.217 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:53.264 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:55:53.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:55:53.264 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:53.311 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:55:53.369 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:55:53.417 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:55:53.418 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:55:53.419 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:55:53.419 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:55:53.419 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:53.419 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:53.419 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:53.419 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:53.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:53.420 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:55:53.468 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:55:53.468 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:55:53.512 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:55:53.527 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:55:53.576 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:55:53.576 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:55:53.576 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:55:53.576 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:55:53.576 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:55:53.576 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:55:53.577 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:53.578 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:53.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:53.579 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:55:53.598 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T08:55:53.606+0000 7f08641b7580 -1 init, newargv = 0x55fa35f49110 newargc=15 2024-04-25T08:55:53.598 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[43020]: starting ceph client 2024-04-25T08:55:53.604 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[43020]: starting fuse 2024-04-25T08:55:54.703 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:55:54.720 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:55:54.772 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:55:54.772 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:55:54.772 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:55:54.772 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:55:54.772 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:55:54.772 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:55:54.772 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:55:54.773 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:54.774 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:54.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:54.775 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:55:54.824 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:55:54.824 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:55:54.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:55:54.824 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T08:55:54.824 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T08:55:54.824 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:55:54.824 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T08:55:54.824 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:55:54.824 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T08:55:54.825 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:55:54.826 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:55:54.826 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T08:55:54.826 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:55:54.951 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.43020.asok 2024-04-25T08:55:54.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:55:54.964 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43020.asok 2024-04-25T08:55:54.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:54.964 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43020.asok status 2024-04-25T08:55:55.124 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:55:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T08:55:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:55:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:55:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T08:55:55.124 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "43020", 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6555, 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 6555 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:55:55.125 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 211787431 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 211787431 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.6555 192.168.0.1:0/211787431", 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/211787431", 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 33, 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 69, 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:55:55.126 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T08:55:55.127 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T08:55:55.127 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:55:55.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:55.139 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T08:55:55.139 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:55.146 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T08:55:55.146 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:55.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:55.146 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:55.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:55.201 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-04-25T08:55:55.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:55.258 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-04-25T08:55:55.310 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T08:55:55.310 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:55:55.310 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T08:55:55.311 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:55:55.311 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-04-25T08:55:55.311 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:55:55.425 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627777 2024-04-25T08:55:55.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:55:55.435 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:55.600 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:55:55.603 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:55:55.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe849f0640 1 Processor -- start 2024-04-25T08:55:55.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe849f0640 1 -- start start 2024-04-25T08:55:55.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe849f0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbe800cf840 0x7fbe800cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:55.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe849f0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe8015d180 0x7fbe8015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:55.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe849f0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8006b710 con 0x7fbe8006fc50 2024-04-25T08:55:55.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe849f0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe80161560 con 0x7fbe800cf840 2024-04-25T08:55:55.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe849f0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe801617b0 con 0x7fbe8015d180 2024-04-25T08:55:55.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe7dd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbe800cf840 0x7fbe800cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:55.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe7dd9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fbe800cf840 0x7fbe800cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57252/0 (socket says 172.21.15.37:57252) 2024-04-25T08:55:55.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe7dd9b640 1 -- 172.21.15.37:0/4233754656 learned_addr learned my addr 172.21.15.37:0/4233754656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:55.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe7ed9d640 1 --2- 172.21.15.37:0/4233754656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe8015d180 0x7fbe8015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:55.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe7dd9b640 1 -- 172.21.15.37:0/4233754656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe8015d180 msgr2=0x7fbe8015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:55.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe7dd9b640 1 --2- 172.21.15.37:0/4233754656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe8015d180 0x7fbe8015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:55.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe7dd9b640 1 -- 172.21.15.37:0/4233754656 >> v1:172.21.15.37:6789/0 conn(0x7fbe8006fc50 legacy=0x7fbe8006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:55.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe7dd9b640 1 -- 172.21.15.37:0/4233754656 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe80161a90 con 0x7fbe800cf840 2024-04-25T08:55:55.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.618+0000 7fbe7d59a640 1 -- 172.21.15.37:0/4233754656 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281072514 0 0) 0x7fbe8006b710 con 0x7fbe8006fc50 2024-04-25T08:55:55.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe7dd9b640 1 --2- 172.21.15.37:0/4233754656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe800cf840 0x7fbe800cfcd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009870 tx=0x7fbe7401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ae2abcf2397195 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:55.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe7d59a640 1 -- 172.21.15.37:0/4233754656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe740b0070 con 0x7fbe800cf840 2024-04-25T08:55:55.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe7d59a640 1 -- 172.21.15.37:0/4233754656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe740ab600 con 0x7fbe800cf840 2024-04-25T08:55:55.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe7d59a640 1 -- 172.21.15.37:0/4233754656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe740ab9d0 con 0x7fbe800cf840 2024-04-25T08:55:55.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 -- 172.21.15.37:0/4233754656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe800cf840 msgr2=0x7fbe800cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:55.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 --2- 172.21.15.37:0/4233754656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe800cf840 0x7fbe800cfcd0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009870 tx=0x7fbe7401c530 comp rx=0 tx=0).stop 2024-04-25T08:55:55.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 -- 172.21.15.37:0/4233754656 shutdown_connections 2024-04-25T08:55:55.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 --2- 172.21.15.37:0/4233754656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe8015d180 0x7fbe8015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:55.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 --2- 172.21.15.37:0/4233754656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe800cf840 0x7fbe800cfcd0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:55.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 -- 172.21.15.37:0/4233754656 >> 172.21.15.37:0/4233754656 conn(0x7fbe80099a70 msgr2=0x7fbe800d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:55.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 -- 172.21.15.37:0/4233754656 shutdown_connections 2024-04-25T08:55:55.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 -- 172.21.15.37:0/4233754656 wait complete. 2024-04-25T08:55:55.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 Processor -- start 2024-04-25T08:55:55.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 -- start start 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe8006fc50 0x7fbe8014f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe800cf840 0x7fbe8015e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe8015d180 0x7fbe800b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80070030 con 0x7fbe8006fc50 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800d0e30 con 0x7fbe800cf840 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.622+0000 7fbe849f0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800701b0 con 0x7fbe8015d180 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe8006fc50 0x7fbe8014f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe8006fc50 0x7fbe8014f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57262/0 (socket says 172.21.15.37:57262) 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7e59c640 1 -- 172.21.15.37:0/3976961288 learned_addr learned my addr 172.21.15.37:0/3976961288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7ed9d640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe8015d180 0x7fbe800b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7dd9b640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe800cf840 0x7fbe8015e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7e59c640 1 -- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe8015d180 msgr2=0x7fbe800b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7e59c640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe8015d180 0x7fbe800b3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:55.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7e59c640 1 -- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe800cf840 msgr2=0x7fbe8015e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:55.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7e59c640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe800cf840 0x7fbe8015e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:55.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7e59c640 1 -- 172.21.15.37:0/3976961288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe800b3120 con 0x7fbe8006fc50 2024-04-25T08:55:55.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7e59c640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe8006fc50 0x7fbe8014f5c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbe70001ec0 tx=0x7fbe7000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:55.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe6effd640 1 -- 172.21.15.37:0/3976961288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70017070 con 0x7fbe8006fc50 2024-04-25T08:55:55.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe6effd640 1 -- 172.21.15.37:0/3976961288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70003600 con 0x7fbe8006fc50 2024-04-25T08:55:55.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe849f0640 1 -- 172.21.15.37:0/3976961288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe800b33c0 con 0x7fbe8006fc50 2024-04-25T08:55:55.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe6effd640 1 -- 172.21.15.37:0/3976961288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe700039f0 con 0x7fbe8006fc50 2024-04-25T08:55:55.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe849f0640 1 -- 172.21.15.37:0/3976961288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe80150d80 con 0x7fbe8006fc50 2024-04-25T08:55:55.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe849f0640 1 -- 172.21.15.37:0/3976961288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe800b38e0 con 0x7fbe8006fc50 2024-04-25T08:55:55.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe6effd640 1 -- 172.21.15.37:0/3976961288 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe7001a070 con 0x7fbe8006fc50 2024-04-25T08:55:55.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe6effd640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe60066f60 0x7fbe60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:55.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe6effd640 1 -- 172.21.15.37:0/3976961288 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbe70084530 con 0x7fbe8006fc50 2024-04-25T08:55:55.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.626+0000 7fbe7dd9b640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe60066f60 0x7fbe60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:55.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.630+0000 7fbe7dd9b640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe60066f60 0x7fbe60069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbe740ad000 tx=0x7fbe74008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.630+0000 7fbe6effd640 1 -- 172.21.15.37:0/3976961288 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbe800b38e0 con 0x7fbe8006fc50 2024-04-25T08:55:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.826+0000 7fbe849f0640 1 -- 172.21.15.37:0/3976961288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe8004efa0 con 0x7fbe8006fc50 2024-04-25T08:55:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.826+0000 7fbe6effd640 1 -- 172.21.15.37:0/3976961288 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fbe70054d50 con 0x7fbe8006fc50 2024-04-25T08:55:55.817 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:55.818 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/486740526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":486740526},{"type":"v1","addr":"172.21.15.37:6833","nonce":486740526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/121484331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":121484331},{"type":"v1","addr":"172.21.15.203:6835","nonce":121484331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2501630566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2501630566},{"type":"v1","addr":"172.21.15.203:6837","nonce":2501630566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:55:46.200939+0000","modified":"2024-04-25T08:55:49.230920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6405":{"gid":6405,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/2189232625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2189232625},{"type":"v1","addr":"172.21.15.37:6835","nonce":2189232625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:55:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-04-25T08:55:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 -- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe60066f60 msgr2=0x7fbe60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe60066f60 0x7fbe60069420 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbe740ad000 tx=0x7fbe74008040 comp rx=0 tx=0).stop 2024-04-25T08:55:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 -- 172.21.15.37:0/3976961288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe8006fc50 msgr2=0x7fbe8014f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe8006fc50 0x7fbe8014f5c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbe70001ec0 tx=0x7fbe7000c8e0 comp rx=0 tx=0).stop 2024-04-25T08:55:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 -- 172.21.15.37:0/3976961288 shutdown_connections 2024-04-25T08:55:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe60066f60 0x7fbe60069420 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe8015d180 0x7fbe800b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe800cf840 0x7fbe8015e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 --2- 172.21.15.37:0/3976961288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe8006fc50 0x7fbe8014f5c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 -- 172.21.15.37:0/3976961288 >> 172.21.15.37:0/3976961288 conn(0x7fbe80099a70 msgr2=0x7fbe8006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 -- 172.21.15.37:0/3976961288 shutdown_connections 2024-04-25T08:55:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:55.830+0000 7fbe849f0640 1 -- 172.21.15.37:0/3976961288 wait complete. 2024-04-25T08:55:55.837 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T08:55:55.997 INFO:teuthology.orchestra.run.smithi037.stdout:{"message":"","return_code":0} 2024-04-25T08:55:55.998 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:55:55.998 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T08:55:56.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4683ca3080 1 Processor -- start 2024-04-25T08:55:56.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4683ca3080 1 -- start start 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4683ca3080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575d2145570 0x5575d21751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4683ca3080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575d2193360 0x5575d22149c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4683ca3080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 0x5575d2214f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4683ca3080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d2179800 con 0x5575d221d1f0 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4683ca3080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d20ec9f0 con 0x5575d2145570 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4683ca3080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575d2179980 con 0x5575d2193360 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4681fc2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 0x5575d2214f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4681fc2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 0x5575d2214f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57282/0 (socket says 172.21.15.37:57282) 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4681fc2640 1 -- 172.21.15.37:0/522376467 learned_addr learned my addr 172.21.15.37:0/522376467 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4680fc0640 1 --2- 172.21.15.37:0/522376467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575d2193360 0x5575d22149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4681fc2640 1 -- 172.21.15.37:0/522376467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575d2193360 msgr2=0x5575d22149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4681fc2640 1 --2- 172.21.15.37:0/522376467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575d2193360 0x5575d22149c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4681fc2640 1 -- 172.21.15.37:0/522376467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575d2145570 msgr2=0x5575d21751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4681fc2640 1 --2- 172.21.15.37:0/522376467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575d2145570 0x5575d21751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.046+0000 7f4681fc2640 1 -- 172.21.15.37:0/522376467 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575d221a0e0 con 0x5575d221d1f0 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4681fc2640 1 --2- 172.21.15.37:0/522376467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 0x5575d2214f00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f467c00b2a0 tx=0x7f467c00b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9418d8c189ab73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f467b7fe640 1 -- 172.21.15.37:0/522376467 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c00d510 con 0x5575d221d1f0 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f467b7fe640 1 -- 172.21.15.37:0/522376467 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c008070 con 0x5575d221d1f0 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f467b7fe640 1 -- 172.21.15.37:0/522376467 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c00d8b0 con 0x5575d221d1f0 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 -- 172.21.15.37:0/522376467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 msgr2=0x5575d2214f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 --2- 172.21.15.37:0/522376467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 0x5575d2214f00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f467c00b2a0 tx=0x7f467c00b760 comp rx=0 tx=0).stop 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 -- 172.21.15.37:0/522376467 shutdown_connections 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 --2- 172.21.15.37:0/522376467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 0x5575d2214f00 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 --2- 172.21.15.37:0/522376467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575d2193360 0x5575d22149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 --2- 172.21.15.37:0/522376467 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575d2145570 0x5575d21751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 -- 172.21.15.37:0/522376467 >> 172.21.15.37:0/522376467 conn(0x5575d214bc90 msgr2=0x5575d2177650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 -- 172.21.15.37:0/522376467 shutdown_connections 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 -- 172.21.15.37:0/522376467 wait complete. 2024-04-25T08:55:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 Processor -- start 2024-04-25T08:55:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 -- start start 2024-04-25T08:55:56.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575d2145570 0x5575d220da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575d2193360 0x5575d2210530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 0x5575d2210a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d221b330 con 0x5575d221d1f0 2024-04-25T08:55:56.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575d221a380 con 0x5575d2193360 2024-04-25T08:55:56.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575d2219dd0 con 0x5575d2145570 2024-04-25T08:55:56.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4681fc2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 0x5575d2210a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4681fc2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 0x5575d2210a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57298/0 (socket says 172.21.15.37:57298) 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4681fc2640 1 -- 172.21.15.37:0/778354513 learned_addr learned my addr 172.21.15.37:0/778354513 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4680fc0640 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575d2193360 0x5575d2210530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f46817c1640 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575d2145570 0x5575d220da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4681fc2640 1 -- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575d2145570 msgr2=0x5575d220da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4681fc2640 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5575d2145570 0x5575d220da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4681fc2640 1 -- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575d2193360 msgr2=0x5575d2210530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4681fc2640 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5575d2193360 0x5575d2210530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4681fc2640 1 -- 172.21.15.37:0/778354513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575d22b83c0 con 0x5575d221d1f0 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4681fc2640 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 0x5575d2210a70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x5575d21781c0 tx=0x7f467c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f467a7fc640 1 -- 172.21.15.37:0/778354513 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c01c650 con 0x5575d221d1f0 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f467a7fc640 1 -- 172.21.15.37:0/778354513 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467c021020 con 0x5575d221d1f0 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 -- 172.21.15.37:0/778354513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5575d22b8660 con 0x5575d221d1f0 2024-04-25T08:55:56.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f467a7fc640 1 -- 172.21.15.37:0/778354513 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c01c830 con 0x5575d221d1f0 2024-04-25T08:55:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.050+0000 7f4683ca3080 1 -- 172.21.15.37:0/778354513 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575d22b8ba0 con 0x5575d221d1f0 2024-04-25T08:55:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f467a7fc640 1 -- 172.21.15.37:0/778354513 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f467c017070 con 0x5575d221d1f0 2024-04-25T08:55:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f467a7fc640 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4664065970 0x7f4664067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f467a7fc640 1 -- 172.21.15.37:0/778354513 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f467c085ec0 con 0x5575d221d1f0 2024-04-25T08:55:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5575d2214590 0x5575d2214970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 -- 172.21.15.37:0/778354513 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e69) v9 -- 0x5575d2214eb0 con 0x5575d2214590 2024-04-25T08:55:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f46817c1640 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4664065970 0x7f4664067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4680fc0640 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5575d2214590 0x5575d2214970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f46817c1640 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4664065970 0x7f4664067e30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f46740ac000 tx=0x7f46740033b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4680fc0640 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5575d2214590 0x5575d2214970 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4680fc0640 1 -- 172.21.15.37:0/778354513 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f466c009820 con 0x5575d2214590 2024-04-25T08:55:56.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 -- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5575d2214590 msgr2=0x5575d2214970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5575d2214590 0x5575d2214970 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 -- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4664065970 msgr2=0x7f4664067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4664065970 0x7f4664067e30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f46740ac000 tx=0x7f46740033b0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 -- 172.21.15.37:0/778354513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 msgr2=0x5575d2210a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 --2- 172.21.15.37:0/778354513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5575d221d1f0 0x5575d2210a70 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x5575d21781c0 tx=0x7f467c000ee0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4681fc2640 1 -- 172.21.15.37:0/778354513 reap_dead start 2024-04-25T08:55:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 -- 172.21.15.37:0/778354513 shutdown_connections 2024-04-25T08:55:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 -- 172.21.15.37:0/778354513 >> 172.21.15.37:0/778354513 conn(0x5575d214bc90 msgr2=0x5575d2211860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 -- 172.21.15.37:0/778354513 shutdown_connections 2024-04-25T08:55:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.054+0000 7f4683ca3080 1 -- 172.21.15.37:0/778354513 wait complete. 2024-04-25T08:55:56.049 INFO:teuthology.orchestra.run.smithi037.stdout:Xfiledir_test 2024-04-25T08:55:56.049 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:56.049 DEBUG:teuthology.orchestra.run.smithi037:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T08:55:56.135 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:55:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627777, 2024-04-25T08:55:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "ancestors": [ 2024-04-25T08:55:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "dirino": 1099511627776, 2024-04-25T08:55:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "dname": "file", 2024-04-25T08:55:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 2 2024-04-25T08:55:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:55:56.135 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: "dirino": 1, 2024-04-25T08:55:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: "dname": "dir_test", 2024-04-25T08:55:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 4 2024-04-25T08:55:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T08:55:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: "pool": 10, 2024-04-25T08:55:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: "old_pools": [] 2024-04-25T08:55:56.136 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:55:56.136 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:56.138 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-04-25T08:55:56.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 Processor -- start 2024-04-25T08:55:56.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 -- start start 2024-04-25T08:55:56.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559f2a5bc570 0x559f2a6917a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a60a360 0x559f2a691ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f2a6941f0 0x559f2a5eaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2a5639f0 con 0x559f2a60a360 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2a567950 con 0x559f2a6941f0 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f2a5ef660 con 0x559f2a5bc570 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa279232640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a60a360 0x559f2a691ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa279232640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a60a360 0x559f2a691ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57302/0 (socket says 172.21.15.37:57302) 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa279232640 1 -- 172.21.15.37:0/2543048604 learned_addr learned my addr 172.21.15.37:0/2543048604 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa279a33640 1 --2- 172.21.15.37:0/2543048604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559f2a5bc570 0x559f2a6917a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27a234640 1 --2- 172.21.15.37:0/2543048604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f2a6941f0 0x559f2a5eaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa279232640 1 -- 172.21.15.37:0/2543048604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559f2a5bc570 msgr2=0x559f2a6917a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa279232640 1 --2- 172.21.15.37:0/2543048604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559f2a5bc570 0x559f2a6917a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa279232640 1 -- 172.21.15.37:0/2543048604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f2a6941f0 msgr2=0x559f2a5eaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa279232640 1 --2- 172.21.15.37:0/2543048604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f2a6941f0 0x559f2a5eaa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa279232640 1 -- 172.21.15.37:0/2543048604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f2a5ee4d0 con 0x559f2a60a360 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa279a33640 1 --2- 172.21.15.37:0/2543048604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559f2a5bc570 0x559f2a6917a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa279232640 1 --2- 172.21.15.37:0/2543048604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a60a360 0x559f2a691ce0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa26400b2d0 tx=0x7fa26400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96b711533ecba62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa273fff640 1 -- 172.21.15.37:0/2543048604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26400a040 con 0x559f2a60a360 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa273fff640 1 -- 172.21.15.37:0/2543048604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26400f620 con 0x559f2a60a360 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa273fff640 1 -- 172.21.15.37:0/2543048604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26400fa10 con 0x559f2a60a360 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 -- 172.21.15.37:0/2543048604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a60a360 msgr2=0x559f2a691ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 --2- 172.21.15.37:0/2543048604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a60a360 0x559f2a691ce0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa26400b2d0 tx=0x7fa26400b790 comp rx=0 tx=0).stop 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 -- 172.21.15.37:0/2543048604 shutdown_connections 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 --2- 172.21.15.37:0/2543048604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f2a6941f0 0x559f2a5eaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 --2- 172.21.15.37:0/2543048604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a60a360 0x559f2a691ce0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 --2- 172.21.15.37:0/2543048604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559f2a5bc570 0x559f2a6917a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 -- 172.21.15.37:0/2543048604 >> 172.21.15.37:0/2543048604 conn(0x559f2a5c2c90 msgr2=0x559f2a5eddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 -- 172.21.15.37:0/2543048604 shutdown_connections 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.230+0000 7fa27bf15080 1 -- 172.21.15.37:0/2543048604 wait complete. 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa27bf15080 1 Processor -- start 2024-04-25T08:55:56.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa27bf15080 1 -- start start 2024-04-25T08:55:56.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa27bf15080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a5bc570 0x559f2a69e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa27bf15080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f2a60a360 0x559f2a6904b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa27bf15080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559f2a6941f0 0x559f2a691650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa27bf15080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2a5ebc10 con 0x559f2a5bc570 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa27bf15080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2a5ec7e0 con 0x559f2a60a360 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa27bf15080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f2a600950 con 0x559f2a6941f0 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa279a33640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a5bc570 0x559f2a69e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa279a33640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a5bc570 0x559f2a69e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57310/0 (socket says 172.21.15.37:57310) 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa279a33640 1 -- 172.21.15.37:0/439321369 learned_addr learned my addr 172.21.15.37:0/439321369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa279232640 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f2a60a360 0x559f2a6904b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa27a234640 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559f2a6941f0 0x559f2a691650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa279a33640 1 -- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559f2a6941f0 msgr2=0x559f2a691650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa279a33640 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559f2a6941f0 0x559f2a691650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa279a33640 1 -- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f2a60a360 msgr2=0x559f2a6904b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa279a33640 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559f2a60a360 0x559f2a6904b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa279a33640 1 -- 172.21.15.37:0/439321369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f2a730f70 con 0x559f2a5bc570 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa279a33640 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a5bc570 0x559f2a69e360 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa274018000 tx=0x7fa274018190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa272ffd640 1 -- 172.21.15.37:0/439321369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2740bf470 con 0x559f2a5bc570 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa272ffd640 1 -- 172.21.15.37:0/439321369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2740ba070 con 0x559f2a5bc570 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa27bf15080 1 -- 172.21.15.37:0/439321369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559f2a5c1fc0 con 0x559f2a5bc570 2024-04-25T08:55:56.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa272ffd640 1 -- 172.21.15.37:0/439321369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2740bf830 con 0x559f2a5bc570 2024-04-25T08:55:56.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa27bf15080 1 -- 172.21.15.37:0/439321369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f2a5c2500 con 0x559f2a5bc570 2024-04-25T08:55:56.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.234+0000 7fa272ffd640 1 -- 172.21.15.37:0/439321369 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2740bf9d0 con 0x559f2a5bc570 2024-04-25T08:55:56.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa272ffd640 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa25c065970 0x7fa25c067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa272ffd640 1 -- 172.21.15.37:0/439321369 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa27411fe50 con 0x559f2a5bc570 2024-04-25T08:55:56.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27bf15080 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x559f2a6909f0 0x559f2a73da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27bf15080 1 -- 172.21.15.37:0/439321369 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:1 10.7 10:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) v9 -- 0x559f2a5c1af0 con 0x559f2a6909f0 2024-04-25T08:55:56.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa279232640 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa25c065970 0x7fa25c067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27a234640 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x559f2a6909f0 0x559f2a73da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa279232640 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa25c065970 0x7fa25c067e30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa26400b7d0 tx=0x7fa264009000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27a234640 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x559f2a6909f0 0x559f2a73da40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27a234640 1 -- 172.21.15.37:0/439321369 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v69'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa26c001810 con 0x559f2a6909f0 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27bf15080 1 -- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x559f2a6909f0 msgr2=0x559f2a73da40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27bf15080 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x559f2a6909f0 0x559f2a73da40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27bf15080 1 -- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa25c065970 msgr2=0x7fa25c067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27bf15080 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa25c065970 0x7fa25c067e30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa26400b7d0 tx=0x7fa264009000 comp rx=0 tx=0).stop 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27bf15080 1 -- 172.21.15.37:0/439321369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a5bc570 msgr2=0x559f2a69e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27bf15080 1 --2- 172.21.15.37:0/439321369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f2a5bc570 0x559f2a69e360 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa274018000 tx=0x7fa274018190 comp rx=0 tx=0).stop 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27a234640 1 -- 172.21.15.37:0/439321369 reap_dead start 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27bf15080 1 -- 172.21.15.37:0/439321369 shutdown_connections 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27bf15080 1 -- 172.21.15.37:0/439321369 >> 172.21.15.37:0/439321369 conn(0x559f2a5c2c90 msgr2=0x559f2a5eddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.238+0000 7fa27bf15080 1 -- 172.21.15.37:0/439321369 shutdown_connections 2024-04-25T08:55:56.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.242+0000 7fa27bf15080 1 -- 172.21.15.37:0/439321369 wait complete. 2024-04-25T08:55:56.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:56.234 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-04-25T08:55:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e0e67f640 1 Processor -- start 2024-04-25T08:55:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e0e67f640 1 -- start start 2024-04-25T08:55:56.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e0e67f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8e0815a3c0 0x7f8e080ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e0e67f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8e0815a7a0 0x7f8e080b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e0e67f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e0815e0f0 con 0x7f8e0815ac30 2024-04-25T08:55:56.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e0e67f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e080b24a0 con 0x7f8e0815a7a0 2024-04-25T08:55:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e0e67f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0815b0b0 con 0x7f8e0815a3c0 2024-04-25T08:55:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e077fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8e0815a7a0 0x7f8e080b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e077fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8e0815a7a0 0x7f8e080b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57324/0 (socket says 172.21.15.37:57324) 2024-04-25T08:55:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e0cc1c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8e0815ac30 0x7f8e080cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41996/0 (socket says 172.21.15.37:41996) 2024-04-25T08:55:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e077fe640 1 -- 172.21.15.37:0/3347300179 learned_addr learned my addr 172.21.15.37:0/3347300179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e07fff640 1 --2- 172.21.15.37:0/3347300179 >> v2:172.21.15.203:3300/0 conn(0x7f8e0815a3c0 0x7f8e080ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e077fe640 1 -- 172.21.15.37:0/3347300179 >> v2:172.21.15.203:3300/0 conn(0x7f8e0815a3c0 msgr2=0x7f8e080ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e077fe640 1 --2- 172.21.15.37:0/3347300179 >> v2:172.21.15.203:3300/0 conn(0x7f8e0815a3c0 0x7f8e080ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e077fe640 1 -- 172.21.15.37:0/3347300179 >> v1:172.21.15.37:6789/0 conn(0x7f8e0815ac30 legacy=0x7f8e080cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.410+0000 7f8e077fe640 1 -- 172.21.15.37:0/3347300179 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0815b390 con 0x7f8e0815a7a0 2024-04-25T08:55:56.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e077fe640 1 --2- 172.21.15.37:0/3347300179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e0815a7a0 0x7f8e080b27a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8e080cbcc0 tx=0x7f8dfc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc293b35fd3b9466 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e06ffd640 1 -- 172.21.15.37:0/3347300179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0aa670 con 0x7f8e0815a7a0 2024-04-25T08:55:56.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e06ffd640 1 -- 172.21.15.37:0/3347300179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc0af070 con 0x7f8e0815a7a0 2024-04-25T08:55:56.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e06ffd640 1 -- 172.21.15.37:0/3347300179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0aaa30 con 0x7f8e0815a7a0 2024-04-25T08:55:56.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e0e67f640 1 -- 172.21.15.37:0/3347300179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e0815a7a0 msgr2=0x7f8e080b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e0e67f640 1 --2- 172.21.15.37:0/3347300179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e0815a7a0 0x7f8e080b27a0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8e080cbcc0 tx=0x7f8dfc01c4f0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e0e67f640 1 -- 172.21.15.37:0/3347300179 shutdown_connections 2024-04-25T08:55:56.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e0e67f640 1 --2- 172.21.15.37:0/3347300179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e0815a7a0 0x7f8e080b27a0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e0e67f640 1 --2- 172.21.15.37:0/3347300179 >> v2:172.21.15.203:3300/0 conn(0x7f8e0815a3c0 0x7f8e080ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e0e67f640 1 -- 172.21.15.37:0/3347300179 >> 172.21.15.37:0/3347300179 conn(0x7f8e0806ae60 msgr2=0x7f8e0806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e0e67f640 1 -- 172.21.15.37:0/3347300179 shutdown_connections 2024-04-25T08:55:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.414+0000 7f8e0e67f640 1 -- 172.21.15.37:0/3347300179 wait complete. 2024-04-25T08:55:56.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0e67f640 1 Processor -- start 2024-04-25T08:55:56.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0e67f640 1 -- start start 2024-04-25T08:55:56.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0e67f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e0815a3c0 0x7f8e08099e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0e67f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e0815a7a0 0x7f8e080afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0e67f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e0815ac30 0x7f8e080c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0e67f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0815edb0 con 0x7f8e0815a3c0 2024-04-25T08:55:56.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0e67f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e080b2ef0 con 0x7f8e0815ac30 2024-04-25T08:55:56.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0e67f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0815ef30 con 0x7f8e0815a7a0 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e0815a3c0 0x7f8e08099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e0815a3c0 0x7f8e08099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57328/0 (socket says 172.21.15.37:57328) 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e07fff640 1 -- 172.21.15.37:0/4182332078 learned_addr learned my addr 172.21.15.37:0/4182332078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e0815ac30 0x7f8e080c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e077fe640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e0815a7a0 0x7f8e080afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e07fff640 1 -- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e0815a7a0 msgr2=0x7f8e080afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e07fff640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e0815a7a0 0x7f8e080afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e07fff640 1 -- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e0815ac30 msgr2=0x7f8e080c3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e07fff640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e0815ac30 0x7f8e080c3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e07fff640 1 -- 172.21.15.37:0/4182332078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e080ac210 con 0x7f8e0815a3c0 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e0815ac30 0x7f8e080c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:56.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e077fe640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e0815a7a0 0x7f8e080afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:56.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e07fff640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e0815a3c0 0x7f8e08099e70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8df800d830 tx=0x7f8df800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e04ff9640 1 -- 172.21.15.37:0/4182332078 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df800b6c0 con 0x7f8e0815a3c0 2024-04-25T08:55:56.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e04ff9640 1 -- 172.21.15.37:0/4182332078 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df8019070 con 0x7f8e0815a3c0 2024-04-25T08:55:56.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0e67f640 1 -- 172.21.15.37:0/4182332078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e080ac4f0 con 0x7f8e0815a3c0 2024-04-25T08:55:56.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e04ff9640 1 -- 172.21.15.37:0/4182332078 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df800ba60 con 0x7f8e0815a3c0 2024-04-25T08:55:56.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e0e67f640 1 -- 172.21.15.37:0/4182332078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e080acab0 con 0x7f8e0815a3c0 2024-04-25T08:55:56.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.418+0000 7f8e04ff9640 1 -- 172.21.15.37:0/4182332078 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8df801e460 con 0x7f8e0815a3c0 2024-04-25T08:55:56.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e04ff9640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8df4066f60 0x7f8df4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e04ff9640 1 -- 172.21.15.37:0/4182332078 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8df8085900 con 0x7f8e0815a3c0 2024-04-25T08:55:56.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e077fe640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8df4066f60 0x7f8df4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 Processor -- start 2024-04-25T08:55:56.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 -- start start 2024-04-25T08:55:56.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e077fe640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8df4066f60 0x7f8df4069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0ae000 tx=0x7f8dfc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f61b0 0x55603a8f6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f6ad0 0x55603a8da9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a8db2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603a6f9fd0 con 0x55603a8f61b0 2024-04-25T08:55:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603a682170 con 0x55603a8f6ad0 2024-04-25T08:55:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55603a6f6070 con 0x55603a8daf10 2024-04-25T08:55:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f61b0 0x55603a8f6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f61b0 0x55603a8f6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57342/0 (socket says 172.21.15.37:57342) 2024-04-25T08:55:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e07fff640 1 -- 172.21.15.37:0/1390840598 learned_addr learned my addr 172.21.15.37:0/1390840598 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e077fe640 1 --2- 172.21.15.37:0/1390840598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f6ad0 0x55603a8da9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/1390840598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a8db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e07fff640 1 -- 172.21.15.37:0/1390840598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 msgr2=0x55603a8db2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e07fff640 1 --2- 172.21.15.37:0/1390840598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a8db2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e07fff640 1 -- 172.21.15.37:0/1390840598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f6ad0 msgr2=0x55603a8da9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e07fff640 1 --2- 172.21.15.37:0/1390840598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f6ad0 0x55603a8da9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e07fff640 1 -- 172.21.15.37:0/1390840598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55603a740d20 con 0x55603a8f61b0 2024-04-25T08:55:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e07fff640 1 --2- 172.21.15.37:0/1390840598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f61b0 0x55603a8f6590 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8df80524e0 tx=0x7f8df80568c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409aed48f435f952 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8dedffb640 1 -- 172.21.15.37:0/1390840598 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df805e790 con 0x55603a8f61b0 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8dedffb640 1 -- 172.21.15.37:0/1390840598 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df800be10 con 0x55603a8f61b0 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8dedffb640 1 -- 172.21.15.37:0/1390840598 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df80526f0 con 0x55603a8f61b0 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 -- 172.21.15.37:0/1390840598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f61b0 msgr2=0x55603a8f6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 --2- 172.21.15.37:0/1390840598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f61b0 0x55603a8f6590 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8df80524e0 tx=0x7f8df80568c0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 -- 172.21.15.37:0/1390840598 shutdown_connections 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 --2- 172.21.15.37:0/1390840598 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a8db2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 --2- 172.21.15.37:0/1390840598 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f6ad0 0x55603a8da9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 --2- 172.21.15.37:0/1390840598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f61b0 0x55603a8f6590 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 -- 172.21.15.37:0/1390840598 >> 172.21.15.37:0/1390840598 conn(0x55603a8b2710 msgr2=0x55603a8b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 -- 172.21.15.37:0/1390840598 shutdown_connections 2024-04-25T08:55:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.422+0000 7f8e117ec000 1 -- 172.21.15.37:0/1390840598 wait complete. 2024-04-25T08:55:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- start start 2024-04-25T08:55:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 Processor -- start 2024-04-25T08:55:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8daf10 0x55603a8fd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8f61b0 0x55603a8fdc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8db2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603a7b7c10 con 0x55603a8f6ad0 2024-04-25T08:55:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603a6b4680 con 0x55603a8daf10 2024-04-25T08:55:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55603a6fe600 con 0x55603a8f61b0 2024-04-25T08:55:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 10 client.0 _ll_register_callbacks cb 0x55603a7af080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e0cc1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e0cc1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8db2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57358/0 (socket says 172.21.15.37:57358) 2024-04-25T08:55:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/4095051001 learned_addr learned my addr 172.21.15.37:0/4095051001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e077fe640 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8f61b0 0x55603a8fdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e07fff640 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8daf10 0x55603a8fd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/4095051001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8f61b0 msgr2=0x55603a8fdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8f61b0 0x55603a8fdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/4095051001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8daf10 msgr2=0x55603a8fd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8daf10 0x55603a8fd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/4095051001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55603a6d1d30 con 0x55603a8f6ad0 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e07fff640 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8daf10 0x55603a8fd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8db2f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000e4d0 tx=0x7f8e0000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61d140125a7a6fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8de6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8de6ffd640 1 -- 172.21.15.37:0/4095051001 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e000070d0 con 0x55603a8f6ad0 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8de6ffd640 1 -- 172.21.15.37:0/4095051001 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e00007270 con 0x55603a8f6ad0 2024-04-25T08:55:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- 172.21.15.37:0/4095051001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8df800be10 con 0x55603a8f6ad0 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8de6ffd640 1 -- 172.21.15.37:0/4095051001 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e0001f050 con 0x55603a8f6ad0 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 10 client.6600 fetch_fsmap learned FSMap version 33 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- 172.21.15.37:0/4095051001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55603a6823c0 con 0x55603a8f6ad0 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8de6ffd640 1 -- 172.21.15.37:0/4095051001 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df800be10 con 0x55603a8f6ad0 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8de6ffd640 1 -- 172.21.15.37:0/4095051001 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e00022020 con 0x55603a8f6ad0 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 10 client.6600 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 10 client.6600 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 20 client.6600 populate_metadata read hostname 'smithi037' 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55603a8dbd00 0x55603a8dc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 4 client.6600 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- 172.21.15.37:0/4095051001 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55603a695e00 con 0x55603a8dbd00 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e07fff640 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55603a8dbd00 0x55603a8dc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8de6ffd640 10 client.6600 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e07fff640 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55603a8dbd00 0x55603a8dc0e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5faa74dbe13bc58 server_cookie=f51a5473d2d79315 in_seq=0 out_seq=0 2024-04-25T08:55:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8de6ffd640 1 -- 172.21.15.37:0/4095051001 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55603a695e00 con 0x55603a8dbd00 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8de6ffd640 10 client.6600 handle_command_reply: tid=0 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 client.6600 shutdown 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- 172.21.15.37:0/4095051001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 msgr2=0x55603a8db2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8db2f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000e4d0 tx=0x7f8e0000e990 comp rx=0 tx=0).stop 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- 172.21.15.37:0/4095051001 shutdown_connections 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55603a8dbd00 0x55603a8dc0e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8db2f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8f61b0 0x55603a8fdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8de6ffd640 0 client.6600 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 --2- 172.21.15.37:0/4095051001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8daf10 0x55603a8fd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- 172.21.15.37:0/4095051001 >> 172.21.15.37:0/4095051001 conn(0x55603a8b2710 msgr2=0x55603a8aa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- 172.21.15.37:0/4095051001 shutdown_connections 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 1 -- 172.21.15.37:0/4095051001 wait complete. 2024-04-25T08:55:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.426+0000 7f8e117ec000 20 client.6600 trim_cache size 0 max 16384 2024-04-25T08:55:56.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.442+0000 7f8e117ec000 1 Processor -- start 2024-04-25T08:55:56.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.442+0000 7f8e117ec000 1 -- start start 2024-04-25T08:55:56.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a8aa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f61b0 0x55603a8aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8aab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e08200d70 con 0x55603a8f6ad0 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e081fffa0 con 0x55603a8f61b0 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e080ad8e0 con 0x55603a8daf10 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8aab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57374/0 (socket says 172.21.15.37:57374) 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/4280529875 learned_addr learned my addr 172.21.15.37:0/4280529875 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e077fe640 1 --2- 172.21.15.37:0/4280529875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f61b0 0x55603a8aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e07fff640 1 --2- 172.21.15.37:0/4280529875 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a8aa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/4280529875 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 msgr2=0x55603a8aa0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/4280529875 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a8aa0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/4280529875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f61b0 msgr2=0x55603a8aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/4280529875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f61b0 0x55603a8aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/4280529875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55603a8db660 con 0x55603a8f6ad0 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e07fff640 1 --2- 172.21.15.37:0/4280529875 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a8aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/4280529875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8aab40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8e000013f0 tx=0x7f8e0000c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6a1ce1cf761734 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8ded7fa640 1 -- 172.21.15.37:0/4280529875 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e000070d0 con 0x55603a8f6ad0 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8ded7fa640 1 -- 172.21.15.37:0/4280529875 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e00027030 con 0x55603a8f6ad0 2024-04-25T08:55:56.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8ded7fa640 1 -- 172.21.15.37:0/4280529875 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00013640 con 0x55603a8f6ad0 2024-04-25T08:55:56.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- 172.21.15.37:0/4280529875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 msgr2=0x55603a8aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 --2- 172.21.15.37:0/4280529875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8aab40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f8e000013f0 tx=0x7f8e0000c000 comp rx=0 tx=0).stop 2024-04-25T08:55:56.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- 172.21.15.37:0/4280529875 shutdown_connections 2024-04-25T08:55:56.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 --2- 172.21.15.37:0/4280529875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a8aab40 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 --2- 172.21.15.37:0/4280529875 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f61b0 0x55603a8aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 --2- 172.21.15.37:0/4280529875 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a8aa0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- 172.21.15.37:0/4280529875 >> 172.21.15.37:0/4280529875 conn(0x55603a8b2710 msgr2=0x55603a8da5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- 172.21.15.37:0/4280529875 shutdown_connections 2024-04-25T08:55:56.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- 172.21.15.37:0/4280529875 wait complete. 2024-04-25T08:55:56.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- start start 2024-04-25T08:55:56.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 Processor -- start 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a956d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f61b0 0x55603a9572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a911fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55603a6a0d40 con 0x55603a8f6ad0 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e080adee0 con 0x55603a8f61b0 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55603a7b4eb0 con 0x55603a8daf10 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 10 client.0 _ll_register_callbacks cb 0x55603a697b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a911fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a911fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57390/0 (socket says 172.21.15.37:57390) 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/3521298175 learned_addr learned my addr 172.21.15.37:0/3521298175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e077fe640 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f61b0 0x55603a9572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e07fff640 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a956d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/3521298175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 msgr2=0x55603a956d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a956d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/3521298175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f61b0 msgr2=0x55603a9572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f61b0 0x55603a9572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 -- 172.21.15.37:0/3521298175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55603a77c0f0 con 0x55603a8f6ad0 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e07fff640 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a956d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e0cc1c640 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a911fe0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8e00007db0 tx=0x7f8e00001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172e34c7d85df33e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8de6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8de6ffd640 1 -- 172.21.15.37:0/3521298175 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00013640 con 0x55603a8f6ad0 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8de6ffd640 1 -- 172.21.15.37:0/3521298175 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8e000137e0 con 0x55603a8f6ad0 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8e117ec000 1 -- 172.21.15.37:0/3521298175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e00027030 con 0x55603a8f6ad0 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.446+0000 7f8de6ffd640 1 -- 172.21.15.37:0/3521298175 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e0003c050 con 0x55603a8f6ad0 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 10 client.6606 fetch_fsmap learned FSMap version 33 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 -- 172.21.15.37:0/3521298175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55603a68ed20 con 0x55603a8f6ad0 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8de6ffd640 1 -- 172.21.15.37:0/3521298175 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e00027030 con 0x55603a8f6ad0 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8de6ffd640 1 -- 172.21.15.37:0/3521298175 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8e0003f020 con 0x55603a8f6ad0 2024-04-25T08:55:56.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 10 client.6606 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 10 client.6606 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 20 client.6606 populate_metadata read hostname 'smithi037' 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55603a912970 0x55603a912d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 4 client.6606 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 -- 172.21.15.37:0/3521298175 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x55603a707510 con 0x55603a912970 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e07fff640 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55603a912970 0x55603a912d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8de6ffd640 10 client.6606 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e07fff640 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55603a912970 0x55603a912d80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=428f400e8b313a90 server_cookie=202132f8c6ae8996 in_seq=0 out_seq=0 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8de6ffd640 1 -- 172.21.15.37:0/3521298175 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55603a707510 con 0x55603a912970 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8de6ffd640 10 client.6606 handle_command_reply: tid=0 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 client.6606 shutdown 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 -- 172.21.15.37:0/3521298175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 msgr2=0x55603a911fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a911fe0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f8e00007db0 tx=0x7f8e00001260 comp rx=0 tx=0).stop 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 -- 172.21.15.37:0/3521298175 shutdown_connections 2024-04-25T08:55:56.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55603a912970 0x55603a912d80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55603a8f6ad0 0x55603a911fe0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55603a8f61b0 0x55603a9572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 --2- 172.21.15.37:0/3521298175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55603a8daf10 0x55603a956d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 -- 172.21.15.37:0/3521298175 >> 172.21.15.37:0/3521298175 conn(0x55603a8b2710 msgr2=0x55603a8b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8de6ffd640 0 client.6606 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 -- 172.21.15.37:0/3521298175 shutdown_connections 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 1 -- 172.21.15.37:0/3521298175 wait complete. 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e117ec000 20 client.6606 trim_cache size 0 max 16384 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "17690648-69be-44c7-bf0c-2047e5a7ebdb", 2024-04-25T08:55:56.445 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-04-25T08:55:56.446 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:55:56.446 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T08:55:56.450+0000 7fac3dd0b640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6555=0-4194304@1} caps={6555=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6555 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55bf555ad600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 -- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8df4066f60 msgr2=0x7f8df4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8df4066f60 0x7f8df4069420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0ae000 tx=0x7f8dfc008040 comp rx=0 tx=0).stop 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 -- 172.21.15.37:0/4182332078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e0815a3c0 msgr2=0x7f8e08099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e0815a3c0 0x7f8e08099e70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8df800d830 tx=0x7f8df800dcf0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 -- 172.21.15.37:0/4182332078 shutdown_connections 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8df4066f60 0x7f8df4069420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8e0815ac30 0x7f8e080c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8e0815a7a0 0x7f8e080afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 --2- 172.21.15.37:0/4182332078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e0815a3c0 0x7f8e08099e70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 -- 172.21.15.37:0/4182332078 >> 172.21.15.37:0/4182332078 conn(0x7f8e0806ae60 msgr2=0x7f8e080cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 -- 172.21.15.37:0/4182332078 shutdown_connections 2024-04-25T08:55:56.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.450+0000 7f8e0e67f640 1 -- 172.21.15.37:0/4182332078 wait complete. 2024-04-25T08:55:56.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:56.458 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T08:55:56.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.582+0000 7f781ddb0640 1 Processor -- start 2024-04-25T08:55:56.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.582+0000 7f781ddb0640 1 -- start start 2024-04-25T08:55:56.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.582+0000 7f781ddb0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7818161c80 0x7f78180b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.582+0000 7f781ddb0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78180c9f70 0x7f78180c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.582+0000 7f781ddb0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7818162060 con 0x7f78180c2870 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.582+0000 7f781ddb0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78180c2cc0 con 0x7f7818161c80 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.582+0000 7f781ddb0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78180ca480 con 0x7f78180c9f70 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.582+0000 7f78177fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7818161c80 0x7f78180b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.582+0000 7f7817fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f78180c2870 0x7f7818165770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42000/0 (socket says 172.21.15.37:42000) 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.582+0000 7f7817fff640 1 -- 172.21.15.37:0/2964553399 learned_addr learned my addr 172.21.15.37:0/2964553399 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.582+0000 7f780effd640 1 --2- 172.21.15.37:0/2964553399 >> v2:172.21.15.203:3300/0 conn(0x7f78180c9f70 0x7f78180c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f78177fe640 1 -- 172.21.15.37:0/2964553399 >> v2:172.21.15.203:3300/0 conn(0x7f78180c9f70 msgr2=0x7f78180c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f78177fe640 1 --2- 172.21.15.37:0/2964553399 >> v2:172.21.15.203:3300/0 conn(0x7f78180c9f70 0x7f78180c2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f78177fe640 1 -- 172.21.15.37:0/2964553399 >> v1:172.21.15.37:6789/0 conn(0x7f78180c2870 legacy=0x7f7818165770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f7816ffd640 1 -- 172.21.15.37:0/2964553399 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749270982 0 0) 0x7f7818162060 con 0x7f78180c2870 2024-04-25T08:55:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f78177fe640 1 -- 172.21.15.37:0/2964553399 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78180ca740 con 0x7f7818161c80 2024-04-25T08:55:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f780effd640 1 --2- 172.21.15.37:0/2964553399 >> v2:172.21.15.203:3300/0 conn(0x7f78180c9f70 0x7f78180c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:56.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f78177fe640 1 --2- 172.21.15.37:0/2964553399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7818161c80 0x7f78180b3450 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f781806bce0 tx=0x7f780801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b206782ac57bb69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f7816ffd640 1 -- 172.21.15.37:0/2964553399 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78080ab650 con 0x7f7818161c80 2024-04-25T08:55:56.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f7816ffd640 1 -- 172.21.15.37:0/2964553399 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78080b0070 con 0x7f7818161c80 2024-04-25T08:55:56.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f7816ffd640 1 -- 172.21.15.37:0/2964553399 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78080aba10 con 0x7f7818161c80 2024-04-25T08:55:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f781ddb0640 1 -- 172.21.15.37:0/2964553399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7818161c80 msgr2=0x7f78180b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f781ddb0640 1 --2- 172.21.15.37:0/2964553399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7818161c80 0x7f78180b3450 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f781806bce0 tx=0x7f780801c450 comp rx=0 tx=0).stop 2024-04-25T08:55:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f781ddb0640 1 -- 172.21.15.37:0/2964553399 shutdown_connections 2024-04-25T08:55:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f781ddb0640 1 --2- 172.21.15.37:0/2964553399 >> v2:172.21.15.203:3300/0 conn(0x7f78180c9f70 0x7f78180c2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f781ddb0640 1 --2- 172.21.15.37:0/2964553399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7818161c80 0x7f78180b3450 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f781ddb0640 1 -- 172.21.15.37:0/2964553399 >> 172.21.15.37:0/2964553399 conn(0x7f781806ae60 msgr2=0x7f781806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f781ddb0640 1 -- 172.21.15.37:0/2964553399 shutdown_connections 2024-04-25T08:55:56.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.586+0000 7f781ddb0640 1 -- 172.21.15.37:0/2964553399 wait complete. 2024-04-25T08:55:56.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f781ddb0640 1 Processor -- start 2024-04-25T08:55:56.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f781ddb0640 1 -- start start 2024-04-25T08:55:56.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f781ddb0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78180c2870 0x7f7818157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f781ddb0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78180c9f70 0x7f78180cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f781ddb0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7818161c80 0x7f78180cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f781ddb0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78180b3b10 con 0x7f7818161c80 2024-04-25T08:55:56.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f781ddb0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78180b3990 con 0x7f78180c9f70 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f781ddb0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781816e8a0 con 0x7f78180c2870 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7817fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7818161c80 0x7f78180cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7817fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7818161c80 0x7f78180cbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57408/0 (socket says 172.21.15.37:57408) 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7817fff640 1 -- 172.21.15.37:0/4137210231 learned_addr learned my addr 172.21.15.37:0/4137210231 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f780effd640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78180c9f70 0x7f78180cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f78177fe640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78180c2870 0x7f7818157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7817fff640 1 -- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78180c2870 msgr2=0x7f7818157470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7817fff640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78180c2870 0x7f7818157470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7817fff640 1 -- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78180c9f70 msgr2=0x7f78180cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7817fff640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78180c9f70 0x7f78180cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7817fff640 1 -- 172.21.15.37:0/4137210231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78180b09c0 con 0x7f7818161c80 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7817fff640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7818161c80 0x7f78180cbb90 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7810001570 tx=0x7f781000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7814ff9640 1 -- 172.21.15.37:0/4137210231 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810018070 con 0x7f7818161c80 2024-04-25T08:55:56.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7814ff9640 1 -- 172.21.15.37:0/4137210231 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781000a640 con 0x7f7818161c80 2024-04-25T08:55:56.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7814ff9640 1 -- 172.21.15.37:0/4137210231 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781000aa30 con 0x7f7818161c80 2024-04-25T08:55:56.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f781ddb0640 1 -- 172.21.15.37:0/4137210231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78180aefa0 con 0x7f7818161c80 2024-04-25T08:55:56.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f781ddb0640 1 -- 172.21.15.37:0/4137210231 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781815f4e0 con 0x7f7818161c80 2024-04-25T08:55:56.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.590+0000 7f7814ff9640 1 -- 172.21.15.37:0/4137210231 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f781000abd0 con 0x7f7818161c80 2024-04-25T08:55:56.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7814ff9640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77f8066f60 0x7f77f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7814ff9640 1 -- 172.21.15.37:0/4137210231 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7810084880 con 0x7f7818161c80 2024-04-25T08:55:56.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f78177fe640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77f8066f60 0x7f77f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 Processor -- start 2024-04-25T08:55:56.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 -- start start 2024-04-25T08:55:56.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f78177fe640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77f8066f60 0x7f77f8069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f78080af000 tx=0x7f78080a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b9579d0 0x55db9b957db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9582f0 0x55db9b90db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b90e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9b75aff0 con 0x55db9b90e0a0 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9b6e3150 con 0x55db9b9582f0 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db9b757090 con 0x55db9b9579d0 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7817fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b90e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7817fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b90e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57420/0 (socket says 172.21.15.37:57420) 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7817fff640 1 -- 172.21.15.37:0/1966995957 learned_addr learned my addr 172.21.15.37:0/1966995957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f780effd640 1 --2- 172.21.15.37:0/1966995957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9582f0 0x55db9b90db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f78177fe640 1 --2- 172.21.15.37:0/1966995957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b9579d0 0x55db9b957db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7817fff640 1 -- 172.21.15.37:0/1966995957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b9579d0 msgr2=0x55db9b957db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7817fff640 1 --2- 172.21.15.37:0/1966995957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b9579d0 0x55db9b957db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7817fff640 1 -- 172.21.15.37:0/1966995957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9582f0 msgr2=0x55db9b90db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7817fff640 1 --2- 172.21.15.37:0/1966995957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9582f0 0x55db9b90db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7817fff640 1 -- 172.21.15.37:0/1966995957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db9b8f10d0 con 0x55db9b90e0a0 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7817fff640 1 --2- 172.21.15.37:0/1966995957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b90e480 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7810082b80 tx=0x7f78100556f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4d524075743995 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f780d7fa640 1 -- 172.21.15.37:0/1966995957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810059140 con 0x55db9b90e0a0 2024-04-25T08:55:56.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f780d7fa640 1 -- 172.21.15.37:0/1966995957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781005d700 con 0x55db9b90e0a0 2024-04-25T08:55:56.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f780d7fa640 1 -- 172.21.15.37:0/1966995957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810051660 con 0x55db9b90e0a0 2024-04-25T08:55:56.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 -- 172.21.15.37:0/1966995957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 msgr2=0x55db9b90e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 --2- 172.21.15.37:0/1966995957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b90e480 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7810082b80 tx=0x7f78100556f0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 -- 172.21.15.37:0/1966995957 shutdown_connections 2024-04-25T08:55:56.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 --2- 172.21.15.37:0/1966995957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b90e480 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 --2- 172.21.15.37:0/1966995957 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9582f0 0x55db9b90db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 --2- 172.21.15.37:0/1966995957 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b9579d0 0x55db9b957db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 -- 172.21.15.37:0/1966995957 >> 172.21.15.37:0/1966995957 conn(0x55db9b90a020 msgr2=0x55db9b90c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 -- 172.21.15.37:0/1966995957 shutdown_connections 2024-04-25T08:55:56.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.594+0000 7f7820f1d000 1 -- 172.21.15.37:0/1966995957 wait complete. 2024-04-25T08:55:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 -- start start 2024-04-25T08:55:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 Processor -- start 2024-04-25T08:55:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b93c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b9579d0 0x55db9b93cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9582f0 0x55db9b93d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9b71f250 con 0x55db9b90e0a0 2024-04-25T08:55:56.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9b715690 con 0x55db9b9582f0 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db9b6e3a40 con 0x55db9b9579d0 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 10 client.0 _ll_register_callbacks cb 0x55db9b8f8e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f78177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b93c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f78177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b93c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57424/0 (socket says 172.21.15.37:57424) 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f78177fe640 1 -- 172.21.15.37:0/669329410 learned_addr learned my addr 172.21.15.37:0/669329410 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f780effd640 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b9579d0 0x55db9b93cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7817fff640 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9582f0 0x55db9b93d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f78177fe640 1 -- 172.21.15.37:0/669329410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b9579d0 msgr2=0x55db9b93cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f78177fe640 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b9579d0 0x55db9b93cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f78177fe640 1 -- 172.21.15.37:0/669329410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9582f0 msgr2=0x55db9b93d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f78177fe640 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9582f0 0x55db9b93d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f78177fe640 1 -- 172.21.15.37:0/669329410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db9b8081c0 con 0x55db9b90e0a0 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7817fff640 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9582f0 0x55db9b93d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:56.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f78177fe640 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b93c7f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f780801a080 tx=0x7f780801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cbff1866819d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f77f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f77f67fc640 1 -- 172.21.15.37:0/669329410 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78080cd060 con 0x55db9b90e0a0 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f77f67fc640 1 -- 172.21.15.37:0/669329410 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f78080bfa50 con 0x55db9b90e0a0 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 -- 172.21.15.37:0/669329410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f781005d700 con 0x55db9b90e0a0 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f77f67fc640 1 -- 172.21.15.37:0/669329410 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78080d3050 con 0x55db9b90e0a0 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 10 client.6621 fetch_fsmap learned FSMap version 33 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 -- 172.21.15.37:0/669329410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55db9b804170 con 0x55db9b90e0a0 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f77f67fc640 1 -- 172.21.15.37:0/669329410 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781005d700 con 0x55db9b90e0a0 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f77f67fc640 1 -- 172.21.15.37:0/669329410 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f78080d6020 con 0x55db9b90e0a0 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 10 client.6621 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 10 client.6621 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 20 client.6621 populate_metadata read hostname 'smithi037' 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55db9b967c80 0x55db9b96a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 4 client.6621 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 -- 172.21.15.37:0/669329410 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db9b6f6c60 con 0x55db9b967c80 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f780effd640 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55db9b967c80 0x55db9b96a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f780effd640 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55db9b967c80 0x55db9b96a070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9046b2fb6208cebc server_cookie=9fabdb0998c5de38 in_seq=0 out_seq=0 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f77f67fc640 10 client.6621 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f77f67fc640 1 -- 172.21.15.37:0/669329410 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55db9b6f6c60 con 0x55db9b967c80 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f77f67fc640 10 client.6621 handle_command_reply: tid=0 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 client.6621 shutdown 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 -- 172.21.15.37:0/669329410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 msgr2=0x55db9b93c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b93c7f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f780801a080 tx=0x7f780801ac90 comp rx=0 tx=0).stop 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 -- 172.21.15.37:0/669329410 shutdown_connections 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55db9b967c80 0x55db9b96a070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9582f0 0x55db9b93d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b9579d0 0x55db9b93cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f77f67fc640 0 client.6621 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 --2- 172.21.15.37:0/669329410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b90e0a0 0x55db9b93c7f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.598+0000 7f7820f1d000 1 -- 172.21.15.37:0/669329410 >> 172.21.15.37:0/669329410 conn(0x55db9b90a020 msgr2=0x55db9b90cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.602+0000 7f7820f1d000 1 -- 172.21.15.37:0/669329410 shutdown_connections 2024-04-25T08:55:56.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.602+0000 7f7820f1d000 1 -- 172.21.15.37:0/669329410 wait complete. 2024-04-25T08:55:56.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.602+0000 7f7820f1d000 20 client.6621 trim_cache size 0 max 16384 2024-04-25T08:55:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.614+0000 7f7820f1d000 1 Processor -- start 2024-04-25T08:55:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.614+0000 7f7820f1d000 1 -- start start 2024-04-25T08:55:56.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b90e0a0 0x55db9b9b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9579d0 0x55db9b9b4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b9b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7818207490 con 0x55db9b9582f0 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7818206670 con 0x55db9b9579d0 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78181588b0 con 0x55db9b90e0a0 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b9b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b9b4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57428/0 (socket says 172.21.15.37:57428) 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 -- 172.21.15.37:0/4169572019 learned_addr learned my addr 172.21.15.37:0/4169572019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f780effd640 1 --2- 172.21.15.37:0/4169572019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9579d0 0x55db9b9b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f78177fe640 1 --2- 172.21.15.37:0/4169572019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b90e0a0 0x55db9b9b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 -- 172.21.15.37:0/4169572019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b90e0a0 msgr2=0x55db9b9b40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 --2- 172.21.15.37:0/4169572019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b90e0a0 0x55db9b9b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 -- 172.21.15.37:0/4169572019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9579d0 msgr2=0x55db9b9b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 --2- 172.21.15.37:0/4169572019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9579d0 0x55db9b9b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 -- 172.21.15.37:0/4169572019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db9b7a1de0 con 0x55db9b9582f0 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 --2- 172.21.15.37:0/4169572019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b9b4b70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f781000a430 tx=0x7f7810084070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb8bb9cfb67bf3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f780cff9640 1 -- 172.21.15.37:0/4169572019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781005d700 con 0x55db9b9582f0 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f780cff9640 1 -- 172.21.15.37:0/4169572019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7810053090 con 0x55db9b9582f0 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f780cff9640 1 -- 172.21.15.37:0/4169572019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810054440 con 0x55db9b9582f0 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- 172.21.15.37:0/4169572019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 msgr2=0x55db9b9b4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 --2- 172.21.15.37:0/4169572019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b9b4b70 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f781000a430 tx=0x7f7810084070 comp rx=0 tx=0).stop 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- 172.21.15.37:0/4169572019 shutdown_connections 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 --2- 172.21.15.37:0/4169572019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b9b4b70 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 --2- 172.21.15.37:0/4169572019 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9579d0 0x55db9b9b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 --2- 172.21.15.37:0/4169572019 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b90e0a0 0x55db9b9b40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- 172.21.15.37:0/4169572019 >> 172.21.15.37:0/4169572019 conn(0x55db9b90a020 msgr2=0x55db9b964fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- 172.21.15.37:0/4169572019 shutdown_connections 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- 172.21.15.37:0/4169572019 wait complete. 2024-04-25T08:55:56.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- start start 2024-04-25T08:55:56.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 Processor -- start 2024-04-25T08:55:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b90e0a0 0x55db9b96a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9579d0 0x55db9b96a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b96ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9b6f5e00 con 0x55db9b9582f0 2024-04-25T08:55:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7818206070 con 0x55db9b9579d0 2024-04-25T08:55:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55db9b75a300 con 0x55db9b90e0a0 2024-04-25T08:55:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 10 client.0 _ll_register_callbacks cb 0x55db9b8d9040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b96ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b96ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57430/0 (socket says 172.21.15.37:57430) 2024-04-25T08:55:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 -- 172.21.15.37:0/3484596743 learned_addr learned my addr 172.21.15.37:0/3484596743 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f780effd640 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9579d0 0x55db9b96a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f78177fe640 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b90e0a0 0x55db9b96a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 -- 172.21.15.37:0/3484596743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b90e0a0 msgr2=0x55db9b96a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b90e0a0 0x55db9b96a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 -- 172.21.15.37:0/3484596743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9579d0 msgr2=0x55db9b96a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9579d0 0x55db9b96a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 -- 172.21.15.37:0/3484596743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db9b6fc8e0 con 0x55db9b9582f0 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7817fff640 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b96ae00 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f781000a430 tx=0x7f7810001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b570515cec1ce32f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f77f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f77f67fc640 1 -- 172.21.15.37:0/3484596743 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7810071060 con 0x55db9b9582f0 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f77f67fc640 1 -- 172.21.15.37:0/3484596743 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f781005b0e0 con 0x55db9b9582f0 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.618+0000 7f7820f1d000 1 -- 172.21.15.37:0/3484596743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7810053090 con 0x55db9b9582f0 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f77f67fc640 1 -- 172.21.15.37:0/3484596743 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7810078050 con 0x55db9b9582f0 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 10 client.6627 fetch_fsmap learned FSMap version 33 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 -- 172.21.15.37:0/3484596743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55db9b7b48f0 con 0x55db9b9582f0 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f77f67fc640 1 -- 172.21.15.37:0/3484596743 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7810053090 con 0x55db9b9582f0 2024-04-25T08:55:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f77f67fc640 1 -- 172.21.15.37:0/3484596743 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7810050d40 con 0x55db9b9582f0 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 10 client.6627 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 10 client.6627 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 20 client.6627 populate_metadata read hostname 'smithi037' 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55db9b96b7f0 0x55db9b9bff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 4 client.6627 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 -- 172.21.15.37:0/3484596743 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55db9b768530 con 0x55db9b96b7f0 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f78177fe640 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55db9b96b7f0 0x55db9b9bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f77f67fc640 10 client.6627 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f78177fe640 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55db9b96b7f0 0x55db9b9bff60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bae5df3a27b04350 server_cookie=2f26440c35e28321 in_seq=0 out_seq=0 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f77f67fc640 1 -- 172.21.15.37:0/3484596743 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55db9b768530 con 0x55db9b96b7f0 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f77f67fc640 10 client.6627 handle_command_reply: tid=0 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 client.6627 shutdown 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 -- 172.21.15.37:0/3484596743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 msgr2=0x55db9b96ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b96ae00 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f781000a430 tx=0x7f7810001260 comp rx=0 tx=0).stop 2024-04-25T08:55:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 -- 172.21.15.37:0/3484596743 shutdown_connections 2024-04-25T08:55:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55db9b96b7f0 0x55db9b9bff60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55db9b9582f0 0x55db9b96ae00 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55db9b9579d0 0x55db9b96a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 --2- 172.21.15.37:0/3484596743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55db9b90e0a0 0x55db9b96a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 -- 172.21.15.37:0/3484596743 >> 172.21.15.37:0/3484596743 conn(0x55db9b90a020 msgr2=0x55db9b9560d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f77f67fc640 0 client.6627 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 -- 172.21.15.37:0/3484596743 shutdown_connections 2024-04-25T08:55:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 1 -- 172.21.15.37:0/3484596743 wait complete. 2024-04-25T08:55:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f7820f1d000 20 client.6627 trim_cache size 0 max 16384 2024-04-25T08:55:56.618 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:55:56.619 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-04-25T08:55:56.619 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-04-25T08:55:56.619 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:55:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 -- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77f8066f60 msgr2=0x7f77f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77f8066f60 0x7f77f8069420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f78080af000 tx=0x7f78080a7000 comp rx=0 tx=0).stop 2024-04-25T08:55:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 -- 172.21.15.37:0/4137210231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7818161c80 msgr2=0x7f78180cbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7818161c80 0x7f78180cbb90 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f7810001570 tx=0x7f781000cca0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 -- 172.21.15.37:0/4137210231 shutdown_connections 2024-04-25T08:55:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77f8066f60 0x7f77f8069420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7818161c80 0x7f78180cbb90 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78180c9f70 0x7f78180cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 --2- 172.21.15.37:0/4137210231 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78180c2870 0x7f7818157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 -- 172.21.15.37:0/4137210231 >> 172.21.15.37:0/4137210231 conn(0x7f781806ae60 msgr2=0x7f781806d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 -- 172.21.15.37:0/4137210231 shutdown_connections 2024-04-25T08:55:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.622+0000 7f781ddb0640 1 -- 172.21.15.37:0/4137210231 wait complete. 2024-04-25T08:55:56.629 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:55:56.629 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:56.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.742+0000 7f7881548640 1 Processor -- start 2024-04-25T08:55:56.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.742+0000 7f7881548640 1 -- start start 2024-04-25T08:55:56.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7881548640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787c15ecc0 0x7f787c06e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7881548640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c0b2090 con 0x7f787c0b3f50 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7881548640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c0b43e0 con 0x7f787c15a700 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7881548640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0b46c0 con 0x7f787c15ecc0 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f787a7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f787c0b3f50 0x7f787c0b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42008/0 (socket says 172.21.15.37:42008) 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f787a7fc640 1 -- 172.21.15.37:0/780354862 learned_addr learned my addr 172.21.15.37:0/780354862 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f787b7fe640 1 --2- 172.21.15.37:0/780354862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787c15ecc0 0x7f787c06e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518533284 0 0) 0x7f787c0b2090 con 0x7f787c0b3f50 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7864003530 con 0x7f787c0b3f50 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4177747960 0 0) 0x7f7864003530 con 0x7f787c0b3f50 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787c15ecc0 msgr2=0x7f787c06e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 --2- 172.21.15.37:0/780354862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787c15ecc0 0x7f787c06e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 >> v1:172.21.15.203:6789/0 conn(0x7f787c15a700 legacy=0x7f787c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0cd6f0 con 0x7f787c0b3f50 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f78700016a0 con 0x7f787c0b3f50 2024-04-25T08:55:56.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 >> v1:172.21.15.37:6789/0 conn(0x7f787c0b3f50 legacy=0x7f787c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 --2- 172.21.15.37:0/780354862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7864003a60 0x7f7864003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 --2- 172.21.15.37:0/780354862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787c15ecc0 0x7f7864004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 --2- 172.21.15.37:0/780354862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7864004d50 0x7f78640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0b46c0 con 0x7f7864004d50 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786400bc00 con 0x7f7864003a60 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786400e010 con 0x7f787c15ecc0 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f787b7fe640 1 --2- 172.21.15.37:0/780354862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7864004d50 0x7f78640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f787affd640 1 --2- 172.21.15.37:0/780354862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7864003a60 0x7f7864003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f787b7fe640 1 -- 172.21.15.37:0/780354862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787c15ecc0 msgr2=0x7f7864004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f787b7fe640 1 --2- 172.21.15.37:0/780354862 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f787c15ecc0 0x7f7864004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f787b7fe640 1 -- 172.21.15.37:0/780354862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7864003a60 msgr2=0x7f7864003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f787b7fe640 1 --2- 172.21.15.37:0/780354862 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7864003a60 0x7f7864003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f787b7fe640 1 -- 172.21.15.37:0/780354862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864003530 con 0x7f7864004d50 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f787b7fe640 1 --2- 172.21.15.37:0/780354862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7864004d50 0x7f78640051d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7874009fc0 tx=0x7f787401cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8616be5266371c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78740b2070 con 0x7f7864004d50 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78740a5be0 con 0x7f7864004d50 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.746+0000 7f7879ffb640 1 -- 172.21.15.37:0/780354862 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874009c10 con 0x7f7864004d50 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- 172.21.15.37:0/780354862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7864004d50 msgr2=0x7f78640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 --2- 172.21.15.37:0/780354862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7864004d50 0x7f78640051d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7874009fc0 tx=0x7f787401cc90 comp rx=0 tx=0).stop 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787b7fe640 1 -- 172.21.15.37:0/780354862 reap_dead start 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- 172.21.15.37:0/780354862 shutdown_connections 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- 172.21.15.37:0/780354862 >> 172.21.15.37:0/780354862 conn(0x7f787c0b11a0 msgr2=0x7f787c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- 172.21.15.37:0/780354862 shutdown_connections 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- 172.21.15.37:0/780354862 wait complete. 2024-04-25T08:55:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 Processor -- start 2024-04-25T08:55:56.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- start start 2024-04-25T08:55:56.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7864004d50 0x7f787c144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f787c1449f0 0x7f787c160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f787c160950 0x7f787c15c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0cd310 con 0x7f787c160950 2024-04-25T08:55:56.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c15f0d0 con 0x7f787c1449f0 2024-04-25T08:55:56.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0cd490 con 0x7f7864004d50 2024-04-25T08:55:56.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f787c160950 0x7f787c15c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f787c160950 0x7f787c15c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57438/0 (socket says 172.21.15.37:57438) 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787b7fe640 1 -- 172.21.15.37:0/894985387 learned_addr learned my addr 172.21.15.37:0/894985387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787affd640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7864004d50 0x7f787c144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787a7fc640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f787c1449f0 0x7f787c160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787b7fe640 1 -- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7864004d50 msgr2=0x7f787c144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787b7fe640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7864004d50 0x7f787c144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787b7fe640 1 -- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f787c1449f0 msgr2=0x7f787c160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787b7fe640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f787c1449f0 0x7f787c160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787b7fe640 1 -- 172.21.15.37:0/894985387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c160ed0 con 0x7f787c160950 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f787b7fe640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f787c160950 0x7f787c15c950 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7874002240 tx=0x7f7874009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f78637fe640 1 -- 172.21.15.37:0/894985387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78740bd070 con 0x7f787c160950 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f78637fe640 1 -- 172.21.15.37:0/894985387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78740a59e0 con 0x7f787c160950 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- 172.21.15.37:0/894985387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c1611b0 con 0x7f787c160950 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f78637fe640 1 -- 172.21.15.37:0/894985387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78740b2040 con 0x7f787c160950 2024-04-25T08:55:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- 172.21.15.37:0/894985387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c161770 con 0x7f787c160950 2024-04-25T08:55:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.750+0000 7f7881548640 1 -- 172.21.15.37:0/894985387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c04efa0 con 0x7f787c160950 2024-04-25T08:55:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.754+0000 7f78637fe640 1 -- 172.21.15.37:0/894985387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78740c0070 con 0x7f787c160950 2024-04-25T08:55:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.754+0000 7f78637fe640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f90 0x7f7868069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.754+0000 7f78637fe640 1 -- 172.21.15.37:0/894985387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f787411ee10 con 0x7f787c160950 2024-04-25T08:55:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.754+0000 7f787affd640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f90 0x7f7868069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:56.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.754+0000 7f787affd640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f90 0x7f7868069450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f787c0b0fc0 tx=0x7f786c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:56.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.758+0000 7f78637fe640 1 -- 172.21.15.37:0/894985387 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7874122030 con 0x7f787c160950 2024-04-25T08:55:56.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.942+0000 7f7881548640 1 -- 172.21.15.37:0/894985387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f787c06f260 con 0x7f787c160950 2024-04-25T08:55:56.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.942+0000 7f78637fe640 1 -- 172.21.15.37:0/894985387 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f78740169d0 con 0x7f787c160950 2024-04-25T08:55:56.934 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:56.934 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/486740526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":486740526},{"type":"v1","addr":"172.21.15.37:6833","nonce":486740526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/121484331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":121484331},{"type":"v1","addr":"172.21.15.203:6835","nonce":121484331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2501630566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2501630566},{"type":"v1","addr":"172.21.15.203:6837","nonce":2501630566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:55:46.200939+0000","modified":"2024-04-25T08:55:49.230920+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6405":{"gid":6405,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/2189232625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2189232625},{"type":"v1","addr":"172.21.15.37:6835","nonce":2189232625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:55:56.935 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 33 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 -- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f90 msgr2=0x7f7868069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f90 0x7f7868069450 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f787c0b0fc0 tx=0x7f786c009040 comp rx=0 tx=0).stop 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 -- 172.21.15.37:0/894985387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f787c160950 msgr2=0x7f787c15c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f787c160950 0x7f787c15c950 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7874002240 tx=0x7f7874009750 comp rx=0 tx=0).stop 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 -- 172.21.15.37:0/894985387 shutdown_connections 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f90 0x7f7868069450 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f787c160950 0x7f787c15c950 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f787c1449f0 0x7f787c160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 --2- 172.21.15.37:0/894985387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7864004d50 0x7f787c144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 -- 172.21.15.37:0/894985387 >> 172.21.15.37:0/894985387 conn(0x7f787c0b11a0 msgr2=0x7f787c159e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 -- 172.21.15.37:0/894985387 shutdown_connections 2024-04-25T08:55:56.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:56.946+0000 7f7881548640 1 -- 172.21.15.37:0/894985387 wait complete. 2024-04-25T08:55:56.952 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T08:55:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.082+0000 7f807bd04640 1 Processor -- start 2024-04-25T08:55:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.082+0000 7f807bd04640 1 -- start start 2024-04-25T08:55:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807bd04640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80740cf590 0x7f80740cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807bd04640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8074072000 0x7f8074072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807bd04640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80740cbd20 con 0x7f807415b6b0 2024-04-25T08:55:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807bd04640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807415ba90 con 0x7f80740cf590 2024-04-25T08:55:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807bd04640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80740cc000 con 0x7f8074072000 2024-04-25T08:55:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807929f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80740cf590 0x7f80740cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807929f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80740cf590 0x7f80740cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57450/0 (socket says 172.21.15.37:57450) 2024-04-25T08:55:57.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807929f640 1 -- 172.21.15.37:0/3623390884 learned_addr learned my addr 172.21.15.37:0/3623390884 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807a2a1640 1 --2- 172.21.15.37:0/3623390884 >> v2:172.21.15.203:3300/0 conn(0x7f8074072000 0x7f8074072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807929f640 1 -- 172.21.15.37:0/3623390884 >> v2:172.21.15.203:3300/0 conn(0x7f8074072000 msgr2=0x7f8074072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807929f640 1 --2- 172.21.15.37:0/3623390884 >> v2:172.21.15.203:3300/0 conn(0x7f8074072000 0x7f8074072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807929f640 1 -- 172.21.15.37:0/3623390884 >> v1:172.21.15.203:6789/0 conn(0x7f807415b6b0 legacy=0x7f80740ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:57.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807929f640 1 -- 172.21.15.37:0/3623390884 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80740cc2e0 con 0x7f80740cf590 2024-04-25T08:55:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f807929f640 1 --2- 172.21.15.37:0/3623390884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80740cf590 0x7f80740cfa20 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f807406f000 tx=0x7f806801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154371882897c0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f8078a9e640 1 -- 172.21.15.37:0/3623390884 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680a5760 con 0x7f80740cf590 2024-04-25T08:55:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f8078a9e640 1 -- 172.21.15.37:0/3623390884 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80680a5900 con 0x7f80740cf590 2024-04-25T08:55:57.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.086+0000 7f8078a9e640 1 -- 172.21.15.37:0/3623390884 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680a5cd0 con 0x7f80740cf590 2024-04-25T08:55:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 -- 172.21.15.37:0/3623390884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80740cf590 msgr2=0x7f80740cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 --2- 172.21.15.37:0/3623390884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80740cf590 0x7f80740cfa20 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f807406f000 tx=0x7f806801c9b0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 -- 172.21.15.37:0/3623390884 shutdown_connections 2024-04-25T08:55:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 --2- 172.21.15.37:0/3623390884 >> v2:172.21.15.203:3300/0 conn(0x7f8074072000 0x7f8074072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 --2- 172.21.15.37:0/3623390884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80740cf590 0x7f80740cfa20 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 -- 172.21.15.37:0/3623390884 >> 172.21.15.37:0/3623390884 conn(0x7f807406ae60 msgr2=0x7f807406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 -- 172.21.15.37:0/3623390884 shutdown_connections 2024-04-25T08:55:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 -- 172.21.15.37:0/3623390884 wait complete. 2024-04-25T08:55:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 Processor -- start 2024-04-25T08:55:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 -- start start 2024-04-25T08:55:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8074072000 0x7f80740c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80740cf590 0x7f807406e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807415b6b0 0x7f80740c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80740caf30 con 0x7f80740cf590 2024-04-25T08:55:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80740cadb0 con 0x7f8074072000 2024-04-25T08:55:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80741682a0 con 0x7f807415b6b0 2024-04-25T08:55:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807929f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80740cf590 0x7f807406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807929f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80740cf590 0x7f807406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57454/0 (socket says 172.21.15.37:57454) 2024-04-25T08:55:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807929f640 1 -- 172.21.15.37:0/2740409048 learned_addr learned my addr 172.21.15.37:0/2740409048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f8079aa0640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8074072000 0x7f80740c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807a2a1640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807415b6b0 0x7f80740c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807929f640 1 -- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807415b6b0 msgr2=0x7f80740c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807929f640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807415b6b0 0x7f80740c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807929f640 1 -- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8074072000 msgr2=0x7f80740c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807929f640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8074072000 0x7f80740c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807929f640 1 -- 172.21.15.37:0/2740409048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80740ac170 con 0x7f80740cf590 2024-04-25T08:55:57.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f8079aa0640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8074072000 0x7f80740c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:57.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807929f640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80740cf590 0x7f807406e940 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f806801a5e0 tx=0x7f8068019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:57.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f80667fc640 1 -- 172.21.15.37:0/2740409048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680bc020 con 0x7f80740cf590 2024-04-25T08:55:57.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f80667fc640 1 -- 172.21.15.37:0/2740409048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80680b8670 con 0x7f80740cf590 2024-04-25T08:55:57.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 -- 172.21.15.37:0/2740409048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80740c5b60 con 0x7f80740cf590 2024-04-25T08:55:57.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f80667fc640 1 -- 172.21.15.37:0/2740409048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680b8a60 con 0x7f80740cf590 2024-04-25T08:55:57.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.090+0000 7f807bd04640 1 -- 172.21.15.37:0/2740409048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80740ac710 con 0x7f80740cf590 2024-04-25T08:55:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f80667fc640 1 -- 172.21.15.37:0/2740409048 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80680bf020 con 0x7f80740cf590 2024-04-25T08:55:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f80667fc640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8058066f60 0x7f8058069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f80667fc640 1 -- 172.21.15.37:0/2740409048 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8068120210 con 0x7f80740cf590 2024-04-25T08:55:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f8079aa0640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8058066f60 0x7f8058069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807ee71000 1 Processor -- start 2024-04-25T08:55:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807ee71000 1 -- start start 2024-04-25T08:55:57.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f8079aa0640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8058066f60 0x7f8058069420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80700098b0 tx=0x7f8070008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807ee71000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86916c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807ee71000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c8691c00 0x5592c86473a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807ee71000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c86478e0 0x5592c8647cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807ee71000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c852eba0 con 0x5592c8691c00 2024-04-25T08:55:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807ee71000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c8494090 con 0x5592c86912e0 2024-04-25T08:55:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807ee71000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592c841c160 con 0x5592c86478e0 2024-04-25T08:55:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807929f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c8691c00 0x5592c86473a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807929f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c8691c00 0x5592c86473a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57470/0 (socket says 172.21.15.37:57470) 2024-04-25T08:55:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807929f640 1 -- 172.21.15.37:0/4027563738 learned_addr learned my addr 172.21.15.37:0/4027563738 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:57.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807a2a1640 1 --2- 172.21.15.37:0/4027563738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c86478e0 0x5592c8647cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807929f640 1 -- 172.21.15.37:0/4027563738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c86478e0 msgr2=0x5592c8647cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807929f640 1 --2- 172.21.15.37:0/4027563738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c86478e0 0x5592c8647cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807929f640 1 -- 172.21.15.37:0/4027563738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 msgr2=0x5592c86916c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f8079aa0640 1 --2- 172.21.15.37:0/4027563738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86916c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807929f640 1 --2- 172.21.15.37:0/4027563738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86916c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f807929f640 1 -- 172.21.15.37:0/4027563738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592c853dc20 con 0x5592c8691c00 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.094+0000 7f8079aa0640 1 --2- 172.21.15.37:0/4027563738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86916c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807929f640 1 --2- 172.21.15.37:0/4027563738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c8691c00 0x5592c86473a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f80680f0840 tx=0x7f80680f04a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=741f7fbd2f7a62fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f80577fe640 1 -- 172.21.15.37:0/4027563738 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8068120050 con 0x5592c8691c00 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f80577fe640 1 -- 172.21.15.37:0/4027563738 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80680f1240 con 0x5592c8691c00 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f80577fe640 1 -- 172.21.15.37:0/4027563738 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680f0b40 con 0x5592c8691c00 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- 172.21.15.37:0/4027563738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c8691c00 msgr2=0x5592c86473a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 --2- 172.21.15.37:0/4027563738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c8691c00 0x5592c86473a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f80680f0840 tx=0x7f80680f04a0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- 172.21.15.37:0/4027563738 shutdown_connections 2024-04-25T08:55:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 --2- 172.21.15.37:0/4027563738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c86478e0 0x5592c8647cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 --2- 172.21.15.37:0/4027563738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c8691c00 0x5592c86473a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 --2- 172.21.15.37:0/4027563738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86916c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- 172.21.15.37:0/4027563738 >> 172.21.15.37:0/4027563738 conn(0x5592c8643890 msgr2=0x5592c8645c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- 172.21.15.37:0/4027563738 shutdown_connections 2024-04-25T08:55:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- 172.21.15.37:0/4027563738 wait complete. 2024-04-25T08:55:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- start start 2024-04-25T08:55:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 Processor -- start 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c86478e0 0x5592c8652480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86912e0 0x5592c86529c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c8691c00 0x5592c8652f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c8490130 con 0x5592c86912e0 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c8458130 con 0x5592c8691c00 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592c8552300 con 0x5592c86478e0 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 10 client.0 _ll_register_callbacks cb 0x5592c8632bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807929f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86912e0 0x5592c86529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807929f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86912e0 0x5592c86529c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57472/0 (socket says 172.21.15.37:57472) 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807929f640 1 -- 172.21.15.37:0/3389679582 learned_addr learned my addr 172.21.15.37:0/3389679582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f8079aa0640 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c86478e0 0x5592c8652480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807a2a1640 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c8691c00 0x5592c8652f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807929f640 1 -- 172.21.15.37:0/3389679582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c86478e0 msgr2=0x5592c8652480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807929f640 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c86478e0 0x5592c8652480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807929f640 1 -- 172.21.15.37:0/3389679582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c8691c00 msgr2=0x5592c8652f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807929f640 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c8691c00 0x5592c8652f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807929f640 1 -- 172.21.15.37:0/3389679582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592c8471060 con 0x5592c86912e0 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807929f640 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86912e0 0x5592c86529c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f806801cf20 tx=0x7f8068009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154efc82940c1cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f8054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f8054ff9640 1 -- 172.21.15.37:0/3389679582 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680f4c90 con 0x5592c86912e0 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f8054ff9640 1 -- 172.21.15.37:0/3389679582 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80680f0da0 con 0x5592c86912e0 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- 172.21.15.37:0/3389679582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80680f1240 con 0x5592c86912e0 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f8054ff9640 1 -- 172.21.15.37:0/3389679582 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80680fe050 con 0x5592c86912e0 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 10 client.6645 fetch_fsmap learned FSMap version 33 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- 172.21.15.37:0/3389679582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5592c84357c0 con 0x5592c86912e0 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f8054ff9640 1 -- 172.21.15.37:0/3389679582 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80680f1240 con 0x5592c86912e0 2024-04-25T08:55:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f8054ff9640 1 -- 172.21.15.37:0/3389679582 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8068101080 con 0x5592c86912e0 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 10 client.6645 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 10 client.6645 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 20 client.6645 populate_metadata read hostname 'smithi037' 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5592c8653770 0x5592c86a2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 4 client.6645 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f807ee71000 1 -- 172.21.15.37:0/3389679582 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592c842fcd0 con 0x5592c8653770 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.098+0000 7f8079aa0640 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5592c8653770 0x5592c86a2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f8054ff9640 10 client.6645 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f8079aa0640 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5592c8653770 0x5592c86a2850 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c52061f60d808dcd server_cookie=1613c66f00428162 in_seq=0 out_seq=0 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f8054ff9640 1 -- 172.21.15.37:0/3389679582 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592c842fcd0 con 0x5592c8653770 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f8054ff9640 10 client.6645 handle_command_reply: tid=0 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 1 client.6645 shutdown 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 1 -- 172.21.15.37:0/3389679582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86912e0 msgr2=0x5592c86529c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86912e0 0x5592c86529c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f806801cf20 tx=0x7f8068009810 comp rx=0 tx=0).stop 2024-04-25T08:55:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 1 -- 172.21.15.37:0/3389679582 shutdown_connections 2024-04-25T08:55:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5592c8653770 0x5592c86a2850 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c8691c00 0x5592c8652f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86912e0 0x5592c86529c0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f8054ff9640 0 client.6645 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 1 --2- 172.21.15.37:0/3389679582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c86478e0 0x5592c8652480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 1 -- 172.21.15.37:0/3389679582 >> 172.21.15.37:0/3389679582 conn(0x5592c8643890 msgr2=0x5592c8646510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 1 -- 172.21.15.37:0/3389679582 shutdown_connections 2024-04-25T08:55:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 1 -- 172.21.15.37:0/3389679582 wait complete. 2024-04-25T08:55:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.102+0000 7f807ee71000 20 client.6645 trim_cache size 0 max 16384 2024-04-25T08:55:57.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 Processor -- start 2024-04-25T08:55:57.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 -- start start 2024-04-25T08:55:57.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c8643890 0x5592c864ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c8646c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86471d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8074202310 con 0x5592c86478e0 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80740c3a00 con 0x5592c86912e0 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80740c4000 con 0x5592c8643890 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807929f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c8646c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807929f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c8646c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57484/0 (socket says 172.21.15.37:57484) 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807929f640 1 -- 172.21.15.37:0/1444428133 learned_addr learned my addr 172.21.15.37:0/1444428133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807929f640 1 -- 172.21.15.37:0/1444428133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c8643890 msgr2=0x5592c864ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f8079aa0640 1 --2- 172.21.15.37:0/1444428133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c8643890 0x5592c864ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807a2a1640 1 --2- 172.21.15.37:0/1444428133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807929f640 1 --2- 172.21.15.37:0/1444428133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c8643890 0x5592c864ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807929f640 1 -- 172.21.15.37:0/1444428133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 msgr2=0x5592c86471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807929f640 1 --2- 172.21.15.37:0/1444428133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86471d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807929f640 1 -- 172.21.15.37:0/1444428133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592c844ffc0 con 0x5592c86478e0 2024-04-25T08:55:57.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807929f640 1 --2- 172.21.15.37:0/1444428133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c8646c90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f80680ecfa0 tx=0x7f8068106070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce5579ab6200319 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:57.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f8056ffd640 1 -- 172.21.15.37:0/1444428133 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8068120040 con 0x5592c86478e0 2024-04-25T08:55:57.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f8056ffd640 1 -- 172.21.15.37:0/1444428133 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8068108070 con 0x5592c86478e0 2024-04-25T08:55:57.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f8056ffd640 1 -- 172.21.15.37:0/1444428133 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80680f0fa0 con 0x5592c86478e0 2024-04-25T08:55:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 -- 172.21.15.37:0/1444428133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 msgr2=0x5592c8646c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 --2- 172.21.15.37:0/1444428133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c8646c90 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f80680ecfa0 tx=0x7f8068106070 comp rx=0 tx=0).stop 2024-04-25T08:55:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 -- 172.21.15.37:0/1444428133 shutdown_connections 2024-04-25T08:55:57.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 --2- 172.21.15.37:0/1444428133 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 --2- 172.21.15.37:0/1444428133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c8646c90 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 --2- 172.21.15.37:0/1444428133 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c8643890 0x5592c864ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 -- 172.21.15.37:0/1444428133 >> 172.21.15.37:0/1444428133 conn(0x5592c8652ec0 msgr2=0x5592c864bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 -- 172.21.15.37:0/1444428133 shutdown_connections 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.114+0000 7f807ee71000 1 -- 172.21.15.37:0/1444428133 wait complete. 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- start start 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 Processor -- start 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c8643890 0x5592c86f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c86f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c842ee70 con 0x5592c86478e0 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592c841ca50 con 0x5592c86912e0 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592c84933a0 con 0x5592c8643890 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 10 client.0 _ll_register_callbacks cb 0x5592c8440240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807929f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c86f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807929f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c86f1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57494/0 (socket says 172.21.15.37:57494) 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807929f640 1 -- 172.21.15.37:0/669638371 learned_addr learned my addr 172.21.15.37:0/669638371 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807929f640 1 -- 172.21.15.37:0/669638371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c8643890 msgr2=0x5592c86f0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807a2a1640 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807929f640 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c8643890 0x5592c86f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807929f640 1 -- 172.21.15.37:0/669638371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 msgr2=0x5592c86ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807929f640 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807929f640 1 -- 172.21.15.37:0/669638371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592c84ed9e0 con 0x5592c86478e0 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807929f640 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c86f1190 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f806801b060 tx=0x7f8068009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bf93c892010be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8054ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8054ff9640 1 -- 172.21.15.37:0/669638371 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8068120070 con 0x5592c86478e0 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8054ff9640 1 -- 172.21.15.37:0/669638371 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80680f8fd0 con 0x5592c86478e0 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- 172.21.15.37:0/669638371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8068108070 con 0x5592c86478e0 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8054ff9640 1 -- 172.21.15.37:0/669638371 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80680f1240 con 0x5592c86478e0 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 10 client.6651 fetch_fsmap learned FSMap version 33 2024-04-25T08:55:57.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- 172.21.15.37:0/669638371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x5592c8433630 con 0x5592c86478e0 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8054ff9640 1 -- 172.21.15.37:0/669638371 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8068108070 con 0x5592c86478e0 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8054ff9640 1 -- 172.21.15.37:0/669638371 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8068101030 con 0x5592c86478e0 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 10 client.6651 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 10 client.6651 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 20 client.6651 populate_metadata read hostname 'smithi037' 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5592c8691c00 0x5592c86acc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 4 client.6651 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- 172.21.15.37:0/669638371 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5592c84a15d0 con 0x5592c8691c00 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8079aa0640 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5592c8691c00 0x5592c86acc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8054ff9640 10 client.6651 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8079aa0640 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5592c8691c00 0x5592c86acc90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99a7d9b94ef14547 server_cookie=ba792167631e6ae2 in_seq=0 out_seq=0 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8054ff9640 1 -- 172.21.15.37:0/669638371 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+140 (crc 0 0 0) 0x5592c84a15d0 con 0x5592c8691c00 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8054ff9640 10 client.6651 handle_command_reply: tid=0 2024-04-25T08:55:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 client.6651 shutdown 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- 172.21.15.37:0/669638371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 msgr2=0x5592c86f1190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c86f1190 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f806801b060 tx=0x7f8068009040 comp rx=0 tx=0).stop 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- 172.21.15.37:0/669638371 shutdown_connections 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5592c8691c00 0x5592c86acc90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592c86912e0 0x5592c86ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f8054ff9640 0 client.6651 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592c86478e0 0x5592c86f1190 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 --2- 172.21.15.37:0/669638371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592c8643890 0x5592c86f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- 172.21.15.37:0/669638371 >> 172.21.15.37:0/669638371 conn(0x5592c8652ec0 msgr2=0x5592c8690ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- 172.21.15.37:0/669638371 shutdown_connections 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 1 -- 172.21.15.37:0/669638371 wait complete. 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807ee71000 20 client.6651 trim_cache size 0 max 16384 2024-04-25T08:55:57.115 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:57.116 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T08:55:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:55:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "backtrace", 2024-04-25T08:55:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 589047987, 2024-04-25T08:55:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627777, 2024-04-25T08:55:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/dir_test/file" 2024-04-25T08:55:57.116 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:55:57.116 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T08:55:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807bd04640 1 -- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8058066f60 msgr2=0x7f8058069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807bd04640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8058066f60 0x7f8058069420 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80700098b0 tx=0x7f8070008040 comp rx=0 tx=0).stop 2024-04-25T08:55:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807bd04640 1 -- 172.21.15.37:0/2740409048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80740cf590 msgr2=0x7f807406e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807bd04640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80740cf590 0x7f807406e940 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f806801a5e0 tx=0x7f8068019750 comp rx=0 tx=0).stop 2024-04-25T08:55:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807bd04640 1 -- 172.21.15.37:0/2740409048 shutdown_connections 2024-04-25T08:55:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807bd04640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8058066f60 0x7f8058069420 secure :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f80700098b0 tx=0x7f8070008040 comp rx=0 tx=0).stop 2024-04-25T08:55:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807bd04640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f807415b6b0 0x7f80740c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807bd04640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80740cf590 0x7f807406e940 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807bd04640 1 --2- 172.21.15.37:0/2740409048 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8074072000 0x7f80740c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.118+0000 7f807bd04640 1 -- 172.21.15.37:0/2740409048 >> 172.21.15.37:0/2740409048 conn(0x7f807406ae60 msgr2=0x7f80740b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:57.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.122+0000 7f807bd04640 1 -- 172.21.15.37:0/2740409048 shutdown_connections 2024-04-25T08:55:57.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.122+0000 7f807bd04640 1 -- 172.21.15.37:0/2740409048 wait complete. 2024-04-25T08:55:57.126 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:55:57.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.246+0000 7f8721dcc640 1 Processor -- start 2024-04-25T08:55:57.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.246+0000 7f8721dcc640 1 -- start start 2024-04-25T08:55:57.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f8721dcc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f871c06f340 0x7f871c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f8721dcc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f871c06c3b0 0x7f871c151e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f8721dcc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f871c0727e0 con 0x7f871c0af170 2024-04-25T08:55:57.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f8721dcc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871c06f720 con 0x7f871c06c3b0 2024-04-25T08:55:57.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f8721dcc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c06d850 con 0x7f871c06f340 2024-04-25T08:55:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f871b7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f871c06f340 0x7f871c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f871bfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f871c06c3b0 0x7f871c151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f8713fff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f871c0af170 0x7f871c06bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:54548/0 (socket says 172.21.15.37:54548) 2024-04-25T08:55:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f8713fff640 1 -- 172.21.15.37:0/3687340682 learned_addr learned my addr 172.21.15.37:0/3687340682 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:55:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f871bfff640 1 -- 172.21.15.37:0/3687340682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f871c06f340 msgr2=0x7f871c0669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f871bfff640 1 --2- 172.21.15.37:0/3687340682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f871c06f340 0x7f871c0669a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f871bfff640 1 -- 172.21.15.37:0/3687340682 >> v1:172.21.15.203:6789/0 conn(0x7f871c0af170 legacy=0x7f871c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f871bfff640 1 -- 172.21.15.37:0/3687340682 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c067110 con 0x7f871c06c3b0 2024-04-25T08:55:57.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f871bfff640 1 --2- 172.21.15.37:0/3687340682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c06c3b0 0x7f871c151e70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8714009d90 tx=0x7f871401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=168aad67fad2b0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:57.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f871affd640 1 -- 172.21.15.37:0/3687340682 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87140b2070 con 0x7f871c06c3b0 2024-04-25T08:55:57.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f871affd640 1 -- 172.21.15.37:0/3687340682 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87140a5830 con 0x7f871c06c3b0 2024-04-25T08:55:57.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.250+0000 7f871affd640 1 -- 172.21.15.37:0/3687340682 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87140a5c00 con 0x7f871c06c3b0 2024-04-25T08:55:57.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 -- 172.21.15.37:0/3687340682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c06c3b0 msgr2=0x7f871c151e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 --2- 172.21.15.37:0/3687340682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c06c3b0 0x7f871c151e70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8714009d90 tx=0x7f871401c970 comp rx=0 tx=0).stop 2024-04-25T08:55:57.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 -- 172.21.15.37:0/3687340682 shutdown_connections 2024-04-25T08:55:57.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 --2- 172.21.15.37:0/3687340682 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c06c3b0 0x7f871c151e70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 --2- 172.21.15.37:0/3687340682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f871c06f340 0x7f871c0669a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 -- 172.21.15.37:0/3687340682 >> 172.21.15.37:0/3687340682 conn(0x7f871c0717f0 msgr2=0x7f871c071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:57.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 -- 172.21.15.37:0/3687340682 shutdown_connections 2024-04-25T08:55:57.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 -- 172.21.15.37:0/3687340682 wait complete. 2024-04-25T08:55:57.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 Processor -- start 2024-04-25T08:55:57.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 -- start start 2024-04-25T08:55:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c06c3b0 0x7f871c14df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f871c06f340 0x7f871c13b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f871c0af170 0x7f871c14aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c0ce9d0 con 0x7f871c06f340 2024-04-25T08:55:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c15bee0 con 0x7f871c06c3b0 2024-04-25T08:55:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8721dcc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871c0ceb50 con 0x7f871c0af170 2024-04-25T08:55:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8713fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f871c06f340 0x7f871c13b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8713fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f871c06f340 0x7f871c13b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57496/0 (socket says 172.21.15.37:57496) 2024-04-25T08:55:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8713fff640 1 -- 172.21.15.37:0/3186946641 learned_addr learned my addr 172.21.15.37:0/3186946641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:55:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f871b7fe640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c06c3b0 0x7f871c14df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f871bfff640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f871c0af170 0x7f871c14aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8713fff640 1 -- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f871c0af170 msgr2=0x7f871c14aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8713fff640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f871c0af170 0x7f871c14aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8713fff640 1 -- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c06c3b0 msgr2=0x7f871c14df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8713fff640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c06c3b0 0x7f871c14df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8713fff640 1 -- 172.21.15.37:0/3186946641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c14af90 con 0x7f871c06f340 2024-04-25T08:55:57.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f871bfff640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f871c0af170 0x7f871c14aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:57.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f871b7fe640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c06c3b0 0x7f871c14df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:57.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.254+0000 7f8713fff640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f871c06f340 0x7f871c13b650 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f870400c370 tx=0x7f870400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:57.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.258+0000 7f8718ff9640 1 -- 172.21.15.37:0/3186946641 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704017070 con 0x7f871c06f340 2024-04-25T08:55:57.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.258+0000 7f8721dcc640 1 -- 172.21.15.37:0/3186946641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c14a1b0 con 0x7f871c06f340 2024-04-25T08:55:57.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.258+0000 7f8718ff9640 1 -- 172.21.15.37:0/3186946641 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8704005cf0 con 0x7f871c06f340 2024-04-25T08:55:57.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.258+0000 7f8718ff9640 1 -- 172.21.15.37:0/3186946641 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87040072f0 con 0x7f871c06f340 2024-04-25T08:55:57.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.258+0000 7f8721dcc640 1 -- 172.21.15.37:0/3186946641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c14a770 con 0x7f871c06f340 2024-04-25T08:55:57.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.258+0000 7f8718ff9640 1 -- 172.21.15.37:0/3186946641 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f870401a070 con 0x7f871c06f340 2024-04-25T08:55:57.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.258+0000 7f8721dcc640 1 -- 172.21.15.37:0/3186946641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c04efa0 con 0x7f871c06f340 2024-04-25T08:55:57.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.258+0000 7f8718ff9640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f86fc066f60 0x7f86fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:57.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.258+0000 7f8718ff9640 1 -- 172.21.15.37:0/3186946641 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87040844a0 con 0x7f871c06f340 2024-04-25T08:55:57.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.258+0000 7f871b7fe640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f86fc066f60 0x7f86fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.262+0000 7f871b7fe640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f86fc066f60 0x7f86fc069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f870c009ae0 tx=0x7f870c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.262+0000 7f8718ff9640 1 -- 172.21.15.37:0/3186946641 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f871c0727e0 con 0x7f871c06f340 2024-04-25T08:55:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.458+0000 7f8721dcc640 1 -- 172.21.15.37:0/3186946641 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f871c14b790 con 0x7f871c06f340 2024-04-25T08:55:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.458+0000 7f8718ff9640 1 -- 172.21.15.37:0/3186946641 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f87040553b0 con 0x7f871c06f340 2024-04-25T08:55:57.450 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:55:57.450 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:55:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 -- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f86fc066f60 msgr2=0x7f86fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f86fc066f60 0x7f86fc069420 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f870c009ae0 tx=0x7f870c008040 comp rx=0 tx=0).stop 2024-04-25T08:55:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 -- 172.21.15.37:0/3186946641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f871c06f340 msgr2=0x7f871c13b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f871c06f340 0x7f871c13b650 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f870400c370 tx=0x7f870400c830 comp rx=0 tx=0).stop 2024-04-25T08:55:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 -- 172.21.15.37:0/3186946641 shutdown_connections 2024-04-25T08:55:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f86fc066f60 0x7f86fc069420 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f871c0af170 0x7f871c14aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f871c06f340 0x7f871c13b650 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 --2- 172.21.15.37:0/3186946641 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f871c06c3b0 0x7f871c14df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 -- 172.21.15.37:0/3186946641 >> 172.21.15.37:0/3186946641 conn(0x7f871c0717f0 msgr2=0x7f871c0b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:57.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 -- 172.21.15.37:0/3186946641 shutdown_connections 2024-04-25T08:55:57.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:55:57.462+0000 7f8721dcc640 1 -- 172.21.15.37:0/3186946641 wait complete. 2024-04-25T08:55:57.469 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=100 retry_count=0)... 2024-04-25T08:55:59.246 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T08:55:59.254+0000 7ff670aed640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:56:02.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:56:02.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.614+0000 7f35a9d53640 1 Processor -- start 2024-04-25T08:56:02.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.614+0000 7f35a9d53640 1 -- start start 2024-04-25T08:56:02.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f35a9d53640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 0x7f35a40ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f35a9d53640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f35a40cec20 0x7f35a415a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f35a9d53640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a40cdca0 con 0x7f35a40ce840 2024-04-25T08:56:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f35a9d53640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a40cb2d0 con 0x7f35a40cec20 2024-04-25T08:56:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f35a9d53640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40cb5b0 con 0x7f35a40cd750 2024-04-25T08:56:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f359bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 0x7f35a40ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f35a3fff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f35a40cec20 0x7f35a415a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f359bfff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 0x7f35a40ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:41950/0 (socket says 172.21.15.37:41950) 2024-04-25T08:56:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f35a37fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f35a40ce840 0x7f35a40cbdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:37658/0 (socket says 172.21.15.37:37658) 2024-04-25T08:56:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f359bfff640 1 -- 172.21.15.37:0/1926871849 learned_addr learned my addr 172.21.15.37:0/1926871849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f359bfff640 1 -- 172.21.15.37:0/1926871849 >> v2:172.21.15.203:3300/0 conn(0x7f35a40cec20 msgr2=0x7f35a415a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f359bfff640 1 --2- 172.21.15.37:0/1926871849 >> v2:172.21.15.203:3300/0 conn(0x7f35a40cec20 0x7f35a415a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f359bfff640 1 -- 172.21.15.37:0/1926871849 >> v1:172.21.15.203:6789/0 conn(0x7f35a40ce840 legacy=0x7f35a40cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.618+0000 7f359bfff640 1 -- 172.21.15.37:0/1926871849 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40b2d50 con 0x7f35a40cd750 2024-04-25T08:56:02.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f359bfff640 1 --2- 172.21.15.37:0/1926871849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 0x7f35a40ce040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f358c009870 tx=0x7f358c01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec670d0391765223 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:02.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a2ffd640 1 -- 172.21.15.37:0/1926871849 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0a7070 con 0x7f35a40cd750 2024-04-25T08:56:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a2ffd640 1 -- 172.21.15.37:0/1926871849 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c0b0070 con 0x7f35a40cd750 2024-04-25T08:56:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a2ffd640 1 -- 172.21.15.37:0/1926871849 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0ab890 con 0x7f35a40cd750 2024-04-25T08:56:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a9d53640 1 -- 172.21.15.37:0/1926871849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 msgr2=0x7f35a40ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a9d53640 1 --2- 172.21.15.37:0/1926871849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 0x7f35a40ce040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f358c009870 tx=0x7f358c01c560 comp rx=0 tx=0).stop 2024-04-25T08:56:02.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a9d53640 1 -- 172.21.15.37:0/1926871849 shutdown_connections 2024-04-25T08:56:02.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a9d53640 1 --2- 172.21.15.37:0/1926871849 >> v2:172.21.15.203:3300/0 conn(0x7f35a40cec20 0x7f35a415a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a9d53640 1 --2- 172.21.15.37:0/1926871849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 0x7f35a40ce040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a9d53640 1 -- 172.21.15.37:0/1926871849 >> 172.21.15.37:0/1926871849 conn(0x7f35a40b1fa0 msgr2=0x7f35a40b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:02.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a9d53640 1 -- 172.21.15.37:0/1926871849 shutdown_connections 2024-04-25T08:56:02.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a9d53640 1 -- 172.21.15.37:0/1926871849 wait complete. 2024-04-25T08:56:02.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a9d53640 1 Processor -- start 2024-04-25T08:56:02.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.622+0000 7f35a9d53640 1 -- start start 2024-04-25T08:56:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a9d53640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 0x7f35a40c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a9d53640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35a40ce840 0x7f35a414e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a9d53640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40cec20 0x7f35a40c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a9d53640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a4167700 con 0x7f35a40ce840 2024-04-25T08:56:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a9d53640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a40adce0 con 0x7f35a40cec20 2024-04-25T08:56:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a9d53640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a4167880 con 0x7f35a40cd750 2024-04-25T08:56:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f359bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35a40ce840 0x7f35a414e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f359bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35a40ce840 0x7f35a414e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46866/0 (socket says 172.21.15.37:46866) 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f359bfff640 1 -- 172.21.15.37:0/3299778477 learned_addr learned my addr 172.21.15.37:0/3299778477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a37fe640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 0x7f35a40c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a3fff640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40cec20 0x7f35a40c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f359bfff640 1 -- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 msgr2=0x7f35a40c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f359bfff640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 0x7f35a40c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f359bfff640 1 -- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40cec20 msgr2=0x7f35a40c9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f359bfff640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40cec20 0x7f35a40c9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f359bfff640 1 -- 172.21.15.37:0/3299778477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40ca380 con 0x7f35a40ce840 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a37fe640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 0x7f35a40c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a3fff640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40cec20 0x7f35a40c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f359bfff640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35a40ce840 0x7f35a414e9e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f358c0ad040 tx=0x7f358c01ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a0ff9640 1 -- 172.21.15.37:0/3299778477 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0be5f0 con 0x7f35a40ce840 2024-04-25T08:56:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a9d53640 1 -- 172.21.15.37:0/3299778477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a414ef20 con 0x7f35a40ce840 2024-04-25T08:56:02.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a0ff9640 1 -- 172.21.15.37:0/3299778477 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c0be7b0 con 0x7f35a40ce840 2024-04-25T08:56:02.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a0ff9640 1 -- 172.21.15.37:0/3299778477 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0beba0 con 0x7f35a40ce840 2024-04-25T08:56:02.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a9d53640 1 -- 172.21.15.37:0/3299778477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a40ca0a0 con 0x7f35a40ce840 2024-04-25T08:56:02.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a9d53640 1 -- 172.21.15.37:0/3299778477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a404efa0 con 0x7f35a40ce840 2024-04-25T08:56:02.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a0ff9640 1 -- 172.21.15.37:0/3299778477 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f358c0b43f0 con 0x7f35a40ce840 2024-04-25T08:56:02.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.626+0000 7f35a0ff9640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3584066f60 0x7f3584069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.630+0000 7f35a0ff9640 1 -- 172.21.15.37:0/3299778477 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f358c11f2e0 con 0x7f35a40ce840 2024-04-25T08:56:02.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.630+0000 7f35a37fe640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3584066f60 0x7f3584069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.630+0000 7f35a37fe640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3584066f60 0x7f3584069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3594000f70 tx=0x7f3594007d50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.634+0000 7f35a0ff9640 1 -- 172.21.15.37:0/3299778477 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f358c016150 con 0x7f35a40ce840 2024-04-25T08:56:02.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.822+0000 7f35a9d53640 1 -- 172.21.15.37:0/3299778477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f35a406ddc0 con 0x7f35a40ce840 2024-04-25T08:56:02.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a0ff9640 1 -- 172.21.15.37:0/3299778477 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f358c0efc40 con 0x7f35a40ce840 2024-04-25T08:56:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:02.815 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T08:56:02.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 -- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3584066f60 msgr2=0x7f3584069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3584066f60 0x7f3584069420 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3594000f70 tx=0x7f3594007d50 comp rx=0 tx=0).stop 2024-04-25T08:56:02.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 -- 172.21.15.37:0/3299778477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35a40ce840 msgr2=0x7f35a414e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35a40ce840 0x7f35a414e9e0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f358c0ad040 tx=0x7f358c01ada0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 -- 172.21.15.37:0/3299778477 shutdown_connections 2024-04-25T08:56:02.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3584066f60 0x7f3584069420 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35a40cec20 0x7f35a40c9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35a40ce840 0x7f35a414e9e0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 --2- 172.21.15.37:0/3299778477 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35a40cd750 0x7f35a40c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 -- 172.21.15.37:0/3299778477 >> 172.21.15.37:0/3299778477 conn(0x7f35a40b1fa0 msgr2=0x7f35a40cbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:02.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 -- 172.21.15.37:0/3299778477 shutdown_connections 2024-04-25T08:56:02.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.826+0000 7f35a9d53640 1 -- 172.21.15.37:0/3299778477 wait complete. 2024-04-25T08:56:02.834 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T08:56:02.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:02.834 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-04-25T08:56:02.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b805c640 1 Processor -- start 2024-04-25T08:56:02.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b805c640 1 -- start start 2024-04-25T08:56:02.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b805c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f62b0158120 0x7f62b015b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b805c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f62b00cef90 0x7f62b015eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b805c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b0158500 con 0x7f62b00b5780 2024-04-25T08:56:02.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b805c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b006bed0 con 0x7f62b0158120 2024-04-25T08:56:02.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b805c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b015ac60 con 0x7f62b00cef90 2024-04-25T08:56:02.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b5df8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f62b0158120 0x7f62b015b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b5df8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f62b0158120 0x7f62b015b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46870/0 (socket says 172.21.15.37:46870) 2024-04-25T08:56:02.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b5df8640 1 -- 172.21.15.37:0/180311610 learned_addr learned my addr 172.21.15.37:0/180311610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:02.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b65f9640 1 --2- 172.21.15.37:0/180311610 >> v2:172.21.15.203:3300/0 conn(0x7f62b00cef90 0x7f62b015eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b5df8640 1 -- 172.21.15.37:0/180311610 >> v2:172.21.15.203:3300/0 conn(0x7f62b00cef90 msgr2=0x7f62b015eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:02.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b5df8640 1 --2- 172.21.15.37:0/180311610 >> v2:172.21.15.203:3300/0 conn(0x7f62b00cef90 0x7f62b015eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b5df8640 1 -- 172.21.15.37:0/180311610 >> v1:172.21.15.203:6789/0 conn(0x7f62b00b5780 legacy=0x7f62b00ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.974+0000 7f62b5df8640 1 -- 172.21.15.37:0/180311610 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b015af40 con 0x7f62b0158120 2024-04-25T08:56:02.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.978+0000 7f62b5df8640 1 --2- 172.21.15.37:0/180311610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b0158120 0x7f62b015b740 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f62b006d020 tx=0x7f62ac01eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864c810378246ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:02.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.978+0000 7f62b4df6640 1 -- 172.21.15.37:0/180311610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0b4070 con 0x7f62b0158120 2024-04-25T08:56:02.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.978+0000 7f62b4df6640 1 -- 172.21.15.37:0/180311610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac0a7ac0 con 0x7f62b0158120 2024-04-25T08:56:02.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.978+0000 7f62b4df6640 1 -- 172.21.15.37:0/180311610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0af440 con 0x7f62b0158120 2024-04-25T08:56:02.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.978+0000 7f62b805c640 1 -- 172.21.15.37:0/180311610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b0158120 msgr2=0x7f62b015b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.978+0000 7f62b805c640 1 --2- 172.21.15.37:0/180311610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b0158120 0x7f62b015b740 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f62b006d020 tx=0x7f62ac01eb20 comp rx=0 tx=0).stop 2024-04-25T08:56:02.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.978+0000 7f62b805c640 1 -- 172.21.15.37:0/180311610 shutdown_connections 2024-04-25T08:56:02.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.978+0000 7f62b805c640 1 --2- 172.21.15.37:0/180311610 >> v2:172.21.15.203:3300/0 conn(0x7f62b00cef90 0x7f62b015eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.978+0000 7f62b805c640 1 --2- 172.21.15.37:0/180311610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b0158120 0x7f62b015b740 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.978+0000 7f62b805c640 1 -- 172.21.15.37:0/180311610 >> 172.21.15.37:0/180311610 conn(0x7f62b00cc680 msgr2=0x7f62b00cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:02.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 -- 172.21.15.37:0/180311610 shutdown_connections 2024-04-25T08:56:02.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 -- 172.21.15.37:0/180311610 wait complete. 2024-04-25T08:56:02.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 Processor -- start 2024-04-25T08:56:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 -- start start 2024-04-25T08:56:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b5780 0x7f62b0154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b00cef90 0x7f62b0164ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b0158120 0x7f62b01561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00b4730 con 0x7f62b00b5780 2024-04-25T08:56:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00b45b0 con 0x7f62b00cef90 2024-04-25T08:56:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b00b48b0 con 0x7f62b0158120 2024-04-25T08:56:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b5df8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b5780 0x7f62b0154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b5df8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b5780 0x7f62b0154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46876/0 (socket says 172.21.15.37:46876) 2024-04-25T08:56:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b5df8640 1 -- 172.21.15.37:0/2721948402 learned_addr learned my addr 172.21.15.37:0/2721948402 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b55f7640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b00cef90 0x7f62b0164ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b65f9640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b0158120 0x7f62b01561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b5df8640 1 -- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b0158120 msgr2=0x7f62b01561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b5df8640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b0158120 0x7f62b01561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b5df8640 1 -- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b00cef90 msgr2=0x7f62b0164ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b5df8640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b00cef90 0x7f62b0164ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b5df8640 1 -- 172.21.15.37:0/2721948402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b006f030 con 0x7f62b00b5780 2024-04-25T08:56:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b5df8640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b5780 0x7f62b0154890 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00b2c0 tx=0x7f62ac01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62a67fc640 1 -- 172.21.15.37:0/2721948402 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0bf070 con 0x7f62b00b5780 2024-04-25T08:56:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62a67fc640 1 -- 172.21.15.37:0/2721948402 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac0ba440 con 0x7f62b00b5780 2024-04-25T08:56:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 -- 172.21.15.37:0/2721948402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b0154f90 con 0x7f62b00b5780 2024-04-25T08:56:02.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62a67fc640 1 -- 172.21.15.37:0/2721948402 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0b4070 con 0x7f62b00b5780 2024-04-25T08:56:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.982+0000 7f62b805c640 1 -- 172.21.15.37:0/2721948402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b0155550 con 0x7f62b00b5780 2024-04-25T08:56:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62a67fc640 1 -- 172.21.15.37:0/2721948402 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62ac0c2070 con 0x7f62b00b5780 2024-04-25T08:56:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62a67fc640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6294066f60 0x7f6294069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62a67fc640 1 -- 172.21.15.37:0/2721948402 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f62ac11f7c0 con 0x7f62b00b5780 2024-04-25T08:56:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b55f7640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6294066f60 0x7f6294069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62bb1c9000 1 Processor -- start 2024-04-25T08:56:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62bb1c9000 1 -- start start 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b55f7640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6294066f60 0x7f6294069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f62b0156010 tx=0x7f62a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6c9c900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c9ce40 0x557bd6c53430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c53970 0x557bd6c53d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6aa0d00 con 0x557bd6c9c520 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6a28df0 con 0x557bd6c53970 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bd6a9cda0 con 0x557bd6c9ce40 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b5df8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6c9c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b5df8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6c9c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46880/0 (socket says 172.21.15.37:46880) 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b5df8640 1 -- 172.21.15.37:0/16644605 learned_addr learned my addr 172.21.15.37:0/16644605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b55f7640 1 --2- 172.21.15.37:0/16644605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c9ce40 0x557bd6c53430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b65f9640 1 --2- 172.21.15.37:0/16644605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c53970 0x557bd6c53d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b5df8640 1 -- 172.21.15.37:0/16644605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c9ce40 msgr2=0x557bd6c53430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b5df8640 1 --2- 172.21.15.37:0/16644605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c9ce40 0x557bd6c53430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b5df8640 1 -- 172.21.15.37:0/16644605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c53970 msgr2=0x557bd6c53d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b5df8640 1 --2- 172.21.15.37:0/16644605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c53970 0x557bd6c53d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b5df8640 1 -- 172.21.15.37:0/16644605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bd6a5b580 con 0x557bd6c9c520 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62b5df8640 1 --2- 172.21.15.37:0/16644605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6c9c900 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0efc30 tx=0x7f62ac01b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=480d0fb782d2c20c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62937fe640 1 -- 172.21.15.37:0/16644605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac107070 con 0x557bd6c9c520 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62937fe640 1 -- 172.21.15.37:0/16644605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac0eff30 con 0x557bd6c9c520 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.986+0000 7f62937fe640 1 -- 172.21.15.37:0/16644605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0f4080 con 0x557bd6c9c520 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- 172.21.15.37:0/16644605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 msgr2=0x557bd6c9c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/16644605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6c9c900 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f62ac0efc30 tx=0x7f62ac01b8c0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- 172.21.15.37:0/16644605 shutdown_connections 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/16644605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c53970 0x557bd6c53d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/16644605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c9ce40 0x557bd6c53430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/16644605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6c9c900 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- 172.21.15.37:0/16644605 >> 172.21.15.37:0/16644605 conn(0x557bd6b3ac10 msgr2=0x557bd6c51d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- 172.21.15.37:0/16644605 shutdown_connections 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- 172.21.15.37:0/16644605 wait complete. 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- start start 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 Processor -- start 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c53970 0x557bd6ca2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6ca30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9ce40 0x557bd6c57df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6a396d0 con 0x557bd6c9c520 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6a5b950 con 0x557bd6c9ce40 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bd6aa0d00 con 0x557bd6c53970 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 10 client.0 _ll_register_callbacks cb 0x557bd6c3f030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b55f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6ca30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b55f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6ca30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46892/0 (socket says 172.21.15.37:46892) 2024-04-25T08:56:02.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b55f7640 1 -- 172.21.15.37:0/920758621 learned_addr learned my addr 172.21.15.37:0/920758621 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b5df8640 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c53970 0x557bd6ca2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b65f9640 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9ce40 0x557bd6c57df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b55f7640 1 -- 172.21.15.37:0/920758621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c53970 msgr2=0x557bd6ca2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b55f7640 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c53970 0x557bd6ca2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b55f7640 1 -- 172.21.15.37:0/920758621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9ce40 msgr2=0x557bd6c57df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b55f7640 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9ce40 0x557bd6c57df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b55f7640 1 -- 172.21.15.37:0/920758621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bd6a3dd40 con 0x557bd6c9c520 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b5df8640 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c53970 0x557bd6ca2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b65f9640 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9ce40 0x557bd6c57df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b55f7640 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6ca30a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f62a00038b0 tx=0x7f62a0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be5de91f957393b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f6290ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f6290ff9640 1 -- 172.21.15.37:0/920758621 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a0027060 con 0x557bd6c9c520 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f6290ff9640 1 -- 172.21.15.37:0/920758621 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f62a0010040 con 0x557bd6c9c520 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- 172.21.15.37:0/920758621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62ac0eff30 con 0x557bd6c9c520 2024-04-25T08:56:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f6290ff9640 1 -- 172.21.15.37:0/920758621 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62a0015110 con 0x557bd6c9c520 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 10 client.6669 fetch_fsmap learned FSMap version 34 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- 172.21.15.37:0/920758621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x557bd6a9bcf0 con 0x557bd6c9c520 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f6290ff9640 1 -- 172.21.15.37:0/920758621 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac0eff30 con 0x557bd6c9c520 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f6290ff9640 1 -- 172.21.15.37:0/920758621 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f62a00159c0 con 0x557bd6c9c520 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 10 client.6669 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 10 client.6669 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 20 client.6669 populate_metadata read hostname 'smithi037' 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x557bd6c58b90 0x557bd6c58fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 4 client.6669 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62bb1c9000 1 -- 172.21.15.37:0/920758621 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557bd6a3c9d0 con 0x557bd6c58b90 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b5df8640 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x557bd6c58b90 0x557bd6c58fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f6290ff9640 10 client.6669 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.990+0000 7f62b5df8640 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x557bd6c58b90 0x557bd6c58fa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5daea5fe736e9dc2 server_cookie=5f53188b1395e334 in_seq=0 out_seq=0 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f6290ff9640 1 -- 172.21.15.37:0/920758621 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557bd6a3c9d0 con 0x557bd6c58b90 2024-04-25T08:56:02.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f6290ff9640 10 client.6669 handle_command_reply: tid=0 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 1 client.6669 shutdown 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 1 -- 172.21.15.37:0/920758621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 msgr2=0x557bd6ca30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6ca30a0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f62a00038b0 tx=0x7f62a0000bd0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 1 -- 172.21.15.37:0/920758621 shutdown_connections 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x557bd6c58b90 0x557bd6c58fa0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9ce40 0x557bd6c57df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f6290ff9640 0 client.6669 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9c520 0x557bd6ca30a0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/920758621 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c53970 0x557bd6ca2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 1 -- 172.21.15.37:0/920758621 >> 172.21.15.37:0/920758621 conn(0x557bd6b3ac10 msgr2=0x557bd6c52420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 1 -- 172.21.15.37:0/920758621 shutdown_connections 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 1 -- 172.21.15.37:0/920758621 wait complete. 2024-04-25T08:56:02.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:02.994+0000 7f62bb1c9000 20 client.6669 trim_cache size 0 max 16384 2024-04-25T08:56:02.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.006+0000 7f62bb1c9000 1 Processor -- start 2024-04-25T08:56:02.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.006+0000 7f62bb1c9000 1 -- start start 2024-04-25T08:56:03.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c53970 0x557bd6c57350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9c520 0x557bd6c57890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c9ce40 0x557bd6c9f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b0203380 con 0x557bd6c53970 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b0202590 con 0x557bd6c9c520 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b014ee10 con 0x557bd6c9ce40 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b5df8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c53970 0x557bd6c57350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b5df8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c53970 0x557bd6c57350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46900/0 (socket says 172.21.15.37:46900) 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b5df8640 1 -- 172.21.15.37:0/2042101383 learned_addr learned my addr 172.21.15.37:0/2042101383 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b5df8640 1 -- 172.21.15.37:0/2042101383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c9ce40 msgr2=0x557bd6c9f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b5df8640 1 --2- 172.21.15.37:0/2042101383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c9ce40 0x557bd6c9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b5df8640 1 -- 172.21.15.37:0/2042101383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9c520 msgr2=0x557bd6c57890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b5df8640 1 --2- 172.21.15.37:0/2042101383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9c520 0x557bd6c57890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b5df8640 1 -- 172.21.15.37:0/2042101383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557bd6c9cc70 con 0x557bd6c53970 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b5df8640 1 --2- 172.21.15.37:0/2042101383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c53970 0x557bd6c57350 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f62ac025170 tx=0x7f62ac0f3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30545602a064d918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f6292ffd640 1 -- 172.21.15.37:0/2042101383 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac10c070 con 0x557bd6c53970 2024-04-25T08:56:03.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f6292ffd640 1 -- 172.21.15.37:0/2042101383 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac0eff30 con 0x557bd6c53970 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f6292ffd640 1 -- 172.21.15.37:0/2042101383 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac107040 con 0x557bd6c53970 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- 172.21.15.37:0/2042101383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c53970 msgr2=0x557bd6c57350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/2042101383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c53970 0x557bd6c57350 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f62ac025170 tx=0x7f62ac0f3000 comp rx=0 tx=0).stop 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- 172.21.15.37:0/2042101383 shutdown_connections 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/2042101383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c9ce40 0x557bd6c9f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/2042101383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9c520 0x557bd6c57890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/2042101383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c53970 0x557bd6c57350 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- 172.21.15.37:0/2042101383 >> 172.21.15.37:0/2042101383 conn(0x557bd6b3ac10 msgr2=0x557bd6c9b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- 172.21.15.37:0/2042101383 shutdown_connections 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- 172.21.15.37:0/2042101383 wait complete. 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- start start 2024-04-25T08:56:03.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 Processor -- start 2024-04-25T08:56:03.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c53970 0x557bd6ca1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9c520 0x557bd6ca2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9ce40 0x557bd6cf9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b014f410 con 0x557bd6c9ce40 2024-04-25T08:56:03.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bd6a297e0 con 0x557bd6c9c520 2024-04-25T08:56:03.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557bd6aa2d50 con 0x557bd6c53970 2024-04-25T08:56:03.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 10 client.0 _ll_register_callbacks cb 0x7f62ac00b3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:03.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b65f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9ce40 0x557bd6cf9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b65f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9ce40 0x557bd6cf9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46908/0 (socket says 172.21.15.37:46908) 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b65f9640 1 -- 172.21.15.37:0/311340911 learned_addr learned my addr 172.21.15.37:0/311340911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b5df8640 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c53970 0x557bd6ca1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b55f7640 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9c520 0x557bd6ca2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b65f9640 1 -- 172.21.15.37:0/311340911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c53970 msgr2=0x557bd6ca1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b65f9640 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c53970 0x557bd6ca1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b65f9640 1 -- 172.21.15.37:0/311340911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9c520 msgr2=0x557bd6ca2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b65f9640 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9c520 0x557bd6ca2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b65f9640 1 -- 172.21.15.37:0/311340911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557bd6b23d30 con 0x557bd6c9ce40 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62b65f9640 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9ce40 0x557bd6cf9740 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007e10 tx=0x7f62a8013620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912cc857e17951e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f6290ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f6290ff9640 1 -- 172.21.15.37:0/311340911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a801d020 con 0x557bd6c9ce40 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f6290ff9640 1 -- 172.21.15.37:0/311340911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f62a8013cb0 con 0x557bd6c9ce40 2024-04-25T08:56:03.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.010+0000 7f62bb1c9000 1 -- 172.21.15.37:0/311340911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62ac0eff30 con 0x557bd6c9ce40 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f6290ff9640 1 -- 172.21.15.37:0/311340911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62a8027050 con 0x557bd6c9ce40 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 10 client.6675 fetch_fsmap learned FSMap version 34 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 -- 172.21.15.37:0/311340911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x557bd6a6d5a0 con 0x557bd6c9ce40 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f6290ff9640 1 -- 172.21.15.37:0/311340911 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac0eff30 con 0x557bd6c9ce40 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f6290ff9640 1 -- 172.21.15.37:0/311340911 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f62a802a020 con 0x557bd6c9ce40 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 10 client.6675 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 10 client.6675 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 20 client.6675 populate_metadata read hostname 'smithi037' 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x557bd6cfa570 0x557bd6d05b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 4 client.6675 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 -- 172.21.15.37:0/311340911 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) v1 -- 0x557bd6a7d610 con 0x557bd6cfa570 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b5df8640 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x557bd6cfa570 0x557bd6d05b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f6290ff9640 10 client.6675 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b5df8640 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x557bd6cfa570 0x557bd6d05b80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=124c5d4fddd33d8c server_cookie=8a9e49db118ec409 in_seq=0 out_seq=0 2024-04-25T08:56:03.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f6290ff9640 1 -- 172.21.15.37:0/311340911 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557bd6a7d610 con 0x557bd6cfa570 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f6290ff9640 10 client.6675 handle_command_reply: tid=0 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 client.6675 shutdown 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 -- 172.21.15.37:0/311340911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9ce40 msgr2=0x557bd6cf9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9ce40 0x557bd6cf9740 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007e10 tx=0x7f62a8013620 comp rx=0 tx=0).stop 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 -- 172.21.15.37:0/311340911 shutdown_connections 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x557bd6cfa570 0x557bd6d05b80 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557bd6c9ce40 0x557bd6cf9740 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557bd6c9c520 0x557bd6ca2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f6290ff9640 0 client.6675 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 --2- 172.21.15.37:0/311340911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557bd6c53970 0x557bd6ca1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 -- 172.21.15.37:0/311340911 >> 172.21.15.37:0/311340911 conn(0x557bd6b3ac10 msgr2=0x557bd6c9d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 -- 172.21.15.37:0/311340911 shutdown_connections 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 1 -- 172.21.15.37:0/311340911 wait complete. 2024-04-25T08:56:03.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62bb1c9000 20 client.6675 trim_cache size 0 max 16384 2024-04-25T08:56:03.010 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:56:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-04-25T08:56:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "b0d4976f-090d-40f7-9d0d-f67e70d4aaf2", 2024-04-25T08:56:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-04-25T08:56:03.010 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:56:03.010 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T08:56:03.014+0000 7fac3dd0b640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-25T08:55:55.321977+0000 1=1+0) (iversion lock) caps={6555=pAsLsXsFs/-@2},l=6555(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x55bf555ad600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:56:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 -- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6294066f60 msgr2=0x7f6294069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6294066f60 0x7f6294069420 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f62b0156010 tx=0x7f62a0009040 comp rx=0 tx=0).stop 2024-04-25T08:56:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 -- 172.21.15.37:0/2721948402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b5780 msgr2=0x7f62b0154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b5780 0x7f62b0154890 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00b2c0 tx=0x7f62ac01d360 comp rx=0 tx=0).stop 2024-04-25T08:56:03.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 -- 172.21.15.37:0/2721948402 shutdown_connections 2024-04-25T08:56:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6294066f60 0x7f6294069420 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62b0158120 0x7f62b01561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62b00cef90 0x7f62b0164ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 --2- 172.21.15.37:0/2721948402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62b00b5780 0x7f62b0154890 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 -- 172.21.15.37:0/2721948402 >> 172.21.15.37:0/2721948402 conn(0x7f62b00cc680 msgr2=0x7f62b0150af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 -- 172.21.15.37:0/2721948402 shutdown_connections 2024-04-25T08:56:03.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.014+0000 7f62b805c640 1 -- 172.21.15.37:0/2721948402 wait complete. 2024-04-25T08:56:03.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:03.021 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T08:56:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed840f640 1 Processor -- start 2024-04-25T08:56:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed840f640 1 -- start start 2024-04-25T08:56:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed840f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed015da80 0x7f4ed00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed840f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4ed00d07f0 0x7f4ed006c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed840f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed006be60 con 0x7f4ed00d1cd0 2024-04-25T08:56:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed840f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed006c440 con 0x7f4ed00d07f0 2024-04-25T08:56:03.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed840f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed015de60 con 0x7f4ed015da80 2024-04-25T08:56:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed69ac640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4ed00d07f0 0x7f4ed006c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed69ac640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4ed00d07f0 0x7f4ed006c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46916/0 (socket says 172.21.15.37:46916) 2024-04-25T08:56:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed69ac640 1 -- 172.21.15.37:0/4012213316 learned_addr learned my addr 172.21.15.37:0/4012213316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/4012213316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed015da80 0x7f4ed00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed69ac640 1 -- 172.21.15.37:0/4012213316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed015da80 msgr2=0x7f4ed00d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed69ac640 1 --2- 172.21.15.37:0/4012213316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed015da80 0x7f4ed00d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed69ac640 1 -- 172.21.15.37:0/4012213316 >> v1:172.21.15.203:6789/0 conn(0x7f4ed00d1cd0 legacy=0x7f4ed00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.150+0000 7f4ed69ac640 1 -- 172.21.15.37:0/4012213316 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed00d2450 con 0x7f4ed00d07f0 2024-04-25T08:56:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed69ac640 1 --2- 172.21.15.37:0/4012213316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed00d07f0 0x7f4ed006c8d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009e50 tx=0x7f4ec801ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3919515e7632656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed51a9640 1 -- 172.21.15.37:0/4012213316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80a5800 con 0x7f4ed00d07f0 2024-04-25T08:56:03.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed51a9640 1 -- 172.21.15.37:0/4012213316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec80a59a0 con 0x7f4ed00d07f0 2024-04-25T08:56:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed51a9640 1 -- 172.21.15.37:0/4012213316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80a5d70 con 0x7f4ed00d07f0 2024-04-25T08:56:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 -- 172.21.15.37:0/4012213316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed00d07f0 msgr2=0x7f4ed006c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 --2- 172.21.15.37:0/4012213316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed00d07f0 0x7f4ed006c8d0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009e50 tx=0x7f4ec801ca00 comp rx=0 tx=0).stop 2024-04-25T08:56:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 -- 172.21.15.37:0/4012213316 shutdown_connections 2024-04-25T08:56:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 --2- 172.21.15.37:0/4012213316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed00d07f0 0x7f4ed006c8d0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 --2- 172.21.15.37:0/4012213316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed015da80 0x7f4ed00d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 -- 172.21.15.37:0/4012213316 >> 172.21.15.37:0/4012213316 conn(0x7f4ed008b000 msgr2=0x7f4ed00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 -- 172.21.15.37:0/4012213316 shutdown_connections 2024-04-25T08:56:03.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 -- 172.21.15.37:0/4012213316 wait complete. 2024-04-25T08:56:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 Processor -- start 2024-04-25T08:56:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 -- start start 2024-04-25T08:56:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed00d07f0 0x7f4ed015c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed00d1cd0 0x7f4ed0160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed015da80 0x7f4ed00b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed006d7a0 con 0x7f4ed015da80 2024-04-25T08:56:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed006d620 con 0x7f4ed00d07f0 2024-04-25T08:56:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.154+0000 7f4ed840f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed006d920 con 0x7f4ed00d1cd0 2024-04-25T08:56:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed69ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed015da80 0x7f4ed00b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed69ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed015da80 0x7f4ed00b3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46928/0 (socket says 172.21.15.37:46928) 2024-04-25T08:56:03.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed69ac640 1 -- 172.21.15.37:0/2709474769 learned_addr learned my addr 172.21.15.37:0/2709474769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed00d07f0 0x7f4ed015c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed00d1cd0 0x7f4ed0160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed69ac640 1 -- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed00d1cd0 msgr2=0x7f4ed0160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed69ac640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed00d1cd0 0x7f4ed0160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed69ac640 1 -- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed00d07f0 msgr2=0x7f4ed015c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed69ac640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed00d07f0 0x7f4ed015c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed69ac640 1 -- 172.21.15.37:0/2709474769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed015e200 con 0x7f4ed015da80 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed00d07f0 0x7f4ed015c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed69ac640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed015da80 0x7f4ed00b3800 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80af040 tx=0x7f4ec80197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ec6ffd640 1 -- 172.21.15.37:0/2709474769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80b8670 con 0x7f4ed015da80 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ec6ffd640 1 -- 172.21.15.37:0/2709474769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec80be070 con 0x7f4ed015da80 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ec6ffd640 1 -- 172.21.15.37:0/2709474769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80b8a90 con 0x7f4ed015da80 2024-04-25T08:56:03.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed840f640 1 -- 172.21.15.37:0/2709474769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed015e4e0 con 0x7f4ed015da80 2024-04-25T08:56:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed840f640 1 -- 172.21.15.37:0/2709474769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed015eaa0 con 0x7f4ed015da80 2024-04-25T08:56:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ec6ffd640 1 -- 172.21.15.37:0/2709474769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ec80b8c30 con 0x7f4ed015da80 2024-04-25T08:56:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ec6ffd640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4eb4066f60 0x7f4eb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ec6ffd640 1 -- 172.21.15.37:0/2709474769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ec8120420 con 0x7f4ed015da80 2024-04-25T08:56:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4eb4066f60 0x7f4eb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4edb57c000 1 Processor -- start 2024-04-25T08:56:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4edb57c000 1 -- start start 2024-04-25T08:56:03.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.158+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4eb4066f60 0x7f4eb4069420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4ed014b8d0 tx=0x7f4ecc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075d9d0 0x55856075ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075e2f0 0x558560713b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585607140a0 0x558560714480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558560560ff0 con 0x55856075e2f0 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585604e9150 con 0x55856075d9d0 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55856055d090 con 0x5585607140a0 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075e2f0 0x558560713b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075e2f0 0x558560713b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46938/0 (socket says 172.21.15.37:46938) 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 -- 172.21.15.37:0/775274166 learned_addr learned my addr 172.21.15.37:0/775274166 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed69ac640 1 --2- 172.21.15.37:0/775274166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585607140a0 0x558560714480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/775274166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075d9d0 0x55856075ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 -- 172.21.15.37:0/775274166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585607140a0 msgr2=0x558560714480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/775274166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585607140a0 0x558560714480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 -- 172.21.15.37:0/775274166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075d9d0 msgr2=0x55856075ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/775274166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075d9d0 0x55856075ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 -- 172.21.15.37:0/775274166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585606f70d0 con 0x55856075e2f0 2024-04-25T08:56:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/775274166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075e2f0 0x558560713b60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00015a0 tx=0x7f4ec0001a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ec9e28ab556d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4eb3fff640 1 -- 172.21.15.37:0/775274166 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0019070 con 0x55856075e2f0 2024-04-25T08:56:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4eb3fff640 1 -- 172.21.15.37:0/775274166 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec00050d0 con 0x55856075e2f0 2024-04-25T08:56:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4eb3fff640 1 -- 172.21.15.37:0/775274166 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec00054a0 con 0x55856075e2f0 2024-04-25T08:56:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- 172.21.15.37:0/775274166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075e2f0 msgr2=0x558560713b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 --2- 172.21.15.37:0/775274166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075e2f0 0x558560713b60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00015a0 tx=0x7f4ec0001a60 comp rx=0 tx=0).stop 2024-04-25T08:56:03.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- 172.21.15.37:0/775274166 shutdown_connections 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 --2- 172.21.15.37:0/775274166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585607140a0 0x558560714480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 --2- 172.21.15.37:0/775274166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075e2f0 0x558560713b60 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 --2- 172.21.15.37:0/775274166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075d9d0 0x55856075ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- 172.21.15.37:0/775274166 >> 172.21.15.37:0/775274166 conn(0x558560710020 msgr2=0x558560712440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- 172.21.15.37:0/775274166 shutdown_connections 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- 172.21.15.37:0/775274166 wait complete. 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- start start 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 Processor -- start 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585607140a0 0x5585607427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x558560742d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55856075e2f0 0x558560743270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558560525250 con 0x55856075d9d0 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55856051b690 con 0x5585607140a0 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585604e9a40 con 0x55856075e2f0 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4edb57c000 10 client.0 _ll_register_callbacks cb 0x5585606fee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x558560742d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x558560742d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46950/0 (socket says 172.21.15.37:46950) 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 -- 172.21.15.37:0/3031046825 learned_addr learned my addr 172.21.15.37:0/3031046825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585607140a0 0x5585607427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed69ac640 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55856075e2f0 0x558560743270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 -- 172.21.15.37:0/3031046825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55856075e2f0 msgr2=0x558560743270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55856075e2f0 0x558560743270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 -- 172.21.15.37:0/3031046825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585607140a0 msgr2=0x5585607427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585607140a0 0x5585607427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 -- 172.21.15.37:0/3031046825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55856060e1c0 con 0x55856075d9d0 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x558560742d30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0013000 tx=0x7f4ec0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e81df76a6e592e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.162+0000 7f4eb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4eb17fa640 1 -- 172.21.15.37:0/3031046825 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0024020 con 0x55856075d9d0 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4eb17fa640 1 -- 172.21.15.37:0/3031046825 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ec000ea10 con 0x55856075d9d0 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 -- 172.21.15.37:0/3031046825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ec00050d0 con 0x55856075d9d0 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4eb17fa640 1 -- 172.21.15.37:0/3031046825 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ec002e050 con 0x55856075d9d0 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 10 client.6687 fetch_fsmap learned FSMap version 34 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 -- 172.21.15.37:0/3031046825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55856060a170 con 0x55856075d9d0 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4eb17fa640 1 -- 172.21.15.37:0/3031046825 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec00050d0 con 0x55856075d9d0 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4eb17fa640 1 -- 172.21.15.37:0/3031046825 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4ec000ec50 con 0x55856075d9d0 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 10 client.6687 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 10 client.6687 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:56:03.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 20 client.6687 populate_metadata read hostname 'smithi037' 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55856076dc80 0x558560770070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 4 client.6687 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 -- 172.21.15.37:0/3031046825 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585604fcc60 con 0x55856076dc80 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55856076dc80 0x558560770070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4eb17fa640 10 client.6687 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55856076dc80 0x558560770070 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a51108ef70eb9e server_cookie=40993c7d1640bbfe in_seq=0 out_seq=0 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4eb17fa640 1 -- 172.21.15.37:0/3031046825 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5585604fcc60 con 0x55856076dc80 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4eb17fa640 10 client.6687 handle_command_reply: tid=0 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 client.6687 shutdown 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 -- 172.21.15.37:0/3031046825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 msgr2=0x558560742d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x558560742d30 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0013000 tx=0x7f4ec0000bd0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 -- 172.21.15.37:0/3031046825 shutdown_connections 2024-04-25T08:56:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55856076dc80 0x558560770070 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55856075e2f0 0x558560743270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x558560742d30 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4eb17fa640 0 client.6687 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 --2- 172.21.15.37:0/3031046825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5585607140a0 0x5585607427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 -- 172.21.15.37:0/3031046825 >> 172.21.15.37:0/3031046825 conn(0x558560710020 msgr2=0x558560712ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 -- 172.21.15.37:0/3031046825 shutdown_connections 2024-04-25T08:56:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 1 -- 172.21.15.37:0/3031046825 wait complete. 2024-04-25T08:56:03.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.166+0000 7f4edb57c000 20 client.6687 trim_cache size 0 max 16384 2024-04-25T08:56:03.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 Processor -- start 2024-04-25T08:56:03.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 -- start start 2024-04-25T08:56:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585607140a0 0x5585607ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x5585607ba630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075e2f0 0x5585607bab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0202d80 con 0x55856075d9d0 2024-04-25T08:56:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0201fc0 con 0x55856075e2f0 2024-04-25T08:56:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed014c460 con 0x5585607140a0 2024-04-25T08:56:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4ed59aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x5585607ba630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4ed59aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x5585607ba630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46954/0 (socket says 172.21.15.37:46954) 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4ed59aa640 1 -- 172.21.15.37:0/3227926140 learned_addr learned my addr 172.21.15.37:0/3227926140 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4ed59aa640 1 -- 172.21.15.37:0/3227926140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585607140a0 msgr2=0x5585607ba0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/3227926140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585607140a0 0x5585607ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4ed59aa640 1 -- 172.21.15.37:0/3227926140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075e2f0 msgr2=0x5585607bab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/3227926140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075e2f0 0x5585607bab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4ed59aa640 1 -- 172.21.15.37:0/3227926140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585605a7de0 con 0x55856075d9d0 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/3227926140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x5585607ba630 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000e5b0 tx=0x7f4ec001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a49b08e4ed3737 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4eb37fe640 1 -- 172.21.15.37:0/3227926140 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0024070 con 0x55856075d9d0 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4eb37fe640 1 -- 172.21.15.37:0/3227926140 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec001a040 con 0x55856075d9d0 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4eb37fe640 1 -- 172.21.15.37:0/3227926140 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec0019070 con 0x55856075d9d0 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 -- 172.21.15.37:0/3227926140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 msgr2=0x5585607ba630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 --2- 172.21.15.37:0/3227926140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x5585607ba630 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000e5b0 tx=0x7f4ec001e000 comp rx=0 tx=0).stop 2024-04-25T08:56:03.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 -- 172.21.15.37:0/3227926140 shutdown_connections 2024-04-25T08:56:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 --2- 172.21.15.37:0/3227926140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075e2f0 0x5585607bab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 --2- 172.21.15.37:0/3227926140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55856075d9d0 0x5585607ba630 secure :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000e5b0 tx=0x7f4ec001e000 comp rx=0 tx=0).stop 2024-04-25T08:56:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 --2- 172.21.15.37:0/3227926140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5585607140a0 0x5585607ba0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 -- 172.21.15.37:0/3227926140 >> 172.21.15.37:0/3227926140 conn(0x558560710020 msgr2=0x55856076afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 -- 172.21.15.37:0/3227926140 shutdown_connections 2024-04-25T08:56:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 -- 172.21.15.37:0/3227926140 wait complete. 2024-04-25T08:56:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 -- start start 2024-04-25T08:56:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.182+0000 7f4edb57c000 1 Processor -- start 2024-04-25T08:56:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585607140a0 0x558560770380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075e2f0 0x5585607708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558560770e00 0x5585607711e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585604fbe00 con 0x5585607140a0 2024-04-25T08:56:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed014ca60 con 0x55856075e2f0 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558560560300 con 0x558560770e00 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 10 client.0 _ll_register_callbacks cb 0x5585606df040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed61ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585607140a0 0x558560770380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed61ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585607140a0 0x558560770380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46958/0 (socket says 172.21.15.37:46958) 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed61ab640 1 -- 172.21.15.37:0/4276987187 learned_addr learned my addr 172.21.15.37:0/4276987187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075e2f0 0x5585607708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed69ac640 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558560770e00 0x5585607711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed61ab640 1 -- 172.21.15.37:0/4276987187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558560770e00 msgr2=0x5585607711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558560770e00 0x5585607711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed61ab640 1 -- 172.21.15.37:0/4276987187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075e2f0 msgr2=0x5585607708c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075e2f0 0x5585607708c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed61ab640 1 -- 172.21.15.37:0/4276987187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585605028e0 con 0x5585607140a0 2024-04-25T08:56:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075e2f0 0x5585607708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed69ac640 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558560770e00 0x5585607711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed61ab640 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585607140a0 0x558560770380 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00fab0 tx=0x7f4ecc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965108486aabe430 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4eb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:56:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4eb17fa640 1 -- 172.21.15.37:0/4276987187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc01b040 con 0x5585607140a0 2024-04-25T08:56:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4eb17fa640 1 -- 172.21.15.37:0/4276987187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(69..69 src has 1..69) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ecc023640 con 0x5585607140a0 2024-04-25T08:56:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 -- 172.21.15.37:0/4276987187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ec001a040 con 0x5585607140a0 2024-04-25T08:56:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4eb17fa640 1 -- 172.21.15.37:0/4276987187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ecc036050 con 0x5585607140a0 2024-04-25T08:56:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 10 client.6693 fetch_fsmap learned FSMap version 34 2024-04-25T08:56:03.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 -- 172.21.15.37:0/4276987187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x5585605ba8f0 con 0x5585607140a0 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4eb17fa640 1 -- 172.21.15.37:0/4276987187 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec001a040 con 0x5585607140a0 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4eb17fa640 1 -- 172.21.15.37:0/4276987187 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 34) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4ecc039020 con 0x5585607140a0 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 10 client.6693 fetch_fsmap finished waiting for FSMap version 34 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 10 client.6693 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 20 client.6693 populate_metadata read hostname 'smithi037' 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5585607c3f50 0x5585607c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 4 client.6693 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 -- 172.21.15.37:0/4276987187 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55856056e530 con 0x5585607c3f50 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5585607c3f50 0x5585607c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4eb17fa640 10 client.6693 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed59aa640 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5585607c3f50 0x5585607c6340 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77450251ca24644a server_cookie=46223b544a4270d1 in_seq=0 out_seq=0 2024-04-25T08:56:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4eb17fa640 1 -- 172.21.15.37:0/4276987187 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x55856056e530 con 0x5585607c3f50 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4eb17fa640 10 client.6693 handle_command_reply: tid=0 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 client.6693 shutdown 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 -- 172.21.15.37:0/4276987187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585607140a0 msgr2=0x558560770380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585607140a0 0x558560770380 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00fab0 tx=0x7f4ecc0046d0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 -- 172.21.15.37:0/4276987187 shutdown_connections 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x5585607c3f50 0x5585607c6340 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558560770e00 0x5585607711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55856075e2f0 0x5585607708c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4eb17fa640 0 client.6693 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 --2- 172.21.15.37:0/4276987187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5585607140a0 0x558560770380 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 -- 172.21.15.37:0/4276987187 >> 172.21.15.37:0/4276987187 conn(0x558560710020 msgr2=0x55856075c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 -- 172.21.15.37:0/4276987187 shutdown_connections 2024-04-25T08:56:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 1 -- 172.21.15.37:0/4276987187 wait complete. 2024-04-25T08:56:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4edb57c000 20 client.6693 trim_cache size 0 max 16384 2024-04-25T08:56:03.185 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:56:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:56:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-04-25T08:56:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "b0d4976f-090d-40f7-9d0d-f67e70d4aaf2": { 2024-04-25T08:56:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-04-25T08:56:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "b0d4976f-090d-40f7-9d0d-f67e70d4aaf2", 2024-04-25T08:56:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,repair,force" 2024-04-25T08:56:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:03.186 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:56:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 -- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4eb4066f60 msgr2=0x7f4eb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4eb4066f60 0x7f4eb4069420 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4ed014b8d0 tx=0x7f4ecc008040 comp rx=0 tx=0).stop 2024-04-25T08:56:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 -- 172.21.15.37:0/2709474769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed015da80 msgr2=0x7f4ed00b3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed015da80 0x7f4ed00b3800 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80af040 tx=0x7f4ec80197a0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 -- 172.21.15.37:0/2709474769 shutdown_connections 2024-04-25T08:56:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4eb4066f60 0x7f4eb4069420 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ed015da80 0x7f4ed00b3800 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4ed00d1cd0 0x7f4ed0160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 --2- 172.21.15.37:0/2709474769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4ed00d07f0 0x7f4ed015c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 -- 172.21.15.37:0/2709474769 >> 172.21.15.37:0/2709474769 conn(0x7f4ed008b000 msgr2=0x7f4ed00d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 -- 172.21.15.37:0/2709474769 shutdown_connections 2024-04-25T08:56:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:03.186+0000 7f4ed840f640 1 -- 172.21.15.37:0/2709474769 wait complete. 2024-04-25T08:56:03.193 INFO:tasks.cephfs.filesystem:scrub status for tag:b0d4976f-090d-40f7-9d0d-f67e70d4aaf2 - {'path': '/', 'tag': 'b0d4976f-090d-40f7-9d0d-f67e70d4aaf2', 'options': 'recursive,repair,force'} 2024-04-25T08:56:25.656 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:56:25.659 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:56:33.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:33.195 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-04-25T08:56:33.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.342+0000 7f975ef98640 1 Processor -- start 2024-04-25T08:56:33.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.342+0000 7f975ef98640 1 -- start start 2024-04-25T08:56:33.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975ef98640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f975815ecb0 0x7f97580c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975ef98640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580c9ef0 0x7f97580ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975ef98640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97580ca4c0 con 0x7f97580cf590 2024-04-25T08:56:33.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975ef98640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97580ca780 con 0x7f975815ecb0 2024-04-25T08:56:33.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975ef98640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975815f0f0 con 0x7f97580c9ef0 2024-04-25T08:56:33.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975d535640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580c9ef0 0x7f97580ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975d535640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580c9ef0 0x7f97580ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:34338/0 (socket says 172.21.15.37:34338) 2024-04-25T08:56:33.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975d535640 1 -- 172.21.15.37:0/4057921308 learned_addr learned my addr 172.21.15.37:0/4057921308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f9757fff640 1 --2- 172.21.15.37:0/4057921308 >> v2:172.21.15.203:3300/0 conn(0x7f975815ecb0 0x7f97580c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975d535640 1 -- 172.21.15.37:0/4057921308 >> v2:172.21.15.203:3300/0 conn(0x7f975815ecb0 msgr2=0x7f97580c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975d535640 1 --2- 172.21.15.37:0/4057921308 >> v2:172.21.15.203:3300/0 conn(0x7f975815ecb0 0x7f97580c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975d535640 1 -- 172.21.15.37:0/4057921308 >> v1:172.21.15.203:6789/0 conn(0x7f97580cf590 legacy=0x7f97580cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.346+0000 7f975d535640 1 -- 172.21.15.37:0/4057921308 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580cd690 con 0x7f97580c9ef0 2024-04-25T08:56:33.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f975d535640 1 --2- 172.21.15.37:0/4057921308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580c9ef0 0x7f97580ce170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9750009e50 tx=0x7f975001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39d0a71efbd5ea90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f97577fe640 1 -- 172.21.15.37:0/4057921308 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97500a5700 con 0x7f97580c9ef0 2024-04-25T08:56:33.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f97577fe640 1 -- 172.21.15.37:0/4057921308 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97500a58a0 con 0x7f97580c9ef0 2024-04-25T08:56:33.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f97577fe640 1 -- 172.21.15.37:0/4057921308 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97500a5c70 con 0x7f97580c9ef0 2024-04-25T08:56:33.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f975ef98640 1 -- 172.21.15.37:0/4057921308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580c9ef0 msgr2=0x7f97580ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f975ef98640 1 --2- 172.21.15.37:0/4057921308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580c9ef0 0x7f97580ce170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9750009e50 tx=0x7f975001ca30 comp rx=0 tx=0).stop 2024-04-25T08:56:33.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f975ef98640 1 -- 172.21.15.37:0/4057921308 shutdown_connections 2024-04-25T08:56:33.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f975ef98640 1 --2- 172.21.15.37:0/4057921308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580c9ef0 0x7f97580ce170 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f975ef98640 1 --2- 172.21.15.37:0/4057921308 >> v2:172.21.15.203:3300/0 conn(0x7f975815ecb0 0x7f97580c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f975ef98640 1 -- 172.21.15.37:0/4057921308 >> 172.21.15.37:0/4057921308 conn(0x7f975806ae60 msgr2=0x7f975806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f975ef98640 1 -- 172.21.15.37:0/4057921308 shutdown_connections 2024-04-25T08:56:33.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f975ef98640 1 -- 172.21.15.37:0/4057921308 wait complete. 2024-04-25T08:56:33.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f975ef98640 1 Processor -- start 2024-04-25T08:56:33.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.350+0000 7f975ef98640 1 -- start start 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975ef98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97580c9ef0 0x7f9758098300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975ef98640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580cf590 0x7f97580ba830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975ef98640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f975815ecb0 0x7f97580bc890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975ef98640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9758167870 con 0x7f97580c9ef0 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975ef98640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97581676f0 con 0x7f975815ecb0 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975ef98640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97581679f0 con 0x7f97580cf590 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975cd34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97580c9ef0 0x7f9758098300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975cd34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97580c9ef0 0x7f9758098300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43370/0 (socket says 172.21.15.37:43370) 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975cd34640 1 -- 172.21.15.37:0/805006711 learned_addr learned my addr 172.21.15.37:0/805006711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975d535640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f975815ecb0 0x7f97580bc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f9757fff640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580cf590 0x7f97580ba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975cd34640 1 -- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580cf590 msgr2=0x7f97580ba830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975cd34640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580cf590 0x7f97580ba830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975cd34640 1 -- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f975815ecb0 msgr2=0x7f97580bc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975cd34640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f975815ecb0 0x7f97580bc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975cd34640 1 -- 172.21.15.37:0/805006711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580bad70 con 0x7f97580c9ef0 2024-04-25T08:56:33.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975cd34640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97580c9ef0 0x7f9758098300 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f974c00c3c0 tx=0x7f974c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f97557fa640 1 -- 172.21.15.37:0/805006711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c017070 con 0x7f97580c9ef0 2024-04-25T08:56:33.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f97557fa640 1 -- 172.21.15.37:0/805006711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c0052d0 con 0x7f97580c9ef0 2024-04-25T08:56:33.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975ef98640 1 -- 172.21.15.37:0/805006711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97580bb010 con 0x7f97580c9ef0 2024-04-25T08:56:33.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f97557fa640 1 -- 172.21.15.37:0/805006711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c003840 con 0x7f97580c9ef0 2024-04-25T08:56:33.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f975ef98640 1 -- 172.21.15.37:0/805006711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97580bd580 con 0x7f97580c9ef0 2024-04-25T08:56:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f97557fa640 1 -- 172.21.15.37:0/805006711 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f974c01a070 con 0x7f97580c9ef0 2024-04-25T08:56:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f97557fa640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9744066f90 0x7f9744069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f97557fa640 1 -- 172.21.15.37:0/805006711 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f974c084500 con 0x7f97580c9ef0 2024-04-25T08:56:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.354+0000 7f9757fff640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9744066f90 0x7f9744069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 Processor -- start 2024-04-25T08:56:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- start start 2024-04-25T08:56:33.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9757fff640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9744066f90 0x7f9744069450 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f97580734b0 tx=0x7f9748008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 0x55f018af3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af4520 0x55f018aa9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018aaa2b0 0x55f018aaa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0188f4ff0 con 0x55f018af4520 2024-04-25T08:56:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01887d150 con 0x55f018aaa2b0 2024-04-25T08:56:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0188f1090 con 0x55f018af3c00 2024-04-25T08:56:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9757fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af4520 0x55f018aa9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9757fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af4520 0x55f018aa9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43384/0 (socket says 172.21.15.37:43384) 2024-04-25T08:56:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9757fff640 1 -- 172.21.15.37:0/2306500497 learned_addr learned my addr 172.21.15.37:0/2306500497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f975cd34640 1 --2- 172.21.15.37:0/2306500497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 0x55f018af3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f975d535640 1 --2- 172.21.15.37:0/2306500497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018aaa2b0 0x55f018aaa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9757fff640 1 -- 172.21.15.37:0/2306500497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 msgr2=0x55f018af3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9757fff640 1 --2- 172.21.15.37:0/2306500497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 0x55f018af3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9757fff640 1 -- 172.21.15.37:0/2306500497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018aaa2b0 msgr2=0x55f018aaa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9757fff640 1 --2- 172.21.15.37:0/2306500497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018aaa2b0 0x55f018aaa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9757fff640 1 -- 172.21.15.37:0/2306500497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f018a8b0d0 con 0x55f018af4520 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f975d535640 1 --2- 172.21.15.37:0/2306500497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018aaa2b0 0x55f018aaa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9757fff640 1 --2- 172.21.15.37:0/2306500497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af4520 0x55f018aa9d70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9748011d20 tx=0x7f9748014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36954316a87ae4f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f973e7fc640 1 -- 172.21.15.37:0/2306500497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748011d50 con 0x55f018af4520 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f973e7fc640 1 -- 172.21.15.37:0/2306500497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9748026070 con 0x55f018af4520 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f973e7fc640 1 -- 172.21.15.37:0/2306500497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748021660 con 0x55f018af4520 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- 172.21.15.37:0/2306500497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af4520 msgr2=0x55f018aa9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 --2- 172.21.15.37:0/2306500497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af4520 0x55f018aa9d70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9748011d20 tx=0x7f9748014000 comp rx=0 tx=0).stop 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- 172.21.15.37:0/2306500497 shutdown_connections 2024-04-25T08:56:33.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 --2- 172.21.15.37:0/2306500497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018aaa2b0 0x55f018aaa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 --2- 172.21.15.37:0/2306500497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af4520 0x55f018aa9d70 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 --2- 172.21.15.37:0/2306500497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 0x55f018af3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- 172.21.15.37:0/2306500497 >> 172.21.15.37:0/2306500497 conn(0x55f018aa4020 msgr2=0x55f018aa6440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- 172.21.15.37:0/2306500497 shutdown_connections 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- 172.21.15.37:0/2306500497 wait complete. 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- start start 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 Processor -- start 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018aaa2b0 0x55f018af3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af3c00 0x55f018ab2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018ab28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0188b9250 con 0x55f018af3c00 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0188af690 con 0x55f018af4520 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.358+0000 7f9762105000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f01887da40 con 0x55f018aaa2b0 2024-04-25T08:56:33.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 10 client.0 _ll_register_callbacks cb 0x55f018a92e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:33.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9757fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af3c00 0x55f018ab2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9757fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af3c00 0x55f018ab2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43392/0 (socket says 172.21.15.37:43392) 2024-04-25T08:56:33.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9757fff640 1 -- 172.21.15.37:0/1813784959 learned_addr learned my addr 172.21.15.37:0/1813784959 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f975cd34640 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018aaa2b0 0x55f018af3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f975d535640 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018ab28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9757fff640 1 -- 172.21.15.37:0/1813784959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018aaa2b0 msgr2=0x55f018af3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9757fff640 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018aaa2b0 0x55f018af3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9757fff640 1 -- 172.21.15.37:0/1813784959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 msgr2=0x55f018ab28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9757fff640 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018ab28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9757fff640 1 -- 172.21.15.37:0/1813784959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0189a21c0 con 0x55f018af3c00 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9757fff640 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af3c00 0x55f018ab2360 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9748008010 tx=0x7f97480046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565643a99990aaa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f97377fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f97377fe640 1 -- 172.21.15.37:0/1813784959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748021b10 con 0x55f018af3c00 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f97377fe640 1 -- 172.21.15.37:0/1813784959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f9748021cb0 con 0x55f018af3c00 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 -- 172.21.15.37:0/1813784959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9748026070 con 0x55f018af3c00 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f97377fe640 1 -- 172.21.15.37:0/1813784959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f974803e050 con 0x55f018af3c00 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 10 client.6702 fetch_fsmap learned FSMap version 35 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 -- 172.21.15.37:0/1813784959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55f01899e170 con 0x55f018af3c00 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f97377fe640 1 -- 172.21.15.37:0/1813784959 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9748026070 con 0x55f018af3c00 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f97377fe640 1 -- 172.21.15.37:0/1813784959 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9748041020 con 0x55f018af3c00 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 10 client.6702 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 10 client.6702 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 20 client.6702 populate_metadata read hostname 'smithi037' 2024-04-25T08:56:33.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55f018ab3580 0x55f018ab3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 4 client.6702 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 -- 172.21.15.37:0/1813784959 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f018890c60 con 0x55f018ab3580 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f975cd34640 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55f018ab3580 0x55f018ab3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f975cd34640 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55f018ab3580 0x55f018ab3960 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79c30c2d019e5126 server_cookie=125a640c0f5363f2 in_seq=0 out_seq=0 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f97377fe640 10 client.6702 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f97377fe640 1 -- 172.21.15.37:0/1813784959 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f018890c60 con 0x55f018ab3580 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f97377fe640 10 client.6702 handle_command_reply: tid=0 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 client.6702 shutdown 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 -- 172.21.15.37:0/1813784959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af3c00 msgr2=0x55f018ab2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af3c00 0x55f018ab2360 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f9748008010 tx=0x7f97480046d0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 -- 172.21.15.37:0/1813784959 shutdown_connections 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55f018ab3580 0x55f018ab3960 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018ab28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018af3c00 0x55f018ab2360 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f97377fe640 0 client.6702 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 --2- 172.21.15.37:0/1813784959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018aaa2b0 0x55f018af3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 -- 172.21.15.37:0/1813784959 >> 172.21.15.37:0/1813784959 conn(0x55f018aa4020 msgr2=0x55f018aa6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 -- 172.21.15.37:0/1813784959 shutdown_connections 2024-04-25T08:56:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 1 -- 172.21.15.37:0/1813784959 wait complete. 2024-04-25T08:56:33.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.362+0000 7f9762105000 20 client.6702 trim_cache size 0 max 16384 2024-04-25T08:56:33.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f9762105000 1 Processor -- start 2024-04-25T08:56:33.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f9762105000 1 -- start start 2024-04-25T08:56:33.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f9762105000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018ac19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f9762105000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 0x55f018ac1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f9762105000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018b082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f9762105000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97581ff360 con 0x55f018aaa2b0 2024-04-25T08:56:33.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f9762105000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97580cc1b0 con 0x55f018af4520 2024-04-25T08:56:33.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f9762105000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97581599a0 con 0x55f018af3c00 2024-04-25T08:56:33.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f975cd34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018ac19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f975cd34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018ac19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43396/0 (socket says 172.21.15.37:43396) 2024-04-25T08:56:33.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f975cd34640 1 -- 172.21.15.37:0/1437893186 learned_addr learned my addr 172.21.15.37:0/1437893186 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f975cd34640 1 -- 172.21.15.37:0/1437893186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 msgr2=0x55f018ac1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f975cd34640 1 --2- 172.21.15.37:0/1437893186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 0x55f018ac1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f975d535640 1 --2- 172.21.15.37:0/1437893186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018b082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f975cd34640 1 -- 172.21.15.37:0/1437893186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 msgr2=0x55f018b082c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f975cd34640 1 --2- 172.21.15.37:0/1437893186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018b082c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f975cd34640 1 -- 172.21.15.37:0/1437893186 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f01893bde0 con 0x55f018aaa2b0 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f975d535640 1 --2- 172.21.15.37:0/1437893186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018b082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f975cd34640 1 --2- 172.21.15.37:0/1437893186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018ac19d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f974c00cd80 tx=0x7f974c057040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591a2f43a7c2d7a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f973dffb640 1 -- 172.21.15.37:0/1437893186 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c05d320 con 0x55f018aaa2b0 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f973dffb640 1 -- 172.21.15.37:0/1437893186 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c052cb0 con 0x55f018aaa2b0 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.378+0000 7f973dffb640 1 -- 172.21.15.37:0/1437893186 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c051280 con 0x55f018aaa2b0 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- 172.21.15.37:0/1437893186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 msgr2=0x55f018ac19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 --2- 172.21.15.37:0/1437893186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018ac19d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f974c00cd80 tx=0x7f974c057040 comp rx=0 tx=0).stop 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- 172.21.15.37:0/1437893186 shutdown_connections 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 --2- 172.21.15.37:0/1437893186 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018b082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 --2- 172.21.15.37:0/1437893186 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 0x55f018ac1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 --2- 172.21.15.37:0/1437893186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018ac19d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- 172.21.15.37:0/1437893186 >> 172.21.15.37:0/1437893186 conn(0x55f018aa4020 msgr2=0x55f018aa6ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- 172.21.15.37:0/1437893186 shutdown_connections 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- 172.21.15.37:0/1437893186 wait complete. 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- start start 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 Processor -- start 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018b563c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 0x55f018b56900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018b11410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f01888fe00 con 0x55f018aaa2b0 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9758159f70 con 0x55f018af4520 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0188f4300 con 0x55f018af3c00 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 10 client.0 _ll_register_callbacks cb 0x55f018a73040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f975cd34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018b563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f975cd34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018b563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43398/0 (socket says 172.21.15.37:43398) 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f975cd34640 1 -- 172.21.15.37:0/3389507012 learned_addr learned my addr 172.21.15.37:0/3389507012 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9757fff640 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 0x55f018b56900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f975d535640 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018b11410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f975cd34640 1 -- 172.21.15.37:0/3389507012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 msgr2=0x55f018b56900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f975cd34640 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 0x55f018b56900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f975cd34640 1 -- 172.21.15.37:0/3389507012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 msgr2=0x55f018b11410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f975cd34640 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018b11410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f975cd34640 1 -- 172.21.15.37:0/3389507012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0188968e0 con 0x55f018aaa2b0 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f975cd34640 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018b563c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f974c084010 tx=0x7f974c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdd0a8b924b7547 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f97377fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f97377fe640 1 -- 172.21.15.37:0/3389507012 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974c07a060 con 0x55f018aaa2b0 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f97377fe640 1 -- 172.21.15.37:0/3389507012 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f974c05ce90 con 0x55f018aaa2b0 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- 172.21.15.37:0/3389507012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f974c052cb0 con 0x55f018aaa2b0 2024-04-25T08:56:33.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f97377fe640 1 -- 172.21.15.37:0/3389507012 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f974c072070 con 0x55f018aaa2b0 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 10 client.6708 fetch_fsmap learned FSMap version 35 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- 172.21.15.37:0/3389507012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x55f01894e8f0 con 0x55f018aaa2b0 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f97377fe640 1 -- 172.21.15.37:0/3389507012 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f974c052cb0 con 0x55f018aaa2b0 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f97377fe640 1 -- 172.21.15.37:0/3389507012 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f974c0840b0 con 0x55f018aaa2b0 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 10 client.6708 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 10 client.6708 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 20 client.6708 populate_metadata read hostname 'smithi037' 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55f018b11e00 0x55f018b12210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 4 client.6708 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9762105000 1 -- 172.21.15.37:0/3389507012 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f018902530 con 0x55f018b11e00 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9757fff640 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55f018b11e00 0x55f018b12210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f97377fe640 10 client.6708 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.382+0000 7f9757fff640 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55f018b11e00 0x55f018b12210 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d56bad04caacfdda server_cookie=cca4d59f00f97f5d in_seq=0 out_seq=0 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f97377fe640 1 -- 172.21.15.37:0/3389507012 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f018902530 con 0x55f018b11e00 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f97377fe640 10 client.6708 handle_command_reply: tid=0 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 1 client.6708 shutdown 2024-04-25T08:56:33.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 1 -- 172.21.15.37:0/3389507012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 msgr2=0x55f018b563c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018b563c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f974c084010 tx=0x7f974c000ee0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 1 -- 172.21.15.37:0/3389507012 shutdown_connections 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55f018b11e00 0x55f018b12210 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f018af4520 0x55f018b11410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f018af3c00 0x55f018b56900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f97377fe640 0 client.6708 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 1 --2- 172.21.15.37:0/3389507012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f018aaa2b0 0x55f018b563c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 1 -- 172.21.15.37:0/3389507012 >> 172.21.15.37:0/3389507012 conn(0x55f018aa4020 msgr2=0x55f018b00960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 1 -- 172.21.15.37:0/3389507012 shutdown_connections 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 1 -- 172.21.15.37:0/3389507012 wait complete. 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f9762105000 20 client.6708 trim_cache size 0 max 16384 2024-04-25T08:56:33.384 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:56:33.385 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-04-25T08:56:33.385 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-04-25T08:56:33.385 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:56:33.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 -- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9744066f90 msgr2=0x7f9744069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9744066f90 0x7f9744069450 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f97580734b0 tx=0x7f9748008040 comp rx=0 tx=0).stop 2024-04-25T08:56:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 -- 172.21.15.37:0/805006711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97580c9ef0 msgr2=0x7f9758098300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97580c9ef0 0x7f9758098300 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f974c00c3c0 tx=0x7f974c00c880 comp rx=0 tx=0).stop 2024-04-25T08:56:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 -- 172.21.15.37:0/805006711 shutdown_connections 2024-04-25T08:56:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9744066f90 0x7f9744069450 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f975815ecb0 0x7f97580bc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97580cf590 0x7f97580ba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 --2- 172.21.15.37:0/805006711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97580c9ef0 0x7f9758098300 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 -- 172.21.15.37:0/805006711 >> 172.21.15.37:0/805006711 conn(0x7f975806ae60 msgr2=0x7f9758157770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 -- 172.21.15.37:0/805006711 shutdown_connections 2024-04-25T08:56:33.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.386+0000 7f975ef98640 1 -- 172.21.15.37:0/805006711 wait complete. 2024-04-25T08:56:33.392 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:56:33.392 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:33.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.518+0000 7fe9c6857640 1 Processor -- start 2024-04-25T08:56:33.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.518+0000 7fe9c6857640 1 -- start start 2024-04-25T08:56:33.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9c6857640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015ba20 0x7fe9c00cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9c6857640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe9c015fd90 0x7fe9c006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9c6857640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c015c260 con 0x7fe9c006b1d0 2024-04-25T08:56:33.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9c6857640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c006b640 con 0x7fe9c015fd90 2024-04-25T08:56:33.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9c6857640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c00d1430 con 0x7fe9c015ba20 2024-04-25T08:56:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9bffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015ba20 0x7fe9c00cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9bffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015ba20 0x7fe9c00cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:34402/0 (socket says 172.21.15.37:34402) 2024-04-25T08:56:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9bffff640 1 -- 172.21.15.37:0/2053574983 learned_addr learned my addr 172.21.15.37:0/2053574983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9b77fe640 1 --2- 172.21.15.37:0/2053574983 >> v2:172.21.15.203:3300/0 conn(0x7fe9c015fd90 0x7fe9c006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9bffff640 1 -- 172.21.15.37:0/2053574983 >> v2:172.21.15.203:3300/0 conn(0x7fe9c015fd90 msgr2=0x7fe9c006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9bffff640 1 --2- 172.21.15.37:0/2053574983 >> v2:172.21.15.203:3300/0 conn(0x7fe9c015fd90 0x7fe9c006ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9bffff640 1 -- 172.21.15.37:0/2053574983 >> v1:172.21.15.203:6789/0 conn(0x7fe9c006b1d0 legacy=0x7fe9c00cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.522+0000 7fe9bffff640 1 -- 172.21.15.37:0/2053574983 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c015c540 con 0x7fe9c015ba20 2024-04-25T08:56:33.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9bffff640 1 --2- 172.21.15.37:0/2053574983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015ba20 0x7fe9c00cc1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009e50 tx=0x7fe9b001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c7c6cf7e7c4c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9bf7fe640 1 -- 172.21.15.37:0/2053574983 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00a5700 con 0x7fe9c015ba20 2024-04-25T08:56:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9bf7fe640 1 -- 172.21.15.37:0/2053574983 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b00a58a0 con 0x7fe9c015ba20 2024-04-25T08:56:33.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9bf7fe640 1 -- 172.21.15.37:0/2053574983 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b00a5c70 con 0x7fe9c015ba20 2024-04-25T08:56:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 -- 172.21.15.37:0/2053574983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015ba20 msgr2=0x7fe9c00cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 --2- 172.21.15.37:0/2053574983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015ba20 0x7fe9c00cc1a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0009e50 tx=0x7fe9b001ca30 comp rx=0 tx=0).stop 2024-04-25T08:56:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 -- 172.21.15.37:0/2053574983 shutdown_connections 2024-04-25T08:56:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 --2- 172.21.15.37:0/2053574983 >> v2:172.21.15.203:3300/0 conn(0x7fe9c015fd90 0x7fe9c006ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 --2- 172.21.15.37:0/2053574983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015ba20 0x7fe9c00cc1a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 -- 172.21.15.37:0/2053574983 >> 172.21.15.37:0/2053574983 conn(0x7fe9c0099a70 msgr2=0x7fe9c00d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 -- 172.21.15.37:0/2053574983 shutdown_connections 2024-04-25T08:56:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 -- 172.21.15.37:0/2053574983 wait complete. 2024-04-25T08:56:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 Processor -- start 2024-04-25T08:56:33.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 -- start start 2024-04-25T08:56:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c006b1d0 0x7fe9c00c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c015ba20 0x7fe9c00b5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015fd90 0x7fe9c0152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c01602f0 con 0x7fe9c015ba20 2024-04-25T08:56:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0160170 con 0x7fe9c006b1d0 2024-04-25T08:56:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c6857640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c015be00 con 0x7fe9c015fd90 2024-04-25T08:56:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c015ba20 0x7fe9c00b5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c015ba20 0x7fe9c00b5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43410/0 (socket says 172.21.15.37:43410) 2024-04-25T08:56:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9b77fe640 1 -- 172.21.15.37:0/2893294096 learned_addr learned my addr 172.21.15.37:0/2893294096 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9bffff640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c006b1d0 0x7fe9c00c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9c4df4640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015fd90 0x7fe9c0152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9b77fe640 1 -- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015fd90 msgr2=0x7fe9c0152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9b77fe640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015fd90 0x7fe9c0152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9b77fe640 1 -- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c006b1d0 msgr2=0x7fe9c00c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9b77fe640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c006b1d0 0x7fe9c00c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.526+0000 7fe9b77fe640 1 -- 172.21.15.37:0/2893294096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00b5f70 con 0x7fe9c015ba20 2024-04-25T08:56:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9b77fe640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c015ba20 0x7fe9c00b5a30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800e870 tx=0x7fe9a800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9bd7fa640 1 -- 172.21.15.37:0/2893294096 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8017070 con 0x7fe9c015ba20 2024-04-25T08:56:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9bd7fa640 1 -- 172.21.15.37:0/2893294096 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a800c940 con 0x7fe9c015ba20 2024-04-25T08:56:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9c6857640 1 -- 172.21.15.37:0/2893294096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c0152840 con 0x7fe9c015ba20 2024-04-25T08:56:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9bd7fa640 1 -- 172.21.15.37:0/2893294096 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a800ccf0 con 0x7fe9c015ba20 2024-04-25T08:56:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9c6857640 1 -- 172.21.15.37:0/2893294096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c00c9ed0 con 0x7fe9c015ba20 2024-04-25T08:56:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9c6857640 1 -- 172.21.15.37:0/2893294096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c004efa0 con 0x7fe9c015ba20 2024-04-25T08:56:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9bd7fa640 1 -- 172.21.15.37:0/2893294096 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9a801a070 con 0x7fe9c015ba20 2024-04-25T08:56:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9bd7fa640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9a0066f90 0x7fe9a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9bd7fa640 1 -- 172.21.15.37:0/2893294096 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fe9a80850b0 con 0x7fe9c015ba20 2024-04-25T08:56:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9bffff640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9a0066f90 0x7fe9a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.530+0000 7fe9bffff640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9a0066f90 0x7fe9a0069450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00ae000 tx=0x7fe9b00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.534+0000 7fe9bd7fa640 1 -- 172.21.15.37:0/2893294096 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9a8055f50 con 0x7fe9c015ba20 2024-04-25T08:56:33.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.726+0000 7fe9c6857640 1 -- 172.21.15.37:0/2893294096 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9c00b6480 con 0x7fe9c015ba20 2024-04-25T08:56:33.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9bd7fa640 1 -- 172.21.15.37:0/2893294096 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fe9a8055850 con 0x7fe9c015ba20 2024-04-25T08:56:33.719 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:33.719 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/486740526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":486740526},{"type":"v1","addr":"172.21.15.37:6833","nonce":486740526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/121484331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":121484331},{"type":"v1","addr":"172.21.15.203:6835","nonce":121484331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2501630566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2501630566},{"type":"v1","addr":"172.21.15.203:6837","nonce":2501630566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:55:46.200939+0000","modified":"2024-04-25T08:56:06.871679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6405":{"gid":6405,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":8,"addr":"172.21.15.37:6835/2189232625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2189232625},{"type":"v1","addr":"172.21.15.37:6835","nonce":2189232625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:56:33.719 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-04-25T08:56:33.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 -- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9a0066f90 msgr2=0x7fe9a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9a0066f90 0x7fe9a0069450 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00ae000 tx=0x7fe9b00a7070 comp rx=0 tx=0).stop 2024-04-25T08:56:33.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 -- 172.21.15.37:0/2893294096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c015ba20 msgr2=0x7fe9c00b5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c015ba20 0x7fe9c00b5a30 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800e870 tx=0x7fe9a800ec70 comp rx=0 tx=0).stop 2024-04-25T08:56:33.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 -- 172.21.15.37:0/2893294096 shutdown_connections 2024-04-25T08:56:33.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9a0066f90 0x7fe9a0069450 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c015fd90 0x7fe9c0152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c015ba20 0x7fe9c00b5a30 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 --2- 172.21.15.37:0/2893294096 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c006b1d0 0x7fe9c00c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 -- 172.21.15.37:0/2893294096 >> 172.21.15.37:0/2893294096 conn(0x7fe9c0099a70 msgr2=0x7fe9c015d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 -- 172.21.15.37:0/2893294096 shutdown_connections 2024-04-25T08:56:33.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.730+0000 7fe9c6857640 1 -- 172.21.15.37:0/2893294096 wait complete. 2024-04-25T08:56:33.736 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T08:56:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.862+0000 7fba70ca7640 1 Processor -- start 2024-04-25T08:56:33.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.862+0000 7fba70ca7640 1 -- start start 2024-04-25T08:56:33.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba70ca7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fba6c0d0130 0x7fba6c0cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba70ca7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba6c0d1fe0 0x7fba6c161c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba70ca7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba6c0d2530 con 0x7fba6c0d0510 2024-04-25T08:56:33.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba70ca7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba6c0d2810 con 0x7fba6c0d0130 2024-04-25T08:56:33.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba70ca7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c0b3160 con 0x7fba6c0d1fe0 2024-04-25T08:56:33.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba6a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fba6c0d0130 0x7fba6c0cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba6a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fba6c0d0130 0x7fba6c0cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43424/0 (socket says 172.21.15.37:43424) 2024-04-25T08:56:33.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba6a59c640 1 -- 172.21.15.37:0/1935098328 learned_addr learned my addr 172.21.15.37:0/1935098328 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba69d9b640 1 --2- 172.21.15.37:0/1935098328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba6c0d1fe0 0x7fba6c161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba6a59c640 1 -- 172.21.15.37:0/1935098328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba6c0d1fe0 msgr2=0x7fba6c161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba6a59c640 1 --2- 172.21.15.37:0/1935098328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba6c0d1fe0 0x7fba6c161c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba6a59c640 1 -- 172.21.15.37:0/1935098328 >> v1:172.21.15.203:6789/0 conn(0x7fba6c0d0510 legacy=0x7fba6c162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.866+0000 7fba6a59c640 1 -- 172.21.15.37:0/1935098328 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c0b3440 con 0x7fba6c0d0130 2024-04-25T08:56:33.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6a59c640 1 --2- 172.21.15.37:0/1935098328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba6c0d0130 0x7fba6c0cf4c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fba64009870 tx=0x7fba6401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c6f89c7abe9cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6959a640 1 -- 172.21.15.37:0/1935098328 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640b0070 con 0x7fba6c0d0130 2024-04-25T08:56:33.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6959a640 1 -- 172.21.15.37:0/1935098328 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba640ab600 con 0x7fba6c0d0130 2024-04-25T08:56:33.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6959a640 1 -- 172.21.15.37:0/1935098328 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640ab9d0 con 0x7fba6c0d0130 2024-04-25T08:56:33.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 -- 172.21.15.37:0/1935098328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba6c0d0130 msgr2=0x7fba6c0cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 --2- 172.21.15.37:0/1935098328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba6c0d0130 0x7fba6c0cf4c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fba64009870 tx=0x7fba6401c520 comp rx=0 tx=0).stop 2024-04-25T08:56:33.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 -- 172.21.15.37:0/1935098328 shutdown_connections 2024-04-25T08:56:33.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 --2- 172.21.15.37:0/1935098328 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba6c0d1fe0 0x7fba6c161c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 --2- 172.21.15.37:0/1935098328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba6c0d0130 0x7fba6c0cf4c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 -- 172.21.15.37:0/1935098328 >> 172.21.15.37:0/1935098328 conn(0x7fba6c08b000 msgr2=0x7fba6c0d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 -- 172.21.15.37:0/1935098328 shutdown_connections 2024-04-25T08:56:33.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 -- 172.21.15.37:0/1935098328 wait complete. 2024-04-25T08:56:33.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 Processor -- start 2024-04-25T08:56:33.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 -- start start 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba6c0d0130 0x7fba6c0cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba6c0d0510 0x7fba6c157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba6c0d1fe0 0x7fba6c0c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c0b4450 con 0x7fba6c0d0130 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c162a80 con 0x7fba6c0d0510 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba70ca7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c0b45d0 con 0x7fba6c0d1fe0 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba6c0d0130 0x7fba6c0cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba6c0d0130 0x7fba6c0cb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43428/0 (socket says 172.21.15.37:43428) 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6a59c640 1 -- 172.21.15.37:0/4017354323 learned_addr learned my addr 172.21.15.37:0/4017354323 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba69d9b640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba6c0d0510 0x7fba6c157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6ad9d640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba6c0d1fe0 0x7fba6c0c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6a59c640 1 -- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba6c0d1fe0 msgr2=0x7fba6c0c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6a59c640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba6c0d1fe0 0x7fba6c0c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6a59c640 1 -- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba6c0d0510 msgr2=0x7fba6c157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6a59c640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba6c0d0510 0x7fba6c157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba6a59c640 1 -- 172.21.15.37:0/4017354323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c0b7b90 con 0x7fba6c0d0130 2024-04-25T08:56:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.870+0000 7fba69d9b640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba6c0d0510 0x7fba6c157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba6a59c640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba6c0d0130 0x7fba6c0cb1b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fba640ae040 tx=0x7fba6401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba62ffd640 1 -- 172.21.15.37:0/4017354323 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640b0040 con 0x7fba6c0d0130 2024-04-25T08:56:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba62ffd640 1 -- 172.21.15.37:0/4017354323 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba640bc070 con 0x7fba6c0d0130 2024-04-25T08:56:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba62ffd640 1 -- 172.21.15.37:0/4017354323 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640b6880 con 0x7fba6c0d0130 2024-04-25T08:56:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba70ca7640 1 -- 172.21.15.37:0/4017354323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba6c157a40 con 0x7fba6c0d0130 2024-04-25T08:56:33.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba70ca7640 1 -- 172.21.15.37:0/4017354323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba6c0cba70 con 0x7fba6c0d0130 2024-04-25T08:56:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba62ffd640 1 -- 172.21.15.37:0/4017354323 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba640ab4c0 con 0x7fba6c0d0130 2024-04-25T08:56:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba62ffd640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba4c066f60 0x7fba4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba62ffd640 1 -- 172.21.15.37:0/4017354323 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fba6411f5e0 con 0x7fba6c0d0130 2024-04-25T08:56:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba69d9b640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba4c066f60 0x7fba4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba73e14000 1 Processor -- start 2024-04-25T08:56:33.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba73e14000 1 -- start start 2024-04-25T08:56:33.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba69d9b640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba4c066f60 0x7fba4c069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0c8f10 tx=0x7fba5c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba73e14000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617f2e0 0x55922617f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba73e14000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x5592261353a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba73e14000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226135cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba73e14000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55922601cba0 con 0x5592261358e0 2024-04-25T08:56:33.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba73e14000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559225f82090 con 0x55922617fc00 2024-04-25T08:56:33.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba73e14000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559225f0a160 con 0x55922617f2e0 2024-04-25T08:56:33.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba6ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226135cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba6ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226135cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43438/0 (socket says 172.21.15.37:43438) 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba6ad9d640 1 -- 172.21.15.37:0/365136238 learned_addr learned my addr 172.21.15.37:0/365136238 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba6a59c640 1 --2- 172.21.15.37:0/365136238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617f2e0 0x55922617f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.874+0000 7fba69d9b640 1 --2- 172.21.15.37:0/365136238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x5592261353a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6ad9d640 1 -- 172.21.15.37:0/365136238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617f2e0 msgr2=0x55922617f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6ad9d640 1 --2- 172.21.15.37:0/365136238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617f2e0 0x55922617f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6ad9d640 1 -- 172.21.15.37:0/365136238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 msgr2=0x5592261353a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6ad9d640 1 --2- 172.21.15.37:0/365136238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x5592261353a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6ad9d640 1 -- 172.21.15.37:0/365136238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55922602bc20 con 0x5592261358e0 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6ad9d640 1 --2- 172.21.15.37:0/365136238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226135cc0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fba54004d20 tx=0x7fba540051e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3f16d7d0ed8424 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba4bfff640 1 -- 172.21.15.37:0/365136238 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba540067e0 con 0x5592261358e0 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba4bfff640 1 -- 172.21.15.37:0/365136238 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba54018070 con 0x5592261358e0 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba4bfff640 1 -- 172.21.15.37:0/365136238 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54006bc0 con 0x5592261358e0 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- 172.21.15.37:0/365136238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 msgr2=0x559226135cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 --2- 172.21.15.37:0/365136238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226135cc0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fba54004d20 tx=0x7fba540051e0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- 172.21.15.37:0/365136238 shutdown_connections 2024-04-25T08:56:33.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 --2- 172.21.15.37:0/365136238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226135cc0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 --2- 172.21.15.37:0/365136238 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x5592261353a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 --2- 172.21.15.37:0/365136238 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617f2e0 0x55922617f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- 172.21.15.37:0/365136238 >> 172.21.15.37:0/365136238 conn(0x559226131890 msgr2=0x559226133c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- 172.21.15.37:0/365136238 shutdown_connections 2024-04-25T08:56:33.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- 172.21.15.37:0/365136238 wait complete. 2024-04-25T08:56:33.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- start start 2024-04-25T08:56:33.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 Processor -- start 2024-04-25T08:56:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226186640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617f2e0 0x559226186b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x55922616d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559225f7e130 con 0x5592261358e0 2024-04-25T08:56:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559225f46130 con 0x55922617fc00 2024-04-25T08:56:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559226040300 con 0x55922617f2e0 2024-04-25T08:56:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 10 client.0 _ll_register_callbacks cb 0x559226120bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226186640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226186640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43446/0 (socket says 172.21.15.37:43446) 2024-04-25T08:56:33.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6a59c640 1 -- 172.21.15.37:0/2058630308 learned_addr learned my addr 172.21.15.37:0/2058630308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba69d9b640 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617f2e0 0x559226186b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6ad9d640 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x55922616d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6a59c640 1 -- 172.21.15.37:0/2058630308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617f2e0 msgr2=0x559226186b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6a59c640 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617f2e0 0x559226186b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6a59c640 1 -- 172.21.15.37:0/2058630308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 msgr2=0x55922616d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6a59c640 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x55922616d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6a59c640 1 -- 172.21.15.37:0/2058630308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559225f5f060 con 0x5592261358e0 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6ad9d640 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x55922616d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba69d9b640 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617f2e0 0x559226186b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba6a59c640 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226186640 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fba640ef990 tx=0x7fba640ef5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596d9fa87264c289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba497fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba497fa640 1 -- 172.21.15.37:0/2058630308 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640f8110 con 0x5592261358e0 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba497fa640 1 -- 172.21.15.37:0/2058630308 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fba6411f0b0 con 0x5592261358e0 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- 172.21.15.37:0/2058630308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba54018070 con 0x5592261358e0 2024-04-25T08:56:33.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba497fa640 1 -- 172.21.15.37:0/2058630308 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba640efc90 con 0x5592261358e0 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 10 client.6723 fetch_fsmap learned FSMap version 35 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- 172.21.15.37:0/2058630308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x559225f237c0 con 0x5592261358e0 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba497fa640 1 -- 172.21.15.37:0/2058630308 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba54018070 con 0x5592261358e0 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba497fa640 1 -- 172.21.15.37:0/2058630308 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fba640b3030 con 0x5592261358e0 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 10 client.6723 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 10 client.6723 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 20 client.6723 populate_metadata read hostname 'smithi037' 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55922616de30 0x55922616e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 4 client.6723 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.878+0000 7fba73e14000 1 -- 172.21.15.37:0/2058630308 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559225f1dcd0 con 0x55922616de30 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba69d9b640 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55922616de30 0x55922616e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba497fa640 10 client.6723 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba69d9b640 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55922616de30 0x55922616e210 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb3da97aadc7a33 server_cookie=ca20903b973b8385 in_seq=0 out_seq=0 2024-04-25T08:56:33.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba497fa640 1 -- 172.21.15.37:0/2058630308 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559225f1dcd0 con 0x55922616de30 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba497fa640 10 client.6723 handle_command_reply: tid=0 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 1 client.6723 shutdown 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 1 -- 172.21.15.37:0/2058630308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 msgr2=0x559226186640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226186640 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fba640ef990 tx=0x7fba640ef5f0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 1 -- 172.21.15.37:0/2058630308 shutdown_connections 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x55922616de30 0x55922616e210 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x55922616d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617f2e0 0x559226186b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 1 --2- 172.21.15.37:0/2058630308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592261358e0 0x559226186640 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 1 -- 172.21.15.37:0/2058630308 >> 172.21.15.37:0/2058630308 conn(0x559226131890 msgr2=0x559226134510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba497fa640 0 client.6723 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 1 -- 172.21.15.37:0/2058630308 shutdown_connections 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 1 -- 172.21.15.37:0/2058630308 wait complete. 2024-04-25T08:56:33.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.882+0000 7fba73e14000 20 client.6723 trim_cache size 0 max 16384 2024-04-25T08:56:33.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 Processor -- start 2024-04-25T08:56:33.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 -- start start 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592261358e0 0x5592261db500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261dba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617fc00 0x5592261dbf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c2022b0 con 0x55922617f2e0 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c0b8ec0 con 0x5592261358e0 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba6c201dc0 con 0x55922617fc00 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba69d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261dba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba69d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261dba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43448/0 (socket says 172.21.15.37:43448) 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba69d9b640 1 -- 172.21.15.37:0/3437578593 learned_addr learned my addr 172.21.15.37:0/3437578593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba69d9b640 1 -- 172.21.15.37:0/3437578593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617fc00 msgr2=0x5592261dbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba69d9b640 1 --2- 172.21.15.37:0/3437578593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617fc00 0x5592261dbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba69d9b640 1 -- 172.21.15.37:0/3437578593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592261358e0 msgr2=0x5592261db500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba69d9b640 1 --2- 172.21.15.37:0/3437578593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592261358e0 0x5592261db500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba69d9b640 1 -- 172.21.15.37:0/3437578593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559225f3dfc0 con 0x55922617f2e0 2024-04-25T08:56:33.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba69d9b640 1 --2- 172.21.15.37:0/3437578593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261dba40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0cd1e0 tx=0x7fba5c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf673c7127c9a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba4b7fe640 1 -- 172.21.15.37:0/3437578593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c025330 con 0x55922617f2e0 2024-04-25T08:56:33.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba4b7fe640 1 -- 172.21.15.37:0/3437578593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c029020 con 0x55922617f2e0 2024-04-25T08:56:33.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba4b7fe640 1 -- 172.21.15.37:0/3437578593 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0254d0 con 0x55922617f2e0 2024-04-25T08:56:33.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 -- 172.21.15.37:0/3437578593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 msgr2=0x5592261dba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 --2- 172.21.15.37:0/3437578593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261dba40 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0cd1e0 tx=0x7fba5c015000 comp rx=0 tx=0).stop 2024-04-25T08:56:33.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 -- 172.21.15.37:0/3437578593 shutdown_connections 2024-04-25T08:56:33.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 --2- 172.21.15.37:0/3437578593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55922617fc00 0x5592261dbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 --2- 172.21.15.37:0/3437578593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261dba40 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 --2- 172.21.15.37:0/3437578593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5592261358e0 0x5592261db500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 -- 172.21.15.37:0/3437578593 >> 172.21.15.37:0/3437578593 conn(0x559226131890 msgr2=0x55922617e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 -- 172.21.15.37:0/3437578593 shutdown_connections 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 -- 172.21.15.37:0/3437578593 wait complete. 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 -- start start 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.894+0000 7fba73e14000 1 Processor -- start 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592261358e0 0x5592261dff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261e04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x5592261844c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559225f1ce70 con 0x55922617f2e0 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559225f0aa50 con 0x55922617fc00 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559225f813a0 con 0x5592261358e0 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 10 client.0 _ll_register_callbacks cb 0x559225f2e240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba69d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261e04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba69d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261e04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43464/0 (socket says 172.21.15.37:43464) 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba69d9b640 1 -- 172.21.15.37:0/932356 learned_addr learned my addr 172.21.15.37:0/932356 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba6ad9d640 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x5592261844c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba6a59c640 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592261358e0 0x5592261dff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba69d9b640 1 -- 172.21.15.37:0/932356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592261358e0 msgr2=0x5592261dff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba69d9b640 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592261358e0 0x5592261dff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba69d9b640 1 -- 172.21.15.37:0/932356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 msgr2=0x5592261844c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba69d9b640 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x5592261844c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba69d9b640 1 -- 172.21.15.37:0/932356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559225fdb9e0 con 0x55922617f2e0 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba69d9b640 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261e04a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009860 tx=0x7fba5c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26de4296f8846c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba497fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba497fa640 1 -- 172.21.15.37:0/932356 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c02e430 con 0x55922617f2e0 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba497fa640 1 -- 172.21.15.37:0/932356 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fba5c02e5d0 con 0x55922617f2e0 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 -- 172.21.15.37:0/932356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba5c029020 con 0x55922617f2e0 2024-04-25T08:56:33.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba497fa640 1 -- 172.21.15.37:0/932356 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba5c03e050 con 0x55922617f2e0 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 10 client.6729 fetch_fsmap learned FSMap version 35 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 -- 172.21.15.37:0/932356 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x559225f21630 con 0x55922617f2e0 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba497fa640 1 -- 172.21.15.37:0/932356 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c029020 con 0x55922617f2e0 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba497fa640 1 -- 172.21.15.37:0/932356 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fba5c041020 con 0x55922617f2e0 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 10 client.6729 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 10 client.6729 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 20 client.6729 populate_metadata read hostname 'smithi037' 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559226184e80 0x559226185260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 4 client.6729 mds_command: new command op to 6405 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 -- 172.21.15.37:0/932356 --> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x559225f8f5d0 con 0x559226184e80 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba6a59c640 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559226184e80 0x559226185260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba6a59c640 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559226184e80 0x559226185260 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cba6f0bdcb75f0a9 server_cookie=9d5b9459d9655cf8 in_seq=0 out_seq=0 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba497fa640 10 client.6729 ms_handle_connect on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba497fa640 1 -- 172.21.15.37:0/932356 <== mds.0 v2:172.21.15.37:6834/2189232625 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559225f8f5d0 con 0x559226184e80 2024-04-25T08:56:33.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba497fa640 10 client.6729 handle_command_reply: tid=0 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 client.6729 shutdown 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 -- 172.21.15.37:0/932356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 msgr2=0x5592261e04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261e04a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fba5c009860 tx=0x7fba5c004ac0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 -- 172.21.15.37:0/932356 shutdown_connections 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.37:6834/2189232625,v1:172.21.15.37:6835/2189232625] conn(0x559226184e80 0x559226185260 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55922617fc00 0x5592261844c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55922617f2e0 0x5592261e04a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba497fa640 0 client.6729 ms_handle_reset on v2:172.21.15.37:6834/2189232625 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 --2- 172.21.15.37:0/932356 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5592261358e0 0x5592261dff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 -- 172.21.15.37:0/932356 >> 172.21.15.37:0/932356 conn(0x559226131890 msgr2=0x55922617eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 -- 172.21.15.37:0/932356 shutdown_connections 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 1 -- 172.21.15.37:0/932356 wait complete. 2024-04-25T08:56:33.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba73e14000 20 client.6729 trim_cache size 0 max 16384 2024-04-25T08:56:33.895 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:33.895 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 -- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba4c066f60 msgr2=0x7fba4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba4c066f60 0x7fba4c069420 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0c8f10 tx=0x7fba5c00b040 comp rx=0 tx=0).stop 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 -- 172.21.15.37:0/4017354323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba6c0d0130 msgr2=0x7fba6c0cb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba6c0d0130 0x7fba6c0cb1b0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fba640ae040 tx=0x7fba6401ad60 comp rx=0 tx=0).stop 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 -- 172.21.15.37:0/4017354323 shutdown_connections 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba4c066f60 0x7fba4c069420 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba6c0d1fe0 0x7fba6c0c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba6c0d0510 0x7fba6c157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 --2- 172.21.15.37:0/4017354323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba6c0d0130 0x7fba6c0cb1b0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 -- 172.21.15.37:0/4017354323 >> 172.21.15.37:0/4017354323 conn(0x7fba6c08b000 msgr2=0x7fba6c0b24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 -- 172.21.15.37:0/4017354323 shutdown_connections 2024-04-25T08:56:33.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:33.898+0000 7fba70ca7640 1 -- 172.21.15.37:0/4017354323 wait complete. 2024-04-25T08:56:33.904 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:56:34.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb58a74b640 1 Processor -- start 2024-04-25T08:56:34.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb58a74b640 1 -- start start 2024-04-25T08:56:34.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb58a74b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb584161e80 0x7fb5840ceca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb58a74b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb5840c2320 0x7fb5840c2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb58a74b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5840ce8b0 con 0x7fb58406cfc0 2024-04-25T08:56:34.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb58a74b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5840c2cc0 con 0x7fb5840c2320 2024-04-25T08:56:34.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb58a74b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840b2c10 con 0x7fb584161e80 2024-04-25T08:56:34.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb57b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb5840c2320 0x7fb5840c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb588ce8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb58406cfc0 0x7fb58406d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38832/0 (socket says 172.21.15.37:38832) 2024-04-25T08:56:34.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb588ce8640 1 -- 172.21.15.37:0/267457206 learned_addr learned my addr 172.21.15.37:0/267457206 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:34.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb583fff640 1 --2- 172.21.15.37:0/267457206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb584161e80 0x7fb5840ceca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb57b7fe640 1 -- 172.21.15.37:0/267457206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb584161e80 msgr2=0x7fb5840ceca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:34.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb57b7fe640 1 --2- 172.21.15.37:0/267457206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb584161e80 0x7fb5840ceca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb57b7fe640 1 -- 172.21.15.37:0/267457206 >> v1:172.21.15.37:6789/0 conn(0x7fb58406cfc0 legacy=0x7fb58406d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb5837fe640 1 -- 172.21.15.37:0/267457206 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187148750 0 0) 0x7fb57c0014a0 con 0x7fb58406cfc0 2024-04-25T08:56:34.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.022+0000 7fb57b7fe640 1 -- 172.21.15.37:0/267457206 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5840cd3c0 con 0x7fb5840c2320 2024-04-25T08:56:34.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb57b7fe640 1 --2- 172.21.15.37:0/267457206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5840c2320 0x7fb5840c2780 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0097c0 tx=0x7fb56c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974117c687fdd992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:34.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb5837fe640 1 -- 172.21.15.37:0/267457206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0b0070 con 0x7fb5840c2320 2024-04-25T08:56:34.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb5837fe640 1 -- 172.21.15.37:0/267457206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56c0ab600 con 0x7fb5840c2320 2024-04-25T08:56:34.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb5837fe640 1 -- 172.21.15.37:0/267457206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0ab9d0 con 0x7fb5840c2320 2024-04-25T08:56:34.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 -- 172.21.15.37:0/267457206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5840c2320 msgr2=0x7fb5840c2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 --2- 172.21.15.37:0/267457206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5840c2320 0x7fb5840c2780 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fb56c0097c0 tx=0x7fb56c01c380 comp rx=0 tx=0).stop 2024-04-25T08:56:34.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 -- 172.21.15.37:0/267457206 shutdown_connections 2024-04-25T08:56:34.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 --2- 172.21.15.37:0/267457206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5840c2320 0x7fb5840c2780 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 --2- 172.21.15.37:0/267457206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb584161e80 0x7fb5840ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 -- 172.21.15.37:0/267457206 >> 172.21.15.37:0/267457206 conn(0x7fb58406ae60 msgr2=0x7fb58406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:34.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 -- 172.21.15.37:0/267457206 shutdown_connections 2024-04-25T08:56:34.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 -- 172.21.15.37:0/267457206 wait complete. 2024-04-25T08:56:34.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 Processor -- start 2024-04-25T08:56:34.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 -- start start 2024-04-25T08:56:34.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58406cfc0 0x7fb5841647c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5840c2320 0x7fb584158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584161e80 0x7fb5841583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840ac1e0 con 0x7fb584161e80 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58406e9a0 con 0x7fb5840c2320 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5840ac360 con 0x7fb58406cfc0 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb588ce8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584161e80 0x7fb5841583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb588ce8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584161e80 0x7fb5841583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43468/0 (socket says 172.21.15.37:43468) 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb588ce8640 1 -- 172.21.15.37:0/3344508331 learned_addr learned my addr 172.21.15.37:0/3344508331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb57b7fe640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5840c2320 0x7fb584158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb583fff640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58406cfc0 0x7fb5841647c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb588ce8640 1 -- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58406cfc0 msgr2=0x7fb5841647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb588ce8640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58406cfc0 0x7fb5841647c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb588ce8640 1 -- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5840c2320 msgr2=0x7fb584158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb588ce8640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5840c2320 0x7fb584158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb588ce8640 1 -- 172.21.15.37:0/3344508331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb584159130 con 0x7fb584161e80 2024-04-25T08:56:34.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb57b7fe640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5840c2320 0x7fb584158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:34.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb588ce8640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584161e80 0x7fb5841583c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c780 tx=0x7fb57c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:34.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb5817fa640 1 -- 172.21.15.37:0/3344508331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c018070 con 0x7fb584161e80 2024-04-25T08:56:34.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb5817fa640 1 -- 172.21.15.37:0/3344508331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c00a640 con 0x7fb584161e80 2024-04-25T08:56:34.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb5817fa640 1 -- 172.21.15.37:0/3344508331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00aa30 con 0x7fb584161e80 2024-04-25T08:56:34.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 -- 172.21.15.37:0/3344508331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5841593d0 con 0x7fb584161e80 2024-04-25T08:56:34.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.026+0000 7fb58a74b640 1 -- 172.21.15.37:0/3344508331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb584159990 con 0x7fb584161e80 2024-04-25T08:56:34.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.030+0000 7fb58a74b640 1 -- 172.21.15.37:0/3344508331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58404efa0 con 0x7fb584161e80 2024-04-25T08:56:34.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.030+0000 7fb5817fa640 1 -- 172.21.15.37:0/3344508331 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb57c00abd0 con 0x7fb584161e80 2024-04-25T08:56:34.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.030+0000 7fb5817fa640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb564066f60 0x7fb564069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.030+0000 7fb5817fa640 1 -- 172.21.15.37:0/3344508331 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fb57c084770 con 0x7fb584161e80 2024-04-25T08:56:34.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.030+0000 7fb583fff640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb564066f60 0x7fb564069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.030+0000 7fb583fff640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb564066f60 0x7fb564069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb574009790 tx=0x7fb574008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:34.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.034+0000 7fb5817fa640 1 -- 172.21.15.37:0/3344508331 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb5840ce8b0 con 0x7fb584161e80 2024-04-25T08:56:34.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 -- 172.21.15.37:0/3344508331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb58406d4f0 con 0x7fb584161e80 2024-04-25T08:56:34.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb5817fa640 1 -- 172.21.15.37:0/3344508331 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb57c082e20 con 0x7fb584161e80 2024-04-25T08:56:34.219 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:34.219 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 -- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb564066f60 msgr2=0x7fb564069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb564066f60 0x7fb564069420 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb574009790 tx=0x7fb574008040 comp rx=0 tx=0).stop 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 -- 172.21.15.37:0/3344508331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584161e80 msgr2=0x7fb5841583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584161e80 0x7fb5841583c0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00c780 tx=0x7fb57c00cc40 comp rx=0 tx=0).stop 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 -- 172.21.15.37:0/3344508331 shutdown_connections 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb564066f60 0x7fb564069420 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb584161e80 0x7fb5841583c0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb5840c2320 0x7fb584158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 --2- 172.21.15.37:0/3344508331 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb58406cfc0 0x7fb5841647c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 -- 172.21.15.37:0/3344508331 >> 172.21.15.37:0/3344508331 conn(0x7fb58406ae60 msgr2=0x7fb5840b06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 -- 172.21.15.37:0/3344508331 shutdown_connections 2024-04-25T08:56:34.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.230+0000 7fb58a74b640 1 -- 172.21.15.37:0/3344508331 wait complete. 2024-04-25T08:56:34.238 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T08:56:34.238 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:56:34.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:56:34.288 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:56:34.340 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:56:34.341 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:56:34.341 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:34.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198f7d3640 1 Processor -- start 2024-04-25T08:56:34.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198f7d3640 1 -- start start 2024-04-25T08:56:34.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198f7d3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1988144ca0 0x7f198806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198f7d3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19880ccc70 con 0x7f19880ce510 2024-04-25T08:56:34.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198f7d3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198806dc40 con 0x7f198806ec60 2024-04-25T08:56:34.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198f7d3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19880cc2b0 con 0x7f1988144ca0 2024-04-25T08:56:34.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198cd6e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1988144ca0 0x7f198806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198cd6e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1988144ca0 0x7f198806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43484/0 (socket says 172.21.15.37:43484) 2024-04-25T08:56:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198d56f640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f19880ce510 0x7f19880ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38842/0 (socket says 172.21.15.37:38842) 2024-04-25T08:56:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198cd6e640 1 -- 172.21.15.37:0/3324115718 learned_addr learned my addr 172.21.15.37:0/3324115718 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198cd6e640 1 -- 172.21.15.37:0/3324115718 >> v1:172.21.15.203:6789/0 conn(0x7f198806ec60 legacy=0x7f1988142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198cd6e640 1 -- 172.21.15.37:0/3324115718 >> v1:172.21.15.37:6789/0 conn(0x7f19880ce510 legacy=0x7f19880ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f198cd6e640 1 -- 172.21.15.37:0/3324115718 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198806df20 con 0x7f1988144ca0 2024-04-25T08:56:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.530+0000 7f197ffff640 1 -- 172.21.15.37:0/3324115718 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058283138 0 0) 0x7f19880ccc70 con 0x7f19880ce510 2024-04-25T08:56:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198cd6e640 1 --2- 172.21.15.37:0/3324115718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1988144ca0 0x7f198806e720 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1978009870 tx=0x7f197801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d84418ff5c86606 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f197ffff640 1 -- 172.21.15.37:0/3324115718 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780b5070 con 0x7f1988144ca0 2024-04-25T08:56:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f197ffff640 1 -- 172.21.15.37:0/3324115718 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19780a8a50 con 0x7f1988144ca0 2024-04-25T08:56:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f197ffff640 1 -- 172.21.15.37:0/3324115718 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19780a8e20 con 0x7f1988144ca0 2024-04-25T08:56:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 -- 172.21.15.37:0/3324115718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1988144ca0 msgr2=0x7f198806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 --2- 172.21.15.37:0/3324115718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1988144ca0 0x7f198806e720 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1978009870 tx=0x7f197801ed50 comp rx=0 tx=0).stop 2024-04-25T08:56:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 -- 172.21.15.37:0/3324115718 shutdown_connections 2024-04-25T08:56:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 --2- 172.21.15.37:0/3324115718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1988144ca0 0x7f198806e720 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 -- 172.21.15.37:0/3324115718 >> 172.21.15.37:0/3324115718 conn(0x7f19880b11a0 msgr2=0x7f19880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 -- 172.21.15.37:0/3324115718 shutdown_connections 2024-04-25T08:56:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 -- 172.21.15.37:0/3324115718 wait complete. 2024-04-25T08:56:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 Processor -- start 2024-04-25T08:56:34.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 -- start start 2024-04-25T08:56:34.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198806ec60 0x7f19880a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19880ce510 0x7f198813ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1988144ca0 0x7f198815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1988142d90 con 0x7f198806ec60 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198806e290 con 0x7f1988144ca0 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198f7d3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1988142f10 con 0x7f19880ce510 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.534+0000 7f198d56f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198806ec60 0x7f19880a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198d56f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198806ec60 0x7f19880a8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43494/0 (socket says 172.21.15.37:43494) 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198d56f640 1 -- 172.21.15.37:0/98140759 learned_addr learned my addr 172.21.15.37:0/98140759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198cd6e640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19880ce510 0x7f198813ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198dd70640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1988144ca0 0x7f198815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198d56f640 1 -- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19880ce510 msgr2=0x7f198813ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198d56f640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19880ce510 0x7f198813ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198d56f640 1 -- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1988144ca0 msgr2=0x7f198815ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198d56f640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1988144ca0 0x7f198815ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198d56f640 1 -- 172.21.15.37:0/98140759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19880a9420 con 0x7f198806ec60 2024-04-25T08:56:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198dd70640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1988144ca0 0x7f198815ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198d56f640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198806ec60 0x7f19880a8ee0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f198400c4d0 tx=0x7f198400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f197dffb640 1 -- 172.21.15.37:0/98140759 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984018070 con 0x7f198806ec60 2024-04-25T08:56:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f197dffb640 1 -- 172.21.15.37:0/98140759 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19840074e0 con 0x7f198806ec60 2024-04-25T08:56:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198f7d3640 1 -- 172.21.15.37:0/98140759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19880a9f70 con 0x7f198806ec60 2024-04-25T08:56:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f197dffb640 1 -- 172.21.15.37:0/98140759 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984004a30 con 0x7f198806ec60 2024-04-25T08:56:34.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198f7d3640 1 -- 172.21.15.37:0/98140759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19880aa810 con 0x7f198806ec60 2024-04-25T08:56:34.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198f7d3640 1 -- 172.21.15.37:0/98140759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19880aa360 con 0x7f198806ec60 2024-04-25T08:56:34.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f197dffb640 1 -- 172.21.15.37:0/98140759 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f198401b070 con 0x7f198806ec60 2024-04-25T08:56:34.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f197dffb640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f196c066f60 0x7f196c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f197dffb640 1 -- 172.21.15.37:0/98140759 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f1984084500 con 0x7f198806ec60 2024-04-25T08:56:34.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.538+0000 7f198cd6e640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f196c066f60 0x7f196c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.542+0000 7f198cd6e640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f196c066f60 0x7f196c069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f19780b2000 tx=0x7f1978008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:34.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.542+0000 7f197dffb640 1 -- 172.21.15.37:0/98140759 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f19840553a0 con 0x7f198806ec60 2024-04-25T08:56:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.702+0000 7f198f7d3640 1 -- 172.21.15.37:0/98140759 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f198804efa0 con 0x7f198806ec60 2024-04-25T08:56:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f197dffb640 1 -- 172.21.15.37:0/98140759 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17040 (secure 0 0 0) 0x7f1984054ca0 con 0x7f198806ec60 2024-04-25T08:56:34.696 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:34.696 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":71,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:56:12.356132+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:44.492629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:45.203787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.9","mappings":[{"from":7,"to":2}]},{"pgid":"10.39","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 -- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f196c066f60 msgr2=0x7f196c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f196c066f60 0x7f196c069420 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f19780b2000 tx=0x7f1978008040 comp rx=0 tx=0).stop 2024-04-25T08:56:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 -- 172.21.15.37:0/98140759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198806ec60 msgr2=0x7f19880a8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198806ec60 0x7f19880a8ee0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f198400c4d0 tx=0x7f198400c990 comp rx=0 tx=0).stop 2024-04-25T08:56:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 -- 172.21.15.37:0/98140759 shutdown_connections 2024-04-25T08:56:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f196c066f60 0x7f196c069420 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1988144ca0 0x7f198815ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19880ce510 0x7f198813ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 --2- 172.21.15.37:0/98140759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f198806ec60 0x7f19880a8ee0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 -- 172.21.15.37:0/98140759 >> 172.21.15.37:0/98140759 conn(0x7f19880b11a0 msgr2=0x7f19880b0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 -- 172.21.15.37:0/98140759 shutdown_connections 2024-04-25T08:56:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.706+0000 7f198f7d3640 1 -- 172.21.15.37:0/98140759 wait complete. 2024-04-25T08:56:34.715 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:55:44.492629+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:55:45.203787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:56:34.716 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:56:34.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.846+0000 7f049efff640 1 Processor -- start 2024-04-25T08:56:34.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.846+0000 7f049efff640 1 -- start start 2024-04-25T08:56:34.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f049efff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f049809a480 0x7f04980cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f049efff640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f049815bd30 con 0x7f04980cfb80 2024-04-25T08:56:34.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f049efff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04980cc0d0 con 0x7f049815b950 2024-04-25T08:56:34.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f049efff640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f049809ab10 con 0x7f049809a480 2024-04-25T08:56:34.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f049d59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f04980cfb80 0x7f04980cff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38844/0 (socket says 172.21.15.37:38844) 2024-04-25T08:56:34.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f049d59c640 1 -- 172.21.15.37:0/2426039130 learned_addr learned my addr 172.21.15.37:0/2426039130 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:34.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830046379 0 0) 0x7f049815bd30 con 0x7f04980cfb80 2024-04-25T08:56:34.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0488003530 con 0x7f04980cfb80 2024-04-25T08:56:34.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f0497fff640 1 --2- 172.21.15.37:0/2426039130 >> v2:172.21.15.203:3300/0 conn(0x7f049809a480 0x7f04980cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2944951520 0 0) 0x7f0488003530 con 0x7f04980cfb80 2024-04-25T08:56:34.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 >> v2:172.21.15.203:3300/0 conn(0x7f049809a480 msgr2=0x7f04980cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:34.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 --2- 172.21.15.37:0/2426039130 >> v2:172.21.15.203:3300/0 conn(0x7f049809a480 0x7f04980cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 >> v1:172.21.15.203:6789/0 conn(0x7f049815b950 legacy=0x7f04980cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04980cc370 con 0x7f04980cfb80 2024-04-25T08:56:34.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0490001a00 con 0x7f04980cfb80 2024-04-25T08:56:34.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 >> v1:172.21.15.37:6789/0 conn(0x7f04980cfb80 legacy=0x7f04980cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 --2- 172.21.15.37:0/2426039130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0488003a80 0x7f0488003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 --2- 172.21.15.37:0/2426039130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 0x7f0488004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 --2- 172.21.15.37:0/2426039130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0488005220 0x7f048800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049809ab10 con 0x7f0488004810 2024-04-25T08:56:34.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f048800c150 con 0x7f0488005220 2024-04-25T08:56:34.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048800c350 con 0x7f0488003a80 2024-04-25T08:56:34.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f0497fff640 1 --2- 172.21.15.37:0/2426039130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 0x7f0488004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f049cd9b640 1 --2- 172.21.15.37:0/2426039130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0488003a80 0x7f0488003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f0497fff640 1 -- 172.21.15.37:0/2426039130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0488003a80 msgr2=0x7f0488003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:34.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f0497fff640 1 --2- 172.21.15.37:0/2426039130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0488003a80 0x7f0488003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f0497fff640 1 -- 172.21.15.37:0/2426039130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0488005220 msgr2=0x7f048800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:34.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f0497fff640 1 --2- 172.21.15.37:0/2426039130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0488005220 0x7f048800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f0497fff640 1 -- 172.21.15.37:0/2426039130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0488003530 con 0x7f0488004810 2024-04-25T08:56:34.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.850+0000 7f049d59c640 1 -- 172.21.15.37:0/2426039130 reap_dead start 2024-04-25T08:56:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f0497fff640 1 --2- 172.21.15.37:0/2426039130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 0x7f0488004c90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0484009ae0 tx=0x7f048401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93bef0116a45aaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840b2070 con 0x7f0488004810 2024-04-25T08:56:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04840a5660 con 0x7f0488004810 2024-04-25T08:56:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f04977fe640 1 -- 172.21.15.37:0/2426039130 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04840096b0 con 0x7f0488004810 2024-04-25T08:56:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 -- 172.21.15.37:0/2426039130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 msgr2=0x7f0488004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 --2- 172.21.15.37:0/2426039130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 0x7f0488004c90 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f0484009ae0 tx=0x7f048401ca70 comp rx=0 tx=0).stop 2024-04-25T08:56:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 -- 172.21.15.37:0/2426039130 shutdown_connections 2024-04-25T08:56:34.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 --2- 172.21.15.37:0/2426039130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 0x7f0488004c90 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 -- 172.21.15.37:0/2426039130 >> 172.21.15.37:0/2426039130 conn(0x7f04980b30b0 msgr2=0x7f04980b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 -- 172.21.15.37:0/2426039130 shutdown_connections 2024-04-25T08:56:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 -- 172.21.15.37:0/2426039130 wait complete. 2024-04-25T08:56:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 Processor -- start 2024-04-25T08:56:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 -- start start 2024-04-25T08:56:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 0x7f0498156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04981f3060 0x7f04980ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04980ae830 0x7f04980b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04980cf420 con 0x7f0488004810 2024-04-25T08:56:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04980cc930 con 0x7f04980ae830 2024-04-25T08:56:34.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04980cf5a0 con 0x7f04981f3060 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049cd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 0x7f0498156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049cd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 0x7f0498156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43504/0 (socket says 172.21.15.37:43504) 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049cd9b640 1 -- 172.21.15.37:0/364534398 learned_addr learned my addr 172.21.15.37:0/364534398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049d59c640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04980ae830 0x7f04980b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f0497fff640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04981f3060 0x7f04980ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049cd9b640 1 -- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04981f3060 msgr2=0x7f04980ae2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049cd9b640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04981f3060 0x7f04980ae2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049cd9b640 1 -- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04980ae830 msgr2=0x7f04980b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049cd9b640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04980ae830 0x7f04980b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049cd9b640 1 -- 172.21.15.37:0/364534398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0498157050 con 0x7f0488004810 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f0497fff640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04981f3060 0x7f04980ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:34.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049cd9b640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 0x7f0498156b10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f048c00e490 tx=0x7f048c00e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f04957fa640 1 -- 172.21.15.37:0/364534398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c0052e0 con 0x7f0488004810 2024-04-25T08:56:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f04957fa640 1 -- 172.21.15.37:0/364534398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f048c018070 con 0x7f0488004810 2024-04-25T08:56:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 -- 172.21.15.37:0/364534398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04980b5740 con 0x7f0488004810 2024-04-25T08:56:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f04957fa640 1 -- 172.21.15.37:0/364534398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048c003840 con 0x7f0488004810 2024-04-25T08:56:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.854+0000 7f049efff640 1 -- 172.21.15.37:0/364534398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04980c6920 con 0x7f0488004810 2024-04-25T08:56:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.858+0000 7f049efff640 1 -- 172.21.15.37:0/364534398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049804efa0 con 0x7f0488004810 2024-04-25T08:56:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.858+0000 7f04957fa640 1 -- 172.21.15.37:0/364534398 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f048c0039e0 con 0x7f0488004810 2024-04-25T08:56:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.858+0000 7f04957fa640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f047c066f90 0x7f047c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.858+0000 7f04957fa640 1 -- 172.21.15.37:0/364534398 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f048c084e30 con 0x7f0488004810 2024-04-25T08:56:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.858+0000 7f0497fff640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f047c066f90 0x7f047c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:34.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.858+0000 7f0497fff640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f047c066f90 0x7f047c069450 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f04840af000 tx=0x7f0484008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:34.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:34.862+0000 7f04957fa640 1 -- 172.21.15.37:0/364534398 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f048c055d00 con 0x7f0488004810 2024-04-25T08:56:35.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.014+0000 7f049efff640 1 -- 172.21.15.37:0/364534398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f04980d13a0 con 0x7f0488004810 2024-04-25T08:56:35.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.014+0000 7f04957fa640 1 -- 172.21.15.37:0/364534398 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f048c005480 con 0x7f0488004810 2024-04-25T08:56:35.005 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 -- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f047c066f90 msgr2=0x7f047c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f047c066f90 0x7f047c069450 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f04840af000 tx=0x7f0484008040 comp rx=0 tx=0).stop 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 -- 172.21.15.37:0/364534398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 msgr2=0x7f0498156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 0x7f0498156b10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f048c00e490 tx=0x7f048c00e950 comp rx=0 tx=0).stop 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 -- 172.21.15.37:0/364534398 shutdown_connections 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f047c066f90 0x7f047c069450 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f04980ae830 0x7f04980b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f04981f3060 0x7f04980ae2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 --2- 172.21.15.37:0/364534398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0488004810 0x7f0498156b10 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 -- 172.21.15.37:0/364534398 >> 172.21.15.37:0/364534398 conn(0x7f04980b30b0 msgr2=0x7f04980d0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 -- 172.21.15.37:0/364534398 shutdown_connections 2024-04-25T08:56:35.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.018+0000 7f049efff640 1 -- 172.21.15.37:0/364534398 wait complete. 2024-04-25T08:56:35.022 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:56:35.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3898229640 1 Processor -- start 2024-04-25T08:56:35.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3898229640 1 -- start start 2024-04-25T08:56:35.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3898229640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f38900cfca0 0x7f3890166290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3898229640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38900cbd20 con 0x7f389015b800 2024-04-25T08:56:35.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3898229640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38900cc000 con 0x7f389015b420 2024-04-25T08:56:35.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3898229640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38900cc930 con 0x7f38900cfca0 2024-04-25T08:56:35.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f38957c4640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f389015b800 0x7f38900cf590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38846/0 (socket says 172.21.15.37:38846) 2024-04-25T08:56:35.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f38957c4640 1 -- 172.21.15.37:0/1937099775 learned_addr learned my addr 172.21.15.37:0/1937099775 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:35.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f38967c6640 1 --2- 172.21.15.37:0/1937099775 >> v2:172.21.15.203:3300/0 conn(0x7f38900cfca0 0x7f3890166290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078439059 0 0) 0x7f38900cbd20 con 0x7f389015b800 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f387c003530 con 0x7f389015b800 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 792444447 0 0) 0x7f387c003530 con 0x7f389015b800 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 >> v2:172.21.15.203:3300/0 conn(0x7f38900cfca0 msgr2=0x7f3890166290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 --2- 172.21.15.37:0/1937099775 >> v2:172.21.15.203:3300/0 conn(0x7f38900cfca0 0x7f3890166290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 >> v1:172.21.15.203:6789/0 conn(0x7f389015b420 legacy=0x7f38900ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38900cc2e0 con 0x7f389015b800 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3880001680 con 0x7f389015b800 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 >> v1:172.21.15.37:6789/0 conn(0x7f389015b800 legacy=0x7f38900cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 --2- 172.21.15.37:0/1937099775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f387c003a30 0x7f387c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 --2- 172.21.15.37:0/1937099775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c004810 0x7f387c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 --2- 172.21.15.37:0/1937099775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0051d0 0x7f387c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38900cc930 con 0x7f387c003a30 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387c00c100 con 0x7f387c004810 2024-04-25T08:56:35.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387c00c300 con 0x7f387c0051d0 2024-04-25T08:56:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3895fc5640 1 --2- 172.21.15.37:0/1937099775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f387c003a30 0x7f387c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f38957c4640 1 --2- 172.21.15.37:0/1937099775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c004810 0x7f387c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3895fc5640 1 -- 172.21.15.37:0/1937099775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0051d0 msgr2=0x7f387c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3895fc5640 1 --2- 172.21.15.37:0/1937099775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f387c0051d0 0x7f387c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3895fc5640 1 -- 172.21.15.37:0/1937099775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c004810 msgr2=0x7f387c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3895fc5640 1 --2- 172.21.15.37:0/1937099775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c004810 0x7f387c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f3895fc5640 1 -- 172.21.15.37:0/1937099775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f387c003530 con 0x7f387c003a30 2024-04-25T08:56:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f38967c6640 1 -- 172.21.15.37:0/1937099775 reap_dead start 2024-04-25T08:56:35.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.154+0000 7f38957c4640 1 --2- 172.21.15.37:0/1937099775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c004810 0x7f387c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:35.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3895fc5640 1 --2- 172.21.15.37:0/1937099775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f387c003a30 0x7f387c003e70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f388c001750 tx=0x7f388c01c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e4801c659b9da10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:35.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c0013a0 con 0x7f387c003a30 2024-04-25T08:56:35.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388c0052f0 con 0x7f387c003a30 2024-04-25T08:56:35.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3894fc3640 1 -- 172.21.15.37:0/1937099775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388c003840 con 0x7f387c003a30 2024-04-25T08:56:35.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 -- 172.21.15.37:0/1937099775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f387c003a30 msgr2=0x7f387c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 --2- 172.21.15.37:0/1937099775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f387c003a30 0x7f387c003e70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f388c001750 tx=0x7f388c01c370 comp rx=0 tx=0).stop 2024-04-25T08:56:35.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 -- 172.21.15.37:0/1937099775 shutdown_connections 2024-04-25T08:56:35.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 --2- 172.21.15.37:0/1937099775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f387c003a30 0x7f387c003e70 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 -- 172.21.15.37:0/1937099775 >> 172.21.15.37:0/1937099775 conn(0x7f389006ae60 msgr2=0x7f389006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:35.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 -- 172.21.15.37:0/1937099775 shutdown_connections 2024-04-25T08:56:35.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 -- 172.21.15.37:0/1937099775 wait complete. 2024-04-25T08:56:35.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 Processor -- start 2024-04-25T08:56:35.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 -- start start 2024-04-25T08:56:35.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c003a30 0x7f389015bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c7420 0x7f38901f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38901f85c0 0x7f3890158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38900cfb20 con 0x7f38901f85c0 2024-04-25T08:56:35.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38900cf9a0 con 0x7f387c003a30 2024-04-25T08:56:35.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f3898229640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38900cadb0 con 0x7f38900c7420 2024-04-25T08:56:35.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.158+0000 7f38967c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38901f85c0 0x7f3890158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38967c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38901f85c0 0x7f3890158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43518/0 (socket says 172.21.15.37:43518) 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38967c6640 1 -- 172.21.15.37:0/646699297 learned_addr learned my addr 172.21.15.37:0/646699297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f3895fc5640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c003a30 0x7f389015bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38957c4640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c7420 0x7f38901f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38967c6640 1 -- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c7420 msgr2=0x7f38901f8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38967c6640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c7420 0x7f38901f8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38967c6640 1 -- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c003a30 msgr2=0x7f389015bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38967c6640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c003a30 0x7f389015bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38967c6640 1 -- 172.21.15.37:0/646699297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389015cf50 con 0x7f38901f85c0 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f3895fc5640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c003a30 0x7f389015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38957c4640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c7420 0x7f38901f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38967c6640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38901f85c0 0x7f3890158830 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f388800ee50 tx=0x7f388800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38867fc640 1 -- 172.21.15.37:0/646699297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888018020 con 0x7f38901f85c0 2024-04-25T08:56:35.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38867fc640 1 -- 172.21.15.37:0/646699297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388800bbf0 con 0x7f38901f85c0 2024-04-25T08:56:35.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f3898229640 1 -- 172.21.15.37:0/646699297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389015c460 con 0x7f38901f85c0 2024-04-25T08:56:35.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38867fc640 1 -- 172.21.15.37:0/646699297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888014400 con 0x7f38901f85c0 2024-04-25T08:56:35.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f3898229640 1 -- 172.21.15.37:0/646699297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f389015ca20 con 0x7f38901f85c0 2024-04-25T08:56:35.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f3898229640 1 -- 172.21.15.37:0/646699297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389004efa0 con 0x7f38901f85c0 2024-04-25T08:56:35.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38867fc640 1 -- 172.21.15.37:0/646699297 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f388801b020 con 0x7f38901f85c0 2024-04-25T08:56:35.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38867fc640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3874066f90 0x7f3874069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f38867fc640 1 -- 172.21.15.37:0/646699297 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f38880856f0 con 0x7f38901f85c0 2024-04-25T08:56:35.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f3895fc5640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3874066f90 0x7f3874069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.162+0000 7f3895fc5640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3874066f90 0x7f3874069450 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f388c004000 tx=0x7f388c0011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:35.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.166+0000 7f38867fc640 1 -- 172.21.15.37:0/646699297 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3888022030 con 0x7f38901f85c0 2024-04-25T08:56:35.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.330+0000 7f3898229640 1 -- 172.21.15.37:0/646699297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38900ae180 con 0x7f38901f85c0 2024-04-25T08:56:35.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.330+0000 7f38867fc640 1 -- 172.21.15.37:0/646699297 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3888056590 con 0x7f38901f85c0 2024-04-25T08:56:35.320 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:56:35.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 -- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3874066f90 msgr2=0x7f3874069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3874066f90 0x7f3874069450 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f388c004000 tx=0x7f388c0011a0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 -- 172.21.15.37:0/646699297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38901f85c0 msgr2=0x7f3890158830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38901f85c0 0x7f3890158830 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f388800ee50 tx=0x7f388800b480 comp rx=0 tx=0).stop 2024-04-25T08:56:35.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 -- 172.21.15.37:0/646699297 shutdown_connections 2024-04-25T08:56:35.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3874066f90 0x7f3874069450 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38901f85c0 0x7f3890158830 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c7420 0x7f38901f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 --2- 172.21.15.37:0/646699297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f387c003a30 0x7f389015bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 -- 172.21.15.37:0/646699297 >> 172.21.15.37:0/646699297 conn(0x7f389006ae60 msgr2=0x7f38900b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:35.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 -- 172.21.15.37:0/646699297 shutdown_connections 2024-04-25T08:56:35.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.334+0000 7f3898229640 1 -- 172.21.15.37:0/646699297 wait complete. 2024-04-25T08:56:35.340 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:56:35.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46ec16640 1 Processor -- start 2024-04-25T08:56:35.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46ec16640 1 -- start start 2024-04-25T08:56:35.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46ec16640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa468161c80 0x7fa4680b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46ec16640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4680c2a40 0x7fa4680ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46ec16640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa468162060 con 0x7fa4680c9f70 2024-04-25T08:56:35.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46ec16640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4680ca480 con 0x7fa468161c80 2024-04-25T08:56:35.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46ec16640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680ca740 con 0x7fa4680c2a40 2024-04-25T08:56:35.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa467fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa4680c9f70 0x7fa4680c2330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38848/0 (socket says 172.21.15.37:38848) 2024-04-25T08:56:35.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa467fff640 1 -- 172.21.15.37:0/2971870556 learned_addr learned my addr 172.21.15.37:0/2971870556 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:35.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46c9b2640 1 --2- 172.21.15.37:0/2971870556 >> v2:172.21.15.203:3300/0 conn(0x7fa468161c80 0x7fa4680b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46d1b3640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4680c2a40 0x7fa4680ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 128981318 0 0) 0x7fa468162060 con 0x7fa4680c9f70 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa450003530 con 0x7fa4680c9f70 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 18320082 0 0) 0x7fa450003530 con 0x7fa4680c9f70 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4680c2a40 msgr2=0x7fa4680ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4680c2a40 0x7fa4680ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 >> v2:172.21.15.203:3300/0 conn(0x7fa468161c80 msgr2=0x7fa4680b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 --2- 172.21.15.37:0/2971870556 >> v2:172.21.15.203:3300/0 conn(0x7fa468161c80 0x7fa4680b3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4680caa20 con 0x7fa4680c9f70 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fa4580016a0 con 0x7fa4680c9f70 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 >> v1:172.21.15.37:6789/0 conn(0x7fa4680c9f70 legacy=0x7fa4680c2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4680c2a40 0x7fa450003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa450003fb0 0x7fa450004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa450004d50 0x7fa4500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680ca480 con 0x7fa450004d50 2024-04-25T08:56:35.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680ca740 con 0x7fa450003fb0 2024-04-25T08:56:35.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45000ba00 con 0x7fa4680c2a40 2024-04-25T08:56:35.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46d1b3640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa450004d50 0x7fa4500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa467fff640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa450003fb0 0x7fa450004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46c9b2640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4680c2a40 0x7fa450003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46d1b3640 1 -- 172.21.15.37:0/2971870556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4680c2a40 msgr2=0x7fa450003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:35.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46d1b3640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa4680c2a40 0x7fa450003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46d1b3640 1 -- 172.21.15.37:0/2971870556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa450003fb0 msgr2=0x7fa450004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:35.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46d1b3640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa450003fb0 0x7fa450004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.474+0000 7fa46d1b3640 1 -- 172.21.15.37:0/2971870556 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa450003530 con 0x7fa450004d50 2024-04-25T08:56:35.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa46d1b3640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa450004d50 0x7fa4500051d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa4680cfb30 tx=0x7fa46001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce40e034d1bad0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:35.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4600b2070 con 0x7fa450004d50 2024-04-25T08:56:35.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4600a5c30 con 0x7fa450004d50 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa4677fe640 1 -- 172.21.15.37:0/2971870556 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460009c10 con 0x7fa450004d50 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa46ec16640 1 -- 172.21.15.37:0/2971870556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa450004d50 msgr2=0x7fa4500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa46ec16640 1 --2- 172.21.15.37:0/2971870556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa450004d50 0x7fa4500051d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fa4680cfb30 tx=0x7fa46001cc20 comp rx=0 tx=0).stop 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa46d1b3640 1 -- 172.21.15.37:0/2971870556 reap_dead start 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa46ec16640 1 -- 172.21.15.37:0/2971870556 shutdown_connections 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa46ec16640 1 -- 172.21.15.37:0/2971870556 >> 172.21.15.37:0/2971870556 conn(0x7fa46806ae60 msgr2=0x7fa46806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa46ec16640 1 -- 172.21.15.37:0/2971870556 shutdown_connections 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa46ec16640 1 -- 172.21.15.37:0/2971870556 wait complete. 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa46ec16640 1 Processor -- start 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.478+0000 7fa46ec16640 1 -- start start 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa46ec16640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa468154190 0x7fa46815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa46ec16640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468154570 0x7fa4681604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa46ec16640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa468160be0 0x7fa468162400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa46ec16640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680b3990 con 0x7fa468154570 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa46ec16640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680cafe0 con 0x7fa468154190 2024-04-25T08:56:35.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa46ec16640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4680b3b10 con 0x7fa468160be0 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa467fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468154570 0x7fa4681604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa467fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468154570 0x7fa4681604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43540/0 (socket says 172.21.15.37:43540) 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa467fff640 1 -- 172.21.15.37:0/1549402089 learned_addr learned my addr 172.21.15.37:0/1549402089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa46c9b2640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa468154190 0x7fa46815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa46d1b3640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa468160be0 0x7fa468162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa467fff640 1 -- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa468160be0 msgr2=0x7fa468162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa467fff640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa468160be0 0x7fa468162400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa467fff640 1 -- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa468154190 msgr2=0x7fa46815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa467fff640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa468154190 0x7fa46815ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa467fff640 1 -- 172.21.15.37:0/1549402089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4680ad6f0 con 0x7fa468154570 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa467fff640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468154570 0x7fa4681604d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa45800eaf0 tx=0x7fa4580016b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa4657fa640 1 -- 172.21.15.37:0/1549402089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45800c870 con 0x7fa468154570 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa4657fa640 1 -- 172.21.15.37:0/1549402089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa458018070 con 0x7fa468154570 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa46ec16640 1 -- 172.21.15.37:0/1549402089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4680ad9d0 con 0x7fa468154570 2024-04-25T08:56:35.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa4657fa640 1 -- 172.21.15.37:0/1549402089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45800cc50 con 0x7fa468154570 2024-04-25T08:56:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa46ec16640 1 -- 172.21.15.37:0/1549402089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4680adf90 con 0x7fa468154570 2024-04-25T08:56:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa46ec16640 1 -- 172.21.15.37:0/1549402089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46804efa0 con 0x7fa468154570 2024-04-25T08:56:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa4657fa640 1 -- 172.21.15.37:0/1549402089 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa45801d460 con 0x7fa468154570 2024-04-25T08:56:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.482+0000 7fa4657fa640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa454066f60 0x7fa454069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.486+0000 7fa4657fa640 1 -- 172.21.15.37:0/1549402089 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fa458085a80 con 0x7fa468154570 2024-04-25T08:56:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.486+0000 7fa46c9b2640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa454066f60 0x7fa454069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.486+0000 7fa46c9b2640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa454066f60 0x7fa454069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009b60 tx=0x7fa45c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:35.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.486+0000 7fa4657fa640 1 -- 172.21.15.37:0/1549402089 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa458056970 con 0x7fa468154570 2024-04-25T08:56:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.646+0000 7fa46ec16640 1 -- 172.21.15.37:0/1549402089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa46806bfc0 con 0x7fa468154570 2024-04-25T08:56:35.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.646+0000 7fa4657fa640 1 -- 172.21.15.37:0/1549402089 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa458056270 con 0x7fa468154570 2024-04-25T08:56:35.637 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 -- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa454066f60 msgr2=0x7fa454069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa454066f60 0x7fa454069420 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009b60 tx=0x7fa45c008040 comp rx=0 tx=0).stop 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 -- 172.21.15.37:0/1549402089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468154570 msgr2=0x7fa4681604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468154570 0x7fa4681604d0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fa45800eaf0 tx=0x7fa4580016b0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 -- 172.21.15.37:0/1549402089 shutdown_connections 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa454066f60 0x7fa454069420 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa468160be0 0x7fa468162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa468154570 0x7fa4681604d0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 --2- 172.21.15.37:0/1549402089 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa468154190 0x7fa46815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 -- 172.21.15.37:0/1549402089 >> 172.21.15.37:0/1549402089 conn(0x7fa46806ae60 msgr2=0x7fa4680c2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 -- 172.21.15.37:0/1549402089 shutdown_connections 2024-04-25T08:56:35.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.650+0000 7fa46ec16640 1 -- 172.21.15.37:0/1549402089 wait complete. 2024-04-25T08:56:35.657 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:56:35.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc50d0c9640 1 Processor -- start 2024-04-25T08:56:35.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc50d0c9640 1 -- start start 2024-04-25T08:56:35.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc50d0c9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc50815aac0 0x7fc5080ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc50d0c9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5080cf0e0 0x7fc5080cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc50d0c9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5080ce320 con 0x7fc50815aea0 2024-04-25T08:56:35.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc50d0c9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5080ce0d0 con 0x7fc50815aac0 2024-04-25T08:56:35.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc50d0c9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5080ce600 con 0x7fc5080cf0e0 2024-04-25T08:56:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc506d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc50815aac0 0x7fc5080ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc506d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc50815aac0 0x7fc5080ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43550/0 (socket says 172.21.15.37:43550) 2024-04-25T08:56:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc506d9d640 1 -- 172.21.15.37:0/846414995 learned_addr learned my addr 172.21.15.37:0/846414995 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc50659c640 1 --2- 172.21.15.37:0/846414995 >> v2:172.21.15.203:3300/0 conn(0x7fc5080cf0e0 0x7fc5080cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc506d9d640 1 -- 172.21.15.37:0/846414995 >> v2:172.21.15.203:3300/0 conn(0x7fc5080cf0e0 msgr2=0x7fc5080cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc506d9d640 1 --2- 172.21.15.37:0/846414995 >> v2:172.21.15.203:3300/0 conn(0x7fc5080cf0e0 0x7fc5080cd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc506d9d640 1 -- 172.21.15.37:0/846414995 >> v1:172.21.15.203:6789/0 conn(0x7fc50815aea0 legacy=0x7fc50815b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:35.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.786+0000 7fc506d9d640 1 -- 172.21.15.37:0/846414995 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5080ce8e0 con 0x7fc50815aac0 2024-04-25T08:56:35.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc506d9d640 1 --2- 172.21.15.37:0/846414995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50815aac0 0x7fc5080ccc40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc50815e1b0 tx=0x7fc50001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ccc6ce58ef95ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:35.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc505d9b640 1 -- 172.21.15.37:0/846414995 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5000a5780 con 0x7fc50815aac0 2024-04-25T08:56:35.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc505d9b640 1 -- 172.21.15.37:0/846414995 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5000a5920 con 0x7fc50815aac0 2024-04-25T08:56:35.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc505d9b640 1 -- 172.21.15.37:0/846414995 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5000a5cf0 con 0x7fc50815aac0 2024-04-25T08:56:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc50d0c9640 1 -- 172.21.15.37:0/846414995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50815aac0 msgr2=0x7fc5080ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc50d0c9640 1 --2- 172.21.15.37:0/846414995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50815aac0 0x7fc5080ccc40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fc50815e1b0 tx=0x7fc50001c9b0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc50d0c9640 1 -- 172.21.15.37:0/846414995 shutdown_connections 2024-04-25T08:56:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc50d0c9640 1 --2- 172.21.15.37:0/846414995 >> v2:172.21.15.203:3300/0 conn(0x7fc5080cf0e0 0x7fc5080cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc50d0c9640 1 --2- 172.21.15.37:0/846414995 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50815aac0 0x7fc5080ccc40 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc50d0c9640 1 -- 172.21.15.37:0/846414995 >> 172.21.15.37:0/846414995 conn(0x7fc508099a70 msgr2=0x7fc5080d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc50d0c9640 1 -- 172.21.15.37:0/846414995 shutdown_connections 2024-04-25T08:56:35.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.790+0000 7fc50d0c9640 1 -- 172.21.15.37:0/846414995 wait complete. 2024-04-25T08:56:35.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50d0c9640 1 Processor -- start 2024-04-25T08:56:35.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50d0c9640 1 -- start start 2024-04-25T08:56:35.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50d0c9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080cf0e0 0x7fc5080c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50d0c9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50815aac0 0x7fc50806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50d0c9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc50815aea0 0x7fc50806b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50d0c9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080cbbb0 con 0x7fc50815aac0 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50d0c9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080cba30 con 0x7fc50815aea0 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50d0c9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5080cbd30 con 0x7fc5080cf0e0 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50815aac0 0x7fc50806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc506d9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080cf0e0 0x7fc5080c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50815aac0 0x7fc50806b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43554/0 (socket says 172.21.15.37:43554) 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50659c640 1 -- 172.21.15.37:0/3594034237 learned_addr learned my addr 172.21.15.37:0/3594034237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50759e640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc50815aea0 0x7fc50806b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50659c640 1 -- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080cf0e0 msgr2=0x7fc5080c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50659c640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080cf0e0 0x7fc5080c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50659c640 1 -- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc50815aea0 msgr2=0x7fc50806b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50659c640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc50815aea0 0x7fc50806b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50659c640 1 -- 172.21.15.37:0/3594034237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5080c55f0 con 0x7fc50815aac0 2024-04-25T08:56:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50759e640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc50815aea0 0x7fc50806b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50659c640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50815aac0 0x7fc50806b1b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000d800 tx=0x7fc4f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc4ff7fe640 1 -- 172.21.15.37:0/3594034237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000b6c0 con 0x7fc50815aac0 2024-04-25T08:56:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc4ff7fe640 1 -- 172.21.15.37:0/3594034237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f0019070 con 0x7fc50815aac0 2024-04-25T08:56:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50d0c9640 1 -- 172.21.15.37:0/3594034237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5080c58d0 con 0x7fc50815aac0 2024-04-25T08:56:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc4ff7fe640 1 -- 172.21.15.37:0/3594034237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000ba60 con 0x7fc50815aac0 2024-04-25T08:56:35.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50d0c9640 1 -- 172.21.15.37:0/3594034237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5080c5e90 con 0x7fc50815aac0 2024-04-25T08:56:35.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.794+0000 7fc50d0c9640 1 -- 172.21.15.37:0/3594034237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5080c6260 con 0x7fc50815aac0 2024-04-25T08:56:35.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.798+0000 7fc4ff7fe640 1 -- 172.21.15.37:0/3594034237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4f000bc00 con 0x7fc50815aac0 2024-04-25T08:56:35.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.798+0000 7fc4ff7fe640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f60 0x7fc4e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:35.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.798+0000 7fc4ff7fe640 1 -- 172.21.15.37:0/3594034237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc4f0085670 con 0x7fc50815aac0 2024-04-25T08:56:35.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.798+0000 7fc506d9d640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f60 0x7fc4e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:35.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.798+0000 7fc506d9d640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f60 0x7fc4e8069420 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc5000ae000 tx=0x7fc5000a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:35.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.802+0000 7fc4ff7fe640 1 -- 172.21.15.37:0/3594034237 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4f0056480 con 0x7fc50815aac0 2024-04-25T08:56:35.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.974+0000 7fc50d0c9640 1 -- 172.21.15.37:0/3594034237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc508159da0 con 0x7fc50815aac0 2024-04-25T08:56:35.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.974+0000 7fc4ff7fe640 1 -- 172.21.15.37:0/3594034237 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc4f0055d80 con 0x7fc50815aac0 2024-04-25T08:56:35.966 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:56:35.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 -- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f60 msgr2=0x7fc4e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f60 0x7fc4e8069420 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc5000ae000 tx=0x7fc5000a7070 comp rx=0 tx=0).stop 2024-04-25T08:56:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 -- 172.21.15.37:0/3594034237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50815aac0 msgr2=0x7fc50806b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50815aac0 0x7fc50806b1b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fc4f000d800 tx=0x7fc4f000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 -- 172.21.15.37:0/3594034237 shutdown_connections 2024-04-25T08:56:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f60 0x7fc4e8069420 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc50815aea0 0x7fc50806b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50815aac0 0x7fc50806b1b0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 --2- 172.21.15.37:0/3594034237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5080cf0e0 0x7fc5080c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 -- 172.21.15.37:0/3594034237 >> 172.21.15.37:0/3594034237 conn(0x7fc508099a70 msgr2=0x7fc50814a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 -- 172.21.15.37:0/3594034237 shutdown_connections 2024-04-25T08:56:35.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:35.978+0000 7fc50d0c9640 1 -- 172.21.15.37:0/3594034237 wait complete. 2024-04-25T08:56:35.985 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T08:56:35.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:35.986 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:56:35.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[43020]: fuse finished with error 0 and tester_r 0 2024-04-25T08:56:35.999 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:56:35.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:35.999 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:56:36.165 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:56:36.165 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:36.171 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:56:36.172 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:56:36.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:36.172 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:56:36.219 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:56:36.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:56:36.220 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:56:36.220 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:36.226 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:56:36.226 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:56:36.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:36.227 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:56:36.275 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:56:36.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:56:36.276 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:56:36.276 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:36.282 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:56:36.282 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:56:36.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:36.283 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:56:36.331 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:56:36.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:56:36.332 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea8aafe640 1 Processor -- start 2024-04-25T08:56:36.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea8aafe640 1 -- start start 2024-04-25T08:56:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea8aafe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fea840d4230 0x7fea840d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea8aafe640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea8406be60 con 0x7fea840d07f0 2024-04-25T08:56:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea8aafe640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea840d4650 con 0x7fea8415da80 2024-04-25T08:56:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea8aafe640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea840d2210 con 0x7fea840d4230 2024-04-25T08:56:36.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea83fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fea840d07f0 0x7fea8406c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38852/0 (socket says 172.21.15.37:38852) 2024-04-25T08:56:36.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea83fff640 1 -- 172.21.15.37:0/1446688187 learned_addr learned my addr 172.21.15.37:0/1446688187 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:36.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea8909b640 1 --2- 172.21.15.37:0/1446688187 >> v2:172.21.15.37:3300/0 conn(0x7fea840d4230 0x7fea840d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:36.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea837fe640 1 -- 172.21.15.37:0/1446688187 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750425098 0 0) 0x7fea8406be60 con 0x7fea840d07f0 2024-04-25T08:56:36.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea837fe640 1 -- 172.21.15.37:0/1446688187 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea6c003530 con 0x7fea840d07f0 2024-04-25T08:56:36.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea8909b640 1 -- 172.21.15.37:0/1446688187 >> v1:172.21.15.203:6789/0 conn(0x7fea8415da80 legacy=0x7fea840d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea8909b640 1 -- 172.21.15.37:0/1446688187 >> v1:172.21.15.37:6789/0 conn(0x7fea840d07f0 legacy=0x7fea8406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.478+0000 7fea8909b640 1 -- 172.21.15.37:0/1446688187 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8415de60 con 0x7fea840d4230 2024-04-25T08:56:36.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8909b640 1 --2- 172.21.15.37:0/1446688187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea840d4230 0x7fea840d1cd0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009d90 tx=0x7fea7c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab0e1a23a99cab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:36.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea837fe640 1 -- 172.21.15.37:0/1446688187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0b2070 con 0x7fea840d4230 2024-04-25T08:56:36.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea837fe640 1 -- 172.21.15.37:0/1446688187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7c0a5830 con 0x7fea840d4230 2024-04-25T08:56:36.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea837fe640 1 -- 172.21.15.37:0/1446688187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0a5c00 con 0x7fea840d4230 2024-04-25T08:56:36.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 -- 172.21.15.37:0/1446688187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea840d4230 msgr2=0x7fea840d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 --2- 172.21.15.37:0/1446688187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea840d4230 0x7fea840d1cd0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009d90 tx=0x7fea7c01c970 comp rx=0 tx=0).stop 2024-04-25T08:56:36.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 -- 172.21.15.37:0/1446688187 shutdown_connections 2024-04-25T08:56:36.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 --2- 172.21.15.37:0/1446688187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea840d4230 0x7fea840d1cd0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 -- 172.21.15.37:0/1446688187 >> 172.21.15.37:0/1446688187 conn(0x7fea8408b000 msgr2=0x7fea840d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:36.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 -- 172.21.15.37:0/1446688187 shutdown_connections 2024-04-25T08:56:36.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 -- 172.21.15.37:0/1446688187 wait complete. 2024-04-25T08:56:36.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 Processor -- start 2024-04-25T08:56:36.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 -- start start 2024-04-25T08:56:36.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea840d07f0 0x7fea841f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:36.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea840d4230 0x7fea840b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:36.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8415da80 0x7fea840b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:36.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8406dbc0 con 0x7fea840d4230 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8406da40 con 0x7fea840d07f0 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8aafe640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8416b5c0 con 0x7fea8415da80 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea83fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea840d4230 0x7fea840b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea83fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea840d4230 0x7fea840b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43570/0 (socket says 172.21.15.37:43570) 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea83fff640 1 -- 172.21.15.37:0/2834256764 learned_addr learned my addr 172.21.15.37:0/2834256764 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8889a640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea840d07f0 0x7fea841f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.482+0000 7fea8909b640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8415da80 0x7fea840b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea83fff640 1 -- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8415da80 msgr2=0x7fea840b7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea83fff640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8415da80 0x7fea840b7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea83fff640 1 -- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea840d07f0 msgr2=0x7fea841f9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea83fff640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea840d07f0 0x7fea841f9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea83fff640 1 -- 172.21.15.37:0/2834256764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea840b8d10 con 0x7fea840d4230 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea8889a640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea840d07f0 0x7fea841f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea83fff640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea840d4230 0x7fea840b6310 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fea7800c680 tx=0x7fea7800cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:36.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea817fa640 1 -- 172.21.15.37:0/2834256764 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea78017070 con 0x7fea840d4230 2024-04-25T08:56:36.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea817fa640 1 -- 172.21.15.37:0/2834256764 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea78007270 con 0x7fea840d4230 2024-04-25T08:56:36.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea817fa640 1 -- 172.21.15.37:0/2834256764 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea780057b0 con 0x7fea840d4230 2024-04-25T08:56:36.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea8aafe640 1 -- 172.21.15.37:0/2834256764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea840b8ff0 con 0x7fea840d4230 2024-04-25T08:56:36.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea8aafe640 1 -- 172.21.15.37:0/2834256764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea840cacf0 con 0x7fea840d4230 2024-04-25T08:56:36.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea8aafe640 1 -- 172.21.15.37:0/2834256764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea8404efa0 con 0x7fea840d4230 2024-04-25T08:56:36.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea817fa640 1 -- 172.21.15.37:0/2834256764 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fea7801a070 con 0x7fea840d4230 2024-04-25T08:56:36.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea817fa640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fea70066f90 0x7fea70069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:36.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea817fa640 1 -- 172.21.15.37:0/2834256764 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fea78084610 con 0x7fea840d4230 2024-04-25T08:56:36.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea8889a640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fea70066f90 0x7fea70069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:36.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.486+0000 7fea8889a640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fea70066f90 0x7fea70069450 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fea74001670 tx=0x7fea74009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:36.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.490+0000 7fea817fa640 1 -- 172.21.15.37:0/2834256764 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fea78055410 con 0x7fea840d4230 2024-04-25T08:56:36.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.682+0000 7fea8aafe640 1 -- 172.21.15.37:0/2834256764 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea840b9570 con 0x7fea840d4230 2024-04-25T08:56:36.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea817fa640 1 -- 172.21.15.37:0/2834256764 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fea78054d10 con 0x7fea840d4230 2024-04-25T08:56:36.675 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:36.675 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/486740526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":486740526},{"type":"v1","addr":"172.21.15.37:6833","nonce":486740526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/121484331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":121484331},{"type":"v1","addr":"172.21.15.203:6835","nonce":121484331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2501630566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2501630566},{"type":"v1","addr":"172.21.15.203:6837","nonce":2501630566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:55:46.200939+0000","modified":"2024-04-25T08:56:06.871679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6405":{"gid":6405,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":8,"addr":"172.21.15.37:6835/2189232625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2189232625},{"type":"v1","addr":"172.21.15.37:6835","nonce":2189232625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:56:36.675 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-04-25T08:56:36.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 -- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fea70066f90 msgr2=0x7fea70069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fea70066f90 0x7fea70069450 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fea74001670 tx=0x7fea74009040 comp rx=0 tx=0).stop 2024-04-25T08:56:36.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 -- 172.21.15.37:0/2834256764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea840d4230 msgr2=0x7fea840b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea840d4230 0x7fea840b6310 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fea7800c680 tx=0x7fea7800cb40 comp rx=0 tx=0).stop 2024-04-25T08:56:36.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 -- 172.21.15.37:0/2834256764 shutdown_connections 2024-04-25T08:56:36.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fea70066f90 0x7fea70069450 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fea8415da80 0x7fea840b7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea840d4230 0x7fea840b6310 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 --2- 172.21.15.37:0/2834256764 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fea840d07f0 0x7fea841f9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 -- 172.21.15.37:0/2834256764 >> 172.21.15.37:0/2834256764 conn(0x7fea8408b000 msgr2=0x7fea840ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:36.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 -- 172.21.15.37:0/2834256764 shutdown_connections 2024-04-25T08:56:36.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.686+0000 7fea8aafe640 1 -- 172.21.15.37:0/2834256764 wait complete. 2024-04-25T08:56:36.694 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:36.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.822+0000 7fdf438f2640 1 Processor -- start 2024-04-25T08:56:36.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.822+0000 7fdf438f2640 1 -- start start 2024-04-25T08:56:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf438f2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdf3c0ce510 0x7fdf3c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf438f2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c0cee30 con 0x7fdf3c1451f0 2024-04-25T08:56:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf438f2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3c0ccc70 con 0x7fdf3c144ca0 2024-04-25T08:56:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf438f2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3c0cc2b0 con 0x7fdf3c0ce510 2024-04-25T08:56:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf4168e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fdf3c0ce510 0x7fdf3c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf41e8f640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdf3c1451f0 0x7fdf3c142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38862/0 (socket says 172.21.15.37:38862) 2024-04-25T08:56:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf41e8f640 1 -- 172.21.15.37:0/3963822135 learned_addr learned my addr 172.21.15.37:0/3963822135 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf4168e640 1 -- 172.21.15.37:0/3963822135 >> v1:172.21.15.203:6789/0 conn(0x7fdf3c144ca0 legacy=0x7fdf3c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf4168e640 1 -- 172.21.15.37:0/3963822135 >> v1:172.21.15.37:6789/0 conn(0x7fdf3c1451f0 legacy=0x7fdf3c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf33fff640 1 -- 172.21.15.37:0/3963822135 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338710284 0 0) 0x7fdf3c0cee30 con 0x7fdf3c1451f0 2024-04-25T08:56:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf4168e640 1 -- 172.21.15.37:0/3963822135 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c06dc40 con 0x7fdf3c0ce510 2024-04-25T08:56:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf4168e640 1 --2- 172.21.15.37:0/3963822135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf3c0ce510 0x7fdf3c0ce8f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdf380097c0 tx=0x7fdf3801ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b306af302ba96f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf33fff640 1 -- 172.21.15.37:0/3963822135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf380b5070 con 0x7fdf3c0ce510 2024-04-25T08:56:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf33fff640 1 -- 172.21.15.37:0/3963822135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf380a8990 con 0x7fdf3c0ce510 2024-04-25T08:56:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.826+0000 7fdf33fff640 1 -- 172.21.15.37:0/3963822135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf380a8d60 con 0x7fdf3c0ce510 2024-04-25T08:56:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 -- 172.21.15.37:0/3963822135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf3c0ce510 msgr2=0x7fdf3c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 --2- 172.21.15.37:0/3963822135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf3c0ce510 0x7fdf3c0ce8f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdf380097c0 tx=0x7fdf3801ec90 comp rx=0 tx=0).stop 2024-04-25T08:56:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 -- 172.21.15.37:0/3963822135 shutdown_connections 2024-04-25T08:56:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 --2- 172.21.15.37:0/3963822135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf3c0ce510 0x7fdf3c0ce8f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 -- 172.21.15.37:0/3963822135 >> 172.21.15.37:0/3963822135 conn(0x7fdf3c0b11a0 msgr2=0x7fdf3c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 -- 172.21.15.37:0/3963822135 shutdown_connections 2024-04-25T08:56:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 -- 172.21.15.37:0/3963822135 wait complete. 2024-04-25T08:56:36.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 Processor -- start 2024-04-25T08:56:36.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 -- start start 2024-04-25T08:56:36.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3c0ce510 0x7fdf3c1e5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf3c144ca0 0x7fdf3c1e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c1451f0 0x7fdf3c1e5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c06ee30 con 0x7fdf3c144ca0 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c06e290 con 0x7fdf3c1451f0 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf438f2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3c0cc900 con 0x7fdf3c0ce510 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf40e8d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf3c144ca0 0x7fdf3c1e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf40e8d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf3c144ca0 0x7fdf3c1e4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43588/0 (socket says 172.21.15.37:43588) 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf40e8d640 1 -- 172.21.15.37:0/2997107468 learned_addr learned my addr 172.21.15.37:0/2997107468 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf41e8f640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c1451f0 0x7fdf3c1e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf4168e640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3c0ce510 0x7fdf3c1e5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf40e8d640 1 -- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3c0ce510 msgr2=0x7fdf3c1e5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf40e8d640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3c0ce510 0x7fdf3c1e5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf40e8d640 1 -- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c1451f0 msgr2=0x7fdf3c1e5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf40e8d640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c1451f0 0x7fdf3c1e5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf40e8d640 1 -- 172.21.15.37:0/2997107468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3c1e3850 con 0x7fdf3c144ca0 2024-04-25T08:56:36.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.830+0000 7fdf4168e640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3c0ce510 0x7fdf3c1e5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:36.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf40e8d640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf3c144ca0 0x7fdf3c1e4b00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c4d0 tx=0x7fdf2c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:36.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf31ffb640 1 -- 172.21.15.37:0/2997107468 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c0052d0 con 0x7fdf3c144ca0 2024-04-25T08:56:36.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf31ffb640 1 -- 172.21.15.37:0/2997107468 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2c005470 con 0x7fdf3c144ca0 2024-04-25T08:56:36.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf31ffb640 1 -- 172.21.15.37:0/2997107468 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c012830 con 0x7fdf3c144ca0 2024-04-25T08:56:36.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf438f2640 1 -- 172.21.15.37:0/2997107468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3c1e3b30 con 0x7fdf3c144ca0 2024-04-25T08:56:36.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf438f2640 1 -- 172.21.15.37:0/2997107468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf3c1e40f0 con 0x7fdf3c144ca0 2024-04-25T08:56:36.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf438f2640 1 -- 172.21.15.37:0/2997107468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3c04efa0 con 0x7fdf3c144ca0 2024-04-25T08:56:36.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf31ffb640 1 -- 172.21.15.37:0/2997107468 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf2c0129d0 con 0x7fdf3c144ca0 2024-04-25T08:56:36.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf31ffb640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf20066f60 0x7fdf20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:36.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf31ffb640 1 -- 172.21.15.37:0/2997107468 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fdf2c084e20 con 0x7fdf3c144ca0 2024-04-25T08:56:36.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf4168e640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf20066f60 0x7fdf20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:36.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.834+0000 7fdf4168e640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf20066f60 0x7fdf20069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdf380b2000 tx=0x7fdf38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:36.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:36.838+0000 7fdf31ffb640 1 -- 172.21.15.37:0/2997107468 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdf2c055b90 con 0x7fdf3c144ca0 2024-04-25T08:56:36.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.002+0000 7fdf438f2640 1 -- 172.21.15.37:0/2997107468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf3c0b1cc0 con 0x7fdf3c144ca0 2024-04-25T08:56:36.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.002+0000 7fdf31ffb640 1 -- 172.21.15.37:0/2997107468 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17040 (secure 0 0 0) 0x7fdf2c055490 con 0x7fdf3c144ca0 2024-04-25T08:56:36.994 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:36.994 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":71,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:56:12.356132+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:44.492629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:45.203787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.9","mappings":[{"from":7,"to":2}]},{"pgid":"10.39","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:36.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 -- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf20066f60 msgr2=0x7fdf20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf20066f60 0x7fdf20069420 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdf380b2000 tx=0x7fdf38008040 comp rx=0 tx=0).stop 2024-04-25T08:56:36.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 -- 172.21.15.37:0/2997107468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf3c144ca0 msgr2=0x7fdf3c1e4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:36.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf3c144ca0 0x7fdf3c1e4b00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c4d0 tx=0x7fdf2c00c990 comp rx=0 tx=0).stop 2024-04-25T08:56:36.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 -- 172.21.15.37:0/2997107468 shutdown_connections 2024-04-25T08:56:36.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf20066f60 0x7fdf20069420 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf3c1451f0 0x7fdf3c1e5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf3c144ca0 0x7fdf3c1e4b00 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 --2- 172.21.15.37:0/2997107468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf3c0ce510 0x7fdf3c1e5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:36.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 -- 172.21.15.37:0/2997107468 >> 172.21.15.37:0/2997107468 conn(0x7fdf3c0b11a0 msgr2=0x7fdf3c0cac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:36.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 -- 172.21.15.37:0/2997107468 shutdown_connections 2024-04-25T08:56:36.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.006+0000 7fdf438f2640 1 -- 172.21.15.37:0/2997107468 wait complete. 2024-04-25T08:56:37.013 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:55:44.492629+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:55:45.203787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:56:37.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:56:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d8276a640 1 Processor -- start 2024-04-25T08:56:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d8276a640 1 -- start start 2024-04-25T08:56:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d8276a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4d7c15b6b0 0x7f4d7c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d8276a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4d7c073010 0x7f4d7c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d8276a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7c0cbd20 con 0x7f4d7c0cf590 2024-04-25T08:56:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d8276a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c15ba90 con 0x7f4d7c073010 2024-04-25T08:56:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d8276a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c0cc000 con 0x7f4d7c15b6b0 2024-04-25T08:56:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d80d07640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4d7c073010 0x7f4d7c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d80d07640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4d7c073010 0x7f4d7c0cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43592/0 (socket says 172.21.15.37:43592) 2024-04-25T08:56:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d80d07640 1 -- 172.21.15.37:0/4018599704 learned_addr learned my addr 172.21.15.37:0/4018599704 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d7bfff640 1 --2- 172.21.15.37:0/4018599704 >> v2:172.21.15.203:3300/0 conn(0x7f4d7c15b6b0 0x7f4d7c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d80d07640 1 -- 172.21.15.37:0/4018599704 >> v2:172.21.15.203:3300/0 conn(0x7f4d7c15b6b0 msgr2=0x7f4d7c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d80d07640 1 --2- 172.21.15.37:0/4018599704 >> v2:172.21.15.203:3300/0 conn(0x7f4d7c15b6b0 0x7f4d7c0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d80d07640 1 -- 172.21.15.37:0/4018599704 >> v1:172.21.15.203:6789/0 conn(0x7f4d7c0cf590 legacy=0x7f4d7c15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d80d07640 1 -- 172.21.15.37:0/4018599704 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c0cc2e0 con 0x7f4d7c073010 2024-04-25T08:56:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.158+0000 7f4d7bfff640 1 --2- 172.21.15.37:0/4018599704 >> v2:172.21.15.203:3300/0 conn(0x7f4d7c15b6b0 0x7f4d7c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d80d07640 1 --2- 172.21.15.37:0/4018599704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d7c073010 0x7f4d7c0cfa20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0d0180 tx=0x7f4d7401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d1182d0bd1554c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7affd640 1 -- 172.21.15.37:0/4018599704 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d740a56c0 con 0x7f4d7c073010 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7affd640 1 -- 172.21.15.37:0/4018599704 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d740a5860 con 0x7f4d7c073010 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7affd640 1 -- 172.21.15.37:0/4018599704 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d740a5c30 con 0x7f4d7c073010 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 -- 172.21.15.37:0/4018599704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d7c073010 msgr2=0x7f4d7c0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 --2- 172.21.15.37:0/4018599704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d7c073010 0x7f4d7c0cfa20 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0d0180 tx=0x7f4d7401c9f0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 -- 172.21.15.37:0/4018599704 shutdown_connections 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 --2- 172.21.15.37:0/4018599704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d7c073010 0x7f4d7c0cfa20 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 --2- 172.21.15.37:0/4018599704 >> v2:172.21.15.203:3300/0 conn(0x7f4d7c15b6b0 0x7f4d7c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 -- 172.21.15.37:0/4018599704 >> 172.21.15.37:0/4018599704 conn(0x7f4d7c06ae60 msgr2=0x7f4d7c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 -- 172.21.15.37:0/4018599704 shutdown_connections 2024-04-25T08:56:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 -- 172.21.15.37:0/4018599704 wait complete. 2024-04-25T08:56:37.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 Processor -- start 2024-04-25T08:56:37.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 -- start start 2024-04-25T08:56:37.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d7c073010 0x7f4d7c097dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d7c0cf590 0x7f4d7c164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d7c15b6b0 0x7f4d7c164bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0cadb0 con 0x7f4d7c0cf590 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0ca900 con 0x7f4d7c073010 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d8276a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c0caf30 con 0x7f4d7c15b6b0 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d7c0cf590 0x7f4d7c164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d7c0cf590 0x7f4d7c164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43604/0 (socket says 172.21.15.37:43604) 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7b7fe640 1 -- 172.21.15.37:0/51754362 learned_addr learned my addr 172.21.15.37:0/51754362 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7bfff640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d7c073010 0x7f4d7c097dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d80d07640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d7c15b6b0 0x7f4d7c164bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7b7fe640 1 -- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d7c15b6b0 msgr2=0x7f4d7c164bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7b7fe640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d7c15b6b0 0x7f4d7c164bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7b7fe640 1 -- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d7c073010 msgr2=0x7f4d7c097dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7b7fe640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d7c073010 0x7f4d7c097dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d7b7fe640 1 -- 172.21.15.37:0/51754362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c098300 con 0x7f4d7c0cf590 2024-04-25T08:56:37.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.162+0000 7f4d80d07640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d7c15b6b0 0x7f4d7c164bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:37.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d7b7fe640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d7c0cf590 0x7f4d7c164670 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000c420 tx=0x7f4d7000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:37.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d78ff9640 1 -- 172.21.15.37:0/51754362 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d700052d0 con 0x7f4d7c0cf590 2024-04-25T08:56:37.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d78ff9640 1 -- 172.21.15.37:0/51754362 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d70005470 con 0x7f4d7c0cf590 2024-04-25T08:56:37.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d8276a640 1 -- 172.21.15.37:0/51754362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7c0b20d0 con 0x7f4d7c0cf590 2024-04-25T08:56:37.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d78ff9640 1 -- 172.21.15.37:0/51754362 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70007260 con 0x7f4d7c0cf590 2024-04-25T08:56:37.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d8276a640 1 -- 172.21.15.37:0/51754362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7c15a7c0 con 0x7f4d7c0cf590 2024-04-25T08:56:37.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d8276a640 1 -- 172.21.15.37:0/51754362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7c15a2d0 con 0x7f4d7c0cf590 2024-04-25T08:56:37.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d78ff9640 1 -- 172.21.15.37:0/51754362 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4d70007400 con 0x7f4d7c0cf590 2024-04-25T08:56:37.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d78ff9640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4d68066f90 0x7f4d68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d78ff9640 1 -- 172.21.15.37:0/51754362 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f4d700847d0 con 0x7f4d7c0cf590 2024-04-25T08:56:37.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d7bfff640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4d68066f90 0x7f4d68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.166+0000 7f4d7bfff640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4d68066f90 0x7f4d68069450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c006fd0 tx=0x7f4d6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.170+0000 7f4d78ff9640 1 -- 172.21.15.37:0/51754362 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4d70055540 con 0x7f4d7c0cf590 2024-04-25T08:56:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.330+0000 7f4d8276a640 1 -- 172.21.15.37:0/51754362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4d7c15a990 con 0x7f4d7c0cf590 2024-04-25T08:56:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.330+0000 7f4d78ff9640 1 -- 172.21.15.37:0/51754362 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4d70054e40 con 0x7f4d7c0cf590 2024-04-25T08:56:37.323 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:56:37.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 -- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4d68066f90 msgr2=0x7f4d68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4d68066f90 0x7f4d68069450 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c006fd0 tx=0x7f4d6c008040 comp rx=0 tx=0).stop 2024-04-25T08:56:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 -- 172.21.15.37:0/51754362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d7c0cf590 msgr2=0x7f4d7c164670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d7c0cf590 0x7f4d7c164670 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000c420 tx=0x7f4d7000c8e0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 -- 172.21.15.37:0/51754362 shutdown_connections 2024-04-25T08:56:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4d68066f90 0x7f4d68069450 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4d7c15b6b0 0x7f4d7c164bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d7c0cf590 0x7f4d7c164670 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 --2- 172.21.15.37:0/51754362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4d7c073010 0x7f4d7c097dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 -- 172.21.15.37:0/51754362 >> 172.21.15.37:0/51754362 conn(0x7f4d7c06ae60 msgr2=0x7f4d7c1562e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 -- 172.21.15.37:0/51754362 shutdown_connections 2024-04-25T08:56:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.334+0000 7f4d8276a640 1 -- 172.21.15.37:0/51754362 wait complete. 2024-04-25T08:56:37.341 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:56:37.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fcc1e2640 1 Processor -- start 2024-04-25T08:56:37.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fcc1e2640 1 -- start start 2024-04-25T08:56:37.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fcc1e2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9fc415b950 0x7f9fc409a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fcc1e2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc415bd30 con 0x7f9fc40cfb80 2024-04-25T08:56:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fcc1e2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fc40cc0d0 con 0x7f9fc40cf420 2024-04-25T08:56:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fcc1e2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fc409ab10 con 0x7f9fc415b950 2024-04-25T08:56:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fc9f7e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9fc415b950 0x7f9fc409a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fca77f640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9fc40cfb80 0x7f9fc40cff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38866/0 (socket says 172.21.15.37:38866) 2024-04-25T08:56:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fca77f640 1 -- 172.21.15.37:0/3032676709 learned_addr learned my addr 172.21.15.37:0/3032676709 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fc9f7e640 1 -- 172.21.15.37:0/3032676709 >> v1:172.21.15.203:6789/0 conn(0x7f9fc40cf420 legacy=0x7f9fc40cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fc9f7e640 1 -- 172.21.15.37:0/3032676709 >> v1:172.21.15.37:6789/0 conn(0x7f9fc40cfb80 legacy=0x7f9fc40cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.486+0000 7f9fc9f7e640 1 -- 172.21.15.37:0/3032676709 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40cc370 con 0x7f9fc415b950 2024-04-25T08:56:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc9f7e640 1 --2- 172.21.15.37:0/3032676709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc415b950 0x7f9fc409a480 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00097c0 tx=0x7f9fc001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d0c24db5c66ea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc8f7c640 1 -- 172.21.15.37:0/3032676709 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00b1070 con 0x7f9fc415b950 2024-04-25T08:56:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc8f7c640 1 -- 172.21.15.37:0/3032676709 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc00ac5f0 con 0x7f9fc415b950 2024-04-25T08:56:37.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc8f7c640 1 -- 172.21.15.37:0/3032676709 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00ac9c0 con 0x7f9fc415b950 2024-04-25T08:56:37.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/3032676709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc415b950 msgr2=0x7f9fc409a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 --2- 172.21.15.37:0/3032676709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc415b950 0x7f9fc409a480 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00097c0 tx=0x7f9fc001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/3032676709 shutdown_connections 2024-04-25T08:56:37.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 --2- 172.21.15.37:0/3032676709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc415b950 0x7f9fc409a480 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/3032676709 >> 172.21.15.37:0/3032676709 conn(0x7f9fc40b30b0 msgr2=0x7f9fc40b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:37.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/3032676709 shutdown_connections 2024-04-25T08:56:37.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/3032676709 wait complete. 2024-04-25T08:56:37.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 Processor -- start 2024-04-25T08:56:37.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 -- start start 2024-04-25T08:56:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fc40cf420 0x7f9fc41526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc40cfb80 0x7f9fc40b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fc415b950 0x7f9fc406fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc406fc20 con 0x7f9fc40cfb80 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc406faa0 con 0x7f9fc40cf420 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fcc1e2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fc4169780 con 0x7f9fc415b950 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc977d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc40cfb80 0x7f9fc40b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc977d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc40cfb80 0x7f9fc40b6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43628/0 (socket says 172.21.15.37:43628) 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc977d640 1 -- 172.21.15.37:0/2415493825 learned_addr learned my addr 172.21.15.37:0/2415493825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc9f7e640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fc40cf420 0x7f9fc41526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fca77f640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fc415b950 0x7f9fc406fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc977d640 1 -- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fc415b950 msgr2=0x7f9fc406fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc977d640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fc415b950 0x7f9fc406fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc977d640 1 -- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fc40cf420 msgr2=0x7f9fc41526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc977d640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fc40cf420 0x7f9fc41526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc977d640 1 -- 172.21.15.37:0/2415493825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40b5a50 con 0x7f9fc40cfb80 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fca77f640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fc415b950 0x7f9fc406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.490+0000 7f9fc9f7e640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fc40cf420 0x7f9fc41526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fc977d640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc40cfb80 0x7f9fc40b6810 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400c3c0 tx=0x7f9fb400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fba7fc640 1 -- 172.21.15.37:0/2415493825 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4017070 con 0x7f9fc40cfb80 2024-04-25T08:56:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fba7fc640 1 -- 172.21.15.37:0/2415493825 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb40052d0 con 0x7f9fc40cfb80 2024-04-25T08:56:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/2415493825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc40b5d30 con 0x7f9fc40cfb80 2024-04-25T08:56:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fba7fc640 1 -- 172.21.15.37:0/2415493825 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb4003840 con 0x7f9fc40cfb80 2024-04-25T08:56:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/2415493825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fc40b62f0 con 0x7f9fc40cfb80 2024-04-25T08:56:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/2415493825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fc404efa0 con 0x7f9fc40cfb80 2024-04-25T08:56:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fba7fc640 1 -- 172.21.15.37:0/2415493825 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9fb401a070 con 0x7f9fc40cfb80 2024-04-25T08:56:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fba7fc640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9fa8066f60 0x7f9fa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fba7fc640 1 -- 172.21.15.37:0/2415493825 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f9fb4084530 con 0x7f9fc40cfb80 2024-04-25T08:56:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fc9f7e640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9fa8066f60 0x7f9fa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.494+0000 7f9fc9f7e640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9fa8066f60 0x7f9fa8069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00ae000 tx=0x7f9fc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:37.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.498+0000 7f9fba7fc640 1 -- 172.21.15.37:0/2415493825 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9fb4005480 con 0x7f9fc40cfb80 2024-04-25T08:56:37.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.658+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/2415493825 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fc40d13a0 con 0x7f9fc40cfb80 2024-04-25T08:56:37.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.658+0000 7f9fba7fc640 1 -- 172.21.15.37:0/2415493825 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9fb4055340 con 0x7f9fc40cfb80 2024-04-25T08:56:37.649 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:56:37.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9fa8066f60 msgr2=0x7f9fa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9fa8066f60 0x7f9fa8069420 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00ae000 tx=0x7f9fc0008040 comp rx=0 tx=0).stop 2024-04-25T08:56:37.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/2415493825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc40cfb80 msgr2=0x7f9fc40b6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc40cfb80 0x7f9fc40b6810 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400c3c0 tx=0x7f9fb400c880 comp rx=0 tx=0).stop 2024-04-25T08:56:37.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/2415493825 shutdown_connections 2024-04-25T08:56:37.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9fa8066f60 0x7f9fa8069420 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fc415b950 0x7f9fc406fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fc40cfb80 0x7f9fc40b6810 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 --2- 172.21.15.37:0/2415493825 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fc40cf420 0x7f9fc41526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/2415493825 >> 172.21.15.37:0/2415493825 conn(0x7f9fc40b30b0 msgr2=0x7f9fc40ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:37.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/2415493825 shutdown_connections 2024-04-25T08:56:37.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.662+0000 7f9fcc1e2640 1 -- 172.21.15.37:0/2415493825 wait complete. 2024-04-25T08:56:37.668 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:56:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6527598640 1 Processor -- start 2024-04-25T08:56:37.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6527598640 1 -- start start 2024-04-25T08:56:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6527598640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652009a180 0x7f65200cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6527598640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f65200afb90 0x7f65200cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6527598640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652009a950 con 0x7f652015b920 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6527598640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f652009ac30 con 0x7f65200afb90 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6527598640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65200ae380 con 0x7f652009a180 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6525334640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f652015b920 0x7f6520099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38880/0 (socket says 172.21.15.37:38880) 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6525334640 1 -- 172.21.15.37:0/1045409297 learned_addr learned my addr 172.21.15.37:0/1045409297 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6524b33640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652009a180 0x7f65200cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6525b35640 1 --2- 172.21.15.37:0/1045409297 >> v2:172.21.15.203:3300/0 conn(0x7f65200afb90 0x7f65200cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601456903 0 0) 0x7f652009a950 con 0x7f652015b920 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6508003530 con 0x7f652015b920 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3910477697 0 0) 0x7f6508003530 con 0x7f652015b920 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652009a180 msgr2=0x7f65200cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652009a180 0x7f65200cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 >> v2:172.21.15.203:3300/0 conn(0x7f65200afb90 msgr2=0x7f65200cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 --2- 172.21.15.37:0/1045409297 >> v2:172.21.15.203:3300/0 conn(0x7f65200afb90 0x7f65200cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200ae660 con 0x7f652015b920 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f6518001a00 con 0x7f652015b920 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 >> v1:172.21.15.37:6789/0 conn(0x7f652015b920 legacy=0x7f6520099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 0x7f6508003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6508004810 0x7f6508004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652009a180 0x7f65080051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f652009ac30 con 0x7f6508003a30 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65200ae380 con 0x7f6508004810 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650800bab0 con 0x7f652009a180 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6525334640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 0x7f6508003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6524b33640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6508004810 0x7f6508004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6525b35640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652009a180 0x7f65080051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6525334640 1 -- 172.21.15.37:0/1045409297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652009a180 msgr2=0x7f65080051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6525334640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f652009a180 0x7f65080051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6525334640 1 -- 172.21.15.37:0/1045409297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6508004810 msgr2=0x7f6508004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6525334640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6508004810 0x7f6508004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.806+0000 7f6525334640 1 -- 172.21.15.37:0/1045409297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6508003530 con 0x7f6508003a30 2024-04-25T08:56:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.810+0000 7f6525334640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 0x7f6508003e50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f651800a8a0 tx=0x7f651801d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e49ff20fef5cba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.810+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651800a4c0 con 0x7f6508003a30 2024-04-25T08:56:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.810+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65180ad650 con 0x7f6508003a30 2024-04-25T08:56:37.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.810+0000 7f651ffff640 1 -- 172.21.15.37:0/1045409297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65180a5e60 con 0x7f6508003a30 2024-04-25T08:56:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.810+0000 7f6527598640 1 -- 172.21.15.37:0/1045409297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 msgr2=0x7f6508003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.810+0000 7f6527598640 1 --2- 172.21.15.37:0/1045409297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 0x7f6508003e50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f651800a8a0 tx=0x7f651801d300 comp rx=0 tx=0).stop 2024-04-25T08:56:37.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.810+0000 7f6525b35640 1 -- 172.21.15.37:0/1045409297 reap_dead start 2024-04-25T08:56:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.810+0000 7f6527598640 1 -- 172.21.15.37:0/1045409297 shutdown_connections 2024-04-25T08:56:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.810+0000 7f6527598640 1 -- 172.21.15.37:0/1045409297 >> 172.21.15.37:0/1045409297 conn(0x7f65200b3010 msgr2=0x7f65200b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.810+0000 7f6527598640 1 -- 172.21.15.37:0/1045409297 shutdown_connections 2024-04-25T08:56:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.810+0000 7f6527598640 1 -- 172.21.15.37:0/1045409297 wait complete. 2024-04-25T08:56:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6527598640 1 Processor -- start 2024-04-25T08:56:37.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6527598640 1 -- start start 2024-04-25T08:56:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6527598640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 0x7f65201fa410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6527598640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6520156fa0 0x7f652015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6527598640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65200703f0 0x7f65200708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6527598640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65201664b0 con 0x7f6508003a30 2024-04-25T08:56:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6527598640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65200cd5d0 con 0x7f65200703f0 2024-04-25T08:56:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6527598640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6520166630 con 0x7f6520156fa0 2024-04-25T08:56:37.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6525334640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 0x7f65201fa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6525334640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 0x7f65201fa410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43648/0 (socket says 172.21.15.37:43648) 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6525334640 1 -- 172.21.15.37:0/4146972387 learned_addr learned my addr 172.21.15.37:0/4146972387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6524b33640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6520156fa0 0x7f652015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6525b35640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65200703f0 0x7f65200708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6525334640 1 -- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6520156fa0 msgr2=0x7f652015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6525334640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6520156fa0 0x7f652015d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6525334640 1 -- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65200703f0 msgr2=0x7f65200708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6525334640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65200703f0 0x7f65200708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6525334640 1 -- 172.21.15.37:0/4146972387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65201fad10 con 0x7f6508003a30 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6525334640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 0x7f65201fa410 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f651800a8a0 tx=0x7f6508005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f651dffb640 1 -- 172.21.15.37:0/4146972387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65180ad430 con 0x7f6508003a30 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f651dffb640 1 -- 172.21.15.37:0/4146972387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6518004b70 con 0x7f6508003a30 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6527598640 1 -- 172.21.15.37:0/4146972387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65201faff0 con 0x7f6508003a30 2024-04-25T08:56:37.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6527598640 1 -- 172.21.15.37:0/4146972387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f652006c1b0 con 0x7f6508003a30 2024-04-25T08:56:37.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f651dffb640 1 -- 172.21.15.37:0/4146972387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6518004d40 con 0x7f6508003a30 2024-04-25T08:56:37.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.814+0000 7f6527598640 1 -- 172.21.15.37:0/4146972387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65000052a0 con 0x7f6508003a30 2024-04-25T08:56:37.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.818+0000 7f651dffb640 1 -- 172.21.15.37:0/4146972387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f65180bd020 con 0x7f6508003a30 2024-04-25T08:56:37.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.818+0000 7f651dffb640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f90 0x7f650c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:37.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.818+0000 7f651dffb640 1 -- 172.21.15.37:0/4146972387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f651811f520 con 0x7f6508003a30 2024-04-25T08:56:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.818+0000 7f6524b33640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f90 0x7f650c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:37.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.818+0000 7f6524b33640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f90 0x7f650c069450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f652009a020 tx=0x7f6514008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:37.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.822+0000 7f651dffb640 1 -- 172.21.15.37:0/4146972387 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6518017010 con 0x7f6508003a30 2024-04-25T08:56:37.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.982+0000 7f6527598640 1 -- 172.21.15.37:0/4146972387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6500005860 con 0x7f6508003a30 2024-04-25T08:56:37.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.982+0000 7f651dffb640 1 -- 172.21.15.37:0/4146972387 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7f65180f03c0 con 0x7f6508003a30 2024-04-25T08:56:37.972 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.982+0000 7f6527598640 1 -- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f90 msgr2=0x7f650c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.982+0000 7f6527598640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f90 0x7f650c069450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f652009a020 tx=0x7f6514008040 comp rx=0 tx=0).stop 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.982+0000 7f6527598640 1 -- 172.21.15.37:0/4146972387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 msgr2=0x7f65201fa410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.982+0000 7f6527598640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 0x7f65201fa410 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f651800a8a0 tx=0x7f6508005840 comp rx=0 tx=0).stop 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.982+0000 7f6527598640 1 -- 172.21.15.37:0/4146972387 shutdown_connections 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.982+0000 7f6527598640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f650c066f90 0x7f650c069450 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.986+0000 7f6527598640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f65200703f0 0x7f65200708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.986+0000 7f6527598640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6520156fa0 0x7f652015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.986+0000 7f6527598640 1 --2- 172.21.15.37:0/4146972387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6508003a30 0x7f65201fa410 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.986+0000 7f6527598640 1 -- 172.21.15.37:0/4146972387 >> 172.21.15.37:0/4146972387 conn(0x7f65200b3010 msgr2=0x7f652006f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.986+0000 7f6527598640 1 -- 172.21.15.37:0/4146972387 shutdown_connections 2024-04-25T08:56:37.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:37.986+0000 7f6527598640 1 -- 172.21.15.37:0/4146972387 wait complete. 2024-04-25T08:56:37.992 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88cd8e640 1 Processor -- start 2024-04-25T08:56:38.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88cd8e640 1 -- start start 2024-04-25T08:56:38.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88cd8e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880d0870 0x7ff8880d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88cd8e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8880ce6b0 con 0x7ff88815dc30 2024-04-25T08:56:38.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88cd8e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8880ce990 con 0x7ff88815d850 2024-04-25T08:56:38.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88cd8e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880cec70 con 0x7ff8880d0870 2024-04-25T08:56:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff886d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff88815dc30 0x7ff888167910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38894/0 (socket says 172.21.15.37:38894) 2024-04-25T08:56:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff886d9d640 1 -- 172.21.15.37:0/2735203408 learned_addr learned my addr 172.21.15.37:0/2735203408 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff885d9b640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880d0870 0x7ff8880d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541747926 0 0) 0x7ff8880ce6b0 con 0x7ff88815dc30 2024-04-25T08:56:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff870003530 con 0x7ff88815dc30 2024-04-25T08:56:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1980170309 0 0) 0x7ff870003530 con 0x7ff88815dc30 2024-04-25T08:56:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880d0870 msgr2=0x7ff8880d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880d0870 0x7ff8880d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 >> v1:172.21.15.203:6789/0 conn(0x7ff88815d850 legacy=0x7ff8880d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8880cef50 con 0x7ff88815dc30 2024-04-25T08:56:38.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7ff880001a00 con 0x7ff88815dc30 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 >> v1:172.21.15.37:6789/0 conn(0x7ff88815dc30 legacy=0x7ff888167910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880d0870 0x7ff870003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff870003fe0 0x7ff870004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff870004d50 0x7ff8700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880cec70 con 0x7ff870004d50 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87000bb30 con 0x7ff870003fe0 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87000bd30 con 0x7ff8880d0870 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff886d9d640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff870004d50 0x7ff8700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff885d9b640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff870003fe0 0x7ff870004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88659c640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880d0870 0x7ff870003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff886d9d640 1 -- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880d0870 msgr2=0x7ff870003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff886d9d640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880d0870 0x7ff870003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff886d9d640 1 -- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff870003fe0 msgr2=0x7ff870004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff886d9d640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff870003fe0 0x7ff870004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff886d9d640 1 -- 172.21.15.37:0/2735203408 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff870003530 con 0x7ff870004d50 2024-04-25T08:56:38.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.126+0000 7ff88659c640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880d0870 0x7ff870003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.130+0000 7ff886d9d640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff870004d50 0x7ff8700051d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff88000a850 tx=0x7ff88001d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a789f565867698e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.130+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff88000a4a0 con 0x7ff870004d50 2024-04-25T08:56:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.130+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8800ad650 con 0x7ff870004d50 2024-04-25T08:56:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.130+0000 7ff88559a640 1 -- 172.21.15.37:0/2735203408 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8800ada80 con 0x7ff870004d50 2024-04-25T08:56:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.130+0000 7ff88cd8e640 1 -- 172.21.15.37:0/2735203408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff870004d50 msgr2=0x7ff8700051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.130+0000 7ff88cd8e640 1 --2- 172.21.15.37:0/2735203408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff870004d50 0x7ff8700051d0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7ff88000a850 tx=0x7ff88001d3a0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.130+0000 7ff886d9d640 1 -- 172.21.15.37:0/2735203408 reap_dead start 2024-04-25T08:56:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.130+0000 7ff88cd8e640 1 -- 172.21.15.37:0/2735203408 shutdown_connections 2024-04-25T08:56:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.130+0000 7ff88cd8e640 1 -- 172.21.15.37:0/2735203408 >> 172.21.15.37:0/2735203408 conn(0x7ff88808b000 msgr2=0x7ff8880d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.130+0000 7ff88cd8e640 1 -- 172.21.15.37:0/2735203408 shutdown_connections 2024-04-25T08:56:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.130+0000 7ff88cd8e640 1 -- 172.21.15.37:0/2735203408 wait complete. 2024-04-25T08:56:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88cd8e640 1 Processor -- start 2024-04-25T08:56:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88cd8e640 1 -- start start 2024-04-25T08:56:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88cd8e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff88815a5f0 0x7ff8880ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88cd8e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff88815a9d0 0x7ff8880ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88cd8e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880cad80 0x7ff8880cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88cd8e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880ce490 con 0x7ff88815a5f0 2024-04-25T08:56:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88cd8e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880ce310 con 0x7ff88815a9d0 2024-04-25T08:56:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88cd8e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8880d1de0 con 0x7ff8880cad80 2024-04-25T08:56:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff88815a5f0 0x7ff8880ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff88815a5f0 0x7ff8880ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43676/0 (socket says 172.21.15.37:43676) 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88659c640 1 -- 172.21.15.37:0/223288829 learned_addr learned my addr 172.21.15.37:0/223288829 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff885d9b640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff88815a9d0 0x7ff8880ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff886d9d640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880cad80 0x7ff8880cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88659c640 1 -- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880cad80 msgr2=0x7ff8880cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88659c640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880cad80 0x7ff8880cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88659c640 1 -- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff88815a9d0 msgr2=0x7ff8880ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88659c640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff88815a9d0 0x7ff8880ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88659c640 1 -- 172.21.15.37:0/223288829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8880cb290 con 0x7ff88815a5f0 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88659c640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff88815a5f0 0x7ff8880ca840 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e7e0 tx=0x7ff87c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff876ffd640 1 -- 172.21.15.37:0/223288829 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c0050d0 con 0x7ff88815a5f0 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff876ffd640 1 -- 172.21.15.37:0/223288829 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87c018070 con 0x7ff88815a5f0 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff876ffd640 1 -- 172.21.15.37:0/223288829 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87c00c6c0 con 0x7ff88815a5f0 2024-04-25T08:56:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88cd8e640 1 -- 172.21.15.37:0/223288829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8880cb570 con 0x7ff88815a5f0 2024-04-25T08:56:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88cd8e640 1 -- 172.21.15.37:0/223288829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8880cbb30 con 0x7ff88815a5f0 2024-04-25T08:56:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.134+0000 7ff88cd8e640 1 -- 172.21.15.37:0/223288829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff88804efa0 con 0x7ff88815a5f0 2024-04-25T08:56:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.138+0000 7ff876ffd640 1 -- 172.21.15.37:0/223288829 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff87c00c860 con 0x7ff88815a5f0 2024-04-25T08:56:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.138+0000 7ff876ffd640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff868066f60 0x7ff868069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.138+0000 7ff876ffd640 1 -- 172.21.15.37:0/223288829 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff87c084fe0 con 0x7ff88815a5f0 2024-04-25T08:56:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.138+0000 7ff885d9b640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff868066f60 0x7ff868069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.138+0000 7ff885d9b640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff868066f60 0x7ff868069420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff8780099b0 tx=0x7ff878008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.142+0000 7ff876ffd640 1 -- 172.21.15.37:0/223288829 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff87c055eb0 con 0x7ff88815a5f0 2024-04-25T08:56:38.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.334+0000 7ff88cd8e640 1 -- 172.21.15.37:0/223288829 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8880cdc30 con 0x7ff88815a5f0 2024-04-25T08:56:38.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.334+0000 7ff876ffd640 1 -- 172.21.15.37:0/223288829 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7ff87c0557b0 con 0x7ff88815a5f0 2024-04-25T08:56:38.325 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:38.325 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/486740526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":486740526},{"type":"v1","addr":"172.21.15.37:6833","nonce":486740526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/121484331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":121484331},{"type":"v1","addr":"172.21.15.203:6835","nonce":121484331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2501630566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2501630566},{"type":"v1","addr":"172.21.15.203:6837","nonce":2501630566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:55:46.200939+0000","modified":"2024-04-25T08:56:06.871679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6405":{"gid":6405,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":8,"addr":"172.21.15.37:6835/2189232625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2189232625},{"type":"v1","addr":"172.21.15.37:6835","nonce":2189232625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:56:38.325 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-04-25T08:56:38.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 -- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff868066f60 msgr2=0x7ff868069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff868066f60 0x7ff868069420 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff8780099b0 tx=0x7ff878008040 comp rx=0 tx=0).stop 2024-04-25T08:56:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 -- 172.21.15.37:0/223288829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff88815a5f0 msgr2=0x7ff8880ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff88815a5f0 0x7ff8880ca840 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ff87c00e7e0 tx=0x7ff87c00eca0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 -- 172.21.15.37:0/223288829 shutdown_connections 2024-04-25T08:56:38.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff868066f60 0x7ff868069420 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8880cad80 0x7ff8880cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff88815a9d0 0x7ff8880ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 --2- 172.21.15.37:0/223288829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff88815a5f0 0x7ff8880ca840 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 -- 172.21.15.37:0/223288829 >> 172.21.15.37:0/223288829 conn(0x7ff88808b000 msgr2=0x7ff8880b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 -- 172.21.15.37:0/223288829 shutdown_connections 2024-04-25T08:56:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.338+0000 7ff88cd8e640 1 -- 172.21.15.37:0/223288829 wait complete. 2024-04-25T08:56:38.344 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:38.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68d1719640 1 Processor -- start 2024-04-25T08:56:38.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68d1719640 1 -- start start 2024-04-25T08:56:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68d1719640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f68cc06b3a0 0x7f68cc0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68d1719640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68cc15c260 con 0x7f68cc0cc1a0 2024-04-25T08:56:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68d1719640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68cc15c540 con 0x7f68cc15ba20 2024-04-25T08:56:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68d1719640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc0d1430 con 0x7f68cc06b3a0 2024-04-25T08:56:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68ca7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f68cc0cc1a0 0x7f68cc06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38896/0 (socket says 172.21.15.37:38896) 2024-04-25T08:56:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68ca7fc640 1 -- 172.21.15.37:0/1442085973 learned_addr learned my addr 172.21.15.37:0/1442085973 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68cb7fe640 1 --2- 172.21.15.37:0/1442085973 >> v2:172.21.15.37:3300/0 conn(0x7f68cc06b3a0 0x7f68cc0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68c9ffb640 1 -- 172.21.15.37:0/1442085973 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849421242 0 0) 0x7f68cc15c260 con 0x7f68cc0cc1a0 2024-04-25T08:56:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68c9ffb640 1 -- 172.21.15.37:0/1442085973 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68b4003530 con 0x7f68cc0cc1a0 2024-04-25T08:56:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68cb7fe640 1 -- 172.21.15.37:0/1442085973 >> v1:172.21.15.203:6789/0 conn(0x7f68cc15ba20 legacy=0x7f68cc15fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68cb7fe640 1 -- 172.21.15.37:0/1442085973 >> v1:172.21.15.37:6789/0 conn(0x7f68cc0cc1a0 legacy=0x7f68cc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.466+0000 7f68cb7fe640 1 -- 172.21.15.37:0/1442085973 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc15c820 con 0x7f68cc06b3a0 2024-04-25T08:56:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68cb7fe640 1 --2- 172.21.15.37:0/1442085973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68cc06b3a0 0x7f68cc0cf6d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009d90 tx=0x7f68c401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47050321180e19fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68c9ffb640 1 -- 172.21.15.37:0/1442085973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c40b2070 con 0x7f68cc06b3a0 2024-04-25T08:56:38.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68c9ffb640 1 -- 172.21.15.37:0/1442085973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c40a5940 con 0x7f68cc06b3a0 2024-04-25T08:56:38.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68c9ffb640 1 -- 172.21.15.37:0/1442085973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c40a5d10 con 0x7f68cc06b3a0 2024-04-25T08:56:38.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 -- 172.21.15.37:0/1442085973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68cc06b3a0 msgr2=0x7f68cc0cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 --2- 172.21.15.37:0/1442085973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68cc06b3a0 0x7f68cc0cf6d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009d90 tx=0x7f68c401c970 comp rx=0 tx=0).stop 2024-04-25T08:56:38.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 -- 172.21.15.37:0/1442085973 shutdown_connections 2024-04-25T08:56:38.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 --2- 172.21.15.37:0/1442085973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68cc06b3a0 0x7f68cc0cf6d0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 -- 172.21.15.37:0/1442085973 >> 172.21.15.37:0/1442085973 conn(0x7f68cc099a70 msgr2=0x7f68cc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:38.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 -- 172.21.15.37:0/1442085973 shutdown_connections 2024-04-25T08:56:38.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 -- 172.21.15.37:0/1442085973 wait complete. 2024-04-25T08:56:38.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 Processor -- start 2024-04-25T08:56:38.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 -- start start 2024-04-25T08:56:38.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68cc06b3a0 0x7f68cc0b0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68cc0cc1a0 0x7f68cc06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68cc15ba20 0x7f68cc155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc1698b0 con 0x7f68cc15ba20 2024-04-25T08:56:38.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc169730 con 0x7f68cc06b3a0 2024-04-25T08:56:38.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.470+0000 7f68d1719640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc169a30 con 0x7f68cc0cc1a0 2024-04-25T08:56:38.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68cb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68cc15ba20 0x7f68cc155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68cb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68cc15ba20 0x7f68cc155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43704/0 (socket says 172.21.15.37:43704) 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68cb7fe640 1 -- 172.21.15.37:0/3662640237 learned_addr learned my addr 172.21.15.37:0/3662640237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68caffd640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68cc06b3a0 0x7f68cc0b0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68ca7fc640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68cc0cc1a0 0x7f68cc06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68cb7fe640 1 -- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68cc0cc1a0 msgr2=0x7f68cc06f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68cb7fe640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68cc0cc1a0 0x7f68cc06f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68cb7fe640 1 -- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68cc06b3a0 msgr2=0x7f68cc0b0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68cb7fe640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68cc06b3a0 0x7f68cc0b0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68cb7fe640 1 -- 172.21.15.37:0/3662640237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc156080 con 0x7f68cc15ba20 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68ca7fc640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68cc0cc1a0 0x7f68cc06f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68cb7fe640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68cc15ba20 0x7f68cc155590 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f68c401a5a0 tx=0x7f68c4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68b37fe640 1 -- 172.21.15.37:0/3662640237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c40b2040 con 0x7f68cc15ba20 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68b37fe640 1 -- 172.21.15.37:0/3662640237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c40bf070 con 0x7f68cc15ba20 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68b37fe640 1 -- 172.21.15.37:0/3662640237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c40b8890 con 0x7f68cc15ba20 2024-04-25T08:56:38.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68d1719640 1 -- 172.21.15.37:0/3662640237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68cc0b0850 con 0x7f68cc15ba20 2024-04-25T08:56:38.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68d1719640 1 -- 172.21.15.37:0/3662640237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68cc0c56a0 con 0x7f68cc15ba20 2024-04-25T08:56:38.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68d1719640 1 -- 172.21.15.37:0/3662640237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68cc0b0d70 con 0x7f68cc15ba20 2024-04-25T08:56:38.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68b37fe640 1 -- 172.21.15.37:0/3662640237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f68c40a57c0 con 0x7f68cc15ba20 2024-04-25T08:56:38.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68b37fe640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f68b8066f60 0x7f68b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68b37fe640 1 -- 172.21.15.37:0/3662640237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f68c41204e0 con 0x7f68cc15ba20 2024-04-25T08:56:38.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68caffd640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f68b8066f60 0x7f68b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.474+0000 7f68caffd640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f68b8066f60 0x7f68b8069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0b2f40 tx=0x7f68bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:38.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.478+0000 7f68b37fe640 1 -- 172.21.15.37:0/3662640237 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f68cc156080 con 0x7f68cc15ba20 2024-04-25T08:56:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.630+0000 7f68d1719640 1 -- 172.21.15.37:0/3662640237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68cc04efa0 con 0x7f68cc15ba20 2024-04-25T08:56:38.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68b37fe640 1 -- 172.21.15.37:0/3662640237 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17040 (secure 0 0 0) 0x7f68c4016560 con 0x7f68cc15ba20 2024-04-25T08:56:38.623 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:38.623 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":71,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:56:12.356132+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:44.492629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:45.203787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.9","mappings":[{"from":7,"to":2}]},{"pgid":"10.39","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:38.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 -- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f68b8066f60 msgr2=0x7f68b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f68b8066f60 0x7f68b8069420 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0b2f40 tx=0x7f68bc009040 comp rx=0 tx=0).stop 2024-04-25T08:56:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 -- 172.21.15.37:0/3662640237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68cc15ba20 msgr2=0x7f68cc155590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68cc15ba20 0x7f68cc155590 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f68c401a5a0 tx=0x7f68c4019710 comp rx=0 tx=0).stop 2024-04-25T08:56:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 -- 172.21.15.37:0/3662640237 shutdown_connections 2024-04-25T08:56:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f68b8066f60 0x7f68b8069420 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68cc15ba20 0x7f68cc155590 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f68cc0cc1a0 0x7f68cc06f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 --2- 172.21.15.37:0/3662640237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f68cc06b3a0 0x7f68cc0b0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 -- 172.21.15.37:0/3662640237 >> 172.21.15.37:0/3662640237 conn(0x7f68cc099a70 msgr2=0x7f68cc0ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 -- 172.21.15.37:0/3662640237 shutdown_connections 2024-04-25T08:56:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.634+0000 7f68d1719640 1 -- 172.21.15.37:0/3662640237 wait complete. 2024-04-25T08:56:38.642 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:56:38.642 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff137a2e640 1 Processor -- start 2024-04-25T08:56:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff137a2e640 1 -- start start 2024-04-25T08:56:38.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff137a2e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff1300b1ae0 0x7ff130151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff137a2e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff130160310 con 0x7ff1300cc820 2024-04-25T08:56:38.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff137a2e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff13015f550 con 0x7ff1300b0b00 2024-04-25T08:56:38.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff137a2e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13015cfa0 con 0x7ff1300b1ae0 2024-04-25T08:56:38.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff134fc9640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff1300cc820 0x7ff1300b13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38908/0 (socket says 172.21.15.37:38908) 2024-04-25T08:56:38.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff134fc9640 1 -- 172.21.15.37:0/2972611835 learned_addr learned my addr 172.21.15.37:0/2972611835 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:38.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff135fcb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff1300b1ae0 0x7ff130151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff127fff640 1 -- 172.21.15.37:0/2972611835 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679951613 0 0) 0x7ff130160310 con 0x7ff1300cc820 2024-04-25T08:56:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff127fff640 1 -- 172.21.15.37:0/2972611835 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff11c003530 con 0x7ff1300cc820 2024-04-25T08:56:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff135fcb640 1 -- 172.21.15.37:0/2972611835 >> v1:172.21.15.203:6789/0 conn(0x7ff1300b0b00 legacy=0x7ff1300cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff135fcb640 1 -- 172.21.15.37:0/2972611835 >> v1:172.21.15.37:6789/0 conn(0x7ff1300cc820 legacy=0x7ff1300b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.774+0000 7ff135fcb640 1 -- 172.21.15.37:0/2972611835 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13015f830 con 0x7ff1300b1ae0 2024-04-25T08:56:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff135fcb640 1 --2- 172.21.15.37:0/2972611835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1300b1ae0 0x7ff130151fd0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0097c0 tx=0x7ff12c01c2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd26227930cf0e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff127fff640 1 -- 172.21.15.37:0/2972611835 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0b0070 con 0x7ff1300b1ae0 2024-04-25T08:56:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff127fff640 1 -- 172.21.15.37:0/2972611835 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c0ab600 con 0x7ff1300b1ae0 2024-04-25T08:56:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff127fff640 1 -- 172.21.15.37:0/2972611835 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c0ab9d0 con 0x7ff1300b1ae0 2024-04-25T08:56:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff137a2e640 1 -- 172.21.15.37:0/2972611835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1300b1ae0 msgr2=0x7ff130151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff137a2e640 1 --2- 172.21.15.37:0/2972611835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1300b1ae0 0x7ff130151fd0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0097c0 tx=0x7ff12c01c2c0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff137a2e640 1 -- 172.21.15.37:0/2972611835 shutdown_connections 2024-04-25T08:56:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff137a2e640 1 --2- 172.21.15.37:0/2972611835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1300b1ae0 0x7ff130151fd0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff137a2e640 1 -- 172.21.15.37:0/2972611835 >> 172.21.15.37:0/2972611835 conn(0x7ff1300b1f10 msgr2=0x7ff1300b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff137a2e640 1 -- 172.21.15.37:0/2972611835 shutdown_connections 2024-04-25T08:56:38.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff137a2e640 1 -- 172.21.15.37:0/2972611835 wait complete. 2024-04-25T08:56:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.778+0000 7ff137a2e640 1 Processor -- start 2024-04-25T08:56:38.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff137a2e640 1 -- start start 2024-04-25T08:56:38.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff137a2e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1300b0b00 0x7ff1300c3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff137a2e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1300b1ae0 0x7ff130149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff137a2e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff130147690 0x7ff13014b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff137a2e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130169900 con 0x7ff1300b1ae0 2024-04-25T08:56:38.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff137a2e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130169780 con 0x7ff1300b0b00 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff137a2e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff130169a80 con 0x7ff130147690 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff134fc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1300b1ae0 0x7ff130149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff134fc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1300b1ae0 0x7ff130149be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43724/0 (socket says 172.21.15.37:43724) 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff134fc9640 1 -- 172.21.15.37:0/1648128079 learned_addr learned my addr 172.21.15.37:0/1648128079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff1357ca640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1300b0b00 0x7ff1300c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff135fcb640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff130147690 0x7ff13014b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff134fc9640 1 -- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff130147690 msgr2=0x7ff13014b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff134fc9640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff130147690 0x7ff13014b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff134fc9640 1 -- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1300b0b00 msgr2=0x7ff1300c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff134fc9640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1300b0b00 0x7ff1300c3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff134fc9640 1 -- 172.21.15.37:0/1648128079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff130150310 con 0x7ff1300b1ae0 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff134fc9640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1300b1ae0 0x7ff130149be0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff12000c420 tx=0x7ff12000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff125ffb640 1 -- 172.21.15.37:0/1648128079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff120017070 con 0x7ff1300b1ae0 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff125ffb640 1 -- 172.21.15.37:0/1648128079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff120005cf0 con 0x7ff1300b1ae0 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff137a2e640 1 -- 172.21.15.37:0/1648128079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff130149700 con 0x7ff1300b1ae0 2024-04-25T08:56:38.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff125ffb640 1 -- 172.21.15.37:0/1648128079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1200072d0 con 0x7ff1300b1ae0 2024-04-25T08:56:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff137a2e640 1 -- 172.21.15.37:0/1648128079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1300cfd50 con 0x7ff1300b1ae0 2024-04-25T08:56:38.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff137a2e640 1 -- 172.21.15.37:0/1648128079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13014ce50 con 0x7ff1300b1ae0 2024-04-25T08:56:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff125ffb640 1 -- 172.21.15.37:0/1648128079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff12001a070 con 0x7ff1300b1ae0 2024-04-25T08:56:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff125ffb640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff114066f60 0x7ff114069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff125ffb640 1 -- 172.21.15.37:0/1648128079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7ff120084470 con 0x7ff1300b1ae0 2024-04-25T08:56:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.782+0000 7ff1357ca640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff114066f60 0x7ff114069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:38.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.786+0000 7ff1357ca640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff114066f60 0x7ff114069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff1300c66a0 tx=0x7ff128009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:38.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.786+0000 7ff125ffb640 1 -- 172.21.15.37:0/1648128079 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff120054cc0 con 0x7ff1300b1ae0 2024-04-25T08:56:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.982+0000 7ff137a2e640 1 -- 172.21.15.37:0/1648128079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff13004efa0 con 0x7ff1300b1ae0 2024-04-25T08:56:38.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff125ffb640 1 -- 172.21.15.37:0/1648128079 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7ff120058d60 con 0x7ff1300b1ae0 2024-04-25T08:56:38.975 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:38.975 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/486740526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":486740526},{"type":"v1","addr":"172.21.15.37:6833","nonce":486740526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/121484331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":121484331},{"type":"v1","addr":"172.21.15.203:6835","nonce":121484331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2501630566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2501630566},{"type":"v1","addr":"172.21.15.203:6837","nonce":2501630566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:55:46.200939+0000","modified":"2024-04-25T08:56:06.871679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6405":{"gid":6405,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":8,"addr":"172.21.15.37:6835/2189232625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2189232625},{"type":"v1","addr":"172.21.15.37:6835","nonce":2189232625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:56:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-04-25T08:56:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 -- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff114066f60 msgr2=0x7ff114069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff114066f60 0x7ff114069420 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ff1300c66a0 tx=0x7ff128009040 comp rx=0 tx=0).stop 2024-04-25T08:56:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 -- 172.21.15.37:0/1648128079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1300b1ae0 msgr2=0x7ff130149be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1300b1ae0 0x7ff130149be0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff12000c420 tx=0x7ff12000c8e0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 -- 172.21.15.37:0/1648128079 shutdown_connections 2024-04-25T08:56:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff114066f60 0x7ff114069420 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff130147690 0x7ff13014b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1300b1ae0 0x7ff130149be0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 --2- 172.21.15.37:0/1648128079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff1300b0b00 0x7ff1300c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 -- 172.21.15.37:0/1648128079 >> 172.21.15.37:0/1648128079 conn(0x7ff1300b1f10 msgr2=0x7ff13014ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 -- 172.21.15.37:0/1648128079 shutdown_connections 2024-04-25T08:56:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:38.986+0000 7ff137a2e640 1 -- 172.21.15.37:0/1648128079 wait complete. 2024-04-25T08:56:38.994 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:39.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4fbc9640 1 Processor -- start 2024-04-25T08:56:39.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4fbc9640 1 -- start start 2024-04-25T08:56:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4fbc9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3f4815da80 0x7f3f480d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4fbc9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f480d1cd0 0x7f3f480d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4fbc9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4806be60 con 0x7f3f480d0620 2024-04-25T08:56:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4fbc9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f480d0a70 con 0x7f3f4815da80 2024-04-25T08:56:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4fbc9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4806c440 con 0x7f3f480d1cd0 2024-04-25T08:56:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4d965640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3f4815da80 0x7f3f480d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4d965640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3f4815da80 0x7f3f480d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43734/0 (socket says 172.21.15.37:43734) 2024-04-25T08:56:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4d965640 1 -- 172.21.15.37:0/60351303 learned_addr learned my addr 172.21.15.37:0/60351303 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4d164640 1 --2- 172.21.15.37:0/60351303 >> v2:172.21.15.203:3300/0 conn(0x7f3f480d1cd0 0x7f3f480d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4d965640 1 -- 172.21.15.37:0/60351303 >> v2:172.21.15.203:3300/0 conn(0x7f3f480d1cd0 msgr2=0x7f3f480d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4d965640 1 --2- 172.21.15.37:0/60351303 >> v2:172.21.15.203:3300/0 conn(0x7f3f480d1cd0 0x7f3f480d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4d965640 1 -- 172.21.15.37:0/60351303 >> v1:172.21.15.203:6789/0 conn(0x7f3f480d0620 legacy=0x7f3f480ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4d965640 1 -- 172.21.15.37:0/60351303 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4815de60 con 0x7f3f4815da80 2024-04-25T08:56:39.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.130+0000 7f3f4d965640 1 --2- 172.21.15.37:0/60351303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4815da80 0x7f3f480d4230 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009e50 tx=0x7f3f4401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa749ff0f3aa6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:39.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4c963640 1 -- 172.21.15.37:0/60351303 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440a5710 con 0x7f3f4815da80 2024-04-25T08:56:39.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4c963640 1 -- 172.21.15.37:0/60351303 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f440a58b0 con 0x7f3f4815da80 2024-04-25T08:56:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4c963640 1 -- 172.21.15.37:0/60351303 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440a5c80 con 0x7f3f4815da80 2024-04-25T08:56:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/60351303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4815da80 msgr2=0x7f3f480d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 --2- 172.21.15.37:0/60351303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4815da80 0x7f3f480d4230 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009e50 tx=0x7f3f4401c9f0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/60351303 shutdown_connections 2024-04-25T08:56:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 --2- 172.21.15.37:0/60351303 >> v2:172.21.15.203:3300/0 conn(0x7f3f480d1cd0 0x7f3f480d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 --2- 172.21.15.37:0/60351303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f4815da80 0x7f3f480d4230 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/60351303 >> 172.21.15.37:0/60351303 conn(0x7f3f4808b000 msgr2=0x7f3f480d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/60351303 shutdown_connections 2024-04-25T08:56:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/60351303 wait complete. 2024-04-25T08:56:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 Processor -- start 2024-04-25T08:56:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 -- start start 2024-04-25T08:56:39.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f480d0620 0x7f3f48153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480d1cd0 0x7f3f4815b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4815da80 0x7f3f4815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4816a730 con 0x7f3f480d0620 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4816a5b0 con 0x7f3f4815da80 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4fbc9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4816a8b0 con 0x7f3f480d1cd0 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4d965640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f480d0620 0x7f3f48153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4d965640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f480d0620 0x7f3f48153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43736/0 (socket says 172.21.15.37:43736) 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4d965640 1 -- 172.21.15.37:0/1668447344 learned_addr learned my addr 172.21.15.37:0/1668447344 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4d164640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480d1cd0 0x7f3f4815b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4e166640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4815da80 0x7f3f4815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4d965640 1 -- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480d1cd0 msgr2=0x7f3f4815b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4d965640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480d1cd0 0x7f3f4815b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4d965640 1 -- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4815da80 msgr2=0x7f3f4815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4d965640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4815da80 0x7f3f4815bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4d965640 1 -- 172.21.15.37:0/1668447344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f481541b0 con 0x7f3f480d0620 2024-04-25T08:56:39.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f4d965640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f480d0620 0x7f3f48153840 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3f440af040 tx=0x7f3f44019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:39.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f3e7fc640 1 -- 172.21.15.37:0/1668447344 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440b8670 con 0x7f3f480d0620 2024-04-25T08:56:39.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f3e7fc640 1 -- 172.21.15.37:0/1668447344 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f440be070 con 0x7f3f480d0620 2024-04-25T08:56:39.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.134+0000 7f3f3e7fc640 1 -- 172.21.15.37:0/1668447344 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f440b8a70 con 0x7f3f480d0620 2024-04-25T08:56:39.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.138+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/1668447344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f48154490 con 0x7f3f480d0620 2024-04-25T08:56:39.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.138+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/1668447344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f48154a50 con 0x7f3f480d0620 2024-04-25T08:56:39.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.138+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/1668447344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4804efa0 con 0x7f3f480d0620 2024-04-25T08:56:39.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.138+0000 7f3f3e7fc640 1 -- 172.21.15.37:0/1668447344 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3f440b8c10 con 0x7f3f480d0620 2024-04-25T08:56:39.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.138+0000 7f3f3e7fc640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f2c066f90 0x7f3f2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.138+0000 7f3f3e7fc640 1 -- 172.21.15.37:0/1668447344 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f3f4411fd30 con 0x7f3f480d0620 2024-04-25T08:56:39.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.138+0000 7f3f4d164640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f2c066f90 0x7f3f2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.138+0000 7f3f4d164640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f2c066f90 0x7f3f2c069450 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3f481f9cf0 tx=0x7f3f38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:39.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.142+0000 7f3f3e7fc640 1 -- 172.21.15.37:0/1668447344 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3f440c7020 con 0x7f3f480d0620 2024-04-25T08:56:39.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.330+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/1668447344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f480c6dc0 con 0x7f3f480d0620 2024-04-25T08:56:39.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f3e7fc640 1 -- 172.21.15.37:0/1668447344 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f3f440165e0 con 0x7f3f480d0620 2024-04-25T08:56:39.323 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:39.323 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/486740526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":486740526},{"type":"v1","addr":"172.21.15.37:6833","nonce":486740526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/121484331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":121484331},{"type":"v1","addr":"172.21.15.203:6835","nonce":121484331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2501630566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2501630566},{"type":"v1","addr":"172.21.15.203:6837","nonce":2501630566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:55:46.200939+0000","modified":"2024-04-25T08:56:06.871679+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6405},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6405":{"gid":6405,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":8,"addr":"172.21.15.37:6835/2189232625","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2189232625},{"type":"v1","addr":"172.21.15.37:6835","nonce":2189232625}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:56:39.323 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 35 2024-04-25T08:56:39.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f2c066f90 msgr2=0x7f3f2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f2c066f90 0x7f3f2c069450 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3f481f9cf0 tx=0x7f3f38008040 comp rx=0 tx=0).stop 2024-04-25T08:56:39.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/1668447344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f480d0620 msgr2=0x7f3f48153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f480d0620 0x7f3f48153840 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3f440af040 tx=0x7f3f44019790 comp rx=0 tx=0).stop 2024-04-25T08:56:39.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/1668447344 shutdown_connections 2024-04-25T08:56:39.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3f2c066f90 0x7f3f2c069450 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f4815da80 0x7f3f4815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f480d1cd0 0x7f3f4815b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 --2- 172.21.15.37:0/1668447344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f480d0620 0x7f3f48153840 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/1668447344 >> 172.21.15.37:0/1668447344 conn(0x7f3f4808b000 msgr2=0x7f3f480b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:39.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/1668447344 shutdown_connections 2024-04-25T08:56:39.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.334+0000 7f3f4fbc9640 1 -- 172.21.15.37:0/1668447344 wait complete. 2024-04-25T08:56:39.341 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.466+0000 7f9118a97640 1 Processor -- start 2024-04-25T08:56:39.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.466+0000 7f9118a97640 1 -- start start 2024-04-25T08:56:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9118a97640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f911415b920 0x7f9114099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9118a97640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9114099fb0 0x7f91140cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9118a97640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911409a3c0 con 0x7f91140b0440 2024-04-25T08:56:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9118a97640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91140cd680 con 0x7f911415b920 2024-04-25T08:56:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9118a97640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911409a950 con 0x7f9114099fb0 2024-04-25T08:56:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f911259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f911415b920 0x7f9114099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9109d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f91140b0440 0x7f91140aeb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56508/0 (socket says 172.21.15.37:56508) 2024-04-25T08:56:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9109d9b640 1 -- 172.21.15.37:0/706583647 learned_addr learned my addr 172.21.15.37:0/706583647 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9112d9d640 1 --2- 172.21.15.37:0/706583647 >> v2:172.21.15.203:3300/0 conn(0x7f9114099fb0 0x7f91140cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f911259c640 1 -- 172.21.15.37:0/706583647 >> v2:172.21.15.203:3300/0 conn(0x7f9114099fb0 msgr2=0x7f91140cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f911259c640 1 --2- 172.21.15.37:0/706583647 >> v2:172.21.15.203:3300/0 conn(0x7f9114099fb0 0x7f91140cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f911259c640 1 -- 172.21.15.37:0/706583647 >> v1:172.21.15.37:6789/0 conn(0x7f91140b0440 legacy=0x7f91140aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f911259c640 1 -- 172.21.15.37:0/706583647 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911409ac30 con 0x7f911415b920 2024-04-25T08:56:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9111d9b640 1 -- 172.21.15.37:0/706583647 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892623299 0 0) 0x7f90fc001120 con 0x7f91140b0440 2024-04-25T08:56:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9112d9d640 1 --2- 172.21.15.37:0/706583647 >> v2:172.21.15.203:3300/0 conn(0x7f9114099fb0 0x7f91140cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:39.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f911259c640 1 --2- 172.21.15.37:0/706583647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911415b920 0x7f9114099a70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9114072910 tx=0x7f910401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d3f46d3d5c2849 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:39.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9111d9b640 1 -- 172.21.15.37:0/706583647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040b0070 con 0x7f911415b920 2024-04-25T08:56:39.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9111d9b640 1 -- 172.21.15.37:0/706583647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91040ab600 con 0x7f911415b920 2024-04-25T08:56:39.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.470+0000 7f9111d9b640 1 -- 172.21.15.37:0/706583647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91040ab9d0 con 0x7f911415b920 2024-04-25T08:56:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 -- 172.21.15.37:0/706583647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911415b920 msgr2=0x7f9114099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 --2- 172.21.15.37:0/706583647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911415b920 0x7f9114099a70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9114072910 tx=0x7f910401c480 comp rx=0 tx=0).stop 2024-04-25T08:56:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 -- 172.21.15.37:0/706583647 shutdown_connections 2024-04-25T08:56:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 --2- 172.21.15.37:0/706583647 >> v2:172.21.15.203:3300/0 conn(0x7f9114099fb0 0x7f91140cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 --2- 172.21.15.37:0/706583647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f911415b920 0x7f9114099a70 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 -- 172.21.15.37:0/706583647 >> 172.21.15.37:0/706583647 conn(0x7f91140b3010 msgr2=0x7f91140b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 -- 172.21.15.37:0/706583647 shutdown_connections 2024-04-25T08:56:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 -- 172.21.15.37:0/706583647 wait complete. 2024-04-25T08:56:39.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 Processor -- start 2024-04-25T08:56:39.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 -- start start 2024-04-25T08:56:39.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9114099fb0 0x7f9114156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91140b0440 0x7f911415d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911415b920 0x7f9114153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91140afc20 con 0x7f91140b0440 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91140afaa0 con 0x7f911415b920 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91140afda0 con 0x7f9114099fb0 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9109d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91140b0440 0x7f911415d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9109d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91140b0440 0x7f911415d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60632/0 (socket says 172.21.15.37:60632) 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9109d9b640 1 -- 172.21.15.37:0/1488471182 learned_addr learned my addr 172.21.15.37:0/1488471182 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f911259c640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9114099fb0 0x7f9114156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9109d9b640 1 -- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9114099fb0 msgr2=0x7f9114156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9112d9d640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911415b920 0x7f9114153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9109d9b640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9114099fb0 0x7f9114156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9109d9b640 1 -- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911415b920 msgr2=0x7f9114153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9109d9b640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911415b920 0x7f9114153620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9109d9b640 1 -- 172.21.15.37:0/1488471182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91141573d0 con 0x7f91140b0440 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f911259c640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9114099fb0 0x7f9114156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9109d9b640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91140b0440 0x7f911415d220 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c4b0 tx=0x7f90fc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f910b7fe640 1 -- 172.21.15.37:0/1488471182 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc018070 con 0x7f91140b0440 2024-04-25T08:56:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f910b7fe640 1 -- 172.21.15.37:0/1488471182 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90fc007500 con 0x7f91140b0440 2024-04-25T08:56:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 -- 172.21.15.37:0/1488471182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91141556d0 con 0x7f91140b0440 2024-04-25T08:56:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f910b7fe640 1 -- 172.21.15.37:0/1488471182 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90fc004a30 con 0x7f91140b0440 2024-04-25T08:56:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.474+0000 7f9118a97640 1 -- 172.21.15.37:0/1488471182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9114155c90 con 0x7f91140b0440 2024-04-25T08:56:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.478+0000 7f9118a97640 1 -- 172.21.15.37:0/1488471182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911404efa0 con 0x7f91140b0440 2024-04-25T08:56:39.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.478+0000 7f910b7fe640 1 -- 172.21.15.37:0/1488471182 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90fc01b070 con 0x7f91140b0440 2024-04-25T08:56:39.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.478+0000 7f910b7fe640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90f4066f60 0x7f90f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.478+0000 7f910b7fe640 1 -- 172.21.15.37:0/1488471182 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f90fc084570 con 0x7f91140b0440 2024-04-25T08:56:39.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.478+0000 7f911259c640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90f4066f60 0x7f90f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.478+0000 7f911259c640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90f4066f60 0x7f90f4069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f91040ad000 tx=0x7f9104008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.482+0000 7f910b7fe640 1 -- 172.21.15.37:0/1488471182 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f911409a3c0 con 0x7f91140b0440 2024-04-25T08:56:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.642+0000 7f9118a97640 1 -- 172.21.15.37:0/1488471182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91140b2930 con 0x7f91140b0440 2024-04-25T08:56:39.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.646+0000 7f910b7fe640 1 -- 172.21.15.37:0/1488471182 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+17040 (secure 0 0 0) 0x7f90fc055380 con 0x7f91140b0440 2024-04-25T08:56:39.637 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:39.637 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":71,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:56:12.356132+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:44.492629+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:45.203787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.9","mappings":[{"from":7,"to":2}]},{"pgid":"10.39","mappings":[{"from":7,"to":3}]},{"pgid":"10.3b","mappings":[{"from":4,"to":3}]},{"pgid":"10.3c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:39.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 -- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90f4066f60 msgr2=0x7f90f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90f4066f60 0x7f90f4069420 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f91040ad000 tx=0x7f9104008040 comp rx=0 tx=0).stop 2024-04-25T08:56:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 -- 172.21.15.37:0/1488471182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91140b0440 msgr2=0x7f911415d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91140b0440 0x7f911415d220 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00c4b0 tx=0x7f90fc00c970 comp rx=0 tx=0).stop 2024-04-25T08:56:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 -- 172.21.15.37:0/1488471182 shutdown_connections 2024-04-25T08:56:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90f4066f60 0x7f90f4069420 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f911415b920 0x7f9114153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91140b0440 0x7f911415d220 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 --2- 172.21.15.37:0/1488471182 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9114099fb0 0x7f9114156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 -- 172.21.15.37:0/1488471182 >> 172.21.15.37:0/1488471182 conn(0x7f91140b3010 msgr2=0x7f91140b0ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 -- 172.21.15.37:0/1488471182 shutdown_connections 2024-04-25T08:56:39.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.650+0000 7f9118a97640 1 -- 172.21.15.37:0/1488471182 wait complete. 2024-04-25T08:56:39.656 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:56:39.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a2b09640 1 Processor -- start 2024-04-25T08:56:39.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a2b09640 1 -- start start 2024-04-25T08:56:39.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a2b09640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f479c0ce510 0x7f479c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a2b09640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f479c1451f0 0x7f479c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a2b09640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f479c0cee30 con 0x7f479c144ca0 2024-04-25T08:56:39.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a2b09640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f479c0cc2b0 con 0x7f479c1451f0 2024-04-25T08:56:39.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a2b09640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c06e290 con 0x7f479c0ce510 2024-04-25T08:56:39.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a10a6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f479c1451f0 0x7f479c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a10a6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f479c1451f0 0x7f479c142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60636/0 (socket says 172.21.15.37:60636) 2024-04-25T08:56:39.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a10a6640 1 -- 172.21.15.37:0/1805820409 learned_addr learned my addr 172.21.15.37:0/1805820409 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:39.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a08a5640 1 --2- 172.21.15.37:0/1805820409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f479c0ce510 0x7f479c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a10a6640 1 -- 172.21.15.37:0/1805820409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f479c0ce510 msgr2=0x7f479c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:39.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a10a6640 1 --2- 172.21.15.37:0/1805820409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f479c0ce510 0x7f479c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a10a6640 1 -- 172.21.15.37:0/1805820409 >> v1:172.21.15.203:6789/0 conn(0x7f479c144ca0 legacy=0x7f479c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:39.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.782+0000 7f47a10a6640 1 -- 172.21.15.37:0/1805820409 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c0ccc70 con 0x7f479c1451f0 2024-04-25T08:56:39.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f47a10a6640 1 --2- 172.21.15.37:0/1805820409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c1451f0 0x7f479c142680 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4794009e50 tx=0x7f479401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105835d5dd0c6447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:39.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f479b7fe640 1 -- 172.21.15.37:0/1805820409 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47940b4070 con 0x7f479c1451f0 2024-04-25T08:56:39.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f479b7fe640 1 -- 172.21.15.37:0/1805820409 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47940af400 con 0x7f479c1451f0 2024-04-25T08:56:39.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f479b7fe640 1 -- 172.21.15.37:0/1805820409 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47940af7d0 con 0x7f479c1451f0 2024-04-25T08:56:39.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f47a2b09640 1 -- 172.21.15.37:0/1805820409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c1451f0 msgr2=0x7f479c142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f47a2b09640 1 --2- 172.21.15.37:0/1805820409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c1451f0 0x7f479c142680 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4794009e50 tx=0x7f479401f1e0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f47a2b09640 1 -- 172.21.15.37:0/1805820409 shutdown_connections 2024-04-25T08:56:39.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f47a2b09640 1 --2- 172.21.15.37:0/1805820409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c1451f0 0x7f479c142680 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f47a2b09640 1 --2- 172.21.15.37:0/1805820409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f479c0ce510 0x7f479c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f47a2b09640 1 -- 172.21.15.37:0/1805820409 >> 172.21.15.37:0/1805820409 conn(0x7f479c0b11a0 msgr2=0x7f479c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:39.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f47a2b09640 1 -- 172.21.15.37:0/1805820409 shutdown_connections 2024-04-25T08:56:39.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.786+0000 7f47a2b09640 1 -- 172.21.15.37:0/1805820409 wait complete. 2024-04-25T08:56:39.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a2b09640 1 Processor -- start 2024-04-25T08:56:39.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a2b09640 1 -- start start 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a2b09640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f479c0ce510 0x7f479c139220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a2b09640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f479c144ca0 0x7f479c1e50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a2b09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c1451f0 0x7f479c1e5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a2b09640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c06ee30 con 0x7f479c1451f0 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a2b09640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c06df20 con 0x7f479c0ce510 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a2b09640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f479c0cc900 con 0x7f479c144ca0 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a10a6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c1451f0 0x7f479c1e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a10a6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c1451f0 0x7f479c1e5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60652/0 (socket says 172.21.15.37:60652) 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a10a6640 1 -- 172.21.15.37:0/886392045 learned_addr learned my addr 172.21.15.37:0/886392045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f479bfff640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f479c144ca0 0x7f479c1e50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a08a5640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f479c0ce510 0x7f479c139220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a10a6640 1 -- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f479c144ca0 msgr2=0x7f479c1e50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a10a6640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f479c144ca0 0x7f479c1e50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a10a6640 1 -- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f479c0ce510 msgr2=0x7f479c139220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a10a6640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f479c0ce510 0x7f479c139220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a10a6640 1 -- 172.21.15.37:0/886392045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c0aa1c0 con 0x7f479c1451f0 2024-04-25T08:56:39.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a08a5640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f479c0ce510 0x7f479c139220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:39.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a10a6640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c1451f0 0x7f479c1e5610 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f479401d890 tx=0x7f479401bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:39.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47997fa640 1 -- 172.21.15.37:0/886392045 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47940b4040 con 0x7f479c1451f0 2024-04-25T08:56:39.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47997fa640 1 -- 172.21.15.37:0/886392045 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47940c1070 con 0x7f479c1451f0 2024-04-25T08:56:39.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a2b09640 1 -- 172.21.15.37:0/886392045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479c0a9120 con 0x7f479c1451f0 2024-04-25T08:56:39.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47997fa640 1 -- 172.21.15.37:0/886392045 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4794009510 con 0x7f479c1451f0 2024-04-25T08:56:39.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a2b09640 1 -- 172.21.15.37:0/886392045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479c0a96e0 con 0x7f479c1451f0 2024-04-25T08:56:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47a2b09640 1 -- 172.21.15.37:0/886392045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479c04efa0 con 0x7f479c1451f0 2024-04-25T08:56:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47997fa640 1 -- 172.21.15.37:0/886392045 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47940b7070 con 0x7f479c1451f0 2024-04-25T08:56:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.790+0000 7f47997fa640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4788066f60 0x7f4788069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.794+0000 7f47997fa640 1 -- 172.21.15.37:0/886392045 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f4794123210 con 0x7f479c1451f0 2024-04-25T08:56:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.794+0000 7f47a08a5640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4788066f60 0x7f4788069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.794+0000 7f47a08a5640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4788066f60 0x7f4788069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f478c009840 tx=0x7f478c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.798+0000 7f47997fa640 1 -- 172.21.15.37:0/886392045 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4794018d40 con 0x7f479c1451f0 2024-04-25T08:56:39.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.982+0000 7f47a2b09640 1 -- 172.21.15.37:0/886392045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f479c0b1cc0 con 0x7f479c1451f0 2024-04-25T08:56:39.973 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T08:56:39.982+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:56:39.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.986+0000 7f47997fa640 1 -- 172.21.15.37:0/886392045 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7f47940f4200 con 0x7f479c1451f0 2024-04-25T08:56:39.976 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:56:39.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 -- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4788066f60 msgr2=0x7f4788069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4788066f60 0x7f4788069420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f478c009840 tx=0x7f478c008040 comp rx=0 tx=0).stop 2024-04-25T08:56:39.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 -- 172.21.15.37:0/886392045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c1451f0 msgr2=0x7f479c1e5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:39.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c1451f0 0x7f479c1e5610 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f479401d890 tx=0x7f479401bf80 comp rx=0 tx=0).stop 2024-04-25T08:56:39.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 -- 172.21.15.37:0/886392045 shutdown_connections 2024-04-25T08:56:39.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4788066f60 0x7f4788069420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f479c1451f0 0x7f479c1e5610 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f479c144ca0 0x7f479c1e50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 --2- 172.21.15.37:0/886392045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f479c0ce510 0x7f479c139220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:39.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 -- 172.21.15.37:0/886392045 >> 172.21.15.37:0/886392045 conn(0x7f479c0b11a0 msgr2=0x7f479c0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:39.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 -- 172.21.15.37:0/886392045 shutdown_connections 2024-04-25T08:56:39.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:39.990+0000 7f47a2b09640 1 -- 172.21.15.37:0/886392045 wait complete. 2024-04-25T08:56:39.988 INFO:tasks.ceph.mds.c.smithi037.stderr: -5343> 2024-04-25T08:55:52.458+0000 7fac41d13640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:56:39.993 INFO:tasks.ceph.mds.c.smithi037.stderr: -3684> 2024-04-25T08:55:56.450+0000 7fac3dd0b640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6555=0-4194304@1} caps={6555=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6555 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55bf555ad600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:56:39.995 INFO:tasks.ceph.mds.c.smithi037.stderr: -2863> 2024-04-25T08:56:03.014+0000 7fac3dd0b640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 ap=1 DIRTYPARENT s=0 n(v0 rc2024-04-25T08:55:55.321977+0000 1=1+0) (iversion lock) caps={6555=pAsLsXsFs/-@2},l=6555(-1) | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=1 authpin=1 scrubqueue=0 0x55bf555ad600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T08:56:39.996 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:56:40.031 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T08:56:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.122+0000 7f5333a3f640 1 Processor -- start 2024-04-25T08:56:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.122+0000 7f5333a3f640 1 -- start start 2024-04-25T08:56:40.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.122+0000 7f5333a3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f532c15ba20 0x7f532c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.122+0000 7f5333a3f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c15c260 con 0x7f532c15fd90 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.122+0000 7f5333a3f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c15c540 con 0x7f532c06b3a0 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.122+0000 7f5333a3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c0d1430 con 0x7f532c15ba20 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.122+0000 7f5330fda640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f532c15fd90 0x7f532c06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56512/0 (socket says 172.21.15.37:56512) 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.122+0000 7f5330fda640 1 -- 172.21.15.37:0/1886354437 learned_addr learned my addr 172.21.15.37:0/1886354437 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f53317db640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f532c15ba20 0x7f532c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1765892429 0 0) 0x7f532c15c260 con 0x7f532c15fd90 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5314003530 con 0x7f532c15fd90 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3442291050 0 0) 0x7f5314003530 con 0x7f532c15fd90 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f532c15ba20 msgr2=0x7f532c0cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f532c15ba20 0x7f532c0cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 >> v1:172.21.15.203:6789/0 conn(0x7f532c06b3a0 legacy=0x7f532c0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c15c820 con 0x7f532c15fd90 2024-04-25T08:56:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f531c001a00 con 0x7f532c15fd90 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f53317db640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f532c15ba20 0x7f532c0cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 >> v1:172.21.15.37:6789/0 conn(0x7f532c15fd90 legacy=0x7f532c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f532c15ba20 0x7f5314003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5314004050 0x7f5314004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5314004d50 0x7f53140051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531400bb90 con 0x7f5314004d50 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531400bd40 con 0x7f5314004050 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531400bf40 con 0x7f532c15ba20 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5331fdc640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5314004d50 0x7f53140051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f53317db640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f532c15ba20 0x7f5314003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5330fda640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5314004050 0x7f5314004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5331fdc640 1 -- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f532c15ba20 msgr2=0x7f5314003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5331fdc640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f532c15ba20 0x7f5314003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5331fdc640 1 -- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5314004050 msgr2=0x7f5314004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:40.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5331fdc640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5314004050 0x7f5314004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:40.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5331fdc640 1 -- 172.21.15.37:0/1886354437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5314003530 con 0x7f5314004d50 2024-04-25T08:56:40.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5330fda640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5314004050 0x7f5314004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5331fdc640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5314004d50 0x7f53140051d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f532c072050 tx=0x7f532801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8718b6f032c4077e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53280013a0 con 0x7f5314004d50 2024-04-25T08:56:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53280052f0 con 0x7f5314004d50 2024-04-25T08:56:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5323fff640 1 -- 172.21.15.37:0/1886354437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328003840 con 0x7f5314004d50 2024-04-25T08:56:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5333a3f640 1 -- 172.21.15.37:0/1886354437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5314004d50 msgr2=0x7f53140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5333a3f640 1 --2- 172.21.15.37:0/1886354437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5314004d50 0x7f53140051d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f532c072050 tx=0x7f532801c3c0 comp rx=0 tx=0).stop 2024-04-25T08:56:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5331fdc640 1 -- 172.21.15.37:0/1886354437 reap_dead start 2024-04-25T08:56:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5333a3f640 1 -- 172.21.15.37:0/1886354437 shutdown_connections 2024-04-25T08:56:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5333a3f640 1 -- 172.21.15.37:0/1886354437 >> 172.21.15.37:0/1886354437 conn(0x7f532c099a70 msgr2=0x7f532c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:40.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5333a3f640 1 -- 172.21.15.37:0/1886354437 shutdown_connections 2024-04-25T08:56:40.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.126+0000 7f5333a3f640 1 -- 172.21.15.37:0/1886354437 wait complete. 2024-04-25T08:56:40.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5333a3f640 1 Processor -- start 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5333a3f640 1 -- start start 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5333a3f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314004d50 0x7f532c159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5333a3f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f532c15a2e0 0x7f532c0c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5333a3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c0c7080 0x7f532c0ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5333a3f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c15be00 con 0x7f532c0c7080 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5333a3f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c1601f0 con 0x7f532c15a2e0 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5333a3f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c15bf80 con 0x7f5314004d50 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5331fdc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c0c7080 0x7f532c0ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5331fdc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c0c7080 0x7f532c0ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60674/0 (socket says 172.21.15.37:60674) 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5331fdc640 1 -- 172.21.15.37:0/1213201559 learned_addr learned my addr 172.21.15.37:0/1213201559 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5330fda640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f532c15a2e0 0x7f532c0c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f53317db640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314004d50 0x7f532c159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:40.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5331fdc640 1 -- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314004d50 msgr2=0x7f532c159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5331fdc640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314004d50 0x7f532c159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5331fdc640 1 -- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f532c15a2e0 msgr2=0x7f532c0c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5331fdc640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f532c15a2e0 0x7f532c0c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5331fdc640 1 -- 172.21.15.37:0/1213201559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c0c6190 con 0x7f532c0c7080 2024-04-25T08:56:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f53317db640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314004d50 0x7f532c159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5331fdc640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c0c7080 0x7f532c0ca140 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f532801c8a0 tx=0x7f5314005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5321ffb640 1 -- 172.21.15.37:0/1213201559 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53280013a0 con 0x7f532c0c7080 2024-04-25T08:56:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5321ffb640 1 -- 172.21.15.37:0/1213201559 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53280054a0 con 0x7f532c0c7080 2024-04-25T08:56:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5321ffb640 1 -- 172.21.15.37:0/1213201559 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328007a40 con 0x7f532c0c7080 2024-04-25T08:56:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5333a3f640 1 -- 172.21.15.37:0/1213201559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c09a580 con 0x7f532c0c7080 2024-04-25T08:56:40.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.130+0000 7f5333a3f640 1 -- 172.21.15.37:0/1213201559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c09ab40 con 0x7f532c0c7080 2024-04-25T08:56:40.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.134+0000 7f5333a3f640 1 -- 172.21.15.37:0/1213201559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c09e410 con 0x7f532c0c7080 2024-04-25T08:56:40.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.134+0000 7f5321ffb640 1 -- 172.21.15.37:0/1213201559 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53280bd020 con 0x7f532c0c7080 2024-04-25T08:56:40.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.134+0000 7f5321ffb640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f90 0x7f5318069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:40.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.134+0000 7f5321ffb640 1 -- 172.21.15.37:0/1213201559 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f532811ef40 con 0x7f532c0c7080 2024-04-25T08:56:40.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.134+0000 7f53317db640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f90 0x7f5318069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:40.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.134+0000 7f53317db640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f90 0x7f5318069450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f532400e940 tx=0x7f5324008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:40.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.138+0000 7f5321ffb640 1 -- 172.21.15.37:0/1213201559 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f53280c4020 con 0x7f532c0c7080 2024-04-25T08:56:40.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.330+0000 7f5333a3f640 1 -- 172.21.15.37:0/1213201559 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f532c04efa0 con 0x7f532c0c7080 2024-04-25T08:56:40.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.990+0000 7f5321ffb640 1 -- 172.21.15.37:0/1213201559 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f53280160d0 con 0x7f532c0c7080 2024-04-25T08:56:40.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 -- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f90 msgr2=0x7f5318069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:40.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f90 0x7f5318069450 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f532400e940 tx=0x7f5324008040 comp rx=0 tx=0).stop 2024-04-25T08:56:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 -- 172.21.15.37:0/1213201559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c0c7080 msgr2=0x7f532c0ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c0c7080 0x7f532c0ca140 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f532801c8a0 tx=0x7f5314005840 comp rx=0 tx=0).stop 2024-04-25T08:56:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 -- 172.21.15.37:0/1213201559 shutdown_connections 2024-04-25T08:56:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f90 0x7f5318069450 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c0c7080 0x7f532c0ca140 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f532c15a2e0 0x7f532c0c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 --2- 172.21.15.37:0/1213201559 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314004d50 0x7f532c159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 -- 172.21.15.37:0/1213201559 >> 172.21.15.37:0/1213201559 conn(0x7f532c099a70 msgr2=0x7f532c15ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 -- 172.21.15.37:0/1213201559 shutdown_connections 2024-04-25T08:56:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:40.994+0000 7f5333a3f640 1 -- 172.21.15.37:0/1213201559 wait complete. 2024-04-25T08:56:41.009 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:56:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539ea87640 1 Processor -- start 2024-04-25T08:56:41.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539ea87640 1 -- start start 2024-04-25T08:56:41.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539ea87640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f53980adc70 0x7f539806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:41.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539ea87640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f539806e9f0 0x7f53980cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:41.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539ea87640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53980b28c0 con 0x7f53980cfa50 2024-04-25T08:56:41.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539ea87640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f539815bb20 con 0x7f53980adc70 2024-04-25T08:56:41.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539ea87640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53980b2b60 con 0x7f539806e9f0 2024-04-25T08:56:41.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539c823640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f53980adc70 0x7f539806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:41.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539c823640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f53980adc70 0x7f539806e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60688/0 (socket says 172.21.15.37:60688) 2024-04-25T08:56:41.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539c823640 1 -- 172.21.15.37:0/3816845433 learned_addr learned my addr 172.21.15.37:0/3816845433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:41.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f5397fff640 1 --2- 172.21.15.37:0/3816845433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f539806e9f0 0x7f53980cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:41.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539c823640 1 -- 172.21.15.37:0/3816845433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f539806e9f0 msgr2=0x7f53980cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:41.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539c823640 1 --2- 172.21.15.37:0/3816845433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f539806e9f0 0x7f53980cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:41.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539c823640 1 -- 172.21.15.37:0/3816845433 >> v1:172.21.15.37:6789/0 conn(0x7f53980cfa50 legacy=0x7f539815f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:41.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f53977fe640 1 -- 172.21.15.37:0/3816845433 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490583678 0 0) 0x7f53980b28c0 con 0x7f53980cfa50 2024-04-25T08:56:41.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.202+0000 7f539c823640 1 -- 172.21.15.37:0/3816845433 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53980b2e40 con 0x7f53980adc70 2024-04-25T08:56:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f539c823640 1 --2- 172.21.15.37:0/3816845433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53980adc70 0x7f539806e4b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5388009870 tx=0x7f538801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6374b3cf7a016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f53977fe640 1 -- 172.21.15.37:0/3816845433 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53880b0070 con 0x7f53980adc70 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f53977fe640 1 -- 172.21.15.37:0/3816845433 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53880ab600 con 0x7f53980adc70 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f53977fe640 1 -- 172.21.15.37:0/3816845433 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53880ab9d0 con 0x7f53980adc70 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f539ea87640 1 -- 172.21.15.37:0/3816845433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53980adc70 msgr2=0x7f539806e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f539ea87640 1 --2- 172.21.15.37:0/3816845433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53980adc70 0x7f539806e4b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f5388009870 tx=0x7f538801c560 comp rx=0 tx=0).stop 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f539ea87640 1 -- 172.21.15.37:0/3816845433 shutdown_connections 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f539ea87640 1 --2- 172.21.15.37:0/3816845433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f539806e9f0 0x7f53980cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f539ea87640 1 --2- 172.21.15.37:0/3816845433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53980adc70 0x7f539806e4b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f539ea87640 1 -- 172.21.15.37:0/3816845433 >> 172.21.15.37:0/3816845433 conn(0x7f539806ae60 msgr2=0x7f539806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f539ea87640 1 -- 172.21.15.37:0/3816845433 shutdown_connections 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f539ea87640 1 -- 172.21.15.37:0/3816845433 wait complete. 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f539ea87640 1 Processor -- start 2024-04-25T08:56:41.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.206+0000 7f539ea87640 1 -- start start 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539ea87640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539806e9f0 0x7f539815e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539ea87640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53980adc70 0x7f53980c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539ea87640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53980cfa50 0x7f53980c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539ea87640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539806edd0 con 0x7f53980cfa50 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539ea87640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53980cfea0 con 0x7f539806e9f0 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539ea87640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53980b3c00 con 0x7f53980adc70 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539d024640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53980cfa50 0x7f53980c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539d024640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53980cfa50 0x7f53980c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60696/0 (socket says 172.21.15.37:60696) 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539d024640 1 -- 172.21.15.37:0/318224482 learned_addr learned my addr 172.21.15.37:0/318224482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f5397fff640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53980adc70 0x7f53980c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539c823640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539806e9f0 0x7f539815e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539d024640 1 -- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53980adc70 msgr2=0x7f53980c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:41.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539d024640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53980adc70 0x7f53980c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539d024640 1 -- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539806e9f0 msgr2=0x7f539815e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539d024640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539806e9f0 0x7f539815e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539d024640 1 -- 172.21.15.37:0/318224482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539815e960 con 0x7f53980cfa50 2024-04-25T08:56:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539d024640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53980cfa50 0x7f53980c54e0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5390002240 tx=0x7f539000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f53957fa640 1 -- 172.21.15.37:0/318224482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390017070 con 0x7f53980cfa50 2024-04-25T08:56:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f53957fa640 1 -- 172.21.15.37:0/318224482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5390003db0 con 0x7f53980cfa50 2024-04-25T08:56:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539ea87640 1 -- 172.21.15.37:0/318224482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539815c8e0 con 0x7f53980cfa50 2024-04-25T08:56:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f53957fa640 1 -- 172.21.15.37:0/318224482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5390005270 con 0x7f53980cfa50 2024-04-25T08:56:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539ea87640 1 -- 172.21.15.37:0/318224482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f539815ce60 con 0x7f53980cfa50 2024-04-25T08:56:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539ea87640 1 -- 172.21.15.37:0/318224482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5398067c30 con 0x7f53980cfa50 2024-04-25T08:56:41.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f53957fa640 1 -- 172.21.15.37:0/318224482 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f539001a070 con 0x7f53980cfa50 2024-04-25T08:56:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f53957fa640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5384066f60 0x7f5384069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f53957fa640 1 -- 172.21.15.37:0/318224482 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f53900846e0 con 0x7f53980cfa50 2024-04-25T08:56:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.210+0000 7f539c823640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5384066f60 0x7f5384069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:41.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.214+0000 7f539c823640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5384066f60 0x7f5384069420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f53880ad000 tx=0x7f53880a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:41.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.214+0000 7f53957fa640 1 -- 172.21.15.37:0/318224482 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f539815e960 con 0x7f53980cfa50 2024-04-25T08:56:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.374+0000 7f539ea87640 1 -- 172.21.15.37:0/318224482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f53980cb4f0 con 0x7f53980cfa50 2024-04-25T08:56:41.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.998+0000 7f53957fa640 1 -- 172.21.15.37:0/318224482 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v73) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5390054ee0 con 0x7f53980cfa50 2024-04-25T08:56:41.987 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:56:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.998+0000 7f539ea87640 1 -- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5384066f60 msgr2=0x7f5384069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.998+0000 7f539ea87640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5384066f60 0x7f5384069420 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f53880ad000 tx=0x7f53880a6000 comp rx=0 tx=0).stop 2024-04-25T08:56:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.998+0000 7f539ea87640 1 -- 172.21.15.37:0/318224482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53980cfa50 msgr2=0x7f53980c54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.998+0000 7f539ea87640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53980cfa50 0x7f53980c54e0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f5390002240 tx=0x7f539000cbe0 comp rx=0 tx=0).stop 2024-04-25T08:56:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.998+0000 7f539ea87640 1 -- 172.21.15.37:0/318224482 shutdown_connections 2024-04-25T08:56:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.998+0000 7f539ea87640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5384066f60 0x7f5384069420 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:41.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.998+0000 7f539ea87640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53980cfa50 0x7f53980c54e0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.998+0000 7f539ea87640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f53980adc70 0x7f53980c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.998+0000 7f539ea87640 1 --2- 172.21.15.37:0/318224482 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f539806e9f0 0x7f539815e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:41.998+0000 7f539ea87640 1 -- 172.21.15.37:0/318224482 >> 172.21.15.37:0/318224482 conn(0x7f539806ae60 msgr2=0x7f5398071810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:41.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.002+0000 7f539ea87640 1 -- 172.21.15.37:0/318224482 shutdown_connections 2024-04-25T08:56:41.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.002+0000 7f539ea87640 1 -- 172.21.15.37:0/318224482 wait complete. 2024-04-25T08:56:42.017 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:56:42.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.154+0000 7fddcc684640 1 Processor -- start 2024-04-25T08:56:42.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.154+0000 7fddcc684640 1 -- start start 2024-04-25T08:56:42.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.154+0000 7fddcc684640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fddc406ac90 0x7fddc406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:42.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.154+0000 7fddcc684640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddc40b0310 0x7fddc40b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:42.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.154+0000 7fddcc684640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc415cb80 con 0x7fddc40b2ac0 2024-04-25T08:56:42.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.154+0000 7fddcc684640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc406b5b0 con 0x7fddc406ac90 2024-04-25T08:56:42.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.154+0000 7fddcc684640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc40afe10 con 0x7fddc40b0310 2024-04-25T08:56:42.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddca420640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fddc406ac90 0x7fddc406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:42.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddca420640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fddc406ac90 0x7fddc406b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60702/0 (socket says 172.21.15.37:60702) 2024-04-25T08:56:42.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddca420640 1 -- 172.21.15.37:0/2719983894 learned_addr learned my addr 172.21.15.37:0/2719983894 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:42.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddcac21640 1 --2- 172.21.15.37:0/2719983894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddc40b0310 0x7fddc40b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:42.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddca420640 1 -- 172.21.15.37:0/2719983894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddc40b0310 msgr2=0x7fddc40b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:42.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddca420640 1 --2- 172.21.15.37:0/2719983894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddc40b0310 0x7fddc40b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:42.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddca420640 1 -- 172.21.15.37:0/2719983894 >> v1:172.21.15.203:6789/0 conn(0x7fddc40b2ac0 legacy=0x7fddc40ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:42.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddca420640 1 -- 172.21.15.37:0/2719983894 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40cd300 con 0x7fddc406ac90 2024-04-25T08:56:42.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddca420640 1 --2- 172.21.15.37:0/2719983894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc406ac90 0x7fddc406b070 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009870 tx=0x7fddc001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb5d0f0147a8dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:42.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddc941e640 1 -- 172.21.15.37:0/2719983894 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc00a95f0 con 0x7fddc406ac90 2024-04-25T08:56:42.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddc941e640 1 -- 172.21.15.37:0/2719983894 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc00a9790 con 0x7fddc406ac90 2024-04-25T08:56:42.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.158+0000 7fddc941e640 1 -- 172.21.15.37:0/2719983894 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc00a9b60 con 0x7fddc406ac90 2024-04-25T08:56:42.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 -- 172.21.15.37:0/2719983894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc406ac90 msgr2=0x7fddc406b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:42.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 --2- 172.21.15.37:0/2719983894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc406ac90 0x7fddc406b070 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009870 tx=0x7fddc001c530 comp rx=0 tx=0).stop 2024-04-25T08:56:42.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 -- 172.21.15.37:0/2719983894 shutdown_connections 2024-04-25T08:56:42.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 --2- 172.21.15.37:0/2719983894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddc40b0310 0x7fddc40b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:42.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 --2- 172.21.15.37:0/2719983894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc406ac90 0x7fddc406b070 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:42.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 -- 172.21.15.37:0/2719983894 >> 172.21.15.37:0/2719983894 conn(0x7fddc40b1fa0 msgr2=0x7fddc40b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:42.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 -- 172.21.15.37:0/2719983894 shutdown_connections 2024-04-25T08:56:42.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 -- 172.21.15.37:0/2719983894 wait complete. 2024-04-25T08:56:42.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 Processor -- start 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 -- start start 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddc406ac90 0x7fddc41f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc40b0310 0x7fddc40c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddc40b2ac0 0x7fddc40c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4168470 con 0x7fddc40b0310 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc41682f0 con 0x7fddc406ac90 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc41685f0 con 0x7fddc40b2ac0 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddc9c1f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc40b0310 0x7fddc40c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddc9c1f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc40b0310 0x7fddc40c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60708/0 (socket says 172.21.15.37:60708) 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddc9c1f640 1 -- 172.21.15.37:0/2174749382 learned_addr learned my addr 172.21.15.37:0/2174749382 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddca420640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddc406ac90 0x7fddc41f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:42.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcac21640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddc40b2ac0 0x7fddc40c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddc9c1f640 1 -- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddc40b2ac0 msgr2=0x7fddc40c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddc9c1f640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddc40b2ac0 0x7fddc40c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddc9c1f640 1 -- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddc406ac90 msgr2=0x7fddc41f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddc9c1f640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddc406ac90 0x7fddc41f48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddc9c1f640 1 -- 172.21.15.37:0/2174749382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc40c82f0 con 0x7fddc40b0310 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddca420640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddc406ac90 0x7fddc41f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddc9c1f640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc40b0310 0x7fddc40c9770 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fddb400c420 tx=0x7fddb400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddbaffd640 1 -- 172.21.15.37:0/2174749382 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb40052d0 con 0x7fddc40b0310 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddbaffd640 1 -- 172.21.15.37:0/2174749382 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb4005470 con 0x7fddc40b0310 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddbaffd640 1 -- 172.21.15.37:0/2174749382 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb4007260 con 0x7fddc40b0310 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.162+0000 7fddcc684640 1 -- 172.21.15.37:0/2174749382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc40c7d30 con 0x7fddc40b0310 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.166+0000 7fddcc684640 1 -- 172.21.15.37:0/2174749382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc4153a80 con 0x7fddc40b0310 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.166+0000 7fddcc684640 1 -- 172.21.15.37:0/2174749382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc4068340 con 0x7fddc40b0310 2024-04-25T08:56:42.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.166+0000 7fddbaffd640 1 -- 172.21.15.37:0/2174749382 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fddb4007400 con 0x7fddc40b0310 2024-04-25T08:56:42.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.166+0000 7fddbaffd640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdda8066f60 0x7fdda8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:42.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.166+0000 7fddbaffd640 1 -- 172.21.15.37:0/2174749382 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5836+0+0 (secure 0 0 0) 0x7fddb4084550 con 0x7fddc40b0310 2024-04-25T08:56:42.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.166+0000 7fddca420640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdda8066f60 0x7fdda8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:42.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.166+0000 7fddca420640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdda8066f60 0x7fdda8069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fddc00ae000 tx=0x7fddc00b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:42.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.170+0000 7fddbaffd640 1 -- 172.21.15.37:0/2174749382 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fddb4055570 con 0x7fddc40b0310 2024-04-25T08:56:42.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.330+0000 7fddcc684640 1 -- 172.21.15.37:0/2174749382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fddc40afbf0 con 0x7fddc40b0310 2024-04-25T08:56:42.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:42.998+0000 7fddbaffd640 1 -- 172.21.15.37:0/2174749382 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v74) v1 ==== 169+0+0 (secure 0 0 0) 0x7fddb4082e20 con 0x7fddc40b0310 2024-04-25T08:56:42.989 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T08:56:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 -- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdda8066f60 msgr2=0x7fdda8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdda8066f60 0x7fdda8069420 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fddc00ae000 tx=0x7fddc00b3040 comp rx=0 tx=0).stop 2024-04-25T08:56:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 -- 172.21.15.37:0/2174749382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc40b0310 msgr2=0x7fddc40c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc40b0310 0x7fddc40c9770 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fddb400c420 tx=0x7fddb400c8e0 comp rx=0 tx=0).stop 2024-04-25T08:56:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 -- 172.21.15.37:0/2174749382 shutdown_connections 2024-04-25T08:56:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdda8066f60 0x7fdda8069420 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:42.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddc40b2ac0 0x7fddc40c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:42.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc40b0310 0x7fddc40c9770 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:42.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 --2- 172.21.15.37:0/2174749382 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddc406ac90 0x7fddc41f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:42.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 -- 172.21.15.37:0/2174749382 >> 172.21.15.37:0/2174749382 conn(0x7fddc40b1fa0 msgr2=0x7fddc40737a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:42.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 -- 172.21.15.37:0/2174749382 shutdown_connections 2024-04-25T08:56:42.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.002+0000 7fddcc684640 1 -- 172.21.15.37:0/2174749382 wait complete. 2024-04-25T08:56:43.019 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:56:43.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.154+0000 7f15d8feb640 1 Processor -- start 2024-04-25T08:56:43.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.154+0000 7f15d8feb640 1 -- start start 2024-04-25T08:56:43.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d8feb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d415a600 0x7f15d40cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d8feb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f15d40b2ac0 0x7f15d40cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d8feb640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d40cf370 con 0x7f15d415e1e0 2024-04-25T08:56:43.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d8feb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d40ce070 con 0x7f15d40b2ac0 2024-04-25T08:56:43.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d8feb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40ce350 con 0x7f15d415a600 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d2d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f15d415e1e0 0x7f15d40b3250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56520/0 (socket says 172.21.15.37:56520) 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d2d9d640 1 -- 172.21.15.37:0/2533267851 learned_addr learned my addr 172.21.15.37:0/2533267851 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d1d9b640 1 --2- 172.21.15.37:0/2533267851 >> v2:172.21.15.203:3300/0 conn(0x7f15d40b2ac0 0x7f15d40cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385983101 0 0) 0x7f15d40cf370 con 0x7f15d415e1e0 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15c0003530 con 0x7f15d415e1e0 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d259c640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d415a600 0x7f15d40cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 319659468 0 0) 0x7f15c0003530 con 0x7f15d415e1e0 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d415a600 msgr2=0x7f15d40cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d415a600 0x7f15d40cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 >> v2:172.21.15.203:3300/0 conn(0x7f15d40b2ac0 msgr2=0x7f15d40cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 --2- 172.21.15.37:0/2533267851 >> v2:172.21.15.203:3300/0 conn(0x7f15d40b2ac0 0x7f15d40cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d40ce630 con 0x7f15d415e1e0 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f15cc001a00 con 0x7f15d415e1e0 2024-04-25T08:56:43.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 >> v1:172.21.15.37:6789/0 conn(0x7f15d415e1e0 legacy=0x7f15d40b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 0x7f15c0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d415a600 0x7f15c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c0004d50 0x7f15c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40ce070 con 0x7f15c0003a30 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40ce350 con 0x7f15c0004d50 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15c000bb10 con 0x7f15d415a600 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d259c640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 0x7f15c0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d1d9b640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d415a600 0x7f15c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d259c640 1 -- 172.21.15.37:0/2533267851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d415a600 msgr2=0x7f15c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d259c640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d415a600 0x7f15c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d259c640 1 -- 172.21.15.37:0/2533267851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c0004d50 msgr2=0x7f15c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d259c640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15c0004d50 0x7f15c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d259c640 1 -- 172.21.15.37:0/2533267851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c0003530 con 0x7f15c0003a30 2024-04-25T08:56:43.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.158+0000 7f15d1d9b640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d415a600 0x7f15c0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d259c640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 0x7f15c0003e50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00ba10 tx=0x7f15bc01c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a3f14e79e1b5c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0ae070 con 0x7f15c0003a30 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc0a9660 con 0x7f15c0003a30 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d159a640 1 -- 172.21.15.37:0/2533267851 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00b5e0 con 0x7f15c0003a30 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 -- 172.21.15.37:0/2533267851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 msgr2=0x7f15c0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 --2- 172.21.15.37:0/2533267851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 0x7f15c0003e50 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f15bc00ba10 tx=0x7f15bc01c680 comp rx=0 tx=0).stop 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d2d9d640 1 -- 172.21.15.37:0/2533267851 reap_dead start 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 -- 172.21.15.37:0/2533267851 shutdown_connections 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 -- 172.21.15.37:0/2533267851 >> 172.21.15.37:0/2533267851 conn(0x7f15d40b1f10 msgr2=0x7f15d40b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 -- 172.21.15.37:0/2533267851 shutdown_connections 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 -- 172.21.15.37:0/2533267851 wait complete. 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 Processor -- start 2024-04-25T08:56:43.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 -- start start 2024-04-25T08:56:43.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 0x7f15d414d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d40b5dd0 0x7f15d41f24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15d406f510 0x7f15d406f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40cc1e0 con 0x7f15c0003a30 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d40cbef0 con 0x7f15d406f510 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d8feb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d415a9e0 con 0x7f15d40b5dd0 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 0x7f15d414d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 0x7f15d414d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60728/0 (socket says 172.21.15.37:60728) 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d259c640 1 -- 172.21.15.37:0/4022155404 learned_addr learned my addr 172.21.15.37:0/4022155404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d1d9b640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d40b5dd0 0x7f15d41f24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d259c640 1 -- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d40b5dd0 msgr2=0x7f15d41f24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d2d9d640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15d406f510 0x7f15d406f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d259c640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d40b5dd0 0x7f15d41f24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d259c640 1 -- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15d406f510 msgr2=0x7f15d406f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d259c640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15d406f510 0x7f15d406f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d259c640 1 -- 172.21.15.37:0/4022155404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d41f2d40 con 0x7f15c0003a30 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.162+0000 7f15d2d9d640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15d406f510 0x7f15d406f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:43.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15d259c640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 0x7f15d414d410 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f15bc098090 tx=0x7f15c0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:43.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15caffd640 1 -- 172.21.15.37:0/4022155404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0ae040 con 0x7f15c0003a30 2024-04-25T08:56:43.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15caffd640 1 -- 172.21.15.37:0/4022155404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc0b7030 con 0x7f15c0003a30 2024-04-25T08:56:43.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15d8feb640 1 -- 172.21.15.37:0/4022155404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d41f3020 con 0x7f15c0003a30 2024-04-25T08:56:43.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15caffd640 1 -- 172.21.15.37:0/4022155404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc0b3880 con 0x7f15c0003a30 2024-04-25T08:56:43.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15d8feb640 1 -- 172.21.15.37:0/4022155404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d41f00d0 con 0x7f15c0003a30 2024-04-25T08:56:43.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15d8feb640 1 -- 172.21.15.37:0/4022155404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d404efa0 con 0x7f15c0003a30 2024-04-25T08:56:43.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15caffd640 1 -- 172.21.15.37:0/4022155404 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f15bc0b1070 con 0x7f15c0003a30 2024-04-25T08:56:43.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15caffd640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15b4066f90 0x7f15b4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15caffd640 1 -- 172.21.15.37:0/4022155404 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f15bc11f6d0 con 0x7f15c0003a30 2024-04-25T08:56:43.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15d1d9b640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15b4066f90 0x7f15b4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.166+0000 7f15d1d9b640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15b4066f90 0x7f15b4069450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f15d406b530 tx=0x7f15c4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:43.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.170+0000 7f15caffd640 1 -- 172.21.15.37:0/4022155404 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f15bc016390 con 0x7f15c0003a30 2024-04-25T08:56:43.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.346+0000 7f15d8feb640 1 -- 172.21.15.37:0/4022155404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f15d40b6380 con 0x7f15c0003a30 2024-04-25T08:56:43.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.350+0000 7f15caffd640 1 -- 172.21.15.37:0/4022155404 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f15bc0f0850 con 0x7f15c0003a30 2024-04-25T08:56:43.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 -- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15b4066f90 msgr2=0x7f15b4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:43.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15b4066f90 0x7f15b4069450 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f15d406b530 tx=0x7f15c4008040 comp rx=0 tx=0).stop 2024-04-25T08:56:43.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 -- 172.21.15.37:0/4022155404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 msgr2=0x7f15d414d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:43.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 0x7f15d414d410 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f15bc098090 tx=0x7f15c0005840 comp rx=0 tx=0).stop 2024-04-25T08:56:43.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 -- 172.21.15.37:0/4022155404 shutdown_connections 2024-04-25T08:56:43.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f15b4066f90 0x7f15b4069450 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15d406f510 0x7f15d406f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15d40b5dd0 0x7f15d41f24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 --2- 172.21.15.37:0/4022155404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15c0003a30 0x7f15d414d410 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 -- 172.21.15.37:0/4022155404 >> 172.21.15.37:0/4022155404 conn(0x7f15d40b1f10 msgr2=0x7f15d4159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:43.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 -- 172.21.15.37:0/4022155404 shutdown_connections 2024-04-25T08:56:43.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.354+0000 7f15d8feb640 1 -- 172.21.15.37:0/4022155404 wait complete. 2024-04-25T08:56:43.361 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-04-25T08:56:43.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61f125d640 1 Processor -- start 2024-04-25T08:56:43.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61f125d640 1 -- start start 2024-04-25T08:56:43.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61f125d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec06bca0 0x7f61ec0cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61f125d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f61ec0af370 0x7f61ec0ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61f125d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61ec06ec30 con 0x7f61ec066e30 2024-04-25T08:56:43.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61f125d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61ec0669a0 con 0x7f61ec0af370 2024-04-25T08:56:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61f125d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec0cd5f0 con 0x7f61ec06bca0 2024-04-25T08:56:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61ead9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec06bca0 0x7f61ec0cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61eb59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f61ec0af370 0x7f61ec0ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61ead9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec06bca0 0x7f61ec0cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:57186/0 (socket says 172.21.15.37:57186) 2024-04-25T08:56:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61eb59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f61ec0af370 0x7f61ec0ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:46882/0 (socket says 172.21.15.37:46882) 2024-04-25T08:56:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61ead9d640 1 -- 172.21.15.37:0/97479682 learned_addr learned my addr 172.21.15.37:0/97479682 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61ead9d640 1 -- 172.21.15.37:0/97479682 >> v2:172.21.15.203:3300/0 conn(0x7f61ec0af370 msgr2=0x7f61ec0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:43.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61ead9d640 1 --2- 172.21.15.37:0/97479682 >> v2:172.21.15.203:3300/0 conn(0x7f61ec0af370 0x7f61ec0ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61ead9d640 1 -- 172.21.15.37:0/97479682 >> v1:172.21.15.203:6789/0 conn(0x7f61ec066e30 legacy=0x7f61ec06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:43.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.498+0000 7f61ead9d640 1 -- 172.21.15.37:0/97479682 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec06f230 con 0x7f61ec06bca0 2024-04-25T08:56:43.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61ead9d640 1 --2- 172.21.15.37:0/97479682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec06bca0 0x7f61ec0cd0b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009e50 tx=0x7f61dc01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae0141f6f1213613 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:43.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61e9d9b640 1 -- 172.21.15.37:0/97479682 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0a5850 con 0x7f61ec06bca0 2024-04-25T08:56:43.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61e9d9b640 1 -- 172.21.15.37:0/97479682 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc0a59f0 con 0x7f61ec06bca0 2024-04-25T08:56:43.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61e9d9b640 1 -- 172.21.15.37:0/97479682 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc0a5dc0 con 0x7f61ec06bca0 2024-04-25T08:56:43.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61f125d640 1 -- 172.21.15.37:0/97479682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec06bca0 msgr2=0x7f61ec0cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:43.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61f125d640 1 --2- 172.21.15.37:0/97479682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec06bca0 0x7f61ec0cd0b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f61dc009e50 tx=0x7f61dc01c970 comp rx=0 tx=0).stop 2024-04-25T08:56:43.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61f125d640 1 -- 172.21.15.37:0/97479682 shutdown_connections 2024-04-25T08:56:43.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61f125d640 1 --2- 172.21.15.37:0/97479682 >> v2:172.21.15.203:3300/0 conn(0x7f61ec0af370 0x7f61ec0ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61f125d640 1 --2- 172.21.15.37:0/97479682 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec06bca0 0x7f61ec0cd0b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61f125d640 1 -- 172.21.15.37:0/97479682 >> 172.21.15.37:0/97479682 conn(0x7f61ec071810 msgr2=0x7f61ec071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:43.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61f125d640 1 -- 172.21.15.37:0/97479682 shutdown_connections 2024-04-25T08:56:43.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.502+0000 7f61f125d640 1 -- 172.21.15.37:0/97479682 wait complete. 2024-04-25T08:56:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61f125d640 1 Processor -- start 2024-04-25T08:56:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61f125d640 1 -- start start 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61f125d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec066e30 0x7f61ec0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61f125d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61ec06bca0 0x7f61ec149c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61f125d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ec0af370 0x7f61ec1558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61f125d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec1687b0 con 0x7f61ec0af370 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61f125d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec168630 con 0x7f61ec06bca0 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61f125d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61ec168930 con 0x7f61ec066e30 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61eb59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ec0af370 0x7f61ec1558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61eb59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ec0af370 0x7f61ec1558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60732/0 (socket says 172.21.15.37:60732) 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61eb59e640 1 -- 172.21.15.37:0/3703617919 learned_addr learned my addr 172.21.15.37:0/3703617919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61ea59c640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61ec06bca0 0x7f61ec149c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61ead9d640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec066e30 0x7f61ec0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61eb59e640 1 -- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec066e30 msgr2=0x7f61ec0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:43.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61eb59e640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec066e30 0x7f61ec0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61eb59e640 1 -- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61ec06bca0 msgr2=0x7f61ec149c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61eb59e640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61ec06bca0 0x7f61ec149c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61eb59e640 1 -- 172.21.15.37:0/3703617919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec0acec0 con 0x7f61ec0af370 2024-04-25T08:56:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61ead9d640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec066e30 0x7f61ec0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61eb59e640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ec0af370 0x7f61ec1558a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f61e400e960 tx=0x7f61e400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61d37fe640 1 -- 172.21.15.37:0/3703617919 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e4017070 con 0x7f61ec0af370 2024-04-25T08:56:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61d37fe640 1 -- 172.21.15.37:0/3703617919 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e400c9a0 con 0x7f61ec0af370 2024-04-25T08:56:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61f125d640 1 -- 172.21.15.37:0/3703617919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61ec0ad1a0 con 0x7f61ec0af370 2024-04-25T08:56:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61d37fe640 1 -- 172.21.15.37:0/3703617919 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e400cd70 con 0x7f61ec0af370 2024-04-25T08:56:43.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61f125d640 1 -- 172.21.15.37:0/3703617919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61ec0ad760 con 0x7f61ec0af370 2024-04-25T08:56:43.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61f125d640 1 -- 172.21.15.37:0/3703617919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61ec04efa0 con 0x7f61ec0af370 2024-04-25T08:56:43.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.506+0000 7f61d37fe640 1 -- 172.21.15.37:0/3703617919 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f61e401a070 con 0x7f61ec0af370 2024-04-25T08:56:43.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.510+0000 7f61d37fe640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f61d8066f60 0x7f61d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:43.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.510+0000 7f61d37fe640 1 -- 172.21.15.37:0/3703617919 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f61e4085130 con 0x7f61ec0af370 2024-04-25T08:56:43.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.510+0000 7f61ead9d640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f61d8066f60 0x7f61d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:43.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.510+0000 7f61ead9d640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f61d8066f60 0x7f61d8069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0ae000 tx=0x7f61dc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:43.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.514+0000 7f61d37fe640 1 -- 172.21.15.37:0/3703617919 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f61e4055cf0 con 0x7f61ec0af370 2024-04-25T08:56:43.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:43.734+0000 7f61f125d640 1 -- 172.21.15.37:0/3703617919 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7f61ec072400 con 0x7f61ec0af370 2024-04-25T08:56:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.018+0000 7f61d37fe640 1 -- 172.21.15.37:0/3703617919 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v235) v1 ==== 169+0+0 (secure 0 0 0) 0x7f61e4059d90 con 0x7f61ec0af370 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 -- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f61d8066f60 msgr2=0x7f61d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f61d8066f60 0x7f61d8069420 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f61dc0ae000 tx=0x7f61dc0a7070 comp rx=0 tx=0).stop 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 -- 172.21.15.37:0/3703617919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ec0af370 msgr2=0x7f61ec1558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ec0af370 0x7f61ec1558a0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f61e400e960 tx=0x7f61e400ee20 comp rx=0 tx=0).stop 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 -- 172.21.15.37:0/3703617919 shutdown_connections 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f61d8066f60 0x7f61d8069420 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ec0af370 0x7f61ec1558a0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f61ec06bca0 0x7f61ec149c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 --2- 172.21.15.37:0/3703617919 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61ec066e30 0x7f61ec0ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 -- 172.21.15.37:0/3703617919 >> 172.21.15.37:0/3703617919 conn(0x7f61ec071810 msgr2=0x7f61ec0b4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 -- 172.21.15.37:0/3703617919 shutdown_connections 2024-04-25T08:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.022+0000 7f61f125d640 1 -- 172.21.15.37:0/3703617919 wait complete. 2024-04-25T08:56:44.035 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T08:56:44.036 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T08:56:44.036 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:56:44.036 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50def6e640 1 Processor -- start 2024-04-25T08:56:44.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50def6e640 1 -- start start 2024-04-25T08:56:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50def6e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d80d07f0 0x7f50d80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50def6e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d806be60 con 0x7f50d815da80 2024-04-25T08:56:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50def6e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d815de60 con 0x7f50d80d1cd0 2024-04-25T08:56:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50def6e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d806c440 con 0x7f50d80d07f0 2024-04-25T08:56:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50dcd0a640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f50d815da80 0x7f50d80d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56522/0 (socket says 172.21.15.37:56522) 2024-04-25T08:56:44.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50dcd0a640 1 -- 172.21.15.37:0/3617756668 learned_addr learned my addr 172.21.15.37:0/3617756668 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:44.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d7fff640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d80d07f0 0x7f50d80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719569153 0 0) 0x7f50d806be60 con 0x7f50d815da80 2024-04-25T08:56:44.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c4003530 con 0x7f50d815da80 2024-04-25T08:56:44.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3173898708 0 0) 0x7f50c4003530 con 0x7f50d815da80 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d80d07f0 msgr2=0x7f50d80d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d80d07f0 0x7f50d80d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 >> v1:172.21.15.203:6789/0 conn(0x7f50d80d1cd0 legacy=0x7f50d806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d80d2450 con 0x7f50d815da80 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f50c0001680 con 0x7f50d815da80 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 >> v1:172.21.15.37:6789/0 conn(0x7f50d815da80 legacy=0x7f50d80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 0x7f50c4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d80d07f0 0x7f50c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4004d50 0x7f50c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d806c440 con 0x7f50c4003a60 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c400bb10 con 0x7f50c4004d50 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c400bd10 con 0x7f50d80d07f0 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50dcd0a640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 0x7f50c4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50d7fff640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d80d07f0 0x7f50c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50dd50b640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4004d50 0x7f50c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50dcd0a640 1 -- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d80d07f0 msgr2=0x7f50c4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50dcd0a640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d80d07f0 0x7f50c4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50dcd0a640 1 -- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4004d50 msgr2=0x7f50c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50dcd0a640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4004d50 0x7f50c40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50dcd0a640 1 -- 172.21.15.37:0/3617756668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c4003530 con 0x7f50c4003a60 2024-04-25T08:56:44.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.174+0000 7f50dd50b640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50c4004d50 0x7f50c40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:44.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50dcd0a640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 0x7f50c4003e80 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c210 tx=0x7f50c001cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33adc6e0dd015859 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0007d20 con 0x7f50c4003a60 2024-04-25T08:56:44.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c00a9070 con 0x7f50c4003a60 2024-04-25T08:56:44.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50d77fe640 1 -- 172.21.15.37:0/3617756668 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0005d20 con 0x7f50c4003a60 2024-04-25T08:56:44.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50def6e640 1 -- 172.21.15.37:0/3617756668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 msgr2=0x7f50c4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50def6e640 1 --2- 172.21.15.37:0/3617756668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 0x7f50c4003e80 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c210 tx=0x7f50c001cb70 comp rx=0 tx=0).stop 2024-04-25T08:56:44.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50dd50b640 1 -- 172.21.15.37:0/3617756668 reap_dead start 2024-04-25T08:56:44.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50def6e640 1 -- 172.21.15.37:0/3617756668 shutdown_connections 2024-04-25T08:56:44.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50def6e640 1 -- 172.21.15.37:0/3617756668 >> 172.21.15.37:0/3617756668 conn(0x7f50d808b000 msgr2=0x7f50d80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50def6e640 1 -- 172.21.15.37:0/3617756668 shutdown_connections 2024-04-25T08:56:44.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50def6e640 1 -- 172.21.15.37:0/3617756668 wait complete. 2024-04-25T08:56:44.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50def6e640 1 Processor -- start 2024-04-25T08:56:44.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.178+0000 7f50def6e640 1 -- start start 2024-04-25T08:56:44.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50def6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 0x7f50d81612c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50def6e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d8161800 0x7f50d80b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50def6e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50d80b56b0 0x7f50d81472e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50def6e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d806d970 con 0x7f50c4003a60 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50def6e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d806d7f0 con 0x7f50d80b56b0 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50def6e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d806daf0 con 0x7f50d8161800 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50dcd0a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 0x7f50d81612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50dcd0a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 0x7f50d81612c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60744/0 (socket says 172.21.15.37:60744) 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50dcd0a640 1 -- 172.21.15.37:0/1556815192 learned_addr learned my addr 172.21.15.37:0/1556815192 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50d7fff640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d8161800 0x7f50d80b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50dcd0a640 1 -- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d8161800 msgr2=0x7f50d80b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50dd50b640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50d80b56b0 0x7f50d81472e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50dcd0a640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d8161800 0x7f50d80b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50dcd0a640 1 -- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50d80b56b0 msgr2=0x7f50d81472e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50dcd0a640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50d80b56b0 0x7f50d81472e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50dcd0a640 1 -- 172.21.15.37:0/1556815192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d8161d10 con 0x7f50c4003a60 2024-04-25T08:56:44.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50dcd0a640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 0x7f50d81612c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f50c001c5e0 tx=0x7f50c4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50d57fa640 1 -- 172.21.15.37:0/1556815192 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c00047f0 con 0x7f50c4003a60 2024-04-25T08:56:44.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50d57fa640 1 -- 172.21.15.37:0/1556815192 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c0004990 con 0x7f50c4003a60 2024-04-25T08:56:44.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50d57fa640 1 -- 172.21.15.37:0/1556815192 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0004d40 con 0x7f50c4003a60 2024-04-25T08:56:44.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50def6e640 1 -- 172.21.15.37:0/1556815192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d80cb210 con 0x7f50c4003a60 2024-04-25T08:56:44.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50def6e640 1 -- 172.21.15.37:0/1556815192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d80cb7d0 con 0x7f50c4003a60 2024-04-25T08:56:44.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50def6e640 1 -- 172.21.15.37:0/1556815192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d804efa0 con 0x7f50c4003a60 2024-04-25T08:56:44.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50d57fa640 1 -- 172.21.15.37:0/1556815192 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f50c0007070 con 0x7f50c4003a60 2024-04-25T08:56:44.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50d57fa640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f50b8066f60 0x7f50b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50d57fa640 1 -- 172.21.15.37:0/1556815192 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f50c011e4d0 con 0x7f50c4003a60 2024-04-25T08:56:44.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.182+0000 7f50d7fff640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f50b8066f60 0x7f50b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.186+0000 7f50d7fff640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f50b8066f60 0x7f50b8069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f50d81fa070 tx=0x7f50c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.190+0000 7f50d57fa640 1 -- 172.21.15.37:0/1556815192 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f50c00168b0 con 0x7f50c4003a60 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.346+0000 7f50def6e640 1 -- 172.21.15.37:0/1556815192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50d8164360 con 0x7f50c4003a60 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50d57fa640 1 -- 172.21.15.37:0/1556815192 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+13258 (secure 0 0 0) 0x7f50c00ef6e0 con 0x7f50c4003a60 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":74,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:56:42.998971+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2189232625":"2024-04-26T08:56:39.986283+0000","172.21.15.37:6834/2189232625":"2024-04-26T08:56:39.986283+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 -- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f50b8066f60 msgr2=0x7f50b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f50b8066f60 0x7f50b8069420 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f50d81fa070 tx=0x7f50c8008040 comp rx=0 tx=0).stop 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 -- 172.21.15.37:0/1556815192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 msgr2=0x7f50d81612c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 0x7f50d81612c0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f50c001c5e0 tx=0x7f50c4005840 comp rx=0 tx=0).stop 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 -- 172.21.15.37:0/1556815192 shutdown_connections 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f50b8066f60 0x7f50b8069420 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f50d80b56b0 0x7f50d81472e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f50d8161800 0x7f50d80b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 --2- 172.21.15.37:0/1556815192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f50c4003a60 0x7f50d81612c0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 -- 172.21.15.37:0/1556815192 >> 172.21.15.37:0/1556815192 conn(0x7f50d808b000 msgr2=0x7f50d80d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 -- 172.21.15.37:0/1556815192 shutdown_connections 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.350+0000 7f50def6e640 1 -- 172.21.15.37:0/1556815192 wait complete. 2024-04-25T08:56:44.359 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:56:44.359 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:56:44.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6d50f9640 1 Processor -- start 2024-04-25T08:56:44.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6d50f9640 1 -- start start 2024-04-25T08:56:44.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6d50f9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd6d015d8d0 0x7fd6d015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6d50f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d006d330 0x7fd6d00cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6d50f9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d006c570 con 0x7fd6d00ce370 2024-04-25T08:56:44.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6d50f9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d006cea0 con 0x7fd6d015d8d0 2024-04-25T08:56:44.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6d50f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d006c850 con 0x7fd6d006d330 2024-04-25T08:56:44.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6ced9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd6d015d8d0 0x7fd6d015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6ced9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd6d015d8d0 0x7fd6d015dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60754/0 (socket says 172.21.15.37:60754) 2024-04-25T08:56:44.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6ced9d640 1 -- 172.21.15.37:0/199712247 learned_addr learned my addr 172.21.15.37:0/199712247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:44.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6ce59c640 1 --2- 172.21.15.37:0/199712247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d006d330 0x7fd6d00cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6ced9d640 1 -- 172.21.15.37:0/199712247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d006d330 msgr2=0x7fd6d00cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6ced9d640 1 --2- 172.21.15.37:0/199712247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d006d330 0x7fd6d00cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6ced9d640 1 -- 172.21.15.37:0/199712247 >> v1:172.21.15.203:6789/0 conn(0x7fd6d00ce370 legacy=0x7fd6d0099df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.498+0000 7fd6ced9d640 1 -- 172.21.15.37:0/199712247 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d006cb30 con 0x7fd6d015d8d0 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6ced9d640 1 --2- 172.21.15.37:0/199712247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d015d8d0 0x7fd6d015dcb0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009870 tx=0x7fd6c001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bfb30a8a083f0cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6cdd9b640 1 -- 172.21.15.37:0/199712247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c00ab5f0 con 0x7fd6d015d8d0 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6cdd9b640 1 -- 172.21.15.37:0/199712247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c00b0070 con 0x7fd6d015d8d0 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6cdd9b640 1 -- 172.21.15.37:0/199712247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c00ab9b0 con 0x7fd6d015d8d0 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 -- 172.21.15.37:0/199712247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d015d8d0 msgr2=0x7fd6d015dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 --2- 172.21.15.37:0/199712247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d015d8d0 0x7fd6d015dcb0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009870 tx=0x7fd6c001c520 comp rx=0 tx=0).stop 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 -- 172.21.15.37:0/199712247 shutdown_connections 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 --2- 172.21.15.37:0/199712247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d006d330 0x7fd6d00cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 --2- 172.21.15.37:0/199712247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d015d8d0 0x7fd6d015dcb0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 -- 172.21.15.37:0/199712247 >> 172.21.15.37:0/199712247 conn(0x7fd6d00d15b0 msgr2=0x7fd6d00d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 -- 172.21.15.37:0/199712247 shutdown_connections 2024-04-25T08:56:44.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 -- 172.21.15.37:0/199712247 wait complete. 2024-04-25T08:56:44.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 Processor -- start 2024-04-25T08:56:44.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 -- start start 2024-04-25T08:56:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d006d330 0x7fd6d0155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d00ce370 0x7fd6d01564b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d015d8d0 0x7fd6d00c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d00cc7e0 con 0x7fd6d015d8d0 2024-04-25T08:56:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d00cc660 con 0x7fd6d006d330 2024-04-25T08:56:44.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6d50f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d00cc960 con 0x7fd6d00ce370 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6cf59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d015d8d0 0x7fd6d00c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6cf59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d015d8d0 0x7fd6d00c50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60756/0 (socket says 172.21.15.37:60756) 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6cf59e640 1 -- 172.21.15.37:0/4118572722 learned_addr learned my addr 172.21.15.37:0/4118572722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.502+0000 7fd6ce59c640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d00ce370 0x7fd6d01564b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6ced9d640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d006d330 0x7fd6d0155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6cf59e640 1 -- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d00ce370 msgr2=0x7fd6d01564b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6cf59e640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d00ce370 0x7fd6d01564b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6cf59e640 1 -- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d006d330 msgr2=0x7fd6d0155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6cf59e640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d006d330 0x7fd6d0155d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6cf59e640 1 -- 172.21.15.37:0/4118572722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d01562b0 con 0x7fd6d015d8d0 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6ced9d640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d006d330 0x7fd6d0155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6cf59e640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d015d8d0 0x7fd6d00c50e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c780 tx=0x7fd6c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6b77fe640 1 -- 172.21.15.37:0/4118572722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8004b70 con 0x7fd6d015d8d0 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6b77fe640 1 -- 172.21.15.37:0/4118572722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c8004d10 con 0x7fd6d015d8d0 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6d50f9640 1 -- 172.21.15.37:0/4118572722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d01577a0 con 0x7fd6d015d8d0 2024-04-25T08:56:44.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6b77fe640 1 -- 172.21.15.37:0/4118572722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c800aa90 con 0x7fd6d015d8d0 2024-04-25T08:56:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6d50f9640 1 -- 172.21.15.37:0/4118572722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d01571e0 con 0x7fd6d015d8d0 2024-04-25T08:56:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6d50f9640 1 -- 172.21.15.37:0/4118572722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d004efa0 con 0x7fd6d015d8d0 2024-04-25T08:56:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6b77fe640 1 -- 172.21.15.37:0/4118572722 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6c800ac30 con 0x7fd6d015d8d0 2024-04-25T08:56:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6b77fe640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd6bc066f60 0x7fd6bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6b77fe640 1 -- 172.21.15.37:0/4118572722 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd6c8085130 con 0x7fd6d015d8d0 2024-04-25T08:56:44.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6ced9d640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd6bc066f60 0x7fd6bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.506+0000 7fd6ced9d640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd6bc066f60 0x7fd6bc069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00af000 tx=0x7fd6c0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.510+0000 7fd6b77fe640 1 -- 172.21.15.37:0/4118572722 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6c8020070 con 0x7fd6d015d8d0 2024-04-25T08:56:44.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.662+0000 7fd6d50f9640 1 -- 172.21.15.37:0/4118572722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6d00cb7c0 con 0x7fd6d015d8d0 2024-04-25T08:56:44.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6b77fe640 1 -- 172.21.15.37:0/4118572722 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd6d01562b0 con 0x7fd6d015d8d0 2024-04-25T08:56:44.655 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:56:44.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 -- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd6bc066f60 msgr2=0x7fd6bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd6bc066f60 0x7fd6bc069420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00af000 tx=0x7fd6c0008040 comp rx=0 tx=0).stop 2024-04-25T08:56:44.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 -- 172.21.15.37:0/4118572722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d015d8d0 msgr2=0x7fd6d00c50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d015d8d0 0x7fd6d00c50e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd6c800c780 tx=0x7fd6c800cc40 comp rx=0 tx=0).stop 2024-04-25T08:56:44.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 -- 172.21.15.37:0/4118572722 shutdown_connections 2024-04-25T08:56:44.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd6bc066f60 0x7fd6bc069420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6d015d8d0 0x7fd6d00c50e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6d00ce370 0x7fd6d01564b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 --2- 172.21.15.37:0/4118572722 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6d006d330 0x7fd6d0155d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 -- 172.21.15.37:0/4118572722 >> 172.21.15.37:0/4118572722 conn(0x7fd6d00d15b0 msgr2=0x7fd6d0073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 -- 172.21.15.37:0/4118572722 shutdown_connections 2024-04-25T08:56:44.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.666+0000 7fd6d50f9640 1 -- 172.21.15.37:0/4118572722 wait complete. 2024-04-25T08:56:44.672 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T08:56:44.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a9632640 1 Processor -- start 2024-04-25T08:56:44.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a9632640 1 -- start start 2024-04-25T08:56:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a9632640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f22a415da80 0x7f22a40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a9632640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f22a40d1cd0 0x7f22a40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a9632640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a406be60 con 0x7f22a40d0620 2024-04-25T08:56:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a9632640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a40d0a70 con 0x7f22a415da80 2024-04-25T08:56:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a9632640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a406c440 con 0x7f22a40d1cd0 2024-04-25T08:56:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f22a415da80 0x7f22a40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f22a415da80 0x7f22a40d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60758/0 (socket says 172.21.15.37:60758) 2024-04-25T08:56:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a2ffd640 1 -- 172.21.15.37:0/3474480671 learned_addr learned my addr 172.21.15.37:0/3474480671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a27fc640 1 --2- 172.21.15.37:0/3474480671 >> v2:172.21.15.203:3300/0 conn(0x7f22a40d1cd0 0x7f22a40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a2ffd640 1 -- 172.21.15.37:0/3474480671 >> v2:172.21.15.203:3300/0 conn(0x7f22a40d1cd0 msgr2=0x7f22a40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a2ffd640 1 --2- 172.21.15.37:0/3474480671 >> v2:172.21.15.203:3300/0 conn(0x7f22a40d1cd0 0x7f22a40d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a2ffd640 1 -- 172.21.15.37:0/3474480671 >> v1:172.21.15.203:6789/0 conn(0x7f22a40d0620 legacy=0x7f22a40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.794+0000 7f22a2ffd640 1 -- 172.21.15.37:0/3474480671 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a415de60 con 0x7f22a415da80 2024-04-25T08:56:44.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a2ffd640 1 --2- 172.21.15.37:0/3474480671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a415da80 0x7f22a40d4230 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f229c009e50 tx=0x7f229c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e484b35d76286e11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a1ffb640 1 -- 172.21.15.37:0/3474480671 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0a5710 con 0x7f22a415da80 2024-04-25T08:56:44.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a1ffb640 1 -- 172.21.15.37:0/3474480671 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c0a58b0 con 0x7f22a415da80 2024-04-25T08:56:44.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a1ffb640 1 -- 172.21.15.37:0/3474480671 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0a5c80 con 0x7f22a415da80 2024-04-25T08:56:44.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a9632640 1 -- 172.21.15.37:0/3474480671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a415da80 msgr2=0x7f22a40d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a9632640 1 --2- 172.21.15.37:0/3474480671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a415da80 0x7f22a40d4230 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f229c009e50 tx=0x7f229c01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a9632640 1 -- 172.21.15.37:0/3474480671 shutdown_connections 2024-04-25T08:56:44.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a9632640 1 --2- 172.21.15.37:0/3474480671 >> v2:172.21.15.203:3300/0 conn(0x7f22a40d1cd0 0x7f22a40d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a9632640 1 --2- 172.21.15.37:0/3474480671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a415da80 0x7f22a40d4230 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a9632640 1 -- 172.21.15.37:0/3474480671 >> 172.21.15.37:0/3474480671 conn(0x7f22a408b000 msgr2=0x7f22a40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a9632640 1 -- 172.21.15.37:0/3474480671 shutdown_connections 2024-04-25T08:56:44.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.798+0000 7f22a9632640 1 -- 172.21.15.37:0/3474480671 wait complete. 2024-04-25T08:56:44.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a9632640 1 Processor -- start 2024-04-25T08:56:44.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a9632640 1 -- start start 2024-04-25T08:56:44.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a9632640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a40d0620 0x7f22a41531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a9632640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22a40d1cd0 0x7f22a4153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a9632640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a415da80 0x7f22a40b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a9632640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a416a730 con 0x7f22a415da80 2024-04-25T08:56:44.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a9632640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a416a5b0 con 0x7f22a40d0620 2024-04-25T08:56:44.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a9632640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a416a8b0 con 0x7f22a40d1cd0 2024-04-25T08:56:44.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a415da80 0x7f22a40b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a415da80 0x7f22a40b1e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60768/0 (socket says 172.21.15.37:60768) 2024-04-25T08:56:44.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a37fe640 1 -- 172.21.15.37:0/3445754065 learned_addr learned my addr 172.21.15.37:0/3445754065 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:44.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a27fc640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22a40d1cd0 0x7f22a4153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a2ffd640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a40d0620 0x7f22a41531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a37fe640 1 -- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22a40d1cd0 msgr2=0x7f22a4153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a37fe640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22a40d1cd0 0x7f22a4153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a37fe640 1 -- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a40d0620 msgr2=0x7f22a41531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a37fe640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a40d0620 0x7f22a41531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a37fe640 1 -- 172.21.15.37:0/3445754065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4153700 con 0x7f22a415da80 2024-04-25T08:56:44.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a27fc640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22a40d1cd0 0x7f22a4153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:44.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a37fe640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a415da80 0x7f22a40b1e60 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f229400c460 tx=0x7f229400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f229b7fe640 1 -- 172.21.15.37:0/3445754065 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294016020 con 0x7f22a415da80 2024-04-25T08:56:44.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f229b7fe640 1 -- 172.21.15.37:0/3445754065 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2294005220 con 0x7f22a415da80 2024-04-25T08:56:44.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f229b7fe640 1 -- 172.21.15.37:0/3445754065 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294007030 con 0x7f22a415da80 2024-04-25T08:56:44.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a9632640 1 -- 172.21.15.37:0/3445754065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a4154150 con 0x7f22a415da80 2024-04-25T08:56:44.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a9632640 1 -- 172.21.15.37:0/3445754065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a4154710 con 0x7f22a415da80 2024-04-25T08:56:44.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f22a9632640 1 -- 172.21.15.37:0/3445754065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a404efa0 con 0x7f22a415da80 2024-04-25T08:56:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f229b7fe640 1 -- 172.21.15.37:0/3445754065 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f22940071d0 con 0x7f22a415da80 2024-04-25T08:56:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f229b7fe640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2284066f60 0x7f2284069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.802+0000 7f229b7fe640 1 -- 172.21.15.37:0/3445754065 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f22940847e0 con 0x7f22a415da80 2024-04-25T08:56:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.806+0000 7f22a2ffd640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2284066f60 0x7f2284069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.806+0000 7f22a2ffd640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2284066f60 0x7f2284069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f229c0ae000 tx=0x7f229c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:44.806+0000 7f229b7fe640 1 -- 172.21.15.37:0/3445754065 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2294020020 con 0x7f22a415da80 2024-04-25T08:56:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:45.018+0000 7f22a9632640 1 -- 172.21.15.37:0/3445754065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7f22a4153e40 con 0x7f22a415da80 2024-04-25T08:56:46.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.030+0000 7f229b7fe640 1 -- 172.21.15.37:0/3445754065 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v237) v1 ==== 169+0+0 (secure 0 0 0) 0x7f22a4153700 con 0x7f22a415da80 2024-04-25T08:56:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.030+0000 7f22a9632640 1 -- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2284066f60 msgr2=0x7f2284069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.030+0000 7f22a9632640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2284066f60 0x7f2284069420 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f229c0ae000 tx=0x7f229c0a7070 comp rx=0 tx=0).stop 2024-04-25T08:56:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.034+0000 7f22a9632640 1 -- 172.21.15.37:0/3445754065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a415da80 msgr2=0x7f22a40b1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.034+0000 7f22a9632640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a415da80 0x7f22a40b1e60 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f229400c460 tx=0x7f229400c920 comp rx=0 tx=0).stop 2024-04-25T08:56:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.034+0000 7f22a9632640 1 -- 172.21.15.37:0/3445754065 shutdown_connections 2024-04-25T08:56:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.034+0000 7f22a9632640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2284066f60 0x7f2284069420 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.034+0000 7f22a9632640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22a415da80 0x7f22a40b1e60 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.034+0000 7f22a9632640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22a40d1cd0 0x7f22a4153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.034+0000 7f22a9632640 1 --2- 172.21.15.37:0/3445754065 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22a40d0620 0x7f22a41531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.034+0000 7f22a9632640 1 -- 172.21.15.37:0/3445754065 >> 172.21.15.37:0/3445754065 conn(0x7f22a408b000 msgr2=0x7f22a40b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.034+0000 7f22a9632640 1 -- 172.21.15.37:0/3445754065 shutdown_connections 2024-04-25T08:56:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.034+0000 7f22a9632640 1 -- 172.21.15.37:0/3445754065 wait complete. 2024-04-25T08:56:46.048 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:56:46.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.194+0000 7f2c368f1640 1 Processor -- start 2024-04-25T08:56:46.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.194+0000 7f2c368f1640 1 -- start start 2024-04-25T08:56:46.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c368f1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2c3015da80 0x7f2c300d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c368f1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3006be60 con 0x7f2c300d07f0 2024-04-25T08:56:46.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c368f1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c3015de60 con 0x7f2c300d1cd0 2024-04-25T08:56:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c368f1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3006c440 con 0x7f2c3015da80 2024-04-25T08:56:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c2ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2c3015da80 0x7f2c300d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c34e8e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2c300d07f0 0x7f2c3006c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56524/0 (socket says 172.21.15.37:56524) 2024-04-25T08:56:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c34e8e640 1 -- 172.21.15.37:0/3338790994 learned_addr learned my addr 172.21.15.37:0/3338790994 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c2ffff640 1 -- 172.21.15.37:0/3338790994 >> v1:172.21.15.203:6789/0 conn(0x7f2c300d1cd0 legacy=0x7f2c300d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c2ffff640 1 -- 172.21.15.37:0/3338790994 >> v1:172.21.15.37:6789/0 conn(0x7f2c300d07f0 legacy=0x7f2c3006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c2effd640 1 -- 172.21.15.37:0/3338790994 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1562491543 0 0) 0x7f2c3006be60 con 0x7f2c300d07f0 2024-04-25T08:56:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c2ffff640 1 -- 172.21.15.37:0/3338790994 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c300d2450 con 0x7f2c3015da80 2024-04-25T08:56:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c2ffff640 1 --2- 172.21.15.37:0/3338790994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c3015da80 0x7f2c300d4230 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2c3006c5c0 tx=0x7f2c2001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91884bf7d0483e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c2effd640 1 -- 172.21.15.37:0/3338790994 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200b1070 con 0x7f2c3015da80 2024-04-25T08:56:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c2effd640 1 -- 172.21.15.37:0/3338790994 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c200ac410 con 0x7f2c3015da80 2024-04-25T08:56:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c2effd640 1 -- 172.21.15.37:0/3338790994 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200ac7e0 con 0x7f2c3015da80 2024-04-25T08:56:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c368f1640 1 -- 172.21.15.37:0/3338790994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c3015da80 msgr2=0x7f2c300d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.198+0000 7f2c368f1640 1 --2- 172.21.15.37:0/3338790994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c3015da80 0x7f2c300d4230 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f2c3006c5c0 tx=0x7f2c2001c410 comp rx=0 tx=0).stop 2024-04-25T08:56:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 -- 172.21.15.37:0/3338790994 shutdown_connections 2024-04-25T08:56:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 --2- 172.21.15.37:0/3338790994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c3015da80 0x7f2c300d4230 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 -- 172.21.15.37:0/3338790994 >> 172.21.15.37:0/3338790994 conn(0x7f2c3008b000 msgr2=0x7f2c300d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 -- 172.21.15.37:0/3338790994 shutdown_connections 2024-04-25T08:56:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 -- 172.21.15.37:0/3338790994 wait complete. 2024-04-25T08:56:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 Processor -- start 2024-04-25T08:56:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 -- start start 2024-04-25T08:56:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c300d07f0 0x7f2c30151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c300d1cd0 0x7f2c300c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c3015da80 0x7f2c300b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3006d970 con 0x7f2c300d1cd0 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3006d7f0 con 0x7f2c3015da80 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3006daf0 con 0x7f2c300d07f0 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c300d1cd0 0x7f2c300c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c300d1cd0 0x7f2c300c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60776/0 (socket says 172.21.15.37:60776) 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2f7fe640 1 -- 172.21.15.37:0/3165408079 learned_addr learned my addr 172.21.15.37:0/3165408079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2ffff640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c300d07f0 0x7f2c30151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c34e8e640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c3015da80 0x7f2c300b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2f7fe640 1 -- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c300d07f0 msgr2=0x7f2c30151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2f7fe640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c300d07f0 0x7f2c30151070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2f7fe640 1 -- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c3015da80 msgr2=0x7f2c300b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2f7fe640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c3015da80 0x7f2c300b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2f7fe640 1 -- 172.21.15.37:0/3165408079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c301515b0 con 0x7f2c300d1cd0 2024-04-25T08:56:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2f7fe640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c300d1cd0 0x7f2c300c8940 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400c430 tx=0x7f2c2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2cff9640 1 -- 172.21.15.37:0/3165408079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24011790 con 0x7f2c300d1cd0 2024-04-25T08:56:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2cff9640 1 -- 172.21.15.37:0/3165408079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c24017070 con 0x7f2c300d1cd0 2024-04-25T08:56:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 -- 172.21.15.37:0/3165408079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c30151870 con 0x7f2c300d1cd0 2024-04-25T08:56:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c2cff9640 1 -- 172.21.15.37:0/3165408079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24011bb0 con 0x7f2c300d1cd0 2024-04-25T08:56:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 -- 172.21.15.37:0/3165408079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c30151e30 con 0x7f2c300d1cd0 2024-04-25T08:56:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.202+0000 7f2c368f1640 1 -- 172.21.15.37:0/3165408079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3004efa0 con 0x7f2c300d1cd0 2024-04-25T08:56:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.206+0000 7f2c2cff9640 1 -- 172.21.15.37:0/3165408079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c24012420 con 0x7f2c300d1cd0 2024-04-25T08:56:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.206+0000 7f2c2cff9640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c1c066f60 0x7f2c1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.206+0000 7f2c2cff9640 1 -- 172.21.15.37:0/3165408079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f2c24084930 con 0x7f2c300d1cd0 2024-04-25T08:56:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.206+0000 7f2c2ffff640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c1c066f60 0x7f2c1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.206+0000 7f2c2ffff640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c1c066f60 0x7f2c1c069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2c200ae000 tx=0x7f2c20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.210+0000 7f2c2cff9640 1 -- 172.21.15.37:0/3165408079 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c24055b40 con 0x7f2c300d1cd0 2024-04-25T08:56:46.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.370+0000 7f2c368f1640 1 -- 172.21.15.37:0/3165408079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2c300b25d0 con 0x7f2c300d1cd0 2024-04-25T08:56:46.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c2cff9640 1 -- 172.21.15.37:0/3165408079 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2c24055440 con 0x7f2c300d1cd0 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 -- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c1c066f60 msgr2=0x7f2c1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c1c066f60 0x7f2c1c069420 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2c200ae000 tx=0x7f2c20008040 comp rx=0 tx=0).stop 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 -- 172.21.15.37:0/3165408079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c300d1cd0 msgr2=0x7f2c300c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c300d1cd0 0x7f2c300c8940 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f2c2400c430 tx=0x7f2c2400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 -- 172.21.15.37:0/3165408079 shutdown_connections 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c1c066f60 0x7f2c1c069420 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c3015da80 0x7f2c300b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c300d1cd0 0x7f2c300c8940 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 --2- 172.21.15.37:0/3165408079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c300d07f0 0x7f2c30151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 -- 172.21.15.37:0/3165408079 >> 172.21.15.37:0/3165408079 conn(0x7f2c3008b000 msgr2=0x7f2c300b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 -- 172.21.15.37:0/3165408079 shutdown_connections 2024-04-25T08:56:46.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.378+0000 7f2c368f1640 1 -- 172.21.15.37:0/3165408079 wait complete. 2024-04-25T08:56:46.387 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:56:46.388 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:56:46.440 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:56:46.492 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:56:46.492 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:56:46.492 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:46.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:56:46.498 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:56:46.498 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:46.504 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:56:46.504 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:56:46.505 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:46.548 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:56:46.548 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:56:46.549 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:46.554 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:56:46.554 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:56:46.554 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1aea921640 1 Processor -- start 2024-04-25T08:56:46.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1aea921640 1 -- start start 2024-04-25T08:56:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1aea921640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ae415da80 0x7f1ae40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1aea921640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d1cd0 0x7f1ae40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1aea921640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ae406be60 con 0x7f1ae40d0620 2024-04-25T08:56:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1aea921640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ae406c440 con 0x7f1ae415da80 2024-04-25T08:56:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1aea921640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae415de60 con 0x7f1ae40d1cd0 2024-04-25T08:56:46.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1ae3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ae415da80 0x7f1ae40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1ae3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ae415da80 0x7f1ae40d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60788/0 (socket says 172.21.15.37:60788) 2024-04-25T08:56:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1ae3fff640 1 -- 172.21.15.37:0/2856146108 learned_addr learned my addr 172.21.15.37:0/2856146108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1ae37fe640 1 --2- 172.21.15.37:0/2856146108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d1cd0 0x7f1ae40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1ae3fff640 1 -- 172.21.15.37:0/2856146108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d1cd0 msgr2=0x7f1ae40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1ae3fff640 1 --2- 172.21.15.37:0/2856146108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d1cd0 0x7f1ae40d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1ae3fff640 1 -- 172.21.15.37:0/2856146108 >> v1:172.21.15.203:6789/0 conn(0x7f1ae40d0620 legacy=0x7f1ae40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:46.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.734+0000 7f1ae3fff640 1 -- 172.21.15.37:0/2856146108 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae40d2450 con 0x7f1ae415da80 2024-04-25T08:56:46.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1ae3fff640 1 --2- 172.21.15.37:0/2856146108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae415da80 0x7f1ae40d4230 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009870 tx=0x7f1ad401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f5f5852dbdb42c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1ae2ffd640 1 -- 172.21.15.37:0/2856146108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad40b0070 con 0x7f1ae415da80 2024-04-25T08:56:46.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1ae2ffd640 1 -- 172.21.15.37:0/2856146108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad40ab600 con 0x7f1ae415da80 2024-04-25T08:56:46.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1ae2ffd640 1 -- 172.21.15.37:0/2856146108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad40ab9d0 con 0x7f1ae415da80 2024-04-25T08:56:46.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 -- 172.21.15.37:0/2856146108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae415da80 msgr2=0x7f1ae40d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 --2- 172.21.15.37:0/2856146108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae415da80 0x7f1ae40d4230 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009870 tx=0x7f1ad401c520 comp rx=0 tx=0).stop 2024-04-25T08:56:46.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 -- 172.21.15.37:0/2856146108 shutdown_connections 2024-04-25T08:56:46.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 --2- 172.21.15.37:0/2856146108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae40d1cd0 0x7f1ae40d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 --2- 172.21.15.37:0/2856146108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae415da80 0x7f1ae40d4230 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 -- 172.21.15.37:0/2856146108 >> 172.21.15.37:0/2856146108 conn(0x7f1ae408b000 msgr2=0x7f1ae40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:46.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 -- 172.21.15.37:0/2856146108 shutdown_connections 2024-04-25T08:56:46.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 -- 172.21.15.37:0/2856146108 wait complete. 2024-04-25T08:56:46.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 Processor -- start 2024-04-25T08:56:46.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 -- start start 2024-04-25T08:56:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae40d0620 0x7f1ae40cc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae40d1cd0 0x7f1ae40c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae415da80 0x7f1ae40b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae416a730 con 0x7f1ae40d0620 2024-04-25T08:56:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae416a5b0 con 0x7f1ae40d1cd0 2024-04-25T08:56:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.738+0000 7f1aea921640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae416a8b0 con 0x7f1ae415da80 2024-04-25T08:56:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae40d0620 0x7f1ae40cc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae40d0620 0x7f1ae40cc0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60794/0 (socket says 172.21.15.37:60794) 2024-04-25T08:56:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae3fff640 1 -- 172.21.15.37:0/175546093 learned_addr learned my addr 172.21.15.37:0/175546093 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae8ebe640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae415da80 0x7f1ae40b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae3fff640 1 -- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae415da80 msgr2=0x7f1ae40b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae3fff640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae415da80 0x7f1ae40b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae3fff640 1 -- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae40d1cd0 msgr2=0x7f1ae40c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae3fff640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae40d1cd0 0x7f1ae40c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae3fff640 1 -- 172.21.15.37:0/175546093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae40ccc80 con 0x7f1ae40d0620 2024-04-25T08:56:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae3fff640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae40d0620 0x7f1ae40cc0c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40ae040 tx=0x7f1ad40192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae0ff9640 1 -- 172.21.15.37:0/175546093 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad40ba070 con 0x7f1ae40d0620 2024-04-25T08:56:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae0ff9640 1 -- 172.21.15.37:0/175546093 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad40b4660 con 0x7f1ae40d0620 2024-04-25T08:56:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1aea921640 1 -- 172.21.15.37:0/175546093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae40cc9a0 con 0x7f1ae40d0620 2024-04-25T08:56:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae0ff9640 1 -- 172.21.15.37:0/175546093 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad40b0040 con 0x7f1ae40d0620 2024-04-25T08:56:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1aea921640 1 -- 172.21.15.37:0/175546093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae4155db0 con 0x7f1ae40d0620 2024-04-25T08:56:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1aea921640 1 -- 172.21.15.37:0/175546093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae404efa0 con 0x7f1ae40d0620 2024-04-25T08:56:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae0ff9640 1 -- 172.21.15.37:0/175546093 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ad40ab4c0 con 0x7f1ae40d0620 2024-04-25T08:56:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae0ff9640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ac4066f60 0x7f1ac4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae0ff9640 1 -- 172.21.15.37:0/175546093 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1ad411e100 con 0x7f1ae40d0620 2024-04-25T08:56:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.742+0000 7f1ae37fe640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ac4066f60 0x7f1ac4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.746+0000 7f1ae37fe640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ac4066f60 0x7f1ac4069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0070b0 tx=0x7f1acc007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.746+0000 7f1ae0ff9640 1 -- 172.21.15.37:0/175546093 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1ad4016110 con 0x7f1ae40d0620 2024-04-25T08:56:46.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.942+0000 7f1aea921640 1 -- 172.21.15.37:0/175546093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ae4160bb0 con 0x7f1ae40d0620 2024-04-25T08:56:46.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.942+0000 7f1ae0ff9640 1 -- 172.21.15.37:0/175546093 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f1ad40eebb0 con 0x7f1ae40d0620 2024-04-25T08:56:46.934 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:46.934 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5237,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/486740526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":486740526},{"type":"v1","addr":"172.21.15.37:6833","nonce":486740526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5249,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/121484331","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":121484331},{"type":"v1","addr":"172.21.15.203:6835","nonce":121484331}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5746,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2501630566","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2501630566},{"type":"v1","addr":"172.21.15.203:6837","nonce":2501630566}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1741046847","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1741046847},{"type":"v1","addr":"172.21.15.37:6835","nonce":1741046847}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[]} 2024-04-25T08:56:46.935 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 37 2024-04-25T08:56:46.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 -- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ac4066f60 msgr2=0x7f1ac4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ac4066f60 0x7f1ac4069420 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1acc0070b0 tx=0x7f1acc007040 comp rx=0 tx=0).stop 2024-04-25T08:56:46.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 -- 172.21.15.37:0/175546093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae40d0620 msgr2=0x7f1ae40cc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae40d0620 0x7f1ae40cc0c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1ad40ae040 tx=0x7f1ad40192c0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 -- 172.21.15.37:0/175546093 shutdown_connections 2024-04-25T08:56:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ac4066f60 0x7f1ac4069420 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ae415da80 0x7f1ae40b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ae40d1cd0 0x7f1ae40c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 --2- 172.21.15.37:0/175546093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae40d0620 0x7f1ae40cc0c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 -- 172.21.15.37:0/175546093 >> 172.21.15.37:0/175546093 conn(0x7f1ae408b000 msgr2=0x7f1ae40b51c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 -- 172.21.15.37:0/175546093 shutdown_connections 2024-04-25T08:56:46.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:46.946+0000 7f1aea921640 1 -- 172.21.15.37:0/175546093 wait complete. 2024-04-25T08:56:46.952 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:56:46.952 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:56:46.953 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:56:46.953 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:56:46.953 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:56:46.953 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:56:46.953 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:56:46.953 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:56:46.953 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:56:46.954 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:56:46.954 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:56:46.954 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:56:46.954 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:56:46.954 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:56:46.954 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:56:46.954 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:56:46.964 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:56:46.964 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:56:46.965 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:56:46.965 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:56:46.967 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:56:46.967 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:56:46.968 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:56:46.968 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:56:47.008 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:56:47.008 DEBUG:teuthology.parallel:result is None 2024-04-25T08:56:47.009 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:56:47.009 DEBUG:teuthology.parallel:result is None 2024-04-25T08:56:47.012 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:56:47.012 DEBUG:teuthology.parallel:result is None 2024-04-25T08:56:47.012 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:56:47.013 DEBUG:teuthology.parallel:result is None 2024-04-25T08:56:47.013 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:56:47.069 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T08:56:47.070 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T08:56:47.081 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-25T08:56:47.082 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T08:56:47.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fcd0701b640 1 Processor -- start 2024-04-25T08:56:47.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fcd0701b640 1 -- start start 2024-04-25T08:56:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fcd0701b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcd0015ba20 0x7fcd000cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fcd0701b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcd0015fd90 0x7fcd0006ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fcd0701b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd0006b5e0 con 0x7fcd0006b1d0 2024-04-25T08:56:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fcd0701b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd000d1430 con 0x7fcd0015fd90 2024-04-25T08:56:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fcd0701b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd000cc6e0 con 0x7fcd0015ba20 2024-04-25T08:56:47.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fccf7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcd0015fd90 0x7fcd0006ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fccf7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcd0015fd90 0x7fcd0006ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60824/0 (socket says 172.21.15.37:60824) 2024-04-25T08:56:47.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fccf7fff640 1 -- 172.21.15.37:0/1631551746 learned_addr learned my addr 172.21.15.37:0/1631551746 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:47.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fcd04db7640 1 --2- 172.21.15.37:0/1631551746 >> v2:172.21.15.203:3300/0 conn(0x7fcd0015ba20 0x7fcd000cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fccf7fff640 1 -- 172.21.15.37:0/1631551746 >> v2:172.21.15.203:3300/0 conn(0x7fcd0015ba20 msgr2=0x7fcd000cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:47.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fccf7fff640 1 --2- 172.21.15.37:0/1631551746 >> v2:172.21.15.203:3300/0 conn(0x7fcd0015ba20 0x7fcd000cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fccf7fff640 1 -- 172.21.15.37:0/1631551746 >> v1:172.21.15.203:6789/0 conn(0x7fcd0006b1d0 legacy=0x7fcd000cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fccf7fff640 1 -- 172.21.15.37:0/1631551746 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0015c260 con 0x7fcd0015fd90 2024-04-25T08:56:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fccf7fff640 1 --2- 172.21.15.37:0/1631551746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd0015fd90 0x7fcd0006ac90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fcd0009a1a0 tx=0x7fccf001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413f5680c5940c2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fccf77fe640 1 -- 172.21.15.37:0/1631551746 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf00ab610 con 0x7fcd0015fd90 2024-04-25T08:56:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fccf77fe640 1 -- 172.21.15.37:0/1631551746 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf00b0070 con 0x7fcd0015fd90 2024-04-25T08:56:47.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.138+0000 7fccf77fe640 1 -- 172.21.15.37:0/1631551746 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf00ab9d0 con 0x7fcd0015fd90 2024-04-25T08:56:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 -- 172.21.15.37:0/1631551746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd0015fd90 msgr2=0x7fcd0006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 --2- 172.21.15.37:0/1631551746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd0015fd90 0x7fcd0006ac90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fcd0009a1a0 tx=0x7fccf001c520 comp rx=0 tx=0).stop 2024-04-25T08:56:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 -- 172.21.15.37:0/1631551746 shutdown_connections 2024-04-25T08:56:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 --2- 172.21.15.37:0/1631551746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd0015fd90 0x7fcd0006ac90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 --2- 172.21.15.37:0/1631551746 >> v2:172.21.15.203:3300/0 conn(0x7fcd0015ba20 0x7fcd000cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 -- 172.21.15.37:0/1631551746 >> 172.21.15.37:0/1631551746 conn(0x7fcd00099a70 msgr2=0x7fcd000d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:47.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 -- 172.21.15.37:0/1631551746 shutdown_connections 2024-04-25T08:56:47.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 -- 172.21.15.37:0/1631551746 wait complete. 2024-04-25T08:56:47.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 Processor -- start 2024-04-25T08:56:47.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 -- start start 2024-04-25T08:56:47.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd0006b1d0 0x7fcd0014a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0015ba20 0x7fcd00158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd0015fd90 0x7fcd001575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000d01f0 con 0x7fcd0006b1d0 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd00160220 con 0x7fcd0015ba20 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000c40d0 con 0x7fcd0015fd90 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd04db7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd0006b1d0 0x7fcd0014a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd04db7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd0006b1d0 0x7fcd0014a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60826/0 (socket says 172.21.15.37:60826) 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd04db7640 1 -- 172.21.15.37:0/2890269769 learned_addr learned my addr 172.21.15.37:0/2890269769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fccf7fff640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0015ba20 0x7fcd00158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd055b8640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd0015fd90 0x7fcd001575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd04db7640 1 -- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd0015fd90 msgr2=0x7fcd001575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd04db7640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd0015fd90 0x7fcd001575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd04db7640 1 -- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0015ba20 msgr2=0x7fcd00158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd04db7640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0015ba20 0x7fcd00158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd04db7640 1 -- 172.21.15.37:0/2890269769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd000c4370 con 0x7fcd0006b1d0 2024-04-25T08:56:47.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fccf7fff640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0015ba20 0x7fcd00158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:47.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd04db7640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd0006b1d0 0x7fcd0014a2b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00e9b0 tx=0x7fccfc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fccf57fa640 1 -- 172.21.15.37:0/2890269769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc017070 con 0x7fcd0006b1d0 2024-04-25T08:56:47.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fccf57fa640 1 -- 172.21.15.37:0/2890269769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc00c9a0 con 0x7fcd0006b1d0 2024-04-25T08:56:47.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 -- 172.21.15.37:0/2890269769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd000c4650 con 0x7fcd0006b1d0 2024-04-25T08:56:47.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fccf57fa640 1 -- 172.21.15.37:0/2890269769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc00cd70 con 0x7fcd0006b1d0 2024-04-25T08:56:47.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.142+0000 7fcd0701b640 1 -- 172.21.15.37:0/2890269769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd000c4ca0 con 0x7fcd0006b1d0 2024-04-25T08:56:47.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.146+0000 7fcd0701b640 1 -- 172.21.15.37:0/2890269769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0004efa0 con 0x7fcd0006b1d0 2024-04-25T08:56:47.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.146+0000 7fccf57fa640 1 -- 172.21.15.37:0/2890269769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fccfc01a070 con 0x7fcd0006b1d0 2024-04-25T08:56:47.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.146+0000 7fccf57fa640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fccec066f60 0x7fccec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.146+0000 7fccf57fa640 1 -- 172.21.15.37:0/2890269769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fccfc0853b0 con 0x7fcd0006b1d0 2024-04-25T08:56:47.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.146+0000 7fccf7fff640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fccec066f60 0x7fccec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.146+0000 7fccf7fff640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fccec066f60 0x7fccec069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fccf00af000 tx=0x7fccf00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.150+0000 7fccf57fa640 1 -- 172.21.15.37:0/2890269769 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fccfc0564a0 con 0x7fcd0006b1d0 2024-04-25T08:56:47.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:47.322+0000 7fcd0701b640 1 -- 172.21.15.37:0/2890269769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcd0006b7c0 con 0x7fcd0006b1d0 2024-04-25T08:56:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fccf57fa640 1 -- 172.21.15.37:0/2890269769 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v76) v1 ==== 97+0+0 (secure 0 0 0) 0x7fccfc055da0 con 0x7fcd0006b1d0 2024-04-25T08:56:48.039 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-04-25T08:56:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 -- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fccec066f60 msgr2=0x7fccec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fccec066f60 0x7fccec069420 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fccf00af000 tx=0x7fccf00a7000 comp rx=0 tx=0).stop 2024-04-25T08:56:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 -- 172.21.15.37:0/2890269769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd0006b1d0 msgr2=0x7fcd0014a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd0006b1d0 0x7fcd0014a2b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fccfc00e9b0 tx=0x7fccfc00ee70 comp rx=0 tx=0).stop 2024-04-25T08:56:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 -- 172.21.15.37:0/2890269769 shutdown_connections 2024-04-25T08:56:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fccec066f60 0x7fccec069420 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcd0015fd90 0x7fcd001575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcd0015ba20 0x7fcd00158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 --2- 172.21.15.37:0/2890269769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd0006b1d0 0x7fcd0014a2b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 -- 172.21.15.37:0/2890269769 >> 172.21.15.37:0/2890269769 conn(0x7fcd00099a70 msgr2=0x7fcd000aea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 -- 172.21.15.37:0/2890269769 shutdown_connections 2024-04-25T08:56:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.050+0000 7fcd0701b640 1 -- 172.21.15.37:0/2890269769 wait complete. 2024-04-25T08:56:48.064 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:56:48.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9bee519640 1 Processor -- start 2024-04-25T08:56:48.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9bee519640 1 -- start start 2024-04-25T08:56:48.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9bee519640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9be80b0a30 0x7f9be80cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9bee519640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be809a180 0x7f9be815d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9bee519640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be80b0e10 con 0x7f9be806f4a0 2024-04-25T08:56:48.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9bee519640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be80d0d90 con 0x7f9be80b0a30 2024-04-25T08:56:48.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9bee519640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80d0180 con 0x7f9be809a180 2024-04-25T08:56:48.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9be7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9be80b0a30 0x7f9be80cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9be77fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9be806f4a0 0x7f9be8099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56530/0 (socket says 172.21.15.37:56530) 2024-04-25T08:56:48.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9be7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9be80b0a30 0x7f9be80cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60842/0 (socket says 172.21.15.37:60842) 2024-04-25T08:56:48.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9be77fe640 1 -- 172.21.15.37:0/3349743590 learned_addr learned my addr 172.21.15.37:0/3349743590 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:48.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9becab6640 1 --2- 172.21.15.37:0/3349743590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be809a180 0x7f9be815d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9be7fff640 1 -- 172.21.15.37:0/3349743590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be809a180 msgr2=0x7f9be815d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9be7fff640 1 --2- 172.21.15.37:0/3349743590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be809a180 0x7f9be815d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9be7fff640 1 -- 172.21.15.37:0/3349743590 >> v1:172.21.15.37:6789/0 conn(0x7f9be806f4a0 legacy=0x7f9be8099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9be6ffd640 1 -- 172.21.15.37:0/3349743590 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392665544 0 0) 0x7f9be80b0e10 con 0x7f9be806f4a0 2024-04-25T08:56:48.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.194+0000 7f9be7fff640 1 -- 172.21.15.37:0/3349743590 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80d0460 con 0x7f9be80b0a30 2024-04-25T08:56:48.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9be7fff640 1 --2- 172.21.15.37:0/3349743590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be80b0a30 0x7f9be80cf890 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80097c0 tx=0x7f9bd801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c82b161de8d2db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9be6ffd640 1 -- 172.21.15.37:0/3349743590 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80a7070 con 0x7f9be80b0a30 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9be6ffd640 1 -- 172.21.15.37:0/3349743590 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd80b0070 con 0x7f9be80b0a30 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9be6ffd640 1 -- 172.21.15.37:0/3349743590 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80ab890 con 0x7f9be80b0a30 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9bee519640 1 -- 172.21.15.37:0/3349743590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be80b0a30 msgr2=0x7f9be80cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9bee519640 1 --2- 172.21.15.37:0/3349743590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be80b0a30 0x7f9be80cf890 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80097c0 tx=0x7f9bd801c470 comp rx=0 tx=0).stop 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9bee519640 1 -- 172.21.15.37:0/3349743590 shutdown_connections 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9bee519640 1 --2- 172.21.15.37:0/3349743590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be809a180 0x7f9be815d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9bee519640 1 --2- 172.21.15.37:0/3349743590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be80b0a30 0x7f9be80cf890 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9bee519640 1 -- 172.21.15.37:0/3349743590 >> 172.21.15.37:0/3349743590 conn(0x7f9be80b30b0 msgr2=0x7f9be80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9bee519640 1 -- 172.21.15.37:0/3349743590 shutdown_connections 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.198+0000 7f9bee519640 1 -- 172.21.15.37:0/3349743590 wait complete. 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9bee519640 1 Processor -- start 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9bee519640 1 -- start start 2024-04-25T08:56:48.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9bee519640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be806f4a0 0x7f9be80b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9bee519640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be809a180 0x7f9be806d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9bee519640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be80b0a30 0x7f9be80c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9bee519640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80cbd90 con 0x7f9be806f4a0 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9bee519640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80cbc10 con 0x7f9be80b0a30 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9bee519640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be80cbf10 con 0x7f9be809a180 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be806f4a0 0x7f9be80b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be806f4a0 0x7f9be80b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60846/0 (socket says 172.21.15.37:60846) 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be7fff640 1 -- 172.21.15.37:0/2799159107 learned_addr learned my addr 172.21.15.37:0/2799159107 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be77fe640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be809a180 0x7f9be806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9becab6640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be80b0a30 0x7f9be80c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be7fff640 1 -- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be809a180 msgr2=0x7f9be806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be7fff640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be809a180 0x7f9be806d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be7fff640 1 -- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be80b0a30 msgr2=0x7f9be80c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be7fff640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be80b0a30 0x7f9be80c5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be7fff640 1 -- 172.21.15.37:0/2799159107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80c9a80 con 0x7f9be806f4a0 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be77fe640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be809a180 0x7f9be806d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be7fff640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be806f4a0 0x7f9be80b4100 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9bd801ab20 tx=0x7f9bd801acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be4ff9640 1 -- 172.21.15.37:0/2799159107 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80bf610 con 0x7f9be806f4a0 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be4ff9640 1 -- 172.21.15.37:0/2799159107 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd80bf7b0 con 0x7f9be806f4a0 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9bee519640 1 -- 172.21.15.37:0/2799159107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be80c9d60 con 0x7f9be806f4a0 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9be4ff9640 1 -- 172.21.15.37:0/2799159107 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80bfba0 con 0x7f9be806f4a0 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9bee519640 1 -- 172.21.15.37:0/2799159107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be80ca320 con 0x7f9be806f4a0 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.202+0000 7f9bee519640 1 -- 172.21.15.37:0/2799159107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be804efa0 con 0x7f9be806f4a0 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.206+0000 7f9be4ff9640 1 -- 172.21.15.37:0/2799159107 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bd80bd020 con 0x7f9be806f4a0 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.206+0000 7f9be4ff9640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bd4066f60 0x7f9bd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.206+0000 7f9be4ff9640 1 -- 172.21.15.37:0/2799159107 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9bd811f6f0 con 0x7f9be806f4a0 2024-04-25T08:56:48.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.206+0000 7f9be77fe640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bd4066f60 0x7f9bd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.206+0000 7f9be77fe640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bd4066f60 0x7f9bd4069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9be80b71b0 tx=0x7f9bdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.210+0000 7f9be4ff9640 1 -- 172.21.15.37:0/2799159107 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bd8016090 con 0x7f9be806f4a0 2024-04-25T08:56:48.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.426+0000 7f9bee519640 1 -- 172.21.15.37:0/2799159107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9be80b3ee0 con 0x7f9be806f4a0 2024-04-25T08:56:48.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.426+0000 7f9be4ff9640 1 -- 172.21.15.37:0/2799159107 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9bd80f0a70 con 0x7f9be806f4a0 2024-04-25T08:56:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:48.417 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:56:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpm2pLfABAAJbn2dGKdl8SHS8pApIWC1Q==", 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmqB+jDxAAmRnBzN3vqO8HIf/cd3QoJQ==", 2024-04-25T08:56:48.418 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmiXRqCRAACtY8cFxwqOT4WNzj9WmeKA==", 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmX3ARGBAAW/ZUr85NsKtd8HzxE+Yefw==", 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-04-25T08:56:48.420 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCSGSpmwhMiJhAAbmQz36LujHeYXaKIOdLLSA==", 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCVGSpm/OLPKxAA9ZmrrWxvt4QDj2lHATnzSg==", 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCYGSpmq7tEMxAA/wjwgkrtnTkrMoC6G+YIlQ==", 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCbGSpmReB1ORAAkXje/r11I3ArZHErIvijgw==", 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:48.422 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQChGSpm4XfLORAAGj+R1Nk/EGmvEdpt8kQJlg==", 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-04-25T08:56:48.423 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQClGSpmd5f1ARAAKF3XG//v3Uqx4bckcD4JMw==", 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCoGSpmi9OECBAAJBskMWGCNlAaej9xUc6g8w==", 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCrGSpmunQiDhAA3UZo2vzT44HEYUBUia75NA==", 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA==", 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.425 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmu8UoJhAAyutD5hzKLZmr57WuAWScuQ==", 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmGVVdIxAAY7/VOuqF5OoomqqVxE1Ifw==", 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmjFNDKxAAmo6DB9W17TZaGlvkdqkSuQ==", 2024-04-25T08:56:48.427 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm+mGgJRAA9oTRTlZpGBdCZ0/bj580Bg==", 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.428 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpmLChTORAAo6fs/9PDJ5YqqcbfHfPO9Q==", 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm3NLHNhAAeqSA658L6P9OcrSO5JVzLg==", 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:56:48.429 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T08:56:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:56:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:48.430 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T08:56:48.430 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:56:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 -- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bd4066f60 msgr2=0x7f9bd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bd4066f60 0x7f9bd4069420 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9be80b71b0 tx=0x7f9bdc009040 comp rx=0 tx=0).stop 2024-04-25T08:56:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 -- 172.21.15.37:0/2799159107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be806f4a0 msgr2=0x7f9be80b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be806f4a0 0x7f9be80b4100 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f9bd801ab20 tx=0x7f9bd801acb0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 -- 172.21.15.37:0/2799159107 shutdown_connections 2024-04-25T08:56:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bd4066f60 0x7f9bd4069420 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be80b0a30 0x7f9be80c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be809a180 0x7f9be806d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 --2- 172.21.15.37:0/2799159107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be806f4a0 0x7f9be80b4100 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 -- 172.21.15.37:0/2799159107 >> 172.21.15.37:0/2799159107 conn(0x7f9be80b30b0 msgr2=0x7f9be80b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 -- 172.21.15.37:0/2799159107 shutdown_connections 2024-04-25T08:56:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.430+0000 7f9bee519640 1 -- 172.21.15.37:0/2799159107 wait complete. 2024-04-25T08:56:48.437 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.570+0000 7fec4c65c640 1 Processor -- start 2024-04-25T08:56:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.570+0000 7fec4c65c640 1 -- start start 2024-04-25T08:56:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4c65c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44071fd0 0x7fec4415a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4c65c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4415ae30 con 0x7fec44159ea0 2024-04-25T08:56:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4c65c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4415b110 con 0x7fec4415a280 2024-04-25T08:56:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4c65c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec440b2990 con 0x7fec44071fd0 2024-04-25T08:56:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4a3f8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fec44159ea0 0x7fec440b2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56532/0 (socket says 172.21.15.37:56532) 2024-04-25T08:56:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4a3f8640 1 -- 172.21.15.37:0/3839213173 learned_addr learned my addr 172.21.15.37:0/3839213173 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4abf9640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44071fd0 0x7fec4415a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599894145 0 0) 0x7fec4415ae30 con 0x7fec44159ea0 2024-04-25T08:56:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec30003530 con 0x7fec44159ea0 2024-04-25T08:56:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243673631 0 0) 0x7fec4415b110 con 0x7fec4415a280 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec4415ae30 con 0x7fec4415a280 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1228708923 0 0) 0x7fec30003530 con 0x7fec44159ea0 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44071fd0 msgr2=0x7fec4415a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44071fd0 0x7fec4415a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 >> v1:172.21.15.203:6789/0 conn(0x7fec4415a280 legacy=0x7fec440718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440b2620 con 0x7fec44159ea0 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fec40001600 con 0x7fec44159ea0 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 >> v1:172.21.15.37:6789/0 conn(0x7fec44159ea0 legacy=0x7fec440b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec30003bf0 0x7fec30004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44071fd0 0x7fec30004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec30004d50 0x7fec300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec440b2990 con 0x7fec30004d50 2024-04-25T08:56:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3000bb90 con 0x7fec30003bf0 2024-04-25T08:56:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3000bd90 con 0x7fec44071fd0 2024-04-25T08:56:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4abf9640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec30004d50 0x7fec300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec49bf7640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44071fd0 0x7fec30004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4a3f8640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec30003bf0 0x7fec30004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4abf9640 1 -- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44071fd0 msgr2=0x7fec30004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4abf9640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44071fd0 0x7fec30004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4abf9640 1 -- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec30003bf0 msgr2=0x7fec30004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4abf9640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec30003bf0 0x7fec30004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec4abf9640 1 -- 172.21.15.37:0/3839213173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec30003530 con 0x7fec30004d50 2024-04-25T08:56:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.574+0000 7fec49bf7640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec44071fd0 0x7fec30004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:48.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.578+0000 7fec4abf9640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec30004d50 0x7fec300051d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00f270 tx=0x7fec3c01dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e09fe16ebee464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.578+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00ad20 con 0x7fec30004d50 2024-04-25T08:56:48.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.578+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c0a6b80 con 0x7fec30004d50 2024-04-25T08:56:48.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.578+0000 7fec493f6640 1 -- 172.21.15.37:0/3839213173 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c008440 con 0x7fec30004d50 2024-04-25T08:56:48.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.578+0000 7fec4c65c640 1 -- 172.21.15.37:0/3839213173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec30004d50 msgr2=0x7fec300051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.578+0000 7fec4c65c640 1 --2- 172.21.15.37:0/3839213173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec30004d50 0x7fec300051d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00f270 tx=0x7fec3c01dd80 comp rx=0 tx=0).stop 2024-04-25T08:56:48.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.578+0000 7fec4abf9640 1 -- 172.21.15.37:0/3839213173 reap_dead start 2024-04-25T08:56:48.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.578+0000 7fec4c65c640 1 -- 172.21.15.37:0/3839213173 shutdown_connections 2024-04-25T08:56:48.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.578+0000 7fec4c65c640 1 -- 172.21.15.37:0/3839213173 >> 172.21.15.37:0/3839213173 conn(0x7fec440b11a0 msgr2=0x7fec440b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.578+0000 7fec4c65c640 1 -- 172.21.15.37:0/3839213173 shutdown_connections 2024-04-25T08:56:48.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.578+0000 7fec4c65c640 1 -- 172.21.15.37:0/3839213173 wait complete. 2024-04-25T08:56:48.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4c65c640 1 Processor -- start 2024-04-25T08:56:48.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4c65c640 1 -- start start 2024-04-25T08:56:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4c65c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44142630 0x7fec440a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4c65c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec440af940 0x7fec440afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4c65c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec44144ed0 0x7fec44145350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4c65c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4415a700 con 0x7fec44144ed0 2024-04-25T08:56:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4c65c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec440b3530 con 0x7fec44142630 2024-04-25T08:56:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4c65c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4415a880 con 0x7fec440af940 2024-04-25T08:56:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4abf9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec44144ed0 0x7fec44145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4abf9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec44144ed0 0x7fec44145350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60868/0 (socket says 172.21.15.37:60868) 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4abf9640 1 -- 172.21.15.37:0/2632817123 learned_addr learned my addr 172.21.15.37:0/2632817123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec49bf7640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec440af940 0x7fec440afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4a3f8640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44142630 0x7fec440a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4abf9640 1 -- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec440af940 msgr2=0x7fec440afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4abf9640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec440af940 0x7fec440afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4abf9640 1 -- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44142630 msgr2=0x7fec440a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4abf9640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44142630 0x7fec440a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4abf9640 1 -- 172.21.15.37:0/2632817123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec44150d00 con 0x7fec44144ed0 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4a3f8640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44142630 0x7fec440a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4abf9640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec44144ed0 0x7fec44145350 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fec3c01f460 tx=0x7fec3c00a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec3affd640 1 -- 172.21.15.37:0/2632817123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34001520 con 0x7fec44144ed0 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec3affd640 1 -- 172.21.15.37:0/2632817123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c0039a0 con 0x7fec44144ed0 2024-04-25T08:56:48.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec3affd640 1 -- 172.21.15.37:0/2632817123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00ad20 con 0x7fec44144ed0 2024-04-25T08:56:48.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4c65c640 1 -- 172.21.15.37:0/2632817123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec44150fe0 con 0x7fec44144ed0 2024-04-25T08:56:48.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4c65c640 1 -- 172.21.15.37:0/2632817123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec441515a0 con 0x7fec44144ed0 2024-04-25T08:56:48.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.582+0000 7fec4c65c640 1 -- 172.21.15.37:0/2632817123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4404efa0 con 0x7fec44144ed0 2024-04-25T08:56:48.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.586+0000 7fec3affd640 1 -- 172.21.15.37:0/2632817123 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec3c0bd020 con 0x7fec44144ed0 2024-04-25T08:56:48.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.586+0000 7fec3affd640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec28066f90 0x7fec28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.586+0000 7fec3affd640 1 -- 172.21.15.37:0/2632817123 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fec3c11f210 con 0x7fec44144ed0 2024-04-25T08:56:48.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.586+0000 7fec4a3f8640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec28066f90 0x7fec28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.586+0000 7fec4a3f8640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec28066f90 0x7fec28069450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec400050c0 tx=0x7fec40001ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.590+0000 7fec3affd640 1 -- 172.21.15.37:0/2632817123 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec3c017a90 con 0x7fec44144ed0 2024-04-25T08:56:48.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.754+0000 7fec4c65c640 1 -- 172.21.15.37:0/2632817123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec4406e6b0 con 0x7fec44144ed0 2024-04-25T08:56:48.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.754+0000 7fec3affd640 1 -- 172.21.15.37:0/2632817123 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+13129 (secure 0 0 0) 0x7fec3c0f03d0 con 0x7fec44144ed0 2024-04-25T08:56:48.745 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:48.745 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":78,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:56:48.062822+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 -- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec28066f90 msgr2=0x7fec28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec28066f90 0x7fec28069450 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fec400050c0 tx=0x7fec40001ce0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 -- 172.21.15.37:0/2632817123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec44144ed0 msgr2=0x7fec44145350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec44144ed0 0x7fec44145350 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fec3c01f460 tx=0x7fec3c00a860 comp rx=0 tx=0).stop 2024-04-25T08:56:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 -- 172.21.15.37:0/2632817123 shutdown_connections 2024-04-25T08:56:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec28066f90 0x7fec28069450 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec44144ed0 0x7fec44145350 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec440af940 0x7fec440afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 --2- 172.21.15.37:0/2632817123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec44142630 0x7fec440a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 -- 172.21.15.37:0/2632817123 >> 172.21.15.37:0/2632817123 conn(0x7fec440b11a0 msgr2=0x7fec4406d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 -- 172.21.15.37:0/2632817123 shutdown_connections 2024-04-25T08:56:48.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.758+0000 7fec4c65c640 1 -- 172.21.15.37:0/2632817123 wait complete. 2024-04-25T08:56:48.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:56:48.765 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:56:48.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b04275640 1 Processor -- start 2024-04-25T08:56:48.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b04275640 1 -- start start 2024-04-25T08:56:48.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b04275640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc0cf590 0x7f2afc15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b04275640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2afc0cf970 0x7f2afc0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b04275640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2afc0c9ef0 0x7f2afc0b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b04275640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc0ca780 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b04275640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc0cd440 con 0x7f2afc0cf970 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b04275640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0cdce0 con 0x7f2afc0cf590 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b02812640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2afc0c9ef0 0x7f2afc0b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b02812640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2afc0c9ef0 0x7f2afc0b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60884/0 (socket says 172.21.15.37:60884) 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b02812640 1 -- 172.21.15.37:0/3418907195 learned_addr learned my addr 172.21.15.37:0/3418907195 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b02011640 1 --2- 172.21.15.37:0/3418907195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc0cf590 0x7f2afc15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b01810640 1 --2- 172.21.15.37:0/3418907195 >> v2:172.21.15.203:3300/0 conn(0x7f2afc0cf970 0x7f2afc0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b02812640 1 -- 172.21.15.37:0/3418907195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc0cf590 msgr2=0x7f2afc15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b02812640 1 --2- 172.21.15.37:0/3418907195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc0cf590 0x7f2afc15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b02812640 1 -- 172.21.15.37:0/3418907195 >> v2:172.21.15.203:3300/0 conn(0x7f2afc0cf970 msgr2=0x7f2afc0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b02812640 1 --2- 172.21.15.37:0/3418907195 >> v2:172.21.15.203:3300/0 conn(0x7f2afc0cf970 0x7f2afc0c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b02812640 1 -- 172.21.15.37:0/3418907195 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc0ca4c0 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b02011640 1 --2- 172.21.15.37:0/3418907195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc0cf590 0x7f2afc15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:48.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.898+0000 7f2b01810640 1 --2- 172.21.15.37:0/3418907195 >> v2:172.21.15.203:3300/0 conn(0x7f2afc0cf970 0x7f2afc0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:48.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b02812640 1 --2- 172.21.15.37:0/3418907195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc0c9ef0 0x7f2afc0b21b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0d0110 tx=0x7f2af401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68929215bb20ee39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b0100f640 1 -- 172.21.15.37:0/3418907195 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af40a5580 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b0100f640 1 -- 172.21.15.37:0/3418907195 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af40a5720 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b0100f640 1 -- 172.21.15.37:0/3418907195 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af40a5af0 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 -- 172.21.15.37:0/3418907195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc0c9ef0 msgr2=0x7f2afc0b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 --2- 172.21.15.37:0/3418907195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc0c9ef0 0x7f2afc0b21b0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0d0110 tx=0x7f2af401c700 comp rx=0 tx=0).stop 2024-04-25T08:56:48.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 -- 172.21.15.37:0/3418907195 shutdown_connections 2024-04-25T08:56:48.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 --2- 172.21.15.37:0/3418907195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc0c9ef0 0x7f2afc0b21b0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 --2- 172.21.15.37:0/3418907195 >> v2:172.21.15.203:3300/0 conn(0x7f2afc0cf970 0x7f2afc0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 --2- 172.21.15.37:0/3418907195 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc0cf590 0x7f2afc15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 -- 172.21.15.37:0/3418907195 >> 172.21.15.37:0/3418907195 conn(0x7f2afc06ae60 msgr2=0x7f2afc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 -- 172.21.15.37:0/3418907195 shutdown_connections 2024-04-25T08:56:48.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 -- 172.21.15.37:0/3418907195 wait complete. 2024-04-25T08:56:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 Processor -- start 2024-04-25T08:56:48.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 -- start start 2024-04-25T08:56:48.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc0c9ef0 0x7f2afc164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc0cf590 0x7f2afc156cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2afc0cf970 0x7f2afc157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0ac2f0 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0ac170 con 0x7f2afc0cf970 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b04275640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0ce150 con 0x7f2afc0cf590 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b02011640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc0c9ef0 0x7f2afc164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b02011640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc0c9ef0 0x7f2afc164eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60892/0 (socket says 172.21.15.37:60892) 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b02011640 1 -- 172.21.15.37:0/2981349229 learned_addr learned my addr 172.21.15.37:0/2981349229 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b02812640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2afc0cf970 0x7f2afc157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.902+0000 7f2b01810640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc0cf590 0x7f2afc156cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2b02011640 1 -- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc0cf590 msgr2=0x7f2afc156cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2b02011640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc0cf590 0x7f2afc156cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2b02011640 1 -- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2afc0cf970 msgr2=0x7f2afc157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2b02011640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2afc0cf970 0x7f2afc157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2b02011640 1 -- 172.21.15.37:0/2981349229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc1553b0 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2b02011640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc0c9ef0 0x7f2afc164eb0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c7c0 tx=0x7f2af800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2af2ffd640 1 -- 172.21.15.37:0/2981349229 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8015070 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2af2ffd640 1 -- 172.21.15.37:0/2981349229 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af8007a30 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2b04275640 1 -- 172.21.15.37:0/2981349229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc155690 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2af2ffd640 1 -- 172.21.15.37:0/2981349229 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8007e00 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2b04275640 1 -- 172.21.15.37:0/2981349229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc164990 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2b04275640 1 -- 172.21.15.37:0/2981349229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc04efa0 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2af2ffd640 1 -- 172.21.15.37:0/2981349229 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2af8018070 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2af2ffd640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2ae0066f60 0x7f2ae0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2af2ffd640 1 -- 172.21.15.37:0/2981349229 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2af8084db0 con 0x7f2afc0c9ef0 2024-04-25T08:56:48.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2b01810640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2ae0066f60 0x7f2ae0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.906+0000 7f2b01810640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2ae0066f60 0x7f2ae0069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2afc06b500 tx=0x7f2aec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:48.910+0000 7f2af2ffd640 1 -- 172.21.15.37:0/2981349229 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2af8089070 con 0x7f2afc0c9ef0 2024-04-25T08:56:49.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.070+0000 7f2b04275640 1 -- 172.21.15.37:0/2981349229 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2afc0c9720 con 0x7f2afc0c9ef0 2024-04-25T08:56:49.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2af2ffd640 1 -- 172.21.15.37:0/2981349229 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2af8055890 con 0x7f2afc0c9ef0 2024-04-25T08:56:49.063 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 -- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2ae0066f60 msgr2=0x7f2ae0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2ae0066f60 0x7f2ae0069420 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2afc06b500 tx=0x7f2aec008040 comp rx=0 tx=0).stop 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 -- 172.21.15.37:0/2981349229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc0c9ef0 msgr2=0x7f2afc164eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc0c9ef0 0x7f2afc164eb0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c7c0 tx=0x7f2af800cc80 comp rx=0 tx=0).stop 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 -- 172.21.15.37:0/2981349229 shutdown_connections 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2ae0066f60 0x7f2ae0069420 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2afc0cf970 0x7f2afc157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2afc0cf590 0x7f2afc156cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 --2- 172.21.15.37:0/2981349229 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc0c9ef0 0x7f2afc164eb0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 -- 172.21.15.37:0/2981349229 >> 172.21.15.37:0/2981349229 conn(0x7f2afc06ae60 msgr2=0x7f2afc0ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 -- 172.21.15.37:0/2981349229 shutdown_connections 2024-04-25T08:56:49.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.074+0000 7f2b04275640 1 -- 172.21.15.37:0/2981349229 wait complete. 2024-04-25T08:56:49.084 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:56:49.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.218+0000 7f195989a640 1 Processor -- start 2024-04-25T08:56:49.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.218+0000 7f195989a640 1 -- start start 2024-04-25T08:56:49.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f195989a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f19540b24a0 0x7f19540ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f195989a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19540ae860 con 0x7f1954071740 2024-04-25T08:56:49.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f195989a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195415e210 con 0x7f195415da10 2024-04-25T08:56:49.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f195989a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195415e4f0 con 0x7f19540b24a0 2024-04-25T08:56:49.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f19527fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1954071740 0x7f19540acbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56534/0 (socket says 172.21.15.37:56534) 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f19527fc640 1 -- 172.21.15.37:0/697417594 learned_addr learned my addr 172.21.15.37:0/697417594 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f19537fe640 1 --2- 172.21.15.37:0/697417594 >> v2:172.21.15.203:3300/0 conn(0x7f19540b24a0 0x7f19540ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243602180 0 0) 0x7f19540ae860 con 0x7f1954071740 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f193c003530 con 0x7f1954071740 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3745107920 0 0) 0x7f193c003530 con 0x7f1954071740 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 >> v2:172.21.15.203:3300/0 conn(0x7f19540b24a0 msgr2=0x7f19540ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 --2- 172.21.15.37:0/697417594 >> v2:172.21.15.203:3300/0 conn(0x7f19540b24a0 0x7f19540ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 >> v1:172.21.15.203:6789/0 conn(0x7f195415da10 legacy=0x7f1954071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19540ae4c0 con 0x7f1954071740 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1948001680 con 0x7f1954071740 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 >> v1:172.21.15.37:6789/0 conn(0x7f1954071740 legacy=0x7f19540acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f193c003a30 0x7f193c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f193c004810 0x7f193c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 0x7f193c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195415e4f0 con 0x7f193c0052d0 2024-04-25T08:56:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c00c210 con 0x7f193c003a30 2024-04-25T08:56:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c00c410 con 0x7f193c004810 2024-04-25T08:56:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f19537fe640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 0x7f193c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f19527fc640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f193c004810 0x7f193c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1952ffd640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f193c003a30 0x7f193c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f19537fe640 1 -- 172.21.15.37:0/697417594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f193c004810 msgr2=0x7f193c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f19537fe640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f193c004810 0x7f193c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f19537fe640 1 -- 172.21.15.37:0/697417594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f193c003a30 msgr2=0x7f193c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f19537fe640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f193c003a30 0x7f193c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f19537fe640 1 -- 172.21.15.37:0/697417594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c003530 con 0x7f193c0052d0 2024-04-25T08:56:49.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.222+0000 7f1952ffd640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f193c003a30 0x7f193c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f19537fe640 1 -- 172.21.15.37:0/697417594 reap_dead start 2024-04-25T08:56:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f19537fe640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 0x7f193c00b9d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f194c009fa0 tx=0x7f194c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1313a36d904e9b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c0b2070 con 0x7f193c0052d0 2024-04-25T08:56:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c0a5b30 con 0x7f193c0052d0 2024-04-25T08:56:49.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f1951ffb640 1 -- 172.21.15.37:0/697417594 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c009bf0 con 0x7f193c0052d0 2024-04-25T08:56:49.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 -- 172.21.15.37:0/697417594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 msgr2=0x7f193c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 0x7f193c00b9d0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f194c009fa0 tx=0x7f194c01cc80 comp rx=0 tx=0).stop 2024-04-25T08:56:49.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 -- 172.21.15.37:0/697417594 shutdown_connections 2024-04-25T08:56:49.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 --2- 172.21.15.37:0/697417594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 0x7f193c00b9d0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 -- 172.21.15.37:0/697417594 >> 172.21.15.37:0/697417594 conn(0x7f19540b11a0 msgr2=0x7f19540b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:49.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 -- 172.21.15.37:0/697417594 shutdown_connections 2024-04-25T08:56:49.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 -- 172.21.15.37:0/697417594 wait complete. 2024-04-25T08:56:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 Processor -- start 2024-04-25T08:56:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 -- start start 2024-04-25T08:56:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 0x7f19541514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954154a40 0x7f19540afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f195414e590 0x7f195414ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19540713c0 con 0x7f193c0052d0 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1954071240 con 0x7f1954154a40 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f195989a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1954071540 con 0x7f195414e590 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f1952ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 0x7f19541514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f1952ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 0x7f19541514c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60916/0 (socket says 172.21.15.37:60916) 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f1952ffd640 1 -- 172.21.15.37:0/4286346622 learned_addr learned my addr 172.21.15.37:0/4286346622 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f19527fc640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954154a40 0x7f19540afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f19537fe640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f195414e590 0x7f195414ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f1952ffd640 1 -- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f195414e590 msgr2=0x7f195414ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f1952ffd640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f195414e590 0x7f195414ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f1952ffd640 1 -- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954154a40 msgr2=0x7f19540afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f1952ffd640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954154a40 0x7f19540afd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.226+0000 7f1952ffd640 1 -- 172.21.15.37:0/4286346622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1954151cf0 con 0x7f193c0052d0 2024-04-25T08:56:49.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f1952ffd640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 0x7f19541514c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f194400d570 tx=0x7f194400da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f193bfff640 1 -- 172.21.15.37:0/4286346622 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944005480 con 0x7f193c0052d0 2024-04-25T08:56:49.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f193bfff640 1 -- 172.21.15.37:0/4286346622 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1944017070 con 0x7f193c0052d0 2024-04-25T08:56:49.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f195989a640 1 -- 172.21.15.37:0/4286346622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1954151fd0 con 0x7f193c0052d0 2024-04-25T08:56:49.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f193bfff640 1 -- 172.21.15.37:0/4286346622 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1944003840 con 0x7f193c0052d0 2024-04-25T08:56:49.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f195989a640 1 -- 172.21.15.37:0/4286346622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1954159ee0 con 0x7f193c0052d0 2024-04-25T08:56:49.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f195989a640 1 -- 172.21.15.37:0/4286346622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19541599f0 con 0x7f193c0052d0 2024-04-25T08:56:49.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f193bfff640 1 -- 172.21.15.37:0/4286346622 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19440039e0 con 0x7f193c0052d0 2024-04-25T08:56:49.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f193bfff640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1940066f90 0x7f1940069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f193bfff640 1 -- 172.21.15.37:0/4286346622 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1944084cb0 con 0x7f193c0052d0 2024-04-25T08:56:49.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f19527fc640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1940066f90 0x7f1940069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.230+0000 7f19527fc640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1940066f90 0x7f1940069450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1948001130 tx=0x7f1948009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.234+0000 7f193bfff640 1 -- 172.21.15.37:0/4286346622 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1944019310 con 0x7f193c0052d0 2024-04-25T08:56:49.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.390+0000 7f195989a640 1 -- 172.21.15.37:0/4286346622 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f195404efa0 con 0x7f193c0052d0 2024-04-25T08:56:49.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.390+0000 7f193bfff640 1 -- 172.21.15.37:0/4286346622 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1944055e00 con 0x7f193c0052d0 2024-04-25T08:56:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:49.382 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T08:56:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:56:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T08:56:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T08:56:49.382 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:56:49.382 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T08:56:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 -- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1940066f90 msgr2=0x7f1940069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1940066f90 0x7f1940069450 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1948001130 tx=0x7f1948009040 comp rx=0 tx=0).stop 2024-04-25T08:56:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 -- 172.21.15.37:0/4286346622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 msgr2=0x7f19541514c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 0x7f19541514c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f194400d570 tx=0x7f194400da30 comp rx=0 tx=0).stop 2024-04-25T08:56:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 -- 172.21.15.37:0/4286346622 shutdown_connections 2024-04-25T08:56:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1940066f90 0x7f1940069450 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f195414e590 0x7f195414ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1954154a40 0x7f19540afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 --2- 172.21.15.37:0/4286346622 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f193c0052d0 0x7f19541514c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 -- 172.21.15.37:0/4286346622 >> 172.21.15.37:0/4286346622 conn(0x7f19540b11a0 msgr2=0x7f195406e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:49.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 -- 172.21.15.37:0/4286346622 shutdown_connections 2024-04-25T08:56:49.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.394+0000 7f195989a640 1 -- 172.21.15.37:0/4286346622 wait complete. 2024-04-25T08:56:49.399 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:56:49.400 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:56:49.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752cf79640 1 Processor -- start 2024-04-25T08:56:49.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752cf79640 1 -- start start 2024-04-25T08:56:49.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752cf79640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7528161050 0x7f75280d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752cf79640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f752808b4c0 0x7f75280d2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752cf79640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75280b1290 con 0x7f75280ce390 2024-04-25T08:56:49.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752cf79640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7528161430 con 0x7f7528161050 2024-04-25T08:56:49.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752cf79640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75280ce870 con 0x7f752808b4c0 2024-04-25T08:56:49.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7528161050 0x7f75280d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7528161050 0x7f75280d1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38960/0 (socket says 172.21.15.37:38960) 2024-04-25T08:56:49.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752659c640 1 -- 172.21.15.37:0/1046971123 learned_addr learned my addr 172.21.15.37:0/1046971123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:49.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f7525d9b640 1 --2- 172.21.15.37:0/1046971123 >> v2:172.21.15.203:3300/0 conn(0x7f752808b4c0 0x7f75280d2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752659c640 1 -- 172.21.15.37:0/1046971123 >> v2:172.21.15.203:3300/0 conn(0x7f752808b4c0 msgr2=0x7f75280d2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:49.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752659c640 1 --2- 172.21.15.37:0/1046971123 >> v2:172.21.15.203:3300/0 conn(0x7f752808b4c0 0x7f75280d2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752659c640 1 -- 172.21.15.37:0/1046971123 >> v1:172.21.15.203:6789/0 conn(0x7f75280ce390 legacy=0x7f75280d0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:49.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.526+0000 7f752659c640 1 -- 172.21.15.37:0/1046971123 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75280ceb50 con 0x7f7528161050 2024-04-25T08:56:49.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752659c640 1 --2- 172.21.15.37:0/1046971123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7528161050 0x7f75280d1ae0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f75281615b0 tx=0x7f751001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab18b7c61b322ded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752559a640 1 -- 172.21.15.37:0/1046971123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75100b1070 con 0x7f7528161050 2024-04-25T08:56:49.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752559a640 1 -- 172.21.15.37:0/1046971123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75100ac630 con 0x7f7528161050 2024-04-25T08:56:49.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752559a640 1 -- 172.21.15.37:0/1046971123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75100a8050 con 0x7f7528161050 2024-04-25T08:56:49.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752cf79640 1 -- 172.21.15.37:0/1046971123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7528161050 msgr2=0x7f75280d1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752cf79640 1 --2- 172.21.15.37:0/1046971123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7528161050 0x7f75280d1ae0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f75281615b0 tx=0x7f751001c4e0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752cf79640 1 -- 172.21.15.37:0/1046971123 shutdown_connections 2024-04-25T08:56:49.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752cf79640 1 --2- 172.21.15.37:0/1046971123 >> v2:172.21.15.203:3300/0 conn(0x7f752808b4c0 0x7f75280d2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752cf79640 1 --2- 172.21.15.37:0/1046971123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7528161050 0x7f75280d1ae0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752cf79640 1 -- 172.21.15.37:0/1046971123 >> 172.21.15.37:0/1046971123 conn(0x7f752808b000 msgr2=0x7f75280d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:49.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752cf79640 1 -- 172.21.15.37:0/1046971123 shutdown_connections 2024-04-25T08:56:49.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752cf79640 1 -- 172.21.15.37:0/1046971123 wait complete. 2024-04-25T08:56:49.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752cf79640 1 Processor -- start 2024-04-25T08:56:49.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.530+0000 7f752cf79640 1 -- start start 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f752cf79640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f752808b4c0 0x7f75280b8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f752cf79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75280ce390 0x7f752806c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f752cf79640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7528161050 0x7f752806cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f752cf79640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f752816a6a0 con 0x7f75280ce390 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f752cf79640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75280cf910 con 0x7f752808b4c0 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f752cf79640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f752816a820 con 0x7f7528161050 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f7525d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75280ce390 0x7f752806c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f7525d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75280ce390 0x7f752806c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38968/0 (socket says 172.21.15.37:38968) 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f7525d9b640 1 -- 172.21.15.37:0/3483518980 learned_addr learned my addr 172.21.15.37:0/3483518980 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f7525d9b640 1 -- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7528161050 msgr2=0x7f752806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f7525d9b640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7528161050 0x7f752806cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f7525d9b640 1 -- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f752808b4c0 msgr2=0x7f75280b8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f7525d9b640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f752808b4c0 0x7f75280b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f7525d9b640 1 -- 172.21.15.37:0/3483518980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75280c7480 con 0x7f75280ce390 2024-04-25T08:56:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f7525d9b640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75280ce390 0x7f752806c690 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7518007ec0 tx=0x7f751800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f751effd640 1 -- 172.21.15.37:0/3483518980 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518014620 con 0x7f75280ce390 2024-04-25T08:56:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f751effd640 1 -- 172.21.15.37:0/3483518980 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7518019070 con 0x7f75280ce390 2024-04-25T08:56:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f752cf79640 1 -- 172.21.15.37:0/3483518980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75280b8b30 con 0x7f75280ce390 2024-04-25T08:56:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f751effd640 1 -- 172.21.15.37:0/3483518980 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518014a20 con 0x7f75280ce390 2024-04-25T08:56:49.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f752cf79640 1 -- 172.21.15.37:0/3483518980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75280c7b40 con 0x7f75280ce390 2024-04-25T08:56:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f752cf79640 1 -- 172.21.15.37:0/3483518980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7528067c30 con 0x7f75280ce390 2024-04-25T08:56:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f751effd640 1 -- 172.21.15.37:0/3483518980 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7518014bc0 con 0x7f75280ce390 2024-04-25T08:56:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f751effd640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7508066f60 0x7f7508069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f751effd640 1 -- 172.21.15.37:0/3483518980 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7518085760 con 0x7f75280ce390 2024-04-25T08:56:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.534+0000 7f752659c640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7508066f60 0x7f7508069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.538+0000 7f752659c640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7508066f60 0x7f7508069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f75100ae000 tx=0x7f7510004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.538+0000 7f751effd640 1 -- 172.21.15.37:0/3483518980 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75180569c0 con 0x7f75280ce390 2024-04-25T08:56:49.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:49.694+0000 7f752cf79640 1 -- 172.21.15.37:0/3483518980 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f752815b910 con 0x7f75280ce390 2024-04-25T08:56:50.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.058+0000 7f751effd640 1 -- 172.21.15.37:0/3483518980 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 138+0+0 (secure 0 0 0) 0x7f75180562c0 con 0x7f75280ce390 2024-04-25T08:56:50.048 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T08:56:50.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 -- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7508066f60 msgr2=0x7f7508069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7508066f60 0x7f7508069420 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f75100ae000 tx=0x7f7510004740 comp rx=0 tx=0).stop 2024-04-25T08:56:50.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 -- 172.21.15.37:0/3483518980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75280ce390 msgr2=0x7f752806c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75280ce390 0x7f752806c690 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7518007ec0 tx=0x7f751800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 -- 172.21.15.37:0/3483518980 shutdown_connections 2024-04-25T08:56:50.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7508066f60 0x7f7508069420 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7528161050 0x7f752806cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75280ce390 0x7f752806c690 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 --2- 172.21.15.37:0/3483518980 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f752808b4c0 0x7f75280b8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 -- 172.21.15.37:0/3483518980 >> 172.21.15.37:0/3483518980 conn(0x7f752808b000 msgr2=0x7f7528157db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:50.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 -- 172.21.15.37:0/3483518980 shutdown_connections 2024-04-25T08:56:50.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.062+0000 7f752cf79640 1 -- 172.21.15.37:0/3483518980 wait complete. 2024-04-25T08:56:50.078 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:56:50.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.218+0000 7f1f91c1f640 1 Processor -- start 2024-04-25T08:56:50.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.218+0000 7f1f91c1f640 1 -- start start 2024-04-25T08:56:50.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.218+0000 7f1f91c1f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1f8c143790 0x7f1f8c0669a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:50.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.218+0000 7f1f91c1f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c06e170 con 0x7f1f8c06bca0 2024-04-25T08:56:50.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.218+0000 7f1f91c1f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8c06e450 con 0x7f1f8c15da40 2024-04-25T08:56:50.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.218+0000 7f1f91c1f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8c143bf0 con 0x7f1f8c143790 2024-04-25T08:56:50.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.218+0000 7f1f8bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1f8c06bca0 0x7f1f8c164d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44696/0 (socket says 172.21.15.37:44696) 2024-04-25T08:56:50.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.218+0000 7f1f8bfff640 1 -- 172.21.15.37:0/2528050535 learned_addr learned my addr 172.21.15.37:0/2528050535 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:50.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.218+0000 7f1f8affd640 1 --2- 172.21.15.37:0/2528050535 >> v2:172.21.15.37:3300/0 conn(0x7f1f8c143790 0x7f1f8c0669a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f8a7fc640 1 -- 172.21.15.37:0/2528050535 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942536980 0 0) 0x7f1f8c06e170 con 0x7f1f8c06bca0 2024-04-25T08:56:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f8a7fc640 1 -- 172.21.15.37:0/2528050535 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f78005740 con 0x7f1f8c06bca0 2024-04-25T08:56:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f8affd640 1 -- 172.21.15.37:0/2528050535 >> v1:172.21.15.203:6789/0 conn(0x7f1f8c15da40 legacy=0x7f1f8c06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f8affd640 1 -- 172.21.15.37:0/2528050535 >> v1:172.21.15.37:6789/0 conn(0x7f1f8c06bca0 legacy=0x7f1f8c164d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f8affd640 1 -- 172.21.15.37:0/2528050535 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c06e730 con 0x7f1f8c143790 2024-04-25T08:56:50.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f8affd640 1 --2- 172.21.15.37:0/2528050535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f8c143790 0x7f1f8c0669a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1f740097c0 tx=0x7f1f7401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb3c2027e40ce12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:50.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f8a7fc640 1 -- 172.21.15.37:0/2528050535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f740b0070 con 0x7f1f8c143790 2024-04-25T08:56:50.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f8a7fc640 1 -- 172.21.15.37:0/2528050535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f740ab600 con 0x7f1f8c143790 2024-04-25T08:56:50.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f8a7fc640 1 -- 172.21.15.37:0/2528050535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f740ab9d0 con 0x7f1f8c143790 2024-04-25T08:56:50.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f91c1f640 1 -- 172.21.15.37:0/2528050535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f8c143790 msgr2=0x7f1f8c0669a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f91c1f640 1 --2- 172.21.15.37:0/2528050535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f8c143790 0x7f1f8c0669a0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1f740097c0 tx=0x7f1f7401c410 comp rx=0 tx=0).stop 2024-04-25T08:56:50.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f91c1f640 1 -- 172.21.15.37:0/2528050535 shutdown_connections 2024-04-25T08:56:50.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f91c1f640 1 --2- 172.21.15.37:0/2528050535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f8c143790 0x7f1f8c0669a0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f91c1f640 1 -- 172.21.15.37:0/2528050535 >> 172.21.15.37:0/2528050535 conn(0x7f1f8c071810 msgr2=0x7f1f8c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:50.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f91c1f640 1 -- 172.21.15.37:0/2528050535 shutdown_connections 2024-04-25T08:56:50.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.222+0000 7f1f91c1f640 1 -- 172.21.15.37:0/2528050535 wait complete. 2024-04-25T08:56:50.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f91c1f640 1 Processor -- start 2024-04-25T08:56:50.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f91c1f640 1 -- start start 2024-04-25T08:56:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f91c1f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f8c06bca0 0x7f1f8c155890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f91c1f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f8c15da40 0x7f1f8c13c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f91c1f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f8c13ce90 0x7f1f8c13a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f91c1f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0b3b60 con 0x7f1f8c13ce90 2024-04-25T08:56:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f91c1f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0b39e0 con 0x7f1f8c06bca0 2024-04-25T08:56:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f91c1f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c0b3ce0 con 0x7f1f8c15da40 2024-04-25T08:56:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f8bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f8c13ce90 0x7f1f8c13a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f8bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f8c13ce90 0x7f1f8c13a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38990/0 (socket says 172.21.15.37:38990) 2024-04-25T08:56:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f8bfff640 1 -- 172.21.15.37:0/1626042751 learned_addr learned my addr 172.21.15.37:0/1626042751 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:50.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f8affd640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f8c15da40 0x7f1f8c13c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f8b7fe640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f8c06bca0 0x7f1f8c155890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f8bfff640 1 -- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f8c15da40 msgr2=0x7f1f8c13c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f8bfff640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f8c15da40 0x7f1f8c13c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f8bfff640 1 -- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f8c06bca0 msgr2=0x7f1f8c155890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f8bfff640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f8c06bca0 0x7f1f8c155890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f8bfff640 1 -- 172.21.15.37:0/1626042751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c158a20 con 0x7f1f8c13ce90 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f8bfff640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f8c13ce90 0x7f1f8c13a3a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00cb40 tx=0x7f1f7c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f83fff640 1 -- 172.21.15.37:0/1626042751 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c017070 con 0x7f1f8c13ce90 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f83fff640 1 -- 172.21.15.37:0/1626042751 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7c00aca0 con 0x7f1f8c13ce90 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f91c1f640 1 -- 172.21.15.37:0/1626042751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8c13af80 con 0x7f1f8c13ce90 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f83fff640 1 -- 172.21.15.37:0/1626042751 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c012620 con 0x7f1f8c13ce90 2024-04-25T08:56:50.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f91c1f640 1 -- 172.21.15.37:0/1626042751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c13b540 con 0x7f1f8c13ce90 2024-04-25T08:56:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f91c1f640 1 -- 172.21.15.37:0/1626042751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8c04efa0 con 0x7f1f8c13ce90 2024-04-25T08:56:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.226+0000 7f1f83fff640 1 -- 172.21.15.37:0/1626042751 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f7c01a070 con 0x7f1f8c13ce90 2024-04-25T08:56:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.230+0000 7f1f83fff640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f6c066f90 0x7f1f6c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.230+0000 7f1f83fff640 1 -- 172.21.15.37:0/1626042751 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f1f7c084c60 con 0x7f1f8c13ce90 2024-04-25T08:56:50.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.230+0000 7f1f8b7fe640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f6c066f90 0x7f1f6c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:50.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.230+0000 7f1f8b7fe640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f6c066f90 0x7f1f6c069450 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1f84000f70 tx=0x7f1f84009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:50.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.234+0000 7f1f83fff640 1 -- 172.21.15.37:0/1626042751 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f8c158a20 con 0x7f1f8c13ce90 2024-04-25T08:56:50.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:50.394+0000 7f1f91c1f640 1 -- 172.21.15.37:0/1626042751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1f8c142fc0 con 0x7f1f8c13ce90 2024-04-25T08:56:51.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.070+0000 7f1f83fff640 1 -- 172.21.15.37:0/1626042751 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1f7c055700 con 0x7f1f8c13ce90 2024-04-25T08:56:51.060 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.070+0000 7f1f91c1f640 1 -- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f6c066f90 msgr2=0x7f1f6c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.070+0000 7f1f91c1f640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f6c066f90 0x7f1f6c069450 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1f84000f70 tx=0x7f1f84009040 comp rx=0 tx=0).stop 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.070+0000 7f1f91c1f640 1 -- 172.21.15.37:0/1626042751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f8c13ce90 msgr2=0x7f1f8c13a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.070+0000 7f1f91c1f640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f8c13ce90 0x7f1f8c13a3a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00cb40 tx=0x7f1f7c00a060 comp rx=0 tx=0).stop 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.074+0000 7f1f91c1f640 1 -- 172.21.15.37:0/1626042751 shutdown_connections 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.074+0000 7f1f91c1f640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1f6c066f90 0x7f1f6c069450 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.074+0000 7f1f91c1f640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f8c13ce90 0x7f1f8c13a3a0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.074+0000 7f1f91c1f640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f8c15da40 0x7f1f8c13c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.074+0000 7f1f91c1f640 1 --2- 172.21.15.37:0/1626042751 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f8c06bca0 0x7f1f8c155890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.074+0000 7f1f91c1f640 1 -- 172.21.15.37:0/1626042751 >> 172.21.15.37:0/1626042751 conn(0x7f1f8c071810 msgr2=0x7f1f8c0b0010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.074+0000 7f1f91c1f640 1 -- 172.21.15.37:0/1626042751 shutdown_connections 2024-04-25T08:56:51.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.074+0000 7f1f91c1f640 1 -- 172.21.15.37:0/1626042751 wait complete. 2024-04-25T08:56:51.094 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:56:51.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f3577ac3640 1 Processor -- start 2024-04-25T08:56:51.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f3577ac3640 1 -- start start 2024-04-25T08:56:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f3577ac3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35700cd350 0x7f357015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f3577ac3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35700ce320 con 0x7f35700cf0e0 2024-04-25T08:56:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f3577ac3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35700ce600 con 0x7f357015aac0 2024-04-25T08:56:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f3577ac3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35700ce0d0 con 0x7f35700cd350 2024-04-25T08:56:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f357505e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35700cd350 0x7f357015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f357505e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f35700cd350 0x7f357015aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39000/0 (socket says 172.21.15.37:39000) 2024-04-25T08:56:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f3576060640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f35700cf0e0 0x7f357015b3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44706/0 (socket says 172.21.15.37:44706) 2024-04-25T08:56:51.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f357505e640 1 -- 172.21.15.37:0/2022280507 learned_addr learned my addr 172.21.15.37:0/2022280507 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:51.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f357505e640 1 -- 172.21.15.37:0/2022280507 >> v1:172.21.15.203:6789/0 conn(0x7f357015aac0 legacy=0x7f35700ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:51.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f357505e640 1 -- 172.21.15.37:0/2022280507 >> v1:172.21.15.37:6789/0 conn(0x7f35700cf0e0 legacy=0x7f357015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:51.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f357505e640 1 -- 172.21.15.37:0/2022280507 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35700ce8e0 con 0x7f35700cd350 2024-04-25T08:56:51.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.234+0000 7f357485d640 1 -- 172.21.15.37:0/2022280507 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493790475 0 0) 0x7f35700ce320 con 0x7f35700cf0e0 2024-04-25T08:56:51.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f357505e640 1 --2- 172.21.15.37:0/2022280507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35700cd350 0x7f357015aea0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f3564009870 tx=0x7f356401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3422afd4d2018b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f357485d640 1 -- 172.21.15.37:0/2022280507 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35640b2070 con 0x7f35700cd350 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f357485d640 1 -- 172.21.15.37:0/2022280507 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35640a9070 con 0x7f35700cd350 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f357485d640 1 -- 172.21.15.37:0/2022280507 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35640ad880 con 0x7f35700cd350 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f3577ac3640 1 -- 172.21.15.37:0/2022280507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35700cd350 msgr2=0x7f357015aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f3577ac3640 1 --2- 172.21.15.37:0/2022280507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35700cd350 0x7f357015aea0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f3564009870 tx=0x7f356401c560 comp rx=0 tx=0).stop 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f3577ac3640 1 -- 172.21.15.37:0/2022280507 shutdown_connections 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f3577ac3640 1 --2- 172.21.15.37:0/2022280507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35700cd350 0x7f357015aea0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f3577ac3640 1 -- 172.21.15.37:0/2022280507 >> 172.21.15.37:0/2022280507 conn(0x7f3570099a70 msgr2=0x7f35700d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f3577ac3640 1 -- 172.21.15.37:0/2022280507 shutdown_connections 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f3577ac3640 1 -- 172.21.15.37:0/2022280507 wait complete. 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f3577ac3640 1 Processor -- start 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.238+0000 7f3577ac3640 1 -- start start 2024-04-25T08:56:51.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3577ac3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35700cd350 0x7f3570155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3577ac3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35700cf0e0 0x7f35700d0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3577ac3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f357015aac0 0x7f35700b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3577ac3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3570168150 con 0x7f357015aac0 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3577ac3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3570167fd0 con 0x7f35700cd350 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3577ac3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35701682d0 con 0x7f35700cf0e0 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3576060640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f357015aac0 0x7f35700b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3576060640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f357015aac0 0x7f35700b3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39006/0 (socket says 172.21.15.37:39006) 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3576060640 1 -- 172.21.15.37:0/3755544323 learned_addr learned my addr 172.21.15.37:0/3755544323 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f357585f640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35700cd350 0x7f3570155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f357505e640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35700cf0e0 0x7f35700d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3576060640 1 -- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35700cf0e0 msgr2=0x7f35700d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3576060640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35700cf0e0 0x7f35700d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3576060640 1 -- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35700cd350 msgr2=0x7f3570155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:51.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3576060640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35700cd350 0x7f3570155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3576060640 1 -- 172.21.15.37:0/3755544323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3570156050 con 0x7f357015aac0 2024-04-25T08:56:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3576060640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f357015aac0 0x7f35700b3320 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f356800c7a0 tx=0x7f356800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f35627fc640 1 -- 172.21.15.37:0/3755544323 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3568018070 con 0x7f357015aac0 2024-04-25T08:56:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f35627fc640 1 -- 172.21.15.37:0/3755544323 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356800a640 con 0x7f357015aac0 2024-04-25T08:56:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f35627fc640 1 -- 172.21.15.37:0/3755544323 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356800aab0 con 0x7f357015aac0 2024-04-25T08:56:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3577ac3640 1 -- 172.21.15.37:0/3755544323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3570154a80 con 0x7f357015aac0 2024-04-25T08:56:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3577ac3640 1 -- 172.21.15.37:0/3755544323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35701553a0 con 0x7f357015aac0 2024-04-25T08:56:51.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f3577ac3640 1 -- 172.21.15.37:0/3755544323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3570154380 con 0x7f357015aac0 2024-04-25T08:56:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f35627fc640 1 -- 172.21.15.37:0/3755544323 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f356801b070 con 0x7f357015aac0 2024-04-25T08:56:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f35627fc640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3554066f60 0x7f3554069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f35627fc640 1 -- 172.21.15.37:0/3755544323 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f35680847b0 con 0x7f357015aac0 2024-04-25T08:56:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.242+0000 7f357585f640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3554066f60 0x7f3554069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:51.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.246+0000 7f357585f640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3554066f60 0x7f3554069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f356c000f70 tx=0x7f356c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.250+0000 7f35627fc640 1 -- 172.21.15.37:0/3755544323 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3568055690 con 0x7f357015aac0 2024-04-25T08:56:51.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:51.442+0000 7f3577ac3640 1 -- 172.21.15.37:0/3755544323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f357004efa0 con 0x7f357015aac0 2024-04-25T08:56:52.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.106+0000 7f35627fc640 1 -- 172.21.15.37:0/3755544323 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3568054f90 con 0x7f357015aac0 2024-04-25T08:56:52.097 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T08:56:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.110+0000 7f3577ac3640 1 -- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3554066f60 msgr2=0x7f3554069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.110+0000 7f3577ac3640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3554066f60 0x7f3554069420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f356c000f70 tx=0x7f356c009040 comp rx=0 tx=0).stop 2024-04-25T08:56:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.110+0000 7f3577ac3640 1 -- 172.21.15.37:0/3755544323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f357015aac0 msgr2=0x7f35700b3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.110+0000 7f3577ac3640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f357015aac0 0x7f35700b3320 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f356800c7a0 tx=0x7f356800cc60 comp rx=0 tx=0).stop 2024-04-25T08:56:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.110+0000 7f3577ac3640 1 -- 172.21.15.37:0/3755544323 shutdown_connections 2024-04-25T08:56:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.110+0000 7f3577ac3640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3554066f60 0x7f3554069420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.110+0000 7f3577ac3640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f357015aac0 0x7f35700b3320 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.114+0000 7f3577ac3640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f35700cf0e0 0x7f35700d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.114+0000 7f3577ac3640 1 --2- 172.21.15.37:0/3755544323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f35700cd350 0x7f3570155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.114+0000 7f3577ac3640 1 -- 172.21.15.37:0/3755544323 >> 172.21.15.37:0/3755544323 conn(0x7f3570099a70 msgr2=0x7f357006b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:52.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.114+0000 7f3577ac3640 1 -- 172.21.15.37:0/3755544323 shutdown_connections 2024-04-25T08:56:52.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.114+0000 7f3577ac3640 1 -- 172.21.15.37:0/3755544323 wait complete. 2024-04-25T08:56:52.135 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:52.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bce676640 1 Processor -- start 2024-04-25T08:56:52.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bce676640 1 -- start start 2024-04-25T08:56:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bce676640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc815a700 0x7f3bc815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bce676640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3bc80b3f50 0x7f3bc80cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bce676640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc80b2090 con 0x7f3bc80ce430 2024-04-25T08:56:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bce676640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc815f200 con 0x7f3bc80b3f50 2024-04-25T08:56:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bce676640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80b43e0 con 0x7f3bc815a700 2024-04-25T08:56:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bccc13640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3bc80ce430 0x7f3bc80b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44710/0 (socket says 172.21.15.37:44710) 2024-04-25T08:56:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bccc13640 1 -- 172.21.15.37:0/1586170502 learned_addr learned my addr 172.21.15.37:0/1586170502 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bbf7fe640 1 --2- 172.21.15.37:0/1586170502 >> v2:172.21.15.203:3300/0 conn(0x7f3bc80b3f50 0x7f3bc80cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727595123 0 0) 0x7f3bc80b2090 con 0x7f3bc80ce430 2024-04-25T08:56:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc7fff640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc815a700 0x7f3bc815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bb4003530 con 0x7f3bc80ce430 2024-04-25T08:56:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2824816457 0 0) 0x7f3bb4003530 con 0x7f3bc80ce430 2024-04-25T08:56:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc815a700 msgr2=0x7f3bc815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc815a700 0x7f3bc815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 >> v2:172.21.15.203:3300/0 conn(0x7f3bc80b3f50 msgr2=0x7f3bc80cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 --2- 172.21.15.37:0/1586170502 >> v2:172.21.15.203:3300/0 conn(0x7f3bc80b3f50 0x7f3bc80cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc80b46c0 con 0x7f3bc80ce430 2024-04-25T08:56:52.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3bc0001680 con 0x7f3bc80ce430 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 >> v1:172.21.15.37:6789/0 conn(0x7f3bc80ce430 legacy=0x7f3bc80b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc815a700 0x7f3bb4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 0x7f3bb4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bb4004d50 0x7f3bb40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc815f200 con 0x7f3bb4003fb0 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80b43e0 con 0x7f3bb4004d50 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb400bb30 con 0x7f3bc815a700 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bbf7fe640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 0x7f3bb4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bccc13640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bb4004d50 0x7f3bb40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bbf7fe640 1 -- 172.21.15.37:0/1586170502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc815a700 msgr2=0x7f3bb4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bbf7fe640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc815a700 0x7f3bb4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bbf7fe640 1 -- 172.21.15.37:0/1586170502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bb4004d50 msgr2=0x7f3bb40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bbf7fe640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bb4004d50 0x7f3bb40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bbf7fe640 1 -- 172.21.15.37:0/1586170502 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb4003530 con 0x7f3bb4003fb0 2024-04-25T08:56:52.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.262+0000 7f3bccc13640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bb4004d50 0x7f3bb40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:52.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bbf7fe640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 0x7f3bb4004810 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009ae0 tx=0x7f3bb001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540b0558eeed7388 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb00b4070 con 0x7f3bb4003fb0 2024-04-25T08:56:52.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb00a7a00 con 0x7f3bb4003fb0 2024-04-25T08:56:52.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bc77fe640 1 -- 172.21.15.37:0/1586170502 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb00096b0 con 0x7f3bb4003fb0 2024-04-25T08:56:52.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bce676640 1 -- 172.21.15.37:0/1586170502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 msgr2=0x7f3bb4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bce676640 1 --2- 172.21.15.37:0/1586170502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 0x7f3bb4004810 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009ae0 tx=0x7f3bb001ed10 comp rx=0 tx=0).stop 2024-04-25T08:56:52.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bccc13640 1 -- 172.21.15.37:0/1586170502 reap_dead start 2024-04-25T08:56:52.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bce676640 1 -- 172.21.15.37:0/1586170502 shutdown_connections 2024-04-25T08:56:52.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bce676640 1 -- 172.21.15.37:0/1586170502 >> 172.21.15.37:0/1586170502 conn(0x7f3bc80b11a0 msgr2=0x7f3bc80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:52.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bce676640 1 -- 172.21.15.37:0/1586170502 shutdown_connections 2024-04-25T08:56:52.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bce676640 1 -- 172.21.15.37:0/1586170502 wait complete. 2024-04-25T08:56:52.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bce676640 1 Processor -- start 2024-04-25T08:56:52.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.266+0000 7f3bce676640 1 -- start start 2024-04-25T08:56:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bce676640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 0x7f3bc8141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bce676640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc81520b0 0x7f3bc815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bce676640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc8140be0 0x7f3bc8141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bce676640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80cd490 con 0x7f3bb4003fb0 2024-04-25T08:56:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bce676640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80cd310 con 0x7f3bc81520b0 2024-04-25T08:56:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bce676640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80b2bb0 con 0x7f3bc8140be0 2024-04-25T08:56:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 0x7f3bc8141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 0x7f3bc8141820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39020/0 (socket says 172.21.15.37:39020) 2024-04-25T08:56:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc7fff640 1 -- 172.21.15.37:0/3144694047 learned_addr learned my addr 172.21.15.37:0/3144694047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bbf7fe640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc81520b0 0x7f3bc815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc7fff640 1 -- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc8140be0 msgr2=0x7f3bc8141060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc7fff640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc8140be0 0x7f3bc8141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc7fff640 1 -- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc81520b0 msgr2=0x7f3bc815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc7fff640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc81520b0 0x7f3bc815bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc7fff640 1 -- 172.21.15.37:0/3144694047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc81415a0 con 0x7f3bb4003fb0 2024-04-25T08:56:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc7fff640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 0x7f3bc8141820 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800ef10 tx=0x7f3bb800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc57fa640 1 -- 172.21.15.37:0/3144694047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8018020 con 0x7f3bb4003fb0 2024-04-25T08:56:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc57fa640 1 -- 172.21.15.37:0/3144694047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb8014400 con 0x7f3bb4003fb0 2024-04-25T08:56:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bce676640 1 -- 172.21.15.37:0/3144694047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc815af20 con 0x7f3bb4003fb0 2024-04-25T08:56:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc57fa640 1 -- 172.21.15.37:0/3144694047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb80147d0 con 0x7f3bb4003fb0 2024-04-25T08:56:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bce676640 1 -- 172.21.15.37:0/3144694047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc815b4e0 con 0x7f3bb4003fb0 2024-04-25T08:56:52.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bce676640 1 -- 172.21.15.37:0/3144694047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc804efa0 con 0x7f3bb4003fb0 2024-04-25T08:56:52.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc57fa640 1 -- 172.21.15.37:0/3144694047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3bb801b020 con 0x7f3bb4003fb0 2024-04-25T08:56:52.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc57fa640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ba8066f60 0x7f3ba8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bc57fa640 1 -- 172.21.15.37:0/3144694047 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3bb8085fd0 con 0x7f3bb4003fb0 2024-04-25T08:56:52.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.270+0000 7f3bbf7fe640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ba8066f60 0x7f3ba8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.274+0000 7f3bbf7fe640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ba8066f60 0x7f3ba8069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00b1000 tx=0x7f3bb00046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.274+0000 7f3bc57fa640 1 -- 172.21.15.37:0/3144694047 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3bb8022030 con 0x7f3bb4003fb0 2024-04-25T08:56:52.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.438+0000 7f3bce676640 1 -- 172.21.15.37:0/3144694047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bc806f260 con 0x7f3bb4003fb0 2024-04-25T08:56:52.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.438+0000 7f3bc57fa640 1 -- 172.21.15.37:0/3144694047 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16922 (secure 0 0 0) 0x7f3bb8056ea0 con 0x7f3bb4003fb0 2024-04-25T08:56:52.430 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:52.430 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:56:51.447041+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:56:49.737591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:56:50.436964+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:52.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 -- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ba8066f60 msgr2=0x7f3ba8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ba8066f60 0x7f3ba8069420 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3bb00b1000 tx=0x7f3bb00046d0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 -- 172.21.15.37:0/3144694047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 msgr2=0x7f3bc8141820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 0x7f3bc8141820 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800ef10 tx=0x7f3bb800b480 comp rx=0 tx=0).stop 2024-04-25T08:56:52.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 -- 172.21.15.37:0/3144694047 shutdown_connections 2024-04-25T08:56:52.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ba8066f60 0x7f3ba8069420 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bc8140be0 0x7f3bc8141060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bc81520b0 0x7f3bc815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 --2- 172.21.15.37:0/3144694047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bb4003fb0 0x7f3bc8141820 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 -- 172.21.15.37:0/3144694047 >> 172.21.15.37:0/3144694047 conn(0x7f3bc80b11a0 msgr2=0x7f3bc806e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 -- 172.21.15.37:0/3144694047 shutdown_connections 2024-04-25T08:56:52.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.442+0000 7f3bce676640 1 -- 172.21.15.37:0/3144694047 wait complete. 2024-04-25T08:56:52.450 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:52.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.582+0000 7f2763785640 1 Processor -- start 2024-04-25T08:56:52.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.582+0000 7f2763785640 1 -- start start 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2763785640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f275c15a700 0x7f275c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2763785640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f275c0b3f50 0x7f275c0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2763785640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275c0b2090 con 0x7f275c0ce430 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2763785640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275c15f200 con 0x7f275c0b3f50 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2763785640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0b43e0 con 0x7f275c15a700 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2761d22640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f275c0ce430 0x7f275c0b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44724/0 (socket says 172.21.15.37:44724) 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2760d20640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f275c0b3f50 0x7f275c0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2761d22640 1 -- 172.21.15.37:0/22930652 learned_addr learned my addr 172.21.15.37:0/22930652 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2761521640 1 --2- 172.21.15.37:0/22930652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f275c15a700 0x7f275c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2760d20640 1 -- 172.21.15.37:0/22930652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f275c15a700 msgr2=0x7f275c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2760d20640 1 --2- 172.21.15.37:0/22930652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f275c15a700 0x7f275c15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2760d20640 1 -- 172.21.15.37:0/22930652 >> v1:172.21.15.37:6789/0 conn(0x7f275c0ce430 legacy=0x7f275c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2760d20640 1 -- 172.21.15.37:0/22930652 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c0b46c0 con 0x7f275c0b3f50 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f274ffff640 1 -- 172.21.15.37:0/22930652 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650024760 0 0) 0x7f275c0b2090 con 0x7f275c0ce430 2024-04-25T08:56:52.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f2760d20640 1 --2- 172.21.15.37:0/22930652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275c0b3f50 0x7f275c0cdef0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f27500097c0 tx=0x7f275001c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fa43b842fef6a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f274ffff640 1 -- 172.21.15.37:0/22930652 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27500b0070 con 0x7f275c0b3f50 2024-04-25T08:56:52.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f274ffff640 1 -- 172.21.15.37:0/22930652 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2750004230 con 0x7f275c0b3f50 2024-04-25T08:56:52.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.586+0000 7f274ffff640 1 -- 172.21.15.37:0/22930652 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750003620 con 0x7f275c0b3f50 2024-04-25T08:56:52.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 -- 172.21.15.37:0/22930652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275c0b3f50 msgr2=0x7f275c0cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 --2- 172.21.15.37:0/22930652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275c0b3f50 0x7f275c0cdef0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f27500097c0 tx=0x7f275001c350 comp rx=0 tx=0).stop 2024-04-25T08:56:52.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 -- 172.21.15.37:0/22930652 shutdown_connections 2024-04-25T08:56:52.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 --2- 172.21.15.37:0/22930652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275c0b3f50 0x7f275c0cdef0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 --2- 172.21.15.37:0/22930652 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f275c15a700 0x7f275c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 -- 172.21.15.37:0/22930652 >> 172.21.15.37:0/22930652 conn(0x7f275c0b11a0 msgr2=0x7f275c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:52.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 -- 172.21.15.37:0/22930652 shutdown_connections 2024-04-25T08:56:52.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 -- 172.21.15.37:0/22930652 wait complete. 2024-04-25T08:56:52.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 Processor -- start 2024-04-25T08:56:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 -- start start 2024-04-25T08:56:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f275c0ce430 0x7f275c0afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f275c15a700 0x7f275c14cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275c14d930 0x7f275c14d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0cd490 con 0x7f275c14d930 2024-04-25T08:56:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0cd310 con 0x7f275c15a700 2024-04-25T08:56:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0b2bb0 con 0x7f275c0ce430 2024-04-25T08:56:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2761d22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275c14d930 0x7f275c14d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2761d22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275c14d930 0x7f275c14d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39052/0 (socket says 172.21.15.37:39052) 2024-04-25T08:56:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2761d22640 1 -- 172.21.15.37:0/1915886741 learned_addr learned my addr 172.21.15.37:0/1915886741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:52.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2761521640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f275c0ce430 0x7f275c0afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2760d20640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f275c15a700 0x7f275c14cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2761d22640 1 -- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f275c0ce430 msgr2=0x7f275c0afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2761d22640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f275c0ce430 0x7f275c0afdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2761d22640 1 -- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f275c15a700 msgr2=0x7f275c14cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2761d22640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f275c15a700 0x7f275c14cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2761d22640 1 -- 172.21.15.37:0/1915886741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c14f050 con 0x7f275c14d930 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2761d22640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275c14d930 0x7f275c14d2c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f275400c7a0 tx=0x7f275400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f274dffb640 1 -- 172.21.15.37:0/1915886741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2754018070 con 0x7f275c14d930 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f274dffb640 1 -- 172.21.15.37:0/1915886741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f275400a640 con 0x7f275c14d930 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 -- 172.21.15.37:0/1915886741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275c1518f0 con 0x7f275c14d930 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f274dffb640 1 -- 172.21.15.37:0/1915886741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275400aab0 con 0x7f275c14d930 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.590+0000 7f2763785640 1 -- 172.21.15.37:0/1915886741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c14f600 con 0x7f275c14d930 2024-04-25T08:56:52.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.594+0000 7f2763785640 1 -- 172.21.15.37:0/1915886741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c14f3f0 con 0x7f275c14d930 2024-04-25T08:56:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.594+0000 7f274dffb640 1 -- 172.21.15.37:0/1915886741 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f275401b070 con 0x7f275c14d930 2024-04-25T08:56:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.594+0000 7f274dffb640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2740066f90 0x7f2740069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.594+0000 7f274dffb640 1 -- 172.21.15.37:0/1915886741 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27540846f0 con 0x7f275c14d930 2024-04-25T08:56:52.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.594+0000 7f2761521640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2740066f90 0x7f2740069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.594+0000 7f2761521640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2740066f90 0x7f2740069450 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f275c06d1a0 tx=0x7f2758008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.598+0000 7f274dffb640 1 -- 172.21.15.37:0/1915886741 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2754055600 con 0x7f275c14d930 2024-04-25T08:56:52.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.762+0000 7f2763785640 1 -- 172.21.15.37:0/1915886741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f275c06f260 con 0x7f275c14d930 2024-04-25T08:56:52.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.762+0000 7f274dffb640 1 -- 172.21.15.37:0/1915886741 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16922 (secure 0 0 0) 0x7f2754054f00 con 0x7f275c14d930 2024-04-25T08:56:52.753 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:52.753 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:56:51.447041+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:56:49.737591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:56:50.436964+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 -- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2740066f90 msgr2=0x7f2740069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2740066f90 0x7f2740069450 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f275c06d1a0 tx=0x7f2758008040 comp rx=0 tx=0).stop 2024-04-25T08:56:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 -- 172.21.15.37:0/1915886741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275c14d930 msgr2=0x7f275c14d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275c14d930 0x7f275c14d2c0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f275400c7a0 tx=0x7f275400cc60 comp rx=0 tx=0).stop 2024-04-25T08:56:52.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 -- 172.21.15.37:0/1915886741 shutdown_connections 2024-04-25T08:56:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2740066f90 0x7f2740069450 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f275c14d930 0x7f275c14d2c0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f275c15a700 0x7f275c14cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 --2- 172.21.15.37:0/1915886741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f275c0ce430 0x7f275c0afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 -- 172.21.15.37:0/1915886741 >> 172.21.15.37:0/1915886741 conn(0x7f275c0b11a0 msgr2=0x7f275c0ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 -- 172.21.15.37:0/1915886741 shutdown_connections 2024-04-25T08:56:52.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.766+0000 7f2763785640 1 -- 172.21.15.37:0/1915886741 wait complete. 2024-04-25T08:56:52.773 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:56:52.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d8cf8640 1 Processor -- start 2024-04-25T08:56:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d8cf8640 1 -- start start 2024-04-25T08:56:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d8cf8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7d40d07f0 0x7fc7d40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d8cf8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc7d40d1cd0 0x7fc7d406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d8cf8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d406be60 con 0x7fc7d415da80 2024-04-25T08:56:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d8cf8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d406c440 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d8cf8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d415de60 con 0x7fc7d40d07f0 2024-04-25T08:56:52.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d2d9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc7d40d1cd0 0x7fc7d406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d1d9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7d40d07f0 0x7fc7d40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d2d9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc7d40d1cd0 0x7fc7d406c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:57074/0 (socket says 172.21.15.37:57074) 2024-04-25T08:56:52.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d2d9d640 1 -- 172.21.15.37:0/1476275557 learned_addr learned my addr 172.21.15.37:0/1476275557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:52.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d2d9d640 1 -- 172.21.15.37:0/1476275557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7d40d07f0 msgr2=0x7fc7d40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:52.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d2d9d640 1 --2- 172.21.15.37:0/1476275557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7d40d07f0 0x7fc7d40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d2d9d640 1 -- 172.21.15.37:0/1476275557 >> v1:172.21.15.203:6789/0 conn(0x7fc7d415da80 legacy=0x7fc7d40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.906+0000 7fc7d2d9d640 1 -- 172.21.15.37:0/1476275557 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d40d2450 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d2d9d640 1 --2- 172.21.15.37:0/1476275557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7d40d1cd0 0x7fc7d406c8d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009e50 tx=0x7fc7cc01ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed2715248d218f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d159a640 1 -- 172.21.15.37:0/1476275557 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0a58b0 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d159a640 1 -- 172.21.15.37:0/1476275557 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc0a5a50 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d159a640 1 -- 172.21.15.37:0/1476275557 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc0a5e20 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/1476275557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7d40d1cd0 msgr2=0x7fc7d406c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d8cf8640 1 --2- 172.21.15.37:0/1476275557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7d40d1cd0 0x7fc7d406c8d0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009e50 tx=0x7fc7cc01ca00 comp rx=0 tx=0).stop 2024-04-25T08:56:52.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/1476275557 shutdown_connections 2024-04-25T08:56:52.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d8cf8640 1 --2- 172.21.15.37:0/1476275557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7d40d1cd0 0x7fc7d406c8d0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d8cf8640 1 --2- 172.21.15.37:0/1476275557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7d40d07f0 0x7fc7d40d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/1476275557 >> 172.21.15.37:0/1476275557 conn(0x7fc7d408b000 msgr2=0x7fc7d40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:52.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/1476275557 shutdown_connections 2024-04-25T08:56:52.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.910+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/1476275557 wait complete. 2024-04-25T08:56:52.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d8cf8640 1 Processor -- start 2024-04-25T08:56:52.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d8cf8640 1 -- start start 2024-04-25T08:56:52.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d8cf8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7d40d07f0 0x7fc7d40b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d8cf8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7d40d1cd0 0x7fc7d40b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d8cf8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7d415da80 0x7fc7d415f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d8cf8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d406d7a0 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d8cf8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d406d620 con 0x7fc7d415da80 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d8cf8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d406d920 con 0x7fc7d40d07f0 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7d40d1cd0 0x7fc7d40b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7d40d1cd0 0x7fc7d40b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39056/0 (socket says 172.21.15.37:39056) 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d1d9b640 1 -- 172.21.15.37:0/3449717497 learned_addr learned my addr 172.21.15.37:0/3449717497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d2d9d640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7d415da80 0x7fc7d415f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d259c640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7d40d07f0 0x7fc7d40b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d1d9b640 1 -- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7d40d07f0 msgr2=0x7fc7d40b8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d1d9b640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7d40d07f0 0x7fc7d40b8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d1d9b640 1 -- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7d415da80 msgr2=0x7fc7d415f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d1d9b640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7d415da80 0x7fc7d415f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d1d9b640 1 -- 172.21.15.37:0/3449717497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d40b7260 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d1d9b640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7d40d1cd0 0x7fc7d40b6d20 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00e9e0 tx=0x7fc7bc00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7caffd640 1 -- 172.21.15.37:0/3449717497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc017070 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7caffd640 1 -- 172.21.15.37:0/3449717497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc00c9d0 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/3449717497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d4155e10 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7caffd640 1 -- 172.21.15.37:0/3449717497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc00cd80 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/3449717497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d40b7da0 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/3449717497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d404efa0 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.914+0000 7fc7caffd640 1 -- 172.21.15.37:0/3449717497 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc7bc01a070 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.918+0000 7fc7caffd640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc7b4066f60 0x7fc7b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.918+0000 7fc7caffd640 1 -- 172.21.15.37:0/3449717497 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc7bc085590 con 0x7fc7d40d1cd0 2024-04-25T08:56:52.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.918+0000 7fc7d259c640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc7b4066f60 0x7fc7b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.918+0000 7fc7d259c640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc7b4066f60 0x7fc7b4069420 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40015b0 tx=0x7fc7c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:52.922+0000 7fc7caffd640 1 -- 172.21.15.37:0/3449717497 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc7d40b7260 con 0x7fc7d40d1cd0 2024-04-25T08:56:53.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:53.122+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/3449717497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc7d40b38b0 con 0x7fc7d40d1cd0 2024-04-25T08:56:54.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.122+0000 7fc7caffd640 1 -- 172.21.15.37:0/3449717497 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc7bc056410 con 0x7fc7d40d1cd0 2024-04-25T08:56:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc7b4066f60 msgr2=0x7fc7b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc7b4066f60 0x7fc7b4069420 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40015b0 tx=0x7fc7c4009040 comp rx=0 tx=0).stop 2024-04-25T08:56:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/3449717497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7d40d1cd0 msgr2=0x7fc7d40b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7d40d1cd0 0x7fc7d40b6d20 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00e9e0 tx=0x7fc7bc00eea0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/3449717497 shutdown_connections 2024-04-25T08:56:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc7b4066f60 0x7fc7b4069420 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc7d415da80 0x7fc7d415f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7d40d1cd0 0x7fc7d40b6d20 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 --2- 172.21.15.37:0/3449717497 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc7d40d07f0 0x7fc7d40b8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/3449717497 >> 172.21.15.37:0/3449717497 conn(0x7fc7d408b000 msgr2=0x7fc7d40b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/3449717497 shutdown_connections 2024-04-25T08:56:54.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.126+0000 7fc7d8cf8640 1 -- 172.21.15.37:0/3449717497 wait complete. 2024-04-25T08:56:54.138 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:56:54.138 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:56:54.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a8c438640 1 Processor -- start 2024-04-25T08:56:54.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a8c438640 1 -- start start 2024-04-25T08:56:54.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a8c438640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2a840c9f70 0x7f2a840c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a8c438640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a840b3b60 con 0x7f2a84161c80 2024-04-25T08:56:54.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a8c438640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a84162060 con 0x7f2a840c2870 2024-04-25T08:56:54.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a8c438640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a840c2cc0 con 0x7f2a840c9f70 2024-04-25T08:56:54.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a8a1d4640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2a84161c80 0x7f2a840b3450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44736/0 (socket says 172.21.15.37:44736) 2024-04-25T08:56:54.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a8a1d4640 1 -- 172.21.15.37:0/4286698207 learned_addr learned my addr 172.21.15.37:0/4286698207 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:54.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a899d3640 1 --2- 172.21.15.37:0/4286698207 >> v2:172.21.15.37:3300/0 conn(0x7f2a840c9f70 0x7f2a840c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a891d2640 1 -- 172.21.15.37:0/4286698207 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026843786 0 0) 0x7f2a840b3b60 con 0x7f2a84161c80 2024-04-25T08:56:54.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a891d2640 1 -- 172.21.15.37:0/4286698207 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a70003530 con 0x7f2a84161c80 2024-04-25T08:56:54.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a899d3640 1 -- 172.21.15.37:0/4286698207 >> v1:172.21.15.203:6789/0 conn(0x7f2a840c2870 legacy=0x7f2a840ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:54.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a899d3640 1 -- 172.21.15.37:0/4286698207 >> v1:172.21.15.37:6789/0 conn(0x7f2a84161c80 legacy=0x7f2a840b3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.274+0000 7f2a899d3640 1 -- 172.21.15.37:0/4286698207 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a840ca480 con 0x7f2a840c9f70 2024-04-25T08:56:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a899d3640 1 --2- 172.21.15.37:0/4286698207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a840c9f70 0x7f2a840c2330 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2a740097c0 tx=0x7f2a7401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14de7e4762ef6e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a891d2640 1 -- 172.21.15.37:0/4286698207 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a740ab430 con 0x7f2a840c9f70 2024-04-25T08:56:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a891d2640 1 -- 172.21.15.37:0/4286698207 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a740b0070 con 0x7f2a840c9f70 2024-04-25T08:56:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a891d2640 1 -- 172.21.15.37:0/4286698207 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a740ab7f0 con 0x7f2a840c9f70 2024-04-25T08:56:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a8c438640 1 -- 172.21.15.37:0/4286698207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a840c9f70 msgr2=0x7f2a840c2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a8c438640 1 --2- 172.21.15.37:0/4286698207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a840c9f70 0x7f2a840c2330 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f2a740097c0 tx=0x7f2a7401c430 comp rx=0 tx=0).stop 2024-04-25T08:56:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a8c438640 1 -- 172.21.15.37:0/4286698207 shutdown_connections 2024-04-25T08:56:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a8c438640 1 --2- 172.21.15.37:0/4286698207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a840c9f70 0x7f2a840c2330 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a8c438640 1 -- 172.21.15.37:0/4286698207 >> 172.21.15.37:0/4286698207 conn(0x7f2a8406ae60 msgr2=0x7f2a8406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:54.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a8c438640 1 -- 172.21.15.37:0/4286698207 shutdown_connections 2024-04-25T08:56:54.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a8c438640 1 -- 172.21.15.37:0/4286698207 wait complete. 2024-04-25T08:56:54.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a8c438640 1 Processor -- start 2024-04-25T08:56:54.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.278+0000 7f2a8c438640 1 -- start start 2024-04-25T08:56:54.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8c438640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a840c2870 0x7f2a840c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8c438640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a840c9f70 0x7f2a84072c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8c438640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a84161c80 0x7f2a84160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8c438640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a840cde50 con 0x7f2a84161c80 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8c438640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a840cafe0 con 0x7f2a840c9f70 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8c438640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a840cdfd0 con 0x7f2a840c2870 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8a9d5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a84161c80 0x7f2a84160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8a9d5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a84161c80 0x7f2a84160e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39072/0 (socket says 172.21.15.37:39072) 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8a9d5640 1 -- 172.21.15.37:0/1801337897 learned_addr learned my addr 172.21.15.37:0/1801337897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a899d3640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a840c9f70 0x7f2a84072c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8a1d4640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a840c2870 0x7f2a840c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8a9d5640 1 -- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a840c2870 msgr2=0x7f2a840c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8a9d5640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a840c2870 0x7f2a840c5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8a9d5640 1 -- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a840c9f70 msgr2=0x7f2a84072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8a9d5640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a840c9f70 0x7f2a84072c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8a9d5640 1 -- 172.21.15.37:0/1801337897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a84161800 con 0x7f2a84161c80 2024-04-25T08:56:54.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a899d3640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a840c9f70 0x7f2a84072c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:54.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8a9d5640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a84161c80 0x7f2a84160e20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c780 tx=0x7f2a7c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:54.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a7affd640 1 -- 172.21.15.37:0/1801337897 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c004b70 con 0x7f2a84161c80 2024-04-25T08:56:54.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a7affd640 1 -- 172.21.15.37:0/1801337897 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c004d10 con 0x7f2a84161c80 2024-04-25T08:56:54.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8c438640 1 -- 172.21.15.37:0/1801337897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a84161520 con 0x7f2a84161c80 2024-04-25T08:56:54.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a7affd640 1 -- 172.21.15.37:0/1801337897 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00aa90 con 0x7f2a84161c80 2024-04-25T08:56:54.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8c438640 1 -- 172.21.15.37:0/1801337897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a840c5150 con 0x7f2a84161c80 2024-04-25T08:56:54.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8c438640 1 -- 172.21.15.37:0/1801337897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8404efa0 con 0x7f2a84161c80 2024-04-25T08:56:54.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a7affd640 1 -- 172.21.15.37:0/1801337897 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2a7c00ac30 con 0x7f2a84161c80 2024-04-25T08:56:54.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a7affd640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2a68066f90 0x7f2a68069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a7affd640 1 -- 172.21.15.37:0/1801337897 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2a7c085450 con 0x7f2a84161c80 2024-04-25T08:56:54.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.282+0000 7f2a8a1d4640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2a68066f90 0x7f2a68069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.286+0000 7f2a8a1d4640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2a68066f90 0x7f2a68069450 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2a80007fd0 tx=0x7f2a80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:54.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.286+0000 7f2a7affd640 1 -- 172.21.15.37:0/1801337897 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2a7c020070 con 0x7f2a84161c80 2024-04-25T08:56:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:54.482+0000 7f2a8c438640 1 -- 172.21.15.37:0/1801337897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2a8416cf40 con 0x7f2a84161c80 2024-04-25T08:56:55.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.134+0000 7f2a7affd640 1 -- 172.21.15.37:0/1801337897 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2a7c0562c0 con 0x7f2a84161c80 2024-04-25T08:56:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 -- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2a68066f90 msgr2=0x7f2a68069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2a68066f90 0x7f2a68069450 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f2a80007fd0 tx=0x7f2a80009040 comp rx=0 tx=0).stop 2024-04-25T08:56:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 -- 172.21.15.37:0/1801337897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a84161c80 msgr2=0x7f2a84160e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a84161c80 0x7f2a84160e20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c780 tx=0x7f2a7c00cc40 comp rx=0 tx=0).stop 2024-04-25T08:56:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 -- 172.21.15.37:0/1801337897 shutdown_connections 2024-04-25T08:56:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2a68066f90 0x7f2a68069450 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2a84161c80 0x7f2a84160e20 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2a840c9f70 0x7f2a84072c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 --2- 172.21.15.37:0/1801337897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2a840c2870 0x7f2a840c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 -- 172.21.15.37:0/1801337897 >> 172.21.15.37:0/1801337897 conn(0x7f2a8406ae60 msgr2=0x7f2a84073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 -- 172.21.15.37:0/1801337897 shutdown_connections 2024-04-25T08:56:55.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.138+0000 7f2a8c438640 1 -- 172.21.15.37:0/1801337897 wait complete. 2024-04-25T08:56:55.155 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:56:55.155 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:55.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.294+0000 7fd6552b4640 1 Processor -- start 2024-04-25T08:56:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd6552b4640 1 -- start start 2024-04-25T08:56:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd6552b4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6500d07f0 0x7fd6500d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd6552b4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65006be60 con 0x7fd65015da80 2024-04-25T08:56:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd6552b4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd65015de60 con 0x7fd6500d1cd0 2024-04-25T08:56:55.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd6552b4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65006c440 con 0x7fd6500d07f0 2024-04-25T08:56:55.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64effd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd65015da80 0x7fd6500d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44750/0 (socket says 172.21.15.37:44750) 2024-04-25T08:56:55.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64effd640 1 -- 172.21.15.37:0/1504423489 learned_addr learned my addr 172.21.15.37:0/1504423489 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:55.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd6467fc640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6500d07f0 0x7fd6500d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337931382 0 0) 0x7fd65006be60 con 0x7fd65015da80 2024-04-25T08:56:55.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd63c003530 con 0x7fd65015da80 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 923313015 0 0) 0x7fd63c003530 con 0x7fd65015da80 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6500d07f0 msgr2=0x7fd6500d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6500d07f0 0x7fd6500d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 >> v1:172.21.15.203:6789/0 conn(0x7fd6500d1cd0 legacy=0x7fd65006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6500d2450 con 0x7fd65015da80 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fd640001680 con 0x7fd65015da80 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 >> v1:172.21.15.37:6789/0 conn(0x7fd65015da80 legacy=0x7fd6500d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd63c003a60 0x7fd63c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6500d07f0 0x7fd63c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd63c004d50 0x7fd63c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65006c440 con 0x7fd63c003a60 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c00bb10 con 0x7fd63c004d50 2024-04-25T08:56:55.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c00bd10 con 0x7fd6500d07f0 2024-04-25T08:56:55.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64effd640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd63c003a60 0x7fd63c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd6467fc640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6500d07f0 0x7fd63c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64f7fe640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd63c004d50 0x7fd63c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64effd640 1 -- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6500d07f0 msgr2=0x7fd63c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:55.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64effd640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6500d07f0 0x7fd63c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64effd640 1 -- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd63c004d50 msgr2=0x7fd63c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:55.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64effd640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd63c004d50 0x7fd63c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd64effd640 1 -- 172.21.15.37:0/1504423489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c003530 con 0x7fd63c003a60 2024-04-25T08:56:55.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.298+0000 7fd6467fc640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6500d07f0 0x7fd63c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:55.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd64effd640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd63c003a60 0x7fd63c003e80 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd64000c210 tx=0x7fd64001cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a033c518ccd53419 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640007d20 con 0x7fd63c003a60 2024-04-25T08:56:55.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6400a9070 con 0x7fd63c003a60 2024-04-25T08:56:55.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd64e7fc640 1 -- 172.21.15.37:0/1504423489 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640005d40 con 0x7fd63c003a60 2024-04-25T08:56:55.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd6552b4640 1 -- 172.21.15.37:0/1504423489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd63c003a60 msgr2=0x7fd63c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd6552b4640 1 --2- 172.21.15.37:0/1504423489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd63c003a60 0x7fd63c003e80 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fd64000c210 tx=0x7fd64001cb70 comp rx=0 tx=0).stop 2024-04-25T08:56:55.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd64f7fe640 1 -- 172.21.15.37:0/1504423489 reap_dead start 2024-04-25T08:56:55.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd6552b4640 1 -- 172.21.15.37:0/1504423489 shutdown_connections 2024-04-25T08:56:55.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd6552b4640 1 -- 172.21.15.37:0/1504423489 >> 172.21.15.37:0/1504423489 conn(0x7fd65008b000 msgr2=0x7fd6500d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:55.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd6552b4640 1 -- 172.21.15.37:0/1504423489 shutdown_connections 2024-04-25T08:56:55.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd6552b4640 1 -- 172.21.15.37:0/1504423489 wait complete. 2024-04-25T08:56:55.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd6552b4640 1 Processor -- start 2024-04-25T08:56:55.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd6552b4640 1 -- start start 2024-04-25T08:56:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd6552b4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd63c003a60 0x7fd6500b2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.302+0000 7fd6552b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd650157ff0 0x7fd650158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6552b4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6500b07e0 0x7fd6500b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6552b4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65006d970 con 0x7fd650157ff0 2024-04-25T08:56:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6552b4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd65006d7f0 con 0x7fd6500b07e0 2024-04-25T08:56:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6552b4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd65006daf0 con 0x7fd63c003a60 2024-04-25T08:56:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6467fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd650157ff0 0x7fd650158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6467fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd650157ff0 0x7fd650158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39094/0 (socket says 172.21.15.37:39094) 2024-04-25T08:56:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6467fc640 1 -- 172.21.15.37:0/1695526803 learned_addr learned my addr 172.21.15.37:0/1695526803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6467fc640 1 -- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd63c003a60 msgr2=0x7fd6500b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd64effd640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd63c003a60 0x7fd6500b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6467fc640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd63c003a60 0x7fd6500b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6467fc640 1 -- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6500b07e0 msgr2=0x7fd6500b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6467fc640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6500b07e0 0x7fd6500b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6467fc640 1 -- 172.21.15.37:0/1695526803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6500b7920 con 0x7fd650157ff0 2024-04-25T08:56:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6467fc640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd650157ff0 0x7fd650158470 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd638010910 tx=0x7fd638010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd647fff640 1 -- 172.21.15.37:0/1695526803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63800f040 con 0x7fd650157ff0 2024-04-25T08:56:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd647fff640 1 -- 172.21.15.37:0/1695526803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63800ec20 con 0x7fd650157ff0 2024-04-25T08:56:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6552b4640 1 -- 172.21.15.37:0/1695526803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6500b7c00 con 0x7fd650157ff0 2024-04-25T08:56:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd647fff640 1 -- 172.21.15.37:0/1695526803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638004020 con 0x7fd650157ff0 2024-04-25T08:56:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6552b4640 1 -- 172.21.15.37:0/1695526803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6500b81c0 con 0x7fd650157ff0 2024-04-25T08:56:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd6552b4640 1 -- 172.21.15.37:0/1695526803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd65004efa0 con 0x7fd650157ff0 2024-04-25T08:56:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd647fff640 1 -- 172.21.15.37:0/1695526803 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6380041c0 con 0x7fd650157ff0 2024-04-25T08:56:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd647fff640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd630066f90 0x7fd630069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd647fff640 1 -- 172.21.15.37:0/1695526803 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6380855a0 con 0x7fd650157ff0 2024-04-25T08:56:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.306+0000 7fd64effd640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd630066f90 0x7fd630069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.310+0000 7fd64effd640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd630066f90 0x7fd630069450 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd6400ab000 tx=0x7fd6400a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.310+0000 7fd647fff640 1 -- 172.21.15.37:0/1695526803 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd638018070 con 0x7fd650157ff0 2024-04-25T08:56:55.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.498+0000 7fd6552b4640 1 -- 172.21.15.37:0/1695526803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6500b2b60 con 0x7fd650157ff0 2024-04-25T08:56:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.498+0000 7fd647fff640 1 -- 172.21.15.37:0/1695526803 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fd638056420 con 0x7fd650157ff0 2024-04-25T08:56:55.490 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:55.490 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:56:54.486633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/4130042812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4130042812},{"type":"v1","addr":"172.21.15.37:6835","nonce":4130042812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:56:55.490 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 44 2024-04-25T08:56:55.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 -- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd630066f90 msgr2=0x7fd630069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd630066f90 0x7fd630069450 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd6400ab000 tx=0x7fd6400a7070 comp rx=0 tx=0).stop 2024-04-25T08:56:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 -- 172.21.15.37:0/1695526803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd650157ff0 msgr2=0x7fd650158470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd650157ff0 0x7fd650158470 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fd638010910 tx=0x7fd638010dd0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 -- 172.21.15.37:0/1695526803 shutdown_connections 2024-04-25T08:56:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd630066f90 0x7fd630069450 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6500b07e0 0x7fd6500b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd650157ff0 0x7fd650158470 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 --2- 172.21.15.37:0/1695526803 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd63c003a60 0x7fd6500b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 -- 172.21.15.37:0/1695526803 >> 172.21.15.37:0/1695526803 conn(0x7fd65008b000 msgr2=0x7fd65015cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 -- 172.21.15.37:0/1695526803 shutdown_connections 2024-04-25T08:56:55.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.502+0000 7fd6552b4640 1 -- 172.21.15.37:0/1695526803 wait complete. 2024-04-25T08:56:55.507 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1cab018640 1 Processor -- start 2024-04-25T08:56:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1cab018640 1 -- start start 2024-04-25T08:56:55.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1cab018640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40cc560 0x7f1ca40d0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1cab018640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ca40ce720 0x7f1ca415b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1cab018640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca40cf170 con 0x7f1ca4071560 2024-04-25T08:56:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1cab018640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca40cf450 con 0x7f1ca40ce720 2024-04-25T08:56:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1cab018640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca40cf730 con 0x7f1ca40cc560 2024-04-25T08:56:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1ca95b5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ca40ce720 0x7f1ca415b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1ca95b5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ca40ce720 0x7f1ca415b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39106/0 (socket says 172.21.15.37:39106) 2024-04-25T08:56:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1ca95b5640 1 -- 172.21.15.37:0/3473287818 learned_addr learned my addr 172.21.15.37:0/3473287818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1ca8db4640 1 --2- 172.21.15.37:0/3473287818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40cc560 0x7f1ca40d0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1ca95b5640 1 -- 172.21.15.37:0/3473287818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40cc560 msgr2=0x7f1ca40d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1ca95b5640 1 --2- 172.21.15.37:0/3473287818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40cc560 0x7f1ca40d0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1ca95b5640 1 -- 172.21.15.37:0/3473287818 >> v1:172.21.15.203:6789/0 conn(0x7f1ca4071560 legacy=0x7f1ca40cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.642+0000 7f1ca95b5640 1 -- 172.21.15.37:0/3473287818 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca40cfa10 con 0x7f1ca40ce720 2024-04-25T08:56:55.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1ca95b5640 1 --2- 172.21.15.37:0/3473287818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca40ce720 0x7f1ca415b900 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009e50 tx=0x7f1c9c01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48745f5f7e72c106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1ca37fe640 1 -- 172.21.15.37:0/3473287818 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0a57e0 con 0x7f1ca40ce720 2024-04-25T08:56:55.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1ca37fe640 1 -- 172.21.15.37:0/3473287818 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c0a5980 con 0x7f1ca40ce720 2024-04-25T08:56:55.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1ca37fe640 1 -- 172.21.15.37:0/3473287818 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0a5d50 con 0x7f1ca40ce720 2024-04-25T08:56:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1cab018640 1 -- 172.21.15.37:0/3473287818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca40ce720 msgr2=0x7f1ca415b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1cab018640 1 --2- 172.21.15.37:0/3473287818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca40ce720 0x7f1ca415b900 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009e50 tx=0x7f1c9c01ca00 comp rx=0 tx=0).stop 2024-04-25T08:56:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1cab018640 1 -- 172.21.15.37:0/3473287818 shutdown_connections 2024-04-25T08:56:55.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1cab018640 1 --2- 172.21.15.37:0/3473287818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca40ce720 0x7f1ca415b900 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1cab018640 1 --2- 172.21.15.37:0/3473287818 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40cc560 0x7f1ca40d0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1cab018640 1 -- 172.21.15.37:0/3473287818 >> 172.21.15.37:0/3473287818 conn(0x7f1ca406ac90 msgr2=0x7f1ca406b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:55.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1cab018640 1 -- 172.21.15.37:0/3473287818 shutdown_connections 2024-04-25T08:56:55.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1cab018640 1 -- 172.21.15.37:0/3473287818 wait complete. 2024-04-25T08:56:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1cab018640 1 Processor -- start 2024-04-25T08:56:55.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.646+0000 7f1cab018640 1 -- start start 2024-04-25T08:56:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1cab018640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca4071560 0x7f1ca414af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1cab018640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca40cc560 0x7f1ca41371e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1cab018640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40ce720 0x7f1ca414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1cab018640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca415c9e0 con 0x7f1ca4071560 2024-04-25T08:56:55.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1cab018640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca415c860 con 0x7f1ca40cc560 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1cab018640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca41681d0 con 0x7f1ca40ce720 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca8db4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca4071560 0x7f1ca414af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca8db4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca4071560 0x7f1ca414af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39118/0 (socket says 172.21.15.37:39118) 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca8db4640 1 -- 172.21.15.37:0/3187543021 learned_addr learned my addr 172.21.15.37:0/3187543021 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca3fff640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca40cc560 0x7f1ca41371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca95b5640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40ce720 0x7f1ca414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca8db4640 1 -- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40ce720 msgr2=0x7f1ca414e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca8db4640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40ce720 0x7f1ca414e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca8db4640 1 -- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca40cc560 msgr2=0x7f1ca41371e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca8db4640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca40cc560 0x7f1ca41371e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca8db4640 1 -- 172.21.15.37:0/3187543021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4137720 con 0x7f1ca4071560 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca95b5640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40ce720 0x7f1ca414e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca8db4640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca4071560 0x7f1ca414af20 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e930 tx=0x7f1c9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca17fa640 1 -- 172.21.15.37:0/3187543021 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94017070 con 0x7f1ca4071560 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca17fa640 1 -- 172.21.15.37:0/3187543021 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9400ca00 con 0x7f1ca4071560 2024-04-25T08:56:55.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1cab018640 1 -- 172.21.15.37:0/3187543021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca414ea10 con 0x7f1ca4071560 2024-04-25T08:56:55.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca17fa640 1 -- 172.21.15.37:0/3187543021 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400cdb0 con 0x7f1ca4071560 2024-04-25T08:56:55.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1cab018640 1 -- 172.21.15.37:0/3187543021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca414efd0 con 0x7f1ca4071560 2024-04-25T08:56:55.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1cab018640 1 -- 172.21.15.37:0/3187543021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca404efa0 con 0x7f1ca4071560 2024-04-25T08:56:55.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca17fa640 1 -- 172.21.15.37:0/3187543021 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c9401a070 con 0x7f1ca4071560 2024-04-25T08:56:55.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca17fa640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066f60 0x7f1c90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca17fa640 1 -- 172.21.15.37:0/3187543021 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c940852a0 con 0x7f1ca4071560 2024-04-25T08:56:55.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.650+0000 7f1ca3fff640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066f60 0x7f1c90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.654+0000 7f1ca3fff640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066f60 0x7f1c90069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40b1560 tx=0x7f1c98009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.658+0000 7f1ca17fa640 1 -- 172.21.15.37:0/3187543021 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c94056140 con 0x7f1ca4071560 2024-04-25T08:56:55.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:56:55.741 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:56:55.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.814+0000 7f1cab018640 1 -- 172.21.15.37:0/3187543021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ca415ae90 con 0x7f1ca4071560 2024-04-25T08:56:55.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.814+0000 7f1ca17fa640 1 -- 172.21.15.37:0/3187543021 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16909 (secure 0 0 0) 0x7f1c94055a40 con 0x7f1ca4071560 2024-04-25T08:56:55.805 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:55.805 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":82,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:56:53.098938+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:56:49.737591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:56:50.436964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 -- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066f60 msgr2=0x7f1c90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066f60 0x7f1c90069420 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1ca40b1560 tx=0x7f1c98009040 comp rx=0 tx=0).stop 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 -- 172.21.15.37:0/3187543021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca4071560 msgr2=0x7f1ca414af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca4071560 0x7f1ca414af20 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400e930 tx=0x7f1c9400edf0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 -- 172.21.15.37:0/3187543021 shutdown_connections 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c90066f60 0x7f1c90069420 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ca40ce720 0x7f1ca414e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ca40cc560 0x7f1ca41371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 --2- 172.21.15.37:0/3187543021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ca4071560 0x7f1ca414af20 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 -- 172.21.15.37:0/3187543021 >> 172.21.15.37:0/3187543021 conn(0x7f1ca406ac90 msgr2=0x7f1ca4151ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 -- 172.21.15.37:0/3187543021 shutdown_connections 2024-04-25T08:56:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.818+0000 7f1cab018640 1 -- 172.21.15.37:0/3187543021 wait complete. 2024-04-25T08:56:55.823 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:56:55.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:56:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e66bd640 1 Processor -- start 2024-04-25T08:56:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e66bd640 1 -- start start 2024-04-25T08:56:55.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e66bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e0144ca0 0x7ff4e006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e66bd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff4e006ec60 0x7ff4e0142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e66bd640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e00ccc70 con 0x7ff4e00ce510 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e66bd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e00cc2b0 con 0x7ff4e006ec60 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e66bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e006e290 con 0x7ff4e0144ca0 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4dffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff4e00ce510 0x7ff4e00ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44756/0 (socket says 172.21.15.37:44756) 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4dffff640 1 -- 172.21.15.37:0/169734276 learned_addr learned my addr 172.21.15.37:0/169734276 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4df7fe640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e0144ca0 0x7ff4e006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e4c5a640 1 --2- 172.21.15.37:0/169734276 >> v2:172.21.15.203:3300/0 conn(0x7ff4e006ec60 0x7ff4e0142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2830724488 0 0) 0x7ff4e00ccc70 con 0x7ff4e00ce510 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4c8003530 con 0x7ff4e00ce510 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 498085817 0 0) 0x7ff4c8003530 con 0x7ff4e00ce510 2024-04-25T08:56:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e0144ca0 msgr2=0x7ff4e006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e0144ca0 0x7ff4e006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 >> v2:172.21.15.203:3300/0 conn(0x7ff4e006ec60 msgr2=0x7ff4e0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 --2- 172.21.15.37:0/169734276 >> v2:172.21.15.203:3300/0 conn(0x7ff4e006ec60 0x7ff4e0142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e006dc40 con 0x7ff4e00ce510 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7ff4d4001680 con 0x7ff4e00ce510 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e4c5a640 1 --2- 172.21.15.37:0/169734276 >> v2:172.21.15.203:3300/0 conn(0x7ff4e006ec60 0x7ff4e0142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 >> v1:172.21.15.37:6789/0 conn(0x7ff4e00ce510 legacy=0x7ff4e00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c8003a30 0x7ff4c8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e0144ca0 0x7ff4c8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c8004d50 0x7ff4c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e00cc2b0 con 0x7ff4c8004d50 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e006e290 con 0x7ff4c8003a30 2024-04-25T08:56:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4c800bae0 con 0x7ff4e0144ca0 2024-04-25T08:56:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e4c5a640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c8004d50 0x7ff4c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4dffff640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c8003a30 0x7ff4c8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4df7fe640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e0144ca0 0x7ff4c8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e4c5a640 1 -- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e0144ca0 msgr2=0x7ff4c8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e4c5a640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e0144ca0 0x7ff4c8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e4c5a640 1 -- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c8003a30 msgr2=0x7ff4c8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e4c5a640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c8003a30 0x7ff4c8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4e4c5a640 1 -- 172.21.15.37:0/169734276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4c8003530 con 0x7ff4c8004d50 2024-04-25T08:56:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.994+0000 7ff4dffff640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4c8003a30 0x7ff4c8003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:55.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e4c5a640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c8004d50 0x7ff4c80051d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff4e006f0c0 tx=0x7ff4d801cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef76c13efd3ae0fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8009c80 con 0x7ff4c8004d50 2024-04-25T08:56:55.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d80a5de0 con 0x7ff4c8004d50 2024-04-25T08:56:55.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4deffd640 1 -- 172.21.15.37:0/169734276 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8003ba0 con 0x7ff4c8004d50 2024-04-25T08:56:55.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 -- 172.21.15.37:0/169734276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c8004d50 msgr2=0x7ff4c80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 --2- 172.21.15.37:0/169734276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4c8004d50 0x7ff4c80051d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff4e006f0c0 tx=0x7ff4d801cee0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e4c5a640 1 -- 172.21.15.37:0/169734276 reap_dead start 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 -- 172.21.15.37:0/169734276 shutdown_connections 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 -- 172.21.15.37:0/169734276 >> 172.21.15.37:0/169734276 conn(0x7ff4e00b11a0 msgr2=0x7ff4e00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 -- 172.21.15.37:0/169734276 shutdown_connections 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 -- 172.21.15.37:0/169734276 wait complete. 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 Processor -- start 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 -- start start 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4e01ed400 0x7ff4e00b3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e00afc60 0x7ff4e00b00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4e01f34f0 0x7ff4e01f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e01451f0 con 0x7ff4e01f34f0 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e00cc900 con 0x7ff4e01ed400 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e66bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e0145370 con 0x7ff4e00afc60 2024-04-25T08:56:55.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e4c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4e01f34f0 0x7ff4e01f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e4c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4e01f34f0 0x7ff4e01f3970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39142/0 (socket says 172.21.15.37:39142) 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:55.998+0000 7ff4e4c5a640 1 -- 172.21.15.37:0/3229971736 learned_addr learned my addr 172.21.15.37:0/3229971736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4df7fe640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e00afc60 0x7ff4e00b00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4dffff640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4e01ed400 0x7ff4e00b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4e4c5a640 1 -- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e00afc60 msgr2=0x7ff4e00b00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4e4c5a640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e00afc60 0x7ff4e00b00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4e4c5a640 1 -- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4e01ed400 msgr2=0x7ff4e00b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4e4c5a640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4e01ed400 0x7ff4e00b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4e4c5a640 1 -- 172.21.15.37:0/3229971736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e01f43b0 con 0x7ff4e01f34f0 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4e4c5a640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4e01f34f0 0x7ff4e01f3970 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8098900 tx=0x7ff4d8019c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4dcff9640 1 -- 172.21.15.37:0/3229971736 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8009c80 con 0x7ff4e01f34f0 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4dcff9640 1 -- 172.21.15.37:0/3229971736 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d80a5be0 con 0x7ff4e01f34f0 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4e66bd640 1 -- 172.21.15.37:0/3229971736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e01f40d0 con 0x7ff4e01f34f0 2024-04-25T08:56:55.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4dcff9640 1 -- 172.21.15.37:0/3229971736 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d80095f0 con 0x7ff4e01f34f0 2024-04-25T08:56:55.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4e66bd640 1 -- 172.21.15.37:0/3229971736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e00b2920 con 0x7ff4e01f34f0 2024-04-25T08:56:55.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4e66bd640 1 -- 172.21.15.37:0/3229971736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e004efa0 con 0x7ff4e01f34f0 2024-04-25T08:56:55.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4dcff9640 1 -- 172.21.15.37:0/3229971736 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff4d8004410 con 0x7ff4e01f34f0 2024-04-25T08:56:55.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4dcff9640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4cc066f60 0x7ff4cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4dcff9640 1 -- 172.21.15.37:0/3229971736 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff4d811ef10 con 0x7ff4e01f34f0 2024-04-25T08:56:55.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4dffff640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4cc066f60 0x7ff4cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.002+0000 7ff4dffff640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4cc066f60 0x7ff4cc069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40017f0 tx=0x7ff4d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.006+0000 7ff4dcff9640 1 -- 172.21.15.37:0/3229971736 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff4d8016bf0 con 0x7ff4e01f34f0 2024-04-25T08:56:56.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.166+0000 7ff4e66bd640 1 -- 172.21.15.37:0/3229971736 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4e015b810 con 0x7ff4cc066f60 2024-04-25T08:56:56.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.182+0000 7ff4dcff9640 1 -- 172.21.15.37:0/3229971736 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252051 (secure 0 0 0) 0x7ff4e015b810 con 0x7ff4cc066f60 2024-04-25T08:56:56.176 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:56.178 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:56:56.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 -- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4cc066f60 msgr2=0x7ff4cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4cc066f60 0x7ff4cc069420 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40017f0 tx=0x7ff4d4009040 comp rx=0 tx=0).stop 2024-04-25T08:56:56.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 -- 172.21.15.37:0/3229971736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4e01f34f0 msgr2=0x7ff4e01f3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4e01f34f0 0x7ff4e01f3970 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8098900 tx=0x7ff4d8019c80 comp rx=0 tx=0).stop 2024-04-25T08:56:56.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 -- 172.21.15.37:0/3229971736 shutdown_connections 2024-04-25T08:56:56.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff4cc066f60 0x7ff4cc069420 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4e01f34f0 0x7ff4e01f3970 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff4e00afc60 0x7ff4e00b00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 --2- 172.21.15.37:0/3229971736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff4e01ed400 0x7ff4e00b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 -- 172.21.15.37:0/3229971736 >> 172.21.15.37:0/3229971736 conn(0x7ff4e00b11a0 msgr2=0x7ff4e00718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:56.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 -- 172.21.15.37:0/3229971736 shutdown_connections 2024-04-25T08:56:56.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.186+0000 7ff4e66bd640 1 -- 172.21.15.37:0/3229971736 wait complete. 2024-04-25T08:56:56.195 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-04-25T08:56:55.211965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258440,"kb_used_data":8980,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716280,"statfs":{"total":773094113280,"available":772829470720,"internally_reserved":0,"allocated":9195520,"data_stored":4830592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.146926"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108560+0000","last_change":"2024-04-25T08:56:52.120967+0000","last_active":"2024-04-25T08:56:53.108560+0000","last_peered":"2024-04-25T08:56:53.108560+0000","last_clean":"2024-04-25T08:56:53.108560+0000","last_became_active":"2024-04-25T08:56:52.120243+0000","last_became_peered":"2024-04-25T08:56:52.120243+0000","last_unstale":"2024-04-25T08:56:53.108560+0000","last_undegraded":"2024-04-25T08:56:53.108560+0000","last_fullsized":"2024-04-25T08:56:53.108560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:58.573901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105574+0000","last_change":"2024-04-25T08:56:52.123315+0000","last_active":"2024-04-25T08:56:53.105574+0000","last_peered":"2024-04-25T08:56:53.105574+0000","last_clean":"2024-04-25T08:56:53.105574+0000","last_became_active":"2024-04-25T08:56:52.123027+0000","last_became_peered":"2024-04-25T08:56:52.123027+0000","last_unstale":"2024-04-25T08:56:53.105574+0000","last_undegraded":"2024-04-25T08:56:53.105574+0000","last_fullsized":"2024-04-25T08:56:53.105574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:16.735000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172403+0000","last_change":"2024-04-25T08:56:52.111597+0000","last_active":"2024-04-25T08:56:53.172403+0000","last_peered":"2024-04-25T08:56:53.172403+0000","last_clean":"2024-04-25T08:56:53.172403+0000","last_became_active":"2024-04-25T08:56:52.111256+0000","last_became_peered":"2024-04-25T08:56:52.111256+0000","last_unstale":"2024-04-25T08:56:53.172403+0000","last_undegraded":"2024-04-25T08:56:53.172403+0000","last_fullsized":"2024-04-25T08:56:53.172403+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:28.290610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.107452+0000","last_change":"2024-04-25T08:56:52.100023+0000","last_active":"2024-04-25T08:56:52.107452+0000","last_peered":"2024-04-25T08:56:52.107452+0000","last_clean":"2024-04-25T08:56:52.107452+0000","last_became_active":"2024-04-25T08:56:52.099205+0000","last_became_peered":"2024-04-25T08:56:52.099205+0000","last_unstale":"2024-04-25T08:56:52.107452+0000","last_undegraded":"2024-04-25T08:56:52.107452+0000","last_fullsized":"2024-04-25T08:56:52.107452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:28.290610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116379+0000","last_change":"2024-04-25T08:56:52.106706+0000","last_active":"2024-04-25T08:56:52.116379+0000","last_peered":"2024-04-25T08:56:52.116379+0000","last_clean":"2024-04-25T08:56:52.116379+0000","last_became_active":"2024-04-25T08:56:52.106354+0000","last_became_peered":"2024-04-25T08:56:52.106354+0000","last_unstale":"2024-04-25T08:56:52.116379+0000","last_undegraded":"2024-04-25T08:56:52.116379+0000","last_fullsized":"2024-04-25T08:56:52.116379+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:15.208661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.127136+0000","last_change":"2024-04-25T08:56:52.117998+0000","last_active":"2024-04-25T08:56:52.127136+0000","last_peered":"2024-04-25T08:56:52.127136+0000","last_clean":"2024-04-25T08:56:52.127136+0000","last_became_active":"2024-04-25T08:56:52.117702+0000","last_became_peered":"2024-04-25T08:56:52.117702+0000","last_unstale":"2024-04-25T08:56:52.127136+0000","last_undegraded":"2024-04-25T08:56:52.127136+0000","last_fullsized":"2024-04-25T08:56:52.127136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.191980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469411+0000","last_change":"2024-04-25T08:56:52.116445+0000","last_active":"2024-04-25T08:56:53.469411+0000","last_peered":"2024-04-25T08:56:53.469411+0000","last_clean":"2024-04-25T08:56:53.469411+0000","last_became_active":"2024-04-25T08:56:52.114423+0000","last_became_peered":"2024-04-25T08:56:52.114423+0000","last_unstale":"2024-04-25T08:56:53.469411+0000","last_undegraded":"2024-04-25T08:56:53.469411+0000","last_fullsized":"2024-04-25T08:56:53.469411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:58.760488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116665+0000","last_change":"2024-04-25T08:56:52.106148+0000","last_active":"2024-04-25T08:56:52.116665+0000","last_peered":"2024-04-25T08:56:52.116665+0000","last_clean":"2024-04-25T08:56:52.116665+0000","last_became_active":"2024-04-25T08:56:52.105797+0000","last_became_peered":"2024-04-25T08:56:52.105797+0000","last_unstale":"2024-04-25T08:56:52.116665+0000","last_undegraded":"2024-04-25T08:56:52.116665+0000","last_fullsized":"2024-04-25T08:56:52.116665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:37.569468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.110138+0000","last_change":"2024-04-25T08:56:52.110253+0000","last_active":"2024-04-25T08:56:53.110138+0000","last_peered":"2024-04-25T08:56:53.110138+0000","last_clean":"2024-04-25T08:56:53.110138+0000","last_became_active":"2024-04-25T08:56:52.108998+0000","last_became_peered":"2024-04-25T08:56:52.108998+0000","last_unstale":"2024-04-25T08:56:53.110138+0000","last_undegraded":"2024-04-25T08:56:53.110138+0000","last_fullsized":"2024-04-25T08:56:53.110138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:59.878728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.110551+0000","last_change":"2024-04-25T08:56:52.108706+0000","last_active":"2024-04-25T08:56:52.110551+0000","last_peered":"2024-04-25T08:56:52.110551+0000","last_clean":"2024-04-25T08:56:52.110551+0000","last_became_active":"2024-04-25T08:56:52.108243+0000","last_became_peered":"2024-04-25T08:56:52.108243+0000","last_unstale":"2024-04-25T08:56:52.110551+0000","last_undegraded":"2024-04-25T08:56:52.110551+0000","last_fullsized":"2024-04-25T08:56:52.110551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:52.935915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105259+0000","last_change":"2024-04-25T08:56:52.128945+0000","last_active":"2024-04-25T08:56:53.105259+0000","last_peered":"2024-04-25T08:56:53.105259+0000","last_clean":"2024-04-25T08:56:53.105259+0000","last_became_active":"2024-04-25T08:56:52.128666+0000","last_became_peered":"2024-04-25T08:56:52.128666+0000","last_unstale":"2024-04-25T08:56:53.105259+0000","last_undegraded":"2024-04-25T08:56:53.105259+0000","last_fullsized":"2024-04-25T08:56:53.105259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:42.991363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.130660+0000","last_change":"2024-04-25T08:56:52.123450+0000","last_active":"2024-04-25T08:56:52.130660+0000","last_peered":"2024-04-25T08:56:52.130660+0000","last_clean":"2024-04-25T08:56:52.130660+0000","last_became_active":"2024-04-25T08:56:52.122737+0000","last_became_peered":"2024-04-25T08:56:52.122737+0000","last_unstale":"2024-04-25T08:56:52.130660+0000","last_undegraded":"2024-04-25T08:56:52.130660+0000","last_fullsized":"2024-04-25T08:56:52.130660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:48.492455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.122076+0000","last_change":"2024-04-25T08:56:52.114022+0000","last_active":"2024-04-25T08:56:52.122076+0000","last_peered":"2024-04-25T08:56:52.122076+0000","last_clean":"2024-04-25T08:56:52.122076+0000","last_became_active":"2024-04-25T08:56:52.113222+0000","last_became_peered":"2024-04-25T08:56:52.113222+0000","last_unstale":"2024-04-25T08:56:52.122076+0000","last_undegraded":"2024-04-25T08:56:52.122076+0000","last_fullsized":"2024-04-25T08:56:52.122076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:10.735890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.121788+0000","last_change":"2024-04-25T08:56:52.115122+0000","last_active":"2024-04-25T08:56:52.121788+0000","last_peered":"2024-04-25T08:56:52.121788+0000","last_clean":"2024-04-25T08:56:52.121788+0000","last_became_active":"2024-04-25T08:56:52.114811+0000","last_became_peered":"2024-04-25T08:56:52.114811+0000","last_unstale":"2024-04-25T08:56:52.121788+0000","last_undegraded":"2024-04-25T08:56:52.121788+0000","last_fullsized":"2024-04-25T08:56:52.121788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:40.610515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.104723+0000","last_change":"2024-04-25T08:56:52.119694+0000","last_active":"2024-04-25T08:56:53.104723+0000","last_peered":"2024-04-25T08:56:53.104723+0000","last_clean":"2024-04-25T08:56:53.104723+0000","last_became_active":"2024-04-25T08:56:52.119387+0000","last_became_peered":"2024-04-25T08:56:52.119387+0000","last_unstale":"2024-04-25T08:56:53.104723+0000","last_undegraded":"2024-04-25T08:56:53.104723+0000","last_fullsized":"2024-04-25T08:56:53.104723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:44.138372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108027+0000","last_change":"2024-04-25T08:56:52.109950+0000","last_active":"2024-04-25T08:56:53.108027+0000","last_peered":"2024-04-25T08:56:53.108027+0000","last_clean":"2024-04-25T08:56:53.108027+0000","last_became_active":"2024-04-25T08:56:52.109259+0000","last_became_peered":"2024-04-25T08:56:52.109259+0000","last_unstale":"2024-04-25T08:56:53.108027+0000","last_undegraded":"2024-04-25T08:56:53.108027+0000","last_fullsized":"2024-04-25T08:56:53.108027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:26.865946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116347+0000","last_change":"2024-04-25T08:56:52.105179+0000","last_active":"2024-04-25T08:56:52.116347+0000","last_peered":"2024-04-25T08:56:52.116347+0000","last_clean":"2024-04-25T08:56:52.116347+0000","last_became_active":"2024-04-25T08:56:52.104310+0000","last_became_peered":"2024-04-25T08:56:52.104310+0000","last_unstale":"2024-04-25T08:56:52.116347+0000","last_undegraded":"2024-04-25T08:56:52.116347+0000","last_fullsized":"2024-04-25T08:56:52.116347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:56.251550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172549+0000","last_change":"2024-04-25T08:56:52.099522+0000","last_active":"2024-04-25T08:56:53.172549+0000","last_peered":"2024-04-25T08:56:53.172549+0000","last_clean":"2024-04-25T08:56:53.172549+0000","last_became_active":"2024-04-25T08:56:52.098507+0000","last_became_peered":"2024-04-25T08:56:52.098507+0000","last_unstale":"2024-04-25T08:56:53.172549+0000","last_undegraded":"2024-04-25T08:56:53.172549+0000","last_fullsized":"2024-04-25T08:56:53.172549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:56.009236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.115898+0000","last_change":"2024-04-25T08:56:52.105847+0000","last_active":"2024-04-25T08:56:52.115898+0000","last_peered":"2024-04-25T08:56:52.115898+0000","last_clean":"2024-04-25T08:56:52.115898+0000","last_became_active":"2024-04-25T08:56:52.105159+0000","last_became_peered":"2024-04-25T08:56:52.105159+0000","last_unstale":"2024-04-25T08:56:52.115898+0000","last_undegraded":"2024-04-25T08:56:52.115898+0000","last_fullsized":"2024-04-25T08:56:52.115898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:15.561525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116803+0000","last_change":"2024-04-25T08:56:52.109584+0000","last_active":"2024-04-25T08:56:52.116803+0000","last_peered":"2024-04-25T08:56:52.116803+0000","last_clean":"2024-04-25T08:56:52.116803+0000","last_became_active":"2024-04-25T08:56:52.109228+0000","last_became_peered":"2024-04-25T08:56:52.109228+0000","last_unstale":"2024-04-25T08:56:52.116803+0000","last_undegraded":"2024-04-25T08:56:52.116803+0000","last_fullsized":"2024-04-25T08:56:52.116803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:36.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.109808+0000","last_change":"2024-04-25T08:56:52.117308+0000","last_active":"2024-04-25T08:56:53.109808+0000","last_peered":"2024-04-25T08:56:53.109808+0000","last_clean":"2024-04-25T08:56:53.109808+0000","last_became_active":"2024-04-25T08:56:52.116500+0000","last_became_peered":"2024-04-25T08:56:52.116500+0000","last_unstale":"2024-04-25T08:56:53.109808+0000","last_undegraded":"2024-04-25T08:56:53.109808+0000","last_fullsized":"2024-04-25T08:56:53.109808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.669375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.122029+0000","last_change":"2024-04-25T08:56:52.113612+0000","last_active":"2024-04-25T08:56:52.122029+0000","last_peered":"2024-04-25T08:56:52.122029+0000","last_clean":"2024-04-25T08:56:52.122029+0000","last_became_active":"2024-04-25T08:56:52.113333+0000","last_became_peered":"2024-04-25T08:56:52.113333+0000","last_unstale":"2024-04-25T08:56:52.122029+0000","last_undegraded":"2024-04-25T08:56:52.122029+0000","last_fullsized":"2024-04-25T08:56:52.122029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:38.584586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.109432+0000","last_change":"2024-04-25T08:56:52.098022+0000","last_active":"2024-04-25T08:56:53.109432+0000","last_peered":"2024-04-25T08:56:53.109432+0000","last_clean":"2024-04-25T08:56:53.109432+0000","last_became_active":"2024-04-25T08:56:52.097587+0000","last_became_peered":"2024-04-25T08:56:52.097587+0000","last_unstale":"2024-04-25T08:56:53.109432+0000","last_undegraded":"2024-04-25T08:56:53.109432+0000","last_fullsized":"2024-04-25T08:56:53.109432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:14.768231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105490+0000","last_change":"2024-04-25T08:56:52.125229+0000","last_active":"2024-04-25T08:56:53.105490+0000","last_peered":"2024-04-25T08:56:53.105490+0000","last_clean":"2024-04-25T08:56:53.105490+0000","last_became_active":"2024-04-25T08:56:52.122239+0000","last_became_peered":"2024-04-25T08:56:52.122239+0000","last_unstale":"2024-04-25T08:56:53.105490+0000","last_undegraded":"2024-04-25T08:56:53.105490+0000","last_fullsized":"2024-04-25T08:56:53.105490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:40.610515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105021+0000","last_change":"2024-04-25T08:56:52.122510+0000","last_active":"2024-04-25T08:56:53.105021+0000","last_peered":"2024-04-25T08:56:53.105021+0000","last_clean":"2024-04-25T08:56:53.105021+0000","last_became_active":"2024-04-25T08:56:52.122181+0000","last_became_peered":"2024-04-25T08:56:52.122181+0000","last_unstale":"2024-04-25T08:56:53.105021+0000","last_undegraded":"2024-04-25T08:56:53.105021+0000","last_fullsized":"2024-04-25T08:56:53.105021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:50.067483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.130474+0000","last_change":"2024-04-25T08:56:52.123288+0000","last_active":"2024-04-25T08:56:52.130474+0000","last_peered":"2024-04-25T08:56:52.130474+0000","last_clean":"2024-04-25T08:56:52.130474+0000","last_became_active":"2024-04-25T08:56:52.122607+0000","last_became_peered":"2024-04-25T08:56:52.122607+0000","last_unstale":"2024-04-25T08:56:52.130474+0000","last_undegraded":"2024-04-25T08:56:52.130474+0000","last_fullsized":"2024-04-25T08:56:52.130474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:09.821880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.109045+0000","last_change":"2024-04-25T08:56:52.106104+0000","last_active":"2024-04-25T08:56:53.109045+0000","last_peered":"2024-04-25T08:56:53.109045+0000","last_clean":"2024-04-25T08:56:53.109045+0000","last_became_active":"2024-04-25T08:56:52.105631+0000","last_became_peered":"2024-04-25T08:56:52.105631+0000","last_unstale":"2024-04-25T08:56:53.109045+0000","last_undegraded":"2024-04-25T08:56:53.109045+0000","last_fullsized":"2024-04-25T08:56:53.109045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.636840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116832+0000","last_change":"2024-04-25T08:56:52.110246+0000","last_active":"2024-04-25T08:56:52.116832+0000","last_peered":"2024-04-25T08:56:52.116832+0000","last_clean":"2024-04-25T08:56:52.116832+0000","last_became_active":"2024-04-25T08:56:52.109951+0000","last_became_peered":"2024-04-25T08:56:52.109951+0000","last_unstale":"2024-04-25T08:56:52.116832+0000","last_undegraded":"2024-04-25T08:56:52.116832+0000","last_fullsized":"2024-04-25T08:56:52.116832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:56.943561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.104477+0000","last_change":"2024-04-25T08:56:52.112100+0000","last_active":"2024-04-25T08:56:53.104477+0000","last_peered":"2024-04-25T08:56:53.104477+0000","last_clean":"2024-04-25T08:56:53.104477+0000","last_became_active":"2024-04-25T08:56:52.111703+0000","last_became_peered":"2024-04-25T08:56:52.111703+0000","last_unstale":"2024-04-25T08:56:53.104477+0000","last_undegraded":"2024-04-25T08:56:53.104477+0000","last_fullsized":"2024-04-25T08:56:53.104477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:52.090957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.107554+0000","last_change":"2024-04-25T08:56:52.115343+0000","last_active":"2024-04-25T08:56:53.107554+0000","last_peered":"2024-04-25T08:56:53.107554+0000","last_clean":"2024-04-25T08:56:53.107554+0000","last_became_active":"2024-04-25T08:56:52.114598+0000","last_became_peered":"2024-04-25T08:56:52.114598+0000","last_unstale":"2024-04-25T08:56:53.107554+0000","last_undegraded":"2024-04-25T08:56:53.107554+0000","last_fullsized":"2024-04-25T08:56:53.107554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:37.302261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.171853+0000","last_change":"2024-04-25T08:56:52.105089+0000","last_active":"2024-04-25T08:56:53.171853+0000","last_peered":"2024-04-25T08:56:53.171853+0000","last_clean":"2024-04-25T08:56:53.171853+0000","last_became_active":"2024-04-25T08:56:52.104491+0000","last_became_peered":"2024-04-25T08:56:52.104491+0000","last_unstale":"2024-04-25T08:56:53.171853+0000","last_undegraded":"2024-04-25T08:56:53.171853+0000","last_fullsized":"2024-04-25T08:56:53.171853+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:27.589864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.107616+0000","last_change":"2024-04-25T08:56:52.121158+0000","last_active":"2024-04-25T08:56:53.107616+0000","last_peered":"2024-04-25T08:56:53.107616+0000","last_clean":"2024-04-25T08:56:53.107616+0000","last_became_active":"2024-04-25T08:56:52.120433+0000","last_became_peered":"2024-04-25T08:56:52.120433+0000","last_unstale":"2024-04-25T08:56:53.107616+0000","last_undegraded":"2024-04-25T08:56:53.107616+0000","last_fullsized":"2024-04-25T08:56:53.107616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:39.844113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.171576+0000","last_change":"2024-04-25T08:56:52.099148+0000","last_active":"2024-04-25T08:56:53.171576+0000","last_peered":"2024-04-25T08:56:53.171576+0000","last_clean":"2024-04-25T08:56:53.171576+0000","last_became_active":"2024-04-25T08:56:52.098367+0000","last_became_peered":"2024-04-25T08:56:52.098367+0000","last_unstale":"2024-04-25T08:56:53.171576+0000","last_undegraded":"2024-04-25T08:56:53.171576+0000","last_fullsized":"2024-04-25T08:56:53.171576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:59.878728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.133839+0000","last_change":"2024-04-25T08:56:51.107533+0000","last_active":"2024-04-25T08:56:52.133839+0000","last_peered":"2024-04-25T08:56:52.133839+0000","last_clean":"2024-04-25T08:56:52.133839+0000","last_became_active":"2024-04-25T08:56:51.101692+0000","last_became_peered":"2024-04-25T08:56:51.101692+0000","last_unstale":"2024-04-25T08:56:52.133839+0000","last_undegraded":"2024-04-25T08:56:52.133839+0000","last_fullsized":"2024-04-25T08:56:52.133839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:55.594310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.104328+0000","last_change":"2024-04-25T08:56:52.131959+0000","last_active":"2024-04-25T08:56:53.104328+0000","last_peered":"2024-04-25T08:56:53.104328+0000","last_clean":"2024-04-25T08:56:53.104328+0000","last_became_active":"2024-04-25T08:56:52.131754+0000","last_became_peered":"2024-04-25T08:56:52.131754+0000","last_unstale":"2024-04-25T08:56:53.104328+0000","last_undegraded":"2024-04-25T08:56:53.104328+0000","last_fullsized":"2024-04-25T08:56:53.104328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:10.961781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.107987+0000","last_change":"2024-04-25T08:56:51.119798+0000","last_active":"2024-04-25T08:56:53.107987+0000","last_peered":"2024-04-25T08:56:53.107987+0000","last_clean":"2024-04-25T08:56:53.107987+0000","last_became_active":"2024-04-25T08:56:51.119016+0000","last_became_peered":"2024-04-25T08:56:51.119016+0000","last_unstale":"2024-04-25T08:56:53.107987+0000","last_undegraded":"2024-04-25T08:56:53.107987+0000","last_fullsized":"2024-04-25T08:56:53.107987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:27.279583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.109687+0000","last_change":"2024-04-25T08:56:52.112616+0000","last_active":"2024-04-25T08:56:53.109687+0000","last_peered":"2024-04-25T08:56:53.109687+0000","last_clean":"2024-04-25T08:56:53.109687+0000","last_became_active":"2024-04-25T08:56:52.111863+0000","last_became_peered":"2024-04-25T08:56:52.111863+0000","last_unstale":"2024-04-25T08:56:53.109687+0000","last_undegraded":"2024-04-25T08:56:53.109687+0000","last_fullsized":"2024-04-25T08:56:53.109687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:36.553929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108036+0000","last_change":"2024-04-25T08:56:51.120584+0000","last_active":"2024-04-25T08:56:53.108036+0000","last_peered":"2024-04-25T08:56:53.108036+0000","last_clean":"2024-04-25T08:56:53.108036+0000","last_became_active":"2024-04-25T08:56:51.115950+0000","last_became_peered":"2024-04-25T08:56:51.115950+0000","last_unstale":"2024-04-25T08:56:53.108036+0000","last_undegraded":"2024-04-25T08:56:53.108036+0000","last_fullsized":"2024-04-25T08:56:53.108036+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:57.749461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.110120+0000","last_change":"2024-04-25T08:56:52.097648+0000","last_active":"2024-04-25T08:56:53.110120+0000","last_peered":"2024-04-25T08:56:53.110120+0000","last_clean":"2024-04-25T08:56:53.110120+0000","last_became_active":"2024-04-25T08:56:52.096860+0000","last_became_peered":"2024-04-25T08:56:52.096860+0000","last_unstale":"2024-04-25T08:56:53.110120+0000","last_undegraded":"2024-04-25T08:56:53.110120+0000","last_fullsized":"2024-04-25T08:56:53.110120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:37.523964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469641+0000","last_change":"2024-04-25T08:56:51.116983+0000","last_active":"2024-04-25T08:56:53.469641+0000","last_peered":"2024-04-25T08:56:53.469641+0000","last_clean":"2024-04-25T08:56:53.469641+0000","last_became_active":"2024-04-25T08:56:51.116329+0000","last_became_peered":"2024-04-25T08:56:51.116329+0000","last_unstale":"2024-04-25T08:56:53.469641+0000","last_undegraded":"2024-04-25T08:56:53.469641+0000","last_fullsized":"2024-04-25T08:56:53.469641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:14.662323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.104762+0000","last_change":"2024-04-25T08:56:52.117640+0000","last_active":"2024-04-25T08:56:53.104762+0000","last_peered":"2024-04-25T08:56:53.104762+0000","last_clean":"2024-04-25T08:56:53.104762+0000","last_became_active":"2024-04-25T08:56:52.117336+0000","last_became_peered":"2024-04-25T08:56:52.117336+0000","last_unstale":"2024-04-25T08:56:53.104762+0000","last_undegraded":"2024-04-25T08:56:53.104762+0000","last_fullsized":"2024-04-25T08:56:53.104762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:45.559340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.110104+0000","last_change":"2024-04-25T08:56:51.101068+0000","last_active":"2024-04-25T08:56:53.110104+0000","last_peered":"2024-04-25T08:56:53.110104+0000","last_clean":"2024-04-25T08:56:53.110104+0000","last_became_active":"2024-04-25T08:56:51.100290+0000","last_became_peered":"2024-04-25T08:56:51.100290+0000","last_unstale":"2024-04-25T08:56:53.110104+0000","last_undegraded":"2024-04-25T08:56:53.110104+0000","last_fullsized":"2024-04-25T08:56:53.110104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:22.270137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105291+0000","last_change":"2024-04-25T08:56:52.128956+0000","last_active":"2024-04-25T08:56:53.105291+0000","last_peered":"2024-04-25T08:56:53.105291+0000","last_clean":"2024-04-25T08:56:53.105291+0000","last_became_active":"2024-04-25T08:56:52.128721+0000","last_became_peered":"2024-04-25T08:56:52.128721+0000","last_unstale":"2024-04-25T08:56:53.105291+0000","last_undegraded":"2024-04-25T08:56:53.105291+0000","last_fullsized":"2024-04-25T08:56:53.105291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:39.954646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.133631+0000","last_change":"2024-04-25T08:56:51.107942+0000","last_active":"2024-04-25T08:56:52.133631+0000","last_peered":"2024-04-25T08:56:52.133631+0000","last_clean":"2024-04-25T08:56:52.133631+0000","last_became_active":"2024-04-25T08:56:51.107154+0000","last_became_peered":"2024-04-25T08:56:51.107154+0000","last_unstale":"2024-04-25T08:56:52.133631+0000","last_undegraded":"2024-04-25T08:56:52.133631+0000","last_fullsized":"2024-04-25T08:56:52.133631+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:36.512937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.110429+0000","last_change":"2024-04-25T08:56:52.110931+0000","last_active":"2024-04-25T08:56:53.110429+0000","last_peered":"2024-04-25T08:56:53.110429+0000","last_clean":"2024-04-25T08:56:53.110429+0000","last_became_active":"2024-04-25T08:56:52.110242+0000","last_became_peered":"2024-04-25T08:56:52.110242+0000","last_unstale":"2024-04-25T08:56:53.110429+0000","last_undegraded":"2024-04-25T08:56:53.110429+0000","last_fullsized":"2024-04-25T08:56:53.110429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:27.589864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105838+0000","last_change":"2024-04-25T08:56:51.119436+0000","last_active":"2024-04-25T08:56:53.105838+0000","last_peered":"2024-04-25T08:56:53.105838+0000","last_clean":"2024-04-25T08:56:53.105838+0000","last_became_active":"2024-04-25T08:56:51.119122+0000","last_became_peered":"2024-04-25T08:56:51.119122+0000","last_unstale":"2024-04-25T08:56:53.105838+0000","last_undegraded":"2024-04-25T08:56:53.105838+0000","last_fullsized":"2024-04-25T08:56:53.105838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:36.291234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469987+0000","last_change":"2024-04-25T08:56:52.107107+0000","last_active":"2024-04-25T08:56:53.469987+0000","last_peered":"2024-04-25T08:56:53.469987+0000","last_clean":"2024-04-25T08:56:53.469987+0000","last_became_active":"2024-04-25T08:56:52.103569+0000","last_became_peered":"2024-04-25T08:56:52.103569+0000","last_unstale":"2024-04-25T08:56:53.469987+0000","last_undegraded":"2024-04-25T08:56:53.469987+0000","last_fullsized":"2024-04-25T08:56:53.469987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:52.935915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.087273+0000","last_change":"2024-04-25T08:56:51.100730+0000","last_active":"2024-04-25T08:56:52.087273+0000","last_peered":"2024-04-25T08:56:52.087273+0000","last_clean":"2024-04-25T08:56:52.087273+0000","last_became_active":"2024-04-25T08:56:51.100326+0000","last_became_peered":"2024-04-25T08:56:51.100326+0000","last_unstale":"2024-04-25T08:56:52.087273+0000","last_undegraded":"2024-04-25T08:56:52.087273+0000","last_fullsized":"2024-04-25T08:56:52.087273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:28.323423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105237+0000","last_change":"2024-04-25T08:56:52.133708+0000","last_active":"2024-04-25T08:56:53.105237+0000","last_peered":"2024-04-25T08:56:53.105237+0000","last_clean":"2024-04-25T08:56:53.105237+0000","last_became_active":"2024-04-25T08:56:52.133466+0000","last_became_peered":"2024-04-25T08:56:52.133466+0000","last_unstale":"2024-04-25T08:56:53.105237+0000","last_undegraded":"2024-04-25T08:56:53.105237+0000","last_fullsized":"2024-04-25T08:56:53.105237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:02.899860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108475+0000","last_change":"2024-04-25T08:56:51.118078+0000","last_active":"2024-04-25T08:56:53.108475+0000","last_peered":"2024-04-25T08:56:53.108475+0000","last_clean":"2024-04-25T08:56:53.108475+0000","last_became_active":"2024-04-25T08:56:51.115969+0000","last_became_peered":"2024-04-25T08:56:51.115969+0000","last_unstale":"2024-04-25T08:56:53.108475+0000","last_undegraded":"2024-04-25T08:56:53.108475+0000","last_fullsized":"2024-04-25T08:56:53.108475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:51.924888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.107470+0000","last_change":"2024-04-25T08:56:52.100662+0000","last_active":"2024-04-25T08:56:52.107470+0000","last_peered":"2024-04-25T08:56:52.107470+0000","last_clean":"2024-04-25T08:56:52.107470+0000","last_became_active":"2024-04-25T08:56:52.099084+0000","last_became_peered":"2024-04-25T08:56:52.099084+0000","last_unstale":"2024-04-25T08:56:52.107470+0000","last_undegraded":"2024-04-25T08:56:52.107470+0000","last_fullsized":"2024-04-25T08:56:52.107470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:43.107847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.080868+0000","last_change":"2024-04-25T08:56:51.116625+0000","last_active":"2024-04-25T08:56:52.080868+0000","last_peered":"2024-04-25T08:56:52.080868+0000","last_clean":"2024-04-25T08:56:52.080868+0000","last_became_active":"2024-04-25T08:56:51.116012+0000","last_became_peered":"2024-04-25T08:56:51.116012+0000","last_unstale":"2024-04-25T08:56:52.080868+0000","last_undegraded":"2024-04-25T08:56:52.080868+0000","last_fullsized":"2024-04-25T08:56:52.080868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:26.578837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116211+0000","last_change":"2024-04-25T08:56:52.106550+0000","last_active":"2024-04-25T08:56:52.116211+0000","last_peered":"2024-04-25T08:56:52.116211+0000","last_clean":"2024-04-25T08:56:52.116211+0000","last_became_active":"2024-04-25T08:56:52.105973+0000","last_became_peered":"2024-04-25T08:56:52.105973+0000","last_unstale":"2024-04-25T08:56:52.116211+0000","last_undegraded":"2024-04-25T08:56:52.116211+0000","last_fullsized":"2024-04-25T08:56:52.116211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:32.910438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.133509+0000","last_change":"2024-04-25T08:56:51.095525+0000","last_active":"2024-04-25T08:56:52.133509+0000","last_peered":"2024-04-25T08:56:52.133509+0000","last_clean":"2024-04-25T08:56:52.133509+0000","last_became_active":"2024-04-25T08:56:51.095223+0000","last_became_peered":"2024-04-25T08:56:51.095223+0000","last_unstale":"2024-04-25T08:56:52.133509+0000","last_undegraded":"2024-04-25T08:56:52.133509+0000","last_fullsized":"2024-04-25T08:56:52.133509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:39.853485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172275+0000","last_change":"2024-04-25T08:56:52.111648+0000","last_active":"2024-04-25T08:56:53.172275+0000","last_peered":"2024-04-25T08:56:53.172275+0000","last_clean":"2024-04-25T08:56:53.172275+0000","last_became_active":"2024-04-25T08:56:52.111251+0000","last_became_peered":"2024-04-25T08:56:52.111251+0000","last_unstale":"2024-04-25T08:56:53.172275+0000","last_undegraded":"2024-04-25T08:56:53.172275+0000","last_fullsized":"2024-04-25T08:56:53.172275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:58.760488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469717+0000","last_change":"2024-04-25T08:56:51.096512+0000","last_active":"2024-04-25T08:56:53.469717+0000","last_peered":"2024-04-25T08:56:53.469717+0000","last_clean":"2024-04-25T08:56:53.469717+0000","last_became_active":"2024-04-25T08:56:51.095725+0000","last_became_peered":"2024-04-25T08:56:51.095725+0000","last_unstale":"2024-04-25T08:56:53.469717+0000","last_undegraded":"2024-04-25T08:56:53.469717+0000","last_fullsized":"2024-04-25T08:56:53.469717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:07.001522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.107510+0000","last_change":"2024-04-25T08:56:52.103550+0000","last_active":"2024-04-25T08:56:52.107510+0000","last_peered":"2024-04-25T08:56:52.107510+0000","last_clean":"2024-04-25T08:56:52.107510+0000","last_became_active":"2024-04-25T08:56:52.102576+0000","last_became_peered":"2024-04-25T08:56:52.102576+0000","last_unstale":"2024-04-25T08:56:52.107510+0000","last_undegraded":"2024-04-25T08:56:52.107510+0000","last_fullsized":"2024-04-25T08:56:52.107510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:56.009236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.138514+0000","last_change":"2024-04-25T08:56:51.108723+0000","last_active":"2024-04-25T08:56:52.138514+0000","last_peered":"2024-04-25T08:56:52.138514+0000","last_clean":"2024-04-25T08:56:52.138514+0000","last_became_active":"2024-04-25T08:56:51.107894+0000","last_became_peered":"2024-04-25T08:56:51.107894+0000","last_unstale":"2024-04-25T08:56:52.138514+0000","last_undegraded":"2024-04-25T08:56:52.138514+0000","last_fullsized":"2024-04-25T08:56:52.138514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:57.749461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.470265+0000","last_change":"2024-04-25T08:56:52.102592+0000","last_active":"2024-04-25T08:56:53.470265+0000","last_peered":"2024-04-25T08:56:53.470265+0000","last_clean":"2024-04-25T08:56:53.470265+0000","last_became_active":"2024-04-25T08:56:52.102050+0000","last_became_peered":"2024-04-25T08:56:52.102050+0000","last_unstale":"2024-04-25T08:56:53.470265+0000","last_undegraded":"2024-04-25T08:56:53.470265+0000","last_fullsized":"2024-04-25T08:56:53.470265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:04.895571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108037+0000","last_change":"2024-04-25T08:56:51.116396+0000","last_active":"2024-04-25T08:56:53.108037+0000","last_peered":"2024-04-25T08:56:53.108037+0000","last_clean":"2024-04-25T08:56:53.108037+0000","last_became_active":"2024-04-25T08:56:51.115635+0000","last_became_peered":"2024-04-25T08:56:51.115635+0000","last_unstale":"2024-04-25T08:56:53.108037+0000","last_undegraded":"2024-04-25T08:56:53.108037+0000","last_fullsized":"2024-04-25T08:56:53.108037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:26.578837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.470202+0000","last_change":"2024-04-25T08:56:52.104020+0000","last_active":"2024-04-25T08:56:53.470202+0000","last_peered":"2024-04-25T08:56:53.470202+0000","last_clean":"2024-04-25T08:56:53.470202+0000","last_became_active":"2024-04-25T08:56:52.103266+0000","last_became_peered":"2024-04-25T08:56:52.103266+0000","last_unstale":"2024-04-25T08:56:53.470202+0000","last_undegraded":"2024-04-25T08:56:53.470202+0000","last_fullsized":"2024-04-25T08:56:53.470202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:48.492455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.094595+0000","last_change":"2024-04-25T08:56:51.108606+0000","last_active":"2024-04-25T08:56:52.094595+0000","last_peered":"2024-04-25T08:56:52.094595+0000","last_clean":"2024-04-25T08:56:52.094595+0000","last_became_active":"2024-04-25T08:56:51.107964+0000","last_became_peered":"2024-04-25T08:56:51.107964+0000","last_unstale":"2024-04-25T08:56:52.094595+0000","last_undegraded":"2024-04-25T08:56:52.094595+0000","last_fullsized":"2024-04-25T08:56:52.094595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:05.324319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.470105+0000","last_change":"2024-04-25T08:56:52.109009+0000","last_active":"2024-04-25T08:56:53.470105+0000","last_peered":"2024-04-25T08:56:53.470105+0000","last_clean":"2024-04-25T08:56:53.470105+0000","last_became_active":"2024-04-25T08:56:52.108260+0000","last_became_peered":"2024-04-25T08:56:52.108260+0000","last_unstale":"2024-04-25T08:56:53.470105+0000","last_undegraded":"2024-04-25T08:56:53.470105+0000","last_fullsized":"2024-04-25T08:56:53.470105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:09.821880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105528+0000","last_change":"2024-04-25T08:56:51.113838+0000","last_active":"2024-04-25T08:56:53.105528+0000","last_peered":"2024-04-25T08:56:53.105528+0000","last_clean":"2024-04-25T08:56:53.105528+0000","last_became_active":"2024-04-25T08:56:51.113500+0000","last_became_peered":"2024-04-25T08:56:51.113500+0000","last_unstale":"2024-04-25T08:56:53.105528+0000","last_undegraded":"2024-04-25T08:56:53.105528+0000","last_fullsized":"2024-04-25T08:56:53.105528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:25.854919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116756+0000","last_change":"2024-04-25T08:56:52.107089+0000","last_active":"2024-04-25T08:56:52.116756+0000","last_peered":"2024-04-25T08:56:52.116756+0000","last_clean":"2024-04-25T08:56:52.116756+0000","last_became_active":"2024-04-25T08:56:52.106667+0000","last_became_peered":"2024-04-25T08:56:52.106667+0000","last_unstale":"2024-04-25T08:56:52.116756+0000","last_undegraded":"2024-04-25T08:56:52.116756+0000","last_fullsized":"2024-04-25T08:56:52.116756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:10.961781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108685+0000","last_change":"2024-04-25T08:56:51.118338+0000","last_active":"2024-04-25T08:56:53.108685+0000","last_peered":"2024-04-25T08:56:53.108685+0000","last_clean":"2024-04-25T08:56:53.108685+0000","last_became_active":"2024-04-25T08:56:51.116647+0000","last_became_peered":"2024-04-25T08:56:51.116647+0000","last_unstale":"2024-04-25T08:56:53.108685+0000","last_undegraded":"2024-04-25T08:56:53.108685+0000","last_fullsized":"2024-04-25T08:56:53.108685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:08.810853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.127331+0000","last_change":"2024-04-25T08:56:52.120630+0000","last_active":"2024-04-25T08:56:52.127331+0000","last_peered":"2024-04-25T08:56:52.127331+0000","last_clean":"2024-04-25T08:56:52.127331+0000","last_became_active":"2024-04-25T08:56:52.120307+0000","last_became_peered":"2024-04-25T08:56:52.120307+0000","last_unstale":"2024-04-25T08:56:52.127331+0000","last_undegraded":"2024-04-25T08:56:52.127331+0000","last_fullsized":"2024-04-25T08:56:52.127331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:39.303259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.110710+0000","last_change":"2024-04-25T08:56:51.096606+0000","last_active":"2024-04-25T08:56:53.110710+0000","last_peered":"2024-04-25T08:56:53.110710+0000","last_clean":"2024-04-25T08:56:53.110710+0000","last_became_active":"2024-04-25T08:56:51.095790+0000","last_became_peered":"2024-04-25T08:56:51.095790+0000","last_unstale":"2024-04-25T08:56:53.110710+0000","last_undegraded":"2024-04-25T08:56:53.110710+0000","last_fullsized":"2024-04-25T08:56:53.110710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:38.672049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.127318+0000","last_change":"2024-04-25T08:56:52.117496+0000","last_active":"2024-04-25T08:56:52.127318+0000","last_peered":"2024-04-25T08:56:52.127318+0000","last_clean":"2024-04-25T08:56:52.127318+0000","last_became_active":"2024-04-25T08:56:52.117109+0000","last_became_peered":"2024-04-25T08:56:52.117109+0000","last_unstale":"2024-04-25T08:56:52.127318+0000","last_undegraded":"2024-04-25T08:56:52.127318+0000","last_fullsized":"2024-04-25T08:56:52.127318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:04.214004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.138477+0000","last_change":"2024-04-25T08:56:51.107381+0000","last_active":"2024-04-25T08:56:52.138477+0000","last_peered":"2024-04-25T08:56:52.138477+0000","last_clean":"2024-04-25T08:56:52.138477+0000","last_became_active":"2024-04-25T08:56:51.101514+0000","last_became_peered":"2024-04-25T08:56:51.101514+0000","last_unstale":"2024-04-25T08:56:52.138477+0000","last_undegraded":"2024-04-25T08:56:52.138477+0000","last_fullsized":"2024-04-25T08:56:52.138477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:28.713552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116716+0000","last_change":"2024-04-25T08:56:52.107852+0000","last_active":"2024-04-25T08:56:52.116716+0000","last_peered":"2024-04-25T08:56:52.116716+0000","last_clean":"2024-04-25T08:56:52.116716+0000","last_became_active":"2024-04-25T08:56:52.107501+0000","last_became_peered":"2024-04-25T08:56:52.107501+0000","last_unstale":"2024-04-25T08:56:52.116716+0000","last_undegraded":"2024-04-25T08:56:52.116716+0000","last_fullsized":"2024-04-25T08:56:52.116716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:52.090957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172143+0000","last_change":"2024-04-25T08:56:51.096926+0000","last_active":"2024-04-25T08:56:53.172143+0000","last_peered":"2024-04-25T08:56:53.172143+0000","last_clean":"2024-04-25T08:56:53.172143+0000","last_became_active":"2024-04-25T08:56:51.096509+0000","last_became_peered":"2024-04-25T08:56:51.096509+0000","last_unstale":"2024-04-25T08:56:53.172143+0000","last_undegraded":"2024-04-25T08:56:53.172143+0000","last_fullsized":"2024-04-25T08:56:53.172143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:50.344453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105916+0000","last_change":"2024-04-25T08:56:52.132092+0000","last_active":"2024-04-25T08:56:53.105916+0000","last_peered":"2024-04-25T08:56:53.105916+0000","last_clean":"2024-04-25T08:56:53.105916+0000","last_became_active":"2024-04-25T08:56:52.131800+0000","last_became_peered":"2024-04-25T08:56:52.131800+0000","last_unstale":"2024-04-25T08:56:53.105916+0000","last_undegraded":"2024-04-25T08:56:53.105916+0000","last_fullsized":"2024-04-25T08:56:53.105916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:22.710685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.130206+0000","last_change":"2024-04-25T08:56:51.092743+0000","last_active":"2024-04-25T08:56:52.130206+0000","last_peered":"2024-04-25T08:56:52.130206+0000","last_clean":"2024-04-25T08:56:52.130206+0000","last_became_active":"2024-04-25T08:56:51.092409+0000","last_became_peered":"2024-04-25T08:56:51.092409+0000","last_unstale":"2024-04-25T08:56:52.130206+0000","last_undegraded":"2024-04-25T08:56:52.130206+0000","last_fullsized":"2024-04-25T08:56:52.130206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:01.132439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105968+0000","last_change":"2024-04-25T08:56:52.122590+0000","last_active":"2024-04-25T08:56:53.105968+0000","last_peered":"2024-04-25T08:56:53.105968+0000","last_clean":"2024-04-25T08:56:53.105968+0000","last_became_active":"2024-04-25T08:56:52.118307+0000","last_became_peered":"2024-04-25T08:56:52.118307+0000","last_unstale":"2024-04-25T08:56:53.105968+0000","last_undegraded":"2024-04-25T08:56:53.105968+0000","last_fullsized":"2024-04-25T08:56:53.105968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:56.517596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.091013+0000","last_change":"2024-04-25T08:56:51.107076+0000","last_active":"2024-04-25T08:56:52.091013+0000","last_peered":"2024-04-25T08:56:52.091013+0000","last_clean":"2024-04-25T08:56:52.091013+0000","last_became_active":"2024-04-25T08:56:51.106255+0000","last_became_peered":"2024-04-25T08:56:51.106255+0000","last_unstale":"2024-04-25T08:56:52.091013+0000","last_undegraded":"2024-04-25T08:56:52.091013+0000","last_fullsized":"2024-04-25T08:56:52.091013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:27.458796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.470240+0000","last_change":"2024-04-25T08:56:52.102863+0000","last_active":"2024-04-25T08:56:53.470240+0000","last_peered":"2024-04-25T08:56:53.470240+0000","last_clean":"2024-04-25T08:56:53.470240+0000","last_became_active":"2024-04-25T08:56:52.101860+0000","last_became_peered":"2024-04-25T08:56:52.101860+0000","last_unstale":"2024-04-25T08:56:53.470240+0000","last_undegraded":"2024-04-25T08:56:53.470240+0000","last_fullsized":"2024-04-25T08:56:53.470240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:40.947555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.095601+0000","last_change":"2024-04-25T08:56:51.103941+0000","last_active":"2024-04-25T08:56:52.095601+0000","last_peered":"2024-04-25T08:56:52.095601+0000","last_clean":"2024-04-25T08:56:52.095601+0000","last_became_active":"2024-04-25T08:56:51.103666+0000","last_became_peered":"2024-04-25T08:56:51.103666+0000","last_unstale":"2024-04-25T08:56:52.095601+0000","last_undegraded":"2024-04-25T08:56:52.095601+0000","last_fullsized":"2024-04-25T08:56:52.095601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:45.435859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108930+0000","last_change":"2024-04-25T08:56:52.116135+0000","last_active":"2024-04-25T08:56:53.108930+0000","last_peered":"2024-04-25T08:56:53.108930+0000","last_clean":"2024-04-25T08:56:53.108930+0000","last_became_active":"2024-04-25T08:56:52.115350+0000","last_became_peered":"2024-04-25T08:56:52.115350+0000","last_unstale":"2024-04-25T08:56:53.108930+0000","last_undegraded":"2024-04-25T08:56:53.108930+0000","last_fullsized":"2024-04-25T08:56:53.108930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:39.598347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.134029+0000","last_change":"2024-04-25T08:56:51.096677+0000","last_active":"2024-04-25T08:56:52.134029+0000","last_peered":"2024-04-25T08:56:52.134029+0000","last_clean":"2024-04-25T08:56:52.134029+0000","last_became_active":"2024-04-25T08:56:51.095944+0000","last_became_peered":"2024-04-25T08:56:51.095944+0000","last_unstale":"2024-04-25T08:56:52.134029+0000","last_undegraded":"2024-04-25T08:56:52.134029+0000","last_fullsized":"2024-04-25T08:56:52.134029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:58.867701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469426+0000","last_change":"2024-04-25T08:56:52.117121+0000","last_active":"2024-04-25T08:56:53.469426+0000","last_peered":"2024-04-25T08:56:53.469426+0000","last_clean":"2024-04-25T08:56:53.469426+0000","last_became_active":"2024-04-25T08:56:52.115346+0000","last_became_peered":"2024-04-25T08:56:52.115346+0000","last_unstale":"2024-04-25T08:56:53.469426+0000","last_undegraded":"2024-04-25T08:56:53.469426+0000","last_fullsized":"2024-04-25T08:56:53.469426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:28.290610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.106063+0000","last_change":"2024-04-25T08:56:51.111445+0000","last_active":"2024-04-25T08:56:53.106063+0000","last_peered":"2024-04-25T08:56:53.106063+0000","last_clean":"2024-04-25T08:56:53.106063+0000","last_became_active":"2024-04-25T08:56:51.111060+0000","last_became_peered":"2024-04-25T08:56:51.111060+0000","last_unstale":"2024-04-25T08:56:53.106063+0000","last_undegraded":"2024-04-25T08:56:53.106063+0000","last_fullsized":"2024-04-25T08:56:53.106063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:38.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.109757+0000","last_change":"2024-04-25T08:56:52.112324+0000","last_active":"2024-04-25T08:56:53.109757+0000","last_peered":"2024-04-25T08:56:53.109757+0000","last_clean":"2024-04-25T08:56:53.109757+0000","last_became_active":"2024-04-25T08:56:52.110888+0000","last_became_peered":"2024-04-25T08:56:52.110888+0000","last_unstale":"2024-04-25T08:56:53.109757+0000","last_undegraded":"2024-04-25T08:56:53.109757+0000","last_fullsized":"2024-04-25T08:56:53.109757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:24.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108694+0000","last_change":"2024-04-25T08:56:51.118804+0000","last_active":"2024-04-25T08:56:53.108694+0000","last_peered":"2024-04-25T08:56:53.108694+0000","last_clean":"2024-04-25T08:56:53.108694+0000","last_became_active":"2024-04-25T08:56:51.116751+0000","last_became_peered":"2024-04-25T08:56:51.116751+0000","last_unstale":"2024-04-25T08:56:53.108694+0000","last_undegraded":"2024-04-25T08:56:53.108694+0000","last_fullsized":"2024-04-25T08:56:53.108694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:54.998209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.104647+0000","last_change":"2024-04-25T08:56:52.113845+0000","last_active":"2024-04-25T08:56:53.104647+0000","last_peered":"2024-04-25T08:56:53.104647+0000","last_clean":"2024-04-25T08:56:53.104647+0000","last_became_active":"2024-04-25T08:56:52.113464+0000","last_became_peered":"2024-04-25T08:56:52.113464+0000","last_unstale":"2024-04-25T08:56:53.104647+0000","last_undegraded":"2024-04-25T08:56:53.104647+0000","last_fullsized":"2024-04-25T08:56:53.104647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.191980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.107720+0000","last_change":"2024-04-25T08:56:51.116453+0000","last_active":"2024-04-25T08:56:53.107720+0000","last_peered":"2024-04-25T08:56:53.107720+0000","last_clean":"2024-04-25T08:56:53.107720+0000","last_became_active":"2024-04-25T08:56:51.115817+0000","last_became_peered":"2024-04-25T08:56:51.115817+0000","last_unstale":"2024-04-25T08:56:53.107720+0000","last_undegraded":"2024-04-25T08:56:53.107720+0000","last_fullsized":"2024-04-25T08:56:53.107720+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:42.096820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469547+0000","last_change":"2024-04-25T08:56:52.119509+0000","last_active":"2024-04-25T08:56:53.469547+0000","last_peered":"2024-04-25T08:56:53.469547+0000","last_clean":"2024-04-25T08:56:53.469547+0000","last_became_active":"2024-04-25T08:56:52.118846+0000","last_became_peered":"2024-04-25T08:56:52.118846+0000","last_unstale":"2024-04-25T08:56:53.469547+0000","last_undegraded":"2024-04-25T08:56:53.469547+0000","last_fullsized":"2024-04-25T08:56:53.469547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:43.107847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469427+0000","last_change":"2024-04-25T08:56:51.103526+0000","last_active":"2024-04-25T08:56:53.469427+0000","last_peered":"2024-04-25T08:56:53.469427+0000","last_clean":"2024-04-25T08:56:53.469427+0000","last_became_active":"2024-04-25T08:56:51.102732+0000","last_became_peered":"2024-04-25T08:56:51.102732+0000","last_unstale":"2024-04-25T08:56:53.469427+0000","last_undegraded":"2024-04-25T08:56:53.469427+0000","last_fullsized":"2024-04-25T08:56:53.469427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:50.344453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469874+0000","last_change":"2024-04-25T08:56:52.107818+0000","last_active":"2024-04-25T08:56:53.469874+0000","last_peered":"2024-04-25T08:56:53.469874+0000","last_clean":"2024-04-25T08:56:53.469874+0000","last_became_active":"2024-04-25T08:56:52.103932+0000","last_became_peered":"2024-04-25T08:56:52.103932+0000","last_unstale":"2024-04-25T08:56:53.469874+0000","last_undegraded":"2024-04-25T08:56:53.469874+0000","last_fullsized":"2024-04-25T08:56:53.469874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:56.009236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.087611+0000","last_change":"2024-04-25T08:56:51.108190+0000","last_active":"2024-04-25T08:56:52.087611+0000","last_peered":"2024-04-25T08:56:52.087611+0000","last_clean":"2024-04-25T08:56:52.087611+0000","last_became_active":"2024-04-25T08:56:51.107716+0000","last_became_peered":"2024-04-25T08:56:51.107716+0000","last_unstale":"2024-04-25T08:56:52.087611+0000","last_undegraded":"2024-04-25T08:56:52.087611+0000","last_fullsized":"2024-04-25T08:56:52.087611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:14.284634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172529+0000","last_change":"2024-04-25T08:56:52.110793+0000","last_active":"2024-04-25T08:56:53.172529+0000","last_peered":"2024-04-25T08:56:53.172529+0000","last_clean":"2024-04-25T08:56:53.172529+0000","last_became_active":"2024-04-25T08:56:52.110411+0000","last_became_peered":"2024-04-25T08:56:52.110411+0000","last_unstale":"2024-04-25T08:56:53.172529+0000","last_undegraded":"2024-04-25T08:56:53.172529+0000","last_fullsized":"2024-04-25T08:56:53.172529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:43.107847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469682+0000","last_change":"2024-04-25T08:56:51.105116+0000","last_active":"2024-04-25T08:56:53.469682+0000","last_peered":"2024-04-25T08:56:53.469682+0000","last_clean":"2024-04-25T08:56:53.469682+0000","last_became_active":"2024-04-25T08:56:51.104439+0000","last_became_peered":"2024-04-25T08:56:51.104439+0000","last_unstale":"2024-04-25T08:56:53.469682+0000","last_undegraded":"2024-04-25T08:56:53.469682+0000","last_fullsized":"2024-04-25T08:56:53.469682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:47.326064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172671+0000","last_change":"2024-04-25T08:56:52.101776+0000","last_active":"2024-04-25T08:56:53.172671+0000","last_peered":"2024-04-25T08:56:53.172671+0000","last_clean":"2024-04-25T08:56:53.172671+0000","last_became_active":"2024-04-25T08:56:52.101123+0000","last_became_peered":"2024-04-25T08:56:52.101123+0000","last_unstale":"2024-04-25T08:56:53.172671+0000","last_undegraded":"2024-04-25T08:56:53.172671+0000","last_fullsized":"2024-04-25T08:56:53.172671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:52.935915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.171474+0000","last_change":"2024-04-25T08:56:51.102221+0000","last_active":"2024-04-25T08:56:53.171474+0000","last_peered":"2024-04-25T08:56:53.171474+0000","last_clean":"2024-04-25T08:56:53.171474+0000","last_became_active":"2024-04-25T08:56:51.101938+0000","last_became_peered":"2024-04-25T08:56:51.101938+0000","last_unstale":"2024-04-25T08:56:53.171474+0000","last_undegraded":"2024-04-25T08:56:53.171474+0000","last_fullsized":"2024-04-25T08:56:53.171474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:14.662323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108428+0000","last_change":"2024-04-25T08:56:52.107464+0000","last_active":"2024-04-25T08:56:53.108428+0000","last_peered":"2024-04-25T08:56:53.108428+0000","last_clean":"2024-04-25T08:56:53.108428+0000","last_became_active":"2024-04-25T08:56:52.107112+0000","last_became_peered":"2024-04-25T08:56:52.107112+0000","last_unstale":"2024-04-25T08:56:53.108428+0000","last_undegraded":"2024-04-25T08:56:53.108428+0000","last_fullsized":"2024-04-25T08:56:53.108428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:49.015906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.092346+0000","last_change":"2024-04-25T08:56:51.104828+0000","last_active":"2024-04-25T08:56:52.092346+0000","last_peered":"2024-04-25T08:56:52.092346+0000","last_clean":"2024-04-25T08:56:52.092346+0000","last_became_active":"2024-04-25T08:56:51.104398+0000","last_became_peered":"2024-04-25T08:56:51.104398+0000","last_unstale":"2024-04-25T08:56:52.092346+0000","last_undegraded":"2024-04-25T08:56:52.092346+0000","last_fullsized":"2024-04-25T08:56:52.092346+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:25.941535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":286,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.107955+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:56:53.107955+0000","last_peered":"2024-04-25T08:56:53.107955+0000","last_clean":"2024-04-25T08:56:53.107955+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:56:53.107955+0000","last_undegraded":"2024-04-25T08:56:53.107955+0000","last_fullsized":"2024-04-25T08:56:53.107955+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1589248,"data_stored":1045183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1003520,"data_stored":456405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":454827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":456509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":455375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1003520,"data_stored":455535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":999424,"data_stored":454905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1597440,"data_stored":1051853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:56:56.200 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:56:56.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f6976045640 1 Processor -- start 2024-04-25T08:56:56.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f6976045640 1 -- start start 2024-04-25T08:56:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f6976045640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f697015ca90 0x7f69700d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f6976045640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697015f360 con 0x7f697006ac90 2024-04-25T08:56:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f6976045640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697015f640 con 0x7f69700cc130 2024-04-25T08:56:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f6976045640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697015ff70 con 0x7f697015ca90 2024-04-25T08:56:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f696f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f697006ac90 0x7f697006b070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44762/0 (socket says 172.21.15.37:44762) 2024-04-25T08:56:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f696effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f697015ca90 0x7f69700d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f696f7fe640 1 -- 172.21.15.37:0/4103102496 learned_addr learned my addr 172.21.15.37:0/4103102496 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f696effd640 1 -- 172.21.15.37:0/4103102496 >> v1:172.21.15.203:6789/0 conn(0x7f69700cc130 legacy=0x7f69700cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f696effd640 1 -- 172.21.15.37:0/4103102496 >> v1:172.21.15.37:6789/0 conn(0x7f697006ac90 legacy=0x7f697006b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f696effd640 1 -- 172.21.15.37:0/4103102496 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697015f920 con 0x7f697015ca90 2024-04-25T08:56:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.370+0000 7f696e7fc640 1 -- 172.21.15.37:0/4103102496 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948915601 0 0) 0x7f697015f360 con 0x7f697006ac90 2024-04-25T08:56:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f696effd640 1 --2- 172.21.15.37:0/4103102496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f697015ca90 0x7f69700d0c10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f69580097c0 tx=0x7f695801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96325eabb103c754 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f696e7fc640 1 -- 172.21.15.37:0/4103102496 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69580b0070 con 0x7f697015ca90 2024-04-25T08:56:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f696e7fc640 1 -- 172.21.15.37:0/4103102496 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69580ab600 con 0x7f697015ca90 2024-04-25T08:56:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f696e7fc640 1 -- 172.21.15.37:0/4103102496 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69580ab9d0 con 0x7f697015ca90 2024-04-25T08:56:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f6976045640 1 -- 172.21.15.37:0/4103102496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f697015ca90 msgr2=0x7f69700d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f6976045640 1 --2- 172.21.15.37:0/4103102496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f697015ca90 0x7f69700d0c10 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f69580097c0 tx=0x7f695801c430 comp rx=0 tx=0).stop 2024-04-25T08:56:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f6976045640 1 -- 172.21.15.37:0/4103102496 shutdown_connections 2024-04-25T08:56:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f6976045640 1 --2- 172.21.15.37:0/4103102496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f697015ca90 0x7f69700d0c10 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f6976045640 1 -- 172.21.15.37:0/4103102496 >> 172.21.15.37:0/4103102496 conn(0x7f6970099a70 msgr2=0x7f69700d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f6976045640 1 -- 172.21.15.37:0/4103102496 shutdown_connections 2024-04-25T08:56:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f6976045640 1 -- 172.21.15.37:0/4103102496 wait complete. 2024-04-25T08:56:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.374+0000 7f6976045640 1 Processor -- start 2024-04-25T08:56:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6976045640 1 -- start start 2024-04-25T08:56:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6976045640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f697006ac90 0x7f69700c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6976045640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69700cc130 0x7f6970155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6976045640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f697015ca90 0x7f697006fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6976045640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6970169a00 con 0x7f69700cc130 2024-04-25T08:56:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6976045640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6970169880 con 0x7f697006ac90 2024-04-25T08:56:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6976045640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6970169b80 con 0x7f697015ca90 2024-04-25T08:56:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f696effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69700cc130 0x7f6970155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f696effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69700cc130 0x7f6970155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39146/0 (socket says 172.21.15.37:39146) 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f696effd640 1 -- 172.21.15.37:0/3053853104 learned_addr learned my addr 172.21.15.37:0/3053853104 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f696effd640 1 -- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f697015ca90 msgr2=0x7f697006fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f696effd640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f697015ca90 0x7f697006fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f696f7fe640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f697006ac90 0x7f69700c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f696effd640 1 -- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f697006ac90 msgr2=0x7f69700c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f696effd640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f697006ac90 0x7f69700c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f696effd640 1 -- 172.21.15.37:0/3053853104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697006ec10 con 0x7f69700cc130 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f696effd640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69700cc130 0x7f6970155670 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6958019040 tx=0x7f69580191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6967fff640 1 -- 172.21.15.37:0/3053853104 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69580b0040 con 0x7f69700cc130 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6967fff640 1 -- 172.21.15.37:0/3053853104 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69580bb070 con 0x7f69700cc130 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6976045640 1 -- 172.21.15.37:0/3053853104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697006eef0 con 0x7f69700cc130 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6967fff640 1 -- 172.21.15.37:0/3053853104 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69580b58c0 con 0x7f69700cc130 2024-04-25T08:56:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6976045640 1 -- 172.21.15.37:0/3053853104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697006f4b0 con 0x7f69700cc130 2024-04-25T08:56:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6976045640 1 -- 172.21.15.37:0/3053853104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697004efa0 con 0x7f69700cc130 2024-04-25T08:56:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6967fff640 1 -- 172.21.15.37:0/3053853104 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f69580b3070 con 0x7f69700cc130 2024-04-25T08:56:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6967fff640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6950066f60 0x7f6950069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f6967fff640 1 -- 172.21.15.37:0/3053853104 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f695811f0b0 con 0x7f69700cc130 2024-04-25T08:56:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.378+0000 7f696f7fe640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6950066f60 0x7f6950069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.382+0000 7f696f7fe640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6950066f60 0x7f6950069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f69680012f0 tx=0x7f6968009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.382+0000 7f6967fff640 1 -- 172.21.15.37:0/3053853104 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6958016050 con 0x7f69700cc130 2024-04-25T08:56:56.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.534+0000 7f6976045640 1 -- 172.21.15.37:0/3053853104 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69700b6480 con 0x7f6950066f60 2024-04-25T08:56:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.554+0000 7f6967fff640 1 -- 172.21.15.37:0/3053853104 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252051 (secure 0 0 0) 0x7f69700b6480 con 0x7f6950066f60 2024-04-25T08:56:56.546 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:56.547 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 -- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6950066f60 msgr2=0x7f6950069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6950066f60 0x7f6950069420 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f69680012f0 tx=0x7f6968009040 comp rx=0 tx=0).stop 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 -- 172.21.15.37:0/3053853104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69700cc130 msgr2=0x7f6970155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69700cc130 0x7f6970155670 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6958019040 tx=0x7f69580191d0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 -- 172.21.15.37:0/3053853104 shutdown_connections 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6950066f60 0x7f6950069420 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f697015ca90 0x7f697006fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69700cc130 0x7f6970155670 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 --2- 172.21.15.37:0/3053853104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f697006ac90 0x7f69700c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 -- 172.21.15.37:0/3053853104 >> 172.21.15.37:0/3053853104 conn(0x7f6970099a70 msgr2=0x7f6970073760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 -- 172.21.15.37:0/3053853104 shutdown_connections 2024-04-25T08:56:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.558+0000 7f6976045640 1 -- 172.21.15.37:0/3053853104 wait complete. 2024-04-25T08:56:56.563 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-04-25T08:56:55.211965+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":115,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258440,"kb_used_data":8980,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716280,"statfs":{"total":773094113280,"available":772829470720,"internally_reserved":0,"allocated":9195520,"data_stored":4830592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":45,"apply_latency_ms":45,"commit_latency_ns":45000000,"apply_latency_ns":45000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.146926"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108560+0000","last_change":"2024-04-25T08:56:52.120967+0000","last_active":"2024-04-25T08:56:53.108560+0000","last_peered":"2024-04-25T08:56:53.108560+0000","last_clean":"2024-04-25T08:56:53.108560+0000","last_became_active":"2024-04-25T08:56:52.120243+0000","last_became_peered":"2024-04-25T08:56:52.120243+0000","last_unstale":"2024-04-25T08:56:53.108560+0000","last_undegraded":"2024-04-25T08:56:53.108560+0000","last_fullsized":"2024-04-25T08:56:53.108560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:58.573901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105574+0000","last_change":"2024-04-25T08:56:52.123315+0000","last_active":"2024-04-25T08:56:53.105574+0000","last_peered":"2024-04-25T08:56:53.105574+0000","last_clean":"2024-04-25T08:56:53.105574+0000","last_became_active":"2024-04-25T08:56:52.123027+0000","last_became_peered":"2024-04-25T08:56:52.123027+0000","last_unstale":"2024-04-25T08:56:53.105574+0000","last_undegraded":"2024-04-25T08:56:53.105574+0000","last_fullsized":"2024-04-25T08:56:53.105574+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:16.735000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172403+0000","last_change":"2024-04-25T08:56:52.111597+0000","last_active":"2024-04-25T08:56:53.172403+0000","last_peered":"2024-04-25T08:56:53.172403+0000","last_clean":"2024-04-25T08:56:53.172403+0000","last_became_active":"2024-04-25T08:56:52.111256+0000","last_became_peered":"2024-04-25T08:56:52.111256+0000","last_unstale":"2024-04-25T08:56:53.172403+0000","last_undegraded":"2024-04-25T08:56:53.172403+0000","last_fullsized":"2024-04-25T08:56:53.172403+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:28.290610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.107452+0000","last_change":"2024-04-25T08:56:52.100023+0000","last_active":"2024-04-25T08:56:52.107452+0000","last_peered":"2024-04-25T08:56:52.107452+0000","last_clean":"2024-04-25T08:56:52.107452+0000","last_became_active":"2024-04-25T08:56:52.099205+0000","last_became_peered":"2024-04-25T08:56:52.099205+0000","last_unstale":"2024-04-25T08:56:52.107452+0000","last_undegraded":"2024-04-25T08:56:52.107452+0000","last_fullsized":"2024-04-25T08:56:52.107452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:28.290610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116379+0000","last_change":"2024-04-25T08:56:52.106706+0000","last_active":"2024-04-25T08:56:52.116379+0000","last_peered":"2024-04-25T08:56:52.116379+0000","last_clean":"2024-04-25T08:56:52.116379+0000","last_became_active":"2024-04-25T08:56:52.106354+0000","last_became_peered":"2024-04-25T08:56:52.106354+0000","last_unstale":"2024-04-25T08:56:52.116379+0000","last_undegraded":"2024-04-25T08:56:52.116379+0000","last_fullsized":"2024-04-25T08:56:52.116379+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:39:15.208661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.127136+0000","last_change":"2024-04-25T08:56:52.117998+0000","last_active":"2024-04-25T08:56:52.127136+0000","last_peered":"2024-04-25T08:56:52.127136+0000","last_clean":"2024-04-25T08:56:52.127136+0000","last_became_active":"2024-04-25T08:56:52.117702+0000","last_became_peered":"2024-04-25T08:56:52.117702+0000","last_unstale":"2024-04-25T08:56:52.127136+0000","last_undegraded":"2024-04-25T08:56:52.127136+0000","last_fullsized":"2024-04-25T08:56:52.127136+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.191980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469411+0000","last_change":"2024-04-25T08:56:52.116445+0000","last_active":"2024-04-25T08:56:53.469411+0000","last_peered":"2024-04-25T08:56:53.469411+0000","last_clean":"2024-04-25T08:56:53.469411+0000","last_became_active":"2024-04-25T08:56:52.114423+0000","last_became_peered":"2024-04-25T08:56:52.114423+0000","last_unstale":"2024-04-25T08:56:53.469411+0000","last_undegraded":"2024-04-25T08:56:53.469411+0000","last_fullsized":"2024-04-25T08:56:53.469411+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:58.760488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116665+0000","last_change":"2024-04-25T08:56:52.106148+0000","last_active":"2024-04-25T08:56:52.116665+0000","last_peered":"2024-04-25T08:56:52.116665+0000","last_clean":"2024-04-25T08:56:52.116665+0000","last_became_active":"2024-04-25T08:56:52.105797+0000","last_became_peered":"2024-04-25T08:56:52.105797+0000","last_unstale":"2024-04-25T08:56:52.116665+0000","last_undegraded":"2024-04-25T08:56:52.116665+0000","last_fullsized":"2024-04-25T08:56:52.116665+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:37.569468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.110138+0000","last_change":"2024-04-25T08:56:52.110253+0000","last_active":"2024-04-25T08:56:53.110138+0000","last_peered":"2024-04-25T08:56:53.110138+0000","last_clean":"2024-04-25T08:56:53.110138+0000","last_became_active":"2024-04-25T08:56:52.108998+0000","last_became_peered":"2024-04-25T08:56:52.108998+0000","last_unstale":"2024-04-25T08:56:53.110138+0000","last_undegraded":"2024-04-25T08:56:53.110138+0000","last_fullsized":"2024-04-25T08:56:53.110138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:59.878728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.110551+0000","last_change":"2024-04-25T08:56:52.108706+0000","last_active":"2024-04-25T08:56:52.110551+0000","last_peered":"2024-04-25T08:56:52.110551+0000","last_clean":"2024-04-25T08:56:52.110551+0000","last_became_active":"2024-04-25T08:56:52.108243+0000","last_became_peered":"2024-04-25T08:56:52.108243+0000","last_unstale":"2024-04-25T08:56:52.110551+0000","last_undegraded":"2024-04-25T08:56:52.110551+0000","last_fullsized":"2024-04-25T08:56:52.110551+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:52.935915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105259+0000","last_change":"2024-04-25T08:56:52.128945+0000","last_active":"2024-04-25T08:56:53.105259+0000","last_peered":"2024-04-25T08:56:53.105259+0000","last_clean":"2024-04-25T08:56:53.105259+0000","last_became_active":"2024-04-25T08:56:52.128666+0000","last_became_peered":"2024-04-25T08:56:52.128666+0000","last_unstale":"2024-04-25T08:56:53.105259+0000","last_undegraded":"2024-04-25T08:56:53.105259+0000","last_fullsized":"2024-04-25T08:56:53.105259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:42.991363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.130660+0000","last_change":"2024-04-25T08:56:52.123450+0000","last_active":"2024-04-25T08:56:52.130660+0000","last_peered":"2024-04-25T08:56:52.130660+0000","last_clean":"2024-04-25T08:56:52.130660+0000","last_became_active":"2024-04-25T08:56:52.122737+0000","last_became_peered":"2024-04-25T08:56:52.122737+0000","last_unstale":"2024-04-25T08:56:52.130660+0000","last_undegraded":"2024-04-25T08:56:52.130660+0000","last_fullsized":"2024-04-25T08:56:52.130660+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:48.492455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.122076+0000","last_change":"2024-04-25T08:56:52.114022+0000","last_active":"2024-04-25T08:56:52.122076+0000","last_peered":"2024-04-25T08:56:52.122076+0000","last_clean":"2024-04-25T08:56:52.122076+0000","last_became_active":"2024-04-25T08:56:52.113222+0000","last_became_peered":"2024-04-25T08:56:52.113222+0000","last_unstale":"2024-04-25T08:56:52.122076+0000","last_undegraded":"2024-04-25T08:56:52.122076+0000","last_fullsized":"2024-04-25T08:56:52.122076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:10.735890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.121788+0000","last_change":"2024-04-25T08:56:52.115122+0000","last_active":"2024-04-25T08:56:52.121788+0000","last_peered":"2024-04-25T08:56:52.121788+0000","last_clean":"2024-04-25T08:56:52.121788+0000","last_became_active":"2024-04-25T08:56:52.114811+0000","last_became_peered":"2024-04-25T08:56:52.114811+0000","last_unstale":"2024-04-25T08:56:52.121788+0000","last_undegraded":"2024-04-25T08:56:52.121788+0000","last_fullsized":"2024-04-25T08:56:52.121788+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:40.610515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.104723+0000","last_change":"2024-04-25T08:56:52.119694+0000","last_active":"2024-04-25T08:56:53.104723+0000","last_peered":"2024-04-25T08:56:53.104723+0000","last_clean":"2024-04-25T08:56:53.104723+0000","last_became_active":"2024-04-25T08:56:52.119387+0000","last_became_peered":"2024-04-25T08:56:52.119387+0000","last_unstale":"2024-04-25T08:56:53.104723+0000","last_undegraded":"2024-04-25T08:56:53.104723+0000","last_fullsized":"2024-04-25T08:56:53.104723+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:10:44.138372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108027+0000","last_change":"2024-04-25T08:56:52.109950+0000","last_active":"2024-04-25T08:56:53.108027+0000","last_peered":"2024-04-25T08:56:53.108027+0000","last_clean":"2024-04-25T08:56:53.108027+0000","last_became_active":"2024-04-25T08:56:52.109259+0000","last_became_peered":"2024-04-25T08:56:52.109259+0000","last_unstale":"2024-04-25T08:56:53.108027+0000","last_undegraded":"2024-04-25T08:56:53.108027+0000","last_fullsized":"2024-04-25T08:56:53.108027+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:26.865946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116347+0000","last_change":"2024-04-25T08:56:52.105179+0000","last_active":"2024-04-25T08:56:52.116347+0000","last_peered":"2024-04-25T08:56:52.116347+0000","last_clean":"2024-04-25T08:56:52.116347+0000","last_became_active":"2024-04-25T08:56:52.104310+0000","last_became_peered":"2024-04-25T08:56:52.104310+0000","last_unstale":"2024-04-25T08:56:52.116347+0000","last_undegraded":"2024-04-25T08:56:52.116347+0000","last_fullsized":"2024-04-25T08:56:52.116347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:56.251550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172549+0000","last_change":"2024-04-25T08:56:52.099522+0000","last_active":"2024-04-25T08:56:53.172549+0000","last_peered":"2024-04-25T08:56:53.172549+0000","last_clean":"2024-04-25T08:56:53.172549+0000","last_became_active":"2024-04-25T08:56:52.098507+0000","last_became_peered":"2024-04-25T08:56:52.098507+0000","last_unstale":"2024-04-25T08:56:53.172549+0000","last_undegraded":"2024-04-25T08:56:53.172549+0000","last_fullsized":"2024-04-25T08:56:53.172549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:56.009236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.115898+0000","last_change":"2024-04-25T08:56:52.105847+0000","last_active":"2024-04-25T08:56:52.115898+0000","last_peered":"2024-04-25T08:56:52.115898+0000","last_clean":"2024-04-25T08:56:52.115898+0000","last_became_active":"2024-04-25T08:56:52.105159+0000","last_became_peered":"2024-04-25T08:56:52.105159+0000","last_unstale":"2024-04-25T08:56:52.115898+0000","last_undegraded":"2024-04-25T08:56:52.115898+0000","last_fullsized":"2024-04-25T08:56:52.115898+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:15.561525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116803+0000","last_change":"2024-04-25T08:56:52.109584+0000","last_active":"2024-04-25T08:56:52.116803+0000","last_peered":"2024-04-25T08:56:52.116803+0000","last_clean":"2024-04-25T08:56:52.116803+0000","last_became_active":"2024-04-25T08:56:52.109228+0000","last_became_peered":"2024-04-25T08:56:52.109228+0000","last_unstale":"2024-04-25T08:56:52.116803+0000","last_undegraded":"2024-04-25T08:56:52.116803+0000","last_fullsized":"2024-04-25T08:56:52.116803+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:36.623445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.109808+0000","last_change":"2024-04-25T08:56:52.117308+0000","last_active":"2024-04-25T08:56:53.109808+0000","last_peered":"2024-04-25T08:56:53.109808+0000","last_clean":"2024-04-25T08:56:53.109808+0000","last_became_active":"2024-04-25T08:56:52.116500+0000","last_became_peered":"2024-04-25T08:56:52.116500+0000","last_unstale":"2024-04-25T08:56:53.109808+0000","last_undegraded":"2024-04-25T08:56:53.109808+0000","last_fullsized":"2024-04-25T08:56:53.109808+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:39.669375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.122029+0000","last_change":"2024-04-25T08:56:52.113612+0000","last_active":"2024-04-25T08:56:52.122029+0000","last_peered":"2024-04-25T08:56:52.122029+0000","last_clean":"2024-04-25T08:56:52.122029+0000","last_became_active":"2024-04-25T08:56:52.113333+0000","last_became_peered":"2024-04-25T08:56:52.113333+0000","last_unstale":"2024-04-25T08:56:52.122029+0000","last_undegraded":"2024-04-25T08:56:52.122029+0000","last_fullsized":"2024-04-25T08:56:52.122029+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:38.584586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.109432+0000","last_change":"2024-04-25T08:56:52.098022+0000","last_active":"2024-04-25T08:56:53.109432+0000","last_peered":"2024-04-25T08:56:53.109432+0000","last_clean":"2024-04-25T08:56:53.109432+0000","last_became_active":"2024-04-25T08:56:52.097587+0000","last_became_peered":"2024-04-25T08:56:52.097587+0000","last_unstale":"2024-04-25T08:56:53.109432+0000","last_undegraded":"2024-04-25T08:56:53.109432+0000","last_fullsized":"2024-04-25T08:56:53.109432+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:14.768231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105490+0000","last_change":"2024-04-25T08:56:52.125229+0000","last_active":"2024-04-25T08:56:53.105490+0000","last_peered":"2024-04-25T08:56:53.105490+0000","last_clean":"2024-04-25T08:56:53.105490+0000","last_became_active":"2024-04-25T08:56:52.122239+0000","last_became_peered":"2024-04-25T08:56:52.122239+0000","last_unstale":"2024-04-25T08:56:53.105490+0000","last_undegraded":"2024-04-25T08:56:53.105490+0000","last_fullsized":"2024-04-25T08:56:53.105490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:40.610515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105021+0000","last_change":"2024-04-25T08:56:52.122510+0000","last_active":"2024-04-25T08:56:53.105021+0000","last_peered":"2024-04-25T08:56:53.105021+0000","last_clean":"2024-04-25T08:56:53.105021+0000","last_became_active":"2024-04-25T08:56:52.122181+0000","last_became_peered":"2024-04-25T08:56:52.122181+0000","last_unstale":"2024-04-25T08:56:53.105021+0000","last_undegraded":"2024-04-25T08:56:53.105021+0000","last_fullsized":"2024-04-25T08:56:53.105021+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:50.067483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.130474+0000","last_change":"2024-04-25T08:56:52.123288+0000","last_active":"2024-04-25T08:56:52.130474+0000","last_peered":"2024-04-25T08:56:52.130474+0000","last_clean":"2024-04-25T08:56:52.130474+0000","last_became_active":"2024-04-25T08:56:52.122607+0000","last_became_peered":"2024-04-25T08:56:52.122607+0000","last_unstale":"2024-04-25T08:56:52.130474+0000","last_undegraded":"2024-04-25T08:56:52.130474+0000","last_fullsized":"2024-04-25T08:56:52.130474+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:09.821880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.109045+0000","last_change":"2024-04-25T08:56:52.106104+0000","last_active":"2024-04-25T08:56:53.109045+0000","last_peered":"2024-04-25T08:56:53.109045+0000","last_clean":"2024-04-25T08:56:53.109045+0000","last_became_active":"2024-04-25T08:56:52.105631+0000","last_became_peered":"2024-04-25T08:56:52.105631+0000","last_unstale":"2024-04-25T08:56:53.109045+0000","last_undegraded":"2024-04-25T08:56:53.109045+0000","last_fullsized":"2024-04-25T08:56:53.109045+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.636840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116832+0000","last_change":"2024-04-25T08:56:52.110246+0000","last_active":"2024-04-25T08:56:52.116832+0000","last_peered":"2024-04-25T08:56:52.116832+0000","last_clean":"2024-04-25T08:56:52.116832+0000","last_became_active":"2024-04-25T08:56:52.109951+0000","last_became_peered":"2024-04-25T08:56:52.109951+0000","last_unstale":"2024-04-25T08:56:52.116832+0000","last_undegraded":"2024-04-25T08:56:52.116832+0000","last_fullsized":"2024-04-25T08:56:52.116832+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:56.943561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.104477+0000","last_change":"2024-04-25T08:56:52.112100+0000","last_active":"2024-04-25T08:56:53.104477+0000","last_peered":"2024-04-25T08:56:53.104477+0000","last_clean":"2024-04-25T08:56:53.104477+0000","last_became_active":"2024-04-25T08:56:52.111703+0000","last_became_peered":"2024-04-25T08:56:52.111703+0000","last_unstale":"2024-04-25T08:56:53.104477+0000","last_undegraded":"2024-04-25T08:56:53.104477+0000","last_fullsized":"2024-04-25T08:56:53.104477+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:52.090957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.107554+0000","last_change":"2024-04-25T08:56:52.115343+0000","last_active":"2024-04-25T08:56:53.107554+0000","last_peered":"2024-04-25T08:56:53.107554+0000","last_clean":"2024-04-25T08:56:53.107554+0000","last_became_active":"2024-04-25T08:56:52.114598+0000","last_became_peered":"2024-04-25T08:56:52.114598+0000","last_unstale":"2024-04-25T08:56:53.107554+0000","last_undegraded":"2024-04-25T08:56:53.107554+0000","last_fullsized":"2024-04-25T08:56:53.107554+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:37.302261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.171853+0000","last_change":"2024-04-25T08:56:52.105089+0000","last_active":"2024-04-25T08:56:53.171853+0000","last_peered":"2024-04-25T08:56:53.171853+0000","last_clean":"2024-04-25T08:56:53.171853+0000","last_became_active":"2024-04-25T08:56:52.104491+0000","last_became_peered":"2024-04-25T08:56:52.104491+0000","last_unstale":"2024-04-25T08:56:53.171853+0000","last_undegraded":"2024-04-25T08:56:53.171853+0000","last_fullsized":"2024-04-25T08:56:53.171853+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:27.589864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.107616+0000","last_change":"2024-04-25T08:56:52.121158+0000","last_active":"2024-04-25T08:56:53.107616+0000","last_peered":"2024-04-25T08:56:53.107616+0000","last_clean":"2024-04-25T08:56:53.107616+0000","last_became_active":"2024-04-25T08:56:52.120433+0000","last_became_peered":"2024-04-25T08:56:52.120433+0000","last_unstale":"2024-04-25T08:56:53.107616+0000","last_undegraded":"2024-04-25T08:56:53.107616+0000","last_fullsized":"2024-04-25T08:56:53.107616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:39.844113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.171576+0000","last_change":"2024-04-25T08:56:52.099148+0000","last_active":"2024-04-25T08:56:53.171576+0000","last_peered":"2024-04-25T08:56:53.171576+0000","last_clean":"2024-04-25T08:56:53.171576+0000","last_became_active":"2024-04-25T08:56:52.098367+0000","last_became_peered":"2024-04-25T08:56:52.098367+0000","last_unstale":"2024-04-25T08:56:53.171576+0000","last_undegraded":"2024-04-25T08:56:53.171576+0000","last_fullsized":"2024-04-25T08:56:53.171576+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:59.878728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.133839+0000","last_change":"2024-04-25T08:56:51.107533+0000","last_active":"2024-04-25T08:56:52.133839+0000","last_peered":"2024-04-25T08:56:52.133839+0000","last_clean":"2024-04-25T08:56:52.133839+0000","last_became_active":"2024-04-25T08:56:51.101692+0000","last_became_peered":"2024-04-25T08:56:51.101692+0000","last_unstale":"2024-04-25T08:56:52.133839+0000","last_undegraded":"2024-04-25T08:56:52.133839+0000","last_fullsized":"2024-04-25T08:56:52.133839+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:55.594310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.104328+0000","last_change":"2024-04-25T08:56:52.131959+0000","last_active":"2024-04-25T08:56:53.104328+0000","last_peered":"2024-04-25T08:56:53.104328+0000","last_clean":"2024-04-25T08:56:53.104328+0000","last_became_active":"2024-04-25T08:56:52.131754+0000","last_became_peered":"2024-04-25T08:56:52.131754+0000","last_unstale":"2024-04-25T08:56:53.104328+0000","last_undegraded":"2024-04-25T08:56:53.104328+0000","last_fullsized":"2024-04-25T08:56:53.104328+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:10.961781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.107987+0000","last_change":"2024-04-25T08:56:51.119798+0000","last_active":"2024-04-25T08:56:53.107987+0000","last_peered":"2024-04-25T08:56:53.107987+0000","last_clean":"2024-04-25T08:56:53.107987+0000","last_became_active":"2024-04-25T08:56:51.119016+0000","last_became_peered":"2024-04-25T08:56:51.119016+0000","last_unstale":"2024-04-25T08:56:53.107987+0000","last_undegraded":"2024-04-25T08:56:53.107987+0000","last_fullsized":"2024-04-25T08:56:53.107987+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:27.279583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.109687+0000","last_change":"2024-04-25T08:56:52.112616+0000","last_active":"2024-04-25T08:56:53.109687+0000","last_peered":"2024-04-25T08:56:53.109687+0000","last_clean":"2024-04-25T08:56:53.109687+0000","last_became_active":"2024-04-25T08:56:52.111863+0000","last_became_peered":"2024-04-25T08:56:52.111863+0000","last_unstale":"2024-04-25T08:56:53.109687+0000","last_undegraded":"2024-04-25T08:56:53.109687+0000","last_fullsized":"2024-04-25T08:56:53.109687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:36.553929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108036+0000","last_change":"2024-04-25T08:56:51.120584+0000","last_active":"2024-04-25T08:56:53.108036+0000","last_peered":"2024-04-25T08:56:53.108036+0000","last_clean":"2024-04-25T08:56:53.108036+0000","last_became_active":"2024-04-25T08:56:51.115950+0000","last_became_peered":"2024-04-25T08:56:51.115950+0000","last_unstale":"2024-04-25T08:56:53.108036+0000","last_undegraded":"2024-04-25T08:56:53.108036+0000","last_fullsized":"2024-04-25T08:56:53.108036+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:57.749461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.110120+0000","last_change":"2024-04-25T08:56:52.097648+0000","last_active":"2024-04-25T08:56:53.110120+0000","last_peered":"2024-04-25T08:56:53.110120+0000","last_clean":"2024-04-25T08:56:53.110120+0000","last_became_active":"2024-04-25T08:56:52.096860+0000","last_became_peered":"2024-04-25T08:56:52.096860+0000","last_unstale":"2024-04-25T08:56:53.110120+0000","last_undegraded":"2024-04-25T08:56:53.110120+0000","last_fullsized":"2024-04-25T08:56:53.110120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:37.523964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469641+0000","last_change":"2024-04-25T08:56:51.116983+0000","last_active":"2024-04-25T08:56:53.469641+0000","last_peered":"2024-04-25T08:56:53.469641+0000","last_clean":"2024-04-25T08:56:53.469641+0000","last_became_active":"2024-04-25T08:56:51.116329+0000","last_became_peered":"2024-04-25T08:56:51.116329+0000","last_unstale":"2024-04-25T08:56:53.469641+0000","last_undegraded":"2024-04-25T08:56:53.469641+0000","last_fullsized":"2024-04-25T08:56:53.469641+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:14.662323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.104762+0000","last_change":"2024-04-25T08:56:52.117640+0000","last_active":"2024-04-25T08:56:53.104762+0000","last_peered":"2024-04-25T08:56:53.104762+0000","last_clean":"2024-04-25T08:56:53.104762+0000","last_became_active":"2024-04-25T08:56:52.117336+0000","last_became_peered":"2024-04-25T08:56:52.117336+0000","last_unstale":"2024-04-25T08:56:53.104762+0000","last_undegraded":"2024-04-25T08:56:53.104762+0000","last_fullsized":"2024-04-25T08:56:53.104762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:45.559340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.110104+0000","last_change":"2024-04-25T08:56:51.101068+0000","last_active":"2024-04-25T08:56:53.110104+0000","last_peered":"2024-04-25T08:56:53.110104+0000","last_clean":"2024-04-25T08:56:53.110104+0000","last_became_active":"2024-04-25T08:56:51.100290+0000","last_became_peered":"2024-04-25T08:56:51.100290+0000","last_unstale":"2024-04-25T08:56:53.110104+0000","last_undegraded":"2024-04-25T08:56:53.110104+0000","last_fullsized":"2024-04-25T08:56:53.110104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:22.270137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105291+0000","last_change":"2024-04-25T08:56:52.128956+0000","last_active":"2024-04-25T08:56:53.105291+0000","last_peered":"2024-04-25T08:56:53.105291+0000","last_clean":"2024-04-25T08:56:53.105291+0000","last_became_active":"2024-04-25T08:56:52.128721+0000","last_became_peered":"2024-04-25T08:56:52.128721+0000","last_unstale":"2024-04-25T08:56:53.105291+0000","last_undegraded":"2024-04-25T08:56:53.105291+0000","last_fullsized":"2024-04-25T08:56:53.105291+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:39.954646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.133631+0000","last_change":"2024-04-25T08:56:51.107942+0000","last_active":"2024-04-25T08:56:52.133631+0000","last_peered":"2024-04-25T08:56:52.133631+0000","last_clean":"2024-04-25T08:56:52.133631+0000","last_became_active":"2024-04-25T08:56:51.107154+0000","last_became_peered":"2024-04-25T08:56:51.107154+0000","last_unstale":"2024-04-25T08:56:52.133631+0000","last_undegraded":"2024-04-25T08:56:52.133631+0000","last_fullsized":"2024-04-25T08:56:52.133631+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:36.512937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.110429+0000","last_change":"2024-04-25T08:56:52.110931+0000","last_active":"2024-04-25T08:56:53.110429+0000","last_peered":"2024-04-25T08:56:53.110429+0000","last_clean":"2024-04-25T08:56:53.110429+0000","last_became_active":"2024-04-25T08:56:52.110242+0000","last_became_peered":"2024-04-25T08:56:52.110242+0000","last_unstale":"2024-04-25T08:56:53.110429+0000","last_undegraded":"2024-04-25T08:56:53.110429+0000","last_fullsized":"2024-04-25T08:56:53.110429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:27.589864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105838+0000","last_change":"2024-04-25T08:56:51.119436+0000","last_active":"2024-04-25T08:56:53.105838+0000","last_peered":"2024-04-25T08:56:53.105838+0000","last_clean":"2024-04-25T08:56:53.105838+0000","last_became_active":"2024-04-25T08:56:51.119122+0000","last_became_peered":"2024-04-25T08:56:51.119122+0000","last_unstale":"2024-04-25T08:56:53.105838+0000","last_undegraded":"2024-04-25T08:56:53.105838+0000","last_fullsized":"2024-04-25T08:56:53.105838+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:36.291234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469987+0000","last_change":"2024-04-25T08:56:52.107107+0000","last_active":"2024-04-25T08:56:53.469987+0000","last_peered":"2024-04-25T08:56:53.469987+0000","last_clean":"2024-04-25T08:56:53.469987+0000","last_became_active":"2024-04-25T08:56:52.103569+0000","last_became_peered":"2024-04-25T08:56:52.103569+0000","last_unstale":"2024-04-25T08:56:53.469987+0000","last_undegraded":"2024-04-25T08:56:53.469987+0000","last_fullsized":"2024-04-25T08:56:53.469987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:52.935915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.087273+0000","last_change":"2024-04-25T08:56:51.100730+0000","last_active":"2024-04-25T08:56:52.087273+0000","last_peered":"2024-04-25T08:56:52.087273+0000","last_clean":"2024-04-25T08:56:52.087273+0000","last_became_active":"2024-04-25T08:56:51.100326+0000","last_became_peered":"2024-04-25T08:56:51.100326+0000","last_unstale":"2024-04-25T08:56:52.087273+0000","last_undegraded":"2024-04-25T08:56:52.087273+0000","last_fullsized":"2024-04-25T08:56:52.087273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:28.323423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105237+0000","last_change":"2024-04-25T08:56:52.133708+0000","last_active":"2024-04-25T08:56:53.105237+0000","last_peered":"2024-04-25T08:56:53.105237+0000","last_clean":"2024-04-25T08:56:53.105237+0000","last_became_active":"2024-04-25T08:56:52.133466+0000","last_became_peered":"2024-04-25T08:56:52.133466+0000","last_unstale":"2024-04-25T08:56:53.105237+0000","last_undegraded":"2024-04-25T08:56:53.105237+0000","last_fullsized":"2024-04-25T08:56:53.105237+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:02.899860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108475+0000","last_change":"2024-04-25T08:56:51.118078+0000","last_active":"2024-04-25T08:56:53.108475+0000","last_peered":"2024-04-25T08:56:53.108475+0000","last_clean":"2024-04-25T08:56:53.108475+0000","last_became_active":"2024-04-25T08:56:51.115969+0000","last_became_peered":"2024-04-25T08:56:51.115969+0000","last_unstale":"2024-04-25T08:56:53.108475+0000","last_undegraded":"2024-04-25T08:56:53.108475+0000","last_fullsized":"2024-04-25T08:56:53.108475+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:51.924888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.107470+0000","last_change":"2024-04-25T08:56:52.100662+0000","last_active":"2024-04-25T08:56:52.107470+0000","last_peered":"2024-04-25T08:56:52.107470+0000","last_clean":"2024-04-25T08:56:52.107470+0000","last_became_active":"2024-04-25T08:56:52.099084+0000","last_became_peered":"2024-04-25T08:56:52.099084+0000","last_unstale":"2024-04-25T08:56:52.107470+0000","last_undegraded":"2024-04-25T08:56:52.107470+0000","last_fullsized":"2024-04-25T08:56:52.107470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:43.107847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.080868+0000","last_change":"2024-04-25T08:56:51.116625+0000","last_active":"2024-04-25T08:56:52.080868+0000","last_peered":"2024-04-25T08:56:52.080868+0000","last_clean":"2024-04-25T08:56:52.080868+0000","last_became_active":"2024-04-25T08:56:51.116012+0000","last_became_peered":"2024-04-25T08:56:51.116012+0000","last_unstale":"2024-04-25T08:56:52.080868+0000","last_undegraded":"2024-04-25T08:56:52.080868+0000","last_fullsized":"2024-04-25T08:56:52.080868+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:26.578837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116211+0000","last_change":"2024-04-25T08:56:52.106550+0000","last_active":"2024-04-25T08:56:52.116211+0000","last_peered":"2024-04-25T08:56:52.116211+0000","last_clean":"2024-04-25T08:56:52.116211+0000","last_became_active":"2024-04-25T08:56:52.105973+0000","last_became_peered":"2024-04-25T08:56:52.105973+0000","last_unstale":"2024-04-25T08:56:52.116211+0000","last_undegraded":"2024-04-25T08:56:52.116211+0000","last_fullsized":"2024-04-25T08:56:52.116211+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:32.910438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.133509+0000","last_change":"2024-04-25T08:56:51.095525+0000","last_active":"2024-04-25T08:56:52.133509+0000","last_peered":"2024-04-25T08:56:52.133509+0000","last_clean":"2024-04-25T08:56:52.133509+0000","last_became_active":"2024-04-25T08:56:51.095223+0000","last_became_peered":"2024-04-25T08:56:51.095223+0000","last_unstale":"2024-04-25T08:56:52.133509+0000","last_undegraded":"2024-04-25T08:56:52.133509+0000","last_fullsized":"2024-04-25T08:56:52.133509+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:39.853485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172275+0000","last_change":"2024-04-25T08:56:52.111648+0000","last_active":"2024-04-25T08:56:53.172275+0000","last_peered":"2024-04-25T08:56:53.172275+0000","last_clean":"2024-04-25T08:56:53.172275+0000","last_became_active":"2024-04-25T08:56:52.111251+0000","last_became_peered":"2024-04-25T08:56:52.111251+0000","last_unstale":"2024-04-25T08:56:53.172275+0000","last_undegraded":"2024-04-25T08:56:53.172275+0000","last_fullsized":"2024-04-25T08:56:53.172275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:58.760488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"81'6","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469717+0000","last_change":"2024-04-25T08:56:51.096512+0000","last_active":"2024-04-25T08:56:53.469717+0000","last_peered":"2024-04-25T08:56:53.469717+0000","last_clean":"2024-04-25T08:56:53.469717+0000","last_became_active":"2024-04-25T08:56:51.095725+0000","last_became_peered":"2024-04-25T08:56:51.095725+0000","last_unstale":"2024-04-25T08:56:53.469717+0000","last_undegraded":"2024-04-25T08:56:53.469717+0000","last_fullsized":"2024-04-25T08:56:53.469717+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:07.001522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.107510+0000","last_change":"2024-04-25T08:56:52.103550+0000","last_active":"2024-04-25T08:56:52.107510+0000","last_peered":"2024-04-25T08:56:52.107510+0000","last_clean":"2024-04-25T08:56:52.107510+0000","last_became_active":"2024-04-25T08:56:52.102576+0000","last_became_peered":"2024-04-25T08:56:52.102576+0000","last_unstale":"2024-04-25T08:56:52.107510+0000","last_undegraded":"2024-04-25T08:56:52.107510+0000","last_fullsized":"2024-04-25T08:56:52.107510+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:56.009236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.138514+0000","last_change":"2024-04-25T08:56:51.108723+0000","last_active":"2024-04-25T08:56:52.138514+0000","last_peered":"2024-04-25T08:56:52.138514+0000","last_clean":"2024-04-25T08:56:52.138514+0000","last_became_active":"2024-04-25T08:56:51.107894+0000","last_became_peered":"2024-04-25T08:56:51.107894+0000","last_unstale":"2024-04-25T08:56:52.138514+0000","last_undegraded":"2024-04-25T08:56:52.138514+0000","last_fullsized":"2024-04-25T08:56:52.138514+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:57.749461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.470265+0000","last_change":"2024-04-25T08:56:52.102592+0000","last_active":"2024-04-25T08:56:53.470265+0000","last_peered":"2024-04-25T08:56:53.470265+0000","last_clean":"2024-04-25T08:56:53.470265+0000","last_became_active":"2024-04-25T08:56:52.102050+0000","last_became_peered":"2024-04-25T08:56:52.102050+0000","last_unstale":"2024-04-25T08:56:53.470265+0000","last_undegraded":"2024-04-25T08:56:53.470265+0000","last_fullsized":"2024-04-25T08:56:53.470265+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:04.895571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108037+0000","last_change":"2024-04-25T08:56:51.116396+0000","last_active":"2024-04-25T08:56:53.108037+0000","last_peered":"2024-04-25T08:56:53.108037+0000","last_clean":"2024-04-25T08:56:53.108037+0000","last_became_active":"2024-04-25T08:56:51.115635+0000","last_became_peered":"2024-04-25T08:56:51.115635+0000","last_unstale":"2024-04-25T08:56:53.108037+0000","last_undegraded":"2024-04-25T08:56:53.108037+0000","last_fullsized":"2024-04-25T08:56:53.108037+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:26.578837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.470202+0000","last_change":"2024-04-25T08:56:52.104020+0000","last_active":"2024-04-25T08:56:53.470202+0000","last_peered":"2024-04-25T08:56:53.470202+0000","last_clean":"2024-04-25T08:56:53.470202+0000","last_became_active":"2024-04-25T08:56:52.103266+0000","last_became_peered":"2024-04-25T08:56:52.103266+0000","last_unstale":"2024-04-25T08:56:53.470202+0000","last_undegraded":"2024-04-25T08:56:53.470202+0000","last_fullsized":"2024-04-25T08:56:53.470202+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:48.492455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.094595+0000","last_change":"2024-04-25T08:56:51.108606+0000","last_active":"2024-04-25T08:56:52.094595+0000","last_peered":"2024-04-25T08:56:52.094595+0000","last_clean":"2024-04-25T08:56:52.094595+0000","last_became_active":"2024-04-25T08:56:51.107964+0000","last_became_peered":"2024-04-25T08:56:51.107964+0000","last_unstale":"2024-04-25T08:56:52.094595+0000","last_undegraded":"2024-04-25T08:56:52.094595+0000","last_fullsized":"2024-04-25T08:56:52.094595+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:05.324319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.470105+0000","last_change":"2024-04-25T08:56:52.109009+0000","last_active":"2024-04-25T08:56:53.470105+0000","last_peered":"2024-04-25T08:56:53.470105+0000","last_clean":"2024-04-25T08:56:53.470105+0000","last_became_active":"2024-04-25T08:56:52.108260+0000","last_became_peered":"2024-04-25T08:56:52.108260+0000","last_unstale":"2024-04-25T08:56:53.470105+0000","last_undegraded":"2024-04-25T08:56:53.470105+0000","last_fullsized":"2024-04-25T08:56:53.470105+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:09.821880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105528+0000","last_change":"2024-04-25T08:56:51.113838+0000","last_active":"2024-04-25T08:56:53.105528+0000","last_peered":"2024-04-25T08:56:53.105528+0000","last_clean":"2024-04-25T08:56:53.105528+0000","last_became_active":"2024-04-25T08:56:51.113500+0000","last_became_peered":"2024-04-25T08:56:51.113500+0000","last_unstale":"2024-04-25T08:56:53.105528+0000","last_undegraded":"2024-04-25T08:56:53.105528+0000","last_fullsized":"2024-04-25T08:56:53.105528+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:25.854919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116756+0000","last_change":"2024-04-25T08:56:52.107089+0000","last_active":"2024-04-25T08:56:52.116756+0000","last_peered":"2024-04-25T08:56:52.116756+0000","last_clean":"2024-04-25T08:56:52.116756+0000","last_became_active":"2024-04-25T08:56:52.106667+0000","last_became_peered":"2024-04-25T08:56:52.106667+0000","last_unstale":"2024-04-25T08:56:52.116756+0000","last_undegraded":"2024-04-25T08:56:52.116756+0000","last_fullsized":"2024-04-25T08:56:52.116756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:10.961781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108685+0000","last_change":"2024-04-25T08:56:51.118338+0000","last_active":"2024-04-25T08:56:53.108685+0000","last_peered":"2024-04-25T08:56:53.108685+0000","last_clean":"2024-04-25T08:56:53.108685+0000","last_became_active":"2024-04-25T08:56:51.116647+0000","last_became_peered":"2024-04-25T08:56:51.116647+0000","last_unstale":"2024-04-25T08:56:53.108685+0000","last_undegraded":"2024-04-25T08:56:53.108685+0000","last_fullsized":"2024-04-25T08:56:53.108685+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:08.810853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.127331+0000","last_change":"2024-04-25T08:56:52.120630+0000","last_active":"2024-04-25T08:56:52.127331+0000","last_peered":"2024-04-25T08:56:52.127331+0000","last_clean":"2024-04-25T08:56:52.127331+0000","last_became_active":"2024-04-25T08:56:52.120307+0000","last_became_peered":"2024-04-25T08:56:52.120307+0000","last_unstale":"2024-04-25T08:56:52.127331+0000","last_undegraded":"2024-04-25T08:56:52.127331+0000","last_fullsized":"2024-04-25T08:56:52.127331+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:39.303259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.110710+0000","last_change":"2024-04-25T08:56:51.096606+0000","last_active":"2024-04-25T08:56:53.110710+0000","last_peered":"2024-04-25T08:56:53.110710+0000","last_clean":"2024-04-25T08:56:53.110710+0000","last_became_active":"2024-04-25T08:56:51.095790+0000","last_became_peered":"2024-04-25T08:56:51.095790+0000","last_unstale":"2024-04-25T08:56:53.110710+0000","last_undegraded":"2024-04-25T08:56:53.110710+0000","last_fullsized":"2024-04-25T08:56:53.110710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:38.672049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.127318+0000","last_change":"2024-04-25T08:56:52.117496+0000","last_active":"2024-04-25T08:56:52.127318+0000","last_peered":"2024-04-25T08:56:52.127318+0000","last_clean":"2024-04-25T08:56:52.127318+0000","last_became_active":"2024-04-25T08:56:52.117109+0000","last_became_peered":"2024-04-25T08:56:52.117109+0000","last_unstale":"2024-04-25T08:56:52.127318+0000","last_undegraded":"2024-04-25T08:56:52.127318+0000","last_fullsized":"2024-04-25T08:56:52.127318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:04.214004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.138477+0000","last_change":"2024-04-25T08:56:51.107381+0000","last_active":"2024-04-25T08:56:52.138477+0000","last_peered":"2024-04-25T08:56:52.138477+0000","last_clean":"2024-04-25T08:56:52.138477+0000","last_became_active":"2024-04-25T08:56:51.101514+0000","last_became_peered":"2024-04-25T08:56:51.101514+0000","last_unstale":"2024-04-25T08:56:52.138477+0000","last_undegraded":"2024-04-25T08:56:52.138477+0000","last_fullsized":"2024-04-25T08:56:52.138477+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:28.713552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.116716+0000","last_change":"2024-04-25T08:56:52.107852+0000","last_active":"2024-04-25T08:56:52.116716+0000","last_peered":"2024-04-25T08:56:52.116716+0000","last_clean":"2024-04-25T08:56:52.116716+0000","last_became_active":"2024-04-25T08:56:52.107501+0000","last_became_peered":"2024-04-25T08:56:52.107501+0000","last_unstale":"2024-04-25T08:56:52.116716+0000","last_undegraded":"2024-04-25T08:56:52.116716+0000","last_fullsized":"2024-04-25T08:56:52.116716+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:52.090957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172143+0000","last_change":"2024-04-25T08:56:51.096926+0000","last_active":"2024-04-25T08:56:53.172143+0000","last_peered":"2024-04-25T08:56:53.172143+0000","last_clean":"2024-04-25T08:56:53.172143+0000","last_became_active":"2024-04-25T08:56:51.096509+0000","last_became_peered":"2024-04-25T08:56:51.096509+0000","last_unstale":"2024-04-25T08:56:53.172143+0000","last_undegraded":"2024-04-25T08:56:53.172143+0000","last_fullsized":"2024-04-25T08:56:53.172143+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:50.344453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105916+0000","last_change":"2024-04-25T08:56:52.132092+0000","last_active":"2024-04-25T08:56:53.105916+0000","last_peered":"2024-04-25T08:56:53.105916+0000","last_clean":"2024-04-25T08:56:53.105916+0000","last_became_active":"2024-04-25T08:56:52.131800+0000","last_became_peered":"2024-04-25T08:56:52.131800+0000","last_unstale":"2024-04-25T08:56:53.105916+0000","last_undegraded":"2024-04-25T08:56:53.105916+0000","last_fullsized":"2024-04-25T08:56:53.105916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:22.710685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.130206+0000","last_change":"2024-04-25T08:56:51.092743+0000","last_active":"2024-04-25T08:56:52.130206+0000","last_peered":"2024-04-25T08:56:52.130206+0000","last_clean":"2024-04-25T08:56:52.130206+0000","last_became_active":"2024-04-25T08:56:51.092409+0000","last_became_peered":"2024-04-25T08:56:51.092409+0000","last_unstale":"2024-04-25T08:56:52.130206+0000","last_undegraded":"2024-04-25T08:56:52.130206+0000","last_fullsized":"2024-04-25T08:56:52.130206+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:01.132439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.105968+0000","last_change":"2024-04-25T08:56:52.122590+0000","last_active":"2024-04-25T08:56:53.105968+0000","last_peered":"2024-04-25T08:56:53.105968+0000","last_clean":"2024-04-25T08:56:53.105968+0000","last_became_active":"2024-04-25T08:56:52.118307+0000","last_became_peered":"2024-04-25T08:56:52.118307+0000","last_unstale":"2024-04-25T08:56:53.105968+0000","last_undegraded":"2024-04-25T08:56:53.105968+0000","last_fullsized":"2024-04-25T08:56:53.105968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:56.517596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.091013+0000","last_change":"2024-04-25T08:56:51.107076+0000","last_active":"2024-04-25T08:56:52.091013+0000","last_peered":"2024-04-25T08:56:52.091013+0000","last_clean":"2024-04-25T08:56:52.091013+0000","last_became_active":"2024-04-25T08:56:51.106255+0000","last_became_peered":"2024-04-25T08:56:51.106255+0000","last_unstale":"2024-04-25T08:56:52.091013+0000","last_undegraded":"2024-04-25T08:56:52.091013+0000","last_fullsized":"2024-04-25T08:56:52.091013+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:27.458796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.470240+0000","last_change":"2024-04-25T08:56:52.102863+0000","last_active":"2024-04-25T08:56:53.470240+0000","last_peered":"2024-04-25T08:56:53.470240+0000","last_clean":"2024-04-25T08:56:53.470240+0000","last_became_active":"2024-04-25T08:56:52.101860+0000","last_became_peered":"2024-04-25T08:56:52.101860+0000","last_unstale":"2024-04-25T08:56:53.470240+0000","last_undegraded":"2024-04-25T08:56:53.470240+0000","last_fullsized":"2024-04-25T08:56:53.470240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:40.947555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.095601+0000","last_change":"2024-04-25T08:56:51.103941+0000","last_active":"2024-04-25T08:56:52.095601+0000","last_peered":"2024-04-25T08:56:52.095601+0000","last_clean":"2024-04-25T08:56:52.095601+0000","last_became_active":"2024-04-25T08:56:51.103666+0000","last_became_peered":"2024-04-25T08:56:51.103666+0000","last_unstale":"2024-04-25T08:56:52.095601+0000","last_undegraded":"2024-04-25T08:56:52.095601+0000","last_fullsized":"2024-04-25T08:56:52.095601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:45.435859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108930+0000","last_change":"2024-04-25T08:56:52.116135+0000","last_active":"2024-04-25T08:56:53.108930+0000","last_peered":"2024-04-25T08:56:53.108930+0000","last_clean":"2024-04-25T08:56:53.108930+0000","last_became_active":"2024-04-25T08:56:52.115350+0000","last_became_peered":"2024-04-25T08:56:52.115350+0000","last_unstale":"2024-04-25T08:56:53.108930+0000","last_undegraded":"2024-04-25T08:56:53.108930+0000","last_fullsized":"2024-04-25T08:56:53.108930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:39.598347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.134029+0000","last_change":"2024-04-25T08:56:51.096677+0000","last_active":"2024-04-25T08:56:52.134029+0000","last_peered":"2024-04-25T08:56:52.134029+0000","last_clean":"2024-04-25T08:56:52.134029+0000","last_became_active":"2024-04-25T08:56:51.095944+0000","last_became_peered":"2024-04-25T08:56:51.095944+0000","last_unstale":"2024-04-25T08:56:52.134029+0000","last_undegraded":"2024-04-25T08:56:52.134029+0000","last_fullsized":"2024-04-25T08:56:52.134029+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:58.867701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469426+0000","last_change":"2024-04-25T08:56:52.117121+0000","last_active":"2024-04-25T08:56:53.469426+0000","last_peered":"2024-04-25T08:56:53.469426+0000","last_clean":"2024-04-25T08:56:53.469426+0000","last_became_active":"2024-04-25T08:56:52.115346+0000","last_became_peered":"2024-04-25T08:56:52.115346+0000","last_unstale":"2024-04-25T08:56:53.469426+0000","last_undegraded":"2024-04-25T08:56:53.469426+0000","last_fullsized":"2024-04-25T08:56:53.469426+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:28.290610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.106063+0000","last_change":"2024-04-25T08:56:51.111445+0000","last_active":"2024-04-25T08:56:53.106063+0000","last_peered":"2024-04-25T08:56:53.106063+0000","last_clean":"2024-04-25T08:56:53.106063+0000","last_became_active":"2024-04-25T08:56:51.111060+0000","last_became_peered":"2024-04-25T08:56:51.111060+0000","last_unstale":"2024-04-25T08:56:53.106063+0000","last_undegraded":"2024-04-25T08:56:53.106063+0000","last_fullsized":"2024-04-25T08:56:53.106063+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:20:38.833086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.109757+0000","last_change":"2024-04-25T08:56:52.112324+0000","last_active":"2024-04-25T08:56:53.109757+0000","last_peered":"2024-04-25T08:56:53.109757+0000","last_clean":"2024-04-25T08:56:53.109757+0000","last_became_active":"2024-04-25T08:56:52.110888+0000","last_became_peered":"2024-04-25T08:56:52.110888+0000","last_unstale":"2024-04-25T08:56:53.109757+0000","last_undegraded":"2024-04-25T08:56:53.109757+0000","last_fullsized":"2024-04-25T08:56:53.109757+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:24.734075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108694+0000","last_change":"2024-04-25T08:56:51.118804+0000","last_active":"2024-04-25T08:56:53.108694+0000","last_peered":"2024-04-25T08:56:53.108694+0000","last_clean":"2024-04-25T08:56:53.108694+0000","last_became_active":"2024-04-25T08:56:51.116751+0000","last_became_peered":"2024-04-25T08:56:51.116751+0000","last_unstale":"2024-04-25T08:56:53.108694+0000","last_undegraded":"2024-04-25T08:56:53.108694+0000","last_fullsized":"2024-04-25T08:56:53.108694+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:54.998209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.104647+0000","last_change":"2024-04-25T08:56:52.113845+0000","last_active":"2024-04-25T08:56:53.104647+0000","last_peered":"2024-04-25T08:56:53.104647+0000","last_clean":"2024-04-25T08:56:53.104647+0000","last_became_active":"2024-04-25T08:56:52.113464+0000","last_became_peered":"2024-04-25T08:56:52.113464+0000","last_unstale":"2024-04-25T08:56:53.104647+0000","last_undegraded":"2024-04-25T08:56:53.104647+0000","last_fullsized":"2024-04-25T08:56:53.104647+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:48.191980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.107720+0000","last_change":"2024-04-25T08:56:51.116453+0000","last_active":"2024-04-25T08:56:53.107720+0000","last_peered":"2024-04-25T08:56:53.107720+0000","last_clean":"2024-04-25T08:56:53.107720+0000","last_became_active":"2024-04-25T08:56:51.115817+0000","last_became_peered":"2024-04-25T08:56:51.115817+0000","last_unstale":"2024-04-25T08:56:53.107720+0000","last_undegraded":"2024-04-25T08:56:53.107720+0000","last_fullsized":"2024-04-25T08:56:53.107720+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:42.096820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469547+0000","last_change":"2024-04-25T08:56:52.119509+0000","last_active":"2024-04-25T08:56:53.469547+0000","last_peered":"2024-04-25T08:56:53.469547+0000","last_clean":"2024-04-25T08:56:53.469547+0000","last_became_active":"2024-04-25T08:56:52.118846+0000","last_became_peered":"2024-04-25T08:56:52.118846+0000","last_unstale":"2024-04-25T08:56:53.469547+0000","last_undegraded":"2024-04-25T08:56:53.469547+0000","last_fullsized":"2024-04-25T08:56:53.469547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:43.107847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469427+0000","last_change":"2024-04-25T08:56:51.103526+0000","last_active":"2024-04-25T08:56:53.469427+0000","last_peered":"2024-04-25T08:56:53.469427+0000","last_clean":"2024-04-25T08:56:53.469427+0000","last_became_active":"2024-04-25T08:56:51.102732+0000","last_became_peered":"2024-04-25T08:56:51.102732+0000","last_unstale":"2024-04-25T08:56:53.469427+0000","last_undegraded":"2024-04-25T08:56:53.469427+0000","last_fullsized":"2024-04-25T08:56:53.469427+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:50.344453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469874+0000","last_change":"2024-04-25T08:56:52.107818+0000","last_active":"2024-04-25T08:56:53.469874+0000","last_peered":"2024-04-25T08:56:53.469874+0000","last_clean":"2024-04-25T08:56:53.469874+0000","last_became_active":"2024-04-25T08:56:52.103932+0000","last_became_peered":"2024-04-25T08:56:52.103932+0000","last_unstale":"2024-04-25T08:56:53.469874+0000","last_undegraded":"2024-04-25T08:56:53.469874+0000","last_fullsized":"2024-04-25T08:56:53.469874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:56.009236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.087611+0000","last_change":"2024-04-25T08:56:51.108190+0000","last_active":"2024-04-25T08:56:52.087611+0000","last_peered":"2024-04-25T08:56:52.087611+0000","last_clean":"2024-04-25T08:56:52.087611+0000","last_became_active":"2024-04-25T08:56:51.107716+0000","last_became_peered":"2024-04-25T08:56:51.107716+0000","last_unstale":"2024-04-25T08:56:52.087611+0000","last_undegraded":"2024-04-25T08:56:52.087611+0000","last_fullsized":"2024-04-25T08:56:52.087611+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:14.284634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172529+0000","last_change":"2024-04-25T08:56:52.110793+0000","last_active":"2024-04-25T08:56:53.172529+0000","last_peered":"2024-04-25T08:56:53.172529+0000","last_clean":"2024-04-25T08:56:53.172529+0000","last_became_active":"2024-04-25T08:56:52.110411+0000","last_became_peered":"2024-04-25T08:56:52.110411+0000","last_unstale":"2024-04-25T08:56:53.172529+0000","last_undegraded":"2024-04-25T08:56:53.172529+0000","last_fullsized":"2024-04-25T08:56:53.172529+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:43.107847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.469682+0000","last_change":"2024-04-25T08:56:51.105116+0000","last_active":"2024-04-25T08:56:53.469682+0000","last_peered":"2024-04-25T08:56:53.469682+0000","last_clean":"2024-04-25T08:56:53.469682+0000","last_became_active":"2024-04-25T08:56:51.104439+0000","last_became_peered":"2024-04-25T08:56:51.104439+0000","last_unstale":"2024-04-25T08:56:53.469682+0000","last_undegraded":"2024-04-25T08:56:53.469682+0000","last_fullsized":"2024-04-25T08:56:53.469682+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:47.326064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.172671+0000","last_change":"2024-04-25T08:56:52.101776+0000","last_active":"2024-04-25T08:56:53.172671+0000","last_peered":"2024-04-25T08:56:53.172671+0000","last_clean":"2024-04-25T08:56:53.172671+0000","last_became_active":"2024-04-25T08:56:52.101123+0000","last_became_peered":"2024-04-25T08:56:52.101123+0000","last_unstale":"2024-04-25T08:56:53.172671+0000","last_undegraded":"2024-04-25T08:56:53.172671+0000","last_fullsized":"2024-04-25T08:56:53.172671+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:52.935915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.171474+0000","last_change":"2024-04-25T08:56:51.102221+0000","last_active":"2024-04-25T08:56:53.171474+0000","last_peered":"2024-04-25T08:56:53.171474+0000","last_clean":"2024-04-25T08:56:53.171474+0000","last_became_active":"2024-04-25T08:56:51.101938+0000","last_became_peered":"2024-04-25T08:56:51.101938+0000","last_unstale":"2024-04-25T08:56:53.171474+0000","last_undegraded":"2024-04-25T08:56:53.171474+0000","last_fullsized":"2024-04-25T08:56:53.171474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:14.662323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.108428+0000","last_change":"2024-04-25T08:56:52.107464+0000","last_active":"2024-04-25T08:56:53.108428+0000","last_peered":"2024-04-25T08:56:53.108428+0000","last_clean":"2024-04-25T08:56:53.108428+0000","last_became_active":"2024-04-25T08:56:52.107112+0000","last_became_peered":"2024-04-25T08:56:52.107112+0000","last_unstale":"2024-04-25T08:56:53.108428+0000","last_undegraded":"2024-04-25T08:56:53.108428+0000","last_fullsized":"2024-04-25T08:56:53.108428+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:51.060412+0000","last_clean_scrub_stamp":"2024-04-25T08:56:51.060412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:32:49.015906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T08:56:52.092346+0000","last_change":"2024-04-25T08:56:51.104828+0000","last_active":"2024-04-25T08:56:52.092346+0000","last_peered":"2024-04-25T08:56:52.092346+0000","last_clean":"2024-04-25T08:56:52.092346+0000","last_became_active":"2024-04-25T08:56:51.104398+0000","last_became_peered":"2024-04-25T08:56:51.104398+0000","last_unstale":"2024-04-25T08:56:52.092346+0000","last_undegraded":"2024-04-25T08:56:52.092346+0000","last_fullsized":"2024-04-25T08:56:52.092346+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:56:50.049385+0000","last_clean_scrub_stamp":"2024-04-25T08:56:50.049385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:25.941535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":286,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T08:56:53.107955+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:56:53.107955+0000","last_peered":"2024-04-25T08:56:53.107955+0000","last_clean":"2024-04-25T08:56:53.107955+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:56:53.107955+0000","last_undegraded":"2024-04-25T08:56:53.107955+0000","last_fullsized":"2024-04-25T08:56:53.107955+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1589248,"data_stored":1045183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1003520,"data_stored":456405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":454827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1011712,"data_stored":456509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":995328,"data_stored":455375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":1003520,"data_stored":455535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":999424,"data_stored":454905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1597440,"data_stored":1051853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:56:56.567 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:56:56.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:56:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.746+0000 7f80dd5a4640 1 Processor -- start 2024-04-25T08:56:56.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.746+0000 7f80dd5a4640 1 -- start start 2024-04-25T08:56:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80dd5a4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80d80cf590 0x7f80d80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80dd5a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80d8072000 0x7f80d8072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80dd5a4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80d80cbd20 con 0x7f80d815b6b0 2024-04-25T08:56:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80dd5a4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80d815ba90 con 0x7f80d80cf590 2024-04-25T08:56:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80dd5a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80cc000 con 0x7f80d8072000 2024-04-25T08:56:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d6ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f80d815b6b0 0x7f80d80ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44774/0 (socket says 172.21.15.37:44774) 2024-04-25T08:56:56.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d6ffd640 1 -- 172.21.15.37:0/2666279286 learned_addr learned my addr 172.21.15.37:0/2666279286 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:56.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d67fc640 1 --2- 172.21.15.37:0/2666279286 >> v2:172.21.15.37:3300/0 conn(0x7f80d80cf590 0x7f80d80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d77fe640 1 --2- 172.21.15.37:0/2666279286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80d8072000 0x7f80d8072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d5ffb640 1 -- 172.21.15.37:0/2666279286 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963657655 0 0) 0x7f80d80cbd20 con 0x7f80d815b6b0 2024-04-25T08:56:56.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d5ffb640 1 -- 172.21.15.37:0/2666279286 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80c0003530 con 0x7f80d815b6b0 2024-04-25T08:56:56.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d67fc640 1 -- 172.21.15.37:0/2666279286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80d8072000 msgr2=0x7f80d8072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:56.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d67fc640 1 --2- 172.21.15.37:0/2666279286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80d8072000 0x7f80d8072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d67fc640 1 -- 172.21.15.37:0/2666279286 >> v1:172.21.15.37:6789/0 conn(0x7f80d815b6b0 legacy=0x7f80d80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d67fc640 1 -- 172.21.15.37:0/2666279286 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d80cc2e0 con 0x7f80d80cf590 2024-04-25T08:56:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d67fc640 1 --2- 172.21.15.37:0/2666279286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80d80cf590 0x7f80d80cfa20 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0097c0 tx=0x7f80cc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ff68f15c8bb3a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d5ffb640 1 -- 172.21.15.37:0/2666279286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0b2070 con 0x7f80d80cf590 2024-04-25T08:56:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d5ffb640 1 -- 172.21.15.37:0/2666279286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80cc0ad640 con 0x7f80d80cf590 2024-04-25T08:56:56.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.750+0000 7f80d5ffb640 1 -- 172.21.15.37:0/2666279286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc0ada10 con 0x7f80d80cf590 2024-04-25T08:56:56.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2666279286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80d80cf590 msgr2=0x7f80d80cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 --2- 172.21.15.37:0/2666279286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80d80cf590 0x7f80d80cfa20 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0097c0 tx=0x7f80cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2666279286 shutdown_connections 2024-04-25T08:56:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 --2- 172.21.15.37:0/2666279286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80d8072000 0x7f80d8072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 --2- 172.21.15.37:0/2666279286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80d80cf590 0x7f80d80cfa20 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2666279286 >> 172.21.15.37:0/2666279286 conn(0x7f80d806ae60 msgr2=0x7f80d806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2666279286 shutdown_connections 2024-04-25T08:56:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2666279286 wait complete. 2024-04-25T08:56:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 Processor -- start 2024-04-25T08:56:56.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 -- start start 2024-04-25T08:56:56.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80d8072000 0x7f80d8098210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80d80cf590 0x7f80d80c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d815b6b0 0x7f80d8152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80caf30 con 0x7f80d8072000 2024-04-25T08:56:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d80cadb0 con 0x7f80d815b6b0 2024-04-25T08:56:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80d81682a0 con 0x7f80d80cf590 2024-04-25T08:56:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80d8072000 0x7f80d8098210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80d8072000 0x7f80d8098210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39158/0 (socket says 172.21.15.37:39158) 2024-04-25T08:56:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d6ffd640 1 -- 172.21.15.37:0/2963532756 learned_addr learned my addr 172.21.15.37:0/2963532756 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d67fc640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80d80cf590 0x7f80d80c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d77fe640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d815b6b0 0x7f80d8152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d6ffd640 1 -- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80d80cf590 msgr2=0x7f80d80c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d6ffd640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80d80cf590 0x7f80d80c1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d6ffd640 1 -- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d815b6b0 msgr2=0x7f80d8152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d6ffd640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d815b6b0 0x7f80d8152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d6ffd640 1 -- 172.21.15.37:0/2963532756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d8098970 con 0x7f80d8072000 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d77fe640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d815b6b0 0x7f80d8152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80d6ffd640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80d8072000 0x7f80d8098210 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f80c800c3f0 tx=0x7f80c800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80bf7fe640 1 -- 172.21.15.37:0/2963532756 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8017070 con 0x7f80d8072000 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80bf7fe640 1 -- 172.21.15.37:0/2963532756 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c8005cf0 con 0x7f80d8072000 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2963532756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d8151cb0 con 0x7f80d8072000 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80bf7fe640 1 -- 172.21.15.37:0/2963532756 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80072f0 con 0x7f80d8072000 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.754+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2963532756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d8097cf0 con 0x7f80d8072000 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.758+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2963532756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d804ef80 con 0x7f80d8072000 2024-04-25T08:56:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.758+0000 7f80bf7fe640 1 -- 172.21.15.37:0/2963532756 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80c801a070 con 0x7f80d8072000 2024-04-25T08:56:56.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.758+0000 7f80bf7fe640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80c4066f60 0x7f80c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.758+0000 7f80bf7fe640 1 -- 172.21.15.37:0/2963532756 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80c80844c0 con 0x7f80d8072000 2024-04-25T08:56:56.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.758+0000 7f80d67fc640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80c4066f60 0x7f80c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.758+0000 7f80d67fc640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80c4066f60 0x7f80c4069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0af000 tx=0x7f80cc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:56.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.762+0000 7f80bf7fe640 1 -- 172.21.15.37:0/2963532756 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f80c8054d80 con 0x7f80d8072000 2024-04-25T08:56:56.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.982+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2963532756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f80d8097f30 con 0x7f80d8072000 2024-04-25T08:56:56.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.986+0000 7f80bf7fe640 1 -- 172.21.15.37:0/2963532756 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f80c805b0d0 con 0x7f80d8072000 2024-04-25T08:56:56.976 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T08:56:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA== 2024-04-25T08:56:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-04-25T08:56:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-04-25T08:56:56.976 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:56:56.977 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.986+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80c4066f60 msgr2=0x7f80c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.986+0000 7f80dd5a4640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80c4066f60 0x7f80c4069420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f80cc0af000 tx=0x7f80cc008040 comp rx=0 tx=0).stop 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.986+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2963532756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80d8072000 msgr2=0x7f80d8098210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.986+0000 7f80dd5a4640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80d8072000 0x7f80d8098210 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f80c800c3f0 tx=0x7f80c800c8b0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.986+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2963532756 shutdown_connections 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.986+0000 7f80dd5a4640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80c4066f60 0x7f80c4069420 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.990+0000 7f80dd5a4640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80d815b6b0 0x7f80d8152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.990+0000 7f80dd5a4640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80d80cf590 0x7f80d80c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.990+0000 7f80dd5a4640 1 --2- 172.21.15.37:0/2963532756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80d8072000 0x7f80d8098210 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.990+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2963532756 >> 172.21.15.37:0/2963532756 conn(0x7f80d806ae60 msgr2=0x7f80d80b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.990+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2963532756 shutdown_connections 2024-04-25T08:56:56.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:56.990+0000 7f80dd5a4640 1 -- 172.21.15.37:0/2963532756 wait complete. 2024-04-25T08:56:56.996 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:57.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85cee8d640 1 Processor -- start 2024-04-25T08:56:57.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85cee8d640 1 -- start start 2024-04-25T08:56:57.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85cee8d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f85c80ce510 0x7f85c80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85cee8d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f85c8144ca0 0x7f85c806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85cee8d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c80cee30 con 0x7f85c806ec60 2024-04-25T08:56:57.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85cee8d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80cc2b0 con 0x7f85c80ce510 2024-04-25T08:56:57.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85cee8d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c806e290 con 0x7f85c8144ca0 2024-04-25T08:56:57.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85ccc29640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f85c80ce510 0x7f85c80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85cd42a640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f85c806ec60 0x7f85c8142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44786/0 (socket says 172.21.15.37:44786) 2024-04-25T08:56:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85cd42a640 1 -- 172.21.15.37:0/1765084266 learned_addr learned my addr 172.21.15.37:0/1765084266 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85c7fff640 1 --2- 172.21.15.37:0/1765084266 >> v2:172.21.15.203:3300/0 conn(0x7f85c8144ca0 0x7f85c806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85ccc29640 1 -- 172.21.15.37:0/1765084266 >> v2:172.21.15.203:3300/0 conn(0x7f85c8144ca0 msgr2=0x7f85c806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85ccc29640 1 --2- 172.21.15.37:0/1765084266 >> v2:172.21.15.203:3300/0 conn(0x7f85c8144ca0 0x7f85c806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85ccc29640 1 -- 172.21.15.37:0/1765084266 >> v1:172.21.15.37:6789/0 conn(0x7f85c806ec60 legacy=0x7f85c8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85c77fe640 1 -- 172.21.15.37:0/1765084266 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4088858502 0 0) 0x7f85c80cee30 con 0x7f85c806ec60 2024-04-25T08:56:57.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.126+0000 7f85ccc29640 1 -- 172.21.15.37:0/1765084266 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80ccc70 con 0x7f85c80ce510 2024-04-25T08:56:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85ccc29640 1 --2- 172.21.15.37:0/1765084266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c80ce510 0x7f85c80ce8f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f85c8070fe0 tx=0x7f85b801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc2e133879534dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85c77fe640 1 -- 172.21.15.37:0/1765084266 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b80b5070 con 0x7f85c80ce510 2024-04-25T08:56:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85c77fe640 1 -- 172.21.15.37:0/1765084266 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b80a8970 con 0x7f85c80ce510 2024-04-25T08:56:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85c77fe640 1 -- 172.21.15.37:0/1765084266 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b80a8d40 con 0x7f85c80ce510 2024-04-25T08:56:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 -- 172.21.15.37:0/1765084266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c80ce510 msgr2=0x7f85c80ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 --2- 172.21.15.37:0/1765084266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c80ce510 0x7f85c80ce8f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f85c8070fe0 tx=0x7f85b801ec70 comp rx=0 tx=0).stop 2024-04-25T08:56:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 -- 172.21.15.37:0/1765084266 shutdown_connections 2024-04-25T08:56:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 --2- 172.21.15.37:0/1765084266 >> v2:172.21.15.203:3300/0 conn(0x7f85c8144ca0 0x7f85c806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 --2- 172.21.15.37:0/1765084266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c80ce510 0x7f85c80ce8f0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 -- 172.21.15.37:0/1765084266 >> 172.21.15.37:0/1765084266 conn(0x7f85c80b11a0 msgr2=0x7f85c80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 -- 172.21.15.37:0/1765084266 shutdown_connections 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 -- 172.21.15.37:0/1765084266 wait complete. 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 Processor -- start 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 -- start start 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c806ec60 0x7f85c80add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c80ce510 0x7f85c813bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c8144ca0 0x7f85c80b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80cc900 con 0x7f85c8144ca0 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c806df20 con 0x7f85c80ce510 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8142d90 con 0x7f85c806ec60 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cd42a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c8144ca0 0x7f85c80b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cd42a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c8144ca0 0x7f85c80b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39174/0 (socket says 172.21.15.37:39174) 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cd42a640 1 -- 172.21.15.37:0/2005404473 learned_addr learned my addr 172.21.15.37:0/2005404473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:57.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85ccc29640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c806ec60 0x7f85c80add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85c7fff640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c80ce510 0x7f85c813bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cd42a640 1 -- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c806ec60 msgr2=0x7f85c80add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cd42a640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c806ec60 0x7f85c80add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cd42a640 1 -- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c80ce510 msgr2=0x7f85c813bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cd42a640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c80ce510 0x7f85c813bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cd42a640 1 -- 172.21.15.37:0/2005404473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c81e0c90 con 0x7f85c8144ca0 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85c7fff640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c80ce510 0x7f85c813bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cd42a640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c8144ca0 0x7f85c80b3ff0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f85c000c7a0 tx=0x7f85c000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85c57fa640 1 -- 172.21.15.37:0/2005404473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0018070 con 0x7f85c8144ca0 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85c57fa640 1 -- 172.21.15.37:0/2005404473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c000a640 con 0x7f85c8144ca0 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85c57fa640 1 -- 172.21.15.37:0/2005404473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c000aab0 con 0x7f85c8144ca0 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 -- 172.21.15.37:0/2005404473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c81e0f30 con 0x7f85c8144ca0 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.130+0000 7f85cee8d640 1 -- 172.21.15.37:0/2005404473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c80ad500 con 0x7f85c8144ca0 2024-04-25T08:56:57.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.134+0000 7f85cee8d640 1 -- 172.21.15.37:0/2005404473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c80ad2f0 con 0x7f85c8144ca0 2024-04-25T08:56:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.134+0000 7f85c57fa640 1 -- 172.21.15.37:0/2005404473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f85c001b070 con 0x7f85c8144ca0 2024-04-25T08:56:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.134+0000 7f85c57fa640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f85b4066f60 0x7f85b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.134+0000 7f85c57fa640 1 -- 172.21.15.37:0/2005404473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85c00847b0 con 0x7f85c8144ca0 2024-04-25T08:56:57.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.134+0000 7f85ccc29640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f85b4066f60 0x7f85b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.134+0000 7f85ccc29640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f85b4066f60 0x7f85b4069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f85b80b2000 tx=0x7f85b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.138+0000 7f85c57fa640 1 -- 172.21.15.37:0/2005404473 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f85c0055630 con 0x7f85c8144ca0 2024-04-25T08:56:57.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 -- 172.21.15.37:0/2005404473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85c80b1cc0 con 0x7f85c8144ca0 2024-04-25T08:56:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85c57fa640 1 -- 172.21.15.37:0/2005404473 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f85c0054f30 con 0x7f85c8144ca0 2024-04-25T08:56:57.320 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:56:57.320 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:56:54.486633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/4130042812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4130042812},{"type":"v1","addr":"172.21.15.37:6835","nonce":4130042812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:56:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 44 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 -- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f85b4066f60 msgr2=0x7f85b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f85b4066f60 0x7f85b4069420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f85b80b2000 tx=0x7f85b8008040 comp rx=0 tx=0).stop 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 -- 172.21.15.37:0/2005404473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c8144ca0 msgr2=0x7f85c80b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c8144ca0 0x7f85c80b3ff0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f85c000c7a0 tx=0x7f85c000cc60 comp rx=0 tx=0).stop 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 -- 172.21.15.37:0/2005404473 shutdown_connections 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f85b4066f60 0x7f85b4069420 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85c8144ca0 0x7f85c80b3ff0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f85c80ce510 0x7f85c813bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 --2- 172.21.15.37:0/2005404473 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f85c806ec60 0x7f85c80add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 -- 172.21.15.37:0/2005404473 >> 172.21.15.37:0/2005404473 conn(0x7f85c80b11a0 msgr2=0x7f85c8071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 -- 172.21.15.37:0/2005404473 shutdown_connections 2024-04-25T08:56:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.330+0000 7f85cee8d640 1 -- 172.21.15.37:0/2005404473 wait complete. 2024-04-25T08:56:57.740 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:56:51.447805+0000', 'modified': '2024-04-25T08:56:54.486633+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6930}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6930': {'gid': 6930, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/4130042812', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 4130042812}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 4130042812}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:56:57.741 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:56:57.741 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:56:57.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aae307640 1 Processor -- start 2024-04-25T08:56:57.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aae307640 1 -- start start 2024-04-25T08:56:57.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aae307640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3aa81451f0 0x7f3aa8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aae307640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa80ccc70 con 0x7f3aa8144ca0 2024-04-25T08:56:57.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aae307640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aa806dc40 con 0x7f3aa80ce510 2024-04-25T08:56:57.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aae307640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aa80cc2b0 con 0x7f3aa81451f0 2024-04-25T08:56:57.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa77fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3aa8144ca0 0x7f3aa806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44796/0 (socket says 172.21.15.37:44796) 2024-04-25T08:56:57.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa77fe640 1 -- 172.21.15.37:0/1375844114 learned_addr learned my addr 172.21.15.37:0/1375844114 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aac8a4640 1 --2- 172.21.15.37:0/1375844114 >> v2:172.21.15.203:3300/0 conn(0x7f3aa81451f0 0x7f3aa8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2310365291 0 0) 0x7f3aa80ccc70 con 0x7f3aa8144ca0 2024-04-25T08:56:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a90003530 con 0x7f3aa8144ca0 2024-04-25T08:56:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3122378119 0 0) 0x7f3a90003530 con 0x7f3aa8144ca0 2024-04-25T08:56:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 >> v2:172.21.15.203:3300/0 conn(0x7f3aa81451f0 msgr2=0x7f3aa8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/1375844114 >> v2:172.21.15.203:3300/0 conn(0x7f3aa81451f0 0x7f3aa8142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 >> v1:172.21.15.203:6789/0 conn(0x7f3aa80ce510 legacy=0x7f3aa80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa806df20 con 0x7f3aa8144ca0 2024-04-25T08:56:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3a9c001680 con 0x7f3aa8144ca0 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 >> v1:172.21.15.37:6789/0 conn(0x7f3aa8144ca0 legacy=0x7f3aa806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a90003a60 0x7f3a90003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a90004810 0x7f3a90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a900051d0 0x7f3a9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80cc2b0 con 0x7f3a90003a60 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9000c150 con 0x7f3a90004810 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9000c350 con 0x7f3a900051d0 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a90003a60 0x7f3a90003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aac8a4640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a900051d0 0x7f3a9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a90004810 0x7f3a90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa7fff640 1 -- 172.21.15.37:0/1375844114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a900051d0 msgr2=0x7f3a9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a900051d0 0x7f3a9000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa7fff640 1 -- 172.21.15.37:0/1375844114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a90004810 msgr2=0x7f3a90004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a90004810 0x7f3a90004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa7fff640 1 -- 172.21.15.37:0/1375844114 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a90003530 con 0x7f3a90003a60 2024-04-25T08:56:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aac8a4640 1 -- 172.21.15.37:0/1375844114 reap_dead start 2024-04-25T08:56:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.886+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3a90004810 0x7f3a90004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a90003a60 0x7f3a90003e80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f3a98001780 tx=0x7f3a9801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8e9acd88702918a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a980013a0 con 0x7f3a90003a60 2024-04-25T08:56:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a980052f0 con 0x7f3a90003a60 2024-04-25T08:56:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aa6ffd640 1 -- 172.21.15.37:0/1375844114 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98003840 con 0x7f3a90003a60 2024-04-25T08:56:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aae307640 1 -- 172.21.15.37:0/1375844114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a90003a60 msgr2=0x7f3a90003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aae307640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a90003a60 0x7f3a90003e80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f3a98001780 tx=0x7f3a9801c400 comp rx=0 tx=0).stop 2024-04-25T08:56:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aae307640 1 -- 172.21.15.37:0/1375844114 shutdown_connections 2024-04-25T08:56:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aae307640 1 --2- 172.21.15.37:0/1375844114 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a90003a60 0x7f3a90003e80 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aae307640 1 -- 172.21.15.37:0/1375844114 >> 172.21.15.37:0/1375844114 conn(0x7f3aa80b11a0 msgr2=0x7f3aa80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aae307640 1 -- 172.21.15.37:0/1375844114 shutdown_connections 2024-04-25T08:56:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aae307640 1 -- 172.21.15.37:0/1375844114 wait complete. 2024-04-25T08:56:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aae307640 1 Processor -- start 2024-04-25T08:56:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.890+0000 7f3aae307640 1 -- start start 2024-04-25T08:56:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aae307640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a90003a60 0x7f3aa8141870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aae307640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa81e26c0 0x7f3aa80ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aae307640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa80ae600 0x7f3aa80c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aae307640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80ce8f0 con 0x7f3aa81e26c0 2024-04-25T08:56:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aae307640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa806e290 con 0x7f3aa80ae600 2024-04-25T08:56:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aae307640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa80cea70 con 0x7f3a90003a60 2024-04-25T08:56:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa81e26c0 0x7f3aa80ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa81e26c0 0x7f3aa80ae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39184/0 (socket says 172.21.15.37:39184) 2024-04-25T08:56:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa77fe640 1 -- 172.21.15.37:0/3731148647 learned_addr learned my addr 172.21.15.37:0/3731148647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aac8a4640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa80ae600 0x7f3aa80c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a90003a60 0x7f3aa8141870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa77fe640 1 -- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a90003a60 msgr2=0x7f3aa8141870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a90003a60 0x7f3aa8141870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa77fe640 1 -- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa80ae600 msgr2=0x7f3aa80c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa80ae600 0x7f3aa80c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa77fe640 1 -- 172.21.15.37:0/3731148647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa80b0db0 con 0x7f3aa81e26c0 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aac8a4640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa80ae600 0x7f3aa80c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa81e26c0 0x7f3aa80ae0c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c010870 tx=0x7f3a9c010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa4ff9640 1 -- 172.21.15.37:0/3731148647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c01b070 con 0x7f3aa81e26c0 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa4ff9640 1 -- 172.21.15.37:0/3731148647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c011040 con 0x7f3aa81e26c0 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aae307640 1 -- 172.21.15.37:0/3731148647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aa81e2b80 con 0x7f3aa81e26c0 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa4ff9640 1 -- 172.21.15.37:0/3731148647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c00ec90 con 0x7f3aa81e26c0 2024-04-25T08:56:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aae307640 1 -- 172.21.15.37:0/3731148647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aa81e3140 con 0x7f3aa81e26c0 2024-04-25T08:56:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa4ff9640 1 -- 172.21.15.37:0/3731148647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a9c01e070 con 0x7f3aa81e26c0 2024-04-25T08:56:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa4ff9640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a94066f60 0x7f3a94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa4ff9640 1 -- 172.21.15.37:0/3731148647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a9c087af0 con 0x7f3aa81e26c0 2024-04-25T08:56:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.894+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a94066f60 0x7f3a94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 Processor -- start 2024-04-25T08:56:57.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 -- start start 2024-04-25T08:56:57.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a94066f60 0x7f3a94069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3a98004000 tx=0x7f3a980011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc366d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc367240 0x556bfc31c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc31d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bfc2045f0 con 0x556bfc366920 2024-04-25T08:56:57.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bfc16bda0 con 0x556bfc31cf30 2024-04-25T08:56:57.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bfc0f4150 con 0x556bfc367240 2024-04-25T08:56:57.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc366d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aa7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc366d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39186/0 (socket says 172.21.15.37:39186) 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aa7fff640 1 -- 172.21.15.37:0/1091646671 learned_addr learned my addr 172.21.15.37:0/1091646671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aac8a4640 1 --2- 172.21.15.37:0/1091646671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc31d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/1091646671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc367240 0x556bfc31c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aa7fff640 1 -- 172.21.15.37:0/1091646671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc367240 msgr2=0x556bfc31c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/1091646671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc367240 0x556bfc31c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aa7fff640 1 -- 172.21.15.37:0/1091646671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 msgr2=0x556bfc31d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/1091646671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc31d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aa7fff640 1 -- 172.21.15.37:0/1091646671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bfc10dca0 con 0x556bfc366920 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/1091646671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc366d00 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f3a98097e10 tx=0x7f3a98001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f788ba6084f5609 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3a8dffb640 1 -- 172.21.15.37:0/1091646671 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a980c5070 con 0x556bfc366920 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3a8dffb640 1 -- 172.21.15.37:0/1091646671 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a98007690 con 0x556bfc366920 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3a8dffb640 1 -- 172.21.15.37:0/1091646671 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a980069e0 con 0x556bfc366920 2024-04-25T08:56:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 -- 172.21.15.37:0/1091646671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 msgr2=0x556bfc366d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 --2- 172.21.15.37:0/1091646671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc366d00 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f3a98097e10 tx=0x7f3a98001130 comp rx=0 tx=0).stop 2024-04-25T08:56:57.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 -- 172.21.15.37:0/1091646671 shutdown_connections 2024-04-25T08:56:57.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 --2- 172.21.15.37:0/1091646671 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc31d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 --2- 172.21.15.37:0/1091646671 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc367240 0x556bfc31c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 --2- 172.21.15.37:0/1091646671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc366d00 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 -- 172.21.15.37:0/1091646671 >> 172.21.15.37:0/1091646671 conn(0x556bfc3232a0 msgr2=0x556bfc31b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:57.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 -- 172.21.15.37:0/1091646671 shutdown_connections 2024-04-25T08:56:57.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 -- 172.21.15.37:0/1091646671 wait complete. 2024-04-25T08:56:57.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 -- start start 2024-04-25T08:56:57.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 Processor -- start 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.898+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc36dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc366920 0x556bfc36e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc320aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bfc126990 con 0x556bfc367240 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bfc167e40 con 0x556bfc31cf30 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bfc0f7900 con 0x556bfc366920 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 10 client.0 _ll_register_callbacks cb 0x556bfc21f1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aac8a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc320aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aac8a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc320aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39200/0 (socket says 172.21.15.37:39200) 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aac8a4640 1 -- 172.21.15.37:0/2737726362 learned_addr learned my addr 172.21.15.37:0/2737726362 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc36dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc366920 0x556bfc36e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aac8a4640 1 -- 172.21.15.37:0/2737726362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc366920 msgr2=0x556bfc36e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aac8a4640 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc366920 0x556bfc36e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aac8a4640 1 -- 172.21.15.37:0/2737726362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 msgr2=0x556bfc36dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aac8a4640 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc36dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aac8a4640 1 -- 172.21.15.37:0/2737726362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556bfc124630 con 0x556bfc367240 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aac8a4640 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc320aa0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3aa806c9d0 tx=0x7f3aa00046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ae9a29db5d1da5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3a86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3a86ffd640 1 -- 172.21.15.37:0/2737726362 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa001b030 con 0x556bfc367240 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 -- 172.21.15.37:0/2737726362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556bfc107a50 con 0x556bfc367240 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3a86ffd640 1 -- 172.21.15.37:0/2737726362 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3aa0012040 con 0x556bfc367240 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3a86ffd640 1 -- 172.21.15.37:0/2737726362 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aa0014a00 con 0x556bfc367240 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 10 client.7083 fetch_fsmap learned FSMap version 44 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 -- 172.21.15.37:0/2737726362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x556bfc0f43a0 con 0x556bfc367240 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3a86ffd640 1 -- 172.21.15.37:0/2737726362 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556bfc107a50 con 0x556bfc367240 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3a86ffd640 1 -- 172.21.15.37:0/2737726362 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3aa002a080 con 0x556bfc367240 2024-04-25T08:56:57.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 10 client.7083 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 10 client.7083 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 20 client.7083 populate_metadata read hostname 'smithi037' 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.37:6834/4130042812,v1:172.21.15.37:6835/4130042812] conn(0x556bfc322160 0x556bfc377ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 4 client.7083 mds_command: new command op to 6930 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 -- 172.21.15.37:0/2737726362 --> [v2:172.21.15.37:6834/4130042812,v1:172.21.15.37:6835/4130042812] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556bfc1792e0 con 0x556bfc322160 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.37:6834/4130042812,v1:172.21.15.37:6835/4130042812] conn(0x556bfc322160 0x556bfc377ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.37:6834/4130042812,v1:172.21.15.37:6835/4130042812] conn(0x556bfc322160 0x556bfc377ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb4e7c3bce5b36d server_cookie=23181aac7a9ea2a9 in_seq=0 out_seq=0 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3a86ffd640 10 client.7083 ms_handle_connect on v2:172.21.15.37:6834/4130042812 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3a86ffd640 1 -- 172.21.15.37:0/2737726362 <== mds.0 v2:172.21.15.37:6834/4130042812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556bfc1792e0 con 0x556bfc322160 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3a86ffd640 10 client.7083 handle_command_reply: tid=0 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 client.7083 shutdown 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 -- 172.21.15.37:0/2737726362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 msgr2=0x556bfc320aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc320aa0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3aa806c9d0 tx=0x7f3aa00046e0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 -- 172.21.15.37:0/2737726362 shutdown_connections 2024-04-25T08:56:57.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.37:6834/4130042812,v1:172.21.15.37:6835/4130042812] conn(0x556bfc322160 0x556bfc377ce0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc320aa0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc366920 0x556bfc36e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 --2- 172.21.15.37:0/2737726362 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc36dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 -- 172.21.15.37:0/2737726362 >> 172.21.15.37:0/2737726362 conn(0x556bfc3232a0 msgr2=0x556bfc31b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:57.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3a86ffd640 0 client.7083 ms_handle_reset on v2:172.21.15.37:6834/4130042812 2024-04-25T08:56:57.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 -- 172.21.15.37:0/2737726362 shutdown_connections 2024-04-25T08:56:57.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 1 -- 172.21.15.37:0/2737726362 wait complete. 2024-04-25T08:56:57.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.902+0000 7f3ab1474000 20 client.7083 trim_cache size 0 max 16384 2024-04-25T08:56:57.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3ab1474000 1 Processor -- start 2024-04-25T08:56:57.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3ab1474000 1 -- start start 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc3202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc320800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc367240 0x556bfc36f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3ab1474000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa81ffb00 con 0x556bfc366920 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3ab1474000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa81fef30 con 0x556bfc31cf30 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3ab1474000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aa81ff500 con 0x556bfc367240 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc320800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3aa77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc320800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39204/0 (socket says 172.21.15.37:39204) 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3aa77fe640 1 -- 172.21.15.37:0/659737560 learned_addr learned my addr 172.21.15.37:0/659737560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3aa77fe640 1 -- 172.21.15.37:0/659737560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc367240 msgr2=0x556bfc36f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/659737560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc367240 0x556bfc36f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3aa77fe640 1 -- 172.21.15.37:0/659737560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 msgr2=0x556bfc3202c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/659737560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc3202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3aa77fe640 1 -- 172.21.15.37:0/659737560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556bfc367070 con 0x556bfc366920 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/659737560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc320800 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c057f70 tx=0x7f3a9c057d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f518f2869df4c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3a8d7fa640 1 -- 172.21.15.37:0/659737560 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c05c3c0 con 0x556bfc366920 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3a8d7fa640 1 -- 172.21.15.37:0/659737560 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c084070 con 0x556bfc366920 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.918+0000 7f3a8d7fa640 1 -- 172.21.15.37:0/659737560 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c060980 con 0x556bfc366920 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/659737560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 msgr2=0x556bfc320800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- 172.21.15.37:0/659737560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc320800 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c057f70 tx=0x7f3a9c057d00 comp rx=0 tx=0).stop 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/659737560 shutdown_connections 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- 172.21.15.37:0/659737560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc367240 0x556bfc36f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- 172.21.15.37:0/659737560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc366920 0x556bfc320800 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- 172.21.15.37:0/659737560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc3202c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/659737560 >> 172.21.15.37:0/659737560 conn(0x556bfc3232a0 msgr2=0x556bfc31fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/659737560 shutdown_connections 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/659737560 wait complete. 2024-04-25T08:56:57.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- start start 2024-04-25T08:56:57.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 Processor -- start 2024-04-25T08:56:57.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc372e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc366920 0x556bfc3aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc3aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bfc1703d0 con 0x556bfc367240 2024-04-25T08:56:57.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556bfc227d50 con 0x556bfc31cf30 2024-04-25T08:56:57.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x556bfc106980 con 0x556bfc366920 2024-04-25T08:56:57.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 10 client.0 _ll_register_callbacks cb 0x7f3a90003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aac8a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc3aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aac8a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc3aaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39214/0 (socket says 172.21.15.37:39214) 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aac8a4640 1 -- 172.21.15.37:0/641513097 learned_addr learned my addr 172.21.15.37:0/641513097 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc372e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aa77fe640 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc366920 0x556bfc3aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aac8a4640 1 -- 172.21.15.37:0/641513097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc366920 msgr2=0x556bfc3aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aac8a4640 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc366920 0x556bfc3aa4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aac8a4640 1 -- 172.21.15.37:0/641513097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 msgr2=0x556bfc372e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aac8a4640 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc372e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aac8a4640 1 -- 172.21.15.37:0/641513097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556bfc1ed600 con 0x556bfc367240 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc372e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aac8a4640 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc3aaa20 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0000f90 tx=0x7f3aa0004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a09318d5b3cd48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3a86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3a86ffd640 1 -- 172.21.15.37:0/641513097 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa001b040 con 0x556bfc367240 2024-04-25T08:56:57.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3a86ffd640 1 -- 172.21.15.37:0/641513097 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3aa0020410 con 0x556bfc367240 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/641513097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a9c084070 con 0x556bfc367240 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3a86ffd640 1 -- 172.21.15.37:0/641513097 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3aa0037050 con 0x556bfc367240 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 10 client.7089 fetch_fsmap learned FSMap version 44 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/641513097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x556bfc1009b0 con 0x556bfc367240 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3a86ffd640 1 -- 172.21.15.37:0/641513097 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c084070 con 0x556bfc367240 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3a86ffd640 1 -- 172.21.15.37:0/641513097 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3aa003a020 con 0x556bfc367240 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 10 client.7089 fetch_fsmap finished waiting for FSMap version 44 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 10 client.7089 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 20 client.7089 populate_metadata read hostname 'smithi037' 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.37:6834/4130042812,v1:172.21.15.37:6835/4130042812] conn(0x556bfc3733b0 0x556bfc3ab230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 4 client.7089 mds_command: new command op to 6930 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/641513097 --> [v2:172.21.15.37:6834/4130042812,v1:172.21.15.37:6835/4130042812] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556bfc173630 con 0x556bfc3733b0 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.37:6834/4130042812,v1:172.21.15.37:6835/4130042812] conn(0x556bfc3733b0 0x556bfc3ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3aa7fff640 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.37:6834/4130042812,v1:172.21.15.37:6835/4130042812] conn(0x556bfc3733b0 0x556bfc3ab230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22ec828a9f2b42a8 server_cookie=ede61ee68658a84a in_seq=0 out_seq=0 2024-04-25T08:56:57.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3a86ffd640 10 client.7089 ms_handle_connect on v2:172.21.15.37:6834/4130042812 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3a86ffd640 1 -- 172.21.15.37:0/641513097 <== mds.0 v2:172.21.15.37:6834/4130042812 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x556bfc173630 con 0x556bfc3733b0 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3a86ffd640 10 client.7089 handle_command_reply: tid=0 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 client.7089 shutdown 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/641513097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 msgr2=0x556bfc3aaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc3aaa20 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0000f90 tx=0x7f3aa0004a50 comp rx=0 tx=0).stop 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/641513097 shutdown_connections 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.37:6834/4130042812,v1:172.21.15.37:6835/4130042812] conn(0x556bfc3733b0 0x556bfc3ab230 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556bfc367240 0x556bfc3aaa20 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556bfc366920 0x556bfc3aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 --2- 172.21.15.37:0/641513097 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556bfc31cf30 0x556bfc372e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3a86ffd640 0 client.7089 ms_handle_reset on v2:172.21.15.37:6834/4130042812 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/641513097 >> 172.21.15.37:0/641513097 conn(0x556bfc3232a0 msgr2=0x556bfc3202c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/641513097 shutdown_connections 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 1 -- 172.21.15.37:0/641513097 wait complete. 2024-04-25T08:56:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.922+0000 7f3ab1474000 20 client.7089 trim_cache size 0 max 16384 2024-04-25T08:56:57.920 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:56:57.920 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T08:56:57.920 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T08:56:57.920 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 6930, 2024-04-25T08:56:57.920 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T08:56:57.920 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T08:56:57.920 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T08:56:57.920 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 5.8236200340000002, 2024-04-25T08:56:57.920 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 44, 2024-04-25T08:56:57.920 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 82, 2024-04-25T08:56:57.920 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 82, 2024-04-25T08:56:57.921 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 10.84193245 2024-04-25T08:56:57.921 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:56:57.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 -- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a94066f60 msgr2=0x7f3a94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a94066f60 0x7f3a94069420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3a98004000 tx=0x7f3a980011a0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 -- 172.21.15.37:0/3731148647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa81e26c0 msgr2=0x7f3aa80ae0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa81e26c0 0x7f3aa80ae0c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c010870 tx=0x7f3a9c010d30 comp rx=0 tx=0).stop 2024-04-25T08:56:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 -- 172.21.15.37:0/3731148647 shutdown_connections 2024-04-25T08:56:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a94066f60 0x7f3a94069420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aa80ae600 0x7f3aa80c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aa81e26c0 0x7f3aa80ae0c0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 --2- 172.21.15.37:0/3731148647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3a90003a60 0x7f3aa8141870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 -- 172.21.15.37:0/3731148647 >> 172.21.15.37:0/3731148647 conn(0x7f3aa80b11a0 msgr2=0x7f3aa80a8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 -- 172.21.15.37:0/3731148647 shutdown_connections 2024-04-25T08:56:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:57.926+0000 7f3aae307640 1 -- 172.21.15.37:0/3731148647 wait complete. 2024-04-25T08:56:57.931 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:56:58.044 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T08:56:58.054+0000 7f3ab9a74640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:56:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.054+0000 7fa34e302640 1 Processor -- start 2024-04-25T08:56:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.054+0000 7fa34e302640 1 -- start start 2024-04-25T08:56:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa34e302640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa34815d180 0x7fa34815d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa34e302640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa34806b710 con 0x7fa3480cf840 2024-04-25T08:56:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa34e302640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3481617b0 con 0x7fa34806fc50 2024-04-25T08:56:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa34e302640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa348161560 con 0x7fa34815d180 2024-04-25T08:56:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa33f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa3480cf840 0x7fa3480cfcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44806/0 (socket says 172.21.15.37:44806) 2024-04-25T08:56:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa33f7fe640 1 -- 172.21.15.37:0/878951717 learned_addr learned my addr 172.21.15.37:0/878951717 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:56:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa34c89f640 1 --2- 172.21.15.37:0/878951717 >> v2:172.21.15.203:3300/0 conn(0x7fa34815d180 0x7fa34815d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395786382 0 0) 0x7fa34806b710 con 0x7fa3480cf840 2024-04-25T08:56:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa334003530 con 0x7fa3480cf840 2024-04-25T08:56:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 694454495 0 0) 0x7fa334003530 con 0x7fa3480cf840 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 >> v2:172.21.15.203:3300/0 conn(0x7fa34815d180 msgr2=0x7fa34815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 --2- 172.21.15.37:0/878951717 >> v2:172.21.15.203:3300/0 conn(0x7fa34815d180 0x7fa34815d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 >> v1:172.21.15.203:6789/0 conn(0x7fa34806fc50 legacy=0x7fa34806ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa348161a90 con 0x7fa3480cf840 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fa330001680 con 0x7fa3480cf840 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 >> v1:172.21.15.37:6789/0 conn(0x7fa3480cf840 legacy=0x7fa3480cfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa334003a30 0x7fa334003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa334004810 0x7fa334004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa334005240 0x7fa33400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa348161560 con 0x7fa334005240 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33400c1a0 con 0x7fa334003a30 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33400c3a0 con 0x7fa334004810 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa34c89f640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa334005240 0x7fa33400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa33f7fe640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa334004810 0x7fa334004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa347fff640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa334003a30 0x7fa334003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa34c89f640 1 -- 172.21.15.37:0/878951717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa334004810 msgr2=0x7fa334004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:58.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa34c89f640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa334004810 0x7fa334004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa34c89f640 1 -- 172.21.15.37:0/878951717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa334003a30 msgr2=0x7fa334003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:58.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa34c89f640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa334003a30 0x7fa334003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa34c89f640 1 -- 172.21.15.37:0/878951717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa334003530 con 0x7fa334005240 2024-04-25T08:56:58.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.058+0000 7fa347fff640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa334003a30 0x7fa334003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:58.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34c89f640 1 -- 172.21.15.37:0/878951717 reap_dead start 2024-04-25T08:56:58.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34c89f640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa334005240 0x7fa33400b9d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa340009fc0 tx=0x7fa34001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e764c12d7222494 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:58.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340009c10 con 0x7fa334005240 2024-04-25T08:56:58.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa340003ba0 con 0x7fa334005240 2024-04-25T08:56:58.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa3477fe640 1 -- 172.21.15.37:0/878951717 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3400a5900 con 0x7fa334005240 2024-04-25T08:56:58.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 -- 172.21.15.37:0/878951717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa334005240 msgr2=0x7fa33400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa334005240 0x7fa33400b9d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa340009fc0 tx=0x7fa34001cc00 comp rx=0 tx=0).stop 2024-04-25T08:56:58.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 -- 172.21.15.37:0/878951717 shutdown_connections 2024-04-25T08:56:58.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 --2- 172.21.15.37:0/878951717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa334005240 0x7fa33400b9d0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 -- 172.21.15.37:0/878951717 >> 172.21.15.37:0/878951717 conn(0x7fa348099a70 msgr2=0x7fa3480d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:58.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 -- 172.21.15.37:0/878951717 shutdown_connections 2024-04-25T08:56:58.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 -- 172.21.15.37:0/878951717 wait complete. 2024-04-25T08:56:58.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 Processor -- start 2024-04-25T08:56:58.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 -- start start 2024-04-25T08:56:58.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa334005240 0x7fa3481e85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa348158c80 0x7fa3480b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3480b2670 0x7fa3480ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa348070030 con 0x7fa348158c80 2024-04-25T08:56:58.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3480d0e30 con 0x7fa334005240 2024-04-25T08:56:58.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34e302640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3480701b0 con 0x7fa3480b2670 2024-04-25T08:56:58.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa33f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa348158c80 0x7fa3480b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa33f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa348158c80 0x7fa3480b2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39246/0 (socket says 172.21.15.37:39246) 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa33f7fe640 1 -- 172.21.15.37:0/1683693174 learned_addr learned my addr 172.21.15.37:0/1683693174 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa347fff640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa334005240 0x7fa3481e85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.062+0000 7fa34c89f640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3480b2670 0x7fa3480ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa33f7fe640 1 -- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3480b2670 msgr2=0x7fa3480ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa33f7fe640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3480b2670 0x7fa3480ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa33f7fe640 1 -- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa334005240 msgr2=0x7fa3481e85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa33f7fe640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa334005240 0x7fa3481e85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa33f7fe640 1 -- 172.21.15.37:0/1683693174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3481e7520 con 0x7fa348158c80 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa33f7fe640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa348158c80 0x7fa3480b2130 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa330011790 tx=0x7fa330011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa3457fa640 1 -- 172.21.15.37:0/1683693174 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330010040 con 0x7fa348158c80 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa3457fa640 1 -- 172.21.15.37:0/1683693174 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33001a070 con 0x7fa348158c80 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa34e302640 1 -- 172.21.15.37:0/1683693174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3481e7800 con 0x7fa348158c80 2024-04-25T08:56:58.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa3457fa640 1 -- 172.21.15.37:0/1683693174 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33000fb90 con 0x7fa348158c80 2024-04-25T08:56:58.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa34e302640 1 -- 172.21.15.37:0/1683693174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3481e7dc0 con 0x7fa348158c80 2024-04-25T08:56:58.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa34e302640 1 -- 172.21.15.37:0/1683693174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa34804efa0 con 0x7fa348158c80 2024-04-25T08:56:58.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa3457fa640 1 -- 172.21.15.37:0/1683693174 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa33001f460 con 0x7fa348158c80 2024-04-25T08:56:58.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa3457fa640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa328066f90 0x7fa328069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa3457fa640 1 -- 172.21.15.37:0/1683693174 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa330088220 con 0x7fa348158c80 2024-04-25T08:56:58.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa347fff640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa328066f90 0x7fa328069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.066+0000 7fa347fff640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa328066f90 0x7fa328069450 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa338000c10 tx=0x7fa338009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:58.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.070+0000 7fa3457fa640 1 -- 172.21.15.37:0/1683693174 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa330059130 con 0x7fa348158c80 2024-04-25T08:56:58.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.262+0000 7fa34e302640 1 -- 172.21.15.37:0/1683693174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa3480b6480 con 0x7fa348158c80 2024-04-25T08:56:58.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.262+0000 7fa3457fa640 1 -- 172.21.15.37:0/1683693174 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa330058a30 con 0x7fa348158c80 2024-04-25T08:56:58.252 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:56:58.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.262+0000 7fa34e302640 1 -- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa328066f90 msgr2=0x7fa328069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.262+0000 7fa34e302640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa328066f90 0x7fa328069450 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa338000c10 tx=0x7fa338009040 comp rx=0 tx=0).stop 2024-04-25T08:56:58.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.262+0000 7fa34e302640 1 -- 172.21.15.37:0/1683693174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa348158c80 msgr2=0x7fa3480b2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.262+0000 7fa34e302640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa348158c80 0x7fa3480b2130 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fa330011790 tx=0x7fa330011c50 comp rx=0 tx=0).stop 2024-04-25T08:56:58.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.266+0000 7fa34e302640 1 -- 172.21.15.37:0/1683693174 shutdown_connections 2024-04-25T08:56:58.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.266+0000 7fa34e302640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa328066f90 0x7fa328069450 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.266+0000 7fa34e302640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa3480b2670 0x7fa3480ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.266+0000 7fa34e302640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa348158c80 0x7fa3480b2130 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.266+0000 7fa34e302640 1 --2- 172.21.15.37:0/1683693174 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa334005240 0x7fa3481e85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.266+0000 7fa34e302640 1 -- 172.21.15.37:0/1683693174 >> 172.21.15.37:0/1683693174 conn(0x7fa348099a70 msgr2=0x7fa3480adcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:58.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.266+0000 7fa34e302640 1 -- 172.21.15.37:0/1683693174 shutdown_connections 2024-04-25T08:56:58.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:56:58.266+0000 7fa34e302640 1 -- 172.21.15.37:0/1683693174 wait complete. 2024-04-25T08:56:58.270 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:56:58.270 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:56:58.270 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T08:56:58.270 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:56:58.270 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:56:58.270 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:56:58.270 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:56:58.270 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:56:58.270 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:56:58.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:58.271 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T08:56:58.277 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:56:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:56:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:56:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:56:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:56:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:56:58.277 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:56:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:56:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T08:56:58.277 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:56:58.277 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42222sec preferred_lft 42222sec 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:56:58.278 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:56:58.279 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:56:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:56:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:56:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:56:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:56:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:56:58.279 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:56:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:56:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:56:58.279 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:56:58.280 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:56:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:56:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:56:58.280 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:56:58.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:58.280 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:56:58.331 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:56:58.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:58.332 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T08:56:58.380 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T08:56:58.380 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:56:58.380 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:56:58.380 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:56:58.380 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:56:58.380 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:56:58.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:58.380 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:56:58.381 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:56:58.381 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:56:58.381 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:56:58.381 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:56:58.381 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:56:58.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:58.455 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:56:58.503 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:56:58.504 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:56:58.504 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:56:58.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:58.504 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T08:56:58.559 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:56:58.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:56:58.560 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:56:58.607 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:56:58.608 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:56:58.608 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:56:58.655 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:56:58.713 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:56:58.761 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:56:58.761 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:56:58.762 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:56:58.763 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:56:58.764 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:56:58.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:58.764 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:56:58.811 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:56:58.811 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:56:58.856 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:56:58.871 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:56:58.920 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:56:58.920 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:56:58.920 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:56:58.920 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:56:58.920 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:56:58.920 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:56:58.920 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:56:58.920 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:56:58.920 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:56:58.920 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:56:58.920 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:56:58.921 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:56:58.922 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:56:58.922 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:56:58.922 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:56:58.922 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:56:58.922 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:56:58.922 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:56:58.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:58.922 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:56:58.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T08:56:58.942+0000 7faec9ee6580 -1 init, newargv = 0x55c07d064d00 newargc=15 2024-04-25T08:56:58.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[44994]: starting ceph client 2024-04-25T08:56:58.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[44994]: starting fuse 2024-04-25T08:57:00.039 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:57:00.056 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:57:00.108 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:57:00.108 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:57:00.108 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:57:00.109 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:00.110 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:00.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:00.111 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:57:00.160 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:57:00.160 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:57:00.160 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:57:00.160 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T08:57:00.160 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T08:57:00.160 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:57:00.160 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T08:57:00.160 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:57:00.160 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T08:57:00.160 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:57:00.160 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:57:00.161 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:57:00.162 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T08:57:00.162 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:57:00.291 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.44994.asok 2024-04-25T08:57:00.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:57:00.303 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44994.asok 2024-04-25T08:57:00.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:00.303 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44994.asok status 2024-04-25T08:57:00.464 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:57:00.464 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "44994", 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7101, 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T08:57:00.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 7101 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1097420832 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1097420832 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.7101 192.168.0.1:0/1097420832", 2024-04-25T08:57:00.466 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/1097420832", 2024-04-25T08:57:00.467 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T08:57:00.467 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 44, 2024-04-25T08:57:00.467 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 82, 2024-04-25T08:57:00.467 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:57:00.467 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T08:57:00.467 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T08:57:00.467 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:57:00.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:00.479 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T08:57:00.479 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:00.485 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T08:57:00.485 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:00.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:00.485 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:00.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:00.539 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-04-25T08:57:00.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:00.596 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-04-25T08:57:00.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:00.648 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-04-25T08:57:00.699 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T08:57:00.699 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:57:00.699 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T08:57:00.699 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:57:00.699 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-04-25T08:57:00.699 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:57:00.811 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627776 2024-04-25T08:57:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:57:00.821 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:57:00.821 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:00.868 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:01.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa7010b6640 1 Processor -- start 2024-04-25T08:57:01.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa7010b6640 1 -- start start 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa7010b6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc144ca0 0x7fa6fc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa7010b6640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc0ccc70 con 0x7fa6fc06ec60 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa7010b6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc06dc40 con 0x7fa6fc0ce510 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa7010b6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0cc2b0 con 0x7fa6fc144ca0 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fb59e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa6fc06ec60 0x7fa6fc142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:43962/0 (socket says 172.21.15.37:43962) 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fb59e640 1 -- 172.21.15.37:0/141317265 learned_addr learned my addr 172.21.15.37:0/141317265 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fa59c640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc144ca0 0x7fa6fc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425599095 0 0) 0x7fa6fc0ccc70 con 0x7fa6fc06ec60 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e4003530 con 0x7fa6fc06ec60 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3096601808 0 0) 0x7fa6e4003530 con 0x7fa6fc06ec60 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc144ca0 msgr2=0x7fa6fc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:01.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc144ca0 0x7fa6fc06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 >> v1:172.21.15.203:6789/0 conn(0x7fa6fc0ce510 legacy=0x7fa6fc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc06df20 con 0x7fa6fc06ec60 2024-04-25T08:57:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fa6f4001a00 con 0x7fa6fc06ec60 2024-04-25T08:57:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 >> v1:172.21.15.37:6789/0 conn(0x7fa6fc06ec60 legacy=0x7fa6fc142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 0x7fa6e4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6e4004810 0x7fa6e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc144ca0 0x7fa6e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0cc2b0 con 0x7fa6e4003a60 2024-04-25T08:57:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e400bb60 con 0x7fa6e4004810 2024-04-25T08:57:01.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e400bd60 con 0x7fa6fc144ca0 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fad9d640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 0x7fa6e4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fa59c640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6e4004810 0x7fa6e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fb59e640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc144ca0 0x7fa6e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fad9d640 1 -- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc144ca0 msgr2=0x7fa6e40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fad9d640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc144ca0 0x7fa6e40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fad9d640 1 -- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6e4004810 msgr2=0x7fa6e4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fad9d640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6e4004810 0x7fa6e4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fad9d640 1 -- 172.21.15.37:0/141317265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e4003530 con 0x7fa6e4003a60 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.058+0000 7fa6fb59e640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc144ca0 0x7fa6e40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa6fad9d640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 0x7fa6e4003e80 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec001780 tx=0x7fa6ec01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a94783f687506d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0013a0 con 0x7fa6e4003a60 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec0052f0 con 0x7fa6e4003a60 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa6f9d9b640 1 -- 172.21.15.37:0/141317265 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec003840 con 0x7fa6e4003a60 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa7010b6640 1 -- 172.21.15.37:0/141317265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 msgr2=0x7fa6e4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa7010b6640 1 --2- 172.21.15.37:0/141317265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 0x7fa6e4003e80 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec001780 tx=0x7fa6ec01c420 comp rx=0 tx=0).stop 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa6fb59e640 1 -- 172.21.15.37:0/141317265 reap_dead start 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa7010b6640 1 -- 172.21.15.37:0/141317265 shutdown_connections 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa7010b6640 1 -- 172.21.15.37:0/141317265 >> 172.21.15.37:0/141317265 conn(0x7fa6fc0b11a0 msgr2=0x7fa6fc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa7010b6640 1 -- 172.21.15.37:0/141317265 shutdown_connections 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa7010b6640 1 -- 172.21.15.37:0/141317265 wait complete. 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa7010b6640 1 Processor -- start 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.062+0000 7fa7010b6640 1 -- start start 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa7010b6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 0x7fa6fc15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa7010b6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc13f220 0x7fa6fc1e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa7010b6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6fc140830 0x7fa6fc1e1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa7010b6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc142d90 con 0x7fa6e4003a60 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa7010b6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc06e290 con 0x7fa6fc140830 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa7010b6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc142f10 con 0x7fa6fc13f220 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 0x7fa6fc15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 0x7fa6fc15a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38102/0 (socket says 172.21.15.37:38102) 2024-04-25T08:57:01.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fad9d640 1 -- 172.21.15.37:0/1451892242 learned_addr learned my addr 172.21.15.37:0/1451892242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fa59c640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc13f220 0x7fa6fc1e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fad9d640 1 -- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc13f220 msgr2=0x7fa6fc1e0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fb59e640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6fc140830 0x7fa6fc1e1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fad9d640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc13f220 0x7fa6fc1e0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fad9d640 1 -- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6fc140830 msgr2=0x7fa6fc1e1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fad9d640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6fc140830 0x7fa6fc1e1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fad9d640 1 -- 172.21.15.37:0/1451892242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc13f790 con 0x7fa6e4003a60 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fb59e640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6fc140830 0x7fa6fc1e1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fad9d640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 0x7fa6fc15a140 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec001780 tx=0x7fa6e4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6e37fe640 1 -- 172.21.15.37:0/1451892242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec007a40 con 0x7fa6e4003a60 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6e37fe640 1 -- 172.21.15.37:0/1451892242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec0bb070 con 0x7fa6e4003a60 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa7010b6640 1 -- 172.21.15.37:0/1451892242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc13fa70 con 0x7fa6e4003a60 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6e37fe640 1 -- 172.21.15.37:0/1451892242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0013a0 con 0x7fa6e4003a60 2024-04-25T08:57:01.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa7010b6640 1 -- 172.21.15.37:0/1451892242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc140030 con 0x7fa6e4003a60 2024-04-25T08:57:01.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa7010b6640 1 -- 172.21.15.37:0/1451892242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6fc04efa0 con 0x7fa6e4003a60 2024-04-25T08:57:01.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6e37fe640 1 -- 172.21.15.37:0/1451892242 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa6ec0b4040 con 0x7fa6e4003a60 2024-04-25T08:57:01.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6e37fe640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa6e8066f60 0x7fa6e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6e37fe640 1 -- 172.21.15.37:0/1451892242 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa6ec120060 con 0x7fa6e4003a60 2024-04-25T08:57:01.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.066+0000 7fa6fa59c640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa6e8066f60 0x7fa6e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.070+0000 7fa6fa59c640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa6e8066f60 0x7fa6e8069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc072240 tx=0x7fa6f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.070+0000 7fa6e37fe640 1 -- 172.21.15.37:0/1451892242 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6ec016130 con 0x7fa6e4003a60 2024-04-25T08:57:01.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.238+0000 7fa7010b6640 1 -- 172.21.15.37:0/1451892242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6fc0b1cc0 con 0x7fa6e4003a60 2024-04-25T08:57:01.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.238+0000 7fa6e37fe640 1 -- 172.21.15.37:0/1451892242 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16909 (secure 0 0 0) 0x7fa6ec0f0f50 con 0x7fa6e4003a60 2024-04-25T08:57:01.229 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:57:01.229 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":82,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:56:53.098938+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:56:49.737591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:56:50.436964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 -- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa6e8066f60 msgr2=0x7fa6e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa6e8066f60 0x7fa6e8069420 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc072240 tx=0x7fa6f0008040 comp rx=0 tx=0).stop 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 -- 172.21.15.37:0/1451892242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 msgr2=0x7fa6fc15a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 0x7fa6fc15a140 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec001780 tx=0x7fa6e4005840 comp rx=0 tx=0).stop 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 -- 172.21.15.37:0/1451892242 shutdown_connections 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa6e8066f60 0x7fa6e8069420 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6fc140830 0x7fa6fc1e1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa6fc13f220 0x7fa6fc1e0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 --2- 172.21.15.37:0/1451892242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6e4003a60 0x7fa6fc15a140 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 -- 172.21.15.37:0/1451892242 >> 172.21.15.37:0/1451892242 conn(0x7fa6fc0b11a0 msgr2=0x7fa6fc0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 -- 172.21.15.37:0/1451892242 shutdown_connections 2024-04-25T08:57:01.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.242+0000 7fa7010b6640 1 -- 172.21.15.37:0/1451892242 wait complete. 2024-04-25T08:57:01.249 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:56:49.737591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:56:50.436964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:57:01.249 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:57:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.382+0000 7fe15d17a640 1 Processor -- start 2024-04-25T08:57:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.382+0000 7fe15d17a640 1 -- start start 2024-04-25T08:57:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15d17a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe1580cf590 0x7fe1580cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15d17a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe158072000 0x7fe158072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15d17a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1580cbd20 con 0x7fe15815b6b0 2024-04-25T08:57:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15d17a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15815ba90 con 0x7fe1580cf590 2024-04-25T08:57:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15d17a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1580cc000 con 0x7fe158072000 2024-04-25T08:57:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe1580cf590 0x7fe1580cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe156d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe15815b6b0 0x7fe1580ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:43964/0 (socket says 172.21.15.37:43964) 2024-04-25T08:57:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe1580cf590 0x7fe1580cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38106/0 (socket says 172.21.15.37:38106) 2024-04-25T08:57:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe156d9d640 1 -- 172.21.15.37:0/3527880153 learned_addr learned my addr 172.21.15.37:0/3527880153 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:01.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15759e640 1 --2- 172.21.15.37:0/3527880153 >> v2:172.21.15.203:3300/0 conn(0x7fe158072000 0x7fe158072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15659c640 1 -- 172.21.15.37:0/3527880153 >> v2:172.21.15.203:3300/0 conn(0x7fe158072000 msgr2=0x7fe158072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:01.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15659c640 1 --2- 172.21.15.37:0/3527880153 >> v2:172.21.15.203:3300/0 conn(0x7fe158072000 0x7fe158072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15659c640 1 -- 172.21.15.37:0/3527880153 >> v1:172.21.15.37:6789/0 conn(0x7fe15815b6b0 legacy=0x7fe1580ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe15659c640 1 -- 172.21.15.37:0/3527880153 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580cc2e0 con 0x7fe1580cf590 2024-04-25T08:57:01.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.386+0000 7fe155d9b640 1 -- 172.21.15.37:0/3527880153 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2926134117 0 0) 0x7fe1580cbd20 con 0x7fe15815b6b0 2024-04-25T08:57:01.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15659c640 1 --2- 172.21.15.37:0/3527880153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1580cf590 0x7fe1580cfa20 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe15806f000 tx=0x7fe14001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e22cb72e439131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe155d9b640 1 -- 172.21.15.37:0/3527880153 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1400b0070 con 0x7fe1580cf590 2024-04-25T08:57:01.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe155d9b640 1 -- 172.21.15.37:0/3527880153 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1400ab600 con 0x7fe1580cf590 2024-04-25T08:57:01.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe155d9b640 1 -- 172.21.15.37:0/3527880153 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1400ab9d0 con 0x7fe1580cf590 2024-04-25T08:57:01.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 -- 172.21.15.37:0/3527880153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1580cf590 msgr2=0x7fe1580cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 --2- 172.21.15.37:0/3527880153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1580cf590 0x7fe1580cfa20 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe15806f000 tx=0x7fe14001c480 comp rx=0 tx=0).stop 2024-04-25T08:57:01.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 -- 172.21.15.37:0/3527880153 shutdown_connections 2024-04-25T08:57:01.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 --2- 172.21.15.37:0/3527880153 >> v2:172.21.15.203:3300/0 conn(0x7fe158072000 0x7fe158072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 --2- 172.21.15.37:0/3527880153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1580cf590 0x7fe1580cfa20 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 -- 172.21.15.37:0/3527880153 >> 172.21.15.37:0/3527880153 conn(0x7fe15806ae60 msgr2=0x7fe15806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:01.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 -- 172.21.15.37:0/3527880153 shutdown_connections 2024-04-25T08:57:01.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 -- 172.21.15.37:0/3527880153 wait complete. 2024-04-25T08:57:01.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 Processor -- start 2024-04-25T08:57:01.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 -- start start 2024-04-25T08:57:01.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe158072000 0x7fe1581534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1580cf590 0x7fe15815a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15815b6b0 0x7fe15815df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580caf30 con 0x7fe15815b6b0 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580cadb0 con 0x7fe158072000 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15d17a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1581682a0 con 0x7fe1580cf590 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15815b6b0 0x7fe15815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15815b6b0 0x7fe15815df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38116/0 (socket says 172.21.15.37:38116) 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15759e640 1 -- 172.21.15.37:0/1252719486 learned_addr learned my addr 172.21.15.37:0/1252719486 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15659c640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1580cf590 0x7fe15815a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe156d9d640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe158072000 0x7fe1581534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15759e640 1 -- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1580cf590 msgr2=0x7fe15815a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15759e640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1580cf590 0x7fe15815a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15759e640 1 -- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe158072000 msgr2=0x7fe1581534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15759e640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe158072000 0x7fe1581534a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15759e640 1 -- 172.21.15.37:0/1252719486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580b3e50 con 0x7fe15815b6b0 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.390+0000 7fe15759e640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15815b6b0 0x7fe15815df70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe150007e40 tx=0x7fe15000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.394+0000 7fe14f7fe640 1 -- 172.21.15.37:0/1252719486 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe150014620 con 0x7fe15815b6b0 2024-04-25T08:57:01.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.394+0000 7fe14f7fe640 1 -- 172.21.15.37:0/1252719486 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe150019070 con 0x7fe15815b6b0 2024-04-25T08:57:01.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.394+0000 7fe15d17a640 1 -- 172.21.15.37:0/1252719486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1580b3920 con 0x7fe15815b6b0 2024-04-25T08:57:01.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.394+0000 7fe14f7fe640 1 -- 172.21.15.37:0/1252719486 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe150014a20 con 0x7fe15815b6b0 2024-04-25T08:57:01.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.394+0000 7fe15d17a640 1 -- 172.21.15.37:0/1252719486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1581539e0 con 0x7fe15815b6b0 2024-04-25T08:57:01.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.394+0000 7fe15d17a640 1 -- 172.21.15.37:0/1252719486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15804efa0 con 0x7fe15815b6b0 2024-04-25T08:57:01.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.394+0000 7fe14f7fe640 1 -- 172.21.15.37:0/1252719486 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe150014bc0 con 0x7fe15815b6b0 2024-04-25T08:57:01.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.394+0000 7fe14f7fe640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe138066f60 0x7fe138069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.394+0000 7fe14f7fe640 1 -- 172.21.15.37:0/1252719486 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1500859f0 con 0x7fe15815b6b0 2024-04-25T08:57:01.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.394+0000 7fe156d9d640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe138066f60 0x7fe138069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.394+0000 7fe156d9d640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe138066f60 0x7fe138069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe148000f70 tx=0x7fe148007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.398+0000 7fe14f7fe640 1 -- 172.21.15.37:0/1252719486 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe150056900 con 0x7fe15815b6b0 2024-04-25T08:57:01.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.558+0000 7fe15d17a640 1 -- 172.21.15.37:0/1252719486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe15806bfc0 con 0x7fe15815b6b0 2024-04-25T08:57:01.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.558+0000 7fe14f7fe640 1 -- 172.21.15.37:0/1252719486 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe150085020 con 0x7fe15815b6b0 2024-04-25T08:57:01.549 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 -- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe138066f60 msgr2=0x7fe138069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe138066f60 0x7fe138069420 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe148000f70 tx=0x7fe148007c10 comp rx=0 tx=0).stop 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 -- 172.21.15.37:0/1252719486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15815b6b0 msgr2=0x7fe15815df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15815b6b0 0x7fe15815df70 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe150007e40 tx=0x7fe15000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 -- 172.21.15.37:0/1252719486 shutdown_connections 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe138066f60 0x7fe138069420 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15815b6b0 0x7fe15815df70 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe1580cf590 0x7fe15815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 --2- 172.21.15.37:0/1252719486 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe158072000 0x7fe1581534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 -- 172.21.15.37:0/1252719486 >> 172.21.15.37:0/1252719486 conn(0x7fe15806ae60 msgr2=0x7fe158164670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 -- 172.21.15.37:0/1252719486 shutdown_connections 2024-04-25T08:57:01.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.562+0000 7fe15d17a640 1 -- 172.21.15.37:0/1252719486 wait complete. 2024-04-25T08:57:01.567 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:57:01.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148a3c2640 1 Processor -- start 2024-04-25T08:57:01.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148a3c2640 1 -- start start 2024-04-25T08:57:01.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148a3c2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f148406b440 0x7f14840b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148a3c2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f14840b1ac0 0x7f14840cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148a3c2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14840afb20 con 0x7f148406b820 2024-04-25T08:57:01.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148a3c2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14840b06c0 con 0x7f14840b1ac0 2024-04-25T08:57:01.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148a3c2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14840afd90 con 0x7f148406b440 2024-04-25T08:57:01.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148895f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f14840b1ac0 0x7f14840cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148895f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f14840b1ac0 0x7f14840cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38130/0 (socket says 172.21.15.37:38130) 2024-04-25T08:57:01.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148895f640 1 -- 172.21.15.37:0/3315390619 learned_addr learned my addr 172.21.15.37:0/3315390619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:01.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f1483fff640 1 --2- 172.21.15.37:0/3315390619 >> v2:172.21.15.203:3300/0 conn(0x7f148406b440 0x7f14840b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f1482ffd640 1 -- 172.21.15.37:0/3315390619 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406724069 0 0) 0x7f14840afb20 con 0x7f148406b820 2024-04-25T08:57:01.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148895f640 1 -- 172.21.15.37:0/3315390619 >> v2:172.21.15.203:3300/0 conn(0x7f148406b440 msgr2=0x7f14840b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:01.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148895f640 1 --2- 172.21.15.37:0/3315390619 >> v2:172.21.15.203:3300/0 conn(0x7f148406b440 0x7f14840b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148895f640 1 -- 172.21.15.37:0/3315390619 >> v1:172.21.15.37:6789/0 conn(0x7f148406b820 legacy=0x7f14840b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.706+0000 7f148895f640 1 -- 172.21.15.37:0/3315390619 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14840b0070 con 0x7f14840b1ac0 2024-04-25T08:57:01.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f148895f640 1 --2- 172.21.15.37:0/3315390619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14840b1ac0 0x7f14840cf650 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f14840724c0 tx=0x7f147c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a40884cc466d7c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f1482ffd640 1 -- 172.21.15.37:0/3315390619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c0a57d0 con 0x7f14840b1ac0 2024-04-25T08:57:01.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f1482ffd640 1 -- 172.21.15.37:0/3315390619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147c0a5970 con 0x7f14840b1ac0 2024-04-25T08:57:01.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f1482ffd640 1 -- 172.21.15.37:0/3315390619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c0a5d40 con 0x7f14840b1ac0 2024-04-25T08:57:01.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f148a3c2640 1 -- 172.21.15.37:0/3315390619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14840b1ac0 msgr2=0x7f14840cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f148a3c2640 1 --2- 172.21.15.37:0/3315390619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14840b1ac0 0x7f14840cf650 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f14840724c0 tx=0x7f147c01c9f0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f148a3c2640 1 -- 172.21.15.37:0/3315390619 shutdown_connections 2024-04-25T08:57:01.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f148a3c2640 1 --2- 172.21.15.37:0/3315390619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14840b1ac0 0x7f14840cf650 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f148a3c2640 1 --2- 172.21.15.37:0/3315390619 >> v2:172.21.15.203:3300/0 conn(0x7f148406b440 0x7f14840b0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f148a3c2640 1 -- 172.21.15.37:0/3315390619 >> 172.21.15.37:0/3315390619 conn(0x7f14840b1f10 msgr2=0x7f14840b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f148a3c2640 1 -- 172.21.15.37:0/3315390619 shutdown_connections 2024-04-25T08:57:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.710+0000 7f148a3c2640 1 -- 172.21.15.37:0/3315390619 wait complete. 2024-04-25T08:57:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148a3c2640 1 Processor -- start 2024-04-25T08:57:01.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148a3c2640 1 -- start start 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148a3c2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f148406b440 0x7f148414e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148a3c2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f148406b820 0x7f14840b2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148a3c2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14840b1ac0 0x7f14840c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148a3c2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14840cd170 con 0x7f148406b440 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148a3c2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14840ccff0 con 0x7f14840b1ac0 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148a3c2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148415d080 con 0x7f148406b820 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1483fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f148406b440 0x7f148414e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1483fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f148406b440 0x7f148414e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38134/0 (socket says 172.21.15.37:38134) 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1483fff640 1 -- 172.21.15.37:0/280952605 learned_addr learned my addr 172.21.15.37:0/280952605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1483fff640 1 -- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f148406b820 msgr2=0x7f14840b2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f14837fe640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f148406b820 0x7f14840b2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148895f640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14840b1ac0 0x7f14840c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1483fff640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f148406b820 0x7f14840b2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1483fff640 1 -- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14840b1ac0 msgr2=0x7f14840c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1483fff640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14840b1ac0 0x7f14840c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1483fff640 1 -- 172.21.15.37:0/280952605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148414edf0 con 0x7f148406b440 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f14837fe640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f148406b820 0x7f14840b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148895f640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14840b1ac0 0x7f14840c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1483fff640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f148406b440 0x7f148414e8b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f147400e930 tx=0x7f147400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1480ff9640 1 -- 172.21.15.37:0/280952605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474017070 con 0x7f148406b440 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1480ff9640 1 -- 172.21.15.37:0/280952605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147400c9a0 con 0x7f148406b440 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1480ff9640 1 -- 172.21.15.37:0/280952605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147400cd50 con 0x7f148406b440 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148a3c2640 1 -- 172.21.15.37:0/280952605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148414f0a0 con 0x7f148406b440 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148a3c2640 1 -- 172.21.15.37:0/280952605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148414f660 con 0x7f148406b440 2024-04-25T08:57:01.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f148a3c2640 1 -- 172.21.15.37:0/280952605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148404efa0 con 0x7f148406b440 2024-04-25T08:57:01.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1480ff9640 1 -- 172.21.15.37:0/280952605 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f147401a070 con 0x7f148406b440 2024-04-25T08:57:01.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.714+0000 7f1480ff9640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1464066f90 0x7f1464069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.718+0000 7f1480ff9640 1 -- 172.21.15.37:0/280952605 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14740851b0 con 0x7f148406b440 2024-04-25T08:57:01.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.718+0000 7f14837fe640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1464066f90 0x7f1464069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.718+0000 7f14837fe640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1464066f90 0x7f1464069450 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1484154610 tx=0x7f146c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.722+0000 7f1480ff9640 1 -- 172.21.15.37:0/280952605 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14740560e0 con 0x7f148406b440 2024-04-25T08:57:01.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 -- 172.21.15.37:0/280952605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f14840b6380 con 0x7f148406b440 2024-04-25T08:57:01.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f1480ff9640 1 -- 172.21.15.37:0/280952605 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f14740559e0 con 0x7f148406b440 2024-04-25T08:57:01.872 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:57:01.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 -- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1464066f90 msgr2=0x7f1464069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1464066f90 0x7f1464069450 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1484154610 tx=0x7f146c009040 comp rx=0 tx=0).stop 2024-04-25T08:57:01.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 -- 172.21.15.37:0/280952605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f148406b440 msgr2=0x7f148414e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f148406b440 0x7f148414e8b0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f147400e930 tx=0x7f147400edf0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 -- 172.21.15.37:0/280952605 shutdown_connections 2024-04-25T08:57:01.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1464066f90 0x7f1464069450 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14840b1ac0 0x7f14840c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f148406b820 0x7f14840b2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 --2- 172.21.15.37:0/280952605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f148406b440 0x7f148414e8b0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 -- 172.21.15.37:0/280952605 >> 172.21.15.37:0/280952605 conn(0x7f14840b1f10 msgr2=0x7f148415d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:01.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 -- 172.21.15.37:0/280952605 shutdown_connections 2024-04-25T08:57:01.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:01.882+0000 7f148a3c2640 1 -- 172.21.15.37:0/280952605 wait complete. 2024-04-25T08:57:01.890 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:57:02.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7fb0023a9640 1 Processor -- start 2024-04-25T08:57:02.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7fb0023a9640 1 -- start start 2024-04-25T08:57:02.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7fb0023a9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faffc06bca0 0x7faffc0cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7fb0023a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc066e30 0x7faffc06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7fb0023a9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faffc0af370 0x7faffc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7fb0023a9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faffc0669a0 con 0x7faffc06bca0 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7fb0023a9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faffc06ec30 con 0x7faffc0af370 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7fb0023a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0cd5f0 con 0x7faffc066e30 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7faffbfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faffc06bca0 0x7faffc0cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7faffbfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faffc06bca0 0x7faffc0cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38142/0 (socket says 172.21.15.37:38142) 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7faffbfff640 1 -- 172.21.15.37:0/3169648807 learned_addr learned my addr 172.21.15.37:0/3169648807 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7faffb7fe640 1 --2- 172.21.15.37:0/3169648807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc066e30 0x7faffc06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7fb000946640 1 --2- 172.21.15.37:0/3169648807 >> v2:172.21.15.203:3300/0 conn(0x7faffc0af370 0x7faffc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7faffbfff640 1 -- 172.21.15.37:0/3169648807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc066e30 msgr2=0x7faffc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7faffbfff640 1 --2- 172.21.15.37:0/3169648807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc066e30 0x7faffc06c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7faffbfff640 1 -- 172.21.15.37:0/3169648807 >> v2:172.21.15.203:3300/0 conn(0x7faffc0af370 msgr2=0x7faffc06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:02.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7faffbfff640 1 --2- 172.21.15.37:0/3169648807 >> v2:172.21.15.203:3300/0 conn(0x7faffc0af370 0x7faffc06d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7faffbfff640 1 -- 172.21.15.37:0/3169648807 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc06c5c0 con 0x7faffc06bca0 2024-04-25T08:57:02.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7fb000946640 1 --2- 172.21.15.37:0/3169648807 >> v2:172.21.15.203:3300/0 conn(0x7faffc0af370 0x7faffc06d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:02.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.022+0000 7faffb7fe640 1 --2- 172.21.15.37:0/3169648807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc066e30 0x7faffc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:02.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7faffbfff640 1 --2- 172.21.15.37:0/3169648807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faffc06bca0 0x7faffc0cd0b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7faffc06d160 tx=0x7fafec01c1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19e4c25b0b951b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7faffaffd640 1 -- 172.21.15.37:0/3169648807 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec0b0070 con 0x7faffc06bca0 2024-04-25T08:57:02.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7faffaffd640 1 -- 172.21.15.37:0/3169648807 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafec0ab600 con 0x7faffc06bca0 2024-04-25T08:57:02.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7faffaffd640 1 -- 172.21.15.37:0/3169648807 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafec0ab9d0 con 0x7faffc06bca0 2024-04-25T08:57:02.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7fb0023a9640 1 -- 172.21.15.37:0/3169648807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faffc06bca0 msgr2=0x7faffc0cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7fb0023a9640 1 --2- 172.21.15.37:0/3169648807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faffc06bca0 0x7faffc0cd0b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7faffc06d160 tx=0x7fafec01c1a0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7fb0023a9640 1 -- 172.21.15.37:0/3169648807 shutdown_connections 2024-04-25T08:57:02.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7fb0023a9640 1 --2- 172.21.15.37:0/3169648807 >> v2:172.21.15.203:3300/0 conn(0x7faffc0af370 0x7faffc06d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7fb0023a9640 1 --2- 172.21.15.37:0/3169648807 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc066e30 0x7faffc06c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7fb0023a9640 1 --2- 172.21.15.37:0/3169648807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faffc06bca0 0x7faffc0cd0b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7fb0023a9640 1 -- 172.21.15.37:0/3169648807 >> 172.21.15.37:0/3169648807 conn(0x7faffc071810 msgr2=0x7faffc071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:02.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7fb0023a9640 1 -- 172.21.15.37:0/3169648807 shutdown_connections 2024-04-25T08:57:02.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7fb0023a9640 1 -- 172.21.15.37:0/3169648807 wait complete. 2024-04-25T08:57:02.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7fb0023a9640 1 Processor -- start 2024-04-25T08:57:02.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.026+0000 7fb0023a9640 1 -- start start 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7fb0023a9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc066e30 0x7faffc142660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7fb0023a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faffc06bca0 0x7faffc0ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7fb0023a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc0af370 0x7faffc0ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7fb0023a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc15cc80 con 0x7faffc06bca0 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7fb0023a9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faffc15cb00 con 0x7faffc066e30 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7fb0023a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faffc0ceaf0 con 0x7faffc0af370 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faffc06bca0 0x7faffc0ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faffc06bca0 0x7faffc0ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38148/0 (socket says 172.21.15.37:38148) 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffb7fe640 1 -- 172.21.15.37:0/3750964914 learned_addr learned my addr 172.21.15.37:0/3750964914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffbfff640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc066e30 0x7faffc142660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7fb000946640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc0af370 0x7faffc0ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffb7fe640 1 -- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc0af370 msgr2=0x7faffc0ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffb7fe640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc0af370 0x7faffc0ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffb7fe640 1 -- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc066e30 msgr2=0x7faffc142660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffb7fe640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc066e30 0x7faffc142660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffb7fe640 1 -- 172.21.15.37:0/3750964914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faffc0b2d30 con 0x7faffc06bca0 2024-04-25T08:57:02.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffbfff640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc066e30 0x7faffc142660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:02.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7fb000946640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc0af370 0x7faffc0ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:02.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffb7fe640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faffc06bca0 0x7faffc0ad420 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fafe4007f20 tx=0x7fafe400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faff8ff9640 1 -- 172.21.15.37:0/3750964914 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe401a070 con 0x7faffc06bca0 2024-04-25T08:57:02.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faff8ff9640 1 -- 172.21.15.37:0/3750964914 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe4003ba0 con 0x7faffc06bca0 2024-04-25T08:57:02.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7fb0023a9640 1 -- 172.21.15.37:0/3750964914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faffc0b2ff0 con 0x7faffc06bca0 2024-04-25T08:57:02.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faff8ff9640 1 -- 172.21.15.37:0/3750964914 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe4004030 con 0x7faffc06bca0 2024-04-25T08:57:02.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7fb0023a9640 1 -- 172.21.15.37:0/3750964914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faffc0b35b0 con 0x7faffc06bca0 2024-04-25T08:57:02.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7fb0023a9640 1 -- 172.21.15.37:0/3750964914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faffc04efa0 con 0x7faffc06bca0 2024-04-25T08:57:02.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faff8ff9640 1 -- 172.21.15.37:0/3750964914 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafe401d070 con 0x7faffc06bca0 2024-04-25T08:57:02.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faff8ff9640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafdc066f60 0x7fafdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faff8ff9640 1 -- 172.21.15.37:0/3750964914 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fafe4085290 con 0x7faffc06bca0 2024-04-25T08:57:02.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.030+0000 7faffbfff640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafdc066f60 0x7fafdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.034+0000 7faffbfff640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafdc066f60 0x7fafdc069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fafec0ad000 tx=0x7fafec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.034+0000 7faff8ff9640 1 -- 172.21.15.37:0/3750964914 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fafe4010070 con 0x7faffc06bca0 2024-04-25T08:57:02.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 -- 172.21.15.37:0/3750964914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faffc072910 con 0x7faffc06bca0 2024-04-25T08:57:02.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7faff8ff9640 1 -- 172.21.15.37:0/3750964914 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fafe4055b50 con 0x7faffc06bca0 2024-04-25T08:57:02.203 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:57:02.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 -- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafdc066f60 msgr2=0x7fafdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafdc066f60 0x7fafdc069420 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fafec0ad000 tx=0x7fafec008040 comp rx=0 tx=0).stop 2024-04-25T08:57:02.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 -- 172.21.15.37:0/3750964914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faffc06bca0 msgr2=0x7faffc0ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faffc06bca0 0x7faffc0ad420 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fafe4007f20 tx=0x7fafe400d560 comp rx=0 tx=0).stop 2024-04-25T08:57:02.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 -- 172.21.15.37:0/3750964914 shutdown_connections 2024-04-25T08:57:02.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafdc066f60 0x7fafdc069420 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faffc0af370 0x7faffc0ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faffc06bca0 0x7faffc0ad420 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 --2- 172.21.15.37:0/3750964914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faffc066e30 0x7faffc142660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 -- 172.21.15.37:0/3750964914 >> 172.21.15.37:0/3750964914 conn(0x7faffc071810 msgr2=0x7faffc0b0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:02.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 -- 172.21.15.37:0/3750964914 shutdown_connections 2024-04-25T08:57:02.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.214+0000 7fb0023a9640 1 -- 172.21.15.37:0/3750964914 wait complete. 2024-04-25T08:57:02.223 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:57:02.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c28b67640 1 Processor -- start 2024-04-25T08:57:02.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c28b67640 1 -- start start 2024-04-25T08:57:02.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c28b67640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0c240b3f50 0x7f0c2415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c28b67640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c240b24a0 0x7f0c240b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c28b67640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c240b2090 con 0x7f0c2415a700 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c28b67640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2415f200 con 0x7f0c240b3f50 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c28b67640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240b43e0 con 0x7f0c240b24a0 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c21d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0c240b3f50 0x7f0c2415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c21d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0c240b3f50 0x7f0c2415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38162/0 (socket says 172.21.15.37:38162) 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c21d9b640 1 -- 172.21.15.37:0/2678299633 learned_addr learned my addr 172.21.15.37:0/2678299633 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c22d9d640 1 --2- 172.21.15.37:0/2678299633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c240b24a0 0x7f0c240b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.350+0000 7f0c21d9b640 1 -- 172.21.15.37:0/2678299633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c240b24a0 msgr2=0x7f0c240b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c21d9b640 1 --2- 172.21.15.37:0/2678299633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c240b24a0 0x7f0c240b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c21d9b640 1 -- 172.21.15.37:0/2678299633 >> v1:172.21.15.203:6789/0 conn(0x7f0c2415a700 legacy=0x7f0c240cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c21d9b640 1 -- 172.21.15.37:0/2678299633 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c240b46c0 con 0x7f0c240b3f50 2024-04-25T08:57:02.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c22d9d640 1 --2- 172.21.15.37:0/2678299633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c240b24a0 0x7f0c240b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:02.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c21d9b640 1 --2- 172.21.15.37:0/2678299633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c240b3f50 0x7f0c2415ecc0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0c240b3e20 tx=0x7f0c1801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4442aa30585d0e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c2159a640 1 -- 172.21.15.37:0/2678299633 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c180b0070 con 0x7f0c240b3f50 2024-04-25T08:57:02.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c2159a640 1 -- 172.21.15.37:0/2678299633 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c180ab600 con 0x7f0c240b3f50 2024-04-25T08:57:02.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c2159a640 1 -- 172.21.15.37:0/2678299633 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c180ab9d0 con 0x7f0c240b3f50 2024-04-25T08:57:02.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c28b67640 1 -- 172.21.15.37:0/2678299633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c240b3f50 msgr2=0x7f0c2415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c28b67640 1 --2- 172.21.15.37:0/2678299633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c240b3f50 0x7f0c2415ecc0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f0c240b3e20 tx=0x7f0c1801c3c0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c28b67640 1 -- 172.21.15.37:0/2678299633 shutdown_connections 2024-04-25T08:57:02.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c28b67640 1 --2- 172.21.15.37:0/2678299633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c240b24a0 0x7f0c240b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c28b67640 1 --2- 172.21.15.37:0/2678299633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c240b3f50 0x7f0c2415ecc0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c28b67640 1 -- 172.21.15.37:0/2678299633 >> 172.21.15.37:0/2678299633 conn(0x7f0c240b11a0 msgr2=0x7f0c240b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:02.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c28b67640 1 -- 172.21.15.37:0/2678299633 shutdown_connections 2024-04-25T08:57:02.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.354+0000 7f0c28b67640 1 -- 172.21.15.37:0/2678299633 wait complete. 2024-04-25T08:57:02.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c28b67640 1 Processor -- start 2024-04-25T08:57:02.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c28b67640 1 -- start start 2024-04-25T08:57:02.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c28b67640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c240b24a0 0x7f0c24153120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c28b67640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c2415a700 0x7f0c24151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c28b67640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c24158510 0x7f0c24154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c28b67640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240cd310 con 0x7f0c24158510 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c28b67640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240ce600 con 0x7f0c2415a700 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c28b67640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c240cd490 con 0x7f0c240b24a0 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c22d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c24158510 0x7f0c24154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c22d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c24158510 0x7f0c24154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38172/0 (socket says 172.21.15.37:38172) 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c22d9d640 1 -- 172.21.15.37:0/3851993861 learned_addr learned my addr 172.21.15.37:0/3851993861 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c2259c640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c240b24a0 0x7f0c24153120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c21d9b640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c2415a700 0x7f0c24151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c22d9d640 1 -- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c240b24a0 msgr2=0x7f0c24153120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c22d9d640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c240b24a0 0x7f0c24153120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c22d9d640 1 -- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c2415a700 msgr2=0x7f0c24151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c22d9d640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c2415a700 0x7f0c24151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c22d9d640 1 -- 172.21.15.37:0/3851993861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c24153890 con 0x7f0c24158510 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c22d9d640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c24158510 0x7f0c24154b60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00cd80 tx=0x7f0c1c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c12ffd640 1 -- 172.21.15.37:0/3851993861 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00f680 con 0x7f0c24158510 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c12ffd640 1 -- 172.21.15.37:0/3851993861 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c00f840 con 0x7f0c24158510 2024-04-25T08:57:02.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c28b67640 1 -- 172.21.15.37:0/3851993861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2415f4b0 con 0x7f0c24158510 2024-04-25T08:57:02.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c12ffd640 1 -- 172.21.15.37:0/3851993861 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c00fc30 con 0x7f0c24158510 2024-04-25T08:57:02.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c28b67640 1 -- 172.21.15.37:0/3851993861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c241625e0 con 0x7f0c24158510 2024-04-25T08:57:02.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c28b67640 1 -- 172.21.15.37:0/3851993861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c2404efa0 con 0x7f0c24158510 2024-04-25T08:57:02.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c12ffd640 1 -- 172.21.15.37:0/3851993861 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0c1c0213f0 con 0x7f0c24158510 2024-04-25T08:57:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.358+0000 7f0c12ffd640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c04066f60 0x7f0c04069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.362+0000 7f0c12ffd640 1 -- 172.21.15.37:0/3851993861 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c1c085ea0 con 0x7f0c24158510 2024-04-25T08:57:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.362+0000 7f0c2259c640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c04066f60 0x7f0c04069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.362+0000 7f0c2259c640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c04066f60 0x7f0c04069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0c2415ec00 tx=0x7f0c14009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.366+0000 7f0c12ffd640 1 -- 172.21.15.37:0/3851993861 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0c1c014030 con 0x7f0c24158510 2024-04-25T08:57:02.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.530+0000 7f0c28b67640 1 -- 172.21.15.37:0/3851993861 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0c241653e0 con 0x7f0c24158510 2024-04-25T08:57:02.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c12ffd640 1 -- 172.21.15.37:0/3851993861 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v82) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0c1c056db0 con 0x7f0c24158510 2024-04-25T08:57:02.522 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:57:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 -- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c04066f60 msgr2=0x7f0c04069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c04066f60 0x7f0c04069420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f0c2415ec00 tx=0x7f0c14009040 comp rx=0 tx=0).stop 2024-04-25T08:57:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 -- 172.21.15.37:0/3851993861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c24158510 msgr2=0x7f0c24154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c24158510 0x7f0c24154b60 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c00cd80 tx=0x7f0c1c00a650 comp rx=0 tx=0).stop 2024-04-25T08:57:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 -- 172.21.15.37:0/3851993861 shutdown_connections 2024-04-25T08:57:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0c04066f60 0x7f0c04069420 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c24158510 0x7f0c24154b60 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0c2415a700 0x7f0c24151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 --2- 172.21.15.37:0/3851993861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0c240b24a0 0x7f0c24153120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 -- 172.21.15.37:0/3851993861 >> 172.21.15.37:0/3851993861 conn(0x7f0c240b11a0 msgr2=0x7f0c240b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 -- 172.21.15.37:0/3851993861 shutdown_connections 2024-04-25T08:57:02.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.534+0000 7f0c28b67640 1 -- 172.21.15.37:0/3851993861 wait complete. 2024-04-25T08:57:02.540 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:57:02.540 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:02.546 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:02.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.714+0000 7f51507ee640 1 Processor -- start 2024-04-25T08:57:02.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.714+0000 7f51507ee640 1 -- start start 2024-04-25T08:57:02.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f51507ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480d4230 0x7f51480d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f51507ee640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514806be60 con 0x7f51480d07f0 2024-04-25T08:57:02.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f51507ee640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514815de60 con 0x7f514815da80 2024-04-25T08:57:02.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f51507ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51480d46a0 con 0x7f51480d4230 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514dd89640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f51480d07f0 0x7f514806c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:43986/0 (socket says 172.21.15.37:43986) 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514dd89640 1 -- 172.21.15.37:0/2503269149 learned_addr learned my addr 172.21.15.37:0/2503269149 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514ed8b640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480d4230 0x7f51480d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201597364 0 0) 0x7f514806be60 con 0x7f51480d07f0 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5130003530 con 0x7f51480d07f0 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3630239252 0 0) 0x7f5130003530 con 0x7f51480d07f0 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480d4230 msgr2=0x7f51480d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480d4230 0x7f51480d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 >> v1:172.21.15.203:6789/0 conn(0x7f514815da80 legacy=0x7f51480d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51480d2450 con 0x7f51480d07f0 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f5144001ba0 con 0x7f51480d07f0 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 >> v1:172.21.15.37:6789/0 conn(0x7f51480d07f0 legacy=0x7f514806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480d4230 0x7f5130003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5130004000 0x7f5130004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5130004d50 0x7f51300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51480d46a0 con 0x7f5130004000 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513000bb60 con 0x7f5130004d50 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513000bd60 con 0x7f51480d4230 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514dd89640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5130004000 0x7f5130004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514e58a640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480d4230 0x7f5130003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514ed8b640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5130004d50 0x7f51300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514dd89640 1 -- 172.21.15.37:0/2503269149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480d4230 msgr2=0x7f5130003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514dd89640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480d4230 0x7f5130003a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514dd89640 1 -- 172.21.15.37:0/2503269149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5130004d50 msgr2=0x7f51300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514dd89640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5130004d50 0x7f51300051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514dd89640 1 -- 172.21.15.37:0/2503269149 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130003530 con 0x7f5130004000 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514dd89640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5130004000 0x7f5130004810 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f514806de60 tx=0x7f514401f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f4f334bf45d326 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.718+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514400c1a0 con 0x7f5130004000 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5144005df0 con 0x7f5130004000 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514d588640 1 -- 172.21.15.37:0/2503269149 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51440072f0 con 0x7f5130004000 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 -- 172.21.15.37:0/2503269149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5130004000 msgr2=0x7f5130004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 --2- 172.21.15.37:0/2503269149 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5130004000 0x7f5130004810 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f514806de60 tx=0x7f514401f740 comp rx=0 tx=0).stop 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514ed8b640 1 -- 172.21.15.37:0/2503269149 reap_dead start 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 -- 172.21.15.37:0/2503269149 shutdown_connections 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 -- 172.21.15.37:0/2503269149 >> 172.21.15.37:0/2503269149 conn(0x7f514808b000 msgr2=0x7f51480d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 -- 172.21.15.37:0/2503269149 shutdown_connections 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 -- 172.21.15.37:0/2503269149 wait complete. 2024-04-25T08:57:02.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 Processor -- start 2024-04-25T08:57:02.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 -- start start 2024-04-25T08:57:02.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5148157ee0 0x7f51480b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480ca7c0 0x7f51480b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51480b1160 0x7f51480b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51480d26f0 con 0x7f51480ca7c0 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514806c440 con 0x7f51480b1160 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51480d2870 con 0x7f5148157ee0 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514dd89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480ca7c0 0x7f51480b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514dd89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480ca7c0 0x7f51480b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38188/0 (socket says 172.21.15.37:38188) 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514dd89640 1 -- 172.21.15.37:0/43665315 learned_addr learned my addr 172.21.15.37:0/43665315 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514ed8b640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51480b1160 0x7f51480b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514e58a640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5148157ee0 0x7f51480b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514dd89640 1 -- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5148157ee0 msgr2=0x7f51480b8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514dd89640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5148157ee0 0x7f51480b8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514dd89640 1 -- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51480b1160 msgr2=0x7f51480b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514dd89640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51480b1160 0x7f51480b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514dd89640 1 -- 172.21.15.37:0/43665315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51480cacc0 con 0x7f51480ca7c0 2024-04-25T08:57:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f514dd89640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480ca7c0 0x7f51480b0c20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f514401f570 tx=0x7f5130005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f513effd640 1 -- 172.21.15.37:0/43665315 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514400c1a0 con 0x7f51480ca7c0 2024-04-25T08:57:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f513effd640 1 -- 172.21.15.37:0/43665315 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51440070d0 con 0x7f51480ca7c0 2024-04-25T08:57:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f513effd640 1 -- 172.21.15.37:0/43665315 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144001300 con 0x7f51480ca7c0 2024-04-25T08:57:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 -- 172.21.15.37:0/43665315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51480cafa0 con 0x7f51480ca7c0 2024-04-25T08:57:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.722+0000 7f51507ee640 1 -- 172.21.15.37:0/43665315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51480cb560 con 0x7f51480ca7c0 2024-04-25T08:57:02.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.726+0000 7f51507ee640 1 -- 172.21.15.37:0/43665315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514804efa0 con 0x7f51480ca7c0 2024-04-25T08:57:02.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.726+0000 7f513effd640 1 -- 172.21.15.37:0/43665315 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51440074c0 con 0x7f51480ca7c0 2024-04-25T08:57:02.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.726+0000 7f513effd640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f90 0x7f5134069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.726+0000 7f513effd640 1 -- 172.21.15.37:0/43665315 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f51441224f0 con 0x7f51480ca7c0 2024-04-25T08:57:02.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.726+0000 7f514e58a640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f90 0x7f5134069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.726+0000 7f514e58a640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f90 0x7f5134069450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5140000c10 tx=0x7f5140009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.730+0000 7f513effd640 1 -- 172.21.15.37:0/43665315 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51440c4030 con 0x7f51480ca7c0 2024-04-25T08:57:02.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.886+0000 7f51507ee640 1 -- 172.21.15.37:0/43665315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51480b1540 con 0x7f51480ca7c0 2024-04-25T08:57:02.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.890+0000 7f513effd640 1 -- 172.21.15.37:0/43665315 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16909 (secure 0 0 0) 0x7f5144019450 con 0x7f51480ca7c0 2024-04-25T08:57:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:57:02.881 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":82,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:56:53.098938+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:56:49.737591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:56:50.436964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:02.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.890+0000 7f51507ee640 1 -- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f90 msgr2=0x7f5134069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.890+0000 7f51507ee640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f90 0x7f5134069450 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5140000c10 tx=0x7f5140009040 comp rx=0 tx=0).stop 2024-04-25T08:57:02.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.890+0000 7f51507ee640 1 -- 172.21.15.37:0/43665315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480ca7c0 msgr2=0x7f51480b0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.890+0000 7f51507ee640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480ca7c0 0x7f51480b0c20 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f514401f570 tx=0x7f5130005840 comp rx=0 tx=0).stop 2024-04-25T08:57:02.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.894+0000 7f51507ee640 1 -- 172.21.15.37:0/43665315 shutdown_connections 2024-04-25T08:57:02.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.894+0000 7f51507ee640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5134066f90 0x7f5134069450 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.894+0000 7f51507ee640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51480b1160 0x7f51480b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.894+0000 7f51507ee640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480ca7c0 0x7f51480b0c20 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.894+0000 7f51507ee640 1 --2- 172.21.15.37:0/43665315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5148157ee0 0x7f51480b8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.894+0000 7f51507ee640 1 -- 172.21.15.37:0/43665315 >> 172.21.15.37:0/43665315 conn(0x7f514808b000 msgr2=0x7f51480d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:02.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.894+0000 7f51507ee640 1 -- 172.21.15.37:0/43665315 shutdown_connections 2024-04-25T08:57:02.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:02.894+0000 7f51507ee640 1 -- 172.21.15.37:0/43665315 wait complete. 2024-04-25T08:57:02.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:56:49.737591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:56:50.436964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:57:02.898 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:57:03.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7af0009640 1 Processor -- start 2024-04-25T08:57:03.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7af0009640 1 -- start start 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7af0009640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7ae8159bd0 0x7f7ae80c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7af0009640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ae8159fb0 0x7f7ae80b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7af0009640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ae80b26f0 0x7f7ae80b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7af0009640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae8072780 con 0x7f7ae8159bd0 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7af0009640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae80c9ef0 con 0x7f7ae8159fb0 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7af0009640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8071b70 con 0x7f7ae80b26f0 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7aedda5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7ae8159bd0 0x7f7ae80c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7aedda5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7ae8159bd0 0x7f7ae80c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38192/0 (socket says 172.21.15.37:38192) 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7aedda5640 1 -- 172.21.15.37:0/1822643700 learned_addr learned my addr 172.21.15.37:0/1822643700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7aee5a6640 1 --2- 172.21.15.37:0/1822643700 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ae80b26f0 0x7f7ae80b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7aed5a4640 1 --2- 172.21.15.37:0/1822643700 >> v2:172.21.15.203:3300/0 conn(0x7f7ae8159fb0 0x7f7ae80b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7aedda5640 1 -- 172.21.15.37:0/1822643700 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ae80b26f0 msgr2=0x7f7ae80b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7aedda5640 1 --2- 172.21.15.37:0/1822643700 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ae80b26f0 0x7f7ae80b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7aedda5640 1 -- 172.21.15.37:0/1822643700 >> v2:172.21.15.203:3300/0 conn(0x7f7ae8159fb0 msgr2=0x7f7ae80b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:03.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7aedda5640 1 --2- 172.21.15.37:0/1822643700 >> v2:172.21.15.203:3300/0 conn(0x7f7ae8159fb0 0x7f7ae80b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.026+0000 7f7aedda5640 1 -- 172.21.15.37:0/1822643700 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae8071e50 con 0x7f7ae8159bd0 2024-04-25T08:57:03.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aedda5640 1 --2- 172.21.15.37:0/1822643700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae8159bd0 0x7f7ae80c99b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7ae806bcc0 tx=0x7f7ae401e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79fefd36bf4a1c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aecda3640 1 -- 172.21.15.37:0/1822643700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40ad070 con 0x7f7ae8159bd0 2024-04-25T08:57:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aecda3640 1 -- 172.21.15.37:0/1822643700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae4004230 con 0x7f7ae8159bd0 2024-04-25T08:57:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aecda3640 1 -- 172.21.15.37:0/1822643700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae40a77b0 con 0x7f7ae8159bd0 2024-04-25T08:57:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 -- 172.21.15.37:0/1822643700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae8159bd0 msgr2=0x7f7ae80c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 --2- 172.21.15.37:0/1822643700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae8159bd0 0x7f7ae80c99b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f7ae806bcc0 tx=0x7f7ae401e9a0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 -- 172.21.15.37:0/1822643700 shutdown_connections 2024-04-25T08:57:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 --2- 172.21.15.37:0/1822643700 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ae80b26f0 0x7f7ae80b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 --2- 172.21.15.37:0/1822643700 >> v2:172.21.15.203:3300/0 conn(0x7f7ae8159fb0 0x7f7ae80b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 --2- 172.21.15.37:0/1822643700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae8159bd0 0x7f7ae80c99b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 -- 172.21.15.37:0/1822643700 >> 172.21.15.37:0/1822643700 conn(0x7f7ae806ae60 msgr2=0x7f7ae806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 -- 172.21.15.37:0/1822643700 shutdown_connections 2024-04-25T08:57:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 -- 172.21.15.37:0/1822643700 wait complete. 2024-04-25T08:57:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 Processor -- start 2024-04-25T08:57:03.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 -- start start 2024-04-25T08:57:03.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ae80b26f0 0x7f7ae8098940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae8159bd0 0x7f7ae815c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ae8159fb0 0x7f7ae815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80cb500 con 0x7f7ae8159bd0 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80cb380 con 0x7f7ae8159fb0 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7af0009640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80cb680 con 0x7f7ae80b26f0 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aed5a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae8159bd0 0x7f7ae815c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aed5a4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae8159bd0 0x7f7ae815c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38202/0 (socket says 172.21.15.37:38202) 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aed5a4640 1 -- 172.21.15.37:0/1303035042 learned_addr learned my addr 172.21.15.37:0/1303035042 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aedda5640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ae80b26f0 0x7f7ae8098940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aee5a6640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ae8159fb0 0x7f7ae815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aed5a4640 1 -- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ae80b26f0 msgr2=0x7f7ae8098940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aed5a4640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ae80b26f0 0x7f7ae8098940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aed5a4640 1 -- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ae8159fb0 msgr2=0x7f7ae815ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aed5a4640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ae8159fb0 0x7f7ae815ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aed5a4640 1 -- 172.21.15.37:0/1303035042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae80c9220 con 0x7f7ae8159bd0 2024-04-25T08:57:03.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aedda5640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ae80b26f0 0x7f7ae8098940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:03.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.030+0000 7f7aed5a4640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae8159bd0 0x7f7ae815c970 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800d7d0 tx=0x7f7ad800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.034+0000 7f7ade7fc640 1 -- 172.21.15.37:0/1303035042 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8004020 con 0x7f7ae8159bd0 2024-04-25T08:57:03.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.034+0000 7f7ade7fc640 1 -- 172.21.15.37:0/1303035042 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad80041c0 con 0x7f7ae8159bd0 2024-04-25T08:57:03.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.034+0000 7f7af0009640 1 -- 172.21.15.37:0/1303035042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae80c6600 con 0x7f7ae8159bd0 2024-04-25T08:57:03.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.034+0000 7f7ade7fc640 1 -- 172.21.15.37:0/1303035042 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80043b0 con 0x7f7ae8159bd0 2024-04-25T08:57:03.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.034+0000 7f7af0009640 1 -- 172.21.15.37:0/1303035042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae80c6bc0 con 0x7f7ae8159bd0 2024-04-25T08:57:03.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.034+0000 7f7af0009640 1 -- 172.21.15.37:0/1303035042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae804efa0 con 0x7f7ae8159bd0 2024-04-25T08:57:03.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.034+0000 7f7ade7fc640 1 -- 172.21.15.37:0/1303035042 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7ad801b020 con 0x7f7ae8159bd0 2024-04-25T08:57:03.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.034+0000 7f7ade7fc640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7acc066f60 0x7f7acc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.034+0000 7f7ade7fc640 1 -- 172.21.15.37:0/1303035042 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7ad8084f30 con 0x7f7ae8159bd0 2024-04-25T08:57:03.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.034+0000 7f7aedda5640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7acc066f60 0x7f7acc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.034+0000 7f7aedda5640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7acc066f60 0x7f7acc069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40b1000 tx=0x7f7ae40a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.038+0000 7f7ade7fc640 1 -- 172.21.15.37:0/1303035042 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ad8055db0 con 0x7f7ae8159bd0 2024-04-25T08:57:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.198+0000 7f7af0009640 1 -- 172.21.15.37:0/1303035042 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7ae8098e80 con 0x7f7ae8159bd0 2024-04-25T08:57:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.198+0000 7f7ade7fc640 1 -- 172.21.15.37:0/1303035042 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ad80556b0 con 0x7f7ae8159bd0 2024-04-25T08:57:03.189 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:57:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 -- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7acc066f60 msgr2=0x7f7acc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7acc066f60 0x7f7acc069420 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40b1000 tx=0x7f7ae40a9070 comp rx=0 tx=0).stop 2024-04-25T08:57:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 -- 172.21.15.37:0/1303035042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae8159bd0 msgr2=0x7f7ae815c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae8159bd0 0x7f7ae815c970 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800d7d0 tx=0x7f7ad800dc90 comp rx=0 tx=0).stop 2024-04-25T08:57:03.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 -- 172.21.15.37:0/1303035042 shutdown_connections 2024-04-25T08:57:03.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7acc066f60 0x7f7acc069420 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ae8159fb0 0x7f7ae815ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ae8159bd0 0x7f7ae815c970 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 --2- 172.21.15.37:0/1303035042 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ae80b26f0 0x7f7ae8098940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 -- 172.21.15.37:0/1303035042 >> 172.21.15.37:0/1303035042 conn(0x7f7ae806ae60 msgr2=0x7f7ae80cd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:03.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 -- 172.21.15.37:0/1303035042 shutdown_connections 2024-04-25T08:57:03.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.202+0000 7f7af0009640 1 -- 172.21.15.37:0/1303035042 wait complete. 2024-04-25T08:57:03.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:57:03.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f32b19640 1 Processor -- start 2024-04-25T08:57:03.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f32b19640 1 -- start start 2024-04-25T08:57:03.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f32b19640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0f2c0cf590 0x7f0f2c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f32b19640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0f2c0ce170 0x7f0f2c0ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f32b19640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2c0ca4c0 con 0x7f0f2c0cf970 2024-04-25T08:57:03.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f32b19640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2c0ca780 con 0x7f0f2c0ce170 2024-04-25T08:57:03.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f32b19640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2c0cd440 con 0x7f0f2c0cf590 2024-04-25T08:57:03.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f310b6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0f2c0ce170 0x7f0f2c0ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f2bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0f2c0cf970 0x7f0f2c0c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44000/0 (socket says 172.21.15.37:44000) 2024-04-25T08:57:03.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f310b6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0f2c0ce170 0x7f0f2c0ce600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38216/0 (socket says 172.21.15.37:38216) 2024-04-25T08:57:03.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f2bfff640 1 -- 172.21.15.37:0/3954205739 learned_addr learned my addr 172.21.15.37:0/3954205739 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:03.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f308b5640 1 --2- 172.21.15.37:0/3954205739 >> v2:172.21.15.203:3300/0 conn(0x7f0f2c0cf590 0x7f0f2c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f310b6640 1 -- 172.21.15.37:0/3954205739 >> v2:172.21.15.203:3300/0 conn(0x7f0f2c0cf590 msgr2=0x7f0f2c15ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:03.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f310b6640 1 --2- 172.21.15.37:0/3954205739 >> v2:172.21.15.203:3300/0 conn(0x7f0f2c0cf590 0x7f0f2c15ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f310b6640 1 -- 172.21.15.37:0/3954205739 >> v1:172.21.15.37:6789/0 conn(0x7f0f2c0cf970 legacy=0x7f0f2c0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.346+0000 7f0f310b6640 1 -- 172.21.15.37:0/3954205739 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c0cd690 con 0x7f0f2c0ce170 2024-04-25T08:57:03.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f310b6640 1 --2- 172.21.15.37:0/3954205739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f2c0ce170 0x7f0f2c0ce600 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0d0430 tx=0x7f0f2401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc2b5f5e9694f7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2b7fe640 1 -- 172.21.15.37:0/3954205739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f240b2070 con 0x7f0f2c0ce170 2024-04-25T08:57:03.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2b7fe640 1 -- 172.21.15.37:0/3954205739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f240a5900 con 0x7f0f2c0ce170 2024-04-25T08:57:03.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2b7fe640 1 -- 172.21.15.37:0/3954205739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f240a5cd0 con 0x7f0f2c0ce170 2024-04-25T08:57:03.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 -- 172.21.15.37:0/3954205739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f2c0ce170 msgr2=0x7f0f2c0ce600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 --2- 172.21.15.37:0/3954205739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f2c0ce170 0x7f0f2c0ce600 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0d0430 tx=0x7f0f2401c930 comp rx=0 tx=0).stop 2024-04-25T08:57:03.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 -- 172.21.15.37:0/3954205739 shutdown_connections 2024-04-25T08:57:03.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 --2- 172.21.15.37:0/3954205739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f2c0ce170 0x7f0f2c0ce600 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 --2- 172.21.15.37:0/3954205739 >> v2:172.21.15.203:3300/0 conn(0x7f0f2c0cf590 0x7f0f2c15ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 -- 172.21.15.37:0/3954205739 >> 172.21.15.37:0/3954205739 conn(0x7f0f2c06ae60 msgr2=0x7f0f2c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:03.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 -- 172.21.15.37:0/3954205739 shutdown_connections 2024-04-25T08:57:03.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 -- 172.21.15.37:0/3954205739 wait complete. 2024-04-25T08:57:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 Processor -- start 2024-04-25T08:57:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 -- start start 2024-04-25T08:57:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f2c0ce170 0x7f0f2c0c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f2c0cf590 0x7f0f2c0c2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2c0cf970 0x7f0f2c0c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0cfed0 con 0x7f0f2c0cf590 2024-04-25T08:57:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0cfd50 con 0x7f0f2c0ce170 2024-04-25T08:57:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0ac170 con 0x7f0f2c0cf970 2024-04-25T08:57:03.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f2c0cf590 0x7f0f2c0c2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f2c0cf590 0x7f0f2c0c2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38230/0 (socket says 172.21.15.37:38230) 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2bfff640 1 -- 172.21.15.37:0/2141050610 learned_addr learned my addr 172.21.15.37:0/2141050610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f308b5640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f2c0ce170 0x7f0f2c0c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f310b6640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2c0cf970 0x7f0f2c0c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2bfff640 1 -- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2c0cf970 msgr2=0x7f0f2c0c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2bfff640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2c0cf970 0x7f0f2c0c33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2bfff640 1 -- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f2c0ce170 msgr2=0x7f0f2c0c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2bfff640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f2c0ce170 0x7f0f2c0c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2bfff640 1 -- 172.21.15.37:0/2141050610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c15d870 con 0x7f0f2c0cf590 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f308b5640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f2c0ce170 0x7f0f2c0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f310b6640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2c0cf970 0x7f0f2c0c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f2bfff640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f2c0cf590 0x7f0f2c0c2e90 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800c500 tx=0x7f0f1800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f297fa640 1 -- 172.21.15.37:0/2141050610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f18017070 con 0x7f0f2c0cf590 2024-04-25T08:57:03.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f297fa640 1 -- 172.21.15.37:0/2141050610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f18005cf0 con 0x7f0f2c0cf590 2024-04-25T08:57:03.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 -- 172.21.15.37:0/2141050610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2c0c3c10 con 0x7f0f2c0cf590 2024-04-25T08:57:03.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f297fa640 1 -- 172.21.15.37:0/2141050610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f180072f0 con 0x7f0f2c0cf590 2024-04-25T08:57:03.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.350+0000 7f0f32b19640 1 -- 172.21.15.37:0/2141050610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2c0c3910 con 0x7f0f2c0cf590 2024-04-25T08:57:03.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.354+0000 7f0f32b19640 1 -- 172.21.15.37:0/2141050610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2c04efa0 con 0x7f0f2c0cf590 2024-04-25T08:57:03.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.354+0000 7f0f297fa640 1 -- 172.21.15.37:0/2141050610 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f1801a070 con 0x7f0f2c0cf590 2024-04-25T08:57:03.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.354+0000 7f0f297fa640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0f10066f90 0x7f0f10069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.354+0000 7f0f297fa640 1 -- 172.21.15.37:0/2141050610 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f180843c0 con 0x7f0f2c0cf590 2024-04-25T08:57:03.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.354+0000 7f0f308b5640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0f10066f90 0x7f0f10069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.354+0000 7f0f308b5640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0f10066f90 0x7f0f10069450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009840 tx=0x7f0f20008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.358+0000 7f0f297fa640 1 -- 172.21.15.37:0/2141050610 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f18054c80 con 0x7f0f2c0cf590 2024-04-25T08:57:03.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 -- 172.21.15.37:0/2141050610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f2c0af380 con 0x7f0f2c0cf590 2024-04-25T08:57:03.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f297fa640 1 -- 172.21.15.37:0/2141050610 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f18058d20 con 0x7f0f2c0cf590 2024-04-25T08:57:03.504 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:57:03.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 -- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0f10066f90 msgr2=0x7f0f10069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0f10066f90 0x7f0f10069450 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009840 tx=0x7f0f20008040 comp rx=0 tx=0).stop 2024-04-25T08:57:03.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 -- 172.21.15.37:0/2141050610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f2c0cf590 msgr2=0x7f0f2c0c2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f2c0cf590 0x7f0f2c0c2e90 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f0f1800c500 tx=0x7f0f1800c9c0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 -- 172.21.15.37:0/2141050610 shutdown_connections 2024-04-25T08:57:03.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0f10066f90 0x7f0f10069450 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f2c0cf970 0x7f0f2c0c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f2c0cf590 0x7f0f2c0c2e90 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 --2- 172.21.15.37:0/2141050610 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f2c0ce170 0x7f0f2c0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 -- 172.21.15.37:0/2141050610 >> 172.21.15.37:0/2141050610 conn(0x7f0f2c06ae60 msgr2=0x7f0f2c0ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:03.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 -- 172.21.15.37:0/2141050610 shutdown_connections 2024-04-25T08:57:03.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.514+0000 7f0f32b19640 1 -- 172.21.15.37:0/2141050610 wait complete. 2024-04-25T08:57:03.521 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:57:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f52a0172640 1 Processor -- start 2024-04-25T08:57:03.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f52a0172640 1 -- start start 2024-04-25T08:57:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f52a0172640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f529806bca0 0x7f52980cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f52a0172640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5298066e30 0x7f529806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f52a0172640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529806ec30 con 0x7f52980af370 2024-04-25T08:57:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f52a0172640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52980669a0 con 0x7f529806bca0 2024-04-25T08:57:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f52a0172640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52980cd5f0 con 0x7f5298066e30 2024-04-25T08:57:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f529df0e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f529806bca0 0x7f52980cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f529df0e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f529806bca0 0x7f52980cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38242/0 (socket says 172.21.15.37:38242) 2024-04-25T08:57:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f529df0e640 1 -- 172.21.15.37:0/1505164469 learned_addr learned my addr 172.21.15.37:0/1505164469 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f529d70d640 1 --2- 172.21.15.37:0/1505164469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5298066e30 0x7f529806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f529df0e640 1 -- 172.21.15.37:0/1505164469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5298066e30 msgr2=0x7f529806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:03.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f529df0e640 1 --2- 172.21.15.37:0/1505164469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5298066e30 0x7f529806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f529df0e640 1 -- 172.21.15.37:0/1505164469 >> v1:172.21.15.203:6789/0 conn(0x7f52980af370 legacy=0x7f529806d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:03.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f529df0e640 1 -- 172.21.15.37:0/1505164469 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529806c5c0 con 0x7f529806bca0 2024-04-25T08:57:03.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.650+0000 7f529d70d640 1 --2- 172.21.15.37:0/1505164469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5298066e30 0x7f529806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:03.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f529df0e640 1 --2- 172.21.15.37:0/1505164469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529806bca0 0x7f52980cd0b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5290009e50 tx=0x7f529001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e961ca5c80d595c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f529cf0c640 1 -- 172.21.15.37:0/1505164469 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52900a56f0 con 0x7f529806bca0 2024-04-25T08:57:03.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f529cf0c640 1 -- 172.21.15.37:0/1505164469 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52900a5890 con 0x7f529806bca0 2024-04-25T08:57:03.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f529cf0c640 1 -- 172.21.15.37:0/1505164469 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52900a5c60 con 0x7f529806bca0 2024-04-25T08:57:03.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f52a0172640 1 -- 172.21.15.37:0/1505164469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529806bca0 msgr2=0x7f52980cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f52a0172640 1 --2- 172.21.15.37:0/1505164469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529806bca0 0x7f52980cd0b0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5290009e50 tx=0x7f529001c930 comp rx=0 tx=0).stop 2024-04-25T08:57:03.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f52a0172640 1 -- 172.21.15.37:0/1505164469 shutdown_connections 2024-04-25T08:57:03.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f52a0172640 1 --2- 172.21.15.37:0/1505164469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5298066e30 0x7f529806c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f52a0172640 1 --2- 172.21.15.37:0/1505164469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529806bca0 0x7f52980cd0b0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f52a0172640 1 -- 172.21.15.37:0/1505164469 >> 172.21.15.37:0/1505164469 conn(0x7f5298071810 msgr2=0x7f5298071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:03.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f52a0172640 1 -- 172.21.15.37:0/1505164469 shutdown_connections 2024-04-25T08:57:03.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f52a0172640 1 -- 172.21.15.37:0/1505164469 wait complete. 2024-04-25T08:57:03.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f52a0172640 1 Processor -- start 2024-04-25T08:57:03.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.654+0000 7f52a0172640 1 -- start start 2024-04-25T08:57:03.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52a0172640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5298066e30 0x7f5298158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52a0172640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529806bca0 0x7f52980ad660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52a0172640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52980af370 0x7f5298157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52a0172640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529815cc80 con 0x7f529806bca0 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52a0172640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529815cb00 con 0x7f52980af370 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52a0172640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5298168630 con 0x7f5298066e30 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529d70d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529806bca0 0x7f52980ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529d70d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529806bca0 0x7f52980ad660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38246/0 (socket says 172.21.15.37:38246) 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529d70d640 1 -- 172.21.15.37:0/94597647 learned_addr learned my addr 172.21.15.37:0/94597647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529df0e640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5298066e30 0x7f5298158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529e70f640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52980af370 0x7f5298157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529d70d640 1 -- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5298066e30 msgr2=0x7f5298158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529d70d640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5298066e30 0x7f5298158e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529d70d640 1 -- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52980af370 msgr2=0x7f5298157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529d70d640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52980af370 0x7f5298157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529d70d640 1 -- 172.21.15.37:0/94597647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52980b3ca0 con 0x7f529806bca0 2024-04-25T08:57:03.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529d70d640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529806bca0 0x7f52980ad660 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c7f0 tx=0x7f528c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52967fc640 1 -- 172.21.15.37:0/94597647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c015070 con 0x7f529806bca0 2024-04-25T08:57:03.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52967fc640 1 -- 172.21.15.37:0/94597647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c007820 con 0x7f529806bca0 2024-04-25T08:57:03.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52a0172640 1 -- 172.21.15.37:0/94597647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52980b4230 con 0x7f529806bca0 2024-04-25T08:57:03.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52967fc640 1 -- 172.21.15.37:0/94597647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c007c10 con 0x7f529806bca0 2024-04-25T08:57:03.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52a0172640 1 -- 172.21.15.37:0/94597647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5298159350 con 0x7f529806bca0 2024-04-25T08:57:03.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52a0172640 1 -- 172.21.15.37:0/94597647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529804efa0 con 0x7f529806bca0 2024-04-25T08:57:03.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52967fc640 1 -- 172.21.15.37:0/94597647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f528c018070 con 0x7f529806bca0 2024-04-25T08:57:03.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52967fc640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 0x7f5284069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f52967fc640 1 -- 172.21.15.37:0/94597647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f528c0852d0 con 0x7f529806bca0 2024-04-25T08:57:03.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.658+0000 7f529df0e640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 0x7f5284069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.662+0000 7f529df0e640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 0x7f5284069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f52900ae000 tx=0x7f52900a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.662+0000 7f52967fc640 1 -- 172.21.15.37:0/94597647 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f528c056150 con 0x7f529806bca0 2024-04-25T08:57:03.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 -- 172.21.15.37:0/94597647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5298072910 con 0x7f529806bca0 2024-04-25T08:57:03.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52967fc640 1 -- 172.21.15.37:0/94597647 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7f528c0850f0 con 0x7f529806bca0 2024-04-25T08:57:03.812 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:57:03.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 -- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 msgr2=0x7f5284069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 0x7f5284069420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f52900ae000 tx=0x7f52900a7070 comp rx=0 tx=0).stop 2024-04-25T08:57:03.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 -- 172.21.15.37:0/94597647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529806bca0 msgr2=0x7f52980ad660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529806bca0 0x7f52980ad660 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f528c00c7f0 tx=0x7f528c00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 -- 172.21.15.37:0/94597647 shutdown_connections 2024-04-25T08:57:03.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 0x7f5284069420 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52980af370 0x7f5298157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529806bca0 0x7f52980ad660 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 --2- 172.21.15.37:0/94597647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5298066e30 0x7f5298158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 -- 172.21.15.37:0/94597647 >> 172.21.15.37:0/94597647 conn(0x7f5298071810 msgr2=0x7f5298159680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 -- 172.21.15.37:0/94597647 shutdown_connections 2024-04-25T08:57:03.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.822+0000 7f52a0172640 1 -- 172.21.15.37:0/94597647 wait complete. 2024-04-25T08:57:03.831 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:57:03.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.954+0000 7f45846c4640 1 Processor -- start 2024-04-25T08:57:03.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.954+0000 7f45846c4640 1 -- start start 2024-04-25T08:57:03.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.954+0000 7f45846c4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f457c159ea0 0x7f457c0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.954+0000 7f45846c4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f457c15ae30 con 0x7f457c15a280 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.954+0000 7f45846c4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f457c15b110 con 0x7f457c071fd0 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.954+0000 7f45846c4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457c0b2990 con 0x7f457c159ea0 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.954+0000 7f4581c5f640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f457c15a280 0x7f457c0718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:44004/0 (socket says 172.21.15.37:44004) 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.954+0000 7f4581c5f640 1 -- 172.21.15.37:0/138619944 learned_addr learned my addr 172.21.15.37:0/138619944 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f4582460640 1 --2- 172.21.15.37:0/138619944 >> v2:172.21.15.203:3300/0 conn(0x7f457c159ea0 0x7f457c0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1333720829 0 0) 0x7f457c15ae30 con 0x7f457c15a280 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4568003450 con 0x7f457c15a280 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1717077945 0 0) 0x7f4568003450 con 0x7f457c15a280 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 >> v2:172.21.15.203:3300/0 conn(0x7f457c159ea0 msgr2=0x7f457c0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 --2- 172.21.15.37:0/138619944 >> v2:172.21.15.203:3300/0 conn(0x7f457c159ea0 0x7f457c0b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 >> v1:172.21.15.203:6789/0 conn(0x7f457c071fd0 legacy=0x7f457c06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c0b2620 con 0x7f457c15a280 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f456c001680 con 0x7f457c15a280 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 >> v1:172.21.15.37:6789/0 conn(0x7f457c15a280 legacy=0x7f457c0718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 --2- 172.21.15.37:0/138619944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4568003b20 0x7f4568003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 --2- 172.21.15.37:0/138619944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680088b0 0x7f456800ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 --2- 172.21.15.37:0/138619944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f456800b540 0x7f456800da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c0b2990 con 0x7f45680088b0 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456800e2e0 con 0x7f456800b540 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456800e4e0 con 0x7f4568003b20 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f4581c5f640 1 --2- 172.21.15.37:0/138619944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680088b0 0x7f456800ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f4582460640 1 --2- 172.21.15.37:0/138619944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4568003b20 0x7f4568003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f4582c61640 1 --2- 172.21.15.37:0/138619944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f456800b540 0x7f456800da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f4581c5f640 1 -- 172.21.15.37:0/138619944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4568003b20 msgr2=0x7f4568003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f4581c5f640 1 --2- 172.21.15.37:0/138619944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4568003b20 0x7f4568003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f4581c5f640 1 -- 172.21.15.37:0/138619944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f456800b540 msgr2=0x7f456800da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:03.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f4581c5f640 1 --2- 172.21.15.37:0/138619944 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f456800b540 0x7f456800da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f4581c5f640 1 -- 172.21.15.37:0/138619944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4568003450 con 0x7f45680088b0 2024-04-25T08:57:03.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f4582c61640 1 -- 172.21.15.37:0/138619944 reap_dead start 2024-04-25T08:57:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f4581c5f640 1 --2- 172.21.15.37:0/138619944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680088b0 0x7f456800ad40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f456c00c1c0 tx=0x7f456c01f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbc40f363ab3644 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c007d00 con 0x7f45680088b0 2024-04-25T08:57:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c005b00 con 0x7f45680088b0 2024-04-25T08:57:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f458145e640 1 -- 172.21.15.37:0/138619944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c007070 con 0x7f45680088b0 2024-04-25T08:57:03.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f45846c4640 1 -- 172.21.15.37:0/138619944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680088b0 msgr2=0x7f456800ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f45846c4640 1 --2- 172.21.15.37:0/138619944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680088b0 0x7f456800ad40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f456c00c1c0 tx=0x7f456c01f3d0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f45846c4640 1 -- 172.21.15.37:0/138619944 shutdown_connections 2024-04-25T08:57:03.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f45846c4640 1 --2- 172.21.15.37:0/138619944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45680088b0 0x7f456800ad40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f45846c4640 1 -- 172.21.15.37:0/138619944 >> 172.21.15.37:0/138619944 conn(0x7f457c0b11a0 msgr2=0x7f457c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:03.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f45846c4640 1 -- 172.21.15.37:0/138619944 shutdown_connections 2024-04-25T08:57:03.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.958+0000 7f45846c4640 1 -- 172.21.15.37:0/138619944 wait complete. 2024-04-25T08:57:03.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f45846c4640 1 Processor -- start 2024-04-25T08:57:03.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f45846c4640 1 -- start start 2024-04-25T08:57:03.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f45846c4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45680088b0 0x7f457c152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f45846c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f457c1eed80 0x7f457c1f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f45846c4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f457c1ef200 0x7f457c1f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f45846c4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c15a880 con 0x7f457c1eed80 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f45846c4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c15a700 con 0x7f45680088b0 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f45846c4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457c15aa00 con 0x7f457c1ef200 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4581c5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f457c1eed80 0x7f457c1f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4581c5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f457c1eed80 0x7f457c1f2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38274/0 (socket says 172.21.15.37:38274) 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4581c5f640 1 -- 172.21.15.37:0/3424608071 learned_addr learned my addr 172.21.15.37:0/3424608071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4582460640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45680088b0 0x7f457c152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4581c5f640 1 -- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f457c1ef200 msgr2=0x7f457c1f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4581c5f640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f457c1ef200 0x7f457c1f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4581c5f640 1 -- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45680088b0 msgr2=0x7f457c152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4581c5f640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45680088b0 0x7f457c152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4581c5f640 1 -- 172.21.15.37:0/3424608071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c1526c0 con 0x7f457c1eed80 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4582460640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45680088b0 0x7f457c152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:03.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4581c5f640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f457c1eed80 0x7f457c1f2c00 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f456c01f200 tx=0x7f4568003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4572ffd640 1 -- 172.21.15.37:0/3424608071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c0bf070 con 0x7f457c1eed80 2024-04-25T08:57:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4572ffd640 1 -- 172.21.15.37:0/3424608071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456c000ee0 con 0x7f457c1eed80 2024-04-25T08:57:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f45846c4640 1 -- 172.21.15.37:0/3424608071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457c150390 con 0x7f457c1eed80 2024-04-25T08:57:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4572ffd640 1 -- 172.21.15.37:0/3424608071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456c007d00 con 0x7f457c1eed80 2024-04-25T08:57:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f45846c4640 1 -- 172.21.15.37:0/3424608071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457c150950 con 0x7f457c1eed80 2024-04-25T08:57:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f45846c4640 1 -- 172.21.15.37:0/3424608071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457c04efa0 con 0x7f457c1eed80 2024-04-25T08:57:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.962+0000 7f4572ffd640 1 -- 172.21.15.37:0/3424608071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f456c0c2070 con 0x7f457c1eed80 2024-04-25T08:57:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.966+0000 7f4572ffd640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4560066f90 0x7f4560069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.966+0000 7f4572ffd640 1 -- 172.21.15.37:0/3424608071 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f456c1221b0 con 0x7f457c1eed80 2024-04-25T08:57:03.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.966+0000 7f4582460640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4560066f90 0x7f4560069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.966+0000 7f4582460640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4560066f90 0x7f4560069450 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4578009aa0 tx=0x7f4578008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:03.970+0000 7f4572ffd640 1 -- 172.21.15.37:0/3424608071 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f456c0190e0 con 0x7f457c1eed80 2024-04-25T08:57:04.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 -- 172.21.15.37:0/3424608071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f457c06e6b0 con 0x7f457c1eed80 2024-04-25T08:57:04.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f4572ffd640 1 -- 172.21.15.37:0/3424608071 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v82) v1 ==== 80+0+0 (secure 0 0 0) 0x7f456c0f30c0 con 0x7f457c1eed80 2024-04-25T08:57:04.132 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:57:04.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 -- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4560066f90 msgr2=0x7f4560069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4560066f90 0x7f4560069450 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4578009aa0 tx=0x7f4578008040 comp rx=0 tx=0).stop 2024-04-25T08:57:04.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 -- 172.21.15.37:0/3424608071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f457c1eed80 msgr2=0x7f457c1f2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f457c1eed80 0x7f457c1f2c00 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f456c01f200 tx=0x7f4568003740 comp rx=0 tx=0).stop 2024-04-25T08:57:04.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 -- 172.21.15.37:0/3424608071 shutdown_connections 2024-04-25T08:57:04.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4560066f90 0x7f4560069450 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f457c1ef200 0x7f457c1f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f457c1eed80 0x7f457c1f2c00 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 --2- 172.21.15.37:0/3424608071 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45680088b0 0x7f457c152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 -- 172.21.15.37:0/3424608071 >> 172.21.15.37:0/3424608071 conn(0x7f457c0b11a0 msgr2=0x7f457c158f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:04.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 -- 172.21.15.37:0/3424608071 shutdown_connections 2024-04-25T08:57:04.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:04.142+0000 7f45846c4640 1 -- 172.21.15.37:0/3424608071 wait complete. 2024-04-25T08:57:04.150 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T08:57:04.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:04.150 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:04.161 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[44994]: fuse finished with error 0 and tester_r 0 2024-04-25T08:57:04.162 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:57:10.264 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:57:10.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:10.265 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:57:10.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:10.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f626ae43640 1 Processor -- start 2024-04-25T08:57:10.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f626ae43640 1 -- start start 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f626ae43640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626406f4a0 0x7f6264099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f626ae43640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6264099fb0 0x7f626415d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f626ae43640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62640b0e10 con 0x7f62640b0a30 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f626ae43640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62640d0d90 con 0x7f6264099fb0 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f626ae43640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62640d0180 con 0x7f626406f4a0 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f62693e0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6264099fb0 0x7f626415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f62693e0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6264099fb0 0x7f626415d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40500/0 (socket says 172.21.15.37:40500) 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f62693e0640 1 -- 172.21.15.37:0/2578578662 learned_addr learned my addr 172.21.15.37:0/2578578662 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f6263fff640 1 --2- 172.21.15.37:0/2578578662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626406f4a0 0x7f6264099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f62693e0640 1 -- 172.21.15.37:0/2578578662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626406f4a0 msgr2=0x7f6264099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f62693e0640 1 --2- 172.21.15.37:0/2578578662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626406f4a0 0x7f6264099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f62693e0640 1 -- 172.21.15.37:0/2578578662 >> v1:172.21.15.203:6789/0 conn(0x7f62640b0a30 legacy=0x7f62640cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:10.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.457+0000 7f62693e0640 1 -- 172.21.15.37:0/2578578662 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640d0460 con 0x7f6264099fb0 2024-04-25T08:57:10.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f62693e0640 1 --2- 172.21.15.37:0/2578578662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6264099fb0 0x7f626415d230 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f625c009e50 tx=0x7f625c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac08d2043e4612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:10.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f62637fe640 1 -- 172.21.15.37:0/2578578662 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c0a5700 con 0x7f6264099fb0 2024-04-25T08:57:10.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f62637fe640 1 -- 172.21.15.37:0/2578578662 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c0a58a0 con 0x7f6264099fb0 2024-04-25T08:57:10.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f62637fe640 1 -- 172.21.15.37:0/2578578662 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c0a5c70 con 0x7f6264099fb0 2024-04-25T08:57:10.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f626ae43640 1 -- 172.21.15.37:0/2578578662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6264099fb0 msgr2=0x7f626415d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:10.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f626ae43640 1 --2- 172.21.15.37:0/2578578662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6264099fb0 0x7f626415d230 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f625c009e50 tx=0x7f625c01ca30 comp rx=0 tx=0).stop 2024-04-25T08:57:10.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f626ae43640 1 -- 172.21.15.37:0/2578578662 shutdown_connections 2024-04-25T08:57:10.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f626ae43640 1 --2- 172.21.15.37:0/2578578662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6264099fb0 0x7f626415d230 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f626ae43640 1 --2- 172.21.15.37:0/2578578662 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f626406f4a0 0x7f6264099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f626ae43640 1 -- 172.21.15.37:0/2578578662 >> 172.21.15.37:0/2578578662 conn(0x7f62640b30b0 msgr2=0x7f62640b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:10.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f626ae43640 1 -- 172.21.15.37:0/2578578662 shutdown_connections 2024-04-25T08:57:10.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f626ae43640 1 -- 172.21.15.37:0/2578578662 wait complete. 2024-04-25T08:57:10.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f626ae43640 1 Processor -- start 2024-04-25T08:57:10.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.461+0000 7f626ae43640 1 -- start start 2024-04-25T08:57:10.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f626ae43640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f626406f4a0 0x7f62640c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f626ae43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6264099fb0 0x7f626414c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f626ae43640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62640b0a30 0x7f6264160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f626ae43640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640cbfa0 con 0x7f6264099fb0 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f626ae43640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640cbe20 con 0x7f626406f4a0 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f626ae43640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62640cc120 con 0x7f62640b0a30 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f6263fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6264099fb0 0x7f626414c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f6263fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6264099fb0 0x7f626414c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40508/0 (socket says 172.21.15.37:40508) 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f6263fff640 1 -- 172.21.15.37:0/2565248770 learned_addr learned my addr 172.21.15.37:0/2565248770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f6268bdf640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f626406f4a0 0x7f62640c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f62693e0640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62640b0a30 0x7f6264160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f6263fff640 1 -- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62640b0a30 msgr2=0x7f6264160250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f6263fff640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62640b0a30 0x7f6264160250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f6263fff640 1 -- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f626406f4a0 msgr2=0x7f62640c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f6263fff640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f626406f4a0 0x7f62640c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f6263fff640 1 -- 172.21.15.37:0/2565248770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640c8c00 con 0x7f6264099fb0 2024-04-25T08:57:10.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f6263fff640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6264099fb0 0x7f626414c120 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f625400e930 tx=0x7f625400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:10.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f62617fa640 1 -- 172.21.15.37:0/2565248770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254017070 con 0x7f6264099fb0 2024-04-25T08:57:10.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f62617fa640 1 -- 172.21.15.37:0/2565248770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625400ca00 con 0x7f6264099fb0 2024-04-25T08:57:10.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f62617fa640 1 -- 172.21.15.37:0/2565248770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625400cdb0 con 0x7f6264099fb0 2024-04-25T08:57:10.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f626ae43640 1 -- 172.21.15.37:0/2565248770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62640c8ee0 con 0x7f6264099fb0 2024-04-25T08:57:10.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f626ae43640 1 -- 172.21.15.37:0/2565248770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62640c94a0 con 0x7f6264099fb0 2024-04-25T08:57:10.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f626ae43640 1 -- 172.21.15.37:0/2565248770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626404efa0 con 0x7f6264099fb0 2024-04-25T08:57:10.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f62617fa640 1 -- 172.21.15.37:0/2565248770 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f625401a070 con 0x7f6264099fb0 2024-04-25T08:57:10.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f62617fa640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6244066f60 0x7f6244069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f62617fa640 1 -- 172.21.15.37:0/2565248770 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6254085510 con 0x7f6264099fb0 2024-04-25T08:57:10.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.465+0000 7f6268bdf640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6244066f60 0x7f6244069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.469+0000 7f6268bdf640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6244066f60 0x7f6244069420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f624c001770 tx=0x7f624c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:10.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.469+0000 7f62617fa640 1 -- 172.21.15.37:0/2565248770 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6254055e00 con 0x7f6264099fb0 2024-04-25T08:57:10.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.665+0000 7f626ae43640 1 -- 172.21.15.37:0/2565248770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f62640b3ee0 con 0x7f6264099fb0 2024-04-25T08:57:10.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f62617fa640 1 -- 172.21.15.37:0/2565248770 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f6254059ea0 con 0x7f6264099fb0 2024-04-25T08:57:10.659 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:57:10.659 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:56:54.486633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6930},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6930":{"gid":6930,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/4130042812","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":4130042812},{"type":"v1","addr":"172.21.15.37:6835","nonce":4130042812}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:57:10.659 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 44 2024-04-25T08:57:10.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 -- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6244066f60 msgr2=0x7f6244069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:10.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6244066f60 0x7f6244069420 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f624c001770 tx=0x7f624c009040 comp rx=0 tx=0).stop 2024-04-25T08:57:10.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 -- 172.21.15.37:0/2565248770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6264099fb0 msgr2=0x7f626414c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:10.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6264099fb0 0x7f626414c120 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f625400e930 tx=0x7f625400edf0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 -- 172.21.15.37:0/2565248770 shutdown_connections 2024-04-25T08:57:10.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6244066f60 0x7f6244069420 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62640b0a30 0x7f6264160250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6264099fb0 0x7f626414c120 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 --2- 172.21.15.37:0/2565248770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f626406f4a0 0x7f62640c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 -- 172.21.15.37:0/2565248770 >> 172.21.15.37:0/2565248770 conn(0x7f62640b30b0 msgr2=0x7f62640cc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:10.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 -- 172.21.15.37:0/2565248770 shutdown_connections 2024-04-25T08:57:10.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.669+0000 7f626ae43640 1 -- 172.21.15.37:0/2565248770 wait complete. 2024-04-25T08:57:10.678 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T08:57:10.833 INFO:teuthology.orchestra.run.smithi037.stdout:{"message":"","return_code":0} 2024-04-25T08:57:10.833 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:57:10.834 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:57:10.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74d6700640 1 Processor -- start 2024-04-25T08:57:10.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74d6700640 1 -- start start 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74d6700640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f74d006ac90 0x7f74d006b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74d6700640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f74d00cc130 0x7f74d00cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74d6700640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d006b5b0 con 0x7f74d015ca90 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74d6700640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d015ff70 con 0x7f74d006ac90 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74d6700640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d015f360 con 0x7f74d00cc130 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74cffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f74d006ac90 0x7f74d006b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74cf7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f74d015ca90 0x7f74d00d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58180/0 (socket says 172.21.15.37:58180) 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74cf7fe640 1 -- 172.21.15.37:0/3552296493 learned_addr learned my addr 172.21.15.37:0/3552296493 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74d4c9d640 1 --2- 172.21.15.37:0/3552296493 >> v2:172.21.15.203:3300/0 conn(0x7f74d00cc130 0x7f74d00cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74cffff640 1 -- 172.21.15.37:0/3552296493 >> v2:172.21.15.203:3300/0 conn(0x7f74d00cc130 msgr2=0x7f74d00cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74cffff640 1 --2- 172.21.15.37:0/3552296493 >> v2:172.21.15.203:3300/0 conn(0x7f74d00cc130 0x7f74d00cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74cffff640 1 -- 172.21.15.37:0/3552296493 >> v1:172.21.15.37:6789/0 conn(0x7f74d015ca90 legacy=0x7f74d00d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74ceffd640 1 -- 172.21.15.37:0/3552296493 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148063876 0 0) 0x7f74d006b5b0 con 0x7f74d015ca90 2024-04-25T08:57:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.965+0000 7f74cffff640 1 -- 172.21.15.37:0/3552296493 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d015f640 con 0x7f74d006ac90 2024-04-25T08:57:10.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74cffff640 1 --2- 172.21.15.37:0/3552296493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d006ac90 0x7f74d006b070 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f74d006bd50 tx=0x7f74c801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f940e8bc43e2a1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:10.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74ceffd640 1 -- 172.21.15.37:0/3552296493 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c80b2070 con 0x7f74d006ac90 2024-04-25T08:57:10.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74ceffd640 1 -- 172.21.15.37:0/3552296493 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c80a59c0 con 0x7f74d006ac90 2024-04-25T08:57:10.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74ceffd640 1 -- 172.21.15.37:0/3552296493 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c80a5d90 con 0x7f74d006ac90 2024-04-25T08:57:10.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74d6700640 1 -- 172.21.15.37:0/3552296493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d006ac90 msgr2=0x7f74d006b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74d6700640 1 --2- 172.21.15.37:0/3552296493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d006ac90 0x7f74d006b070 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f74d006bd50 tx=0x7f74c801c900 comp rx=0 tx=0).stop 2024-04-25T08:57:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74d6700640 1 -- 172.21.15.37:0/3552296493 shutdown_connections 2024-04-25T08:57:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74d6700640 1 --2- 172.21.15.37:0/3552296493 >> v2:172.21.15.203:3300/0 conn(0x7f74d00cc130 0x7f74d00cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74d6700640 1 --2- 172.21.15.37:0/3552296493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d006ac90 0x7f74d006b070 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74d6700640 1 -- 172.21.15.37:0/3552296493 >> 172.21.15.37:0/3552296493 conn(0x7f74d0099a70 msgr2=0x7f74d00d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74d6700640 1 -- 172.21.15.37:0/3552296493 shutdown_connections 2024-04-25T08:57:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74d6700640 1 -- 172.21.15.37:0/3552296493 wait complete. 2024-04-25T08:57:10.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74d6700640 1 Processor -- start 2024-04-25T08:57:10.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.969+0000 7f74d6700640 1 -- start start 2024-04-25T08:57:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74d6700640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74d006ac90 0x7f74d0157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74d6700640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d00cc130 0x7f74d0158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74d6700640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74d015ca90 0x7f74d01fa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74d6700640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d0159750 con 0x7f74d00cc130 2024-04-25T08:57:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74d6700640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d01595d0 con 0x7f74d015ca90 2024-04-25T08:57:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74d6700640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d00d0040 con 0x7f74d006ac90 2024-04-25T08:57:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d00cc130 0x7f74d0158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d00cc130 0x7f74d0158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40522/0 (socket says 172.21.15.37:40522) 2024-04-25T08:57:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74cf7fe640 1 -- 172.21.15.37:0/2224506255 learned_addr learned my addr 172.21.15.37:0/2224506255 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74cffff640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74d006ac90 0x7f74d0157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74d4c9d640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74d015ca90 0x7f74d01fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74cf7fe640 1 -- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74d006ac90 msgr2=0x7f74d0157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74cf7fe640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74d006ac90 0x7f74d0157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74cf7fe640 1 -- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74d015ca90 msgr2=0x7f74d01fa050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74cf7fe640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74d015ca90 0x7f74d01fa050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74cf7fe640 1 -- 172.21.15.37:0/2224506255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d00c5da0 con 0x7f74d00cc130 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74d4c9d640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74d015ca90 0x7f74d01fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74cf7fe640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d00cc130 0x7f74d0158140 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c480 tx=0x7f74c400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74ccff9640 1 -- 172.21.15.37:0/2224506255 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4018070 con 0x7f74d00cc130 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74ccff9640 1 -- 172.21.15.37:0/2224506255 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c4007500 con 0x7f74d00cc130 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74d6700640 1 -- 172.21.15.37:0/2224506255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d01537d0 con 0x7f74d00cc130 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74ccff9640 1 -- 172.21.15.37:0/2224506255 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4004a30 con 0x7f74d00cc130 2024-04-25T08:57:10.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74d6700640 1 -- 172.21.15.37:0/2224506255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d0153d90 con 0x7f74d00cc130 2024-04-25T08:57:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74d6700640 1 -- 172.21.15.37:0/2224506255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d004efa0 con 0x7f74d00cc130 2024-04-25T08:57:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74ccff9640 1 -- 172.21.15.37:0/2224506255 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f74c401b070 con 0x7f74d00cc130 2024-04-25T08:57:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74ccff9640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f74bc066f60 0x7f74bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74ccff9640 1 -- 172.21.15.37:0/2224506255 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f74c4084500 con 0x7f74d00cc130 2024-04-25T08:57:10.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.973+0000 7f74cffff640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f74bc066f60 0x7f74bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:10.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.977+0000 7f74cffff640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f74bc066f60 0x7f74bc069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f74c80af000 tx=0x7f74c80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:10.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:10.977+0000 7f74ccff9640 1 -- 172.21.15.37:0/2224506255 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f74c4055410 con 0x7f74d00cc130 2024-04-25T08:57:11.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.177+0000 7f74d6700640 1 -- 172.21.15.37:0/2224506255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f74d015bd60 con 0x7f74d00cc130 2024-04-25T08:57:11.168 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T08:57:11.177+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:57:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74ccff9640 1 -- 172.21.15.37:0/2224506255 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7f74c4054d10 con 0x7f74d00cc130 2024-04-25T08:57:11.170 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:57:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 -- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f74bc066f60 msgr2=0x7f74bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f74bc066f60 0x7f74bc069420 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f74c80af000 tx=0x7f74c80a7070 comp rx=0 tx=0).stop 2024-04-25T08:57:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 -- 172.21.15.37:0/2224506255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d00cc130 msgr2=0x7f74d0158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d00cc130 0x7f74d0158140 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f74c400c480 tx=0x7f74c400c940 comp rx=0 tx=0).stop 2024-04-25T08:57:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 -- 172.21.15.37:0/2224506255 shutdown_connections 2024-04-25T08:57:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f74bc066f60 0x7f74bc069420 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f74d015ca90 0x7f74d01fa050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d00cc130 0x7f74d0158140 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 --2- 172.21.15.37:0/2224506255 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f74d006ac90 0x7f74d0157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 -- 172.21.15.37:0/2224506255 >> 172.21.15.37:0/2224506255 conn(0x7f74d0099a70 msgr2=0x7f74d006fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:11.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 -- 172.21.15.37:0/2224506255 shutdown_connections 2024-04-25T08:57:11.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.181+0000 7f74d6700640 1 -- 172.21.15.37:0/2224506255 wait complete. 2024-04-25T08:57:11.183 INFO:tasks.ceph.mds.c.smithi037.stderr: -3424> 2024-04-25T08:56:58.054+0000 7f3ab9a74640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:57:11.191 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T08:57:11.221 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T08:57:11.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 Processor -- start 2024-04-25T08:57:11.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- start start 2024-04-25T08:57:11.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c58989c3c0 0x55c58993dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c7a0 0x55c58991fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c589940b80 0x55c58993e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c589942a10 con 0x55c58989c7a0 2024-04-25T08:57:11.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5898129f0 con 0x55c589940b80 2024-04-25T08:57:11.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c589942b90 con 0x55c58989c3c0 2024-04-25T08:57:11.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccf213640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c7a0 0x55c58991fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccf213640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c7a0 0x55c58991fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40562/0 (socket says 172.21.15.37:40562) 2024-04-25T08:57:11.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccf213640 1 -- 172.21.15.37:0/2633794181 learned_addr learned my addr 172.21.15.37:0/2633794181 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:11.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccf213640 1 -- 172.21.15.37:0/2633794181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c58989c3c0 msgr2=0x55c58993dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:11.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd0215640 1 --2- 172.21.15.37:0/2633794181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c589940b80 0x55c58993e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccf213640 1 --2- 172.21.15.37:0/2633794181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c58989c3c0 0x55c58993dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccf213640 1 -- 172.21.15.37:0/2633794181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c589940b80 msgr2=0x55c58993e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:11.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccf213640 1 --2- 172.21.15.37:0/2633794181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c589940b80 0x55c58993e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccf213640 1 -- 172.21.15.37:0/2633794181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5899204a0 con 0x55c58989c7a0 2024-04-25T08:57:11.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd0215640 1 --2- 172.21.15.37:0/2633794181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c589940b80 0x55c58993e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:11.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccf213640 1 --2- 172.21.15.37:0/2633794181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c7a0 0x55c58991fd90 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400a750 tx=0x7f4cc400ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5c1c0e3efe050f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:11.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cce211640 1 -- 172.21.15.37:0/2633794181 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400ae60 con 0x55c58989c7a0 2024-04-25T08:57:11.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cce211640 1 -- 172.21.15.37:0/2633794181 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc4008c00 con 0x55c58989c7a0 2024-04-25T08:57:11.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cce211640 1 -- 172.21.15.37:0/2633794181 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc40133f0 con 0x55c58989c7a0 2024-04-25T08:57:11.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/2633794181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c7a0 msgr2=0x55c58991fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 --2- 172.21.15.37:0/2633794181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c7a0 0x55c58991fd90 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f4cc400a750 tx=0x7f4cc400ac10 comp rx=0 tx=0).stop 2024-04-25T08:57:11.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/2633794181 shutdown_connections 2024-04-25T08:57:11.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 --2- 172.21.15.37:0/2633794181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c589940b80 0x55c58993e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 --2- 172.21.15.37:0/2633794181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c7a0 0x55c58991fd90 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 --2- 172.21.15.37:0/2633794181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c58989c3c0 0x55c58993dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/2633794181 >> 172.21.15.37:0/2633794181 conn(0x55c589896ab0 msgr2=0x55c58986f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:11.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/2633794181 shutdown_connections 2024-04-25T08:57:11.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/2633794181 wait complete. 2024-04-25T08:57:11.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 Processor -- start 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- start start 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c3c0 0x55c5899d4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c58989c7a0 0x55c589945f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c589940b80 0x55c58993b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5899209e0 con 0x55c58989c3c0 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c589920740 con 0x55c58989c7a0 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4cd1ef6080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c589920c80 con 0x55c589940b80 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccfa14640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c3c0 0x55c5899d4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccfa14640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c3c0 0x55c5899d4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40576/0 (socket says 172.21.15.37:40576) 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.237+0000 7f4ccfa14640 1 -- 172.21.15.37:0/3045668936 learned_addr learned my addr 172.21.15.37:0/3045668936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4cd0215640 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c589940b80 0x55c58993b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccf213640 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c58989c7a0 0x55c589945f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccfa14640 1 -- 172.21.15.37:0/3045668936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c589940b80 msgr2=0x55c58993b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccfa14640 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c589940b80 0x55c58993b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccfa14640 1 -- 172.21.15.37:0/3045668936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c58989c7a0 msgr2=0x55c589945f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccfa14640 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c58989c7a0 0x55c589945f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccfa14640 1 -- 172.21.15.37:0/3045668936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5899d4b60 con 0x55c58989c3c0 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccfa14640 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c3c0 0x55c5899d4450 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000a500 tx=0x7f4cc0005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccd20f640 1 -- 172.21.15.37:0/3045668936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00af040 con 0x55c58989c3c0 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccd20f640 1 -- 172.21.15.37:0/3045668936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc00b5690 con 0x55c58989c3c0 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/3045668936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c5899d4e00 con 0x55c58989c3c0 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccd20f640 1 -- 172.21.15.37:0/3045668936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc00b5a80 con 0x55c58989c3c0 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/3045668936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5899d5340 con 0x55c58989c3c0 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccd20f640 1 -- 172.21.15.37:0/3045668936 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4cc00b5c20 con 0x55c58989c3c0 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccd20f640 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4cbc065940 0x7f4cbc067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccd20f640 1 -- 172.21.15.37:0/3045668936 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4cc011f7c0 con 0x55c58989c3c0 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4cd1ef6080 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55c58993bd40 0x55c5899d9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccf213640 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4cbc065940 0x7f4cbc067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/3045668936 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v9 -- 0x55c5899da350 con 0x55c58993bd40 2024-04-25T08:57:11.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4cd0215640 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55c58993bd40 0x55c5899d9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4cd0215640 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55c58993bd40 0x55c5899d9e10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:11.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.241+0000 7f4ccf213640 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4cbc065940 0x7f4cbc067e00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000c10 tx=0x7f4cc400c000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:11.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd0215640 1 -- 172.21.15.37:0/3045668936 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v83'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4cc8001670 con 0x55c58993bd40 2024-04-25T08:57:11.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/3045668936 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55c58993bd40 msgr2=0x55c5899d9e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd1ef6080 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55c58993bd40 0x55c5899d9e10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/3045668936 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4cbc065940 msgr2=0x7f4cbc067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd1ef6080 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4cbc065940 0x7f4cbc067e00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4000c10 tx=0x7f4cc400c000 comp rx=0 tx=0).stop 2024-04-25T08:57:11.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/3045668936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c3c0 msgr2=0x55c5899d4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd1ef6080 1 --2- 172.21.15.37:0/3045668936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c58989c3c0 0x55c5899d4450 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000a500 tx=0x7f4cc0005710 comp rx=0 tx=0).stop 2024-04-25T08:57:11.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd0215640 1 -- 172.21.15.37:0/3045668936 reap_dead start 2024-04-25T08:57:11.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/3045668936 shutdown_connections 2024-04-25T08:57:11.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/3045668936 >> 172.21.15.37:0/3045668936 conn(0x55c589896ab0 msgr2=0x55c58993e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:11.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/3045668936 shutdown_connections 2024-04-25T08:57:11.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.245+0000 7f4cd1ef6080 1 -- 172.21.15.37:0/3045668936 wait complete. 2024-04-25T08:57:11.241 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T08:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.417+0000 7f05421d3640 1 Processor -- start 2024-04-25T08:57:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.417+0000 7f05421d3640 1 -- start start 2024-04-25T08:57:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.417+0000 7f05421d3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c15d9c0 0x7f053c0d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.417+0000 7f05421d3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053c0d1950 con 0x7f053c0d3df0 2024-04-25T08:57:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f05421d3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053c0d1bf0 con 0x7f053c0cf270 2024-04-25T08:57:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f05421d3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053c0d2520 con 0x7f053c15d9c0 2024-04-25T08:57:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f053c0d3df0 0x7f053c06c590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58210/0 (socket says 172.21.15.37:58210) 2024-04-25T08:57:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053bfff640 1 -- 172.21.15.37:0/253465603 learned_addr learned my addr 172.21.15.37:0/253465603 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053b7fe640 1 --2- 172.21.15.37:0/253465603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c15d9c0 0x7f053c0d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220567790 0 0) 0x7f053c0d1950 con 0x7f053c0d3df0 2024-04-25T08:57:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0524003530 con 0x7f053c0d3df0 2024-04-25T08:57:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167473723 0 0) 0x7f053c0d1bf0 con 0x7f053c0cf270 2024-04-25T08:57:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f053c0d1950 con 0x7f053c0cf270 2024-04-25T08:57:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 377865726 0 0) 0x7f0524003530 con 0x7f053c0d3df0 2024-04-25T08:57:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c15d9c0 msgr2=0x7f053c0d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 --2- 172.21.15.37:0/253465603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c15d9c0 0x7f053c0d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 >> v1:172.21.15.203:6789/0 conn(0x7f053c0cf270 legacy=0x7f053c0d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c0d1ed0 con 0x7f053c0d3df0 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0534001980 con 0x7f053c0d3df0 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 >> v1:172.21.15.37:6789/0 conn(0x7f053c0d3df0 legacy=0x7f053c06c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 --2- 172.21.15.37:0/253465603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0524003bf0 0x7f0524004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 --2- 172.21.15.37:0/253465603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0524004810 0x7f0524004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 --2- 172.21.15.37:0/253465603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c15d9c0 0x7f0524005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c0d2520 con 0x7f0524003bf0 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052400bb90 con 0x7f0524004810 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052400bd90 con 0x7f053c15d9c0 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053b7fe640 1 --2- 172.21.15.37:0/253465603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0524003bf0 0x7f0524004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053affd640 1 --2- 172.21.15.37:0/253465603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0524004810 0x7f0524004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053b7fe640 1 -- 172.21.15.37:0/253465603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c15d9c0 msgr2=0x7f0524005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053b7fe640 1 --2- 172.21.15.37:0/253465603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c15d9c0 0x7f0524005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053b7fe640 1 -- 172.21.15.37:0/253465603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0524004810 msgr2=0x7f0524004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053b7fe640 1 --2- 172.21.15.37:0/253465603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0524004810 0x7f0524004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.421+0000 7f053b7fe640 1 -- 172.21.15.37:0/253465603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0524003530 con 0x7f0524003bf0 2024-04-25T08:57:11.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f053b7fe640 1 --2- 172.21.15.37:0/253465603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0524003bf0 0x7f0524004010 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f052c009ae0 tx=0x7f052c01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42a77622f5dd5a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:11.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0b2070 con 0x7f0524003bf0 2024-04-25T08:57:11.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c0a5560 con 0x7f0524003bf0 2024-04-25T08:57:11.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f053a7fc640 1 -- 172.21.15.37:0/253465603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0096b0 con 0x7f0524003bf0 2024-04-25T08:57:11.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 -- 172.21.15.37:0/253465603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0524003bf0 msgr2=0x7f0524004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 --2- 172.21.15.37:0/253465603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0524003bf0 0x7f0524004010 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f052c009ae0 tx=0x7f052c01ca80 comp rx=0 tx=0).stop 2024-04-25T08:57:11.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f053bfff640 1 -- 172.21.15.37:0/253465603 reap_dead start 2024-04-25T08:57:11.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 -- 172.21.15.37:0/253465603 shutdown_connections 2024-04-25T08:57:11.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 -- 172.21.15.37:0/253465603 >> 172.21.15.37:0/253465603 conn(0x7f053c09cad0 msgr2=0x7f053c08b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:11.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 -- 172.21.15.37:0/253465603 shutdown_connections 2024-04-25T08:57:11.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 -- 172.21.15.37:0/253465603 wait complete. 2024-04-25T08:57:11.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 Processor -- start 2024-04-25T08:57:11.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 -- start start 2024-04-25T08:57:11.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0524003bf0 0x7f053c14e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c09d440 0x7f053c09d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f053c0a0200 0x7f053c0a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c06d630 con 0x7f053c0a0200 2024-04-25T08:57:11.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c06d4b0 con 0x7f0524003bf0 2024-04-25T08:57:11.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f05421d3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053c06d7b0 con 0x7f053c09d440 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f053bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f053c0a0200 0x7f053c0a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f053bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f053c0a0200 0x7f053c0a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40596/0 (socket says 172.21.15.37:40596) 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f053bfff640 1 -- 172.21.15.37:0/137696997 learned_addr learned my addr 172.21.15.37:0/137696997 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f053b7fe640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0524003bf0 0x7f053c14e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.425+0000 7f053affd640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c09d440 0x7f053c09d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f053bfff640 1 -- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c09d440 msgr2=0x7f053c09d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f053bfff640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c09d440 0x7f053c09d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f053bfff640 1 -- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0524003bf0 msgr2=0x7f053c14e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f053bfff640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0524003bf0 0x7f053c14e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f053bfff640 1 -- 172.21.15.37:0/137696997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c14e770 con 0x7f053c0a0200 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f053affd640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c09d440 0x7f053c09d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f053bfff640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f053c0a0200 0x7f053c0a0680 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f053c0d12d0 tx=0x7f053400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f0523fff640 1 -- 172.21.15.37:0/137696997 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534004d80 con 0x7f053c0a0200 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f0523fff640 1 -- 172.21.15.37:0/137696997 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f053400a5e0 con 0x7f053c0a0200 2024-04-25T08:57:11.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f05421d3640 1 -- 172.21.15.37:0/137696997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053c0a1430 con 0x7f053c0a0200 2024-04-25T08:57:11.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f0523fff640 1 -- 172.21.15.37:0/137696997 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f053400a9d0 con 0x7f053c0a0200 2024-04-25T08:57:11.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f05421d3640 1 -- 172.21.15.37:0/137696997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053c15a720 con 0x7f053c0a0200 2024-04-25T08:57:11.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f05421d3640 1 -- 172.21.15.37:0/137696997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053c04efa0 con 0x7f053c0a0200 2024-04-25T08:57:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f0523fff640 1 -- 172.21.15.37:0/137696997 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f053400ab70 con 0x7f053c0a0200 2024-04-25T08:57:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f0523fff640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0528066f90 0x7f0528069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f0523fff640 1 -- 172.21.15.37:0/137696997 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0534084910 con 0x7f053c0a0200 2024-04-25T08:57:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f053b7fe640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0528066f90 0x7f0528069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.429+0000 7f053b7fe640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0528066f90 0x7f0528069450 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f052c0af000 tx=0x7f052c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:11.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.433+0000 7f0523fff640 1 -- 172.21.15.37:0/137696997 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f053c14e770 con 0x7f053c0a0200 2024-04-25T08:57:11.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:11.633+0000 7f05421d3640 1 -- 172.21.15.37:0/137696997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f053c0cb010 con 0x7f053c0a0200 2024-04-25T08:57:12.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.189+0000 7f0523fff640 1 -- 172.21.15.37:0/137696997 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v46) v1 ==== 162+0+0 (secure 0 0 0) 0x7f053400ae60 con 0x7f053c0a0200 2024-04-25T08:57:12.181 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T08:57:12.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 -- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0528066f90 msgr2=0x7f0528069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:12.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0528066f90 0x7f0528069450 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f052c0af000 tx=0x7f052c0046d0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 -- 172.21.15.37:0/137696997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f053c0a0200 msgr2=0x7f053c0a0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:12.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f053c0a0200 0x7f053c0a0680 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f053c0d12d0 tx=0x7f053400cbf0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 -- 172.21.15.37:0/137696997 shutdown_connections 2024-04-25T08:57:12.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0528066f90 0x7f0528069450 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f053c0a0200 0x7f053c0a0680 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f053c09d440 0x7f053c09d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 --2- 172.21.15.37:0/137696997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0524003bf0 0x7f053c14e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 -- 172.21.15.37:0/137696997 >> 172.21.15.37:0/137696997 conn(0x7f053c09cad0 msgr2=0x7f053c0b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:12.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 -- 172.21.15.37:0/137696997 shutdown_connections 2024-04-25T08:57:12.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.193+0000 7f05421d3640 1 -- 172.21.15.37:0/137696997 wait complete. 2024-04-25T08:57:12.209 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:12.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.337+0000 7fb41d313640 1 Processor -- start 2024-04-25T08:57:12.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.337+0000 7fb41d313640 1 -- start start 2024-04-25T08:57:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb41d313640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb418144ca0 0x7fb41806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb41d313640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb41806ec60 0x7fb418142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb41d313640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4180ccc70 con 0x7fb4180ce510 2024-04-25T08:57:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb41d313640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4180cc2b0 con 0x7fb418144ca0 2024-04-25T08:57:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb41d313640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41806e290 con 0x7fb41806ec60 2024-04-25T08:57:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb416ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb4180ce510 0x7fb4180ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58224/0 (socket says 172.21.15.37:58224) 2024-04-25T08:57:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb416ffd640 1 -- 172.21.15.37:0/371558343 learned_addr learned my addr 172.21.15.37:0/371558343 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367944860 0 0) 0x7fb4180ccc70 con 0x7fb4180ce510 2024-04-25T08:57:12.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb4177fe640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb41806ec60 0x7fb418142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb400003530 con 0x7fb4180ce510 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb4167fc640 1 --2- 172.21.15.37:0/371558343 >> v2:172.21.15.203:3300/0 conn(0x7fb418144ca0 0x7fb41806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3606790000 0 0) 0x7fb400003530 con 0x7fb4180ce510 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb41806ec60 msgr2=0x7fb418142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb41806ec60 0x7fb418142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 >> v2:172.21.15.203:3300/0 conn(0x7fb418144ca0 msgr2=0x7fb41806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 --2- 172.21.15.37:0/371558343 >> v2:172.21.15.203:3300/0 conn(0x7fb418144ca0 0x7fb41806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41806dc40 con 0x7fb4180ce510 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fb408001680 con 0x7fb4180ce510 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb4167fc640 1 --2- 172.21.15.37:0/371558343 >> v2:172.21.15.203:3300/0 conn(0x7fb418144ca0 0x7fb41806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 >> v1:172.21.15.37:6789/0 conn(0x7fb4180ce510 legacy=0x7fb4180ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb400003a30 0x7fb400003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb400004810 0x7fb400004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:12.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4000051d0 0x7fb40000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180cc2b0 con 0x7fb400003a30 2024-04-25T08:57:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41806e290 con 0x7fb4000051d0 2024-04-25T08:57:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40000c050 con 0x7fb400004810 2024-04-25T08:57:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb4177fe640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb41806ec60 0x7fb418142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb416ffd640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb400003a30 0x7fb400003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb4167fc640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb400004810 0x7fb400004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb416ffd640 1 -- 172.21.15.37:0/371558343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb400004810 msgr2=0x7fb400004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb416ffd640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb400004810 0x7fb400004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb416ffd640 1 -- 172.21.15.37:0/371558343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4000051d0 msgr2=0x7fb40000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb416ffd640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4000051d0 0x7fb40000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.341+0000 7fb416ffd640 1 -- 172.21.15.37:0/371558343 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400003530 con 0x7fb400003a30 2024-04-25T08:57:12.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb416ffd640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb400003a30 0x7fb400003e50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb40800c240 tx=0x7fb40801f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d945a1936fee0ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:12.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408007d60 con 0x7fb400003a30 2024-04-25T08:57:12.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb408005b00 con 0x7fb400003a30 2024-04-25T08:57:12.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb415ffb640 1 -- 172.21.15.37:0/371558343 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408007070 con 0x7fb400003a30 2024-04-25T08:57:12.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 -- 172.21.15.37:0/371558343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb400003a30 msgr2=0x7fb400003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:12.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 --2- 172.21.15.37:0/371558343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb400003a30 0x7fb400003e50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb40800c240 tx=0x7fb40801f430 comp rx=0 tx=0).stop 2024-04-25T08:57:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb4177fe640 1 -- 172.21.15.37:0/371558343 reap_dead start 2024-04-25T08:57:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 -- 172.21.15.37:0/371558343 shutdown_connections 2024-04-25T08:57:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 -- 172.21.15.37:0/371558343 >> 172.21.15.37:0/371558343 conn(0x7fb4180b11a0 msgr2=0x7fb4180b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 -- 172.21.15.37:0/371558343 shutdown_connections 2024-04-25T08:57:12.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 -- 172.21.15.37:0/371558343 wait complete. 2024-04-25T08:57:12.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 Processor -- start 2024-04-25T08:57:12.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 -- start start 2024-04-25T08:57:12.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb400003a30 0x7fb41813dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:12.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4181e62f0 0x7fb4181f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:12.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4181e6770 0x7fb4181f4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:12.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4181451f0 con 0x7fb4181e6770 2024-04-25T08:57:12.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4180cc900 con 0x7fb400003a30 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb41d313640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb418145370 con 0x7fb4181e62f0 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb4177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4181e6770 0x7fb4181f4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb4177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4181e6770 0x7fb4181f4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40608/0 (socket says 172.21.15.37:40608) 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb4177fe640 1 -- 172.21.15.37:0/402718499 learned_addr learned my addr 172.21.15.37:0/402718499 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb416ffd640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb400003a30 0x7fb41813dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb4167fc640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4181e62f0 0x7fb4181f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb4177fe640 1 -- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4181e62f0 msgr2=0x7fb4181f4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb4177fe640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4181e62f0 0x7fb4181f4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb4177fe640 1 -- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb400003a30 msgr2=0x7fb41813dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb4177fe640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb400003a30 0x7fb41813dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.345+0000 7fb4177fe640 1 -- 172.21.15.37:0/402718499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4181e1100 con 0x7fb4181e6770 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb4177fe640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4181e6770 0x7fb4181f4cb0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb41804f950 tx=0x7fb41000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb3ff7fe640 1 -- 172.21.15.37:0/402718499 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41001a070 con 0x7fb4181e6770 2024-04-25T08:57:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb3ff7fe640 1 -- 172.21.15.37:0/402718499 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4100039a0 con 0x7fb4181e6770 2024-04-25T08:57:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb41d313640 1 -- 172.21.15.37:0/402718499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4181e13e0 con 0x7fb4181e6770 2024-04-25T08:57:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb3ff7fe640 1 -- 172.21.15.37:0/402718499 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb410003d90 con 0x7fb4181e6770 2024-04-25T08:57:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb41d313640 1 -- 172.21.15.37:0/402718499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4181e19a0 con 0x7fb4181e6770 2024-04-25T08:57:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb41d313640 1 -- 172.21.15.37:0/402718499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41804efa0 con 0x7fb4181e6770 2024-04-25T08:57:12.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb3ff7fe640 1 -- 172.21.15.37:0/402718499 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb41001d070 con 0x7fb4181e6770 2024-04-25T08:57:12.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb3ff7fe640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb404066f60 0x7fb404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:12.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb3ff7fe640 1 -- 172.21.15.37:0/402718499 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb410085160 con 0x7fb4181e6770 2024-04-25T08:57:12.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb416ffd640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb404066f60 0x7fb404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:12.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.349+0000 7fb416ffd640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb404066f60 0x7fb404069420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb4080ae000 tx=0x7fb408001ac0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.353+0000 7fb3ff7fe640 1 -- 172.21.15.37:0/402718499 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb410010070 con 0x7fb4181e6770 2024-04-25T08:57:12.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.545+0000 7fb41d313640 1 -- 172.21.15.37:0/402718499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb41806e8c0 con 0x7fb4181e6770 2024-04-25T08:57:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.545+0000 7fb3ff7fe640 1 -- 172.21.15.37:0/402718499 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fb410055ef0 con 0x7fb4181e6770 2024-04-25T08:57:12.537 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:57:12.537 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:57:12.198131+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"c","rank":0,"incarnation":47,"state":"up:replay","state_seq":1,"addr":"172.21.15.37:6835/3927528604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3927528604},{"type":"v1","addr":"172.21.15.37:6835","nonce":3927528604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:57:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 47 2024-04-25T08:57:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 -- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb404066f60 msgr2=0x7fb404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb404066f60 0x7fb404069420 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb4080ae000 tx=0x7fb408001ac0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 -- 172.21.15.37:0/402718499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4181e6770 msgr2=0x7fb4181f4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4181e6770 0x7fb4181f4cb0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fb41804f950 tx=0x7fb41000d4a0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 -- 172.21.15.37:0/402718499 shutdown_connections 2024-04-25T08:57:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb404066f60 0x7fb404069420 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4181e6770 0x7fb4181f4cb0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4181e62f0 0x7fb4181f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 --2- 172.21.15.37:0/402718499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb400003a30 0x7fb41813dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 -- 172.21.15.37:0/402718499 >> 172.21.15.37:0/402718499 conn(0x7fb4180b11a0 msgr2=0x7fb4180a8640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 -- 172.21.15.37:0/402718499 shutdown_connections 2024-04-25T08:57:12.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:12.549+0000 7fb41d313640 1 -- 172.21.15.37:0/402718499 wait complete. 2024-04-25T08:57:12.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:56:51.447805+0000', 'modified': '2024-04-25T08:57:12.198131+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7200}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7200': {'gid': 7200, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.37:6835/3927528604', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 3927528604}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 3927528604}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:57:12.958 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7200:up:replay 2024-04-25T08:57:13.959 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:14.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb38798e640 1 Processor -- start 2024-04-25T08:57:14.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb38798e640 1 -- start start 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb38798e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb38015da80 0x7fb3800d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb38798e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb3800d1cd0 0x7fb3800d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb38798e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb3800d0620 0x7fb3800ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb38798e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38006c440 con 0x7fb3800d1cd0 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb38798e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38006be60 con 0x7fb3800d0620 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb38798e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38015de60 con 0x7fb38015da80 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb384f29640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb3800d1cd0 0x7fb3800d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb384f29640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb3800d1cd0 0x7fb3800d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40612/0 (socket says 172.21.15.37:40612) 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb384f29640 1 -- 172.21.15.37:0/4233560953 learned_addr learned my addr 172.21.15.37:0/4233560953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb385f2b640 1 --2- 172.21.15.37:0/4233560953 >> v2:172.21.15.203:3300/0 conn(0x7fb3800d0620 0x7fb3800ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb38572a640 1 --2- 172.21.15.37:0/4233560953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb38015da80 0x7fb3800d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:14.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb384f29640 1 -- 172.21.15.37:0/4233560953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb38015da80 msgr2=0x7fb3800d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:14.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb384f29640 1 --2- 172.21.15.37:0/4233560953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb38015da80 0x7fb3800d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb384f29640 1 -- 172.21.15.37:0/4233560953 >> v2:172.21.15.203:3300/0 conn(0x7fb3800d0620 msgr2=0x7fb3800ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:14.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb384f29640 1 --2- 172.21.15.37:0/4233560953 >> v2:172.21.15.203:3300/0 conn(0x7fb3800d0620 0x7fb3800ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.105+0000 7fb384f29640 1 -- 172.21.15.37:0/4233560953 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3800d2450 con 0x7fb3800d1cd0 2024-04-25T08:57:14.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb384f29640 1 --2- 172.21.15.37:0/4233560953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3800d1cd0 0x7fb3800d00e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb38006de60 tx=0x7fb37c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb656abfb0f43afa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:14.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb377fff640 1 -- 172.21.15.37:0/4233560953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0ab5f0 con 0x7fb3800d1cd0 2024-04-25T08:57:14.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb377fff640 1 -- 172.21.15.37:0/4233560953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c0b0070 con 0x7fb3800d1cd0 2024-04-25T08:57:14.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb377fff640 1 -- 172.21.15.37:0/4233560953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c0ab9b0 con 0x7fb3800d1cd0 2024-04-25T08:57:14.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb38798e640 1 -- 172.21.15.37:0/4233560953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3800d1cd0 msgr2=0x7fb3800d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:14.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb38798e640 1 --2- 172.21.15.37:0/4233560953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3800d1cd0 0x7fb3800d00e0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fb38006de60 tx=0x7fb37c01c520 comp rx=0 tx=0).stop 2024-04-25T08:57:14.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb38798e640 1 -- 172.21.15.37:0/4233560953 shutdown_connections 2024-04-25T08:57:14.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb38798e640 1 --2- 172.21.15.37:0/4233560953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3800d1cd0 0x7fb3800d00e0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb38798e640 1 --2- 172.21.15.37:0/4233560953 >> v2:172.21.15.203:3300/0 conn(0x7fb3800d0620 0x7fb3800ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb38798e640 1 --2- 172.21.15.37:0/4233560953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb38015da80 0x7fb3800d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb38798e640 1 -- 172.21.15.37:0/4233560953 >> 172.21.15.37:0/4233560953 conn(0x7fb38008b000 msgr2=0x7fb3800d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:14.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb38798e640 1 -- 172.21.15.37:0/4233560953 shutdown_connections 2024-04-25T08:57:14.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb38798e640 1 -- 172.21.15.37:0/4233560953 wait complete. 2024-04-25T08:57:14.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb38798e640 1 Processor -- start 2024-04-25T08:57:14.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.109+0000 7fb38798e640 1 -- start start 2024-04-25T08:57:14.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb38798e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3800d0620 0x7fb380159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:14.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb38798e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3800d1cd0 0x7fb3800cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb38798e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38015da80 0x7fb38014b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb38798e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38006ca30 con 0x7fb38015da80 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb38798e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38006c8b0 con 0x7fb3800d1cd0 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb38798e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38006cbb0 con 0x7fb3800d0620 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb385f2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38015da80 0x7fb38014b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb385f2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38015da80 0x7fb38014b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40616/0 (socket says 172.21.15.37:40616) 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb385f2b640 1 -- 172.21.15.37:0/1887711881 learned_addr learned my addr 172.21.15.37:0/1887711881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb38572a640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3800d0620 0x7fb380159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb384f29640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3800d1cd0 0x7fb3800cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb385f2b640 1 -- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3800d0620 msgr2=0x7fb380159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb385f2b640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3800d0620 0x7fb380159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb385f2b640 1 -- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3800d1cd0 msgr2=0x7fb3800cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:14.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb385f2b640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3800d1cd0 0x7fb3800cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb385f2b640 1 -- 172.21.15.37:0/1887711881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb380068520 con 0x7fb38015da80 2024-04-25T08:57:14.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb385f2b640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38015da80 0x7fb38014b9d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb378007e90 tx=0x7fb37800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:14.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb375ffb640 1 -- 172.21.15.37:0/1887711881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378014420 con 0x7fb38015da80 2024-04-25T08:57:14.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb375ffb640 1 -- 172.21.15.37:0/1887711881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb378019070 con 0x7fb38015da80 2024-04-25T08:57:14.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb38798e640 1 -- 172.21.15.37:0/1887711881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38015a220 con 0x7fb38015da80 2024-04-25T08:57:14.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb375ffb640 1 -- 172.21.15.37:0/1887711881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb378014820 con 0x7fb38015da80 2024-04-25T08:57:14.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb38798e640 1 -- 172.21.15.37:0/1887711881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38015a7e0 con 0x7fb38015da80 2024-04-25T08:57:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb38798e640 1 -- 172.21.15.37:0/1887711881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38004efa0 con 0x7fb38015da80 2024-04-25T08:57:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb375ffb640 1 -- 172.21.15.37:0/1887711881 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3780149c0 con 0x7fb38015da80 2024-04-25T08:57:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb375ffb640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb364066f60 0x7fb364069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb375ffb640 1 -- 172.21.15.37:0/1887711881 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb378085ae0 con 0x7fb38015da80 2024-04-25T08:57:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.113+0000 7fb38572a640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb364066f60 0x7fb364069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:14.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.117+0000 7fb38572a640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb364066f60 0x7fb364069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb37000b930 tx=0x7fb370008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:14.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.117+0000 7fb375ffb640 1 -- 172.21.15.37:0/1887711881 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb378056910 con 0x7fb38015da80 2024-04-25T08:57:14.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.309+0000 7fb38798e640 1 -- 172.21.15.37:0/1887711881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3800b03f0 con 0x7fb38015da80 2024-04-25T08:57:14.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb375ffb640 1 -- 172.21.15.37:0/1887711881 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fb378056210 con 0x7fb38015da80 2024-04-25T08:57:14.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:57:14.303 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":49,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:57:13.208255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"c","rank":0,"incarnation":47,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.37:6835/3927528604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3927528604},{"type":"v1","addr":"172.21.15.37:6835","nonce":3927528604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:57:14.303 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 49 2024-04-25T08:57:14.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 -- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb364066f60 msgr2=0x7fb364069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:14.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb364066f60 0x7fb364069420 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb37000b930 tx=0x7fb370008040 comp rx=0 tx=0).stop 2024-04-25T08:57:14.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 -- 172.21.15.37:0/1887711881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38015da80 msgr2=0x7fb38014b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:14.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38015da80 0x7fb38014b9d0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb378007e90 tx=0x7fb37800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 -- 172.21.15.37:0/1887711881 shutdown_connections 2024-04-25T08:57:14.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb364066f60 0x7fb364069420 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb38015da80 0x7fb38014b9d0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3800d1cd0 0x7fb3800cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 --2- 172.21.15.37:0/1887711881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3800d0620 0x7fb380159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:14.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 -- 172.21.15.37:0/1887711881 >> 172.21.15.37:0/1887711881 conn(0x7fb38008b000 msgr2=0x7fb380160070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:14.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 -- 172.21.15.37:0/1887711881 shutdown_connections 2024-04-25T08:57:14.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:14.313+0000 7fb38798e640 1 -- 172.21.15.37:0/1887711881 wait complete. 2024-04-25T08:57:14.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:56:51.447805+0000', 'modified': '2024-04-25T08:57:13.208255+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7200}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7200': {'gid': 7200, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.37:6835/3927528604', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 3927528604}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 3927528604}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:57:14.724 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7200:up:rejoin 2024-04-25T08:57:15.725 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:15.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.869+0000 7fc5139fe640 1 Processor -- start 2024-04-25T08:57:15.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.869+0000 7fc5139fe640 1 -- start start 2024-04-25T08:57:15.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.869+0000 7fc5139fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc50c066e30 0x7fc50c06c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:15.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.869+0000 7fc5139fe640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50c06ec30 con 0x7fc50c06bca0 2024-04-25T08:57:15.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.869+0000 7fc5139fe640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc50c06c5c0 con 0x7fc50c0af370 2024-04-25T08:57:15.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.869+0000 7fc5139fe640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc50c0669a0 con 0x7fc50c066e30 2024-04-25T08:57:15.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.869+0000 7fc510f99640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc50c066e30 0x7fc50c06c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:15.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.869+0000 7fc51179a640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc50c06bca0 0x7fc50c0cd0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58238/0 (socket says 172.21.15.37:58238) 2024-04-25T08:57:15.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.869+0000 7fc51179a640 1 -- 172.21.15.37:0/2969301587 learned_addr learned my addr 172.21.15.37:0/2969301587 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.869+0000 7fc510f99640 1 -- 172.21.15.37:0/2969301587 >> v1:172.21.15.203:6789/0 conn(0x7fc50c0af370 legacy=0x7fc50c0ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc510f99640 1 -- 172.21.15.37:0/2969301587 >> v1:172.21.15.37:6789/0 conn(0x7fc50c06bca0 legacy=0x7fc50c0cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc503fff640 1 -- 172.21.15.37:0/2969301587 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654040814 0 0) 0x7fc50c06ec30 con 0x7fc50c06bca0 2024-04-25T08:57:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc510f99640 1 -- 172.21.15.37:0/2969301587 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c06f230 con 0x7fc50c066e30 2024-04-25T08:57:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc510f99640 1 --2- 172.21.15.37:0/2969301587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c066e30 0x7fc50c06c080 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0097c0 tx=0x7fc4fc01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad256dd4b4083bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc503fff640 1 -- 172.21.15.37:0/2969301587 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0b0070 con 0x7fc50c066e30 2024-04-25T08:57:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc503fff640 1 -- 172.21.15.37:0/2969301587 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc0ab600 con 0x7fc50c066e30 2024-04-25T08:57:15.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc503fff640 1 -- 172.21.15.37:0/2969301587 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0ab9d0 con 0x7fc50c066e30 2024-04-25T08:57:15.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc5139fe640 1 -- 172.21.15.37:0/2969301587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c066e30 msgr2=0x7fc50c06c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:15.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc5139fe640 1 --2- 172.21.15.37:0/2969301587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c066e30 0x7fc50c06c080 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc0097c0 tx=0x7fc4fc01c310 comp rx=0 tx=0).stop 2024-04-25T08:57:15.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc5139fe640 1 -- 172.21.15.37:0/2969301587 shutdown_connections 2024-04-25T08:57:15.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc5139fe640 1 --2- 172.21.15.37:0/2969301587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c066e30 0x7fc50c06c080 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:15.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc5139fe640 1 -- 172.21.15.37:0/2969301587 >> 172.21.15.37:0/2969301587 conn(0x7fc50c071810 msgr2=0x7fc50c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:15.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc5139fe640 1 -- 172.21.15.37:0/2969301587 shutdown_connections 2024-04-25T08:57:15.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.873+0000 7fc5139fe640 1 -- 172.21.15.37:0/2969301587 wait complete. 2024-04-25T08:57:15.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc5139fe640 1 Processor -- start 2024-04-25T08:57:15.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc5139fe640 1 -- start start 2024-04-25T08:57:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc5139fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc50c066e30 0x7fc50c14abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc5139fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc50c06bca0 0x7fc50c14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc5139fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c0af370 0x7fc50c156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc5139fe640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c1697c0 con 0x7fc50c0af370 2024-04-25T08:57:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc5139fe640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c169640 con 0x7fc50c066e30 2024-04-25T08:57:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc5139fe640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc50c169940 con 0x7fc50c06bca0 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc511f9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c0af370 0x7fc50c156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc511f9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c0af370 0x7fc50c156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40634/0 (socket says 172.21.15.37:40634) 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc511f9b640 1 -- 172.21.15.37:0/476499717 learned_addr learned my addr 172.21.15.37:0/476499717 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc510f99640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc50c06bca0 0x7fc50c14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc51179a640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc50c066e30 0x7fc50c14abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc511f9b640 1 -- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc50c06bca0 msgr2=0x7fc50c14fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc511f9b640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc50c06bca0 0x7fc50c14fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc511f9b640 1 -- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc50c066e30 msgr2=0x7fc50c14abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc511f9b640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc50c066e30 0x7fc50c14abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc511f9b640 1 -- 172.21.15.37:0/476499717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc50c14ece0 con 0x7fc50c0af370 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc511f9b640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c0af370 0x7fc50c156e80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc50400c3c0 tx=0x7fc50400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc501ffb640 1 -- 172.21.15.37:0/476499717 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504011790 con 0x7fc50c0af370 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc501ffb640 1 -- 172.21.15.37:0/476499717 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc504017070 con 0x7fc50c0af370 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc5139fe640 1 -- 172.21.15.37:0/476499717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc50c14efc0 con 0x7fc50c0af370 2024-04-25T08:57:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc501ffb640 1 -- 172.21.15.37:0/476499717 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504011b70 con 0x7fc50c0af370 2024-04-25T08:57:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc5139fe640 1 -- 172.21.15.37:0/476499717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50c14f580 con 0x7fc50c0af370 2024-04-25T08:57:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc5139fe640 1 -- 172.21.15.37:0/476499717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc50c04efa0 con 0x7fc50c0af370 2024-04-25T08:57:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.877+0000 7fc501ffb640 1 -- 172.21.15.37:0/476499717 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc504011d10 con 0x7fc50c0af370 2024-04-25T08:57:15.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.881+0000 7fc501ffb640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4f0066f90 0x7fc4f0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:15.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.881+0000 7fc501ffb640 1 -- 172.21.15.37:0/476499717 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc504084f20 con 0x7fc50c0af370 2024-04-25T08:57:15.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.881+0000 7fc51179a640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4f0066f90 0x7fc4f0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:15.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.881+0000 7fc51179a640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4f0066f90 0x7fc4f0069450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc508007fd0 tx=0x7fc508007ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:15.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:15.885+0000 7fc501ffb640 1 -- 172.21.15.37:0/476499717 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc504055d50 con 0x7fc50c0af370 2024-04-25T08:57:16.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 -- 172.21.15.37:0/476499717 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc50c072910 con 0x7fc50c0af370 2024-04-25T08:57:16.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc501ffb640 1 -- 172.21.15.37:0/476499717 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fc504055650 con 0x7fc50c0af370 2024-04-25T08:57:16.068 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:57:16.068 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":50,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":50,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:57:14.218022+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/3927528604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3927528604},{"type":"v1","addr":"172.21.15.37:6835","nonce":3927528604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:57:16.068 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 50 2024-04-25T08:57:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 -- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4f0066f90 msgr2=0x7fc4f0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4f0066f90 0x7fc4f0069450 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc508007fd0 tx=0x7fc508007ee0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 -- 172.21.15.37:0/476499717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c0af370 msgr2=0x7fc50c156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c0af370 0x7fc50c156e80 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc50400c3c0 tx=0x7fc50400c880 comp rx=0 tx=0).stop 2024-04-25T08:57:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 -- 172.21.15.37:0/476499717 shutdown_connections 2024-04-25T08:57:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4f0066f90 0x7fc4f0069450 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc50c0af370 0x7fc50c156e80 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc50c06bca0 0x7fc50c14fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 --2- 172.21.15.37:0/476499717 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc50c066e30 0x7fc50c14abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 -- 172.21.15.37:0/476499717 >> 172.21.15.37:0/476499717 conn(0x7fc50c071810 msgr2=0x7fc50c154050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:16.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 -- 172.21.15.37:0/476499717 shutdown_connections 2024-04-25T08:57:16.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.077+0000 7fc5139fe640 1 -- 172.21.15.37:0/476499717 wait complete. 2024-04-25T08:57:16.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 50, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:56:51.447805+0000', 'modified': '2024-04-25T08:57:14.218022+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 83, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7200}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7200': {'gid': 7200, 'name': 'c', 'rank': 0, 'incarnation': 47, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.37:6835/3927528604', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 3927528604}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 3927528604}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:57:16.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:57:16.488 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:57:16.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.633+0000 7f0ce4d6e640 1 Processor -- start 2024-04-25T08:57:16.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.633+0000 7f0ce4d6e640 1 -- start start 2024-04-25T08:57:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.633+0000 7f0ce4d6e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce0144ca0 0x7f0ce006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.633+0000 7f0ce4d6e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce00ccc70 con 0x7f0ce006ec60 2024-04-25T08:57:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.633+0000 7f0ce4d6e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce006dc40 con 0x7f0ce00ce510 2024-04-25T08:57:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.633+0000 7f0ce4d6e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00cc2b0 con 0x7f0ce0144ca0 2024-04-25T08:57:16.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.633+0000 7f0cded9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0ce006ec60 0x7f0ce0142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58242/0 (socket says 172.21.15.37:58242) 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.633+0000 7f0cded9d640 1 -- 172.21.15.37:0/1452298688 learned_addr learned my addr 172.21.15.37:0/1452298688 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.633+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214414815 0 0) 0x7f0ce00ccc70 con 0x7f0ce006ec60 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.633+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce0144ca0 0x7f0ce006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.633+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ccc003530 con 0x7f0ce006ec60 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2860671553 0 0) 0x7f0ccc003530 con 0x7f0ce006ec60 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce0144ca0 msgr2=0x7f0ce006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce0144ca0 0x7f0ce006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 >> v1:172.21.15.203:6789/0 conn(0x7f0ce00ce510 legacy=0x7f0ce00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce006df20 con 0x7f0ce006ec60 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0cd8001a00 con 0x7f0ce006ec60 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 >> v1:172.21.15.37:6789/0 conn(0x7f0ce006ec60 legacy=0x7f0ce0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ccc003a60 0x7f0ccc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce0144ca0 0x7f0ccc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ccc004d50 0x7f0ccc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00cc2b0 con 0x7f0ccc004d50 2024-04-25T08:57:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc00bb70 con 0x7f0ccc003a60 2024-04-25T08:57:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ccc00bd70 con 0x7f0ce0144ca0 2024-04-25T08:57:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cded9d640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ccc004d50 0x7f0ccc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cde59c640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ccc003a60 0x7f0ccc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cded9d640 1 -- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce0144ca0 msgr2=0x7f0ccc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce0144ca0 0x7f0ccc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cded9d640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce0144ca0 0x7f0ccc004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cded9d640 1 -- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ccc003a60 msgr2=0x7f0ccc003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cded9d640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ccc003a60 0x7f0ccc003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cded9d640 1 -- 172.21.15.37:0/1452298688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ccc003530 con 0x7f0ccc004d50 2024-04-25T08:57:16.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cde59c640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ccc003a60 0x7f0ccc003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:16.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cded9d640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ccc004d50 0x7f0ccc0051d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8002240 tx=0x7f0cd801d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bbd7d763bfaa855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800a4a0 con 0x7f0ccc004d50 2024-04-25T08:57:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd80ad650 con 0x7f0ccc004d50 2024-04-25T08:57:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cddd9b640 1 -- 172.21.15.37:0/1452298688 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80ada40 con 0x7f0ccc004d50 2024-04-25T08:57:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/1452298688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ccc004d50 msgr2=0x7f0ccc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0ce4d6e640 1 --2- 172.21.15.37:0/1452298688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ccc004d50 0x7f0ccc0051d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f0cd8002240 tx=0x7f0cd801d300 comp rx=0 tx=0).stop 2024-04-25T08:57:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0cded9d640 1 -- 172.21.15.37:0/1452298688 reap_dead start 2024-04-25T08:57:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/1452298688 shutdown_connections 2024-04-25T08:57:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/1452298688 >> 172.21.15.37:0/1452298688 conn(0x7f0ce00b11a0 msgr2=0x7f0ce00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/1452298688 shutdown_connections 2024-04-25T08:57:16.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.637+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/1452298688 wait complete. 2024-04-25T08:57:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0ce4d6e640 1 Processor -- start 2024-04-25T08:57:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0ce4d6e640 1 -- start start 2024-04-25T08:57:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0ce4d6e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ccc004d50 0x7f0ce00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0ce4d6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ce00ae130 0x7f0ce01e1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0ce4d6e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce013e8a0 0x7f0ce013ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0ce4d6e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0142d90 con 0x7f0ce00ae130 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0ce4d6e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce006e290 con 0x7f0ccc004d50 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0ce4d6e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0142f10 con 0x7f0ce013e8a0 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ce00ae130 0x7f0ce01e1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ce00ae130 0x7f0ce01e1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40652/0 (socket says 172.21.15.37:40652) 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/3871586762 learned_addr learned my addr 172.21.15.37:0/3871586762 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cded9d640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce013e8a0 0x7f0ce013ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cde59c640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ccc004d50 0x7f0ce00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce013e8a0 msgr2=0x7f0ce013ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce013e8a0 0x7f0ce013ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ccc004d50 msgr2=0x7f0ce00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ccc004d50 0x7f0ce00ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/3871586762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce01f3460 con 0x7f0ce00ae130 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cded9d640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce013e8a0 0x7f0ce013ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:16.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ce00ae130 0x7f0ce01e1030 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8010840 tx=0x7f0cc8010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd77fe640 1 -- 172.21.15.37:0/3871586762 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc800f040 con 0x7f0ce00ae130 2024-04-25T08:57:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd77fe640 1 -- 172.21.15.37:0/3871586762 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc800eb80 con 0x7f0ce00ae130 2024-04-25T08:57:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/3871586762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce01e1570 con 0x7f0ce00ae130 2024-04-25T08:57:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0cd77fe640 1 -- 172.21.15.37:0/3871586762 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc801c400 con 0x7f0ce00ae130 2024-04-25T08:57:16.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.641+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/3871586762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce00acfe0 con 0x7f0ce00ae130 2024-04-25T08:57:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd77fe640 1 -- 172.21.15.37:0/3871586762 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0cc80253f0 con 0x7f0ce00ae130 2024-04-25T08:57:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd77fe640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0cc0066f60 0x7f0cc0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd77fe640 1 -- 172.21.15.37:0/3871586762 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0cc8085910 con 0x7f0ce00ae130 2024-04-25T08:57:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cde59c640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0cc0066f60 0x7f0cc0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0ce7edb000 1 Processor -- start 2024-04-25T08:57:16.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0ce7edb000 1 -- start start 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cde59c640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0cc0066f60 0x7f0cc0069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0001640 tx=0x7f0cd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f8920 0x55807e0f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e0af310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807df965f0 con 0x55807e0f9240 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807defdda0 con 0x55807e0f8920 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55807de86150 con 0x55807e0aef30 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40664/0 (socket says 172.21.15.37:40664) 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/739849675 learned_addr learned my addr 172.21.15.37:0/739849675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cde59c640 1 --2- 172.21.15.37:0/739849675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f8920 0x55807e0f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cded9d640 1 --2- 172.21.15.37:0/739849675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e0af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/739849675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 msgr2=0x55807e0af310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/739849675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e0af310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/739849675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f8920 msgr2=0x55807e0f8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/739849675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f8920 0x55807e0f8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/739849675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55807de9fca0 con 0x55807e0f9240 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cded9d640 1 --2- 172.21.15.37:0/739849675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/739849675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0ae9f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f0cc805e180 tx=0x7f0cc8055ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277ca0b7b0450758 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cbffff640 1 -- 172.21.15.37:0/739849675 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc806d030 con 0x55807e0f9240 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cbffff640 1 -- 172.21.15.37:0/739849675 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc8054350 con 0x55807e0f9240 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0cbffff640 1 -- 172.21.15.37:0/739849675 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc805e220 con 0x55807e0f9240 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0ce7edb000 1 -- 172.21.15.37:0/739849675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 msgr2=0x55807e0ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.645+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/739849675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0ae9f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f0cc805e180 tx=0x7f0cc8055ae0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 -- 172.21.15.37:0/739849675 shutdown_connections 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/739849675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e0af310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/739849675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0ae9f0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/739849675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f8920 0x55807e0f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 -- 172.21.15.37:0/739849675 >> 172.21.15.37:0/739849675 conn(0x55807e0b52a0 msgr2=0x55807e0ad2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 -- 172.21.15.37:0/739849675 shutdown_connections 2024-04-25T08:57:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 -- 172.21.15.37:0/739849675 wait complete. 2024-04-25T08:57:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 -- start start 2024-04-25T08:57:16.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 Processor -- start 2024-04-25T08:57:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e0ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f8920 0x55807e100330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807deb8990 con 0x55807e0f9240 2024-04-25T08:57:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807def9e40 con 0x55807e0f8920 2024-04-25T08:57:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55807de89900 con 0x55807e0aef30 2024-04-25T08:57:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 10 client.0 _ll_register_callbacks cb 0x55807dfb11c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:16.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cded9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cded9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0b2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40672/0 (socket says 172.21.15.37:40672) 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cded9d640 1 -- 172.21.15.37:0/545069418 learned_addr learned my addr 172.21.15.37:0/545069418 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f8920 0x55807e100330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cde59c640 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e0ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cded9d640 1 -- 172.21.15.37:0/545069418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 msgr2=0x55807e0ffdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cded9d640 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e0ffdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cded9d640 1 -- 172.21.15.37:0/545069418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f8920 msgr2=0x55807e100330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cded9d640 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f8920 0x55807e100330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cded9d640 1 -- 172.21.15.37:0/545069418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55807deb6630 con 0x55807e0f9240 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cded9d640 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0b2aa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f0cd801dba0 tx=0x7f0cd801a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c36ec52a30846d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/545069418 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800a480 con 0x55807e0f9240 2024-04-25T08:57:16.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/545069418 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0cd80aecc0 con 0x55807e0f9240 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 -- 172.21.15.37:0/545069418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cc8054350 con 0x55807e0f9240 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/545069418 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cd80017c0 con 0x55807e0f9240 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 10 client.7245 fetch_fsmap learned FSMap version 50 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 -- 172.21.15.37:0/545069418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55807de863a0 con 0x55807e0f9240 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/545069418 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc8054350 con 0x55807e0f9240 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/545069418 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0cd8004b70 con 0x55807e0f9240 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 10 client.7245 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 10 client.7245 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 20 client.7245 populate_metadata read hostname 'smithi037' 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55807e0b4160 0x55807e109e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 4 client.7245 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:16.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0ce7edb000 1 -- 172.21.15.37:0/545069418 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55807de99a50 con 0x55807e0b4160 2024-04-25T08:57:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cde59c640 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55807e0b4160 0x55807e109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cde59c640 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55807e0b4160 0x55807e109e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f6fb1171e0ebe7 server_cookie=818a747acba2ab54 in_seq=0 out_seq=0 2024-04-25T08:57:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.649+0000 7f0cbd7fa640 10 client.7245 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/545069418 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55807de99a50 con 0x55807e0b4160 2024-04-25T08:57:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0cbd7fa640 10 client.7245 handle_command_reply: tid=0 2024-04-25T08:57:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 1 client.7245 shutdown 2024-04-25T08:57:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 1 -- 172.21.15.37:0/545069418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 msgr2=0x55807e0b2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0b2aa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f0cd801dba0 tx=0x7f0cd801a0a0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 1 -- 172.21.15.37:0/545069418 shutdown_connections 2024-04-25T08:57:16.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55807e0b4160 0x55807e109e50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f9240 0x55807e0b2aa0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0cbd7fa640 0 client.7245 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:16.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f8920 0x55807e100330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/545069418 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e0ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 1 -- 172.21.15.37:0/545069418 >> 172.21.15.37:0/545069418 conn(0x55807e0b52a0 msgr2=0x55807e0ad9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:16.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 1 -- 172.21.15.37:0/545069418 shutdown_connections 2024-04-25T08:57:16.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 1 -- 172.21.15.37:0/545069418 wait complete. 2024-04-25T08:57:16.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.653+0000 7f0ce7edb000 20 client.7245 trim_cache size 0 max 16384 2024-04-25T08:57:16.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.665+0000 7f0ce7edb000 1 Processor -- start 2024-04-25T08:57:16.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.665+0000 7f0ce7edb000 1 -- start start 2024-04-25T08:57:16.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0aef30 0x55807e153ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0f8920 0x55807e154420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f9240 0x55807e154960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01ffca0 con 0x55807e0aef30 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01ff0a0 con 0x55807e0f9240 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce01ff6a0 con 0x55807e0f8920 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cde59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0aef30 0x55807e153ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cde59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0aef30 0x55807e153ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40680/0 (socket says 172.21.15.37:40680) 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cde59c640 1 -- 172.21.15.37:0/4021097443 learned_addr learned my addr 172.21.15.37:0/4021097443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cde59c640 1 -- 172.21.15.37:0/4021097443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0f8920 msgr2=0x55807e154420 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cde59c640 1 --2- 172.21.15.37:0/4021097443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0f8920 0x55807e154420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cde59c640 1 -- 172.21.15.37:0/4021097443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f9240 msgr2=0x55807e154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cded9d640 1 --2- 172.21.15.37:0/4021097443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f9240 0x55807e154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cde59c640 1 --2- 172.21.15.37:0/4021097443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f9240 0x55807e154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cde59c640 1 -- 172.21.15.37:0/4021097443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55807e0f9070 con 0x55807e0aef30 2024-04-25T08:57:16.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cde59c640 1 --2- 172.21.15.37:0/4021097443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0aef30 0x55807e153ee0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0001550 tx=0x7f0cd0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ca5a02968fa414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:16.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cbf7fe640 1 -- 172.21.15.37:0/4021097443 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00075f0 con 0x55807e0aef30 2024-04-25T08:57:16.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cbf7fe640 1 -- 172.21.15.37:0/4021097443 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0028030 con 0x55807e0aef30 2024-04-25T08:57:16.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cbf7fe640 1 -- 172.21.15.37:0/4021097443 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd000ea90 con 0x55807e0aef30 2024-04-25T08:57:16.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4021097443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0aef30 msgr2=0x55807e153ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/4021097443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0aef30 0x55807e153ee0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0001550 tx=0x7f0cd0014000 comp rx=0 tx=0).stop 2024-04-25T08:57:16.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4021097443 shutdown_connections 2024-04-25T08:57:16.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/4021097443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f9240 0x55807e154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/4021097443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0f8920 0x55807e154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/4021097443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0aef30 0x55807e153ee0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4021097443 >> 172.21.15.37:0/4021097443 conn(0x55807e0b52a0 msgr2=0x55807e0b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:16.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4021097443 shutdown_connections 2024-04-25T08:57:16.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4021097443 wait complete. 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- start start 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 Processor -- start 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e1589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f8920 0x55807e1138d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f9240 0x55807e113e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807df023d0 con 0x55807e0f8920 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55807dfb9d50 con 0x55807e0f9240 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55807de98980 con 0x55807e0aef30 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0ce7edb000 10 client.0 _ll_register_callbacks cb 0x7f0cc80095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cd5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f8920 0x55807e1138d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cd5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f8920 0x55807e1138d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40688/0 (socket says 172.21.15.37:40688) 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/4237453327 learned_addr learned my addr 172.21.15.37:0/4237453327 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cde59c640 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e1589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/4237453327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 msgr2=0x55807e1589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:16.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e1589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/4237453327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f9240 msgr2=0x55807e113e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f9240 0x55807e113e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.669+0000 7f0cd5d9b640 1 -- 172.21.15.37:0/4237453327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55807df7f600 con 0x55807e0f8920 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cd5d9b640 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f8920 0x55807e1138d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8056210 tx=0x7f0cc8059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea26d7908f8d0396 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/4237453327 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc806d040 con 0x55807e0f8920 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/4237453327 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0cc80554a0 con 0x55807e0f8920 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4237453327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0cd0028030 con 0x55807e0f8920 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/4237453327 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cc8054410 con 0x55807e0f8920 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 10 client.7251 fetch_fsmap learned FSMap version 50 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4237453327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=2+}) v3 -- 0x55807de929b0 con 0x55807e0f8920 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/4237453327 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0028030 con 0x55807e0f8920 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/4237453327 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 50) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0cc8051da0 con 0x55807e0f8920 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 10 client.7251 fetch_fsmap finished waiting for FSMap version 50 2024-04-25T08:57:16.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 10 client.7251 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 20 client.7251 populate_metadata read hostname 'smithi037' 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55807e114350 0x55807e114730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 4 client.7251 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4237453327 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55807df0b2e0 con 0x55807e114350 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cde59c640 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55807e114350 0x55807e114730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cde59c640 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55807e114350 0x55807e114730 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69a8649cb62adf5c server_cookie=bc952107d30572f1 in_seq=0 out_seq=0 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cbd7fa640 10 client.7251 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cbd7fa640 1 -- 172.21.15.37:0/4237453327 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55807df0b2e0 con 0x55807e114350 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cbd7fa640 10 client.7251 handle_command_reply: tid=0 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 client.7251 shutdown 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4237453327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f8920 msgr2=0x55807e1138d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f8920 0x55807e1138d0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8056210 tx=0x7f0cc8059000 comp rx=0 tx=0).stop 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4237453327 shutdown_connections 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55807e114350 0x55807e114730 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55807e0f9240 0x55807e113e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55807e0f8920 0x55807e1138d0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0cbd7fa640 0 client.7251 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 --2- 172.21.15.37:0/4237453327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55807e0aef30 0x55807e1589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4237453327 >> 172.21.15.37:0/4237453327 conn(0x55807e0b52a0 msgr2=0x55807e1009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4237453327 shutdown_connections 2024-04-25T08:57:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 1 -- 172.21.15.37:0/4237453327 wait complete. 2024-04-25T08:57:16.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce7edb000 20 client.7251 trim_cache size 0 max 16384 2024-04-25T08:57:16.669 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:57:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T08:57:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T08:57:16.669 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7200, 2024-04-25T08:57:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T08:57:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T08:57:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T08:57:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 4.472785172, 2024-04-25T08:57:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 50, 2024-04-25T08:57:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 83, 2024-04-25T08:57:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 83, 2024-04-25T08:57:16.670 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 5.4412258690000002 2024-04-25T08:57:16.670 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:57:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0cc0066f60 msgr2=0x7f0cc0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0cc0066f60 0x7f0cc0069420 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0001640 tx=0x7f0cd0009040 comp rx=0 tx=0).stop 2024-04-25T08:57:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/3871586762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ce00ae130 msgr2=0x7f0ce01e1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ce00ae130 0x7f0ce01e1030 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8010840 tx=0x7f0cc8010d00 comp rx=0 tx=0).stop 2024-04-25T08:57:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/3871586762 shutdown_connections 2024-04-25T08:57:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0cc0066f60 0x7f0cc0069420 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ce013e8a0 0x7f0ce013ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ce00ae130 0x7f0ce01e1030 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 --2- 172.21.15.37:0/3871586762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ccc004d50 0x7f0ce00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/3871586762 >> 172.21.15.37:0/3871586762 conn(0x7f0ce00b11a0 msgr2=0x7f0ce00af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/3871586762 shutdown_connections 2024-04-25T08:57:16.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.673+0000 7f0ce4d6e640 1 -- 172.21.15.37:0/3871586762 wait complete. 2024-04-25T08:57:16.681 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:57:16.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.805+0000 7f0a16f05640 1 Processor -- start 2024-04-25T08:57:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.805+0000 7f0a16f05640 1 -- start start 2024-04-25T08:57:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a16f05640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0a1006b820 0x7f0a100b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a16f05640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0a100b1ac0 0x7f0a100cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a16f05640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a100afb20 con 0x7f0a1006b440 2024-04-25T08:57:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a16f05640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a100b06c0 con 0x7f0a1006b820 2024-04-25T08:57:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a16f05640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a100afd90 con 0x7f0a100b1ac0 2024-04-25T08:57:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a14ca1640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0a1006b440 0x7f0a100b13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:58248/0 (socket says 172.21.15.37:58248) 2024-04-25T08:57:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a14ca1640 1 -- 172.21.15.37:0/230114710 learned_addr learned my addr 172.21.15.37:0/230114710 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a0ffff640 1 --2- 172.21.15.37:0/230114710 >> v2:172.21.15.37:3300/0 conn(0x7f0a1006b820 0x7f0a100b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a154a2640 1 --2- 172.21.15.37:0/230114710 >> v2:172.21.15.203:3300/0 conn(0x7f0a100b1ac0 0x7f0a100cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a0f7fe640 1 -- 172.21.15.37:0/230114710 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665294802 0 0) 0x7f0a100afb20 con 0x7f0a1006b440 2024-04-25T08:57:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a0f7fe640 1 -- 172.21.15.37:0/230114710 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09fc003530 con 0x7f0a1006b440 2024-04-25T08:57:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a0ffff640 1 -- 172.21.15.37:0/230114710 >> v2:172.21.15.203:3300/0 conn(0x7f0a100b1ac0 msgr2=0x7f0a100cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a0ffff640 1 --2- 172.21.15.37:0/230114710 >> v2:172.21.15.203:3300/0 conn(0x7f0a100b1ac0 0x7f0a100cf650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a0ffff640 1 -- 172.21.15.37:0/230114710 >> v1:172.21.15.37:6789/0 conn(0x7f0a1006b440 legacy=0x7f0a100b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a0ffff640 1 -- 172.21.15.37:0/230114710 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a100b0070 con 0x7f0a1006b820 2024-04-25T08:57:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a0ffff640 1 --2- 172.21.15.37:0/230114710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a1006b820 0x7f0a100b0b50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f0a10073280 tx=0x7f09f801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a7c7b927b56a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a0f7fe640 1 -- 172.21.15.37:0/230114710 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80b1070 con 0x7f0a1006b820 2024-04-25T08:57:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a0f7fe640 1 -- 172.21.15.37:0/230114710 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f80ac5f0 con 0x7f0a1006b820 2024-04-25T08:57:16.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a0f7fe640 1 -- 172.21.15.37:0/230114710 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80ac9c0 con 0x7f0a1006b820 2024-04-25T08:57:16.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a16f05640 1 -- 172.21.15.37:0/230114710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a1006b820 msgr2=0x7f0a100b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a16f05640 1 --2- 172.21.15.37:0/230114710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a1006b820 0x7f0a100b0b50 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f0a10073280 tx=0x7f09f801c480 comp rx=0 tx=0).stop 2024-04-25T08:57:16.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a16f05640 1 -- 172.21.15.37:0/230114710 shutdown_connections 2024-04-25T08:57:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a16f05640 1 --2- 172.21.15.37:0/230114710 >> v2:172.21.15.203:3300/0 conn(0x7f0a100b1ac0 0x7f0a100cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.809+0000 7f0a16f05640 1 --2- 172.21.15.37:0/230114710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a1006b820 0x7f0a100b0b50 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 -- 172.21.15.37:0/230114710 >> 172.21.15.37:0/230114710 conn(0x7f0a100b1f10 msgr2=0x7f0a100b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 -- 172.21.15.37:0/230114710 shutdown_connections 2024-04-25T08:57:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 -- 172.21.15.37:0/230114710 wait complete. 2024-04-25T08:57:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 Processor -- start 2024-04-25T08:57:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 -- start start 2024-04-25T08:57:16.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a1006b440 0x7f0a10152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a1006b820 0x7f0a100c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a100b1ac0 0x7f0a1006f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a100cd130 con 0x7f0a1006b440 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a100ccfb0 con 0x7f0a100b1ac0 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1015d080 con 0x7f0a1006b820 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a14ca1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a1006b440 0x7f0a10152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a14ca1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a1006b440 0x7f0a10152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40704/0 (socket says 172.21.15.37:40704) 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a14ca1640 1 -- 172.21.15.37:0/2271467981 learned_addr learned my addr 172.21.15.37:0/2271467981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a0ffff640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a1006b820 0x7f0a100c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a154a2640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a100b1ac0 0x7f0a1006f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a14ca1640 1 -- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a1006b820 msgr2=0x7f0a100c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a14ca1640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a1006b820 0x7f0a100c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a14ca1640 1 -- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a100b1ac0 msgr2=0x7f0a1006f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a14ca1640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a100b1ac0 0x7f0a1006f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a14ca1640 1 -- 172.21.15.37:0/2271467981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a100c4830 con 0x7f0a1006b440 2024-04-25T08:57:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a154a2640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a100b1ac0 0x7f0a1006f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a14ca1640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a1006b440 0x7f0a10152390 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c480 tx=0x7f0a0000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a0d7fa640 1 -- 172.21.15.37:0/2271467981 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00017070 con 0x7f0a1006b440 2024-04-25T08:57:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a0d7fa640 1 -- 172.21.15.37:0/2271467981 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a00005cf0 con 0x7f0a1006b440 2024-04-25T08:57:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a0d7fa640 1 -- 172.21.15.37:0/2271467981 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00007290 con 0x7f0a1006b440 2024-04-25T08:57:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 -- 172.21.15.37:0/2271467981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a100c4b10 con 0x7f0a1006b440 2024-04-25T08:57:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.813+0000 7f0a16f05640 1 -- 172.21.15.37:0/2271467981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a100c50d0 con 0x7f0a1006b440 2024-04-25T08:57:16.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.817+0000 7f0a16f05640 1 -- 172.21.15.37:0/2271467981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1004efa0 con 0x7f0a1006b440 2024-04-25T08:57:16.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.817+0000 7f0a0d7fa640 1 -- 172.21.15.37:0/2271467981 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a0001a070 con 0x7f0a1006b440 2024-04-25T08:57:16.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.817+0000 7f0a0d7fa640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f0066f60 0x7f09f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:16.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.817+0000 7f0a0d7fa640 1 -- 172.21.15.37:0/2271467981 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0a00084650 con 0x7f0a1006b440 2024-04-25T08:57:16.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.817+0000 7f0a0ffff640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f0066f60 0x7f09f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:16.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.817+0000 7f0a0ffff640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f0066f60 0x7f09f0069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f09f80ae000 tx=0x7f09f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:16.821+0000 7f0a0d7fa640 1 -- 172.21.15.37:0/2271467981 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a00055480 con 0x7f0a1006b440 2024-04-25T08:57:17.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.017+0000 7f0a16f05640 1 -- 172.21.15.37:0/2271467981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0a100cc160 con 0x7f0a1006b440 2024-04-25T08:57:17.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.017+0000 7f0a0d7fa640 1 -- 172.21.15.37:0/2271467981 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v50) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0a00054d80 con 0x7f0a1006b440 2024-04-25T08:57:17.008 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 -- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f0066f60 msgr2=0x7f09f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f0066f60 0x7f09f0069420 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f09f80ae000 tx=0x7f09f8008040 comp rx=0 tx=0).stop 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 -- 172.21.15.37:0/2271467981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a1006b440 msgr2=0x7f0a10152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a1006b440 0x7f0a10152390 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0a0000c480 tx=0x7f0a0000c940 comp rx=0 tx=0).stop 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 -- 172.21.15.37:0/2271467981 shutdown_connections 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f0066f60 0x7f09f0069420 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0a100b1ac0 0x7f0a1006f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a1006b820 0x7f0a100c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 --2- 172.21.15.37:0/2271467981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a1006b440 0x7f0a10152390 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 -- 172.21.15.37:0/2271467981 >> 172.21.15.37:0/2271467981 conn(0x7f0a100b1f10 msgr2=0x7f0a100cb630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 -- 172.21.15.37:0/2271467981 shutdown_connections 2024-04-25T08:57:17.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:17.021+0000 7f0a16f05640 1 -- 172.21.15.37:0/2271467981 wait complete. 2024-04-25T08:57:17.029 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:57:17.029 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:57:17.029 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T08:57:17.029 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:57:17.029 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:17.029 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:57:17.029 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:57:17.029 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:57:17.029 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:57:17.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:17.030 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T08:57:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:57:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:57:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:57:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:57:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:17.036 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:57:17.036 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42204sec preferred_lft 42204sec 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:17.037 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:57:17.038 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:57:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:57:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:17.039 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:57:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:57:17.039 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:17.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:17.039 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:57:17.089 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:57:17.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:17.091 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T08:57:17.140 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T08:57:17.141 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:57:17.141 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:57:17.141 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:57:17.141 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:57:17.141 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:57:17.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:17.141 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:57:17.141 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:57:17.141 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:57:17.142 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:57:17.142 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:57:17.142 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:57:17.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:17.220 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:57:17.267 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:57:17.267 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:57:17.267 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:57:17.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:17.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T08:57:17.324 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:57:17.325 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:57:17.325 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:17.375 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:57:17.376 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:57:17.376 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:17.423 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:57:17.484 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:57:17.533 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:57:17.533 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:57:17.533 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:57:17.534 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:57:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:57:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:57:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:57:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:57:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:57:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:57:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:57:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:57:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:57:17.535 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:57:17.536 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:57:17.536 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:57:17.536 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:17.536 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:17.536 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:17.536 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:17.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:17.537 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:57:17.583 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:57:17.583 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:57:17.628 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:57:17.643 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:57:17.696 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:57:17.696 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:57:17.696 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:57:17.696 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:57:17.696 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:57:17.696 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:57:17.696 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:57:17.696 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:57:17.696 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:57:17.696 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:57:17.697 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:57:17.698 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:57:17.698 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:57:17.698 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:57:17.698 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:57:17.698 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:17.698 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:17.698 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:17.698 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:17.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:17.698 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:57:17.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T08:57:17.721+0000 7f97ed0d1580 -1 init, newargv = 0x55d9fe4dbfc0 newargc=15 2024-04-25T08:57:17.713 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[45642]: starting ceph client 2024-04-25T08:57:17.720 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[45642]: starting fuse 2024-04-25T08:57:18.192 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T08:57:18.201+0000 7efd5f9cb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:57:18.819 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:57:18.836 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:57:18.886 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:57:18.886 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:57:18.886 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:57:18.886 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:57:18.887 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:57:18.887 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:57:18.887 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:57:18.887 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:57:18.887 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:57:18.887 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:57:18.888 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:57:18.888 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:57:18.888 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:57:18.888 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:57:18.888 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:57:18.888 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:57:18.888 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:57:18.889 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:57:18.889 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:57:18.889 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:57:18.889 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:57:18.889 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:57:18.889 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:57:18.890 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:57:18.890 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:57:18.890 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:57:18.890 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:57:18.890 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:57:18.890 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:57:18.891 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:18.891 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:18.891 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:18.891 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:18.891 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:18.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:18.893 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:57:18.939 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:57:18.940 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:57:18.940 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:57:18.940 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T08:57:18.940 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T08:57:18.940 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:57:18.940 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T08:57:18.940 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:57:18.940 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T08:57:18.940 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:57:18.940 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T08:57:18.940 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:57:18.940 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:57:18.941 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T08:57:18.941 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T08:57:18.941 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T08:57:18.941 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T08:57:18.941 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:57:18.941 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T08:57:18.941 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:57:18.941 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:57:18.941 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:57:18.941 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T08:57:18.941 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T08:57:18.942 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T08:57:18.942 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:57:18.942 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:57:18.942 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T08:57:18.942 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:57:19.067 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.45642.asok 2024-04-25T08:57:19.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:57:19.079 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45642.asok 2024-04-25T08:57:19.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:19.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45642.asok status 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "45642", 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7263, 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 7263 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3798505295 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3798505295 2024-04-25T08:57:19.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:57:19.237 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.7263 192.168.0.1:0/3798505295", 2024-04-25T08:57:19.237 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/3798505295", 2024-04-25T08:57:19.237 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T08:57:19.237 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 50, 2024-04-25T08:57:19.237 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 83, 2024-04-25T08:57:19.237 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:57:19.237 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T08:57:19.237 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T08:57:19.237 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:57:19.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:19.248 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T08:57:19.249 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:19.255 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T08:57:19.255 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:19.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:19.255 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:19.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:19.311 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-04-25T08:57:19.366 INFO:teuthology.orchestra.run.smithi037.stdout:file_undamaged 2024-04-25T08:57:19.367 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T08:57:19.367 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:57:19.367 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T08:57:19.367 DEBUG:teuthology.orchestra.run.smithi037:> import json 2024-04-25T08:57:19.367 DEBUG:teuthology.orchestra.run.smithi037:> import sys 2024-04-25T08:57:19.367 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:57:19.367 DEBUG:teuthology.orchestra.run.smithi037:> try: 2024-04-25T08:57:19.368 DEBUG:teuthology.orchestra.run.smithi037:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-04-25T08:57:19.368 DEBUG:teuthology.orchestra.run.smithi037:> except OSError as e: 2024-04-25T08:57:19.368 DEBUG:teuthology.orchestra.run.smithi037:> sys.exit(e.errno) 2024-04-25T08:57:19.368 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:57:19.368 DEBUG:teuthology.orchestra.run.smithi037:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T08:57:19.368 DEBUG:teuthology.orchestra.run.smithi037:> print(json.dumps( 2024-04-25T08:57:19.368 DEBUG:teuthology.orchestra.run.smithi037:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T08:57:19.368 DEBUG:teuthology.orchestra.run.smithi037:> indent=2)) 2024-04-25T08:57:19.368 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:57:19.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T08:57:19.480 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T08:57:19.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:19.481 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T08:57:19.529 INFO:teuthology.orchestra.run.smithi037.stdout:2 2024-04-25T08:57:19.530 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:57:19.530 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:19.576 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:19.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.761+0000 7f80882bf640 1 Processor -- start 2024-04-25T08:57:19.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.761+0000 7f80882bf640 1 -- start start 2024-04-25T08:57:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.761+0000 7f80882bf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80800b0a30 0x7f80800cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.761+0000 7f80882bf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f808006f4a0 0x7f8080099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.761+0000 7f80882bf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80800b0e10 con 0x7f8080099fb0 2024-04-25T08:57:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.761+0000 7f80882bf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80800d0d90 con 0x7f80800b0a30 2024-04-25T08:57:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.761+0000 7f80882bf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80800d0180 con 0x7f808006f4a0 2024-04-25T08:57:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.761+0000 7f808685c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8080099fb0 0x7f808015d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36454/0 (socket says 172.21.15.37:36454) 2024-04-25T08:57:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.761+0000 7f808605b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f80800b0a30 0x7f80800cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.761+0000 7f808685c640 1 -- 172.21.15.37:0/685724272 learned_addr learned my addr 172.21.15.37:0/685724272 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f808585a640 1 --2- 172.21.15.37:0/685724272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f808006f4a0 0x7f8080099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f808605b640 1 -- 172.21.15.37:0/685724272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f808006f4a0 msgr2=0x7f8080099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f808605b640 1 --2- 172.21.15.37:0/685724272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f808006f4a0 0x7f8080099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f808605b640 1 -- 172.21.15.37:0/685724272 >> v1:172.21.15.37:6789/0 conn(0x7f8080099fb0 legacy=0x7f808015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f808605b640 1 -- 172.21.15.37:0/685724272 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800d0460 con 0x7f80800b0a30 2024-04-25T08:57:19.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f808585a640 1 --2- 172.21.15.37:0/685724272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f808006f4a0 0x7f8080099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:19.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f808605b640 1 --2- 172.21.15.37:0/685724272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80800b0a30 0x7f80800cf890 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f807c0097c0 tx=0x7f807c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6889f894d2494bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f8085059640 1 -- 172.21.15.37:0/685724272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0b2070 con 0x7f80800b0a30 2024-04-25T08:57:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f8085059640 1 -- 172.21.15.37:0/685724272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c0ad640 con 0x7f80800b0a30 2024-04-25T08:57:19.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f8085059640 1 -- 172.21.15.37:0/685724272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0ada10 con 0x7f80800b0a30 2024-04-25T08:57:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f80882bf640 1 -- 172.21.15.37:0/685724272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80800b0a30 msgr2=0x7f80800cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f80882bf640 1 --2- 172.21.15.37:0/685724272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80800b0a30 0x7f80800cf890 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f807c0097c0 tx=0x7f807c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f80882bf640 1 -- 172.21.15.37:0/685724272 shutdown_connections 2024-04-25T08:57:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f80882bf640 1 --2- 172.21.15.37:0/685724272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f808006f4a0 0x7f8080099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f80882bf640 1 --2- 172.21.15.37:0/685724272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80800b0a30 0x7f80800cf890 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f80882bf640 1 -- 172.21.15.37:0/685724272 >> 172.21.15.37:0/685724272 conn(0x7f80800b30b0 msgr2=0x7f80800b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f80882bf640 1 -- 172.21.15.37:0/685724272 shutdown_connections 2024-04-25T08:57:19.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.765+0000 7f80882bf640 1 -- 172.21.15.37:0/685724272 wait complete. 2024-04-25T08:57:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f80882bf640 1 Processor -- start 2024-04-25T08:57:19.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f80882bf640 1 -- start start 2024-04-25T08:57:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f80882bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f808006f4a0 0x7f808014f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f80882bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8080099fb0 0x7f80800c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f80882bf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80800b0a30 0x7f8080154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f80882bf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808015ded0 con 0x7f808006f4a0 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f80882bf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808015dd50 con 0x7f8080099fb0 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f80882bf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80800cc420 con 0x7f80800b0a30 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f808605b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f808006f4a0 0x7f808014f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f808605b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f808006f4a0 0x7f808014f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49954/0 (socket says 172.21.15.37:49954) 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f808605b640 1 -- 172.21.15.37:0/4231195543 learned_addr learned my addr 172.21.15.37:0/4231195543 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f808685c640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80800b0a30 0x7f8080154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f808605b640 1 -- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80800b0a30 msgr2=0x7f8080154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f808605b640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80800b0a30 0x7f8080154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f808605b640 1 -- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8080099fb0 msgr2=0x7f80800c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f808605b640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8080099fb0 0x7f80800c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f808605b640 1 -- 172.21.15.37:0/4231195543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800c8690 con 0x7f808006f4a0 2024-04-25T08:57:19.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f808685c640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80800b0a30 0x7f8080154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f808605b640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f808006f4a0 0x7f808014f700 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f807c0b0040 tx=0x7f807c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f8072ffd640 1 -- 172.21.15.37:0/4231195543 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0ba020 con 0x7f808006f4a0 2024-04-25T08:57:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f8072ffd640 1 -- 172.21.15.37:0/4231195543 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807c0add20 con 0x7f808006f4a0 2024-04-25T08:57:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f8072ffd640 1 -- 172.21.15.37:0/4231195543 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807c0b2040 con 0x7f808006f4a0 2024-04-25T08:57:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f80882bf640 1 -- 172.21.15.37:0/4231195543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80800c8970 con 0x7f808006f4a0 2024-04-25T08:57:19.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f80882bf640 1 -- 172.21.15.37:0/4231195543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8080151960 con 0x7f808006f4a0 2024-04-25T08:57:19.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f80882bf640 1 -- 172.21.15.37:0/4231195543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808004efa0 con 0x7f808006f4a0 2024-04-25T08:57:19.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.769+0000 7f8072ffd640 1 -- 172.21.15.37:0/4231195543 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f807c0bd020 con 0x7f808006f4a0 2024-04-25T08:57:19.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.773+0000 7f8072ffd640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8064066f60 0x7f8064069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.773+0000 7f8072ffd640 1 -- 172.21.15.37:0/4231195543 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f807c11ef20 con 0x7f808006f4a0 2024-04-25T08:57:19.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.773+0000 7f808585a640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8064066f60 0x7f8064069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.773+0000 7f808585a640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8064066f60 0x7f8064069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8080068850 tx=0x7f8074008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.777+0000 7f8072ffd640 1 -- 172.21.15.37:0/4231195543 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f807c0c4030 con 0x7f808006f4a0 2024-04-25T08:57:19.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.937+0000 7f80882bf640 1 -- 172.21.15.37:0/4231195543 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8080151750 con 0x7f808006f4a0 2024-04-25T08:57:19.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f8072ffd640 1 -- 172.21.15.37:0/4231195543 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17038 (secure 0 0 0) 0x7f807c016090 con 0x7f808006f4a0 2024-04-25T08:57:19.931 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:57:19.931 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":83,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:57:11.180680+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:56:49.737591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:56:50.436964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/4130042812":"2024-04-26T08:57:11.180643+0000","172.21.15.37:6834/4130042812":"2024-04-26T08:57:11.180643+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 -- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8064066f60 msgr2=0x7f8064069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8064066f60 0x7f8064069420 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f8080068850 tx=0x7f8074008040 comp rx=0 tx=0).stop 2024-04-25T08:57:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 -- 172.21.15.37:0/4231195543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f808006f4a0 msgr2=0x7f808014f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f808006f4a0 0x7f808014f700 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f807c0b0040 tx=0x7f807c019240 comp rx=0 tx=0).stop 2024-04-25T08:57:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 -- 172.21.15.37:0/4231195543 shutdown_connections 2024-04-25T08:57:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8064066f60 0x7f8064069420 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80800b0a30 0x7f8080154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8080099fb0 0x7f80800c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 --2- 172.21.15.37:0/4231195543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f808006f4a0 0x7f808014f700 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 -- 172.21.15.37:0/4231195543 >> 172.21.15.37:0/4231195543 conn(0x7f80800b30b0 msgr2=0x7f80800afa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 -- 172.21.15.37:0/4231195543 shutdown_connections 2024-04-25T08:57:19.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:19.941+0000 7f80882bf640 1 -- 172.21.15.37:0/4231195543 wait complete. 2024-04-25T08:57:19.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:56:49.737591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:56:50.436964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:57:19.951 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:57:20.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf6c6d640 1 Processor -- start 2024-04-25T08:57:20.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf6c6d640 1 -- start start 2024-04-25T08:57:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf6c6d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7feaf00d4230 0x7feaf00d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf6c6d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf006be60 con 0x7feaf015da80 2024-04-25T08:57:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf6c6d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaf00d4650 con 0x7feaf00d07f0 2024-04-25T08:57:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf6c6d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaf00d2210 con 0x7feaf00d4230 2024-04-25T08:57:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf4a09640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7feaf015da80 0x7feaf00d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36456/0 (socket says 172.21.15.37:36456) 2024-04-25T08:57:20.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf4a09640 1 -- 172.21.15.37:0/2663687211 learned_addr learned my addr 172.21.15.37:0/2663687211 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf520a640 1 --2- 172.21.15.37:0/2663687211 >> v2:172.21.15.203:3300/0 conn(0x7feaf00d4230 0x7feaf00d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101963929 0 0) 0x7feaf006be60 con 0x7feaf015da80 2024-04-25T08:57:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fead8003530 con 0x7feaf015da80 2024-04-25T08:57:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1310248756 0 0) 0x7fead8003530 con 0x7feaf015da80 2024-04-25T08:57:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 >> v2:172.21.15.203:3300/0 conn(0x7feaf00d4230 msgr2=0x7feaf00d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 --2- 172.21.15.37:0/2663687211 >> v2:172.21.15.203:3300/0 conn(0x7feaf00d4230 0x7feaf00d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 >> v1:172.21.15.203:6789/0 conn(0x7feaf00d07f0 legacy=0x7feaf006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf015de60 con 0x7feaf015da80 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7feae0001680 con 0x7feaf015da80 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 >> v1:172.21.15.37:6789/0 conn(0x7feaf015da80 legacy=0x7feaf00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fead8003a30 0x7fead8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 0x7fead8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fead80051d0 0x7fead800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf00d2210 con 0x7fead8004810 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead800c100 con 0x7fead80051d0 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead800c300 con 0x7fead8003a30 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaeffff640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 0x7fead8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf4a09640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fead8003a30 0x7fead8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf520a640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fead80051d0 0x7fead800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaeffff640 1 -- 172.21.15.37:0/2663687211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fead8003a30 msgr2=0x7fead8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaeffff640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fead8003a30 0x7fead8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaeffff640 1 -- 172.21.15.37:0/2663687211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fead80051d0 msgr2=0x7fead800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:20.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaeffff640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fead80051d0 0x7fead800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaeffff640 1 -- 172.21.15.37:0/2663687211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8003530 con 0x7fead8004810 2024-04-25T08:57:20.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf4a09640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fead8003a30 0x7fead8003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:20.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf520a640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fead80051d0 0x7fead800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:20.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.093+0000 7feaf520a640 1 -- 172.21.15.37:0/2663687211 reap_dead start 2024-04-25T08:57:20.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaeffff640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 0x7fead8004c90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7feae40017b0 tx=0x7feae401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3246a818307195ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae40013d0 con 0x7fead8004810 2024-04-25T08:57:20.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae40052d0 con 0x7fead8004810 2024-04-25T08:57:20.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaef7fe640 1 -- 172.21.15.37:0/2663687211 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae4003650 con 0x7fead8004810 2024-04-25T08:57:20.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaf6c6d640 1 -- 172.21.15.37:0/2663687211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 msgr2=0x7fead8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaf6c6d640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 0x7fead8004c90 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7feae40017b0 tx=0x7feae401c3e0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaf6c6d640 1 -- 172.21.15.37:0/2663687211 shutdown_connections 2024-04-25T08:57:20.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaf6c6d640 1 --2- 172.21.15.37:0/2663687211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 0x7fead8004c90 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaf6c6d640 1 -- 172.21.15.37:0/2663687211 >> 172.21.15.37:0/2663687211 conn(0x7feaf008b000 msgr2=0x7feaf00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:20.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaf6c6d640 1 -- 172.21.15.37:0/2663687211 shutdown_connections 2024-04-25T08:57:20.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaf6c6d640 1 -- 172.21.15.37:0/2663687211 wait complete. 2024-04-25T08:57:20.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaf6c6d640 1 Processor -- start 2024-04-25T08:57:20.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.097+0000 7feaf6c6d640 1 -- start start 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf6c6d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 0x7feaf00c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf6c6d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf00ca7c0 0x7feaf00c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf6c6d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf015aec0 0x7feaf014e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf6c6d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf00d2450 con 0x7fead8004810 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf6c6d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaf006c440 con 0x7feaf015aec0 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf6c6d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaf00d26f0 con 0x7feaf00ca7c0 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf4a09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 0x7feaf00c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf4a09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 0x7feaf00c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49974/0 (socket says 172.21.15.37:49974) 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf4a09640 1 -- 172.21.15.37:0/3584282107 learned_addr learned my addr 172.21.15.37:0/3584282107 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf520a640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf015aec0 0x7feaf014e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaeffff640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf00ca7c0 0x7feaf00c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf4a09640 1 -- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf00ca7c0 msgr2=0x7feaf00c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf4a09640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf00ca7c0 0x7feaf00c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf4a09640 1 -- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf015aec0 msgr2=0x7feaf014e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf4a09640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf015aec0 0x7feaf014e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf4a09640 1 -- 172.21.15.37:0/3584282107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf00c9d80 con 0x7fead8004810 2024-04-25T08:57:20.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf4a09640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 0x7feaf00c9840 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7feae0010b40 tx=0x7feae00056e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaed7fa640 1 -- 172.21.15.37:0/3584282107 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae000e9b0 con 0x7fead8004810 2024-04-25T08:57:20.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaed7fa640 1 -- 172.21.15.37:0/3584282107 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae0011040 con 0x7fead8004810 2024-04-25T08:57:20.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf6c6d640 1 -- 172.21.15.37:0/3584282107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaf0150610 con 0x7fead8004810 2024-04-25T08:57:20.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaed7fa640 1 -- 172.21.15.37:0/3584282107 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae000ed50 con 0x7fead8004810 2024-04-25T08:57:20.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf6c6d640 1 -- 172.21.15.37:0/3584282107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaf0150bd0 con 0x7fead8004810 2024-04-25T08:57:20.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaf6c6d640 1 -- 172.21.15.37:0/3584282107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaf004efa0 con 0x7fead8004810 2024-04-25T08:57:20.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaed7fa640 1 -- 172.21.15.37:0/3584282107 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feae001d020 con 0x7fead8004810 2024-04-25T08:57:20.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaed7fa640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feadc066f90 0x7feadc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaed7fa640 1 -- 172.21.15.37:0/3584282107 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7feae0087f30 con 0x7fead8004810 2024-04-25T08:57:20.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.101+0000 7feaeffff640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feadc066f90 0x7feadc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.105+0000 7feaeffff640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feadc066f90 0x7feadc069450 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7feae40ac000 tx=0x7feae40011f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.105+0000 7feaed7fa640 1 -- 172.21.15.37:0/3584282107 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feae0058d60 con 0x7fead8004810 2024-04-25T08:57:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 -- 172.21.15.37:0/3584282107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feaf00b0640 con 0x7fead8004810 2024-04-25T08:57:20.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaed7fa640 1 -- 172.21.15.37:0/3584282107 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7feae0058660 con 0x7fead8004810 2024-04-25T08:57:20.256 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:57:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 -- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feadc066f90 msgr2=0x7feadc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feadc066f90 0x7feadc069450 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7feae40ac000 tx=0x7feae40011f0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 -- 172.21.15.37:0/3584282107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 msgr2=0x7feaf00c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 0x7feaf00c9840 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7feae0010b40 tx=0x7feae00056e0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 -- 172.21.15.37:0/3584282107 shutdown_connections 2024-04-25T08:57:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feadc066f90 0x7feadc069450 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7feaf015aec0 0x7feaf014e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7feaf00ca7c0 0x7feaf00c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 --2- 172.21.15.37:0/3584282107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fead8004810 0x7feaf00c9840 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 -- 172.21.15.37:0/3584282107 >> 172.21.15.37:0/3584282107 conn(0x7feaf008b000 msgr2=0x7feaf00ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 -- 172.21.15.37:0/3584282107 shutdown_connections 2024-04-25T08:57:20.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.265+0000 7feaf6c6d640 1 -- 172.21.15.37:0/3584282107 wait complete. 2024-04-25T08:57:20.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:57:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb708cbc640 1 Processor -- start 2024-04-25T08:57:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb708cbc640 1 -- start start 2024-04-25T08:57:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb708cbc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb7040cf590 0x7fb70415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb708cbc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7040ce170 0x7fb7040ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb708cbc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7040ca4c0 con 0x7fb7040cf970 2024-04-25T08:57:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb708cbc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7040ca780 con 0x7fb7040cf590 2024-04-25T08:57:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb708cbc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040cd440 con 0x7fb7040ce170 2024-04-25T08:57:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb70259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb7040cf590 0x7fb70415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb6fbfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb7040cf970 0x7fb7040c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36460/0 (socket says 172.21.15.37:36460) 2024-04-25T08:57:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb70259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb7040cf590 0x7fb70415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49986/0 (socket says 172.21.15.37:49986) 2024-04-25T08:57:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb6fbfff640 1 -- 172.21.15.37:0/2273252624 learned_addr learned my addr 172.21.15.37:0/2273252624 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb702d9d640 1 --2- 172.21.15.37:0/2273252624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7040ce170 0x7fb7040ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb70259c640 1 -- 172.21.15.37:0/2273252624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7040ce170 msgr2=0x7fb7040ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb70259c640 1 --2- 172.21.15.37:0/2273252624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7040ce170 0x7fb7040ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb70259c640 1 -- 172.21.15.37:0/2273252624 >> v1:172.21.15.37:6789/0 conn(0x7fb7040cf970 legacy=0x7fb7040c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb70259c640 1 -- 172.21.15.37:0/2273252624 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7040cd690 con 0x7fb7040cf590 2024-04-25T08:57:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.405+0000 7fb701d9b640 1 -- 172.21.15.37:0/2273252624 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2016775381 0 0) 0x7fb7040ca4c0 con 0x7fb7040cf970 2024-04-25T08:57:20.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb70259c640 1 --2- 172.21.15.37:0/2273252624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7040cf590 0x7fb70415ecb0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00ba60 tx=0x7fb6ec01eb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4c787259c037f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb701d9b640 1 -- 172.21.15.37:0/2273252624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0b4070 con 0x7fb7040cf590 2024-04-25T08:57:20.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb701d9b640 1 -- 172.21.15.37:0/2273252624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec0a7a50 con 0x7fb7040cf590 2024-04-25T08:57:20.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb701d9b640 1 -- 172.21.15.37:0/2273252624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0a7e20 con 0x7fb7040cf590 2024-04-25T08:57:20.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 -- 172.21.15.37:0/2273252624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7040cf590 msgr2=0x7fb70415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 --2- 172.21.15.37:0/2273252624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7040cf590 0x7fb70415ecb0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec00ba60 tx=0x7fb6ec01eb70 comp rx=0 tx=0).stop 2024-04-25T08:57:20.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 -- 172.21.15.37:0/2273252624 shutdown_connections 2024-04-25T08:57:20.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 --2- 172.21.15.37:0/2273252624 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7040ce170 0x7fb7040ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 --2- 172.21.15.37:0/2273252624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7040cf590 0x7fb70415ecb0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 -- 172.21.15.37:0/2273252624 >> 172.21.15.37:0/2273252624 conn(0x7fb70406ae60 msgr2=0x7fb70406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:20.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 -- 172.21.15.37:0/2273252624 shutdown_connections 2024-04-25T08:57:20.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 -- 172.21.15.37:0/2273252624 wait complete. 2024-04-25T08:57:20.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 Processor -- start 2024-04-25T08:57:20.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 -- start start 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7040ce170 0x7fb70408b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7040cf590 0x7fb704098900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7040cf970 0x7fb704151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040ca240 con 0x7fb7040cf590 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040ca0c0 con 0x7fb7040ce170 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb708cbc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040ac170 con 0x7fb7040cf970 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb6fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7040cf590 0x7fb704098900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb6fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7040cf590 0x7fb704098900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49992/0 (socket says 172.21.15.37:49992) 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb6fbfff640 1 -- 172.21.15.37:0/3085796763 learned_addr learned my addr 172.21.15.37:0/3085796763 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb70259c640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7040ce170 0x7fb70408b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb702d9d640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7040cf970 0x7fb704151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb6fbfff640 1 -- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7040cf970 msgr2=0x7fb704151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb6fbfff640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7040cf970 0x7fb704151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb6fbfff640 1 -- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7040ce170 msgr2=0x7fb70408b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb6fbfff640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7040ce170 0x7fb70408b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb6fbfff640 1 -- 172.21.15.37:0/3085796763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb704073310 con 0x7fb7040cf590 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb70259c640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7040ce170 0x7fb70408b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb6fbfff640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7040cf590 0x7fb704098900 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400c480 tx=0x7fb6f400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb6faffd640 1 -- 172.21.15.37:0/3085796763 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4018070 con 0x7fb7040cf590 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.409+0000 7fb6faffd640 1 -- 172.21.15.37:0/3085796763 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f4007500 con 0x7fb7040cf590 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.413+0000 7fb708cbc640 1 -- 172.21.15.37:0/3085796763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70408b540 con 0x7fb7040cf590 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.413+0000 7fb6faffd640 1 -- 172.21.15.37:0/3085796763 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4004a30 con 0x7fb7040cf590 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.413+0000 7fb708cbc640 1 -- 172.21.15.37:0/3085796763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7040c7900 con 0x7fb7040cf590 2024-04-25T08:57:20.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.413+0000 7fb708cbc640 1 -- 172.21.15.37:0/3085796763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70404efa0 con 0x7fb7040cf590 2024-04-25T08:57:20.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.413+0000 7fb6faffd640 1 -- 172.21.15.37:0/3085796763 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb6f401b070 con 0x7fb7040cf590 2024-04-25T08:57:20.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.413+0000 7fb6faffd640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6e4066fe0 0x7fb6e40694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.413+0000 7fb6faffd640 1 -- 172.21.15.37:0/3085796763 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb6f40845e0 con 0x7fb7040cf590 2024-04-25T08:57:20.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.413+0000 7fb70259c640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6e4066fe0 0x7fb6e40694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.413+0000 7fb70259c640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6e4066fe0 0x7fb6e40694a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0b1000 tx=0x7fb6ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.417+0000 7fb6faffd640 1 -- 172.21.15.37:0/3085796763 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb6f4055480 con 0x7fb7040cf590 2024-04-25T08:57:20.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 -- 172.21.15.37:0/3085796763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7040b4170 con 0x7fb7040cf590 2024-04-25T08:57:20.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb6faffd640 1 -- 172.21.15.37:0/3085796763 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb6f4054d80 con 0x7fb7040cf590 2024-04-25T08:57:20.568 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:57:20.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 -- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6e4066fe0 msgr2=0x7fb6e40694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6e4066fe0 0x7fb6e40694a0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb6ec0b1000 tx=0x7fb6ec008040 comp rx=0 tx=0).stop 2024-04-25T08:57:20.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 -- 172.21.15.37:0/3085796763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7040cf590 msgr2=0x7fb704098900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7040cf590 0x7fb704098900 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fb6f400c480 tx=0x7fb6f400c940 comp rx=0 tx=0).stop 2024-04-25T08:57:20.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 -- 172.21.15.37:0/3085796763 shutdown_connections 2024-04-25T08:57:20.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb6e4066fe0 0x7fb6e40694a0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7040cf970 0x7fb704151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7040cf590 0x7fb704098900 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 --2- 172.21.15.37:0/3085796763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb7040ce170 0x7fb70408b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 -- 172.21.15.37:0/3085796763 >> 172.21.15.37:0/3085796763 conn(0x7fb70406ae60 msgr2=0x7fb70406ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:20.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 -- 172.21.15.37:0/3085796763 shutdown_connections 2024-04-25T08:57:20.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.577+0000 7fb708cbc640 1 -- 172.21.15.37:0/3085796763 wait complete. 2024-04-25T08:57:20.586 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:57:20.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05f38a640 1 Processor -- start 2024-04-25T08:57:20.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05f38a640 1 -- start start 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05f38a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa05815da80 0x7fa0580d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05f38a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0580d1cd0 0x7fa0580d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05f38a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa0580d0620 0x7fa0580ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05f38a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05806c440 con 0x7fa0580d0620 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05f38a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05806be60 con 0x7fa0580d1cd0 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05f38a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05815de60 con 0x7fa05815da80 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05d927640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa0580d0620 0x7fa0580ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05d927640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa0580d0620 0x7fa0580ce3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50000/0 (socket says 172.21.15.37:50000) 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05d927640 1 -- 172.21.15.37:0/3252441531 learned_addr learned my addr 172.21.15.37:0/3252441531 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05c925640 1 --2- 172.21.15.37:0/3252441531 >> v2:172.21.15.203:3300/0 conn(0x7fa0580d1cd0 0x7fa0580d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05d126640 1 --2- 172.21.15.37:0/3252441531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa05815da80 0x7fa0580d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05d927640 1 -- 172.21.15.37:0/3252441531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa05815da80 msgr2=0x7fa0580d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05d927640 1 --2- 172.21.15.37:0/3252441531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa05815da80 0x7fa0580d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05d927640 1 -- 172.21.15.37:0/3252441531 >> v2:172.21.15.203:3300/0 conn(0x7fa0580d1cd0 msgr2=0x7fa0580d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:20.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05d927640 1 --2- 172.21.15.37:0/3252441531 >> v2:172.21.15.203:3300/0 conn(0x7fa0580d1cd0 0x7fa0580d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.721+0000 7fa05d927640 1 -- 172.21.15.37:0/3252441531 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580d2450 con 0x7fa0580d0620 2024-04-25T08:57:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05d927640 1 --2- 172.21.15.37:0/3252441531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0580d0620 0x7fa0580ce3b0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa05806bd60 tx=0x7fa05401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8063a6d61b300b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa04bfff640 1 -- 172.21.15.37:0/3252441531 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540b0070 con 0x7fa0580d0620 2024-04-25T08:57:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa04bfff640 1 -- 172.21.15.37:0/3252441531 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0540ab600 con 0x7fa0580d0620 2024-04-25T08:57:20.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa04bfff640 1 -- 172.21.15.37:0/3252441531 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540ab9d0 con 0x7fa0580d0620 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 -- 172.21.15.37:0/3252441531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0580d0620 msgr2=0x7fa0580ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 --2- 172.21.15.37:0/3252441531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0580d0620 0x7fa0580ce3b0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fa05806bd60 tx=0x7fa05401c520 comp rx=0 tx=0).stop 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 -- 172.21.15.37:0/3252441531 shutdown_connections 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 --2- 172.21.15.37:0/3252441531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0580d0620 0x7fa0580ce3b0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 --2- 172.21.15.37:0/3252441531 >> v2:172.21.15.203:3300/0 conn(0x7fa0580d1cd0 0x7fa0580d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 --2- 172.21.15.37:0/3252441531 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa05815da80 0x7fa0580d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 -- 172.21.15.37:0/3252441531 >> 172.21.15.37:0/3252441531 conn(0x7fa05808b000 msgr2=0x7fa0580d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 -- 172.21.15.37:0/3252441531 shutdown_connections 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 -- 172.21.15.37:0/3252441531 wait complete. 2024-04-25T08:57:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 Processor -- start 2024-04-25T08:57:20.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 -- start start 2024-04-25T08:57:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0580d0620 0x7fa0580b8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0580d1cd0 0x7fa058153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05815da80 0x7fa0581545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05806ca30 con 0x7fa05815da80 2024-04-25T08:57:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05806c8b0 con 0x7fa0580d0620 2024-04-25T08:57:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05f38a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05806cbb0 con 0x7fa0580d1cd0 2024-04-25T08:57:20.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05d927640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05815da80 0x7fa0581545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05d927640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05815da80 0x7fa0581545c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50014/0 (socket says 172.21.15.37:50014) 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.725+0000 7fa05d927640 1 -- 172.21.15.37:0/536745643 learned_addr learned my addr 172.21.15.37:0/536745643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05c925640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0580d1cd0 0x7fa058153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05d126640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0580d0620 0x7fa0580b8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05d927640 1 -- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0580d1cd0 msgr2=0x7fa058153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05d927640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0580d1cd0 0x7fa058153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05d927640 1 -- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0580d0620 msgr2=0x7fa0580b8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05d927640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0580d0620 0x7fa0580b8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05d927640 1 -- 172.21.15.37:0/536745643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0580b76a0 con 0x7fa05815da80 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05d126640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0580d0620 0x7fa0580b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05d927640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05815da80 0x7fa0581545c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa05401cdc0 tx=0x7fa0540192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa049ffb640 1 -- 172.21.15.37:0/536745643 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540b0040 con 0x7fa05815da80 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa049ffb640 1 -- 172.21.15.37:0/536745643 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0540b9070 con 0x7fa05815da80 2024-04-25T08:57:20.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05f38a640 1 -- 172.21.15.37:0/536745643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0580b7980 con 0x7fa05815da80 2024-04-25T08:57:20.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa049ffb640 1 -- 172.21.15.37:0/536745643 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0540b4670 con 0x7fa05815da80 2024-04-25T08:57:20.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05f38a640 1 -- 172.21.15.37:0/536745643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0580b7f40 con 0x7fa05815da80 2024-04-25T08:57:20.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05f38a640 1 -- 172.21.15.37:0/536745643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05804efa0 con 0x7fa05815da80 2024-04-25T08:57:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa049ffb640 1 -- 172.21.15.37:0/536745643 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0540b4810 con 0x7fa05815da80 2024-04-25T08:57:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa049ffb640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa03c066f60 0x7fa03c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa049ffb640 1 -- 172.21.15.37:0/536745643 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa05411f2c0 con 0x7fa05815da80 2024-04-25T08:57:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.729+0000 7fa05d126640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa03c066f60 0x7fa03c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.733+0000 7fa05d126640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa03c066f60 0x7fa03c069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa050009f50 tx=0x7fa05000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.733+0000 7fa049ffb640 1 -- 172.21.15.37:0/536745643 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0540c2020 con 0x7fa05815da80 2024-04-25T08:57:20.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.893+0000 7fa05f38a640 1 -- 172.21.15.37:0/536745643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0580b4e10 con 0x7fa05815da80 2024-04-25T08:57:20.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.893+0000 7fa049ffb640 1 -- 172.21.15.37:0/536745643 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa054016110 con 0x7fa05815da80 2024-04-25T08:57:20.884 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:57:20.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 -- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa03c066f60 msgr2=0x7fa03c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa03c066f60 0x7fa03c069420 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa050009f50 tx=0x7fa05000b040 comp rx=0 tx=0).stop 2024-04-25T08:57:20.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 -- 172.21.15.37:0/536745643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05815da80 msgr2=0x7fa0581545c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05815da80 0x7fa0581545c0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fa05401cdc0 tx=0x7fa0540192c0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 -- 172.21.15.37:0/536745643 shutdown_connections 2024-04-25T08:57:20.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa03c066f60 0x7fa03c069420 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05815da80 0x7fa0581545c0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0580d1cd0 0x7fa058153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 --2- 172.21.15.37:0/536745643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0580d0620 0x7fa0580b8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 -- 172.21.15.37:0/536745643 >> 172.21.15.37:0/536745643 conn(0x7fa05808b000 msgr2=0x7fa0580d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:20.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 -- 172.21.15.37:0/536745643 shutdown_connections 2024-04-25T08:57:20.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:20.897+0000 7fa05f38a640 1 -- 172.21.15.37:0/536745643 wait complete. 2024-04-25T08:57:20.903 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:57:21.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.029+0000 7fa680764640 1 Processor -- start 2024-04-25T08:57:21.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.029+0000 7fa680764640 1 -- start start 2024-04-25T08:57:21.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa680764640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa67809a180 0x7fa6780b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa680764640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa67809a7b0 0x7fa6780ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa680764640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6780cbec0 con 0x7fa6780b2c30 2024-04-25T08:57:21.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa680764640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6780cc1a0 con 0x7fa67809a7b0 2024-04-25T08:57:21.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa680764640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6780cc480 con 0x7fa67809a180 2024-04-25T08:57:21.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67ed01640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa67809a7b0 0x7fa6780ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67ed01640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa67809a7b0 0x7fa6780ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50026/0 (socket says 172.21.15.37:50026) 2024-04-25T08:57:21.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67ed01640 1 -- 172.21.15.37:0/591228701 learned_addr learned my addr 172.21.15.37:0/591228701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:21.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67dcff640 1 --2- 172.21.15.37:0/591228701 >> v2:172.21.15.203:3300/0 conn(0x7fa67809a180 0x7fa6780b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67ed01640 1 -- 172.21.15.37:0/591228701 >> v2:172.21.15.203:3300/0 conn(0x7fa67809a180 msgr2=0x7fa6780b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67ed01640 1 --2- 172.21.15.37:0/591228701 >> v2:172.21.15.203:3300/0 conn(0x7fa67809a180 0x7fa6780b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67ed01640 1 -- 172.21.15.37:0/591228701 >> v1:172.21.15.203:6789/0 conn(0x7fa6780b2c30 legacy=0x7fa678099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67ed01640 1 -- 172.21.15.37:0/591228701 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780cc760 con 0x7fa67809a7b0 2024-04-25T08:57:21.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67ed01640 1 --2- 172.21.15.37:0/591228701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa67809a7b0 0x7fa6780ccf60 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa678072190 tx=0x7fa67001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9577974266fb2595 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67d4fe640 1 -- 172.21.15.37:0/591228701 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6700090d0 con 0x7fa67809a7b0 2024-04-25T08:57:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67d4fe640 1 -- 172.21.15.37:0/591228701 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6700a5860 con 0x7fa67809a7b0 2024-04-25T08:57:21.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.033+0000 7fa67d4fe640 1 -- 172.21.15.37:0/591228701 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670009270 con 0x7fa67809a7b0 2024-04-25T08:57:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- 172.21.15.37:0/591228701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa67809a7b0 msgr2=0x7fa6780ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 --2- 172.21.15.37:0/591228701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa67809a7b0 0x7fa6780ccf60 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fa678072190 tx=0x7fa67001c980 comp rx=0 tx=0).stop 2024-04-25T08:57:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- 172.21.15.37:0/591228701 shutdown_connections 2024-04-25T08:57:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 --2- 172.21.15.37:0/591228701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa67809a7b0 0x7fa6780ccf60 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 --2- 172.21.15.37:0/591228701 >> v2:172.21.15.203:3300/0 conn(0x7fa67809a180 0x7fa6780b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- 172.21.15.37:0/591228701 >> 172.21.15.37:0/591228701 conn(0x7fa6780b3010 msgr2=0x7fa6780b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- 172.21.15.37:0/591228701 shutdown_connections 2024-04-25T08:57:21.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- 172.21.15.37:0/591228701 wait complete. 2024-04-25T08:57:21.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 Processor -- start 2024-04-25T08:57:21.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- start start 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa67809a180 0x7fa678149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa67809a7b0 0x7fa6780af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6780b2c30 0x7fa678150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67806c380 con 0x7fa67809a7b0 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67806c200 con 0x7fa6780b2c30 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67815d190 con 0x7fa67809a180 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa67dcff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa67809a7b0 0x7fa6780af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa67dcff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa67809a7b0 0x7fa6780af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50042/0 (socket says 172.21.15.37:50042) 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa67dcff640 1 -- 172.21.15.37:0/4113827647 learned_addr learned my addr 172.21.15.37:0/4113827647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa67ed01640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6780b2c30 0x7fa678150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa67e500640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa67809a180 0x7fa678149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa67dcff640 1 -- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa67809a180 msgr2=0x7fa678149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa67dcff640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa67809a180 0x7fa678149570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa67dcff640 1 -- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6780b2c30 msgr2=0x7fa678150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa67dcff640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6780b2c30 0x7fa678150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa67dcff640 1 -- 172.21.15.37:0/4113827647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6780c9490 con 0x7fa67809a7b0 2024-04-25T08:57:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa67dcff640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa67809a7b0 0x7fa6780af690 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa66800d800 tx=0x7fa66800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa66effd640 1 -- 172.21.15.37:0/4113827647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800b6c0 con 0x7fa67809a7b0 2024-04-25T08:57:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa66effd640 1 -- 172.21.15.37:0/4113827647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa668019070 con 0x7fa67809a7b0 2024-04-25T08:57:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- 172.21.15.37:0/4113827647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6780c9770 con 0x7fa67809a7b0 2024-04-25T08:57:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa66effd640 1 -- 172.21.15.37:0/4113827647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800ba60 con 0x7fa67809a7b0 2024-04-25T08:57:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- 172.21.15.37:0/4113827647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6780c9d30 con 0x7fa67809a7b0 2024-04-25T08:57:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.037+0000 7fa680764640 1 -- 172.21.15.37:0/4113827647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67804efa0 con 0x7fa67809a7b0 2024-04-25T08:57:21.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.041+0000 7fa66effd640 1 -- 172.21.15.37:0/4113827647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa66800bc00 con 0x7fa67809a7b0 2024-04-25T08:57:21.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.041+0000 7fa66effd640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa65c066f90 0x7fa65c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.041+0000 7fa66effd640 1 -- 172.21.15.37:0/4113827647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fa668085680 con 0x7fa67809a7b0 2024-04-25T08:57:21.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.041+0000 7fa67e500640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa65c066f90 0x7fa65c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.041+0000 7fa67e500640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa65c066f90 0x7fa65c069450 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa6781fa060 tx=0x7fa674008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.045+0000 7fa66effd640 1 -- 172.21.15.37:0/4113827647 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa6680564a0 con 0x7fa67809a7b0 2024-04-25T08:57:21.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.209+0000 7fa680764640 1 -- 172.21.15.37:0/4113827647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa67806ba80 con 0x7fa67809a7b0 2024-04-25T08:57:21.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.209+0000 7fa66effd640 1 -- 172.21.15.37:0/4113827647 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v83) v1 ==== 80+0+122 (secure 0 0 0) 0x7fa668055da0 con 0x7fa67809a7b0 2024-04-25T08:57:21.201 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/4130042812 2024-04-26T08:57:11.180643+0000 2024-04-25T08:57:21.201 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6834/4130042812 2024-04-26T08:57:11.180643+0000 2024-04-25T08:57:21.201 INFO:teuthology.orchestra.run.smithi037.stderr:listed 2 entries 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 -- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa65c066f90 msgr2=0x7fa65c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa65c066f90 0x7fa65c069450 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa6781fa060 tx=0x7fa674008040 comp rx=0 tx=0).stop 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 -- 172.21.15.37:0/4113827647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa67809a7b0 msgr2=0x7fa6780af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa67809a7b0 0x7fa6780af690 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa66800d800 tx=0x7fa66800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 -- 172.21.15.37:0/4113827647 shutdown_connections 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa65c066f90 0x7fa65c069450 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa6780b2c30 0x7fa678150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa67809a7b0 0x7fa6780af690 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 --2- 172.21.15.37:0/4113827647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa67809a180 0x7fa678149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 -- 172.21.15.37:0/4113827647 >> 172.21.15.37:0/4113827647 conn(0x7fa6780b3010 msgr2=0x7fa67806f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 -- 172.21.15.37:0/4113827647 shutdown_connections 2024-04-25T08:57:21.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.213+0000 7fa680764640 1 -- 172.21.15.37:0/4113827647 wait complete. 2024-04-25T08:57:21.218 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:57:21.218 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:21.224 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:21.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.397+0000 7ff2b5124640 1 Processor -- start 2024-04-25T08:57:21.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.397+0000 7ff2b5124640 1 -- start start 2024-04-25T08:57:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2b5124640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff2b00ce510 0x7ff2b00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2b5124640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff2b01451f0 0x7ff2b0142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2b5124640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b00cee30 con 0x7ff2b0144ca0 2024-04-25T08:57:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2b5124640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b00cc2b0 con 0x7ff2b00ce510 2024-04-25T08:57:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2b5124640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b006e290 con 0x7ff2b01451f0 2024-04-25T08:57:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2aed9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff2b00ce510 0x7ff2b00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2aed9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff2b00ce510 0x7ff2b00ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50046/0 (socket says 172.21.15.37:50046) 2024-04-25T08:57:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2a7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff2b0144ca0 0x7ff2b006e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36464/0 (socket says 172.21.15.37:36464) 2024-04-25T08:57:21.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2aed9d640 1 -- 172.21.15.37:0/2736657485 learned_addr learned my addr 172.21.15.37:0/2736657485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:21.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2af59e640 1 --2- 172.21.15.37:0/2736657485 >> v2:172.21.15.203:3300/0 conn(0x7ff2b01451f0 0x7ff2b0142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2aed9d640 1 -- 172.21.15.37:0/2736657485 >> v2:172.21.15.203:3300/0 conn(0x7ff2b01451f0 msgr2=0x7ff2b0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2aed9d640 1 --2- 172.21.15.37:0/2736657485 >> v2:172.21.15.203:3300/0 conn(0x7ff2b01451f0 0x7ff2b0142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2aed9d640 1 -- 172.21.15.37:0/2736657485 >> v1:172.21.15.37:6789/0 conn(0x7ff2b0144ca0 legacy=0x7ff2b006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.401+0000 7ff2aed9d640 1 -- 172.21.15.37:0/2736657485 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00ccc70 con 0x7ff2b00ce510 2024-04-25T08:57:21.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2aed9d640 1 --2- 172.21.15.37:0/2736657485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00ce510 0x7ff2b00ce8f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00b48f0 tx=0x7ff2a001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef096d7cebffa14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2ae59c640 1 -- 172.21.15.37:0/2736657485 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00b5070 con 0x7ff2b00ce510 2024-04-25T08:57:21.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2ae59c640 1 -- 172.21.15.37:0/2736657485 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00a8a10 con 0x7ff2b00ce510 2024-04-25T08:57:21.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2ae59c640 1 -- 172.21.15.37:0/2736657485 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00a8de0 con 0x7ff2b00ce510 2024-04-25T08:57:21.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 -- 172.21.15.37:0/2736657485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00ce510 msgr2=0x7ff2b00ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 --2- 172.21.15.37:0/2736657485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00ce510 0x7ff2b00ce8f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00b48f0 tx=0x7ff2a001ed10 comp rx=0 tx=0).stop 2024-04-25T08:57:21.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 -- 172.21.15.37:0/2736657485 shutdown_connections 2024-04-25T08:57:21.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 --2- 172.21.15.37:0/2736657485 >> v2:172.21.15.203:3300/0 conn(0x7ff2b01451f0 0x7ff2b0142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 --2- 172.21.15.37:0/2736657485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00ce510 0x7ff2b00ce8f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 -- 172.21.15.37:0/2736657485 >> 172.21.15.37:0/2736657485 conn(0x7ff2b00b11a0 msgr2=0x7ff2b00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 -- 172.21.15.37:0/2736657485 shutdown_connections 2024-04-25T08:57:21.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 -- 172.21.15.37:0/2736657485 wait complete. 2024-04-25T08:57:21.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 Processor -- start 2024-04-25T08:57:21.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 -- start start 2024-04-25T08:57:21.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00ce510 0x7ff2b01e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b0144ca0 0x7ff2b01e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2b01451f0 0x7ff2b01e3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b006ee30 con 0x7ff2b00ce510 2024-04-25T08:57:21.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b006df20 con 0x7ff2b0144ca0 2024-04-25T08:57:21.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b00cc900 con 0x7ff2b01451f0 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00ce510 0x7ff2b01e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00ce510 0x7ff2b01e5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50048/0 (socket says 172.21.15.37:50048) 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2aed9d640 1 -- 172.21.15.37:0/3078849874 learned_addr learned my addr 172.21.15.37:0/3078849874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2aed9d640 1 -- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2b01451f0 msgr2=0x7ff2b01e3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2aed9d640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2b01451f0 0x7ff2b01e3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2aed9d640 1 -- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b0144ca0 msgr2=0x7ff2b01e64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2aed9d640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b0144ca0 0x7ff2b01e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2aed9d640 1 -- 172.21.15.37:0/3078849874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b01e3b00 con 0x7ff2b00ce510 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2aed9d640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00ce510 0x7ff2b01e5f90 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00b3040 tx=0x7ff2a001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2a77fe640 1 -- 172.21.15.37:0/3078849874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00b5070 con 0x7ff2b00ce510 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2a77fe640 1 -- 172.21.15.37:0/3078849874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a00b0b40 con 0x7ff2b00ce510 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2a77fe640 1 -- 172.21.15.37:0/3078849874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a00bb410 con 0x7ff2b00ce510 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 -- 172.21.15.37:0/3078849874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b01e1790 con 0x7ff2b00ce510 2024-04-25T08:57:21.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.405+0000 7ff2b5124640 1 -- 172.21.15.37:0/3078849874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b01e1d50 con 0x7ff2b00ce510 2024-04-25T08:57:21.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.409+0000 7ff2b5124640 1 -- 172.21.15.37:0/3078849874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b004efa0 con 0x7ff2b00ce510 2024-04-25T08:57:21.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.409+0000 7ff2a77fe640 1 -- 172.21.15.37:0/3078849874 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff2a00c2060 con 0x7ff2b00ce510 2024-04-25T08:57:21.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.409+0000 7ff2a77fe640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 0x7ff290069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.409+0000 7ff2a77fe640 1 -- 172.21.15.37:0/3078849874 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff2a01223c0 con 0x7ff2b00ce510 2024-04-25T08:57:21.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.409+0000 7ff2a7fff640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 0x7ff290069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.409+0000 7ff2a7fff640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 0x7ff290069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff298000f70 tx=0x7ff298007a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.413+0000 7ff2a77fe640 1 -- 172.21.15.37:0/3078849874 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff2a0018870 con 0x7ff2b00ce510 2024-04-25T08:57:21.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.577+0000 7ff2b5124640 1 -- 172.21.15.37:0/3078849874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2b01e3d70 con 0x7ff2b00ce510 2024-04-25T08:57:21.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.577+0000 7ff2a77fe640 1 -- 172.21.15.37:0/3078849874 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17038 (secure 0 0 0) 0x7ff2a00f31f0 con 0x7ff2b00ce510 2024-04-25T08:57:21.570 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:57:21.570 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":83,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:57:11.180680+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:56:49.737591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:56:50.436964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/4130042812":"2024-04-26T08:57:11.180643+0000","172.21.15.37:6834/4130042812":"2024-04-26T08:57:11.180643+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 -- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 msgr2=0x7ff290069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 0x7ff290069420 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff298000f70 tx=0x7ff298007a10 comp rx=0 tx=0).stop 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 -- 172.21.15.37:0/3078849874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00ce510 msgr2=0x7ff2b01e5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00ce510 0x7ff2b01e5f90 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7ff2a00b3040 tx=0x7ff2a001bab0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 -- 172.21.15.37:0/3078849874 shutdown_connections 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 0x7ff290069420 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2b01451f0 0x7ff2b01e3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2b0144ca0 0x7ff2b01e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 --2- 172.21.15.37:0/3078849874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b00ce510 0x7ff2b01e5f90 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 -- 172.21.15.37:0/3078849874 >> 172.21.15.37:0/3078849874 conn(0x7ff2b00b11a0 msgr2=0x7ff2b00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 -- 172.21.15.37:0/3078849874 shutdown_connections 2024-04-25T08:57:21.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.585+0000 7ff2b5124640 1 -- 172.21.15.37:0/3078849874 wait complete. 2024-04-25T08:57:21.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:56:49.737591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:56:50.436964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:57:21.591 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:57:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a98b38640 1 Processor -- start 2024-04-25T08:57:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a98b38640 1 -- start start 2024-04-25T08:57:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a98b38640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9a9415b6b0 0x7f9a940ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a98b38640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9a940cf590 0x7f9a940cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a98b38640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a9415f130 0x7f9a9415f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a98b38640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9415ba90 con 0x7f9a940cf590 2024-04-25T08:57:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a98b38640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a940cbd20 con 0x7f9a9415b6b0 2024-04-25T08:57:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a98b38640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940cc000 con 0x7f9a9415f130 2024-04-25T08:57:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a91d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9a940cf590 0x7f9a940cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a91d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9a940cf590 0x7f9a940cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50064/0 (socket says 172.21.15.37:50064) 2024-04-25T08:57:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a91d9b640 1 -- 172.21.15.37:0/3382143670 learned_addr learned my addr 172.21.15.37:0/3382143670 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a92d9d640 1 --2- 172.21.15.37:0/3382143670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a9415f130 0x7f9a9415f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a9259c640 1 --2- 172.21.15.37:0/3382143670 >> v2:172.21.15.203:3300/0 conn(0x7f9a9415b6b0 0x7f9a940ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a91d9b640 1 -- 172.21.15.37:0/3382143670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a9415f130 msgr2=0x7f9a9415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a91d9b640 1 --2- 172.21.15.37:0/3382143670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a9415f130 0x7f9a9415f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a91d9b640 1 -- 172.21.15.37:0/3382143670 >> v2:172.21.15.203:3300/0 conn(0x7f9a9415b6b0 msgr2=0x7f9a940ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a91d9b640 1 --2- 172.21.15.37:0/3382143670 >> v2:172.21.15.203:3300/0 conn(0x7f9a9415b6b0 0x7f9a940ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a91d9b640 1 -- 172.21.15.37:0/3382143670 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a940cc2e0 con 0x7f9a940cf590 2024-04-25T08:57:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.729+0000 7f9a9259c640 1 --2- 172.21.15.37:0/3382143670 >> v2:172.21.15.203:3300/0 conn(0x7f9a9415b6b0 0x7f9a940ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a91d9b640 1 --2- 172.21.15.37:0/3382143670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a940cf590 0x7f9a940cfa20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9a940d0180 tx=0x7f9a7c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b399c9a29ca535e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a9159a640 1 -- 172.21.15.37:0/3382143670 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c0b1070 con 0x7f9a940cf590 2024-04-25T08:57:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a9159a640 1 -- 172.21.15.37:0/3382143670 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c0ac630 con 0x7f9a940cf590 2024-04-25T08:57:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a9159a640 1 -- 172.21.15.37:0/3382143670 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c0a8050 con 0x7f9a940cf590 2024-04-25T08:57:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a98b38640 1 -- 172.21.15.37:0/3382143670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a940cf590 msgr2=0x7f9a940cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a98b38640 1 --2- 172.21.15.37:0/3382143670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a940cf590 0x7f9a940cfa20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f9a940d0180 tx=0x7f9a7c01c520 comp rx=0 tx=0).stop 2024-04-25T08:57:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a98b38640 1 -- 172.21.15.37:0/3382143670 shutdown_connections 2024-04-25T08:57:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a98b38640 1 --2- 172.21.15.37:0/3382143670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a940cf590 0x7f9a940cfa20 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a98b38640 1 --2- 172.21.15.37:0/3382143670 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a9415f130 0x7f9a9415f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a98b38640 1 --2- 172.21.15.37:0/3382143670 >> v2:172.21.15.203:3300/0 conn(0x7f9a9415b6b0 0x7f9a940ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a98b38640 1 -- 172.21.15.37:0/3382143670 >> 172.21.15.37:0/3382143670 conn(0x7f9a9406ae60 msgr2=0x7f9a9406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a98b38640 1 -- 172.21.15.37:0/3382143670 shutdown_connections 2024-04-25T08:57:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.733+0000 7f9a98b38640 1 -- 172.21.15.37:0/3382143670 wait complete. 2024-04-25T08:57:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a98b38640 1 Processor -- start 2024-04-25T08:57:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a98b38640 1 -- start start 2024-04-25T08:57:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a98b38640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a940cf590 0x7f9a94098330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a98b38640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a9415b6b0 0x7f9a940acb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a98b38640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a9415f130 0x7f9a9415c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a98b38640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940ca900 con 0x7f9a9415b6b0 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a98b38640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9415fef0 con 0x7f9a940cf590 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a98b38640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a940cadb0 con 0x7f9a9415f130 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a91d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a9415b6b0 0x7f9a940acb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a91d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a9415b6b0 0x7f9a940acb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50070/0 (socket says 172.21.15.37:50070) 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a91d9b640 1 -- 172.21.15.37:0/313881155 learned_addr learned my addr 172.21.15.37:0/313881155 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a91d9b640 1 -- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a9415f130 msgr2=0x7f9a9415c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a92d9d640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a9415f130 0x7f9a9415c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a9259c640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a940cf590 0x7f9a94098330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a91d9b640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a9415f130 0x7f9a9415c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a91d9b640 1 -- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a940cf590 msgr2=0x7f9a94098330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a91d9b640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a940cf590 0x7f9a94098330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a91d9b640 1 -- 172.21.15.37:0/313881155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a94159190 con 0x7f9a9415b6b0 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a91d9b640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a9415b6b0 0x7f9a940acb80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0af040 tx=0x7f9a7c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a8affd640 1 -- 172.21.15.37:0/313881155 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c0bc070 con 0x7f9a9415b6b0 2024-04-25T08:57:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a8affd640 1 -- 172.21.15.37:0/313881155 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c0ab5e0 con 0x7f9a9415b6b0 2024-04-25T08:57:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a98b38640 1 -- 172.21.15.37:0/313881155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a94159400 con 0x7f9a9415b6b0 2024-04-25T08:57:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a8affd640 1 -- 172.21.15.37:0/313881155 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c0b1070 con 0x7f9a9415b6b0 2024-04-25T08:57:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a98b38640 1 -- 172.21.15.37:0/313881155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a941599c0 con 0x7f9a9415b6b0 2024-04-25T08:57:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a98b38640 1 -- 172.21.15.37:0/313881155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9404efa0 con 0x7f9a9415b6b0 2024-04-25T08:57:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.737+0000 7f9a8affd640 1 -- 172.21.15.37:0/313881155 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9a7c0bf070 con 0x7f9a9415b6b0 2024-04-25T08:57:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.741+0000 7f9a8affd640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a74066f60 0x7f9a74069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.741+0000 7f9a8affd640 1 -- 172.21.15.37:0/313881155 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9a7c11eb90 con 0x7f9a9415b6b0 2024-04-25T08:57:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.741+0000 7f9a9259c640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a74066f60 0x7f9a74069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.741+0000 7f9a9259c640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a74066f60 0x7f9a74069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9a940b2910 tx=0x7f9a8c00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.745+0000 7f9a8affd640 1 -- 172.21.15.37:0/313881155 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9a7c016110 con 0x7f9a9415b6b0 2024-04-25T08:57:21.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.905+0000 7f9a98b38640 1 -- 172.21.15.37:0/313881155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9a940713f0 con 0x7f9a9415b6b0 2024-04-25T08:57:21.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.905+0000 7f9a8affd640 1 -- 172.21.15.37:0/313881155 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9a7c0ef920 con 0x7f9a9415b6b0 2024-04-25T08:57:21.897 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:57:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 -- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a74066f60 msgr2=0x7f9a74069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a74066f60 0x7f9a74069420 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9a940b2910 tx=0x7f9a8c00b040 comp rx=0 tx=0).stop 2024-04-25T08:57:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 -- 172.21.15.37:0/313881155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a9415b6b0 msgr2=0x7f9a940acb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a9415b6b0 0x7f9a940acb80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c0af040 tx=0x7f9a7c0192c0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 -- 172.21.15.37:0/313881155 shutdown_connections 2024-04-25T08:57:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9a74066f60 0x7f9a74069420 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9a9415f130 0x7f9a9415c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a9415b6b0 0x7f9a940acb80 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 --2- 172.21.15.37:0/313881155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9a940cf590 0x7f9a94098330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 -- 172.21.15.37:0/313881155 >> 172.21.15.37:0/313881155 conn(0x7f9a9406ae60 msgr2=0x7f9a940b3690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 -- 172.21.15.37:0/313881155 shutdown_connections 2024-04-25T08:57:21.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:21.909+0000 7f9a98b38640 1 -- 172.21.15.37:0/313881155 wait complete. 2024-04-25T08:57:21.918 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:57:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b7399640 1 Processor -- start 2024-04-25T08:57:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b7399640 1 -- start start 2024-04-25T08:57:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b7399640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff9b00cfa20 0x7ff9b0072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b7399640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b00cbd20 con 0x7ff9b00cf590 2024-04-25T08:57:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b7399640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b00cc000 con 0x7ff9b015b6b0 2024-04-25T08:57:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b7399640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b015ba90 con 0x7ff9b00cfa20 2024-04-25T08:57:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b4934640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff9b00cf590 0x7ff9b0072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36474/0 (socket says 172.21.15.37:36474) 2024-04-25T08:57:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b4934640 1 -- 172.21.15.37:0/1620675458 learned_addr learned my addr 172.21.15.37:0/1620675458 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b5936640 1 --2- 172.21.15.37:0/1620675458 >> v2:172.21.15.203:3300/0 conn(0x7ff9b00cfa20 0x7ff9b0072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276086488 0 0) 0x7ff9b00cbd20 con 0x7ff9b00cf590 2024-04-25T08:57:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff99c003530 con 0x7ff9b00cf590 2024-04-25T08:57:22.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1566247655 0 0) 0x7ff99c003530 con 0x7ff9b00cf590 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 >> v2:172.21.15.203:3300/0 conn(0x7ff9b00cfa20 msgr2=0x7ff9b0072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 --2- 172.21.15.37:0/1620675458 >> v2:172.21.15.203:3300/0 conn(0x7ff9b00cfa20 0x7ff9b0072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 >> v1:172.21.15.203:6789/0 conn(0x7ff9b015b6b0 legacy=0x7ff9b00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00cc2e0 con 0x7ff9b00cf590 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7ff9ac001680 con 0x7ff9b00cf590 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 >> v1:172.21.15.37:6789/0 conn(0x7ff9b00cf590 legacy=0x7ff9b0072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff99c003a30 0x7ff99c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99c004810 0x7ff99c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99c0051d0 0x7ff99c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b015ba90 con 0x7ff99c0051d0 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c00c100 con 0x7ff99c004810 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff99c00c300 con 0x7ff99c003a30 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b5936640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99c0051d0 0x7ff99c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b4934640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99c004810 0x7ff99c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b5135640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff99c003a30 0x7ff99c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b5936640 1 -- 172.21.15.37:0/1620675458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff99c003a30 msgr2=0x7ff99c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b5936640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff99c003a30 0x7ff99c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b5936640 1 -- 172.21.15.37:0/1620675458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99c004810 msgr2=0x7ff99c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b5936640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99c004810 0x7ff99c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b5936640 1 -- 172.21.15.37:0/1620675458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c003530 con 0x7ff99c0051d0 2024-04-25T08:57:22.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.053+0000 7ff9b4934640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff99c004810 0x7ff99c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9b5936640 1 -- 172.21.15.37:0/1620675458 reap_dead start 2024-04-25T08:57:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9b5936640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99c0051d0 0x7ff99c00b9d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009fc0 tx=0x7ff9a801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532bdb38fbdce563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8009c10 con 0x7ff99c0051d0 2024-04-25T08:57:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a80a5d10 con 0x7ff99c0051d0 2024-04-25T08:57:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9a7fff640 1 -- 172.21.15.37:0/1620675458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8003be0 con 0x7ff99c0051d0 2024-04-25T08:57:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9b7399640 1 -- 172.21.15.37:0/1620675458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99c0051d0 msgr2=0x7ff99c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9b7399640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99c0051d0 0x7ff99c00b9d0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009fc0 tx=0x7ff9a801ccc0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9b7399640 1 -- 172.21.15.37:0/1620675458 shutdown_connections 2024-04-25T08:57:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9b7399640 1 --2- 172.21.15.37:0/1620675458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff99c0051d0 0x7ff99c00b9d0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9b7399640 1 -- 172.21.15.37:0/1620675458 >> 172.21.15.37:0/1620675458 conn(0x7ff9b006ae60 msgr2=0x7ff9b006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:22.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9b7399640 1 -- 172.21.15.37:0/1620675458 shutdown_connections 2024-04-25T08:57:22.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9b7399640 1 -- 172.21.15.37:0/1620675458 wait complete. 2024-04-25T08:57:22.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9b7399640 1 Processor -- start 2024-04-25T08:57:22.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.057+0000 7ff9b7399640 1 -- start start 2024-04-25T08:57:22.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b7399640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff99c0051d0 0x7ff9b0154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b7399640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b00c6430 0x7ff9b015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b7399640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9b015c5f0 0x7ff9b00ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b7399640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00cfe50 con 0x7ff9b00c6430 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b7399640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00cc8a0 con 0x7ff9b015c5f0 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b7399640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00cba30 con 0x7ff99c0051d0 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b4934640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b00c6430 0x7ff9b015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b4934640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b00c6430 0x7ff9b015a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50080/0 (socket says 172.21.15.37:50080) 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b4934640 1 -- 172.21.15.37:0/2848499168 learned_addr learned my addr 172.21.15.37:0/2848499168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b5135640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff99c0051d0 0x7ff9b0154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b5936640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9b015c5f0 0x7ff9b00ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b4934640 1 -- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff99c0051d0 msgr2=0x7ff9b0154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b4934640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff99c0051d0 0x7ff9b0154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b4934640 1 -- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9b015c5f0 msgr2=0x7ff9b00ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b4934640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9b015c5f0 0x7ff9b00ad1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b4934640 1 -- 172.21.15.37:0/2848499168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b0155170 con 0x7ff9b00c6430 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b4934640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b00c6430 0x7ff9b015a5d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00e7b0 tx=0x7ff9ac00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:22.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9a5ffb640 1 -- 172.21.15.37:0/2848499168 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00c6c0 con 0x7ff9b00c6430 2024-04-25T08:57:22.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9a5ffb640 1 -- 172.21.15.37:0/2848499168 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac018070 con 0x7ff9b00c6430 2024-04-25T08:57:22.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9a5ffb640 1 -- 172.21.15.37:0/2848499168 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac00ca60 con 0x7ff9b00c6430 2024-04-25T08:57:22.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b7399640 1 -- 172.21.15.37:0/2848499168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b00c3fe0 con 0x7ff9b00c6430 2024-04-25T08:57:22.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b7399640 1 -- 172.21.15.37:0/2848499168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b00c45a0 con 0x7ff9b00c6430 2024-04-25T08:57:22.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b7399640 1 -- 172.21.15.37:0/2848499168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b004efa0 con 0x7ff9b00c6430 2024-04-25T08:57:22.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9a5ffb640 1 -- 172.21.15.37:0/2848499168 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff9ac00cc00 con 0x7ff9b00c6430 2024-04-25T08:57:22.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9a5ffb640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff994066f60 0x7ff994069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9a5ffb640 1 -- 172.21.15.37:0/2848499168 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff9ac084fd0 con 0x7ff9b00c6430 2024-04-25T08:57:22.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.061+0000 7ff9b5135640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff994066f60 0x7ff994069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.065+0000 7ff9b5135640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff994066f60 0x7ff994069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0000c10 tx=0x7ff9a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:22.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.065+0000 7ff9a5ffb640 1 -- 172.21.15.37:0/2848499168 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff9ac055e90 con 0x7ff9b00c6430 2024-04-25T08:57:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.225+0000 7ff9b7399640 1 -- 172.21.15.37:0/2848499168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9b00ca130 con 0x7ff9b00c6430 2024-04-25T08:57:22.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9a5ffb640 1 -- 172.21.15.37:0/2848499168 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9ac055790 con 0x7ff9b00c6430 2024-04-25T08:57:22.218 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:57:22.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 -- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff994066f60 msgr2=0x7ff994069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff994066f60 0x7ff994069420 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff9a0000c10 tx=0x7ff9a0009040 comp rx=0 tx=0).stop 2024-04-25T08:57:22.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 -- 172.21.15.37:0/2848499168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b00c6430 msgr2=0x7ff9b015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b00c6430 0x7ff9b015a5d0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac00e7b0 tx=0x7ff9ac00ec70 comp rx=0 tx=0).stop 2024-04-25T08:57:22.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 -- 172.21.15.37:0/2848499168 shutdown_connections 2024-04-25T08:57:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff994066f60 0x7ff994069420 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff9b015c5f0 0x7ff9b00ad1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9b00c6430 0x7ff9b015a5d0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 --2- 172.21.15.37:0/2848499168 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff99c0051d0 0x7ff9b0154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 -- 172.21.15.37:0/2848499168 >> 172.21.15.37:0/2848499168 conn(0x7ff9b006ae60 msgr2=0x7ff9b00caf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 -- 172.21.15.37:0/2848499168 shutdown_connections 2024-04-25T08:57:22.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.229+0000 7ff9b7399640 1 -- 172.21.15.37:0/2848499168 wait complete. 2024-04-25T08:57:22.238 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:57:22.339 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T08:57:22.349+0000 7ff670aed640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:57:22.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4e0a6c640 1 Processor -- start 2024-04-25T08:57:22.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4e0a6c640 1 -- start start 2024-04-25T08:57:22.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4e0a6c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe4dc06b820 0x7fe4dc0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4e0a6c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4dc0afb20 con 0x7fe4dc06b440 2024-04-25T08:57:22.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4e0a6c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4dc0afd90 con 0x7fe4dc0b1ac0 2024-04-25T08:57:22.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4e0a6c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4dc0b06c0 con 0x7fe4dc06b820 2024-04-25T08:57:22.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4da59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe4dc06b440 0x7fe4dc0b13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36486/0 (socket says 172.21.15.37:36486) 2024-04-25T08:57:22.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4da59c640 1 -- 172.21.15.37:0/7150958 learned_addr learned my addr 172.21.15.37:0/7150958 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:22.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4d9d9b640 1 --2- 172.21.15.37:0/7150958 >> v2:172.21.15.37:3300/0 conn(0x7fe4dc06b820 0x7fe4dc0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4d9d9b640 1 -- 172.21.15.37:0/7150958 >> v1:172.21.15.203:6789/0 conn(0x7fe4dc0b1ac0 legacy=0x7fe4dc0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:22.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4d9d9b640 1 -- 172.21.15.37:0/7150958 >> v1:172.21.15.37:6789/0 conn(0x7fe4dc06b440 legacy=0x7fe4dc0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4d959a640 1 -- 172.21.15.37:0/7150958 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669187522 0 0) 0x7fe4dc0afb20 con 0x7fe4dc06b440 2024-04-25T08:57:22.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.385+0000 7fe4d9d9b640 1 -- 172.21.15.37:0/7150958 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc0b0070 con 0x7fe4dc06b820 2024-04-25T08:57:22.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4d9d9b640 1 --2- 172.21.15.37:0/7150958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc06b820 0x7fe4dc0b0b50 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00097c0 tx=0x7fe4d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821bda395cba907a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:22.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4d959a640 1 -- 172.21.15.37:0/7150958 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00b0070 con 0x7fe4dc06b820 2024-04-25T08:57:22.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4d959a640 1 -- 172.21.15.37:0/7150958 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d00ab600 con 0x7fe4dc06b820 2024-04-25T08:57:22.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4d959a640 1 -- 172.21.15.37:0/7150958 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d00ab9d0 con 0x7fe4dc06b820 2024-04-25T08:57:22.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/7150958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc06b820 msgr2=0x7fe4dc0b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 --2- 172.21.15.37:0/7150958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc06b820 0x7fe4dc0b0b50 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00097c0 tx=0x7fe4d001c460 comp rx=0 tx=0).stop 2024-04-25T08:57:22.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/7150958 shutdown_connections 2024-04-25T08:57:22.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 --2- 172.21.15.37:0/7150958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc06b820 0x7fe4dc0b0b50 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/7150958 >> 172.21.15.37:0/7150958 conn(0x7fe4dc0b1f10 msgr2=0x7fe4dc0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:22.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/7150958 shutdown_connections 2024-04-25T08:57:22.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/7150958 wait complete. 2024-04-25T08:57:22.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 Processor -- start 2024-04-25T08:57:22.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 -- start start 2024-04-25T08:57:22.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc06b440 0x7fe4dc1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4dc06b820 0x7fe4dc0c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4dc0b1ac0 0x7fe4dc0ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc0cd1f0 con 0x7fe4dc06b440 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc0cd070 con 0x7fe4dc0b1ac0 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4e0a6c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc16b980 con 0x7fe4dc06b820 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4da59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc06b440 0x7fe4dc1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4da59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc06b440 0x7fe4dc1580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50094/0 (socket says 172.21.15.37:50094) 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4da59c640 1 -- 172.21.15.37:0/885584699 learned_addr learned my addr 172.21.15.37:0/885584699 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4dad9d640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4dc0b1ac0 0x7fe4dc0ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4d9d9b640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4dc06b820 0x7fe4dc0c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4da59c640 1 -- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4dc06b820 msgr2=0x7fe4dc0c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4da59c640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4dc06b820 0x7fe4dc0c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4da59c640 1 -- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4dc0b1ac0 msgr2=0x7fe4dc0ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4da59c640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4dc0b1ac0 0x7fe4dc0ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4da59c640 1 -- 172.21.15.37:0/885584699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc0c3e10 con 0x7fe4dc06b440 2024-04-25T08:57:22.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4dad9d640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4dc0b1ac0 0x7fe4dc0ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:22.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4d9d9b640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4dc06b820 0x7fe4dc0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:22.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.389+0000 7fe4da59c640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc06b440 0x7fe4dc1580e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0011f0 tx=0x7fe4cc00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:22.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.393+0000 7fe4caffd640 1 -- 172.21.15.37:0/885584699 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc018070 con 0x7fe4dc06b440 2024-04-25T08:57:22.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.393+0000 7fe4caffd640 1 -- 172.21.15.37:0/885584699 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc0074e0 con 0x7fe4dc06b440 2024-04-25T08:57:22.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.393+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/885584699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc0c40f0 con 0x7fe4dc06b440 2024-04-25T08:57:22.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.393+0000 7fe4caffd640 1 -- 172.21.15.37:0/885584699 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc004a10 con 0x7fe4dc06b440 2024-04-25T08:57:22.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.393+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/885584699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc0c46b0 con 0x7fe4dc06b440 2024-04-25T08:57:22.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.393+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/885584699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4dc04efa0 con 0x7fe4dc06b440 2024-04-25T08:57:22.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.393+0000 7fe4caffd640 1 -- 172.21.15.37:0/885584699 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe4cc01b070 con 0x7fe4dc06b440 2024-04-25T08:57:22.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.393+0000 7fe4caffd640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4bc066f60 0x7fe4bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.393+0000 7fe4caffd640 1 -- 172.21.15.37:0/885584699 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe4cc084530 con 0x7fe4dc06b440 2024-04-25T08:57:22.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.393+0000 7fe4d9d9b640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4bc066f60 0x7fe4bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.393+0000 7fe4d9d9b640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4bc066f60 0x7fe4bc069420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00ad000 tx=0x7fe4d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:22.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.397+0000 7fe4caffd640 1 -- 172.21.15.37:0/885584699 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe4cc055360 con 0x7fe4dc06b440 2024-04-25T08:57:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/885584699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4dc0b6380 con 0x7fe4dc06b440 2024-04-25T08:57:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4caffd640 1 -- 172.21.15.37:0/885584699 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4cc054c60 con 0x7fe4dc06b440 2024-04-25T08:57:22.556 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:57:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4bc066f60 msgr2=0x7fe4bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4bc066f60 0x7fe4bc069420 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe4d00ad000 tx=0x7fe4d0008040 comp rx=0 tx=0).stop 2024-04-25T08:57:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/885584699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc06b440 msgr2=0x7fe4dc1580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc06b440 0x7fe4dc1580e0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc0011f0 tx=0x7fe4cc00c9b0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/885584699 shutdown_connections 2024-04-25T08:57:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe4bc066f60 0x7fe4bc069420 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4dc0b1ac0 0x7fe4dc0ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4dc06b820 0x7fe4dc0c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 --2- 172.21.15.37:0/885584699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4dc06b440 0x7fe4dc1580e0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/885584699 >> 172.21.15.37:0/885584699 conn(0x7fe4dc0b1f10 msgr2=0x7fe4dc15cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:22.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/885584699 shutdown_connections 2024-04-25T08:57:22.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.565+0000 7fe4e0a6c640 1 -- 172.21.15.37:0/885584699 wait complete. 2024-04-25T08:57:22.575 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:57:22.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342892c640 1 Processor -- start 2024-04-25T08:57:22.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342892c640 1 -- start start 2024-04-25T08:57:22.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342892c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34240af280 0x7f342415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342892c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f342406c3b0 0x7f342406df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342892c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3424066c30 con 0x7f34240b2c30 2024-04-25T08:57:22.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342892c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342406daa0 con 0x7f342406c3b0 2024-04-25T08:57:22.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342892c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34240af660 con 0x7f34240af280 2024-04-25T08:57:22.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f3421d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f34240b2c30 0x7f342406bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:36492/0 (socket says 172.21.15.37:36492) 2024-04-25T08:57:22.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f3421d9b640 1 -- 172.21.15.37:0/4176494135 learned_addr learned my addr 172.21.15.37:0/4176494135 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:22.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342259c640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34240af280 0x7f342415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f3422d9d640 1 --2- 172.21.15.37:0/4176494135 >> v2:172.21.15.203:3300/0 conn(0x7f342406c3b0 0x7f342406df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675125780 0 0) 0x7f3424066c30 con 0x7f34240b2c30 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f340c003530 con 0x7f34240b2c30 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1823784737 0 0) 0x7f340c003530 con 0x7f34240b2c30 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34240af280 msgr2=0x7f342415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34240af280 0x7f342415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 >> v2:172.21.15.203:3300/0 conn(0x7f342406c3b0 msgr2=0x7f342406df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 --2- 172.21.15.37:0/4176494135 >> v2:172.21.15.203:3300/0 conn(0x7f342406c3b0 0x7f342406df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342415e640 con 0x7f34240b2c30 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3418001680 con 0x7f34240b2c30 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 >> v1:172.21.15.37:6789/0 conn(0x7f34240b2c30 legacy=0x7f342406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c003a30 0x7f340c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34240af280 0x7f340c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f340c004d50 0x7f340c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34240af660 con 0x7f340c003a30 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c00b9d0 con 0x7f340c004d50 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c00bb50 con 0x7f34240af280 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342259c640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c003a30 0x7f340c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f3422d9d640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f340c004d50 0x7f340c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f3421d9b640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34240af280 0x7f340c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342259c640 1 -- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34240af280 msgr2=0x7f340c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342259c640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34240af280 0x7f340c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342259c640 1 -- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f340c004d50 msgr2=0x7f340c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342259c640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f340c004d50 0x7f340c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f342259c640 1 -- 172.21.15.37:0/4176494135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c003530 con 0x7f340c003a30 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f3422d9d640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f340c004d50 0x7f340c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.709+0000 7f3421d9b640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34240af280 0x7f340c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342259c640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c003a30 0x7f340c003e50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f3414009ae0 tx=0x7f341401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a072687a6c8b5308 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34140b2070 con 0x7f340c003a30 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34140a5530 con 0x7f340c003a30 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342159a640 1 -- 172.21.15.37:0/4176494135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34140096b0 con 0x7f340c003a30 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342892c640 1 -- 172.21.15.37:0/4176494135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c003a30 msgr2=0x7f340c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342892c640 1 --2- 172.21.15.37:0/4176494135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f340c003a30 0x7f340c003e50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f3414009ae0 tx=0x7f341401ca70 comp rx=0 tx=0).stop 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f3422d9d640 1 -- 172.21.15.37:0/4176494135 reap_dead start 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342892c640 1 -- 172.21.15.37:0/4176494135 shutdown_connections 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342892c640 1 -- 172.21.15.37:0/4176494135 >> 172.21.15.37:0/4176494135 conn(0x7f3424071810 msgr2=0x7f3424071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342892c640 1 -- 172.21.15.37:0/4176494135 shutdown_connections 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342892c640 1 -- 172.21.15.37:0/4176494135 wait complete. 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342892c640 1 Processor -- start 2024-04-25T08:57:22.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.713+0000 7f342892c640 1 -- start start 2024-04-25T08:57:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f342892c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f340c003a30 0x7f34241564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f342892c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424152a10 0x7f34241612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f342892c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3424146ed0 0x7f3424157d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f342892c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342406ee00 con 0x7f3424152a10 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f342892c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f342406ec80 con 0x7f340c003a30 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f342892c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f342406ef80 con 0x7f3424146ed0 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3421d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424152a10 0x7f34241612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3421d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424152a10 0x7f34241612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50098/0 (socket says 172.21.15.37:50098) 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3421d9b640 1 -- 172.21.15.37:0/1073570706 learned_addr learned my addr 172.21.15.37:0/1073570706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f342259c640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f340c003a30 0x7f34241564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3422d9d640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3424146ed0 0x7f3424157d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3421d9b640 1 -- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3424146ed0 msgr2=0x7f3424157d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3421d9b640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3424146ed0 0x7f3424157d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3421d9b640 1 -- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f340c003a30 msgr2=0x7f34241564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3421d9b640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f340c003a30 0x7f34241564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3421d9b640 1 -- 172.21.15.37:0/1073570706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3424156cf0 con 0x7f3424152a10 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3421d9b640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424152a10 0x7f34241612b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f341800d810 tx=0x7f341800dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3412ffd640 1 -- 172.21.15.37:0/1073570706 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418019070 con 0x7f3424152a10 2024-04-25T08:57:22.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3412ffd640 1 -- 172.21.15.37:0/1073570706 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34180074e0 con 0x7f3424152a10 2024-04-25T08:57:22.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3412ffd640 1 -- 172.21.15.37:0/1073570706 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418004a10 con 0x7f3424152a10 2024-04-25T08:57:22.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f342892c640 1 -- 172.21.15.37:0/1073570706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3424156fd0 con 0x7f3424152a10 2024-04-25T08:57:22.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f342892c640 1 -- 172.21.15.37:0/1073570706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3424157590 con 0x7f3424152a10 2024-04-25T08:57:22.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f342892c640 1 -- 172.21.15.37:0/1073570706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342404efa0 con 0x7f3424152a10 2024-04-25T08:57:22.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3412ffd640 1 -- 172.21.15.37:0/1073570706 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3418004bb0 con 0x7f3424152a10 2024-04-25T08:57:22.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3412ffd640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3404066f60 0x7f3404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:22.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.717+0000 7f3412ffd640 1 -- 172.21.15.37:0/1073570706 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3418084a60 con 0x7f3424152a10 2024-04-25T08:57:22.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.721+0000 7f342259c640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3404066f60 0x7f3404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:22.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.721+0000 7f342259c640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3404066f60 0x7f3404069420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f34140af000 tx=0x7f34140046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:22.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.725+0000 7f3412ffd640 1 -- 172.21.15.37:0/1073570706 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3418055230 con 0x7f3424152a10 2024-04-25T08:57:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 -- 172.21.15.37:0/1073570706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f34240aca80 con 0x7f3424152a10 2024-04-25T08:57:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f3412ffd640 1 -- 172.21.15.37:0/1073570706 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v83) v1 ==== 80+0+122 (secure 0 0 0) 0x7f34180592d0 con 0x7f3424152a10 2024-04-25T08:57:22.888 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/4130042812 2024-04-26T08:57:11.180643+0000 2024-04-25T08:57:22.888 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6834/4130042812 2024-04-26T08:57:11.180643+0000 2024-04-25T08:57:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:listed 2 entries 2024-04-25T08:57:22.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 -- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3404066f60 msgr2=0x7f3404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3404066f60 0x7f3404069420 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f34140af000 tx=0x7f34140046d0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 -- 172.21.15.37:0/1073570706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424152a10 msgr2=0x7f34241612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:22.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424152a10 0x7f34241612b0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f341800d810 tx=0x7f341800dcd0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 -- 172.21.15.37:0/1073570706 shutdown_connections 2024-04-25T08:57:22.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3404066f60 0x7f3404069420 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3424146ed0 0x7f3424157d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3424152a10 0x7f34241612b0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 --2- 172.21.15.37:0/1073570706 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f340c003a30 0x7f34241564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:22.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 -- 172.21.15.37:0/1073570706 >> 172.21.15.37:0/1073570706 conn(0x7f3424071810 msgr2=0x7f34240b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:22.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 -- 172.21.15.37:0/1073570706 shutdown_connections 2024-04-25T08:57:22.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:22.897+0000 7f342892c640 1 -- 172.21.15.37:0/1073570706 wait complete. 2024-04-25T08:57:22.906 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T08:57:22.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:22.907 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:22.918 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[45642]: fuse finished with error 0 and tester_r 0 2024-04-25T08:57:22.920 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:57:25.785 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:57:25.788 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:57:29.021 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:57:29.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:29.022 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:57:29.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:29.030 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-04-25T08:57:29.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31c707640 1 Processor -- start 2024-04-25T08:57:29.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31c707640 1 -- start start 2024-04-25T08:57:29.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31c707640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe31415b350 0x7fe31415ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31c707640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3140cc450 con 0x7fe314066bb0 2024-04-25T08:57:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31c707640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3140b31b0 con 0x7fe31406c3b0 2024-04-25T08:57:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31c707640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3140b2ed0 con 0x7fe31415b350 2024-04-25T08:57:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31a4a3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe31415b350 0x7fe31415ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31a4a3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe31415b350 0x7fe31415ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50102/0 (socket says 172.21.15.37:50102) 2024-04-25T08:57:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31a4a3640 1 -- 172.21.15.37:0/3156895999 learned_addr learned my addr 172.21.15.37:0/3156895999 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31a4a3640 1 -- 172.21.15.37:0/3156895999 >> v1:172.21.15.203:6789/0 conn(0x7fe31406c3b0 legacy=0x7fe31406d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31a4a3640 1 -- 172.21.15.37:0/3156895999 >> v1:172.21.15.37:6789/0 conn(0x7fe314066bb0 legacy=0x7fe31406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.225+0000 7fe31a4a3640 1 -- 172.21.15.37:0/3156895999 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31415df40 con 0x7fe31415b350 2024-04-25T08:57:29.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/3156895999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe31415b350 0x7fe31415ea20 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe310009870 tx=0x7fe31001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db8c5e424ad81b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe3194a1640 1 -- 172.21.15.37:0/3156895999 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3100aa650 con 0x7fe31415b350 2024-04-25T08:57:29.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe3194a1640 1 -- 172.21.15.37:0/3156895999 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3100af070 con 0x7fe31415b350 2024-04-25T08:57:29.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe3194a1640 1 -- 172.21.15.37:0/3156895999 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3100aaa10 con 0x7fe31415b350 2024-04-25T08:57:29.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 -- 172.21.15.37:0/3156895999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe31415b350 msgr2=0x7fe31415ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 --2- 172.21.15.37:0/3156895999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe31415b350 0x7fe31415ea20 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fe310009870 tx=0x7fe31001c4f0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 -- 172.21.15.37:0/3156895999 shutdown_connections 2024-04-25T08:57:29.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 --2- 172.21.15.37:0/3156895999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe31415b350 0x7fe31415ea20 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 -- 172.21.15.37:0/3156895999 >> 172.21.15.37:0/3156895999 conn(0x7fe3140717f0 msgr2=0x7fe314071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 -- 172.21.15.37:0/3156895999 shutdown_connections 2024-04-25T08:57:29.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 -- 172.21.15.37:0/3156895999 wait complete. 2024-04-25T08:57:29.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 Processor -- start 2024-04-25T08:57:29.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 -- start start 2024-04-25T08:57:29.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe314066bb0 0x7fe31414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe31406c3b0 0x7fe3140ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31415b350 0x7fe3140ce580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31415b8b0 con 0x7fe31406c3b0 2024-04-25T08:57:29.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe31415b730 con 0x7fe31415b350 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.229+0000 7fe31c707640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe314072940 con 0x7fe314066bb0 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe319ca2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe31406c3b0 0x7fe3140ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe319ca2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe31406c3b0 0x7fe3140ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50112/0 (socket says 172.21.15.37:50112) 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe319ca2640 1 -- 172.21.15.37:0/1990662285 learned_addr learned my addr 172.21.15.37:0/1990662285 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe314066bb0 0x7fe31414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe31aca4640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31415b350 0x7fe3140ce580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe319ca2640 1 -- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe314066bb0 msgr2=0x7fe31414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe319ca2640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe314066bb0 0x7fe31414fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe319ca2640 1 -- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31415b350 msgr2=0x7fe3140ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe319ca2640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31415b350 0x7fe3140ce580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe319ca2640 1 -- 172.21.15.37:0/1990662285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31414e5a0 con 0x7fe31406c3b0 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe319ca2640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe31406c3b0 0x7fe3140ce040 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c420 tx=0x7fe30400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe30affd640 1 -- 172.21.15.37:0/1990662285 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3040052d0 con 0x7fe31406c3b0 2024-04-25T08:57:29.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe30affd640 1 -- 172.21.15.37:0/1990662285 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe304005470 con 0x7fe31406c3b0 2024-04-25T08:57:29.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe30affd640 1 -- 172.21.15.37:0/1990662285 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304007260 con 0x7fe31406c3b0 2024-04-25T08:57:29.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe31c707640 1 -- 172.21.15.37:0/1990662285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe31414e810 con 0x7fe31406c3b0 2024-04-25T08:57:29.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe31c707640 1 -- 172.21.15.37:0/1990662285 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe31414edd0 con 0x7fe31406c3b0 2024-04-25T08:57:29.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe30affd640 1 -- 172.21.15.37:0/1990662285 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe304007400 con 0x7fe31406c3b0 2024-04-25T08:57:29.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe30affd640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f8066f60 0x7fe2f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe30affd640 1 -- 172.21.15.37:0/1990662285 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe3040846b0 con 0x7fe31406c3b0 2024-04-25T08:57:29.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.233+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f8066f60 0x7fe2f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 Processor -- start 2024-04-25T08:57:29.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 -- start start 2024-04-25T08:57:29.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f8066f60 0x7fe2f8069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe3100ae000 tx=0x7fe3100a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82c8bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82c41110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82c41a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f82a8fdd0 con 0x560f82c8b8c0 2024-04-25T08:57:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f82a18010 con 0x560f82c8c1e0 2024-04-25T08:57:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f82a8be70 con 0x560f82c41650 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31a4a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82c8bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31a4a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82c8bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50124/0 (socket says 172.21.15.37:50124) 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31a4a3640 1 -- 172.21.15.37:0/139030431 learned_addr learned my addr 172.21.15.37:0/139030431 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe319ca2640 1 --2- 172.21.15.37:0/139030431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82c41110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31aca4640 1 --2- 172.21.15.37:0/139030431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82c41a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31a4a3640 1 -- 172.21.15.37:0/139030431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 msgr2=0x560f82c41a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/139030431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82c41a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31a4a3640 1 -- 172.21.15.37:0/139030431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 msgr2=0x560f82c41110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/139030431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82c41110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31a4a3640 1 -- 172.21.15.37:0/139030431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f82a489c0 con 0x560f82c8b8c0 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/139030431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82c8bca0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe310000c10 tx=0x7fe310004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bf9fa20217672f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe2f7fff640 1 -- 172.21.15.37:0/139030431 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3100b9070 con 0x560f82c8b8c0 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe2f7fff640 1 -- 172.21.15.37:0/139030431 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3100af040 con 0x560f82c8b8c0 2024-04-25T08:57:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe2f7fff640 1 -- 172.21.15.37:0/139030431 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3100be8d0 con 0x560f82c8b8c0 2024-04-25T08:57:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 -- 172.21.15.37:0/139030431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 msgr2=0x560f82c8bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 --2- 172.21.15.37:0/139030431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82c8bca0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fe310000c10 tx=0x7fe310004740 comp rx=0 tx=0).stop 2024-04-25T08:57:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 -- 172.21.15.37:0/139030431 shutdown_connections 2024-04-25T08:57:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 --2- 172.21.15.37:0/139030431 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82c41a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 --2- 172.21.15.37:0/139030431 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82c41110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 --2- 172.21.15.37:0/139030431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82c8bca0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 -- 172.21.15.37:0/139030431 >> 172.21.15.37:0/139030431 conn(0x560f82b29870 msgr2=0x560f82c3f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 -- 172.21.15.37:0/139030431 shutdown_connections 2024-04-25T08:57:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 -- 172.21.15.37:0/139030431 wait complete. 2024-04-25T08:57:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 -- start start 2024-04-25T08:57:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.237+0000 7fe31f874000 1 Processor -- start 2024-04-25T08:57:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82c45d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8b8c0 0x560f82c462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8c1e0 0x560f82c467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f82a4a420 con 0x560f82c8c1e0 2024-04-25T08:57:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f82b12470 con 0x560f82c8b8c0 2024-04-25T08:57:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f82c4ac20 con 0x560f82c41650 2024-04-25T08:57:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 10 client.0 _ll_register_callbacks cb 0x560f82c2cf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31aca4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8c1e0 0x560f82c467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31aca4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8c1e0 0x560f82c467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50126/0 (socket says 172.21.15.37:50126) 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31aca4640 1 -- 172.21.15.37:0/3861354779 learned_addr learned my addr 172.21.15.37:0/3861354779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82c45d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe319ca2640 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8b8c0 0x560f82c462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31aca4640 1 -- 172.21.15.37:0/3861354779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 msgr2=0x560f82c45d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31aca4640 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82c45d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31aca4640 1 -- 172.21.15.37:0/3861354779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8b8c0 msgr2=0x560f82c462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31aca4640 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8b8c0 0x560f82c462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31aca4640 1 -- 172.21.15.37:0/3861354779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f82a18260 con 0x560f82c8c1e0 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe319ca2640 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8b8c0 0x560f82c462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31aca4640 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8c1e0 0x560f82c467e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe30c010510 tx=0x7fe30c0109d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58702206f1ff5e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe2f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe2f57fa640 1 -- 172.21.15.37:0/3861354779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30c019020 con 0x560f82c8c1e0 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 -- 172.21.15.37:0/3861354779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3100af040 con 0x560f82c8c1e0 2024-04-25T08:57:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe2f57fa640 1 -- 172.21.15.37:0/3861354779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe30c015870 con 0x560f82c8c1e0 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe2f57fa640 1 -- 172.21.15.37:0/3861354779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe30c023050 con 0x560f82c8c1e0 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 10 client.7350 fetch_fsmap learned FSMap version 51 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 -- 172.21.15.37:0/3861354779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x560f82a4ab30 con 0x560f82c8c1e0 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe2f57fa640 1 -- 172.21.15.37:0/3861354779 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3100af040 con 0x560f82c8c1e0 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe2f57fa640 1 -- 172.21.15.37:0/3861354779 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe30c026020 con 0x560f82c8c1e0 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 10 client.7350 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 10 client.7350 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 20 client.7350 populate_metadata read hostname 'smithi037' 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x560f82c9a950 0x560f82c9cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 4 client.7350 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 -- 172.21.15.37:0/3861354779 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f82a2bc80 con 0x560f82c9a950 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x560f82c9a950 0x560f82c9cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x560f82c9a950 0x560f82c9cd40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4072a5994c249c server_cookie=1238c1ec4fa46835 in_seq=0 out_seq=0 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe2f57fa640 10 client.7350 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe2f57fa640 1 -- 172.21.15.37:0/3861354779 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560f82a2bc80 con 0x560f82c9a950 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe2f57fa640 10 client.7350 handle_command_reply: tid=0 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 client.7350 shutdown 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 -- 172.21.15.37:0/3861354779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8c1e0 msgr2=0x560f82c467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8c1e0 0x560f82c467e0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe30c010510 tx=0x7fe30c0109d0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 -- 172.21.15.37:0/3861354779 shutdown_connections 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x560f82c9a950 0x560f82c9cd40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8c1e0 0x560f82c467e0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8b8c0 0x560f82c462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 --2- 172.21.15.37:0/3861354779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82c45d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe2f57fa640 0 client.7350 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 -- 172.21.15.37:0/3861354779 >> 172.21.15.37:0/3861354779 conn(0x560f82b29870 msgr2=0x560f82c3f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 -- 172.21.15.37:0/3861354779 shutdown_connections 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 1 -- 172.21.15.37:0/3861354779 wait complete. 2024-04-25T08:57:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.241+0000 7fe31f874000 20 client.7350 trim_cache size 0 max 16384 2024-04-25T08:57:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe31f874000 1 Processor -- start 2024-04-25T08:57:29.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe31f874000 1 -- start start 2024-04-25T08:57:29.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe31f874000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82ce6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe31f874000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82ce7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe31f874000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82ce7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe31f874000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe314203110 con 0x560f82c8b8c0 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe31f874000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe314202310 con 0x560f82c8c1e0 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe31f874000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe314151a70 con 0x560f82c41650 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe319ca2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82ce7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe319ca2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82ce7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50128/0 (socket says 172.21.15.37:50128) 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe319ca2640 1 -- 172.21.15.37:0/1778894842 learned_addr learned my addr 172.21.15.37:0/1778894842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/1778894842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82ce6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe31aca4640 1 --2- 172.21.15.37:0/1778894842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82ce7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe319ca2640 1 -- 172.21.15.37:0/1778894842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 msgr2=0x560f82ce6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe319ca2640 1 --2- 172.21.15.37:0/1778894842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82ce6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe319ca2640 1 -- 172.21.15.37:0/1778894842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 msgr2=0x560f82ce7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe319ca2640 1 --2- 172.21.15.37:0/1778894842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82ce7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe319ca2640 1 -- 172.21.15.37:0/1778894842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f82b156d0 con 0x560f82c8b8c0 2024-04-25T08:57:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.257+0000 7fe319ca2640 1 --2- 172.21.15.37:0/1778894842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82ce7520 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe30405d050 tx=0x7fe304000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794c481698f21eab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe2f77fe640 1 -- 172.21.15.37:0/1778894842 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304081070 con 0x560f82c8b8c0 2024-04-25T08:57:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe2f77fe640 1 -- 172.21.15.37:0/1778894842 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30405d580 con 0x560f82c8b8c0 2024-04-25T08:57:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe2f77fe640 1 -- 172.21.15.37:0/1778894842 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30406c070 con 0x560f82c8b8c0 2024-04-25T08:57:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- 172.21.15.37:0/1778894842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 msgr2=0x560f82ce7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 --2- 172.21.15.37:0/1778894842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82ce7520 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe30405d050 tx=0x7fe304000ee0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- 172.21.15.37:0/1778894842 shutdown_connections 2024-04-25T08:57:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 --2- 172.21.15.37:0/1778894842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82ce7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 --2- 172.21.15.37:0/1778894842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c8b8c0 0x560f82ce7520 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 --2- 172.21.15.37:0/1778894842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c41650 0x560f82ce6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- 172.21.15.37:0/1778894842 >> 172.21.15.37:0/1778894842 conn(0x560f82b29870 msgr2=0x560f82c41a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- 172.21.15.37:0/1778894842 shutdown_connections 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- 172.21.15.37:0/1778894842 wait complete. 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- start start 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 Processor -- start 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c41650 0x560f82cec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c8b8c0 0x560f82cec620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82ca67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f82a8f0e0 con 0x560f82c41650 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe314152070 con 0x560f82c8c1e0 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x560f82a36ae0 con 0x560f82c8b8c0 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 10 client.0 _ll_register_callbacks cb 0x560f82a2c200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31a4a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c41650 0x560f82cec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31a4a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c41650 0x560f82cec0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50132/0 (socket says 172.21.15.37:50132) 2024-04-25T08:57:29.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31a4a3640 1 -- 172.21.15.37:0/4093990524 learned_addr learned my addr 172.21.15.37:0/4093990524 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe319ca2640 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c8b8c0 0x560f82cec620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31aca4640 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82ca67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31a4a3640 1 -- 172.21.15.37:0/4093990524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c8b8c0 msgr2=0x560f82cec620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c8b8c0 0x560f82cec620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31a4a3640 1 -- 172.21.15.37:0/4093990524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 msgr2=0x560f82ca67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82ca67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31a4a3640 1 -- 172.21.15.37:0/4093990524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f82b12900 con 0x560f82c41650 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31a4a3640 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c41650 0x560f82cec0e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe3100ac040 tx=0x7fe3100046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c424641f51c427a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe2f57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe2f57fa640 1 -- 172.21.15.37:0/4093990524 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3100b9040 con 0x560f82c41650 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- 172.21.15.37:0/4093990524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe30405d580 con 0x560f82c41650 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe2f57fa640 1 -- 172.21.15.37:0/4093990524 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe3100d2610 con 0x560f82c41650 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe2f57fa640 1 -- 172.21.15.37:0/4093990524 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3100e1050 con 0x560f82c41650 2024-04-25T08:57:29.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 10 client.7356 fetch_fsmap learned FSMap version 51 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- 172.21.15.37:0/4093990524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x560f82a36f20 con 0x560f82c41650 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe2f57fa640 1 -- 172.21.15.37:0/4093990524 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe30405d580 con 0x560f82c41650 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe2f57fa640 1 -- 172.21.15.37:0/4093990524 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe3100e4020 con 0x560f82c41650 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 10 client.7356 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 10 client.7356 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 20 client.7356 populate_metadata read hostname 'smithi037' 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x560f82ca6d00 0x560f82ca7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 4 client.7356 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe31f874000 1 -- 172.21.15.37:0/4093990524 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) v1 -- 0x560f82a9d310 con 0x560f82ca6d00 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.261+0000 7fe319ca2640 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x560f82ca6d00 0x560f82ca7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe319ca2640 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x560f82ca6d00 0x560f82ca7110 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97a02e1acbe78be1 server_cookie=e29755107049812f in_seq=0 out_seq=0 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe2f57fa640 10 client.7356 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe2f57fa640 1 -- 172.21.15.37:0/4093990524 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560f82a9d310 con 0x560f82ca6d00 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe2f57fa640 10 client.7356 handle_command_reply: tid=0 2024-04-25T08:57:29.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 1 client.7356 shutdown 2024-04-25T08:57:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 1 -- 172.21.15.37:0/4093990524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c41650 msgr2=0x560f82cec0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c41650 0x560f82cec0e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fe3100ac040 tx=0x7fe3100046d0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 1 -- 172.21.15.37:0/4093990524 shutdown_connections 2024-04-25T08:57:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x560f82ca6d00 0x560f82ca7110 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560f82c8c1e0 0x560f82ca67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560f82c8b8c0 0x560f82cec620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe2f57fa640 0 client.7356 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 1 --2- 172.21.15.37:0/4093990524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f82c41650 0x560f82cec0e0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 1 -- 172.21.15.37:0/4093990524 >> 172.21.15.37:0/4093990524 conn(0x560f82b29870 msgr2=0x560f82c422c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 1 -- 172.21.15.37:0/4093990524 shutdown_connections 2024-04-25T08:57:29.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 1 -- 172.21.15.37:0/4093990524 wait complete. 2024-04-25T08:57:29.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31f874000 20 client.7356 trim_cache size 0 max 16384 2024-04-25T08:57:29.263 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:57:29.263 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-04-25T08:57:29.263 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "39caadc4-8e58-4384-b7e6-09f5b29d202e", 2024-04-25T08:57:29.263 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-04-25T08:57:29.263 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:57:29.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 -- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f8066f60 msgr2=0x7fe2f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f8066f60 0x7fe2f8069420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe3100ae000 tx=0x7fe3100a7000 comp rx=0 tx=0).stop 2024-04-25T08:57:29.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 -- 172.21.15.37:0/1990662285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe31406c3b0 msgr2=0x7fe3140ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe31406c3b0 0x7fe3140ce040 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c420 tx=0x7fe30400c8e0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 -- 172.21.15.37:0/1990662285 shutdown_connections 2024-04-25T08:57:29.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f8066f60 0x7fe2f8069420 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe31415b350 0x7fe3140ce580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe31406c3b0 0x7fe3140ce040 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 --2- 172.21.15.37:0/1990662285 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe314066bb0 0x7fe31414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 -- 172.21.15.37:0/1990662285 >> 172.21.15.37:0/1990662285 conn(0x7fe3140717f0 msgr2=0x7fe314153770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 -- 172.21.15.37:0/1990662285 shutdown_connections 2024-04-25T08:57:29.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.265+0000 7fe31c707640 1 -- 172.21.15.37:0/1990662285 wait complete. 2024-04-25T08:57:29.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:29.273 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:57:29.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd36b6cf640 1 Processor -- start 2024-04-25T08:57:29.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd36b6cf640 1 -- start start 2024-04-25T08:57:29.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd36b6cf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd3640b0380 0x7fd3640d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd36b6cf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd3640d4440 0x7fd36406d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd36b6cf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36406bff0 con 0x7fd3640b5270 2024-04-25T08:57:29.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd36b6cf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36406cc00 con 0x7fd3640b0380 2024-04-25T08:57:29.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd36b6cf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36406c2d0 con 0x7fd3640d4440 2024-04-25T08:57:29.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd368c6a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd3640b0380 0x7fd3640d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd36946b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd3640b5270 0x7fd3640b5650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:35544/0 (socket says 172.21.15.37:35544) 2024-04-25T08:57:29.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd36946b640 1 -- 172.21.15.37:0/2710094891 learned_addr learned my addr 172.21.15.37:0/2710094891 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:29.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd368c6a640 1 --2- 172.21.15.37:0/2710094891 >> v2:172.21.15.37:3300/0 conn(0x7fd3640b0380 0x7fd3640d3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48754/0 (socket says 172.21.15.37:48754) 2024-04-25T08:57:29.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd369c6c640 1 --2- 172.21.15.37:0/2710094891 >> v2:172.21.15.203:3300/0 conn(0x7fd3640d4440 0x7fd36406d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd368c6a640 1 -- 172.21.15.37:0/2710094891 >> v2:172.21.15.203:3300/0 conn(0x7fd3640d4440 msgr2=0x7fd36406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd368c6a640 1 --2- 172.21.15.37:0/2710094891 >> v2:172.21.15.203:3300/0 conn(0x7fd3640d4440 0x7fd36406d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd368c6a640 1 -- 172.21.15.37:0/2710094891 >> v1:172.21.15.37:6789/0 conn(0x7fd3640b5270 legacy=0x7fd3640b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd368c6a640 1 -- 172.21.15.37:0/2710094891 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36406c5b0 con 0x7fd3640b0380 2024-04-25T08:57:29.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.401+0000 7fd35bfff640 1 -- 172.21.15.37:0/2710094891 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278454305 0 0) 0x7fd36406bff0 con 0x7fd3640b5270 2024-04-25T08:57:29.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd368c6a640 1 --2- 172.21.15.37:0/2710094891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3640b0380 0x7fd3640d3f00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fd3640d0210 tx=0x7fd35401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f963af18996df887 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd35bfff640 1 -- 172.21.15.37:0/2710094891 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3540b0070 con 0x7fd3640b0380 2024-04-25T08:57:29.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd35bfff640 1 -- 172.21.15.37:0/2710094891 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3540ab600 con 0x7fd3640b0380 2024-04-25T08:57:29.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd35bfff640 1 -- 172.21.15.37:0/2710094891 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3540ab9d0 con 0x7fd3640b0380 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 -- 172.21.15.37:0/2710094891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3640b0380 msgr2=0x7fd3640d3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 --2- 172.21.15.37:0/2710094891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3640b0380 0x7fd3640d3f00 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fd3640d0210 tx=0x7fd35401c460 comp rx=0 tx=0).stop 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 -- 172.21.15.37:0/2710094891 shutdown_connections 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 --2- 172.21.15.37:0/2710094891 >> v2:172.21.15.203:3300/0 conn(0x7fd3640d4440 0x7fd36406d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 --2- 172.21.15.37:0/2710094891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3640b0380 0x7fd3640d3f00 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 -- 172.21.15.37:0/2710094891 >> 172.21.15.37:0/2710094891 conn(0x7fd36408b000 msgr2=0x7fd3640d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 -- 172.21.15.37:0/2710094891 shutdown_connections 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 -- 172.21.15.37:0/2710094891 wait complete. 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 Processor -- start 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 -- start start 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3640b0380 0x7fd364158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3640b5270 0x7fd36415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3640d4440 0x7fd36415c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36416a8a0 con 0x7fd3640b5270 2024-04-25T08:57:29.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36416a720 con 0x7fd3640d4440 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36b6cf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36416aa20 con 0x7fd3640b0380 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd368c6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3640b5270 0x7fd36415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd368c6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3640b5270 0x7fd36415c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48762/0 (socket says 172.21.15.37:48762) 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd368c6a640 1 -- 172.21.15.37:0/4103413404 learned_addr learned my addr 172.21.15.37:0/4103413404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd369c6c640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3640d4440 0x7fd36415c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd36946b640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3640b0380 0x7fd364158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd368c6a640 1 -- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3640b0380 msgr2=0x7fd364158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd368c6a640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3640b0380 0x7fd364158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd368c6a640 1 -- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3640d4440 msgr2=0x7fd36415c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd368c6a640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3640d4440 0x7fd36415c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.405+0000 7fd368c6a640 1 -- 172.21.15.37:0/4103413404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3641590d0 con 0x7fd3640b5270 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd368c6a640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3640b5270 0x7fd36415c2a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd3540ae040 tx=0x7fd35401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd359ffb640 1 -- 172.21.15.37:0/4103413404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3540b0050 con 0x7fd3640b5270 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd359ffb640 1 -- 172.21.15.37:0/4103413404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3540b1040 con 0x7fd3640b5270 2024-04-25T08:57:29.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd36b6cf640 1 -- 172.21.15.37:0/4103413404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3640b75f0 con 0x7fd3640b5270 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd359ffb640 1 -- 172.21.15.37:0/4103413404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd354005bc0 con 0x7fd3640b5270 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd36b6cf640 1 -- 172.21.15.37:0/4103413404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3640b7bb0 con 0x7fd3640b5270 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd359ffb640 1 -- 172.21.15.37:0/4103413404 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd3540be060 con 0x7fd3640b5270 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd359ffb640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd348066f90 0x7fd348069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd359ffb640 1 -- 172.21.15.37:0/4103413404 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd35411ebb0 con 0x7fd3640b5270 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd36946b640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd348066f90 0x7fd348069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd36e83c000 1 Processor -- start 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd36e83c000 1 -- start start 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.409+0000 7fd36946b640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd348066f90 0x7fd348069450 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd360007fd0 tx=0x7fd360009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58ca2f0 0x55aab587fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58800a0 0x55aab5880480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab56ccff0 con 0x55aab58c99d0 2024-04-25T08:57:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab5655150 con 0x55aab58ca2f0 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aab56c9090 con 0x55aab58800a0 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36946b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36946b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58c9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48770/0 (socket says 172.21.15.37:48770) 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36946b640 1 -- 172.21.15.37:0/2189044275 learned_addr learned my addr 172.21.15.37:0/2189044275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd368c6a640 1 --2- 172.21.15.37:0/2189044275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58ca2f0 0x55aab587fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd369c6c640 1 --2- 172.21.15.37:0/2189044275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58800a0 0x55aab5880480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36946b640 1 -- 172.21.15.37:0/2189044275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58800a0 msgr2=0x55aab5880480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36946b640 1 --2- 172.21.15.37:0/2189044275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58800a0 0x55aab5880480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36946b640 1 -- 172.21.15.37:0/2189044275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58ca2f0 msgr2=0x55aab587fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36946b640 1 --2- 172.21.15.37:0/2189044275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58ca2f0 0x55aab587fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36946b640 1 -- 172.21.15.37:0/2189044275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab58630d0 con 0x55aab58c99d0 2024-04-25T08:57:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36946b640 1 --2- 172.21.15.37:0/2189044275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58c9db0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd360001ea0 tx=0x7fd360011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7237dd759ab13f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd34effd640 1 -- 172.21.15.37:0/2189044275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36000e7b0 con 0x55aab58c99d0 2024-04-25T08:57:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd34effd640 1 -- 172.21.15.37:0/2189044275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36000e950 con 0x55aab58c99d0 2024-04-25T08:57:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd34effd640 1 -- 172.21.15.37:0/2189044275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36000ed00 con 0x55aab58c99d0 2024-04-25T08:57:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- 172.21.15.37:0/2189044275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 msgr2=0x55aab58c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 --2- 172.21.15.37:0/2189044275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58c9db0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fd360001ea0 tx=0x7fd360011000 comp rx=0 tx=0).stop 2024-04-25T08:57:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- 172.21.15.37:0/2189044275 shutdown_connections 2024-04-25T08:57:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 --2- 172.21.15.37:0/2189044275 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58800a0 0x55aab5880480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 --2- 172.21.15.37:0/2189044275 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58ca2f0 0x55aab587fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 --2- 172.21.15.37:0/2189044275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58c9db0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- 172.21.15.37:0/2189044275 >> 172.21.15.37:0/2189044275 conn(0x55aab587c020 msgr2=0x55aab587e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- 172.21.15.37:0/2189044275 shutdown_connections 2024-04-25T08:57:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- 172.21.15.37:0/2189044275 wait complete. 2024-04-25T08:57:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- start start 2024-04-25T08:57:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 Processor -- start 2024-04-25T08:57:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58800a0 0x55aab58ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58c99d0 0x55aab58aed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab5691250 con 0x55aab58ca2f0 2024-04-25T08:57:29.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab5687690 con 0x55aab58800a0 2024-04-25T08:57:29.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aab5655a40 con 0x55aab58c99d0 2024-04-25T08:57:29.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36e83c000 10 client.0 _ll_register_callbacks cb 0x55aab586ae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:29.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd369c6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd369c6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58af270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48782/0 (socket says 172.21.15.37:48782) 2024-04-25T08:57:29.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd369c6c640 1 -- 172.21.15.37:0/2431152085 learned_addr learned my addr 172.21.15.37:0/2431152085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:29.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd36946b640 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58800a0 0x55aab58ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd368c6a640 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58c99d0 0x55aab58aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd369c6c640 1 -- 172.21.15.37:0/2431152085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58c99d0 msgr2=0x55aab58aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd369c6c640 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58c99d0 0x55aab58aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd369c6c640 1 -- 172.21.15.37:0/2431152085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58800a0 msgr2=0x55aab58ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd369c6c640 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58800a0 0x55aab58ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.413+0000 7fd369c6c640 1 -- 172.21.15.37:0/2431152085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aab577a1c0 con 0x55aab58ca2f0 2024-04-25T08:57:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd369c6c640 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58af270 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0079d0 tx=0x7fd35c007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0bf2970299d148 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd343fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:57:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd343fff640 1 -- 172.21.15.37:0/2431152085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c01b020 con 0x55aab58ca2f0 2024-04-25T08:57:29.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 -- 172.21.15.37:0/2431152085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd36000e950 con 0x55aab58ca2f0 2024-04-25T08:57:29.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd343fff640 1 -- 172.21.15.37:0/2431152085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd35c011a80 con 0x55aab58ca2f0 2024-04-25T08:57:29.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd343fff640 1 -- 172.21.15.37:0/2431152085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd35c025050 con 0x55aab58ca2f0 2024-04-25T08:57:29.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 10 client.7371 fetch_fsmap learned FSMap version 51 2024-04-25T08:57:29.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 -- 172.21.15.37:0/2431152085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55aab5776170 con 0x55aab58ca2f0 2024-04-25T08:57:29.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd343fff640 1 -- 172.21.15.37:0/2431152085 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd36000e950 con 0x55aab58ca2f0 2024-04-25T08:57:29.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd343fff640 1 -- 172.21.15.37:0/2431152085 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd35c028020 con 0x55aab58ca2f0 2024-04-25T08:57:29.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 10 client.7371 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:57:29.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 10 client.7371 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:57:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 20 client.7371 populate_metadata read hostname 'smithi037' 2024-04-25T08:57:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55aab58d9c80 0x55aab58dc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 4 client.7371 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 -- 172.21.15.37:0/2431152085 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aab5668c60 con 0x55aab58d9c80 2024-04-25T08:57:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36946b640 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55aab58d9c80 0x55aab58dc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd343fff640 10 client.7371 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36946b640 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55aab58d9c80 0x55aab58dc070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0d4ab8b11450d77 server_cookie=cd06b75bb3c835be in_seq=0 out_seq=0 2024-04-25T08:57:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd343fff640 1 -- 172.21.15.37:0/2431152085 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aab5668c60 con 0x55aab58d9c80 2024-04-25T08:57:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd343fff640 10 client.7371 handle_command_reply: tid=0 2024-04-25T08:57:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 client.7371 shutdown 2024-04-25T08:57:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 -- 172.21.15.37:0/2431152085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 msgr2=0x55aab58af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58af270 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0079d0 tx=0x7fd35c007e90 comp rx=0 tx=0).stop 2024-04-25T08:57:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 -- 172.21.15.37:0/2431152085 shutdown_connections 2024-04-25T08:57:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55aab58d9c80 0x55aab58dc070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58af270 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58c99d0 0x55aab58aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 --2- 172.21.15.37:0/2431152085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58800a0 0x55aab58ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd343fff640 0 client.7371 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 -- 172.21.15.37:0/2431152085 >> 172.21.15.37:0/2431152085 conn(0x55aab587c020 msgr2=0x55aab587eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 -- 172.21.15.37:0/2431152085 shutdown_connections 2024-04-25T08:57:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 1 -- 172.21.15.37:0/2431152085 wait complete. 2024-04-25T08:57:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.417+0000 7fd36e83c000 20 client.7371 trim_cache size 0 max 16384 2024-04-25T08:57:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 Processor -- start 2024-04-25T08:57:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 -- start start 2024-04-25T08:57:29.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58800a0 0x55aab587fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58d6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58ca2f0 0x55aab58d7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd364203240 con 0x55aab58c99d0 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd364202440 con 0x55aab58800a0 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640b6f00 con 0x55aab58ca2f0 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd368c6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58d6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd368c6a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58d6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48788/0 (socket says 172.21.15.37:48788) 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd368c6a640 1 -- 172.21.15.37:0/302359615 learned_addr learned my addr 172.21.15.37:0/302359615 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36946b640 1 --2- 172.21.15.37:0/302359615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58800a0 0x55aab587fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd369c6c640 1 --2- 172.21.15.37:0/302359615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58ca2f0 0x55aab58d7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd368c6a640 1 -- 172.21.15.37:0/302359615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58ca2f0 msgr2=0x55aab58d7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd368c6a640 1 --2- 172.21.15.37:0/302359615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58ca2f0 0x55aab58d7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd368c6a640 1 -- 172.21.15.37:0/302359615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58800a0 msgr2=0x55aab587fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd368c6a640 1 --2- 172.21.15.37:0/302359615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58800a0 0x55aab587fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd368c6a640 1 -- 172.21.15.37:0/302359615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aab5713de0 con 0x55aab58c99d0 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd368c6a640 1 --2- 172.21.15.37:0/302359615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58d6e40 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd3540eb6a0 tx=0x7fd354019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49201fd493944288 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd34e7fc640 1 -- 172.21.15.37:0/302359615 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3540f3170 con 0x55aab58c99d0 2024-04-25T08:57:29.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd34e7fc640 1 -- 172.21.15.37:0/302359615 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3540f3390 con 0x55aab58c99d0 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd34e7fc640 1 -- 172.21.15.37:0/302359615 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3540f7950 con 0x55aab58c99d0 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 -- 172.21.15.37:0/302359615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 msgr2=0x55aab58d6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 --2- 172.21.15.37:0/302359615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58d6e40 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fd3540eb6a0 tx=0x7fd354019200 comp rx=0 tx=0).stop 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 -- 172.21.15.37:0/302359615 shutdown_connections 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 --2- 172.21.15.37:0/302359615 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58ca2f0 0x55aab58d7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 --2- 172.21.15.37:0/302359615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58c99d0 0x55aab58d6e40 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 --2- 172.21.15.37:0/302359615 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58800a0 0x55aab587fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 -- 172.21.15.37:0/302359615 >> 172.21.15.37:0/302359615 conn(0x55aab587c020 msgr2=0x55aab58d6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 -- 172.21.15.37:0/302359615 shutdown_connections 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.433+0000 7fd36e83c000 1 -- 172.21.15.37:0/302359615 wait complete. 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- start start 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 Processor -- start 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58800a0 0x55aab5929580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58c99d0 0x55aab5929ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58dba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aab5667e00 con 0x55aab58ca2f0 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3640b8d90 con 0x55aab58c99d0 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aab56cc300 con 0x55aab58800a0 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 10 client.0 _ll_register_callbacks cb 0x55aab584b040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd369c6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58dba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd369c6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58dba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48802/0 (socket says 172.21.15.37:48802) 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd369c6c640 1 -- 172.21.15.37:0/1047911925 learned_addr learned my addr 172.21.15.37:0/1047911925 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36946b640 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58800a0 0x55aab5929580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd369c6c640 1 -- 172.21.15.37:0/1047911925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58800a0 msgr2=0x55aab5929580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd368c6a640 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58c99d0 0x55aab5929ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd369c6c640 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58800a0 0x55aab5929580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd369c6c640 1 -- 172.21.15.37:0/1047911925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58c99d0 msgr2=0x55aab5929ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd369c6c640 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58c99d0 0x55aab5929ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd369c6c640 1 -- 172.21.15.37:0/1047911925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aab566e8e0 con 0x55aab58ca2f0 2024-04-25T08:57:29.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd369c6c640 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58dba20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd35c020720 tx=0x7fd35c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ef1fc034600de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd343fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd343fff640 1 -- 172.21.15.37:0/1047911925 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c02d030 con 0x55aab58ca2f0 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd343fff640 1 -- 172.21.15.37:0/1047911925 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd35c002050 con 0x55aab58ca2f0 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- 172.21.15.37:0/1047911925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3540f3390 con 0x55aab58ca2f0 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd343fff640 1 -- 172.21.15.37:0/1047911925 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd35c039050 con 0x55aab58ca2f0 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 10 client.7377 fetch_fsmap learned FSMap version 51 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- 172.21.15.37:0/1047911925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55aab57268f0 con 0x55aab58ca2f0 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd343fff640 1 -- 172.21.15.37:0/1047911925 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3540f3390 con 0x55aab58ca2f0 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd343fff640 1 -- 172.21.15.37:0/1047911925 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd35c03c020 con 0x55aab58ca2f0 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 10 client.7377 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 10 client.7377 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:57:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 20 client.7377 populate_metadata read hostname 'smithi037' 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55aab58dc410 0x55aab58dc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 4 client.7377 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- 172.21.15.37:0/1047911925 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aab56da530 con 0x55aab58dc410 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36946b640 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55aab58dc410 0x55aab58dc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd343fff640 10 client.7377 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36946b640 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55aab58dc410 0x55aab58dc820 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=afeef01d9ff342ae server_cookie=5332ce6c8c528ccd in_seq=0 out_seq=0 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd343fff640 1 -- 172.21.15.37:0/1047911925 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55aab56da530 con 0x55aab58dc410 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd343fff640 10 client.7377 handle_command_reply: tid=0 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 client.7377 shutdown 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- 172.21.15.37:0/1047911925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 msgr2=0x55aab58dba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58dba20 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fd35c020720 tx=0x7fd35c004a50 comp rx=0 tx=0).stop 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- 172.21.15.37:0/1047911925 shutdown_connections 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55aab58dc410 0x55aab58dc820 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55aab58ca2f0 0x55aab58dba20 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55aab58c99d0 0x55aab5929ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd343fff640 0 client.7377 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 --2- 172.21.15.37:0/1047911925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55aab58800a0 0x55aab5929580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- 172.21.15.37:0/1047911925 >> 172.21.15.37:0/1047911925 conn(0x55aab587c020 msgr2=0x55aab58d6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- 172.21.15.37:0/1047911925 shutdown_connections 2024-04-25T08:57:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 1 -- 172.21.15.37:0/1047911925 wait complete. 2024-04-25T08:57:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36e83c000 20 client.7377 trim_cache size 0 max 16384 2024-04-25T08:57:29.436 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:57:29.436 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:57:29.436 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-04-25T08:57:29.436 INFO:teuthology.orchestra.run.smithi037.stdout: "39caadc4-8e58-4384-b7e6-09f5b29d202e": { 2024-04-25T08:57:29.437 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/subdir", 2024-04-25T08:57:29.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "39caadc4-8e58-4384-b7e6-09f5b29d202e", 2024-04-25T08:57:29.437 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive" 2024-04-25T08:57:29.437 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:57:29.437 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:57:29.437 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:57:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36b6cf640 1 -- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd348066f90 msgr2=0x7fd348069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36b6cf640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd348066f90 0x7fd348069450 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd360007fd0 tx=0x7fd360009040 comp rx=0 tx=0).stop 2024-04-25T08:57:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36b6cf640 1 -- 172.21.15.37:0/4103413404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3640b5270 msgr2=0x7fd36415c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36b6cf640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3640b5270 0x7fd36415c2a0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fd3540ae040 tx=0x7fd35401aca0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36b6cf640 1 -- 172.21.15.37:0/4103413404 shutdown_connections 2024-04-25T08:57:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36b6cf640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd348066f90 0x7fd348069450 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36b6cf640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd3640d4440 0x7fd36415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36b6cf640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd3640b5270 0x7fd36415c2a0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36b6cf640 1 --2- 172.21.15.37:0/4103413404 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd3640b0380 0x7fd364158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.437+0000 7fd36b6cf640 1 -- 172.21.15.37:0/4103413404 >> 172.21.15.37:0/4103413404 conn(0x7fd36408b000 msgr2=0x7fd3640d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.441+0000 7fd36b6cf640 1 -- 172.21.15.37:0/4103413404 shutdown_connections 2024-04-25T08:57:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:29.441+0000 7fd36b6cf640 1 -- 172.21.15.37:0/4103413404 wait complete. 2024-04-25T08:57:29.447 INFO:tasks.cephfs.filesystem:scrub status for tag:39caadc4-8e58-4384-b7e6-09f5b29d202e - {'path': '/subdir', 'tag': '39caadc4-8e58-4384-b7e6-09f5b29d202e', 'options': 'recursive'} 2024-04-25T08:57:55.925 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:57:55.929 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:57:59.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:59.448 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:57:59.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.597+0000 7fb1f01f0640 1 Processor -- start 2024-04-25T08:57:59.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.597+0000 7fb1f01f0640 1 -- start start 2024-04-25T08:57:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.601+0000 7fb1f01f0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb1e815da80 0x7fb1e80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.601+0000 7fb1f01f0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e806be60 con 0x7fb1e80d07f0 2024-04-25T08:57:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.601+0000 7fb1f01f0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e815de60 con 0x7fb1e80d1cd0 2024-04-25T08:57:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.601+0000 7fb1f01f0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e806c440 con 0x7fb1e815da80 2024-04-25T08:57:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.601+0000 7fb1edf8c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb1e815da80 0x7fb1e80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.601+0000 7fb1ee78d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb1e80d07f0 0x7fb1e806c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48058/0 (socket says 172.21.15.37:48058) 2024-04-25T08:57:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.601+0000 7fb1ee78d640 1 -- 172.21.15.37:0/999785065 learned_addr learned my addr 172.21.15.37:0/999785065 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:57:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.601+0000 7fb1edf8c640 1 -- 172.21.15.37:0/999785065 >> v1:172.21.15.203:6789/0 conn(0x7fb1e80d1cd0 legacy=0x7fb1e80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.601+0000 7fb1edf8c640 1 -- 172.21.15.37:0/999785065 >> v1:172.21.15.37:6789/0 conn(0x7fb1e80d07f0 legacy=0x7fb1e806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.601+0000 7fb1ecf8a640 1 -- 172.21.15.37:0/999785065 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560927623 0 0) 0x7fb1e806be60 con 0x7fb1e80d07f0 2024-04-25T08:57:59.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.601+0000 7fb1edf8c640 1 -- 172.21.15.37:0/999785065 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e80d2450 con 0x7fb1e815da80 2024-04-25T08:57:59.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/999785065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e815da80 0x7fb1e80d4230 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb1e806c5c0 tx=0x7fb1e401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9596422e214f1047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1ecf8a640 1 -- 172.21.15.37:0/999785065 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40b1070 con 0x7fb1e815da80 2024-04-25T08:57:59.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1ecf8a640 1 -- 172.21.15.37:0/999785065 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e40ac410 con 0x7fb1e815da80 2024-04-25T08:57:59.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1ecf8a640 1 -- 172.21.15.37:0/999785065 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40ac7e0 con 0x7fb1e815da80 2024-04-25T08:57:59.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1f01f0640 1 -- 172.21.15.37:0/999785065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e815da80 msgr2=0x7fb1e80d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1f01f0640 1 --2- 172.21.15.37:0/999785065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e815da80 0x7fb1e80d4230 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb1e806c5c0 tx=0x7fb1e401c410 comp rx=0 tx=0).stop 2024-04-25T08:57:59.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1f01f0640 1 -- 172.21.15.37:0/999785065 shutdown_connections 2024-04-25T08:57:59.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1f01f0640 1 --2- 172.21.15.37:0/999785065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e815da80 0x7fb1e80d4230 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1f01f0640 1 -- 172.21.15.37:0/999785065 >> 172.21.15.37:0/999785065 conn(0x7fb1e808b000 msgr2=0x7fb1e80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1f01f0640 1 -- 172.21.15.37:0/999785065 shutdown_connections 2024-04-25T08:57:59.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1f01f0640 1 -- 172.21.15.37:0/999785065 wait complete. 2024-04-25T08:57:59.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1f01f0640 1 Processor -- start 2024-04-25T08:57:59.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.605+0000 7fb1f01f0640 1 -- start start 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1f01f0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1e80d07f0 0x7fb1e80b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1f01f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e80d1cd0 0x7fb1e80cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1f01f0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1e815da80 0x7fb1e815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1f01f0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e806d970 con 0x7fb1e80d1cd0 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1f01f0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e806d7f0 con 0x7fb1e815da80 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1f01f0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e806daf0 con 0x7fb1e80d07f0 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1ed78b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e80d1cd0 0x7fb1e80cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1ed78b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e80d1cd0 0x7fb1e80cd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42156/0 (socket says 172.21.15.37:42156) 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1ed78b640 1 -- 172.21.15.37:0/484378734 learned_addr learned my addr 172.21.15.37:0/484378734 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1ee78d640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1e815da80 0x7fb1e815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1e80d07f0 0x7fb1e80b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1ed78b640 1 -- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1e80d07f0 msgr2=0x7fb1e80b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1e80d07f0 0x7fb1e80b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1ed78b640 1 -- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1e815da80 msgr2=0x7fb1e815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1e815da80 0x7fb1e815ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1ed78b640 1 -- 172.21.15.37:0/484378734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e80b68f0 con 0x7fb1e80d1cd0 2024-04-25T08:57:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e80d1cd0 0x7fb1e80cd150 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800c4a0 tx=0x7fb1d800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1de7fc640 1 -- 172.21.15.37:0/484378734 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8016020 con 0x7fb1e80d1cd0 2024-04-25T08:57:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1de7fc640 1 -- 172.21.15.37:0/484378734 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d8012830 con 0x7fb1e80d1cd0 2024-04-25T08:57:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1f01f0640 1 -- 172.21.15.37:0/484378734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e80b6bd0 con 0x7fb1e80d1cd0 2024-04-25T08:57:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1de7fc640 1 -- 172.21.15.37:0/484378734 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8012c60 con 0x7fb1e80d1cd0 2024-04-25T08:57:59.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1f01f0640 1 -- 172.21.15.37:0/484378734 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e815ea60 con 0x7fb1e80d1cd0 2024-04-25T08:57:59.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1de7fc640 1 -- 172.21.15.37:0/484378734 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1d8019020 con 0x7fb1e80d1cd0 2024-04-25T08:57:59.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1de7fc640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1cc066f60 0x7fb1cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.609+0000 7fb1de7fc640 1 -- 172.21.15.37:0/484378734 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb1d8085320 con 0x7fb1e80d1cd0 2024-04-25T08:57:59.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1cc066f60 0x7fb1cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 Processor -- start 2024-04-25T08:57:59.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 -- start start 2024-04-25T08:57:59.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1cc066f60 0x7fb1cc069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40ae000 tx=0x7fb1e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887efdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887f02f0 0x55f6887a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887a60a0 0x55f6887a6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6885f2ff0 con 0x55f6887ef9d0 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f68857b150 con 0x55f6887f02f0 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6885ef090 con 0x55f6887a60a0 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1edf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887efdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1edf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887efdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42170/0 (socket says 172.21.15.37:42170) 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1edf8c640 1 -- 172.21.15.37:0/1441620905 learned_addr learned my addr 172.21.15.37:0/1441620905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/1441620905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887f02f0 0x55f6887a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1ee78d640 1 --2- 172.21.15.37:0/1441620905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887a60a0 0x55f6887a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1edf8c640 1 -- 172.21.15.37:0/1441620905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887a60a0 msgr2=0x55f6887a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/1441620905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887a60a0 0x55f6887a6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1edf8c640 1 -- 172.21.15.37:0/1441620905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887f02f0 msgr2=0x55f6887a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/1441620905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887f02f0 0x55f6887a5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1edf8c640 1 -- 172.21.15.37:0/1441620905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6887890d0 con 0x55f6887ef9d0 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/1441620905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887efdb0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40af040 tx=0x7fb1e40a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29abbede6e51b5dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1cb7fe640 1 -- 172.21.15.37:0/1441620905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40c8070 con 0x55f6887ef9d0 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1cb7fe640 1 -- 172.21.15.37:0/1441620905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e40acd20 con 0x55f6887ef9d0 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1cb7fe640 1 -- 172.21.15.37:0/1441620905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40b1040 con 0x55f6887ef9d0 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 -- 172.21.15.37:0/1441620905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 msgr2=0x55f6887efdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 --2- 172.21.15.37:0/1441620905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887efdb0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40af040 tx=0x7fb1e40a6040 comp rx=0 tx=0).stop 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 -- 172.21.15.37:0/1441620905 shutdown_connections 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 --2- 172.21.15.37:0/1441620905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887a60a0 0x55f6887a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 --2- 172.21.15.37:0/1441620905 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887f02f0 0x55f6887a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 --2- 172.21.15.37:0/1441620905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887efdb0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 -- 172.21.15.37:0/1441620905 >> 172.21.15.37:0/1441620905 conn(0x55f6887a2020 msgr2=0x55f6887a4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 -- 172.21.15.37:0/1441620905 shutdown_connections 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 -- 172.21.15.37:0/1441620905 wait complete. 2024-04-25T08:57:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 -- start start 2024-04-25T08:57:59.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.613+0000 7fb1f335d000 1 Processor -- start 2024-04-25T08:57:59.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f6887d47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887ef9d0 0x55f6887d4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887f02f0 0x55f6887d5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6885b7250 con 0x55f6887a60a0 2024-04-25T08:57:59.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6885ad690 con 0x55f6887ef9d0 2024-04-25T08:57:59.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f68857ba40 con 0x55f6887f02f0 2024-04-25T08:57:59.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 10 client.0 _ll_register_callbacks cb 0x55f688790e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:59.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1edf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f6887d47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1edf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f6887d47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42176/0 (socket says 172.21.15.37:42176) 2024-04-25T08:57:59.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1edf8c640 1 -- 172.21.15.37:0/3543380377 learned_addr learned my addr 172.21.15.37:0/3543380377 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1ee78d640 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887f02f0 0x55f6887d5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887ef9d0 0x55f6887d4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1edf8c640 1 -- 172.21.15.37:0/3543380377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887f02f0 msgr2=0x55f6887d5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887f02f0 0x55f6887d5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1edf8c640 1 -- 172.21.15.37:0/3543380377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887ef9d0 msgr2=0x55f6887d4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887ef9d0 0x55f6887d4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1edf8c640 1 -- 172.21.15.37:0/3543380377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6886a01c0 con 0x55f6887a60a0 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887ef9d0 0x55f6887d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f6887d47f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4000c10 tx=0x7fb1e401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3143e9d3fe7c8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3543380377 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40c8070 con 0x55f6887a60a0 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3543380377 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb1e40cfca0 con 0x55f6887a60a0 2024-04-25T08:57:59.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 -- 172.21.15.37:0/3543380377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1e40acd20 con 0x55f6887a60a0 2024-04-25T08:57:59.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3543380377 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1e40b1040 con 0x55f6887a60a0 2024-04-25T08:57:59.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 10 client.7392 fetch_fsmap learned FSMap version 51 2024-04-25T08:57:59.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 -- 172.21.15.37:0/3543380377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55f68869c170 con 0x55f6887a60a0 2024-04-25T08:57:59.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3543380377 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1e40acd20 con 0x55f6887a60a0 2024-04-25T08:57:59.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3543380377 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb1e40c2070 con 0x55f6887a60a0 2024-04-25T08:57:59.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 10 client.7392 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:57:59.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 10 client.7392 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:57:59.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 20 client.7392 populate_metadata read hostname 'smithi037' 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55f6887ffc80 0x55f688802070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 4 client.7392 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 -- 172.21.15.37:0/3543380377 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f68858ec60 con 0x55f6887ffc80 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55f6887ffc80 0x55f688802070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55f6887ffc80 0x55f688802070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d2be058e27cc44d server_cookie=689e3777341fa0f4 in_seq=0 out_seq=0 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1c8ff9640 10 client.7392 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3543380377 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f68858ec60 con 0x55f6887ffc80 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1c8ff9640 10 client.7392 handle_command_reply: tid=0 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 client.7392 shutdown 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 -- 172.21.15.37:0/3543380377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 msgr2=0x55f6887d47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f6887d47f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4000c10 tx=0x7fb1e401ac50 comp rx=0 tx=0).stop 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 -- 172.21.15.37:0/3543380377 shutdown_connections 2024-04-25T08:57:59.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55f6887ffc80 0x55f688802070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887f02f0 0x55f6887d5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887ef9d0 0x55f6887d4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3543380377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f6887d47f0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1c8ff9640 0 client.7392 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 -- 172.21.15.37:0/3543380377 >> 172.21.15.37:0/3543380377 conn(0x55f6887a2020 msgr2=0x55f6887a4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 -- 172.21.15.37:0/3543380377 shutdown_connections 2024-04-25T08:57:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 1 -- 172.21.15.37:0/3543380377 wait complete. 2024-04-25T08:57:59.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.617+0000 7fb1f335d000 20 client.7392 trim_cache size 0 max 16384 2024-04-25T08:57:59.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1f335d000 1 Processor -- start 2024-04-25T08:57:59.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1f335d000 1 -- start start 2024-04-25T08:57:59.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887a60a0 0x55f6887eede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887f02f0 0x55f6887a5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1f335d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e8202ea0 con 0x55f6887ef9d0 2024-04-25T08:57:59.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1f335d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e82020a0 con 0x55f6887f02f0 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1f335d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e815fef0 con 0x55f6887a60a0 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1ed78b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1ed78b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42188/0 (socket says 172.21.15.37:42188) 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1ed78b640 1 -- 172.21.15.37:0/2433970970 learned_addr learned my addr 172.21.15.37:0/2433970970 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1ed78b640 1 -- 172.21.15.37:0/2433970970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887a60a0 msgr2=0x55f6887eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/2433970970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887a60a0 0x55f6887eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1ed78b640 1 -- 172.21.15.37:0/2433970970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887f02f0 msgr2=0x55f6887a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/2433970970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887f02f0 0x55f6887a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.633+0000 7fb1ed78b640 1 -- 172.21.15.37:0/2433970970 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f688639de0 con 0x55f6887ef9d0 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/2433970970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887a5420 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800ce60 tx=0x7fb1d80834a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d148ebc8cfbf6a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1caffd640 1 -- 172.21.15.37:0/2433970970 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d8081070 con 0x55f6887ef9d0 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1caffd640 1 -- 172.21.15.37:0/2433970970 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d8053a50 con 0x55f6887ef9d0 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1caffd640 1 -- 172.21.15.37:0/2433970970 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1d806d030 con 0x55f6887ef9d0 2024-04-25T08:57:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- 172.21.15.37:0/2433970970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 msgr2=0x55f6887a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 --2- 172.21.15.37:0/2433970970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887a5420 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800ce60 tx=0x7fb1d80834a0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- 172.21.15.37:0/2433970970 shutdown_connections 2024-04-25T08:57:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 --2- 172.21.15.37:0/2433970970 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887f02f0 0x55f6887a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 --2- 172.21.15.37:0/2433970970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887ef9d0 0x55f6887a5420 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 --2- 172.21.15.37:0/2433970970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887a60a0 0x55f6887eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- 172.21.15.37:0/2433970970 >> 172.21.15.37:0/2433970970 conn(0x55f6887a2020 msgr2=0x55f6887ee0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- 172.21.15.37:0/2433970970 shutdown_connections 2024-04-25T08:57:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- 172.21.15.37:0/2433970970 wait complete. 2024-04-25T08:57:59.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- start start 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 Processor -- start 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f688850520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887ef9d0 0x55f688850a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887f02f0 0x55f688802380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f68858de00 con 0x55f6887a60a0 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e81604f0 con 0x55f6887ef9d0 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f6885f2300 con 0x55f6887f02f0 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 10 client.0 _ll_register_callbacks cb 0x55f688771040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1edf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f688850520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1edf8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f688850520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42202/0 (socket says 172.21.15.37:42202) 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1edf8c640 1 -- 172.21.15.37:0/3577982037 learned_addr learned my addr 172.21.15.37:0/3577982037 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1edf8c640 1 -- 172.21.15.37:0/3577982037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887f02f0 msgr2=0x55f688802380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1ee78d640 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887f02f0 0x55f688802380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887ef9d0 0x55f688850a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887f02f0 0x55f688802380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1edf8c640 1 -- 172.21.15.37:0/3577982037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887ef9d0 msgr2=0x55f688850a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887ef9d0 0x55f688850a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1edf8c640 1 -- 172.21.15.37:0/3577982037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6885948e0 con 0x55f6887a60a0 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1edf8c640 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f688850520 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fb1e401ccb0 tx=0x7fb1e40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba2823ee5a2b0de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3577982037 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40dc020 con 0x55f6887a60a0 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3577982037 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb1e40d0900 con 0x55f6887a60a0 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- 172.21.15.37:0/3577982037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb1d8053a50 con 0x55f6887a60a0 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3577982037 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb1e40d0b00 con 0x55f6887a60a0 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 10 client.7398 fetch_fsmap learned FSMap version 51 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- 172.21.15.37:0/3577982037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x55f68864c8f0 con 0x55f6887a60a0 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3577982037 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1d8053a50 con 0x55f6887a60a0 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3577982037 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb1e40df020 con 0x55f6887a60a0 2024-04-25T08:57:59.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 10 client.7398 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 10 client.7398 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 20 client.7398 populate_metadata read hostname 'smithi037' 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55f688802d40 0x55f688803150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 4 client.7398 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1f335d000 1 -- 172.21.15.37:0/3577982037 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f688600530 con 0x55f688802d40 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.637+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55f688802d40 0x55f688803150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1ed78b640 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55f688802d40 0x55f688803150 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93a34b52bcc1a1fa server_cookie=8249f10dffbc293c in_seq=0 out_seq=0 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1c8ff9640 10 client.7398 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1c8ff9640 1 -- 172.21.15.37:0/3577982037 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f688600530 con 0x55f688802d40 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1c8ff9640 10 client.7398 handle_command_reply: tid=0 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 1 client.7398 shutdown 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 1 -- 172.21.15.37:0/3577982037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 msgr2=0x55f688850520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f688850520 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fb1e401ccb0 tx=0x7fb1e40191b0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 1 -- 172.21.15.37:0/3577982037 shutdown_connections 2024-04-25T08:57:59.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55f688802d40 0x55f688803150 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f6887f02f0 0x55f688802380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f6887ef9d0 0x55f688850a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1c8ff9640 0 client.7398 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 1 --2- 172.21.15.37:0/3577982037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6887a60a0 0x55f688850520 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 1 -- 172.21.15.37:0/3577982037 >> 172.21.15.37:0/3577982037 conn(0x55f6887a2020 msgr2=0x55f6887ee930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 1 -- 172.21.15.37:0/3577982037 shutdown_connections 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 1 -- 172.21.15.37:0/3577982037 wait complete. 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f335d000 20 client.7398 trim_cache size 0 max 16384 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-04-25T08:57:59.637 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:57:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 -- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1cc066f60 msgr2=0x7fb1cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1cc066f60 0x7fb1cc069420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb1e40ae000 tx=0x7fb1e4008040 comp rx=0 tx=0).stop 2024-04-25T08:57:59.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 -- 172.21.15.37:0/484378734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e80d1cd0 msgr2=0x7fb1e80cd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e80d1cd0 0x7fb1e80cd150 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fb1d800c4a0 tx=0x7fb1d800c960 comp rx=0 tx=0).stop 2024-04-25T08:57:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 -- 172.21.15.37:0/484378734 shutdown_connections 2024-04-25T08:57:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1cc066f60 0x7fb1cc069420 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1e815da80 0x7fb1e815ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1e80d1cd0 0x7fb1e80cd150 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 --2- 172.21.15.37:0/484378734 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1e80d07f0 0x7fb1e80b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 -- 172.21.15.37:0/484378734 >> 172.21.15.37:0/484378734 conn(0x7fb1e808b000 msgr2=0x7fb1e80b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 -- 172.21.15.37:0/484378734 shutdown_connections 2024-04-25T08:57:59.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.641+0000 7fb1f01f0640 1 -- 172.21.15.37:0/484378734 wait complete. 2024-04-25T08:57:59.648 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:57:59.648 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:59.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b98e55640 1 Processor -- start 2024-04-25T08:57:59.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b98e55640 1 -- start start 2024-04-25T08:57:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b98e55640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406fc50 0x7f9b9406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b98e55640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9b9406b1d0 0x7f9b940cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b98e55640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9406b710 con 0x7f9b940cfd80 2024-04-25T08:57:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b98e55640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b94161560 con 0x7f9b9406b1d0 2024-04-25T08:57:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b98e55640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b941617b0 con 0x7f9b9406fc50 2024-04-25T08:57:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b9259c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406fc50 0x7f9b9406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b91d9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9b9406b1d0 0x7f9b940cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b9259c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406fc50 0x7f9b9406ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:54220/0 (socket says 172.21.15.37:54220) 2024-04-25T08:57:59.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b9259c640 1 -- 172.21.15.37:0/1020988283 learned_addr learned my addr 172.21.15.37:0/1020988283 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:59.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b9259c640 1 -- 172.21.15.37:0/1020988283 >> v2:172.21.15.203:3300/0 conn(0x7f9b9406b1d0 msgr2=0x7f9b940cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b9259c640 1 --2- 172.21.15.37:0/1020988283 >> v2:172.21.15.203:3300/0 conn(0x7f9b9406b1d0 0x7f9b940cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b9259c640 1 -- 172.21.15.37:0/1020988283 >> v1:172.21.15.203:6789/0 conn(0x7f9b940cfd80 legacy=0x7f9b9415d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b9259c640 1 -- 172.21.15.37:0/1020988283 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b94161a90 con 0x7f9b9406fc50 2024-04-25T08:57:59.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.773+0000 7f9b91d9b640 1 --2- 172.21.15.37:0/1020988283 >> v2:172.21.15.203:3300/0 conn(0x7f9b9406b1d0 0x7f9b940cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:59.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b9259c640 1 --2- 172.21.15.37:0/1020988283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406fc50 0x7f9b9406ac90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009870 tx=0x7f9b8801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dccd5f446b210a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b9159a640 1 -- 172.21.15.37:0/1020988283 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b880ab640 con 0x7f9b9406fc50 2024-04-25T08:57:59.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b9159a640 1 -- 172.21.15.37:0/1020988283 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b880b0070 con 0x7f9b9406fc50 2024-04-25T08:57:59.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b9159a640 1 -- 172.21.15.37:0/1020988283 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b880aba00 con 0x7f9b9406fc50 2024-04-25T08:57:59.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b98e55640 1 -- 172.21.15.37:0/1020988283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406fc50 msgr2=0x7f9b9406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b98e55640 1 --2- 172.21.15.37:0/1020988283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406fc50 0x7f9b9406ac90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b88009870 tx=0x7f9b8801c530 comp rx=0 tx=0).stop 2024-04-25T08:57:59.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b98e55640 1 -- 172.21.15.37:0/1020988283 shutdown_connections 2024-04-25T08:57:59.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b98e55640 1 --2- 172.21.15.37:0/1020988283 >> v2:172.21.15.203:3300/0 conn(0x7f9b9406b1d0 0x7f9b940cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b98e55640 1 --2- 172.21.15.37:0/1020988283 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406fc50 0x7f9b9406ac90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b98e55640 1 -- 172.21.15.37:0/1020988283 >> 172.21.15.37:0/1020988283 conn(0x7f9b94099a70 msgr2=0x7f9b940d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b98e55640 1 -- 172.21.15.37:0/1020988283 shutdown_connections 2024-04-25T08:57:59.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b98e55640 1 -- 172.21.15.37:0/1020988283 wait complete. 2024-04-25T08:57:59.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b98e55640 1 Processor -- start 2024-04-25T08:57:59.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.777+0000 7f9b98e55640 1 -- start start 2024-04-25T08:57:59.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b98e55640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406b1d0 0x7f9b9414fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b98e55640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b9406fc50 0x7f9b940c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b98e55640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b940cfd80 0x7f9b941fa0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b98e55640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940701b0 con 0x7f9b940cfd80 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b98e55640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b94070030 con 0x7f9b9406fc50 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b98e55640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b94162290 con 0x7f9b9406b1d0 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b92d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b940cfd80 0x7f9b941fa0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b92d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b940cfd80 0x7f9b941fa0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42208/0 (socket says 172.21.15.37:42208) 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b92d9d640 1 -- 172.21.15.37:0/476548586 learned_addr learned my addr 172.21.15.37:0/476548586 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b91d9b640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b9406fc50 0x7f9b940c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b9259c640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406b1d0 0x7f9b9414fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b92d9d640 1 -- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406b1d0 msgr2=0x7f9b9414fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b92d9d640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406b1d0 0x7f9b9414fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b92d9d640 1 -- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b9406fc50 msgr2=0x7f9b940c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b92d9d640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b9406fc50 0x7f9b940c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b92d9d640 1 -- 172.21.15.37:0/476548586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b940b1c00 con 0x7f9b940cfd80 2024-04-25T08:57:59.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b9259c640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406b1d0 0x7f9b9414fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:59.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b92d9d640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b940cfd80 0x7f9b941fa0f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c730 tx=0x7f9b8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b82ffd640 1 -- 172.21.15.37:0/476548586 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c003b80 con 0x7f9b940cfd80 2024-04-25T08:57:59.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b82ffd640 1 -- 172.21.15.37:0/476548586 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c003d20 con 0x7f9b940cfd80 2024-04-25T08:57:59.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b98e55640 1 -- 172.21.15.37:0/476548586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b940b1ea0 con 0x7f9b940cfd80 2024-04-25T08:57:59.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b82ffd640 1 -- 172.21.15.37:0/476548586 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00aa30 con 0x7f9b940cfd80 2024-04-25T08:57:59.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b98e55640 1 -- 172.21.15.37:0/476548586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b941fadf0 con 0x7f9b940cfd80 2024-04-25T08:57:59.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b98e55640 1 -- 172.21.15.37:0/476548586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b940b2350 con 0x7f9b940cfd80 2024-04-25T08:57:59.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b82ffd640 1 -- 172.21.15.37:0/476548586 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9b8c00abd0 con 0x7f9b940cfd80 2024-04-25T08:57:59.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b82ffd640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b74066f90 0x7f9b74069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b82ffd640 1 -- 172.21.15.37:0/476548586 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9b8c084fc0 con 0x7f9b940cfd80 2024-04-25T08:57:59.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.781+0000 7f9b9259c640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b74066f90 0x7f9b74069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.785+0000 7f9b9259c640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b74066f90 0x7f9b74069450 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9b880af000 tx=0x7f9b880a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.789+0000 7f9b82ffd640 1 -- 172.21.15.37:0/476548586 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9b8c055e80 con 0x7f9b940cfd80 2024-04-25T08:57:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.977+0000 7f9b98e55640 1 -- 172.21.15.37:0/476548586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b9404efa0 con 0x7f9b940cfd80 2024-04-25T08:57:59.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.977+0000 7f9b82ffd640 1 -- 172.21.15.37:0/476548586 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f9b8c055780 con 0x7f9b940cfd80 2024-04-25T08:57:59.970 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:57:59.970 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":51,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:57:22.214792+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/3927528604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3927528604},{"type":"v1","addr":"172.21.15.37:6835","nonce":3927528604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:57:59.970 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 51 2024-04-25T08:57:59.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 -- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b74066f90 msgr2=0x7f9b74069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b74066f90 0x7f9b74069450 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9b880af000 tx=0x7f9b880a7000 comp rx=0 tx=0).stop 2024-04-25T08:57:59.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 -- 172.21.15.37:0/476548586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b940cfd80 msgr2=0x7f9b941fa0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b940cfd80 0x7f9b941fa0f0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c00c730 tx=0x7f9b8c00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 -- 172.21.15.37:0/476548586 shutdown_connections 2024-04-25T08:57:59.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9b74066f90 0x7f9b74069450 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b940cfd80 0x7f9b941fa0f0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9b9406fc50 0x7f9b940c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 --2- 172.21.15.37:0/476548586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9b9406b1d0 0x7f9b9414fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 -- 172.21.15.37:0/476548586 >> 172.21.15.37:0/476548586 conn(0x7f9b94099a70 msgr2=0x7f9b940aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 -- 172.21.15.37:0/476548586 shutdown_connections 2024-04-25T08:57:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:57:59.981+0000 7f9b98e55640 1 -- 172.21.15.37:0/476548586 wait complete. 2024-04-25T08:57:59.989 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T08:58:00.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde908f3640 1 Processor -- start 2024-04-25T08:58:00.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde908f3640 1 -- start start 2024-04-25T08:58:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde908f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c15d9e0 0x7fde8c06c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde908f3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fde8c066a80 0x7fde8c0cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde908f3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8c0ca3e0 con 0x7fde8c15ddc0 2024-04-25T08:58:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde908f3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8c06c630 con 0x7fde8c066a80 2024-04-25T08:58:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde908f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c06b6a0 con 0x7fde8c15d9e0 2024-04-25T08:58:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde89d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fde8c066a80 0x7fde8c0cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde8ad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fde8c15ddc0 0x7fde8c0cefc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48062/0 (socket says 172.21.15.37:48062) 2024-04-25T08:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde89d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fde8c066a80 0x7fde8c0cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42224/0 (socket says 172.21.15.37:42224) 2024-04-25T08:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde8ad9d640 1 -- 172.21.15.37:0/1272336970 learned_addr learned my addr 172.21.15.37:0/1272336970 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde8a59c640 1 --2- 172.21.15.37:0/1272336970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c15d9e0 0x7fde8c06c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde89d9b640 1 -- 172.21.15.37:0/1272336970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c15d9e0 msgr2=0x7fde8c06c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde89d9b640 1 --2- 172.21.15.37:0/1272336970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c15d9e0 0x7fde8c06c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde89d9b640 1 -- 172.21.15.37:0/1272336970 >> v1:172.21.15.37:6789/0 conn(0x7fde8c15ddc0 legacy=0x7fde8c0cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde8959a640 1 -- 172.21.15.37:0/1272336970 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975412757 0 0) 0x7fde8c0ca3e0 con 0x7fde8c15ddc0 2024-04-25T08:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde89d9b640 1 -- 172.21.15.37:0/1272336970 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c0ca6c0 con 0x7fde8c066a80 2024-04-25T08:58:00.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.113+0000 7fde8a59c640 1 --2- 172.21.15.37:0/1272336970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c15d9e0 0x7fde8c06c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde89d9b640 1 --2- 172.21.15.37:0/1272336970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c066a80 0x7fde8c0cbd30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fde800097c0 tx=0x7fde8001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26013a589d36d15d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8959a640 1 -- 172.21.15.37:0/1272336970 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800b2070 con 0x7fde8c066a80 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8959a640 1 -- 172.21.15.37:0/1272336970 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde800ad640 con 0x7fde8c066a80 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8959a640 1 -- 172.21.15.37:0/1272336970 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800ada10 con 0x7fde8c066a80 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 -- 172.21.15.37:0/1272336970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c066a80 msgr2=0x7fde8c0cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 --2- 172.21.15.37:0/1272336970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c066a80 0x7fde8c0cbd30 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fde800097c0 tx=0x7fde8001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 -- 172.21.15.37:0/1272336970 shutdown_connections 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 --2- 172.21.15.37:0/1272336970 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c066a80 0x7fde8c0cbd30 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 --2- 172.21.15.37:0/1272336970 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c15d9e0 0x7fde8c06c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 -- 172.21.15.37:0/1272336970 >> 172.21.15.37:0/1272336970 conn(0x7fde8c0cc670 msgr2=0x7fde8c0cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 -- 172.21.15.37:0/1272336970 shutdown_connections 2024-04-25T08:58:00.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 -- 172.21.15.37:0/1272336970 wait complete. 2024-04-25T08:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 Processor -- start 2024-04-25T08:58:00.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 -- start start 2024-04-25T08:58:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c066a80 0x7fde8c0bed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde8c15d9e0 0x7fde8c153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c15ddc0 0x7fde8c0c3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0cf850 con 0x7fde8c066a80 2024-04-25T08:58:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0cf6d0 con 0x7fde8c15d9e0 2024-04-25T08:58:00.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c06e020 con 0x7fde8c15ddc0 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c066a80 0x7fde8c0bed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c066a80 0x7fde8c0bed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42236/0 (socket says 172.21.15.37:42236) 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8a59c640 1 -- 172.21.15.37:0/1243452777 learned_addr learned my addr 172.21.15.37:0/1243452777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8a59c640 1 -- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c15ddc0 msgr2=0x7fde8c0c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8a59c640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c15ddc0 0x7fde8c0c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8a59c640 1 -- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde8c15d9e0 msgr2=0x7fde8c153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8a59c640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde8c15d9e0 0x7fde8c153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8a59c640 1 -- 172.21.15.37:0/1243452777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c155040 con 0x7fde8c066a80 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde8a59c640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c066a80 0x7fde8c0bed30 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00e810 tx=0x7fde7c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde7affd640 1 -- 172.21.15.37:0/1243452777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c017070 con 0x7fde8c066a80 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde7affd640 1 -- 172.21.15.37:0/1243452777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c00c8d0 con 0x7fde8c066a80 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde7affd640 1 -- 172.21.15.37:0/1243452777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00f040 con 0x7fde8c066a80 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 -- 172.21.15.37:0/1243452777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde8c155320 con 0x7fde8c066a80 2024-04-25T08:58:00.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.117+0000 7fde908f3640 1 -- 172.21.15.37:0/1243452777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde8c157970 con 0x7fde8c066a80 2024-04-25T08:58:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde7affd640 1 -- 172.21.15.37:0/1243452777 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde7c01a070 con 0x7fde8c066a80 2024-04-25T08:58:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde7affd640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde6c066f60 0x7fde6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde7affd640 1 -- 172.21.15.37:0/1243452777 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fde7c084df0 con 0x7fde8c066a80 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde89d9b640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde6c066f60 0x7fde6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde93a60000 1 Processor -- start 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde93a60000 1 -- start start 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde89d9b640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde6c066f60 0x7fde6c069420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fde800af000 tx=0x7fde80004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde93a60000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 0x555eefa546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde93a60000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa54c00 0x555eefa0a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde93a60000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefa0acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde93a60000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eef8f1ba0 con 0x555eefa0a8e0 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde93a60000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eef857090 con 0x555eefa54c00 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde93a60000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555eef7df160 con 0x555eefa542e0 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde8ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefa0acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde8ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefa0acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42248/0 (socket says 172.21.15.37:42248) 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde8ad9d640 1 -- 172.21.15.37:0/1071864443 learned_addr learned my addr 172.21.15.37:0/1071864443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde89d9b640 1 --2- 172.21.15.37:0/1071864443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa54c00 0x555eefa0a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde8a59c640 1 --2- 172.21.15.37:0/1071864443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 0x555eefa546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde8ad9d640 1 -- 172.21.15.37:0/1071864443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 msgr2=0x555eefa546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde8ad9d640 1 --2- 172.21.15.37:0/1071864443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 0x555eefa546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde8ad9d640 1 -- 172.21.15.37:0/1071864443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa54c00 msgr2=0x555eefa0a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde8ad9d640 1 --2- 172.21.15.37:0/1071864443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa54c00 0x555eefa0a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde8ad9d640 1 -- 172.21.15.37:0/1071864443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eef900c20 con 0x555eefa0a8e0 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde8ad9d640 1 --2- 172.21.15.37:0/1071864443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefa0acc0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fde84001c60 tx=0x7fde84002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6855b74805cdc2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde6bfff640 1 -- 172.21.15.37:0/1071864443 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8400c7e0 con 0x555eefa0a8e0 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde6bfff640 1 -- 172.21.15.37:0/1071864443 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8400c980 con 0x555eefa0a8e0 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.121+0000 7fde6bfff640 1 -- 172.21.15.37:0/1071864443 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8400cd50 con 0x555eefa0a8e0 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- 172.21.15.37:0/1071864443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 msgr2=0x555eefa0acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 --2- 172.21.15.37:0/1071864443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefa0acc0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fde84001c60 tx=0x7fde84002120 comp rx=0 tx=0).stop 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- 172.21.15.37:0/1071864443 shutdown_connections 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 --2- 172.21.15.37:0/1071864443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefa0acc0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 --2- 172.21.15.37:0/1071864443 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa54c00 0x555eefa0a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 --2- 172.21.15.37:0/1071864443 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 0x555eefa546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- 172.21.15.37:0/1071864443 >> 172.21.15.37:0/1071864443 conn(0x555eefa06890 msgr2=0x555eefa08c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- 172.21.15.37:0/1071864443 shutdown_connections 2024-04-25T08:58:00.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- 172.21.15.37:0/1071864443 wait complete. 2024-04-25T08:58:00.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- start start 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 Processor -- start 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa0a8e0 0x555eefa5b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 0x555eefa5bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa54c00 0x555eefa42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eef853130 con 0x555eefa54c00 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eef81b130 con 0x555eefa0a8e0 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555eef915300 con 0x555eefa542e0 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 10 client.0 _ll_register_callbacks cb 0x555eef9f5bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa54c00 0x555eefa42090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa54c00 0x555eefa42090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42262/0 (socket says 172.21.15.37:42262) 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8ad9d640 1 -- 172.21.15.37:0/4223946 learned_addr learned my addr 172.21.15.37:0/4223946 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde89d9b640 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 0x555eefa5bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8a59c640 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa0a8e0 0x555eefa5b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8ad9d640 1 -- 172.21.15.37:0/4223946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 msgr2=0x555eefa5bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8ad9d640 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 0x555eefa5bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8ad9d640 1 -- 172.21.15.37:0/4223946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa0a8e0 msgr2=0x555eefa5b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8ad9d640 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa0a8e0 0x555eefa5b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8ad9d640 1 -- 172.21.15.37:0/4223946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555eef834060 con 0x555eefa54c00 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8a59c640 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa0a8e0 0x555eefa5b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8ad9d640 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa54c00 0x555eefa42090 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fde84000f90 tx=0x7fde84001410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daebbae8e8ccb366 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde697fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde697fa640 1 -- 172.21.15.37:0/4223946 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde84025060 con 0x555eefa54c00 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde697fa640 1 -- 172.21.15.37:0/4223946 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fde84015cb0 con 0x555eefa54c00 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- 172.21.15.37:0/4223946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde8400c980 con 0x555eefa54c00 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde697fa640 1 -- 172.21.15.37:0/4223946 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde840072e0 con 0x555eefa54c00 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 10 client.7416 fetch_fsmap learned FSMap version 51 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- 172.21.15.37:0/4223946 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x555eef7f87c0 con 0x555eefa54c00 2024-04-25T08:58:00.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde697fa640 1 -- 172.21.15.37:0/4223946 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8400c980 con 0x555eefa54c00 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde697fa640 1 -- 172.21.15.37:0/4223946 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde84007480 con 0x555eefa54c00 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 10 client.7416 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 10 client.7416 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 20 client.7416 populate_metadata read hostname 'smithi037' 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x555eefa42e30 0x555eefa43210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 4 client.7416 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde93a60000 1 -- 172.21.15.37:0/4223946 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555eef7f2cd0 con 0x555eefa42e30 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8a59c640 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x555eefa42e30 0x555eefa43210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde697fa640 10 client.7416 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde8a59c640 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x555eefa42e30 0x555eefa43210 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63708493d25f7b61 server_cookie=649a044764b740f5 in_seq=0 out_seq=0 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde697fa640 1 -- 172.21.15.37:0/4223946 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555eef7f2cd0 con 0x555eefa42e30 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.125+0000 7fde697fa640 10 client.7416 handle_command_reply: tid=0 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 1 client.7416 shutdown 2024-04-25T08:58:00.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 1 -- 172.21.15.37:0/4223946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa54c00 msgr2=0x555eefa42090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa54c00 0x555eefa42090 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fde84000f90 tx=0x7fde84001410 comp rx=0 tx=0).stop 2024-04-25T08:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 1 -- 172.21.15.37:0/4223946 shutdown_connections 2024-04-25T08:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x555eefa42e30 0x555eefa43210 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa54c00 0x555eefa42090 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 0x555eefa5bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde697fa640 0 client.7416 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 1 --2- 172.21.15.37:0/4223946 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa0a8e0 0x555eefa5b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 1 -- 172.21.15.37:0/4223946 >> 172.21.15.37:0/4223946 conn(0x555eefa06890 msgr2=0x555eefa09510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 1 -- 172.21.15.37:0/4223946 shutdown_connections 2024-04-25T08:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 1 -- 172.21.15.37:0/4223946 wait complete. 2024-04-25T08:58:00.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.129+0000 7fde93a60000 20 client.7416 trim_cache size 0 max 16384 2024-04-25T08:58:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.137+0000 7fde93a60000 1 Processor -- start 2024-04-25T08:58:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.137+0000 7fde93a60000 1 -- start start 2024-04-25T08:58:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa06890 0x555eefab0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 0x555eefab0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c202230 con 0x555eefa0a8e0 2024-04-25T08:58:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c0c1e50 con 0x555eefa06890 2024-04-25T08:58:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c158900 con 0x555eefa542e0 2024-04-25T08:58:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42272/0 (socket says 172.21.15.37:42272) 2024-04-25T08:58:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 -- 172.21.15.37:0/2840388793 learned_addr learned my addr 172.21.15.37:0/2840388793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 -- 172.21.15.37:0/2840388793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 msgr2=0x555eefab0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 --2- 172.21.15.37:0/2840388793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 0x555eefab0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 -- 172.21.15.37:0/2840388793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa06890 msgr2=0x555eefab0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 --2- 172.21.15.37:0/2840388793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa06890 0x555eefab0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 -- 172.21.15.37:0/2840388793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eef812fc0 con 0x555eefa0a8e0 2024-04-25T08:58:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 --2- 172.21.15.37:0/2840388793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab0a40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fde80097e60 tx=0x7fde800a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed111aacc79bc0ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde6b7fe640 1 -- 172.21.15.37:0/2840388793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800b2040 con 0x555eefa0a8e0 2024-04-25T08:58:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde6b7fe640 1 -- 172.21.15.37:0/2840388793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde800c9070 con 0x555eefa0a8e0 2024-04-25T08:58:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde6b7fe640 1 -- 172.21.15.37:0/2840388793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800ade50 con 0x555eefa0a8e0 2024-04-25T08:58:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- 172.21.15.37:0/2840388793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 msgr2=0x555eefab0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 --2- 172.21.15.37:0/2840388793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab0a40 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fde80097e60 tx=0x7fde800a7000 comp rx=0 tx=0).stop 2024-04-25T08:58:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- 172.21.15.37:0/2840388793 shutdown_connections 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 --2- 172.21.15.37:0/2840388793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa542e0 0x555eefab0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 --2- 172.21.15.37:0/2840388793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab0a40 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 --2- 172.21.15.37:0/2840388793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa06890 0x555eefab0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- 172.21.15.37:0/2840388793 >> 172.21.15.37:0/2840388793 conn(0x555eefa0fba0 msgr2=0x555eefa53290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- 172.21.15.37:0/2840388793 shutdown_connections 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- 172.21.15.37:0/2840388793 wait complete. 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- start start 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 Processor -- start 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa06890 0x555eefab4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa542e0 0x555eefa589b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eef7f1e70 con 0x555eefa0a8e0 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eef7dfa50 con 0x555eefa542e0 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555eef8563a0 con 0x555eefa06890 2024-04-25T08:58:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde93a60000 10 client.0 _ll_register_callbacks cb 0x555eef803240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42282/0 (socket says 172.21.15.37:42282) 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 -- 172.21.15.37:0/3350707153 learned_addr learned my addr 172.21.15.37:0/3350707153 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 -- 172.21.15.37:0/3350707153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa06890 msgr2=0x555eefab4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde8a59c640 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa06890 0x555eefab4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa06890 0x555eefab4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 -- 172.21.15.37:0/3350707153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa542e0 msgr2=0x555eefa589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde8ad9d640 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa542e0 0x555eefa589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa542e0 0x555eefa589b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 -- 172.21.15.37:0/3350707153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555eef8b09e0 con 0x555eefa0a8e0 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde89d9b640 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab4e70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fde8001ab50 tx=0x7fde8001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864a7f29709d53ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.141+0000 7fde697fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde697fa640 1 -- 172.21.15.37:0/3350707153 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde800b2070 con 0x555eefa0a8e0 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde697fa640 1 -- 172.21.15.37:0/3350707153 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fde80004230 con 0x555eefa0a8e0 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 -- 172.21.15.37:0/3350707153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde800c9070 con 0x555eefa0a8e0 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde697fa640 1 -- 172.21.15.37:0/3350707153 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde800da050 con 0x555eefa0a8e0 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 10 client.7422 fetch_fsmap learned FSMap version 51 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 -- 172.21.15.37:0/3350707153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x555eef7f6630 con 0x555eefa0a8e0 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde697fa640 1 -- 172.21.15.37:0/3350707153 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde800c9070 con 0x555eefa0a8e0 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde697fa640 1 -- 172.21.15.37:0/3350707153 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde800dd020 con 0x555eefa0a8e0 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 10 client.7422 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 10 client.7422 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 20 client.7422 populate_metadata read hostname 'smithi037' 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x555eefa54c00 0x555eefa59310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 4 client.7422 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 -- 172.21.15.37:0/3350707153 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x555eef8645d0 con 0x555eefa54c00 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde8a59c640 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x555eefa54c00 0x555eefa59310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde697fa640 10 client.7422 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde8a59c640 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x555eefa54c00 0x555eefa59310 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d23fb72563d797 server_cookie=6dfd638d7531e878 in_seq=0 out_seq=0 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde697fa640 1 -- 172.21.15.37:0/3350707153 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+236 (crc 0 0 0) 0x555eef8645d0 con 0x555eefa54c00 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde697fa640 10 client.7422 handle_command_reply: tid=0 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 client.7422 shutdown 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 -- 172.21.15.37:0/3350707153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 msgr2=0x555eefab4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab4e70 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fde8001ab50 tx=0x7fde8001ace0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 -- 172.21.15.37:0/3350707153 shutdown_connections 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x555eefa54c00 0x555eefa59310 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555eefa542e0 0x555eefa589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555eefa0a8e0 0x555eefab4e70 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 --2- 172.21.15.37:0/3350707153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555eefa06890 0x555eefab4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde697fa640 0 client.7422 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 -- 172.21.15.37:0/3350707153 >> 172.21.15.37:0/3350707153 conn(0x555eefa0fba0 msgr2=0x555eefa53b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 -- 172.21.15.37:0/3350707153 shutdown_connections 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 1 -- 172.21.15.37:0/3350707153 wait complete. 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde93a60000 20 client.7422 trim_cache size 0 max 16384 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "dentry", 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 586362977, 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627776, 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stdout: "frag": "*", 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stdout: "dname": "file_to_be_damaged", 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stdout: "snap_id": "head", 2024-04-25T08:58:00.141 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/subdir/file_to_be_damaged" 2024-04-25T08:58:00.142 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:00.142 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 -- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde6c066f60 msgr2=0x7fde6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde6c066f60 0x7fde6c069420 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fde800af000 tx=0x7fde80004740 comp rx=0 tx=0).stop 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 -- 172.21.15.37:0/1243452777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c066a80 msgr2=0x7fde8c0bed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c066a80 0x7fde8c0bed30 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00e810 tx=0x7fde7c00ecd0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 -- 172.21.15.37:0/1243452777 shutdown_connections 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde6c066f60 0x7fde6c069420 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde8c15ddc0 0x7fde8c0c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde8c15d9e0 0x7fde8c153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 --2- 172.21.15.37:0/1243452777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde8c066a80 0x7fde8c0bed30 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 -- 172.21.15.37:0/1243452777 >> 172.21.15.37:0/1243452777 conn(0x7fde8c0cc670 msgr2=0x7fde8c0b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 -- 172.21.15.37:0/1243452777 shutdown_connections 2024-04-25T08:58:00.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.145+0000 7fde908f3640 1 -- 172.21.15.37:0/1243452777 wait complete. 2024-04-25T08:58:00.150 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:58:00.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0fa53640 1 Processor -- start 2024-04-25T08:58:00.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0fa53640 1 -- start start 2024-04-25T08:58:00.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0fa53640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d080b2720 0x7f3d080b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0fa53640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d080b2ca0 con 0x7f3d080cf560 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0fa53640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d080b2f40 con 0x7f3d0815a600 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0fa53640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d080b37d0 con 0x7f3d080b2720 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0dff0640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3d080cf560 0x7f3d080ac1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48072/0 (socket says 172.21.15.37:48072) 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0dff0640 1 -- 172.21.15.37:0/2740507741 learned_addr learned my addr 172.21.15.37:0/2740507741 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0cfee640 1 --2- 172.21.15.37:0/2740507741 >> v2:172.21.15.203:3300/0 conn(0x7f3d080b2720 0x7f3d080b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987001272 0 0) 0x7f3d080b2ca0 con 0x7f3d080cf560 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf4005740 con 0x7f3d080cf560 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 316945899 0 0) 0x7f3cf4005740 con 0x7f3d080cf560 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 >> v2:172.21.15.203:3300/0 conn(0x7f3d080b2720 msgr2=0x7f3d080b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 --2- 172.21.15.37:0/2740507741 >> v2:172.21.15.203:3300/0 conn(0x7f3d080b2720 0x7f3d080b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 >> v1:172.21.15.203:6789/0 conn(0x7f3d0815a600 legacy=0x7f3d0806e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d080b31e0 con 0x7f3d080cf560 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3d00001a00 con 0x7f3d080cf560 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 >> v1:172.21.15.37:6789/0 conn(0x7f3d080cf560 legacy=0x7f3d080ac1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 --2- 172.21.15.37:0/2740507741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 0x7f3cf4006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 --2- 172.21.15.37:0/2740507741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cf4006a20 0x7f3cf4006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 --2- 172.21.15.37:0/2740507741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cf40073e0 0x7f3cf400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d080b37d0 con 0x7f3cf4005c40 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf400e310 con 0x7f3cf40073e0 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf400e510 con 0x7f3cf4006a20 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0d7ef640 1 --2- 172.21.15.37:0/2740507741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 0x7f3cf4006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0d7ef640 1 -- 172.21.15.37:0/2740507741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cf4006a20 msgr2=0x7f3cf4006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0d7ef640 1 --2- 172.21.15.37:0/2740507741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cf4006a20 0x7f3cf4006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0d7ef640 1 -- 172.21.15.37:0/2740507741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cf40073e0 msgr2=0x7f3cf400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0d7ef640 1 --2- 172.21.15.37:0/2740507741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cf40073e0 0x7f3cf400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0d7ef640 1 -- 172.21.15.37:0/2740507741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf4005740 con 0x7f3cf4005c40 2024-04-25T08:58:00.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.265+0000 7f3d0dff0640 1 -- 172.21.15.37:0/2740507741 reap_dead start 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0d7ef640 1 --2- 172.21.15.37:0/2740507741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 0x7f3cf4006060 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8001750 tx=0x7f3cf801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59ee5ebcf9901b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80013a0 con 0x7f3cf4005c40 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf80052d0 con 0x7f3cf4005c40 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3cfffff640 1 -- 172.21.15.37:0/2740507741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80054c0 con 0x7f3cf4005c40 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2740507741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 msgr2=0x7f3cf4006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 --2- 172.21.15.37:0/2740507741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 0x7f3cf4006060 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8001750 tx=0x7f3cf801c3e0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2740507741 shutdown_connections 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 --2- 172.21.15.37:0/2740507741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 0x7f3cf4006060 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2740507741 >> 172.21.15.37:0/2740507741 conn(0x7f3d0806ae60 msgr2=0x7f3d0806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2740507741 shutdown_connections 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2740507741 wait complete. 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 Processor -- start 2024-04-25T08:58:00.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 -- start start 2024-04-25T08:58:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 0x7f3d081f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d0815b5b0 0x7f3d0815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d081553c0 0x7f3d08155840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0806eec0 con 0x7f3cf4005c40 2024-04-25T08:58:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0806ed40 con 0x7f3d0815b5b0 2024-04-25T08:58:00.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.269+0000 7f3d0fa53640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d080ac1e0 con 0x7f3d081553c0 2024-04-25T08:58:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0d7ef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 0x7f3d081f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0d7ef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 0x7f3d081f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42296/0 (socket says 172.21.15.37:42296) 2024-04-25T08:58:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0d7ef640 1 -- 172.21.15.37:0/2897968432 learned_addr learned my addr 172.21.15.37:0/2897968432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0dff0640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d081553c0 0x7f3d08155840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0cfee640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d0815b5b0 0x7f3d0815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0d7ef640 1 -- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d081553c0 msgr2=0x7f3d08155840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0d7ef640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d081553c0 0x7f3d08155840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0d7ef640 1 -- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d0815b5b0 msgr2=0x7f3d0815ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0d7ef640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d0815b5b0 0x7f3d0815ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0d7ef640 1 -- 172.21.15.37:0/2897968432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0815c050 con 0x7f3cf4005c40 2024-04-25T08:58:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0d7ef640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 0x7f3d081f53f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8001750 tx=0x7f3cf4007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3cfdffb640 1 -- 172.21.15.37:0/2897968432 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80013a0 con 0x7f3cf4005c40 2024-04-25T08:58:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3cfdffb640 1 -- 172.21.15.37:0/2897968432 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf8005480 con 0x7f3cf4005c40 2024-04-25T08:58:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2897968432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0815c330 con 0x7f3cf4005c40 2024-04-25T08:58:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3cfdffb640 1 -- 172.21.15.37:0/2897968432 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8007a60 con 0x7f3cf4005c40 2024-04-25T08:58:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2897968432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d080c2f00 con 0x7f3cf4005c40 2024-04-25T08:58:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2897968432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0804efa0 con 0x7f3cf4005c40 2024-04-25T08:58:00.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3cfdffb640 1 -- 172.21.15.37:0/2897968432 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3cf8006a20 con 0x7f3cf4005c40 2024-04-25T08:58:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3cfdffb640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cec066f60 0x7f3cec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3cfdffb640 1 -- 172.21.15.37:0/2897968432 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3cf811f700 con 0x7f3cf4005c40 2024-04-25T08:58:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0cfee640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cec066f60 0x7f3cec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.273+0000 7f3d0cfee640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cec066f60 0x7f3cec069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3d0806b520 tx=0x7f3d04008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.277+0000 7f3cfdffb640 1 -- 172.21.15.37:0/2897968432 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3cf80b8020 con 0x7f3cf4005c40 2024-04-25T08:58:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.465+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2897968432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3d080af8b0 con 0x7f3cf4005c40 2024-04-25T08:58:00.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.465+0000 7f3cfdffb640 1 -- 172.21.15.37:0/2897968432 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f3cf80160f0 con 0x7f3cf4005c40 2024-04-25T08:58:00.457 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:00.457 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T08:58:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cec066f60 msgr2=0x7f3cec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cec066f60 0x7f3cec069420 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f3d0806b520 tx=0x7f3d04008040 comp rx=0 tx=0).stop 2024-04-25T08:58:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2897968432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 msgr2=0x7f3d081f53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 0x7f3d081f53f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8001750 tx=0x7f3cf4007a50 comp rx=0 tx=0).stop 2024-04-25T08:58:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2897968432 shutdown_connections 2024-04-25T08:58:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cec066f60 0x7f3cec069420 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d081553c0 0x7f3d08155840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d0815b5b0 0x7f3d0815ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 --2- 172.21.15.37:0/2897968432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cf4005c40 0x7f3d081f53f0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2897968432 >> 172.21.15.37:0/2897968432 conn(0x7f3d0806ae60 msgr2=0x7f3d0815e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2897968432 shutdown_connections 2024-04-25T08:58:00.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.469+0000 7f3d0fa53640 1 -- 172.21.15.37:0/2897968432 wait complete. 2024-04-25T08:58:00.474 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T08:58:00.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:00.474 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-04-25T08:58:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.613+0000 7f2dd21d7640 1 Processor -- start 2024-04-25T08:58:00.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.613+0000 7f2dd21d7640 1 -- start start 2024-04-25T08:58:00.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.613+0000 7f2dd21d7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2dcc0b36d0 0x7f2dcc0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.613+0000 7f2dd21d7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2dcc0cd000 0x7f2dcc0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.613+0000 7f2dd21d7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dcc0b3bd0 con 0x7f2dcc159f20 2024-04-25T08:58:00.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.613+0000 7f2dd21d7640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc0b3eb0 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.613+0000 7f2dd21d7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc0b4190 con 0x7f2dcc0cd000 2024-04-25T08:58:00.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.613+0000 7f2dcb7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2dcc159f20 0x7f2dcc06d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48078/0 (socket says 172.21.15.37:48078) 2024-04-25T08:58:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.613+0000 7f2dcb7fe640 1 -- 172.21.15.37:0/648595174 learned_addr learned my addr 172.21.15.37:0/648595174 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.613+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/648595174 >> v2:172.21.15.37:3300/0 conn(0x7f2dcc0b36d0 0x7f2dcc0b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dcbfff640 1 --2- 172.21.15.37:0/648595174 >> v2:172.21.15.203:3300/0 conn(0x7f2dcc0cd000 0x7f2dcc0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dca7fc640 1 -- 172.21.15.37:0/648595174 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661090991 0 0) 0x7f2dcc0b3bd0 con 0x7f2dcc159f20 2024-04-25T08:58:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dca7fc640 1 -- 172.21.15.37:0/648595174 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2db4003530 con 0x7f2dcc159f20 2024-04-25T08:58:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dcaffd640 1 -- 172.21.15.37:0/648595174 >> v2:172.21.15.203:3300/0 conn(0x7f2dcc0cd000 msgr2=0x7f2dcc0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/648595174 >> v2:172.21.15.203:3300/0 conn(0x7f2dcc0cd000 0x7f2dcc0b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dcaffd640 1 -- 172.21.15.37:0/648595174 >> v1:172.21.15.37:6789/0 conn(0x7f2dcc159f20 legacy=0x7f2dcc06d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dcaffd640 1 -- 172.21.15.37:0/648595174 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc06cf80 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/648595174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0b36d0 0x7f2dcc0b2190 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0d0350 tx=0x7f2dc001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4638f9ee745e9912 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dca7fc640 1 -- 172.21.15.37:0/648595174 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00b0070 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dca7fc640 1 -- 172.21.15.37:0/648595174 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc00ab600 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dca7fc640 1 -- 172.21.15.37:0/648595174 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00ab9b0 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dd21d7640 1 -- 172.21.15.37:0/648595174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0b36d0 msgr2=0x7f2dcc0b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dd21d7640 1 --2- 172.21.15.37:0/648595174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0b36d0 0x7f2dcc0b2190 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0d0350 tx=0x7f2dc001c450 comp rx=0 tx=0).stop 2024-04-25T08:58:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dd21d7640 1 -- 172.21.15.37:0/648595174 shutdown_connections 2024-04-25T08:58:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dd21d7640 1 --2- 172.21.15.37:0/648595174 >> v2:172.21.15.203:3300/0 conn(0x7f2dcc0cd000 0x7f2dcc0b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dd21d7640 1 --2- 172.21.15.37:0/648595174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0b36d0 0x7f2dcc0b2190 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dd21d7640 1 -- 172.21.15.37:0/648595174 >> 172.21.15.37:0/648595174 conn(0x7f2dcc06ae60 msgr2=0x7f2dcc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dd21d7640 1 -- 172.21.15.37:0/648595174 shutdown_connections 2024-04-25T08:58:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.617+0000 7f2dd21d7640 1 -- 172.21.15.37:0/648595174 wait complete. 2024-04-25T08:58:00.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dd21d7640 1 Processor -- start 2024-04-25T08:58:00.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dd21d7640 1 -- start start 2024-04-25T08:58:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dd21d7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0b36d0 0x7f2dcc098810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dd21d7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dcc0cd000 0x7f2dcc0c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dd21d7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2dcc159f20 0x7f2dcc150ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dd21d7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0cbd30 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dd21d7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0cbbb0 con 0x7f2dcc0cd000 2024-04-25T08:58:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dd21d7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc0cbeb0 con 0x7f2dcc159f20 2024-04-25T08:58:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0b36d0 0x7f2dcc098810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0b36d0 0x7f2dcc098810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42310/0 (socket says 172.21.15.37:42310) 2024-04-25T08:58:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcb7fe640 1 -- 172.21.15.37:0/1291806874 learned_addr learned my addr 172.21.15.37:0/1291806874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dcc0cd000 0x7f2dcc0c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcbfff640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2dcc159f20 0x7f2dcc150ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcb7fe640 1 -- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2dcc159f20 msgr2=0x7f2dcc150ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcb7fe640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2dcc159f20 0x7f2dcc150ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcb7fe640 1 -- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dcc0cd000 msgr2=0x7f2dcc0c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcb7fe640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dcc0cd000 0x7f2dcc0c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcb7fe640 1 -- 172.21.15.37:0/1291806874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc098d50 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dcc0cd000 0x7f2dcc0c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dcb7fe640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0b36d0 0x7f2dcc098810 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00c3f0 tx=0x7f2dbc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2db3fff640 1 -- 172.21.15.37:0/1291806874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc017070 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2db3fff640 1 -- 172.21.15.37:0/1291806874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc005cf0 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dd21d7640 1 -- 172.21.15.37:0/1291806874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc153bf0 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2db3fff640 1 -- 172.21.15.37:0/1291806874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc0072f0 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2dd21d7640 1 -- 172.21.15.37:0/1291806874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc0ca670 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.621+0000 7f2db3fff640 1 -- 172.21.15.37:0/1291806874 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2dbc01a070 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2db3fff640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2db8066f90 0x7f2db8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2db3fff640 1 -- 172.21.15.37:0/1291806874 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2dbc084750 con 0x7f2dcc0b36d0 2024-04-25T08:58:00.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2db8066f90 0x7f2db8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dd5344000 1 Processor -- start 2024-04-25T08:58:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dd5344000 1 -- start start 2024-04-25T08:58:00.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2db8066f90 0x7f2db8069450 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00ad000 tx=0x7f2dc0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69f421a0 0x556f69f42580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69ef84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69ef8a00 0x556f69ef8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dd5344000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f69d42c40 con 0x556f69f42ac0 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dd5344000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f69ccb1d0 con 0x556f69ef8a00 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dd5344000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f69d3ece0 con 0x556f69f421a0 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69ef84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69ef84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42314/0 (socket says 172.21.15.37:42314) 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcaffd640 1 -- 172.21.15.37:0/2198759169 learned_addr learned my addr 172.21.15.37:0/2198759169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcbfff640 1 --2- 172.21.15.37:0/2198759169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69ef8a00 0x556f69ef8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcb7fe640 1 --2- 172.21.15.37:0/2198759169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69f421a0 0x556f69f42580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcaffd640 1 -- 172.21.15.37:0/2198759169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69f421a0 msgr2=0x556f69f42580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/2198759169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69f421a0 0x556f69f42580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcaffd640 1 -- 172.21.15.37:0/2198759169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69ef8a00 msgr2=0x556f69ef8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/2198759169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69ef8a00 0x556f69ef8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcaffd640 1 -- 172.21.15.37:0/2198759169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f69d882a0 con 0x556f69f42ac0 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/2198759169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69ef84c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0008010 tx=0x7f2dc00a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5d4a425bf7db6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2db17fa640 1 -- 172.21.15.37:0/2198759169 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00c7070 con 0x556f69f42ac0 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2db17fa640 1 -- 172.21.15.37:0/2198759169 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc00b1040 con 0x556f69f42ac0 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2db17fa640 1 -- 172.21.15.37:0/2198759169 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00b7830 con 0x556f69f42ac0 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.625+0000 7f2dd5344000 1 -- 172.21.15.37:0/2198759169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 msgr2=0x556f69ef84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 --2- 172.21.15.37:0/2198759169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69ef84c0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0008010 tx=0x7f2dc00a6000 comp rx=0 tx=0).stop 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 -- 172.21.15.37:0/2198759169 shutdown_connections 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 --2- 172.21.15.37:0/2198759169 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69ef8a00 0x556f69ef8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 --2- 172.21.15.37:0/2198759169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69ef84c0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 --2- 172.21.15.37:0/2198759169 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69f421a0 0x556f69f42580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 -- 172.21.15.37:0/2198759169 >> 172.21.15.37:0/2198759169 conn(0x556f69ef05a0 msgr2=0x556f69ef2990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 -- 172.21.15.37:0/2198759169 shutdown_connections 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 -- 172.21.15.37:0/2198759169 wait complete. 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 -- start start 2024-04-25T08:58:00.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 Processor -- start 2024-04-25T08:58:00.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69ef8a00 0x556f69f49500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f421a0 0x556f69f49a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69f42ac0 0x556f69f25db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f69cfda90 con 0x556f69ef8a00 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f69cf9d60 con 0x556f69f421a0 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f69d0eb50 con 0x556f69f42ac0 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 10 client.0 _ll_register_callbacks cb 0x556f69edf410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69ef8a00 0x556f69f49500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69ef8a00 0x556f69f49500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42316/0 (socket says 172.21.15.37:42316) 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcb7fe640 1 -- 172.21.15.37:0/1595696329 learned_addr learned my addr 172.21.15.37:0/1595696329 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f421a0 0x556f69f49a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcb7fe640 1 -- 172.21.15.37:0/1595696329 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69f42ac0 msgr2=0x556f69f25db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcb7fe640 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69f42ac0 0x556f69f25db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcb7fe640 1 -- 172.21.15.37:0/1595696329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f421a0 msgr2=0x556f69f49a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcb7fe640 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f421a0 0x556f69f49a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcb7fe640 1 -- 172.21.15.37:0/1595696329 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f69cfb730 con 0x556f69ef8a00 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcb7fe640 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69ef8a00 0x556f69f49500 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc054ce0 tx=0x7f2dbc058ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792536498f8b1241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2daa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2daa7fc640 1 -- 172.21.15.37:0/1595696329 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc06c070 con 0x556f69ef8a00 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2daa7fc640 1 -- 172.21.15.37:0/1595696329 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2dbc07fca0 con 0x556f69ef8a00 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 -- 172.21.15.37:0/1595696329 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2dc00b1040 con 0x556f69ef8a00 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2daa7fc640 1 -- 172.21.15.37:0/1595696329 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dbc082e60 con 0x556f69ef8a00 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 10 client.7443 fetch_fsmap learned FSMap version 51 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 -- 172.21.15.37:0/1595696329 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x556f69ccb420 con 0x556f69ef8a00 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2daa7fc640 1 -- 172.21.15.37:0/1595696329 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc00b1040 con 0x556f69ef8a00 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2daa7fc640 1 -- 172.21.15.37:0/1595696329 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2dbc06cb70 con 0x556f69ef8a00 2024-04-25T08:58:00.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 10 client.7443 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 10 client.7443 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 20 client.7443 populate_metadata read hostname 'smithi037' 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556f69f26af0 0x556f69f26ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 4 client.7443 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 -- 172.21.15.37:0/1595696329 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f69cdeb50 con 0x556f69f26af0 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556f69f26af0 0x556f69f26ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556f69f26af0 0x556f69f26ed0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45467f5319c282a server_cookie=8601d4b0ac602185 in_seq=0 out_seq=0 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2daa7fc640 10 client.7443 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2daa7fc640 1 -- 172.21.15.37:0/1595696329 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556f69cdeb50 con 0x556f69f26af0 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2daa7fc640 10 client.7443 handle_command_reply: tid=0 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.629+0000 7f2dd5344000 1 client.7443 shutdown 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2dd5344000 1 -- 172.21.15.37:0/1595696329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69ef8a00 msgr2=0x556f69f49500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2dd5344000 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69ef8a00 0x556f69f49500 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc054ce0 tx=0x7f2dbc058ea0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2dd5344000 1 -- 172.21.15.37:0/1595696329 shutdown_connections 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2dd5344000 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556f69f26af0 0x556f69f26ed0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2dd5344000 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69f42ac0 0x556f69f25db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2daa7fc640 0 client.7443 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2dd5344000 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f421a0 0x556f69f49a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2dd5344000 1 --2- 172.21.15.37:0/1595696329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69ef8a00 0x556f69f49500 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2dd5344000 1 -- 172.21.15.37:0/1595696329 >> 172.21.15.37:0/1595696329 conn(0x556f69ef05a0 msgr2=0x556f69ef9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2dd5344000 1 -- 172.21.15.37:0/1595696329 shutdown_connections 2024-04-25T08:58:00.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2dd5344000 1 -- 172.21.15.37:0/1595696329 wait complete. 2024-04-25T08:58:00.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.633+0000 7f2dd5344000 20 client.7443 trim_cache size 0 max 16384 2024-04-25T08:58:00.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.645+0000 7f2dd5344000 1 Processor -- start 2024-04-25T08:58:00.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.645+0000 7f2dd5344000 1 -- start start 2024-04-25T08:58:00.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69ef8a00 0x556f69ef8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f421a0 0x556f69ef9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f42ac0 0x556f69f24970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc1fee90 con 0x556f69f421a0 2024-04-25T08:58:00.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc1686f0 con 0x556f69f42ac0 2024-04-25T08:58:00.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc1524a0 con 0x556f69ef8a00 2024-04-25T08:58:00.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f421a0 0x556f69ef9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f421a0 0x556f69ef9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42330/0 (socket says 172.21.15.37:42330) 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcaffd640 1 -- 172.21.15.37:0/3243214616 learned_addr learned my addr 172.21.15.37:0/3243214616 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcb7fe640 1 --2- 172.21.15.37:0/3243214616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69ef8a00 0x556f69ef8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcbfff640 1 --2- 172.21.15.37:0/3243214616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f42ac0 0x556f69f24970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcaffd640 1 -- 172.21.15.37:0/3243214616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69ef8a00 msgr2=0x556f69ef8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/3243214616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69ef8a00 0x556f69ef8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcaffd640 1 -- 172.21.15.37:0/3243214616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f42ac0 msgr2=0x556f69f24970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/3243214616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f42ac0 0x556f69f24970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcaffd640 1 -- 172.21.15.37:0/3243214616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f69dc7550 con 0x556f69f421a0 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcbfff640 1 --2- 172.21.15.37:0/3243214616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f42ac0 0x556f69f24970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/3243214616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f421a0 0x556f69ef9320 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2dc001c930 tx=0x7f2dc0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87cbfb74a2d6e09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2db0ff9640 1 -- 172.21.15.37:0/3243214616 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00d9070 con 0x556f69f421a0 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2db0ff9640 1 -- 172.21.15.37:0/3243214616 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc00b1040 con 0x556f69f421a0 2024-04-25T08:58:00.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2db0ff9640 1 -- 172.21.15.37:0/3243214616 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00c7040 con 0x556f69f421a0 2024-04-25T08:58:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- 172.21.15.37:0/3243214616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f421a0 msgr2=0x556f69ef9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 --2- 172.21.15.37:0/3243214616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f421a0 0x556f69ef9320 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f2dc001c930 tx=0x7f2dc0004740 comp rx=0 tx=0).stop 2024-04-25T08:58:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- 172.21.15.37:0/3243214616 shutdown_connections 2024-04-25T08:58:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 --2- 172.21.15.37:0/3243214616 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f42ac0 0x556f69f24970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 --2- 172.21.15.37:0/3243214616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f421a0 0x556f69ef9320 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 --2- 172.21.15.37:0/3243214616 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69ef8a00 0x556f69ef8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- 172.21.15.37:0/3243214616 >> 172.21.15.37:0/3243214616 conn(0x556f69ef05a0 msgr2=0x556f69f4fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- 172.21.15.37:0/3243214616 shutdown_connections 2024-04-25T08:58:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- 172.21.15.37:0/3243214616 wait complete. 2024-04-25T08:58:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- start start 2024-04-25T08:58:00.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 Processor -- start 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69ef8a00 0x556f69fa3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f421a0 0x556f69fa40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69f44eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f69ce98b0 con 0x556f69f42ac0 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc1680f0 con 0x556f69f421a0 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f69cfda90 con 0x556f69ef8a00 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dd5344000 10 client.0 _ll_register_callbacks cb 0x7f2dc00ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69f44eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69f44eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42346/0 (socket says 172.21.15.37:42346) 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcbfff640 1 -- 172.21.15.37:0/2774219138 learned_addr learned my addr 172.21.15.37:0/2774219138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcaffd640 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f421a0 0x556f69fa40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcb7fe640 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69ef8a00 0x556f69fa3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcbfff640 1 -- 172.21.15.37:0/2774219138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69ef8a00 msgr2=0x556f69fa3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcbfff640 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69ef8a00 0x556f69fa3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcbfff640 1 -- 172.21.15.37:0/2774219138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f421a0 msgr2=0x556f69fa40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcbfff640 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f421a0 0x556f69fa40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcbfff640 1 -- 172.21.15.37:0/2774219138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f69cf5490 con 0x556f69f42ac0 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2dcbfff640 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69f44eb0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4007620 tx=0x7f2dc4007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e97b505b97e2fd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2daa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.649+0000 7f2daa7fc640 1 -- 172.21.15.37:0/2774219138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc401b020 con 0x556f69f42ac0 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2daa7fc640 1 -- 172.21.15.37:0/2774219138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2dc4017890 con 0x556f69f42ac0 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 -- 172.21.15.37:0/2774219138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2dc00b1040 con 0x556f69f42ac0 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2daa7fc640 1 -- 172.21.15.37:0/2774219138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2dc4025050 con 0x556f69f42ac0 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 10 client.7449 fetch_fsmap learned FSMap version 51 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 -- 172.21.15.37:0/2774219138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x556f69cea1b0 con 0x556f69f42ac0 2024-04-25T08:58:00.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2daa7fc640 1 -- 172.21.15.37:0/2774219138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc00b1040 con 0x556f69f42ac0 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2daa7fc640 1 -- 172.21.15.37:0/2774219138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2dc4017a90 con 0x556f69f42ac0 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 10 client.7449 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 10 client.7449 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 20 client.7449 populate_metadata read hostname 'smithi037' 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556f69f453f0 0x556f69f45800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 4 client.7449 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 -- 172.21.15.37:0/2774219138 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) v1 -- 0x556f69d50180 con 0x556f69f453f0 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dcb7fe640 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556f69f453f0 0x556f69f45800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2daa7fc640 10 client.7449 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dcb7fe640 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556f69f453f0 0x556f69f45800 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dce3d5e9f05606b5 server_cookie=c463e7a9646fcbc9 in_seq=0 out_seq=0 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2daa7fc640 1 -- 172.21.15.37:0/2774219138 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x556f69d50180 con 0x556f69f453f0 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2daa7fc640 10 client.7449 handle_command_reply: tid=0 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 client.7449 shutdown 2024-04-25T08:58:00.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 -- 172.21.15.37:0/2774219138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 msgr2=0x556f69f44eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69f44eb0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4007620 tx=0x7f2dc4007ae0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 -- 172.21.15.37:0/2774219138 shutdown_connections 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556f69f453f0 0x556f69f45800 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556f69f42ac0 0x556f69f44eb0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556f69f421a0 0x556f69fa40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 --2- 172.21.15.37:0/2774219138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556f69ef8a00 0x556f69fa3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 -- 172.21.15.37:0/2774219138 >> 172.21.15.37:0/2774219138 conn(0x556f69ef05a0 msgr2=0x556f69f209f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2daa7fc640 0 client.7449 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 -- 172.21.15.37:0/2774219138 shutdown_connections 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 1 -- 172.21.15.37:0/2774219138 wait complete. 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd5344000 20 client.7449 trim_cache size 0 max 16384 2024-04-25T08:58:00.649 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:58:00.650 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-04-25T08:58:00.650 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "405323a4-c6fa-4133-96bb-dc1b3c6a8b13", 2024-04-25T08:58:00.650 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-04-25T08:58:00.650 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 -- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2db8066f90 msgr2=0x7f2db8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2db8066f90 0x7f2db8069450 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00ad000 tx=0x7f2dc0008040 comp rx=0 tx=0).stop 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 -- 172.21.15.37:0/1291806874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0b36d0 msgr2=0x7f2dcc098810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0b36d0 0x7f2dcc098810 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00c3f0 tx=0x7f2dbc00c8b0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 -- 172.21.15.37:0/1291806874 shutdown_connections 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2db8066f90 0x7f2db8069450 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2dcc159f20 0x7f2dcc150ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2dcc0cd000 0x7f2dcc0c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 --2- 172.21.15.37:0/1291806874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2dcc0b36d0 0x7f2dcc098810 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 -- 172.21.15.37:0/1291806874 >> 172.21.15.37:0/1291806874 conn(0x7f2dcc06ae60 msgr2=0x7f2dcc0adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 -- 172.21.15.37:0/1291806874 shutdown_connections 2024-04-25T08:58:00.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.653+0000 7f2dd21d7640 1 -- 172.21.15.37:0/1291806874 wait complete. 2024-04-25T08:58:00.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:00.661 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:58:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.781+0000 7f55c3909640 1 Processor -- start 2024-04-25T08:58:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.781+0000 7f55c3909640 1 -- start start 2024-04-25T08:58:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c3909640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f55bc15d9c0 0x7f55bc0d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c3909640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55bc0cf270 0x7f55bc0d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c3909640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0d1950 con 0x7f55bc0d3c20 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c3909640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55bc0d2520 con 0x7f55bc15d9c0 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c3909640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0d1bf0 con 0x7f55bc0cf270 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c16a5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f55bc15d9c0 0x7f55bc0d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c16a5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f55bc15d9c0 0x7f55bc0d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42350/0 (socket says 172.21.15.37:42350) 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c16a5640 1 -- 172.21.15.37:0/1962324523 learned_addr learned my addr 172.21.15.37:0/1962324523 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c0ea4640 1 --2- 172.21.15.37:0/1962324523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55bc0cf270 0x7f55bc0d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c16a5640 1 -- 172.21.15.37:0/1962324523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55bc0cf270 msgr2=0x7f55bc0d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c16a5640 1 --2- 172.21.15.37:0/1962324523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55bc0cf270 0x7f55bc0d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c16a5640 1 -- 172.21.15.37:0/1962324523 >> v1:172.21.15.203:6789/0 conn(0x7f55bc0d3c20 legacy=0x7f55bc0d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c16a5640 1 -- 172.21.15.37:0/1962324523 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0d1ed0 con 0x7f55bc15d9c0 2024-04-25T08:58:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c0ea4640 1 --2- 172.21.15.37:0/1962324523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55bc0cf270 0x7f55bc0d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55c16a5640 1 --2- 172.21.15.37:0/1962324523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc15d9c0 0x7f55bc0d1320 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009e50 tx=0x7f55b401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e988d6a507683c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55b3fff640 1 -- 172.21.15.37:0/1962324523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40a5700 con 0x7f55bc15d9c0 2024-04-25T08:58:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55b3fff640 1 -- 172.21.15.37:0/1962324523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b40a58a0 con 0x7f55bc15d9c0 2024-04-25T08:58:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.785+0000 7f55b3fff640 1 -- 172.21.15.37:0/1962324523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40a5c70 con 0x7f55bc15d9c0 2024-04-25T08:58:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 -- 172.21.15.37:0/1962324523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc15d9c0 msgr2=0x7f55bc0d1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 --2- 172.21.15.37:0/1962324523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc15d9c0 0x7f55bc0d1320 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009e50 tx=0x7f55b401c9f0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 -- 172.21.15.37:0/1962324523 shutdown_connections 2024-04-25T08:58:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 --2- 172.21.15.37:0/1962324523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55bc0cf270 0x7f55bc0d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 --2- 172.21.15.37:0/1962324523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc15d9c0 0x7f55bc0d1320 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 -- 172.21.15.37:0/1962324523 >> 172.21.15.37:0/1962324523 conn(0x7f55bc09cad0 msgr2=0x7f55bc08b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 -- 172.21.15.37:0/1962324523 shutdown_connections 2024-04-25T08:58:00.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 -- 172.21.15.37:0/1962324523 wait complete. 2024-04-25T08:58:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 Processor -- start 2024-04-25T08:58:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 -- start start 2024-04-25T08:58:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55bc0cf270 0x7f55bc0c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55bc0d3c20 0x7f55bc0cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc15d9c0 0x7f55bc152d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc16a720 con 0x7f55bc15d9c0 2024-04-25T08:58:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc16a5a0 con 0x7f55bc0cf270 2024-04-25T08:58:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc16a8a0 con 0x7f55bc0d3c20 2024-04-25T08:58:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c1ea6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc15d9c0 0x7f55bc152d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c1ea6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc15d9c0 0x7f55bc152d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42354/0 (socket says 172.21.15.37:42354) 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c1ea6640 1 -- 172.21.15.37:0/2546516020 learned_addr learned my addr 172.21.15.37:0/2546516020 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c0ea4640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55bc0d3c20 0x7f55bc0cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c16a5640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55bc0cf270 0x7f55bc0c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c1ea6640 1 -- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55bc0d3c20 msgr2=0x7f55bc0cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c1ea6640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55bc0d3c20 0x7f55bc0cba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c1ea6640 1 -- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55bc0cf270 msgr2=0x7f55bc0c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c1ea6640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55bc0cf270 0x7f55bc0c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c1ea6640 1 -- 172.21.15.37:0/2546516020 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc154d70 con 0x7f55bc15d9c0 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c1ea6640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc15d9c0 0x7f55bc152d40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f55b800c3c0 tx=0x7f55b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55b1ffb640 1 -- 172.21.15.37:0/2546516020 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8011790 con 0x7f55bc15d9c0 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55b1ffb640 1 -- 172.21.15.37:0/2546516020 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b8017070 con 0x7f55bc15d9c0 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 -- 172.21.15.37:0/2546516020 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc153730 con 0x7f55bc15d9c0 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55b1ffb640 1 -- 172.21.15.37:0/2546516020 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b8011b70 con 0x7f55bc15d9c0 2024-04-25T08:58:00.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.789+0000 7f55c3909640 1 -- 172.21.15.37:0/2546516020 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc153cf0 con 0x7f55bc15d9c0 2024-04-25T08:58:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55b1ffb640 1 -- 172.21.15.37:0/2546516020 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55b8011d10 con 0x7f55bc15d9c0 2024-04-25T08:58:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55b1ffb640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55a0066f60 0x7f55a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55b1ffb640 1 -- 172.21.15.37:0/2546516020 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f55b8084f60 con 0x7f55bc15d9c0 2024-04-25T08:58:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c16a5640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55a0066f60 0x7f55a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c6a76000 1 Processor -- start 2024-04-25T08:58:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c6a76000 1 -- start start 2024-04-25T08:58:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c16a5640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55a0066f60 0x7f55a0069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f55b40ae000 tx=0x7f55b40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf659db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf65a2f0 0x556adf60fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf610480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c6a76000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556adf45cff0 con 0x556adf6599d0 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c6a76000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556adf3e5150 con 0x556adf6100a0 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c6a76000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x556adf459090 con 0x556adf65a2f0 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c16a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf659db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c16a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf659db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42370/0 (socket says 172.21.15.37:42370) 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c16a5640 1 -- 172.21.15.37:0/3909314954 learned_addr learned my addr 172.21.15.37:0/3909314954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c0ea4640 1 --2- 172.21.15.37:0/3909314954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf65a2f0 0x556adf60fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c16a5640 1 -- 172.21.15.37:0/3909314954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf65a2f0 msgr2=0x556adf60fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c16a5640 1 --2- 172.21.15.37:0/3909314954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf65a2f0 0x556adf60fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c16a5640 1 -- 172.21.15.37:0/3909314954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 msgr2=0x556adf610480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c1ea6640 1 --2- 172.21.15.37:0/3909314954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf610480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c16a5640 1 --2- 172.21.15.37:0/3909314954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf610480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c16a5640 1 -- 172.21.15.37:0/3909314954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556adf5f30d0 con 0x556adf6599d0 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55c16a5640 1 --2- 172.21.15.37:0/3909314954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf659db0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f55b401ca30 tx=0x7f55b4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad656bcf05826d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.793+0000 7f55a6ffd640 1 -- 172.21.15.37:0/3909314954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40c7070 con 0x556adf6599d0 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55a6ffd640 1 -- 172.21.15.37:0/3909314954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b40b6420 con 0x556adf6599d0 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55a6ffd640 1 -- 172.21.15.37:0/3909314954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40b67d0 con 0x556adf6599d0 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- 172.21.15.37:0/3909314954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 msgr2=0x556adf659db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 --2- 172.21.15.37:0/3909314954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf659db0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f55b401ca30 tx=0x7f55b4004ac0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- 172.21.15.37:0/3909314954 shutdown_connections 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 --2- 172.21.15.37:0/3909314954 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf610480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 --2- 172.21.15.37:0/3909314954 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf65a2f0 0x556adf60fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 --2- 172.21.15.37:0/3909314954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf659db0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- 172.21.15.37:0/3909314954 >> 172.21.15.37:0/3909314954 conn(0x556adf60c020 msgr2=0x556adf60e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- 172.21.15.37:0/3909314954 shutdown_connections 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- 172.21.15.37:0/3909314954 wait complete. 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- start start 2024-04-25T08:58:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 Processor -- start 2024-04-25T08:58:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf63e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf63ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf65a2f0 0x556adf63f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556adf421250 con 0x556adf6599d0 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x556adf417690 con 0x556adf6100a0 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x556adf3e5a40 con 0x556adf65a2f0 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 10 client.0 _ll_register_callbacks cb 0x556adf5fae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c0ea4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf63ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c0ea4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf63ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42372/0 (socket says 172.21.15.37:42372) 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c0ea4640 1 -- 172.21.15.37:0/4011114757 learned_addr learned my addr 172.21.15.37:0/4011114757 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c16a5640 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf63e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c1ea6640 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf65a2f0 0x556adf63f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c0ea4640 1 -- 172.21.15.37:0/4011114757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf65a2f0 msgr2=0x556adf63f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c0ea4640 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf65a2f0 0x556adf63f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c0ea4640 1 -- 172.21.15.37:0/4011114757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 msgr2=0x556adf63e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c0ea4640 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf63e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c0ea4640 1 -- 172.21.15.37:0/4011114757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556adf50a1c0 con 0x556adf6599d0 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c0ea4640 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf63ed30 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f55ac010550 tx=0x7f55ac010a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8679b991a9276e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f559bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f559bfff640 1 -- 172.21.15.37:0/4011114757 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac014660 con 0x556adf6599d0 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f559bfff640 1 -- 172.21.15.37:0/4011114757 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f55ac014800 con 0x556adf6599d0 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- 172.21.15.37:0/4011114757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55b40b6420 con 0x556adf6599d0 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f559bfff640 1 -- 172.21.15.37:0/4011114757 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55ac024050 con 0x556adf6599d0 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 10 client.7461 fetch_fsmap learned FSMap version 51 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- 172.21.15.37:0/4011114757 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x556adf506170 con 0x556adf6599d0 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f559bfff640 1 -- 172.21.15.37:0/4011114757 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b40b6420 con 0x556adf6599d0 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f559bfff640 1 -- 172.21.15.37:0/4011114757 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f55ac027020 con 0x556adf6599d0 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 10 client.7461 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 10 client.7461 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 20 client.7461 populate_metadata read hostname 'smithi037' 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556adf669c80 0x556adf66c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 4 client.7461 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c6a76000 1 -- 172.21.15.37:0/4011114757 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556adf3f8c60 con 0x556adf669c80 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c16a5640 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556adf669c80 0x556adf66c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f559bfff640 10 client.7461 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.797+0000 7f55c16a5640 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556adf669c80 0x556adf66c070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7b49c55e638f2ab server_cookie=ca1f0843ce364dd3 in_seq=0 out_seq=0 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f559bfff640 1 -- 172.21.15.37:0/4011114757 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556adf3f8c60 con 0x556adf669c80 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f559bfff640 10 client.7461 handle_command_reply: tid=0 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 1 client.7461 shutdown 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 1 -- 172.21.15.37:0/4011114757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 msgr2=0x556adf63ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf63ed30 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f55ac010550 tx=0x7f55ac010a10 comp rx=0 tx=0).stop 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 1 -- 172.21.15.37:0/4011114757 shutdown_connections 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556adf669c80 0x556adf66c070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf65a2f0 0x556adf63f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f559bfff640 0 client.7461 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6599d0 0x556adf63ed30 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 1 --2- 172.21.15.37:0/4011114757 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf63e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 1 -- 172.21.15.37:0/4011114757 >> 172.21.15.37:0/4011114757 conn(0x556adf60c020 msgr2=0x556adf60eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 1 -- 172.21.15.37:0/4011114757 shutdown_connections 2024-04-25T08:58:00.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 1 -- 172.21.15.37:0/4011114757 wait complete. 2024-04-25T08:58:00.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.801+0000 7f55c6a76000 20 client.7461 trim_cache size 0 max 16384 2024-04-25T08:58:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.813+0000 7f55c6a76000 1 Processor -- start 2024-04-25T08:58:00.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.813+0000 7f55c6a76000 1 -- start start 2024-04-25T08:58:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.813+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf6b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.813+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf6599d0 0x556adf6b6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.813+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf65a2f0 0x556adf6b6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.813+0000 7f55c6a76000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc203410 con 0x556adf65a2f0 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.813+0000 7f55c6a76000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc2025f0 con 0x556adf6100a0 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.813+0000 7f55c6a76000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc0c8480 con 0x556adf6599d0 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.813+0000 7f55c1ea6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf65a2f0 0x556adf6b6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.813+0000 7f55c1ea6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf65a2f0 0x556adf6b6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42382/0 (socket says 172.21.15.37:42382) 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.813+0000 7f55c1ea6640 1 -- 172.21.15.37:0/2126347218 learned_addr learned my addr 172.21.15.37:0/2126347218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c16a5640 1 --2- 172.21.15.37:0/2126347218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf6b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c1ea6640 1 -- 172.21.15.37:0/2126347218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf6599d0 msgr2=0x556adf6b6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c1ea6640 1 --2- 172.21.15.37:0/2126347218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf6599d0 0x556adf6b6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c1ea6640 1 -- 172.21.15.37:0/2126347218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 msgr2=0x556adf6b60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c1ea6640 1 --2- 172.21.15.37:0/2126347218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf6b60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c1ea6640 1 -- 172.21.15.37:0/2126347218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556adf4a3de0 con 0x556adf65a2f0 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c1ea6640 1 --2- 172.21.15.37:0/2126347218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf65a2f0 0x556adf6b6b70 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f55b805d900 tx=0x7f55b8055e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474965e4aa5cb145 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55a67fc640 1 -- 172.21.15.37:0/2126347218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b806d030 con 0x556adf65a2f0 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55a67fc640 1 -- 172.21.15.37:0/2126347218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b8059870 con 0x556adf65a2f0 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55a67fc640 1 -- 172.21.15.37:0/2126347218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b805de30 con 0x556adf65a2f0 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- 172.21.15.37:0/2126347218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf65a2f0 msgr2=0x556adf6b6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 --2- 172.21.15.37:0/2126347218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf65a2f0 0x556adf6b6b70 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f55b805d900 tx=0x7f55b8055e20 comp rx=0 tx=0).stop 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- 172.21.15.37:0/2126347218 shutdown_connections 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 --2- 172.21.15.37:0/2126347218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf65a2f0 0x556adf6b6b70 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 --2- 172.21.15.37:0/2126347218 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf6599d0 0x556adf6b6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 --2- 172.21.15.37:0/2126347218 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf6100a0 0x556adf6b60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- 172.21.15.37:0/2126347218 >> 172.21.15.37:0/2126347218 conn(0x556adf60c020 msgr2=0x556adf6580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- 172.21.15.37:0/2126347218 shutdown_connections 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- 172.21.15.37:0/2126347218 wait complete. 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- start start 2024-04-25T08:58:00.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 Processor -- start 2024-04-25T08:58:00.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6100a0 0x556adf66c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf6599d0 0x556adf66ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf65a2f0 0x556adf66d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x556adf3f7e00 con 0x556adf6100a0 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc201ff0 con 0x556adf65a2f0 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x556adf45c300 con 0x556adf6599d0 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 10 client.0 _ll_register_callbacks cb 0x556adf5db040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c16a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6100a0 0x556adf66c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c16a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6100a0 0x556adf66c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42384/0 (socket says 172.21.15.37:42384) 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c16a5640 1 -- 172.21.15.37:0/379950090 learned_addr learned my addr 172.21.15.37:0/379950090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c16a5640 1 -- 172.21.15.37:0/379950090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf6599d0 msgr2=0x556adf66ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c0ea4640 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf6599d0 0x556adf66ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c16a5640 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf6599d0 0x556adf66ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c16a5640 1 -- 172.21.15.37:0/379950090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf65a2f0 msgr2=0x556adf66d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:00.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c16a5640 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf65a2f0 0x556adf66d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c16a5640 1 -- 172.21.15.37:0/379950090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556adf3fe8e0 con 0x556adf6100a0 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c16a5640 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6100a0 0x556adf66c770 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009e50 tx=0x7f55b401b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d7ce86491455b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f559bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f559bfff640 1 -- 172.21.15.37:0/379950090 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b40c7040 con 0x556adf6100a0 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f559bfff640 1 -- 172.21.15.37:0/379950090 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f55b40b69b0 con 0x556adf6100a0 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- 172.21.15.37:0/379950090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55b8059870 con 0x556adf6100a0 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f559bfff640 1 -- 172.21.15.37:0/379950090 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55b40dd050 con 0x556adf6100a0 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 10 client.7467 fetch_fsmap learned FSMap version 51 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- 172.21.15.37:0/379950090 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=2+}) v3 -- 0x556adf4b68f0 con 0x556adf6100a0 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f559bfff640 1 -- 172.21.15.37:0/379950090 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b8059870 con 0x556adf6100a0 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f559bfff640 1 -- 172.21.15.37:0/379950090 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 51) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f55b40e0020 con 0x556adf6100a0 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 10 client.7467 fetch_fsmap finished waiting for FSMap version 51 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 10 client.7467 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 20 client.7467 populate_metadata read hostname 'smithi037' 2024-04-25T08:58:00.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556adf6bff50 0x556adf6c2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 4 client.7467 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c6a76000 1 -- 172.21.15.37:0/379950090 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556adf46a530 con 0x556adf6bff50 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.817+0000 7f55c0ea4640 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556adf6bff50 0x556adf6c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c0ea4640 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556adf6bff50 0x556adf6c2340 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4fdae15216cde81 server_cookie=a26b8af4fb343d9d in_seq=0 out_seq=0 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f559bfff640 10 client.7467 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f559bfff640 1 -- 172.21.15.37:0/379950090 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+266 (crc 0 0 0) 0x556adf46a530 con 0x556adf6bff50 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f559bfff640 10 client.7467 handle_command_reply: tid=0 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 1 client.7467 shutdown 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 1 -- 172.21.15.37:0/379950090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6100a0 msgr2=0x556adf66c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6100a0 0x556adf66c770 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009e50 tx=0x7f55b401b230 comp rx=0 tx=0).stop 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 1 -- 172.21.15.37:0/379950090 shutdown_connections 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x556adf6bff50 0x556adf6c2340 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x556adf65a2f0 0x556adf66d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x556adf6599d0 0x556adf66ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 1 --2- 172.21.15.37:0/379950090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556adf6100a0 0x556adf66c770 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f559bfff640 0 client.7467 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 1 -- 172.21.15.37:0/379950090 >> 172.21.15.37:0/379950090 conn(0x556adf60c020 msgr2=0x556adf658930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 1 -- 172.21.15.37:0/379950090 shutdown_connections 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 1 -- 172.21.15.37:0/379950090 wait complete. 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c6a76000 20 client.7467 trim_cache size 0 max 16384 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "405323a4-c6fa-4133-96bb-dc1b3c6a8b13": { 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/subdir", 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "405323a4-c6fa-4133-96bb-dc1b3c6a8b13", 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,repair" 2024-04-25T08:58:00.816 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:00.817 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:00.817 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:58:00.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 -- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55a0066f60 msgr2=0x7f55a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55a0066f60 0x7f55a0069420 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f55b40ae000 tx=0x7f55b40a7070 comp rx=0 tx=0).stop 2024-04-25T08:58:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 -- 172.21.15.37:0/2546516020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc15d9c0 msgr2=0x7f55bc152d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc15d9c0 0x7f55bc152d40 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f55b800c3c0 tx=0x7f55b800c880 comp rx=0 tx=0).stop 2024-04-25T08:58:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 -- 172.21.15.37:0/2546516020 shutdown_connections 2024-04-25T08:58:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55a0066f60 0x7f55a0069420 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55bc15d9c0 0x7f55bc152d40 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55bc0d3c20 0x7f55bc0cba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 --2- 172.21.15.37:0/2546516020 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55bc0cf270 0x7f55bc0c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 -- 172.21.15.37:0/2546516020 >> 172.21.15.37:0/2546516020 conn(0x7f55bc09cad0 msgr2=0x7f55bc0b59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 -- 172.21.15.37:0/2546516020 shutdown_connections 2024-04-25T08:58:00.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:00.821+0000 7f55c3909640 1 -- 172.21.15.37:0/2546516020 wait complete. 2024-04-25T08:58:00.826 INFO:tasks.cephfs.filesystem:scrub status for tag:405323a4-c6fa-4133-96bb-dc1b3c6a8b13 - {'path': '/subdir', 'tag': '405323a4-c6fa-4133-96bb-dc1b3c6a8b13', 'options': 'recursive,repair'} 2024-04-25T08:58:26.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:26.050 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:30.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:30.827 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:58:30.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.972+0000 7f8cad8ab640 1 Processor -- start 2024-04-25T08:58:30.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.972+0000 7f8cad8ab640 1 -- start start 2024-04-25T08:58:30.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8cad8ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca815b950 0x7f8ca809a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8cad8ab640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ca80cf420 0x7f8ca80cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8cad8ab640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca815bd30 con 0x7f8ca80cfb80 2024-04-25T08:58:30.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8cad8ab640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca80cc0d0 con 0x7f8ca80cf420 2024-04-25T08:58:30.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8cad8ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca809ab10 con 0x7f8ca815b950 2024-04-25T08:58:30.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca77fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8ca80cfb80 0x7f8ca806f290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:54454/0 (socket says 172.21.15.37:54454) 2024-04-25T08:58:30.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca77fe640 1 -- 172.21.15.37:0/762137237 learned_addr learned my addr 172.21.15.37:0/762137237 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:30.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/762137237 >> v2:172.21.15.203:3300/0 conn(0x7f8ca80cf420 0x7f8ca80cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca815b950 0x7f8ca809a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753156373 0 0) 0x7f8ca815bd30 con 0x7f8ca80cfb80 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c94003530 con 0x7f8ca80cfb80 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2214034143 0 0) 0x7f8c94003530 con 0x7f8ca80cfb80 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca815b950 msgr2=0x7f8ca809a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca815b950 0x7f8ca809a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 >> v2:172.21.15.203:3300/0 conn(0x7f8ca80cf420 msgr2=0x7f8ca80cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 --2- 172.21.15.37:0/762137237 >> v2:172.21.15.203:3300/0 conn(0x7f8ca80cf420 0x7f8ca80cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca80cc370 con 0x7f8ca80cfb80 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f8ca0001680 con 0x7f8ca80cfb80 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 >> v1:172.21.15.37:6789/0 conn(0x7f8ca80cfb80 legacy=0x7f8ca806f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca815b950 0x7f8c94003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c94003fb0 0x7f8c94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c94004d50 0x7f8c940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80cc0d0 con 0x7f8c94003fb0 2024-04-25T08:58:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca809ab10 con 0x7f8c94004d50 2024-04-25T08:58:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9400ba00 con 0x7f8ca815b950 2024-04-25T08:58:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c94003fb0 0x7f8c94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca77fe640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c94004d50 0x7f8c940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca67fc640 1 -- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca815b950 msgr2=0x7f8c94003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca815b950 0x7f8c94003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca815b950 0x7f8c94003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca67fc640 1 -- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c94004d50 msgr2=0x7f8c940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c94004d50 0x7f8c940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca67fc640 1 -- 172.21.15.37:0/762137237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c94003530 con 0x7f8c94003fb0 2024-04-25T08:58:30.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.976+0000 7f8ca77fe640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c94004d50 0x7f8c940051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:30.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c94003fb0 0x7f8c94004810 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009ae0 tx=0x7f8c9001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6e5fdbd0f81b25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900b2070 con 0x7f8c94003fb0 2024-04-25T08:58:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c900a56a0 con 0x7f8c94003fb0 2024-04-25T08:58:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8ca5ffb640 1 -- 172.21.15.37:0/762137237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900096b0 con 0x7f8c94003fb0 2024-04-25T08:58:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 -- 172.21.15.37:0/762137237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c94003fb0 msgr2=0x7f8c94004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 --2- 172.21.15.37:0/762137237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c94003fb0 0x7f8c94004810 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009ae0 tx=0x7f8c9001cab0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8ca77fe640 1 -- 172.21.15.37:0/762137237 reap_dead start 2024-04-25T08:58:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 -- 172.21.15.37:0/762137237 shutdown_connections 2024-04-25T08:58:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 -- 172.21.15.37:0/762137237 >> 172.21.15.37:0/762137237 conn(0x7f8ca80b30b0 msgr2=0x7f8ca80b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 -- 172.21.15.37:0/762137237 shutdown_connections 2024-04-25T08:58:30.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 -- 172.21.15.37:0/762137237 wait complete. 2024-04-25T08:58:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 Processor -- start 2024-04-25T08:58:30.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 -- start start 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c94003fb0 0x7f8ca8152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ca80b5e10 0x7f8ca80b6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca80c8ea0 0x7f8ca80c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80cf880 con 0x7f8ca80b5e10 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80cc930 con 0x7f8c94003fb0 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8cad8ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca80cd390 con 0x7f8ca80c8ea0 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8ca67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ca80b5e10 0x7f8ca80b6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8ca67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ca80b5e10 0x7f8ca80b6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49514/0 (socket says 172.21.15.37:49514) 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8ca67fc640 1 -- 172.21.15.37:0/2595360384 learned_addr learned my addr 172.21.15.37:0/2595360384 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c94003fb0 0x7f8ca8152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.980+0000 7f8ca77fe640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca80c8ea0 0x7f8ca80c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8ca67fc640 1 -- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca80c8ea0 msgr2=0x7f8ca80c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca80c8ea0 0x7f8ca80c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8ca67fc640 1 -- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c94003fb0 msgr2=0x7f8ca8152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c94003fb0 0x7f8ca8152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8ca67fc640 1 -- 172.21.15.37:0/2595360384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca80b4d70 con 0x7f8ca80b5e10 2024-04-25T08:58:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ca80b5e10 0x7f8ca80b6290 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009ae0 tx=0x7f8c94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8c9ffff640 1 -- 172.21.15.37:0/2595360384 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900b2040 con 0x7f8ca80b5e10 2024-04-25T08:58:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8c9ffff640 1 -- 172.21.15.37:0/2595360384 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c90004230 con 0x7f8ca80b5e10 2024-04-25T08:58:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8cad8ab640 1 -- 172.21.15.37:0/2595360384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca80b5050 con 0x7f8ca80b5e10 2024-04-25T08:58:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8c9ffff640 1 -- 172.21.15.37:0/2595360384 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90003620 con 0x7f8ca80b5e10 2024-04-25T08:58:30.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8cad8ab640 1 -- 172.21.15.37:0/2595360384 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca80b5610 con 0x7f8ca80b5e10 2024-04-25T08:58:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8c9ffff640 1 -- 172.21.15.37:0/2595360384 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c900037c0 con 0x7f8ca80b5e10 2024-04-25T08:58:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8c9ffff640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c88066f90 0x7f8c88069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8c9ffff640 1 -- 172.21.15.37:0/2595360384 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8c9011faa0 con 0x7f8ca80b5e10 2024-04-25T08:58:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c88066f90 0x7f8c88069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8cb0a18000 1 Processor -- start 2024-04-25T08:58:30.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8cb0a18000 1 -- start start 2024-04-25T08:58:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.984+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c88066f90 0x7f8c88069450 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80d0230 tx=0x7f8c9800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35be30 0x55d7de35c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35c750 0x55d7de311f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de312890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7de15aff0 con 0x55d7de3124b0 2024-04-25T08:58:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7de0e3150 con 0x55d7de35c750 2024-04-25T08:58:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7de157090 con 0x55d7de35be30 2024-04-25T08:58:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de312890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de312890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49522/0 (socket says 172.21.15.37:49522) 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca77fe640 1 -- 172.21.15.37:0/3370380405 learned_addr learned my addr 172.21.15.37:0/3370380405 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/3370380405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35c750 0x55d7de311f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/3370380405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35be30 0x55d7de35c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca77fe640 1 -- 172.21.15.37:0/3370380405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35be30 msgr2=0x55d7de35c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca77fe640 1 --2- 172.21.15.37:0/3370380405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35be30 0x55d7de35c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca77fe640 1 -- 172.21.15.37:0/3370380405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35c750 msgr2=0x55d7de311f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca77fe640 1 --2- 172.21.15.37:0/3370380405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35c750 0x55d7de311f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca77fe640 1 -- 172.21.15.37:0/3370380405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7de2f10d0 con 0x55d7de3124b0 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca77fe640 1 --2- 172.21.15.37:0/3370380405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de312890 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8ca815bf00 tx=0x7f8ca0005550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c489e3375f242cfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8c9d7fa640 1 -- 172.21.15.37:0/3370380405 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0010040 con 0x55d7de3124b0 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8c9d7fa640 1 -- 172.21.15.37:0/3370380405 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca00139a0 con 0x55d7de3124b0 2024-04-25T08:58:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8c9d7fa640 1 -- 172.21.15.37:0/3370380405 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca0013d90 con 0x55d7de3124b0 2024-04-25T08:58:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3370380405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 msgr2=0x55d7de312890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/3370380405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de312890 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8ca815bf00 tx=0x7f8ca0005550 comp rx=0 tx=0).stop 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3370380405 shutdown_connections 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/3370380405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de312890 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/3370380405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35c750 0x55d7de311f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/3370380405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35be30 0x55d7de35c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3370380405 >> 172.21.15.37:0/3370380405 conn(0x55d7de30a020 msgr2=0x55d7de30c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3370380405 shutdown_connections 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3370380405 wait complete. 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- start start 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 Processor -- start 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de35b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35be30 0x55d7de318360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35c750 0x55d7de3188a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7de15f620 con 0x55d7de3124b0 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7de115690 con 0x55d7de35c750 2024-04-25T08:58:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7de0e3a40 con 0x55d7de35be30 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8cb0a18000 10 client.0 _ll_register_callbacks cb 0x55d7de2f8e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de35b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de35b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49528/0 (socket says 172.21.15.37:49528) 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/3329918906 learned_addr learned my addr 172.21.15.37:0/3329918906 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35be30 0x55d7de318360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.988+0000 7f8ca77fe640 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35c750 0x55d7de3188a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/3329918906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35be30 msgr2=0x55d7de318360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35be30 0x55d7de318360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/3329918906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35c750 msgr2=0x55d7de3188a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35c750 0x55d7de3188a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/3329918906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7de2081c0 con 0x55d7de3124b0 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8ca77fe640 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35c750 0x55d7de3188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de35b790 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8c98017c00 tx=0x7f8c98004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76643dba0491c773 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8c867fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8c867fc640 1 -- 172.21.15.37:0/3329918906 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98018d80 con 0x55d7de3124b0 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3329918906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ca00139a0 con 0x55d7de3124b0 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8c867fc640 1 -- 172.21.15.37:0/3329918906 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8c98023410 con 0x55d7de3124b0 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8c867fc640 1 -- 172.21.15.37:0/3329918906 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c98023650 con 0x55d7de3124b0 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 10 client.7479 fetch_fsmap learned FSMap version 52 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3329918906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55d7de204170 con 0x55d7de3124b0 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8c867fc640 1 -- 172.21.15.37:0/3329918906 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca00139a0 con 0x55d7de3124b0 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8c867fc640 1 -- 172.21.15.37:0/3329918906 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8c980237f0 con 0x55d7de3124b0 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 10 client.7479 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 10 client.7479 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 20 client.7479 populate_metadata read hostname 'smithi037' 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55d7de3195e0 0x55d7de36e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 4 client.7479 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:58:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3329918906 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7de0f6c60 con 0x55d7de3195e0 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55d7de3195e0 0x55d7de36e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8c867fc640 10 client.7479 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55d7de3195e0 0x55d7de36e0f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=420d0bf5c4e55fd0 server_cookie=6140d25e4864952c in_seq=0 out_seq=0 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8c867fc640 1 -- 172.21.15.37:0/3329918906 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d7de0f6c60 con 0x55d7de3195e0 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8c867fc640 10 client.7479 handle_command_reply: tid=0 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 client.7479 shutdown 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3329918906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 msgr2=0x55d7de35b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de35b790 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8c98017c00 tx=0x7f8c98004a50 comp rx=0 tx=0).stop 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3329918906 shutdown_connections 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55d7de3195e0 0x55d7de36e0f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35c750 0x55d7de3188a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35be30 0x55d7de318360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/3329918906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de35b790 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8c867fc640 0 client.7479 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:30.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3329918906 >> 172.21.15.37:0/3329918906 conn(0x55d7de30a020 msgr2=0x55d7de313120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3329918906 shutdown_connections 2024-04-25T08:58:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 1 -- 172.21.15.37:0/3329918906 wait complete. 2024-04-25T08:58:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:30.992+0000 7f8cb0a18000 20 client.7479 trim_cache size 0 max 16384 2024-04-25T08:58:30.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 Processor -- start 2024-04-25T08:58:31.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 -- start start 2024-04-25T08:58:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de33aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35be30 0x55d7de33b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35c750 0x55d7de33b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca81fc680 con 0x55d7de3124b0 2024-04-25T08:58:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca806dbb0 con 0x55d7de35be30 2024-04-25T08:58:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca806cfb0 con 0x55d7de35c750 2024-04-25T08:58:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de33aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de33aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49540/0 (socket says 172.21.15.37:49540) 2024-04-25T08:58:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/1207369663 learned_addr learned my addr 172.21.15.37:0/1207369663 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/1207369663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35be30 0x55d7de33b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca77fe640 1 --2- 172.21.15.37:0/1207369663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35c750 0x55d7de33b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/1207369663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35c750 msgr2=0x55d7de33b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/1207369663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35c750 0x55d7de33b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/1207369663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35be30 msgr2=0x55d7de33b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/1207369663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35be30 0x55d7de33b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/1207369663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7de1a1de0 con 0x55d7de3124b0 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca77fe640 1 --2- 172.21.15.37:0/1207369663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35c750 0x55d7de33b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/1207369663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de33aad0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009f70 tx=0x7f8c98021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53a04cdb33de6b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8c9cff9640 1 -- 172.21.15.37:0/1207369663 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98038030 con 0x55d7de3124b0 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8c9cff9640 1 -- 172.21.15.37:0/1207369663 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c98028070 con 0x55d7de3124b0 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8c9cff9640 1 -- 172.21.15.37:0/1207369663 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c980187c0 con 0x55d7de3124b0 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1207369663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 msgr2=0x55d7de33aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/1207369663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de33aad0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009f70 tx=0x7f8c98021000 comp rx=0 tx=0).stop 2024-04-25T08:58:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1207369663 shutdown_connections 2024-04-25T08:58:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/1207369663 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35c750 0x55d7de33b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/1207369663 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35be30 0x55d7de33b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.008+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/1207369663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de33aad0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1207369663 >> 172.21.15.37:0/1207369663 conn(0x55d7de30a020 msgr2=0x55d7de313120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1207369663 shutdown_connections 2024-04-25T08:58:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1207369663 wait complete. 2024-04-25T08:58:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- start start 2024-04-25T08:58:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 Processor -- start 2024-04-25T08:58:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de370230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35be30 0x55d7de36e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35c750 0x55d7de36e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7de0f5e00 con 0x55d7de3124b0 2024-04-25T08:58:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca806d5b0 con 0x55d7de35be30 2024-04-25T08:58:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7de15a300 con 0x55d7de35c750 2024-04-25T08:58:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 10 client.0 _ll_register_callbacks cb 0x55d7de2d9040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de370230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de370230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49544/0 (socket says 172.21.15.37:49544) 2024-04-25T08:58:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/1425006738 learned_addr learned my addr 172.21.15.37:0/1425006738 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca77fe640 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35c750 0x55d7de36e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35be30 0x55d7de36e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/1425006738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35c750 msgr2=0x55d7de36e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35c750 0x55d7de36e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/1425006738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35be30 msgr2=0x55d7de36e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35be30 0x55d7de36e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca6ffd640 1 -- 172.21.15.37:0/1425006738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d7de0fc8e0 con 0x55d7de3124b0 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca6ffd640 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de370230 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f8c98024d20 tx=0x7f8c98004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8d465121d4211a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8c867fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8c867fc640 1 -- 172.21.15.37:0/1425006738 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98034450 con 0x55d7de3124b0 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8c867fc640 1 -- 172.21.15.37:0/1425006738 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8c980345f0 con 0x55d7de3124b0 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1425006738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c98028070 con 0x55d7de3124b0 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8c867fc640 1 -- 172.21.15.37:0/1425006738 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c9804d050 con 0x55d7de3124b0 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 10 client.7485 fetch_fsmap learned FSMap version 52 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1425006738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x55d7de1b48f0 con 0x55d7de3124b0 2024-04-25T08:58:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8c867fc640 1 -- 172.21.15.37:0/1425006738 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c98028070 con 0x55d7de3124b0 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8c867fc640 1 -- 172.21.15.37:0/1425006738 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8c98050020 con 0x55d7de3124b0 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 10 client.7485 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 10 client.7485 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 20 client.7485 populate_metadata read hostname 'smithi037' 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55d7de36eb70 0x55d7de36ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 4 client.7485 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1425006738 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d7de168530 con 0x55d7de36eb70 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55d7de36eb70 0x55d7de36ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8ca67fc640 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55d7de36eb70 0x55d7de36ef80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ee7f99703ecf14 server_cookie=d9c2d58f259d99d7 in_seq=0 out_seq=0 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8c867fc640 10 client.7485 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8c867fc640 1 -- 172.21.15.37:0/1425006738 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d7de168530 con 0x55d7de36eb70 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8c867fc640 10 client.7485 handle_command_reply: tid=0 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 client.7485 shutdown 2024-04-25T08:58:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1425006738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 msgr2=0x55d7de370230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de370230 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f8c98024d20 tx=0x7f8c98004ac0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1425006738 shutdown_connections 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x55d7de36eb70 0x55d7de36ef80 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d7de35c750 0x55d7de36e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d7de35be30 0x55d7de36e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 --2- 172.21.15.37:0/1425006738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d7de3124b0 0x55d7de370230 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1425006738 >> 172.21.15.37:0/1425006738 conn(0x55d7de30a020 msgr2=0x55d7de368b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8c867fc640 0 client.7485 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1425006738 shutdown_connections 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.012+0000 7f8cb0a18000 1 -- 172.21.15.37:0/1425006738 wait complete. 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cb0a18000 20 client.7485 trim_cache size 0 max 16384 2024-04-25T08:58:31.011 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:58:31.012 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-04-25T08:58:31.012 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-04-25T08:58:31.012 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:58:31.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 -- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c88066f90 msgr2=0x7f8c88069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c88066f90 0x7f8c88069450 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80d0230 tx=0x7f8c9800b040 comp rx=0 tx=0).stop 2024-04-25T08:58:31.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 -- 172.21.15.37:0/2595360384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ca80b5e10 msgr2=0x7f8ca80b6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ca80b5e10 0x7f8ca80b6290 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009ae0 tx=0x7f8c94005840 comp rx=0 tx=0).stop 2024-04-25T08:58:31.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 -- 172.21.15.37:0/2595360384 shutdown_connections 2024-04-25T08:58:31.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c88066f90 0x7f8c88069450 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ca80c8ea0 0x7f8ca80c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ca80b5e10 0x7f8ca80b6290 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 --2- 172.21.15.37:0/2595360384 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c94003fb0 0x7f8ca8152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 -- 172.21.15.37:0/2595360384 >> 172.21.15.37:0/2595360384 conn(0x7f8ca80b30b0 msgr2=0x7f8ca806ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 -- 172.21.15.37:0/2595360384 shutdown_connections 2024-04-25T08:58:31.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.016+0000 7f8cad8ab640 1 -- 172.21.15.37:0/2595360384 wait complete. 2024-04-25T08:58:31.023 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:58:31.023 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.144+0000 7fd771e49640 1 Processor -- start 2024-04-25T08:58:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.144+0000 7fd771e49640 1 -- start start 2024-04-25T08:58:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd771e49640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15da80 0x7fd76c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd771e49640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c06be60 con 0x7fd76c0d07f0 2024-04-25T08:58:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd771e49640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd76c15de60 con 0x7fd76c0d1cd0 2024-04-25T08:58:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd771e49640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c06c440 con 0x7fd76c15da80 2024-04-25T08:58:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd76c0d07f0 0x7fd76c06c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:54468/0 (socket says 172.21.15.37:54468) 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76bfff640 1 -- 172.21.15.37:0/2230370983 learned_addr learned my addr 172.21.15.37:0/2230370983 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76b7fe640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15da80 0x7fd76c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042907359 0 0) 0x7fd76c06be60 con 0x7fd76c0d07f0 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd758003530 con 0x7fd76c0d07f0 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3674067600 0 0) 0x7fd758003530 con 0x7fd76c0d07f0 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15da80 msgr2=0x7fd76c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15da80 0x7fd76c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 >> v1:172.21.15.203:6789/0 conn(0x7fd76c0d1cd0 legacy=0x7fd76c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c0d2450 con 0x7fd76c0d07f0 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fd764001a00 con 0x7fd76c0d07f0 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 >> v1:172.21.15.37:6789/0 conn(0x7fd76c0d07f0 legacy=0x7fd76c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd758003a60 0x7fd758003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15da80 0x7fd758004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 0x7fd7580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c06c440 con 0x7fd758004d50 2024-04-25T08:58:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75800bb00 con 0x7fd758003a60 2024-04-25T08:58:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75800bd00 con 0x7fd76c15da80 2024-04-25T08:58:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76bfff640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 0x7fd7580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76b7fe640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd758003a60 0x7fd758003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76affd640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15da80 0x7fd758004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76bfff640 1 -- 172.21.15.37:0/2230370983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15da80 msgr2=0x7fd758004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76bfff640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15da80 0x7fd758004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76bfff640 1 -- 172.21.15.37:0/2230370983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd758003a60 msgr2=0x7fd758003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76bfff640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd758003a60 0x7fd758003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.148+0000 7fd76bfff640 1 -- 172.21.15.37:0/2230370983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd758003530 con 0x7fd758004d50 2024-04-25T08:58:31.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd76bfff640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 0x7fd7580051d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd76c08d8a0 tx=0x7fd76401d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974cacab788812d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76400a480 con 0x7fd758004d50 2024-04-25T08:58:31.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7640ad650 con 0x7fd758004d50 2024-04-25T08:58:31.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd76a7fc640 1 -- 172.21.15.37:0/2230370983 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7640ada40 con 0x7fd758004d50 2024-04-25T08:58:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 -- 172.21.15.37:0/2230370983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 msgr2=0x7fd7580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 --2- 172.21.15.37:0/2230370983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 0x7fd7580051d0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fd76c08d8a0 tx=0x7fd76401d320 comp rx=0 tx=0).stop 2024-04-25T08:58:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd76bfff640 1 -- 172.21.15.37:0/2230370983 reap_dead start 2024-04-25T08:58:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 -- 172.21.15.37:0/2230370983 shutdown_connections 2024-04-25T08:58:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 -- 172.21.15.37:0/2230370983 >> 172.21.15.37:0/2230370983 conn(0x7fd76c08b000 msgr2=0x7fd76c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 -- 172.21.15.37:0/2230370983 shutdown_connections 2024-04-25T08:58:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 -- 172.21.15.37:0/2230370983 wait complete. 2024-04-25T08:58:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 Processor -- start 2024-04-25T08:58:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 -- start start 2024-04-25T08:58:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 0x7fd76c0b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15b660 0x7fd76c157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd76c0cc7d0 0x7fd76c0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c0d26f0 con 0x7fd758004d50 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c0d20f0 con 0x7fd76c0cc7d0 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd771e49640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76c0d2870 con 0x7fd76c15b660 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd76b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 0x7fd76c0b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd76b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 0x7fd76c0b6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49560/0 (socket says 172.21.15.37:49560) 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd76b7fe640 1 -- 172.21.15.37:0/3047005560 learned_addr learned my addr 172.21.15.37:0/3047005560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd76bfff640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd76c0cc7d0 0x7fd76c0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.152+0000 7fd76affd640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15b660 0x7fd76c157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd76b7fe640 1 -- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15b660 msgr2=0x7fd76c157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd76b7fe640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15b660 0x7fd76c157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd76b7fe640 1 -- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd76c0cc7d0 msgr2=0x7fd76c0ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd76b7fe640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd76c0cc7d0 0x7fd76c0ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd76b7fe640 1 -- 172.21.15.37:0/3047005560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd76c0b7260 con 0x7fd758004d50 2024-04-25T08:58:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd76b7fe640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 0x7fd76c0b6d20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd754011a00 tx=0x7fd754011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd763fff640 1 -- 172.21.15.37:0/3047005560 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75400fb50 con 0x7fd758004d50 2024-04-25T08:58:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd763fff640 1 -- 172.21.15.37:0/3047005560 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75400fcf0 con 0x7fd758004d50 2024-04-25T08:58:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd763fff640 1 -- 172.21.15.37:0/3047005560 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75400fea0 con 0x7fd758004d50 2024-04-25T08:58:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd771e49640 1 -- 172.21.15.37:0/3047005560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76c0b7d90 con 0x7fd758004d50 2024-04-25T08:58:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd771e49640 1 -- 172.21.15.37:0/3047005560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd76c0b8350 con 0x7fd758004d50 2024-04-25T08:58:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd771e49640 1 -- 172.21.15.37:0/3047005560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76c04efa0 con 0x7fd758004d50 2024-04-25T08:58:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd763fff640 1 -- 172.21.15.37:0/3047005560 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd754012040 con 0x7fd758004d50 2024-04-25T08:58:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd763fff640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd74c066f60 0x7fd74c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd763fff640 1 -- 172.21.15.37:0/3047005560 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd754085720 con 0x7fd758004d50 2024-04-25T08:58:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd76affd640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd74c066f60 0x7fd74c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.156+0000 7fd76affd640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd74c066f60 0x7fd74c069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd76c15ba60 tx=0x7fd75c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.160+0000 7fd763fff640 1 -- 172.21.15.37:0/3047005560 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd754019030 con 0x7fd758004d50 2024-04-25T08:58:31.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 -- 172.21.15.37:0/3047005560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd76c0c9d50 con 0x7fd758004d50 2024-04-25T08:58:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd763fff640 1 -- 172.21.15.37:0/3047005560 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fd754055fc0 con 0x7fd758004d50 2024-04-25T08:58:31.344 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:31.344 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:58:02.214606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":16,"addr":"172.21.15.37:6835/3927528604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3927528604},{"type":"v1","addr":"172.21.15.37:6835","nonce":3927528604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:58:31.344 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 52 2024-04-25T08:58:31.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 -- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd74c066f60 msgr2=0x7fd74c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd74c066f60 0x7fd74c069420 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd76c15ba60 tx=0x7fd75c009040 comp rx=0 tx=0).stop 2024-04-25T08:58:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 -- 172.21.15.37:0/3047005560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 msgr2=0x7fd76c0b6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 0x7fd76c0b6d20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd754011a00 tx=0x7fd754011ec0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 -- 172.21.15.37:0/3047005560 shutdown_connections 2024-04-25T08:58:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd74c066f60 0x7fd74c069420 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd76c0cc7d0 0x7fd76c0ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd76c15b660 0x7fd76c157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 --2- 172.21.15.37:0/3047005560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd758004d50 0x7fd76c0b6d20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 -- 172.21.15.37:0/3047005560 >> 172.21.15.37:0/3047005560 conn(0x7fd76c08b000 msgr2=0x7fd76c06c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.352+0000 7fd771e49640 1 -- 172.21.15.37:0/3047005560 shutdown_connections 2024-04-25T08:58:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.356+0000 7fd771e49640 1 -- 172.21.15.37:0/3047005560 wait complete. 2024-04-25T08:58:31.361 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c damage ls --format=json-pretty 2024-04-25T08:58:31.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b70e18640 1 Processor -- start 2024-04-25T08:58:31.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b70e18640 1 -- start start 2024-04-25T08:58:31.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b70e18640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1b6c15ba20 0x7f1b6c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b70e18640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1b6c15fd90 0x7f1b6c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b70e18640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b6c06b1d0 0x7f1b6c0cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b70e18640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6c0d1430 con 0x7f1b6c15ba20 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b70e18640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b6c0cc6e0 con 0x7f1b6c15fd90 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b70e18640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c06b620 con 0x7f1b6c06b1d0 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b6a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1b6c15ba20 0x7f1b6c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b6a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1b6c15ba20 0x7f1b6c0cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49576/0 (socket says 172.21.15.37:49576) 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b6a59c640 1 -- 172.21.15.37:0/4037853264 learned_addr learned my addr 172.21.15.37:0/4037853264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b69d9b640 1 --2- 172.21.15.37:0/4037853264 >> v2:172.21.15.203:3300/0 conn(0x7f1b6c15fd90 0x7f1b6c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b6a59c640 1 -- 172.21.15.37:0/4037853264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b6c06b1d0 msgr2=0x7f1b6c0cf6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/4037853264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b6c06b1d0 0x7f1b6c0cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b6a59c640 1 -- 172.21.15.37:0/4037853264 >> v2:172.21.15.203:3300/0 conn(0x7f1b6c15fd90 msgr2=0x7f1b6c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/4037853264 >> v2:172.21.15.203:3300/0 conn(0x7f1b6c15fd90 0x7f1b6c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b6a59c640 1 -- 172.21.15.37:0/4037853264 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c15c260 con 0x7f1b6c15ba20 2024-04-25T08:58:31.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.484+0000 7f1b69d9b640 1 --2- 172.21.15.37:0/4037853264 >> v2:172.21.15.203:3300/0 conn(0x7f1b6c15fd90 0x7f1b6c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:31.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/4037853264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c15ba20 0x7f1b6c0cc1a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c099f10 tx=0x7f1b6401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369017e801e35c54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b6959a640 1 -- 172.21.15.37:0/4037853264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b640b1070 con 0x7f1b6c15ba20 2024-04-25T08:58:31.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b6959a640 1 -- 172.21.15.37:0/4037853264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b640ac630 con 0x7f1b6c15ba20 2024-04-25T08:58:31.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b6959a640 1 -- 172.21.15.37:0/4037853264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b640a8050 con 0x7f1b6c15ba20 2024-04-25T08:58:31.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b70e18640 1 -- 172.21.15.37:0/4037853264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c15ba20 msgr2=0x7f1b6c0cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b70e18640 1 --2- 172.21.15.37:0/4037853264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c15ba20 0x7f1b6c0cc1a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c099f10 tx=0x7f1b6401c520 comp rx=0 tx=0).stop 2024-04-25T08:58:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b70e18640 1 -- 172.21.15.37:0/4037853264 shutdown_connections 2024-04-25T08:58:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b70e18640 1 --2- 172.21.15.37:0/4037853264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b6c06b1d0 0x7f1b6c0cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b70e18640 1 --2- 172.21.15.37:0/4037853264 >> v2:172.21.15.203:3300/0 conn(0x7f1b6c15fd90 0x7f1b6c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b70e18640 1 --2- 172.21.15.37:0/4037853264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c15ba20 0x7f1b6c0cc1a0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b70e18640 1 -- 172.21.15.37:0/4037853264 >> 172.21.15.37:0/4037853264 conn(0x7f1b6c099a70 msgr2=0x7f1b6c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b70e18640 1 -- 172.21.15.37:0/4037853264 shutdown_connections 2024-04-25T08:58:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b70e18640 1 -- 172.21.15.37:0/4037853264 wait complete. 2024-04-25T08:58:31.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b70e18640 1 Processor -- start 2024-04-25T08:58:31.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.488+0000 7f1b70e18640 1 -- start start 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b70e18640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c06b1d0 0x7f1b6c151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b70e18640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b6c15ba20 0x7f1b6c1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b70e18640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b6c15fd90 0x7f1b6c0ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b70e18640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c1601a0 con 0x7f1b6c06b1d0 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b70e18640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c15be00 con 0x7f1b6c15ba20 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b70e18640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c0c88c0 con 0x7f1b6c15fd90 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c06b1d0 0x7f1b6c151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c06b1d0 0x7f1b6c151ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49588/0 (socket says 172.21.15.37:49588) 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b6a59c640 1 -- 172.21.15.37:0/401379144 learned_addr learned my addr 172.21.15.37:0/401379144 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b69d9b640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b6c15ba20 0x7f1b6c1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b6a59c640 1 -- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b6c15fd90 msgr2=0x7f1b6c0ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b6c15fd90 0x7f1b6c0ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b6a59c640 1 -- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b6c15ba20 msgr2=0x7f1b6c1546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b6c15ba20 0x7f1b6c1546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b6a59c640 1 -- 172.21.15.37:0/401379144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b6c0c4560 con 0x7f1b6c06b1d0 2024-04-25T08:58:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b69d9b640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b6c15ba20 0x7f1b6c1546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c06b1d0 0x7f1b6c151ba0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1b640af040 tx=0x7f1b640192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b62ffd640 1 -- 172.21.15.37:0/401379144 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b640b1040 con 0x7f1b6c06b1d0 2024-04-25T08:58:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b62ffd640 1 -- 172.21.15.37:0/401379144 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b640bc070 con 0x7f1b6c06b1d0 2024-04-25T08:58:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b70e18640 1 -- 172.21.15.37:0/401379144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b6c0c4840 con 0x7f1b6c06b1d0 2024-04-25T08:58:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b62ffd640 1 -- 172.21.15.37:0/401379144 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b640a8040 con 0x7f1b6c06b1d0 2024-04-25T08:58:31.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b70e18640 1 -- 172.21.15.37:0/401379144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6c154c10 con 0x7f1b6c06b1d0 2024-04-25T08:58:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b62ffd640 1 -- 172.21.15.37:0/401379144 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b640b5670 con 0x7f1b6c06b1d0 2024-04-25T08:58:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b62ffd640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b50066f60 0x7f1b50069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b62ffd640 1 -- 172.21.15.37:0/401379144 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1b6411ef80 con 0x7f1b6c06b1d0 2024-04-25T08:58:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b69d9b640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b50066f60 0x7f1b50069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b73f85000 1 Processor -- start 2024-04-25T08:58:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.492+0000 7f1b73f85000 1 -- start start 2024-04-25T08:58:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b69d9b640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b50066f60 0x7f1b50069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0b6300 tx=0x7f1b5c007490 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 0x559fd3b30b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b31060 0x559fd3ae67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd39c9ba0 con 0x559fd3ae6cf0 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd392f090 con 0x559fd3b31060 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fd38b7160 con 0x559fd3b30740 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49602/0 (socket says 172.21.15.37:49602) 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6ad9d640 1 -- 172.21.15.37:0/3505221729 learned_addr learned my addr 172.21.15.37:0/3505221729 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b69d9b640 1 --2- 172.21.15.37:0/3505221729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b31060 0x559fd3ae67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/3505221729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 0x559fd3b30b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6ad9d640 1 -- 172.21.15.37:0/3505221729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 msgr2=0x559fd3b30b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6ad9d640 1 --2- 172.21.15.37:0/3505221729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 0x559fd3b30b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6ad9d640 1 -- 172.21.15.37:0/3505221729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b31060 msgr2=0x559fd3ae67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6ad9d640 1 --2- 172.21.15.37:0/3505221729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b31060 0x559fd3ae67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6ad9d640 1 -- 172.21.15.37:0/3505221729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fd39d8c20 con 0x559fd3ae6cf0 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b69d9b640 1 --2- 172.21.15.37:0/3505221729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b31060 0x559fd3ae67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6ad9d640 1 --2- 172.21.15.37:0/3505221729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae70d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009780 tx=0x7f1b54009c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb9cd055f9f40f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b5bfff640 1 -- 172.21.15.37:0/3505221729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54015020 con 0x559fd3ae6cf0 2024-04-25T08:58:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b5bfff640 1 -- 172.21.15.37:0/3505221729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5400bad0 con 0x559fd3ae6cf0 2024-04-25T08:58:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b5bfff640 1 -- 172.21.15.37:0/3505221729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b54011420 con 0x559fd3ae6cf0 2024-04-25T08:58:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- 172.21.15.37:0/3505221729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 msgr2=0x559fd3ae70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 --2- 172.21.15.37:0/3505221729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae70d0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f1b54009780 tx=0x7f1b54009c40 comp rx=0 tx=0).stop 2024-04-25T08:58:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- 172.21.15.37:0/3505221729 shutdown_connections 2024-04-25T08:58:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 --2- 172.21.15.37:0/3505221729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae70d0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 --2- 172.21.15.37:0/3505221729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b31060 0x559fd3ae67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 --2- 172.21.15.37:0/3505221729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 0x559fd3b30b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- 172.21.15.37:0/3505221729 >> 172.21.15.37:0/3505221729 conn(0x559fd3ade890 msgr2=0x559fd3ae0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- 172.21.15.37:0/3505221729 shutdown_connections 2024-04-25T08:58:31.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- 172.21.15.37:0/3505221729 wait complete. 2024-04-25T08:58:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- start start 2024-04-25T08:58:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 Processor -- start 2024-04-25T08:58:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3b30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 0x559fd3af1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b31060 0x559fd3af21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd392b130 con 0x559fd3ae6cf0 2024-04-25T08:58:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd38f3130 con 0x559fd3b31060 2024-04-25T08:58:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fd39ed300 con 0x559fd3b30740 2024-04-25T08:58:31.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b73f85000 10 client.0 _ll_register_callbacks cb 0x559fd3acdbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3b30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3b30200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49610/0 (socket says 172.21.15.37:49610) 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6a59c640 1 -- 172.21.15.37:0/3375837018 learned_addr learned my addr 172.21.15.37:0/3375837018 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6a59c640 1 -- 172.21.15.37:0/3375837018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 msgr2=0x559fd3af1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/3375837018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 0x559fd3af1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6a59c640 1 -- 172.21.15.37:0/3375837018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b31060 msgr2=0x559fd3af21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/3375837018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b31060 0x559fd3af21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6a59c640 1 -- 172.21.15.37:0/3375837018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fd390c060 con 0x559fd3ae6cf0 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/3375837018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3b30200 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f1b640ef3b0 tx=0x7f1b640efef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195582be149f608f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.496+0000 7f1b597fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b597fa640 1 -- 172.21.15.37:0/3375837018 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b640eeb00 con 0x559fd3ae6cf0 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b597fa640 1 -- 172.21.15.37:0/3375837018 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1b640eeca0 con 0x559fd3ae6cf0 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 -- 172.21.15.37:0/3375837018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b5400bad0 con 0x559fd3ae6cf0 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b597fa640 1 -- 172.21.15.37:0/3375837018 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b6411b050 con 0x559fd3ae6cf0 2024-04-25T08:58:31.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 10 client.7503 fetch_fsmap learned FSMap version 52 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 -- 172.21.15.37:0/3375837018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x559fd38d07c0 con 0x559fd3ae6cf0 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b597fa640 1 -- 172.21.15.37:0/3375837018 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b5400bad0 con 0x559fd3ae6cf0 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b597fa640 1 -- 172.21.15.37:0/3375837018 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1b640b4050 con 0x559fd3ae6cf0 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 10 client.7503 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 10 client.7503 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 20 client.7503 populate_metadata read hostname 'smithi037' 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 --2- 172.21.15.37:0/3375837018 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x559fd3af2f40 0x559fd3b427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 4 client.7503 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 -- 172.21.15.37:0/3375837018 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fd38cacd0 con 0x559fd3af2f40 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b69d9b640 1 --2- 172.21.15.37:0/3375837018 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x559fd3af2f40 0x559fd3b427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b69d9b640 1 --2- 172.21.15.37:0/3375837018 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x559fd3af2f40 0x559fd3b427f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e07cb7c0b629c7b server_cookie=7f58eba9d04de803 in_seq=0 out_seq=0 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b597fa640 10 client.7503 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:31.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b597fa640 1 -- 172.21.15.37:0/3375837018 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559fd38cacd0 con 0x559fd3af2f40 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b597fa640 10 client.7503 handle_command_reply: tid=0 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 client.7503 shutdown 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 -- 172.21.15.37:0/3375837018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 msgr2=0x559fd3b30200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 --2- 172.21.15.37:0/3375837018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3b30200 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f1b640ef3b0 tx=0x7f1b640efef0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 -- 172.21.15.37:0/3375837018 shutdown_connections 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 --2- 172.21.15.37:0/3375837018 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x559fd3af2f40 0x559fd3b427f0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 --2- 172.21.15.37:0/3375837018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b31060 0x559fd3af21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b597fa640 0 client.7503 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 --2- 172.21.15.37:0/3375837018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 0x559fd3af1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 --2- 172.21.15.37:0/3375837018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3b30200 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 -- 172.21.15.37:0/3375837018 >> 172.21.15.37:0/3375837018 conn(0x559fd3ade890 msgr2=0x559fd3ae7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 -- 172.21.15.37:0/3375837018 shutdown_connections 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 1 -- 172.21.15.37:0/3375837018 wait complete. 2024-04-25T08:58:31.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.500+0000 7f1b73f85000 20 client.7503 trim_cache size 0 max 16384 2024-04-25T08:58:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 Processor -- start 2024-04-25T08:58:31.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 -- start start 2024-04-25T08:58:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b30740 0x559fd3ae70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b31060 0x559fd3ae7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c06f730 con 0x559fd3ae6cf0 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c0c8060 con 0x559fd3b30740 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b6c0c8660 con 0x559fd3b31060 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b6a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49618/0 (socket says 172.21.15.37:49618) 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b6a59c640 1 -- 172.21.15.37:0/852538604 learned_addr learned my addr 172.21.15.37:0/852538604 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b6a59c640 1 -- 172.21.15.37:0/852538604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b31060 msgr2=0x559fd3ae7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/852538604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b31060 0x559fd3ae7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b6a59c640 1 -- 172.21.15.37:0/852538604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b30740 msgr2=0x559fd3ae70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/852538604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b30740 0x559fd3ae70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b6a59c640 1 -- 172.21.15.37:0/852538604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fd38eafc0 con 0x559fd3ae6cf0 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/852538604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae8230 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f1b6401ca00 tx=0x7f1b640c3000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8276db6888d780d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b5b7fe640 1 -- 172.21.15.37:0/852538604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b640eb780 con 0x559fd3ae6cf0 2024-04-25T08:58:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b5b7fe640 1 -- 172.21.15.37:0/852538604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b640eb400 con 0x559fd3ae6cf0 2024-04-25T08:58:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b5b7fe640 1 -- 172.21.15.37:0/852538604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b640eb950 con 0x559fd3ae6cf0 2024-04-25T08:58:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 -- 172.21.15.37:0/852538604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 msgr2=0x559fd3ae8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 --2- 172.21.15.37:0/852538604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae8230 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f1b6401ca00 tx=0x7f1b640c3000 comp rx=0 tx=0).stop 2024-04-25T08:58:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 -- 172.21.15.37:0/852538604 shutdown_connections 2024-04-25T08:58:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 --2- 172.21.15.37:0/852538604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b31060 0x559fd3ae7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 --2- 172.21.15.37:0/852538604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3b30740 0x559fd3ae70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 --2- 172.21.15.37:0/852538604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3ae6cf0 0x559fd3ae8230 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 -- 172.21.15.37:0/852538604 >> 172.21.15.37:0/852538604 conn(0x559fd3ade890 msgr2=0x559fd3b3dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 -- 172.21.15.37:0/852538604 shutdown_connections 2024-04-25T08:58:31.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.512+0000 7f1b73f85000 1 -- 172.21.15.37:0/852538604 wait complete. 2024-04-25T08:58:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- start start 2024-04-25T08:58:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 Processor -- start 2024-04-25T08:58:31.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3ae6cf0 0x559fd3b90eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 0x559fd3b913f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3b31060 0x559fd3b42890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd38c9e70 con 0x559fd3b31060 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fd38b7a50 con 0x559fd3ae6cf0 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fd392e3a0 con 0x559fd3b30740 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 10 client.0 _ll_register_callbacks cb 0x559fd38db240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3b31060 0x559fd3b42890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3b31060 0x559fd3b42890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49632/0 (socket says 172.21.15.37:49632) 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6ad9d640 1 -- 172.21.15.37:0/4253412063 learned_addr learned my addr 172.21.15.37:0/4253412063 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3ae6cf0 0x559fd3b90eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6ad9d640 1 -- 172.21.15.37:0/4253412063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 msgr2=0x559fd3b913f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b69d9b640 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 0x559fd3b913f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6ad9d640 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 0x559fd3b913f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6ad9d640 1 -- 172.21.15.37:0/4253412063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3ae6cf0 msgr2=0x559fd3b90eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6ad9d640 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3ae6cf0 0x559fd3b90eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6ad9d640 1 -- 172.21.15.37:0/4253412063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fd39889e0 con 0x559fd3b31060 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3ae6cf0 0x559fd3b90eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6ad9d640 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3b31060 0x559fd3b42890 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f1b54014040 tx=0x7f1b5400bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772d7976b84ec3e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b597fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b597fa640 1 -- 172.21.15.37:0/4253412063 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5401a620 con 0x559fd3b31060 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b597fa640 1 -- 172.21.15.37:0/4253412063 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1b5401a7c0 con 0x559fd3b31060 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- 172.21.15.37:0/4253412063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b640eb400 con 0x559fd3b31060 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b597fa640 1 -- 172.21.15.37:0/4253412063 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b5402b050 con 0x559fd3b31060 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 10 client.7509 fetch_fsmap learned FSMap version 52 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- 172.21.15.37:0/4253412063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x559fd38ce630 con 0x559fd3b31060 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b597fa640 1 -- 172.21.15.37:0/4253412063 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b640eb400 con 0x559fd3b31060 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b597fa640 1 -- 172.21.15.37:0/4253412063 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 52) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1b5402e020 con 0x559fd3b31060 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 10 client.7509 fetch_fsmap finished waiting for FSMap version 52 2024-04-25T08:58:31.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 10 client.7509 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 20 client.7509 populate_metadata read hostname 'smithi037' 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x559fd3b97d10 0x559fd3b9a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 4 client.7509 mds_command: new command op to 7200 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- 172.21.15.37:0/4253412063 --> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x559fd3ade6e0 con 0x559fd3b97d10 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x559fd3b97d10 0x559fd3b9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b6a59c640 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x559fd3b97d10 0x559fd3b9a100 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff6b46dcba496b04 server_cookie=b940990d1b00f19e in_seq=0 out_seq=0 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b597fa640 10 client.7509 ms_handle_connect on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b597fa640 1 -- 172.21.15.37:0/4253412063 <== mds.0 v2:172.21.15.37:6834/3927528604 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559fd3ade6e0 con 0x559fd3b97d10 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b597fa640 10 client.7509 handle_command_reply: tid=0 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 client.7509 shutdown 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- 172.21.15.37:0/4253412063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3b31060 msgr2=0x559fd3b42890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3b31060 0x559fd3b42890 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f1b54014040 tx=0x7f1b5400bb00 comp rx=0 tx=0).stop 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- 172.21.15.37:0/4253412063 shutdown_connections 2024-04-25T08:58:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.37:6834/3927528604,v1:172.21.15.37:6835/3927528604] conn(0x559fd3b97d10 0x559fd3b9a100 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559fd3b31060 0x559fd3b42890 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x559fd3b30740 0x559fd3b913f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b597fa640 0 client.7509 ms_handle_reset on v2:172.21.15.37:6834/3927528604 2024-04-25T08:58:31.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 --2- 172.21.15.37:0/4253412063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x559fd3ae6cf0 0x559fd3b90eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- 172.21.15.37:0/4253412063 >> 172.21.15.37:0/4253412063 conn(0x559fd3ade890 msgr2=0x559fd3ae0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- 172.21.15.37:0/4253412063 shutdown_connections 2024-04-25T08:58:31.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 1 -- 172.21.15.37:0/4253412063 wait complete. 2024-04-25T08:58:31.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b73f85000 20 client.7509 trim_cache size 0 max 16384 2024-04-25T08:58:31.515 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:31.515 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-04-25T08:58:31.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 -- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b50066f60 msgr2=0x7f1b50069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b50066f60 0x7f1b50069420 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c0b6300 tx=0x7f1b5c007490 comp rx=0 tx=0).stop 2024-04-25T08:58:31.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 -- 172.21.15.37:0/401379144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c06b1d0 msgr2=0x7f1b6c151ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c06b1d0 0x7f1b6c151ba0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1b640af040 tx=0x7f1b640192c0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 -- 172.21.15.37:0/401379144 shutdown_connections 2024-04-25T08:58:31.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b50066f60 0x7f1b50069420 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b6c15fd90 0x7f1b6c0ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b6c15ba20 0x7f1b6c1546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 --2- 172.21.15.37:0/401379144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c06b1d0 0x7f1b6c151ba0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 -- 172.21.15.37:0/401379144 >> 172.21.15.37:0/401379144 conn(0x7f1b6c099a70 msgr2=0x7f1b6c15cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 -- 172.21.15.37:0/401379144 shutdown_connections 2024-04-25T08:58:31.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.516+0000 7f1b70e18640 1 -- 172.21.15.37:0/401379144 wait complete. 2024-04-25T08:58:31.524 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af9547640 1 Processor -- start 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af9547640 1 -- start start 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af9547640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5af415db70 0x7f5af40d1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af9547640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af40b3700 0x7f5af40b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af9547640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af40b2f40 con 0x7f5af40d0dc0 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af9547640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af40b4090 con 0x7f5af415db70 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af9547640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40d1fe0 con 0x7f5af40b3700 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5af415db70 0x7f5af40d1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af2ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5af415db70 0x7f5af40d1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49638/0 (socket says 172.21.15.37:49638) 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af2ffd640 1 -- 172.21.15.37:0/3762605918 learned_addr learned my addr 172.21.15.37:0/3762605918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af2ffd640 1 -- 172.21.15.37:0/3762605918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af40b3700 msgr2=0x7f5af40b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af37fe640 1 --2- 172.21.15.37:0/3762605918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af40b3700 0x7f5af40b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/3762605918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af40b3700 0x7f5af40b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af2ffd640 1 -- 172.21.15.37:0/3762605918 >> v1:172.21.15.37:6789/0 conn(0x7f5af40d0dc0 legacy=0x7f5af40ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af2ffd640 1 -- 172.21.15.37:0/3762605918 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af40d22c0 con 0x7f5af415db70 2024-04-25T08:58:31.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.636+0000 7f5af1ffb640 1 -- 172.21.15.37:0/3762605918 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4012830737 0 0) 0x7f5af40b2f40 con 0x7f5af40d0dc0 2024-04-25T08:58:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/3762605918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af415db70 0x7f5af40d1670 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009870 tx=0x7f5ae801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fb954ca61fa0af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af1ffb640 1 -- 172.21.15.37:0/3762605918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80aa600 con 0x7f5af415db70 2024-04-25T08:58:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af1ffb640 1 -- 172.21.15.37:0/3762605918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae80af070 con 0x7f5af415db70 2024-04-25T08:58:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af1ffb640 1 -- 172.21.15.37:0/3762605918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80aa9c0 con 0x7f5af415db70 2024-04-25T08:58:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af9547640 1 -- 172.21.15.37:0/3762605918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af415db70 msgr2=0x7f5af40d1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af9547640 1 --2- 172.21.15.37:0/3762605918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af415db70 0x7f5af40d1670 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009870 tx=0x7f5ae801c4a0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af9547640 1 -- 172.21.15.37:0/3762605918 shutdown_connections 2024-04-25T08:58:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af9547640 1 --2- 172.21.15.37:0/3762605918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af40b3700 0x7f5af40b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af9547640 1 --2- 172.21.15.37:0/3762605918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af415db70 0x7f5af40d1670 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af9547640 1 -- 172.21.15.37:0/3762605918 >> 172.21.15.37:0/3762605918 conn(0x7f5af409cab0 msgr2=0x7f5af408b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af9547640 1 -- 172.21.15.37:0/3762605918 shutdown_connections 2024-04-25T08:58:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af9547640 1 -- 172.21.15.37:0/3762605918 wait complete. 2024-04-25T08:58:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af9547640 1 Processor -- start 2024-04-25T08:58:31.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af9547640 1 -- start start 2024-04-25T08:58:31.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.640+0000 7f5af9547640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af40b3700 0x7f5af40c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af9547640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af40d0dc0 0x7f5af415ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af9547640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5af415db70 0x7f5af415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af9547640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40d1d30 con 0x7f5af40d0dc0 2024-04-25T08:58:31.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af9547640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af40d1bb0 con 0x7f5af415db70 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af9547640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af416a8a0 con 0x7f5af40b3700 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af40d0dc0 0x7f5af415ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af40d0dc0 0x7f5af415ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49640/0 (socket says 172.21.15.37:49640) 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af27fc640 1 -- 172.21.15.37:0/3546934804 learned_addr learned my addr 172.21.15.37:0/3546934804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af37fe640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5af415db70 0x7f5af415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af40b3700 0x7f5af40c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af27fc640 1 -- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af40b3700 msgr2=0x7f5af40c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af27fc640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af40b3700 0x7f5af40c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af27fc640 1 -- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5af415db70 msgr2=0x7f5af415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af27fc640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5af415db70 0x7f5af415c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af27fc640 1 -- 172.21.15.37:0/3546934804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af415d3a0 con 0x7f5af40d0dc0 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af37fe640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5af415db70 0x7f5af415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af27fc640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af40d0dc0 0x7f5af415ce60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400c470 tx=0x7f5ae400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5adb7fe640 1 -- 172.21.15.37:0/3546934804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40052d0 con 0x7f5af40d0dc0 2024-04-25T08:58:31.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5adb7fe640 1 -- 172.21.15.37:0/3546934804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae4005490 con 0x7f5af40d0dc0 2024-04-25T08:58:31.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af9547640 1 -- 172.21.15.37:0/3546934804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af40c9a20 con 0x7f5af40d0dc0 2024-04-25T08:58:31.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5adb7fe640 1 -- 172.21.15.37:0/3546934804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4012850 con 0x7f5af40d0dc0 2024-04-25T08:58:31.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af9547640 1 -- 172.21.15.37:0/3546934804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af40c9fe0 con 0x7f5af40d0dc0 2024-04-25T08:58:31.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af9547640 1 -- 172.21.15.37:0/3546934804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af404efa0 con 0x7f5af40d0dc0 2024-04-25T08:58:31.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5adb7fe640 1 -- 172.21.15.37:0/3546934804 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ae40129f0 con 0x7f5af40d0dc0 2024-04-25T08:58:31.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5adb7fe640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ae0066f60 0x7f5ae0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5adb7fe640 1 -- 172.21.15.37:0/3546934804 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5ae4084e50 con 0x7f5af40d0dc0 2024-04-25T08:58:31.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ae0066f60 0x7f5ae0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.644+0000 7f5af2ffd640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ae0066f60 0x7f5ae0069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80ae000 tx=0x7f5ae8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.648+0000 7f5adb7fe640 1 -- 172.21.15.37:0/3546934804 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ae4055c80 con 0x7f5af40d0dc0 2024-04-25T08:58:31.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 -- 172.21.15.37:0/3546934804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5af406cfa0 con 0x7f5af40d0dc0 2024-04-25T08:58:31.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5adb7fe640 1 -- 172.21.15.37:0/3546934804 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f5ae4055580 con 0x7f5af40d0dc0 2024-04-25T08:58:31.832 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:31.832 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:58:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 -- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ae0066f60 msgr2=0x7f5ae0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ae0066f60 0x7f5ae0069420 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80ae000 tx=0x7f5ae8008040 comp rx=0 tx=0).stop 2024-04-25T08:58:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 -- 172.21.15.37:0/3546934804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af40d0dc0 msgr2=0x7f5af415ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af40d0dc0 0x7f5af415ce60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f5ae400c470 tx=0x7f5ae400c930 comp rx=0 tx=0).stop 2024-04-25T08:58:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 -- 172.21.15.37:0/3546934804 shutdown_connections 2024-04-25T08:58:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5ae0066f60 0x7f5ae0069420 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5af415db70 0x7f5af415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5af40d0dc0 0x7f5af415ce60 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 --2- 172.21.15.37:0/3546934804 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5af40b3700 0x7f5af40c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 -- 172.21.15.37:0/3546934804 >> 172.21.15.37:0/3546934804 conn(0x7f5af409cab0 msgr2=0x7f5af406d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 -- 172.21.15.37:0/3546934804 shutdown_connections 2024-04-25T08:58:31.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.840+0000 7f5af9547640 1 -- 172.21.15.37:0/3546934804 wait complete. 2024-04-25T08:58:31.849 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T08:58:31.850 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:58:31.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb3f5c640 1 Processor -- start 2024-04-25T08:58:31.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb3f5c640 1 -- start start 2024-04-25T08:58:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb3f5c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efcac15a700 0x7efcac15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb3f5c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 0x7efcac0b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb3f5c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac0b2090 con 0x7efcac0b3f50 2024-04-25T08:58:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb3f5c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcac15f200 con 0x7efcac15a700 2024-04-25T08:58:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb3f5c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0b43e0 con 0x7efcac0b24a0 2024-04-25T08:58:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb14f7640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7efcac0b3f50 0x7efcac0cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:40618/0 (socket says 172.21.15.37:40618) 2024-04-25T08:58:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb24f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 0x7efcac0b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb14f7640 1 -- 172.21.15.37:0/1421451932 learned_addr learned my addr 172.21.15.37:0/1421451932 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:31.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb1cf8640 1 --2- 172.21.15.37:0/1421451932 >> v2:172.21.15.203:3300/0 conn(0x7efcac15a700 0x7efcac15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb0cf6640 1 -- 172.21.15.37:0/1421451932 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105264647 0 0) 0x7efcac0b2090 con 0x7efcac0b3f50 2024-04-25T08:58:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb0cf6640 1 -- 172.21.15.37:0/1421451932 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc98003530 con 0x7efcac0b3f50 2024-04-25T08:58:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb24f9640 1 -- 172.21.15.37:0/1421451932 >> v2:172.21.15.203:3300/0 conn(0x7efcac15a700 msgr2=0x7efcac15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb24f9640 1 --2- 172.21.15.37:0/1421451932 >> v2:172.21.15.203:3300/0 conn(0x7efcac15a700 0x7efcac15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb24f9640 1 -- 172.21.15.37:0/1421451932 >> v1:172.21.15.203:6789/0 conn(0x7efcac0b3f50 legacy=0x7efcac0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.976+0000 7efcb24f9640 1 -- 172.21.15.37:0/1421451932 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac0b46c0 con 0x7efcac0b24a0 2024-04-25T08:58:31.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb24f9640 1 --2- 172.21.15.37:0/1421451932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 0x7efcac0b2930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efca4009d90 tx=0x7efca401c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbd0e0522d2ee3c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb0cf6640 1 -- 172.21.15.37:0/1421451932 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40b2070 con 0x7efcac0b24a0 2024-04-25T08:58:31.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb0cf6640 1 -- 172.21.15.37:0/1421451932 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca40a5800 con 0x7efcac0b24a0 2024-04-25T08:58:31.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb0cf6640 1 -- 172.21.15.37:0/1421451932 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40a5bb0 con 0x7efcac0b24a0 2024-04-25T08:58:31.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb3f5c640 1 -- 172.21.15.37:0/1421451932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 msgr2=0x7efcac0b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb3f5c640 1 --2- 172.21.15.37:0/1421451932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 0x7efcac0b2930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efca4009d90 tx=0x7efca401c850 comp rx=0 tx=0).stop 2024-04-25T08:58:31.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb3f5c640 1 -- 172.21.15.37:0/1421451932 shutdown_connections 2024-04-25T08:58:31.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb3f5c640 1 --2- 172.21.15.37:0/1421451932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 0x7efcac0b2930 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb3f5c640 1 --2- 172.21.15.37:0/1421451932 >> v2:172.21.15.203:3300/0 conn(0x7efcac15a700 0x7efcac15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb3f5c640 1 -- 172.21.15.37:0/1421451932 >> 172.21.15.37:0/1421451932 conn(0x7efcac0b11a0 msgr2=0x7efcac0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb3f5c640 1 -- 172.21.15.37:0/1421451932 shutdown_connections 2024-04-25T08:58:31.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb3f5c640 1 -- 172.21.15.37:0/1421451932 wait complete. 2024-04-25T08:58:31.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb3f5c640 1 Processor -- start 2024-04-25T08:58:31.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.980+0000 7efcb3f5c640 1 -- start start 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb3f5c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 0x7efcac0aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb3f5c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcac15a700 0x7efcac0abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb3f5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcac151e20 0x7efcac143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb3f5c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0cd310 con 0x7efcac151e20 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb3f5c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0ce600 con 0x7efcac15a700 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb3f5c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0cd490 con 0x7efcac0b24a0 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb24f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcac151e20 0x7efcac143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb24f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcac151e20 0x7efcac143e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49652/0 (socket says 172.21.15.37:49652) 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb24f9640 1 -- 172.21.15.37:0/421894546 learned_addr learned my addr 172.21.15.37:0/421894546 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb14f7640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcac15a700 0x7efcac0abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb1cf8640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 0x7efcac0aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb24f9640 1 -- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 msgr2=0x7efcac0aa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb24f9640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 0x7efcac0aa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb24f9640 1 -- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcac15a700 msgr2=0x7efcac0abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb24f9640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcac15a700 0x7efcac0abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb24f9640 1 -- 172.21.15.37:0/421894546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac144390 con 0x7efcac151e20 2024-04-25T08:58:31.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb1cf8640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 0x7efcac0aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:31.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb24f9640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcac151e20 0x7efcac143e50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7efca401a480 tx=0x7efca40195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efca27fc640 1 -- 172.21.15.37:0/421894546 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40bd020 con 0x7efcac151e20 2024-04-25T08:58:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efca27fc640 1 -- 172.21.15.37:0/421894546 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca40039a0 con 0x7efcac151e20 2024-04-25T08:58:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb3f5c640 1 -- 172.21.15.37:0/421894546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcac159bc0 con 0x7efcac151e20 2024-04-25T08:58:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efca27fc640 1 -- 172.21.15.37:0/421894546 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40b2040 con 0x7efcac151e20 2024-04-25T08:58:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb3f5c640 1 -- 172.21.15.37:0/421894546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac0af540 con 0x7efcac151e20 2024-04-25T08:58:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb3f5c640 1 -- 172.21.15.37:0/421894546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcac04efa0 con 0x7efcac151e20 2024-04-25T08:58:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efca27fc640 1 -- 172.21.15.37:0/421894546 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efca40c0020 con 0x7efcac151e20 2024-04-25T08:58:31.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efca27fc640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc90066f90 0x7efc90069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efca27fc640 1 -- 172.21.15.37:0/421894546 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7efca411ff50 con 0x7efcac151e20 2024-04-25T08:58:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.984+0000 7efcb1cf8640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc90066f90 0x7efc90069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.988+0000 7efcb1cf8640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc90066f90 0x7efc90069450 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efcac0cb9f0 tx=0x7efca8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:31.992+0000 7efca27fc640 1 -- 172.21.15.37:0/421894546 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efca40c7030 con 0x7efcac151e20 2024-04-25T08:58:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.184+0000 7efcb3f5c640 1 -- 172.21.15.37:0/421894546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efcac06f220 con 0x7efcac151e20 2024-04-25T08:58:32.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.184+0000 7efca27fc640 1 -- 172.21.15.37:0/421894546 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7efca4016560 con 0x7efcac151e20 2024-04-25T08:58:32.177 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:58:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 -- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc90066f90 msgr2=0x7efc90069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc90066f90 0x7efc90069450 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7efcac0cb9f0 tx=0x7efca8008040 comp rx=0 tx=0).stop 2024-04-25T08:58:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 -- 172.21.15.37:0/421894546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcac151e20 msgr2=0x7efcac143e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcac151e20 0x7efcac143e50 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7efca401a480 tx=0x7efca40195f0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 -- 172.21.15.37:0/421894546 shutdown_connections 2024-04-25T08:58:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efc90066f90 0x7efc90069450 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcac151e20 0x7efcac143e50 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcac15a700 0x7efcac0abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 --2- 172.21.15.37:0/421894546 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcac0b24a0 0x7efcac0aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 -- 172.21.15.37:0/421894546 >> 172.21.15.37:0/421894546 conn(0x7efcac0b11a0 msgr2=0x7efcac0cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:32.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 -- 172.21.15.37:0/421894546 shutdown_connections 2024-04-25T08:58:32.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:32.188+0000 7efcb3f5c640 1 -- 172.21.15.37:0/421894546 wait complete. 2024-04-25T08:58:32.194 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:58:32.195 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:58:32.195 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T08:58:32.195 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:58:32.195 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:32.195 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:58:32.195 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:58:32.195 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:58:32.195 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:32.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:32.195 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T08:58:32.202 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:58:32.202 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:58:32.202 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:58:32.202 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:32.202 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:58:32.202 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:32.202 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:58:32.202 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:32.202 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T08:58:32.202 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:58:32.202 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42128sec preferred_lft 42128sec 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:32.203 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:32.204 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:58:32.205 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:32.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:32.205 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:58:32.255 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:58:32.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:32.256 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T08:58:32.304 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T08:58:32.304 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:58:32.304 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:58:32.304 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:58:32.304 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:58:32.304 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:58:32.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:32.305 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:58:32.305 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:58:32.305 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:58:32.305 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:58:32.305 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:58:32.305 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:58:32.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:32.376 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:58:32.423 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:58:32.423 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:58:32.423 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:32.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:32.423 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T08:58:32.479 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:58:32.479 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:58:32.479 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:32.527 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:32.527 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:58:32.527 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:32.575 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:58:32.632 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:58:32.681 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:32.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:32.683 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:58:32.731 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:58:32.731 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:58:32.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:58:32.790 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:58:32.839 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:58:32.839 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:58:32.839 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:58:32.839 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:58:32.839 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:58:32.839 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:58:32.839 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:58:32.839 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:58:32.839 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:58:32.839 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:58:32.840 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:58:32.841 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:58:32.841 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:58:32.841 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:58:32.841 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:58:32.841 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:58:32.841 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:58:32.841 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:58:32.841 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:32.841 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:32.841 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:32.841 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:32.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:32.842 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:58:32.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T08:58:32.868+0000 7f4a51ad4580 -1 init, newargv = 0x55e830812490 newargc=15 2024-04-25T08:58:32.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[46538]: starting ceph client 2024-04-25T08:58:32.866 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[46538]: starting fuse 2024-04-25T08:58:33.964 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:58:33.980 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:58:34.029 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:58:34.030 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:58:34.031 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:58:34.031 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:58:34.031 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:58:34.031 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:58:34.031 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:58:34.031 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:58:34.031 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:58:34.031 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:58:34.031 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:58:34.031 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:58:34.031 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:58:34.032 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:58:34.032 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:58:34.032 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:58:34.032 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:58:34.032 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:58:34.032 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:34.032 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:34.032 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:34.032 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:34.032 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:34.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:34.033 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:58:34.080 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:58:34.080 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:58:34.081 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:58:34.081 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T08:58:34.081 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T08:58:34.081 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:58:34.081 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T08:58:34.082 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:58:34.082 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T08:58:34.082 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:58:34.082 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T08:58:34.082 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:58:34.082 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:58:34.083 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T08:58:34.083 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T08:58:34.083 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T08:58:34.083 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T08:58:34.083 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:58:34.083 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T08:58:34.083 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:58:34.084 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:58:34.084 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:58:34.084 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T08:58:34.084 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T08:58:34.084 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T08:58:34.084 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:58:34.084 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:58:34.084 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T08:58:34.085 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:58:34.207 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.46538.asok 2024-04-25T08:58:34.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:58:34.219 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46538.asok 2024-04-25T08:58:34.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:34.219 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46538.asok status 2024-04-25T08:58:34.374 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:58:34.374 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "46538", 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7527, 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T08:58:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 7527 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1770068767 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1770068767 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.7527 192.168.0.1:0/1770068767", 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/1770068767", 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T08:58:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 52, 2024-04-25T08:58:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 83, 2024-04-25T08:58:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:58:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T08:58:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T08:58:34.377 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:58:34.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:34.388 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T08:58:34.389 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:34.395 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T08:58:34.395 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:34.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:34.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:34.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:34.450 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-04-25T08:58:34.504 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:58:34.505 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-04-25T08:58:34.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 Processor -- start 2024-04-25T08:58:34.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 -- start start 2024-04-25T08:58:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d323292570 0x55d323366a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 0x55d323348000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d32336a1f0 0x55d323348540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3232c6800 con 0x55d3232e0360 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3232399f0 con 0x55d32336a1f0 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3232c6980 con 0x55d323292570 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f14a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 0x55d323348000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f14a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 0x55d323348000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49676/0 (socket says 172.21.15.37:49676) 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f14a8640 1 -- 172.21.15.37:0/998872843 learned_addr learned my addr 172.21.15.37:0/998872843 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f1ca9640 1 --2- 172.21.15.37:0/998872843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d323292570 0x55d323366a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f24aa640 1 --2- 172.21.15.37:0/998872843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d32336a1f0 0x55d323348540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f14a8640 1 -- 172.21.15.37:0/998872843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d323292570 msgr2=0x55d323366a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f14a8640 1 --2- 172.21.15.37:0/998872843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d323292570 0x55d323366a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f14a8640 1 -- 172.21.15.37:0/998872843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d32336a1f0 msgr2=0x55d323348540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f14a8640 1 --2- 172.21.15.37:0/998872843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d32336a1f0 0x55d323348540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f14a8640 1 -- 172.21.15.37:0/998872843 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3233688f0 con 0x55d3232e0360 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f14a8640 1 --2- 172.21.15.37:0/998872843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 0x55d323348000 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00a750 tx=0x7fd4dc00ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19695aa441ff414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4ebfff640 1 -- 172.21.15.37:0/998872843 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00ae60 con 0x55d3232e0360 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4ebfff640 1 -- 172.21.15.37:0/998872843 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc008c00 con 0x55d3232e0360 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4ebfff640 1 -- 172.21.15.37:0/998872843 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0133f0 con 0x55d3232e0360 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 -- 172.21.15.37:0/998872843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 msgr2=0x55d323348000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 --2- 172.21.15.37:0/998872843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 0x55d323348000 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00a750 tx=0x7fd4dc00ac10 comp rx=0 tx=0).stop 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 -- 172.21.15.37:0/998872843 shutdown_connections 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 --2- 172.21.15.37:0/998872843 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d32336a1f0 0x55d323348540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 --2- 172.21.15.37:0/998872843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 0x55d323348000 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 --2- 172.21.15.37:0/998872843 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d323292570 0x55d323366a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 -- 172.21.15.37:0/998872843 >> 172.21.15.37:0/998872843 conn(0x55d323298c90 msgr2=0x55d3232c4650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:34.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 -- 172.21.15.37:0/998872843 shutdown_connections 2024-04-25T08:58:34.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.596+0000 7fd4f418b080 1 -- 172.21.15.37:0/998872843 wait complete. 2024-04-25T08:58:34.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f418b080 1 Processor -- start 2024-04-25T08:58:34.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f418b080 1 -- start start 2024-04-25T08:58:34.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f418b080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d323292570 0x55d3232c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f418b080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 0x55d323366ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f418b080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d32336a1f0 0x55d3233ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f418b080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d323374800 con 0x55d3232e0360 2024-04-25T08:58:34.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f418b080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3232d6a70 con 0x55d323292570 2024-04-25T08:58:34.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f418b080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3232d7590 con 0x55d32336a1f0 2024-04-25T08:58:34.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f14a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 0x55d323366ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f14a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 0x55d323366ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49682/0 (socket says 172.21.15.37:49682) 2024-04-25T08:58:34.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f14a8640 1 -- 172.21.15.37:0/1216411929 learned_addr learned my addr 172.21.15.37:0/1216411929 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:34.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f1ca9640 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d323292570 0x55d3232c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f24aa640 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d32336a1f0 0x55d3233ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f14a8640 1 -- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d32336a1f0 msgr2=0x55d3233ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f14a8640 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55d32336a1f0 0x55d3233ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f14a8640 1 -- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d323292570 msgr2=0x55d3232c22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f14a8640 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d323292570 0x55d3232c22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f14a8640 1 -- 172.21.15.37:0/1216411929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3233fde80 con 0x55d3232e0360 2024-04-25T08:58:34.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f1ca9640 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55d323292570 0x55d3232c22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:34.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f14a8640 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 0x55d323366ba0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0087e0 tx=0x7fd4dc003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4eaffd640 1 -- 172.21.15.37:0/1216411929 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc027410 con 0x55d3232e0360 2024-04-25T08:58:34.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4eaffd640 1 -- 172.21.15.37:0/1216411929 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc0275b0 con 0x55d3232e0360 2024-04-25T08:58:34.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f418b080 1 -- 172.21.15.37:0/1216411929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d3233fe120 con 0x55d3232e0360 2024-04-25T08:58:34.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4eaffd640 1 -- 172.21.15.37:0/1216411929 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0279a0 con 0x55d3232e0360 2024-04-25T08:58:34.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.600+0000 7fd4f418b080 1 -- 172.21.15.37:0/1216411929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3233fe660 con 0x55d3232e0360 2024-04-25T08:58:34.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4eaffd640 1 -- 172.21.15.37:0/1216411929 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4dc025020 con 0x55d3232e0360 2024-04-25T08:58:34.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4eaffd640 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4d4065970 0x7fd4d4067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4eaffd640 1 -- 172.21.15.37:0/1216411929 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd4dc010030 con 0x55d3232e0360 2024-04-25T08:58:34.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f418b080 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55d323296060 0x55d323296440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f418b080 1 -- 172.21.15.37:0/1216411929 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:1 11.10 11:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e83) v9 -- 0x55d323296980 con 0x55d323296060 2024-04-25T08:58:34.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f1ca9640 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4d4065970 0x7fd4d4067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f24aa640 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55d323296060 0x55d323296440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f24aa640 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55d323296060 0x55d323296440 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f1ca9640 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4d4065970 0x7fd4d4067e30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0ab000 tx=0x7fd4ec0a5000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f24aa640 1 -- 172.21.15.37:0/1216411929 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v83'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fd4e400d040 con 0x55d323296060 2024-04-25T08:58:34.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f418b080 1 -- 172.21.15.37:0/1216411929 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55d323296060 msgr2=0x55d323296440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f418b080 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55d323296060 0x55d323296440 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f418b080 1 -- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4d4065970 msgr2=0x7fd4d4067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f418b080 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4d4065970 0x7fd4d4067e30 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0ab000 tx=0x7fd4ec0a5000 comp rx=0 tx=0).stop 2024-04-25T08:58:34.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f418b080 1 -- 172.21.15.37:0/1216411929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 msgr2=0x55d323366ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f418b080 1 --2- 172.21.15.37:0/1216411929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3232e0360 0x55d323366ba0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc0087e0 tx=0x7fd4dc003030 comp rx=0 tx=0).stop 2024-04-25T08:58:34.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.604+0000 7fd4f24aa640 1 -- 172.21.15.37:0/1216411929 reap_dead start 2024-04-25T08:58:34.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.608+0000 7fd4f418b080 1 -- 172.21.15.37:0/1216411929 shutdown_connections 2024-04-25T08:58:34.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.608+0000 7fd4f418b080 1 -- 172.21.15.37:0/1216411929 >> 172.21.15.37:0/1216411929 conn(0x55d323298c90 msgr2=0x55d3232d7740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:34.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.608+0000 7fd4f418b080 1 -- 172.21.15.37:0/1216411929 shutdown_connections 2024-04-25T08:58:34.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.608+0000 7fd4f418b080 1 -- 172.21.15.37:0/1216411929 wait complete. 2024-04-25T08:58:34.602 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:58:34.603 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:58:34.688 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:58:34.730 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:58:34.731 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:58:34.731 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:34.737 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:34.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.920+0000 7f8cd0dd0640 1 Processor -- start 2024-04-25T08:58:34.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.920+0000 7f8cd0dd0640 1 -- start start 2024-04-25T08:58:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8cd0dd0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8ccc0ce510 0x7f8ccc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8cd0dd0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ccc144ca0 0x7f8ccc06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8cd0dd0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8ccc06ec60 0x7f8ccc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8cd0dd0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc0cc2b0 con 0x7f8ccc06ec60 2024-04-25T08:58:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8cd0dd0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ccc06e290 con 0x7f8ccc0ce510 2024-04-25T08:58:34.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8cd0dd0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0cee30 con 0x7f8ccc144ca0 2024-04-25T08:58:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8ccad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8ccc06ec60 0x7f8ccc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8ccad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8ccc06ec60 0x7f8ccc142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49690/0 (socket says 172.21.15.37:49690) 2024-04-25T08:58:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8ccad9d640 1 -- 172.21.15.37:0/2358394932 learned_addr learned my addr 172.21.15.37:0/2358394932 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8cc9d9b640 1 --2- 172.21.15.37:0/2358394932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ccc144ca0 0x7f8ccc06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8cca59c640 1 --2- 172.21.15.37:0/2358394932 >> v2:172.21.15.203:3300/0 conn(0x7f8ccc0ce510 0x7f8ccc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8ccad9d640 1 -- 172.21.15.37:0/2358394932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ccc144ca0 msgr2=0x7f8ccc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8ccad9d640 1 --2- 172.21.15.37:0/2358394932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ccc144ca0 0x7f8ccc06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8ccad9d640 1 -- 172.21.15.37:0/2358394932 >> v2:172.21.15.203:3300/0 conn(0x7f8ccc0ce510 msgr2=0x7f8ccc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8ccad9d640 1 --2- 172.21.15.37:0/2358394932 >> v2:172.21.15.203:3300/0 conn(0x7f8ccc0ce510 0x7f8ccc0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.924+0000 7f8ccad9d640 1 -- 172.21.15.37:0/2358394932 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc0ccc70 con 0x7f8ccc06ec60 2024-04-25T08:58:34.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8ccad9d640 1 --2- 172.21.15.37:0/2358394932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ccc06ec60 0x7f8ccc142680 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc144c50 tx=0x7f8cc401ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2ff1a43884cfb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cc959a640 1 -- 172.21.15.37:0/2358394932 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40b5070 con 0x7f8ccc06ec60 2024-04-25T08:58:34.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cc959a640 1 -- 172.21.15.37:0/2358394932 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc40a89d0 con 0x7f8ccc06ec60 2024-04-25T08:58:34.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cc959a640 1 -- 172.21.15.37:0/2358394932 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40a8da0 con 0x7f8ccc06ec60 2024-04-25T08:58:34.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/2358394932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ccc06ec60 msgr2=0x7f8ccc142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 --2- 172.21.15.37:0/2358394932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ccc06ec60 0x7f8ccc142680 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc144c50 tx=0x7f8cc401ed10 comp rx=0 tx=0).stop 2024-04-25T08:58:34.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/2358394932 shutdown_connections 2024-04-25T08:58:34.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 --2- 172.21.15.37:0/2358394932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ccc06ec60 0x7f8ccc142680 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 --2- 172.21.15.37:0/2358394932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ccc144ca0 0x7f8ccc06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 --2- 172.21.15.37:0/2358394932 >> v2:172.21.15.203:3300/0 conn(0x7f8ccc0ce510 0x7f8ccc0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/2358394932 >> 172.21.15.37:0/2358394932 conn(0x7f8ccc0b11a0 msgr2=0x7f8ccc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:34.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/2358394932 shutdown_connections 2024-04-25T08:58:34.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/2358394932 wait complete. 2024-04-25T08:58:34.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 Processor -- start 2024-04-25T08:58:34.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 -- start start 2024-04-25T08:58:34.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ccc06ec60 0x7f8ccc1f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ccc0ce510 0x7f8ccc1f4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ccc144ca0 0x7f8ccc1316e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc0cc900 con 0x7f8ccc144ca0 2024-04-25T08:58:34.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc06df20 con 0x7f8ccc0ce510 2024-04-25T08:58:34.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cd0dd0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc1451f0 con 0x7f8ccc06ec60 2024-04-25T08:58:34.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8ccad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ccc144ca0 0x7f8ccc1316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8ccad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ccc144ca0 0x7f8ccc1316e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49698/0 (socket says 172.21.15.37:49698) 2024-04-25T08:58:34.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8ccad9d640 1 -- 172.21.15.37:0/4082238130 learned_addr learned my addr 172.21.15.37:0/4082238130 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:34.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cca59c640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ccc06ec60 0x7f8ccc1f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8cc9d9b640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ccc0ce510 0x7f8ccc1f4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8ccad9d640 1 -- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ccc06ec60 msgr2=0x7f8ccc1f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8ccad9d640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ccc06ec60 0x7f8ccc1f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8ccad9d640 1 -- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ccc0ce510 msgr2=0x7f8ccc1f4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8ccad9d640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ccc0ce510 0x7f8ccc1f4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8ccad9d640 1 -- 172.21.15.37:0/4082238130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc12ee90 con 0x7f8ccc144ca0 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.928+0000 7f8ccad9d640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ccc144ca0 0x7f8ccc1316e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8cc401f5b0 tx=0x7f8cc401bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.932+0000 7f8cc2ffd640 1 -- 172.21.15.37:0/4082238130 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40b5070 con 0x7f8ccc144ca0 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.932+0000 7f8cc2ffd640 1 -- 172.21.15.37:0/4082238130 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc40b0a10 con 0x7f8ccc144ca0 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.932+0000 7f8cc2ffd640 1 -- 172.21.15.37:0/4082238130 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc40b0e00 con 0x7f8ccc144ca0 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.932+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/4082238130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc12f150 con 0x7f8ccc144ca0 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.932+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/4082238130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc12f710 con 0x7f8ccc144ca0 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.932+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/4082238130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc04efa0 con 0x7f8ccc144ca0 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.932+0000 7f8cc2ffd640 1 -- 172.21.15.37:0/4082238130 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8cc40c2060 con 0x7f8ccc144ca0 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.932+0000 7f8cc2ffd640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8cac066f60 0x7f8cac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.932+0000 7f8cc2ffd640 1 -- 172.21.15.37:0/4082238130 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8cc4122410 con 0x7f8ccc144ca0 2024-04-25T08:58:34.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.932+0000 7f8cca59c640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8cac066f60 0x7f8cac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.932+0000 7f8cca59c640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8cac066f60 0x7f8cac069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400ba30 tx=0x7f8cb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:34.936+0000 7f8cc2ffd640 1 -- 172.21.15.37:0/4082238130 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8cc4018870 con 0x7f8ccc144ca0 2024-04-25T08:58:35.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.100+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/4082238130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ccc0cea20 con 0x7f8ccc144ca0 2024-04-25T08:58:35.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.100+0000 7f8cc2ffd640 1 -- 172.21.15.37:0/4082238130 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17038 (secure 0 0 0) 0x7f8cc40f31a0 con 0x7f8ccc144ca0 2024-04-25T08:58:35.093 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:35.094 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":83,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:57:11.180680+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:56:49.737591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:56:50.436964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/4130042812":"2024-04-26T08:57:11.180643+0000","172.21.15.37:6834/4130042812":"2024-04-26T08:57:11.180643+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8cac066f60 msgr2=0x7f8cac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8cac066f60 0x7f8cac069420 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400ba30 tx=0x7f8cb4008040 comp rx=0 tx=0).stop 2024-04-25T08:58:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/4082238130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ccc144ca0 msgr2=0x7f8ccc1316e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ccc144ca0 0x7f8ccc1316e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f8cc401f5b0 tx=0x7f8cc401bab0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/4082238130 shutdown_connections 2024-04-25T08:58:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8cac066f60 0x7f8cac069420 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ccc144ca0 0x7f8ccc1316e0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8ccc0ce510 0x7f8ccc1f4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 --2- 172.21.15.37:0/4082238130 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8ccc06ec60 0x7f8ccc1f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/4082238130 >> 172.21.15.37:0/4082238130 conn(0x7f8ccc0b11a0 msgr2=0x7f8ccc0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/4082238130 shutdown_connections 2024-04-25T08:58:35.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.104+0000 7f8cd0dd0640 1 -- 172.21.15.37:0/4082238130 wait complete. 2024-04-25T08:58:35.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:56:49.737591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:56:50.436964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:58:35.113 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:58:35.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.244+0000 7f5ce301c640 1 Processor -- start 2024-04-25T08:58:35.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.244+0000 7f5ce301c640 1 -- start start 2024-04-25T08:58:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce301c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5cdc06fc50 0x7f5cdc06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce301c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cdc0cff50 0x7f5cdc15d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce301c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cdc06b710 con 0x7f5cdc06b1d0 2024-04-25T08:58:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce301c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cdc161560 con 0x7f5cdc06fc50 2024-04-25T08:58:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce301c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc1617b0 con 0x7f5cdc0cff50 2024-04-25T08:58:35.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce0db8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5cdc06fc50 0x7f5cdc06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce0db8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5cdc06fc50 0x7f5cdc06ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49712/0 (socket says 172.21.15.37:49712) 2024-04-25T08:58:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce0db8640 1 -- 172.21.15.37:0/3291369680 learned_addr learned my addr 172.21.15.37:0/3291369680 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce15b9640 1 --2- 172.21.15.37:0/3291369680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cdc0cff50 0x7f5cdc15d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce0db8640 1 -- 172.21.15.37:0/3291369680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cdc0cff50 msgr2=0x7f5cdc15d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce0db8640 1 --2- 172.21.15.37:0/3291369680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cdc0cff50 0x7f5cdc15d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce0db8640 1 -- 172.21.15.37:0/3291369680 >> v1:172.21.15.203:6789/0 conn(0x7f5cdc06b1d0 legacy=0x7f5cdc0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce0db8640 1 -- 172.21.15.37:0/3291369680 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc161a90 con 0x7f5cdc06fc50 2024-04-25T08:58:35.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5ce0db8640 1 --2- 172.21.15.37:0/3291369680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cdc06fc50 0x7f5cdc06ac90 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009870 tx=0x7f5ccc01c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbd5d15f55febf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5cdb7fe640 1 -- 172.21.15.37:0/3291369680 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0ab640 con 0x7f5cdc06fc50 2024-04-25T08:58:35.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5cdb7fe640 1 -- 172.21.15.37:0/3291369680 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc0b0070 con 0x7f5cdc06fc50 2024-04-25T08:58:35.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.248+0000 7f5cdb7fe640 1 -- 172.21.15.37:0/3291369680 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc0aba00 con 0x7f5cdc06fc50 2024-04-25T08:58:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 -- 172.21.15.37:0/3291369680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cdc06fc50 msgr2=0x7f5cdc06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 --2- 172.21.15.37:0/3291369680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cdc06fc50 0x7f5cdc06ac90 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc009870 tx=0x7f5ccc01c530 comp rx=0 tx=0).stop 2024-04-25T08:58:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 -- 172.21.15.37:0/3291369680 shutdown_connections 2024-04-25T08:58:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 --2- 172.21.15.37:0/3291369680 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cdc0cff50 0x7f5cdc15d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 --2- 172.21.15.37:0/3291369680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cdc06fc50 0x7f5cdc06ac90 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 -- 172.21.15.37:0/3291369680 >> 172.21.15.37:0/3291369680 conn(0x7f5cdc099a70 msgr2=0x7f5cdc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 -- 172.21.15.37:0/3291369680 shutdown_connections 2024-04-25T08:58:35.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 -- 172.21.15.37:0/3291369680 wait complete. 2024-04-25T08:58:35.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 Processor -- start 2024-04-25T08:58:35.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 -- start start 2024-04-25T08:58:35.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cdc06b1d0 0x7f5cdc0c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cdc06fc50 0x7f5cdc0b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cdc0cff50 0x7f5cdc15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc162410 con 0x7f5cdc06fc50 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc162290 con 0x7f5cdc0cff50 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc0cc6e0 con 0x7f5cdc06b1d0 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cdbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cdc06fc50 0x7f5cdc0b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cdbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cdc06fc50 0x7f5cdc0b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49718/0 (socket says 172.21.15.37:49718) 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cdbfff640 1 -- 172.21.15.37:0/4265771772 learned_addr learned my addr 172.21.15.37:0/4265771772 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce0db8640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cdc06b1d0 0x7f5cdc0c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce15b9640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cdc0cff50 0x7f5cdc15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cdbfff640 1 -- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cdc06b1d0 msgr2=0x7f5cdc0c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cdbfff640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cdc06b1d0 0x7f5cdc0c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cdbfff640 1 -- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cdc0cff50 msgr2=0x7f5cdc15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cdbfff640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cdc0cff50 0x7f5cdc15a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cdbfff640 1 -- 172.21.15.37:0/4265771772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc0c7940 con 0x7f5cdc06fc50 2024-04-25T08:58:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce15b9640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cdc0cff50 0x7f5cdc15a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cdbfff640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cdc06fc50 0x7f5cdc0b3b30 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000c430 tx=0x7f5cd000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cd97fa640 1 -- 172.21.15.37:0/4265771772 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00052d0 con 0x7f5cdc06fc50 2024-04-25T08:58:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cd97fa640 1 -- 172.21.15.37:0/4265771772 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd0005470 con 0x7f5cdc06fc50 2024-04-25T08:58:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 -- 172.21.15.37:0/4265771772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cdc0c7c20 con 0x7f5cdc06fc50 2024-04-25T08:58:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5cd97fa640 1 -- 172.21.15.37:0/4265771772 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0007260 con 0x7f5cdc06fc50 2024-04-25T08:58:35.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.252+0000 7f5ce301c640 1 -- 172.21.15.37:0/4265771772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cdc0c81e0 con 0x7f5cdc06fc50 2024-04-25T08:58:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.256+0000 7f5ce301c640 1 -- 172.21.15.37:0/4265771772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cdc04efa0 con 0x7f5cdc06fc50 2024-04-25T08:58:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.256+0000 7f5cd97fa640 1 -- 172.21.15.37:0/4265771772 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cd0007400 con 0x7f5cdc06fc50 2024-04-25T08:58:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.256+0000 7f5cd97fa640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cc8066f60 0x7f5cc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.256+0000 7f5cd97fa640 1 -- 172.21.15.37:0/4265771772 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5cd0084670 con 0x7f5cdc06fc50 2024-04-25T08:58:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.256+0000 7f5ce0db8640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cc8066f60 0x7f5cc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.256+0000 7f5ce0db8640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cc8066f60 0x7f5cc8069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0af000 tx=0x7f5ccc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.260+0000 7f5cd97fa640 1 -- 172.21.15.37:0/4265771772 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5cd0054e50 con 0x7f5cdc06fc50 2024-04-25T08:58:35.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.420+0000 7f5ce301c640 1 -- 172.21.15.37:0/4265771772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5cdc0cad10 con 0x7f5cdc06fc50 2024-04-25T08:58:35.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.420+0000 7f5cd97fa640 1 -- 172.21.15.37:0/4265771772 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5cd0058ef0 con 0x7f5cdc06fc50 2024-04-25T08:58:35.414 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:58:35.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 -- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cc8066f60 msgr2=0x7f5cc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cc8066f60 0x7f5cc8069420 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0af000 tx=0x7f5ccc0a7000 comp rx=0 tx=0).stop 2024-04-25T08:58:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 -- 172.21.15.37:0/4265771772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cdc06fc50 msgr2=0x7f5cdc0b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cdc06fc50 0x7f5cdc0b3b30 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000c430 tx=0x7f5cd000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 -- 172.21.15.37:0/4265771772 shutdown_connections 2024-04-25T08:58:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cc8066f60 0x7f5cc8069420 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cdc0cff50 0x7f5cdc15a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cdc06fc50 0x7f5cdc0b3b30 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 --2- 172.21.15.37:0/4265771772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cdc06b1d0 0x7f5cdc0c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 -- 172.21.15.37:0/4265771772 >> 172.21.15.37:0/4265771772 conn(0x7f5cdc099a70 msgr2=0x7f5cdc0ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 -- 172.21.15.37:0/4265771772 shutdown_connections 2024-04-25T08:58:35.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.424+0000 7f5ce301c640 1 -- 172.21.15.37:0/4265771772 wait complete. 2024-04-25T08:58:35.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:58:35.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9cb0e1640 1 Processor -- start 2024-04-25T08:58:35.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9cb0e1640 1 -- start start 2024-04-25T08:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9cb0e1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe9c415b6b0 0x7fe9c40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9cb0e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c40cf590 0x7fe9c40cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9cb0e1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c40cbd20 con 0x7fe9c415f130 2024-04-25T08:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9cb0e1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c415ba90 con 0x7fe9c415b6b0 2024-04-25T08:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9cb0e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40cff30 con 0x7fe9c40cf590 2024-04-25T08:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9c8e7d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe9c415b6b0 0x7fe9c40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9c8e7d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe9c415b6b0 0x7fe9c40ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49730/0 (socket says 172.21.15.37:49730) 2024-04-25T08:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9c8e7d640 1 -- 172.21.15.37:0/2816676077 learned_addr learned my addr 172.21.15.37:0/2816676077 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:35.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9c3fff640 1 --2- 172.21.15.37:0/2816676077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c40cf590 0x7fe9c40cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9c8e7d640 1 -- 172.21.15.37:0/2816676077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c40cf590 msgr2=0x7fe9c40cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:35.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9c8e7d640 1 --2- 172.21.15.37:0/2816676077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c40cf590 0x7fe9c40cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9c8e7d640 1 -- 172.21.15.37:0/2816676077 >> v1:172.21.15.203:6789/0 conn(0x7fe9c415f130 legacy=0x7fe9c415f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.576+0000 7fe9c8e7d640 1 -- 172.21.15.37:0/2816676077 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c40cc000 con 0x7fe9c415b6b0 2024-04-25T08:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c8e7d640 1 --2- 172.21.15.37:0/2816676077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c415b6b0 0x7fe9c40ca3c0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009870 tx=0x7fe9b401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3487d7eeb2f08cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c37fe640 1 -- 172.21.15.37:0/2816676077 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40a95f0 con 0x7fe9c415b6b0 2024-04-25T08:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c37fe640 1 -- 172.21.15.37:0/2816676077 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b40a9790 con 0x7fe9c415b6b0 2024-04-25T08:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c37fe640 1 -- 172.21.15.37:0/2816676077 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40a9b60 con 0x7fe9c415b6b0 2024-04-25T08:58:35.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/2816676077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c415b6b0 msgr2=0x7fe9c40ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 --2- 172.21.15.37:0/2816676077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c415b6b0 0x7fe9c40ca3c0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4009870 tx=0x7fe9b401c530 comp rx=0 tx=0).stop 2024-04-25T08:58:35.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/2816676077 shutdown_connections 2024-04-25T08:58:35.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 --2- 172.21.15.37:0/2816676077 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c40cf590 0x7fe9c40cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 --2- 172.21.15.37:0/2816676077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c415b6b0 0x7fe9c40ca3c0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/2816676077 >> 172.21.15.37:0/2816676077 conn(0x7fe9c406ae60 msgr2=0x7fe9c406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/2816676077 shutdown_connections 2024-04-25T08:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/2816676077 wait complete. 2024-04-25T08:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 Processor -- start 2024-04-25T08:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 -- start start 2024-04-25T08:58:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c40cf590 0x7fe9c4154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c415b6b0 0x7fe9c415e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c415f130 0x7fe9c4152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40ca900 con 0x7fe9c415f130 2024-04-25T08:58:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40cba30 con 0x7fe9c40cf590 2024-04-25T08:58:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9cb0e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c40cadb0 con 0x7fe9c415b6b0 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c967e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c415f130 0x7fe9c4152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c967e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c415f130 0x7fe9c4152890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49740/0 (socket says 172.21.15.37:49740) 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c967e640 1 -- 172.21.15.37:0/840503553 learned_addr learned my addr 172.21.15.37:0/840503553 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c8e7d640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c40cf590 0x7fe9c4154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c3fff640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c415b6b0 0x7fe9c415e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c967e640 1 -- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c415b6b0 msgr2=0x7fe9c415e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c967e640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c415b6b0 0x7fe9c415e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c967e640 1 -- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c40cf590 msgr2=0x7fe9c4154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c967e640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c40cf590 0x7fe9c4154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.580+0000 7fe9c967e640 1 -- 172.21.15.37:0/840503553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c4153030 con 0x7fe9c415f130 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9c967e640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c415f130 0x7fe9c4152890 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00c760 tx=0x7fe9bc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9c17fa640 1 -- 172.21.15.37:0/840503553 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc003b80 con 0x7fe9c415f130 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9c17fa640 1 -- 172.21.15.37:0/840503553 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc003d20 con 0x7fe9c415f130 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9c17fa640 1 -- 172.21.15.37:0/840503553 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00aa30 con 0x7fe9c415f130 2024-04-25T08:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/840503553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c4153310 con 0x7fe9c415f130 2024-04-25T08:58:35.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/840503553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c41538d0 con 0x7fe9c415f130 2024-04-25T08:58:35.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/840503553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c404efa0 con 0x7fe9c415f130 2024-04-25T08:58:35.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9c17fa640 1 -- 172.21.15.37:0/840503553 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe9bc00abd0 con 0x7fe9c415f130 2024-04-25T08:58:35.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9c17fa640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9c17fa640 1 -- 172.21.15.37:0/840503553 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe9bc084f80 con 0x7fe9c415f130 2024-04-25T08:58:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9c8e7d640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.584+0000 7fe9c8e7d640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9b0066f60 0x7fe9b0069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40ae000 tx=0x7fe9b40b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.588+0000 7fe9c17fa640 1 -- 172.21.15.37:0/840503553 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe9bc055750 con 0x7fe9c415f130 2024-04-25T08:58:35.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.748+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/840503553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe9c406b450 con 0x7fe9c415f130 2024-04-25T08:58:35.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.748+0000 7fe9c17fa640 1 -- 172.21.15.37:0/840503553 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9bc0597f0 con 0x7fe9c415f130 2024-04-25T08:58:35.741 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:58:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9b0066f60 msgr2=0x7fe9b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9b0066f60 0x7fe9b0069420 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40ae000 tx=0x7fe9b40b3040 comp rx=0 tx=0).stop 2024-04-25T08:58:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/840503553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c415f130 msgr2=0x7fe9c4152890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c415f130 0x7fe9c4152890 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc00c760 tx=0x7fe9bc00cc20 comp rx=0 tx=0).stop 2024-04-25T08:58:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/840503553 shutdown_connections 2024-04-25T08:58:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe9b0066f60 0x7fe9b0069420 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9c415f130 0x7fe9c4152890 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe9c415b6b0 0x7fe9c415e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 --2- 172.21.15.37:0/840503553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe9c40cf590 0x7fe9c4154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/840503553 >> 172.21.15.37:0/840503553 conn(0x7fe9c406ae60 msgr2=0x7fe9c40af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/840503553 shutdown_connections 2024-04-25T08:58:35.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.752+0000 7fe9cb0e1640 1 -- 172.21.15.37:0/840503553 wait complete. 2024-04-25T08:58:35.760 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:58:35.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a1b7c8640 1 Processor -- start 2024-04-25T08:58:35.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a1b7c8640 1 -- start start 2024-04-25T08:58:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a1b7c8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5a1415da80 0x7f5a140d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a1b7c8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5a140d07f0 0x7f5a1406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a1b7c8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1406be60 con 0x7f5a140d1cd0 2024-04-25T08:58:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a1b7c8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1406c440 con 0x7f5a1415da80 2024-04-25T08:58:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a1b7c8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1415de60 con 0x7f5a140d07f0 2024-04-25T08:58:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a18d63640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5a140d1cd0 0x7f5a140d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:54482/0 (socket says 172.21.15.37:54482) 2024-04-25T08:58:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a18d63640 1 -- 172.21.15.37:0/32987127 learned_addr learned my addr 172.21.15.37:0/32987127 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a19564640 1 --2- 172.21.15.37:0/32987127 >> v2:172.21.15.37:3300/0 conn(0x7f5a1415da80 0x7f5a140d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a19d65640 1 --2- 172.21.15.37:0/32987127 >> v2:172.21.15.203:3300/0 conn(0x7f5a140d07f0 0x7f5a1406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a13fff640 1 -- 172.21.15.37:0/32987127 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2647519479 0 0) 0x7f5a1406be60 con 0x7f5a140d1cd0 2024-04-25T08:58:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a13fff640 1 -- 172.21.15.37:0/32987127 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59fc003530 con 0x7f5a140d1cd0 2024-04-25T08:58:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a19564640 1 -- 172.21.15.37:0/32987127 >> v2:172.21.15.203:3300/0 conn(0x7f5a140d07f0 msgr2=0x7f5a1406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a19564640 1 --2- 172.21.15.37:0/32987127 >> v2:172.21.15.203:3300/0 conn(0x7f5a140d07f0 0x7f5a1406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a19564640 1 -- 172.21.15.37:0/32987127 >> v1:172.21.15.37:6789/0 conn(0x7f5a140d1cd0 legacy=0x7f5a140d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.888+0000 7f5a19564640 1 -- 172.21.15.37:0/32987127 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a140d2450 con 0x7f5a1415da80 2024-04-25T08:58:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a19564640 1 --2- 172.21.15.37:0/32987127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a1415da80 0x7f5a140d4230 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009d90 tx=0x7f5a0c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f9597b5cde79d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a13fff640 1 -- 172.21.15.37:0/32987127 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0b2070 con 0x7f5a1415da80 2024-04-25T08:58:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a13fff640 1 -- 172.21.15.37:0/32987127 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c0a5920 con 0x7f5a1415da80 2024-04-25T08:58:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a13fff640 1 -- 172.21.15.37:0/32987127 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c0a5cf0 con 0x7f5a1415da80 2024-04-25T08:58:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/32987127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a1415da80 msgr2=0x7f5a140d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a1b7c8640 1 --2- 172.21.15.37:0/32987127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a1415da80 0x7f5a140d4230 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009d90 tx=0x7f5a0c01c900 comp rx=0 tx=0).stop 2024-04-25T08:58:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/32987127 shutdown_connections 2024-04-25T08:58:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a1b7c8640 1 --2- 172.21.15.37:0/32987127 >> v2:172.21.15.203:3300/0 conn(0x7f5a140d07f0 0x7f5a1406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a1b7c8640 1 --2- 172.21.15.37:0/32987127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a1415da80 0x7f5a140d4230 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/32987127 >> 172.21.15.37:0/32987127 conn(0x7f5a1408b000 msgr2=0x7f5a140d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/32987127 shutdown_connections 2024-04-25T08:58:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.892+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/32987127 wait complete. 2024-04-25T08:58:35.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a1b7c8640 1 Processor -- start 2024-04-25T08:58:35.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a1b7c8640 1 -- start start 2024-04-25T08:58:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a1b7c8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a140d07f0 0x7f5a14159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a1b7c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a140d1cd0 0x7f5a140cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a1b7c8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a1415da80 0x7f5a140cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a1b7c8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1406d7a0 con 0x7f5a140d1cd0 2024-04-25T08:58:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a1b7c8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1406d620 con 0x7f5a1415da80 2024-04-25T08:58:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a1b7c8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1406d920 con 0x7f5a140d07f0 2024-04-25T08:58:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a18d63640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a140d1cd0 0x7f5a140cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a18d63640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a140d1cd0 0x7f5a140cb3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49748/0 (socket says 172.21.15.37:49748) 2024-04-25T08:58:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a18d63640 1 -- 172.21.15.37:0/4085059998 learned_addr learned my addr 172.21.15.37:0/4085059998 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a19564640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a140d07f0 0x7f5a14159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a19d65640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a1415da80 0x7f5a140cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a18d63640 1 -- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a140d07f0 msgr2=0x7f5a14159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a18d63640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a140d07f0 0x7f5a14159c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a18d63640 1 -- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a1415da80 msgr2=0x7f5a140cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a18d63640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a1415da80 0x7f5a140cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a18d63640 1 -- 172.21.15.37:0/4085059998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a140cbe10 con 0x7f5a140d1cd0 2024-04-25T08:58:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a19564640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a140d07f0 0x7f5a14159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a18d63640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a140d1cd0 0x7f5a140cb3c0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c730 tx=0x7f5a0800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a11ffb640 1 -- 172.21.15.37:0/4085059998 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08017070 con 0x7f5a140d1cd0 2024-04-25T08:58:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a11ffb640 1 -- 172.21.15.37:0/4085059998 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a08007270 con 0x7f5a140d1cd0 2024-04-25T08:58:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/4085059998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a140cc0f0 con 0x7f5a140d1cd0 2024-04-25T08:58:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a11ffb640 1 -- 172.21.15.37:0/4085059998 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08001d50 con 0x7f5a140d1cd0 2024-04-25T08:58:35.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/4085059998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a140cc6b0 con 0x7f5a140d1cd0 2024-04-25T08:58:35.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.896+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/4085059998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1404efa0 con 0x7f5a140d1cd0 2024-04-25T08:58:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.900+0000 7f5a11ffb640 1 -- 172.21.15.37:0/4085059998 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5a0801a070 con 0x7f5a140d1cd0 2024-04-25T08:58:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.900+0000 7f5a11ffb640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a00066f60 0x7f5a00069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.900+0000 7f5a11ffb640 1 -- 172.21.15.37:0/4085059998 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5a08084a80 con 0x7f5a140d1cd0 2024-04-25T08:58:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.900+0000 7f5a19564640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a00066f60 0x7f5a00069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.900+0000 7f5a19564640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a00066f60 0x7f5a00069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0af000 tx=0x7f5a0c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:35.904+0000 7f5a11ffb640 1 -- 172.21.15.37:0/4085059998 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5a08021020 con 0x7f5a140d1cd0 2024-04-25T08:58:36.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/4085059998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5a140d1750 con 0x7f5a140d1cd0 2024-04-25T08:58:36.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a11ffb640 1 -- 172.21.15.37:0/4085059998 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5a08055890 con 0x7f5a140d1cd0 2024-04-25T08:58:36.055 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:58:36.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a00066f60 msgr2=0x7f5a00069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a00066f60 0x7f5a00069420 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c0af000 tx=0x7f5a0c0a7070 comp rx=0 tx=0).stop 2024-04-25T08:58:36.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/4085059998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a140d1cd0 msgr2=0x7f5a140cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a140d1cd0 0x7f5a140cb3c0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f5a0800c730 tx=0x7f5a0800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/4085059998 shutdown_connections 2024-04-25T08:58:36.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5a00066f60 0x7f5a00069420 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5a1415da80 0x7f5a140cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a140d1cd0 0x7f5a140cb3c0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 --2- 172.21.15.37:0/4085059998 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5a140d07f0 0x7f5a14159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/4085059998 >> 172.21.15.37:0/4085059998 conn(0x7f5a1408b000 msgr2=0x7f5a140b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:36.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/4085059998 shutdown_connections 2024-04-25T08:58:36.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.064+0000 7f5a1b7c8640 1 -- 172.21.15.37:0/4085059998 wait complete. 2024-04-25T08:58:36.075 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:58:36.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d350b4640 1 Processor -- start 2024-04-25T08:58:36.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d350b4640 1 -- start start 2024-04-25T08:58:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d350b4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5d300ce510 0x7f5d300ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d350b4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d30144ca0 0x7f5d3006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d350b4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d300cee30 con 0x7f5d3006ec60 2024-04-25T08:58:36.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d350b4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d300cc2b0 con 0x7f5d300ce510 2024-04-25T08:58:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d350b4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3006e290 con 0x7f5d30144ca0 2024-04-25T08:58:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d2ed9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5d300ce510 0x7f5d300ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d2ed9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5d300ce510 0x7f5d300ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49756/0 (socket says 172.21.15.37:49756) 2024-04-25T08:58:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d2ed9d640 1 -- 172.21.15.37:0/2487651657 learned_addr learned my addr 172.21.15.37:0/2487651657 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d2e59c640 1 --2- 172.21.15.37:0/2487651657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d30144ca0 0x7f5d3006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d2ed9d640 1 -- 172.21.15.37:0/2487651657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d30144ca0 msgr2=0x7f5d3006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d2ed9d640 1 --2- 172.21.15.37:0/2487651657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d30144ca0 0x7f5d3006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d2ed9d640 1 -- 172.21.15.37:0/2487651657 >> v1:172.21.15.203:6789/0 conn(0x7f5d3006ec60 legacy=0x7f5d30142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d2ed9d640 1 -- 172.21.15.37:0/2487651657 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d300ccc70 con 0x7f5d300ce510 2024-04-25T08:58:36.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.212+0000 7f5d2e59c640 1 --2- 172.21.15.37:0/2487651657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d30144ca0 0x7f5d3006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d2ed9d640 1 --2- 172.21.15.37:0/2487651657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d300ce510 0x7f5d300ce8f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009e50 tx=0x7f5d2001f210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e9832f33a45415 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d2dd9b640 1 -- 172.21.15.37:0/2487651657 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d200b4070 con 0x7f5d300ce510 2024-04-25T08:58:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d2dd9b640 1 -- 172.21.15.37:0/2487651657 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d200af400 con 0x7f5d300ce510 2024-04-25T08:58:36.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d2dd9b640 1 -- 172.21.15.37:0/2487651657 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d200af7d0 con 0x7f5d300ce510 2024-04-25T08:58:36.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d350b4640 1 -- 172.21.15.37:0/2487651657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d300ce510 msgr2=0x7f5d300ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d350b4640 1 --2- 172.21.15.37:0/2487651657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d300ce510 0x7f5d300ce8f0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f5d20009e50 tx=0x7f5d2001f210 comp rx=0 tx=0).stop 2024-04-25T08:58:36.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d350b4640 1 -- 172.21.15.37:0/2487651657 shutdown_connections 2024-04-25T08:58:36.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d350b4640 1 --2- 172.21.15.37:0/2487651657 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d30144ca0 0x7f5d3006e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d350b4640 1 --2- 172.21.15.37:0/2487651657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d300ce510 0x7f5d300ce8f0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d350b4640 1 -- 172.21.15.37:0/2487651657 >> 172.21.15.37:0/2487651657 conn(0x7f5d300b11a0 msgr2=0x7f5d300b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:36.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d350b4640 1 -- 172.21.15.37:0/2487651657 shutdown_connections 2024-04-25T08:58:36.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.216+0000 7f5d350b4640 1 -- 172.21.15.37:0/2487651657 wait complete. 2024-04-25T08:58:36.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d350b4640 1 Processor -- start 2024-04-25T08:58:36.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d350b4640 1 -- start start 2024-04-25T08:58:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d350b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3006ec60 0x7f5d3013ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d350b4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d300ce510 0x7f5d300caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d350b4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d30144ca0 0x7f5d300caff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d350b4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d300cc900 con 0x7f5d3006ec60 2024-04-25T08:58:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d350b4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3006df20 con 0x7f5d300ce510 2024-04-25T08:58:36.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d350b4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d30142d90 con 0x7f5d30144ca0 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3006ec60 0x7f5d3013ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3006ec60 0x7f5d3013ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49766/0 (socket says 172.21.15.37:49766) 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2ed9d640 1 -- 172.21.15.37:0/612639819 learned_addr learned my addr 172.21.15.37:0/612639819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2f59e640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d30144ca0 0x7f5d300caff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2e59c640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d300ce510 0x7f5d300caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2ed9d640 1 -- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d30144ca0 msgr2=0x7f5d300caff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2ed9d640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d30144ca0 0x7f5d300caff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2ed9d640 1 -- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d300ce510 msgr2=0x7f5d300caab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2ed9d640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d300ce510 0x7f5d300caab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2ed9d640 1 -- 172.21.15.37:0/612639819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d301e0d80 con 0x7f5d3006ec60 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2e59c640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d300ce510 0x7f5d300caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d2ed9d640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3006ec60 0x7f5d3013ce70 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f5d200b2040 tx=0x7f5d2001bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d177fe640 1 -- 172.21.15.37:0/612639819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d200b4040 con 0x7f5d3006ec60 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d177fe640 1 -- 172.21.15.37:0/612639819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d200afe10 con 0x7f5d3006ec60 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d177fe640 1 -- 172.21.15.37:0/612639819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20009510 con 0x7f5d3006ec60 2024-04-25T08:58:36.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d350b4640 1 -- 172.21.15.37:0/612639819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d301e1020 con 0x7f5d3006ec60 2024-04-25T08:58:36.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d350b4640 1 -- 172.21.15.37:0/612639819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d301e15e0 con 0x7f5d3006ec60 2024-04-25T08:58:36.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d350b4640 1 -- 172.21.15.37:0/612639819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3004efa0 con 0x7f5d3006ec60 2024-04-25T08:58:36.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.220+0000 7f5d177fe640 1 -- 172.21.15.37:0/612639819 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d200c2020 con 0x7f5d3006ec60 2024-04-25T08:58:36.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.224+0000 7f5d177fe640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d1c066f60 0x7f5d1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.224+0000 7f5d177fe640 1 -- 172.21.15.37:0/612639819 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5d20122e80 con 0x7f5d3006ec60 2024-04-25T08:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.224+0000 7f5d2e59c640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d1c066f60 0x7f5d1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.224+0000 7f5d2e59c640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d1c066f60 0x7f5d1c069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5d300cbcd0 tx=0x7f5d24008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:36.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.228+0000 7f5d177fe640 1 -- 172.21.15.37:0/612639819 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d20018d70 con 0x7f5d3006ec60 2024-04-25T08:58:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.400+0000 7f5d350b4640 1 -- 172.21.15.37:0/612639819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5d300b1cc0 con 0x7f5d3006ec60 2024-04-25T08:58:36.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.400+0000 7f5d177fe640 1 -- 172.21.15.37:0/612639819 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v83) v1 ==== 80+0+122 (secure 0 0 0) 0x7f5d200f3d70 con 0x7f5d3006ec60 2024-04-25T08:58:36.393 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/4130042812 2024-04-26T08:57:11.180643+0000 2024-04-25T08:58:36.393 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6834/4130042812 2024-04-26T08:57:11.180643+0000 2024-04-25T08:58:36.393 INFO:teuthology.orchestra.run.smithi037.stderr:listed 2 entries 2024-04-25T08:58:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 -- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d1c066f60 msgr2=0x7f5d1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d1c066f60 0x7f5d1c069420 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f5d300cbcd0 tx=0x7f5d24008040 comp rx=0 tx=0).stop 2024-04-25T08:58:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 -- 172.21.15.37:0/612639819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3006ec60 msgr2=0x7f5d3013ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3006ec60 0x7f5d3013ce70 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f5d200b2040 tx=0x7f5d2001bfb0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 -- 172.21.15.37:0/612639819 shutdown_connections 2024-04-25T08:58:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d1c066f60 0x7f5d1c069420 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d30144ca0 0x7f5d300caff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d300ce510 0x7f5d300caab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 --2- 172.21.15.37:0/612639819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d3006ec60 0x7f5d3013ce70 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 -- 172.21.15.37:0/612639819 >> 172.21.15.37:0/612639819 conn(0x7f5d300b11a0 msgr2=0x7f5d300718c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 -- 172.21.15.37:0/612639819 shutdown_connections 2024-04-25T08:58:36.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.404+0000 7f5d350b4640 1 -- 172.21.15.37:0/612639819 wait complete. 2024-04-25T08:58:36.412 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T08:58:36.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:36.412 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:36.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[46538]: fuse finished with error 0 and tester_r 0 2024-04-25T08:58:36.427 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:58:36.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:36.427 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:58:36.498 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:58:36.498 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:36.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:58:36.505 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:58:36.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:36.505 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:58:36.554 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:58:36.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:36.555 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:58:36.556 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:36.562 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:58:36.562 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:58:36.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:36.563 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:58:36.611 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:58:36.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:36.613 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:58:36.613 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:36.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:58:36.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T08:58:36.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:36.621 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:58:36.669 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:58:36.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:36.671 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:36.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.812+0000 7f9248b53640 1 Processor -- start 2024-04-25T08:58:36.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.812+0000 7f9248b53640 1 -- start start 2024-04-25T08:58:36.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.812+0000 7f9248b53640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f92440ce510 0x7f92440ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.812+0000 7f9248b53640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f92441451f0 0x7f9244142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.812+0000 7f9248b53640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92440cee30 con 0x7f9244144ca0 2024-04-25T08:58:36.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f9248b53640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92440cc2b0 con 0x7f92440ce510 2024-04-25T08:58:36.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f9248b53640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924406e290 con 0x7f92441451f0 2024-04-25T08:58:36.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f924259c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f92440ce510 0x7f92440ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f9241d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9244144ca0 0x7f924406e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:54496/0 (socket says 172.21.15.37:54496) 2024-04-25T08:58:36.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f9241d9b640 1 -- 172.21.15.37:0/758003115 learned_addr learned my addr 172.21.15.37:0/758003115 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:36.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f9242d9d640 1 --2- 172.21.15.37:0/758003115 >> v2:172.21.15.203:3300/0 conn(0x7f92441451f0 0x7f9244142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f924159a640 1 -- 172.21.15.37:0/758003115 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1024488407 0 0) 0x7f92440cee30 con 0x7f9244144ca0 2024-04-25T08:58:36.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f924259c640 1 -- 172.21.15.37:0/758003115 >> v2:172.21.15.203:3300/0 conn(0x7f92441451f0 msgr2=0x7f9244142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:36.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f924259c640 1 --2- 172.21.15.37:0/758003115 >> v2:172.21.15.203:3300/0 conn(0x7f92441451f0 0x7f9244142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f924259c640 1 -- 172.21.15.37:0/758003115 >> v1:172.21.15.37:6789/0 conn(0x7f9244144ca0 legacy=0x7f924406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f924259c640 1 -- 172.21.15.37:0/758003115 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92440ccc70 con 0x7f92440ce510 2024-04-25T08:58:36.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f9242d9d640 1 --2- 172.21.15.37:0/758003115 >> v2:172.21.15.203:3300/0 conn(0x7f92441451f0 0x7f9244142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:36.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f924259c640 1 --2- 172.21.15.37:0/758003115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92440ce510 0x7f92440ce8f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f92440b48f0 tx=0x7f923401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c357a888d297e9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:36.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f924159a640 1 -- 172.21.15.37:0/758003115 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92340b5070 con 0x7f92440ce510 2024-04-25T08:58:36.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f924159a640 1 -- 172.21.15.37:0/758003115 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92340a8970 con 0x7f92440ce510 2024-04-25T08:58:36.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.816+0000 7f924159a640 1 -- 172.21.15.37:0/758003115 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92340a8d40 con 0x7f92440ce510 2024-04-25T08:58:36.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 -- 172.21.15.37:0/758003115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92440ce510 msgr2=0x7f92440ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 --2- 172.21.15.37:0/758003115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92440ce510 0x7f92440ce8f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f92440b48f0 tx=0x7f923401ec70 comp rx=0 tx=0).stop 2024-04-25T08:58:36.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 -- 172.21.15.37:0/758003115 shutdown_connections 2024-04-25T08:58:36.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 --2- 172.21.15.37:0/758003115 >> v2:172.21.15.203:3300/0 conn(0x7f92441451f0 0x7f9244142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 --2- 172.21.15.37:0/758003115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92440ce510 0x7f92440ce8f0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 -- 172.21.15.37:0/758003115 >> 172.21.15.37:0/758003115 conn(0x7f92440b11a0 msgr2=0x7f92440b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:36.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 -- 172.21.15.37:0/758003115 shutdown_connections 2024-04-25T08:58:36.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 -- 172.21.15.37:0/758003115 wait complete. 2024-04-25T08:58:36.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 Processor -- start 2024-04-25T08:58:36.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 -- start start 2024-04-25T08:58:36.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92440ce510 0x7f92441f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9244144ca0 0x7f924415afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92441451f0 0x7f92440c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924406ee30 con 0x7f92441451f0 2024-04-25T08:58:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924406df20 con 0x7f9244144ca0 2024-04-25T08:58:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92440cc900 con 0x7f92440ce510 2024-04-25T08:58:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9242d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92441451f0 0x7f92440c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9242d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92441451f0 0x7f92440c9360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49786/0 (socket says 172.21.15.37:49786) 2024-04-25T08:58:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9242d9d640 1 -- 172.21.15.37:0/2911857516 learned_addr learned my addr 172.21.15.37:0/2911857516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9241d9b640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9244144ca0 0x7f924415afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f924259c640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92440ce510 0x7f92441f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9242d9d640 1 -- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92440ce510 msgr2=0x7f92441f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9242d9d640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92440ce510 0x7f92441f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9242d9d640 1 -- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9244144ca0 msgr2=0x7f924415afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9242d9d640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9244144ca0 0x7f924415afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9242d9d640 1 -- 172.21.15.37:0/2911857516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924415b510 con 0x7f92441451f0 2024-04-25T08:58:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9241d9b640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9244144ca0 0x7f924415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9242d9d640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92441451f0 0x7f92440c9360 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f923c007e60 tx=0x7f923c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9232ffd640 1 -- 172.21.15.37:0/2911857516 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c01a070 con 0x7f92441451f0 2024-04-25T08:58:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9232ffd640 1 -- 172.21.15.37:0/2911857516 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c015600 con 0x7f92441451f0 2024-04-25T08:58:36.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 -- 172.21.15.37:0/2911857516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9244131ba0 con 0x7f92441451f0 2024-04-25T08:58:36.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9232ffd640 1 -- 172.21.15.37:0/2911857516 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c0159f0 con 0x7f92441451f0 2024-04-25T08:58:36.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.820+0000 7f9248b53640 1 -- 172.21.15.37:0/2911857516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9244132160 con 0x7f92441451f0 2024-04-25T08:58:36.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.824+0000 7f9248b53640 1 -- 172.21.15.37:0/2911857516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924404efa0 con 0x7f92441451f0 2024-04-25T08:58:36.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.824+0000 7f9232ffd640 1 -- 172.21.15.37:0/2911857516 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f923c01d070 con 0x7f92441451f0 2024-04-25T08:58:36.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.824+0000 7f9232ffd640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9224066f90 0x7f9224069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.824+0000 7f9232ffd640 1 -- 172.21.15.37:0/2911857516 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f923c0854e0 con 0x7f92441451f0 2024-04-25T08:58:36.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.824+0000 7f924259c640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9224066f90 0x7f9224069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.824+0000 7f924259c640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9224066f90 0x7f9224069450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f92340b2000 tx=0x7f92340aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:36.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:36.828+0000 7f9232ffd640 1 -- 172.21.15.37:0/2911857516 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f923c010070 con 0x7f92441451f0 2024-04-25T08:58:37.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.024+0000 7f9248b53640 1 -- 172.21.15.37:0/2911857516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92440b1cc0 con 0x7f92441451f0 2024-04-25T08:58:37.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.024+0000 7f9232ffd640 1 -- 172.21.15.37:0/2911857516 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f923c056270 con 0x7f92441451f0 2024-04-25T08:58:37.018 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:37.018 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:58:02.214606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":16,"addr":"172.21.15.37:6835/3927528604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3927528604},{"type":"v1","addr":"172.21.15.37:6835","nonce":3927528604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:58:37.018 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 52 2024-04-25T08:58:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 -- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9224066f90 msgr2=0x7f9224069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9224066f90 0x7f9224069450 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f92340b2000 tx=0x7f92340aa070 comp rx=0 tx=0).stop 2024-04-25T08:58:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 -- 172.21.15.37:0/2911857516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92441451f0 msgr2=0x7f92440c9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92441451f0 0x7f92440c9360 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f923c007e60 tx=0x7f923c00d4a0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 -- 172.21.15.37:0/2911857516 shutdown_connections 2024-04-25T08:58:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9224066f90 0x7f9224069450 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92441451f0 0x7f92440c9360 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9244144ca0 0x7f924415afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 --2- 172.21.15.37:0/2911857516 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92440ce510 0x7f92441f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 -- 172.21.15.37:0/2911857516 >> 172.21.15.37:0/2911857516 conn(0x7f92440b11a0 msgr2=0x7f92440c6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 -- 172.21.15.37:0/2911857516 shutdown_connections 2024-04-25T08:58:37.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.028+0000 7f9248b53640 1 -- 172.21.15.37:0/2911857516 wait complete. 2024-04-25T08:58:37.037 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:37.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.176+0000 7f1c36d61640 1 Processor -- start 2024-04-25T08:58:37.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.176+0000 7f1c36d61640 1 -- start start 2024-04-25T08:58:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c36d61640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1c3006f4a0 0x7f1c30099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c36d61640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c300b0e10 con 0x7f1c30099fb0 2024-04-25T08:58:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c36d61640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c300d0180 con 0x7f1c300b0a30 2024-04-25T08:58:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c36d61640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c300d0d90 con 0x7f1c3006f4a0 2024-04-25T08:58:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c2ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1c3006f4a0 0x7f1c30099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c352fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1c30099fb0 0x7f1c3015d230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:54508/0 (socket says 172.21.15.37:54508) 2024-04-25T08:58:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c352fe640 1 -- 172.21.15.37:0/1978538777 learned_addr learned my addr 172.21.15.37:0/1978538777 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c2ffff640 1 -- 172.21.15.37:0/1978538777 >> v1:172.21.15.203:6789/0 conn(0x7f1c300b0a30 legacy=0x7f1c300cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c2ffff640 1 -- 172.21.15.37:0/1978538777 >> v1:172.21.15.37:6789/0 conn(0x7f1c30099fb0 legacy=0x7f1c3015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c2f7fe640 1 -- 172.21.15.37:0/1978538777 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 585312675 0 0) 0x7f1c300b0e10 con 0x7f1c30099fb0 2024-04-25T08:58:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c2ffff640 1 -- 172.21.15.37:0/1978538777 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c300d0460 con 0x7f1c3006f4a0 2024-04-25T08:58:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c2ffff640 1 --2- 172.21.15.37:0/1978538777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c3006f4a0 0x7f1c30099a70 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097c0 tx=0x7f1c1c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db86209e977a3ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c2f7fe640 1 -- 172.21.15.37:0/1978538777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0b0070 con 0x7f1c3006f4a0 2024-04-25T08:58:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c2f7fe640 1 -- 172.21.15.37:0/1978538777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0ab600 con 0x7f1c3006f4a0 2024-04-25T08:58:37.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c2f7fe640 1 -- 172.21.15.37:0/1978538777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0ab9d0 con 0x7f1c3006f4a0 2024-04-25T08:58:37.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c36d61640 1 -- 172.21.15.37:0/1978538777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c3006f4a0 msgr2=0x7f1c30099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c36d61640 1 --2- 172.21.15.37:0/1978538777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c3006f4a0 0x7f1c30099a70 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097c0 tx=0x7f1c1c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c36d61640 1 -- 172.21.15.37:0/1978538777 shutdown_connections 2024-04-25T08:58:37.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c36d61640 1 --2- 172.21.15.37:0/1978538777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c3006f4a0 0x7f1c30099a70 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c36d61640 1 -- 172.21.15.37:0/1978538777 >> 172.21.15.37:0/1978538777 conn(0x7f1c300b30b0 msgr2=0x7f1c300b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:37.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c36d61640 1 -- 172.21.15.37:0/1978538777 shutdown_connections 2024-04-25T08:58:37.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.180+0000 7f1c36d61640 1 -- 172.21.15.37:0/1978538777 wait complete. 2024-04-25T08:58:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c36d61640 1 Processor -- start 2024-04-25T08:58:37.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c36d61640 1 -- start start 2024-04-25T08:58:37.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c36d61640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c3006f4a0 0x7f1c300cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c36d61640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c30099fb0 0x7f1c300c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c36d61640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c300b0a30 0x7f1c301547e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c36d61640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3015ded0 con 0x7f1c3006f4a0 2024-04-25T08:58:37.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c36d61640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3015dd50 con 0x7f1c30099fb0 2024-04-25T08:58:37.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c36d61640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3016c580 con 0x7f1c300b0a30 2024-04-25T08:58:37.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c34afd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c3006f4a0 0x7f1c300cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c34afd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c3006f4a0 0x7f1c300cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49798/0 (socket says 172.21.15.37:49798) 2024-04-25T08:58:37.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c34afd640 1 -- 172.21.15.37:0/1056513619 learned_addr learned my addr 172.21.15.37:0/1056513619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c352fe640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c300b0a30 0x7f1c301547e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c2ffff640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c30099fb0 0x7f1c300c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c34afd640 1 -- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c300b0a30 msgr2=0x7f1c301547e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c34afd640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c300b0a30 0x7f1c301547e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c34afd640 1 -- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c30099fb0 msgr2=0x7f1c300c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c34afd640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c30099fb0 0x7f1c300c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c34afd640 1 -- 172.21.15.37:0/1056513619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c300cb4f0 con 0x7f1c3006f4a0 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c2ffff640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c30099fb0 0x7f1c300c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c34afd640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c3006f4a0 0x7f1c300cafb0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400c3c0 tx=0x7f1c2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c2d7fa640 1 -- 172.21.15.37:0/1056513619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24017070 con 0x7f1c3006f4a0 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c2d7fa640 1 -- 172.21.15.37:0/1056513619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c240052d0 con 0x7f1c3006f4a0 2024-04-25T08:58:37.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c36d61640 1 -- 172.21.15.37:0/1056513619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c30153460 con 0x7f1c3006f4a0 2024-04-25T08:58:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c2d7fa640 1 -- 172.21.15.37:0/1056513619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c24003840 con 0x7f1c3006f4a0 2024-04-25T08:58:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.184+0000 7f1c36d61640 1 -- 172.21.15.37:0/1056513619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c30153d00 con 0x7f1c3006f4a0 2024-04-25T08:58:37.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.188+0000 7f1c36d61640 1 -- 172.21.15.37:0/1056513619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3004efa0 con 0x7f1c3006f4a0 2024-04-25T08:58:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.188+0000 7f1c2d7fa640 1 -- 172.21.15.37:0/1056513619 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c2401a070 con 0x7f1c3006f4a0 2024-04-25T08:58:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.188+0000 7f1c2d7fa640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c14066f60 0x7f1c14069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.188+0000 7f1c2d7fa640 1 -- 172.21.15.37:0/1056513619 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1c24084530 con 0x7f1c3006f4a0 2024-04-25T08:58:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.188+0000 7f1c2ffff640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c14066f60 0x7f1c14069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.188+0000 7f1c2ffff640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c14066f60 0x7f1c14069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0ad000 tx=0x7f1c1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.192+0000 7f1c2d7fa640 1 -- 172.21.15.37:0/1056513619 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c24005480 con 0x7f1c3006f4a0 2024-04-25T08:58:37.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.352+0000 7f1c36d61640 1 -- 172.21.15.37:0/1056513619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c30156160 con 0x7f1c3006f4a0 2024-04-25T08:58:37.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c2d7fa640 1 -- 172.21.15.37:0/1056513619 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17038 (secure 0 0 0) 0x7f1c24055360 con 0x7f1c3006f4a0 2024-04-25T08:58:37.347 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:37.347 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":83,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:57:11.180680+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:56:49.737591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:56:50.436964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/4130042812":"2024-04-26T08:57:11.180643+0000","172.21.15.37:6834/4130042812":"2024-04-26T08:57:11.180643+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 -- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c14066f60 msgr2=0x7f1c14069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c14066f60 0x7f1c14069420 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0ad000 tx=0x7f1c1c008040 comp rx=0 tx=0).stop 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 -- 172.21.15.37:0/1056513619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c3006f4a0 msgr2=0x7f1c300cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c3006f4a0 0x7f1c300cafb0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f1c2400c3c0 tx=0x7f1c2400c880 comp rx=0 tx=0).stop 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 -- 172.21.15.37:0/1056513619 shutdown_connections 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c14066f60 0x7f1c14069420 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c300b0a30 0x7f1c301547e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c30099fb0 0x7f1c300c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 --2- 172.21.15.37:0/1056513619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c3006f4a0 0x7f1c300cafb0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 -- 172.21.15.37:0/1056513619 >> 172.21.15.37:0/1056513619 conn(0x7f1c300b30b0 msgr2=0x7f1c3006fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 -- 172.21.15.37:0/1056513619 shutdown_connections 2024-04-25T08:58:37.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.356+0000 7f1c36d61640 1 -- 172.21.15.37:0/1056513619 wait complete. 2024-04-25T08:58:37.367 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:56:49.737591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:56:50.436964+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:58:37.368 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:58:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f5721bc8640 1 Processor -- start 2024-04-25T08:58:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f5721bc8640 1 -- start start 2024-04-25T08:58:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f5721bc8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f571c15b6b0 0x7f571c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f5721bc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f571c0cf590 0x7f571c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f5721bc8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571c0cbd20 con 0x7f571c15f130 2024-04-25T08:58:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f5721bc8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571c15ba90 con 0x7f571c15b6b0 2024-04-25T08:58:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f5721bc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0cff30 con 0x7f571c0cf590 2024-04-25T08:58:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f571b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f571c15b6b0 0x7f571c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f571b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f571c15b6b0 0x7f571c0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49804/0 (socket says 172.21.15.37:49804) 2024-04-25T08:58:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f571b7fe640 1 -- 172.21.15.37:0/3927383176 learned_addr learned my addr 172.21.15.37:0/3927383176 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f571affd640 1 --2- 172.21.15.37:0/3927383176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f571c0cf590 0x7f571c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f571b7fe640 1 -- 172.21.15.37:0/3927383176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f571c0cf590 msgr2=0x7f571c0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f571b7fe640 1 --2- 172.21.15.37:0/3927383176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f571c0cf590 0x7f571c0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f571b7fe640 1 -- 172.21.15.37:0/3927383176 >> v1:172.21.15.203:6789/0 conn(0x7f571c15f130 legacy=0x7f571c15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:37.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.508+0000 7f571b7fe640 1 -- 172.21.15.37:0/3927383176 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c0cc000 con 0x7f571c15b6b0 2024-04-25T08:58:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571b7fe640 1 --2- 172.21.15.37:0/3927383176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571c15b6b0 0x7f571c0ca3c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5714009e50 tx=0x7f571401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f418e6474d0e7da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571a7fc640 1 -- 172.21.15.37:0/3927383176 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57140a58d0 con 0x7f571c15b6b0 2024-04-25T08:58:37.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571a7fc640 1 -- 172.21.15.37:0/3927383176 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57140a5a70 con 0x7f571c15b6b0 2024-04-25T08:58:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571a7fc640 1 -- 172.21.15.37:0/3927383176 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57140a5e40 con 0x7f571c15b6b0 2024-04-25T08:58:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 -- 172.21.15.37:0/3927383176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571c15b6b0 msgr2=0x7f571c0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 --2- 172.21.15.37:0/3927383176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571c15b6b0 0x7f571c0ca3c0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f5714009e50 tx=0x7f571401ca00 comp rx=0 tx=0).stop 2024-04-25T08:58:37.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 -- 172.21.15.37:0/3927383176 shutdown_connections 2024-04-25T08:58:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 --2- 172.21.15.37:0/3927383176 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f571c0cf590 0x7f571c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 --2- 172.21.15.37:0/3927383176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571c15b6b0 0x7f571c0ca3c0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 -- 172.21.15.37:0/3927383176 >> 172.21.15.37:0/3927383176 conn(0x7f571c06ae60 msgr2=0x7f571c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 -- 172.21.15.37:0/3927383176 shutdown_connections 2024-04-25T08:58:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 -- 172.21.15.37:0/3927383176 wait complete. 2024-04-25T08:58:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 Processor -- start 2024-04-25T08:58:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 -- start start 2024-04-25T08:58:37.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571c0cf590 0x7f571c159430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f571c15b6b0 0x7f571c1575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f571c15f130 0x7f571c0c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0ca900 con 0x7f571c0cf590 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0cba30 con 0x7f571c15f130 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f5721bc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0cadb0 con 0x7f571c15b6b0 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571c0cf590 0x7f571c159430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571c0cf590 0x7f571c159430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49810/0 (socket says 172.21.15.37:49810) 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571b7fe640 1 -- 172.21.15.37:0/1708528611 learned_addr learned my addr 172.21.15.37:0/1708528611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571bfff640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f571c15f130 0x7f571c0c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571affd640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f571c15b6b0 0x7f571c1575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571b7fe640 1 -- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f571c15b6b0 msgr2=0x7f571c1575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571b7fe640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f571c15b6b0 0x7f571c1575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571b7fe640 1 -- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f571c15f130 msgr2=0x7f571c0c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571b7fe640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f571c15f130 0x7f571c0c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.512+0000 7f571b7fe640 1 -- 172.21.15.37:0/1708528611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c154790 con 0x7f571c0cf590 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f571b7fe640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571c0cf590 0x7f571c159430 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f571401a630 tx=0x7f57140197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f5713fff640 1 -- 172.21.15.37:0/1708528611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57140a58d0 con 0x7f571c0cf590 2024-04-25T08:58:37.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f5713fff640 1 -- 172.21.15.37:0/1708528611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57140be070 con 0x7f571c0cf590 2024-04-25T08:58:37.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f5721bc8640 1 -- 172.21.15.37:0/1708528611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c0c5810 con 0x7f571c0cf590 2024-04-25T08:58:37.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f5713fff640 1 -- 172.21.15.37:0/1708528611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714009510 con 0x7f571c0cf590 2024-04-25T08:58:37.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f5721bc8640 1 -- 172.21.15.37:0/1708528611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c0c2fd0 con 0x7f571c0cf590 2024-04-25T08:58:37.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f5721bc8640 1 -- 172.21.15.37:0/1708528611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c04efa0 con 0x7f571c0cf590 2024-04-25T08:58:37.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f5713fff640 1 -- 172.21.15.37:0/1708528611 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f57140b86b0 con 0x7f571c0cf590 2024-04-25T08:58:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f5713fff640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f56fc066f60 0x7f56fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f5713fff640 1 -- 172.21.15.37:0/1708528611 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f57141209d0 con 0x7f571c0cf590 2024-04-25T08:58:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f571affd640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f56fc066f60 0x7f56fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.516+0000 7f571affd640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f56fc066f60 0x7f56fc069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f571c0b0250 tx=0x7f5704008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.520+0000 7f5713fff640 1 -- 172.21.15.37:0/1708528611 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5714016620 con 0x7f571c0cf590 2024-04-25T08:58:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.680+0000 7f5721bc8640 1 -- 172.21.15.37:0/1708528611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f571c15f920 con 0x7f571c0cf590 2024-04-25T08:58:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.680+0000 7f5713fff640 1 -- 172.21.15.37:0/1708528611 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57140f1820 con 0x7f571c0cf590 2024-04-25T08:58:37.672 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:58:37.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 -- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f56fc066f60 msgr2=0x7f56fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f56fc066f60 0x7f56fc069420 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f571c0b0250 tx=0x7f5704008040 comp rx=0 tx=0).stop 2024-04-25T08:58:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 -- 172.21.15.37:0/1708528611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571c0cf590 msgr2=0x7f571c159430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571c0cf590 0x7f571c159430 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f571401a630 tx=0x7f57140197a0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 -- 172.21.15.37:0/1708528611 shutdown_connections 2024-04-25T08:58:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f56fc066f60 0x7f56fc069420 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f571c15f130 0x7f571c0c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f571c15b6b0 0x7f571c1575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 --2- 172.21.15.37:0/1708528611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f571c0cf590 0x7f571c159430 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 -- 172.21.15.37:0/1708528611 >> 172.21.15.37:0/1708528611 conn(0x7f571c06ae60 msgr2=0x7f571c0af1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 -- 172.21.15.37:0/1708528611 shutdown_connections 2024-04-25T08:58:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.684+0000 7f5721bc8640 1 -- 172.21.15.37:0/1708528611 wait complete. 2024-04-25T08:58:37.690 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:58:37.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97bdfb4640 1 Processor -- start 2024-04-25T08:58:37.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97bdfb4640 1 -- start start 2024-04-25T08:58:37.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97bdfb4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97b815b6b0 0x7f97b80ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97bdfb4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97b8073010 0x7f97b80cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97bdfb4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b80cbd20 con 0x7f97b80cf590 2024-04-25T08:58:37.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97bdfb4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b815ba90 con 0x7f97b815b6b0 2024-04-25T08:58:37.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97bdfb4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b80cc000 con 0x7f97b8073010 2024-04-25T08:58:37.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97b77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97b815b6b0 0x7f97b80ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97b77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97b815b6b0 0x7f97b80ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49818/0 (socket says 172.21.15.37:49818) 2024-04-25T08:58:37.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97b77fe640 1 -- 172.21.15.37:0/1755189502 learned_addr learned my addr 172.21.15.37:0/1755189502 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:37.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97b7fff640 1 --2- 172.21.15.37:0/1755189502 >> v2:172.21.15.203:3300/0 conn(0x7f97b8073010 0x7f97b80cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97b77fe640 1 -- 172.21.15.37:0/1755189502 >> v2:172.21.15.203:3300/0 conn(0x7f97b8073010 msgr2=0x7f97b80cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:37.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97b77fe640 1 --2- 172.21.15.37:0/1755189502 >> v2:172.21.15.203:3300/0 conn(0x7f97b8073010 0x7f97b80cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97b77fe640 1 -- 172.21.15.37:0/1755189502 >> v1:172.21.15.203:6789/0 conn(0x7f97b80cf590 legacy=0x7f97b815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:37.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.828+0000 7f97b77fe640 1 -- 172.21.15.37:0/1755189502 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b80cc2e0 con 0x7f97b815b6b0 2024-04-25T08:58:37.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97b77fe640 1 --2- 172.21.15.37:0/1755189502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b815b6b0 0x7f97b80ca3c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f97b806f000 tx=0x7f97a801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a5cd1d07e46a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97b6ffd640 1 -- 172.21.15.37:0/1755189502 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a80b0070 con 0x7f97b815b6b0 2024-04-25T08:58:37.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97b6ffd640 1 -- 172.21.15.37:0/1755189502 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a80ab600 con 0x7f97b815b6b0 2024-04-25T08:58:37.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97b6ffd640 1 -- 172.21.15.37:0/1755189502 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a80ab9d0 con 0x7f97b815b6b0 2024-04-25T08:58:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 -- 172.21.15.37:0/1755189502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b815b6b0 msgr2=0x7f97b80ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 --2- 172.21.15.37:0/1755189502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b815b6b0 0x7f97b80ca3c0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f97b806f000 tx=0x7f97a801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 -- 172.21.15.37:0/1755189502 shutdown_connections 2024-04-25T08:58:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 --2- 172.21.15.37:0/1755189502 >> v2:172.21.15.203:3300/0 conn(0x7f97b8073010 0x7f97b80cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 --2- 172.21.15.37:0/1755189502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b815b6b0 0x7f97b80ca3c0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 -- 172.21.15.37:0/1755189502 >> 172.21.15.37:0/1755189502 conn(0x7f97b806ae60 msgr2=0x7f97b806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:37.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 -- 172.21.15.37:0/1755189502 shutdown_connections 2024-04-25T08:58:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 -- 172.21.15.37:0/1755189502 wait complete. 2024-04-25T08:58:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 Processor -- start 2024-04-25T08:58:37.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 -- start start 2024-04-25T08:58:37.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97b8073010 0x7f97b80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b80cf590 0x7f97b8098560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b815b6b0 0x7f97b815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80cadb0 con 0x7f97b80cf590 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80ca900 con 0x7f97b8073010 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97bdfb4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80caf30 con 0x7f97b815b6b0 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97aeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b80cf590 0x7f97b8098560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97aeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b80cf590 0x7f97b8098560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49834/0 (socket says 172.21.15.37:49834) 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97aeffd640 1 -- 172.21.15.37:0/3518475842 learned_addr learned my addr 172.21.15.37:0/3518475842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97b7fff640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b815b6b0 0x7f97b815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97b77fe640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97b8073010 0x7f97b80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97aeffd640 1 -- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b815b6b0 msgr2=0x7f97b815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97aeffd640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b815b6b0 0x7f97b815c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97aeffd640 1 -- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97b8073010 msgr2=0x7f97b80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97aeffd640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97b8073010 0x7f97b80c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.832+0000 7f97aeffd640 1 -- 172.21.15.37:0/3518475842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8156350 con 0x7f97b80cf590 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97aeffd640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b80cf590 0x7f97b8098560 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f97a000c450 tx=0x7f97a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97b4ff9640 1 -- 172.21.15.37:0/3518475842 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0012830 con 0x7f97b80cf590 2024-04-25T08:58:37.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97b4ff9640 1 -- 172.21.15.37:0/3518475842 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a0017070 con 0x7f97b80cf590 2024-04-25T08:58:37.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97bdfb4640 1 -- 172.21.15.37:0/3518475842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b8098ec0 con 0x7f97b80cf590 2024-04-25T08:58:37.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97b4ff9640 1 -- 172.21.15.37:0/3518475842 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a0012c30 con 0x7f97b80cf590 2024-04-25T08:58:37.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97bdfb4640 1 -- 172.21.15.37:0/3518475842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b81575b0 con 0x7f97b80cf590 2024-04-25T08:58:37.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97bdfb4640 1 -- 172.21.15.37:0/3518475842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b804efa0 con 0x7f97b80cf590 2024-04-25T08:58:37.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97b4ff9640 1 -- 172.21.15.37:0/3518475842 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f97a001c460 con 0x7f97b80cf590 2024-04-25T08:58:37.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97b4ff9640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9798066f60 0x7f9798069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97b4ff9640 1 -- 172.21.15.37:0/3518475842 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f97a0084fc0 con 0x7f97b80cf590 2024-04-25T08:58:37.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97b77fe640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9798066f60 0x7f9798069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.836+0000 7f97b77fe640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9798066f60 0x7f9798069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f97a80ad000 tx=0x7f97a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.840+0000 7f97b4ff9640 1 -- 172.21.15.37:0/3518475842 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f97a0055df0 con 0x7f97b80cf590 2024-04-25T08:58:37.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:37.996+0000 7f97bdfb4640 1 -- 172.21.15.37:0/3518475842 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97b806b530 con 0x7f97b80cf590 2024-04-25T08:58:37.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97b4ff9640 1 -- 172.21.15.37:0/3518475842 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97a00556f0 con 0x7f97b80cf590 2024-04-25T08:58:37.991 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:58:37.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 -- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9798066f60 msgr2=0x7f9798069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9798066f60 0x7f9798069420 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f97a80ad000 tx=0x7f97a8008040 comp rx=0 tx=0).stop 2024-04-25T08:58:37.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 -- 172.21.15.37:0/3518475842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b80cf590 msgr2=0x7f97b8098560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b80cf590 0x7f97b8098560 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f97a000c450 tx=0x7f97a000c910 comp rx=0 tx=0).stop 2024-04-25T08:58:37.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 -- 172.21.15.37:0/3518475842 shutdown_connections 2024-04-25T08:58:37.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9798066f60 0x7f9798069420 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97b815b6b0 0x7f97b815c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97b80cf590 0x7f97b8098560 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 --2- 172.21.15.37:0/3518475842 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97b8073010 0x7f97b80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 -- 172.21.15.37:0/3518475842 >> 172.21.15.37:0/3518475842 conn(0x7f97b806ae60 msgr2=0x7f97b80b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:37.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 -- 172.21.15.37:0/3518475842 shutdown_connections 2024-04-25T08:58:37.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.000+0000 7f97bdfb4640 1 -- 172.21.15.37:0/3518475842 wait complete. 2024-04-25T08:58:38.009 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:58:38.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.144+0000 7f2bed0b3640 1 Processor -- start 2024-04-25T08:58:38.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.144+0000 7f2bed0b3640 1 -- start start 2024-04-25T08:58:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2bed0b3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be815ecb0 0x7f2be80c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2bed0b3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be80ca4c0 con 0x7f2be80c9ef0 2024-04-25T08:58:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2bed0b3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be80cd690 con 0x7f2be80cf590 2024-04-25T08:58:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2bed0b3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80ca780 con 0x7f2be815ecb0 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be759e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2be80c9ef0 0x7f2be80ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:54512/0 (socket says 172.21.15.37:54512) 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be759e640 1 -- 172.21.15.37:0/2771575129 learned_addr learned my addr 172.21.15.37:0/2771575129 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be659c640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be815ecb0 0x7f2be80c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390764110 0 0) 0x7f2be80ca4c0 con 0x7f2be80c9ef0 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bd0005740 con 0x7f2be80c9ef0 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 759182130 0 0) 0x7f2bd0005740 con 0x7f2be80c9ef0 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be815ecb0 msgr2=0x7f2be80c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be815ecb0 0x7f2be80c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 >> v1:172.21.15.203:6789/0 conn(0x7f2be80cf590 legacy=0x7f2be80cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be80cd970 con 0x7f2be80c9ef0 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f2bd8001680 con 0x7f2be80c9ef0 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 >> v1:172.21.15.37:6789/0 conn(0x7f2be80c9ef0 legacy=0x7f2be80ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bd0005c70 0x7f2bd0006090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be815ecb0 0x7f2bd0006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd0006f60 0x7f2bd00073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd000dcc0 con 0x7f2bd0006f60 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd000de70 con 0x7f2bd0005c70 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd000e070 con 0x7f2be815ecb0 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be759e640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd0006f60 0x7f2bd00073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be659c640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be815ecb0 0x7f2bd0006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be6d9d640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bd0005c70 0x7f2bd0006090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be759e640 1 -- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be815ecb0 msgr2=0x7f2bd0006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be759e640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be815ecb0 0x7f2bd0006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be759e640 1 -- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bd0005c70 msgr2=0x7f2bd0006090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be759e640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bd0005c70 0x7f2bd0006090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be759e640 1 -- 172.21.15.37:0/2771575129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd0005740 con 0x7f2bd0006f60 2024-04-25T08:58:38.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.148+0000 7f2be659c640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be815ecb0 0x7f2bd0006a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:38.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2be759e640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd0006f60 0x7f2bd00073e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c210 tx=0x7f2bd801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520a16bde08f27ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8007d20 con 0x7f2bd0006f60 2024-04-25T08:58:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd80a9070 con 0x7f2bd0006f60 2024-04-25T08:58:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2be5d9b640 1 -- 172.21.15.37:0/2771575129 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80a5de0 con 0x7f2bd0006f60 2024-04-25T08:58:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2bed0b3640 1 -- 172.21.15.37:0/2771575129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd0006f60 msgr2=0x7f2bd00073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2bed0b3640 1 --2- 172.21.15.37:0/2771575129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bd0006f60 0x7f2bd00073e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c210 tx=0x7f2bd801cc60 comp rx=0 tx=0).stop 2024-04-25T08:58:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2be759e640 1 -- 172.21.15.37:0/2771575129 reap_dead start 2024-04-25T08:58:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2bed0b3640 1 -- 172.21.15.37:0/2771575129 shutdown_connections 2024-04-25T08:58:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2bed0b3640 1 -- 172.21.15.37:0/2771575129 >> 172.21.15.37:0/2771575129 conn(0x7f2be806ae60 msgr2=0x7f2be806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2bed0b3640 1 -- 172.21.15.37:0/2771575129 shutdown_connections 2024-04-25T08:58:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2bed0b3640 1 -- 172.21.15.37:0/2771575129 wait complete. 2024-04-25T08:58:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2bed0b3640 1 Processor -- start 2024-04-25T08:58:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2bed0b3640 1 -- start start 2024-04-25T08:58:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2bed0b3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2be81540f0 0x7f2be8071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2bed0b3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be81f3540 0x7f2be815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.152+0000 7f2bed0b3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815bd00 0x7f2be81f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bed0b3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80cd440 con 0x7f2be815bd00 2024-04-25T08:58:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bed0b3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be815f0f0 con 0x7f2be81540f0 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bed0b3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80cdce0 con 0x7f2be81f3540 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815bd00 0x7f2be81f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815bd00 0x7f2be81f4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49850/0 (socket says 172.21.15.37:49850) 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be759e640 1 -- 172.21.15.37:0/99962209 learned_addr learned my addr 172.21.15.37:0/99962209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be6d9d640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2be81540f0 0x7f2be8071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be659c640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be81f3540 0x7f2be815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be759e640 1 -- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be81f3540 msgr2=0x7f2be815b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be759e640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be81f3540 0x7f2be815b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be759e640 1 -- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2be81540f0 msgr2=0x7f2be8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be759e640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2be81540f0 0x7f2be8071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be759e640 1 -- 172.21.15.37:0/99962209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be815c520 con 0x7f2be815bd00 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be759e640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815bd00 0x7f2be81f4da0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c210 tx=0x7f2bd0007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/99962209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd8007d20 con 0x7f2be815bd00 2024-04-25T08:58:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/99962209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bd80a5b70 con 0x7f2be815bd00 2024-04-25T08:58:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bed0b3640 1 -- 172.21.15.37:0/99962209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be8072680 con 0x7f2be815bd00 2024-04-25T08:58:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/99962209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bd80048b0 con 0x7f2be815bd00 2024-04-25T08:58:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bed0b3640 1 -- 172.21.15.37:0/99962209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be8072c40 con 0x7f2be815bd00 2024-04-25T08:58:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bed0b3640 1 -- 172.21.15.37:0/99962209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be804efa0 con 0x7f2be815bd00 2024-04-25T08:58:38.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/99962209 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bd8004a50 con 0x7f2be815bd00 2024-04-25T08:58:38.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bcf7fe640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2bd4066f90 0x7f2bd4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/99962209 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2bd811e510 con 0x7f2be815bd00 2024-04-25T08:58:38.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be6d9d640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2bd4066f90 0x7f2bd4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.156+0000 7f2be6d9d640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2bd4066f90 0x7f2bd4069450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2be0000f90 tx=0x7f2be0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.160+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/99962209 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bd8016970 con 0x7f2be815bd00 2024-04-25T08:58:38.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 -- 172.21.15.37:0/99962209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2be80b4170 con 0x7f2be815bd00 2024-04-25T08:58:38.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bcf7fe640 1 -- 172.21.15.37:0/99962209 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2bd80ef3d0 con 0x7f2be815bd00 2024-04-25T08:58:38.307 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:58:38.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 -- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2bd4066f90 msgr2=0x7f2bd4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2bd4066f90 0x7f2bd4069450 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2be0000f90 tx=0x7f2be0009040 comp rx=0 tx=0).stop 2024-04-25T08:58:38.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 -- 172.21.15.37:0/99962209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815bd00 msgr2=0x7f2be81f4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815bd00 0x7f2be81f4da0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f2bd800c210 tx=0x7f2bd0007a50 comp rx=0 tx=0).stop 2024-04-25T08:58:38.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 -- 172.21.15.37:0/99962209 shutdown_connections 2024-04-25T08:58:38.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2bd4066f90 0x7f2bd4069450 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2be815bd00 0x7f2be81f4da0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2be81f3540 0x7f2be815b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 --2- 172.21.15.37:0/99962209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2be81540f0 0x7f2be8071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 -- 172.21.15.37:0/99962209 >> 172.21.15.37:0/99962209 conn(0x7f2be806ae60 msgr2=0x7f2be80cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:38.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 -- 172.21.15.37:0/99962209 shutdown_connections 2024-04-25T08:58:38.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.316+0000 7f2bed0b3640 1 -- 172.21.15.37:0/99962209 wait complete. 2024-04-25T08:58:38.324 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:38.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2dd835640 1 Processor -- start 2024-04-25T08:58:38.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2dd835640 1 -- start start 2024-04-25T08:58:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2dd835640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb2d8066e30 0x7fb2d806c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2dd835640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2d80af370 0x7fb2d80ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2dd835640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d806ec30 con 0x7fb2d806bca0 2024-04-25T08:58:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2dd835640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d80669a0 con 0x7fb2d8066e30 2024-04-25T08:58:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2dd835640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d806c5c0 con 0x7fb2d80af370 2024-04-25T08:58:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d6ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb2d806bca0 0x7fb2d80cd0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:54520/0 (socket says 172.21.15.37:54520) 2024-04-25T08:58:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d6ffd640 1 -- 172.21.15.37:0/55754270 learned_addr learned my addr 172.21.15.37:0/55754270 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d77fe640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2d80af370 0x7fb2d80ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d67fc640 1 --2- 172.21.15.37:0/55754270 >> v2:172.21.15.203:3300/0 conn(0x7fb2d8066e30 0x7fb2d806c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3972575162 0 0) 0x7fb2d806ec30 con 0x7fb2d806bca0 2024-04-25T08:58:38.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c4003530 con 0x7fb2d806bca0 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2152172359 0 0) 0x7fb2c4003530 con 0x7fb2d806bca0 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2d80af370 msgr2=0x7fb2d80ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2d80af370 0x7fb2d80ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 >> v2:172.21.15.203:3300/0 conn(0x7fb2d8066e30 msgr2=0x7fb2d806c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 --2- 172.21.15.37:0/55754270 >> v2:172.21.15.203:3300/0 conn(0x7fb2d8066e30 0x7fb2d806c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d806f230 con 0x7fb2d806bca0 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fb2d0001680 con 0x7fb2d806bca0 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 >> v1:172.21.15.37:6789/0 conn(0x7fb2d806bca0 legacy=0x7fb2d80cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2d80af370 0x7fb2c4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4003fb0 0x7fb2c4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c4004d50 0x7fb2c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80669a0 con 0x7fb2c4003fb0 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d806c5c0 con 0x7fb2c4004d50 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c400bb30 con 0x7fb2d80af370 2024-04-25T08:58:38.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d67fc640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4003fb0 0x7fb2c4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d6ffd640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2d80af370 0x7fb2c4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d77fe640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c4004d50 0x7fb2c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d67fc640 1 -- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2d80af370 msgr2=0x7fb2c4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:38.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d67fc640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2d80af370 0x7fb2c4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d67fc640 1 -- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c4004d50 msgr2=0x7fb2c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:38.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d67fc640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c4004d50 0x7fb2c40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d67fc640 1 -- 172.21.15.37:0/55754270 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c4003530 con 0x7fb2c4003fb0 2024-04-25T08:58:38.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d6ffd640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2d80af370 0x7fb2c4003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:38.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.456+0000 7fb2d77fe640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2c4004d50 0x7fb2c40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:38.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.460+0000 7fb2d67fc640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4003fb0 0x7fb2c4004810 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009ae0 tx=0x7fb2c001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce486854ae59261 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.460+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00b2070 con 0x7fb2c4003fb0 2024-04-25T08:58:38.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.460+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c00a5560 con 0x7fb2c4003fb0 2024-04-25T08:58:38.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.460+0000 7fb2d5ffb640 1 -- 172.21.15.37:0/55754270 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00096b0 con 0x7fb2c4003fb0 2024-04-25T08:58:38.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.460+0000 7fb2dd835640 1 -- 172.21.15.37:0/55754270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4003fb0 msgr2=0x7fb2c4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.460+0000 7fb2dd835640 1 --2- 172.21.15.37:0/55754270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2c4003fb0 0x7fb2c4004810 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fb2c0009ae0 tx=0x7fb2c001ca80 comp rx=0 tx=0).stop 2024-04-25T08:58:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.460+0000 7fb2d77fe640 1 -- 172.21.15.37:0/55754270 reap_dead start 2024-04-25T08:58:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.460+0000 7fb2dd835640 1 -- 172.21.15.37:0/55754270 shutdown_connections 2024-04-25T08:58:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.460+0000 7fb2dd835640 1 -- 172.21.15.37:0/55754270 >> 172.21.15.37:0/55754270 conn(0x7fb2d8071810 msgr2=0x7fb2d8071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.460+0000 7fb2dd835640 1 -- 172.21.15.37:0/55754270 shutdown_connections 2024-04-25T08:58:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.460+0000 7fb2dd835640 1 -- 172.21.15.37:0/55754270 wait complete. 2024-04-25T08:58:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2dd835640 1 Processor -- start 2024-04-25T08:58:38.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2dd835640 1 -- start start 2024-04-25T08:58:38.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2dd835640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c4003fb0 0x7fb2d814ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2dd835640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d8158bc0 0x7fb2d815b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2dd835640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d8159040 0x7fb2d815bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2dd835640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80cf480 con 0x7fb2d8159040 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2dd835640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80cf300 con 0x7fb2d8158bc0 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2dd835640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d80cf600 con 0x7fb2c4003fb0 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d8159040 0x7fb2d815bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d8159040 0x7fb2d815bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49870/0 (socket says 172.21.15.37:49870) 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d77fe640 1 -- 172.21.15.37:0/2588209063 learned_addr learned my addr 172.21.15.37:0/2588209063 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d6ffd640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c4003fb0 0x7fb2d814ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d67fc640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d8158bc0 0x7fb2d815b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d77fe640 1 -- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c4003fb0 msgr2=0x7fb2d814ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d77fe640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c4003fb0 0x7fb2d814ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d77fe640 1 -- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d8158bc0 msgr2=0x7fb2d815b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d77fe640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d8158bc0 0x7fb2d815b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d77fe640 1 -- 172.21.15.37:0/2588209063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d8159610 con 0x7fb2d8159040 2024-04-25T08:58:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d6ffd640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c4003fb0 0x7fb2d814ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2d77fe640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d8159040 0x7fb2d815bd20 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80ce510 tx=0x7fb2c8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2cffff640 1 -- 172.21.15.37:0/2588209063 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c801a030 con 0x7fb2d8159040 2024-04-25T08:58:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2cffff640 1 -- 172.21.15.37:0/2588209063 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c8010d80 con 0x7fb2d8159040 2024-04-25T08:58:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2dd835640 1 -- 172.21.15.37:0/2588209063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d81598f0 con 0x7fb2d8159040 2024-04-25T08:58:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2cffff640 1 -- 172.21.15.37:0/2588209063 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8003be0 con 0x7fb2d8159040 2024-04-25T08:58:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2dd835640 1 -- 172.21.15.37:0/2588209063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d8159eb0 con 0x7fb2d8159040 2024-04-25T08:58:38.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2dd835640 1 -- 172.21.15.37:0/2588209063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d815a290 con 0x7fb2d8159040 2024-04-25T08:58:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2cffff640 1 -- 172.21.15.37:0/2588209063 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2c801d070 con 0x7fb2d8159040 2024-04-25T08:58:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.464+0000 7fb2cffff640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b8066f90 0x7fb2b8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.468+0000 7fb2cffff640 1 -- 172.21.15.37:0/2588209063 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb2c8085720 con 0x7fb2d8159040 2024-04-25T08:58:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.468+0000 7fb2d6ffd640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b8066f90 0x7fb2b8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.468+0000 7fb2d6ffd640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b8066f90 0x7fb2b8069450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0001ec0 tx=0x7fb2d0001440 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.472+0000 7fb2cffff640 1 -- 172.21.15.37:0/2588209063 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2d815a290 con 0x7fb2d8159040 2024-04-25T08:58:38.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.664+0000 7fb2dd835640 1 -- 172.21.15.37:0/2588209063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2d804efa0 con 0x7fb2d8159040 2024-04-25T08:58:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.664+0000 7fb2cffff640 1 -- 172.21.15.37:0/2588209063 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fb2c80565e0 con 0x7fb2d8159040 2024-04-25T08:58:38.657 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:38.657 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:58:02.214606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":16,"addr":"172.21.15.37:6835/3927528604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3927528604},{"type":"v1","addr":"172.21.15.37:6835","nonce":3927528604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:58:38.657 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 52 2024-04-25T08:58:38.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 -- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b8066f90 msgr2=0x7fb2b8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b8066f90 0x7fb2b8069450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0001ec0 tx=0x7fb2d0001440 comp rx=0 tx=0).stop 2024-04-25T08:58:38.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 -- 172.21.15.37:0/2588209063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d8159040 msgr2=0x7fb2d815bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d8159040 0x7fb2d815bd20 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80ce510 tx=0x7fb2c8010590 comp rx=0 tx=0).stop 2024-04-25T08:58:38.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 -- 172.21.15.37:0/2588209063 shutdown_connections 2024-04-25T08:58:38.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2b8066f90 0x7fb2b8069450 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2d8159040 0x7fb2d815bd20 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2d8158bc0 0x7fb2d815b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 --2- 172.21.15.37:0/2588209063 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2c4003fb0 0x7fb2d814ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 -- 172.21.15.37:0/2588209063 >> 172.21.15.37:0/2588209063 conn(0x7fb2d8071810 msgr2=0x7fb2d80cd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:38.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 -- 172.21.15.37:0/2588209063 shutdown_connections 2024-04-25T08:58:38.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.668+0000 7fb2dd835640 1 -- 172.21.15.37:0/2588209063 wait complete. 2024-04-25T08:58:38.678 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:38.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca73204640 1 Processor -- start 2024-04-25T08:58:38.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca73204640 1 -- start start 2024-04-25T08:58:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca73204640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fca6c15da80 0x7fca6c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca73204640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fca6c0d07f0 0x7fca6c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca73204640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca6c06be60 con 0x7fca6c0d1cd0 2024-04-25T08:58:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca73204640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6c06c440 con 0x7fca6c0d07f0 2024-04-25T08:58:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca73204640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca6c15de60 con 0x7fca6c15da80 2024-04-25T08:58:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca717a1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fca6c0d07f0 0x7fca6c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca717a1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fca6c0d07f0 0x7fca6c06c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49886/0 (socket says 172.21.15.37:49886) 2024-04-25T08:58:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca717a1640 1 -- 172.21.15.37:0/3696285843 learned_addr learned my addr 172.21.15.37:0/3696285843 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca70fa0640 1 --2- 172.21.15.37:0/3696285843 >> v2:172.21.15.203:3300/0 conn(0x7fca6c15da80 0x7fca6c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca717a1640 1 -- 172.21.15.37:0/3696285843 >> v2:172.21.15.203:3300/0 conn(0x7fca6c15da80 msgr2=0x7fca6c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca717a1640 1 --2- 172.21.15.37:0/3696285843 >> v2:172.21.15.203:3300/0 conn(0x7fca6c15da80 0x7fca6c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca717a1640 1 -- 172.21.15.37:0/3696285843 >> v1:172.21.15.203:6789/0 conn(0x7fca6c0d1cd0 legacy=0x7fca6c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:38.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.804+0000 7fca717a1640 1 -- 172.21.15.37:0/3696285843 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c0d2450 con 0x7fca6c0d07f0 2024-04-25T08:58:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca717a1640 1 --2- 172.21.15.37:0/3696285843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca6c0d07f0 0x7fca6c06c8d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fca64009e50 tx=0x7fca6401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a912213d26428d3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca6b7fe640 1 -- 172.21.15.37:0/3696285843 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca640a57e0 con 0x7fca6c0d07f0 2024-04-25T08:58:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca6b7fe640 1 -- 172.21.15.37:0/3696285843 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca640a5980 con 0x7fca6c0d07f0 2024-04-25T08:58:38.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca6b7fe640 1 -- 172.21.15.37:0/3696285843 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca640a5d50 con 0x7fca6c0d07f0 2024-04-25T08:58:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca73204640 1 -- 172.21.15.37:0/3696285843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca6c0d07f0 msgr2=0x7fca6c06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca73204640 1 --2- 172.21.15.37:0/3696285843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca6c0d07f0 0x7fca6c06c8d0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7fca64009e50 tx=0x7fca6401c980 comp rx=0 tx=0).stop 2024-04-25T08:58:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca73204640 1 -- 172.21.15.37:0/3696285843 shutdown_connections 2024-04-25T08:58:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca73204640 1 --2- 172.21.15.37:0/3696285843 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca6c0d07f0 0x7fca6c06c8d0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca73204640 1 --2- 172.21.15.37:0/3696285843 >> v2:172.21.15.203:3300/0 conn(0x7fca6c15da80 0x7fca6c0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca73204640 1 -- 172.21.15.37:0/3696285843 >> 172.21.15.37:0/3696285843 conn(0x7fca6c08b000 msgr2=0x7fca6c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca73204640 1 -- 172.21.15.37:0/3696285843 shutdown_connections 2024-04-25T08:58:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca73204640 1 -- 172.21.15.37:0/3696285843 wait complete. 2024-04-25T08:58:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca73204640 1 Processor -- start 2024-04-25T08:58:38.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.808+0000 7fca73204640 1 -- start start 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca73204640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c0d07f0 0x7fca6c155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca73204640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca6c0d1cd0 0x7fca6c160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca73204640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca6c15da80 0x7fca6c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca73204640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c06d7a0 con 0x7fca6c15da80 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca73204640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c06d620 con 0x7fca6c0d1cd0 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca73204640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6c06d920 con 0x7fca6c0d07f0 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca717a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca6c15da80 0x7fca6c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca717a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca6c15da80 0x7fca6c0cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49894/0 (socket says 172.21.15.37:49894) 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca717a1640 1 -- 172.21.15.37:0/2426124155 learned_addr learned my addr 172.21.15.37:0/2426124155 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca70fa0640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c0d07f0 0x7fca6c155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca717a1640 1 -- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c0d07f0 msgr2=0x7fca6c155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca717a1640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c0d07f0 0x7fca6c155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca6bfff640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca6c0d1cd0 0x7fca6c160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca717a1640 1 -- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca6c0d1cd0 msgr2=0x7fca6c160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca717a1640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca6c0d1cd0 0x7fca6c160730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca717a1640 1 -- 172.21.15.37:0/2426124155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c155cf0 con 0x7fca6c15da80 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca70fa0640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c0d07f0 0x7fca6c155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca717a1640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca6c15da80 0x7fca6c0cc990 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fca6401a5b0 tx=0x7fca64019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca697fa640 1 -- 172.21.15.37:0/2426124155 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca640b8670 con 0x7fca6c15da80 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca697fa640 1 -- 172.21.15.37:0/2426124155 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca640be070 con 0x7fca6c15da80 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca73204640 1 -- 172.21.15.37:0/2426124155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca6c155fd0 con 0x7fca6c15da80 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca697fa640 1 -- 172.21.15.37:0/2426124155 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca640b8a30 con 0x7fca6c15da80 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca73204640 1 -- 172.21.15.37:0/2426124155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca6c156590 con 0x7fca6c15da80 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca73204640 1 -- 172.21.15.37:0/2426124155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca6c04efa0 con 0x7fca6c15da80 2024-04-25T08:58:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca697fa640 1 -- 172.21.15.37:0/2426124155 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fca640b8bd0 con 0x7fca6c15da80 2024-04-25T08:58:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca697fa640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fca58066f60 0x7fca58069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca697fa640 1 -- 172.21.15.37:0/2426124155 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fca64121030 con 0x7fca6c15da80 2024-04-25T08:58:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.812+0000 7fca70fa0640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fca58066f60 0x7fca58069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.816+0000 7fca70fa0640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fca58066f60 0x7fca58069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0097a0 tx=0x7fca5c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.816+0000 7fca697fa640 1 -- 172.21.15.37:0/2426124155 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fca640c8020 con 0x7fca6c15da80 2024-04-25T08:58:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.980+0000 7fca73204640 1 -- 172.21.15.37:0/2426124155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca6c0b2500 con 0x7fca6c15da80 2024-04-25T08:58:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.980+0000 7fca697fa640 1 -- 172.21.15.37:0/2426124155 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17038 (secure 0 0 0) 0x7fca640165a0 con 0x7fca6c15da80 2024-04-25T08:58:38.974 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:38.974 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":83,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:57:11.180680+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:56:49.737591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:56:50.436964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/4130042812":"2024-04-26T08:57:11.180643+0000","172.21.15.37:6834/4130042812":"2024-04-26T08:57:11.180643+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:38.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 -- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fca58066f60 msgr2=0x7fca58069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fca58066f60 0x7fca58069420 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0097a0 tx=0x7fca5c008040 comp rx=0 tx=0).stop 2024-04-25T08:58:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 -- 172.21.15.37:0/2426124155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca6c15da80 msgr2=0x7fca6c0cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca6c15da80 0x7fca6c0cc990 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fca6401a5b0 tx=0x7fca64019720 comp rx=0 tx=0).stop 2024-04-25T08:58:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 -- 172.21.15.37:0/2426124155 shutdown_connections 2024-04-25T08:58:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fca58066f60 0x7fca58069420 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca6c15da80 0x7fca6c0cc990 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fca6c0d1cd0 0x7fca6c160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 --2- 172.21.15.37:0/2426124155 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fca6c0d07f0 0x7fca6c155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 -- 172.21.15.37:0/2426124155 >> 172.21.15.37:0/2426124155 conn(0x7fca6c08b000 msgr2=0x7fca6c0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 -- 172.21.15.37:0/2426124155 shutdown_connections 2024-04-25T08:58:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:38.984+0000 7fca73204640 1 -- 172.21.15.37:0/2426124155 wait complete. 2024-04-25T08:58:38.991 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:58:38.992 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:39.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27e5002640 1 Processor -- start 2024-04-25T08:58:39.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27e5002640 1 -- start start 2024-04-25T08:58:39.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27e5002640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f27e00b0b00 0x7f27e00cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27e5002640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f27e00cc650 0x7f27e00b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27e5002640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e00b1d40 con 0x7f27e00b1910 2024-04-25T08:58:39.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27e5002640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e015cfa0 con 0x7f27e00b0b00 2024-04-25T08:58:39.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27e5002640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e015cdf0 con 0x7f27e00cc650 2024-04-25T08:58:39.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27ded9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f27e00b0b00 0x7f27e00cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27ded9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f27e00b0b00 0x7f27e00cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49898/0 (socket says 172.21.15.37:49898) 2024-04-25T08:58:39.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27ded9d640 1 -- 172.21.15.37:0/1104529605 learned_addr learned my addr 172.21.15.37:0/1104529605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:39.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27de59c640 1 --2- 172.21.15.37:0/1104529605 >> v2:172.21.15.203:3300/0 conn(0x7f27e00cc650 0x7f27e00b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27ded9d640 1 -- 172.21.15.37:0/1104529605 >> v2:172.21.15.203:3300/0 conn(0x7f27e00cc650 msgr2=0x7f27e00b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:39.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27ded9d640 1 --2- 172.21.15.37:0/1104529605 >> v2:172.21.15.203:3300/0 conn(0x7f27e00cc650 0x7f27e00b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27ded9d640 1 -- 172.21.15.37:0/1104529605 >> v1:172.21.15.203:6789/0 conn(0x7f27e00b1910 legacy=0x7f27e0151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.124+0000 7f27ded9d640 1 -- 172.21.15.37:0/1104529605 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e0160310 con 0x7f27e00b0b00 2024-04-25T08:58:39.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27ded9d640 1 --2- 172.21.15.37:0/1104529605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e00b0b00 0x7f27e00cc110 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f27e00ca880 tx=0x7f27d001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acac839b31391a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27ddd9b640 1 -- 172.21.15.37:0/1104529605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d00b0070 con 0x7f27e00b0b00 2024-04-25T08:58:39.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27ddd9b640 1 -- 172.21.15.37:0/1104529605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d00ab600 con 0x7f27e00b0b00 2024-04-25T08:58:39.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27ddd9b640 1 -- 172.21.15.37:0/1104529605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d00ab9d0 con 0x7f27e00b0b00 2024-04-25T08:58:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 -- 172.21.15.37:0/1104529605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e00b0b00 msgr2=0x7f27e00cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 --2- 172.21.15.37:0/1104529605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e00b0b00 0x7f27e00cc110 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f27e00ca880 tx=0x7f27d001c520 comp rx=0 tx=0).stop 2024-04-25T08:58:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 -- 172.21.15.37:0/1104529605 shutdown_connections 2024-04-25T08:58:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 --2- 172.21.15.37:0/1104529605 >> v2:172.21.15.203:3300/0 conn(0x7f27e00cc650 0x7f27e00b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 --2- 172.21.15.37:0/1104529605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e00b0b00 0x7f27e00cc110 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 -- 172.21.15.37:0/1104529605 >> 172.21.15.37:0/1104529605 conn(0x7f27e00b1f10 msgr2=0x7f27e00b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 -- 172.21.15.37:0/1104529605 shutdown_connections 2024-04-25T08:58:39.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 -- 172.21.15.37:0/1104529605 wait complete. 2024-04-25T08:58:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 Processor -- start 2024-04-25T08:58:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 -- start start 2024-04-25T08:58:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27e00b0b00 0x7f27e0155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e00b1910 0x7f27e0157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e00cc650 0x7f27e014eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00b0ee0 con 0x7f27e00cc650 2024-04-25T08:58:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e006b950 con 0x7f27e00b0b00 2024-04-25T08:58:39.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27e5002640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00b1060 con 0x7f27e00b1910 2024-04-25T08:58:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27df59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e00cc650 0x7f27e014eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27df59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e00cc650 0x7f27e014eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49910/0 (socket says 172.21.15.37:49910) 2024-04-25T08:58:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27df59e640 1 -- 172.21.15.37:0/605485537 learned_addr learned my addr 172.21.15.37:0/605485537 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27ded9d640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27e00b0b00 0x7f27e0155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27de59c640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e00b1910 0x7f27e0157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27df59e640 1 -- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e00b1910 msgr2=0x7f27e0157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27df59e640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e00b1910 0x7f27e0157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27df59e640 1 -- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27e00b0b00 msgr2=0x7f27e0155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27df59e640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27e00b0b00 0x7f27e0155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27df59e640 1 -- 172.21.15.37:0/605485537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e014f040 con 0x7f27e00cc650 2024-04-25T08:58:39.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27ded9d640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27e00b0b00 0x7f27e0155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:39.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.128+0000 7f27de59c640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e00b1910 0x7f27e0157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:39.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27df59e640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e00cc650 0x7f27e014eb00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f27d800c770 tx=0x7f27d800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27c77fe640 1 -- 172.21.15.37:0/605485537 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8003b80 con 0x7f27e00cc650 2024-04-25T08:58:39.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27c77fe640 1 -- 172.21.15.37:0/605485537 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d8003d20 con 0x7f27e00cc650 2024-04-25T08:58:39.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27e5002640 1 -- 172.21.15.37:0/605485537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e0158130 con 0x7f27e00cc650 2024-04-25T08:58:39.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27c77fe640 1 -- 172.21.15.37:0/605485537 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d800aa90 con 0x7f27e00cc650 2024-04-25T08:58:39.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27e5002640 1 -- 172.21.15.37:0/605485537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e01586f0 con 0x7f27e00cc650 2024-04-25T08:58:39.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27e5002640 1 -- 172.21.15.37:0/605485537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e004efa0 con 0x7f27e00cc650 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27c77fe640 1 -- 172.21.15.37:0/605485537 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27d800ac30 con 0x7f27e00cc650 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27c77fe640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27cc066f60 0x7f27cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27c77fe640 1 -- 172.21.15.37:0/605485537 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f27d8085190 con 0x7f27e00cc650 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27ded9d640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27cc066f60 0x7f27cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.132+0000 7f27ded9d640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27cc066f60 0x7f27cc069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f27d00ad000 tx=0x7f27d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.136+0000 7f27c77fe640 1 -- 172.21.15.37:0/605485537 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27d8055fc0 con 0x7f27e00cc650 2024-04-25T08:58:39.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.324+0000 7f27e5002640 1 -- 172.21.15.37:0/605485537 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27e00b6380 con 0x7f27e00cc650 2024-04-25T08:58:39.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.324+0000 7f27c77fe640 1 -- 172.21.15.37:0/605485537 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f27d80558c0 con 0x7f27e00cc650 2024-04-25T08:58:39.316 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:39.316 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:58:02.214606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":16,"addr":"172.21.15.37:6835/3927528604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3927528604},{"type":"v1","addr":"172.21.15.37:6835","nonce":3927528604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:58:39.316 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 52 2024-04-25T08:58:39.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 -- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27cc066f60 msgr2=0x7f27cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27cc066f60 0x7f27cc069420 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f27d00ad000 tx=0x7f27d0008040 comp rx=0 tx=0).stop 2024-04-25T08:58:39.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 -- 172.21.15.37:0/605485537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e00cc650 msgr2=0x7f27e014eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e00cc650 0x7f27e014eb00 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f27d800c770 tx=0x7f27d800cc30 comp rx=0 tx=0).stop 2024-04-25T08:58:39.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 -- 172.21.15.37:0/605485537 shutdown_connections 2024-04-25T08:58:39.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27cc066f60 0x7f27cc069420 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27e00cc650 0x7f27e014eb00 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27e00b1910 0x7f27e0157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 --2- 172.21.15.37:0/605485537 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27e00b0b00 0x7f27e0155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 -- 172.21.15.37:0/605485537 >> 172.21.15.37:0/605485537 conn(0x7f27e00b1f10 msgr2=0x7f27e015df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:39.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 -- 172.21.15.37:0/605485537 shutdown_connections 2024-04-25T08:58:39.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.328+0000 7f27e5002640 1 -- 172.21.15.37:0/605485537 wait complete. 2024-04-25T08:58:39.335 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:39.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb470f08640 1 Processor -- start 2024-04-25T08:58:39.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb470f08640 1 -- start start 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb470f08640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb46c15b950 0x7fb46c09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb470f08640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb46c0cfb80 0x7fb46c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb470f08640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb46c15bd30 con 0x7fb46c0cf420 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb470f08640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46c0d0510 con 0x7fb46c15b950 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb470f08640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0cc0d0 con 0x7fb46c0cfb80 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb469d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb46c0cf420 0x7fb46c0cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52768/0 (socket says 172.21.15.37:52768) 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb46a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb46c15b950 0x7fb46c09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb469d9b640 1 -- 172.21.15.37:0/2744427106 learned_addr learned my addr 172.21.15.37:0/2744427106 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb46ad9d640 1 --2- 172.21.15.37:0/2744427106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb46c0cfb80 0x7fb46c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb46a59c640 1 -- 172.21.15.37:0/2744427106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb46c0cfb80 msgr2=0x7fb46c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb46a59c640 1 --2- 172.21.15.37:0/2744427106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb46c0cfb80 0x7fb46c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb46959a640 1 -- 172.21.15.37:0/2744427106 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764166140 0 0) 0x7fb46c15bd30 con 0x7fb46c0cf420 2024-04-25T08:58:39.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb46a59c640 1 -- 172.21.15.37:0/2744427106 >> v1:172.21.15.37:6789/0 conn(0x7fb46c0cf420 legacy=0x7fb46c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.460+0000 7fb46a59c640 1 -- 172.21.15.37:0/2744427106 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c09ab10 con 0x7fb46c15b950 2024-04-25T08:58:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb46a59c640 1 --2- 172.21.15.37:0/2744427106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb46c15b950 0x7fb46c09a480 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0097c0 tx=0x7fb45c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a9c050ec7c14c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb46959a640 1 -- 172.21.15.37:0/2744427106 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0b2070 con 0x7fb46c15b950 2024-04-25T08:58:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb46959a640 1 -- 172.21.15.37:0/2744427106 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c0ad640 con 0x7fb46c15b950 2024-04-25T08:58:39.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb46959a640 1 -- 172.21.15.37:0/2744427106 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c0ada10 con 0x7fb46c15b950 2024-04-25T08:58:39.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 -- 172.21.15.37:0/2744427106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb46c15b950 msgr2=0x7fb46c09a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 --2- 172.21.15.37:0/2744427106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb46c15b950 0x7fb46c09a480 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0097c0 tx=0x7fb45c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 -- 172.21.15.37:0/2744427106 shutdown_connections 2024-04-25T08:58:39.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 --2- 172.21.15.37:0/2744427106 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb46c0cfb80 0x7fb46c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 --2- 172.21.15.37:0/2744427106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb46c15b950 0x7fb46c09a480 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 -- 172.21.15.37:0/2744427106 >> 172.21.15.37:0/2744427106 conn(0x7fb46c0b30b0 msgr2=0x7fb46c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:39.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 -- 172.21.15.37:0/2744427106 shutdown_connections 2024-04-25T08:58:39.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 -- 172.21.15.37:0/2744427106 wait complete. 2024-04-25T08:58:39.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 Processor -- start 2024-04-25T08:58:39.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 -- start start 2024-04-25T08:58:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb46c0cf420 0x7fb46c159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb46c0cfb80 0x7fb46c0b6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb46c15b950 0x7fb46c0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c06faa0 con 0x7fb46c0cfb80 2024-04-25T08:58:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c0cf870 con 0x7fb46c15b950 2024-04-25T08:58:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb470f08640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46c06fc20 con 0x7fb46c0cf420 2024-04-25T08:58:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb469d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb46c0cfb80 0x7fb46c0b6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb469d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb46c0cfb80 0x7fb46c0b6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55454/0 (socket says 172.21.15.37:55454) 2024-04-25T08:58:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.464+0000 7fb469d9b640 1 -- 172.21.15.37:0/2916828393 learned_addr learned my addr 172.21.15.37:0/2916828393 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb46ad9d640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb46c15b950 0x7fb46c0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb46a59c640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb46c0cf420 0x7fb46c159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb469d9b640 1 -- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb46c0cf420 msgr2=0x7fb46c159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb469d9b640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb46c0cf420 0x7fb46c159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb469d9b640 1 -- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb46c15b950 msgr2=0x7fb46c0b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb469d9b640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb46c15b950 0x7fb46c0b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb469d9b640 1 -- 172.21.15.37:0/2916828393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c0b7170 con 0x7fb46c0cfb80 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb46a59c640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb46c0cf420 0x7fb46c159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb469d9b640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb46c0cfb80 0x7fb46c0b6c30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c4b0 tx=0x7fb46000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb45affd640 1 -- 172.21.15.37:0/2916828393 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460018070 con 0x7fb46c0cfb80 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb45affd640 1 -- 172.21.15.37:0/2916828393 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb460007500 con 0x7fb46c0cfb80 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb45affd640 1 -- 172.21.15.37:0/2916828393 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460004a30 con 0x7fb46c0cfb80 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb470f08640 1 -- 172.21.15.37:0/2916828393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46c158ba0 con 0x7fb46c0cfb80 2024-04-25T08:58:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb470f08640 1 -- 172.21.15.37:0/2916828393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb46c159160 con 0x7fb46c0cfb80 2024-04-25T08:58:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb470f08640 1 -- 172.21.15.37:0/2916828393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb46c04efa0 con 0x7fb46c0cfb80 2024-04-25T08:58:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb45affd640 1 -- 172.21.15.37:0/2916828393 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb460004bd0 con 0x7fb46c0cfb80 2024-04-25T08:58:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb45affd640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb44c066f60 0x7fb44c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb45affd640 1 -- 172.21.15.37:0/2916828393 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb4600844b0 con 0x7fb46c0cfb80 2024-04-25T08:58:39.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.468+0000 7fb46a59c640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb44c066f60 0x7fb44c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.472+0000 7fb46a59c640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb44c066f60 0x7fb44c069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0af000 tx=0x7fb45c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.472+0000 7fb45affd640 1 -- 172.21.15.37:0/2916828393 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb460055240 con 0x7fb46c0cfb80 2024-04-25T08:58:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 -- 172.21.15.37:0/2916828393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb46c0cef70 con 0x7fb46c0cfb80 2024-04-25T08:58:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb45affd640 1 -- 172.21.15.37:0/2916828393 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fb460054b40 con 0x7fb46c0cfb80 2024-04-25T08:58:39.656 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:39.656 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":52,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:56:51.447805+0000","modified":"2024-04-25T08:58:02.214606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7200},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7200":{"gid":7200,"name":"c","rank":0,"incarnation":47,"state":"up:active","state_seq":16,"addr":"172.21.15.37:6835/3927528604","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3927528604},{"type":"v1","addr":"172.21.15.37:6835","nonce":3927528604}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:58:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 52 2024-04-25T08:58:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 -- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb44c066f60 msgr2=0x7fb44c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb44c066f60 0x7fb44c069420 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0af000 tx=0x7fb45c008040 comp rx=0 tx=0).stop 2024-04-25T08:58:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 -- 172.21.15.37:0/2916828393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb46c0cfb80 msgr2=0x7fb46c0b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb46c0cfb80 0x7fb46c0b6c30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fb46000c4b0 tx=0x7fb46000c970 comp rx=0 tx=0).stop 2024-04-25T08:58:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 -- 172.21.15.37:0/2916828393 shutdown_connections 2024-04-25T08:58:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb44c066f60 0x7fb44c069420 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb46c15b950 0x7fb46c0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb46c0cfb80 0x7fb46c0b6c30 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 --2- 172.21.15.37:0/2916828393 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb46c0cf420 0x7fb46c159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 -- 172.21.15.37:0/2916828393 >> 172.21.15.37:0/2916828393 conn(0x7fb46c0b30b0 msgr2=0x7fb46c073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 -- 172.21.15.37:0/2916828393 shutdown_connections 2024-04-25T08:58:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.664+0000 7fb470f08640 1 -- 172.21.15.37:0/2916828393 wait complete. 2024-04-25T08:58:39.674 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f2637b90640 1 Processor -- start 2024-04-25T08:58:39.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f2637b90640 1 -- start start 2024-04-25T08:58:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f2637b90640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2630144ca0 0x7f263006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f2637b90640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26300ccc70 con 0x7f263006ec60 2024-04-25T08:58:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f2637b90640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f263006dc40 con 0x7f26300ce510 2024-04-25T08:58:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f2637b90640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26300cc2b0 con 0x7f2630144ca0 2024-04-25T08:58:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263612d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f263006ec60 0x7f2630142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52770/0 (socket says 172.21.15.37:52770) 2024-04-25T08:58:39.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263612d640 1 -- 172.21.15.37:0/1322959730 learned_addr learned my addr 172.21.15.37:0/1322959730 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263512b640 1 --2- 172.21.15.37:0/1322959730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2630144ca0 0x7f263006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459999823 0 0) 0x7f26300ccc70 con 0x7f263006ec60 2024-04-25T08:58:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f261c003530 con 0x7f263006ec60 2024-04-25T08:58:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 743234938 0 0) 0x7f261c003530 con 0x7f263006ec60 2024-04-25T08:58:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2630144ca0 msgr2=0x7f263006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 --2- 172.21.15.37:0/1322959730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2630144ca0 0x7f263006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 >> v1:172.21.15.203:6789/0 conn(0x7f26300ce510 legacy=0x7f26300ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263006df20 con 0x7f263006ec60 2024-04-25T08:58:39.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f2628001a00 con 0x7f263006ec60 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 >> v1:172.21.15.37:6789/0 conn(0x7f263006ec60 legacy=0x7f2630142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 --2- 172.21.15.37:0/1322959730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2630144ca0 0x7f261c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 --2- 172.21.15.37:0/1322959730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f261c004030 0x7f261c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 --2- 172.21.15.37:0/1322959730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f261c004d50 0x7f261c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26300cc2b0 con 0x7f261c004030 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f261c00bc20 con 0x7f261c004d50 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261c00be20 con 0x7f2630144ca0 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263512b640 1 --2- 172.21.15.37:0/1322959730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f261c004030 0x7f261c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263612d640 1 --2- 172.21.15.37:0/1322959730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f261c004d50 0x7f261c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263512b640 1 -- 172.21.15.37:0/1322959730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2630144ca0 msgr2=0x7f261c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263512b640 1 --2- 172.21.15.37:0/1322959730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2630144ca0 0x7f261c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263512b640 1 -- 172.21.15.37:0/1322959730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f261c004d50 msgr2=0x7f261c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263512b640 1 --2- 172.21.15.37:0/1322959730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f261c004d50 0x7f261c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.792+0000 7f263512b640 1 -- 172.21.15.37:0/1322959730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261c003530 con 0x7f261c004030 2024-04-25T08:58:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f263512b640 1 --2- 172.21.15.37:0/1322959730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f261c004030 0x7f261c004810 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f2620009ab0 tx=0x7f262001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad206eb1a43a176 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26200b2070 con 0x7f261c004030 2024-04-25T08:58:39.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26200a5680 con 0x7f261c004030 2024-04-25T08:58:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f263492a640 1 -- 172.21.15.37:0/1322959730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26200096b0 con 0x7f261c004030 2024-04-25T08:58:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f2637b90640 1 -- 172.21.15.37:0/1322959730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f261c004030 msgr2=0x7f261c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f2637b90640 1 --2- 172.21.15.37:0/1322959730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f261c004030 0x7f261c004810 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f2620009ab0 tx=0x7f262001ca90 comp rx=0 tx=0).stop 2024-04-25T08:58:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f263612d640 1 -- 172.21.15.37:0/1322959730 reap_dead start 2024-04-25T08:58:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f2637b90640 1 -- 172.21.15.37:0/1322959730 shutdown_connections 2024-04-25T08:58:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f2637b90640 1 -- 172.21.15.37:0/1322959730 >> 172.21.15.37:0/1322959730 conn(0x7f26300b11a0 msgr2=0x7f26300b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f2637b90640 1 -- 172.21.15.37:0/1322959730 shutdown_connections 2024-04-25T08:58:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f2637b90640 1 -- 172.21.15.37:0/1322959730 wait complete. 2024-04-25T08:58:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f2637b90640 1 Processor -- start 2024-04-25T08:58:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.796+0000 7f2637b90640 1 -- start start 2024-04-25T08:58:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f2637b90640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f261c004030 0x7f263013cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f2637b90640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2630143120 0x7f26301435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f2637b90640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26300b3470 0x7f26300b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f2637b90640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2630142d90 con 0x7f2630143120 2024-04-25T08:58:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f2637b90640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263006e290 con 0x7f26300b3470 2024-04-25T08:58:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f2637b90640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2630142f10 con 0x7f261c004030 2024-04-25T08:58:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263512b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2630143120 0x7f26301435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263512b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2630143120 0x7f26301435a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55468/0 (socket says 172.21.15.37:55468) 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263512b640 1 -- 172.21.15.37:0/461354613 learned_addr learned my addr 172.21.15.37:0/461354613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263612d640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26300b3470 0x7f26300b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263512b640 1 -- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f261c004030 msgr2=0x7f263013cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263592c640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f261c004030 0x7f263013cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263512b640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f261c004030 0x7f263013cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263512b640 1 -- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26300b3470 msgr2=0x7f26300b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263512b640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26300b3470 0x7f26300b38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263512b640 1 -- 172.21.15.37:0/461354613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f263013d440 con 0x7f2630143120 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263592c640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f261c004030 0x7f263013cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263512b640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2630143120 0x7f26301435a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f262001c8c0 tx=0x7f261c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f26267fc640 1 -- 172.21.15.37:0/461354613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26200b2040 con 0x7f2630143120 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f26267fc640 1 -- 172.21.15.37:0/461354613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26200bb070 con 0x7f2630143120 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f2637b90640 1 -- 172.21.15.37:0/461354613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26300b41b0 con 0x7f2630143120 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f26267fc640 1 -- 172.21.15.37:0/461354613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26200ad840 con 0x7f2630143120 2024-04-25T08:58:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f2637b90640 1 -- 172.21.15.37:0/461354613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26301e1210 con 0x7f2630143120 2024-04-25T08:58:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f2637b90640 1 -- 172.21.15.37:0/461354613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263004efa0 con 0x7f2630143120 2024-04-25T08:58:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f26267fc640 1 -- 172.21.15.37:0/461354613 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26200ad9e0 con 0x7f2630143120 2024-04-25T08:58:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f26267fc640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2614066f60 0x7f2614069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f26267fc640 1 -- 172.21.15.37:0/461354613 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f262011f350 con 0x7f2630143120 2024-04-25T08:58:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263592c640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2614066f60 0x7f2614069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.800+0000 7f263592c640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2614066f60 0x7f2614069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f262c007fd0 tx=0x7f262c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.804+0000 7f26267fc640 1 -- 172.21.15.37:0/461354613 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26200167a0 con 0x7f2630143120 2024-04-25T08:58:39.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.960+0000 7f2637b90640 1 -- 172.21.15.37:0/461354613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f26300b1cc0 con 0x7f2630143120 2024-04-25T08:58:39.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.960+0000 7f26267fc640 1 -- 172.21.15.37:0/461354613 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17038 (secure 0 0 0) 0x7f26200f0210 con 0x7f2630143120 2024-04-25T08:58:39.954 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:39.954 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":83,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:57:11.180680+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:56:49.737591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T08:56:50.436964+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/4130042812":"2024-04-26T08:57:11.180643+0000","172.21.15.37:6834/4130042812":"2024-04-26T08:57:11.180643+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:39.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 -- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2614066f60 msgr2=0x7f2614069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2614066f60 0x7f2614069420 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f262c007fd0 tx=0x7f262c009040 comp rx=0 tx=0).stop 2024-04-25T08:58:39.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 -- 172.21.15.37:0/461354613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2630143120 msgr2=0x7f26301435a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2630143120 0x7f26301435a0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f262001c8c0 tx=0x7f261c005840 comp rx=0 tx=0).stop 2024-04-25T08:58:39.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 -- 172.21.15.37:0/461354613 shutdown_connections 2024-04-25T08:58:39.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2614066f60 0x7f2614069420 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26300b3470 0x7f26300b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2630143120 0x7f26301435a0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 --2- 172.21.15.37:0/461354613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f261c004030 0x7f263013cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 -- 172.21.15.37:0/461354613 >> 172.21.15.37:0/461354613 conn(0x7f26300b11a0 msgr2=0x7f26300cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:39.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 -- 172.21.15.37:0/461354613 shutdown_connections 2024-04-25T08:58:39.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:39.964+0000 7f2637b90640 1 -- 172.21.15.37:0/461354613 wait complete. 2024-04-25T08:58:39.972 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:58:40.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55de890640 1 Processor -- start 2024-04-25T08:58:40.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55de890640 1 -- start start 2024-04-25T08:58:40.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55de890640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f55d80ce510 0x7f55d80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55de890640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f55d81451f0 0x7f55d8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55de890640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d80cee30 con 0x7f55d8144ca0 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55de890640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55d80cc2b0 con 0x7f55d81451f0 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55de890640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55d806e290 con 0x7f55d80ce510 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55dce2d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f55d81451f0 0x7f55d8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55d77fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f55d8144ca0 0x7f55d806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52782/0 (socket says 172.21.15.37:52782) 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55d77fe640 1 -- 172.21.15.37:0/2255981846 learned_addr learned my addr 172.21.15.37:0/2255981846 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55d7fff640 1 --2- 172.21.15.37:0/2255981846 >> v2:172.21.15.203:3300/0 conn(0x7f55d80ce510 0x7f55d80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55d6ffd640 1 -- 172.21.15.37:0/2255981846 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721187757 0 0) 0x7f55d80cee30 con 0x7f55d8144ca0 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55d6ffd640 1 -- 172.21.15.37:0/2255981846 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55c0003530 con 0x7f55d8144ca0 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55dce2d640 1 -- 172.21.15.37:0/2255981846 >> v2:172.21.15.203:3300/0 conn(0x7f55d80ce510 msgr2=0x7f55d80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55dce2d640 1 --2- 172.21.15.37:0/2255981846 >> v2:172.21.15.203:3300/0 conn(0x7f55d80ce510 0x7f55d80ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55dce2d640 1 -- 172.21.15.37:0/2255981846 >> v1:172.21.15.37:6789/0 conn(0x7f55d8144ca0 legacy=0x7f55d806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.092+0000 7f55dce2d640 1 -- 172.21.15.37:0/2255981846 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d80ccc70 con 0x7f55d81451f0 2024-04-25T08:58:40.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55dce2d640 1 --2- 172.21.15.37:0/2255981846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d81451f0 0x7f55d8142680 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f55d8144c50 tx=0x7f55d001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720e12e32bbd5bda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:40.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55d6ffd640 1 -- 172.21.15.37:0/2255981846 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d00b4070 con 0x7f55d81451f0 2024-04-25T08:58:40.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55d6ffd640 1 -- 172.21.15.37:0/2255981846 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d00af600 con 0x7f55d81451f0 2024-04-25T08:58:40.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55d6ffd640 1 -- 172.21.15.37:0/2255981846 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d00af9b0 con 0x7f55d81451f0 2024-04-25T08:58:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55de890640 1 -- 172.21.15.37:0/2255981846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d81451f0 msgr2=0x7f55d8142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55de890640 1 --2- 172.21.15.37:0/2255981846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d81451f0 0x7f55d8142680 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f55d8144c50 tx=0x7f55d001f110 comp rx=0 tx=0).stop 2024-04-25T08:58:40.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55de890640 1 -- 172.21.15.37:0/2255981846 shutdown_connections 2024-04-25T08:58:40.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55de890640 1 --2- 172.21.15.37:0/2255981846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d81451f0 0x7f55d8142680 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55de890640 1 --2- 172.21.15.37:0/2255981846 >> v2:172.21.15.203:3300/0 conn(0x7f55d80ce510 0x7f55d80ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55de890640 1 -- 172.21.15.37:0/2255981846 >> 172.21.15.37:0/2255981846 conn(0x7f55d80b11a0 msgr2=0x7f55d80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:40.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55de890640 1 -- 172.21.15.37:0/2255981846 shutdown_connections 2024-04-25T08:58:40.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55de890640 1 -- 172.21.15.37:0/2255981846 wait complete. 2024-04-25T08:58:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.096+0000 7f55de890640 1 Processor -- start 2024-04-25T08:58:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55de890640 1 -- start start 2024-04-25T08:58:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55de890640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d80ce510 0x7f55d815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55de890640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55d8144ca0 0x7f55d80af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55de890640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d81451f0 0x7f55d80ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55de890640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d806ee30 con 0x7f55d80ce510 2024-04-25T08:58:40.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55de890640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d806df20 con 0x7f55d81451f0 2024-04-25T08:58:40.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55de890640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d80cc900 con 0x7f55d8144ca0 2024-04-25T08:58:40.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d80ce510 0x7f55d815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d80ce510 0x7f55d815b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55482/0 (socket says 172.21.15.37:55482) 2024-04-25T08:58:40.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d7fff640 1 -- 172.21.15.37:0/218539079 learned_addr learned my addr 172.21.15.37:0/218539079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:40.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d77fe640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55d8144ca0 0x7f55d80af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55dce2d640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d81451f0 0x7f55d80ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d7fff640 1 -- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55d8144ca0 msgr2=0x7f55d80af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d7fff640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55d8144ca0 0x7f55d80af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d7fff640 1 -- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d81451f0 msgr2=0x7f55d80ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d7fff640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d81451f0 0x7f55d80ca4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d7fff640 1 -- 172.21.15.37:0/218539079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d80affb0 con 0x7f55d80ce510 2024-04-25T08:58:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d77fe640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55d8144ca0 0x7f55d80af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d7fff640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d80ce510 0x7f55d815b1c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f55c800e960 tx=0x7f55c800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d4ff9640 1 -- 172.21.15.37:0/218539079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8017070 con 0x7f55d80ce510 2024-04-25T08:58:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d4ff9640 1 -- 172.21.15.37:0/218539079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c800ca00 con 0x7f55d80ce510 2024-04-25T08:58:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55de890640 1 -- 172.21.15.37:0/218539079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d80b0290 con 0x7f55d80ce510 2024-04-25T08:58:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d4ff9640 1 -- 172.21.15.37:0/218539079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c800cdb0 con 0x7f55d80ce510 2024-04-25T08:58:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55de890640 1 -- 172.21.15.37:0/218539079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d80aa5a0 con 0x7f55d80ce510 2024-04-25T08:58:40.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55de890640 1 -- 172.21.15.37:0/218539079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d804efa0 con 0x7f55d80ce510 2024-04-25T08:58:40.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d4ff9640 1 -- 172.21.15.37:0/218539079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55c801a070 con 0x7f55d80ce510 2024-04-25T08:58:40.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d4ff9640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55c4066f60 0x7f55c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d4ff9640 1 -- 172.21.15.37:0/218539079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f55c80851c0 con 0x7f55d80ce510 2024-04-25T08:58:40.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.100+0000 7f55d77fe640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55c4066f60 0x7f55c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.104+0000 7f55d77fe640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55c4066f60 0x7f55c4069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f55d80ac7e0 tx=0x7f55cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:40.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.104+0000 7f55d4ff9640 1 -- 172.21.15.37:0/218539079 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f55c8056080 con 0x7f55d80ce510 2024-04-25T08:58:40.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.300+0000 7f55de890640 1 -- 172.21.15.37:0/218539079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f55d80b0460 con 0x7f55d80ce510 2024-04-25T08:58:40.292 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T08:58:40.300+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:58:40.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.304+0000 7f55d4ff9640 1 -- 172.21.15.37:0/218539079 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) v1 ==== 161+0+0 (secure 0 0 0) 0x7f55c8055980 con 0x7f55d80ce510 2024-04-25T08:58:40.295 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:58:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 -- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55c4066f60 msgr2=0x7f55c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55c4066f60 0x7f55c4069420 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f55d80ac7e0 tx=0x7f55cc009040 comp rx=0 tx=0).stop 2024-04-25T08:58:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 -- 172.21.15.37:0/218539079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d80ce510 msgr2=0x7f55d815b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d80ce510 0x7f55d815b1c0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f55c800e960 tx=0x7f55c800ee20 comp rx=0 tx=0).stop 2024-04-25T08:58:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 -- 172.21.15.37:0/218539079 shutdown_connections 2024-04-25T08:58:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55c4066f60 0x7f55c4069420 secure :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f55d80ac7e0 tx=0x7f55cc009040 comp rx=0 tx=0).stop 2024-04-25T08:58:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d81451f0 0x7f55d80ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55d8144ca0 0x7f55d80af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 --2- 172.21.15.37:0/218539079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d80ce510 0x7f55d815b1c0 secure :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f55c800e960 tx=0x7f55c800ee20 comp rx=0 tx=0).stop 2024-04-25T08:58:40.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 -- 172.21.15.37:0/218539079 >> 172.21.15.37:0/218539079 conn(0x7f55d80b11a0 msgr2=0x7f55d8071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 -- 172.21.15.37:0/218539079 shutdown_connections 2024-04-25T08:58:40.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.308+0000 7f55de890640 1 -- 172.21.15.37:0/218539079 wait complete. 2024-04-25T08:58:40.310 INFO:tasks.ceph.mds.c.smithi037.stderr: -4375> 2024-04-25T08:57:18.201+0000 7efd5f9cb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:58:40.318 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:58:40.358 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T08:58:40.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac4d592640 1 Processor -- start 2024-04-25T08:58:40.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac4d592640 1 -- start start 2024-04-25T08:58:40.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac4d592640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fac4815b6b0 0x7fac480ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac4d592640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fac480cf590 0x7fac480cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac4d592640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac480cbd20 con 0x7fac4815f130 2024-04-25T08:58:40.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac4d592640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac4815ba90 con 0x7fac4815b6b0 2024-04-25T08:58:40.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac4d592640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac480cc000 con 0x7fac480cf590 2024-04-25T08:58:40.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac46ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fac4815b6b0 0x7fac480ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac46ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fac4815b6b0 0x7fac480ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55510/0 (socket says 172.21.15.37:55510) 2024-04-25T08:58:40.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac46ffd640 1 -- 172.21.15.37:0/324660611 learned_addr learned my addr 172.21.15.37:0/324660611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:40.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac46ffd640 1 -- 172.21.15.37:0/324660611 >> v2:172.21.15.203:3300/0 conn(0x7fac480cf590 msgr2=0x7fac480cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:40.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac46ffd640 1 --2- 172.21.15.37:0/324660611 >> v2:172.21.15.203:3300/0 conn(0x7fac480cf590 0x7fac480cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac46ffd640 1 -- 172.21.15.37:0/324660611 >> v1:172.21.15.203:6789/0 conn(0x7fac4815f130 legacy=0x7fac4815f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:40.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.444+0000 7fac46ffd640 1 -- 172.21.15.37:0/324660611 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac480cc2e0 con 0x7fac4815b6b0 2024-04-25T08:58:40.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac46ffd640 1 --2- 172.21.15.37:0/324660611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4815b6b0 0x7fac480ca3c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fac480b4430 tx=0x7fac3801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=288dab6cb7acbfc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:40.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac45ffb640 1 -- 172.21.15.37:0/324660611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac380b1070 con 0x7fac4815b6b0 2024-04-25T08:58:40.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac45ffb640 1 -- 172.21.15.37:0/324660611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac380ac630 con 0x7fac4815b6b0 2024-04-25T08:58:40.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac45ffb640 1 -- 172.21.15.37:0/324660611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac380a8050 con 0x7fac4815b6b0 2024-04-25T08:58:40.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac4d592640 1 -- 172.21.15.37:0/324660611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4815b6b0 msgr2=0x7fac480ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac4d592640 1 --2- 172.21.15.37:0/324660611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4815b6b0 0x7fac480ca3c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fac480b4430 tx=0x7fac3801c4e0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac4d592640 1 -- 172.21.15.37:0/324660611 shutdown_connections 2024-04-25T08:58:40.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac4d592640 1 --2- 172.21.15.37:0/324660611 >> v2:172.21.15.203:3300/0 conn(0x7fac480cf590 0x7fac480cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac4d592640 1 --2- 172.21.15.37:0/324660611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4815b6b0 0x7fac480ca3c0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac4d592640 1 -- 172.21.15.37:0/324660611 >> 172.21.15.37:0/324660611 conn(0x7fac4806ae60 msgr2=0x7fac4806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:40.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac4d592640 1 -- 172.21.15.37:0/324660611 shutdown_connections 2024-04-25T08:58:40.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac4d592640 1 -- 172.21.15.37:0/324660611 wait complete. 2024-04-25T08:58:40.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac4d592640 1 Processor -- start 2024-04-25T08:58:40.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.448+0000 7fac4d592640 1 -- start start 2024-04-25T08:58:40.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac4d592640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac480cf590 0x7fac480c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac4d592640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4815b6b0 0x7fac4815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac4d592640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4815f130 0x7fac4815e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac4d592640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac480cadb0 con 0x7fac4815b6b0 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac4d592640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac480ca900 con 0x7fac480cf590 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac4d592640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac480caf30 con 0x7fac4815f130 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac467fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4815b6b0 0x7fac4815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac467fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4815b6b0 0x7fac4815e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55526/0 (socket says 172.21.15.37:55526) 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac467fc640 1 -- 172.21.15.37:0/2147949309 learned_addr learned my addr 172.21.15.37:0/2147949309 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac477fe640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4815f130 0x7fac4815e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac46ffd640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac480cf590 0x7fac480c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac467fc640 1 -- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4815f130 msgr2=0x7fac4815e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac467fc640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4815f130 0x7fac4815e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac467fc640 1 -- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac480cf590 msgr2=0x7fac480c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac467fc640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac480cf590 0x7fac480c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac467fc640 1 -- 172.21.15.37:0/2147949309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac480c69a0 con 0x7fac4815b6b0 2024-04-25T08:58:40.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac46ffd640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac480cf590 0x7fac480c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:40.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac467fc640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4815b6b0 0x7fac4815e110 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00b580 tx=0x7fac3c00ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:40.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac2f7fe640 1 -- 172.21.15.37:0/2147949309 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c014070 con 0x7fac4815b6b0 2024-04-25T08:58:40.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac2f7fe640 1 -- 172.21.15.37:0/2147949309 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3c010890 con 0x7fac4815b6b0 2024-04-25T08:58:40.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac2f7fe640 1 -- 172.21.15.37:0/2147949309 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c010c80 con 0x7fac4815b6b0 2024-04-25T08:58:40.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac4d592640 1 -- 172.21.15.37:0/2147949309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac480c8da0 con 0x7fac4815b6b0 2024-04-25T08:58:40.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac4d592640 1 -- 172.21.15.37:0/2147949309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac480c9360 con 0x7fac4815b6b0 2024-04-25T08:58:40.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac4d592640 1 -- 172.21.15.37:0/2147949309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac4804efa0 con 0x7fac4815b6b0 2024-04-25T08:58:40.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac2f7fe640 1 -- 172.21.15.37:0/2147949309 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fac3c017070 con 0x7fac4815b6b0 2024-04-25T08:58:40.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac2f7fe640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac34066f60 0x7fac34069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac2f7fe640 1 -- 172.21.15.37:0/2147949309 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fac3c085050 con 0x7fac4815b6b0 2024-04-25T08:58:40.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.452+0000 7fac46ffd640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac34066f60 0x7fac34069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.456+0000 7fac46ffd640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac34066f60 0x7fac34069420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fac380ae000 tx=0x7fac38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:40.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.456+0000 7fac2f7fe640 1 -- 172.21.15.37:0/2147949309 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fac3c055e10 con 0x7fac4815b6b0 2024-04-25T08:58:40.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:40.656+0000 7fac4d592640 1 -- 172.21.15.37:0/2147949309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fac48166070 con 0x7fac4815b6b0 2024-04-25T08:58:41.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.384+0000 7fac2f7fe640 1 -- 172.21.15.37:0/2147949309 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 104+0+0 (secure 0 0 0) 0x7fac3c010e20 con 0x7fac4815b6b0 2024-04-25T08:58:41.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 -- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac34066f60 msgr2=0x7fac34069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac34066f60 0x7fac34069420 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fac380ae000 tx=0x7fac38008040 comp rx=0 tx=0).stop 2024-04-25T08:58:41.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 -- 172.21.15.37:0/2147949309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4815b6b0 msgr2=0x7fac4815e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4815b6b0 0x7fac4815e110 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00b580 tx=0x7fac3c00ba40 comp rx=0 tx=0).stop 2024-04-25T08:58:41.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 -- 172.21.15.37:0/2147949309 shutdown_connections 2024-04-25T08:58:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fac34066f60 0x7fac34069420 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fac4815f130 0x7fac4815e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac4815b6b0 0x7fac4815e110 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 --2- 172.21.15.37:0/2147949309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fac480cf590 0x7fac480c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 -- 172.21.15.37:0/2147949309 >> 172.21.15.37:0/2147949309 conn(0x7fac4806ae60 msgr2=0x7fac480b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 -- 172.21.15.37:0/2147949309 shutdown_connections 2024-04-25T08:58:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.392+0000 7fac4d592640 1 -- 172.21.15.37:0/2147949309 wait complete. 2024-04-25T08:58:41.405 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:58:41.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.588+0000 7f011f38e640 1 Processor -- start 2024-04-25T08:58:41.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.588+0000 7f011f38e640 1 -- start start 2024-04-25T08:58:41.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011f38e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f011815d980 0x7f01180d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011f38e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f01180d12c0 0x7f01180d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011f38e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011815dd60 con 0x7f01180d21f0 2024-04-25T08:58:41.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011f38e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01180d2650 con 0x7f01180d12c0 2024-04-25T08:58:41.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011f38e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011806c810 con 0x7f011815d980 2024-04-25T08:58:41.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011d92b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f01180d21f0 0x7f01180ce300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52796/0 (socket says 172.21.15.37:52796) 2024-04-25T08:58:41.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011d92b640 1 -- 172.21.15.37:0/1151981745 learned_addr learned my addr 172.21.15.37:0/1151981745 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:41.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011c929640 1 --2- 172.21.15.37:0/1151981745 >> v2:172.21.15.203:3300/0 conn(0x7f01180d12c0 0x7f01180d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011d12a640 1 --2- 172.21.15.37:0/1151981745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f011815d980 0x7f01180d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753999128 0 0) 0x7f011815dd60 con 0x7f01180d21f0 2024-04-25T08:58:41.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0104003530 con 0x7f01180d21f0 2024-04-25T08:58:41.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1121834792 0 0) 0x7f0104003530 con 0x7f01180d21f0 2024-04-25T08:58:41.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f011815d980 msgr2=0x7f01180d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 --2- 172.21.15.37:0/1151981745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f011815d980 0x7f01180d3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 >> v2:172.21.15.203:3300/0 conn(0x7f01180d12c0 msgr2=0x7f01180d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 --2- 172.21.15.37:0/1151981745 >> v2:172.21.15.203:3300/0 conn(0x7f01180d12c0 0x7f01180d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01180d4340 con 0x7f01180d21f0 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0110001a00 con 0x7f01180d21f0 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 >> v1:172.21.15.37:6789/0 conn(0x7f01180d21f0 legacy=0x7f01180ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 --2- 172.21.15.37:0/1151981745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01040039c0 0x7f0104003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 --2- 172.21.15.37:0/1151981745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f011815d980 0x7f0104004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 --2- 172.21.15.37:0/1151981745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 0x7f01040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011806c810 con 0x7f0104004d50 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010400b9d0 con 0x7f01040039c0 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f010400bb50 con 0x7f011815d980 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011d92b640 1 --2- 172.21.15.37:0/1151981745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 0x7f01040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011c929640 1 --2- 172.21.15.37:0/1151981745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f011815d980 0x7f0104004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011d92b640 1 -- 172.21.15.37:0/1151981745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f011815d980 msgr2=0x7f0104004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:41.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011d92b640 1 --2- 172.21.15.37:0/1151981745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f011815d980 0x7f0104004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011d92b640 1 -- 172.21.15.37:0/1151981745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01040039c0 msgr2=0x7f0104003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:41.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011d92b640 1 --2- 172.21.15.37:0/1151981745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f01040039c0 0x7f0104003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.592+0000 7f011d92b640 1 -- 172.21.15.37:0/1151981745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0104003530 con 0x7f0104004d50 2024-04-25T08:58:41.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011d92b640 1 --2- 172.21.15.37:0/1151981745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 0x7f01040051d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f011000a920 tx=0x7f011001fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cde3db2d1d08c5b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:41.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011000a4c0 con 0x7f0104004d50 2024-04-25T08:58:41.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01100ad020 con 0x7f0104004d50 2024-04-25T08:58:41.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f010ffff640 1 -- 172.21.15.37:0/1151981745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01100a9750 con 0x7f0104004d50 2024-04-25T08:58:41.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 -- 172.21.15.37:0/1151981745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 msgr2=0x7f01040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 --2- 172.21.15.37:0/1151981745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 0x7f01040051d0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f011000a920 tx=0x7f011001fba0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011d92b640 1 -- 172.21.15.37:0/1151981745 reap_dead start 2024-04-25T08:58:41.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 -- 172.21.15.37:0/1151981745 shutdown_connections 2024-04-25T08:58:41.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 -- 172.21.15.37:0/1151981745 >> 172.21.15.37:0/1151981745 conn(0x7f011808b000 msgr2=0x7f01180d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:41.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 -- 172.21.15.37:0/1151981745 shutdown_connections 2024-04-25T08:58:41.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 -- 172.21.15.37:0/1151981745 wait complete. 2024-04-25T08:58:41.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 Processor -- start 2024-04-25T08:58:41.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 -- start start 2024-04-25T08:58:41.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 0x7f011809e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0118159540 0x7f01180a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f011806d020 0x7f011806d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011806bee0 con 0x7f0104004d50 2024-04-25T08:58:41.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01180b02c0 con 0x7f011806d020 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.596+0000 7f011f38e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011806c1c0 con 0x7f0118159540 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011d12a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 0x7f011809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011d12a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 0x7f011809e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55544/0 (socket says 172.21.15.37:55544) 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011d12a640 1 -- 172.21.15.37:0/557146969 learned_addr learned my addr 172.21.15.37:0/557146969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011d12a640 1 -- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0118159540 msgr2=0x7f01180a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011d12a640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0118159540 0x7f01180a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011d12a640 1 -- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f011806d020 msgr2=0x7f011806d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011d12a640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f011806d020 0x7f011806d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011d12a640 1 -- 172.21.15.37:0/557146969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01180a0c70 con 0x7f0104004d50 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011d12a640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 0x7f011809e080 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f011400e7f0 tx=0x7f011400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f010dffb640 1 -- 172.21.15.37:0/557146969 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0114017070 con 0x7f0104004d50 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f010dffb640 1 -- 172.21.15.37:0/557146969 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f011400c860 con 0x7f0104004d50 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011f38e640 1 -- 172.21.15.37:0/557146969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01180a0f50 con 0x7f0104004d50 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f010dffb640 1 -- 172.21.15.37:0/557146969 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011400eea0 con 0x7f0104004d50 2024-04-25T08:58:41.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011f38e640 1 -- 172.21.15.37:0/557146969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01180c86f0 con 0x7f0104004d50 2024-04-25T08:58:41.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011f38e640 1 -- 172.21.15.37:0/557146969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0118067c30 con 0x7f0104004d50 2024-04-25T08:58:41.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f010dffb640 1 -- 172.21.15.37:0/557146969 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f011401a070 con 0x7f0104004d50 2024-04-25T08:58:41.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f010dffb640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00fc066f60 0x7f00fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f010dffb640 1 -- 172.21.15.37:0/557146969 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f01140853c0 con 0x7f0104004d50 2024-04-25T08:58:41.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011c929640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00fc066f60 0x7f00fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.600+0000 7f011c929640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00fc066f60 0x7f00fc069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0118160d80 tx=0x7f0108008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:41.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.604+0000 7f010dffb640 1 -- 172.21.15.37:0/557146969 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0114056180 con 0x7f0104004d50 2024-04-25T08:58:41.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:41.764+0000 7f011f38e640 1 -- 172.21.15.37:0/557146969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01180d3e20 con 0x7f0104004d50 2024-04-25T08:58:42.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.404+0000 7f010dffb640 1 -- 172.21.15.37:0/557146969 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0114055a80 con 0x7f0104004d50 2024-04-25T08:58:42.397 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:58:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.408+0000 7f011f38e640 1 -- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00fc066f60 msgr2=0x7f00fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.408+0000 7f011f38e640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00fc066f60 0x7f00fc069420 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0118160d80 tx=0x7f0108008040 comp rx=0 tx=0).stop 2024-04-25T08:58:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.408+0000 7f011f38e640 1 -- 172.21.15.37:0/557146969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 msgr2=0x7f011809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.408+0000 7f011f38e640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 0x7f011809e080 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f011400e7f0 tx=0x7f011400ecb0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.408+0000 7f011f38e640 1 -- 172.21.15.37:0/557146969 shutdown_connections 2024-04-25T08:58:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.408+0000 7f011f38e640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00fc066f60 0x7f00fc069420 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.408+0000 7f011f38e640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f011806d020 0x7f011806d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.408+0000 7f011f38e640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0118159540 0x7f01180a03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.408+0000 7f011f38e640 1 --2- 172.21.15.37:0/557146969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0104004d50 0x7f011809e080 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.408+0000 7f011f38e640 1 -- 172.21.15.37:0/557146969 >> 172.21.15.37:0/557146969 conn(0x7f011808b000 msgr2=0x7f0118160050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:42.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.412+0000 7f011f38e640 1 -- 172.21.15.37:0/557146969 shutdown_connections 2024-04-25T08:58:42.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.412+0000 7f011f38e640 1 -- 172.21.15.37:0/557146969 wait complete. 2024-04-25T08:58:42.425 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:58:42.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b9d759640 1 Processor -- start 2024-04-25T08:58:42.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b9d759640 1 -- start start 2024-04-25T08:58:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b9d759640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815a3c0 0x7f1b980ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b9d759640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1b9815ac00 0x7f1b980cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b9d759640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9815e0f0 con 0x7f1b9815a7a0 2024-04-25T08:58:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b9d759640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b980b2eb0 con 0x7f1b9815ac00 2024-04-25T08:58:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b9d759640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980b24a0 con 0x7f1b9815a3c0 2024-04-25T08:58:42.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b96ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815a3c0 0x7f1b980ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:42.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b977fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1b9815ac00 0x7f1b980cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:42.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b96ffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815a3c0 0x7f1b980ae6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:43974/0 (socket says 172.21.15.37:43974) 2024-04-25T08:58:42.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b96ffd640 1 -- 172.21.15.37:0/4239252966 learned_addr learned my addr 172.21.15.37:0/4239252966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:42.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b8ffff640 1 --1- 172.21.15.37:0/4239252966 >> v1:172.21.15.203:6789/0 conn(0x7f1b9815a7a0 0x7f1b980b27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:42374/0 (socket says 172.21.15.37:42374) 2024-04-25T08:58:42.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b96ffd640 1 -- 172.21.15.37:0/4239252966 >> v2:172.21.15.203:3300/0 conn(0x7f1b9815ac00 msgr2=0x7f1b980cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:42.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b96ffd640 1 --2- 172.21.15.37:0/4239252966 >> v2:172.21.15.203:3300/0 conn(0x7f1b9815ac00 0x7f1b980cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b96ffd640 1 -- 172.21.15.37:0/4239252966 >> v1:172.21.15.203:6789/0 conn(0x7f1b9815a7a0 legacy=0x7f1b980b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.556+0000 7f1b96ffd640 1 -- 172.21.15.37:0/4239252966 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9815b0b0 con 0x7f1b9815a3c0 2024-04-25T08:58:42.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b96ffd640 1 --2- 172.21.15.37:0/4239252966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815a3c0 0x7f1b980ae6c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009870 tx=0x7f1b8801c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8140d4f609b9c13c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:42.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b967fc640 1 -- 172.21.15.37:0/4239252966 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b880ab640 con 0x7f1b9815a3c0 2024-04-25T08:58:42.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b967fc640 1 -- 172.21.15.37:0/4239252966 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b880b0070 con 0x7f1b9815a3c0 2024-04-25T08:58:42.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b967fc640 1 -- 172.21.15.37:0/4239252966 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b880aba00 con 0x7f1b9815a3c0 2024-04-25T08:58:42.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b9d759640 1 -- 172.21.15.37:0/4239252966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815a3c0 msgr2=0x7f1b980ae6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b9d759640 1 --2- 172.21.15.37:0/4239252966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815a3c0 0x7f1b980ae6c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009870 tx=0x7f1b8801c530 comp rx=0 tx=0).stop 2024-04-25T08:58:42.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b9d759640 1 -- 172.21.15.37:0/4239252966 shutdown_connections 2024-04-25T08:58:42.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b9d759640 1 --2- 172.21.15.37:0/4239252966 >> v2:172.21.15.203:3300/0 conn(0x7f1b9815ac00 0x7f1b980cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b9d759640 1 --2- 172.21.15.37:0/4239252966 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815a3c0 0x7f1b980ae6c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b9d759640 1 -- 172.21.15.37:0/4239252966 >> 172.21.15.37:0/4239252966 conn(0x7f1b9806ae60 msgr2=0x7f1b9806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:42.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b9d759640 1 -- 172.21.15.37:0/4239252966 shutdown_connections 2024-04-25T08:58:42.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b9d759640 1 -- 172.21.15.37:0/4239252966 wait complete. 2024-04-25T08:58:42.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b9d759640 1 Processor -- start 2024-04-25T08:58:42.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.560+0000 7f1b9d759640 1 -- start start 2024-04-25T08:58:42.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b9d759640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b9815a3c0 0x7f1b981f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:42.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b9d759640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b9815a7a0 0x7f1b981560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:42.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b9d759640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815ac00 0x7f1b98070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b9d759640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980cbee0 con 0x7f1b9815a7a0 2024-04-25T08:58:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b9d759640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980cbd60 con 0x7f1b9815a3c0 2024-04-25T08:58:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b9d759640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b980cc060 con 0x7f1b9815ac00 2024-04-25T08:58:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b9815a7a0 0x7f1b981560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:42.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b9815a7a0 0x7f1b981560e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55560/0 (socket says 172.21.15.37:55560) 2024-04-25T08:58:42.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8ffff640 1 -- 172.21.15.37:0/324259475 learned_addr learned my addr 172.21.15.37:0/324259475 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:42.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b96ffd640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b9815a3c0 0x7f1b981f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:42.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b977fe640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815ac00 0x7f1b98070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:42.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8ffff640 1 -- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815ac00 msgr2=0x7f1b98070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8ffff640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815ac00 0x7f1b98070e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8ffff640 1 -- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b9815a3c0 msgr2=0x7f1b981f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8ffff640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b9815a3c0 0x7f1b981f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8ffff640 1 -- 172.21.15.37:0/324259475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9809cfd0 con 0x7f1b9815a7a0 2024-04-25T08:58:42.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8ffff640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b9815a7a0 0x7f1b981560e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c450 tx=0x7f1b8000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:42.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8f7fe640 1 -- 172.21.15.37:0/324259475 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80017070 con 0x7f1b9815a7a0 2024-04-25T08:58:42.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8f7fe640 1 -- 172.21.15.37:0/324259475 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b80006480 con 0x7f1b9815a7a0 2024-04-25T08:58:42.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b9d759640 1 -- 172.21.15.37:0/324259475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b981f7ba0 con 0x7f1b9815a7a0 2024-04-25T08:58:42.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8f7fe640 1 -- 172.21.15.37:0/324259475 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b800049f0 con 0x7f1b9815a7a0 2024-04-25T08:58:42.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b9d759640 1 -- 172.21.15.37:0/324259475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b98072340 con 0x7f1b9815a7a0 2024-04-25T08:58:42.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b9d759640 1 -- 172.21.15.37:0/324259475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b98067c30 con 0x7f1b9815a7a0 2024-04-25T08:58:42.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8f7fe640 1 -- 172.21.15.37:0/324259475 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1b8001a070 con 0x7f1b9815a7a0 2024-04-25T08:58:42.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8f7fe640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b78066f60 0x7f1b78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:42.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.564+0000 7f1b8f7fe640 1 -- 172.21.15.37:0/324259475 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f1b800842f0 con 0x7f1b9815a7a0 2024-04-25T08:58:42.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.568+0000 7f1b96ffd640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b78066f60 0x7f1b78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:42.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.568+0000 7f1b96ffd640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b78066f60 0x7f1b78069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1b880af000 tx=0x7f1b880a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:42.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.572+0000 7f1b8f7fe640 1 -- 172.21.15.37:0/324259475 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1b981f7ba0 con 0x7f1b9815a7a0 2024-04-25T08:58:42.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:42.732+0000 7f1b9d759640 1 -- 172.21.15.37:0/324259475 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1b980b0170 con 0x7f1b9815a7a0 2024-04-25T08:58:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.416+0000 7f1b8f7fe640 1 -- 172.21.15.37:0/324259475 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1b9809cfd0 con 0x7f1b9815a7a0 2024-04-25T08:58:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T08:58:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.420+0000 7f1b9d759640 1 -- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b78066f60 msgr2=0x7f1b78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.420+0000 7f1b9d759640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b78066f60 0x7f1b78069420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1b880af000 tx=0x7f1b880a7000 comp rx=0 tx=0).stop 2024-04-25T08:58:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.420+0000 7f1b9d759640 1 -- 172.21.15.37:0/324259475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b9815a7a0 msgr2=0x7f1b981560e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.420+0000 7f1b9d759640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b9815a7a0 0x7f1b981560e0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000c450 tx=0x7f1b8000c910 comp rx=0 tx=0).stop 2024-04-25T08:58:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.420+0000 7f1b9d759640 1 -- 172.21.15.37:0/324259475 shutdown_connections 2024-04-25T08:58:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.420+0000 7f1b9d759640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1b78066f60 0x7f1b78069420 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.420+0000 7f1b9d759640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1b9815ac00 0x7f1b98070e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.424+0000 7f1b9d759640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b9815a7a0 0x7f1b981560e0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.424+0000 7f1b9d759640 1 --2- 172.21.15.37:0/324259475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1b9815a3c0 0x7f1b981f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.424+0000 7f1b9d759640 1 -- 172.21.15.37:0/324259475 >> 172.21.15.37:0/324259475 conn(0x7f1b9806ae60 msgr2=0x7f1b980aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:43.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.424+0000 7f1b9d759640 1 -- 172.21.15.37:0/324259475 shutdown_connections 2024-04-25T08:58:43.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.424+0000 7f1b9d759640 1 -- 172.21.15.37:0/324259475 wait complete. 2024-04-25T08:58:43.438 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:58:43.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c99b9640 1 Processor -- start 2024-04-25T08:58:43.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c99b9640 1 -- start start 2024-04-25T08:58:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c99b9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f20c40d0950 0x7f20c4168ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c99b9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c40cf7a0 con 0x7f20c41611e0 2024-04-25T08:58:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c99b9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c40b2940 con 0x7f20c406bfb0 2024-04-25T08:58:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c99b9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c406c410 con 0x7f20c40d0950 2024-04-25T08:58:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c2ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f20c41611e0 0x7f20c40cbc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52808/0 (socket says 172.21.15.37:52808) 2024-04-25T08:58:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c2ffd640 1 -- 172.21.15.37:0/3484156512 learned_addr learned my addr 172.21.15.37:0/3484156512 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009445552 0 0) 0x7f20c40cf7a0 con 0x7f20c41611e0 2024-04-25T08:58:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c37fe640 1 --2- 172.21.15.37:0/3484156512 >> v2:172.21.15.203:3300/0 conn(0x7f20c40d0950 0x7f20c4168ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20ac003530 con 0x7f20c41611e0 2024-04-25T08:58:43.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 988407506 0 0) 0x7f20ac003530 con 0x7f20c41611e0 2024-04-25T08:58:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 >> v2:172.21.15.203:3300/0 conn(0x7f20c40d0950 msgr2=0x7f20c4168ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 --2- 172.21.15.37:0/3484156512 >> v2:172.21.15.203:3300/0 conn(0x7f20c40d0950 0x7f20c4168ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 >> v1:172.21.15.203:6789/0 conn(0x7f20c406bfb0 legacy=0x7f20c40d0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40b2c20 con 0x7f20c41611e0 2024-04-25T08:58:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f20b4001680 con 0x7f20c41611e0 2024-04-25T08:58:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 >> v1:172.21.15.37:6789/0 conn(0x7f20c41611e0 legacy=0x7f20c40cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 --2- 172.21.15.37:0/3484156512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20ac003a30 0x7f20ac003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 --2- 172.21.15.37:0/3484156512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20ac004810 0x7f20ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 --2- 172.21.15.37:0/3484156512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20ac005240 0x7f20ac00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c406c410 con 0x7f20ac003a30 2024-04-25T08:58:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac00c1a0 con 0x7f20ac004810 2024-04-25T08:58:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ac00c3a0 con 0x7f20ac005240 2024-04-25T08:58:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c2ffd640 1 --2- 172.21.15.37:0/3484156512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20ac003a30 0x7f20ac003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c37fe640 1 --2- 172.21.15.37:0/3484156512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20ac005240 0x7f20ac00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c2ffd640 1 -- 172.21.15.37:0/3484156512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20ac005240 msgr2=0x7f20ac00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c2ffd640 1 --2- 172.21.15.37:0/3484156512 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20ac005240 0x7f20ac00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c2ffd640 1 -- 172.21.15.37:0/3484156512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20ac004810 msgr2=0x7f20ac004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c2ffd640 1 --2- 172.21.15.37:0/3484156512 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20ac004810 0x7f20ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c2ffd640 1 -- 172.21.15.37:0/3484156512 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ac003530 con 0x7f20ac003a30 2024-04-25T08:58:43.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.580+0000 7f20c37fe640 1 -- 172.21.15.37:0/3484156512 reap_dead start 2024-04-25T08:58:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.584+0000 7f20c2ffd640 1 --2- 172.21.15.37:0/3484156512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20ac003a30 0x7f20ac003e50 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f20b400c240 tx=0x7f20b401cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b1c03658bf374d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.584+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4007d20 con 0x7f20ac003a30 2024-04-25T08:58:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.584+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b40a9070 con 0x7f20ac003a30 2024-04-25T08:58:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.584+0000 7f20c1ffb640 1 -- 172.21.15.37:0/3484156512 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b4005da0 con 0x7f20ac003a30 2024-04-25T08:58:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.584+0000 7f20c99b9640 1 -- 172.21.15.37:0/3484156512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20ac003a30 msgr2=0x7f20ac003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.584+0000 7f20c99b9640 1 --2- 172.21.15.37:0/3484156512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20ac003a30 0x7f20ac003e50 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f20b400c240 tx=0x7f20b401cb40 comp rx=0 tx=0).stop 2024-04-25T08:58:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.584+0000 7f20c99b9640 1 -- 172.21.15.37:0/3484156512 shutdown_connections 2024-04-25T08:58:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.584+0000 7f20c99b9640 1 --2- 172.21.15.37:0/3484156512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20ac003a30 0x7f20ac003e50 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.584+0000 7f20c99b9640 1 -- 172.21.15.37:0/3484156512 >> 172.21.15.37:0/3484156512 conn(0x7f20c40b3100 msgr2=0x7f20c40b3510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:43.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.584+0000 7f20c99b9640 1 -- 172.21.15.37:0/3484156512 shutdown_connections 2024-04-25T08:58:43.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.584+0000 7f20c99b9640 1 -- 172.21.15.37:0/3484156512 wait complete. 2024-04-25T08:58:43.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c99b9640 1 Processor -- start 2024-04-25T08:58:43.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c99b9640 1 -- start start 2024-04-25T08:58:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c99b9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20ac003a30 0x7f20c40c9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c99b9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b6ce0 0x7f20c40c5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c99b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c40c6950 0x7f20c40c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c99b9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c4099a70 con 0x7f20c40c6950 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c99b9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c40b26f0 con 0x7f20c40b6ce0 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c99b9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c4099bf0 con 0x7f20ac003a30 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c40c6950 0x7f20c40c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c40c6950 0x7f20c40c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55586/0 (socket says 172.21.15.37:55586) 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c37fe640 1 -- 172.21.15.37:0/1156303773 learned_addr learned my addr 172.21.15.37:0/1156303773 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c27fc640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b6ce0 0x7f20c40c5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c37fe640 1 -- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20ac003a30 msgr2=0x7f20c40c9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c37fe640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20ac003a30 0x7f20c40c9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c37fe640 1 -- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b6ce0 msgr2=0x7f20c40c5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c37fe640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b6ce0 0x7f20c40c5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c37fe640 1 -- 172.21.15.37:0/1156303773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40ca0e0 con 0x7f20c40c6950 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c27fc640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b6ce0 0x7f20c40c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c37fe640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c40c6950 0x7f20c40c6de0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f20c406f1e0 tx=0x7f20bc0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20abfff640 1 -- 172.21.15.37:0/1156303773 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc011e40 con 0x7f20c40c6950 2024-04-25T08:58:43.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20abfff640 1 -- 172.21.15.37:0/1156303773 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc01b070 con 0x7f20c40c6950 2024-04-25T08:58:43.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c99b9640 1 -- 172.21.15.37:0/1156303773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c40ca3c0 con 0x7f20c40c6950 2024-04-25T08:58:43.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20abfff640 1 -- 172.21.15.37:0/1156303773 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc0039a0 con 0x7f20c40c6950 2024-04-25T08:58:43.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c99b9640 1 -- 172.21.15.37:0/1156303773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c40ca980 con 0x7f20c40c6950 2024-04-25T08:58:43.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20c99b9640 1 -- 172.21.15.37:0/1156303773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c404efa0 con 0x7f20c40c6950 2024-04-25T08:58:43.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20abfff640 1 -- 172.21.15.37:0/1156303773 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20bc003b40 con 0x7f20c40c6950 2024-04-25T08:58:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20abfff640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20b0066f60 0x7f20b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.588+0000 7f20abfff640 1 -- 172.21.15.37:0/1156303773 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f20bc085710 con 0x7f20c40c6950 2024-04-25T08:58:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.592+0000 7f20c2ffd640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20b0066f60 0x7f20b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.592+0000 7f20c2ffd640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20b0066f60 0x7f20b0069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f20b40ab000 tx=0x7f20c40cf550 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.592+0000 7f20abfff640 1 -- 172.21.15.37:0/1156303773 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20bc056840 con 0x7f20c40c6950 2024-04-25T08:58:43.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.752+0000 7f20c99b9640 1 -- 172.21.15.37:0/1156303773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f20c406c590 con 0x7f20c40c6950 2024-04-25T08:58:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.756+0000 7f20abfff640 1 -- 172.21.15.37:0/1156303773 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f20bc05c0a0 con 0x7f20c40c6950 2024-04-25T08:58:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.760+0000 7f20c99b9640 1 -- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20b0066f60 msgr2=0x7f20b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.760+0000 7f20c99b9640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20b0066f60 0x7f20b0069420 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f20b40ab000 tx=0x7f20c40cf550 comp rx=0 tx=0).stop 2024-04-25T08:58:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.760+0000 7f20c99b9640 1 -- 172.21.15.37:0/1156303773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c40c6950 msgr2=0x7f20c40c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.760+0000 7f20c99b9640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c40c6950 0x7f20c40c6de0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f20c406f1e0 tx=0x7f20bc0115a0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.760+0000 7f20c99b9640 1 -- 172.21.15.37:0/1156303773 shutdown_connections 2024-04-25T08:58:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.760+0000 7f20c99b9640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20b0066f60 0x7f20b0069420 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.760+0000 7f20c99b9640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20c40c6950 0x7f20c40c6de0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.760+0000 7f20c99b9640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20c40b6ce0 0x7f20c40c5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.760+0000 7f20c99b9640 1 --2- 172.21.15.37:0/1156303773 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20ac003a30 0x7f20c40c9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.760+0000 7f20c99b9640 1 -- 172.21.15.37:0/1156303773 >> 172.21.15.37:0/1156303773 conn(0x7f20c40b3100 msgr2=0x7f20c40cd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:43.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.764+0000 7f20c99b9640 1 -- 172.21.15.37:0/1156303773 shutdown_connections 2024-04-25T08:58:43.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.764+0000 7f20c99b9640 1 -- 172.21.15.37:0/1156303773 wait complete. 2024-04-25T08:58:43.770 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-04-25T08:58:43.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb35125640 1 Processor -- start 2024-04-25T08:58:43.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb35125640 1 -- start start 2024-04-25T08:58:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb35125640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdb3015ecc0 0x7fdb3006e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb35125640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb300b2090 con 0x7fdb3015a700 2024-04-25T08:58:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb35125640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb300b43e0 con 0x7fdb300b3f50 2024-04-25T08:58:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb35125640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb300b46c0 con 0x7fdb3015ecc0 2024-04-25T08:58:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2ed9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdb3015a700 0x7fdb300cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52814/0 (socket says 172.21.15.37:52814) 2024-04-25T08:58:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2ed9d640 1 -- 172.21.15.37:0/195501480 learned_addr learned my addr 172.21.15.37:0/195501480 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3347703981 0 0) 0x7fdb300b2090 con 0x7fdb3015a700 2024-04-25T08:58:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb18005740 con 0x7fdb3015a700 2024-04-25T08:58:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2f59e640 1 --2- 172.21.15.37:0/195501480 >> v2:172.21.15.203:3300/0 conn(0x7fdb3015ecc0 0x7fdb3006e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4018365876 0 0) 0x7fdb18005740 con 0x7fdb3015a700 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 >> v2:172.21.15.203:3300/0 conn(0x7fdb3015ecc0 msgr2=0x7fdb3006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 --2- 172.21.15.37:0/195501480 >> v2:172.21.15.203:3300/0 conn(0x7fdb3015ecc0 0x7fdb3006e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 >> v1:172.21.15.203:6789/0 conn(0x7fdb300b3f50 legacy=0x7fdb300b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb300cd6f0 con 0x7fdb3015a700 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fdb200016a0 con 0x7fdb3015a700 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2f59e640 1 --2- 172.21.15.37:0/195501480 >> v2:172.21.15.203:3300/0 conn(0x7fdb3015ecc0 0x7fdb3006e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 >> v1:172.21.15.37:6789/0 conn(0x7fdb3015a700 legacy=0x7fdb300cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 0x7fdb18006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb18006a20 0x7fdb18006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb18007590 0x7fdb1800dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300b46c0 con 0x7fdb18005c40 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1800e4e0 con 0x7fdb18006a20 2024-04-25T08:58:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1800e6e0 con 0x7fdb18007590 2024-04-25T08:58:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2ed9d640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 0x7fdb18006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2e59c640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb18006a20 0x7fdb18006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2f59e640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb18007590 0x7fdb1800dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2ed9d640 1 -- 172.21.15.37:0/195501480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb18007590 msgr2=0x7fdb1800dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2ed9d640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb18007590 0x7fdb1800dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2ed9d640 1 -- 172.21.15.37:0/195501480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb18006a20 msgr2=0x7fdb18006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2ed9d640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb18006a20 0x7fdb18006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2ed9d640 1 -- 172.21.15.37:0/195501480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18005740 con 0x7fdb18005c40 2024-04-25T08:58:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2f59e640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb18007590 0x7fdb1800dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:43.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.904+0000 7fdb2f59e640 1 -- 172.21.15.37:0/195501480 reap_dead start 2024-04-25T08:58:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb2ed9d640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 0x7fdb18006060 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000c210 tx=0x7fdb2001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f22c34faa03d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20007d20 con 0x7fdb18005c40 2024-04-25T08:58:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb200a9020 con 0x7fdb18005c40 2024-04-25T08:58:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb2dd9b640 1 -- 172.21.15.37:0/195501480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20005ae0 con 0x7fdb18005c40 2024-04-25T08:58:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb35125640 1 -- 172.21.15.37:0/195501480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 msgr2=0x7fdb18006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb35125640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 0x7fdb18006060 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000c210 tx=0x7fdb2001c9f0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb35125640 1 -- 172.21.15.37:0/195501480 shutdown_connections 2024-04-25T08:58:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb35125640 1 --2- 172.21.15.37:0/195501480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 0x7fdb18006060 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb35125640 1 -- 172.21.15.37:0/195501480 >> 172.21.15.37:0/195501480 conn(0x7fdb300b11a0 msgr2=0x7fdb300b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb35125640 1 -- 172.21.15.37:0/195501480 shutdown_connections 2024-04-25T08:58:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb35125640 1 -- 172.21.15.37:0/195501480 wait complete. 2024-04-25T08:58:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb35125640 1 Processor -- start 2024-04-25T08:58:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.908+0000 7fdb35125640 1 -- start start 2024-04-25T08:58:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb35125640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 0x7fdb3015cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb35125640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb301f66f0 0x7fdb301572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb35125640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb30157ad0 0x7fdb300714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb35125640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300cd450 con 0x7fdb18005c40 2024-04-25T08:58:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb35125640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300cd2d0 con 0x7fdb301f66f0 2024-04-25T08:58:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb35125640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300b24a0 con 0x7fdb30157ad0 2024-04-25T08:58:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 0x7fdb3015cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 0x7fdb3015cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55600/0 (socket says 172.21.15.37:55600) 2024-04-25T08:58:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2ed9d640 1 -- 172.21.15.37:0/14987792 learned_addr learned my addr 172.21.15.37:0/14987792 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2e59c640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb301f66f0 0x7fdb301572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2f59e640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb30157ad0 0x7fdb300714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2ed9d640 1 -- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb30157ad0 msgr2=0x7fdb300714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2ed9d640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb30157ad0 0x7fdb300714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2ed9d640 1 -- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb301f66f0 msgr2=0x7fdb301572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2ed9d640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb301f66f0 0x7fdb301572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2ed9d640 1 -- 172.21.15.37:0/14987792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3015db90 con 0x7fdb18005c40 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2e59c640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb301f66f0 0x7fdb301572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2ed9d640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 0x7fdb3015cc60 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fdb2001c820 tx=0x7fdb18007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb177fe640 1 -- 172.21.15.37:0/14987792 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb200b4070 con 0x7fdb18005c40 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb177fe640 1 -- 172.21.15.37:0/14987792 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb200014a0 con 0x7fdb18005c40 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb177fe640 1 -- 172.21.15.37:0/14987792 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20007d20 con 0x7fdb18005c40 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb35125640 1 -- 172.21.15.37:0/14987792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb301582c0 con 0x7fdb18005c40 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb35125640 1 -- 172.21.15.37:0/14987792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3015d5d0 con 0x7fdb18005c40 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb35125640 1 -- 172.21.15.37:0/14987792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3004efa0 con 0x7fdb18005c40 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb177fe640 1 -- 172.21.15.37:0/14987792 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdb200bf060 con 0x7fdb18005c40 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb177fe640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdb1c066f60 0x7fdb1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb177fe640 1 -- 172.21.15.37:0/14987792 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fdb2011da90 con 0x7fdb18005c40 2024-04-25T08:58:43.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.912+0000 7fdb2e59c640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdb1c066f60 0x7fdb1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.916+0000 7fdb2e59c640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdb1c066f60 0x7fdb1c069420 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdb301eb3f0 tx=0x7fdb24009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:43.916+0000 7fdb177fe640 1 -- 172.21.15.37:0/14987792 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdb300ce700 con 0x7fdb18005c40 2024-04-25T08:58:44.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.136+0000 7fdb35125640 1 -- 172.21.15.37:0/14987792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) v1 -- 0x7fdb3006f260 con 0x7fdb18005c40 2024-04-25T08:58:44.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb177fe640 1 -- 172.21.15.37:0/14987792 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v318) v1 ==== 166+0+0 (secure 0 0 0) 0x7fdb200eebc0 con 0x7fdb18005c40 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 -- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdb1c066f60 msgr2=0x7fdb1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdb1c066f60 0x7fdb1c069420 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdb301eb3f0 tx=0x7fdb24009040 comp rx=0 tx=0).stop 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 -- 172.21.15.37:0/14987792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 msgr2=0x7fdb3015cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 0x7fdb3015cc60 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fdb2001c820 tx=0x7fdb18007a50 comp rx=0 tx=0).stop 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 -- 172.21.15.37:0/14987792 shutdown_connections 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdb1c066f60 0x7fdb1c069420 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdb30157ad0 0x7fdb300714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdb301f66f0 0x7fdb301572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 --2- 172.21.15.37:0/14987792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb18005c40 0x7fdb3015cc60 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 -- 172.21.15.37:0/14987792 >> 172.21.15.37:0/14987792 conn(0x7fdb300b11a0 msgr2=0x7fdb30159ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 -- 172.21.15.37:0/14987792 shutdown_connections 2024-04-25T08:58:44.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.420+0000 7fdb35125640 1 -- 172.21.15.37:0/14987792 wait complete. 2024-04-25T08:58:44.431 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T08:58:44.432 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-04-25T08:58:44.432 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T08:58:44.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:44.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f475360f640 1 Processor -- start 2024-04-25T08:58:44.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f475360f640 1 -- start start 2024-04-25T08:58:44.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f475360f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f474c071030 0x7f474c161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f475360f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f474c071e90 0x7f474c0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f475360f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f474c0b29e0 con 0x7f474c071410 2024-04-25T08:58:44.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f475360f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474c162680 con 0x7f474c071030 2024-04-25T08:58:44.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f475360f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f474c162960 con 0x7f474c071e90 2024-04-25T08:58:44.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f4751bac640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f474c071410 0x7f474c165920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52820/0 (socket says 172.21.15.37:52820) 2024-04-25T08:58:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f47513ab640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f474c071030 0x7f474c161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f4751bac640 1 -- 172.21.15.37:0/3919793806 learned_addr learned my addr 172.21.15.37:0/3919793806 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f4750baa640 1 --2- 172.21.15.37:0/3919793806 >> v2:172.21.15.203:3300/0 conn(0x7f474c071e90 0x7f474c0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f47513ab640 1 -- 172.21.15.37:0/3919793806 >> v2:172.21.15.203:3300/0 conn(0x7f474c071e90 msgr2=0x7f474c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f47513ab640 1 --2- 172.21.15.37:0/3919793806 >> v2:172.21.15.203:3300/0 conn(0x7f474c071e90 0x7f474c0b24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f47513ab640 1 -- 172.21.15.37:0/3919793806 >> v1:172.21.15.37:6789/0 conn(0x7f474c071410 legacy=0x7f474c165920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f47513ab640 1 -- 172.21.15.37:0/3919793806 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c162c40 con 0x7f474c071030 2024-04-25T08:58:44.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.568+0000 7f4750baa640 1 --2- 172.21.15.37:0/3919793806 >> v2:172.21.15.203:3300/0 conn(0x7f474c071e90 0x7f474c0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:44.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f47513ab640 1 --2- 172.21.15.37:0/3919793806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c071030 0x7f474c161fb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f474c0b1130 tx=0x7f474801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7ee2780a95cae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:44.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f4743fff640 1 -- 172.21.15.37:0/3919793806 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47480b2070 con 0x7f474c071030 2024-04-25T08:58:44.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f4743fff640 1 -- 172.21.15.37:0/3919793806 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47480a5870 con 0x7f474c071030 2024-04-25T08:58:44.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f4743fff640 1 -- 172.21.15.37:0/3919793806 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47480a5c40 con 0x7f474c071030 2024-04-25T08:58:44.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 -- 172.21.15.37:0/3919793806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c071030 msgr2=0x7f474c161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 --2- 172.21.15.37:0/3919793806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c071030 0x7f474c161fb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f474c0b1130 tx=0x7f474801c8a0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 -- 172.21.15.37:0/3919793806 shutdown_connections 2024-04-25T08:58:44.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 --2- 172.21.15.37:0/3919793806 >> v2:172.21.15.203:3300/0 conn(0x7f474c071e90 0x7f474c0b24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 --2- 172.21.15.37:0/3919793806 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c071030 0x7f474c161fb0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 -- 172.21.15.37:0/3919793806 >> 172.21.15.37:0/3919793806 conn(0x7f474c0b11a0 msgr2=0x7f474c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:44.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 -- 172.21.15.37:0/3919793806 shutdown_connections 2024-04-25T08:58:44.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 -- 172.21.15.37:0/3919793806 wait complete. 2024-04-25T08:58:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 Processor -- start 2024-04-25T08:58:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 -- start start 2024-04-25T08:58:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f474c071030 0x7f474c1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c071410 0x7f474c0cc190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c071e90 0x7f474c0cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c16f050 con 0x7f474c071410 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474c16eed0 con 0x7f474c071e90 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f475360f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f474c16f1d0 con 0x7f474c071030 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f4750baa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c071410 0x7f474c0cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f4750baa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c071410 0x7f474c0cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55616/0 (socket says 172.21.15.37:55616) 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.572+0000 7f4750baa640 1 -- 172.21.15.37:0/3502632617 learned_addr learned my addr 172.21.15.37:0/3502632617 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f47513ab640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f474c071030 0x7f474c1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4751bac640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c071e90 0x7f474c0cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4750baa640 1 -- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f474c071030 msgr2=0x7f474c1580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4750baa640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f474c071030 0x7f474c1580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4750baa640 1 -- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c071e90 msgr2=0x7f474c0cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4750baa640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c071e90 0x7f474c0cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4750baa640 1 -- 172.21.15.37:0/3502632617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f474c0ce670 con 0x7f474c071410 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4751bac640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c071e90 0x7f474c0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:44.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f47513ab640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f474c071030 0x7f474c1580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:44.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4750baa640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c071410 0x7f474c0cc190 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f473c00d800 tx=0x7f473c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:44.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4741ffb640 1 -- 172.21.15.37:0/3502632617 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c018020 con 0x7f474c071410 2024-04-25T08:58:44.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4741ffb640 1 -- 172.21.15.37:0/3502632617 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f473c0096c0 con 0x7f474c071410 2024-04-25T08:58:44.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f475360f640 1 -- 172.21.15.37:0/3502632617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474c0ce950 con 0x7f474c071410 2024-04-25T08:58:44.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4741ffb640 1 -- 172.21.15.37:0/3502632617 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473c009ab0 con 0x7f474c071410 2024-04-25T08:58:44.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f475360f640 1 -- 172.21.15.37:0/3502632617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f474c0cef10 con 0x7f474c071410 2024-04-25T08:58:44.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f475360f640 1 -- 172.21.15.37:0/3502632617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f474c04efa0 con 0x7f474c071410 2024-04-25T08:58:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4741ffb640 1 -- 172.21.15.37:0/3502632617 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f473c01b020 con 0x7f474c071410 2024-04-25T08:58:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4741ffb640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4730066f60 0x7f4730069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f4741ffb640 1 -- 172.21.15.37:0/3502632617 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f473c085430 con 0x7f474c071410 2024-04-25T08:58:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f47513ab640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4730066f60 0x7f4730069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.576+0000 7f47513ab640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4730066f60 0x7f4730069420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f47480af000 tx=0x7f47480a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:44.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.580+0000 7f4741ffb640 1 -- 172.21.15.37:0/3502632617 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f473c022020 con 0x7f474c071410 2024-04-25T08:58:44.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.740+0000 7f475360f640 1 -- 172.21.15.37:0/3502632617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f474c0ae860 con 0x7f474c071410 2024-04-25T08:58:44.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f4741ffb640 1 -- 172.21.15.37:0/3502632617 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+13388 (secure 0 0 0) 0x7f473c056550 con 0x7f474c071410 2024-04-25T08:58:44.734 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:44.735 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":86,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:58:43.410973+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/3927528604":"2024-04-26T08:58:40.305318+0000","172.21.15.37:6834/3927528604":"2024-04-26T08:58:40.305318+0000","172.21.15.37:6835/4130042812":"2024-04-26T08:57:11.180643+0000","172.21.15.37:6834/4130042812":"2024-04-26T08:57:11.180643+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 -- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4730066f60 msgr2=0x7f4730069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4730066f60 0x7f4730069420 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f47480af000 tx=0x7f47480a7070 comp rx=0 tx=0).stop 2024-04-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 -- 172.21.15.37:0/3502632617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c071410 msgr2=0x7f474c0cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c071410 0x7f474c0cc190 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f473c00d800 tx=0x7f473c00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 -- 172.21.15.37:0/3502632617 shutdown_connections 2024-04-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4730066f60 0x7f4730069420 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f474c071e90 0x7f474c0cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f474c071410 0x7f474c0cc190 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 --2- 172.21.15.37:0/3502632617 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f474c071030 0x7f474c1580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 -- 172.21.15.37:0/3502632617 >> 172.21.15.37:0/3502632617 conn(0x7f474c0b11a0 msgr2=0x7f474c073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:44.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 -- 172.21.15.37:0/3502632617 shutdown_connections 2024-04-25T08:58:44.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.744+0000 7f475360f640 1 -- 172.21.15.37:0/3502632617 wait complete. 2024-04-25T08:58:44.754 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:58:44.754 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:58:44.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed90ae6640 1 Processor -- start 2024-04-25T08:58:44.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed90ae6640 1 -- start start 2024-04-25T08:58:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed90ae6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fed8c15da80 0x7fed8c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed90ae6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed8c0d1cd0 0x7fed8c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed90ae6640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8c06be60 con 0x7fed8c0d0620 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed90ae6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8c06c440 con 0x7fed8c15da80 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed90ae6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c15de60 con 0x7fed8c0d1cd0 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8ad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fed8c0d0620 0x7fed8c0ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52824/0 (socket says 172.21.15.37:52824) 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8ad9d640 1 -- 172.21.15.37:0/3925474691 learned_addr learned my addr 172.21.15.37:0/3925474691 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3900204349 0 0) 0x7fed8c06be60 con 0x7fed8c0d0620 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed89d9b640 1 --2- 172.21.15.37:0/3925474691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed8c0d1cd0 0x7fed8c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8a59c640 1 --2- 172.21.15.37:0/3925474691 >> v2:172.21.15.203:3300/0 conn(0x7fed8c15da80 0x7fed8c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed74003530 con 0x7fed8c0d0620 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3082092469 0 0) 0x7fed74003530 con 0x7fed8c0d0620 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed8c0d1cd0 msgr2=0x7fed8c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:44.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 --2- 172.21.15.37:0/3925474691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed8c0d1cd0 0x7fed8c0d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 >> v2:172.21.15.203:3300/0 conn(0x7fed8c15da80 msgr2=0x7fed8c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 --2- 172.21.15.37:0/3925474691 >> v2:172.21.15.203:3300/0 conn(0x7fed8c15da80 0x7fed8c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c0d2450 con 0x7fed8c0d0620 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fed7c001680 con 0x7fed8c0d0620 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 >> v1:172.21.15.37:6789/0 conn(0x7fed8c0d0620 legacy=0x7fed8c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 --2- 172.21.15.37:0/3925474691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed74003a30 0x7fed74003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 --2- 172.21.15.37:0/3925474691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed8c0d1cd0 0x7fed74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 --2- 172.21.15.37:0/3925474691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed74004d50 0x7fed740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c06c440 con 0x7fed74004d50 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c15de60 con 0x7fed74003a30 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed7400b9d0 con 0x7fed8c0d1cd0 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8ad9d640 1 --2- 172.21.15.37:0/3925474691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed74004d50 0x7fed740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed89d9b640 1 --2- 172.21.15.37:0/3925474691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed8c0d1cd0 0x7fed74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8ad9d640 1 -- 172.21.15.37:0/3925474691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed8c0d1cd0 msgr2=0x7fed74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:44.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8ad9d640 1 --2- 172.21.15.37:0/3925474691 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed8c0d1cd0 0x7fed74004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8ad9d640 1 -- 172.21.15.37:0/3925474691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed74003a30 msgr2=0x7fed74003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:44.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8ad9d640 1 --2- 172.21.15.37:0/3925474691 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed74003a30 0x7fed74003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.892+0000 7fed8ad9d640 1 -- 172.21.15.37:0/3925474691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed74003530 con 0x7fed74004d50 2024-04-25T08:58:44.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed8ad9d640 1 --2- 172.21.15.37:0/3925474691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed74004d50 0x7fed740051d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00c210 tx=0x7fed7c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bd76043a28d63e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:44.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c007d60 con 0x7fed74004d50 2024-04-25T08:58:44.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c0a9020 con 0x7fed74004d50 2024-04-25T08:58:44.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed8959a640 1 -- 172.21.15.37:0/3925474691 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c0a5be0 con 0x7fed74004d50 2024-04-25T08:58:44.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed90ae6640 1 -- 172.21.15.37:0/3925474691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed74004d50 msgr2=0x7fed740051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed90ae6640 1 --2- 172.21.15.37:0/3925474691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed74004d50 0x7fed740051d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fed7c00c210 tx=0x7fed7c01cca0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed8ad9d640 1 -- 172.21.15.37:0/3925474691 reap_dead start 2024-04-25T08:58:44.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed90ae6640 1 -- 172.21.15.37:0/3925474691 shutdown_connections 2024-04-25T08:58:44.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed90ae6640 1 -- 172.21.15.37:0/3925474691 >> 172.21.15.37:0/3925474691 conn(0x7fed8c08b000 msgr2=0x7fed8c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:44.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed90ae6640 1 -- 172.21.15.37:0/3925474691 shutdown_connections 2024-04-25T08:58:44.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed90ae6640 1 -- 172.21.15.37:0/3925474691 wait complete. 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed90ae6640 1 Processor -- start 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.896+0000 7fed90ae6640 1 -- start start 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed90ae6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed74004d50 0x7fed8c1f04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed90ae6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed8c0cc220 0x7fed8c0c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed90ae6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed8c1f6570 0x7fed8c1f6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed90ae6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0d26f0 con 0x7fed8c0cc220 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed90ae6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0d20f0 con 0x7fed8c1f6570 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed90ae6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8c0d2870 con 0x7fed74004d50 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed89d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed8c0cc220 0x7fed8c0c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed89d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed8c0cc220 0x7fed8c0c9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55628/0 (socket says 172.21.15.37:55628) 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed89d9b640 1 -- 172.21.15.37:0/1227970553 learned_addr learned my addr 172.21.15.37:0/1227970553 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed8ad9d640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed8c1f6570 0x7fed8c1f6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed8a59c640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed74004d50 0x7fed8c1f04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed89d9b640 1 -- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed74004d50 msgr2=0x7fed8c1f04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed89d9b640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed74004d50 0x7fed8c1f04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed89d9b640 1 -- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed8c1f6570 msgr2=0x7fed8c1f6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed89d9b640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed8c1f6570 0x7fed8c1f6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed89d9b640 1 -- 172.21.15.37:0/1227970553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c0cc700 con 0x7fed8c0cc220 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed8ad9d640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed8c1f6570 0x7fed8c1f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed89d9b640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed8c0cc220 0x7fed8c0c9f80 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fed80010890 tx=0x7fed80010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed7affd640 1 -- 172.21.15.37:0/1227970553 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8000f040 con 0x7fed8c0cc220 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed7affd640 1 -- 172.21.15.37:0/1227970553 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed80005ba0 con 0x7fed8c0cc220 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed90ae6640 1 -- 172.21.15.37:0/1227970553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed8c0cc9e0 con 0x7fed8c0cc220 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed7affd640 1 -- 172.21.15.37:0/1227970553 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8000e960 con 0x7fed8c0cc220 2024-04-25T08:58:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed90ae6640 1 -- 172.21.15.37:0/1227970553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c0ccfa0 con 0x7fed8c0cc220 2024-04-25T08:58:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed90ae6640 1 -- 172.21.15.37:0/1227970553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8c04efa0 con 0x7fed8c0cc220 2024-04-25T08:58:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed7affd640 1 -- 172.21.15.37:0/1227970553 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fed8000eb00 con 0x7fed8c0cc220 2024-04-25T08:58:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed7affd640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fed6c066f90 0x7fed6c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.900+0000 7fed7affd640 1 -- 172.21.15.37:0/1227970553 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fed80085360 con 0x7fed8c0cc220 2024-04-25T08:58:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.904+0000 7fed8a59c640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fed6c066f90 0x7fed6c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.904+0000 7fed8a59c640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fed6c066f90 0x7fed6c069450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fed84004b40 tx=0x7fed8400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:44.908+0000 7fed7affd640 1 -- 172.21.15.37:0/1227970553 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fed80018030 con 0x7fed8c0cc220 2024-04-25T08:58:45.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.060+0000 7fed90ae6640 1 -- 172.21.15.37:0/1227970553 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed8c06d920 con 0x7fed8c0cc220 2024-04-25T08:58:45.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.060+0000 7fed7affd640 1 -- 172.21.15.37:0/1227970553 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed800563f0 con 0x7fed8c0cc220 2024-04-25T08:58:45.053 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:58:45.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 -- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fed6c066f90 msgr2=0x7fed6c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fed6c066f90 0x7fed6c069450 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fed84004b40 tx=0x7fed8400b040 comp rx=0 tx=0).stop 2024-04-25T08:58:45.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 -- 172.21.15.37:0/1227970553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed8c0cc220 msgr2=0x7fed8c0c9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed8c0cc220 0x7fed8c0c9f80 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fed80010890 tx=0x7fed80010d50 comp rx=0 tx=0).stop 2024-04-25T08:58:45.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 -- 172.21.15.37:0/1227970553 shutdown_connections 2024-04-25T08:58:45.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fed6c066f90 0x7fed6c069450 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fed8c1f6570 0x7fed8c1f6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fed8c0cc220 0x7fed8c0c9f80 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 --2- 172.21.15.37:0/1227970553 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fed74004d50 0x7fed8c1f04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 -- 172.21.15.37:0/1227970553 >> 172.21.15.37:0/1227970553 conn(0x7fed8c08b000 msgr2=0x7fed8c0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:45.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 -- 172.21.15.37:0/1227970553 shutdown_connections 2024-04-25T08:58:45.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.064+0000 7fed90ae6640 1 -- 172.21.15.37:0/1227970553 wait complete. 2024-04-25T08:58:45.071 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T08:58:45.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f5203053640 1 Processor -- start 2024-04-25T08:58:45.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f5203053640 1 -- start start 2024-04-25T08:58:45.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f5203053640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51fc0ce510 0x7f51fc0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f5203053640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51fc1451f0 0x7f51fc142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f5203053640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51fc0cee30 con 0x7f51fc144ca0 2024-04-25T08:58:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f5203053640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc0cc2b0 con 0x7f51fc1451f0 2024-04-25T08:58:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f5203053640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51fc06e290 con 0x7f51fc0ce510 2024-04-25T08:58:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f51fbfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f51fc144ca0 0x7f51fc06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52832/0 (socket says 172.21.15.37:52832) 2024-04-25T08:58:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f51fbfff640 1 -- 172.21.15.37:0/68240598 learned_addr learned my addr 172.21.15.37:0/68240598 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:45.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f52015f0640 1 --2- 172.21.15.37:0/68240598 >> v2:172.21.15.37:3300/0 conn(0x7f51fc1451f0 0x7f51fc142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f5200def640 1 --2- 172.21.15.37:0/68240598 >> v2:172.21.15.203:3300/0 conn(0x7f51fc0ce510 0x7f51fc0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f51fb7fe640 1 -- 172.21.15.37:0/68240598 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 116739639 0 0) 0x7f51fc0cee30 con 0x7f51fc144ca0 2024-04-25T08:58:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f51fb7fe640 1 -- 172.21.15.37:0/68240598 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51e4003530 con 0x7f51fc144ca0 2024-04-25T08:58:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f52015f0640 1 -- 172.21.15.37:0/68240598 >> v2:172.21.15.203:3300/0 conn(0x7f51fc0ce510 msgr2=0x7f51fc0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f52015f0640 1 --2- 172.21.15.37:0/68240598 >> v2:172.21.15.203:3300/0 conn(0x7f51fc0ce510 0x7f51fc0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f52015f0640 1 -- 172.21.15.37:0/68240598 >> v1:172.21.15.37:6789/0 conn(0x7f51fc144ca0 legacy=0x7f51fc06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f52015f0640 1 -- 172.21.15.37:0/68240598 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc0ccc70 con 0x7f51fc1451f0 2024-04-25T08:58:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.192+0000 7f5200def640 1 --2- 172.21.15.37:0/68240598 >> v2:172.21.15.203:3300/0 conn(0x7f51fc0ce510 0x7f51fc0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:45.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f52015f0640 1 --2- 172.21.15.37:0/68240598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc1451f0 0x7f51fc142680 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f51fc144c50 tx=0x7f51f401f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3c927f747bf0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:45.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f51fb7fe640 1 -- 172.21.15.37:0/68240598 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40b4070 con 0x7f51fc1451f0 2024-04-25T08:58:45.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f51fb7fe640 1 -- 172.21.15.37:0/68240598 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f40af600 con 0x7f51fc1451f0 2024-04-25T08:58:45.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f51fb7fe640 1 -- 172.21.15.37:0/68240598 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40af9d0 con 0x7f51fc1451f0 2024-04-25T08:58:45.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 -- 172.21.15.37:0/68240598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc1451f0 msgr2=0x7f51fc142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 --2- 172.21.15.37:0/68240598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc1451f0 0x7f51fc142680 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f51fc144c50 tx=0x7f51f401f110 comp rx=0 tx=0).stop 2024-04-25T08:58:45.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 -- 172.21.15.37:0/68240598 shutdown_connections 2024-04-25T08:58:45.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 --2- 172.21.15.37:0/68240598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc1451f0 0x7f51fc142680 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 --2- 172.21.15.37:0/68240598 >> v2:172.21.15.203:3300/0 conn(0x7f51fc0ce510 0x7f51fc0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 -- 172.21.15.37:0/68240598 >> 172.21.15.37:0/68240598 conn(0x7f51fc0b11a0 msgr2=0x7f51fc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:45.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 -- 172.21.15.37:0/68240598 shutdown_connections 2024-04-25T08:58:45.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 -- 172.21.15.37:0/68240598 wait complete. 2024-04-25T08:58:45.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 Processor -- start 2024-04-25T08:58:45.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 -- start start 2024-04-25T08:58:45.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc133270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51fc144ca0 0x7f51fc1e4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc1451f0 0x7f51fc15b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc06ee30 con 0x7f51fc1451f0 2024-04-25T08:58:45.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc06df20 con 0x7f51fc144ca0 2024-04-25T08:58:45.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc0cc900 con 0x7f51fc0ce510 2024-04-25T08:58:45.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f52015f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc1451f0 0x7f51fc15b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f52015f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc1451f0 0x7f51fc15b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55640/0 (socket says 172.21.15.37:55640) 2024-04-25T08:58:45.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f52015f0640 1 -- 172.21.15.37:0/3135534939 learned_addr learned my addr 172.21.15.37:0/3135534939 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:45.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f51fbfff640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51fc144ca0 0x7f51fc1e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5200def640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc133270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f52015f0640 1 -- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 msgr2=0x7f51fc133270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f52015f0640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc133270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f52015f0640 1 -- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51fc144ca0 msgr2=0x7f51fc1e4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f52015f0640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51fc144ca0 0x7f51fc1e4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f52015f0640 1 -- 172.21.15.37:0/3135534939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc133950 con 0x7f51fc1451f0 2024-04-25T08:58:45.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f51fbfff640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51fc144ca0 0x7f51fc1e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:45.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f52015f0640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc1451f0 0x7f51fc15b0c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f51f401bd20 tx=0x7f51f401beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:45.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f51f97fa640 1 -- 172.21.15.37:0/3135534939 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f40b4040 con 0x7f51fc1451f0 2024-04-25T08:58:45.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f51f97fa640 1 -- 172.21.15.37:0/3135534939 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f40c1070 con 0x7f51fc1451f0 2024-04-25T08:58:45.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 -- 172.21.15.37:0/3135534939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc133c30 con 0x7f51fc1451f0 2024-04-25T08:58:45.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f51f97fa640 1 -- 172.21.15.37:0/3135534939 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4009450 con 0x7f51fc1451f0 2024-04-25T08:58:45.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.196+0000 7f5203053640 1 -- 172.21.15.37:0/3135534939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc1341f0 con 0x7f51fc1451f0 2024-04-25T08:58:45.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.200+0000 7f5203053640 1 -- 172.21.15.37:0/3135534939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc04efa0 con 0x7f51fc1451f0 2024-04-25T08:58:45.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.200+0000 7f51f97fa640 1 -- 172.21.15.37:0/3135534939 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51f40ba6b0 con 0x7f51fc1451f0 2024-04-25T08:58:45.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.200+0000 7f51f97fa640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e8066f60 0x7f51e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.200+0000 7f51f97fa640 1 -- 172.21.15.37:0/3135534939 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f51f4123850 con 0x7f51fc1451f0 2024-04-25T08:58:45.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.200+0000 7f5200def640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e8066f60 0x7f51e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.200+0000 7f5200def640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e8066f60 0x7f51e8069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f51fc1e53f0 tx=0x7f51ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:45.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.204+0000 7f51f97fa640 1 -- 172.21.15.37:0/3135534939 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51f4018c70 con 0x7f51fc1451f0 2024-04-25T08:58:45.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:45.416+0000 7f5203053640 1 -- 172.21.15.37:0/3135534939 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7f51fc0b0f80 con 0x7f51fc1451f0 2024-04-25T08:58:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.428+0000 7f51f97fa640 1 -- 172.21.15.37:0/3135534939 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v320) v1 ==== 170+0+0 (secure 0 0 0) 0x7f51f40f4980 con 0x7f51fc1451f0 2024-04-25T08:58:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 -- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e8066f60 msgr2=0x7f51e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e8066f60 0x7f51e8069420 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f51fc1e53f0 tx=0x7f51ec008040 comp rx=0 tx=0).stop 2024-04-25T08:58:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 -- 172.21.15.37:0/3135534939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc1451f0 msgr2=0x7f51fc15b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc1451f0 0x7f51fc15b0c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f51f401bd20 tx=0x7f51f401beb0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 -- 172.21.15.37:0/3135534939 shutdown_connections 2024-04-25T08:58:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51e8066f60 0x7f51e8069420 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51fc1451f0 0x7f51fc15b0c0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51fc144ca0 0x7f51fc1e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 --2- 172.21.15.37:0/3135534939 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51fc0ce510 0x7f51fc133270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 -- 172.21.15.37:0/3135534939 >> 172.21.15.37:0/3135534939 conn(0x7f51fc0b11a0 msgr2=0x7f51fc0c4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 -- 172.21.15.37:0/3135534939 shutdown_connections 2024-04-25T08:58:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.432+0000 7f5203053640 1 -- 172.21.15.37:0/3135534939 wait complete. 2024-04-25T08:58:46.443 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:58:46.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f569140c640 1 Processor -- start 2024-04-25T08:58:46.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f569140c640 1 -- start start 2024-04-25T08:58:46.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f569140c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f568c0cf590 0x7f568c0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f569140c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f568c072000 0x7f568c072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f569140c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f568c0cbd20 con 0x7f568c15b6b0 2024-04-25T08:58:46.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f569140c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c15ba90 con 0x7f568c0cf590 2024-04-25T08:58:46.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f569140c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f568c0cc000 con 0x7f568c072000 2024-04-25T08:58:46.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f568a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f568c0cf590 0x7f568c0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f568a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f568c0cf590 0x7f568c0cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55652/0 (socket says 172.21.15.37:55652) 2024-04-25T08:58:46.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f568a7fc640 1 -- 172.21.15.37:0/2124997412 learned_addr learned my addr 172.21.15.37:0/2124997412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:46.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f568b7fe640 1 --2- 172.21.15.37:0/2124997412 >> v2:172.21.15.203:3300/0 conn(0x7f568c072000 0x7f568c072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f568a7fc640 1 -- 172.21.15.37:0/2124997412 >> v2:172.21.15.203:3300/0 conn(0x7f568c072000 msgr2=0x7f568c072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:46.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f568a7fc640 1 --2- 172.21.15.37:0/2124997412 >> v2:172.21.15.203:3300/0 conn(0x7f568c072000 0x7f568c072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f568a7fc640 1 -- 172.21.15.37:0/2124997412 >> v1:172.21.15.203:6789/0 conn(0x7f568c15b6b0 legacy=0x7f568c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.584+0000 7f568a7fc640 1 -- 172.21.15.37:0/2124997412 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c0cc2e0 con 0x7f568c0cf590 2024-04-25T08:58:46.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568a7fc640 1 --2- 172.21.15.37:0/2124997412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f568c0cf590 0x7f568c0cfa20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f568c06f000 tx=0x7f567c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee756222ff2a0dda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:46.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f5689ffb640 1 -- 172.21.15.37:0/2124997412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0ab610 con 0x7f568c0cf590 2024-04-25T08:58:46.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f5689ffb640 1 -- 172.21.15.37:0/2124997412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f567c0b0070 con 0x7f568c0cf590 2024-04-25T08:58:46.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f5689ffb640 1 -- 172.21.15.37:0/2124997412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f567c0ab9d0 con 0x7f568c0cf590 2024-04-25T08:58:46.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 -- 172.21.15.37:0/2124997412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f568c0cf590 msgr2=0x7f568c0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 --2- 172.21.15.37:0/2124997412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f568c0cf590 0x7f568c0cfa20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f568c06f000 tx=0x7f567c01c520 comp rx=0 tx=0).stop 2024-04-25T08:58:46.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 -- 172.21.15.37:0/2124997412 shutdown_connections 2024-04-25T08:58:46.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 --2- 172.21.15.37:0/2124997412 >> v2:172.21.15.203:3300/0 conn(0x7f568c072000 0x7f568c072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 --2- 172.21.15.37:0/2124997412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f568c0cf590 0x7f568c0cfa20 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 -- 172.21.15.37:0/2124997412 >> 172.21.15.37:0/2124997412 conn(0x7f568c06ae60 msgr2=0x7f568c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:46.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 -- 172.21.15.37:0/2124997412 shutdown_connections 2024-04-25T08:58:46.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 -- 172.21.15.37:0/2124997412 wait complete. 2024-04-25T08:58:46.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 Processor -- start 2024-04-25T08:58:46.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 -- start start 2024-04-25T08:58:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f568c072000 0x7f568c1f4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568c0cf590 0x7f568c15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f568c15b6b0 0x7f568c156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0caf30 con 0x7f568c15b6b0 2024-04-25T08:58:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f568c0cadb0 con 0x7f568c072000 2024-04-25T08:58:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f569140c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f568c1682a0 con 0x7f568c0cf590 2024-04-25T08:58:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f568c15b6b0 0x7f568c156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f568c15b6b0 0x7f568c156c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55654/0 (socket says 172.21.15.37:55654) 2024-04-25T08:58:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568b7fe640 1 -- 172.21.15.37:0/1865378412 learned_addr learned my addr 172.21.15.37:0/1865378412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568a7fc640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568c0cf590 0x7f568c15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568affd640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f568c072000 0x7f568c1f4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568b7fe640 1 -- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568c0cf590 msgr2=0x7f568c15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568b7fe640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568c0cf590 0x7f568c15bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568b7fe640 1 -- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f568c072000 msgr2=0x7f568c1f4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568b7fe640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f568c072000 0x7f568c1f4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568b7fe640 1 -- 172.21.15.37:0/1865378412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f568c1f53d0 con 0x7f568c15b6b0 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.588+0000 7f568affd640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f568c072000 0x7f568c1f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f568b7fe640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f568c15b6b0 0x7f568c156c70 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5684009dd0 tx=0x7f568400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f56737fe640 1 -- 172.21.15.37:0/1865378412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684018020 con 0x7f568c15b6b0 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f56737fe640 1 -- 172.21.15.37:0/1865378412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5684014480 con 0x7f568c15b6b0 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f569140c640 1 -- 172.21.15.37:0/1865378412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c15c580 con 0x7f568c15b6b0 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f56737fe640 1 -- 172.21.15.37:0/1865378412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684014870 con 0x7f568c15b6b0 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f569140c640 1 -- 172.21.15.37:0/1865378412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568c1f4690 con 0x7f568c15b6b0 2024-04-25T08:58:46.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f569140c640 1 -- 172.21.15.37:0/1865378412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f568c15ead0 con 0x7f568c15b6b0 2024-04-25T08:58:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f56737fe640 1 -- 172.21.15.37:0/1865378412 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f568401b020 con 0x7f568c15b6b0 2024-04-25T08:58:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f56737fe640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5678066f60 0x7f5678069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f56737fe640 1 -- 172.21.15.37:0/1865378412 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f56840859b0 con 0x7f568c15b6b0 2024-04-25T08:58:46.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f568affd640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5678066f60 0x7f5678069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.592+0000 7f568affd640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5678066f60 0x7f5678069420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f568c0b1b20 tx=0x7f5680009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:46.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.596+0000 7f56737fe640 1 -- 172.21.15.37:0/1865378412 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f568c15ead0 con 0x7f568c15b6b0 2024-04-25T08:58:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.756+0000 7f569140c640 1 -- 172.21.15.37:0/1865378412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f568c04efa0 con 0x7f568c15b6b0 2024-04-25T08:58:46.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.760+0000 7f56737fe640 1 -- 172.21.15.37:0/1865378412 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5684022070 con 0x7f568c15b6b0 2024-04-25T08:58:46.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 -- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5678066f60 msgr2=0x7f5678069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5678066f60 0x7f5678069420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f568c0b1b20 tx=0x7f5680009040 comp rx=0 tx=0).stop 2024-04-25T08:58:46.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 -- 172.21.15.37:0/1865378412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f568c15b6b0 msgr2=0x7f568c156c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f568c15b6b0 0x7f568c156c70 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5684009dd0 tx=0x7f568400c490 comp rx=0 tx=0).stop 2024-04-25T08:58:46.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 -- 172.21.15.37:0/1865378412 shutdown_connections 2024-04-25T08:58:46.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5678066f60 0x7f5678069420 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f568c15b6b0 0x7f568c156c70 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f568c0cf590 0x7f568c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 --2- 172.21.15.37:0/1865378412 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f568c072000 0x7f568c1f4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 -- 172.21.15.37:0/1865378412 >> 172.21.15.37:0/1865378412 conn(0x7f568c06ae60 msgr2=0x7f568c0b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 -- 172.21.15.37:0/1865378412 shutdown_connections 2024-04-25T08:58:46.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:46.764+0000 7f569140c640 1 -- 172.21.15.37:0/1865378412 wait complete. 2024-04-25T08:58:46.773 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:58:46.773 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:58:46.803 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:58:46.858 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:58:46.859 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:58:46.859 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:46.865 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:58:46.865 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:58:46.865 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:46.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:58:46.875 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:58:46.875 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:46.910 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:58:46.911 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T08:58:46.911 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:46.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:58:46.921 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:58:46.921 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:47.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb81881640 1 Processor -- start 2024-04-25T08:58:47.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb81881640 1 -- start start 2024-04-25T08:58:47.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb81881640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffb7c15b950 0x7ffb7c09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb81881640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb7c0cfb80 0x7ffb7c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb81881640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb7c15bd30 con 0x7ffb7c0cf420 2024-04-25T08:58:47.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb81881640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7c0d0510 con 0x7ffb7c15b950 2024-04-25T08:58:47.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb81881640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c0cc0d0 con 0x7ffb7c0cfb80 2024-04-25T08:58:47.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb7affd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffb7c15b950 0x7ffb7c09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb73fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ffb7c0cf420 0x7ffb7c0cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52846/0 (socket says 172.21.15.37:52846) 2024-04-25T08:58:47.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb73fff640 1 -- 172.21.15.37:0/1915602574 learned_addr learned my addr 172.21.15.37:0/1915602574 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:47.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb7b7fe640 1 --2- 172.21.15.37:0/1915602574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb7c0cfb80 0x7ffb7c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb7a7fc640 1 -- 172.21.15.37:0/1915602574 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2894457207 0 0) 0x7ffb7c15bd30 con 0x7ffb7c0cf420 2024-04-25T08:58:47.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb7a7fc640 1 -- 172.21.15.37:0/1915602574 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb68003530 con 0x7ffb7c0cf420 2024-04-25T08:58:47.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb7affd640 1 -- 172.21.15.37:0/1915602574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb7c0cfb80 msgr2=0x7ffb7c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:47.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb7affd640 1 --2- 172.21.15.37:0/1915602574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb7c0cfb80 0x7ffb7c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb7affd640 1 -- 172.21.15.37:0/1915602574 >> v1:172.21.15.37:6789/0 conn(0x7ffb7c0cf420 legacy=0x7ffb7c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.096+0000 7ffb7affd640 1 -- 172.21.15.37:0/1915602574 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c09ab10 con 0x7ffb7c15b950 2024-04-25T08:58:47.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7affd640 1 --2- 172.21.15.37:0/1915602574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb7c15b950 0x7ffb7c09a480 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0097c0 tx=0x7ffb6c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=794a6c6824fe945e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:47.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7a7fc640 1 -- 172.21.15.37:0/1915602574 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0b2070 con 0x7ffb7c15b950 2024-04-25T08:58:47.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7a7fc640 1 -- 172.21.15.37:0/1915602574 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c0ad640 con 0x7ffb7c15b950 2024-04-25T08:58:47.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7a7fc640 1 -- 172.21.15.37:0/1915602574 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0ada10 con 0x7ffb7c15b950 2024-04-25T08:58:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 -- 172.21.15.37:0/1915602574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb7c15b950 msgr2=0x7ffb7c09a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 --2- 172.21.15.37:0/1915602574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb7c15b950 0x7ffb7c09a480 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0097c0 tx=0x7ffb6c01c4a0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 -- 172.21.15.37:0/1915602574 shutdown_connections 2024-04-25T08:58:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 --2- 172.21.15.37:0/1915602574 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb7c0cfb80 0x7ffb7c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 --2- 172.21.15.37:0/1915602574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb7c15b950 0x7ffb7c09a480 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 -- 172.21.15.37:0/1915602574 >> 172.21.15.37:0/1915602574 conn(0x7ffb7c0b30b0 msgr2=0x7ffb7c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 -- 172.21.15.37:0/1915602574 shutdown_connections 2024-04-25T08:58:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 -- 172.21.15.37:0/1915602574 wait complete. 2024-04-25T08:58:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 Processor -- start 2024-04-25T08:58:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 -- start start 2024-04-25T08:58:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb7c0cf420 0x7ffb7c156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb7c0cfb80 0x7ffb7c15d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb7c15b950 0x7ffb7c15dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c0cf9b0 con 0x7ffb7c15b950 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c0cf830 con 0x7ffb7c0cfb80 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb81881640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7c06faa0 con 0x7ffb7c0cf420 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb7c15b950 0x7ffb7c15dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb7c15b950 0x7ffb7c15dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55676/0 (socket says 172.21.15.37:55676) 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7b7fe640 1 -- 172.21.15.37:0/3369083510 learned_addr learned my addr 172.21.15.37:0/3369083510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7affd640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb7c0cf420 0x7ffb7c156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb73fff640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb7c0cfb80 0x7ffb7c15d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7b7fe640 1 -- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb7c0cf420 msgr2=0x7ffb7c156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7b7fe640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb7c0cf420 0x7ffb7c156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7b7fe640 1 -- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb7c0cfb80 msgr2=0x7ffb7c15d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7b7fe640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb7c0cfb80 0x7ffb7c15d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.100+0000 7ffb7b7fe640 1 -- 172.21.15.37:0/3369083510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb7c0c5f90 con 0x7ffb7c15b950 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb7b7fe640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb7c15b950 0x7ffb7c15dd50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400efc0 tx=0x7ffb7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb78ff9640 1 -- 172.21.15.37:0/3369083510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74018020 con 0x7ffb7c15b950 2024-04-25T08:58:47.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb78ff9640 1 -- 172.21.15.37:0/3369083510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb74014480 con 0x7ffb7c15b950 2024-04-25T08:58:47.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb78ff9640 1 -- 172.21.15.37:0/3369083510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb74014870 con 0x7ffb7c15b950 2024-04-25T08:58:47.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb81881640 1 -- 172.21.15.37:0/3369083510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb7c0c6270 con 0x7ffb7c15b950 2024-04-25T08:58:47.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb81881640 1 -- 172.21.15.37:0/3369083510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb7c0c6830 con 0x7ffb7c15b950 2024-04-25T08:58:47.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb81881640 1 -- 172.21.15.37:0/3369083510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb7c04efa0 con 0x7ffb7c15b950 2024-04-25T08:58:47.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb78ff9640 1 -- 172.21.15.37:0/3369083510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb7401b020 con 0x7ffb7c15b950 2024-04-25T08:58:47.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb78ff9640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb5c066f60 0x7ffb5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb78ff9640 1 -- 172.21.15.37:0/3369083510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ffb74086060 con 0x7ffb7c15b950 2024-04-25T08:58:47.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb7affd640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb5c066f60 0x7ffb5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.104+0000 7ffb7affd640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb5c066f60 0x7ffb5c069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0af000 tx=0x7ffb6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:47.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.108+0000 7ffb78ff9640 1 -- 172.21.15.37:0/3369083510 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb74022070 con 0x7ffb7c15b950 2024-04-25T08:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.300+0000 7ffb81881640 1 -- 172.21.15.37:0/3369083510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb7c0cf050 con 0x7ffb7c15b950 2024-04-25T08:58:47.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb78ff9640 1 -- 172.21.15.37:0/3369083510 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+3705 (secure 0 0 0) 0x7ffb74010040 con 0x7ffb7c15b950 2024-04-25T08:58:47.295 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:47.295 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":54,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5558,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2702395549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":2702395549},{"type":"v1","addr":"172.21.15.203:6837","nonce":2702395549}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6097,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1942139256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1942139256},{"type":"v1","addr":"172.21.15.203:6835","nonce":1942139256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6927,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1006773121","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1006773121},{"type":"v1","addr":"172.21.15.37:6833","nonce":1006773121}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":7656,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/1156457533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1156457533},{"type":"v1","addr":"172.21.15.37:6835","nonce":1156457533}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":54}],"filesystems":[]} 2024-04-25T08:58:47.295 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 54 2024-04-25T08:58:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 -- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb5c066f60 msgr2=0x7ffb5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb5c066f60 0x7ffb5c069420 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c0af000 tx=0x7ffb6c008040 comp rx=0 tx=0).stop 2024-04-25T08:58:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 -- 172.21.15.37:0/3369083510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb7c15b950 msgr2=0x7ffb7c15dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb7c15b950 0x7ffb7c15dd50 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400efc0 tx=0x7ffb7400c490 comp rx=0 tx=0).stop 2024-04-25T08:58:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 -- 172.21.15.37:0/3369083510 shutdown_connections 2024-04-25T08:58:47.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb5c066f60 0x7ffb5c069420 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb7c15b950 0x7ffb7c15dd50 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb7c0cfb80 0x7ffb7c15d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 --2- 172.21.15.37:0/3369083510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb7c0cf420 0x7ffb7c156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 -- 172.21.15.37:0/3369083510 >> 172.21.15.37:0/3369083510 conn(0x7ffb7c0b30b0 msgr2=0x7ffb7c0ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 -- 172.21.15.37:0/3369083510 shutdown_connections 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.304+0000 7ffb81881640 1 -- 172.21.15.37:0/3369083510 wait complete. 2024-04-25T08:58:47.315 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:58:47.315 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:58:47.315 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:58:47.315 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:47.315 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:58:47.315 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:58:47.316 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:58:47.316 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:47.316 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:58:47.316 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:58:47.316 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:58:47.316 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:47.316 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:58:47.316 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:58:47.317 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:58:47.317 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:47.325 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:58:47.325 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:58:47.326 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:58:47.326 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:58:47.331 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:58:47.331 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:58:47.335 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:58:47.335 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:58:47.368 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:58:47.368 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:47.369 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:58:47.369 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:47.377 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:58:47.378 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:47.379 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:58:47.379 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:47.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:58:47.429 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T08:58:47.436 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T08:58:47.445 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-25T08:58:47.446 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T08:58:47.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9adf62a640 1 Processor -- start 2024-04-25T08:58:47.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9adf62a640 1 -- start start 2024-04-25T08:58:47.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9adf62a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9ad80b24d0 0x7f9ad809aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9adf62a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad8099cc0 con 0x7f9ad806c260 2024-04-25T08:58:47.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9adf62a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad8099fa0 con 0x7f9ad80b28b0 2024-04-25T08:58:47.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9adf62a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad8099a70 con 0x7f9ad80b24d0 2024-04-25T08:58:47.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9add3c6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9ad80b24d0 0x7f9ad809aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9addbc7640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9ad806c260 0x7f9ad806c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52866/0 (socket says 172.21.15.37:52866) 2024-04-25T08:58:47.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9add3c6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9ad80b24d0 0x7f9ad809aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55694/0 (socket says 172.21.15.37:55694) 2024-04-25T08:58:47.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9addbc7640 1 -- 172.21.15.37:0/2443569253 learned_addr learned my addr 172.21.15.37:0/2443569253 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:47.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9add3c6640 1 -- 172.21.15.37:0/2443569253 >> v1:172.21.15.203:6789/0 conn(0x7f9ad80b28b0 legacy=0x7f9ad80b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9add3c6640 1 -- 172.21.15.37:0/2443569253 >> v1:172.21.15.37:6789/0 conn(0x7f9ad806c260 legacy=0x7f9ad806c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9acffff640 1 -- 172.21.15.37:0/2443569253 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702312311 0 0) 0x7f9ad8099cc0 con 0x7f9ad806c260 2024-04-25T08:58:47.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.508+0000 7f9add3c6640 1 -- 172.21.15.37:0/2443569253 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad809a280 con 0x7f9ad80b24d0 2024-04-25T08:58:47.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9add3c6640 1 --2- 172.21.15.37:0/2443569253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad80b24d0 0x7f9ad809aa80 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097c0 tx=0x7f9ad401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=658912237de91928 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:47.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9acffff640 1 -- 172.21.15.37:0/2443569253 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40b0070 con 0x7f9ad80b24d0 2024-04-25T08:58:47.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9acffff640 1 -- 172.21.15.37:0/2443569253 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad40ab600 con 0x7f9ad80b24d0 2024-04-25T08:58:47.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9acffff640 1 -- 172.21.15.37:0/2443569253 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40ab9d0 con 0x7f9ad80b24d0 2024-04-25T08:58:47.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 -- 172.21.15.37:0/2443569253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad80b24d0 msgr2=0x7f9ad809aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 --2- 172.21.15.37:0/2443569253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad80b24d0 0x7f9ad809aa80 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097c0 tx=0x7f9ad401cdb0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 -- 172.21.15.37:0/2443569253 shutdown_connections 2024-04-25T08:58:47.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 --2- 172.21.15.37:0/2443569253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad80b24d0 0x7f9ad809aa80 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 -- 172.21.15.37:0/2443569253 >> 172.21.15.37:0/2443569253 conn(0x7f9ad80b3020 msgr2=0x7f9ad80b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:47.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 -- 172.21.15.37:0/2443569253 shutdown_connections 2024-04-25T08:58:47.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 -- 172.21.15.37:0/2443569253 wait complete. 2024-04-25T08:58:47.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 Processor -- start 2024-04-25T08:58:47.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 -- start start 2024-04-25T08:58:47.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad806c260 0x7f9ad814b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ad80b24d0 0x7f9ad8160230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.512+0000 7f9adf62a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ad80b28b0 0x7f9ad8160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9adf62a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80b2e10 con 0x7f9ad806c260 2024-04-25T08:58:47.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9adf62a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad80b2c90 con 0x7f9ad80b24d0 2024-04-25T08:58:47.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9adf62a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad815dea0 con 0x7f9ad80b28b0 2024-04-25T08:58:47.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9add3c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad806c260 0x7f9ad814b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9add3c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad806c260 0x7f9ad814b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55710/0 (socket says 172.21.15.37:55710) 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9add3c6640 1 -- 172.21.15.37:0/397949620 learned_addr learned my addr 172.21.15.37:0/397949620 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9adcbc5640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ad80b24d0 0x7f9ad8160230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9addbc7640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ad80b28b0 0x7f9ad8160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9add3c6640 1 -- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ad80b28b0 msgr2=0x7f9ad8160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9add3c6640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ad80b28b0 0x7f9ad8160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9add3c6640 1 -- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ad80b24d0 msgr2=0x7f9ad8160230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9add3c6640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ad80b24d0 0x7f9ad8160230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9add3c6640 1 -- 172.21.15.37:0/397949620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad80c7310 con 0x7f9ad806c260 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9addbc7640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ad80b28b0 0x7f9ad8160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9add3c6640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad806c260 0x7f9ad814b7c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40ae040 tx=0x7f9ad401ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9acdffb640 1 -- 172.21.15.37:0/397949620 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40bb070 con 0x7f9ad806c260 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9acdffb640 1 -- 172.21.15.37:0/397949620 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad4004020 con 0x7f9ad806c260 2024-04-25T08:58:47.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9adf62a640 1 -- 172.21.15.37:0/397949620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad815e280 con 0x7f9ad806c260 2024-04-25T08:58:47.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9acdffb640 1 -- 172.21.15.37:0/397949620 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40b0040 con 0x7f9ad806c260 2024-04-25T08:58:47.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9adf62a640 1 -- 172.21.15.37:0/397949620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad80c7890 con 0x7f9ad806c260 2024-04-25T08:58:47.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9adf62a640 1 -- 172.21.15.37:0/397949620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad804efa0 con 0x7f9ad806c260 2024-04-25T08:58:47.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9acdffb640 1 -- 172.21.15.37:0/397949620 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ad4003620 con 0x7f9ad806c260 2024-04-25T08:58:47.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9acdffb640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9abc066f60 0x7f9abc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9acdffb640 1 -- 172.21.15.37:0/397949620 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f9ad411f750 con 0x7f9ad806c260 2024-04-25T08:58:47.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.516+0000 7f9adcbc5640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9abc066f60 0x7f9abc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.520+0000 7f9adcbc5640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9abc066f60 0x7f9abc069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8070d20 tx=0x7f9ac8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:47.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.520+0000 7f9acdffb640 1 -- 172.21.15.37:0/397949620 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9ad4016090 con 0x7f9ad806c260 2024-04-25T08:58:47.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:47.688+0000 7f9adf62a640 1 -- 172.21.15.37:0/397949620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9ad80c4e10 con 0x7f9ad806c260 2024-04-25T08:58:48.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9acdffb640 1 -- 172.21.15.37:0/397949620 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v91) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9ad40aa3a0 con 0x7f9ad806c260 2024-04-25T08:58:48.778 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-04-25T08:58:48.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 -- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9abc066f60 msgr2=0x7f9abc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:48.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9abc066f60 0x7f9abc069420 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8070d20 tx=0x7f9ac8009040 comp rx=0 tx=0).stop 2024-04-25T08:58:48.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 -- 172.21.15.37:0/397949620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad806c260 msgr2=0x7f9ad814b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:48.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad806c260 0x7f9ad814b7c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40ae040 tx=0x7f9ad401ab90 comp rx=0 tx=0).stop 2024-04-25T08:58:48.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 -- 172.21.15.37:0/397949620 shutdown_connections 2024-04-25T08:58:48.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9abc066f60 0x7f9abc069420 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9ad80b28b0 0x7f9ad8160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9ad80b24d0 0x7f9ad8160230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 --2- 172.21.15.37:0/397949620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9ad806c260 0x7f9ad814b7c0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 -- 172.21.15.37:0/397949620 >> 172.21.15.37:0/397949620 conn(0x7f9ad80b3020 msgr2=0x7f9ad80d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:48.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 -- 172.21.15.37:0/397949620 shutdown_connections 2024-04-25T08:58:48.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.788+0000 7f9adf62a640 1 -- 172.21.15.37:0/397949620 wait complete. 2024-04-25T08:58:48.804 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:58:48.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.944+0000 7fcfc2920640 1 Processor -- start 2024-04-25T08:58:48.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.944+0000 7fcfc2920640 1 -- start start 2024-04-25T08:58:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfc2920640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfbc15b950 0x7fcfbc09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfc2920640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcfbc0cf420 0x7fcfbc0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfc2920640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcfbc0cfb80 0x7fcfbc0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfc2920640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfbc15bd30 con 0x7fcfbc0cf420 2024-04-25T08:58:48.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfc2920640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfbc0cc0d0 con 0x7fcfbc0cfb80 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfc2920640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc09ab10 con 0x7fcfbc15b950 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcfbc0cf420 0x7fcfbc0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcfbc0cf420 0x7fcfbc0cce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55720/0 (socket says 172.21.15.37:55720) 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbb7fe640 1 -- 172.21.15.37:0/1989498479 learned_addr learned my addr 172.21.15.37:0/1989498479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfc0ebd640 1 --2- 172.21.15.37:0/1989498479 >> v2:172.21.15.203:3300/0 conn(0x7fcfbc0cfb80 0x7fcfbc0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbbfff640 1 --2- 172.21.15.37:0/1989498479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfbc15b950 0x7fcfbc09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbb7fe640 1 -- 172.21.15.37:0/1989498479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfbc15b950 msgr2=0x7fcfbc09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbb7fe640 1 --2- 172.21.15.37:0/1989498479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfbc15b950 0x7fcfbc09a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbb7fe640 1 -- 172.21.15.37:0/1989498479 >> v2:172.21.15.203:3300/0 conn(0x7fcfbc0cfb80 msgr2=0x7fcfbc0d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbb7fe640 1 --2- 172.21.15.37:0/1989498479 >> v2:172.21.15.203:3300/0 conn(0x7fcfbc0cfb80 0x7fcfbc0d0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbb7fe640 1 -- 172.21.15.37:0/1989498479 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc0cc370 con 0x7fcfbc0cf420 2024-04-25T08:58:48.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfc0ebd640 1 --2- 172.21.15.37:0/1989498479 >> v2:172.21.15.203:3300/0 conn(0x7fcfbc0cfb80 0x7fcfbc0d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:48.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbb7fe640 1 --2- 172.21.15.37:0/1989498479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfbc0cf420 0x7fcfbc0cce50 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0b3bd0 tx=0x7fcfb001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64db2540cb6a75d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:48.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbaffd640 1 -- 172.21.15.37:0/1989498479 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00b1070 con 0x7fcfbc0cf420 2024-04-25T08:58:48.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbaffd640 1 -- 172.21.15.37:0/1989498479 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb00ac630 con 0x7fcfbc0cf420 2024-04-25T08:58:48.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.948+0000 7fcfbaffd640 1 -- 172.21.15.37:0/1989498479 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00a8050 con 0x7fcfbc0cf420 2024-04-25T08:58:48.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 -- 172.21.15.37:0/1989498479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfbc0cf420 msgr2=0x7fcfbc0cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:48.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 --2- 172.21.15.37:0/1989498479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfbc0cf420 0x7fcfbc0cce50 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0b3bd0 tx=0x7fcfb001c520 comp rx=0 tx=0).stop 2024-04-25T08:58:48.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 -- 172.21.15.37:0/1989498479 shutdown_connections 2024-04-25T08:58:48.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 --2- 172.21.15.37:0/1989498479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfbc0cf420 0x7fcfbc0cce50 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 --2- 172.21.15.37:0/1989498479 >> v2:172.21.15.203:3300/0 conn(0x7fcfbc0cfb80 0x7fcfbc0d0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 --2- 172.21.15.37:0/1989498479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfbc15b950 0x7fcfbc09a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 -- 172.21.15.37:0/1989498479 >> 172.21.15.37:0/1989498479 conn(0x7fcfbc0b30b0 msgr2=0x7fcfbc0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 -- 172.21.15.37:0/1989498479 shutdown_connections 2024-04-25T08:58:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 -- 172.21.15.37:0/1989498479 wait complete. 2024-04-25T08:58:48.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 Processor -- start 2024-04-25T08:58:48.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 -- start start 2024-04-25T08:58:48.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfbc0cf420 0x7fcfbc0c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfbc0cfb80 0x7fcfbc1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfbc15b950 0x7fcfbc0c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0cd390 con 0x7fcfbc15b950 2024-04-25T08:58:48.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc06faa0 con 0x7fcfbc0cfb80 2024-04-25T08:58:48.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0b41d0 con 0x7fcfbc0cf420 2024-04-25T08:58:48.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc0ebd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfbc15b950 0x7fcfbc0c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc0ebd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfbc15b950 0x7fcfbc0c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55726/0 (socket says 172.21.15.37:55726) 2024-04-25T08:58:48.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc0ebd640 1 -- 172.21.15.37:0/1926816209 learned_addr learned my addr 172.21.15.37:0/1926816209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:48.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfbbfff640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfbc0cf420 0x7fcfbc0c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfbb7fe640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfbc0cfb80 0x7fcfbc1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc0ebd640 1 -- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfbc0cf420 msgr2=0x7fcfbc0c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:48.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc0ebd640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfbc0cf420 0x7fcfbc0c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc0ebd640 1 -- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfbc0cfb80 msgr2=0x7fcfbc1538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:48.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc0ebd640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfbc0cfb80 0x7fcfbc1538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc0ebd640 1 -- 172.21.15.37:0/1926816209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc0c6350 con 0x7fcfbc15b950 2024-04-25T08:58:48.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc0ebd640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfbc15b950 0x7fcfbc0c7460 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400cef0 tx=0x7fcfb400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:48.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfb8ff9640 1 -- 172.21.15.37:0/1926816209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb401a070 con 0x7fcfbc15b950 2024-04-25T08:58:48.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfb8ff9640 1 -- 172.21.15.37:0/1926816209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfb400d860 con 0x7fcfbc15b950 2024-04-25T08:58:48.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 -- 172.21.15.37:0/1926816209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc0c6630 con 0x7fcfbc15b950 2024-04-25T08:58:48.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfb8ff9640 1 -- 172.21.15.37:0/1926816209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb400dc50 con 0x7fcfbc15b950 2024-04-25T08:58:48.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.952+0000 7fcfc2920640 1 -- 172.21.15.37:0/1926816209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfbc0c6040 con 0x7fcfbc15b950 2024-04-25T08:58:48.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.956+0000 7fcfc2920640 1 -- 172.21.15.37:0/1926816209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfbc04efa0 con 0x7fcfbc15b950 2024-04-25T08:58:48.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.956+0000 7fcfb8ff9640 1 -- 172.21.15.37:0/1926816209 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcfb401d070 con 0x7fcfbc15b950 2024-04-25T08:58:48.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.956+0000 7fcfb8ff9640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcfa8066f60 0x7fcfa8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.956+0000 7fcfb8ff9640 1 -- 172.21.15.37:0/1926816209 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcfb4085690 con 0x7fcfbc15b950 2024-04-25T08:58:48.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.956+0000 7fcfbbfff640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcfa8066f60 0x7fcfa8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.956+0000 7fcfbbfff640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcfa8066f60 0x7fcfa8069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc166730 tx=0x7fcfac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:48.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:48.960+0000 7fcfb8ff9640 1 -- 172.21.15.37:0/1926816209 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcfb40569b0 con 0x7fcfbc15b950 2024-04-25T08:58:49.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.176+0000 7fcfc2920640 1 -- 172.21.15.37:0/1926816209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcfbc0d13a0 con 0x7fcfbc15b950 2024-04-25T08:58:49.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.176+0000 7fcfb8ff9640 1 -- 172.21.15.37:0/1926816209 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fcfb40562b0 con 0x7fcfbc15b950 2024-04-25T08:58:49.169 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:49.169 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:58:49.169 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpm2pLfABAAJbn2dGKdl8SHS8pApIWC1Q==", 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmqB+jDxAAmRnBzN3vqO8HIf/cd3QoJQ==", 2024-04-25T08:58:49.170 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmiXRqCRAACtY8cFxwqOT4WNzj9WmeKA==", 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmX3ARGBAAW/ZUr85NsKtd8HzxE+Yefw==", 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCSGSpmwhMiJhAAbmQz36LujHeYXaKIOdLLSA==", 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.172 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCVGSpm/OLPKxAA9ZmrrWxvt4QDj2lHATnzSg==", 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.173 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCYGSpmq7tEMxAA/wjwgkrtnTkrMoC6G+YIlQ==", 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCbGSpmReB1ORAAkXje/r11I3ArZHErIvijgw==", 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:58:49.174 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQChGSpm4XfLORAAGj+R1Nk/EGmvEdpt8kQJlg==", 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQClGSpmd5f1ARAAKF3XG//v3Uqx4bckcD4JMw==", 2024-04-25T08:58:49.175 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.176 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:49.176 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:58:49.177 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.177 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-04-25T08:58:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCoGSpmi9OECBAAJBskMWGCNlAaej9xUc6g8w==", 2024-04-25T08:58:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:58:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCrGSpmunQiDhAA3UZo2vzT44HEYUBUia75NA==", 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA==", 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmu8UoJhAAyutD5hzKLZmr57WuAWScuQ==", 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmGVVdIxAAY7/VOuqF5OoomqqVxE1Ifw==", 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:58:49.181 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmjFNDKxAAmo6DB9W17TZaGlvkdqkSuQ==", 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.182 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm+mGgJRAA9oTRTlZpGBdCZ0/bj580Bg==", 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpmLChTORAAo6fs/9PDJ5YqqcbfHfPO9Q==", 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T08:58:49.183 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm3NLHNhAAeqSA658L6P9OcrSO5JVzLg==", 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T08:58:49.184 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:58:49.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 -- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcfa8066f60 msgr2=0x7fcfa8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcfa8066f60 0x7fcfa8069420 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc166730 tx=0x7fcfac008040 comp rx=0 tx=0).stop 2024-04-25T08:58:49.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 -- 172.21.15.37:0/1926816209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfbc15b950 msgr2=0x7fcfbc0c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfbc15b950 0x7fcfbc0c7460 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fcfb400cef0 tx=0x7fcfb400a710 comp rx=0 tx=0).stop 2024-04-25T08:58:49.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 -- 172.21.15.37:0/1926816209 shutdown_connections 2024-04-25T08:58:49.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcfa8066f60 0x7fcfa8069420 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcfbc15b950 0x7fcfbc0c7460 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcfbc0cfb80 0x7fcfbc1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 --2- 172.21.15.37:0/1926816209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcfbc0cf420 0x7fcfbc0c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 -- 172.21.15.37:0/1926816209 >> 172.21.15.37:0/1926816209 conn(0x7fcfbc0b30b0 msgr2=0x7fcfbc0b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:49.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 -- 172.21.15.37:0/1926816209 shutdown_connections 2024-04-25T08:58:49.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.180+0000 7fcfc2920640 1 -- 172.21.15.37:0/1926816209 wait complete. 2024-04-25T08:58:49.189 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:49.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f025a565640 1 Processor -- start 2024-04-25T08:58:49.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f025a565640 1 -- start start 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f025a565640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f025406ac90 0x7f025406b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f025a565640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02540cc130 0x7f02540cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f025a565640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025406b5b0 con 0x7f025415ca90 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f025a565640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025415ff70 con 0x7f025406ac90 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f025a565640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025415f360 con 0x7f02540cc130 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f0253fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f025406ac90 0x7f025406b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f0253fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f025406ac90 0x7f025406b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56508/0 (socket says 172.21.15.37:56508) 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f0253fff640 1 -- 172.21.15.37:0/2333430510 learned_addr learned my addr 172.21.15.37:0/2333430510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f0258b02640 1 --2- 172.21.15.37:0/2333430510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02540cc130 0x7f02540cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f0253fff640 1 -- 172.21.15.37:0/2333430510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02540cc130 msgr2=0x7f02540cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f0253fff640 1 --2- 172.21.15.37:0/2333430510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02540cc130 0x7f02540cf2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f0253fff640 1 -- 172.21.15.37:0/2333430510 >> v1:172.21.15.203:6789/0 conn(0x7f025415ca90 legacy=0x7f02540d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f0253fff640 1 -- 172.21.15.37:0/2333430510 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025415f640 con 0x7f025406ac90 2024-04-25T08:58:49.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.324+0000 7f0253fff640 1 --2- 172.21.15.37:0/2333430510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f025406ac90 0x7f025406b070 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f0244009870 tx=0x7f024401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55a99e3a0f00c93b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0252ffd640 1 -- 172.21.15.37:0/2333430510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02440aa610 con 0x7f025406ac90 2024-04-25T08:58:49.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0252ffd640 1 -- 172.21.15.37:0/2333430510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02440af070 con 0x7f025406ac90 2024-04-25T08:58:49.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0252ffd640 1 -- 172.21.15.37:0/2333430510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02440aa9d0 con 0x7f025406ac90 2024-04-25T08:58:49.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 -- 172.21.15.37:0/2333430510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f025406ac90 msgr2=0x7f025406b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 --2- 172.21.15.37:0/2333430510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f025406ac90 0x7f025406b070 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f0244009870 tx=0x7f024401c4f0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 -- 172.21.15.37:0/2333430510 shutdown_connections 2024-04-25T08:58:49.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 --2- 172.21.15.37:0/2333430510 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02540cc130 0x7f02540cf2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 --2- 172.21.15.37:0/2333430510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f025406ac90 0x7f025406b070 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 -- 172.21.15.37:0/2333430510 >> 172.21.15.37:0/2333430510 conn(0x7f0254099a70 msgr2=0x7f02540d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:49.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 -- 172.21.15.37:0/2333430510 shutdown_connections 2024-04-25T08:58:49.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 -- 172.21.15.37:0/2333430510 wait complete. 2024-04-25T08:58:49.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 Processor -- start 2024-04-25T08:58:49.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 -- start start 2024-04-25T08:58:49.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f025406ac90 0x7f025414c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02540cc130 0x7f02540c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f025415ca90 0x7f0254154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0254159750 con 0x7f025415ca90 2024-04-25T08:58:49.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02541595d0 con 0x7f02540cc130 2024-04-25T08:58:49.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f025a565640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02540d0040 con 0x7f025406ac90 2024-04-25T08:58:49.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0258b02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f025415ca90 0x7f0254154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0258b02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f025415ca90 0x7f0254154a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56518/0 (socket says 172.21.15.37:56518) 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0258b02640 1 -- 172.21.15.37:0/2231307339 learned_addr learned my addr 172.21.15.37:0/2231307339 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0253fff640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f025406ac90 0x7f025414c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f02537fe640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02540cc130 0x7f02540c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0258b02640 1 -- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f025406ac90 msgr2=0x7f025414c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0258b02640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f025406ac90 0x7f025414c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0258b02640 1 -- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02540cc130 msgr2=0x7f02540c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0258b02640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02540cc130 0x7f02540c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0258b02640 1 -- 172.21.15.37:0/2231307339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025414cf20 con 0x7f025415ca90 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f02537fe640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02540cc130 0x7f02540c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.328+0000 7f0258b02640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f025415ca90 0x7f0254154a30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f024c007de0 tx=0x7f024c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.332+0000 7f0250ff9640 1 -- 172.21.15.37:0/2231307339 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c014620 con 0x7f025415ca90 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.332+0000 7f0250ff9640 1 -- 172.21.15.37:0/2231307339 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c019070 con 0x7f025415ca90 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.332+0000 7f0250ff9640 1 -- 172.21.15.37:0/2231307339 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c014a20 con 0x7f025415ca90 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.332+0000 7f025a565640 1 -- 172.21.15.37:0/2231307339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02540c4b30 con 0x7f025415ca90 2024-04-25T08:58:49.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.332+0000 7f025a565640 1 -- 172.21.15.37:0/2231307339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02540c50f0 con 0x7f025415ca90 2024-04-25T08:58:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.332+0000 7f025a565640 1 -- 172.21.15.37:0/2231307339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025404efa0 con 0x7f025415ca90 2024-04-25T08:58:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.332+0000 7f0250ff9640 1 -- 172.21.15.37:0/2231307339 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f024c014bc0 con 0x7f025415ca90 2024-04-25T08:58:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.332+0000 7f0250ff9640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0240066f60 0x7f0240069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.332+0000 7f0250ff9640 1 -- 172.21.15.37:0/2231307339 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f024c085840 con 0x7f025415ca90 2024-04-25T08:58:49.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.332+0000 7f0253fff640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0240066f60 0x7f0240069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.332+0000 7f0253fff640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0240066f60 0x7f0240069420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f02440ae000 tx=0x7f02440a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.336+0000 7f0250ff9640 1 -- 172.21.15.37:0/2231307339 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f024c056970 con 0x7f025415ca90 2024-04-25T08:58:49.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.500+0000 7f025a565640 1 -- 172.21.15.37:0/2231307339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02540b6480 con 0x7f025415ca90 2024-04-25T08:58:49.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.500+0000 7f0250ff9640 1 -- 172.21.15.37:0/2231307339 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+13129 (secure 0 0 0) 0x7f024c056270 con 0x7f025415ca90 2024-04-25T08:58:49.492 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:49.492 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:58:48.788339+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:49.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.500+0000 7f025a565640 1 -- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0240066f60 msgr2=0x7f0240069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.500+0000 7f025a565640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0240066f60 0x7f0240069420 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f02440ae000 tx=0x7f02440a7000 comp rx=0 tx=0).stop 2024-04-25T08:58:49.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.500+0000 7f025a565640 1 -- 172.21.15.37:0/2231307339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f025415ca90 msgr2=0x7f0254154a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.500+0000 7f025a565640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f025415ca90 0x7f0254154a30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f024c007de0 tx=0x7f024c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.504+0000 7f025a565640 1 -- 172.21.15.37:0/2231307339 shutdown_connections 2024-04-25T08:58:49.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.504+0000 7f025a565640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0240066f60 0x7f0240069420 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.504+0000 7f025a565640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f025415ca90 0x7f0254154a30 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.504+0000 7f025a565640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02540cc130 0x7f02540c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.504+0000 7f025a565640 1 --2- 172.21.15.37:0/2231307339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f025406ac90 0x7f025414c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.504+0000 7f025a565640 1 -- 172.21.15.37:0/2231307339 >> 172.21.15.37:0/2231307339 conn(0x7f0254099a70 msgr2=0x7f025415a490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.504+0000 7f025a565640 1 -- 172.21.15.37:0/2231307339 shutdown_connections 2024-04-25T08:58:49.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.504+0000 7f025a565640 1 -- 172.21.15.37:0/2231307339 wait complete. 2024-04-25T08:58:49.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:58:49.512 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:58:49.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f15251d3640 1 Processor -- start 2024-04-25T08:58:49.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f15251d3640 1 -- start start 2024-04-25T08:58:49.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f15251d3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f152015b800 0x7f15200cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f15251d3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f15200cfad0 0x7f1520072000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f15251d3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15200cbd20 con 0x7f152015b420 2024-04-25T08:58:49.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f15251d3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15200cff20 con 0x7f15200cfad0 2024-04-25T08:58:49.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f15251d3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15200cc930 con 0x7f152015b800 2024-04-25T08:58:49.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f1517fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f152015b800 0x7f15200cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f151f59e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f15200cfad0 0x7f1520072000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f151ed9d640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f152015b420 0x7f15200ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:60248/0 (socket says 172.21.15.37:60248) 2024-04-25T08:58:49.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f151ed9d640 1 -- 172.21.15.37:0/2766930475 learned_addr learned my addr 172.21.15.37:0/2766930475 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:49.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f1517fff640 1 --2- 172.21.15.37:0/2766930475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f152015b800 0x7f15200cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:44106/0 (socket says 172.21.15.37:44106) 2024-04-25T08:58:49.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f151f59e640 1 -- 172.21.15.37:0/2766930475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f152015b800 msgr2=0x7f15200cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:49.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f151f59e640 1 --2- 172.21.15.37:0/2766930475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f152015b800 0x7f15200cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f151f59e640 1 -- 172.21.15.37:0/2766930475 >> v1:172.21.15.203:6789/0 conn(0x7f152015b420 legacy=0x7f15200ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f151f59e640 1 -- 172.21.15.37:0/2766930475 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200cc000 con 0x7f15200cfad0 2024-04-25T08:58:49.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.652+0000 7f1517fff640 1 --2- 172.21.15.37:0/2766930475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f152015b800 0x7f15200cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:49.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f151f59e640 1 --2- 172.21.15.37:0/2766930475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15200cfad0 0x7f1520072000 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1518009d90 tx=0x7f151801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b860572c9877c2ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f151e59c640 1 -- 172.21.15.37:0/2766930475 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180b2070 con 0x7f15200cfad0 2024-04-25T08:58:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f151e59c640 1 -- 172.21.15.37:0/2766930475 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15180a5920 con 0x7f15200cfad0 2024-04-25T08:58:49.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f151e59c640 1 -- 172.21.15.37:0/2766930475 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180a5cf0 con 0x7f15200cfad0 2024-04-25T08:58:49.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f15251d3640 1 -- 172.21.15.37:0/2766930475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15200cfad0 msgr2=0x7f1520072000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f15251d3640 1 --2- 172.21.15.37:0/2766930475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15200cfad0 0x7f1520072000 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1518009d90 tx=0x7f151801c970 comp rx=0 tx=0).stop 2024-04-25T08:58:49.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f15251d3640 1 -- 172.21.15.37:0/2766930475 shutdown_connections 2024-04-25T08:58:49.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f15251d3640 1 --2- 172.21.15.37:0/2766930475 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15200cfad0 0x7f1520072000 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f15251d3640 1 --2- 172.21.15.37:0/2766930475 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f152015b800 0x7f15200cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f15251d3640 1 -- 172.21.15.37:0/2766930475 >> 172.21.15.37:0/2766930475 conn(0x7f152006ae60 msgr2=0x7f152006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f15251d3640 1 -- 172.21.15.37:0/2766930475 shutdown_connections 2024-04-25T08:58:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f15251d3640 1 -- 172.21.15.37:0/2766930475 wait complete. 2024-04-25T08:58:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f15251d3640 1 Processor -- start 2024-04-25T08:58:49.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.656+0000 7f15251d3640 1 -- start start 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15251d3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15200cfad0 0x7f152015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15251d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f152015b420 0x7f1520098a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15251d3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152015b800 0x7f15200ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15251d3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200caf30 con 0x7f152015b420 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15251d3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200cadb0 con 0x7f152015b800 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15251d3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1520072d50 con 0x7f15200cfad0 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f1517fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f152015b420 0x7f1520098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f1517fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f152015b420 0x7f1520098a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56520/0 (socket says 172.21.15.37:56520) 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f1517fff640 1 -- 172.21.15.37:0/2320759237 learned_addr learned my addr 172.21.15.37:0/2320759237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f151ed9d640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15200cfad0 0x7f152015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f151f59e640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152015b800 0x7f15200ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f1517fff640 1 -- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15200cfad0 msgr2=0x7f152015e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f1517fff640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15200cfad0 0x7f152015e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f1517fff640 1 -- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152015b800 msgr2=0x7f15200ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f1517fff640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152015b800 0x7f15200ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f1517fff640 1 -- 172.21.15.37:0/2320759237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200cd6b0 con 0x7f152015b420 2024-04-25T08:58:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f151f59e640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152015b800 0x7f15200ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f1517fff640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f152015b420 0x7f1520098a20 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f150800e9b0 tx=0x7f150800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15177fe640 1 -- 172.21.15.37:0/2320759237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1508017070 con 0x7f152015b420 2024-04-25T08:58:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15177fe640 1 -- 172.21.15.37:0/2320759237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150800c9a0 con 0x7f152015b420 2024-04-25T08:58:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15251d3640 1 -- 172.21.15.37:0/2320759237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15200cd990 con 0x7f152015b420 2024-04-25T08:58:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15177fe640 1 -- 172.21.15.37:0/2320759237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150800cd90 con 0x7f152015b420 2024-04-25T08:58:49.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15251d3640 1 -- 172.21.15.37:0/2320759237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15200cdf50 con 0x7f152015b420 2024-04-25T08:58:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15251d3640 1 -- 172.21.15.37:0/2320759237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152004efa0 con 0x7f152015b420 2024-04-25T08:58:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15177fe640 1 -- 172.21.15.37:0/2320759237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f150801a070 con 0x7f152015b420 2024-04-25T08:58:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15177fe640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1500066f60 0x7f1500069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.660+0000 7f15177fe640 1 -- 172.21.15.37:0/2320759237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1508084d10 con 0x7f152015b420 2024-04-25T08:58:49.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.664+0000 7f151ed9d640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1500066f60 0x7f1500069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.664+0000 7f151ed9d640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1500066f60 0x7f1500069420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f152015e0f0 tx=0x7f1510007e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.664+0000 7f15177fe640 1 -- 172.21.15.37:0/2320759237 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1508055890 con 0x7f152015b420 2024-04-25T08:58:49.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 -- 172.21.15.37:0/2320759237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1520072210 con 0x7f152015b420 2024-04-25T08:58:49.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15177fe640 1 -- 172.21.15.37:0/2320759237 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1508059930 con 0x7f152015b420 2024-04-25T08:58:49.819 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:58:49.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 -- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1500066f60 msgr2=0x7f1500069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1500066f60 0x7f1500069420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f152015e0f0 tx=0x7f1510007e90 comp rx=0 tx=0).stop 2024-04-25T08:58:49.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 -- 172.21.15.37:0/2320759237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f152015b420 msgr2=0x7f1520098a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f152015b420 0x7f1520098a20 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f150800e9b0 tx=0x7f150800ee70 comp rx=0 tx=0).stop 2024-04-25T08:58:49.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 -- 172.21.15.37:0/2320759237 shutdown_connections 2024-04-25T08:58:49.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1500066f60 0x7f1500069420 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f152015b800 0x7f15200ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f152015b420 0x7f1520098a20 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 --2- 172.21.15.37:0/2320759237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15200cfad0 0x7f152015e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 -- 172.21.15.37:0/2320759237 >> 172.21.15.37:0/2320759237 conn(0x7f152006ae60 msgr2=0x7f15200c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:49.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 -- 172.21.15.37:0/2320759237 shutdown_connections 2024-04-25T08:58:49.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.828+0000 7f15251d3640 1 -- 172.21.15.37:0/2320759237 wait complete. 2024-04-25T08:58:49.838 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:58:49.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff5739a1640 1 Processor -- start 2024-04-25T08:58:49.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff5739a1640 1 -- start start 2024-04-25T08:58:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff5739a1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff56c0b0a30 0x7ff56c0cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff5739a1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff56c09a180 0x7ff56c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff5739a1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56c0b0e10 con 0x7ff56c06f4a0 2024-04-25T08:58:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff5739a1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56c0d0d90 con 0x7ff56c0b0a30 2024-04-25T08:58:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff5739a1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c0d0180 con 0x7ff56c09a180 2024-04-25T08:58:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff57173d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff56c0b0a30 0x7ff56c0cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff57173d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff56c0b0a30 0x7ff56c0cf890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56530/0 (socket says 172.21.15.37:56530) 2024-04-25T08:58:49.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff57173d640 1 -- 172.21.15.37:0/2307627678 learned_addr learned my addr 172.21.15.37:0/2307627678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:49.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff571f3e640 1 --2- 172.21.15.37:0/2307627678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff56c09a180 0x7ff56c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff57173d640 1 -- 172.21.15.37:0/2307627678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff56c09a180 msgr2=0x7ff56c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:49.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff57173d640 1 --2- 172.21.15.37:0/2307627678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff56c09a180 0x7ff56c15d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff57173d640 1 -- 172.21.15.37:0/2307627678 >> v1:172.21.15.203:6789/0 conn(0x7ff56c06f4a0 legacy=0x7ff56c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.972+0000 7ff57173d640 1 -- 172.21.15.37:0/2307627678 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c0d0460 con 0x7ff56c0b0a30 2024-04-25T08:58:49.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff57173d640 1 --2- 172.21.15.37:0/2307627678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56c0b0a30 0x7ff56c0cf890 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ff568009870 tx=0x7ff56801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234ea59f737d261f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff563fff640 1 -- 172.21.15.37:0/2307627678 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5680a95f0 con 0x7ff56c0b0a30 2024-04-25T08:58:49.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff563fff640 1 -- 172.21.15.37:0/2307627678 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5680a9790 con 0x7ff56c0b0a30 2024-04-25T08:58:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff563fff640 1 -- 172.21.15.37:0/2307627678 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5680a9b60 con 0x7ff56c0b0a30 2024-04-25T08:58:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 -- 172.21.15.37:0/2307627678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56c0b0a30 msgr2=0x7ff56c0cf890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 --2- 172.21.15.37:0/2307627678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56c0b0a30 0x7ff56c0cf890 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ff568009870 tx=0x7ff56801c530 comp rx=0 tx=0).stop 2024-04-25T08:58:49.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 -- 172.21.15.37:0/2307627678 shutdown_connections 2024-04-25T08:58:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 --2- 172.21.15.37:0/2307627678 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff56c09a180 0x7ff56c15d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 --2- 172.21.15.37:0/2307627678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56c0b0a30 0x7ff56c0cf890 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 -- 172.21.15.37:0/2307627678 >> 172.21.15.37:0/2307627678 conn(0x7ff56c0b30b0 msgr2=0x7ff56c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 -- 172.21.15.37:0/2307627678 shutdown_connections 2024-04-25T08:58:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 -- 172.21.15.37:0/2307627678 wait complete. 2024-04-25T08:58:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 Processor -- start 2024-04-25T08:58:49.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 -- start start 2024-04-25T08:58:49.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56c06f4a0 0x7ff56c0cb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff56c09a180 0x7ff56c0c6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff56c0b0a30 0x7ff56c0c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c0cbd90 con 0x7ff56c06f4a0 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c0cbc10 con 0x7ff56c09a180 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff5739a1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c0cbf10 con 0x7ff56c0b0a30 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff57173d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56c06f4a0 0x7ff56c0cb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff57173d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56c06f4a0 0x7ff56c0cb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56544/0 (socket says 172.21.15.37:56544) 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff57173d640 1 -- 172.21.15.37:0/4278668394 learned_addr learned my addr 172.21.15.37:0/4278668394 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff571f3e640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff56c0b0a30 0x7ff56c0c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff57173d640 1 -- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff56c0b0a30 msgr2=0x7ff56c0c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff570f3c640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff56c09a180 0x7ff56c0c6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff57173d640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff56c0b0a30 0x7ff56c0c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff57173d640 1 -- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff56c09a180 msgr2=0x7ff56c0c6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff57173d640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff56c09a180 0x7ff56c0c6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.976+0000 7ff57173d640 1 -- 172.21.15.37:0/4278668394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c0c6ae0 con 0x7ff56c06f4a0 2024-04-25T08:58:49.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff57173d640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56c06f4a0 0x7ff56c0cb190 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff56801abe0 tx=0x7ff5680192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff561ffb640 1 -- 172.21.15.37:0/4278668394 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5680b1800 con 0x7ff56c06f4a0 2024-04-25T08:58:49.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff561ffb640 1 -- 172.21.15.37:0/4278668394 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5680b19c0 con 0x7ff56c06f4a0 2024-04-25T08:58:49.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff5739a1640 1 -- 172.21.15.37:0/4278668394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56c0c6dc0 con 0x7ff56c06f4a0 2024-04-25T08:58:49.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff561ffb640 1 -- 172.21.15.37:0/4278668394 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5680b1db0 con 0x7ff56c06f4a0 2024-04-25T08:58:49.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff5739a1640 1 -- 172.21.15.37:0/4278668394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56c0c7380 con 0x7ff56c06f4a0 2024-04-25T08:58:49.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff5739a1640 1 -- 172.21.15.37:0/4278668394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56c04efa0 con 0x7ff56c06f4a0 2024-04-25T08:58:49.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff561ffb640 1 -- 172.21.15.37:0/4278668394 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff5680c43f0 con 0x7ff56c06f4a0 2024-04-25T08:58:49.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff561ffb640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff550066f60 0x7ff550069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff561ffb640 1 -- 172.21.15.37:0/4278668394 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff56811f640 con 0x7ff56c06f4a0 2024-04-25T08:58:49.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff570f3c640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff550066f60 0x7ff550069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.980+0000 7ff570f3c640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff550066f60 0x7ff550069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff55c007fd0 tx=0x7ff55c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:49.984+0000 7ff561ffb640 1 -- 172.21.15.37:0/4278668394 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff5680ad020 con 0x7ff56c06f4a0 2024-04-25T08:58:50.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 -- 172.21.15.37:0/4278668394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff56c0cfb80 con 0x7ff56c06f4a0 2024-04-25T08:58:50.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff561ffb640 1 -- 172.21.15.37:0/4278668394 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v91) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff568016150 con 0x7ff56c06f4a0 2024-04-25T08:58:50.139 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:50.139 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T08:58:50.139 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:58:50.139 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T08:58:50.139 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T08:58:50.139 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:58:50.139 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T08:58:50.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 -- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff550066f60 msgr2=0x7ff550069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff550066f60 0x7ff550069420 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff55c007fd0 tx=0x7ff55c009040 comp rx=0 tx=0).stop 2024-04-25T08:58:50.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 -- 172.21.15.37:0/4278668394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56c06f4a0 msgr2=0x7ff56c0cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56c06f4a0 0x7ff56c0cb190 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff56801abe0 tx=0x7ff5680192d0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 -- 172.21.15.37:0/4278668394 shutdown_connections 2024-04-25T08:58:50.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff550066f60 0x7ff550069420 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff56c0b0a30 0x7ff56c0c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff56c09a180 0x7ff56c0c6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 --2- 172.21.15.37:0/4278668394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff56c06f4a0 0x7ff56c0cb190 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 -- 172.21.15.37:0/4278668394 >> 172.21.15.37:0/4278668394 conn(0x7ff56c0b30b0 msgr2=0x7ff56c0af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 -- 172.21.15.37:0/4278668394 shutdown_connections 2024-04-25T08:58:50.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.148+0000 7ff5739a1640 1 -- 172.21.15.37:0/4278668394 wait complete. 2024-04-25T08:58:50.156 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:58:50.156 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:58:50.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.292+0000 7f7278908640 1 Processor -- start 2024-04-25T08:58:50.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.292+0000 7f7278908640 1 -- start start 2024-04-25T08:58:50.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.292+0000 7f7278908640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f727415a3c0 0x7f72740ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.292+0000 7f7278908640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f727415a7a0 0x7f72740b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.292+0000 7f7278908640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727415e0f0 con 0x7f727415ac30 2024-04-25T08:58:50.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.292+0000 7f7278908640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72740b24a0 con 0x7f727415a7a0 2024-04-25T08:58:50.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.292+0000 7f7278908640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727415b0b0 con 0x7f727415a3c0 2024-04-25T08:58:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.292+0000 7f7272d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f727415ac30 0x7f72740cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37192/0 (socket says 172.21.15.37:37192) 2024-04-25T08:58:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.292+0000 7f7272d9d640 1 -- 172.21.15.37:0/318929755 learned_addr learned my addr 172.21.15.37:0/318929755 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.292+0000 7f7271d9b640 1 --2- 172.21.15.37:0/318929755 >> v2:172.21.15.37:3300/0 conn(0x7f727415a7a0 0x7f72740b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f727259c640 1 --2- 172.21.15.37:0/318929755 >> v2:172.21.15.203:3300/0 conn(0x7f727415a3c0 0x7f72740ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f727159a640 1 -- 172.21.15.37:0/318929755 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646777478 0 0) 0x7f727415e0f0 con 0x7f727415ac30 2024-04-25T08:58:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f727159a640 1 -- 172.21.15.37:0/318929755 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f725c003530 con 0x7f727415ac30 2024-04-25T08:58:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f727159a640 1 -- 172.21.15.37:0/318929755 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2730826803 0 0) 0x7f725c003530 con 0x7f727415ac30 2024-04-25T08:58:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7271d9b640 1 -- 172.21.15.37:0/318929755 >> v2:172.21.15.203:3300/0 conn(0x7f727415a3c0 msgr2=0x7f72740ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7271d9b640 1 --2- 172.21.15.37:0/318929755 >> v2:172.21.15.203:3300/0 conn(0x7f727415a3c0 0x7f72740ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7271d9b640 1 -- 172.21.15.37:0/318929755 >> v1:172.21.15.37:6789/0 conn(0x7f727415ac30 legacy=0x7f72740cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7271d9b640 1 -- 172.21.15.37:0/318929755 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727415b390 con 0x7f727415a7a0 2024-04-25T08:58:50.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7271d9b640 1 --2- 172.21.15.37:0/318929755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f727415a7a0 0x7f72740b27a0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f72740cbcc0 tx=0x7f726801c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca5f3c813bb3238 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f727159a640 1 -- 172.21.15.37:0/318929755 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72680b1070 con 0x7f727415a7a0 2024-04-25T08:58:50.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f727159a640 1 -- 172.21.15.37:0/318929755 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72680ac440 con 0x7f727415a7a0 2024-04-25T08:58:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f727159a640 1 -- 172.21.15.37:0/318929755 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c001a00 con 0x7f727415a7a0 2024-04-25T08:58:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7278908640 1 -- 172.21.15.37:0/318929755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f727415a7a0 msgr2=0x7f72740b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7278908640 1 --2- 172.21.15.37:0/318929755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f727415a7a0 0x7f72740b27a0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f72740cbcc0 tx=0x7f726801c370 comp rx=0 tx=0).stop 2024-04-25T08:58:50.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7278908640 1 -- 172.21.15.37:0/318929755 shutdown_connections 2024-04-25T08:58:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7278908640 1 --2- 172.21.15.37:0/318929755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f727415a7a0 0x7f72740b27a0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7278908640 1 --2- 172.21.15.37:0/318929755 >> v2:172.21.15.203:3300/0 conn(0x7f727415a3c0 0x7f72740ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7278908640 1 -- 172.21.15.37:0/318929755 >> 172.21.15.37:0/318929755 conn(0x7f727406ae60 msgr2=0x7f727406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7278908640 1 -- 172.21.15.37:0/318929755 shutdown_connections 2024-04-25T08:58:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.296+0000 7f7278908640 1 -- 172.21.15.37:0/318929755 wait complete. 2024-04-25T08:58:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7278908640 1 Processor -- start 2024-04-25T08:58:50.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7278908640 1 -- start start 2024-04-25T08:58:50.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7278908640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f727415a3c0 0x7f7274071020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7278908640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f727415a7a0 0x7f72740ac420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7278908640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f727415ac30 0x7f72740c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7278908640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727415edb0 con 0x7f727415a3c0 2024-04-25T08:58:50.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7278908640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72740b2ef0 con 0x7f727415ac30 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7278908640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727415ef30 con 0x7f727415a7a0 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f727259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f727415a3c0 0x7f7274071020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f727259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f727415a3c0 0x7f7274071020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56564/0 (socket says 172.21.15.37:56564) 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f727259c640 1 -- 172.21.15.37:0/3663356932 learned_addr learned my addr 172.21.15.37:0/3663356932 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f727259c640 1 -- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f727415a7a0 msgr2=0x7f72740ac420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f727259c640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f727415a7a0 0x7f72740ac420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f727259c640 1 -- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f727415ac30 msgr2=0x7f72740c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7272d9d640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f727415ac30 0x7f72740c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f727259c640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f727415ac30 0x7f72740c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f727259c640 1 -- 172.21.15.37:0/3663356932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7274071670 con 0x7f727415a3c0 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f727259c640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f727415a3c0 0x7f7274071020 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f726400e930 tx=0x7f726400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7262ffd640 1 -- 172.21.15.37:0/3663356932 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7264017070 con 0x7f727415a3c0 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7262ffd640 1 -- 172.21.15.37:0/3663356932 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726400ca00 con 0x7f727415a3c0 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7278908640 1 -- 172.21.15.37:0/3663356932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7274071950 con 0x7f727415a3c0 2024-04-25T08:58:50.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7262ffd640 1 -- 172.21.15.37:0/3663356932 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726400cdb0 con 0x7f727415a3c0 2024-04-25T08:58:50.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7278908640 1 -- 172.21.15.37:0/3663356932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7274071f10 con 0x7f727415a3c0 2024-04-25T08:58:50.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7278908640 1 -- 172.21.15.37:0/3663356932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7274072350 con 0x7f727415a3c0 2024-04-25T08:58:50.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.300+0000 7f7262ffd640 1 -- 172.21.15.37:0/3663356932 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f726401a070 con 0x7f727415a3c0 2024-04-25T08:58:50.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.304+0000 7f7262ffd640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7254066f60 0x7f7254069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.304+0000 7f7262ffd640 1 -- 172.21.15.37:0/3663356932 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f7264084f40 con 0x7f727415a3c0 2024-04-25T08:58:50.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.304+0000 7f7271d9b640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7254066f60 0x7f7254069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.304+0000 7f7271d9b640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7254066f60 0x7f7254069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f72680ae000 tx=0x7f7268004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.308+0000 7f7262ffd640 1 -- 172.21.15.37:0/3663356932 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7264056130 con 0x7f727415a3c0 2024-04-25T08:58:50.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:50.468+0000 7f7278908640 1 -- 172.21.15.37:0/3663356932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7274067c30 con 0x7f727415a3c0 2024-04-25T08:58:51.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.456+0000 7f7262ffd640 1 -- 172.21.15.37:0/3663356932 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v92) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7264055a30 con 0x7f727415a3c0 2024-04-25T08:58:51.450 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T08:58:51.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.460+0000 7f7278908640 1 -- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7254066f60 msgr2=0x7f7254069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.460+0000 7f7278908640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7254066f60 0x7f7254069420 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f72680ae000 tx=0x7f7268004740 comp rx=0 tx=0).stop 2024-04-25T08:58:51.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.460+0000 7f7278908640 1 -- 172.21.15.37:0/3663356932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f727415a3c0 msgr2=0x7f7274071020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.460+0000 7f7278908640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f727415a3c0 0x7f7274071020 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f726400e930 tx=0x7f726400edf0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.460+0000 7f7278908640 1 -- 172.21.15.37:0/3663356932 shutdown_connections 2024-04-25T08:58:51.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.460+0000 7f7278908640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7254066f60 0x7f7254069420 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.460+0000 7f7278908640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f727415ac30 0x7f72740c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.460+0000 7f7278908640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f727415a7a0 0x7f72740ac420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.460+0000 7f7278908640 1 --2- 172.21.15.37:0/3663356932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f727415a3c0 0x7f7274071020 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.460+0000 7f7278908640 1 -- 172.21.15.37:0/3663356932 >> 172.21.15.37:0/3663356932 conn(0x7f727406ae60 msgr2=0x7f7274153c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.464+0000 7f7278908640 1 -- 172.21.15.37:0/3663356932 shutdown_connections 2024-04-25T08:58:51.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.464+0000 7f7278908640 1 -- 172.21.15.37:0/3663356932 wait complete. 2024-04-25T08:58:51.478 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:58:51.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe8416f3640 1 Processor -- start 2024-04-25T08:58:51.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe8416f3640 1 -- start start 2024-04-25T08:58:51.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe8416f3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe83c0b3cf0 0x7fe83c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe8416f3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe83c0cfad0 0x7fe83c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe8416f3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c0cbf10 con 0x7fe83c15e4f0 2024-04-25T08:58:51.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe8416f3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c0cff40 con 0x7fe83c0cfad0 2024-04-25T08:58:51.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe8416f3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c0cc1f0 con 0x7fe83c0b3cf0 2024-04-25T08:58:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe83a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe83c0cfad0 0x7fe83c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe83a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe83c0cfad0 0x7fe83c15dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56576/0 (socket says 172.21.15.37:56576) 2024-04-25T08:58:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe83a7fc640 1 -- 172.21.15.37:0/480149519 learned_addr learned my addr 172.21.15.37:0/480149519 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe83affd640 1 --2- 172.21.15.37:0/480149519 >> v2:172.21.15.203:3300/0 conn(0x7fe83c0b3cf0 0x7fe83c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe83a7fc640 1 -- 172.21.15.37:0/480149519 >> v2:172.21.15.203:3300/0 conn(0x7fe83c0b3cf0 msgr2=0x7fe83c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe83a7fc640 1 --2- 172.21.15.37:0/480149519 >> v2:172.21.15.203:3300/0 conn(0x7fe83c0b3cf0 0x7fe83c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe83a7fc640 1 -- 172.21.15.37:0/480149519 >> v1:172.21.15.203:6789/0 conn(0x7fe83c15e4f0 legacy=0x7fe83c161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe83a7fc640 1 -- 172.21.15.37:0/480149519 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c0cc4d0 con 0x7fe83c0cfad0 2024-04-25T08:58:51.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.624+0000 7fe83affd640 1 --2- 172.21.15.37:0/480149519 >> v2:172.21.15.203:3300/0 conn(0x7fe83c0b3cf0 0x7fe83c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:51.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe83a7fc640 1 --2- 172.21.15.37:0/480149519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe83c0cfad0 0x7fe83c15dfb0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0cbd40 tx=0x7fe83001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74511c70a1018993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe839ffb640 1 -- 172.21.15.37:0/480149519 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300b0070 con 0x7fe83c0cfad0 2024-04-25T08:58:51.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe839ffb640 1 -- 172.21.15.37:0/480149519 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8300ab600 con 0x7fe83c0cfad0 2024-04-25T08:58:51.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe839ffb640 1 -- 172.21.15.37:0/480149519 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300ab9d0 con 0x7fe83c0cfad0 2024-04-25T08:58:51.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 -- 172.21.15.37:0/480149519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe83c0cfad0 msgr2=0x7fe83c15dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 --2- 172.21.15.37:0/480149519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe83c0cfad0 0x7fe83c15dfb0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0cbd40 tx=0x7fe83001c4e0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 -- 172.21.15.37:0/480149519 shutdown_connections 2024-04-25T08:58:51.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 --2- 172.21.15.37:0/480149519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe83c0cfad0 0x7fe83c15dfb0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 --2- 172.21.15.37:0/480149519 >> v2:172.21.15.203:3300/0 conn(0x7fe83c0b3cf0 0x7fe83c0cf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 -- 172.21.15.37:0/480149519 >> 172.21.15.37:0/480149519 conn(0x7fe83c06ae60 msgr2=0x7fe83c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 -- 172.21.15.37:0/480149519 shutdown_connections 2024-04-25T08:58:51.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 -- 172.21.15.37:0/480149519 wait complete. 2024-04-25T08:58:51.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 Processor -- start 2024-04-25T08:58:51.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 -- start start 2024-04-25T08:58:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe83c0b3cf0 0x7fe83c159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe83c0cfad0 0x7fe83c070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe83c15e4f0 0x7fe83c06e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0b4250 con 0x7fe83c15e4f0 2024-04-25T08:58:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0b40d0 con 0x7fe83c0b3cf0 2024-04-25T08:58:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe8416f3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c15e910 con 0x7fe83c0cfad0 2024-04-25T08:58:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe83b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe83c15e4f0 0x7fe83c06e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe83b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe83c15e4f0 0x7fe83c06e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56590/0 (socket says 172.21.15.37:56590) 2024-04-25T08:58:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe83b7fe640 1 -- 172.21.15.37:0/965792143 learned_addr learned my addr 172.21.15.37:0/965792143 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe83affd640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe83c0b3cf0 0x7fe83c159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe83a7fc640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe83c0cfad0 0x7fe83c070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe83b7fe640 1 -- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe83c0cfad0 msgr2=0x7fe83c070fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe83b7fe640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe83c0cfad0 0x7fe83c070fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe83b7fe640 1 -- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe83c0b3cf0 msgr2=0x7fe83c159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe83b7fe640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe83c0b3cf0 0x7fe83c159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.628+0000 7fe83b7fe640 1 -- 172.21.15.37:0/965792143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c158040 con 0x7fe83c15e4f0 2024-04-25T08:58:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe83b7fe640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe83c15e4f0 0x7fe83c06e290 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe83400b970 tx=0x7fe83400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe8237fe640 1 -- 172.21.15.37:0/965792143 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834019070 con 0x7fe83c15e4f0 2024-04-25T08:58:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe8237fe640 1 -- 172.21.15.37:0/965792143 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83400cb40 con 0x7fe83c15e4f0 2024-04-25T08:58:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe8416f3640 1 -- 172.21.15.37:0/965792143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c0b11a0 con 0x7fe83c15e4f0 2024-04-25T08:58:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe8237fe640 1 -- 172.21.15.37:0/965792143 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834014440 con 0x7fe83c15e4f0 2024-04-25T08:58:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe8416f3640 1 -- 172.21.15.37:0/965792143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c1585c0 con 0x7fe83c15e4f0 2024-04-25T08:58:51.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe8416f3640 1 -- 172.21.15.37:0/965792143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c04efa0 con 0x7fe83c15e4f0 2024-04-25T08:58:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe8237fe640 1 -- 172.21.15.37:0/965792143 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe83401c070 con 0x7fe83c15e4f0 2024-04-25T08:58:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe8237fe640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe828066f90 0x7fe828069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe8237fe640 1 -- 172.21.15.37:0/965792143 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fe834084d10 con 0x7fe83c15e4f0 2024-04-25T08:58:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe83affd640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe828066f90 0x7fe828069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.632+0000 7fe83affd640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe828066f90 0x7fe828069450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00a9a0 tx=0x7fe82c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.636+0000 7fe8237fe640 1 -- 172.21.15.37:0/965792143 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe83408a020 con 0x7fe83c15e4f0 2024-04-25T08:58:51.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:51.804+0000 7fe8416f3640 1 -- 172.21.15.37:0/965792143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe83c0b3060 con 0x7fe83c15e4f0 2024-04-25T08:58:52.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.460+0000 7fe8237fe640 1 -- 172.21.15.37:0/965792143 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v93) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe834055c40 con 0x7fe83c15e4f0 2024-04-25T08:58:52.454 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T08:58:52.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 -- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe828066f90 msgr2=0x7fe828069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe828066f90 0x7fe828069450 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00a9a0 tx=0x7fe82c008040 comp rx=0 tx=0).stop 2024-04-25T08:58:52.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 -- 172.21.15.37:0/965792143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe83c15e4f0 msgr2=0x7fe83c06e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe83c15e4f0 0x7fe83c06e290 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe83400b970 tx=0x7fe83400be30 comp rx=0 tx=0).stop 2024-04-25T08:58:52.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 -- 172.21.15.37:0/965792143 shutdown_connections 2024-04-25T08:58:52.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe828066f90 0x7fe828069450 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe83c15e4f0 0x7fe83c06e290 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe83c0cfad0 0x7fe83c070fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 --2- 172.21.15.37:0/965792143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe83c0b3cf0 0x7fe83c159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 -- 172.21.15.37:0/965792143 >> 172.21.15.37:0/965792143 conn(0x7fe83c06ae60 msgr2=0x7fe83c15a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 -- 172.21.15.37:0/965792143 shutdown_connections 2024-04-25T08:58:52.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.464+0000 7fe8416f3640 1 -- 172.21.15.37:0/965792143 wait complete. 2024-04-25T08:58:52.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:58:52.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f9674866640 1 Processor -- start 2024-04-25T08:58:52.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f9674866640 1 -- start start 2024-04-25T08:58:52.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f9674866640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967006f4a0 0x7f9670099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f9674866640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96700b0e10 con 0x7f96700b0a30 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f9674866640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96700d0180 con 0x7f9670099fb0 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f9674866640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96700d0d90 con 0x7f967006f4a0 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966e59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f96700b0a30 0x7f96700cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37196/0 (socket says 172.21.15.37:37196) 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966e59c640 1 -- 172.21.15.37:0/3986721330 learned_addr learned my addr 172.21.15.37:0/3986721330 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966dd9b640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967006f4a0 0x7f9670099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794240793 0 0) 0x7f96700b0e10 con 0x7f96700b0a30 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9658003530 con 0x7f96700b0a30 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1217878064 0 0) 0x7f9658003530 con 0x7f96700b0a30 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967006f4a0 msgr2=0x7f9670099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967006f4a0 0x7f9670099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 >> v1:172.21.15.203:6789/0 conn(0x7f9670099fb0 legacy=0x7f967015d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96700d0460 con 0x7f96700b0a30 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f9660001680 con 0x7f96700b0a30 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 >> v1:172.21.15.37:6789/0 conn(0x7f96700b0a30 legacy=0x7f96700cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9658003a60 0x7f9658003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967006f4a0 0x7f9658004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9658004d50 0x7f96580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700d0d90 con 0x7f9658004d50 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f965800bae0 con 0x7f9658003a60 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f965800bce0 con 0x7f967006f4a0 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966ed9d640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9658004d50 0x7f96580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966e59c640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9658003a60 0x7f9658003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966dd9b640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967006f4a0 0x7f9658004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966ed9d640 1 -- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967006f4a0 msgr2=0x7f9658004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966ed9d640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f967006f4a0 0x7f9658004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966ed9d640 1 -- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9658003a60 msgr2=0x7f9658003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966ed9d640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9658003a60 0x7f9658003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966ed9d640 1 -- 172.21.15.37:0/3986721330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9658003530 con 0x7f9658004d50 2024-04-25T08:58:52.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.628+0000 7f966e59c640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9658003a60 0x7f9658003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f966ed9d640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9658004d50 0x7f96580051d0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f9668001e40 tx=0x7f966801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f0594ed73261e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668001a90 con 0x7f9658004d50 2024-04-25T08:58:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9668003b80 con 0x7f9658004d50 2024-04-25T08:58:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f966d59a640 1 -- 172.21.15.37:0/3986721330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668005080 con 0x7f9658004d50 2024-04-25T08:58:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f9674866640 1 -- 172.21.15.37:0/3986721330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9658004d50 msgr2=0x7f96580051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f9674866640 1 --2- 172.21.15.37:0/3986721330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9658004d50 0x7f96580051d0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f9668001e40 tx=0x7f966801c760 comp rx=0 tx=0).stop 2024-04-25T08:58:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f966ed9d640 1 -- 172.21.15.37:0/3986721330 reap_dead start 2024-04-25T08:58:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f9674866640 1 -- 172.21.15.37:0/3986721330 shutdown_connections 2024-04-25T08:58:52.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f9674866640 1 -- 172.21.15.37:0/3986721330 >> 172.21.15.37:0/3986721330 conn(0x7f96700b30b0 msgr2=0x7f96700b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f9674866640 1 -- 172.21.15.37:0/3986721330 shutdown_connections 2024-04-25T08:58:52.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f9674866640 1 -- 172.21.15.37:0/3986721330 wait complete. 2024-04-25T08:58:52.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f9674866640 1 Processor -- start 2024-04-25T08:58:52.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.632+0000 7f9674866640 1 -- start start 2024-04-25T08:58:52.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f9674866640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96701f3a90 0x7f96700b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f9674866640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9670152320 0x7f96701515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f9674866640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9670151b30 0x7f96700ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f9674866640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700ccc20 con 0x7f96701f3a90 2024-04-25T08:58:52.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f9674866640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700d0a20 con 0x7f9670151b30 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f9674866640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96700ccda0 con 0x7f9670152320 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96701f3a90 0x7f96700b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96701f3a90 0x7f96700b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56610/0 (socket says 172.21.15.37:56610) 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966e59c640 1 -- 172.21.15.37:0/2124285501 learned_addr learned my addr 172.21.15.37:0/2124285501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966ed9d640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9670151b30 0x7f96700ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966dd9b640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9670152320 0x7f96701515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966e59c640 1 -- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9670152320 msgr2=0x7f96701515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966e59c640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9670152320 0x7f96701515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966e59c640 1 -- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9670151b30 msgr2=0x7f96700ca6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966e59c640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9670151b30 0x7f96700ca6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966e59c640 1 -- 172.21.15.37:0/2124285501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9670071090 con 0x7f96701f3a90 2024-04-25T08:58:52.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966dd9b640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9670152320 0x7f96701515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:52.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966e59c640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96701f3a90 0x7f96700b69d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f966000dae0 tx=0x7f96600016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f965effd640 1 -- 172.21.15.37:0/2124285501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660019070 con 0x7f96701f3a90 2024-04-25T08:58:52.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f965effd640 1 -- 172.21.15.37:0/2124285501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9660004cc0 con 0x7f96701f3a90 2024-04-25T08:58:52.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f9674866640 1 -- 172.21.15.37:0/2124285501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96700b6f10 con 0x7f96701f3a90 2024-04-25T08:58:52.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f965effd640 1 -- 172.21.15.37:0/2124285501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9660007310 con 0x7f96701f3a90 2024-04-25T08:58:52.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f9674866640 1 -- 172.21.15.37:0/2124285501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96700b4f90 con 0x7f96701f3a90 2024-04-25T08:58:52.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f9674866640 1 -- 172.21.15.37:0/2124285501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f967004efa0 con 0x7f96701f3a90 2024-04-25T08:58:52.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f965effd640 1 -- 172.21.15.37:0/2124285501 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f966001c070 con 0x7f96701f3a90 2024-04-25T08:58:52.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f965effd640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9650066f90 0x7f9650069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f965effd640 1 -- 172.21.15.37:0/2124285501 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f9660084b50 con 0x7f96701f3a90 2024-04-25T08:58:52.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.636+0000 7f966dd9b640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9650066f90 0x7f9650069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.640+0000 7f966dd9b640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9650066f90 0x7f9650069450 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f96700b6960 tx=0x7f9664008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.644+0000 7f965effd640 1 -- 172.21.15.37:0/2124285501 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f96600553e0 con 0x7f96701f3a90 2024-04-25T08:58:52.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:52.836+0000 7f9674866640 1 -- 172.21.15.37:0/2124285501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f967006fa80 con 0x7f96701f3a90 2024-04-25T08:58:53.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.524+0000 7f965effd640 1 -- 172.21.15.37:0/2124285501 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v56) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9660059480 con 0x7f96701f3a90 2024-04-25T08:58:53.525 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T08:58:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.536+0000 7f9674866640 1 -- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9650066f90 msgr2=0x7f9650069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.536+0000 7f9674866640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9650066f90 0x7f9650069450 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f96700b6960 tx=0x7f9664008040 comp rx=0 tx=0).stop 2024-04-25T08:58:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.536+0000 7f9674866640 1 -- 172.21.15.37:0/2124285501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96701f3a90 msgr2=0x7f96700b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.536+0000 7f9674866640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96701f3a90 0x7f96700b69d0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f966000dae0 tx=0x7f96600016f0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.536+0000 7f9674866640 1 -- 172.21.15.37:0/2124285501 shutdown_connections 2024-04-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.536+0000 7f9674866640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9650066f90 0x7f9650069450 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.536+0000 7f9674866640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9670151b30 0x7f96700ca6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.536+0000 7f9674866640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9670152320 0x7f96701515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.536+0000 7f9674866640 1 --2- 172.21.15.37:0/2124285501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96701f3a90 0x7f96700b69d0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.536+0000 7f9674866640 1 -- 172.21.15.37:0/2124285501 >> 172.21.15.37:0/2124285501 conn(0x7f96700b30b0 msgr2=0x7f967006fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.540+0000 7f9674866640 1 -- 172.21.15.37:0/2124285501 shutdown_connections 2024-04-25T08:58:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.540+0000 7f9674866640 1 -- 172.21.15.37:0/2124285501 wait complete. 2024-04-25T08:58:53.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.692+0000 7f5d80b59640 1 Processor -- start 2024-04-25T08:58:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d80b59640 1 -- start start 2024-04-25T08:58:53.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d80b59640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5d7c15a700 0x7f5d7c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d80b59640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5d7c0b24a0 0x7f5d7c0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d80b59640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d7c0b2090 con 0x7f5d7c0b3f50 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d80b59640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7c15f200 con 0x7f5d7c0b24a0 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d80b59640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d7c0b43e0 con 0x7f5d7c15a700 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7ad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5d7c0b24a0 0x7f5d7c0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7ad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5d7c0b24a0 0x7f5d7c0b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56624/0 (socket says 172.21.15.37:56624) 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7ad9d640 1 -- 172.21.15.37:0/3363804208 learned_addr learned my addr 172.21.15.37:0/3363804208 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7a59c640 1 --2- 172.21.15.37:0/3363804208 >> v2:172.21.15.203:3300/0 conn(0x7f5d7c15a700 0x7f5d7c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7ad9d640 1 -- 172.21.15.37:0/3363804208 >> v2:172.21.15.203:3300/0 conn(0x7f5d7c15a700 msgr2=0x7f5d7c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7ad9d640 1 --2- 172.21.15.37:0/3363804208 >> v2:172.21.15.203:3300/0 conn(0x7f5d7c15a700 0x7f5d7c15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7ad9d640 1 -- 172.21.15.37:0/3363804208 >> v1:172.21.15.203:6789/0 conn(0x7f5d7c0b3f50 legacy=0x7f5d7c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:53.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7ad9d640 1 -- 172.21.15.37:0/3363804208 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c0b46c0 con 0x7f5d7c0b24a0 2024-04-25T08:58:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7ad9d640 1 --2- 172.21.15.37:0/3363804208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d7c0b24a0 0x7f5d7c0b28f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c06c980 tx=0x7f5d7401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823b2db9ed83753f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:53.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7959a640 1 -- 172.21.15.37:0/3363804208 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d740a5620 con 0x7f5d7c0b24a0 2024-04-25T08:58:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7959a640 1 -- 172.21.15.37:0/3363804208 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d740a57c0 con 0x7f5d7c0b24a0 2024-04-25T08:58:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.696+0000 7f5d7959a640 1 -- 172.21.15.37:0/3363804208 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d740a5b90 con 0x7f5d7c0b24a0 2024-04-25T08:58:53.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 -- 172.21.15.37:0/3363804208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d7c0b24a0 msgr2=0x7f5d7c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 --2- 172.21.15.37:0/3363804208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d7c0b24a0 0x7f5d7c0b28f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c06c980 tx=0x7f5d7401c870 comp rx=0 tx=0).stop 2024-04-25T08:58:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 -- 172.21.15.37:0/3363804208 shutdown_connections 2024-04-25T08:58:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 --2- 172.21.15.37:0/3363804208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d7c0b24a0 0x7f5d7c0b28f0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 --2- 172.21.15.37:0/3363804208 >> v2:172.21.15.203:3300/0 conn(0x7f5d7c15a700 0x7f5d7c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 -- 172.21.15.37:0/3363804208 >> 172.21.15.37:0/3363804208 conn(0x7f5d7c0b11a0 msgr2=0x7f5d7c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 -- 172.21.15.37:0/3363804208 shutdown_connections 2024-04-25T08:58:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 -- 172.21.15.37:0/3363804208 wait complete. 2024-04-25T08:58:53.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 Processor -- start 2024-04-25T08:58:53.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 -- start start 2024-04-25T08:58:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d7c0b24a0 0x7f5d7c161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d7c15a700 0x7f5d7c161f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d7c152c10 0x7f5d7c140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0cd310 con 0x7f5d7c15a700 2024-04-25T08:58:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0ce600 con 0x7f5d7c152c10 2024-04-25T08:58:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7c0cd490 con 0x7f5d7c0b24a0 2024-04-25T08:58:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d7c15a700 0x7f5d7c161f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d7c15a700 0x7f5d7c161f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56636/0 (socket says 172.21.15.37:56636) 2024-04-25T08:58:53.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d79d9b640 1 -- 172.21.15.37:0/2785637223 learned_addr learned my addr 172.21.15.37:0/2785637223 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d7ad9d640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d7c152c10 0x7f5d7c140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d7a59c640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d7c0b24a0 0x7f5d7c161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d79d9b640 1 -- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d7c0b24a0 msgr2=0x7f5d7c161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d79d9b640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d7c0b24a0 0x7f5d7c161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d79d9b640 1 -- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d7c152c10 msgr2=0x7f5d7c140380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d79d9b640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d7c152c10 0x7f5d7c140380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d79d9b640 1 -- 172.21.15.37:0/2785637223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d7c163a30 con 0x7f5d7c15a700 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d7ad9d640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d7c152c10 0x7f5d7c140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d7a59c640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d7c0b24a0 0x7f5d7c161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d79d9b640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d7c15a700 0x7f5d7c161f50 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c450 tx=0x7f5d7000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d6affd640 1 -- 172.21.15.37:0/2785637223 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70012830 con 0x7f5d7c15a700 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d6affd640 1 -- 172.21.15.37:0/2785637223 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d70017070 con 0x7f5d7c15a700 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d80b59640 1 -- 172.21.15.37:0/2785637223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d7c1412e0 con 0x7f5d7c15a700 2024-04-25T08:58:53.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.700+0000 7f5d6affd640 1 -- 172.21.15.37:0/2785637223 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d70012c30 con 0x7f5d7c15a700 2024-04-25T08:58:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.704+0000 7f5d80b59640 1 -- 172.21.15.37:0/2785637223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d7c1418a0 con 0x7f5d7c15a700 2024-04-25T08:58:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.704+0000 7f5d80b59640 1 -- 172.21.15.37:0/2785637223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d7c04efa0 con 0x7f5d7c15a700 2024-04-25T08:58:53.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.704+0000 7f5d6affd640 1 -- 172.21.15.37:0/2785637223 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5d7001c460 con 0x7f5d7c15a700 2024-04-25T08:58:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.704+0000 7f5d6affd640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d5c066f60 0x7f5d5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.704+0000 7f5d6affd640 1 -- 172.21.15.37:0/2785637223 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5d70085ad0 con 0x7f5d7c15a700 2024-04-25T08:58:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.704+0000 7f5d7a59c640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d5c066f60 0x7f5d5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.704+0000 7f5d7a59c640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d5c066f60 0x7f5d5c069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c007a70 tx=0x7f5d6c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.708+0000 7f5d6affd640 1 -- 172.21.15.37:0/2785637223 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5d700569b0 con 0x7f5d7c15a700 2024-04-25T08:58:53.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.868+0000 7f5d80b59640 1 -- 172.21.15.37:0/2785637223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d7c06f260 con 0x7f5d7c15a700 2024-04-25T08:58:53.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.872+0000 7f5d6affd640 1 -- 172.21.15.37:0/2785637223 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16862 (secure 0 0 0) 0x7f5d700562b0 con 0x7f5d7c15a700 2024-04-25T08:58:53.863 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:53.863 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:58:52.842976+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:50.512578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:51.848880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:53.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 -- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d5c066f60 msgr2=0x7f5d5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d5c066f60 0x7f5d5c069420 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c007a70 tx=0x7f5d6c008040 comp rx=0 tx=0).stop 2024-04-25T08:58:53.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 -- 172.21.15.37:0/2785637223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d7c15a700 msgr2=0x7f5d7c161f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d7c15a700 0x7f5d7c161f50 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f5d7000c450 tx=0x7f5d7000c910 comp rx=0 tx=0).stop 2024-04-25T08:58:53.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 -- 172.21.15.37:0/2785637223 shutdown_connections 2024-04-25T08:58:53.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5d5c066f60 0x7f5d5c069420 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5d7c152c10 0x7f5d7c140380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d7c15a700 0x7f5d7c161f50 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 --2- 172.21.15.37:0/2785637223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5d7c0b24a0 0x7f5d7c161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 -- 172.21.15.37:0/2785637223 >> 172.21.15.37:0/2785637223 conn(0x7f5d7c0b11a0 msgr2=0x7f5d7c0cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:53.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 -- 172.21.15.37:0/2785637223 shutdown_connections 2024-04-25T08:58:53.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:53.876+0000 7f5d80b59640 1 -- 172.21.15.37:0/2785637223 wait complete. 2024-04-25T08:58:53.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:54.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ef367640 1 Processor -- start 2024-04-25T08:58:54.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ef367640 1 -- start start 2024-04-25T08:58:54.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ef367640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51e80b2c30 0x7f51e80b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ef367640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51e809a7b0 0x7f51e8099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ef367640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e8099fb0 0x7f51e80ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ef367640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e80cbec0 con 0x7f51e809a7b0 2024-04-25T08:58:54.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ef367640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e80cc1a0 con 0x7f51e80b2c30 2024-04-25T08:58:54.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ef367640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e80cc480 con 0x7f51e8099fb0 2024-04-25T08:58:54.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ec902640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51e809a7b0 0x7f51e8099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ec902640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f51e809a7b0 0x7f51e8099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56638/0 (socket says 172.21.15.37:56638) 2024-04-25T08:58:54.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ec902640 1 -- 172.21.15.37:0/1231077412 learned_addr learned my addr 172.21.15.37:0/1231077412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:54.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ed103640 1 --2- 172.21.15.37:0/1231077412 >> v2:172.21.15.203:3300/0 conn(0x7f51e80b2c30 0x7f51e80b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ed904640 1 --2- 172.21.15.37:0/1231077412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e8099fb0 0x7f51e80ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ec902640 1 -- 172.21.15.37:0/1231077412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e8099fb0 msgr2=0x7f51e80ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:54.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ec902640 1 --2- 172.21.15.37:0/1231077412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e8099fb0 0x7f51e80ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ec902640 1 -- 172.21.15.37:0/1231077412 >> v2:172.21.15.203:3300/0 conn(0x7f51e80b2c30 msgr2=0x7f51e80b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:54.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ec902640 1 --2- 172.21.15.37:0/1231077412 >> v2:172.21.15.203:3300/0 conn(0x7f51e80b2c30 0x7f51e80b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.012+0000 7f51ec902640 1 -- 172.21.15.37:0/1231077412 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e80cc760 con 0x7f51e809a7b0 2024-04-25T08:58:54.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ec902640 1 --2- 172.21.15.37:0/1231077412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e809a7b0 0x7f51e8099a70 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f51e8071fe0 tx=0x7f51dc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab8f9ac213e3e16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:54.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51e7fff640 1 -- 172.21.15.37:0/1231077412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc0b2070 con 0x7f51e809a7b0 2024-04-25T08:58:54.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51e7fff640 1 -- 172.21.15.37:0/1231077412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc0ad440 con 0x7f51e809a7b0 2024-04-25T08:58:54.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51e7fff640 1 -- 172.21.15.37:0/1231077412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc0ad810 con 0x7f51e809a7b0 2024-04-25T08:58:54.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 -- 172.21.15.37:0/1231077412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e809a7b0 msgr2=0x7f51e8099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:54.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 --2- 172.21.15.37:0/1231077412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e809a7b0 0x7f51e8099a70 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f51e8071fe0 tx=0x7f51dc01c520 comp rx=0 tx=0).stop 2024-04-25T08:58:54.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 -- 172.21.15.37:0/1231077412 shutdown_connections 2024-04-25T08:58:54.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 --2- 172.21.15.37:0/1231077412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e809a7b0 0x7f51e8099a70 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 --2- 172.21.15.37:0/1231077412 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e8099fb0 0x7f51e80ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 --2- 172.21.15.37:0/1231077412 >> v2:172.21.15.203:3300/0 conn(0x7f51e80b2c30 0x7f51e80b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 -- 172.21.15.37:0/1231077412 >> 172.21.15.37:0/1231077412 conn(0x7f51e80b3010 msgr2=0x7f51e80b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:54.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 -- 172.21.15.37:0/1231077412 shutdown_connections 2024-04-25T08:58:54.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 -- 172.21.15.37:0/1231077412 wait complete. 2024-04-25T08:58:54.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 Processor -- start 2024-04-25T08:58:54.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 -- start start 2024-04-25T08:58:54.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e8099fb0 0x7f51e814eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e809a7b0 0x7f51e80c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e80b2c30 0x7f51e80c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e806c380 con 0x7f51e809a7b0 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e806c200 con 0x7f51e8099fb0 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ef367640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e815d190 con 0x7f51e80b2c30 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ec902640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e809a7b0 0x7f51e80c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ec902640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e809a7b0 0x7f51e80c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56650/0 (socket says 172.21.15.37:56650) 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ec902640 1 -- 172.21.15.37:0/3786574286 learned_addr learned my addr 172.21.15.37:0/3786574286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ed103640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e8099fb0 0x7f51e814eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ed904640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e80b2c30 0x7f51e80c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ec902640 1 -- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e80b2c30 msgr2=0x7f51e80c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ec902640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e80b2c30 0x7f51e80c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ec902640 1 -- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e8099fb0 msgr2=0x7f51e814eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:54.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ec902640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e8099fb0 0x7f51e814eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ec902640 1 -- 172.21.15.37:0/3786574286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e814f530 con 0x7f51e809a7b0 2024-04-25T08:58:54.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.016+0000 7f51ed103640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e8099fb0 0x7f51e814eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:54.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51ec902640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e809a7b0 0x7f51e80c7510 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0b0040 tx=0x7f51dc0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:54.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51e5ffb640 1 -- 172.21.15.37:0/3786574286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc0bd070 con 0x7f51e809a7b0 2024-04-25T08:58:54.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51e5ffb640 1 -- 172.21.15.37:0/3786574286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc0ade50 con 0x7f51e809a7b0 2024-04-25T08:58:54.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51e5ffb640 1 -- 172.21.15.37:0/3786574286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc0b2070 con 0x7f51e809a7b0 2024-04-25T08:58:54.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51ef367640 1 -- 172.21.15.37:0/3786574286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e80c6470 con 0x7f51e809a7b0 2024-04-25T08:58:54.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51ef367640 1 -- 172.21.15.37:0/3786574286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e80c6a30 con 0x7f51e809a7b0 2024-04-25T08:58:54.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51ef367640 1 -- 172.21.15.37:0/3786574286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e804efa0 con 0x7f51e809a7b0 2024-04-25T08:58:54.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51e5ffb640 1 -- 172.21.15.37:0/3786574286 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51dc0c0070 con 0x7f51e809a7b0 2024-04-25T08:58:54.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51e5ffb640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51d4066f60 0x7f51d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51e5ffb640 1 -- 172.21.15.37:0/3786574286 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f51dc11f210 con 0x7f51e809a7b0 2024-04-25T08:58:54.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51ed103640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51d4066f60 0x7f51d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.020+0000 7f51ed103640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51d4066f60 0x7f51d4069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f51d8003fd0 tx=0x7f51d80039e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:54.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.024+0000 7f51e5ffb640 1 -- 172.21.15.37:0/3786574286 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51dc016110 con 0x7f51e809a7b0 2024-04-25T08:58:54.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.188+0000 7f51ef367640 1 -- 172.21.15.37:0/3786574286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51e80c7e80 con 0x7f51e809a7b0 2024-04-25T08:58:54.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51e5ffb640 1 -- 172.21.15.37:0/3786574286 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16862 (secure 0 0 0) 0x7f51dc0f0200 con 0x7f51e809a7b0 2024-04-25T08:58:54.183 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:54.183 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":94,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:58:52.842976+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:50.512578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:51.848880+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:54.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 -- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51d4066f60 msgr2=0x7f51d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:54.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51d4066f60 0x7f51d4069420 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f51d8003fd0 tx=0x7f51d80039e0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 -- 172.21.15.37:0/3786574286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e809a7b0 msgr2=0x7f51e80c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:54.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e809a7b0 0x7f51e80c7510 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0b0040 tx=0x7f51dc0192c0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 -- 172.21.15.37:0/3786574286 shutdown_connections 2024-04-25T08:58:54.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f51d4066f60 0x7f51d4069420 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51e80b2c30 0x7f51e80c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51e809a7b0 0x7f51e80c7510 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 --2- 172.21.15.37:0/3786574286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51e8099fb0 0x7f51e814eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 -- 172.21.15.37:0/3786574286 >> 172.21.15.37:0/3786574286 conn(0x7f51e80b3010 msgr2=0x7f51e806d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:54.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 -- 172.21.15.37:0/3786574286 shutdown_connections 2024-04-25T08:58:54.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.192+0000 7f51ef367640 1 -- 172.21.15.37:0/3786574286 wait complete. 2024-04-25T08:58:54.202 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:58:54.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.332+0000 7f288cb06640 1 Processor -- start 2024-04-25T08:58:54.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288cb06640 1 -- start start 2024-04-25T08:58:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288cb06640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2888159bd0 0x7f28880c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288cb06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2888159fb0 0x7f28880b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288cb06640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28880c9ef0 con 0x7f28880b26f0 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288cb06640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2888072780 con 0x7f2888159bd0 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288cb06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2888071b70 con 0x7f2888159fb0 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f2886d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f28880b26f0 0x7f28880cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37204/0 (socket says 172.21.15.37:37204) 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f2886d9d640 1 -- 172.21.15.37:0/1566510645 learned_addr learned my addr 172.21.15.37:0/1566510645 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288659c640 1 --2- 172.21.15.37:0/1566510645 >> v2:172.21.15.203:3300/0 conn(0x7f2888159bd0 0x7f28880c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f2885d9b640 1 --2- 172.21.15.37:0/1566510645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2888159fb0 0x7f28880b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2549328688 0 0) 0x7f28880c9ef0 con 0x7f28880b26f0 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2874003530 con 0x7f28880b26f0 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3639503757 0 0) 0x7f2874003530 con 0x7f28880b26f0 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2888159fb0 msgr2=0x7f28880b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 --2- 172.21.15.37:0/1566510645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2888159fb0 0x7f28880b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 >> v2:172.21.15.203:3300/0 conn(0x7f2888159bd0 msgr2=0x7f28880c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 --2- 172.21.15.37:0/1566510645 >> v2:172.21.15.203:3300/0 conn(0x7f2888159bd0 0x7f28880c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2888071e50 con 0x7f28880b26f0 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f2878001680 con 0x7f28880b26f0 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 >> v1:172.21.15.37:6789/0 conn(0x7f28880b26f0 legacy=0x7f28880cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 --2- 172.21.15.37:0/1566510645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2888159fb0 0x7f2874003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 --2- 172.21.15.37:0/1566510645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 0x7f2874004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 --2- 172.21.15.37:0/1566510645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2874004d50 0x7f28740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2888072780 con 0x7f2874004000 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2888071b70 con 0x7f2874004d50 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287400ba50 con 0x7f2888159fb0 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f2885d9b640 1 --2- 172.21.15.37:0/1566510645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 0x7f2874004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f2885d9b640 1 -- 172.21.15.37:0/1566510645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2888159fb0 msgr2=0x7f2874003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:54.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f2885d9b640 1 --2- 172.21.15.37:0/1566510645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2888159fb0 0x7f2874003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f2885d9b640 1 -- 172.21.15.37:0/1566510645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2874004d50 msgr2=0x7f28740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:54.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f2885d9b640 1 --2- 172.21.15.37:0/1566510645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2874004d50 0x7f28740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.336+0000 7f2885d9b640 1 -- 172.21.15.37:0/1566510645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2874003530 con 0x7f2874004000 2024-04-25T08:58:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f2885d9b640 1 --2- 172.21.15.37:0/1566510645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 0x7f2874004810 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f2870009ab0 tx=0x7f287001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f12975fdeda8ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700b2070 con 0x7f2874004000 2024-04-25T08:58:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28700a5680 con 0x7f2874004000 2024-04-25T08:58:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288559a640 1 -- 172.21.15.37:0/1566510645 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28700096b0 con 0x7f2874004000 2024-04-25T08:58:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 -- 172.21.15.37:0/1566510645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 msgr2=0x7f2874004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 --2- 172.21.15.37:0/1566510645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 0x7f2874004810 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f2870009ab0 tx=0x7f287001ca90 comp rx=0 tx=0).stop 2024-04-25T08:58:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f2886d9d640 1 -- 172.21.15.37:0/1566510645 reap_dead start 2024-04-25T08:58:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 -- 172.21.15.37:0/1566510645 shutdown_connections 2024-04-25T08:58:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 -- 172.21.15.37:0/1566510645 >> 172.21.15.37:0/1566510645 conn(0x7f288806ae60 msgr2=0x7f288806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 -- 172.21.15.37:0/1566510645 shutdown_connections 2024-04-25T08:58:54.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 -- 172.21.15.37:0/1566510645 wait complete. 2024-04-25T08:58:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 Processor -- start 2024-04-25T08:58:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 -- start start 2024-04-25T08:58:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 0x7f28880974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f288815cd50 0x7f28880b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28880c1810 0x7f28880c1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28880cefd0 con 0x7f2874004000 2024-04-25T08:58:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2888072410 con 0x7f288815cd50 2024-04-25T08:58:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288cb06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28880b21b0 con 0x7f28880c1810 2024-04-25T08:58:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 0x7f28880974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 0x7f28880974a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56662/0 (socket says 172.21.15.37:56662) 2024-04-25T08:58:54.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.340+0000 7f288659c640 1 -- 172.21.15.37:0/2417988585 learned_addr learned my addr 172.21.15.37:0/2417988585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f2886d9d640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28880c1810 0x7f28880c1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f2885d9b640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f288815cd50 0x7f28880b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f288659c640 1 -- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28880c1810 msgr2=0x7f28880c1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f288659c640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28880c1810 0x7f28880c1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f288659c640 1 -- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f288815cd50 msgr2=0x7f28880b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f288659c640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f288815cd50 0x7f28880b0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f288659c640 1 -- 172.21.15.37:0/2417988585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2888097da0 con 0x7f2874004000 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f2885d9b640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f288815cd50 0x7f28880b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f2886d9d640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28880c1810 0x7f28880c1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f288659c640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 0x7f28880974a0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f2880009db0 tx=0x7f288000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f287effd640 1 -- 172.21.15.37:0/2417988585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880018020 con 0x7f2874004000 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f287effd640 1 -- 172.21.15.37:0/2417988585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2880014480 con 0x7f2874004000 2024-04-25T08:58:54.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f287effd640 1 -- 172.21.15.37:0/2417988585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880014870 con 0x7f2874004000 2024-04-25T08:58:54.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f288cb06640 1 -- 172.21.15.37:0/2417988585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2888098080 con 0x7f2874004000 2024-04-25T08:58:54.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f288cb06640 1 -- 172.21.15.37:0/2417988585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2888098640 con 0x7f2874004000 2024-04-25T08:58:54.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f288cb06640 1 -- 172.21.15.37:0/2417988585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2888098840 con 0x7f2874004000 2024-04-25T08:58:54.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f287effd640 1 -- 172.21.15.37:0/2417988585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f288001b020 con 0x7f2874004000 2024-04-25T08:58:54.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f287effd640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2868066f60 0x7f2868069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:54.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f287effd640 1 -- 172.21.15.37:0/2417988585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2880085d70 con 0x7f2874004000 2024-04-25T08:58:54.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f2885d9b640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2868066f60 0x7f2868069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:54.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.344+0000 7f2885d9b640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2868066f60 0x7f2868069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f28700af000 tx=0x7f28700046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:54.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.348+0000 7f287effd640 1 -- 172.21.15.37:0/2417988585 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2888098840 con 0x7f2874004000 2024-04-25T08:58:54.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:54.552+0000 7f288cb06640 1 -- 172.21.15.37:0/2417988585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f28880cce50 con 0x7f2874004000 2024-04-25T08:58:55.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.544+0000 7f287effd640 1 -- 172.21.15.37:0/2417988585 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v59) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2880022030 con 0x7f2874004000 2024-04-25T08:58:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 -- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2868066f60 msgr2=0x7f2868069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:55.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2868066f60 0x7f2868069420 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f28700af000 tx=0x7f28700046d0 comp rx=0 tx=0).stop 2024-04-25T08:58:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 -- 172.21.15.37:0/2417988585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 msgr2=0x7f28880974a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 0x7f28880974a0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f2880009db0 tx=0x7f288000c490 comp rx=0 tx=0).stop 2024-04-25T08:58:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 -- 172.21.15.37:0/2417988585 shutdown_connections 2024-04-25T08:58:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2868066f60 0x7f2868069420 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28880c1810 0x7f28880c1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f288815cd50 0x7f28880b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 --2- 172.21.15.37:0/2417988585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2874004000 0x7f28880974a0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 -- 172.21.15.37:0/2417988585 >> 172.21.15.37:0/2417988585 conn(0x7f288806ae60 msgr2=0x7f28880af7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 -- 172.21.15.37:0/2417988585 shutdown_connections 2024-04-25T08:58:55.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.548+0000 7f288cb06640 1 -- 172.21.15.37:0/2417988585 wait complete. 2024-04-25T08:58:55.564 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:58:55.565 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:58:55.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10f645e640 1 Processor -- start 2024-04-25T08:58:55.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10f645e640 1 -- start start 2024-04-25T08:58:55.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10f645e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f10f009a480 0x7f10f00cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:55.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10f645e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f00cfb80 0x7f10f00cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:55.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10f645e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f015bd30 con 0x7f10f015b950 2024-04-25T08:58:55.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10f645e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f00d0510 con 0x7f10f009a480 2024-04-25T08:58:55.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10f645e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00cc0d0 con 0x7f10f00cfb80 2024-04-25T08:58:55.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10effff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f10f015b950 0x7f10f00cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37216/0 (socket says 172.21.15.37:37216) 2024-04-25T08:58:55.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10effff640 1 -- 172.21.15.37:0/2168111872 learned_addr learned my addr 172.21.15.37:0/2168111872 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:55.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10ef7fe640 1 --2- 172.21.15.37:0/2168111872 >> v2:172.21.15.37:3300/0 conn(0x7f10f009a480 0x7f10f00cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:55.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10f49fb640 1 --2- 172.21.15.37:0/2168111872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f00cfb80 0x7f10f00cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10eeffd640 1 -- 172.21.15.37:0/2168111872 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1433140953 0 0) 0x7f10f015bd30 con 0x7f10f015b950 2024-04-25T08:58:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10eeffd640 1 -- 172.21.15.37:0/2168111872 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10d8003530 con 0x7f10f015b950 2024-04-25T08:58:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10ef7fe640 1 -- 172.21.15.37:0/2168111872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f00cfb80 msgr2=0x7f10f00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10ef7fe640 1 --2- 172.21.15.37:0/2168111872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f00cfb80 0x7f10f00cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10ef7fe640 1 -- 172.21.15.37:0/2168111872 >> v1:172.21.15.37:6789/0 conn(0x7f10f015b950 legacy=0x7f10f00cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10ef7fe640 1 -- 172.21.15.37:0/2168111872 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f009ab10 con 0x7f10f009a480 2024-04-25T08:58:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.708+0000 7f10f49fb640 1 --2- 172.21.15.37:0/2168111872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f00cfb80 0x7f10f00cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10ef7fe640 1 --2- 172.21.15.37:0/2168111872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10f009a480 0x7f10f00cce50 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f10e00097c0 tx=0x7f10e001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628f670bdf315fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10eeffd640 1 -- 172.21.15.37:0/2168111872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00b2070 con 0x7f10f009a480 2024-04-25T08:58:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10eeffd640 1 -- 172.21.15.37:0/2168111872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e00ad640 con 0x7f10f009a480 2024-04-25T08:58:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10eeffd640 1 -- 172.21.15.37:0/2168111872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e00ad9f0 con 0x7f10f009a480 2024-04-25T08:58:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10f645e640 1 -- 172.21.15.37:0/2168111872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10f009a480 msgr2=0x7f10f00cce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10f645e640 1 --2- 172.21.15.37:0/2168111872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10f009a480 0x7f10f00cce50 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f10e00097c0 tx=0x7f10e001c4a0 comp rx=0 tx=0).stop 2024-04-25T08:58:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10f645e640 1 -- 172.21.15.37:0/2168111872 shutdown_connections 2024-04-25T08:58:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10f645e640 1 --2- 172.21.15.37:0/2168111872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f00cfb80 0x7f10f00cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10f645e640 1 --2- 172.21.15.37:0/2168111872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10f009a480 0x7f10f00cce50 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10f645e640 1 -- 172.21.15.37:0/2168111872 >> 172.21.15.37:0/2168111872 conn(0x7f10f00b30b0 msgr2=0x7f10f00b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10f645e640 1 -- 172.21.15.37:0/2168111872 shutdown_connections 2024-04-25T08:58:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.712+0000 7f10f645e640 1 -- 172.21.15.37:0/2168111872 wait complete. 2024-04-25T08:58:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f645e640 1 Processor -- start 2024-04-25T08:58:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f645e640 1 -- start start 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f645e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f009a480 0x7f10f00c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f645e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f00cfb80 0x7f10f0153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f645e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10f015b950 0x7f10f00ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f645e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f00cd390 con 0x7f10f015b950 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f645e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f006faa0 con 0x7f10f00cfb80 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f645e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0154910 con 0x7f10f009a480 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f49fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10f015b950 0x7f10f00ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f49fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10f015b950 0x7f10f00ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56666/0 (socket says 172.21.15.37:56666) 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f49fb640 1 -- 172.21.15.37:0/2332652567 learned_addr learned my addr 172.21.15.37:0/2332652567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10effff640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f009a480 0x7f10f00c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10ef7fe640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f00cfb80 0x7f10f0153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f49fb640 1 -- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f009a480 msgr2=0x7f10f00c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f49fb640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f009a480 0x7f10f00c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f49fb640 1 -- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f00cfb80 msgr2=0x7f10f0153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f49fb640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f00cfb80 0x7f10f0153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f49fb640 1 -- 172.21.15.37:0/2332652567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0154bf0 con 0x7f10f015b950 2024-04-25T08:58:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10ef7fe640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f00cfb80 0x7f10f0153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f49fb640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10f015b950 0x7f10f00ae2f0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009fa0 tx=0x7f10e800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10ecff9640 1 -- 172.21.15.37:0/2332652567 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e801a070 con 0x7f10f015b950 2024-04-25T08:58:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10ecff9640 1 -- 172.21.15.37:0/2332652567 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e8003bc0 con 0x7f10f015b950 2024-04-25T08:58:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10ecff9640 1 -- 172.21.15.37:0/2332652567 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8004030 con 0x7f10f015b950 2024-04-25T08:58:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f645e640 1 -- 172.21.15.37:0/2332652567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f0154ed0 con 0x7f10f015b950 2024-04-25T08:58:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f645e640 1 -- 172.21.15.37:0/2332652567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f01542d0 con 0x7f10f015b950 2024-04-25T08:58:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10f645e640 1 -- 172.21.15.37:0/2332652567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f01558b0 con 0x7f10f015b950 2024-04-25T08:58:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10ecff9640 1 -- 172.21.15.37:0/2332652567 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10e801d070 con 0x7f10f015b950 2024-04-25T08:58:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10ecff9640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10dc066f60 0x7f10dc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.716+0000 7f10ecff9640 1 -- 172.21.15.37:0/2332652567 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10e8085440 con 0x7f10f015b950 2024-04-25T08:58:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.720+0000 7f10effff640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10dc066f60 0x7f10dc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.720+0000 7f10effff640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10dc066f60 0x7f10dc069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f10e4007fd0 tx=0x7f10e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.724+0000 7f10ecff9640 1 -- 172.21.15.37:0/2332652567 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10f01558b0 con 0x7f10f015b950 2024-04-25T08:58:55.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:55.920+0000 7f10f645e640 1 -- 172.21.15.37:0/2332652567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f10f004efa0 con 0x7f10f015b950 2024-04-25T08:58:56.104 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:56.107 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.556+0000 7f10ecff9640 1 -- 172.21.15.37:0/2332652567 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v60) v1 ==== 122+0+0 (secure 0 0 0) 0x7f10e8010070 con 0x7f10f015b950 2024-04-25T08:58:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 -- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10dc066f60 msgr2=0x7f10dc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10dc066f60 0x7f10dc069420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f10e4007fd0 tx=0x7f10e4009040 comp rx=0 tx=0).stop 2024-04-25T08:58:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 -- 172.21.15.37:0/2332652567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10f015b950 msgr2=0x7f10f00ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10f015b950 0x7f10f00ae2f0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009fa0 tx=0x7f10e800d560 comp rx=0 tx=0).stop 2024-04-25T08:58:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 -- 172.21.15.37:0/2332652567 shutdown_connections 2024-04-25T08:58:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10dc066f60 0x7f10dc069420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10f015b950 0x7f10f00ae2f0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10f00cfb80 0x7f10f0153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 --2- 172.21.15.37:0/2332652567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10f009a480 0x7f10f00c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 -- 172.21.15.37:0/2332652567 >> 172.21.15.37:0/2332652567 conn(0x7f10f00b30b0 msgr2=0x7f10f00ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 -- 172.21.15.37:0/2332652567 shutdown_connections 2024-04-25T08:58:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.560+0000 7f10f645e640 1 -- 172.21.15.37:0/2332652567 wait complete. 2024-04-25T08:58:56.572 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:58:56.572 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:56.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe7228a9640 1 Processor -- start 2024-04-25T08:58:56.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe7228a9640 1 -- start start 2024-04-25T08:58:56.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe7228a9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe71c144ca0 0x7fe71c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:56.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe7228a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c06ec60 0x7fe71c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:56.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe7228a9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71c0ccc70 con 0x7fe71c0ce510 2024-04-25T08:58:56.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe7228a9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe71c0cc2b0 con 0x7fe71c144ca0 2024-04-25T08:58:56.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe7228a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c06e290 con 0x7fe71c06ec60 2024-04-25T08:58:56.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe71b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe71c144ca0 0x7fe71c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:56.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe71b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe71c144ca0 0x7fe71c06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56676/0 (socket says 172.21.15.37:56676) 2024-04-25T08:58:56.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe71b7fe640 1 -- 172.21.15.37:0/3444518905 learned_addr learned my addr 172.21.15.37:0/3444518905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:56.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe720e46640 1 --2- 172.21.15.37:0/3444518905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c06ec60 0x7fe71c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:56.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe71b7fe640 1 -- 172.21.15.37:0/3444518905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c06ec60 msgr2=0x7fe71c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:56.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe71b7fe640 1 --2- 172.21.15.37:0/3444518905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c06ec60 0x7fe71c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe71b7fe640 1 -- 172.21.15.37:0/3444518905 >> v1:172.21.15.203:6789/0 conn(0x7fe71c0ce510 legacy=0x7fe71c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:56.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.712+0000 7fe71b7fe640 1 -- 172.21.15.37:0/3444518905 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c06dc40 con 0x7fe71c144ca0 2024-04-25T08:58:56.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe71b7fe640 1 --2- 172.21.15.37:0/3444518905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c144ca0 0x7fe71c06e720 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe704009870 tx=0x7fe70401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c38998a5f01bb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:56.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe71affd640 1 -- 172.21.15.37:0/3444518905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040b5070 con 0x7fe71c144ca0 2024-04-25T08:58:56.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe71affd640 1 -- 172.21.15.37:0/3444518905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7040a8a50 con 0x7fe71c144ca0 2024-04-25T08:58:56.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe71affd640 1 -- 172.21.15.37:0/3444518905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7040a8e20 con 0x7fe71c144ca0 2024-04-25T08:58:56.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe7228a9640 1 -- 172.21.15.37:0/3444518905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c144ca0 msgr2=0x7fe71c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:56.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe7228a9640 1 --2- 172.21.15.37:0/3444518905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c144ca0 0x7fe71c06e720 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fe704009870 tx=0x7fe70401ed50 comp rx=0 tx=0).stop 2024-04-25T08:58:56.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe7228a9640 1 -- 172.21.15.37:0/3444518905 shutdown_connections 2024-04-25T08:58:56.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe7228a9640 1 --2- 172.21.15.37:0/3444518905 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c06ec60 0x7fe71c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe7228a9640 1 --2- 172.21.15.37:0/3444518905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c144ca0 0x7fe71c06e720 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe7228a9640 1 -- 172.21.15.37:0/3444518905 >> 172.21.15.37:0/3444518905 conn(0x7fe71c0b11a0 msgr2=0x7fe71c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:56.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe7228a9640 1 -- 172.21.15.37:0/3444518905 shutdown_connections 2024-04-25T08:58:56.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe7228a9640 1 -- 172.21.15.37:0/3444518905 wait complete. 2024-04-25T08:58:56.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.716+0000 7fe7228a9640 1 Processor -- start 2024-04-25T08:58:56.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe7228a9640 1 -- start start 2024-04-25T08:58:56.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe7228a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ec60 0x7fe71c1337c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:56.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe7228a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c0ce510 0x7fe71c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:56.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe7228a9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe71c144ca0 0x7fe71c143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:56.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe7228a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c1451f0 con 0x7fe71c06ec60 2024-04-25T08:58:56.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe7228a9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c0cc900 con 0x7fe71c144ca0 2024-04-25T08:58:56.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe7228a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c145370 con 0x7fe71c0ce510 2024-04-25T08:58:56.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe71bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ec60 0x7fe71c1337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:56.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe71bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ec60 0x7fe71c1337c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56684/0 (socket says 172.21.15.37:56684) 2024-04-25T08:58:56.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe71bfff640 1 -- 172.21.15.37:0/3763662841 learned_addr learned my addr 172.21.15.37:0/3763662841 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:56.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe71b7fe640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c0ce510 0x7fe71c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:56.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe720e46640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe71c144ca0 0x7fe71c143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:56.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe71bfff640 1 -- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c0ce510 msgr2=0x7fe71c15afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:56.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe71bfff640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c0ce510 0x7fe71c15afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe71bfff640 1 -- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe71c144ca0 msgr2=0x7fe71c143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:56.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe71bfff640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe71c144ca0 0x7fe71c143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe71bfff640 1 -- 172.21.15.37:0/3763662841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe71c1e5780 con 0x7fe71c06ec60 2024-04-25T08:58:56.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe720e46640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe71c144ca0 0x7fe71c143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:56.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe71bfff640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ec60 0x7fe71c1337c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe71400c790 tx=0x7fe71400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:56.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe718ff9640 1 -- 172.21.15.37:0/3763662841 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714016020 con 0x7fe71c06ec60 2024-04-25T08:58:56.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe718ff9640 1 -- 172.21.15.37:0/3763662841 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe714006d60 con 0x7fe71c06ec60 2024-04-25T08:58:56.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe7228a9640 1 -- 172.21.15.37:0/3763662841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c1e5a60 con 0x7fe71c06ec60 2024-04-25T08:58:56.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe718ff9640 1 -- 172.21.15.37:0/3763662841 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714003bc0 con 0x7fe71c06ec60 2024-04-25T08:58:56.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe7228a9640 1 -- 172.21.15.37:0/3763662841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c1e6020 con 0x7fe71c06ec60 2024-04-25T08:58:56.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe7228a9640 1 -- 172.21.15.37:0/3763662841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71c04efa0 con 0x7fe71c06ec60 2024-04-25T08:58:56.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe718ff9640 1 -- 172.21.15.37:0/3763662841 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe71400a850 con 0x7fe71c06ec60 2024-04-25T08:58:56.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe718ff9640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6fc066f60 0x7fe6fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.720+0000 7fe718ff9640 1 -- 172.21.15.37:0/3763662841 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe714084ec0 con 0x7fe71c06ec60 2024-04-25T08:58:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.724+0000 7fe71b7fe640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6fc066f60 0x7fe6fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:56.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.724+0000 7fe71b7fe640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6fc066f60 0x7fe6fc069420 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe7040b2000 tx=0x7fe704008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:56.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.728+0000 7fe718ff9640 1 -- 172.21.15.37:0/3763662841 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe714020020 con 0x7fe71c06ec60 2024-04-25T08:58:56.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.920+0000 7fe7228a9640 1 -- 172.21.15.37:0/3763662841 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe71c0b1cc0 con 0x7fe71c06ec60 2024-04-25T08:58:56.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.920+0000 7fe718ff9640 1 -- 172.21.15.37:0/3763662841 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe714055dd0 con 0x7fe71c06ec60 2024-04-25T08:58:56.912 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:56.912 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T08:58:55.924579+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7734},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7734":{"gid":7734,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/354285428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":354285428},{"type":"v1","addr":"172.21.15.37:6835","nonce":354285428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:58:56.912 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 60 2024-04-25T08:58:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 -- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6fc066f60 msgr2=0x7fe6fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6fc066f60 0x7fe6fc069420 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe7040b2000 tx=0x7fe704008040 comp rx=0 tx=0).stop 2024-04-25T08:58:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 -- 172.21.15.37:0/3763662841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ec60 msgr2=0x7fe71c1337c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ec60 0x7fe71c1337c0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fe71400c790 tx=0x7fe71400cc50 comp rx=0 tx=0).stop 2024-04-25T08:58:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 -- 172.21.15.37:0/3763662841 shutdown_connections 2024-04-25T08:58:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6fc066f60 0x7fe6fc069420 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe71c144ca0 0x7fe71c143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe71c0ce510 0x7fe71c15afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 --2- 172.21.15.37:0/3763662841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe71c06ec60 0x7fe71c1337c0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 -- 172.21.15.37:0/3763662841 >> 172.21.15.37:0/3763662841 conn(0x7fe71c0b11a0 msgr2=0x7fe71c0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 -- 172.21.15.37:0/3763662841 shutdown_connections 2024-04-25T08:58:56.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:56.924+0000 7fe7228a9640 1 -- 172.21.15.37:0/3763662841 wait complete. 2024-04-25T08:58:56.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:57.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.060+0000 7f96f3668640 1 Processor -- start 2024-04-25T08:58:57.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.060+0000 7f96f3668640 1 -- start start 2024-04-25T08:58:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f3668640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f96ec15d8d0 0x7f96ec15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f3668640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96ec0ce540 0x7f96ec099df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f3668640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96ec06c570 con 0x7f96ec06d330 2024-04-25T08:58:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f3668640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96ec06cea0 con 0x7f96ec15d8d0 2024-04-25T08:58:57.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f3668640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec06c850 con 0x7f96ec0ce540 2024-04-25T08:58:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f1c05640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96ec0ce540 0x7f96ec099df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f1404640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f96ec15d8d0 0x7f96ec15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f1404640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f96ec15d8d0 0x7f96ec15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:49574/0 (socket says 172.21.15.37:49574) 2024-04-25T08:58:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f0c03640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f96ec06d330 0x7f96ec0cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:60304/0 (socket says 172.21.15.37:60304) 2024-04-25T08:58:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f1404640 1 -- 172.21.15.37:0/761985417 learned_addr learned my addr 172.21.15.37:0/761985417 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f1404640 1 -- 172.21.15.37:0/761985417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96ec0ce540 msgr2=0x7f96ec099df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:57.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f1404640 1 --2- 172.21.15.37:0/761985417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96ec0ce540 0x7f96ec099df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f1404640 1 -- 172.21.15.37:0/761985417 >> v1:172.21.15.203:6789/0 conn(0x7f96ec06d330 legacy=0x7f96ec0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f1404640 1 -- 172.21.15.37:0/761985417 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec06cb30 con 0x7f96ec15d8d0 2024-04-25T08:58:57.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.064+0000 7f96f1c05640 1 --2- 172.21.15.37:0/761985417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96ec0ce540 0x7f96ec099df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96f1404640 1 --2- 172.21.15.37:0/761985417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96ec15d8d0 0x7f96ec15dcb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009870 tx=0x7f96e801c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a76a7aefe1bb471 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96dffff640 1 -- 172.21.15.37:0/761985417 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e80aa680 con 0x7f96ec15d8d0 2024-04-25T08:58:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96dffff640 1 -- 172.21.15.37:0/761985417 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e80af070 con 0x7f96ec15d8d0 2024-04-25T08:58:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96dffff640 1 -- 172.21.15.37:0/761985417 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e80aaa40 con 0x7f96ec15d8d0 2024-04-25T08:58:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96f3668640 1 -- 172.21.15.37:0/761985417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96ec15d8d0 msgr2=0x7f96ec15dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96f3668640 1 --2- 172.21.15.37:0/761985417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96ec15d8d0 0x7f96ec15dcb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009870 tx=0x7f96e801c500 comp rx=0 tx=0).stop 2024-04-25T08:58:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96f3668640 1 -- 172.21.15.37:0/761985417 shutdown_connections 2024-04-25T08:58:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96f3668640 1 --2- 172.21.15.37:0/761985417 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96ec0ce540 0x7f96ec099df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96f3668640 1 --2- 172.21.15.37:0/761985417 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96ec15d8d0 0x7f96ec15dcb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96f3668640 1 -- 172.21.15.37:0/761985417 >> 172.21.15.37:0/761985417 conn(0x7f96ec0d15b0 msgr2=0x7f96ec0d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96f3668640 1 -- 172.21.15.37:0/761985417 shutdown_connections 2024-04-25T08:58:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96f3668640 1 -- 172.21.15.37:0/761985417 wait complete. 2024-04-25T08:58:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96f3668640 1 Processor -- start 2024-04-25T08:58:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.068+0000 7f96f3668640 1 -- start start 2024-04-25T08:58:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f3668640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ec06d330 0x7f96ec09eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f3668640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96ec0ce540 0x7f96ec15e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f3668640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96ec15d8d0 0x7f96ec160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f3668640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0cc7a0 con 0x7f96ec06d330 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f3668640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0cc620 con 0x7f96ec0ce540 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f3668640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96ec0cc920 con 0x7f96ec15d8d0 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f1404640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ec06d330 0x7f96ec09eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f1404640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ec06d330 0x7f96ec09eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56698/0 (socket says 172.21.15.37:56698) 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f1404640 1 -- 172.21.15.37:0/3644993924 learned_addr learned my addr 172.21.15.37:0/3644993924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f0c03640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96ec0ce540 0x7f96ec15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f1c05640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96ec15d8d0 0x7f96ec160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f1404640 1 -- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96ec15d8d0 msgr2=0x7f96ec160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f1404640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96ec15d8d0 0x7f96ec160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f1404640 1 -- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96ec0ce540 msgr2=0x7f96ec15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f1404640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96ec0ce540 0x7f96ec15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f1404640 1 -- 172.21.15.37:0/3644993924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96ec0cb040 con 0x7f96ec06d330 2024-04-25T08:58:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f1404640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ec06d330 0x7f96ec09eb10 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f96e801ac10 tx=0x7f96e801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96ddffb640 1 -- 172.21.15.37:0/3644993924 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e80bd5f0 con 0x7f96ec06d330 2024-04-25T08:58:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96ddffb640 1 -- 172.21.15.37:0/3644993924 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e80bd7b0 con 0x7f96ec06d330 2024-04-25T08:58:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f3668640 1 -- 172.21.15.37:0/3644993924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ec09f050 con 0x7f96ec06d330 2024-04-25T08:58:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96ddffb640 1 -- 172.21.15.37:0/3644993924 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e80bdba0 con 0x7f96ec06d330 2024-04-25T08:58:57.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f3668640 1 -- 172.21.15.37:0/3644993924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96ec0ca7a0 con 0x7f96ec06d330 2024-04-25T08:58:57.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f3668640 1 -- 172.21.15.37:0/3644993924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96ec04efa0 con 0x7f96ec06d330 2024-04-25T08:58:57.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96ddffb640 1 -- 172.21.15.37:0/3644993924 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f96e80a8050 con 0x7f96ec06d330 2024-04-25T08:58:57.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96ddffb640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f96d0066f90 0x7f96d0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96ddffb640 1 -- 172.21.15.37:0/3644993924 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f96e811f320 con 0x7f96ec06d330 2024-04-25T08:58:57.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.072+0000 7f96f0c03640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f96d0066f90 0x7f96d0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.076+0000 7f96f0c03640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f96d0066f90 0x7f96d0069450 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0c5070 tx=0x7f96e0007cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:57.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.076+0000 7f96ddffb640 1 -- 172.21.15.37:0/3644993924 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f96e80c3060 con 0x7f96ec06d330 2024-04-25T08:58:57.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.240+0000 7f96f3668640 1 -- 172.21.15.37:0/3644993924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96ec073d90 con 0x7f96ec06d330 2024-04-25T08:58:57.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.240+0000 7f96ddffb640 1 -- 172.21.15.37:0/3644993924 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16849 (secure 0 0 0) 0x7f96e80a6040 con 0x7f96ec06d330 2024-04-25T08:58:57.234 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:57.234 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":95,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:58:54.529461+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:50.512578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:51.848880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:57.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 -- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f96d0066f90 msgr2=0x7f96d0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f96d0066f90 0x7f96d0069450 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0c5070 tx=0x7f96e0007cf0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 -- 172.21.15.37:0/3644993924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ec06d330 msgr2=0x7f96ec09eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ec06d330 0x7f96ec09eb10 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f96e801ac10 tx=0x7f96e801ada0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 -- 172.21.15.37:0/3644993924 shutdown_connections 2024-04-25T08:58:57.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f96d0066f90 0x7f96d0069450 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f96ec15d8d0 0x7f96ec160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f96ec0ce540 0x7f96ec15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 --2- 172.21.15.37:0/3644993924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f96ec06d330 0x7f96ec09eb10 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 -- 172.21.15.37:0/3644993924 >> 172.21.15.37:0/3644993924 conn(0x7f96ec0d15b0 msgr2=0x7f96ec0b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:57.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 -- 172.21.15.37:0/3644993924 shutdown_connections 2024-04-25T08:58:57.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.244+0000 7f96f3668640 1 -- 172.21.15.37:0/3644993924 wait complete. 2024-04-25T08:58:57.252 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:58:57.252 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:58:57.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.380+0000 7fcc90fa3640 1 Processor -- start 2024-04-25T08:58:57.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.380+0000 7fcc90fa3640 1 -- start start 2024-04-25T08:58:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc90fa3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcc8c144ca0 0x7fcc8c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc90fa3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c06ec60 0x7fcc8c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc90fa3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8c0ccc70 con 0x7fcc8c0ce510 2024-04-25T08:58:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc90fa3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc8c0cc2b0 con 0x7fcc8c144ca0 2024-04-25T08:58:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc90fa3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c06e290 con 0x7fcc8c06ec60 2024-04-25T08:58:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8a59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fcc8c0ce510 0x7fcc8c0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37220/0 (socket says 172.21.15.37:37220) 2024-04-25T08:58:57.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8a59c640 1 -- 172.21.15.37:0/2117575534 learned_addr learned my addr 172.21.15.37:0/2117575534 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc89d9b640 1 --2- 172.21.15.37:0/2117575534 >> v2:172.21.15.203:3300/0 conn(0x7fcc8c144ca0 0x7fcc8c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8ad9d640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c06ec60 0x7fcc8c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179776758 0 0) 0x7fcc8c0ccc70 con 0x7fcc8c0ce510 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc74003530 con 0x7fcc8c0ce510 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2943584536 0 0) 0x7fcc74003530 con 0x7fcc8c0ce510 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c06ec60 msgr2=0x7fcc8c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c06ec60 0x7fcc8c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 >> v2:172.21.15.203:3300/0 conn(0x7fcc8c144ca0 msgr2=0x7fcc8c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 --2- 172.21.15.37:0/2117575534 >> v2:172.21.15.203:3300/0 conn(0x7fcc8c144ca0 0x7fcc8c06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c06dc40 con 0x7fcc8c0ce510 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fcc7c001680 con 0x7fcc8c0ce510 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 >> v1:172.21.15.37:6789/0 conn(0x7fcc8c0ce510 legacy=0x7fcc8c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc74003a30 0x7fcc74003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc74004810 0x7fcc74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c06ec60 0x7fcc740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c0cc2b0 con 0x7fcc74004810 2024-04-25T08:58:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c06e290 con 0x7fcc74003a30 2024-04-25T08:58:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7400ba50 con 0x7fcc8c06ec60 2024-04-25T08:58:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc89d9b640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc74004810 0x7fcc74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8a59c640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc74003a30 0x7fcc74003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc89d9b640 1 -- 172.21.15.37:0/2117575534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c06ec60 msgr2=0x7fcc740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc8ad9d640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c06ec60 0x7fcc740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc89d9b640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c06ec60 0x7fcc740051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc89d9b640 1 -- 172.21.15.37:0/2117575534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc74003a30 msgr2=0x7fcc74003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc89d9b640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc74003a30 0x7fcc74003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc89d9b640 1 -- 172.21.15.37:0/2117575534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc74003530 con 0x7fcc74004810 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.384+0000 7fcc89d9b640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc74004810 0x7fcc74004c90 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009ab0 tx=0x7fcc8001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98107281df5c071a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc800b2070 con 0x7fcc74004810 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc800a55a0 con 0x7fcc74004810 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8959a640 1 -- 172.21.15.37:0/2117575534 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc800096b0 con 0x7fcc74004810 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2117575534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc74004810 msgr2=0x7fcc74004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 --2- 172.21.15.37:0/2117575534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc74004810 0x7fcc74004c90 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009ab0 tx=0x7fcc8001ca90 comp rx=0 tx=0).stop 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8ad9d640 1 -- 172.21.15.37:0/2117575534 reap_dead start 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2117575534 shutdown_connections 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2117575534 >> 172.21.15.37:0/2117575534 conn(0x7fcc8c0b11a0 msgr2=0x7fcc8c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2117575534 shutdown_connections 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2117575534 wait complete. 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 Processor -- start 2024-04-25T08:58:57.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 -- start start 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc74004810 0x7fcc8c1f0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c136380 0x7fcc8c0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc8c0a8a10 0x7fcc8c133e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c1451f0 con 0x7fcc8c0a8a10 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c0cc900 con 0x7fcc74004810 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c145370 con 0x7fcc8c136380 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc8c0a8a10 0x7fcc8c133e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc8c0a8a10 0x7fcc8c133e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56712/0 (socket says 172.21.15.37:56712) 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8ad9d640 1 -- 172.21.15.37:0/2949726794 learned_addr learned my addr 172.21.15.37:0/2949726794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc89d9b640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c136380 0x7fcc8c0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8ad9d640 1 -- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c136380 msgr2=0x7fcc8c0a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8a59c640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc74004810 0x7fcc8c1f0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8ad9d640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c136380 0x7fcc8c0a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8ad9d640 1 -- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc74004810 msgr2=0x7fcc8c1f0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8ad9d640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc74004810 0x7fcc8c1f0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8ad9d640 1 -- 172.21.15.37:0/2949726794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c1f14d0 con 0x7fcc8c0a8a10 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8a59c640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc74004810 0x7fcc8c1f0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc8ad9d640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc8c0a8a10 0x7fcc8c133e60 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c06f0c0 tx=0x7fcc8400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc7affd640 1 -- 172.21.15.37:0/2949726794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8401c070 con 0x7fcc8c0a8a10 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc7affd640 1 -- 172.21.15.37:0/2949726794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc84017400 con 0x7fcc8c0a8a10 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc7affd640 1 -- 172.21.15.37:0/2949726794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc840177f0 con 0x7fcc8c0a8a10 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2949726794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8c1330a0 con 0x7fcc8c0a8a10 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.388+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2949726794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8c133660 con 0x7fcc8c0a8a10 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.392+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2949726794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8c04efa0 con 0x7fcc8c0a8a10 2024-04-25T08:58:57.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.392+0000 7fcc7affd640 1 -- 172.21.15.37:0/2949726794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc8401f070 con 0x7fcc8c0a8a10 2024-04-25T08:58:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.392+0000 7fcc7affd640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc6c066f60 0x7fcc6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.392+0000 7fcc7affd640 1 -- 172.21.15.37:0/2949726794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc84085310 con 0x7fcc8c0a8a10 2024-04-25T08:58:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.392+0000 7fcc8a59c640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc6c066f60 0x7fcc6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.392+0000 7fcc8a59c640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc6c066f60 0x7fcc6c069420 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c001210 tx=0x7fcc7c001120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.396+0000 7fcc7affd640 1 -- 172.21.15.37:0/2949726794 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc84012070 con 0x7fcc8c0a8a10 2024-04-25T08:58:57.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.556+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2949726794 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcc8c0c7bb0 con 0x7fcc6c066f60 2024-04-25T08:58:57.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.572+0000 7fcc7affd640 1 -- 172.21.15.37:0/2949726794 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246773 (secure 0 0 0) 0x7fcc8c0c7bb0 con 0x7fcc6c066f60 2024-04-25T08:58:57.567 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:57.569 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:58:57.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc6c066f60 msgr2=0x7fcc6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc6c066f60 0x7fcc6c069420 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c001210 tx=0x7fcc7c001120 comp rx=0 tx=0).stop 2024-04-25T08:58:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2949726794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc8c0a8a10 msgr2=0x7fcc8c133e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc8c0a8a10 0x7fcc8c133e60 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c06f0c0 tx=0x7fcc8400f4a0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2949726794 shutdown_connections 2024-04-25T08:58:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc6c066f60 0x7fcc6c069420 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc8c0a8a10 0x7fcc8c133e60 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc8c136380 0x7fcc8c0a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 --2- 172.21.15.37:0/2949726794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc74004810 0x7fcc8c1f0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2949726794 >> 172.21.15.37:0/2949726794 conn(0x7fcc8c0b11a0 msgr2=0x7fcc8c0c5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2949726794 shutdown_connections 2024-04-25T08:58:57.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.576+0000 7fcc90fa3640 1 -- 172.21.15.37:0/2949726794 wait complete. 2024-04-25T08:58:57.585 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-04-25T08:58:57.256685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259576,"kb_used_data":10120,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715144,"statfs":{"total":773094113280,"available":772828307456,"internally_reserved":0,"allocated":10362880,"data_stored":5404656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.791611"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476372+0000","last_change":"2024-04-25T08:58:52.476372+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476372+0000","last_undegraded":"2024-04-25T08:58:52.476372+0000","last_fullsized":"2024-04-25T08:58:52.476372+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485159+0000","last_change":"2024-04-25T08:58:52.485159+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485159+0000","last_undegraded":"2024-04-25T08:58:52.485159+0000","last_fullsized":"2024-04-25T08:58:52.485159+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476534+0000","last_change":"2024-04-25T08:58:52.476534+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476534+0000","last_undegraded":"2024-04-25T08:58:52.476534+0000","last_fullsized":"2024-04-25T08:58:52.476534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.482393+0000","last_change":"2024-04-25T08:58:52.482393+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.482393+0000","last_undegraded":"2024-04-25T08:58:52.482393+0000","last_fullsized":"2024-04-25T08:58:52.482393+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489217+0000","last_change":"2024-04-25T08:58:52.489217+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489217+0000","last_undegraded":"2024-04-25T08:58:52.489217+0000","last_fullsized":"2024-04-25T08:58:52.489217+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490958+0000","last_change":"2024-04-25T08:58:52.490958+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490958+0000","last_undegraded":"2024-04-25T08:58:52.490958+0000","last_fullsized":"2024-04-25T08:58:52.490958+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.495788+0000","last_change":"2024-04-25T08:58:52.495788+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.495788+0000","last_undegraded":"2024-04-25T08:58:52.495788+0000","last_fullsized":"2024-04-25T08:58:52.495788+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480355+0000","last_change":"2024-04-25T08:58:52.480355+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480355+0000","last_undegraded":"2024-04-25T08:58:52.480355+0000","last_fullsized":"2024-04-25T08:58:52.480355+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479616+0000","last_change":"2024-04-25T08:58:52.479616+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479616+0000","last_undegraded":"2024-04-25T08:58:52.479616+0000","last_fullsized":"2024-04-25T08:58:52.479616+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479974+0000","last_change":"2024-04-25T08:58:52.479974+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479974+0000","last_undegraded":"2024-04-25T08:58:52.479974+0000","last_fullsized":"2024-04-25T08:58:52.479974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491682+0000","last_change":"2024-04-25T08:58:52.491682+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491682+0000","last_undegraded":"2024-04-25T08:58:52.491682+0000","last_fullsized":"2024-04-25T08:58:52.491682+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479047+0000","last_change":"2024-04-25T08:58:52.479047+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479047+0000","last_undegraded":"2024-04-25T08:58:52.479047+0000","last_fullsized":"2024-04-25T08:58:52.479047+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.482025+0000","last_change":"2024-04-25T08:58:52.482025+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.482025+0000","last_undegraded":"2024-04-25T08:58:52.482025+0000","last_fullsized":"2024-04-25T08:58:52.482025+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489553+0000","last_change":"2024-04-25T08:58:52.489553+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489553+0000","last_undegraded":"2024-04-25T08:58:52.489553+0000","last_fullsized":"2024-04-25T08:58:52.489553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478028+0000","last_change":"2024-04-25T08:58:52.478028+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478028+0000","last_undegraded":"2024-04-25T08:58:52.478028+0000","last_fullsized":"2024-04-25T08:58:52.478028+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490790+0000","last_change":"2024-04-25T08:58:52.490790+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490790+0000","last_undegraded":"2024-04-25T08:58:52.490790+0000","last_fullsized":"2024-04-25T08:58:52.490790+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491623+0000","last_change":"2024-04-25T08:58:52.491623+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491623+0000","last_undegraded":"2024-04-25T08:58:52.491623+0000","last_fullsized":"2024-04-25T08:58:52.491623+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478387+0000","last_change":"2024-04-25T08:58:52.478387+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478387+0000","last_undegraded":"2024-04-25T08:58:52.478387+0000","last_fullsized":"2024-04-25T08:58:52.478387+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489792+0000","last_change":"2024-04-25T08:58:52.489792+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489792+0000","last_undegraded":"2024-04-25T08:58:52.489792+0000","last_fullsized":"2024-04-25T08:58:52.489792+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481843+0000","last_change":"2024-04-25T08:58:52.481843+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481843+0000","last_undegraded":"2024-04-25T08:58:52.481843+0000","last_fullsized":"2024-04-25T08:58:52.481843+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490305+0000","last_change":"2024-04-25T08:58:52.490305+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490305+0000","last_undegraded":"2024-04-25T08:58:52.490305+0000","last_fullsized":"2024-04-25T08:58:52.490305+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480491+0000","last_change":"2024-04-25T08:58:52.480491+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480491+0000","last_undegraded":"2024-04-25T08:58:52.480491+0000","last_fullsized":"2024-04-25T08:58:52.480491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481337+0000","last_change":"2024-04-25T08:58:52.481337+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481337+0000","last_undegraded":"2024-04-25T08:58:52.481337+0000","last_fullsized":"2024-04-25T08:58:52.481337+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.475900+0000","last_change":"2024-04-25T08:58:52.475900+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.475900+0000","last_undegraded":"2024-04-25T08:58:52.475900+0000","last_fullsized":"2024-04-25T08:58:52.475900+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487974+0000","last_change":"2024-04-25T08:58:52.487974+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487974+0000","last_undegraded":"2024-04-25T08:58:52.487974+0000","last_fullsized":"2024-04-25T08:58:52.487974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480582+0000","last_change":"2024-04-25T08:58:52.480582+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480582+0000","last_undegraded":"2024-04-25T08:58:52.480582+0000","last_fullsized":"2024-04-25T08:58:52.480582+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478227+0000","last_change":"2024-04-25T08:58:52.478227+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478227+0000","last_undegraded":"2024-04-25T08:58:52.478227+0000","last_fullsized":"2024-04-25T08:58:52.478227+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481255+0000","last_change":"2024-04-25T08:58:52.481255+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481255+0000","last_undegraded":"2024-04-25T08:58:52.481255+0000","last_fullsized":"2024-04-25T08:58:52.481255+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491670+0000","last_change":"2024-04-25T08:58:52.491670+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491670+0000","last_undegraded":"2024-04-25T08:58:52.491670+0000","last_fullsized":"2024-04-25T08:58:52.491670+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479529+0000","last_change":"2024-04-25T08:58:52.479529+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479529+0000","last_undegraded":"2024-04-25T08:58:52.479529+0000","last_fullsized":"2024-04-25T08:58:52.479529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485384+0000","last_change":"2024-04-25T08:58:52.485384+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485384+0000","last_undegraded":"2024-04-25T08:58:52.485384+0000","last_fullsized":"2024-04-25T08:58:52.485384+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490204+0000","last_change":"2024-04-25T08:58:52.490204+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490204+0000","last_undegraded":"2024-04-25T08:58:52.490204+0000","last_fullsized":"2024-04-25T08:58:52.490204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487585+0000","last_change":"2024-04-25T08:58:52.487585+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487585+0000","last_undegraded":"2024-04-25T08:58:52.487585+0000","last_fullsized":"2024-04-25T08:58:52.487585+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.509098+0000","last_change":"2024-04-25T08:58:52.499317+0000","last_active":"2024-04-25T08:58:52.509098+0000","last_peered":"2024-04-25T08:58:52.509098+0000","last_clean":"2024-04-25T08:58:52.509098+0000","last_became_active":"2024-04-25T08:58:52.498145+0000","last_became_peered":"2024-04-25T08:58:52.498145+0000","last_unstale":"2024-04-25T08:58:52.509098+0000","last_undegraded":"2024-04-25T08:58:52.509098+0000","last_fullsized":"2024-04-25T08:58:52.509098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:50.604902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487225+0000","last_change":"2024-04-25T08:58:52.487225+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487225+0000","last_undegraded":"2024-04-25T08:58:52.487225+0000","last_fullsized":"2024-04-25T08:58:52.487225+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518038+0000","last_change":"2024-04-25T08:58:52.513581+0000","last_active":"2024-04-25T08:58:52.518038+0000","last_peered":"2024-04-25T08:58:52.518038+0000","last_clean":"2024-04-25T08:58:52.518038+0000","last_became_active":"2024-04-25T08:58:52.512774+0000","last_became_peered":"2024-04-25T08:58:52.512774+0000","last_unstale":"2024-04-25T08:58:52.518038+0000","last_undegraded":"2024-04-25T08:58:52.518038+0000","last_fullsized":"2024-04-25T08:58:52.518038+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:45.689538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487757+0000","last_change":"2024-04-25T08:58:52.487757+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487757+0000","last_undegraded":"2024-04-25T08:58:52.487757+0000","last_fullsized":"2024-04-25T08:58:52.487757+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512573+0000","last_change":"2024-04-25T08:58:52.508417+0000","last_active":"2024-04-25T08:58:52.512573+0000","last_peered":"2024-04-25T08:58:52.512573+0000","last_clean":"2024-04-25T08:58:52.512573+0000","last_became_active":"2024-04-25T08:58:52.507854+0000","last_became_peered":"2024-04-25T08:58:52.507854+0000","last_unstale":"2024-04-25T08:58:52.512573+0000","last_undegraded":"2024-04-25T08:58:52.512573+0000","last_fullsized":"2024-04-25T08:58:52.512573+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:02.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491046+0000","last_change":"2024-04-25T08:58:52.491046+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491046+0000","last_undegraded":"2024-04-25T08:58:52.491046+0000","last_fullsized":"2024-04-25T08:58:52.491046+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508855+0000","last_change":"2024-04-25T08:58:52.494753+0000","last_active":"2024-04-25T08:58:52.508855+0000","last_peered":"2024-04-25T08:58:52.508855+0000","last_clean":"2024-04-25T08:58:52.508855+0000","last_became_active":"2024-04-25T08:58:52.493972+0000","last_became_peered":"2024-04-25T08:58:52.493972+0000","last_unstale":"2024-04-25T08:58:52.508855+0000","last_undegraded":"2024-04-25T08:58:52.508855+0000","last_fullsized":"2024-04-25T08:58:52.508855+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:14.712772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479776+0000","last_change":"2024-04-25T08:58:52.479776+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479776+0000","last_undegraded":"2024-04-25T08:58:52.479776+0000","last_fullsized":"2024-04-25T08:58:52.479776+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512397+0000","last_change":"2024-04-25T08:58:52.505294+0000","last_active":"2024-04-25T08:58:52.512397+0000","last_peered":"2024-04-25T08:58:52.512397+0000","last_clean":"2024-04-25T08:58:52.512397+0000","last_became_active":"2024-04-25T08:58:52.504512+0000","last_became_peered":"2024-04-25T08:58:52.504512+0000","last_unstale":"2024-04-25T08:58:52.512397+0000","last_undegraded":"2024-04-25T08:58:52.512397+0000","last_fullsized":"2024-04-25T08:58:52.512397+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:56.905836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478321+0000","last_change":"2024-04-25T08:58:52.478321+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478321+0000","last_undegraded":"2024-04-25T08:58:52.478321+0000","last_fullsized":"2024-04-25T08:58:52.478321+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507544+0000","last_change":"2024-04-25T08:58:52.503172+0000","last_active":"2024-04-25T08:58:52.507544+0000","last_peered":"2024-04-25T08:58:52.507544+0000","last_clean":"2024-04-25T08:58:52.507544+0000","last_became_active":"2024-04-25T08:58:52.502702+0000","last_became_peered":"2024-04-25T08:58:52.502702+0000","last_unstale":"2024-04-25T08:58:52.507544+0000","last_undegraded":"2024-04-25T08:58:52.507544+0000","last_fullsized":"2024-04-25T08:58:52.507544+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486416+0000","last_change":"2024-04-25T08:58:52.486416+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486416+0000","last_undegraded":"2024-04-25T08:58:52.486416+0000","last_fullsized":"2024-04-25T08:58:52.486416+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.502729+0000","last_change":"2024-04-25T08:58:52.501970+0000","last_active":"2024-04-25T08:58:52.502729+0000","last_peered":"2024-04-25T08:58:52.502729+0000","last_clean":"2024-04-25T08:58:52.502729+0000","last_became_active":"2024-04-25T08:58:52.500708+0000","last_became_peered":"2024-04-25T08:58:52.500708+0000","last_unstale":"2024-04-25T08:58:52.502729+0000","last_undegraded":"2024-04-25T08:58:52.502729+0000","last_fullsized":"2024-04-25T08:58:52.502729+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:25.488383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480907+0000","last_change":"2024-04-25T08:58:52.480907+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480907+0000","last_undegraded":"2024-04-25T08:58:52.480907+0000","last_fullsized":"2024-04-25T08:58:52.480907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512711+0000","last_change":"2024-04-25T08:58:52.506757+0000","last_active":"2024-04-25T08:58:52.512711+0000","last_peered":"2024-04-25T08:58:52.512711+0000","last_clean":"2024-04-25T08:58:52.512711+0000","last_became_active":"2024-04-25T08:58:52.506014+0000","last_became_peered":"2024-04-25T08:58:52.506014+0000","last_unstale":"2024-04-25T08:58:52.512711+0000","last_undegraded":"2024-04-25T08:58:52.512711+0000","last_fullsized":"2024-04-25T08:58:52.512711+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476026+0000","last_change":"2024-04-25T08:58:52.476026+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476026+0000","last_undegraded":"2024-04-25T08:58:52.476026+0000","last_fullsized":"2024-04-25T08:58:52.476026+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508755+0000","last_change":"2024-04-25T08:58:52.505203+0000","last_active":"2024-04-25T08:58:52.508755+0000","last_peered":"2024-04-25T08:58:52.508755+0000","last_clean":"2024-04-25T08:58:52.508755+0000","last_became_active":"2024-04-25T08:58:52.504503+0000","last_became_peered":"2024-04-25T08:58:52.504503+0000","last_unstale":"2024-04-25T08:58:52.508755+0000","last_undegraded":"2024-04-25T08:58:52.508755+0000","last_fullsized":"2024-04-25T08:58:52.508755+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:15.105995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487774+0000","last_change":"2024-04-25T08:58:52.487774+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487774+0000","last_undegraded":"2024-04-25T08:58:52.487774+0000","last_fullsized":"2024-04-25T08:58:52.487774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.495939+0000","last_change":"2024-04-25T08:58:52.493504+0000","last_active":"2024-04-25T08:58:52.495939+0000","last_peered":"2024-04-25T08:58:52.495939+0000","last_clean":"2024-04-25T08:58:52.495939+0000","last_became_active":"2024-04-25T08:58:52.493077+0000","last_became_peered":"2024-04-25T08:58:52.493077+0000","last_unstale":"2024-04-25T08:58:52.495939+0000","last_undegraded":"2024-04-25T08:58:52.495939+0000","last_fullsized":"2024-04-25T08:58:52.495939+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:12.438660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477801+0000","last_change":"2024-04-25T08:58:52.477801+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477801+0000","last_undegraded":"2024-04-25T08:58:52.477801+0000","last_fullsized":"2024-04-25T08:58:52.477801+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518279+0000","last_change":"2024-04-25T08:58:52.515324+0000","last_active":"2024-04-25T08:58:52.518279+0000","last_peered":"2024-04-25T08:58:52.518279+0000","last_clean":"2024-04-25T08:58:52.518279+0000","last_became_active":"2024-04-25T08:58:52.513958+0000","last_became_peered":"2024-04-25T08:58:52.513958+0000","last_unstale":"2024-04-25T08:58:52.518279+0000","last_undegraded":"2024-04-25T08:58:52.518279+0000","last_fullsized":"2024-04-25T08:58:52.518279+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:07.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477385+0000","last_change":"2024-04-25T08:58:52.477385+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477385+0000","last_undegraded":"2024-04-25T08:58:52.477385+0000","last_fullsized":"2024-04-25T08:58:52.477385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.496529+0000","last_change":"2024-04-25T08:58:52.493433+0000","last_active":"2024-04-25T08:58:52.496529+0000","last_peered":"2024-04-25T08:58:52.496529+0000","last_clean":"2024-04-25T08:58:52.496529+0000","last_became_active":"2024-04-25T08:58:52.493061+0000","last_became_peered":"2024-04-25T08:58:52.493061+0000","last_unstale":"2024-04-25T08:58:52.496529+0000","last_undegraded":"2024-04-25T08:58:52.496529+0000","last_fullsized":"2024-04-25T08:58:52.496529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:04.908717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479199+0000","last_change":"2024-04-25T08:58:52.479199+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479199+0000","last_undegraded":"2024-04-25T08:58:52.479199+0000","last_fullsized":"2024-04-25T08:58:52.479199+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512641+0000","last_change":"2024-04-25T08:58:52.507149+0000","last_active":"2024-04-25T08:58:52.512641+0000","last_peered":"2024-04-25T08:58:52.512641+0000","last_clean":"2024-04-25T08:58:52.512641+0000","last_became_active":"2024-04-25T08:58:52.506632+0000","last_became_peered":"2024-04-25T08:58:52.506632+0000","last_unstale":"2024-04-25T08:58:52.512641+0000","last_undegraded":"2024-04-25T08:58:52.512641+0000","last_fullsized":"2024-04-25T08:58:52.512641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:16.136540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476701+0000","last_change":"2024-04-25T08:58:52.476701+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476701+0000","last_undegraded":"2024-04-25T08:58:52.476701+0000","last_fullsized":"2024-04-25T08:58:52.476701+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513002+0000","last_change":"2024-04-25T08:58:52.508620+0000","last_active":"2024-04-25T08:58:52.513002+0000","last_peered":"2024-04-25T08:58:52.513002+0000","last_clean":"2024-04-25T08:58:52.513002+0000","last_became_active":"2024-04-25T08:58:52.508361+0000","last_became_peered":"2024-04-25T08:58:52.508361+0000","last_unstale":"2024-04-25T08:58:52.513002+0000","last_undegraded":"2024-04-25T08:58:52.513002+0000","last_fullsized":"2024-04-25T08:58:52.513002+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.319442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480028+0000","last_change":"2024-04-25T08:58:52.480028+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480028+0000","last_undegraded":"2024-04-25T08:58:52.480028+0000","last_fullsized":"2024-04-25T08:58:52.480028+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507782+0000","last_change":"2024-04-25T08:58:52.503640+0000","last_active":"2024-04-25T08:58:52.507782+0000","last_peered":"2024-04-25T08:58:52.507782+0000","last_clean":"2024-04-25T08:58:52.507782+0000","last_became_active":"2024-04-25T08:58:52.503271+0000","last_became_peered":"2024-04-25T08:58:52.503271+0000","last_unstale":"2024-04-25T08:58:52.507782+0000","last_undegraded":"2024-04-25T08:58:52.507782+0000","last_fullsized":"2024-04-25T08:58:52.507782+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:02.528462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477547+0000","last_change":"2024-04-25T08:58:52.477547+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477547+0000","last_undegraded":"2024-04-25T08:58:52.477547+0000","last_fullsized":"2024-04-25T08:58:52.477547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507718+0000","last_change":"2024-04-25T08:58:52.503187+0000","last_active":"2024-04-25T08:58:52.507718+0000","last_peered":"2024-04-25T08:58:52.507718+0000","last_clean":"2024-04-25T08:58:52.507718+0000","last_became_active":"2024-04-25T08:58:52.502662+0000","last_became_peered":"2024-04-25T08:58:52.502662+0000","last_unstale":"2024-04-25T08:58:52.507718+0000","last_undegraded":"2024-04-25T08:58:52.507718+0000","last_fullsized":"2024-04-25T08:58:52.507718+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:28.688051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485819+0000","last_change":"2024-04-25T08:58:52.485819+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485819+0000","last_undegraded":"2024-04-25T08:58:52.485819+0000","last_fullsized":"2024-04-25T08:58:52.485819+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513658+0000","last_change":"2024-04-25T08:58:52.502738+0000","last_active":"2024-04-25T08:58:52.513658+0000","last_peered":"2024-04-25T08:58:52.513658+0000","last_clean":"2024-04-25T08:58:52.513658+0000","last_became_active":"2024-04-25T08:58:52.501965+0000","last_became_peered":"2024-04-25T08:58:52.501965+0000","last_unstale":"2024-04-25T08:58:52.513658+0000","last_undegraded":"2024-04-25T08:58:52.513658+0000","last_fullsized":"2024-04-25T08:58:52.513658+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:30.608887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481261+0000","last_change":"2024-04-25T08:58:52.481261+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481261+0000","last_undegraded":"2024-04-25T08:58:52.481261+0000","last_fullsized":"2024-04-25T08:58:52.481261+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513182+0000","last_change":"2024-04-25T08:58:52.510654+0000","last_active":"2024-04-25T08:58:52.513182+0000","last_peered":"2024-04-25T08:58:52.513182+0000","last_clean":"2024-04-25T08:58:52.513182+0000","last_became_active":"2024-04-25T08:58:52.509735+0000","last_became_peered":"2024-04-25T08:58:52.509735+0000","last_unstale":"2024-04-25T08:58:52.513182+0000","last_undegraded":"2024-04-25T08:58:52.513182+0000","last_fullsized":"2024-04-25T08:58:52.513182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:23.084950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476876+0000","last_change":"2024-04-25T08:58:52.476876+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476876+0000","last_undegraded":"2024-04-25T08:58:52.476876+0000","last_fullsized":"2024-04-25T08:58:52.476876+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518225+0000","last_change":"2024-04-25T08:58:52.514656+0000","last_active":"2024-04-25T08:58:52.518225+0000","last_peered":"2024-04-25T08:58:52.518225+0000","last_clean":"2024-04-25T08:58:52.518225+0000","last_became_active":"2024-04-25T08:58:52.513150+0000","last_became_peered":"2024-04-25T08:58:52.513150+0000","last_unstale":"2024-04-25T08:58:52.518225+0000","last_undegraded":"2024-04-25T08:58:52.518225+0000","last_fullsized":"2024-04-25T08:58:52.518225+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:40.111643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476469+0000","last_change":"2024-04-25T08:58:52.476469+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476469+0000","last_undegraded":"2024-04-25T08:58:52.476469+0000","last_fullsized":"2024-04-25T08:58:52.476469+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503595+0000","last_change":"2024-04-25T08:58:52.494154+0000","last_active":"2024-04-25T08:58:52.503595+0000","last_peered":"2024-04-25T08:58:52.503595+0000","last_clean":"2024-04-25T08:58:52.503595+0000","last_became_active":"2024-04-25T08:58:52.493429+0000","last_became_peered":"2024-04-25T08:58:52.493429+0000","last_unstale":"2024-04-25T08:58:52.503595+0000","last_undegraded":"2024-04-25T08:58:52.503595+0000","last_fullsized":"2024-04-25T08:58:52.503595+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:30.608887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.483968+0000","last_change":"2024-04-25T08:58:52.483968+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.483968+0000","last_undegraded":"2024-04-25T08:58:52.483968+0000","last_fullsized":"2024-04-25T08:58:52.483968+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.501305+0000","last_change":"2024-04-25T08:58:52.494299+0000","last_active":"2024-04-25T08:58:52.501305+0000","last_peered":"2024-04-25T08:58:52.501305+0000","last_clean":"2024-04-25T08:58:52.501305+0000","last_became_active":"2024-04-25T08:58:52.493557+0000","last_became_peered":"2024-04-25T08:58:52.493557+0000","last_unstale":"2024-04-25T08:58:52.501305+0000","last_undegraded":"2024-04-25T08:58:52.501305+0000","last_fullsized":"2024-04-25T08:58:52.501305+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:41.335795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486514+0000","last_change":"2024-04-25T08:58:52.486514+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486514+0000","last_undegraded":"2024-04-25T08:58:52.486514+0000","last_fullsized":"2024-04-25T08:58:52.486514+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507821+0000","last_change":"2024-04-25T08:58:52.500643+0000","last_active":"2024-04-25T08:58:52.507821+0000","last_peered":"2024-04-25T08:58:52.507821+0000","last_clean":"2024-04-25T08:58:52.507821+0000","last_became_active":"2024-04-25T08:58:52.499551+0000","last_became_peered":"2024-04-25T08:58:52.499551+0000","last_unstale":"2024-04-25T08:58:52.507821+0000","last_undegraded":"2024-04-25T08:58:52.507821+0000","last_fullsized":"2024-04-25T08:58:52.507821+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:06.052032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487020+0000","last_change":"2024-04-25T08:58:52.487020+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487020+0000","last_undegraded":"2024-04-25T08:58:52.487020+0000","last_fullsized":"2024-04-25T08:58:52.487020+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503818+0000","last_change":"2024-04-25T08:58:52.501153+0000","last_active":"2024-04-25T08:58:52.503818+0000","last_peered":"2024-04-25T08:58:52.503818+0000","last_clean":"2024-04-25T08:58:52.503818+0000","last_became_active":"2024-04-25T08:58:52.497820+0000","last_became_peered":"2024-04-25T08:58:52.497820+0000","last_unstale":"2024-04-25T08:58:52.503818+0000","last_undegraded":"2024-04-25T08:58:52.503818+0000","last_fullsized":"2024-04-25T08:58:52.503818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:05.283812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481896+0000","last_change":"2024-04-25T08:58:52.481896+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481896+0000","last_undegraded":"2024-04-25T08:58:52.481896+0000","last_fullsized":"2024-04-25T08:58:52.481896+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.517562+0000","last_change":"2024-04-25T08:58:52.511956+0000","last_active":"2024-04-25T08:58:52.517562+0000","last_peered":"2024-04-25T08:58:52.517562+0000","last_clean":"2024-04-25T08:58:52.517562+0000","last_became_active":"2024-04-25T08:58:52.511196+0000","last_became_peered":"2024-04-25T08:58:52.511196+0000","last_unstale":"2024-04-25T08:58:52.517562+0000","last_undegraded":"2024-04-25T08:58:52.517562+0000","last_fullsized":"2024-04-25T08:58:52.517562+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:09.214520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476443+0000","last_change":"2024-04-25T08:58:52.476443+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476443+0000","last_undegraded":"2024-04-25T08:58:52.476443+0000","last_fullsized":"2024-04-25T08:58:52.476443+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.517853+0000","last_change":"2024-04-25T08:58:52.512155+0000","last_active":"2024-04-25T08:58:52.517853+0000","last_peered":"2024-04-25T08:58:52.517853+0000","last_clean":"2024-04-25T08:58:52.517853+0000","last_became_active":"2024-04-25T08:58:52.506602+0000","last_became_peered":"2024-04-25T08:58:52.506602+0000","last_unstale":"2024-04-25T08:58:52.517853+0000","last_undegraded":"2024-04-25T08:58:52.517853+0000","last_fullsized":"2024-04-25T08:58:52.517853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:09.340774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476535+0000","last_change":"2024-04-25T08:58:52.476535+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476535+0000","last_undegraded":"2024-04-25T08:58:52.476535+0000","last_fullsized":"2024-04-25T08:58:52.476535+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.515015+0000","last_change":"2024-04-25T08:58:52.508006+0000","last_active":"2024-04-25T08:58:52.515015+0000","last_peered":"2024-04-25T08:58:52.515015+0000","last_clean":"2024-04-25T08:58:52.515015+0000","last_became_active":"2024-04-25T08:58:52.507149+0000","last_became_peered":"2024-04-25T08:58:52.507149+0000","last_unstale":"2024-04-25T08:58:52.515015+0000","last_undegraded":"2024-04-25T08:58:52.515015+0000","last_fullsized":"2024-04-25T08:58:52.515015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:04.306110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486979+0000","last_change":"2024-04-25T08:58:52.486979+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486979+0000","last_undegraded":"2024-04-25T08:58:52.486979+0000","last_fullsized":"2024-04-25T08:58:52.486979+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508975+0000","last_change":"2024-04-25T08:58:52.495139+0000","last_active":"2024-04-25T08:58:52.508975+0000","last_peered":"2024-04-25T08:58:52.508975+0000","last_clean":"2024-04-25T08:58:52.508975+0000","last_became_active":"2024-04-25T08:58:52.494329+0000","last_became_peered":"2024-04-25T08:58:52.494329+0000","last_unstale":"2024-04-25T08:58:52.508975+0000","last_undegraded":"2024-04-25T08:58:52.508975+0000","last_fullsized":"2024-04-25T08:58:52.508975+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:54.968958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478270+0000","last_change":"2024-04-25T08:58:52.478270+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478270+0000","last_undegraded":"2024-04-25T08:58:52.478270+0000","last_fullsized":"2024-04-25T08:58:52.478270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.516378+0000","last_change":"2024-04-25T08:58:52.513900+0000","last_active":"2024-04-25T08:58:52.516378+0000","last_peered":"2024-04-25T08:58:52.516378+0000","last_clean":"2024-04-25T08:58:52.516378+0000","last_became_active":"2024-04-25T08:58:52.513643+0000","last_became_peered":"2024-04-25T08:58:52.513643+0000","last_unstale":"2024-04-25T08:58:52.516378+0000","last_undegraded":"2024-04-25T08:58:52.516378+0000","last_fullsized":"2024-04-25T08:58:52.516378+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:51.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476352+0000","last_change":"2024-04-25T08:58:52.476352+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476352+0000","last_undegraded":"2024-04-25T08:58:52.476352+0000","last_fullsized":"2024-04-25T08:58:52.476352+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503475+0000","last_change":"2024-04-25T08:58:52.493529+0000","last_active":"2024-04-25T08:58:52.503475+0000","last_peered":"2024-04-25T08:58:52.503475+0000","last_clean":"2024-04-25T08:58:52.503475+0000","last_became_active":"2024-04-25T08:58:52.492691+0000","last_became_peered":"2024-04-25T08:58:52.492691+0000","last_unstale":"2024-04-25T08:58:52.503475+0000","last_undegraded":"2024-04-25T08:58:52.503475+0000","last_fullsized":"2024-04-25T08:58:52.503475+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:50.604902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476218+0000","last_change":"2024-04-25T08:58:52.476218+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476218+0000","last_undegraded":"2024-04-25T08:58:52.476218+0000","last_fullsized":"2024-04-25T08:58:52.476218+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.509136+0000","last_change":"2024-04-25T08:58:52.495019+0000","last_active":"2024-04-25T08:58:52.509136+0000","last_peered":"2024-04-25T08:58:52.509136+0000","last_clean":"2024-04-25T08:58:52.509136+0000","last_became_active":"2024-04-25T08:58:52.494237+0000","last_became_peered":"2024-04-25T08:58:52.494237+0000","last_unstale":"2024-04-25T08:58:52.509136+0000","last_undegraded":"2024-04-25T08:58:52.509136+0000","last_fullsized":"2024-04-25T08:58:52.509136+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:35.222412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.475654+0000","last_change":"2024-04-25T08:58:52.475654+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.475654+0000","last_undegraded":"2024-04-25T08:58:52.475654+0000","last_fullsized":"2024-04-25T08:58:52.475654+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503768+0000","last_change":"2024-04-25T08:58:52.502403+0000","last_active":"2024-04-25T08:58:52.503768+0000","last_peered":"2024-04-25T08:58:52.503768+0000","last_clean":"2024-04-25T08:58:52.503768+0000","last_became_active":"2024-04-25T08:58:52.501501+0000","last_became_peered":"2024-04-25T08:58:52.501501+0000","last_unstale":"2024-04-25T08:58:52.503768+0000","last_undegraded":"2024-04-25T08:58:52.503768+0000","last_fullsized":"2024-04-25T08:58:52.503768+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:12.458893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490599+0000","last_change":"2024-04-25T08:58:52.490599+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490599+0000","last_undegraded":"2024-04-25T08:58:52.490599+0000","last_fullsized":"2024-04-25T08:58:52.490599+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.498266+0000","last_change":"2024-04-25T08:58:52.493037+0000","last_active":"2024-04-25T08:58:52.498266+0000","last_peered":"2024-04-25T08:58:52.498266+0000","last_clean":"2024-04-25T08:58:52.498266+0000","last_became_active":"2024-04-25T08:58:52.492644+0000","last_became_peered":"2024-04-25T08:58:52.492644+0000","last_unstale":"2024-04-25T08:58:52.498266+0000","last_undegraded":"2024-04-25T08:58:52.498266+0000","last_fullsized":"2024-04-25T08:58:52.498266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:07.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":307,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.497611+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:58:52.497611+0000","last_peered":"2024-04-25T08:58:52.497611+0000","last_clean":"2024-04-25T08:58:52.497611+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:58:52.497611+0000","last_undegraded":"2024-04-25T08:58:52.497611+0000","last_fullsized":"2024-04-25T08:58:52.497611+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1740800,"data_stored":1118358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1740800,"data_stored":1118358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:58:57.590 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:58:57.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4f049ab640 1 Processor -- start 2024-04-25T08:58:57.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4f049ab640 1 -- start start 2024-04-25T08:58:57.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4f049ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000b3f50 0x7f4f0015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4f049ab640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f000b2090 con 0x7f4f000b24a0 2024-04-25T08:58:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4f049ab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f000b43e0 con 0x7f4f0015a700 2024-04-25T08:58:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4f049ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0015f200 con 0x7f4f000b3f50 2024-04-25T08:58:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efed9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4f000b24a0 0x7f4f0006e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37224/0 (socket says 172.21.15.37:37224) 2024-04-25T08:58:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efed9d640 1 -- 172.21.15.37:0/3036991984 learned_addr learned my addr 172.21.15.37:0/3036991984 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4ef7fff640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000b3f50 0x7f4f0015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914702725 0 0) 0x7f4f000b2090 con 0x7f4f000b24a0 2024-04-25T08:58:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eec003530 con 0x7f4f000b24a0 2024-04-25T08:58:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2027518317 0 0) 0x7f4eec003530 con 0x7f4f000b24a0 2024-04-25T08:58:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000b3f50 msgr2=0x7f4f0015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000b3f50 0x7f4f0015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 >> v1:172.21.15.203:6789/0 conn(0x7f4f0015a700 legacy=0x7f4f000cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f000b46c0 con 0x7f4f000b24a0 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f4ef8001a00 con 0x7f4f000b24a0 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 >> v1:172.21.15.37:6789/0 conn(0x7f4f000b24a0 legacy=0x7f4f0006e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 0x7f4eec003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec004810 0x7f4eec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec0052f0 0x7f4eec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0015f200 con 0x7f4eec003a60 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec00bff0 con 0x7f4eec004810 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec00c1f0 con 0x7f4eec0052f0 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efe59c640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 0x7f4eec003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4ef7fff640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec004810 0x7f4eec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efed9d640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec0052f0 0x7f4eec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efe59c640 1 -- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec0052f0 msgr2=0x7f4eec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efe59c640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec0052f0 0x7f4eec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efe59c640 1 -- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec004810 msgr2=0x7f4eec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efe59c640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec004810 0x7f4eec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efe59c640 1 -- 172.21.15.37:0/3036991984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec003530 con 0x7f4eec003a60 2024-04-25T08:58:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4ef7fff640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4eec004810 0x7f4eec004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.760+0000 7f4efed9d640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4eec0052f0 0x7f4eec00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:57.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4efe59c640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 0x7f4eec003e80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0001780 tx=0x7f4ef001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d9fab4b44d29a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00013a0 con 0x7f4eec003a60 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef00052f0 con 0x7f4eec003a60 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4efdd9b640 1 -- 172.21.15.37:0/3036991984 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0003840 con 0x7f4eec003a60 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 -- 172.21.15.37:0/3036991984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 msgr2=0x7f4eec003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 --2- 172.21.15.37:0/3036991984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 0x7f4eec003e80 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0001780 tx=0x7f4ef001c3f0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4efed9d640 1 -- 172.21.15.37:0/3036991984 reap_dead start 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 -- 172.21.15.37:0/3036991984 shutdown_connections 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 -- 172.21.15.37:0/3036991984 >> 172.21.15.37:0/3036991984 conn(0x7f4f000b11a0 msgr2=0x7f4f000b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 -- 172.21.15.37:0/3036991984 shutdown_connections 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 -- 172.21.15.37:0/3036991984 wait complete. 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 Processor -- start 2024-04-25T08:58:57.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 -- start start 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 0x7f4f00151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f0015cae0 0x7f4f0015cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000af470 0x7f4f000af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000cd310 con 0x7f4eec003a60 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000ce600 con 0x7f4f0015cae0 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4f049ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000cd490 con 0x7f4f000af470 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4efe59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 0x7f4f00151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4efe59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 0x7f4f00151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56728/0 (socket says 172.21.15.37:56728) 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.764+0000 7f4efe59c640 1 -- 172.21.15.37:0/563488233 learned_addr learned my addr 172.21.15.37:0/563488233 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4ef7fff640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f0015cae0 0x7f4f0015cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4efe59c640 1 -- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000af470 msgr2=0x7f4f000af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4efe59c640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000af470 0x7f4f000af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4efe59c640 1 -- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f0015cae0 msgr2=0x7f4f0015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4efe59c640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f0015cae0 0x7f4f0015cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4efe59c640 1 -- 172.21.15.37:0/563488233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f001f80f0 con 0x7f4eec003a60 2024-04-25T08:58:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4ef7fff640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f0015cae0 0x7f4f0015cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4efe59c640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 0x7f4f00151500 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0001780 tx=0x7f4eec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4ef6ffd640 1 -- 172.21.15.37:0/563488233 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0007a40 con 0x7f4eec003a60 2024-04-25T08:58:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4ef6ffd640 1 -- 172.21.15.37:0/563488233 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef00bb070 con 0x7f4eec003a60 2024-04-25T08:58:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4f049ab640 1 -- 172.21.15.37:0/563488233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f001f7330 con 0x7f4eec003a60 2024-04-25T08:58:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4ef6ffd640 1 -- 172.21.15.37:0/563488233 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef00013a0 con 0x7f4eec003a60 2024-04-25T08:58:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4f049ab640 1 -- 172.21.15.37:0/563488233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f001f78f0 con 0x7f4eec003a60 2024-04-25T08:58:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4f049ab640 1 -- 172.21.15.37:0/563488233 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0004efa0 con 0x7f4eec003a60 2024-04-25T08:58:57.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4ef6ffd640 1 -- 172.21.15.37:0/563488233 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4ef00b4040 con 0x7f4eec003a60 2024-04-25T08:58:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4ef6ffd640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ee0066f60 0x7f4ee0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4ef6ffd640 1 -- 172.21.15.37:0/563488233 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4ef011f400 con 0x7f4eec003a60 2024-04-25T08:58:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4ef7fff640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ee0066f60 0x7f4ee0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:57.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.768+0000 7f4ef7fff640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ee0066f60 0x7f4ee0069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4f0015fb10 tx=0x7f4ee8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:57.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.772+0000 7f4ef6ffd640 1 -- 172.21.15.37:0/563488233 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4ef0016130 con 0x7f4eec003a60 2024-04-25T08:58:57.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.928+0000 7f4f049ab640 1 -- 172.21.15.37:0/563488233 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f00151a40 con 0x7f4ee0066f60 2024-04-25T08:58:57.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4ef6ffd640 1 -- 172.21.15.37:0/563488233 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246773 (secure 0 0 0) 0x7f4f00151a40 con 0x7f4ee0066f60 2024-04-25T08:58:57.939 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:57.940 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:58:57.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 -- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ee0066f60 msgr2=0x7f4ee0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ee0066f60 0x7f4ee0069420 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f4f0015fb10 tx=0x7f4ee8008040 comp rx=0 tx=0).stop 2024-04-25T08:58:57.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 -- 172.21.15.37:0/563488233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 msgr2=0x7f4f00151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:57.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 0x7f4f00151500 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0001780 tx=0x7f4eec005840 comp rx=0 tx=0).stop 2024-04-25T08:58:57.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 -- 172.21.15.37:0/563488233 shutdown_connections 2024-04-25T08:58:57.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4ee0066f60 0x7f4ee0069420 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f000af470 0x7f4f000af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f0015cae0 0x7f4f0015cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 --2- 172.21.15.37:0/563488233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4eec003a60 0x7f4f00151500 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:57.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 -- 172.21.15.37:0/563488233 >> 172.21.15.37:0/563488233 conn(0x7f4f000b11a0 msgr2=0x7f4f00165e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:57.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 -- 172.21.15.37:0/563488233 shutdown_connections 2024-04-25T08:58:57.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:57.948+0000 7f4f049ab640 1 -- 172.21.15.37:0/563488233 wait complete. 2024-04-25T08:58:57.957 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-04-25T08:58:57.256685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259576,"kb_used_data":10120,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715144,"statfs":{"total":773094113280,"available":772828307456,"internally_reserved":0,"allocated":10362880,"data_stored":5404656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.791611"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476372+0000","last_change":"2024-04-25T08:58:52.476372+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476372+0000","last_undegraded":"2024-04-25T08:58:52.476372+0000","last_fullsized":"2024-04-25T08:58:52.476372+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485159+0000","last_change":"2024-04-25T08:58:52.485159+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485159+0000","last_undegraded":"2024-04-25T08:58:52.485159+0000","last_fullsized":"2024-04-25T08:58:52.485159+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476534+0000","last_change":"2024-04-25T08:58:52.476534+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476534+0000","last_undegraded":"2024-04-25T08:58:52.476534+0000","last_fullsized":"2024-04-25T08:58:52.476534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.482393+0000","last_change":"2024-04-25T08:58:52.482393+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.482393+0000","last_undegraded":"2024-04-25T08:58:52.482393+0000","last_fullsized":"2024-04-25T08:58:52.482393+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489217+0000","last_change":"2024-04-25T08:58:52.489217+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489217+0000","last_undegraded":"2024-04-25T08:58:52.489217+0000","last_fullsized":"2024-04-25T08:58:52.489217+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490958+0000","last_change":"2024-04-25T08:58:52.490958+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490958+0000","last_undegraded":"2024-04-25T08:58:52.490958+0000","last_fullsized":"2024-04-25T08:58:52.490958+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.495788+0000","last_change":"2024-04-25T08:58:52.495788+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.495788+0000","last_undegraded":"2024-04-25T08:58:52.495788+0000","last_fullsized":"2024-04-25T08:58:52.495788+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480355+0000","last_change":"2024-04-25T08:58:52.480355+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480355+0000","last_undegraded":"2024-04-25T08:58:52.480355+0000","last_fullsized":"2024-04-25T08:58:52.480355+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479616+0000","last_change":"2024-04-25T08:58:52.479616+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479616+0000","last_undegraded":"2024-04-25T08:58:52.479616+0000","last_fullsized":"2024-04-25T08:58:52.479616+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479974+0000","last_change":"2024-04-25T08:58:52.479974+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479974+0000","last_undegraded":"2024-04-25T08:58:52.479974+0000","last_fullsized":"2024-04-25T08:58:52.479974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491682+0000","last_change":"2024-04-25T08:58:52.491682+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491682+0000","last_undegraded":"2024-04-25T08:58:52.491682+0000","last_fullsized":"2024-04-25T08:58:52.491682+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479047+0000","last_change":"2024-04-25T08:58:52.479047+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479047+0000","last_undegraded":"2024-04-25T08:58:52.479047+0000","last_fullsized":"2024-04-25T08:58:52.479047+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.482025+0000","last_change":"2024-04-25T08:58:52.482025+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.482025+0000","last_undegraded":"2024-04-25T08:58:52.482025+0000","last_fullsized":"2024-04-25T08:58:52.482025+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489553+0000","last_change":"2024-04-25T08:58:52.489553+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489553+0000","last_undegraded":"2024-04-25T08:58:52.489553+0000","last_fullsized":"2024-04-25T08:58:52.489553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478028+0000","last_change":"2024-04-25T08:58:52.478028+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478028+0000","last_undegraded":"2024-04-25T08:58:52.478028+0000","last_fullsized":"2024-04-25T08:58:52.478028+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490790+0000","last_change":"2024-04-25T08:58:52.490790+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490790+0000","last_undegraded":"2024-04-25T08:58:52.490790+0000","last_fullsized":"2024-04-25T08:58:52.490790+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491623+0000","last_change":"2024-04-25T08:58:52.491623+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491623+0000","last_undegraded":"2024-04-25T08:58:52.491623+0000","last_fullsized":"2024-04-25T08:58:52.491623+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478387+0000","last_change":"2024-04-25T08:58:52.478387+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478387+0000","last_undegraded":"2024-04-25T08:58:52.478387+0000","last_fullsized":"2024-04-25T08:58:52.478387+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489792+0000","last_change":"2024-04-25T08:58:52.489792+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489792+0000","last_undegraded":"2024-04-25T08:58:52.489792+0000","last_fullsized":"2024-04-25T08:58:52.489792+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481843+0000","last_change":"2024-04-25T08:58:52.481843+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481843+0000","last_undegraded":"2024-04-25T08:58:52.481843+0000","last_fullsized":"2024-04-25T08:58:52.481843+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490305+0000","last_change":"2024-04-25T08:58:52.490305+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490305+0000","last_undegraded":"2024-04-25T08:58:52.490305+0000","last_fullsized":"2024-04-25T08:58:52.490305+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480491+0000","last_change":"2024-04-25T08:58:52.480491+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480491+0000","last_undegraded":"2024-04-25T08:58:52.480491+0000","last_fullsized":"2024-04-25T08:58:52.480491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481337+0000","last_change":"2024-04-25T08:58:52.481337+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481337+0000","last_undegraded":"2024-04-25T08:58:52.481337+0000","last_fullsized":"2024-04-25T08:58:52.481337+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.475900+0000","last_change":"2024-04-25T08:58:52.475900+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.475900+0000","last_undegraded":"2024-04-25T08:58:52.475900+0000","last_fullsized":"2024-04-25T08:58:52.475900+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487974+0000","last_change":"2024-04-25T08:58:52.487974+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487974+0000","last_undegraded":"2024-04-25T08:58:52.487974+0000","last_fullsized":"2024-04-25T08:58:52.487974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480582+0000","last_change":"2024-04-25T08:58:52.480582+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480582+0000","last_undegraded":"2024-04-25T08:58:52.480582+0000","last_fullsized":"2024-04-25T08:58:52.480582+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478227+0000","last_change":"2024-04-25T08:58:52.478227+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478227+0000","last_undegraded":"2024-04-25T08:58:52.478227+0000","last_fullsized":"2024-04-25T08:58:52.478227+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481255+0000","last_change":"2024-04-25T08:58:52.481255+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481255+0000","last_undegraded":"2024-04-25T08:58:52.481255+0000","last_fullsized":"2024-04-25T08:58:52.481255+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491670+0000","last_change":"2024-04-25T08:58:52.491670+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491670+0000","last_undegraded":"2024-04-25T08:58:52.491670+0000","last_fullsized":"2024-04-25T08:58:52.491670+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479529+0000","last_change":"2024-04-25T08:58:52.479529+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479529+0000","last_undegraded":"2024-04-25T08:58:52.479529+0000","last_fullsized":"2024-04-25T08:58:52.479529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485384+0000","last_change":"2024-04-25T08:58:52.485384+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485384+0000","last_undegraded":"2024-04-25T08:58:52.485384+0000","last_fullsized":"2024-04-25T08:58:52.485384+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490204+0000","last_change":"2024-04-25T08:58:52.490204+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490204+0000","last_undegraded":"2024-04-25T08:58:52.490204+0000","last_fullsized":"2024-04-25T08:58:52.490204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487585+0000","last_change":"2024-04-25T08:58:52.487585+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487585+0000","last_undegraded":"2024-04-25T08:58:52.487585+0000","last_fullsized":"2024-04-25T08:58:52.487585+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.509098+0000","last_change":"2024-04-25T08:58:52.499317+0000","last_active":"2024-04-25T08:58:52.509098+0000","last_peered":"2024-04-25T08:58:52.509098+0000","last_clean":"2024-04-25T08:58:52.509098+0000","last_became_active":"2024-04-25T08:58:52.498145+0000","last_became_peered":"2024-04-25T08:58:52.498145+0000","last_unstale":"2024-04-25T08:58:52.509098+0000","last_undegraded":"2024-04-25T08:58:52.509098+0000","last_fullsized":"2024-04-25T08:58:52.509098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:50.604902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487225+0000","last_change":"2024-04-25T08:58:52.487225+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487225+0000","last_undegraded":"2024-04-25T08:58:52.487225+0000","last_fullsized":"2024-04-25T08:58:52.487225+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518038+0000","last_change":"2024-04-25T08:58:52.513581+0000","last_active":"2024-04-25T08:58:52.518038+0000","last_peered":"2024-04-25T08:58:52.518038+0000","last_clean":"2024-04-25T08:58:52.518038+0000","last_became_active":"2024-04-25T08:58:52.512774+0000","last_became_peered":"2024-04-25T08:58:52.512774+0000","last_unstale":"2024-04-25T08:58:52.518038+0000","last_undegraded":"2024-04-25T08:58:52.518038+0000","last_fullsized":"2024-04-25T08:58:52.518038+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:45.689538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487757+0000","last_change":"2024-04-25T08:58:52.487757+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487757+0000","last_undegraded":"2024-04-25T08:58:52.487757+0000","last_fullsized":"2024-04-25T08:58:52.487757+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512573+0000","last_change":"2024-04-25T08:58:52.508417+0000","last_active":"2024-04-25T08:58:52.512573+0000","last_peered":"2024-04-25T08:58:52.512573+0000","last_clean":"2024-04-25T08:58:52.512573+0000","last_became_active":"2024-04-25T08:58:52.507854+0000","last_became_peered":"2024-04-25T08:58:52.507854+0000","last_unstale":"2024-04-25T08:58:52.512573+0000","last_undegraded":"2024-04-25T08:58:52.512573+0000","last_fullsized":"2024-04-25T08:58:52.512573+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:02.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491046+0000","last_change":"2024-04-25T08:58:52.491046+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491046+0000","last_undegraded":"2024-04-25T08:58:52.491046+0000","last_fullsized":"2024-04-25T08:58:52.491046+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508855+0000","last_change":"2024-04-25T08:58:52.494753+0000","last_active":"2024-04-25T08:58:52.508855+0000","last_peered":"2024-04-25T08:58:52.508855+0000","last_clean":"2024-04-25T08:58:52.508855+0000","last_became_active":"2024-04-25T08:58:52.493972+0000","last_became_peered":"2024-04-25T08:58:52.493972+0000","last_unstale":"2024-04-25T08:58:52.508855+0000","last_undegraded":"2024-04-25T08:58:52.508855+0000","last_fullsized":"2024-04-25T08:58:52.508855+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:14.712772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479776+0000","last_change":"2024-04-25T08:58:52.479776+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479776+0000","last_undegraded":"2024-04-25T08:58:52.479776+0000","last_fullsized":"2024-04-25T08:58:52.479776+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512397+0000","last_change":"2024-04-25T08:58:52.505294+0000","last_active":"2024-04-25T08:58:52.512397+0000","last_peered":"2024-04-25T08:58:52.512397+0000","last_clean":"2024-04-25T08:58:52.512397+0000","last_became_active":"2024-04-25T08:58:52.504512+0000","last_became_peered":"2024-04-25T08:58:52.504512+0000","last_unstale":"2024-04-25T08:58:52.512397+0000","last_undegraded":"2024-04-25T08:58:52.512397+0000","last_fullsized":"2024-04-25T08:58:52.512397+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:56.905836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478321+0000","last_change":"2024-04-25T08:58:52.478321+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478321+0000","last_undegraded":"2024-04-25T08:58:52.478321+0000","last_fullsized":"2024-04-25T08:58:52.478321+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507544+0000","last_change":"2024-04-25T08:58:52.503172+0000","last_active":"2024-04-25T08:58:52.507544+0000","last_peered":"2024-04-25T08:58:52.507544+0000","last_clean":"2024-04-25T08:58:52.507544+0000","last_became_active":"2024-04-25T08:58:52.502702+0000","last_became_peered":"2024-04-25T08:58:52.502702+0000","last_unstale":"2024-04-25T08:58:52.507544+0000","last_undegraded":"2024-04-25T08:58:52.507544+0000","last_fullsized":"2024-04-25T08:58:52.507544+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486416+0000","last_change":"2024-04-25T08:58:52.486416+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486416+0000","last_undegraded":"2024-04-25T08:58:52.486416+0000","last_fullsized":"2024-04-25T08:58:52.486416+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.502729+0000","last_change":"2024-04-25T08:58:52.501970+0000","last_active":"2024-04-25T08:58:52.502729+0000","last_peered":"2024-04-25T08:58:52.502729+0000","last_clean":"2024-04-25T08:58:52.502729+0000","last_became_active":"2024-04-25T08:58:52.500708+0000","last_became_peered":"2024-04-25T08:58:52.500708+0000","last_unstale":"2024-04-25T08:58:52.502729+0000","last_undegraded":"2024-04-25T08:58:52.502729+0000","last_fullsized":"2024-04-25T08:58:52.502729+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:25.488383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480907+0000","last_change":"2024-04-25T08:58:52.480907+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480907+0000","last_undegraded":"2024-04-25T08:58:52.480907+0000","last_fullsized":"2024-04-25T08:58:52.480907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512711+0000","last_change":"2024-04-25T08:58:52.506757+0000","last_active":"2024-04-25T08:58:52.512711+0000","last_peered":"2024-04-25T08:58:52.512711+0000","last_clean":"2024-04-25T08:58:52.512711+0000","last_became_active":"2024-04-25T08:58:52.506014+0000","last_became_peered":"2024-04-25T08:58:52.506014+0000","last_unstale":"2024-04-25T08:58:52.512711+0000","last_undegraded":"2024-04-25T08:58:52.512711+0000","last_fullsized":"2024-04-25T08:58:52.512711+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476026+0000","last_change":"2024-04-25T08:58:52.476026+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476026+0000","last_undegraded":"2024-04-25T08:58:52.476026+0000","last_fullsized":"2024-04-25T08:58:52.476026+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508755+0000","last_change":"2024-04-25T08:58:52.505203+0000","last_active":"2024-04-25T08:58:52.508755+0000","last_peered":"2024-04-25T08:58:52.508755+0000","last_clean":"2024-04-25T08:58:52.508755+0000","last_became_active":"2024-04-25T08:58:52.504503+0000","last_became_peered":"2024-04-25T08:58:52.504503+0000","last_unstale":"2024-04-25T08:58:52.508755+0000","last_undegraded":"2024-04-25T08:58:52.508755+0000","last_fullsized":"2024-04-25T08:58:52.508755+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:15.105995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487774+0000","last_change":"2024-04-25T08:58:52.487774+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487774+0000","last_undegraded":"2024-04-25T08:58:52.487774+0000","last_fullsized":"2024-04-25T08:58:52.487774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.495939+0000","last_change":"2024-04-25T08:58:52.493504+0000","last_active":"2024-04-25T08:58:52.495939+0000","last_peered":"2024-04-25T08:58:52.495939+0000","last_clean":"2024-04-25T08:58:52.495939+0000","last_became_active":"2024-04-25T08:58:52.493077+0000","last_became_peered":"2024-04-25T08:58:52.493077+0000","last_unstale":"2024-04-25T08:58:52.495939+0000","last_undegraded":"2024-04-25T08:58:52.495939+0000","last_fullsized":"2024-04-25T08:58:52.495939+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:12.438660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477801+0000","last_change":"2024-04-25T08:58:52.477801+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477801+0000","last_undegraded":"2024-04-25T08:58:52.477801+0000","last_fullsized":"2024-04-25T08:58:52.477801+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518279+0000","last_change":"2024-04-25T08:58:52.515324+0000","last_active":"2024-04-25T08:58:52.518279+0000","last_peered":"2024-04-25T08:58:52.518279+0000","last_clean":"2024-04-25T08:58:52.518279+0000","last_became_active":"2024-04-25T08:58:52.513958+0000","last_became_peered":"2024-04-25T08:58:52.513958+0000","last_unstale":"2024-04-25T08:58:52.518279+0000","last_undegraded":"2024-04-25T08:58:52.518279+0000","last_fullsized":"2024-04-25T08:58:52.518279+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:07.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477385+0000","last_change":"2024-04-25T08:58:52.477385+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477385+0000","last_undegraded":"2024-04-25T08:58:52.477385+0000","last_fullsized":"2024-04-25T08:58:52.477385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.496529+0000","last_change":"2024-04-25T08:58:52.493433+0000","last_active":"2024-04-25T08:58:52.496529+0000","last_peered":"2024-04-25T08:58:52.496529+0000","last_clean":"2024-04-25T08:58:52.496529+0000","last_became_active":"2024-04-25T08:58:52.493061+0000","last_became_peered":"2024-04-25T08:58:52.493061+0000","last_unstale":"2024-04-25T08:58:52.496529+0000","last_undegraded":"2024-04-25T08:58:52.496529+0000","last_fullsized":"2024-04-25T08:58:52.496529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:04.908717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479199+0000","last_change":"2024-04-25T08:58:52.479199+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479199+0000","last_undegraded":"2024-04-25T08:58:52.479199+0000","last_fullsized":"2024-04-25T08:58:52.479199+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512641+0000","last_change":"2024-04-25T08:58:52.507149+0000","last_active":"2024-04-25T08:58:52.512641+0000","last_peered":"2024-04-25T08:58:52.512641+0000","last_clean":"2024-04-25T08:58:52.512641+0000","last_became_active":"2024-04-25T08:58:52.506632+0000","last_became_peered":"2024-04-25T08:58:52.506632+0000","last_unstale":"2024-04-25T08:58:52.512641+0000","last_undegraded":"2024-04-25T08:58:52.512641+0000","last_fullsized":"2024-04-25T08:58:52.512641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:16.136540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476701+0000","last_change":"2024-04-25T08:58:52.476701+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476701+0000","last_undegraded":"2024-04-25T08:58:52.476701+0000","last_fullsized":"2024-04-25T08:58:52.476701+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513002+0000","last_change":"2024-04-25T08:58:52.508620+0000","last_active":"2024-04-25T08:58:52.513002+0000","last_peered":"2024-04-25T08:58:52.513002+0000","last_clean":"2024-04-25T08:58:52.513002+0000","last_became_active":"2024-04-25T08:58:52.508361+0000","last_became_peered":"2024-04-25T08:58:52.508361+0000","last_unstale":"2024-04-25T08:58:52.513002+0000","last_undegraded":"2024-04-25T08:58:52.513002+0000","last_fullsized":"2024-04-25T08:58:52.513002+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.319442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480028+0000","last_change":"2024-04-25T08:58:52.480028+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480028+0000","last_undegraded":"2024-04-25T08:58:52.480028+0000","last_fullsized":"2024-04-25T08:58:52.480028+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507782+0000","last_change":"2024-04-25T08:58:52.503640+0000","last_active":"2024-04-25T08:58:52.507782+0000","last_peered":"2024-04-25T08:58:52.507782+0000","last_clean":"2024-04-25T08:58:52.507782+0000","last_became_active":"2024-04-25T08:58:52.503271+0000","last_became_peered":"2024-04-25T08:58:52.503271+0000","last_unstale":"2024-04-25T08:58:52.507782+0000","last_undegraded":"2024-04-25T08:58:52.507782+0000","last_fullsized":"2024-04-25T08:58:52.507782+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:02.528462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477547+0000","last_change":"2024-04-25T08:58:52.477547+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477547+0000","last_undegraded":"2024-04-25T08:58:52.477547+0000","last_fullsized":"2024-04-25T08:58:52.477547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507718+0000","last_change":"2024-04-25T08:58:52.503187+0000","last_active":"2024-04-25T08:58:52.507718+0000","last_peered":"2024-04-25T08:58:52.507718+0000","last_clean":"2024-04-25T08:58:52.507718+0000","last_became_active":"2024-04-25T08:58:52.502662+0000","last_became_peered":"2024-04-25T08:58:52.502662+0000","last_unstale":"2024-04-25T08:58:52.507718+0000","last_undegraded":"2024-04-25T08:58:52.507718+0000","last_fullsized":"2024-04-25T08:58:52.507718+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:28.688051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485819+0000","last_change":"2024-04-25T08:58:52.485819+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485819+0000","last_undegraded":"2024-04-25T08:58:52.485819+0000","last_fullsized":"2024-04-25T08:58:52.485819+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513658+0000","last_change":"2024-04-25T08:58:52.502738+0000","last_active":"2024-04-25T08:58:52.513658+0000","last_peered":"2024-04-25T08:58:52.513658+0000","last_clean":"2024-04-25T08:58:52.513658+0000","last_became_active":"2024-04-25T08:58:52.501965+0000","last_became_peered":"2024-04-25T08:58:52.501965+0000","last_unstale":"2024-04-25T08:58:52.513658+0000","last_undegraded":"2024-04-25T08:58:52.513658+0000","last_fullsized":"2024-04-25T08:58:52.513658+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:30.608887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481261+0000","last_change":"2024-04-25T08:58:52.481261+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481261+0000","last_undegraded":"2024-04-25T08:58:52.481261+0000","last_fullsized":"2024-04-25T08:58:52.481261+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513182+0000","last_change":"2024-04-25T08:58:52.510654+0000","last_active":"2024-04-25T08:58:52.513182+0000","last_peered":"2024-04-25T08:58:52.513182+0000","last_clean":"2024-04-25T08:58:52.513182+0000","last_became_active":"2024-04-25T08:58:52.509735+0000","last_became_peered":"2024-04-25T08:58:52.509735+0000","last_unstale":"2024-04-25T08:58:52.513182+0000","last_undegraded":"2024-04-25T08:58:52.513182+0000","last_fullsized":"2024-04-25T08:58:52.513182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:23.084950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476876+0000","last_change":"2024-04-25T08:58:52.476876+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476876+0000","last_undegraded":"2024-04-25T08:58:52.476876+0000","last_fullsized":"2024-04-25T08:58:52.476876+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518225+0000","last_change":"2024-04-25T08:58:52.514656+0000","last_active":"2024-04-25T08:58:52.518225+0000","last_peered":"2024-04-25T08:58:52.518225+0000","last_clean":"2024-04-25T08:58:52.518225+0000","last_became_active":"2024-04-25T08:58:52.513150+0000","last_became_peered":"2024-04-25T08:58:52.513150+0000","last_unstale":"2024-04-25T08:58:52.518225+0000","last_undegraded":"2024-04-25T08:58:52.518225+0000","last_fullsized":"2024-04-25T08:58:52.518225+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:40.111643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476469+0000","last_change":"2024-04-25T08:58:52.476469+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476469+0000","last_undegraded":"2024-04-25T08:58:52.476469+0000","last_fullsized":"2024-04-25T08:58:52.476469+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503595+0000","last_change":"2024-04-25T08:58:52.494154+0000","last_active":"2024-04-25T08:58:52.503595+0000","last_peered":"2024-04-25T08:58:52.503595+0000","last_clean":"2024-04-25T08:58:52.503595+0000","last_became_active":"2024-04-25T08:58:52.493429+0000","last_became_peered":"2024-04-25T08:58:52.493429+0000","last_unstale":"2024-04-25T08:58:52.503595+0000","last_undegraded":"2024-04-25T08:58:52.503595+0000","last_fullsized":"2024-04-25T08:58:52.503595+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:30.608887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.483968+0000","last_change":"2024-04-25T08:58:52.483968+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.483968+0000","last_undegraded":"2024-04-25T08:58:52.483968+0000","last_fullsized":"2024-04-25T08:58:52.483968+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.501305+0000","last_change":"2024-04-25T08:58:52.494299+0000","last_active":"2024-04-25T08:58:52.501305+0000","last_peered":"2024-04-25T08:58:52.501305+0000","last_clean":"2024-04-25T08:58:52.501305+0000","last_became_active":"2024-04-25T08:58:52.493557+0000","last_became_peered":"2024-04-25T08:58:52.493557+0000","last_unstale":"2024-04-25T08:58:52.501305+0000","last_undegraded":"2024-04-25T08:58:52.501305+0000","last_fullsized":"2024-04-25T08:58:52.501305+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:41.335795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486514+0000","last_change":"2024-04-25T08:58:52.486514+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486514+0000","last_undegraded":"2024-04-25T08:58:52.486514+0000","last_fullsized":"2024-04-25T08:58:52.486514+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507821+0000","last_change":"2024-04-25T08:58:52.500643+0000","last_active":"2024-04-25T08:58:52.507821+0000","last_peered":"2024-04-25T08:58:52.507821+0000","last_clean":"2024-04-25T08:58:52.507821+0000","last_became_active":"2024-04-25T08:58:52.499551+0000","last_became_peered":"2024-04-25T08:58:52.499551+0000","last_unstale":"2024-04-25T08:58:52.507821+0000","last_undegraded":"2024-04-25T08:58:52.507821+0000","last_fullsized":"2024-04-25T08:58:52.507821+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:06.052032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487020+0000","last_change":"2024-04-25T08:58:52.487020+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487020+0000","last_undegraded":"2024-04-25T08:58:52.487020+0000","last_fullsized":"2024-04-25T08:58:52.487020+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503818+0000","last_change":"2024-04-25T08:58:52.501153+0000","last_active":"2024-04-25T08:58:52.503818+0000","last_peered":"2024-04-25T08:58:52.503818+0000","last_clean":"2024-04-25T08:58:52.503818+0000","last_became_active":"2024-04-25T08:58:52.497820+0000","last_became_peered":"2024-04-25T08:58:52.497820+0000","last_unstale":"2024-04-25T08:58:52.503818+0000","last_undegraded":"2024-04-25T08:58:52.503818+0000","last_fullsized":"2024-04-25T08:58:52.503818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:05.283812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481896+0000","last_change":"2024-04-25T08:58:52.481896+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481896+0000","last_undegraded":"2024-04-25T08:58:52.481896+0000","last_fullsized":"2024-04-25T08:58:52.481896+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.517562+0000","last_change":"2024-04-25T08:58:52.511956+0000","last_active":"2024-04-25T08:58:52.517562+0000","last_peered":"2024-04-25T08:58:52.517562+0000","last_clean":"2024-04-25T08:58:52.517562+0000","last_became_active":"2024-04-25T08:58:52.511196+0000","last_became_peered":"2024-04-25T08:58:52.511196+0000","last_unstale":"2024-04-25T08:58:52.517562+0000","last_undegraded":"2024-04-25T08:58:52.517562+0000","last_fullsized":"2024-04-25T08:58:52.517562+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:09.214520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476443+0000","last_change":"2024-04-25T08:58:52.476443+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476443+0000","last_undegraded":"2024-04-25T08:58:52.476443+0000","last_fullsized":"2024-04-25T08:58:52.476443+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.517853+0000","last_change":"2024-04-25T08:58:52.512155+0000","last_active":"2024-04-25T08:58:52.517853+0000","last_peered":"2024-04-25T08:58:52.517853+0000","last_clean":"2024-04-25T08:58:52.517853+0000","last_became_active":"2024-04-25T08:58:52.506602+0000","last_became_peered":"2024-04-25T08:58:52.506602+0000","last_unstale":"2024-04-25T08:58:52.517853+0000","last_undegraded":"2024-04-25T08:58:52.517853+0000","last_fullsized":"2024-04-25T08:58:52.517853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:09.340774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476535+0000","last_change":"2024-04-25T08:58:52.476535+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476535+0000","last_undegraded":"2024-04-25T08:58:52.476535+0000","last_fullsized":"2024-04-25T08:58:52.476535+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.515015+0000","last_change":"2024-04-25T08:58:52.508006+0000","last_active":"2024-04-25T08:58:52.515015+0000","last_peered":"2024-04-25T08:58:52.515015+0000","last_clean":"2024-04-25T08:58:52.515015+0000","last_became_active":"2024-04-25T08:58:52.507149+0000","last_became_peered":"2024-04-25T08:58:52.507149+0000","last_unstale":"2024-04-25T08:58:52.515015+0000","last_undegraded":"2024-04-25T08:58:52.515015+0000","last_fullsized":"2024-04-25T08:58:52.515015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:04.306110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486979+0000","last_change":"2024-04-25T08:58:52.486979+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486979+0000","last_undegraded":"2024-04-25T08:58:52.486979+0000","last_fullsized":"2024-04-25T08:58:52.486979+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508975+0000","last_change":"2024-04-25T08:58:52.495139+0000","last_active":"2024-04-25T08:58:52.508975+0000","last_peered":"2024-04-25T08:58:52.508975+0000","last_clean":"2024-04-25T08:58:52.508975+0000","last_became_active":"2024-04-25T08:58:52.494329+0000","last_became_peered":"2024-04-25T08:58:52.494329+0000","last_unstale":"2024-04-25T08:58:52.508975+0000","last_undegraded":"2024-04-25T08:58:52.508975+0000","last_fullsized":"2024-04-25T08:58:52.508975+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:54.968958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478270+0000","last_change":"2024-04-25T08:58:52.478270+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478270+0000","last_undegraded":"2024-04-25T08:58:52.478270+0000","last_fullsized":"2024-04-25T08:58:52.478270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.516378+0000","last_change":"2024-04-25T08:58:52.513900+0000","last_active":"2024-04-25T08:58:52.516378+0000","last_peered":"2024-04-25T08:58:52.516378+0000","last_clean":"2024-04-25T08:58:52.516378+0000","last_became_active":"2024-04-25T08:58:52.513643+0000","last_became_peered":"2024-04-25T08:58:52.513643+0000","last_unstale":"2024-04-25T08:58:52.516378+0000","last_undegraded":"2024-04-25T08:58:52.516378+0000","last_fullsized":"2024-04-25T08:58:52.516378+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:51.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476352+0000","last_change":"2024-04-25T08:58:52.476352+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476352+0000","last_undegraded":"2024-04-25T08:58:52.476352+0000","last_fullsized":"2024-04-25T08:58:52.476352+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503475+0000","last_change":"2024-04-25T08:58:52.493529+0000","last_active":"2024-04-25T08:58:52.503475+0000","last_peered":"2024-04-25T08:58:52.503475+0000","last_clean":"2024-04-25T08:58:52.503475+0000","last_became_active":"2024-04-25T08:58:52.492691+0000","last_became_peered":"2024-04-25T08:58:52.492691+0000","last_unstale":"2024-04-25T08:58:52.503475+0000","last_undegraded":"2024-04-25T08:58:52.503475+0000","last_fullsized":"2024-04-25T08:58:52.503475+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:50.604902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476218+0000","last_change":"2024-04-25T08:58:52.476218+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476218+0000","last_undegraded":"2024-04-25T08:58:52.476218+0000","last_fullsized":"2024-04-25T08:58:52.476218+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.509136+0000","last_change":"2024-04-25T08:58:52.495019+0000","last_active":"2024-04-25T08:58:52.509136+0000","last_peered":"2024-04-25T08:58:52.509136+0000","last_clean":"2024-04-25T08:58:52.509136+0000","last_became_active":"2024-04-25T08:58:52.494237+0000","last_became_peered":"2024-04-25T08:58:52.494237+0000","last_unstale":"2024-04-25T08:58:52.509136+0000","last_undegraded":"2024-04-25T08:58:52.509136+0000","last_fullsized":"2024-04-25T08:58:52.509136+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:35.222412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.475654+0000","last_change":"2024-04-25T08:58:52.475654+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.475654+0000","last_undegraded":"2024-04-25T08:58:52.475654+0000","last_fullsized":"2024-04-25T08:58:52.475654+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503768+0000","last_change":"2024-04-25T08:58:52.502403+0000","last_active":"2024-04-25T08:58:52.503768+0000","last_peered":"2024-04-25T08:58:52.503768+0000","last_clean":"2024-04-25T08:58:52.503768+0000","last_became_active":"2024-04-25T08:58:52.501501+0000","last_became_peered":"2024-04-25T08:58:52.501501+0000","last_unstale":"2024-04-25T08:58:52.503768+0000","last_undegraded":"2024-04-25T08:58:52.503768+0000","last_fullsized":"2024-04-25T08:58:52.503768+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:12.458893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490599+0000","last_change":"2024-04-25T08:58:52.490599+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490599+0000","last_undegraded":"2024-04-25T08:58:52.490599+0000","last_fullsized":"2024-04-25T08:58:52.490599+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.498266+0000","last_change":"2024-04-25T08:58:52.493037+0000","last_active":"2024-04-25T08:58:52.498266+0000","last_peered":"2024-04-25T08:58:52.498266+0000","last_clean":"2024-04-25T08:58:52.498266+0000","last_became_active":"2024-04-25T08:58:52.492644+0000","last_became_peered":"2024-04-25T08:58:52.492644+0000","last_unstale":"2024-04-25T08:58:52.498266+0000","last_undegraded":"2024-04-25T08:58:52.498266+0000","last_fullsized":"2024-04-25T08:58:52.498266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:07.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":307,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.497611+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:58:52.497611+0000","last_peered":"2024-04-25T08:58:52.497611+0000","last_clean":"2024-04-25T08:58:52.497611+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:58:52.497611+0000","last_undegraded":"2024-04-25T08:58:52.497611+0000","last_fullsized":"2024-04-25T08:58:52.497611+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1740800,"data_stored":1118358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1740800,"data_stored":1118358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:58:57.961 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:58:58.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.120+0000 7fb1cb3b0640 1 Processor -- start 2024-04-25T08:58:58.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.120+0000 7fb1cb3b0640 1 -- start start 2024-04-25T08:58:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1cb3b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1c4159ea0 0x7fb1c40b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1cb3b0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c415ae30 con 0x7fb1c415a280 2024-04-25T08:58:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1cb3b0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c415b110 con 0x7fb1c4071fd0 2024-04-25T08:58:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1cb3b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40b2990 con 0x7fb1c4159ea0 2024-04-25T08:58:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c894b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb1c415a280 0x7fb1c40718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37240/0 (socket says 172.21.15.37:37240) 2024-04-25T08:58:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c894b640 1 -- 172.21.15.37:0/1548812604 learned_addr learned my addr 172.21.15.37:0/1548812604 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c914c640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1c4159ea0 0x7fb1c40b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277023256 0 0) 0x7fb1c415ae30 con 0x7fb1c415a280 2024-04-25T08:58:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1b0003530 con 0x7fb1c415a280 2024-04-25T08:58:58.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 534904973 0 0) 0x7fb1b0003530 con 0x7fb1c415a280 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1c4159ea0 msgr2=0x7fb1c40b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1c4159ea0 0x7fb1c40b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 >> v1:172.21.15.203:6789/0 conn(0x7fb1c4071fd0 legacy=0x7fb1c406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c40b2620 con 0x7fb1c415a280 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fb1b4001680 con 0x7fb1c415a280 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 >> v1:172.21.15.37:6789/0 conn(0x7fb1c415a280 legacy=0x7fb1c40718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1c4159ea0 0x7fb1b0003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b00040b0 0x7fb1b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1b0004da0 0x7fb1b0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40b2990 con 0x7fb1b00040b0 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b000bc10 con 0x7fb1b0004da0 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b000be10 con 0x7fb1c4159ea0 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c894b640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b00040b0 0x7fb1b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c914c640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1c4159ea0 0x7fb1b0003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c994d640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1b0004da0 0x7fb1b0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c894b640 1 -- 172.21.15.37:0/1548812604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1c4159ea0 msgr2=0x7fb1b0003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:58.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c894b640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1c4159ea0 0x7fb1b0003a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c894b640 1 -- 172.21.15.37:0/1548812604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1b0004da0 msgr2=0x7fb1b0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:58.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c894b640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1b0004da0 0x7fb1b0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c894b640 1 -- 172.21.15.37:0/1548812604 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b0003530 con 0x7fb1b00040b0 2024-04-25T08:58:58.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1c894b640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b00040b0 0x7fb1b0004810 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c1c0 tx=0x7fb1b401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0793beeecbc241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4007d00 con 0x7fb1b00040b0 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b4005b00 con 0x7fb1b00040b0 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.124+0000 7fb1bbfff640 1 -- 172.21.15.37:0/1548812604 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4007070 con 0x7fb1b00040b0 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/1548812604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b00040b0 msgr2=0x7fb1b0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 --2- 172.21.15.37:0/1548812604 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1b00040b0 0x7fb1b0004810 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c1c0 tx=0x7fb1b401f3d0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1c994d640 1 -- 172.21.15.37:0/1548812604 reap_dead start 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/1548812604 shutdown_connections 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/1548812604 >> 172.21.15.37:0/1548812604 conn(0x7fb1c40b11a0 msgr2=0x7fb1c40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/1548812604 shutdown_connections 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/1548812604 wait complete. 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 Processor -- start 2024-04-25T08:58:58.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 -- start start 2024-04-25T08:58:58.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1b00040b0 0x7fb1c40c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c414dfa0 0x7fb1c4151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1c40c5b40 0x7fb1c40c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c406d920 con 0x7fb1c414dfa0 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c406d7a0 con 0x7fb1c40c5b40 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c406daa0 con 0x7fb1b00040b0 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1c894b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c414dfa0 0x7fb1c4151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1c894b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c414dfa0 0x7fb1c4151050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56754/0 (socket says 172.21.15.37:56754) 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1c894b640 1 -- 172.21.15.37:0/2954746709 learned_addr learned my addr 172.21.15.37:0/2954746709 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1c914c640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1b00040b0 0x7fb1c40c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1c894b640 1 -- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1b00040b0 msgr2=0x7fb1c40c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1c894b640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1b00040b0 0x7fb1c40c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1c894b640 1 -- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1c40c5b40 msgr2=0x7fb1c40c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1c894b640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1c40c5b40 0x7fb1c40c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1c894b640 1 -- 172.21.15.37:0/2954746709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c4151590 con 0x7fb1c414dfa0 2024-04-25T08:58:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1c894b640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c414dfa0 0x7fb1c4151050 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fb1b401f200 tx=0x7fb1b0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1b9ffb640 1 -- 172.21.15.37:0/2954746709 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4007d00 con 0x7fb1c414dfa0 2024-04-25T08:58:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1b9ffb640 1 -- 172.21.15.37:0/2954746709 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b4000ee0 con 0x7fb1c414dfa0 2024-04-25T08:58:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/2954746709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c414e420 con 0x7fb1c414dfa0 2024-04-25T08:58:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1b9ffb640 1 -- 172.21.15.37:0/2954746709 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4001c40 con 0x7fb1c414dfa0 2024-04-25T08:58:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.128+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/2954746709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c40c4d80 con 0x7fb1c414dfa0 2024-04-25T08:58:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.132+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/2954746709 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c404efa0 con 0x7fb1c414dfa0 2024-04-25T08:58:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.132+0000 7fb1b9ffb640 1 -- 172.21.15.37:0/2954746709 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb1b4001480 con 0x7fb1c414dfa0 2024-04-25T08:58:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.132+0000 7fb1b9ffb640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1a8066f90 0x7fb1a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.132+0000 7fb1b9ffb640 1 -- 172.21.15.37:0/2954746709 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb1b4122430 con 0x7fb1c414dfa0 2024-04-25T08:58:58.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.132+0000 7fb1c914c640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1a8066f90 0x7fb1a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.132+0000 7fb1c914c640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1a8066f90 0x7fb1a8069450 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000cb00 tx=0x7fb1c000c4e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:58.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.136+0000 7fb1b9ffb640 1 -- 172.21.15.37:0/2954746709 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb1b40190e0 con 0x7fb1c414dfa0 2024-04-25T08:58:58.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.292+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/2954746709 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1c40cedd0 con 0x7fb1a8066f90 2024-04-25T08:58:58.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.308+0000 7fb1b9ffb640 1 -- 172.21.15.37:0/2954746709 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246773 (secure 0 0 0) 0x7fb1c40cedd0 con 0x7fb1a8066f90 2024-04-25T08:58:58.302 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:58.303 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:58:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1a8066f90 msgr2=0x7fb1a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1a8066f90 0x7fb1a8069450 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000cb00 tx=0x7fb1c000c4e0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/2954746709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c414dfa0 msgr2=0x7fb1c4151050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c414dfa0 0x7fb1c4151050 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fb1b401f200 tx=0x7fb1b0005840 comp rx=0 tx=0).stop 2024-04-25T08:58:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/2954746709 shutdown_connections 2024-04-25T08:58:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb1a8066f90 0x7fb1a8069450 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb1c40c5b40 0x7fb1c40c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb1c414dfa0 0x7fb1c4151050 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 --2- 172.21.15.37:0/2954746709 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb1b00040b0 0x7fb1c40c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/2954746709 >> 172.21.15.37:0/2954746709 conn(0x7fb1c40b11a0 msgr2=0x7fb1c40cf030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/2954746709 shutdown_connections 2024-04-25T08:58:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.312+0000 7fb1cb3b0640 1 -- 172.21.15.37:0/2954746709 wait complete. 2024-04-25T08:58:58.320 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-04-25T08:58:57.256685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259576,"kb_used_data":10120,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715144,"statfs":{"total":773094113280,"available":772828307456,"internally_reserved":0,"allocated":10362880,"data_stored":5404656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.791611"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476372+0000","last_change":"2024-04-25T08:58:52.476372+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476372+0000","last_undegraded":"2024-04-25T08:58:52.476372+0000","last_fullsized":"2024-04-25T08:58:52.476372+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485159+0000","last_change":"2024-04-25T08:58:52.485159+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485159+0000","last_undegraded":"2024-04-25T08:58:52.485159+0000","last_fullsized":"2024-04-25T08:58:52.485159+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476534+0000","last_change":"2024-04-25T08:58:52.476534+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476534+0000","last_undegraded":"2024-04-25T08:58:52.476534+0000","last_fullsized":"2024-04-25T08:58:52.476534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.482393+0000","last_change":"2024-04-25T08:58:52.482393+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.482393+0000","last_undegraded":"2024-04-25T08:58:52.482393+0000","last_fullsized":"2024-04-25T08:58:52.482393+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489217+0000","last_change":"2024-04-25T08:58:52.489217+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489217+0000","last_undegraded":"2024-04-25T08:58:52.489217+0000","last_fullsized":"2024-04-25T08:58:52.489217+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490958+0000","last_change":"2024-04-25T08:58:52.490958+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490958+0000","last_undegraded":"2024-04-25T08:58:52.490958+0000","last_fullsized":"2024-04-25T08:58:52.490958+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.495788+0000","last_change":"2024-04-25T08:58:52.495788+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.495788+0000","last_undegraded":"2024-04-25T08:58:52.495788+0000","last_fullsized":"2024-04-25T08:58:52.495788+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480355+0000","last_change":"2024-04-25T08:58:52.480355+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480355+0000","last_undegraded":"2024-04-25T08:58:52.480355+0000","last_fullsized":"2024-04-25T08:58:52.480355+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479616+0000","last_change":"2024-04-25T08:58:52.479616+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479616+0000","last_undegraded":"2024-04-25T08:58:52.479616+0000","last_fullsized":"2024-04-25T08:58:52.479616+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479974+0000","last_change":"2024-04-25T08:58:52.479974+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479974+0000","last_undegraded":"2024-04-25T08:58:52.479974+0000","last_fullsized":"2024-04-25T08:58:52.479974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491682+0000","last_change":"2024-04-25T08:58:52.491682+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491682+0000","last_undegraded":"2024-04-25T08:58:52.491682+0000","last_fullsized":"2024-04-25T08:58:52.491682+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479047+0000","last_change":"2024-04-25T08:58:52.479047+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479047+0000","last_undegraded":"2024-04-25T08:58:52.479047+0000","last_fullsized":"2024-04-25T08:58:52.479047+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.482025+0000","last_change":"2024-04-25T08:58:52.482025+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.482025+0000","last_undegraded":"2024-04-25T08:58:52.482025+0000","last_fullsized":"2024-04-25T08:58:52.482025+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489553+0000","last_change":"2024-04-25T08:58:52.489553+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489553+0000","last_undegraded":"2024-04-25T08:58:52.489553+0000","last_fullsized":"2024-04-25T08:58:52.489553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478028+0000","last_change":"2024-04-25T08:58:52.478028+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478028+0000","last_undegraded":"2024-04-25T08:58:52.478028+0000","last_fullsized":"2024-04-25T08:58:52.478028+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490790+0000","last_change":"2024-04-25T08:58:52.490790+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490790+0000","last_undegraded":"2024-04-25T08:58:52.490790+0000","last_fullsized":"2024-04-25T08:58:52.490790+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491623+0000","last_change":"2024-04-25T08:58:52.491623+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491623+0000","last_undegraded":"2024-04-25T08:58:52.491623+0000","last_fullsized":"2024-04-25T08:58:52.491623+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478387+0000","last_change":"2024-04-25T08:58:52.478387+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478387+0000","last_undegraded":"2024-04-25T08:58:52.478387+0000","last_fullsized":"2024-04-25T08:58:52.478387+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489792+0000","last_change":"2024-04-25T08:58:52.489792+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489792+0000","last_undegraded":"2024-04-25T08:58:52.489792+0000","last_fullsized":"2024-04-25T08:58:52.489792+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481843+0000","last_change":"2024-04-25T08:58:52.481843+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481843+0000","last_undegraded":"2024-04-25T08:58:52.481843+0000","last_fullsized":"2024-04-25T08:58:52.481843+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490305+0000","last_change":"2024-04-25T08:58:52.490305+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490305+0000","last_undegraded":"2024-04-25T08:58:52.490305+0000","last_fullsized":"2024-04-25T08:58:52.490305+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480491+0000","last_change":"2024-04-25T08:58:52.480491+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480491+0000","last_undegraded":"2024-04-25T08:58:52.480491+0000","last_fullsized":"2024-04-25T08:58:52.480491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481337+0000","last_change":"2024-04-25T08:58:52.481337+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481337+0000","last_undegraded":"2024-04-25T08:58:52.481337+0000","last_fullsized":"2024-04-25T08:58:52.481337+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.475900+0000","last_change":"2024-04-25T08:58:52.475900+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.475900+0000","last_undegraded":"2024-04-25T08:58:52.475900+0000","last_fullsized":"2024-04-25T08:58:52.475900+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487974+0000","last_change":"2024-04-25T08:58:52.487974+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487974+0000","last_undegraded":"2024-04-25T08:58:52.487974+0000","last_fullsized":"2024-04-25T08:58:52.487974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480582+0000","last_change":"2024-04-25T08:58:52.480582+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480582+0000","last_undegraded":"2024-04-25T08:58:52.480582+0000","last_fullsized":"2024-04-25T08:58:52.480582+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478227+0000","last_change":"2024-04-25T08:58:52.478227+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478227+0000","last_undegraded":"2024-04-25T08:58:52.478227+0000","last_fullsized":"2024-04-25T08:58:52.478227+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481255+0000","last_change":"2024-04-25T08:58:52.481255+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481255+0000","last_undegraded":"2024-04-25T08:58:52.481255+0000","last_fullsized":"2024-04-25T08:58:52.481255+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491670+0000","last_change":"2024-04-25T08:58:52.491670+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491670+0000","last_undegraded":"2024-04-25T08:58:52.491670+0000","last_fullsized":"2024-04-25T08:58:52.491670+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479529+0000","last_change":"2024-04-25T08:58:52.479529+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479529+0000","last_undegraded":"2024-04-25T08:58:52.479529+0000","last_fullsized":"2024-04-25T08:58:52.479529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485384+0000","last_change":"2024-04-25T08:58:52.485384+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485384+0000","last_undegraded":"2024-04-25T08:58:52.485384+0000","last_fullsized":"2024-04-25T08:58:52.485384+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490204+0000","last_change":"2024-04-25T08:58:52.490204+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490204+0000","last_undegraded":"2024-04-25T08:58:52.490204+0000","last_fullsized":"2024-04-25T08:58:52.490204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487585+0000","last_change":"2024-04-25T08:58:52.487585+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487585+0000","last_undegraded":"2024-04-25T08:58:52.487585+0000","last_fullsized":"2024-04-25T08:58:52.487585+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.509098+0000","last_change":"2024-04-25T08:58:52.499317+0000","last_active":"2024-04-25T08:58:52.509098+0000","last_peered":"2024-04-25T08:58:52.509098+0000","last_clean":"2024-04-25T08:58:52.509098+0000","last_became_active":"2024-04-25T08:58:52.498145+0000","last_became_peered":"2024-04-25T08:58:52.498145+0000","last_unstale":"2024-04-25T08:58:52.509098+0000","last_undegraded":"2024-04-25T08:58:52.509098+0000","last_fullsized":"2024-04-25T08:58:52.509098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:50.604902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487225+0000","last_change":"2024-04-25T08:58:52.487225+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487225+0000","last_undegraded":"2024-04-25T08:58:52.487225+0000","last_fullsized":"2024-04-25T08:58:52.487225+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518038+0000","last_change":"2024-04-25T08:58:52.513581+0000","last_active":"2024-04-25T08:58:52.518038+0000","last_peered":"2024-04-25T08:58:52.518038+0000","last_clean":"2024-04-25T08:58:52.518038+0000","last_became_active":"2024-04-25T08:58:52.512774+0000","last_became_peered":"2024-04-25T08:58:52.512774+0000","last_unstale":"2024-04-25T08:58:52.518038+0000","last_undegraded":"2024-04-25T08:58:52.518038+0000","last_fullsized":"2024-04-25T08:58:52.518038+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:45.689538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487757+0000","last_change":"2024-04-25T08:58:52.487757+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487757+0000","last_undegraded":"2024-04-25T08:58:52.487757+0000","last_fullsized":"2024-04-25T08:58:52.487757+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512573+0000","last_change":"2024-04-25T08:58:52.508417+0000","last_active":"2024-04-25T08:58:52.512573+0000","last_peered":"2024-04-25T08:58:52.512573+0000","last_clean":"2024-04-25T08:58:52.512573+0000","last_became_active":"2024-04-25T08:58:52.507854+0000","last_became_peered":"2024-04-25T08:58:52.507854+0000","last_unstale":"2024-04-25T08:58:52.512573+0000","last_undegraded":"2024-04-25T08:58:52.512573+0000","last_fullsized":"2024-04-25T08:58:52.512573+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:02.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491046+0000","last_change":"2024-04-25T08:58:52.491046+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491046+0000","last_undegraded":"2024-04-25T08:58:52.491046+0000","last_fullsized":"2024-04-25T08:58:52.491046+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508855+0000","last_change":"2024-04-25T08:58:52.494753+0000","last_active":"2024-04-25T08:58:52.508855+0000","last_peered":"2024-04-25T08:58:52.508855+0000","last_clean":"2024-04-25T08:58:52.508855+0000","last_became_active":"2024-04-25T08:58:52.493972+0000","last_became_peered":"2024-04-25T08:58:52.493972+0000","last_unstale":"2024-04-25T08:58:52.508855+0000","last_undegraded":"2024-04-25T08:58:52.508855+0000","last_fullsized":"2024-04-25T08:58:52.508855+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:14.712772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479776+0000","last_change":"2024-04-25T08:58:52.479776+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479776+0000","last_undegraded":"2024-04-25T08:58:52.479776+0000","last_fullsized":"2024-04-25T08:58:52.479776+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512397+0000","last_change":"2024-04-25T08:58:52.505294+0000","last_active":"2024-04-25T08:58:52.512397+0000","last_peered":"2024-04-25T08:58:52.512397+0000","last_clean":"2024-04-25T08:58:52.512397+0000","last_became_active":"2024-04-25T08:58:52.504512+0000","last_became_peered":"2024-04-25T08:58:52.504512+0000","last_unstale":"2024-04-25T08:58:52.512397+0000","last_undegraded":"2024-04-25T08:58:52.512397+0000","last_fullsized":"2024-04-25T08:58:52.512397+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:56.905836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478321+0000","last_change":"2024-04-25T08:58:52.478321+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478321+0000","last_undegraded":"2024-04-25T08:58:52.478321+0000","last_fullsized":"2024-04-25T08:58:52.478321+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507544+0000","last_change":"2024-04-25T08:58:52.503172+0000","last_active":"2024-04-25T08:58:52.507544+0000","last_peered":"2024-04-25T08:58:52.507544+0000","last_clean":"2024-04-25T08:58:52.507544+0000","last_became_active":"2024-04-25T08:58:52.502702+0000","last_became_peered":"2024-04-25T08:58:52.502702+0000","last_unstale":"2024-04-25T08:58:52.507544+0000","last_undegraded":"2024-04-25T08:58:52.507544+0000","last_fullsized":"2024-04-25T08:58:52.507544+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486416+0000","last_change":"2024-04-25T08:58:52.486416+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486416+0000","last_undegraded":"2024-04-25T08:58:52.486416+0000","last_fullsized":"2024-04-25T08:58:52.486416+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.502729+0000","last_change":"2024-04-25T08:58:52.501970+0000","last_active":"2024-04-25T08:58:52.502729+0000","last_peered":"2024-04-25T08:58:52.502729+0000","last_clean":"2024-04-25T08:58:52.502729+0000","last_became_active":"2024-04-25T08:58:52.500708+0000","last_became_peered":"2024-04-25T08:58:52.500708+0000","last_unstale":"2024-04-25T08:58:52.502729+0000","last_undegraded":"2024-04-25T08:58:52.502729+0000","last_fullsized":"2024-04-25T08:58:52.502729+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:25.488383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480907+0000","last_change":"2024-04-25T08:58:52.480907+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480907+0000","last_undegraded":"2024-04-25T08:58:52.480907+0000","last_fullsized":"2024-04-25T08:58:52.480907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512711+0000","last_change":"2024-04-25T08:58:52.506757+0000","last_active":"2024-04-25T08:58:52.512711+0000","last_peered":"2024-04-25T08:58:52.512711+0000","last_clean":"2024-04-25T08:58:52.512711+0000","last_became_active":"2024-04-25T08:58:52.506014+0000","last_became_peered":"2024-04-25T08:58:52.506014+0000","last_unstale":"2024-04-25T08:58:52.512711+0000","last_undegraded":"2024-04-25T08:58:52.512711+0000","last_fullsized":"2024-04-25T08:58:52.512711+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476026+0000","last_change":"2024-04-25T08:58:52.476026+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476026+0000","last_undegraded":"2024-04-25T08:58:52.476026+0000","last_fullsized":"2024-04-25T08:58:52.476026+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508755+0000","last_change":"2024-04-25T08:58:52.505203+0000","last_active":"2024-04-25T08:58:52.508755+0000","last_peered":"2024-04-25T08:58:52.508755+0000","last_clean":"2024-04-25T08:58:52.508755+0000","last_became_active":"2024-04-25T08:58:52.504503+0000","last_became_peered":"2024-04-25T08:58:52.504503+0000","last_unstale":"2024-04-25T08:58:52.508755+0000","last_undegraded":"2024-04-25T08:58:52.508755+0000","last_fullsized":"2024-04-25T08:58:52.508755+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:15.105995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487774+0000","last_change":"2024-04-25T08:58:52.487774+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487774+0000","last_undegraded":"2024-04-25T08:58:52.487774+0000","last_fullsized":"2024-04-25T08:58:52.487774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.495939+0000","last_change":"2024-04-25T08:58:52.493504+0000","last_active":"2024-04-25T08:58:52.495939+0000","last_peered":"2024-04-25T08:58:52.495939+0000","last_clean":"2024-04-25T08:58:52.495939+0000","last_became_active":"2024-04-25T08:58:52.493077+0000","last_became_peered":"2024-04-25T08:58:52.493077+0000","last_unstale":"2024-04-25T08:58:52.495939+0000","last_undegraded":"2024-04-25T08:58:52.495939+0000","last_fullsized":"2024-04-25T08:58:52.495939+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:12.438660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477801+0000","last_change":"2024-04-25T08:58:52.477801+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477801+0000","last_undegraded":"2024-04-25T08:58:52.477801+0000","last_fullsized":"2024-04-25T08:58:52.477801+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518279+0000","last_change":"2024-04-25T08:58:52.515324+0000","last_active":"2024-04-25T08:58:52.518279+0000","last_peered":"2024-04-25T08:58:52.518279+0000","last_clean":"2024-04-25T08:58:52.518279+0000","last_became_active":"2024-04-25T08:58:52.513958+0000","last_became_peered":"2024-04-25T08:58:52.513958+0000","last_unstale":"2024-04-25T08:58:52.518279+0000","last_undegraded":"2024-04-25T08:58:52.518279+0000","last_fullsized":"2024-04-25T08:58:52.518279+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:07.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477385+0000","last_change":"2024-04-25T08:58:52.477385+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477385+0000","last_undegraded":"2024-04-25T08:58:52.477385+0000","last_fullsized":"2024-04-25T08:58:52.477385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.496529+0000","last_change":"2024-04-25T08:58:52.493433+0000","last_active":"2024-04-25T08:58:52.496529+0000","last_peered":"2024-04-25T08:58:52.496529+0000","last_clean":"2024-04-25T08:58:52.496529+0000","last_became_active":"2024-04-25T08:58:52.493061+0000","last_became_peered":"2024-04-25T08:58:52.493061+0000","last_unstale":"2024-04-25T08:58:52.496529+0000","last_undegraded":"2024-04-25T08:58:52.496529+0000","last_fullsized":"2024-04-25T08:58:52.496529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:04.908717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479199+0000","last_change":"2024-04-25T08:58:52.479199+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479199+0000","last_undegraded":"2024-04-25T08:58:52.479199+0000","last_fullsized":"2024-04-25T08:58:52.479199+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512641+0000","last_change":"2024-04-25T08:58:52.507149+0000","last_active":"2024-04-25T08:58:52.512641+0000","last_peered":"2024-04-25T08:58:52.512641+0000","last_clean":"2024-04-25T08:58:52.512641+0000","last_became_active":"2024-04-25T08:58:52.506632+0000","last_became_peered":"2024-04-25T08:58:52.506632+0000","last_unstale":"2024-04-25T08:58:52.512641+0000","last_undegraded":"2024-04-25T08:58:52.512641+0000","last_fullsized":"2024-04-25T08:58:52.512641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:16.136540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476701+0000","last_change":"2024-04-25T08:58:52.476701+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476701+0000","last_undegraded":"2024-04-25T08:58:52.476701+0000","last_fullsized":"2024-04-25T08:58:52.476701+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513002+0000","last_change":"2024-04-25T08:58:52.508620+0000","last_active":"2024-04-25T08:58:52.513002+0000","last_peered":"2024-04-25T08:58:52.513002+0000","last_clean":"2024-04-25T08:58:52.513002+0000","last_became_active":"2024-04-25T08:58:52.508361+0000","last_became_peered":"2024-04-25T08:58:52.508361+0000","last_unstale":"2024-04-25T08:58:52.513002+0000","last_undegraded":"2024-04-25T08:58:52.513002+0000","last_fullsized":"2024-04-25T08:58:52.513002+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.319442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480028+0000","last_change":"2024-04-25T08:58:52.480028+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480028+0000","last_undegraded":"2024-04-25T08:58:52.480028+0000","last_fullsized":"2024-04-25T08:58:52.480028+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507782+0000","last_change":"2024-04-25T08:58:52.503640+0000","last_active":"2024-04-25T08:58:52.507782+0000","last_peered":"2024-04-25T08:58:52.507782+0000","last_clean":"2024-04-25T08:58:52.507782+0000","last_became_active":"2024-04-25T08:58:52.503271+0000","last_became_peered":"2024-04-25T08:58:52.503271+0000","last_unstale":"2024-04-25T08:58:52.507782+0000","last_undegraded":"2024-04-25T08:58:52.507782+0000","last_fullsized":"2024-04-25T08:58:52.507782+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:02.528462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477547+0000","last_change":"2024-04-25T08:58:52.477547+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477547+0000","last_undegraded":"2024-04-25T08:58:52.477547+0000","last_fullsized":"2024-04-25T08:58:52.477547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507718+0000","last_change":"2024-04-25T08:58:52.503187+0000","last_active":"2024-04-25T08:58:52.507718+0000","last_peered":"2024-04-25T08:58:52.507718+0000","last_clean":"2024-04-25T08:58:52.507718+0000","last_became_active":"2024-04-25T08:58:52.502662+0000","last_became_peered":"2024-04-25T08:58:52.502662+0000","last_unstale":"2024-04-25T08:58:52.507718+0000","last_undegraded":"2024-04-25T08:58:52.507718+0000","last_fullsized":"2024-04-25T08:58:52.507718+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:28.688051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485819+0000","last_change":"2024-04-25T08:58:52.485819+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485819+0000","last_undegraded":"2024-04-25T08:58:52.485819+0000","last_fullsized":"2024-04-25T08:58:52.485819+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513658+0000","last_change":"2024-04-25T08:58:52.502738+0000","last_active":"2024-04-25T08:58:52.513658+0000","last_peered":"2024-04-25T08:58:52.513658+0000","last_clean":"2024-04-25T08:58:52.513658+0000","last_became_active":"2024-04-25T08:58:52.501965+0000","last_became_peered":"2024-04-25T08:58:52.501965+0000","last_unstale":"2024-04-25T08:58:52.513658+0000","last_undegraded":"2024-04-25T08:58:52.513658+0000","last_fullsized":"2024-04-25T08:58:52.513658+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:30.608887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481261+0000","last_change":"2024-04-25T08:58:52.481261+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481261+0000","last_undegraded":"2024-04-25T08:58:52.481261+0000","last_fullsized":"2024-04-25T08:58:52.481261+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513182+0000","last_change":"2024-04-25T08:58:52.510654+0000","last_active":"2024-04-25T08:58:52.513182+0000","last_peered":"2024-04-25T08:58:52.513182+0000","last_clean":"2024-04-25T08:58:52.513182+0000","last_became_active":"2024-04-25T08:58:52.509735+0000","last_became_peered":"2024-04-25T08:58:52.509735+0000","last_unstale":"2024-04-25T08:58:52.513182+0000","last_undegraded":"2024-04-25T08:58:52.513182+0000","last_fullsized":"2024-04-25T08:58:52.513182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:23.084950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476876+0000","last_change":"2024-04-25T08:58:52.476876+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476876+0000","last_undegraded":"2024-04-25T08:58:52.476876+0000","last_fullsized":"2024-04-25T08:58:52.476876+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518225+0000","last_change":"2024-04-25T08:58:52.514656+0000","last_active":"2024-04-25T08:58:52.518225+0000","last_peered":"2024-04-25T08:58:52.518225+0000","last_clean":"2024-04-25T08:58:52.518225+0000","last_became_active":"2024-04-25T08:58:52.513150+0000","last_became_peered":"2024-04-25T08:58:52.513150+0000","last_unstale":"2024-04-25T08:58:52.518225+0000","last_undegraded":"2024-04-25T08:58:52.518225+0000","last_fullsized":"2024-04-25T08:58:52.518225+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:40.111643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476469+0000","last_change":"2024-04-25T08:58:52.476469+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476469+0000","last_undegraded":"2024-04-25T08:58:52.476469+0000","last_fullsized":"2024-04-25T08:58:52.476469+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503595+0000","last_change":"2024-04-25T08:58:52.494154+0000","last_active":"2024-04-25T08:58:52.503595+0000","last_peered":"2024-04-25T08:58:52.503595+0000","last_clean":"2024-04-25T08:58:52.503595+0000","last_became_active":"2024-04-25T08:58:52.493429+0000","last_became_peered":"2024-04-25T08:58:52.493429+0000","last_unstale":"2024-04-25T08:58:52.503595+0000","last_undegraded":"2024-04-25T08:58:52.503595+0000","last_fullsized":"2024-04-25T08:58:52.503595+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:30.608887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.483968+0000","last_change":"2024-04-25T08:58:52.483968+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.483968+0000","last_undegraded":"2024-04-25T08:58:52.483968+0000","last_fullsized":"2024-04-25T08:58:52.483968+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.501305+0000","last_change":"2024-04-25T08:58:52.494299+0000","last_active":"2024-04-25T08:58:52.501305+0000","last_peered":"2024-04-25T08:58:52.501305+0000","last_clean":"2024-04-25T08:58:52.501305+0000","last_became_active":"2024-04-25T08:58:52.493557+0000","last_became_peered":"2024-04-25T08:58:52.493557+0000","last_unstale":"2024-04-25T08:58:52.501305+0000","last_undegraded":"2024-04-25T08:58:52.501305+0000","last_fullsized":"2024-04-25T08:58:52.501305+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:41.335795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486514+0000","last_change":"2024-04-25T08:58:52.486514+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486514+0000","last_undegraded":"2024-04-25T08:58:52.486514+0000","last_fullsized":"2024-04-25T08:58:52.486514+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507821+0000","last_change":"2024-04-25T08:58:52.500643+0000","last_active":"2024-04-25T08:58:52.507821+0000","last_peered":"2024-04-25T08:58:52.507821+0000","last_clean":"2024-04-25T08:58:52.507821+0000","last_became_active":"2024-04-25T08:58:52.499551+0000","last_became_peered":"2024-04-25T08:58:52.499551+0000","last_unstale":"2024-04-25T08:58:52.507821+0000","last_undegraded":"2024-04-25T08:58:52.507821+0000","last_fullsized":"2024-04-25T08:58:52.507821+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:06.052032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487020+0000","last_change":"2024-04-25T08:58:52.487020+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487020+0000","last_undegraded":"2024-04-25T08:58:52.487020+0000","last_fullsized":"2024-04-25T08:58:52.487020+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503818+0000","last_change":"2024-04-25T08:58:52.501153+0000","last_active":"2024-04-25T08:58:52.503818+0000","last_peered":"2024-04-25T08:58:52.503818+0000","last_clean":"2024-04-25T08:58:52.503818+0000","last_became_active":"2024-04-25T08:58:52.497820+0000","last_became_peered":"2024-04-25T08:58:52.497820+0000","last_unstale":"2024-04-25T08:58:52.503818+0000","last_undegraded":"2024-04-25T08:58:52.503818+0000","last_fullsized":"2024-04-25T08:58:52.503818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:05.283812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481896+0000","last_change":"2024-04-25T08:58:52.481896+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481896+0000","last_undegraded":"2024-04-25T08:58:52.481896+0000","last_fullsized":"2024-04-25T08:58:52.481896+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.517562+0000","last_change":"2024-04-25T08:58:52.511956+0000","last_active":"2024-04-25T08:58:52.517562+0000","last_peered":"2024-04-25T08:58:52.517562+0000","last_clean":"2024-04-25T08:58:52.517562+0000","last_became_active":"2024-04-25T08:58:52.511196+0000","last_became_peered":"2024-04-25T08:58:52.511196+0000","last_unstale":"2024-04-25T08:58:52.517562+0000","last_undegraded":"2024-04-25T08:58:52.517562+0000","last_fullsized":"2024-04-25T08:58:52.517562+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:09.214520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476443+0000","last_change":"2024-04-25T08:58:52.476443+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476443+0000","last_undegraded":"2024-04-25T08:58:52.476443+0000","last_fullsized":"2024-04-25T08:58:52.476443+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.517853+0000","last_change":"2024-04-25T08:58:52.512155+0000","last_active":"2024-04-25T08:58:52.517853+0000","last_peered":"2024-04-25T08:58:52.517853+0000","last_clean":"2024-04-25T08:58:52.517853+0000","last_became_active":"2024-04-25T08:58:52.506602+0000","last_became_peered":"2024-04-25T08:58:52.506602+0000","last_unstale":"2024-04-25T08:58:52.517853+0000","last_undegraded":"2024-04-25T08:58:52.517853+0000","last_fullsized":"2024-04-25T08:58:52.517853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:09.340774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476535+0000","last_change":"2024-04-25T08:58:52.476535+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476535+0000","last_undegraded":"2024-04-25T08:58:52.476535+0000","last_fullsized":"2024-04-25T08:58:52.476535+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.515015+0000","last_change":"2024-04-25T08:58:52.508006+0000","last_active":"2024-04-25T08:58:52.515015+0000","last_peered":"2024-04-25T08:58:52.515015+0000","last_clean":"2024-04-25T08:58:52.515015+0000","last_became_active":"2024-04-25T08:58:52.507149+0000","last_became_peered":"2024-04-25T08:58:52.507149+0000","last_unstale":"2024-04-25T08:58:52.515015+0000","last_undegraded":"2024-04-25T08:58:52.515015+0000","last_fullsized":"2024-04-25T08:58:52.515015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:04.306110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486979+0000","last_change":"2024-04-25T08:58:52.486979+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486979+0000","last_undegraded":"2024-04-25T08:58:52.486979+0000","last_fullsized":"2024-04-25T08:58:52.486979+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508975+0000","last_change":"2024-04-25T08:58:52.495139+0000","last_active":"2024-04-25T08:58:52.508975+0000","last_peered":"2024-04-25T08:58:52.508975+0000","last_clean":"2024-04-25T08:58:52.508975+0000","last_became_active":"2024-04-25T08:58:52.494329+0000","last_became_peered":"2024-04-25T08:58:52.494329+0000","last_unstale":"2024-04-25T08:58:52.508975+0000","last_undegraded":"2024-04-25T08:58:52.508975+0000","last_fullsized":"2024-04-25T08:58:52.508975+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:54.968958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478270+0000","last_change":"2024-04-25T08:58:52.478270+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478270+0000","last_undegraded":"2024-04-25T08:58:52.478270+0000","last_fullsized":"2024-04-25T08:58:52.478270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.516378+0000","last_change":"2024-04-25T08:58:52.513900+0000","last_active":"2024-04-25T08:58:52.516378+0000","last_peered":"2024-04-25T08:58:52.516378+0000","last_clean":"2024-04-25T08:58:52.516378+0000","last_became_active":"2024-04-25T08:58:52.513643+0000","last_became_peered":"2024-04-25T08:58:52.513643+0000","last_unstale":"2024-04-25T08:58:52.516378+0000","last_undegraded":"2024-04-25T08:58:52.516378+0000","last_fullsized":"2024-04-25T08:58:52.516378+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:51.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476352+0000","last_change":"2024-04-25T08:58:52.476352+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476352+0000","last_undegraded":"2024-04-25T08:58:52.476352+0000","last_fullsized":"2024-04-25T08:58:52.476352+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503475+0000","last_change":"2024-04-25T08:58:52.493529+0000","last_active":"2024-04-25T08:58:52.503475+0000","last_peered":"2024-04-25T08:58:52.503475+0000","last_clean":"2024-04-25T08:58:52.503475+0000","last_became_active":"2024-04-25T08:58:52.492691+0000","last_became_peered":"2024-04-25T08:58:52.492691+0000","last_unstale":"2024-04-25T08:58:52.503475+0000","last_undegraded":"2024-04-25T08:58:52.503475+0000","last_fullsized":"2024-04-25T08:58:52.503475+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:50.604902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476218+0000","last_change":"2024-04-25T08:58:52.476218+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476218+0000","last_undegraded":"2024-04-25T08:58:52.476218+0000","last_fullsized":"2024-04-25T08:58:52.476218+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.509136+0000","last_change":"2024-04-25T08:58:52.495019+0000","last_active":"2024-04-25T08:58:52.509136+0000","last_peered":"2024-04-25T08:58:52.509136+0000","last_clean":"2024-04-25T08:58:52.509136+0000","last_became_active":"2024-04-25T08:58:52.494237+0000","last_became_peered":"2024-04-25T08:58:52.494237+0000","last_unstale":"2024-04-25T08:58:52.509136+0000","last_undegraded":"2024-04-25T08:58:52.509136+0000","last_fullsized":"2024-04-25T08:58:52.509136+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:35.222412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.475654+0000","last_change":"2024-04-25T08:58:52.475654+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.475654+0000","last_undegraded":"2024-04-25T08:58:52.475654+0000","last_fullsized":"2024-04-25T08:58:52.475654+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503768+0000","last_change":"2024-04-25T08:58:52.502403+0000","last_active":"2024-04-25T08:58:52.503768+0000","last_peered":"2024-04-25T08:58:52.503768+0000","last_clean":"2024-04-25T08:58:52.503768+0000","last_became_active":"2024-04-25T08:58:52.501501+0000","last_became_peered":"2024-04-25T08:58:52.501501+0000","last_unstale":"2024-04-25T08:58:52.503768+0000","last_undegraded":"2024-04-25T08:58:52.503768+0000","last_fullsized":"2024-04-25T08:58:52.503768+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:12.458893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490599+0000","last_change":"2024-04-25T08:58:52.490599+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490599+0000","last_undegraded":"2024-04-25T08:58:52.490599+0000","last_fullsized":"2024-04-25T08:58:52.490599+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.498266+0000","last_change":"2024-04-25T08:58:52.493037+0000","last_active":"2024-04-25T08:58:52.498266+0000","last_peered":"2024-04-25T08:58:52.498266+0000","last_clean":"2024-04-25T08:58:52.498266+0000","last_became_active":"2024-04-25T08:58:52.492644+0000","last_became_peered":"2024-04-25T08:58:52.492644+0000","last_unstale":"2024-04-25T08:58:52.498266+0000","last_undegraded":"2024-04-25T08:58:52.498266+0000","last_fullsized":"2024-04-25T08:58:52.498266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:07.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":307,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.497611+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:58:52.497611+0000","last_peered":"2024-04-25T08:58:52.497611+0000","last_clean":"2024-04-25T08:58:52.497611+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:58:52.497611+0000","last_undegraded":"2024-04-25T08:58:52.497611+0000","last_fullsized":"2024-04-25T08:58:52.497611+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1740800,"data_stored":1118358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1740800,"data_stored":1118358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:58:58.323 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3f is not active+clean 2024-04-25T08:58:58.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.476372+0000', 'last_change': '2024-04-25T08:58:52.476372+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.476372+0000', 'last_undegraded': '2024-04-25T08:58:52.476372+0000', 'last_fullsized': '2024-04-25T08:58:52.476372+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:58:58.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3e is not active+clean 2024-04-25T08:58:58.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.485159+0000', 'last_change': '2024-04-25T08:58:52.485159+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.485159+0000', 'last_undegraded': '2024-04-25T08:58:52.485159+0000', 'last_fullsized': '2024-04-25T08:58:52.485159+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:58:58.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3d is not active+clean 2024-04-25T08:58:58.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.476534+0000', 'last_change': '2024-04-25T08:58:52.476534+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.476534+0000', 'last_undegraded': '2024-04-25T08:58:52.476534+0000', 'last_fullsized': '2024-04-25T08:58:52.476534+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:58:58.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3c is not active+clean 2024-04-25T08:58:58.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.482393+0000', 'last_change': '2024-04-25T08:58:52.482393+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.482393+0000', 'last_undegraded': '2024-04-25T08:58:52.482393+0000', 'last_fullsized': '2024-04-25T08:58:52.482393+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:58:58.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3b is not active+clean 2024-04-25T08:58:58.324 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.489217+0000', 'last_change': '2024-04-25T08:58:52.489217+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.489217+0000', 'last_undegraded': '2024-04-25T08:58:52.489217+0000', 'last_fullsized': '2024-04-25T08:58:52.489217+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.324 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3a is not active+clean 2024-04-25T08:58:58.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.490958+0000', 'last_change': '2024-04-25T08:58:52.490958+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.490958+0000', 'last_undegraded': '2024-04-25T08:58:52.490958+0000', 'last_fullsized': '2024-04-25T08:58:52.490958+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.39 is not active+clean 2024-04-25T08:58:58.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.495788+0000', 'last_change': '2024-04-25T08:58:52.495788+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.495788+0000', 'last_undegraded': '2024-04-25T08:58:52.495788+0000', 'last_fullsized': '2024-04-25T08:58:52.495788+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:58:58.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.38 is not active+clean 2024-04-25T08:58:58.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.480355+0000', 'last_change': '2024-04-25T08:58:52.480355+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.480355+0000', 'last_undegraded': '2024-04-25T08:58:52.480355+0000', 'last_fullsized': '2024-04-25T08:58:52.480355+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:58:58.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.37 is not active+clean 2024-04-25T08:58:58.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.479616+0000', 'last_change': '2024-04-25T08:58:52.479616+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.479616+0000', 'last_undegraded': '2024-04-25T08:58:52.479616+0000', 'last_fullsized': '2024-04-25T08:58:52.479616+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.325 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.36 is not active+clean 2024-04-25T08:58:58.325 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.479974+0000', 'last_change': '2024-04-25T08:58:52.479974+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.479974+0000', 'last_undegraded': '2024-04-25T08:58:52.479974+0000', 'last_fullsized': '2024-04-25T08:58:52.479974+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.35 is not active+clean 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.491682+0000', 'last_change': '2024-04-25T08:58:52.491682+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.491682+0000', 'last_undegraded': '2024-04-25T08:58:52.491682+0000', 'last_fullsized': '2024-04-25T08:58:52.491682+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.34 is not active+clean 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.479047+0000', 'last_change': '2024-04-25T08:58:52.479047+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.479047+0000', 'last_undegraded': '2024-04-25T08:58:52.479047+0000', 'last_fullsized': '2024-04-25T08:58:52.479047+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.33 is not active+clean 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.482025+0000', 'last_change': '2024-04-25T08:58:52.482025+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.482025+0000', 'last_undegraded': '2024-04-25T08:58:52.482025+0000', 'last_fullsized': '2024-04-25T08:58:52.482025+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.32 is not active+clean 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.489553+0000', 'last_change': '2024-04-25T08:58:52.489553+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.489553+0000', 'last_undegraded': '2024-04-25T08:58:52.489553+0000', 'last_fullsized': '2024-04-25T08:58:52.489553+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.31 is not active+clean 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.478028+0000', 'last_change': '2024-04-25T08:58:52.478028+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.478028+0000', 'last_undegraded': '2024-04-25T08:58:52.478028+0000', 'last_fullsized': '2024-04-25T08:58:52.478028+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.30 is not active+clean 2024-04-25T08:58:58.326 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.490790+0000', 'last_change': '2024-04-25T08:58:52.490790+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.490790+0000', 'last_undegraded': '2024-04-25T08:58:52.490790+0000', 'last_fullsized': '2024-04-25T08:58:52.490790+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2f is not active+clean 2024-04-25T08:58:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.491623+0000', 'last_change': '2024-04-25T08:58:52.491623+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.491623+0000', 'last_undegraded': '2024-04-25T08:58:52.491623+0000', 'last_fullsized': '2024-04-25T08:58:52.491623+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2e is not active+clean 2024-04-25T08:58:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.478387+0000', 'last_change': '2024-04-25T08:58:52.478387+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.478387+0000', 'last_undegraded': '2024-04-25T08:58:52.478387+0000', 'last_fullsized': '2024-04-25T08:58:52.478387+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:58:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2d is not active+clean 2024-04-25T08:58:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.489792+0000', 'last_change': '2024-04-25T08:58:52.489792+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.489792+0000', 'last_undegraded': '2024-04-25T08:58:52.489792+0000', 'last_fullsized': '2024-04-25T08:58:52.489792+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2c is not active+clean 2024-04-25T08:58:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.481843+0000', 'last_change': '2024-04-25T08:58:52.481843+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.481843+0000', 'last_undegraded': '2024-04-25T08:58:52.481843+0000', 'last_fullsized': '2024-04-25T08:58:52.481843+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:58:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2b is not active+clean 2024-04-25T08:58:58.327 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.490305+0000', 'last_change': '2024-04-25T08:58:52.490305+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.490305+0000', 'last_undegraded': '2024-04-25T08:58:52.490305+0000', 'last_fullsized': '2024-04-25T08:58:52.490305+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2a is not active+clean 2024-04-25T08:58:58.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.480491+0000', 'last_change': '2024-04-25T08:58:52.480491+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.480491+0000', 'last_undegraded': '2024-04-25T08:58:52.480491+0000', 'last_fullsized': '2024-04-25T08:58:52.480491+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.29 is not active+clean 2024-04-25T08:58:58.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.481337+0000', 'last_change': '2024-04-25T08:58:52.481337+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.481337+0000', 'last_undegraded': '2024-04-25T08:58:52.481337+0000', 'last_fullsized': '2024-04-25T08:58:52.481337+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:58:58.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.28 is not active+clean 2024-04-25T08:58:58.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.475900+0000', 'last_change': '2024-04-25T08:58:52.475900+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.475900+0000', 'last_undegraded': '2024-04-25T08:58:52.475900+0000', 'last_fullsized': '2024-04-25T08:58:52.475900+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:58:58.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.27 is not active+clean 2024-04-25T08:58:58.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.487974+0000', 'last_change': '2024-04-25T08:58:52.487974+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.487974+0000', 'last_undegraded': '2024-04-25T08:58:52.487974+0000', 'last_fullsized': '2024-04-25T08:58:52.487974+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:58:58.328 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.26 is not active+clean 2024-04-25T08:58:58.328 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.480582+0000', 'last_change': '2024-04-25T08:58:52.480582+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.480582+0000', 'last_undegraded': '2024-04-25T08:58:52.480582+0000', 'last_fullsized': '2024-04-25T08:58:52.480582+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.25 is not active+clean 2024-04-25T08:58:58.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.478227+0000', 'last_change': '2024-04-25T08:58:52.478227+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.478227+0000', 'last_undegraded': '2024-04-25T08:58:52.478227+0000', 'last_fullsized': '2024-04-25T08:58:52.478227+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:58:58.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.24 is not active+clean 2024-04-25T08:58:58.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.481255+0000', 'last_change': '2024-04-25T08:58:52.481255+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.481255+0000', 'last_undegraded': '2024-04-25T08:58:52.481255+0000', 'last_fullsized': '2024-04-25T08:58:52.481255+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:58:58.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.23 is not active+clean 2024-04-25T08:58:58.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.491670+0000', 'last_change': '2024-04-25T08:58:52.491670+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.491670+0000', 'last_undegraded': '2024-04-25T08:58:52.491670+0000', 'last_fullsized': '2024-04-25T08:58:52.491670+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:58:58.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.22 is not active+clean 2024-04-25T08:58:58.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.479529+0000', 'last_change': '2024-04-25T08:58:52.479529+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.479529+0000', 'last_undegraded': '2024-04-25T08:58:52.479529+0000', 'last_fullsized': '2024-04-25T08:58:52.479529+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.329 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.21 is not active+clean 2024-04-25T08:58:58.329 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.485384+0000', 'last_change': '2024-04-25T08:58:52.485384+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.485384+0000', 'last_undegraded': '2024-04-25T08:58:52.485384+0000', 'last_fullsized': '2024-04-25T08:58:52.485384+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.20 is not active+clean 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.490204+0000', 'last_change': '2024-04-25T08:58:52.490204+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.490204+0000', 'last_undegraded': '2024-04-25T08:58:52.490204+0000', 'last_fullsized': '2024-04-25T08:58:52.490204+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.487585+0000', 'last_change': '2024-04-25T08:58:52.487585+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.487585+0000', 'last_undegraded': '2024-04-25T08:58:52.487585+0000', 'last_fullsized': '2024-04-25T08:58:52.487585+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.487225+0000', 'last_change': '2024-04-25T08:58:52.487225+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.487225+0000', 'last_undegraded': '2024-04-25T08:58:52.487225+0000', 'last_fullsized': '2024-04-25T08:58:52.487225+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.487757+0000', 'last_change': '2024-04-25T08:58:52.487757+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.487757+0000', 'last_undegraded': '2024-04-25T08:58:52.487757+0000', 'last_fullsized': '2024-04-25T08:58:52.487757+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.491046+0000', 'last_change': '2024-04-25T08:58:52.491046+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.491046+0000', 'last_undegraded': '2024-04-25T08:58:52.491046+0000', 'last_fullsized': '2024-04-25T08:58:52.491046+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-04-25T08:58:58.330 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.479776+0000', 'last_change': '2024-04-25T08:58:52.479776+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.479776+0000', 'last_undegraded': '2024-04-25T08:58:52.479776+0000', 'last_fullsized': '2024-04-25T08:58:52.479776+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:58:58.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-25T08:58:58.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.478321+0000', 'last_change': '2024-04-25T08:58:52.478321+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.478321+0000', 'last_undegraded': '2024-04-25T08:58:52.478321+0000', 'last_fullsized': '2024-04-25T08:58:52.478321+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:58:58.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-04-25T08:58:58.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.486416+0000', 'last_change': '2024-04-25T08:58:52.486416+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.486416+0000', 'last_undegraded': '2024-04-25T08:58:52.486416+0000', 'last_fullsized': '2024-04-25T08:58:52.486416+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-04-25T08:58:58.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.480907+0000', 'last_change': '2024-04-25T08:58:52.480907+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.480907+0000', 'last_undegraded': '2024-04-25T08:58:52.480907+0000', 'last_fullsized': '2024-04-25T08:58:52.480907+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:58:58.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-04-25T08:58:58.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.476026+0000', 'last_change': '2024-04-25T08:58:52.476026+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.476026+0000', 'last_undegraded': '2024-04-25T08:58:52.476026+0000', 'last_fullsized': '2024-04-25T08:58:52.476026+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.331 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-04-25T08:58:58.331 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.487774+0000', 'last_change': '2024-04-25T08:58:52.487774+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.487774+0000', 'last_undegraded': '2024-04-25T08:58:52.487774+0000', 'last_fullsized': '2024-04-25T08:58:52.487774+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:58:58.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-04-25T08:58:58.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.477801+0000', 'last_change': '2024-04-25T08:58:52.477801+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.477801+0000', 'last_undegraded': '2024-04-25T08:58:52.477801+0000', 'last_fullsized': '2024-04-25T08:58:52.477801+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:58:58.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-04-25T08:58:58.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.477385+0000', 'last_change': '2024-04-25T08:58:52.477385+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.477385+0000', 'last_undegraded': '2024-04-25T08:58:52.477385+0000', 'last_fullsized': '2024-04-25T08:58:52.477385+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:58:58.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-04-25T08:58:58.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.479199+0000', 'last_change': '2024-04-25T08:58:52.479199+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.479199+0000', 'last_undegraded': '2024-04-25T08:58:52.479199+0000', 'last_fullsized': '2024-04-25T08:58:52.479199+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:58:58.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-04-25T08:58:58.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.476701+0000', 'last_change': '2024-04-25T08:58:52.476701+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.476701+0000', 'last_undegraded': '2024-04-25T08:58:52.476701+0000', 'last_fullsized': '2024-04-25T08:58:52.476701+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-04-25T08:58:58.332 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.480028+0000', 'last_change': '2024-04-25T08:58:52.480028+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.480028+0000', 'last_undegraded': '2024-04-25T08:58:52.480028+0000', 'last_fullsized': '2024-04-25T08:58:52.480028+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:58:58.332 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-04-25T08:58:58.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.477547+0000', 'last_change': '2024-04-25T08:58:52.477547+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.477547+0000', 'last_undegraded': '2024-04-25T08:58:52.477547+0000', 'last_fullsized': '2024-04-25T08:58:52.477547+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:58:58.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-04-25T08:58:58.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.485819+0000', 'last_change': '2024-04-25T08:58:52.485819+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.485819+0000', 'last_undegraded': '2024-04-25T08:58:52.485819+0000', 'last_fullsized': '2024-04-25T08:58:52.485819+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-25T08:58:58.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.481261+0000', 'last_change': '2024-04-25T08:58:52.481261+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.481261+0000', 'last_undegraded': '2024-04-25T08:58:52.481261+0000', 'last_fullsized': '2024-04-25T08:58:52.481261+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:58:58.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-04-25T08:58:58.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.476876+0000', 'last_change': '2024-04-25T08:58:52.476876+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.476876+0000', 'last_undegraded': '2024-04-25T08:58:52.476876+0000', 'last_fullsized': '2024-04-25T08:58:52.476876+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:58:58.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-04-25T08:58:58.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.476469+0000', 'last_change': '2024-04-25T08:58:52.476469+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.476469+0000', 'last_undegraded': '2024-04-25T08:58:52.476469+0000', 'last_fullsized': '2024-04-25T08:58:52.476469+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.333 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-04-25T08:58:58.333 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.483968+0000', 'last_change': '2024-04-25T08:58:52.483968+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.483968+0000', 'last_undegraded': '2024-04-25T08:58:52.483968+0000', 'last_fullsized': '2024-04-25T08:58:52.483968+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.486514+0000', 'last_change': '2024-04-25T08:58:52.486514+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.486514+0000', 'last_undegraded': '2024-04-25T08:58:52.486514+0000', 'last_fullsized': '2024-04-25T08:58:52.486514+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.487020+0000', 'last_change': '2024-04-25T08:58:52.487020+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.487020+0000', 'last_undegraded': '2024-04-25T08:58:52.487020+0000', 'last_fullsized': '2024-04-25T08:58:52.487020+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.481896+0000', 'last_change': '2024-04-25T08:58:52.481896+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.481896+0000', 'last_undegraded': '2024-04-25T08:58:52.481896+0000', 'last_fullsized': '2024-04-25T08:58:52.481896+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.476443+0000', 'last_change': '2024-04-25T08:58:52.476443+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.476443+0000', 'last_undegraded': '2024-04-25T08:58:52.476443+0000', 'last_fullsized': '2024-04-25T08:58:52.476443+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.476535+0000', 'last_change': '2024-04-25T08:58:52.476535+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.476535+0000', 'last_undegraded': '2024-04-25T08:58:52.476535+0000', 'last_fullsized': '2024-04-25T08:58:52.476535+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-04-25T08:58:58.334 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.486979+0000', 'last_change': '2024-04-25T08:58:52.486979+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.486979+0000', 'last_undegraded': '2024-04-25T08:58:52.486979+0000', 'last_fullsized': '2024-04-25T08:58:52.486979+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:58:58.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-04-25T08:58:58.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.478270+0000', 'last_change': '2024-04-25T08:58:52.478270+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.478270+0000', 'last_undegraded': '2024-04-25T08:58:52.478270+0000', 'last_fullsized': '2024-04-25T08:58:52.478270+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T08:58:58.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-04-25T08:58:58.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.476352+0000', 'last_change': '2024-04-25T08:58:52.476352+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.476352+0000', 'last_undegraded': '2024-04-25T08:58:52.476352+0000', 'last_fullsized': '2024-04-25T08:58:52.476352+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:58:58.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-25T08:58:58.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.476218+0000', 'last_change': '2024-04-25T08:58:52.476218+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.476218+0000', 'last_undegraded': '2024-04-25T08:58:52.476218+0000', 'last_fullsized': '2024-04-25T08:58:52.476218+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:58:58.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-04-25T08:58:58.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.475654+0000', 'last_change': '2024-04-25T08:58:52.475654+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.475654+0000', 'last_undegraded': '2024-04-25T08:58:52.475654+0000', 'last_fullsized': '2024-04-25T08:58:52.475654+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:58:58.335 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-04-25T08:58:58.335 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:58:52.490599+0000', 'last_change': '2024-04-25T08:58:52.490599+0000', 'last_active': '2024-04-25T08:58:52.462142+0000', 'last_peered': '2024-04-25T08:58:52.462142+0000', 'last_clean': '2024-04-25T08:58:52.462142+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:58:52.490599+0000', 'last_undegraded': '2024-04-25T08:58:52.490599+0000', 'last_fullsized': '2024-04-25T08:58:52.490599+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'last_clean_scrub_stamp': '2024-04-25T08:58:52.462142+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:58:58.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:58:58.444 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T08:58:58.452+0000 7fa6eda26640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:58:58.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2103e68640 1 Processor -- start 2024-04-25T08:58:58.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2103e68640 1 -- start start 2024-04-25T08:58:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2103e68640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f20fc0b7130 0x7f20fc06bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2103e68640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f20fc06c4f0 0x7f20fc0cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2103e68640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc0cbd10 con 0x7f20fc0b6d50 2024-04-25T08:58:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2103e68640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20fc09ae60 con 0x7f20fc06c4f0 2024-04-25T08:58:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2103e68640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20fc0cc600 con 0x7f20fc0b7130 2024-04-25T08:58:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2102405640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f20fc06c4f0 0x7f20fc0cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2102405640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f20fc06c4f0 0x7f20fc0cf690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56768/0 (socket says 172.21.15.37:56768) 2024-04-25T08:58:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2102405640 1 -- 172.21.15.37:0/1263339893 learned_addr learned my addr 172.21.15.37:0/1263339893 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2101403640 1 --2- 172.21.15.37:0/1263339893 >> v2:172.21.15.203:3300/0 conn(0x7f20fc0b7130 0x7f20fc06bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2102405640 1 -- 172.21.15.37:0/1263339893 >> v2:172.21.15.203:3300/0 conn(0x7f20fc0b7130 msgr2=0x7f20fc06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2102405640 1 --2- 172.21.15.37:0/1263339893 >> v2:172.21.15.203:3300/0 conn(0x7f20fc0b7130 0x7f20fc06bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2102405640 1 -- 172.21.15.37:0/1263339893 >> v1:172.21.15.203:6789/0 conn(0x7f20fc0b6d50 legacy=0x7f20fc09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.500+0000 7f2102405640 1 -- 172.21.15.37:0/1263339893 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc09ab90 con 0x7f20fc06c4f0 2024-04-25T08:58:58.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2102405640 1 --2- 172.21.15.37:0/1263339893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc06c4f0 0x7f20fc0cf690 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009e50 tx=0x7f20f401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5ee0b4a06cb3cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:58.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2100c02640 1 -- 172.21.15.37:0/1263339893 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f40a5670 con 0x7f20fc06c4f0 2024-04-25T08:58:58.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2100c02640 1 -- 172.21.15.37:0/1263339893 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f40a5810 con 0x7f20fc06c4f0 2024-04-25T08:58:58.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2100c02640 1 -- 172.21.15.37:0/1263339893 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f40a5be0 con 0x7f20fc06c4f0 2024-04-25T08:58:58.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2103e68640 1 -- 172.21.15.37:0/1263339893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc06c4f0 msgr2=0x7f20fc0cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2103e68640 1 --2- 172.21.15.37:0/1263339893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc06c4f0 0x7f20fc0cf690 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009e50 tx=0x7f20f401c9b0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2103e68640 1 -- 172.21.15.37:0/1263339893 shutdown_connections 2024-04-25T08:58:58.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2103e68640 1 --2- 172.21.15.37:0/1263339893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc06c4f0 0x7f20fc0cf690 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2103e68640 1 --2- 172.21.15.37:0/1263339893 >> v2:172.21.15.203:3300/0 conn(0x7f20fc0b7130 0x7f20fc06bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2103e68640 1 -- 172.21.15.37:0/1263339893 >> 172.21.15.37:0/1263339893 conn(0x7f20fc0b3010 msgr2=0x7f20fc0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2103e68640 1 -- 172.21.15.37:0/1263339893 shutdown_connections 2024-04-25T08:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2103e68640 1 -- 172.21.15.37:0/1263339893 wait complete. 2024-04-25T08:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2103e68640 1 Processor -- start 2024-04-25T08:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.504+0000 7f2103e68640 1 -- start start 2024-04-25T08:58:58.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2103e68640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20fc06c4f0 0x7f20fc14e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2103e68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc0b6d50 0x7f20fc0c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2103e68640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20fc0b7130 0x7f20fc0c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2103e68640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0b4370 con 0x7f20fc0b6d50 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2103e68640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc06ce40 con 0x7f20fc06c4f0 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2103e68640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0b44f0 con 0x7f20fc0b7130 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2101403640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc0b6d50 0x7f20fc0c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2101403640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc0b6d50 0x7f20fc0c6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56772/0 (socket says 172.21.15.37:56772) 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2101403640 1 -- 172.21.15.37:0/1534778434 learned_addr learned my addr 172.21.15.37:0/1534778434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2101c04640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20fc06c4f0 0x7f20fc14e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2102405640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20fc0b7130 0x7f20fc0c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2101403640 1 -- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20fc0b7130 msgr2=0x7f20fc0c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2101403640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20fc0b7130 0x7f20fc0c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2101403640 1 -- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20fc06c4f0 msgr2=0x7f20fc14e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2101403640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20fc06c4f0 0x7f20fc14e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2101403640 1 -- 172.21.15.37:0/1534778434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc14f040 con 0x7f20fc0b6d50 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2101403640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc0b6d50 0x7f20fc0c6ca0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00d800 tx=0x7f20ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:58.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f20f27fc640 1 -- 172.21.15.37:0/1534778434 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00b6c0 con 0x7f20fc0b6d50 2024-04-25T08:58:58.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f20f27fc640 1 -- 172.21.15.37:0/1534778434 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ec019070 con 0x7f20fc0b6d50 2024-04-25T08:58:58.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2103e68640 1 -- 172.21.15.37:0/1534778434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc14f320 con 0x7f20fc0b6d50 2024-04-25T08:58:58.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f20f27fc640 1 -- 172.21.15.37:0/1534778434 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec00ba60 con 0x7f20fc0b6d50 2024-04-25T08:58:58.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2103e68640 1 -- 172.21.15.37:0/1534778434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc14f8e0 con 0x7f20fc0b6d50 2024-04-25T08:58:58.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2103e68640 1 -- 172.21.15.37:0/1534778434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20fc04efa0 con 0x7f20fc0b6d50 2024-04-25T08:58:58.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f20f27fc640 1 -- 172.21.15.37:0/1534778434 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f20ec00bc00 con 0x7f20fc0b6d50 2024-04-25T08:58:58.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f20f27fc640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20e0066f60 0x7f20e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f20f27fc640 1 -- 172.21.15.37:0/1534778434 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20ec085620 con 0x7f20fc0b6d50 2024-04-25T08:58:58.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.508+0000 7f2101c04640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20e0066f60 0x7f20e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.512+0000 7f2101c04640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20e0066f60 0x7f20e0069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f20f8004e30 tx=0x7f20f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:58.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.512+0000 7f20f27fc640 1 -- 172.21.15.37:0/1534778434 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f20ec0564a0 con 0x7f20fc0b6d50 2024-04-25T08:58:58.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.732+0000 7f2103e68640 1 -- 172.21.15.37:0/1534778434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f20fc06ee30 con 0x7f20fc0b6d50 2024-04-25T08:58:58.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.732+0000 7f20f27fc640 1 -- 172.21.15.37:0/1534778434 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1035 (secure 0 0 0) 0x7f20ec055da0 con 0x7f20fc0b6d50 2024-04-25T08:58:58.726 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:58.726 INFO:teuthology.orchestra.run.smithi037.stdout:{"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":412,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":95,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035137,"num_in_osds":8,"osd_in_since":1714035128,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":265805824,"bytes_avail":772828307456,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":60,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7734}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T08:58:41.251754+0000","services":{}},"progress_events":{}} 2024-04-25T08:58:58.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 -- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20e0066f60 msgr2=0x7f20e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20e0066f60 0x7f20e0069420 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f20f8004e30 tx=0x7f20f8009040 comp rx=0 tx=0).stop 2024-04-25T08:58:58.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 -- 172.21.15.37:0/1534778434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc0b6d50 msgr2=0x7f20fc0c6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc0b6d50 0x7f20fc0c6ca0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f20ec00d800 tx=0x7f20ec00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 -- 172.21.15.37:0/1534778434 shutdown_connections 2024-04-25T08:58:58.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f20e0066f60 0x7f20e0069420 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f20fc0b7130 0x7f20fc0c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20fc0b6d50 0x7f20fc0c6ca0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 --2- 172.21.15.37:0/1534778434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f20fc06c4f0 0x7f20fc14e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 -- 172.21.15.37:0/1534778434 >> 172.21.15.37:0/1534778434 conn(0x7f20fc0b3010 msgr2=0x7f20fc099f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:58.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 -- 172.21.15.37:0/1534778434 shutdown_connections 2024-04-25T08:58:58.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.736+0000 7f2103e68640 1 -- 172.21.15.37:0/1534778434 wait complete. 2024-04-25T08:58:58.745 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:58:58.745 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:58:58.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.876+0000 7f190ec70640 1 Processor -- start 2024-04-25T08:58:58.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.876+0000 7f190ec70640 1 -- start start 2024-04-25T08:58:58.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.876+0000 7f190ec70640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19080ce510 0x7f19080ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.876+0000 7f190ec70640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f19081451f0 0x7f1908142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.876+0000 7f190ec70640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19080cee30 con 0x7f1908144ca0 2024-04-25T08:58:58.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.876+0000 7f190ec70640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19080cc2b0 con 0x7f19081451f0 2024-04-25T08:58:58.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.876+0000 7f190ec70640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190806e290 con 0x7f19080ce510 2024-04-25T08:58:58.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190d20d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f19081451f0 0x7f1908142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f1907fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1908144ca0 0x7f190806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:37250/0 (socket says 172.21.15.37:37250) 2024-04-25T08:58:58.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f1907fff640 1 -- 172.21.15.37:0/2011022102 learned_addr learned my addr 172.21.15.37:0/2011022102 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:58:58.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190ca0c640 1 --2- 172.21.15.37:0/2011022102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19080ce510 0x7f19080ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190d20d640 1 -- 172.21.15.37:0/2011022102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19080ce510 msgr2=0x7f19080ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:58.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190d20d640 1 --2- 172.21.15.37:0/2011022102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19080ce510 0x7f19080ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190d20d640 1 -- 172.21.15.37:0/2011022102 >> v1:172.21.15.37:6789/0 conn(0x7f1908144ca0 legacy=0x7f190806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190d20d640 1 -- 172.21.15.37:0/2011022102 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19080ccc70 con 0x7f19081451f0 2024-04-25T08:58:58.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190d20d640 1 --2- 172.21.15.37:0/2011022102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19081451f0 0x7f1908142680 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f1900009d90 tx=0x7f190001f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad0a04582544eef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:58.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f19077fe640 1 -- 172.21.15.37:0/2011022102 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000b4070 con 0x7f19081451f0 2024-04-25T08:58:58.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f19077fe640 1 -- 172.21.15.37:0/2011022102 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19000af600 con 0x7f19081451f0 2024-04-25T08:58:58.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f19077fe640 1 -- 172.21.15.37:0/2011022102 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19000af9d0 con 0x7f19081451f0 2024-04-25T08:58:58.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190ec70640 1 -- 172.21.15.37:0/2011022102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19081451f0 msgr2=0x7f1908142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190ec70640 1 --2- 172.21.15.37:0/2011022102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19081451f0 0x7f1908142680 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f1900009d90 tx=0x7f190001f150 comp rx=0 tx=0).stop 2024-04-25T08:58:58.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190ec70640 1 -- 172.21.15.37:0/2011022102 shutdown_connections 2024-04-25T08:58:58.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190ec70640 1 --2- 172.21.15.37:0/2011022102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19081451f0 0x7f1908142680 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190ec70640 1 --2- 172.21.15.37:0/2011022102 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19080ce510 0x7f19080ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190ec70640 1 -- 172.21.15.37:0/2011022102 >> 172.21.15.37:0/2011022102 conn(0x7f19080b11a0 msgr2=0x7f19080b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:58.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190ec70640 1 -- 172.21.15.37:0/2011022102 shutdown_connections 2024-04-25T08:58:58.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.880+0000 7f190ec70640 1 -- 172.21.15.37:0/2011022102 wait complete. 2024-04-25T08:58:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190ec70640 1 Processor -- start 2024-04-25T08:58:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190ec70640 1 -- start start 2024-04-25T08:58:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190ec70640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19080ce510 0x7f1908138df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190ec70640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1908144ca0 0x7f19080a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190ec70640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19081451f0 0x7f19081e7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190ec70640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190806ee30 con 0x7f1908144ca0 2024-04-25T08:58:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190ec70640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190806df20 con 0x7f19081451f0 2024-04-25T08:58:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190ec70640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19080cc900 con 0x7f19080ce510 2024-04-25T08:58:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f1907fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1908144ca0 0x7f19080a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f1907fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1908144ca0 0x7f19080a9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56788/0 (socket says 172.21.15.37:56788) 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f1907fff640 1 -- 172.21.15.37:0/1647097960 learned_addr learned my addr 172.21.15.37:0/1647097960 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f1907fff640 1 -- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19080ce510 msgr2=0x7f1908138df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f1907fff640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19080ce510 0x7f1908138df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f1907fff640 1 -- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19081451f0 msgr2=0x7f19081e7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190d20d640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19081451f0 0x7f19081e7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f1907fff640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19081451f0 0x7f19081e7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f1907fff640 1 -- 172.21.15.37:0/1647097960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19080a8e80 con 0x7f1908144ca0 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f1907fff640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1908144ca0 0x7f19080a9f20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c460 tx=0x7f18fc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f19057fa640 1 -- 172.21.15.37:0/1647097960 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc017070 con 0x7f1908144ca0 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f19057fa640 1 -- 172.21.15.37:0/1647097960 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc005cf0 con 0x7f1908144ca0 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190ec70640 1 -- 172.21.15.37:0/1647097960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19080a9160 con 0x7f1908144ca0 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f19057fa640 1 -- 172.21.15.37:0/1647097960 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc007290 con 0x7f1908144ca0 2024-04-25T08:58:58.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190ec70640 1 -- 172.21.15.37:0/1647097960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19080a9720 con 0x7f1908144ca0 2024-04-25T08:58:58.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f190ec70640 1 -- 172.21.15.37:0/1647097960 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190804efa0 con 0x7f1908144ca0 2024-04-25T08:58:58.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.884+0000 7f19057fa640 1 -- 172.21.15.37:0/1647097960 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18fc01a070 con 0x7f1908144ca0 2024-04-25T08:58:58.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.888+0000 7f19057fa640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18f4066f60 0x7f18f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:58.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.888+0000 7f19057fa640 1 -- 172.21.15.37:0/1647097960 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f18fc0844b0 con 0x7f1908144ca0 2024-04-25T08:58:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.888+0000 7f190ca0c640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18f4066f60 0x7f18f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:58.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.888+0000 7f190ca0c640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18f4066f60 0x7f18f4069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f18f80098f0 tx=0x7f18f8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:58.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:58.892+0000 7f19057fa640 1 -- 172.21.15.37:0/1647097960 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18fc055330 con 0x7f1908144ca0 2024-04-25T08:58:59.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.048+0000 7f190ec70640 1 -- 172.21.15.37:0/1647097960 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1908073150 con 0x7f18f4066f60 2024-04-25T08:58:59.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f19057fa640 1 -- 172.21.15.37:0/1647097960 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+246773 (secure 0 0 0) 0x7f1908073150 con 0x7f18f4066f60 2024-04-25T08:58:59.059 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:58:59.060 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:58:59.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 -- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18f4066f60 msgr2=0x7f18f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:59.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18f4066f60 0x7f18f4069420 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f18f80098f0 tx=0x7f18f8008040 comp rx=0 tx=0).stop 2024-04-25T08:58:59.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 -- 172.21.15.37:0/1647097960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1908144ca0 msgr2=0x7f19080a9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:59.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1908144ca0 0x7f19080a9f20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00c460 tx=0x7f18fc00c920 comp rx=0 tx=0).stop 2024-04-25T08:58:59.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 -- 172.21.15.37:0/1647097960 shutdown_connections 2024-04-25T08:58:59.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18f4066f60 0x7f18f4069420 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:59.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19081451f0 0x7f19081e7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:59.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1908144ca0 0x7f19080a9f20 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:59.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 --2- 172.21.15.37:0/1647097960 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19080ce510 0x7f1908138df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:59.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 -- 172.21.15.37:0/1647097960 >> 172.21.15.37:0/1647097960 conn(0x7f19080b11a0 msgr2=0x7f19080c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:59.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 -- 172.21.15.37:0/1647097960 shutdown_connections 2024-04-25T08:58:59.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:58:59.068+0000 7f190ec70640 1 -- 172.21.15.37:0/1647097960 wait complete. 2024-04-25T08:58:59.078 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-04-25T08:58:57.256685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":40,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259576,"kb_used_data":10120,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715144,"statfs":{"total":773094113280,"available":772828307456,"internally_reserved":0,"allocated":10362880,"data_stored":5404656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.791611"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476372+0000","last_change":"2024-04-25T08:58:52.476372+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476372+0000","last_undegraded":"2024-04-25T08:58:52.476372+0000","last_fullsized":"2024-04-25T08:58:52.476372+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485159+0000","last_change":"2024-04-25T08:58:52.485159+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485159+0000","last_undegraded":"2024-04-25T08:58:52.485159+0000","last_fullsized":"2024-04-25T08:58:52.485159+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476534+0000","last_change":"2024-04-25T08:58:52.476534+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476534+0000","last_undegraded":"2024-04-25T08:58:52.476534+0000","last_fullsized":"2024-04-25T08:58:52.476534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.482393+0000","last_change":"2024-04-25T08:58:52.482393+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.482393+0000","last_undegraded":"2024-04-25T08:58:52.482393+0000","last_fullsized":"2024-04-25T08:58:52.482393+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489217+0000","last_change":"2024-04-25T08:58:52.489217+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489217+0000","last_undegraded":"2024-04-25T08:58:52.489217+0000","last_fullsized":"2024-04-25T08:58:52.489217+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490958+0000","last_change":"2024-04-25T08:58:52.490958+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490958+0000","last_undegraded":"2024-04-25T08:58:52.490958+0000","last_fullsized":"2024-04-25T08:58:52.490958+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.495788+0000","last_change":"2024-04-25T08:58:52.495788+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.495788+0000","last_undegraded":"2024-04-25T08:58:52.495788+0000","last_fullsized":"2024-04-25T08:58:52.495788+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480355+0000","last_change":"2024-04-25T08:58:52.480355+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480355+0000","last_undegraded":"2024-04-25T08:58:52.480355+0000","last_fullsized":"2024-04-25T08:58:52.480355+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479616+0000","last_change":"2024-04-25T08:58:52.479616+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479616+0000","last_undegraded":"2024-04-25T08:58:52.479616+0000","last_fullsized":"2024-04-25T08:58:52.479616+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479974+0000","last_change":"2024-04-25T08:58:52.479974+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479974+0000","last_undegraded":"2024-04-25T08:58:52.479974+0000","last_fullsized":"2024-04-25T08:58:52.479974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491682+0000","last_change":"2024-04-25T08:58:52.491682+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491682+0000","last_undegraded":"2024-04-25T08:58:52.491682+0000","last_fullsized":"2024-04-25T08:58:52.491682+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479047+0000","last_change":"2024-04-25T08:58:52.479047+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479047+0000","last_undegraded":"2024-04-25T08:58:52.479047+0000","last_fullsized":"2024-04-25T08:58:52.479047+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.482025+0000","last_change":"2024-04-25T08:58:52.482025+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.482025+0000","last_undegraded":"2024-04-25T08:58:52.482025+0000","last_fullsized":"2024-04-25T08:58:52.482025+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489553+0000","last_change":"2024-04-25T08:58:52.489553+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489553+0000","last_undegraded":"2024-04-25T08:58:52.489553+0000","last_fullsized":"2024-04-25T08:58:52.489553+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478028+0000","last_change":"2024-04-25T08:58:52.478028+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478028+0000","last_undegraded":"2024-04-25T08:58:52.478028+0000","last_fullsized":"2024-04-25T08:58:52.478028+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490790+0000","last_change":"2024-04-25T08:58:52.490790+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490790+0000","last_undegraded":"2024-04-25T08:58:52.490790+0000","last_fullsized":"2024-04-25T08:58:52.490790+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491623+0000","last_change":"2024-04-25T08:58:52.491623+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491623+0000","last_undegraded":"2024-04-25T08:58:52.491623+0000","last_fullsized":"2024-04-25T08:58:52.491623+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478387+0000","last_change":"2024-04-25T08:58:52.478387+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478387+0000","last_undegraded":"2024-04-25T08:58:52.478387+0000","last_fullsized":"2024-04-25T08:58:52.478387+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.489792+0000","last_change":"2024-04-25T08:58:52.489792+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.489792+0000","last_undegraded":"2024-04-25T08:58:52.489792+0000","last_fullsized":"2024-04-25T08:58:52.489792+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481843+0000","last_change":"2024-04-25T08:58:52.481843+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481843+0000","last_undegraded":"2024-04-25T08:58:52.481843+0000","last_fullsized":"2024-04-25T08:58:52.481843+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490305+0000","last_change":"2024-04-25T08:58:52.490305+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490305+0000","last_undegraded":"2024-04-25T08:58:52.490305+0000","last_fullsized":"2024-04-25T08:58:52.490305+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480491+0000","last_change":"2024-04-25T08:58:52.480491+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480491+0000","last_undegraded":"2024-04-25T08:58:52.480491+0000","last_fullsized":"2024-04-25T08:58:52.480491+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481337+0000","last_change":"2024-04-25T08:58:52.481337+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481337+0000","last_undegraded":"2024-04-25T08:58:52.481337+0000","last_fullsized":"2024-04-25T08:58:52.481337+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.475900+0000","last_change":"2024-04-25T08:58:52.475900+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.475900+0000","last_undegraded":"2024-04-25T08:58:52.475900+0000","last_fullsized":"2024-04-25T08:58:52.475900+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487974+0000","last_change":"2024-04-25T08:58:52.487974+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487974+0000","last_undegraded":"2024-04-25T08:58:52.487974+0000","last_fullsized":"2024-04-25T08:58:52.487974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480582+0000","last_change":"2024-04-25T08:58:52.480582+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480582+0000","last_undegraded":"2024-04-25T08:58:52.480582+0000","last_fullsized":"2024-04-25T08:58:52.480582+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478227+0000","last_change":"2024-04-25T08:58:52.478227+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478227+0000","last_undegraded":"2024-04-25T08:58:52.478227+0000","last_fullsized":"2024-04-25T08:58:52.478227+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481255+0000","last_change":"2024-04-25T08:58:52.481255+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481255+0000","last_undegraded":"2024-04-25T08:58:52.481255+0000","last_fullsized":"2024-04-25T08:58:52.481255+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491670+0000","last_change":"2024-04-25T08:58:52.491670+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491670+0000","last_undegraded":"2024-04-25T08:58:52.491670+0000","last_fullsized":"2024-04-25T08:58:52.491670+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479529+0000","last_change":"2024-04-25T08:58:52.479529+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479529+0000","last_undegraded":"2024-04-25T08:58:52.479529+0000","last_fullsized":"2024-04-25T08:58:52.479529+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485384+0000","last_change":"2024-04-25T08:58:52.485384+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485384+0000","last_undegraded":"2024-04-25T08:58:52.485384+0000","last_fullsized":"2024-04-25T08:58:52.485384+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490204+0000","last_change":"2024-04-25T08:58:52.490204+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490204+0000","last_undegraded":"2024-04-25T08:58:52.490204+0000","last_fullsized":"2024-04-25T08:58:52.490204+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487585+0000","last_change":"2024-04-25T08:58:52.487585+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487585+0000","last_undegraded":"2024-04-25T08:58:52.487585+0000","last_fullsized":"2024-04-25T08:58:52.487585+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.509098+0000","last_change":"2024-04-25T08:58:52.499317+0000","last_active":"2024-04-25T08:58:52.509098+0000","last_peered":"2024-04-25T08:58:52.509098+0000","last_clean":"2024-04-25T08:58:52.509098+0000","last_became_active":"2024-04-25T08:58:52.498145+0000","last_became_peered":"2024-04-25T08:58:52.498145+0000","last_unstale":"2024-04-25T08:58:52.509098+0000","last_undegraded":"2024-04-25T08:58:52.509098+0000","last_fullsized":"2024-04-25T08:58:52.509098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:50.604902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487225+0000","last_change":"2024-04-25T08:58:52.487225+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487225+0000","last_undegraded":"2024-04-25T08:58:52.487225+0000","last_fullsized":"2024-04-25T08:58:52.487225+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518038+0000","last_change":"2024-04-25T08:58:52.513581+0000","last_active":"2024-04-25T08:58:52.518038+0000","last_peered":"2024-04-25T08:58:52.518038+0000","last_clean":"2024-04-25T08:58:52.518038+0000","last_became_active":"2024-04-25T08:58:52.512774+0000","last_became_peered":"2024-04-25T08:58:52.512774+0000","last_unstale":"2024-04-25T08:58:52.518038+0000","last_undegraded":"2024-04-25T08:58:52.518038+0000","last_fullsized":"2024-04-25T08:58:52.518038+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:45.689538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487757+0000","last_change":"2024-04-25T08:58:52.487757+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487757+0000","last_undegraded":"2024-04-25T08:58:52.487757+0000","last_fullsized":"2024-04-25T08:58:52.487757+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512573+0000","last_change":"2024-04-25T08:58:52.508417+0000","last_active":"2024-04-25T08:58:52.512573+0000","last_peered":"2024-04-25T08:58:52.512573+0000","last_clean":"2024-04-25T08:58:52.512573+0000","last_became_active":"2024-04-25T08:58:52.507854+0000","last_became_peered":"2024-04-25T08:58:52.507854+0000","last_unstale":"2024-04-25T08:58:52.512573+0000","last_undegraded":"2024-04-25T08:58:52.512573+0000","last_fullsized":"2024-04-25T08:58:52.512573+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:02.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.491046+0000","last_change":"2024-04-25T08:58:52.491046+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.491046+0000","last_undegraded":"2024-04-25T08:58:52.491046+0000","last_fullsized":"2024-04-25T08:58:52.491046+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508855+0000","last_change":"2024-04-25T08:58:52.494753+0000","last_active":"2024-04-25T08:58:52.508855+0000","last_peered":"2024-04-25T08:58:52.508855+0000","last_clean":"2024-04-25T08:58:52.508855+0000","last_became_active":"2024-04-25T08:58:52.493972+0000","last_became_peered":"2024-04-25T08:58:52.493972+0000","last_unstale":"2024-04-25T08:58:52.508855+0000","last_undegraded":"2024-04-25T08:58:52.508855+0000","last_fullsized":"2024-04-25T08:58:52.508855+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:14.712772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479776+0000","last_change":"2024-04-25T08:58:52.479776+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479776+0000","last_undegraded":"2024-04-25T08:58:52.479776+0000","last_fullsized":"2024-04-25T08:58:52.479776+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512397+0000","last_change":"2024-04-25T08:58:52.505294+0000","last_active":"2024-04-25T08:58:52.512397+0000","last_peered":"2024-04-25T08:58:52.512397+0000","last_clean":"2024-04-25T08:58:52.512397+0000","last_became_active":"2024-04-25T08:58:52.504512+0000","last_became_peered":"2024-04-25T08:58:52.504512+0000","last_unstale":"2024-04-25T08:58:52.512397+0000","last_undegraded":"2024-04-25T08:58:52.512397+0000","last_fullsized":"2024-04-25T08:58:52.512397+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:56.905836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478321+0000","last_change":"2024-04-25T08:58:52.478321+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478321+0000","last_undegraded":"2024-04-25T08:58:52.478321+0000","last_fullsized":"2024-04-25T08:58:52.478321+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507544+0000","last_change":"2024-04-25T08:58:52.503172+0000","last_active":"2024-04-25T08:58:52.507544+0000","last_peered":"2024-04-25T08:58:52.507544+0000","last_clean":"2024-04-25T08:58:52.507544+0000","last_became_active":"2024-04-25T08:58:52.502702+0000","last_became_peered":"2024-04-25T08:58:52.502702+0000","last_unstale":"2024-04-25T08:58:52.507544+0000","last_undegraded":"2024-04-25T08:58:52.507544+0000","last_fullsized":"2024-04-25T08:58:52.507544+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486416+0000","last_change":"2024-04-25T08:58:52.486416+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486416+0000","last_undegraded":"2024-04-25T08:58:52.486416+0000","last_fullsized":"2024-04-25T08:58:52.486416+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.502729+0000","last_change":"2024-04-25T08:58:52.501970+0000","last_active":"2024-04-25T08:58:52.502729+0000","last_peered":"2024-04-25T08:58:52.502729+0000","last_clean":"2024-04-25T08:58:52.502729+0000","last_became_active":"2024-04-25T08:58:52.500708+0000","last_became_peered":"2024-04-25T08:58:52.500708+0000","last_unstale":"2024-04-25T08:58:52.502729+0000","last_undegraded":"2024-04-25T08:58:52.502729+0000","last_fullsized":"2024-04-25T08:58:52.502729+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:25.488383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480907+0000","last_change":"2024-04-25T08:58:52.480907+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480907+0000","last_undegraded":"2024-04-25T08:58:52.480907+0000","last_fullsized":"2024-04-25T08:58:52.480907+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512711+0000","last_change":"2024-04-25T08:58:52.506757+0000","last_active":"2024-04-25T08:58:52.512711+0000","last_peered":"2024-04-25T08:58:52.512711+0000","last_clean":"2024-04-25T08:58:52.512711+0000","last_became_active":"2024-04-25T08:58:52.506014+0000","last_became_peered":"2024-04-25T08:58:52.506014+0000","last_unstale":"2024-04-25T08:58:52.512711+0000","last_undegraded":"2024-04-25T08:58:52.512711+0000","last_fullsized":"2024-04-25T08:58:52.512711+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476026+0000","last_change":"2024-04-25T08:58:52.476026+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476026+0000","last_undegraded":"2024-04-25T08:58:52.476026+0000","last_fullsized":"2024-04-25T08:58:52.476026+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508755+0000","last_change":"2024-04-25T08:58:52.505203+0000","last_active":"2024-04-25T08:58:52.508755+0000","last_peered":"2024-04-25T08:58:52.508755+0000","last_clean":"2024-04-25T08:58:52.508755+0000","last_became_active":"2024-04-25T08:58:52.504503+0000","last_became_peered":"2024-04-25T08:58:52.504503+0000","last_unstale":"2024-04-25T08:58:52.508755+0000","last_undegraded":"2024-04-25T08:58:52.508755+0000","last_fullsized":"2024-04-25T08:58:52.508755+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:15.105995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487774+0000","last_change":"2024-04-25T08:58:52.487774+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487774+0000","last_undegraded":"2024-04-25T08:58:52.487774+0000","last_fullsized":"2024-04-25T08:58:52.487774+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.495939+0000","last_change":"2024-04-25T08:58:52.493504+0000","last_active":"2024-04-25T08:58:52.495939+0000","last_peered":"2024-04-25T08:58:52.495939+0000","last_clean":"2024-04-25T08:58:52.495939+0000","last_became_active":"2024-04-25T08:58:52.493077+0000","last_became_peered":"2024-04-25T08:58:52.493077+0000","last_unstale":"2024-04-25T08:58:52.495939+0000","last_undegraded":"2024-04-25T08:58:52.495939+0000","last_fullsized":"2024-04-25T08:58:52.495939+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:12.438660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477801+0000","last_change":"2024-04-25T08:58:52.477801+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477801+0000","last_undegraded":"2024-04-25T08:58:52.477801+0000","last_fullsized":"2024-04-25T08:58:52.477801+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518279+0000","last_change":"2024-04-25T08:58:52.515324+0000","last_active":"2024-04-25T08:58:52.518279+0000","last_peered":"2024-04-25T08:58:52.518279+0000","last_clean":"2024-04-25T08:58:52.518279+0000","last_became_active":"2024-04-25T08:58:52.513958+0000","last_became_peered":"2024-04-25T08:58:52.513958+0000","last_unstale":"2024-04-25T08:58:52.518279+0000","last_undegraded":"2024-04-25T08:58:52.518279+0000","last_fullsized":"2024-04-25T08:58:52.518279+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:07.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477385+0000","last_change":"2024-04-25T08:58:52.477385+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477385+0000","last_undegraded":"2024-04-25T08:58:52.477385+0000","last_fullsized":"2024-04-25T08:58:52.477385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.496529+0000","last_change":"2024-04-25T08:58:52.493433+0000","last_active":"2024-04-25T08:58:52.496529+0000","last_peered":"2024-04-25T08:58:52.496529+0000","last_clean":"2024-04-25T08:58:52.496529+0000","last_became_active":"2024-04-25T08:58:52.493061+0000","last_became_peered":"2024-04-25T08:58:52.493061+0000","last_unstale":"2024-04-25T08:58:52.496529+0000","last_undegraded":"2024-04-25T08:58:52.496529+0000","last_fullsized":"2024-04-25T08:58:52.496529+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:04.908717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.479199+0000","last_change":"2024-04-25T08:58:52.479199+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.479199+0000","last_undegraded":"2024-04-25T08:58:52.479199+0000","last_fullsized":"2024-04-25T08:58:52.479199+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.512641+0000","last_change":"2024-04-25T08:58:52.507149+0000","last_active":"2024-04-25T08:58:52.512641+0000","last_peered":"2024-04-25T08:58:52.512641+0000","last_clean":"2024-04-25T08:58:52.512641+0000","last_became_active":"2024-04-25T08:58:52.506632+0000","last_became_peered":"2024-04-25T08:58:52.506632+0000","last_unstale":"2024-04-25T08:58:52.512641+0000","last_undegraded":"2024-04-25T08:58:52.512641+0000","last_fullsized":"2024-04-25T08:58:52.512641+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:16.136540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476701+0000","last_change":"2024-04-25T08:58:52.476701+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476701+0000","last_undegraded":"2024-04-25T08:58:52.476701+0000","last_fullsized":"2024-04-25T08:58:52.476701+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513002+0000","last_change":"2024-04-25T08:58:52.508620+0000","last_active":"2024-04-25T08:58:52.513002+0000","last_peered":"2024-04-25T08:58:52.513002+0000","last_clean":"2024-04-25T08:58:52.513002+0000","last_became_active":"2024-04-25T08:58:52.508361+0000","last_became_peered":"2024-04-25T08:58:52.508361+0000","last_unstale":"2024-04-25T08:58:52.513002+0000","last_undegraded":"2024-04-25T08:58:52.513002+0000","last_fullsized":"2024-04-25T08:58:52.513002+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.319442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.480028+0000","last_change":"2024-04-25T08:58:52.480028+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.480028+0000","last_undegraded":"2024-04-25T08:58:52.480028+0000","last_fullsized":"2024-04-25T08:58:52.480028+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507782+0000","last_change":"2024-04-25T08:58:52.503640+0000","last_active":"2024-04-25T08:58:52.507782+0000","last_peered":"2024-04-25T08:58:52.507782+0000","last_clean":"2024-04-25T08:58:52.507782+0000","last_became_active":"2024-04-25T08:58:52.503271+0000","last_became_peered":"2024-04-25T08:58:52.503271+0000","last_unstale":"2024-04-25T08:58:52.507782+0000","last_undegraded":"2024-04-25T08:58:52.507782+0000","last_fullsized":"2024-04-25T08:58:52.507782+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:02.528462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.477547+0000","last_change":"2024-04-25T08:58:52.477547+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.477547+0000","last_undegraded":"2024-04-25T08:58:52.477547+0000","last_fullsized":"2024-04-25T08:58:52.477547+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507718+0000","last_change":"2024-04-25T08:58:52.503187+0000","last_active":"2024-04-25T08:58:52.507718+0000","last_peered":"2024-04-25T08:58:52.507718+0000","last_clean":"2024-04-25T08:58:52.507718+0000","last_became_active":"2024-04-25T08:58:52.502662+0000","last_became_peered":"2024-04-25T08:58:52.502662+0000","last_unstale":"2024-04-25T08:58:52.507718+0000","last_undegraded":"2024-04-25T08:58:52.507718+0000","last_fullsized":"2024-04-25T08:58:52.507718+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:28.688051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.485819+0000","last_change":"2024-04-25T08:58:52.485819+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.485819+0000","last_undegraded":"2024-04-25T08:58:52.485819+0000","last_fullsized":"2024-04-25T08:58:52.485819+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513658+0000","last_change":"2024-04-25T08:58:52.502738+0000","last_active":"2024-04-25T08:58:52.513658+0000","last_peered":"2024-04-25T08:58:52.513658+0000","last_clean":"2024-04-25T08:58:52.513658+0000","last_became_active":"2024-04-25T08:58:52.501965+0000","last_became_peered":"2024-04-25T08:58:52.501965+0000","last_unstale":"2024-04-25T08:58:52.513658+0000","last_undegraded":"2024-04-25T08:58:52.513658+0000","last_fullsized":"2024-04-25T08:58:52.513658+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:30.608887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481261+0000","last_change":"2024-04-25T08:58:52.481261+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481261+0000","last_undegraded":"2024-04-25T08:58:52.481261+0000","last_fullsized":"2024-04-25T08:58:52.481261+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.513182+0000","last_change":"2024-04-25T08:58:52.510654+0000","last_active":"2024-04-25T08:58:52.513182+0000","last_peered":"2024-04-25T08:58:52.513182+0000","last_clean":"2024-04-25T08:58:52.513182+0000","last_became_active":"2024-04-25T08:58:52.509735+0000","last_became_peered":"2024-04-25T08:58:52.509735+0000","last_unstale":"2024-04-25T08:58:52.513182+0000","last_undegraded":"2024-04-25T08:58:52.513182+0000","last_fullsized":"2024-04-25T08:58:52.513182+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:23.084950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476876+0000","last_change":"2024-04-25T08:58:52.476876+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476876+0000","last_undegraded":"2024-04-25T08:58:52.476876+0000","last_fullsized":"2024-04-25T08:58:52.476876+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.518225+0000","last_change":"2024-04-25T08:58:52.514656+0000","last_active":"2024-04-25T08:58:52.518225+0000","last_peered":"2024-04-25T08:58:52.518225+0000","last_clean":"2024-04-25T08:58:52.518225+0000","last_became_active":"2024-04-25T08:58:52.513150+0000","last_became_peered":"2024-04-25T08:58:52.513150+0000","last_unstale":"2024-04-25T08:58:52.518225+0000","last_undegraded":"2024-04-25T08:58:52.518225+0000","last_fullsized":"2024-04-25T08:58:52.518225+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:40.111643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476469+0000","last_change":"2024-04-25T08:58:52.476469+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476469+0000","last_undegraded":"2024-04-25T08:58:52.476469+0000","last_fullsized":"2024-04-25T08:58:52.476469+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503595+0000","last_change":"2024-04-25T08:58:52.494154+0000","last_active":"2024-04-25T08:58:52.503595+0000","last_peered":"2024-04-25T08:58:52.503595+0000","last_clean":"2024-04-25T08:58:52.503595+0000","last_became_active":"2024-04-25T08:58:52.493429+0000","last_became_peered":"2024-04-25T08:58:52.493429+0000","last_unstale":"2024-04-25T08:58:52.503595+0000","last_undegraded":"2024-04-25T08:58:52.503595+0000","last_fullsized":"2024-04-25T08:58:52.503595+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:30.608887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.483968+0000","last_change":"2024-04-25T08:58:52.483968+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.483968+0000","last_undegraded":"2024-04-25T08:58:52.483968+0000","last_fullsized":"2024-04-25T08:58:52.483968+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.501305+0000","last_change":"2024-04-25T08:58:52.494299+0000","last_active":"2024-04-25T08:58:52.501305+0000","last_peered":"2024-04-25T08:58:52.501305+0000","last_clean":"2024-04-25T08:58:52.501305+0000","last_became_active":"2024-04-25T08:58:52.493557+0000","last_became_peered":"2024-04-25T08:58:52.493557+0000","last_unstale":"2024-04-25T08:58:52.501305+0000","last_undegraded":"2024-04-25T08:58:52.501305+0000","last_fullsized":"2024-04-25T08:58:52.501305+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:41.335795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486514+0000","last_change":"2024-04-25T08:58:52.486514+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486514+0000","last_undegraded":"2024-04-25T08:58:52.486514+0000","last_fullsized":"2024-04-25T08:58:52.486514+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.507821+0000","last_change":"2024-04-25T08:58:52.500643+0000","last_active":"2024-04-25T08:58:52.507821+0000","last_peered":"2024-04-25T08:58:52.507821+0000","last_clean":"2024-04-25T08:58:52.507821+0000","last_became_active":"2024-04-25T08:58:52.499551+0000","last_became_peered":"2024-04-25T08:58:52.499551+0000","last_unstale":"2024-04-25T08:58:52.507821+0000","last_undegraded":"2024-04-25T08:58:52.507821+0000","last_fullsized":"2024-04-25T08:58:52.507821+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:06.052032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.487020+0000","last_change":"2024-04-25T08:58:52.487020+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.487020+0000","last_undegraded":"2024-04-25T08:58:52.487020+0000","last_fullsized":"2024-04-25T08:58:52.487020+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503818+0000","last_change":"2024-04-25T08:58:52.501153+0000","last_active":"2024-04-25T08:58:52.503818+0000","last_peered":"2024-04-25T08:58:52.503818+0000","last_clean":"2024-04-25T08:58:52.503818+0000","last_became_active":"2024-04-25T08:58:52.497820+0000","last_became_peered":"2024-04-25T08:58:52.497820+0000","last_unstale":"2024-04-25T08:58:52.503818+0000","last_undegraded":"2024-04-25T08:58:52.503818+0000","last_fullsized":"2024-04-25T08:58:52.503818+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:05.283812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.481896+0000","last_change":"2024-04-25T08:58:52.481896+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.481896+0000","last_undegraded":"2024-04-25T08:58:52.481896+0000","last_fullsized":"2024-04-25T08:58:52.481896+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.517562+0000","last_change":"2024-04-25T08:58:52.511956+0000","last_active":"2024-04-25T08:58:52.517562+0000","last_peered":"2024-04-25T08:58:52.517562+0000","last_clean":"2024-04-25T08:58:52.517562+0000","last_became_active":"2024-04-25T08:58:52.511196+0000","last_became_peered":"2024-04-25T08:58:52.511196+0000","last_unstale":"2024-04-25T08:58:52.517562+0000","last_undegraded":"2024-04-25T08:58:52.517562+0000","last_fullsized":"2024-04-25T08:58:52.517562+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:09.214520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476443+0000","last_change":"2024-04-25T08:58:52.476443+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476443+0000","last_undegraded":"2024-04-25T08:58:52.476443+0000","last_fullsized":"2024-04-25T08:58:52.476443+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.517853+0000","last_change":"2024-04-25T08:58:52.512155+0000","last_active":"2024-04-25T08:58:52.517853+0000","last_peered":"2024-04-25T08:58:52.517853+0000","last_clean":"2024-04-25T08:58:52.517853+0000","last_became_active":"2024-04-25T08:58:52.506602+0000","last_became_peered":"2024-04-25T08:58:52.506602+0000","last_unstale":"2024-04-25T08:58:52.517853+0000","last_undegraded":"2024-04-25T08:58:52.517853+0000","last_fullsized":"2024-04-25T08:58:52.517853+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:09.340774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476535+0000","last_change":"2024-04-25T08:58:52.476535+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476535+0000","last_undegraded":"2024-04-25T08:58:52.476535+0000","last_fullsized":"2024-04-25T08:58:52.476535+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.515015+0000","last_change":"2024-04-25T08:58:52.508006+0000","last_active":"2024-04-25T08:58:52.515015+0000","last_peered":"2024-04-25T08:58:52.515015+0000","last_clean":"2024-04-25T08:58:52.515015+0000","last_became_active":"2024-04-25T08:58:52.507149+0000","last_became_peered":"2024-04-25T08:58:52.507149+0000","last_unstale":"2024-04-25T08:58:52.515015+0000","last_undegraded":"2024-04-25T08:58:52.515015+0000","last_fullsized":"2024-04-25T08:58:52.515015+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:04.306110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.486979+0000","last_change":"2024-04-25T08:58:52.486979+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.486979+0000","last_undegraded":"2024-04-25T08:58:52.486979+0000","last_fullsized":"2024-04-25T08:58:52.486979+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.508975+0000","last_change":"2024-04-25T08:58:52.495139+0000","last_active":"2024-04-25T08:58:52.508975+0000","last_peered":"2024-04-25T08:58:52.508975+0000","last_clean":"2024-04-25T08:58:52.508975+0000","last_became_active":"2024-04-25T08:58:52.494329+0000","last_became_peered":"2024-04-25T08:58:52.494329+0000","last_unstale":"2024-04-25T08:58:52.508975+0000","last_undegraded":"2024-04-25T08:58:52.508975+0000","last_fullsized":"2024-04-25T08:58:52.508975+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:54.968958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.478270+0000","last_change":"2024-04-25T08:58:52.478270+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.478270+0000","last_undegraded":"2024-04-25T08:58:52.478270+0000","last_fullsized":"2024-04-25T08:58:52.478270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.516378+0000","last_change":"2024-04-25T08:58:52.513900+0000","last_active":"2024-04-25T08:58:52.516378+0000","last_peered":"2024-04-25T08:58:52.516378+0000","last_clean":"2024-04-25T08:58:52.516378+0000","last_became_active":"2024-04-25T08:58:52.513643+0000","last_became_peered":"2024-04-25T08:58:52.513643+0000","last_unstale":"2024-04-25T08:58:52.516378+0000","last_undegraded":"2024-04-25T08:58:52.516378+0000","last_fullsized":"2024-04-25T08:58:52.516378+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:51.599278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476352+0000","last_change":"2024-04-25T08:58:52.476352+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476352+0000","last_undegraded":"2024-04-25T08:58:52.476352+0000","last_fullsized":"2024-04-25T08:58:52.476352+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503475+0000","last_change":"2024-04-25T08:58:52.493529+0000","last_active":"2024-04-25T08:58:52.503475+0000","last_peered":"2024-04-25T08:58:52.503475+0000","last_clean":"2024-04-25T08:58:52.503475+0000","last_became_active":"2024-04-25T08:58:52.492691+0000","last_became_peered":"2024-04-25T08:58:52.492691+0000","last_unstale":"2024-04-25T08:58:52.503475+0000","last_undegraded":"2024-04-25T08:58:52.503475+0000","last_fullsized":"2024-04-25T08:58:52.503475+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:50.604902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.476218+0000","last_change":"2024-04-25T08:58:52.476218+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.476218+0000","last_undegraded":"2024-04-25T08:58:52.476218+0000","last_fullsized":"2024-04-25T08:58:52.476218+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.509136+0000","last_change":"2024-04-25T08:58:52.495019+0000","last_active":"2024-04-25T08:58:52.509136+0000","last_peered":"2024-04-25T08:58:52.509136+0000","last_clean":"2024-04-25T08:58:52.509136+0000","last_became_active":"2024-04-25T08:58:52.494237+0000","last_became_peered":"2024-04-25T08:58:52.494237+0000","last_unstale":"2024-04-25T08:58:52.509136+0000","last_undegraded":"2024-04-25T08:58:52.509136+0000","last_fullsized":"2024-04-25T08:58:52.509136+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:35.222412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.475654+0000","last_change":"2024-04-25T08:58:52.475654+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.475654+0000","last_undegraded":"2024-04-25T08:58:52.475654+0000","last_fullsized":"2024-04-25T08:58:52.475654+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.503768+0000","last_change":"2024-04-25T08:58:52.502403+0000","last_active":"2024-04-25T08:58:52.503768+0000","last_peered":"2024-04-25T08:58:52.503768+0000","last_clean":"2024-04-25T08:58:52.503768+0000","last_became_active":"2024-04-25T08:58:52.501501+0000","last_became_peered":"2024-04-25T08:58:52.501501+0000","last_unstale":"2024-04-25T08:58:52.503768+0000","last_undegraded":"2024-04-25T08:58:52.503768+0000","last_fullsized":"2024-04-25T08:58:52.503768+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:12.458893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2024-04-25T08:58:52.490599+0000","last_change":"2024-04-25T08:58:52.490599+0000","last_active":"2024-04-25T08:58:52.462142+0000","last_peered":"2024-04-25T08:58:52.462142+0000","last_clean":"2024-04-25T08:58:52.462142+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:58:52.490599+0000","last_undegraded":"2024-04-25T08:58:52.490599+0000","last_fullsized":"2024-04-25T08:58:52.490599+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.498266+0000","last_change":"2024-04-25T08:58:52.493037+0000","last_active":"2024-04-25T08:58:52.498266+0000","last_peered":"2024-04-25T08:58:52.498266+0000","last_clean":"2024-04-25T08:58:52.498266+0000","last_became_active":"2024-04-25T08:58:52.492644+0000","last_became_peered":"2024-04-25T08:58:52.492644+0000","last_unstale":"2024-04-25T08:58:52.498266+0000","last_undegraded":"2024-04-25T08:58:52.498266+0000","last_fullsized":"2024-04-25T08:58:52.498266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:07.296288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":307,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T08:58:52.497611+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:58:52.497611+0000","last_peered":"2024-04-25T08:58:52.497611+0000","last_clean":"2024-04-25T08:58:52.497611+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:58:52.497611+0000","last_undegraded":"2024-04-25T08:58:52.497611+0000","last_fullsized":"2024-04-25T08:58:52.497611+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1740800,"data_stored":1118358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574931,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32308,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339532,"statfs":{"total":96636764160,"available":96603680768,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574931,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":527990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574932,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1740800,"data_stored":1118358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:59:02.083 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:59:02.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe314783640 1 Processor -- start 2024-04-25T08:59:02.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe314783640 1 -- start start 2024-04-25T08:59:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe314783640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c15a700 0x7fe30c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe314783640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe30c0b3f50 0x7fe30c0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe314783640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c0b2090 con 0x7fe30c0ce430 2024-04-25T08:59:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe314783640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c15f200 con 0x7fe30c0b3f50 2024-04-25T08:59:02.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe314783640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0b43e0 con 0x7fe30c15a700 2024-04-25T08:59:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe311d1e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe30c0b3f50 0x7fe30c0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe311d1e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe30c0b3f50 0x7fe30c0cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:48974/0 (socket says 172.21.15.37:48974) 2024-04-25T08:59:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe311d1e640 1 -- 172.21.15.37:0/1173973206 learned_addr learned my addr 172.21.15.37:0/1173973206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe31251f640 1 --2- 172.21.15.37:0/1173973206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c15a700 0x7fe30c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe311d1e640 1 -- 172.21.15.37:0/1173973206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c15a700 msgr2=0x7fe30c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:02.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe311d1e640 1 --2- 172.21.15.37:0/1173973206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c15a700 0x7fe30c15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe311d1e640 1 -- 172.21.15.37:0/1173973206 >> v1:172.21.15.203:6789/0 conn(0x7fe30c0ce430 legacy=0x7fe30c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.228+0000 7fe311d1e640 1 -- 172.21.15.37:0/1173973206 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c0b46c0 con 0x7fe30c0b3f50 2024-04-25T08:59:02.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe311d1e640 1 --2- 172.21.15.37:0/1173973206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c0b3f50 0x7fe30c0cdef0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009870 tx=0x7fe2fc01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26464af108aba739 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:02.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe31151d640 1 -- 172.21.15.37:0/1173973206 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0ab610 con 0x7fe30c0b3f50 2024-04-25T08:59:02.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe31151d640 1 -- 172.21.15.37:0/1173973206 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2fc0b0070 con 0x7fe30c0b3f50 2024-04-25T08:59:02.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe31151d640 1 -- 172.21.15.37:0/1173973206 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2fc0ab9d0 con 0x7fe30c0b3f50 2024-04-25T08:59:02.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 -- 172.21.15.37:0/1173973206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c0b3f50 msgr2=0x7fe30c0cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 --2- 172.21.15.37:0/1173973206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c0b3f50 0x7fe30c0cdef0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc009870 tx=0x7fe2fc01c410 comp rx=0 tx=0).stop 2024-04-25T08:59:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 -- 172.21.15.37:0/1173973206 shutdown_connections 2024-04-25T08:59:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 --2- 172.21.15.37:0/1173973206 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c0b3f50 0x7fe30c0cdef0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 --2- 172.21.15.37:0/1173973206 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c15a700 0x7fe30c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 -- 172.21.15.37:0/1173973206 >> 172.21.15.37:0/1173973206 conn(0x7fe30c0b11a0 msgr2=0x7fe30c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 -- 172.21.15.37:0/1173973206 shutdown_connections 2024-04-25T08:59:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 -- 172.21.15.37:0/1173973206 wait complete. 2024-04-25T08:59:02.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 Processor -- start 2024-04-25T08:59:02.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 -- start start 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c0ce430 0x7fe30c1f48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c15a700 0x7fe30c15f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe30c0b0230 0x7fe30c0af650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0cd490 con 0x7fe30c0b0230 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0cd310 con 0x7fe30c15a700 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe314783640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0b2bb0 con 0x7fe30c0ce430 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe312d20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe30c0b0230 0x7fe30c0af650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe312d20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe30c0b0230 0x7fe30c0af650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52584/0 (socket says 172.21.15.37:52584) 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.232+0000 7fe312d20640 1 -- 172.21.15.37:0/3795716470 learned_addr learned my addr 172.21.15.37:0/3795716470 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe311d1e640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c15a700 0x7fe30c15f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe31251f640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c0ce430 0x7fe30c1f48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe312d20640 1 -- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c0ce430 msgr2=0x7fe30c1f48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe312d20640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c0ce430 0x7fe30c1f48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe312d20640 1 -- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c15a700 msgr2=0x7fe30c15f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe312d20640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c15a700 0x7fe30c15f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe312d20640 1 -- 172.21.15.37:0/3795716470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c155c20 con 0x7fe30c0b0230 2024-04-25T08:59:02.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe312d20640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe30c0b0230 0x7fe30c0af650 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c770 tx=0x7fe30400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:02.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe302ffd640 1 -- 172.21.15.37:0/3795716470 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe304003b80 con 0x7fe30c0b0230 2024-04-25T08:59:02.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe302ffd640 1 -- 172.21.15.37:0/3795716470 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe304003d20 con 0x7fe30c0b0230 2024-04-25T08:59:02.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe314783640 1 -- 172.21.15.37:0/3795716470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c155f00 con 0x7fe30c0b0230 2024-04-25T08:59:02.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe302ffd640 1 -- 172.21.15.37:0/3795716470 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe30400aa30 con 0x7fe30c0b0230 2024-04-25T08:59:02.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe314783640 1 -- 172.21.15.37:0/3795716470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c1f4e20 con 0x7fe30c0b0230 2024-04-25T08:59:02.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe314783640 1 -- 172.21.15.37:0/3795716470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c04efa0 con 0x7fe30c0b0230 2024-04-25T08:59:02.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe302ffd640 1 -- 172.21.15.37:0/3795716470 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe30400abd0 con 0x7fe30c0b0230 2024-04-25T08:59:02.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe302ffd640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f0066f60 0x7fe2f0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe302ffd640 1 -- 172.21.15.37:0/3795716470 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe304084f80 con 0x7fe30c0b0230 2024-04-25T08:59:02.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe31251f640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f0066f60 0x7fe2f0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.236+0000 7fe31251f640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f0066f60 0x7fe2f0069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe308009af0 tx=0x7fe308008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:02.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.240+0000 7fe302ffd640 1 -- 172.21.15.37:0/3795716470 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe304055e90 con 0x7fe30c0b0230 2024-04-25T08:59:02.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.396+0000 7fe314783640 1 -- 172.21.15.37:0/3795716470 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe30c06f260 con 0x7fe2f0066f60 2024-04-25T08:59:02.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.416+0000 7fe302ffd640 1 -- 172.21.15.37:0/3795716470 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252076 (secure 0 0 0) 0x7fe30c06f260 con 0x7fe2f0066f60 2024-04-25T08:59:02.411 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:59:02.412 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T08:59:02.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 -- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f0066f60 msgr2=0x7fe2f0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f0066f60 0x7fe2f0069420 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe308009af0 tx=0x7fe308008040 comp rx=0 tx=0).stop 2024-04-25T08:59:02.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 -- 172.21.15.37:0/3795716470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe30c0b0230 msgr2=0x7fe30c0af650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe30c0b0230 0x7fe30c0af650 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c770 tx=0x7fe30400cc30 comp rx=0 tx=0).stop 2024-04-25T08:59:02.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 -- 172.21.15.37:0/3795716470 shutdown_connections 2024-04-25T08:59:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe2f0066f60 0x7fe2f0069420 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe30c0b0230 0x7fe30c0af650 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe30c15a700 0x7fe30c15f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 --2- 172.21.15.37:0/3795716470 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe30c0ce430 0x7fe30c1f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 -- 172.21.15.37:0/3795716470 >> 172.21.15.37:0/3795716470 conn(0x7fe30c0b11a0 msgr2=0x7fe30c071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 -- 172.21.15.37:0/3795716470 shutdown_connections 2024-04-25T08:59:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.420+0000 7fe314783640 1 -- 172.21.15.37:0/3795716470 wait complete. 2024-04-25T08:59:02.428 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-04-25T08:59:01.258487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259860,"kb_used_data":10376,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714860,"statfs":{"total":773094113280,"available":772828016640,"internally_reserved":0,"allocated":10625024,"data_stored":5520348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003120"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776660+0000","last_change":"2024-04-25T08:58:53.553942+0000","last_active":"2024-04-25T08:58:54.776660+0000","last_peered":"2024-04-25T08:58:54.776660+0000","last_clean":"2024-04-25T08:58:54.776660+0000","last_became_active":"2024-04-25T08:58:53.553572+0000","last_became_peered":"2024-04-25T08:58:53.553572+0000","last_unstale":"2024-04-25T08:58:54.776660+0000","last_undegraded":"2024-04-25T08:58:54.776660+0000","last_fullsized":"2024-04-25T08:58:54.776660+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:41.837491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.778043+0000","last_change":"2024-04-25T08:58:53.552881+0000","last_active":"2024-04-25T08:58:54.778043+0000","last_peered":"2024-04-25T08:58:54.778043+0000","last_clean":"2024-04-25T08:58:54.778043+0000","last_became_active":"2024-04-25T08:58:53.552181+0000","last_became_peered":"2024-04-25T08:58:53.552181+0000","last_unstale":"2024-04-25T08:58:54.778043+0000","last_undegraded":"2024-04-25T08:58:54.778043+0000","last_fullsized":"2024-04-25T08:58:54.778043+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:38.994781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.775240+0000","last_change":"2024-04-25T08:58:53.564540+0000","last_active":"2024-04-25T08:58:54.775240+0000","last_peered":"2024-04-25T08:58:54.775240+0000","last_clean":"2024-04-25T08:58:54.775240+0000","last_became_active":"2024-04-25T08:58:53.564172+0000","last_became_peered":"2024-04-25T08:58:53.564172+0000","last_unstale":"2024-04-25T08:58:54.775240+0000","last_undegraded":"2024-04-25T08:58:54.775240+0000","last_fullsized":"2024-04-25T08:58:54.775240+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:51.999959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777848+0000","last_change":"2024-04-25T08:58:53.536070+0000","last_active":"2024-04-25T08:58:54.777848+0000","last_peered":"2024-04-25T08:58:54.777848+0000","last_clean":"2024-04-25T08:58:54.777848+0000","last_became_active":"2024-04-25T08:58:53.535693+0000","last_became_peered":"2024-04-25T08:58:53.535693+0000","last_unstale":"2024-04-25T08:58:54.777848+0000","last_undegraded":"2024-04-25T08:58:54.777848+0000","last_fullsized":"2024-04-25T08:58:54.777848+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:00.599483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.539124+0000","last_change":"2024-04-25T08:58:53.534117+0000","last_active":"2024-04-25T08:58:54.539124+0000","last_peered":"2024-04-25T08:58:54.539124+0000","last_clean":"2024-04-25T08:58:54.539124+0000","last_became_active":"2024-04-25T08:58:53.533420+0000","last_became_peered":"2024-04-25T08:58:53.533420+0000","last_unstale":"2024-04-25T08:58:54.539124+0000","last_undegraded":"2024-04-25T08:58:54.539124+0000","last_fullsized":"2024-04-25T08:58:54.539124+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:21.102659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557114+0000","last_change":"2024-04-25T08:58:53.552803+0000","last_active":"2024-04-25T08:58:54.557114+0000","last_peered":"2024-04-25T08:58:54.557114+0000","last_clean":"2024-04-25T08:58:54.557114+0000","last_became_active":"2024-04-25T08:58:53.552502+0000","last_became_peered":"2024-04-25T08:58:53.552502+0000","last_unstale":"2024-04-25T08:58:54.557114+0000","last_undegraded":"2024-04-25T08:58:54.557114+0000","last_fullsized":"2024-04-25T08:58:54.557114+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:12.527505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776657+0000","last_change":"2024-04-25T08:58:53.557105+0000","last_active":"2024-04-25T08:58:54.776657+0000","last_peered":"2024-04-25T08:58:54.776657+0000","last_clean":"2024-04-25T08:58:54.776657+0000","last_became_active":"2024-04-25T08:58:53.556378+0000","last_became_peered":"2024-04-25T08:58:53.556378+0000","last_unstale":"2024-04-25T08:58:54.776657+0000","last_undegraded":"2024-04-25T08:58:54.776657+0000","last_fullsized":"2024-04-25T08:58:54.776657+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:49.262454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.775043+0000","last_change":"2024-04-25T08:58:53.549952+0000","last_active":"2024-04-25T08:58:54.775043+0000","last_peered":"2024-04-25T08:58:54.775043+0000","last_clean":"2024-04-25T08:58:54.775043+0000","last_became_active":"2024-04-25T08:58:53.549584+0000","last_became_peered":"2024-04-25T08:58:53.549584+0000","last_unstale":"2024-04-25T08:58:54.775043+0000","last_undegraded":"2024-04-25T08:58:54.775043+0000","last_fullsized":"2024-04-25T08:58:54.775043+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:23.105511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.539908+0000","last_change":"2024-04-25T08:58:53.543764+0000","last_active":"2024-04-25T08:58:54.539908+0000","last_peered":"2024-04-25T08:58:54.539908+0000","last_clean":"2024-04-25T08:58:54.539908+0000","last_became_active":"2024-04-25T08:58:53.543036+0000","last_became_peered":"2024-04-25T08:58:53.543036+0000","last_unstale":"2024-04-25T08:58:54.539908+0000","last_undegraded":"2024-04-25T08:58:54.539908+0000","last_fullsized":"2024-04-25T08:58:54.539908+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:17.456909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.536580+0000","last_change":"2024-04-25T08:58:53.548622+0000","last_active":"2024-04-25T08:58:54.536580+0000","last_peered":"2024-04-25T08:58:54.536580+0000","last_clean":"2024-04-25T08:58:54.536580+0000","last_became_active":"2024-04-25T08:58:53.548315+0000","last_became_peered":"2024-04-25T08:58:53.548315+0000","last_unstale":"2024-04-25T08:58:54.536580+0000","last_undegraded":"2024-04-25T08:58:54.536580+0000","last_fullsized":"2024-04-25T08:58:54.536580+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:41.333637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.539974+0000","last_change":"2024-04-25T08:58:53.564972+0000","last_active":"2024-04-25T08:58:54.539974+0000","last_peered":"2024-04-25T08:58:54.539974+0000","last_clean":"2024-04-25T08:58:54.539974+0000","last_became_active":"2024-04-25T08:58:53.563915+0000","last_became_peered":"2024-04-25T08:58:53.563915+0000","last_unstale":"2024-04-25T08:58:54.539974+0000","last_undegraded":"2024-04-25T08:58:54.539974+0000","last_fullsized":"2024-04-25T08:58:54.539974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:00.599483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.536267+0000","last_change":"2024-04-25T08:58:53.542960+0000","last_active":"2024-04-25T08:58:54.536267+0000","last_peered":"2024-04-25T08:58:54.536267+0000","last_clean":"2024-04-25T08:58:54.536267+0000","last_became_active":"2024-04-25T08:58:53.542701+0000","last_became_peered":"2024-04-25T08:58:53.542701+0000","last_unstale":"2024-04-25T08:58:54.536267+0000","last_undegraded":"2024-04-25T08:58:54.536267+0000","last_fullsized":"2024-04-25T08:58:54.536267+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:47.694335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.775897+0000","last_change":"2024-04-25T08:58:53.547089+0000","last_active":"2024-04-25T08:58:54.775897+0000","last_peered":"2024-04-25T08:58:54.775897+0000","last_clean":"2024-04-25T08:58:54.775897+0000","last_became_active":"2024-04-25T08:58:53.546787+0000","last_became_peered":"2024-04-25T08:58:53.546787+0000","last_unstale":"2024-04-25T08:58:54.775897+0000","last_undegraded":"2024-04-25T08:58:54.775897+0000","last_fullsized":"2024-04-25T08:58:54.775897+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:21.102659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.538109+0000","last_change":"2024-04-25T08:58:53.563677+0000","last_active":"2024-04-25T08:58:54.538109+0000","last_peered":"2024-04-25T08:58:54.538109+0000","last_clean":"2024-04-25T08:58:54.538109+0000","last_became_active":"2024-04-25T08:58:53.559489+0000","last_became_peered":"2024-04-25T08:58:53.559489+0000","last_unstale":"2024-04-25T08:58:54.538109+0000","last_undegraded":"2024-04-25T08:58:54.538109+0000","last_fullsized":"2024-04-25T08:58:54.538109+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:23.105511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.775706+0000","last_change":"2024-04-25T08:58:53.552971+0000","last_active":"2024-04-25T08:58:54.775706+0000","last_peered":"2024-04-25T08:58:54.775706+0000","last_clean":"2024-04-25T08:58:54.775706+0000","last_became_active":"2024-04-25T08:58:53.552346+0000","last_became_peered":"2024-04-25T08:58:53.552346+0000","last_unstale":"2024-04-25T08:58:54.775706+0000","last_undegraded":"2024-04-25T08:58:54.775706+0000","last_fullsized":"2024-04-25T08:58:54.775706+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:11.999725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.556961+0000","last_change":"2024-04-25T08:58:53.550482+0000","last_active":"2024-04-25T08:58:54.556961+0000","last_peered":"2024-04-25T08:58:54.556961+0000","last_clean":"2024-04-25T08:58:54.556961+0000","last_became_active":"2024-04-25T08:58:53.550121+0000","last_became_peered":"2024-04-25T08:58:53.550121+0000","last_unstale":"2024-04-25T08:58:54.556961+0000","last_undegraded":"2024-04-25T08:58:54.556961+0000","last_fullsized":"2024-04-25T08:58:54.556961+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:59.338873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.558011+0000","last_change":"2024-04-25T08:58:53.546933+0000","last_active":"2024-04-25T08:58:54.558011+0000","last_peered":"2024-04-25T08:58:54.558011+0000","last_clean":"2024-04-25T08:58:54.558011+0000","last_became_active":"2024-04-25T08:58:53.546569+0000","last_became_peered":"2024-04-25T08:58:53.546569+0000","last_unstale":"2024-04-25T08:58:54.558011+0000","last_undegraded":"2024-04-25T08:58:54.558011+0000","last_fullsized":"2024-04-25T08:58:54.558011+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:38.466863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776384+0000","last_change":"2024-04-25T08:58:53.548950+0000","last_active":"2024-04-25T08:58:54.776384+0000","last_peered":"2024-04-25T08:58:54.776384+0000","last_clean":"2024-04-25T08:58:54.776384+0000","last_became_active":"2024-04-25T08:58:53.548568+0000","last_became_peered":"2024-04-25T08:58:53.548568+0000","last_unstale":"2024-04-25T08:58:54.776384+0000","last_undegraded":"2024-04-25T08:58:54.776384+0000","last_fullsized":"2024-04-25T08:58:54.776384+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:31.277146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.540255+0000","last_change":"2024-04-25T08:58:53.563733+0000","last_active":"2024-04-25T08:58:54.540255+0000","last_peered":"2024-04-25T08:58:54.540255+0000","last_clean":"2024-04-25T08:58:54.540255+0000","last_became_active":"2024-04-25T08:58:53.563124+0000","last_became_peered":"2024-04-25T08:58:53.563124+0000","last_unstale":"2024-04-25T08:58:54.540255+0000","last_undegraded":"2024-04-25T08:58:54.540255+0000","last_fullsized":"2024-04-25T08:58:54.540255+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:11.697774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.778071+0000","last_change":"2024-04-25T08:58:53.553595+0000","last_active":"2024-04-25T08:58:54.778071+0000","last_peered":"2024-04-25T08:58:54.778071+0000","last_clean":"2024-04-25T08:58:54.778071+0000","last_became_active":"2024-04-25T08:58:53.553248+0000","last_became_peered":"2024-04-25T08:58:53.553248+0000","last_unstale":"2024-04-25T08:58:54.778071+0000","last_undegraded":"2024-04-25T08:58:54.778071+0000","last_fullsized":"2024-04-25T08:58:54.778071+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:38.994781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.538399+0000","last_change":"2024-04-25T08:58:53.543000+0000","last_active":"2024-04-25T08:58:54.538399+0000","last_peered":"2024-04-25T08:58:54.538399+0000","last_clean":"2024-04-25T08:58:54.538399+0000","last_became_active":"2024-04-25T08:58:53.542111+0000","last_became_peered":"2024-04-25T08:58:53.542111+0000","last_unstale":"2024-04-25T08:58:54.538399+0000","last_undegraded":"2024-04-25T08:58:54.538399+0000","last_fullsized":"2024-04-25T08:58:54.538399+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:49.452386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.538509+0000","last_change":"2024-04-25T08:58:53.563150+0000","last_active":"2024-04-25T08:58:54.538509+0000","last_peered":"2024-04-25T08:58:54.538509+0000","last_clean":"2024-04-25T08:58:54.538509+0000","last_became_active":"2024-04-25T08:58:53.558866+0000","last_became_peered":"2024-04-25T08:58:53.558866+0000","last_unstale":"2024-04-25T08:58:54.538509+0000","last_undegraded":"2024-04-25T08:58:54.538509+0000","last_fullsized":"2024-04-25T08:58:54.538509+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:47.742025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.539365+0000","last_change":"2024-04-25T08:58:53.541729+0000","last_active":"2024-04-25T08:58:54.539365+0000","last_peered":"2024-04-25T08:58:54.539365+0000","last_clean":"2024-04-25T08:58:54.539365+0000","last_became_active":"2024-04-25T08:58:53.540968+0000","last_became_peered":"2024-04-25T08:58:53.540968+0000","last_unstale":"2024-04-25T08:58:54.539365+0000","last_undegraded":"2024-04-25T08:58:54.539365+0000","last_fullsized":"2024-04-25T08:58:54.539365+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:11.999725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.775194+0000","last_change":"2024-04-25T08:58:53.559090+0000","last_active":"2024-04-25T08:58:54.775194+0000","last_peered":"2024-04-25T08:58:54.775194+0000","last_clean":"2024-04-25T08:58:54.775194+0000","last_became_active":"2024-04-25T08:58:53.558562+0000","last_became_peered":"2024-04-25T08:58:53.558562+0000","last_unstale":"2024-04-25T08:58:54.775194+0000","last_undegraded":"2024-04-25T08:58:54.775194+0000","last_fullsized":"2024-04-25T08:58:54.775194+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:10.736412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.778216+0000","last_change":"2024-04-25T08:58:53.539977+0000","last_active":"2024-04-25T08:58:54.778216+0000","last_peered":"2024-04-25T08:58:54.778216+0000","last_clean":"2024-04-25T08:58:54.778216+0000","last_became_active":"2024-04-25T08:58:53.539235+0000","last_became_peered":"2024-04-25T08:58:53.539235+0000","last_unstale":"2024-04-25T08:58:54.778216+0000","last_undegraded":"2024-04-25T08:58:54.778216+0000","last_fullsized":"2024-04-25T08:58:54.778216+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:11.697774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.540367+0000","last_change":"2024-04-25T08:58:53.565299+0000","last_active":"2024-04-25T08:58:54.540367+0000","last_peered":"2024-04-25T08:58:54.540367+0000","last_clean":"2024-04-25T08:58:54.540367+0000","last_became_active":"2024-04-25T08:58:53.559199+0000","last_became_peered":"2024-04-25T08:58:53.559199+0000","last_unstale":"2024-04-25T08:58:54.540367+0000","last_undegraded":"2024-04-25T08:58:54.540367+0000","last_fullsized":"2024-04-25T08:58:54.540367+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:49.262454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.536532+0000","last_change":"2024-04-25T08:58:53.535959+0000","last_active":"2024-04-25T08:58:54.536532+0000","last_peered":"2024-04-25T08:58:54.536532+0000","last_clean":"2024-04-25T08:58:54.536532+0000","last_became_active":"2024-04-25T08:58:53.531775+0000","last_became_peered":"2024-04-25T08:58:53.531775+0000","last_unstale":"2024-04-25T08:58:54.536532+0000","last_undegraded":"2024-04-25T08:58:54.536532+0000","last_fullsized":"2024-04-25T08:58:54.536532+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:24.588569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.778275+0000","last_change":"2024-04-25T08:58:53.558481+0000","last_active":"2024-04-25T08:58:54.778275+0000","last_peered":"2024-04-25T08:58:54.778275+0000","last_clean":"2024-04-25T08:58:54.778275+0000","last_became_active":"2024-04-25T08:58:53.558215+0000","last_became_peered":"2024-04-25T08:58:53.558215+0000","last_unstale":"2024-04-25T08:58:54.778275+0000","last_undegraded":"2024-04-25T08:58:54.778275+0000","last_fullsized":"2024-04-25T08:58:54.778275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:11.697774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777929+0000","last_change":"2024-04-25T08:58:53.542277+0000","last_active":"2024-04-25T08:58:54.777929+0000","last_peered":"2024-04-25T08:58:54.777929+0000","last_clean":"2024-04-25T08:58:54.777929+0000","last_became_active":"2024-04-25T08:58:53.541475+0000","last_became_peered":"2024-04-25T08:58:53.541475+0000","last_unstale":"2024-04-25T08:58:54.777929+0000","last_undegraded":"2024-04-25T08:58:54.777929+0000","last_fullsized":"2024-04-25T08:58:54.777929+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:05.922206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.538565+0000","last_change":"2024-04-25T08:58:53.559467+0000","last_active":"2024-04-25T08:58:54.538565+0000","last_peered":"2024-04-25T08:58:54.538565+0000","last_clean":"2024-04-25T08:58:54.538565+0000","last_became_active":"2024-04-25T08:58:53.558494+0000","last_became_peered":"2024-04-25T08:58:53.558494+0000","last_unstale":"2024-04-25T08:58:54.538565+0000","last_undegraded":"2024-04-25T08:58:54.538565+0000","last_fullsized":"2024-04-25T08:58:54.538565+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:11.999725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776500+0000","last_change":"2024-04-25T08:58:53.550302+0000","last_active":"2024-04-25T08:58:54.776500+0000","last_peered":"2024-04-25T08:58:54.776500+0000","last_clean":"2024-04-25T08:58:54.776500+0000","last_became_active":"2024-04-25T08:58:53.549546+0000","last_became_peered":"2024-04-25T08:58:53.549546+0000","last_unstale":"2024-04-25T08:58:54.776500+0000","last_undegraded":"2024-04-25T08:58:54.776500+0000","last_fullsized":"2024-04-25T08:58:54.776500+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:21.102659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557216+0000","last_change":"2024-04-25T08:58:53.550343+0000","last_active":"2024-04-25T08:58:54.557216+0000","last_peered":"2024-04-25T08:58:54.557216+0000","last_clean":"2024-04-25T08:58:54.557216+0000","last_became_active":"2024-04-25T08:58:53.550031+0000","last_became_peered":"2024-04-25T08:58:53.550031+0000","last_unstale":"2024-04-25T08:58:54.557216+0000","last_undegraded":"2024-04-25T08:58:54.557216+0000","last_fullsized":"2024-04-25T08:58:54.557216+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:07.476214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557869+0000","last_change":"2024-04-25T08:58:53.560860+0000","last_active":"2024-04-25T08:58:54.557869+0000","last_peered":"2024-04-25T08:58:54.557869+0000","last_clean":"2024-04-25T08:58:54.557869+0000","last_became_active":"2024-04-25T08:58:53.560589+0000","last_became_peered":"2024-04-25T08:58:53.560589+0000","last_unstale":"2024-04-25T08:58:54.557869+0000","last_undegraded":"2024-04-25T08:58:54.557869+0000","last_fullsized":"2024-04-25T08:58:54.557869+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:07.149548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"94'4","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.540563+0000","last_change":"2024-04-25T08:58:52.499317+0000","last_active":"2024-04-25T08:58:54.540563+0000","last_peered":"2024-04-25T08:58:54.540563+0000","last_clean":"2024-04-25T08:58:54.540563+0000","last_became_active":"2024-04-25T08:58:52.498145+0000","last_became_peered":"2024-04-25T08:58:52.498145+0000","last_unstale":"2024-04-25T08:58:54.540563+0000","last_undegraded":"2024-04-25T08:58:54.540563+0000","last_fullsized":"2024-04-25T08:58:54.540563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:12.458893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557423+0000","last_change":"2024-04-25T08:58:53.532360+0000","last_active":"2024-04-25T08:58:54.557423+0000","last_peered":"2024-04-25T08:58:54.557423+0000","last_clean":"2024-04-25T08:58:54.557423+0000","last_became_active":"2024-04-25T08:58:53.531624+0000","last_became_peered":"2024-04-25T08:58:53.531624+0000","last_unstale":"2024-04-25T08:58:54.557423+0000","last_undegraded":"2024-04-25T08:58:54.557423+0000","last_fullsized":"2024-04-25T08:58:54.557423+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:51.288751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.540985+0000","last_change":"2024-04-25T08:58:52.513581+0000","last_active":"2024-04-25T08:58:54.540985+0000","last_peered":"2024-04-25T08:58:54.540985+0000","last_clean":"2024-04-25T08:58:54.540985+0000","last_became_active":"2024-04-25T08:58:52.512774+0000","last_became_peered":"2024-04-25T08:58:52.512774+0000","last_unstale":"2024-04-25T08:58:54.540985+0000","last_undegraded":"2024-04-25T08:58:54.540985+0000","last_fullsized":"2024-04-25T08:58:54.540985+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:53.203027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777537+0000","last_change":"2024-04-25T08:58:53.555821+0000","last_active":"2024-04-25T08:58:54.777537+0000","last_peered":"2024-04-25T08:58:54.777537+0000","last_clean":"2024-04-25T08:58:54.777537+0000","last_became_active":"2024-04-25T08:58:53.555095+0000","last_became_peered":"2024-04-25T08:58:53.555095+0000","last_unstale":"2024-04-25T08:58:54.777537+0000","last_undegraded":"2024-04-25T08:58:54.777537+0000","last_fullsized":"2024-04-25T08:58:54.777537+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:26.501873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777142+0000","last_change":"2024-04-25T08:58:52.508417+0000","last_active":"2024-04-25T08:58:54.777142+0000","last_peered":"2024-04-25T08:58:54.777142+0000","last_clean":"2024-04-25T08:58:54.777142+0000","last_became_active":"2024-04-25T08:58:52.507854+0000","last_became_peered":"2024-04-25T08:58:52.507854+0000","last_unstale":"2024-04-25T08:58:54.777142+0000","last_undegraded":"2024-04-25T08:58:54.777142+0000","last_fullsized":"2024-04-25T08:58:54.777142+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:21.053948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.558143+0000","last_change":"2024-04-25T08:58:53.529885+0000","last_active":"2024-04-25T08:58:54.558143+0000","last_peered":"2024-04-25T08:58:54.558143+0000","last_clean":"2024-04-25T08:58:54.558143+0000","last_became_active":"2024-04-25T08:58:53.529189+0000","last_became_peered":"2024-04-25T08:58:53.529189+0000","last_unstale":"2024-04-25T08:58:54.558143+0000","last_undegraded":"2024-04-25T08:58:54.558143+0000","last_fullsized":"2024-04-25T08:58:54.558143+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:51.624687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.539906+0000","last_change":"2024-04-25T08:58:52.494753+0000","last_active":"2024-04-25T08:58:54.539906+0000","last_peered":"2024-04-25T08:58:54.539906+0000","last_clean":"2024-04-25T08:58:54.539906+0000","last_became_active":"2024-04-25T08:58:52.493972+0000","last_became_peered":"2024-04-25T08:58:52.493972+0000","last_unstale":"2024-04-25T08:58:54.539906+0000","last_undegraded":"2024-04-25T08:58:54.539906+0000","last_fullsized":"2024-04-25T08:58:54.539906+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:06.052032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.775132+0000","last_change":"2024-04-25T08:58:53.557703+0000","last_active":"2024-04-25T08:58:54.775132+0000","last_peered":"2024-04-25T08:58:54.775132+0000","last_clean":"2024-04-25T08:58:54.775132+0000","last_became_active":"2024-04-25T08:58:53.557028+0000","last_became_peered":"2024-04-25T08:58:53.557028+0000","last_unstale":"2024-04-25T08:58:54.775132+0000","last_undegraded":"2024-04-25T08:58:54.775132+0000","last_fullsized":"2024-04-25T08:58:54.775132+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:22.067437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.539385+0000","last_change":"2024-04-25T08:58:52.505294+0000","last_active":"2024-04-25T08:58:54.539385+0000","last_peered":"2024-04-25T08:58:54.539385+0000","last_clean":"2024-04-25T08:58:54.539385+0000","last_became_active":"2024-04-25T08:58:52.504512+0000","last_became_peered":"2024-04-25T08:58:52.504512+0000","last_unstale":"2024-04-25T08:58:54.539385+0000","last_undegraded":"2024-04-25T08:58:54.539385+0000","last_fullsized":"2024-04-25T08:58:54.539385+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:14.712772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.539617+0000","last_change":"2024-04-25T08:58:53.528475+0000","last_active":"2024-04-25T08:58:54.539617+0000","last_peered":"2024-04-25T08:58:54.539617+0000","last_clean":"2024-04-25T08:58:54.539617+0000","last_became_active":"2024-04-25T08:58:53.527605+0000","last_became_peered":"2024-04-25T08:58:53.527605+0000","last_unstale":"2024-04-25T08:58:54.539617+0000","last_undegraded":"2024-04-25T08:58:54.539617+0000","last_fullsized":"2024-04-25T08:58:54.539617+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:00:21.102659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777963+0000","last_change":"2024-04-25T08:58:52.503172+0000","last_active":"2024-04-25T08:58:54.777963+0000","last_peered":"2024-04-25T08:58:54.777963+0000","last_clean":"2024-04-25T08:58:54.777963+0000","last_became_active":"2024-04-25T08:58:52.502702+0000","last_became_peered":"2024-04-25T08:58:52.502702+0000","last_unstale":"2024-04-25T08:58:54.777963+0000","last_undegraded":"2024-04-25T08:58:54.777963+0000","last_fullsized":"2024-04-25T08:58:54.777963+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:30.608887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557120+0000","last_change":"2024-04-25T08:58:53.552811+0000","last_active":"2024-04-25T08:58:54.557120+0000","last_peered":"2024-04-25T08:58:54.557120+0000","last_clean":"2024-04-25T08:58:54.557120+0000","last_became_active":"2024-04-25T08:58:53.552468+0000","last_became_peered":"2024-04-25T08:58:53.552468+0000","last_unstale":"2024-04-25T08:58:54.557120+0000","last_undegraded":"2024-04-25T08:58:54.557120+0000","last_fullsized":"2024-04-25T08:58:54.557120+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:32.053119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776886+0000","last_change":"2024-04-25T08:58:52.501970+0000","last_active":"2024-04-25T08:58:54.776886+0000","last_peered":"2024-04-25T08:58:54.776886+0000","last_clean":"2024-04-25T08:58:54.776886+0000","last_became_active":"2024-04-25T08:58:52.500708+0000","last_became_peered":"2024-04-25T08:58:52.500708+0000","last_unstale":"2024-04-25T08:58:54.776886+0000","last_undegraded":"2024-04-25T08:58:54.776886+0000","last_fullsized":"2024-04-25T08:58:54.776886+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.319442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776232+0000","last_change":"2024-04-25T08:58:53.536434+0000","last_active":"2024-04-25T08:58:54.776232+0000","last_peered":"2024-04-25T08:58:54.776232+0000","last_clean":"2024-04-25T08:58:54.776232+0000","last_became_active":"2024-04-25T08:58:53.536140+0000","last_became_peered":"2024-04-25T08:58:53.536140+0000","last_unstale":"2024-04-25T08:58:54.776232+0000","last_undegraded":"2024-04-25T08:58:54.776232+0000","last_fullsized":"2024-04-25T08:58:54.776232+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:17.456909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776508+0000","last_change":"2024-04-25T08:58:52.506757+0000","last_active":"2024-04-25T08:58:54.776508+0000","last_peered":"2024-04-25T08:58:54.776508+0000","last_clean":"2024-04-25T08:58:54.776508+0000","last_became_active":"2024-04-25T08:58:52.506014+0000","last_became_peered":"2024-04-25T08:58:52.506014+0000","last_unstale":"2024-04-25T08:58:54.776508+0000","last_undegraded":"2024-04-25T08:58:54.776508+0000","last_fullsized":"2024-04-25T08:58:54.776508+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:56.507945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.540115+0000","last_change":"2024-04-25T08:58:53.556984+0000","last_active":"2024-04-25T08:58:54.540115+0000","last_peered":"2024-04-25T08:58:54.540115+0000","last_clean":"2024-04-25T08:58:54.540115+0000","last_became_active":"2024-04-25T08:58:53.556290+0000","last_became_peered":"2024-04-25T08:58:53.556290+0000","last_unstale":"2024-04-25T08:58:54.540115+0000","last_undegraded":"2024-04-25T08:58:54.540115+0000","last_fullsized":"2024-04-25T08:58:54.540115+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:38.994781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.778364+0000","last_change":"2024-04-25T08:58:52.505203+0000","last_active":"2024-04-25T08:58:54.778364+0000","last_peered":"2024-04-25T08:58:54.778364+0000","last_clean":"2024-04-25T08:58:54.778364+0000","last_became_active":"2024-04-25T08:58:52.504503+0000","last_became_peered":"2024-04-25T08:58:52.504503+0000","last_unstale":"2024-04-25T08:58:54.778364+0000","last_undegraded":"2024-04-25T08:58:54.778364+0000","last_fullsized":"2024-04-25T08:58:54.778364+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:09.214520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777270+0000","last_change":"2024-04-25T08:58:53.572499+0000","last_active":"2024-04-25T08:58:54.777270+0000","last_peered":"2024-04-25T08:58:54.777270+0000","last_clean":"2024-04-25T08:58:54.777270+0000","last_became_active":"2024-04-25T08:58:53.572005+0000","last_became_peered":"2024-04-25T08:58:53.572005+0000","last_unstale":"2024-04-25T08:58:54.777270+0000","last_undegraded":"2024-04-25T08:58:54.777270+0000","last_fullsized":"2024-04-25T08:58:54.777270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:54.842417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776553+0000","last_change":"2024-04-25T08:58:52.493504+0000","last_active":"2024-04-25T08:58:54.776553+0000","last_peered":"2024-04-25T08:58:54.776553+0000","last_clean":"2024-04-25T08:58:54.776553+0000","last_became_active":"2024-04-25T08:58:52.493077+0000","last_became_peered":"2024-04-25T08:58:52.493077+0000","last_unstale":"2024-04-25T08:58:54.776553+0000","last_undegraded":"2024-04-25T08:58:54.776553+0000","last_fullsized":"2024-04-25T08:58:54.776553+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:04.908717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776581+0000","last_change":"2024-04-25T08:58:53.548142+0000","last_active":"2024-04-25T08:58:54.776581+0000","last_peered":"2024-04-25T08:58:54.776581+0000","last_clean":"2024-04-25T08:58:54.776581+0000","last_became_active":"2024-04-25T08:58:53.547407+0000","last_became_peered":"2024-04-25T08:58:53.547407+0000","last_unstale":"2024-04-25T08:58:54.776581+0000","last_undegraded":"2024-04-25T08:58:54.776581+0000","last_fullsized":"2024-04-25T08:58:54.776581+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:16.687826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"94'3","reported_seq":17,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.540518+0000","last_change":"2024-04-25T08:58:52.515324+0000","last_active":"2024-04-25T08:58:54.540518+0000","last_peered":"2024-04-25T08:58:54.540518+0000","last_clean":"2024-04-25T08:58:54.540518+0000","last_became_active":"2024-04-25T08:58:52.513958+0000","last_became_peered":"2024-04-25T08:58:52.513958+0000","last_unstale":"2024-04-25T08:58:54.540518+0000","last_undegraded":"2024-04-25T08:58:54.540518+0000","last_fullsized":"2024-04-25T08:58:54.540518+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:02.528462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.775728+0000","last_change":"2024-04-25T08:58:53.544146+0000","last_active":"2024-04-25T08:58:54.775728+0000","last_peered":"2024-04-25T08:58:54.775728+0000","last_clean":"2024-04-25T08:58:54.775728+0000","last_became_active":"2024-04-25T08:58:53.543857+0000","last_became_peered":"2024-04-25T08:58:53.543857+0000","last_unstale":"2024-04-25T08:58:54.775728+0000","last_undegraded":"2024-04-25T08:58:54.775728+0000","last_fullsized":"2024-04-25T08:58:54.775728+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:47.742025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"94'6","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557943+0000","last_change":"2024-04-25T08:58:52.493433+0000","last_active":"2024-04-25T08:58:54.557943+0000","last_peered":"2024-04-25T08:58:54.557943+0000","last_clean":"2024-04-25T08:58:54.557943+0000","last_became_active":"2024-04-25T08:58:52.493061+0000","last_became_peered":"2024-04-25T08:58:52.493061+0000","last_unstale":"2024-04-25T08:58:54.557943+0000","last_undegraded":"2024-04-25T08:58:54.557943+0000","last_fullsized":"2024-04-25T08:58:54.557943+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:30.093202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.775478+0000","last_change":"2024-04-25T08:58:53.543162+0000","last_active":"2024-04-25T08:58:54.775478+0000","last_peered":"2024-04-25T08:58:54.775478+0000","last_clean":"2024-04-25T08:58:54.775478+0000","last_became_active":"2024-04-25T08:58:53.542788+0000","last_became_peered":"2024-04-25T08:58:53.542788+0000","last_unstale":"2024-04-25T08:58:54.775478+0000","last_undegraded":"2024-04-25T08:58:54.775478+0000","last_fullsized":"2024-04-25T08:58:54.775478+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:49.452386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777066+0000","last_change":"2024-04-25T08:58:52.507149+0000","last_active":"2024-04-25T08:58:54.777066+0000","last_peered":"2024-04-25T08:58:54.777066+0000","last_clean":"2024-04-25T08:58:54.777066+0000","last_became_active":"2024-04-25T08:58:52.506632+0000","last_became_peered":"2024-04-25T08:58:52.506632+0000","last_unstale":"2024-04-25T08:58:54.777066+0000","last_undegraded":"2024-04-25T08:58:54.777066+0000","last_fullsized":"2024-04-25T08:58:54.777066+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:46.728536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.538141+0000","last_change":"2024-04-25T08:58:53.534092+0000","last_active":"2024-04-25T08:58:54.538141+0000","last_peered":"2024-04-25T08:58:54.538141+0000","last_clean":"2024-04-25T08:58:54.538141+0000","last_became_active":"2024-04-25T08:58:53.533302+0000","last_became_peered":"2024-04-25T08:58:53.533302+0000","last_unstale":"2024-04-25T08:58:54.538141+0000","last_undegraded":"2024-04-25T08:58:54.538141+0000","last_fullsized":"2024-04-25T08:58:54.538141+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:26:22.067437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557239+0000","last_change":"2024-04-25T08:58:52.508620+0000","last_active":"2024-04-25T08:58:54.557239+0000","last_peered":"2024-04-25T08:58:54.557239+0000","last_clean":"2024-04-25T08:58:54.557239+0000","last_became_active":"2024-04-25T08:58:52.508361+0000","last_became_peered":"2024-04-25T08:58:52.508361+0000","last_unstale":"2024-04-25T08:58:54.557239+0000","last_undegraded":"2024-04-25T08:58:54.557239+0000","last_fullsized":"2024-04-25T08:58:54.557239+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:28.172857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.535991+0000","last_change":"2024-04-25T08:58:53.553609+0000","last_active":"2024-04-25T08:58:54.535991+0000","last_peered":"2024-04-25T08:58:54.535991+0000","last_clean":"2024-04-25T08:58:54.535991+0000","last_became_active":"2024-04-25T08:58:53.553311+0000","last_became_peered":"2024-04-25T08:58:53.553311+0000","last_unstale":"2024-04-25T08:58:54.535991+0000","last_undegraded":"2024-04-25T08:58:54.535991+0000","last_fullsized":"2024-04-25T08:58:54.535991+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:41.038569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.775956+0000","last_change":"2024-04-25T08:58:52.503640+0000","last_active":"2024-04-25T08:58:54.775956+0000","last_peered":"2024-04-25T08:58:54.775956+0000","last_clean":"2024-04-25T08:58:54.775956+0000","last_became_active":"2024-04-25T08:58:52.503271+0000","last_became_peered":"2024-04-25T08:58:52.503271+0000","last_unstale":"2024-04-25T08:58:54.775956+0000","last_undegraded":"2024-04-25T08:58:54.775956+0000","last_fullsized":"2024-04-25T08:58:54.775956+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:54.968958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.536605+0000","last_change":"2024-04-25T08:58:53.558433+0000","last_active":"2024-04-25T08:58:54.536605+0000","last_peered":"2024-04-25T08:58:54.536605+0000","last_clean":"2024-04-25T08:58:54.536605+0000","last_became_active":"2024-04-25T08:58:53.558133+0000","last_became_peered":"2024-04-25T08:58:53.558133+0000","last_unstale":"2024-04-25T08:58:54.536605+0000","last_undegraded":"2024-04-25T08:58:54.536605+0000","last_fullsized":"2024-04-25T08:58:54.536605+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:04.646452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"94'8","reported_seq":22,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.775421+0000","last_change":"2024-04-25T08:58:52.503187+0000","last_active":"2024-04-25T08:58:54.775421+0000","last_peered":"2024-04-25T08:58:54.775421+0000","last_clean":"2024-04-25T08:58:54.775421+0000","last_became_active":"2024-04-25T08:58:52.502662+0000","last_became_peered":"2024-04-25T08:58:52.502662+0000","last_unstale":"2024-04-25T08:58:54.775421+0000","last_undegraded":"2024-04-25T08:58:54.775421+0000","last_fullsized":"2024-04-25T08:58:54.775421+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:50.604902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557654+0000","last_change":"2024-04-25T08:58:53.560951+0000","last_active":"2024-04-25T08:58:54.557654+0000","last_peered":"2024-04-25T08:58:54.557654+0000","last_clean":"2024-04-25T08:58:54.557654+0000","last_became_active":"2024-04-25T08:58:53.560668+0000","last_became_peered":"2024-04-25T08:58:53.560668+0000","last_unstale":"2024-04-25T08:58:54.557654+0000","last_undegraded":"2024-04-25T08:58:54.557654+0000","last_fullsized":"2024-04-25T08:58:54.557654+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:38.316257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.540861+0000","last_change":"2024-04-25T08:58:52.502738+0000","last_active":"2024-04-25T08:58:54.540861+0000","last_peered":"2024-04-25T08:58:54.540861+0000","last_clean":"2024-04-25T08:58:54.540861+0000","last_became_active":"2024-04-25T08:58:52.501965+0000","last_became_peered":"2024-04-25T08:58:52.501965+0000","last_unstale":"2024-04-25T08:58:54.540861+0000","last_undegraded":"2024-04-25T08:58:54.540861+0000","last_fullsized":"2024-04-25T08:58:54.540861+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:25.953382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.540348+0000","last_change":"2024-04-25T08:58:53.538948+0000","last_active":"2024-04-25T08:58:54.540348+0000","last_peered":"2024-04-25T08:58:54.540348+0000","last_clean":"2024-04-25T08:58:54.540348+0000","last_became_active":"2024-04-25T08:58:53.538105+0000","last_became_peered":"2024-04-25T08:58:53.538105+0000","last_unstale":"2024-04-25T08:58:54.540348+0000","last_undegraded":"2024-04-25T08:58:54.540348+0000","last_fullsized":"2024-04-25T08:58:54.540348+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:17.456909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777516+0000","last_change":"2024-04-25T08:58:52.510654+0000","last_active":"2024-04-25T08:58:54.777516+0000","last_peered":"2024-04-25T08:58:54.777516+0000","last_clean":"2024-04-25T08:58:54.777516+0000","last_became_active":"2024-04-25T08:58:52.509735+0000","last_became_peered":"2024-04-25T08:58:52.509735+0000","last_unstale":"2024-04-25T08:58:54.777516+0000","last_undegraded":"2024-04-25T08:58:54.777516+0000","last_fullsized":"2024-04-25T08:58:54.777516+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:10.986236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.540829+0000","last_change":"2024-04-25T08:58:53.555057+0000","last_active":"2024-04-25T08:58:54.540829+0000","last_peered":"2024-04-25T08:58:54.540829+0000","last_clean":"2024-04-25T08:58:54.540829+0000","last_became_active":"2024-04-25T08:58:53.544134+0000","last_became_peered":"2024-04-25T08:58:53.544134+0000","last_unstale":"2024-04-25T08:58:54.540829+0000","last_undegraded":"2024-04-25T08:58:54.540829+0000","last_fullsized":"2024-04-25T08:58:54.540829+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:54.842417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"94'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.540640+0000","last_change":"2024-04-25T08:58:52.514656+0000","last_active":"2024-04-25T08:58:54.540640+0000","last_peered":"2024-04-25T08:58:54.540640+0000","last_clean":"2024-04-25T08:58:54.540640+0000","last_became_active":"2024-04-25T08:58:52.513150+0000","last_became_peered":"2024-04-25T08:58:52.513150+0000","last_unstale":"2024-04-25T08:58:54.540640+0000","last_undegraded":"2024-04-25T08:58:54.540640+0000","last_fullsized":"2024-04-25T08:58:54.540640+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:25.329455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557370+0000","last_change":"2024-04-25T08:58:53.545929+0000","last_active":"2024-04-25T08:58:54.557370+0000","last_peered":"2024-04-25T08:58:54.557370+0000","last_clean":"2024-04-25T08:58:54.557370+0000","last_became_active":"2024-04-25T08:58:53.545605+0000","last_became_peered":"2024-04-25T08:58:53.545605+0000","last_unstale":"2024-04-25T08:58:54.557370+0000","last_undegraded":"2024-04-25T08:58:54.557370+0000","last_fullsized":"2024-04-25T08:58:54.557370+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:12.278738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777525+0000","last_change":"2024-04-25T08:58:52.494154+0000","last_active":"2024-04-25T08:58:54.777525+0000","last_peered":"2024-04-25T08:58:54.777525+0000","last_clean":"2024-04-25T08:58:54.777525+0000","last_became_active":"2024-04-25T08:58:52.493429+0000","last_became_peered":"2024-04-25T08:58:52.493429+0000","last_unstale":"2024-04-25T08:58:54.777525+0000","last_undegraded":"2024-04-25T08:58:54.777525+0000","last_fullsized":"2024-04-25T08:58:54.777525+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:16.136540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776925+0000","last_change":"2024-04-25T08:58:53.554811+0000","last_active":"2024-04-25T08:58:54.776925+0000","last_peered":"2024-04-25T08:58:54.776925+0000","last_clean":"2024-04-25T08:58:54.776925+0000","last_became_active":"2024-04-25T08:58:53.554089+0000","last_became_peered":"2024-04-25T08:58:53.554089+0000","last_unstale":"2024-04-25T08:58:54.776925+0000","last_undegraded":"2024-04-25T08:58:54.776925+0000","last_fullsized":"2024-04-25T08:58:54.776925+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:17.456909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.535897+0000","last_change":"2024-04-25T08:58:52.494299+0000","last_active":"2024-04-25T08:58:54.535897+0000","last_peered":"2024-04-25T08:58:54.535897+0000","last_clean":"2024-04-25T08:58:54.535897+0000","last_became_active":"2024-04-25T08:58:52.493557+0000","last_became_peered":"2024-04-25T08:58:52.493557+0000","last_unstale":"2024-04-25T08:58:54.535897+0000","last_undegraded":"2024-04-25T08:58:54.535897+0000","last_fullsized":"2024-04-25T08:58:54.535897+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:08.011622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777439+0000","last_change":"2024-04-25T08:58:53.562791+0000","last_active":"2024-04-25T08:58:54.777439+0000","last_peered":"2024-04-25T08:58:54.777439+0000","last_clean":"2024-04-25T08:58:54.777439+0000","last_became_active":"2024-04-25T08:58:53.562044+0000","last_became_peered":"2024-04-25T08:58:53.562044+0000","last_unstale":"2024-04-25T08:58:54.777439+0000","last_undegraded":"2024-04-25T08:58:54.777439+0000","last_fullsized":"2024-04-25T08:58:54.777439+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:42:00.599483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777220+0000","last_change":"2024-04-25T08:58:52.500643+0000","last_active":"2024-04-25T08:58:54.777220+0000","last_peered":"2024-04-25T08:58:54.777220+0000","last_clean":"2024-04-25T08:58:54.777220+0000","last_became_active":"2024-04-25T08:58:52.499551+0000","last_became_peered":"2024-04-25T08:58:52.499551+0000","last_unstale":"2024-04-25T08:58:54.777220+0000","last_undegraded":"2024-04-25T08:58:54.777220+0000","last_fullsized":"2024-04-25T08:58:54.777220+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:02.008997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557354+0000","last_change":"2024-04-25T08:58:53.531977+0000","last_active":"2024-04-25T08:58:54.557354+0000","last_peered":"2024-04-25T08:58:54.557354+0000","last_clean":"2024-04-25T08:58:54.557354+0000","last_became_active":"2024-04-25T08:58:53.531595+0000","last_became_peered":"2024-04-25T08:58:53.531595+0000","last_unstale":"2024-04-25T08:58:54.557354+0000","last_undegraded":"2024-04-25T08:58:54.557354+0000","last_fullsized":"2024-04-25T08:58:54.557354+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:59.060164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.535904+0000","last_change":"2024-04-25T08:58:52.501153+0000","last_active":"2024-04-25T08:58:54.535904+0000","last_peered":"2024-04-25T08:58:54.535904+0000","last_clean":"2024-04-25T08:58:54.535904+0000","last_became_active":"2024-04-25T08:58:52.497820+0000","last_became_peered":"2024-04-25T08:58:52.497820+0000","last_unstale":"2024-04-25T08:58:54.535904+0000","last_undegraded":"2024-04-25T08:58:54.535904+0000","last_fullsized":"2024-04-25T08:58:54.535904+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:31.167974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.539533+0000","last_change":"2024-04-25T08:58:53.537001+0000","last_active":"2024-04-25T08:58:54.539533+0000","last_peered":"2024-04-25T08:58:54.539533+0000","last_clean":"2024-04-25T08:58:54.539533+0000","last_became_active":"2024-04-25T08:58:53.536178+0000","last_became_peered":"2024-04-25T08:58:53.536178+0000","last_unstale":"2024-04-25T08:58:54.539533+0000","last_undegraded":"2024-04-25T08:58:54.539533+0000","last_fullsized":"2024-04-25T08:58:54.539533+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:47.742025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"94'2","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777535+0000","last_change":"2024-04-25T08:58:52.511956+0000","last_active":"2024-04-25T08:58:54.777535+0000","last_peered":"2024-04-25T08:58:54.777535+0000","last_clean":"2024-04-25T08:58:54.777535+0000","last_became_active":"2024-04-25T08:58:52.511196+0000","last_became_peered":"2024-04-25T08:58:52.511196+0000","last_unstale":"2024-04-25T08:58:54.777535+0000","last_undegraded":"2024-04-25T08:58:54.777535+0000","last_fullsized":"2024-04-25T08:58:54.777535+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:48.438896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557532+0000","last_change":"2024-04-25T08:58:53.570180+0000","last_active":"2024-04-25T08:58:54.557532+0000","last_peered":"2024-04-25T08:58:54.557532+0000","last_clean":"2024-04-25T08:58:54.557532+0000","last_became_active":"2024-04-25T08:58:53.569811+0000","last_became_peered":"2024-04-25T08:58:53.569811+0000","last_unstale":"2024-04-25T08:58:54.557532+0000","last_undegraded":"2024-04-25T08:58:54.557532+0000","last_fullsized":"2024-04-25T08:58:54.557532+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:04.276526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"95'1","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.541189+0000","last_change":"2024-04-25T08:58:52.512155+0000","last_active":"2024-04-25T08:58:54.541189+0000","last_peered":"2024-04-25T08:58:54.541189+0000","last_clean":"2024-04-25T08:58:54.541189+0000","last_became_active":"2024-04-25T08:58:52.506602+0000","last_became_peered":"2024-04-25T08:58:52.506602+0000","last_unstale":"2024-04-25T08:58:54.541189+0000","last_undegraded":"2024-04-25T08:58:54.541189+0000","last_fullsized":"2024-04-25T08:58:54.541189+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:28.688051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557701+0000","last_change":"2024-04-25T08:58:53.532384+0000","last_active":"2024-04-25T08:58:54.557701+0000","last_peered":"2024-04-25T08:58:54.557701+0000","last_clean":"2024-04-25T08:58:54.557701+0000","last_became_active":"2024-04-25T08:58:53.531777+0000","last_became_peered":"2024-04-25T08:58:53.531777+0000","last_unstale":"2024-04-25T08:58:54.557701+0000","last_undegraded":"2024-04-25T08:58:54.557701+0000","last_fullsized":"2024-04-25T08:58:54.557701+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:16:14.900483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.540801+0000","last_change":"2024-04-25T08:58:52.508006+0000","last_active":"2024-04-25T08:58:54.540801+0000","last_peered":"2024-04-25T08:58:54.540801+0000","last_clean":"2024-04-25T08:58:54.540801+0000","last_became_active":"2024-04-25T08:58:52.507149+0000","last_became_peered":"2024-04-25T08:58:52.507149+0000","last_unstale":"2024-04-25T08:58:54.540801+0000","last_undegraded":"2024-04-25T08:58:54.540801+0000","last_fullsized":"2024-04-25T08:58:54.540801+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:17.559547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.778371+0000","last_change":"2024-04-25T08:58:53.560714+0000","last_active":"2024-04-25T08:58:54.778371+0000","last_peered":"2024-04-25T08:58:54.778371+0000","last_clean":"2024-04-25T08:58:54.778371+0000","last_became_active":"2024-04-25T08:58:53.559982+0000","last_became_peered":"2024-04-25T08:58:53.559982+0000","last_unstale":"2024-04-25T08:58:54.778371+0000","last_undegraded":"2024-04-25T08:58:54.778371+0000","last_fullsized":"2024-04-25T08:58:54.778371+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:34:49.262454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.539923+0000","last_change":"2024-04-25T08:58:52.495139+0000","last_active":"2024-04-25T08:58:54.539923+0000","last_peered":"2024-04-25T08:58:54.539923+0000","last_clean":"2024-04-25T08:58:54.539923+0000","last_became_active":"2024-04-25T08:58:52.494329+0000","last_became_peered":"2024-04-25T08:58:52.494329+0000","last_unstale":"2024-04-25T08:58:54.539923+0000","last_undegraded":"2024-04-25T08:58:54.539923+0000","last_fullsized":"2024-04-25T08:58:54.539923+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:15.105995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.537130+0000","last_change":"2024-04-25T08:58:53.542904+0000","last_active":"2024-04-25T08:58:54.537130+0000","last_peered":"2024-04-25T08:58:54.537130+0000","last_clean":"2024-04-25T08:58:54.537130+0000","last_became_active":"2024-04-25T08:58:53.542587+0000","last_became_peered":"2024-04-25T08:58:53.542587+0000","last_unstale":"2024-04-25T08:58:54.537130+0000","last_undegraded":"2024-04-25T08:58:54.537130+0000","last_fullsized":"2024-04-25T08:58:54.537130+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:02.355099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557648+0000","last_change":"2024-04-25T08:58:52.513900+0000","last_active":"2024-04-25T08:58:54.557648+0000","last_peered":"2024-04-25T08:58:54.557648+0000","last_clean":"2024-04-25T08:58:54.557648+0000","last_became_active":"2024-04-25T08:58:52.513643+0000","last_became_peered":"2024-04-25T08:58:52.513643+0000","last_unstale":"2024-04-25T08:58:54.557648+0000","last_undegraded":"2024-04-25T08:58:54.557648+0000","last_fullsized":"2024-04-25T08:58:54.557648+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:59:55.119564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776377+0000","last_change":"2024-04-25T08:58:53.551590+0000","last_active":"2024-04-25T08:58:54.776377+0000","last_peered":"2024-04-25T08:58:54.776377+0000","last_clean":"2024-04-25T08:58:54.776377+0000","last_became_active":"2024-04-25T08:58:53.551304+0000","last_became_peered":"2024-04-25T08:58:53.551304+0000","last_unstale":"2024-04-25T08:58:54.776377+0000","last_undegraded":"2024-04-25T08:58:54.776377+0000","last_fullsized":"2024-04-25T08:58:54.776377+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:45.241142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777700+0000","last_change":"2024-04-25T08:58:52.493529+0000","last_active":"2024-04-25T08:58:54.777700+0000","last_peered":"2024-04-25T08:58:54.777700+0000","last_clean":"2024-04-25T08:58:54.777700+0000","last_became_active":"2024-04-25T08:58:52.492691+0000","last_became_peered":"2024-04-25T08:58:52.492691+0000","last_unstale":"2024-04-25T08:58:54.777700+0000","last_undegraded":"2024-04-25T08:58:54.777700+0000","last_fullsized":"2024-04-25T08:58:54.777700+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:23.084950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.539160+0000","last_change":"2024-04-25T08:58:53.558432+0000","last_active":"2024-04-25T08:58:54.539160+0000","last_peered":"2024-04-25T08:58:54.539160+0000","last_clean":"2024-04-25T08:58:54.539160+0000","last_became_active":"2024-04-25T08:58:53.554606+0000","last_became_peered":"2024-04-25T08:58:53.554606+0000","last_unstale":"2024-04-25T08:58:54.539160+0000","last_undegraded":"2024-04-25T08:58:54.539160+0000","last_fullsized":"2024-04-25T08:58:54.539160+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:49.452386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.539672+0000","last_change":"2024-04-25T08:58:52.495019+0000","last_active":"2024-04-25T08:58:54.539672+0000","last_peered":"2024-04-25T08:58:54.539672+0000","last_clean":"2024-04-25T08:58:54.539672+0000","last_became_active":"2024-04-25T08:58:52.494237+0000","last_became_peered":"2024-04-25T08:58:52.494237+0000","last_unstale":"2024-04-25T08:58:54.539672+0000","last_undegraded":"2024-04-25T08:58:54.539672+0000","last_fullsized":"2024-04-25T08:58:54.539672+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:16.136540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.557561+0000","last_change":"2024-04-25T08:58:53.544303+0000","last_active":"2024-04-25T08:58:54.557561+0000","last_peered":"2024-04-25T08:58:54.557561+0000","last_clean":"2024-04-25T08:58:54.557561+0000","last_became_active":"2024-04-25T08:58:53.543982+0000","last_became_peered":"2024-04-25T08:58:53.543982+0000","last_unstale":"2024-04-25T08:58:54.557561+0000","last_undegraded":"2024-04-25T08:58:54.557561+0000","last_fullsized":"2024-04-25T08:58:54.557561+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:01.440419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776574+0000","last_change":"2024-04-25T08:58:52.502403+0000","last_active":"2024-04-25T08:58:54.776574+0000","last_peered":"2024-04-25T08:58:54.776574+0000","last_clean":"2024-04-25T08:58:54.776574+0000","last_became_active":"2024-04-25T08:58:52.501501+0000","last_became_peered":"2024-04-25T08:58:52.501501+0000","last_unstale":"2024-04-25T08:58:54.776574+0000","last_undegraded":"2024-04-25T08:58:54.776574+0000","last_fullsized":"2024-04-25T08:58:54.776574+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:22.092021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.777276+0000","last_change":"2024-04-25T08:58:53.545666+0000","last_active":"2024-04-25T08:58:54.777276+0000","last_peered":"2024-04-25T08:58:54.777276+0000","last_clean":"2024-04-25T08:58:54.777276+0000","last_became_active":"2024-04-25T08:58:53.544855+0000","last_became_peered":"2024-04-25T08:58:53.544855+0000","last_unstale":"2024-04-25T08:58:54.777276+0000","last_undegraded":"2024-04-25T08:58:54.777276+0000","last_fullsized":"2024-04-25T08:58:54.777276+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:52.462142+0000","last_clean_scrub_stamp":"2024-04-25T08:58:52.462142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:13.452149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.778276+0000","last_change":"2024-04-25T08:58:52.493037+0000","last_active":"2024-04-25T08:58:54.778276+0000","last_peered":"2024-04-25T08:58:54.778276+0000","last_clean":"2024-04-25T08:58:54.778276+0000","last_became_active":"2024-04-25T08:58:52.492644+0000","last_became_peered":"2024-04-25T08:58:52.492644+0000","last_unstale":"2024-04-25T08:58:54.778276+0000","last_undegraded":"2024-04-25T08:58:54.778276+0000","last_fullsized":"2024-04-25T08:58:54.778276+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:51.448652+0000","last_clean_scrub_stamp":"2024-04-25T08:58:51.448652+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:17.559547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":311,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T08:58:54.776773+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T08:58:54.776773+0000","last_peered":"2024-04-25T08:58:54.776773+0000","last_clean":"2024-04-25T08:58:54.776773+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T08:58:54.776773+0000","last_undegraded":"2024-04-25T08:58:54.776773+0000","last_fullsized":"2024-04-25T08:58:54.776773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1773568,"data_stored":1132273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32348,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339492,"statfs":{"total":96636764160,"available":96603639808,"internally_reserved":0,"allocated":1191936,"data_stored":544065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574932,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1171456,"data_stored":541849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574931,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1179648,"data_stored":542009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574931,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1171456,"data_stored":541849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1171456,"data_stored":541849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1175552,"data_stored":541919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32928,"kb_used_data":1748,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338912,"statfs":{"total":96636764160,"available":96603045888,"internally_reserved":0,"allocated":1789952,"data_stored":1134535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:59:02.432 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:59:02.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:59:02.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.612+0000 7f349f1bc640 1 Processor -- start 2024-04-25T08:59:02.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.612+0000 7f349f1bc640 1 -- start start 2024-04-25T08:59:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f349f1bc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34980cf590 0x7f349815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f349f1bc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f34980ce170 0x7f34980ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f349f1bc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34980ca4c0 con 0x7f34980cf970 2024-04-25T08:59:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f349f1bc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34980ca780 con 0x7f34980ce170 2024-04-25T08:59:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f349f1bc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34980cd440 con 0x7f34980cf590 2024-04-25T08:59:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f3497fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f34980cf970 0x7f34980c99b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38408/0 (socket says 172.21.15.37:38408) 2024-04-25T08:59:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f3497fff640 1 -- 172.21.15.37:0/2546045687 learned_addr learned my addr 172.21.15.37:0/2546045687 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:02.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f349d759640 1 --2- 172.21.15.37:0/2546045687 >> v2:172.21.15.203:3300/0 conn(0x7f34980ce170 0x7f34980ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f349cf58640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34980cf590 0x7f349815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280330066 0 0) 0x7f34980ca4c0 con 0x7f34980cf970 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3480003530 con 0x7f34980cf970 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3917303974 0 0) 0x7f3480003530 con 0x7f34980cf970 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34980cf590 msgr2=0x7f349815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34980cf590 0x7f349815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 >> v2:172.21.15.203:3300/0 conn(0x7f34980ce170 msgr2=0x7f34980ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 --2- 172.21.15.37:0/2546045687 >> v2:172.21.15.203:3300/0 conn(0x7f34980ce170 0x7f34980ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34980cd690 con 0x7f34980cf970 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f348c001680 con 0x7f34980cf970 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 >> v1:172.21.15.37:6789/0 conn(0x7f34980cf970 legacy=0x7f34980c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34980cf590 0x7f3480003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3480004000 0x7f3480004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3480004d50 0x7f34800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34980cd440 con 0x7f3480004000 2024-04-25T08:59:02.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348000b9d0 con 0x7f3480004d50 2024-04-25T08:59:02.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348000bbd0 con 0x7f34980cf590 2024-04-25T08:59:02.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f3497fff640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3480004000 0x7f3480004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f349cf58640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34980cf590 0x7f3480003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f349d759640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3480004d50 0x7f34800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f3497fff640 1 -- 172.21.15.37:0/2546045687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34980cf590 msgr2=0x7f3480003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:02.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f3497fff640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34980cf590 0x7f3480003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f3497fff640 1 -- 172.21.15.37:0/2546045687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3480004d50 msgr2=0x7f34800051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:02.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f3497fff640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3480004d50 0x7f34800051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.616+0000 7f3497fff640 1 -- 172.21.15.37:0/2546045687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3480003530 con 0x7f3480004000 2024-04-25T08:59:02.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f3497fff640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3480004000 0x7f3480004810 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f348c00c160 tx=0x7f348c01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4661139b05194a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:02.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c007c90 con 0x7f3480004000 2024-04-25T08:59:02.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c005b00 con 0x7f3480004000 2024-04-25T08:59:02.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f34977fe640 1 -- 172.21.15.37:0/2546045687 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c007070 con 0x7f3480004000 2024-04-25T08:59:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 -- 172.21.15.37:0/2546045687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3480004000 msgr2=0x7f3480004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 --2- 172.21.15.37:0/2546045687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3480004000 0x7f3480004810 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f348c00c160 tx=0x7f348c01f330 comp rx=0 tx=0).stop 2024-04-25T08:59:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349d759640 1 -- 172.21.15.37:0/2546045687 reap_dead start 2024-04-25T08:59:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 -- 172.21.15.37:0/2546045687 shutdown_connections 2024-04-25T08:59:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 -- 172.21.15.37:0/2546045687 >> 172.21.15.37:0/2546045687 conn(0x7f349806ae60 msgr2=0x7f349806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 -- 172.21.15.37:0/2546045687 shutdown_connections 2024-04-25T08:59:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 -- 172.21.15.37:0/2546045687 wait complete. 2024-04-25T08:59:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 Processor -- start 2024-04-25T08:59:02.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 -- start start 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34981572c0 0x7f34980c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34980aee70 0x7f34980c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34981576a0 0x7f349815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34980cfdb0 con 0x7f34980aee70 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34980cdce0 con 0x7f34981572c0 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f349f1bc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349815bd80 con 0x7f34981576a0 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.620+0000 7f3497fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34980aee70 0x7f34980c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f3497fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34980aee70 0x7f34980c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52602/0 (socket says 172.21.15.37:52602) 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f3497fff640 1 -- 172.21.15.37:0/2703226277 learned_addr learned my addr 172.21.15.37:0/2703226277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f349cf58640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34981572c0 0x7f34980c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f349d759640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34981576a0 0x7f349815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f3497fff640 1 -- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34981576a0 msgr2=0x7f349815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f3497fff640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34981576a0 0x7f349815abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f3497fff640 1 -- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34981572c0 msgr2=0x7f34980c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f3497fff640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34981572c0 0x7f34980c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f3497fff640 1 -- 172.21.15.37:0/2703226277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349815c060 con 0x7f34980aee70 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f349cf58640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34981572c0 0x7f34980c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f349d759640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34981576a0 0x7f349815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f3497fff640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34980aee70 0x7f34980c3f00 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f348c000c10 tx=0x7f3480005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f34957fa640 1 -- 172.21.15.37:0/2703226277 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c0048b0 con 0x7f34980aee70 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f34957fa640 1 -- 172.21.15.37:0/2703226277 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348c004a50 con 0x7f34980aee70 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f349f1bc640 1 -- 172.21.15.37:0/2703226277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349815c340 con 0x7f34980aee70 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f34957fa640 1 -- 172.21.15.37:0/2703226277 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c004e00 con 0x7f34980aee70 2024-04-25T08:59:02.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f349f1bc640 1 -- 172.21.15.37:0/2703226277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34980c2e10 con 0x7f34980aee70 2024-04-25T08:59:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f349f1bc640 1 -- 172.21.15.37:0/2703226277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349804efa0 con 0x7f34980aee70 2024-04-25T08:59:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f34957fa640 1 -- 172.21.15.37:0/2703226277 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f348c0b9030 con 0x7f34980aee70 2024-04-25T08:59:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f34957fa640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3484066f60 0x7f3484069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f34957fa640 1 -- 172.21.15.37:0/2703226277 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f348c1220c0 con 0x7f34980aee70 2024-04-25T08:59:02.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.624+0000 7f349cf58640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3484066f60 0x7f3484069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.628+0000 7f349cf58640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3484066f60 0x7f3484069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3498157df0 tx=0x7f3488008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:02.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.628+0000 7f34957fa640 1 -- 172.21.15.37:0/2703226277 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f348c019040 con 0x7f34980aee70 2024-04-25T08:59:02.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.844+0000 7f349f1bc640 1 -- 172.21.15.37:0/2703226277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f34980b4170 con 0x7f34980aee70 2024-04-25T08:59:02.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.848+0000 7f34957fa640 1 -- 172.21.15.37:0/2703226277 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f348c0f2fd0 con 0x7f34980aee70 2024-04-25T08:59:02.841 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T08:59:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA== 2024-04-25T08:59:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-04-25T08:59:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-04-25T08:59:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:59:02.841 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-04-25T08:59:02.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 -- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3484066f60 msgr2=0x7f3484069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3484066f60 0x7f3484069420 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f3498157df0 tx=0x7f3488008040 comp rx=0 tx=0).stop 2024-04-25T08:59:02.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 -- 172.21.15.37:0/2703226277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34980aee70 msgr2=0x7f34980c3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34980aee70 0x7f34980c3f00 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f348c000c10 tx=0x7f3480005840 comp rx=0 tx=0).stop 2024-04-25T08:59:02.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 -- 172.21.15.37:0/2703226277 shutdown_connections 2024-04-25T08:59:02.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3484066f60 0x7f3484069420 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34981576a0 0x7f349815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34980aee70 0x7f34980c3f00 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 --2- 172.21.15.37:0/2703226277 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34981572c0 0x7f34980c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 -- 172.21.15.37:0/2703226277 >> 172.21.15.37:0/2703226277 conn(0x7f349806ae60 msgr2=0x7f34980ce550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:02.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 -- 172.21.15.37:0/2703226277 shutdown_connections 2024-04-25T08:59:02.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.852+0000 7f349f1bc640 1 -- 172.21.15.37:0/2703226277 wait complete. 2024-04-25T08:59:02.861 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f3873640 1 Processor -- start 2024-04-25T08:59:02.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f3873640 1 -- start start 2024-04-25T08:59:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f3873640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f87ec0ce510 0x7f87ec0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f3873640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f87ec144ca0 0x7f87ec06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f3873640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec0cee30 con 0x7f87ec06ec60 2024-04-25T08:59:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f3873640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec0cc2b0 con 0x7f87ec0ce510 2024-04-25T08:59:02.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f3873640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec06e290 con 0x7f87ec144ca0 2024-04-25T08:59:02.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f160f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f87ec0ce510 0x7f87ec0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f1e10640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f87ec06ec60 0x7f87ec142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38410/0 (socket says 172.21.15.37:38410) 2024-04-25T08:59:02.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f1e10640 1 -- 172.21.15.37:0/1825361605 learned_addr learned my addr 172.21.15.37:0/1825361605 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:02.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f0e0e640 1 --2- 172.21.15.37:0/1825361605 >> v2:172.21.15.203:3300/0 conn(0x7f87ec144ca0 0x7f87ec06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f160f640 1 -- 172.21.15.37:0/1825361605 >> v2:172.21.15.203:3300/0 conn(0x7f87ec144ca0 msgr2=0x7f87ec06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f160f640 1 --2- 172.21.15.37:0/1825361605 >> v2:172.21.15.203:3300/0 conn(0x7f87ec144ca0 0x7f87ec06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f160f640 1 -- 172.21.15.37:0/1825361605 >> v1:172.21.15.37:6789/0 conn(0x7f87ec06ec60 legacy=0x7f87ec142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87f160f640 1 -- 172.21.15.37:0/1825361605 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec0ccc70 con 0x7f87ec0ce510 2024-04-25T08:59:02.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.984+0000 7f87e3fff640 1 -- 172.21.15.37:0/1825361605 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984571877 0 0) 0x7f87ec0cee30 con 0x7f87ec06ec60 2024-04-25T08:59:02.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f160f640 1 --2- 172.21.15.37:0/1825361605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ec0ce510 0x7f87ec0ce8f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f87ec070fe0 tx=0x7f87e801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42cc4d3ddacdbd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:02.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87e3fff640 1 -- 172.21.15.37:0/1825361605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e80b5070 con 0x7f87ec0ce510 2024-04-25T08:59:02.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87e3fff640 1 -- 172.21.15.37:0/1825361605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e80a8970 con 0x7f87ec0ce510 2024-04-25T08:59:02.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87e3fff640 1 -- 172.21.15.37:0/1825361605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e80a8d40 con 0x7f87ec0ce510 2024-04-25T08:59:02.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 -- 172.21.15.37:0/1825361605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ec0ce510 msgr2=0x7f87ec0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 --2- 172.21.15.37:0/1825361605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ec0ce510 0x7f87ec0ce8f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f87ec070fe0 tx=0x7f87e801ec70 comp rx=0 tx=0).stop 2024-04-25T08:59:02.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 -- 172.21.15.37:0/1825361605 shutdown_connections 2024-04-25T08:59:02.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 --2- 172.21.15.37:0/1825361605 >> v2:172.21.15.203:3300/0 conn(0x7f87ec144ca0 0x7f87ec06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 --2- 172.21.15.37:0/1825361605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ec0ce510 0x7f87ec0ce8f0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 -- 172.21.15.37:0/1825361605 >> 172.21.15.37:0/1825361605 conn(0x7f87ec0b11a0 msgr2=0x7f87ec0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:02.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 -- 172.21.15.37:0/1825361605 shutdown_connections 2024-04-25T08:59:02.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 -- 172.21.15.37:0/1825361605 wait complete. 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 Processor -- start 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 -- start start 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87ec06ec60 0x7f87ec0add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ec0ce510 0x7f87ec13bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec144ca0 0x7f87ec0b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0cc900 con 0x7f87ec0ce510 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec06df20 con 0x7f87ec144ca0 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec142d90 con 0x7f87ec06ec60 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f0e0e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ec0ce510 0x7f87ec13bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f0e0e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ec0ce510 0x7f87ec13bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52618/0 (socket says 172.21.15.37:52618) 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f0e0e640 1 -- 172.21.15.37:0/337707547 learned_addr learned my addr 172.21.15.37:0/337707547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f1e10640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec144ca0 0x7f87ec0b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f160f640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87ec06ec60 0x7f87ec0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f0e0e640 1 -- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87ec06ec60 msgr2=0x7f87ec0add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f0e0e640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87ec06ec60 0x7f87ec0add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f0e0e640 1 -- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec144ca0 msgr2=0x7f87ec0b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f0e0e640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec144ca0 0x7f87ec0b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f0e0e640 1 -- 172.21.15.37:0/337707547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec1e0c90 con 0x7f87ec0ce510 2024-04-25T08:59:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f1e10640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec144ca0 0x7f87ec0b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f0e0e640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ec0ce510 0x7f87ec13bf80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00d5d0 tx=0x7f87dc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87e1ffb640 1 -- 172.21.15.37:0/337707547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc004240 con 0x7f87ec0ce510 2024-04-25T08:59:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87e1ffb640 1 -- 172.21.15.37:0/337707547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc0043e0 con 0x7f87ec0ce510 2024-04-25T08:59:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 -- 172.21.15.37:0/337707547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec1e0f30 con 0x7f87ec0ce510 2024-04-25T08:59:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87e1ffb640 1 -- 172.21.15.37:0/337707547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc003840 con 0x7f87ec0ce510 2024-04-25T08:59:02.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.988+0000 7f87f3873640 1 -- 172.21.15.37:0/337707547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec0ad500 con 0x7f87ec0ce510 2024-04-25T08:59:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.992+0000 7f87f3873640 1 -- 172.21.15.37:0/337707547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec0ad2f0 con 0x7f87ec0ce510 2024-04-25T08:59:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.992+0000 7f87e1ffb640 1 -- 172.21.15.37:0/337707547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87dc0039e0 con 0x7f87ec0ce510 2024-04-25T08:59:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.992+0000 7f87e1ffb640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87d0066f60 0x7f87d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.992+0000 7f87e1ffb640 1 -- 172.21.15.37:0/337707547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87dc085480 con 0x7f87ec0ce510 2024-04-25T08:59:02.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.992+0000 7f87f160f640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87d0066f60 0x7f87d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:02.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.992+0000 7f87f160f640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87d0066f60 0x7f87d0069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f87e80b2000 tx=0x7f87e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:02.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:02.996+0000 7f87e1ffb640 1 -- 172.21.15.37:0/337707547 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87dc01c360 con 0x7f87ec0ce510 2024-04-25T08:59:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.188+0000 7f87f3873640 1 -- 172.21.15.37:0/337707547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87ec0b1cc0 con 0x7f87ec0ce510 2024-04-25T08:59:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.188+0000 7f87e1ffb640 1 -- 172.21.15.37:0/337707547 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f87dc056300 con 0x7f87ec0ce510 2024-04-25T08:59:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:59:03.180 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T08:58:55.924579+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7734},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7734":{"gid":7734,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/354285428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":354285428},{"type":"v1","addr":"172.21.15.37:6835","nonce":354285428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:59:03.180 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 60 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 -- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87d0066f60 msgr2=0x7f87d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87d0066f60 0x7f87d0069420 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f87e80b2000 tx=0x7f87e8008040 comp rx=0 tx=0).stop 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 -- 172.21.15.37:0/337707547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ec0ce510 msgr2=0x7f87ec13bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ec0ce510 0x7f87ec13bf80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00d5d0 tx=0x7f87dc00da90 comp rx=0 tx=0).stop 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 -- 172.21.15.37:0/337707547 shutdown_connections 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87d0066f60 0x7f87d0069420 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f87ec144ca0 0x7f87ec0b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ec0ce510 0x7f87ec13bf80 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 --2- 172.21.15.37:0/337707547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f87ec06ec60 0x7f87ec0add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 -- 172.21.15.37:0/337707547 >> 172.21.15.37:0/337707547 conn(0x7f87ec0b11a0 msgr2=0x7f87ec071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 -- 172.21.15.37:0/337707547 shutdown_connections 2024-04-25T08:59:03.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.192+0000 7f87f3873640 1 -- 172.21.15.37:0/337707547 wait complete. 2024-04-25T08:59:03.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:58:52.843744+0000', 'modified': '2024-04-25T08:58:55.924579+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7734}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7734': {'gid': 7734, 'name': 'a', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/354285428', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 354285428}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 354285428}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:59:03.601 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:59:03.602 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:59:03.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f30a0ae3640 1 Processor -- start 2024-04-25T08:59:03.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f30a0ae3640 1 -- start start 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f30a0ae3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 0x7f309c06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f30a0ae3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f309c06b1d0 0x7f309c0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f30a0ae3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f309c0cfd80 0x7f309c15d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f30a0ae3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c161560 con 0x7f309c0cfd80 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f30a0ae3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f309c06b710 con 0x7f309c06b1d0 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f30a0ae3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c1617b0 con 0x7f309c06fc50 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f309ad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f309c0cfd80 0x7f309c15d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f309ad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f309c0cfd80 0x7f309c15d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52634/0 (socket says 172.21.15.37:52634) 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f309ad9d640 1 -- 172.21.15.37:0/3305214825 learned_addr learned my addr 172.21.15.37:0/3305214825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f3099d9b640 1 --2- 172.21.15.37:0/3305214825 >> v2:172.21.15.203:3300/0 conn(0x7f309c06b1d0 0x7f309c0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f309a59c640 1 --2- 172.21.15.37:0/3305214825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 0x7f309c06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f309ad9d640 1 -- 172.21.15.37:0/3305214825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 msgr2=0x7f309c06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f309ad9d640 1 --2- 172.21.15.37:0/3305214825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 0x7f309c06ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f309ad9d640 1 -- 172.21.15.37:0/3305214825 >> v2:172.21.15.203:3300/0 conn(0x7f309c06b1d0 msgr2=0x7f309c0cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f309ad9d640 1 --2- 172.21.15.37:0/3305214825 >> v2:172.21.15.203:3300/0 conn(0x7f309c06b1d0 0x7f309c0cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f309ad9d640 1 -- 172.21.15.37:0/3305214825 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c161a90 con 0x7f309c0cfd80 2024-04-25T08:59:03.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.744+0000 7f309a59c640 1 --2- 172.21.15.37:0/3305214825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 0x7f309c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:03.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f309ad9d640 1 --2- 172.21.15.37:0/3305214825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c0cfd80 0x7f309c15d180 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f309c0cdff0 tx=0x7f309401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c3a87d81c9b04f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:03.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f309959a640 1 -- 172.21.15.37:0/3305214825 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30940a5860 con 0x7f309c0cfd80 2024-04-25T08:59:03.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f309959a640 1 -- 172.21.15.37:0/3305214825 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30940a5a00 con 0x7f309c0cfd80 2024-04-25T08:59:03.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f309959a640 1 -- 172.21.15.37:0/3305214825 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30940a5dd0 con 0x7f309c0cfd80 2024-04-25T08:59:03.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f30a0ae3640 1 -- 172.21.15.37:0/3305214825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c0cfd80 msgr2=0x7f309c15d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f30a0ae3640 1 --2- 172.21.15.37:0/3305214825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c0cfd80 0x7f309c15d180 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f309c0cdff0 tx=0x7f309401c9c0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f30a0ae3640 1 -- 172.21.15.37:0/3305214825 shutdown_connections 2024-04-25T08:59:03.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f30a0ae3640 1 --2- 172.21.15.37:0/3305214825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c0cfd80 0x7f309c15d180 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f30a0ae3640 1 --2- 172.21.15.37:0/3305214825 >> v2:172.21.15.203:3300/0 conn(0x7f309c06b1d0 0x7f309c0cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f30a0ae3640 1 --2- 172.21.15.37:0/3305214825 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 0x7f309c06ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f30a0ae3640 1 -- 172.21.15.37:0/3305214825 >> 172.21.15.37:0/3305214825 conn(0x7f309c099a70 msgr2=0x7f309c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:03.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f30a0ae3640 1 -- 172.21.15.37:0/3305214825 shutdown_connections 2024-04-25T08:59:03.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.748+0000 7f30a0ae3640 1 -- 172.21.15.37:0/3305214825 wait complete. 2024-04-25T08:59:03.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f30a0ae3640 1 Processor -- start 2024-04-25T08:59:03.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f30a0ae3640 1 -- start start 2024-04-25T08:59:03.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f30a0ae3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c06b1d0 0x7f309c15aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f30a0ae3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 0x7f309c0b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f30a0ae3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f309c0cfd80 0x7f309c0b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f30a0ae3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c0701b0 con 0x7f309c06b1d0 2024-04-25T08:59:03.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f30a0ae3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c070030 con 0x7f309c0cfd80 2024-04-25T08:59:03.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f30a0ae3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c162290 con 0x7f309c06fc50 2024-04-25T08:59:03.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f309a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c06b1d0 0x7f309c15aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f309a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c06b1d0 0x7f309c15aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52640/0 (socket says 172.21.15.37:52640) 2024-04-25T08:59:03.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f309a59c640 1 -- 172.21.15.37:0/2698774311 learned_addr learned my addr 172.21.15.37:0/2698774311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:03.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f3099d9b640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 0x7f309c0b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f309ad9d640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f309c0cfd80 0x7f309c0b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f309a59c640 1 -- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 msgr2=0x7f309c0b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f309a59c640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 0x7f309c0b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f309a59c640 1 -- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f309c0cfd80 msgr2=0x7f309c0b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f309a59c640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f309c0cfd80 0x7f309c0b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f309a59c640 1 -- 172.21.15.37:0/2698774311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309c15b2d0 con 0x7f309c06b1d0 2024-04-25T08:59:03.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f3099d9b640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 0x7f309c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:03.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f309ad9d640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f309c0cfd80 0x7f309c0b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:03.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f309a59c640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c06b1d0 0x7f309c15aba0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f308c00c7c0 tx=0x7f308c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:03.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f308affd640 1 -- 172.21.15.37:0/2698774311 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c007a30 con 0x7f309c06b1d0 2024-04-25T08:59:03.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f308affd640 1 -- 172.21.15.37:0/2698774311 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c007bd0 con 0x7f309c06b1d0 2024-04-25T08:59:03.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f308affd640 1 -- 172.21.15.37:0/2698774311 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c0113f0 con 0x7f309c06b1d0 2024-04-25T08:59:03.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f30a0ae3640 1 -- 172.21.15.37:0/2698774311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309c15b5b0 con 0x7f309c06b1d0 2024-04-25T08:59:03.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.752+0000 7f30a0ae3640 1 -- 172.21.15.37:0/2698774311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309c15bb70 con 0x7f309c06b1d0 2024-04-25T08:59:03.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f308affd640 1 -- 172.21.15.37:0/2698774311 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f308c011590 con 0x7f309c06b1d0 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f308affd640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f307c066f90 0x7f307c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f308affd640 1 -- 172.21.15.37:0/2698774311 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f308c085370 con 0x7f309c06b1d0 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f3099d9b640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f307c066f90 0x7f307c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 Processor -- start 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 -- start start 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f3099d9b640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f307c066f90 0x7f307c069450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f309c15d280 tx=0x7f3090008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 0x563eda0bdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0739f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda074310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ed9f5b5f0 con 0x563eda0be240 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ed9ec2da0 con 0x563eda073f30 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ed9e4b150 con 0x563eda0bd920 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f3099d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0739f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f3099d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0739f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52646/0 (socket says 172.21.15.37:52646) 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f3099d9b640 1 -- 172.21.15.37:0/476889196 learned_addr learned my addr 172.21.15.37:0/476889196 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f309a59c640 1 --2- 172.21.15.37:0/476889196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 0x563eda0bdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f309ad9d640 1 --2- 172.21.15.37:0/476889196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f3099d9b640 1 -- 172.21.15.37:0/476889196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 msgr2=0x563eda0bdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f3099d9b640 1 --2- 172.21.15.37:0/476889196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 0x563eda0bdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f3099d9b640 1 -- 172.21.15.37:0/476889196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 msgr2=0x563eda074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f3099d9b640 1 --2- 172.21.15.37:0/476889196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda074310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f3099d9b640 1 -- 172.21.15.37:0/476889196 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ed9e64ca0 con 0x563eda0be240 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f3099d9b640 1 --2- 172.21.15.37:0/476889196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0739f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f3090004380 tx=0x7f3090011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28611c14dd714824 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f307bfff640 1 -- 172.21.15.37:0/476889196 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309000f040 con 0x563eda0be240 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f307bfff640 1 -- 172.21.15.37:0/476889196 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3090003620 con 0x563eda0be240 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f307bfff640 1 -- 172.21.15.37:0/476889196 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30900044f0 con 0x563eda0be240 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 -- 172.21.15.37:0/476889196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 msgr2=0x563eda0739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 --2- 172.21.15.37:0/476889196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0739f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f3090004380 tx=0x7f3090011000 comp rx=0 tx=0).stop 2024-04-25T08:59:03.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 -- 172.21.15.37:0/476889196 shutdown_connections 2024-04-25T08:59:03.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 --2- 172.21.15.37:0/476889196 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 --2- 172.21.15.37:0/476889196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0739f0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 --2- 172.21.15.37:0/476889196 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 0x563eda0bdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 -- 172.21.15.37:0/476889196 >> 172.21.15.37:0/476889196 conn(0x563eda07a2a0 msgr2=0x563eda0722d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:03.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 -- 172.21.15.37:0/476889196 shutdown_connections 2024-04-25T08:59:03.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.756+0000 7f30a3c50000 1 -- 172.21.15.37:0/476889196 wait complete. 2024-04-25T08:59:03.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 -- start start 2024-04-25T08:59:03.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 Processor -- start 2024-04-25T08:59:03.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda0c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0bd920 0x563eda0c5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0be240 0x563eda077aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ed9e7d990 con 0x563eda0bd920 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ed9ebee40 con 0x563eda073f30 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ed9e4e900 con 0x563eda0be240 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 10 client.0 _ll_register_callbacks cb 0x563ed9f761c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f3099d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0bd920 0x563eda0c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f3099d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0bd920 0x563eda0c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52660/0 (socket says 172.21.15.37:52660) 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f3099d9b640 1 -- 172.21.15.37:0/1706095125 learned_addr learned my addr 172.21.15.37:0/1706095125 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f309ad9d640 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0be240 0x563eda077aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f309a59c640 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda0c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f3099d9b640 1 -- 172.21.15.37:0/1706095125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0be240 msgr2=0x563eda077aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f3099d9b640 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0be240 0x563eda077aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f3099d9b640 1 -- 172.21.15.37:0/1706095125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 msgr2=0x563eda0c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f3099d9b640 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda0c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f3099d9b640 1 -- 172.21.15.37:0/1706095125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ed9e7b630 con 0x563eda0bd920 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f309a59c640 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda0c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f3099d9b640 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0bd920 0x563eda0c5230 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f3090018040 tx=0x7f3090004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4920a78059283c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30797fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30797fa640 1 -- 172.21.15.37:0/1706095125 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090013640 con 0x563eda0bd920 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30797fa640 1 -- 172.21.15.37:0/1706095125 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f309001e040 con 0x563eda0bd920 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 -- 172.21.15.37:0/1706095125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3090003620 con 0x563eda0bd920 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30797fa640 1 -- 172.21.15.37:0/1706095125 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3090039050 con 0x563eda0bd920 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 10 client.7899 fetch_fsmap learned FSMap version 60 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 -- 172.21.15.37:0/1706095125 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x563ed9e4b3a0 con 0x563eda0bd920 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30797fa640 1 -- 172.21.15.37:0/1706095125 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3090003620 con 0x563eda0bd920 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30797fa640 1 -- 172.21.15.37:0/1706095125 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f309003c020 con 0x563eda0bd920 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 10 client.7899 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 10 client.7899 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:59:03.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 20 client.7899 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.37:6834/354285428,v1:172.21.15.37:6835/354285428] conn(0x563eda0784b0 0x563eda078890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 4 client.7899 mds_command: new command op to 7734 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 -- 172.21.15.37:0/1706095125 --> [v2:172.21.15.37:6834/354285428,v1:172.21.15.37:6835/354285428] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ed9e5ea50 con 0x563eda0784b0 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f309a59c640 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.37:6834/354285428,v1:172.21.15.37:6835/354285428] conn(0x563eda0784b0 0x563eda078890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f309a59c640 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.37:6834/354285428,v1:172.21.15.37:6835/354285428] conn(0x563eda0784b0 0x563eda078890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f180ad2927d1d2b5 server_cookie=88a7a2324876d02b in_seq=0 out_seq=0 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30797fa640 10 client.7899 ms_handle_connect on v2:172.21.15.37:6834/354285428 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30797fa640 1 -- 172.21.15.37:0/1706095125 <== mds.0 v2:172.21.15.37:6834/354285428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ed9e5ea50 con 0x563eda0784b0 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30797fa640 10 client.7899 handle_command_reply: tid=0 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.760+0000 7f30a3c50000 1 client.7899 shutdown 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30a3c50000 1 -- 172.21.15.37:0/1706095125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0bd920 msgr2=0x563eda0c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30a3c50000 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0bd920 0x563eda0c5230 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f3090018040 tx=0x7f3090004740 comp rx=0 tx=0).stop 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30a3c50000 1 -- 172.21.15.37:0/1706095125 shutdown_connections 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30a3c50000 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.37:6834/354285428,v1:172.21.15.37:6835/354285428] conn(0x563eda0784b0 0x563eda078890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30a3c50000 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0be240 0x563eda077aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30a3c50000 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0bd920 0x563eda0c5230 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30797fa640 0 client.7899 ms_handle_reset on v2:172.21.15.37:6834/354285428 2024-04-25T08:59:03.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30a3c50000 1 --2- 172.21.15.37:0/1706095125 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda0c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30a3c50000 1 -- 172.21.15.37:0/1706095125 >> 172.21.15.37:0/1706095125 conn(0x563eda07a2a0 msgr2=0x563eda0729e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:03.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30a3c50000 1 -- 172.21.15.37:0/1706095125 shutdown_connections 2024-04-25T08:59:03.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30a3c50000 1 -- 172.21.15.37:0/1706095125 wait complete. 2024-04-25T08:59:03.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.764+0000 7f30a3c50000 20 client.7899 trim_cache size 0 max 16384 2024-04-25T08:59:03.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.776+0000 7f30a3c50000 1 Processor -- start 2024-04-25T08:59:03.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.776+0000 7f30a3c50000 1 -- start start 2024-04-25T08:59:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda073f30 0x563eda119040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 0x563eda119580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda0be240 0x563eda119ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c2020e0 con 0x563eda073f30 2024-04-25T08:59:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309c1f7170 con 0x563eda0be240 2024-04-25T08:59:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f309c1f7770 con 0x563eda0bd920 2024-04-25T08:59:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda073f30 0x563eda119040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda073f30 0x563eda119040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52672/0 (socket says 172.21.15.37:52672) 2024-04-25T08:59:03.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309a59c640 1 -- 172.21.15.37:0/2908089119 learned_addr learned my addr 172.21.15.37:0/2908089119 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309a59c640 1 -- 172.21.15.37:0/2908089119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 msgr2=0x563eda119580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309a59c640 1 --2- 172.21.15.37:0/2908089119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 0x563eda119580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309a59c640 1 -- 172.21.15.37:0/2908089119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda0be240 msgr2=0x563eda119ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309a59c640 1 --2- 172.21.15.37:0/2908089119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda0be240 0x563eda119ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309a59c640 1 -- 172.21.15.37:0/2908089119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563eda0be070 con 0x563eda073f30 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309a59c640 1 --2- 172.21.15.37:0/2908089119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda073f30 0x563eda119040 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f308c055a80 tx=0x7f308c0835e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99f8fa0269e9f5b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f307b7fe640 1 -- 172.21.15.37:0/2908089119 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c01e040 con 0x563eda073f30 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f307b7fe640 1 -- 172.21.15.37:0/2908089119 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c053b90 con 0x563eda073f30 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f307b7fe640 1 -- 172.21.15.37:0/2908089119 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308c0850d0 con 0x563eda073f30 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- 172.21.15.37:0/2908089119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda073f30 msgr2=0x563eda119040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 --2- 172.21.15.37:0/2908089119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda073f30 0x563eda119040 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f308c055a80 tx=0x7f308c0835e0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- 172.21.15.37:0/2908089119 shutdown_connections 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 --2- 172.21.15.37:0/2908089119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda0be240 0x563eda119ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 --2- 172.21.15.37:0/2908089119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 0x563eda119580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 --2- 172.21.15.37:0/2908089119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda073f30 0x563eda119040 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- 172.21.15.37:0/2908089119 >> 172.21.15.37:0/2908089119 conn(0x563eda07a2a0 msgr2=0x563eda077220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- 172.21.15.37:0/2908089119 shutdown_connections 2024-04-25T08:59:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- 172.21.15.37:0/2908089119 wait complete. 2024-04-25T08:59:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- start start 2024-04-25T08:59:03.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 Processor -- start 2024-04-25T08:59:03.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 0x563eda0d9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0d9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ed9ec73d0 con 0x563eda0be240 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ed9f7ed50 con 0x563eda073f30 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ed9e5d980 con 0x563eda0bd920 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30a3c50000 10 client.0 _ll_register_callbacks cb 0x7f3090009630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0d9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0d9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52684/0 (socket says 172.21.15.37:52684) 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309ad9d640 1 -- 172.21.15.37:0/2033559811 learned_addr learned my addr 172.21.15.37:0/2033559811 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309a59c640 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309ad9d640 1 -- 172.21.15.37:0/2033559811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 msgr2=0x563eda0d9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309ad9d640 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 0x563eda0d9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309ad9d640 1 -- 172.21.15.37:0/2033559811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 msgr2=0x563eda0ca620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309ad9d640 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda0ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309ad9d640 1 -- 172.21.15.37:0/2033559811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ed9f44600 con 0x563eda0be240 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f309ad9d640 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0d9950 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f309401d260 tx=0x7f3094019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b692c91b6359fcea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.780+0000 7f30797fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30797fa640 1 -- 172.21.15.37:0/2033559811 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30940bc070 con 0x563eda0be240 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30797fa640 1 -- 172.21.15.37:0/2033559811 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f30940c1c40 con 0x563eda0be240 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 -- 172.21.15.37:0/2033559811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f308c053b90 con 0x563eda0be240 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30797fa640 1 -- 172.21.15.37:0/2033559811 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30940c1e40 con 0x563eda0be240 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 10 client.7905 fetch_fsmap learned FSMap version 60 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 -- 172.21.15.37:0/2033559811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=2+}) v3 -- 0x563ed9e579b0 con 0x563eda0be240 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30797fa640 1 -- 172.21.15.37:0/2033559811 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308c053b90 con 0x563eda0be240 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30797fa640 1 -- 172.21.15.37:0/2033559811 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 60) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3094008040 con 0x563eda0be240 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 10 client.7905 fetch_fsmap finished waiting for FSMap version 60 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 10 client.7905 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:59:03.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 20 client.7905 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.37:6834/354285428,v1:172.21.15.37:6835/354285428] conn(0x563eda0d9e90 0x563eda11e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 4 client.7905 mds_command: new command op to 7734 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 -- 172.21.15.37:0/2033559811 --> [v2:172.21.15.37:6834/354285428,v1:172.21.15.37:6835/354285428] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563ed9ed02e0 con 0x563eda0d9e90 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f309a59c640 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.37:6834/354285428,v1:172.21.15.37:6835/354285428] conn(0x563eda0d9e90 0x563eda11e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f309a59c640 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.37:6834/354285428,v1:172.21.15.37:6835/354285428] conn(0x563eda0d9e90 0x563eda11e1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aa9a478c4353c0b server_cookie=a294c3301ee4afd5 in_seq=0 out_seq=0 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30797fa640 10 client.7905 ms_handle_connect on v2:172.21.15.37:6834/354285428 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30797fa640 1 -- 172.21.15.37:0/2033559811 <== mds.0 v2:172.21.15.37:6834/354285428 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x563ed9ed02e0 con 0x563eda0d9e90 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30797fa640 10 client.7905 handle_command_reply: tid=0 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 client.7905 shutdown 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 -- 172.21.15.37:0/2033559811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 msgr2=0x563eda0d9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0d9950 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f309401d260 tx=0x7f3094019760 comp rx=0 tx=0).stop 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 -- 172.21.15.37:0/2033559811 shutdown_connections 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.37:6834/354285428,v1:172.21.15.37:6835/354285428] conn(0x563eda0d9e90 0x563eda11e1b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eda0be240 0x563eda0d9950 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563eda0bd920 0x563eda0d9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 --2- 172.21.15.37:0/2033559811 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563eda073f30 0x563eda0ca620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30797fa640 0 client.7905 ms_handle_reset on v2:172.21.15.37:6834/354285428 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 -- 172.21.15.37:0/2033559811 >> 172.21.15.37:0/2033559811 conn(0x563eda07a2a0 msgr2=0x563eda0c58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 -- 172.21.15.37:0/2033559811 shutdown_connections 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 1 -- 172.21.15.37:0/2033559811 wait complete. 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a3c50000 20 client.7905 trim_cache size 0 max 16384 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7734, 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T08:59:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T08:59:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T08:59:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 10.256318439999999, 2024-04-25T08:59:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 60, 2024-04-25T08:59:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 95, 2024-04-25T08:59:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 95, 2024-04-25T08:59:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 16.337672054999999 2024-04-25T08:59:03.782 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 -- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f307c066f90 msgr2=0x7f307c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f307c066f90 0x7f307c069450 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f309c15d280 tx=0x7f3090008040 comp rx=0 tx=0).stop 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 -- 172.21.15.37:0/2698774311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c06b1d0 msgr2=0x7f309c15aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c06b1d0 0x7f309c15aba0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f308c00c7c0 tx=0x7f308c00cc80 comp rx=0 tx=0).stop 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 -- 172.21.15.37:0/2698774311 shutdown_connections 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f307c066f90 0x7f307c069450 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f309c0cfd80 0x7f309c0b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f309c06fc50 0x7f309c0b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 --2- 172.21.15.37:0/2698774311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f309c06b1d0 0x7f309c15aba0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 -- 172.21.15.37:0/2698774311 >> 172.21.15.37:0/2698774311 conn(0x7f309c099a70 msgr2=0x7f309c0c8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 -- 172.21.15.37:0/2698774311 shutdown_connections 2024-04-25T08:59:03.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.784+0000 7f30a0ae3640 1 -- 172.21.15.37:0/2698774311 wait complete. 2024-04-25T08:59:03.792 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:59:03.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c607c7640 1 Processor -- start 2024-04-25T08:59:03.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c607c7640 1 -- start start 2024-04-25T08:59:03.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c607c7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c580cc050 0x7f8c580cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c607c7640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c580b1240 con 0x7f8c580cd220 2024-04-25T08:59:03.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c607c7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c580cc490 con 0x7f8c580cfe40 2024-04-25T08:59:03.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c607c7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580d02d0 con 0x7f8c580cc050 2024-04-25T08:59:03.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5e563640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8c580cd220 0x7f8c580b2740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38416/0 (socket says 172.21.15.37:38416) 2024-04-25T08:59:03.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5e563640 1 -- 172.21.15.37:0/3190321246 learned_addr learned my addr 172.21.15.37:0/3190321246 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:03.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5dd62640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c580cc050 0x7f8c580cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418109343 0 0) 0x7f8c48001120 con 0x7f8c580cd220 2024-04-25T08:59:03.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c44003530 con 0x7f8c580cd220 2024-04-25T08:59:03.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4164133017 0 0) 0x7f8c44003530 con 0x7f8c580cd220 2024-04-25T08:59:03.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c580cc050 msgr2=0x7f8c580cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c580cc050 0x7f8c580cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 >> v1:172.21.15.203:6789/0 conn(0x7f8c580cfe40 legacy=0x7f8c58099cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c580b2e50 con 0x7f8c580cd220 2024-04-25T08:59:03.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f8c48001850 con 0x7f8c580cd220 2024-04-25T08:59:03.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 >> v1:172.21.15.37:6789/0 conn(0x7f8c580cd220 legacy=0x7f8c580b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c580cc050 0x7f8c44003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c44003fa0 0x7f8c44004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44004d50 0x7f8c440051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580d02d0 con 0x7f8c44003fa0 2024-04-25T08:59:03.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4400bb60 con 0x7f8c44004d50 2024-04-25T08:59:03.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4400bd30 con 0x7f8c580cc050 2024-04-25T08:59:03.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5dd62640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c44003fa0 0x7f8c44004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5ed64640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44004d50 0x7f8c440051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5dd62640 1 -- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c580cc050 msgr2=0x7f8c44003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5e563640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c580cc050 0x7f8c44003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5dd62640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c580cc050 0x7f8c44003a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5dd62640 1 -- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44004d50 msgr2=0x7f8c440051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:03.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5dd62640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44004d50 0x7f8c440051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5dd62640 1 -- 172.21.15.37:0/3190321246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c44003530 con 0x7f8c44003fa0 2024-04-25T08:59:03.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.908+0000 7f8c5ed64640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c44004d50 0x7f8c440051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:03.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c5dd62640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c44003fa0 0x7f8c44004810 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009ae0 tx=0x7f8c5401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930bc68387ff0ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:03.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c540b2070 con 0x7f8c44003fa0 2024-04-25T08:59:03.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c540a5660 con 0x7f8c44003fa0 2024-04-25T08:59:03.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c5d561640 1 -- 172.21.15.37:0/3190321246 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c540096b0 con 0x7f8c44003fa0 2024-04-25T08:59:03.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c607c7640 1 -- 172.21.15.37:0/3190321246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c44003fa0 msgr2=0x7f8c44004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c607c7640 1 --2- 172.21.15.37:0/3190321246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c44003fa0 0x7f8c44004810 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009ae0 tx=0x7f8c5401ca70 comp rx=0 tx=0).stop 2024-04-25T08:59:03.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c5ed64640 1 -- 172.21.15.37:0/3190321246 reap_dead start 2024-04-25T08:59:03.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c607c7640 1 -- 172.21.15.37:0/3190321246 shutdown_connections 2024-04-25T08:59:03.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c607c7640 1 -- 172.21.15.37:0/3190321246 >> 172.21.15.37:0/3190321246 conn(0x7f8c580d15b0 msgr2=0x7f8c580d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:03.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c607c7640 1 -- 172.21.15.37:0/3190321246 shutdown_connections 2024-04-25T08:59:03.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c607c7640 1 -- 172.21.15.37:0/3190321246 wait complete. 2024-04-25T08:59:03.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c607c7640 1 Processor -- start 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.912+0000 7f8c607c7640 1 -- start start 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c607c7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c44003fa0 0x7f8c580c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c607c7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c580c6140 0x7f8c5806d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c607c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5815a1f0 0x7f8c580af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c607c7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5815f920 con 0x7f8c5815a1f0 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c607c7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5815f640 con 0x7f8c580c6140 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c607c7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5815fc00 con 0x7f8c44003fa0 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c5ed64640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5815a1f0 0x7f8c580af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c5ed64640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5815a1f0 0x7f8c580af8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52702/0 (socket says 172.21.15.37:52702) 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c5ed64640 1 -- 172.21.15.37:0/930798107 learned_addr learned my addr 172.21.15.37:0/930798107 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c5e563640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c44003fa0 0x7f8c580c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c5dd62640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c580c6140 0x7f8c5806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c5ed64640 1 -- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c44003fa0 msgr2=0x7f8c580c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c5ed64640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c44003fa0 0x7f8c580c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c5ed64640 1 -- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c580c6140 msgr2=0x7f8c5806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c5ed64640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c580c6140 0x7f8c5806d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c5ed64640 1 -- 172.21.15.37:0/930798107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c580c50a0 con 0x7f8c5815a1f0 2024-04-25T08:59:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c5ed64640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5815a1f0 0x7f8c580af8a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8c500057f0 tx=0x7f8c5000e4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c4effd640 1 -- 172.21.15.37:0/930798107 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c50011040 con 0x7f8c5815a1f0 2024-04-25T08:59:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c4effd640 1 -- 172.21.15.37:0/930798107 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5000ec90 con 0x7f8c5815a1f0 2024-04-25T08:59:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c607c7640 1 -- 172.21.15.37:0/930798107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c580c5380 con 0x7f8c5815a1f0 2024-04-25T08:59:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c4effd640 1 -- 172.21.15.37:0/930798107 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c50006b60 con 0x7f8c5815a1f0 2024-04-25T08:59:03.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c607c7640 1 -- 172.21.15.37:0/930798107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c580c5940 con 0x7f8c5815a1f0 2024-04-25T08:59:03.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c607c7640 1 -- 172.21.15.37:0/930798107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5804efa0 con 0x7f8c5815a1f0 2024-04-25T08:59:03.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c4effd640 1 -- 172.21.15.37:0/930798107 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8c5001c020 con 0x7f8c5815a1f0 2024-04-25T08:59:03.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c4effd640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c3c066f90 0x7f8c3c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:03.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.916+0000 7f8c4effd640 1 -- 172.21.15.37:0/930798107 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c50087ca0 con 0x7f8c5815a1f0 2024-04-25T08:59:03.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.920+0000 7f8c5e563640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c3c066f90 0x7f8c3c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:03.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.920+0000 7f8c5e563640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c3c066f90 0x7f8c3c069450 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8c48001ec0 tx=0x7f8c48001030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:03.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:03.924+0000 7f8c4effd640 1 -- 172.21.15.37:0/930798107 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8c50058b20 con 0x7f8c5815a1f0 2024-04-25T08:59:04.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.120+0000 7f8c607c7640 1 -- 172.21.15.37:0/930798107 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8c5809a9a0 con 0x7f8c5815a1f0 2024-04-25T08:59:04.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.120+0000 7f8c4effd640 1 -- 172.21.15.37:0/930798107 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v60) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8c50058420 con 0x7f8c5815a1f0 2024-04-25T08:59:04.112 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:59:04.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 -- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c3c066f90 msgr2=0x7f8c3c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:04.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c3c066f90 0x7f8c3c069450 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8c48001ec0 tx=0x7f8c48001030 comp rx=0 tx=0).stop 2024-04-25T08:59:04.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 -- 172.21.15.37:0/930798107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5815a1f0 msgr2=0x7f8c580af8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:04.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5815a1f0 0x7f8c580af8a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8c500057f0 tx=0x7f8c5000e4c0 comp rx=0 tx=0).stop 2024-04-25T08:59:04.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 -- 172.21.15.37:0/930798107 shutdown_connections 2024-04-25T08:59:04.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8c3c066f90 0x7f8c3c069450 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:04.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c5815a1f0 0x7f8c580af8a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:04.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8c580c6140 0x7f8c5806d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:04.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 --2- 172.21.15.37:0/930798107 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8c44003fa0 0x7f8c580c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:04.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 -- 172.21.15.37:0/930798107 >> 172.21.15.37:0/930798107 conn(0x7f8c580d15b0 msgr2=0x7f8c580d1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:04.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 -- 172.21.15.37:0/930798107 shutdown_connections 2024-04-25T08:59:04.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:04.124+0000 7f8c607c7640 1 -- 172.21.15.37:0/930798107 wait complete. 2024-04-25T08:59:04.132 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:59:04.132 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:59:04.132 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T08:59:04.132 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:59:04.132 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:04.132 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:59:04.132 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:59:04.132 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:59:04.132 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:59:04.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:04.132 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:04.138 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42097sec preferred_lft 42097sec 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:04.139 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:59:04.140 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:04.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:04.141 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:59:04.191 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:59:04.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:04.193 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T08:59:04.241 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T08:59:04.241 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:59:04.241 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:59:04.241 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:59:04.241 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:59:04.242 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:59:04.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:04.242 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:59:04.242 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:59:04.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:59:04.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:59:04.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:59:04.242 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:59:04.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:04.318 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:59:04.367 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:59:04.367 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:59:04.367 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:59:04.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:04.368 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T08:59:04.424 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:59:04.424 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:04.424 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:04.471 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:59:04.471 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:04.471 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:04.518 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:59:04.577 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:04.623 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:04.624 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:04.625 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:59:04.626 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:04.626 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:04.626 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:04.626 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:04.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:04.627 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:59:04.676 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:59:04.677 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:59:04.719 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:59:04.735 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:59:04.783 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:04.784 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:59:04.785 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:04.785 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:04.785 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:04.785 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:04.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:04.786 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:59:04.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T08:59:04.812+0000 7ff4c27a0580 -1 init, newargv = 0x556a1913df10 newargc=15 2024-04-25T08:59:04.806 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48073]: starting ceph client 2024-04-25T08:59:04.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48073]: starting fuse 2024-04-25T08:59:05.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:59:05.930 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:05.981 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:59:05.982 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:59:05.983 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:05.984 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:05.984 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:05.984 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:05.984 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:05.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:05.985 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:59:06.031 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:59:06.031 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:59:06.031 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:59:06.031 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:59:06.032 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T08:59:06.033 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:59:06.033 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:59:06.033 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:59:06.033 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T08:59:06.033 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T08:59:06.033 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T08:59:06.033 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:59:06.033 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:59:06.033 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T08:59:06.033 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:59:06.159 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.48073.asok 2024-04-25T08:59:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:59:06.171 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48073.asok 2024-04-25T08:59:06.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:06.172 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48073.asok status 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "48073", 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T08:59:06.329 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7917, 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 7917 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3917224285 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:59:06.330 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:59:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3917224285 2024-04-25T08:59:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:59:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.7917 192.168.0.1:0/3917224285", 2024-04-25T08:59:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/3917224285", 2024-04-25T08:59:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T08:59:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 60, 2024-04-25T08:59:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 95, 2024-04-25T08:59:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:59:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T08:59:06.331 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T08:59:06.331 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:59:06.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:06.344 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T08:59:06.345 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:06.351 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T08:59:06.351 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:06.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:06.352 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:06.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:06.406 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-04-25T08:59:06.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:06.468 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-04-25T08:59:06.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:06.520 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-04-25T08:59:06.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:06.572 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-04-25T08:59:06.630 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T08:59:06.630 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:59:06.631 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T08:59:06.631 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:59:06.631 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-04-25T08:59:06.631 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:59:06.746 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627776 2024-04-25T08:59:06.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:59:06.754 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:59:06.754 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:59:06.804 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788ec01640 1 Processor -- start 2024-04-25T08:59:06.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788ec01640 1 -- start start 2024-04-25T08:59:06.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788ec01640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7888071740 0x7f78880acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:06.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788ec01640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f78880b24a0 0x7f78880ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:06.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788ec01640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78880ae860 con 0x7f788815da10 2024-04-25T08:59:06.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788ec01640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788815e210 con 0x7f78880b24a0 2024-04-25T08:59:06.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788ec01640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788815e4f0 con 0x7f7888071740 2024-04-25T08:59:06.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788d19e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f78880b24a0 0x7f78880ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:06.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788d19e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f78880b24a0 0x7f78880ad100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52710/0 (socket says 172.21.15.37:52710) 2024-04-25T08:59:06.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788d19e640 1 -- 172.21.15.37:0/2721663858 learned_addr learned my addr 172.21.15.37:0/2721663858 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:06.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f7887fff640 1 --2- 172.21.15.37:0/2721663858 >> v2:172.21.15.203:3300/0 conn(0x7f7888071740 0x7f78880acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:06.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788d19e640 1 -- 172.21.15.37:0/2721663858 >> v2:172.21.15.203:3300/0 conn(0x7f7888071740 msgr2=0x7f78880acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:06.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788d19e640 1 --2- 172.21.15.37:0/2721663858 >> v2:172.21.15.203:3300/0 conn(0x7f7888071740 0x7f78880acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:06.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788d19e640 1 -- 172.21.15.37:0/2721663858 >> v1:172.21.15.203:6789/0 conn(0x7f788815da10 legacy=0x7f7888071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:06.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.988+0000 7f788d19e640 1 -- 172.21.15.37:0/2721663858 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880ae4c0 con 0x7f78880b24a0 2024-04-25T08:59:06.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788d19e640 1 --2- 172.21.15.37:0/2721663858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78880b24a0 0x7f78880ad100 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f788806c980 tx=0x7f788001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3225588f9c17c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:06.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f78877fe640 1 -- 172.21.15.37:0/2721663858 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78800a5550 con 0x7f78880b24a0 2024-04-25T08:59:06.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f78877fe640 1 -- 172.21.15.37:0/2721663858 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78800a56f0 con 0x7f78880b24a0 2024-04-25T08:59:06.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f78877fe640 1 -- 172.21.15.37:0/2721663858 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78800a5ac0 con 0x7f78880b24a0 2024-04-25T08:59:06.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 -- 172.21.15.37:0/2721663858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78880b24a0 msgr2=0x7f78880ad100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:06.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 --2- 172.21.15.37:0/2721663858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78880b24a0 0x7f78880ad100 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f788806c980 tx=0x7f788001c890 comp rx=0 tx=0).stop 2024-04-25T08:59:06.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 -- 172.21.15.37:0/2721663858 shutdown_connections 2024-04-25T08:59:06.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 --2- 172.21.15.37:0/2721663858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78880b24a0 0x7f78880ad100 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:06.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 --2- 172.21.15.37:0/2721663858 >> v2:172.21.15.203:3300/0 conn(0x7f7888071740 0x7f78880acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:06.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 -- 172.21.15.37:0/2721663858 >> 172.21.15.37:0/2721663858 conn(0x7f78880b11a0 msgr2=0x7f78880b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:06.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 -- 172.21.15.37:0/2721663858 shutdown_connections 2024-04-25T08:59:06.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 -- 172.21.15.37:0/2721663858 wait complete. 2024-04-25T08:59:06.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 Processor -- start 2024-04-25T08:59:06.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 -- start start 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7888071740 0x7f788814e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78880b24a0 0x7f788814e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f788815da10 0x7f78880af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788815ef00 con 0x7f78880b24a0 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78880adc60 con 0x7f7888071740 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.992+0000 7f788ec01640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788816b0a0 con 0x7f788815da10 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f7887fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78880b24a0 0x7f788814e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f7887fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78880b24a0 0x7f788814e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52724/0 (socket says 172.21.15.37:52724) 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f7887fff640 1 -- 172.21.15.37:0/137119402 learned_addr learned my addr 172.21.15.37:0/137119402 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f788c99d640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7888071740 0x7f788814e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f788d19e640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f788815da10 0x7f78880af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f7887fff640 1 -- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f788815da10 msgr2=0x7f78880af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:06.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f7887fff640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f788815da10 0x7f78880af3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:06.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f7887fff640 1 -- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7888071740 msgr2=0x7f788814e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:06.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f7887fff640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7888071740 0x7f788814e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:06.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f7887fff640 1 -- 172.21.15.37:0/137119402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78881505a0 con 0x7f78880b24a0 2024-04-25T08:59:06.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f788d19e640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f788815da10 0x7f78880af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:06.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f7887fff640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78880b24a0 0x7f788814e210 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f787000c610 tx=0x7f787000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:06.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f78857fa640 1 -- 172.21.15.37:0/137119402 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870004230 con 0x7f78880b24a0 2024-04-25T08:59:06.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f78857fa640 1 -- 172.21.15.37:0/137119402 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78700043f0 con 0x7f78880b24a0 2024-04-25T08:59:06.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f788ec01640 1 -- 172.21.15.37:0/137119402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788814efd0 con 0x7f78880b24a0 2024-04-25T08:59:06.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f78857fa640 1 -- 172.21.15.37:0/137119402 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870003860 con 0x7f78880b24a0 2024-04-25T08:59:06.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f788ec01640 1 -- 172.21.15.37:0/137119402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788814f590 con 0x7f78880b24a0 2024-04-25T08:59:06.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f788ec01640 1 -- 172.21.15.37:0/137119402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788804efa0 con 0x7f78880b24a0 2024-04-25T08:59:06.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f78857fa640 1 -- 172.21.15.37:0/137119402 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7870003a00 con 0x7f78880b24a0 2024-04-25T08:59:06.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f78857fa640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f60 0x7f7868069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:06.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f78857fa640 1 -- 172.21.15.37:0/137119402 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7870085590 con 0x7f78880b24a0 2024-04-25T08:59:06.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:06.996+0000 7f788c99d640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f60 0x7f7868069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:06.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.000+0000 7f788c99d640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f60 0x7f7868069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7878004e30 tx=0x7f7878009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:06.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.000+0000 7f78857fa640 1 -- 172.21.15.37:0/137119402 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7870015020 con 0x7f78880b24a0 2024-04-25T08:59:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.160+0000 7f788ec01640 1 -- 172.21.15.37:0/137119402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78880b1dc0 con 0x7f78880b24a0 2024-04-25T08:59:07.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f78857fa640 1 -- 172.21.15.37:0/137119402 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16849 (secure 0 0 0) 0x7f78700563f0 con 0x7f78880b24a0 2024-04-25T08:59:07.155 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:59:07.156 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":95,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:58:54.529461+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:50.512578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:51.848880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:07.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 -- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f60 msgr2=0x7f7868069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f60 0x7f7868069420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7878004e30 tx=0x7f7878009040 comp rx=0 tx=0).stop 2024-04-25T08:59:07.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 -- 172.21.15.37:0/137119402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78880b24a0 msgr2=0x7f788814e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78880b24a0 0x7f788814e210 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f787000c610 tx=0x7f787000cad0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 -- 172.21.15.37:0/137119402 shutdown_connections 2024-04-25T08:59:07.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7868066f60 0x7f7868069420 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f788815da10 0x7f78880af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78880b24a0 0x7f788814e210 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 --2- 172.21.15.37:0/137119402 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7888071740 0x7f788814e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 -- 172.21.15.37:0/137119402 >> 172.21.15.37:0/137119402 conn(0x7f78880b11a0 msgr2=0x7f78880b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:07.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 -- 172.21.15.37:0/137119402 shutdown_connections 2024-04-25T08:59:07.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.164+0000 7f788ec01640 1 -- 172.21.15.37:0/137119402 wait complete. 2024-04-25T08:59:07.175 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:58:50.512578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:58:51.848880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:59:07.175 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:59:07.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41c11d5640 1 Processor -- start 2024-04-25T08:59:07.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41c11d5640 1 -- start start 2024-04-25T08:59:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41c11d5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f41bc0af280 0x7f41bc15df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41c11d5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc06c3b0 0x7f41bc06df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41c11d5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc066c30 con 0x7f41bc0b2c30 2024-04-25T08:59:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41c11d5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc06daa0 con 0x7f41bc0af280 2024-04-25T08:59:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41c11d5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0af660 con 0x7f41bc06c3b0 2024-04-25T08:59:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41ba59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f41bc0b2c30 0x7f41bc06bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38428/0 (socket says 172.21.15.37:38428) 2024-04-25T08:59:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41ba59c640 1 -- 172.21.15.37:0/179403349 learned_addr learned my addr 172.21.15.37:0/179403349 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41bad9d640 1 --2- 172.21.15.37:0/179403349 >> v2:172.21.15.203:3300/0 conn(0x7f41bc0af280 0x7f41bc15df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41bb59e640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc06c3b0 0x7f41bc06df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020818381 0 0) 0x7f41bc066c30 con 0x7f41bc0b2c30 2024-04-25T08:59:07.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41a4003530 con 0x7f41bc0b2c30 2024-04-25T08:59:07.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 600661740 0 0) 0x7f41a4003530 con 0x7f41bc0b2c30 2024-04-25T08:59:07.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc06c3b0 msgr2=0x7f41bc06df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc06c3b0 0x7f41bc06df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 >> v2:172.21.15.203:3300/0 conn(0x7f41bc0af280 msgr2=0x7f41bc15df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 --2- 172.21.15.37:0/179403349 >> v2:172.21.15.203:3300/0 conn(0x7f41bc0af280 0x7f41bc15df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc15e640 con 0x7f41bc0b2c30 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f41b0001680 con 0x7f41bc0b2c30 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 >> v1:172.21.15.37:6789/0 conn(0x7f41bc0b2c30 legacy=0x7f41bc06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a4003a30 0x7f41a4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc06c3b0 0x7f41a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a4004d50 0x7f41a40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0af660 con 0x7f41a4003a30 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a400b9d0 con 0x7f41a4004d50 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a400bb80 con 0x7f41bc06c3b0 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41bad9d640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a4003a30 0x7f41a4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41ba59c640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc06c3b0 0x7f41a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41bb59e640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a4004d50 0x7f41a40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41bad9d640 1 -- 172.21.15.37:0/179403349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc06c3b0 msgr2=0x7f41a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41bad9d640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc06c3b0 0x7f41a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41bad9d640 1 -- 172.21.15.37:0/179403349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a4004d50 msgr2=0x7f41a40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41bad9d640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a4004d50 0x7f41a40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.316+0000 7f41bad9d640 1 -- 172.21.15.37:0/179403349 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a4003530 con 0x7f41a4003a30 2024-04-25T08:59:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.320+0000 7f41bad9d640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a4003a30 0x7f41a4003e50 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009ae0 tx=0x7f41ac01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b01069894f0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.320+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0096b0 con 0x7f41a4003a30 2024-04-25T08:59:07.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.320+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac0a6490 con 0x7f41a4003a30 2024-04-25T08:59:07.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.320+0000 7f41b9d9b640 1 -- 172.21.15.37:0/179403349 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac0a6880 con 0x7f41a4003a30 2024-04-25T08:59:07.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.320+0000 7f41c11d5640 1 -- 172.21.15.37:0/179403349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a4003a30 msgr2=0x7f41a4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.320+0000 7f41c11d5640 1 --2- 172.21.15.37:0/179403349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a4003a30 0x7f41a4003e50 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009ae0 tx=0x7f41ac01cab0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.320+0000 7f41bb59e640 1 -- 172.21.15.37:0/179403349 reap_dead start 2024-04-25T08:59:07.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.320+0000 7f41c11d5640 1 -- 172.21.15.37:0/179403349 shutdown_connections 2024-04-25T08:59:07.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.320+0000 7f41c11d5640 1 -- 172.21.15.37:0/179403349 >> 172.21.15.37:0/179403349 conn(0x7f41bc071810 msgr2=0x7f41bc071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:07.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.320+0000 7f41c11d5640 1 -- 172.21.15.37:0/179403349 shutdown_connections 2024-04-25T08:59:07.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.320+0000 7f41c11d5640 1 -- 172.21.15.37:0/179403349 wait complete. 2024-04-25T08:59:07.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41c11d5640 1 Processor -- start 2024-04-25T08:59:07.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41c11d5640 1 -- start start 2024-04-25T08:59:07.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41c11d5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a4003a30 0x7f41bc160050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41c11d5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41bc15f040 0x7f41bc15cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41c11d5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc166340 0x7f41bc15d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41c11d5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc06eab0 con 0x7f41bc15f040 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41c11d5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc06e930 con 0x7f41a4003a30 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41c11d5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc06ec30 con 0x7f41bc166340 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41bc15f040 0x7f41bc15cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41bc15f040 0x7f41bc15cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52742/0 (socket says 172.21.15.37:52742) 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41ba59c640 1 -- 172.21.15.37:0/3319980479 learned_addr learned my addr 172.21.15.37:0/3319980479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41bad9d640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a4003a30 0x7f41bc160050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41bb59e640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc166340 0x7f41bc15d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41ba59c640 1 -- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc166340 msgr2=0x7f41bc15d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41ba59c640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc166340 0x7f41bc15d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41ba59c640 1 -- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a4003a30 msgr2=0x7f41bc160050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41ba59c640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a4003a30 0x7f41bc160050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41ba59c640 1 -- 172.21.15.37:0/3319980479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc163c80 con 0x7f41bc15f040 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41bad9d640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a4003a30 0x7f41bc160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.324+0000 7f41ba59c640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41bc15f040 0x7f41bc15cf10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f41b0004e40 tx=0x7f41b0007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.328+0000 7f41a37fe640 1 -- 172.21.15.37:0/3319980479 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b000f750 con 0x7f41bc15f040 2024-04-25T08:59:07.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.328+0000 7f41a37fe640 1 -- 172.21.15.37:0/3319980479 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b0018070 con 0x7f41bc15f040 2024-04-25T08:59:07.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.328+0000 7f41c11d5640 1 -- 172.21.15.37:0/3319980479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc161910 con 0x7f41bc15f040 2024-04-25T08:59:07.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.328+0000 7f41a37fe640 1 -- 172.21.15.37:0/3319980479 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b000fb50 con 0x7f41bc15f040 2024-04-25T08:59:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.328+0000 7f41c11d5640 1 -- 172.21.15.37:0/3319980479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc163f00 con 0x7f41bc15f040 2024-04-25T08:59:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.328+0000 7f41c11d5640 1 -- 172.21.15.37:0/3319980479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc161e30 con 0x7f41bc15f040 2024-04-25T08:59:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.328+0000 7f41a37fe640 1 -- 172.21.15.37:0/3319980479 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f41b000fcf0 con 0x7f41bc15f040 2024-04-25T08:59:07.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.328+0000 7f41a37fe640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41a8066f90 0x7f41a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.328+0000 7f41a37fe640 1 -- 172.21.15.37:0/3319980479 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f41b0085000 con 0x7f41bc15f040 2024-04-25T08:59:07.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.328+0000 7f41bad9d640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41a8066f90 0x7f41a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.328+0000 7f41bad9d640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41a8066f90 0x7f41a8069450 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0ad000 tx=0x7f41ac0b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.332+0000 7f41a37fe640 1 -- 172.21.15.37:0/3319980479 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f41bc161e30 con 0x7f41bc15f040 2024-04-25T08:59:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.492+0000 7f41c11d5640 1 -- 172.21.15.37:0/3319980479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41bc04efa0 con 0x7f41bc15f040 2024-04-25T08:59:07.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.492+0000 7f41a37fe640 1 -- 172.21.15.37:0/3319980479 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41bc163c80 con 0x7f41bc15f040 2024-04-25T08:59:07.486 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:59:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 -- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41a8066f90 msgr2=0x7f41a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41a8066f90 0x7f41a8069450 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f41ac0ad000 tx=0x7f41ac0b2040 comp rx=0 tx=0).stop 2024-04-25T08:59:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 -- 172.21.15.37:0/3319980479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41bc15f040 msgr2=0x7f41bc15cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41bc15f040 0x7f41bc15cf10 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f41b0004e40 tx=0x7f41b0007430 comp rx=0 tx=0).stop 2024-04-25T08:59:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 -- 172.21.15.37:0/3319980479 shutdown_connections 2024-04-25T08:59:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f41a8066f90 0x7f41a8069450 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f41bc166340 0x7f41bc15d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41bc15f040 0x7f41bc15cf10 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 --2- 172.21.15.37:0/3319980479 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f41a4003a30 0x7f41bc160050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 -- 172.21.15.37:0/3319980479 >> 172.21.15.37:0/3319980479 conn(0x7f41bc071810 msgr2=0x7f41bc0b0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:07.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 -- 172.21.15.37:0/3319980479 shutdown_connections 2024-04-25T08:59:07.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.496+0000 7f41c11d5640 1 -- 172.21.15.37:0/3319980479 wait complete. 2024-04-25T08:59:07.505 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:59:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.632+0000 7f2b24b99640 1 Processor -- start 2024-04-25T08:59:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.632+0000 7f2b24b99640 1 -- start start 2024-04-25T08:59:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b24b99640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2b200cba90 0x7f2b200cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b24b99640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b20071b70 con 0x7f2b20159fb0 2024-04-25T08:59:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b24b99640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b20071e50 con 0x7f2b20159bd0 2024-04-25T08:59:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b24b99640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b20072780 con 0x7f2b200cba90 2024-04-25T08:59:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b17fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2b20159fb0 0x7f2b200cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38440/0 (socket says 172.21.15.37:38440) 2024-04-25T08:59:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b17fff640 1 -- 172.21.15.37:0/1998294454 learned_addr learned my addr 172.21.15.37:0/1998294454 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:07.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1ed9d640 1 --2- 172.21.15.37:0/1998294454 >> v2:172.21.15.203:3300/0 conn(0x7f2b200cba90 0x7f2b200cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710813164 0 0) 0x7f2b20071b70 con 0x7f2b20159fb0 2024-04-25T08:59:07.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b0c004640 con 0x7f2b20159fb0 2024-04-25T08:59:07.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 780912020 0 0) 0x7f2b0c004640 con 0x7f2b20159fb0 2024-04-25T08:59:07.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 >> v2:172.21.15.203:3300/0 conn(0x7f2b200cba90 msgr2=0x7f2b200cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:07.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 --2- 172.21.15.37:0/1998294454 >> v2:172.21.15.203:3300/0 conn(0x7f2b200cba90 0x7f2b200cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 >> v1:172.21.15.203:6789/0 conn(0x7f2b20159bd0 legacy=0x7f2b200c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b20072130 con 0x7f2b20159fb0 2024-04-25T08:59:07.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f2b08001680 con 0x7f2b20159fb0 2024-04-25T08:59:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 >> v1:172.21.15.37:6789/0 conn(0x7f2b20159fb0 legacy=0x7f2b200cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 --2- 172.21.15.37:0/1998294454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 0x7f2b0c004f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 --2- 172.21.15.37:0/1998294454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b0c005920 0x7f2b0c005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 --2- 172.21.15.37:0/1998294454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b0c0062e0 0x7f2b0c003450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b20072780 con 0x7f2b0c004b40 2024-04-25T08:59:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c003ba0 con 0x7f2b0c005920 2024-04-25T08:59:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0c003da0 con 0x7f2b0c0062e0 2024-04-25T08:59:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1e59c640 1 --2- 172.21.15.37:0/1998294454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 0x7f2b0c004f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b17fff640 1 --2- 172.21.15.37:0/1998294454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b0c005920 0x7f2b0c005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1ed9d640 1 --2- 172.21.15.37:0/1998294454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b0c0062e0 0x7f2b0c003450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1e59c640 1 -- 172.21.15.37:0/1998294454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b0c0062e0 msgr2=0x7f2b0c003450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:07.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1e59c640 1 --2- 172.21.15.37:0/1998294454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b0c0062e0 0x7f2b0c003450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1e59c640 1 -- 172.21.15.37:0/1998294454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b0c005920 msgr2=0x7f2b0c005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:07.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1e59c640 1 --2- 172.21.15.37:0/1998294454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b0c005920 0x7f2b0c005da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1e59c640 1 -- 172.21.15.37:0/1998294454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c004640 con 0x7f2b0c004b40 2024-04-25T08:59:07.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1ed9d640 1 -- 172.21.15.37:0/1998294454 reap_dead start 2024-04-25T08:59:07.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1e59c640 1 --2- 172.21.15.37:0/1998294454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 0x7f2b0c004f60 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2b10001780 tx=0x7f2b1001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b3a6143fecf2e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b100013a0 con 0x7f2b0c004b40 2024-04-25T08:59:07.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b100052f0 con 0x7f2b0c004b40 2024-04-25T08:59:07.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.636+0000 7f2b1dd9b640 1 -- 172.21.15.37:0/1998294454 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10003840 con 0x7f2b0c004b40 2024-04-25T08:59:07.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 -- 172.21.15.37:0/1998294454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 msgr2=0x7f2b0c004f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 --2- 172.21.15.37:0/1998294454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 0x7f2b0c004f60 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f2b10001780 tx=0x7f2b1001c3d0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 -- 172.21.15.37:0/1998294454 shutdown_connections 2024-04-25T08:59:07.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 --2- 172.21.15.37:0/1998294454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 0x7f2b0c004f60 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 -- 172.21.15.37:0/1998294454 >> 172.21.15.37:0/1998294454 conn(0x7f2b2006ae60 msgr2=0x7f2b2006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:07.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 -- 172.21.15.37:0/1998294454 shutdown_connections 2024-04-25T08:59:07.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 -- 172.21.15.37:0/1998294454 wait complete. 2024-04-25T08:59:07.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 Processor -- start 2024-04-25T08:59:07.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 -- start start 2024-04-25T08:59:07.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 0x7f2b200b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b200c2a10 0x7f2b2015b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b2015bda0 0x7f2b200c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200c9f40 con 0x7f2b0c004b40 2024-04-25T08:59:07.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200c9dc0 con 0x7f2b2015bda0 2024-04-25T08:59:07.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200cb2d0 con 0x7f2b200c2a10 2024-04-25T08:59:07.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b1e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 0x7f2b200b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b1e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 0x7f2b200b22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52750/0 (socket says 172.21.15.37:52750) 2024-04-25T08:59:07.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b1e59c640 1 -- 172.21.15.37:0/3100020346 learned_addr learned my addr 172.21.15.37:0/3100020346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:07.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b17fff640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b200c2a10 0x7f2b2015b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b1ed9d640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b2015bda0 0x7f2b200c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b1e59c640 1 -- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b200c2a10 msgr2=0x7f2b2015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b1e59c640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b200c2a10 0x7f2b2015b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b1e59c640 1 -- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b2015bda0 msgr2=0x7f2b200c5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b1e59c640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b2015bda0 0x7f2b200c5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b1e59c640 1 -- 172.21.15.37:0/3100020346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b200b1250 con 0x7f2b0c004b40 2024-04-25T08:59:07.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b1e59c640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 0x7f2b200b22f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f2b1001c200 tx=0x7f2b0c006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b16ffd640 1 -- 172.21.15.37:0/3100020346 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10007a40 con 0x7f2b0c004b40 2024-04-25T08:59:07.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b16ffd640 1 -- 172.21.15.37:0/3100020346 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b100ba070 con 0x7f2b0c004b40 2024-04-25T08:59:07.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 -- 172.21.15.37:0/3100020346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b200b1530 con 0x7f2b0c004b40 2024-04-25T08:59:07.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b16ffd640 1 -- 172.21.15.37:0/3100020346 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b100054a0 con 0x7f2b0c004b40 2024-04-25T08:59:07.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.640+0000 7f2b24b99640 1 -- 172.21.15.37:0/3100020346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b200b1af0 con 0x7f2b0c004b40 2024-04-25T08:59:07.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.644+0000 7f2b24b99640 1 -- 172.21.15.37:0/3100020346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b200b2830 con 0x7f2b0c004b40 2024-04-25T08:59:07.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.644+0000 7f2b16ffd640 1 -- 172.21.15.37:0/3100020346 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b100b4070 con 0x7f2b0c004b40 2024-04-25T08:59:07.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.644+0000 7f2b16ffd640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b00066f00 0x7f2b000693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.644+0000 7f2b16ffd640 1 -- 172.21.15.37:0/3100020346 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2b1011ecc0 con 0x7f2b0c004b40 2024-04-25T08:59:07.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.644+0000 7f2b17fff640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b00066f00 0x7f2b000693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.644+0000 7f2b17fff640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b00066f00 0x7f2b000693c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2b08001060 tx=0x7f2b08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.648+0000 7f2b16ffd640 1 -- 172.21.15.37:0/3100020346 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b200b2830 con 0x7f2b0c004b40 2024-04-25T08:59:07.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.804+0000 7f2b24b99640 1 -- 172.21.15.37:0/3100020346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b2004efa0 con 0x7f2b0c004b40 2024-04-25T08:59:07.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b16ffd640 1 -- 172.21.15.37:0/3100020346 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b10016110 con 0x7f2b0c004b40 2024-04-25T08:59:07.798 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:59:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 -- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b00066f00 msgr2=0x7f2b000693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b00066f00 0x7f2b000693c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2b08001060 tx=0x7f2b08009040 comp rx=0 tx=0).stop 2024-04-25T08:59:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 -- 172.21.15.37:0/3100020346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 msgr2=0x7f2b200b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 0x7f2b200b22f0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f2b1001c200 tx=0x7f2b0c006950 comp rx=0 tx=0).stop 2024-04-25T08:59:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 -- 172.21.15.37:0/3100020346 shutdown_connections 2024-04-25T08:59:07.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b00066f00 0x7f2b000693c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b2015bda0 0x7f2b200c5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b200c2a10 0x7f2b2015b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 --2- 172.21.15.37:0/3100020346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b0c004b40 0x7f2b200b22f0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 -- 172.21.15.37:0/3100020346 >> 172.21.15.37:0/3100020346 conn(0x7f2b2006ae60 msgr2=0x7f2b20159190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 -- 172.21.15.37:0/3100020346 shutdown_connections 2024-04-25T08:59:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.808+0000 7f2b24b99640 1 -- 172.21.15.37:0/3100020346 wait complete. 2024-04-25T08:59:07.819 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:59:07.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034f279640 1 Processor -- start 2024-04-25T08:59:07.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034f279640 1 -- start start 2024-04-25T08:59:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034f279640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f034809a480 0x7f03480cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034f279640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f03480cfb80 0x7f03480d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034f279640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f034815bd30 con 0x7f034815b950 2024-04-25T08:59:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034f279640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03480cc0d0 con 0x7f03480cfb80 2024-04-25T08:59:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034f279640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f034809ab10 con 0x7f034809a480 2024-04-25T08:59:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034d015640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f034815b950 0x7f03480cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38452/0 (socket says 172.21.15.37:38452) 2024-04-25T08:59:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034d015640 1 -- 172.21.15.37:0/2744349465 learned_addr learned my addr 172.21.15.37:0/2744349465 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:07.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034d816640 1 --2- 172.21.15.37:0/2744349465 >> v2:172.21.15.37:3300/0 conn(0x7f03480cfb80 0x7f03480d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034c814640 1 --2- 172.21.15.37:0/2744349465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f034809a480 0x7f03480cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f0347fff640 1 -- 172.21.15.37:0/2744349465 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243120221 0 0) 0x7f034815bd30 con 0x7f034815b950 2024-04-25T08:59:07.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f0347fff640 1 -- 172.21.15.37:0/2744349465 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0330003530 con 0x7f034815b950 2024-04-25T08:59:07.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034d816640 1 -- 172.21.15.37:0/2744349465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f034809a480 msgr2=0x7f03480cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:07.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034d816640 1 --2- 172.21.15.37:0/2744349465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f034809a480 0x7f03480cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034d816640 1 -- 172.21.15.37:0/2744349465 >> v1:172.21.15.37:6789/0 conn(0x7f034815b950 legacy=0x7f03480cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.952+0000 7f034d816640 1 -- 172.21.15.37:0/2744349465 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03480cc370 con 0x7f03480cfb80 2024-04-25T08:59:07.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034d816640 1 --2- 172.21.15.37:0/2744349465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03480cfb80 0x7f03480d0010 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0338009d90 tx=0x7f033801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f915b0c719bec028 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f0347fff640 1 -- 172.21.15.37:0/2744349465 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380b2070 con 0x7f03480cfb80 2024-04-25T08:59:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f0347fff640 1 -- 172.21.15.37:0/2744349465 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03380a5830 con 0x7f03480cfb80 2024-04-25T08:59:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f0347fff640 1 -- 172.21.15.37:0/2744349465 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380a5c00 con 0x7f03480cfb80 2024-04-25T08:59:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 -- 172.21.15.37:0/2744349465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03480cfb80 msgr2=0x7f03480d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 --2- 172.21.15.37:0/2744349465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03480cfb80 0x7f03480d0010 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f0338009d90 tx=0x7f033801c970 comp rx=0 tx=0).stop 2024-04-25T08:59:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 -- 172.21.15.37:0/2744349465 shutdown_connections 2024-04-25T08:59:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 --2- 172.21.15.37:0/2744349465 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03480cfb80 0x7f03480d0010 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 --2- 172.21.15.37:0/2744349465 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f034809a480 0x7f03480cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 -- 172.21.15.37:0/2744349465 >> 172.21.15.37:0/2744349465 conn(0x7f03480b30b0 msgr2=0x7f03480b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 -- 172.21.15.37:0/2744349465 shutdown_connections 2024-04-25T08:59:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 -- 172.21.15.37:0/2744349465 wait complete. 2024-04-25T08:59:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 Processor -- start 2024-04-25T08:59:07.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 -- start start 2024-04-25T08:59:07.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f034809a480 0x7f03480af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03480cfb80 0x7f034815f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f034815b950 0x7f034815bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034806fc20 con 0x7f03480cfb80 2024-04-25T08:59:07.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034806faa0 con 0x7f034809a480 2024-04-25T08:59:07.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034f279640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0348168770 con 0x7f034815b950 2024-04-25T08:59:07.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034c814640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03480cfb80 0x7f034815f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034c814640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03480cfb80 0x7f034815f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52768/0 (socket says 172.21.15.37:52768) 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034c814640 1 -- 172.21.15.37:0/3018145505 learned_addr learned my addr 172.21.15.37:0/3018145505 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034d015640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f034809a480 0x7f03480af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.956+0000 7f034d816640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f034815b950 0x7f034815bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034c814640 1 -- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f034815b950 msgr2=0x7f034815bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034c814640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f034815b950 0x7f034815bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034c814640 1 -- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f034809a480 msgr2=0x7f03480af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034c814640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f034809a480 0x7f03480af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034c814640 1 -- 172.21.15.37:0/3018145505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034815c270 con 0x7f03480cfb80 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034d015640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f034809a480 0x7f03480af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034c814640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03480cfb80 0x7f034815f690 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f033c00d620 tx=0x7f033c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f0345ffb640 1 -- 172.21.15.37:0/3018145505 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c00b620 con 0x7f03480cfb80 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f0345ffb640 1 -- 172.21.15.37:0/3018145505 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033c019070 con 0x7f03480cfb80 2024-04-25T08:59:07.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034f279640 1 -- 172.21.15.37:0/3018145505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03480afd30 con 0x7f03480cfb80 2024-04-25T08:59:07.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f0345ffb640 1 -- 172.21.15.37:0/3018145505 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033c00b9e0 con 0x7f03480cfb80 2024-04-25T08:59:07.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034f279640 1 -- 172.21.15.37:0/3018145505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03480b02f0 con 0x7f03480cfb80 2024-04-25T08:59:07.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034f279640 1 -- 172.21.15.37:0/3018145505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f034804efa0 con 0x7f03480cfb80 2024-04-25T08:59:07.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f0345ffb640 1 -- 172.21.15.37:0/3018145505 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f033c00bb80 con 0x7f03480cfb80 2024-04-25T08:59:07.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f0345ffb640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0334066f90 0x7f0334069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:07.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f0345ffb640 1 -- 172.21.15.37:0/3018145505 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f033c085540 con 0x7f03480cfb80 2024-04-25T08:59:07.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034d015640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0334066f90 0x7f0334069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:07.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.960+0000 7f034d015640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0334066f90 0x7f0334069450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0340005890 tx=0x7f0340009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:07.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:07.964+0000 7f0345ffb640 1 -- 172.21.15.37:0/3018145505 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f033c056450 con 0x7f03480cfb80 2024-04-25T08:59:08.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.124+0000 7f034f279640 1 -- 172.21.15.37:0/3018145505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03480d13a0 con 0x7f03480cfb80 2024-04-25T08:59:08.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.124+0000 7f0345ffb640 1 -- 172.21.15.37:0/3018145505 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f033c055d50 con 0x7f03480cfb80 2024-04-25T08:59:08.117 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:59:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 -- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0334066f90 msgr2=0x7f0334069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0334066f90 0x7f0334069450 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0340005890 tx=0x7f0340009040 comp rx=0 tx=0).stop 2024-04-25T08:59:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 -- 172.21.15.37:0/3018145505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03480cfb80 msgr2=0x7f034815f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03480cfb80 0x7f034815f690 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f033c00d620 tx=0x7f033c00dae0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 -- 172.21.15.37:0/3018145505 shutdown_connections 2024-04-25T08:59:08.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0334066f90 0x7f0334069450 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f034815b950 0x7f034815bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f03480cfb80 0x7f034815f690 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 --2- 172.21.15.37:0/3018145505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f034809a480 0x7f03480af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 -- 172.21.15.37:0/3018145505 >> 172.21.15.37:0/3018145505 conn(0x7f03480b30b0 msgr2=0x7f0348073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 -- 172.21.15.37:0/3018145505 shutdown_connections 2024-04-25T08:59:08.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.128+0000 7f034f279640 1 -- 172.21.15.37:0/3018145505 wait complete. 2024-04-25T08:59:08.137 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:59:08.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.272+0000 7fb28b78b640 1 Processor -- start 2024-04-25T08:59:08.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.272+0000 7fb28b78b640 1 -- start start 2024-04-25T08:59:08.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb28b78b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb28415ecc0 0x7fb28406e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb28b78b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2840b2090 con 0x7fb28415a700 2024-04-25T08:59:08.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb28b78b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2840b43e0 con 0x7fb2840b3f50 2024-04-25T08:59:08.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb28b78b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2840b46c0 con 0x7fb28415ecc0 2024-04-25T08:59:08.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb289d28640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb28415ecc0 0x7fb28406e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb289527640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb28415a700 0x7fb2840cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38466/0 (socket says 172.21.15.37:38466) 2024-04-25T08:59:08.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb289d28640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb28415ecc0 0x7fb28406e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52772/0 (socket says 172.21.15.37:52772) 2024-04-25T08:59:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb289527640 1 -- 172.21.15.37:0/1718944147 learned_addr learned my addr 172.21.15.37:0/1718944147 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb289d28640 1 -- 172.21.15.37:0/1718944147 >> v1:172.21.15.203:6789/0 conn(0x7fb2840b3f50 legacy=0x7fb2840b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb289d28640 1 -- 172.21.15.37:0/1718944147 >> v1:172.21.15.37:6789/0 conn(0x7fb28415a700 legacy=0x7fb2840cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb289d28640 1 -- 172.21.15.37:0/1718944147 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840cd6f0 con 0x7fb28415ecc0 2024-04-25T08:59:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb27bfff640 1 -- 172.21.15.37:0/1718944147 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098010823 0 0) 0x7fb2840b2090 con 0x7fb28415a700 2024-04-25T08:59:08.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb289d28640 1 --2- 172.21.15.37:0/1718944147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb28415ecc0 0x7fb28406e1a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009d90 tx=0x7fb27c01c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddd71f073536e7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb27bfff640 1 -- 172.21.15.37:0/1718944147 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0b2070 con 0x7fb28415ecc0 2024-04-25T08:59:08.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb27bfff640 1 -- 172.21.15.37:0/1718944147 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c0a57f0 con 0x7fb28415ecc0 2024-04-25T08:59:08.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb27bfff640 1 -- 172.21.15.37:0/1718944147 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0a5bc0 con 0x7fb28415ecc0 2024-04-25T08:59:08.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb28b78b640 1 -- 172.21.15.37:0/1718944147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb28415ecc0 msgr2=0x7fb28406e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb28b78b640 1 --2- 172.21.15.37:0/1718944147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb28415ecc0 0x7fb28406e1a0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fb27c009d90 tx=0x7fb27c01c820 comp rx=0 tx=0).stop 2024-04-25T08:59:08.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb28b78b640 1 -- 172.21.15.37:0/1718944147 shutdown_connections 2024-04-25T08:59:08.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb28b78b640 1 --2- 172.21.15.37:0/1718944147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb28415ecc0 0x7fb28406e1a0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb28b78b640 1 -- 172.21.15.37:0/1718944147 >> 172.21.15.37:0/1718944147 conn(0x7fb2840b11a0 msgr2=0x7fb2840b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb28b78b640 1 -- 172.21.15.37:0/1718944147 shutdown_connections 2024-04-25T08:59:08.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.276+0000 7fb28b78b640 1 -- 172.21.15.37:0/1718944147 wait complete. 2024-04-25T08:59:08.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb28b78b640 1 Processor -- start 2024-04-25T08:59:08.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb28b78b640 1 -- start start 2024-04-25T08:59:08.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb28b78b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28415a700 0x7fb284140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb28b78b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28415ecc0 0x7fb28415cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb28b78b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb28415d130 0x7fb2841f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb28b78b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840cd310 con 0x7fb28415d130 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb28b78b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840ce600 con 0x7fb28415a700 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb28b78b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840cd490 con 0x7fb28415ecc0 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb289d28640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb28415d130 0x7fb2841f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb289d28640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb28415d130 0x7fb2841f6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52774/0 (socket says 172.21.15.37:52774) 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb289d28640 1 -- 172.21.15.37:0/3115412613 learned_addr learned my addr 172.21.15.37:0/3115412613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb289527640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28415a700 0x7fb284140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb289d28640 1 -- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28415ecc0 msgr2=0x7fb28415cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb289d28640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28415ecc0 0x7fb28415cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb289d28640 1 -- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28415a700 msgr2=0x7fb284140f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb289d28640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28415a700 0x7fb284140f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb289d28640 1 -- 172.21.15.37:0/3115412613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28415d5f0 con 0x7fb28415d130 2024-04-25T08:59:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb289527640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28415a700 0x7fb284140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb289d28640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb28415d130 0x7fb2841f6db0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0b0040 tx=0x7fb27c0195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb279ffb640 1 -- 172.21.15.37:0/3115412613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0bd030 con 0x7fb28415d130 2024-04-25T08:59:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb279ffb640 1 -- 172.21.15.37:0/3115412613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c0b8530 con 0x7fb28415d130 2024-04-25T08:59:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb28b78b640 1 -- 172.21.15.37:0/3115412613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28415d8d0 con 0x7fb28415d130 2024-04-25T08:59:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb279ffb640 1 -- 172.21.15.37:0/3115412613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c0b2040 con 0x7fb28415d130 2024-04-25T08:59:08.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb28b78b640 1 -- 172.21.15.37:0/3115412613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2841f6890 con 0x7fb28415d130 2024-04-25T08:59:08.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.280+0000 7fb28b78b640 1 -- 172.21.15.37:0/3115412613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28406f260 con 0x7fb28415d130 2024-04-25T08:59:08.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.284+0000 7fb279ffb640 1 -- 172.21.15.37:0/3115412613 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb27c0c0070 con 0x7fb28415d130 2024-04-25T08:59:08.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.284+0000 7fb279ffb640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb268066f60 0x7fb268069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.284+0000 7fb279ffb640 1 -- 172.21.15.37:0/3115412613 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb27c11ff60 con 0x7fb28415d130 2024-04-25T08:59:08.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.284+0000 7fb289527640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb268066f60 0x7fb268069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.284+0000 7fb289527640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb268066f60 0x7fb268069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb280000f70 tx=0x7fb280009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.288+0000 7fb279ffb640 1 -- 172.21.15.37:0/3115412613 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb27c0c7070 con 0x7fb28415d130 2024-04-25T08:59:08.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.460+0000 7fb28b78b640 1 -- 172.21.15.37:0/3115412613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb28404ef80 con 0x7fb28415d130 2024-04-25T08:59:08.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.460+0000 7fb279ffb640 1 -- 172.21.15.37:0/3115412613 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v95) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb27c016560 con 0x7fb28415d130 2024-04-25T08:59:08.452 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:59:08.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 -- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb268066f60 msgr2=0x7fb268069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb268066f60 0x7fb268069420 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb280000f70 tx=0x7fb280009040 comp rx=0 tx=0).stop 2024-04-25T08:59:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 -- 172.21.15.37:0/3115412613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb28415d130 msgr2=0x7fb2841f6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb28415d130 0x7fb2841f6db0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fb27c0b0040 tx=0x7fb27c0195c0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 -- 172.21.15.37:0/3115412613 shutdown_connections 2024-04-25T08:59:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb268066f60 0x7fb268069420 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb28415d130 0x7fb2841f6db0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb28415ecc0 0x7fb28415cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 --2- 172.21.15.37:0/3115412613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb28415a700 0x7fb284140f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 -- 172.21.15.37:0/3115412613 >> 172.21.15.37:0/3115412613 conn(0x7fb2840b11a0 msgr2=0x7fb284155ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 -- 172.21.15.37:0/3115412613 shutdown_connections 2024-04-25T08:59:08.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.464+0000 7fb28b78b640 1 -- 172.21.15.37:0/3115412613 wait complete. 2024-04-25T08:59:08.471 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T08:59:08.471 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:59:08.477 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:59:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e7f01640 1 Processor -- start 2024-04-25T08:59:08.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e7f01640 1 -- start start 2024-04-25T08:59:08.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e7f01640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f18e00b5270 0x7f18e00b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e7f01640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00b5650 0x7f18e00d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e7f01640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18e006bff0 con 0x7f18e00d4440 2024-04-25T08:59:08.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e7f01640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18e006cc00 con 0x7f18e00b5270 2024-04-25T08:59:08.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e7f01640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e006c2d0 con 0x7f18e00b5650 2024-04-25T08:59:08.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e5c9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f18e00b5270 0x7f18e00b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e549c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00b5650 0x7f18e00d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e5c9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f18e00b5270 0x7f18e00b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:49184/0 (socket says 172.21.15.37:49184) 2024-04-25T08:59:08.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e649e640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f18e00d4440 0x7f18e006d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:44494/0 (socket says 172.21.15.37:44494) 2024-04-25T08:59:08.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e5c9d640 1 -- 172.21.15.37:0/3961737780 learned_addr learned my addr 172.21.15.37:0/3961737780 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:08.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e5c9d640 1 -- 172.21.15.37:0/3961737780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00b5650 msgr2=0x7f18e00d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:08.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e5c9d640 1 --2- 172.21.15.37:0/3961737780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00b5650 0x7f18e00d3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e5c9d640 1 -- 172.21.15.37:0/3961737780 >> v1:172.21.15.203:6789/0 conn(0x7f18e00d4440 legacy=0x7f18e006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.652+0000 7f18e5c9d640 1 -- 172.21.15.37:0/3961737780 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e006c5b0 con 0x7f18e00b5270 2024-04-25T08:59:08.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e5c9d640 1 --2- 172.21.15.37:0/3961737780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00b5270 0x7f18e00b0380 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ea0 tx=0x7f18dc01ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dc8899bba6401e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e4c9b640 1 -- 172.21.15.37:0/3961737780 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0a5960 con 0x7f18e00b5270 2024-04-25T08:59:08.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e4c9b640 1 -- 172.21.15.37:0/3961737780 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc0a5b00 con 0x7f18e00b5270 2024-04-25T08:59:08.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e4c9b640 1 -- 172.21.15.37:0/3961737780 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc0ad3f0 con 0x7f18e00b5270 2024-04-25T08:59:08.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e7f01640 1 -- 172.21.15.37:0/3961737780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00b5270 msgr2=0x7f18e00b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e7f01640 1 --2- 172.21.15.37:0/3961737780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00b5270 0x7f18e00b0380 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f18dc009ea0 tx=0x7f18dc01ca70 comp rx=0 tx=0).stop 2024-04-25T08:59:08.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e7f01640 1 -- 172.21.15.37:0/3961737780 shutdown_connections 2024-04-25T08:59:08.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e7f01640 1 --2- 172.21.15.37:0/3961737780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00b5650 0x7f18e00d3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e7f01640 1 --2- 172.21.15.37:0/3961737780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00b5270 0x7f18e00b0380 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e7f01640 1 -- 172.21.15.37:0/3961737780 >> 172.21.15.37:0/3961737780 conn(0x7f18e008b000 msgr2=0x7f18e00d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e7f01640 1 -- 172.21.15.37:0/3961737780 shutdown_connections 2024-04-25T08:59:08.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.656+0000 7f18e7f01640 1 -- 172.21.15.37:0/3961737780 wait complete. 2024-04-25T08:59:08.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e7f01640 1 Processor -- start 2024-04-25T08:59:08.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e7f01640 1 -- start start 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e7f01640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00b5270 0x7f18e0155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e7f01640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e00b5650 0x7f18e00b7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e7f01640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00d4440 0x7f18e00b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e7f01640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e015dfa0 con 0x7f18e00b5650 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e7f01640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18e015de20 con 0x7f18e00d4440 2024-04-25T08:59:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e7f01640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18e016a720 con 0x7f18e00b5270 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e549c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e00b5650 0x7f18e00b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e549c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e00b5650 0x7f18e00b7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52780/0 (socket says 172.21.15.37:52780) 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e549c640 1 -- 172.21.15.37:0/2447878674 learned_addr learned my addr 172.21.15.37:0/2447878674 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e5c9d640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00b5270 0x7f18e0155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e649e640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00d4440 0x7f18e00b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e549c640 1 -- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00b5270 msgr2=0x7f18e0155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e549c640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00b5270 0x7f18e0155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e549c640 1 -- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00d4440 msgr2=0x7f18e00b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e549c640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00d4440 0x7f18e00b7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e549c640 1 -- 172.21.15.37:0/2447878674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18e00b8100 con 0x7f18e00b5650 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e549c640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e00b5650 0x7f18e00b7590 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f18d000e9b0 tx=0x7f18d000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18d67fc640 1 -- 172.21.15.37:0/2447878674 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d0017070 con 0x7f18e00b5650 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18d67fc640 1 -- 172.21.15.37:0/2447878674 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d000c9a0 con 0x7f18e00b5650 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e7f01640 1 -- 172.21.15.37:0/2447878674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18e00b83e0 con 0x7f18e00b5650 2024-04-25T08:59:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18d67fc640 1 -- 172.21.15.37:0/2447878674 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d000cd50 con 0x7f18e00b5650 2024-04-25T08:59:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e7f01640 1 -- 172.21.15.37:0/2447878674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18e00b89a0 con 0x7f18e00b5650 2024-04-25T08:59:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18e7f01640 1 -- 172.21.15.37:0/2447878674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18e004efa0 con 0x7f18e00b5650 2024-04-25T08:59:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.660+0000 7f18d67fc640 1 -- 172.21.15.37:0/2447878674 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18d001a070 con 0x7f18e00b5650 2024-04-25T08:59:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.664+0000 7f18d67fc640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18c4066f60 0x7f18c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.664+0000 7f18d67fc640 1 -- 172.21.15.37:0/2447878674 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f18d00855d0 con 0x7f18e00b5650 2024-04-25T08:59:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.664+0000 7f18e5c9d640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18c4066f60 0x7f18c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.664+0000 7f18e5c9d640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18c4066f60 0x7f18c4069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0ae000 tx=0x7f18dc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.668+0000 7f18d67fc640 1 -- 172.21.15.37:0/2447878674 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18d0056450 con 0x7f18e00b5650 2024-04-25T08:59:08.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.824+0000 7f18e7f01640 1 -- 172.21.15.37:0/2447878674 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18e00d1b20 con 0x7f18e00b5650 2024-04-25T08:59:08.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18d67fc640 1 -- 172.21.15.37:0/2447878674 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16849 (secure 0 0 0) 0x7f18d0055d50 con 0x7f18e00b5650 2024-04-25T08:59:08.818 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:59:08.819 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":95,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:58:54.529461+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:50.512578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:51.848880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:59:08.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 -- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18c4066f60 msgr2=0x7f18c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18c4066f60 0x7f18c4069420 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f18dc0ae000 tx=0x7f18dc0a7070 comp rx=0 tx=0).stop 2024-04-25T08:59:08.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 -- 172.21.15.37:0/2447878674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e00b5650 msgr2=0x7f18e00b7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e00b5650 0x7f18e00b7590 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f18d000e9b0 tx=0x7f18d000ee70 comp rx=0 tx=0).stop 2024-04-25T08:59:08.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 -- 172.21.15.37:0/2447878674 shutdown_connections 2024-04-25T08:59:08.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18c4066f60 0x7f18c4069420 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18e00d4440 0x7f18e00b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18e00b5650 0x7f18e00b7590 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 --2- 172.21.15.37:0/2447878674 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18e00b5270 0x7f18e0155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 -- 172.21.15.37:0/2447878674 >> 172.21.15.37:0/2447878674 conn(0x7f18e008b000 msgr2=0x7f18e00b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 -- 172.21.15.37:0/2447878674 shutdown_connections 2024-04-25T08:59:08.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.828+0000 7f18e7f01640 1 -- 172.21.15.37:0/2447878674 wait complete. 2024-04-25T08:59:08.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:58:50.512578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:58:51.848880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:59:08.838 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:59:08.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10eabf8640 1 Processor -- start 2024-04-25T08:59:08.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10eabf8640 1 -- start start 2024-04-25T08:59:08.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10eabf8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e415b420 0x7f10e40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10eabf8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f10e415b800 0x7f10e40cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10eabf8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f10e40cfad0 0x7f10e415f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10eabf8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e40cff20 con 0x7f10e415b800 2024-04-25T08:59:08.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10eabf8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e40cc930 con 0x7f10e40cfad0 2024-04-25T08:59:08.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10eabf8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40cbd20 con 0x7f10e415b420 2024-04-25T08:59:08.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10e3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f10e415b800 0x7f10e40cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10e3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f10e415b800 0x7f10e40cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52788/0 (socket says 172.21.15.37:52788) 2024-04-25T08:59:08.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10e3fff640 1 -- 172.21.15.37:0/719391311 learned_addr learned my addr 172.21.15.37:0/719391311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:08.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10e9195640 1 --2- 172.21.15.37:0/719391311 >> v2:172.21.15.203:3300/0 conn(0x7f10e40cfad0 0x7f10e415f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10e8994640 1 --2- 172.21.15.37:0/719391311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e415b420 0x7f10e40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10e3fff640 1 -- 172.21.15.37:0/719391311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e415b420 msgr2=0x7f10e40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:08.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10e3fff640 1 --2- 172.21.15.37:0/719391311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e415b420 0x7f10e40ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10e3fff640 1 -- 172.21.15.37:0/719391311 >> v2:172.21.15.203:3300/0 conn(0x7f10e40cfad0 msgr2=0x7f10e415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:08.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10e3fff640 1 --2- 172.21.15.37:0/719391311 >> v2:172.21.15.203:3300/0 conn(0x7f10e40cfad0 0x7f10e415f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.964+0000 7f10e3fff640 1 -- 172.21.15.37:0/719391311 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e40cc000 con 0x7f10e415b800 2024-04-25T08:59:08.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10e3fff640 1 --2- 172.21.15.37:0/719391311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e415b800 0x7f10e40cf590 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f10e40d0440 tx=0x7f10d401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f7126cdbd9165ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10e37fe640 1 -- 172.21.15.37:0/719391311 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40b0070 con 0x7f10e415b800 2024-04-25T08:59:08.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10e37fe640 1 -- 172.21.15.37:0/719391311 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d40ab600 con 0x7f10e415b800 2024-04-25T08:59:08.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10e37fe640 1 -- 172.21.15.37:0/719391311 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40ab9d0 con 0x7f10e415b800 2024-04-25T08:59:08.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 -- 172.21.15.37:0/719391311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e415b800 msgr2=0x7f10e40cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 --2- 172.21.15.37:0/719391311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e415b800 0x7f10e40cf590 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f10e40d0440 tx=0x7f10d401c230 comp rx=0 tx=0).stop 2024-04-25T08:59:08.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 -- 172.21.15.37:0/719391311 shutdown_connections 2024-04-25T08:59:08.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 --2- 172.21.15.37:0/719391311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e415b800 0x7f10e40cf590 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 --2- 172.21.15.37:0/719391311 >> v2:172.21.15.203:3300/0 conn(0x7f10e40cfad0 0x7f10e415f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 --2- 172.21.15.37:0/719391311 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e415b420 0x7f10e40ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 -- 172.21.15.37:0/719391311 >> 172.21.15.37:0/719391311 conn(0x7f10e406ae60 msgr2=0x7f10e406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:08.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 -- 172.21.15.37:0/719391311 shutdown_connections 2024-04-25T08:59:08.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 -- 172.21.15.37:0/719391311 wait complete. 2024-04-25T08:59:08.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 Processor -- start 2024-04-25T08:59:08.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 -- start start 2024-04-25T08:59:08.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e40cfad0 0x7f10e4155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e415b420 0x7f10e41570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e415b800 0x7f10e41575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40caa80 con 0x7f10e415b420 2024-04-25T08:59:08.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40ca900 con 0x7f10e40cfad0 2024-04-25T08:59:08.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.968+0000 7f10eabf8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e40cadb0 con 0x7f10e415b800 2024-04-25T08:59:08.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e415b420 0x7f10e41570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e415b420 0x7f10e41570b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52796/0 (socket says 172.21.15.37:52796) 2024-04-25T08:59:08.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e3fff640 1 -- 172.21.15.37:0/1619176567 learned_addr learned my addr 172.21.15.37:0/1619176567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:08.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e8994640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e40cfad0 0x7f10e4155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e3fff640 1 -- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e415b800 msgr2=0x7f10e41575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e3fff640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e415b800 0x7f10e41575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e3fff640 1 -- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e40cfad0 msgr2=0x7f10e4155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:08.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e3fff640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e40cfad0 0x7f10e4155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:08.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e3fff640 1 -- 172.21.15.37:0/1619176567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e415cf20 con 0x7f10e415b420 2024-04-25T08:59:08.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e8994640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e40cfad0 0x7f10e4155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:08.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e3fff640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e415b420 0x7f10e41570b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f10d401a8e0 tx=0x7f10d4018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e17fa640 1 -- 172.21.15.37:0/1619176567 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40ba070 con 0x7f10e415b420 2024-04-25T08:59:08.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e17fa640 1 -- 172.21.15.37:0/1619176567 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d40b4620 con 0x7f10e415b420 2024-04-25T08:59:08.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10eabf8640 1 -- 172.21.15.37:0/1619176567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e4157c20 con 0x7f10e415b420 2024-04-25T08:59:08.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e17fa640 1 -- 172.21.15.37:0/1619176567 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40b0040 con 0x7f10e415b420 2024-04-25T08:59:08.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10eabf8640 1 -- 172.21.15.37:0/1619176567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e40c6dd0 con 0x7f10e415b420 2024-04-25T08:59:08.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10eabf8640 1 -- 172.21.15.37:0/1619176567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e404efa0 con 0x7f10e415b420 2024-04-25T08:59:08.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e17fa640 1 -- 172.21.15.37:0/1619176567 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f10d40bd070 con 0x7f10e415b420 2024-04-25T08:59:08.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e17fa640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10d0066f60 0x7f10d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:08.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e17fa640 1 -- 172.21.15.37:0/1619176567 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10d411e2f0 con 0x7f10e415b420 2024-04-25T08:59:08.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e8994640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10d0066f60 0x7f10d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:08.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.972+0000 7f10e8994640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10d0066f60 0x7f10d0069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f10d80097c0 tx=0x7f10d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:08.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:08.976+0000 7f10e17fa640 1 -- 172.21.15.37:0/1619176567 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f10d4016110 con 0x7f10e415b420 2024-04-25T08:59:09.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 -- 172.21.15.37:0/1619176567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10e40ca3a0 con 0x7f10e415b420 2024-04-25T08:59:09.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10e17fa640 1 -- 172.21.15.37:0/1619176567 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10d40ef0d0 con 0x7f10e415b420 2024-04-25T08:59:09.128 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T08:59:09.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 -- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10d0066f60 msgr2=0x7f10d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10d0066f60 0x7f10d0069420 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f10d80097c0 tx=0x7f10d8008040 comp rx=0 tx=0).stop 2024-04-25T08:59:09.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 -- 172.21.15.37:0/1619176567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e415b420 msgr2=0x7f10e41570b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e415b420 0x7f10e41570b0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f10d401a8e0 tx=0x7f10d4018fd0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 -- 172.21.15.37:0/1619176567 shutdown_connections 2024-04-25T08:59:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f10d0066f60 0x7f10d0069420 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f10e415b800 0x7f10e41575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e415b420 0x7f10e41570b0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 --2- 172.21.15.37:0/1619176567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f10e40cfad0 0x7f10e4155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 -- 172.21.15.37:0/1619176567 >> 172.21.15.37:0/1619176567 conn(0x7f10e406ae60 msgr2=0x7f10e406e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 -- 172.21.15.37:0/1619176567 shutdown_connections 2024-04-25T08:59:09.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.136+0000 7f10eabf8640 1 -- 172.21.15.37:0/1619176567 wait complete. 2024-04-25T08:59:09.145 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:59:09.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1323cf3640 1 Processor -- start 2024-04-25T08:59:09.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1323cf3640 1 -- start start 2024-04-25T08:59:09.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1323cf3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c15a600 0x7f131c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1323cf3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f131c0b2ac0 0x7f131c0cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1323cf3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f131c15e1e0 0x7f131c15e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1323cf3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131c15eb90 con 0x7f131c15e1e0 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1323cf3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f131c0cf370 con 0x7f131c0b2ac0 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1323cf3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0ce070 con 0x7f131c15a600 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1322290640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f131c15e1e0 0x7f131c15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1322290640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f131c15e1e0 0x7f131c15e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52810/0 (socket says 172.21.15.37:52810) 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1322290640 1 -- 172.21.15.37:0/3247244037 learned_addr learned my addr 172.21.15.37:0/3247244037 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1321a8f640 1 --2- 172.21.15.37:0/3247244037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c15a600 0x7f131c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f132128e640 1 --2- 172.21.15.37:0/3247244037 >> v2:172.21.15.203:3300/0 conn(0x7f131c0b2ac0 0x7f131c0cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1322290640 1 -- 172.21.15.37:0/3247244037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c15a600 msgr2=0x7f131c0cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1322290640 1 --2- 172.21.15.37:0/3247244037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c15a600 0x7f131c0cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1322290640 1 -- 172.21.15.37:0/3247244037 >> v2:172.21.15.203:3300/0 conn(0x7f131c0b2ac0 msgr2=0x7f131c0cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1322290640 1 --2- 172.21.15.37:0/3247244037 >> v2:172.21.15.203:3300/0 conn(0x7f131c0b2ac0 0x7f131c0cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1322290640 1 -- 172.21.15.37:0/3247244037 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c0ce350 con 0x7f131c15e1e0 2024-04-25T08:59:09.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f132128e640 1 --2- 172.21.15.37:0/3247244037 >> v2:172.21.15.203:3300/0 conn(0x7f131c0b2ac0 0x7f131c0cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:09.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.284+0000 7f1321a8f640 1 --2- 172.21.15.37:0/3247244037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c15a600 0x7f131c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:09.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1322290640 1 --2- 172.21.15.37:0/3247244037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f131c15e1e0 0x7f131c15e650 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f131c0747c0 tx=0x7f130c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a40ef3f313790c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:09.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1320a8d640 1 -- 172.21.15.37:0/3247244037 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c0b5070 con 0x7f131c15e1e0 2024-04-25T08:59:09.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1320a8d640 1 -- 172.21.15.37:0/3247244037 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130c0a89d0 con 0x7f131c15e1e0 2024-04-25T08:59:09.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1320a8d640 1 -- 172.21.15.37:0/3247244037 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c0a8da0 con 0x7f131c15e1e0 2024-04-25T08:59:09.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 -- 172.21.15.37:0/3247244037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f131c15e1e0 msgr2=0x7f131c15e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 --2- 172.21.15.37:0/3247244037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f131c15e1e0 0x7f131c15e650 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f131c0747c0 tx=0x7f130c01ed10 comp rx=0 tx=0).stop 2024-04-25T08:59:09.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 -- 172.21.15.37:0/3247244037 shutdown_connections 2024-04-25T08:59:09.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 --2- 172.21.15.37:0/3247244037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f131c15e1e0 0x7f131c15e650 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 --2- 172.21.15.37:0/3247244037 >> v2:172.21.15.203:3300/0 conn(0x7f131c0b2ac0 0x7f131c0cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 --2- 172.21.15.37:0/3247244037 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c15a600 0x7f131c0cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 -- 172.21.15.37:0/3247244037 >> 172.21.15.37:0/3247244037 conn(0x7f131c0b1f10 msgr2=0x7f131c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:09.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 -- 172.21.15.37:0/3247244037 shutdown_connections 2024-04-25T08:59:09.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 -- 172.21.15.37:0/3247244037 wait complete. 2024-04-25T08:59:09.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 Processor -- start 2024-04-25T08:59:09.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 -- start start 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c0b2ac0 0x7f131c0c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131c15a600 0x7f131c09c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f131c15e1e0 0x7f131c09cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0cbef0 con 0x7f131c15e1e0 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0ce910 con 0x7f131c15a600 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1323cf3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131c0cc1e0 con 0x7f131c0b2ac0 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1322290640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f131c15e1e0 0x7f131c09cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1322290640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f131c15e1e0 0x7f131c09cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52826/0 (socket says 172.21.15.37:52826) 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.288+0000 7f1322290640 1 -- 172.21.15.37:0/1726837252 learned_addr learned my addr 172.21.15.37:0/1726837252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f132128e640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131c15a600 0x7f131c09c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1321a8f640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c0b2ac0 0x7f131c0c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1322290640 1 -- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c0b2ac0 msgr2=0x7f131c0c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1322290640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c0b2ac0 0x7f131c0c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1322290640 1 -- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131c15a600 msgr2=0x7f131c09c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1322290640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131c15a600 0x7f131c09c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1322290640 1 -- 172.21.15.37:0/1726837252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f131c09d5b0 con 0x7f131c15e1e0 2024-04-25T08:59:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1321a8f640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c0b2ac0 0x7f131c0c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1322290640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f131c15e1e0 0x7f131c09cf30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f130c0b3040 tx=0x7f130c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f131a7fc640 1 -- 172.21.15.37:0/1726837252 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c0b5070 con 0x7f131c15e1e0 2024-04-25T08:59:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f131a7fc640 1 -- 172.21.15.37:0/1726837252 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f130c0b0ae0 con 0x7f131c15e1e0 2024-04-25T08:59:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f131a7fc640 1 -- 172.21.15.37:0/1726837252 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130c0bb410 con 0x7f131c15e1e0 2024-04-25T08:59:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1323cf3640 1 -- 172.21.15.37:0/1726837252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c09d890 con 0x7f131c15e1e0 2024-04-25T08:59:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1323cf3640 1 -- 172.21.15.37:0/1726837252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131c09de50 con 0x7f131c15e1e0 2024-04-25T08:59:09.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1323cf3640 1 -- 172.21.15.37:0/1726837252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131c0b6380 con 0x7f131c15e1e0 2024-04-25T08:59:09.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f131a7fc640 1 -- 172.21.15.37:0/1726837252 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f130c0c2060 con 0x7f131c15e1e0 2024-04-25T08:59:09.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f131a7fc640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1308066f60 0x7f1308069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f131a7fc640 1 -- 172.21.15.37:0/1726837252 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f130c1223c0 con 0x7f131c15e1e0 2024-04-25T08:59:09.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1321a8f640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1308066f60 0x7f1308069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.292+0000 7f1321a8f640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1308066f60 0x7f1308069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f131400d430 tx=0x7f1314007530 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:09.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.296+0000 7f131a7fc640 1 -- 172.21.15.37:0/1726837252 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f130c018870 con 0x7f131c15e1e0 2024-04-25T08:59:09.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.456+0000 7f1323cf3640 1 -- 172.21.15.37:0/1726837252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f131c04ef80 con 0x7f131c15e1e0 2024-04-25T08:59:09.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f131a7fc640 1 -- 172.21.15.37:0/1726837252 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7f130c0f31a0 con 0x7f131c15e1e0 2024-04-25T08:59:09.450 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T08:59:09.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 -- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1308066f60 msgr2=0x7f1308069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1308066f60 0x7f1308069420 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f131400d430 tx=0x7f1314007530 comp rx=0 tx=0).stop 2024-04-25T08:59:09.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 -- 172.21.15.37:0/1726837252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f131c15e1e0 msgr2=0x7f131c09cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f131c15e1e0 0x7f131c09cf30 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f130c0b3040 tx=0x7f130c01bab0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 -- 172.21.15.37:0/1726837252 shutdown_connections 2024-04-25T08:59:09.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1308066f60 0x7f1308069420 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f131c15e1e0 0x7f131c09cf30 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f131c15a600 0x7f131c09c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 --2- 172.21.15.37:0/1726837252 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f131c0b2ac0 0x7f131c0c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 -- 172.21.15.37:0/1726837252 >> 172.21.15.37:0/1726837252 conn(0x7f131c0b1f10 msgr2=0x7f131c15ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:09.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 -- 172.21.15.37:0/1726837252 shutdown_connections 2024-04-25T08:59:09.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.460+0000 7f1323cf3640 1 -- 172.21.15.37:0/1726837252 wait complete. 2024-04-25T08:59:09.468 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:59:09.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f8630906640 1 Processor -- start 2024-04-25T08:59:09.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f8630906640 1 -- start start 2024-04-25T08:59:09.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f8630906640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f862c0b3cf0 0x7f862c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f8630906640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0cfad0 0x7f862c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f8630906640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f862c0cbf10 con 0x7f862c15e4f0 2024-04-25T08:59:09.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f8630906640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f862c0cff40 con 0x7f862c0b3cf0 2024-04-25T08:59:09.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f8630906640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0cc1f0 con 0x7f862c0cfad0 2024-04-25T08:59:09.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f862a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f862c0b3cf0 0x7f862c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f862a59c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f862c0b3cf0 0x7f862c0cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38458/0 (socket says 172.21.15.37:38458) 2024-04-25T08:59:09.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f862a59c640 1 -- 172.21.15.37:0/1765431982 learned_addr learned my addr 172.21.15.37:0/1765431982 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:09.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f8621d9b640 1 --2- 172.21.15.37:0/1765431982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0cfad0 0x7f862c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f862a59c640 1 -- 172.21.15.37:0/1765431982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0cfad0 msgr2=0x7f862c15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:09.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f862a59c640 1 --2- 172.21.15.37:0/1765431982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0cfad0 0x7f862c15dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f862a59c640 1 -- 172.21.15.37:0/1765431982 >> v1:172.21.15.203:6789/0 conn(0x7f862c15e4f0 legacy=0x7f862c161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:09.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.592+0000 7f862a59c640 1 -- 172.21.15.37:0/1765431982 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c0cc4d0 con 0x7f862c0b3cf0 2024-04-25T08:59:09.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f862a59c640 1 --2- 172.21.15.37:0/1765431982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f862c0b3cf0 0x7f862c0cf590 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f861c009870 tx=0x7f861c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783f8b8446c4e564 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:09.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8629d9b640 1 -- 172.21.15.37:0/1765431982 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c0b0070 con 0x7f862c0b3cf0 2024-04-25T08:59:09.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8629d9b640 1 -- 172.21.15.37:0/1765431982 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c0ab600 con 0x7f862c0b3cf0 2024-04-25T08:59:09.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8629d9b640 1 -- 172.21.15.37:0/1765431982 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c0ab9d0 con 0x7f862c0b3cf0 2024-04-25T08:59:09.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 -- 172.21.15.37:0/1765431982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f862c0b3cf0 msgr2=0x7f862c0cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 --2- 172.21.15.37:0/1765431982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f862c0b3cf0 0x7f862c0cf590 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f861c009870 tx=0x7f861c01c520 comp rx=0 tx=0).stop 2024-04-25T08:59:09.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 -- 172.21.15.37:0/1765431982 shutdown_connections 2024-04-25T08:59:09.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 --2- 172.21.15.37:0/1765431982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0cfad0 0x7f862c15dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 --2- 172.21.15.37:0/1765431982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f862c0b3cf0 0x7f862c0cf590 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 -- 172.21.15.37:0/1765431982 >> 172.21.15.37:0/1765431982 conn(0x7f862c06ae60 msgr2=0x7f862c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:09.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 -- 172.21.15.37:0/1765431982 shutdown_connections 2024-04-25T08:59:09.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 -- 172.21.15.37:0/1765431982 wait complete. 2024-04-25T08:59:09.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 Processor -- start 2024-04-25T08:59:09.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 -- start start 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f862c0b3cf0 0x7f862c155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0cfad0 0x7f862c0c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f862c15e4f0 0x7f862c1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0b4250 con 0x7f862c0b3cf0 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f862c0b40d0 con 0x7f862c15e4f0 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8630906640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f862c15e920 con 0x7f862c0cfad0 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f862a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f862c0b3cf0 0x7f862c155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f862a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f862c0b3cf0 0x7f862c155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38460/0 (socket says 172.21.15.37:38460) 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f862a59c640 1 -- 172.21.15.37:0/4149283109 learned_addr learned my addr 172.21.15.37:0/4149283109 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f8621d9b640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0cfad0 0x7f862c0c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f862a59c640 1 -- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0cfad0 msgr2=0x7f862c0c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f862a59c640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0cfad0 0x7f862c0c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f862a59c640 1 -- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f862c15e4f0 msgr2=0x7f862c1559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f862ad9d640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f862c15e4f0 0x7f862c1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f862a59c640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f862c15e4f0 0x7f862c1559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f862a59c640 1 -- 172.21.15.37:0/4149283109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f862c159a30 con 0x7f862c0b3cf0 2024-04-25T08:59:09.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.596+0000 7f862a59c640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f862c0b3cf0 0x7f862c155310 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f861c0ae040 tx=0x7f861c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:09.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.600+0000 7f86237fe640 1 -- 172.21.15.37:0/4149283109 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c0b0040 con 0x7f862c0b3cf0 2024-04-25T08:59:09.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.600+0000 7f86237fe640 1 -- 172.21.15.37:0/4149283109 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861c0b9070 con 0x7f862c0b3cf0 2024-04-25T08:59:09.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.600+0000 7f8630906640 1 -- 172.21.15.37:0/4149283109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f862c15d440 con 0x7f862c0b3cf0 2024-04-25T08:59:09.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.600+0000 7f86237fe640 1 -- 172.21.15.37:0/4149283109 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861c0b4660 con 0x7f862c0b3cf0 2024-04-25T08:59:09.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.600+0000 7f8630906640 1 -- 172.21.15.37:0/4149283109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f862c15d6e0 con 0x7f862c0b3cf0 2024-04-25T08:59:09.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.600+0000 7f8630906640 1 -- 172.21.15.37:0/4149283109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f862c04efa0 con 0x7f862c0b3cf0 2024-04-25T08:59:09.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.600+0000 7f86237fe640 1 -- 172.21.15.37:0/4149283109 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f861c0b4800 con 0x7f862c0b3cf0 2024-04-25T08:59:09.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.600+0000 7f86237fe640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8610066f90 0x7f8610069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.600+0000 7f86237fe640 1 -- 172.21.15.37:0/4149283109 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f861c11edf0 con 0x7f862c0b3cf0 2024-04-25T08:59:09.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.600+0000 7f8621d9b640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8610066f90 0x7f8610069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.600+0000 7f8621d9b640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8610066f90 0x7f8610069450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f862c1604d0 tx=0x7f8614008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:09.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.604+0000 7f86237fe640 1 -- 172.21.15.37:0/4149283109 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f861c016110 con 0x7f862c0b3cf0 2024-04-25T08:59:09.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.756+0000 7f8630906640 1 -- 172.21.15.37:0/4149283109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f862c15eeb0 con 0x7f862c0b3cf0 2024-04-25T08:59:09.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.756+0000 7f86237fe640 1 -- 172.21.15.37:0/4149283109 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f861c0efd00 con 0x7f862c0b3cf0 2024-04-25T08:59:09.749 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T08:59:09.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 -- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8610066f90 msgr2=0x7f8610069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8610066f90 0x7f8610069450 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f862c1604d0 tx=0x7f8614008040 comp rx=0 tx=0).stop 2024-04-25T08:59:09.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 -- 172.21.15.37:0/4149283109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f862c0b3cf0 msgr2=0x7f862c155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f862c0b3cf0 0x7f862c155310 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f861c0ae040 tx=0x7f861c0192c0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 -- 172.21.15.37:0/4149283109 shutdown_connections 2024-04-25T08:59:09.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8610066f90 0x7f8610069450 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f862c15e4f0 0x7f862c1559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f862c0cfad0 0x7f862c0c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 --2- 172.21.15.37:0/4149283109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f862c0b3cf0 0x7f862c155310 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 -- 172.21.15.37:0/4149283109 >> 172.21.15.37:0/4149283109 conn(0x7f862c06ae60 msgr2=0x7f862c15f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:09.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 -- 172.21.15.37:0/4149283109 shutdown_connections 2024-04-25T08:59:09.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.760+0000 7f8630906640 1 -- 172.21.15.37:0/4149283109 wait complete. 2024-04-25T08:59:09.767 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:59:09.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bc0805640 1 Processor -- start 2024-04-25T08:59:09.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bc0805640 1 -- start start 2024-04-25T08:59:09.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bc0805640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2bbc159ea0 0x7f2bbc0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bc0805640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2bbc15a280 0x7f2bbc15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bc0805640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc15ae30 con 0x7f2bbc0718c0 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bc0805640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bbc0b2990 con 0x7f2bbc15a280 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bc0805640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bbc15b110 con 0x7f2bbc159ea0 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bb9d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2bbc15a280 0x7f2bbc15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bbad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2bbc0718c0 0x7f2bbc06d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51832/0 (socket says 172.21.15.37:51832) 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bbad9d640 1 -- 172.21.15.37:0/747316891 learned_addr learned my addr 172.21.15.37:0/747316891 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bba59c640 1 --2- 172.21.15.37:0/747316891 >> v2:172.21.15.203:3300/0 conn(0x7f2bbc159ea0 0x7f2bbc0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bb9d9b640 1 -- 172.21.15.37:0/747316891 >> v2:172.21.15.203:3300/0 conn(0x7f2bbc159ea0 msgr2=0x7f2bbc0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bb9d9b640 1 --2- 172.21.15.37:0/747316891 >> v2:172.21.15.203:3300/0 conn(0x7f2bbc159ea0 0x7f2bbc0b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bb9d9b640 1 -- 172.21.15.37:0/747316891 >> v1:172.21.15.37:6789/0 conn(0x7f2bbc0718c0 legacy=0x7f2bbc06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.888+0000 7f2bb9d9b640 1 -- 172.21.15.37:0/747316891 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc0b2620 con 0x7f2bbc15a280 2024-04-25T08:59:09.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bb9d9b640 1 --2- 172.21.15.37:0/747316891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc15a280 0x7f2bbc15a700 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0d0510 tx=0x7f2bb001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e578708ef924a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:09.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bb959a640 1 -- 172.21.15.37:0/747316891 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00b1070 con 0x7f2bbc15a280 2024-04-25T08:59:09.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bb959a640 1 -- 172.21.15.37:0/747316891 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb00ac5f0 con 0x7f2bbc15a280 2024-04-25T08:59:09.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bb959a640 1 -- 172.21.15.37:0/747316891 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00ac9c0 con 0x7f2bbc15a280 2024-04-25T08:59:09.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bc0805640 1 -- 172.21.15.37:0/747316891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc15a280 msgr2=0x7f2bbc15a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bc0805640 1 --2- 172.21.15.37:0/747316891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc15a280 0x7f2bbc15a700 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0d0510 tx=0x7f2bb001c480 comp rx=0 tx=0).stop 2024-04-25T08:59:09.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bc0805640 1 -- 172.21.15.37:0/747316891 shutdown_connections 2024-04-25T08:59:09.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bc0805640 1 --2- 172.21.15.37:0/747316891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc15a280 0x7f2bbc15a700 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bc0805640 1 --2- 172.21.15.37:0/747316891 >> v2:172.21.15.203:3300/0 conn(0x7f2bbc159ea0 0x7f2bbc0b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bc0805640 1 -- 172.21.15.37:0/747316891 >> 172.21.15.37:0/747316891 conn(0x7f2bbc0b11a0 msgr2=0x7f2bbc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:09.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bc0805640 1 -- 172.21.15.37:0/747316891 shutdown_connections 2024-04-25T08:59:09.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.892+0000 7f2bc0805640 1 -- 172.21.15.37:0/747316891 wait complete. 2024-04-25T08:59:09.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bc0805640 1 Processor -- start 2024-04-25T08:59:09.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bc0805640 1 -- start start 2024-04-25T08:59:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bc0805640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bbc0718c0 0x7f2bbc1508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bc0805640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc159ea0 0x7f2bbc0a85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bc0805640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc15a280 0x7f2bbc151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bc0805640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc164b20 con 0x7f2bbc15a280 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bc0805640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc1649a0 con 0x7f2bbc159ea0 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bc0805640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc164ca0 con 0x7f2bbc0718c0 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bbad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc15a280 0x7f2bbc151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bbad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc15a280 0x7f2bbc151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38486/0 (socket says 172.21.15.37:38486) 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bbad9d640 1 -- 172.21.15.37:0/3639200704 learned_addr learned my addr 172.21.15.37:0/3639200704 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bb9d9b640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc159ea0 0x7f2bbc0a85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bba59c640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bbc0718c0 0x7f2bbc1508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bbad9d640 1 -- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bbc0718c0 msgr2=0x7f2bbc1508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bbad9d640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bbc0718c0 0x7f2bbc1508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bbad9d640 1 -- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc159ea0 msgr2=0x7f2bbc0a85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:09.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bbad9d640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc159ea0 0x7f2bbc0a85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bbad9d640 1 -- 172.21.15.37:0/3639200704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc157200 con 0x7f2bbc15a280 2024-04-25T08:59:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bbad9d640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc15a280 0x7f2bbc151a30 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400cbc0 tx=0x7f2bb400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2baaffd640 1 -- 172.21.15.37:0/3639200704 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4017070 con 0x7f2bbc15a280 2024-04-25T08:59:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2baaffd640 1 -- 172.21.15.37:0/3639200704 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb400ad40 con 0x7f2bbc15a280 2024-04-25T08:59:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bc0805640 1 -- 172.21.15.37:0/3639200704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc1574e0 con 0x7f2bbc15a280 2024-04-25T08:59:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2baaffd640 1 -- 172.21.15.37:0/3639200704 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4012600 con 0x7f2bbc15a280 2024-04-25T08:59:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bc0805640 1 -- 172.21.15.37:0/3639200704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc157aa0 con 0x7f2bbc15a280 2024-04-25T08:59:09.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2bc0805640 1 -- 172.21.15.37:0/3639200704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bbc04efa0 con 0x7f2bbc15a280 2024-04-25T08:59:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2baaffd640 1 -- 172.21.15.37:0/3639200704 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bb401a070 con 0x7f2bbc15a280 2024-04-25T08:59:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2baaffd640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b9c066f60 0x7f2b9c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.896+0000 7f2baaffd640 1 -- 172.21.15.37:0/3639200704 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2bb4084ef0 con 0x7f2bbc15a280 2024-04-25T08:59:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.900+0000 7f2bba59c640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b9c066f60 0x7f2b9c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:09.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.900+0000 7f2bba59c640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b9c066f60 0x7f2b9c069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009790 tx=0x7f2bac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:09.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:09.904+0000 7f2baaffd640 1 -- 172.21.15.37:0/3639200704 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2bb4055680 con 0x7f2bbc15a280 2024-04-25T08:59:10.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 -- 172.21.15.37:0/3639200704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2bbc06e6b0 con 0x7f2bbc15a280 2024-04-25T08:59:10.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2baaffd640 1 -- 172.21.15.37:0/3639200704 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v95) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2bb4059720 con 0x7f2bbc15a280 2024-04-25T08:59:10.059 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T08:59:10.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 -- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b9c066f60 msgr2=0x7f2b9c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:10.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b9c066f60 0x7f2b9c069420 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2bac009790 tx=0x7f2bac008040 comp rx=0 tx=0).stop 2024-04-25T08:59:10.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 -- 172.21.15.37:0/3639200704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc15a280 msgr2=0x7f2bbc151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:10.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc15a280 0x7f2bbc151a30 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2bb400cbc0 tx=0x7f2bb400a430 comp rx=0 tx=0).stop 2024-04-25T08:59:10.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 -- 172.21.15.37:0/3639200704 shutdown_connections 2024-04-25T08:59:10.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b9c066f60 0x7f2b9c069420 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:10.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bbc15a280 0x7f2bbc151a30 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:10.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2bbc159ea0 0x7f2bbc0a85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:10.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 --2- 172.21.15.37:0/3639200704 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2bbc0718c0 0x7f2bbc1508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:10.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 -- 172.21.15.37:0/3639200704 >> 172.21.15.37:0/3639200704 conn(0x7f2bbc0b11a0 msgr2=0x7f2bbc0b0610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:10.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 -- 172.21.15.37:0/3639200704 shutdown_connections 2024-04-25T08:59:10.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:10.068+0000 7f2bc0805640 1 -- 172.21.15.37:0/3639200704 wait complete. 2024-04-25T08:59:10.077 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T08:59:10.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:10.077 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:10.086 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48073]: fuse finished with error 0 and tester_r 0 2024-04-25T08:59:10.088 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:59:16.190 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T08:59:16.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:16.191 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:59:16.199 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.380+0000 7f0ab9a73640 1 Processor -- start 2024-04-25T08:59:16.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab9a73640 1 -- start start 2024-04-25T08:59:16.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab9a73640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ab415da80 0x7f0ab40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab9a73640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ab40d1cd0 0x7f0ab40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab9a73640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab406be60 con 0x7f0ab40d0620 2024-04-25T08:59:16.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab9a73640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab406c440 con 0x7f0ab40d1cd0 2024-04-25T08:59:16.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab9a73640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab415de60 con 0x7f0ab415da80 2024-04-25T08:59:16.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab2ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ab40d1cd0 0x7f0ab40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab37fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ab415da80 0x7f0ab40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab2ffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ab40d1cd0 0x7f0ab40d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:40720/0 (socket says 172.21.15.37:40720) 2024-04-25T08:59:16.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab2ffd640 1 -- 172.21.15.37:0/2432067243 learned_addr learned my addr 172.21.15.37:0/2432067243 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:16.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab3fff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f0ab40d0620 0x7f0ab40ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:33766/0 (socket says 172.21.15.37:33766) 2024-04-25T08:59:16.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab2ffd640 1 -- 172.21.15.37:0/2432067243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ab415da80 msgr2=0x7f0ab40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:16.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab2ffd640 1 --2- 172.21.15.37:0/2432067243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ab415da80 0x7f0ab40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab2ffd640 1 -- 172.21.15.37:0/2432067243 >> v1:172.21.15.203:6789/0 conn(0x7f0ab40d0620 legacy=0x7f0ab40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.384+0000 7f0ab2ffd640 1 -- 172.21.15.37:0/2432067243 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40d2450 con 0x7f0ab40d1cd0 2024-04-25T08:59:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab2ffd640 1 --2- 172.21.15.37:0/2432067243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ab40d1cd0 0x7f0ab40d00e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c006fd0 tx=0x7f0a9c01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56704784710f9e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab27fc640 1 -- 172.21.15.37:0/2432067243 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0ab650 con 0x7f0ab40d1cd0 2024-04-25T08:59:16.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab27fc640 1 -- 172.21.15.37:0/2432067243 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c0b0070 con 0x7f0ab40d1cd0 2024-04-25T08:59:16.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab27fc640 1 -- 172.21.15.37:0/2432067243 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0aba10 con 0x7f0ab40d1cd0 2024-04-25T08:59:16.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 -- 172.21.15.37:0/2432067243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ab40d1cd0 msgr2=0x7f0ab40d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 --2- 172.21.15.37:0/2432067243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ab40d1cd0 0x7f0ab40d00e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c006fd0 tx=0x7f0a9c01c540 comp rx=0 tx=0).stop 2024-04-25T08:59:16.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 -- 172.21.15.37:0/2432067243 shutdown_connections 2024-04-25T08:59:16.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 --2- 172.21.15.37:0/2432067243 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ab40d1cd0 0x7f0ab40d00e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 --2- 172.21.15.37:0/2432067243 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ab415da80 0x7f0ab40d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 -- 172.21.15.37:0/2432067243 >> 172.21.15.37:0/2432067243 conn(0x7f0ab408b000 msgr2=0x7f0ab40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:16.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 -- 172.21.15.37:0/2432067243 shutdown_connections 2024-04-25T08:59:16.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 -- 172.21.15.37:0/2432067243 wait complete. 2024-04-25T08:59:16.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 Processor -- start 2024-04-25T08:59:16.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 -- start start 2024-04-25T08:59:16.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab40d0620 0x7f0ab4158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ab40d1cd0 0x7f0ab415fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ab415da80 0x7f0ab40b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab416a730 con 0x7f0ab40d0620 2024-04-25T08:59:16.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab416a5b0 con 0x7f0ab40d1cd0 2024-04-25T08:59:16.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab9a73640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab416a8b0 con 0x7f0ab415da80 2024-04-25T08:59:16.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab40d0620 0x7f0ab4158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab40d0620 0x7f0ab4158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38502/0 (socket says 172.21.15.37:38502) 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab37fe640 1 -- 172.21.15.37:0/844607991 learned_addr learned my addr 172.21.15.37:0/844607991 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab3fff640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ab415da80 0x7f0ab40b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.388+0000 7f0ab2ffd640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ab40d1cd0 0x7f0ab415fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab37fe640 1 -- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ab415da80 msgr2=0x7f0ab40b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab37fe640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ab415da80 0x7f0ab40b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab37fe640 1 -- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ab40d1cd0 msgr2=0x7f0ab415fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab37fe640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ab40d1cd0 0x7f0ab415fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab37fe640 1 -- 172.21.15.37:0/844607991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40cc1c0 con 0x7f0ab40d0620 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab3fff640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ab415da80 0x7f0ab40b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab2ffd640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ab40d1cd0 0x7f0ab415fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab37fe640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab40d0620 0x7f0ab4158110 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00efc0 tx=0x7f0aac00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0aabfff640 1 -- 172.21.15.37:0/844607991 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac018020 con 0x7f0ab40d0620 2024-04-25T08:59:16.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0aabfff640 1 -- 172.21.15.37:0/844607991 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aac014480 con 0x7f0ab40d0620 2024-04-25T08:59:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab9a73640 1 -- 172.21.15.37:0/844607991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab40cc4a0 con 0x7f0ab40d0620 2024-04-25T08:59:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0aabfff640 1 -- 172.21.15.37:0/844607991 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aac014850 con 0x7f0ab40d0620 2024-04-25T08:59:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab9a73640 1 -- 172.21.15.37:0/844607991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab40cca60 con 0x7f0ab40d0620 2024-04-25T08:59:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab9a73640 1 -- 172.21.15.37:0/844607991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab404efa0 con 0x7f0ab40d0620 2024-04-25T08:59:16.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0aabfff640 1 -- 172.21.15.37:0/844607991 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0aac01b020 con 0x7f0ab40d0620 2024-04-25T08:59:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0aabfff640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a94066f90 0x7f0a94069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0aabfff640 1 -- 172.21.15.37:0/844607991 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f0aac085cc0 con 0x7f0ab40d0620 2024-04-25T08:59:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab2ffd640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a94066f90 0x7f0a94069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.392+0000 7f0ab2ffd640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a94066f90 0x7f0a94069450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0af000 tx=0x7f0a9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:16.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.396+0000 7f0aabfff640 1 -- 172.21.15.37:0/844607991 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0aac08c080 con 0x7f0ab40d0620 2024-04-25T08:59:16.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.592+0000 7f0ab9a73640 1 -- 172.21.15.37:0/844607991 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ab40d1750 con 0x7f0ab40d0620 2024-04-25T08:59:16.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.592+0000 7f0aabfff640 1 -- 172.21.15.37:0/844607991 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0aac022070 con 0x7f0ab40d0620 2024-04-25T08:59:16.586 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:59:16.586 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":60,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T08:58:55.924579+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7734},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7734":{"gid":7734,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/354285428","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":354285428},{"type":"v1","addr":"172.21.15.37:6835","nonce":354285428}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:59:16.587 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 60 2024-04-25T08:59:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 -- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a94066f90 msgr2=0x7f0a94069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a94066f90 0x7f0a94069450 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0af000 tx=0x7f0a9c009040 comp rx=0 tx=0).stop 2024-04-25T08:59:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 -- 172.21.15.37:0/844607991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab40d0620 msgr2=0x7f0ab4158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab40d0620 0x7f0ab4158110 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f0aac00efc0 tx=0x7f0aac00c490 comp rx=0 tx=0).stop 2024-04-25T08:59:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 -- 172.21.15.37:0/844607991 shutdown_connections 2024-04-25T08:59:16.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a94066f90 0x7f0a94069450 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ab415da80 0x7f0ab40b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ab40d1cd0 0x7f0ab415fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 --2- 172.21.15.37:0/844607991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ab40d0620 0x7f0ab4158110 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 -- 172.21.15.37:0/844607991 >> 172.21.15.37:0/844607991 conn(0x7f0ab408b000 msgr2=0x7f0ab41608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:16.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 -- 172.21.15.37:0/844607991 shutdown_connections 2024-04-25T08:59:16.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.596+0000 7f0ab9a73640 1 -- 172.21.15.37:0/844607991 wait complete. 2024-04-25T08:59:16.604 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T08:59:16.750 INFO:teuthology.orchestra.run.smithi037.stdout:{"message":"","return_code":0} 2024-04-25T08:59:16.751 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:59:16.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:59:16.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d21242640 1 Processor -- start 2024-04-25T08:59:16.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d21242640 1 -- start start 2024-04-25T08:59:16.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d21242640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0d1c0ce510 0x7f0d1c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d21242640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0d1c1451f0 0x7f0d1c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d21242640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1c0cee30 con 0x7f0d1c144ca0 2024-04-25T08:59:16.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d21242640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c0cc2b0 con 0x7f0d1c0ce510 2024-04-25T08:59:16.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d21242640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1c06e290 con 0x7f0d1c1451f0 2024-04-25T08:59:16.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d1ad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0d1c0ce510 0x7f0d1c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d1a59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0d1c144ca0 0x7f0d1c06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51842/0 (socket says 172.21.15.37:51842) 2024-04-25T08:59:16.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d1a59c640 1 -- 172.21.15.37:0/1857577799 learned_addr learned my addr 172.21.15.37:0/1857577799 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d1b59e640 1 --2- 172.21.15.37:0/1857577799 >> v2:172.21.15.203:3300/0 conn(0x7f0d1c1451f0 0x7f0d1c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d1ad9d640 1 -- 172.21.15.37:0/1857577799 >> v2:172.21.15.203:3300/0 conn(0x7f0d1c1451f0 msgr2=0x7f0d1c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d1ad9d640 1 --2- 172.21.15.37:0/1857577799 >> v2:172.21.15.203:3300/0 conn(0x7f0d1c1451f0 0x7f0d1c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d1ad9d640 1 -- 172.21.15.37:0/1857577799 >> v1:172.21.15.37:6789/0 conn(0x7f0d1c144ca0 legacy=0x7f0d1c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d19d9b640 1 -- 172.21.15.37:0/1857577799 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298677450 0 0) 0x7f0d1c0cee30 con 0x7f0d1c144ca0 2024-04-25T08:59:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.884+0000 7f0d1ad9d640 1 -- 172.21.15.37:0/1857577799 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c0ccc70 con 0x7f0d1c0ce510 2024-04-25T08:59:16.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1ad9d640 1 --2- 172.21.15.37:0/1857577799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c0ce510 0x7f0d1c0ce8f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0b48f0 tx=0x7f0d0c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de24790f60816155 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:16.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d19d9b640 1 -- 172.21.15.37:0/1857577799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0b5070 con 0x7f0d1c0ce510 2024-04-25T08:59:16.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d19d9b640 1 -- 172.21.15.37:0/1857577799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c0a8970 con 0x7f0d1c0ce510 2024-04-25T08:59:16.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d19d9b640 1 -- 172.21.15.37:0/1857577799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0a8d40 con 0x7f0d1c0ce510 2024-04-25T08:59:16.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 -- 172.21.15.37:0/1857577799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c0ce510 msgr2=0x7f0d1c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 --2- 172.21.15.37:0/1857577799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c0ce510 0x7f0d1c0ce8f0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0b48f0 tx=0x7f0d0c01ec70 comp rx=0 tx=0).stop 2024-04-25T08:59:16.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 -- 172.21.15.37:0/1857577799 shutdown_connections 2024-04-25T08:59:16.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 --2- 172.21.15.37:0/1857577799 >> v2:172.21.15.203:3300/0 conn(0x7f0d1c1451f0 0x7f0d1c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 --2- 172.21.15.37:0/1857577799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c0ce510 0x7f0d1c0ce8f0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 -- 172.21.15.37:0/1857577799 >> 172.21.15.37:0/1857577799 conn(0x7f0d1c0b11a0 msgr2=0x7f0d1c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 -- 172.21.15.37:0/1857577799 shutdown_connections 2024-04-25T08:59:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 -- 172.21.15.37:0/1857577799 wait complete. 2024-04-25T08:59:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 Processor -- start 2024-04-25T08:59:16.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 -- start start 2024-04-25T08:59:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d1c0ce510 0x7f0d1c1f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c144ca0 0x7f0d1c15afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c1451f0 0x7f0d1c0c9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c06ee30 con 0x7f0d1c144ca0 2024-04-25T08:59:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c06df20 con 0x7f0d1c1451f0 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d21242640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1c0cc900 con 0x7f0d1c0ce510 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c144ca0 0x7f0d1c15afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c144ca0 0x7f0d1c15afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38522/0 (socket says 172.21.15.37:38522) 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1a59c640 1 -- 172.21.15.37:0/4255573509 learned_addr learned my addr 172.21.15.37:0/4255573509 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1b59e640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c1451f0 0x7f0d1c0c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1ad9d640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d1c0ce510 0x7f0d1c1f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1a59c640 1 -- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d1c0ce510 msgr2=0x7f0d1c1f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1a59c640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d1c0ce510 0x7f0d1c1f5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1a59c640 1 -- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c1451f0 msgr2=0x7f0d1c0c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1a59c640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c1451f0 0x7f0d1c0c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1a59c640 1 -- 172.21.15.37:0/4255573509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1c15b510 con 0x7f0d1c144ca0 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.888+0000 7f0d1b59e640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c1451f0 0x7f0d1c0c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d1a59c640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c144ca0 0x7f0d1c15afd0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000c4b0 tx=0x7f0d1000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d037fe640 1 -- 172.21.15.37:0/4255573509 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10018070 con 0x7f0d1c144ca0 2024-04-25T08:59:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d037fe640 1 -- 172.21.15.37:0/4255573509 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d10007500 con 0x7f0d1c144ca0 2024-04-25T08:59:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d21242640 1 -- 172.21.15.37:0/4255573509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1c131ba0 con 0x7f0d1c144ca0 2024-04-25T08:59:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d037fe640 1 -- 172.21.15.37:0/4255573509 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10004a30 con 0x7f0d1c144ca0 2024-04-25T08:59:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d21242640 1 -- 172.21.15.37:0/4255573509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d1c132160 con 0x7f0d1c144ca0 2024-04-25T08:59:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d21242640 1 -- 172.21.15.37:0/4255573509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1c04efa0 con 0x7f0d1c144ca0 2024-04-25T08:59:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d037fe640 1 -- 172.21.15.37:0/4255573509 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0d1001b070 con 0x7f0d1c144ca0 2024-04-25T08:59:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d037fe640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0d08066f90 0x7f0d08069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d037fe640 1 -- 172.21.15.37:0/4255573509 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f0d10084520 con 0x7f0d1c144ca0 2024-04-25T08:59:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d1ad9d640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0d08066f90 0x7f0d08069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.892+0000 7f0d1ad9d640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0d08066f90 0x7f0d08069450 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0b2000 tx=0x7f0d0c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:16.896+0000 7f0d037fe640 1 -- 172.21.15.37:0/4255573509 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0d100553f0 con 0x7f0d1c144ca0 2024-04-25T08:59:17.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.088+0000 7f0d21242640 1 -- 172.21.15.37:0/4255573509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0d1c0b1cc0 con 0x7f0d1c144ca0 2024-04-25T08:59:17.082 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T08:59:17.092+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:59:17.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.092+0000 7f0d037fe640 1 -- 172.21.15.37:0/4255573509 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0d10054cf0 con 0x7f0d1c144ca0 2024-04-25T08:59:17.084 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:59:17.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 -- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0d08066f90 msgr2=0x7f0d08069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0d08066f90 0x7f0d08069450 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c0b2000 tx=0x7f0d0c008040 comp rx=0 tx=0).stop 2024-04-25T08:59:17.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 -- 172.21.15.37:0/4255573509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c144ca0 msgr2=0x7f0d1c15afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c144ca0 0x7f0d1c15afd0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f0d1000c4b0 tx=0x7f0d1000c970 comp rx=0 tx=0).stop 2024-04-25T08:59:17.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 -- 172.21.15.37:0/4255573509 shutdown_connections 2024-04-25T08:59:17.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0d08066f90 0x7f0d08069450 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0d1c1451f0 0x7f0d1c0c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d1c144ca0 0x7f0d1c15afd0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 --2- 172.21.15.37:0/4255573509 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0d1c0ce510 0x7f0d1c1f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 -- 172.21.15.37:0/4255573509 >> 172.21.15.37:0/4255573509 conn(0x7f0d1c0b11a0 msgr2=0x7f0d1c0c6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:17.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 -- 172.21.15.37:0/4255573509 shutdown_connections 2024-04-25T08:59:17.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.096+0000 7f0d21242640 1 -- 172.21.15.37:0/4255573509 wait complete. 2024-04-25T08:59:17.089 INFO:tasks.ceph.mds.a.smithi037.stderr: -4071> 2024-04-25T08:58:58.452+0000 7fa6eda26640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:59:17.104 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-04-25T08:59:17.119 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T08:59:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.148+0000 7f733c3ee080 1 Processor -- start 2024-04-25T08:59:17.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.148+0000 7f733c3ee080 1 -- start start 2024-04-25T08:59:17.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bf01f2c110 0x55bf01eff300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf01f2c4f0 0x55bf01f03030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01fba340 0x55bf01fb7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf01f30a30 con 0x55bf01fba340 2024-04-25T08:59:17.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf01ea69f0 con 0x55bf01f2c4f0 2024-04-25T08:59:17.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf01f49ae0 con 0x55bf01f2c110 2024-04-25T08:59:17.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733a70d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01fba340 0x55bf01fb7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733a70d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01fba340 0x55bf01fb7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38526/0 (socket says 172.21.15.37:38526) 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733a70d640 1 -- 172.21.15.37:0/2440150434 learned_addr learned my addr 172.21.15.37:0/2440150434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733970b640 1 --2- 172.21.15.37:0/2440150434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf01f2c4f0 0x55bf01f03030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f7339f0c640 1 --2- 172.21.15.37:0/2440150434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bf01f2c110 0x55bf01eff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733a70d640 1 -- 172.21.15.37:0/2440150434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bf01f2c110 msgr2=0x55bf01eff300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733a70d640 1 --2- 172.21.15.37:0/2440150434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bf01f2c110 0x55bf01eff300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733a70d640 1 -- 172.21.15.37:0/2440150434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf01f2c4f0 msgr2=0x55bf01f03030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733a70d640 1 --2- 172.21.15.37:0/2440150434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf01f2c4f0 0x55bf01f03030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733a70d640 1 -- 172.21.15.37:0/2440150434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf01f450f0 con 0x55bf01fba340 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733970b640 1 --2- 172.21.15.37:0/2440150434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf01f2c4f0 0x55bf01f03030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733a70d640 1 --2- 172.21.15.37:0/2440150434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01fba340 0x55bf01fb7b60 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f733400a590 tx=0x7f73340033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac729e75803aac8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f732bfff640 1 -- 172.21.15.37:0/2440150434 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73340b0040 con 0x55bf01fba340 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f732bfff640 1 -- 172.21.15.37:0/2440150434 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73340ab620 con 0x55bf01fba340 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f732bfff640 1 -- 172.21.15.37:0/2440150434 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73340aba10 con 0x55bf01fba340 2024-04-25T08:59:17.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- 172.21.15.37:0/2440150434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01fba340 msgr2=0x55bf01fb7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 --2- 172.21.15.37:0/2440150434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01fba340 0x55bf01fb7b60 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f733400a590 tx=0x7f73340033b0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- 172.21.15.37:0/2440150434 shutdown_connections 2024-04-25T08:59:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 --2- 172.21.15.37:0/2440150434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01fba340 0x55bf01fb7b60 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 --2- 172.21.15.37:0/2440150434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf01f2c4f0 0x55bf01f03030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 --2- 172.21.15.37:0/2440150434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bf01f2c110 0x55bf01eff300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- 172.21.15.37:0/2440150434 >> 172.21.15.37:0/2440150434 conn(0x55bf01f2aa80 msgr2=0x55bf01f44190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- 172.21.15.37:0/2440150434 shutdown_connections 2024-04-25T08:59:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- 172.21.15.37:0/2440150434 wait complete. 2024-04-25T08:59:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 Processor -- start 2024-04-25T08:59:17.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- start start 2024-04-25T08:59:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01f2c110 0x55bf01fb9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bf01f2c4f0 0x55bf01fb4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf01fba340 0x55bf01fb5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf01f45630 con 0x55bf01f2c110 2024-04-25T08:59:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf01f45390 con 0x55bf01fba340 2024-04-25T08:59:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf01f458d0 con 0x55bf01f2c4f0 2024-04-25T08:59:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f7339f0c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01f2c110 0x55bf01fb9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f7339f0c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01f2c110 0x55bf01fb9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38532/0 (socket says 172.21.15.37:38532) 2024-04-25T08:59:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f7339f0c640 1 -- 172.21.15.37:0/1131189775 learned_addr learned my addr 172.21.15.37:0/1131189775 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f7339f0c640 1 -- 172.21.15.37:0/1131189775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bf01f2c4f0 msgr2=0x55bf01fb4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f7339f0c640 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55bf01f2c4f0 0x55bf01fb4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f7339f0c640 1 -- 172.21.15.37:0/1131189775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf01fba340 msgr2=0x55bf01fb5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f7339f0c640 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55bf01fba340 0x55bf01fb5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f7339f0c640 1 -- 172.21.15.37:0/1131189775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf01fd9760 con 0x55bf01f2c110 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f7339f0c640 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01f2c110 0x55bf01fb9080 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f732c0086c0 tx=0x7f732c008b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f732affd640 1 -- 172.21.15.37:0/1131189775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c006b10 con 0x55bf01f2c110 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f732affd640 1 -- 172.21.15.37:0/1131189775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f732c019070 con 0x55bf01f2c110 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- 172.21.15.37:0/1131189775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55bf01fd9a00 con 0x55bf01f2c110 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f732affd640 1 -- 172.21.15.37:0/1131189775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732c0143f0 con 0x55bf01f2c110 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.152+0000 7f733c3ee080 1 -- 172.21.15.37:0/1131189775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf01fd9f40 con 0x55bf01f2c110 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.156+0000 7f732affd640 1 -- 172.21.15.37:0/1131189775 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f732c014590 con 0x55bf01f2c110 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.156+0000 7f732affd640 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f731c0659a0 0x7f731c067e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.156+0000 7f732affd640 1 -- 172.21.15.37:0/1131189775 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f732c0842c0 con 0x55bf01f2c110 2024-04-25T08:59:17.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.156+0000 7f733c3ee080 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55bf01fb95c0 0x55bf01fd8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.156+0000 7f733c3ee080 1 -- 172.21.15.37:0/1131189775 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:1 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e98) v9 -- 0x55bf01fd64b0 con 0x55bf01fb95c0 2024-04-25T08:59:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.156+0000 7f733970b640 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f731c0659a0 0x7f731c067e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.156+0000 7f733a70d640 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55bf01fb95c0 0x55bf01fd8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.156+0000 7f733a70d640 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55bf01fb95c0 0x55bf01fd8f10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.156+0000 7f733970b640 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f731c0659a0 0x7f731c067e60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7330002ed0 tx=0x7f7330002450 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733a70d640 1 -- 172.21.15.37:0/1131189775 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v98'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f73340ab890 con 0x55bf01fb95c0 2024-04-25T08:59:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733c3ee080 1 -- 172.21.15.37:0/1131189775 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55bf01fb95c0 msgr2=0x55bf01fd8f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733c3ee080 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55bf01fb95c0 0x55bf01fd8f10 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733c3ee080 1 -- 172.21.15.37:0/1131189775 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f731c0659a0 msgr2=0x7f731c067e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733c3ee080 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f731c0659a0 0x7f731c067e60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7330002ed0 tx=0x7f7330002450 comp rx=0 tx=0).stop 2024-04-25T08:59:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733c3ee080 1 -- 172.21.15.37:0/1131189775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01f2c110 msgr2=0x55bf01fb9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733c3ee080 1 --2- 172.21.15.37:0/1131189775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bf01f2c110 0x55bf01fb9080 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f732c0086c0 tx=0x7f732c008b80 comp rx=0 tx=0).stop 2024-04-25T08:59:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733a70d640 1 -- 172.21.15.37:0/1131189775 reap_dead start 2024-04-25T08:59:17.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733c3ee080 1 -- 172.21.15.37:0/1131189775 shutdown_connections 2024-04-25T08:59:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733c3ee080 1 -- 172.21.15.37:0/1131189775 >> 172.21.15.37:0/1131189775 conn(0x55bf01f2aa80 msgr2=0x55bf01fb7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733c3ee080 1 -- 172.21.15.37:0/1131189775 shutdown_connections 2024-04-25T08:59:17.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.160+0000 7f733c3ee080 1 -- 172.21.15.37:0/1131189775 wait complete. 2024-04-25T08:59:17.155 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset 2024-04-25T08:59:17.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 1 -- start start 2024-04-25T08:59:17.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 1 Processor -- start 2024-04-25T08:59:17.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07b91800 0x558e07d4a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e07b91400 0x558e07d4ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e07b91000 0x558e07d4b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08954180 con 0x558e07b91800 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08954300 con 0x558e07b91000 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e08954480 con 0x558e07b91400 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541ec79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07b91800 0x558e07d4a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541ec79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07b91800 0x558e07d4a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38544/0 (socket says 172.21.15.37:38544) 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541ec79640 1 -- 172.21.15.37:0/2809313324 learned_addr learned my addr 172.21.15.37:0/2809313324 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541ec79640 1 -- 172.21.15.37:0/2809313324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e07b91400 msgr2=0x558e07d4ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541ec79640 1 --2- 172.21.15.37:0/2809313324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e07b91400 0x558e07d4ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541f47a640 1 --2- 172.21.15.37:0/2809313324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e07b91000 0x558e07d4b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541ec79640 1 -- 172.21.15.37:0/2809313324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e07b91000 msgr2=0x558e07d4b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541ec79640 1 --2- 172.21.15.37:0/2809313324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e07b91000 0x558e07d4b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541ec79640 1 -- 172.21.15.37:0/2809313324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e0893e960 con 0x558e07b91800 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541ec79640 1 --2- 172.21.15.37:0/2809313324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07b91800 0x558e07d4a580 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x558e089f1380 tx=0x558e07c54ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99388ec2bb697614 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541c474640 1 -- 172.21.15.37:0/2809313324 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e07bc64e0 con 0x558e07b91800 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541c474640 1 -- 172.21.15.37:0/2809313324 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558e07bda1c0 con 0x558e07b91800 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541c474640 1 -- 172.21.15.37:0/2809313324 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e07bc6b60 con 0x558e07b91800 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 1 -- 172.21.15.37:0/2809313324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e0893ed20 con 0x558e07b91800 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 1 -- 172.21.15.37:0/2809313324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x558e0893ef00 con 0x558e07b91800 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 4 waiting for MDS map... 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541c474640 1 -- 172.21.15.37:0/2809313324 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x558e08a96600 con 0x558e07b91800 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f541c474640 1 -- 172.21.15.37:0/2809313324 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 61) v1 ==== 2723+0+0 (secure 0 0 0) 0x558e07be7180 con 0x558e07b91800 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.236+0000 7f54204ba400 4 Got MDS map 61 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 10 main: JournalTool::main 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 4 main: JournalTool: connecting to RADOS... 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 Processor -- start 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- start start 2024-04-25T08:59:17.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1000 0x558e07d4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e08aa1400 0x558e07d4c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08954600 con 0x558e08aa0c00 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08954780 con 0x558e08aa1400 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e08954900 con 0x558e08aa1000 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38546/0 (socket says 172.21.15.37:38546) 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 -- 172.21.15.37:0/2134040274 learned_addr learned my addr 172.21.15.37:0/2134040274 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 -- 172.21.15.37:0/2134040274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1000 msgr2=0x558e07d4bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 --2- 172.21.15.37:0/2134040274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1000 0x558e07d4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 -- 172.21.15.37:0/2134040274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e08aa1400 msgr2=0x558e07d4c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 --2- 172.21.15.37:0/2134040274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e08aa1400 0x558e07d4c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 -- 172.21.15.37:0/2134040274 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e0893ed20 con 0x558e08aa0c00 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 --2- 172.21.15.37:0/2134040274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4b600 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x558e089f09f0 tx=0x558e0894c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416dfbb6319a2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541a470640 1 -- 172.21.15.37:0/2134040274 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e07bc69c0 con 0x558e08aa0c00 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541a470640 1 -- 172.21.15.37:0/2134040274 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558e07bdac40 con 0x558e08aa0c00 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541a470640 1 -- 172.21.15.37:0/2134040274 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e07bc64e0 con 0x558e08aa0c00 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- 172.21.15.37:0/2134040274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 msgr2=0x558e07d4b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 --2- 172.21.15.37:0/2134040274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4b600 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x558e089f09f0 tx=0x558e0894c000 comp rx=0 tx=0).stop 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- 172.21.15.37:0/2134040274 shutdown_connections 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 --2- 172.21.15.37:0/2134040274 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e08aa1400 0x558e07d4c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 --2- 172.21.15.37:0/2134040274 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1000 0x558e07d4bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 --2- 172.21.15.37:0/2134040274 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4b600 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- 172.21.15.37:0/2134040274 >> 172.21.15.37:0/2134040274 conn(0x558e07b90c00 msgr2=0x558e07bbf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- 172.21.15.37:0/2134040274 shutdown_connections 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- 172.21.15.37:0/2134040274 wait complete. 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 Processor -- start 2024-04-25T08:59:17.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- start start 2024-04-25T08:59:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e08aa1000 0x558e07d4cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1400 0x558e07d4c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08955b00 con 0x558e08aa0c00 2024-04-25T08:59:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08955980 con 0x558e08aa1000 2024-04-25T08:59:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f54204ba400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e08955800 con 0x558e08aa1400 2024-04-25T08:59:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38552/0 (socket says 172.21.15.37:38552) 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 -- 172.21.15.37:0/599824521 learned_addr learned my addr 172.21.15.37:0/599824521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 -- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1400 msgr2=0x558e07d4c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1400 0x558e07d4c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 -- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e08aa1000 msgr2=0x558e07d4cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e08aa1000 0x558e07d4cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.240+0000 7f541f47a640 1 -- 172.21.15.37:0/599824521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e0893fa40 con 0x558e08aa0c00 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541f47a640 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4d180 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x558e08aba810 tx=0x558e0894c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f5418c6d640 1 -- 172.21.15.37:0/599824521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e07bc6b60 con 0x558e08aa0c00 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f5418c6d640 1 -- 172.21.15.37:0/599824521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558e07bdaa80 con 0x558e08aa0c00 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 1 -- 172.21.15.37:0/599824521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558e0893fc20 con 0x558e08aa0c00 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f5418c6d640 1 -- 172.21.15.37:0/599824521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e07bc6000 con 0x558e08aa0c00 2024-04-25T08:59:17.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 1 -- 172.21.15.37:0/599824521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e0893e1e0 con 0x558e08aa0c00 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 4 main: JournalTool: resolving pool 13 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f5418c6d640 1 -- 172.21.15.37:0/599824521 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x558e07ec0c00 con 0x558e08aa0c00 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f5418c6d640 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x558e07fa2000 0x558e07d4d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f5418c6d640 1 -- 172.21.15.37:0/599824521 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x558e07ec2600 con 0x558e08aa0c00 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 4 main: JournalTool: creating IoCtx.. 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 4 main: Executing for rank 0 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541ec79640 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x558e07fa2000 0x558e07d4d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 1 -- start start 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 1 Processor -- start 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541ec79640 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x558e07fa2000 0x558e07d4d700 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x558e089f1ce0 tx=0x558e07c54a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1c00 0x558e07d4c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07fa2400 0x558e07d4bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e07fa2800 0x558e07d4b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08955500 con 0x558e07fa2400 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e08954780 con 0x558e07fa2800 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f54204ba400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e08954a80 con 0x558e08aa1c00 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541ec79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07fa2400 0x558e07d4bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541ec79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07fa2400 0x558e07d4bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38566/0 (socket says 172.21.15.37:38566) 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541ec79640 1 -- 172.21.15.37:0/3592009711 learned_addr learned my addr 172.21.15.37:0/3592009711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541f47a640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1c00 0x558e07d4c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541e478640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e07fa2800 0x558e07d4b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541ec79640 1 -- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1c00 msgr2=0x558e07d4c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541ec79640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1c00 0x558e07d4c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541ec79640 1 -- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e07fa2800 msgr2=0x558e07d4b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541ec79640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e07fa2800 0x558e07d4b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541ec79640 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e080025a0 con 0x558e07fa2400 2024-04-25T08:59:17.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541e478640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e07fa2800 0x558e07d4b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:17.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f541ec79640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07fa2400 0x558e07d4bb80 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x558e07f61620 tx=0x558e07fb5920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df827d586202162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f5415c67640 1 -- 172.21.15.37:0/3592009711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e07bc6340 con 0x558e07fa2400 2024-04-25T08:59:17.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f5415c67640 1 -- 172.21.15.37:0/3592009711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558e07bdae00 con 0x558e07fa2400 2024-04-25T08:59:17.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.244+0000 7f5415c67640 1 -- 172.21.15.37:0/3592009711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e07ef44e0 con 0x558e07fa2400 2024-04-25T08:59:17.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e08002780 con 0x558e07fa2400 2024-04-25T08:59:17.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x558e08002960 con 0x558e07fa2400 2024-04-25T08:59:17.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f54204ba400 4 waiting for MDS map... 2024-04-25T08:59:17.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f5415c67640 1 -- 172.21.15.37:0/3592009711 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x558e08010200 con 0x558e07fa2400 2024-04-25T08:59:17.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f5415c67640 1 -- 172.21.15.37:0/3592009711 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 61) v1 ==== 2723+0+0 (secure 0 0 0) 0x558e07be7880 con 0x558e07fa2400 2024-04-25T08:59:17.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f54204ba400 4 Got MDS map 61 2024-04-25T08:59:17.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x558e07fa2c00 0x558e08028000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e07fa3000 con 0x558e07fa2c00 2024-04-25T08:59:17.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f541f47a640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x558e07fa2c00 0x558e08028000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f541f47a640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x558e07fa2c00 0x558e08028000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=6daeae7997c8545f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f541f47a640 1 -- 172.21.15.37:0/3592009711 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x558e07d0c000 con 0x558e07fa2c00 2024-04-25T08:59:17.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x558e07fa3400 0x558e08028580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e07fa3800 con 0x558e07fa3400 2024-04-25T08:59:17.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f541e478640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x558e07fa3400 0x558e08028580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.248+0000 7f541e478640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x558e07fa3400 0x558e08028580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=23727bcd547257ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.252+0000 7f541e478640 1 -- 172.21.15.37:0/3592009711 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x558e07d0c500 con 0x558e07fa3400 2024-04-25T08:59:17.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.252+0000 7f5414464640 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e0800f400 con 0x558e07fa3400 2024-04-25T08:59:17.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.252+0000 7f5414464640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x558e0800f800 0x558e0802a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.252+0000 7f5414464640 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e0800fc00 con 0x558e0800f800 2024-04-25T08:59:17.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.252+0000 7f541e478640 1 -- 172.21.15.37:0/3592009711 <== osd.0 v2:172.21.15.37:6824/1116141266 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+16 (crc 0 0 0) 0x558e07d0c500 con 0x558e07fa3400 2024-04-25T08:59:17.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.252+0000 7f541f47a640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x558e0800f800 0x558e0802a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.252+0000 7f541f47a640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x558e0800f800 0x558e0802a680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=619aa5c55989664e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.252+0000 7f541f47a640 1 -- 172.21.15.37:0/3592009711 <== osd.4 v2:172.21.15.203:6802/714177927 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558e07d0c000 con 0x558e0800f800 2024-04-25T08:59:17.249 INFO:teuthology.orchestra.run.smithi037.stdout:old journal was 4209834~892 2024-04-25T08:59:17.249 INFO:teuthology.orchestra.run.smithi037.stdout:new journal start will be 8388608 (4177882 bytes past old end) 2024-04-25T08:59:17.250 INFO:teuthology.orchestra.run.smithi037.stdout:writing journal head 2024-04-25T08:59:17.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.252+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:5 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e07fa3c00 con 0x558e07fa3400 2024-04-25T08:59:17.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f541e478640 1 -- 172.21.15.37:0/3592009711 <== osd.0 v2:172.21.15.37:6824/1116141266 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v98'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x558e07d0c500 con 0x558e07fa3400 2024-04-25T08:59:17.250 INFO:teuthology.orchestra.run.smithi037.stdout:writing EResetJournal entry 2024-04-25T08:59:17.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:6 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e0800f400 con 0x558e0800f800 2024-04-25T08:59:17.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x558e07fa3800 0x558e08028b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:7 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e0804a000 con 0x558e07fa3800 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:8 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e0804a400 con 0x558e07fa2c00 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x558e0804a800 0x558e08029080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:9 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e0804ac00 con 0x558e0804a800 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:10 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e0804b000 con 0x558e0800f800 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x558e0804b400 0x558e08029600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:11 13.1d 13:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e0804b800 con 0x558e0804b400 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f541e478640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x558e0804a800 0x558e08029080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f541ec79640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x558e07fa3800 0x558e08028b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f541f47a640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x558e0804b400 0x558e08029600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f541e478640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x558e0804a800 0x558e08029080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=d24413fd1deba64d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f541ec79640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x558e07fa3800 0x558e08028b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=59846950a22c46d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f541f47a640 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x558e0804b400 0x558e08029600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=3ed51b563697ee41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f541e478640 1 -- 172.21.15.37:0/3592009711 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(9 200.00000005 [delete] v98'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558e07d0c500 con 0x558e0804a800 2024-04-25T08:59:17.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f541f47a640 1 -- 172.21.15.37:0/3592009711 <== osd.6 v2:172.21.15.203:6800/467003745 2 ==== osd_op_reply(8 200.00000004 [delete] v98'7 uv6 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558e07d0c000 con 0x558e07fa2c00 2024-04-25T08:59:17.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.256+0000 7f541f47a640 1 -- 172.21.15.37:0/3592009711 <== osd.4 v2:172.21.15.203:6802/714177927 2 ==== osd_op_reply(6 200.00000002 [delete] v98'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558e07d0c000 con 0x558e0800f800 2024-04-25T08:59:17.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.260+0000 7f541f47a640 1 -- 172.21.15.37:0/3592009711 <== osd.4 v2:172.21.15.203:6802/714177927 3 ==== osd_op_reply(10 200.00000006 [delete] v98'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558e07d0c000 con 0x558e0800f800 2024-04-25T08:59:17.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.264+0000 7f541f47a640 1 -- 172.21.15.37:0/3592009711 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== osd_op_reply(11 200.00000007 [delete] v98'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558e07d0c000 con 0x558e0804b400 2024-04-25T08:59:17.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.264+0000 7f541ec79640 1 -- 172.21.15.37:0/3592009711 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== osd_op_reply(7 200.00000003 [delete] v98'4 uv3 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x558e07d0c780 con 0x558e07fa3800 2024-04-25T08:59:17.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.264+0000 7f5414464640 1 -- 172.21.15.37:0/3592009711 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:12 13.1b 13:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e98) v9 -- 0x558e0800fc00 con 0x558e0800f800 2024-04-25T08:59:17.259 INFO:teuthology.orchestra.run.smithi037.stdout:done 2024-04-25T08:59:17.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f541f47a640 1 -- 172.21.15.37:0/3592009711 <== osd.4 v2:172.21.15.203:6802/714177927 4 ==== osd_op_reply(12 200.00000002 [write 0~48] v98'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x558e07d0c000 con 0x558e0800f800 2024-04-25T08:59:17.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x558e07fa3400 msgr2=0x558e08028580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x558e07fa3400 0x558e08028580 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x558e0804a800 msgr2=0x558e08029080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x558e0804a800 0x558e08029080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x558e0800f800 msgr2=0x558e0802a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x558e0800f800 0x558e0802a680 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x558e07fa3800 msgr2=0x558e08028b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x558e07fa3800 0x558e08028b00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x558e07fa2c00 msgr2=0x558e08028000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x558e07fa2c00 0x558e08028000 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x558e0804b400 msgr2=0x558e08029600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x558e0804b400 0x558e08029600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f541e478640 1 -- 172.21.15.37:0/3592009711 reap_dead start 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f541e478640 1 -- 172.21.15.37:0/3592009711 reap_dead start 2024-04-25T08:59:17.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07fa2400 msgr2=0x558e07d4bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07fa2400 0x558e07d4bb80 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x558e07f61620 tx=0x558e07fb5920 comp rx=0 tx=0).stop 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 shutdown_connections 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/3592009711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07fa2400 0x558e07d4bb80 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 >> 172.21.15.37:0/3592009711 conn(0x558e08aa1800 msgr2=0x558e07ffc000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 shutdown_connections 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/3592009711 wait complete. 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x558e07fa2000 msgr2=0x558e07d4d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x558e07fa2000 0x558e07d4d700 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x558e089f1ce0 tx=0x558e07c54a80 comp rx=0 tx=0).stop 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/599824521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 msgr2=0x558e07d4d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4d180 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x558e08aba810 tx=0x558e0894c300 comp rx=0 tx=0).stop 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/599824521 shutdown_connections 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x558e07fa2000 0x558e07d4d700 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e08aa1400 0x558e07d4c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e08aa1000 0x558e07d4cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/599824521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e08aa0c00 0x558e07d4d180 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/599824521 >> 172.21.15.37:0/599824521 conn(0x558e07b90c00 msgr2=0x558e07bbf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/599824521 shutdown_connections 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/599824521 wait complete. 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/2809313324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07b91800 msgr2=0x558e07d4a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/2809313324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07b91800 0x558e07d4a580 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x558e089f1380 tx=0x558e07c54ea0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/2809313324 shutdown_connections 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/2809313324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x558e07b91000 0x558e07d4b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/2809313324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x558e07b91400 0x558e07d4ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 --2- 172.21.15.37:0/2809313324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e07b91800 0x558e07d4a580 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/2809313324 >> 172.21.15.37:0/2809313324 conn(0x558e07b91c00 msgr2=0x558e07bbf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/2809313324 shutdown_connections 2024-04-25T08:59:17.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.268+0000 7f54204ba400 1 -- 172.21.15.37:0/2809313324 wait complete. 2024-04-25T08:59:17.265 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset'] in time 0:00:00.110627, result: old journal was 4209834~892 new journal start will be 8388608 (4177882 bytes past old end) writing journal head writing EResetJournal entry done 2024-04-25T08:59:17.265 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T08:59:17.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f8312640 1 Processor -- start 2024-04-25T08:59:17.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f8312640 1 -- start start 2024-04-25T08:59:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f8312640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f58f01634c0 0x7f58f01658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f8312640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f00ca4c0 con 0x7f58f00cf590 2024-04-25T08:59:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f8312640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f00cd690 con 0x7f58f015ecb0 2024-04-25T08:59:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f8312640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f00ca780 con 0x7f58f01634c0 2024-04-25T08:59:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f68af640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f58f01634c0 0x7f58f01658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f60ae640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f58f00cf590 0x7f58f00cf970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51864/0 (socket says 172.21.15.37:51864) 2024-04-25T08:59:17.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f68af640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f58f01634c0 0x7f58f01658b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38574/0 (socket says 172.21.15.37:38574) 2024-04-25T08:59:17.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f60ae640 1 -- 172.21.15.37:0/3022089919 learned_addr learned my addr 172.21.15.37:0/3022089919 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:17.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f68af640 1 -- 172.21.15.37:0/3022089919 >> v1:172.21.15.203:6789/0 conn(0x7f58f015ecb0 legacy=0x7f58f00c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:17.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f68af640 1 -- 172.21.15.37:0/3022089919 >> v1:172.21.15.37:6789/0 conn(0x7f58f00cf590 legacy=0x7f58f00cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f50ac640 1 -- 172.21.15.37:0/3022089919 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3719369418 0 0) 0x7f58f00ca4c0 con 0x7f58f00cf590 2024-04-25T08:59:17.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.444+0000 7f58f68af640 1 -- 172.21.15.37:0/3022089919 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f00cd970 con 0x7f58f01634c0 2024-04-25T08:59:17.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f68af640 1 --2- 172.21.15.37:0/3022089919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f01634c0 0x7f58f01658b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009d90 tx=0x7f58e801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc644cc44874a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f50ac640 1 -- 172.21.15.37:0/3022089919 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e80b2070 con 0x7f58f01634c0 2024-04-25T08:59:17.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f50ac640 1 -- 172.21.15.37:0/3022089919 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e80a57f0 con 0x7f58f01634c0 2024-04-25T08:59:17.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f50ac640 1 -- 172.21.15.37:0/3022089919 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e80a5bc0 con 0x7f58f01634c0 2024-04-25T08:59:17.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 -- 172.21.15.37:0/3022089919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f01634c0 msgr2=0x7f58f01658b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 --2- 172.21.15.37:0/3022089919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f01634c0 0x7f58f01658b0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009d90 tx=0x7f58e801c930 comp rx=0 tx=0).stop 2024-04-25T08:59:17.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 -- 172.21.15.37:0/3022089919 shutdown_connections 2024-04-25T08:59:17.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 --2- 172.21.15.37:0/3022089919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f01634c0 0x7f58f01658b0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 -- 172.21.15.37:0/3022089919 >> 172.21.15.37:0/3022089919 conn(0x7f58f006ae60 msgr2=0x7f58f006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:17.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 -- 172.21.15.37:0/3022089919 shutdown_connections 2024-04-25T08:59:17.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 -- 172.21.15.37:0/3022089919 wait complete. 2024-04-25T08:59:17.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 Processor -- start 2024-04-25T08:59:17.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 -- start start 2024-04-25T08:59:17.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58f00cf590 0x7f58f0072f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f015ecb0 0x7f58f0155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f01634c0 0x7f58f00cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00cdce0 con 0x7f58f015ecb0 2024-04-25T08:59:17.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00cd440 con 0x7f58f01634c0 2024-04-25T08:59:17.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f0168820 con 0x7f58f00cf590 2024-04-25T08:59:17.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f58ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f015ecb0 0x7f58f0155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f58ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f015ecb0 0x7f58f0155710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38584/0 (socket says 172.21.15.37:38584) 2024-04-25T08:59:17.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f58ad640 1 -- 172.21.15.37:0/3514782887 learned_addr learned my addr 172.21.15.37:0/3514782887 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f60ae640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58f00cf590 0x7f58f0072f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f68af640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f01634c0 0x7f58f00cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f58ad640 1 -- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58f00cf590 msgr2=0x7f58f0072f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f58ad640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58f00cf590 0x7f58f0072f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f58ad640 1 -- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f01634c0 msgr2=0x7f58f00cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f58ad640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f01634c0 0x7f58f00cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f58ad640 1 -- 172.21.15.37:0/3514782887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f00c35a0 con 0x7f58f015ecb0 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f58ad640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f015ecb0 0x7f58f0155710 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f58e000c450 tx=0x7f58e000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58e6ffd640 1 -- 172.21.15.37:0/3514782887 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0012830 con 0x7f58f015ecb0 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58e6ffd640 1 -- 172.21.15.37:0/3514782887 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e0017070 con 0x7f58f015ecb0 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 -- 172.21.15.37:0/3514782887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f00c3880 con 0x7f58f015ecb0 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58e6ffd640 1 -- 172.21.15.37:0/3514782887 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0012c30 con 0x7f58f015ecb0 2024-04-25T08:59:17.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.448+0000 7f58f8312640 1 -- 172.21.15.37:0/3514782887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f0155fe0 con 0x7f58f015ecb0 2024-04-25T08:59:17.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.452+0000 7f58f8312640 1 -- 172.21.15.37:0/3514782887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f004efa0 con 0x7f58f015ecb0 2024-04-25T08:59:17.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.452+0000 7f58e6ffd640 1 -- 172.21.15.37:0/3514782887 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58e001c460 con 0x7f58f015ecb0 2024-04-25T08:59:17.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.452+0000 7f58e6ffd640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58d4066f60 0x7f58d4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:17.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.452+0000 7f58e6ffd640 1 -- 172.21.15.37:0/3514782887 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f58e00850d0 con 0x7f58f015ecb0 2024-04-25T08:59:17.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.452+0000 7f58f60ae640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58d4066f60 0x7f58d4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:17.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.452+0000 7f58f60ae640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58d4066f60 0x7f58d4069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f58ec000f70 tx=0x7f58ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:17.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.456+0000 7f58e6ffd640 1 -- 172.21.15.37:0/3514782887 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f58e0055e80 con 0x7f58f015ecb0 2024-04-25T08:59:17.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:17.660+0000 7f58f8312640 1 -- 172.21.15.37:0/3514782887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f58f00b4170 con 0x7f58f015ecb0 2024-04-25T08:59:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.100+0000 7f58e6ffd640 1 -- 172.21.15.37:0/3514782887 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v62) v1 ==== 162+0+0 (secure 0 0 0) 0x7f58e0055780 con 0x7f58f015ecb0 2024-04-25T08:59:18.093 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T08:59:18.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 -- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58d4066f60 msgr2=0x7f58d4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58d4066f60 0x7f58d4069420 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f58ec000f70 tx=0x7f58ec009040 comp rx=0 tx=0).stop 2024-04-25T08:59:18.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 -- 172.21.15.37:0/3514782887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f015ecb0 msgr2=0x7f58f0155710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f015ecb0 0x7f58f0155710 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f58e000c450 tx=0x7f58e000c910 comp rx=0 tx=0).stop 2024-04-25T08:59:18.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 -- 172.21.15.37:0/3514782887 shutdown_connections 2024-04-25T08:59:18.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f58d4066f60 0x7f58d4069420 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58f01634c0 0x7f58f00cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58f015ecb0 0x7f58f0155710 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 --2- 172.21.15.37:0/3514782887 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58f00cf590 0x7f58f0072f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 -- 172.21.15.37:0/3514782887 >> 172.21.15.37:0/3514782887 conn(0x7f58f006ae60 msgr2=0x7f58f015e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:18.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 -- 172.21.15.37:0/3514782887 shutdown_connections 2024-04-25T08:59:18.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.104+0000 7f58f8312640 1 -- 172.21.15.37:0/3514782887 wait complete. 2024-04-25T08:59:18.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:18.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa95974d640 1 Processor -- start 2024-04-25T08:59:18.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa95974d640 1 -- start start 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa95974d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa954066e30 0x7fa95406c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa95974d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa9540af370 0x7fa9540ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa95974d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95406ec30 con 0x7fa95406bca0 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa95974d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9540669a0 con 0x7fa954066e30 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa95974d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95406c5c0 con 0x7fa9540af370 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa9527fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa954066e30 0x7fa95406c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa9527fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa954066e30 0x7fa95406c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38596/0 (socket says 172.21.15.37:38596) 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa9527fc640 1 -- 172.21.15.37:0/3284294649 learned_addr learned my addr 172.21.15.37:0/3284294649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa9537fe640 1 --2- 172.21.15.37:0/3284294649 >> v2:172.21.15.203:3300/0 conn(0x7fa9540af370 0x7fa9540ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa9527fc640 1 -- 172.21.15.37:0/3284294649 >> v2:172.21.15.203:3300/0 conn(0x7fa9540af370 msgr2=0x7fa9540ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa9527fc640 1 --2- 172.21.15.37:0/3284294649 >> v2:172.21.15.203:3300/0 conn(0x7fa9540af370 0x7fa9540ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa9527fc640 1 -- 172.21.15.37:0/3284294649 >> v1:172.21.15.203:6789/0 conn(0x7fa95406bca0 legacy=0x7fa9540cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:18.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.248+0000 7fa9527fc640 1 -- 172.21.15.37:0/3284294649 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95406f230 con 0x7fa954066e30 2024-04-25T08:59:18.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa9527fc640 1 --2- 172.21.15.37:0/3284294649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa954066e30 0x7fa95406c080 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa95406d160 tx=0x7fa94801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199cc604a0a6285f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:18.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa951ffb640 1 -- 172.21.15.37:0/3284294649 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9480b1070 con 0x7fa954066e30 2024-04-25T08:59:18.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa951ffb640 1 -- 172.21.15.37:0/3284294649 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9480ac410 con 0x7fa954066e30 2024-04-25T08:59:18.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa951ffb640 1 -- 172.21.15.37:0/3284294649 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9480ac7e0 con 0x7fa954066e30 2024-04-25T08:59:18.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 -- 172.21.15.37:0/3284294649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa954066e30 msgr2=0x7fa95406c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 --2- 172.21.15.37:0/3284294649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa954066e30 0x7fa95406c080 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fa95406d160 tx=0x7fa94801c390 comp rx=0 tx=0).stop 2024-04-25T08:59:18.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 -- 172.21.15.37:0/3284294649 shutdown_connections 2024-04-25T08:59:18.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 --2- 172.21.15.37:0/3284294649 >> v2:172.21.15.203:3300/0 conn(0x7fa9540af370 0x7fa9540ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 --2- 172.21.15.37:0/3284294649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa954066e30 0x7fa95406c080 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 -- 172.21.15.37:0/3284294649 >> 172.21.15.37:0/3284294649 conn(0x7fa954071810 msgr2=0x7fa954071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:18.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 -- 172.21.15.37:0/3284294649 shutdown_connections 2024-04-25T08:59:18.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 -- 172.21.15.37:0/3284294649 wait complete. 2024-04-25T08:59:18.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 Processor -- start 2024-04-25T08:59:18.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 -- start start 2024-04-25T08:59:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa954066e30 0x7fa9540ad970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa95406bca0 0x7fa954150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9540af370 0x7fa9540b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9541687b0 con 0x7fa9540af370 2024-04-25T08:59:18.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa954168630 con 0x7fa95406bca0 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa95974d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa954168930 con 0x7fa954066e30 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa9537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9540af370 0x7fa9540b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa9537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9540af370 0x7fa9540b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38598/0 (socket says 172.21.15.37:38598) 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa9537fe640 1 -- 172.21.15.37:0/2425389560 learned_addr learned my addr 172.21.15.37:0/2425389560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa9527fc640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa95406bca0 0x7fa954150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa952ffd640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa954066e30 0x7fa9540ad970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa9537fe640 1 -- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa954066e30 msgr2=0x7fa9540ad970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa9537fe640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa954066e30 0x7fa9540ad970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa9537fe640 1 -- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa95406bca0 msgr2=0x7fa954150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa9537fe640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa95406bca0 0x7fa954150f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa9537fe640 1 -- 172.21.15.37:0/2425389560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9541486a0 con 0x7fa9540af370 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.252+0000 7fa9537fe640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9540af370 0x7fa9540b2d30 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa94c007e40 tx=0x7fa94c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.256+0000 7fa93b7fe640 1 -- 172.21.15.37:0/2425389560 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c014620 con 0x7fa9540af370 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.256+0000 7fa93b7fe640 1 -- 172.21.15.37:0/2425389560 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c019070 con 0x7fa9540af370 2024-04-25T08:59:18.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.256+0000 7fa95974d640 1 -- 172.21.15.37:0/2425389560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa954165fb0 con 0x7fa9540af370 2024-04-25T08:59:18.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.256+0000 7fa93b7fe640 1 -- 172.21.15.37:0/2425389560 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c014a20 con 0x7fa9540af370 2024-04-25T08:59:18.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.256+0000 7fa95974d640 1 -- 172.21.15.37:0/2425389560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa954150a40 con 0x7fa9540af370 2024-04-25T08:59:18.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.256+0000 7fa95974d640 1 -- 172.21.15.37:0/2425389560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95404efa0 con 0x7fa9540af370 2024-04-25T08:59:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.256+0000 7fa93b7fe640 1 -- 172.21.15.37:0/2425389560 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa94c014bc0 con 0x7fa9540af370 2024-04-25T08:59:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.256+0000 7fa93b7fe640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa940066f60 0x7fa940069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.256+0000 7fa93b7fe640 1 -- 172.21.15.37:0/2425389560 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa94c085b10 con 0x7fa9540af370 2024-04-25T08:59:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.256+0000 7fa952ffd640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa940066f60 0x7fa940069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:18.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.256+0000 7fa952ffd640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa940066f60 0x7fa940069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa94400b810 tx=0x7fa944008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:18.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.260+0000 7fa93b7fe640 1 -- 172.21.15.37:0/2425389560 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa94c0568d0 con 0x7fa9540af370 2024-04-25T08:59:18.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.448+0000 7fa95974d640 1 -- 172.21.15.37:0/2425389560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa954072910 con 0x7fa9540af370 2024-04-25T08:59:18.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa93b7fe640 1 -- 172.21.15.37:0/2425389560 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fa94c0561d0 con 0x7fa9540af370 2024-04-25T08:59:18.442 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:59:18.443 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":63,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T08:59:18.110244+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":63,"state":"up:replay","state_seq":1,"addr":"172.21.15.37:6835/1462327322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1462327322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1462327322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:59:18.443 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 63 2024-04-25T08:59:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 -- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa940066f60 msgr2=0x7fa940069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa940066f60 0x7fa940069420 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa94400b810 tx=0x7fa944008040 comp rx=0 tx=0).stop 2024-04-25T08:59:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 -- 172.21.15.37:0/2425389560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9540af370 msgr2=0x7fa9540b2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9540af370 0x7fa9540b2d30 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fa94c007e40 tx=0x7fa94c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 -- 172.21.15.37:0/2425389560 shutdown_connections 2024-04-25T08:59:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa940066f60 0x7fa940069420 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9540af370 0x7fa9540b2d30 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa95406bca0 0x7fa954150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 --2- 172.21.15.37:0/2425389560 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa954066e30 0x7fa9540ad970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:18.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 -- 172.21.15.37:0/2425389560 >> 172.21.15.37:0/2425389560 conn(0x7fa954071810 msgr2=0x7fa9540b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 -- 172.21.15.37:0/2425389560 shutdown_connections 2024-04-25T08:59:18.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:18.452+0000 7fa95974d640 1 -- 172.21.15.37:0/2425389560 wait complete. 2024-04-25T08:59:18.864 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:58:52.843744+0000', 'modified': '2024-04-25T08:59:18.110244+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7998}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7998': {'gid': 7998, 'name': 'a', 'rank': 0, 'incarnation': 63, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.37:6835/1462327322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 1462327322}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1462327322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:59:18.864 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7998:up:replay 2024-04-25T08:59:19.865 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.008+0000 7f143b836640 1 Processor -- start 2024-04-25T08:59:20.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.008+0000 7f143b836640 1 -- start start 2024-04-25T08:59:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.008+0000 7f143b836640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f14340b5270 0x7f14340b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f143b836640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143406bff0 con 0x7f14340b5650 2024-04-25T08:59:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f143b836640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143406c2d0 con 0x7f143406d090 2024-04-25T08:59:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f143b836640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143406cc00 con 0x7f14340b5270 2024-04-25T08:59:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f1438dd1640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f14340b5650 0x7f14340d3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49570/0 (socket says 172.21.15.37:49570) 2024-04-25T08:59:20.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f1438dd1640 1 -- 172.21.15.37:0/688502137 learned_addr learned my addr 172.21.15.37:0/688502137 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f14395d2640 1 --2- 172.21.15.37:0/688502137 >> v2:172.21.15.203:3300/0 conn(0x7f14340b5270 0x7f14340b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 628867466 0 0) 0x7f143406bff0 con 0x7f14340b5650 2024-04-25T08:59:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1420003530 con 0x7f14340b5650 2024-04-25T08:59:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 630142837 0 0) 0x7f1420003530 con 0x7f14340b5650 2024-04-25T08:59:20.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 >> v2:172.21.15.203:3300/0 conn(0x7f14340b5270 msgr2=0x7f14340b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 --2- 172.21.15.37:0/688502137 >> v2:172.21.15.203:3300/0 conn(0x7f14340b5270 0x7f14340b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 >> v1:172.21.15.203:6789/0 conn(0x7f143406d090 legacy=0x7f143406d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143406c5b0 con 0x7f14340b5650 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1424001800 con 0x7f14340b5650 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 >> v1:172.21.15.37:6789/0 conn(0x7f14340b5650 legacy=0x7f14340d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 --2- 172.21.15.37:0/688502137 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1420003a30 0x7f1420003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 --2- 172.21.15.37:0/688502137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1420004810 0x7f1420004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 --2- 172.21.15.37:0/688502137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14200051d0 0x7f142000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143406cc00 con 0x7f14200051d0 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142000c100 con 0x7f1420004810 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142000c300 con 0x7f1420003a30 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f1439dd3640 1 --2- 172.21.15.37:0/688502137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14200051d0 0x7f142000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f14395d2640 1 --2- 172.21.15.37:0/688502137 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1420003a30 0x7f1420003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f1438dd1640 1 --2- 172.21.15.37:0/688502137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1420004810 0x7f1420004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f1439dd3640 1 -- 172.21.15.37:0/688502137 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1420003a30 msgr2=0x7f1420003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f1439dd3640 1 --2- 172.21.15.37:0/688502137 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1420003a30 0x7f1420003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f1439dd3640 1 -- 172.21.15.37:0/688502137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1420004810 msgr2=0x7f1420004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f1439dd3640 1 --2- 172.21.15.37:0/688502137 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1420004810 0x7f1420004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f1439dd3640 1 -- 172.21.15.37:0/688502137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1420003530 con 0x7f14200051d0 2024-04-25T08:59:20.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f1439dd3640 1 -- 172.21.15.37:0/688502137 reap_dead start 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.012+0000 7f1439dd3640 1 --2- 172.21.15.37:0/688502137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14200051d0 0x7f142000b9d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f142c001780 tx=0x7f142c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9251f3d83b7aa7b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0013a0 con 0x7f14200051d0 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c0a9070 con 0x7f14200051d0 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f142bfff640 1 -- 172.21.15.37:0/688502137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c003600 con 0x7f14200051d0 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 -- 172.21.15.37:0/688502137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14200051d0 msgr2=0x7f142000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 --2- 172.21.15.37:0/688502137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14200051d0 0x7f142000b9d0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f142c001780 tx=0x7f142c01c420 comp rx=0 tx=0).stop 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 -- 172.21.15.37:0/688502137 shutdown_connections 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 --2- 172.21.15.37:0/688502137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14200051d0 0x7f142000b9d0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 -- 172.21.15.37:0/688502137 >> 172.21.15.37:0/688502137 conn(0x7f143408b000 msgr2=0x7f14340d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 -- 172.21.15.37:0/688502137 shutdown_connections 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 -- 172.21.15.37:0/688502137 wait complete. 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 Processor -- start 2024-04-25T08:59:20.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 -- start start 2024-04-25T08:59:20.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14200051d0 0x7f14340cb880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14341f44e0 0x7f14341ed670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14340b84c0 0x7f14340b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340b5ac0 con 0x7f14341f44e0 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340d1760 con 0x7f14200051d0 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143415de20 con 0x7f14340b84c0 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1438dd1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14341f44e0 0x7f14341ed670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1438dd1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14341f44e0 0x7f14341ed670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53960/0 (socket says 172.21.15.37:53960) 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1438dd1640 1 -- 172.21.15.37:0/3277536743 learned_addr learned my addr 172.21.15.37:0/3277536743 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f14395d2640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14200051d0 0x7f14340cb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1439dd3640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14340b84c0 0x7f14340b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1438dd1640 1 -- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14340b84c0 msgr2=0x7f14340b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1438dd1640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14340b84c0 0x7f14340b8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1438dd1640 1 -- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14200051d0 msgr2=0x7f14340cb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1438dd1640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14200051d0 0x7f14340cb880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1438dd1640 1 -- 172.21.15.37:0/3277536743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340cc0d0 con 0x7f14341f44e0 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f14395d2640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14200051d0 0x7f14340cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1439dd3640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14340b84c0 0x7f14340b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1438dd1640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14341f44e0 0x7f14341ed670 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f142400e800 tx=0x7f142400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1429ffb640 1 -- 172.21.15.37:0/3277536743 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14240070d0 con 0x7f14341f44e0 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1429ffb640 1 -- 172.21.15.37:0/3277536743 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1424018070 con 0x7f14341f44e0 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 -- 172.21.15.37:0/3277536743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14340cc3b0 con 0x7f14341f44e0 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f1429ffb640 1 -- 172.21.15.37:0/3277536743 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14240074b0 con 0x7f14341f44e0 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.016+0000 7f143b836640 1 -- 172.21.15.37:0/3277536743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14340cc970 con 0x7f14341f44e0 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.020+0000 7f143b836640 1 -- 172.21.15.37:0/3277536743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143404efa0 con 0x7f14341f44e0 2024-04-25T08:59:20.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.020+0000 7f1429ffb640 1 -- 172.21.15.37:0/3277536743 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14240057b0 con 0x7f14341f44e0 2024-04-25T08:59:20.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.020+0000 7f1429ffb640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1418066f90 0x7f1418069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:20.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.020+0000 7f1429ffb640 1 -- 172.21.15.37:0/3277536743 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1424085170 con 0x7f14341f44e0 2024-04-25T08:59:20.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.020+0000 7f14395d2640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1418066f90 0x7f1418069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:20.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.020+0000 7f14395d2640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1418066f90 0x7f1418069450 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f143415a2e0 tx=0x7f143000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:20.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.024+0000 7f1429ffb640 1 -- 172.21.15.37:0/3277536743 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1424055f20 con 0x7f14341f44e0 2024-04-25T08:59:20.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.216+0000 7f143b836640 1 -- 172.21.15.37:0/3277536743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14340c7070 con 0x7f14341f44e0 2024-04-25T08:59:20.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.216+0000 7f1429ffb640 1 -- 172.21.15.37:0/3277536743 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f142401be40 con 0x7f14341f44e0 2024-04-25T08:59:20.209 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:59:20.209 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T08:59:19.119216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":63,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.37:6835/1462327322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1462327322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1462327322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:59:20.210 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 65 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 -- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1418066f90 msgr2=0x7f1418069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1418066f90 0x7f1418069450 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f143415a2e0 tx=0x7f143000b040 comp rx=0 tx=0).stop 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 -- 172.21.15.37:0/3277536743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14341f44e0 msgr2=0x7f14341ed670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14341f44e0 0x7f14341ed670 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f142400e800 tx=0x7f142400ecc0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 -- 172.21.15.37:0/3277536743 shutdown_connections 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1418066f90 0x7f1418069450 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14340b84c0 0x7f14340b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14341f44e0 0x7f14341ed670 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 --2- 172.21.15.37:0/3277536743 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14200051d0 0x7f14340cb880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 -- 172.21.15.37:0/3277536743 >> 172.21.15.37:0/3277536743 conn(0x7f143408b000 msgr2=0x7f14340b4750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 -- 172.21.15.37:0/3277536743 shutdown_connections 2024-04-25T08:59:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:20.220+0000 7f143b836640 1 -- 172.21.15.37:0/3277536743 wait complete. 2024-04-25T08:59:20.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:58:52.843744+0000', 'modified': '2024-04-25T08:59:19.119216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7998}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7998': {'gid': 7998, 'name': 'a', 'rank': 0, 'incarnation': 63, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.37:6835/1462327322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 1462327322}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1462327322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:59:20.630 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7998:up:rejoin 2024-04-25T08:59:21.632 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:21.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.772+0000 7fb3af65c640 1 Processor -- start 2024-04-25T08:59:21.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.772+0000 7fb3af65c640 1 -- start start 2024-04-25T08:59:21.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3af65c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb3a815a700 0x7fb3a815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3af65c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a80b3f50 0x7fb3a80cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3af65c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a80b2090 con 0x7fb3a80ce430 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3af65c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3a815f200 con 0x7fb3a815a700 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3af65c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80b43e0 con 0x7fb3a80b3f50 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3adbf9640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb3a80ce430 0x7fb3a80b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49582/0 (socket says 172.21.15.37:49582) 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3adbf9640 1 -- 172.21.15.37:0/1491258693 learned_addr learned my addr 172.21.15.37:0/1491258693 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3ad3f8640 1 --2- 172.21.15.37:0/1491258693 >> v2:172.21.15.203:3300/0 conn(0x7fb3a815a700 0x7fb3a815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3acbf7640 1 --2- 172.21.15.37:0/1491258693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a80b3f50 0x7fb3a80cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077094402 0 0) 0x7fb3a80b2090 con 0x7fb3a80ce430 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb394005740 con 0x7fb3a80ce430 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2418627393 0 0) 0x7fb394005740 con 0x7fb3a80ce430 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a80b3f50 msgr2=0x7fb3a80cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 --2- 172.21.15.37:0/1491258693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a80b3f50 0x7fb3a80cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 >> v2:172.21.15.203:3300/0 conn(0x7fb3a815a700 msgr2=0x7fb3a815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:21.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 --2- 172.21.15.37:0/1491258693 >> v2:172.21.15.203:3300/0 conn(0x7fb3a815a700 0x7fb3a815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a80b46c0 con 0x7fb3a80ce430 2024-04-25T08:59:21.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fb3a0001a00 con 0x7fb3a80ce430 2024-04-25T08:59:21.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 >> v1:172.21.15.37:6789/0 conn(0x7fb3a80ce430 legacy=0x7fb3a80b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 --2- 172.21.15.37:0/1491258693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb394005c40 0x7fb394006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 --2- 172.21.15.37:0/1491258693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb394006a20 0x7fb394006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 --2- 172.21.15.37:0/1491258693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3940073e0 0x7fb39400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a815f200 con 0x7fb394006a20 2024-04-25T08:59:21.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80b43e0 con 0x7fb394005c40 2024-04-25T08:59:21.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb39400e120 con 0x7fb3940073e0 2024-04-25T08:59:21.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3acbf7640 1 --2- 172.21.15.37:0/1491258693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb394006a20 0x7fb394006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3ad3f8640 1 --2- 172.21.15.37:0/1491258693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb394005c40 0x7fb394006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3acbf7640 1 -- 172.21.15.37:0/1491258693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3940073e0 msgr2=0x7fb39400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:21.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3acbf7640 1 --2- 172.21.15.37:0/1491258693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3940073e0 0x7fb39400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3acbf7640 1 -- 172.21.15.37:0/1491258693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb394005c40 msgr2=0x7fb394006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:21.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3acbf7640 1 --2- 172.21.15.37:0/1491258693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb394005c40 0x7fb394006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.776+0000 7fb3acbf7640 1 -- 172.21.15.37:0/1491258693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb394005740 con 0x7fb394006a20 2024-04-25T08:59:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3acbf7640 1 --2- 172.21.15.37:0/1491258693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb394006a20 0x7fb394006ea0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fb398009ac0 tx=0x7fb39801ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad3bd229affff7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:21.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3980b2070 con 0x7fb394006a20 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3980a5540 con 0x7fb394006a20 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb39ffff640 1 -- 172.21.15.37:0/1491258693 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb398009690 con 0x7fb394006a20 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 -- 172.21.15.37:0/1491258693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb394006a20 msgr2=0x7fb394006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 --2- 172.21.15.37:0/1491258693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb394006a20 0x7fb394006ea0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fb398009ac0 tx=0x7fb39801ca60 comp rx=0 tx=0).stop 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3adbf9640 1 -- 172.21.15.37:0/1491258693 reap_dead start 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 -- 172.21.15.37:0/1491258693 shutdown_connections 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 -- 172.21.15.37:0/1491258693 >> 172.21.15.37:0/1491258693 conn(0x7fb3a80b11a0 msgr2=0x7fb3a80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 -- 172.21.15.37:0/1491258693 shutdown_connections 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 -- 172.21.15.37:0/1491258693 wait complete. 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 Processor -- start 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 -- start start 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a814fa50 0x7fb3a81556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a80af0f0 0x7fb3a80af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a8153fa0 0x7fb3a8154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80cd490 con 0x7fb3a80af0f0 2024-04-25T08:59:21.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80cd310 con 0x7fb3a814fa50 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3af65c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a80b2bb0 con 0x7fb3a8153fa0 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3acbf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a80af0f0 0x7fb3a80af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3acbf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a80af0f0 0x7fb3a80af570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53964/0 (socket says 172.21.15.37:53964) 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3acbf7640 1 -- 172.21.15.37:0/955208931 learned_addr learned my addr 172.21.15.37:0/955208931 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3ad3f8640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a814fa50 0x7fb3a81556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3adbf9640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a8153fa0 0x7fb3a8154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3acbf7640 1 -- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a8153fa0 msgr2=0x7fb3a8154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3acbf7640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a8153fa0 0x7fb3a8154420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3acbf7640 1 -- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a814fa50 msgr2=0x7fb3a81556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3acbf7640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a814fa50 0x7fb3a81556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3acbf7640 1 -- 172.21.15.37:0/955208931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a81503c0 con 0x7fb3a80af0f0 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.780+0000 7fb3ad3f8640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a814fa50 0x7fb3a81556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.784+0000 7fb3acbf7640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a80af0f0 0x7fb3a80af570 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb39801d300 tx=0x7fb398019800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:21.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.784+0000 7fb39dffb640 1 -- 172.21.15.37:0/955208931 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3980bd070 con 0x7fb3a80af0f0 2024-04-25T08:59:21.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.784+0000 7fb39dffb640 1 -- 172.21.15.37:0/955208931 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3980ad440 con 0x7fb3a80af0f0 2024-04-25T08:59:21.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.784+0000 7fb3af65c640 1 -- 172.21.15.37:0/955208931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a815b140 con 0x7fb3a80af0f0 2024-04-25T08:59:21.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.784+0000 7fb39dffb640 1 -- 172.21.15.37:0/955208931 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3980b2040 con 0x7fb3a80af0f0 2024-04-25T08:59:21.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.784+0000 7fb3af65c640 1 -- 172.21.15.37:0/955208931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a815b700 con 0x7fb3a80af0f0 2024-04-25T08:59:21.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.784+0000 7fb3af65c640 1 -- 172.21.15.37:0/955208931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3a804efa0 con 0x7fb3a80af0f0 2024-04-25T08:59:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.784+0000 7fb39dffb640 1 -- 172.21.15.37:0/955208931 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb3980c0070 con 0x7fb3a80af0f0 2024-04-25T08:59:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.784+0000 7fb39dffb640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb38c066f60 0x7fb38c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.784+0000 7fb39dffb640 1 -- 172.21.15.37:0/955208931 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb39811ee60 con 0x7fb3a80af0f0 2024-04-25T08:59:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.784+0000 7fb3ad3f8640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb38c066f60 0x7fb38c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:21.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.788+0000 7fb3ad3f8640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb38c066f60 0x7fb38c069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009aa0 tx=0x7fb3a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:21.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.788+0000 7fb39dffb640 1 -- 172.21.15.37:0/955208931 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb3980167a0 con 0x7fb3a80af0f0 2024-04-25T08:59:21.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.980+0000 7fb3af65c640 1 -- 172.21.15.37:0/955208931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3a806f260 con 0x7fb3a80af0f0 2024-04-25T08:59:21.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb39dffb640 1 -- 172.21.15.37:0/955208931 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fb3980ef640 con 0x7fb3a80af0f0 2024-04-25T08:59:21.975 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:59:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T08:59:20.127517+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/1462327322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1462327322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1462327322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:59:21.976 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 66 2024-04-25T08:59:21.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 -- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb38c066f60 msgr2=0x7fb38c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb38c066f60 0x7fb38c069420 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009aa0 tx=0x7fb3a4008040 comp rx=0 tx=0).stop 2024-04-25T08:59:21.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 -- 172.21.15.37:0/955208931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a80af0f0 msgr2=0x7fb3a80af570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:21.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a80af0f0 0x7fb3a80af570 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fb39801d300 tx=0x7fb398019800 comp rx=0 tx=0).stop 2024-04-25T08:59:21.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 -- 172.21.15.37:0/955208931 shutdown_connections 2024-04-25T08:59:21.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb38c066f60 0x7fb38c069420 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb3a8153fa0 0x7fb3a8154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3a80af0f0 0x7fb3a80af570 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 --2- 172.21.15.37:0/955208931 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb3a814fa50 0x7fb3a81556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 -- 172.21.15.37:0/955208931 >> 172.21.15.37:0/955208931 conn(0x7fb3a80b11a0 msgr2=0x7fb3a80b0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 -- 172.21.15.37:0/955208931 shutdown_connections 2024-04-25T08:59:21.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:21.984+0000 7fb3af65c640 1 -- 172.21.15.37:0/955208931 wait complete. 2024-04-25T08:59:22.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:58:52.843744+0000', 'modified': '2024-04-25T08:59:20.127517+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 98, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7998}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7998': {'gid': 7998, 'name': 'a', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.37:6835/1462327322', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 1462327322}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1462327322}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:59:22.398 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:59:22.398 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T08:59:22.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d17aa4640 1 Processor -- start 2024-04-25T08:59:22.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d17aa4640 1 -- start start 2024-04-25T08:59:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d17aa4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3d100cc860 0x7f3d100b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d17aa4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3d100b35e0 0x7f3d1006e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d17aa4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d100b3d60 con 0x7f3d1015a540 2024-04-25T08:59:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d17aa4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d100b4040 con 0x7f3d100b35e0 2024-04-25T08:59:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d17aa4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1006e150 con 0x7f3d100cc860 2024-04-25T08:59:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d15840640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3d1015a540 0x7f3d100cc150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49592/0 (socket says 172.21.15.37:49592) 2024-04-25T08:59:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d15840640 1 -- 172.21.15.37:0/270172810 learned_addr learned my addr 172.21.15.37:0/270172810 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d16041640 1 --2- 172.21.15.37:0/270172810 >> v2:172.21.15.37:3300/0 conn(0x7f3d100b35e0 0x7f3d1006e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d1503f640 1 --2- 172.21.15.37:0/270172810 >> v2:172.21.15.203:3300/0 conn(0x7f3d100cc860 0x7f3d100b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d1483e640 1 -- 172.21.15.37:0/270172810 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 39162680 0 0) 0x7f3d100b3d60 con 0x7f3d1015a540 2024-04-25T08:59:22.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d1483e640 1 -- 172.21.15.37:0/270172810 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cfc003530 con 0x7f3d1015a540 2024-04-25T08:59:22.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d16041640 1 -- 172.21.15.37:0/270172810 >> v2:172.21.15.203:3300/0 conn(0x7f3d100cc860 msgr2=0x7f3d100b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d16041640 1 --2- 172.21.15.37:0/270172810 >> v2:172.21.15.203:3300/0 conn(0x7f3d100cc860 0x7f3d100b30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d16041640 1 -- 172.21.15.37:0/270172810 >> v1:172.21.15.37:6789/0 conn(0x7f3d1015a540 legacy=0x7f3d100cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.544+0000 7f3d16041640 1 -- 172.21.15.37:0/270172810 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d1006e430 con 0x7f3d100b35e0 2024-04-25T08:59:22.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d16041640 1 --2- 172.21.15.37:0/270172810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d100b35e0 0x7f3d1006e950 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3d1006c6a0 tx=0x7f3d0c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b71d331ce0e52f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d1483e640 1 -- 172.21.15.37:0/270172810 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0b0070 con 0x7f3d100b35e0 2024-04-25T08:59:22.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d1483e640 1 -- 172.21.15.37:0/270172810 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c004230 con 0x7f3d100b35e0 2024-04-25T08:59:22.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d1483e640 1 -- 172.21.15.37:0/270172810 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c003620 con 0x7f3d100b35e0 2024-04-25T08:59:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d17aa4640 1 -- 172.21.15.37:0/270172810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d100b35e0 msgr2=0x7f3d1006e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d17aa4640 1 --2- 172.21.15.37:0/270172810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d100b35e0 0x7f3d1006e950 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f3d1006c6a0 tx=0x7f3d0c01c3f0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d17aa4640 1 -- 172.21.15.37:0/270172810 shutdown_connections 2024-04-25T08:59:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d17aa4640 1 --2- 172.21.15.37:0/270172810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d100b35e0 0x7f3d1006e950 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d17aa4640 1 --2- 172.21.15.37:0/270172810 >> v2:172.21.15.203:3300/0 conn(0x7f3d100cc860 0x7f3d100b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d17aa4640 1 -- 172.21.15.37:0/270172810 >> 172.21.15.37:0/270172810 conn(0x7f3d100b14b0 msgr2=0x7f3d100b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d17aa4640 1 -- 172.21.15.37:0/270172810 shutdown_connections 2024-04-25T08:59:22.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d17aa4640 1 -- 172.21.15.37:0/270172810 wait complete. 2024-04-25T08:59:22.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d17aa4640 1 Processor -- start 2024-04-25T08:59:22.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.548+0000 7f3d17aa4640 1 -- start start 2024-04-25T08:59:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d17aa4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d100b35e0 0x7f3d1015fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d17aa4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d100cc860 0x7f3d10160240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d17aa4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1015a540 0x7f3d10162360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d17aa4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1006f820 con 0x7f3d100b35e0 2024-04-25T08:59:22.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d17aa4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1006f6a0 con 0x7f3d100cc860 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d17aa4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d1006f9a0 con 0x7f3d1015a540 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d15840640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d100b35e0 0x7f3d1015fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d15840640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d100b35e0 0x7f3d1015fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53982/0 (socket says 172.21.15.37:53982) 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d15840640 1 -- 172.21.15.37:0/1320177530 learned_addr learned my addr 172.21.15.37:0/1320177530 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d1503f640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d100cc860 0x7f3d10160240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d15840640 1 -- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1015a540 msgr2=0x7f3d10162360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d16041640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1015a540 0x7f3d10162360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d15840640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1015a540 0x7f3d10162360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d15840640 1 -- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d100cc860 msgr2=0x7f3d10160240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d15840640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d100cc860 0x7f3d10160240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d15840640 1 -- 172.21.15.37:0/1320177530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d10151900 con 0x7f3d100b35e0 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d1503f640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d100cc860 0x7f3d10160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d16041640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1015a540 0x7f3d10162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:22.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d15840640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d100b35e0 0x7f3d1015fd00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800caa0 tx=0x7f3d0800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d067fc640 1 -- 172.21.15.37:0/1320177530 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08017070 con 0x7f3d100b35e0 2024-04-25T08:59:22.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d067fc640 1 -- 172.21.15.37:0/1320177530 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0800ac00 con 0x7f3d100b35e0 2024-04-25T08:59:22.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d067fc640 1 -- 172.21.15.37:0/1320177530 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08012400 con 0x7f3d100b35e0 2024-04-25T08:59:22.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d17aa4640 1 -- 172.21.15.37:0/1320177530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d10161e80 con 0x7f3d100b35e0 2024-04-25T08:59:22.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d17aa4640 1 -- 172.21.15.37:0/1320177530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d100cfd90 con 0x7f3d100b35e0 2024-04-25T08:59:22.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d067fc640 1 -- 172.21.15.37:0/1320177530 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d0801a070 con 0x7f3d100b35e0 2024-04-25T08:59:22.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d067fc640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cf4066f60 0x7f3cf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d067fc640 1 -- 172.21.15.37:0/1320177530 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f3d080850d0 con 0x7f3d100b35e0 2024-04-25T08:59:22.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.552+0000 7f3d1503f640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cf4066f60 0x7f3cf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 Processor -- start 2024-04-25T08:59:22.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 -- start start 2024-04-25T08:59:22.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1503f640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cf4066f60 0x7f3cf4069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3d10072640 tx=0x7f3d00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac53920 0x562e4ac53d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac54240 0x562e4ac099f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac09f30 0x562e4ac0a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4aaf15f0 con 0x562e4ac53920 2024-04-25T08:59:22.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4aa58da0 con 0x562e4ac54240 2024-04-25T08:59:22.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e4a9e1150 con 0x562e4ac09f30 2024-04-25T08:59:22.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d15840640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac53920 0x562e4ac53d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d15840640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac53920 0x562e4ac53d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53992/0 (socket says 172.21.15.37:53992) 2024-04-25T08:59:22.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d15840640 1 -- 172.21.15.37:0/742958245 learned_addr learned my addr 172.21.15.37:0/742958245 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:22.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1503f640 1 --2- 172.21.15.37:0/742958245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac54240 0x562e4ac099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d16041640 1 --2- 172.21.15.37:0/742958245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac09f30 0x562e4ac0a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d15840640 1 -- 172.21.15.37:0/742958245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac09f30 msgr2=0x562e4ac0a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d15840640 1 --2- 172.21.15.37:0/742958245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac09f30 0x562e4ac0a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d15840640 1 -- 172.21.15.37:0/742958245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac54240 msgr2=0x562e4ac099f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d15840640 1 --2- 172.21.15.37:0/742958245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac54240 0x562e4ac099f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d15840640 1 -- 172.21.15.37:0/742958245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e4a9faca0 con 0x562e4ac53920 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d16041640 1 --2- 172.21.15.37:0/742958245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac09f30 0x562e4ac0a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d15840640 1 --2- 172.21.15.37:0/742958245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac53920 0x562e4ac53d00 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f3d0805d8d0 tx=0x7f3d08059840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ced886ec97a4a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3cf37fe640 1 -- 172.21.15.37:0/742958245 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d080539f0 con 0x562e4ac53920 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3cf37fe640 1 -- 172.21.15.37:0/742958245 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d08051d60 con 0x562e4ac53920 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3cf37fe640 1 -- 172.21.15.37:0/742958245 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0805dbb0 con 0x562e4ac53920 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 -- 172.21.15.37:0/742958245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac53920 msgr2=0x562e4ac53d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/742958245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac53920 0x562e4ac53d00 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f3d0805d8d0 tx=0x7f3d08059840 comp rx=0 tx=0).stop 2024-04-25T08:59:22.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 -- 172.21.15.37:0/742958245 shutdown_connections 2024-04-25T08:59:22.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/742958245 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac09f30 0x562e4ac0a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/742958245 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac54240 0x562e4ac099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/742958245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac53920 0x562e4ac53d00 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 -- 172.21.15.37:0/742958245 >> 172.21.15.37:0/742958245 conn(0x562e4ac102a0 msgr2=0x562e4ac082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 -- 172.21.15.37:0/742958245 shutdown_connections 2024-04-25T08:59:22.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 -- 172.21.15.37:0/742958245 wait complete. 2024-04-25T08:59:22.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 -- start start 2024-04-25T08:59:22.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.556+0000 7f3d1ac11000 1 Processor -- start 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac5ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac53920 0x562e4ac5b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac54240 0x562e4ac0daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4aa13990 con 0x562e4ac09f30 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4aa54e40 con 0x562e4ac54240 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e4a9e4900 con 0x562e4ac53920 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 10 client.0 _ll_register_callbacks cb 0x562e4ab0c1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d15840640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac5ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d15840640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac5ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53998/0 (socket says 172.21.15.37:53998) 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d15840640 1 -- 172.21.15.37:0/671421126 learned_addr learned my addr 172.21.15.37:0/671421126 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1503f640 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac53920 0x562e4ac5b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d16041640 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac54240 0x562e4ac0daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d15840640 1 -- 172.21.15.37:0/671421126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac53920 msgr2=0x562e4ac5b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d15840640 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac53920 0x562e4ac5b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d15840640 1 -- 172.21.15.37:0/671421126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac54240 msgr2=0x562e4ac0daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d15840640 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac54240 0x562e4ac0daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d15840640 1 -- 172.21.15.37:0/671421126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e4aa11630 con 0x562e4ac09f30 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d16041640 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac54240 0x562e4ac0daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d15840640 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac5ac80 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f3d0805d8d0 tx=0x7f3d08000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8afebe768c7fd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3cf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/671421126 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d08054d40 con 0x562e4ac09f30 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/671421126 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f3d0806ed80 con 0x562e4ac09f30 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 -- 172.21.15.37:0/671421126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d08051d60 con 0x562e4ac09f30 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/671421126 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d08073b30 con 0x562e4ac09f30 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 10 client.8064 fetch_fsmap learned FSMap version 66 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 -- 172.21.15.37:0/671421126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x562e4a9e13a0 con 0x562e4ac09f30 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/671421126 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d08051d60 con 0x562e4ac09f30 2024-04-25T08:59:22.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/671421126 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3d08051440 con 0x562e4ac09f30 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 10 client.8064 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 10 client.8064 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 20 client.8064 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x562e4ac0e4b0 0x562e4ac0e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 4 client.8064 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 -- 172.21.15.37:0/671421126 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e4a9f4a50 con 0x562e4ac0e4b0 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1503f640 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x562e4ac0e4b0 0x562e4ac0e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3cf0ff9640 10 client.8064 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1503f640 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x562e4ac0e4b0 0x562e4ac0e890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34189bbda6cfb603 server_cookie=e3c3bfb701b3cbaa in_seq=0 out_seq=0 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/671421126 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e4a9f4a50 con 0x562e4ac0e4b0 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3cf0ff9640 10 client.8064 handle_command_reply: tid=0 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 client.8064 shutdown 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 -- 172.21.15.37:0/671421126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 msgr2=0x562e4ac5ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac5ac80 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f3d0805d8d0 tx=0x7f3d08000f50 comp rx=0 tx=0).stop 2024-04-25T08:59:22.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 -- 172.21.15.37:0/671421126 shutdown_connections 2024-04-25T08:59:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x562e4ac0e4b0 0x562e4ac0e890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac54240 0x562e4ac0daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac53920 0x562e4ac5b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3cf0ff9640 0 client.8064 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/671421126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac5ac80 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 -- 172.21.15.37:0/671421126 >> 172.21.15.37:0/671421126 conn(0x562e4ac102a0 msgr2=0x562e4ac089e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 -- 172.21.15.37:0/671421126 shutdown_connections 2024-04-25T08:59:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 1 -- 172.21.15.37:0/671421126 wait complete. 2024-04-25T08:59:22.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.560+0000 7f3d1ac11000 20 client.8064 trim_cache size 0 max 16384 2024-04-25T08:59:22.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.576+0000 7f3d1ac11000 1 Processor -- start 2024-04-25T08:59:22.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.576+0000 7f3d1ac11000 1 -- start start 2024-04-25T08:59:22.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.576+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac09f30 0x562e4ac0d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.576+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac53920 0x562e4ac0d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.576+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac54240 0x562e4ac5d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.576+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d101ffee0 con 0x562e4ac54240 2024-04-25T08:59:22.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.576+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d100ce580 con 0x562e4ac53920 2024-04-25T08:59:22.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.576+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d101ff8e0 con 0x562e4ac09f30 2024-04-25T08:59:22.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.576+0000 7f3d16041640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac54240 0x562e4ac5d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.576+0000 7f3d16041640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac54240 0x562e4ac5d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54004/0 (socket says 172.21.15.37:54004) 2024-04-25T08:59:22.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.576+0000 7f3d16041640 1 -- 172.21.15.37:0/2668430029 learned_addr learned my addr 172.21.15.37:0/2668430029 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:22.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1503f640 1 --2- 172.21.15.37:0/2668430029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac53920 0x562e4ac0d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d15840640 1 --2- 172.21.15.37:0/2668430029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac09f30 0x562e4ac0d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d16041640 1 -- 172.21.15.37:0/2668430029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac09f30 msgr2=0x562e4ac0d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d16041640 1 --2- 172.21.15.37:0/2668430029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac09f30 0x562e4ac0d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d16041640 1 -- 172.21.15.37:0/2668430029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac53920 msgr2=0x562e4ac0d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d16041640 1 --2- 172.21.15.37:0/2668430029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac53920 0x562e4ac0d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d16041640 1 -- 172.21.15.37:0/2668430029 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e4ac54070 con 0x562e4ac54240 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d16041640 1 --2- 172.21.15.37:0/2668430029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac54240 0x562e4ac5d8b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c01a020 tx=0x7f3d0c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e845e13de6799650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3cf2ffd640 1 -- 172.21.15.37:0/2668430029 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0ba030 con 0x562e4ac54240 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3cf2ffd640 1 -- 172.21.15.37:0/2668430029 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c004370 con 0x562e4ac54240 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3cf2ffd640 1 -- 172.21.15.37:0/2668430029 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0c0b0040 con 0x562e4ac54240 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 -- 172.21.15.37:0/2668430029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac54240 msgr2=0x562e4ac5d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/2668430029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac54240 0x562e4ac5d8b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f3d0c01a020 tx=0x7f3d0c004740 comp rx=0 tx=0).stop 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 -- 172.21.15.37:0/2668430029 shutdown_connections 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/2668430029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac54240 0x562e4ac5d8b0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/2668430029 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac53920 0x562e4ac0d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/2668430029 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac09f30 0x562e4ac0d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 -- 172.21.15.37:0/2668430029 >> 172.21.15.37:0/2668430029 conn(0x562e4ac102a0 msgr2=0x562e4ac0cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 -- 172.21.15.37:0/2668430029 shutdown_connections 2024-04-25T08:59:22.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 -- 172.21.15.37:0/2668430029 wait complete. 2024-04-25T08:59:22.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 -- start start 2024-04-25T08:59:22.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 Processor -- start 2024-04-25T08:59:22.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac60790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac53920 0x562e4ac73740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac54240 0x562e4ac73c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4aa5d3d0 con 0x562e4ac09f30 2024-04-25T08:59:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e4ab14d50 con 0x562e4ac53920 2024-04-25T08:59:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e4a9f3980 con 0x562e4ac54240 2024-04-25T08:59:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 10 client.0 _ll_register_callbacks cb 0x562e4aa5e2c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d15840640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac60790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d15840640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac60790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54014/0 (socket says 172.21.15.37:54014) 2024-04-25T08:59:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d15840640 1 -- 172.21.15.37:0/3281805138 learned_addr learned my addr 172.21.15.37:0/3281805138 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1503f640 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac53920 0x562e4ac73740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d16041640 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac54240 0x562e4ac73c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d15840640 1 -- 172.21.15.37:0/3281805138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac54240 msgr2=0x562e4ac73c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d15840640 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac54240 0x562e4ac73c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d15840640 1 -- 172.21.15.37:0/3281805138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac53920 msgr2=0x562e4ac73740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d15840640 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac53920 0x562e4ac73740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d15840640 1 -- 172.21.15.37:0/3281805138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e4aada600 con 0x562e4ac09f30 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d16041640 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac54240 0x562e4ac73c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d15840640 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac60790 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f3d08005c30 tx=0x7f3d08055540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68afd1debb393971 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3cf0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/3281805138 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d080545b0 con 0x562e4ac09f30 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/3281805138 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f3d08054750 con 0x562e4ac09f30 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 -- 172.21.15.37:0/3281805138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d0c004370 con 0x562e4ac09f30 2024-04-25T08:59:22.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/3281805138 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d08056a20 con 0x562e4ac09f30 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 10 client.8070 fetch_fsmap learned FSMap version 66 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3d1ac11000 1 -- 172.21.15.37:0/3281805138 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x562e4a9ed9b0 con 0x562e4ac09f30 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.580+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/3281805138 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0c004370 con 0x562e4ac09f30 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/3281805138 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3d08056bc0 con 0x562e4ac09f30 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 10 client.8070 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 10 client.8070 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 20 client.8070 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x562e4ac741c0 0x562e4acb81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 4 client.8070 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 -- 172.21.15.37:0/3281805138 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562e4aa662e0 con 0x562e4ac741c0 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1503f640 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x562e4ac741c0 0x562e4acb81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3cf0ff9640 10 client.8070 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:22.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1503f640 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x562e4ac741c0 0x562e4acb81b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=510c13139b41d7de server_cookie=c19951fe6b4a047e in_seq=0 out_seq=0 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3cf0ff9640 1 -- 172.21.15.37:0/3281805138 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x562e4aa662e0 con 0x562e4ac741c0 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3cf0ff9640 10 client.8070 handle_command_reply: tid=0 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 client.8070 shutdown 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 -- 172.21.15.37:0/3281805138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 msgr2=0x562e4ac60790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac60790 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f3d08005c30 tx=0x7f3d08055540 comp rx=0 tx=0).stop 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 -- 172.21.15.37:0/3281805138 shutdown_connections 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x562e4ac741c0 0x562e4acb81b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3cf0ff9640 0 client.8070 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562e4ac54240 0x562e4ac73c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562e4ac53920 0x562e4ac73740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 --2- 172.21.15.37:0/3281805138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e4ac09f30 0x562e4ac60790 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 -- 172.21.15.37:0/3281805138 >> 172.21.15.37:0/3281805138 conn(0x562e4ac102a0 msgr2=0x562e4ac51c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 -- 172.21.15.37:0/3281805138 shutdown_connections 2024-04-25T08:59:22.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 1 -- 172.21.15.37:0/3281805138 wait complete. 2024-04-25T08:59:22.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d1ac11000 20 client.8070 trim_cache size 0 max 16384 2024-04-25T08:59:22.582 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:59:22.582 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T08:59:22.582 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T08:59:22.582 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 7998, 2024-04-25T08:59:22.582 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T08:59:22.582 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T08:59:22.583 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T08:59:22.583 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 4.4708823750000004, 2024-04-25T08:59:22.583 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 66, 2024-04-25T08:59:22.583 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 98, 2024-04-25T08:59:22.583 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 98, 2024-04-25T08:59:22.583 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 5.454198324 2024-04-25T08:59:22.583 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:59:22.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 -- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cf4066f60 msgr2=0x7f3cf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cf4066f60 0x7f3cf4069420 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3d10072640 tx=0x7f3d00008040 comp rx=0 tx=0).stop 2024-04-25T08:59:22.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 -- 172.21.15.37:0/1320177530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d100b35e0 msgr2=0x7f3d1015fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d100b35e0 0x7f3d1015fd00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3d0800caa0 tx=0x7f3d0800cf60 comp rx=0 tx=0).stop 2024-04-25T08:59:22.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 -- 172.21.15.37:0/1320177530 shutdown_connections 2024-04-25T08:59:22.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cf4066f60 0x7f3cf4069420 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d1015a540 0x7f3d10162360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d100cc860 0x7f3d10160240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 --2- 172.21.15.37:0/1320177530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d100b35e0 0x7f3d1015fd00 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 -- 172.21.15.37:0/1320177530 >> 172.21.15.37:0/1320177530 conn(0x7f3d100b14b0 msgr2=0x7f3d100b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 -- 172.21.15.37:0/1320177530 shutdown_connections 2024-04-25T08:59:22.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.584+0000 7f3d17aa4640 1 -- 172.21.15.37:0/1320177530 wait complete. 2024-04-25T08:59:22.592 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:59:22.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.712+0000 7f4f4fe4f640 1 Processor -- start 2024-04-25T08:59:22.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.712+0000 7f4f4fe4f640 1 -- start start 2024-04-25T08:59:22.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.712+0000 7f4f4fe4f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4f480ce510 0x7f4f480ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.712+0000 7f4f4fe4f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481451f0 0x7f4f48142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.712+0000 7f4f4fe4f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f480cee30 con 0x7f4f48144ca0 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.712+0000 7f4f4fe4f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f480cc2b0 con 0x7f4f480ce510 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.712+0000 7f4f4fe4f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4806e290 con 0x7f4f481451f0 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.712+0000 7f4f4d3ea640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4f48144ca0 0x7f4f4806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49608/0 (socket says 172.21.15.37:49608) 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.712+0000 7f4f4d3ea640 1 -- 172.21.15.37:0/668095178 learned_addr learned my addr 172.21.15.37:0/668095178 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4dbeb640 1 --2- 172.21.15.37:0/668095178 >> v2:172.21.15.203:3300/0 conn(0x7f4f480ce510 0x7f4f480ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4e3ec640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481451f0 0x7f4f48142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003671037 0 0) 0x7f4f480cee30 con 0x7f4f48144ca0 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f34003530 con 0x7f4f48144ca0 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2533746707 0 0) 0x7f4f34003530 con 0x7f4f48144ca0 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481451f0 msgr2=0x7f4f48142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481451f0 0x7f4f48142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 >> v2:172.21.15.203:3300/0 conn(0x7f4f480ce510 msgr2=0x7f4f480ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 --2- 172.21.15.37:0/668095178 >> v2:172.21.15.203:3300/0 conn(0x7f4f480ce510 0x7f4f480ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f480ccc70 con 0x7f4f48144ca0 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f4f38001680 con 0x7f4f48144ca0 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 >> v1:172.21.15.37:6789/0 conn(0x7f4f48144ca0 legacy=0x7f4f4806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f34003a30 0x7f4f34003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481451f0 0x7f4f34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f34004d50 0x7f4f340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480cc2b0 con 0x7f4f34003a30 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4806e290 con 0x7f4f34004d50 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3400ba80 con 0x7f4f481451f0 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4dbeb640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f34003a30 0x7f4f34003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4e3ec640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f34004d50 0x7f4f340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4d3ea640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481451f0 0x7f4f34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4dbeb640 1 -- 172.21.15.37:0/668095178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481451f0 msgr2=0x7f4f34004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4dbeb640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481451f0 0x7f4f34004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4dbeb640 1 -- 172.21.15.37:0/668095178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f34004d50 msgr2=0x7f4f340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:22.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4dbeb640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f34004d50 0x7f4f340051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4dbeb640 1 -- 172.21.15.37:0/668095178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f34003530 con 0x7f4f34003a30 2024-04-25T08:59:22.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4dbeb640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f34003a30 0x7f4f34003e50 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009ae0 tx=0x7f4f4401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be72c5b7485cde16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440b2070 con 0x7f4f34003a30 2024-04-25T08:59:22.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f440a56a0 con 0x7f4f34003a30 2024-04-25T08:59:22.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.716+0000 7f4f4cbe9640 1 -- 172.21.15.37:0/668095178 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f440096b0 con 0x7f4f34003a30 2024-04-25T08:59:22.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/668095178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f34003a30 msgr2=0x7f4f34003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 --2- 172.21.15.37:0/668095178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f34003a30 0x7f4f34003e50 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009ae0 tx=0x7f4f4401cab0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4e3ec640 1 -- 172.21.15.37:0/668095178 reap_dead start 2024-04-25T08:59:22.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/668095178 shutdown_connections 2024-04-25T08:59:22.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/668095178 >> 172.21.15.37:0/668095178 conn(0x7f4f480b11a0 msgr2=0x7f4f480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/668095178 shutdown_connections 2024-04-25T08:59:22.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/668095178 wait complete. 2024-04-25T08:59:22.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 Processor -- start 2024-04-25T08:59:22.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 -- start start 2024-04-25T08:59:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f481e6120 0x7f4f481f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f481433f0 0x7f4f480b3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481e6500 0x7f4f481e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4806ee30 con 0x7f4f481433f0 2024-04-25T08:59:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4806df20 con 0x7f4f481e6120 2024-04-25T08:59:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f480cc900 con 0x7f4f481e6500 2024-04-25T08:59:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4d3ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f481433f0 0x7f4f480b3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4d3ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f481433f0 0x7f4f480b3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54032/0 (socket says 172.21.15.37:54032) 2024-04-25T08:59:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4d3ea640 1 -- 172.21.15.37:0/873529545 learned_addr learned my addr 172.21.15.37:0/873529545 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:22.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4dbeb640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f481e6120 0x7f4f481f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4e3ec640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481e6500 0x7f4f481e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4d3ea640 1 -- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481e6500 msgr2=0x7f4f481e69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4d3ea640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481e6500 0x7f4f481e69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4d3ea640 1 -- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f481e6120 msgr2=0x7f4f481f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4d3ea640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f481e6120 0x7f4f481f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4d3ea640 1 -- 172.21.15.37:0/873529545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f480a84d0 con 0x7f4f481433f0 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4dbeb640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f481e6120 0x7f4f481f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4d3ea640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f481433f0 0x7f4f480b3370 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f4f38011840 tx=0x7f4f38011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f3e7fc640 1 -- 172.21.15.37:0/873529545 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f38010040 con 0x7f4f481433f0 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f3e7fc640 1 -- 172.21.15.37:0/873529545 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f3801a070 con 0x7f4f481433f0 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f3e7fc640 1 -- 172.21.15.37:0/873529545 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800fd60 con 0x7f4f481433f0 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/873529545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f480a8770 con 0x7f4f481433f0 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.720+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/873529545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f480a8d30 con 0x7f4f481433f0 2024-04-25T08:59:22.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.724+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/873529545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f4804efa0 con 0x7f4f481433f0 2024-04-25T08:59:22.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.724+0000 7f4f3e7fc640 1 -- 172.21.15.37:0/873529545 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4f3801f460 con 0x7f4f481433f0 2024-04-25T08:59:22.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.724+0000 7f4f3e7fc640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f2c066f90 0x7f4f2c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:22.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.724+0000 7f4f3e7fc640 1 -- 172.21.15.37:0/873529545 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f4f38088460 con 0x7f4f481433f0 2024-04-25T08:59:22.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.724+0000 7f4f4dbeb640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f2c066f90 0x7f4f2c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:22.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.724+0000 7f4f4dbeb640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f2c066f90 0x7f4f2c069450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4f440af000 tx=0x7f4f44008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:22.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.728+0000 7f4f3e7fc640 1 -- 172.21.15.37:0/873529545 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4f38059220 con 0x7f4f481433f0 2024-04-25T08:59:22.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.924+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/873529545 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4f480b1cc0 con 0x7f4f481433f0 2024-04-25T08:59:22.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.924+0000 7f4f3e7fc640 1 -- 172.21.15.37:0/873529545 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v66) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4f38058b20 con 0x7f4f481433f0 2024-04-25T08:59:22.916 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:59:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.924+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f2c066f90 msgr2=0x7f4f2c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.924+0000 7f4f4fe4f640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f2c066f90 0x7f4f2c069450 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4f440af000 tx=0x7f4f44008040 comp rx=0 tx=0).stop 2024-04-25T08:59:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.924+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/873529545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f481433f0 msgr2=0x7f4f480b3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.924+0000 7f4f4fe4f640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f481433f0 0x7f4f480b3370 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f4f38011840 tx=0x7f4f38011d00 comp rx=0 tx=0).stop 2024-04-25T08:59:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.928+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/873529545 shutdown_connections 2024-04-25T08:59:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.928+0000 7f4f4fe4f640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4f2c066f90 0x7f4f2c069450 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.928+0000 7f4f4fe4f640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4f481e6500 0x7f4f481e69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.928+0000 7f4f4fe4f640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f481433f0 0x7f4f480b3370 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.928+0000 7f4f4fe4f640 1 --2- 172.21.15.37:0/873529545 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4f481e6120 0x7f4f481f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.928+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/873529545 >> 172.21.15.37:0/873529545 conn(0x7f4f480b11a0 msgr2=0x7f4f480c6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.928+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/873529545 shutdown_connections 2024-04-25T08:59:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:22.928+0000 7f4f4fe4f640 1 -- 172.21.15.37:0/873529545 wait complete. 2024-04-25T08:59:22.933 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:59:22.934 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:59:22.934 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T08:59:22.934 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:59:22.934 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:22.934 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:59:22.934 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:59:22.934 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:59:22.934 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:59:22.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.935 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T08:59:22.942 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:59:22.942 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:59:22.942 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:59:22.942 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.943 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T08:59:22.943 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.943 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:22.943 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:22.943 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T08:59:22.943 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:22.943 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:22.943 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T08:59:22.943 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:59:22.943 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:22.944 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T08:59:22.944 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T08:59:22.944 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42078sec preferred_lft 42078sec 2024-04-25T08:59:22.944 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T08:59:22.944 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.945 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T08:59:22.945 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:22.945 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T08:59:22.945 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T08:59:22.945 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:22.945 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T08:59:22.945 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.945 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:59:22.945 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T08:59:22.945 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:59:22.946 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.946 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T08:59:22.946 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.946 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:59:22.946 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:22.946 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T08:59:22.946 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.946 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:59:22.946 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:22.946 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T08:59:22.947 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:59:22.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.947 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:59:22.996 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T08:59:22.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.998 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T08:59:23.047 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T08:59:23.048 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:59:23.048 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T08:59:23.048 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T08:59:23.048 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T08:59:23.048 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:59:23.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:23.048 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T08:59:23.048 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T08:59:23.048 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:59:23.048 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:59:23.049 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:59:23.049 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T08:59:23.105 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T08:59:23.112+0000 7f7c4b029640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:59:23.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:23.120 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T08:59:23.166 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:59:23.167 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:59:23.167 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:59:23.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:23.167 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T08:59:23.226 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:59:23.227 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:23.227 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:23.274 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:59:23.274 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:59:23.275 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:23.322 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:59:23.380 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:59:23.429 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:23.430 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:59:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:23.431 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:23.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:23.432 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:59:23.479 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:59:23.479 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:59:23.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:59:23.538 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:23.587 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:59:23.587 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:59:23.588 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:59:23.589 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:23.590 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:23.590 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:23.590 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:23.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:23.590 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:59:23.604 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T08:59:23.612+0000 7ffa23d57580 -1 init, newargv = 0x55bb1c9b31f0 newargc=15 2024-04-25T08:59:23.604 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48758]: starting ceph client 2024-04-25T08:59:23.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48758]: starting fuse 2024-04-25T08:59:24.710 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T08:59:24.727 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:59:24.777 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T08:59:24.777 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T08:59:24.777 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T08:59:24.777 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T08:59:24.778 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:59:24.779 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:24.780 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:59:24.780 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:24.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:24.781 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T08:59:24.827 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T08:59:24.827 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T08:59:24.827 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:59:24.827 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T08:59:24.827 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T08:59:24.828 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:59:24.829 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T08:59:24.829 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:59:24.829 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:59:24.829 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:59:24.829 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T08:59:24.829 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T08:59:24.829 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T08:59:24.829 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:59:24.829 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T08:59:24.829 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T08:59:24.829 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T08:59:24.954 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.48758.asok 2024-04-25T08:59:24.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:59:24.966 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48758.asok 2024-04-25T08:59:24.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:24.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48758.asok status 2024-04-25T08:59:25.128 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:59:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T08:59:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:59:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:59:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T08:59:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T08:59:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:59:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "48758", 2024-04-25T08:59:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T08:59:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 8082, 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 8082 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3221765314 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:59:25.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3221765314 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.8082 192.168.0.1:0/3221765314", 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/3221765314", 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 66, 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 98, 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T08:59:25.130 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:59:25.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:25.142 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T08:59:25.142 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:25.147 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T08:59:25.147 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:25.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:25.148 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:59:25.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:25.200 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-04-25T08:59:25.252 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T08:59:25.260+0000 7f7c47021640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T08:59:25.253 INFO:teuthology.orchestra.run.smithi037.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-04-25T08:59:25.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:59:25.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:25.255 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-04-25T08:59:25.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.444+0000 7f3269d20640 1 Processor -- start 2024-04-25T08:59:25.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.444+0000 7f3269d20640 1 -- start start 2024-04-25T08:59:25.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3269d20640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f326415d980 0x7f32640d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3269d20640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f32640d12c0 0x7f32640d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3269d20640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326415dd60 con 0x7f32640d21f0 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3269d20640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326406c810 con 0x7f326415d980 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3269d20640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32640d4340 con 0x7f32640d12c0 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f32637fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f326415d980 0x7f32640d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3263fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f32640d21f0 0x7f32640ce300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49614/0 (socket says 172.21.15.37:49614) 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3263fff640 1 -- 172.21.15.37:0/3006621004 learned_addr learned my addr 172.21.15.37:0/3006621004 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f325bfff640 1 --2- 172.21.15.37:0/3006621004 >> v2:172.21.15.203:3300/0 conn(0x7f32640d12c0 0x7f32640d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f32637fe640 1 -- 172.21.15.37:0/3006621004 >> v2:172.21.15.203:3300/0 conn(0x7f32640d12c0 msgr2=0x7f32640d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f32637fe640 1 --2- 172.21.15.37:0/3006621004 >> v2:172.21.15.203:3300/0 conn(0x7f32640d12c0 0x7f32640d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f32637fe640 1 -- 172.21.15.37:0/3006621004 >> v1:172.21.15.37:6789/0 conn(0x7f32640d21f0 legacy=0x7f32640ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f32637fe640 1 -- 172.21.15.37:0/3006621004 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32640d45e0 con 0x7f326415d980 2024-04-25T08:59:25.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f32637fe640 1 --2- 172.21.15.37:0/3006621004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f326415d980 0x7f32640d3e00 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f32640b61f0 tx=0x7f325401c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc117f7e51506ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3262ffd640 1 -- 172.21.15.37:0/3006621004 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540b0070 con 0x7f326415d980 2024-04-25T08:59:25.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3262ffd640 1 -- 172.21.15.37:0/3006621004 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32540ab600 con 0x7f326415d980 2024-04-25T08:59:25.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3262ffd640 1 -- 172.21.15.37:0/3006621004 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540ab9d0 con 0x7f326415d980 2024-04-25T08:59:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3269d20640 1 -- 172.21.15.37:0/3006621004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f326415d980 msgr2=0x7f32640d3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.448+0000 7f3269d20640 1 --2- 172.21.15.37:0/3006621004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f326415d980 0x7f32640d3e00 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f32640b61f0 tx=0x7f325401c280 comp rx=0 tx=0).stop 2024-04-25T08:59:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 -- 172.21.15.37:0/3006621004 shutdown_connections 2024-04-25T08:59:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 --2- 172.21.15.37:0/3006621004 >> v2:172.21.15.203:3300/0 conn(0x7f32640d12c0 0x7f32640d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 --2- 172.21.15.37:0/3006621004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f326415d980 0x7f32640d3e00 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 -- 172.21.15.37:0/3006621004 >> 172.21.15.37:0/3006621004 conn(0x7f326408b000 msgr2=0x7f32640d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 -- 172.21.15.37:0/3006621004 shutdown_connections 2024-04-25T08:59:25.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 -- 172.21.15.37:0/3006621004 wait complete. 2024-04-25T08:59:25.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 Processor -- start 2024-04-25T08:59:25.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 -- start start 2024-04-25T08:59:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32640d12c0 0x7f326414fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32640d21f0 0x7f32640c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f326415d980 0x7f32640c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326406c1c0 con 0x7f32640d12c0 2024-04-25T08:59:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326406bee0 con 0x7f32640d21f0 2024-04-25T08:59:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326406c4a0 con 0x7f326415d980 2024-04-25T08:59:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f32637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32640d12c0 0x7f326414fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f32637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32640d12c0 0x7f326414fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54044/0 (socket says 172.21.15.37:54044) 2024-04-25T08:59:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f32637fe640 1 -- 172.21.15.37:0/4154443794 learned_addr learned my addr 172.21.15.37:0/4154443794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f325bfff640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32640d21f0 0x7f32640c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3263fff640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f326415d980 0x7f32640c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f32637fe640 1 -- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f326415d980 msgr2=0x7f32640c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f32637fe640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f326415d980 0x7f32640c8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f32637fe640 1 -- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32640d21f0 msgr2=0x7f32640c91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f32637fe640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32640d21f0 0x7f32640c91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f32637fe640 1 -- 172.21.15.37:0/4154443794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32640c9890 con 0x7f32640d12c0 2024-04-25T08:59:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f32637fe640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32640d12c0 0x7f326414fcd0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f325401a930 tx=0x7f3254019020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3260ff9640 1 -- 172.21.15.37:0/4154443794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540b8020 con 0x7f32640d12c0 2024-04-25T08:59:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3260ff9640 1 -- 172.21.15.37:0/4154443794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32540aa730 con 0x7f32640d12c0 2024-04-25T08:59:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 -- 172.21.15.37:0/4154443794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32640c9b70 con 0x7f32640d12c0 2024-04-25T08:59:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3260ff9640 1 -- 172.21.15.37:0/4154443794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540b0040 con 0x7f32640d12c0 2024-04-25T08:59:25.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.452+0000 7f3269d20640 1 -- 172.21.15.37:0/4154443794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32640ca130 con 0x7f32640d12c0 2024-04-25T08:59:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3260ff9640 1 -- 172.21.15.37:0/4154443794 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f32540bb020 con 0x7f32640d12c0 2024-04-25T08:59:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3260ff9640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3244066f60 0x7f3244069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3260ff9640 1 -- 172.21.15.37:0/4154443794 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f325411f3e0 con 0x7f32640d12c0 2024-04-25T08:59:25.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f325bfff640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3244066f60 0x7f3244069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f326ce8d000 1 Processor -- start 2024-04-25T08:59:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f326ce8d000 1 -- start start 2024-04-25T08:59:25.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f325bfff640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3244066f60 0x7f3244069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f326415d180 tx=0x7f324c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e111571b0 0x561e11157590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e11157ad0 0x561e1113b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e1113bf10 0x561e1113c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f326ce8d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e10f5afd0 con 0x561e1113bf10 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f326ce8d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e10ee3170 con 0x561e11157ad0 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f326ce8d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e10f57070 con 0x561e111571b0 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3263fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e1113bf10 0x561e1113c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3263fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e1113bf10 0x561e1113c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54054/0 (socket says 172.21.15.37:54054) 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3263fff640 1 -- 172.21.15.37:0/1000355002 learned_addr learned my addr 172.21.15.37:0/1000355002 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f325bfff640 1 --2- 172.21.15.37:0/1000355002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e11157ad0 0x561e1113b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f32637fe640 1 --2- 172.21.15.37:0/1000355002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e111571b0 0x561e11157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3263fff640 1 -- 172.21.15.37:0/1000355002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e111571b0 msgr2=0x561e11157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3263fff640 1 --2- 172.21.15.37:0/1000355002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e111571b0 0x561e11157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3263fff640 1 -- 172.21.15.37:0/1000355002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e11157ad0 msgr2=0x561e1113b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3263fff640 1 --2- 172.21.15.37:0/1000355002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e11157ad0 0x561e1113b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3263fff640 1 -- 172.21.15.37:0/1000355002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e10fa1d20 con 0x561e1113bf10 2024-04-25T08:59:25.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f3263fff640 1 --2- 172.21.15.37:0/1000355002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e1113bf10 0x561e1113c2f0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f325c00d8a0 tx=0x7f325c00dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea3c7d9ca9d0628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f32597fa640 1 -- 172.21.15.37:0/1000355002 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c00fb40 con 0x561e1113bf10 2024-04-25T08:59:25.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f32597fa640 1 -- 172.21.15.37:0/1000355002 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c019070 con 0x561e1113bf10 2024-04-25T08:59:25.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.456+0000 7f32597fa640 1 -- 172.21.15.37:0/1000355002 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c014410 con 0x561e1113bf10 2024-04-25T08:59:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- 172.21.15.37:0/1000355002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e1113bf10 msgr2=0x561e1113c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 --2- 172.21.15.37:0/1000355002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e1113bf10 0x561e1113c2f0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f325c00d8a0 tx=0x7f325c00dd60 comp rx=0 tx=0).stop 2024-04-25T08:59:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- 172.21.15.37:0/1000355002 shutdown_connections 2024-04-25T08:59:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 --2- 172.21.15.37:0/1000355002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e1113bf10 0x561e1113c2f0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 --2- 172.21.15.37:0/1000355002 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e11157ad0 0x561e1113b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 --2- 172.21.15.37:0/1000355002 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e111571b0 0x561e11157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- 172.21.15.37:0/1000355002 >> 172.21.15.37:0/1000355002 conn(0x561e11113710 msgr2=0x561e11113af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- 172.21.15.37:0/1000355002 shutdown_connections 2024-04-25T08:59:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- 172.21.15.37:0/1000355002 wait complete. 2024-04-25T08:59:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- start start 2024-04-25T08:59:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 Processor -- start 2024-04-25T08:59:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e1113bf10 0x561e1115e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e111571b0 0x561e1115ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11157ad0 0x561e1113c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e11018c10 con 0x561e11157ad0 2024-04-25T08:59:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e10f15680 con 0x561e111571b0 2024-04-25T08:59:25.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e10f5f600 con 0x561e1113bf10 2024-04-25T08:59:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 10 client.0 _ll_register_callbacks cb 0x561e11010080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f3263fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11157ad0 0x561e1113c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f3263fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11157ad0 0x561e1113c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54056/0 (socket says 172.21.15.37:54056) 2024-04-25T08:59:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f3263fff640 1 -- 172.21.15.37:0/2232459338 learned_addr learned my addr 172.21.15.37:0/2232459338 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:25.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f32637fe640 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e1113bf10 0x561e1115e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f325bfff640 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e111571b0 0x561e1115ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f3263fff640 1 -- 172.21.15.37:0/2232459338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e1113bf10 msgr2=0x561e1115e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f3263fff640 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e1113bf10 0x561e1115e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f3263fff640 1 -- 172.21.15.37:0/2232459338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e111571b0 msgr2=0x561e1115ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f3263fff640 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e111571b0 0x561e1115ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f3263fff640 1 -- 172.21.15.37:0/2232459338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e10f32d30 con 0x561e11157ad0 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f3263fff640 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11157ad0 0x561e1113c2f0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f325c016040 tx=0x7f325c000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=157cf434a715cc5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f32427fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f32427fc640 1 -- 172.21.15.37:0/2232459338 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c014960 con 0x561e11157ad0 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f32427fc640 1 -- 172.21.15.37:0/2232459338 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f325c014b00 con 0x561e11157ad0 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- 172.21.15.37:0/2232459338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f325c019070 con 0x561e11157ad0 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f32427fc640 1 -- 172.21.15.37:0/2232459338 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f325c031050 con 0x561e11157ad0 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 10 client.8097 fetch_fsmap learned FSMap version 66 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- 172.21.15.37:0/2232459338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x561e10ee33c0 con 0x561e11157ad0 2024-04-25T08:59:25.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f32427fc640 1 -- 172.21.15.37:0/2232459338 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325c019070 con 0x561e11157ad0 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f32427fc640 1 -- 172.21.15.37:0/2232459338 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f325c034020 con 0x561e11157ad0 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 10 client.8097 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 10 client.8097 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 20 client.8097 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x561e1113d090 0x561e1113d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 4 client.8097 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f326ce8d000 1 -- 172.21.15.37:0/2232459338 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e10ef6e00 con 0x561e1113d090 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.460+0000 7f32637fe640 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x561e1113d090 0x561e1113d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f32637fe640 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x561e1113d090 0x561e1113d470 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0af5a14cdcb084 server_cookie=6ab0e36d5b847bd6 in_seq=0 out_seq=0 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f32427fc640 10 client.8097 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f32427fc640 1 -- 172.21.15.37:0/2232459338 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e10ef6e00 con 0x561e1113d090 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f32427fc640 10 client.8097 handle_command_reply: tid=0 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 1 client.8097 shutdown 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 1 -- 172.21.15.37:0/2232459338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11157ad0 msgr2=0x561e1113c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11157ad0 0x561e1113c2f0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f325c016040 tx=0x7f325c000f50 comp rx=0 tx=0).stop 2024-04-25T08:59:25.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 1 -- 172.21.15.37:0/2232459338 shutdown_connections 2024-04-25T08:59:25.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x561e1113d090 0x561e1113d470 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11157ad0 0x561e1113c2f0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e111571b0 0x561e1115ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 1 --2- 172.21.15.37:0/2232459338 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e1113bf10 0x561e1115e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 1 -- 172.21.15.37:0/2232459338 >> 172.21.15.37:0/2232459338 conn(0x561e11113710 msgr2=0x561e1110b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f32427fc640 0 client.8097 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:25.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 1 -- 172.21.15.37:0/2232459338 shutdown_connections 2024-04-25T08:59:25.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 1 -- 172.21.15.37:0/2232459338 wait complete. 2024-04-25T08:59:25.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.464+0000 7f326ce8d000 20 client.8097 trim_cache size 0 max 16384 2024-04-25T08:59:25.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 Processor -- start 2024-04-25T08:59:25.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 -- start start 2024-04-25T08:59:25.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11113710 0x561e1110b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e1113bf10 0x561e1110b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e111571b0 0x561e1110bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3264200600 con 0x561e11113710 2024-04-25T08:59:25.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32641ff720 con 0x561e1113bf10 2024-04-25T08:59:25.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32640a12c0 con 0x561e111571b0 2024-04-25T08:59:25.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f32637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11113710 0x561e1110b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f32637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11113710 0x561e1110b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54072/0 (socket says 172.21.15.37:54072) 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f32637fe640 1 -- 172.21.15.37:0/2335524469 learned_addr learned my addr 172.21.15.37:0/2335524469 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f3263fff640 1 --2- 172.21.15.37:0/2335524469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e111571b0 0x561e1110bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f325bfff640 1 --2- 172.21.15.37:0/2335524469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e1113bf10 0x561e1110b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f32637fe640 1 -- 172.21.15.37:0/2335524469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e111571b0 msgr2=0x561e1110bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f32637fe640 1 --2- 172.21.15.37:0/2335524469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e111571b0 0x561e1110bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f32637fe640 1 -- 172.21.15.37:0/2335524469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e1113bf10 msgr2=0x561e1110b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f32637fe640 1 --2- 172.21.15.37:0/2335524469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e1113bf10 0x561e1110b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f32637fe640 1 -- 172.21.15.37:0/2335524469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e10fdd0f0 con 0x561e11113710 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f3263fff640 1 --2- 172.21.15.37:0/2335524469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e111571b0 0x561e1110bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f325bfff640 1 --2- 172.21.15.37:0/2335524469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e1113bf10 0x561e1110b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f32637fe640 1 --2- 172.21.15.37:0/2335524469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11113710 0x561e1110b0c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f32540ebe60 tx=0x7f325411d4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3ef6a45e698fdef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f3258ff9640 1 -- 172.21.15.37:0/2335524469 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540edd00 con 0x561e11113710 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f3258ff9640 1 -- 172.21.15.37:0/2335524469 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325411f0b0 con 0x561e11113710 2024-04-25T08:59:25.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f3258ff9640 1 -- 172.21.15.37:0/2335524469 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540f7c80 con 0x561e11113710 2024-04-25T08:59:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 -- 172.21.15.37:0/2335524469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11113710 msgr2=0x561e1110b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 --2- 172.21.15.37:0/2335524469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11113710 0x561e1110b0c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f32540ebe60 tx=0x7f325411d4f0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 -- 172.21.15.37:0/2335524469 shutdown_connections 2024-04-25T08:59:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 --2- 172.21.15.37:0/2335524469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e111571b0 0x561e1110bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 --2- 172.21.15.37:0/2335524469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e1113bf10 0x561e1110b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 --2- 172.21.15.37:0/2335524469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e11113710 0x561e1110b0c0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 -- 172.21.15.37:0/2335524469 >> 172.21.15.37:0/2335524469 conn(0x561e11164b10 msgr2=0x561e1113b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 -- 172.21.15.37:0/2335524469 shutdown_connections 2024-04-25T08:59:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 -- 172.21.15.37:0/2335524469 wait complete. 2024-04-25T08:59:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 -- start start 2024-04-25T08:59:25.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.480+0000 7f326ce8d000 1 Processor -- start 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e11113710 0x561e1110dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e1113bf10 0x561e11173e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e111571b0 0x561e11174360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e10f1e860 con 0x561e111571b0 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32641ff100 con 0x561e1113bf10 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e10f5afd0 con 0x561e11113710 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 10 client.0 _ll_register_callbacks cb 0x561e10ef8b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3263fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e111571b0 0x561e11174360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3263fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e111571b0 0x561e11174360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54086/0 (socket says 172.21.15.37:54086) 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3263fff640 1 -- 172.21.15.37:0/1415947083 learned_addr learned my addr 172.21.15.37:0/1415947083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f325bfff640 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e1113bf10 0x561e11173e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32637fe640 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e11113710 0x561e1110dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3263fff640 1 -- 172.21.15.37:0/1415947083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e11113710 msgr2=0x561e1110dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3263fff640 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e11113710 0x561e1110dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3263fff640 1 -- 172.21.15.37:0/1415947083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e1113bf10 msgr2=0x561e11173e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3263fff640 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e1113bf10 0x561e11173e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3263fff640 1 -- 172.21.15.37:0/1415947083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e10eefd20 con 0x561e111571b0 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32637fe640 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e11113710 0x561e1110dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3263fff640 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e111571b0 0x561e11174360 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f325c001230 tx=0x7f325c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb866763fbc4dc73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32427fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32427fc640 1 -- 172.21.15.37:0/1415947083 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325c039070 con 0x561e111571b0 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32427fc640 1 -- 172.21.15.37:0/1415947083 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f325c029ac0 con 0x561e111571b0 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 -- 172.21.15.37:0/1415947083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f325411f0b0 con 0x561e111571b0 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32427fc640 1 -- 172.21.15.37:0/1415947083 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f325c044050 con 0x561e111571b0 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 10 client.8103 fetch_fsmap learned FSMap version 66 2024-04-25T08:59:25.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 -- 172.21.15.37:0/1415947083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x561e10f020c0 con 0x561e111571b0 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32427fc640 1 -- 172.21.15.37:0/1415947083 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325411f0b0 con 0x561e111571b0 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32427fc640 1 -- 172.21.15.37:0/1415947083 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f325c047020 con 0x561e111571b0 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 10 client.8103 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 10 client.8103 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 20 client.8103 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x561e11157ad0 0x561e111b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 4 client.8103 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 -- 172.21.15.37:0/1415947083 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) v1 -- 0x561e10f68510 con 0x561e11157ad0 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32637fe640 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x561e11157ad0 0x561e111b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32427fc640 10 client.8103 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32637fe640 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x561e11157ad0 0x561e111b7a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57c3c7ac8dc65dfc server_cookie=ee6dcd751be40986 in_seq=0 out_seq=0 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32427fc640 1 -- 172.21.15.37:0/1415947083 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561e10f68510 con 0x561e11157ad0 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32427fc640 10 client.8103 handle_command_reply: tid=0 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 client.8103 shutdown 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 -- 172.21.15.37:0/1415947083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e111571b0 msgr2=0x561e11174360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e111571b0 0x561e11174360 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f325c001230 tx=0x7f325c00b040 comp rx=0 tx=0).stop 2024-04-25T08:59:25.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 -- 172.21.15.37:0/1415947083 shutdown_connections 2024-04-25T08:59:25.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x561e11157ad0 0x561e111b7a90 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561e111571b0 0x561e11174360 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x561e1113bf10 0x561e11173e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 --2- 172.21.15.37:0/1415947083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x561e11113710 0x561e1110dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 -- 172.21.15.37:0/1415947083 >> 172.21.15.37:0/1415947083 conn(0x561e11164b10 msgr2=0x561e11111a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f32427fc640 0 client.8103 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:25.482 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f7c47021640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-25T08:59:06.530103+0000 1=1+0) n(v0 rc2024-04-25T08:59:06.639471+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x560a400ebb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T08:59:06.530103+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T08:59:06.639471+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T08:59:25.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 -- 172.21.15.37:0/1415947083 shutdown_connections 2024-04-25T08:59:25.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 1 -- 172.21.15.37:0/1415947083 wait complete. 2024-04-25T08:59:25.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f326ce8d000 20 client.8103 trim_cache size 0 max 16384 2024-04-25T08:59:25.483 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:59:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-04-25T08:59:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "12f075cd-defa-4e99-8d54-d2f1d91bd495", 2024-04-25T08:59:25.483 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-04-25T08:59:25.483 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:59:25.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3269d20640 1 -- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3244066f60 msgr2=0x7f3244069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3269d20640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3244066f60 0x7f3244069420 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f326415d180 tx=0x7f324c008040 comp rx=0 tx=0).stop 2024-04-25T08:59:25.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3269d20640 1 -- 172.21.15.37:0/4154443794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32640d12c0 msgr2=0x7f326414fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3269d20640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32640d12c0 0x7f326414fcd0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f325401a930 tx=0x7f3254019020 comp rx=0 tx=0).stop 2024-04-25T08:59:25.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3269d20640 1 -- 172.21.15.37:0/4154443794 shutdown_connections 2024-04-25T08:59:25.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3269d20640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3244066f60 0x7f3244069420 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3269d20640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f326415d980 0x7f32640c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3269d20640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f32640d21f0 0x7f32640c91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3269d20640 1 --2- 172.21.15.37:0/4154443794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f32640d12c0 0x7f326414fcd0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.484+0000 7f3269d20640 1 -- 172.21.15.37:0/4154443794 >> 172.21.15.37:0/4154443794 conn(0x7f326408b000 msgr2=0x7f32640b8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.488+0000 7f3269d20640 1 -- 172.21.15.37:0/4154443794 shutdown_connections 2024-04-25T08:59:25.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.488+0000 7f3269d20640 1 -- 172.21.15.37:0/4154443794 wait complete. 2024-04-25T08:59:25.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:25.494 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:59:25.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.620+0000 7f605cdc8640 1 Processor -- start 2024-04-25T08:59:25.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.620+0000 7f605cdc8640 1 -- start start 2024-04-25T08:59:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605cdc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f605815b6b0 0x7f60580ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605cdc8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605815ff50 con 0x7f60580cf590 2024-04-25T08:59:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605cdc8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60580cbd20 con 0x7f6058073010 2024-04-25T08:59:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605cdc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605815ba90 con 0x7f605815b6b0 2024-04-25T08:59:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f6055d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f60580cf590 0x7f605815f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49616/0 (socket says 172.21.15.37:49616) 2024-04-25T08:59:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f6055d9b640 1 -- 172.21.15.37:0/1946262460 learned_addr learned my addr 172.21.15.37:0/1946262460 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605659c640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f605815b6b0 0x7f60580ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487175052 0 0) 0x7f605815ff50 con 0x7f60580cf590 2024-04-25T08:59:25.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6040003530 con 0x7f60580cf590 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1824954855 0 0) 0x7f6040003530 con 0x7f60580cf590 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f605815b6b0 msgr2=0x7f60580ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f605815b6b0 0x7f60580ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 >> v1:172.21.15.203:6789/0 conn(0x7f6058073010 legacy=0x7f605815f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60580cc000 con 0x7f60580cf590 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f604c001680 con 0x7f60580cf590 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 >> v1:172.21.15.37:6789/0 conn(0x7f60580cf590 legacy=0x7f605815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f605815b6b0 0x7f6040003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040003fe0 0x7f6040004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6040004d50 0x7f60400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605815ba90 con 0x7f6040003fe0 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f604000bb30 con 0x7f6040004d50 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f604000bd30 con 0x7f605815b6b0 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f6055d9b640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040003fe0 0x7f6040004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f605659c640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f605815b6b0 0x7f6040003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f6056d9d640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6040004d50 0x7f60400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f6055d9b640 1 -- 172.21.15.37:0/1946262460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f605815b6b0 msgr2=0x7f6040003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f6055d9b640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f605815b6b0 0x7f6040003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f6055d9b640 1 -- 172.21.15.37:0/1946262460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6040004d50 msgr2=0x7f60400051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f6055d9b640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6040004d50 0x7f60400051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.624+0000 7f6055d9b640 1 -- 172.21.15.37:0/1946262460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6040003530 con 0x7f6040003fe0 2024-04-25T08:59:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f6055d9b640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040003fe0 0x7f6040004810 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f604c00c210 tx=0x7f604c01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc2140daa5ebf4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c007d40 con 0x7f6040003fe0 2024-04-25T08:59:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604c005b00 con 0x7f6040003fe0 2024-04-25T08:59:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605559a640 1 -- 172.21.15.37:0/1946262460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c007070 con 0x7f6040003fe0 2024-04-25T08:59:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 -- 172.21.15.37:0/1946262460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040003fe0 msgr2=0x7f6040004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 --2- 172.21.15.37:0/1946262460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6040003fe0 0x7f6040004810 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f604c00c210 tx=0x7f604c01f330 comp rx=0 tx=0).stop 2024-04-25T08:59:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f6056d9d640 1 -- 172.21.15.37:0/1946262460 reap_dead start 2024-04-25T08:59:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 -- 172.21.15.37:0/1946262460 shutdown_connections 2024-04-25T08:59:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 -- 172.21.15.37:0/1946262460 >> 172.21.15.37:0/1946262460 conn(0x7f605806ae60 msgr2=0x7f605806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 -- 172.21.15.37:0/1946262460 shutdown_connections 2024-04-25T08:59:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 -- 172.21.15.37:0/1946262460 wait complete. 2024-04-25T08:59:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 Processor -- start 2024-04-25T08:59:25.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 -- start start 2024-04-25T08:59:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6040003fe0 0x7f6058154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60580ce830 0x7f60580accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60580cecb0 0x7f60580ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60580cc8a0 con 0x7f60580cecb0 2024-04-25T08:59:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60580cc5c0 con 0x7f60580ce830 2024-04-25T08:59:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605cdc8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60580cba30 con 0x7f6040003fe0 2024-04-25T08:59:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f6056d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60580cecb0 0x7f60580ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f6056d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60580cecb0 0x7f60580ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54112/0 (socket says 172.21.15.37:54112) 2024-04-25T08:59:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f6056d9d640 1 -- 172.21.15.37:0/3405744707 learned_addr learned my addr 172.21.15.37:0/3405744707 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f6055d9b640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60580ce830 0x7f60580accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f605659c640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6040003fe0 0x7f6058154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f6056d9d640 1 -- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6040003fe0 msgr2=0x7f6058154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f6056d9d640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6040003fe0 0x7f6058154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f6056d9d640 1 -- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60580ce830 msgr2=0x7f60580accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f6056d9d640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60580ce830 0x7f60580accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.628+0000 7f6056d9d640 1 -- 172.21.15.37:0/3405744707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605815ddc0 con 0x7f60580cecb0 2024-04-25T08:59:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f6056d9d640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60580cecb0 0x7f60580ad210 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f60580d0440 tx=0x7f605000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f6046ffd640 1 -- 172.21.15.37:0/3405744707 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050003980 con 0x7f60580cecb0 2024-04-25T08:59:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f6046ffd640 1 -- 172.21.15.37:0/3405744707 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6050018070 con 0x7f60580cecb0 2024-04-25T08:59:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f6046ffd640 1 -- 172.21.15.37:0/3405744707 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050003d60 con 0x7f60580cecb0 2024-04-25T08:59:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f605cdc8640 1 -- 172.21.15.37:0/3405744707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605815e0a0 con 0x7f60580cecb0 2024-04-25T08:59:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f605cdc8640 1 -- 172.21.15.37:0/3405744707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605815e660 con 0x7f60580cecb0 2024-04-25T08:59:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f6046ffd640 1 -- 172.21.15.37:0/3405744707 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f60500050d0 con 0x7f60580cecb0 2024-04-25T08:59:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f6046ffd640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6038066f60 0x7f6038069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f6046ffd640 1 -- 172.21.15.37:0/3405744707 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6050084fe0 con 0x7f60580cecb0 2024-04-25T08:59:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f605659c640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6038066f60 0x7f6038069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f605ff35000 1 Processor -- start 2024-04-25T08:59:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f605ff35000 1 -- start start 2024-04-25T08:59:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.632+0000 7f605659c640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6038066f60 0x7f6038069420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f605815d400 tx=0x7f6048008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27d9d0 0x557c2a27ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a233b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a234480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2a080ff0 con 0x557c2a27e2f0 2024-04-25T08:59:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2a009150 con 0x557c2a27d9d0 2024-04-25T08:59:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c2a07d090 con 0x557c2a2340a0 2024-04-25T08:59:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a233b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a233b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54120/0 (socket says 172.21.15.37:54120) 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 -- 172.21.15.37:0/3822025054 learned_addr learned my addr 172.21.15.37:0/3822025054 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605659c640 1 --2- 172.21.15.37:0/3822025054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27d9d0 0x557c2a27ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6056d9d640 1 --2- 172.21.15.37:0/3822025054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a234480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 -- 172.21.15.37:0/3822025054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 msgr2=0x557c2a234480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 --2- 172.21.15.37:0/3822025054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a234480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 -- 172.21.15.37:0/3822025054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27d9d0 msgr2=0x557c2a27ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 --2- 172.21.15.37:0/3822025054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27d9d0 0x557c2a27ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 -- 172.21.15.37:0/3822025054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c2a2170d0 con 0x557c2a27e2f0 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 --2- 172.21.15.37:0/3822025054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a233b60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f604c000c10 tx=0x7f604c001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c666960bd888ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6037fff640 1 -- 172.21.15.37:0/3822025054 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c001700 con 0x557c2a27e2f0 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6037fff640 1 -- 172.21.15.37:0/3822025054 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604c0047f0 con 0x557c2a27e2f0 2024-04-25T08:59:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6037fff640 1 -- 172.21.15.37:0/3822025054 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c004ba0 con 0x557c2a27e2f0 2024-04-25T08:59:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- 172.21.15.37:0/3822025054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 msgr2=0x557c2a233b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 --2- 172.21.15.37:0/3822025054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a233b60 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f604c000c10 tx=0x7f604c001a50 comp rx=0 tx=0).stop 2024-04-25T08:59:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- 172.21.15.37:0/3822025054 shutdown_connections 2024-04-25T08:59:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 --2- 172.21.15.37:0/3822025054 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a234480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 --2- 172.21.15.37:0/3822025054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a233b60 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 --2- 172.21.15.37:0/3822025054 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27d9d0 0x557c2a27ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- 172.21.15.37:0/3822025054 >> 172.21.15.37:0/3822025054 conn(0x557c2a230020 msgr2=0x557c2a232440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- 172.21.15.37:0/3822025054 shutdown_connections 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- 172.21.15.37:0/3822025054 wait complete. 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- start start 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 Processor -- start 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a2627f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27d9d0 0x557c2a262d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27e2f0 0x557c2a263270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2a045250 con 0x557c2a27d9d0 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2a03b690 con 0x557c2a27e2f0 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c2a009a40 con 0x557c2a2340a0 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f605ff35000 10 client.0 _ll_register_callbacks cb 0x557c2a21ee90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27d9d0 0x557c2a262d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27d9d0 0x557c2a262d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54130/0 (socket says 172.21.15.37:54130) 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.636+0000 7f6055d9b640 1 -- 172.21.15.37:0/2739059817 learned_addr learned my addr 172.21.15.37:0/2739059817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605659c640 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a2627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f6056d9d640 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27e2f0 0x557c2a263270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f6055d9b640 1 -- 172.21.15.37:0/2739059817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 msgr2=0x557c2a2627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f6055d9b640 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a2627f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f6055d9b640 1 -- 172.21.15.37:0/2739059817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27e2f0 msgr2=0x557c2a263270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f6055d9b640 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27e2f0 0x557c2a263270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f6055d9b640 1 -- 172.21.15.37:0/2739059817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c2a12e1c0 con 0x557c2a27d9d0 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f6055d9b640 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27d9d0 0x557c2a262d30 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f604c001ec0 tx=0x7f604c01c0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ced8742e914d06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f60357fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f60357fa640 1 -- 172.21.15.37:0/2739059817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c0016a0 con 0x557c2a27d9d0 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f60357fa640 1 -- 172.21.15.37:0/2739059817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f604c0c8d70 con 0x557c2a27d9d0 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 -- 172.21.15.37:0/2739059817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f604c0047f0 con 0x557c2a27d9d0 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f60357fa640 1 -- 172.21.15.37:0/2739059817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f604c0049a0 con 0x557c2a27d9d0 2024-04-25T08:59:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 10 client.8115 fetch_fsmap learned FSMap version 66 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 -- 172.21.15.37:0/2739059817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x557c2a12a170 con 0x557c2a27d9d0 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f60357fa640 1 -- 172.21.15.37:0/2739059817 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604c0047f0 con 0x557c2a27d9d0 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f60357fa640 1 -- 172.21.15.37:0/2739059817 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f604c004b40 con 0x557c2a27d9d0 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 10 client.8115 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 10 client.8115 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 20 client.8115 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x557c2a28dc80 0x557c2a290070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 4 client.8115 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 -- 172.21.15.37:0/2739059817 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c2a01cc60 con 0x557c2a28dc80 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605659c640 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x557c2a28dc80 0x557c2a290070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605659c640 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x557c2a28dc80 0x557c2a290070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed29e9b900baffe2 server_cookie=a7db77e253cc06b2 in_seq=0 out_seq=0 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f60357fa640 10 client.8115 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f60357fa640 1 -- 172.21.15.37:0/2739059817 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c2a01cc60 con 0x557c2a28dc80 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f60357fa640 10 client.8115 handle_command_reply: tid=0 2024-04-25T08:59:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 client.8115 shutdown 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 -- 172.21.15.37:0/2739059817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27d9d0 msgr2=0x557c2a262d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27d9d0 0x557c2a262d30 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f604c001ec0 tx=0x7f604c01c0d0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 -- 172.21.15.37:0/2739059817 shutdown_connections 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x557c2a28dc80 0x557c2a290070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27e2f0 0x557c2a263270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27d9d0 0x557c2a262d30 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 --2- 172.21.15.37:0/2739059817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a2627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f60357fa640 0 client.8115 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 -- 172.21.15.37:0/2739059817 >> 172.21.15.37:0/2739059817 conn(0x557c2a230020 msgr2=0x557c2a232ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 -- 172.21.15.37:0/2739059817 shutdown_connections 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 1 -- 172.21.15.37:0/2739059817 wait complete. 2024-04-25T08:59:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.640+0000 7f605ff35000 20 client.8115 trim_cache size 0 max 16384 2024-04-25T08:59:25.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605ff35000 1 Processor -- start 2024-04-25T08:59:25.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605ff35000 1 -- start start 2024-04-25T08:59:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605ff35000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a2340a0 0x557c2a2da300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605ff35000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a27d9d0 0x557c2a2da840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605ff35000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27e2f0 0x557c2a2dad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605ff35000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6058200c00 con 0x557c2a2340a0 2024-04-25T08:59:25.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605ff35000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60581ffdf0 con 0x557c2a27e2f0 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605ff35000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60580c59e0 con 0x557c2a27d9d0 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a2340a0 0x557c2a2da300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a2340a0 0x557c2a2da300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54136/0 (socket says 172.21.15.37:54136) 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605659c640 1 -- 172.21.15.37:0/3105408666 learned_addr learned my addr 172.21.15.37:0/3105408666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f6056d9d640 1 --2- 172.21.15.37:0/3105408666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27e2f0 0x557c2a2dad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f6055d9b640 1 --2- 172.21.15.37:0/3105408666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a27d9d0 0x557c2a2da840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605659c640 1 -- 172.21.15.37:0/3105408666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a27d9d0 msgr2=0x557c2a2da840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605659c640 1 --2- 172.21.15.37:0/3105408666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a27d9d0 0x557c2a2da840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605659c640 1 -- 172.21.15.37:0/3105408666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27e2f0 msgr2=0x557c2a2dad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605659c640 1 --2- 172.21.15.37:0/3105408666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27e2f0 0x557c2a2dad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605659c640 1 -- 172.21.15.37:0/3105408666 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c2a0c7de0 con 0x557c2a2340a0 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f6056d9d640 1 --2- 172.21.15.37:0/3105408666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27e2f0 0x557c2a2dad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605659c640 1 --2- 172.21.15.37:0/3105408666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a2340a0 0x557c2a2da300 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f60480098f0 tx=0x7f6048016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c036a60986c630e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f60377fe640 1 -- 172.21.15.37:0/3105408666 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048003b20 con 0x557c2a2340a0 2024-04-25T08:59:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f60377fe640 1 -- 172.21.15.37:0/3105408666 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6048031070 con 0x557c2a2340a0 2024-04-25T08:59:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f60377fe640 1 -- 172.21.15.37:0/3105408666 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604802b150 con 0x557c2a2340a0 2024-04-25T08:59:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605ff35000 1 -- 172.21.15.37:0/3105408666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a2340a0 msgr2=0x557c2a2da300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.656+0000 7f605ff35000 1 --2- 172.21.15.37:0/3105408666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a2340a0 0x557c2a2da300 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f60480098f0 tx=0x7f6048016000 comp rx=0 tx=0).stop 2024-04-25T08:59:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/3105408666 shutdown_connections 2024-04-25T08:59:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- 172.21.15.37:0/3105408666 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27e2f0 0x557c2a2dad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- 172.21.15.37:0/3105408666 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a27d9d0 0x557c2a2da840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- 172.21.15.37:0/3105408666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a2340a0 0x557c2a2da300 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/3105408666 >> 172.21.15.37:0/3105408666 conn(0x557c2a230020 msgr2=0x557c2a28afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/3105408666 shutdown_connections 2024-04-25T08:59:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/3105408666 wait complete. 2024-04-25T08:59:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- start start 2024-04-25T08:59:25.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 Processor -- start 2024-04-25T08:59:25.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a2de310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27d9d0 0x557c2a2de850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a290380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c2a01be00 con 0x557c2a27e2f0 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60581ff7f0 con 0x557c2a27d9d0 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c2a080300 con 0x557c2a2340a0 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 10 client.0 _ll_register_callbacks cb 0x557c2a1ff040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f6056d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a290380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f6056d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a290380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54142/0 (socket says 172.21.15.37:54142) 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f6056d9d640 1 -- 172.21.15.37:0/2858174187 learned_addr learned my addr 172.21.15.37:0/2858174187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f6055d9b640 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27d9d0 0x557c2a2de850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605659c640 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a2de310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f6056d9d640 1 -- 172.21.15.37:0/2858174187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 msgr2=0x557c2a2de310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f6056d9d640 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a2de310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f6056d9d640 1 -- 172.21.15.37:0/2858174187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27d9d0 msgr2=0x557c2a2de850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:25.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f6056d9d640 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27d9d0 0x557c2a2de850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f6056d9d640 1 -- 172.21.15.37:0/2858174187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c2a0228e0 con 0x557c2a27e2f0 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f6056d9d640 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a290380 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f60500835f0 tx=0x7f60500550b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2be1203ce2cba3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f60357fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f60357fa640 1 -- 172.21.15.37:0/2858174187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605005dd70 con 0x557c2a27e2f0 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f60357fa640 1 -- 172.21.15.37:0/2858174187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6050051cd0 con 0x557c2a27e2f0 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/2858174187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6048031070 con 0x557c2a27e2f0 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f60357fa640 1 -- 172.21.15.37:0/2858174187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f605007cbe0 con 0x557c2a27e2f0 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 10 client.8121 fetch_fsmap learned FSMap version 66 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/2858174187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=2+}) v3 -- 0x557c2a0da8f0 con 0x557c2a27e2f0 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f60357fa640 1 -- 172.21.15.37:0/2858174187 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6048031070 con 0x557c2a27e2f0 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f60357fa640 1 -- 172.21.15.37:0/2858174187 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 66) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6050054ab0 con 0x557c2a27e2f0 2024-04-25T08:59:25.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 10 client.8121 fetch_fsmap finished waiting for FSMap version 66 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 10 client.8121 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 20 client.8121 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x557c2a290fd0 0x557c2a2913e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 4 client.8121 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/2858174187 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557c2a08e530 con 0x557c2a290fd0 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605659c640 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x557c2a290fd0 0x557c2a2913e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605659c640 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x557c2a290fd0 0x557c2a2913e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db7ece805656e448 server_cookie=f1d6a24cbd1c1b76 in_seq=0 out_seq=0 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f60357fa640 10 client.8121 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f60357fa640 1 -- 172.21.15.37:0/2858174187 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+256 (crc 0 0 0) 0x557c2a08e530 con 0x557c2a290fd0 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f60357fa640 10 client.8121 handle_command_reply: tid=0 2024-04-25T08:59:25.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 client.8121 shutdown 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/2858174187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 msgr2=0x557c2a290380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a290380 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f60500835f0 tx=0x7f60500550b0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/2858174187 shutdown_connections 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x557c2a290fd0 0x557c2a2913e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c2a27e2f0 0x557c2a290380 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c2a27d9d0 0x557c2a2de850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f60357fa640 0 client.8121 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 --2- 172.21.15.37:0/2858174187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c2a2340a0 0x557c2a2de310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/2858174187 >> 172.21.15.37:0/2858174187 conn(0x557c2a230020 msgr2=0x557c2a27c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/2858174187 shutdown_connections 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 1 -- 172.21.15.37:0/2858174187 wait complete. 2024-04-25T08:59:25.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.660+0000 7f605ff35000 20 client.8121 trim_cache size 0 max 16384 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "12f075cd-defa-4e99-8d54-d2f1d91bd495": { 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/dir", 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "12f075cd-defa-4e99-8d54-d2f1d91bd495", 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive" 2024-04-25T08:59:25.661 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:59:25.662 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:59:25.662 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:59:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 -- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6038066f60 msgr2=0x7f6038069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6038066f60 0x7f6038069420 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f605815d400 tx=0x7f6048008040 comp rx=0 tx=0).stop 2024-04-25T08:59:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 -- 172.21.15.37:0/3405744707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60580cecb0 msgr2=0x7f60580ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60580cecb0 0x7f60580ad210 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f60580d0440 tx=0x7f605000ec10 comp rx=0 tx=0).stop 2024-04-25T08:59:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 -- 172.21.15.37:0/3405744707 shutdown_connections 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6038066f60 0x7f6038069420 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60580cecb0 0x7f60580ad210 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f60580ce830 0x7f60580accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 --2- 172.21.15.37:0/3405744707 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6040003fe0 0x7f6058154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 -- 172.21.15.37:0/3405744707 >> 172.21.15.37:0/3405744707 conn(0x7f605806ae60 msgr2=0x7f60580b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 -- 172.21.15.37:0/3405744707 shutdown_connections 2024-04-25T08:59:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:25.664+0000 7f605cdc8640 1 -- 172.21.15.37:0/3405744707 wait complete. 2024-04-25T08:59:25.670 INFO:tasks.cephfs.filesystem:scrub status for tag:12f075cd-defa-4e99-8d54-d2f1d91bd495 - {'path': '/dir', 'tag': '12f075cd-defa-4e99-8d54-d2f1d91bd495', 'options': 'recursive'} 2024-04-25T08:59:26.157 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:26.160 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:28.383 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T08:59:28.392+0000 7ff670aed640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:59:55.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:55.672 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:59:55.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.815+0000 7fdad3ca4640 1 Processor -- start 2024-04-25T08:59:55.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.815+0000 7fdad3ca4640 1 -- start start 2024-04-25T08:59:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad3ca4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdacc0b2c60 0x7fdacc06ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad3ca4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0af520 0x7fdacc06b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad3ca4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdacc0ccfb0 con 0x7fdacc0cf720 2024-04-25T08:59:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad3ca4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdacc0cdd70 con 0x7fdacc0b2c60 2024-04-25T08:59:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad3ca4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc0afa20 con 0x7fdacc0af520 2024-04-25T08:59:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad2241640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0af520 0x7fdacc06b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad123f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdacc0b2c60 0x7fdacc06ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad2241640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0af520 0x7fdacc06b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:54124/0 (socket says 172.21.15.37:54124) 2024-04-25T08:59:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad2241640 1 -- 172.21.15.37:0/2250015270 learned_addr learned my addr 172.21.15.37:0/2250015270 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad123f640 1 -- 172.21.15.37:0/2250015270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0af520 msgr2=0x7fdacc06b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad123f640 1 --2- 172.21.15.37:0/2250015270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0af520 0x7fdacc06b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad123f640 1 -- 172.21.15.37:0/2250015270 >> v1:172.21.15.203:6789/0 conn(0x7fdacc0cf720 legacy=0x7fdacc0cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad123f640 1 -- 172.21.15.37:0/2250015270 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc0cd290 con 0x7fdacc0b2c60 2024-04-25T08:59:55.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad2241640 1 --2- 172.21.15.37:0/2250015270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0af520 0x7fdacc06b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:59:55.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.819+0000 7fdad0a3e640 1 -- 172.21.15.37:0/2250015270 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662894073 0 0) 0x7fdacc0ccfb0 con 0x7fdacc0cf720 2024-04-25T08:59:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad123f640 1 --2- 172.21.15.37:0/2250015270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdacc0b2c60 0x7fdacc06ac90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0097f0 tx=0x7fdabc01c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b51d4ef0d9e0eba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad0a3e640 1 -- 172.21.15.37:0/2250015270 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0ab600 con 0x7fdacc0b2c60 2024-04-25T08:59:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad0a3e640 1 -- 172.21.15.37:0/2250015270 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc0b0070 con 0x7fdacc0b2c60 2024-04-25T08:59:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad0a3e640 1 -- 172.21.15.37:0/2250015270 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0ab9c0 con 0x7fdacc0b2c60 2024-04-25T08:59:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 -- 172.21.15.37:0/2250015270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdacc0b2c60 msgr2=0x7fdacc06ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 --2- 172.21.15.37:0/2250015270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdacc0b2c60 0x7fdacc06ac90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdabc0097f0 tx=0x7fdabc01c3e0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 -- 172.21.15.37:0/2250015270 shutdown_connections 2024-04-25T08:59:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 --2- 172.21.15.37:0/2250015270 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0af520 0x7fdacc06b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 --2- 172.21.15.37:0/2250015270 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdacc0b2c60 0x7fdacc06ac90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 -- 172.21.15.37:0/2250015270 >> 172.21.15.37:0/2250015270 conn(0x7fdacc099a70 msgr2=0x7fdacc0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 -- 172.21.15.37:0/2250015270 shutdown_connections 2024-04-25T08:59:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 -- 172.21.15.37:0/2250015270 wait complete. 2024-04-25T08:59:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 Processor -- start 2024-04-25T08:59:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 -- start start 2024-04-25T08:59:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdacc0af520 0x7fdacc156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0b2c60 0x7fdacc158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc0cf720 0x7fdacc152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc1611e0 con 0x7fdacc0cf720 2024-04-25T08:59:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc06bb20 con 0x7fdacc0af520 2024-04-25T08:59:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad3ca4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc16ada0 con 0x7fdacc0b2c60 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad123f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0b2c60 0x7fdacc158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.823+0000 7fdad2241640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc0cf720 0x7fdacc152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad2241640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc0cf720 0x7fdacc152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36072/0 (socket says 172.21.15.37:36072) 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad123f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0b2c60 0x7fdacc158800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:54132/0 (socket says 172.21.15.37:54132) 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad2241640 1 -- 172.21.15.37:0/3266073597 learned_addr learned my addr 172.21.15.37:0/3266073597 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad1a40640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdacc0af520 0x7fdacc156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad2241640 1 -- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0b2c60 msgr2=0x7fdacc158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad2241640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0b2c60 0x7fdacc158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad2241640 1 -- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdacc0af520 msgr2=0x7fdacc156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad2241640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdacc0af520 0x7fdacc156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad2241640 1 -- 172.21.15.37:0/3266073597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc15b400 con 0x7fdacc0cf720 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad2241640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc0cf720 0x7fdacc152f10 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fdac400ef90 tx=0x7fdac400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:55.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdac27fc640 1 -- 172.21.15.37:0/3266073597 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac400cbe0 con 0x7fdacc0cf720 2024-04-25T08:59:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdac27fc640 1 -- 172.21.15.37:0/3266073597 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac400cd80 con 0x7fdacc0cf720 2024-04-25T08:59:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad3ca4640 1 -- 172.21.15.37:0/3266073597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdacc159790 con 0x7fdacc0cf720 2024-04-25T08:59:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdac27fc640 1 -- 172.21.15.37:0/3266073597 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac4003be0 con 0x7fdacc0cf720 2024-04-25T08:59:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad3ca4640 1 -- 172.21.15.37:0/3266073597 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdacc15b6c0 con 0x7fdacc0cf720 2024-04-25T08:59:55.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdac27fc640 1 -- 172.21.15.37:0/3266073597 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdac401b030 con 0x7fdacc0cf720 2024-04-25T08:59:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdac27fc640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdab0066f60 0x7fdab0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdac27fc640 1 -- 172.21.15.37:0/3266073597 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fdac40854d0 con 0x7fdacc0cf720 2024-04-25T08:59:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.827+0000 7fdad1a40640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdab0066f60 0x7fdab0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 Processor -- start 2024-04-25T08:59:55.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 -- start start 2024-04-25T08:59:55.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad1a40640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdab0066f60 0x7fdab0069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdac8000f70 tx=0x7fdac8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b860cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b85c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b85c30a0 0x5605b85c3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b840fff0 con 0x5605b860c9d0 2024-04-25T08:59:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b8398150 con 0x5605b860d2f0 2024-04-25T08:59:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605b840c090 con 0x5605b85c30a0 2024-04-25T08:59:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad1a40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b860cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad1a40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b860cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36088/0 (socket says 172.21.15.37:36088) 2024-04-25T08:59:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad1a40640 1 -- 172.21.15.37:0/658008958 learned_addr learned my addr 172.21.15.37:0/658008958 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad123f640 1 --2- 172.21.15.37:0/658008958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b85c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad2241640 1 --2- 172.21.15.37:0/658008958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b85c30a0 0x5605b85c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad1a40640 1 -- 172.21.15.37:0/658008958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b85c30a0 msgr2=0x5605b85c3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad1a40640 1 --2- 172.21.15.37:0/658008958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b85c30a0 0x5605b85c3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad1a40640 1 -- 172.21.15.37:0/658008958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 msgr2=0x5605b85c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad1a40640 1 --2- 172.21.15.37:0/658008958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b85c2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad1a40640 1 -- 172.21.15.37:0/658008958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605b85a60d0 con 0x5605b860c9d0 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad2241640 1 --2- 172.21.15.37:0/658008958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b85c30a0 0x5605b85c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad1a40640 1 --2- 172.21.15.37:0/658008958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b860cdb0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fdac800fb40 tx=0x7fdac8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c029e45efed12c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdaaf7fe640 1 -- 172.21.15.37:0/658008958 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8025070 con 0x5605b860c9d0 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdaaf7fe640 1 -- 172.21.15.37:0/658008958 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac801b050 con 0x5605b860c9d0 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdaaf7fe640 1 -- 172.21.15.37:0/658008958 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8014e40 con 0x5605b860c9d0 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 -- 172.21.15.37:0/658008958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 msgr2=0x5605b860cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 --2- 172.21.15.37:0/658008958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b860cdb0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fdac800fb40 tx=0x7fdac8012000 comp rx=0 tx=0).stop 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 -- 172.21.15.37:0/658008958 shutdown_connections 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 --2- 172.21.15.37:0/658008958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b85c30a0 0x5605b85c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 --2- 172.21.15.37:0/658008958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b85c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 --2- 172.21.15.37:0/658008958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b860cdb0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 -- 172.21.15.37:0/658008958 >> 172.21.15.37:0/658008958 conn(0x5605b85bf020 msgr2=0x5605b85c1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:55.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 -- 172.21.15.37:0/658008958 shutdown_connections 2024-04-25T08:59:55.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 -- 172.21.15.37:0/658008958 wait complete. 2024-04-25T08:59:55.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 -- start start 2024-04-25T08:59:55.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.831+0000 7fdad6e11000 1 Processor -- start 2024-04-25T08:59:55.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b85f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860c9d0 0x5605b85f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b85f2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b83d4250 con 0x5605b85c30a0 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b83ca690 con 0x5605b860d2f0 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605b8398a40 con 0x5605b860c9d0 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 10 client.0 _ll_register_callbacks cb 0x5605b85ade90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad1a40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b85f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad1a40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b85f17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36094/0 (socket says 172.21.15.37:36094) 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad1a40640 1 -- 172.21.15.37:0/2444552746 learned_addr learned my addr 172.21.15.37:0/2444552746 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad123f640 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860c9d0 0x5605b85f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad2241640 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b85f2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad1a40640 1 -- 172.21.15.37:0/2444552746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860c9d0 msgr2=0x5605b85f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad1a40640 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860c9d0 0x5605b85f1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad1a40640 1 -- 172.21.15.37:0/2444552746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 msgr2=0x5605b85f2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad1a40640 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b85f2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad1a40640 1 -- 172.21.15.37:0/2444552746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605b84bd1c0 con 0x5605b85c30a0 2024-04-25T08:59:55.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad1a40640 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b85f17f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fdac8023040 tx=0x7fdac8001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730f7f7530b57516 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdaacff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdaacff9640 1 -- 172.21.15.37:0/2444552746 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac802f060 con 0x5605b85c30a0 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdaacff9640 1 -- 172.21.15.37:0/2444552746 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fdac8020b00 con 0x5605b85c30a0 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 -- 172.21.15.37:0/2444552746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdac801b050 con 0x5605b85c30a0 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdaacff9640 1 -- 172.21.15.37:0/2444552746 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdac8035050 con 0x5605b85c30a0 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 10 client.8130 fetch_fsmap learned FSMap version 67 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 -- 172.21.15.37:0/2444552746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5605b84b9170 con 0x5605b85c30a0 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdaacff9640 1 -- 172.21.15.37:0/2444552746 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac801b050 con 0x5605b85c30a0 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdaacff9640 1 -- 172.21.15.37:0/2444552746 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdac8038020 con 0x5605b85c30a0 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 10 client.8130 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 10 client.8130 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 20 client.8130 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5605b861cc80 0x5605b861f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 4 client.8130 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:55.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 -- 172.21.15.37:0/2444552746 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605b83abc60 con 0x5605b861cc80 2024-04-25T08:59:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad123f640 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5605b861cc80 0x5605b861f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdaacff9640 10 client.8130 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad123f640 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5605b861cc80 0x5605b861f070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f7b9b0f3a3f692 server_cookie=23b4871b335286ba in_seq=0 out_seq=0 2024-04-25T08:59:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdaacff9640 1 -- 172.21.15.37:0/2444552746 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5605b83abc60 con 0x5605b861cc80 2024-04-25T08:59:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdaacff9640 10 client.8130 handle_command_reply: tid=0 2024-04-25T08:59:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 client.8130 shutdown 2024-04-25T08:59:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 -- 172.21.15.37:0/2444552746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 msgr2=0x5605b85f17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b85f17f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fdac8023040 tx=0x7fdac8001090 comp rx=0 tx=0).stop 2024-04-25T08:59:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 -- 172.21.15.37:0/2444552746 shutdown_connections 2024-04-25T08:59:55.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5605b861cc80 0x5605b861f070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b85f2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860c9d0 0x5605b85f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdaacff9640 0 client.8130 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 --2- 172.21.15.37:0/2444552746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b85f17f0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 -- 172.21.15.37:0/2444552746 >> 172.21.15.37:0/2444552746 conn(0x5605b85bf020 msgr2=0x5605b85c1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 -- 172.21.15.37:0/2444552746 shutdown_connections 2024-04-25T08:59:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 1 -- 172.21.15.37:0/2444552746 wait complete. 2024-04-25T08:59:55.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.835+0000 7fdad6e11000 20 client.8130 trim_cache size 0 max 16384 2024-04-25T08:59:55.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad6e11000 1 Processor -- start 2024-04-25T08:59:55.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad6e11000 1 -- start start 2024-04-25T08:59:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b8668ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860c9d0 0x5605b8669420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b8669960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad6e11000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc203960 con 0x5605b85c30a0 2024-04-25T08:59:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad6e11000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc15a510 con 0x5605b860d2f0 2024-04-25T08:59:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad6e11000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc15de50 con 0x5605b860c9d0 2024-04-25T08:59:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad1a40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b8668ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad1a40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b8668ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36102/0 (socket says 172.21.15.37:36102) 2024-04-25T08:59:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad1a40640 1 -- 172.21.15.37:0/2093465781 learned_addr learned my addr 172.21.15.37:0/2093465781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:55.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad123f640 1 --2- 172.21.15.37:0/2093465781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860c9d0 0x5605b8669420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad2241640 1 --2- 172.21.15.37:0/2093465781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b8669960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad1a40640 1 -- 172.21.15.37:0/2093465781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860c9d0 msgr2=0x5605b8669420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad1a40640 1 --2- 172.21.15.37:0/2093465781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860c9d0 0x5605b8669420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad1a40640 1 -- 172.21.15.37:0/2093465781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 msgr2=0x5605b8669960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad1a40640 1 --2- 172.21.15.37:0/2093465781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b8669960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad1a40640 1 -- 172.21.15.37:0/2093465781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605b8456de0 con 0x5605b85c30a0 2024-04-25T08:59:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.851+0000 7fdad2241640 1 --2- 172.21.15.37:0/2093465781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b8669960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad1a40640 1 --2- 172.21.15.37:0/2093465781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b8668ee0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdac8007ad0 tx=0x7fdac8028000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea5e59d00768721 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdaaeffd640 1 -- 172.21.15.37:0/2093465781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac803e070 con 0x5605b85c30a0 2024-04-25T08:59:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdaaeffd640 1 -- 172.21.15.37:0/2093465781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac801b040 con 0x5605b85c30a0 2024-04-25T08:59:55.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdaaeffd640 1 -- 172.21.15.37:0/2093465781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac8025040 con 0x5605b85c30a0 2024-04-25T08:59:55.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 -- 172.21.15.37:0/2093465781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 msgr2=0x5605b8668ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 --2- 172.21.15.37:0/2093465781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b8668ee0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdac8007ad0 tx=0x7fdac8028000 comp rx=0 tx=0).stop 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 -- 172.21.15.37:0/2093465781 shutdown_connections 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 --2- 172.21.15.37:0/2093465781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b860d2f0 0x5605b8669960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 --2- 172.21.15.37:0/2093465781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860c9d0 0x5605b8669420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 --2- 172.21.15.37:0/2093465781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b85c30a0 0x5605b8668ee0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 -- 172.21.15.37:0/2093465781 >> 172.21.15.37:0/2093465781 conn(0x5605b85bf020 msgr2=0x5605b860b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 -- 172.21.15.37:0/2093465781 shutdown_connections 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 -- 172.21.15.37:0/2093465781 wait complete. 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 -- start start 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 Processor -- start 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b85c30a0 0x5605b861f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b861f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860d2f0 0x5605b861fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605b83aae00 con 0x5605b860c9d0 2024-04-25T08:59:55.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdacc159f10 con 0x5605b85c30a0 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5605b840f300 con 0x5605b860d2f0 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 10 client.0 _ll_register_callbacks cb 0x5605b858e040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad123f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b861f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad123f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b861f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36104/0 (socket says 172.21.15.37:36104) 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad123f640 1 -- 172.21.15.37:0/3581024792 learned_addr learned my addr 172.21.15.37:0/3581024792 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad1a40640 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b85c30a0 0x5605b861f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad2241640 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860d2f0 0x5605b861fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad123f640 1 -- 172.21.15.37:0/3581024792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860d2f0 msgr2=0x5605b861fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad123f640 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860d2f0 0x5605b861fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad123f640 1 -- 172.21.15.37:0/3581024792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b85c30a0 msgr2=0x5605b861f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad123f640 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b85c30a0 0x5605b861f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad123f640 1 -- 172.21.15.37:0/3581024792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605b83b18e0 con 0x5605b860c9d0 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad1a40640 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b85c30a0 0x5605b861f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad123f640 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b861f8c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdabc01c420 tx=0x7fdabc01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600a3d99007cc3fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:55.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdaacff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdaacff9640 1 -- 172.21.15.37:0/3581024792 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0c0700 con 0x5605b860c9d0 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdaacff9640 1 -- 172.21.15.37:0/3581024792 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fdabc0b5500 con 0x5605b860c9d0 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 -- 172.21.15.37:0/3581024792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdac801b040 con 0x5605b860c9d0 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdaacff9640 1 -- 172.21.15.37:0/3581024792 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdabc0d0050 con 0x5605b860c9d0 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 10 client.8136 fetch_fsmap learned FSMap version 67 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdad6e11000 1 -- 172.21.15.37:0/3581024792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5605b84698f0 con 0x5605b860c9d0 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdaacff9640 1 -- 172.21.15.37:0/3581024792 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac801b040 con 0x5605b860c9d0 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.855+0000 7fdaacff9640 1 -- 172.21.15.37:0/3581024792 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdabc0d3020 con 0x5605b860c9d0 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 10 client.8136 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 10 client.8136 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 20 client.8136 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5605b8672f50 0x5605b8675340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 4 client.8136 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 -- 172.21.15.37:0/3581024792 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5605b841d530 con 0x5605b8672f50 2024-04-25T08:59:55.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad1a40640 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5605b8672f50 0x5605b8675340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdaacff9640 10 client.8136 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad1a40640 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5605b8672f50 0x5605b8675340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e70cd04c584ed91a server_cookie=da3664a8e9400ad in_seq=0 out_seq=0 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdaacff9640 1 -- 172.21.15.37:0/3581024792 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5605b841d530 con 0x5605b8672f50 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdaacff9640 10 client.8136 handle_command_reply: tid=0 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 client.8136 shutdown 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 -- 172.21.15.37:0/3581024792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 msgr2=0x5605b861f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b861f8c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdabc01c420 tx=0x7fdabc01ac20 comp rx=0 tx=0).stop 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 -- 172.21.15.37:0/3581024792 shutdown_connections 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5605b8672f50 0x5605b8675340 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5605b860d2f0 0x5605b861fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605b860c9d0 0x5605b861f8c0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdaacff9640 0 client.8136 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 --2- 172.21.15.37:0/3581024792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5605b85c30a0 0x5605b861f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 -- 172.21.15.37:0/3581024792 >> 172.21.15.37:0/3581024792 conn(0x5605b85bf020 msgr2=0x5605b860b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 -- 172.21.15.37:0/3581024792 shutdown_connections 2024-04-25T08:59:55.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 1 -- 172.21.15.37:0/3581024792 wait complete. 2024-04-25T08:59:55.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad6e11000 20 client.8136 trim_cache size 0 max 16384 2024-04-25T08:59:55.856 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:59:55.856 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-04-25T08:59:55.856 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-04-25T08:59:55.856 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:59:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 -- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdab0066f60 msgr2=0x7fdab0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdab0066f60 0x7fdab0069420 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdac8000f70 tx=0x7fdac8009040 comp rx=0 tx=0).stop 2024-04-25T08:59:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 -- 172.21.15.37:0/3266073597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc0cf720 msgr2=0x7fdacc152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc0cf720 0x7fdacc152f10 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fdac400ef90 tx=0x7fdac400c490 comp rx=0 tx=0).stop 2024-04-25T08:59:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 -- 172.21.15.37:0/3266073597 shutdown_connections 2024-04-25T08:59:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdab0066f60 0x7fdab0069420 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdacc0cf720 0x7fdacc152f10 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdacc0b2c60 0x7fdacc158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 --2- 172.21.15.37:0/3266073597 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdacc0af520 0x7fdacc156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 -- 172.21.15.37:0/3266073597 >> 172.21.15.37:0/3266073597 conn(0x7fdacc099a70 msgr2=0x7fdacc0cfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 -- 172.21.15.37:0/3266073597 shutdown_connections 2024-04-25T08:59:55.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.859+0000 7fdad3ca4640 1 -- 172.21.15.37:0/3266073597 wait complete. 2024-04-25T08:59:55.868 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:59:55.868 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:59:55.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd4452f640 1 Processor -- start 2024-04-25T08:59:55.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd4452f640 1 -- start start 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd4452f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efd3c0ce510 0x7efd3c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd4452f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efd3c144ca0 0x7efd3c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd4452f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd3c0cee30 con 0x7efd3c06ec60 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd4452f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c0cc2b0 con 0x7efd3c144ca0 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd4452f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd3c06e290 con 0x7efd3c0ce510 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd41aca640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efd3c144ca0 0x7efd3c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd41aca640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efd3c144ca0 0x7efd3c06e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36106/0 (socket says 172.21.15.37:36106) 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd41aca640 1 -- 172.21.15.37:0/3529997786 learned_addr learned my addr 172.21.15.37:0/3529997786 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd422cb640 1 --2- 172.21.15.37:0/3529997786 >> v2:172.21.15.203:3300/0 conn(0x7efd3c0ce510 0x7efd3c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd41aca640 1 -- 172.21.15.37:0/3529997786 >> v2:172.21.15.203:3300/0 conn(0x7efd3c0ce510 msgr2=0x7efd3c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd41aca640 1 --2- 172.21.15.37:0/3529997786 >> v2:172.21.15.203:3300/0 conn(0x7efd3c0ce510 0x7efd3c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd41aca640 1 -- 172.21.15.37:0/3529997786 >> v1:172.21.15.203:6789/0 conn(0x7efd3c06ec60 legacy=0x7efd3c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd41aca640 1 -- 172.21.15.37:0/3529997786 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c0ccc70 con 0x7efd3c144ca0 2024-04-25T08:59:55.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.987+0000 7efd422cb640 1 --2- 172.21.15.37:0/3529997786 >> v2:172.21.15.203:3300/0 conn(0x7efd3c0ce510 0x7efd3c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd41aca640 1 --2- 172.21.15.37:0/3529997786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd3c144ca0 0x7efd3c06e720 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7efd3c144c50 tx=0x7efd2c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ec57672f0f0cea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd412c9640 1 -- 172.21.15.37:0/3529997786 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0b5070 con 0x7efd3c144ca0 2024-04-25T08:59:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd412c9640 1 -- 172.21.15.37:0/3529997786 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c0a89d0 con 0x7efd3c144ca0 2024-04-25T08:59:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd412c9640 1 -- 172.21.15.37:0/3529997786 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0a8da0 con 0x7efd3c144ca0 2024-04-25T08:59:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 -- 172.21.15.37:0/3529997786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd3c144ca0 msgr2=0x7efd3c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 --2- 172.21.15.37:0/3529997786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd3c144ca0 0x7efd3c06e720 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7efd3c144c50 tx=0x7efd2c01ecd0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 -- 172.21.15.37:0/3529997786 shutdown_connections 2024-04-25T08:59:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 --2- 172.21.15.37:0/3529997786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd3c144ca0 0x7efd3c06e720 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 --2- 172.21.15.37:0/3529997786 >> v2:172.21.15.203:3300/0 conn(0x7efd3c0ce510 0x7efd3c0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 -- 172.21.15.37:0/3529997786 >> 172.21.15.37:0/3529997786 conn(0x7efd3c0b11a0 msgr2=0x7efd3c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 -- 172.21.15.37:0/3529997786 shutdown_connections 2024-04-25T08:59:55.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 -- 172.21.15.37:0/3529997786 wait complete. 2024-04-25T08:59:55.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 Processor -- start 2024-04-25T08:59:55.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 -- start start 2024-04-25T08:59:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd3c06ec60 0x7efd3c13c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd3c0ce510 0x7efd3c13ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c144ca0 0x7efd3c13f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c0cc900 con 0x7efd3c0ce510 2024-04-25T08:59:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c06df20 con 0x7efd3c144ca0 2024-04-25T08:59:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3c142d90 con 0x7efd3c06ec60 2024-04-25T08:59:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd41aca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd3c0ce510 0x7efd3c13ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd41aca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd3c0ce510 0x7efd3c13ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36122/0 (socket says 172.21.15.37:36122) 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd41aca640 1 -- 172.21.15.37:0/2896093877 learned_addr learned my addr 172.21.15.37:0/2896093877 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd422cb640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd3c06ec60 0x7efd3c13c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd42acc640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c144ca0 0x7efd3c13f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd41aca640 1 -- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd3c06ec60 msgr2=0x7efd3c13c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd41aca640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd3c06ec60 0x7efd3c13c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd41aca640 1 -- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c144ca0 msgr2=0x7efd3c13f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd41aca640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c144ca0 0x7efd3c13f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd41aca640 1 -- 172.21.15.37:0/2896093877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c13c8f0 con 0x7efd3c0ce510 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd422cb640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd3c06ec60 0x7efd3c13c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd42acc640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c144ca0 0x7efd3c13f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd41aca640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd3c0ce510 0x7efd3c13ecd0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0b3040 tx=0x7efd2c01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd32ffd640 1 -- 172.21.15.37:0/2896093877 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0b5040 con 0x7efd3c0ce510 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd32ffd640 1 -- 172.21.15.37:0/2896093877 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd2c0bf070 con 0x7efd3c0ce510 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 -- 172.21.15.37:0/2896093877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd3c152190 con 0x7efd3c0ce510 2024-04-25T08:59:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd32ffd640 1 -- 172.21.15.37:0/2896093877 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c0b0a80 con 0x7efd3c0ce510 2024-04-25T08:59:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.991+0000 7efd4452f640 1 -- 172.21.15.37:0/2896093877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd3c152750 con 0x7efd3c0ce510 2024-04-25T08:59:55.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.995+0000 7efd4452f640 1 -- 172.21.15.37:0/2896093877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd3c04efa0 con 0x7efd3c0ce510 2024-04-25T08:59:55.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.995+0000 7efd32ffd640 1 -- 172.21.15.37:0/2896093877 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efd2c0b0c20 con 0x7efd3c0ce510 2024-04-25T08:59:55.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.995+0000 7efd32ffd640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efd20066f60 0x7efd20069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:55.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.995+0000 7efd32ffd640 1 -- 172.21.15.37:0/2896093877 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7efd2c123390 con 0x7efd3c0ce510 2024-04-25T08:59:55.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.995+0000 7efd422cb640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efd20066f60 0x7efd20069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:55.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.995+0000 7efd422cb640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efd20066f60 0x7efd20069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7efd34008930 tx=0x7efd34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:55.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:55.999+0000 7efd32ffd640 1 -- 172.21.15.37:0/2896093877 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efd2c018830 con 0x7efd3c0ce510 2024-04-25T08:59:56.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.191+0000 7efd4452f640 1 -- 172.21.15.37:0/2896093877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd3c0ccf70 con 0x7efd3c0ce510 2024-04-25T08:59:56.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.191+0000 7efd32ffd640 1 -- 172.21.15.37:0/2896093877 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4947 (secure 0 0 0) 0x7efd2c0f4130 con 0x7efd3c0ce510 2024-04-25T08:59:56.185 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:59:56.185 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T08:59:28.123805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/1462327322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1462327322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1462327322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T08:59:56.186 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 67 2024-04-25T08:59:56.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 -- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efd20066f60 msgr2=0x7efd20069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efd20066f60 0x7efd20069420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7efd34008930 tx=0x7efd34008040 comp rx=0 tx=0).stop 2024-04-25T08:59:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 -- 172.21.15.37:0/2896093877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd3c0ce510 msgr2=0x7efd3c13ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd3c0ce510 0x7efd3c13ecd0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7efd2c0b3040 tx=0x7efd2c01ba70 comp rx=0 tx=0).stop 2024-04-25T08:59:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 -- 172.21.15.37:0/2896093877 shutdown_connections 2024-04-25T08:59:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efd20066f60 0x7efd20069420 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efd3c144ca0 0x7efd3c13f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd3c0ce510 0x7efd3c13ecd0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 --2- 172.21.15.37:0/2896093877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efd3c06ec60 0x7efd3c13c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 -- 172.21.15.37:0/2896093877 >> 172.21.15.37:0/2896093877 conn(0x7efd3c0b11a0 msgr2=0x7efd3c0b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 -- 172.21.15.37:0/2896093877 shutdown_connections 2024-04-25T08:59:56.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.195+0000 7efd4452f640 1 -- 172.21.15.37:0/2896093877 wait complete. 2024-04-25T08:59:56.203 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T08:59:56.271 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:56.274 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:56.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1adb91d640 1 Processor -- start 2024-04-25T08:59:56.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1adb91d640 1 -- start start 2024-04-25T08:59:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1adb91d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40cf590 0x7f1ad415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1adb91d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1ad40cf970 0x7f1ad40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1adb91d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ad40ca4c0 con 0x7f1ad40c9ef0 2024-04-25T08:59:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1adb91d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad40ca780 con 0x7f1ad40cf970 2024-04-25T08:59:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1adb91d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40cd440 con 0x7f1ad40cf590 2024-04-25T08:59:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad9eba640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1ad40c9ef0 0x7f1ad40b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42288/0 (socket says 172.21.15.37:42288) 2024-04-25T08:59:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad9eba640 1 -- 172.21.15.37:0/2733626116 learned_addr learned my addr 172.21.15.37:0/2733626116 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/2733626116 >> v2:172.21.15.203:3300/0 conn(0x7f1ad40cf970 0x7f1ad40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40cf590 0x7f1ad415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827286727 0 0) 0x7f1ad40ca4c0 con 0x7f1ad40c9ef0 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ac0003530 con 0x7f1ad40c9ef0 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2714125865 0 0) 0x7f1ac0003530 con 0x7f1ad40c9ef0 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40cf590 msgr2=0x7f1ad415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40cf590 0x7f1ad415ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 >> v2:172.21.15.203:3300/0 conn(0x7f1ad40cf970 msgr2=0x7f1ad40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 --2- 172.21.15.37:0/2733626116 >> v2:172.21.15.203:3300/0 conn(0x7f1ad40cf970 0x7f1ad40c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad40cd690 con 0x7f1ad40c9ef0 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1acc001a00 con 0x7f1ad40c9ef0 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 >> v1:172.21.15.37:6789/0 conn(0x7f1ad40c9ef0 legacy=0x7f1ad40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ac0003a30 0x7f1ac0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac0004810 0x7f1ac0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40cf590 0x7f1ac00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40cd440 con 0x7f1ac0003a30 2024-04-25T08:59:56.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac000b9d0 con 0x7f1ac0004810 2024-04-25T08:59:56.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac000bb50 con 0x7f1ad40cf590 2024-04-25T08:59:56.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ac0003a30 0x7f1ac0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac0004810 0x7f1ac0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad9eba640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40cf590 0x7f1ac00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad96b9640 1 -- 172.21.15.37:0/2733626116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40cf590 msgr2=0x7f1ac00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40cf590 0x7f1ac00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad96b9640 1 -- 172.21.15.37:0/2733626116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac0004810 msgr2=0x7f1ac0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ac0004810 0x7f1ac0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.339+0000 7f1ad96b9640 1 -- 172.21.15.37:0/2733626116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac0003530 con 0x7f1ac0003a30 2024-04-25T08:59:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ac0003a30 0x7f1ac0003e50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009af0 tx=0x7f1ad001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf218ea40bea669 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad0009690 con 0x7f1ac0003a30 2024-04-25T08:59:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad00a6440 con 0x7f1ac0003a30 2024-04-25T08:59:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1acbfff640 1 -- 172.21.15.37:0/2733626116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad00a6810 con 0x7f1ac0003a30 2024-04-25T08:59:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1adb91d640 1 -- 172.21.15.37:0/2733626116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ac0003a30 msgr2=0x7f1ac0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1adb91d640 1 --2- 172.21.15.37:0/2733626116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ac0003a30 0x7f1ac0003e50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009af0 tx=0x7f1ad001ca30 comp rx=0 tx=0).stop 2024-04-25T08:59:56.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1ad9eba640 1 -- 172.21.15.37:0/2733626116 reap_dead start 2024-04-25T08:59:56.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1adb91d640 1 -- 172.21.15.37:0/2733626116 shutdown_connections 2024-04-25T08:59:56.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1adb91d640 1 -- 172.21.15.37:0/2733626116 >> 172.21.15.37:0/2733626116 conn(0x7f1ad406ae60 msgr2=0x7f1ad406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1adb91d640 1 -- 172.21.15.37:0/2733626116 shutdown_connections 2024-04-25T08:59:56.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1adb91d640 1 -- 172.21.15.37:0/2733626116 wait complete. 2024-04-25T08:59:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1adb91d640 1 Processor -- start 2024-04-25T08:59:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.343+0000 7f1adb91d640 1 -- start start 2024-04-25T08:59:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1adb91d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40c3eb0 0x7f1ad40721e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1adb91d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ad40c4290 0x7f1ad40c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1adb91d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad40c7380 0x7f1ad41fa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1adb91d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40cbde0 con 0x7f1ad40c4290 2024-04-25T08:59:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1adb91d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad415f0c0 con 0x7f1ad40c7380 2024-04-25T08:59:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1adb91d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad40cbf60 con 0x7f1ad40c3eb0 2024-04-25T08:59:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad8eb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ad40c4290 0x7f1ad40c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad8eb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ad40c4290 0x7f1ad40c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36130/0 (socket says 172.21.15.37:36130) 2024-04-25T08:59:56.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/1877161222 learned_addr learned my addr 172.21.15.37:0/1877161222 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40c3eb0 0x7f1ad40721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad9eba640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad40c7380 0x7f1ad41fa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40c3eb0 msgr2=0x7f1ad40721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40c3eb0 0x7f1ad40721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad40c7380 msgr2=0x7f1ad41fa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad40c7380 0x7f1ad41fa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/1877161222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad40cec90 con 0x7f1ad40c4290 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad9eba640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad40c7380 0x7f1ad41fa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ad40c4290 0x7f1ad40c6e40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4011690 tx=0x7f1ac4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ac9ffb640 1 -- 172.21.15.37:0/1877161222 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac400f9c0 con 0x7f1ad40c4290 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ac9ffb640 1 -- 172.21.15.37:0/1877161222 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac400fb60 con 0x7f1ad40c4290 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1adb91d640 1 -- 172.21.15.37:0/1877161222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad40cef00 con 0x7f1ad40c4290 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ac9ffb640 1 -- 172.21.15.37:0/1877161222 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4015460 con 0x7f1ad40c4290 2024-04-25T08:59:56.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1adb91d640 1 -- 172.21.15.37:0/1877161222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad41586b0 con 0x7f1ad40c4290 2024-04-25T08:59:56.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ac9ffb640 1 -- 172.21.15.37:0/1877161222 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1ac4015600 con 0x7f1ad40c4290 2024-04-25T08:59:56.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ac9ffb640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ab8066f30 0x7f1ab80693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ac9ffb640 1 -- 172.21.15.37:0/1877161222 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1ac4085b70 con 0x7f1ad40c4290 2024-04-25T08:59:56.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ab8066f30 0x7f1ab80693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.347+0000 7f1adea8a000 1 Processor -- start 2024-04-25T08:59:56.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- start start 2024-04-25T08:59:56.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ab8066f30 0x7f1ab80693f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00ad000 tx=0x7f1ad00b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd472e0 0x5558ecd476c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558eccfd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558eccfdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ecbe4ba0 con 0x5558ecd47c00 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ecb4a090 con 0x5558eccfd8e0 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558ecad2160 con 0x5558ecd472e0 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad8eb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558eccfd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad8eb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558eccfd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36142/0 (socket says 172.21.15.37:36142) 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/415263170 learned_addr learned my addr 172.21.15.37:0/415263170 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/415263170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd472e0 0x5558ecd476c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/415263170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd472e0 msgr2=0x5558ecd476c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/415263170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd472e0 0x5558ecd476c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/415263170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 msgr2=0x5558eccfdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/415263170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558eccfdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/415263170 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558ecbf3c20 con 0x5558ecd47c00 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/415263170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd472e0 0x5558ecd476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/415263170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558eccfd3a0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1ac405e4a0 tx=0x7f1ac4055d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff66909c4fa9bada server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1abeffd640 1 -- 172.21.15.37:0/415263170 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4082070 con 0x5558ecd47c00 2024-04-25T08:59:56.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1abeffd640 1 -- 172.21.15.37:0/415263170 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac405e9d0 con 0x5558ecd47c00 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1abeffd640 1 -- 172.21.15.37:0/415263170 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac40545c0 con 0x5558ecd47c00 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- 172.21.15.37:0/415263170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 msgr2=0x5558eccfd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 --2- 172.21.15.37:0/415263170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558eccfd3a0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f1ac405e4a0 tx=0x7f1ac4055d20 comp rx=0 tx=0).stop 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- 172.21.15.37:0/415263170 shutdown_connections 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 --2- 172.21.15.37:0/415263170 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558eccfdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 --2- 172.21.15.37:0/415263170 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558eccfd3a0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 --2- 172.21.15.37:0/415263170 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd472e0 0x5558ecd476c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- 172.21.15.37:0/415263170 >> 172.21.15.37:0/415263170 conn(0x5558eccf9890 msgr2=0x5558eccfbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- 172.21.15.37:0/415263170 shutdown_connections 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- 172.21.15.37:0/415263170 wait complete. 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- start start 2024-04-25T08:59:56.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 Processor -- start 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558eccfd8e0 0x5558ecd08480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558ecd472e0 0x5558ecd089c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd08f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ecb46130 con 0x5558ecd47c00 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ecb0e130 con 0x5558ecd472e0 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558ecc08300 con 0x5558eccfd8e0 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1adea8a000 10 client.0 _ll_register_callbacks cb 0x5558ecce8bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad9eba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd08f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad9eba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd08f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36148/0 (socket says 172.21.15.37:36148) 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad9eba640 1 -- 172.21.15.37:0/2048583224 learned_addr learned my addr 172.21.15.37:0/2048583224 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558eccfd8e0 0x5558ecd08480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.351+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558ecd472e0 0x5558ecd089c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ad9eba640 1 -- 172.21.15.37:0/2048583224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558eccfd8e0 msgr2=0x5558ecd08480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ad9eba640 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558eccfd8e0 0x5558ecd08480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ad9eba640 1 -- 172.21.15.37:0/2048583224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558ecd472e0 msgr2=0x5558ecd089c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ad9eba640 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558ecd472e0 0x5558ecd089c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ad9eba640 1 -- 172.21.15.37:0/2048583224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558ecb27060 con 0x5558ecd47c00 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ad9eba640 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd08f00 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1acc002240 tx=0x7f1acc00f540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba6b5f0cd4d55de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ab3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ab3fff640 1 -- 172.21.15.37:0/2048583224 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc010040 con 0x5558ecd47c00 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 -- 172.21.15.37:0/2048583224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ac405e9d0 con 0x5558ecd47c00 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ab3fff640 1 -- 172.21.15.37:0/2048583224 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1acc00fb20 con 0x5558ecd47c00 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ab3fff640 1 -- 172.21.15.37:0/2048583224 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1acc021050 con 0x5558ecd47c00 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 10 client.8154 fetch_fsmap learned FSMap version 67 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 -- 172.21.15.37:0/2048583224 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5558ecaeb7c0 con 0x5558ecd47c00 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ab3fff640 1 -- 172.21.15.37:0/2048583224 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac405e9d0 con 0x5558ecd47c00 2024-04-25T08:59:56.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ab3fff640 1 -- 172.21.15.37:0/2048583224 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1acc024020 con 0x5558ecd47c00 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 10 client.8154 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 10 client.8154 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 20 client.8154 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5558ecd09770 0x5558ecd58850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 4 client.8154 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 -- 172.21.15.37:0/2048583224 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558ecae5cd0 con 0x5558ecd09770 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5558ecd09770 0x5558ecd58850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5558ecd09770 0x5558ecd58850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c1f2c3254a15f11 server_cookie=6227384f712860c2 in_seq=0 out_seq=0 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ab3fff640 10 client.8154 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ab3fff640 1 -- 172.21.15.37:0/2048583224 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5558ecae5cd0 con 0x5558ecd09770 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ab3fff640 10 client.8154 handle_command_reply: tid=0 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 client.8154 shutdown 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 -- 172.21.15.37:0/2048583224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 msgr2=0x5558ecd08f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd08f00 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f1acc002240 tx=0x7f1acc00f540 comp rx=0 tx=0).stop 2024-04-25T08:59:56.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 -- 172.21.15.37:0/2048583224 shutdown_connections 2024-04-25T08:59:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5558ecd09770 0x5558ecd58850 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd08f00 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558ecd472e0 0x5558ecd089c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1ab3fff640 0 client.8154 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 --2- 172.21.15.37:0/2048583224 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558eccfd8e0 0x5558ecd08480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 -- 172.21.15.37:0/2048583224 >> 172.21.15.37:0/2048583224 conn(0x5558eccf9890 msgr2=0x5558eccfc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 -- 172.21.15.37:0/2048583224 shutdown_connections 2024-04-25T08:59:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 1 -- 172.21.15.37:0/2048583224 wait complete. 2024-04-25T08:59:56.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.355+0000 7f1adea8a000 20 client.8154 trim_cache size 0 max 16384 2024-04-25T08:59:56.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1adea8a000 1 Processor -- start 2024-04-25T08:59:56.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1adea8a000 1 -- start start 2024-04-25T08:59:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558ecd46860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd472e0 0x5558ecd46da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd47c00 0x5558eccfcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1adea8a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4203550 con 0x5558ecd472e0 2024-04-25T08:59:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1adea8a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4202a20 con 0x5558eccfd8e0 2024-04-25T08:59:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1adea8a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4202f50 con 0x5558ecd47c00 2024-04-25T08:59:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1ad8eb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd472e0 0x5558ecd46da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1ad8eb8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd472e0 0x5558ecd46da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36162/0 (socket says 172.21.15.37:36162) 2024-04-25T08:59:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/1079431117 learned_addr learned my addr 172.21.15.37:0/1079431117 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/1079431117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558ecd46860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/1079431117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd47c00 msgr2=0x5558eccfcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/1079431117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd47c00 0x5558eccfcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/1079431117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 msgr2=0x5558ecd46860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/1079431117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558ecd46860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1ad8eb8640 1 -- 172.21.15.37:0/1079431117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558ecb05fc0 con 0x5558ecd472e0 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/1079431117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd472e0 0x5558ecd46da0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4085e90 tx=0x7f1ac4058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33030f8acfd756ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1abe7fc640 1 -- 172.21.15.37:0/1079431117 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4071070 con 0x5558ecd472e0 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1abe7fc640 1 -- 172.21.15.37:0/1079431117 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac405a170 con 0x5558ecd472e0 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.367+0000 7f1abe7fc640 1 -- 172.21.15.37:0/1079431117 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4082040 con 0x5558ecd472e0 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/1079431117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd472e0 msgr2=0x5558ecd46da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- 172.21.15.37:0/1079431117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd472e0 0x5558ecd46da0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4085e90 tx=0x7f1ac4058000 comp rx=0 tx=0).stop 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/1079431117 shutdown_connections 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- 172.21.15.37:0/1079431117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd47c00 0x5558eccfcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- 172.21.15.37:0/1079431117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd472e0 0x5558ecd46da0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- 172.21.15.37:0/1079431117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558ecd46860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/1079431117 >> 172.21.15.37:0/1079431117 conn(0x5558eccf9890 msgr2=0x5558ecd45d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/1079431117 shutdown_connections 2024-04-25T08:59:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/1079431117 wait complete. 2024-04-25T08:59:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- start start 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 Processor -- start 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558ecda6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd472e0 0x5558ecda7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd622d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ecae4e70 con 0x5558ecd47c00 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558ecad2a50 con 0x5558eccfd8e0 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5558ecb493a0 con 0x5558ecd472e0 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 10 client.0 _ll_register_callbacks cb 0x5558ecaf6240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad9eba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad9eba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd622d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36172/0 (socket says 172.21.15.37:36172) 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad9eba640 1 -- 172.21.15.37:0/4261076979 learned_addr learned my addr 172.21.15.37:0/4261076979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558ecda6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad8eb8640 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd472e0 0x5558ecda7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad9eba640 1 -- 172.21.15.37:0/4261076979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd472e0 msgr2=0x5558ecda7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad9eba640 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd472e0 0x5558ecda7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad9eba640 1 -- 172.21.15.37:0/4261076979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 msgr2=0x5558ecda6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad9eba640 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558ecda6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad9eba640 1 -- 172.21.15.37:0/4261076979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5558ecba39e0 con 0x5558ecd47c00 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558ecda6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad9eba640 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd622d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f1acc01ca00 tx=0x7f1acc001370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750029a9e75fc966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ab3fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ab3fff640 1 -- 172.21.15.37:0/4261076979 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1acc02b070 con 0x5558ecd47c00 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ab3fff640 1 -- 172.21.15.37:0/4261076979 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1acc001950 con 0x5558ecd47c00 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/4261076979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ac405a170 con 0x5558ecd47c00 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ab3fff640 1 -- 172.21.15.37:0/4261076979 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1acc035050 con 0x5558ecd47c00 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 10 client.8160 fetch_fsmap learned FSMap version 67 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/4261076979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5558ecae9630 con 0x5558ecd47c00 2024-04-25T08:59:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ab3fff640 1 -- 172.21.15.37:0/4261076979 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac405a170 con 0x5558ecd47c00 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ab3fff640 1 -- 172.21.15.37:0/4261076979 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1acc038020 con 0x5558ecd47c00 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 10 client.8160 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 10 client.8160 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 20 client.8160 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5558ecdadab0 0x5558ecdafea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 4 client.8160 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/4261076979 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5558ecb575d0 con 0x5558ecdadab0 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5558ecdadab0 0x5558ecdafea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ad96b9640 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5558ecdadab0 0x5558ecdafea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be5d3889bae3c659 server_cookie=298241efa7654b9b in_seq=0 out_seq=0 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ab3fff640 10 client.8160 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ab3fff640 1 -- 172.21.15.37:0/4261076979 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+419 (crc 0 0 0) 0x5558ecb575d0 con 0x5558ecdadab0 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ab3fff640 10 client.8160 handle_command_reply: tid=0 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 client.8160 shutdown 2024-04-25T08:59:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/4261076979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 msgr2=0x5558ecd622d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd622d0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f1acc01ca00 tx=0x7f1acc001370 comp rx=0 tx=0).stop 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/4261076979 shutdown_connections 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5558ecdadab0 0x5558ecdafea0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558ecd47c00 0x5558ecd622d0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5558ecd472e0 0x5558ecda7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 --2- 172.21.15.37:0/4261076979 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5558eccfd8e0 0x5558ecda6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1ab3fff640 0 client.8160 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/4261076979 >> 172.21.15.37:0/4261076979 conn(0x5558eccf9890 msgr2=0x5558ecd465c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.371+0000 7f1adea8a000 1 -- 172.21.15.37:0/4261076979 shutdown_connections 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adea8a000 1 -- 172.21.15.37:0/4261076979 wait complete. 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adea8a000 20 client.8160 trim_cache size 0 max 16384 2024-04-25T08:59:56.371 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "backtrace", 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 242230831, 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627776, 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/dir" 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "backtrace", 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 2722350428, 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627777, 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/testdir/hardlink" 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T08:59:56.372 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "dir_frag", 2024-04-25T08:59:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 2757202718, 2024-04-25T08:59:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627776, 2024-04-25T08:59:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "frag": "*", 2024-04-25T08:59:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/dir" 2024-04-25T08:59:56.373 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:59:56.373 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T08:59:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 -- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ab8066f30 msgr2=0x7f1ab80693f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ab8066f30 0x7f1ab80693f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f1ad00ad000 tx=0x7f1ad00b2040 comp rx=0 tx=0).stop 2024-04-25T08:59:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 -- 172.21.15.37:0/1877161222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ad40c4290 msgr2=0x7f1ad40c6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ad40c4290 0x7f1ad40c6e40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4011690 tx=0x7f1ac4011b50 comp rx=0 tx=0).stop 2024-04-25T08:59:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 -- 172.21.15.37:0/1877161222 shutdown_connections 2024-04-25T08:59:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ab8066f30 0x7f1ab80693f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ad40c7380 0x7f1ad41fa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ad40c4290 0x7f1ad40c6e40 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 --2- 172.21.15.37:0/1877161222 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ad40c3eb0 0x7f1ad40721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 -- 172.21.15.37:0/1877161222 >> 172.21.15.37:0/1877161222 conn(0x7f1ad406ae60 msgr2=0x7f1ad40acc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 -- 172.21.15.37:0/1877161222 shutdown_connections 2024-04-25T08:59:56.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.375+0000 7f1adb91d640 1 -- 172.21.15.37:0/1877161222 wait complete. 2024-04-25T08:59:56.382 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:59:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.547+0000 7f72fcfed640 1 Processor -- start 2024-04-25T08:59:56.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.547+0000 7f72fcfed640 1 -- start start 2024-04-25T08:59:56.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.547+0000 7f72fcfed640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f72f80ceca0 0x7f72f80cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.547+0000 7f72fcfed640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f80cd3c0 con 0x7f72f8161e80 2024-04-25T08:59:56.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.547+0000 7f72fcfed640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72f806ecf0 con 0x7f72f80c2a30 2024-04-25T08:59:56.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.547+0000 7f72fcfed640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72f80b2c10 con 0x7f72f80ceca0 2024-04-25T08:59:56.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.547+0000 7f72f659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f72f8161e80 0x7f72f80c2320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42292/0 (socket says 172.21.15.37:42292) 2024-04-25T08:59:56.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.547+0000 7f72f659c640 1 -- 172.21.15.37:0/3257099111 learned_addr learned my addr 172.21.15.37:0/3257099111 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:56.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.547+0000 7f72f6d9d640 1 --2- 172.21.15.37:0/3257099111 >> v2:172.21.15.203:3300/0 conn(0x7f72f80ceca0 0x7f72f80cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051156685 0 0) 0x7f72f80cd3c0 con 0x7f72f8161e80 2024-04-25T08:59:56.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e4003530 con 0x7f72f8161e80 2024-04-25T08:59:56.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2001877828 0 0) 0x7f72e4003530 con 0x7f72f8161e80 2024-04-25T08:59:56.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 >> v2:172.21.15.203:3300/0 conn(0x7f72f80ceca0 msgr2=0x7f72f80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 --2- 172.21.15.37:0/3257099111 >> v2:172.21.15.203:3300/0 conn(0x7f72f80ceca0 0x7f72f80cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 >> v1:172.21.15.203:6789/0 conn(0x7f72f80c2a30 legacy=0x7f72f80cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f80b2e60 con 0x7f72f8161e80 2024-04-25T08:59:56.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f72e8001680 con 0x7f72f8161e80 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 >> v1:172.21.15.37:6789/0 conn(0x7f72f8161e80 legacy=0x7f72f80c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 --2- 172.21.15.37:0/3257099111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e4003a30 0x7f72e4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 --2- 172.21.15.37:0/3257099111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 0x7f72e4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 --2- 172.21.15.37:0/3257099111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72e40052f0 0x7f72e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f80b2c10 con 0x7f72e4004810 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e400c240 con 0x7f72e4003a30 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e400c440 con 0x7f72e40052f0 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f5d9b640 1 --2- 172.21.15.37:0/3257099111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 0x7f72e4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f659c640 1 --2- 172.21.15.37:0/3257099111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e4003a30 0x7f72e4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f6d9d640 1 --2- 172.21.15.37:0/3257099111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72e40052f0 0x7f72e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f5d9b640 1 -- 172.21.15.37:0/3257099111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72e40052f0 msgr2=0x7f72e400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f5d9b640 1 --2- 172.21.15.37:0/3257099111 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72e40052f0 0x7f72e400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f5d9b640 1 -- 172.21.15.37:0/3257099111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e4003a30 msgr2=0x7f72e4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f5d9b640 1 --2- 172.21.15.37:0/3257099111 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72e4003a30 0x7f72e4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f5d9b640 1 -- 172.21.15.37:0/3257099111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4003530 con 0x7f72e4004810 2024-04-25T08:59:56.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f6d9d640 1 -- 172.21.15.37:0/3257099111 reap_dead start 2024-04-25T08:59:56.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f5d9b640 1 --2- 172.21.15.37:0/3257099111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 0x7f72e4004c90 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f72e0001e40 tx=0x7f72e001c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdc97f46f897c30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0001a90 con 0x7f72e4004810 2024-04-25T08:59:56.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e0003b80 con 0x7f72e4004810 2024-04-25T08:59:56.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72f559a640 1 -- 172.21.15.37:0/3257099111 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0005080 con 0x7f72e4004810 2024-04-25T08:59:56.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72fcfed640 1 -- 172.21.15.37:0/3257099111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 msgr2=0x7f72e4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72fcfed640 1 --2- 172.21.15.37:0/3257099111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 0x7f72e4004c90 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f72e0001e40 tx=0x7f72e001c720 comp rx=0 tx=0).stop 2024-04-25T08:59:56.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72fcfed640 1 -- 172.21.15.37:0/3257099111 shutdown_connections 2024-04-25T08:59:56.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72fcfed640 1 --2- 172.21.15.37:0/3257099111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 0x7f72e4004c90 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72fcfed640 1 -- 172.21.15.37:0/3257099111 >> 172.21.15.37:0/3257099111 conn(0x7f72f806ae60 msgr2=0x7f72f806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72fcfed640 1 -- 172.21.15.37:0/3257099111 shutdown_connections 2024-04-25T08:59:56.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.551+0000 7f72fcfed640 1 -- 172.21.15.37:0/3257099111 wait complete. 2024-04-25T08:59:56.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72fcfed640 1 Processor -- start 2024-04-25T08:59:56.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72fcfed640 1 -- start start 2024-04-25T08:59:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72fcfed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 0x7f72f81625a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72fcfed640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72f8162ae0 0x7f72f80ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72fcfed640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72f80aca20 0x7f72f8071ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72fcfed640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f80c28b0 con 0x7f72e4004810 2024-04-25T08:59:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72fcfed640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f80c2730 con 0x7f72f80aca20 2024-04-25T08:59:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72fcfed640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f80ac1e0 con 0x7f72f8162ae0 2024-04-25T08:59:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72f659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 0x7f72f81625a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72f659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 0x7f72f81625a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36192/0 (socket says 172.21.15.37:36192) 2024-04-25T08:59:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72f659c640 1 -- 172.21.15.37:0/3505247519 learned_addr learned my addr 172.21.15.37:0/3505247519 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72f6d9d640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72f80aca20 0x7f72f8071ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72f5d9b640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72f8162ae0 0x7f72f80ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72f659c640 1 -- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72f8162ae0 msgr2=0x7f72f80ac4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72f659c640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72f8162ae0 0x7f72f80ac4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72f659c640 1 -- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72f80aca20 msgr2=0x7f72f8071ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72f659c640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72f80aca20 0x7f72f8071ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72f659c640 1 -- 172.21.15.37:0/3505247519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f80acea0 con 0x7f72e4004810 2024-04-25T08:59:56.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72f659c640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 0x7f72f81625a0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f72e800e810 tx=0x7f72e800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72eeffd640 1 -- 172.21.15.37:0/3505247519 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e800c6c0 con 0x7f72e4004810 2024-04-25T08:59:56.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72eeffd640 1 -- 172.21.15.37:0/3505247519 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e8018070 con 0x7f72e4004810 2024-04-25T08:59:56.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72fcfed640 1 -- 172.21.15.37:0/3505247519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f80c3500 con 0x7f72e4004810 2024-04-25T08:59:56.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72eeffd640 1 -- 172.21.15.37:0/3505247519 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e800cae0 con 0x7f72e4004810 2024-04-25T08:59:56.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72fcfed640 1 -- 172.21.15.37:0/3505247519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f80c3ac0 con 0x7f72e4004810 2024-04-25T08:59:56.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.555+0000 7f72fcfed640 1 -- 172.21.15.37:0/3505247519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f804efa0 con 0x7f72e4004810 2024-04-25T08:59:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.559+0000 7f72eeffd640 1 -- 172.21.15.37:0/3505247519 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f72e800cc80 con 0x7f72e4004810 2024-04-25T08:59:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.559+0000 7f72eeffd640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72d8066f90 0x7f72d8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.559+0000 7f72eeffd640 1 -- 172.21.15.37:0/3505247519 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f72e8084b90 con 0x7f72e4004810 2024-04-25T08:59:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.559+0000 7f72f5d9b640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72d8066f90 0x7f72d8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.559+0000 7f72f5d9b640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72d8066f90 0x7f72d8069450 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f72e0005000 tx=0x7f72e0001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.563+0000 7f72eeffd640 1 -- 172.21.15.37:0/3505247519 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f72e8055300 con 0x7f72e4004810 2024-04-25T08:59:56.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.751+0000 7f72fcfed640 1 -- 172.21.15.37:0/3505247519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f72f80ccb10 con 0x7f72e4004810 2024-04-25T08:59:56.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72eeffd640 1 -- 172.21.15.37:0/3505247519 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f72e80593a0 con 0x7f72e4004810 2024-04-25T08:59:56.746 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T08:59:56.746 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-04-25T08:59:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 -- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72d8066f90 msgr2=0x7f72d8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72d8066f90 0x7f72d8069450 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f72e0005000 tx=0x7f72e0001520 comp rx=0 tx=0).stop 2024-04-25T08:59:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 -- 172.21.15.37:0/3505247519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 msgr2=0x7f72f81625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 0x7f72f81625a0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f72e800e810 tx=0x7f72e800ecd0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 -- 172.21.15.37:0/3505247519 shutdown_connections 2024-04-25T08:59:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f72d8066f90 0x7f72d8069450 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f72f80aca20 0x7f72f8071ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f72f8162ae0 0x7f72f80ac4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 --2- 172.21.15.37:0/3505247519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f72e4004810 0x7f72f81625a0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 -- 172.21.15.37:0/3505247519 >> 172.21.15.37:0/3505247519 conn(0x7f72f806ae60 msgr2=0x7f72f806b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 -- 172.21.15.37:0/3505247519 shutdown_connections 2024-04-25T08:59:56.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.755+0000 7f72fcfed640 1 -- 172.21.15.37:0/3505247519 wait complete. 2024-04-25T08:59:56.764 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T08:59:56.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T08:59:56.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7328a3a640 1 Processor -- start 2024-04-25T08:59:56.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7328a3a640 1 -- start start 2024-04-25T08:59:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7328a3a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415a3c0 0x7f73240ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7328a3a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f732415a7a0 0x7f73240b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7328a3a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f732415e0f0 con 0x7f732415ac00 2024-04-25T08:59:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7328a3a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73240b24a0 con 0x7f732415a7a0 2024-04-25T08:59:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7328a3a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732415b0b0 con 0x7f732415a3c0 2024-04-25T08:59:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7321d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f732415a7a0 0x7f73240b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7321d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f732415a7a0 0x7f73240b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36194/0 (socket says 172.21.15.37:36194) 2024-04-25T08:59:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7321d9b640 1 -- 172.21.15.37:0/128071085 learned_addr learned my addr 172.21.15.37:0/128071085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f732259c640 1 --2- 172.21.15.37:0/128071085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415a3c0 0x7f73240ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7321d9b640 1 -- 172.21.15.37:0/128071085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415a3c0 msgr2=0x7f73240ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7321d9b640 1 --2- 172.21.15.37:0/128071085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415a3c0 0x7f73240ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7321d9b640 1 -- 172.21.15.37:0/128071085 >> v1:172.21.15.37:6789/0 conn(0x7f732415ac00 legacy=0x7f73240cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f7321d9b640 1 -- 172.21.15.37:0/128071085 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732415b390 con 0x7f732415a7a0 2024-04-25T08:59:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.887+0000 7f732159a640 1 -- 172.21.15.37:0/128071085 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104069028 0 0) 0x7f732415e0f0 con 0x7f732415ac00 2024-04-25T08:59:56.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f7321d9b640 1 --2- 172.21.15.37:0/128071085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a7a0 0x7f73240b27a0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f7318009870 tx=0x7f731801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab2ae94eefcd9a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f732159a640 1 -- 172.21.15.37:0/128071085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73180a9610 con 0x7f732415a7a0 2024-04-25T08:59:56.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f732159a640 1 -- 172.21.15.37:0/128071085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73180a97b0 con 0x7f732415a7a0 2024-04-25T08:59:56.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f732159a640 1 -- 172.21.15.37:0/128071085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73180a9b80 con 0x7f732415a7a0 2024-04-25T08:59:56.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f7328a3a640 1 -- 172.21.15.37:0/128071085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a7a0 msgr2=0x7f73240b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f7328a3a640 1 --2- 172.21.15.37:0/128071085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a7a0 0x7f73240b27a0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f7318009870 tx=0x7f731801c530 comp rx=0 tx=0).stop 2024-04-25T08:59:56.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f7328a3a640 1 -- 172.21.15.37:0/128071085 shutdown_connections 2024-04-25T08:59:56.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f7328a3a640 1 --2- 172.21.15.37:0/128071085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a7a0 0x7f73240b27a0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f7328a3a640 1 --2- 172.21.15.37:0/128071085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415a3c0 0x7f73240ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f7328a3a640 1 -- 172.21.15.37:0/128071085 >> 172.21.15.37:0/128071085 conn(0x7f732406ae60 msgr2=0x7f732406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f7328a3a640 1 -- 172.21.15.37:0/128071085 shutdown_connections 2024-04-25T08:59:56.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f7328a3a640 1 -- 172.21.15.37:0/128071085 wait complete. 2024-04-25T08:59:56.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f7328a3a640 1 Processor -- start 2024-04-25T08:59:56.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.891+0000 7f7328a3a640 1 -- start start 2024-04-25T08:59:56.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7328a3a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a3c0 0x7f73240c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7328a3a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732415a7a0 0x7f73240b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7328a3a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415ac00 0x7f73240b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7328a3a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732415edb0 con 0x7f732415a3c0 2024-04-25T08:59:56.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7328a3a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73240b2ef0 con 0x7f732415a7a0 2024-04-25T08:59:56.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7328a3a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f732415ef30 con 0x7f732415ac00 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f732259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a3c0 0x7f73240c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f732259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a3c0 0x7f73240c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36198/0 (socket says 172.21.15.37:36198) 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f732259c640 1 -- 172.21.15.37:0/3628575558 learned_addr learned my addr 172.21.15.37:0/3628575558 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7322d9d640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415ac00 0x7f73240b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7321d9b640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732415a7a0 0x7f73240b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f732259c640 1 -- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415ac00 msgr2=0x7f73240b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f732259c640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415ac00 0x7f73240b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f732259c640 1 -- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732415a7a0 msgr2=0x7f73240b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f732259c640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732415a7a0 0x7f73240b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f732259c640 1 -- 172.21.15.37:0/3628575558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f732415c370 con 0x7f732415a3c0 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7322d9d640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415ac00 0x7f73240b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7321d9b640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732415a7a0 0x7f73240b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f732259c640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a3c0 0x7f73240c46b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f731400d800 tx=0x7f731400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7312ffd640 1 -- 172.21.15.37:0/3628575558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731400b6c0 con 0x7f732415a3c0 2024-04-25T08:59:56.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7312ffd640 1 -- 172.21.15.37:0/3628575558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314019070 con 0x7f732415a3c0 2024-04-25T08:59:56.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7328a3a640 1 -- 172.21.15.37:0/3628575558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f732415c650 con 0x7f732415a3c0 2024-04-25T08:59:56.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7312ffd640 1 -- 172.21.15.37:0/3628575558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731400bae0 con 0x7f732415a3c0 2024-04-25T08:59:56.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7328a3a640 1 -- 172.21.15.37:0/3628575558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7324071010 con 0x7f732415a3c0 2024-04-25T08:59:56.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7312ffd640 1 -- 172.21.15.37:0/3628575558 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f731400bc80 con 0x7f732415a3c0 2024-04-25T08:59:56.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7312ffd640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7304066f60 0x7f7304069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.895+0000 7f7312ffd640 1 -- 172.21.15.37:0/3628575558 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f73140857c0 con 0x7f732415a3c0 2024-04-25T08:59:56.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7321d9b640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7304066f60 0x7f7304069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 Processor -- start 2024-04-25T08:59:56.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 -- start start 2024-04-25T08:59:56.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7321d9b640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7304066f60 0x7f7304069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f73180ae000 tx=0x7f73180b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48ac90 0x5601de48b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48b5b0 0x5601de440d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de441290 0x5601de441670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601de327f40 con 0x5601de48b5b0 2024-04-25T08:59:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601de28f070 con 0x5601de441290 2024-04-25T08:59:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601de2171a0 con 0x5601de48ac90 2024-04-25T08:59:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7321d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48b5b0 0x5601de440d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7321d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48b5b0 0x5601de440d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36208/0 (socket says 172.21.15.37:36208) 2024-04-25T08:59:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7321d9b640 1 -- 172.21.15.37:0/783126779 learned_addr learned my addr 172.21.15.37:0/783126779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7321d9b640 1 -- 172.21.15.37:0/783126779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48ac90 msgr2=0x5601de48b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7321d9b640 1 --2- 172.21.15.37:0/783126779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48ac90 0x5601de48b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7321d9b640 1 -- 172.21.15.37:0/783126779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de441290 msgr2=0x5601de441670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7321d9b640 1 --2- 172.21.15.37:0/783126779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de441290 0x5601de441670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7321d9b640 1 -- 172.21.15.37:0/783126779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601de215510 con 0x5601de48b5b0 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7321d9b640 1 --2- 172.21.15.37:0/783126779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48b5b0 0x5601de440d50 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7318008010 tx=0x7f7318004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a3846e2e0ca18b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7303fff640 1 -- 172.21.15.37:0/783126779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73180a9df0 con 0x5601de48b5b0 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7303fff640 1 -- 172.21.15.37:0/783126779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7318004020 con 0x5601de48b5b0 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f7303fff640 1 -- 172.21.15.37:0/783126779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73180043d0 con 0x5601de48b5b0 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 -- 172.21.15.37:0/783126779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48b5b0 msgr2=0x5601de440d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 --2- 172.21.15.37:0/783126779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48b5b0 0x5601de440d50 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f7318008010 tx=0x7f7318004740 comp rx=0 tx=0).stop 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 -- 172.21.15.37:0/783126779 shutdown_connections 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 --2- 172.21.15.37:0/783126779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de441290 0x5601de441670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 --2- 172.21.15.37:0/783126779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48b5b0 0x5601de440d50 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 --2- 172.21.15.37:0/783126779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48ac90 0x5601de48b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 -- 172.21.15.37:0/783126779 >> 172.21.15.37:0/783126779 conn(0x5601de43d240 msgr2=0x5601de43f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 -- 172.21.15.37:0/783126779 shutdown_connections 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 -- 172.21.15.37:0/783126779 wait complete. 2024-04-25T08:59:56.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 -- start start 2024-04-25T08:59:56.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.899+0000 7f732bba7000 1 Processor -- start 2024-04-25T08:59:56.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de441290 0x5601de44be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de44c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de48b5b0 0x5601de44c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601de2496c0 con 0x5601de48ac90 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601de28b110 con 0x5601de48b5b0 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601de2fcc10 con 0x5601de441290 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 10 client.0 _ll_register_callbacks cb 0x5601de42c570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f7321d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de44c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f7321d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de44c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36212/0 (socket says 172.21.15.37:36212) 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f7321d9b640 1 -- 172.21.15.37:0/1405835008 learned_addr learned my addr 172.21.15.37:0/1405835008 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732259c640 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de441290 0x5601de44be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f7322d9d640 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de48b5b0 0x5601de44c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f7321d9b640 1 -- 172.21.15.37:0/1405835008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de441290 msgr2=0x5601de44be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f7321d9b640 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de441290 0x5601de44be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f7321d9b640 1 -- 172.21.15.37:0/1405835008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de48b5b0 msgr2=0x5601de44c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f7321d9b640 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de48b5b0 0x5601de44c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f7321d9b640 1 -- 172.21.15.37:0/1405835008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601de2173b0 con 0x5601de48ac90 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f7321d9b640 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de44c370 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f731801ca10 tx=0x7f731801ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670b2f1b025820bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f73017fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:56.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f73017fa640 1 -- 172.21.15.37:0/1405835008 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73180d5060 con 0x5601de48ac90 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f73017fa640 1 -- 172.21.15.37:0/1405835008 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f73180bcb50 con 0x5601de48ac90 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 -- 172.21.15.37:0/1405835008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7318004020 con 0x5601de48ac90 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f73017fa640 1 -- 172.21.15.37:0/1405835008 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73180bcd50 con 0x5601de48ac90 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 10 client.8181 fetch_fsmap learned FSMap version 67 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 -- 172.21.15.37:0/1405835008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5601de310e00 con 0x5601de48ac90 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f73017fa640 1 -- 172.21.15.37:0/1405835008 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7318004020 con 0x5601de48ac90 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f73017fa640 1 -- 172.21.15.37:0/1405835008 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f73180a8050 con 0x5601de48ac90 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 10 client.8181 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 10 client.8181 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 20 client.8181 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5601de44d150 0x5601de49c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 4 client.8181 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 -- 172.21.15.37:0/1405835008 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601de22ad60 con 0x5601de44d150 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732259c640 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5601de44d150 0x5601de49c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732259c640 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5601de44d150 0x5601de49c410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610229c125ff808a server_cookie=26003c979b13b492 in_seq=0 out_seq=0 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f73017fa640 10 client.8181 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f73017fa640 1 -- 172.21.15.37:0/1405835008 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5601de22ad60 con 0x5601de44d150 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f73017fa640 10 client.8181 handle_command_reply: tid=0 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 client.8181 shutdown 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 -- 172.21.15.37:0/1405835008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 msgr2=0x5601de44c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de44c370 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f731801ca10 tx=0x7f731801ad70 comp rx=0 tx=0).stop 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 -- 172.21.15.37:0/1405835008 shutdown_connections 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5601de44d150 0x5601de49c410 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de48b5b0 0x5601de44c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de44c370 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 --2- 172.21.15.37:0/1405835008 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de441290 0x5601de44be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f73017fa640 0 client.8181 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 -- 172.21.15.37:0/1405835008 >> 172.21.15.37:0/1405835008 conn(0x5601de43d240 msgr2=0x5601de43fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 -- 172.21.15.37:0/1405835008 shutdown_connections 2024-04-25T08:59:56.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 1 -- 172.21.15.37:0/1405835008 wait complete. 2024-04-25T08:59:56.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.903+0000 7f732bba7000 20 client.8181 trim_cache size 0 max 16384 2024-04-25T08:59:56.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 Processor -- start 2024-04-25T08:59:56.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 -- start start 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de441290 0x5601de440640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48ac90 0x5601de440b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de48b5b0 0x5601de4a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7324200dc0 con 0x5601de441290 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73241fff10 con 0x5601de48b5b0 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7324155f40 con 0x5601de48ac90 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de441290 0x5601de440640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de441290 0x5601de440640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36218/0 (socket says 172.21.15.37:36218) 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732259c640 1 -- 172.21.15.37:0/1897645104 learned_addr learned my addr 172.21.15.37:0/1897645104 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f7322d9d640 1 --2- 172.21.15.37:0/1897645104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de48b5b0 0x5601de4a7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732259c640 1 -- 172.21.15.37:0/1897645104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48ac90 msgr2=0x5601de440b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732259c640 1 --2- 172.21.15.37:0/1897645104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48ac90 0x5601de440b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732259c640 1 -- 172.21.15.37:0/1897645104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de48b5b0 msgr2=0x5601de4a7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732259c640 1 --2- 172.21.15.37:0/1897645104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de48b5b0 0x5601de4a7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732259c640 1 -- 172.21.15.37:0/1897645104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601de2e6c50 con 0x5601de441290 2024-04-25T08:59:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732259c640 1 --2- 172.21.15.37:0/1897645104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de441290 0x5601de440640 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7314055b70 tx=0x7f7314074040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b7eea4484bed90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f73037fe640 1 -- 172.21.15.37:0/1897645104 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731406d070 con 0x5601de441290 2024-04-25T08:59:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f73037fe640 1 -- 172.21.15.37:0/1897645104 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314059fc0 con 0x5601de441290 2024-04-25T08:59:56.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f73037fe640 1 -- 172.21.15.37:0/1897645104 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314083e60 con 0x5601de441290 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 -- 172.21.15.37:0/1897645104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de441290 msgr2=0x5601de440640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 --2- 172.21.15.37:0/1897645104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de441290 0x5601de440640 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f7314055b70 tx=0x7f7314074040 comp rx=0 tx=0).stop 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 -- 172.21.15.37:0/1897645104 shutdown_connections 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 --2- 172.21.15.37:0/1897645104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de48b5b0 0x5601de4a7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 --2- 172.21.15.37:0/1897645104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48ac90 0x5601de440b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 --2- 172.21.15.37:0/1897645104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de441290 0x5601de440640 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 -- 172.21.15.37:0/1897645104 >> 172.21.15.37:0/1897645104 conn(0x5601de43d240 msgr2=0x5601de445d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.919+0000 7f732bba7000 1 -- 172.21.15.37:0/1897645104 shutdown_connections 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- 172.21.15.37:0/1897645104 wait complete. 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- start start 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 Processor -- start 2024-04-25T08:59:56.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de441290 0x5601de4ef380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de4ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48b5b0 0x5601de4aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601de446b70 con 0x5601de48ac90 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73241ff9e0 con 0x5601de441290 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601de44a200 con 0x5601de48b5b0 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 10 client.0 _ll_register_callbacks cb 0x7f73180096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f7321d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de4ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f7321d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de4ef8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36232/0 (socket says 172.21.15.37:36232) 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f7321d9b640 1 -- 172.21.15.37:0/1998255185 learned_addr learned my addr 172.21.15.37:0/1998255185 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f7322d9d640 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48b5b0 0x5601de4aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f7321d9b640 1 -- 172.21.15.37:0/1998255185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48b5b0 msgr2=0x5601de4aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f7321d9b640 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48b5b0 0x5601de4aa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f7321d9b640 1 -- 172.21.15.37:0/1998255185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de441290 msgr2=0x5601de4ef380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f7321d9b640 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de441290 0x5601de4ef380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f7321d9b640 1 -- 172.21.15.37:0/1998255185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601de257120 con 0x5601de48ac90 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f7321d9b640 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de4ef8c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f73180bcf10 tx=0x7f73180a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b21ce4fb4589a717 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f73017fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f73017fa640 1 -- 172.21.15.37:0/1998255185 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7318003aa0 con 0x5601de48ac90 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- 172.21.15.37:0/1998255185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7314059fc0 con 0x5601de48ac90 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f73017fa640 1 -- 172.21.15.37:0/1998255185 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f7318003c40 con 0x5601de48ac90 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f73017fa640 1 -- 172.21.15.37:0/1998255185 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7318003e40 con 0x5601de48ac90 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 10 client.8187 fetch_fsmap learned FSMap version 67 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- 172.21.15.37:0/1998255185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5601de236170 con 0x5601de48ac90 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f73017fa640 1 -- 172.21.15.37:0/1998255185 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314059fc0 con 0x5601de48ac90 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f73017fa640 1 -- 172.21.15.37:0/1998255185 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f73180a8040 con 0x5601de48ac90 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 10 client.8187 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 10 client.8187 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 20 client.8187 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5601de4aab50 0x5601de4aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 4 client.8187 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T08:59:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- 172.21.15.37:0/1998255185 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x5601de29c5b0 con 0x5601de4aab50 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732259c640 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5601de4aab50 0x5601de4aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732259c640 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5601de4aab50 0x5601de4aaf30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23c06a204bedb538 server_cookie=3ca4085cf7b2c362 in_seq=0 out_seq=0 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f73017fa640 10 client.8187 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f73017fa640 1 -- 172.21.15.37:0/1998255185 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5601de29c5b0 con 0x5601de4aab50 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f73017fa640 10 client.8187 handle_command_reply: tid=0 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 client.8187 shutdown 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- 172.21.15.37:0/1998255185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 msgr2=0x5601de4ef8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de4ef8c0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f73180bcf10 tx=0x7f73180a7000 comp rx=0 tx=0).stop 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- 172.21.15.37:0/1998255185 shutdown_connections 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5601de4aab50 0x5601de4aaf30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5601de48b5b0 0x5601de4aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601de48ac90 0x5601de4ef8c0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f73017fa640 0 client.8187 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 --2- 172.21.15.37:0/1998255185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5601de441290 0x5601de4ef380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- 172.21.15.37:0/1998255185 >> 172.21.15.37:0/1998255185 conn(0x5601de43d240 msgr2=0x5601de43f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- 172.21.15.37:0/1998255185 shutdown_connections 2024-04-25T08:59:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 1 -- 172.21.15.37:0/1998255185 wait complete. 2024-04-25T08:59:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.923+0000 7f732bba7000 20 client.8187 trim_cache size 0 max 16384 2024-04-25T08:59:56.922 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:59:56.922 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-04-25T08:59:56.922 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "e091b5b3-66ab-4dd8-8606-a34019146efa", 2024-04-25T08:59:56.922 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-04-25T08:59:56.922 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:59:56.923 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7c47021640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-25T08:59:06.530103+0000 1=1+0)->f(v1 m2024-04-25T08:59:06.530103+0000) n(v0 rc2024-04-25T08:59:06.639471+0000 2=1+1)->n(v0 rc2024-04-25T08:59:06.639471+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x560a400ebb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T08:59:06.530103+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T08:59:06.639471+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T08:59:56.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 -- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7304066f60 msgr2=0x7f7304069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7304066f60 0x7f7304069420 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f73180ae000 tx=0x7f73180b3040 comp rx=0 tx=0).stop 2024-04-25T08:59:56.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 -- 172.21.15.37:0/3628575558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a3c0 msgr2=0x7f73240c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a3c0 0x7f73240c46b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f731400d800 tx=0x7f731400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 -- 172.21.15.37:0/3628575558 shutdown_connections 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7304066f60 0x7f7304069420 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415ac00 0x7f73240b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f732415a7a0 0x7f73240b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 --2- 172.21.15.37:0/3628575558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a3c0 0x7f73240c46b0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 -- 172.21.15.37:0/3628575558 >> 172.21.15.37:0/3628575558 conn(0x7f732406ae60 msgr2=0x7f73240af920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 -- 172.21.15.37:0/3628575558 shutdown_connections 2024-04-25T08:59:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:56.927+0000 7f7328a3a640 1 -- 172.21.15.37:0/3628575558 wait complete. 2024-04-25T08:59:56.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:56.933 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T08:59:57.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.047+0000 7fcf2b6bd640 1 Processor -- start 2024-04-25T08:59:57.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.047+0000 7fcf2b6bd640 1 -- start start 2024-04-25T08:59:57.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf2b6bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf240cfa20 0x7fcf24072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf2b6bd640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf240cfe90 con 0x7fcf240cf590 2024-04-25T08:59:57.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf2b6bd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf240cbd20 con 0x7fcf2415b6b0 2024-04-25T08:59:57.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf2b6bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2415ba90 con 0x7fcf240cfa20 2024-04-25T08:59:57.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf28c58640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fcf240cf590 0x7fcf24072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42308/0 (socket says 172.21.15.37:42308) 2024-04-25T08:59:57.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf28c58640 1 -- 172.21.15.37:0/2775866774 learned_addr learned my addr 172.21.15.37:0/2775866774 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T08:59:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf240cfa20 0x7fcf24072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912373468 0 0) 0x7fcf240cfe90 con 0x7fcf240cf590 2024-04-25T08:59:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf10003530 con 0x7fcf240cf590 2024-04-25T08:59:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 183581401 0 0) 0x7fcf10003530 con 0x7fcf240cf590 2024-04-25T08:59:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf240cfa20 msgr2=0x7fcf24072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf240cfa20 0x7fcf24072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 >> v1:172.21.15.203:6789/0 conn(0x7fcf2415b6b0 legacy=0x7fcf240ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:57.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf240cc000 con 0x7fcf240cf590 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fcf14001680 con 0x7fcf240cf590 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 >> v1:172.21.15.37:6789/0 conn(0x7fcf240cf590 legacy=0x7fcf24072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf240cfa20 0x7fcf10003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf10003fe0 0x7fcf10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf10004d50 0x7fcf100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2415ba90 con 0x7fcf10004d50 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000bb30 con 0x7fcf10003fe0 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1000bd30 con 0x7fcf240cfa20 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf10004d50 0x7fcf100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf28c58640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf10003fe0 0x7fcf10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf29459640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf240cfa20 0x7fcf10003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf29c5a640 1 -- 172.21.15.37:0/2775866774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf240cfa20 msgr2=0x7fcf10003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf240cfa20 0x7fcf10003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf29c5a640 1 -- 172.21.15.37:0/2775866774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf10003fe0 msgr2=0x7fcf10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf10003fe0 0x7fcf10004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf29c5a640 1 -- 172.21.15.37:0/2775866774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf10003530 con 0x7fcf10004d50 2024-04-25T08:59:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf10004d50 0x7fcf100051d0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009fa0 tx=0x7fcf1c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcce4781623b62ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0b2070 con 0x7fcf10004d50 2024-04-25T08:59:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c0a5d10 con 0x7fcf10004d50 2024-04-25T08:59:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.051+0000 7fcf1bfff640 1 -- 172.21.15.37:0/2775866774 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c009bf0 con 0x7fcf10004d50 2024-04-25T08:59:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/2775866774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf10004d50 msgr2=0x7fcf100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 --2- 172.21.15.37:0/2775866774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf10004d50 0x7fcf100051d0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009fa0 tx=0x7fcf1c01cc40 comp rx=0 tx=0).stop 2024-04-25T08:59:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf29c5a640 1 -- 172.21.15.37:0/2775866774 reap_dead start 2024-04-25T08:59:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/2775866774 shutdown_connections 2024-04-25T08:59:57.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/2775866774 >> 172.21.15.37:0/2775866774 conn(0x7fcf2406ae60 msgr2=0x7fcf2406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/2775866774 shutdown_connections 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/2775866774 wait complete. 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 Processor -- start 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 -- start start 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf241f4810 0x7fcf2415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf2406e390 0x7fcf2415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf241f4bf0 0x7fcf2415de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf240cc8a0 con 0x7fcf241f4810 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf240cc5c0 con 0x7fcf241f4bf0 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf240cba30 con 0x7fcf2406e390 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf29459640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf241f4810 0x7fcf2415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf29459640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf241f4810 0x7fcf2415cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36260/0 (socket says 172.21.15.37:36260) 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf29459640 1 -- 172.21.15.37:0/1824330810 learned_addr learned my addr 172.21.15.37:0/1824330810 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:57.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf28c58640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf2406e390 0x7fcf2415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf241f4bf0 0x7fcf2415de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf29459640 1 -- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf2406e390 msgr2=0x7fcf2415d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf29459640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf2406e390 0x7fcf2415d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf29459640 1 -- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf241f4bf0 msgr2=0x7fcf2415de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf29459640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf241f4bf0 0x7fcf2415de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf29459640 1 -- 172.21.15.37:0/1824330810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf240b1ca0 con 0x7fcf241f4810 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf28c58640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf2406e390 0x7fcf2415d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf29459640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf241f4810 0x7fcf2415cb00 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000e550 tx=0x7fcf2000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf19ffb640 1 -- 172.21.15.37:0/1824330810 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf200052e0 con 0x7fcf241f4810 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf19ffb640 1 -- 172.21.15.37:0/1824330810 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf20018070 con 0x7fcf241f4810 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/1824330810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2415a970 con 0x7fcf241f4810 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf19ffb640 1 -- 172.21.15.37:0/1824330810 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20003820 con 0x7fcf241f4810 2024-04-25T08:59:57.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.055+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/1824330810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf240b2d70 con 0x7fcf241f4810 2024-04-25T08:59:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf19ffb640 1 -- 172.21.15.37:0/1824330810 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcf200039c0 con 0x7fcf241f4810 2024-04-25T08:59:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf19ffb640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf08066f90 0x7fcf08069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf19ffb640 1 -- 172.21.15.37:0/1824330810 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fcf20084bc0 con 0x7fcf241f4810 2024-04-25T08:59:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf28c58640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf08066f90 0x7fcf08069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 Processor -- start 2024-04-25T08:59:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 -- start start 2024-04-25T08:59:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf28c58640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf08066f90 0x7fcf08069450 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcf14001060 tx=0x7fcf14001510 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55887186a9d0 0x55887186adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186b2f0 0x558871820b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588718210a0 0x558871821480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55887166dff0 con 0x5588718210a0 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588715f6150 con 0x55887186a9d0 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55887166a090 con 0x55887186b2f0 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf29c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588718210a0 0x558871821480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf29c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588718210a0 0x558871821480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36264/0 (socket says 172.21.15.37:36264) 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf29c5a640 1 -- 172.21.15.37:0/3840276741 learned_addr learned my addr 172.21.15.37:0/3840276741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf28c58640 1 --2- 172.21.15.37:0/3840276741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186b2f0 0x558871820b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf29459640 1 --2- 172.21.15.37:0/3840276741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55887186a9d0 0x55887186adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf29c5a640 1 -- 172.21.15.37:0/3840276741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186b2f0 msgr2=0x558871820b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/3840276741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186b2f0 0x558871820b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf29c5a640 1 -- 172.21.15.37:0/3840276741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55887186a9d0 msgr2=0x55887186adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/3840276741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55887186a9d0 0x55887186adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf29c5a640 1 -- 172.21.15.37:0/3840276741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588718040d0 con 0x5588718210a0 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf28c58640 1 --2- 172.21.15.37:0/3840276741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186b2f0 0x558871820b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/3840276741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588718210a0 0x558871821480 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c002240 tx=0x7fcf1c01b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5564e4484ffc74a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:57.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf0effd640 1 -- 172.21.15.37:0/3840276741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0b2070 con 0x5588718210a0 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf0effd640 1 -- 172.21.15.37:0/3840276741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c0bc070 con 0x5588718210a0 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf0effd640 1 -- 172.21.15.37:0/3840276741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0a5b10 con 0x5588718210a0 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 -- 172.21.15.37:0/3840276741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588718210a0 msgr2=0x558871821480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/3840276741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588718210a0 0x558871821480 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c002240 tx=0x7fcf1c01b480 comp rx=0 tx=0).stop 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 -- 172.21.15.37:0/3840276741 shutdown_connections 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/3840276741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5588718210a0 0x558871821480 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/3840276741 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186b2f0 0x558871820b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/3840276741 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55887186a9d0 0x55887186adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.059+0000 7fcf2e82a000 1 -- 172.21.15.37:0/3840276741 >> 172.21.15.37:0/3840276741 conn(0x55887181d020 msgr2=0x55887181f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- 172.21.15.37:0/3840276741 shutdown_connections 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- 172.21.15.37:0/3840276741 wait complete. 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- start start 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 Processor -- start 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 0x55887184f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 0x55887184fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871850270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558871632250 con 0x55887186b2f0 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558871628690 con 0x5588718210a0 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588715f6a40 con 0x55887186a9d0 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 10 client.0 _ll_register_callbacks cb 0x55887180be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871850270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871850270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36270/0 (socket says 172.21.15.37:36270) 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29c5a640 1 -- 172.21.15.37:0/1293809039 learned_addr learned my addr 172.21.15.37:0/1293809039 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29459640 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 0x55887184f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf28c58640 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 0x55887184fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29c5a640 1 -- 172.21.15.37:0/1293809039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 msgr2=0x55887184fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 0x55887184fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29c5a640 1 -- 172.21.15.37:0/1293809039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 msgr2=0x55887184f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 0x55887184f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29c5a640 1 -- 172.21.15.37:0/1293809039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55887171b1c0 con 0x55887186b2f0 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871850270 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0b9040 tx=0x7fcf10005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ab6fcf29c80452 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf03fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf03fff640 1 -- 172.21.15.37:0/1293809039 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0b2050 con 0x55887186b2f0 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf03fff640 1 -- 172.21.15.37:0/1293809039 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fcf1c0c2460 con 0x55887186b2f0 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- 172.21.15.37:0/1293809039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf1c0bc070 con 0x55887186b2f0 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf03fff640 1 -- 172.21.15.37:0/1293809039 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf1c0cf050 con 0x55887186b2f0 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 10 client.8199 fetch_fsmap learned FSMap version 67 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- 172.21.15.37:0/1293809039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x558871717170 con 0x55887186b2f0 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf03fff640 1 -- 172.21.15.37:0/1293809039 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c0bc070 con 0x55887186b2f0 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf03fff640 1 -- 172.21.15.37:0/1293809039 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcf1c0d2020 con 0x55887186b2f0 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 10 client.8199 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 10 client.8199 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 20 client.8199 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x55887187ac80 0x55887187d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 4 client.8199 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- 172.21.15.37:0/1293809039 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558871609c60 con 0x55887187ac80 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29459640 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x55887187ac80 0x55887187d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf03fff640 10 client.8199 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf29459640 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x55887187ac80 0x55887187d070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4b499a486fc413 server_cookie=c5ae4a55d46e9879 in_seq=0 out_seq=0 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf03fff640 1 -- 172.21.15.37:0/1293809039 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558871609c60 con 0x55887187ac80 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf03fff640 10 client.8199 handle_command_reply: tid=0 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 client.8199 shutdown 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- 172.21.15.37:0/1293809039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 msgr2=0x558871850270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871850270 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c0b9040 tx=0x7fcf10005840 comp rx=0 tx=0).stop 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- 172.21.15.37:0/1293809039 shutdown_connections 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x55887187ac80 0x55887187d070 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871850270 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 0x55887184fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/1293809039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 0x55887184f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- 172.21.15.37:0/1293809039 >> 172.21.15.37:0/1293809039 conn(0x55887181d020 msgr2=0x55887181fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf03fff640 0 client.8199 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- 172.21.15.37:0/1293809039 shutdown_connections 2024-04-25T08:59:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.063+0000 7fcf2e82a000 1 -- 172.21.15.37:0/1293809039 wait complete. 2024-04-25T08:59:57.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.067+0000 7fcf2e82a000 20 client.8199 trim_cache size 0 max 16384 2024-04-25T08:59:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf2e82a000 1 Processor -- start 2024-04-25T08:59:57.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf2e82a000 1 -- start start 2024-04-25T08:59:57.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 0x558871877e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 0x558871878380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x55887181f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf241fee90 con 0x55887186b2f0 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf241fdfe0 con 0x5588718210a0 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf241fd420 con 0x55887186a9d0 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf29c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x55887181f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf29c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x55887181f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36280/0 (socket says 172.21.15.37:36280) 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf29c5a640 1 -- 172.21.15.37:0/3088857758 learned_addr learned my addr 172.21.15.37:0/3088857758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf29c5a640 1 -- 172.21.15.37:0/3088857758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 msgr2=0x558871878380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/3088857758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 0x558871878380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf29c5a640 1 -- 172.21.15.37:0/3088857758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 msgr2=0x558871877e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/3088857758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 0x558871877e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf29c5a640 1 -- 172.21.15.37:0/3088857758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588716b4de0 con 0x55887186b2f0 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.079+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/3088857758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x55887181f410 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009e20 tx=0x7fcf1c0af040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853c620b69a24c63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:57.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf0e7fc640 1 -- 172.21.15.37:0/3088857758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0b2070 con 0x55887186b2f0 2024-04-25T08:59:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf0e7fc640 1 -- 172.21.15.37:0/3088857758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c0bc070 con 0x55887186b2f0 2024-04-25T08:59:57.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf0e7fc640 1 -- 172.21.15.37:0/3088857758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0c2460 con 0x55887186b2f0 2024-04-25T08:59:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- 172.21.15.37:0/3088857758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 msgr2=0x55887181f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/3088857758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x55887181f410 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009e20 tx=0x7fcf1c0af040 comp rx=0 tx=0).stop 2024-04-25T08:59:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- 172.21.15.37:0/3088857758 shutdown_connections 2024-04-25T08:59:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/3088857758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x55887181f410 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/3088857758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 0x558871878380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/3088857758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 0x558871877e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- 172.21.15.37:0/3088857758 >> 172.21.15.37:0/3088857758 conn(0x55887181d020 msgr2=0x558871877730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:57.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- 172.21.15.37:0/3088857758 shutdown_connections 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- 172.21.15.37:0/3088857758 wait complete. 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- start start 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 Processor -- start 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 0x5588718ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 0x5588718cea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871889b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558871608e00 con 0x55887186b2f0 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf241fd9c0 con 0x5588718210a0 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55887166d300 con 0x55887186a9d0 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 10 client.0 _ll_register_callbacks cb 0x5588717ec040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871889b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29c5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871889b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36292/0 (socket says 172.21.15.37:36292) 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29c5a640 1 -- 172.21.15.37:0/2488892148 learned_addr learned my addr 172.21.15.37:0/2488892148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29459640 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 0x5588718ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf28c58640 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 0x5588718cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29c5a640 1 -- 172.21.15.37:0/2488892148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 msgr2=0x5588718cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 0x5588718cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29c5a640 1 -- 172.21.15.37:0/2488892148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 msgr2=0x5588718ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 0x5588718ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29c5a640 1 -- 172.21.15.37:0/2488892148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55887160f8e0 con 0x55887186b2f0 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29c5a640 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871889b50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009670 tx=0x7fcf1c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c7472cfd09f144 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf03fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf03fff640 1 -- 172.21.15.37:0/2488892148 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0b2040 con 0x55887186b2f0 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf03fff640 1 -- 172.21.15.37:0/2488892148 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fcf1c0c2c20 con 0x55887186b2f0 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- 172.21.15.37:0/2488892148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf1c0bc070 con 0x55887186b2f0 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf03fff640 1 -- 172.21.15.37:0/2488892148 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf1c0c2e20 con 0x55887186b2f0 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 10 client.8205 fetch_fsmap learned FSMap version 67 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- 172.21.15.37:0/2488892148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5588716c78f0 con 0x55887186b2f0 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf03fff640 1 -- 172.21.15.37:0/2488892148 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf1c0bc070 con 0x55887186b2f0 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf03fff640 1 -- 172.21.15.37:0/2488892148 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcf1c0d2070 con 0x55887186b2f0 2024-04-25T08:59:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 10 client.8205 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 10 client.8205 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 20 client.8205 populate_metadata read hostname 'smithi037' 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x55887188a510 0x55887188a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 4 client.8205 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- 172.21.15.37:0/2488892148 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55887167b530 con 0x55887188a510 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29459640 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x55887188a510 0x55887188a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf29459640 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x55887188a510 0x55887188a920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad52b8756e64f70 server_cookie=284c531534946fb4 in_seq=0 out_seq=0 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf03fff640 10 client.8205 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf03fff640 1 -- 172.21.15.37:0/2488892148 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x55887167b530 con 0x55887188a510 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf03fff640 10 client.8205 handle_command_reply: tid=0 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 client.8205 shutdown 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.083+0000 7fcf2e82a000 1 -- 172.21.15.37:0/2488892148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 msgr2=0x558871889b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871889b50 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009670 tx=0x7fcf1c004ac0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2e82a000 1 -- 172.21.15.37:0/2488892148 shutdown_connections 2024-04-25T08:59:57.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x55887188a510 0x55887188a920 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55887186b2f0 0x558871889b50 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55887186a9d0 0x5588718cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf03fff640 0 client.8205 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T08:59:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2e82a000 1 --2- 172.21.15.37:0/2488892148 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5588718210a0 0x5588718ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2e82a000 1 -- 172.21.15.37:0/2488892148 >> 172.21.15.37:0/2488892148 conn(0x55887181d020 msgr2=0x558871877fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2e82a000 1 -- 172.21.15.37:0/2488892148 shutdown_connections 2024-04-25T08:59:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2e82a000 1 -- 172.21.15.37:0/2488892148 wait complete. 2024-04-25T08:59:57.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2e82a000 20 client.8205 trim_cache size 0 max 16384 2024-04-25T08:59:57.083 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T08:59:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:59:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-04-25T08:59:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "e091b5b3-66ab-4dd8-8606-a34019146efa": { 2024-04-25T08:59:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/dir", 2024-04-25T08:59:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "e091b5b3-66ab-4dd8-8606-a34019146efa", 2024-04-25T08:59:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,repair" 2024-04-25T08:59:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:59:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T08:59:57.084 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf08066f90 msgr2=0x7fcf08069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf08066f90 0x7fcf08069450 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcf14001060 tx=0x7fcf14001510 comp rx=0 tx=0).stop 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/1824330810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf241f4810 msgr2=0x7fcf2415cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf241f4810 0x7fcf2415cb00 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7fcf2000e550 tx=0x7fcf2000ea10 comp rx=0 tx=0).stop 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/1824330810 shutdown_connections 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcf08066f90 0x7fcf08069450 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcf241f4bf0 0x7fcf2415de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcf2406e390 0x7fcf2415d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 --2- 172.21.15.37:0/1824330810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf241f4810 0x7fcf2415cb00 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/1824330810 >> 172.21.15.37:0/1824330810 conn(0x7fcf2406ae60 msgr2=0x7fcf24071fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/1824330810 shutdown_connections 2024-04-25T08:59:57.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T08:59:57.087+0000 7fcf2b6bd640 1 -- 172.21.15.37:0/1824330810 wait complete. 2024-04-25T08:59:57.093 INFO:tasks.cephfs.filesystem:scrub status for tag:e091b5b3-66ab-4dd8-8606-a34019146efa - {'path': '/dir', 'tag': 'e091b5b3-66ab-4dd8-8606-a34019146efa', 'options': 'recursive,repair'} 2024-04-25T08:59:59.986 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T08:59:59.995+0000 7ff670aed640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-04-25T09:00:26.326 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:00:26.329 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:00:27.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:27.093 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:00:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c9884640 1 Processor -- start 2024-04-25T09:00:27.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c9884640 1 -- start start 2024-04-25T09:00:27.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c9884640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f13c40cfca0 0x7f13c415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c9884640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13c415e4f0 0x7f13c40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c9884640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13c40cbf10 con 0x7f13c40b3cf0 2024-04-25T09:00:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c9884640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c40cc1f0 con 0x7f13c40cfca0 2024-04-25T09:00:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c9884640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13c40cc4d0 con 0x7f13c415e4f0 2024-04-25T09:00:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c27fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f13c40cfca0 0x7f13c415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c27fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f13c40cfca0 0x7f13c415dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57560/0 (socket says 172.21.15.37:57560) 2024-04-25T09:00:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c27fc640 1 -- 172.21.15.37:0/794873837 learned_addr learned my addr 172.21.15.37:0/794873837 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c37fe640 1 --2- 172.21.15.37:0/794873837 >> v2:172.21.15.203:3300/0 conn(0x7f13c415e4f0 0x7f13c40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c27fc640 1 -- 172.21.15.37:0/794873837 >> v2:172.21.15.203:3300/0 conn(0x7f13c415e4f0 msgr2=0x7f13c40caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c27fc640 1 --2- 172.21.15.37:0/794873837 >> v2:172.21.15.203:3300/0 conn(0x7f13c415e4f0 0x7f13c40caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c27fc640 1 -- 172.21.15.37:0/794873837 >> v1:172.21.15.203:6789/0 conn(0x7f13c40b3cf0 legacy=0x7f13c40cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.243+0000 7f13c27fc640 1 -- 172.21.15.37:0/794873837 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c40cc7b0 con 0x7f13c40cfca0 2024-04-25T09:00:27.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c27fc640 1 --2- 172.21.15.37:0/794873837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c40cfca0 0x7f13c415dfb0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f13c40b2be0 tx=0x7f13ac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876c8d8a31d0de58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c1ffb640 1 -- 172.21.15.37:0/794873837 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0b0070 con 0x7f13c40cfca0 2024-04-25T09:00:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c1ffb640 1 -- 172.21.15.37:0/794873837 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ac0ab600 con 0x7f13c40cfca0 2024-04-25T09:00:27.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c1ffb640 1 -- 172.21.15.37:0/794873837 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0ab9d0 con 0x7f13c40cfca0 2024-04-25T09:00:27.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c9884640 1 -- 172.21.15.37:0/794873837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c40cfca0 msgr2=0x7f13c415dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c9884640 1 --2- 172.21.15.37:0/794873837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c40cfca0 0x7f13c415dfb0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f13c40b2be0 tx=0x7f13ac01c520 comp rx=0 tx=0).stop 2024-04-25T09:00:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c9884640 1 -- 172.21.15.37:0/794873837 shutdown_connections 2024-04-25T09:00:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c9884640 1 --2- 172.21.15.37:0/794873837 >> v2:172.21.15.203:3300/0 conn(0x7f13c415e4f0 0x7f13c40caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c9884640 1 --2- 172.21.15.37:0/794873837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c40cfca0 0x7f13c415dfb0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c9884640 1 -- 172.21.15.37:0/794873837 >> 172.21.15.37:0/794873837 conn(0x7f13c406ae60 msgr2=0x7f13c406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c9884640 1 -- 172.21.15.37:0/794873837 shutdown_connections 2024-04-25T09:00:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c9884640 1 -- 172.21.15.37:0/794873837 wait complete. 2024-04-25T09:00:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c9884640 1 Processor -- start 2024-04-25T09:00:27.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.247+0000 7f13c9884640 1 -- start start 2024-04-25T09:00:27.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c9884640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40b3cf0 0x7f13c41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c9884640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13c40cfca0 0x7f13c415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c9884640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c415e4f0 0x7f13c4156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c9884640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40cb9c0 con 0x7f13c415e4f0 2024-04-25T09:00:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c9884640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40cb840 con 0x7f13c40cfca0 2024-04-25T09:00:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c9884640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c40cbb40 con 0x7f13c40b3cf0 2024-04-25T09:00:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c415e4f0 0x7f13c4156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c415e4f0 0x7f13c4156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57570/0 (socket says 172.21.15.37:57570) 2024-04-25T09:00:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c37fe640 1 -- 172.21.15.37:0/2831099216 learned_addr learned my addr 172.21.15.37:0/2831099216 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c27fc640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13c40cfca0 0x7f13c415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40b3cf0 0x7f13c41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c37fe640 1 -- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40b3cf0 msgr2=0x7f13c41552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c37fe640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40b3cf0 0x7f13c41552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c37fe640 1 -- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13c40cfca0 msgr2=0x7f13c415a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c37fe640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13c40cfca0 0x7f13c415a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c37fe640 1 -- 172.21.15.37:0/2831099216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13c415a630 con 0x7f13c415e4f0 2024-04-25T09:00:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c27fc640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13c40cfca0 0x7f13c415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c37fe640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c415e4f0 0x7f13c4156df0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00d560 tx=0x7f13bc00da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13bbfff640 1 -- 172.21.15.37:0/2831099216 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0125f0 con 0x7f13c415e4f0 2024-04-25T09:00:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13bbfff640 1 -- 172.21.15.37:0/2831099216 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc019070 con 0x7f13c415e4f0 2024-04-25T09:00:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c9884640 1 -- 172.21.15.37:0/2831099216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c41577d0 con 0x7f13c415e4f0 2024-04-25T09:00:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13bbfff640 1 -- 172.21.15.37:0/2831099216 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc0129f0 con 0x7f13c415e4f0 2024-04-25T09:00:27.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c9884640 1 -- 172.21.15.37:0/2831099216 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c4159610 con 0x7f13c415e4f0 2024-04-25T09:00:27.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13bbfff640 1 -- 172.21.15.37:0/2831099216 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13bc012b90 con 0x7f13c415e4f0 2024-04-25T09:00:27.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13bbfff640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a4066f60 0x7f13a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13bbfff640 1 -- 172.21.15.37:0/2831099216 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f13bc0856a0 con 0x7f13c415e4f0 2024-04-25T09:00:27.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.251+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a4066f60 0x7f13a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 Processor -- start 2024-04-25T09:00:27.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 -- start start 2024-04-25T09:00:27.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a4066f60 0x7f13a4069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f13c41f9d50 tx=0x7f13b4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa1f9d0 0x56437aa1fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa202f0 0x56437a9d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437a822ff0 con 0x56437a9d60a0 2024-04-25T09:00:27.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437a7ab150 con 0x56437aa202f0 2024-04-25T09:00:27.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56437a81f090 con 0x56437aa1f9d0 2024-04-25T09:00:27.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57586/0 (socket says 172.21.15.37:57586) 2024-04-25T09:00:27.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c37fe640 1 -- 172.21.15.37:0/2280701582 learned_addr learned my addr 172.21.15.37:0/2280701582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c27fc640 1 --2- 172.21.15.37:0/2280701582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa202f0 0x56437a9d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/2280701582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa1f9d0 0x56437aa1fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c37fe640 1 -- 172.21.15.37:0/2280701582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa1f9d0 msgr2=0x56437aa1fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c37fe640 1 --2- 172.21.15.37:0/2280701582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa1f9d0 0x56437aa1fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c37fe640 1 -- 172.21.15.37:0/2280701582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa202f0 msgr2=0x56437a9d5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c37fe640 1 --2- 172.21.15.37:0/2280701582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa202f0 0x56437a9d5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c37fe640 1 -- 172.21.15.37:0/2280701582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56437a9b90d0 con 0x56437a9d60a0 2024-04-25T09:00:27.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/2280701582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa1f9d0 0x56437aa1fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13c37fe640 1 --2- 172.21.15.37:0/2280701582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d6480 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f13bc05dea0 tx=0x7f13bc0564f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1891a382a0c5a7b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13b97fa640 1 -- 172.21.15.37:0/2280701582 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc05c0c0 con 0x56437a9d60a0 2024-04-25T09:00:27.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13b97fa640 1 -- 172.21.15.37:0/2280701582 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc012e80 con 0x56437a9d60a0 2024-04-25T09:00:27.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13b97fa640 1 -- 172.21.15.37:0/2280701582 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13bc052330 con 0x56437a9d60a0 2024-04-25T09:00:27.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 -- 172.21.15.37:0/2280701582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 msgr2=0x56437a9d6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/2280701582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d6480 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f13bc05dea0 tx=0x7f13bc0564f0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 -- 172.21.15.37:0/2280701582 shutdown_connections 2024-04-25T09:00:27.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/2280701582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d6480 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/2280701582 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa202f0 0x56437a9d5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/2280701582 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa1f9d0 0x56437aa1fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 -- 172.21.15.37:0/2280701582 >> 172.21.15.37:0/2280701582 conn(0x56437a9d2020 msgr2=0x56437a9d4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 -- 172.21.15.37:0/2280701582 shutdown_connections 2024-04-25T09:00:27.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 -- 172.21.15.37:0/2280701582 wait complete. 2024-04-25T09:00:27.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 -- start start 2024-04-25T09:00:27.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.255+0000 7f13cc9f1000 1 Processor -- start 2024-04-25T09:00:27.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437aa047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa1f9d0 0x56437aa04d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 0x56437aa05270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437a7e7250 con 0x56437a9d60a0 2024-04-25T09:00:27.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437a7dd690 con 0x56437aa1f9d0 2024-04-25T09:00:27.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56437a7aba40 con 0x56437aa202f0 2024-04-25T09:00:27.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 10 client.0 _ll_register_callbacks cb 0x56437a9c0e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:27.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437aa047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437aa047f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57590/0 (socket says 172.21.15.37:57590) 2024-04-25T09:00:27.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c2ffd640 1 -- 172.21.15.37:0/877450522 learned_addr learned my addr 172.21.15.37:0/877450522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c27fc640 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa1f9d0 0x56437aa04d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c37fe640 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 0x56437aa05270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c2ffd640 1 -- 172.21.15.37:0/877450522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 msgr2=0x56437aa05270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 0x56437aa05270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c2ffd640 1 -- 172.21.15.37:0/877450522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa1f9d0 msgr2=0x56437aa04d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa1f9d0 0x56437aa04d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c2ffd640 1 -- 172.21.15.37:0/877450522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56437a8d01c0 con 0x56437a9d60a0 2024-04-25T09:00:27.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437aa047f0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f13b400eb50 tx=0x7f13b4000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c1f13a3281b7f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:00:27.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13a27fc640 1 -- 172.21.15.37:0/877450522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4005830 con 0x56437a9d60a0 2024-04-25T09:00:27.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13a27fc640 1 -- 172.21.15.37:0/877450522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f13b4021cd0 con 0x56437a9d60a0 2024-04-25T09:00:27.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 -- 172.21.15.37:0/877450522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13bc012e80 con 0x56437a9d60a0 2024-04-25T09:00:27.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13a27fc640 1 -- 172.21.15.37:0/877450522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13b4022430 con 0x56437a9d60a0 2024-04-25T09:00:27.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 10 client.8217 fetch_fsmap learned FSMap version 67 2024-04-25T09:00:27.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 -- 172.21.15.37:0/877450522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x56437a8cc170 con 0x56437a9d60a0 2024-04-25T09:00:27.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13a27fc640 1 -- 172.21.15.37:0/877450522 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13bc012e80 con 0x56437a9d60a0 2024-04-25T09:00:27.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13a27fc640 1 -- 172.21.15.37:0/877450522 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f13b4021420 con 0x56437a9d60a0 2024-04-25T09:00:27.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 10 client.8217 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T09:00:27.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 10 client.8217 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:00:27.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 20 client.8217 populate_metadata read hostname 'smithi037' 2024-04-25T09:00:27.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x56437aa2fc80 0x56437aa32070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 4 client.8217 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:00:27.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 -- 172.21.15.37:0/877450522 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56437a7bec60 con 0x56437aa2fc80 2024-04-25T09:00:27.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c27fc640 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x56437aa2fc80 0x56437aa32070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13c27fc640 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x56437aa2fc80 0x56437aa32070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60994d2b5ecefac2 server_cookie=73b2477d2c1f600d in_seq=0 out_seq=0 2024-04-25T09:00:27.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13a27fc640 10 client.8217 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T09:00:27.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13a27fc640 1 -- 172.21.15.37:0/877450522 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56437a7bec60 con 0x56437aa2fc80 2024-04-25T09:00:27.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13a27fc640 10 client.8217 handle_command_reply: tid=0 2024-04-25T09:00:27.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 client.8217 shutdown 2024-04-25T09:00:27.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 -- 172.21.15.37:0/877450522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 msgr2=0x56437aa047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437aa047f0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f13b400eb50 tx=0x7f13b4000f50 comp rx=0 tx=0).stop 2024-04-25T09:00:27.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 -- 172.21.15.37:0/877450522 shutdown_connections 2024-04-25T09:00:27.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x56437aa2fc80 0x56437aa32070 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 0x56437aa05270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13a27fc640 0 client.8217 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T09:00:27.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa1f9d0 0x56437aa04d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/877450522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437aa047f0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 -- 172.21.15.37:0/877450522 >> 172.21.15.37:0/877450522 conn(0x56437a9d2020 msgr2=0x56437a9d4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 -- 172.21.15.37:0/877450522 shutdown_connections 2024-04-25T09:00:27.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 1 -- 172.21.15.37:0/877450522 wait complete. 2024-04-25T09:00:27.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.259+0000 7f13cc9f1000 20 client.8217 trim_cache size 0 max 16384 2024-04-25T09:00:27.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13cc9f1000 1 Processor -- start 2024-04-25T09:00:27.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13cc9f1000 1 -- start start 2024-04-25T09:00:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa1f9d0 0x56437aa2ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 0x56437aa2d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c4203320 con 0x56437a9d60a0 2024-04-25T09:00:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c42024a0 con 0x56437aa1f9d0 2024-04-25T09:00:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13c41566c0 con 0x56437aa202f0 2024-04-25T09:00:27.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13c2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13c2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57602/0 (socket says 172.21.15.37:57602) 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13c2ffd640 1 -- 172.21.15.37:0/255967447 learned_addr learned my addr 172.21.15.37:0/255967447 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13c27fc640 1 --2- 172.21.15.37:0/255967447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa1f9d0 0x56437aa2ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13c2ffd640 1 -- 172.21.15.37:0/255967447 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 msgr2=0x56437aa2d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/255967447 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 0x56437aa2d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13c2ffd640 1 -- 172.21.15.37:0/255967447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa1f9d0 msgr2=0x56437aa2ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/255967447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa1f9d0 0x56437aa2ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13c2ffd640 1 -- 172.21.15.37:0/255967447 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56437a869de0 con 0x56437a9d60a0 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13c27fc640 1 --2- 172.21.15.37:0/255967447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa1f9d0 0x56437aa2ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/255967447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d5a50 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f13b4000f90 tx=0x7f13b401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185787b153bd686e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13b8ff9640 1 -- 172.21.15.37:0/255967447 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4030040 con 0x56437a9d60a0 2024-04-25T09:00:27.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13b8ff9640 1 -- 172.21.15.37:0/255967447 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b4005830 con 0x56437a9d60a0 2024-04-25T09:00:27.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.275+0000 7f13b8ff9640 1 -- 172.21.15.37:0/255967447 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13b4021420 con 0x56437a9d60a0 2024-04-25T09:00:27.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- 172.21.15.37:0/255967447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 msgr2=0x56437a9d5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/255967447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d5a50 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f13b4000f90 tx=0x7f13b401e000 comp rx=0 tx=0).stop 2024-04-25T09:00:27.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- 172.21.15.37:0/255967447 shutdown_connections 2024-04-25T09:00:27.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/255967447 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 0x56437aa2d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/255967447 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437aa1f9d0 0x56437aa2ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/255967447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437a9d60a0 0x56437a9d5a50 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- 172.21.15.37:0/255967447 >> 172.21.15.37:0/255967447 conn(0x56437a9d2020 msgr2=0x56437aa2c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- 172.21.15.37:0/255967447 shutdown_connections 2024-04-25T09:00:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- 172.21.15.37:0/255967447 wait complete. 2024-04-25T09:00:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- start start 2024-04-25T09:00:27.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 Processor -- start 2024-04-25T09:00:27.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437a9d60a0 0x56437aa7f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437aa1f9d0 0x56437aa7fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 0x56437aa31ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56437a7bde00 con 0x56437aa1f9d0 2024-04-25T09:00:27.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c4202040 con 0x56437a9d60a0 2024-04-25T09:00:27.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56437a822300 con 0x56437aa202f0 2024-04-25T09:00:27.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 10 client.0 _ll_register_callbacks cb 0x56437a9a1040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:27.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437aa1f9d0 0x56437aa7fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437aa1f9d0 0x56437aa7fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57608/0 (socket says 172.21.15.37:57608) 2024-04-25T09:00:27.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c27fc640 1 -- 172.21.15.37:0/1467591585 learned_addr learned my addr 172.21.15.37:0/1467591585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437a9d60a0 0x56437aa7f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c37fe640 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 0x56437aa31ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c27fc640 1 -- 172.21.15.37:0/1467591585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 msgr2=0x56437aa31ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c27fc640 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 0x56437aa31ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c27fc640 1 -- 172.21.15.37:0/1467591585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437a9d60a0 msgr2=0x56437aa7f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c27fc640 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437a9d60a0 0x56437aa7f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c27fc640 1 -- 172.21.15.37:0/1467591585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56437a7c48e0 con 0x56437aa1f9d0 2024-04-25T09:00:27.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437a9d60a0 0x56437aa7f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c27fc640 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437aa1f9d0 0x56437aa7fe70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f13ac000c10 tx=0x7f13ac0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b4bfb2bc5b7ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:00:27.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13a27fc640 1 -- 172.21.15.37:0/1467591585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ac0c3060 con 0x56437aa1f9d0 2024-04-25T09:00:27.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- 172.21.15.37:0/1467591585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f13b4005830 con 0x56437aa1f9d0 2024-04-25T09:00:27.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13a27fc640 1 -- 172.21.15.37:0/1467591585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f13ac0bed80 con 0x56437aa1f9d0 2024-04-25T09:00:27.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13a27fc640 1 -- 172.21.15.37:0/1467591585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13ac0c9050 con 0x56437aa1f9d0 2024-04-25T09:00:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 10 client.8223 fetch_fsmap learned FSMap version 67 2024-04-25T09:00:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- 172.21.15.37:0/1467591585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x56437a87c8f0 con 0x56437aa1f9d0 2024-04-25T09:00:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13a27fc640 1 -- 172.21.15.37:0/1467591585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13b4005830 con 0x56437aa1f9d0 2024-04-25T09:00:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13a27fc640 1 -- 172.21.15.37:0/1467591585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f13ac0cc020 con 0x56437aa1f9d0 2024-04-25T09:00:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 10 client.8223 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T09:00:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 10 client.8223 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:00:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 20 client.8223 populate_metadata read hostname 'smithi037' 2024-04-25T09:00:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x56437aa32860 0x56437aa32c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 4 client.8223 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:00:27.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13cc9f1000 1 -- 172.21.15.37:0/1467591585 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56437a830530 con 0x56437aa32860 2024-04-25T09:00:27.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x56437aa32860 0x56437aa32c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13a27fc640 10 client.8223 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T09:00:27.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.279+0000 7f13c2ffd640 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x56437aa32860 0x56437aa32c70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440031a841789ee9 server_cookie=9db87fcd00bd4595 in_seq=0 out_seq=0 2024-04-25T09:00:27.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13a27fc640 1 -- 172.21.15.37:0/1467591585 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56437a830530 con 0x56437aa32860 2024-04-25T09:00:27.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13a27fc640 10 client.8223 handle_command_reply: tid=0 2024-04-25T09:00:27.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 1 client.8223 shutdown 2024-04-25T09:00:27.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 1 -- 172.21.15.37:0/1467591585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437aa1f9d0 msgr2=0x56437aa7fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437aa1f9d0 0x56437aa7fe70 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f13ac000c10 tx=0x7f13ac0046d0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 1 -- 172.21.15.37:0/1467591585 shutdown_connections 2024-04-25T09:00:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x56437aa32860 0x56437aa32c70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56437aa202f0 0x56437aa31ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56437aa1f9d0 0x56437aa7fe70 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 1 --2- 172.21.15.37:0/1467591585 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56437a9d60a0 0x56437aa7f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13a27fc640 0 client.8223 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T09:00:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 1 -- 172.21.15.37:0/1467591585 >> 172.21.15.37:0/1467591585 conn(0x56437a9d2020 msgr2=0x56437aa2cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 1 -- 172.21.15.37:0/1467591585 shutdown_connections 2024-04-25T09:00:27.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 1 -- 172.21.15.37:0/1467591585 wait complete. 2024-04-25T09:00:27.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13cc9f1000 20 client.8223 trim_cache size 0 max 16384 2024-04-25T09:00:27.285 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:00:27.285 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-04-25T09:00:27.285 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-04-25T09:00:27.285 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:00:27.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 -- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a4066f60 msgr2=0x7f13a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a4066f60 0x7f13a4069420 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f13c41f9d50 tx=0x7f13b4009040 comp rx=0 tx=0).stop 2024-04-25T09:00:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 -- 172.21.15.37:0/2831099216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c415e4f0 msgr2=0x7f13c4156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c415e4f0 0x7f13c4156df0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f13bc00d560 tx=0x7f13bc00da20 comp rx=0 tx=0).stop 2024-04-25T09:00:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 -- 172.21.15.37:0/2831099216 shutdown_connections 2024-04-25T09:00:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13a4066f60 0x7f13a4069420 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13c415e4f0 0x7f13c4156df0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13c40cfca0 0x7f13c415a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 --2- 172.21.15.37:0/2831099216 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f13c40b3cf0 0x7f13c41552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 -- 172.21.15.37:0/2831099216 >> 172.21.15.37:0/2831099216 conn(0x7f13c406ae60 msgr2=0x7f13c406e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 -- 172.21.15.37:0/2831099216 shutdown_connections 2024-04-25T09:00:27.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.283+0000 7f13c9884640 1 -- 172.21.15.37:0/2831099216 wait complete. 2024-04-25T09:00:27.291 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:00:27.291 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:27.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.411+0000 7f75c7c08640 1 Processor -- start 2024-04-25T09:00:27.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.411+0000 7f75c7c08640 1 -- start start 2024-04-25T09:00:27.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c7c08640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f75c00d0870 0x7f75c00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c7c08640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c00ce6b0 con 0x7f75c015d850 2024-04-25T09:00:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c7c08640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c00ce990 con 0x7f75c015dc30 2024-04-25T09:00:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c7c08640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c00cec70 con 0x7f75c00d0870 2024-04-25T09:00:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c59a4640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f75c015d850 0x7f75c00d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32778/0 (socket says 172.21.15.37:32778) 2024-04-25T09:00:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c59a4640 1 -- 172.21.15.37:0/1693638753 learned_addr learned my addr 172.21.15.37:0/1693638753 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c51a3640 1 --2- 172.21.15.37:0/1693638753 >> v2:172.21.15.203:3300/0 conn(0x7f75c00d0870 0x7f75c00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 357863833 0 0) 0x7f75c00ce6b0 con 0x7f75c015d850 2024-04-25T09:00:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75a8003530 con 0x7f75c015d850 2024-04-25T09:00:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2677980832 0 0) 0x7f75a8003530 con 0x7f75c015d850 2024-04-25T09:00:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 >> v2:172.21.15.203:3300/0 conn(0x7f75c00d0870 msgr2=0x7f75c00d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 --2- 172.21.15.37:0/1693638753 >> v2:172.21.15.203:3300/0 conn(0x7f75c00d0870 0x7f75c00d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 >> v1:172.21.15.203:6789/0 conn(0x7f75c015dc30 legacy=0x7f75c0167910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c00cef50 con 0x7f75c015d850 2024-04-25T09:00:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f75b0001680 con 0x7f75c015d850 2024-04-25T09:00:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 >> v1:172.21.15.37:6789/0 conn(0x7f75c015d850 legacy=0x7f75c00d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 --2- 172.21.15.37:0/1693638753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75a8003a30 0x7f75a8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 --2- 172.21.15.37:0/1693638753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75a8004810 0x7f75a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 --2- 172.21.15.37:0/1693638753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75a80051d0 0x7f75a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c00cec70 con 0x7f75a8003a30 2024-04-25T09:00:27.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75a800c100 con 0x7f75a80051d0 2024-04-25T09:00:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75a800c300 con 0x7f75a8004810 2024-04-25T09:00:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c59a4640 1 --2- 172.21.15.37:0/1693638753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75a8003a30 0x7f75a8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c61a5640 1 --2- 172.21.15.37:0/1693638753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75a80051d0 0x7f75a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c51a3640 1 --2- 172.21.15.37:0/1693638753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75a8004810 0x7f75a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c59a4640 1 -- 172.21.15.37:0/1693638753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75a8004810 msgr2=0x7f75a8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c59a4640 1 --2- 172.21.15.37:0/1693638753 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75a8004810 0x7f75a8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c59a4640 1 -- 172.21.15.37:0/1693638753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75a80051d0 msgr2=0x7f75a800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c59a4640 1 --2- 172.21.15.37:0/1693638753 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75a80051d0 0x7f75a800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c59a4640 1 -- 172.21.15.37:0/1693638753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a8003530 con 0x7f75a8003a30 2024-04-25T09:00:27.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.415+0000 7f75c61a5640 1 -- 172.21.15.37:0/1693638753 reap_dead start 2024-04-25T09:00:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c59a4640 1 --2- 172.21.15.37:0/1693638753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75a8003a30 0x7f75a8003e50 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c210 tx=0x7f75b001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b44380e6a684f5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0007d40 con 0x7f75a8003a30 2024-04-25T09:00:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b00a9070 con 0x7f75a8003a30 2024-04-25T09:00:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c49a2640 1 -- 172.21.15.37:0/1693638753 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0005d80 con 0x7f75a8003a30 2024-04-25T09:00:27.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 -- 172.21.15.37:0/1693638753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75a8003a30 msgr2=0x7f75a8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 --2- 172.21.15.37:0/1693638753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75a8003a30 0x7f75a8003e50 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f75b000c210 tx=0x7f75b001cca0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 -- 172.21.15.37:0/1693638753 shutdown_connections 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 --2- 172.21.15.37:0/1693638753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75a8003a30 0x7f75a8003e50 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 -- 172.21.15.37:0/1693638753 >> 172.21.15.37:0/1693638753 conn(0x7f75c008b000 msgr2=0x7f75c00d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 -- 172.21.15.37:0/1693638753 shutdown_connections 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 -- 172.21.15.37:0/1693638753 wait complete. 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 Processor -- start 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 -- start start 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75a8003a30 0x7f75c00c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75c01eb1a0 0x7f75c00c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75c00c9130 0x7f75c00b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c0161760 con 0x7f75c00c9130 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c01615e0 con 0x7f75c01eb1a0 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c7c08640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c01618e0 con 0x7f75a8003a30 2024-04-25T09:00:27.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c61a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75c00c9130 0x7f75c00b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c61a5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75c00c9130 0x7f75c00b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57636/0 (socket says 172.21.15.37:57636) 2024-04-25T09:00:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c61a5640 1 -- 172.21.15.37:0/100452050 learned_addr learned my addr 172.21.15.37:0/100452050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c61a5640 1 -- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75a8003a30 msgr2=0x7f75c00c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c61a5640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75a8003a30 0x7f75c00c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c61a5640 1 -- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75c01eb1a0 msgr2=0x7f75c00c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c51a3640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75c01eb1a0 0x7f75c00c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c61a5640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75c01eb1a0 0x7f75c00c8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c61a5640 1 -- 172.21.15.37:0/100452050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c00d1c10 con 0x7f75c00c9130 2024-04-25T09:00:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75c61a5640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75c00c9130 0x7f75c00b43a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f75c00c70a0 tx=0x7f75b800e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.419+0000 7f75be7fc640 1 -- 172.21.15.37:0/100452050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8019070 con 0x7f75c00c9130 2024-04-25T09:00:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.423+0000 7f75be7fc640 1 -- 172.21.15.37:0/100452050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b8006e30 con 0x7f75c00c9130 2024-04-25T09:00:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.423+0000 7f75c7c08640 1 -- 172.21.15.37:0/100452050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c00c95e0 con 0x7f75c00c9130 2024-04-25T09:00:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.423+0000 7f75be7fc640 1 -- 172.21.15.37:0/100452050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b8003ba0 con 0x7f75c00c9130 2024-04-25T09:00:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.423+0000 7f75c7c08640 1 -- 172.21.15.37:0/100452050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c01eb660 con 0x7f75c00c9130 2024-04-25T09:00:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.423+0000 7f75c7c08640 1 -- 172.21.15.37:0/100452050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c004efa0 con 0x7f75c00c9130 2024-04-25T09:00:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.423+0000 7f75be7fc640 1 -- 172.21.15.37:0/100452050 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f75b801c070 con 0x7f75c00c9130 2024-04-25T09:00:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.423+0000 7f75be7fc640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75ac066f90 0x7f75ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.423+0000 7f75be7fc640 1 -- 172.21.15.37:0/100452050 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f75b8084830 con 0x7f75c00c9130 2024-04-25T09:00:27.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.423+0000 7f75c59a4640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75ac066f90 0x7f75ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.423+0000 7f75c59a4640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75ac066f90 0x7f75ac069450 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f75b00ab000 tx=0x7f75b0001a50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.427+0000 7f75be7fc640 1 -- 172.21.15.37:0/100452050 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f75c00c95e0 con 0x7f75c00c9130 2024-04-25T09:00:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.627+0000 7f75c7c08640 1 -- 172.21.15.37:0/100452050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75c01e7070 con 0x7f75c00c9130 2024-04-25T09:00:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.627+0000 7f75be7fc640 1 -- 172.21.15.37:0/100452050 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f75b8055550 con 0x7f75c00c9130 2024-04-25T09:00:27.621 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:00:27.621 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T08:59:28.123805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/1462327322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1462327322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1462327322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:00:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 67 2024-04-25T09:00:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 -- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75ac066f90 msgr2=0x7f75ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75ac066f90 0x7f75ac069450 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f75b00ab000 tx=0x7f75b0001a50 comp rx=0 tx=0).stop 2024-04-25T09:00:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 -- 172.21.15.37:0/100452050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75c00c9130 msgr2=0x7f75c00b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75c00c9130 0x7f75c00b43a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f75c00c70a0 tx=0x7f75b800e960 comp rx=0 tx=0).stop 2024-04-25T09:00:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 -- 172.21.15.37:0/100452050 shutdown_connections 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f75ac066f90 0x7f75ac069450 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75c00c9130 0x7f75c00b43a0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75c01eb1a0 0x7f75c00c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 --2- 172.21.15.37:0/100452050 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75a8003a30 0x7f75c00c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 -- 172.21.15.37:0/100452050 >> 172.21.15.37:0/100452050 conn(0x7f75c008b000 msgr2=0x7f75c015e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 -- 172.21.15.37:0/100452050 shutdown_connections 2024-04-25T09:00:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.631+0000 7f75c7c08640 1 -- 172.21.15.37:0/100452050 wait complete. 2024-04-25T09:00:27.639 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T09:00:27.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.767+0000 7fc4295a9640 1 Processor -- start 2024-04-25T09:00:27.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.767+0000 7fc4295a9640 1 -- start start 2024-04-25T09:00:27.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4295a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4240cf590 0x7fc4240cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4295a9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4240cbd20 con 0x7fc42415b6b0 2024-04-25T09:00:27.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4295a9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4240cc000 con 0x7fc424072000 2024-04-25T09:00:27.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4295a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42415ba90 con 0x7fc4240cf590 2024-04-25T09:00:27.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc422ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc42415b6b0 0x7fc4240ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32782/0 (socket says 172.21.15.37:32782) 2024-04-25T09:00:27.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc422ffd640 1 -- 172.21.15.37:0/3760428499 learned_addr learned my addr 172.21.15.37:0/3760428499 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:27.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4240cf590 0x7fc4240cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939835987 0 0) 0x7fc4240cbd20 con 0x7fc42415b6b0 2024-04-25T09:00:27.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc410003530 con 0x7fc42415b6b0 2024-04-25T09:00:27.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2226062814 0 0) 0x7fc410003530 con 0x7fc42415b6b0 2024-04-25T09:00:27.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4240cf590 msgr2=0x7fc4240cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4240cf590 0x7fc4240cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 >> v1:172.21.15.203:6789/0 conn(0x7fc424072000 legacy=0x7fc424072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240cc2e0 con 0x7fc42415b6b0 2024-04-25T09:00:27.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fc414001680 con 0x7fc42415b6b0 2024-04-25T09:00:27.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 >> v1:172.21.15.37:6789/0 conn(0x7fc42415b6b0 legacy=0x7fc4240ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc410003a60 0x7fc410003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4240cf590 0x7fc410004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc410004d50 0x7fc4100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc42415ba90 con 0x7fc410004d50 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41000bb60 con 0x7fc410003a60 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41000bd30 con 0x7fc4240cf590 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4237fe640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc410004d50 0x7fc4100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc422ffd640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc410003a60 0x7fc410003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4240cf590 0x7fc410004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4237fe640 1 -- 172.21.15.37:0/3760428499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4240cf590 msgr2=0x7fc410004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4237fe640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4240cf590 0x7fc410004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4237fe640 1 -- 172.21.15.37:0/3760428499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc410003a60 msgr2=0x7fc410003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4237fe640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc410003a60 0x7fc410003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4237fe640 1 -- 172.21.15.37:0/3760428499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410003530 con 0x7fc410004d50 2024-04-25T09:00:27.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.771+0000 7fc4237fe640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc410004d50 0x7fc4100051d0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fc41c001e70 tx=0x7fc41c01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdd67c85a1e04e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c001a90 con 0x7fc410004d50 2024-04-25T09:00:27.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c003b80 con 0x7fc410004d50 2024-04-25T09:00:27.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4227fc640 1 -- 172.21.15.37:0/3760428499 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c005080 con 0x7fc410004d50 2024-04-25T09:00:27.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 -- 172.21.15.37:0/3760428499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc410004d50 msgr2=0x7fc4100051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 --2- 172.21.15.37:0/3760428499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc410004d50 0x7fc4100051d0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fc41c001e70 tx=0x7fc41c01c740 comp rx=0 tx=0).stop 2024-04-25T09:00:27.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4237fe640 1 -- 172.21.15.37:0/3760428499 reap_dead start 2024-04-25T09:00:27.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 -- 172.21.15.37:0/3760428499 shutdown_connections 2024-04-25T09:00:27.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 -- 172.21.15.37:0/3760428499 >> 172.21.15.37:0/3760428499 conn(0x7fc42406ae60 msgr2=0x7fc42406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 -- 172.21.15.37:0/3760428499 shutdown_connections 2024-04-25T09:00:27.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 -- 172.21.15.37:0/3760428499 wait complete. 2024-04-25T09:00:27.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 Processor -- start 2024-04-25T09:00:27.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 -- start start 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc410004d50 0x7fc4240c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc424072af0 0x7fc424158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc424157d40 0x7fc4241591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240cba30 con 0x7fc424157d40 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240cc8a0 con 0x7fc410004d50 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240cadb0 con 0x7fc424072af0 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc424157d40 0x7fc4241591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc424157d40 0x7fc4241591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57648/0 (socket says 172.21.15.37:57648) 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4237fe640 1 -- 172.21.15.37:0/2671899917 learned_addr learned my addr 172.21.15.37:0/2671899917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc422ffd640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc410004d50 0x7fc4240c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc424072af0 0x7fc424158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4237fe640 1 -- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc424072af0 msgr2=0x7fc424158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4237fe640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc424072af0 0x7fc424158cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4237fe640 1 -- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc410004d50 msgr2=0x7fc4240c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4237fe640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc410004d50 0x7fc4240c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4237fe640 1 -- 172.21.15.37:0/2671899917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc42415aee0 con 0x7fc424157d40 2024-04-25T09:00:27.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4237fe640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc424157d40 0x7fc4241591f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc41c01c570 tx=0x7fc410005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc41bfff640 1 -- 172.21.15.37:0/2671899917 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c0b8020 con 0x7fc424157d40 2024-04-25T09:00:27.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc41bfff640 1 -- 172.21.15.37:0/2671899917 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41c0012e0 con 0x7fc424157d40 2024-04-25T09:00:27.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 -- 172.21.15.37:0/2671899917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4240c3610 con 0x7fc424157d40 2024-04-25T09:00:27.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc41bfff640 1 -- 172.21.15.37:0/2671899917 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c005030 con 0x7fc424157d40 2024-04-25T09:00:27.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.775+0000 7fc4295a9640 1 -- 172.21.15.37:0/2671899917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4240711b0 con 0x7fc424157d40 2024-04-25T09:00:27.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc41bfff640 1 -- 172.21.15.37:0/2671899917 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc41c0051d0 con 0x7fc424157d40 2024-04-25T09:00:27.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc41bfff640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc404066f60 0x7fc404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc41bfff640 1 -- 172.21.15.37:0/2671899917 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc41c11eb50 con 0x7fc424157d40 2024-04-25T09:00:27.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc422ffd640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc404066f60 0x7fc404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc42c716000 1 Processor -- start 2024-04-25T09:00:27.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc42c716000 1 -- start start 2024-04-25T09:00:27.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc422ffd640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc404066f60 0x7fc404069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc414000c10 tx=0x7fc414009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc42c716000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0a2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc42c716000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab0a3060 0x5584ab0587b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc42c716000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab058cf0 0x5584ab0590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc42c716000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584aaf3bba0 con 0x5584ab0a2740 2024-04-25T09:00:27.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc42c716000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584aaea1090 con 0x5584ab0a3060 2024-04-25T09:00:27.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc42c716000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584aae29160 con 0x5584ab058cf0 2024-04-25T09:00:27.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc422ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0a2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc422ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0a2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57664/0 (socket says 172.21.15.37:57664) 2024-04-25T09:00:27.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc422ffd640 1 -- 172.21.15.37:0/3457419060 learned_addr learned my addr 172.21.15.37:0/3457419060 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/3457419060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab0a3060 0x5584ab0587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.779+0000 7fc4237fe640 1 --2- 172.21.15.37:0/3457419060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab058cf0 0x5584ab0590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc422ffd640 1 -- 172.21.15.37:0/3457419060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab058cf0 msgr2=0x5584ab0590d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc422ffd640 1 --2- 172.21.15.37:0/3457419060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab058cf0 0x5584ab0590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc422ffd640 1 -- 172.21.15.37:0/3457419060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab0a3060 msgr2=0x5584ab0587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc422ffd640 1 --2- 172.21.15.37:0/3457419060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab0a3060 0x5584ab0587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc422ffd640 1 -- 172.21.15.37:0/3457419060 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584aaf4ac20 con 0x5584ab0a2740 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/3457419060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab0a3060 0x5584ab0587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc422ffd640 1 --2- 172.21.15.37:0/3457419060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0a2b20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc414007390 tx=0x7fc414014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d74387ebc831aa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc418ff9640 1 -- 172.21.15.37:0/3457419060 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc414028070 con 0x5584ab0a2740 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc418ff9640 1 -- 172.21.15.37:0/3457419060 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc414015c00 con 0x5584ab0a2740 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc418ff9640 1 -- 172.21.15.37:0/3457419060 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4140143d0 con 0x5584ab0a2740 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 -- 172.21.15.37:0/3457419060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 msgr2=0x5584ab0a2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 --2- 172.21.15.37:0/3457419060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0a2b20 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc414007390 tx=0x7fc414014000 comp rx=0 tx=0).stop 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 -- 172.21.15.37:0/3457419060 shutdown_connections 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 --2- 172.21.15.37:0/3457419060 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab058cf0 0x5584ab0590d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 --2- 172.21.15.37:0/3457419060 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab0a3060 0x5584ab0587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 --2- 172.21.15.37:0/3457419060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0a2b20 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 -- 172.21.15.37:0/3457419060 >> 172.21.15.37:0/3457419060 conn(0x5584ab050890 msgr2=0x5584ab052c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 -- 172.21.15.37:0/3457419060 shutdown_connections 2024-04-25T09:00:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 -- 172.21.15.37:0/3457419060 wait complete. 2024-04-25T09:00:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 -- start start 2024-04-25T09:00:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 Processor -- start 2024-04-25T09:00:27.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab058cf0 0x5584ab0a21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab063c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab0a3060 0x5584ab0641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584aae9d130 con 0x5584ab0a2740 2024-04-25T09:00:27.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584aae65130 con 0x5584ab0a3060 2024-04-25T09:00:27.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584aaf5f300 con 0x5584ab058cf0 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 10 client.0 _ll_register_callbacks cb 0x5584ab03fbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc41a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab063c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc41a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab063c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57672/0 (socket says 172.21.15.37:57672) 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc41a7fc640 1 -- 172.21.15.37:0/1609982105 learned_addr learned my addr 172.21.15.37:0/1609982105 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc41a7fc640 1 -- 172.21.15.37:0/1609982105 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab058cf0 msgr2=0x5584ab0a21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab058cf0 0x5584ab0a21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc4237fe640 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab0a3060 0x5584ab0641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc41a7fc640 1 -- 172.21.15.37:0/1609982105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab0a3060 msgr2=0x5584ab0641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab0a3060 0x5584ab0641a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc41a7fc640 1 -- 172.21.15.37:0/1609982105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584aae7e060 con 0x5584ab0a2740 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab063c60 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0135c0 tx=0x7fc40c013a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4b5750997718c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc401ffb640 1 -- 172.21.15.37:0/1609982105 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c0158f0 con 0x5584ab0a2740 2024-04-25T09:00:27.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc401ffb640 1 -- 172.21.15.37:0/1609982105 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc40c015a90 con 0x5584ab0a2740 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 -- 172.21.15.37:0/1609982105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc414015c00 con 0x5584ab0a2740 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc401ffb640 1 -- 172.21.15.37:0/1609982105 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc40c027050 con 0x5584ab0a2740 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 10 client.8241 fetch_fsmap learned FSMap version 67 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 1 -- 172.21.15.37:0/1609982105 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5584aae427c0 con 0x5584ab0a2740 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc401ffb640 1 -- 172.21.15.37:0/1609982105 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc414015c00 con 0x5584ab0a2740 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc401ffb640 1 -- 172.21.15.37:0/1609982105 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc40c02a020 con 0x5584ab0a2740 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.783+0000 7fc42c716000 10 client.8241 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 10 client.8241 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 20 client.8241 populate_metadata read hostname 'smithi037' 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5584ab064ee0 0x5584ab0b47f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 4 client.8241 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 -- 172.21.15.37:0/1609982105 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584aae3ccd0 con 0x5584ab064ee0 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc422ffd640 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5584ab064ee0 0x5584ab0b47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc422ffd640 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5584ab064ee0 0x5584ab0b47f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40d897b6943f875e server_cookie=e7feb9ffe8d3841b in_seq=0 out_seq=0 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc401ffb640 10 client.8241 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T09:00:27.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc401ffb640 1 -- 172.21.15.37:0/1609982105 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584aae3ccd0 con 0x5584ab064ee0 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc401ffb640 10 client.8241 handle_command_reply: tid=0 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 client.8241 shutdown 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 -- 172.21.15.37:0/1609982105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 msgr2=0x5584ab063c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab063c60 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc40c0135c0 tx=0x7fc40c013a80 comp rx=0 tx=0).stop 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 -- 172.21.15.37:0/1609982105 shutdown_connections 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5584ab064ee0 0x5584ab0b47f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab0a3060 0x5584ab0641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc401ffb640 0 client.8241 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab063c60 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 --2- 172.21.15.37:0/1609982105 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab058cf0 0x5584ab0a21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 -- 172.21.15.37:0/1609982105 >> 172.21.15.37:0/1609982105 conn(0x5584ab050890 msgr2=0x5584ab059b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 -- 172.21.15.37:0/1609982105 shutdown_connections 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 1 -- 172.21.15.37:0/1609982105 wait complete. 2024-04-25T09:00:27.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.787+0000 7fc42c716000 20 client.8241 trim_cache size 0 max 16384 2024-04-25T09:00:27.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 Processor -- start 2024-04-25T09:00:27.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 -- start start 2024-04-25T09:00:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab058cf0 0x5584ab059e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0590d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab0a3060 0x5584ab059610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4241fba80 con 0x5584ab0a2740 2024-04-25T09:00:27.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240cee10 con 0x5584ab058cf0 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4241509f0 con 0x5584ab0a3060 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc41a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc41a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57674/0 (socket says 172.21.15.37:57674) 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc41a7fc640 1 -- 172.21.15.37:0/904428831 learned_addr learned my addr 172.21.15.37:0/904428831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc41a7fc640 1 -- 172.21.15.37:0/904428831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab0a3060 msgr2=0x5584ab059610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/904428831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab0a3060 0x5584ab059610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc41a7fc640 1 -- 172.21.15.37:0/904428831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab058cf0 msgr2=0x5584ab059e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/904428831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab058cf0 0x5584ab059e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc41a7fc640 1 -- 172.21.15.37:0/904428831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584ab060960 con 0x5584ab0a2740 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/904428831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0590d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009540 tx=0x7fc40c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c2b23b4d7ebc41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc403fff640 1 -- 172.21.15.37:0/904428831 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c02f070 con 0x5584ab0a2740 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc403fff640 1 -- 172.21.15.37:0/904428831 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c000e50 con 0x5584ab0a2740 2024-04-25T09:00:27.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc403fff640 1 -- 172.21.15.37:0/904428831 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c004040 con 0x5584ab0a2740 2024-04-25T09:00:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 -- 172.21.15.37:0/904428831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 msgr2=0x5584ab0590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 --2- 172.21.15.37:0/904428831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0590d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009540 tx=0x7fc40c018000 comp rx=0 tx=0).stop 2024-04-25T09:00:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 -- 172.21.15.37:0/904428831 shutdown_connections 2024-04-25T09:00:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 --2- 172.21.15.37:0/904428831 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab0a3060 0x5584ab059610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 --2- 172.21.15.37:0/904428831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a2740 0x5584ab0590d0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 --2- 172.21.15.37:0/904428831 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab058cf0 0x5584ab059e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 -- 172.21.15.37:0/904428831 >> 172.21.15.37:0/904428831 conn(0x5584ab050890 msgr2=0x5584ab0afd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 -- 172.21.15.37:0/904428831 shutdown_connections 2024-04-25T09:00:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.799+0000 7fc42c716000 1 -- 172.21.15.37:0/904428831 wait complete. 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- start start 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 Processor -- start 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab058cf0 0x5584ab102ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab0a2740 0x5584ab1031e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a3060 0x5584ab0b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584aae3be70 con 0x5584ab0a3060 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584aae29a50 con 0x5584ab058cf0 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584aaea03a0 con 0x5584ab0a2740 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 10 client.0 _ll_register_callbacks cb 0x5584aae4d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a3060 0x5584ab0b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a3060 0x5584ab0b4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57678/0 (socket says 172.21.15.37:57678) 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4237fe640 1 -- 172.21.15.37:0/1062203264 learned_addr learned my addr 172.21.15.37:0/1062203264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4237fe640 1 -- 172.21.15.37:0/1062203264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab0a2740 msgr2=0x5584ab1031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc41a7fc640 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab0a2740 0x5584ab1031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc422ffd640 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab058cf0 0x5584ab102ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4237fe640 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab0a2740 0x5584ab1031e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4237fe640 1 -- 172.21.15.37:0/1062203264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab058cf0 msgr2=0x5584ab102ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4237fe640 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab058cf0 0x5584ab102ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4237fe640 1 -- 172.21.15.37:0/1062203264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584aae5cfc0 con 0x5584ab0a3060 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4237fe640 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a3060 0x5584ab0b4b00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0eee70 tx=0x7fc41c0ef930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d01caf50d6213f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc401ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc401ffb640 1 -- 172.21.15.37:0/1062203264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41c10b060 con 0x5584ab0a3060 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- 172.21.15.37:0/1062203264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc40c000e50 con 0x5584ab0a3060 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc401ffb640 1 -- 172.21.15.37:0/1062203264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc41c0ed470 con 0x5584ab0a3060 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc401ffb640 1 -- 172.21.15.37:0/1062203264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc41c11b050 con 0x5584ab0a3060 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 10 client.8247 fetch_fsmap learned FSMap version 67 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- 172.21.15.37:0/1062203264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) v3 -- 0x5584aaefa9e0 con 0x5584ab0a3060 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc401ffb640 1 -- 172.21.15.37:0/1062203264 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c000e50 con 0x5584ab0a3060 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc401ffb640 1 -- 172.21.15.37:0/1062203264 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 67) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc41c0f73f0 con 0x5584ab0a3060 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 10 client.8247 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T09:00:27.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 10 client.8247 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 20 client.8247 populate_metadata read hostname 'smithi037' 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5584ab0b5400 0x5584ab0b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 4 client.8247 mds_command: new command op to 7998 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- 172.21.15.37:0/1062203264 --> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x5584aaeae5d0 con 0x5584ab0b5400 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc422ffd640 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5584ab0b5400 0x5584ab0b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc401ffb640 10 client.8247 ms_handle_connect on v2:172.21.15.37:6834/1462327322 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc422ffd640 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5584ab0b5400 0x5584ab0b57e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=136b9068a7fd8d47 server_cookie=4521d3a24b146056 in_seq=0 out_seq=0 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc401ffb640 1 -- 172.21.15.37:0/1062203264 <== mds.0 v2:172.21.15.37:6834/1462327322 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+144 (crc 0 0 0) 0x5584aaeae5d0 con 0x5584ab0b5400 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc401ffb640 10 client.8247 handle_command_reply: tid=0 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 client.8247 shutdown 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- 172.21.15.37:0/1062203264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a3060 msgr2=0x5584ab0b4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a3060 0x5584ab0b4b00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc41c0eee70 tx=0x7fc41c0ef930 comp rx=0 tx=0).stop 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- 172.21.15.37:0/1062203264 shutdown_connections 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.37:6834/1462327322,v1:172.21.15.37:6835/1462327322] conn(0x5584ab0b5400 0x5584ab0b57e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584ab0a3060 0x5584ab0b4b00 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5584ab0a2740 0x5584ab1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc401ffb640 0 client.8247 ms_handle_reset on v2:172.21.15.37:6834/1462327322 2024-04-25T09:00:27.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 --2- 172.21.15.37:0/1062203264 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5584ab058cf0 0x5584ab102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- 172.21.15.37:0/1062203264 >> 172.21.15.37:0/1062203264 conn(0x5584ab050890 msgr2=0x5584ab052c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- 172.21.15.37:0/1062203264 shutdown_connections 2024-04-25T09:00:27.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 1 -- 172.21.15.37:0/1062203264 wait complete. 2024-04-25T09:00:27.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc42c716000 20 client.8247 trim_cache size 0 max 16384 2024-04-25T09:00:27.802 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:00:27.802 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T09:00:27.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:00:27.802 INFO:teuthology.orchestra.run.smithi037.stdout: "damage_type": "backtrace", 2024-04-25T09:00:27.803 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 2722350428, 2024-04-25T09:00:27.803 INFO:teuthology.orchestra.run.smithi037.stdout: "ino": 1099511627777, 2024-04-25T09:00:27.803 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/testdir/hardlink" 2024-04-25T09:00:27.803 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:00:27.803 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4295a9640 1 -- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc404066f60 msgr2=0x7fc404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4295a9640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc404066f60 0x7fc404069420 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc414000c10 tx=0x7fc414009040 comp rx=0 tx=0).stop 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4295a9640 1 -- 172.21.15.37:0/2671899917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc424157d40 msgr2=0x7fc4241591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.803+0000 7fc4295a9640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc424157d40 0x7fc4241591f0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fc41c01c570 tx=0x7fc410005840 comp rx=0 tx=0).stop 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.807+0000 7fc4295a9640 1 -- 172.21.15.37:0/2671899917 shutdown_connections 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.807+0000 7fc4295a9640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc404066f60 0x7fc404069420 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.807+0000 7fc4295a9640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc424157d40 0x7fc4241591f0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.807+0000 7fc4295a9640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc424072af0 0x7fc424158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.807+0000 7fc4295a9640 1 --2- 172.21.15.37:0/2671899917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc410004d50 0x7fc4240c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.807+0000 7fc4295a9640 1 -- 172.21.15.37:0/2671899917 >> 172.21.15.37:0/2671899917 conn(0x7fc42406ae60 msgr2=0x7fc4240723e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.807+0000 7fc4295a9640 1 -- 172.21.15.37:0/2671899917 shutdown_connections 2024-04-25T09:00:27.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.807+0000 7fc4295a9640 1 -- 172.21.15.37:0/2671899917 wait complete. 2024-04-25T09:00:27.813 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:00:27.813 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:27.819 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f6980f1d640 1 Processor -- start 2024-04-25T09:00:27.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f6980f1d640 1 -- start start 2024-04-25T09:00:27.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f6980f1d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f697c0ce510 0x7f697c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f6980f1d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c0cee30 con 0x7f697c144ca0 2024-04-25T09:00:27.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f6980f1d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c0ccc70 con 0x7f697c1451f0 2024-04-25T09:00:27.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f6980f1d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697c0cc2b0 con 0x7f697c0ce510 2024-04-25T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f6979d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f697c144ca0 0x7f697c06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32790/0 (socket says 172.21.15.37:32790) 2024-04-25T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f6979d9b640 1 -- 172.21.15.37:0/1124980693 learned_addr learned my addr 172.21.15.37:0/1124980693 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637361207 0 0) 0x7f697c0cee30 con 0x7f697c144ca0 2024-04-25T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697a59c640 1 --2- 172.21.15.37:0/1124980693 >> v2:172.21.15.203:3300/0 conn(0x7f697c0ce510 0x7f697c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6964003530 con 0x7f697c144ca0 2024-04-25T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1303733374 0 0) 0x7f6964003530 con 0x7f697c144ca0 2024-04-25T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 >> v2:172.21.15.203:3300/0 conn(0x7f697c0ce510 msgr2=0x7f697c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 --2- 172.21.15.37:0/1124980693 >> v2:172.21.15.203:3300/0 conn(0x7f697c0ce510 0x7f697c0ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 >> v1:172.21.15.203:6789/0 conn(0x7f697c1451f0 legacy=0x7f697c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c06dc40 con 0x7f697c144ca0 2024-04-25T09:00:27.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f6970001680 con 0x7f697c144ca0 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 >> v1:172.21.15.37:6789/0 conn(0x7f697c144ca0 legacy=0x7f697c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6964003a80 0x7f6964003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6964004810 0x7f6964004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6964005220 0x7f696400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0cc2b0 con 0x7f6964005220 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696400c130 con 0x7f6964003a80 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f696400c330 con 0x7f6964004810 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697ad9d640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6964005220 0x7f696400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697a59c640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6964003a80 0x7f6964003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f6979d9b640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6964004810 0x7f6964004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697ad9d640 1 -- 172.21.15.37:0/1124980693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6964004810 msgr2=0x7f6964004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697ad9d640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6964004810 0x7f6964004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697ad9d640 1 -- 172.21.15.37:0/1124980693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6964003a80 msgr2=0x7f6964003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697ad9d640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6964003a80 0x7f6964003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697ad9d640 1 -- 172.21.15.37:0/1124980693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6964003530 con 0x7f6964005220 2024-04-25T09:00:27.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.979+0000 7f697a59c640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6964003a80 0x7f6964003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f697ad9d640 1 -- 172.21.15.37:0/1124980693 reap_dead start 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f697ad9d640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6964005220 0x7f696400b9d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6974001e40 tx=0x7f697401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1893583380ba343f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974001a90 con 0x7f6964005220 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974003b80 con 0x7f6964005220 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f697959a640 1 -- 172.21.15.37:0/1124980693 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974005020 con 0x7f6964005220 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f6980f1d640 1 -- 172.21.15.37:0/1124980693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6964005220 msgr2=0x7f696400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f6980f1d640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6964005220 0x7f696400b9d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f6974001e40 tx=0x7f697401c760 comp rx=0 tx=0).stop 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f6980f1d640 1 -- 172.21.15.37:0/1124980693 shutdown_connections 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f6980f1d640 1 --2- 172.21.15.37:0/1124980693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6964005220 0x7f696400b9d0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f6980f1d640 1 -- 172.21.15.37:0/1124980693 >> 172.21.15.37:0/1124980693 conn(0x7f697c0b11a0 msgr2=0x7f697c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f6980f1d640 1 -- 172.21.15.37:0/1124980693 shutdown_connections 2024-04-25T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f6980f1d640 1 -- 172.21.15.37:0/1124980693 wait complete. 2024-04-25T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f6980f1d640 1 Processor -- start 2024-04-25T09:00:27.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.983+0000 7f6980f1d640 1 -- start start 2024-04-25T09:00:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6980f1d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6964005220 0x7f697c15b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6980f1d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f697c1314e0 0x7f697c0add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6980f1d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f697c0b3fa0 0x7f697c0ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6980f1d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0cc900 con 0x7f697c1314e0 2024-04-25T09:00:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6980f1d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c06e290 con 0x7f697c0b3fa0 2024-04-25T09:00:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6980f1d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0cca80 con 0x7f6964005220 2024-04-25T09:00:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6979d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f697c1314e0 0x7f697c0add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6979d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f697c1314e0 0x7f697c0add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57688/0 (socket says 172.21.15.37:57688) 2024-04-25T09:00:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6979d9b640 1 -- 172.21.15.37:0/3620224110 learned_addr learned my addr 172.21.15.37:0/3620224110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f697ad9d640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f697c0b3fa0 0x7f697c0ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f697a59c640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6964005220 0x7f697c15b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6979d9b640 1 -- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6964005220 msgr2=0x7f697c15b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6979d9b640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6964005220 0x7f697c15b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6979d9b640 1 -- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f697c0b3fa0 msgr2=0x7f697c0ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6979d9b640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f697c0b3fa0 0x7f697c0ae240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6979d9b640 1 -- 172.21.15.37:0/3620224110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c1319f0 con 0x7f697c1314e0 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f697ad9d640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f697c0b3fa0 0x7f697c0ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6979d9b640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f697c1314e0 0x7f697c0add00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f697000d880 tx=0x7f697000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f696affd640 1 -- 172.21.15.37:0/3620224110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970019070 con 0x7f697c1314e0 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f696affd640 1 -- 172.21.15.37:0/3620224110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6970007e50 con 0x7f697c1314e0 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6980f1d640 1 -- 172.21.15.37:0/3620224110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c131cd0 con 0x7f697c1314e0 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f696affd640 1 -- 172.21.15.37:0/3620224110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700047f0 con 0x7f697c1314e0 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6980f1d640 1 -- 172.21.15.37:0/3620224110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c132290 con 0x7f697c1314e0 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f6980f1d640 1 -- 172.21.15.37:0/3620224110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c04efa0 con 0x7f697c1314e0 2024-04-25T09:00:27.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f696affd640 1 -- 172.21.15.37:0/3620224110 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f697001c070 con 0x7f697c1314e0 2024-04-25T09:00:27.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f696affd640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f695c066f90 0x7f695c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f696affd640 1 -- 172.21.15.37:0/3620224110 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6970084c00 con 0x7f697c1314e0 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.987+0000 7f697a59c640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f695c066f90 0x7f695c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.991+0000 7f697a59c640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f695c066f90 0x7f695c069450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f696c009830 tx=0x7f696c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:27.991+0000 7f696affd640 1 -- 172.21.15.37:0/3620224110 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f69700559b0 con 0x7f697c1314e0 2024-04-25T09:00:28.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.151+0000 7f6980f1d640 1 -- 172.21.15.37:0/3620224110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f697c0aeeb0 con 0x7f697c1314e0 2024-04-25T09:00:28.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.151+0000 7f696affd640 1 -- 172.21.15.37:0/3620224110 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+17169 (secure 0 0 0) 0x7f69700552b0 con 0x7f697c1314e0 2024-04-25T09:00:28.145 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:00:28.145 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":98,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:59:17.095374+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:50.512578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:51.848880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/354285428":"2024-04-26T08:59:17.095322+0000","172.21.15.37:6834/354285428":"2024-04-26T08:59:17.095322+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 -- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f695c066f90 msgr2=0x7f695c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f695c066f90 0x7f695c069450 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f696c009830 tx=0x7f696c008040 comp rx=0 tx=0).stop 2024-04-25T09:00:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 -- 172.21.15.37:0/3620224110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f697c1314e0 msgr2=0x7f697c0add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f697c1314e0 0x7f697c0add00 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f697000d880 tx=0x7f697000dd40 comp rx=0 tx=0).stop 2024-04-25T09:00:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 -- 172.21.15.37:0/3620224110 shutdown_connections 2024-04-25T09:00:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f695c066f90 0x7f695c069450 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f697c0b3fa0 0x7f697c0ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f697c1314e0 0x7f697c0add00 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 --2- 172.21.15.37:0/3620224110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6964005220 0x7f697c15b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 -- 172.21.15.37:0/3620224110 >> 172.21.15.37:0/3620224110 conn(0x7f697c0b11a0 msgr2=0x7f697c071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 -- 172.21.15.37:0/3620224110 shutdown_connections 2024-04-25T09:00:28.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.155+0000 7f6980f1d640 1 -- 172.21.15.37:0/3620224110 wait complete. 2024-04-25T09:00:28.163 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:58:50.512578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:58:51.848880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:00:28.163 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:28.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f13a4a1e640 1 Processor -- start 2024-04-25T09:00:28.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f13a4a1e640 1 -- start start 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f13a4a1e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f13a00cd350 0x7f13a015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f13a4a1e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a00ce320 con 0x7f13a00cf0e0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f13a4a1e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a00ce600 con 0x7f13a015aac0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f13a4a1e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a00ce0d0 con 0x7f13a00cd350 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139ed9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f13a00cf0e0 0x7f13a015b3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32804/0 (socket says 172.21.15.37:32804) 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139ed9d640 1 -- 172.21.15.37:0/1256862429 learned_addr learned my addr 172.21.15.37:0/1256862429 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139dd9b640 1 --2- 172.21.15.37:0/1256862429 >> v2:172.21.15.203:3300/0 conn(0x7f13a00cd350 0x7f13a015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663130938 0 0) 0x7f13a00ce320 con 0x7f13a00cf0e0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1388003530 con 0x7f13a00cf0e0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1658248241 0 0) 0x7f1388003530 con 0x7f13a00cf0e0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 >> v2:172.21.15.203:3300/0 conn(0x7f13a00cd350 msgr2=0x7f13a015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 --2- 172.21.15.37:0/1256862429 >> v2:172.21.15.203:3300/0 conn(0x7f13a00cd350 0x7f13a015aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 >> v1:172.21.15.203:6789/0 conn(0x7f13a015aac0 legacy=0x7f13a00ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a00ce8e0 con 0x7f13a00cf0e0 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1390001680 con 0x7f13a00cf0e0 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 >> v1:172.21.15.37:6789/0 conn(0x7f13a00cf0e0 legacy=0x7f13a015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 --2- 172.21.15.37:0/1256862429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1388003a30 0x7f1388003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 --2- 172.21.15.37:0/1256862429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1388004810 0x7f1388004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 --2- 172.21.15.37:0/1256862429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13880051d0 0x7f138800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a00ce0d0 con 0x7f1388003a30 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138800c100 con 0x7f13880051d0 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138800c300 con 0x7f1388004810 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139e59c640 1 --2- 172.21.15.37:0/1256862429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1388003a30 0x7f1388003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139ed9d640 1 --2- 172.21.15.37:0/1256862429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13880051d0 0x7f138800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.299+0000 7f139dd9b640 1 --2- 172.21.15.37:0/1256862429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1388004810 0x7f1388004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f139e59c640 1 -- 172.21.15.37:0/1256862429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1388004810 msgr2=0x7f1388004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f139e59c640 1 --2- 172.21.15.37:0/1256862429 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1388004810 0x7f1388004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f139e59c640 1 -- 172.21.15.37:0/1256862429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13880051d0 msgr2=0x7f138800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f139e59c640 1 --2- 172.21.15.37:0/1256862429 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13880051d0 0x7f138800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f139ed9d640 1 -- 172.21.15.37:0/1256862429 reap_dead start 2024-04-25T09:00:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f139e59c640 1 -- 172.21.15.37:0/1256862429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1388003530 con 0x7f1388003a30 2024-04-25T09:00:28.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f139e59c640 1 --2- 172.21.15.37:0/1256862429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1388003a30 0x7f1388003e50 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f1398001e70 tx=0x7f139801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5a7ace5500e18e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13980af070 con 0x7f1388003a30 2024-04-25T09:00:28.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398003b80 con 0x7f1388003a30 2024-04-25T09:00:28.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f139d59a640 1 -- 172.21.15.37:0/1256862429 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398001a90 con 0x7f1388003a30 2024-04-25T09:00:28.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f13a4a1e640 1 -- 172.21.15.37:0/1256862429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1388003a30 msgr2=0x7f1388003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f13a4a1e640 1 --2- 172.21.15.37:0/1256862429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1388003a30 0x7f1388003e50 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f1398001e70 tx=0x7f139801c760 comp rx=0 tx=0).stop 2024-04-25T09:00:28.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f13a4a1e640 1 -- 172.21.15.37:0/1256862429 shutdown_connections 2024-04-25T09:00:28.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f13a4a1e640 1 --2- 172.21.15.37:0/1256862429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1388003a30 0x7f1388003e50 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f13a4a1e640 1 -- 172.21.15.37:0/1256862429 >> 172.21.15.37:0/1256862429 conn(0x7f13a0099a70 msgr2=0x7f13a00d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f13a4a1e640 1 -- 172.21.15.37:0/1256862429 shutdown_connections 2024-04-25T09:00:28.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.303+0000 7f13a4a1e640 1 -- 172.21.15.37:0/1256862429 wait complete. 2024-04-25T09:00:28.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f13a4a1e640 1 Processor -- start 2024-04-25T09:00:28.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f13a4a1e640 1 -- start start 2024-04-25T09:00:28.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f13a4a1e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1388003a30 0x7f13a01498b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f13a4a1e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a009dcd0 0x7f13a015e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f13a4a1e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a009e150 0x7f13a015eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f13a4a1e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a015ba40 con 0x7f13a009e150 2024-04-25T09:00:28.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f13a4a1e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a015b8c0 con 0x7f13a009dcd0 2024-04-25T09:00:28.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f13a4a1e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a015e270 con 0x7f1388003a30 2024-04-25T09:00:28.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f139ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a009e150 0x7f13a015eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f139ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a009e150 0x7f13a015eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57708/0 (socket says 172.21.15.37:57708) 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f139ed9d640 1 -- 172.21.15.37:0/2808304172 learned_addr learned my addr 172.21.15.37:0/2808304172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f139e59c640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1388003a30 0x7f13a01498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f139dd9b640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a009dcd0 0x7f13a015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f139ed9d640 1 -- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1388003a30 msgr2=0x7f13a01498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f139ed9d640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1388003a30 0x7f13a01498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f139ed9d640 1 -- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a009dcd0 msgr2=0x7f13a015e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f139ed9d640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a009dcd0 0x7f13a015e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f139ed9d640 1 -- 172.21.15.37:0/2808304172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a01564e0 con 0x7f13a009e150 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f139ed9d640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a009e150 0x7f13a015eb40 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f13a006df50 tx=0x7f139000e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f138effd640 1 -- 172.21.15.37:0/2808304172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1390019070 con 0x7f13a009e150 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f138effd640 1 -- 172.21.15.37:0/2808304172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1390010510 con 0x7f13a009e150 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f13a4a1e640 1 -- 172.21.15.37:0/2808304172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a014cdc0 con 0x7f13a009e150 2024-04-25T09:00:28.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f138effd640 1 -- 172.21.15.37:0/2808304172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13900108e0 con 0x7f13a009e150 2024-04-25T09:00:28.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f13a4a1e640 1 -- 172.21.15.37:0/2808304172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a009d1f0 con 0x7f13a009e150 2024-04-25T09:00:28.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f13a4a1e640 1 -- 172.21.15.37:0/2808304172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a004efa0 con 0x7f13a009e150 2024-04-25T09:00:28.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f138effd640 1 -- 172.21.15.37:0/2808304172 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1390010a80 con 0x7f13a009e150 2024-04-25T09:00:28.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f138effd640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1380066f90 0x7f1380069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.307+0000 7f138effd640 1 -- 172.21.15.37:0/2808304172 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1390084850 con 0x7f13a009e150 2024-04-25T09:00:28.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.311+0000 7f139e59c640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1380066f90 0x7f1380069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.311+0000 7f139e59c640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1380066f90 0x7f1380069450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f13980ac000 tx=0x7f1398001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.311+0000 7f138effd640 1 -- 172.21.15.37:0/2808304172 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f13a01564e0 con 0x7f13a009e150 2024-04-25T09:00:28.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.475+0000 7f13a4a1e640 1 -- 172.21.15.37:0/2808304172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13a00b6480 con 0x7f13a009e150 2024-04-25T09:00:28.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f138effd640 1 -- 172.21.15.37:0/2808304172 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1390055610 con 0x7f13a009e150 2024-04-25T09:00:28.470 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:00:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 -- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1380066f90 msgr2=0x7f1380069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1380066f90 0x7f1380069450 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f13980ac000 tx=0x7f1398001520 comp rx=0 tx=0).stop 2024-04-25T09:00:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 -- 172.21.15.37:0/2808304172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a009e150 msgr2=0x7f13a015eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a009e150 0x7f13a015eb40 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f13a006df50 tx=0x7f139000e9e0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 -- 172.21.15.37:0/2808304172 shutdown_connections 2024-04-25T09:00:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1380066f90 0x7f1380069450 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13a009e150 0x7f13a015eb40 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f13a009dcd0 0x7f13a015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 --2- 172.21.15.37:0/2808304172 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1388003a30 0x7f13a01498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 -- 172.21.15.37:0/2808304172 >> 172.21.15.37:0/2808304172 conn(0x7f13a0099a70 msgr2=0x7f13a006f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 -- 172.21.15.37:0/2808304172 shutdown_connections 2024-04-25T09:00:28.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.479+0000 7f13a4a1e640 1 -- 172.21.15.37:0/2808304172 wait complete. 2024-04-25T09:00:28.489 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:28.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0f4f4640 1 Processor -- start 2024-04-25T09:00:28.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0f4f4640 1 -- start start 2024-04-25T09:00:28.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0f4f4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c0815d9c0 0x7f1c080d1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0f4f4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1c080cf270 0x7f1c080d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0f4f4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c080d1950 con 0x7f1c080d3c20 2024-04-25T09:00:28.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0f4f4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c080d2520 con 0x7f1c080cf270 2024-04-25T09:00:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0f4f4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c080d1bf0 con 0x7f1c0815d9c0 2024-04-25T09:00:28.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0d290640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c0815d9c0 0x7f1c080d1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0ca8f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1c080cf270 0x7f1c080d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0d290640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c0815d9c0 0x7f1c080d1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:53350/0 (socket says 172.21.15.37:53350) 2024-04-25T09:00:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0d290640 1 -- 172.21.15.37:0/1252612587 learned_addr learned my addr 172.21.15.37:0/1252612587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0da91640 1 --1- 172.21.15.37:0/1252612587 >> v1:172.21.15.203:6789/0 conn(0x7f1c080d3c20 0x7f1c080d0120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:34724/0 (socket says 172.21.15.37:34724) 2024-04-25T09:00:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0ca8f640 1 -- 172.21.15.37:0/1252612587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c0815d9c0 msgr2=0x7f1c080d1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0ca8f640 1 --2- 172.21.15.37:0/1252612587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c0815d9c0 0x7f1c080d1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0ca8f640 1 -- 172.21.15.37:0/1252612587 >> v1:172.21.15.203:6789/0 conn(0x7f1c080d3c20 legacy=0x7f1c080d0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0ca8f640 1 -- 172.21.15.37:0/1252612587 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c080d1ed0 con 0x7f1c080cf270 2024-04-25T09:00:28.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.623+0000 7f1c0d290640 1 --2- 172.21.15.37:0/1252612587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c0815d9c0 0x7f1c080d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:28.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1c0ca8f640 1 --2- 172.21.15.37:0/1252612587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c080cf270 0x7f1c080d36e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80097a0 tx=0x7f1bf801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d60ff8f00d4f97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1bfffff640 1 -- 172.21.15.37:0/1252612587 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80b1070 con 0x7f1c080cf270 2024-04-25T09:00:28.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1bfffff640 1 -- 172.21.15.37:0/1252612587 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf80ac640 con 0x7f1c080cf270 2024-04-25T09:00:28.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1bfffff640 1 -- 172.21.15.37:0/1252612587 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80aca10 con 0x7f1c080cf270 2024-04-25T09:00:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1252612587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c080cf270 msgr2=0x7f1c080d36e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1c0f4f4640 1 --2- 172.21.15.37:0/1252612587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c080cf270 0x7f1c080d36e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1bf80097a0 tx=0x7f1bf801c480 comp rx=0 tx=0).stop 2024-04-25T09:00:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1252612587 shutdown_connections 2024-04-25T09:00:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1c0f4f4640 1 --2- 172.21.15.37:0/1252612587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c080cf270 0x7f1c080d36e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1c0f4f4640 1 --2- 172.21.15.37:0/1252612587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c0815d9c0 0x7f1c080d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1252612587 >> 172.21.15.37:0/1252612587 conn(0x7f1c0809cad0 msgr2=0x7f1c0808b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1252612587 shutdown_connections 2024-04-25T09:00:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.627+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1252612587 wait complete. 2024-04-25T09:00:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0f4f4640 1 Processor -- start 2024-04-25T09:00:28.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0f4f4640 1 -- start start 2024-04-25T09:00:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0f4f4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c080cf270 0x7f1c08153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0f4f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c080d3c20 0x7f1c080b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0f4f4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c0815d9c0 0x7f1c08154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0f4f4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0816a720 con 0x7f1c080d3c20 2024-04-25T09:00:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0f4f4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0816a5a0 con 0x7f1c0815d9c0 2024-04-25T09:00:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0f4f4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0816a8a0 con 0x7f1c080cf270 2024-04-25T09:00:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0ca8f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c080d3c20 0x7f1c080b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0ca8f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c080d3c20 0x7f1c080b6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57712/0 (socket says 172.21.15.37:57712) 2024-04-25T09:00:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0ca8f640 1 -- 172.21.15.37:0/1829778812 learned_addr learned my addr 172.21.15.37:0/1829778812 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0da91640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c0815d9c0 0x7f1c08154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0d290640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c080cf270 0x7f1c08153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0ca8f640 1 -- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c080cf270 msgr2=0x7f1c08153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0ca8f640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c080cf270 0x7f1c08153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0ca8f640 1 -- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c0815d9c0 msgr2=0x7f1c08154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0ca8f640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c0815d9c0 0x7f1c08154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0ca8f640 1 -- 172.21.15.37:0/1829778812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c08153f60 con 0x7f1c080d3c20 2024-04-25T09:00:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0ca8f640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c080d3c20 0x7f1c080b6eb0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f1bf801a0b0 tx=0x7f1bf8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1bfdffb640 1 -- 172.21.15.37:0/1829778812 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80b9020 con 0x7f1c080d3c20 2024-04-25T09:00:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1bfdffb640 1 -- 172.21.15.37:0/1829778812 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf80b5410 con 0x7f1c080d3c20 2024-04-25T09:00:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1bfdffb640 1 -- 172.21.15.37:0/1829778812 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf80b1040 con 0x7f1c080d3c20 2024-04-25T09:00:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1829778812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c08154240 con 0x7f1c080d3c20 2024-04-25T09:00:28.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1829778812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c080b90f0 con 0x7f1c080d3c20 2024-04-25T09:00:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1829778812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0804efa0 con 0x7f1c080d3c20 2024-04-25T09:00:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.631+0000 7f1bfdffb640 1 -- 172.21.15.37:0/1829778812 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1bf80bc020 con 0x7f1c080d3c20 2024-04-25T09:00:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.635+0000 7f1bfdffb640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bec066f60 0x7f1bec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.635+0000 7f1bfdffb640 1 -- 172.21.15.37:0/1829778812 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1bf811ee30 con 0x7f1c080d3c20 2024-04-25T09:00:28.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.635+0000 7f1c0d290640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bec066f60 0x7f1bec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.635+0000 7f1c0d290640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bec066f60 0x7f1bec069420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1c08157bd0 tx=0x7f1c0000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.639+0000 7f1bfdffb640 1 -- 172.21.15.37:0/1829778812 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1bf80c3030 con 0x7f1c080d3c20 2024-04-25T09:00:28.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.791+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1829778812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c0815e440 con 0x7f1c080d3c20 2024-04-25T09:00:28.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.791+0000 7f1bfdffb640 1 -- 172.21.15.37:0/1829778812 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1bf8016070 con 0x7f1c080d3c20 2024-04-25T09:00:28.783 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:00:28.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.791+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bec066f60 msgr2=0x7f1bec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.791+0000 7f1c0f4f4640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bec066f60 0x7f1bec069420 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1c08157bd0 tx=0x7f1c0000b040 comp rx=0 tx=0).stop 2024-04-25T09:00:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.791+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1829778812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c080d3c20 msgr2=0x7f1c080b6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.791+0000 7f1c0f4f4640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c080d3c20 0x7f1c080b6eb0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f1bf801a0b0 tx=0x7f1bf8019220 comp rx=0 tx=0).stop 2024-04-25T09:00:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.791+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1829778812 shutdown_connections 2024-04-25T09:00:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.791+0000 7f1c0f4f4640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bec066f60 0x7f1bec069420 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.795+0000 7f1c0f4f4640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c0815d9c0 0x7f1c08154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.795+0000 7f1c0f4f4640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c080d3c20 0x7f1c080b6eb0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.795+0000 7f1c0f4f4640 1 --2- 172.21.15.37:0/1829778812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c080cf270 0x7f1c08153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.795+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1829778812 >> 172.21.15.37:0/1829778812 conn(0x7f1c0809cad0 msgr2=0x7f1c0808b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.795+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1829778812 shutdown_connections 2024-04-25T09:00:28.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.795+0000 7f1c0f4f4640 1 -- 172.21.15.37:0/1829778812 wait complete. 2024-04-25T09:00:28.801 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:28.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f52570eb640 1 Processor -- start 2024-04-25T09:00:28.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f52570eb640 1 -- start start 2024-04-25T09:00:28.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f52570eb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f525006ac90 0x7f525006b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f52570eb640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f525006b5e0 con 0x7f525006f610 2024-04-25T09:00:28.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f52570eb640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52500701b0 con 0x7f52500afb60 2024-04-25T09:00:28.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f52570eb640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f525006fbf0 con 0x7f525006ac90 2024-04-25T09:00:28.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f5255688640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f525006ac90 0x7f525006b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f5255688640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f525006ac90 0x7f525006b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57724/0 (socket says 172.21.15.37:57724) 2024-04-25T09:00:28.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f524ffff640 1 --1- 172.21.15.37:0/1876292076 >> v1:172.21.15.37:6789/0 conn(0x7f525006f610 0x7f52500cd840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:32810/0 (socket says 172.21.15.37:32810) 2024-04-25T09:00:28.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f5255688640 1 -- 172.21.15.37:0/1876292076 learned_addr learned my addr 172.21.15.37:0/1876292076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:28.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f5255688640 1 -- 172.21.15.37:0/1876292076 >> v1:172.21.15.203:6789/0 conn(0x7f52500afb60 legacy=0x7f52500cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f5255688640 1 -- 172.21.15.37:0/1876292076 >> v1:172.21.15.37:6789/0 conn(0x7f525006f610 legacy=0x7f52500cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.935+0000 7f5255688640 1 -- 172.21.15.37:0/1876292076 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525006fed0 con 0x7f525006ac90 2024-04-25T09:00:28.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f5255688640 1 --2- 172.21.15.37:0/1876292076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f525006ac90 0x7f525006b0a0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f5240009870 tx=0x7f524001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6315f282dfaa768 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f524f7fe640 1 -- 172.21.15.37:0/1876292076 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52400b0070 con 0x7f525006ac90 2024-04-25T09:00:28.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f524f7fe640 1 -- 172.21.15.37:0/1876292076 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52400ab600 con 0x7f525006ac90 2024-04-25T09:00:28.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f524f7fe640 1 -- 172.21.15.37:0/1876292076 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52400ab9d0 con 0x7f525006ac90 2024-04-25T09:00:28.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 -- 172.21.15.37:0/1876292076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f525006ac90 msgr2=0x7f525006b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 --2- 172.21.15.37:0/1876292076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f525006ac90 0x7f525006b0a0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f5240009870 tx=0x7f524001c410 comp rx=0 tx=0).stop 2024-04-25T09:00:28.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 -- 172.21.15.37:0/1876292076 shutdown_connections 2024-04-25T09:00:28.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 --2- 172.21.15.37:0/1876292076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f525006ac90 0x7f525006b0a0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 -- 172.21.15.37:0/1876292076 >> 172.21.15.37:0/1876292076 conn(0x7f5250099a70 msgr2=0x7f52500d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 -- 172.21.15.37:0/1876292076 shutdown_connections 2024-04-25T09:00:28.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 -- 172.21.15.37:0/1876292076 wait complete. 2024-04-25T09:00:28.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 Processor -- start 2024-04-25T09:00:28.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 -- start start 2024-04-25T09:00:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f525006ac90 0x7f525014e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f525006f610 0x7f52500b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52500afb60 0x7f525006e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525015e4a0 con 0x7f525006f610 2024-04-25T09:00:28.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525015e320 con 0x7f525006ac90 2024-04-25T09:00:28.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.939+0000 7f52570eb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525015e620 con 0x7f52500afb60 2024-04-25T09:00:28.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f525006f610 0x7f52500b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f525006f610 0x7f52500b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57730/0 (socket says 172.21.15.37:57730) 2024-04-25T09:00:28.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524ffff640 1 -- 172.21.15.37:0/132960490 learned_addr learned my addr 172.21.15.37:0/132960490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:28.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f5254e87640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f525006ac90 0x7f525014e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f5255688640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52500afb60 0x7f525006e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524ffff640 1 -- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52500afb60 msgr2=0x7f525006e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524ffff640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52500afb60 0x7f525006e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524ffff640 1 -- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f525006ac90 msgr2=0x7f525014e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524ffff640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f525006ac90 0x7f525014e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524ffff640 1 -- 172.21.15.37:0/132960490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525014ed10 con 0x7f525006f610 2024-04-25T09:00:28.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f5254e87640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f525006ac90 0x7f525014e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:28.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524ffff640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f525006f610 0x7f52500b3b00 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f523800c490 tx=0x7f523800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524d7fa640 1 -- 172.21.15.37:0/132960490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238018070 con 0x7f525006f610 2024-04-25T09:00:28.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524d7fa640 1 -- 172.21.15.37:0/132960490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5238007480 con 0x7f525006f610 2024-04-25T09:00:28.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524d7fa640 1 -- 172.21.15.37:0/132960490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238004a10 con 0x7f525006f610 2024-04-25T09:00:28.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f52570eb640 1 -- 172.21.15.37:0/132960490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52500b4c50 con 0x7f525006f610 2024-04-25T09:00:28.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f52570eb640 1 -- 172.21.15.37:0/132960490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52500b5210 con 0x7f525006f610 2024-04-25T09:00:28.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f52570eb640 1 -- 172.21.15.37:0/132960490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f525004efa0 con 0x7f525006f610 2024-04-25T09:00:28.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524d7fa640 1 -- 172.21.15.37:0/132960490 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f523801b070 con 0x7f525006f610 2024-04-25T09:00:28.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524d7fa640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5230066f60 0x7f5230069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f524d7fa640 1 -- 172.21.15.37:0/132960490 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5238084540 con 0x7f525006f610 2024-04-25T09:00:28.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f5254e87640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5230066f60 0x7f5230069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.943+0000 7f5254e87640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5230066f60 0x7f5230069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5248007fd0 tx=0x7f5248009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:28.947+0000 7f524d7fa640 1 -- 172.21.15.37:0/132960490 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f525006b5e0 con 0x7f525006f610 2024-04-25T09:00:29.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 -- 172.21.15.37:0/132960490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f525006b7c0 con 0x7f525006f610 2024-04-25T09:00:29.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f524d7fa640 1 -- 172.21.15.37:0/132960490 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5238055260 con 0x7f525006f610 2024-04-25T09:00:29.096 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:00:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 -- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5230066f60 msgr2=0x7f5230069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5230066f60 0x7f5230069420 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f5248007fd0 tx=0x7f5248009040 comp rx=0 tx=0).stop 2024-04-25T09:00:29.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 -- 172.21.15.37:0/132960490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f525006f610 msgr2=0x7f52500b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f525006f610 0x7f52500b3b00 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f523800c490 tx=0x7f523800c950 comp rx=0 tx=0).stop 2024-04-25T09:00:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 -- 172.21.15.37:0/132960490 shutdown_connections 2024-04-25T09:00:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5230066f60 0x7f5230069420 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52500afb60 0x7f525006e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f525006f610 0x7f52500b3b00 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 --2- 172.21.15.37:0/132960490 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f525006ac90 0x7f525014e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 -- 172.21.15.37:0/132960490 >> 172.21.15.37:0/132960490 conn(0x7f5250099a70 msgr2=0x7f52500aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 -- 172.21.15.37:0/132960490 shutdown_connections 2024-04-25T09:00:29.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.103+0000 7f52570eb640 1 -- 172.21.15.37:0/132960490 wait complete. 2024-04-25T09:00:29.114 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:00:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef11029640 1 Processor -- start 2024-04-25T09:00:29.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef11029640 1 -- start start 2024-04-25T09:00:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef11029640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fef0c06b440 0x7fef0c0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef11029640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fef0c0b1ac0 0x7fef0c0cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef11029640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0c0afb20 con 0x7fef0c06b820 2024-04-25T09:00:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef11029640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0c0b06c0 con 0x7fef0c06b440 2024-04-25T09:00:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef11029640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0c0afd90 con 0x7fef0c0b1ac0 2024-04-25T09:00:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef0ad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fef0c06b440 0x7fef0c0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef0ad9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fef0c06b440 0x7fef0c0b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57734/0 (socket says 172.21.15.37:57734) 2024-04-25T09:00:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef0ad9d640 1 -- 172.21.15.37:0/1833893511 learned_addr learned my addr 172.21.15.37:0/1833893511 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef0ad9d640 1 -- 172.21.15.37:0/1833893511 >> v2:172.21.15.203:3300/0 conn(0x7fef0c0b1ac0 msgr2=0x7fef0c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef0ad9d640 1 --2- 172.21.15.37:0/1833893511 >> v2:172.21.15.203:3300/0 conn(0x7fef0c0b1ac0 0x7fef0c0cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef0ad9d640 1 -- 172.21.15.37:0/1833893511 >> v1:172.21.15.203:6789/0 conn(0x7fef0c06b820 legacy=0x7fef0c0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:29.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.235+0000 7fef0ad9d640 1 -- 172.21.15.37:0/1833893511 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c0b0070 con 0x7fef0c06b440 2024-04-25T09:00:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef0ad9d640 1 --2- 172.21.15.37:0/1833893511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef0c06b440 0x7fef0c0b0b50 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fef0c073280 tx=0x7feef401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcd3da277de214cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef09d9b640 1 -- 172.21.15.37:0/1833893511 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef40ab5f0 con 0x7fef0c06b440 2024-04-25T09:00:29.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef09d9b640 1 -- 172.21.15.37:0/1833893511 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef40b0070 con 0x7fef0c06b440 2024-04-25T09:00:29.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef09d9b640 1 -- 172.21.15.37:0/1833893511 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef40ab9b0 con 0x7fef0c06b440 2024-04-25T09:00:29.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef11029640 1 -- 172.21.15.37:0/1833893511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef0c06b440 msgr2=0x7fef0c0b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef11029640 1 --2- 172.21.15.37:0/1833893511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef0c06b440 0x7fef0c0b0b50 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7fef0c073280 tx=0x7feef401c4e0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef11029640 1 -- 172.21.15.37:0/1833893511 shutdown_connections 2024-04-25T09:00:29.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef11029640 1 --2- 172.21.15.37:0/1833893511 >> v2:172.21.15.203:3300/0 conn(0x7fef0c0b1ac0 0x7fef0c0cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef11029640 1 --2- 172.21.15.37:0/1833893511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef0c06b440 0x7fef0c0b0b50 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef11029640 1 -- 172.21.15.37:0/1833893511 >> 172.21.15.37:0/1833893511 conn(0x7fef0c0b1f10 msgr2=0x7fef0c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef11029640 1 -- 172.21.15.37:0/1833893511 shutdown_connections 2024-04-25T09:00:29.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.239+0000 7fef11029640 1 -- 172.21.15.37:0/1833893511 wait complete. 2024-04-25T09:00:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef11029640 1 Processor -- start 2024-04-25T09:00:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef11029640 1 -- start start 2024-04-25T09:00:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef11029640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef0c06b440 0x7fef0c14e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef11029640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef0c06b820 0x7fef0c14ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef11029640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef0c0b1ac0 0x7fef0c0cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef11029640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0cd170 con 0x7fef0c06b440 2024-04-25T09:00:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef11029640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c0ccff0 con 0x7fef0c0b1ac0 2024-04-25T09:00:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef11029640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0c15d080 con 0x7fef0c06b820 2024-04-25T09:00:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef0c06b440 0x7fef0c14e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef0c06b440 0x7fef0c14e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57746/0 (socket says 172.21.15.37:57746) 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0ad9d640 1 -- 172.21.15.37:0/3735172146 learned_addr learned my addr 172.21.15.37:0/3735172146 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0b59e640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef0c0b1ac0 0x7fef0c0cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0a59c640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef0c06b820 0x7fef0c14ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0ad9d640 1 -- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef0c06b820 msgr2=0x7fef0c14ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0ad9d640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef0c06b820 0x7fef0c14ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0ad9d640 1 -- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef0c0b1ac0 msgr2=0x7fef0c0cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0ad9d640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef0c0b1ac0 0x7fef0c0cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0ad9d640 1 -- 172.21.15.37:0/3735172146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c14e9f0 con 0x7fef0c06b440 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0b59e640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef0c0b1ac0 0x7fef0c0cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef0ad9d640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef0c06b440 0x7fef0c14e4b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7feef401a110 tx=0x7feef401ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef037fe640 1 -- 172.21.15.37:0/3735172146 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef40be5f0 con 0x7fef0c06b440 2024-04-25T09:00:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef037fe640 1 -- 172.21.15.37:0/3735172146 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef40be790 con 0x7fef0c06b440 2024-04-25T09:00:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef11029640 1 -- 172.21.15.37:0/3735172146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef0c155d70 con 0x7fef0c06b440 2024-04-25T09:00:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef037fe640 1 -- 172.21.15.37:0/3735172146 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef40beb80 con 0x7fef0c06b440 2024-04-25T09:00:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef11029640 1 -- 172.21.15.37:0/3735172146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef0c0b3180 con 0x7fef0c06b440 2024-04-25T09:00:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef11029640 1 -- 172.21.15.37:0/3735172146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef0c04efa0 con 0x7fef0c06b440 2024-04-25T09:00:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef037fe640 1 -- 172.21.15.37:0/3735172146 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7feef40b43f0 con 0x7fef0c06b440 2024-04-25T09:00:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef037fe640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feeec066f60 0x7feeec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.243+0000 7fef037fe640 1 -- 172.21.15.37:0/3735172146 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7feef411fdd0 con 0x7fef0c06b440 2024-04-25T09:00:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.247+0000 7fef0a59c640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feeec066f60 0x7feeec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.247+0000 7fef0a59c640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feeec066f60 0x7feeec069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fef0c154d80 tx=0x7feefc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.247+0000 7fef037fe640 1 -- 172.21.15.37:0/3735172146 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7feef40c4060 con 0x7fef0c06b440 2024-04-25T09:00:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.419+0000 7fef11029640 1 -- 172.21.15.37:0/3735172146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fef0c0cc160 con 0x7fef0c06b440 2024-04-25T09:00:29.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef037fe640 1 -- 172.21.15.37:0/3735172146 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v98) v1 ==== 80+0+120 (secure 0 0 0) 0x7feef40160d0 con 0x7fef0c06b440 2024-04-25T09:00:29.414 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/354285428 2024-04-26T08:59:17.095322+0000 2024-04-25T09:00:29.414 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6834/354285428 2024-04-26T08:59:17.095322+0000 2024-04-25T09:00:29.414 INFO:teuthology.orchestra.run.smithi037.stderr:listed 2 entries 2024-04-25T09:00:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 -- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feeec066f60 msgr2=0x7feeec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feeec066f60 0x7feeec069420 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fef0c154d80 tx=0x7feefc009040 comp rx=0 tx=0).stop 2024-04-25T09:00:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 -- 172.21.15.37:0/3735172146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef0c06b440 msgr2=0x7fef0c14e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef0c06b440 0x7fef0c14e4b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7feef401a110 tx=0x7feef401ad20 comp rx=0 tx=0).stop 2024-04-25T09:00:29.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 -- 172.21.15.37:0/3735172146 shutdown_connections 2024-04-25T09:00:29.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feeec066f60 0x7feeec069420 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fef0c0b1ac0 0x7fef0c0cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fef0c06b820 0x7fef0c14ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 --2- 172.21.15.37:0/3735172146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef0c06b440 0x7fef0c14e4b0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 -- 172.21.15.37:0/3735172146 >> 172.21.15.37:0/3735172146 conn(0x7fef0c0b1f10 msgr2=0x7fef0c06f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 -- 172.21.15.37:0/3735172146 shutdown_connections 2024-04-25T09:00:29.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.423+0000 7fef11029640 1 -- 172.21.15.37:0/3735172146 wait complete. 2024-04-25T09:00:29.433 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:00:29.433 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:29.439 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:29.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.619+0000 7f7add955640 1 Processor -- start 2024-04-25T09:00:29.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.619+0000 7f7add955640 1 -- start start 2024-04-25T09:00:29.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.619+0000 7f7add955640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7ad80ce510 0x7f7ad80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.619+0000 7f7add955640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad8144ca0 0x7f7ad806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.619+0000 7f7add955640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad80cee30 con 0x7f7ad806ec60 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.619+0000 7f7add955640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad80cc2b0 con 0x7f7ad80ce510 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.619+0000 7f7add955640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad806e290 con 0x7f7ad8144ca0 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.619+0000 7f7ad77fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7ad806ec60 0x7f7ad8142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52438/0 (socket says 172.21.15.37:52438) 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.619+0000 7f7ad77fe640 1 -- 172.21.15.37:0/1842629146 learned_addr learned my addr 172.21.15.37:0/1842629146 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.619+0000 7f7ad6ffd640 1 --2- 172.21.15.37:0/1842629146 >> v2:172.21.15.203:3300/0 conn(0x7f7ad80ce510 0x7f7ad80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.619+0000 7f7ace7fc640 1 --2- 172.21.15.37:0/1842629146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad8144ca0 0x7f7ad806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.623+0000 7f7ad67fc640 1 -- 172.21.15.37:0/1842629146 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692682015 0 0) 0x7f7ad80cee30 con 0x7f7ad806ec60 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.623+0000 7f7ad67fc640 1 -- 172.21.15.37:0/1842629146 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab8003530 con 0x7f7ad806ec60 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.623+0000 7f7ad6ffd640 1 -- 172.21.15.37:0/1842629146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad8144ca0 msgr2=0x7f7ad806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.623+0000 7f7ad6ffd640 1 --2- 172.21.15.37:0/1842629146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad8144ca0 0x7f7ad806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.623+0000 7f7ad6ffd640 1 -- 172.21.15.37:0/1842629146 >> v1:172.21.15.37:6789/0 conn(0x7f7ad806ec60 legacy=0x7f7ad8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.623+0000 7f7ad6ffd640 1 -- 172.21.15.37:0/1842629146 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad80ccc70 con 0x7f7ad80ce510 2024-04-25T09:00:29.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.623+0000 7f7ad6ffd640 1 --2- 172.21.15.37:0/1842629146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad80ce510 0x7f7ad80ce8f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80097c0 tx=0x7f7ac801ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c35cb344e8909ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.623+0000 7f7ad67fc640 1 -- 172.21.15.37:0/1842629146 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80b5070 con 0x7f7ad80ce510 2024-04-25T09:00:29.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ad67fc640 1 -- 172.21.15.37:0/1842629146 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac80a8990 con 0x7f7ad80ce510 2024-04-25T09:00:29.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ad67fc640 1 -- 172.21.15.37:0/1842629146 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac80a8d40 con 0x7f7ad80ce510 2024-04-25T09:00:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 -- 172.21.15.37:0/1842629146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad80ce510 msgr2=0x7f7ad80ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 --2- 172.21.15.37:0/1842629146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad80ce510 0x7f7ad80ce8f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80097c0 tx=0x7f7ac801ec90 comp rx=0 tx=0).stop 2024-04-25T09:00:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 -- 172.21.15.37:0/1842629146 shutdown_connections 2024-04-25T09:00:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 --2- 172.21.15.37:0/1842629146 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad8144ca0 0x7f7ad806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 --2- 172.21.15.37:0/1842629146 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad80ce510 0x7f7ad80ce8f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 -- 172.21.15.37:0/1842629146 >> 172.21.15.37:0/1842629146 conn(0x7f7ad80b11a0 msgr2=0x7f7ad80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 -- 172.21.15.37:0/1842629146 shutdown_connections 2024-04-25T09:00:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 -- 172.21.15.37:0/1842629146 wait complete. 2024-04-25T09:00:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 Processor -- start 2024-04-25T09:00:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 -- start start 2024-04-25T09:00:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad806ec60 0x7f7ad81e0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ad80ce510 0x7f7ad81f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad8144ca0 0x7f7ad81f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad80cc900 con 0x7f7ad80ce510 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad806df20 con 0x7f7ad806ec60 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad8142d90 con 0x7f7ad8144ca0 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ace7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ad80ce510 0x7f7ad81f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ace7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ad80ce510 0x7f7ad81f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46720/0 (socket says 172.21.15.37:46720) 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ace7fc640 1 -- 172.21.15.37:0/2084808085 learned_addr learned my addr 172.21.15.37:0/2084808085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ad6ffd640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad806ec60 0x7f7ad81e0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ad77fe640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad8144ca0 0x7f7ad81f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ace7fc640 1 -- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad8144ca0 msgr2=0x7f7ad81f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ace7fc640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad8144ca0 0x7f7ad81f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ace7fc640 1 -- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad806ec60 msgr2=0x7f7ad81e0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ace7fc640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad806ec60 0x7f7ad81e0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ace7fc640 1 -- 172.21.15.37:0/2084808085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad80c6830 con 0x7f7ad80ce510 2024-04-25T09:00:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ad6ffd640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad806ec60 0x7f7ad81e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ad77fe640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad8144ca0 0x7f7ad81f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ace7fc640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ad80ce510 0x7f7ad81f1930 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000e840 tx=0x7f7ac000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ad4ff9640 1 -- 172.21.15.37:0/2084808085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac0017070 con 0x7f7ad80ce510 2024-04-25T09:00:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ad4ff9640 1 -- 172.21.15.37:0/2084808085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac000ee90 con 0x7f7ad80ce510 2024-04-25T09:00:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 -- 172.21.15.37:0/2084808085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad80c6b10 con 0x7f7ad80ce510 2024-04-25T09:00:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7ad4ff9640 1 -- 172.21.15.37:0/2084808085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac000cb60 con 0x7f7ad80ce510 2024-04-25T09:00:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.627+0000 7f7add955640 1 -- 172.21.15.37:0/2084808085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad80c70d0 con 0x7f7ad80ce510 2024-04-25T09:00:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.631+0000 7f7add955640 1 -- 172.21.15.37:0/2084808085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad804efa0 con 0x7f7ad80ce510 2024-04-25T09:00:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.631+0000 7f7ad4ff9640 1 -- 172.21.15.37:0/2084808085 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7ac001a070 con 0x7f7ad80ce510 2024-04-25T09:00:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.631+0000 7f7ad4ff9640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7abc066f90 0x7f7abc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.631+0000 7f7ad4ff9640 1 -- 172.21.15.37:0/2084808085 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f7ac0085540 con 0x7f7ad80ce510 2024-04-25T09:00:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.631+0000 7f7ad6ffd640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7abc066f90 0x7f7abc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.631+0000 7f7ad6ffd640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7abc066f90 0x7f7abc069450 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80b2000 tx=0x7f7ac80aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.635+0000 7f7ad4ff9640 1 -- 172.21.15.37:0/2084808085 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7ac0056260 con 0x7f7ad80ce510 2024-04-25T09:00:29.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.795+0000 7f7add955640 1 -- 172.21.15.37:0/2084808085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ad80c5b60 con 0x7f7ad80ce510 2024-04-25T09:00:29.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7ad4ff9640 1 -- 172.21.15.37:0/2084808085 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+17169 (secure 0 0 0) 0x7f7ac0055b60 con 0x7f7ad80ce510 2024-04-25T09:00:29.791 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:00:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":98,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T08:59:17.095374+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:50.512578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:51.848880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/354285428":"2024-04-26T08:59:17.095322+0000","172.21.15.37:6834/354285428":"2024-04-26T08:59:17.095322+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:29.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 -- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7abc066f90 msgr2=0x7f7abc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7abc066f90 0x7f7abc069450 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7ac80b2000 tx=0x7f7ac80aa070 comp rx=0 tx=0).stop 2024-04-25T09:00:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 -- 172.21.15.37:0/2084808085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ad80ce510 msgr2=0x7f7ad81f1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ad80ce510 0x7f7ad81f1930 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f7ac000e840 tx=0x7f7ac000ec40 comp rx=0 tx=0).stop 2024-04-25T09:00:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 -- 172.21.15.37:0/2084808085 shutdown_connections 2024-04-25T09:00:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7abc066f90 0x7f7abc069450 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7ad8144ca0 0x7f7ad81f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ad80ce510 0x7f7ad81f1930 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 --2- 172.21.15.37:0/2084808085 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7ad806ec60 0x7f7ad81e0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 -- 172.21.15.37:0/2084808085 >> 172.21.15.37:0/2084808085 conn(0x7f7ad80b11a0 msgr2=0x7f7ad80ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 -- 172.21.15.37:0/2084808085 shutdown_connections 2024-04-25T09:00:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.799+0000 7f7add955640 1 -- 172.21.15.37:0/2084808085 wait complete. 2024-04-25T09:00:29.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:58:50.512578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:58:51.848880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:00:29.810 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:29.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c527fa640 1 Processor -- start 2024-04-25T09:00:29.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c527fa640 1 -- start start 2024-04-25T09:00:29.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c527fa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2c4c09a180 0x7f2c4c0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c527fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c4c0afb90 0x7f2c4c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c527fa640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c09a950 con 0x7f2c4c15b920 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c527fa640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c09ac30 con 0x7f2c4c09a180 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c527fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0ae380 con 0x7f2c4c0afb90 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c4bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2c4c15b920 0x7f2c4c099a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52444/0 (socket says 172.21.15.37:52444) 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c4b7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2c4c09a180 0x7f2c4c0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c4bfff640 1 -- 172.21.15.37:0/3829352239 learned_addr learned my addr 172.21.15.37:0/3829352239 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c50d97640 1 --2- 172.21.15.37:0/3829352239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c4c0afb90 0x7f2c4c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c4affd640 1 -- 172.21.15.37:0/3829352239 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2282786529 0 0) 0x7f2c4c09a950 con 0x7f2c4c15b920 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c4affd640 1 -- 172.21.15.37:0/3829352239 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c34003530 con 0x7f2c4c15b920 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c4b7fe640 1 -- 172.21.15.37:0/3829352239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c4c0afb90 msgr2=0x7f2c4c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c4b7fe640 1 --2- 172.21.15.37:0/3829352239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c4c0afb90 0x7f2c4c0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c4b7fe640 1 -- 172.21.15.37:0/3829352239 >> v1:172.21.15.37:6789/0 conn(0x7f2c4c15b920 legacy=0x7f2c4c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.951+0000 7f2c4b7fe640 1 -- 172.21.15.37:0/3829352239 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c0ae660 con 0x7f2c4c09a180 2024-04-25T09:00:29.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c4b7fe640 1 --2- 172.21.15.37:0/3829352239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c4c09a180 0x7f2c4c0cc130 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2c400097c0 tx=0x7f2c4001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf2a1f1c70267c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c4affd640 1 -- 172.21.15.37:0/3829352239 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c400b0070 con 0x7f2c4c09a180 2024-04-25T09:00:29.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c4affd640 1 -- 172.21.15.37:0/3829352239 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c400ab600 con 0x7f2c4c09a180 2024-04-25T09:00:29.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c4affd640 1 -- 172.21.15.37:0/3829352239 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c400ab9d0 con 0x7f2c4c09a180 2024-04-25T09:00:29.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 -- 172.21.15.37:0/3829352239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c4c09a180 msgr2=0x7f2c4c0cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 --2- 172.21.15.37:0/3829352239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c4c09a180 0x7f2c4c0cc130 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f2c400097c0 tx=0x7f2c4001c410 comp rx=0 tx=0).stop 2024-04-25T09:00:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 -- 172.21.15.37:0/3829352239 shutdown_connections 2024-04-25T09:00:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 --2- 172.21.15.37:0/3829352239 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c4c0afb90 0x7f2c4c0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 --2- 172.21.15.37:0/3829352239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c4c09a180 0x7f2c4c0cc130 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 -- 172.21.15.37:0/3829352239 >> 172.21.15.37:0/3829352239 conn(0x7f2c4c0b3010 msgr2=0x7f2c4c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 -- 172.21.15.37:0/3829352239 shutdown_connections 2024-04-25T09:00:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 -- 172.21.15.37:0/3829352239 wait complete. 2024-04-25T09:00:29.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 Processor -- start 2024-04-25T09:00:29.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 -- start start 2024-04-25T09:00:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c4c09a180 0x7f2c4c06d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c4c0afb90 0x7f2c4c153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c4c15b920 0x7f2c4c165a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c1684c0 con 0x7f2c4c15b920 2024-04-25T09:00:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0cd5d0 con 0x7f2c4c09a180 2024-04-25T09:00:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.955+0000 7f2c527fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c168640 con 0x7f2c4c0afb90 2024-04-25T09:00:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c50d97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c4c15b920 0x7f2c4c165a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c50d97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c4c15b920 0x7f2c4c165a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46746/0 (socket says 172.21.15.37:46746) 2024-04-25T09:00:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c50d97640 1 -- 172.21.15.37:0/174055519 learned_addr learned my addr 172.21.15.37:0/174055519 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c4b7fe640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c4c0afb90 0x7f2c4c153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c4bfff640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c4c09a180 0x7f2c4c06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c50d97640 1 -- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c4c0afb90 msgr2=0x7f2c4c153a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c50d97640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c4c0afb90 0x7f2c4c153a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c50d97640 1 -- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c4c09a180 msgr2=0x7f2c4c06d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c50d97640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c4c09a180 0x7f2c4c06d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c50d97640 1 -- 172.21.15.37:0/174055519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c06c7b0 con 0x7f2c4c15b920 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c4bfff640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c4c09a180 0x7f2c4c06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c50d97640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c4c15b920 0x7f2c4c165a80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f2c44007e60 tx=0x7f2c4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c48ff9640 1 -- 172.21.15.37:0/174055519 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44014420 con 0x7f2c4c15b920 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c48ff9640 1 -- 172.21.15.37:0/174055519 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c44019070 con 0x7f2c4c15b920 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c48ff9640 1 -- 172.21.15.37:0/174055519 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44014820 con 0x7f2c4c15b920 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c527fa640 1 -- 172.21.15.37:0/174055519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c06d790 con 0x7f2c4c15b920 2024-04-25T09:00:29.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c527fa640 1 -- 172.21.15.37:0/174055519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c159100 con 0x7f2c4c15b920 2024-04-25T09:00:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c527fa640 1 -- 172.21.15.37:0/174055519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c04efa0 con 0x7f2c4c15b920 2024-04-25T09:00:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c48ff9640 1 -- 172.21.15.37:0/174055519 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c440149c0 con 0x7f2c4c15b920 2024-04-25T09:00:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c48ff9640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c38066f60 0x7f2c38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c48ff9640 1 -- 172.21.15.37:0/174055519 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f2c44085a90 con 0x7f2c4c15b920 2024-04-25T09:00:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.959+0000 7f2c4bfff640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c38066f60 0x7f2c38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.963+0000 7f2c4bfff640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c38066f60 0x7f2c38069420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0011f0 tx=0x7f2c3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:29.963+0000 7f2c48ff9640 1 -- 172.21.15.37:0/174055519 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c44056850 con 0x7f2c4c15b920 2024-04-25T09:00:30.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.127+0000 7f2c527fa640 1 -- 172.21.15.37:0/174055519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c4c0c7960 con 0x7f2c4c15b920 2024-04-25T09:00:30.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.127+0000 7f2c48ff9640 1 -- 172.21.15.37:0/174055519 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c44056150 con 0x7f2c4c15b920 2024-04-25T09:00:30.120 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:00:30.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 -- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c38066f60 msgr2=0x7f2c38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c38066f60 0x7f2c38069420 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c0011f0 tx=0x7f2c3c009040 comp rx=0 tx=0).stop 2024-04-25T09:00:30.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 -- 172.21.15.37:0/174055519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c4c15b920 msgr2=0x7f2c4c165a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c4c15b920 0x7f2c4c165a80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f2c44007e60 tx=0x7f2c4400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 -- 172.21.15.37:0/174055519 shutdown_connections 2024-04-25T09:00:30.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c38066f60 0x7f2c38069420 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c4c15b920 0x7f2c4c165a80 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c4c0afb90 0x7f2c4c153a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 --2- 172.21.15.37:0/174055519 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c4c09a180 0x7f2c4c06d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 -- 172.21.15.37:0/174055519 >> 172.21.15.37:0/174055519 conn(0x7f2c4c0b3010 msgr2=0x7f2c4c158110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 -- 172.21.15.37:0/174055519 shutdown_connections 2024-04-25T09:00:30.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.131+0000 7f2c527fa640 1 -- 172.21.15.37:0/174055519 wait complete. 2024-04-25T09:00:30.139 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:30.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcd5610640 1 Processor -- start 2024-04-25T09:00:30.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcd5610640 1 -- start start 2024-04-25T09:00:30.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcd5610640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efcd0159bd0 0x7efcd00c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcd5610640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcd0159fb0 0x7efcd00b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcd5610640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd00c9ef0 con 0x7efcd00b26f0 2024-04-25T09:00:30.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcd5610640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd0072780 con 0x7efcd0159bd0 2024-04-25T09:00:30.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcd5610640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd0071b70 con 0x7efcd0159fb0 2024-04-25T09:00:30.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcceffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efcd0159bd0 0x7efcd00c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efccf7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7efcd00b26f0 0x7efcd00cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52456/0 (socket says 172.21.15.37:52456) 2024-04-25T09:00:30.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efccf7fe640 1 -- 172.21.15.37:0/3070022467 learned_addr learned my addr 172.21.15.37:0/3070022467 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:30.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcce7fc640 1 --2- 172.21.15.37:0/3070022467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcd0159fb0 0x7efcd00b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcceffd640 1 -- 172.21.15.37:0/3070022467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcd0159fb0 msgr2=0x7efcd00b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:30.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcceffd640 1 --2- 172.21.15.37:0/3070022467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcd0159fb0 0x7efcd00b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcceffd640 1 -- 172.21.15.37:0/3070022467 >> v1:172.21.15.37:6789/0 conn(0x7efcd00b26f0 legacy=0x7efcd00cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.271+0000 7efcceffd640 1 -- 172.21.15.37:0/3070022467 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd0071e50 con 0x7efcd0159bd0 2024-04-25T09:00:30.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcceffd640 1 --2- 172.21.15.37:0/3070022467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd0159bd0 0x7efcd00c99b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009d90 tx=0x7efcc001da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ef3fe784ddb0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efccdffb640 1 -- 172.21.15.37:0/3070022467 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc00b3070 con 0x7efcd0159bd0 2024-04-25T09:00:30.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efccdffb640 1 -- 172.21.15.37:0/3070022467 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc00a6a90 con 0x7efcd0159bd0 2024-04-25T09:00:30.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efccdffb640 1 -- 172.21.15.37:0/3070022467 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc00a6e60 con 0x7efcd0159bd0 2024-04-25T09:00:30.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 -- 172.21.15.37:0/3070022467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd0159bd0 msgr2=0x7efcd00c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 --2- 172.21.15.37:0/3070022467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd0159bd0 0x7efcd00c99b0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7efcc0009d90 tx=0x7efcc001da50 comp rx=0 tx=0).stop 2024-04-25T09:00:30.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 -- 172.21.15.37:0/3070022467 shutdown_connections 2024-04-25T09:00:30.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 --2- 172.21.15.37:0/3070022467 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcd0159fb0 0x7efcd00b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 --2- 172.21.15.37:0/3070022467 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd0159bd0 0x7efcd00c99b0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 -- 172.21.15.37:0/3070022467 >> 172.21.15.37:0/3070022467 conn(0x7efcd006ae60 msgr2=0x7efcd006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 -- 172.21.15.37:0/3070022467 shutdown_connections 2024-04-25T09:00:30.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 -- 172.21.15.37:0/3070022467 wait complete. 2024-04-25T09:00:30.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 Processor -- start 2024-04-25T09:00:30.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 -- start start 2024-04-25T09:00:30.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcd00b26f0 0x7efcd0154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd0159bd0 0x7efcd0158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcd0159fb0 0x7efcd0070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00cba90 con 0x7efcd0159bd0 2024-04-25T09:00:30.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00cefd0 con 0x7efcd0159fb0 2024-04-25T09:00:30.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcd5610640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd00cbc10 con 0x7efcd00b26f0 2024-04-25T09:00:30.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcce7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd0159bd0 0x7efcd0158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcce7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd0159bd0 0x7efcd0158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46762/0 (socket says 172.21.15.37:46762) 2024-04-25T09:00:30.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcce7fc640 1 -- 172.21.15.37:0/2172968316 learned_addr learned my addr 172.21.15.37:0/2172968316 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:30.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efccf7fe640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcd0159fb0 0x7efcd0070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcce7fc640 1 -- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcd00b26f0 msgr2=0x7efcd0154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcce7fc640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcd00b26f0 0x7efcd0154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcce7fc640 1 -- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcd0159fb0 msgr2=0x7efcd0070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcce7fc640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcd0159fb0 0x7efcd0070ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcce7fc640 1 -- 172.21.15.37:0/2172968316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd015c050 con 0x7efcd0159bd0 2024-04-25T09:00:30.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efccf7fe640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcd0159fb0 0x7efcd0070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:30.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.275+0000 7efcce7fc640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd0159bd0 0x7efcd0158640 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7efcc400d7d0 tx=0x7efcc400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.279+0000 7efcb77fe640 1 -- 172.21.15.37:0/2172968316 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4004020 con 0x7efcd0159bd0 2024-04-25T09:00:30.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.279+0000 7efcb77fe640 1 -- 172.21.15.37:0/2172968316 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc40041c0 con 0x7efcd0159bd0 2024-04-25T09:00:30.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.279+0000 7efcd5610640 1 -- 172.21.15.37:0/2172968316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd015c330 con 0x7efcd0159bd0 2024-04-25T09:00:30.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.279+0000 7efcb77fe640 1 -- 172.21.15.37:0/2172968316 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc40043b0 con 0x7efcd0159bd0 2024-04-25T09:00:30.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.279+0000 7efcd5610640 1 -- 172.21.15.37:0/2172968316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd015cbd0 con 0x7efcd0159bd0 2024-04-25T09:00:30.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.279+0000 7efcd5610640 1 -- 172.21.15.37:0/2172968316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd004efa0 con 0x7efcd0159bd0 2024-04-25T09:00:30.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.279+0000 7efcb77fe640 1 -- 172.21.15.37:0/2172968316 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efcc401b020 con 0x7efcd0159bd0 2024-04-25T09:00:30.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.279+0000 7efcb77fe640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efcbc066f60 0x7efcbc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.279+0000 7efcb77fe640 1 -- 172.21.15.37:0/2172968316 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7efcc40850c0 con 0x7efcd0159bd0 2024-04-25T09:00:30.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.279+0000 7efcceffd640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efcbc066f60 0x7efcbc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.279+0000 7efcceffd640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efcbc066f60 0x7efcbc069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efcc00b0000 tx=0x7efcc000d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.283+0000 7efcb77fe640 1 -- 172.21.15.37:0/2172968316 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efcc4055e80 con 0x7efcd0159bd0 2024-04-25T09:00:30.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 -- 172.21.15.37:0/2172968316 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efcd006bfc0 con 0x7efcd0159bd0 2024-04-25T09:00:30.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcb77fe640 1 -- 172.21.15.37:0/2172968316 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) v1 ==== 104+0+11 (secure 0 0 0) 0x7efcc4055780 con 0x7efcd0159bd0 2024-04-25T09:00:30.438 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 -- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efcbc066f60 msgr2=0x7efcbc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efcbc066f60 0x7efcbc069420 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7efcc00b0000 tx=0x7efcc000d040 comp rx=0 tx=0).stop 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 -- 172.21.15.37:0/2172968316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd0159bd0 msgr2=0x7efcd0158640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd0159bd0 0x7efcd0158640 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7efcc400d7d0 tx=0x7efcc400dc90 comp rx=0 tx=0).stop 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 -- 172.21.15.37:0/2172968316 shutdown_connections 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efcbc066f60 0x7efcbc069420 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efcd0159fb0 0x7efcd0070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcd0159bd0 0x7efcd0158640 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 --2- 172.21.15.37:0/2172968316 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efcd00b26f0 0x7efcd0154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 -- 172.21.15.37:0/2172968316 >> 172.21.15.37:0/2172968316 conn(0x7efcd006ae60 msgr2=0x7efcd00afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 -- 172.21.15.37:0/2172968316 shutdown_connections 2024-04-25T09:00:30.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.447+0000 7efcd5610640 1 -- 172.21.15.37:0/2172968316 wait complete. 2024-04-25T09:00:30.457 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:30.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e330a640 1 Processor -- start 2024-04-25T09:00:30.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e330a640 1 -- start start 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e330a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa7dc15b6b0 0x7fa7dc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e330a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa7dc073010 0x7fa7dc0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e330a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc0cbd20 con 0x7fa7dc0cf590 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e330a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7dc15ba90 con 0x7fa7dc073010 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e330a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7dc0cc000 con 0x7fa7dc15b6b0 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e18a7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa7dc073010 0x7fa7dc0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e18a7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa7dc073010 0x7fa7dc0cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46770/0 (socket says 172.21.15.37:46770) 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e18a7640 1 -- 172.21.15.37:0/56605726 learned_addr learned my addr 172.21.15.37:0/56605726 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e10a6640 1 --2- 172.21.15.37:0/56605726 >> v2:172.21.15.203:3300/0 conn(0x7fa7dc15b6b0 0x7fa7dc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e18a7640 1 -- 172.21.15.37:0/56605726 >> v2:172.21.15.203:3300/0 conn(0x7fa7dc15b6b0 msgr2=0x7fa7dc0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e18a7640 1 --2- 172.21.15.37:0/56605726 >> v2:172.21.15.203:3300/0 conn(0x7fa7dc15b6b0 0x7fa7dc0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e18a7640 1 -- 172.21.15.37:0/56605726 >> v1:172.21.15.203:6789/0 conn(0x7fa7dc0cf590 legacy=0x7fa7dc15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e18a7640 1 -- 172.21.15.37:0/56605726 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc0cc2e0 con 0x7fa7dc073010 2024-04-25T09:00:30.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.591+0000 7fa7e10a6640 1 --2- 172.21.15.37:0/56605726 >> v2:172.21.15.203:3300/0 conn(0x7fa7dc15b6b0 0x7fa7dc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e18a7640 1 --2- 172.21.15.37:0/56605726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7dc073010 0x7fa7dc0cfa20 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0d0180 tx=0x7fa7d401c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a80617a9affe475 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7d3fff640 1 -- 172.21.15.37:0/56605726 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40a5680 con 0x7fa7dc073010 2024-04-25T09:00:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7d3fff640 1 -- 172.21.15.37:0/56605726 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d40a5820 con 0x7fa7dc073010 2024-04-25T09:00:30.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7d3fff640 1 -- 172.21.15.37:0/56605726 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d40a5bf0 con 0x7fa7dc073010 2024-04-25T09:00:30.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 -- 172.21.15.37:0/56605726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7dc073010 msgr2=0x7fa7dc0cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 --2- 172.21.15.37:0/56605726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7dc073010 0x7fa7dc0cfa20 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc0d0180 tx=0x7fa7d401c9b0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 -- 172.21.15.37:0/56605726 shutdown_connections 2024-04-25T09:00:30.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 --2- 172.21.15.37:0/56605726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7dc073010 0x7fa7dc0cfa20 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 --2- 172.21.15.37:0/56605726 >> v2:172.21.15.203:3300/0 conn(0x7fa7dc15b6b0 0x7fa7dc0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 -- 172.21.15.37:0/56605726 >> 172.21.15.37:0/56605726 conn(0x7fa7dc06ae60 msgr2=0x7fa7dc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 -- 172.21.15.37:0/56605726 shutdown_connections 2024-04-25T09:00:30.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 -- 172.21.15.37:0/56605726 wait complete. 2024-04-25T09:00:30.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 Processor -- start 2024-04-25T09:00:30.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 -- start start 2024-04-25T09:00:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7dc073010 0x7fa7dc0cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7dc0cf590 0x7fa7dc0985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc15b6b0 0x7fa7dc152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0cadb0 con 0x7fa7dc073010 2024-04-25T09:00:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0ca900 con 0x7fa7dc0cf590 2024-04-25T09:00:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e330a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0caf30 con 0x7fa7dc15b6b0 2024-04-25T09:00:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e10a6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7dc073010 0x7fa7dc0cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e10a6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7dc073010 0x7fa7dc0cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46776/0 (socket says 172.21.15.37:46776) 2024-04-25T09:00:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e10a6640 1 -- 172.21.15.37:0/68431444 learned_addr learned my addr 172.21.15.37:0/68431444 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:30.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e18a7640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc15b6b0 0x7fa7dc152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e08a5640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7dc0cf590 0x7fa7dc0985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e10a6640 1 -- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc15b6b0 msgr2=0x7fa7dc152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e10a6640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc15b6b0 0x7fa7dc152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e10a6640 1 -- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7dc0cf590 msgr2=0x7fa7dc0985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e10a6640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7dc0cf590 0x7fa7dc0985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e10a6640 1 -- 172.21.15.37:0/68431444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc0cd970 con 0x7fa7dc073010 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.595+0000 7fa7e18a7640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc15b6b0 0x7fa7dc152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7e10a6640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7dc073010 0x7fa7dc0cd390 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c7f0 tx=0x7fa7d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7d1ffb640 1 -- 172.21.15.37:0/68431444 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8015070 con 0x7fa7dc073010 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7d1ffb640 1 -- 172.21.15.37:0/68431444 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8007820 con 0x7fa7dc073010 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7d1ffb640 1 -- 172.21.15.37:0/68431444 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8007c10 con 0x7fa7dc073010 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7e330a640 1 -- 172.21.15.37:0/68431444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc0cdc50 con 0x7fa7dc073010 2024-04-25T09:00:30.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7e330a640 1 -- 172.21.15.37:0/68431444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7dc0ce4f0 con 0x7fa7dc073010 2024-04-25T09:00:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7e330a640 1 -- 172.21.15.37:0/68431444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7dc0cde20 con 0x7fa7dc073010 2024-04-25T09:00:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7d1ffb640 1 -- 172.21.15.37:0/68431444 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa7d8018070 con 0x7fa7dc073010 2024-04-25T09:00:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7d1ffb640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c0066f60 0x7fa7c0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7d1ffb640 1 -- 172.21.15.37:0/68431444 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa7d80853f0 con 0x7fa7dc073010 2024-04-25T09:00:30.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7e08a5640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c0066f60 0x7fa7c0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.599+0000 7fa7e08a5640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c0066f60 0x7fa7c0069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc156c00 tx=0x7fa7cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.603+0000 7fa7d1ffb640 1 -- 172.21.15.37:0/68431444 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa7d80561b0 con 0x7fa7dc073010 2024-04-25T09:00:30.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 -- 172.21.15.37:0/68431444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa7dc04efa0 con 0x7fa7dc073010 2024-04-25T09:00:30.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7d1ffb640 1 -- 172.21.15.37:0/68431444 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa7d8055ab0 con 0x7fa7dc073010 2024-04-25T09:00:30.755 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:00:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 -- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c0066f60 msgr2=0x7fa7c0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c0066f60 0x7fa7c0069420 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc156c00 tx=0x7fa7cc009040 comp rx=0 tx=0).stop 2024-04-25T09:00:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 -- 172.21.15.37:0/68431444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7dc073010 msgr2=0x7fa7dc0cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7dc073010 0x7fa7dc0cd390 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fa7d800c7f0 tx=0x7fa7d800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 -- 172.21.15.37:0/68431444 shutdown_connections 2024-04-25T09:00:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa7c0066f60 0x7fa7c0069420 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa7dc15b6b0 0x7fa7dc152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7dc0cf590 0x7fa7dc0985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 --2- 172.21.15.37:0/68431444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7dc073010 0x7fa7dc0cd390 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 -- 172.21.15.37:0/68431444 >> 172.21.15.37:0/68431444 conn(0x7fa7dc06ae60 msgr2=0x7fa7dc164fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.763+0000 7fa7e330a640 1 -- 172.21.15.37:0/68431444 shutdown_connections 2024-04-25T09:00:30.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.767+0000 7fa7e330a640 1 -- 172.21.15.37:0/68431444 wait complete. 2024-04-25T09:00:30.776 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:00:30.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd416c640 1 Processor -- start 2024-04-25T09:00:30.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd416c640 1 -- start start 2024-04-25T09:00:30.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd416c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc0cf840 0x7fddcc0cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd416c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc06b710 con 0x7fddcc06fc50 2024-04-25T09:00:30.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd416c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddcc1617b0 con 0x7fddcc15d180 2024-04-25T09:00:30.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd416c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc161560 con 0x7fddcc0cf840 2024-04-25T09:00:30.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd1f08640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fddcc06fc50 0x7fddcc06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52466/0 (socket says 172.21.15.37:52466) 2024-04-25T09:00:30.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd1f08640 1 -- 172.21.15.37:0/2212326389 learned_addr learned my addr 172.21.15.37:0/2212326389 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:30.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd1707640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc0cf840 0x7fddcc0cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001862982 0 0) 0x7fddcc06b710 con 0x7fddcc06fc50 2024-04-25T09:00:30.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddb8003530 con 0x7fddcc06fc50 2024-04-25T09:00:30.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4199623794 0 0) 0x7fddb8003530 con 0x7fddcc06fc50 2024-04-25T09:00:30.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc0cf840 msgr2=0x7fddcc0cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:30.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc0cf840 0x7fddcc0cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 >> v1:172.21.15.203:6789/0 conn(0x7fddcc15d180 legacy=0x7fddcc15d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc161a90 con 0x7fddcc06fc50 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fddc8001680 con 0x7fddcc06fc50 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 >> v1:172.21.15.37:6789/0 conn(0x7fddcc06fc50 legacy=0x7fddcc06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb8003a60 0x7fddb8003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc0cf840 0x7fddb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb8004d50 0x7fddb80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc161560 con 0x7fddb8003a60 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb800bb60 con 0x7fddb8004d50 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb800bd60 con 0x7fddcc0cf840 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd1f08640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb8003a60 0x7fddb8003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd1707640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc0cf840 0x7fddb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd2709640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb8004d50 0x7fddb80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd1f08640 1 -- 172.21.15.37:0/2212326389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc0cf840 msgr2=0x7fddb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd1f08640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc0cf840 0x7fddb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd1f08640 1 -- 172.21.15.37:0/2212326389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb8004d50 msgr2=0x7fddb80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd1f08640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb8004d50 0x7fddb80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.915+0000 7fddd1f08640 1 -- 172.21.15.37:0/2212326389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb8003530 con 0x7fddb8003a60 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd1f08640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb8003a60 0x7fddb8003e80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fddc800c160 tx=0x7fddc801cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c34d040add6daf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8007c70 con 0x7fddb8003a60 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc80a9070 con 0x7fddb8003a60 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd0f06640 1 -- 172.21.15.37:0/2212326389 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8005d80 con 0x7fddb8003a60 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd416c640 1 -- 172.21.15.37:0/2212326389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb8003a60 msgr2=0x7fddb8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd416c640 1 --2- 172.21.15.37:0/2212326389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb8003a60 0x7fddb8003e80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fddc800c160 tx=0x7fddc801cb60 comp rx=0 tx=0).stop 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd2709640 1 -- 172.21.15.37:0/2212326389 reap_dead start 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd416c640 1 -- 172.21.15.37:0/2212326389 shutdown_connections 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd416c640 1 -- 172.21.15.37:0/2212326389 >> 172.21.15.37:0/2212326389 conn(0x7fddcc099a70 msgr2=0x7fddcc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd416c640 1 -- 172.21.15.37:0/2212326389 shutdown_connections 2024-04-25T09:00:30.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd416c640 1 -- 172.21.15.37:0/2212326389 wait complete. 2024-04-25T09:00:30.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd416c640 1 Processor -- start 2024-04-25T09:00:30.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.919+0000 7fddd416c640 1 -- start start 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd416c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb8003a60 0x7fddcc154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd416c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddcc155c30 0x7fddcc157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd416c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc1556e0 0x7fddcc157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd416c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc15de80 con 0x7fddcc155c30 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd416c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc15dd00 con 0x7fddb8003a60 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd416c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc070030 con 0x7fddcc1556e0 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1707640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddcc155c30 0x7fddcc157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1707640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddcc155c30 0x7fddcc157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46790/0 (socket says 172.21.15.37:46790) 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1707640 1 -- 172.21.15.37:0/3589910104 learned_addr learned my addr 172.21.15.37:0/3589910104 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd2709640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc1556e0 0x7fddcc157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1f08640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb8003a60 0x7fddcc154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1707640 1 -- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc1556e0 msgr2=0x7fddcc157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1707640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc1556e0 0x7fddcc157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1707640 1 -- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb8003a60 msgr2=0x7fddcc154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1707640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb8003a60 0x7fddcc154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1707640 1 -- 172.21.15.37:0/3589910104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc1554d0 con 0x7fddcc155c30 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd2709640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc1556e0 0x7fddcc157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1f08640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb8003a60 0x7fddcc154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1707640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddcc155c30 0x7fddcc157550 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fddbc011640 tx=0x7fddbc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddc27fc640 1 -- 172.21.15.37:0/3589910104 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00f920 con 0x7fddcc155c30 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddc27fc640 1 -- 172.21.15.37:0/3589910104 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc00fac0 con 0x7fddcc155c30 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd416c640 1 -- 172.21.15.37:0/3589910104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddcc0b32c0 con 0x7fddcc155c30 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddc27fc640 1 -- 172.21.15.37:0/3589910104 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00fe90 con 0x7fddcc155c30 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd416c640 1 -- 172.21.15.37:0/3589910104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddcc0b3880 con 0x7fddcc155c30 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd416c640 1 -- 172.21.15.37:0/3589910104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddcc04efa0 con 0x7fddcc155c30 2024-04-25T09:00:30.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddc27fc640 1 -- 172.21.15.37:0/3589910104 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fddbc015460 con 0x7fddcc155c30 2024-04-25T09:00:30.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddc27fc640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fddb0067100 0x7fddb00695c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddc27fc640 1 -- 172.21.15.37:0/3589910104 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fddbc085ba0 con 0x7fddcc155c30 2024-04-25T09:00:30.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.923+0000 7fddd1f08640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fddb0067100 0x7fddb00695c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.927+0000 7fddd1f08640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fddb0067100 0x7fddb00695c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fddc80ab000 tx=0x7fddc80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:30.927+0000 7fddc27fc640 1 -- 172.21.15.37:0/3589910104 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fddbc019030 con 0x7fddcc155c30 2024-04-25T09:00:31.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.095+0000 7fddd416c640 1 -- 172.21.15.37:0/3589910104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fddcc0cac10 con 0x7fddcc155c30 2024-04-25T09:00:31.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.095+0000 7fddc27fc640 1 -- 172.21.15.37:0/3589910104 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v98) v1 ==== 80+0+120 (secure 0 0 0) 0x7fddbc056960 con 0x7fddcc155c30 2024-04-25T09:00:31.087 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/354285428 2024-04-26T08:59:17.095322+0000 2024-04-25T09:00:31.087 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6834/354285428 2024-04-26T08:59:17.095322+0000 2024-04-25T09:00:31.088 INFO:teuthology.orchestra.run.smithi037.stderr:listed 2 entries 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.095+0000 7fddd416c640 1 -- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fddb0067100 msgr2=0x7fddb00695c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.095+0000 7fddd416c640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fddb0067100 0x7fddb00695c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fddc80ab000 tx=0x7fddc80a7070 comp rx=0 tx=0).stop 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.095+0000 7fddd416c640 1 -- 172.21.15.37:0/3589910104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddcc155c30 msgr2=0x7fddcc157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.095+0000 7fddd416c640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddcc155c30 0x7fddcc157550 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fddbc011640 tx=0x7fddbc011b00 comp rx=0 tx=0).stop 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.099+0000 7fddd416c640 1 -- 172.21.15.37:0/3589910104 shutdown_connections 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.099+0000 7fddd416c640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fddb0067100 0x7fddb00695c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.099+0000 7fddd416c640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddcc1556e0 0x7fddcc157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.099+0000 7fddd416c640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddcc155c30 0x7fddcc157550 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.099+0000 7fddd416c640 1 --2- 172.21.15.37:0/3589910104 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb8003a60 0x7fddcc154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.099+0000 7fddd416c640 1 -- 172.21.15.37:0/3589910104 >> 172.21.15.37:0/3589910104 conn(0x7fddcc099a70 msgr2=0x7fddcc15c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.099+0000 7fddd416c640 1 -- 172.21.15.37:0/3589910104 shutdown_connections 2024-04-25T09:00:31.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:31.099+0000 7fddd416c640 1 -- 172.21.15.37:0/3589910104 wait complete. 2024-04-25T09:00:31.105 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T09:00:31.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:31.105 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:31.116 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[48758]: fuse finished with error 0 and tester_r 0 2024-04-25T09:00:31.117 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:37.220 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:00:37.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:37.220 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:00:37.229 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:37.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faecc64f640 1 Processor -- start 2024-04-25T09:00:37.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faecc64f640 1 -- start start 2024-04-25T09:00:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faecc64f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7faec40b3f50 0x7faec415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faecc64f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec40b2090 con 0x7faec40b24a0 2024-04-25T09:00:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faecc64f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec40b43e0 con 0x7faec415a700 2024-04-25T09:00:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faecc64f640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec415f200 con 0x7faec40b3f50 2024-04-25T09:00:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faecabec640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7faec40b24a0 0x7faec406e1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52474/0 (socket says 172.21.15.37:52474) 2024-04-25T09:00:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faecabec640 1 -- 172.21.15.37:0/3166720819 learned_addr learned my addr 172.21.15.37:0/3166720819 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec9bea640 1 --2- 172.21.15.37:0/3166720819 >> v2:172.21.15.203:3300/0 conn(0x7faec40b3f50 0x7faec415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4183942163 0 0) 0x7faec40b2090 con 0x7faec40b24a0 2024-04-25T09:00:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faeb0003530 con 0x7faec40b24a0 2024-04-25T09:00:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 783413959 0 0) 0x7faeb0003530 con 0x7faec40b24a0 2024-04-25T09:00:37.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 >> v2:172.21.15.203:3300/0 conn(0x7faec40b3f50 msgr2=0x7faec415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 --2- 172.21.15.37:0/3166720819 >> v2:172.21.15.203:3300/0 conn(0x7faec40b3f50 0x7faec415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 >> v1:172.21.15.203:6789/0 conn(0x7faec415a700 legacy=0x7faec40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec40b46c0 con 0x7faec40b24a0 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7faebc001a20 con 0x7faec40b24a0 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 >> v1:172.21.15.37:6789/0 conn(0x7faec40b24a0 legacy=0x7faec406e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 --2- 172.21.15.37:0/3166720819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0003a30 0x7faeb0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 --2- 172.21.15.37:0/3166720819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faeb0004810 0x7faeb0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 --2- 172.21.15.37:0/3166720819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faeb0005380 0x7faeb000b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec415f200 con 0x7faeb0004810 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb000c2d0 con 0x7faeb0003a30 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb000c4d0 con 0x7faeb0005380 2024-04-25T09:00:37.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec9bea640 1 --2- 172.21.15.37:0/3166720819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faeb0004810 0x7faeb0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faecabec640 1 --2- 172.21.15.37:0/3166720819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faeb0005380 0x7faeb000b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec9bea640 1 -- 172.21.15.37:0/3166720819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faeb0005380 msgr2=0x7faeb000b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:37.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faeca3eb640 1 --2- 172.21.15.37:0/3166720819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0003a30 0x7faeb0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec9bea640 1 --2- 172.21.15.37:0/3166720819 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faeb0005380 0x7faeb000b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec9bea640 1 -- 172.21.15.37:0/3166720819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0003a30 msgr2=0x7faeb0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:37.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec9bea640 1 --2- 172.21.15.37:0/3166720819 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0003a30 0x7faeb0003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faec9bea640 1 -- 172.21.15.37:0/3166720819 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb0003530 con 0x7faeb0004810 2024-04-25T09:00:37.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.419+0000 7faecabec640 1 -- 172.21.15.37:0/3166720819 reap_dead start 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faec9bea640 1 --2- 172.21.15.37:0/3166720819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faeb0004810 0x7faeb0004c90 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7faeb400ba30 tx=0x7faeb401c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e2b46a41de51d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb400b600 con 0x7faeb0004810 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb40b1070 con 0x7faeb0004810 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faec93e9640 1 -- 172.21.15.37:0/3166720819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb40ac850 con 0x7faeb0004810 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faecc64f640 1 -- 172.21.15.37:0/3166720819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faeb0004810 msgr2=0x7faeb0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faecc64f640 1 --2- 172.21.15.37:0/3166720819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faeb0004810 0x7faeb0004c90 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7faeb400ba30 tx=0x7faeb401c670 comp rx=0 tx=0).stop 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faecc64f640 1 -- 172.21.15.37:0/3166720819 shutdown_connections 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faecc64f640 1 --2- 172.21.15.37:0/3166720819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faeb0004810 0x7faeb0004c90 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faecc64f640 1 -- 172.21.15.37:0/3166720819 >> 172.21.15.37:0/3166720819 conn(0x7faec40b11a0 msgr2=0x7faec40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faecc64f640 1 -- 172.21.15.37:0/3166720819 shutdown_connections 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faecc64f640 1 -- 172.21.15.37:0/3166720819 wait complete. 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faecc64f640 1 Processor -- start 2024-04-25T09:00:37.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.423+0000 7faecc64f640 1 -- start start 2024-04-25T09:00:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecc64f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0004810 0x7faec41f0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecc64f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faec40cb790 0x7faec41f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecc64f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faec40ac8e0 0x7faec415c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecc64f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec40b2c90 con 0x7faec40ac8e0 2024-04-25T09:00:37.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecc64f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec40ce700 con 0x7faeb0004810 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecc64f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec40cd2d0 con 0x7faec40cb790 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecabec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faec40ac8e0 0x7faec415c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecabec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faec40ac8e0 0x7faec415c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46800/0 (socket says 172.21.15.37:46800) 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecabec640 1 -- 172.21.15.37:0/876462523 learned_addr learned my addr 172.21.15.37:0/876462523 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faeca3eb640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0004810 0x7faec41f0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecabec640 1 -- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faec40cb790 msgr2=0x7faec41f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecabec640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faec40cb790 0x7faec41f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecabec640 1 -- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0004810 msgr2=0x7faec41f0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecabec640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0004810 0x7faec41f0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecabec640 1 -- 172.21.15.37:0/876462523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec41f7560 con 0x7faec40ac8e0 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faeca3eb640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0004810 0x7faec41f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecabec640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faec40ac8e0 0x7faec415c2e0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7faec406cfb0 tx=0x7faebc00b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faebaffd640 1 -- 172.21.15.37:0/876462523 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc00b870 con 0x7faec40ac8e0 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faebaffd640 1 -- 172.21.15.37:0/876462523 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc00ba30 con 0x7faec40ac8e0 2024-04-25T09:00:37.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecc64f640 1 -- 172.21.15.37:0/876462523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec41f7840 con 0x7faec40ac8e0 2024-04-25T09:00:37.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faebaffd640 1 -- 172.21.15.37:0/876462523 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc00be20 con 0x7faec40ac8e0 2024-04-25T09:00:37.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecc64f640 1 -- 172.21.15.37:0/876462523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec414fcd0 con 0x7faec40ac8e0 2024-04-25T09:00:37.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faecc64f640 1 -- 172.21.15.37:0/876462523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec404efa0 con 0x7faec40ac8e0 2024-04-25T09:00:37.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faebaffd640 1 -- 172.21.15.37:0/876462523 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faebc01b020 con 0x7faec40ac8e0 2024-04-25T09:00:37.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faebaffd640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faea8066fe0 0x7faea80694a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faebaffd640 1 -- 172.21.15.37:0/876462523 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7faebc0850c0 con 0x7faec40ac8e0 2024-04-25T09:00:37.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.427+0000 7faeca3eb640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faea8066fe0 0x7faea80694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.431+0000 7faeca3eb640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faea8066fe0 0x7faea80694a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7faec415a6d0 tx=0x7faec0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.431+0000 7faebaffd640 1 -- 172.21.15.37:0/876462523 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faebc055910 con 0x7faec40ac8e0 2024-04-25T09:00:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 -- 172.21.15.37:0/876462523 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faec406f260 con 0x7faec40ac8e0 2024-04-25T09:00:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faebaffd640 1 -- 172.21.15.37:0/876462523 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4947 (secure 0 0 0) 0x7faebc0599b0 con 0x7faec40ac8e0 2024-04-25T09:00:37.623 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:00:37.623 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T08:59:28.123805+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":98,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7998":{"gid":7998,"name":"a","rank":0,"incarnation":63,"state":"up:active","state_seq":6,"addr":"172.21.15.37:6835/1462327322","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1462327322},{"type":"v1","addr":"172.21.15.37:6835","nonce":1462327322}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:00:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 67 2024-04-25T09:00:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 -- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faea8066fe0 msgr2=0x7faea80694a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faea8066fe0 0x7faea80694a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7faec415a6d0 tx=0x7faec0009040 comp rx=0 tx=0).stop 2024-04-25T09:00:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 -- 172.21.15.37:0/876462523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faec40ac8e0 msgr2=0x7faec415c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faec40ac8e0 0x7faec415c2e0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7faec406cfb0 tx=0x7faebc00b0d0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 -- 172.21.15.37:0/876462523 shutdown_connections 2024-04-25T09:00:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faea8066fe0 0x7faea80694a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faec40ac8e0 0x7faec415c2e0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faec40cb790 0x7faec41f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 --2- 172.21.15.37:0/876462523 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faeb0004810 0x7faec41f0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 -- 172.21.15.37:0/876462523 >> 172.21.15.37:0/876462523 conn(0x7faec40b11a0 msgr2=0x7faec40b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 -- 172.21.15.37:0/876462523 shutdown_connections 2024-04-25T09:00:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.631+0000 7faecc64f640 1 -- 172.21.15.37:0/876462523 wait complete. 2024-04-25T09:00:37.642 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T09:00:37.796 INFO:teuthology.orchestra.run.smithi037.stdout:{"message":"","return_code":0} 2024-04-25T09:00:37.796 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:00:37.797 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:00:37.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f2312eaf640 1 Processor -- start 2024-04-25T09:00:37.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f2312eaf640 1 -- start start 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f2312eaf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f230c15da80 0x7f230c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f2312eaf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f230c0d07f0 0x7f230c06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f2312eaf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f230c06be60 con 0x7f230c0d1cd0 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f2312eaf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f230c06c440 con 0x7f230c15da80 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f2312eaf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230c15de60 con 0x7f230c0d07f0 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f230c0d1cd0 0x7f230c0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52476/0 (socket says 172.21.15.37:52476) 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230bfff640 1 -- 172.21.15.37:0/2689892108 learned_addr learned my addr 172.21.15.37:0/2689892108 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f2310c4b640 1 --2- 172.21.15.37:0/2689892108 >> v2:172.21.15.203:3300/0 conn(0x7f230c15da80 0x7f230c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f231144c640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f230c0d07f0 0x7f230c06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043952305 0 0) 0x7f230c06be60 con 0x7f230c0d1cd0 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22f4003530 con 0x7f230c0d1cd0 2024-04-25T09:00:37.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3319147863 0 0) 0x7f22f4003530 con 0x7f230c0d1cd0 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f230c0d07f0 msgr2=0x7f230c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f230c0d07f0 0x7f230c06c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 >> v2:172.21.15.203:3300/0 conn(0x7f230c15da80 msgr2=0x7f230c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 --2- 172.21.15.37:0/2689892108 >> v2:172.21.15.203:3300/0 conn(0x7f230c15da80 0x7f230c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c0d2450 con 0x7f230c0d1cd0 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f2300001820 con 0x7f230c0d1cd0 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 >> v1:172.21.15.37:6789/0 conn(0x7f230c0d1cd0 legacy=0x7f230c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22f4003a30 0x7f22f4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f4004810 0x7f22f4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f230c0d07f0 0x7f22f40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c06c440 con 0x7f22f4004810 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c15de60 con 0x7f22f4003a30 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f400ba00 con 0x7f230c0d07f0 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.923+0000 7f230bfff640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f4004810 0x7f22f4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f231144c640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f230c0d07f0 0x7f22f40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f2310c4b640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22f4003a30 0x7f22f4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f230bfff640 1 -- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f230c0d07f0 msgr2=0x7f22f40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:37.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f230bfff640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f230c0d07f0 0x7f22f40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f230bfff640 1 -- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22f4003a30 msgr2=0x7f22f4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:37.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f230bfff640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22f4003a30 0x7f22f4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f230bfff640 1 -- 172.21.15.37:0/2689892108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f4003530 con 0x7f22f4004810 2024-04-25T09:00:37.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f2310c4b640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f22f4003a30 0x7f22f4003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:37.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f230bfff640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f4004810 0x7f22f4004c90 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f230c08d9d0 tx=0x7f230001f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff050c874ce2cd64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300007d30 con 0x7f22f4004810 2024-04-25T09:00:37.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300007270 con 0x7f22f4004810 2024-04-25T09:00:37.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f230b7fe640 1 -- 172.21.15.37:0/2689892108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000057b0 con 0x7f22f4004810 2024-04-25T09:00:37.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f2312eaf640 1 -- 172.21.15.37:0/2689892108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f4004810 msgr2=0x7f22f4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f2312eaf640 1 --2- 172.21.15.37:0/2689892108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f22f4004810 0x7f22f4004c90 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f230c08d9d0 tx=0x7f230001f330 comp rx=0 tx=0).stop 2024-04-25T09:00:37.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f231144c640 1 -- 172.21.15.37:0/2689892108 reap_dead start 2024-04-25T09:00:37.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f2312eaf640 1 -- 172.21.15.37:0/2689892108 shutdown_connections 2024-04-25T09:00:37.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f2312eaf640 1 -- 172.21.15.37:0/2689892108 >> 172.21.15.37:0/2689892108 conn(0x7f230c08b000 msgr2=0x7f230c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:37.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f2312eaf640 1 -- 172.21.15.37:0/2689892108 shutdown_connections 2024-04-25T09:00:37.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.927+0000 7f2312eaf640 1 -- 172.21.15.37:0/2689892108 wait complete. 2024-04-25T09:00:37.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2312eaf640 1 Processor -- start 2024-04-25T09:00:37.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2312eaf640 1 -- start start 2024-04-25T09:00:37.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2312eaf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22f4004810 0x7f230c1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2312eaf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c06ca00 0x7f230c1f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2312eaf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f230c1f3f40 0x7f230c15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2312eaf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c0d26f0 con 0x7f230c06ca00 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2312eaf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f230c0d2120 con 0x7f230c1f3f40 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2312eaf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f230c0d2870 con 0x7f22f4004810 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f230bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c06ca00 0x7f230c1f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f230bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c06ca00 0x7f230c1f3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46822/0 (socket says 172.21.15.37:46822) 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f230bfff640 1 -- 172.21.15.37:0/910546963 learned_addr learned my addr 172.21.15.37:0/910546963 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2310c4b640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22f4004810 0x7f230c1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f231144c640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f230c1f3f40 0x7f230c15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f230bfff640 1 -- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22f4004810 msgr2=0x7f230c1542b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f230bfff640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22f4004810 0x7f230c1542b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f230bfff640 1 -- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f230c1f3f40 msgr2=0x7f230c15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f230bfff640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f230c1f3f40 0x7f230c15e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f230bfff640 1 -- 172.21.15.37:0/910546963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f230c160650 con 0x7f230c06ca00 2024-04-25T09:00:37.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f231144c640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f230c1f3f40 0x7f230c15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:37.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f230bfff640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c06ca00 0x7f230c1f3a00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2300000f00 tx=0x7f22f4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f23097fa640 1 -- 172.21.15.37:0/910546963 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000bf070 con 0x7f230c06ca00 2024-04-25T09:00:37.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f23097fa640 1 -- 172.21.15.37:0/910546963 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23000015e0 con 0x7f230c06ca00 2024-04-25T09:00:37.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2312eaf640 1 -- 172.21.15.37:0/910546963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f230c160930 con 0x7f230c06ca00 2024-04-25T09:00:37.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f23097fa640 1 -- 172.21.15.37:0/910546963 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000019b0 con 0x7f230c06ca00 2024-04-25T09:00:37.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2312eaf640 1 -- 172.21.15.37:0/910546963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f230c160270 con 0x7f230c06ca00 2024-04-25T09:00:37.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f2312eaf640 1 -- 172.21.15.37:0/910546963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f230c04efa0 con 0x7f230c06ca00 2024-04-25T09:00:37.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f23097fa640 1 -- 172.21.15.37:0/910546963 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23000c2070 con 0x7f230c06ca00 2024-04-25T09:00:37.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f23097fa640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f22f8066f60 0x7f22f8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.931+0000 7f23097fa640 1 -- 172.21.15.37:0/910546963 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f2300121f70 con 0x7f230c06ca00 2024-04-25T09:00:37.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.935+0000 7f2310c4b640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f22f8066f60 0x7f22f8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.935+0000 7f2310c4b640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f22f8066f60 0x7f22f8069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f230c0cfd10 tx=0x7f22fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:37.939+0000 7f23097fa640 1 -- 172.21.15.37:0/910546963 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23000b7070 con 0x7f230c06ca00 2024-04-25T09:00:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.131+0000 7f2312eaf640 1 -- 172.21.15.37:0/910546963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f230c0b28f0 con 0x7f230c06ca00 2024-04-25T09:00:38.126 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T09:00:38.135+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:00:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.135+0000 7f23097fa640 1 -- 172.21.15.37:0/910546963 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) v1 ==== 161+0+0 (secure 0 0 0) 0x7f23000190a0 con 0x7f230c06ca00 2024-04-25T09:00:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:00:38.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 -- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f22f8066f60 msgr2=0x7f22f8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f22f8066f60 0x7f22f8069420 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f230c0cfd10 tx=0x7f22fc008040 comp rx=0 tx=0).stop 2024-04-25T09:00:38.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 -- 172.21.15.37:0/910546963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c06ca00 msgr2=0x7f230c1f3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c06ca00 0x7f230c1f3a00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f2300000f00 tx=0x7f22f4005840 comp rx=0 tx=0).stop 2024-04-25T09:00:38.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 -- 172.21.15.37:0/910546963 shutdown_connections 2024-04-25T09:00:38.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f22f8066f60 0x7f22f8069420 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f230c1f3f40 0x7f230c15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c06ca00 0x7f230c1f3a00 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 --2- 172.21.15.37:0/910546963 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f22f4004810 0x7f230c1542b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 -- 172.21.15.37:0/910546963 >> 172.21.15.37:0/910546963 conn(0x7f230c08b000 msgr2=0x7f230c0b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 -- 172.21.15.37:0/910546963 shutdown_connections 2024-04-25T09:00:38.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.139+0000 7f2312eaf640 1 -- 172.21.15.37:0/910546963 wait complete. 2024-04-25T09:00:38.139 INFO:tasks.ceph.mds.a.smithi037.stderr: -4770> 2024-04-25T08:59:23.112+0000 7f7c4b029640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:00:38.140 INFO:tasks.ceph.mds.a.smithi037.stderr: -4383> 2024-04-25T08:59:25.260+0000 7f7c47021640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-04-25T09:00:38.140 INFO:tasks.ceph.mds.a.smithi037.stderr: -4221> 2024-04-25T08:59:25.484+0000 7f7c47021640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 ap=1 f(v0 m2024-04-25T08:59:06.530103+0000 1=1+0) n(v0 rc2024-04-25T08:59:06.639471+0000 2=1+1) (iversion lock) | dirfrag=0 authpin=1 scrubqueue=0 0x560a400ebb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T08:59:06.530103+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T08:59:06.639471+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-04-25T09:00:38.144 INFO:tasks.ceph.mds.a.smithi037.stderr: -2501> 2024-04-25T08:59:56.927+0000 7f7c47021640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v13 pv17 ap=3 DIRTYPARENT f(v0 m2024-04-25T08:59:06.530103+0000 1=1+0)->f(v1 m2024-04-25T08:59:06.530103+0000) n(v0 rc2024-04-25T08:59:06.639471+0000 2=1+1)->n(v0 rc2024-04-25T08:59:06.639471+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x560a400ebb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/dir v13>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-04-25T08:59:06.530103+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-04-25T08:59:06.639471+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-04-25T09:00:38.149 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T09:00:38.183 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T09:00:38.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5876e8c0 1 -- start start 2024-04-25T09:00:38.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5876e8c0 1 Processor -- start 2024-04-25T09:00:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5876e8c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56344fc6b800 0x56344fe24580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5876e8c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56344fc6b400 0x56344fe24b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5876e8c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56344fc6b000 0x56344fe25080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5876e8c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344fd00180 con 0x56344fc6b000 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5876e8c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344fd00300 con 0x56344fc6b400 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5876e8c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56344fd00480 con 0x56344fc6b800 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5772e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56344fc6b000 0x56344fe25080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5772e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56344fc6b000 0x56344fe25080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46844/0 (socket says 172.21.15.37:46844) 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5772e640 1 -- 172.21.15.37:0/3989929396 learned_addr learned my addr 172.21.15.37:0/3989929396 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5672c640 1 --2- 172.21.15.37:0/3989929396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56344fc6b400 0x56344fe24b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c56f2d640 1 --2- 172.21.15.37:0/3989929396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56344fc6b800 0x56344fe24580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5772e640 1 -- 172.21.15.37:0/3989929396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56344fc6b800 msgr2=0x56344fe24580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5772e640 1 --2- 172.21.15.37:0/3989929396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56344fc6b800 0x56344fe24580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5772e640 1 -- 172.21.15.37:0/3989929396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56344fc6b400 msgr2=0x56344fe24b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5772e640 1 --2- 172.21.15.37:0/3989929396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56344fc6b400 0x56344fe24b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c5772e640 1 -- 172.21.15.37:0/3989929396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56344fc901e0 con 0x56344fc6b000 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.191+0000 7f8c56f2d640 1 --2- 172.21.15.37:0/3989929396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56344fc6b800 0x56344fe24580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5772e640 1 --2- 172.21.15.37:0/3989929396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56344fc6b000 0x56344fe25080 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x563450ab1860 tx=0x56344fd24c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79e2a06eb39cf25b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c54728640 1 -- 172.21.15.37:0/3989929396 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56344fca04e0 con 0x56344fc6b000 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c54728640 1 -- 172.21.15.37:0/3989929396 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56344fcb4e00 con 0x56344fc6b000 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c54728640 1 -- 172.21.15.37:0/3989929396 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56344fca0b60 con 0x56344fc6b000 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3989929396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56344fc914a0 con 0x56344fc6b000 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3989929396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56344fc91a40 con 0x56344fc6b000 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 4 waiting for MDS map... 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c54728640 1 -- 172.21.15.37:0/3989929396 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x563450b7e600 con 0x56344fc6b000 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c54728640 1 -- 172.21.15.37:0/3989929396 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 68) v1 ==== 2723+0+0 (secure 0 0 0) 0x56344fcc1180 con 0x56344fc6b000 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 4 Got MDS map 68 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 4 datascan.main: Using metadata pool output 2024-04-25T09:00:38.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 4 datascan.main: connecting to RADOS... 2024-04-25T09:00:38.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 Processor -- start 2024-04-25T09:00:38.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 -- start start 2024-04-25T09:00:38.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563450b82c00 0x56344fe25600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563450b83000 0x56344fe25b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344fd00600 con 0x563450b83400 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344fd00780 con 0x563450b82c00 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56344fd00900 con 0x563450b83000 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5672c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5672c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46858/0 (socket says 172.21.15.37:46858) 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5672c640 1 -- 172.21.15.37:0/3817351965 learned_addr learned my addr 172.21.15.37:0/3817351965 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c56f2d640 1 --2- 172.21.15.37:0/3817351965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563450b83000 0x56344fe25b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5672c640 1 --2- 172.21.15.37:0/3817351965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563450b82c00 0x56344fe25600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5672c640 1 -- 172.21.15.37:0/3817351965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563450b83000 msgr2=0x56344fe25b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5672c640 1 --2- 172.21.15.37:0/3817351965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563450b83000 0x56344fe25b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5672c640 1 -- 172.21.15.37:0/3817351965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563450b82c00 msgr2=0x56344fe25600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5672c640 1 --2- 172.21.15.37:0/3817351965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563450b82c00 0x56344fe25600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5672c640 1 -- 172.21.15.37:0/3817351965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56344fc91c20 con 0x563450b83400 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5672c640 1 --2- 172.21.15.37:0/3817351965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26100 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x563450b72cc0 tx=0x563450b9ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f03232951e72a6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c52724640 1 -- 172.21.15.37:0/3817351965 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56344fca04e0 con 0x563450b83400 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c52724640 1 -- 172.21.15.37:0/3817351965 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56344fcb4540 con 0x563450b83400 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c52724640 1 -- 172.21.15.37:0/3817351965 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56344fca0b60 con 0x563450b83400 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3817351965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 msgr2=0x56344fe26100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/3817351965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26100 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x563450b72cc0 tx=0x563450b9ac00 comp rx=0 tx=0).stop 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3817351965 shutdown_connections 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/3817351965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26100 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/3817351965 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563450b83000 0x56344fe25b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/3817351965 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563450b82c00 0x56344fe25600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3817351965 >> 172.21.15.37:0/3817351965 conn(0x56344fc6ac00 msgr2=0x56344fc99800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3817351965 shutdown_connections 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.195+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3817351965 wait complete. 2024-04-25T09:00:38.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 Processor -- start 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 -- start start 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563450b82c00 0x56344fe27180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563450b83000 0x56344fe26c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344fd01c80 con 0x563450b83400 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56344fd01b00 con 0x563450b83000 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56344fd01980 con 0x563450b82c00 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46862/0 (socket says 172.21.15.37:46862) 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 learned_addr learned my addr 172.21.15.37:0/384082263 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c56f2d640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563450b83000 0x56344fe26c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563450b82c00 0x56344fe27180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563450b82c00 msgr2=0x56344fe27180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563450b82c00 0x56344fe27180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563450b83000 msgr2=0x56344fe26c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563450b83000 0x56344fe26c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56344fc901e0 con 0x563450b83400 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26680 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x563450b73980 tx=0x563450b9ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c50f21640 1 -- 172.21.15.37:0/384082263 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56344fca0000 con 0x563450b83400 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c50f21640 1 -- 172.21.15.37:0/384082263 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56344fcb4c40 con 0x563450b83400 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c50f21640 1 -- 172.21.15.37:0/384082263 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56344fca04e0 con 0x563450b83400 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56344fc914a0 con 0x563450b83400 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56344fc91e00 con 0x563450b83400 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 4 datascan.init: resolving metadata pool 13 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c50f21640 1 -- 172.21.15.37:0/384082263 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x563450bf2c00 con 0x563450b83400 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c50f21640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56344ff96c00 0x56344fe27700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c50f21640 1 -- 172.21.15.37:0/384082263 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x563450bf0a00 con 0x563450b83400 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-25T09:00:38.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T09:00:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x563450b83800 0x56344fe26100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450b83c00 con 0x563450b83800 2024-04-25T09:00:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x563450b83800 0x56344fe26100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c56f2d640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56344ff96c00 0x56344fe27700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.199+0000 7f8c5672c640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x563450b83800 0x56344fe26100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.203+0000 7f8c56f2d640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56344ff96c00 0x56344fe27700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x56344fd461e0 tx=0x56344fd68000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.203+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.2 v2:172.21.15.37:6808/332967247 1 ==== osd_op_reply(1 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x563450b83800 2024-04-25T09:00:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.203+0000 7f8c51f23640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56344ff97000 0x563450096000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.203+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x56344ff97400 con 0x56344ff97000 2024-04-25T09:00:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.203+0000 7f8c5772e640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56344ff97000 0x563450096000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.203+0000 7f8c5772e640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56344ff97000 0x563450096000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== osd_op_reply(2 [pgnls start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c52724640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x56344ff97c00 0x563450096580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a0000 con 0x56344ff97c00 2024-04-25T09:00:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c56f2d640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x56344ff97c00 0x563450096580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c56f2d640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x56344ff97c00 0x563450096580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(3 [pgnls start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x56344ff97c00 2024-04-25T09:00:38.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x56344ff97800 con 0x56344ff97000 2024-04-25T09:00:38.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 2 ==== osd_op_reply(4 [pgnls start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a0400 con 0x56344ff97000 2024-04-25T09:00:38.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 3 ==== osd_op_reply(5 [pgnls start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c51f23640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x56344ff97400 0x563450096b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450b82800 con 0x56344ff97400 2024-04-25T09:00:38.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c5672c640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x56344ff97400 0x563450096b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c5672c640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x56344ff97400 0x563450096b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.3 v2:172.21.15.37:6800/4126478460 1 ==== osd_op_reply(6 [pgnls start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x56344ff97400 2024-04-25T09:00:38.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.207+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a0400 con 0x56344ff97c00 2024-04-25T09:00:38.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.1 v2:172.21.15.37:6816/2944303862 2 ==== osd_op_reply(7 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x56344ff97c00 2024-04-25T09:00:38.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450b82400 con 0x56344ff97c00 2024-04-25T09:00:38.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.1 v2:172.21.15.37:6816/2944303862 3 ==== osd_op_reply(8 [pgnls start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x56344ff97c00 2024-04-25T09:00:38.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c52724640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5634500a0800 0x563450097600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x56344ff97800 con 0x5634500a0800 2024-04-25T09:00:38.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c5772e640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5634500a0800 0x563450097600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c5772e640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5634500a0800 0x563450097600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(9 [pgnls start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x5634500a0800 2024-04-25T09:00:38.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a0000 con 0x56344ff97c00 2024-04-25T09:00:38.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.1 v2:172.21.15.37:6816/2944303862 4 ==== osd_op_reply(10 [pgnls start_epoch 99 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x56344ff97c00 2024-04-25T09:00:38.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c52724640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x5634500a0400 0x563450097b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a0c00 con 0x5634500a0400 2024-04-25T09:00:38.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c56f2d640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x5634500a0400 0x563450097b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.211+0000 7f8c56f2d640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x5634500a0400 0x563450097b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 1 ==== osd_op_reply(11 [pgnls start_epoch 99 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450b82400 con 0x563450b83800 2024-04-25T09:00:38.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.2 v2:172.21.15.37:6808/332967247 2 ==== osd_op_reply(12 [pgnls start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x563450b83800 2024-04-25T09:00:38.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1000 con 0x56344ff97000 2024-04-25T09:00:38.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 4 ==== osd_op_reply(13 [pgnls start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450a3b400 con 0x56344ff97c00 2024-04-25T09:00:38.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.1 v2:172.21.15.37:6816/2944303862 5 ==== osd_op_reply(14 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x56344ff97c00 2024-04-25T09:00:38.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1400 con 0x56344ff97000 2024-04-25T09:00:38.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 5 ==== osd_op_reply(15 [pgnls start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450a3b000 con 0x56344ff97c00 2024-04-25T09:00:38.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.1 v2:172.21.15.37:6816/2944303862 6 ==== osd_op_reply(16 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x56344ff97c00 2024-04-25T09:00:38.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1800 con 0x5634500a0800 2024-04-25T09:00:38.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.6 v2:172.21.15.203:6800/467003745 2 ==== osd_op_reply(17 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x5634500a0800 2024-04-25T09:00:38.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b0000 con 0x5634500a0400 2024-04-25T09:00:38.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 2 ==== osd_op_reply(18 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1c00 con 0x563450b83800 2024-04-25T09:00:38.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.2 v2:172.21.15.37:6808/332967247 3 ==== osd_op_reply(19 [pgnls start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x563450b83800 2024-04-25T09:00:38.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b0400 con 0x5634500a0400 2024-04-25T09:00:38.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 3 ==== osd_op_reply(20 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c52724640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x5634500a0000 0x563450098100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.219+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450b83c00 con 0x5634500a0000 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c5672c640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x5634500a0000 0x563450098100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c5672c640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x5634500a0000 0x563450098100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== osd_op_reply(21 [pgnls start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x5634500a0000 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b0800 con 0x5634500a0400 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 4 ==== osd_op_reply(22 [pgnls start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:23 14.1a 14:58000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450b82800 con 0x56344ff97000 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 6 ==== osd_op_reply(23 [pgnls start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:24 14.3a 14:5c000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b0c00 con 0x56344ff97000 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 7 ==== osd_op_reply(24 [pgnls start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:25 14.6 14:60000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450b82400 con 0x56344ff97000 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 8 ==== osd_op_reply(25 [pgnls start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:26 14.26 14:64000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b1000 con 0x5634500a0400 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 5 ==== osd_op_reply(26 [pgnls start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.223+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:27 14.16 14:68000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450a3b400 con 0x56344ff97400 2024-04-25T09:00:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.227+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.3 v2:172.21.15.37:6800/4126478460 2 ==== osd_op_reply(27 [pgnls start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x56344ff97400 2024-04-25T09:00:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.227+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:28 14.36 14:6c000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1000 con 0x563450b83800 2024-04-25T09:00:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.227+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.2 v2:172.21.15.37:6808/332967247 4 ==== osd_op_reply(28 [pgnls start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x563450b83800 2024-04-25T09:00:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.227+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:29 14.e 14:70000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450a3b000 con 0x5634500a0400 2024-04-25T09:00:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.227+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 6 ==== osd_op_reply(29 [pgnls start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.227+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:30 14.2e 14:74000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1400 con 0x56344ff97400 2024-04-25T09:00:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.227+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.3 v2:172.21.15.37:6800/4126478460 3 ==== osd_op_reply(30 [pgnls start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x56344ff97400 2024-04-25T09:00:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.227+0000 7f8c52724640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5634500b0c00 0x563450098680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.227+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:31 14.1e 14:78000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b0000 con 0x5634500b0c00 2024-04-25T09:00:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.227+0000 7f8c5772e640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5634500b0c00 0x563450098680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.227+0000 7f8c5772e640 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5634500b0c00 0x563450098680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.231+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== osd_op_reply(31 [pgnls start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x5634500b0c00 2024-04-25T09:00:38.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.231+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:32 14.3e 14:7c000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1800 con 0x5634500a0000 2024-04-25T09:00:38.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.231+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.5 v2:172.21.15.203:6808/213476662 2 ==== osd_op_reply(32 [pgnls start_epoch 99 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x5634500a0000 2024-04-25T09:00:38.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.231+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:33 14.1 14:80000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b0400 con 0x56344ff97400 2024-04-25T09:00:38.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.231+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.3 v2:172.21.15.37:6800/4126478460 4 ==== osd_op_reply(33 [pgnls start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x56344ff97400 2024-04-25T09:00:38.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.231+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:34 14.21 14:84000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a0c00 con 0x5634500a0000 2024-04-25T09:00:38.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.5 v2:172.21.15.203:6808/213476662 3 ==== osd_op_reply(34 [pgnls start_epoch 99 out=49b] v95'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x5634500a0000 2024-04-25T09:00:38.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:35 14.11 14:88000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b1400 con 0x5634500a0400 2024-04-25T09:00:38.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 7 ==== osd_op_reply(35 [pgnls start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:36 14.31 14:8c000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450b82800 con 0x56344ff97c00 2024-04-25T09:00:38.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.1 v2:172.21.15.37:6816/2944303862 7 ==== osd_op_reply(36 [pgnls start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x56344ff97c00 2024-04-25T09:00:38.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:37 14.9 14:90000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b1800 con 0x5634500a0000 2024-04-25T09:00:38.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.5 v2:172.21.15.203:6808/213476662 4 ==== osd_op_reply(37 [pgnls start_epoch 99 out=49b] v95'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x5634500a0000 2024-04-25T09:00:38.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:38 14.29 14:94000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x56344ff97800 con 0x5634500a0800 2024-04-25T09:00:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.6 v2:172.21.15.203:6800/467003745 3 ==== osd_op_reply(38 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x5634500a0800 2024-04-25T09:00:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:39 14.19 14:98000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b1c00 con 0x5634500a0800 2024-04-25T09:00:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.6 v2:172.21.15.203:6800/467003745 4 ==== osd_op_reply(39 [pgnls start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x5634500a0800 2024-04-25T09:00:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.235+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:40 14.39 14:9c000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1000 con 0x5634500b0c00 2024-04-25T09:00:38.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.7 v2:172.21.15.203:6824/2957998038 2 ==== osd_op_reply(40 [pgnls start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x5634500b0c00 2024-04-25T09:00:38.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:41 14.5 14:a0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1c00 con 0x56344ff97000 2024-04-25T09:00:38.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 9 ==== osd_op_reply(41 [pgnls start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:42 14.25 14:a4000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1400 con 0x563450b83800 2024-04-25T09:00:38.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.2 v2:172.21.15.37:6808/332967247 5 ==== osd_op_reply(42 [pgnls start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x563450b83800 2024-04-25T09:00:38.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:43 14.15 14:a8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500c2000 con 0x5634500b0c00 2024-04-25T09:00:38.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.7 v2:172.21.15.203:6824/2957998038 3 ==== osd_op_reply(43 [pgnls start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x5634500b0c00 2024-04-25T09:00:38.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:44 14.35 14:ac000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1800 con 0x5634500a0400 2024-04-25T09:00:38.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 8 ==== osd_op_reply(44 [pgnls start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:45 14.d 14:b0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500c2400 con 0x5634500a0800 2024-04-25T09:00:38.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.6 v2:172.21.15.203:6800/467003745 5 ==== osd_op_reply(45 [pgnls start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x5634500a0800 2024-04-25T09:00:38.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:46 14.2d 14:b4000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450a3b400 con 0x5634500a0400 2024-04-25T09:00:38.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 9 ==== osd_op_reply(46 [pgnls start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.239+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:47 14.1d 14:b8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500c2800 con 0x56344ff97000 2024-04-25T09:00:38.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 10 ==== osd_op_reply(47 [pgnls start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:48 14.3d 14:bc000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450b83c00 con 0x56344ff97400 2024-04-25T09:00:38.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.3 v2:172.21.15.37:6800/4126478460 5 ==== osd_op_reply(48 [pgnls start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x56344ff97400 2024-04-25T09:00:38.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:49 14.3 14:c0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500c2c00 con 0x56344ff97000 2024-04-25T09:00:38.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 11 ==== osd_op_reply(49 [pgnls start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:50 14.23 14:c4000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b0800 con 0x5634500b0c00 2024-04-25T09:00:38.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.7 v2:172.21.15.203:6824/2957998038 4 ==== osd_op_reply(50 [pgnls start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x5634500b0c00 2024-04-25T09:00:38.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:51 14.13 14:c8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500c3000 con 0x563450b83800 2024-04-25T09:00:38.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.2 v2:172.21.15.37:6808/332967247 6 ==== osd_op_reply(51 [pgnls start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x563450b83800 2024-04-25T09:00:38.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:52 14.33 14:cc000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b1000 con 0x56344ff97c00 2024-04-25T09:00:38.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.1 v2:172.21.15.37:6816/2944303862 8 ==== osd_op_reply(52 [pgnls start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x56344ff97c00 2024-04-25T09:00:38.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:53 14.b 14:d0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500c3400 con 0x56344ff97000 2024-04-25T09:00:38.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 12 ==== osd_op_reply(53 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:54 14.2b 14:d4000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b1400 con 0x5634500a0400 2024-04-25T09:00:38.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 10 ==== osd_op_reply(54 [pgnls start_epoch 99 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:55 14.1b 14:d8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500c3800 con 0x5634500a0000 2024-04-25T09:00:38.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.5 v2:172.21.15.203:6808/213476662 5 ==== osd_op_reply(55 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x5634500a0000 2024-04-25T09:00:38.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.243+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:56 14.3b 14:dc000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x56344ff97800 con 0x5634500a0400 2024-04-25T09:00:38.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.247+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 11 ==== osd_op_reply(56 [pgnls start_epoch 99 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.247+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:57 14.7 14:e0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500c3c00 con 0x5634500a0000 2024-04-25T09:00:38.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.247+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.5 v2:172.21.15.203:6808/213476662 6 ==== osd_op_reply(57 [pgnls start_epoch 99 out=81b] v97'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56344fd1e000 con 0x5634500a0000 2024-04-25T09:00:38.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.247+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:58 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a0c00 con 0x5634500a0000 2024-04-25T09:00:38.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.247+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.5 v2:172.21.15.203:6808/213476662 7 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x56344fd1e000 con 0x5634500a0000 2024-04-25T09:00:38.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.247+0000 7f8c5876e8c0 10 datascan.operator(): handling object 1099511627777.0 2024-04-25T09:00:38.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.247+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:59 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1400 con 0x5634500a0000 2024-04-25T09:00:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.5 v2:172.21.15.203:6808/213476662 8 ==== osd_op_reply(59 10000000001.00000000 [call] v99'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56344fd1e000 con 0x5634500a0000 2024-04-25T09:00:38.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:60 14.27 14:e4000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500b0400 con 0x5634500a0000 2024-04-25T09:00:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.5 v2:172.21.15.203:6808/213476662 9 ==== osd_op_reply(60 [pgnls start_epoch 99 out=49b] v95'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x5634500a0000 2024-04-25T09:00:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:61 14.17 14:e8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500c3000 con 0x56344ff97c00 2024-04-25T09:00:38.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.1 v2:172.21.15.37:6816/2944303862 9 ==== osd_op_reply(61 [pgnls start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x56344ff97c00 2024-04-25T09:00:38.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:62 14.37 14:ec000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450b82800 con 0x5634500a0400 2024-04-25T09:00:38.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 <== osd.4 v2:172.21.15.203:6802/714177927 12 ==== osd_op_reply(62 [pgnls start_epoch 99 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e780 con 0x5634500a0400 2024-04-25T09:00:38.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:63 14.f 14:f0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500c2000 con 0x5634500b0c00 2024-04-25T09:00:38.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.7 v2:172.21.15.203:6824/2957998038 5 ==== osd_op_reply(63 [pgnls start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x5634500b0c00 2024-04-25T09:00:38.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:64 14.2f 14:f4000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450b82400 con 0x56344ff97000 2024-04-25T09:00:38.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 13 ==== osd_op_reply(64 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.251+0000 7f8c52724640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:65 14.1f 14:f8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x5634500a1800 con 0x56344ff97000 2024-04-25T09:00:38.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5772e640 1 -- 172.21.15.37:0/384082263 <== osd.0 v2:172.21.15.37:6824/1116141266 14 ==== osd_op_reply(65 [pgnls start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e500 con 0x56344ff97000 2024-04-25T09:00:38.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c51f23640 1 -- 172.21.15.37:0/384082263 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:66 14.3f 14:fc000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x563450a3b000 con 0x56344ff97400 2024-04-25T09:00:38.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5672c640 1 -- 172.21.15.37:0/384082263 <== osd.3 v2:172.21.15.37:6800/4126478460 6 ==== osd_op_reply(66 [pgnls start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56344fd1e000 con 0x56344ff97400 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56344ff97000 msgr2=0x563450096000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56344ff97000 0x563450096000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x56344ff97c00 msgr2=0x563450096580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x56344ff97c00 0x563450096580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x563450b83800 msgr2=0x56344fe26100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x563450b83800 0x56344fe26100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x56344ff97400 msgr2=0x563450096b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x56344ff97400 0x563450096b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x5634500a0400 msgr2=0x563450097b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x5634500a0400 0x563450097b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x5634500a0000 msgr2=0x563450098100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x5634500a0000 0x563450098100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5634500a0800 msgr2=0x563450097600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5634500a0800 0x563450097600 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5634500b0c00 msgr2=0x563450098680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 reap_dead start 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5634500b0c00 0x563450098680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 reap_dead start 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 reap_dead start 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c56f2d640 1 -- 172.21.15.37:0/384082263 reap_dead start 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56344ff96c00 msgr2=0x56344fe27700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56344ff96c00 0x56344fe27700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x56344fd461e0 tx=0x56344fd68000 comp rx=0 tx=0).stop 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 msgr2=0x56344fe26680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26680 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x563450b73980 tx=0x563450b9ac60 comp rx=0 tx=0).stop 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 shutdown_connections 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56344ff96c00 0x56344fe27700 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/384082263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563450b83400 0x56344fe26680 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 >> 172.21.15.37:0/384082263 conn(0x56344fc6ac00 msgr2=0x56344fc99800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 shutdown_connections 2024-04-25T09:00:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/384082263 wait complete. 2024-04-25T09:00:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3989929396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56344fc6b000 msgr2=0x56344fe25080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/3989929396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56344fc6b000 0x56344fe25080 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x563450ab1860 tx=0x56344fd24c60 comp rx=0 tx=0).stop 2024-04-25T09:00:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3989929396 shutdown_connections 2024-04-25T09:00:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/3989929396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56344fc6b000 0x56344fe25080 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/3989929396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56344fc6b400 0x56344fe24b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 --2- 172.21.15.37:0/3989929396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56344fc6b800 0x56344fe24580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3989929396 >> 172.21.15.37:0/3989929396 conn(0x56344fc6bc00 msgr2=0x56344fc99000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3989929396 shutdown_connections 2024-04-25T09:00:38.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.255+0000 7f8c5876e8c0 1 -- 172.21.15.37:0/3989929396 wait complete. 2024-04-25T09:00:38.251 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.101894, result: 2024-04-25T09:00:38.251 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-04-25T09:00:38.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 1 -- start start 2024-04-25T09:00:38.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 1 Processor -- start 2024-04-25T09:00:38.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c3772b3000 0x55c37746db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3772b3400 0x55c37746d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3772b3800 0x55c37746d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378096180 con 0x55c3772b3800 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c377312900 con 0x55c3772b3400 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c377312480 con 0x55c3772b3000 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b89c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3772b3800 0x55c37746d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b89c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3772b3800 0x55c37746d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46874/0 (socket says 172.21.15.37:46874) 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b89c0640 1 -- 172.21.15.37:0/1148232153 learned_addr learned my addr 172.21.15.37:0/1148232153 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b81bf640 1 --2- 172.21.15.37:0/1148232153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c3772b3000 0x55c37746db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b79be640 1 --2- 172.21.15.37:0/1148232153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3772b3400 0x55c37746d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b89c0640 1 -- 172.21.15.37:0/1148232153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c3772b3000 msgr2=0x55c37746db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b89c0640 1 --2- 172.21.15.37:0/1148232153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c3772b3000 0x55c37746db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b89c0640 1 -- 172.21.15.37:0/1148232153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3772b3400 msgr2=0x55c37746d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b89c0640 1 --2- 172.21.15.37:0/1148232153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3772b3400 0x55c37746d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b89c0640 1 -- 172.21.15.37:0/1148232153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3772d81e0 con 0x55c3772b3800 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b89c0640 1 --2- 172.21.15.37:0/1148232153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3772b3800 0x55c37746d080 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x55c378105470 tx=0x55c37734a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3837cbe6cd78e2bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b59ba640 1 -- 172.21.15.37:0/1148232153 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3772e89c0 con 0x55c3772b3800 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b59ba640 1 -- 172.21.15.37:0/1148232153 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3772fc1c0 con 0x55c3772b3800 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b59ba640 1 -- 172.21.15.37:0/1148232153 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3772e84e0 con 0x55c3772b3800 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 1 -- 172.21.15.37:0/1148232153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3772d94a0 con 0x55c3772b3800 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 1 -- 172.21.15.37:0/1148232153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c3772d83c0 con 0x55c3772b3800 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 4 waiting for MDS map... 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b59ba640 1 -- 172.21.15.37:0/1148232153 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x55c3781d0200 con 0x55c3772b3800 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b59ba640 1 -- 172.21.15.37:0/1148232153 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 68) v1 ==== 2723+0+0 (secure 0 0 0) 0x55c377309180 con 0x55c3772b3800 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 4 Got MDS map 68 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 4 datascan.main: Using metadata pool output 2024-04-25T09:00:38.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.335+0000 7f08b9a008c0 4 datascan.main: connecting to RADOS... 2024-04-25T09:00:38.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 Processor -- start 2024-04-25T09:00:38.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- start start 2024-04-25T09:00:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c378081800 0x55c37746cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378081c00 0x55c37746c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c378236000 0x55c37746e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c377312180 con 0x55c378081c00 2024-04-25T09:00:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378096d80 con 0x55c378236000 2024-04-25T09:00:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c378096f00 con 0x55c378081800 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b81bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378081c00 0x55c37746c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b81bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378081c00 0x55c37746c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46878/0 (socket says 172.21.15.37:46878) 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b81bf640 1 -- 172.21.15.37:0/2409076780 learned_addr learned my addr 172.21.15.37:0/2409076780 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b79be640 1 --2- 172.21.15.37:0/2409076780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c378236000 0x55c37746e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b79be640 1 --2- 172.21.15.37:0/2409076780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c378081800 0x55c37746cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b81bf640 1 -- 172.21.15.37:0/2409076780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c378081800 msgr2=0x55c37746cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b81bf640 1 --2- 172.21.15.37:0/2409076780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c378081800 0x55c37746cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b81bf640 1 -- 172.21.15.37:0/2409076780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c378236000 msgr2=0x55c37746e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b81bf640 1 --2- 172.21.15.37:0/2409076780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c378236000 0x55c37746e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b81bf640 1 -- 172.21.15.37:0/2409076780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3772d81e0 con 0x55c378081c00 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b81bf640 1 --2- 172.21.15.37:0/2409076780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378081c00 0x55c37746c580 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x55c378094840 tx=0x55c3781e4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f082848ecf93472d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b39b6640 1 -- 172.21.15.37:0/2409076780 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3772e89c0 con 0x55c378081c00 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b39b6640 1 -- 172.21.15.37:0/2409076780 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3772fcc40 con 0x55c378081c00 2024-04-25T09:00:38.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b39b6640 1 -- 172.21.15.37:0/2409076780 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3772e84e0 con 0x55c378081c00 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- 172.21.15.37:0/2409076780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378081c00 msgr2=0x55c37746c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/2409076780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378081c00 0x55c37746c580 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x55c378094840 tx=0x55c3781e4c00 comp rx=0 tx=0).stop 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- 172.21.15.37:0/2409076780 shutdown_connections 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/2409076780 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c378236000 0x55c37746e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/2409076780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378081c00 0x55c37746c580 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/2409076780 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c378081800 0x55c37746cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- 172.21.15.37:0/2409076780 >> 172.21.15.37:0/2409076780 conn(0x55c378081400 msgr2=0x55c3772e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- 172.21.15.37:0/2409076780 shutdown_connections 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- 172.21.15.37:0/2409076780 wait complete. 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 Processor -- start 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- start start 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c378081c00 0x55c37746cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c378081800 0x55c37746c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378237000 0x55c37746e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378097200 con 0x55c378237000 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c378097080 con 0x55c378081800 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b9a008c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c378097380 con 0x55c378081c00 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b79be640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378237000 0x55c37746e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b79be640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378237000 0x55c37746e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46894/0 (socket says 172.21.15.37:46894) 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 learned_addr learned my addr 172.21.15.37:0/720830454 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b79be640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c378081c00 0x55c37746cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.339+0000 7f08b81bf640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c378081800 0x55c37746c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c378081c00 msgr2=0x55c37746cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b79be640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c378081c00 0x55c37746cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c378081800 msgr2=0x55c37746c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b79be640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c378081800 0x55c37746c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3772d92c0 con 0x55c378237000 2024-04-25T09:00:38.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b81bf640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c378081800 0x55c37746c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b79be640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378237000 0x55c37746e100 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x55c378095b60 tx=0x55c3781e4d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b21b3640 1 -- 172.21.15.37:0/720830454 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3772e89c0 con 0x55c378237000 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b21b3640 1 -- 172.21.15.37:0/720830454 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3772fc540 con 0x55c378237000 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b21b3640 1 -- 172.21.15.37:0/720830454 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c3772e84e0 con 0x55c378237000 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c3772d9a40 con 0x55c378237000 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3772d9c20 con 0x55c378237000 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b9a008c0 4 datascan.init: resolving metadata pool 13 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b21b3640 1 -- 172.21.15.37:0/720830454 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55c3775e0c00 con 0x55c378237000 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b21b3640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55c3775f4c00 0x55c3776d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b21b3640 1 -- 172.21.15.37:0/720830454 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x55c3775dac00 con 0x55c378237000 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b9a008c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b9a008c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b9a008c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55c378236c00 0x55c37746e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236800 con 0x55c378236c00 2024-04-25T09:00:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b79be640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55c378236c00 0x55c37746e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b81bf640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55c3775f4c00 0x55c3776d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b79be640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55c378236c00 0x55c37746e680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55c378236400 0x55c37746ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236000 con 0x55c378236400 2024-04-25T09:00:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b81bf640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55c3775f4c00 0x55c3776d0000 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55c378094840 tx=0x55c37734a120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.343+0000 7f08b89c0640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55c378236400 0x55c37746ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b89c0640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55c378236400 0x55c37746ec00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55c3775f5000 0x55c37746f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3775f5400 con 0x55c3775f5000 2024-04-25T09:00:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b81bf640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55c3775f5000 0x55c37746f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b81bf640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55c3775f5000 0x55c37746f180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ee000 con 0x55c378236c00 2024-04-25T09:00:38.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b39b6640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c3776ee400 0x55c3776d0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ee800 con 0x55c3776ee400 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b79be640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c3776ee400 0x55c3776d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b79be640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c3776ee400 0x55c3776d0580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ef000 con 0x55c378236c00 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776eec00 con 0x55c378236c00 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b31b5640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55c3776ef400 0x55c3776d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ee000 con 0x55c3776ef400 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b89c0640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55c3776ef400 0x55c3776d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.347+0000 7f08b89c0640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55c3776ef400 0x55c3776d0b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.3 v2:172.21.15.37:6800/4126478460 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ef400 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ef000 con 0x55c3776ee400 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ee800 con 0x55c3776ee400 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ef800 con 0x55c378236400 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236800 con 0x55c3776ee400 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 99 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b39b6640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55c3776efc00 0x55c3776d1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706000 con 0x55c3776efc00 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b81bf640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55c3776efc00 0x55c3776d1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b81bf640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55c3776efc00 0x55c3776d1080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 99 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707000 con 0x55c3775f5000 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 2 ==== osd_op_reply(14 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706400 con 0x55c378236c00 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.351+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:16 14.2c 14:34000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707400 con 0x55c3776ee400 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 5 ==== osd_op_reply(16 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:17 14.1c 14:38000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706800 con 0x55c378236c00 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:18 14.3c 14:3c000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707800 con 0x55c3776ee400 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 6 ==== osd_op_reply(18 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:19 14.2 14:40000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706c00 con 0x55c378236400 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:20 14.22 14:44000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707c00 con 0x55c3776efc00 2024-04-25T09:00:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:21 14.12 14:48000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ee000 con 0x55c3775f5000 2024-04-25T09:00:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:22 14.32 14:4c000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3775f5400 con 0x55c3776efc00 2024-04-25T09:00:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b39b6640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55c3776ef800 0x55c3776d1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:23 14.a 14:50000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776eec00 con 0x55c3776ef800 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.355+0000 7f08b79be640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55c3776ef800 0x55c3776d1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b79be640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55c3776ef800 0x55c3776d1600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== osd_op_reply(23 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:24 14.2a 14:54000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236000 con 0x55c3776efc00 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ef000 con 0x55c378236c00 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 7 ==== osd_op_reply(25 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236800 con 0x55c378236c00 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 8 ==== osd_op_reply(26 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378081000 con 0x55c378236c00 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 9 ==== osd_op_reply(27 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378237c00 con 0x55c3776efc00 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236000 con 0x55c3776ef400 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.3 v2:172.21.15.37:6800/4126478460 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ef400 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ee800 con 0x55c3775f5000 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:31 14.e 14:70000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378237c00 con 0x55c3776efc00 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 6 ==== osd_op_reply(31 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.359+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:32 14.2e 14:74000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236800 con 0x55c3776ef400 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.3 v2:172.21.15.37:6800/4126478460 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ef400 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b39b6640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55c3776ee000 0x55c3776d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:33 14.1e 14:78000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776eec00 con 0x55c3776ee000 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b89c0640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55c3776ee000 0x55c3776d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b89c0640 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55c3776ee000 0x55c3776d1b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ee000 2024-04-25T09:00:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:34 14.3e 14:7c000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706000 con 0x55c3776ef800 2024-04-25T09:00:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 2 ==== osd_op_reply(34 [pgnls-filter start_epoch 99 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:35 14.1 14:80000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ef000 con 0x55c3776ef400 2024-04-25T09:00:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.3 v2:172.21.15.37:6800/4126478460 4 ==== osd_op_reply(35 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ef400 2024-04-25T09:00:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:36 14.21 14:84000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706400 con 0x55c3776ef800 2024-04-25T09:00:38.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 99 out=49b] v95'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:37 14.11 14:88000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3775f5400 con 0x55c3776efc00 2024-04-25T09:00:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.363+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:38 14.31 14:8c000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706800 con 0x55c3776ee400 2024-04-25T09:00:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 7 ==== osd_op_reply(38 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:39 14.9 14:90000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707c00 con 0x55c3776ef800 2024-04-25T09:00:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 4 ==== osd_op_reply(39 [pgnls-filter start_epoch 99 out=49b] v95'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:40 14.29 14:94000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706c00 con 0x55c378236400 2024-04-25T09:00:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:41 14.19 14:98000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c000 con 0x55c378236400 2024-04-25T09:00:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:42 14.39 14:9c000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707000 con 0x55c3776ee000 2024-04-25T09:00:38.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.7 v2:172.21.15.203:6824/2957998038 2 ==== osd_op_reply(42 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ee000 2024-04-25T09:00:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:43 14.5 14:a0000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c400 con 0x55c378236c00 2024-04-25T09:00:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 10 ==== osd_op_reply(43 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:44 14.25 14:a4000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707400 con 0x55c3775f5000 2024-04-25T09:00:38.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.367+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:45 14.15 14:a8000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c800 con 0x55c3776ee000 2024-04-25T09:00:38.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.7 v2:172.21.15.203:6824/2957998038 3 ==== osd_op_reply(45 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ee000 2024-04-25T09:00:38.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:46 14.35 14:ac000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707800 con 0x55c3776efc00 2024-04-25T09:00:38.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:47 14.d 14:b0000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771cc00 con 0x55c378236400 2024-04-25T09:00:38.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:48 14.2d 14:b4000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c000 con 0x55c3776efc00 2024-04-25T09:00:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:49 14.1d 14:b8000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d000 con 0x55c378236c00 2024-04-25T09:00:38.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 11 ==== osd_op_reply(49 [pgnls-filter start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:50 14.3d 14:bc000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706c00 con 0x55c3776ef400 2024-04-25T09:00:38.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.3 v2:172.21.15.37:6800/4126478460 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ef400 2024-04-25T09:00:38.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:51 14.3 14:c0000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d400 con 0x55c378236c00 2024-04-25T09:00:38.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 12 ==== osd_op_reply(51 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:52 14.23 14:c4000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776eec00 con 0x55c3776ee000 2024-04-25T09:00:38.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.7 v2:172.21.15.203:6824/2957998038 4 ==== osd_op_reply(52 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ee000 2024-04-25T09:00:38.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:53 14.13 14:c8000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d800 con 0x55c3775f5000 2024-04-25T09:00:38.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 6 ==== osd_op_reply(53 [pgnls-filter start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.371+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:54 14.33 14:cc000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ef000 con 0x55c3776ee400 2024-04-25T09:00:38.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:55 14.b 14:d0000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771dc00 con 0x55c378236c00 2024-04-25T09:00:38.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 13 ==== osd_op_reply(55 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:56 14.2b 14:d4000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236000 con 0x55c3776efc00 2024-04-25T09:00:38.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 99 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:57 14.1b 14:d8000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706000 con 0x55c3776ef800 2024-04-25T09:00:38.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 5 ==== osd_op_reply(57 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:58 14.3b 14:dc000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236800 con 0x55c3776efc00 2024-04-25T09:00:38.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 11 ==== osd_op_reply(58 [pgnls-filter start_epoch 99 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:59 14.7 14:e0000000::::head [pgnls-filter start_epoch 99 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706400 con 0x55c3776ef800 2024-04-25T09:00:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 6 ==== osd_op_reply(59 [pgnls-filter start_epoch 99 out=81b] v99'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:60 14.0 14:00000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3775f5800 con 0x55c3775f5000 2024-04-25T09:00:38.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:61 14.20 14:04000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706800 con 0x55c378236c00 2024-04-25T09:00:38.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:62 14.10 14:08000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c400 con 0x55c3776ee400 2024-04-25T09:00:38.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.375+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:63 14.30 14:0c000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707c00 con 0x55c378236c00 2024-04-25T09:00:38.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 15 ==== osd_op_reply(63 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:64 14.8 14:10000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d000 con 0x55c378236c00 2024-04-25T09:00:38.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 16 ==== osd_op_reply(64 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:65 14.28 14:14000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707400 con 0x55c3776ef400 2024-04-25T09:00:38.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.3 v2:172.21.15.37:6800/4126478460 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ef400 2024-04-25T09:00:38.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:66 14.18 14:18000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c000 con 0x55c3776ee400 2024-04-25T09:00:38.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 10 ==== osd_op_reply(66 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:67 14.38 14:1c000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3775f5400 con 0x55c3776ee400 2024-04-25T09:00:38.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 11 ==== osd_op_reply(67 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:68 14.4 14:20000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707800 con 0x55c378236400 2024-04-25T09:00:38.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 7 ==== osd_op_reply(68 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:69 14.24 14:24000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378237c00 con 0x55c3776ee400 2024-04-25T09:00:38.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 99 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:70 14.14 14:28000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c800 con 0x55c3776efc00 2024-04-25T09:00:38.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 12 ==== osd_op_reply(70 [pgnls-filter start_epoch 99 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:71 14.34 14:2c000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ee800 con 0x55c3775f5000 2024-04-25T09:00:38.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:72 14.c 14:30000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771cc00 con 0x55c378236c00 2024-04-25T09:00:38.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 17 ==== osd_op_reply(72 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:73 14.2c 14:34000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378081000 con 0x55c3776ee400 2024-04-25T09:00:38.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 13 ==== osd_op_reply(73 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.379+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:74 14.1c 14:38000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d400 con 0x55c378236c00 2024-04-25T09:00:38.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 18 ==== osd_op_reply(74 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:75 14.3c 14:3c000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706000 con 0x55c3776ee400 2024-04-25T09:00:38.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:76 14.2 14:40000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771dc00 con 0x55c378236400 2024-04-25T09:00:38.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:77 14.22 14:44000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706400 con 0x55c3776efc00 2024-04-25T09:00:38.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 13 ==== osd_op_reply(77 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:78 14.12 14:48000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c400 con 0x55c3775f5000 2024-04-25T09:00:38.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 9 ==== osd_op_reply(78 [pgnls-filter start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:79 14.32 14:4c000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706800 con 0x55c3776efc00 2024-04-25T09:00:38.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:80 14.a 14:50000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776eec00 con 0x55c3776ef800 2024-04-25T09:00:38.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 7 ==== osd_op_reply(80 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:81 14.2a 14:54000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706c00 con 0x55c3776efc00 2024-04-25T09:00:38.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.383+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 15 ==== osd_op_reply(81 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:82 14.1a 14:58000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ef000 con 0x55c378236c00 2024-04-25T09:00:38.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 19 ==== osd_op_reply(82 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:83 14.3a 14:5c000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707000 con 0x55c378236c00 2024-04-25T09:00:38.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 20 ==== osd_op_reply(83 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:84 14.6 14:60000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d000 con 0x55c378236c00 2024-04-25T09:00:38.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 21 ==== osd_op_reply(84 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:85 14.26 14:64000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707400 con 0x55c3776efc00 2024-04-25T09:00:38.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 16 ==== osd_op_reply(85 [pgnls-filter start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:86 14.16 14:68000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c000 con 0x55c3776ef400 2024-04-25T09:00:38.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.3 v2:172.21.15.37:6800/4126478460 7 ==== osd_op_reply(86 [pgnls-filter start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ef400 2024-04-25T09:00:38.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:87 14.36 14:6c000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707800 con 0x55c3775f5000 2024-04-25T09:00:38.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 10 ==== osd_op_reply(87 [pgnls-filter start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:88 14.e 14:70000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771cc00 con 0x55c3776efc00 2024-04-25T09:00:38.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 17 ==== osd_op_reply(88 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:89 14.2e 14:74000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707c00 con 0x55c3776ef400 2024-04-25T09:00:38.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.3 v2:172.21.15.37:6800/4126478460 8 ==== osd_op_reply(89 [pgnls-filter start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ef400 2024-04-25T09:00:38.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:90 14.1e 14:78000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d400 con 0x55c3776ee000 2024-04-25T09:00:38.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.7 v2:172.21.15.203:6824/2957998038 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ee000 2024-04-25T09:00:38.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.387+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:91 14.3e 14:7c000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706000 con 0x55c3776ef800 2024-04-25T09:00:38.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 8 ==== osd_op_reply(91 [pgnls-filter start_epoch 99 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:92 14.1 14:80000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d800 con 0x55c3776ef400 2024-04-25T09:00:38.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.3 v2:172.21.15.37:6800/4126478460 9 ==== osd_op_reply(92 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ef400 2024-04-25T09:00:38.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:93 14.21 14:84000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706400 con 0x55c3776ef800 2024-04-25T09:00:38.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 9 ==== osd_op_reply(93 [pgnls-filter start_epoch 99 out=49b] v95'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:94 14.11 14:88000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3775f5400 con 0x55c3776efc00 2024-04-25T09:00:38.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 18 ==== osd_op_reply(94 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:95 14.31 14:8c000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776eec00 con 0x55c3776ee400 2024-04-25T09:00:38.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 15 ==== osd_op_reply(95 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:96 14.9 14:90000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3775f5800 con 0x55c3776ef800 2024-04-25T09:00:38.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 99 out=49b] v95'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:97 14.29 14:94000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ee800 con 0x55c378236400 2024-04-25T09:00:38.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 9 ==== osd_op_reply(97 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.391+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:98 14.19 14:98000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378237c00 con 0x55c378236400 2024-04-25T09:00:38.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 10 ==== osd_op_reply(98 [pgnls-filter start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:99 14.39 14:9c000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c800 con 0x55c3776ee000 2024-04-25T09:00:38.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.7 v2:172.21.15.203:6824/2957998038 6 ==== osd_op_reply(99 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ee000 2024-04-25T09:00:38.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:100 14.5 14:a0000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236000 con 0x55c378236c00 2024-04-25T09:00:38.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 22 ==== osd_op_reply(100 [pgnls-filter start_epoch 99 out=49b] v95'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:101 14.25 14:a4000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771dc00 con 0x55c3775f5000 2024-04-25T09:00:38.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 11 ==== osd_op_reply(101 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:102 14.15 14:a8000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236800 con 0x55c3776ee000 2024-04-25T09:00:38.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.7 v2:172.21.15.203:6824/2957998038 7 ==== osd_op_reply(102 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ee000 2024-04-25T09:00:38.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:103 14.35 14:ac000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c000 con 0x55c3776efc00 2024-04-25T09:00:38.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 19 ==== osd_op_reply(103 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.395+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:104 14.d 14:b0000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378081000 con 0x55c378236400 2024-04-25T09:00:38.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 11 ==== osd_op_reply(104 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:105 14.2d 14:b4000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d400 con 0x55c3776efc00 2024-04-25T09:00:38.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 20 ==== osd_op_reply(105 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:106 14.1d 14:b8000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c400 con 0x55c378236c00 2024-04-25T09:00:38.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 23 ==== osd_op_reply(106 [pgnls-filter start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:107 14.3d 14:bc000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d800 con 0x55c3776ef400 2024-04-25T09:00:38.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.3 v2:172.21.15.37:6800/4126478460 10 ==== osd_op_reply(107 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ef400 2024-04-25T09:00:38.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:108 14.3 14:c0000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771cc00 con 0x55c378236c00 2024-04-25T09:00:38.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 24 ==== osd_op_reply(108 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:109 14.23 14:c4000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c800 con 0x55c3776ee000 2024-04-25T09:00:38.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.7 v2:172.21.15.203:6824/2957998038 8 ==== osd_op_reply(109 [pgnls-filter start_epoch 99 out=49b] v95'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ee000 2024-04-25T09:00:38.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:110 14.13 14:c8000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771dc00 con 0x55c3775f5000 2024-04-25T09:00:38.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.2 v2:172.21.15.37:6808/332967247 12 ==== osd_op_reply(110 [pgnls-filter start_epoch 99 out=49b] v95'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3775f5000 2024-04-25T09:00:38.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:111 14.33 14:cc000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378237c00 con 0x55c3776ee400 2024-04-25T09:00:38.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 16 ==== osd_op_reply(111 [pgnls-filter start_epoch 99 out=49b] v95'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:112 14.b 14:d0000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3775f5400 con 0x55c378236c00 2024-04-25T09:00:38.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 25 ==== osd_op_reply(112 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.399+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:113 14.2b 14:d4000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ee800 con 0x55c3776efc00 2024-04-25T09:00:38.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.403+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 99 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.403+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:114 14.1b 14:d8000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706800 con 0x55c3776ef800 2024-04-25T09:00:38.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.403+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 11 ==== osd_op_reply(114 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.403+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:115 14.3b 14:dc000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707c00 con 0x55c3776efc00 2024-04-25T09:00:38.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.403+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 22 ==== osd_op_reply(115 [pgnls-filter start_epoch 99 out=49b] v95'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.403+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:116 14.7 14:e0000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377706c00 con 0x55c3776ef800 2024-04-25T09:00:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.403+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 12 ==== osd_op_reply(116 [pgnls-filter start_epoch 99 out=81b] v99'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.403+0000 7f08b9a008c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-04-25T09:00:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.403+0000 7f08b9a008c0 10 datascan.operator(): handling object 10000000001.0 2024-04-25T09:00:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.403+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:117 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d000 con 0x55c3776ef800 2024-04-25T09:00:38.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 13 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-04-25T09:00:38.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-04-25T09:00:38.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-04-25T09:00:38.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:118 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c400 con 0x55c3776ee400 2024-04-25T09:00:38.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 17 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+59 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-04-25T09:00:38.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:119 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c378236000 con 0x55c378236400 2024-04-25T09:00:38.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.6 v2:172.21.15.203:6800/467003745 12 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=520b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+520 (crc 0 0 0) 0x55c377366500 con 0x55c378236400 2024-04-25T09:00:38.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-04-25T09:00:38.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T09:00:38.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 20 datascan.get_frag_of: frag is * 2024-04-25T09:00:38.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:120 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3775f5800 con 0x55c3776ee400 2024-04-25T09:00:38.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 18 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 10 datascan.find_or_create_dirfrag: read_version = 17 2024-04-25T09:00:38.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-04-25T09:00:38.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:121 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776ef000 con 0x55c3776ee400 2024-04-25T09:00:38.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 19 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T09:00:38.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 20 datascan.read_dentry: file_head not found in result 2024-04-25T09:00:38.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-04-25T09:00:38.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:122 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c3776eec00 con 0x55c3776ee400 2024-04-25T09:00:38.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 20 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v99'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T09:00:38.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.407+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:123 14.27 14:e4000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707000 con 0x55c3776ef800 2024-04-25T09:00:38.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.5 v2:172.21.15.203:6808/213476662 14 ==== osd_op_reply(123 [pgnls-filter start_epoch 99 out=49b] v95'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ef800 2024-04-25T09:00:38.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:124 14.17 14:e8000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707400 con 0x55c3776ee400 2024-04-25T09:00:38.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.1 v2:172.21.15.37:6816/2944303862 21 ==== osd_op_reply(124 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c3776ee400 2024-04-25T09:00:38.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:125 14.37 14:ec000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771d000 con 0x55c3776efc00 2024-04-25T09:00:38.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 <== osd.4 v2:172.21.15.203:6802/714177927 23 ==== osd_op_reply(125 [pgnls-filter start_epoch 99 out=49b] v95'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366780 con 0x55c3776efc00 2024-04-25T09:00:38.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:126 14.f 14:f0000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377707800 con 0x55c3776ee000 2024-04-25T09:00:38.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.7 v2:172.21.15.203:6824/2957998038 9 ==== osd_op_reply(126 [pgnls-filter start_epoch 99 out=49b] v95'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ee000 2024-04-25T09:00:38.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:127 14.2f 14:f4000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771c400 con 0x55c378236c00 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 26 ==== osd_op_reply(127 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b31b5640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:128 14.1f 14:f8000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c377722000 con 0x55c378236c00 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b79be640 1 -- 172.21.15.37:0/720830454 <== osd.0 v2:172.21.15.37:6824/1116141266 27 ==== osd_op_reply(128 [pgnls-filter start_epoch 99 out=49b] v95'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366000 con 0x55c378236c00 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b39b6640 1 -- 172.21.15.37:0/720830454 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:129 14.3f 14:fc000000::::head [pgnls-filter start_epoch 99 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55c37771cc00 con 0x55c3776ef400 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b89c0640 1 -- 172.21.15.37:0/720830454 <== osd.3 v2:172.21.15.37:6800/4126478460 11 ==== osd_op_reply(129 [pgnls-filter start_epoch 99 out=49b] v95'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c377366500 con 0x55c3776ef400 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55c378236c00 msgr2=0x55c37746e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55c378236c00 0x55c37746e680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c3776ee400 msgr2=0x55c3776d0580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55c3776ee400 0x55c3776d0580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55c3775f5000 msgr2=0x55c37746f180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55c3775f5000 0x55c37746f180 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55c3776ef400 msgr2=0x55c3776d0b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55c3776ef400 0x55c3776d0b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 reap_dead start 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55c3776efc00 msgr2=0x55c3776d1080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55c3776efc00 0x55c3776d1080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55c3776ef800 msgr2=0x55c3776d1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55c3776ef800 0x55c3776d1600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55c378236400 msgr2=0x55c37746ec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55c378236400 0x55c37746ec00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55c3776ee000 msgr2=0x55c3776d1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55c3776ee000 0x55c3776d1b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b81bf640 1 -- 172.21.15.37:0/720830454 reap_dead start 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55c3775f4c00 msgr2=0x55c3776d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55c3775f4c00 0x55c3776d0000 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x55c378094840 tx=0x55c37734a120 comp rx=0 tx=0).stop 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378237000 msgr2=0x55c37746e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378237000 0x55c37746e100 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x55c378095b60 tx=0x55c3781e4d80 comp rx=0 tx=0).stop 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 shutdown_connections 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55c3775f4c00 0x55c3776d0000 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/720830454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c378237000 0x55c37746e100 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 >> 172.21.15.37:0/720830454 conn(0x55c378081400 msgr2=0x55c3772e1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 shutdown_connections 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.411+0000 7f08b9a008c0 1 -- 172.21.15.37:0/720830454 wait complete. 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.415+0000 7f08b9a008c0 1 -- 172.21.15.37:0/1148232153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3772b3800 msgr2=0x55c37746d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.415+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/1148232153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3772b3800 0x55c37746d080 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x55c378105470 tx=0x55c37734a000 comp rx=0 tx=0).stop 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.415+0000 7f08b9a008c0 1 -- 172.21.15.37:0/1148232153 shutdown_connections 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.415+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/1148232153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3772b3800 0x55c37746d080 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.415+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/1148232153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55c3772b3400 0x55c37746d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.415+0000 7f08b9a008c0 1 --2- 172.21.15.37:0/1148232153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55c3772b3000 0x55c37746db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.415+0000 7f08b9a008c0 1 -- 172.21.15.37:0/1148232153 >> 172.21.15.37:0/1148232153 conn(0x55c3772b2c00 msgr2=0x55c3772e1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.415+0000 7f08b9a008c0 1 -- 172.21.15.37:0/1148232153 shutdown_connections 2024-04-25T09:00:38.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.415+0000 7f08b9a008c0 1 -- 172.21.15.37:0/1148232153 wait complete. 2024-04-25T09:00:38.409 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.158096, result: 2024-04-25T09:00:38.410 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T09:00:38.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 -- start start 2024-04-25T09:00:38.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 Processor -- start 2024-04-25T09:00:38.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ca9f8ef800 0x55ca9faa9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca9f8ef400 0x55ca9faa9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ca9f8ef000 0x55ca9faa9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa06c0180 con 0x55ca9f8ef400 2024-04-25T09:00:38.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9f94e900 con 0x55ca9f8ef800 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ca9f94e480 con 0x55ca9f8ef000 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea3dd9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca9f8ef400 0x55ca9faa9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea3dd9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca9f8ef400 0x55ca9faa9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46920/0 (socket says 172.21.15.37:46920) 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/171668913 learned_addr learned my addr 172.21.15.37:0/171668913 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/171668913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ca9f8ef000 0x55ca9faa9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea45da640 1 --2- 172.21.15.37:0/171668913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ca9f8ef800 0x55ca9faa9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/171668913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ca9f8ef000 msgr2=0x55ca9faa9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/171668913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ca9f8ef000 0x55ca9faa9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/171668913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ca9f8ef800 msgr2=0x55ca9faa9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/171668913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ca9f8ef800 0x55ca9faa9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/171668913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca9f9141e0 con 0x55ca9f8ef400 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/171668913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ca9f8ef000 0x55ca9faa9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/171668913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca9f8ef400 0x55ca9faa9600 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x55caa0733860 tx=0x55ca9f9a8f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7d627245a1269a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea1dd5640 1 -- 172.21.15.37:0/171668913 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca9f9244e0 con 0x55ca9f8ef400 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea1dd5640 1 -- 172.21.15.37:0/171668913 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca9f9381c0 con 0x55ca9f8ef400 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea1dd5640 1 -- 172.21.15.37:0/171668913 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca9f924b60 con 0x55ca9f8ef400 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/171668913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca9f9152c0 con 0x55ca9f8ef400 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/171668913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ca9f915a40 con 0x55ca9f8ef400 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 4 waiting for MDS map... 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea1dd5640 1 -- 172.21.15.37:0/171668913 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x55caa0802600 con 0x55ca9f8ef400 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea1dd5640 1 -- 172.21.15.37:0/171668913 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 68) v1 ==== 2723+0+0 (secure 0 0 0) 0x55ca9f945180 con 0x55ca9f8ef400 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 4 Got MDS map 68 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 4 datascan.main: Using metadata pool output 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 4 datascan.main: connecting to RADOS... 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 Processor -- start 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 -- start start 2024-04-25T09:00:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.495+0000 7f8ea5e1b8c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55caa0806c00 0x55ca9faa8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55caa0807000 0x55ca9faa8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0807400 0x55ca9faaa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca9f94e180 con 0x55caa0807400 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa06c0d80 con 0x55caa0806c00 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55caa06c0f00 con 0x55caa0807000 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea3dd9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0807400 0x55ca9faaa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea3dd9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0807400 0x55ca9faaa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46924/0 (socket says 172.21.15.37:46924) 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1208240456 learned_addr learned my addr 172.21.15.37:0/1208240456 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1208240456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55caa0806c00 0x55ca9faa8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1208240456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55caa0807000 msgr2=0x55ca9faa8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea45da640 1 --2- 172.21.15.37:0/1208240456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55caa0807000 0x55ca9faa8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1208240456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55caa0807000 0x55ca9faa8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1208240456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55caa0806c00 msgr2=0x55ca9faa8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1208240456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55caa0806c00 0x55ca9faa8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1208240456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca9f915c20 con 0x55caa0807400 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1208240456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55caa0806c00 0x55ca9faa8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1208240456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0807400 0x55ca9faaa100 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x55caa0814660 tx=0x55ca9f9a8300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f50ee54d75bbb87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8e9fdd1640 1 -- 172.21.15.37:0/1208240456 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca9f924340 con 0x55caa0807400 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8e9fdd1640 1 -- 172.21.15.37:0/1208240456 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca9f938e00 con 0x55caa0807400 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8e9fdd1640 1 -- 172.21.15.37:0/1208240456 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca9f9244e0 con 0x55caa0807400 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1208240456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0807400 msgr2=0x55ca9faaa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1208240456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0807400 0x55ca9faaa100 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x55caa0814660 tx=0x55ca9f9a8300 comp rx=0 tx=0).stop 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1208240456 shutdown_connections 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1208240456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0807400 0x55ca9faaa100 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1208240456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55caa0807000 0x55ca9faa8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1208240456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55caa0806c00 0x55ca9faa8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1208240456 >> 172.21.15.37:0/1208240456 conn(0x55ca9f8eec00 msgr2=0x55ca9f91d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1208240456 shutdown_connections 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1208240456 wait complete. 2024-04-25T09:00:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 Processor -- start 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- start start 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0806c00 0x55ca9faa8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55caa0807000 0x55ca9faa8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55caa0807400 0x55ca9faaa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa06c1200 con 0x55caa0806c00 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55caa06c1080 con 0x55caa0807400 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55caa06c1380 con 0x55caa0807000 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea4ddb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0806c00 0x55ca9faa8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea4ddb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0806c00 0x55ca9faa8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46930/0 (socket says 172.21.15.37:46930) 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 learned_addr learned my addr 172.21.15.37:0/1074366945 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea45da640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55caa0807000 0x55ca9faa8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55caa0807400 0x55ca9faaa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55caa0807000 msgr2=0x55ca9faa8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55caa0807000 0x55ca9faa8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55caa0807400 msgr2=0x55ca9faaa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55caa0807400 0x55ca9faaa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca9f9141e0 con 0x55caa0806c00 2024-04-25T09:00:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea45da640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55caa0807000 0x55ca9faa8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0806c00 0x55ca9faa8b00 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x55caa0815980 tx=0x55caa081af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8e9e5ce640 1 -- 172.21.15.37:0/1074366945 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca9f924340 con 0x55caa0806c00 2024-04-25T09:00:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8e9e5ce640 1 -- 172.21.15.37:0/1074366945 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ca9f938540 con 0x55caa0806c00 2024-04-25T09:00:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ca9f9152c0 con 0x55caa0806c00 2024-04-25T09:00:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8e9e5ce640 1 -- 172.21.15.37:0/1074366945 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ca9f9244e0 con 0x55caa0806c00 2024-04-25T09:00:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ca9f915e00 con 0x55caa0806c00 2024-04-25T09:00:38.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.499+0000 7f8ea5e1b8c0 4 datascan.init: resolving metadata pool 13 2024-04-25T09:00:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8e9e5ce640 1 -- 172.21.15.37:0/1074366945 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55ca9fc16c00 con 0x55caa0806c00 2024-04-25T09:00:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8e9e5ce640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55ca9fc24c00 0x55ca9fd14000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8e9e5ce640 1 -- 172.21.15.37:0/1074366945 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x55ca9fc26a00 con 0x55caa0806c00 2024-04-25T09:00:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea5e1b8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T09:00:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea5e1b8c0 4 datascan.main: resolving metadata pool 13 2024-04-25T09:00:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55caa0807800 0x55ca9faaa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55caa0807c00 con 0x55caa0807800 2024-04-25T09:00:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea45da640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55ca9fc24c00 0x55ca9fd14000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55caa0807800 0x55ca9faaa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea45da640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55ca9fc24c00 0x55ca9fd14000 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55ca9f9ca150 tx=0x55ca9f9c6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55caa0807800 0x55ca9faaa680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== osd_op_reply(1 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55caa0807c00 0x55ca9fd14580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55caa0806800 con 0x55caa0807c00 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55caa0807c00 0x55ca9fd14580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55caa0807c00 0x55ca9fd14580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(2 [pgnls start_epoch 99 out=177b] v99'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55caa0806800 0x55ca9fd14b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.503+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fc25000 con 0x55caa0806800 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea45da640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55caa0806800 0x55ca9fd14b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea45da640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55caa0806800 0x55ca9fd14b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.2 v2:172.21.15.37:6808/332967247 1 ==== osd_op_reply(3 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe780 con 0x55caa0806800 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fc25000 con 0x55caa0807800 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 2 ==== osd_op_reply(4 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55ca9fc25000 0x55ca9fd15080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55caa0806400 con 0x55ca9fc25000 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55ca9fc25000 0x55ca9fd15080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55ca9fc25000 0x55ca9fd15080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(5 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55caa0806400 0x55ca9fd15600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fc25c00 con 0x55caa0806400 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55caa0806400 0x55ca9fd15600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55caa0806400 0x55ca9fd15600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.507+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== osd_op_reply(6 [pgnls start_epoch 99 out=149b] v98'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0806400 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fc25c00 con 0x55caa0807c00 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 2 ==== osd_op_reply(7 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fc25c00 con 0x55ca9fc25000 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 2 ==== osd_op_reply(8 [pgnls start_epoch 99 out=73b] v98'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55ca9fc25c00 0x55ca9fd15b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c000 con 0x55ca9fc25c00 2024-04-25T09:00:38.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea45da640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55ca9fc25c00 0x55ca9fd15b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea45da640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55ca9fc25c00 0x55ca9fd15b80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== osd_op_reply(9 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe780 con 0x55ca9fc25c00 2024-04-25T09:00:38.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c000 con 0x55caa0806400 2024-04-25T09:00:38.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.0 v2:172.21.15.37:6824/1116141266 2 ==== osd_op_reply(10 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0806400 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c000 con 0x55caa0806800 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.2 v2:172.21.15.37:6808/332967247 2 ==== osd_op_reply(11 [pgnls start_epoch 99 out=81b] v98'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ca9f9fe780 con 0x55caa0806800 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c000 con 0x55caa0807c00 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 3 ==== osd_op_reply(12 [pgnls start_epoch 99 out=49b] v98'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55ca9fd3c000 0x55ca9fd16c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fc25400 con 0x55ca9fd3c000 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.511+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55ca9fd3c000 0x55ca9fd16c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55ca9fd3c000 0x55ca9fd16c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 1 ==== osd_op_reply(13 [pgnls start_epoch 99 out=77b] v98'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea3dd9640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55ca9fc25400 0x55ca9fd16100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fc25400 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55ca9fc25400 0x55ca9fd16100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea4ddb640 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55ca9fc25400 0x55ca9fd16100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.3 v2:172.21.15.37:6800/4126478460 1 ==== osd_op_reply(14 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55ca9fc25400 2024-04-25T09:00:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55caa0807800 2024-04-25T09:00:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 3 ==== osd_op_reply(15 [pgnls start_epoch 99 out=73b] v98'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fd3c000 2024-04-25T09:00:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 2 ==== osd_op_reply(16 [pgnls start_epoch 99 out=49b] v98'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fc25000 2024-04-25T09:00:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 3 ==== osd_op_reply(17 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55caa0807c00 2024-04-25T09:00:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 4 ==== osd_op_reply(18 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.515+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fc25c00 2024-04-25T09:00:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.5 v2:172.21.15.203:6808/213476662 2 ==== osd_op_reply(19 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe780 con 0x55ca9fc25c00 2024-04-25T09:00:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fc25400 2024-04-25T09:00:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.3 v2:172.21.15.37:6800/4126478460 2 ==== osd_op_reply(20 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55ca9fc25400 2024-04-25T09:00:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fd3c000 2024-04-25T09:00:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 3 ==== osd_op_reply(21 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fd3c000 2024-04-25T09:00:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 4 ==== osd_op_reply(22 [pgnls start_epoch 99 out=97b] v98'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fd3c000 2024-04-25T09:00:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 5 ==== osd_op_reply(23 [pgnls start_epoch 99 out=74b] v94'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55caa0807800 2024-04-25T09:00:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 4 ==== osd_op_reply(24 [pgnls start_epoch 99 out=73b] v98'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55caa0806400 2024-04-25T09:00:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.0 v2:172.21.15.37:6824/1116141266 3 ==== osd_op_reply(25 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0806400 2024-04-25T09:00:38.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.519+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55caa0807800 2024-04-25T09:00:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.523+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 5 ==== osd_op_reply(26 [pgnls start_epoch 99 out=98b] v98'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.523+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fc25c00 2024-04-25T09:00:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.523+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.5 v2:172.21.15.203:6808/213476662 3 ==== osd_op_reply(27 [pgnls start_epoch 99 out=76b] v98'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55ca9f9fe780 con 0x55ca9fc25c00 2024-04-25T09:00:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.523+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fd3c000 2024-04-25T09:00:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.523+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 6 ==== osd_op_reply(28 [pgnls start_epoch 99 out=73b] v98'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.523+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55caa0807800 2024-04-25T09:00:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.523+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 6 ==== osd_op_reply(29 [pgnls start_epoch 99 out=73b] v98'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.523+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fc25c00 2024-04-25T09:00:38.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.523+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.5 v2:172.21.15.203:6808/213476662 4 ==== osd_op_reply(30 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe780 con 0x55ca9fc25c00 2024-04-25T09:00:38.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.523+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fc25000 2024-04-25T09:00:38.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 4 ==== osd_op_reply(31 [pgnls start_epoch 99 out=73b] v94'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55ca9fc25000 2024-04-25T09:00:38.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 5 ==== osd_op_reply(32 [pgnls start_epoch 99 out=101b] v98'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T09:00:38.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3d400 con 0x55caa0807c00 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 5 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3d800 con 0x55caa0807c00 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 6 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3dc00 con 0x55caa0807c00 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 7 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 20 datascan.scan_links: injected ino detected 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd48000 con 0x55caa0807c00 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 8 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd48400 con 0x55caa0807c00 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 9 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:38 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd48800 con 0x55caa0806400 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.0 v2:172.21.15.37:6824/1116141266 4 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0806400 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T09:00:38.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:39 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd48c00 con 0x55ca9fc25000 2024-04-25T09:00:38.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 6 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-04-25T09:00:38.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.527+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:40 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd49000 con 0x55caa0806800 2024-04-25T09:00:38.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.2 v2:172.21.15.37:6808/332967247 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55ca9f9fe780 con 0x55caa0806800 2024-04-25T09:00:38.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T09:00:38.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T09:00:38.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T09:00:38.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:41 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd49400 con 0x55caa0807800 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 7 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:42 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd49800 con 0x55ca9fd3c000 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 7 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:43 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd49c00 con 0x55caa0807800 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 8 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T09:00:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:44 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd48c00 con 0x55caa0807800 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 9 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 200.00000002 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.531+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:45 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3d000 con 0x55caa0807800 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 10 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T09:00:38.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T09:00:38.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T09:00:38.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:46 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3cc00 con 0x55ca9fc25000 2024-04-25T09:00:38.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 7 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:47 13.0 13:00000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0807800 2024-04-25T09:00:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 11 ==== osd_op_reply(47 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:48 13.10 13:08000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0807c00 2024-04-25T09:00:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 10 ==== osd_op_reply(48 [pgnls start_epoch 99 out=177b] v99'18 uv18 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:49 13.8 13:10000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0806800 2024-04-25T09:00:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.2 v2:172.21.15.37:6808/332967247 4 ==== osd_op_reply(49 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe780 con 0x55caa0806800 2024-04-25T09:00:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:50 13.18 13:18000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0807800 2024-04-25T09:00:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 12 ==== osd_op_reply(50 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.535+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:51 13.4 13:20000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fc25000 2024-04-25T09:00:38.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 8 ==== osd_op_reply(51 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:52 13.14 13:28000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0806400 2024-04-25T09:00:38.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.0 v2:172.21.15.37:6824/1116141266 5 ==== osd_op_reply(52 [pgnls start_epoch 99 out=149b] v98'28 uv28 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0806400 2024-04-25T09:00:38.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:53 13.c 13:30000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0807c00 2024-04-25T09:00:38.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 11 ==== osd_op_reply(53 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:54 13.1c 13:38000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fc25000 2024-04-25T09:00:38.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 9 ==== osd_op_reply(54 [pgnls start_epoch 99 out=73b] v98'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:55 13.2 13:40000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fc25c00 2024-04-25T09:00:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.5 v2:172.21.15.203:6808/213476662 5 ==== osd_op_reply(55 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe780 con 0x55ca9fc25c00 2024-04-25T09:00:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:56 13.12 13:48000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0806400 2024-04-25T09:00:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.0 v2:172.21.15.37:6824/1116141266 6 ==== osd_op_reply(56 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0806400 2024-04-25T09:00:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:57 13.a 13:50000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0806800 2024-04-25T09:00:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.2 v2:172.21.15.37:6808/332967247 5 ==== osd_op_reply(57 [pgnls start_epoch 99 out=81b] v98'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ca9f9fe780 con 0x55caa0806800 2024-04-25T09:00:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:58 13.1a 13:58000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0807c00 2024-04-25T09:00:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 12 ==== osd_op_reply(58 [pgnls start_epoch 99 out=49b] v98'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:59 13.6 13:60000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fd3c000 2024-04-25T09:00:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 8 ==== osd_op_reply(59 [pgnls start_epoch 99 out=77b] v98'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:60 13.16 13:68000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fc25400 2024-04-25T09:00:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.3 v2:172.21.15.37:6800/4126478460 3 ==== osd_op_reply(60 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55ca9fc25400 2024-04-25T09:00:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.539+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:61 13.e 13:70000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0807800 2024-04-25T09:00:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 13 ==== osd_op_reply(61 [pgnls start_epoch 99 out=73b] v98'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:62 13.1e 13:78000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fd3c000 2024-04-25T09:00:38.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 9 ==== osd_op_reply(62 [pgnls start_epoch 99 out=49b] v98'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:63 13.1 13:80000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fc25000 2024-04-25T09:00:38.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 10 ==== osd_op_reply(63 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:64 13.11 13:88000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0807c00 2024-04-25T09:00:38.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 13 ==== osd_op_reply(64 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:65 13.9 13:90000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fc25c00 2024-04-25T09:00:38.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.5 v2:172.21.15.203:6808/213476662 6 ==== osd_op_reply(65 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe780 con 0x55ca9fc25c00 2024-04-25T09:00:38.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:66 13.19 13:98000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fc25400 2024-04-25T09:00:38.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.3 v2:172.21.15.37:6800/4126478460 4 ==== osd_op_reply(66 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55ca9fc25400 2024-04-25T09:00:38.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:67 13.5 13:a0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fd3c000 2024-04-25T09:00:38.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 10 ==== osd_op_reply(67 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:68 13.15 13:a8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fd3c000 2024-04-25T09:00:38.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 11 ==== osd_op_reply(68 [pgnls start_epoch 99 out=97b] v98'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.543+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:69 13.d 13:b0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fd3c000 2024-04-25T09:00:38.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 12 ==== osd_op_reply(69 [pgnls start_epoch 99 out=74b] v94'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:70 13.1d 13:b8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0807800 2024-04-25T09:00:38.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 14 ==== osd_op_reply(70 [pgnls start_epoch 99 out=73b] v98'5 uv5 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:71 13.3 13:c0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0806400 2024-04-25T09:00:38.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.0 v2:172.21.15.37:6824/1116141266 7 ==== osd_op_reply(71 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0806400 2024-04-25T09:00:38.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:72 13.13 13:c8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0807800 2024-04-25T09:00:38.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 15 ==== osd_op_reply(72 [pgnls start_epoch 99 out=98b] v98'4 uv4 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:73 13.b 13:d0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fc25c00 2024-04-25T09:00:38.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.5 v2:172.21.15.203:6808/213476662 7 ==== osd_op_reply(73 [pgnls start_epoch 99 out=76b] v98'6 uv6 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55ca9f9fe780 con 0x55ca9fc25c00 2024-04-25T09:00:38.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:74 13.1b 13:d8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fd3c000 2024-04-25T09:00:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 13 ==== osd_op_reply(74 [pgnls start_epoch 99 out=73b] v98'10 uv10 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.547+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:75 13.7 13:e0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55caa0807800 2024-04-25T09:00:38.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 16 ==== osd_op_reply(75 [pgnls start_epoch 99 out=73b] v98'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:76 13.17 13:e8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fc25c00 2024-04-25T09:00:38.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.5 v2:172.21.15.203:6808/213476662 8 ==== osd_op_reply(76 [pgnls start_epoch 99 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ca9f9fe780 con 0x55ca9fc25c00 2024-04-25T09:00:38.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:77 13.f 13:f0000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fc25000 2024-04-25T09:00:38.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 11 ==== osd_op_reply(77 [pgnls start_epoch 99 out=73b] v94'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:78 13.1f 13:f8000000::::head [pgnls start_epoch 99 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c800 con 0x55ca9fc25000 2024-04-25T09:00:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 12 ==== osd_op_reply(78 [pgnls start_epoch 99 out=101b] v98'9 uv9 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T09:00:38.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:79 13.10 13:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55caa0807c00 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 14 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:80 13.10 13:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3d400 con 0x55caa0807c00 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 15 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.551+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3d800 con 0x55caa0807c00 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 16 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 20 datascan.scan_links: injected ino detected 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:82 13.10 13:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3dc00 con 0x55caa0807c00 2024-04-25T09:00:38.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 17 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T09:00:38.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:83 13.10 13:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fc25800 con 0x55caa0807c00 2024-04-25T09:00:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 18 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T09:00:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T09:00:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T09:00:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T09:00:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T09:00:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T09:00:38.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:84 13.14 13:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd48400 con 0x55caa0806400 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.0 v2:172.21.15.37:6824/1116141266 8 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0806400 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:85 13.1c 13:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd48800 con 0x55ca9fc25000 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 13 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:86 13.a 13:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd49400 con 0x55caa0806800 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 <== osd.2 v2:172.21.15.37:6808/332967247 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+54 (crc 0 0 0) 0x55ca9f9fe780 con 0x55caa0806800 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:87 13.e 13:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd49800 con 0x55caa0807800 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 17 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T09:00:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:88 13.15 13:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd48000 con 0x55ca9fd3c000 2024-04-25T09:00:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 14 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T09:00:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T09:00:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T09:00:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T09:00:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T09:00:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.555+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:89 13.1d 13:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3c400 con 0x55caa0807800 2024-04-25T09:00:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 18 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T09:00:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T09:00:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T09:00:38.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:90 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3d400 con 0x55caa0807800 2024-04-25T09:00:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 19 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T09:00:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T09:00:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 200.00000002 2024-04-25T09:00:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-04-25T09:00:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T09:00:38.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:91 13.7 13:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3d800 con 0x55caa0807800 2024-04-25T09:00:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 20 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T09:00:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T09:00:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T09:00:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T09:00:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T09:00:38.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:92 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3dc00 con 0x55ca9fc25000 2024-04-25T09:00:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.6 v2:172.21.15.203:6800/467003745 14 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1041b] v0'0 uv9 ondisk = 0) v8 ==== 154+0+1041 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fc25000 2024-04-25T09:00:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-04-25T09:00:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T09:00:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-04-25T09:00:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-04-25T09:00:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.559+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:93 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd49c00 con 0x55caa0807c00 2024-04-25T09:00:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 19 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T09:00:38.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:94 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd48c00 con 0x55caa0807c00 2024-04-25T09:00:38.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 20 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v99'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:95 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd48800 con 0x55caa0807c00 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 21 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+505 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 20 datascan.scan_links: first is now 2 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:96 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd49800 con 0x55caa0807c00 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea4ddb640 1 -- 172.21.15.37:0/1074366945 <== osd.1 v2:172.21.15.37:6816/2944303862 22 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v99'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ca9f9fe500 con 0x55caa0807c00 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 10 datascan.scan_links: updating inotable 2024-04-25T09:00:38.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:97 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd48000 con 0x55caa0807800 2024-04-25T09:00:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 21 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-04-25T09:00:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-04-25T09:00:38.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.563+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:98 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3d000 con 0x55caa0807800 2024-04-25T09:00:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.7 v2:172.21.15.203:6824/2957998038 22 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v99'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55ca9f9fe000 con 0x55caa0807800 2024-04-25T09:00:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 10 datascan.scan_links: updating snaptable 2024-04-25T09:00:38.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:99 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e99) v9 -- 0x55ca9fd3cc00 con 0x55ca9fd3c000 2024-04-25T09:00:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea3dd9640 1 -- 172.21.15.37:0/1074366945 <== osd.4 v2:172.21.15.203:6802/714177927 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55ca9f9fe000 con 0x55ca9fd3c000 2024-04-25T09:00:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55caa0806400 msgr2=0x55ca9fd15600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55caa0806400 0x55ca9fd15600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55caa0807c00 msgr2=0x55ca9fd14580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55caa0807c00 0x55ca9fd14580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55caa0806800 msgr2=0x55ca9fd14b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55caa0806800 0x55ca9fd14b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55ca9fc25400 msgr2=0x55ca9fd16100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55ca9fc25400 0x55ca9fd16100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55ca9fd3c000 msgr2=0x55ca9fd16c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55ca9fd3c000 0x55ca9fd16c00 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55ca9fc25c00 msgr2=0x55ca9fd15b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55ca9fc25c00 0x55ca9fd15b80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55ca9fc25000 msgr2=0x55ca9fd15080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 reap_dead start 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55ca9fc25000 0x55ca9fd15080 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55caa0807800 msgr2=0x55ca9faaa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55caa0807800 0x55ca9faaa680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 reap_dead start 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea45da640 1 -- 172.21.15.37:0/1074366945 reap_dead start 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55ca9fc24c00 msgr2=0x55ca9fd14000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55ca9fc24c00 0x55ca9fd14000 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x55ca9f9ca150 tx=0x55ca9f9c6000 comp rx=0 tx=0).stop 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0806c00 msgr2=0x55ca9faa8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0806c00 0x55ca9faa8b00 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x55caa0815980 tx=0x55caa081af00 comp rx=0 tx=0).stop 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 shutdown_connections 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55ca9fc24c00 0x55ca9fd14000 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/1074366945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa0806c00 0x55ca9faa8b00 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.567+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 >> 172.21.15.37:0/1074366945 conn(0x55ca9f8eec00 msgr2=0x55ca9f91d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.571+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 shutdown_connections 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.571+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/1074366945 wait complete. 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.571+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/171668913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca9f8ef400 msgr2=0x55ca9faa9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.571+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/171668913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca9f8ef400 0x55ca9faa9600 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x55caa0733860 tx=0x55ca9f9a8f00 comp rx=0 tx=0).stop 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.571+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/171668913 shutdown_connections 2024-04-25T09:00:38.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.571+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/171668913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ca9f8ef000 0x55ca9faa9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.571+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/171668913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ca9f8ef400 0x55ca9faa9600 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.571+0000 7f8ea5e1b8c0 1 --2- 172.21.15.37:0/171668913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ca9f8ef800 0x55ca9faa9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.571+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/171668913 >> 172.21.15.37:0/171668913 conn(0x55ca9f8efc00 msgr2=0x55ca9f91d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.571+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/171668913 shutdown_connections 2024-04-25T09:00:38.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.571+0000 7f8ea5e1b8c0 1 -- 172.21.15.37:0/171668913 wait complete. 2024-04-25T09:00:38.566 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.156031, result: 2024-04-25T09:00:38.566 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T09:00:38.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5333d1c640 1 Processor -- start 2024-04-25T09:00:38.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5333d1c640 1 -- start start 2024-04-25T09:00:38.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5333d1c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f532c06c3b0 0x7f532c164d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5333d1c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c143b70 con 0x7f532c15da40 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5333d1c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532c06e170 con 0x7f532c143790 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5333d1c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532c06e450 con 0x7f532c06c3b0 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f53312b7640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f532c15da40 0x7f532c06bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52498/0 (socket says 172.21.15.37:52498) 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f53312b7640 1 -- 172.21.15.37:0/426144135 learned_addr learned my addr 172.21.15.37:0/426144135 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f53322b9640 1 --2- 172.21.15.37:0/426144135 >> v2:172.21.15.203:3300/0 conn(0x7f532c06c3b0 0x7f532c164d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919021707 0 0) 0x7f532c143b70 con 0x7f532c15da40 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5314003530 con 0x7f532c15da40 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1571502316 0 0) 0x7f5314003530 con 0x7f532c15da40 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 >> v2:172.21.15.203:3300/0 conn(0x7f532c06c3b0 msgr2=0x7f532c164d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 --2- 172.21.15.37:0/426144135 >> v2:172.21.15.203:3300/0 conn(0x7f532c06c3b0 0x7f532c164d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 >> v1:172.21.15.203:6789/0 conn(0x7f532c143790 legacy=0x7f532c06d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c06e730 con 0x7f532c15da40 2024-04-25T09:00:38.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f5320001a00 con 0x7f532c15da40 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 >> v1:172.21.15.37:6789/0 conn(0x7f532c15da40 legacy=0x7f532c06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314003a30 0x7f5314003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5314004810 0x7f5314004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53140051d0 0x7f531400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c06e450 con 0x7f5314004810 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531400c100 con 0x7f53140051d0 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531400c300 con 0x7f5314003a30 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f53312b7640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5314004810 0x7f5314004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f53322b9640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53140051d0 0x7f531400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5331ab8640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314003a30 0x7f5314003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f53312b7640 1 -- 172.21.15.37:0/426144135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314003a30 msgr2=0x7f5314003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f53312b7640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314003a30 0x7f5314003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f53312b7640 1 -- 172.21.15.37:0/426144135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53140051d0 msgr2=0x7f531400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f53312b7640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f53140051d0 0x7f531400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f53312b7640 1 -- 172.21.15.37:0/426144135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5314003530 con 0x7f5314004810 2024-04-25T09:00:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f53322b9640 1 -- 172.21.15.37:0/426144135 reap_dead start 2024-04-25T09:00:38.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.747+0000 7f5331ab8640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314003a30 0x7f5314003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.751+0000 7f53312b7640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5314004810 0x7f5314004c90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f532000a810 tx=0x7f532001fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0fee96c3b5aede server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.751+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532000a460 con 0x7f5314004810 2024-04-25T09:00:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.751+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53200aa690 con 0x7f5314004810 2024-04-25T09:00:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.751+0000 7f5330ab6640 1 -- 172.21.15.37:0/426144135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53200aaac0 con 0x7f5314004810 2024-04-25T09:00:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.751+0000 7f5333d1c640 1 -- 172.21.15.37:0/426144135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5314004810 msgr2=0x7f5314004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.751+0000 7f5333d1c640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5314004810 0x7f5314004c90 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f532000a810 tx=0x7f532001fa60 comp rx=0 tx=0).stop 2024-04-25T09:00:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.751+0000 7f5333d1c640 1 -- 172.21.15.37:0/426144135 shutdown_connections 2024-04-25T09:00:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.751+0000 7f5333d1c640 1 --2- 172.21.15.37:0/426144135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5314004810 0x7f5314004c90 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.751+0000 7f5333d1c640 1 -- 172.21.15.37:0/426144135 >> 172.21.15.37:0/426144135 conn(0x7f532c071810 msgr2=0x7f532c071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.751+0000 7f5333d1c640 1 -- 172.21.15.37:0/426144135 shutdown_connections 2024-04-25T09:00:38.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.751+0000 7f5333d1c640 1 -- 172.21.15.37:0/426144135 wait complete. 2024-04-25T09:00:38.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5333d1c640 1 Processor -- start 2024-04-25T09:00:38.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5333d1c640 1 -- start start 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5333d1c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314004810 0x7f532c13c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5333d1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c13a430 0x7f532c13a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5333d1c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f532c13d530 0x7f532c13d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5333d1c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c06be20 con 0x7f532c13a430 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5333d1c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532c06bca0 con 0x7f532c13d530 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5333d1c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f532c06bfa0 con 0x7f5314004810 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f53312b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c13a430 0x7f532c13a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f53312b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c13a430 0x7f532c13a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46948/0 (socket says 172.21.15.37:46948) 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f53312b7640 1 -- 172.21.15.37:0/1921067587 learned_addr learned my addr 172.21.15.37:0/1921067587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5331ab8640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314004810 0x7f532c13c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f53322b9640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f532c13d530 0x7f532c13d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f53312b7640 1 -- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314004810 msgr2=0x7f532c13c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f53312b7640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314004810 0x7f532c13c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f53312b7640 1 -- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f532c13d530 msgr2=0x7f532c13d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:38.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f53312b7640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f532c13d530 0x7f532c13d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f53312b7640 1 -- 172.21.15.37:0/1921067587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532c13c840 con 0x7f532c13a430 2024-04-25T09:00:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f53312b7640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c13a430 0x7f532c13a8b0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f532001f890 tx=0x7f5314005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f532a7fc640 1 -- 172.21.15.37:0/1921067587 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53200aa490 con 0x7f532c13a430 2024-04-25T09:00:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f532a7fc640 1 -- 172.21.15.37:0/1921067587 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53200aa840 con 0x7f532c13a430 2024-04-25T09:00:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5333d1c640 1 -- 172.21.15.37:0/1921067587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532c13b260 con 0x7f532c13a430 2024-04-25T09:00:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f532a7fc640 1 -- 172.21.15.37:0/1921067587 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53200aabf0 con 0x7f532c13a430 2024-04-25T09:00:38.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5333d1c640 1 -- 172.21.15.37:0/1921067587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532c13b820 con 0x7f532c13a430 2024-04-25T09:00:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f5333d1c640 1 -- 172.21.15.37:0/1921067587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532c0cf8a0 con 0x7f532c13a430 2024-04-25T09:00:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.755+0000 7f532a7fc640 1 -- 172.21.15.37:0/1921067587 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53200ab470 con 0x7f532c13a430 2024-04-25T09:00:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.759+0000 7f532a7fc640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f60 0x7f5318069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.759+0000 7f532a7fc640 1 -- 172.21.15.37:0/1921067587 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f5320122f40 con 0x7f532c13a430 2024-04-25T09:00:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.759+0000 7f5331ab8640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f60 0x7f5318069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.759+0000 7f5331ab8640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f60 0x7f5318069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f531c000c10 tx=0x7f531c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:38.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.763+0000 7f532a7fc640 1 -- 172.21.15.37:0/1921067587 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f53200af020 con 0x7f532c13a430 2024-04-25T09:00:38.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:38.959+0000 7f5333d1c640 1 -- 172.21.15.37:0/1921067587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f532c04ef80 con 0x7f532c13a430 2024-04-25T09:00:39.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.139+0000 7f532a7fc640 1 -- 172.21.15.37:0/1921067587 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v69) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5320019770 con 0x7f532c13a430 2024-04-25T09:00:39.131 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T09:00:39.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 -- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f60 msgr2=0x7f5318069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f60 0x7f5318069420 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f531c000c10 tx=0x7f531c009040 comp rx=0 tx=0).stop 2024-04-25T09:00:39.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 -- 172.21.15.37:0/1921067587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c13a430 msgr2=0x7f532c13a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c13a430 0x7f532c13a8b0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f532001f890 tx=0x7f5314005840 comp rx=0 tx=0).stop 2024-04-25T09:00:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 -- 172.21.15.37:0/1921067587 shutdown_connections 2024-04-25T09:00:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5318066f60 0x7f5318069420 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f532c13d530 0x7f532c13d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f532c13a430 0x7f532c13a8b0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 --2- 172.21.15.37:0/1921067587 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5314004810 0x7f532c13c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 -- 172.21.15.37:0/1921067587 >> 172.21.15.37:0/1921067587 conn(0x7f532c071810 msgr2=0x7f532c0b4310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 -- 172.21.15.37:0/1921067587 shutdown_connections 2024-04-25T09:00:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.143+0000 7f5333d1c640 1 -- 172.21.15.37:0/1921067587 wait complete. 2024-04-25T09:00:39.156 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c0848b640 1 Processor -- start 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c0848b640 1 -- start start 2024-04-25T09:00:39.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c0848b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c0015a600 0x7f2c000cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c0848b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c000b2ac0 0x7f2c000cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c0848b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c000cf370 con 0x7f2c0015e1e0 2024-04-25T09:00:39.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c0848b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c000ce070 con 0x7f2c0015a600 2024-04-25T09:00:39.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c0848b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000ce350 con 0x7f2c000b2ac0 2024-04-25T09:00:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c06227640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c0015a600 0x7f2c000cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c05a26640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c000b2ac0 0x7f2c000cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c06227640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c0015a600 0x7f2c000cee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:58470/0 (socket says 172.21.15.37:58470) 2024-04-25T09:00:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c06227640 1 -- 172.21.15.37:0/1479851883 learned_addr learned my addr 172.21.15.37:0/1479851883 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c05a26640 1 -- 172.21.15.37:0/1479851883 >> v2:172.21.15.203:3300/0 conn(0x7f2c0015a600 msgr2=0x7f2c000cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c05a26640 1 --2- 172.21.15.37:0/1479851883 >> v2:172.21.15.203:3300/0 conn(0x7f2c0015a600 0x7f2c000cee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c05a26640 1 -- 172.21.15.37:0/1479851883 >> v1:172.21.15.203:6789/0 conn(0x7f2c0015e1e0 legacy=0x7f2c000b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c05a26640 1 -- 172.21.15.37:0/1479851883 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c000ce630 con 0x7f2c000b2ac0 2024-04-25T09:00:39.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.279+0000 7f2c06227640 1 --2- 172.21.15.37:0/1479851883 >> v2:172.21.15.203:3300/0 conn(0x7f2c0015a600 0x7f2c000cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:39.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c05a26640 1 --2- 172.21.15.37:0/1479851883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c000b2ac0 0x7f2c000cb9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0097c0 tx=0x7f2bfc01ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b3505e0ff5a0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c05225640 1 -- 172.21.15.37:0/1479851883 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0b5070 con 0x7f2c000b2ac0 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c05225640 1 -- 172.21.15.37:0/1479851883 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bfc0a8990 con 0x7f2c000b2ac0 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c05225640 1 -- 172.21.15.37:0/1479851883 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc0a8d60 con 0x7f2c000b2ac0 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c0848b640 1 -- 172.21.15.37:0/1479851883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c000b2ac0 msgr2=0x7f2c000cb9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c0848b640 1 --2- 172.21.15.37:0/1479851883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c000b2ac0 0x7f2c000cb9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0097c0 tx=0x7f2bfc01ec90 comp rx=0 tx=0).stop 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c0848b640 1 -- 172.21.15.37:0/1479851883 shutdown_connections 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c0848b640 1 --2- 172.21.15.37:0/1479851883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c000b2ac0 0x7f2c000cb9b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c0848b640 1 --2- 172.21.15.37:0/1479851883 >> v2:172.21.15.203:3300/0 conn(0x7f2c0015a600 0x7f2c000cee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c0848b640 1 -- 172.21.15.37:0/1479851883 >> 172.21.15.37:0/1479851883 conn(0x7f2c000b1f10 msgr2=0x7f2c000b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c0848b640 1 -- 172.21.15.37:0/1479851883 shutdown_connections 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c0848b640 1 -- 172.21.15.37:0/1479851883 wait complete. 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c0848b640 1 Processor -- start 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.283+0000 7f2c0848b640 1 -- start start 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c0848b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c000b2ac0 0x7f2c0015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c0848b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c0015a600 0x7f2c000b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c0848b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c0015e1e0 0x7f2c000b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c0848b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000cc1e0 con 0x7f2c000b2ac0 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c0848b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000cbef0 con 0x7f2c0015e1e0 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c0848b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0015a9e0 con 0x7f2c0015a600 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c06227640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c000b2ac0 0x7f2c0015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c06227640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c000b2ac0 0x7f2c0015d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46958/0 (socket says 172.21.15.37:46958) 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c06227640 1 -- 172.21.15.37:0/2313086870 learned_addr learned my addr 172.21.15.37:0/2313086870 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c06a28640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c0015e1e0 0x7f2c000b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c05a26640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c0015a600 0x7f2c000b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c06227640 1 -- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c0015a600 msgr2=0x7f2c000b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c06227640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c0015a600 0x7f2c000b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c06227640 1 -- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c0015e1e0 msgr2=0x7f2c000b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c06227640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c0015e1e0 0x7f2c000b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c06227640 1 -- 172.21.15.37:0/2313086870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0015d680 con 0x7f2c000b2ac0 2024-04-25T09:00:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c05a26640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c0015a600 0x7f2c000b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c06227640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c000b2ac0 0x7f2c0015d140 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0010f40 tx=0x7f2bf000e7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2bf6ffd640 1 -- 172.21.15.37:0/2313086870 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf001e070 con 0x7f2c000b2ac0 2024-04-25T09:00:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2bf6ffd640 1 -- 172.21.15.37:0/2313086870 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf0011860 con 0x7f2c000b2ac0 2024-04-25T09:00:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c0848b640 1 -- 172.21.15.37:0/2313086870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c000b5140 con 0x7f2c000b2ac0 2024-04-25T09:00:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2bf6ffd640 1 -- 172.21.15.37:0/2313086870 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0011c30 con 0x7f2c000b2ac0 2024-04-25T09:00:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c0848b640 1 -- 172.21.15.37:0/2313086870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c000b5700 con 0x7f2c000b2ac0 2024-04-25T09:00:39.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c0848b640 1 -- 172.21.15.37:0/2313086870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c000b5dc0 con 0x7f2c000b2ac0 2024-04-25T09:00:39.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2bf6ffd640 1 -- 172.21.15.37:0/2313086870 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2bf0021070 con 0x7f2c000b2ac0 2024-04-25T09:00:39.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2bf6ffd640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2be4066f60 0x7f2be4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2bf6ffd640 1 -- 172.21.15.37:0/2313086870 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f2bf0087e40 con 0x7f2c000b2ac0 2024-04-25T09:00:39.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.287+0000 7f2c05a26640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2be4066f60 0x7f2be4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.291+0000 7f2c05a26640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2be4066f60 0x7f2be4069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0b2000 tx=0x7f2bfc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.291+0000 7f2bf6ffd640 1 -- 172.21.15.37:0/2313086870 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c000b5dc0 con 0x7f2c000b2ac0 2024-04-25T09:00:39.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.479+0000 7f2c0848b640 1 -- 172.21.15.37:0/2313086870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c0004efa0 con 0x7f2c000b2ac0 2024-04-25T09:00:39.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2bf6ffd640 1 -- 172.21.15.37:0/2313086870 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f2bf0058560 con 0x7f2c000b2ac0 2024-04-25T09:00:39.474 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:00:39.474 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":70,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T09:00:39.149190+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"a","rank":0,"incarnation":70,"state":"up:replay","state_seq":1,"addr":"172.21.15.37:6835/3366017865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3366017865},{"type":"v1","addr":"172.21.15.37:6835","nonce":3366017865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:00:39.475 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 70 2024-04-25T09:00:39.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 -- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2be4066f60 msgr2=0x7f2be4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2be4066f60 0x7f2be4069420 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc0b2000 tx=0x7f2bfc008040 comp rx=0 tx=0).stop 2024-04-25T09:00:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 -- 172.21.15.37:0/2313086870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c000b2ac0 msgr2=0x7f2c0015d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c000b2ac0 0x7f2c0015d140 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0010f40 tx=0x7f2bf000e7f0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 -- 172.21.15.37:0/2313086870 shutdown_connections 2024-04-25T09:00:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2be4066f60 0x7f2be4069420 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c0015e1e0 0x7f2c000b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c0015a600 0x7f2c000b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 --2- 172.21.15.37:0/2313086870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c000b2ac0 0x7f2c0015d140 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 -- 172.21.15.37:0/2313086870 >> 172.21.15.37:0/2313086870 conn(0x7f2c000b1f10 msgr2=0x7f2c0006f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 -- 172.21.15.37:0/2313086870 shutdown_connections 2024-04-25T09:00:39.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:39.483+0000 7f2c0848b640 1 -- 172.21.15.37:0/2313086870 wait complete. 2024-04-25T09:00:39.896 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:58:52.843744+0000', 'modified': '2024-04-25T09:00:39.149190+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8340}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8340': {'gid': 8340, 'name': 'a', 'rank': 0, 'incarnation': 70, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.37:6835/3366017865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 3366017865}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 3366017865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:00:39.896 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8340:up:replay 2024-04-25T09:00:40.897 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:41.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f115c8de640 1 Processor -- start 2024-04-25T09:00:41.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f115c8de640 1 -- start start 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f115c8de640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f115815a700 0x7f115815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f115c8de640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f11580b3f50 0x7f11580cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f115c8de640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11580b2090 con 0x7f11580ce430 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f115c8de640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115815f200 con 0x7f11580b3f50 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f115c8de640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11580b43e0 con 0x7f115815a700 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f1155d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f11580b3f50 0x7f11580cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f1156d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f11580ce430 0x7f11580b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50706/0 (socket says 172.21.15.37:50706) 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f1156d9d640 1 -- 172.21.15.37:0/2567434821 learned_addr learned my addr 172.21.15.37:0/2567434821 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f115659c640 1 --2- 172.21.15.37:0/2567434821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f115815a700 0x7f115815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f115559a640 1 -- 172.21.15.37:0/2567434821 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3976656473 0 0) 0x7f11580b2090 con 0x7f11580ce430 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f115559a640 1 -- 172.21.15.37:0/2567434821 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1140003530 con 0x7f11580ce430 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f1155d9b640 1 -- 172.21.15.37:0/2567434821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f115815a700 msgr2=0x7f115815ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:41.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f1155d9b640 1 --2- 172.21.15.37:0/2567434821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f115815a700 0x7f115815ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f1155d9b640 1 -- 172.21.15.37:0/2567434821 >> v1:172.21.15.37:6789/0 conn(0x7f11580ce430 legacy=0x7f11580b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f1155d9b640 1 -- 172.21.15.37:0/2567434821 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11580b46c0 con 0x7f11580b3f50 2024-04-25T09:00:41.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.043+0000 7f115659c640 1 --2- 172.21.15.37:0/2567434821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f115815a700 0x7f115815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:41.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f1155d9b640 1 --2- 172.21.15.37:0/2567434821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11580b3f50 0x7f11580cdef0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f11480097c0 tx=0x7f114801c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c8462125c7c542 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115559a640 1 -- 172.21.15.37:0/2567434821 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11480b0070 con 0x7f11580b3f50 2024-04-25T09:00:41.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115559a640 1 -- 172.21.15.37:0/2567434821 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148004230 con 0x7f11580b3f50 2024-04-25T09:00:41.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115559a640 1 -- 172.21.15.37:0/2567434821 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148003620 con 0x7f11580b3f50 2024-04-25T09:00:41.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115c8de640 1 -- 172.21.15.37:0/2567434821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11580b3f50 msgr2=0x7f11580cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115c8de640 1 --2- 172.21.15.37:0/2567434821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11580b3f50 0x7f11580cdef0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f11480097c0 tx=0x7f114801c350 comp rx=0 tx=0).stop 2024-04-25T09:00:41.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115c8de640 1 -- 172.21.15.37:0/2567434821 shutdown_connections 2024-04-25T09:00:41.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115c8de640 1 --2- 172.21.15.37:0/2567434821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11580b3f50 0x7f11580cdef0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115c8de640 1 --2- 172.21.15.37:0/2567434821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f115815a700 0x7f115815ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115c8de640 1 -- 172.21.15.37:0/2567434821 >> 172.21.15.37:0/2567434821 conn(0x7f11580b11a0 msgr2=0x7f11580b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:41.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115c8de640 1 -- 172.21.15.37:0/2567434821 shutdown_connections 2024-04-25T09:00:41.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115c8de640 1 -- 172.21.15.37:0/2567434821 wait complete. 2024-04-25T09:00:41.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115c8de640 1 Processor -- start 2024-04-25T09:00:41.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.047+0000 7f115c8de640 1 -- start start 2024-04-25T09:00:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f115c8de640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11580ce430 0x7f1158152b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f115c8de640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115815a700 0x7f11581f6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f115c8de640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158153240 0x7f11581586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f115c8de640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11580cd490 con 0x7f1158153240 2024-04-25T09:00:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f115c8de640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11580cd310 con 0x7f115815a700 2024-04-25T09:00:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f115c8de640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11580b2bb0 con 0x7f11580ce430 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1156d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158153240 0x7f11581586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1156d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158153240 0x7f11581586c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54306/0 (socket says 172.21.15.37:54306) 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1156d9d640 1 -- 172.21.15.37:0/1779122839 learned_addr learned my addr 172.21.15.37:0/1779122839 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1155d9b640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115815a700 0x7f11581f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f115659c640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11580ce430 0x7f1158152b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1156d9d640 1 -- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11580ce430 msgr2=0x7f1158152b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1156d9d640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11580ce430 0x7f1158152b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1156d9d640 1 -- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115815a700 msgr2=0x7f11581f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1156d9d640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115815a700 0x7f11581f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1156d9d640 1 -- 172.21.15.37:0/1779122839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1158153050 con 0x7f1158153240 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1155d9b640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115815a700 0x7f11581f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1156d9d640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158153240 0x7f11581586c0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f115000cb70 tx=0x7f115000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1146ffd640 1 -- 172.21.15.37:0/1779122839 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150017070 con 0x7f1158153240 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1146ffd640 1 -- 172.21.15.37:0/1779122839 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115000ad40 con 0x7f1158153240 2024-04-25T09:00:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f115c8de640 1 -- 172.21.15.37:0/1779122839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1158157f70 con 0x7f1158153240 2024-04-25T09:00:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1146ffd640 1 -- 172.21.15.37:0/1779122839 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150012620 con 0x7f1158153240 2024-04-25T09:00:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f115c8de640 1 -- 172.21.15.37:0/1779122839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115815b130 con 0x7f1158153240 2024-04-25T09:00:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f115c8de640 1 -- 172.21.15.37:0/1779122839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115804efa0 con 0x7f1158153240 2024-04-25T09:00:41.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1146ffd640 1 -- 172.21.15.37:0/1779122839 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f115001a070 con 0x7f1158153240 2024-04-25T09:00:41.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1146ffd640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1138066f60 0x7f1138069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f1146ffd640 1 -- 172.21.15.37:0/1779122839 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f11500851a0 con 0x7f1158153240 2024-04-25T09:00:41.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.051+0000 7f115659c640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1138066f60 0x7f1138069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.055+0000 7f115659c640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1138066f60 0x7f1138069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f114c00c900 tx=0x7f114c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.055+0000 7f1146ffd640 1 -- 172.21.15.37:0/1779122839 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1150055e60 con 0x7f1158153240 2024-04-25T09:00:41.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.255+0000 7f115c8de640 1 -- 172.21.15.37:0/1779122839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f115806f260 con 0x7f1158153240 2024-04-25T09:00:41.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f1146ffd640 1 -- 172.21.15.37:0/1779122839 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f1150055760 con 0x7f1158153240 2024-04-25T09:00:41.250 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:00:41.250 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":72,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T09:00:40.161964+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"a","rank":0,"incarnation":70,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.37:6835/3366017865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3366017865},{"type":"v1","addr":"172.21.15.37:6835","nonce":3366017865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:00:41.250 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 72 2024-04-25T09:00:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 -- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1138066f60 msgr2=0x7f1138069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1138066f60 0x7f1138069420 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f114c00c900 tx=0x7f114c008040 comp rx=0 tx=0).stop 2024-04-25T09:00:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 -- 172.21.15.37:0/1779122839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158153240 msgr2=0x7f11581586c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158153240 0x7f11581586c0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f115000cb70 tx=0x7f115000a430 comp rx=0 tx=0).stop 2024-04-25T09:00:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 -- 172.21.15.37:0/1779122839 shutdown_connections 2024-04-25T09:00:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1138066f60 0x7f1138069420 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1158153240 0x7f11581586c0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f115815a700 0x7f11581f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 --2- 172.21.15.37:0/1779122839 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f11580ce430 0x7f1158152b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 -- 172.21.15.37:0/1779122839 >> 172.21.15.37:0/1779122839 conn(0x7f11580b11a0 msgr2=0x7f1158071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 -- 172.21.15.37:0/1779122839 shutdown_connections 2024-04-25T09:00:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:41.259+0000 7f115c8de640 1 -- 172.21.15.37:0/1779122839 wait complete. 2024-04-25T09:00:41.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:58:52.843744+0000', 'modified': '2024-04-25T09:00:40.161964+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8340}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8340': {'gid': 8340, 'name': 'a', 'rank': 0, 'incarnation': 70, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.37:6835/3366017865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 3366017865}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 3366017865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:00:41.672 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8340:up:rejoin 2024-04-25T09:00:42.673 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:42.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb3206640 1 Processor -- start 2024-04-25T09:00:42.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb3206640 1 -- start start 2024-04-25T09:00:42.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb3206640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15d970 0x7f8bac0c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb3206640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8bac0cf9c0 0x7f8bac0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb3206640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bac0cec00 con 0x7f8bac0cb010 2024-04-25T09:00:42.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb3206640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bac0c73a0 con 0x7f8bac0cf9c0 2024-04-25T09:00:42.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb3206640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac0cf530 con 0x7f8bac15d970 2024-04-25T09:00:42.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb17a3640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8bac0cb010 0x7f8bac06ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50710/0 (socket says 172.21.15.37:50710) 2024-04-25T09:00:42.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb17a3640 1 -- 172.21.15.37:0/220330572 learned_addr learned my addr 172.21.15.37:0/220330572 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:42.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8babfff640 1 --2- 172.21.15.37:0/220330572 >> v2:172.21.15.203:3300/0 conn(0x7f8bac0cf9c0 0x7f8bac0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb0fa2640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15d970 0x7f8bac0c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123666612 0 0) 0x7f8bac0cec00 con 0x7f8bac0cb010 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b94003530 con 0x7f8bac0cb010 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2743764863 0 0) 0x7f8b94003530 con 0x7f8bac0cb010 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15d970 msgr2=0x7f8bac0c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15d970 0x7f8bac0c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 >> v2:172.21.15.203:3300/0 conn(0x7f8bac0cf9c0 msgr2=0x7f8bac0caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 --2- 172.21.15.37:0/220330572 >> v2:172.21.15.203:3300/0 conn(0x7f8bac0cf9c0 0x7f8bac0caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac0ceee0 con 0x7f8bac0cb010 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f8ba4001a00 con 0x7f8bac0cb010 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb0fa2640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15d970 0x7f8bac0c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 >> v1:172.21.15.37:6789/0 conn(0x7f8bac0cb010 legacy=0x7f8bac06ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b94003a30 0x7f8b94003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15d970 0x7f8b94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b94004d50 0x7f8b940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac0cf530 con 0x7f8b94004d50 2024-04-25T09:00:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9400b9d0 con 0x7f8b94003a30 2024-04-25T09:00:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9400bb80 con 0x7f8bac15d970 2024-04-25T09:00:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb17a3640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b94004d50 0x7f8b940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb0fa2640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b94003a30 0x7f8b94003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8babfff640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15d970 0x7f8b94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb17a3640 1 -- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15d970 msgr2=0x7f8b94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb17a3640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15d970 0x7f8b94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb17a3640 1 -- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b94003a30 msgr2=0x7f8b94003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb17a3640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8b94003a30 0x7f8b94003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.819+0000 7f8bb17a3640 1 -- 172.21.15.37:0/220330572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b94003530 con 0x7f8b94004d50 2024-04-25T09:00:42.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bb17a3640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b94004d50 0x7f8b940051d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400a8a0 tx=0x7f8ba401d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a7cb256374f9664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400a4c0 con 0x7f8b94004d50 2024-04-25T09:00:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba40a9020 con 0x7f8b94004d50 2024-04-25T09:00:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bab7fe640 1 -- 172.21.15.37:0/220330572 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40072d0 con 0x7f8b94004d50 2024-04-25T09:00:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bb3206640 1 -- 172.21.15.37:0/220330572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b94004d50 msgr2=0x7f8b940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bb3206640 1 --2- 172.21.15.37:0/220330572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b94004d50 0x7f8b940051d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400a8a0 tx=0x7f8ba401d3c0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bb17a3640 1 -- 172.21.15.37:0/220330572 reap_dead start 2024-04-25T09:00:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bb3206640 1 -- 172.21.15.37:0/220330572 shutdown_connections 2024-04-25T09:00:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bb3206640 1 -- 172.21.15.37:0/220330572 >> 172.21.15.37:0/220330572 conn(0x7f8bac099a70 msgr2=0x7f8bac0d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bb3206640 1 -- 172.21.15.37:0/220330572 shutdown_connections 2024-04-25T09:00:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bb3206640 1 -- 172.21.15.37:0/220330572 wait complete. 2024-04-25T09:00:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bb3206640 1 Processor -- start 2024-04-25T09:00:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.823+0000 7f8bb3206640 1 -- start start 2024-04-25T09:00:42.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb3206640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bac159350 0x7f8bac160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb3206640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac159730 0x7f8bac160b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb3206640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15e1e0 0x7f8bac15e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb3206640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac0cb4a0 con 0x7f8bac159730 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb3206640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac15dd50 con 0x7f8bac159350 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb3206640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bac0cb620 con 0x7f8bac15e1e0 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8babfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac159730 0x7f8bac160b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8babfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac159730 0x7f8bac160b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54322/0 (socket says 172.21.15.37:54322) 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8babfff640 1 -- 172.21.15.37:0/2196031339 learned_addr learned my addr 172.21.15.37:0/2196031339 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb0fa2640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bac159350 0x7f8bac160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb17a3640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15e1e0 0x7f8bac15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8babfff640 1 -- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15e1e0 msgr2=0x7f8bac15e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8babfff640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15e1e0 0x7f8bac15e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8babfff640 1 -- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bac159350 msgr2=0x7f8bac160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8babfff640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bac159350 0x7f8bac160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8babfff640 1 -- 172.21.15.37:0/2196031339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac15ef30 con 0x7f8bac159730 2024-04-25T09:00:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb17a3640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15e1e0 0x7f8bac15e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8babfff640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac159730 0x7f8bac160b70 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0011660 tx=0x7f8ba0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8ba97fa640 1 -- 172.21.15.37:0/2196031339 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba000f960 con 0x7f8bac159730 2024-04-25T09:00:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8ba97fa640 1 -- 172.21.15.37:0/2196031339 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba000fb00 con 0x7f8bac159730 2024-04-25T09:00:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb3206640 1 -- 172.21.15.37:0/2196031339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bac15f210 con 0x7f8bac159730 2024-04-25T09:00:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8ba97fa640 1 -- 172.21.15.37:0/2196031339 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0015460 con 0x7f8bac159730 2024-04-25T09:00:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb3206640 1 -- 172.21.15.37:0/2196031339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bac15f7d0 con 0x7f8bac159730 2024-04-25T09:00:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb3206640 1 -- 172.21.15.37:0/2196031339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bac04efa0 con 0x7f8bac159730 2024-04-25T09:00:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8ba97fa640 1 -- 172.21.15.37:0/2196031339 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ba0015600 con 0x7f8bac159730 2024-04-25T09:00:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8ba97fa640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8b98066f60 0x7f8b98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8ba97fa640 1 -- 172.21.15.37:0/2196031339 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f8ba0085cf0 con 0x7f8bac159730 2024-04-25T09:00:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.827+0000 7f8bb0fa2640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8b98066f60 0x7f8b98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.831+0000 7f8bb0fa2640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8b98066f60 0x7f8b98069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c010820 tx=0x7f8b9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:42.831+0000 7f8ba97fa640 1 -- 172.21.15.37:0/2196031339 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ba008a070 con 0x7f8bac159730 2024-04-25T09:00:43.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 -- 172.21.15.37:0/2196031339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bac099fc0 con 0x7f8bac159730 2024-04-25T09:00:43.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8ba97fa640 1 -- 172.21.15.37:0/2196031339 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f8ba0019030 con 0x7f8bac159730 2024-04-25T09:00:43.019 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:00:43.019 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T09:00:41.164474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/3366017865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3366017865},{"type":"v1","addr":"172.21.15.37:6835","nonce":3366017865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:00:43.019 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 73 2024-04-25T09:00:43.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 -- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8b98066f60 msgr2=0x7f8b98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8b98066f60 0x7f8b98069420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c010820 tx=0x7f8b9c008040 comp rx=0 tx=0).stop 2024-04-25T09:00:43.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 -- 172.21.15.37:0/2196031339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac159730 msgr2=0x7f8bac160b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac159730 0x7f8bac160b70 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0011660 tx=0x7f8ba0011b20 comp rx=0 tx=0).stop 2024-04-25T09:00:43.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 -- 172.21.15.37:0/2196031339 shutdown_connections 2024-04-25T09:00:43.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8b98066f60 0x7f8b98069420 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bac15e1e0 0x7f8bac15e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8bac159730 0x7f8bac160b70 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 --2- 172.21.15.37:0/2196031339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bac159350 0x7f8bac160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 -- 172.21.15.37:0/2196031339 >> 172.21.15.37:0/2196031339 conn(0x7f8bac099a70 msgr2=0x7f8bac073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 -- 172.21.15.37:0/2196031339 shutdown_connections 2024-04-25T09:00:43.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.027+0000 7f8bb3206640 1 -- 172.21.15.37:0/2196031339 wait complete. 2024-04-25T09:00:43.441 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:58:52.843744+0000', 'modified': '2024-04-25T09:00:41.164474+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8340}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8340': {'gid': 8340, 'name': 'a', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.37:6835/3366017865', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 3366017865}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 3366017865}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:00:43.441 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:00:43.442 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:00:43.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ead3ed640 1 Processor -- start 2024-04-25T09:00:43.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ead3ed640 1 -- start start 2024-04-25T09:00:43.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ead3ed640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ea815a280 0x7f1ea815a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ead3ed640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ea806d090 0x7f1ea806d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ead3ed640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea815ae30 con 0x7f1ea8159ea0 2024-04-25T09:00:43.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ead3ed640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea80b2990 con 0x7f1ea815a280 2024-04-25T09:00:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ead3ed640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea80b3530 con 0x7f1ea806d090 2024-04-25T09:00:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ea67fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ea815a280 0x7f1ea815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ea67fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1ea815a280 0x7f1ea815a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54326/0 (socket says 172.21.15.37:54326) 2024-04-25T09:00:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ea67fc640 1 -- 172.21.15.37:0/94067166 learned_addr learned my addr 172.21.15.37:0/94067166 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/94067166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ea806d090 0x7f1ea806d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ea67fc640 1 -- 172.21.15.37:0/94067166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ea806d090 msgr2=0x7f1ea806d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/94067166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ea806d090 0x7f1ea806d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ea67fc640 1 -- 172.21.15.37:0/94067166 >> v1:172.21.15.203:6789/0 conn(0x7f1ea8159ea0 legacy=0x7f1ea80b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.587+0000 7f1ea67fc640 1 -- 172.21.15.37:0/94067166 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea815b110 con 0x7f1ea815a280 2024-04-25T09:00:43.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/94067166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea815a280 0x7f1ea815a700 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009e50 tx=0x7f1e9801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf0e7b10434e847 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ea5ffb640 1 -- 172.21.15.37:0/94067166 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980a5810 con 0x7f1ea815a280 2024-04-25T09:00:43.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ea5ffb640 1 -- 172.21.15.37:0/94067166 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e980a59b0 con 0x7f1ea815a280 2024-04-25T09:00:43.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ea5ffb640 1 -- 172.21.15.37:0/94067166 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980a5d80 con 0x7f1ea815a280 2024-04-25T09:00:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ead3ed640 1 -- 172.21.15.37:0/94067166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea815a280 msgr2=0x7f1ea815a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ead3ed640 1 --2- 172.21.15.37:0/94067166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea815a280 0x7f1ea815a700 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009e50 tx=0x7f1e9801ca30 comp rx=0 tx=0).stop 2024-04-25T09:00:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ead3ed640 1 -- 172.21.15.37:0/94067166 shutdown_connections 2024-04-25T09:00:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ead3ed640 1 --2- 172.21.15.37:0/94067166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ea806d090 0x7f1ea806d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ead3ed640 1 --2- 172.21.15.37:0/94067166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea815a280 0x7f1ea815a700 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ead3ed640 1 -- 172.21.15.37:0/94067166 >> 172.21.15.37:0/94067166 conn(0x7f1ea80b11a0 msgr2=0x7f1ea80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ead3ed640 1 -- 172.21.15.37:0/94067166 shutdown_connections 2024-04-25T09:00:43.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ead3ed640 1 -- 172.21.15.37:0/94067166 wait complete. 2024-04-25T09:00:43.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ead3ed640 1 Processor -- start 2024-04-25T09:00:43.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.591+0000 7f1ead3ed640 1 -- start start 2024-04-25T09:00:43.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ead3ed640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ea806d090 0x7f1ea80cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ead3ed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8159ea0 0x7f1ea80cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ead3ed640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ea815a280 0x7f1ea8144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ead3ed640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea80719c0 con 0x7f1ea8159ea0 2024-04-25T09:00:43.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ead3ed640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8071840 con 0x7f1ea806d090 2024-04-25T09:00:43.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ead3ed640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8071b40 con 0x7f1ea815a280 2024-04-25T09:00:43.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8159ea0 0x7f1ea80cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8159ea0 0x7f1ea80cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54338/0 (socket says 172.21.15.37:54338) 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea67fc640 1 -- 172.21.15.37:0/1111963505 learned_addr learned my addr 172.21.15.37:0/1111963505 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea6ffd640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ea806d090 0x7f1ea80cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ea815a280 0x7f1ea8144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea67fc640 1 -- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ea815a280 msgr2=0x7f1ea8144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ea815a280 0x7f1ea8144090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea67fc640 1 -- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ea806d090 msgr2=0x7f1ea80cee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ea806d090 0x7f1ea80cee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea67fc640 1 -- 172.21.15.37:0/1111963505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea80cddb0 con 0x7f1ea8159ea0 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8159ea0 0x7f1ea80cd2a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f1e9801a660 tx=0x7f1e980197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1e8f7fe640 1 -- 172.21.15.37:0/1111963505 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980b8470 con 0x7f1ea8159ea0 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1e8f7fe640 1 -- 172.21.15.37:0/1111963505 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e980be070 con 0x7f1ea8159ea0 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ead3ed640 1 -- 172.21.15.37:0/1111963505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea80ce090 con 0x7f1ea8159ea0 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1e8f7fe640 1 -- 172.21.15.37:0/1111963505 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980b8870 con 0x7f1ea8159ea0 2024-04-25T09:00:43.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ead3ed640 1 -- 172.21.15.37:0/1111963505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea80ce650 con 0x7f1ea8159ea0 2024-04-25T09:00:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1e8f7fe640 1 -- 172.21.15.37:0/1111963505 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1e980b8a10 con 0x7f1ea8159ea0 2024-04-25T09:00:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1e8f7fe640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1e94066f60 0x7f1e94069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1e8f7fe640 1 -- 172.21.15.37:0/1111963505 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f1e98120050 con 0x7f1ea8159ea0 2024-04-25T09:00:43.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.595+0000 7f1ea6ffd640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1e94066f60 0x7f1e94069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 Processor -- start 2024-04-25T09:00:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 -- start start 2024-04-25T09:00:43.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea6ffd640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1e94066f60 0x7f1e94069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c000f70 tx=0x7f1e9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff8031920 0x563ff8031d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff7fe79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff7fe7f30 0x563ff7fe8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff7ecf5f0 con 0x563ff8032240 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff7e36da0 con 0x563ff8031920 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ff7dbf150 con 0x563ff7fe7f30 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff7fe79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff7fe79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54342/0 (socket says 172.21.15.37:54342) 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea67fc640 1 -- 172.21.15.37:0/2925013614 learned_addr learned my addr 172.21.15.37:0/2925013614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea6ffd640 1 --2- 172.21.15.37:0/2925013614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff8031920 0x563ff8031d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/2925013614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff7fe7f30 0x563ff7fe8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea67fc640 1 -- 172.21.15.37:0/2925013614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff7fe7f30 msgr2=0x563ff7fe8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/2925013614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff7fe7f30 0x563ff7fe8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea67fc640 1 -- 172.21.15.37:0/2925013614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff8031920 msgr2=0x563ff8031d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/2925013614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff8031920 0x563ff8031d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea67fc640 1 -- 172.21.15.37:0/2925013614 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ff7dd8ca0 con 0x563ff8032240 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/2925013614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff7fe79f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f1e980f0640 tx=0x7f1e98009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a9e8cac33e8505 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1e8cff9640 1 -- 172.21.15.37:0/2925013614 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980f06b0 con 0x563ff8032240 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1e8cff9640 1 -- 172.21.15.37:0/2925013614 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e980f4800 con 0x563ff8032240 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1e8cff9640 1 -- 172.21.15.37:0/2925013614 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980f8dc0 con 0x563ff8032240 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 -- 172.21.15.37:0/2925013614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 msgr2=0x563ff7fe79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 --2- 172.21.15.37:0/2925013614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff7fe79f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f1e980f0640 tx=0x7f1e98009810 comp rx=0 tx=0).stop 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 -- 172.21.15.37:0/2925013614 shutdown_connections 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 --2- 172.21.15.37:0/2925013614 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff7fe7f30 0x563ff7fe8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 --2- 172.21.15.37:0/2925013614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff7fe79f0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 --2- 172.21.15.37:0/2925013614 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff8031920 0x563ff8031d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 -- 172.21.15.37:0/2925013614 >> 172.21.15.37:0/2925013614 conn(0x563ff7fee2a0 msgr2=0x563ff7fe62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 -- 172.21.15.37:0/2925013614 shutdown_connections 2024-04-25T09:00:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 -- 172.21.15.37:0/2925013614 wait complete. 2024-04-25T09:00:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 -- start start 2024-04-25T09:00:43.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.599+0000 7f1eb055a000 1 Processor -- start 2024-04-25T09:00:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff7fe7f30 0x563ff8038c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff80391c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff8032240 0x563ff7febaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff7df1990 con 0x563ff8031920 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff7e32e40 con 0x563ff8032240 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ff7dc2900 con 0x563ff7fe7f30 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 10 client.0 _ll_register_callbacks cb 0x563ff7eea1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff80391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff80391c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54346/0 (socket says 172.21.15.37:54346) 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea67fc640 1 -- 172.21.15.37:0/4218758655 learned_addr learned my addr 172.21.15.37:0/4218758655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff8032240 0x563ff7febaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea6ffd640 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff7fe7f30 0x563ff8038c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea67fc640 1 -- 172.21.15.37:0/4218758655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff7fe7f30 msgr2=0x563ff8038c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff7fe7f30 0x563ff8038c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea67fc640 1 -- 172.21.15.37:0/4218758655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff8032240 msgr2=0x563ff7febaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff8032240 0x563ff7febaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea67fc640 1 -- 172.21.15.37:0/4218758655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ff7def630 con 0x563ff8031920 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff80391c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f1e980f0640 tx=0x7f1e98000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a773eee63ac57970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1e85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1e85ffb640 1 -- 172.21.15.37:0/4218758655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980f8dc0 con 0x563ff8031920 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1e85ffb640 1 -- 172.21.15.37:0/4218758655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f1e980f0910 con 0x563ff8031920 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 -- 172.21.15.37:0/4218758655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1e980f4800 con 0x563ff8031920 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1e85ffb640 1 -- 172.21.15.37:0/4218758655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e980f0b50 con 0x563ff8031920 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 10 client.8406 fetch_fsmap learned FSMap version 73 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 -- 172.21.15.37:0/4218758655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x563ff7dbf3a0 con 0x563ff8031920 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1e85ffb640 1 -- 172.21.15.37:0/4218758655 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e980f4800 con 0x563ff8031920 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1e85ffb640 1 -- 172.21.15.37:0/4218758655 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1e980f8930 con 0x563ff8031920 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 10 client.8406 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 10 client.8406 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 20 client.8406 populate_metadata read hostname 'smithi037' 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x563ff7fec4b0 0x563ff7fec890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 4 client.8406 mds_command: new command op to 8340 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 -- 172.21.15.37:0/4218758655 --> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ff7dd2a50 con 0x563ff7fec4b0 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea6ffd640 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x563ff7fec4b0 0x563ff7fec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1e85ffb640 10 client.8406 ms_handle_connect on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1ea6ffd640 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x563ff7fec4b0 0x563ff7fec890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c473032ca3e20a5c server_cookie=55acd051328f4a8a in_seq=0 out_seq=0 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1e85ffb640 1 -- 172.21.15.37:0/4218758655 <== mds.0 v2:172.21.15.37:6834/3366017865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ff7dd2a50 con 0x563ff7fec4b0 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1e85ffb640 10 client.8406 handle_command_reply: tid=0 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 client.8406 shutdown 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 -- 172.21.15.37:0/4218758655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 msgr2=0x563ff80391c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff80391c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f1e980f0640 tx=0x7f1e98000f50 comp rx=0 tx=0).stop 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 -- 172.21.15.37:0/4218758655 shutdown_connections 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff8032240 0x563ff7febaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x563ff7fec4b0 0x563ff7fec890 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff80391c0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 --2- 172.21.15.37:0/4218758655 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff7fe7f30 0x563ff8038c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1e85ffb640 0 client.8406 ms_handle_reset on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 -- 172.21.15.37:0/4218758655 >> 172.21.15.37:0/4218758655 conn(0x563ff7fee2a0 msgr2=0x563ff7fe69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 -- 172.21.15.37:0/4218758655 shutdown_connections 2024-04-25T09:00:43.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 1 -- 172.21.15.37:0/4218758655 wait complete. 2024-04-25T09:00:43.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.603+0000 7f1eb055a000 20 client.8406 trim_cache size 0 max 16384 2024-04-25T09:00:43.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.619+0000 7f1eb055a000 1 Processor -- start 2024-04-25T09:00:43.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.619+0000 7f1eb055a000 1 -- start start 2024-04-25T09:00:43.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.619+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff7fe7f30 0x563ff808cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.619+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff8031920 0x563ff808d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.619+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff808d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8202730 con 0x563ff8032240 2024-04-25T09:00:43.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea81518b0 con 0x563ff7fe7f30 2024-04-25T09:00:43.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea81fae00 con 0x563ff8031920 2024-04-25T09:00:43.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff808d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff808d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54360/0 (socket says 172.21.15.37:54360) 2024-04-25T09:00:43.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea77fe640 1 -- 172.21.15.37:0/565109003 learned_addr learned my addr 172.21.15.37:0/565109003 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea77fe640 1 -- 172.21.15.37:0/565109003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff8031920 msgr2=0x563ff808d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/565109003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff8031920 0x563ff808d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea77fe640 1 -- 172.21.15.37:0/565109003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff7fe7f30 msgr2=0x563ff808cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/565109003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff7fe7f30 0x563ff808cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea77fe640 1 -- 172.21.15.37:0/565109003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ff7eb8600 con 0x563ff8032240 2024-04-25T09:00:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/565109003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff808d9e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009820 tx=0x7f1ea0009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab55d74ff400ea10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1e87fff640 1 -- 172.21.15.37:0/565109003 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0019030 con 0x563ff8032240 2024-04-25T09:00:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1e87fff640 1 -- 172.21.15.37:0/565109003 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea00127d0 con 0x563ff8032240 2024-04-25T09:00:43.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1e87fff640 1 -- 172.21.15.37:0/565109003 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000f040 con 0x563ff8032240 2024-04-25T09:00:43.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- 172.21.15.37:0/565109003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 msgr2=0x563ff808d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 --2- 172.21.15.37:0/565109003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff808d9e0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009820 tx=0x7f1ea0009ce0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- 172.21.15.37:0/565109003 shutdown_connections 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 --2- 172.21.15.37:0/565109003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8032240 0x563ff808d9e0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 --2- 172.21.15.37:0/565109003 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff8031920 0x563ff808d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 --2- 172.21.15.37:0/565109003 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff7fe7f30 0x563ff808cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- 172.21.15.37:0/565109003 >> 172.21.15.37:0/565109003 conn(0x563ff7fee2a0 msgr2=0x563ff7feb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- 172.21.15.37:0/565109003 shutdown_connections 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- 172.21.15.37:0/565109003 wait complete. 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- start start 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 Processor -- start 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff7fe7f30 0x563ff803e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff808d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff8032240 0x563ff808df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff7e3b3d0 con 0x563ff8031920 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ff7ef2d50 con 0x563ff7fe7f30 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ff7dd1980 con 0x563ff8032240 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 10 client.0 _ll_register_callbacks cb 0x7f1e980092f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff808d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff808d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54370/0 (socket says 172.21.15.37:54370) 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea67fc640 1 -- 172.21.15.37:0/1083653838 learned_addr learned my addr 172.21.15.37:0/1083653838 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea6ffd640 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff7fe7f30 0x563ff803e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff8032240 0x563ff808df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea67fc640 1 -- 172.21.15.37:0/1083653838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff8032240 msgr2=0x563ff808df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff8032240 0x563ff808df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea67fc640 1 -- 172.21.15.37:0/1083653838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff7fe7f30 msgr2=0x563ff803e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff7fe7f30 0x563ff803e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea67fc640 1 -- 172.21.15.37:0/1083653838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ff7dcb9b0 con 0x563ff8031920 2024-04-25T09:00:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff8032240 0x563ff808df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea6ffd640 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff7fe7f30 0x563ff803e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1ea67fc640 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff808d9f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1e980ee960 tx=0x7f1e9811e1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22c3eb484e33137 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1e85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1e85ffb640 1 -- 172.21.15.37:0/1083653838 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e98108040 con 0x563ff8031920 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1e85ffb640 1 -- 172.21.15.37:0/1083653838 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f1e980ec400 con 0x563ff8031920 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- 172.21.15.37:0/1083653838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ea00127d0 con 0x563ff8031920 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1e85ffb640 1 -- 172.21.15.37:0/1083653838 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e98113050 con 0x563ff8031920 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 10 client.8412 fetch_fsmap learned FSMap version 73 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.623+0000 7f1eb055a000 1 -- 172.21.15.37:0/1083653838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x563ff7dde0b0 con 0x563ff8031920 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1e85ffb640 1 -- 172.21.15.37:0/1083653838 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea00127d0 con 0x563ff8031920 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1e85ffb640 1 -- 172.21.15.37:0/1083653838 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1e98116020 con 0x563ff8031920 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 10 client.8412 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 10 client.8412 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:00:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 20 client.8412 populate_metadata read hostname 'smithi037' 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x563ff808e470 0x563ff808e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 4 client.8412 mds_command: new command op to 8340 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 -- 172.21.15.37:0/1083653838 --> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563ff7e442e0 con 0x563ff808e470 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ea6ffd640 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x563ff808e470 0x563ff808e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1e85ffb640 10 client.8412 ms_handle_connect on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ea6ffd640 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x563ff808e470 0x563ff808e850 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8759f0c82662983 server_cookie=64e108f450a3e74 in_seq=0 out_seq=0 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1e85ffb640 1 -- 172.21.15.37:0/1083653838 <== mds.0 v2:172.21.15.37:6834/3366017865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x563ff7e442e0 con 0x563ff808e470 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1e85ffb640 10 client.8412 handle_command_reply: tid=0 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 client.8412 shutdown 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 -- 172.21.15.37:0/1083653838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 msgr2=0x563ff808d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff808d9f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f1e980ee960 tx=0x7f1e9811e1a0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 -- 172.21.15.37:0/1083653838 shutdown_connections 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563ff8032240 0x563ff808df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x563ff808e470 0x563ff808e850 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563ff8031920 0x563ff808d9f0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 --2- 172.21.15.37:0/1083653838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563ff7fe7f30 0x563ff803e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1e85ffb640 0 client.8412 ms_handle_reset on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 -- 172.21.15.37:0/1083653838 >> 172.21.15.37:0/1083653838 conn(0x563ff7fee2a0 msgr2=0x563ff80398a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 -- 172.21.15.37:0/1083653838 shutdown_connections 2024-04-25T09:00:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 1 -- 172.21.15.37:0/1083653838 wait complete. 2024-04-25T09:00:43.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1eb055a000 20 client.8412 trim_cache size 0 max 16384 2024-04-25T09:00:43.624 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:00:43.624 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T09:00:43.624 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T09:00:43.624 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 8340, 2024-04-25T09:00:43.624 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T09:00:43.625 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T09:00:43.625 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T09:00:43.625 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 4.4767133799999996, 2024-04-25T09:00:43.625 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 73, 2024-04-25T09:00:43.625 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 99, 2024-04-25T09:00:43.625 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 99, 2024-04-25T09:00:43.625 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 5.4336141500000004 2024-04-25T09:00:43.625 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 -- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1e94066f60 msgr2=0x7f1e94069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1e94066f60 0x7f1e94069420 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c000f70 tx=0x7f1e9c009040 comp rx=0 tx=0).stop 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 -- 172.21.15.37:0/1111963505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8159ea0 msgr2=0x7f1ea80cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8159ea0 0x7f1ea80cd2a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f1e9801a660 tx=0x7f1e980197d0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 -- 172.21.15.37:0/1111963505 shutdown_connections 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1e94066f60 0x7f1e94069420 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ea815a280 0x7f1ea8144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8159ea0 0x7f1ea80cd2a0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 --2- 172.21.15.37:0/1111963505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ea806d090 0x7f1ea80cee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 -- 172.21.15.37:0/1111963505 >> 172.21.15.37:0/1111963505 conn(0x7f1ea80b11a0 msgr2=0x7f1ea815cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 -- 172.21.15.37:0/1111963505 shutdown_connections 2024-04-25T09:00:43.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.627+0000 7f1ead3ed640 1 -- 172.21.15.37:0/1111963505 wait complete. 2024-04-25T09:00:43.636 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:00:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dd08b4640 1 Processor -- start 2024-04-25T09:00:43.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dd08b4640 1 -- start start 2024-04-25T09:00:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dd08b4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0dcc0b3a70 0x7f0dcc06d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dd08b4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dcc06ee20 con 0x7f0dcc06f120 2024-04-25T09:00:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dd08b4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dcc06d670 con 0x7f0dcc15ac90 2024-04-25T09:00:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dd08b4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dcc06d950 con 0x7f0dcc0b3a70 2024-04-25T09:00:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dcad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0dcc06f120 0x7f0dcc15b200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50724/0 (socket says 172.21.15.37:50724) 2024-04-25T09:00:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dcad9d640 1 -- 172.21.15.37:0/1233625586 learned_addr learned my addr 172.21.15.37:0/1233625586 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:43.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dca59c640 1 --2- 172.21.15.37:0/1233625586 >> v2:172.21.15.203:3300/0 conn(0x7f0dcc0b3a70 0x7f0dcc06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458366655 0 0) 0x7f0dcc06ee20 con 0x7f0dcc06f120 2024-04-25T09:00:43.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0db8005740 con 0x7f0dcc06f120 2024-04-25T09:00:43.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 781607489 0 0) 0x7f0db8005740 con 0x7f0dcc06f120 2024-04-25T09:00:43.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 >> v2:172.21.15.203:3300/0 conn(0x7f0dcc0b3a70 msgr2=0x7f0dcc06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 --2- 172.21.15.37:0/1233625586 >> v2:172.21.15.203:3300/0 conn(0x7f0dcc0b3a70 0x7f0dcc06d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 >> v1:172.21.15.203:6789/0 conn(0x7f0dcc15ac90 legacy=0x7f0dcc06e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc06dc30 con 0x7f0dcc06f120 2024-04-25T09:00:43.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0dc4001a00 con 0x7f0dcc06f120 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 >> v1:172.21.15.37:6789/0 conn(0x7f0dcc06f120 legacy=0x7f0dcc15b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 --2- 172.21.15.37:0/1233625586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db8005c40 0x7f0db8006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 --2- 172.21.15.37:0/1233625586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db8006a20 0x7f0db8006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 --2- 172.21.15.37:0/1233625586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80073e0 0x7f0db800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc06d950 con 0x7f0db80073e0 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db800e310 con 0x7f0db8005c40 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db800e510 con 0x7f0db8006a20 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dcad9d640 1 --2- 172.21.15.37:0/1233625586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80073e0 0x7f0db800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dca59c640 1 --2- 172.21.15.37:0/1233625586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db8005c40 0x7f0db8006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dc9d9b640 1 --2- 172.21.15.37:0/1233625586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db8006a20 0x7f0db8006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dcad9d640 1 -- 172.21.15.37:0/1233625586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db8006a20 msgr2=0x7f0db8006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dcad9d640 1 --2- 172.21.15.37:0/1233625586 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0db8006a20 0x7f0db8006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dcad9d640 1 -- 172.21.15.37:0/1233625586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db8005c40 msgr2=0x7f0db8006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dcad9d640 1 --2- 172.21.15.37:0/1233625586 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db8005c40 0x7f0db8006060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.759+0000 7f0dcad9d640 1 -- 172.21.15.37:0/1233625586 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db8005740 con 0x7f0db80073e0 2024-04-25T09:00:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dcad9d640 1 -- 172.21.15.37:0/1233625586 reap_dead start 2024-04-25T09:00:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dcad9d640 1 --2- 172.21.15.37:0/1233625586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80073e0 0x7f0db800dbe0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400a830 tx=0x7f0dc401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d93f400e2a2c3ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc400a480 con 0x7f0db80073e0 2024-04-25T09:00:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc40ad630 con 0x7f0db80073e0 2024-04-25T09:00:43.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dc959a640 1 -- 172.21.15.37:0/1233625586 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40ada20 con 0x7f0db80073e0 2024-04-25T09:00:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dd08b4640 1 -- 172.21.15.37:0/1233625586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80073e0 msgr2=0x7f0db800dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dd08b4640 1 --2- 172.21.15.37:0/1233625586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80073e0 0x7f0db800dbe0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400a830 tx=0x7f0dc401d360 comp rx=0 tx=0).stop 2024-04-25T09:00:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dd08b4640 1 -- 172.21.15.37:0/1233625586 shutdown_connections 2024-04-25T09:00:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dd08b4640 1 --2- 172.21.15.37:0/1233625586 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0db80073e0 0x7f0db800dbe0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dd08b4640 1 -- 172.21.15.37:0/1233625586 >> 172.21.15.37:0/1233625586 conn(0x7f0dcc0b11a0 msgr2=0x7f0dcc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dd08b4640 1 -- 172.21.15.37:0/1233625586 shutdown_connections 2024-04-25T09:00:43.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dd08b4640 1 -- 172.21.15.37:0/1233625586 wait complete. 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dd08b4640 1 Processor -- start 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.763+0000 7f0dd08b4640 1 -- start start 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dd08b4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db80073e0 0x7f0dcc0c5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dd08b4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dcc0cde70 0x7f0dcc14ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dd08b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dcc1f3980 0x7f0dcc1f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dd08b4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0b29d0 con 0x7f0dcc1f3980 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dd08b4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc06e1f0 con 0x7f0db80073e0 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dd08b4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dcc0b2b50 con 0x7f0dcc0cde70 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dcad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dcc1f3980 0x7f0dcc1f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dcad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dcc1f3980 0x7f0dcc1f3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54386/0 (socket says 172.21.15.37:54386) 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dcad9d640 1 -- 172.21.15.37:0/2337169630 learned_addr learned my addr 172.21.15.37:0/2337169630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dca59c640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db80073e0 0x7f0dcc0c5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dc9d9b640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dcc0cde70 0x7f0dcc14ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dcad9d640 1 -- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dcc0cde70 msgr2=0x7f0dcc14ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dcad9d640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dcc0cde70 0x7f0dcc14ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dcad9d640 1 -- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db80073e0 msgr2=0x7f0dcc0c5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dcad9d640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db80073e0 0x7f0dcc0c5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dcad9d640 1 -- 172.21.15.37:0/2337169630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dcc0cf6f0 con 0x7f0dcc1f3980 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dcad9d640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dcc1f3980 0x7f0dcc1f3e30 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400a830 tx=0x7f0db8007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dc2ffd640 1 -- 172.21.15.37:0/2337169630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40bc070 con 0x7f0dcc1f3980 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dc2ffd640 1 -- 172.21.15.37:0/2337169630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc40019d0 con 0x7f0dcc1f3980 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dc2ffd640 1 -- 172.21.15.37:0/2337169630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc40ad9e0 con 0x7f0dcc1f3980 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dd08b4640 1 -- 172.21.15.37:0/2337169630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dcc0ce650 con 0x7f0dcc1f3980 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dd08b4640 1 -- 172.21.15.37:0/2337169630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dcc0cec10 con 0x7f0dcc1f3980 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dd08b4640 1 -- 172.21.15.37:0/2337169630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dcc04efa0 con 0x7f0dcc1f3980 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dc2ffd640 1 -- 172.21.15.37:0/2337169630 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0dc40adb80 con 0x7f0dcc1f3980 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dc2ffd640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dac066f60 0x7f0dac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dc2ffd640 1 -- 172.21.15.37:0/2337169630 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f0dc411f2c0 con 0x7f0dcc1f3980 2024-04-25T09:00:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.767+0000 7f0dca59c640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dac066f60 0x7f0dac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.771+0000 7f0dca59c640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dac066f60 0x7f0dac069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009aa0 tx=0x7f0db4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.775+0000 7f0dc2ffd640 1 -- 172.21.15.37:0/2337169630 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0dc40b4070 con 0x7f0dcc1f3980 2024-04-25T09:00:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.967+0000 7f0dd08b4640 1 -- 172.21.15.37:0/2337169630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0dcc06d430 con 0x7f0dcc1f3980 2024-04-25T09:00:43.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.967+0000 7f0dc2ffd640 1 -- 172.21.15.37:0/2337169630 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v73) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0dc4017070 con 0x7f0dcc1f3980 2024-04-25T09:00:43.960 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 -- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dac066f60 msgr2=0x7f0dac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dac066f60 0x7f0dac069420 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009aa0 tx=0x7f0db4008040 comp rx=0 tx=0).stop 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 -- 172.21.15.37:0/2337169630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dcc1f3980 msgr2=0x7f0dcc1f3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dcc1f3980 0x7f0dcc1f3e30 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f0dc400a830 tx=0x7f0db8007a50 comp rx=0 tx=0).stop 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 -- 172.21.15.37:0/2337169630 shutdown_connections 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0dac066f60 0x7f0dac069420 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dcc1f3980 0x7f0dcc1f3e30 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0dcc0cde70 0x7f0dcc14ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 --2- 172.21.15.37:0/2337169630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0db80073e0 0x7f0dcc0c5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 -- 172.21.15.37:0/2337169630 >> 172.21.15.37:0/2337169630 conn(0x7f0dcc0b11a0 msgr2=0x7f0dcc15c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 -- 172.21.15.37:0/2337169630 shutdown_connections 2024-04-25T09:00:43.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:43.971+0000 7f0dd08b4640 1 -- 172.21.15.37:0/2337169630 wait complete. 2024-04-25T09:00:43.978 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:00:43.978 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:00:43.978 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T09:00:43.978 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:00:43.978 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:43.978 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:00:43.978 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:00:43.978 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:00:43.979 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:00:43.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:43.979 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:43.985 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41997sec preferred_lft 41997sec 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:00:43.986 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:00:43.987 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:43.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:43.988 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:00:44.040 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T09:00:44.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:44.041 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T09:00:44.091 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T09:00:44.091 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:00:44.091 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T09:00:44.091 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:00:44.091 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T09:00:44.091 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:00:44.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:44.092 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T09:00:44.092 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T09:00:44.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:00:44.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:00:44.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:00:44.092 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T09:00:44.143 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T09:00:44.151+0000 7f793f332640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:00:44.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:44.169 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T09:00:44.214 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:00:44.214 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:00:44.215 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:00:44.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:44.215 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T09:00:44.275 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:00:44.275 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:00:44.275 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:44.322 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:00:44.322 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:00:44.322 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:44.370 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:00:44.429 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:00:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:00:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:00:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:00:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:00:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:00:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:00:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:00:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:00:44.479 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:00:44.480 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:00:44.481 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:00:44.481 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:00:44.481 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:00:44.481 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:00:44.481 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:00:44.481 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:44.481 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:44.481 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:44.481 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:44.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:44.482 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:00:44.530 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:00:44.532 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:00:44.575 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:00:44.591 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:00:44.639 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:00:44.639 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:00:44.639 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:00:44.639 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:00:44.639 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:00:44.639 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:00:44.640 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:44.641 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:44.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:44.642 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:00:44.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T09:00:44.667+0000 7fbee2134580 -1 init, newargv = 0x56298d582c40 newargc=15 2024-04-25T09:00:44.658 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[49947]: starting ceph client 2024-04-25T09:00:44.664 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[49947]: starting fuse 2024-04-25T09:00:45.764 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:00:45.780 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:00:45.829 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:00:45.829 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:00:45.829 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:00:45.829 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:00:45.829 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:00:45.829 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:00:45.829 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:00:45.829 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:00:45.829 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:00:45.829 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:00:45.830 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:00:45.831 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:00:45.831 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:00:45.831 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:00:45.831 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:00:45.831 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:45.831 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:45.831 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:45.831 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:45.831 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:45.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:45.832 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:00:45.878 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T09:00:45.879 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:00:45.879 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T09:00:45.880 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:00:46.008 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.49947.asok 2024-04-25T09:00:46.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:00:46.020 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49947.asok 2024-04-25T09:00:46.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:46.021 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49947.asok status 2024-04-25T09:00:46.185 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "49947", 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 8427, 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T09:00:46.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 8427 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1459854019 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1459854019 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:00:46.187 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.8427 192.168.0.1:0/1459854019", 2024-04-25T09:00:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/1459854019", 2024-04-25T09:00:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T09:00:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 73, 2024-04-25T09:00:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 99, 2024-04-25T09:00:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:00:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T09:00:46.188 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T09:00:46.188 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:00:46.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:46.200 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T09:00:46.200 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:46.206 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T09:00:46.207 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:46.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:46.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:46.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:46.259 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-04-25T09:00:46.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f50b79640 1 Processor -- start 2024-04-25T09:00:46.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f50b79640 1 -- start start 2024-04-25T09:00:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f50b79640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8f4c0cf510 0x7f8f4c0cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f50b79640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f4c0ac170 0x7f8f4c0ac600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f50b79640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f4c0b28c0 con 0x7f8f4c0adc70 2024-04-25T09:00:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f50b79640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f4c0cfec0 con 0x7f8f4c0cf510 2024-04-25T09:00:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f50b79640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c15bb20 con 0x7f8f4c0ac170 2024-04-25T09:00:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f49d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8f4c0cf510 0x7f8f4c0cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f49d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8f4c0cf510 0x7f8f4c0cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54400/0 (socket says 172.21.15.37:54400) 2024-04-25T09:00:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f49d9b640 1 -- 172.21.15.37:0/3824292999 learned_addr learned my addr 172.21.15.37:0/3824292999 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f4a59c640 1 --1- 172.21.15.37:0/3824292999 >> v1:172.21.15.37:6789/0 conn(0x7f8f4c0adc70 0x7f8f4c06e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50726/0 (socket says 172.21.15.37:50726) 2024-04-25T09:00:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f4ad9d640 1 --2- 172.21.15.37:0/3824292999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f4c0ac170 0x7f8f4c0ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f49d9b640 1 -- 172.21.15.37:0/3824292999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f4c0ac170 msgr2=0x7f8f4c0ac600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/3824292999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f4c0ac170 0x7f8f4c0ac600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.443+0000 7f8f49d9b640 1 -- 172.21.15.37:0/3824292999 >> v1:172.21.15.37:6789/0 conn(0x7f8f4c0adc70 legacy=0x7f8f4c06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f49d9b640 1 -- 172.21.15.37:0/3824292999 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c0b2b60 con 0x7f8f4c0cf510 2024-04-25T09:00:46.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f4959a640 1 -- 172.21.15.37:0/3824292999 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 34477105 0 0) 0x7f8f4c0b28c0 con 0x7f8f4c0adc70 2024-04-25T09:00:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/3824292999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c0cf510 0x7f8f4c0cf980 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009870 tx=0x7f8f4001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc89340f35960e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f4959a640 1 -- 172.21.15.37:0/3824292999 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400ab650 con 0x7f8f4c0cf510 2024-04-25T09:00:46.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f4959a640 1 -- 172.21.15.37:0/3824292999 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400b0070 con 0x7f8f4c0cf510 2024-04-25T09:00:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f4959a640 1 -- 172.21.15.37:0/3824292999 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400aba10 con 0x7f8f4c0cf510 2024-04-25T09:00:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f50b79640 1 -- 172.21.15.37:0/3824292999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c0cf510 msgr2=0x7f8f4c0cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f50b79640 1 --2- 172.21.15.37:0/3824292999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c0cf510 0x7f8f4c0cf980 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009870 tx=0x7f8f4001c560 comp rx=0 tx=0).stop 2024-04-25T09:00:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f50b79640 1 -- 172.21.15.37:0/3824292999 shutdown_connections 2024-04-25T09:00:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f50b79640 1 --2- 172.21.15.37:0/3824292999 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f4c0ac170 0x7f8f4c0ac600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f50b79640 1 --2- 172.21.15.37:0/3824292999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c0cf510 0x7f8f4c0cf980 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f50b79640 1 -- 172.21.15.37:0/3824292999 >> 172.21.15.37:0/3824292999 conn(0x7f8f4c06ae60 msgr2=0x7f8f4c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f50b79640 1 -- 172.21.15.37:0/3824292999 shutdown_connections 2024-04-25T09:00:46.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.447+0000 7f8f50b79640 1 -- 172.21.15.37:0/3824292999 wait complete. 2024-04-25T09:00:46.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f50b79640 1 Processor -- start 2024-04-25T09:00:46.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f50b79640 1 -- start start 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f50b79640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f4c0ac170 0x7f8f4c157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f50b79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c0adc70 0x7f8f4c0b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f50b79640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f4c0cf510 0x7f8f4c1f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f50b79640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0b3c00 con 0x7f8f4c0adc70 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f50b79640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0b36e0 con 0x7f8f4c0cf510 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f50b79640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0b15e0 con 0x7f8f4c0ac170 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f49d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c0adc70 0x7f8f4c0b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f49d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c0adc70 0x7f8f4c0b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54414/0 (socket says 172.21.15.37:54414) 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f49d9b640 1 -- 172.21.15.37:0/834013140 learned_addr learned my addr 172.21.15.37:0/834013140 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f4c0ac170 0x7f8f4c157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f4ad9d640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f4c0cf510 0x7f8f4c1f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f49d9b640 1 -- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f4c0ac170 msgr2=0x7f8f4c157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f4c0ac170 0x7f8f4c157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f49d9b640 1 -- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f4c0cf510 msgr2=0x7f8f4c1f7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f4c0cf510 0x7f8f4c1f7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f49d9b640 1 -- 172.21.15.37:0/834013140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c158190 con 0x7f8f4c0adc70 2024-04-25T09:00:46.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f4ad9d640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f4c0cf510 0x7f8f4c1f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:46.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c0adc70 0x7f8f4c0b10a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f8f400ad040 tx=0x7f8f40019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f3affd640 1 -- 172.21.15.37:0/834013140 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400bf610 con 0x7f8f4c0adc70 2024-04-25T09:00:46.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f3affd640 1 -- 172.21.15.37:0/834013140 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400bf7d0 con 0x7f8f4c0adc70 2024-04-25T09:00:46.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f50b79640 1 -- 172.21.15.37:0/834013140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f4c0c2490 con 0x7f8f4c0adc70 2024-04-25T09:00:46.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f3affd640 1 -- 172.21.15.37:0/834013140 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400bfbc0 con 0x7f8f4c0adc70 2024-04-25T09:00:46.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f50b79640 1 -- 172.21.15.37:0/834013140 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f4c0c2a50 con 0x7f8f4c0adc70 2024-04-25T09:00:46.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f3affd640 1 -- 172.21.15.37:0/834013140 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8f400bd020 con 0x7f8f4c0adc70 2024-04-25T09:00:46.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.451+0000 7f8f3affd640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8f2c066f60 0x7f8f2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f3affd640 1 -- 172.21.15.37:0/834013140 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f8f4011f6e0 con 0x7f8f4c0adc70 2024-04-25T09:00:46.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8f2c066f60 0x7f8f2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 Processor -- start 2024-04-25T09:00:46.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 -- start start 2024-04-25T09:00:46.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8f2c066f60 0x7f8f2c069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c000f70 tx=0x7f8f3c007b60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560475508c90 0x560475509070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x5604754bed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5604754bf290 0x5604754bf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604753a5f40 con 0x5604755095b0 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047530d070 con 0x560475508c90 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604752951a0 con 0x5604754bf290 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f49d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x5604754bed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f49d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x5604754bed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54416/0 (socket says 172.21.15.37:54416) 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f49d9b640 1 -- 172.21.15.37:0/302370934 learned_addr learned my addr 172.21.15.37:0/302370934 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f4ad9d640 1 --2- 172.21.15.37:0/302370934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5604754bf290 0x5604754bf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/302370934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560475508c90 0x560475509070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f49d9b640 1 -- 172.21.15.37:0/302370934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5604754bf290 msgr2=0x5604754bf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/302370934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5604754bf290 0x5604754bf670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f49d9b640 1 -- 172.21.15.37:0/302370934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560475508c90 msgr2=0x560475509070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/302370934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560475508c90 0x560475509070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f49d9b640 1 -- 172.21.15.37:0/302370934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560475293510 con 0x5604755095b0 2024-04-25T09:00:46.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/302370934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x5604754bed50 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f8f4001cad0 tx=0x7f8f400abf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34c315a514b5f5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f2bfff640 1 -- 172.21.15.37:0/302370934 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400efc20 con 0x5604755095b0 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f2bfff640 1 -- 172.21.15.37:0/302370934 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400b0070 con 0x5604755095b0 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f2bfff640 1 -- 172.21.15.37:0/302370934 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400f3d70 con 0x5604755095b0 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 -- 172.21.15.37:0/302370934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 msgr2=0x5604754bed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/302370934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x5604754bed50 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f8f4001cad0 tx=0x7f8f400abf90 comp rx=0 tx=0).stop 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 -- 172.21.15.37:0/302370934 shutdown_connections 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/302370934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5604754bf290 0x5604754bf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/302370934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x5604754bed50 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/302370934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x560475508c90 0x560475509070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 -- 172.21.15.37:0/302370934 >> 172.21.15.37:0/302370934 conn(0x5604754bb240 msgr2=0x5604754bd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 -- 172.21.15.37:0/302370934 shutdown_connections 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 -- 172.21.15.37:0/302370934 wait complete. 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.455+0000 7f8f53ce6000 1 -- start start 2024-04-25T09:00:46.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 Processor -- start 2024-04-25T09:00:46.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5604754bf290 0x5604754c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560475508c90 0x5604754ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604755095b0 0x5604754ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604752c76c0 con 0x560475508c90 2024-04-25T09:00:46.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x560475309110 con 0x5604755095b0 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604753a5f40 con 0x5604754bf290 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 10 client.0 _ll_register_callbacks cb 0x5604754aa570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f49d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560475508c90 0x5604754ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f49d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560475508c90 0x5604754ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54422/0 (socket says 172.21.15.37:54422) 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f49d9b640 1 -- 172.21.15.37:0/444604775 learned_addr learned my addr 172.21.15.37:0/444604775 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f4ad9d640 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604755095b0 0x5604754ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5604754bf290 0x5604754c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f49d9b640 1 -- 172.21.15.37:0/444604775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5604754bf290 msgr2=0x5604754c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5604754bf290 0x5604754c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f49d9b640 1 -- 172.21.15.37:0/444604775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604755095b0 msgr2=0x5604754ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604755095b0 0x5604754ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f49d9b640 1 -- 172.21.15.37:0/444604775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604752953b0 con 0x560475508c90 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5604754bf290 0x5604754c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560475508c90 0x5604754ca370 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f8f4001cad0 tx=0x7f8f4011f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635fe7b94d4e54a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f297fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f297fa640 1 -- 172.21.15.37:0/444604775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f400c3030 con 0x560475508c90 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f297fa640 1 -- 172.21.15.37:0/444604775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f8f40103de0 con 0x560475508c90 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 -- 172.21.15.37:0/444604775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f400b0070 con 0x560475508c90 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f297fa640 1 -- 172.21.15.37:0/444604775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f400efc20 con 0x560475508c90 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 10 client.8442 fetch_fsmap learned FSMap version 73 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 -- 172.21.15.37:0/444604775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x56047538ee00 con 0x560475508c90 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f297fa640 1 -- 172.21.15.37:0/444604775 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400b0070 con 0x560475508c90 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f297fa640 1 -- 172.21.15.37:0/444604775 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f400ef070 con 0x560475508c90 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 10 client.8442 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 10 client.8442 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 20 client.8442 populate_metadata read hostname 'smithi037' 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5604754cb180 0x56047551a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 4 client.8442 mds_command: new command op to 8340 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 -- 172.21.15.37:0/444604775 --> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604752a8d60 con 0x5604754cb180 2024-04-25T09:00:46.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5604754cb180 0x56047551a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f297fa640 10 client.8442 ms_handle_connect on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5604754cb180 0x56047551a410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bf75a5f28bf5c72 server_cookie=35c398dd21c40154 in_seq=0 out_seq=0 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f297fa640 1 -- 172.21.15.37:0/444604775 <== mds.0 v2:172.21.15.37:6834/3366017865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604752a8d60 con 0x5604754cb180 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f297fa640 10 client.8442 handle_command_reply: tid=0 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 client.8442 shutdown 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 -- 172.21.15.37:0/444604775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560475508c90 msgr2=0x5604754ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560475508c90 0x5604754ca370 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f8f4001cad0 tx=0x7f8f4011f070 comp rx=0 tx=0).stop 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 -- 172.21.15.37:0/444604775 shutdown_connections 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604755095b0 0x5604754ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5604754cb180 0x56047551a410 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560475508c90 0x5604754ca370 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/444604775 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5604754bf290 0x5604754c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f297fa640 0 client.8442 ms_handle_reset on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:46.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 -- 172.21.15.37:0/444604775 >> 172.21.15.37:0/444604775 conn(0x5604754bb240 msgr2=0x5604754bdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 -- 172.21.15.37:0/444604775 shutdown_connections 2024-04-25T09:00:46.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 1 -- 172.21.15.37:0/444604775 wait complete. 2024-04-25T09:00:46.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.459+0000 7f8f53ce6000 20 client.8442 trim_cache size 0 max 16384 2024-04-25T09:00:46.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.475+0000 7f8f53ce6000 1 Processor -- start 2024-04-25T09:00:46.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.475+0000 7f8f53ce6000 1 -- start start 2024-04-25T09:00:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.475+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604754bf290 0x560475564ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.475+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560475508c90 0x560475565220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.475+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604755095b0 0x560475565760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.475+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c2011a0 con 0x5604754bf290 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.475+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0b0560 con 0x5604755095b0 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c155e50 con 0x560475508c90 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604754bf290 0x560475564ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604754bf290 0x560475564ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54432/0 (socket says 172.21.15.37:54432) 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4a59c640 1 -- 172.21.15.37:0/3666338781 learned_addr learned my addr 172.21.15.37:0/3666338781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/3666338781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560475508c90 0x560475565220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4ad9d640 1 --2- 172.21.15.37:0/3666338781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604755095b0 0x560475565760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4a59c640 1 -- 172.21.15.37:0/3666338781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560475508c90 msgr2=0x560475565220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/3666338781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560475508c90 0x560475565220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4a59c640 1 -- 172.21.15.37:0/3666338781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604755095b0 msgr2=0x560475565760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/3666338781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604755095b0 0x560475565760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4a59c640 1 -- 172.21.15.37:0/3666338781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560475364c50 con 0x5604754bf290 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/3666338781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560475508c90 0x560475565220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/3666338781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604754bf290 0x560475564ce0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0011f0 tx=0x7f8f3c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a093f62521bcc799 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f2b7fe640 1 -- 172.21.15.37:0/3666338781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c023830 con 0x5604754bf290 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f2b7fe640 1 -- 172.21.15.37:0/3666338781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c013420 con 0x5604754bf290 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f2b7fe640 1 -- 172.21.15.37:0/3666338781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c0137d0 con 0x5604754bf290 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3666338781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604754bf290 msgr2=0x560475564ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/3666338781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604754bf290 0x560475564ce0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c0011f0 tx=0x7f8f3c012000 comp rx=0 tx=0).stop 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3666338781 shutdown_connections 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/3666338781 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604755095b0 0x560475565760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/3666338781 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560475508c90 0x560475565220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/3666338781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604754bf290 0x560475564ce0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3666338781 >> 172.21.15.37:0/3666338781 conn(0x5604754bb240 msgr2=0x5604754c3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3666338781 shutdown_connections 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3666338781 wait complete. 2024-04-25T09:00:46.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 -- start start 2024-04-25T09:00:46.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 Processor -- start 2024-04-25T09:00:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604754bf290 0x560475568a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560475508c90 0x560475568f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x560475523e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c0aff60 con 0x5604755095b0 2024-04-25T09:00:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604753c3f20 con 0x5604754bf290 2024-04-25T09:00:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604752c4430 con 0x560475508c90 2024-04-25T09:00:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 10 client.0 _ll_register_callbacks cb 0x7f8f400069d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x560475523e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x560475523e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54448/0 (socket says 172.21.15.37:54448) 2024-04-25T09:00:46.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4ad9d640 1 -- 172.21.15.37:0/3397301572 learned_addr learned my addr 172.21.15.37:0/3397301572 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f49d9b640 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560475508c90 0x560475568f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604754bf290 0x560475568a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4ad9d640 1 -- 172.21.15.37:0/3397301572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560475508c90 msgr2=0x560475568f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4ad9d640 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560475508c90 0x560475568f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4ad9d640 1 -- 172.21.15.37:0/3397301572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604754bf290 msgr2=0x560475568a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4ad9d640 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604754bf290 0x560475568a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4ad9d640 1 -- 172.21.15.37:0/3397301572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604752d5120 con 0x5604755095b0 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604754bf290 0x560475568a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f4ad9d640 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x560475523e90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f8f44007920 tx=0x7f8f44007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e80b8d66ff893c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f297fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f297fa640 1 -- 172.21.15.37:0/3397301572 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4401a070 con 0x5604755095b0 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f297fa640 1 -- 172.21.15.37:0/3397301572 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f8f44012c90 con 0x5604755095b0 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.479+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3397301572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f3c013420 con 0x5604755095b0 2024-04-25T09:00:46.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f297fa640 1 -- 172.21.15.37:0/3397301572 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f44025050 con 0x5604755095b0 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 10 client.8448 fetch_fsmap learned FSMap version 73 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3397301572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5604752b4170 con 0x5604755095b0 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f297fa640 1 -- 172.21.15.37:0/3397301572 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f3c013420 con 0x5604755095b0 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f297fa640 1 -- 172.21.15.37:0/3397301572 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8f44028020 con 0x5604755095b0 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 10 client.8448 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 10 client.8448 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 20 client.8448 populate_metadata read hostname 'smithi037' 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x560475524ce0 0x5604755250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 4 client.8448 mds_command: new command op to 8340 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3397301572 --> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) v1 -- 0x56047531a5b0 con 0x560475524ce0 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x560475524ce0 0x5604755250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f4a59c640 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x560475524ce0 0x5604755250f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=977399bcef188e87 server_cookie=acd3950e3a71fe83 in_seq=0 out_seq=0 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f297fa640 10 client.8448 ms_handle_connect on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f297fa640 1 -- 172.21.15.37:0/3397301572 <== mds.0 v2:172.21.15.37:6834/3366017865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56047531a5b0 con 0x560475524ce0 2024-04-25T09:00:46.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f297fa640 10 client.8448 handle_command_reply: tid=0 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 client.8448 shutdown 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3397301572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 msgr2=0x560475523e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x560475523e90 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f8f44007920 tx=0x7f8f44007de0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3397301572 shutdown_connections 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x560475524ce0 0x5604755250f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5604755095b0 0x560475523e90 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x560475508c90 0x560475568f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f297fa640 0 client.8448 ms_handle_reset on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 --2- 172.21.15.37:0/3397301572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5604754bf290 0x560475568a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3397301572 >> 172.21.15.37:0/3397301572 conn(0x5604754bb240 msgr2=0x5604754c3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3397301572 shutdown_connections 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 1 -- 172.21.15.37:0/3397301572 wait complete. 2024-04-25T09:00:46.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f53ce6000 20 client.8448 trim_cache size 0 max 16384 2024-04-25T09:00:46.482 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:00:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-04-25T09:00:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "3b2106ec-485c-488c-8d61-1eace6129d5a", 2024-04-25T09:00:46.482 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-04-25T09:00:46.482 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:00:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 -- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8f2c066f60 msgr2=0x7f8f2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8f2c066f60 0x7f8f2c069420 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c000f70 tx=0x7f8f3c007b60 comp rx=0 tx=0).stop 2024-04-25T09:00:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 -- 172.21.15.37:0/834013140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c0adc70 msgr2=0x7f8f4c0b10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c0adc70 0x7f8f4c0b10a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f8f400ad040 tx=0x7f8f40019300 comp rx=0 tx=0).stop 2024-04-25T09:00:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 -- 172.21.15.37:0/834013140 shutdown_connections 2024-04-25T09:00:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8f2c066f60 0x7f8f2c069420 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8f4c0cf510 0x7f8f4c1f7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f4c0adc70 0x7f8f4c0b10a0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 --2- 172.21.15.37:0/834013140 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8f4c0ac170 0x7f8f4c157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 -- 172.21.15.37:0/834013140 >> 172.21.15.37:0/834013140 conn(0x7f8f4c06ae60 msgr2=0x7f8f4c0cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 -- 172.21.15.37:0/834013140 shutdown_connections 2024-04-25T09:00:46.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f8f50b79640 1 -- 172.21.15.37:0/834013140 wait complete. 2024-04-25T09:00:46.484 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T09:00:46.483+0000 7f793b32a640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5639a9c50100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T09:00:46.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:46.493 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:00:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.615+0000 7fafec7e8640 1 Processor -- start 2024-04-25T09:00:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafec7e8640 1 -- start start 2024-04-25T09:00:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafec7e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe40d07f0 0x7fafe40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafec7e8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fafe40d1cd0 0x7fafe406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafec7e8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafe406be60 con 0x7fafe415da80 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafec7e8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafe406c440 con 0x7fafe40d1cd0 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafec7e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe415de60 con 0x7fafe40d07f0 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafea584640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fafe415da80 0x7fafe40d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50730/0 (socket says 172.21.15.37:50730) 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafea584640 1 -- 172.21.15.37:0/1240644173 learned_addr learned my addr 172.21.15.37:0/1240644173 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafead85640 1 --2- 172.21.15.37:0/1240644173 >> v2:172.21.15.203:3300/0 conn(0x7fafe40d1cd0 0x7fafe406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe40d07f0 0x7fafe40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613515354 0 0) 0x7fafe406be60 con 0x7fafe415da80 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafd0003530 con 0x7fafe415da80 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2194399852 0 0) 0x7fafd0003530 con 0x7fafe415da80 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe40d07f0 msgr2=0x7fafe40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe40d07f0 0x7fafe40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 >> v2:172.21.15.203:3300/0 conn(0x7fafe40d1cd0 msgr2=0x7fafe406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 --2- 172.21.15.37:0/1240644173 >> v2:172.21.15.203:3300/0 conn(0x7fafe40d1cd0 0x7fafe406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe40d2450 con 0x7fafe415da80 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fafe0001680 con 0x7fafe415da80 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 >> v1:172.21.15.37:6789/0 conn(0x7fafe415da80 legacy=0x7fafe40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafd0003a30 0x7fafd0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd0004810 0x7fafd0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe40d07f0 0x7fafd00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe406c440 con 0x7fafd0004810 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe415de60 con 0x7fafd0003a30 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd000b9d0 con 0x7fafe40d07f0 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd0004810 0x7fafd0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafea584640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafd0003a30 0x7fafd0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9d83640 1 -- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe40d07f0 msgr2=0x7fafd00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafead85640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe40d07f0 0x7fafd00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe40d07f0 0x7fafd00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9d83640 1 -- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafd0003a30 msgr2=0x7fafd0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafd0003a30 0x7fafd0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafe9d83640 1 -- 172.21.15.37:0/1240644173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd0003530 con 0x7fafd0004810 2024-04-25T09:00:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafea584640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafd0003a30 0x7fafd0003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.619+0000 7fafead85640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe40d07f0 0x7fafd00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd0004810 0x7fafd0004c90 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009af0 tx=0x7fafd401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d9f3f28cd95b8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4009690 con 0x7fafd0004810 2024-04-25T09:00:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd40059a0 con 0x7fafd0004810 2024-04-25T09:00:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafe9582640 1 -- 172.21.15.37:0/1240644173 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4005d90 con 0x7fafd0004810 2024-04-25T09:00:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafec7e8640 1 -- 172.21.15.37:0/1240644173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd0004810 msgr2=0x7fafd0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafec7e8640 1 --2- 172.21.15.37:0/1240644173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafd0004810 0x7fafd0004c90 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009af0 tx=0x7fafd401ca30 comp rx=0 tx=0).stop 2024-04-25T09:00:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafead85640 1 -- 172.21.15.37:0/1240644173 reap_dead start 2024-04-25T09:00:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafec7e8640 1 -- 172.21.15.37:0/1240644173 shutdown_connections 2024-04-25T09:00:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafec7e8640 1 -- 172.21.15.37:0/1240644173 >> 172.21.15.37:0/1240644173 conn(0x7fafe408b000 msgr2=0x7fafe40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafec7e8640 1 -- 172.21.15.37:0/1240644173 shutdown_connections 2024-04-25T09:00:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafec7e8640 1 -- 172.21.15.37:0/1240644173 wait complete. 2024-04-25T09:00:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafec7e8640 1 Processor -- start 2024-04-25T09:00:46.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.623+0000 7fafec7e8640 1 -- start start 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafec7e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe4151af0 0x7fafe40c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafec7e8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe415acf0 0x7fafe40cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafec7e8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe40cc970 0x7fafe41f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafec7e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40d26f0 con 0x7fafe4151af0 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafec7e8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40d2160 con 0x7fafe40cc970 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafec7e8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe40d2870 con 0x7fafe415acf0 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafea584640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe4151af0 0x7fafe40c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafea584640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe4151af0 0x7fafe40c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54466/0 (socket says 172.21.15.37:54466) 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafea584640 1 -- 172.21.15.37:0/1578348333 learned_addr learned my addr 172.21.15.37:0/1578348333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafead85640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe40cc970 0x7fafe41f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafea584640 1 -- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe415acf0 msgr2=0x7fafe40cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe415acf0 0x7fafe40cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafea584640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe415acf0 0x7fafe40cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafea584640 1 -- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe40cc970 msgr2=0x7fafe41f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafea584640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe40cc970 0x7fafe41f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafea584640 1 -- 172.21.15.37:0/1578348333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe4160af0 con 0x7fafe4151af0 2024-04-25T09:00:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafead85640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe40cc970 0x7fafe41f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafea584640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe4151af0 0x7fafe40c82e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fafe0001930 tx=0x7fafe0010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafdaffd640 1 -- 172.21.15.37:0/1578348333 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe001b070 con 0x7fafe4151af0 2024-04-25T09:00:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafdaffd640 1 -- 172.21.15.37:0/1578348333 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe0011040 con 0x7fafe4151af0 2024-04-25T09:00:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafec7e8640 1 -- 172.21.15.37:0/1578348333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe40c8820 con 0x7fafe4151af0 2024-04-25T09:00:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafdaffd640 1 -- 172.21.15.37:0/1578348333 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe000ee60 con 0x7fafe4151af0 2024-04-25T09:00:46.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafec7e8640 1 -- 172.21.15.37:0/1578348333 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe40c96b0 con 0x7fafe4151af0 2024-04-25T09:00:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafdaffd640 1 -- 172.21.15.37:0/1578348333 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fafe001e070 con 0x7fafe4151af0 2024-04-25T09:00:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafdaffd640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafc8066f60 0x7fafc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafdaffd640 1 -- 172.21.15.37:0/1578348333 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fafe0087eb0 con 0x7fafe4151af0 2024-04-25T09:00:46.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.627+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafc8066f60 0x7fafc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 Processor -- start 2024-04-25T09:00:46.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 -- start start 2024-04-25T09:00:46.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafc8066f60 0x7fafc8069420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fafd40ae000 tx=0x7fafd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65b19d0 0x5569d65b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b22f0 0x5569d6567b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65680a0 0x5569d6568480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d63b4ff0 con 0x5569d65680a0 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d633d150 con 0x5569d65b19d0 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569d63b1090 con 0x5569d65b22f0 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafead85640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65680a0 0x5569d6568480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafead85640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65680a0 0x5569d6568480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54474/0 (socket says 172.21.15.37:54474) 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafead85640 1 -- 172.21.15.37:0/1187758458 learned_addr learned my addr 172.21.15.37:0/1187758458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1187758458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b22f0 0x5569d6567b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafea584640 1 --2- 172.21.15.37:0/1187758458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65b19d0 0x5569d65b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafead85640 1 -- 172.21.15.37:0/1187758458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b22f0 msgr2=0x5569d6567b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafead85640 1 --2- 172.21.15.37:0/1187758458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b22f0 0x5569d6567b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafead85640 1 -- 172.21.15.37:0/1187758458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65b19d0 msgr2=0x5569d65b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafead85640 1 --2- 172.21.15.37:0/1187758458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65b19d0 0x5569d65b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafead85640 1 -- 172.21.15.37:0/1187758458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569d654b0d0 con 0x5569d65680a0 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafead85640 1 --2- 172.21.15.37:0/1187758458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65680a0 0x5569d6568480 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fafe406bc80 tx=0x7fafdc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f3ef0475d3c497 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafc7fff640 1 -- 172.21.15.37:0/1187758458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc003ba0 con 0x5569d65680a0 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafc7fff640 1 -- 172.21.15.37:0/1187758458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc003d40 con 0x5569d65680a0 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafc7fff640 1 -- 172.21.15.37:0/1187758458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc015890 con 0x5569d65680a0 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 -- 172.21.15.37:0/1187758458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65680a0 msgr2=0x5569d6568480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 --2- 172.21.15.37:0/1187758458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65680a0 0x5569d6568480 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fafe406bc80 tx=0x7fafdc009750 comp rx=0 tx=0).stop 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 -- 172.21.15.37:0/1187758458 shutdown_connections 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 --2- 172.21.15.37:0/1187758458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65680a0 0x5569d6568480 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 --2- 172.21.15.37:0/1187758458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b22f0 0x5569d6567b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 --2- 172.21.15.37:0/1187758458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65b19d0 0x5569d65b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 -- 172.21.15.37:0/1187758458 >> 172.21.15.37:0/1187758458 conn(0x5569d6564020 msgr2=0x5569d6566440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 -- 172.21.15.37:0/1187758458 shutdown_connections 2024-04-25T09:00:46.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 -- 172.21.15.37:0/1187758458 wait complete. 2024-04-25T09:00:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 -- start start 2024-04-25T09:00:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.631+0000 7fafef955000 1 Processor -- start 2024-04-25T09:00:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65680a0 0x5569d65967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b19d0 0x5569d6596d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d6597270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d6379250 con 0x5569d65b22f0 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d636f690 con 0x5569d65680a0 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569d633da40 con 0x5569d65b19d0 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 10 client.0 _ll_register_callbacks cb 0x5569d6552e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafead85640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d6597270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafead85640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d6597270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54478/0 (socket says 172.21.15.37:54478) 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafead85640 1 -- 172.21.15.37:0/3321122004 learned_addr learned my addr 172.21.15.37:0/3321122004 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafe9d83640 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b19d0 0x5569d6596d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafea584640 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65680a0 0x5569d65967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafead85640 1 -- 172.21.15.37:0/3321122004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b19d0 msgr2=0x5569d6596d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafead85640 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b19d0 0x5569d6596d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafead85640 1 -- 172.21.15.37:0/3321122004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65680a0 msgr2=0x5569d65967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafead85640 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65680a0 0x5569d65967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafead85640 1 -- 172.21.15.37:0/3321122004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569d64621c0 con 0x5569d65b22f0 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafead85640 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d6597270 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009790 tx=0x7fafdc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4f4a655800d2e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafc57fa640 1 -- 172.21.15.37:0/3321122004 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc015c00 con 0x5569d65b22f0 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafc57fa640 1 -- 172.21.15.37:0/3321122004 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fafdc015da0 con 0x5569d65b22f0 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 -- 172.21.15.37:0/3321122004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafdc003d40 con 0x5569d65b22f0 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafc57fa640 1 -- 172.21.15.37:0/3321122004 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafdc004030 con 0x5569d65b22f0 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 10 client.8460 fetch_fsmap learned FSMap version 73 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 -- 172.21.15.37:0/3321122004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5569d645e170 con 0x5569d65b22f0 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafc57fa640 1 -- 172.21.15.37:0/3321122004 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafdc003d40 con 0x5569d65b22f0 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafc57fa640 1 -- 172.21.15.37:0/3321122004 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fafdc0041d0 con 0x5569d65b22f0 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 10 client.8460 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T09:00:46.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 10 client.8460 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 20 client.8460 populate_metadata read hostname 'smithi037' 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5569d65c1c80 0x5569d65c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 4 client.8460 mds_command: new command op to 8340 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 -- 172.21.15.37:0/3321122004 --> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5569d6350c60 con 0x5569d65c1c80 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafea584640 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5569d65c1c80 0x5569d65c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafc57fa640 10 client.8460 ms_handle_connect on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafea584640 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5569d65c1c80 0x5569d65c4070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a862f69bd0cf955c server_cookie=6f871917dee64395 in_seq=0 out_seq=0 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafc57fa640 1 -- 172.21.15.37:0/3321122004 <== mds.0 v2:172.21.15.37:6834/3366017865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5569d6350c60 con 0x5569d65c1c80 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafc57fa640 10 client.8460 handle_command_reply: tid=0 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 client.8460 shutdown 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 -- 172.21.15.37:0/3321122004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 msgr2=0x5569d6597270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d6597270 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fafdc009790 tx=0x7fafdc004a50 comp rx=0 tx=0).stop 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 -- 172.21.15.37:0/3321122004 shutdown_connections 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5569d65c1c80 0x5569d65c4070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d6597270 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b19d0 0x5569d6596d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 --2- 172.21.15.37:0/3321122004 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65680a0 0x5569d65967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafc57fa640 0 client.8460 ms_handle_reset on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 -- 172.21.15.37:0/3321122004 >> 172.21.15.37:0/3321122004 conn(0x5569d6564020 msgr2=0x5569d6566ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 -- 172.21.15.37:0/3321122004 shutdown_connections 2024-04-25T09:00:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 1 -- 172.21.15.37:0/3321122004 wait complete. 2024-04-25T09:00:46.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.635+0000 7fafef955000 20 client.8460 trim_cache size 0 max 16384 2024-04-25T09:00:46.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafef955000 1 Processor -- start 2024-04-25T09:00:46.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafef955000 1 -- start start 2024-04-25T09:00:46.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafef955000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65680a0 0x5569d660dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafef955000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b19d0 0x5569d660e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafef955000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b22f0 0x5569d660e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafef955000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe4200f20 con 0x5569d65b19d0 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafef955000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe4200060 con 0x5569d65680a0 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafef955000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafe41ff580 con 0x5569d65b22f0 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafe9d83640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b19d0 0x5569d660e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafe9d83640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b19d0 0x5569d660e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54490/0 (socket says 172.21.15.37:54490) 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafe9d83640 1 -- 172.21.15.37:0/1175435053 learned_addr learned my addr 172.21.15.37:0/1175435053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafe9d83640 1 -- 172.21.15.37:0/1175435053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b22f0 msgr2=0x5569d660e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafea584640 1 --2- 172.21.15.37:0/1175435053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65680a0 0x5569d660dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafead85640 1 --2- 172.21.15.37:0/1175435053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b22f0 0x5569d660e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1175435053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b22f0 0x5569d660e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafe9d83640 1 -- 172.21.15.37:0/1175435053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65680a0 msgr2=0x5569d660dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1175435053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65680a0 0x5569d660dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafe9d83640 1 -- 172.21.15.37:0/1175435053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569d63fbde0 con 0x5569d65b19d0 2024-04-25T09:00:46.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafe9d83640 1 --2- 172.21.15.37:0/1175435053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b19d0 0x5569d660e420 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009a90 tx=0x7fafd4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1115feef344664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafc77fe640 1 -- 172.21.15.37:0/1175435053 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd4006700 con 0x5569d65b19d0 2024-04-25T09:00:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafc77fe640 1 -- 172.21.15.37:0/1175435053 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd40c9070 con 0x5569d65b19d0 2024-04-25T09:00:46.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.651+0000 7fafc77fe640 1 -- 172.21.15.37:0/1175435053 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40a5740 con 0x5569d65b19d0 2024-04-25T09:00:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- 172.21.15.37:0/1175435053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b19d0 msgr2=0x5569d660e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 --2- 172.21.15.37:0/1175435053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b19d0 0x5569d660e420 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009a90 tx=0x7fafd4004740 comp rx=0 tx=0).stop 2024-04-25T09:00:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- 172.21.15.37:0/1175435053 shutdown_connections 2024-04-25T09:00:46.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 --2- 172.21.15.37:0/1175435053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65b22f0 0x5569d660e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 --2- 172.21.15.37:0/1175435053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b19d0 0x5569d660e420 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 --2- 172.21.15.37:0/1175435053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65680a0 0x5569d660dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- 172.21.15.37:0/1175435053 >> 172.21.15.37:0/1175435053 conn(0x5569d6564020 msgr2=0x5569d65befc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- 172.21.15.37:0/1175435053 shutdown_connections 2024-04-25T09:00:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- 172.21.15.37:0/1175435053 wait complete. 2024-04-25T09:00:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- start start 2024-04-25T09:00:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 Processor -- start 2024-04-25T09:00:46.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65680a0 0x5569d6611d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65b19d0 0x5569d6612290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d65c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569d634fe00 con 0x5569d65b22f0 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe41ffa60 con 0x5569d65b19d0 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5569d63b4300 con 0x5569d65680a0 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 10 client.0 _ll_register_callbacks cb 0x5569d6533040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafead85640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d65c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafead85640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d65c3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54496/0 (socket says 172.21.15.37:54496) 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafead85640 1 -- 172.21.15.37:0/875677326 learned_addr learned my addr 172.21.15.37:0/875677326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafea584640 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65680a0 0x5569d6611d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafe9d83640 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65b19d0 0x5569d6612290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafead85640 1 -- 172.21.15.37:0/875677326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65680a0 msgr2=0x5569d6611d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafead85640 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65680a0 0x5569d6611d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafead85640 1 -- 172.21.15.37:0/875677326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65b19d0 msgr2=0x5569d6612290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafead85640 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65b19d0 0x5569d6612290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafead85640 1 -- 172.21.15.37:0/875677326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5569d63568e0 con 0x5569d65b22f0 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafead85640 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d65c3fc0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0070e0 tx=0x7fafdc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac540fae545591e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafc57fa640 1 -- 172.21.15.37:0/875677326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafdc041060 con 0x5569d65b22f0 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafc57fa640 1 -- 172.21.15.37:0/875677326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fafdc007140 con 0x5569d65b22f0 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- 172.21.15.37:0/875677326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fafd40c9070 con 0x5569d65b22f0 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafc57fa640 1 -- 172.21.15.37:0/875677326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafdc047050 con 0x5569d65b22f0 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 10 client.8466 fetch_fsmap learned FSMap version 73 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- 172.21.15.37:0/875677326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5569d640e8f0 con 0x5569d65b22f0 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafc57fa640 1 -- 172.21.15.37:0/875677326 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd40c9070 con 0x5569d65b22f0 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafc57fa640 1 -- 172.21.15.37:0/875677326 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fafdc04a020 con 0x5569d65b22f0 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 10 client.8466 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 10 client.8466 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:00:46.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 20 client.8466 populate_metadata read hostname 'smithi037' 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5569d65c4920 0x5569d65c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 4 client.8466 mds_command: new command op to 8340 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafef955000 1 -- 172.21.15.37:0/875677326 --> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5569d63c2530 con 0x5569d65c4920 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafea584640 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5569d65c4920 0x5569d65c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafea584640 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5569d65c4920 0x5569d65c4d30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4d69104523ea39c server_cookie=498f20b1811de8c in_seq=0 out_seq=0 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.655+0000 7fafc57fa640 10 client.8466 ms_handle_connect on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafc57fa640 1 -- 172.21.15.37:0/875677326 <== mds.0 v2:172.21.15.37:6834/3366017865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+263 (crc 0 0 0) 0x5569d63c2530 con 0x5569d65c4920 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafc57fa640 10 client.8466 handle_command_reply: tid=0 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 1 client.8466 shutdown 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 1 -- 172.21.15.37:0/875677326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 msgr2=0x5569d65c3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d65c3fc0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fafdc0070e0 tx=0x7fafdc00b040 comp rx=0 tx=0).stop 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 1 -- 172.21.15.37:0/875677326 shutdown_connections 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5569d65c4920 0x5569d65c4d30 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5569d65b22f0 0x5569d65c3fc0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5569d65b19d0 0x5569d6612290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafc57fa640 0 client.8466 ms_handle_reset on v2:172.21.15.37:6834/3366017865 2024-04-25T09:00:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 1 --2- 172.21.15.37:0/875677326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5569d65680a0 0x5569d6611d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 1 -- 172.21.15.37:0/875677326 >> 172.21.15.37:0/875677326 conn(0x5569d6564020 msgr2=0x5569d65b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 1 -- 172.21.15.37:0/875677326 shutdown_connections 2024-04-25T09:00:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 1 -- 172.21.15.37:0/875677326 wait complete. 2024-04-25T09:00:46.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafef955000 20 client.8466 trim_cache size 0 max 16384 2024-04-25T09:00:46.656 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:00:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:00:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-04-25T09:00:46.656 INFO:teuthology.orchestra.run.smithi037.stdout: "3b2106ec-485c-488c-8d61-1eace6129d5a": { 2024-04-25T09:00:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/dir", 2024-04-25T09:00:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "3b2106ec-485c-488c-8d61-1eace6129d5a", 2024-04-25T09:00:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,repair" 2024-04-25T09:00:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:00:46.657 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:00:46.657 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:00:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 -- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafc8066f60 msgr2=0x7fafc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafc8066f60 0x7fafc8069420 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fafd40ae000 tx=0x7fafd4008040 comp rx=0 tx=0).stop 2024-04-25T09:00:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 -- 172.21.15.37:0/1578348333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe4151af0 msgr2=0x7fafe40c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe4151af0 0x7fafe40c82e0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fafe0001930 tx=0x7fafe0010ef0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 -- 172.21.15.37:0/1578348333 shutdown_connections 2024-04-25T09:00:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fafc8066f60 0x7fafc8069420 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fafe40cc970 0x7fafe41f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fafe415acf0 0x7fafe40cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 --2- 172.21.15.37:0/1578348333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fafe4151af0 0x7fafe40c82e0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 -- 172.21.15.37:0/1578348333 >> 172.21.15.37:0/1578348333 conn(0x7fafe408b000 msgr2=0x7fafe40b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 -- 172.21.15.37:0/1578348333 shutdown_connections 2024-04-25T09:00:46.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:00:46.659+0000 7fafec7e8640 1 -- 172.21.15.37:0/1578348333 wait complete. 2024-04-25T09:00:46.667 INFO:tasks.cephfs.filesystem:scrub status for tag:3b2106ec-485c-488c-8d61-1eace6129d5a - {'path': '/dir', 'tag': '3b2106ec-485c-488c-8d61-1eace6129d5a', 'options': 'recursive,repair'} 2024-04-25T09:00:56.433 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:00:56.436 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:16.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:16.669 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:01:16.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.819+0000 7f55f241e640 1 Processor -- start 2024-04-25T09:01:16.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.819+0000 7f55f241e640 1 -- start start 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.819+0000 7f55f241e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f55ec0b2690 0x7f55ec06f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.819+0000 7f55f241e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55ec15ef50 con 0x7f55ec15ea90 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.819+0000 7f55f241e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55ec15f230 con 0x7f55ec0af2b0 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.819+0000 7f55f241e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55ec06fe50 con 0x7f55ec0b2690 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eb7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f55ec15ea90 0x7f55ec0adc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40708/0 (socket says 172.21.15.37:40708) 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eb7fe640 1 -- 172.21.15.37:0/2507349425 learned_addr learned my addr 172.21.15.37:0/2507349425 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55f09bb640 1 --2- 172.21.15.37:0/2507349425 >> v2:172.21.15.203:3300/0 conn(0x7f55ec0b2690 0x7f55ec06f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2001001623 0 0) 0x7f55ec15ef50 con 0x7f55ec15ea90 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55d4003530 con 0x7f55ec15ea90 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2801515542 0 0) 0x7f55d4003530 con 0x7f55ec15ea90 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 >> v2:172.21.15.203:3300/0 conn(0x7f55ec0b2690 msgr2=0x7f55ec06f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 --2- 172.21.15.37:0/2507349425 >> v2:172.21.15.203:3300/0 conn(0x7f55ec0b2690 0x7f55ec06f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 >> v1:172.21.15.203:6789/0 conn(0x7f55ec0af2b0 legacy=0x7f55ec15e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec0ceaf0 con 0x7f55ec15ea90 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f55e0001680 con 0x7f55ec15ea90 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 >> v1:172.21.15.37:6789/0 conn(0x7f55ec15ea90 legacy=0x7f55ec0adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d4003a30 0x7f55d4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d4004810 0x7f55d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55d40051d0 0x7f55d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec06fe50 con 0x7f55d4004810 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d400c100 con 0x7f55d4003a30 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d400c300 con 0x7f55d40051d0 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d4004810 0x7f55d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55ebfff640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d4003a30 0x7f55d4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55f09bb640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55d40051d0 0x7f55d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eb7fe640 1 -- 172.21.15.37:0/2507349425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55d40051d0 msgr2=0x7f55d400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55d40051d0 0x7f55d400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eb7fe640 1 -- 172.21.15.37:0/2507349425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d4003a30 msgr2=0x7f55d4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d4003a30 0x7f55d4003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eb7fe640 1 -- 172.21.15.37:0/2507349425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d4003530 con 0x7f55d4004810 2024-04-25T09:01:16.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55f09bb640 1 -- 172.21.15.37:0/2507349425 reap_dead start 2024-04-25T09:01:16.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55ebfff640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d4003a30 0x7f55d4003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:16.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d4004810 0x7f55d4004c90 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0b3040 tx=0x7f55e001f2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=427770ee5551b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0007d40 con 0x7f55d4004810 2024-04-25T09:01:16.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e00ac070 con 0x7f55d4004810 2024-04-25T09:01:16.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.823+0000 7f55eaffd640 1 -- 172.21.15.37:0/2507349425 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0005b20 con 0x7f55d4004810 2024-04-25T09:01:16.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 -- 172.21.15.37:0/2507349425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d4004810 msgr2=0x7f55d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d4004810 0x7f55d4004c90 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0b3040 tx=0x7f55e001f2d0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 -- 172.21.15.37:0/2507349425 shutdown_connections 2024-04-25T09:01:16.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 --2- 172.21.15.37:0/2507349425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55d4004810 0x7f55d4004c90 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 -- 172.21.15.37:0/2507349425 >> 172.21.15.37:0/2507349425 conn(0x7f55ec099a70 msgr2=0x7f55ec0b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 -- 172.21.15.37:0/2507349425 shutdown_connections 2024-04-25T09:01:16.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 -- 172.21.15.37:0/2507349425 wait complete. 2024-04-25T09:01:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 Processor -- start 2024-04-25T09:01:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 -- start start 2024-04-25T09:01:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d4004810 0x7f55ec14afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55ec14b510 0x7f55ec15c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55ec15cd90 0x7f55ec0c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec0cf010 con 0x7f55ec15cd90 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec070130 con 0x7f55d4004810 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec15bb30 con 0x7f55ec14b510 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f09bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55ec15cd90 0x7f55ec0c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f09bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55ec15cd90 0x7f55ec0c86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33738/0 (socket says 172.21.15.37:33738) 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f09bb640 1 -- 172.21.15.37:0/3862413563 learned_addr learned my addr 172.21.15.37:0/3862413563 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55ebfff640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d4004810 0x7f55ec14afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55ec14b510 0x7f55ec15c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f09bb640 1 -- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55ec14b510 msgr2=0x7f55ec15c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f09bb640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55ec14b510 0x7f55ec15c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f09bb640 1 -- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d4004810 msgr2=0x7f55ec14afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f09bb640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d4004810 0x7f55ec14afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f09bb640 1 -- 172.21.15.37:0/3862413563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec15bd90 con 0x7f55ec15cd90 2024-04-25T09:01:16.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55ebfff640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d4004810 0x7f55ec14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:16.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f09bb640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55ec15cd90 0x7f55ec0c86c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0b2db0 tx=0x7f55e400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55e8ff9640 1 -- 172.21.15.37:0/3862413563 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4016420 con 0x7f55ec15cd90 2024-04-25T09:01:16.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55e8ff9640 1 -- 172.21.15.37:0/3862413563 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e401b070 con 0x7f55ec15cd90 2024-04-25T09:01:16.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 -- 172.21.15.37:0/3862413563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ec14e7c0 con 0x7f55ec15cd90 2024-04-25T09:01:16.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55e8ff9640 1 -- 172.21.15.37:0/3862413563 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e4016820 con 0x7f55ec15cd90 2024-04-25T09:01:16.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.827+0000 7f55f241e640 1 -- 172.21.15.37:0/3862413563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ec15c330 con 0x7f55ec15cd90 2024-04-25T09:01:16.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55e8ff9640 1 -- 172.21.15.37:0/3862413563 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f55e40169c0 con 0x7f55ec15cd90 2024-04-25T09:01:16.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55e8ff9640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55d8066f60 0x7f55d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55e8ff9640 1 -- 172.21.15.37:0/3862413563 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f55e4085f30 con 0x7f55ec15cd90 2024-04-25T09:01:16.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55ebfff640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55d8066f60 0x7f55d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55f558b000 1 Processor -- start 2024-04-25T09:01:16.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55f558b000 1 -- start start 2024-04-25T09:01:16.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55ebfff640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55d8066f60 0x7f55d8069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f55dc007fd0 tx=0x7f55dc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55f558b000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586612309d0 0x558661230db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55f558b000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586611e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55f558b000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586611e70a0 0x5586611e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55f558b000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558661033ff0 con 0x5586612312f0 2024-04-25T09:01:16.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55f558b000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558660fbc150 con 0x5586612309d0 2024-04-25T09:01:16.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55f558b000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558661030090 con 0x5586611e70a0 2024-04-25T09:01:16.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586611e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586611e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33742/0 (socket says 172.21.15.37:33742) 2024-04-25T09:01:16.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55eb7fe640 1 -- 172.21.15.37:0/1902678852 learned_addr learned my addr 172.21.15.37:0/1902678852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:16.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55ebfff640 1 --2- 172.21.15.37:0/1902678852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586612309d0 0x558661230db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55f09bb640 1 --2- 172.21.15.37:0/1902678852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586611e70a0 0x5586611e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55eb7fe640 1 -- 172.21.15.37:0/1902678852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586611e70a0 msgr2=0x5586611e7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/1902678852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586611e70a0 0x5586611e7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55eb7fe640 1 -- 172.21.15.37:0/1902678852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586612309d0 msgr2=0x558661230db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/1902678852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586612309d0 0x558661230db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.831+0000 7f55eb7fe640 1 -- 172.21.15.37:0/1902678852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586611ca0d0 con 0x5586612312f0 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/1902678852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586611e6b60 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f55e001f310 tx=0x7f55e00aa070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897aa4b1ffbd0a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55d1ffb640 1 -- 172.21.15.37:0/1902678852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e00047f0 con 0x5586612312f0 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55d1ffb640 1 -- 172.21.15.37:0/1902678852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e00ac040 con 0x5586612312f0 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55d1ffb640 1 -- 172.21.15.37:0/1902678852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0004b90 con 0x5586612312f0 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 -- 172.21.15.37:0/1902678852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 msgr2=0x5586611e6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 --2- 172.21.15.37:0/1902678852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586611e6b60 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f55e001f310 tx=0x7f55e00aa070 comp rx=0 tx=0).stop 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 -- 172.21.15.37:0/1902678852 shutdown_connections 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 --2- 172.21.15.37:0/1902678852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586611e70a0 0x5586611e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 --2- 172.21.15.37:0/1902678852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586611e6b60 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 --2- 172.21.15.37:0/1902678852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586612309d0 0x558661230db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 -- 172.21.15.37:0/1902678852 >> 172.21.15.37:0/1902678852 conn(0x5586611e3020 msgr2=0x5586611e5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 -- 172.21.15.37:0/1902678852 shutdown_connections 2024-04-25T09:01:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 -- 172.21.15.37:0/1902678852 wait complete. 2024-04-25T09:01:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 -- start start 2024-04-25T09:01:16.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 Processor -- start 2024-04-25T09:01:16.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x5586612157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612309d0 0x558661215d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x558661216270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558660ff8250 con 0x5586612312f0 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558660fee690 con 0x5586611e70a0 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558660fbca40 con 0x5586612309d0 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 10 client.0 _ll_register_callbacks cb 0x5586611d1e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f09bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x558661216270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f09bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x558661216270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33756/0 (socket says 172.21.15.37:33756) 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f09bb640 1 -- 172.21.15.37:0/1047811605 learned_addr learned my addr 172.21.15.37:0/1047811605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55ebfff640 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x5586612157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612309d0 0x558661215d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f09bb640 1 -- 172.21.15.37:0/1047811605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612309d0 msgr2=0x558661215d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f09bb640 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612309d0 0x558661215d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f09bb640 1 -- 172.21.15.37:0/1047811605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 msgr2=0x5586612157f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f09bb640 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x5586612157f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f09bb640 1 -- 172.21.15.37:0/1047811605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5586610e11c0 con 0x5586612312f0 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f09bb640 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x558661216270 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f55e405e6d0 tx=0x7f55e4056d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d5a1d41d90703e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55caffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55caffd640 1 -- 172.21.15.37:0/1047811605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e405ec00 con 0x5586612312f0 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55caffd640 1 -- 172.21.15.37:0/1047811605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f55e405e770 con 0x5586612312f0 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 -- 172.21.15.37:0/1047811605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55e00ac040 con 0x5586612312f0 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55caffd640 1 -- 172.21.15.37:0/1047811605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55e405e9b0 con 0x5586612312f0 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 10 client.8478 fetch_fsmap learned FSMap version 73 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 -- 172.21.15.37:0/1047811605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x5586610dd170 con 0x5586612312f0 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55caffd640 1 -- 172.21.15.37:0/1047811605 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e00ac040 con 0x5586612312f0 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55caffd640 1 -- 172.21.15.37:0/1047811605 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f55e405a180 con 0x5586612312f0 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 10 client.8478 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 10 client.8478 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:01:16.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 20 client.8478 populate_metadata read hostname 'smithi037' 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.835+0000 7f55f558b000 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x558661240c80 0x558661243070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 4 client.8478 mds_command: new command op to 8340 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 -- 172.21.15.37:0/1047811605 --> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558660fcfc60 con 0x558661240c80 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55ebfff640 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x558661240c80 0x558661243070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55ebfff640 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x558661240c80 0x558661243070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2189937118632c5 server_cookie=1f6b4274f804bf0 in_seq=0 out_seq=0 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55caffd640 10 client.8478 ms_handle_connect on v2:172.21.15.37:6834/3366017865 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55caffd640 1 -- 172.21.15.37:0/1047811605 <== mds.0 v2:172.21.15.37:6834/3366017865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558660fcfc60 con 0x558661240c80 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55caffd640 10 client.8478 handle_command_reply: tid=0 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 client.8478 shutdown 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 -- 172.21.15.37:0/1047811605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 msgr2=0x558661216270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x558661216270 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f55e405e6d0 tx=0x7f55e4056d30 comp rx=0 tx=0).stop 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 -- 172.21.15.37:0/1047811605 shutdown_connections 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x558661240c80 0x558661243070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x558661216270 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612309d0 0x558661215d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 --2- 172.21.15.37:0/1047811605 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x5586612157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 -- 172.21.15.37:0/1047811605 >> 172.21.15.37:0/1047811605 conn(0x5586611e3020 msgr2=0x5586611e5ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55caffd640 0 client.8478 ms_handle_reset on v2:172.21.15.37:6834/3366017865 2024-04-25T09:01:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 -- 172.21.15.37:0/1047811605 shutdown_connections 2024-04-25T09:01:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 1 -- 172.21.15.37:0/1047811605 wait complete. 2024-04-25T09:01:16.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.839+0000 7f55f558b000 20 client.8478 trim_cache size 0 max 16384 2024-04-25T09:01:16.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 Processor -- start 2024-04-25T09:01:16.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 -- start start 2024-04-25T09:01:16.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x5586611e6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612309d0 0x55866123de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612312f0 0x55866123e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec200b20 con 0x5586612309d0 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec1ffca0 con 0x5586611e70a0 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec1ff0a0 con 0x5586612312f0 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612309d0 0x55866123de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612309d0 0x55866123de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33758/0 (socket says 172.21.15.37:33758) 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55eb7fe640 1 -- 172.21.15.37:0/3858144872 learned_addr learned my addr 172.21.15.37:0/3858144872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f09bb640 1 --2- 172.21.15.37:0/3858144872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612312f0 0x55866123e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55eb7fe640 1 -- 172.21.15.37:0/3858144872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612312f0 msgr2=0x55866123e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55ebfff640 1 --2- 172.21.15.37:0/3858144872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x5586611e6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/3858144872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612312f0 0x55866123e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55eb7fe640 1 -- 172.21.15.37:0/3858144872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 msgr2=0x5586611e6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/3858144872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x5586611e6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55eb7fe640 1 -- 172.21.15.37:0/3858144872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55866107ade0 con 0x5586612309d0 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55ebfff640 1 --2- 172.21.15.37:0/3858144872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x5586611e6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/3858144872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612309d0 0x55866123de40 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f55e001fb70 tx=0x7f55e00aa000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c6e6d8dd9fc05ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55d17fa640 1 -- 172.21.15.37:0/3858144872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e0001480 con 0x5586612309d0 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55d17fa640 1 -- 172.21.15.37:0/3858144872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e00ac070 con 0x5586612309d0 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55d17fa640 1 -- 172.21.15.37:0/3858144872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e00047f0 con 0x5586612309d0 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 -- 172.21.15.37:0/3858144872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612309d0 msgr2=0x55866123de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 --2- 172.21.15.37:0/3858144872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612309d0 0x55866123de40 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f55e001fb70 tx=0x7f55e00aa000 comp rx=0 tx=0).stop 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 -- 172.21.15.37:0/3858144872 shutdown_connections 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 --2- 172.21.15.37:0/3858144872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612312f0 0x55866123e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 --2- 172.21.15.37:0/3858144872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612309d0 0x55866123de40 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 --2- 172.21.15.37:0/3858144872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x5586611e6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 -- 172.21.15.37:0/3858144872 >> 172.21.15.37:0/3858144872 conn(0x5586611e3020 msgr2=0x55866123d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 -- 172.21.15.37:0/3858144872 shutdown_connections 2024-04-25T09:01:16.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 -- 172.21.15.37:0/3858144872 wait complete. 2024-04-25T09:01:16.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 -- start start 2024-04-25T09:01:16.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.855+0000 7f55f558b000 1 Processor -- start 2024-04-25T09:01:16.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x558661290510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612309d0 0x558661290a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586612426d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558660fcee00 con 0x5586612312f0 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ec1ff6a0 con 0x5586611e70a0 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558661033300 con 0x5586612309d0 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 10 client.0 _ll_register_callbacks cb 0x5586611b2040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f09bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586612426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f09bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586612426d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33770/0 (socket says 172.21.15.37:33770) 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f09bb640 1 -- 172.21.15.37:0/1655745738 learned_addr learned my addr 172.21.15.37:0/1655745738 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55ebfff640 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x558661290510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55eb7fe640 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612309d0 0x558661290a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f09bb640 1 -- 172.21.15.37:0/1655745738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612309d0 msgr2=0x558661290a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f09bb640 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612309d0 0x558661290a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f09bb640 1 -- 172.21.15.37:0/1655745738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 msgr2=0x558661290510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f09bb640 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x558661290510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f09bb640 1 -- 172.21.15.37:0/1655745738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558660fd58e0 con 0x5586612312f0 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55ebfff640 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x558661290510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f09bb640 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586612426d0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f55e40547a0 tx=0x7f55e4059070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd2dc4039a91eda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55caffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55caffd640 1 -- 172.21.15.37:0/1655745738 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e405ec00 con 0x5586612312f0 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55caffd640 1 -- 172.21.15.37:0/1655745738 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f55e4055940 con 0x5586612312f0 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 -- 172.21.15.37:0/1655745738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55e00ac070 con 0x5586612312f0 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55caffd640 1 -- 172.21.15.37:0/1655745738 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55e4055b40 con 0x5586612312f0 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 10 client.8484 fetch_fsmap learned FSMap version 73 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 -- 172.21.15.37:0/1655745738 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55866108d8f0 con 0x5586612312f0 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55caffd640 1 -- 172.21.15.37:0/1655745738 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e00ac070 con 0x5586612312f0 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55caffd640 1 -- 172.21.15.37:0/1655745738 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f55e4052240 con 0x5586612312f0 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 10 client.8484 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 10 client.8484 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 20 client.8484 populate_metadata read hostname 'smithi037' 2024-04-25T09:01:16.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5586612431b0 0x5586612435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 4 client.8484 mds_command: new command op to 8340 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 -- 172.21.15.37:0/1655745738 --> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558661041530 con 0x5586612431b0 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55ebfff640 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5586612431b0 0x5586612435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55ebfff640 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5586612431b0 0x5586612435c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad8c89951509c347 server_cookie=44e197f359f8927e in_seq=0 out_seq=0 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55caffd640 10 client.8484 ms_handle_connect on v2:172.21.15.37:6834/3366017865 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55caffd640 1 -- 172.21.15.37:0/1655745738 <== mds.0 v2:172.21.15.37:6834/3366017865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558661041530 con 0x5586612431b0 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55caffd640 10 client.8484 handle_command_reply: tid=0 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 client.8484 shutdown 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 -- 172.21.15.37:0/1655745738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 msgr2=0x5586612426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586612426d0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f55e40547a0 tx=0x7f55e4059070 comp rx=0 tx=0).stop 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 -- 172.21.15.37:0/1655745738 shutdown_connections 2024-04-25T09:01:16.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x5586612431b0 0x5586612435c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586612312f0 0x5586612426d0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586612309d0 0x558661290a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 --2- 172.21.15.37:0/1655745738 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586611e70a0 0x558661290510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55caffd640 0 client.8484 ms_handle_reset on v2:172.21.15.37:6834/3366017865 2024-04-25T09:01:16.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 -- 172.21.15.37:0/1655745738 >> 172.21.15.37:0/1655745738 conn(0x5586611e3020 msgr2=0x55866123dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 -- 172.21.15.37:0/1655745738 shutdown_connections 2024-04-25T09:01:16.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 1 -- 172.21.15.37:0/1655745738 wait complete. 2024-04-25T09:01:16.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f558b000 20 client.8484 trim_cache size 0 max 16384 2024-04-25T09:01:16.854 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:01:16.854 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-04-25T09:01:16.854 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-04-25T09:01:16.855 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:01:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 -- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55d8066f60 msgr2=0x7f55d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55d8066f60 0x7f55d8069420 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f55dc007fd0 tx=0x7f55dc009040 comp rx=0 tx=0).stop 2024-04-25T09:01:16.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 -- 172.21.15.37:0/3862413563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55ec15cd90 msgr2=0x7f55ec0c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55ec15cd90 0x7f55ec0c86c0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f55ec0b2db0 tx=0x7f55e400f520 comp rx=0 tx=0).stop 2024-04-25T09:01:16.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 -- 172.21.15.37:0/3862413563 shutdown_connections 2024-04-25T09:01:16.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f55d8066f60 0x7f55d8069420 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55ec15cd90 0x7f55ec0c86c0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f55ec14b510 0x7f55ec15c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 --2- 172.21.15.37:0/3862413563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f55d4004810 0x7f55ec14afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 -- 172.21.15.37:0/3862413563 >> 172.21.15.37:0/3862413563 conn(0x7f55ec099a70 msgr2=0x7f55ec0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 -- 172.21.15.37:0/3862413563 shutdown_connections 2024-04-25T09:01:16.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.859+0000 7f55f241e640 1 -- 172.21.15.37:0/3862413563 wait complete. 2024-04-25T09:01:16.865 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:01:16.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.991+0000 7fb39f5e1640 1 Processor -- start 2024-04-25T09:01:16.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.991+0000 7fb39f5e1640 1 -- start start 2024-04-25T09:01:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb39f5e1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb39815ecc0 0x7fb39806e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb39f5e1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3980b2090 con 0x7fb39815a700 2024-04-25T09:01:16.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb39f5e1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3980b43e0 con 0x7fb3980b3f50 2024-04-25T09:01:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb39f5e1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3980b46c0 con 0x7fb39815ecc0 2024-04-25T09:01:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb39db7e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb39815ecc0 0x7fb39806e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb39d37d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb39815a700 0x7fb3980cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40710/0 (socket says 172.21.15.37:40710) 2024-04-25T09:01:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb39d37d640 1 -- 172.21.15.37:0/198324471 learned_addr learned my addr 172.21.15.37:0/198324471 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb39db7e640 1 -- 172.21.15.37:0/198324471 >> v1:172.21.15.203:6789/0 conn(0x7fb3980b3f50 legacy=0x7fb3980b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb39db7e640 1 -- 172.21.15.37:0/198324471 >> v1:172.21.15.37:6789/0 conn(0x7fb39815a700 legacy=0x7fb3980cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb39db7e640 1 -- 172.21.15.37:0/198324471 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3980cd6f0 con 0x7fb39815ecc0 2024-04-25T09:01:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb39db7e640 1 --2- 172.21.15.37:0/198324471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb39815ecc0 0x7fb39806e1a0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb390009d90 tx=0x7fb39001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c1dce85218e6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb38ffff640 1 -- 172.21.15.37:0/198324471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3900b2070 con 0x7fb39815ecc0 2024-04-25T09:01:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb38ffff640 1 -- 172.21.15.37:0/198324471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3900a57b0 con 0x7fb39815ecc0 2024-04-25T09:01:16.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.995+0000 7fb38ffff640 1 -- 172.21.15.37:0/198324471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3900a5b80 con 0x7fb39815ecc0 2024-04-25T09:01:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 -- 172.21.15.37:0/198324471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb39815ecc0 msgr2=0x7fb39806e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 --2- 172.21.15.37:0/198324471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb39815ecc0 0x7fb39806e1a0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fb390009d90 tx=0x7fb39001c7e0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 -- 172.21.15.37:0/198324471 shutdown_connections 2024-04-25T09:01:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 --2- 172.21.15.37:0/198324471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb39815ecc0 0x7fb39806e1a0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 -- 172.21.15.37:0/198324471 >> 172.21.15.37:0/198324471 conn(0x7fb3980b11a0 msgr2=0x7fb3980b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 -- 172.21.15.37:0/198324471 shutdown_connections 2024-04-25T09:01:16.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 -- 172.21.15.37:0/198324471 wait complete. 2024-04-25T09:01:16.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 Processor -- start 2024-04-25T09:01:16.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 -- start start 2024-04-25T09:01:16.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb39815a700 0x7fb39814b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb39815ecc0 0x7fb39815afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb398135d90 0x7fb39814d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3980cd310 con 0x7fb39815a700 2024-04-25T09:01:16.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3980ce600 con 0x7fb39815ecc0 2024-04-25T09:01:16.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3980cd490 con 0x7fb398135d90 2024-04-25T09:01:16.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39d37d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb39815a700 0x7fb39814b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39d37d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb39815a700 0x7fb39814b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33798/0 (socket says 172.21.15.37:33798) 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39d37d640 1 -- 172.21.15.37:0/2690928656 learned_addr learned my addr 172.21.15.37:0/2690928656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39db7e640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb398135d90 0x7fb39814d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39cb7c640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb39815ecc0 0x7fb39815afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39d37d640 1 -- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb398135d90 msgr2=0x7fb39814d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39d37d640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb398135d90 0x7fb39814d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39d37d640 1 -- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb39815ecc0 msgr2=0x7fb39815afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39d37d640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb39815ecc0 0x7fb39815afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39d37d640 1 -- 172.21.15.37:0/2690928656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb39814be10 con 0x7fb39815a700 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39d37d640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb39815a700 0x7fb39814b730 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fb39400c370 tx=0x7fb39400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb38dffb640 1 -- 172.21.15.37:0/2690928656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb394017070 con 0x7fb39815a700 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb38dffb640 1 -- 172.21.15.37:0/2690928656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb394005cf0 con 0x7fb39815a700 2024-04-25T09:01:16.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb38dffb640 1 -- 172.21.15.37:0/2690928656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3940072f0 con 0x7fb39815a700 2024-04-25T09:01:16.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 -- 172.21.15.37:0/2690928656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39814e320 con 0x7fb39815a700 2024-04-25T09:01:16.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:16.999+0000 7fb39f5e1640 1 -- 172.21.15.37:0/2690928656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb398136220 con 0x7fb39815a700 2024-04-25T09:01:16.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.003+0000 7fb39f5e1640 1 -- 172.21.15.37:0/2690928656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39804efa0 con 0x7fb39815a700 2024-04-25T09:01:16.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.003+0000 7fb38dffb640 1 -- 172.21.15.37:0/2690928656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb39401a070 con 0x7fb39815a700 2024-04-25T09:01:16.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.003+0000 7fb38dffb640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb37c066f60 0x7fb37c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.003+0000 7fb38dffb640 1 -- 172.21.15.37:0/2690928656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fb394084630 con 0x7fb39815a700 2024-04-25T09:01:16.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.003+0000 7fb39cb7c640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb37c066f60 0x7fb37c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.003+0000 7fb39cb7c640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb37c066f60 0x7fb37c069420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb39814b6e0 tx=0x7fb388008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.007+0000 7fb38dffb640 1 -- 172.21.15.37:0/2690928656 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb394055390 con 0x7fb39815a700 2024-04-25T09:01:17.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.199+0000 7fb39f5e1640 1 -- 172.21.15.37:0/2690928656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3981663f0 con 0x7fb39815a700 2024-04-25T09:01:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.199+0000 7fb38dffb640 1 -- 172.21.15.37:0/2690928656 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fb394054c90 con 0x7fb39815a700 2024-04-25T09:01:17.189 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:17.189 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T09:00:41.164474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/3366017865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3366017865},{"type":"v1","addr":"172.21.15.37:6835","nonce":3366017865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:17.189 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 73 2024-04-25T09:01:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 -- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb37c066f60 msgr2=0x7fb37c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb37c066f60 0x7fb37c069420 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb39814b6e0 tx=0x7fb388008040 comp rx=0 tx=0).stop 2024-04-25T09:01:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 -- 172.21.15.37:0/2690928656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb39815a700 msgr2=0x7fb39814b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb39815a700 0x7fb39814b730 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fb39400c370 tx=0x7fb39400c830 comp rx=0 tx=0).stop 2024-04-25T09:01:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 -- 172.21.15.37:0/2690928656 shutdown_connections 2024-04-25T09:01:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb37c066f60 0x7fb37c069420 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb398135d90 0x7fb39814d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb39815ecc0 0x7fb39815afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 --2- 172.21.15.37:0/2690928656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb39815a700 0x7fb39814b730 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 -- 172.21.15.37:0/2690928656 >> 172.21.15.37:0/2690928656 conn(0x7fb3980b11a0 msgr2=0x7fb3980afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 -- 172.21.15.37:0/2690928656 shutdown_connections 2024-04-25T09:01:17.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.203+0000 7fb39f5e1640 1 -- 172.21.15.37:0/2690928656 wait complete. 2024-04-25T09:01:17.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-04-25T09:01:17.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.331+0000 7f7de9131640 1 Processor -- start 2024-04-25T09:01:17.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.331+0000 7f7de9131640 1 -- start start 2024-04-25T09:01:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.331+0000 7f7de9131640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7de4159bd0 0x7f7de40c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.331+0000 7f7de9131640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7de40b28c0 0x7f7de4163960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.331+0000 7f7de9131640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7de40c9ef0 con 0x7f7de4159fb0 2024-04-25T09:01:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.331+0000 7f7de9131640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7de4072780 con 0x7f7de4159bd0 2024-04-25T09:01:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.331+0000 7f7de9131640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de4071b70 con 0x7f7de40b28c0 2024-04-25T09:01:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de2d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7de4159bd0 0x7f7de40c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de2d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7de4159bd0 0x7f7de40c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33804/0 (socket says 172.21.15.37:33804) 2024-04-25T09:01:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de2d9d640 1 -- 172.21.15.37:0/2821352099 learned_addr learned my addr 172.21.15.37:0/2821352099 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de359e640 1 --2- 172.21.15.37:0/2821352099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7de40b28c0 0x7f7de4163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de2d9d640 1 -- 172.21.15.37:0/2821352099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7de40b28c0 msgr2=0x7f7de4163960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/2821352099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7de40b28c0 0x7f7de4163960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de2d9d640 1 -- 172.21.15.37:0/2821352099 >> v1:172.21.15.203:6789/0 conn(0x7f7de4159fb0 legacy=0x7f7de40b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de2d9d640 1 -- 172.21.15.37:0/2821352099 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de4071e50 con 0x7f7de4159bd0 2024-04-25T09:01:17.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/2821352099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7de4159bd0 0x7f7de40c99b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009e50 tx=0x7f7ddc01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a60895986449c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de1d9b640 1 -- 172.21.15.37:0/2821352099 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0a5840 con 0x7f7de4159bd0 2024-04-25T09:01:17.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de1d9b640 1 -- 172.21.15.37:0/2821352099 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc0a59e0 con 0x7f7de4159bd0 2024-04-25T09:01:17.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de1d9b640 1 -- 172.21.15.37:0/2821352099 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0a5db0 con 0x7f7de4159bd0 2024-04-25T09:01:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de9131640 1 -- 172.21.15.37:0/2821352099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7de4159bd0 msgr2=0x7f7de40c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de9131640 1 --2- 172.21.15.37:0/2821352099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7de4159bd0 0x7f7de40c99b0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009e50 tx=0x7f7ddc01c9c0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de9131640 1 -- 172.21.15.37:0/2821352099 shutdown_connections 2024-04-25T09:01:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de9131640 1 --2- 172.21.15.37:0/2821352099 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7de40b28c0 0x7f7de4163960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de9131640 1 --2- 172.21.15.37:0/2821352099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7de4159bd0 0x7f7de40c99b0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de9131640 1 -- 172.21.15.37:0/2821352099 >> 172.21.15.37:0/2821352099 conn(0x7f7de406ae60 msgr2=0x7f7de406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de9131640 1 -- 172.21.15.37:0/2821352099 shutdown_connections 2024-04-25T09:01:17.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.335+0000 7f7de9131640 1 -- 172.21.15.37:0/2821352099 wait complete. 2024-04-25T09:01:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de9131640 1 Processor -- start 2024-04-25T09:01:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de9131640 1 -- start start 2024-04-25T09:01:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de9131640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de40b28c0 0x7f7de415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de9131640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7de4159bd0 0x7f7de40b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de9131640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7de4159fb0 0x7f7de40977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de9131640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de4166840 con 0x7f7de4159bd0 2024-04-25T09:01:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de9131640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de41666c0 con 0x7f7de40b28c0 2024-04-25T09:01:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de9131640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de41669c0 con 0x7f7de4159fb0 2024-04-25T09:01:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7de4159bd0 0x7f7de40b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de2d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de40b28c0 0x7f7de415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7de4159fb0 0x7f7de40977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de2d9d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de40b28c0 0x7f7de415a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:40468/0 (socket says 172.21.15.37:40468) 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de2d9d640 1 -- 172.21.15.37:0/551680711 learned_addr learned my addr 172.21.15.37:0/551680711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de259c640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7de4159bd0 0x7f7de40b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33808/0 (socket says 172.21.15.37:33808) 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de259c640 1 -- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7de4159fb0 msgr2=0x7f7de40977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de259c640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7de4159fb0 0x7f7de40977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de259c640 1 -- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de40b28c0 msgr2=0x7f7de415a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de259c640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de40b28c0 0x7f7de415a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de259c640 1 -- 172.21.15.37:0/551680711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de40c5ea0 con 0x7f7de4159bd0 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de259c640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7de4159bd0 0x7f7de40b10d0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f7de415a3f0 tx=0x7f7dcc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7ddb7fe640 1 -- 172.21.15.37:0/551680711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc011790 con 0x7f7de4159bd0 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7ddb7fe640 1 -- 172.21.15.37:0/551680711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc017070 con 0x7f7de4159bd0 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7ddb7fe640 1 -- 172.21.15.37:0/551680711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc011bb0 con 0x7f7de4159bd0 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de9131640 1 -- 172.21.15.37:0/551680711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de40989e0 con 0x7f7de4159bd0 2024-04-25T09:01:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.339+0000 7f7de9131640 1 -- 172.21.15.37:0/551680711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de40c5bb0 con 0x7f7de4159bd0 2024-04-25T09:01:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7ddb7fe640 1 -- 172.21.15.37:0/551680711 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7dcc01c460 con 0x7f7de4159bd0 2024-04-25T09:01:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7ddb7fe640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc4066f60 0x7f7dc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7ddb7fe640 1 -- 172.21.15.37:0/551680711 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f7dcc084f40 con 0x7f7de4159bd0 2024-04-25T09:01:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc4066f60 0x7f7dc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7dec29e000 1 Processor -- start 2024-04-25T09:01:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7dec29e000 1 -- start start 2024-04-25T09:01:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc4066f60 0x7f7dc4069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009e50 tx=0x7f7ddc0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6536b96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6536b9c00 0x55a65366f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a65366f8e0 0x55a65366fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7dec29e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a653556ba0 con 0x55a6536b92e0 2024-04-25T09:01:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7dec29e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6534bc090 con 0x55a6536b9c00 2024-04-25T09:01:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7dec29e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a653444160 con 0x55a65366f8e0 2024-04-25T09:01:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6536b96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6536b96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33824/0 (socket says 172.21.15.37:33824) 2024-04-25T09:01:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de2d9d640 1 -- 172.21.15.37:0/1686382755 learned_addr learned my addr 172.21.15.37:0/1686382755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:17.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de259c640 1 --2- 172.21.15.37:0/1686382755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6536b9c00 0x55a65366f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de359e640 1 --2- 172.21.15.37:0/1686382755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a65366f8e0 0x55a65366fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de2d9d640 1 -- 172.21.15.37:0/1686382755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a65366f8e0 msgr2=0x55a65366fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/1686382755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a65366f8e0 0x55a65366fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de2d9d640 1 -- 172.21.15.37:0/1686382755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6536b9c00 msgr2=0x55a65366f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/1686382755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6536b9c00 0x55a65366f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de2d9d640 1 -- 172.21.15.37:0/1686382755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a653565c20 con 0x55a6536b92e0 2024-04-25T09:01:17.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/1686382755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6536b96c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc098560 tx=0x7f7ddc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d750fdaaaf3b41c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7dd8ff9640 1 -- 172.21.15.37:0/1686382755 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0c6070 con 0x55a6536b92e0 2024-04-25T09:01:17.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7dd8ff9640 1 -- 172.21.15.37:0/1686382755 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc0c26e0 con 0x55a6536b92e0 2024-04-25T09:01:17.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.343+0000 7f7dd8ff9640 1 -- 172.21.15.37:0/1686382755 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0c2a90 con 0x55a6536b92e0 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- 172.21.15.37:0/1686382755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 msgr2=0x55a6536b96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 --2- 172.21.15.37:0/1686382755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6536b96c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc098560 tx=0x7f7ddc004ac0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- 172.21.15.37:0/1686382755 shutdown_connections 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 --2- 172.21.15.37:0/1686382755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a65366f8e0 0x55a65366fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 --2- 172.21.15.37:0/1686382755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6536b9c00 0x55a65366f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 --2- 172.21.15.37:0/1686382755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6536b96c0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- 172.21.15.37:0/1686382755 >> 172.21.15.37:0/1686382755 conn(0x55a65366b890 msgr2=0x55a65366dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- 172.21.15.37:0/1686382755 shutdown_connections 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- 172.21.15.37:0/1686382755 wait complete. 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- start start 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 Processor -- start 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a6536c0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a6536b92e0 0x55a6536c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6536b9c00 0x55a6536a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6534b8130 con 0x55a65366f8e0 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a653480130 con 0x55a6536b9c00 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a65357a300 con 0x55a6536b92e0 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 10 client.0 _ll_register_callbacks cb 0x55a65365abc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a6536c0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a6536c0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33828/0 (socket says 172.21.15.37:33828) 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de2d9d640 1 -- 172.21.15.37:0/863913027 learned_addr learned my addr 172.21.15.37:0/863913027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de2d9d640 1 -- 172.21.15.37:0/863913027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a6536b92e0 msgr2=0x55a6536c0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a6536b92e0 0x55a6536c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de2d9d640 1 -- 172.21.15.37:0/863913027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6536b9c00 msgr2=0x55a6536a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de359e640 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6536b9c00 0x55a6536a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6536b9c00 0x55a6536a7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de2d9d640 1 -- 172.21.15.37:0/863913027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a653499060 con 0x55a65366f8e0 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a6536c0640 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc01cea0 tx=0x7f7ddc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec372b7d03374d91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/863913027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ddc0ba070 con 0x55a65366f8e0 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/863913027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f7ddc0d5d60 con 0x55a65366f8e0 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- 172.21.15.37:0/863913027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ddc0c26e0 con 0x55a65366f8e0 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/863913027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ddc0c6050 con 0x55a65366f8e0 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 10 client.8505 fetch_fsmap learned FSMap version 73 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- 172.21.15.37:0/863913027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55a65345d7c0 con 0x55a65366f8e0 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/863913027 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ddc0c26e0 con 0x55a65366f8e0 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/863913027 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7ddc008030 con 0x55a65366f8e0 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 10 client.8505 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 10 client.8505 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 20 client.8505 populate_metadata read hostname 'smithi037' 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x55a6536a7e30 0x55a6536a8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 4 client.8505 mds_command: new command op to 8340 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 -- 172.21.15.37:0/863913027 --> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a653457cd0 con 0x55a6536a7e30 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de259c640 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x55a6536a7e30 0x55a6536a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dc1ffb640 10 client.8505 ms_handle_connect on v2:172.21.15.37:6834/3366017865 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7de259c640 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x55a6536a7e30 0x55a6536a8210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99c15f1c65c9da8 server_cookie=6981742d3de05079 in_seq=0 out_seq=0 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/863913027 <== mds.0 v2:172.21.15.37:6834/3366017865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a653457cd0 con 0x55a6536a7e30 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dc1ffb640 10 client.8505 handle_command_reply: tid=0 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.347+0000 7f7dec29e000 1 client.8505 shutdown 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dec29e000 1 -- 172.21.15.37:0/863913027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 msgr2=0x55a6536c0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dec29e000 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a6536c0640 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc01cea0 tx=0x7f7ddc004a50 comp rx=0 tx=0).stop 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dec29e000 1 -- 172.21.15.37:0/863913027 shutdown_connections 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dec29e000 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a6536b9c00 0x55a6536a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dec29e000 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a6536b92e0 0x55a6536c0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dec29e000 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x55a6536a7e30 0x55a6536a8210 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dec29e000 1 --2- 172.21.15.37:0/863913027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a6536c0640 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dec29e000 1 -- 172.21.15.37:0/863913027 >> 172.21.15.37:0/863913027 conn(0x55a65366b890 msgr2=0x55a65366e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dc1ffb640 0 client.8505 ms_handle_reset on v2:172.21.15.37:6834/3366017865 2024-04-25T09:01:17.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dec29e000 1 -- 172.21.15.37:0/863913027 shutdown_connections 2024-04-25T09:01:17.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dec29e000 1 -- 172.21.15.37:0/863913027 wait complete. 2024-04-25T09:01:17.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.351+0000 7f7dec29e000 20 client.8505 trim_cache size 0 max 16384 2024-04-25T09:01:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.359+0000 7f7dec29e000 1 Processor -- start 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.359+0000 7f7dec29e000 1 -- start start 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a65366b890 0x55a653715920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a65366f8e0 0x55a653715e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6537163a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de4200640 con 0x55a6536b92e0 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de415ce00 con 0x55a65366f8e0 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7de4200210 con 0x55a65366b890 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6537163a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6537163a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33832/0 (socket says 172.21.15.37:33832) 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de359e640 1 -- 172.21.15.37:0/1199945625 learned_addr learned my addr 172.21.15.37:0/1199945625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de359e640 1 -- 172.21.15.37:0/1199945625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a65366b890 msgr2=0x55a653715920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de359e640 1 --2- 172.21.15.37:0/1199945625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a65366b890 0x55a653715920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de359e640 1 -- 172.21.15.37:0/1199945625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a65366f8e0 msgr2=0x55a653715e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de359e640 1 --2- 172.21.15.37:0/1199945625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a65366f8e0 0x55a653715e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de359e640 1 -- 172.21.15.37:0/1199945625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a653477fc0 con 0x55a6536b92e0 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de359e640 1 --2- 172.21.15.37:0/1199945625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6537163a0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4004660 tx=0x7f7dd400ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8dd07088a93687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dc3fff640 1 -- 172.21.15.37:0/1199945625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd401a070 con 0x55a6536b92e0 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dc3fff640 1 -- 172.21.15.37:0/1199945625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd4012be0 con 0x55a6536b92e0 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dc3fff640 1 -- 172.21.15.37:0/1199945625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4015450 con 0x55a6536b92e0 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- 172.21.15.37:0/1199945625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 msgr2=0x55a6537163a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 --2- 172.21.15.37:0/1199945625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6537163a0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4004660 tx=0x7f7dd400ad10 comp rx=0 tx=0).stop 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- 172.21.15.37:0/1199945625 shutdown_connections 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 --2- 172.21.15.37:0/1199945625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a6536b92e0 0x55a6537163a0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 --2- 172.21.15.37:0/1199945625 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a65366f8e0 0x55a653715e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 --2- 172.21.15.37:0/1199945625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a65366b890 0x55a653715920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- 172.21.15.37:0/1199945625 >> 172.21.15.37:0/1199945625 conn(0x55a653674bc0 msgr2=0x55a6536b8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- 172.21.15.37:0/1199945625 shutdown_connections 2024-04-25T09:01:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- 172.21.15.37:0/1199945625 wait complete. 2024-04-25T09:01:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- start start 2024-04-25T09:01:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 Processor -- start 2024-04-25T09:01:17.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a65366b890 0x55a653719b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a65371a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a6536b92e0 0x55a6536bd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a653456e70 con 0x55a65366f8e0 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a653444a50 con 0x55a65366b890 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6534bb3a0 con 0x55a6536b92e0 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 10 client.0 _ll_register_callbacks cb 0x55a653468240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a65371a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a65371a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33838/0 (socket says 172.21.15.37:33838) 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de259c640 1 -- 172.21.15.37:0/2707044200 learned_addr learned my addr 172.21.15.37:0/2707044200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de259c640 1 -- 172.21.15.37:0/2707044200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a6536b92e0 msgr2=0x55a6536bd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de259c640 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a6536b92e0 0x55a6536bd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a65366b890 0x55a653719b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de259c640 1 -- 172.21.15.37:0/2707044200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a65366b890 msgr2=0x55a653719b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de259c640 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a65366b890 0x55a653719b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de259c640 1 -- 172.21.15.37:0/2707044200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6535159e0 con 0x55a65366f8e0 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a65366b890 0x55a653719b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7de259c640 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a65371a080 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00cd70 tx=0x7f7dcc051a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64154b06538d70a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dc1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/2707044200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc054a90 con 0x55a65366f8e0 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/2707044200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f7dcc054c30 con 0x55a65366f8e0 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.363+0000 7f7dec29e000 1 -- 172.21.15.37:0/2707044200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dd4012be0 con 0x55a65366f8e0 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/2707044200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dcc051390 con 0x55a65366f8e0 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 10 client.8511 fetch_fsmap learned FSMap version 73 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 -- 172.21.15.37:0/2707044200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55a65345b630 con 0x55a65366f8e0 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/2707044200 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd4012be0 con 0x55a65366f8e0 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/2707044200 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 73) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7dcc051530 con 0x55a65366f8e0 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 10 client.8511 fetch_fsmap finished waiting for FSMap version 73 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 10 client.8511 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:01:17.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 20 client.8511 populate_metadata read hostname 'smithi037' 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x55a6536b9c00 0x55a653722bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 4 client.8511 mds_command: new command op to 8340 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 -- 172.21.15.37:0/2707044200 --> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x55a6534c95d0 con 0x55a6536b9c00 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x55a6536b9c00 0x55a653722bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dc1ffb640 10 client.8511 ms_handle_connect on v2:172.21.15.37:6834/3366017865 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de2d9d640 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x55a6536b9c00 0x55a653722bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5661c0c5022be server_cookie=167862f4196fcf3b in_seq=0 out_seq=0 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dc1ffb640 1 -- 172.21.15.37:0/2707044200 <== mds.0 v2:172.21.15.37:6834/3366017865 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a6534c95d0 con 0x55a6536b9c00 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dc1ffb640 10 client.8511 handle_command_reply: tid=0 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 client.8511 shutdown 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 -- 172.21.15.37:0/2707044200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 msgr2=0x55a65371a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a65371a080 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc00cd70 tx=0x7f7dcc051a00 comp rx=0 tx=0).stop 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 -- 172.21.15.37:0/2707044200 shutdown_connections 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a6536b92e0 0x55a6536bd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.37:6834/3366017865,v1:172.21.15.37:6835/3366017865] conn(0x55a6536b9c00 0x55a653722bc0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a65366f8e0 0x55a65371a080 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dc1ffb640 0 client.8511 ms_handle_reset on v2:172.21.15.37:6834/3366017865 2024-04-25T09:01:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 --2- 172.21.15.37:0/2707044200 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a65366b890 0x55a653719b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 -- 172.21.15.37:0/2707044200 >> 172.21.15.37:0/2707044200 conn(0x55a653674bc0 msgr2=0x55a6536b8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 -- 172.21.15.37:0/2707044200 shutdown_connections 2024-04-25T09:01:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 1 -- 172.21.15.37:0/2707044200 wait complete. 2024-04-25T09:01:17.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7dec29e000 20 client.8511 trim_cache size 0 max 16384 2024-04-25T09:01:17.361 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:17.361 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-04-25T09:01:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 -- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc4066f60 msgr2=0x7f7dc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc4066f60 0x7f7dc4069420 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc009e50 tx=0x7f7ddc0a7070 comp rx=0 tx=0).stop 2024-04-25T09:01:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 -- 172.21.15.37:0/551680711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7de4159bd0 msgr2=0x7f7de40b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7de4159bd0 0x7f7de40b10d0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f7de415a3f0 tx=0x7f7dcc00c870 comp rx=0 tx=0).stop 2024-04-25T09:01:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 -- 172.21.15.37:0/551680711 shutdown_connections 2024-04-25T09:01:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7dc4066f60 0x7f7dc4069420 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7de4159fb0 0x7f7de40977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7de4159bd0 0x7f7de40b10d0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 --2- 172.21.15.37:0/551680711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7de40b28c0 0x7f7de415a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 -- 172.21.15.37:0/551680711 >> 172.21.15.37:0/551680711 conn(0x7f7de406ae60 msgr2=0x7f7de4070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 -- 172.21.15.37:0/551680711 shutdown_connections 2024-04-25T09:01:17.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.367+0000 7f7de9131640 1 -- 172.21.15.37:0/551680711 wait complete. 2024-04-25T09:01:17.370 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T09:01:17.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.491+0000 7fb4466ed640 1 Processor -- start 2024-04-25T09:01:17.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.491+0000 7fb4466ed640 1 -- start start 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb4466ed640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4400b2720 0x7fb4400b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb4466ed640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb4400cf560 0x7fb4400cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb4466ed640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4400b2ca0 con 0x7fb44015a600 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb4466ed640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4400cfef0 con 0x7fb4400cf560 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb4466ed640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400b37d0 con 0x7fb4400b2720 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb444c8a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb4400cf560 0x7fb4400cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb444c8a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb4400cf560 0x7fb4400cf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33854/0 (socket says 172.21.15.37:33854) 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb444c8a640 1 -- 172.21.15.37:0/3137434287 learned_addr learned my addr 172.21.15.37:0/3137434287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb43ffff640 1 --1- 172.21.15.37:0/3137434287 >> v1:172.21.15.37:6789/0 conn(0x7fb44015a600 0x7fb44006e930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40726/0 (socket says 172.21.15.37:40726) 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb437fff640 1 --2- 172.21.15.37:0/3137434287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4400b2720 0x7fb4400b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb444c8a640 1 -- 172.21.15.37:0/3137434287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4400b2720 msgr2=0x7fb4400b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb444c8a640 1 --2- 172.21.15.37:0/3137434287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4400b2720 0x7fb4400b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb444c8a640 1 -- 172.21.15.37:0/3137434287 >> v1:172.21.15.37:6789/0 conn(0x7fb44015a600 legacy=0x7fb44006e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb444c8a640 1 -- 172.21.15.37:0/3137434287 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4400b2f40 con 0x7fb4400cf560 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb43f7fe640 1 -- 172.21.15.37:0/3137434287 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469294192 0 0) 0x7fb4400b2ca0 con 0x7fb44015a600 2024-04-25T09:01:17.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb444c8a640 1 --2- 172.21.15.37:0/3137434287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4400cf560 0x7fb4400cf9b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fb438009e50 tx=0x7fb43801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1cece429957607b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb43f7fe640 1 -- 172.21.15.37:0/3137434287 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4380a5810 con 0x7fb4400cf560 2024-04-25T09:01:17.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb43f7fe640 1 -- 172.21.15.37:0/3137434287 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4380a59b0 con 0x7fb4400cf560 2024-04-25T09:01:17.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.495+0000 7fb43f7fe640 1 -- 172.21.15.37:0/3137434287 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4380a5d80 con 0x7fb4400cf560 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 -- 172.21.15.37:0/3137434287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4400cf560 msgr2=0x7fb4400cf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 --2- 172.21.15.37:0/3137434287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4400cf560 0x7fb4400cf9b0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fb438009e50 tx=0x7fb43801ca30 comp rx=0 tx=0).stop 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 -- 172.21.15.37:0/3137434287 shutdown_connections 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 --2- 172.21.15.37:0/3137434287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4400cf560 0x7fb4400cf9b0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 --2- 172.21.15.37:0/3137434287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4400b2720 0x7fb4400b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 -- 172.21.15.37:0/3137434287 >> 172.21.15.37:0/3137434287 conn(0x7fb44006ae60 msgr2=0x7fb44006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 -- 172.21.15.37:0/3137434287 shutdown_connections 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 -- 172.21.15.37:0/3137434287 wait complete. 2024-04-25T09:01:17.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 Processor -- start 2024-04-25T09:01:17.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 -- start start 2024-04-25T09:01:17.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4400b2720 0x7fb4400c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4400cf560 0x7fb4400988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44015a600 0x7fb4400ccec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400acb70 con 0x7fb4400b2720 2024-04-25T09:01:17.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400ac9f0 con 0x7fb44015a600 2024-04-25T09:01:17.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4400accf0 con 0x7fb4400cf560 2024-04-25T09:01:17.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4400b2720 0x7fb4400c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4400b2720 0x7fb4400c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33858/0 (socket says 172.21.15.37:33858) 2024-04-25T09:01:17.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43ffff640 1 -- 172.21.15.37:0/11042570 learned_addr learned my addr 172.21.15.37:0/11042570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb437fff640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4400cf560 0x7fb4400988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb444c8a640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44015a600 0x7fb4400ccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43ffff640 1 -- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4400cf560 msgr2=0x7fb4400988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43ffff640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4400cf560 0x7fb4400988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43ffff640 1 -- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44015a600 msgr2=0x7fb4400ccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43ffff640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44015a600 0x7fb4400ccec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43ffff640 1 -- 172.21.15.37:0/11042570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4400c5e40 con 0x7fb4400b2720 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb437fff640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4400cf560 0x7fb4400988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43ffff640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4400b2720 0x7fb4400c5900 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c480 tx=0x7fb43000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43d7fa640 1 -- 172.21.15.37:0/11042570 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430018070 con 0x7fb4400b2720 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43d7fa640 1 -- 172.21.15.37:0/11042570 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb430007500 con 0x7fb4400b2720 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb43d7fa640 1 -- 172.21.15.37:0/11042570 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430004a30 con 0x7fb4400b2720 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.499+0000 7fb4466ed640 1 -- 172.21.15.37:0/11042570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44015c6a0 con 0x7fb4400b2720 2024-04-25T09:01:17.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.503+0000 7fb4466ed640 1 -- 172.21.15.37:0/11042570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44015cc60 con 0x7fb4400b2720 2024-04-25T09:01:17.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.503+0000 7fb4466ed640 1 -- 172.21.15.37:0/11042570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44004efa0 con 0x7fb4400b2720 2024-04-25T09:01:17.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.503+0000 7fb43d7fa640 1 -- 172.21.15.37:0/11042570 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb430004bd0 con 0x7fb4400b2720 2024-04-25T09:01:17.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.503+0000 7fb43d7fa640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb420066f60 0x7fb420069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.503+0000 7fb43d7fa640 1 -- 172.21.15.37:0/11042570 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fb430084580 con 0x7fb4400b2720 2024-04-25T09:01:17.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.503+0000 7fb437fff640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb420066f60 0x7fb420069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.503+0000 7fb437fff640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb420066f60 0x7fb420069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb428009840 tx=0x7fb428008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.507+0000 7fb43d7fa640 1 -- 172.21.15.37:0/11042570 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4300552d0 con 0x7fb4400b2720 2024-04-25T09:01:17.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.695+0000 7fb4466ed640 1 -- 172.21.15.37:0/11042570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb4400ad840 con 0x7fb4400b2720 2024-04-25T09:01:17.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.695+0000 7fb43d7fa640 1 -- 172.21.15.37:0/11042570 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fb430054bd0 con 0x7fb4400b2720 2024-04-25T09:01:17.685 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:17.685 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T09:01:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 -- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb420066f60 msgr2=0x7fb420069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb420066f60 0x7fb420069420 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb428009840 tx=0x7fb428008040 comp rx=0 tx=0).stop 2024-04-25T09:01:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 -- 172.21.15.37:0/11042570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4400b2720 msgr2=0x7fb4400c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4400b2720 0x7fb4400c5900 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fb43000c480 tx=0x7fb43000c940 comp rx=0 tx=0).stop 2024-04-25T09:01:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 -- 172.21.15.37:0/11042570 shutdown_connections 2024-04-25T09:01:17.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb420066f60 0x7fb420069420 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb44015a600 0x7fb4400ccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4400cf560 0x7fb4400988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 --2- 172.21.15.37:0/11042570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4400b2720 0x7fb4400c5900 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 -- 172.21.15.37:0/11042570 >> 172.21.15.37:0/11042570 conn(0x7fb44006ae60 msgr2=0x7fb4400af790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 -- 172.21.15.37:0/11042570 shutdown_connections 2024-04-25T09:01:17.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.699+0000 7fb4466ed640 1 -- 172.21.15.37:0/11042570 wait complete. 2024-04-25T09:01:17.702 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T09:01:17.703 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:01:17.703 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:01:17.751 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:01:17.804 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:01:17.804 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:01:17.804 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:17.810 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:17.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62f1009640 1 Processor -- start 2024-04-25T09:01:17.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62f1009640 1 -- start start 2024-04-25T09:01:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62f1009640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f62ec0ce510 0x7f62ec0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62f1009640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f62ec1451f0 0x7f62ec142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62f1009640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ec0cee30 con 0x7f62ec144ca0 2024-04-25T09:01:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62f1009640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec0cc2b0 con 0x7f62ec0ce510 2024-04-25T09:01:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62f1009640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec06e290 con 0x7f62ec1451f0 2024-04-25T09:01:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62ead9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f62ec0ce510 0x7f62ec0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62ead9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f62ec0ce510 0x7f62ec0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33868/0 (socket says 172.21.15.37:33868) 2024-04-25T09:01:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62ead9d640 1 -- 172.21.15.37:0/2326309212 learned_addr learned my addr 172.21.15.37:0/2326309212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62eb59e640 1 --2- 172.21.15.37:0/2326309212 >> v2:172.21.15.203:3300/0 conn(0x7f62ec1451f0 0x7f62ec142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62ead9d640 1 -- 172.21.15.37:0/2326309212 >> v2:172.21.15.203:3300/0 conn(0x7f62ec1451f0 msgr2=0x7f62ec142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62ead9d640 1 --2- 172.21.15.37:0/2326309212 >> v2:172.21.15.203:3300/0 conn(0x7f62ec1451f0 0x7f62ec142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62ead9d640 1 -- 172.21.15.37:0/2326309212 >> v1:172.21.15.203:6789/0 conn(0x7f62ec144ca0 legacy=0x7f62ec06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.991+0000 7f62ead9d640 1 -- 172.21.15.37:0/2326309212 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec0ccc70 con 0x7f62ec0ce510 2024-04-25T09:01:17.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62ead9d640 1 --2- 172.21.15.37:0/2326309212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec0ce510 0x7f62ec0ce8f0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0b48f0 tx=0x7f62dc01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61405d300c803a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62e9d9b640 1 -- 172.21.15.37:0/2326309212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc0b5070 con 0x7f62ec0ce510 2024-04-25T09:01:17.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62e9d9b640 1 -- 172.21.15.37:0/2326309212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62dc0a89d0 con 0x7f62ec0ce510 2024-04-25T09:01:17.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62e9d9b640 1 -- 172.21.15.37:0/2326309212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc0a8da0 con 0x7f62ec0ce510 2024-04-25T09:01:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 -- 172.21.15.37:0/2326309212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec0ce510 msgr2=0x7f62ec0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 --2- 172.21.15.37:0/2326309212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec0ce510 0x7f62ec0ce8f0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0b48f0 tx=0x7f62dc01ecd0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 -- 172.21.15.37:0/2326309212 shutdown_connections 2024-04-25T09:01:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 --2- 172.21.15.37:0/2326309212 >> v2:172.21.15.203:3300/0 conn(0x7f62ec1451f0 0x7f62ec142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 --2- 172.21.15.37:0/2326309212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec0ce510 0x7f62ec0ce8f0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 -- 172.21.15.37:0/2326309212 >> 172.21.15.37:0/2326309212 conn(0x7f62ec0b11a0 msgr2=0x7f62ec0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 -- 172.21.15.37:0/2326309212 shutdown_connections 2024-04-25T09:01:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 -- 172.21.15.37:0/2326309212 wait complete. 2024-04-25T09:01:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 Processor -- start 2024-04-25T09:01:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 -- start start 2024-04-25T09:01:17.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62ec0ce510 0x7f62ec0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec144ca0 0x7f62ec13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec1451f0 0x7f62ec13ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec06ee30 con 0x7f62ec144ca0 2024-04-25T09:01:17.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec06df20 con 0x7f62ec1451f0 2024-04-25T09:01:17.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62f1009640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0cc900 con 0x7f62ec0ce510 2024-04-25T09:01:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62ea59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec144ca0 0x7f62ec13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62ea59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec144ca0 0x7f62ec13fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33880/0 (socket says 172.21.15.37:33880) 2024-04-25T09:01:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.995+0000 7f62ea59c640 1 -- 172.21.15.37:0/2892392711 learned_addr learned my addr 172.21.15.37:0/2892392711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62ead9d640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62ec0ce510 0x7f62ec0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62eb59e640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec1451f0 0x7f62ec13ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62ea59c640 1 -- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62ec0ce510 msgr2=0x7f62ec0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62ea59c640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62ec0ce510 0x7f62ec0ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62ea59c640 1 -- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec1451f0 msgr2=0x7f62ec13ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62ea59c640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec1451f0 0x7f62ec13ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62ea59c640 1 -- 172.21.15.37:0/2892392711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec1403a0 con 0x7f62ec144ca0 2024-04-25T09:01:17.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62eb59e640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec1451f0 0x7f62ec13ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62ead9d640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62ec0ce510 0x7f62ec0ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62ea59c640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec144ca0 0x7f62ec13fc00 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f62e000c410 tx=0x7f62e000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62d37fe640 1 -- 172.21.15.37:0/2892392711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0011790 con 0x7f62ec144ca0 2024-04-25T09:01:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62d37fe640 1 -- 172.21.15.37:0/2892392711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e0017070 con 0x7f62ec144ca0 2024-04-25T09:01:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62f1009640 1 -- 172.21.15.37:0/2892392711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ec140680 con 0x7f62ec144ca0 2024-04-25T09:01:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62d37fe640 1 -- 172.21.15.37:0/2892392711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e0011bb0 con 0x7f62ec144ca0 2024-04-25T09:01:17.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62f1009640 1 -- 172.21.15.37:0/2892392711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ec140c40 con 0x7f62ec144ca0 2024-04-25T09:01:17.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62f1009640 1 -- 172.21.15.37:0/2892392711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec04efa0 con 0x7f62ec144ca0 2024-04-25T09:01:17.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62d37fe640 1 -- 172.21.15.37:0/2892392711 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f62e001c460 con 0x7f62ec144ca0 2024-04-25T09:01:17.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62d37fe640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f62d8066f60 0x7f62d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62d37fe640 1 -- 172.21.15.37:0/2892392711 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f62e0084f50 con 0x7f62ec144ca0 2024-04-25T09:01:17.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62ead9d640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f62d8066f60 0x7f62d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:17.999+0000 7f62ead9d640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f62d8066f60 0x7f62d8069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0b2000 tx=0x7f62dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.003+0000 7f62d37fe640 1 -- 172.21.15.37:0/2892392711 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f62e0021020 con 0x7f62ec144ca0 2024-04-25T09:01:18.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.159+0000 7f62f1009640 1 -- 172.21.15.37:0/2892392711 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62ec0b0f80 con 0x7f62ec144ca0 2024-04-25T09:01:18.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.159+0000 7f62d37fe640 1 -- 172.21.15.37:0/2892392711 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17299 (secure 0 0 0) 0x7f62e0055cb0 con 0x7f62ec144ca0 2024-04-25T09:01:18.149 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:18.149 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:00:38.139693+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:50.512578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:51.848880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1462327322":"2024-04-26T09:00:38.139659+0000","172.21.15.37:6834/1462327322":"2024-04-26T09:00:38.139659+0000","172.21.15.37:6835/354285428":"2024-04-26T08:59:17.095322+0000","172.21.15.37:6834/354285428":"2024-04-26T08:59:17.095322+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:18.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 -- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f62d8066f60 msgr2=0x7f62d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f62d8066f60 0x7f62d8069420 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0b2000 tx=0x7f62dc008040 comp rx=0 tx=0).stop 2024-04-25T09:01:18.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 -- 172.21.15.37:0/2892392711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec144ca0 msgr2=0x7f62ec13fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec144ca0 0x7f62ec13fc00 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f62e000c410 tx=0x7f62e000c8d0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 -- 172.21.15.37:0/2892392711 shutdown_connections 2024-04-25T09:01:18.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f62d8066f60 0x7f62d8069420 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f62ec1451f0 0x7f62ec13ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec144ca0 0x7f62ec13fc00 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 --2- 172.21.15.37:0/2892392711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f62ec0ce510 0x7f62ec0ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 -- 172.21.15.37:0/2892392711 >> 172.21.15.37:0/2892392711 conn(0x7f62ec0b11a0 msgr2=0x7f62ec0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:18.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 -- 172.21.15.37:0/2892392711 shutdown_connections 2024-04-25T09:01:18.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.163+0000 7f62f1009640 1 -- 172.21.15.37:0/2892392711 wait complete. 2024-04-25T09:01:18.167 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:58:50.512578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:58:51.848880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:01:18.167 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:01:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7183194640 1 Processor -- start 2024-04-25T09:01:18.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7183194640 1 -- start start 2024-04-25T09:01:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7183194640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0d07f0 0x7f717c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7183194640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717c06be60 con 0x7f717c15da80 2024-04-25T09:01:18.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7183194640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717c15de60 con 0x7f717c0d1cd0 2024-04-25T09:01:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7183194640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c06c440 con 0x7f717c0d07f0 2024-04-25T09:01:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7180f30640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f717c15da80 0x7f717c0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40742/0 (socket says 172.21.15.37:40742) 2024-04-25T09:01:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7180f30640 1 -- 172.21.15.37:0/3591749113 learned_addr learned my addr 172.21.15.37:0/3591749113 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:18.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717bfff640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0d07f0 0x7f717c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817285130 0 0) 0x7f717c06be60 con 0x7f717c15da80 2024-04-25T09:01:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7164003530 con 0x7f717c15da80 2024-04-25T09:01:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565399335 0 0) 0x7f717c15de60 con 0x7f717c0d1cd0 2024-04-25T09:01:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 --> v1:172.21.15.203:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f717c06be60 con 0x7f717c0d1cd0 2024-04-25T09:01:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4248751140 0 0) 0x7f7164003530 con 0x7f717c15da80 2024-04-25T09:01:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0d07f0 msgr2=0x7f717c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0d07f0 0x7f717c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 >> v1:172.21.15.203:6789/0 conn(0x7f717c0d1cd0 legacy=0x7f717c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0d2450 con 0x7f717c15da80 2024-04-25T09:01:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f716c001600 con 0x7f717c15da80 2024-04-25T09:01:18.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 >> v1:172.21.15.37:6789/0 conn(0x7f717c15da80 legacy=0x7f717c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0d07f0 0x7f7164003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71640041a0 0x7f7164004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7164004d50 0x7f71640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c06c440 con 0x7f7164004d50 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716400bb60 con 0x7f71640041a0 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716400bd60 con 0x7f717c0d07f0 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7181731640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7164004d50 0x7f71640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7180f30640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0d07f0 0x7f7164003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f717bfff640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71640041a0 0x7f7164004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7181731640 1 -- 172.21.15.37:0/3591749113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0d07f0 msgr2=0x7f7164003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7181731640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0d07f0 0x7f7164003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7181731640 1 -- 172.21.15.37:0/3591749113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71640041a0 msgr2=0x7f7164004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7181731640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71640041a0 0x7f7164004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.307+0000 7f7181731640 1 -- 172.21.15.37:0/3591749113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7164003530 con 0x7f7164004d50 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f7181731640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7164004d50 0x7f71640051d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7174001fd0 tx=0x7f717401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532b63bfbe977169 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7174001c20 con 0x7f7164004d50 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7174003ba0 con 0x7f7164004d50 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f717b7fe640 1 -- 172.21.15.37:0/3591749113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71740050d0 con 0x7f7164004d50 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f7183194640 1 -- 172.21.15.37:0/3591749113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7164004d50 msgr2=0x7f71640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f7183194640 1 --2- 172.21.15.37:0/3591749113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7164004d50 0x7f71640051d0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f7174001fd0 tx=0x7f717401c900 comp rx=0 tx=0).stop 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f7181731640 1 -- 172.21.15.37:0/3591749113 reap_dead start 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f7183194640 1 -- 172.21.15.37:0/3591749113 shutdown_connections 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f7183194640 1 -- 172.21.15.37:0/3591749113 >> 172.21.15.37:0/3591749113 conn(0x7f717c08b000 msgr2=0x7f717c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f7183194640 1 -- 172.21.15.37:0/3591749113 shutdown_connections 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f7183194640 1 -- 172.21.15.37:0/3591749113 wait complete. 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f7183194640 1 Processor -- start 2024-04-25T09:01:18.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.311+0000 7f7183194640 1 -- start start 2024-04-25T09:01:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7183194640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7164004d50 0x7f717c151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7183194640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f717c0b6ac0 0x7f717c0cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7183194640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0b6f40 0x7f717c1f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7183194640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0d26f0 con 0x7f717c0b6ac0 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7183194640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0d2160 con 0x7f7164004d50 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7183194640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0d2870 con 0x7f717c0b6f40 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f717bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f717c0b6ac0 0x7f717c0cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f717bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f717c0b6ac0 0x7f717c0cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33894/0 (socket says 172.21.15.37:33894) 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f717bfff640 1 -- 172.21.15.37:0/1085067319 learned_addr learned my addr 172.21.15.37:0/1085067319 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7181731640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0b6f40 0x7f717c1f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7180f30640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7164004d50 0x7f717c151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f717bfff640 1 -- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0b6f40 msgr2=0x7f717c1f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f717bfff640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0b6f40 0x7f717c1f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f717bfff640 1 -- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7164004d50 msgr2=0x7f717c151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f717bfff640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7164004d50 0x7f717c151bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f717bfff640 1 -- 172.21.15.37:0/1085067319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0b7780 con 0x7f717c0b6ac0 2024-04-25T09:01:18.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f717bfff640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f717c0b6ac0 0x7f717c0cd1e0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7170011600 tx=0x7f7170011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f71797fa640 1 -- 172.21.15.37:0/1085067319 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170019070 con 0x7f717c0b6ac0 2024-04-25T09:01:18.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f71797fa640 1 -- 172.21.15.37:0/1085067319 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7170005db0 con 0x7f717c0b6ac0 2024-04-25T09:01:18.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7183194640 1 -- 172.21.15.37:0/1085067319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717c152110 con 0x7f717c0b6ac0 2024-04-25T09:01:18.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f71797fa640 1 -- 172.21.15.37:0/1085067319 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170006250 con 0x7f717c0b6ac0 2024-04-25T09:01:18.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7183194640 1 -- 172.21.15.37:0/1085067319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717c0cc700 con 0x7f717c0b6ac0 2024-04-25T09:01:18.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7183194640 1 -- 172.21.15.37:0/1085067319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717c04efa0 con 0x7f717c0b6ac0 2024-04-25T09:01:18.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f71797fa640 1 -- 172.21.15.37:0/1085067319 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f717001c070 con 0x7f717c0b6ac0 2024-04-25T09:01:18.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f71797fa640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7168066f60 0x7f7168069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f71797fa640 1 -- 172.21.15.37:0/1085067319 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f71700857b0 con 0x7f717c0b6ac0 2024-04-25T09:01:18.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.315+0000 7f7180f30640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7168066f60 0x7f7168069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.319+0000 7f7180f30640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7168066f60 0x7f7168069420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f716c001060 tx=0x7f716c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.319+0000 7f71797fa640 1 -- 172.21.15.37:0/1085067319 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7170056500 con 0x7f717c0b6ac0 2024-04-25T09:01:18.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.479+0000 7f7183194640 1 -- 172.21.15.37:0/1085067319 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f717c0cfc20 con 0x7f717c0b6ac0 2024-04-25T09:01:18.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.479+0000 7f71797fa640 1 -- 172.21.15.37:0/1085067319 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7170055e00 con 0x7f717c0b6ac0 2024-04-25T09:01:18.466 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:01:18.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.479+0000 7f7183194640 1 -- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7168066f60 msgr2=0x7f7168069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.479+0000 7f7183194640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7168066f60 0x7f7168069420 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f716c001060 tx=0x7f716c009040 comp rx=0 tx=0).stop 2024-04-25T09:01:18.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.479+0000 7f7183194640 1 -- 172.21.15.37:0/1085067319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f717c0b6ac0 msgr2=0x7f717c0cd1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.479+0000 7f7183194640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f717c0b6ac0 0x7f717c0cd1e0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f7170011600 tx=0x7f7170011ac0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.483+0000 7f7183194640 1 -- 172.21.15.37:0/1085067319 shutdown_connections 2024-04-25T09:01:18.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.483+0000 7f7183194640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7168066f60 0x7f7168069420 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.483+0000 7f7183194640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f717c0b6f40 0x7f717c1f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.483+0000 7f7183194640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f717c0b6ac0 0x7f717c0cd1e0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.483+0000 7f7183194640 1 --2- 172.21.15.37:0/1085067319 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7164004d50 0x7f717c151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.483+0000 7f7183194640 1 -- 172.21.15.37:0/1085067319 >> 172.21.15.37:0/1085067319 conn(0x7f717c08b000 msgr2=0x7f717c0d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:18.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.483+0000 7f7183194640 1 -- 172.21.15.37:0/1085067319 shutdown_connections 2024-04-25T09:01:18.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.483+0000 7f7183194640 1 -- 172.21.15.37:0/1085067319 wait complete. 2024-04-25T09:01:18.485 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:01:18.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.623+0000 7fd78eac1640 1 Processor -- start 2024-04-25T09:01:18.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.623+0000 7fd78eac1640 1 -- start start 2024-04-25T09:01:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78eac1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd78815b6b0 0x7fd7880ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78eac1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd788073010 0x7fd7880cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78eac1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7880cbd20 con 0x7fd7880cf590 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78eac1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78815ba90 con 0x7fd78815b6b0 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78eac1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7880cc000 con 0x7fd788073010 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78c85d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd78815b6b0 0x7fd7880ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd787fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd7880cf590 0x7fd78815f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40752/0 (socket says 172.21.15.37:40752) 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd787fff640 1 -- 172.21.15.37:0/1415668777 learned_addr learned my addr 172.21.15.37:0/1415668777 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78d05e640 1 --2- 172.21.15.37:0/1415668777 >> v2:172.21.15.203:3300/0 conn(0x7fd788073010 0x7fd7880cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd7877fe640 1 -- 172.21.15.37:0/1415668777 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462472495 0 0) 0x7fd7880cbd20 con 0x7fd7880cf590 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd7877fe640 1 -- 172.21.15.37:0/1415668777 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd770003530 con 0x7fd7880cf590 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78c85d640 1 -- 172.21.15.37:0/1415668777 >> v2:172.21.15.203:3300/0 conn(0x7fd788073010 msgr2=0x7fd7880cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78c85d640 1 --2- 172.21.15.37:0/1415668777 >> v2:172.21.15.203:3300/0 conn(0x7fd788073010 0x7fd7880cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78c85d640 1 -- 172.21.15.37:0/1415668777 >> v1:172.21.15.37:6789/0 conn(0x7fd7880cf590 legacy=0x7fd78815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78c85d640 1 -- 172.21.15.37:0/1415668777 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7880cc2e0 con 0x7fd78815b6b0 2024-04-25T09:01:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd78c85d640 1 --2- 172.21.15.37:0/1415668777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd78815b6b0 0x7fd7880ca3c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fd78806f000 tx=0x7fd77801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1572cf1c732bcb64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd7877fe640 1 -- 172.21.15.37:0/1415668777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7780b0070 con 0x7fd78815b6b0 2024-04-25T09:01:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd7877fe640 1 -- 172.21.15.37:0/1415668777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7780ab600 con 0x7fd78815b6b0 2024-04-25T09:01:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.627+0000 7fd7877fe640 1 -- 172.21.15.37:0/1415668777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7780ab9d0 con 0x7fd78815b6b0 2024-04-25T09:01:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 -- 172.21.15.37:0/1415668777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd78815b6b0 msgr2=0x7fd7880ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 --2- 172.21.15.37:0/1415668777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd78815b6b0 0x7fd7880ca3c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fd78806f000 tx=0x7fd77801c480 comp rx=0 tx=0).stop 2024-04-25T09:01:18.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 -- 172.21.15.37:0/1415668777 shutdown_connections 2024-04-25T09:01:18.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 --2- 172.21.15.37:0/1415668777 >> v2:172.21.15.203:3300/0 conn(0x7fd788073010 0x7fd7880cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 --2- 172.21.15.37:0/1415668777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd78815b6b0 0x7fd7880ca3c0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 -- 172.21.15.37:0/1415668777 >> 172.21.15.37:0/1415668777 conn(0x7fd78806ae60 msgr2=0x7fd78806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:18.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 -- 172.21.15.37:0/1415668777 shutdown_connections 2024-04-25T09:01:18.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 -- 172.21.15.37:0/1415668777 wait complete. 2024-04-25T09:01:18.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 Processor -- start 2024-04-25T09:01:18.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 -- start start 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd788073010 0x7fd78815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7880cf590 0x7fd78815d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd78815b6b0 0x7fd7880c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880cadb0 con 0x7fd78815b6b0 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880ca900 con 0x7fd788073010 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7880caf30 con 0x7fd7880cf590 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78d05e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd78815b6b0 0x7fd7880c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78d05e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd78815b6b0 0x7fd7880c1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33904/0 (socket says 172.21.15.37:33904) 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78d05e640 1 -- 172.21.15.37:0/2049156656 learned_addr learned my addr 172.21.15.37:0/2049156656 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78c85d640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd788073010 0x7fd78815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd787fff640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7880cf590 0x7fd78815d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78d05e640 1 -- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7880cf590 msgr2=0x7fd78815d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78d05e640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7880cf590 0x7fd78815d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78d05e640 1 -- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd788073010 msgr2=0x7fd78815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78d05e640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd788073010 0x7fd78815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78d05e640 1 -- 172.21.15.37:0/2049156656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78815e160 con 0x7fd78815b6b0 2024-04-25T09:01:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd787fff640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7880cf590 0x7fd78815d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78d05e640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd78815b6b0 0x7fd7880c1a10 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fd780007e60 tx=0x7fd78000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd7857fa640 1 -- 172.21.15.37:0/2049156656 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd780014420 con 0x7fd78815b6b0 2024-04-25T09:01:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd7857fa640 1 -- 172.21.15.37:0/2049156656 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd780019070 con 0x7fd78815b6b0 2024-04-25T09:01:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 -- 172.21.15.37:0/2049156656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78815e440 con 0x7fd78815b6b0 2024-04-25T09:01:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd7857fa640 1 -- 172.21.15.37:0/2049156656 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd780014820 con 0x7fd78815b6b0 2024-04-25T09:01:18.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.631+0000 7fd78eac1640 1 -- 172.21.15.37:0/2049156656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78815de30 con 0x7fd78815b6b0 2024-04-25T09:01:18.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.635+0000 7fd78eac1640 1 -- 172.21.15.37:0/2049156656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78804efa0 con 0x7fd78815b6b0 2024-04-25T09:01:18.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.635+0000 7fd7857fa640 1 -- 172.21.15.37:0/2049156656 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd7800149c0 con 0x7fd78815b6b0 2024-04-25T09:01:18.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.635+0000 7fd7857fa640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd774066f90 0x7fd774069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.635+0000 7fd7857fa640 1 -- 172.21.15.37:0/2049156656 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd780085b90 con 0x7fd78815b6b0 2024-04-25T09:01:18.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.635+0000 7fd78c85d640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd774066f90 0x7fd774069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.635+0000 7fd78c85d640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd774066f90 0x7fd774069450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd7780ad000 tx=0x7fd778008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.639+0000 7fd7857fa640 1 -- 172.21.15.37:0/2049156656 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd7800568e0 con 0x7fd78815b6b0 2024-04-25T09:01:18.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 -- 172.21.15.37:0/2049156656 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd788073d90 con 0x7fd78815b6b0 2024-04-25T09:01:18.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd7857fa640 1 -- 172.21.15.37:0/2049156656 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd7800561e0 con 0x7fd78815b6b0 2024-04-25T09:01:18.787 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:01:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 -- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd774066f90 msgr2=0x7fd774069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd774066f90 0x7fd774069450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd7780ad000 tx=0x7fd778008040 comp rx=0 tx=0).stop 2024-04-25T09:01:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 -- 172.21.15.37:0/2049156656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd78815b6b0 msgr2=0x7fd7880c1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd78815b6b0 0x7fd7880c1a10 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fd780007e60 tx=0x7fd78000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 -- 172.21.15.37:0/2049156656 shutdown_connections 2024-04-25T09:01:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd774066f90 0x7fd774069450 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd78815b6b0 0x7fd7880c1a10 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd7880cf590 0x7fd78815d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 --2- 172.21.15.37:0/2049156656 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd788073010 0x7fd78815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 -- 172.21.15.37:0/2049156656 >> 172.21.15.37:0/2049156656 conn(0x7fd78806ae60 msgr2=0x7fd7880af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:18.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 -- 172.21.15.37:0/2049156656 shutdown_connections 2024-04-25T09:01:18.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.799+0000 7fd78eac1640 1 -- 172.21.15.37:0/2049156656 wait complete. 2024-04-25T09:01:18.806 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:01:18.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616bd26640 1 Processor -- start 2024-04-25T09:01:18.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616bd26640 1 -- start start 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616bd26640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6164159bd0 0x7f61640c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616bd26640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f61640b28c0 0x7f6164163960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616bd26640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61640c9ef0 con 0x7f6164159fb0 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616bd26640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6164072780 con 0x7f61640b28c0 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616bd26640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6164071b70 con 0x7f6164159bd0 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f61692c1640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6164159fb0 0x7f61640b21b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40758/0 (socket says 172.21.15.37:40758) 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f61692c1640 1 -- 172.21.15.37:0/368180265 learned_addr learned my addr 172.21.15.37:0/368180265 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6169ac2640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6164159bd0 0x7f61640c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616a2c3640 1 --2- 172.21.15.37:0/368180265 >> v2:172.21.15.203:3300/0 conn(0x7f61640b28c0 0x7f6164163960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2045227434 0 0) 0x7f61640c9ef0 con 0x7f6164159fb0 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6150003530 con 0x7f6164159fb0 2024-04-25T09:01:18.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 407917422 0 0) 0x7f6150003530 con 0x7f6164159fb0 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6164159bd0 msgr2=0x7f61640c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6164159bd0 0x7f61640c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 >> v2:172.21.15.203:3300/0 conn(0x7f61640b28c0 msgr2=0x7f6164163960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 --2- 172.21.15.37:0/368180265 >> v2:172.21.15.203:3300/0 conn(0x7f61640b28c0 0x7f6164163960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6164071e50 con 0x7f6164159fb0 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f6154001680 con 0x7f6164159fb0 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 >> v1:172.21.15.37:6789/0 conn(0x7f6164159fb0 legacy=0x7f61640b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6150003a30 0x7f6150003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6164159bd0 0x7f6150004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 0x7f61500051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164072780 con 0x7f6150004d50 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164071b70 con 0x7f6150003a30 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615000b9d0 con 0x7f6164159bd0 2024-04-25T09:01:18.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616a2c3640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 0x7f61500051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f61692c1640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6164159bd0 0x7f6150004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f6169ac2640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6150003a30 0x7f6150003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616a2c3640 1 -- 172.21.15.37:0/368180265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6164159bd0 msgr2=0x7f6150004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616a2c3640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6164159bd0 0x7f6150004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616a2c3640 1 -- 172.21.15.37:0/368180265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6150003a30 msgr2=0x7f6150003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616a2c3640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6150003a30 0x7f6150003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.943+0000 7f616a2c3640 1 -- 172.21.15.37:0/368180265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6150003530 con 0x7f6150004d50 2024-04-25T09:01:18.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f616a2c3640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 0x7f61500051d0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f615c00bfc0 tx=0x7f615c01ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4fa79205c686f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c0b4070 con 0x7f6150004d50 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615c0a7c90 con 0x7f6150004d50 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f6168ac0640 1 -- 172.21.15.37:0/368180265 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615c00bc10 con 0x7f6150004d50 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f616bd26640 1 -- 172.21.15.37:0/368180265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 msgr2=0x7f61500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f616bd26640 1 --2- 172.21.15.37:0/368180265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 0x7f61500051d0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f615c00bfc0 tx=0x7f615c01ec80 comp rx=0 tx=0).stop 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f616a2c3640 1 -- 172.21.15.37:0/368180265 reap_dead start 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f616bd26640 1 -- 172.21.15.37:0/368180265 shutdown_connections 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f616bd26640 1 -- 172.21.15.37:0/368180265 >> 172.21.15.37:0/368180265 conn(0x7f616406ae60 msgr2=0x7f616406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f616bd26640 1 -- 172.21.15.37:0/368180265 shutdown_connections 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f616bd26640 1 -- 172.21.15.37:0/368180265 wait complete. 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f616bd26640 1 Processor -- start 2024-04-25T09:01:18.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.947+0000 7f616bd26640 1 -- start start 2024-04-25T09:01:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f616bd26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 0x7f616414bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f616bd26640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f616415a420 0x7f616415a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f616bd26640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61641e4270 0x7f61641e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f616bd26640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61640b2d70 con 0x7f6150004d50 2024-04-25T09:01:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f616bd26640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164072410 con 0x7f616415a420 2024-04-25T09:01:18.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f616bd26640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61640b2ef0 con 0x7f61641e4270 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f6169ac2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 0x7f616414bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f6169ac2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 0x7f616414bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33912/0 (socket says 172.21.15.37:33912) 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f6169ac2640 1 -- 172.21.15.37:0/11966633 learned_addr learned my addr 172.21.15.37:0/11966633 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f6169ac2640 1 -- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61641e4270 msgr2=0x7f61641e46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f6169ac2640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61641e4270 0x7f61641e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f6169ac2640 1 -- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f616415a420 msgr2=0x7f616415a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f6169ac2640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f616415a420 0x7f616415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f6169ac2640 1 -- 172.21.15.37:0/11966633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616415aea0 con 0x7f6150004d50 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f6169ac2640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 0x7f616414bfe0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f61600114e0 tx=0x7f61600119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f615a7fc640 1 -- 172.21.15.37:0/11966633 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6160015930 con 0x7f6150004d50 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f615a7fc640 1 -- 172.21.15.37:0/11966633 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6160015ad0 con 0x7f6150004d50 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f616bd26640 1 -- 172.21.15.37:0/11966633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616415b180 con 0x7f6150004d50 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f615a7fc640 1 -- 172.21.15.37:0/11966633 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616001d400 con 0x7f6150004d50 2024-04-25T09:01:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f616bd26640 1 -- 172.21.15.37:0/11966633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616415b740 con 0x7f6150004d50 2024-04-25T09:01:18.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f616bd26640 1 -- 172.21.15.37:0/11966633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616404ef80 con 0x7f6150004d50 2024-04-25T09:01:18.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f615a7fc640 1 -- 172.21.15.37:0/11966633 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f616001d5a0 con 0x7f6150004d50 2024-04-25T09:01:18.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f615a7fc640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6148066f90 0x7f6148069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f615a7fc640 1 -- 172.21.15.37:0/11966633 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f6160085a90 con 0x7f6150004d50 2024-04-25T09:01:18.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.951+0000 7f61692c1640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6148066f90 0x7f6148069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.955+0000 7f61692c1640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6148066f90 0x7f6148069450 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6154000c10 tx=0x7f6154000ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:18.955+0000 7f615a7fc640 1 -- 172.21.15.37:0/11966633 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6160019030 con 0x7f6150004d50 2024-04-25T09:01:19.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 -- 172.21.15.37:0/11966633 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f616406b310 con 0x7f6150004d50 2024-04-25T09:01:19.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f615a7fc640 1 -- 172.21.15.37:0/11966633 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7f61600567f0 con 0x7f6150004d50 2024-04-25T09:01:19.103 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:01:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 -- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6148066f90 msgr2=0x7f6148069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6148066f90 0x7f6148069450 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6154000c10 tx=0x7f6154000ff0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 -- 172.21.15.37:0/11966633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 msgr2=0x7f616414bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 0x7f616414bfe0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f61600114e0 tx=0x7f61600119a0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 -- 172.21.15.37:0/11966633 shutdown_connections 2024-04-25T09:01:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6148066f90 0x7f6148069450 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f61641e4270 0x7f61641e46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f616415a420 0x7f616415a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 --2- 172.21.15.37:0/11966633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150004d50 0x7f616414bfe0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 -- 172.21.15.37:0/11966633 >> 172.21.15.37:0/11966633 conn(0x7f616406ae60 msgr2=0x7f61640acb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 -- 172.21.15.37:0/11966633 shutdown_connections 2024-04-25T09:01:19.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.115+0000 7f616bd26640 1 -- 172.21.15.37:0/11966633 wait complete. 2024-04-25T09:01:19.121 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:01:19.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.247+0000 7f4828904640 1 Processor -- start 2024-04-25T09:01:19.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.247+0000 7f4828904640 1 -- start start 2024-04-25T09:01:19.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f4828904640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f482415da80 0x7f48240d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f4828904640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48240d07f0 0x7f482406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f4828904640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482406be60 con 0x7f48240d1cd0 2024-04-25T09:01:19.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f4828904640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482406c440 con 0x7f482415da80 2024-04-25T09:01:19.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f4828904640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482415de60 con 0x7f48240d07f0 2024-04-25T09:01:19.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f4821d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f48240d1cd0 0x7f48240d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40766/0 (socket says 172.21.15.37:40766) 2024-04-25T09:01:19.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f4821d9b640 1 -- 172.21.15.37:0/2914725589 learned_addr learned my addr 172.21.15.37:0/2914725589 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:19.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482259c640 1 --2- 172.21.15.37:0/2914725589 >> v2:172.21.15.203:3300/0 conn(0x7f482415da80 0x7f48240d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458112806 0 0) 0x7f482406be60 con 0x7f48240d1cd0 2024-04-25T09:01:19.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f480c003530 con 0x7f48240d1cd0 2024-04-25T09:01:19.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f4822d9d640 1 --2- 172.21.15.37:0/2914725589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48240d07f0 0x7f482406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3690639546 0 0) 0x7f480c003530 con 0x7f48240d1cd0 2024-04-25T09:01:19.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48240d07f0 msgr2=0x7f482406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:19.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 --2- 172.21.15.37:0/2914725589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48240d07f0 0x7f482406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 >> v2:172.21.15.203:3300/0 conn(0x7f482415da80 msgr2=0x7f48240d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:19.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 --2- 172.21.15.37:0/2914725589 >> v2:172.21.15.203:3300/0 conn(0x7f482415da80 0x7f48240d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48240d2450 con 0x7f48240d1cd0 2024-04-25T09:01:19.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f4818001820 con 0x7f48240d1cd0 2024-04-25T09:01:19.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 >> v1:172.21.15.37:6789/0 conn(0x7f48240d1cd0 legacy=0x7f48240d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 --2- 172.21.15.37:0/2914725589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c003a30 0x7f480c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 --2- 172.21.15.37:0/2914725589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48240d07f0 0x7f480c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 --2- 172.21.15.37:0/2914725589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f480c004d50 0x7f480c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482406c440 con 0x7f480c003a30 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482415de60 con 0x7f480c004d50 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480c00bb00 con 0x7f48240d07f0 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482259c640 1 --2- 172.21.15.37:0/2914725589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c003a30 0x7f480c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f4821d9b640 1 --2- 172.21.15.37:0/2914725589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48240d07f0 0x7f480c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482259c640 1 -- 172.21.15.37:0/2914725589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48240d07f0 msgr2=0x7f480c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482259c640 1 --2- 172.21.15.37:0/2914725589 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48240d07f0 0x7f480c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482259c640 1 -- 172.21.15.37:0/2914725589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f480c004d50 msgr2=0x7f480c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482259c640 1 --2- 172.21.15.37:0/2914725589 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f480c004d50 0x7f480c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.251+0000 7f482259c640 1 -- 172.21.15.37:0/2914725589 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f480c003530 con 0x7f480c003a30 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f482259c640 1 --2- 172.21.15.37:0/2914725589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c003a30 0x7f480c003e50 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4814009ae0 tx=0x7f481401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7857d7e3bcdfd798 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48140b2070 con 0x7f480c003a30 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48140a55b0 con 0x7f480c003a30 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f482159a640 1 -- 172.21.15.37:0/2914725589 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48140096b0 con 0x7f480c003a30 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 -- 172.21.15.37:0/2914725589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c003a30 msgr2=0x7f480c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 --2- 172.21.15.37:0/2914725589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f480c003a30 0x7f480c003e50 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f4814009ae0 tx=0x7f481401ca80 comp rx=0 tx=0).stop 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4822d9d640 1 -- 172.21.15.37:0/2914725589 reap_dead start 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 -- 172.21.15.37:0/2914725589 shutdown_connections 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 -- 172.21.15.37:0/2914725589 >> 172.21.15.37:0/2914725589 conn(0x7f482408b000 msgr2=0x7f48240d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 -- 172.21.15.37:0/2914725589 shutdown_connections 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 -- 172.21.15.37:0/2914725589 wait complete. 2024-04-25T09:01:19.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 Processor -- start 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 -- start start 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f480c003a30 0x7f48240ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4824155e30 0x7f48241566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48241562b0 0x7f48241584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482406d240 con 0x7f4824155e30 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482406d0c0 con 0x7f480c003a30 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482406d3c0 con 0x7f48241562b0 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4821d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4824155e30 0x7f48241566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4821d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4824155e30 0x7f48241566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33932/0 (socket says 172.21.15.37:33932) 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4821d9b640 1 -- 172.21.15.37:0/1019613324 learned_addr learned my addr 172.21.15.37:0/1019613324 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f482259c640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f480c003a30 0x7f48240ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4822d9d640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48241562b0 0x7f48241584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4821d9b640 1 -- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48241562b0 msgr2=0x7f48241584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4821d9b640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48241562b0 0x7f48241584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4821d9b640 1 -- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f480c003a30 msgr2=0x7f48240ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4821d9b640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f480c003a30 0x7f48240ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4821d9b640 1 -- 172.21.15.37:0/1019613324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4824159160 con 0x7f4824155e30 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f482259c640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f480c003a30 0x7f48240ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4822d9d640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48241562b0 0x7f48241584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4821d9b640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4824155e30 0x7f48241566d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f48180011e0 tx=0x7f481800dc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4812ffd640 1 -- 172.21.15.37:0/1019613324 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800de60 con 0x7f4824155e30 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4812ffd640 1 -- 172.21.15.37:0/1019613324 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4818017070 con 0x7f4824155e30 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4812ffd640 1 -- 172.21.15.37:0/1019613324 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48180057b0 con 0x7f4824155e30 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 -- 172.21.15.37:0/1019613324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4824159440 con 0x7f4824155e30 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.255+0000 7f4828904640 1 -- 172.21.15.37:0/1019613324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48240b44e0 con 0x7f4824155e30 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.259+0000 7f4828904640 1 -- 172.21.15.37:0/1019613324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48240b46e0 con 0x7f4824155e30 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.259+0000 7f4812ffd640 1 -- 172.21.15.37:0/1019613324 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4818005950 con 0x7f4824155e30 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.259+0000 7f4812ffd640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4804066f90 0x7f4804069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.259+0000 7f4812ffd640 1 -- 172.21.15.37:0/1019613324 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f48180853c0 con 0x7f4824155e30 2024-04-25T09:01:19.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.259+0000 7f482259c640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4804066f90 0x7f4804069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.259+0000 7f482259c640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4804066f90 0x7f4804069450 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f48140af000 tx=0x7f48140046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.263+0000 7f4812ffd640 1 -- 172.21.15.37:0/1019613324 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f48240b46e0 con 0x7f4824155e30 2024-04-25T09:01:19.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.435+0000 7f4828904640 1 -- 172.21.15.37:0/1019613324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f482404efa0 con 0x7f4824155e30 2024-04-25T09:01:19.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.435+0000 7f4812ffd640 1 -- 172.21.15.37:0/1019613324 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v99) v1 ==== 80+0+242 (secure 0 0 0) 0x7f4818019310 con 0x7f4824155e30 2024-04-25T09:01:19.425 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/1462327322 2024-04-26T09:00:38.139659+0000 2024-04-25T09:01:19.425 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6834/1462327322 2024-04-26T09:00:38.139659+0000 2024-04-25T09:01:19.425 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/354285428 2024-04-26T08:59:17.095322+0000 2024-04-25T09:01:19.425 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6834/354285428 2024-04-26T08:59:17.095322+0000 2024-04-25T09:01:19.425 INFO:teuthology.orchestra.run.smithi037.stderr:listed 4 entries 2024-04-25T09:01:19.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 -- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4804066f90 msgr2=0x7f4804069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4804066f90 0x7f4804069450 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f48140af000 tx=0x7f48140046d0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 -- 172.21.15.37:0/1019613324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4824155e30 msgr2=0x7f48241566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4824155e30 0x7f48241566d0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f48180011e0 tx=0x7f481800dc70 comp rx=0 tx=0).stop 2024-04-25T09:01:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 -- 172.21.15.37:0/1019613324 shutdown_connections 2024-04-25T09:01:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4804066f90 0x7f4804069450 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f48241562b0 0x7f48241584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4824155e30 0x7f48241566d0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 --2- 172.21.15.37:0/1019613324 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f480c003a30 0x7f48240ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 -- 172.21.15.37:0/1019613324 >> 172.21.15.37:0/1019613324 conn(0x7f482408b000 msgr2=0x7f482415e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 -- 172.21.15.37:0/1019613324 shutdown_connections 2024-04-25T09:01:19.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.439+0000 7f4828904640 1 -- 172.21.15.37:0/1019613324 wait complete. 2024-04-25T09:01:19.443 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T09:01:19.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:19.444 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:19.455 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[49947]: fuse finished with error 0 and tester_r 0 2024-04-25T09:01:19.456 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:01:19.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:19.456 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:01:19.537 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:01:19.537 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:19.544 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:01:19.544 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T09:01:19.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:19.544 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:01:19.591 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:01:19.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:01:19.593 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:01:19.593 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:19.598 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:01:19.598 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:01:19.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:19.601 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:01:19.646 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:01:19.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:01:19.647 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:01:19.647 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:19.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:01:19.653 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T09:01:19.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:19.653 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:01:19.699 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:01:19.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:01:19.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.843+0000 7f90bed2a640 1 Processor -- start 2024-04-25T09:01:19.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.843+0000 7f90bed2a640 1 -- start start 2024-04-25T09:01:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bed2a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f90b80ce510 0x7f90b80ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bed2a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f90b8144ca0 0x7f90b806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bed2a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b80cee30 con 0x7f90b806ec60 2024-04-25T09:01:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bed2a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b80cc2b0 con 0x7f90b80ce510 2024-04-25T09:01:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bed2a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b806e290 con 0x7f90b8144ca0 2024-04-25T09:01:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bd2c7640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f90b806ec60 0x7f90b8142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48806/0 (socket says 172.21.15.37:48806) 2024-04-25T09:01:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bd2c7640 1 -- 172.21.15.37:0/2018382252 learned_addr learned my addr 172.21.15.37:0/2018382252 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bcac6640 1 --2- 172.21.15.37:0/2018382252 >> v2:172.21.15.37:3300/0 conn(0x7f90b80ce510 0x7f90b80ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90b7fff640 1 --2- 172.21.15.37:0/2018382252 >> v2:172.21.15.203:3300/0 conn(0x7f90b8144ca0 0x7f90b806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bcac6640 1 -- 172.21.15.37:0/2018382252 >> v2:172.21.15.203:3300/0 conn(0x7f90b8144ca0 msgr2=0x7f90b806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bcac6640 1 --2- 172.21.15.37:0/2018382252 >> v2:172.21.15.203:3300/0 conn(0x7f90b8144ca0 0x7f90b806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bcac6640 1 -- 172.21.15.37:0/2018382252 >> v1:172.21.15.37:6789/0 conn(0x7f90b806ec60 legacy=0x7f90b8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90b77fe640 1 -- 172.21.15.37:0/2018382252 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694567769 0 0) 0x7f90b80cee30 con 0x7f90b806ec60 2024-04-25T09:01:19.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.847+0000 7f90bcac6640 1 -- 172.21.15.37:0/2018382252 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b80ccc70 con 0x7f90b80ce510 2024-04-25T09:01:19.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90bcac6640 1 --2- 172.21.15.37:0/2018382252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90b80ce510 0x7f90b80ce8f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f90b8070fe0 tx=0x7f90a801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36b2fd13677d2e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90b77fe640 1 -- 172.21.15.37:0/2018382252 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a80b5070 con 0x7f90b80ce510 2024-04-25T09:01:19.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90b77fe640 1 -- 172.21.15.37:0/2018382252 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90a80a8970 con 0x7f90b80ce510 2024-04-25T09:01:19.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90b77fe640 1 -- 172.21.15.37:0/2018382252 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a80a8d40 con 0x7f90b80ce510 2024-04-25T09:01:19.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90bed2a640 1 -- 172.21.15.37:0/2018382252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90b80ce510 msgr2=0x7f90b80ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90bed2a640 1 --2- 172.21.15.37:0/2018382252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90b80ce510 0x7f90b80ce8f0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f90b8070fe0 tx=0x7f90a801ec70 comp rx=0 tx=0).stop 2024-04-25T09:01:19.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90bed2a640 1 -- 172.21.15.37:0/2018382252 shutdown_connections 2024-04-25T09:01:19.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90bed2a640 1 --2- 172.21.15.37:0/2018382252 >> v2:172.21.15.203:3300/0 conn(0x7f90b8144ca0 0x7f90b806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90bed2a640 1 --2- 172.21.15.37:0/2018382252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90b80ce510 0x7f90b80ce8f0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90bed2a640 1 -- 172.21.15.37:0/2018382252 >> 172.21.15.37:0/2018382252 conn(0x7f90b80b11a0 msgr2=0x7f90b80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:19.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90bed2a640 1 -- 172.21.15.37:0/2018382252 shutdown_connections 2024-04-25T09:01:19.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90bed2a640 1 -- 172.21.15.37:0/2018382252 wait complete. 2024-04-25T09:01:19.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90bed2a640 1 Processor -- start 2024-04-25T09:01:19.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.851+0000 7f90bed2a640 1 -- start start 2024-04-25T09:01:19.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bed2a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90b806ec60 0x7f90b80add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bed2a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b80ce510 0x7f90b813bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bed2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90b8144ca0 0x7f90b80b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bed2a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b80cc900 con 0x7f90b8144ca0 2024-04-25T09:01:19.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bed2a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b806df20 con 0x7f90b806ec60 2024-04-25T09:01:19.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bed2a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b8142d90 con 0x7f90b80ce510 2024-04-25T09:01:19.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bd2c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90b8144ca0 0x7f90b80b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bd2c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90b8144ca0 0x7f90b80b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60456/0 (socket says 172.21.15.37:60456) 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bd2c7640 1 -- 172.21.15.37:0/2951197209 learned_addr learned my addr 172.21.15.37:0/2951197209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90b7fff640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b80ce510 0x7f90b813bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bcac6640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90b806ec60 0x7f90b80add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bd2c7640 1 -- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b80ce510 msgr2=0x7f90b813bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bd2c7640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b80ce510 0x7f90b813bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bd2c7640 1 -- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90b806ec60 msgr2=0x7f90b80add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bd2c7640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90b806ec60 0x7f90b80add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bd2c7640 1 -- 172.21.15.37:0/2951197209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b81e0c90 con 0x7f90b8144ca0 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bd2c7640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90b8144ca0 0x7f90b80b3ff0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c7a0 tx=0x7f90b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90b57fa640 1 -- 172.21.15.37:0/2951197209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0018070 con 0x7f90b8144ca0 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90b57fa640 1 -- 172.21.15.37:0/2951197209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b000a640 con 0x7f90b8144ca0 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bed2a640 1 -- 172.21.15.37:0/2951197209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b81e0f30 con 0x7f90b8144ca0 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90b57fa640 1 -- 172.21.15.37:0/2951197209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b000aab0 con 0x7f90b8144ca0 2024-04-25T09:01:19.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bed2a640 1 -- 172.21.15.37:0/2951197209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b80ad500 con 0x7f90b8144ca0 2024-04-25T09:01:19.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bed2a640 1 -- 172.21.15.37:0/2951197209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b80ad2f0 con 0x7f90b8144ca0 2024-04-25T09:01:19.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90b57fa640 1 -- 172.21.15.37:0/2951197209 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f90b001b070 con 0x7f90b8144ca0 2024-04-25T09:01:19.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90b57fa640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90a4066f60 0x7f90a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90b57fa640 1 -- 172.21.15.37:0/2951197209 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f90b00848d0 con 0x7f90b8144ca0 2024-04-25T09:01:19.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.855+0000 7f90bcac6640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90a4066f60 0x7f90a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.859+0000 7f90bcac6640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90a4066f60 0x7f90a4069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f90a80b2000 tx=0x7f90a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:19.859+0000 7f90b57fa640 1 -- 172.21.15.37:0/2951197209 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f90b0055630 con 0x7f90b8144ca0 2024-04-25T09:01:20.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.055+0000 7f90bed2a640 1 -- 172.21.15.37:0/2951197209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90b80b1cc0 con 0x7f90b8144ca0 2024-04-25T09:01:20.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.055+0000 7f90b57fa640 1 -- 172.21.15.37:0/2951197209 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f90b0054f30 con 0x7f90b8144ca0 2024-04-25T09:01:20.044 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:20.044 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T09:00:41.164474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/3366017865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3366017865},{"type":"v1","addr":"172.21.15.37:6835","nonce":3366017865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:20.045 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 73 2024-04-25T09:01:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 -- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90a4066f60 msgr2=0x7f90a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90a4066f60 0x7f90a4069420 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f90a80b2000 tx=0x7f90a8008040 comp rx=0 tx=0).stop 2024-04-25T09:01:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 -- 172.21.15.37:0/2951197209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90b8144ca0 msgr2=0x7f90b80b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90b8144ca0 0x7f90b80b3ff0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c7a0 tx=0x7f90b000cc60 comp rx=0 tx=0).stop 2024-04-25T09:01:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 -- 172.21.15.37:0/2951197209 shutdown_connections 2024-04-25T09:01:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f90a4066f60 0x7f90a4069420 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90b8144ca0 0x7f90b80b3ff0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f90b80ce510 0x7f90b813bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 --2- 172.21.15.37:0/2951197209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90b806ec60 0x7f90b80add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 -- 172.21.15.37:0/2951197209 >> 172.21.15.37:0/2951197209 conn(0x7f90b80b11a0 msgr2=0x7f90b8071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 -- 172.21.15.37:0/2951197209 shutdown_connections 2024-04-25T09:01:20.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.059+0000 7f90bed2a640 1 -- 172.21.15.37:0/2951197209 wait complete. 2024-04-25T09:01:20.064 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:20.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5ae56f9640 1 Processor -- start 2024-04-25T09:01:20.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5ae56f9640 1 -- start start 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5ae56f9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5ae0160080 0x7f5ae00c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5ae56f9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae00affa0 con 0x7f5ae00c6e50 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5ae56f9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae00c72c0 con 0x7f5ae015ed40 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5ae56f9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae015f120 con 0x7f5ae0160080 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5ade7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5ae0160080 0x7f5ae00c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5adf7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5ae00c6e50 0x7f5ae00caad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48818/0 (socket says 172.21.15.37:48818) 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5adf7fe640 1 -- 172.21.15.37:0/3422891257 learned_addr learned my addr 172.21.15.37:0/3422891257 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:20.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5ade7fc640 1 -- 172.21.15.37:0/3422891257 >> v1:172.21.15.203:6789/0 conn(0x7f5ae015ed40 legacy=0x7f5ae00d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:20.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5ade7fc640 1 -- 172.21.15.37:0/3422891257 >> v1:172.21.15.37:6789/0 conn(0x7f5ae00c6e50 legacy=0x7f5ae00caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.203+0000 7f5ade7fc640 1 -- 172.21.15.37:0/3422891257 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae00b0240 con 0x7f5ae0160080 2024-04-25T09:01:20.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ade7fc640 1 --2- 172.21.15.37:0/3422891257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae0160080 0x7f5ae00c6910 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40097c0 tx=0x7f5ad401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cb491ce52d088f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5addffb640 1 -- 172.21.15.37:0/3422891257 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40b1070 con 0x7f5ae0160080 2024-04-25T09:01:20.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5addffb640 1 -- 172.21.15.37:0/3422891257 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad40a8050 con 0x7f5ae0160080 2024-04-25T09:01:20.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5addffb640 1 -- 172.21.15.37:0/3422891257 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad40ac860 con 0x7f5ae0160080 2024-04-25T09:01:20.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 -- 172.21.15.37:0/3422891257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae0160080 msgr2=0x7f5ae00c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 --2- 172.21.15.37:0/3422891257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae0160080 0x7f5ae00c6910 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40097c0 tx=0x7f5ad401c3e0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 -- 172.21.15.37:0/3422891257 shutdown_connections 2024-04-25T09:01:20.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 --2- 172.21.15.37:0/3422891257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae0160080 0x7f5ae00c6910 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 -- 172.21.15.37:0/3422891257 >> 172.21.15.37:0/3422891257 conn(0x7f5ae0099a70 msgr2=0x7f5ae00d0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 -- 172.21.15.37:0/3422891257 shutdown_connections 2024-04-25T09:01:20.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 -- 172.21.15.37:0/3422891257 wait complete. 2024-04-25T09:01:20.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 Processor -- start 2024-04-25T09:01:20.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 -- start start 2024-04-25T09:01:20.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae00c6e50 0x7f5ae00c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae015ed40 0x7f5ae0157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae0160080 0x7f5ae00c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae00cbb70 con 0x7f5ae0160080 2024-04-25T09:01:20.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae00cb9f0 con 0x7f5ae015ed40 2024-04-25T09:01:20.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae00cbcf0 con 0x7f5ae00c6e50 2024-04-25T09:01:20.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5adf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae0160080 0x7f5ae00c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5adf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae0160080 0x7f5ae00c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60482/0 (socket says 172.21.15.37:60482) 2024-04-25T09:01:20.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5adf7fe640 1 -- 172.21.15.37:0/2279824463 learned_addr learned my addr 172.21.15.37:0/2279824463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5adf7fe640 1 -- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae00c6e50 msgr2=0x7f5ae00c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ade7fc640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae015ed40 0x7f5ae0157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5adf7fe640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae00c6e50 0x7f5ae00c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5adf7fe640 1 -- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae015ed40 msgr2=0x7f5ae0157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5adf7fe640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae015ed40 0x7f5ae0157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5adf7fe640 1 -- 172.21.15.37:0/2279824463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae0157bc0 con 0x7f5ae0160080 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5adf7fe640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae0160080 0x7f5ae00c99e0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800cbc0 tx=0x7f5ad800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ac77fe640 1 -- 172.21.15.37:0/2279824463 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8017070 con 0x7f5ae0160080 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ac77fe640 1 -- 172.21.15.37:0/2279824463 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad800ad40 con 0x7f5ae0160080 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.207+0000 7f5ae56f9640 1 -- 172.21.15.37:0/2279824463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae0157ea0 con 0x7f5ae0160080 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.211+0000 7f5ac77fe640 1 -- 172.21.15.37:0/2279824463 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8012600 con 0x7f5ae0160080 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.211+0000 7f5ae56f9640 1 -- 172.21.15.37:0/2279824463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae0158460 con 0x7f5ae0160080 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.211+0000 7f5ae56f9640 1 -- 172.21.15.37:0/2279824463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae004efa0 con 0x7f5ae0160080 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.211+0000 7f5ac77fe640 1 -- 172.21.15.37:0/2279824463 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5ad801a070 con 0x7f5ae0160080 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.211+0000 7f5ac77fe640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5acc066f60 0x7f5acc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.211+0000 7f5ac77fe640 1 -- 172.21.15.37:0/2279824463 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f5ad8084f80 con 0x7f5ae0160080 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.211+0000 7f5adeffd640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5acc066f60 0x7f5acc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.211+0000 7f5adeffd640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5acc066f60 0x7f5acc069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0007fd0 tx=0x7f5ad0007960 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.215+0000 7f5ac77fe640 1 -- 172.21.15.37:0/2279824463 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5ad8055680 con 0x7f5ae0160080 2024-04-25T09:01:20.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.367+0000 7f5ae56f9640 1 -- 172.21.15.37:0/2279824463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ae00680c0 con 0x7f5ae0160080 2024-04-25T09:01:20.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ac77fe640 1 -- 172.21.15.37:0/2279824463 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17299 (secure 0 0 0) 0x7f5ad8059720 con 0x7f5ae0160080 2024-04-25T09:01:20.358 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:20.359 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:00:38.139693+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:50.512578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:51.848880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1462327322":"2024-04-26T09:00:38.139659+0000","172.21.15.37:6834/1462327322":"2024-04-26T09:00:38.139659+0000","172.21.15.37:6835/354285428":"2024-04-26T08:59:17.095322+0000","172.21.15.37:6834/354285428":"2024-04-26T08:59:17.095322+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 -- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5acc066f60 msgr2=0x7f5acc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5acc066f60 0x7f5acc069420 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0007fd0 tx=0x7f5ad0007960 comp rx=0 tx=0).stop 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 -- 172.21.15.37:0/2279824463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae0160080 msgr2=0x7f5ae00c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae0160080 0x7f5ae00c99e0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800cbc0 tx=0x7f5ad800a430 comp rx=0 tx=0).stop 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 -- 172.21.15.37:0/2279824463 shutdown_connections 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5acc066f60 0x7f5acc069420 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ae0160080 0x7f5ae00c99e0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5ae015ed40 0x7f5ae0157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 --2- 172.21.15.37:0/2279824463 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5ae00c6e50 0x7f5ae00c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 -- 172.21.15.37:0/2279824463 >> 172.21.15.37:0/2279824463 conn(0x7f5ae0099a70 msgr2=0x7f5ae006f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 -- 172.21.15.37:0/2279824463 shutdown_connections 2024-04-25T09:01:20.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.371+0000 7f5ae56f9640 1 -- 172.21.15.37:0/2279824463 wait complete. 2024-04-25T09:01:20.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:58:50.512578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:58:51.848880+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:01:20.377 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:01:20.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc7023c640 1 Processor -- start 2024-04-25T09:01:20.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc7023c640 1 -- start start 2024-04-25T09:01:20.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc7023c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcc680b3cf0 0x7fcc680cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc7023c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc680cfad0 0x7fcc6815dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc7023c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680cbf10 con 0x7fcc6815e4f0 2024-04-25T09:01:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc7023c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc680cff40 con 0x7fcc680b3cf0 2024-04-25T09:01:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc7023c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680cc1f0 con 0x7fcc680cfad0 2024-04-25T09:01:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc6dfd8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcc680b3cf0 0x7fcc680cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc6d7d7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc680cfad0 0x7fcc6815dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc6dfd8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcc680b3cf0 0x7fcc680cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:59516/0 (socket says 172.21.15.37:59516) 2024-04-25T09:01:20.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc6dfd8640 1 -- 172.21.15.37:0/3188562731 learned_addr learned my addr 172.21.15.37:0/3188562731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc6dfd8640 1 -- 172.21.15.37:0/3188562731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc680cfad0 msgr2=0x7fcc6815dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc6dfd8640 1 --2- 172.21.15.37:0/3188562731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc680cfad0 0x7fcc6815dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc6dfd8640 1 -- 172.21.15.37:0/3188562731 >> v1:172.21.15.203:6789/0 conn(0x7fcc6815e4f0 legacy=0x7fcc68161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc6dfd8640 1 -- 172.21.15.37:0/3188562731 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680cc4d0 con 0x7fcc680b3cf0 2024-04-25T09:01:20.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.507+0000 7fcc6d7d7640 1 --2- 172.21.15.37:0/3188562731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc680cfad0 0x7fcc6815dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:20.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc6dfd8640 1 --2- 172.21.15.37:0/3188562731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc680b3cf0 0x7fcc680cf590 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009870 tx=0x7fcc6401c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5385e69e29b8dff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc6cfd6640 1 -- 172.21.15.37:0/3188562731 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc640b0070 con 0x7fcc680b3cf0 2024-04-25T09:01:20.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc6cfd6640 1 -- 172.21.15.37:0/3188562731 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc640ab600 con 0x7fcc680b3cf0 2024-04-25T09:01:20.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc6cfd6640 1 -- 172.21.15.37:0/3188562731 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc640ab9d0 con 0x7fcc680b3cf0 2024-04-25T09:01:20.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc7023c640 1 -- 172.21.15.37:0/3188562731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc680b3cf0 msgr2=0x7fcc680cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc7023c640 1 --2- 172.21.15.37:0/3188562731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc680b3cf0 0x7fcc680cf590 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009870 tx=0x7fcc6401c560 comp rx=0 tx=0).stop 2024-04-25T09:01:20.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc7023c640 1 -- 172.21.15.37:0/3188562731 shutdown_connections 2024-04-25T09:01:20.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc7023c640 1 --2- 172.21.15.37:0/3188562731 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc680cfad0 0x7fcc6815dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc7023c640 1 --2- 172.21.15.37:0/3188562731 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc680b3cf0 0x7fcc680cf590 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc7023c640 1 -- 172.21.15.37:0/3188562731 >> 172.21.15.37:0/3188562731 conn(0x7fcc6806ae60 msgr2=0x7fcc6806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc7023c640 1 -- 172.21.15.37:0/3188562731 shutdown_connections 2024-04-25T09:01:20.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.511+0000 7fcc7023c640 1 -- 172.21.15.37:0/3188562731 wait complete. 2024-04-25T09:01:20.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc7023c640 1 Processor -- start 2024-04-25T09:01:20.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc7023c640 1 -- start start 2024-04-25T09:01:20.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc7023c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc680b3cf0 0x7fcc68098920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc7023c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc680cfad0 0x7fcc681538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc7023c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6815e4f0 0x7fcc6815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc7023c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680b4250 con 0x7fcc680b3cf0 2024-04-25T09:01:20.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc7023c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680b40d0 con 0x7fcc6815e4f0 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc7023c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6815e900 con 0x7fcc680cfad0 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6dfd8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc680b3cf0 0x7fcc68098920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6dfd8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc680b3cf0 0x7fcc68098920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60488/0 (socket says 172.21.15.37:60488) 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6dfd8640 1 -- 172.21.15.37:0/253092877 learned_addr learned my addr 172.21.15.37:0/253092877 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6e7d9640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6815e4f0 0x7fcc6815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6d7d7640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc680cfad0 0x7fcc681538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6dfd8640 1 -- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc680cfad0 msgr2=0x7fcc681538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6dfd8640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc680cfad0 0x7fcc681538e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6dfd8640 1 -- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6815e4f0 msgr2=0x7fcc6815c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6dfd8640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6815e4f0 0x7fcc6815c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6dfd8640 1 -- 172.21.15.37:0/253092877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68152e40 con 0x7fcc680b3cf0 2024-04-25T09:01:20.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6e7d9640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6815e4f0 0x7fcc6815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:20.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc6dfd8640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc680b3cf0 0x7fcc68098920 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fcc640ae040 tx=0x7fcc64019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc5e7fc640 1 -- 172.21.15.37:0/253092877 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc640ba070 con 0x7fcc680b3cf0 2024-04-25T09:01:20.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc5e7fc640 1 -- 172.21.15.37:0/253092877 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc640b4640 con 0x7fcc680b3cf0 2024-04-25T09:01:20.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc5e7fc640 1 -- 172.21.15.37:0/253092877 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc640b0040 con 0x7fcc680b3cf0 2024-04-25T09:01:20.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc7023c640 1 -- 172.21.15.37:0/253092877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc680c7190 con 0x7fcc680b3cf0 2024-04-25T09:01:20.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc7023c640 1 -- 172.21.15.37:0/253092877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc68098e60 con 0x7fcc680b3cf0 2024-04-25T09:01:20.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc7023c640 1 -- 172.21.15.37:0/253092877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6804efa0 con 0x7fcc680b3cf0 2024-04-25T09:01:20.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.515+0000 7fcc5e7fc640 1 -- 172.21.15.37:0/253092877 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcc640ab4c0 con 0x7fcc680b3cf0 2024-04-25T09:01:20.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.519+0000 7fcc5e7fc640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc4c066f60 0x7fcc4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.519+0000 7fcc5e7fc640 1 -- 172.21.15.37:0/253092877 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fcc6411e550 con 0x7fcc680b3cf0 2024-04-25T09:01:20.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.519+0000 7fcc6d7d7640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc4c066f60 0x7fcc4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.519+0000 7fcc6d7d7640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc4c066f60 0x7fcc4c069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009580 tx=0x7fcc58008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.523+0000 7fcc5e7fc640 1 -- 172.21.15.37:0/253092877 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcc64016150 con 0x7fcc680b3cf0 2024-04-25T09:01:20.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.683+0000 7fcc7023c640 1 -- 172.21.15.37:0/253092877 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc680b3060 con 0x7fcc680b3cf0 2024-04-25T09:01:20.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.683+0000 7fcc5e7fc640 1 -- 172.21.15.37:0/253092877 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc640ef2b0 con 0x7fcc680b3cf0 2024-04-25T09:01:20.671 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:01:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 -- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc4c066f60 msgr2=0x7fcc4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc4c066f60 0x7fcc4c069420 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcc58009580 tx=0x7fcc58008040 comp rx=0 tx=0).stop 2024-04-25T09:01:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 -- 172.21.15.37:0/253092877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc680b3cf0 msgr2=0x7fcc68098920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc680b3cf0 0x7fcc68098920 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fcc640ae040 tx=0x7fcc64019300 comp rx=0 tx=0).stop 2024-04-25T09:01:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 -- 172.21.15.37:0/253092877 shutdown_connections 2024-04-25T09:01:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcc4c066f60 0x7fcc4c069420 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcc6815e4f0 0x7fcc6815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcc680cfad0 0x7fcc681538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 --2- 172.21.15.37:0/253092877 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc680b3cf0 0x7fcc68098920 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 -- 172.21.15.37:0/253092877 >> 172.21.15.37:0/253092877 conn(0x7fcc6806ae60 msgr2=0x7fcc680cdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 -- 172.21.15.37:0/253092877 shutdown_connections 2024-04-25T09:01:20.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.687+0000 7fcc7023c640 1 -- 172.21.15.37:0/253092877 wait complete. 2024-04-25T09:01:20.690 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:01:20.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f0411bd9640 1 Processor -- start 2024-04-25T09:01:20.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f0411bd9640 1 -- start start 2024-04-25T09:01:20.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f0411bd9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f040c15b6b0 0x7f040c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f0411bd9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f040c073010 0x7f040c0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f0411bd9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040c0cbd20 con 0x7f040c0cf590 2024-04-25T09:01:20.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f0411bd9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040c15ba90 con 0x7f040c073010 2024-04-25T09:01:20.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f0411bd9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0cff30 con 0x7f040c15b6b0 2024-04-25T09:01:20.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f040bfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f040c073010 0x7f040c0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f040b7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f040c15b6b0 0x7f040c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f040affd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f040c0cf590 0x7f040c15f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:52006/0 (socket says 172.21.15.37:52006) 2024-04-25T09:01:20.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f040affd640 1 -- 172.21.15.37:0/947810150 learned_addr learned my addr 172.21.15.37:0/947810150 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:20.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f040bfff640 1 -- 172.21.15.37:0/947810150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f040c15b6b0 msgr2=0x7f040c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:20.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f040bfff640 1 --2- 172.21.15.37:0/947810150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f040c15b6b0 0x7f040c0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f040bfff640 1 -- 172.21.15.37:0/947810150 >> v1:172.21.15.203:6789/0 conn(0x7f040c0cf590 legacy=0x7f040c15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.823+0000 7f040bfff640 1 -- 172.21.15.37:0/947810150 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c0cc000 con 0x7f040c073010 2024-04-25T09:01:20.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f040bfff640 1 --2- 172.21.15.37:0/947810150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f040c073010 0x7f040c0cf9f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0404009d90 tx=0x7f040401c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=112a7d17344b40e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f040a7fc640 1 -- 172.21.15.37:0/947810150 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04040b2070 con 0x7f040c073010 2024-04-25T09:01:20.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f040a7fc640 1 -- 172.21.15.37:0/947810150 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04040a5920 con 0x7f040c073010 2024-04-25T09:01:20.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f040a7fc640 1 -- 172.21.15.37:0/947810150 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04040a5cf0 con 0x7f040c073010 2024-04-25T09:01:20.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f0411bd9640 1 -- 172.21.15.37:0/947810150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f040c073010 msgr2=0x7f040c0cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f0411bd9640 1 --2- 172.21.15.37:0/947810150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f040c073010 0x7f040c0cf9f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0404009d90 tx=0x7f040401c940 comp rx=0 tx=0).stop 2024-04-25T09:01:20.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f0411bd9640 1 -- 172.21.15.37:0/947810150 shutdown_connections 2024-04-25T09:01:20.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f0411bd9640 1 --2- 172.21.15.37:0/947810150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f040c073010 0x7f040c0cf9f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f0411bd9640 1 --2- 172.21.15.37:0/947810150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f040c15b6b0 0x7f040c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f0411bd9640 1 -- 172.21.15.37:0/947810150 >> 172.21.15.37:0/947810150 conn(0x7f040c06ae60 msgr2=0x7f040c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f0411bd9640 1 -- 172.21.15.37:0/947810150 shutdown_connections 2024-04-25T09:01:20.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f0411bd9640 1 -- 172.21.15.37:0/947810150 wait complete. 2024-04-25T09:01:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.827+0000 7f0411bd9640 1 Processor -- start 2024-04-25T09:01:20.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f0411bd9640 1 -- start start 2024-04-25T09:01:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f0411bd9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f040c073010 0x7f040c098a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f0411bd9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f040c0cf590 0x7f040c150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f0411bd9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f040c15b6b0 0x7f040c156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f0411bd9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0ca900 con 0x7f040c15b6b0 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f0411bd9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0cba30 con 0x7f040c073010 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f0411bd9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040c0cadb0 con 0x7f040c0cf590 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f040bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f040c15b6b0 0x7f040c156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f040bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f040c15b6b0 0x7f040c156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60500/0 (socket says 172.21.15.37:60500) 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f040bfff640 1 -- 172.21.15.37:0/4278745552 learned_addr learned my addr 172.21.15.37:0/4278745552 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f040affd640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f040c0cf590 0x7f040c150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f040b7fe640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f040c073010 0x7f040c098a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f040bfff640 1 -- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f040c0cf590 msgr2=0x7f040c150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f040bfff640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f040c0cf590 0x7f040c150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f040bfff640 1 -- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f040c073010 msgr2=0x7f040c098a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f040bfff640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f040c073010 0x7f040c098a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f040bfff640 1 -- 172.21.15.37:0/4278745552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c151820 con 0x7f040c15b6b0 2024-04-25T09:01:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f040bfff640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f040c15b6b0 0x7f040c156570 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f0404009d90 tx=0x7f04040196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f03f3fff640 1 -- 172.21.15.37:0/4278745552 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04040b2050 con 0x7f040c15b6b0 2024-04-25T09:01:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f03f3fff640 1 -- 172.21.15.37:0/4278745552 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04040be070 con 0x7f040c15b6b0 2024-04-25T09:01:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f03f3fff640 1 -- 172.21.15.37:0/4278745552 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04040ada90 con 0x7f040c15b6b0 2024-04-25T09:01:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f0411bd9640 1 -- 172.21.15.37:0/4278745552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f040c151b00 con 0x7f040c15b6b0 2024-04-25T09:01:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f0411bd9640 1 -- 172.21.15.37:0/4278745552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f040c1520c0 con 0x7f040c15b6b0 2024-04-25T09:01:20.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f0411bd9640 1 -- 172.21.15.37:0/4278745552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040c04efa0 con 0x7f040c15b6b0 2024-04-25T09:01:20.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.831+0000 7f03f3fff640 1 -- 172.21.15.37:0/4278745552 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f04040adc30 con 0x7f040c15b6b0 2024-04-25T09:01:20.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.835+0000 7f03f3fff640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03f8066f90 0x7f03f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.835+0000 7f03f3fff640 1 -- 172.21.15.37:0/4278745552 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f04041203b0 con 0x7f040c15b6b0 2024-04-25T09:01:20.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.835+0000 7f040b7fe640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03f8066f90 0x7f03f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.835+0000 7f040b7fe640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03f8066f90 0x7f03f8069450 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009ac0 tx=0x7f03fc009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.839+0000 7f03f3fff640 1 -- 172.21.15.37:0/4278745552 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0404016560 con 0x7f040c15b6b0 2024-04-25T09:01:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 -- 172.21.15.37:0/4278745552 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f040c0cb530 con 0x7f040c15b6b0 2024-04-25T09:01:20.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f03f3fff640 1 -- 172.21.15.37:0/4278745552 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v99) v1 ==== 104+0+11 (secure 0 0 0) 0x7f04040f0a20 con 0x7f040c15b6b0 2024-04-25T09:01:20.986 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:01:20.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 -- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03f8066f90 msgr2=0x7f03f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03f8066f90 0x7f03f8069450 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009ac0 tx=0x7f03fc009190 comp rx=0 tx=0).stop 2024-04-25T09:01:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 -- 172.21.15.37:0/4278745552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f040c15b6b0 msgr2=0x7f040c156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f040c15b6b0 0x7f040c156570 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f0404009d90 tx=0x7f04040196e0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 -- 172.21.15.37:0/4278745552 shutdown_connections 2024-04-25T09:01:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f03f8066f90 0x7f03f8069450 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f040c15b6b0 0x7f040c156570 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f040c0cf590 0x7f040c150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 --2- 172.21.15.37:0/4278745552 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f040c073010 0x7f040c098a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 -- 172.21.15.37:0/4278745552 >> 172.21.15.37:0/4278745552 conn(0x7f040c06ae60 msgr2=0x7f040c0b3630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 -- 172.21.15.37:0/4278745552 shutdown_connections 2024-04-25T09:01:20.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:20.999+0000 7f0411bd9640 1 -- 172.21.15.37:0/4278745552 wait complete. 2024-04-25T09:01:21.007 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:01:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a77dc640 1 Processor -- start 2024-04-25T09:01:21.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a77dc640 1 -- start start 2024-04-25T09:01:21.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a77dc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc2a015b6b0 0x7fc2a00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a77dc640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a015ff50 con 0x7fc2a00cf590 2024-04-25T09:01:21.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a77dc640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a00cbd20 con 0x7fc2a0073010 2024-04-25T09:01:21.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a77dc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a015ba90 con 0x7fc2a015b6b0 2024-04-25T09:01:21.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a5578640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc2a015b6b0 0x7fc2a00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a4d77640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc2a00cf590 0x7fc2a015f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48822/0 (socket says 172.21.15.37:48822) 2024-04-25T09:01:21.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a4d77640 1 -- 172.21.15.37:0/2096250160 learned_addr learned my addr 172.21.15.37:0/2096250160 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:21.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a5578640 1 -- 172.21.15.37:0/2096250160 >> v1:172.21.15.203:6789/0 conn(0x7fc2a0073010 legacy=0x7fc2a015f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:21.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a5578640 1 -- 172.21.15.37:0/2096250160 >> v1:172.21.15.37:6789/0 conn(0x7fc2a00cf590 legacy=0x7fc2a015f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc297fff640 1 -- 172.21.15.37:0/2096250160 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899868483 0 0) 0x7fc2a015ff50 con 0x7fc2a00cf590 2024-04-25T09:01:21.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.147+0000 7fc2a5578640 1 -- 172.21.15.37:0/2096250160 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a00cc000 con 0x7fc2a015b6b0 2024-04-25T09:01:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5578640 1 --2- 172.21.15.37:0/2096250160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a015b6b0 0x7fc2a00ca3c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0097c0 tx=0x7fc29c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f563bce1787fcb87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc297fff640 1 -- 172.21.15.37:0/2096250160 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0a7070 con 0x7fc2a015b6b0 2024-04-25T09:01:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc297fff640 1 -- 172.21.15.37:0/2096250160 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc29c0b0070 con 0x7fc2a015b6b0 2024-04-25T09:01:21.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc297fff640 1 -- 172.21.15.37:0/2096250160 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29c0ab890 con 0x7fc2a015b6b0 2024-04-25T09:01:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 -- 172.21.15.37:0/2096250160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a015b6b0 msgr2=0x7fc2a00ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 --2- 172.21.15.37:0/2096250160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a015b6b0 0x7fc2a00ca3c0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0097c0 tx=0x7fc29c01c470 comp rx=0 tx=0).stop 2024-04-25T09:01:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 -- 172.21.15.37:0/2096250160 shutdown_connections 2024-04-25T09:01:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 --2- 172.21.15.37:0/2096250160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a015b6b0 0x7fc2a00ca3c0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 -- 172.21.15.37:0/2096250160 >> 172.21.15.37:0/2096250160 conn(0x7fc2a006ae60 msgr2=0x7fc2a006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 -- 172.21.15.37:0/2096250160 shutdown_connections 2024-04-25T09:01:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 -- 172.21.15.37:0/2096250160 wait complete. 2024-04-25T09:01:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 Processor -- start 2024-04-25T09:01:21.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 -- start start 2024-04-25T09:01:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2a0073010 0x7fc2a01525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00cf590 0x7fc2a0095e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a015b6b0 0x7fc2a0096370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a00ca900 con 0x7fc2a015b6b0 2024-04-25T09:01:21.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a00cba30 con 0x7fc2a0073010 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a00cfdf0 con 0x7fc2a00cf590 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5d79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a015b6b0 0x7fc2a0096370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5d79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a015b6b0 0x7fc2a0096370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60530/0 (socket says 172.21.15.37:60530) 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5d79640 1 -- 172.21.15.37:0/459797988 learned_addr learned my addr 172.21.15.37:0/459797988 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5578640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2a0073010 0x7fc2a01525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a4d77640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00cf590 0x7fc2a0095e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5d79640 1 -- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00cf590 msgr2=0x7fc2a0095e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5d79640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00cf590 0x7fc2a0095e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5d79640 1 -- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2a0073010 msgr2=0x7fc2a01525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5d79640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2a0073010 0x7fc2a01525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5d79640 1 -- 172.21.15.37:0/459797988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a01f5400 con 0x7fc2a015b6b0 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5578640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2a0073010 0x7fc2a01525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a5d79640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a015b6b0 0x7fc2a0096370 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c730 tx=0x7fc29800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc295ffb640 1 -- 172.21.15.37:0/459797988 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298003b80 con 0x7fc2a015b6b0 2024-04-25T09:01:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc295ffb640 1 -- 172.21.15.37:0/459797988 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc298003d20 con 0x7fc2a015b6b0 2024-04-25T09:01:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 -- 172.21.15.37:0/459797988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a01f56e0 con 0x7fc2a015b6b0 2024-04-25T09:01:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc295ffb640 1 -- 172.21.15.37:0/459797988 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc29800aa30 con 0x7fc2a015b6b0 2024-04-25T09:01:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.151+0000 7fc2a77dc640 1 -- 172.21.15.37:0/459797988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a01f5ca0 con 0x7fc2a015b6b0 2024-04-25T09:01:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.155+0000 7fc2a77dc640 1 -- 172.21.15.37:0/459797988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a004efa0 con 0x7fc2a015b6b0 2024-04-25T09:01:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.155+0000 7fc295ffb640 1 -- 172.21.15.37:0/459797988 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc29800abd0 con 0x7fc2a015b6b0 2024-04-25T09:01:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.155+0000 7fc295ffb640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc284066f60 0x7fc284069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.155+0000 7fc295ffb640 1 -- 172.21.15.37:0/459797988 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fc298085180 con 0x7fc2a015b6b0 2024-04-25T09:01:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.155+0000 7fc2a5578640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc284066f60 0x7fc284069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.155+0000 7fc2a5578640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc284066f60 0x7fc284069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0af000 tx=0x7fc29c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.159+0000 7fc295ffb640 1 -- 172.21.15.37:0/459797988 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc298055ee0 con 0x7fc2a015b6b0 2024-04-25T09:01:21.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.311+0000 7fc2a77dc640 1 -- 172.21.15.37:0/459797988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2a00b4100 con 0x7fc2a015b6b0 2024-04-25T09:01:21.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc295ffb640 1 -- 172.21.15.37:0/459797988 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v99) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc2980557e0 con 0x7fc2a015b6b0 2024-04-25T09:01:21.302 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:01:21.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 -- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc284066f60 msgr2=0x7fc284069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc284066f60 0x7fc284069420 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0af000 tx=0x7fc29c008040 comp rx=0 tx=0).stop 2024-04-25T09:01:21.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 -- 172.21.15.37:0/459797988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a015b6b0 msgr2=0x7fc2a0096370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a015b6b0 0x7fc2a0096370 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fc29800c730 tx=0x7fc29800cbf0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 -- 172.21.15.37:0/459797988 shutdown_connections 2024-04-25T09:01:21.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc284066f60 0x7fc284069420 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2a015b6b0 0x7fc2a0096370 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc2a00cf590 0x7fc2a0095e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 --2- 172.21.15.37:0/459797988 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc2a0073010 0x7fc2a01525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 -- 172.21.15.37:0/459797988 >> 172.21.15.37:0/459797988 conn(0x7fc2a006ae60 msgr2=0x7fc2a0152c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 -- 172.21.15.37:0/459797988 shutdown_connections 2024-04-25T09:01:21.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.315+0000 7fc2a77dc640 1 -- 172.21.15.37:0/459797988 wait complete. 2024-04-25T09:01:21.320 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:21.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0db913640 1 Processor -- start 2024-04-25T09:01:21.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0db913640 1 -- start start 2024-04-25T09:01:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0db913640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd0d406ddb0 0x7fd0d406f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0db913640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d406e2b0 con 0x7fd0d4161390 2024-04-25T09:01:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0db913640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d406e590 con 0x7fd0d4158040 2024-04-25T09:01:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0db913640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d406eec0 con 0x7fd0d406ddb0 2024-04-25T09:01:21.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0d8eae640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd0d406ddb0 0x7fd0d406f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0d9eb0640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd0d4161390 0x7fd0d406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48832/0 (socket says 172.21.15.37:48832) 2024-04-25T09:01:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0d9eb0640 1 -- 172.21.15.37:0/3516671044 learned_addr learned my addr 172.21.15.37:0/3516671044 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0d8eae640 1 -- 172.21.15.37:0/3516671044 >> v1:172.21.15.203:6789/0 conn(0x7fd0d4158040 legacy=0x7fd0d40b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0d8eae640 1 -- 172.21.15.37:0/3516671044 >> v1:172.21.15.37:6789/0 conn(0x7fd0d4161390 legacy=0x7fd0d406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0d8eae640 1 -- 172.21.15.37:0/3516671044 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d406e870 con 0x7fd0d406ddb0 2024-04-25T09:01:21.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.455+0000 7fd0cbfff640 1 -- 172.21.15.37:0/3516671044 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561615218 0 0) 0x7fd0d406e2b0 con 0x7fd0d4161390 2024-04-25T09:01:21.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d8eae640 1 --2- 172.21.15.37:0/3516671044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d406ddb0 0x7fd0d406f350 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40097c0 tx=0x7fd0c401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf9b492b1994225 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0cbfff640 1 -- 172.21.15.37:0/3516671044 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40b1070 con 0x7fd0d406ddb0 2024-04-25T09:01:21.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0cbfff640 1 -- 172.21.15.37:0/3516671044 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c40ac5f0 con 0x7fd0d406ddb0 2024-04-25T09:01:21.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0cbfff640 1 -- 172.21.15.37:0/3516671044 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40ac9c0 con 0x7fd0d406ddb0 2024-04-25T09:01:21.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 -- 172.21.15.37:0/3516671044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d406ddb0 msgr2=0x7fd0d406f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 --2- 172.21.15.37:0/3516671044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d406ddb0 0x7fd0d406f350 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fd0c40097c0 tx=0x7fd0c401c430 comp rx=0 tx=0).stop 2024-04-25T09:01:21.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 -- 172.21.15.37:0/3516671044 shutdown_connections 2024-04-25T09:01:21.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 --2- 172.21.15.37:0/3516671044 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d406ddb0 0x7fd0d406f350 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 -- 172.21.15.37:0/3516671044 >> 172.21.15.37:0/3516671044 conn(0x7fd0d40cfb50 msgr2=0x7fd0d40cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 -- 172.21.15.37:0/3516671044 shutdown_connections 2024-04-25T09:01:21.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 -- 172.21.15.37:0/3516671044 wait complete. 2024-04-25T09:01:21.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 Processor -- start 2024-04-25T09:01:21.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 -- start start 2024-04-25T09:01:21.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0d406ddb0 0x7fd0d41570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0d4158040 0x7fd0d4143a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d4161390 0x7fd0d40b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d416bad0 con 0x7fd0d4161390 2024-04-25T09:01:21.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d416b950 con 0x7fd0d4158040 2024-04-25T09:01:21.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d416bc50 con 0x7fd0d406ddb0 2024-04-25T09:01:21.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d9eb0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d4161390 0x7fd0d40b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d9eb0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d4161390 0x7fd0d40b0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60542/0 (socket says 172.21.15.37:60542) 2024-04-25T09:01:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d9eb0640 1 -- 172.21.15.37:0/2075170618 learned_addr learned my addr 172.21.15.37:0/2075170618 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d8eae640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0d4158040 0x7fd0d4143a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d96af640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0d406ddb0 0x7fd0d41570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d9eb0640 1 -- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0d406ddb0 msgr2=0x7fd0d41570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d9eb0640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0d406ddb0 0x7fd0d41570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d9eb0640 1 -- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0d4158040 msgr2=0x7fd0d4143a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d9eb0640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0d4158040 0x7fd0d4143a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d9eb0640 1 -- 172.21.15.37:0/2075170618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d40b0fe0 con 0x7fd0d4161390 2024-04-25T09:01:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d96af640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0d406ddb0 0x7fd0d41570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:21.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0d9eb0640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d4161390 0x7fd0d40b0aa0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c7d0 tx=0x7fd0cc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0c9ffb640 1 -- 172.21.15.37:0/2075170618 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc018070 con 0x7fd0d4161390 2024-04-25T09:01:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0c9ffb640 1 -- 172.21.15.37:0/2075170618 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0cc00a640 con 0x7fd0d4161390 2024-04-25T09:01:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 -- 172.21.15.37:0/2075170618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d40b1530 con 0x7fd0d4161390 2024-04-25T09:01:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0c9ffb640 1 -- 172.21.15.37:0/2075170618 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc00aa70 con 0x7fd0d4161390 2024-04-25T09:01:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.459+0000 7fd0db913640 1 -- 172.21.15.37:0/2075170618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d415de40 con 0x7fd0d4161390 2024-04-25T09:01:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.463+0000 7fd0db913640 1 -- 172.21.15.37:0/2075170618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d404ef30 con 0x7fd0d4161390 2024-04-25T09:01:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.463+0000 7fd0c9ffb640 1 -- 172.21.15.37:0/2075170618 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0cc01b070 con 0x7fd0d4161390 2024-04-25T09:01:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.463+0000 7fd0c9ffb640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0b8066f60 0x7fd0b8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.463+0000 7fd0c9ffb640 1 -- 172.21.15.37:0/2075170618 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd0cc084910 con 0x7fd0d4161390 2024-04-25T09:01:21.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.463+0000 7fd0d96af640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0b8066f60 0x7fd0b8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.463+0000 7fd0d96af640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0b8066f60 0x7fd0b8069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0003e40 tx=0x7fd0d0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.467+0000 7fd0c9ffb640 1 -- 172.21.15.37:0/2075170618 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd0cc055660 con 0x7fd0d4161390 2024-04-25T09:01:21.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.659+0000 7fd0db913640 1 -- 172.21.15.37:0/2075170618 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0d4066f60 con 0x7fd0d4161390 2024-04-25T09:01:21.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.659+0000 7fd0c9ffb640 1 -- 172.21.15.37:0/2075170618 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fd0cc054f60 con 0x7fd0d4161390 2024-04-25T09:01:21.649 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:21.649 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T09:00:41.164474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/3366017865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3366017865},{"type":"v1","addr":"172.21.15.37:6835","nonce":3366017865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:21.649 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 73 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 -- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0b8066f60 msgr2=0x7fd0b8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0b8066f60 0x7fd0b8069420 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0003e40 tx=0x7fd0d0009040 comp rx=0 tx=0).stop 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 -- 172.21.15.37:0/2075170618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d4161390 msgr2=0x7fd0d40b0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d4161390 0x7fd0d40b0aa0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00c7d0 tx=0x7fd0cc00cc90 comp rx=0 tx=0).stop 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 -- 172.21.15.37:0/2075170618 shutdown_connections 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0b8066f60 0x7fd0b8069420 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0d4161390 0x7fd0d40b0aa0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0d4158040 0x7fd0d4143a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 --2- 172.21.15.37:0/2075170618 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0d406ddb0 0x7fd0d41570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 -- 172.21.15.37:0/2075170618 >> 172.21.15.37:0/2075170618 conn(0x7fd0d40cfb50 msgr2=0x7fd0d415c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 -- 172.21.15.37:0/2075170618 shutdown_connections 2024-04-25T09:01:21.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.663+0000 7fd0db913640 1 -- 172.21.15.37:0/2075170618 wait complete. 2024-04-25T09:01:21.669 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:21.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ec220640 1 Processor -- start 2024-04-25T09:01:21.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ec220640 1 -- start start 2024-04-25T09:01:21.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ec220640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd4e415a700 0x7fd4e415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ec220640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd4e40b24a0 0x7fd4e40b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ec220640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e40b2090 con 0x7fd4e40b3f50 2024-04-25T09:01:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ec220640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e415f200 con 0x7fd4e40b24a0 2024-04-25T09:01:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ec220640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e40b43e0 con 0x7fd4e415a700 2024-04-25T09:01:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ea7bd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd4e40b24a0 0x7fd4e40b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ea7bd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd4e40b24a0 0x7fd4e40b28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60548/0 (socket says 172.21.15.37:60548) 2024-04-25T09:01:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ea7bd640 1 -- 172.21.15.37:0/4044796244 learned_addr learned my addr 172.21.15.37:0/4044796244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4e9fbc640 1 --2- 172.21.15.37:0/4044796244 >> v2:172.21.15.203:3300/0 conn(0x7fd4e415a700 0x7fd4e415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ea7bd640 1 -- 172.21.15.37:0/4044796244 >> v2:172.21.15.203:3300/0 conn(0x7fd4e415a700 msgr2=0x7fd4e415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ea7bd640 1 --2- 172.21.15.37:0/4044796244 >> v2:172.21.15.203:3300/0 conn(0x7fd4e415a700 0x7fd4e415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ea7bd640 1 -- 172.21.15.37:0/4044796244 >> v1:172.21.15.203:6789/0 conn(0x7fd4e40b3f50 legacy=0x7fd4e40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.807+0000 7fd4ea7bd640 1 -- 172.21.15.37:0/4044796244 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e40b46c0 con 0x7fd4e40b24a0 2024-04-25T09:01:21.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ea7bd640 1 --2- 172.21.15.37:0/4044796244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e40b24a0 0x7fd4e40b28f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fd4e406bd20 tx=0x7fd4dc01eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a9393547334cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4e8fba640 1 -- 172.21.15.37:0/4044796244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0a78a0 con 0x7fd4e40b24a0 2024-04-25T09:01:21.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4e8fba640 1 -- 172.21.15.37:0/4044796244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc0a7a40 con 0x7fd4e40b24a0 2024-04-25T09:01:21.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4e8fba640 1 -- 172.21.15.37:0/4044796244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0a7e10 con 0x7fd4e40b24a0 2024-04-25T09:01:21.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 -- 172.21.15.37:0/4044796244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e40b24a0 msgr2=0x7fd4e40b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 --2- 172.21.15.37:0/4044796244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e40b24a0 0x7fd4e40b28f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fd4e406bd20 tx=0x7fd4dc01eae0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 -- 172.21.15.37:0/4044796244 shutdown_connections 2024-04-25T09:01:21.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 --2- 172.21.15.37:0/4044796244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e40b24a0 0x7fd4e40b28f0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 --2- 172.21.15.37:0/4044796244 >> v2:172.21.15.203:3300/0 conn(0x7fd4e415a700 0x7fd4e415ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 -- 172.21.15.37:0/4044796244 >> 172.21.15.37:0/4044796244 conn(0x7fd4e40b11a0 msgr2=0x7fd4e40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 -- 172.21.15.37:0/4044796244 shutdown_connections 2024-04-25T09:01:21.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 -- 172.21.15.37:0/4044796244 wait complete. 2024-04-25T09:01:21.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 Processor -- start 2024-04-25T09:01:21.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 -- start start 2024-04-25T09:01:21.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e40b24a0 0x7fd4e4155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e415a700 0x7fd4e4164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4e41f2480 0x7fd4e41f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e40cd310 con 0x7fd4e415a700 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e40ce600 con 0x7fd4e41f2480 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ec220640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e40cd490 con 0x7fd4e40b24a0 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4e97bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e415a700 0x7fd4e4164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4e97bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e415a700 0x7fd4e4164cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60556/0 (socket says 172.21.15.37:60556) 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4e97bb640 1 -- 172.21.15.37:0/201039626 learned_addr learned my addr 172.21.15.37:0/201039626 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.811+0000 7fd4ea7bd640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4e41f2480 0x7fd4e41f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4e9fbc640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e40b24a0 0x7fd4e4155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4e97bb640 1 -- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e40b24a0 msgr2=0x7fd4e4155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4e97bb640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e40b24a0 0x7fd4e4155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4e97bb640 1 -- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4e41f2480 msgr2=0x7fd4e41f2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4e97bb640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4e41f2480 0x7fd4e41f2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4e97bb640 1 -- 172.21.15.37:0/201039626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e40d02e0 con 0x7fd4e415a700 2024-04-25T09:01:21.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4e97bb640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e415a700 0x7fd4e4164cd0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4007fd0 tx=0x7fd4d400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4da7fc640 1 -- 172.21.15.37:0/201039626 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4017070 con 0x7fd4e415a700 2024-04-25T09:01:21.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4da7fc640 1 -- 172.21.15.37:0/201039626 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d40052d0 con 0x7fd4e415a700 2024-04-25T09:01:21.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4ec220640 1 -- 172.21.15.37:0/201039626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e4160580 con 0x7fd4e415a700 2024-04-25T09:01:21.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4da7fc640 1 -- 172.21.15.37:0/201039626 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d4003840 con 0x7fd4e415a700 2024-04-25T09:01:21.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4ec220640 1 -- 172.21.15.37:0/201039626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e4160b40 con 0x7fd4e415a700 2024-04-25T09:01:21.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4ec220640 1 -- 172.21.15.37:0/201039626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e404efa0 con 0x7fd4e415a700 2024-04-25T09:01:21.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4da7fc640 1 -- 172.21.15.37:0/201039626 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd4d401a070 con 0x7fd4e415a700 2024-04-25T09:01:21.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4da7fc640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4c8066f90 0x7fd4c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4da7fc640 1 -- 172.21.15.37:0/201039626 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7fd4d40845a0 con 0x7fd4e415a700 2024-04-25T09:01:21.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4e9fbc640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4c8066f90 0x7fd4c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.815+0000 7fd4e9fbc640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4c8066f90 0x7fd4c8069450 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009af0 tx=0x7fd4e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.819+0000 7fd4da7fc640 1 -- 172.21.15.37:0/201039626 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd4d4005480 con 0x7fd4e415a700 2024-04-25T09:01:21.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.979+0000 7fd4ec220640 1 -- 172.21.15.37:0/201039626 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4e406f260 con 0x7fd4e415a700 2024-04-25T09:01:21.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.979+0000 7fd4da7fc640 1 -- 172.21.15.37:0/201039626 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17299 (secure 0 0 0) 0x7fd4d4054cb0 con 0x7fd4e415a700 2024-04-25T09:01:21.968 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:21.968 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:00:38.139693+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:50.512578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:51.848880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1462327322":"2024-04-26T09:00:38.139659+0000","172.21.15.37:6834/1462327322":"2024-04-26T09:00:38.139659+0000","172.21.15.37:6835/354285428":"2024-04-26T08:59:17.095322+0000","172.21.15.37:6834/354285428":"2024-04-26T08:59:17.095322+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:21.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 -- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4c8066f90 msgr2=0x7fd4c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4c8066f90 0x7fd4c8069450 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009af0 tx=0x7fd4e0008040 comp rx=0 tx=0).stop 2024-04-25T09:01:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 -- 172.21.15.37:0/201039626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e415a700 msgr2=0x7fd4e4164cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e415a700 0x7fd4e4164cd0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4007fd0 tx=0x7fd4d400c890 comp rx=0 tx=0).stop 2024-04-25T09:01:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 -- 172.21.15.37:0/201039626 shutdown_connections 2024-04-25T09:01:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd4c8066f90 0x7fd4c8069450 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd4e41f2480 0x7fd4e41f2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4e415a700 0x7fd4e4164cd0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 --2- 172.21.15.37:0/201039626 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd4e40b24a0 0x7fd4e4155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 -- 172.21.15.37:0/201039626 >> 172.21.15.37:0/201039626 conn(0x7fd4e40b11a0 msgr2=0x7fd4e40ce910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 -- 172.21.15.37:0/201039626 shutdown_connections 2024-04-25T09:01:21.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:21.983+0000 7fd4ec220640 1 -- 172.21.15.37:0/201039626 wait complete. 2024-04-25T09:01:21.987 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:01:21.987 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:22.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.111+0000 7f1f15e19640 1 Processor -- start 2024-04-25T09:01:22.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.111+0000 7f1f15e19640 1 -- start start 2024-04-25T09:01:22.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f15e19640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100ce510 0x7f1f100ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f15e19640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1f101451f0 0x7f1f10142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f15e19640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f100cee30 con 0x7f1f10144ca0 2024-04-25T09:01:22.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f15e19640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f100cc2b0 con 0x7f1f101451f0 2024-04-25T09:01:22.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f15e19640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1006e290 con 0x7f1f100ce510 2024-04-25T09:01:22.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0f7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100ce510 0x7f1f100ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0ffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1f101451f0 0x7f1f10142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0effd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f1f10144ca0 0x7f1f1006e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:52038/0 (socket says 172.21.15.37:52038) 2024-04-25T09:01:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0effd640 1 -- 172.21.15.37:0/2582058179 learned_addr learned my addr 172.21.15.37:0/2582058179 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0ffff640 1 -- 172.21.15.37:0/2582058179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100ce510 msgr2=0x7f1f100ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0ffff640 1 --2- 172.21.15.37:0/2582058179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100ce510 0x7f1f100ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0ffff640 1 -- 172.21.15.37:0/2582058179 >> v1:172.21.15.203:6789/0 conn(0x7f1f10144ca0 legacy=0x7f1f1006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0ffff640 1 -- 172.21.15.37:0/2582058179 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100ccc70 con 0x7f1f101451f0 2024-04-25T09:01:22.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0ffff640 1 --2- 172.21.15.37:0/2582058179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f101451f0 0x7f1f10142680 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009d90 tx=0x7f1f0801f150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17970dee9dcb6c4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0e7fc640 1 -- 172.21.15.37:0/2582058179 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080b4070 con 0x7f1f101451f0 2024-04-25T09:01:22.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0e7fc640 1 -- 172.21.15.37:0/2582058179 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f080af600 con 0x7f1f101451f0 2024-04-25T09:01:22.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.115+0000 7f1f0e7fc640 1 -- 172.21.15.37:0/2582058179 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080af9d0 con 0x7f1f101451f0 2024-04-25T09:01:22.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- 172.21.15.37:0/2582058179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f101451f0 msgr2=0x7f1f10142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 --2- 172.21.15.37:0/2582058179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f101451f0 0x7f1f10142680 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009d90 tx=0x7f1f0801f150 comp rx=0 tx=0).stop 2024-04-25T09:01:22.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- 172.21.15.37:0/2582058179 shutdown_connections 2024-04-25T09:01:22.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 --2- 172.21.15.37:0/2582058179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f101451f0 0x7f1f10142680 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 --2- 172.21.15.37:0/2582058179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100ce510 0x7f1f100ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- 172.21.15.37:0/2582058179 >> 172.21.15.37:0/2582058179 conn(0x7f1f100b11a0 msgr2=0x7f1f100b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:22.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- 172.21.15.37:0/2582058179 shutdown_connections 2024-04-25T09:01:22.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- 172.21.15.37:0/2582058179 wait complete. 2024-04-25T09:01:22.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 Processor -- start 2024-04-25T09:01:22.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- start start 2024-04-25T09:01:22.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100ce510 0x7f1f1015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f10144ca0 0x7f1f100ca600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f101451f0 0x7f1f100cab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1006ee30 con 0x7f1f101451f0 2024-04-25T09:01:22.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1006df20 con 0x7f1f10144ca0 2024-04-25T09:01:22.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f100cc900 con 0x7f1f100ce510 2024-04-25T09:01:22.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f101451f0 0x7f1f100cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f101451f0 0x7f1f100cab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60570/0 (socket says 172.21.15.37:60570) 2024-04-25T09:01:22.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f0ffff640 1 -- 172.21.15.37:0/1628662623 learned_addr learned my addr 172.21.15.37:0/1628662623 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:22.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f0effd640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f10144ca0 0x7f1f100ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f0f7fe640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100ce510 0x7f1f1015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f0ffff640 1 -- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100ce510 msgr2=0x7f1f1015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f0ffff640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100ce510 0x7f1f1015b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f0ffff640 1 -- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f10144ca0 msgr2=0x7f1f100ca600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f0ffff640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f10144ca0 0x7f1f100ca600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f0ffff640 1 -- 172.21.15.37:0/1628662623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f100b1620 con 0x7f1f101451f0 2024-04-25T09:01:22.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f0ffff640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f101451f0 0x7f1f100cab40 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f1f080b2040 tx=0x7f1f0801bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1ef7fff640 1 -- 172.21.15.37:0/1628662623 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f080b4040 con 0x7f1f101451f0 2024-04-25T09:01:22.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1ef7fff640 1 -- 172.21.15.37:0/1628662623 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f080c1070 con 0x7f1f101451f0 2024-04-25T09:01:22.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- 172.21.15.37:0/1628662623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1015b900 con 0x7f1f101451f0 2024-04-25T09:01:22.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1ef7fff640 1 -- 172.21.15.37:0/1628662623 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08009450 con 0x7f1f101451f0 2024-04-25T09:01:22.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- 172.21.15.37:0/1628662623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f101e33d0 con 0x7f1f101451f0 2024-04-25T09:01:22.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.119+0000 7f1f15e19640 1 -- 172.21.15.37:0/1628662623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f1004efa0 con 0x7f1f101451f0 2024-04-25T09:01:22.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.123+0000 7f1ef7fff640 1 -- 172.21.15.37:0/1628662623 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1f080ba6b0 con 0x7f1f101451f0 2024-04-25T09:01:22.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.123+0000 7f1ef7fff640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1efc066f60 0x7f1efc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.123+0000 7f1ef7fff640 1 -- 172.21.15.37:0/1628662623 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f1f08124680 con 0x7f1f101451f0 2024-04-25T09:01:22.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.123+0000 7f1f0f7fe640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1efc066f60 0x7f1efc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.123+0000 7f1f0f7fe640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1efc066f60 0x7f1efc069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009a30 tx=0x7f1f00008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.127+0000 7f1ef7fff640 1 -- 172.21.15.37:0/1628662623 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1f08018cb0 con 0x7f1f101451f0 2024-04-25T09:01:22.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.310+0000 7f1f15e19640 1 -- 172.21.15.37:0/1628662623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f10071a60 con 0x7f1f101451f0 2024-04-25T09:01:22.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1ef7fff640 1 -- 172.21.15.37:0/1628662623 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f1f080f5520 con 0x7f1f101451f0 2024-04-25T09:01:22.302 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:22.302 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T09:00:41.164474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/3366017865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3366017865},{"type":"v1","addr":"172.21.15.37:6835","nonce":3366017865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:22.302 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 73 2024-04-25T09:01:22.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 -- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1efc066f60 msgr2=0x7f1efc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1efc066f60 0x7f1efc069420 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009a30 tx=0x7f1f00008040 comp rx=0 tx=0).stop 2024-04-25T09:01:22.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 -- 172.21.15.37:0/1628662623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f101451f0 msgr2=0x7f1f100cab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f101451f0 0x7f1f100cab40 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f1f080b2040 tx=0x7f1f0801bef0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 -- 172.21.15.37:0/1628662623 shutdown_connections 2024-04-25T09:01:22.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1efc066f60 0x7f1efc069420 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f101451f0 0x7f1f100cab40 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f10144ca0 0x7f1f100ca600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 --2- 172.21.15.37:0/1628662623 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f100ce510 0x7f1f1015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 -- 172.21.15.37:0/1628662623 >> 172.21.15.37:0/1628662623 conn(0x7f1f100b11a0 msgr2=0x7f1f101429b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:22.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 -- 172.21.15.37:0/1628662623 shutdown_connections 2024-04-25T09:01:22.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.314+0000 7f1f15e19640 1 -- 172.21.15.37:0/1628662623 wait complete. 2024-04-25T09:01:22.319 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:22.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.438+0000 7f2f4e2f1640 1 Processor -- start 2024-04-25T09:01:22.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.438+0000 7f2f4e2f1640 1 -- start start 2024-04-25T09:01:22.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4e2f1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2f480ce510 0x7f2f480ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4e2f1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48144ca0 0x7f2f4806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4e2f1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2f4806ec60 0x7f2f48142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4e2f1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f480cc2b0 con 0x7f2f4806ec60 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4e2f1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4806e290 con 0x7f2f480ce510 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4e2f1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480cee30 con 0x7f2f48144ca0 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4c88e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2f4806ec60 0x7f2f48142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4c88e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2f4806ec60 0x7f2f48142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60572/0 (socket says 172.21.15.37:60572) 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4c88e640 1 -- 172.21.15.37:0/2780897413 learned_addr learned my addr 172.21.15.37:0/2780897413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f47fff640 1 --2- 172.21.15.37:0/2780897413 >> v2:172.21.15.203:3300/0 conn(0x7f2f480ce510 0x7f2f480ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f477fe640 1 --2- 172.21.15.37:0/2780897413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48144ca0 0x7f2f4806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4c88e640 1 -- 172.21.15.37:0/2780897413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48144ca0 msgr2=0x7f2f4806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4c88e640 1 --2- 172.21.15.37:0/2780897413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48144ca0 0x7f2f4806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4c88e640 1 -- 172.21.15.37:0/2780897413 >> v2:172.21.15.203:3300/0 conn(0x7f2f480ce510 msgr2=0x7f2f480ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:22.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4c88e640 1 --2- 172.21.15.37:0/2780897413 >> v2:172.21.15.203:3300/0 conn(0x7f2f480ce510 0x7f2f480ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4c88e640 1 -- 172.21.15.37:0/2780897413 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480ccc70 con 0x7f2f4806ec60 2024-04-25T09:01:22.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f477fe640 1 --2- 172.21.15.37:0/2780897413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48144ca0 0x7f2f4806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:22.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f4c88e640 1 --2- 172.21.15.37:0/2780897413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f4806ec60 0x7f2f48142680 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f2f48144c50 tx=0x7f2f4001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2ef196573297fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f46ffd640 1 -- 172.21.15.37:0/2780897413 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f400b5070 con 0x7f2f4806ec60 2024-04-25T09:01:22.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f46ffd640 1 -- 172.21.15.37:0/2780897413 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f400a89d0 con 0x7f2f4806ec60 2024-04-25T09:01:22.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.442+0000 7f2f46ffd640 1 -- 172.21.15.37:0/2780897413 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f400a8da0 con 0x7f2f4806ec60 2024-04-25T09:01:22.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/2780897413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f4806ec60 msgr2=0x7f2f48142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 --2- 172.21.15.37:0/2780897413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f4806ec60 0x7f2f48142680 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f2f48144c50 tx=0x7f2f4001ed10 comp rx=0 tx=0).stop 2024-04-25T09:01:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/2780897413 shutdown_connections 2024-04-25T09:01:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 --2- 172.21.15.37:0/2780897413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f4806ec60 0x7f2f48142680 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 --2- 172.21.15.37:0/2780897413 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48144ca0 0x7f2f4806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 --2- 172.21.15.37:0/2780897413 >> v2:172.21.15.203:3300/0 conn(0x7f2f480ce510 0x7f2f480ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/2780897413 >> 172.21.15.37:0/2780897413 conn(0x7f2f480b11a0 msgr2=0x7f2f480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/2780897413 shutdown_connections 2024-04-25T09:01:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/2780897413 wait complete. 2024-04-25T09:01:22.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 Processor -- start 2024-04-25T09:01:22.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 -- start start 2024-04-25T09:01:22.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4806ec60 0x7f2f4815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f480ce510 0x7f2f481e3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48144ca0 0x7f2f481e58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480cc900 con 0x7f2f480ce510 2024-04-25T09:01:22.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4806df20 con 0x7f2f4806ec60 2024-04-25T09:01:22.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f481451f0 con 0x7f2f48144ca0 2024-04-25T09:01:22.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f480ce510 0x7f2f481e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f480ce510 0x7f2f481e3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60580/0 (socket says 172.21.15.37:60580) 2024-04-25T09:01:22.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f477fe640 1 -- 172.21.15.37:0/3166214287 learned_addr learned my addr 172.21.15.37:0/3166214287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:22.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4c88e640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48144ca0 0x7f2f481e58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f477fe640 1 -- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48144ca0 msgr2=0x7f2f481e58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f477fe640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48144ca0 0x7f2f481e58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f477fe640 1 -- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4806ec60 msgr2=0x7f2f4815b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f477fe640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4806ec60 0x7f2f4815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f477fe640 1 -- 172.21.15.37:0/3166214287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f481e6790 con 0x7f2f480ce510 2024-04-25T09:01:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f477fe640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f480ce510 0x7f2f481e3260 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000c7f0 tx=0x7f2f3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f44ff9640 1 -- 172.21.15.37:0/3166214287 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30015070 con 0x7f2f480ce510 2024-04-25T09:01:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f44ff9640 1 -- 172.21.15.37:0/3166214287 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f300077c0 con 0x7f2f480ce510 2024-04-25T09:01:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/3166214287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f481e37a0 con 0x7f2f480ce510 2024-04-25T09:01:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f44ff9640 1 -- 172.21.15.37:0/3166214287 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f30007b70 con 0x7f2f480ce510 2024-04-25T09:01:22.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.446+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/3166214287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f481e4980 con 0x7f2f480ce510 2024-04-25T09:01:22.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.450+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/3166214287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4804efa0 con 0x7f2f480ce510 2024-04-25T09:01:22.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.450+0000 7f2f44ff9640 1 -- 172.21.15.37:0/3166214287 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2f30018070 con 0x7f2f480ce510 2024-04-25T09:01:22.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.450+0000 7f2f44ff9640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f28066f90 0x7f2f28069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.450+0000 7f2f44ff9640 1 -- 172.21.15.37:0/3166214287 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f2f300856a0 con 0x7f2f480ce510 2024-04-25T09:01:22.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.450+0000 7f2f47fff640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f28066f90 0x7f2f28069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.454+0000 7f2f47fff640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f28066f90 0x7f2f28069450 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009540 tx=0x7f2f38009450 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.454+0000 7f2f44ff9640 1 -- 172.21.15.37:0/3166214287 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2f300564f0 con 0x7f2f480ce510 2024-04-25T09:01:22.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/3166214287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f481e39d0 con 0x7f2f480ce510 2024-04-25T09:01:22.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f44ff9640 1 -- 172.21.15.37:0/3166214287 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f2f30055df0 con 0x7f2f480ce510 2024-04-25T09:01:22.630 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:22.630 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:52.843744+0000","modified":"2024-04-25T09:00:41.164474+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8340},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8340":{"gid":8340,"name":"a","rank":0,"incarnation":70,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/3366017865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3366017865},{"type":"v1","addr":"172.21.15.37:6835","nonce":3366017865}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:22.631 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 73 2024-04-25T09:01:22.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f28066f90 msgr2=0x7f2f28069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f28066f90 0x7f2f28069450 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f2f38009540 tx=0x7f2f38009450 comp rx=0 tx=0).stop 2024-04-25T09:01:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/3166214287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f480ce510 msgr2=0x7f2f481e3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f480ce510 0x7f2f481e3260 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000c7f0 tx=0x7f2f3000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/3166214287 shutdown_connections 2024-04-25T09:01:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2f28066f90 0x7f2f28069450 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2f48144ca0 0x7f2f481e58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f480ce510 0x7f2f481e3260 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 --2- 172.21.15.37:0/3166214287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2f4806ec60 0x7f2f4815b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/3166214287 >> 172.21.15.37:0/3166214287 conn(0x7f2f480b11a0 msgr2=0x7f2f480b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/3166214287 shutdown_connections 2024-04-25T09:01:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.642+0000 7f2f4e2f1640 1 -- 172.21.15.37:0/3166214287 wait complete. 2024-04-25T09:01:22.648 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f27aa562640 1 Processor -- start 2024-04-25T09:01:22.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f27aa562640 1 -- start start 2024-04-25T09:01:22.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f27aa562640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f27a415a700 0x7f27a415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f27aa562640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27a40b24a0 0x7f27a40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f27aa562640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a40b2090 con 0x7f27a40b3f50 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f27aa562640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a415f200 con 0x7f27a415a700 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f27aa562640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a40b43e0 con 0x7f27a40b24a0 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f279b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f27a40b3f50 0x7f27a40cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48838/0 (socket says 172.21.15.37:48838) 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f279b7fe640 1 -- 172.21.15.37:0/3204919726 learned_addr learned my addr 172.21.15.37:0/3204919726 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f27a3fff640 1 --2- 172.21.15.37:0/3204919726 >> v2:172.21.15.203:3300/0 conn(0x7f27a415a700 0x7f27a415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f27a8aff640 1 --2- 172.21.15.37:0/3204919726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27a40b24a0 0x7f27a40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657742354 0 0) 0x7f27a40b2090 con 0x7f27a40b3f50 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.766+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2790005740 con 0x7f27a40b3f50 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4178701180 0 0) 0x7f2790005740 con 0x7f27a40b3f50 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27a40b24a0 msgr2=0x7f27a40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 --2- 172.21.15.37:0/3204919726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27a40b24a0 0x7f27a40b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 >> v2:172.21.15.203:3300/0 conn(0x7f27a415a700 msgr2=0x7f27a415ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 --2- 172.21.15.37:0/3204919726 >> v2:172.21.15.203:3300/0 conn(0x7f27a415a700 0x7f27a415ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a40b46c0 con 0x7f27a40b3f50 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f278c0016a0 con 0x7f27a40b3f50 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 >> v1:172.21.15.37:6789/0 conn(0x7f27a40b3f50 legacy=0x7f27a40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 --2- 172.21.15.37:0/3204919726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2790005c40 0x7f2790006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 --2- 172.21.15.37:0/3204919726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790006a20 0x7f2790006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 --2- 172.21.15.37:0/3204919726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27a40b24a0 0x7f27900073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a415f200 con 0x7f2790006a20 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a40b43e0 con 0x7f2790005c40 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279000dd40 con 0x7f27a40b24a0 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f279b7fe640 1 --2- 172.21.15.37:0/3204919726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790006a20 0x7f2790006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f279b7fe640 1 -- 172.21.15.37:0/3204919726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27a40b24a0 msgr2=0x7f27900073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a8aff640 1 --2- 172.21.15.37:0/3204919726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27a40b24a0 0x7f27900073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f279b7fe640 1 --2- 172.21.15.37:0/3204919726 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27a40b24a0 0x7f27900073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f279b7fe640 1 -- 172.21.15.37:0/3204919726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2790005c40 msgr2=0x7f2790006060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f279b7fe640 1 --2- 172.21.15.37:0/3204919726 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2790005c40 0x7f2790006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f279b7fe640 1 -- 172.21.15.37:0/3204919726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2790005740 con 0x7f2790006a20 2024-04-25T09:01:22.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f279b7fe640 1 --2- 172.21.15.37:0/3204919726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790006a20 0x7f2790006ea0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f278c001ec0 tx=0x7f278c01cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81dd92cbec2ce213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c007cf0 con 0x7f2790006a20 2024-04-25T09:01:22.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c0a9070 con 0x7f2790006a20 2024-04-25T09:01:22.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a37fe640 1 -- 172.21.15.37:0/3204919726 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c005d20 con 0x7f2790006a20 2024-04-25T09:01:22.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27aa562640 1 -- 172.21.15.37:0/3204919726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790006a20 msgr2=0x7f2790006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27aa562640 1 --2- 172.21.15.37:0/3204919726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2790006a20 0x7f2790006ea0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f278c001ec0 tx=0x7f278c01cb90 comp rx=0 tx=0).stop 2024-04-25T09:01:22.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27a8aff640 1 -- 172.21.15.37:0/3204919726 reap_dead start 2024-04-25T09:01:22.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27aa562640 1 -- 172.21.15.37:0/3204919726 shutdown_connections 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27aa562640 1 -- 172.21.15.37:0/3204919726 >> 172.21.15.37:0/3204919726 conn(0x7f27a40b11a0 msgr2=0x7f27a40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27aa562640 1 -- 172.21.15.37:0/3204919726 shutdown_connections 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.770+0000 7f27aa562640 1 -- 172.21.15.37:0/3204919726 wait complete. 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27aa562640 1 Processor -- start 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27aa562640 1 -- start start 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27aa562640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2790006a20 0x7f27a41585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27aa562640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a40cbb50 0x7f27a40af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27aa562640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a4153820 0x7f27a4152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27aa562640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a40ce600 con 0x7f27a4153820 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27aa562640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a415aae0 con 0x7f27a40cbb50 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27aa562640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a40cd310 con 0x7f2790006a20 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a8aff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a4153820 0x7f27a4152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a8aff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a4153820 0x7f27a4152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60598/0 (socket says 172.21.15.37:60598) 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a8aff640 1 -- 172.21.15.37:0/1946976637 learned_addr learned my addr 172.21.15.37:0/1946976637 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:22.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a3fff640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2790006a20 0x7f27a41585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f279b7fe640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a40cbb50 0x7f27a40af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a8aff640 1 -- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2790006a20 msgr2=0x7f27a41585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a8aff640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2790006a20 0x7f27a41585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a8aff640 1 -- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a40cbb50 msgr2=0x7f27a40af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a8aff640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a40cbb50 0x7f27a40af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a8aff640 1 -- 172.21.15.37:0/1946976637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a4158ae0 con 0x7f27a4153820 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a8aff640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a4153820 0x7f27a4152c50 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f279c007e90 tx=0x7f279c00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a17fa640 1 -- 172.21.15.37:0/1946976637 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c01c070 con 0x7f27a4153820 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a17fa640 1 -- 172.21.15.37:0/1946976637 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c017400 con 0x7f27a4153820 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27aa562640 1 -- 172.21.15.37:0/1946976637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a41633c0 con 0x7f27a4153820 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a17fa640 1 -- 172.21.15.37:0/1946976637 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c0177f0 con 0x7f27a4153820 2024-04-25T09:01:22.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27aa562640 1 -- 172.21.15.37:0/1946976637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a4157da0 con 0x7f27a4153820 2024-04-25T09:01:22.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27aa562640 1 -- 172.21.15.37:0/1946976637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a404efa0 con 0x7f27a4153820 2024-04-25T09:01:22.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a17fa640 1 -- 172.21.15.37:0/1946976637 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f279c01f070 con 0x7f27a4153820 2024-04-25T09:01:22.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a17fa640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2784066f90 0x7f2784069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.774+0000 7f27a17fa640 1 -- 172.21.15.37:0/1946976637 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f279c085390 con 0x7f27a4153820 2024-04-25T09:01:22.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.778+0000 7f27a3fff640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2784066f90 0x7f2784069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.778+0000 7f27a3fff640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2784066f90 0x7f2784069450 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f27940047c0 tx=0x7f2794008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.778+0000 7f27a17fa640 1 -- 172.21.15.37:0/1946976637 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f279c012070 con 0x7f27a4153820 2024-04-25T09:01:22.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.938+0000 7f27aa562640 1 -- 172.21.15.37:0/1946976637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27a40aecc0 con 0x7f27a4153820 2024-04-25T09:01:22.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27a17fa640 1 -- 172.21.15.37:0/1946976637 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+17299 (secure 0 0 0) 0x7f279c0560d0 con 0x7f27a4153820 2024-04-25T09:01:22.930 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:22.930 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":99,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:00:38.139693+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:50.512578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:51.848880+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.8","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":5}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/1462327322":"2024-04-26T09:00:38.139659+0000","172.21.15.37:6834/1462327322":"2024-04-26T09:00:38.139659+0000","172.21.15.37:6835/354285428":"2024-04-26T08:59:17.095322+0000","172.21.15.37:6834/354285428":"2024-04-26T08:59:17.095322+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:22.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 -- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2784066f90 msgr2=0x7f2784069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2784066f90 0x7f2784069450 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f27940047c0 tx=0x7f2794008040 comp rx=0 tx=0).stop 2024-04-25T09:01:22.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 -- 172.21.15.37:0/1946976637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a4153820 msgr2=0x7f27a4152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a4153820 0x7f27a4152c50 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f279c007e90 tx=0x7f279c00f4a0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 -- 172.21.15.37:0/1946976637 shutdown_connections 2024-04-25T09:01:22.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2784066f90 0x7f2784069450 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a4153820 0x7f27a4152c50 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a40cbb50 0x7f27a40af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 --2- 172.21.15.37:0/1946976637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2790006a20 0x7f27a41585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 -- 172.21.15.37:0/1946976637 >> 172.21.15.37:0/1946976637 conn(0x7f27a40b11a0 msgr2=0x7f27a406d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:22.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 -- 172.21.15.37:0/1946976637 shutdown_connections 2024-04-25T09:01:22.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:22.942+0000 7f27aa562640 1 -- 172.21.15.37:0/1946976637 wait complete. 2024-04-25T09:01:22.949 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:01:23.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1dbc9ce640 1 Processor -- start 2024-04-25T09:01:23.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1dbc9ce640 1 -- start start 2024-04-25T09:01:23.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1dbc9ce640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1db806c3b0 0x7f1db806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1dbc9ce640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1db8066bb0 0x7f1db80b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1dbc9ce640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1db815ab30 con 0x7f1db815a750 2024-04-25T09:01:23.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1dbc9ce640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db8067100 con 0x7f1db806c3b0 2024-04-25T09:01:23.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1dbc9ce640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1db80cf560 con 0x7f1db8066bb0 2024-04-25T09:01:23.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1db5d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1db806c3b0 0x7f1db806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1db659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1db815a750 0x7f1db806bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48846/0 (socket says 172.21.15.37:48846) 2024-04-25T09:01:23.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1db659c640 1 -- 172.21.15.37:0/1786972951 learned_addr learned my addr 172.21.15.37:0/1786972951 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:23.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1db6d9d640 1 --2- 172.21.15.37:0/1786972951 >> v2:172.21.15.203:3300/0 conn(0x7f1db8066bb0 0x7f1db80b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1db5d9b640 1 -- 172.21.15.37:0/1786972951 >> v2:172.21.15.203:3300/0 conn(0x7f1db8066bb0 msgr2=0x7f1db80b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:23.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1db5d9b640 1 --2- 172.21.15.37:0/1786972951 >> v2:172.21.15.203:3300/0 conn(0x7f1db8066bb0 0x7f1db80b2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1db5d9b640 1 -- 172.21.15.37:0/1786972951 >> v1:172.21.15.37:6789/0 conn(0x7f1db815a750 legacy=0x7f1db806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1db5d9b640 1 -- 172.21.15.37:0/1786972951 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db80cf840 con 0x7f1db806c3b0 2024-04-25T09:01:23.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1db559a640 1 -- 172.21.15.37:0/1786972951 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68933512 0 0) 0x7f1db815ab30 con 0x7f1db815a750 2024-04-25T09:01:23.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.070+0000 7f1db6d9d640 1 --2- 172.21.15.37:0/1786972951 >> v2:172.21.15.203:3300/0 conn(0x7f1db8066bb0 0x7f1db80b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:23.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db5d9b640 1 --2- 172.21.15.37:0/1786972951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1db806c3b0 0x7f1db806ee60 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1db80cf1d0 tx=0x7f1dac01c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238558ef010ea9d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:23.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db559a640 1 -- 172.21.15.37:0/1786972951 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0b1070 con 0x7f1db806c3b0 2024-04-25T09:01:23.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db559a640 1 -- 172.21.15.37:0/1786972951 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dac0a4d40 con 0x7f1db806c3b0 2024-04-25T09:01:23.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db559a640 1 -- 172.21.15.37:0/1786972951 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dac0ac660 con 0x7f1db806c3b0 2024-04-25T09:01:23.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/1786972951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1db806c3b0 msgr2=0x7f1db806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 --2- 172.21.15.37:0/1786972951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1db806c3b0 0x7f1db806ee60 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f1db80cf1d0 tx=0x7f1dac01c360 comp rx=0 tx=0).stop 2024-04-25T09:01:23.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/1786972951 shutdown_connections 2024-04-25T09:01:23.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 --2- 172.21.15.37:0/1786972951 >> v2:172.21.15.203:3300/0 conn(0x7f1db8066bb0 0x7f1db80b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 --2- 172.21.15.37:0/1786972951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1db806c3b0 0x7f1db806ee60 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/1786972951 >> 172.21.15.37:0/1786972951 conn(0x7f1db8071810 msgr2=0x7f1db8071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:23.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/1786972951 shutdown_connections 2024-04-25T09:01:23.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/1786972951 wait complete. 2024-04-25T09:01:23.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 Processor -- start 2024-04-25T09:01:23.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 -- start start 2024-04-25T09:01:23.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1db8066bb0 0x7f1db8156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db806c3b0 0x7f1db8156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815a750 0x7f1db8157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db81677b0 con 0x7f1db8066bb0 2024-04-25T09:01:23.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1db80ceda0 con 0x7f1db806c3b0 2024-04-25T09:01:23.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1db8167930 con 0x7f1db815a750 2024-04-25T09:01:23.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1db8066bb0 0x7f1db8156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1db8066bb0 0x7f1db8156030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60612/0 (socket says 172.21.15.37:60612) 2024-04-25T09:01:23.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db659c640 1 -- 172.21.15.37:0/3899164185 learned_addr learned my addr 172.21.15.37:0/3899164185 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:23.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db5d9b640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db806c3b0 0x7f1db8156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db6d9d640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815a750 0x7f1db8157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db659c640 1 -- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815a750 msgr2=0x7f1db8157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db659c640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815a750 0x7f1db8157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db659c640 1 -- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db806c3b0 msgr2=0x7f1db8156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db659c640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db806c3b0 0x7f1db8156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db659c640 1 -- 172.21.15.37:0/3899164185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db8156b50 con 0x7f1db8066bb0 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db6d9d640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815a750 0x7f1db8157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1db659c640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1db8066bb0 0x7f1db8156030 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f1da800e710 tx=0x7f1da800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1da6ffd640 1 -- 172.21.15.37:0/3899164185 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da800ee20 con 0x7f1db8066bb0 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1da6ffd640 1 -- 172.21.15.37:0/3899164185 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da8016020 con 0x7f1db8066bb0 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.074+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/3899164185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1db81548a0 con 0x7f1db8066bb0 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.078+0000 7f1da6ffd640 1 -- 172.21.15.37:0/3899164185 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da8007500 con 0x7f1db8066bb0 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.078+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/3899164185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db8154e60 con 0x7f1db8066bb0 2024-04-25T09:01:23.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.078+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/3899164185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1db804efa0 con 0x7f1db8066bb0 2024-04-25T09:01:23.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.078+0000 7f1da6ffd640 1 -- 172.21.15.37:0/3899164185 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1da8001740 con 0x7f1db8066bb0 2024-04-25T09:01:23.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.078+0000 7f1da6ffd640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1d98066f60 0x7f1d98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.078+0000 7f1da6ffd640 1 -- 172.21.15.37:0/3899164185 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f1da8087e90 con 0x7f1db8066bb0 2024-04-25T09:01:23.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.078+0000 7f1db5d9b640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1d98066f60 0x7f1d98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.078+0000 7f1db5d9b640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1d98066f60 0x7f1d98069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0ae000 tx=0x7f1dac0a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:23.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.082+0000 7f1da6ffd640 1 -- 172.21.15.37:0/3899164185 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1da8020020 con 0x7f1db8066bb0 2024-04-25T09:01:23.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.266+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/3899164185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1db80cbe30 con 0x7f1db8066bb0 2024-04-25T09:01:23.256 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T09:01:23.266+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:01:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.270+0000 7f1da6ffd640 1 -- 172.21.15.37:0/3899164185 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v74) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1da800f360 con 0x7f1db8066bb0 2024-04-25T09:01:23.258 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:01:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1d98066f60 msgr2=0x7f1d98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1d98066f60 0x7f1d98069420 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0ae000 tx=0x7f1dac0a6070 comp rx=0 tx=0).stop 2024-04-25T09:01:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/3899164185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1db8066bb0 msgr2=0x7f1db8156030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1db8066bb0 0x7f1db8156030 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f1da800e710 tx=0x7f1da800ebd0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/3899164185 shutdown_connections 2024-04-25T09:01:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1d98066f60 0x7f1d98069420 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1db815a750 0x7f1db8157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1db806c3b0 0x7f1db8156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 --2- 172.21.15.37:0/3899164185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1db8066bb0 0x7f1db8156030 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/3899164185 >> 172.21.15.37:0/3899164185 conn(0x7f1db8071810 msgr2=0x7f1db8152be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:23.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/3899164185 shutdown_connections 2024-04-25T09:01:23.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.274+0000 7f1dbc9ce640 1 -- 172.21.15.37:0/3899164185 wait complete. 2024-04-25T09:01:23.267 INFO:tasks.ceph.mds.a.smithi037.stderr: -2339> 2024-04-25T09:00:44.151+0000 7f793f332640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:01:23.268 INFO:tasks.ceph.mds.a.smithi037.stderr: -1916> 2024-04-25T09:00:46.483+0000 7f793b32a640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5639a9c50100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(14)0x10000000001:[<0x10000000000/file v6>,<0x1/dir v13>]//[]","memoryvalue":"(14)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v21>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T09:01:23.280 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:01:23.312 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T09:01:23.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.406+0000 7f88cd008640 1 Processor -- start 2024-04-25T09:01:23.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.406+0000 7f88cd008640 1 -- start start 2024-04-25T09:01:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88cd008640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80d4230 0x7f88c80d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88cd008640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c806be60 con 0x7f88c80d07f0 2024-04-25T09:01:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88cd008640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c815de60 con 0x7f88c815da80 2024-04-25T09:01:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88cd008640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80d46a0 con 0x7f88c80d4230 2024-04-25T09:01:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f88c80d07f0 0x7f88c806c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48862/0 (socket says 172.21.15.37:48862) 2024-04-25T09:01:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c659c640 1 -- 172.21.15.37:0/2685415836 learned_addr learned my addr 172.21.15.37:0/2685415836 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c759e640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80d4230 0x7f88c80d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885413794 0 0) 0x7f88c806be60 con 0x7f88c80d07f0 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88b0003530 con 0x7f88c80d07f0 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 52397457 0 0) 0x7f88b0003530 con 0x7f88c80d07f0 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80d4230 msgr2=0x7f88c80d1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80d4230 0x7f88c80d1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 >> v1:172.21.15.203:6789/0 conn(0x7f88c815da80 legacy=0x7f88c80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c80d2450 con 0x7f88c80d07f0 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f88bc001820 con 0x7f88c80d07f0 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 >> v1:172.21.15.37:6789/0 conn(0x7f88c80d07f0 legacy=0x7f88c806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b0003a60 0x7f88b0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80d4230 0x7f88b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b0004d50 0x7f88b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c80d46a0 con 0x7f88b0003a60 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b000bb00 con 0x7f88b0004d50 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b000bd00 con 0x7f88c80d4230 2024-04-25T09:01:23.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c6d9d640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b0003a60 0x7f88b0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c759e640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b0004d50 0x7f88b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c659c640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80d4230 0x7f88b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c6d9d640 1 -- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80d4230 msgr2=0x7f88b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c6d9d640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80d4230 0x7f88b0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c6d9d640 1 -- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b0004d50 msgr2=0x7f88b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c6d9d640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b0004d50 0x7f88b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c6d9d640 1 -- 172.21.15.37:0/2685415836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b0003530 con 0x7f88b0003a60 2024-04-25T09:01:23.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.410+0000 7f88c759e640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b0004d50 0x7f88b00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c6d9d640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b0003a60 0x7f88b0003ea0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f88b8001780 tx=0x7f88b801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e87b7bfe6428d3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b80013a0 con 0x7f88b0003a60 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b80052f0 con 0x7f88b0003a60 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c5d9b640 1 -- 172.21.15.37:0/2685415836 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b8003840 con 0x7f88b0003a60 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 -- 172.21.15.37:0/2685415836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b0003a60 msgr2=0x7f88b0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 --2- 172.21.15.37:0/2685415836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b0003a60 0x7f88b0003ea0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f88b8001780 tx=0x7f88b801c360 comp rx=0 tx=0).stop 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c759e640 1 -- 172.21.15.37:0/2685415836 reap_dead start 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 -- 172.21.15.37:0/2685415836 shutdown_connections 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 -- 172.21.15.37:0/2685415836 >> 172.21.15.37:0/2685415836 conn(0x7f88c808b000 msgr2=0x7f88c80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 -- 172.21.15.37:0/2685415836 shutdown_connections 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 -- 172.21.15.37:0/2685415836 wait complete. 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 Processor -- start 2024-04-25T09:01:23.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 -- start start 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b0003a60 0x7f88c80b8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88c81547d0 0x7f88c80c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80b6310 0x7f88c80b6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c806d970 con 0x7f88c81547d0 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c806d7f0 con 0x7f88b0003a60 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c806daf0 con 0x7f88c80b6310 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88c81547d0 0x7f88c80c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88c81547d0 0x7f88c80c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60648/0 (socket says 172.21.15.37:60648) 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c659c640 1 -- 172.21.15.37:0/3600827492 learned_addr learned my addr 172.21.15.37:0/3600827492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c6d9d640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b0003a60 0x7f88c80b8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c759e640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80b6310 0x7f88c80b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c659c640 1 -- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80b6310 msgr2=0x7f88c80b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c659c640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80b6310 0x7f88c80b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c659c640 1 -- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b0003a60 msgr2=0x7f88c80b8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c659c640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b0003a60 0x7f88c80b8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c659c640 1 -- 172.21.15.37:0/3600827492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c80c89d0 con 0x7f88c81547d0 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88c659c640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88c81547d0 0x7f88c80c8490 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00d800 tx=0x7f88bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88af7fe640 1 -- 172.21.15.37:0/3600827492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc007270 con 0x7f88c81547d0 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88af7fe640 1 -- 172.21.15.37:0/3600827492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc017070 con 0x7f88c81547d0 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.414+0000 7f88cd008640 1 -- 172.21.15.37:0/3600827492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c80c8c90 con 0x7f88c81547d0 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.418+0000 7f88af7fe640 1 -- 172.21.15.37:0/3600827492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc007610 con 0x7f88c81547d0 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.418+0000 7f88cd008640 1 -- 172.21.15.37:0/3600827492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c80c9250 con 0x7f88c81547d0 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.418+0000 7f88cd008640 1 -- 172.21.15.37:0/3600827492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c804efa0 con 0x7f88c81547d0 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.418+0000 7f88af7fe640 1 -- 172.21.15.37:0/3600827492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88bc0057b0 con 0x7f88c81547d0 2024-04-25T09:01:23.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.418+0000 7f88af7fe640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88b4066f60 0x7f88b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.418+0000 7f88af7fe640 1 -- 172.21.15.37:0/3600827492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f88bc0851f0 con 0x7f88c81547d0 2024-04-25T09:01:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.418+0000 7f88c6d9d640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88b4066f60 0x7f88b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.418+0000 7f88c6d9d640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88b4066f60 0x7f88b4069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f88b8001750 tx=0x7f88b80011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:23.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.422+0000 7f88af7fe640 1 -- 172.21.15.37:0/3600827492 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88bc055f00 con 0x7f88c81547d0 2024-04-25T09:01:23.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:23.618+0000 7f88cd008640 1 -- 172.21.15.37:0/3600827492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f88c806cda0 con 0x7f88c81547d0 2024-04-25T09:01:24.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.278+0000 7f88af7fe640 1 -- 172.21.15.37:0/3600827492 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7f88bc01ae40 con 0x7f88c81547d0 2024-04-25T09:01:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 -- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88b4066f60 msgr2=0x7f88b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88b4066f60 0x7f88b4069420 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f88b8001750 tx=0x7f88b80011a0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 -- 172.21.15.37:0/3600827492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88c81547d0 msgr2=0x7f88c80c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88c81547d0 0x7f88c80c8490 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00d800 tx=0x7f88bc00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 -- 172.21.15.37:0/3600827492 shutdown_connections 2024-04-25T09:01:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88b4066f60 0x7f88b4069420 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88c80b6310 0x7f88c80b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88c81547d0 0x7f88c80c8490 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 --2- 172.21.15.37:0/3600827492 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b0003a60 0x7f88c80b8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 -- 172.21.15.37:0/3600827492 >> 172.21.15.37:0/3600827492 conn(0x7f88c808b000 msgr2=0x7f88c815ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:24.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 -- 172.21.15.37:0/3600827492 shutdown_connections 2024-04-25T09:01:24.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.282+0000 7f88cd008640 1 -- 172.21.15.37:0/3600827492 wait complete. 2024-04-25T09:01:24.295 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:01:24.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02eb3fb640 1 Processor -- start 2024-04-25T09:01:24.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02eb3fb640 1 -- start start 2024-04-25T09:01:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02eb3fb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f02e40cf510 0x7f02e40cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02eb3fb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f02e40ac170 0x7f02e40ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02eb3fb640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e40b28c0 con 0x7f02e40adc70 2024-04-25T09:01:24.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02eb3fb640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e40cfec0 con 0x7f02e40cf510 2024-04-25T09:01:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02eb3fb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e415bb20 con 0x7f02e40ac170 2024-04-25T09:01:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02e9197640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f02e40adc70 0x7f02e406e4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48870/0 (socket says 172.21.15.37:48870) 2024-04-25T09:01:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02e9197640 1 -- 172.21.15.37:0/42333640 learned_addr learned my addr 172.21.15.37:0/42333640 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02e8996640 1 --2- 172.21.15.37:0/42333640 >> v2:172.21.15.37:3300/0 conn(0x7f02e40cf510 0x7f02e40cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02e9998640 1 --2- 172.21.15.37:0/42333640 >> v2:172.21.15.203:3300/0 conn(0x7f02e40ac170 0x7f02e40ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02d7fff640 1 -- 172.21.15.37:0/42333640 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509524576 0 0) 0x7f02e40b28c0 con 0x7f02e40adc70 2024-04-25T09:01:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02d7fff640 1 -- 172.21.15.37:0/42333640 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02d0003530 con 0x7f02e40adc70 2024-04-25T09:01:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02e8996640 1 -- 172.21.15.37:0/42333640 >> v2:172.21.15.203:3300/0 conn(0x7f02e40ac170 msgr2=0x7f02e40ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02e8996640 1 --2- 172.21.15.37:0/42333640 >> v2:172.21.15.203:3300/0 conn(0x7f02e40ac170 0x7f02e40ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02e8996640 1 -- 172.21.15.37:0/42333640 >> v1:172.21.15.37:6789/0 conn(0x7f02e40adc70 legacy=0x7f02e406e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.486+0000 7f02e8996640 1 -- 172.21.15.37:0/42333640 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e40b2b60 con 0x7f02e40cf510 2024-04-25T09:01:24.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02e8996640 1 --2- 172.21.15.37:0/42333640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e40cf510 0x7f02e40cf980 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f02e4073220 tx=0x7f02d801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d663e4b71705cc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:24.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02d7fff640 1 -- 172.21.15.37:0/42333640 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d80b2070 con 0x7f02e40cf510 2024-04-25T09:01:24.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02d7fff640 1 -- 172.21.15.37:0/42333640 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02d80a58e0 con 0x7f02e40cf510 2024-04-25T09:01:24.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02d7fff640 1 -- 172.21.15.37:0/42333640 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02d80a5cb0 con 0x7f02e40cf510 2024-04-25T09:01:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02eb3fb640 1 -- 172.21.15.37:0/42333640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e40cf510 msgr2=0x7f02e40cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02eb3fb640 1 --2- 172.21.15.37:0/42333640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e40cf510 0x7f02e40cf980 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f02e4073220 tx=0x7f02d801c930 comp rx=0 tx=0).stop 2024-04-25T09:01:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02eb3fb640 1 -- 172.21.15.37:0/42333640 shutdown_connections 2024-04-25T09:01:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02eb3fb640 1 --2- 172.21.15.37:0/42333640 >> v2:172.21.15.203:3300/0 conn(0x7f02e40ac170 0x7f02e40ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02eb3fb640 1 --2- 172.21.15.37:0/42333640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e40cf510 0x7f02e40cf980 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02eb3fb640 1 -- 172.21.15.37:0/42333640 >> 172.21.15.37:0/42333640 conn(0x7f02e406ae60 msgr2=0x7f02e406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:24.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02eb3fb640 1 -- 172.21.15.37:0/42333640 shutdown_connections 2024-04-25T09:01:24.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.490+0000 7f02eb3fb640 1 -- 172.21.15.37:0/42333640 wait complete. 2024-04-25T09:01:24.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02eb3fb640 1 Processor -- start 2024-04-25T09:01:24.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02eb3fb640 1 -- start start 2024-04-25T09:01:24.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02eb3fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e40ac170 0x7f02e40c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02eb3fb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02e40adc70 0x7f02e40b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02eb3fb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02e40cf510 0x7f02e40af010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02eb3fb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40b3c00 con 0x7f02e40ac170 2024-04-25T09:01:24.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02eb3fb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40b36e0 con 0x7f02e40adc70 2024-04-25T09:01:24.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02eb3fb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40c30d0 con 0x7f02e40cf510 2024-04-25T09:01:24.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e9197640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e40ac170 0x7f02e40c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:24.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e9197640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e40ac170 0x7f02e40c2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60674/0 (socket says 172.21.15.37:60674) 2024-04-25T09:01:24.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e9197640 1 -- 172.21.15.37:0/1962503565 learned_addr learned my addr 172.21.15.37:0/1962503565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:24.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e8996640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02e40adc70 0x7f02e40b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:24.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e9998640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02e40cf510 0x7f02e40af010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:24.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e9197640 1 -- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02e40cf510 msgr2=0x7f02e40af010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e9197640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02e40cf510 0x7f02e40af010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e9197640 1 -- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02e40adc70 msgr2=0x7f02e40b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e9197640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02e40adc70 0x7f02e40b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e9197640 1 -- 172.21.15.37:0/1962503565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e40cdd90 con 0x7f02e40ac170 2024-04-25T09:01:24.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e8996640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02e40adc70 0x7f02e40b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:24.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02e9197640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e40ac170 0x7f02e40c2b90 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f02e000c460 tx=0x7f02e000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:24.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02d5ffb640 1 -- 172.21.15.37:0/1962503565 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0017070 con 0x7f02e40ac170 2024-04-25T09:01:24.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02d5ffb640 1 -- 172.21.15.37:0/1962503565 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e0005d10 con 0x7f02e40ac170 2024-04-25T09:01:24.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02d5ffb640 1 -- 172.21.15.37:0/1962503565 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e00072b0 con 0x7f02e40ac170 2024-04-25T09:01:24.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02eb3fb640 1 -- 172.21.15.37:0/1962503565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e40ce070 con 0x7f02e40ac170 2024-04-25T09:01:24.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02eb3fb640 1 -- 172.21.15.37:0/1962503565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e40ce630 con 0x7f02e40ac170 2024-04-25T09:01:24.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02c77fe640 1 -- 172.21.15.37:0/1962503565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c00052a0 con 0x7f02e40ac170 2024-04-25T09:01:24.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02d5ffb640 1 -- 172.21.15.37:0/1962503565 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02e001a070 con 0x7f02e40ac170 2024-04-25T09:01:24.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.494+0000 7f02d5ffb640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f60 0x7f02c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.498+0000 7f02d5ffb640 1 -- 172.21.15.37:0/1962503565 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6475+0+0 (secure 0 0 0) 0x7f02e0084630 con 0x7f02e40ac170 2024-04-25T09:01:24.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.498+0000 7f02e8996640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f60 0x7f02c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:24.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.498+0000 7f02e8996640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f60 0x7f02c8069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f02d80af000 tx=0x7f02d80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:24.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.502+0000 7f02d5ffb640 1 -- 172.21.15.37:0/1962503565 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f02e00553d0 con 0x7f02e40ac170 2024-04-25T09:01:24.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:24.666+0000 7f02c77fe640 1 -- 172.21.15.37:0/1962503565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f02c0005590 con 0x7f02e40ac170 2024-04-25T09:01:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.462+0000 7f02d5ffb640 1 -- 172.21.15.37:0/1962503565 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v101) v1 ==== 181+0+0 (secure 0 0 0) 0x7f02e0054cd0 con 0x7f02e40ac170 2024-04-25T09:01:25.453 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 -- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f60 msgr2=0x7f02c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f60 0x7f02c8069420 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f02d80af000 tx=0x7f02d80a7070 comp rx=0 tx=0).stop 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 -- 172.21.15.37:0/1962503565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e40ac170 msgr2=0x7f02e40c2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e40ac170 0x7f02e40c2b90 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f02e000c460 tx=0x7f02e000c920 comp rx=0 tx=0).stop 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 -- 172.21.15.37:0/1962503565 shutdown_connections 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02c8066f60 0x7f02c8069420 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02e40cf510 0x7f02e40af010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02e40adc70 0x7f02e40b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 --2- 172.21.15.37:0/1962503565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02e40ac170 0x7f02e40c2b90 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 -- 172.21.15.37:0/1962503565 >> 172.21.15.37:0/1962503565 conn(0x7f02e406ae60 msgr2=0x7f02e4154290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 -- 172.21.15.37:0/1962503565 shutdown_connections 2024-04-25T09:01:25.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.466+0000 7f02c77fe640 1 -- 172.21.15.37:0/1962503565 wait complete. 2024-04-25T09:01:25.477 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:01:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc6f6b640 1 Processor -- start 2024-04-25T09:01:25.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc6f6b640 1 -- start start 2024-04-25T09:01:25.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc6f6b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3cc015d980 0x7f3cc00d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc6f6b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3cc00d12c0 0x7f3cc00d1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc6f6b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc00d21f0 0x7f3cc00ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc6f6b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc006c810 con 0x7f3cc015d980 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc6f6b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc015dd60 con 0x7f3cc00d12c0 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc6f6b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc00d4340 con 0x7f3cc00d21f0 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc4d07640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3cc015d980 0x7f3cc00d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc4d07640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3cc015d980 0x7f3cc00d3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60678/0 (socket says 172.21.15.37:60678) 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc4d07640 1 -- 172.21.15.37:0/4280664529 learned_addr learned my addr 172.21.15.37:0/4280664529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cbffff640 1 --2- 172.21.15.37:0/4280664529 >> v2:172.21.15.203:3300/0 conn(0x7f3cc00d12c0 0x7f3cc00d1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc5508640 1 --2- 172.21.15.37:0/4280664529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc00d21f0 0x7f3cc00ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc4d07640 1 -- 172.21.15.37:0/4280664529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc00d21f0 msgr2=0x7f3cc00ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:25.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc4d07640 1 --2- 172.21.15.37:0/4280664529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc00d21f0 0x7f3cc00ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc4d07640 1 -- 172.21.15.37:0/4280664529 >> v2:172.21.15.203:3300/0 conn(0x7f3cc00d12c0 msgr2=0x7f3cc00d1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc4d07640 1 --2- 172.21.15.37:0/4280664529 >> v2:172.21.15.203:3300/0 conn(0x7f3cc00d12c0 0x7f3cc00d1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.634+0000 7f3cc4d07640 1 -- 172.21.15.37:0/4280664529 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc00d45e0 con 0x7f3cc015d980 2024-04-25T09:01:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc4d07640 1 --2- 172.21.15.37:0/4280664529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc015d980 0x7f3cc00d3e00 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00b61f0 tx=0x7f3cb401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89a139554924653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cbf7fe640 1 -- 172.21.15.37:0/4280664529 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb40b0070 con 0x7f3cc015d980 2024-04-25T09:01:25.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cbf7fe640 1 -- 172.21.15.37:0/4280664529 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb40ab600 con 0x7f3cc015d980 2024-04-25T09:01:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cbf7fe640 1 -- 172.21.15.37:0/4280664529 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb40ab9d0 con 0x7f3cc015d980 2024-04-25T09:01:25.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/4280664529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc015d980 msgr2=0x7f3cc00d3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc6f6b640 1 --2- 172.21.15.37:0/4280664529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc015d980 0x7f3cc00d3e00 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00b61f0 tx=0x7f3cb401c490 comp rx=0 tx=0).stop 2024-04-25T09:01:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/4280664529 shutdown_connections 2024-04-25T09:01:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc6f6b640 1 --2- 172.21.15.37:0/4280664529 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc00d21f0 0x7f3cc00ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc6f6b640 1 --2- 172.21.15.37:0/4280664529 >> v2:172.21.15.203:3300/0 conn(0x7f3cc00d12c0 0x7f3cc00d1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc6f6b640 1 --2- 172.21.15.37:0/4280664529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc015d980 0x7f3cc00d3e00 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/4280664529 >> 172.21.15.37:0/4280664529 conn(0x7f3cc008b000 msgr2=0x7f3cc00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:25.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/4280664529 shutdown_connections 2024-04-25T09:01:25.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/4280664529 wait complete. 2024-04-25T09:01:25.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc6f6b640 1 Processor -- start 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.638+0000 7f3cc6f6b640 1 -- start start 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc6f6b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc00d12c0 0x7f3cc0155750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc6f6b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc00d21f0 0x7f3cc00cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc6f6b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cc015d980 0x7f3cc0158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc6f6b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc006cca0 con 0x7f3cc00d21f0 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc6f6b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc006c4a0 con 0x7f3cc015d980 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc6f6b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc006ce20 con 0x7f3cc00d12c0 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc00d21f0 0x7f3cc00cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc00d21f0 0x7f3cc00cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60686/0 (socket says 172.21.15.37:60686) 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbffff640 1 -- 172.21.15.37:0/1190156914 learned_addr learned my addr 172.21.15.37:0/1190156914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc4d07640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc00d12c0 0x7f3cc0155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc5508640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cc015d980 0x7f3cc0158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbffff640 1 -- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc00d12c0 msgr2=0x7f3cc0155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbffff640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc00d12c0 0x7f3cc0155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbffff640 1 -- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cc015d980 msgr2=0x7f3cc0158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbffff640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cc015d980 0x7f3cc0158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbffff640 1 -- 172.21.15.37:0/1190156914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc0155e00 con 0x7f3cc00d21f0 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc4d07640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc00d12c0 0x7f3cc0155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc5508640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cc015d980 0x7f3cc0158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbffff640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc00d21f0 0x7f3cc00cb1d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00d800 tx=0x7f3cac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbd7fa640 1 -- 172.21.15.37:0/1190156914 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00b6c0 con 0x7f3cc00d21f0 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbd7fa640 1 -- 172.21.15.37:0/1190156914 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac019070 con 0x7f3cc00d21f0 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/1190156914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc01560e0 con 0x7f3cc00d21f0 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbd7fa640 1 -- 172.21.15.37:0/1190156914 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac00ba60 con 0x7f3cc00d21f0 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/1190156914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc0156bc0 con 0x7f3cc00d21f0 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/1190156914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc0067c30 con 0x7f3cc00d21f0 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbd7fa640 1 -- 172.21.15.37:0/1190156914 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3cac00bc00 con 0x7f3cc00d21f0 2024-04-25T09:01:25.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbd7fa640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca4066f60 0x7f3ca4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cbd7fa640 1 -- 172.21.15.37:0/1190156914 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6037+0+0 (secure 0 0 0) 0x7f3cac085620 con 0x7f3cc00d21f0 2024-04-25T09:01:25.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.642+0000 7f3cc4d07640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca4066f60 0x7f3ca4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.646+0000 7f3cc4d07640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca4066f60 0x7f3ca4069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40ad000 tx=0x7f3cb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:25.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.646+0000 7f3cbd7fa640 1 -- 172.21.15.37:0/1190156914 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3cac0564e0 con 0x7f3cc00d21f0 2024-04-25T09:01:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:25.810+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/1190156914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3cc006ac90 con 0x7f3cc00d21f0 2024-04-25T09:01:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.466+0000 7f3cbd7fa640 1 -- 172.21.15.37:0/1190156914 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v102) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3cac083e20 con 0x7f3cc00d21f0 2024-04-25T09:01:26.455 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T09:01:26.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.470+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca4066f60 msgr2=0x7f3ca4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.470+0000 7f3cc6f6b640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca4066f60 0x7f3ca4069420 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40ad000 tx=0x7f3cb4008040 comp rx=0 tx=0).stop 2024-04-25T09:01:26.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.470+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/1190156914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc00d21f0 msgr2=0x7f3cc00cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.470+0000 7f3cc6f6b640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc00d21f0 0x7f3cc00cb1d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00d800 tx=0x7f3cac00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.470+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/1190156914 shutdown_connections 2024-04-25T09:01:26.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.470+0000 7f3cc6f6b640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca4066f60 0x7f3ca4069420 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.470+0000 7f3cc6f6b640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cc015d980 0x7f3cc0158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.470+0000 7f3cc6f6b640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc00d21f0 0x7f3cc00cb1d0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.470+0000 7f3cc6f6b640 1 --2- 172.21.15.37:0/1190156914 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc00d12c0 0x7f3cc0155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.470+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/1190156914 >> 172.21.15.37:0/1190156914 conn(0x7f3cc008b000 msgr2=0x7f3cc00b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:26.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.474+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/1190156914 shutdown_connections 2024-04-25T09:01:26.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.474+0000 7f3cc6f6b640 1 -- 172.21.15.37:0/1190156914 wait complete. 2024-04-25T09:01:26.500 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:01:26.501 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:26.503 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa18d3f5640 1 Processor -- start 2024-04-25T09:01:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa18d3f5640 1 -- start start 2024-04-25T09:01:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa18d3f5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa18815b6b0 0x7fa1880ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa18d3f5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa18815ff50 con 0x7fa188073010 2024-04-25T09:01:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa18d3f5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1880cbd20 con 0x7fa1880cf590 2024-04-25T09:01:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa18d3f5640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa18815ba90 con 0x7fa18815b6b0 2024-04-25T09:01:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa186ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa18815b6b0 0x7fa1880ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa1877fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa188073010 0x7fa18815f840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48872/0 (socket says 172.21.15.37:48872) 2024-04-25T09:01:26.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa1877fe640 1 -- 172.21.15.37:0/4291044607 learned_addr learned my addr 172.21.15.37:0/4291044607 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:26.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa185ffb640 1 -- 172.21.15.37:0/4291044607 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402088272 0 0) 0x7fa18815ff50 con 0x7fa188073010 2024-04-25T09:01:26.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa185ffb640 1 -- 172.21.15.37:0/4291044607 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa170003530 con 0x7fa188073010 2024-04-25T09:01:26.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa186ffd640 1 -- 172.21.15.37:0/4291044607 >> v1:172.21.15.203:6789/0 conn(0x7fa1880cf590 legacy=0x7fa18815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:26.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa186ffd640 1 -- 172.21.15.37:0/4291044607 >> v1:172.21.15.37:6789/0 conn(0x7fa188073010 legacy=0x7fa18815f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa185ffb640 1 -- 172.21.15.37:0/4291044607 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4033493111 0 0) 0x7fa170003530 con 0x7fa188073010 2024-04-25T09:01:26.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.634+0000 7fa186ffd640 1 -- 172.21.15.37:0/4291044607 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1880cc000 con 0x7fa18815b6b0 2024-04-25T09:01:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.638+0000 7fa186ffd640 1 --2- 172.21.15.37:0/4291044607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa18815b6b0 0x7fa1880ca3c0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fa1780097c0 tx=0x7fa17801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cda244b57a275c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.638+0000 7fa185ffb640 1 -- 172.21.15.37:0/4291044607 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1780b1070 con 0x7fa18815b6b0 2024-04-25T09:01:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.638+0000 7fa185ffb640 1 -- 172.21.15.37:0/4291044607 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1780ac440 con 0x7fa18815b6b0 2024-04-25T09:01:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.638+0000 7fa185ffb640 1 -- 172.21.15.37:0/4291044607 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa180001a00 con 0x7fa18815b6b0 2024-04-25T09:01:26.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.638+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4291044607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa18815b6b0 msgr2=0x7fa1880ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.638+0000 7fa18d3f5640 1 --2- 172.21.15.37:0/4291044607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa18815b6b0 0x7fa1880ca3c0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fa1780097c0 tx=0x7fa17801c390 comp rx=0 tx=0).stop 2024-04-25T09:01:26.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.638+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4291044607 shutdown_connections 2024-04-25T09:01:26.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.638+0000 7fa18d3f5640 1 --2- 172.21.15.37:0/4291044607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa18815b6b0 0x7fa1880ca3c0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.638+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4291044607 >> 172.21.15.37:0/4291044607 conn(0x7fa18806ae60 msgr2=0x7fa18806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:26.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.638+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4291044607 shutdown_connections 2024-04-25T09:01:26.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.638+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4291044607 wait complete. 2024-04-25T09:01:26.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa18d3f5640 1 Processor -- start 2024-04-25T09:01:26.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa18d3f5640 1 -- start start 2024-04-25T09:01:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa18d3f5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa188073010 0x7fa1881524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa18d3f5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1880cf590 0x7fa188152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa18d3f5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa18815b6b0 0x7fa1881f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa18d3f5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880ca900 con 0x7fa18815b6b0 2024-04-25T09:01:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa18d3f5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880cba30 con 0x7fa188073010 2024-04-25T09:01:26.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa18d3f5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1880cfdb0 con 0x7fa1880cf590 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa1877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa18815b6b0 0x7fa1881f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa1877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa18815b6b0 0x7fa1881f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60702/0 (socket says 172.21.15.37:60702) 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa1877fe640 1 -- 172.21.15.37:0/4273248088 learned_addr learned my addr 172.21.15.37:0/4273248088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa186ffd640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa188073010 0x7fa1881524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa1877fe640 1 -- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1880cf590 msgr2=0x7fa188152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa1877fe640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1880cf590 0x7fa188152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa1877fe640 1 -- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa188073010 msgr2=0x7fa1881524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa1877fe640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa188073010 0x7fa1881524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa1877fe640 1 -- 172.21.15.37:0/4273248088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa188098e30 con 0x7fa18815b6b0 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa186ffd640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa188073010 0x7fa1881524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa1877fe640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa18815b6b0 0x7fa1881f7940 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fa1800018d0 tx=0x7fa18000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa16f7fe640 1 -- 172.21.15.37:0/4273248088 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa180012400 con 0x7fa18815b6b0 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa16f7fe640 1 -- 172.21.15.37:0/4273248088 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1800125c0 con 0x7fa18815b6b0 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4273248088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa188153360 con 0x7fa18815b6b0 2024-04-25T09:01:26.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa16f7fe640 1 -- 172.21.15.37:0/4273248088 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1800129b0 con 0x7fa18815b6b0 2024-04-25T09:01:26.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4273248088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa188153080 con 0x7fa18815b6b0 2024-04-25T09:01:26.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4273248088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa18804efa0 con 0x7fa18815b6b0 2024-04-25T09:01:26.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa16f7fe640 1 -- 172.21.15.37:0/4273248088 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa180012b50 con 0x7fa18815b6b0 2024-04-25T09:01:26.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa16f7fe640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa174066f60 0x7fa174069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.642+0000 7fa16f7fe640 1 -- 172.21.15.37:0/4273248088 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa180085760 con 0x7fa18815b6b0 2024-04-25T09:01:26.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.646+0000 7fa186ffd640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa174066f60 0x7fa174069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.646+0000 7fa186ffd640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa174066f60 0x7fa174069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa1780ae000 tx=0x7fa1780a6070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.650+0000 7fa16f7fe640 1 -- 172.21.15.37:0/4273248088 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa1800182b0 con 0x7fa18815b6b0 2024-04-25T09:01:26.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.806+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4273248088 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa1880b1ad0 con 0x7fa18815b6b0 2024-04-25T09:01:26.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa16f7fe640 1 -- 172.21.15.37:0/4273248088 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa180056980 con 0x7fa18815b6b0 2024-04-25T09:01:26.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa174066f60 msgr2=0x7fa174069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa18d3f5640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa174066f60 0x7fa174069420 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fa1780ae000 tx=0x7fa1780a6070 comp rx=0 tx=0).stop 2024-04-25T09:01:26.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4273248088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa18815b6b0 msgr2=0x7fa1881f7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa18d3f5640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa18815b6b0 0x7fa1881f7940 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fa1800018d0 tx=0x7fa18000a430 comp rx=0 tx=0).stop 2024-04-25T09:01:26.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4273248088 shutdown_connections 2024-04-25T09:01:26.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa18d3f5640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa174066f60 0x7fa174069420 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa18d3f5640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa18815b6b0 0x7fa1881f7940 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa18d3f5640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa1880cf590 0x7fa188152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa18d3f5640 1 --2- 172.21.15.37:0/4273248088 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa188073010 0x7fa1881524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4273248088 >> 172.21.15.37:0/4273248088 conn(0x7fa18806ae60 msgr2=0x7fa1880b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.814+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4273248088 shutdown_connections 2024-04-25T09:01:26.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.818+0000 7fa18d3f5640 1 -- 172.21.15.37:0/4273248088 wait complete. 2024-04-25T09:01:26.822 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-04-25T09:01:26.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd206eb3640 1 Processor -- start 2024-04-25T09:01:26.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd206eb3640 1 -- start start 2024-04-25T09:01:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd206eb3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd20006ea70 0x7fd2000af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd206eb3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd20006c3b0 0x7fd2000cd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd206eb3640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd20015bb30 con 0x7fd20015b750 2024-04-25T09:01:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd206eb3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2000cd570 con 0x7fd20006c3b0 2024-04-25T09:01:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd206eb3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20006ef00 con 0x7fd20006ea70 2024-04-25T09:01:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd204c4f640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd20015b750 0x7fd20006bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48878/0 (socket says 172.21.15.37:48878) 2024-04-25T09:01:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd204c4f640 1 -- 172.21.15.37:0/2707398668 learned_addr learned my addr 172.21.15.37:0/2707398668 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:26.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd205450640 1 --2- 172.21.15.37:0/2707398668 >> v2:172.21.15.203:3300/0 conn(0x7fd20006c3b0 0x7fd2000cd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd1fffff640 1 --2- 172.21.15.37:0/2707398668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd20006ea70 0x7fd2000af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd1ff7fe640 1 -- 172.21.15.37:0/2707398668 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378599025 0 0) 0x7fd20015bb30 con 0x7fd20015b750 2024-04-25T09:01:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd1ff7fe640 1 -- 172.21.15.37:0/2707398668 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1e0003530 con 0x7fd20015b750 2024-04-25T09:01:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd205450640 1 -- 172.21.15.37:0/2707398668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd20006ea70 msgr2=0x7fd2000af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd205450640 1 --2- 172.21.15.37:0/2707398668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd20006ea70 0x7fd2000af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd205450640 1 -- 172.21.15.37:0/2707398668 >> v1:172.21.15.37:6789/0 conn(0x7fd20015b750 legacy=0x7fd20006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.958+0000 7fd205450640 1 -- 172.21.15.37:0/2707398668 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20006f260 con 0x7fd20006c3b0 2024-04-25T09:01:26.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd205450640 1 --2- 172.21.15.37:0/2707398668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd20006c3b0 0x7fd2000cd030 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80097c0 tx=0x7fd1f801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9849beb841ebf296 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd1ff7fe640 1 -- 172.21.15.37:0/2707398668 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80b0070 con 0x7fd20006c3b0 2024-04-25T09:01:26.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd1ff7fe640 1 -- 172.21.15.37:0/2707398668 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f80ab600 con 0x7fd20006c3b0 2024-04-25T09:01:26.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd1ff7fe640 1 -- 172.21.15.37:0/2707398668 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80ab9d0 con 0x7fd20006c3b0 2024-04-25T09:01:26.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd206eb3640 1 -- 172.21.15.37:0/2707398668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd20006c3b0 msgr2=0x7fd2000cd030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd206eb3640 1 --2- 172.21.15.37:0/2707398668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd20006c3b0 0x7fd2000cd030 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd1f80097c0 tx=0x7fd1f801c410 comp rx=0 tx=0).stop 2024-04-25T09:01:26.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd206eb3640 1 -- 172.21.15.37:0/2707398668 shutdown_connections 2024-04-25T09:01:26.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd206eb3640 1 --2- 172.21.15.37:0/2707398668 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd20006c3b0 0x7fd2000cd030 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd206eb3640 1 --2- 172.21.15.37:0/2707398668 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd20006ea70 0x7fd2000af240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd206eb3640 1 -- 172.21.15.37:0/2707398668 >> 172.21.15.37:0/2707398668 conn(0x7fd2000717f0 msgr2=0x7fd200071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:26.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd206eb3640 1 -- 172.21.15.37:0/2707398668 shutdown_connections 2024-04-25T09:01:26.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd206eb3640 1 -- 172.21.15.37:0/2707398668 wait complete. 2024-04-25T09:01:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd206eb3640 1 Processor -- start 2024-04-25T09:01:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.962+0000 7fd206eb3640 1 -- start start 2024-04-25T09:01:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd206eb3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd20006c3b0 0x7fd2000cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd206eb3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd20015b750 0x7fd2001542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd206eb3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd20014ef30 0x7fd20013e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd206eb3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000cf420 con 0x7fd20014ef30 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd206eb3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000cf2a0 con 0x7fd20015b750 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd206eb3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2000cf5a0 con 0x7fd20006c3b0 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd205450640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd20014ef30 0x7fd20013e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd205450640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd20014ef30 0x7fd20013e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60716/0 (socket says 172.21.15.37:60716) 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd205450640 1 -- 172.21.15.37:0/1765494645 learned_addr learned my addr 172.21.15.37:0/1765494645 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd1fffff640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd20015b750 0x7fd2001542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd205450640 1 -- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd20006c3b0 msgr2=0x7fd2000cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd204c4f640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd20006c3b0 0x7fd2000cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd205450640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd20006c3b0 0x7fd2000cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd205450640 1 -- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd20015b750 msgr2=0x7fd2001542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd205450640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd20015b750 0x7fd2001542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd205450640 1 -- 172.21.15.37:0/1765494645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2001549d0 con 0x7fd20014ef30 2024-04-25T09:01:26.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd204c4f640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd20006c3b0 0x7fd2000cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd205450640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd20014ef30 0x7fd20013e6c0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd1f801aac0 tx=0x7fd1f80191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd1fd7fa640 1 -- 172.21.15.37:0/1765494645 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80b0040 con 0x7fd20014ef30 2024-04-25T09:01:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd1fd7fa640 1 -- 172.21.15.37:0/1765494645 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1f80b9070 con 0x7fd20014ef30 2024-04-25T09:01:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd206eb3640 1 -- 172.21.15.37:0/1765494645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd200154cb0 con 0x7fd20014ef30 2024-04-25T09:01:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd1fd7fa640 1 -- 172.21.15.37:0/1765494645 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8006e20 con 0x7fd20014ef30 2024-04-25T09:01:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd206eb3640 1 -- 172.21.15.37:0/1765494645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20013dec0 con 0x7fd20014ef30 2024-04-25T09:01:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd206eb3640 1 -- 172.21.15.37:0/1765494645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd20004efa0 con 0x7fd20014ef30 2024-04-25T09:01:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd1fd7fa640 1 -- 172.21.15.37:0/1765494645 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd1f80b4650 con 0x7fd20014ef30 2024-04-25T09:01:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd1fd7fa640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd1e4066f60 0x7fd1e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd1fd7fa640 1 -- 172.21.15.37:0/1765494645 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fd1f811ec00 con 0x7fd20014ef30 2024-04-25T09:01:26.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.966+0000 7fd204c4f640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd1e4066f60 0x7fd1e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.970+0000 7fd204c4f640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd1e4066f60 0x7fd1e4069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005890 tx=0x7fd1f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:26.970+0000 7fd1fd7fa640 1 -- 172.21.15.37:0/1765494645 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd1f8016090 con 0x7fd20014ef30 2024-04-25T09:01:27.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.194+0000 7fd206eb3640 1 -- 172.21.15.37:0/1765494645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) v1 -- 0x7fd200072450 con 0x7fd20014ef30 2024-04-25T09:01:27.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.482+0000 7fd1fd7fa640 1 -- 172.21.15.37:0/1765494645 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v428) v1 ==== 167+0+0 (secure 0 0 0) 0x7fd1f80efc30 con 0x7fd20014ef30 2024-04-25T09:01:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 -- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd1e4066f60 msgr2=0x7fd1e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd1e4066f60 0x7fd1e4069420 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0005890 tx=0x7fd1f0009040 comp rx=0 tx=0).stop 2024-04-25T09:01:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 -- 172.21.15.37:0/1765494645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd20014ef30 msgr2=0x7fd20013e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd20014ef30 0x7fd20013e6c0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd1f801aac0 tx=0x7fd1f80191b0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 -- 172.21.15.37:0/1765494645 shutdown_connections 2024-04-25T09:01:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd1e4066f60 0x7fd1e4069420 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd20014ef30 0x7fd20013e6c0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd20015b750 0x7fd2001542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 --2- 172.21.15.37:0/1765494645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd20006c3b0 0x7fd2000cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 -- 172.21.15.37:0/1765494645 >> 172.21.15.37:0/1765494645 conn(0x7fd2000717f0 msgr2=0x7fd20006d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 -- 172.21.15.37:0/1765494645 shutdown_connections 2024-04-25T09:01:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.486+0000 7fd206eb3640 1 -- 172.21.15.37:0/1765494645 wait complete. 2024-04-25T09:01:27.495 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-04-25T09:01:27.495 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T09:01:27.496 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:27.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf7b6e640 1 Processor -- start 2024-04-25T09:01:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf7b6e640 1 -- start start 2024-04-25T09:01:27.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf7b6e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cf0159ea0 0x7f2cf00b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf7b6e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2cf0071fd0 0x7f2cf015a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf7b6e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf015ae30 con 0x7f2cf015a280 2024-04-25T09:01:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf7b6e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf00b2990 con 0x7f2cf0071fd0 2024-04-25T09:01:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf7b6e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf015b110 con 0x7f2cf0159ea0 2024-04-25T09:01:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf5109640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2cf015a280 0x7f2cf00718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48884/0 (socket says 172.21.15.37:48884) 2024-04-25T09:01:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf610b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2cf0071fd0 0x7f2cf015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf5109640 1 -- 172.21.15.37:0/2998586072 learned_addr learned my addr 172.21.15.37:0/2998586072 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf590a640 1 --2- 172.21.15.37:0/2998586072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cf0159ea0 0x7f2cf00b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf610b640 1 -- 172.21.15.37:0/2998586072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cf0159ea0 msgr2=0x7f2cf00b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf610b640 1 --2- 172.21.15.37:0/2998586072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cf0159ea0 0x7f2cf00b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf4908640 1 -- 172.21.15.37:0/2998586072 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290354671 0 0) 0x7f2cf015ae30 con 0x7f2cf015a280 2024-04-25T09:01:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf610b640 1 -- 172.21.15.37:0/2998586072 >> v1:172.21.15.37:6789/0 conn(0x7f2cf015a280 legacy=0x7f2cf00718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.630+0000 7f2cf610b640 1 -- 172.21.15.37:0/2998586072 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf00b2620 con 0x7f2cf0071fd0 2024-04-25T09:01:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf610b640 1 --2- 172.21.15.37:0/2998586072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf0071fd0 0x7f2cf015a700 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009d90 tx=0x7f2ce801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c06030bad84ce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf4908640 1 -- 172.21.15.37:0/2998586072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80b2070 con 0x7f2cf0071fd0 2024-04-25T09:01:27.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf4908640 1 -- 172.21.15.37:0/2998586072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce80a58c0 con 0x7f2cf0071fd0 2024-04-25T09:01:27.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf4908640 1 -- 172.21.15.37:0/2998586072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce80a5c90 con 0x7f2cf0071fd0 2024-04-25T09:01:27.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/2998586072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf0071fd0 msgr2=0x7f2cf015a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 --2- 172.21.15.37:0/2998586072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf0071fd0 0x7f2cf015a700 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f2ce8009d90 tx=0x7f2ce801c970 comp rx=0 tx=0).stop 2024-04-25T09:01:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/2998586072 shutdown_connections 2024-04-25T09:01:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 --2- 172.21.15.37:0/2998586072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf0071fd0 0x7f2cf015a700 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 --2- 172.21.15.37:0/2998586072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cf0159ea0 0x7f2cf00b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/2998586072 >> 172.21.15.37:0/2998586072 conn(0x7f2cf00b11a0 msgr2=0x7f2cf00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/2998586072 shutdown_connections 2024-04-25T09:01:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/2998586072 wait complete. 2024-04-25T09:01:27.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 Processor -- start 2024-04-25T09:01:27.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 -- start start 2024-04-25T09:01:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cf0071fd0 0x7f2cf014ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf0159ea0 0x7f2cf0157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cf015a280 0x7f2cf00af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf006d420 con 0x7f2cf0159ea0 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf006d2a0 con 0x7f2cf0071fd0 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf7b6e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf006d5a0 con 0x7f2cf015a280 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf5109640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf0159ea0 0x7f2cf0157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf5109640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf0159ea0 0x7f2cf0157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60728/0 (socket says 172.21.15.37:60728) 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf5109640 1 -- 172.21.15.37:0/1613706312 learned_addr learned my addr 172.21.15.37:0/1613706312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf610b640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cf015a280 0x7f2cf00af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf590a640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cf0071fd0 0x7f2cf014ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf5109640 1 -- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cf015a280 msgr2=0x7f2cf00af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf5109640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cf015a280 0x7f2cf00af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf5109640 1 -- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cf0071fd0 msgr2=0x7f2cf014ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf5109640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cf0071fd0 0x7f2cf014ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf5109640 1 -- 172.21.15.37:0/1613706312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0158100 con 0x7f2cf0159ea0 2024-04-25T09:01:27.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.634+0000 7f2cf5109640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf0159ea0 0x7f2cf0157790 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c4b0 tx=0x7f2ce000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:27.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.638+0000 7f2ce67fc640 1 -- 172.21.15.37:0/1613706312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0018070 con 0x7f2cf0159ea0 2024-04-25T09:01:27.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.638+0000 7f2ce67fc640 1 -- 172.21.15.37:0/1613706312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce0007500 con 0x7f2cf0159ea0 2024-04-25T09:01:27.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.638+0000 7f2ce67fc640 1 -- 172.21.15.37:0/1613706312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0004a30 con 0x7f2cf0159ea0 2024-04-25T09:01:27.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.638+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/1613706312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf01f9920 con 0x7f2cf0159ea0 2024-04-25T09:01:27.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.638+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/1613706312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf014f010 con 0x7f2cf0159ea0 2024-04-25T09:01:27.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.638+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/1613706312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf004efa0 con 0x7f2cf0159ea0 2024-04-25T09:01:27.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.638+0000 7f2ce67fc640 1 -- 172.21.15.37:0/1613706312 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2ce0004bd0 con 0x7f2cf0159ea0 2024-04-25T09:01:27.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.638+0000 7f2ce67fc640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2cd4066f60 0x7f2cd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.638+0000 7f2ce67fc640 1 -- 172.21.15.37:0/1613706312 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f2ce0084130 con 0x7f2cf0159ea0 2024-04-25T09:01:27.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.638+0000 7f2cf590a640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2cd4066f60 0x7f2cd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.638+0000 7f2cf590a640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2cd4066f60 0x7f2cd4069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2cf015c060 tx=0x7f2cec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:27.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.642+0000 7f2ce67fc640 1 -- 172.21.15.37:0/1613706312 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2ce0055200 con 0x7f2cf0159ea0 2024-04-25T09:01:27.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.806+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/1613706312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2cf01f9f10 con 0x7f2cf0159ea0 2024-04-25T09:01:27.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.806+0000 7f2ce67fc640 1 -- 172.21.15.37:0/1613706312 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+13517 (secure 0 0 0) 0x7f2ce0054b00 con 0x7f2cf0159ea0 2024-04-25T09:01:27.796 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:27.796 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":102,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:01:26.466148+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/3366017865":"2024-04-26T09:01:23.269994+0000","172.21.15.37:6834/3366017865":"2024-04-26T09:01:23.269994+0000","172.21.15.37:6835/1462327322":"2024-04-26T09:00:38.139659+0000","172.21.15.37:6834/1462327322":"2024-04-26T09:00:38.139659+0000","172.21.15.37:6835/354285428":"2024-04-26T08:59:17.095322+0000","172.21.15.37:6834/354285428":"2024-04-26T08:59:17.095322+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2cd4066f60 msgr2=0x7f2cd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2cd4066f60 0x7f2cd4069420 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f2cf015c060 tx=0x7f2cec008040 comp rx=0 tx=0).stop 2024-04-25T09:01:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/1613706312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf0159ea0 msgr2=0x7f2cf0157790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf0159ea0 0x7f2cf0157790 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f2ce000c4b0 tx=0x7f2ce000c970 comp rx=0 tx=0).stop 2024-04-25T09:01:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/1613706312 shutdown_connections 2024-04-25T09:01:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2cd4066f60 0x7f2cd4069420 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2cf015a280 0x7f2cf00af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2cf0159ea0 0x7f2cf0157790 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 --2- 172.21.15.37:0/1613706312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2cf0071fd0 0x7f2cf014ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/1613706312 >> 172.21.15.37:0/1613706312 conn(0x7f2cf00b11a0 msgr2=0x7f2cf00b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/1613706312 shutdown_connections 2024-04-25T09:01:27.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.810+0000 7f2cf7b6e640 1 -- 172.21.15.37:0/1613706312 wait complete. 2024-04-25T09:01:27.816 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:01:27.817 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:01:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.958+0000 7f4729e77640 1 Processor -- start 2024-04-25T09:01:27.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.958+0000 7f4729e77640 1 -- start start 2024-04-25T09:01:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.958+0000 7f4729e77640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47240cf590 0x7f47240cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.958+0000 7f4729e77640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4724072000 0x7f4724072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.958+0000 7f4729e77640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47240cbd20 con 0x7f472415b6b0 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.958+0000 7f4729e77640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f472415ba90 con 0x7f4724072000 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.958+0000 7f4729e77640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47240cff30 con 0x7f47240cf590 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.958+0000 7f47237fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f472415b6b0 0x7f47240ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48896/0 (socket says 172.21.15.37:48896) 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.958+0000 7f47237fe640 1 -- 172.21.15.37:0/1977479549 learned_addr learned my addr 172.21.15.37:0/1977479549 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.958+0000 7f4723fff640 1 --2- 172.21.15.37:0/1977479549 >> v2:172.21.15.37:3300/0 conn(0x7f4724072000 0x7f4724072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4722ffd640 1 --2- 172.21.15.37:0/1977479549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47240cf590 0x7f47240cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f47227fc640 1 -- 172.21.15.37:0/1977479549 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1700255026 0 0) 0x7f47240cbd20 con 0x7f472415b6b0 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f47227fc640 1 -- 172.21.15.37:0/1977479549 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f470c003530 con 0x7f472415b6b0 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4723fff640 1 -- 172.21.15.37:0/1977479549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47240cf590 msgr2=0x7f47240cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4723fff640 1 --2- 172.21.15.37:0/1977479549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47240cf590 0x7f47240cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4723fff640 1 -- 172.21.15.37:0/1977479549 >> v1:172.21.15.37:6789/0 conn(0x7f472415b6b0 legacy=0x7f47240ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4723fff640 1 -- 172.21.15.37:0/1977479549 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47240cc000 con 0x7f4724072000 2024-04-25T09:01:27.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4723fff640 1 --2- 172.21.15.37:0/1977479549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4724072000 0x7f4724072470 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f471c009d90 tx=0x7f471c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da77f3580835d005 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:27.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f47227fc640 1 -- 172.21.15.37:0/1977479549 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c0b2070 con 0x7f4724072000 2024-04-25T09:01:27.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f47227fc640 1 -- 172.21.15.37:0/1977479549 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471c0a5a00 con 0x7f4724072000 2024-04-25T09:01:27.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f47227fc640 1 -- 172.21.15.37:0/1977479549 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471c0a5db0 con 0x7f4724072000 2024-04-25T09:01:27.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4729e77640 1 -- 172.21.15.37:0/1977479549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4724072000 msgr2=0x7f4724072470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4729e77640 1 --2- 172.21.15.37:0/1977479549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4724072000 0x7f4724072470 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f471c009d90 tx=0x7f471c01c940 comp rx=0 tx=0).stop 2024-04-25T09:01:27.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4729e77640 1 -- 172.21.15.37:0/1977479549 shutdown_connections 2024-04-25T09:01:27.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4729e77640 1 --2- 172.21.15.37:0/1977479549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4724072000 0x7f4724072470 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4729e77640 1 --2- 172.21.15.37:0/1977479549 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47240cf590 0x7f47240cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4729e77640 1 -- 172.21.15.37:0/1977479549 >> 172.21.15.37:0/1977479549 conn(0x7f472406ae60 msgr2=0x7f472406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:27.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4729e77640 1 -- 172.21.15.37:0/1977479549 shutdown_connections 2024-04-25T09:01:27.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.962+0000 7f4729e77640 1 -- 172.21.15.37:0/1977479549 wait complete. 2024-04-25T09:01:27.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4729e77640 1 Processor -- start 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4729e77640 1 -- start start 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4729e77640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4724072000 0x7f47241551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4729e77640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47240cf590 0x7f472415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4729e77640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f472415b6b0 0x7f472415bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4729e77640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240cadb0 con 0x7f4724072000 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4729e77640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47240cba30 con 0x7f47240cf590 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4729e77640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47240caf30 con 0x7f472415b6b0 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f47237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4724072000 0x7f47241551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f47237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4724072000 0x7f47241551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60746/0 (socket says 172.21.15.37:60746) 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f47237fe640 1 -- 172.21.15.37:0/3198500865 learned_addr learned my addr 172.21.15.37:0/3198500865 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4723fff640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f472415b6b0 0x7f472415bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4722ffd640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47240cf590 0x7f472415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f47237fe640 1 -- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f472415b6b0 msgr2=0x7f472415bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f47237fe640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f472415b6b0 0x7f472415bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f47237fe640 1 -- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47240cf590 msgr2=0x7f472415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f47237fe640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47240cf590 0x7f472415a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f47237fe640 1 -- 172.21.15.37:0/3198500865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47240ccfd0 con 0x7f4724072000 2024-04-25T09:01:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f47237fe640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4724072000 0x7f47241551a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f471400c3c0 tx=0x7f471400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f470bfff640 1 -- 172.21.15.37:0/3198500865 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4714017070 con 0x7f4724072000 2024-04-25T09:01:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f470bfff640 1 -- 172.21.15.37:0/3198500865 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4714005cf0 con 0x7f4724072000 2024-04-25T09:01:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4729e77640 1 -- 172.21.15.37:0/3198500865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47241556e0 con 0x7f4724072000 2024-04-25T09:01:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f470bfff640 1 -- 172.21.15.37:0/3198500865 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140072f0 con 0x7f4724072000 2024-04-25T09:01:27.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4729e77640 1 -- 172.21.15.37:0/3198500865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47240cd240 con 0x7f4724072000 2024-04-25T09:01:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f4729e77640 1 -- 172.21.15.37:0/3198500865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472404efa0 con 0x7f4724072000 2024-04-25T09:01:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.966+0000 7f470bfff640 1 -- 172.21.15.37:0/3198500865 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f471401a070 con 0x7f4724072000 2024-04-25T09:01:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.970+0000 7f470bfff640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4710066f60 0x7f4710069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.970+0000 7f470bfff640 1 -- 172.21.15.37:0/3198500865 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f47140842e0 con 0x7f4724072000 2024-04-25T09:01:27.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.970+0000 7f4722ffd640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4710066f60 0x7f4710069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.970+0000 7f4722ffd640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4710066f60 0x7f4710069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4724073f50 tx=0x7f4718008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:27.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:27.974+0000 7f470bfff640 1 -- 172.21.15.37:0/3198500865 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4714055440 con 0x7f4724072000 2024-04-25T09:01:28.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.134+0000 7f4729e77640 1 -- 172.21.15.37:0/3198500865 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47240b3180 con 0x7f4724072000 2024-04-25T09:01:28.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.134+0000 7f470bfff640 1 -- 172.21.15.37:0/3198500865 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4714054d40 con 0x7f4724072000 2024-04-25T09:01:28.123 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:01:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.134+0000 7f4729e77640 1 -- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4710066f60 msgr2=0x7f4710069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.134+0000 7f4729e77640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4710066f60 0x7f4710069420 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4724073f50 tx=0x7f4718008040 comp rx=0 tx=0).stop 2024-04-25T09:01:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.134+0000 7f4729e77640 1 -- 172.21.15.37:0/3198500865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4724072000 msgr2=0x7f47241551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.134+0000 7f4729e77640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4724072000 0x7f47241551a0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f471400c3c0 tx=0x7f471400c880 comp rx=0 tx=0).stop 2024-04-25T09:01:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.138+0000 7f4729e77640 1 -- 172.21.15.37:0/3198500865 shutdown_connections 2024-04-25T09:01:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.138+0000 7f4729e77640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4710066f60 0x7f4710069420 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.138+0000 7f4729e77640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f472415b6b0 0x7f472415bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.138+0000 7f4729e77640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47240cf590 0x7f472415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.138+0000 7f4729e77640 1 --2- 172.21.15.37:0/3198500865 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4724072000 0x7f47241551a0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.138+0000 7f4729e77640 1 -- 172.21.15.37:0/3198500865 >> 172.21.15.37:0/3198500865 conn(0x7f472406ae60 msgr2=0x7f472415fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.138+0000 7f4729e77640 1 -- 172.21.15.37:0/3198500865 shutdown_connections 2024-04-25T09:01:28.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.138+0000 7f4729e77640 1 -- 172.21.15.37:0/3198500865 wait complete. 2024-04-25T09:01:28.143 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f700b17c640 1 Processor -- start 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f700b17c640 1 -- start start 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f700b17c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f70040d07f0 0x7f70040d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f700b17c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70040d1cd0 0x7f700406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f700b17c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700406be60 con 0x7f700415da80 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f700b17c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700406c440 con 0x7f70040d07f0 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f700b17c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700415de60 con 0x7f70040d1cd0 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f7003fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f70040d07f0 0x7f70040d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f7003fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f70040d07f0 0x7f70040d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60754/0 (socket says 172.21.15.37:60754) 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f7003fff640 1 -- 172.21.15.37:0/74119902 learned_addr learned my addr 172.21.15.37:0/74119902 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f7009719640 1 --2- 172.21.15.37:0/74119902 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70040d1cd0 0x7f700406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f7003fff640 1 -- 172.21.15.37:0/74119902 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70040d1cd0 msgr2=0x7f700406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f7003fff640 1 --2- 172.21.15.37:0/74119902 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70040d1cd0 0x7f700406c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f7003fff640 1 -- 172.21.15.37:0/74119902 >> v1:172.21.15.203:6789/0 conn(0x7f700415da80 legacy=0x7f70040d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.282+0000 7f7003fff640 1 -- 172.21.15.37:0/74119902 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70040d2450 con 0x7f70040d07f0 2024-04-25T09:01:28.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f7003fff640 1 --2- 172.21.15.37:0/74119902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70040d07f0 0x7f70040d4230 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f6fec006fd0 tx=0x7f6fec01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff6c21ee2ac592d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:28.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f70037fe640 1 -- 172.21.15.37:0/74119902 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0a7070 con 0x7f70040d07f0 2024-04-25T09:01:28.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f70037fe640 1 -- 172.21.15.37:0/74119902 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec0b0070 con 0x7f70040d07f0 2024-04-25T09:01:28.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f70037fe640 1 -- 172.21.15.37:0/74119902 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec0ab890 con 0x7f70040d07f0 2024-04-25T09:01:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 -- 172.21.15.37:0/74119902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70040d07f0 msgr2=0x7f70040d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 --2- 172.21.15.37:0/74119902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70040d07f0 0x7f70040d4230 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f6fec006fd0 tx=0x7f6fec01c510 comp rx=0 tx=0).stop 2024-04-25T09:01:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 -- 172.21.15.37:0/74119902 shutdown_connections 2024-04-25T09:01:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 --2- 172.21.15.37:0/74119902 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70040d1cd0 0x7f700406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 --2- 172.21.15.37:0/74119902 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70040d07f0 0x7f70040d4230 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 -- 172.21.15.37:0/74119902 >> 172.21.15.37:0/74119902 conn(0x7f700408b000 msgr2=0x7f70040d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 -- 172.21.15.37:0/74119902 shutdown_connections 2024-04-25T09:01:28.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 -- 172.21.15.37:0/74119902 wait complete. 2024-04-25T09:01:28.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 Processor -- start 2024-04-25T09:01:28.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 -- start start 2024-04-25T09:01:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70040d07f0 0x7f70040cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70040d1cd0 0x7f7004153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700415da80 0x7f70040cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700406d7a0 con 0x7f700415da80 2024-04-25T09:01:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700406d620 con 0x7f70040d07f0 2024-04-25T09:01:28.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f700b17c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700406d920 con 0x7f70040d1cd0 2024-04-25T09:01:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f7009719640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700415da80 0x7f70040cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f7009719640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700415da80 0x7f70040cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60764/0 (socket says 172.21.15.37:60764) 2024-04-25T09:01:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.286+0000 7f7009719640 1 -- 172.21.15.37:0/1832447323 learned_addr learned my addr 172.21.15.37:0/1832447323 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f7008f18640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70040d07f0 0x7f70040cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f7003fff640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70040d1cd0 0x7f7004153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f7009719640 1 -- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70040d1cd0 msgr2=0x7f7004153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f7009719640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70040d1cd0 0x7f7004153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f7009719640 1 -- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70040d07f0 msgr2=0x7f70040cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f7009719640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70040d07f0 0x7f70040cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f7009719640 1 -- 172.21.15.37:0/1832447323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700415ba50 con 0x7f700415da80 2024-04-25T09:01:28.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f7003fff640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70040d1cd0 0x7f7004153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:28.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f7009719640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700415da80 0x7f70040cc9e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400c820 tx=0x7f6ff400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:28.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f70017fa640 1 -- 172.21.15.37:0/1832447323 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4015070 con 0x7f700415da80 2024-04-25T09:01:28.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f70017fa640 1 -- 172.21.15.37:0/1832447323 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff4004220 con 0x7f700415da80 2024-04-25T09:01:28.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f70017fa640 1 -- 172.21.15.37:0/1832447323 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff400ce10 con 0x7f700415da80 2024-04-25T09:01:28.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f700b17c640 1 -- 172.21.15.37:0/1832447323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700415bd30 con 0x7f700415da80 2024-04-25T09:01:28.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f700b17c640 1 -- 172.21.15.37:0/1832447323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700415c5d0 con 0x7f700415da80 2024-04-25T09:01:28.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f700b17c640 1 -- 172.21.15.37:0/1832447323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700415c120 con 0x7f700415da80 2024-04-25T09:01:28.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f70017fa640 1 -- 172.21.15.37:0/1832447323 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ff4007830 con 0x7f700415da80 2024-04-25T09:01:28.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f70017fa640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fe4066f60 0x7f6fe4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f70017fa640 1 -- 172.21.15.37:0/1832447323 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f6ff40851e0 con 0x7f700415da80 2024-04-25T09:01:28.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f7008f18640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fe4066f60 0x7f6fe4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.290+0000 7f7008f18640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fe4066f60 0x7f6fe4069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f70041530c0 tx=0x7f6ffc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:28.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.294+0000 7f70017fa640 1 -- 172.21.15.37:0/1832447323 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ff40563e0 con 0x7f700415da80 2024-04-25T09:01:28.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:28.510+0000 7f700b17c640 1 -- 172.21.15.37:0/1832447323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7f700404efa0 con 0x7f700415da80 2024-04-25T09:01:29.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.494+0000 7f70017fa640 1 -- 172.21.15.37:0/1832447323 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v430) v1 ==== 149+0+0 (secure 0 0 0) 0x7f6ff4055ce0 con 0x7f700415da80 2024-04-25T09:01:29.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 -- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fe4066f60 msgr2=0x7f6fe4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fe4066f60 0x7f6fe4069420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f70041530c0 tx=0x7f6ffc009040 comp rx=0 tx=0).stop 2024-04-25T09:01:29.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 -- 172.21.15.37:0/1832447323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700415da80 msgr2=0x7f70040cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700415da80 0x7f70040cc9e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400c820 tx=0x7f6ff400cce0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 -- 172.21.15.37:0/1832447323 shutdown_connections 2024-04-25T09:01:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6fe4066f60 0x7f6fe4069420 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700415da80 0x7f70040cc9e0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f70040d1cd0 0x7f7004153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 --2- 172.21.15.37:0/1832447323 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f70040d07f0 0x7f70040cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 -- 172.21.15.37:0/1832447323 >> 172.21.15.37:0/1832447323 conn(0x7f700408b000 msgr2=0x7f70040b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 -- 172.21.15.37:0/1832447323 shutdown_connections 2024-04-25T09:01:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.498+0000 7f700b17c640 1 -- 172.21.15.37:0/1832447323 wait complete. 2024-04-25T09:01:29.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:01:29.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.650+0000 7f76ad7cf640 1 Processor -- start 2024-04-25T09:01:29.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.650+0000 7f76ad7cf640 1 -- start start 2024-04-25T09:01:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.650+0000 7f76ad7cf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f76a80d07f0 0x7f76a80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.650+0000 7f76ad7cf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a806be60 con 0x7f76a815da80 2024-04-25T09:01:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.650+0000 7f76ad7cf640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a815de60 con 0x7f76a80d1cd0 2024-04-25T09:01:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.650+0000 7f76ad7cf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a806c440 con 0x7f76a80d07f0 2024-04-25T09:01:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.650+0000 7f76a6ffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f76a815da80 0x7f76a80d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50466/0 (socket says 172.21.15.37:50466) 2024-04-25T09:01:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.650+0000 7f76a6ffd640 1 -- 172.21.15.37:0/1458607347 learned_addr learned my addr 172.21.15.37:0/1458607347 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a67fc640 1 --2- 172.21.15.37:0/1458607347 >> v2:172.21.15.203:3300/0 conn(0x7f76a80d07f0 0x7f76a80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3748784772 0 0) 0x7f76a806be60 con 0x7f76a815da80 2024-04-25T09:01:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7690003530 con 0x7f76a815da80 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3151533468 0 0) 0x7f7690003530 con 0x7f76a815da80 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 >> v2:172.21.15.203:3300/0 conn(0x7f76a80d07f0 msgr2=0x7f76a80d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 --2- 172.21.15.37:0/1458607347 >> v2:172.21.15.203:3300/0 conn(0x7f76a80d07f0 0x7f76a80d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 >> v1:172.21.15.203:6789/0 conn(0x7f76a80d1cd0 legacy=0x7f76a806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80d2450 con 0x7f76a815da80 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f7698001680 con 0x7f76a815da80 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 >> v1:172.21.15.37:6789/0 conn(0x7f76a815da80 legacy=0x7f76a80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7690003a30 0x7f7690003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7690004810 0x7f7690004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76900051d0 0x7f769000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a806c440 con 0x7f76900051d0 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769000c100 con 0x7f7690003a30 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769000c300 con 0x7f7690004810 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a77fe640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76900051d0 0x7f769000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a67fc640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7690004810 0x7f7690004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a6ffd640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7690003a30 0x7f7690003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a77fe640 1 -- 172.21.15.37:0/1458607347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7690004810 msgr2=0x7f7690004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:29.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a77fe640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7690004810 0x7f7690004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a77fe640 1 -- 172.21.15.37:0/1458607347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7690003a30 msgr2=0x7f7690003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:29.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a77fe640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7690003a30 0x7f7690003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a77fe640 1 -- 172.21.15.37:0/1458607347 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690003530 con 0x7f76900051d0 2024-04-25T09:01:29.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a6ffd640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7690003a30 0x7f7690003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:29.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a77fe640 1 -- 172.21.15.37:0/1458607347 reap_dead start 2024-04-25T09:01:29.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a77fe640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76900051d0 0x7f769000b9d0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f76a0001e70 tx=0x7f76a001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb188a72442a9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:29.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0001a90 con 0x7f76900051d0 2024-04-25T09:01:29.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a0003bc0 con 0x7f76900051d0 2024-04-25T09:01:29.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.654+0000 7f76a5ffb640 1 -- 172.21.15.37:0/1458607347 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0005020 con 0x7f76900051d0 2024-04-25T09:01:29.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 -- 172.21.15.37:0/1458607347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76900051d0 msgr2=0x7f769000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76900051d0 0x7f769000b9d0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f76a0001e70 tx=0x7f76a001c780 comp rx=0 tx=0).stop 2024-04-25T09:01:29.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 -- 172.21.15.37:0/1458607347 shutdown_connections 2024-04-25T09:01:29.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 --2- 172.21.15.37:0/1458607347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76900051d0 0x7f769000b9d0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 -- 172.21.15.37:0/1458607347 >> 172.21.15.37:0/1458607347 conn(0x7f76a808b000 msgr2=0x7f76a80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:29.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 -- 172.21.15.37:0/1458607347 shutdown_connections 2024-04-25T09:01:29.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 -- 172.21.15.37:0/1458607347 wait complete. 2024-04-25T09:01:29.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 Processor -- start 2024-04-25T09:01:29.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 -- start start 2024-04-25T09:01:29.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76900051d0 0x7f76a814fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76a80b6e30 0x7f76a80cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a80cb710 0x7f76a80cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80d2870 con 0x7f76a80cb710 2024-04-25T09:01:29.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80d26f0 con 0x7f76a80b6e30 2024-04-25T09:01:29.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76ad7cf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a80d20f0 con 0x7f76900051d0 2024-04-25T09:01:29.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a80cb710 0x7f76a80cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76a77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a80cb710 0x7f76a80cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35266/0 (socket says 172.21.15.37:35266) 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76a67fc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76a80b6e30 0x7f76a80cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76a77fe640 1 -- 172.21.15.37:0/2307082684 learned_addr learned my addr 172.21.15.37:0/2307082684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76a6ffd640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76900051d0 0x7f76a814fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76a77fe640 1 -- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76900051d0 msgr2=0x7f76a814fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76a77fe640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76900051d0 0x7f76a814fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76a77fe640 1 -- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76a80b6e30 msgr2=0x7f76a80cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76a77fe640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76a80b6e30 0x7f76a80cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76a77fe640 1 -- 172.21.15.37:0/2307082684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a80cbc20 con 0x7f76a80cb710 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.658+0000 7f76a77fe640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a80cb710 0x7f76a80cc460 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f76a001ccf0 tx=0x7f7690005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.662+0000 7f768f7fe640 1 -- 172.21.15.37:0/2307082684 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0001a90 con 0x7f76a80cb710 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.662+0000 7f768f7fe640 1 -- 172.21.15.37:0/2307082684 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a0005820 con 0x7f76a80cb710 2024-04-25T09:01:29.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.662+0000 7f768f7fe640 1 -- 172.21.15.37:0/2307082684 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0007dc0 con 0x7f76a80cb710 2024-04-25T09:01:29.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.662+0000 7f76ad7cf640 1 -- 172.21.15.37:0/2307082684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a815e280 con 0x7f76a80cb710 2024-04-25T09:01:29.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.662+0000 7f76ad7cf640 1 -- 172.21.15.37:0/2307082684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a815e840 con 0x7f76a80cb710 2024-04-25T09:01:29.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.662+0000 7f76ad7cf640 1 -- 172.21.15.37:0/2307082684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a804efa0 con 0x7f76a80cb710 2024-04-25T09:01:29.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.662+0000 7f768f7fe640 1 -- 172.21.15.37:0/2307082684 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76a00a6480 con 0x7f76a80cb710 2024-04-25T09:01:29.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.662+0000 7f768f7fe640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7694066f90 0x7f7694069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.662+0000 7f768f7fe640 1 -- 172.21.15.37:0/2307082684 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f76a011e3d0 con 0x7f76a80cb710 2024-04-25T09:01:29.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.662+0000 7f76a6ffd640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7694066f90 0x7f7694069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.662+0000 7f76a6ffd640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7694066f90 0x7f7694069450 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f76980017f0 tx=0x7f7698009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:29.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.666+0000 7f768f7fe640 1 -- 172.21.15.37:0/2307082684 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76a0016490 con 0x7f76a80cb710 2024-04-25T09:01:29.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.826+0000 7f76ad7cf640 1 -- 172.21.15.37:0/2307082684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f76a80b20a0 con 0x7f76a80cb710 2024-04-25T09:01:29.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.830+0000 7f768f7fe640 1 -- 172.21.15.37:0/2307082684 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f76a00eeee0 con 0x7f76a80cb710 2024-04-25T09:01:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 -- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7694066f90 msgr2=0x7f7694069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7694066f90 0x7f7694069450 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f76980017f0 tx=0x7f7698009040 comp rx=0 tx=0).stop 2024-04-25T09:01:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 -- 172.21.15.37:0/2307082684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a80cb710 msgr2=0x7f76a80cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a80cb710 0x7f76a80cc460 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f76a001ccf0 tx=0x7f7690005840 comp rx=0 tx=0).stop 2024-04-25T09:01:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 -- 172.21.15.37:0/2307082684 shutdown_connections 2024-04-25T09:01:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7694066f90 0x7f7694069450 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76a80cb710 0x7f76a80cc460 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76a80b6e30 0x7f76a80cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 --2- 172.21.15.37:0/2307082684 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76900051d0 0x7f76a814fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 -- 172.21.15.37:0/2307082684 >> 172.21.15.37:0/2307082684 conn(0x7f76a808b000 msgr2=0x7f76a80b2660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 -- 172.21.15.37:0/2307082684 shutdown_connections 2024-04-25T09:01:29.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:29.834+0000 7f76ad7cf640 1 -- 172.21.15.37:0/2307082684 wait complete. 2024-04-25T09:01:29.840 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:01:29.840 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:01:29.883 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:01:29.952 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:01:29.952 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:01:29.953 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:29.959 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:01:29.959 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:01:29.959 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:29.964 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:01:29.964 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:01:29.964 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:30.006 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:01:30.006 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:01:30.006 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:30.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:01:30.012 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:01:30.012 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:30.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.190+0000 7f6b6f8fb640 1 Processor -- start 2024-04-25T09:01:30.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6f8fb640 1 -- start start 2024-04-25T09:01:30.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6f8fb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6b6809a480 0x7f6b680cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6f8fb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6b680cfb80 0x7f6b680cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6f8fb640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b680d04d0 con 0x7f6b6815b950 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6f8fb640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b6815bd30 con 0x7f6b680cfb80 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6f8fb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b680cc0d0 con 0x7f6b6809a480 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6de98640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6b680cfb80 0x7f6b680cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6d697640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6b6815b950 0x7f6b680cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50478/0 (socket says 172.21.15.37:50478) 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6d697640 1 -- 172.21.15.37:0/3981987715 learned_addr learned my addr 172.21.15.37:0/3981987715 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6ce96640 1 --2- 172.21.15.37:0/3981987715 >> v2:172.21.15.203:3300/0 conn(0x7f6b6809a480 0x7f6b680cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6de98640 1 -- 172.21.15.37:0/3981987715 >> v2:172.21.15.203:3300/0 conn(0x7f6b6809a480 msgr2=0x7f6b680cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6de98640 1 --2- 172.21.15.37:0/3981987715 >> v2:172.21.15.203:3300/0 conn(0x7f6b6809a480 0x7f6b680cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6de98640 1 -- 172.21.15.37:0/3981987715 >> v1:172.21.15.37:6789/0 conn(0x7f6b6815b950 legacy=0x7f6b680cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6de98640 1 -- 172.21.15.37:0/3981987715 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b6809ab10 con 0x7f6b680cfb80 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b5ffff640 1 -- 172.21.15.37:0/3981987715 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 74160207 0 0) 0x7f6b58001120 con 0x7f6b6815b950 2024-04-25T09:01:30.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6ce96640 1 --2- 172.21.15.37:0/3981987715 >> v2:172.21.15.203:3300/0 conn(0x7f6b6809a480 0x7f6b680cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.194+0000 7f6b6de98640 1 --2- 172.21.15.37:0/3981987715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b680cfb80 0x7f6b680cff90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f6b68072710 tx=0x7f6b6001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4d3ce1b27ec548 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b5ffff640 1 -- 172.21.15.37:0/3981987715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600b2070 con 0x7f6b680cfb80 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b5ffff640 1 -- 172.21.15.37:0/3981987715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b600a58e0 con 0x7f6b680cfb80 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b5ffff640 1 -- 172.21.15.37:0/3981987715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600a5cb0 con 0x7f6b680cfb80 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/3981987715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b680cfb80 msgr2=0x7f6b680cff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 --2- 172.21.15.37:0/3981987715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b680cfb80 0x7f6b680cff90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f6b68072710 tx=0x7f6b6001c930 comp rx=0 tx=0).stop 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/3981987715 shutdown_connections 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 --2- 172.21.15.37:0/3981987715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b680cfb80 0x7f6b680cff90 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 --2- 172.21.15.37:0/3981987715 >> v2:172.21.15.203:3300/0 conn(0x7f6b6809a480 0x7f6b680cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/3981987715 >> 172.21.15.37:0/3981987715 conn(0x7f6b680b30b0 msgr2=0x7f6b680b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/3981987715 shutdown_connections 2024-04-25T09:01:30.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/3981987715 wait complete. 2024-04-25T09:01:30.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 Processor -- start 2024-04-25T09:01:30.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 -- start start 2024-04-25T09:01:30.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b6809a480 0x7f6b680c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b680cfb80 0x7f6b6814fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b6815b950 0x7f6b680c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680cd390 con 0x7f6b6815b950 2024-04-25T09:01:30.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6806faa0 con 0x7f6b680cfb80 2024-04-25T09:01:30.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b680b57e0 con 0x7f6b6809a480 2024-04-25T09:01:30.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6de98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b6815b950 0x7f6b680c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6de98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b6815b950 0x7f6b680c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35286/0 (socket says 172.21.15.37:35286) 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6de98640 1 -- 172.21.15.37:0/600337631 learned_addr learned my addr 172.21.15.37:0/600337631 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6d697640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b6809a480 0x7f6b680c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6ce96640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b680cfb80 0x7f6b6814fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6de98640 1 -- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b6809a480 msgr2=0x7f6b680c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6de98640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b6809a480 0x7f6b680c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6de98640 1 -- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b680cfb80 msgr2=0x7f6b6814fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6de98640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b680cfb80 0x7f6b6814fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6de98640 1 -- 172.21.15.37:0/600337631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b68151ac0 con 0x7f6b6815b950 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6d697640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b6809a480 0x7f6b680c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6de98640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b6815b950 0x7f6b680c7600 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f6b6001a560 tx=0x7f6b600196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b5dffb640 1 -- 172.21.15.37:0/600337631 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600b2050 con 0x7f6b6815b950 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b5dffb640 1 -- 172.21.15.37:0/600337631 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b600be070 con 0x7f6b6815b950 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/600337631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b68151210 con 0x7f6b6815b950 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b5dffb640 1 -- 172.21.15.37:0/600337631 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b600adb10 con 0x7f6b6815b950 2024-04-25T09:01:30.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.198+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/600337631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b681514f0 con 0x7f6b6815b950 2024-04-25T09:01:30.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.202+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/600337631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b6804efa0 con 0x7f6b6815b950 2024-04-25T09:01:30.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.202+0000 7f6b5dffb640 1 -- 172.21.15.37:0/600337631 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6b600adcb0 con 0x7f6b6815b950 2024-04-25T09:01:30.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.202+0000 7f6b5dffb640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6b4c066f60 0x7f6b4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.202+0000 7f6b5dffb640 1 -- 172.21.15.37:0/600337631 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f6b601202e0 con 0x7f6b6815b950 2024-04-25T09:01:30.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.202+0000 7f6b6d697640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6b4c066f60 0x7f6b4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.202+0000 7f6b6d697640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6b4c066f60 0x7f6b4c069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007fd0 tx=0x7f6b58009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.206+0000 7f6b5dffb640 1 -- 172.21.15.37:0/600337631 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6b68151ac0 con 0x7f6b6815b950 2024-04-25T09:01:30.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.402+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/600337631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b680cb980 con 0x7f6b6815b950 2024-04-25T09:01:30.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b5dffb640 1 -- 172.21.15.37:0/600337631 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f6b60016520 con 0x7f6b6815b950 2024-04-25T09:01:30.394 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:30.394 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":75,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5921,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2946083597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2946083597},{"type":"v1","addr":"172.21.15.203:6837","nonce":2946083597}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":6577,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/2477669794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":2477669794},{"type":"v1","addr":"172.21.15.203:6836","nonce":2477669794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":7728,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/3878782532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3878782532},{"type":"v1","addr":"172.21.15.37:6833","nonce":3878782532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":55},{"gid":8634,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2194953282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2194953282},{"type":"v1","addr":"172.21.15.37:6835","nonce":2194953282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75}],"filesystems":[]} 2024-04-25T09:01:30.394 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 75 2024-04-25T09:01:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6b4c066f60 msgr2=0x7f6b4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6b4c066f60 0x7f6b4c069420 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6b58007fd0 tx=0x7f6b58009040 comp rx=0 tx=0).stop 2024-04-25T09:01:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/600337631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b6815b950 msgr2=0x7f6b680c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b6815b950 0x7f6b680c7600 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f6b6001a560 tx=0x7f6b600196d0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/600337631 shutdown_connections 2024-04-25T09:01:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6b4c066f60 0x7f6b4c069420 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b6815b950 0x7f6b680c7600 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6b680cfb80 0x7f6b6814fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 --2- 172.21.15.37:0/600337631 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6b6809a480 0x7f6b680c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/600337631 >> 172.21.15.37:0/600337631 conn(0x7f6b680b30b0 msgr2=0x7f6b68073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:30.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/600337631 shutdown_connections 2024-04-25T09:01:30.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.406+0000 7f6b6f8fb640 1 -- 172.21.15.37:0/600337631 wait complete. 2024-04-25T09:01:30.414 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:01:30.414 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:01:30.414 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:01:30.414 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:01:30.414 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:01:30.415 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:01:30.415 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:01:30.415 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:01:30.415 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:01:30.415 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:01:30.415 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:01:30.415 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:01:30.415 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:01:30.416 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:01:30.416 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:01:30.416 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:01:30.424 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:01:30.424 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:01:30.427 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:01:30.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:01:30.428 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:01:30.429 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:01:30.433 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:01:30.434 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:01:30.470 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:01:30.470 DEBUG:teuthology.parallel:result is None 2024-04-25T09:01:30.471 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:01:30.471 DEBUG:teuthology.parallel:result is None 2024-04-25T09:01:30.475 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:01:30.475 DEBUG:teuthology.parallel:result is None 2024-04-25T09:01:30.477 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:01:30.477 DEBUG:teuthology.parallel:result is None 2024-04-25T09:01:30.477 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:01:30.534 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T09:01:30.542 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T09:01:30.545 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-25T09:01:30.546 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T09:01:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2bb3ee640 1 Processor -- start 2024-04-25T09:01:30.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2bb3ee640 1 -- start start 2024-04-25T09:01:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2bb3ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2b40d07f0 0x7fa2b40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2bb3ee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa2b40d1cd0 0x7fa2b406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2bb3ee640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2b406be60 con 0x7fa2b415da80 2024-04-25T09:01:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2bb3ee640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2b406c440 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2bb3ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b415de60 con 0x7fa2b40d07f0 2024-04-25T09:01:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2b8989640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2b40d07f0 0x7fa2b40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2b8989640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2b40d07f0 0x7fa2b40d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:57112/0 (socket says 172.21.15.37:57112) 2024-04-25T09:01:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2b8989640 1 -- 172.21.15.37:0/1800742337 learned_addr learned my addr 172.21.15.37:0/1800742337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2b998b640 1 --2- 172.21.15.37:0/1800742337 >> v2:172.21.15.37:3300/0 conn(0x7fa2b40d1cd0 0x7fa2b406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2b8989640 1 -- 172.21.15.37:0/1800742337 >> v2:172.21.15.37:3300/0 conn(0x7fa2b40d1cd0 msgr2=0x7fa2b406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2b8989640 1 --2- 172.21.15.37:0/1800742337 >> v2:172.21.15.37:3300/0 conn(0x7fa2b40d1cd0 0x7fa2b406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2b8989640 1 -- 172.21.15.37:0/1800742337 >> v1:172.21.15.37:6789/0 conn(0x7fa2b415da80 legacy=0x7fa2b40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.606+0000 7fa2b8989640 1 -- 172.21.15.37:0/1800742337 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b40d2450 con 0x7fa2b40d07f0 2024-04-25T09:01:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2b8989640 1 --2- 172.21.15.37:0/1800742337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2b40d07f0 0x7fa2b40d4230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4006fd0 tx=0x7fa2a401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=174ff3e5a6eb8c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2abfff640 1 -- 172.21.15.37:0/1800742337 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40a7070 con 0x7fa2b40d07f0 2024-04-25T09:01:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2abfff640 1 -- 172.21.15.37:0/1800742337 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a40b0070 con 0x7fa2b40d07f0 2024-04-25T09:01:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2abfff640 1 -- 172.21.15.37:0/1800742337 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40ab890 con 0x7fa2b40d07f0 2024-04-25T09:01:30.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/1800742337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2b40d07f0 msgr2=0x7fa2b40d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2bb3ee640 1 --2- 172.21.15.37:0/1800742337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2b40d07f0 0x7fa2b40d4230 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa2a4006fd0 tx=0x7fa2a401c510 comp rx=0 tx=0).stop 2024-04-25T09:01:30.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/1800742337 shutdown_connections 2024-04-25T09:01:30.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2bb3ee640 1 --2- 172.21.15.37:0/1800742337 >> v2:172.21.15.37:3300/0 conn(0x7fa2b40d1cd0 0x7fa2b406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2bb3ee640 1 --2- 172.21.15.37:0/1800742337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2b40d07f0 0x7fa2b40d4230 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/1800742337 >> 172.21.15.37:0/1800742337 conn(0x7fa2b408b000 msgr2=0x7fa2b40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:30.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/1800742337 shutdown_connections 2024-04-25T09:01:30.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/1800742337 wait complete. 2024-04-25T09:01:30.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.610+0000 7fa2bb3ee640 1 Processor -- start 2024-04-25T09:01:30.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2bb3ee640 1 -- start start 2024-04-25T09:01:30.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2bb3ee640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40d07f0 0x7fa2b41554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2bb3ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b40d1cd0 0x7fa2b40ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2bb3ee640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2b415da80 0x7fa2b41586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2bb3ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b406d7a0 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2bb3ee640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b406d620 con 0x7fa2b40d07f0 2024-04-25T09:01:30.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2bb3ee640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b406d920 con 0x7fa2b415da80 2024-04-25T09:01:30.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2b8989640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b40d1cd0 0x7fa2b40ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2b8989640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b40d1cd0 0x7fa2b40ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35320/0 (socket says 172.21.15.37:35320) 2024-04-25T09:01:30.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2b8989640 1 -- 172.21.15.37:0/2492471732 learned_addr learned my addr 172.21.15.37:0/2492471732 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:30.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2b8989640 1 -- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2b415da80 msgr2=0x7fa2b41586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2b8989640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2b415da80 0x7fa2b41586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2b8989640 1 -- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40d07f0 msgr2=0x7fa2b41554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2b8989640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40d07f0 0x7fa2b41554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2b8989640 1 -- 172.21.15.37:0/2492471732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b415aba0 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2b8989640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b40d1cd0 0x7fa2b40ccff0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fa2a401abc0 tx=0x7fa2a40192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2a9ffb640 1 -- 172.21.15.37:0/2492471732 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a4003b70 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2a9ffb640 1 -- 172.21.15.37:0/2492471732 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2a4003d10 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/2492471732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b415ae40 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2a9ffb640 1 -- 172.21.15.37:0/2492471732 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2a40ab670 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/2492471732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b415b400 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/2492471732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b404efa0 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2a9ffb640 1 -- 172.21.15.37:0/2492471732 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa2a40ab810 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2a9ffb640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa298066f60 0x7fa298069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2a9ffb640 1 -- 172.21.15.37:0/2492471732 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa2a411f3a0 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.614+0000 7fa2b918a640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa298066f60 0x7fa298069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.618+0000 7fa2b918a640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa298066f60 0x7fa298069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0000f70 tx=0x7fa2b00079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.618+0000 7fa2a9ffb640 1 -- 172.21.15.37:0/2492471732 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa2a4016130 con 0x7fa2b40d1cd0 2024-04-25T09:01:30.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:30.790+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/2492471732 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa2b4156970 con 0x7fa2b40d1cd0 2024-04-25T09:01:31.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.514+0000 7fa2a9ffb640 1 -- 172.21.15.37:0/2492471732 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v104) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa2a40f0470 con 0x7fa2b40d1cd0 2024-04-25T09:01:31.505 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-04-25T09:01:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa298066f60 msgr2=0x7fa298069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa298066f60 0x7fa298069420 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0000f70 tx=0x7fa2b00079e0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/2492471732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b40d1cd0 msgr2=0x7fa2b40ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b40d1cd0 0x7fa2b40ccff0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fa2a401abc0 tx=0x7fa2a40192b0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/2492471732 shutdown_connections 2024-04-25T09:01:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa298066f60 0x7fa298069420 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa2b415da80 0x7fa2b41586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2b40d1cd0 0x7fa2b40ccff0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 --2- 172.21.15.37:0/2492471732 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa2b40d07f0 0x7fa2b41554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/2492471732 >> 172.21.15.37:0/2492471732 conn(0x7fa2b408b000 msgr2=0x7fa2b40d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:31.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/2492471732 shutdown_connections 2024-04-25T09:01:31.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.518+0000 7fa2bb3ee640 1 -- 172.21.15.37:0/2492471732 wait complete. 2024-04-25T09:01:31.534 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:01:31.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf769ab640 1 Processor -- start 2024-04-25T09:01:31.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf769ab640 1 -- start start 2024-04-25T09:01:31.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf769ab640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faf700af280 0x7faf7015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf769ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf700b2c30 0x7faf7006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf769ab640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf70066c30 con 0x7faf7006c1e0 2024-04-25T09:01:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf769ab640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7006daa0 con 0x7faf700af280 2024-04-25T09:01:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf769ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf700af660 con 0x7faf700b2c30 2024-04-25T09:01:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf6ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faf700af280 0x7faf7015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf6ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7faf700af280 0x7faf7015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35328/0 (socket says 172.21.15.37:35328) 2024-04-25T09:01:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf6ffff640 1 -- 172.21.15.37:0/2746567821 learned_addr learned my addr 172.21.15.37:0/2746567821 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf6f7fe640 1 --2- 172.21.15.37:0/2746567821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf700b2c30 0x7faf7006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf6ffff640 1 -- 172.21.15.37:0/2746567821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf700b2c30 msgr2=0x7faf7006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf6ffff640 1 --2- 172.21.15.37:0/2746567821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf700b2c30 0x7faf7006bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf6ffff640 1 -- 172.21.15.37:0/2746567821 >> v1:172.21.15.203:6789/0 conn(0x7faf7006c1e0 legacy=0x7faf70161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.674+0000 7faf6ffff640 1 -- 172.21.15.37:0/2746567821 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7015e640 con 0x7faf700af280 2024-04-25T09:01:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf6ffff640 1 --2- 172.21.15.37:0/2746567821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf700af280 0x7faf7015df10 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7faf68009e50 tx=0x7faf6801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a120f61df17aef7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf6effd640 1 -- 172.21.15.37:0/2746567821 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf680a5700 con 0x7faf700af280 2024-04-25T09:01:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf6effd640 1 -- 172.21.15.37:0/2746567821 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf680a58a0 con 0x7faf700af280 2024-04-25T09:01:31.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf6effd640 1 -- 172.21.15.37:0/2746567821 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf680a5c70 con 0x7faf700af280 2024-04-25T09:01:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 -- 172.21.15.37:0/2746567821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf700af280 msgr2=0x7faf7015df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 --2- 172.21.15.37:0/2746567821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf700af280 0x7faf7015df10 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7faf68009e50 tx=0x7faf6801ca30 comp rx=0 tx=0).stop 2024-04-25T09:01:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 -- 172.21.15.37:0/2746567821 shutdown_connections 2024-04-25T09:01:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 --2- 172.21.15.37:0/2746567821 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf700b2c30 0x7faf7006bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 --2- 172.21.15.37:0/2746567821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf700af280 0x7faf7015df10 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 -- 172.21.15.37:0/2746567821 >> 172.21.15.37:0/2746567821 conn(0x7faf70071810 msgr2=0x7faf70071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 -- 172.21.15.37:0/2746567821 shutdown_connections 2024-04-25T09:01:31.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 -- 172.21.15.37:0/2746567821 wait complete. 2024-04-25T09:01:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 Processor -- start 2024-04-25T09:01:31.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 -- start start 2024-04-25T09:01:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7006c1e0 0x7faf701542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf700af280 0x7faf701523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf700b2c30 0x7faf70162510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7016aab0 con 0x7faf7006c1e0 2024-04-25T09:01:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7006c5e0 con 0x7faf700b2c30 2024-04-25T09:01:31.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf769ab640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7016ac30 con 0x7faf700af280 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf6ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7006c1e0 0x7faf701542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf6ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7006c1e0 0x7faf701542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35342/0 (socket says 172.21.15.37:35342) 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.678+0000 7faf6ffff640 1 -- 172.21.15.37:0/1586433874 learned_addr learned my addr 172.21.15.37:0/1586433874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf74f48640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf700b2c30 0x7faf70162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6f7fe640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf700af280 0x7faf701523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6ffff640 1 -- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf700af280 msgr2=0x7faf701523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6ffff640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf700af280 0x7faf701523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6ffff640 1 -- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf700b2c30 msgr2=0x7faf70162510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6ffff640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf700b2c30 0x7faf70162510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6ffff640 1 -- 172.21.15.37:0/1586433874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7015e200 con 0x7faf7006c1e0 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf74f48640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf700b2c30 0x7faf70162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6f7fe640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf700af280 0x7faf701523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6ffff640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7006c1e0 0x7faf701542e0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7faf680af040 tx=0x7faf680197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6cff9640 1 -- 172.21.15.37:0/1586433874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf680b8670 con 0x7faf7006c1e0 2024-04-25T09:01:31.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6cff9640 1 -- 172.21.15.37:0/1586433874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf680be070 con 0x7faf7006c1e0 2024-04-25T09:01:31.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf769ab640 1 -- 172.21.15.37:0/1586433874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf70152e50 con 0x7faf7006c1e0 2024-04-25T09:01:31.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6cff9640 1 -- 172.21.15.37:0/1586433874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf680b8a70 con 0x7faf7006c1e0 2024-04-25T09:01:31.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf769ab640 1 -- 172.21.15.37:0/1586433874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf70153410 con 0x7faf7006c1e0 2024-04-25T09:01:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf769ab640 1 -- 172.21.15.37:0/1586433874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7004efa0 con 0x7faf7006c1e0 2024-04-25T09:01:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6cff9640 1 -- 172.21.15.37:0/1586433874 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7faf680b8c10 con 0x7faf7006c1e0 2024-04-25T09:01:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6cff9640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf5c066f60 0x7faf5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6cff9640 1 -- 172.21.15.37:0/1586433874 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7faf68120070 con 0x7faf7006c1e0 2024-04-25T09:01:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6f7fe640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf5c066f60 0x7faf5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.682+0000 7faf6f7fe640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf5c066f60 0x7faf5c069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7faf7006f710 tx=0x7faf64008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.686+0000 7faf6cff9640 1 -- 172.21.15.37:0/1586433874 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7faf68016620 con 0x7faf7006c1e0 2024-04-25T09:01:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.898+0000 7faf769ab640 1 -- 172.21.15.37:0/1586433874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7faf700727c0 con 0x7faf7006c1e0 2024-04-25T09:01:31.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.898+0000 7faf6cff9640 1 -- 172.21.15.37:0/1586433874 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7faf680f11a0 con 0x7faf7006c1e0 2024-04-25T09:01:31.889 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:31.889 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:01:31.889 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-04-25T09:01:31.889 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.889 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-04-25T09:01:31.889 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpm2pLfABAAJbn2dGKdl8SHS8pApIWC1Q==", 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmqB+jDxAAmRnBzN3vqO8HIf/cd3QoJQ==", 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T09:01:31.890 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T09:01:31.891 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.891 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.891 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-04-25T09:01:31.891 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmiXRqCRAACtY8cFxwqOT4WNzj9WmeKA==", 2024-04-25T09:01:31.891 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.891 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:01:31.892 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T09:01:31.892 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T09:01:31.892 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.892 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-04-25T09:01:31.892 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmX3ARGBAAW/ZUr85NsKtd8HzxE+Yefw==", 2024-04-25T09:01:31.893 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.893 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:01:31.893 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T09:01:31.893 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T09:01:31.893 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.893 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.894 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-04-25T09:01:31.894 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCSGSpmwhMiJhAAbmQz36LujHeYXaKIOdLLSA==", 2024-04-25T09:01:31.894 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.894 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:31.894 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:01:31.895 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.895 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.895 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-04-25T09:01:31.895 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCVGSpm/OLPKxAA9ZmrrWxvt4QDj2lHATnzSg==", 2024-04-25T09:01:31.895 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.896 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:31.896 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:01:31.896 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.896 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.896 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-04-25T09:01:31.896 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCYGSpmq7tEMxAA/wjwgkrtnTkrMoC6G+YIlQ==", 2024-04-25T09:01:31.897 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.897 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:31.897 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:01:31.897 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.897 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.897 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-04-25T09:01:31.898 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCbGSpmReB1ORAAkXje/r11I3ArZHErIvijgw==", 2024-04-25T09:01:31.898 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.898 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:31.898 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:01:31.898 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.899 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.899 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-04-25T09:01:31.899 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQChGSpm4XfLORAAGj+R1Nk/EGmvEdpt8kQJlg==", 2024-04-25T09:01:31.899 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.899 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:31.900 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:01:31.900 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.900 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.900 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.900 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-04-25T09:01:31.900 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQClGSpmd5f1ARAAKF3XG//v3Uqx4bckcD4JMw==", 2024-04-25T09:01:31.901 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.901 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:31.901 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:01:31.901 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.901 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.901 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.901 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-04-25T09:01:31.901 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCoGSpmi9OECBAAJBskMWGCNlAaej9xUc6g8w==", 2024-04-25T09:01:31.902 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.902 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:31.902 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:01:31.902 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.902 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.902 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.902 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-04-25T09:01:31.902 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCrGSpmunQiDhAA3UZo2vzT44HEYUBUia75NA==", 2024-04-25T09:01:31.902 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.902 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA==", 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.903 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmu8UoJhAAyutD5hzKLZmr57WuAWScuQ==", 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmGVVdIxAAY7/VOuqF5OoomqqVxE1Ifw==", 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:01:31.904 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmjFNDKxAAmo6DB9W17TZaGlvkdqkSuQ==", 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm+mGgJRAA9oTRTlZpGBdCZ0/bj580Bg==", 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpmLChTORAAo6fs/9PDJ5YqqcbfHfPO9Q==", 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T09:01:31.906 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm3NLHNhAAeqSA658L6P9OcrSO5JVzLg==", 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T09:01:31.907 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:01:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 -- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf5c066f60 msgr2=0x7faf5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf5c066f60 0x7faf5c069420 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7faf7006f710 tx=0x7faf64008040 comp rx=0 tx=0).stop 2024-04-25T09:01:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 -- 172.21.15.37:0/1586433874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7006c1e0 msgr2=0x7faf701542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7006c1e0 0x7faf701542e0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7faf680af040 tx=0x7faf680197d0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 -- 172.21.15.37:0/1586433874 shutdown_connections 2024-04-25T09:01:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7faf5c066f60 0x7faf5c069420 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7faf700b2c30 0x7faf70162510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7faf700af280 0x7faf701523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 --2- 172.21.15.37:0/1586433874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf7006c1e0 0x7faf701542e0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 -- 172.21.15.37:0/1586433874 >> 172.21.15.37:0/1586433874 conn(0x7faf70071810 msgr2=0x7faf7015bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:31.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 -- 172.21.15.37:0/1586433874 shutdown_connections 2024-04-25T09:01:31.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:31.902+0000 7faf769ab640 1 -- 172.21.15.37:0/1586433874 wait complete. 2024-04-25T09:01:31.910 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:32.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.046+0000 7f3acf434640 1 Processor -- start 2024-04-25T09:01:32.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.046+0000 7f3acf434640 1 -- start start 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acf434640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac815ecc0 0x7f3ac806e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acf434640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac80b2090 con 0x7f3ac80b3f50 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acf434640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ac80b43e0 con 0x7f3ac815a700 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acf434640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80b46c0 con 0x7f3ac815ecc0 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acc9cf640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3ac80b3f50 0x7f3ac80b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50510/0 (socket says 172.21.15.37:50510) 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acc9cf640 1 -- 172.21.15.37:0/1787037164 learned_addr learned my addr 172.21.15.37:0/1787037164 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acd9d1640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac815ecc0 0x7f3ac806e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075630934 0 0) 0x7f3ac80b2090 con 0x7f3ac80b3f50 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab4003530 con 0x7f3ac80b3f50 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4093055420 0 0) 0x7f3ab4003530 con 0x7f3ac80b3f50 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac815ecc0 msgr2=0x7f3ac806e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac815ecc0 0x7f3ac806e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 >> v1:172.21.15.203:6789/0 conn(0x7f3ac815a700 legacy=0x7f3ac80cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac80cd6f0 con 0x7f3ac80b3f50 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3ab80016a0 con 0x7f3ac80b3f50 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 >> v1:172.21.15.37:6789/0 conn(0x7f3ac80b3f50 legacy=0x7f3ac80b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab4003a60 0x7f3ab4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac815ecc0 0x7f3ab4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab4004d50 0x7f3ab40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80b46c0 con 0x7f3ab4004d50 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab400bc00 con 0x7f3ab4003a60 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab400be00 con 0x7f3ac815ecc0 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acd9d1640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab4004d50 0x7f3ab40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acc9cf640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac815ecc0 0x7f3ab4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acd1d0640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab4003a60 0x7f3ab4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acd9d1640 1 -- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac815ecc0 msgr2=0x7f3ab4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acd9d1640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac815ecc0 0x7f3ab4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acd9d1640 1 -- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab4003a60 msgr2=0x7f3ab4003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acd9d1640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab4003a60 0x7f3ab4003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acd9d1640 1 -- 172.21.15.37:0/1787037164 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab4003530 con 0x7f3ab4004d50 2024-04-25T09:01:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.050+0000 7f3acd1d0640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab4003a60 0x7f3ab4003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd9d1640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab4004d50 0x7f3ab40051d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009fc0 tx=0x7f3ac001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f8caa2479e8566 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00b2070 con 0x7f3ab4004d50 2024-04-25T09:01:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac00a5ba0 con 0x7f3ab4004d50 2024-04-25T09:01:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3abffff640 1 -- 172.21.15.37:0/1787037164 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac0009c10 con 0x7f3ab4004d50 2024-04-25T09:01:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 -- 172.21.15.37:0/1787037164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab4004d50 msgr2=0x7f3ab40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 --2- 172.21.15.37:0/1787037164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ab4004d50 0x7f3ab40051d0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009fc0 tx=0x7f3ac001cc50 comp rx=0 tx=0).stop 2024-04-25T09:01:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd9d1640 1 -- 172.21.15.37:0/1787037164 reap_dead start 2024-04-25T09:01:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 -- 172.21.15.37:0/1787037164 shutdown_connections 2024-04-25T09:01:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 -- 172.21.15.37:0/1787037164 >> 172.21.15.37:0/1787037164 conn(0x7f3ac80b11a0 msgr2=0x7f3ac80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 -- 172.21.15.37:0/1787037164 shutdown_connections 2024-04-25T09:01:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 -- 172.21.15.37:0/1787037164 wait complete. 2024-04-25T09:01:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 Processor -- start 2024-04-25T09:01:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 -- start start 2024-04-25T09:01:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab4004d50 0x7f3ac8161df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac8155060 0x7f3ac806d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac806d5d0 0x7f3ac8163140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80cd310 con 0x7f3ac806d5d0 2024-04-25T09:01:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac815f0d0 con 0x7f3ab4004d50 2024-04-25T09:01:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac80cd490 con 0x7f3ac8155060 2024-04-25T09:01:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd9d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac806d5d0 0x7f3ac8163140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd9d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac806d5d0 0x7f3ac8163140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35360/0 (socket says 172.21.15.37:35360) 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd9d1640 1 -- 172.21.15.37:0/1695397921 learned_addr learned my addr 172.21.15.37:0/1695397921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd1d0640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab4004d50 0x7f3ac8161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acc9cf640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac8155060 0x7f3ac806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd9d1640 1 -- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac8155060 msgr2=0x7f3ac806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd9d1640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac8155060 0x7f3ac806d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd9d1640 1 -- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab4004d50 msgr2=0x7f3ac8161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd9d1640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab4004d50 0x7f3ac8161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd9d1640 1 -- 172.21.15.37:0/1695397921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac81554e0 con 0x7f3ac806d5d0 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acd9d1640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac806d5d0 0x7f3ac8163140 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00b0040 tx=0x7f3ac0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3abdffb640 1 -- 172.21.15.37:0/1695397921 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00bd070 con 0x7f3ac806d5d0 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3abdffb640 1 -- 172.21.15.37:0/1695397921 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ac00a59a0 con 0x7f3ac806d5d0 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 -- 172.21.15.37:0/1695397921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac8161030 con 0x7f3ac806d5d0 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3abdffb640 1 -- 172.21.15.37:0/1695397921 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ac00b2040 con 0x7f3ac806d5d0 2024-04-25T09:01:32.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.054+0000 7f3acf434640 1 -- 172.21.15.37:0/1695397921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac81615f0 con 0x7f3ac806d5d0 2024-04-25T09:01:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.058+0000 7f3acf434640 1 -- 172.21.15.37:0/1695397921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac804efa0 con 0x7f3ac806d5d0 2024-04-25T09:01:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.058+0000 7f3abdffb640 1 -- 172.21.15.37:0/1695397921 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ac00c0070 con 0x7f3ac806d5d0 2024-04-25T09:01:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.058+0000 7f3abdffb640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3aac066f60 0x7f3aac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.058+0000 7f3abdffb640 1 -- 172.21.15.37:0/1695397921 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3ac011f730 con 0x7f3ac806d5d0 2024-04-25T09:01:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.058+0000 7f3acd1d0640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3aac066f60 0x7f3aac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.058+0000 7f3acd1d0640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3aac066f60 0x7f3aac069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4000c10 tx=0x7f3ac4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.062+0000 7f3abdffb640 1 -- 172.21.15.37:0/1695397921 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ac0016990 con 0x7f3ac806d5d0 2024-04-25T09:01:32.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.222+0000 7f3acf434640 1 -- 172.21.15.37:0/1695397921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ac8158960 con 0x7f3ac806d5d0 2024-04-25T09:01:32.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3abdffb640 1 -- 172.21.15.37:0/1695397921 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+13130 (secure 0 0 0) 0x7f3ac00ef6a0 con 0x7f3ac806d5d0 2024-04-25T09:01:32.215 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:32.215 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":106,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:01:31.529568+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":93,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 -- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3aac066f60 msgr2=0x7f3aac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3aac066f60 0x7f3aac069420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4000c10 tx=0x7f3ac4009040 comp rx=0 tx=0).stop 2024-04-25T09:01:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 -- 172.21.15.37:0/1695397921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac806d5d0 msgr2=0x7f3ac8163140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac806d5d0 0x7f3ac8163140 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3ac00b0040 tx=0x7f3ac0009750 comp rx=0 tx=0).stop 2024-04-25T09:01:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 -- 172.21.15.37:0/1695397921 shutdown_connections 2024-04-25T09:01:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3aac066f60 0x7f3aac069420 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac806d5d0 0x7f3ac8163140 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac8155060 0x7f3ac806d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 --2- 172.21.15.37:0/1695397921 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ab4004d50 0x7f3ac8161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 -- 172.21.15.37:0/1695397921 >> 172.21.15.37:0/1695397921 conn(0x7f3ac80b11a0 msgr2=0x7f3ac80b09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:32.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 -- 172.21.15.37:0/1695397921 shutdown_connections 2024-04-25T09:01:32.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.226+0000 7f3acf434640 1 -- 172.21.15.37:0/1695397921 wait complete. 2024-04-25T09:01:32.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:01:32.235 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:01:32.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461ebd8640 1 Processor -- start 2024-04-25T09:01:32.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461ebd8640 1 -- start start 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461ebd8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46180cf590 0x7f461815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461ebd8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46180cbde0 0x7f46180cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461ebd8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46180ca4c0 con 0x7f46180cf970 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461ebd8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46180ca780 con 0x7f46180cbde0 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461ebd8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46180cd440 con 0x7f46180cf590 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461d175640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46180cbde0 0x7f46180cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461d175640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f46180cbde0 0x7f46180cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35372/0 (socket says 172.21.15.37:35372) 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461d175640 1 -- 172.21.15.37:0/1947028808 learned_addr learned my addr 172.21.15.37:0/1947028808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461c974640 1 --2- 172.21.15.37:0/1947028808 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46180cf590 0x7f461815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461d175640 1 -- 172.21.15.37:0/1947028808 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46180cf590 msgr2=0x7f461815ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461d175640 1 --2- 172.21.15.37:0/1947028808 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46180cf590 0x7f461815ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461d175640 1 -- 172.21.15.37:0/1947028808 >> v1:172.21.15.203:6789/0 conn(0x7f46180cf970 legacy=0x7f46180c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.366+0000 7f461d175640 1 -- 172.21.15.37:0/1947028808 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46180cd690 con 0x7f46180cbde0 2024-04-25T09:01:32.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461d175640 1 --2- 172.21.15.37:0/1947028808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46180cbde0 0x7f46180cc270 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f4610009870 tx=0x7f461001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9f1c47e77a7694 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f46177fe640 1 -- 172.21.15.37:0/1947028808 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100a7070 con 0x7f46180cbde0 2024-04-25T09:01:32.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f46177fe640 1 -- 172.21.15.37:0/1947028808 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46100b0070 con 0x7f46180cbde0 2024-04-25T09:01:32.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f46177fe640 1 -- 172.21.15.37:0/1947028808 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46100ab890 con 0x7f46180cbde0 2024-04-25T09:01:32.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 -- 172.21.15.37:0/1947028808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46180cbde0 msgr2=0x7f46180cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 --2- 172.21.15.37:0/1947028808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46180cbde0 0x7f46180cc270 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f4610009870 tx=0x7f461001c560 comp rx=0 tx=0).stop 2024-04-25T09:01:32.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 -- 172.21.15.37:0/1947028808 shutdown_connections 2024-04-25T09:01:32.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 --2- 172.21.15.37:0/1947028808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46180cbde0 0x7f46180cc270 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 --2- 172.21.15.37:0/1947028808 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46180cf590 0x7f461815ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 -- 172.21.15.37:0/1947028808 >> 172.21.15.37:0/1947028808 conn(0x7f461806ae60 msgr2=0x7f461806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:32.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 -- 172.21.15.37:0/1947028808 shutdown_connections 2024-04-25T09:01:32.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 -- 172.21.15.37:0/1947028808 wait complete. 2024-04-25T09:01:32.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 Processor -- start 2024-04-25T09:01:32.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 -- start start 2024-04-25T09:01:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46180cbde0 0x7f461815abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46180cf590 0x7f4618153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46180cf970 0x7f4618158470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180cfed0 con 0x7f46180cf590 2024-04-25T09:01:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180cfd50 con 0x7f46180cf970 2024-04-25T09:01:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461ebd8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46180ac170 con 0x7f46180cbde0 2024-04-25T09:01:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f4617fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46180cf590 0x7f4618153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f4617fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46180cf590 0x7f4618153ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35374/0 (socket says 172.21.15.37:35374) 2024-04-25T09:01:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f4617fff640 1 -- 172.21.15.37:0/2605707223 learned_addr learned my addr 172.21.15.37:0/2605707223 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:32.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461c974640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46180cbde0 0x7f461815abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f461d175640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46180cf970 0x7f4618158470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f4617fff640 1 -- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46180cbde0 msgr2=0x7f461815abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f4617fff640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46180cbde0 0x7f461815abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f4617fff640 1 -- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46180cf970 msgr2=0x7f4618158470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.370+0000 7f4617fff640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46180cf970 0x7f4618158470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f4617fff640 1 -- 172.21.15.37:0/2605707223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618098eb0 con 0x7f46180cf590 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f4617fff640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46180cf590 0x7f4618153ba0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f460800c7c0 tx=0x7f460800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f46157fa640 1 -- 172.21.15.37:0/2605707223 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608016020 con 0x7f46180cf590 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f46157fa640 1 -- 172.21.15.37:0/2605707223 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4608006d60 con 0x7f46180cf590 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f46157fa640 1 -- 172.21.15.37:0/2605707223 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608003bc0 con 0x7f46180cf590 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f461ebd8640 1 -- 172.21.15.37:0/2605707223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46181540e0 con 0x7f46180cf590 2024-04-25T09:01:32.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f461ebd8640 1 -- 172.21.15.37:0/2605707223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461815b540 con 0x7f46180cf590 2024-04-25T09:01:32.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f461ebd8640 1 -- 172.21.15.37:0/2605707223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461804efa0 con 0x7f46180cf590 2024-04-25T09:01:32.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f46157fa640 1 -- 172.21.15.37:0/2605707223 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f460800a850 con 0x7f46180cf590 2024-04-25T09:01:32.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f46157fa640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4604066f60 0x7f4604069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f46157fa640 1 -- 172.21.15.37:0/2605707223 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4608084b80 con 0x7f46180cf590 2024-04-25T09:01:32.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f461c974640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4604066f60 0x7f4604069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.374+0000 7f461c974640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4604066f60 0x7f4604069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f460c009a30 tx=0x7f460c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.378+0000 7f46157fa640 1 -- 172.21.15.37:0/2605707223 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4608020020 con 0x7f46180cf590 2024-04-25T09:01:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.538+0000 7f461ebd8640 1 -- 172.21.15.37:0/2605707223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46180b4170 con 0x7f46180cf590 2024-04-25T09:01:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.538+0000 7f46157fa640 1 -- 172.21.15.37:0/2605707223 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4608019310 con 0x7f46180cf590 2024-04-25T09:01:32.529 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:01:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 -- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4604066f60 msgr2=0x7f4604069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4604066f60 0x7f4604069420 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f460c009a30 tx=0x7f460c008040 comp rx=0 tx=0).stop 2024-04-25T09:01:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 -- 172.21.15.37:0/2605707223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46180cf590 msgr2=0x7f4618153ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46180cf590 0x7f4618153ba0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f460800c7c0 tx=0x7f460800cc80 comp rx=0 tx=0).stop 2024-04-25T09:01:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 -- 172.21.15.37:0/2605707223 shutdown_connections 2024-04-25T09:01:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4604066f60 0x7f4604069420 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f46180cf970 0x7f4618158470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46180cf590 0x7f4618153ba0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 --2- 172.21.15.37:0/2605707223 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f46180cbde0 0x7f461815abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 -- 172.21.15.37:0/2605707223 >> 172.21.15.37:0/2605707223 conn(0x7f461806ae60 msgr2=0x7f46181558b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 -- 172.21.15.37:0/2605707223 shutdown_connections 2024-04-25T09:01:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.542+0000 7f461ebd8640 1 -- 172.21.15.37:0/2605707223 wait complete. 2024-04-25T09:01:32.547 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:01:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f23aabef640 1 Processor -- start 2024-04-25T09:01:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f23aabef640 1 -- start start 2024-04-25T09:01:32.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f23aabef640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f23a40af280 0x7f23a415df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f23aabef640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f23a40b2c30 0x7f23a406bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f23aabef640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23a406c1e0 0x7f23a4161500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f23aabef640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a406daa0 con 0x7f23a40b2c30 2024-04-25T09:01:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f23aabef640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a4066c30 con 0x7f23a40af280 2024-04-25T09:01:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f23aabef640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a40af660 con 0x7f23a406c1e0 2024-04-25T09:01:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f239bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f23a40b2c30 0x7f23a406bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f239bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f23a40b2c30 0x7f23a406bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35382/0 (socket says 172.21.15.37:35382) 2024-04-25T09:01:32.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f239bfff640 1 -- 172.21.15.37:0/1185320486 learned_addr learned my addr 172.21.15.37:0/1185320486 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:32.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f23a918c640 1 --2- 172.21.15.37:0/1185320486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23a406c1e0 0x7f23a4161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f23a898b640 1 --2- 172.21.15.37:0/1185320486 >> v2:172.21.15.203:3300/0 conn(0x7f23a40af280 0x7f23a415df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f239bfff640 1 -- 172.21.15.37:0/1185320486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23a406c1e0 msgr2=0x7f23a4161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:32.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f239bfff640 1 --2- 172.21.15.37:0/1185320486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23a406c1e0 0x7f23a4161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f239bfff640 1 -- 172.21.15.37:0/1185320486 >> v2:172.21.15.203:3300/0 conn(0x7f23a40af280 msgr2=0x7f23a415df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:32.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f239bfff640 1 --2- 172.21.15.37:0/1185320486 >> v2:172.21.15.203:3300/0 conn(0x7f23a40af280 0x7f23a415df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.678+0000 7f239bfff640 1 -- 172.21.15.37:0/1185320486 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a415e640 con 0x7f23a40b2c30 2024-04-25T09:01:32.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f239bfff640 1 --2- 172.21.15.37:0/1185320486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a40b2c30 0x7f23a406bca0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f23a40cfbf0 tx=0x7f239401c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7294c2be3ea98458 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f239b7fe640 1 -- 172.21.15.37:0/1185320486 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23940b0070 con 0x7f23a40b2c30 2024-04-25T09:01:32.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f239b7fe640 1 -- 172.21.15.37:0/1185320486 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23940ab600 con 0x7f23a40b2c30 2024-04-25T09:01:32.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f239b7fe640 1 -- 172.21.15.37:0/1185320486 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23940ab9d0 con 0x7f23a40b2c30 2024-04-25T09:01:32.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 -- 172.21.15.37:0/1185320486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a40b2c30 msgr2=0x7f23a406bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 --2- 172.21.15.37:0/1185320486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a40b2c30 0x7f23a406bca0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f23a40cfbf0 tx=0x7f239401c260 comp rx=0 tx=0).stop 2024-04-25T09:01:32.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 -- 172.21.15.37:0/1185320486 shutdown_connections 2024-04-25T09:01:32.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 --2- 172.21.15.37:0/1185320486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a40b2c30 0x7f23a406bca0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 --2- 172.21.15.37:0/1185320486 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23a406c1e0 0x7f23a4161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 --2- 172.21.15.37:0/1185320486 >> v2:172.21.15.203:3300/0 conn(0x7f23a40af280 0x7f23a415df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 -- 172.21.15.37:0/1185320486 >> 172.21.15.37:0/1185320486 conn(0x7f23a4071810 msgr2=0x7f23a4071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:32.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 -- 172.21.15.37:0/1185320486 shutdown_connections 2024-04-25T09:01:32.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 -- 172.21.15.37:0/1185320486 wait complete. 2024-04-25T09:01:32.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 Processor -- start 2024-04-25T09:01:32.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 -- start start 2024-04-25T09:01:32.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a406c1e0 0x7f23a40ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23a40af280 0x7f23a4149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23a40b2c30 0x7f23a4149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a406e2a0 con 0x7f23a406c1e0 2024-04-25T09:01:32.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a406e120 con 0x7f23a40b2c30 2024-04-25T09:01:32.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23aabef640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a406e420 con 0x7f23a40af280 2024-04-25T09:01:32.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23a898b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a406c1e0 0x7f23a40ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23a898b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a406c1e0 0x7f23a40ad7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35388/0 (socket says 172.21.15.37:35388) 2024-04-25T09:01:32.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.682+0000 7f23a898b640 1 -- 172.21.15.37:0/4236551562 learned_addr learned my addr 172.21.15.37:0/4236551562 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:32.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23a918c640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23a40b2c30 0x7f23a4149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f239bfff640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23a40af280 0x7f23a4149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23a898b640 1 -- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23a40af280 msgr2=0x7f23a4149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23a898b640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23a40af280 0x7f23a4149500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23a898b640 1 -- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23a40b2c30 msgr2=0x7f23a4149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23a898b640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23a40b2c30 0x7f23a4149a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23a898b640 1 -- 172.21.15.37:0/4236551562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a40acdd0 con 0x7f23a406c1e0 2024-04-25T09:01:32.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23a898b640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a406c1e0 0x7f23a40ad7a0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f23a000d830 tx=0x7f23a000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23997fa640 1 -- 172.21.15.37:0/4236551562 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a000b6c0 con 0x7f23a406c1e0 2024-04-25T09:01:32.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23997fa640 1 -- 172.21.15.37:0/4236551562 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a0019070 con 0x7f23a406c1e0 2024-04-25T09:01:32.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23aabef640 1 -- 172.21.15.37:0/4236551562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a40ad0b0 con 0x7f23a406c1e0 2024-04-25T09:01:32.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23997fa640 1 -- 172.21.15.37:0/4236551562 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a000ba60 con 0x7f23a406c1e0 2024-04-25T09:01:32.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23aabef640 1 -- 172.21.15.37:0/4236551562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a414b020 con 0x7f23a406c1e0 2024-04-25T09:01:32.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23aabef640 1 -- 172.21.15.37:0/4236551562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a404efa0 con 0x7f23a406c1e0 2024-04-25T09:01:32.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23997fa640 1 -- 172.21.15.37:0/4236551562 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f23a000bc00 con 0x7f23a406c1e0 2024-04-25T09:01:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23997fa640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2390066f90 0x7f2390069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f23997fa640 1 -- 172.21.15.37:0/4236551562 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f23a0085300 con 0x7f23a406c1e0 2024-04-25T09:01:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f239bfff640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2390066f90 0x7f2390069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.686+0000 7f239bfff640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2390066f90 0x7f2390069450 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f23940ad000 tx=0x7f2394008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.690+0000 7f23997fa640 1 -- 172.21.15.37:0/4236551562 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f23a00564d0 con 0x7f23a406c1e0 2024-04-25T09:01:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.854+0000 7f23aabef640 1 -- 172.21.15.37:0/4236551562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f23a414ab10 con 0x7f23a406c1e0 2024-04-25T09:01:32.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.854+0000 7f23997fa640 1 -- 172.21.15.37:0/4236551562 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v106) v1 ==== 84+0+65 (secure 0 0 0) 0x7f23a0055dd0 con 0x7f23a406c1e0 2024-04-25T09:01:32.844 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:32.844 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T09:01:32.845 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:01:32.845 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T09:01:32.845 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T09:01:32.845 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:32.845 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 -- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2390066f90 msgr2=0x7f2390069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2390066f90 0x7f2390069450 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f23940ad000 tx=0x7f2394008040 comp rx=0 tx=0).stop 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 -- 172.21.15.37:0/4236551562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a406c1e0 msgr2=0x7f23a40ad7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a406c1e0 0x7f23a40ad7a0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f23a000d830 tx=0x7f23a000dcf0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 -- 172.21.15.37:0/4236551562 shutdown_connections 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2390066f90 0x7f2390069450 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f23a40b2c30 0x7f23a4149a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f23a40af280 0x7f23a4149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 --2- 172.21.15.37:0/4236551562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23a406c1e0 0x7f23a40ad7a0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 -- 172.21.15.37:0/4236551562 >> 172.21.15.37:0/4236551562 conn(0x7f23a4071810 msgr2=0x7f23a415b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 -- 172.21.15.37:0/4236551562 shutdown_connections 2024-04-25T09:01:32.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.858+0000 7f23aabef640 1 -- 172.21.15.37:0/4236551562 wait complete. 2024-04-25T09:01:32.864 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:01:32.864 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:01:32.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d39d9640 1 Processor -- start 2024-04-25T09:01:32.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d39d9640 1 -- start start 2024-04-25T09:01:32.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d39d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ae6c0 0x7f80cc0b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d39d9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80cc15e0f0 con 0x7f80cc15a3c0 2024-04-25T09:01:32.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d39d9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80cc0b24a0 con 0x7f80cc0cf510 2024-04-25T09:01:32.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d39d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc15b0b0 con 0x7f80cc0ae6c0 2024-04-25T09:01:32.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d1775640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f80cc15a3c0 0x7f80cc15a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50522/0 (socket says 172.21.15.37:50522) 2024-04-25T09:01:32.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d1775640 1 -- 172.21.15.37:0/2928996472 learned_addr learned my addr 172.21.15.37:0/2928996472 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:32.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d0f74640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ae6c0 0x7f80cc0b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028123629 0 0) 0x7f80cc15e0f0 con 0x7f80cc15a3c0 2024-04-25T09:01:32.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80b8003530 con 0x7f80cc15a3c0 2024-04-25T09:01:32.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1078155790 0 0) 0x7f80b8003530 con 0x7f80cc15a3c0 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ae6c0 msgr2=0x7f80cc0b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ae6c0 0x7f80cc0b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 >> v1:172.21.15.203:6789/0 conn(0x7f80cc0cf510 legacy=0x7f80cc15e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc15b390 con 0x7f80cc15a3c0 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f80c8001a00 con 0x7f80cc15a3c0 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 >> v1:172.21.15.37:6789/0 conn(0x7f80cc15a3c0 legacy=0x7f80cc15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b80039f0 0x7f80b8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 0x7f80b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ae6c0 0x7f80b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc15b0b0 con 0x7f80b8004810 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b800dcc0 con 0x7f80b80039f0 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b800dec0 con 0x7f80cc0ae6c0 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d0f74640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 0x7f80b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d1775640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b80039f0 0x7f80b8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d0f74640 1 -- 172.21.15.37:0/2928996472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ae6c0 msgr2=0x7f80b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d0f74640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ae6c0 0x7f80b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d0f74640 1 -- 172.21.15.37:0/2928996472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b80039f0 msgr2=0x7f80b8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d0f74640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b80039f0 0x7f80b8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d0f74640 1 -- 172.21.15.37:0/2928996472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b8003530 con 0x7f80b8004810 2024-04-25T09:01:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.994+0000 7f80d1775640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80b80039f0 0x7f80b8003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:32.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.998+0000 7f80d0f74640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 0x7f80b8004c90 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009ab0 tx=0x7f80bc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c46798c7c92c93d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.998+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc0b2070 con 0x7f80b8004810 2024-04-25T09:01:32.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.998+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80bc0a5660 con 0x7f80b8004810 2024-04-25T09:01:32.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.998+0000 7f80c3fff640 1 -- 172.21.15.37:0/2928996472 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc0096b0 con 0x7f80b8004810 2024-04-25T09:01:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.998+0000 7f80d39d9640 1 -- 172.21.15.37:0/2928996472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 msgr2=0x7f80b8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.998+0000 7f80d39d9640 1 --2- 172.21.15.37:0/2928996472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 0x7f80b8004c90 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009ab0 tx=0x7f80bc01ca70 comp rx=0 tx=0).stop 2024-04-25T09:01:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.998+0000 7f80d1f76640 1 -- 172.21.15.37:0/2928996472 reap_dead start 2024-04-25T09:01:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.998+0000 7f80d39d9640 1 -- 172.21.15.37:0/2928996472 shutdown_connections 2024-04-25T09:01:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.998+0000 7f80d39d9640 1 -- 172.21.15.37:0/2928996472 >> 172.21.15.37:0/2928996472 conn(0x7f80cc06ae60 msgr2=0x7f80cc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.998+0000 7f80d39d9640 1 -- 172.21.15.37:0/2928996472 shutdown_connections 2024-04-25T09:01:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:32.998+0000 7f80d39d9640 1 -- 172.21.15.37:0/2928996472 wait complete. 2024-04-25T09:01:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d39d9640 1 Processor -- start 2024-04-25T09:01:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d39d9640 1 -- start start 2024-04-25T09:01:32.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d39d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 0x7f80cc14f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d39d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ca7c0 0x7f80cc0cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d39d9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc0710f0 0x7f80cc071570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d39d9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc15ecb0 con 0x7f80b8004810 2024-04-25T09:01:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d39d9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc15b950 con 0x7f80cc0710f0 2024-04-25T09:01:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d39d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc15ee30 con 0x7f80cc0ca7c0 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d1775640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 0x7f80cc14f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d1775640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 0x7f80cc14f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35414/0 (socket says 172.21.15.37:35414) 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d1775640 1 -- 172.21.15.37:0/3006570930 learned_addr learned my addr 172.21.15.37:0/3006570930 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d0f74640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ca7c0 0x7f80cc0cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d1f76640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc0710f0 0x7f80cc071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d1775640 1 -- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ca7c0 msgr2=0x7f80cc0cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d1775640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ca7c0 0x7f80cc0cac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d1775640 1 -- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc0710f0 msgr2=0x7f80cc071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d1775640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc0710f0 0x7f80cc071570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d1775640 1 -- 172.21.15.37:0/3006570930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80cc071e60 con 0x7f80b8004810 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d0f74640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ca7c0 0x7f80cc0cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d1775640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 0x7f80cc14f0b0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f80c8001340 tx=0x7f80c800e100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80c1ffb640 1 -- 172.21.15.37:0/3006570930 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c801b070 con 0x7f80b8004810 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80c1ffb640 1 -- 172.21.15.37:0/3006570930 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c8011040 con 0x7f80b8004810 2024-04-25T09:01:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d39d9640 1 -- 172.21.15.37:0/3006570930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc072140 con 0x7f80b8004810 2024-04-25T09:01:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80c1ffb640 1 -- 172.21.15.37:0/3006570930 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c80143a0 con 0x7f80b8004810 2024-04-25T09:01:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d39d9640 1 -- 172.21.15.37:0/3006570930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc072700 con 0x7f80b8004810 2024-04-25T09:01:32.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80d39d9640 1 -- 172.21.15.37:0/3006570930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc0cb180 con 0x7f80b8004810 2024-04-25T09:01:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80c1ffb640 1 -- 172.21.15.37:0/3006570930 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f80c801e070 con 0x7f80b8004810 2024-04-25T09:01:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.002+0000 7f80c1ffb640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80b0066f60 0x7f80b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.006+0000 7f80c1ffb640 1 -- 172.21.15.37:0/3006570930 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f80c8087bb0 con 0x7f80b8004810 2024-04-25T09:01:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.006+0000 7f80d0f74640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80b0066f60 0x7f80b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.006+0000 7f80d0f74640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80b0066f60 0x7f80b0069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0af000 tx=0x7f80bc0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.010+0000 7f80c1ffb640 1 -- 172.21.15.37:0/3006570930 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f80c8058ce0 con 0x7f80b8004810 2024-04-25T09:01:33.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.166+0000 7f80d39d9640 1 -- 172.21.15.37:0/3006570930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f80cc067c30 con 0x7f80b8004810 2024-04-25T09:01:33.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.522+0000 7f80c1ffb640 1 -- 172.21.15.37:0/3006570930 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v107) v1 ==== 138+0+0 (secure 0 0 0) 0x7f80c80585e0 con 0x7f80b8004810 2024-04-25T09:01:33.513 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 -- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80b0066f60 msgr2=0x7f80b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80b0066f60 0x7f80b0069420 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0af000 tx=0x7f80bc0046d0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 -- 172.21.15.37:0/3006570930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 msgr2=0x7f80cc14f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 0x7f80cc14f0b0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f80c8001340 tx=0x7f80c800e100 comp rx=0 tx=0).stop 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 -- 172.21.15.37:0/3006570930 shutdown_connections 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f80b0066f60 0x7f80b0069420 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f80cc0710f0 0x7f80cc071570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f80cc0ca7c0 0x7f80cc0cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 --2- 172.21.15.37:0/3006570930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f80b8004810 0x7f80cc14f0b0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 -- 172.21.15.37:0/3006570930 >> 172.21.15.37:0/3006570930 conn(0x7f80cc06ae60 msgr2=0x7f80cc0aed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 -- 172.21.15.37:0/3006570930 shutdown_connections 2024-04-25T09:01:33.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.526+0000 7f80d39d9640 1 -- 172.21.15.37:0/3006570930 wait complete. 2024-04-25T09:01:33.542 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:01:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.686+0000 7f71c9d38640 1 Processor -- start 2024-04-25T09:01:33.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.686+0000 7f71c9d38640 1 -- start start 2024-04-25T09:01:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.686+0000 7f71c9d38640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f71c409a480 0x7f71c40cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.686+0000 7f71c9d38640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f71c40cfb80 0x7f71c40cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.686+0000 7f71c9d38640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c40d04d0 con 0x7f71c415b950 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.686+0000 7f71c9d38640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c415bd30 con 0x7f71c40cfb80 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.686+0000 7f71c9d38640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c40cc0d0 con 0x7f71c409a480 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.686+0000 7f71c3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f71c40cfb80 0x7f71c40cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c37fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f71c415b950 0x7f71c40cf420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50528/0 (socket says 172.21.15.37:50528) 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f71c40cfb80 0x7f71c40cff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35422/0 (socket says 172.21.15.37:35422) 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c37fe640 1 -- 172.21.15.37:0/548628163 learned_addr learned my addr 172.21.15.37:0/548628163 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71bbfff640 1 --2- 172.21.15.37:0/548628163 >> v2:172.21.15.203:3300/0 conn(0x7f71c409a480 0x7f71c40cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c3fff640 1 -- 172.21.15.37:0/548628163 >> v2:172.21.15.203:3300/0 conn(0x7f71c409a480 msgr2=0x7f71c40cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c3fff640 1 --2- 172.21.15.37:0/548628163 >> v2:172.21.15.203:3300/0 conn(0x7f71c409a480 0x7f71c40cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c3fff640 1 -- 172.21.15.37:0/548628163 >> v1:172.21.15.37:6789/0 conn(0x7f71c415b950 legacy=0x7f71c40cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c3fff640 1 -- 172.21.15.37:0/548628163 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c409ab10 con 0x7f71c40cfb80 2024-04-25T09:01:33.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c2ffd640 1 -- 172.21.15.37:0/548628163 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4148291246 0 0) 0x7f71b4001120 con 0x7f71c415b950 2024-04-25T09:01:33.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c3fff640 1 --2- 172.21.15.37:0/548628163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c40cfb80 0x7f71c40cff90 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f71c4072710 tx=0x7f71bc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3b4e7b8e509e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:33.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c2ffd640 1 -- 172.21.15.37:0/548628163 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0b2070 con 0x7f71c40cfb80 2024-04-25T09:01:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c2ffd640 1 -- 172.21.15.37:0/548628163 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc0a58e0 con 0x7f71c40cfb80 2024-04-25T09:01:33.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c2ffd640 1 -- 172.21.15.37:0/548628163 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0a5cb0 con 0x7f71c40cfb80 2024-04-25T09:01:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c9d38640 1 -- 172.21.15.37:0/548628163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c40cfb80 msgr2=0x7f71c40cff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c9d38640 1 --2- 172.21.15.37:0/548628163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c40cfb80 0x7f71c40cff90 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f71c4072710 tx=0x7f71bc01c930 comp rx=0 tx=0).stop 2024-04-25T09:01:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c9d38640 1 -- 172.21.15.37:0/548628163 shutdown_connections 2024-04-25T09:01:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c9d38640 1 --2- 172.21.15.37:0/548628163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c40cfb80 0x7f71c40cff90 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c9d38640 1 --2- 172.21.15.37:0/548628163 >> v2:172.21.15.203:3300/0 conn(0x7f71c409a480 0x7f71c40cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c9d38640 1 -- 172.21.15.37:0/548628163 >> 172.21.15.37:0/548628163 conn(0x7f71c40b30b0 msgr2=0x7f71c40b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c9d38640 1 -- 172.21.15.37:0/548628163 shutdown_connections 2024-04-25T09:01:33.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.690+0000 7f71c9d38640 1 -- 172.21.15.37:0/548628163 wait complete. 2024-04-25T09:01:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c9d38640 1 Processor -- start 2024-04-25T09:01:33.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c9d38640 1 -- start start 2024-04-25T09:01:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c9d38640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71c409a480 0x7f71c40b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c9d38640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71c40cfb80 0x7f71c4150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c9d38640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c415b950 0x7f71c41652d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c9d38640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c40cd390 con 0x7f71c415b950 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c9d38640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c406faa0 con 0x7f71c409a480 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c9d38640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c415b180 con 0x7f71c40cfb80 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c415b950 0x7f71c41652d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c415b950 0x7f71c41652d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35428/0 (socket says 172.21.15.37:35428) 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c3fff640 1 -- 172.21.15.37:0/2867432433 learned_addr learned my addr 172.21.15.37:0/2867432433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c37fe640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71c409a480 0x7f71c40b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71bbfff640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71c40cfb80 0x7f71c4150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c3fff640 1 -- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71c40cfb80 msgr2=0x7f71c4150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c3fff640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71c40cfb80 0x7f71c4150bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c3fff640 1 -- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71c409a480 msgr2=0x7f71c40b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c3fff640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71c409a480 0x7f71c40b64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c3fff640 1 -- 172.21.15.37:0/2867432433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c40b6bb0 con 0x7f71c415b950 2024-04-25T09:01:33.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c3fff640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c415b950 0x7f71c41652d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f71bc019540 tx=0x7f71bc0196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:33.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c0ff9640 1 -- 172.21.15.37:0/2867432433 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0b2040 con 0x7f71c415b950 2024-04-25T09:01:33.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c0ff9640 1 -- 172.21.15.37:0/2867432433 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc0b8470 con 0x7f71c415b950 2024-04-25T09:01:33.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c9d38640 1 -- 172.21.15.37:0/2867432433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c40b6e90 con 0x7f71c415b950 2024-04-25T09:01:33.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c0ff9640 1 -- 172.21.15.37:0/2867432433 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc0b88a0 con 0x7f71c415b950 2024-04-25T09:01:33.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c9d38640 1 -- 172.21.15.37:0/2867432433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c4164db0 con 0x7f71c415b950 2024-04-25T09:01:33.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.694+0000 7f71c9d38640 1 -- 172.21.15.37:0/2867432433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c404efa0 con 0x7f71c415b950 2024-04-25T09:01:33.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.698+0000 7f71c0ff9640 1 -- 172.21.15.37:0/2867432433 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f71bc0c1020 con 0x7f71c415b950 2024-04-25T09:01:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.698+0000 7f71c0ff9640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f71a4066f60 0x7f71a4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.698+0000 7f71c0ff9640 1 -- 172.21.15.37:0/2867432433 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f71bc11fd10 con 0x7f71c415b950 2024-04-25T09:01:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.698+0000 7f71c37fe640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f71a4066f60 0x7f71a4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.698+0000 7f71c37fe640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f71a4066f60 0x7f71a4069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f71b4000f70 tx=0x7f71b4007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:33.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.702+0000 7f71c0ff9640 1 -- 172.21.15.37:0/2867432433 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f71bc016520 con 0x7f71c415b950 2024-04-25T09:01:33.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:33.854+0000 7f71c9d38640 1 -- 172.21.15.37:0/2867432433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f71c40b0270 con 0x7f71c415b950 2024-04-25T09:01:34.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.530+0000 7f71c0ff9640 1 -- 172.21.15.37:0/2867432433 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v108) v1 ==== 170+0+0 (secure 0 0 0) 0x7f71bc0f0e90 con 0x7f71c415b950 2024-04-25T09:01:34.518 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T09:01:34.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 -- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f71a4066f60 msgr2=0x7f71a4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f71a4066f60 0x7f71a4069420 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f71b4000f70 tx=0x7f71b4007c10 comp rx=0 tx=0).stop 2024-04-25T09:01:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 -- 172.21.15.37:0/2867432433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c415b950 msgr2=0x7f71c41652d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c415b950 0x7f71c41652d0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f71bc019540 tx=0x7f71bc0196d0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 -- 172.21.15.37:0/2867432433 shutdown_connections 2024-04-25T09:01:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f71a4066f60 0x7f71a4069420 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71c415b950 0x7f71c41652d0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71c40cfb80 0x7f71c4150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 --2- 172.21.15.37:0/2867432433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f71c409a480 0x7f71c40b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 -- 172.21.15.37:0/2867432433 >> 172.21.15.37:0/2867432433 conn(0x7f71c40b30b0 msgr2=0x7f71c40b18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 -- 172.21.15.37:0/2867432433 shutdown_connections 2024-04-25T09:01:34.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.534+0000 7f71c9d38640 1 -- 172.21.15.37:0/2867432433 wait complete. 2024-04-25T09:01:34.558 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:01:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe076dc6640 1 Processor -- start 2024-04-25T09:01:34.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe076dc6640 1 -- start start 2024-04-25T09:01:34.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe076dc6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0700cc950 0x7fe07015c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe076dc6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe07006ac90 0x7fe07006b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe076dc6640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0700cfbb0 con 0x7fe0700b2980 2024-04-25T09:01:34.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe076dc6640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07006b600 con 0x7fe07006ac90 2024-04-25T09:01:34.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe076dc6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0700cfe60 con 0x7fe0700cc950 2024-04-25T09:01:34.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe075363640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe07006ac90 0x7fe07006b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe06ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe0700b2980 0x7fe0700cf4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50536/0 (socket says 172.21.15.37:50536) 2024-04-25T09:01:34.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe06ffff640 1 -- 172.21.15.37:0/786504112 learned_addr learned my addr 172.21.15.37:0/786504112 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:34.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe074b62640 1 --2- 172.21.15.37:0/786504112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0700cc950 0x7fe07015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe075363640 1 -- 172.21.15.37:0/786504112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0700cc950 msgr2=0x7fe07015c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:34.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe075363640 1 --2- 172.21.15.37:0/786504112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0700cc950 0x7fe07015c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe075363640 1 -- 172.21.15.37:0/786504112 >> v1:172.21.15.37:6789/0 conn(0x7fe0700b2980 legacy=0x7fe0700cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.702+0000 7fe075363640 1 -- 172.21.15.37:0/786504112 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0700d0140 con 0x7fe07006ac90 2024-04-25T09:01:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe075363640 1 --2- 172.21.15.37:0/786504112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe07006ac90 0x7fe07006b0c0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe068009d90 tx=0x7fe06801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bac3dd3bb6d6741c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe06f7fe640 1 -- 172.21.15.37:0/786504112 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0680b2070 con 0x7fe07006ac90 2024-04-25T09:01:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe06f7fe640 1 -- 172.21.15.37:0/786504112 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0680a5910 con 0x7fe07006ac90 2024-04-25T09:01:34.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe06f7fe640 1 -- 172.21.15.37:0/786504112 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0680a5ce0 con 0x7fe07006ac90 2024-04-25T09:01:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 -- 172.21.15.37:0/786504112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe07006ac90 msgr2=0x7fe07006b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 --2- 172.21.15.37:0/786504112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe07006ac90 0x7fe07006b0c0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fe068009d90 tx=0x7fe06801c850 comp rx=0 tx=0).stop 2024-04-25T09:01:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 -- 172.21.15.37:0/786504112 shutdown_connections 2024-04-25T09:01:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 --2- 172.21.15.37:0/786504112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe07006ac90 0x7fe07006b0c0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 --2- 172.21.15.37:0/786504112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0700cc950 0x7fe07015c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 -- 172.21.15.37:0/786504112 >> 172.21.15.37:0/786504112 conn(0x7fe070099a70 msgr2=0x7fe0700b1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 -- 172.21.15.37:0/786504112 shutdown_connections 2024-04-25T09:01:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 -- 172.21.15.37:0/786504112 wait complete. 2024-04-25T09:01:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 Processor -- start 2024-04-25T09:01:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 -- start start 2024-04-25T09:01:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe07006ac90 0x7fe0701f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700b2980 0x7fe070153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0700cc950 0x7fe0700c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07015fb60 con 0x7fe07006ac90 2024-04-25T09:01:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07015f9e0 con 0x7fe0700b2980 2024-04-25T09:01:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe076dc6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07015fce0 con 0x7fe0700cc950 2024-04-25T09:01:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe074b62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe07006ac90 0x7fe0701f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe074b62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe07006ac90 0x7fe0701f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35452/0 (socket says 172.21.15.37:35452) 2024-04-25T09:01:34.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.706+0000 7fe074b62640 1 -- 172.21.15.37:0/954057967 learned_addr learned my addr 172.21.15.37:0/954057967 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe06ffff640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700b2980 0x7fe070153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe075363640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0700cc950 0x7fe0700c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe074b62640 1 -- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0700cc950 msgr2=0x7fe0700c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe074b62640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0700cc950 0x7fe0700c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe074b62640 1 -- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700b2980 msgr2=0x7fe070153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe074b62640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700b2980 0x7fe070153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe074b62640 1 -- 172.21.15.37:0/954057967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0701f8b30 con 0x7fe07006ac90 2024-04-25T09:01:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe074b62640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe07006ac90 0x7fe0701f7c40 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fe06000d5f0 tx=0x7fe06000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe06d7fa640 1 -- 172.21.15.37:0/954057967 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600125f0 con 0x7fe07006ac90 2024-04-25T09:01:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe06d7fa640 1 -- 172.21.15.37:0/954057967 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe060019070 con 0x7fe07006ac90 2024-04-25T09:01:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe076dc6640 1 -- 172.21.15.37:0/954057967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0701f8570 con 0x7fe07006ac90 2024-04-25T09:01:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe06d7fa640 1 -- 172.21.15.37:0/954057967 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0600129b0 con 0x7fe07006ac90 2024-04-25T09:01:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe076dc6640 1 -- 172.21.15.37:0/954057967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07014d5a0 con 0x7fe07006ac90 2024-04-25T09:01:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe076dc6640 1 -- 172.21.15.37:0/954057967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07004efa0 con 0x7fe07006ac90 2024-04-25T09:01:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe06d7fa640 1 -- 172.21.15.37:0/954057967 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe060012b50 con 0x7fe07006ac90 2024-04-25T09:01:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe06d7fa640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe05c066f60 0x7fe05c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe06d7fa640 1 -- 172.21.15.37:0/954057967 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fe060085580 con 0x7fe07006ac90 2024-04-25T09:01:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe06ffff640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe05c066f60 0x7fe05c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.710+0000 7fe06ffff640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe05c066f60 0x7fe05c069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe0700b60b0 tx=0x7fe064009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:34.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.714+0000 7fe06d7fa640 1 -- 172.21.15.37:0/954057967 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe060056450 con 0x7fe07006ac90 2024-04-25T09:01:34.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:34.910+0000 7fe076dc6640 1 -- 172.21.15.37:0/954057967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe0700cb400 con 0x7fe07006ac90 2024-04-25T09:01:35.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.566+0000 7fe06d7fa640 1 -- 172.21.15.37:0/954057967 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v78) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe060055d50 con 0x7fe07006ac90 2024-04-25T09:01:35.556 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T09:01:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 -- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe05c066f60 msgr2=0x7fe05c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe05c066f60 0x7fe05c069420 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe0700b60b0 tx=0x7fe064009040 comp rx=0 tx=0).stop 2024-04-25T09:01:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 -- 172.21.15.37:0/954057967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe07006ac90 msgr2=0x7fe0701f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe07006ac90 0x7fe0701f7c40 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fe06000d5f0 tx=0x7fe06000dab0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 -- 172.21.15.37:0/954057967 shutdown_connections 2024-04-25T09:01:35.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe05c066f60 0x7fe05c069420 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0700cc950 0x7fe0700c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0700b2980 0x7fe070153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 --2- 172.21.15.37:0/954057967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe07006ac90 0x7fe0701f7c40 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 -- 172.21.15.37:0/954057967 >> 172.21.15.37:0/954057967 conn(0x7fe070099a70 msgr2=0x7fe0700cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:35.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 -- 172.21.15.37:0/954057967 shutdown_connections 2024-04-25T09:01:35.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.570+0000 7fe076dc6640 1 -- 172.21.15.37:0/954057967 wait complete. 2024-04-25T09:01:35.598 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:35.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd752469640 1 Processor -- start 2024-04-25T09:01:35.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd752469640 1 -- start start 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd752469640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c15a280 0x7fd74c15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd752469640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd74c0718c0 0x7fd74c071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd752469640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74c15ae30 con 0x7fd74c159ea0 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd752469640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd74c0b2990 con 0x7fd74c0718c0 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd752469640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c0b3530 con 0x7fd74c15a280 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd750a06640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd74c0718c0 0x7fd74c071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd74b7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c15a280 0x7fd74c15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd750a06640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd74c0718c0 0x7fd74c071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35460/0 (socket says 172.21.15.37:35460) 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd74bfff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fd74c159ea0 0x7fd74c0b2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:33596/0 (socket says 172.21.15.37:33596) 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd74b7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c15a280 0x7fd74c15a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:57284/0 (socket says 172.21.15.37:57284) 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd750a06640 1 -- 172.21.15.37:0/3444708989 learned_addr learned my addr 172.21.15.37:0/3444708989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd750a06640 1 -- 172.21.15.37:0/3444708989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c15a280 msgr2=0x7fd74c15a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:35.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd750a06640 1 --2- 172.21.15.37:0/3444708989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c15a280 0x7fd74c15a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd750a06640 1 -- 172.21.15.37:0/3444708989 >> v1:172.21.15.203:6789/0 conn(0x7fd74c159ea0 legacy=0x7fd74c0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.730+0000 7fd750a06640 1 -- 172.21.15.37:0/3444708989 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c15b110 con 0x7fd74c0718c0 2024-04-25T09:01:35.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd750a06640 1 --2- 172.21.15.37:0/3444708989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c0718c0 0x7fd74c071d50 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd744009e50 tx=0x7fd74401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c996f2fa841cdfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:35.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd74affd640 1 -- 172.21.15.37:0/3444708989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440a5810 con 0x7fd74c0718c0 2024-04-25T09:01:35.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd74affd640 1 -- 172.21.15.37:0/3444708989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7440a59b0 con 0x7fd74c0718c0 2024-04-25T09:01:35.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd74affd640 1 -- 172.21.15.37:0/3444708989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7440a5d80 con 0x7fd74c0718c0 2024-04-25T09:01:35.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd752469640 1 -- 172.21.15.37:0/3444708989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c0718c0 msgr2=0x7fd74c071d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd752469640 1 --2- 172.21.15.37:0/3444708989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c0718c0 0x7fd74c071d50 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd744009e50 tx=0x7fd74401ca30 comp rx=0 tx=0).stop 2024-04-25T09:01:35.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd752469640 1 -- 172.21.15.37:0/3444708989 shutdown_connections 2024-04-25T09:01:35.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd752469640 1 --2- 172.21.15.37:0/3444708989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c0718c0 0x7fd74c071d50 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd752469640 1 --2- 172.21.15.37:0/3444708989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c15a280 0x7fd74c15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd752469640 1 -- 172.21.15.37:0/3444708989 >> 172.21.15.37:0/3444708989 conn(0x7fd74c0b11a0 msgr2=0x7fd74c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:35.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd752469640 1 -- 172.21.15.37:0/3444708989 shutdown_connections 2024-04-25T09:01:35.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.734+0000 7fd752469640 1 -- 172.21.15.37:0/3444708989 wait complete. 2024-04-25T09:01:35.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd752469640 1 Processor -- start 2024-04-25T09:01:35.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd752469640 1 -- start start 2024-04-25T09:01:35.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd752469640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c0718c0 0x7fd74c0cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd752469640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c159ea0 0x7fd74c1585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd752469640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd74c15a280 0x7fd74c143a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd752469640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c0713c0 con 0x7fd74c159ea0 2024-04-25T09:01:35.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd752469640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c071240 con 0x7fd74c15a280 2024-04-25T09:01:35.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd752469640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c071540 con 0x7fd74c0718c0 2024-04-25T09:01:35.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd74b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c159ea0 0x7fd74c1585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd74b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c159ea0 0x7fd74c1585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35464/0 (socket says 172.21.15.37:35464) 2024-04-25T09:01:35.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd74b7fe640 1 -- 172.21.15.37:0/2126570432 learned_addr learned my addr 172.21.15.37:0/2126570432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:35.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd74bfff640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c0718c0 0x7fd74c0cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd750a06640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd74c15a280 0x7fd74c143a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd74b7fe640 1 -- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c0718c0 msgr2=0x7fd74c0cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd74b7fe640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c0718c0 0x7fd74c0cddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd74b7fe640 1 -- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd74c15a280 msgr2=0x7fd74c143a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd74b7fe640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd74c15a280 0x7fd74c143a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd74b7fe640 1 -- 172.21.15.37:0/2126570432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c156e20 con 0x7fd74c159ea0 2024-04-25T09:01:35.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd74bfff640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c0718c0 0x7fd74c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:35.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd750a06640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd74c15a280 0x7fd74c143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:35.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd74b7fe640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c159ea0 0x7fd74c1585f0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd74000e930 tx=0x7fd74000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:35.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd748ff9640 1 -- 172.21.15.37:0/2126570432 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740017070 con 0x7fd74c159ea0 2024-04-25T09:01:35.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd748ff9640 1 -- 172.21.15.37:0/2126570432 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74000ca00 con 0x7fd74c159ea0 2024-04-25T09:01:35.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd752469640 1 -- 172.21.15.37:0/2126570432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd74c157100 con 0x7fd74c159ea0 2024-04-25T09:01:35.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd748ff9640 1 -- 172.21.15.37:0/2126570432 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74000cdb0 con 0x7fd74c159ea0 2024-04-25T09:01:35.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd752469640 1 -- 172.21.15.37:0/2126570432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd74c1576c0 con 0x7fd74c159ea0 2024-04-25T09:01:35.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd752469640 1 -- 172.21.15.37:0/2126570432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74c04efa0 con 0x7fd74c159ea0 2024-04-25T09:01:35.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.738+0000 7fd748ff9640 1 -- 172.21.15.37:0/2126570432 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd74001a070 con 0x7fd74c159ea0 2024-04-25T09:01:35.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.742+0000 7fd748ff9640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd738066f60 0x7fd738069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.742+0000 7fd748ff9640 1 -- 172.21.15.37:0/2126570432 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7400855c0 con 0x7fd74c159ea0 2024-04-25T09:01:35.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.742+0000 7fd74bfff640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd738066f60 0x7fd738069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.742+0000 7fd74bfff640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd738066f60 0x7fd738069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd73c007fd0 tx=0x7fd73c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:35.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.746+0000 7fd748ff9640 1 -- 172.21.15.37:0/2126570432 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd740056500 con 0x7fd74c159ea0 2024-04-25T09:01:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.906+0000 7fd752469640 1 -- 172.21.15.37:0/2126570432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd74c157ac0 con 0x7fd74c159ea0 2024-04-25T09:01:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd748ff9640 1 -- 172.21.15.37:0/2126570432 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16861 (secure 0 0 0) 0x7fd740055e00 con 0x7fd74c159ea0 2024-04-25T09:01:35.898 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:35.898 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":109,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:01:34.915368+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:33.212328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:33.895814+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 -- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd738066f60 msgr2=0x7fd738069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd738066f60 0x7fd738069420 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd73c007fd0 tx=0x7fd73c009040 comp rx=0 tx=0).stop 2024-04-25T09:01:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 -- 172.21.15.37:0/2126570432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c159ea0 msgr2=0x7fd74c1585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c159ea0 0x7fd74c1585f0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd74000e930 tx=0x7fd74000edf0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 -- 172.21.15.37:0/2126570432 shutdown_connections 2024-04-25T09:01:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd738066f60 0x7fd738069420 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd74c15a280 0x7fd74c143a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd74c159ea0 0x7fd74c1585f0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 --2- 172.21.15.37:0/2126570432 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd74c0718c0 0x7fd74c0cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 -- 172.21.15.37:0/2126570432 >> 172.21.15.37:0/2126570432 conn(0x7fd74c0b11a0 msgr2=0x7fd74c15cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:35.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 -- 172.21.15.37:0/2126570432 shutdown_connections 2024-04-25T09:01:35.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:35.910+0000 7fd752469640 1 -- 172.21.15.37:0/2126570432 wait complete. 2024-04-25T09:01:35.919 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.054+0000 7f4bb259e640 1 Processor -- start 2024-04-25T09:01:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.054+0000 7f4bb259e640 1 -- start start 2024-04-25T09:01:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4bac06b820 0x7f4bac0b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f4bac0b1ac0 0x7f4bac0cf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bac0afb20 con 0x7f4bac06b440 2024-04-25T09:01:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bac0b06c0 con 0x7f4bac06b820 2024-04-25T09:01:36.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bac0afd90 con 0x7f4bac0b1ac0 2024-04-25T09:01:36.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bab7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4bac06b820 0x7f4bac0b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bab7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f4bac06b820 0x7f4bac0b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35478/0 (socket says 172.21.15.37:35478) 2024-04-25T09:01:36.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bab7fe640 1 -- 172.21.15.37:0/4024659829 learned_addr learned my addr 172.21.15.37:0/4024659829 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:36.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb0b3b640 1 --2- 172.21.15.37:0/4024659829 >> v2:172.21.15.203:3300/0 conn(0x7f4bac0b1ac0 0x7f4bac0cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bab7fe640 1 -- 172.21.15.37:0/4024659829 >> v2:172.21.15.203:3300/0 conn(0x7f4bac0b1ac0 msgr2=0x7f4bac0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:36.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bab7fe640 1 --2- 172.21.15.37:0/4024659829 >> v2:172.21.15.203:3300/0 conn(0x7f4bac0b1ac0 0x7f4bac0cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bab7fe640 1 -- 172.21.15.37:0/4024659829 >> v1:172.21.15.203:6789/0 conn(0x7f4bac06b440 legacy=0x7f4bac0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:36.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bab7fe640 1 -- 172.21.15.37:0/4024659829 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac0b0070 con 0x7f4bac06b820 2024-04-25T09:01:36.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bab7fe640 1 --2- 172.21.15.37:0/4024659829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bac06b820 0x7f4bac0b0b50 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f4bac073280 tx=0x7f4ba001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2878957ae759d35e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4baaffd640 1 -- 172.21.15.37:0/4024659829 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00ab5f0 con 0x7f4bac06b820 2024-04-25T09:01:36.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4baaffd640 1 -- 172.21.15.37:0/4024659829 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba00b0070 con 0x7f4bac06b820 2024-04-25T09:01:36.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4baaffd640 1 -- 172.21.15.37:0/4024659829 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00ab9b0 con 0x7f4bac06b820 2024-04-25T09:01:36.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 -- 172.21.15.37:0/4024659829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bac06b820 msgr2=0x7f4bac0b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 --2- 172.21.15.37:0/4024659829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bac06b820 0x7f4bac0b0b50 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f4bac073280 tx=0x7f4ba001c4e0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 -- 172.21.15.37:0/4024659829 shutdown_connections 2024-04-25T09:01:36.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 --2- 172.21.15.37:0/4024659829 >> v2:172.21.15.203:3300/0 conn(0x7f4bac0b1ac0 0x7f4bac0cf650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 --2- 172.21.15.37:0/4024659829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bac06b820 0x7f4bac0b0b50 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 -- 172.21.15.37:0/4024659829 >> 172.21.15.37:0/4024659829 conn(0x7f4bac0b1f10 msgr2=0x7f4bac0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:36.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 -- 172.21.15.37:0/4024659829 shutdown_connections 2024-04-25T09:01:36.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.058+0000 7f4bb259e640 1 -- 172.21.15.37:0/4024659829 wait complete. 2024-04-25T09:01:36.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4bb259e640 1 Processor -- start 2024-04-25T09:01:36.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4bb259e640 1 -- start start 2024-04-25T09:01:36.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4bb259e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bac06b440 0x7f4bac15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4bb259e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bac06b820 0x7f4bac15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4bb259e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bac0b1ac0 0x7f4bac15f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4bb259e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0cd130 con 0x7f4bac06b440 2024-04-25T09:01:36.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4bb259e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac0ccfb0 con 0x7f4bac0b1ac0 2024-04-25T09:01:36.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4bb259e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac15d080 con 0x7f4bac06b820 2024-04-25T09:01:36.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4babfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bac06b440 0x7f4bac15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4babfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bac06b440 0x7f4bac15c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35486/0 (socket says 172.21.15.37:35486) 2024-04-25T09:01:36.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4babfff640 1 -- 172.21.15.37:0/3184444459 learned_addr learned my addr 172.21.15.37:0/3184444459 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:36.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4babfff640 1 -- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bac06b820 msgr2=0x7f4bac15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4babfff640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bac06b820 0x7f4bac15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4babfff640 1 -- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bac0b1ac0 msgr2=0x7f4bac15f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4babfff640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bac0b1ac0 0x7f4bac15f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4babfff640 1 -- 172.21.15.37:0/3184444459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac155af0 con 0x7f4bac06b440 2024-04-25T09:01:36.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4babfff640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bac06b440 0x7f4bac15c6a0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c00c450 tx=0x7f4b9c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4ba8ff9640 1 -- 172.21.15.37:0/3184444459 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c012830 con 0x7f4bac06b440 2024-04-25T09:01:36.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4ba8ff9640 1 -- 172.21.15.37:0/3184444459 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9c017070 con 0x7f4bac06b440 2024-04-25T09:01:36.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4bb259e640 1 -- 172.21.15.37:0/3184444459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bac155dd0 con 0x7f4bac06b440 2024-04-25T09:01:36.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4ba8ff9640 1 -- 172.21.15.37:0/3184444459 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9c012c30 con 0x7f4bac06b440 2024-04-25T09:01:36.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4bb259e640 1 -- 172.21.15.37:0/3184444459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bac156390 con 0x7f4bac06b440 2024-04-25T09:01:36.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.062+0000 7f4bb259e640 1 -- 172.21.15.37:0/3184444459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bac04efa0 con 0x7f4bac06b440 2024-04-25T09:01:36.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.066+0000 7f4ba8ff9640 1 -- 172.21.15.37:0/3184444459 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b9c01c460 con 0x7f4bac06b440 2024-04-25T09:01:36.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.066+0000 7f4ba8ff9640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4b98066f60 0x7f4b98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.066+0000 7f4ba8ff9640 1 -- 172.21.15.37:0/3184444459 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b9c084f90 con 0x7f4bac06b440 2024-04-25T09:01:36.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.066+0000 7f4bab7fe640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4b98066f60 0x7f4b98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.066+0000 7f4bab7fe640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4b98066f60 0x7f4b98069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00af000 tx=0x7f4ba0004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.070+0000 7f4ba8ff9640 1 -- 172.21.15.37:0/3184444459 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4b9c055e10 con 0x7f4bac06b440 2024-04-25T09:01:36.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.226+0000 7f4bb259e640 1 -- 172.21.15.37:0/3184444459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4bac0cc160 con 0x7f4bac06b440 2024-04-25T09:01:36.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.230+0000 7f4ba8ff9640 1 -- 172.21.15.37:0/3184444459 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16861 (secure 0 0 0) 0x7f4b9c055710 con 0x7f4bac06b440 2024-04-25T09:01:36.219 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:36.220 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":109,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:01:34.915368+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:33.212328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:33.895814+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:36.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.230+0000 7f4bb259e640 1 -- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4b98066f60 msgr2=0x7f4b98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.230+0000 7f4bb259e640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4b98066f60 0x7f4b98069420 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4ba00af000 tx=0x7f4ba0004740 comp rx=0 tx=0).stop 2024-04-25T09:01:36.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.230+0000 7f4bb259e640 1 -- 172.21.15.37:0/3184444459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bac06b440 msgr2=0x7f4bac15c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.230+0000 7f4bb259e640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bac06b440 0x7f4bac15c6a0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c00c450 tx=0x7f4b9c00c910 comp rx=0 tx=0).stop 2024-04-25T09:01:36.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.230+0000 7f4bb259e640 1 -- 172.21.15.37:0/3184444459 shutdown_connections 2024-04-25T09:01:36.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.230+0000 7f4bb259e640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4b98066f60 0x7f4b98069420 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.230+0000 7f4bb259e640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4bac0b1ac0 0x7f4bac15f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.234+0000 7f4bb259e640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4bac06b820 0x7f4bac15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.234+0000 7f4bb259e640 1 --2- 172.21.15.37:0/3184444459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4bac06b440 0x7f4bac15c6a0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.234+0000 7f4bb259e640 1 -- 172.21.15.37:0/3184444459 >> 172.21.15.37:0/3184444459 conn(0x7f4bac0b1f10 msgr2=0x7f4bac1613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:36.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.234+0000 7f4bb259e640 1 -- 172.21.15.37:0/3184444459 shutdown_connections 2024-04-25T09:01:36.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.234+0000 7f4bb259e640 1 -- 172.21.15.37:0/3184444459 wait complete. 2024-04-25T09:01:36.237 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:01:36.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f4460235640 1 Processor -- start 2024-04-25T09:01:36.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f4460235640 1 -- start start 2024-04-25T09:01:36.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f4460235640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f445815da80 0x7f44580d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f4460235640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 0x7f445806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f4460235640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f445806be60 con 0x7f44580d1cd0 2024-04-25T09:01:36.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f4460235640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f445806c440 con 0x7f445815da80 2024-04-25T09:01:36.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f4460235640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445815de60 con 0x7f44580d07f0 2024-04-25T09:01:36.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f445e7d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 0x7f445806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f445dfd1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f445815da80 0x7f44580d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f445d7d0640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f44580d1cd0 0x7f44580d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:33612/0 (socket says 172.21.15.37:33612) 2024-04-25T09:01:36.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f445d7d0640 1 -- 172.21.15.37:0/922489082 learned_addr learned my addr 172.21.15.37:0/922489082 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:36.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f445e7d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 0x7f445806c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:57304/0 (socket says 172.21.15.37:57304) 2024-04-25T09:01:36.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f445e7d2640 1 -- 172.21.15.37:0/922489082 >> v2:172.21.15.203:3300/0 conn(0x7f445815da80 msgr2=0x7f44580d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:36.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f445e7d2640 1 --2- 172.21.15.37:0/922489082 >> v2:172.21.15.203:3300/0 conn(0x7f445815da80 0x7f44580d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f445e7d2640 1 -- 172.21.15.37:0/922489082 >> v1:172.21.15.203:6789/0 conn(0x7f44580d1cd0 legacy=0x7f44580d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.378+0000 7f445e7d2640 1 -- 172.21.15.37:0/922489082 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44580d2450 con 0x7f44580d07f0 2024-04-25T09:01:36.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f445e7d2640 1 --2- 172.21.15.37:0/922489082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 0x7f445806c8d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4450009dc0 tx=0x7f445001c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=872a444ab32c3339 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f445cfcf640 1 -- 172.21.15.37:0/922489082 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44500b2070 con 0x7f44580d07f0 2024-04-25T09:01:36.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f445cfcf640 1 -- 172.21.15.37:0/922489082 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44500a5940 con 0x7f44580d07f0 2024-04-25T09:01:36.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f445cfcf640 1 -- 172.21.15.37:0/922489082 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44500a5d10 con 0x7f44580d07f0 2024-04-25T09:01:36.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f4460235640 1 -- 172.21.15.37:0/922489082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 msgr2=0x7f445806c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f4460235640 1 --2- 172.21.15.37:0/922489082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 0x7f445806c8d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4450009dc0 tx=0x7f445001c920 comp rx=0 tx=0).stop 2024-04-25T09:01:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f4460235640 1 -- 172.21.15.37:0/922489082 shutdown_connections 2024-04-25T09:01:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f4460235640 1 --2- 172.21.15.37:0/922489082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 0x7f445806c8d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f4460235640 1 --2- 172.21.15.37:0/922489082 >> v2:172.21.15.203:3300/0 conn(0x7f445815da80 0x7f44580d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f4460235640 1 -- 172.21.15.37:0/922489082 >> 172.21.15.37:0/922489082 conn(0x7f445808b000 msgr2=0x7f44580d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f4460235640 1 -- 172.21.15.37:0/922489082 shutdown_connections 2024-04-25T09:01:36.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.382+0000 7f4460235640 1 -- 172.21.15.37:0/922489082 wait complete. 2024-04-25T09:01:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f4460235640 1 Processor -- start 2024-04-25T09:01:36.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f4460235640 1 -- start start 2024-04-25T09:01:36.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f4460235640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 0x7f4458160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f4460235640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d1cd0 0x7f445814c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f4460235640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445815da80 0x7f445814ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f4460235640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445806d7a0 con 0x7f44580d1cd0 2024-04-25T09:01:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f4460235640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445806d620 con 0x7f445815da80 2024-04-25T09:01:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f4460235640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f445806d920 con 0x7f44580d07f0 2024-04-25T09:01:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445d7d0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d1cd0 0x7f445814c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445e7d2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445815da80 0x7f445814ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445e7d2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445815da80 0x7f445814ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:36970/0 (socket says 172.21.15.37:36970) 2024-04-25T09:01:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445e7d2640 1 -- 172.21.15.37:0/3633393286 learned_addr learned my addr 172.21.15.37:0/3633393286 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445d7d0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d1cd0 0x7f445814c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35496/0 (socket says 172.21.15.37:35496) 2024-04-25T09:01:36.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445dfd1640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 0x7f4458160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445d7d0640 1 -- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 msgr2=0x7f4458160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445d7d0640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 0x7f4458160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445d7d0640 1 -- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445815da80 msgr2=0x7f445814ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445d7d0640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445815da80 0x7f445814ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445d7d0640 1 -- 172.21.15.37:0/3633393286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44580c7ff0 con 0x7f44580d1cd0 2024-04-25T09:01:36.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f445d7d0640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d1cd0 0x7f445814c3f0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f445808d9d0 tx=0x7f444800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f444e7fc640 1 -- 172.21.15.37:0/3633393286 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448017070 con 0x7f44580d1cd0 2024-04-25T09:01:36.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f444e7fc640 1 -- 172.21.15.37:0/3633393286 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4448005cf0 con 0x7f44580d1cd0 2024-04-25T09:01:36.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f4460235640 1 -- 172.21.15.37:0/3633393286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44580c8570 con 0x7f44580d1cd0 2024-04-25T09:01:36.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f444e7fc640 1 -- 172.21.15.37:0/3633393286 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448007310 con 0x7f44580d1cd0 2024-04-25T09:01:36.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f4460235640 1 -- 172.21.15.37:0/3633393286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44580c8290 con 0x7f44580d1cd0 2024-04-25T09:01:36.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f4460235640 1 -- 172.21.15.37:0/3633393286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f445804efa0 con 0x7f44580d1cd0 2024-04-25T09:01:36.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.386+0000 7f444e7fc640 1 -- 172.21.15.37:0/3633393286 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f444801a070 con 0x7f44580d1cd0 2024-04-25T09:01:36.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.390+0000 7f444e7fc640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f443c066f60 0x7f443c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.390+0000 7f444e7fc640 1 -- 172.21.15.37:0/3633393286 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4448084650 con 0x7f44580d1cd0 2024-04-25T09:01:36.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.390+0000 7f445dfd1640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f443c066f60 0x7f443c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.390+0000 7f445dfd1640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f443c066f60 0x7f443c069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4454009ac0 tx=0x7f4454009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.394+0000 7f444e7fc640 1 -- 172.21.15.37:0/3633393286 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4448054cd0 con 0x7f44580d1cd0 2024-04-25T09:01:36.510 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T09:01:36.522+0000 7fc23f839640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:01:36.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:36.586+0000 7f4460235640 1 -- 172.21.15.37:0/3633393286 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f445815d290 con 0x7f44580d1cd0 2024-04-25T09:01:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.570+0000 7f444e7fc640 1 -- 172.21.15.37:0/3633393286 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4448058d70 con 0x7f44580d1cd0 2024-04-25T09:01:37.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 -- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f443c066f60 msgr2=0x7f443c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f443c066f60 0x7f443c069420 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4454009ac0 tx=0x7f4454009190 comp rx=0 tx=0).stop 2024-04-25T09:01:37.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 -- 172.21.15.37:0/3633393286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d1cd0 msgr2=0x7f445814c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d1cd0 0x7f445814c3f0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f445808d9d0 tx=0x7f444800c820 comp rx=0 tx=0).stop 2024-04-25T09:01:37.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 -- 172.21.15.37:0/3633393286 shutdown_connections 2024-04-25T09:01:37.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f443c066f60 0x7f443c069420 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f445815da80 0x7f445814ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44580d1cd0 0x7f445814c3f0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 --2- 172.21.15.37:0/3633393286 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44580d07f0 0x7f4458160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 -- 172.21.15.37:0/3633393286 >> 172.21.15.37:0/3633393286 conn(0x7f445808b000 msgr2=0x7f44580ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:37.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 -- 172.21.15.37:0/3633393286 shutdown_connections 2024-04-25T09:01:37.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.574+0000 7f4460235640 1 -- 172.21.15.37:0/3633393286 wait complete. 2024-04-25T09:01:37.584 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:01:37.585 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:01:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce15e75640 1 Processor -- start 2024-04-25T09:01:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce15e75640 1 -- start start 2024-04-25T09:01:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce15e75640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fce100cf590 0x7fce100cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce15e75640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fce10072000 0x7fce10072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce15e75640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce100cbd20 con 0x7fce1015b6b0 2024-04-25T09:01:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce15e75640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce1015ba90 con 0x7fce100cf590 2024-04-25T09:01:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce15e75640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce100cc000 con 0x7fce10072000 2024-04-25T09:01:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce0effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fce100cf590 0x7fce100cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce0effd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fce100cf590 0x7fce100cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35512/0 (socket says 172.21.15.37:35512) 2024-04-25T09:01:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce0effd640 1 -- 172.21.15.37:0/756978264 learned_addr learned my addr 172.21.15.37:0/756978264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce0ffff640 1 --2- 172.21.15.37:0/756978264 >> v2:172.21.15.203:3300/0 conn(0x7fce10072000 0x7fce10072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce0effd640 1 -- 172.21.15.37:0/756978264 >> v2:172.21.15.203:3300/0 conn(0x7fce10072000 msgr2=0x7fce10072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce0effd640 1 --2- 172.21.15.37:0/756978264 >> v2:172.21.15.203:3300/0 conn(0x7fce10072000 0x7fce10072430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce0effd640 1 -- 172.21.15.37:0/756978264 >> v1:172.21.15.203:6789/0 conn(0x7fce1015b6b0 legacy=0x7fce100ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.730+0000 7fce0effd640 1 -- 172.21.15.37:0/756978264 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce100cc2e0 con 0x7fce100cf590 2024-04-25T09:01:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce0effd640 1 --2- 172.21.15.37:0/756978264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce100cf590 0x7fce100cfa20 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fce1006f000 tx=0x7fce0401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec8c50a82e0b922 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce0e7fc640 1 -- 172.21.15.37:0/756978264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce040b1070 con 0x7fce100cf590 2024-04-25T09:01:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce0e7fc640 1 -- 172.21.15.37:0/756978264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce040ac630 con 0x7fce100cf590 2024-04-25T09:01:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce0e7fc640 1 -- 172.21.15.37:0/756978264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce040a8050 con 0x7fce100cf590 2024-04-25T09:01:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 -- 172.21.15.37:0/756978264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce100cf590 msgr2=0x7fce100cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 --2- 172.21.15.37:0/756978264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce100cf590 0x7fce100cfa20 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fce1006f000 tx=0x7fce0401c4e0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 -- 172.21.15.37:0/756978264 shutdown_connections 2024-04-25T09:01:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 --2- 172.21.15.37:0/756978264 >> v2:172.21.15.203:3300/0 conn(0x7fce10072000 0x7fce10072430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 --2- 172.21.15.37:0/756978264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce100cf590 0x7fce100cfa20 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 -- 172.21.15.37:0/756978264 >> 172.21.15.37:0/756978264 conn(0x7fce1006ae60 msgr2=0x7fce1006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 -- 172.21.15.37:0/756978264 shutdown_connections 2024-04-25T09:01:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 -- 172.21.15.37:0/756978264 wait complete. 2024-04-25T09:01:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 Processor -- start 2024-04-25T09:01:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 -- start start 2024-04-25T09:01:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce10072000 0x7fce10159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce100cf590 0x7fce100c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1015b6b0 0x7fce10097eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce100caf30 con 0x7fce1015b6b0 2024-04-25T09:01:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce100cadb0 con 0x7fce100cf590 2024-04-25T09:01:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.734+0000 7fce15e75640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce101682a0 con 0x7fce10072000 2024-04-25T09:01:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1015b6b0 0x7fce10097eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1015b6b0 0x7fce10097eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35514/0 (socket says 172.21.15.37:35514) 2024-04-25T09:01:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0ffff640 1 -- 172.21.15.37:0/3926681420 learned_addr learned my addr 172.21.15.37:0/3926681420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0f7fe640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce10072000 0x7fce10159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0effd640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce100cf590 0x7fce100c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0ffff640 1 -- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce10072000 msgr2=0x7fce10159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0ffff640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce10072000 0x7fce10159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0ffff640 1 -- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce100cf590 msgr2=0x7fce100c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0ffff640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce100cf590 0x7fce100c1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0ffff640 1 -- 172.21.15.37:0/3926681420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce10098d30 con 0x7fce1015b6b0 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0ffff640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1015b6b0 0x7fce10097eb0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fce0800d800 tx=0x7fce0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fcdf7fff640 1 -- 172.21.15.37:0/3926681420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0800b6c0 con 0x7fce1015b6b0 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fcdf7fff640 1 -- 172.21.15.37:0/3926681420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce08019070 con 0x7fce1015b6b0 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fcdf7fff640 1 -- 172.21.15.37:0/3926681420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0800baa0 con 0x7fce1015b6b0 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce15e75640 1 -- 172.21.15.37:0/3926681420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce10158d50 con 0x7fce1015b6b0 2024-04-25T09:01:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce15e75640 1 -- 172.21.15.37:0/3926681420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce101592d0 con 0x7fce1015b6b0 2024-04-25T09:01:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce15e75640 1 -- 172.21.15.37:0/3926681420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1004efa0 con 0x7fce1015b6b0 2024-04-25T09:01:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fcdf7fff640 1 -- 172.21.15.37:0/3926681420 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fce0800bc40 con 0x7fce1015b6b0 2024-04-25T09:01:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fcdf7fff640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdfc066f60 0x7fcdfc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fcdf7fff640 1 -- 172.21.15.37:0/3926681420 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fce08085550 con 0x7fce1015b6b0 2024-04-25T09:01:37.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.738+0000 7fce0f7fe640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdfc066f60 0x7fcdfc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.742+0000 7fce0f7fe640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdfc066f60 0x7fcdfc069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fce101f7880 tx=0x7fce0000d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.742+0000 7fcdf7fff640 1 -- 172.21.15.37:0/3926681420 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fce10098d30 con 0x7fce1015b6b0 2024-04-25T09:01:37.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:37.942+0000 7fce15e75640 1 -- 172.21.15.37:0/3926681420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fce1015faf0 con 0x7fce1015b6b0 2024-04-25T09:01:38.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.610+0000 7fcdf7fff640 1 -- 172.21.15.37:0/3926681420 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) v1 ==== 122+0+0 (secure 0 0 0) 0x7fce080563d0 con 0x7fce1015b6b0 2024-04-25T09:01:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 -- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdfc066f60 msgr2=0x7fcdfc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdfc066f60 0x7fcdfc069420 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fce101f7880 tx=0x7fce0000d040 comp rx=0 tx=0).stop 2024-04-25T09:01:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 -- 172.21.15.37:0/3926681420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1015b6b0 msgr2=0x7fce10097eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1015b6b0 0x7fce10097eb0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7fce0800d800 tx=0x7fce0800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 -- 172.21.15.37:0/3926681420 shutdown_connections 2024-04-25T09:01:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdfc066f60 0x7fcdfc069420 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce1015b6b0 0x7fce10097eb0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fce100cf590 0x7fce100c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 --2- 172.21.15.37:0/3926681420 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fce10072000 0x7fce10159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 -- 172.21.15.37:0/3926681420 >> 172.21.15.37:0/3926681420 conn(0x7fce1006ae60 msgr2=0x7fce100b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 -- 172.21.15.37:0/3926681420 shutdown_connections 2024-04-25T09:01:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.614+0000 7fce15e75640 1 -- 172.21.15.37:0/3926681420 wait complete. 2024-04-25T09:01:38.622 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:01:38.623 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4976529640 1 Processor -- start 2024-04-25T09:01:38.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4976529640 1 -- start start 2024-04-25T09:01:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4976529640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f497009ac40 0x7f49700cd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4976529640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4970099a70 0x7f4970099f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4976529640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49700cc1b0 con 0x7f497015b780 2024-04-25T09:01:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4976529640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49700ccd80 con 0x7f497009ac40 2024-04-25T09:01:38.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4976529640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49700cc450 con 0x7f4970099a70 2024-04-25T09:01:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f496ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f497015b780 0x7f49700cf1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50552/0 (socket says 172.21.15.37:50552) 2024-04-25T09:01:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f496ffff640 1 -- 172.21.15.37:0/1283950537 learned_addr learned my addr 172.21.15.37:0/1283950537 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4974ac6640 1 --2- 172.21.15.37:0/1283950537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4970099a70 0x7f4970099f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f496f7fe640 1 --2- 172.21.15.37:0/1283950537 >> v2:172.21.15.203:3300/0 conn(0x7f497009ac40 0x7f49700cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f496effd640 1 -- 172.21.15.37:0/1283950537 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3395964099 0 0) 0x7f49700cc1b0 con 0x7f497015b780 2024-04-25T09:01:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f496effd640 1 -- 172.21.15.37:0/1283950537 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4958003530 con 0x7f497015b780 2024-04-25T09:01:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f496effd640 1 -- 172.21.15.37:0/1283950537 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4075219734 0 0) 0x7f4958003530 con 0x7f497015b780 2024-04-25T09:01:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4974ac6640 1 -- 172.21.15.37:0/1283950537 >> v2:172.21.15.203:3300/0 conn(0x7f497009ac40 msgr2=0x7f49700cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:38.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4974ac6640 1 --2- 172.21.15.37:0/1283950537 >> v2:172.21.15.203:3300/0 conn(0x7f497009ac40 0x7f49700cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4974ac6640 1 -- 172.21.15.37:0/1283950537 >> v1:172.21.15.37:6789/0 conn(0x7f497015b780 legacy=0x7f49700cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.762+0000 7f4974ac6640 1 -- 172.21.15.37:0/1283950537 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700cc730 con 0x7f4970099a70 2024-04-25T09:01:38.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f4974ac6640 1 --2- 172.21.15.37:0/1283950537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4970099a70 0x7f4970099f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4968009d90 tx=0x7f496801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c40e1fe9e689869 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f496effd640 1 -- 172.21.15.37:0/1283950537 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960001680 con 0x7f4970099a70 2024-04-25T09:01:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f496effd640 1 -- 172.21.15.37:0/1283950537 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49680b0400 con 0x7f4970099a70 2024-04-25T09:01:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f496effd640 1 -- 172.21.15.37:0/1283950537 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49680b07d0 con 0x7f4970099a70 2024-04-25T09:01:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f4976529640 1 -- 172.21.15.37:0/1283950537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4970099a70 msgr2=0x7f4970099f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f4976529640 1 --2- 172.21.15.37:0/1283950537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4970099a70 0x7f4970099f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4968009d90 tx=0x7f496801c890 comp rx=0 tx=0).stop 2024-04-25T09:01:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f4976529640 1 -- 172.21.15.37:0/1283950537 shutdown_connections 2024-04-25T09:01:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f4976529640 1 --2- 172.21.15.37:0/1283950537 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4970099a70 0x7f4970099f00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f4976529640 1 --2- 172.21.15.37:0/1283950537 >> v2:172.21.15.203:3300/0 conn(0x7f497009ac40 0x7f49700cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f4976529640 1 -- 172.21.15.37:0/1283950537 >> 172.21.15.37:0/1283950537 conn(0x7f49700b3100 msgr2=0x7f49700b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:38.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f4976529640 1 -- 172.21.15.37:0/1283950537 shutdown_connections 2024-04-25T09:01:38.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.766+0000 7f4976529640 1 -- 172.21.15.37:0/1283950537 wait complete. 2024-04-25T09:01:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4976529640 1 Processor -- start 2024-04-25T09:01:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4976529640 1 -- start start 2024-04-25T09:01:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4976529640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4970099a70 0x7f49700c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4976529640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f497009ac40 0x7f49700c5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4976529640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f497015b780 0x7f49700c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4976529640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700b5080 con 0x7f497009ac40 2024-04-25T09:01:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4976529640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700b4f00 con 0x7f497015b780 2024-04-25T09:01:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4976529640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49700b5200 con 0x7f4970099a70 2024-04-25T09:01:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f497009ac40 0x7f49700c5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f497009ac40 0x7f49700c5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35522/0 (socket says 172.21.15.37:35522) 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496f7fe640 1 -- 172.21.15.37:0/3610264234 learned_addr learned my addr 172.21.15.37:0/3610264234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496ffff640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4970099a70 0x7f49700c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4974ac6640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f497015b780 0x7f49700c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496f7fe640 1 -- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4970099a70 msgr2=0x7f49700c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496f7fe640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4970099a70 0x7f49700c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496f7fe640 1 -- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f497015b780 msgr2=0x7f49700c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496f7fe640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f497015b780 0x7f49700c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496f7fe640 1 -- 172.21.15.37:0/3610264234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700c7030 con 0x7f497009ac40 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496f7fe640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f497009ac40 0x7f49700c5100 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f496400e930 tx=0x7f496400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496cff9640 1 -- 172.21.15.37:0/3610264234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964017070 con 0x7f497009ac40 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496cff9640 1 -- 172.21.15.37:0/3610264234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496400ca00 con 0x7f497009ac40 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496cff9640 1 -- 172.21.15.37:0/3610264234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496400cdb0 con 0x7f497009ac40 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4976529640 1 -- 172.21.15.37:0/3610264234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49701fa910 con 0x7f497009ac40 2024-04-25T09:01:38.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4976529640 1 -- 172.21.15.37:0/3610264234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497006d7e0 con 0x7f497009ac40 2024-04-25T09:01:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f4976529640 1 -- 172.21.15.37:0/3610264234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497004efa0 con 0x7f497009ac40 2024-04-25T09:01:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.770+0000 7f496cff9640 1 -- 172.21.15.37:0/3610264234 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f496401a070 con 0x7f497009ac40 2024-04-25T09:01:38.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.774+0000 7f496cff9640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f495c066f60 0x7f495c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.774+0000 7f496cff9640 1 -- 172.21.15.37:0/3610264234 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f49640855d0 con 0x7f497009ac40 2024-04-25T09:01:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.774+0000 7f496ffff640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f495c066f60 0x7f495c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.774+0000 7f496ffff640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f495c066f60 0x7f495c069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4960007a10 tx=0x7f4960009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:38.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.778+0000 7f496cff9640 1 -- 172.21.15.37:0/3610264234 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4964056500 con 0x7f497009ac40 2024-04-25T09:01:38.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.966+0000 7f4976529640 1 -- 172.21.15.37:0/3610264234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49700c4ee0 con 0x7f497009ac40 2024-04-25T09:01:38.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.966+0000 7f496cff9640 1 -- 172.21.15.37:0/3610264234 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f4964055e00 con 0x7f497009ac40 2024-04-25T09:01:38.957 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:38.957 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:01:37.945874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8727":{"gid":8727,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1266122577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1266122577},{"type":"v1","addr":"172.21.15.37:6835","nonce":1266122577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:01:38.957 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 81 2024-04-25T09:01:38.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 -- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f495c066f60 msgr2=0x7f495c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f495c066f60 0x7f495c069420 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f4960007a10 tx=0x7f4960009040 comp rx=0 tx=0).stop 2024-04-25T09:01:38.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 -- 172.21.15.37:0/3610264234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f497009ac40 msgr2=0x7f49700c5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f497009ac40 0x7f49700c5100 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f496400e930 tx=0x7f496400edf0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 -- 172.21.15.37:0/3610264234 shutdown_connections 2024-04-25T09:01:38.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f495c066f60 0x7f495c069420 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f497015b780 0x7f49700c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f497009ac40 0x7f49700c5100 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 --2- 172.21.15.37:0/3610264234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4970099a70 0x7f49700c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 -- 172.21.15.37:0/3610264234 >> 172.21.15.37:0/3610264234 conn(0x7f49700b3100 msgr2=0x7f497015e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:38.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 -- 172.21.15.37:0/3610264234 shutdown_connections 2024-04-25T09:01:38.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:38.970+0000 7f4976529640 1 -- 172.21.15.37:0/3610264234 wait complete. 2024-04-25T09:01:38.975 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcff76640 1 Processor -- start 2024-04-25T09:01:39.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcff76640 1 -- start start 2024-04-25T09:01:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcff76640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fedc80d07f0 0x7fedc80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcff76640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc80d1cd0 0x7fedc806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcff76640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc806be60 con 0x7fedc815da80 2024-04-25T09:01:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcff76640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc806c440 con 0x7fedc80d07f0 2024-04-25T09:01:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcff76640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc815de60 con 0x7fedc80d1cd0 2024-04-25T09:01:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcd511640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fedc80d07f0 0x7fedc80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcd511640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fedc80d07f0 0x7fedc80d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35526/0 (socket says 172.21.15.37:35526) 2024-04-25T09:01:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcd511640 1 -- 172.21.15.37:0/837247864 learned_addr learned my addr 172.21.15.37:0/837247864 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedce513640 1 --2- 172.21.15.37:0/837247864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc80d1cd0 0x7fedc806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcd511640 1 -- 172.21.15.37:0/837247864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc80d1cd0 msgr2=0x7fedc806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcd511640 1 --2- 172.21.15.37:0/837247864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc80d1cd0 0x7fedc806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcd511640 1 -- 172.21.15.37:0/837247864 >> v1:172.21.15.203:6789/0 conn(0x7fedc815da80 legacy=0x7fedc80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:39.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.110+0000 7fedcd511640 1 -- 172.21.15.37:0/837247864 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80d2450 con 0x7fedc80d07f0 2024-04-25T09:01:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcd511640 1 --2- 172.21.15.37:0/837247864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc80d07f0 0x7fedc80d4230 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fedc808d880 tx=0x7fedb801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2129d5cffebde53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:39.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedccd10640 1 -- 172.21.15.37:0/837247864 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb80b1070 con 0x7fedc80d07f0 2024-04-25T09:01:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedccd10640 1 -- 172.21.15.37:0/837247864 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb80ac630 con 0x7fedc80d07f0 2024-04-25T09:01:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedccd10640 1 -- 172.21.15.37:0/837247864 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb80a8050 con 0x7fedc80d07f0 2024-04-25T09:01:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 -- 172.21.15.37:0/837247864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc80d07f0 msgr2=0x7fedc80d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 --2- 172.21.15.37:0/837247864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc80d07f0 0x7fedc80d4230 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fedc808d880 tx=0x7fedb801c4b0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 -- 172.21.15.37:0/837247864 shutdown_connections 2024-04-25T09:01:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 --2- 172.21.15.37:0/837247864 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc80d1cd0 0x7fedc806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 --2- 172.21.15.37:0/837247864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc80d07f0 0x7fedc80d4230 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 -- 172.21.15.37:0/837247864 >> 172.21.15.37:0/837247864 conn(0x7fedc808b000 msgr2=0x7fedc80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 -- 172.21.15.37:0/837247864 shutdown_connections 2024-04-25T09:01:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 -- 172.21.15.37:0/837247864 wait complete. 2024-04-25T09:01:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 Processor -- start 2024-04-25T09:01:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 -- start start 2024-04-25T09:01:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc80d07f0 0x7fedc8151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc80d1cd0 0x7fedc8158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc815da80 0x7fedc8158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc806d7a0 con 0x7fedc80d07f0 2024-04-25T09:01:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc806d620 con 0x7fedc80d1cd0 2024-04-25T09:01:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcff76640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc806d920 con 0x7fedc815da80 2024-04-25T09:01:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcdd12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc80d07f0 0x7fedc8151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcdd12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc80d07f0 0x7fedc8151e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35534/0 (socket says 172.21.15.37:35534) 2024-04-25T09:01:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcdd12640 1 -- 172.21.15.37:0/3223784045 learned_addr learned my addr 172.21.15.37:0/3223784045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedce513640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc815da80 0x7fedc8158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcdd12640 1 -- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc815da80 msgr2=0x7fedc8158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcdd12640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc815da80 0x7fedc8158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcd511640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc80d1cd0 0x7fedc8158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcdd12640 1 -- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc80d1cd0 msgr2=0x7fedc8158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcdd12640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc80d1cd0 0x7fedc8158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedcdd12640 1 -- 172.21.15.37:0/3223784045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc8152ed0 con 0x7fedc80d07f0 2024-04-25T09:01:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.114+0000 7fedce513640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc815da80 0x7fedc8158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedcdd12640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc80d07f0 0x7fedc8151e10 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fedc400b410 tx=0x7fedc400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedbe7fc640 1 -- 172.21.15.37:0/3223784045 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4015030 con 0x7fedc80d07f0 2024-04-25T09:01:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedbe7fc640 1 -- 172.21.15.37:0/3223784045 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc4004020 con 0x7fedc80d07f0 2024-04-25T09:01:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedcff76640 1 -- 172.21.15.37:0/3223784045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc815cf40 con 0x7fedc80d07f0 2024-04-25T09:01:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedbe7fc640 1 -- 172.21.15.37:0/3223784045 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc4004410 con 0x7fedc80d07f0 2024-04-25T09:01:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedcff76640 1 -- 172.21.15.37:0/3223784045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc8157740 con 0x7fedc80d07f0 2024-04-25T09:01:39.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedcff76640 1 -- 172.21.15.37:0/3223784045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc804efa0 con 0x7fedc80d07f0 2024-04-25T09:01:39.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedbe7fc640 1 -- 172.21.15.37:0/3223784045 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fedc4018070 con 0x7fedc80d07f0 2024-04-25T09:01:39.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedbe7fc640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fedac066f60 0x7fedac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedbe7fc640 1 -- 172.21.15.37:0/3223784045 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fedc4084d00 con 0x7fedc80d07f0 2024-04-25T09:01:39.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedcd511640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fedac066f60 0x7fedac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.118+0000 7fedcd511640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fedac066f60 0x7fedac069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fedb80ae000 tx=0x7fedb80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:39.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.122+0000 7fedbe7fc640 1 -- 172.21.15.37:0/3223784045 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fedc4055b80 con 0x7fedc80d07f0 2024-04-25T09:01:39.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.286+0000 7fedcff76640 1 -- 172.21.15.37:0/3223784045 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedc80c6960 con 0x7fedc80d07f0 2024-04-25T09:01:39.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedbe7fc640 1 -- 172.21.15.37:0/3223784045 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16848 (secure 0 0 0) 0x7fedc4055480 con 0x7fedc80d07f0 2024-04-25T09:01:39.278 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:39.278 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:01:36.569159+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:33.212328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:33.895814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 -- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fedac066f60 msgr2=0x7fedac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fedac066f60 0x7fedac069420 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fedb80ae000 tx=0x7fedb80a7000 comp rx=0 tx=0).stop 2024-04-25T09:01:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 -- 172.21.15.37:0/3223784045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc80d07f0 msgr2=0x7fedc8151e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc80d07f0 0x7fedc8151e10 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fedc400b410 tx=0x7fedc400b8d0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 -- 172.21.15.37:0/3223784045 shutdown_connections 2024-04-25T09:01:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fedac066f60 0x7fedac069420 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fedc815da80 0x7fedc8158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fedc80d1cd0 0x7fedc8158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 --2- 172.21.15.37:0/3223784045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fedc80d07f0 0x7fedc8151e10 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 -- 172.21.15.37:0/3223784045 >> 172.21.15.37:0/3223784045 conn(0x7fedc808b000 msgr2=0x7fedc80b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:39.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 -- 172.21.15.37:0/3223784045 shutdown_connections 2024-04-25T09:01:39.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.290+0000 7fedcff76640 1 -- 172.21.15.37:0/3223784045 wait complete. 2024-04-25T09:01:39.297 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:01:39.297 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:01:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bdd000640 1 Processor -- start 2024-04-25T09:01:39.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bdd000640 1 -- start start 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bdd000640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3bd8144ca0 0x7f3bd806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bdd000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd806ec60 0x7f3bd8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bdd000640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd80ccc70 con 0x7f3bd80ce510 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bdd000640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd80cc2b0 con 0x7f3bd8144ca0 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bdd000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd806e290 con 0x7f3bd806ec60 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bd659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3bd80ce510 0x7f3bd80ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40960/0 (socket says 172.21.15.37:40960) 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bd659c640 1 -- 172.21.15.37:0/3872327661 learned_addr learned my addr 172.21.15.37:0/3872327661 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bd5d9b640 1 --2- 172.21.15.37:0/3872327661 >> v2:172.21.15.37:3300/0 conn(0x7f3bd8144ca0 0x7f3bd806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bd6d9d640 1 --2- 172.21.15.37:0/3872327661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd806ec60 0x7f3bd8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bd559a640 1 -- 172.21.15.37:0/3872327661 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1042019259 0 0) 0x7f3bd80ccc70 con 0x7f3bd80ce510 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bd559a640 1 -- 172.21.15.37:0/3872327661 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bc0003530 con 0x7f3bd80ce510 2024-04-25T09:01:39.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bd5d9b640 1 -- 172.21.15.37:0/3872327661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd806ec60 msgr2=0x7f3bd8142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:39.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bd5d9b640 1 --2- 172.21.15.37:0/3872327661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd806ec60 0x7f3bd8142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bd559a640 1 -- 172.21.15.37:0/3872327661 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 119913682 0 0) 0x7f3bc0003530 con 0x7f3bd80ce510 2024-04-25T09:01:39.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bd5d9b640 1 -- 172.21.15.37:0/3872327661 >> v1:172.21.15.37:6789/0 conn(0x7f3bd80ce510 legacy=0x7f3bd80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.434+0000 7f3bd5d9b640 1 -- 172.21.15.37:0/3872327661 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd806dc40 con 0x7f3bd8144ca0 2024-04-25T09:01:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bd5d9b640 1 --2- 172.21.15.37:0/3872327661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8144ca0 0x7f3bd806e720 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0097c0 tx=0x7f3bcc01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16ca2b540ad60af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bd559a640 1 -- 172.21.15.37:0/3872327661 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bcc0b5070 con 0x7f3bd8144ca0 2024-04-25T09:01:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bd559a640 1 -- 172.21.15.37:0/3872327661 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bcc0a85f0 con 0x7f3bd8144ca0 2024-04-25T09:01:39.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bd559a640 1 -- 172.21.15.37:0/3872327661 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0001a00 con 0x7f3bd8144ca0 2024-04-25T09:01:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bdd000640 1 -- 172.21.15.37:0/3872327661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8144ca0 msgr2=0x7f3bd806e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bdd000640 1 --2- 172.21.15.37:0/3872327661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8144ca0 0x7f3bd806e720 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f3bcc0097c0 tx=0x7f3bcc01ebb0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bdd000640 1 -- 172.21.15.37:0/3872327661 shutdown_connections 2024-04-25T09:01:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bdd000640 1 --2- 172.21.15.37:0/3872327661 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd806ec60 0x7f3bd8142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bdd000640 1 --2- 172.21.15.37:0/3872327661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8144ca0 0x7f3bd806e720 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bdd000640 1 -- 172.21.15.37:0/3872327661 >> 172.21.15.37:0/3872327661 conn(0x7f3bd80b11a0 msgr2=0x7f3bd80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:39.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bdd000640 1 -- 172.21.15.37:0/3872327661 shutdown_connections 2024-04-25T09:01:39.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.438+0000 7f3bdd000640 1 -- 172.21.15.37:0/3872327661 wait complete. 2024-04-25T09:01:39.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bdd000640 1 Processor -- start 2024-04-25T09:01:39.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bdd000640 1 -- start start 2024-04-25T09:01:39.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bdd000640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd806ec60 0x7f3bd81f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bdd000640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd80ce510 0x7f3bd81e26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bdd000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8144ca0 0x7f3bd80ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bdd000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd81451f0 con 0x7f3bd8144ca0 2024-04-25T09:01:39.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bdd000640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd80cc900 con 0x7f3bd80ce510 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bdd000640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd8145370 con 0x7f3bd806ec60 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bd6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8144ca0 0x7f3bd80ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bd6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8144ca0 0x7f3bd80ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51792/0 (socket says 172.21.15.37:51792) 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bd6d9d640 1 -- 172.21.15.37:0/3385134888 learned_addr learned my addr 172.21.15.37:0/3385134888 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bd5d9b640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd80ce510 0x7f3bd81e26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bd659c640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd806ec60 0x7f3bd81f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bd6d9d640 1 -- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd806ec60 msgr2=0x7f3bd81f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bd6d9d640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd806ec60 0x7f3bd81f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bd6d9d640 1 -- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd80ce510 msgr2=0x7f3bd81e26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bd6d9d640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd80ce510 0x7f3bd81e26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bd6d9d640 1 -- 172.21.15.37:0/3385134888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd81e3dd0 con 0x7f3bd8144ca0 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bd6d9d640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8144ca0 0x7f3bd80ca1f0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800e8b0 tx=0x7f3bc800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bc6ffd640 1 -- 172.21.15.37:0/3385134888 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8017070 con 0x7f3bd8144ca0 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bc6ffd640 1 -- 172.21.15.37:0/3385134888 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc800c940 con 0x7f3bd8144ca0 2024-04-25T09:01:39.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bdd000640 1 -- 172.21.15.37:0/3385134888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd81e31c0 con 0x7f3bd8144ca0 2024-04-25T09:01:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bc6ffd640 1 -- 172.21.15.37:0/3385134888 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc800cd10 con 0x7f3bd8144ca0 2024-04-25T09:01:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bdd000640 1 -- 172.21.15.37:0/3385134888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd81e3780 con 0x7f3bd8144ca0 2024-04-25T09:01:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bdd000640 1 -- 172.21.15.37:0/3385134888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd81f1340 con 0x7f3bd8144ca0 2024-04-25T09:01:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.442+0000 7f3bc6ffd640 1 -- 172.21.15.37:0/3385134888 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3bc801a070 con 0x7f3bd8144ca0 2024-04-25T09:01:39.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.446+0000 7f3bc6ffd640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3bb8066f60 0x7f3bb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.446+0000 7f3bc6ffd640 1 -- 172.21.15.37:0/3385134888 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3bc8085190 con 0x7f3bd8144ca0 2024-04-25T09:01:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.446+0000 7f3bd659c640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3bb8066f60 0x7f3bb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.446+0000 7f3bd659c640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3bb8066f60 0x7f3bb8069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000e270 tx=0x7f3bd0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:39.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.450+0000 7f3bc6ffd640 1 -- 172.21.15.37:0/3385134888 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3bc8056010 con 0x7f3bd8144ca0 2024-04-25T09:01:39.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.602+0000 7f3bdd000640 1 -- 172.21.15.37:0/3385134888 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3bd804efa0 con 0x7f3bb8066f60 2024-04-25T09:01:39.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.618+0000 7f3bc6ffd640 1 -- 172.21.15.37:0/3385134888 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252494 (secure 0 0 0) 0x7f3bd804efa0 con 0x7f3bb8066f60 2024-04-25T09:01:39.609 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:39.613 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T09:01:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 -- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3bb8066f60 msgr2=0x7f3bb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3bb8066f60 0x7f3bb8069420 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000e270 tx=0x7f3bd0009040 comp rx=0 tx=0).stop 2024-04-25T09:01:39.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 -- 172.21.15.37:0/3385134888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8144ca0 msgr2=0x7f3bd80ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8144ca0 0x7f3bd80ca1f0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800e8b0 tx=0x7f3bc800ed70 comp rx=0 tx=0).stop 2024-04-25T09:01:39.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 -- 172.21.15.37:0/3385134888 shutdown_connections 2024-04-25T09:01:39.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3bb8066f60 0x7f3bb8069420 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3bd8144ca0 0x7f3bd80ca1f0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3bd80ce510 0x7f3bd81e26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 --2- 172.21.15.37:0/3385134888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3bd806ec60 0x7f3bd81f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 -- 172.21.15.37:0/3385134888 >> 172.21.15.37:0/3385134888 conn(0x7f3bd80b11a0 msgr2=0x7f3bd80c4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:39.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 -- 172.21.15.37:0/3385134888 shutdown_connections 2024-04-25T09:01:39.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.626+0000 7f3bdd000640 1 -- 172.21.15.37:0/3385134888 wait complete. 2024-04-25T09:01:39.629 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-25T09:01:39.316629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261360,"kb_used_data":11880,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713360,"statfs":{"total":773094113280,"available":772826480640,"internally_reserved":0,"allocated":12165120,"data_stored":6279436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.784572"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581092+0000","last_change":"2024-04-25T09:01:35.598819+0000","last_active":"2024-04-25T09:01:36.581092+0000","last_peered":"2024-04-25T09:01:36.581092+0000","last_clean":"2024-04-25T09:01:36.581092+0000","last_became_active":"2024-04-25T09:01:35.598510+0000","last_became_peered":"2024-04-25T09:01:35.598510+0000","last_unstale":"2024-04-25T09:01:36.581092+0000","last_undegraded":"2024-04-25T09:01:36.581092+0000","last_fullsized":"2024-04-25T09:01:36.581092+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:30.609001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582503+0000","last_change":"2024-04-25T09:01:35.622024+0000","last_active":"2024-04-25T09:01:36.582503+0000","last_peered":"2024-04-25T09:01:36.582503+0000","last_clean":"2024-04-25T09:01:36.582503+0000","last_became_active":"2024-04-25T09:01:35.621240+0000","last_became_peered":"2024-04-25T09:01:35.621240+0000","last_unstale":"2024-04-25T09:01:36.582503+0000","last_undegraded":"2024-04-25T09:01:36.582503+0000","last_fullsized":"2024-04-25T09:01:36.582503+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:32.485434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580948+0000","last_change":"2024-04-25T09:01:35.634574+0000","last_active":"2024-04-25T09:01:36.580948+0000","last_peered":"2024-04-25T09:01:36.580948+0000","last_clean":"2024-04-25T09:01:36.580948+0000","last_became_active":"2024-04-25T09:01:35.634294+0000","last_became_peered":"2024-04-25T09:01:35.634294+0000","last_unstale":"2024-04-25T09:01:36.580948+0000","last_undegraded":"2024-04-25T09:01:36.580948+0000","last_fullsized":"2024-04-25T09:01:36.580948+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:16.196961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.943906+0000","last_change":"2024-04-25T09:01:35.601636+0000","last_active":"2024-04-25T09:01:36.943906+0000","last_peered":"2024-04-25T09:01:36.943906+0000","last_clean":"2024-04-25T09:01:36.943906+0000","last_became_active":"2024-04-25T09:01:35.601313+0000","last_became_peered":"2024-04-25T09:01:35.601313+0000","last_unstale":"2024-04-25T09:01:36.943906+0000","last_undegraded":"2024-04-25T09:01:36.943906+0000","last_fullsized":"2024-04-25T09:01:36.943906+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:55.994484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945252+0000","last_change":"2024-04-25T09:01:35.602254+0000","last_active":"2024-04-25T09:01:36.945252+0000","last_peered":"2024-04-25T09:01:36.945252+0000","last_clean":"2024-04-25T09:01:36.945252+0000","last_became_active":"2024-04-25T09:01:35.601977+0000","last_became_peered":"2024-04-25T09:01:35.601977+0000","last_unstale":"2024-04-25T09:01:36.945252+0000","last_undegraded":"2024-04-25T09:01:36.945252+0000","last_fullsized":"2024-04-25T09:01:36.945252+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:07.559766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581624+0000","last_change":"2024-04-25T09:01:35.619211+0000","last_active":"2024-04-25T09:01:36.581624+0000","last_peered":"2024-04-25T09:01:36.581624+0000","last_clean":"2024-04-25T09:01:36.581624+0000","last_became_active":"2024-04-25T09:01:35.617848+0000","last_became_peered":"2024-04-25T09:01:35.617848+0000","last_unstale":"2024-04-25T09:01:36.581624+0000","last_undegraded":"2024-04-25T09:01:36.581624+0000","last_fullsized":"2024-04-25T09:01:36.581624+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:46.480732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580301+0000","last_change":"2024-04-25T09:01:35.617628+0000","last_active":"2024-04-25T09:01:36.580301+0000","last_peered":"2024-04-25T09:01:36.580301+0000","last_clean":"2024-04-25T09:01:36.580301+0000","last_became_active":"2024-04-25T09:01:35.616809+0000","last_became_peered":"2024-04-25T09:01:35.616809+0000","last_unstale":"2024-04-25T09:01:36.580301+0000","last_undegraded":"2024-04-25T09:01:36.580301+0000","last_fullsized":"2024-04-25T09:01:36.580301+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:35.354671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.574932+0000","last_change":"2024-04-25T09:01:35.592323+0000","last_active":"2024-04-25T09:01:36.574932+0000","last_peered":"2024-04-25T09:01:36.574932+0000","last_clean":"2024-04-25T09:01:36.574932+0000","last_became_active":"2024-04-25T09:01:35.591443+0000","last_became_peered":"2024-04-25T09:01:35.591443+0000","last_unstale":"2024-04-25T09:01:36.574932+0000","last_undegraded":"2024-04-25T09:01:36.574932+0000","last_fullsized":"2024-04-25T09:01:36.574932+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:33.353156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582183+0000","last_change":"2024-04-25T09:01:35.618923+0000","last_active":"2024-04-25T09:01:36.582183+0000","last_peered":"2024-04-25T09:01:36.582183+0000","last_clean":"2024-04-25T09:01:36.582183+0000","last_became_active":"2024-04-25T09:01:35.608809+0000","last_became_peered":"2024-04-25T09:01:35.608809+0000","last_unstale":"2024-04-25T09:01:36.582183+0000","last_undegraded":"2024-04-25T09:01:36.582183+0000","last_fullsized":"2024-04-25T09:01:36.582183+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:22.247172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945929+0000","last_change":"2024-04-25T09:01:35.620324+0000","last_active":"2024-04-25T09:01:36.945929+0000","last_peered":"2024-04-25T09:01:36.945929+0000","last_clean":"2024-04-25T09:01:36.945929+0000","last_became_active":"2024-04-25T09:01:35.620049+0000","last_became_peered":"2024-04-25T09:01:35.620049+0000","last_unstale":"2024-04-25T09:01:36.945929+0000","last_undegraded":"2024-04-25T09:01:36.945929+0000","last_fullsized":"2024-04-25T09:01:36.945929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:32.485434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.588589+0000","last_change":"2024-04-25T09:01:35.635963+0000","last_active":"2024-04-25T09:01:36.588589+0000","last_peered":"2024-04-25T09:01:36.588589+0000","last_clean":"2024-04-25T09:01:36.588589+0000","last_became_active":"2024-04-25T09:01:35.635417+0000","last_became_peered":"2024-04-25T09:01:35.635417+0000","last_unstale":"2024-04-25T09:01:36.588589+0000","last_undegraded":"2024-04-25T09:01:36.588589+0000","last_fullsized":"2024-04-25T09:01:36.588589+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:35.354671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583536+0000","last_change":"2024-04-25T09:01:35.600850+0000","last_active":"2024-04-25T09:01:36.583536+0000","last_peered":"2024-04-25T09:01:36.583536+0000","last_clean":"2024-04-25T09:01:36.583536+0000","last_became_active":"2024-04-25T09:01:35.600122+0000","last_became_peered":"2024-04-25T09:01:35.600122+0000","last_unstale":"2024-04-25T09:01:36.583536+0000","last_undegraded":"2024-04-25T09:01:36.583536+0000","last_fullsized":"2024-04-25T09:01:36.583536+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:22.161066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.943719+0000","last_change":"2024-04-25T09:01:35.634588+0000","last_active":"2024-04-25T09:01:36.943719+0000","last_peered":"2024-04-25T09:01:36.943719+0000","last_clean":"2024-04-25T09:01:36.943719+0000","last_became_active":"2024-04-25T09:01:35.634278+0000","last_became_peered":"2024-04-25T09:01:35.634278+0000","last_unstale":"2024-04-25T09:01:36.943719+0000","last_undegraded":"2024-04-25T09:01:36.943719+0000","last_fullsized":"2024-04-25T09:01:36.943719+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:05.385775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946078+0000","last_change":"2024-04-25T09:01:35.612780+0000","last_active":"2024-04-25T09:01:36.946078+0000","last_peered":"2024-04-25T09:01:36.946078+0000","last_clean":"2024-04-25T09:01:36.946078+0000","last_became_active":"2024-04-25T09:01:35.611972+0000","last_became_peered":"2024-04-25T09:01:35.611972+0000","last_unstale":"2024-04-25T09:01:36.946078+0000","last_undegraded":"2024-04-25T09:01:36.946078+0000","last_fullsized":"2024-04-25T09:01:36.946078+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:35.354671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946324+0000","last_change":"2024-04-25T09:01:35.639776+0000","last_active":"2024-04-25T09:01:36.946324+0000","last_peered":"2024-04-25T09:01:36.946324+0000","last_clean":"2024-04-25T09:01:36.946324+0000","last_became_active":"2024-04-25T09:01:35.639301+0000","last_became_peered":"2024-04-25T09:01:35.639301+0000","last_unstale":"2024-04-25T09:01:36.946324+0000","last_undegraded":"2024-04-25T09:01:36.946324+0000","last_fullsized":"2024-04-25T09:01:36.946324+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:07.559766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946004+0000","last_change":"2024-04-25T09:01:35.611878+0000","last_active":"2024-04-25T09:01:36.946004+0000","last_peered":"2024-04-25T09:01:36.946004+0000","last_clean":"2024-04-25T09:01:36.946004+0000","last_became_active":"2024-04-25T09:01:35.611228+0000","last_became_peered":"2024-04-25T09:01:35.611228+0000","last_unstale":"2024-04-25T09:01:36.946004+0000","last_undegraded":"2024-04-25T09:01:36.946004+0000","last_fullsized":"2024-04-25T09:01:36.946004+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:00.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581246+0000","last_change":"2024-04-25T09:01:35.629412+0000","last_active":"2024-04-25T09:01:36.581246+0000","last_peered":"2024-04-25T09:01:36.581246+0000","last_clean":"2024-04-25T09:01:36.581246+0000","last_became_active":"2024-04-25T09:01:35.629080+0000","last_became_peered":"2024-04-25T09:01:35.629080+0000","last_unstale":"2024-04-25T09:01:36.581246+0000","last_undegraded":"2024-04-25T09:01:36.581246+0000","last_fullsized":"2024-04-25T09:01:36.581246+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:11.426534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582537+0000","last_change":"2024-04-25T09:01:35.616209+0000","last_active":"2024-04-25T09:01:36.582537+0000","last_peered":"2024-04-25T09:01:36.582537+0000","last_clean":"2024-04-25T09:01:36.582537+0000","last_became_active":"2024-04-25T09:01:35.609544+0000","last_became_peered":"2024-04-25T09:01:35.609544+0000","last_unstale":"2024-04-25T09:01:36.582537+0000","last_undegraded":"2024-04-25T09:01:36.582537+0000","last_fullsized":"2024-04-25T09:01:36.582537+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:15.579088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581055+0000","last_change":"2024-04-25T09:01:35.619101+0000","last_active":"2024-04-25T09:01:36.581055+0000","last_peered":"2024-04-25T09:01:36.581055+0000","last_clean":"2024-04-25T09:01:36.581055+0000","last_became_active":"2024-04-25T09:01:35.618851+0000","last_became_peered":"2024-04-25T09:01:35.618851+0000","last_unstale":"2024-04-25T09:01:36.581055+0000","last_undegraded":"2024-04-25T09:01:36.581055+0000","last_fullsized":"2024-04-25T09:01:36.581055+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:29.138706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583715+0000","last_change":"2024-04-25T09:01:35.615968+0000","last_active":"2024-04-25T09:01:36.583715+0000","last_peered":"2024-04-25T09:01:36.583715+0000","last_clean":"2024-04-25T09:01:36.583715+0000","last_became_active":"2024-04-25T09:01:35.614878+0000","last_became_peered":"2024-04-25T09:01:35.614878+0000","last_unstale":"2024-04-25T09:01:36.583715+0000","last_undegraded":"2024-04-25T09:01:36.583715+0000","last_fullsized":"2024-04-25T09:01:36.583715+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:26.715787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575852+0000","last_change":"2024-04-25T09:01:35.628633+0000","last_active":"2024-04-25T09:01:36.575852+0000","last_peered":"2024-04-25T09:01:36.575852+0000","last_clean":"2024-04-25T09:01:36.575852+0000","last_became_active":"2024-04-25T09:01:35.628302+0000","last_became_peered":"2024-04-25T09:01:35.628302+0000","last_unstale":"2024-04-25T09:01:36.575852+0000","last_undegraded":"2024-04-25T09:01:36.575852+0000","last_fullsized":"2024-04-25T09:01:36.575852+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:20.531268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945303+0000","last_change":"2024-04-25T09:01:35.634747+0000","last_active":"2024-04-25T09:01:36.945303+0000","last_peered":"2024-04-25T09:01:36.945303+0000","last_clean":"2024-04-25T09:01:36.945303+0000","last_became_active":"2024-04-25T09:01:35.634264+0000","last_became_peered":"2024-04-25T09:01:35.634264+0000","last_unstale":"2024-04-25T09:01:36.945303+0000","last_undegraded":"2024-04-25T09:01:36.945303+0000","last_fullsized":"2024-04-25T09:01:36.945303+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:46.480732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580011+0000","last_change":"2024-04-25T09:01:35.629381+0000","last_active":"2024-04-25T09:01:36.580011+0000","last_peered":"2024-04-25T09:01:36.580011+0000","last_clean":"2024-04-25T09:01:36.580011+0000","last_became_active":"2024-04-25T09:01:35.629055+0000","last_became_peered":"2024-04-25T09:01:35.629055+0000","last_unstale":"2024-04-25T09:01:36.580011+0000","last_undegraded":"2024-04-25T09:01:36.580011+0000","last_fullsized":"2024-04-25T09:01:36.580011+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:11.367992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944550+0000","last_change":"2024-04-25T09:01:35.596313+0000","last_active":"2024-04-25T09:01:36.944550+0000","last_peered":"2024-04-25T09:01:36.944550+0000","last_clean":"2024-04-25T09:01:36.944550+0000","last_became_active":"2024-04-25T09:01:35.595868+0000","last_became_peered":"2024-04-25T09:01:35.595868+0000","last_unstale":"2024-04-25T09:01:36.944550+0000","last_undegraded":"2024-04-25T09:01:36.944550+0000","last_fullsized":"2024-04-25T09:01:36.944550+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:00.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583597+0000","last_change":"2024-04-25T09:01:35.637670+0000","last_active":"2024-04-25T09:01:36.583597+0000","last_peered":"2024-04-25T09:01:36.583597+0000","last_clean":"2024-04-25T09:01:36.583597+0000","last_became_active":"2024-04-25T09:01:35.637126+0000","last_became_peered":"2024-04-25T09:01:35.637126+0000","last_unstale":"2024-04-25T09:01:36.583597+0000","last_undegraded":"2024-04-25T09:01:36.583597+0000","last_fullsized":"2024-04-25T09:01:36.583597+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.998517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583836+0000","last_change":"2024-04-25T09:01:35.624323+0000","last_active":"2024-04-25T09:01:36.583836+0000","last_peered":"2024-04-25T09:01:36.583836+0000","last_clean":"2024-04-25T09:01:36.583836+0000","last_became_active":"2024-04-25T09:01:35.623726+0000","last_became_peered":"2024-04-25T09:01:35.623726+0000","last_unstale":"2024-04-25T09:01:36.583836+0000","last_undegraded":"2024-04-25T09:01:36.583836+0000","last_fullsized":"2024-04-25T09:01:36.583836+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:30.110950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.588811+0000","last_change":"2024-04-25T09:01:35.631687+0000","last_active":"2024-04-25T09:01:36.588811+0000","last_peered":"2024-04-25T09:01:36.588811+0000","last_clean":"2024-04-25T09:01:36.588811+0000","last_became_active":"2024-04-25T09:01:35.626024+0000","last_became_peered":"2024-04-25T09:01:35.626024+0000","last_unstale":"2024-04-25T09:01:36.588811+0000","last_undegraded":"2024-04-25T09:01:36.588811+0000","last_fullsized":"2024-04-25T09:01:36.588811+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:46.480732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582636+0000","last_change":"2024-04-25T09:01:35.628421+0000","last_active":"2024-04-25T09:01:36.582636+0000","last_peered":"2024-04-25T09:01:36.582636+0000","last_clean":"2024-04-25T09:01:36.582636+0000","last_became_active":"2024-04-25T09:01:35.627742+0000","last_became_peered":"2024-04-25T09:01:35.627742+0000","last_unstale":"2024-04-25T09:01:36.582636+0000","last_undegraded":"2024-04-25T09:01:36.582636+0000","last_fullsized":"2024-04-25T09:01:36.582636+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:22.161066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.576271+0000","last_change":"2024-04-25T09:01:35.610750+0000","last_active":"2024-04-25T09:01:36.576271+0000","last_peered":"2024-04-25T09:01:36.576271+0000","last_clean":"2024-04-25T09:01:36.576271+0000","last_became_active":"2024-04-25T09:01:35.610419+0000","last_became_peered":"2024-04-25T09:01:35.610419+0000","last_unstale":"2024-04-25T09:01:36.576271+0000","last_undegraded":"2024-04-25T09:01:36.576271+0000","last_fullsized":"2024-04-25T09:01:36.576271+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:27.858482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581203+0000","last_change":"2024-04-25T09:01:35.639629+0000","last_active":"2024-04-25T09:01:36.581203+0000","last_peered":"2024-04-25T09:01:36.581203+0000","last_clean":"2024-04-25T09:01:36.581203+0000","last_became_active":"2024-04-25T09:01:35.639167+0000","last_became_peered":"2024-04-25T09:01:35.639167+0000","last_unstale":"2024-04-25T09:01:36.581203+0000","last_undegraded":"2024-04-25T09:01:36.581203+0000","last_fullsized":"2024-04-25T09:01:36.581203+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:56.955538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580893+0000","last_change":"2024-04-25T09:01:35.622072+0000","last_active":"2024-04-25T09:01:36.580893+0000","last_peered":"2024-04-25T09:01:36.580893+0000","last_clean":"2024-04-25T09:01:36.580893+0000","last_became_active":"2024-04-25T09:01:35.621406+0000","last_became_peered":"2024-04-25T09:01:35.621406+0000","last_unstale":"2024-04-25T09:01:36.580893+0000","last_undegraded":"2024-04-25T09:01:36.580893+0000","last_fullsized":"2024-04-25T09:01:36.580893+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:07.559766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.574808+0000","last_change":"2024-04-25T09:01:35.581383+0000","last_active":"2024-04-25T09:01:36.574808+0000","last_peered":"2024-04-25T09:01:36.574808+0000","last_clean":"2024-04-25T09:01:36.574808+0000","last_became_active":"2024-04-25T09:01:35.581077+0000","last_became_peered":"2024-04-25T09:01:35.581077+0000","last_unstale":"2024-04-25T09:01:36.574808+0000","last_undegraded":"2024-04-25T09:01:36.574808+0000","last_fullsized":"2024-04-25T09:01:36.574808+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:41.124569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580389+0000","last_change":"2024-04-25T09:01:35.608337+0000","last_active":"2024-04-25T09:01:36.580389+0000","last_peered":"2024-04-25T09:01:36.580389+0000","last_clean":"2024-04-25T09:01:36.580389+0000","last_became_active":"2024-04-25T09:01:35.607939+0000","last_became_peered":"2024-04-25T09:01:35.607939+0000","last_unstale":"2024-04-25T09:01:36.580389+0000","last_undegraded":"2024-04-25T09:01:36.580389+0000","last_fullsized":"2024-04-25T09:01:36.580389+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:07.681259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"109'4","reported_seq":18,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944319+0000","last_change":"2024-04-25T09:01:34.579573+0000","last_active":"2024-04-25T09:01:36.944319+0000","last_peered":"2024-04-25T09:01:36.944319+0000","last_clean":"2024-04-25T09:01:36.944319+0000","last_became_active":"2024-04-25T09:01:34.579214+0000","last_became_peered":"2024-04-25T09:01:34.579214+0000","last_unstale":"2024-04-25T09:01:36.944319+0000","last_undegraded":"2024-04-25T09:01:36.944319+0000","last_fullsized":"2024-04-25T09:01:36.944319+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:21.148978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575154+0000","last_change":"2024-04-25T09:01:35.591851+0000","last_active":"2024-04-25T09:01:36.575154+0000","last_peered":"2024-04-25T09:01:36.575154+0000","last_clean":"2024-04-25T09:01:36.575154+0000","last_became_active":"2024-04-25T09:01:35.591660+0000","last_became_peered":"2024-04-25T09:01:35.591660+0000","last_unstale":"2024-04-25T09:01:36.575154+0000","last_undegraded":"2024-04-25T09:01:36.575154+0000","last_fullsized":"2024-04-25T09:01:36.575154+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:43.504824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575628+0000","last_change":"2024-04-25T09:01:34.579195+0000","last_active":"2024-04-25T09:01:36.575628+0000","last_peered":"2024-04-25T09:01:36.575628+0000","last_clean":"2024-04-25T09:01:36.575628+0000","last_became_active":"2024-04-25T09:01:34.578755+0000","last_became_peered":"2024-04-25T09:01:34.578755+0000","last_unstale":"2024-04-25T09:01:36.575628+0000","last_undegraded":"2024-04-25T09:01:36.575628+0000","last_fullsized":"2024-04-25T09:01:36.575628+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:26.293458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581582+0000","last_change":"2024-04-25T09:01:35.599297+0000","last_active":"2024-04-25T09:01:36.581582+0000","last_peered":"2024-04-25T09:01:36.581582+0000","last_clean":"2024-04-25T09:01:36.581582+0000","last_became_active":"2024-04-25T09:01:35.598386+0000","last_became_peered":"2024-04-25T09:01:35.598386+0000","last_unstale":"2024-04-25T09:01:36.581582+0000","last_undegraded":"2024-04-25T09:01:36.581582+0000","last_fullsized":"2024-04-25T09:01:36.581582+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:15.579088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583367+0000","last_change":"2024-04-25T09:01:34.563633+0000","last_active":"2024-04-25T09:01:36.583367+0000","last_peered":"2024-04-25T09:01:36.583367+0000","last_clean":"2024-04-25T09:01:36.583367+0000","last_became_active":"2024-04-25T09:01:34.562955+0000","last_became_peered":"2024-04-25T09:01:34.562955+0000","last_unstale":"2024-04-25T09:01:36.583367+0000","last_undegraded":"2024-04-25T09:01:36.583367+0000","last_fullsized":"2024-04-25T09:01:36.583367+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:25.364305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.588165+0000","last_change":"2024-04-25T09:01:35.626092+0000","last_active":"2024-04-25T09:01:36.588165+0000","last_peered":"2024-04-25T09:01:36.588165+0000","last_clean":"2024-04-25T09:01:36.588165+0000","last_became_active":"2024-04-25T09:01:35.625490+0000","last_became_peered":"2024-04-25T09:01:35.625490+0000","last_unstale":"2024-04-25T09:01:36.588165+0000","last_undegraded":"2024-04-25T09:01:36.588165+0000","last_fullsized":"2024-04-25T09:01:36.588165+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:00.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945443+0000","last_change":"2024-04-25T09:01:34.572762+0000","last_active":"2024-04-25T09:01:36.945443+0000","last_peered":"2024-04-25T09:01:36.945443+0000","last_clean":"2024-04-25T09:01:36.945443+0000","last_became_active":"2024-04-25T09:01:34.572438+0000","last_became_peered":"2024-04-25T09:01:34.572438+0000","last_unstale":"2024-04-25T09:01:36.945443+0000","last_undegraded":"2024-04-25T09:01:36.945443+0000","last_fullsized":"2024-04-25T09:01:36.945443+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:14.312491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944547+0000","last_change":"2024-04-25T09:01:35.595056+0000","last_active":"2024-04-25T09:01:36.944547+0000","last_peered":"2024-04-25T09:01:36.944547+0000","last_clean":"2024-04-25T09:01:36.944547+0000","last_became_active":"2024-04-25T09:01:35.594690+0000","last_became_peered":"2024-04-25T09:01:35.594690+0000","last_unstale":"2024-04-25T09:01:36.944547+0000","last_undegraded":"2024-04-25T09:01:36.944547+0000","last_fullsized":"2024-04-25T09:01:36.944547+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:16.186800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946181+0000","last_change":"2024-04-25T09:01:34.583811+0000","last_active":"2024-04-25T09:01:36.946181+0000","last_peered":"2024-04-25T09:01:36.946181+0000","last_clean":"2024-04-25T09:01:36.946181+0000","last_became_active":"2024-04-25T09:01:34.583046+0000","last_became_peered":"2024-04-25T09:01:34.583046+0000","last_unstale":"2024-04-25T09:01:36.946181+0000","last_undegraded":"2024-04-25T09:01:36.946181+0000","last_fullsized":"2024-04-25T09:01:36.946181+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:08.313494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581452+0000","last_change":"2024-04-25T09:01:35.619247+0000","last_active":"2024-04-25T09:01:36.581452+0000","last_peered":"2024-04-25T09:01:36.581452+0000","last_clean":"2024-04-25T09:01:36.581452+0000","last_became_active":"2024-04-25T09:01:35.618925+0000","last_became_peered":"2024-04-25T09:01:35.618925+0000","last_unstale":"2024-04-25T09:01:36.581452+0000","last_undegraded":"2024-04-25T09:01:36.581452+0000","last_fullsized":"2024-04-25T09:01:36.581452+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:34.661103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.587730+0000","last_change":"2024-04-25T09:01:34.567928+0000","last_active":"2024-04-25T09:01:36.587730+0000","last_peered":"2024-04-25T09:01:36.587730+0000","last_clean":"2024-04-25T09:01:36.587730+0000","last_became_active":"2024-04-25T09:01:34.563362+0000","last_became_peered":"2024-04-25T09:01:34.563362+0000","last_unstale":"2024-04-25T09:01:36.587730+0000","last_undegraded":"2024-04-25T09:01:36.587730+0000","last_fullsized":"2024-04-25T09:01:36.587730+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:25.909339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582949+0000","last_change":"2024-04-25T09:01:35.633300+0000","last_active":"2024-04-25T09:01:36.582949+0000","last_peered":"2024-04-25T09:01:36.582949+0000","last_clean":"2024-04-25T09:01:36.582949+0000","last_became_active":"2024-04-25T09:01:35.632636+0000","last_became_peered":"2024-04-25T09:01:35.632636+0000","last_unstale":"2024-04-25T09:01:36.582949+0000","last_undegraded":"2024-04-25T09:01:36.582949+0000","last_fullsized":"2024-04-25T09:01:36.582949+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.998517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581281+0000","last_change":"2024-04-25T09:01:34.562921+0000","last_active":"2024-04-25T09:01:36.581281+0000","last_peered":"2024-04-25T09:01:36.581281+0000","last_clean":"2024-04-25T09:01:36.581281+0000","last_became_active":"2024-04-25T09:01:34.562136+0000","last_became_peered":"2024-04-25T09:01:34.562136+0000","last_unstale":"2024-04-25T09:01:36.581281+0000","last_undegraded":"2024-04-25T09:01:36.581281+0000","last_fullsized":"2024-04-25T09:01:36.581281+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:18.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944492+0000","last_change":"2024-04-25T09:01:35.598252+0000","last_active":"2024-04-25T09:01:36.944492+0000","last_peered":"2024-04-25T09:01:36.944492+0000","last_clean":"2024-04-25T09:01:36.944492+0000","last_became_active":"2024-04-25T09:01:35.597495+0000","last_became_peered":"2024-04-25T09:01:35.597495+0000","last_unstale":"2024-04-25T09:01:36.944492+0000","last_undegraded":"2024-04-25T09:01:36.944492+0000","last_fullsized":"2024-04-25T09:01:36.944492+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:29.756119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.943444+0000","last_change":"2024-04-25T09:01:34.561292+0000","last_active":"2024-04-25T09:01:36.943444+0000","last_peered":"2024-04-25T09:01:36.943444+0000","last_clean":"2024-04-25T09:01:36.943444+0000","last_became_active":"2024-04-25T09:01:34.560996+0000","last_became_peered":"2024-04-25T09:01:34.560996+0000","last_unstale":"2024-04-25T09:01:36.943444+0000","last_undegraded":"2024-04-25T09:01:36.943444+0000","last_fullsized":"2024-04-25T09:01:36.943444+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:55.943449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580205+0000","last_change":"2024-04-25T09:01:35.608324+0000","last_active":"2024-04-25T09:01:36.580205+0000","last_peered":"2024-04-25T09:01:36.580205+0000","last_clean":"2024-04-25T09:01:36.580205+0000","last_became_active":"2024-04-25T09:01:35.607963+0000","last_became_peered":"2024-04-25T09:01:35.607963+0000","last_unstale":"2024-04-25T09:01:36.580205+0000","last_undegraded":"2024-04-25T09:01:36.580205+0000","last_fullsized":"2024-04-25T09:01:36.580205+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:54.729004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583591+0000","last_change":"2024-04-25T09:01:34.571791+0000","last_active":"2024-04-25T09:01:36.583591+0000","last_peered":"2024-04-25T09:01:36.583591+0000","last_clean":"2024-04-25T09:01:36.583591+0000","last_became_active":"2024-04-25T09:01:34.564950+0000","last_became_peered":"2024-04-25T09:01:34.564950+0000","last_unstale":"2024-04-25T09:01:36.583591+0000","last_undegraded":"2024-04-25T09:01:36.583591+0000","last_fullsized":"2024-04-25T09:01:36.583591+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:47.800143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944582+0000","last_change":"2024-04-25T09:01:35.600817+0000","last_active":"2024-04-25T09:01:36.944582+0000","last_peered":"2024-04-25T09:01:36.944582+0000","last_clean":"2024-04-25T09:01:36.944582+0000","last_became_active":"2024-04-25T09:01:35.600548+0000","last_became_peered":"2024-04-25T09:01:35.600548+0000","last_unstale":"2024-04-25T09:01:36.944582+0000","last_undegraded":"2024-04-25T09:01:36.944582+0000","last_fullsized":"2024-04-25T09:01:36.944582+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:35.354671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580161+0000","last_change":"2024-04-25T09:01:34.568280+0000","last_active":"2024-04-25T09:01:36.580161+0000","last_peered":"2024-04-25T09:01:36.580161+0000","last_clean":"2024-04-25T09:01:36.580161+0000","last_became_active":"2024-04-25T09:01:34.567918+0000","last_became_peered":"2024-04-25T09:01:34.567918+0000","last_unstale":"2024-04-25T09:01:36.580161+0000","last_undegraded":"2024-04-25T09:01:36.580161+0000","last_fullsized":"2024-04-25T09:01:36.580161+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:17.580384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575499+0000","last_change":"2024-04-25T09:01:35.592084+0000","last_active":"2024-04-25T09:01:36.575499+0000","last_peered":"2024-04-25T09:01:36.575499+0000","last_clean":"2024-04-25T09:01:36.575499+0000","last_became_active":"2024-04-25T09:01:35.591787+0000","last_became_peered":"2024-04-25T09:01:35.591787+0000","last_unstale":"2024-04-25T09:01:36.575499+0000","last_undegraded":"2024-04-25T09:01:36.575499+0000","last_fullsized":"2024-04-25T09:01:36.575499+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:56.964888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"109'3","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946271+0000","last_change":"2024-04-25T09:01:34.578215+0000","last_active":"2024-04-25T09:01:36.946271+0000","last_peered":"2024-04-25T09:01:36.946271+0000","last_clean":"2024-04-25T09:01:36.946271+0000","last_became_active":"2024-04-25T09:01:34.577464+0000","last_became_peered":"2024-04-25T09:01:34.577464+0000","last_unstale":"2024-04-25T09:01:36.946271+0000","last_undegraded":"2024-04-25T09:01:36.946271+0000","last_fullsized":"2024-04-25T09:01:36.946271+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:32.786071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945682+0000","last_change":"2024-04-25T09:01:35.600419+0000","last_active":"2024-04-25T09:01:36.945682+0000","last_peered":"2024-04-25T09:01:36.945682+0000","last_clean":"2024-04-25T09:01:36.945682+0000","last_became_active":"2024-04-25T09:01:35.600121+0000","last_became_peered":"2024-04-25T09:01:35.600121+0000","last_unstale":"2024-04-25T09:01:36.945682+0000","last_undegraded":"2024-04-25T09:01:36.945682+0000","last_fullsized":"2024-04-25T09:01:36.945682+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:22.247172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"109'6","reported_seq":20,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582927+0000","last_change":"2024-04-25T09:01:34.563869+0000","last_active":"2024-04-25T09:01:36.582927+0000","last_peered":"2024-04-25T09:01:36.582927+0000","last_clean":"2024-04-25T09:01:36.582927+0000","last_became_active":"2024-04-25T09:01:34.562417+0000","last_became_peered":"2024-04-25T09:01:34.562417+0000","last_unstale":"2024-04-25T09:01:36.582927+0000","last_undegraded":"2024-04-25T09:01:36.582927+0000","last_fullsized":"2024-04-25T09:01:36.582927+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:14.312491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.587947+0000","last_change":"2024-04-25T09:01:35.624505+0000","last_active":"2024-04-25T09:01:36.587947+0000","last_peered":"2024-04-25T09:01:36.587947+0000","last_clean":"2024-04-25T09:01:36.587947+0000","last_became_active":"2024-04-25T09:01:35.614092+0000","last_became_peered":"2024-04-25T09:01:35.614092+0000","last_unstale":"2024-04-25T09:01:36.587947+0000","last_undegraded":"2024-04-25T09:01:36.587947+0000","last_fullsized":"2024-04-25T09:01:36.587947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:48.812232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945662+0000","last_change":"2024-04-25T09:01:34.575412+0000","last_active":"2024-04-25T09:01:36.945662+0000","last_peered":"2024-04-25T09:01:36.945662+0000","last_clean":"2024-04-25T09:01:36.945662+0000","last_became_active":"2024-04-25T09:01:34.574972+0000","last_became_peered":"2024-04-25T09:01:34.574972+0000","last_unstale":"2024-04-25T09:01:36.945662+0000","last_undegraded":"2024-04-25T09:01:36.945662+0000","last_fullsized":"2024-04-25T09:01:36.945662+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:25.909339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582509+0000","last_change":"2024-04-25T09:01:35.608312+0000","last_active":"2024-04-25T09:01:36.582509+0000","last_peered":"2024-04-25T09:01:36.582509+0000","last_clean":"2024-04-25T09:01:36.582509+0000","last_became_active":"2024-04-25T09:01:35.607545+0000","last_became_peered":"2024-04-25T09:01:35.607545+0000","last_unstale":"2024-04-25T09:01:36.582509+0000","last_undegraded":"2024-04-25T09:01:36.582509+0000","last_fullsized":"2024-04-25T09:01:36.582509+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:56.955538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944044+0000","last_change":"2024-04-25T09:01:34.577629+0000","last_active":"2024-04-25T09:01:36.944044+0000","last_peered":"2024-04-25T09:01:36.944044+0000","last_clean":"2024-04-25T09:01:36.944044+0000","last_became_active":"2024-04-25T09:01:34.577195+0000","last_became_peered":"2024-04-25T09:01:34.577195+0000","last_unstale":"2024-04-25T09:01:36.944044+0000","last_undegraded":"2024-04-25T09:01:36.944044+0000","last_fullsized":"2024-04-25T09:01:36.944044+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:14.566999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575530+0000","last_change":"2024-04-25T09:01:35.582611+0000","last_active":"2024-04-25T09:01:36.575530+0000","last_peered":"2024-04-25T09:01:36.575530+0000","last_clean":"2024-04-25T09:01:36.575530+0000","last_became_active":"2024-04-25T09:01:35.582255+0000","last_became_peered":"2024-04-25T09:01:35.582255+0000","last_unstale":"2024-04-25T09:01:36.575530+0000","last_undegraded":"2024-04-25T09:01:36.575530+0000","last_fullsized":"2024-04-25T09:01:36.575530+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:54.343143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575449+0000","last_change":"2024-04-25T09:01:34.564481+0000","last_active":"2024-04-25T09:01:36.575449+0000","last_peered":"2024-04-25T09:01:36.575449+0000","last_clean":"2024-04-25T09:01:36.575449+0000","last_became_active":"2024-04-25T09:01:34.564183+0000","last_became_peered":"2024-04-25T09:01:34.564183+0000","last_unstale":"2024-04-25T09:01:36.575449+0000","last_undegraded":"2024-04-25T09:01:36.575449+0000","last_fullsized":"2024-04-25T09:01:36.575449+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:51.319222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581467+0000","last_change":"2024-04-25T09:01:35.635354+0000","last_active":"2024-04-25T09:01:36.581467+0000","last_peered":"2024-04-25T09:01:36.581467+0000","last_clean":"2024-04-25T09:01:36.581467+0000","last_became_active":"2024-04-25T09:01:35.635038+0000","last_became_peered":"2024-04-25T09:01:35.635038+0000","last_unstale":"2024-04-25T09:01:36.581467+0000","last_undegraded":"2024-04-25T09:01:36.581467+0000","last_fullsized":"2024-04-25T09:01:36.581467+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:31.842568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"109'8","reported_seq":22,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580668+0000","last_change":"2024-04-25T09:01:34.562989+0000","last_active":"2024-04-25T09:01:36.580668+0000","last_peered":"2024-04-25T09:01:36.580668+0000","last_clean":"2024-04-25T09:01:36.580668+0000","last_became_active":"2024-04-25T09:01:34.562243+0000","last_became_peered":"2024-04-25T09:01:34.562243+0000","last_unstale":"2024-04-25T09:01:36.580668+0000","last_undegraded":"2024-04-25T09:01:36.580668+0000","last_fullsized":"2024-04-25T09:01:36.580668+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:47.904471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575011+0000","last_change":"2024-04-25T09:01:35.592933+0000","last_active":"2024-04-25T09:01:36.575011+0000","last_peered":"2024-04-25T09:01:36.575011+0000","last_clean":"2024-04-25T09:01:36.575011+0000","last_became_active":"2024-04-25T09:01:35.592653+0000","last_became_peered":"2024-04-25T09:01:35.592653+0000","last_unstale":"2024-04-25T09:01:36.575011+0000","last_undegraded":"2024-04-25T09:01:36.575011+0000","last_fullsized":"2024-04-25T09:01:36.575011+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:46.340931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582237+0000","last_change":"2024-04-25T09:01:34.565923+0000","last_active":"2024-04-25T09:01:36.582237+0000","last_peered":"2024-04-25T09:01:36.582237+0000","last_clean":"2024-04-25T09:01:36.582237+0000","last_became_active":"2024-04-25T09:01:34.565063+0000","last_became_peered":"2024-04-25T09:01:34.565063+0000","last_unstale":"2024-04-25T09:01:36.582237+0000","last_undegraded":"2024-04-25T09:01:36.582237+0000","last_fullsized":"2024-04-25T09:01:36.582237+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:53.717265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582041+0000","last_change":"2024-04-25T09:01:35.622490+0000","last_active":"2024-04-25T09:01:36.582041+0000","last_peered":"2024-04-25T09:01:36.582041+0000","last_clean":"2024-04-25T09:01:36.582041+0000","last_became_active":"2024-04-25T09:01:35.620731+0000","last_became_peered":"2024-04-25T09:01:35.620731+0000","last_unstale":"2024-04-25T09:01:36.582041+0000","last_undegraded":"2024-04-25T09:01:36.582041+0000","last_fullsized":"2024-04-25T09:01:36.582041+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:57.793012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946385+0000","last_change":"2024-04-25T09:01:34.585991+0000","last_active":"2024-04-25T09:01:36.946385+0000","last_peered":"2024-04-25T09:01:36.946385+0000","last_clean":"2024-04-25T09:01:36.946385+0000","last_became_active":"2024-04-25T09:01:34.585265+0000","last_became_peered":"2024-04-25T09:01:34.585265+0000","last_unstale":"2024-04-25T09:01:36.946385+0000","last_undegraded":"2024-04-25T09:01:36.946385+0000","last_fullsized":"2024-04-25T09:01:36.946385+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:18.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945370+0000","last_change":"2024-04-25T09:01:35.610613+0000","last_active":"2024-04-25T09:01:36.945370+0000","last_peered":"2024-04-25T09:01:36.945370+0000","last_clean":"2024-04-25T09:01:36.945370+0000","last_became_active":"2024-04-25T09:01:35.610293+0000","last_became_peered":"2024-04-25T09:01:35.610293+0000","last_unstale":"2024-04-25T09:01:36.945370+0000","last_undegraded":"2024-04-25T09:01:36.945370+0000","last_fullsized":"2024-04-25T09:01:36.945370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:57.793012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.587832+0000","last_change":"2024-04-25T09:01:34.571639+0000","last_active":"2024-04-25T09:01:36.587832+0000","last_peered":"2024-04-25T09:01:36.587832+0000","last_clean":"2024-04-25T09:01:36.587832+0000","last_became_active":"2024-04-25T09:01:34.570919+0000","last_became_peered":"2024-04-25T09:01:34.570919+0000","last_unstale":"2024-04-25T09:01:36.587832+0000","last_undegraded":"2024-04-25T09:01:36.587832+0000","last_fullsized":"2024-04-25T09:01:36.587832+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:14.312491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580619+0000","last_change":"2024-04-25T09:01:35.597119+0000","last_active":"2024-04-25T09:01:36.580619+0000","last_peered":"2024-04-25T09:01:36.580619+0000","last_clean":"2024-04-25T09:01:36.580619+0000","last_became_active":"2024-04-25T09:01:35.596843+0000","last_became_peered":"2024-04-25T09:01:35.596843+0000","last_unstale":"2024-04-25T09:01:36.580619+0000","last_undegraded":"2024-04-25T09:01:36.580619+0000","last_fullsized":"2024-04-25T09:01:36.580619+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:18.951739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.576554+0000","last_change":"2024-04-25T09:01:34.563183+0000","last_active":"2024-04-25T09:01:36.576554+0000","last_peered":"2024-04-25T09:01:36.576554+0000","last_clean":"2024-04-25T09:01:36.576554+0000","last_became_active":"2024-04-25T09:01:34.552350+0000","last_became_peered":"2024-04-25T09:01:34.552350+0000","last_unstale":"2024-04-25T09:01:36.576554+0000","last_undegraded":"2024-04-25T09:01:36.576554+0000","last_fullsized":"2024-04-25T09:01:36.576554+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:57.740142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581342+0000","last_change":"2024-04-25T09:01:35.593304+0000","last_active":"2024-04-25T09:01:36.581342+0000","last_peered":"2024-04-25T09:01:36.581342+0000","last_clean":"2024-04-25T09:01:36.581342+0000","last_became_active":"2024-04-25T09:01:35.589329+0000","last_became_peered":"2024-04-25T09:01:35.589329+0000","last_unstale":"2024-04-25T09:01:36.581342+0000","last_undegraded":"2024-04-25T09:01:36.581342+0000","last_fullsized":"2024-04-25T09:01:36.581342+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:10.798939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.574958+0000","last_change":"2024-04-25T09:01:34.572962+0000","last_active":"2024-04-25T09:01:36.574958+0000","last_peered":"2024-04-25T09:01:36.574958+0000","last_clean":"2024-04-25T09:01:36.574958+0000","last_became_active":"2024-04-25T09:01:34.572395+0000","last_became_peered":"2024-04-25T09:01:34.572395+0000","last_unstale":"2024-04-25T09:01:36.574958+0000","last_undegraded":"2024-04-25T09:01:36.574958+0000","last_fullsized":"2024-04-25T09:01:36.574958+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.394912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944393+0000","last_change":"2024-04-25T09:01:35.621012+0000","last_active":"2024-04-25T09:01:36.944393+0000","last_peered":"2024-04-25T09:01:36.944393+0000","last_clean":"2024-04-25T09:01:36.944393+0000","last_became_active":"2024-04-25T09:01:35.620775+0000","last_became_peered":"2024-04-25T09:01:35.620775+0000","last_unstale":"2024-04-25T09:01:36.944393+0000","last_undegraded":"2024-04-25T09:01:36.944393+0000","last_fullsized":"2024-04-25T09:01:36.944393+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:46.183788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581273+0000","last_change":"2024-04-25T09:01:34.563744+0000","last_active":"2024-04-25T09:01:36.581273+0000","last_peered":"2024-04-25T09:01:36.581273+0000","last_clean":"2024-04-25T09:01:36.581273+0000","last_became_active":"2024-04-25T09:01:34.555693+0000","last_became_peered":"2024-04-25T09:01:34.555693+0000","last_unstale":"2024-04-25T09:01:36.581273+0000","last_undegraded":"2024-04-25T09:01:36.581273+0000","last_fullsized":"2024-04-25T09:01:36.581273+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:12.841471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.576135+0000","last_change":"2024-04-25T09:01:35.590817+0000","last_active":"2024-04-25T09:01:36.576135+0000","last_peered":"2024-04-25T09:01:36.576135+0000","last_clean":"2024-04-25T09:01:36.576135+0000","last_became_active":"2024-04-25T09:01:35.590502+0000","last_became_peered":"2024-04-25T09:01:35.590502+0000","last_unstale":"2024-04-25T09:01:36.576135+0000","last_undegraded":"2024-04-25T09:01:36.576135+0000","last_fullsized":"2024-04-25T09:01:36.576135+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:33.689092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575422+0000","last_change":"2024-04-25T09:01:34.572454+0000","last_active":"2024-04-25T09:01:36.575422+0000","last_peered":"2024-04-25T09:01:36.575422+0000","last_clean":"2024-04-25T09:01:36.575422+0000","last_became_active":"2024-04-25T09:01:34.571813+0000","last_became_peered":"2024-04-25T09:01:34.571813+0000","last_unstale":"2024-04-25T09:01:36.575422+0000","last_undegraded":"2024-04-25T09:01:36.575422+0000","last_fullsized":"2024-04-25T09:01:36.575422+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.788729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580168+0000","last_change":"2024-04-25T09:01:35.597530+0000","last_active":"2024-04-25T09:01:36.580168+0000","last_peered":"2024-04-25T09:01:36.580168+0000","last_clean":"2024-04-25T09:01:36.580168+0000","last_became_active":"2024-04-25T09:01:35.597127+0000","last_became_peered":"2024-04-25T09:01:35.597127+0000","last_unstale":"2024-04-25T09:01:36.580168+0000","last_undegraded":"2024-04-25T09:01:36.580168+0000","last_fullsized":"2024-04-25T09:01:36.580168+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:48.092531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945816+0000","last_change":"2024-04-25T09:01:34.575210+0000","last_active":"2024-04-25T09:01:36.945816+0000","last_peered":"2024-04-25T09:01:36.945816+0000","last_clean":"2024-04-25T09:01:36.945816+0000","last_became_active":"2024-04-25T09:01:34.574914+0000","last_became_peered":"2024-04-25T09:01:34.574914+0000","last_unstale":"2024-04-25T09:01:36.945816+0000","last_undegraded":"2024-04-25T09:01:36.945816+0000","last_fullsized":"2024-04-25T09:01:36.945816+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:53.717265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581014+0000","last_change":"2024-04-25T09:01:35.600699+0000","last_active":"2024-04-25T09:01:36.581014+0000","last_peered":"2024-04-25T09:01:36.581014+0000","last_clean":"2024-04-25T09:01:36.581014+0000","last_became_active":"2024-04-25T09:01:35.599981+0000","last_became_peered":"2024-04-25T09:01:35.599981+0000","last_unstale":"2024-04-25T09:01:36.581014+0000","last_undegraded":"2024-04-25T09:01:36.581014+0000","last_fullsized":"2024-04-25T09:01:36.581014+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:32.485434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"110'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583939+0000","last_change":"2024-04-25T09:01:34.570714+0000","last_active":"2024-04-25T09:01:36.583939+0000","last_peered":"2024-04-25T09:01:36.583939+0000","last_clean":"2024-04-25T09:01:36.583939+0000","last_became_active":"2024-04-25T09:01:34.570358+0000","last_became_peered":"2024-04-25T09:01:34.570358+0000","last_unstale":"2024-04-25T09:01:36.583939+0000","last_undegraded":"2024-04-25T09:01:36.583939+0000","last_fullsized":"2024-04-25T09:01:36.583939+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:31.032184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944335+0000","last_change":"2024-04-25T09:01:35.607939+0000","last_active":"2024-04-25T09:01:36.944335+0000","last_peered":"2024-04-25T09:01:36.944335+0000","last_clean":"2024-04-25T09:01:36.944335+0000","last_became_active":"2024-04-25T09:01:35.607591+0000","last_became_peered":"2024-04-25T09:01:35.607591+0000","last_unstale":"2024-04-25T09:01:36.944335+0000","last_undegraded":"2024-04-25T09:01:36.944335+0000","last_fullsized":"2024-04-25T09:01:36.944335+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:55.983973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582565+0000","last_change":"2024-04-25T09:01:34.570021+0000","last_active":"2024-04-25T09:01:36.582565+0000","last_peered":"2024-04-25T09:01:36.582565+0000","last_clean":"2024-04-25T09:01:36.582565+0000","last_became_active":"2024-04-25T09:01:34.566318+0000","last_became_peered":"2024-04-25T09:01:34.566318+0000","last_unstale":"2024-04-25T09:01:36.582565+0000","last_undegraded":"2024-04-25T09:01:36.582565+0000","last_fullsized":"2024-04-25T09:01:36.582565+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:07.276093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.588143+0000","last_change":"2024-04-25T09:01:35.624833+0000","last_active":"2024-04-25T09:01:36.588143+0000","last_peered":"2024-04-25T09:01:36.588143+0000","last_clean":"2024-04-25T09:01:36.588143+0000","last_became_active":"2024-04-25T09:01:35.614154+0000","last_became_peered":"2024-04-25T09:01:35.614154+0000","last_unstale":"2024-04-25T09:01:36.588143+0000","last_undegraded":"2024-04-25T09:01:36.588143+0000","last_fullsized":"2024-04-25T09:01:36.588143+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:47.743189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945605+0000","last_change":"2024-04-25T09:01:34.578871+0000","last_active":"2024-04-25T09:01:36.945605+0000","last_peered":"2024-04-25T09:01:36.945605+0000","last_clean":"2024-04-25T09:01:36.945605+0000","last_became_active":"2024-04-25T09:01:34.578150+0000","last_became_peered":"2024-04-25T09:01:34.578150+0000","last_unstale":"2024-04-25T09:01:36.945605+0000","last_undegraded":"2024-04-25T09:01:36.945605+0000","last_fullsized":"2024-04-25T09:01:36.945605+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:07.276093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575465+0000","last_change":"2024-04-25T09:01:35.592748+0000","last_active":"2024-04-25T09:01:36.575465+0000","last_peered":"2024-04-25T09:01:36.575465+0000","last_clean":"2024-04-25T09:01:36.575465+0000","last_became_active":"2024-04-25T09:01:35.592409+0000","last_became_peered":"2024-04-25T09:01:35.592409+0000","last_unstale":"2024-04-25T09:01:36.575465+0000","last_undegraded":"2024-04-25T09:01:36.575465+0000","last_fullsized":"2024-04-25T09:01:36.575465+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:20.380662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583025+0000","last_change":"2024-04-25T09:01:34.582493+0000","last_active":"2024-04-25T09:01:36.583025+0000","last_peered":"2024-04-25T09:01:36.583025+0000","last_clean":"2024-04-25T09:01:36.583025+0000","last_became_active":"2024-04-25T09:01:34.581783+0000","last_became_peered":"2024-04-25T09:01:34.581783+0000","last_unstale":"2024-04-25T09:01:36.583025+0000","last_undegraded":"2024-04-25T09:01:36.583025+0000","last_fullsized":"2024-04-25T09:01:36.583025+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:32.786071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.576024+0000","last_change":"2024-04-25T09:01:35.590890+0000","last_active":"2024-04-25T09:01:36.576024+0000","last_peered":"2024-04-25T09:01:36.576024+0000","last_clean":"2024-04-25T09:01:36.576024+0000","last_became_active":"2024-04-25T09:01:35.590581+0000","last_became_peered":"2024-04-25T09:01:35.590581+0000","last_unstale":"2024-04-25T09:01:36.576024+0000","last_undegraded":"2024-04-25T09:01:36.576024+0000","last_fullsized":"2024-04-25T09:01:36.576024+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:49.213953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575219+0000","last_change":"2024-04-25T09:01:34.564697+0000","last_active":"2024-04-25T09:01:36.575219+0000","last_peered":"2024-04-25T09:01:36.575219+0000","last_clean":"2024-04-25T09:01:36.575219+0000","last_became_active":"2024-04-25T09:01:34.563926+0000","last_became_peered":"2024-04-25T09:01:34.563926+0000","last_unstale":"2024-04-25T09:01:36.575219+0000","last_undegraded":"2024-04-25T09:01:36.575219+0000","last_fullsized":"2024-04-25T09:01:36.575219+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:12.329463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946741+0000","last_change":"2024-04-25T09:01:35.640140+0000","last_active":"2024-04-25T09:01:36.946741+0000","last_peered":"2024-04-25T09:01:36.946741+0000","last_clean":"2024-04-25T09:01:36.946741+0000","last_became_active":"2024-04-25T09:01:35.639625+0000","last_became_peered":"2024-04-25T09:01:35.639625+0000","last_unstale":"2024-04-25T09:01:36.946741+0000","last_undegraded":"2024-04-25T09:01:36.946741+0000","last_fullsized":"2024-04-25T09:01:36.946741+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:46.480732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581564+0000","last_change":"2024-04-25T09:01:34.568304+0000","last_active":"2024-04-25T09:01:36.581564+0000","last_peered":"2024-04-25T09:01:36.581564+0000","last_clean":"2024-04-25T09:01:36.581564+0000","last_became_active":"2024-04-25T09:01:34.566058+0000","last_became_peered":"2024-04-25T09:01:34.566058+0000","last_unstale":"2024-04-25T09:01:36.581564+0000","last_undegraded":"2024-04-25T09:01:36.581564+0000","last_fullsized":"2024-04-25T09:01:36.581564+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:58.658639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.588662+0000","last_change":"2024-04-25T09:01:35.635598+0000","last_active":"2024-04-25T09:01:36.588662+0000","last_peered":"2024-04-25T09:01:36.588662+0000","last_clean":"2024-04-25T09:01:36.588662+0000","last_became_active":"2024-04-25T09:01:35.631405+0000","last_became_peered":"2024-04-25T09:01:35.631405+0000","last_unstale":"2024-04-25T09:01:36.588662+0000","last_undegraded":"2024-04-25T09:01:36.588662+0000","last_fullsized":"2024-04-25T09:01:36.588662+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:07.559766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575126+0000","last_change":"2024-04-25T09:01:34.570629+0000","last_active":"2024-04-25T09:01:36.575126+0000","last_peered":"2024-04-25T09:01:36.575126+0000","last_clean":"2024-04-25T09:01:36.575126+0000","last_became_active":"2024-04-25T09:01:34.565444+0000","last_became_peered":"2024-04-25T09:01:34.565444+0000","last_unstale":"2024-04-25T09:01:36.575126+0000","last_undegraded":"2024-04-25T09:01:36.575126+0000","last_fullsized":"2024-04-25T09:01:36.575126+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:33.052275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.587994+0000","last_change":"2024-04-25T09:01:35.632674+0000","last_active":"2024-04-25T09:01:36.587994+0000","last_peered":"2024-04-25T09:01:36.587994+0000","last_clean":"2024-04-25T09:01:36.587994+0000","last_became_active":"2024-04-25T09:01:35.631162+0000","last_became_peered":"2024-04-25T09:01:35.631162+0000","last_unstale":"2024-04-25T09:01:36.587994+0000","last_undegraded":"2024-04-25T09:01:36.587994+0000","last_fullsized":"2024-04-25T09:01:36.587994+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:26.376393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575806+0000","last_change":"2024-04-25T09:01:34.570226+0000","last_active":"2024-04-25T09:01:36.575806+0000","last_peered":"2024-04-25T09:01:36.575806+0000","last_clean":"2024-04-25T09:01:36.575806+0000","last_became_active":"2024-04-25T09:01:34.565514+0000","last_became_peered":"2024-04-25T09:01:34.565514+0000","last_unstale":"2024-04-25T09:01:36.575806+0000","last_undegraded":"2024-04-25T09:01:36.575806+0000","last_fullsized":"2024-04-25T09:01:36.575806+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:22.889807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":339,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945736+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T09:01:36.945736+0000","last_peered":"2024-04-25T09:01:36.945736+0000","last_clean":"2024-04-25T09:01:36.945736+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T09:01:36.945736+0000","last_undegraded":"2024-04-25T09:01:36.945736+0000","last_fullsized":"2024-04-25T09:01:36.945736+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":1227263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1384448,"data_stored":638951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1363968,"data_stored":636735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1372160,"data_stored":636827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1368064,"data_stored":637317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1368064,"data_stored":636769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1368064,"data_stored":636769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1974272,"data_stored":1228805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:01:39.634 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:01:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.798+0000 7f566c2e1640 1 Processor -- start 2024-04-25T09:01:39.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.798+0000 7f566c2e1640 1 -- start start 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566c2e1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f566415b770 0x7f566406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566c2e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56640cf1c0 0x7f56640cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566c2e1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56640713f0 con 0x7f56640cc6d0 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566c2e1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56640716d0 con 0x7f566415b770 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566c2e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56640719b0 con 0x7f56640cf1c0 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566a07d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f566415b770 0x7f566406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566a87e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f56640cc6d0 0x7f56640cd680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40966/0 (socket says 172.21.15.37:40966) 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566a87e640 1 -- 172.21.15.37:0/3732615698 learned_addr learned my addr 172.21.15.37:0/3732615698 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566987c640 1 --2- 172.21.15.37:0/3732615698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56640cf1c0 0x7f56640cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566907b640 1 -- 172.21.15.37:0/3732615698 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803819565 0 0) 0x7f56640713f0 con 0x7f56640cc6d0 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566907b640 1 -- 172.21.15.37:0/3732615698 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f564c003530 con 0x7f56640cc6d0 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566a07d640 1 -- 172.21.15.37:0/3732615698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56640cf1c0 msgr2=0x7f56640cc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:39.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566a07d640 1 --2- 172.21.15.37:0/3732615698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56640cf1c0 0x7f56640cc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566a07d640 1 -- 172.21.15.37:0/3732615698 >> v1:172.21.15.37:6789/0 conn(0x7f56640cc6d0 legacy=0x7f56640cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.802+0000 7f566a07d640 1 -- 172.21.15.37:0/3732615698 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5664071c90 con 0x7f566415b770 2024-04-25T09:01:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566a07d640 1 --2- 172.21.15.37:0/3732615698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566415b770 0x7f566406d3b0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f5654009d90 tx=0x7f565401f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f7ad2b25617f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566907b640 1 -- 172.21.15.37:0/3732615698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56540b4070 con 0x7f566415b770 2024-04-25T09:01:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566907b640 1 -- 172.21.15.37:0/3732615698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56540af600 con 0x7f566415b770 2024-04-25T09:01:39.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566907b640 1 -- 172.21.15.37:0/3732615698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56540af9d0 con 0x7f566415b770 2024-04-25T09:01:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 -- 172.21.15.37:0/3732615698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566415b770 msgr2=0x7f566406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 --2- 172.21.15.37:0/3732615698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566415b770 0x7f566406d3b0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f5654009d90 tx=0x7f565401f150 comp rx=0 tx=0).stop 2024-04-25T09:01:39.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 -- 172.21.15.37:0/3732615698 shutdown_connections 2024-04-25T09:01:39.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 --2- 172.21.15.37:0/3732615698 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f56640cf1c0 0x7f56640cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 --2- 172.21.15.37:0/3732615698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f566415b770 0x7f566406d3b0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 -- 172.21.15.37:0/3732615698 >> 172.21.15.37:0/3732615698 conn(0x7f56640b14b0 msgr2=0x7f56640b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:39.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 -- 172.21.15.37:0/3732615698 shutdown_connections 2024-04-25T09:01:39.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 -- 172.21.15.37:0/3732615698 wait complete. 2024-04-25T09:01:39.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 Processor -- start 2024-04-25T09:01:39.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 -- start start 2024-04-25T09:01:39.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56640cc6d0 0x7f566413f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56640cf1c0 0x7f56641e1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f566415b770 0x7f56640b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566406da70 con 0x7f56640cc6d0 2024-04-25T09:01:39.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566406d8f0 con 0x7f56640cf1c0 2024-04-25T09:01:39.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566c2e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566415cb60 con 0x7f566415b770 2024-04-25T09:01:39.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566a07d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56640cc6d0 0x7f566413f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566a07d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56640cc6d0 0x7f566413f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51814/0 (socket says 172.21.15.37:51814) 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.806+0000 7f566a07d640 1 -- 172.21.15.37:0/3829927593 learned_addr learned my addr 172.21.15.37:0/3829927593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566987c640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56640cf1c0 0x7f56641e1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566a87e640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f566415b770 0x7f56640b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566a07d640 1 -- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f566415b770 msgr2=0x7f56640b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566a07d640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f566415b770 0x7f56640b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566a07d640 1 -- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56640cf1c0 msgr2=0x7f56641e1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566a07d640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56640cf1c0 0x7f56641e1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566a07d640 1 -- 172.21.15.37:0/3829927593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56641e2900 con 0x7f56640cc6d0 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566987c640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56640cf1c0 0x7f56641e1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566a07d640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56640cc6d0 0x7f566413f200 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f565401cd80 tx=0x7f565401bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f5662ffd640 1 -- 172.21.15.37:0/3829927593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56540b4040 con 0x7f56640cc6d0 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f5662ffd640 1 -- 172.21.15.37:0/3829927593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56540c1070 con 0x7f56640cc6d0 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566c2e1640 1 -- 172.21.15.37:0/3829927593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56641e2be0 con 0x7f56640cc6d0 2024-04-25T09:01:39.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f5662ffd640 1 -- 172.21.15.37:0/3829927593 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654009450 con 0x7f56640cc6d0 2024-04-25T09:01:39.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566c2e1640 1 -- 172.21.15.37:0/3829927593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56641e31a0 con 0x7f56640cc6d0 2024-04-25T09:01:39.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566c2e1640 1 -- 172.21.15.37:0/3829927593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f566404efa0 con 0x7f56640cc6d0 2024-04-25T09:01:39.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f5662ffd640 1 -- 172.21.15.37:0/3829927593 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f56540ba690 con 0x7f56640cc6d0 2024-04-25T09:01:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f5662ffd640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5650066f60 0x7f5650069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f5662ffd640 1 -- 172.21.15.37:0/3829927593 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5654123ad0 con 0x7f56640cc6d0 2024-04-25T09:01:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566987c640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5650066f60 0x7f5650069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.810+0000 7f566987c640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5650066f60 0x7f5650069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f566406f0e0 tx=0x7f5658008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:39.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.814+0000 7f5662ffd640 1 -- 172.21.15.37:0/3829927593 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5654018cb0 con 0x7f56640cc6d0 2024-04-25T09:01:39.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.966+0000 7f566c2e1640 1 -- 172.21.15.37:0/3829927593 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f56640cec50 con 0x7f5650066f60 2024-04-25T09:01:39.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.986+0000 7f5662ffd640 1 -- 172.21.15.37:0/3829927593 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252494 (secure 0 0 0) 0x7f56640cec50 con 0x7f5650066f60 2024-04-25T09:01:39.978 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:39.979 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T09:01:39.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 -- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5650066f60 msgr2=0x7f5650069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5650066f60 0x7f5650069420 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f566406f0e0 tx=0x7f5658008040 comp rx=0 tx=0).stop 2024-04-25T09:01:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 -- 172.21.15.37:0/3829927593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56640cc6d0 msgr2=0x7f566413f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56640cc6d0 0x7f566413f200 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f565401cd80 tx=0x7f565401bef0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 -- 172.21.15.37:0/3829927593 shutdown_connections 2024-04-25T09:01:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5650066f60 0x7f5650069420 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f566415b770 0x7f56640b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f56640cf1c0 0x7f56641e1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 --2- 172.21.15.37:0/3829927593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56640cc6d0 0x7f566413f200 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 -- 172.21.15.37:0/3829927593 >> 172.21.15.37:0/3829927593 conn(0x7f56640b14b0 msgr2=0x7f566406f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 -- 172.21.15.37:0/3829927593 shutdown_connections 2024-04-25T09:01:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:39.990+0000 7f566c2e1640 1 -- 172.21.15.37:0/3829927593 wait complete. 2024-04-25T09:01:39.996 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-25T09:01:39.316629+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261360,"kb_used_data":11880,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713360,"statfs":{"total":773094113280,"available":772826480640,"internally_reserved":0,"allocated":12165120,"data_stored":6279436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.784572"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581092+0000","last_change":"2024-04-25T09:01:35.598819+0000","last_active":"2024-04-25T09:01:36.581092+0000","last_peered":"2024-04-25T09:01:36.581092+0000","last_clean":"2024-04-25T09:01:36.581092+0000","last_became_active":"2024-04-25T09:01:35.598510+0000","last_became_peered":"2024-04-25T09:01:35.598510+0000","last_unstale":"2024-04-25T09:01:36.581092+0000","last_undegraded":"2024-04-25T09:01:36.581092+0000","last_fullsized":"2024-04-25T09:01:36.581092+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:30.609001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582503+0000","last_change":"2024-04-25T09:01:35.622024+0000","last_active":"2024-04-25T09:01:36.582503+0000","last_peered":"2024-04-25T09:01:36.582503+0000","last_clean":"2024-04-25T09:01:36.582503+0000","last_became_active":"2024-04-25T09:01:35.621240+0000","last_became_peered":"2024-04-25T09:01:35.621240+0000","last_unstale":"2024-04-25T09:01:36.582503+0000","last_undegraded":"2024-04-25T09:01:36.582503+0000","last_fullsized":"2024-04-25T09:01:36.582503+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:32.485434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580948+0000","last_change":"2024-04-25T09:01:35.634574+0000","last_active":"2024-04-25T09:01:36.580948+0000","last_peered":"2024-04-25T09:01:36.580948+0000","last_clean":"2024-04-25T09:01:36.580948+0000","last_became_active":"2024-04-25T09:01:35.634294+0000","last_became_peered":"2024-04-25T09:01:35.634294+0000","last_unstale":"2024-04-25T09:01:36.580948+0000","last_undegraded":"2024-04-25T09:01:36.580948+0000","last_fullsized":"2024-04-25T09:01:36.580948+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:16.196961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.943906+0000","last_change":"2024-04-25T09:01:35.601636+0000","last_active":"2024-04-25T09:01:36.943906+0000","last_peered":"2024-04-25T09:01:36.943906+0000","last_clean":"2024-04-25T09:01:36.943906+0000","last_became_active":"2024-04-25T09:01:35.601313+0000","last_became_peered":"2024-04-25T09:01:35.601313+0000","last_unstale":"2024-04-25T09:01:36.943906+0000","last_undegraded":"2024-04-25T09:01:36.943906+0000","last_fullsized":"2024-04-25T09:01:36.943906+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:55.994484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945252+0000","last_change":"2024-04-25T09:01:35.602254+0000","last_active":"2024-04-25T09:01:36.945252+0000","last_peered":"2024-04-25T09:01:36.945252+0000","last_clean":"2024-04-25T09:01:36.945252+0000","last_became_active":"2024-04-25T09:01:35.601977+0000","last_became_peered":"2024-04-25T09:01:35.601977+0000","last_unstale":"2024-04-25T09:01:36.945252+0000","last_undegraded":"2024-04-25T09:01:36.945252+0000","last_fullsized":"2024-04-25T09:01:36.945252+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:07.559766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581624+0000","last_change":"2024-04-25T09:01:35.619211+0000","last_active":"2024-04-25T09:01:36.581624+0000","last_peered":"2024-04-25T09:01:36.581624+0000","last_clean":"2024-04-25T09:01:36.581624+0000","last_became_active":"2024-04-25T09:01:35.617848+0000","last_became_peered":"2024-04-25T09:01:35.617848+0000","last_unstale":"2024-04-25T09:01:36.581624+0000","last_undegraded":"2024-04-25T09:01:36.581624+0000","last_fullsized":"2024-04-25T09:01:36.581624+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:46.480732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580301+0000","last_change":"2024-04-25T09:01:35.617628+0000","last_active":"2024-04-25T09:01:36.580301+0000","last_peered":"2024-04-25T09:01:36.580301+0000","last_clean":"2024-04-25T09:01:36.580301+0000","last_became_active":"2024-04-25T09:01:35.616809+0000","last_became_peered":"2024-04-25T09:01:35.616809+0000","last_unstale":"2024-04-25T09:01:36.580301+0000","last_undegraded":"2024-04-25T09:01:36.580301+0000","last_fullsized":"2024-04-25T09:01:36.580301+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:35.354671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.574932+0000","last_change":"2024-04-25T09:01:35.592323+0000","last_active":"2024-04-25T09:01:36.574932+0000","last_peered":"2024-04-25T09:01:36.574932+0000","last_clean":"2024-04-25T09:01:36.574932+0000","last_became_active":"2024-04-25T09:01:35.591443+0000","last_became_peered":"2024-04-25T09:01:35.591443+0000","last_unstale":"2024-04-25T09:01:36.574932+0000","last_undegraded":"2024-04-25T09:01:36.574932+0000","last_fullsized":"2024-04-25T09:01:36.574932+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:33.353156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582183+0000","last_change":"2024-04-25T09:01:35.618923+0000","last_active":"2024-04-25T09:01:36.582183+0000","last_peered":"2024-04-25T09:01:36.582183+0000","last_clean":"2024-04-25T09:01:36.582183+0000","last_became_active":"2024-04-25T09:01:35.608809+0000","last_became_peered":"2024-04-25T09:01:35.608809+0000","last_unstale":"2024-04-25T09:01:36.582183+0000","last_undegraded":"2024-04-25T09:01:36.582183+0000","last_fullsized":"2024-04-25T09:01:36.582183+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:22.247172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945929+0000","last_change":"2024-04-25T09:01:35.620324+0000","last_active":"2024-04-25T09:01:36.945929+0000","last_peered":"2024-04-25T09:01:36.945929+0000","last_clean":"2024-04-25T09:01:36.945929+0000","last_became_active":"2024-04-25T09:01:35.620049+0000","last_became_peered":"2024-04-25T09:01:35.620049+0000","last_unstale":"2024-04-25T09:01:36.945929+0000","last_undegraded":"2024-04-25T09:01:36.945929+0000","last_fullsized":"2024-04-25T09:01:36.945929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:32.485434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.588589+0000","last_change":"2024-04-25T09:01:35.635963+0000","last_active":"2024-04-25T09:01:36.588589+0000","last_peered":"2024-04-25T09:01:36.588589+0000","last_clean":"2024-04-25T09:01:36.588589+0000","last_became_active":"2024-04-25T09:01:35.635417+0000","last_became_peered":"2024-04-25T09:01:35.635417+0000","last_unstale":"2024-04-25T09:01:36.588589+0000","last_undegraded":"2024-04-25T09:01:36.588589+0000","last_fullsized":"2024-04-25T09:01:36.588589+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:35.354671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583536+0000","last_change":"2024-04-25T09:01:35.600850+0000","last_active":"2024-04-25T09:01:36.583536+0000","last_peered":"2024-04-25T09:01:36.583536+0000","last_clean":"2024-04-25T09:01:36.583536+0000","last_became_active":"2024-04-25T09:01:35.600122+0000","last_became_peered":"2024-04-25T09:01:35.600122+0000","last_unstale":"2024-04-25T09:01:36.583536+0000","last_undegraded":"2024-04-25T09:01:36.583536+0000","last_fullsized":"2024-04-25T09:01:36.583536+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:22.161066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.943719+0000","last_change":"2024-04-25T09:01:35.634588+0000","last_active":"2024-04-25T09:01:36.943719+0000","last_peered":"2024-04-25T09:01:36.943719+0000","last_clean":"2024-04-25T09:01:36.943719+0000","last_became_active":"2024-04-25T09:01:35.634278+0000","last_became_peered":"2024-04-25T09:01:35.634278+0000","last_unstale":"2024-04-25T09:01:36.943719+0000","last_undegraded":"2024-04-25T09:01:36.943719+0000","last_fullsized":"2024-04-25T09:01:36.943719+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:05.385775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946078+0000","last_change":"2024-04-25T09:01:35.612780+0000","last_active":"2024-04-25T09:01:36.946078+0000","last_peered":"2024-04-25T09:01:36.946078+0000","last_clean":"2024-04-25T09:01:36.946078+0000","last_became_active":"2024-04-25T09:01:35.611972+0000","last_became_peered":"2024-04-25T09:01:35.611972+0000","last_unstale":"2024-04-25T09:01:36.946078+0000","last_undegraded":"2024-04-25T09:01:36.946078+0000","last_fullsized":"2024-04-25T09:01:36.946078+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:35.354671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946324+0000","last_change":"2024-04-25T09:01:35.639776+0000","last_active":"2024-04-25T09:01:36.946324+0000","last_peered":"2024-04-25T09:01:36.946324+0000","last_clean":"2024-04-25T09:01:36.946324+0000","last_became_active":"2024-04-25T09:01:35.639301+0000","last_became_peered":"2024-04-25T09:01:35.639301+0000","last_unstale":"2024-04-25T09:01:36.946324+0000","last_undegraded":"2024-04-25T09:01:36.946324+0000","last_fullsized":"2024-04-25T09:01:36.946324+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:07.559766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946004+0000","last_change":"2024-04-25T09:01:35.611878+0000","last_active":"2024-04-25T09:01:36.946004+0000","last_peered":"2024-04-25T09:01:36.946004+0000","last_clean":"2024-04-25T09:01:36.946004+0000","last_became_active":"2024-04-25T09:01:35.611228+0000","last_became_peered":"2024-04-25T09:01:35.611228+0000","last_unstale":"2024-04-25T09:01:36.946004+0000","last_undegraded":"2024-04-25T09:01:36.946004+0000","last_fullsized":"2024-04-25T09:01:36.946004+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:00.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581246+0000","last_change":"2024-04-25T09:01:35.629412+0000","last_active":"2024-04-25T09:01:36.581246+0000","last_peered":"2024-04-25T09:01:36.581246+0000","last_clean":"2024-04-25T09:01:36.581246+0000","last_became_active":"2024-04-25T09:01:35.629080+0000","last_became_peered":"2024-04-25T09:01:35.629080+0000","last_unstale":"2024-04-25T09:01:36.581246+0000","last_undegraded":"2024-04-25T09:01:36.581246+0000","last_fullsized":"2024-04-25T09:01:36.581246+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:11.426534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582537+0000","last_change":"2024-04-25T09:01:35.616209+0000","last_active":"2024-04-25T09:01:36.582537+0000","last_peered":"2024-04-25T09:01:36.582537+0000","last_clean":"2024-04-25T09:01:36.582537+0000","last_became_active":"2024-04-25T09:01:35.609544+0000","last_became_peered":"2024-04-25T09:01:35.609544+0000","last_unstale":"2024-04-25T09:01:36.582537+0000","last_undegraded":"2024-04-25T09:01:36.582537+0000","last_fullsized":"2024-04-25T09:01:36.582537+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:15.579088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581055+0000","last_change":"2024-04-25T09:01:35.619101+0000","last_active":"2024-04-25T09:01:36.581055+0000","last_peered":"2024-04-25T09:01:36.581055+0000","last_clean":"2024-04-25T09:01:36.581055+0000","last_became_active":"2024-04-25T09:01:35.618851+0000","last_became_peered":"2024-04-25T09:01:35.618851+0000","last_unstale":"2024-04-25T09:01:36.581055+0000","last_undegraded":"2024-04-25T09:01:36.581055+0000","last_fullsized":"2024-04-25T09:01:36.581055+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:29.138706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583715+0000","last_change":"2024-04-25T09:01:35.615968+0000","last_active":"2024-04-25T09:01:36.583715+0000","last_peered":"2024-04-25T09:01:36.583715+0000","last_clean":"2024-04-25T09:01:36.583715+0000","last_became_active":"2024-04-25T09:01:35.614878+0000","last_became_peered":"2024-04-25T09:01:35.614878+0000","last_unstale":"2024-04-25T09:01:36.583715+0000","last_undegraded":"2024-04-25T09:01:36.583715+0000","last_fullsized":"2024-04-25T09:01:36.583715+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:26.715787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575852+0000","last_change":"2024-04-25T09:01:35.628633+0000","last_active":"2024-04-25T09:01:36.575852+0000","last_peered":"2024-04-25T09:01:36.575852+0000","last_clean":"2024-04-25T09:01:36.575852+0000","last_became_active":"2024-04-25T09:01:35.628302+0000","last_became_peered":"2024-04-25T09:01:35.628302+0000","last_unstale":"2024-04-25T09:01:36.575852+0000","last_undegraded":"2024-04-25T09:01:36.575852+0000","last_fullsized":"2024-04-25T09:01:36.575852+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:20.531268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945303+0000","last_change":"2024-04-25T09:01:35.634747+0000","last_active":"2024-04-25T09:01:36.945303+0000","last_peered":"2024-04-25T09:01:36.945303+0000","last_clean":"2024-04-25T09:01:36.945303+0000","last_became_active":"2024-04-25T09:01:35.634264+0000","last_became_peered":"2024-04-25T09:01:35.634264+0000","last_unstale":"2024-04-25T09:01:36.945303+0000","last_undegraded":"2024-04-25T09:01:36.945303+0000","last_fullsized":"2024-04-25T09:01:36.945303+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:46.480732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580011+0000","last_change":"2024-04-25T09:01:35.629381+0000","last_active":"2024-04-25T09:01:36.580011+0000","last_peered":"2024-04-25T09:01:36.580011+0000","last_clean":"2024-04-25T09:01:36.580011+0000","last_became_active":"2024-04-25T09:01:35.629055+0000","last_became_peered":"2024-04-25T09:01:35.629055+0000","last_unstale":"2024-04-25T09:01:36.580011+0000","last_undegraded":"2024-04-25T09:01:36.580011+0000","last_fullsized":"2024-04-25T09:01:36.580011+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:11.367992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944550+0000","last_change":"2024-04-25T09:01:35.596313+0000","last_active":"2024-04-25T09:01:36.944550+0000","last_peered":"2024-04-25T09:01:36.944550+0000","last_clean":"2024-04-25T09:01:36.944550+0000","last_became_active":"2024-04-25T09:01:35.595868+0000","last_became_peered":"2024-04-25T09:01:35.595868+0000","last_unstale":"2024-04-25T09:01:36.944550+0000","last_undegraded":"2024-04-25T09:01:36.944550+0000","last_fullsized":"2024-04-25T09:01:36.944550+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:00.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583597+0000","last_change":"2024-04-25T09:01:35.637670+0000","last_active":"2024-04-25T09:01:36.583597+0000","last_peered":"2024-04-25T09:01:36.583597+0000","last_clean":"2024-04-25T09:01:36.583597+0000","last_became_active":"2024-04-25T09:01:35.637126+0000","last_became_peered":"2024-04-25T09:01:35.637126+0000","last_unstale":"2024-04-25T09:01:36.583597+0000","last_undegraded":"2024-04-25T09:01:36.583597+0000","last_fullsized":"2024-04-25T09:01:36.583597+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.998517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583836+0000","last_change":"2024-04-25T09:01:35.624323+0000","last_active":"2024-04-25T09:01:36.583836+0000","last_peered":"2024-04-25T09:01:36.583836+0000","last_clean":"2024-04-25T09:01:36.583836+0000","last_became_active":"2024-04-25T09:01:35.623726+0000","last_became_peered":"2024-04-25T09:01:35.623726+0000","last_unstale":"2024-04-25T09:01:36.583836+0000","last_undegraded":"2024-04-25T09:01:36.583836+0000","last_fullsized":"2024-04-25T09:01:36.583836+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:30.110950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.588811+0000","last_change":"2024-04-25T09:01:35.631687+0000","last_active":"2024-04-25T09:01:36.588811+0000","last_peered":"2024-04-25T09:01:36.588811+0000","last_clean":"2024-04-25T09:01:36.588811+0000","last_became_active":"2024-04-25T09:01:35.626024+0000","last_became_peered":"2024-04-25T09:01:35.626024+0000","last_unstale":"2024-04-25T09:01:36.588811+0000","last_undegraded":"2024-04-25T09:01:36.588811+0000","last_fullsized":"2024-04-25T09:01:36.588811+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:46.480732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582636+0000","last_change":"2024-04-25T09:01:35.628421+0000","last_active":"2024-04-25T09:01:36.582636+0000","last_peered":"2024-04-25T09:01:36.582636+0000","last_clean":"2024-04-25T09:01:36.582636+0000","last_became_active":"2024-04-25T09:01:35.627742+0000","last_became_peered":"2024-04-25T09:01:35.627742+0000","last_unstale":"2024-04-25T09:01:36.582636+0000","last_undegraded":"2024-04-25T09:01:36.582636+0000","last_fullsized":"2024-04-25T09:01:36.582636+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:22.161066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.576271+0000","last_change":"2024-04-25T09:01:35.610750+0000","last_active":"2024-04-25T09:01:36.576271+0000","last_peered":"2024-04-25T09:01:36.576271+0000","last_clean":"2024-04-25T09:01:36.576271+0000","last_became_active":"2024-04-25T09:01:35.610419+0000","last_became_peered":"2024-04-25T09:01:35.610419+0000","last_unstale":"2024-04-25T09:01:36.576271+0000","last_undegraded":"2024-04-25T09:01:36.576271+0000","last_fullsized":"2024-04-25T09:01:36.576271+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:27.858482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581203+0000","last_change":"2024-04-25T09:01:35.639629+0000","last_active":"2024-04-25T09:01:36.581203+0000","last_peered":"2024-04-25T09:01:36.581203+0000","last_clean":"2024-04-25T09:01:36.581203+0000","last_became_active":"2024-04-25T09:01:35.639167+0000","last_became_peered":"2024-04-25T09:01:35.639167+0000","last_unstale":"2024-04-25T09:01:36.581203+0000","last_undegraded":"2024-04-25T09:01:36.581203+0000","last_fullsized":"2024-04-25T09:01:36.581203+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:56.955538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580893+0000","last_change":"2024-04-25T09:01:35.622072+0000","last_active":"2024-04-25T09:01:36.580893+0000","last_peered":"2024-04-25T09:01:36.580893+0000","last_clean":"2024-04-25T09:01:36.580893+0000","last_became_active":"2024-04-25T09:01:35.621406+0000","last_became_peered":"2024-04-25T09:01:35.621406+0000","last_unstale":"2024-04-25T09:01:36.580893+0000","last_undegraded":"2024-04-25T09:01:36.580893+0000","last_fullsized":"2024-04-25T09:01:36.580893+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:07.559766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.574808+0000","last_change":"2024-04-25T09:01:35.581383+0000","last_active":"2024-04-25T09:01:36.574808+0000","last_peered":"2024-04-25T09:01:36.574808+0000","last_clean":"2024-04-25T09:01:36.574808+0000","last_became_active":"2024-04-25T09:01:35.581077+0000","last_became_peered":"2024-04-25T09:01:35.581077+0000","last_unstale":"2024-04-25T09:01:36.574808+0000","last_undegraded":"2024-04-25T09:01:36.574808+0000","last_fullsized":"2024-04-25T09:01:36.574808+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:41.124569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580389+0000","last_change":"2024-04-25T09:01:35.608337+0000","last_active":"2024-04-25T09:01:36.580389+0000","last_peered":"2024-04-25T09:01:36.580389+0000","last_clean":"2024-04-25T09:01:36.580389+0000","last_became_active":"2024-04-25T09:01:35.607939+0000","last_became_peered":"2024-04-25T09:01:35.607939+0000","last_unstale":"2024-04-25T09:01:36.580389+0000","last_undegraded":"2024-04-25T09:01:36.580389+0000","last_fullsized":"2024-04-25T09:01:36.580389+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:07.681259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"109'4","reported_seq":18,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944319+0000","last_change":"2024-04-25T09:01:34.579573+0000","last_active":"2024-04-25T09:01:36.944319+0000","last_peered":"2024-04-25T09:01:36.944319+0000","last_clean":"2024-04-25T09:01:36.944319+0000","last_became_active":"2024-04-25T09:01:34.579214+0000","last_became_peered":"2024-04-25T09:01:34.579214+0000","last_unstale":"2024-04-25T09:01:36.944319+0000","last_undegraded":"2024-04-25T09:01:36.944319+0000","last_fullsized":"2024-04-25T09:01:36.944319+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:21.148978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575154+0000","last_change":"2024-04-25T09:01:35.591851+0000","last_active":"2024-04-25T09:01:36.575154+0000","last_peered":"2024-04-25T09:01:36.575154+0000","last_clean":"2024-04-25T09:01:36.575154+0000","last_became_active":"2024-04-25T09:01:35.591660+0000","last_became_peered":"2024-04-25T09:01:35.591660+0000","last_unstale":"2024-04-25T09:01:36.575154+0000","last_undegraded":"2024-04-25T09:01:36.575154+0000","last_fullsized":"2024-04-25T09:01:36.575154+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:43.504824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575628+0000","last_change":"2024-04-25T09:01:34.579195+0000","last_active":"2024-04-25T09:01:36.575628+0000","last_peered":"2024-04-25T09:01:36.575628+0000","last_clean":"2024-04-25T09:01:36.575628+0000","last_became_active":"2024-04-25T09:01:34.578755+0000","last_became_peered":"2024-04-25T09:01:34.578755+0000","last_unstale":"2024-04-25T09:01:36.575628+0000","last_undegraded":"2024-04-25T09:01:36.575628+0000","last_fullsized":"2024-04-25T09:01:36.575628+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:26.293458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581582+0000","last_change":"2024-04-25T09:01:35.599297+0000","last_active":"2024-04-25T09:01:36.581582+0000","last_peered":"2024-04-25T09:01:36.581582+0000","last_clean":"2024-04-25T09:01:36.581582+0000","last_became_active":"2024-04-25T09:01:35.598386+0000","last_became_peered":"2024-04-25T09:01:35.598386+0000","last_unstale":"2024-04-25T09:01:36.581582+0000","last_undegraded":"2024-04-25T09:01:36.581582+0000","last_fullsized":"2024-04-25T09:01:36.581582+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:15.579088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583367+0000","last_change":"2024-04-25T09:01:34.563633+0000","last_active":"2024-04-25T09:01:36.583367+0000","last_peered":"2024-04-25T09:01:36.583367+0000","last_clean":"2024-04-25T09:01:36.583367+0000","last_became_active":"2024-04-25T09:01:34.562955+0000","last_became_peered":"2024-04-25T09:01:34.562955+0000","last_unstale":"2024-04-25T09:01:36.583367+0000","last_undegraded":"2024-04-25T09:01:36.583367+0000","last_fullsized":"2024-04-25T09:01:36.583367+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:25.364305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.588165+0000","last_change":"2024-04-25T09:01:35.626092+0000","last_active":"2024-04-25T09:01:36.588165+0000","last_peered":"2024-04-25T09:01:36.588165+0000","last_clean":"2024-04-25T09:01:36.588165+0000","last_became_active":"2024-04-25T09:01:35.625490+0000","last_became_peered":"2024-04-25T09:01:35.625490+0000","last_unstale":"2024-04-25T09:01:36.588165+0000","last_undegraded":"2024-04-25T09:01:36.588165+0000","last_fullsized":"2024-04-25T09:01:36.588165+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:00.626616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945443+0000","last_change":"2024-04-25T09:01:34.572762+0000","last_active":"2024-04-25T09:01:36.945443+0000","last_peered":"2024-04-25T09:01:36.945443+0000","last_clean":"2024-04-25T09:01:36.945443+0000","last_became_active":"2024-04-25T09:01:34.572438+0000","last_became_peered":"2024-04-25T09:01:34.572438+0000","last_unstale":"2024-04-25T09:01:36.945443+0000","last_undegraded":"2024-04-25T09:01:36.945443+0000","last_fullsized":"2024-04-25T09:01:36.945443+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:14.312491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944547+0000","last_change":"2024-04-25T09:01:35.595056+0000","last_active":"2024-04-25T09:01:36.944547+0000","last_peered":"2024-04-25T09:01:36.944547+0000","last_clean":"2024-04-25T09:01:36.944547+0000","last_became_active":"2024-04-25T09:01:35.594690+0000","last_became_peered":"2024-04-25T09:01:35.594690+0000","last_unstale":"2024-04-25T09:01:36.944547+0000","last_undegraded":"2024-04-25T09:01:36.944547+0000","last_fullsized":"2024-04-25T09:01:36.944547+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:16.186800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946181+0000","last_change":"2024-04-25T09:01:34.583811+0000","last_active":"2024-04-25T09:01:36.946181+0000","last_peered":"2024-04-25T09:01:36.946181+0000","last_clean":"2024-04-25T09:01:36.946181+0000","last_became_active":"2024-04-25T09:01:34.583046+0000","last_became_peered":"2024-04-25T09:01:34.583046+0000","last_unstale":"2024-04-25T09:01:36.946181+0000","last_undegraded":"2024-04-25T09:01:36.946181+0000","last_fullsized":"2024-04-25T09:01:36.946181+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:08.313494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581452+0000","last_change":"2024-04-25T09:01:35.619247+0000","last_active":"2024-04-25T09:01:36.581452+0000","last_peered":"2024-04-25T09:01:36.581452+0000","last_clean":"2024-04-25T09:01:36.581452+0000","last_became_active":"2024-04-25T09:01:35.618925+0000","last_became_peered":"2024-04-25T09:01:35.618925+0000","last_unstale":"2024-04-25T09:01:36.581452+0000","last_undegraded":"2024-04-25T09:01:36.581452+0000","last_fullsized":"2024-04-25T09:01:36.581452+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:34.661103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.587730+0000","last_change":"2024-04-25T09:01:34.567928+0000","last_active":"2024-04-25T09:01:36.587730+0000","last_peered":"2024-04-25T09:01:36.587730+0000","last_clean":"2024-04-25T09:01:36.587730+0000","last_became_active":"2024-04-25T09:01:34.563362+0000","last_became_peered":"2024-04-25T09:01:34.563362+0000","last_unstale":"2024-04-25T09:01:36.587730+0000","last_undegraded":"2024-04-25T09:01:36.587730+0000","last_fullsized":"2024-04-25T09:01:36.587730+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:25.909339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582949+0000","last_change":"2024-04-25T09:01:35.633300+0000","last_active":"2024-04-25T09:01:36.582949+0000","last_peered":"2024-04-25T09:01:36.582949+0000","last_clean":"2024-04-25T09:01:36.582949+0000","last_became_active":"2024-04-25T09:01:35.632636+0000","last_became_peered":"2024-04-25T09:01:35.632636+0000","last_unstale":"2024-04-25T09:01:36.582949+0000","last_undegraded":"2024-04-25T09:01:36.582949+0000","last_fullsized":"2024-04-25T09:01:36.582949+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.998517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581281+0000","last_change":"2024-04-25T09:01:34.562921+0000","last_active":"2024-04-25T09:01:36.581281+0000","last_peered":"2024-04-25T09:01:36.581281+0000","last_clean":"2024-04-25T09:01:36.581281+0000","last_became_active":"2024-04-25T09:01:34.562136+0000","last_became_peered":"2024-04-25T09:01:34.562136+0000","last_unstale":"2024-04-25T09:01:36.581281+0000","last_undegraded":"2024-04-25T09:01:36.581281+0000","last_fullsized":"2024-04-25T09:01:36.581281+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:18.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944492+0000","last_change":"2024-04-25T09:01:35.598252+0000","last_active":"2024-04-25T09:01:36.944492+0000","last_peered":"2024-04-25T09:01:36.944492+0000","last_clean":"2024-04-25T09:01:36.944492+0000","last_became_active":"2024-04-25T09:01:35.597495+0000","last_became_peered":"2024-04-25T09:01:35.597495+0000","last_unstale":"2024-04-25T09:01:36.944492+0000","last_undegraded":"2024-04-25T09:01:36.944492+0000","last_fullsized":"2024-04-25T09:01:36.944492+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:29.756119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.943444+0000","last_change":"2024-04-25T09:01:34.561292+0000","last_active":"2024-04-25T09:01:36.943444+0000","last_peered":"2024-04-25T09:01:36.943444+0000","last_clean":"2024-04-25T09:01:36.943444+0000","last_became_active":"2024-04-25T09:01:34.560996+0000","last_became_peered":"2024-04-25T09:01:34.560996+0000","last_unstale":"2024-04-25T09:01:36.943444+0000","last_undegraded":"2024-04-25T09:01:36.943444+0000","last_fullsized":"2024-04-25T09:01:36.943444+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:55.943449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580205+0000","last_change":"2024-04-25T09:01:35.608324+0000","last_active":"2024-04-25T09:01:36.580205+0000","last_peered":"2024-04-25T09:01:36.580205+0000","last_clean":"2024-04-25T09:01:36.580205+0000","last_became_active":"2024-04-25T09:01:35.607963+0000","last_became_peered":"2024-04-25T09:01:35.607963+0000","last_unstale":"2024-04-25T09:01:36.580205+0000","last_undegraded":"2024-04-25T09:01:36.580205+0000","last_fullsized":"2024-04-25T09:01:36.580205+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:54.729004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583591+0000","last_change":"2024-04-25T09:01:34.571791+0000","last_active":"2024-04-25T09:01:36.583591+0000","last_peered":"2024-04-25T09:01:36.583591+0000","last_clean":"2024-04-25T09:01:36.583591+0000","last_became_active":"2024-04-25T09:01:34.564950+0000","last_became_peered":"2024-04-25T09:01:34.564950+0000","last_unstale":"2024-04-25T09:01:36.583591+0000","last_undegraded":"2024-04-25T09:01:36.583591+0000","last_fullsized":"2024-04-25T09:01:36.583591+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:47.800143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944582+0000","last_change":"2024-04-25T09:01:35.600817+0000","last_active":"2024-04-25T09:01:36.944582+0000","last_peered":"2024-04-25T09:01:36.944582+0000","last_clean":"2024-04-25T09:01:36.944582+0000","last_became_active":"2024-04-25T09:01:35.600548+0000","last_became_peered":"2024-04-25T09:01:35.600548+0000","last_unstale":"2024-04-25T09:01:36.944582+0000","last_undegraded":"2024-04-25T09:01:36.944582+0000","last_fullsized":"2024-04-25T09:01:36.944582+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:35.354671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580161+0000","last_change":"2024-04-25T09:01:34.568280+0000","last_active":"2024-04-25T09:01:36.580161+0000","last_peered":"2024-04-25T09:01:36.580161+0000","last_clean":"2024-04-25T09:01:36.580161+0000","last_became_active":"2024-04-25T09:01:34.567918+0000","last_became_peered":"2024-04-25T09:01:34.567918+0000","last_unstale":"2024-04-25T09:01:36.580161+0000","last_undegraded":"2024-04-25T09:01:36.580161+0000","last_fullsized":"2024-04-25T09:01:36.580161+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:17.580384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575499+0000","last_change":"2024-04-25T09:01:35.592084+0000","last_active":"2024-04-25T09:01:36.575499+0000","last_peered":"2024-04-25T09:01:36.575499+0000","last_clean":"2024-04-25T09:01:36.575499+0000","last_became_active":"2024-04-25T09:01:35.591787+0000","last_became_peered":"2024-04-25T09:01:35.591787+0000","last_unstale":"2024-04-25T09:01:36.575499+0000","last_undegraded":"2024-04-25T09:01:36.575499+0000","last_fullsized":"2024-04-25T09:01:36.575499+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:56.964888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"109'3","reported_seq":17,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946271+0000","last_change":"2024-04-25T09:01:34.578215+0000","last_active":"2024-04-25T09:01:36.946271+0000","last_peered":"2024-04-25T09:01:36.946271+0000","last_clean":"2024-04-25T09:01:36.946271+0000","last_became_active":"2024-04-25T09:01:34.577464+0000","last_became_peered":"2024-04-25T09:01:34.577464+0000","last_unstale":"2024-04-25T09:01:36.946271+0000","last_undegraded":"2024-04-25T09:01:36.946271+0000","last_fullsized":"2024-04-25T09:01:36.946271+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:32.786071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945682+0000","last_change":"2024-04-25T09:01:35.600419+0000","last_active":"2024-04-25T09:01:36.945682+0000","last_peered":"2024-04-25T09:01:36.945682+0000","last_clean":"2024-04-25T09:01:36.945682+0000","last_became_active":"2024-04-25T09:01:35.600121+0000","last_became_peered":"2024-04-25T09:01:35.600121+0000","last_unstale":"2024-04-25T09:01:36.945682+0000","last_undegraded":"2024-04-25T09:01:36.945682+0000","last_fullsized":"2024-04-25T09:01:36.945682+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:22.247172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"109'6","reported_seq":20,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582927+0000","last_change":"2024-04-25T09:01:34.563869+0000","last_active":"2024-04-25T09:01:36.582927+0000","last_peered":"2024-04-25T09:01:36.582927+0000","last_clean":"2024-04-25T09:01:36.582927+0000","last_became_active":"2024-04-25T09:01:34.562417+0000","last_became_peered":"2024-04-25T09:01:34.562417+0000","last_unstale":"2024-04-25T09:01:36.582927+0000","last_undegraded":"2024-04-25T09:01:36.582927+0000","last_fullsized":"2024-04-25T09:01:36.582927+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:14.312491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.587947+0000","last_change":"2024-04-25T09:01:35.624505+0000","last_active":"2024-04-25T09:01:36.587947+0000","last_peered":"2024-04-25T09:01:36.587947+0000","last_clean":"2024-04-25T09:01:36.587947+0000","last_became_active":"2024-04-25T09:01:35.614092+0000","last_became_peered":"2024-04-25T09:01:35.614092+0000","last_unstale":"2024-04-25T09:01:36.587947+0000","last_undegraded":"2024-04-25T09:01:36.587947+0000","last_fullsized":"2024-04-25T09:01:36.587947+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:48.812232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945662+0000","last_change":"2024-04-25T09:01:34.575412+0000","last_active":"2024-04-25T09:01:36.945662+0000","last_peered":"2024-04-25T09:01:36.945662+0000","last_clean":"2024-04-25T09:01:36.945662+0000","last_became_active":"2024-04-25T09:01:34.574972+0000","last_became_peered":"2024-04-25T09:01:34.574972+0000","last_unstale":"2024-04-25T09:01:36.945662+0000","last_undegraded":"2024-04-25T09:01:36.945662+0000","last_fullsized":"2024-04-25T09:01:36.945662+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:25.909339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582509+0000","last_change":"2024-04-25T09:01:35.608312+0000","last_active":"2024-04-25T09:01:36.582509+0000","last_peered":"2024-04-25T09:01:36.582509+0000","last_clean":"2024-04-25T09:01:36.582509+0000","last_became_active":"2024-04-25T09:01:35.607545+0000","last_became_peered":"2024-04-25T09:01:35.607545+0000","last_unstale":"2024-04-25T09:01:36.582509+0000","last_undegraded":"2024-04-25T09:01:36.582509+0000","last_fullsized":"2024-04-25T09:01:36.582509+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:56.955538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944044+0000","last_change":"2024-04-25T09:01:34.577629+0000","last_active":"2024-04-25T09:01:36.944044+0000","last_peered":"2024-04-25T09:01:36.944044+0000","last_clean":"2024-04-25T09:01:36.944044+0000","last_became_active":"2024-04-25T09:01:34.577195+0000","last_became_peered":"2024-04-25T09:01:34.577195+0000","last_unstale":"2024-04-25T09:01:36.944044+0000","last_undegraded":"2024-04-25T09:01:36.944044+0000","last_fullsized":"2024-04-25T09:01:36.944044+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:14.566999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575530+0000","last_change":"2024-04-25T09:01:35.582611+0000","last_active":"2024-04-25T09:01:36.575530+0000","last_peered":"2024-04-25T09:01:36.575530+0000","last_clean":"2024-04-25T09:01:36.575530+0000","last_became_active":"2024-04-25T09:01:35.582255+0000","last_became_peered":"2024-04-25T09:01:35.582255+0000","last_unstale":"2024-04-25T09:01:36.575530+0000","last_undegraded":"2024-04-25T09:01:36.575530+0000","last_fullsized":"2024-04-25T09:01:36.575530+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:54.343143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575449+0000","last_change":"2024-04-25T09:01:34.564481+0000","last_active":"2024-04-25T09:01:36.575449+0000","last_peered":"2024-04-25T09:01:36.575449+0000","last_clean":"2024-04-25T09:01:36.575449+0000","last_became_active":"2024-04-25T09:01:34.564183+0000","last_became_peered":"2024-04-25T09:01:34.564183+0000","last_unstale":"2024-04-25T09:01:36.575449+0000","last_undegraded":"2024-04-25T09:01:36.575449+0000","last_fullsized":"2024-04-25T09:01:36.575449+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:51.319222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581467+0000","last_change":"2024-04-25T09:01:35.635354+0000","last_active":"2024-04-25T09:01:36.581467+0000","last_peered":"2024-04-25T09:01:36.581467+0000","last_clean":"2024-04-25T09:01:36.581467+0000","last_became_active":"2024-04-25T09:01:35.635038+0000","last_became_peered":"2024-04-25T09:01:35.635038+0000","last_unstale":"2024-04-25T09:01:36.581467+0000","last_undegraded":"2024-04-25T09:01:36.581467+0000","last_fullsized":"2024-04-25T09:01:36.581467+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:31.842568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"109'8","reported_seq":22,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580668+0000","last_change":"2024-04-25T09:01:34.562989+0000","last_active":"2024-04-25T09:01:36.580668+0000","last_peered":"2024-04-25T09:01:36.580668+0000","last_clean":"2024-04-25T09:01:36.580668+0000","last_became_active":"2024-04-25T09:01:34.562243+0000","last_became_peered":"2024-04-25T09:01:34.562243+0000","last_unstale":"2024-04-25T09:01:36.580668+0000","last_undegraded":"2024-04-25T09:01:36.580668+0000","last_fullsized":"2024-04-25T09:01:36.580668+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:47.904471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575011+0000","last_change":"2024-04-25T09:01:35.592933+0000","last_active":"2024-04-25T09:01:36.575011+0000","last_peered":"2024-04-25T09:01:36.575011+0000","last_clean":"2024-04-25T09:01:36.575011+0000","last_became_active":"2024-04-25T09:01:35.592653+0000","last_became_peered":"2024-04-25T09:01:35.592653+0000","last_unstale":"2024-04-25T09:01:36.575011+0000","last_undegraded":"2024-04-25T09:01:36.575011+0000","last_fullsized":"2024-04-25T09:01:36.575011+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:46.340931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582237+0000","last_change":"2024-04-25T09:01:34.565923+0000","last_active":"2024-04-25T09:01:36.582237+0000","last_peered":"2024-04-25T09:01:36.582237+0000","last_clean":"2024-04-25T09:01:36.582237+0000","last_became_active":"2024-04-25T09:01:34.565063+0000","last_became_peered":"2024-04-25T09:01:34.565063+0000","last_unstale":"2024-04-25T09:01:36.582237+0000","last_undegraded":"2024-04-25T09:01:36.582237+0000","last_fullsized":"2024-04-25T09:01:36.582237+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:53.717265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582041+0000","last_change":"2024-04-25T09:01:35.622490+0000","last_active":"2024-04-25T09:01:36.582041+0000","last_peered":"2024-04-25T09:01:36.582041+0000","last_clean":"2024-04-25T09:01:36.582041+0000","last_became_active":"2024-04-25T09:01:35.620731+0000","last_became_peered":"2024-04-25T09:01:35.620731+0000","last_unstale":"2024-04-25T09:01:36.582041+0000","last_undegraded":"2024-04-25T09:01:36.582041+0000","last_fullsized":"2024-04-25T09:01:36.582041+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:57.793012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"109'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946385+0000","last_change":"2024-04-25T09:01:34.585991+0000","last_active":"2024-04-25T09:01:36.946385+0000","last_peered":"2024-04-25T09:01:36.946385+0000","last_clean":"2024-04-25T09:01:36.946385+0000","last_became_active":"2024-04-25T09:01:34.585265+0000","last_became_peered":"2024-04-25T09:01:34.585265+0000","last_unstale":"2024-04-25T09:01:36.946385+0000","last_undegraded":"2024-04-25T09:01:36.946385+0000","last_fullsized":"2024-04-25T09:01:36.946385+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:18.766303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945370+0000","last_change":"2024-04-25T09:01:35.610613+0000","last_active":"2024-04-25T09:01:36.945370+0000","last_peered":"2024-04-25T09:01:36.945370+0000","last_clean":"2024-04-25T09:01:36.945370+0000","last_became_active":"2024-04-25T09:01:35.610293+0000","last_became_peered":"2024-04-25T09:01:35.610293+0000","last_unstale":"2024-04-25T09:01:36.945370+0000","last_undegraded":"2024-04-25T09:01:36.945370+0000","last_fullsized":"2024-04-25T09:01:36.945370+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:57.793012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.587832+0000","last_change":"2024-04-25T09:01:34.571639+0000","last_active":"2024-04-25T09:01:36.587832+0000","last_peered":"2024-04-25T09:01:36.587832+0000","last_clean":"2024-04-25T09:01:36.587832+0000","last_became_active":"2024-04-25T09:01:34.570919+0000","last_became_peered":"2024-04-25T09:01:34.570919+0000","last_unstale":"2024-04-25T09:01:36.587832+0000","last_undegraded":"2024-04-25T09:01:36.587832+0000","last_fullsized":"2024-04-25T09:01:36.587832+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:14.312491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580619+0000","last_change":"2024-04-25T09:01:35.597119+0000","last_active":"2024-04-25T09:01:36.580619+0000","last_peered":"2024-04-25T09:01:36.580619+0000","last_clean":"2024-04-25T09:01:36.580619+0000","last_became_active":"2024-04-25T09:01:35.596843+0000","last_became_peered":"2024-04-25T09:01:35.596843+0000","last_unstale":"2024-04-25T09:01:36.580619+0000","last_undegraded":"2024-04-25T09:01:36.580619+0000","last_fullsized":"2024-04-25T09:01:36.580619+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:18.951739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.576554+0000","last_change":"2024-04-25T09:01:34.563183+0000","last_active":"2024-04-25T09:01:36.576554+0000","last_peered":"2024-04-25T09:01:36.576554+0000","last_clean":"2024-04-25T09:01:36.576554+0000","last_became_active":"2024-04-25T09:01:34.552350+0000","last_became_peered":"2024-04-25T09:01:34.552350+0000","last_unstale":"2024-04-25T09:01:36.576554+0000","last_undegraded":"2024-04-25T09:01:36.576554+0000","last_fullsized":"2024-04-25T09:01:36.576554+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:13:57.740142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581342+0000","last_change":"2024-04-25T09:01:35.593304+0000","last_active":"2024-04-25T09:01:36.581342+0000","last_peered":"2024-04-25T09:01:36.581342+0000","last_clean":"2024-04-25T09:01:36.581342+0000","last_became_active":"2024-04-25T09:01:35.589329+0000","last_became_peered":"2024-04-25T09:01:35.589329+0000","last_unstale":"2024-04-25T09:01:36.581342+0000","last_undegraded":"2024-04-25T09:01:36.581342+0000","last_fullsized":"2024-04-25T09:01:36.581342+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:10.798939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.574958+0000","last_change":"2024-04-25T09:01:34.572962+0000","last_active":"2024-04-25T09:01:36.574958+0000","last_peered":"2024-04-25T09:01:36.574958+0000","last_clean":"2024-04-25T09:01:36.574958+0000","last_became_active":"2024-04-25T09:01:34.572395+0000","last_became_peered":"2024-04-25T09:01:34.572395+0000","last_unstale":"2024-04-25T09:01:36.574958+0000","last_undegraded":"2024-04-25T09:01:36.574958+0000","last_fullsized":"2024-04-25T09:01:36.574958+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.394912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944393+0000","last_change":"2024-04-25T09:01:35.621012+0000","last_active":"2024-04-25T09:01:36.944393+0000","last_peered":"2024-04-25T09:01:36.944393+0000","last_clean":"2024-04-25T09:01:36.944393+0000","last_became_active":"2024-04-25T09:01:35.620775+0000","last_became_peered":"2024-04-25T09:01:35.620775+0000","last_unstale":"2024-04-25T09:01:36.944393+0000","last_undegraded":"2024-04-25T09:01:36.944393+0000","last_fullsized":"2024-04-25T09:01:36.944393+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:46.183788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581273+0000","last_change":"2024-04-25T09:01:34.563744+0000","last_active":"2024-04-25T09:01:36.581273+0000","last_peered":"2024-04-25T09:01:36.581273+0000","last_clean":"2024-04-25T09:01:36.581273+0000","last_became_active":"2024-04-25T09:01:34.555693+0000","last_became_peered":"2024-04-25T09:01:34.555693+0000","last_unstale":"2024-04-25T09:01:36.581273+0000","last_undegraded":"2024-04-25T09:01:36.581273+0000","last_fullsized":"2024-04-25T09:01:36.581273+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:12.841471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.576135+0000","last_change":"2024-04-25T09:01:35.590817+0000","last_active":"2024-04-25T09:01:36.576135+0000","last_peered":"2024-04-25T09:01:36.576135+0000","last_clean":"2024-04-25T09:01:36.576135+0000","last_became_active":"2024-04-25T09:01:35.590502+0000","last_became_peered":"2024-04-25T09:01:35.590502+0000","last_unstale":"2024-04-25T09:01:36.576135+0000","last_undegraded":"2024-04-25T09:01:36.576135+0000","last_fullsized":"2024-04-25T09:01:36.576135+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:33.689092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575422+0000","last_change":"2024-04-25T09:01:34.572454+0000","last_active":"2024-04-25T09:01:36.575422+0000","last_peered":"2024-04-25T09:01:36.575422+0000","last_clean":"2024-04-25T09:01:36.575422+0000","last_became_active":"2024-04-25T09:01:34.571813+0000","last_became_peered":"2024-04-25T09:01:34.571813+0000","last_unstale":"2024-04-25T09:01:36.575422+0000","last_undegraded":"2024-04-25T09:01:36.575422+0000","last_fullsized":"2024-04-25T09:01:36.575422+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:51.788729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.580168+0000","last_change":"2024-04-25T09:01:35.597530+0000","last_active":"2024-04-25T09:01:36.580168+0000","last_peered":"2024-04-25T09:01:36.580168+0000","last_clean":"2024-04-25T09:01:36.580168+0000","last_became_active":"2024-04-25T09:01:35.597127+0000","last_became_peered":"2024-04-25T09:01:35.597127+0000","last_unstale":"2024-04-25T09:01:36.580168+0000","last_undegraded":"2024-04-25T09:01:36.580168+0000","last_fullsized":"2024-04-25T09:01:36.580168+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:48.092531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945816+0000","last_change":"2024-04-25T09:01:34.575210+0000","last_active":"2024-04-25T09:01:36.945816+0000","last_peered":"2024-04-25T09:01:36.945816+0000","last_clean":"2024-04-25T09:01:36.945816+0000","last_became_active":"2024-04-25T09:01:34.574914+0000","last_became_peered":"2024-04-25T09:01:34.574914+0000","last_unstale":"2024-04-25T09:01:36.945816+0000","last_undegraded":"2024-04-25T09:01:36.945816+0000","last_fullsized":"2024-04-25T09:01:36.945816+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:53.717265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581014+0000","last_change":"2024-04-25T09:01:35.600699+0000","last_active":"2024-04-25T09:01:36.581014+0000","last_peered":"2024-04-25T09:01:36.581014+0000","last_clean":"2024-04-25T09:01:36.581014+0000","last_became_active":"2024-04-25T09:01:35.599981+0000","last_became_peered":"2024-04-25T09:01:35.599981+0000","last_unstale":"2024-04-25T09:01:36.581014+0000","last_undegraded":"2024-04-25T09:01:36.581014+0000","last_fullsized":"2024-04-25T09:01:36.581014+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:32.485434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"110'1","reported_seq":15,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583939+0000","last_change":"2024-04-25T09:01:34.570714+0000","last_active":"2024-04-25T09:01:36.583939+0000","last_peered":"2024-04-25T09:01:36.583939+0000","last_clean":"2024-04-25T09:01:36.583939+0000","last_became_active":"2024-04-25T09:01:34.570358+0000","last_became_peered":"2024-04-25T09:01:34.570358+0000","last_unstale":"2024-04-25T09:01:36.583939+0000","last_undegraded":"2024-04-25T09:01:36.583939+0000","last_fullsized":"2024-04-25T09:01:36.583939+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:31.032184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.944335+0000","last_change":"2024-04-25T09:01:35.607939+0000","last_active":"2024-04-25T09:01:36.944335+0000","last_peered":"2024-04-25T09:01:36.944335+0000","last_clean":"2024-04-25T09:01:36.944335+0000","last_became_active":"2024-04-25T09:01:35.607591+0000","last_became_peered":"2024-04-25T09:01:35.607591+0000","last_unstale":"2024-04-25T09:01:36.944335+0000","last_undegraded":"2024-04-25T09:01:36.944335+0000","last_fullsized":"2024-04-25T09:01:36.944335+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:55.983973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.582565+0000","last_change":"2024-04-25T09:01:34.570021+0000","last_active":"2024-04-25T09:01:36.582565+0000","last_peered":"2024-04-25T09:01:36.582565+0000","last_clean":"2024-04-25T09:01:36.582565+0000","last_became_active":"2024-04-25T09:01:34.566318+0000","last_became_peered":"2024-04-25T09:01:34.566318+0000","last_unstale":"2024-04-25T09:01:36.582565+0000","last_undegraded":"2024-04-25T09:01:36.582565+0000","last_fullsized":"2024-04-25T09:01:36.582565+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:07.276093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.588143+0000","last_change":"2024-04-25T09:01:35.624833+0000","last_active":"2024-04-25T09:01:36.588143+0000","last_peered":"2024-04-25T09:01:36.588143+0000","last_clean":"2024-04-25T09:01:36.588143+0000","last_became_active":"2024-04-25T09:01:35.614154+0000","last_became_peered":"2024-04-25T09:01:35.614154+0000","last_unstale":"2024-04-25T09:01:36.588143+0000","last_undegraded":"2024-04-25T09:01:36.588143+0000","last_fullsized":"2024-04-25T09:01:36.588143+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:47.743189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945605+0000","last_change":"2024-04-25T09:01:34.578871+0000","last_active":"2024-04-25T09:01:36.945605+0000","last_peered":"2024-04-25T09:01:36.945605+0000","last_clean":"2024-04-25T09:01:36.945605+0000","last_became_active":"2024-04-25T09:01:34.578150+0000","last_became_peered":"2024-04-25T09:01:34.578150+0000","last_unstale":"2024-04-25T09:01:36.945605+0000","last_undegraded":"2024-04-25T09:01:36.945605+0000","last_fullsized":"2024-04-25T09:01:36.945605+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:07.276093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575465+0000","last_change":"2024-04-25T09:01:35.592748+0000","last_active":"2024-04-25T09:01:36.575465+0000","last_peered":"2024-04-25T09:01:36.575465+0000","last_clean":"2024-04-25T09:01:36.575465+0000","last_became_active":"2024-04-25T09:01:35.592409+0000","last_became_peered":"2024-04-25T09:01:35.592409+0000","last_unstale":"2024-04-25T09:01:36.575465+0000","last_undegraded":"2024-04-25T09:01:36.575465+0000","last_fullsized":"2024-04-25T09:01:36.575465+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:20.380662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.583025+0000","last_change":"2024-04-25T09:01:34.582493+0000","last_active":"2024-04-25T09:01:36.583025+0000","last_peered":"2024-04-25T09:01:36.583025+0000","last_clean":"2024-04-25T09:01:36.583025+0000","last_became_active":"2024-04-25T09:01:34.581783+0000","last_became_peered":"2024-04-25T09:01:34.581783+0000","last_unstale":"2024-04-25T09:01:36.583025+0000","last_undegraded":"2024-04-25T09:01:36.583025+0000","last_fullsized":"2024-04-25T09:01:36.583025+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:35:32.786071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.576024+0000","last_change":"2024-04-25T09:01:35.590890+0000","last_active":"2024-04-25T09:01:36.576024+0000","last_peered":"2024-04-25T09:01:36.576024+0000","last_clean":"2024-04-25T09:01:36.576024+0000","last_became_active":"2024-04-25T09:01:35.590581+0000","last_became_peered":"2024-04-25T09:01:35.590581+0000","last_unstale":"2024-04-25T09:01:36.576024+0000","last_undegraded":"2024-04-25T09:01:36.576024+0000","last_fullsized":"2024-04-25T09:01:36.576024+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:49.213953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575219+0000","last_change":"2024-04-25T09:01:34.564697+0000","last_active":"2024-04-25T09:01:36.575219+0000","last_peered":"2024-04-25T09:01:36.575219+0000","last_clean":"2024-04-25T09:01:36.575219+0000","last_became_active":"2024-04-25T09:01:34.563926+0000","last_became_peered":"2024-04-25T09:01:34.563926+0000","last_unstale":"2024-04-25T09:01:36.575219+0000","last_undegraded":"2024-04-25T09:01:36.575219+0000","last_fullsized":"2024-04-25T09:01:36.575219+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:43:12.329463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.946741+0000","last_change":"2024-04-25T09:01:35.640140+0000","last_active":"2024-04-25T09:01:36.946741+0000","last_peered":"2024-04-25T09:01:36.946741+0000","last_clean":"2024-04-25T09:01:36.946741+0000","last_became_active":"2024-04-25T09:01:35.639625+0000","last_became_peered":"2024-04-25T09:01:35.639625+0000","last_unstale":"2024-04-25T09:01:36.946741+0000","last_undegraded":"2024-04-25T09:01:36.946741+0000","last_fullsized":"2024-04-25T09:01:36.946741+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:46.480732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.581564+0000","last_change":"2024-04-25T09:01:34.568304+0000","last_active":"2024-04-25T09:01:36.581564+0000","last_peered":"2024-04-25T09:01:36.581564+0000","last_clean":"2024-04-25T09:01:36.581564+0000","last_became_active":"2024-04-25T09:01:34.566058+0000","last_became_peered":"2024-04-25T09:01:34.566058+0000","last_unstale":"2024-04-25T09:01:36.581564+0000","last_undegraded":"2024-04-25T09:01:36.581564+0000","last_fullsized":"2024-04-25T09:01:36.581564+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:58.658639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.588662+0000","last_change":"2024-04-25T09:01:35.635598+0000","last_active":"2024-04-25T09:01:36.588662+0000","last_peered":"2024-04-25T09:01:36.588662+0000","last_clean":"2024-04-25T09:01:36.588662+0000","last_became_active":"2024-04-25T09:01:35.631405+0000","last_became_peered":"2024-04-25T09:01:35.631405+0000","last_unstale":"2024-04-25T09:01:36.588662+0000","last_undegraded":"2024-04-25T09:01:36.588662+0000","last_fullsized":"2024-04-25T09:01:36.588662+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:07.559766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575126+0000","last_change":"2024-04-25T09:01:34.570629+0000","last_active":"2024-04-25T09:01:36.575126+0000","last_peered":"2024-04-25T09:01:36.575126+0000","last_clean":"2024-04-25T09:01:36.575126+0000","last_became_active":"2024-04-25T09:01:34.565444+0000","last_became_peered":"2024-04-25T09:01:34.565444+0000","last_unstale":"2024-04-25T09:01:36.575126+0000","last_undegraded":"2024-04-25T09:01:36.575126+0000","last_fullsized":"2024-04-25T09:01:36.575126+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:33.052275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.587994+0000","last_change":"2024-04-25T09:01:35.632674+0000","last_active":"2024-04-25T09:01:36.587994+0000","last_peered":"2024-04-25T09:01:36.587994+0000","last_clean":"2024-04-25T09:01:36.587994+0000","last_became_active":"2024-04-25T09:01:35.631162+0000","last_became_peered":"2024-04-25T09:01:35.631162+0000","last_unstale":"2024-04-25T09:01:36.587994+0000","last_undegraded":"2024-04-25T09:01:36.587994+0000","last_fullsized":"2024-04-25T09:01:36.587994+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:34.526546+0000","last_clean_scrub_stamp":"2024-04-25T09:01:34.526546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:26.376393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"109'2","reported_seq":16,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.575806+0000","last_change":"2024-04-25T09:01:34.570226+0000","last_active":"2024-04-25T09:01:36.575806+0000","last_peered":"2024-04-25T09:01:36.575806+0000","last_clean":"2024-04-25T09:01:36.575806+0000","last_became_active":"2024-04-25T09:01:34.565514+0000","last_became_peered":"2024-04-25T09:01:34.565514+0000","last_unstale":"2024-04-25T09:01:36.575806+0000","last_undegraded":"2024-04-25T09:01:36.575806+0000","last_fullsized":"2024-04-25T09:01:36.575806+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.514458+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.514458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:22.889807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":339,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.945736+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T09:01:36.945736+0000","last_peered":"2024-04-25T09:01:36.945736+0000","last_clean":"2024-04-25T09:01:36.945736+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T09:01:36.945736+0000","last_undegraded":"2024-04-25T09:01:36.945736+0000","last_fullsized":"2024-04-25T09:01:36.945736+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1966080,"data_stored":1227263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574964,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32536,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339304,"statfs":{"total":96636764160,"available":96603447296,"internally_reserved":0,"allocated":1384448,"data_stored":638951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32516,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1363968,"data_stored":636735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32524,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339316,"statfs":{"total":96636764160,"available":96603459584,"internally_reserved":0,"allocated":1372160,"data_stored":636827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1368064,"data_stored":637317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1368064,"data_stored":636769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32520,"kb_used_data":1336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339320,"statfs":{"total":96636764160,"available":96603463680,"internally_reserved":0,"allocated":1368064,"data_stored":636769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574965,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1974272,"data_stored":1228805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:01:39.999 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:01:40.000 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:01:40.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.178+0000 7f164b850640 1 Processor -- start 2024-04-25T09:01:40.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.178+0000 7f164b850640 1 -- start start 2024-04-25T09:01:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.178+0000 7f164b850640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f16440b0a30 0x7f16440cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.178+0000 7f164b850640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f164406f4a0 0x7f1644099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.178+0000 7f164b850640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1644099fb0 0x7f164415d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.178+0000 7f164b850640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16440d0d90 con 0x7f164406f4a0 2024-04-25T09:01:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.178+0000 7f164b850640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16440b0e10 con 0x7f16440b0a30 2024-04-25T09:01:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.178+0000 7f164b850640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16440d0180 con 0x7f1644099fb0 2024-04-25T09:01:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f1648deb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f164406f4a0 0x7f1644099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f1648deb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f164406f4a0 0x7f1644099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51818/0 (socket says 172.21.15.37:51818) 2024-04-25T09:01:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f1648deb640 1 -- 172.21.15.37:0/321387110 learned_addr learned my addr 172.21.15.37:0/321387110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f1649ded640 1 --2- 172.21.15.37:0/321387110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1644099fb0 0x7f164415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f16495ec640 1 --2- 172.21.15.37:0/321387110 >> v2:172.21.15.203:3300/0 conn(0x7f16440b0a30 0x7f16440cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f1648deb640 1 -- 172.21.15.37:0/321387110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1644099fb0 msgr2=0x7f164415d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f1648deb640 1 --2- 172.21.15.37:0/321387110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1644099fb0 0x7f164415d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f1648deb640 1 -- 172.21.15.37:0/321387110 >> v2:172.21.15.203:3300/0 conn(0x7f16440b0a30 msgr2=0x7f16440cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f1648deb640 1 --2- 172.21.15.37:0/321387110 >> v2:172.21.15.203:3300/0 conn(0x7f16440b0a30 0x7f16440cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f1648deb640 1 -- 172.21.15.37:0/321387110 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16440d0460 con 0x7f164406f4a0 2024-04-25T09:01:40.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f1648deb640 1 --2- 172.21.15.37:0/321387110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f164406f4a0 0x7f1644099a70 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1644072ac0 tx=0x7f163401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14aafebff9ca3a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:40.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f163bfff640 1 -- 172.21.15.37:0/321387110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16340b1070 con 0x7f164406f4a0 2024-04-25T09:01:40.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f163bfff640 1 -- 172.21.15.37:0/321387110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16340ac630 con 0x7f164406f4a0 2024-04-25T09:01:40.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f163bfff640 1 -- 172.21.15.37:0/321387110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16340a8050 con 0x7f164406f4a0 2024-04-25T09:01:40.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f164b850640 1 -- 172.21.15.37:0/321387110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f164406f4a0 msgr2=0x7f1644099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:40.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f164b850640 1 --2- 172.21.15.37:0/321387110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f164406f4a0 0x7f1644099a70 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f1644072ac0 tx=0x7f163401c520 comp rx=0 tx=0).stop 2024-04-25T09:01:40.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f164b850640 1 -- 172.21.15.37:0/321387110 shutdown_connections 2024-04-25T09:01:40.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f164b850640 1 --2- 172.21.15.37:0/321387110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f164406f4a0 0x7f1644099a70 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f164b850640 1 --2- 172.21.15.37:0/321387110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1644099fb0 0x7f164415d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f164b850640 1 --2- 172.21.15.37:0/321387110 >> v2:172.21.15.203:3300/0 conn(0x7f16440b0a30 0x7f16440cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f164b850640 1 -- 172.21.15.37:0/321387110 >> 172.21.15.37:0/321387110 conn(0x7f16440b30b0 msgr2=0x7f16440b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f164b850640 1 -- 172.21.15.37:0/321387110 shutdown_connections 2024-04-25T09:01:40.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.182+0000 7f164b850640 1 -- 172.21.15.37:0/321387110 wait complete. 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f164b850640 1 Processor -- start 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f164b850640 1 -- start start 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f164b850640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f164406f4a0 0x7f16441558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f164b850640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1644099fb0 0x7f16440c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f164b850640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16440b0a30 0x7f16440c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f164b850640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644161790 con 0x7f16440b0a30 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f164b850640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644161610 con 0x7f164406f4a0 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f164b850640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1644161910 con 0x7f1644099fb0 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1649ded640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16440b0a30 0x7f16440c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1649ded640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16440b0a30 0x7f16440c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51820/0 (socket says 172.21.15.37:51820) 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1649ded640 1 -- 172.21.15.37:0/477613421 learned_addr learned my addr 172.21.15.37:0/477613421 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f16495ec640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f164406f4a0 0x7f16441558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1648deb640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1644099fb0 0x7f16440c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1649ded640 1 -- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1644099fb0 msgr2=0x7f16440c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1649ded640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1644099fb0 0x7f16440c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1649ded640 1 -- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f164406f4a0 msgr2=0x7f16441558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1649ded640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f164406f4a0 0x7f16441558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1649ded640 1 -- 172.21.15.37:0/477613421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16441563c0 con 0x7f16440b0a30 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1648deb640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1644099fb0 0x7f16440c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1649ded640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16440b0a30 0x7f16440c55f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f163c007e70 tx=0x7f163c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1639ffb640 1 -- 172.21.15.37:0/477613421 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c014620 con 0x7f16440b0a30 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1639ffb640 1 -- 172.21.15.37:0/477613421 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c019070 con 0x7f16440b0a30 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f164b850640 1 -- 172.21.15.37:0/477613421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16440c7d80 con 0x7f16440b0a30 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f1639ffb640 1 -- 172.21.15.37:0/477613421 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c014a20 con 0x7f16440b0a30 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f164b850640 1 -- 172.21.15.37:0/477613421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164415ebb0 con 0x7f16440b0a30 2024-04-25T09:01:40.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.186+0000 7f164b850640 1 -- 172.21.15.37:0/477613421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164415eee0 con 0x7f16440b0a30 2024-04-25T09:01:40.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.190+0000 7f1639ffb640 1 -- 172.21.15.37:0/477613421 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f163c014bc0 con 0x7f16440b0a30 2024-04-25T09:01:40.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.190+0000 7f1639ffb640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1628066f60 0x7f1628069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.190+0000 7f1639ffb640 1 -- 172.21.15.37:0/477613421 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f163c085a50 con 0x7f16440b0a30 2024-04-25T09:01:40.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.190+0000 7f16495ec640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1628066f60 0x7f1628069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.190+0000 7f16495ec640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1628066f60 0x7f1628069420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f164415a120 tx=0x7f1640008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:40.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.194+0000 7f1639ffb640 1 -- 172.21.15.37:0/477613421 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f16441563c0 con 0x7f16440b0a30 2024-04-25T09:01:40.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.410+0000 7f164b850640 1 -- 172.21.15.37:0/477613421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f164404efa0 con 0x7f16440b0a30 2024-04-25T09:01:40.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.414+0000 7f1639ffb640 1 -- 172.21.15.37:0/477613421 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f163c056960 con 0x7f16440b0a30 2024-04-25T09:01:40.403 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T09:01:40.404 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA== 2024-04-25T09:01:40.404 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-04-25T09:01:40.404 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-04-25T09:01:40.404 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:01:40.404 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 -- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1628066f60 msgr2=0x7f1628069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1628066f60 0x7f1628069420 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f164415a120 tx=0x7f1640008040 comp rx=0 tx=0).stop 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 -- 172.21.15.37:0/477613421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16440b0a30 msgr2=0x7f16440c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16440b0a30 0x7f16440c55f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f163c007e70 tx=0x7f163c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 -- 172.21.15.37:0/477613421 shutdown_connections 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1628066f60 0x7f1628069420 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16440b0a30 0x7f16440c55f0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1644099fb0 0x7f16440c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 --2- 172.21.15.37:0/477613421 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f164406f4a0 0x7f16441558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 -- 172.21.15.37:0/477613421 >> 172.21.15.37:0/477613421 conn(0x7f16440b30b0 msgr2=0x7f16440b2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 -- 172.21.15.37:0/477613421 shutdown_connections 2024-04-25T09:01:40.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.418+0000 7f164b850640 1 -- 172.21.15.37:0/477613421 wait complete. 2024-04-25T09:01:40.424 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:40.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140ed75640 1 Processor -- start 2024-04-25T09:01:40.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140ed75640 1 -- start start 2024-04-25T09:01:40.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140ed75640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f140815b920 0x7f1408099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140ed75640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1408099fb0 0x7f14080cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140ed75640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f140809a3c0 con 0x7f14080b0440 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140ed75640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14080cd680 con 0x7f140815b920 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140ed75640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140809a950 con 0x7f1408099fb0 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f1407fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f14080b0440 0x7f14080aeb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40970/0 (socket says 172.21.15.37:40970) 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140cb11640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f140815b920 0x7f1408099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f1407fff640 1 -- 172.21.15.37:0/1884413371 learned_addr learned my addr 172.21.15.37:0/1884413371 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140d312640 1 --2- 172.21.15.37:0/1884413371 >> v2:172.21.15.203:3300/0 conn(0x7f1408099fb0 0x7f14080cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140cb11640 1 -- 172.21.15.37:0/1884413371 >> v2:172.21.15.203:3300/0 conn(0x7f1408099fb0 msgr2=0x7f14080cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140cb11640 1 --2- 172.21.15.37:0/1884413371 >> v2:172.21.15.203:3300/0 conn(0x7f1408099fb0 0x7f14080cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140cb11640 1 -- 172.21.15.37:0/1884413371 >> v1:172.21.15.37:6789/0 conn(0x7f14080b0440 legacy=0x7f14080aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f14077fe640 1 -- 172.21.15.37:0/1884413371 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553216950 0 0) 0x7f13fc001120 con 0x7f14080b0440 2024-04-25T09:01:40.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.562+0000 7f140cb11640 1 -- 172.21.15.37:0/1884413371 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140809ac30 con 0x7f140815b920 2024-04-25T09:01:40.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f140cb11640 1 --2- 172.21.15.37:0/1884413371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140815b920 0x7f1408099a70 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f1408072910 tx=0x7f13f801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d96d1f0bb10dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:40.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f14077fe640 1 -- 172.21.15.37:0/1884413371 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f80b2070 con 0x7f140815b920 2024-04-25T09:01:40.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f14077fe640 1 -- 172.21.15.37:0/1884413371 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f80a58e0 con 0x7f140815b920 2024-04-25T09:01:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f14077fe640 1 -- 172.21.15.37:0/1884413371 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f80a5cb0 con 0x7f140815b920 2024-04-25T09:01:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f140ed75640 1 -- 172.21.15.37:0/1884413371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140815b920 msgr2=0x7f1408099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:40.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f140ed75640 1 --2- 172.21.15.37:0/1884413371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140815b920 0x7f1408099a70 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f1408072910 tx=0x7f13f801c930 comp rx=0 tx=0).stop 2024-04-25T09:01:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f140ed75640 1 -- 172.21.15.37:0/1884413371 shutdown_connections 2024-04-25T09:01:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f140ed75640 1 --2- 172.21.15.37:0/1884413371 >> v2:172.21.15.203:3300/0 conn(0x7f1408099fb0 0x7f14080cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f140ed75640 1 --2- 172.21.15.37:0/1884413371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f140815b920 0x7f1408099a70 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f140ed75640 1 -- 172.21.15.37:0/1884413371 >> 172.21.15.37:0/1884413371 conn(0x7f14080b3010 msgr2=0x7f14080b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f140ed75640 1 -- 172.21.15.37:0/1884413371 shutdown_connections 2024-04-25T09:01:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f140ed75640 1 -- 172.21.15.37:0/1884413371 wait complete. 2024-04-25T09:01:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f140ed75640 1 Processor -- start 2024-04-25T09:01:40.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.566+0000 7f140ed75640 1 -- start start 2024-04-25T09:01:40.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f140ed75640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1408099fb0 0x7f14081539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f140ed75640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080b0440 0x7f14081553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f140ed75640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140815b920 0x7f14080b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f140ed75640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14080afc20 con 0x7f14080b0440 2024-04-25T09:01:40.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f140ed75640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14080afaa0 con 0x7f1408099fb0 2024-04-25T09:01:40.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f140ed75640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14080afda0 con 0x7f140815b920 2024-04-25T09:01:40.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f1407fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080b0440 0x7f14081553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f1407fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080b0440 0x7f14081553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51838/0 (socket says 172.21.15.37:51838) 2024-04-25T09:01:40.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f1407fff640 1 -- 172.21.15.37:0/3789469353 learned_addr learned my addr 172.21.15.37:0/3789469353 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:40.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f140d312640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140815b920 0x7f14080b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f140cb11640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1408099fb0 0x7f14081539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f1407fff640 1 -- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140815b920 msgr2=0x7f14080b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f1407fff640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140815b920 0x7f14080b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f1407fff640 1 -- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1408099fb0 msgr2=0x7f14081539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f1407fff640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1408099fb0 0x7f14081539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f1407fff640 1 -- 172.21.15.37:0/3789469353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1408158090 con 0x7f14080b0440 2024-04-25T09:01:40.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f1407fff640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080b0440 0x7f14081553a0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00c4b0 tx=0x7f13fc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f14057fa640 1 -- 172.21.15.37:0/3789469353 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc018070 con 0x7f14080b0440 2024-04-25T09:01:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f14057fa640 1 -- 172.21.15.37:0/3789469353 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13fc007500 con 0x7f14080b0440 2024-04-25T09:01:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f140ed75640 1 -- 172.21.15.37:0/3789469353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14080c85f0 con 0x7f14080b0440 2024-04-25T09:01:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f14057fa640 1 -- 172.21.15.37:0/3789469353 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc004a30 con 0x7f14080b0440 2024-04-25T09:01:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f140ed75640 1 -- 172.21.15.37:0/3789469353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1408158300 con 0x7f14080b0440 2024-04-25T09:01:40.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f140ed75640 1 -- 172.21.15.37:0/3789469353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f140804efa0 con 0x7f14080b0440 2024-04-25T09:01:40.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f14057fa640 1 -- 172.21.15.37:0/3789469353 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f13fc01b070 con 0x7f14080b0440 2024-04-25T09:01:40.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.570+0000 7f14057fa640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13f4066f60 0x7f13f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:40.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.574+0000 7f14057fa640 1 -- 172.21.15.37:0/3789469353 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13fc084500 con 0x7f14080b0440 2024-04-25T09:01:40.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.574+0000 7f140cb11640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13f4066f60 0x7f13f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:40.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.574+0000 7f140cb11640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13f4066f60 0x7f13f4069420 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f13f80af000 tx=0x7f13f80a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:40.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.578+0000 7f14057fa640 1 -- 172.21.15.37:0/3789469353 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f140809a3c0 con 0x7f14080b0440 2024-04-25T09:01:40.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.774+0000 7f140ed75640 1 -- 172.21.15.37:0/3789469353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14081586a0 con 0x7f14080b0440 2024-04-25T09:01:40.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.774+0000 7f14057fa640 1 -- 172.21.15.37:0/3789469353 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f13fc055380 con 0x7f14080b0440 2024-04-25T09:01:40.763 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:40.764 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:01:37.945874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8727":{"gid":8727,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1266122577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1266122577},{"type":"v1","addr":"172.21.15.37:6835","nonce":1266122577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:01:40.764 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 81 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 -- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13f4066f60 msgr2=0x7f13f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13f4066f60 0x7f13f4069420 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f13f80af000 tx=0x7f13f80a7070 comp rx=0 tx=0).stop 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 -- 172.21.15.37:0/3789469353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080b0440 msgr2=0x7f14081553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080b0440 0x7f14081553a0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00c4b0 tx=0x7f13fc00c970 comp rx=0 tx=0).stop 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 -- 172.21.15.37:0/3789469353 shutdown_connections 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f13f4066f60 0x7f13f4069420 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f140815b920 0x7f14080b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14080b0440 0x7f14081553a0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 --2- 172.21.15.37:0/3789469353 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1408099fb0 0x7f14081539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 -- 172.21.15.37:0/3789469353 >> 172.21.15.37:0/3789469353 conn(0x7f14080b3010 msgr2=0x7f14080b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 -- 172.21.15.37:0/3789469353 shutdown_connections 2024-04-25T09:01:40.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:40.778+0000 7f140ed75640 1 -- 172.21.15.37:0/3789469353 wait complete. 2024-04-25T09:01:41.185 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:01:34.915809+0000', 'modified': '2024-04-25T09:01:37.945874+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8727}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8727': {'gid': 8727, 'name': 'c', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/1266122577', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 1266122577}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 1266122577}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:01:41.185 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:01:41.185 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:01:41.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.326+0000 7f8d80f75640 1 Processor -- start 2024-04-25T09:01:41.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.326+0000 7f8d80f75640 1 -- start start 2024-04-25T09:01:41.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.326+0000 7f8d80f75640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0b1ac0 0x7f8d7c0cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.326+0000 7f8d80f75640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7c0afb20 con 0x7f8d7c06b820 2024-04-25T09:01:41.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.326+0000 7f8d80f75640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d7c0afd90 con 0x7f8d7c06b440 2024-04-25T09:01:41.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.326+0000 7f8d80f75640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0b06c0 con 0x7f8d7c0b1ac0 2024-04-25T09:01:41.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.326+0000 7f8d73fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8d7c06b820 0x7f8d7c0cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40976/0 (socket says 172.21.15.37:40976) 2024-04-25T09:01:41.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.326+0000 7f8d73fff640 1 -- 172.21.15.37:0/3629106695 learned_addr learned my addr 172.21.15.37:0/3629106695 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d7ad9d640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0b1ac0 0x7f8d7c0cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 815117920 0 0) 0x7f8d7c0afb20 con 0x7f8d7c06b820 2024-04-25T09:01:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d68003530 con 0x7f8d7c06b820 2024-04-25T09:01:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 888483428 0 0) 0x7f8d68003530 con 0x7f8d7c06b820 2024-04-25T09:01:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0b1ac0 msgr2=0x7f8d7c0cfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0b1ac0 0x7f8d7c0cfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 >> v1:172.21.15.203:6789/0 conn(0x7f8d7c06b440 legacy=0x7f8d7c0b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c0b0070 con 0x7f8d7c06b820 2024-04-25T09:01:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f8d64001680 con 0x7f8d7c06b820 2024-04-25T09:01:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 >> v1:172.21.15.37:6789/0 conn(0x7f8d7c06b820 legacy=0x7f8d7c0cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68003a60 0x7f8d68003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d68004810 0x7f8d68004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0b1ac0 0x7f8d680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0b06c0 con 0x7f8d68004810 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6800bb00 con 0x7f8d68003a60 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6800bd00 con 0x7f8d7c0b1ac0 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d73fff640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d68004810 0x7f8d68004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68003a60 0x7f8d68003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d7ad9d640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0b1ac0 0x7f8d680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d73fff640 1 -- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0b1ac0 msgr2=0x7f8d680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d73fff640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0b1ac0 0x7f8d680051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d73fff640 1 -- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68003a60 msgr2=0x7f8d68003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d73fff640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68003a60 0x7f8d68003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d73fff640 1 -- 172.21.15.37:0/3629106695 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68003530 con 0x7f8d68004810 2024-04-25T09:01:41.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68003a60 0x7f8d68003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:41.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d73fff640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d68004810 0x7f8d68004c90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c06bc20 tx=0x7f8d6401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6de7cf1f67520b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64007d40 con 0x7f8d68004810 2024-04-25T09:01:41.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d640ac070 con 0x7f8d68004810 2024-04-25T09:01:41.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d79d9b640 1 -- 172.21.15.37:0/3629106695 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64005b20 con 0x7f8d68004810 2024-04-25T09:01:41.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d80f75640 1 -- 172.21.15.37:0/3629106695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d68004810 msgr2=0x7f8d68004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d80f75640 1 --2- 172.21.15.37:0/3629106695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d68004810 0x7f8d68004c90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c06bc20 tx=0x7f8d6401f3d0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d7ad9d640 1 -- 172.21.15.37:0/3629106695 reap_dead start 2024-04-25T09:01:41.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d80f75640 1 -- 172.21.15.37:0/3629106695 shutdown_connections 2024-04-25T09:01:41.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.330+0000 7f8d80f75640 1 -- 172.21.15.37:0/3629106695 >> 172.21.15.37:0/3629106695 conn(0x7f8d7c0b1f10 msgr2=0x7f8d7c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:41.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 -- 172.21.15.37:0/3629106695 shutdown_connections 2024-04-25T09:01:41.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 -- 172.21.15.37:0/3629106695 wait complete. 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 Processor -- start 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 -- start start 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68004810 0x7f8d7c0c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d7c09c980 0x7f8d7c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0ad960 0x7f8d7c0adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0cfee0 con 0x7f8d7c09c980 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0cfd60 con 0x7f8d68004810 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c0d0060 con 0x7f8d7c0ad960 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d73fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d7c09c980 0x7f8d7c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d73fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d7c09c980 0x7f8d7c15b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51842/0 (socket says 172.21.15.37:51842) 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d73fff640 1 -- 172.21.15.37:0/2671402312 learned_addr learned my addr 172.21.15.37:0/2671402312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d7ad9d640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0ad960 0x7f8d7c0adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68004810 0x7f8d7c0c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d73fff640 1 -- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0ad960 msgr2=0x7f8d7c0adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d73fff640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0ad960 0x7f8d7c0adde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d73fff640 1 -- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68004810 msgr2=0x7f8d7c0c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d73fff640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68004810 0x7f8d7c0c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d73fff640 1 -- 172.21.15.37:0/2671402312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7c09d110 con 0x7f8d7c09c980 2024-04-25T09:01:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d73fff640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d7c09c980 0x7f8d7c15b460 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f8d6409ba20 tx=0x7f8d68005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d72ffd640 1 -- 172.21.15.37:0/2671402312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640047f0 con 0x7f8d7c09c980 2024-04-25T09:01:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d72ffd640 1 -- 172.21.15.37:0/2671402312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d64004990 con 0x7f8d7c09c980 2024-04-25T09:01:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d72ffd640 1 -- 172.21.15.37:0/2671402312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64004d40 con 0x7f8d7c09c980 2024-04-25T09:01:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 -- 172.21.15.37:0/2671402312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d7c09d3f0 con 0x7f8d7c09c980 2024-04-25T09:01:41.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.334+0000 7f8d80f75640 1 -- 172.21.15.37:0/2671402312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7c09d9b0 con 0x7f8d7c09c980 2024-04-25T09:01:41.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d72ffd640 1 -- 172.21.15.37:0/2671402312 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8d640070d0 con 0x7f8d7c09c980 2024-04-25T09:01:41.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d72ffd640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d5c066f60 0x7f8d5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d72ffd640 1 -- 172.21.15.37:0/2671402312 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d641220f0 con 0x7f8d7c09c980 2024-04-25T09:01:41.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d5c066f60 0x7f8d5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d840e2000 1 Processor -- start 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d840e2000 1 -- start start 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d5c066f60 0x7f8d5c069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005450 tx=0x7f8d6c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63318920 0x562c63318d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c632ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d840e2000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c631b65f0 con 0x562c632cef30 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d840e2000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c6311dda0 con 0x562c63319240 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d840e2000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c630a6150 con 0x562c63318920 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51858/0 (socket says 172.21.15.37:51858) 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7ad9d640 1 -- 172.21.15.37:0/3283755281 learned_addr learned my addr 172.21.15.37:0/3283755281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:41.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d73fff640 1 --2- 172.21.15.37:0/3283755281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c632ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/3283755281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63318920 0x562c63318d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7ad9d640 1 -- 172.21.15.37:0/3283755281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63318920 msgr2=0x562c63318d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7ad9d640 1 --2- 172.21.15.37:0/3283755281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63318920 0x562c63318d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7ad9d640 1 -- 172.21.15.37:0/3283755281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 msgr2=0x562c632ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7ad9d640 1 --2- 172.21.15.37:0/3283755281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c632ce9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7ad9d640 1 -- 172.21.15.37:0/3283755281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c630bfca0 con 0x562c632cef30 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d73fff640 1 --2- 172.21.15.37:0/3283755281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c632ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d7ad9d640 1 --2- 172.21.15.37:0/3283755281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632cf310 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c162690 tx=0x7f8d74009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e226c06333ba27a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d5bfff640 1 -- 172.21.15.37:0/3283755281 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7401b020 con 0x562c632cef30 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d5bfff640 1 -- 172.21.15.37:0/3283755281 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d74003c70 con 0x562c632cef30 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.338+0000 7f8d5bfff640 1 -- 172.21.15.37:0/3283755281 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74003e20 con 0x562c632cef30 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- 172.21.15.37:0/3283755281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 msgr2=0x562c632cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 --2- 172.21.15.37:0/3283755281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632cf310 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c162690 tx=0x7f8d74009750 comp rx=0 tx=0).stop 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- 172.21.15.37:0/3283755281 shutdown_connections 2024-04-25T09:01:41.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 --2- 172.21.15.37:0/3283755281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632cf310 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 --2- 172.21.15.37:0/3283755281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c632ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 --2- 172.21.15.37:0/3283755281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63318920 0x562c63318d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- 172.21.15.37:0/3283755281 >> 172.21.15.37:0/3283755281 conn(0x562c632d52a0 msgr2=0x562c632cd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- 172.21.15.37:0/3283755281 shutdown_connections 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- 172.21.15.37:0/3283755281 wait complete. 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- start start 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 Processor -- start 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c6331fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63318920 0x562c633201c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63319240 0x562c632d2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c630d8990 con 0x562c632cef30 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c63119e40 con 0x562c63318920 2024-04-25T09:01:41.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c630a9900 con 0x562c63319240 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 10 client.0 _ll_register_callbacks cb 0x562c631d11c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c6331fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c6331fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51864/0 (socket says 172.21.15.37:51864) 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d7a59c640 1 -- 172.21.15.37:0/2238500460 learned_addr learned my addr 172.21.15.37:0/2238500460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d7ad9d640 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63319240 0x562c632d2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d73fff640 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63318920 0x562c633201c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d7a59c640 1 -- 172.21.15.37:0/2238500460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63319240 msgr2=0x562c632d2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63319240 0x562c632d2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d7a59c640 1 -- 172.21.15.37:0/2238500460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63318920 msgr2=0x562c633201c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63318920 0x562c633201c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d7a59c640 1 -- 172.21.15.37:0/2238500460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c630d6630 con 0x562c632cef30 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c6331fc80 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c001b80 tx=0x7f8d6c001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6e75ca8837de1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d597fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d597fa640 1 -- 172.21.15.37:0/2238500460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c01a070 con 0x562c632cef30 2024-04-25T09:01:41.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d597fa640 1 -- 172.21.15.37:0/2238500460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d6c001bb0 con 0x562c632cef30 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- 172.21.15.37:0/2238500460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d74003c70 con 0x562c632cef30 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d597fa640 1 -- 172.21.15.37:0/2238500460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d6c02c410 con 0x562c632cef30 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 10 client.8868 fetch_fsmap learned FSMap version 81 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- 172.21.15.37:0/2238500460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x562c630a63a0 con 0x562c632cef30 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d597fa640 1 -- 172.21.15.37:0/2238500460 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d74003c70 con 0x562c632cef30 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d597fa640 1 -- 172.21.15.37:0/2238500460 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8d6c020070 con 0x562c632cef30 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 10 client.8868 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 10 client.8868 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 20 client.8868 populate_metadata read hostname 'smithi037' 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x562c632d4160 0x562c63329ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 4 client.8868 mds_command: new command op to 8727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d840e2000 1 -- 172.21.15.37:0/2238500460 --> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c630b9a50 con 0x562c632d4160 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d73fff640 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x562c632d4160 0x562c63329ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d73fff640 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x562c632d4160 0x562c63329ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=544be6c2b507da6f server_cookie=20530c635758e212 in_seq=0 out_seq=0 2024-04-25T09:01:41.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.342+0000 7f8d597fa640 10 client.8868 ms_handle_connect on v2:172.21.15.37:6834/1266122577 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d597fa640 1 -- 172.21.15.37:0/2238500460 <== mds.0 v2:172.21.15.37:6834/1266122577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562c630b9a50 con 0x562c632d4160 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d597fa640 10 client.8868 handle_command_reply: tid=0 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 1 client.8868 shutdown 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 1 -- 172.21.15.37:0/2238500460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 msgr2=0x562c6331fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c6331fc80 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c001b80 tx=0x7f8d6c001130 comp rx=0 tx=0).stop 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 1 -- 172.21.15.37:0/2238500460 shutdown_connections 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x562c632d4160 0x562c63329ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63319240 0x562c632d2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d597fa640 0 client.8868 ms_handle_reset on v2:172.21.15.37:6834/1266122577 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63318920 0x562c633201c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 1 --2- 172.21.15.37:0/2238500460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c6331fc80 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 1 -- 172.21.15.37:0/2238500460 >> 172.21.15.37:0/2238500460 conn(0x562c632d52a0 msgr2=0x562c632cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 1 -- 172.21.15.37:0/2238500460 shutdown_connections 2024-04-25T09:01:41.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 1 -- 172.21.15.37:0/2238500460 wait complete. 2024-04-25T09:01:41.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.346+0000 7f8d840e2000 20 client.8868 trim_cache size 0 max 16384 2024-04-25T09:01:41.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 Processor -- start 2024-04-25T09:01:41.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 -- start start 2024-04-25T09:01:41.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632d22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63318920 0x562c632d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c63321830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c1ffb10 con 0x562c632cef30 2024-04-25T09:01:41.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c1ff060 con 0x562c63319240 2024-04-25T09:01:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7c1ff510 con 0x562c63318920 2024-04-25T09:01:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632d22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632d22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51880/0 (socket says 172.21.15.37:51880) 2024-04-25T09:01:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d7a59c640 1 -- 172.21.15.37:0/1719746672 learned_addr learned my addr 172.21.15.37:0/1719746672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d73fff640 1 --2- 172.21.15.37:0/1719746672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63318920 0x562c632d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d7ad9d640 1 --2- 172.21.15.37:0/1719746672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c63321830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d7a59c640 1 -- 172.21.15.37:0/1719746672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63318920 msgr2=0x562c632d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/1719746672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63318920 0x562c632d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d7a59c640 1 -- 172.21.15.37:0/1719746672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 msgr2=0x562c63321830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/1719746672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c63321830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d7a59c640 1 -- 172.21.15.37:0/1719746672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c63319070 con 0x562c632cef30 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d73fff640 1 --2- 172.21.15.37:0/1719746672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63318920 0x562c632d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d7ad9d640 1 --2- 172.21.15.37:0/1719746672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c63321830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/1719746672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632d22c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c021460 tx=0x7f8d6c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5f454f00032b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d5b7fe640 1 -- 172.21.15.37:0/1719746672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c039070 con 0x562c632cef30 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d5b7fe640 1 -- 172.21.15.37:0/1719746672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c014ba0 con 0x562c632cef30 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d5b7fe640 1 -- 172.21.15.37:0/1719746672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c007030 con 0x562c632cef30 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 -- 172.21.15.37:0/1719746672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 msgr2=0x562c632d22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 --2- 172.21.15.37:0/1719746672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632d22c0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c021460 tx=0x7f8d6c016000 comp rx=0 tx=0).stop 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 -- 172.21.15.37:0/1719746672 shutdown_connections 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 --2- 172.21.15.37:0/1719746672 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c63321830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 --2- 172.21.15.37:0/1719746672 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c63318920 0x562c632d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 --2- 172.21.15.37:0/1719746672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c632cef30 0x562c632d22c0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 -- 172.21.15.37:0/1719746672 >> 172.21.15.37:0/1719746672 conn(0x562c632d52a0 msgr2=0x562c632d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:41.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 -- 172.21.15.37:0/1719746672 shutdown_connections 2024-04-25T09:01:41.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 -- 172.21.15.37:0/1719746672 wait complete. 2024-04-25T09:01:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 -- start start 2024-04-25T09:01:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.362+0000 7f8d840e2000 1 Processor -- start 2024-04-25T09:01:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c632cef30 0x562c6335cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c63318920 0x562c6335d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c6335d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c631223d0 con 0x562c63318920 2024-04-25T09:01:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c631d9d50 con 0x562c63319240 2024-04-25T09:01:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x562c630b8980 con 0x562c632cef30 2024-04-25T09:01:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 10 client.0 _ll_register_callbacks cb 0x7f8d74009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:41.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d73fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c63318920 0x562c6335d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d73fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c63318920 0x562c6335d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51894/0 (socket says 172.21.15.37:51894) 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d73fff640 1 -- 172.21.15.37:0/4034558247 learned_addr learned my addr 172.21.15.37:0/4034558247 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c632cef30 0x562c6335cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d7ad9d640 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c6335d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d73fff640 1 -- 172.21.15.37:0/4034558247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c632cef30 msgr2=0x562c6335cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d73fff640 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c632cef30 0x562c6335cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d73fff640 1 -- 172.21.15.37:0/4034558247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 msgr2=0x562c6335d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d73fff640 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c6335d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d73fff640 1 -- 172.21.15.37:0/4034558247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562c6319f600 con 0x562c63318920 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d73fff640 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c63318920 0x562c6335d3c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f8d640f2800 tx=0x7f8d640011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3deb9dd57808629 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d597fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d597fa640 1 -- 172.21.15.37:0/4034558247 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640f69c0 con 0x562c63318920 2024-04-25T09:01:41.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d597fa640 1 -- 172.21.15.37:0/4034558247 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d640f0b70 con 0x562c63318920 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 -- 172.21.15.37:0/4034558247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d6c014ba0 con 0x562c63318920 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d597fa640 1 -- 172.21.15.37:0/4034558247 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d640faf80 con 0x562c63318920 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 10 client.8874 fetch_fsmap learned FSMap version 81 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 -- 172.21.15.37:0/4034558247 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x562c630b29b0 con 0x562c63318920 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d597fa640 1 -- 172.21.15.37:0/4034558247 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6c014ba0 con 0x562c63318920 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d597fa640 1 -- 172.21.15.37:0/4034558247 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8d640c4030 con 0x562c63318920 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 10 client.8874 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 10 client.8874 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 20 client.8874 populate_metadata read hostname 'smithi037' 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x562c632d4e50 0x562c6335de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 4 client.8874 mds_command: new command op to 8727 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 -- 172.21.15.37:0/4034558247 --> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562c633205f0 con 0x562c632d4e50 2024-04-25T09:01:41.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x562c632d4e50 0x562c6335de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d597fa640 10 client.8874 ms_handle_connect on v2:172.21.15.37:6834/1266122577 2024-04-25T09:01:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d7a59c640 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x562c632d4e50 0x562c6335de40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5af71be9beb8338c server_cookie=a6a15c6c3cf2b354 in_seq=0 out_seq=0 2024-04-25T09:01:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d597fa640 1 -- 172.21.15.37:0/4034558247 <== mds.0 v2:172.21.15.37:6834/1266122577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x562c633205f0 con 0x562c632d4e50 2024-04-25T09:01:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d597fa640 10 client.8874 handle_command_reply: tid=0 2024-04-25T09:01:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 client.8874 shutdown 2024-04-25T09:01:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 -- 172.21.15.37:0/4034558247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c63318920 msgr2=0x562c6335d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c63318920 0x562c6335d3c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f8d640f2800 tx=0x7f8d640011a0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 -- 172.21.15.37:0/4034558247 shutdown_connections 2024-04-25T09:01:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x562c63319240 0x562c6335d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562c63318920 0x562c6335d3c0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x562c632d4e50 0x562c6335de40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 --2- 172.21.15.37:0/4034558247 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x562c632cef30 0x562c6335cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d597fa640 0 client.8874 ms_handle_reset on v2:172.21.15.37:6834/1266122577 2024-04-25T09:01:41.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 -- 172.21.15.37:0/4034558247 >> 172.21.15.37:0/4034558247 conn(0x562c632d52a0 msgr2=0x562c632d22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:41.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 -- 172.21.15.37:0/4034558247 shutdown_connections 2024-04-25T09:01:41.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 1 -- 172.21.15.37:0/4034558247 wait complete. 2024-04-25T09:01:41.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d840e2000 20 client.8874 trim_cache size 0 max 16384 2024-04-25T09:01:41.363 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:01:41.363 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T09:01:41.363 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T09:01:41.363 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 8727, 2024-04-25T09:01:41.363 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T09:01:41.364 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T09:01:41.364 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T09:01:41.364 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 5.7989105319999998, 2024-04-25T09:01:41.364 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 81, 2024-04-25T09:01:41.364 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 110, 2024-04-25T09:01:41.364 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 110, 2024-04-25T09:01:41.364 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 10.812300226 2024-04-25T09:01:41.364 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:01:41.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 -- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d5c066f60 msgr2=0x7f8d5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d5c066f60 0x7f8d5c069420 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005450 tx=0x7f8d6c009040 comp rx=0 tx=0).stop 2024-04-25T09:01:41.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 -- 172.21.15.37:0/2671402312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d7c09c980 msgr2=0x7f8d7c15b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d7c09c980 0x7f8d7c15b460 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f8d6409ba20 tx=0x7f8d68005840 comp rx=0 tx=0).stop 2024-04-25T09:01:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 -- 172.21.15.37:0/2671402312 shutdown_connections 2024-04-25T09:01:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8d5c066f60 0x7f8d5c069420 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8d7c0ad960 0x7f8d7c0adde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8d7c09c980 0x7f8d7c15b460 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 --2- 172.21.15.37:0/2671402312 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8d68004810 0x7f8d7c0c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 -- 172.21.15.37:0/2671402312 >> 172.21.15.37:0/2671402312 conn(0x7f8d7c0b1f10 msgr2=0x7f8d7c161cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 -- 172.21.15.37:0/2671402312 shutdown_connections 2024-04-25T09:01:41.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.366+0000 7f8d80f75640 1 -- 172.21.15.37:0/2671402312 wait complete. 2024-04-25T09:01:41.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:01:41.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.494+0000 7f34c1fec640 1 Processor -- start 2024-04-25T09:01:41.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34c1fec640 1 -- start start 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34c1fec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34bc06ac90 0x7f34bc06b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34c1fec640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f34bc0cc130 0x7f34bc0cf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34c1fec640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34bc06b5b0 con 0x7f34bc15ca90 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34c1fec640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34bc15ff70 con 0x7f34bc0cc130 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34c1fec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc15f360 con 0x7f34bc06ac90 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34bbfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f34bc0cc130 0x7f34bc0cf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34baffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f34bc15ca90 0x7f34bc0d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40986/0 (socket says 172.21.15.37:40986) 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34baffd640 1 -- 172.21.15.37:0/3355706861 learned_addr learned my addr 172.21.15.37:0/3355706861 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34bb7fe640 1 --2- 172.21.15.37:0/3355706861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34bc06ac90 0x7f34bc06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34bbfff640 1 -- 172.21.15.37:0/3355706861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34bc06ac90 msgr2=0x7f34bc06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34bbfff640 1 --2- 172.21.15.37:0/3355706861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34bc06ac90 0x7f34bc06b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34bbfff640 1 -- 172.21.15.37:0/3355706861 >> v1:172.21.15.37:6789/0 conn(0x7f34bc15ca90 legacy=0x7f34bc0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34ba7fc640 1 -- 172.21.15.37:0/3355706861 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524863235 0 0) 0x7f34bc06b5b0 con 0x7f34bc15ca90 2024-04-25T09:01:41.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34bbfff640 1 -- 172.21.15.37:0/3355706861 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc15f640 con 0x7f34bc0cc130 2024-04-25T09:01:41.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34bbfff640 1 --2- 172.21.15.37:0/3355706861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34bc0cc130 0x7f34bc0cf2f0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f34a40097c0 tx=0x7f34a401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89c48c6766dfb25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34ba7fc640 1 -- 172.21.15.37:0/3355706861 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a40a7070 con 0x7f34bc0cc130 2024-04-25T09:01:41.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34ba7fc640 1 -- 172.21.15.37:0/3355706861 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a40b0070 con 0x7f34bc0cc130 2024-04-25T09:01:41.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.498+0000 7f34ba7fc640 1 -- 172.21.15.37:0/3355706861 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a40ab890 con 0x7f34bc0cc130 2024-04-25T09:01:41.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 -- 172.21.15.37:0/3355706861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34bc0cc130 msgr2=0x7f34bc0cf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 --2- 172.21.15.37:0/3355706861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34bc0cc130 0x7f34bc0cf2f0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f34a40097c0 tx=0x7f34a401c470 comp rx=0 tx=0).stop 2024-04-25T09:01:41.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 -- 172.21.15.37:0/3355706861 shutdown_connections 2024-04-25T09:01:41.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 --2- 172.21.15.37:0/3355706861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34bc0cc130 0x7f34bc0cf2f0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 --2- 172.21.15.37:0/3355706861 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34bc06ac90 0x7f34bc06b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 -- 172.21.15.37:0/3355706861 >> 172.21.15.37:0/3355706861 conn(0x7f34bc099a70 msgr2=0x7f34bc0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:41.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 -- 172.21.15.37:0/3355706861 shutdown_connections 2024-04-25T09:01:41.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 -- 172.21.15.37:0/3355706861 wait complete. 2024-04-25T09:01:41.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 Processor -- start 2024-04-25T09:01:41.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 -- start start 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34bc06ac90 0x7f34bc152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34bc0cc130 0x7f34bc159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34bc15ca90 0x7f34bc14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc159750 con 0x7f34bc15ca90 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc1595d0 con 0x7f34bc0cc130 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34bc0d0040 con 0x7f34bc06ac90 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34bc15ca90 0x7f34bc14e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34bc15ca90 0x7f34bc14e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51916/0 (socket says 172.21.15.37:51916) 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34bbfff640 1 -- 172.21.15.37:0/3454767179 learned_addr learned my addr 172.21.15.37:0/3454767179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34baffd640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34bc0cc130 0x7f34bc159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34bb7fe640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34bc06ac90 0x7f34bc152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34bbfff640 1 -- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34bc06ac90 msgr2=0x7f34bc152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34bbfff640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34bc06ac90 0x7f34bc152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34bbfff640 1 -- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34bc0cc130 msgr2=0x7f34bc159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34bbfff640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34bc0cc130 0x7f34bc159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34bbfff640 1 -- 172.21.15.37:0/3454767179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc0b3330 con 0x7f34bc15ca90 2024-04-25T09:01:41.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34bbfff640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34bc15ca90 0x7f34bc14e9c0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f34a40097c0 tx=0x7f34a401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34b3fff640 1 -- 172.21.15.37:0/3454767179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a40bf610 con 0x7f34bc15ca90 2024-04-25T09:01:41.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34b3fff640 1 -- 172.21.15.37:0/3454767179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a40bf7b0 con 0x7f34bc15ca90 2024-04-25T09:01:41.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34b3fff640 1 -- 172.21.15.37:0/3454767179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a40bfb80 con 0x7f34bc15ca90 2024-04-25T09:01:41.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 -- 172.21.15.37:0/3454767179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34bc0b3610 con 0x7f34bc15ca90 2024-04-25T09:01:41.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.502+0000 7f34c1fec640 1 -- 172.21.15.37:0/3454767179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34bc0b3bd0 con 0x7f34bc15ca90 2024-04-25T09:01:41.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.506+0000 7f34c1fec640 1 -- 172.21.15.37:0/3454767179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34bc04efa0 con 0x7f34bc15ca90 2024-04-25T09:01:41.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.506+0000 7f34b3fff640 1 -- 172.21.15.37:0/3454767179 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f34a40bd020 con 0x7f34bc15ca90 2024-04-25T09:01:41.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.506+0000 7f34b3fff640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f34a0066f60 0x7f34a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.506+0000 7f34b3fff640 1 -- 172.21.15.37:0/3454767179 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f34a411fdb0 con 0x7f34bc15ca90 2024-04-25T09:01:41.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.506+0000 7f34bb7fe640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f34a0066f60 0x7f34a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.506+0000 7f34bb7fe640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f34a0066f60 0x7f34a0069420 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f34b400b9d0 tx=0x7f34b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.510+0000 7f34b3fff640 1 -- 172.21.15.37:0/3454767179 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f34a40c6070 con 0x7f34bc15ca90 2024-04-25T09:01:41.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.698+0000 7f34c1fec640 1 -- 172.21.15.37:0/3454767179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f34bc06b7c0 con 0x7f34bc15ca90 2024-04-25T09:01:41.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.698+0000 7f34b3fff640 1 -- 172.21.15.37:0/3454767179 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) v1 ==== 53+0+73 (secure 0 0 0) 0x7f34a4016090 con 0x7f34bc15ca90 2024-04-25T09:01:41.688 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:01:41.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 -- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f34a0066f60 msgr2=0x7f34a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f34a0066f60 0x7f34a0069420 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f34b400b9d0 tx=0x7f34b4008040 comp rx=0 tx=0).stop 2024-04-25T09:01:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 -- 172.21.15.37:0/3454767179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34bc15ca90 msgr2=0x7f34bc14e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34bc15ca90 0x7f34bc14e9c0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f34a40097c0 tx=0x7f34a401acb0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 -- 172.21.15.37:0/3454767179 shutdown_connections 2024-04-25T09:01:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f34a0066f60 0x7f34a0069420 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34bc15ca90 0x7f34bc14e9c0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34bc0cc130 0x7f34bc159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 --2- 172.21.15.37:0/3454767179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34bc06ac90 0x7f34bc152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 -- 172.21.15.37:0/3454767179 >> 172.21.15.37:0/3454767179 conn(0x7f34bc099a70 msgr2=0x7f34bc15d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 -- 172.21.15.37:0/3454767179 shutdown_connections 2024-04-25T09:01:41.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:41.702+0000 7f34c1fec640 1 -- 172.21.15.37:0/3454767179 wait complete. 2024-04-25T09:01:41.704 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:01:41.704 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:01:41.704 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T09:01:41.704 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:01:41.704 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:41.705 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:01:41.705 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:01:41.705 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:01:41.705 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:01:41.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:41.705 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T09:01:41.710 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:01:41.710 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:01:41.710 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:01:41.710 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:41.710 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T09:01:41.710 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:41.710 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:01:41.710 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41939sec preferred_lft 41939sec 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T09:01:41.711 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:41.712 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:01:41.713 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:41.713 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:01:41.713 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:41.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:41.713 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:01:41.764 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T09:01:41.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:41.765 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T09:01:41.819 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T09:01:41.819 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:01:41.819 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T09:01:41.819 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:01:41.819 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T09:01:41.820 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:01:41.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:41.820 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T09:01:41.820 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T09:01:41.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:01:41.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:01:41.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:01:41.820 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T09:01:41.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:41.893 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T09:01:41.946 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:01:41.946 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:01:41.947 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:01:41.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:41.947 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T09:01:42.003 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:01:42.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:01:42.003 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:42.050 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:01:42.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:01:42.050 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:42.097 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:01:42.156 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:01:42.203 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:01:42.203 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:01:42.203 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:01:42.203 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:01:42.203 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:01:42.203 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:01:42.203 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:01:42.203 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:01:42.203 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:01:42.203 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:01:42.204 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:01:42.205 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:01:42.205 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:01:42.205 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:01:42.205 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:01:42.205 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:01:42.205 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:42.205 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:42.205 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:42.205 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:42.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:42.206 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:01:42.254 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:01:42.255 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:01:42.299 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:01:42.314 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:01:42.362 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:01:42.362 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:01:42.362 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:01:42.362 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:01:42.362 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:01:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:01:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:01:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:01:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:01:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:01:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:01:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:01:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:01:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:01:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:01:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:01:42.364 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:01:42.365 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:01:42.365 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:42.365 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:42.365 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:42.365 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:42.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:42.366 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:01:42.377 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T09:01:42.390+0000 7f0d8ca54580 -1 init, newargv = 0x561cb60bcd70 newargc=15 2024-04-25T09:01:42.378 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[51611]: starting ceph client 2024-04-25T09:01:42.384 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[51611]: starting fuse 2024-04-25T09:01:43.483 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:01:43.501 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:01:43.556 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:01:43.557 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:01:43.558 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:01:43.559 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:43.559 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:43.559 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:43.559 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:43.559 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:43.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:43.560 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:01:43.606 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T09:01:43.607 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T09:01:43.607 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T09:01:43.608 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:01:43.733 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.51611.asok 2024-04-25T09:01:43.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:01:43.745 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.51611.asok 2024-04-25T09:01:43.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:43.746 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.51611.asok status 2024-04-25T09:01:43.905 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:01:43.906 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T09:01:43.906 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:01:43.906 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:01:43.906 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T09:01:43.906 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T09:01:43.906 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:01:43.906 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "51611", 2024-04-25T09:01:43.906 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T09:01:43.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:43.906 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 8889, 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 8889 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1736919475 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:43.907 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1736919475 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.8889 192.168.0.1:0/1736919475", 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/1736919475", 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 81, 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 110, 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T09:01:43.908 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T09:01:43.909 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:01:43.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:43.920 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T09:01:43.920 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:43.926 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T09:01:43.926 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:43.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:43.927 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:43.981 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:01:43.981 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:44.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:44.030 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T09:01:44.168 INFO:teuthology.orchestra.run.smithi037.stderr:6+0 records in 2024-04-25T09:01:44.168 INFO:teuthology.orchestra.run.smithi037.stderr:6+0 records out 2024-04-25T09:01:44.168 INFO:teuthology.orchestra.run.smithi037.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0778474 s, 80.8 MB/s 2024-04-25T09:01:44.169 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:01:44.169 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:44.175 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:44.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a7bc5640 1 Processor -- start 2024-04-25T09:01:44.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a7bc5640 1 -- start start 2024-04-25T09:01:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a7bc5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a015da10 0x7f52a0071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a7bc5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f52a0071e90 0x7f52a0071570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a7bc5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a00adbd0 con 0x7f52a00b24a0 2024-04-25T09:01:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a7bc5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a00ae860 con 0x7f52a0071e90 2024-04-25T09:01:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a7bc5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a015e210 con 0x7f52a015da10 2024-04-25T09:01:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a5961640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a015da10 0x7f52a0071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:44.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a6162640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f52a0071e90 0x7f52a0071570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:44.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a5961640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a015da10 0x7f52a0071030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:59782/0 (socket says 172.21.15.37:59782) 2024-04-25T09:01:44.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a5961640 1 -- 172.21.15.37:0/1760668878 learned_addr learned my addr 172.21.15.37:0/1760668878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:44.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a6162640 1 -- 172.21.15.37:0/1760668878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a015da10 msgr2=0x7f52a0071030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:44.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a6162640 1 --2- 172.21.15.37:0/1760668878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a015da10 0x7f52a0071030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a6162640 1 -- 172.21.15.37:0/1760668878 >> v1:172.21.15.203:6789/0 conn(0x7f52a00b24a0 legacy=0x7f52a01614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a6162640 1 -- 172.21.15.37:0/1760668878 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a015e4f0 con 0x7f52a0071e90 2024-04-25T09:01:44.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.362+0000 7f52a5961640 1 --2- 172.21.15.37:0/1760668878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a015da10 0x7f52a0071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:44.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a6162640 1 --2- 172.21.15.37:0/1760668878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a0071e90 0x7f52a0071570 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5298009d90 tx=0x7f529801c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce54007b208a78af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:44.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a495f640 1 -- 172.21.15.37:0/1760668878 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52980b2070 con 0x7f52a0071e90 2024-04-25T09:01:44.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a495f640 1 -- 172.21.15.37:0/1760668878 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52980a5740 con 0x7f52a0071e90 2024-04-25T09:01:44.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a495f640 1 -- 172.21.15.37:0/1760668878 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52980a5b10 con 0x7f52a0071e90 2024-04-25T09:01:44.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a7bc5640 1 -- 172.21.15.37:0/1760668878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a0071e90 msgr2=0x7f52a0071570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a7bc5640 1 --2- 172.21.15.37:0/1760668878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a0071e90 0x7f52a0071570 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5298009d90 tx=0x7f529801c850 comp rx=0 tx=0).stop 2024-04-25T09:01:44.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a7bc5640 1 -- 172.21.15.37:0/1760668878 shutdown_connections 2024-04-25T09:01:44.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a7bc5640 1 --2- 172.21.15.37:0/1760668878 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a0071e90 0x7f52a0071570 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a7bc5640 1 --2- 172.21.15.37:0/1760668878 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a015da10 0x7f52a0071030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a7bc5640 1 -- 172.21.15.37:0/1760668878 >> 172.21.15.37:0/1760668878 conn(0x7f52a00b11a0 msgr2=0x7f52a00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:44.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a7bc5640 1 -- 172.21.15.37:0/1760668878 shutdown_connections 2024-04-25T09:01:44.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a7bc5640 1 -- 172.21.15.37:0/1760668878 wait complete. 2024-04-25T09:01:44.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a7bc5640 1 Processor -- start 2024-04-25T09:01:44.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.366+0000 7f52a7bc5640 1 -- start start 2024-04-25T09:01:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a7bc5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a0071e90 0x7f52a00af020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:44.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a7bc5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a00b24a0 0x7f52a00af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a7bc5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a015da10 0x7f52a00afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a7bc5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a016a890 con 0x7f52a015da10 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a7bc5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a015ef00 con 0x7f52a00b24a0 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a7bc5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a016aa10 con 0x7f52a0071e90 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a6162640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a015da10 0x7f52a00afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a6162640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a015da10 0x7f52a00afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51920/0 (socket says 172.21.15.37:51920) 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a6162640 1 -- 172.21.15.37:0/561530774 learned_addr learned my addr 172.21.15.37:0/561530774 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a5160640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a00b24a0 0x7f52a00af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a5961640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a0071e90 0x7f52a00af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a6162640 1 -- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a0071e90 msgr2=0x7f52a00af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a6162640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a0071e90 0x7f52a00af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a6162640 1 -- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a00b24a0 msgr2=0x7f52a00af560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a6162640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a00b24a0 0x7f52a00af560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a6162640 1 -- 172.21.15.37:0/561530774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a014e7b0 con 0x7f52a015da10 2024-04-25T09:01:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a6162640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a015da10 0x7f52a00afaa0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f52980b0040 tx=0x7f52980195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52967fc640 1 -- 172.21.15.37:0/561530774 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52980b2050 con 0x7f52a015da10 2024-04-25T09:01:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52967fc640 1 -- 172.21.15.37:0/561530774 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52980be070 con 0x7f52a015da10 2024-04-25T09:01:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a7bc5640 1 -- 172.21.15.37:0/561530774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a014ea90 con 0x7f52a015da10 2024-04-25T09:01:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52967fc640 1 -- 172.21.15.37:0/561530774 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52980ad890 con 0x7f52a015da10 2024-04-25T09:01:44.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a7bc5640 1 -- 172.21.15.37:0/561530774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a014f050 con 0x7f52a015da10 2024-04-25T09:01:44.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a7bc5640 1 -- 172.21.15.37:0/561530774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a004efa0 con 0x7f52a015da10 2024-04-25T09:01:44.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52967fc640 1 -- 172.21.15.37:0/561530774 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f52980ada30 con 0x7f52a015da10 2024-04-25T09:01:44.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52967fc640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 0x7f5284069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:44.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52967fc640 1 -- 172.21.15.37:0/561530774 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5298120a40 con 0x7f52a015da10 2024-04-25T09:01:44.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.370+0000 7f52a5961640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 0x7f5284069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:44.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.374+0000 7f52a5961640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 0x7f5284069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f529c006cb0 tx=0x7f529c009240 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:44.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.374+0000 7f52967fc640 1 -- 172.21.15.37:0/561530774 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5298016560 con 0x7f52a015da10 2024-04-25T09:01:44.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.538+0000 7f52a7bc5640 1 -- 172.21.15.37:0/561530774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52a006c030 con 0x7f52a015da10 2024-04-25T09:01:44.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.538+0000 7f52967fc640 1 -- 172.21.15.37:0/561530774 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16848 (secure 0 0 0) 0x7f52980f14a0 con 0x7f52a015da10 2024-04-25T09:01:44.528 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:44.528 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:01:36.569159+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:33.212328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:33.895814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:44.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 -- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 msgr2=0x7f5284069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 0x7f5284069420 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f529c006cb0 tx=0x7f529c009240 comp rx=0 tx=0).stop 2024-04-25T09:01:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 -- 172.21.15.37:0/561530774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a015da10 msgr2=0x7f52a00afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a015da10 0x7f52a00afaa0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f52980b0040 tx=0x7f52980195f0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 -- 172.21.15.37:0/561530774 shutdown_connections 2024-04-25T09:01:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5284066f60 0x7f5284069420 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52a015da10 0x7f52a00afaa0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f52a00b24a0 0x7f52a00af560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 --2- 172.21.15.37:0/561530774 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52a0071e90 0x7f52a00af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 -- 172.21.15.37:0/561530774 >> 172.21.15.37:0/561530774 conn(0x7f52a00b11a0 msgr2=0x7f52a006d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 -- 172.21.15.37:0/561530774 shutdown_connections 2024-04-25T09:01:44.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.542+0000 7f52a7bc5640 1 -- 172.21.15.37:0/561530774 wait complete. 2024-04-25T09:01:44.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:01:33.212328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:01:33.895814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:01:44.548 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:01:44.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3ff6d8c640 1 Processor -- start 2024-04-25T09:01:44.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3ff6d8c640 1 -- start start 2024-04-25T09:01:44.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3ff6d8c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3ff00cf590 0x7f3ff00cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:44.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3ff6d8c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff00cbd20 con 0x7f3ff015b6b0 2024-04-25T09:01:44.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3ff6d8c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff00cc000 con 0x7f3ff0072000 2024-04-25T09:01:44.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3ff6d8c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff015ba90 con 0x7f3ff00cf590 2024-04-25T09:01:44.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3feffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3ff00cf590 0x7f3ff00cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:44.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3ff4b28640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3ff015b6b0 0x7f3ff00ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:40998/0 (socket says 172.21.15.37:40998) 2024-04-25T09:01:44.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3ff4b28640 1 -- 172.21.15.37:0/3752374777 learned_addr learned my addr 172.21.15.37:0/3752374777 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:44.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3feffff640 1 -- 172.21.15.37:0/3752374777 >> v1:172.21.15.203:6789/0 conn(0x7f3ff0072000 legacy=0x7f3ff0072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:44.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3feffff640 1 -- 172.21.15.37:0/3752374777 >> v1:172.21.15.37:6789/0 conn(0x7f3ff015b6b0 legacy=0x7f3ff00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3fef7fe640 1 -- 172.21.15.37:0/3752374777 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184248042 0 0) 0x7f3ff00cbd20 con 0x7f3ff015b6b0 2024-04-25T09:01:44.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3feffff640 1 -- 172.21.15.37:0/3752374777 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff00cc2e0 con 0x7f3ff00cf590 2024-04-25T09:01:44.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3feffff640 1 --2- 172.21.15.37:0/3752374777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff00cf590 0x7f3ff00cfa20 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40097c0 tx=0x7f3fe401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db4cd515b54d0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:44.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3fef7fe640 1 -- 172.21.15.37:0/3752374777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe40b0070 con 0x7f3ff00cf590 2024-04-25T09:01:44.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3fef7fe640 1 -- 172.21.15.37:0/3752374777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe4004230 con 0x7f3ff00cf590 2024-04-25T09:01:44.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.694+0000 7f3fef7fe640 1 -- 172.21.15.37:0/3752374777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4003620 con 0x7f3ff00cf590 2024-04-25T09:01:44.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/3752374777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff00cf590 msgr2=0x7f3ff00cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 --2- 172.21.15.37:0/3752374777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff00cf590 0x7f3ff00cfa20 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f3fe40097c0 tx=0x7f3fe401c460 comp rx=0 tx=0).stop 2024-04-25T09:01:44.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/3752374777 shutdown_connections 2024-04-25T09:01:44.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 --2- 172.21.15.37:0/3752374777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff00cf590 0x7f3ff00cfa20 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/3752374777 >> 172.21.15.37:0/3752374777 conn(0x7f3ff006ae60 msgr2=0x7f3ff006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:44.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/3752374777 shutdown_connections 2024-04-25T09:01:44.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/3752374777 wait complete. 2024-04-25T09:01:44.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 Processor -- start 2024-04-25T09:01:44.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 -- start start 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff0072000 0x7f3ff00afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff00cf590 0x7f3ff00989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015b6b0 0x7f3ff01f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00caf30 con 0x7f3ff015b6b0 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff00cadb0 con 0x7f3ff0072000 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff01692b0 con 0x7f3ff00cf590 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff5329640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015b6b0 0x7f3ff01f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff5329640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015b6b0 0x7f3ff01f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51948/0 (socket says 172.21.15.37:51948) 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff5329640 1 -- 172.21.15.37:0/80576001 learned_addr learned my addr 172.21.15.37:0/80576001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3feffff640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff00cf590 0x7f3ff00989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff4b28640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff0072000 0x7f3ff00afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff5329640 1 -- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff00cf590 msgr2=0x7f3ff00989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff5329640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff00cf590 0x7f3ff00989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff5329640 1 -- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff0072000 msgr2=0x7f3ff00afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff5329640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff0072000 0x7f3ff00afbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff5329640 1 -- 172.21.15.37:0/80576001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff0098ef0 con 0x7f3ff015b6b0 2024-04-25T09:01:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff5329640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015b6b0 0x7f3ff01f7b70 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c7b0 tx=0x7f3fe800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3fed7fa640 1 -- 172.21.15.37:0/80576001 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8004b70 con 0x7f3ff015b6b0 2024-04-25T09:01:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3fed7fa640 1 -- 172.21.15.37:0/80576001 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe8004d10 con 0x7f3ff015b6b0 2024-04-25T09:01:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3fed7fa640 1 -- 172.21.15.37:0/80576001 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe800aa90 con 0x7f3ff015b6b0 2024-04-25T09:01:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/80576001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff01f8980 con 0x7f3ff015b6b0 2024-04-25T09:01:44.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.698+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/80576001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff015cf40 con 0x7f3ff015b6b0 2024-04-25T09:01:44.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.702+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/80576001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff004efa0 con 0x7f3ff015b6b0 2024-04-25T09:01:44.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.702+0000 7f3fed7fa640 1 -- 172.21.15.37:0/80576001 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3fe800ac30 con 0x7f3ff015b6b0 2024-04-25T09:01:44.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.702+0000 7f3fed7fa640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fdc066f60 0x7f3fdc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:44.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.702+0000 7f3fed7fa640 1 -- 172.21.15.37:0/80576001 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3fe8085530 con 0x7f3ff015b6b0 2024-04-25T09:01:44.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.702+0000 7f3ff4b28640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fdc066f60 0x7f3fdc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:44.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.702+0000 7f3ff4b28640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fdc066f60 0x7f3fdc069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000f70 tx=0x7f3fe0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:44.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.706+0000 7f3fed7fa640 1 -- 172.21.15.37:0/80576001 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3fe8020070 con 0x7f3ff015b6b0 2024-04-25T09:01:44.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.870+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/80576001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3ff006ede0 con 0x7f3ff015b6b0 2024-04-25T09:01:44.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3fed7fa640 1 -- 172.21.15.37:0/80576001 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fe8056320 con 0x7f3ff015b6b0 2024-04-25T09:01:44.861 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fdc066f60 msgr2=0x7f3fdc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fdc066f60 0x7f3fdc069420 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000f70 tx=0x7f3fe0009040 comp rx=0 tx=0).stop 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/80576001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015b6b0 msgr2=0x7f3ff01f7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015b6b0 0x7f3ff01f7b70 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c7b0 tx=0x7f3fe800cc70 comp rx=0 tx=0).stop 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/80576001 shutdown_connections 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3fdc066f60 0x7f3fdc069420 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ff015b6b0 0x7f3ff01f7b70 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ff00cf590 0x7f3ff00989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 --2- 172.21.15.37:0/80576001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ff0072000 0x7f3ff00afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/80576001 >> 172.21.15.37:0/80576001 conn(0x7f3ff006ae60 msgr2=0x7f3ff00b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/80576001 shutdown_connections 2024-04-25T09:01:44.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:44.874+0000 7f3ff6d8c640 1 -- 172.21.15.37:0/80576001 wait complete. 2024-04-25T09:01:44.881 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:01:45.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddbb776640 1 Processor -- start 2024-04-25T09:01:45.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddbb776640 1 -- start start 2024-04-25T09:01:45.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddbb776640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fddb40d4230 0x7fddb40d1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddbb776640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb406be60 con 0x7fddb40d07f0 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddbb776640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb40d4650 con 0x7fddb415da80 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddbb776640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb40d2210 con 0x7fddb40d4230 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddb9d13640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fddb40d4230 0x7fddb40d1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddb8d11640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fddb40d07f0 0x7fddb406c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41010/0 (socket says 172.21.15.37:41010) 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddb8d11640 1 -- 172.21.15.37:0/175997708 learned_addr learned my addr 172.21.15.37:0/175997708 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddb9d13640 1 -- 172.21.15.37:0/175997708 >> v1:172.21.15.203:6789/0 conn(0x7fddb415da80 legacy=0x7fddb40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddb9d13640 1 -- 172.21.15.37:0/175997708 >> v1:172.21.15.37:6789/0 conn(0x7fddb40d07f0 legacy=0x7fddb406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fdda7fff640 1 -- 172.21.15.37:0/175997708 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531388620 0 0) 0x7fddb406be60 con 0x7fddb40d07f0 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.014+0000 7fddb9d13640 1 -- 172.21.15.37:0/175997708 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb415de60 con 0x7fddb40d4230 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fddb9d13640 1 --2- 172.21.15.37:0/175997708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb40d4230 0x7fddb40d1cd0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fddac009d90 tx=0x7fddac01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68204ce2a0afd406 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fdda7fff640 1 -- 172.21.15.37:0/175997708 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac0b2070 con 0x7fddb40d4230 2024-04-25T09:01:45.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fdda7fff640 1 -- 172.21.15.37:0/175997708 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddac0a5830 con 0x7fddb40d4230 2024-04-25T09:01:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fdda7fff640 1 -- 172.21.15.37:0/175997708 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddac0a5c00 con 0x7fddb40d4230 2024-04-25T09:01:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fddbb776640 1 -- 172.21.15.37:0/175997708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb40d4230 msgr2=0x7fddb40d1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fddbb776640 1 --2- 172.21.15.37:0/175997708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb40d4230 0x7fddb40d1cd0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fddac009d90 tx=0x7fddac01c970 comp rx=0 tx=0).stop 2024-04-25T09:01:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fddbb776640 1 -- 172.21.15.37:0/175997708 shutdown_connections 2024-04-25T09:01:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fddbb776640 1 --2- 172.21.15.37:0/175997708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb40d4230 0x7fddb40d1cd0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fddbb776640 1 -- 172.21.15.37:0/175997708 >> 172.21.15.37:0/175997708 conn(0x7fddb408b000 msgr2=0x7fddb40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fddbb776640 1 -- 172.21.15.37:0/175997708 shutdown_connections 2024-04-25T09:01:45.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fddbb776640 1 -- 172.21.15.37:0/175997708 wait complete. 2024-04-25T09:01:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fddbb776640 1 Processor -- start 2024-04-25T09:01:45.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.018+0000 7fddbb776640 1 -- start start 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddbb776640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb40d07f0 0x7fddb4156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddbb776640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb40d4230 0x7fddb40b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddbb776640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddb415da80 0x7fddb40c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddbb776640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb406dbc0 con 0x7fddb40d4230 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddbb776640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb406da40 con 0x7fddb40d07f0 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddbb776640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb416b5c0 con 0x7fddb415da80 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb8d11640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb40d4230 0x7fddb40b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb8d11640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb40d4230 0x7fddb40b6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51966/0 (socket says 172.21.15.37:51966) 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb8d11640 1 -- 172.21.15.37:0/698566651 learned_addr learned my addr 172.21.15.37:0/698566651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb9512640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb40d07f0 0x7fddb4156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb9d13640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddb415da80 0x7fddb40c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb8d11640 1 -- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddb415da80 msgr2=0x7fddb40c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb8d11640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddb415da80 0x7fddb40c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb8d11640 1 -- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb40d07f0 msgr2=0x7fddb4156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb8d11640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb40d07f0 0x7fddb4156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb8d11640 1 -- 172.21.15.37:0/698566651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb40b7200 con 0x7fddb40d4230 2024-04-25T09:01:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb8d11640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb40d4230 0x7fddb40b6b00 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fdda800c760 tx=0x7fdda800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fdda5ffb640 1 -- 172.21.15.37:0/698566651 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda8007270 con 0x7fddb40d4230 2024-04-25T09:01:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fdda5ffb640 1 -- 172.21.15.37:0/698566651 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda8007430 con 0x7fddb40d4230 2024-04-25T09:01:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddbb776640 1 -- 172.21.15.37:0/698566651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb40cbf30 con 0x7fddb40d4230 2024-04-25T09:01:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fdda5ffb640 1 -- 172.21.15.37:0/698566651 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda80059f0 con 0x7fddb40d4230 2024-04-25T09:01:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddbb776640 1 -- 172.21.15.37:0/698566651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb4158880 con 0x7fddb40d4230 2024-04-25T09:01:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddbb776640 1 -- 172.21.15.37:0/698566651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb404efa0 con 0x7fddb40d4230 2024-04-25T09:01:45.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fdda5ffb640 1 -- 172.21.15.37:0/698566651 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdda8005b90 con 0x7fddb40d4230 2024-04-25T09:01:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fdda5ffb640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdd98066f60 0x7fdd98069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fdda5ffb640 1 -- 172.21.15.37:0/698566651 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdda8084d50 con 0x7fddb40d4230 2024-04-25T09:01:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.022+0000 7fddb9512640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdd98066f60 0x7fdd98069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.026+0000 7fddb9512640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdd98066f60 0x7fdd98069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fddb0007fd0 tx=0x7fddb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:45.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.026+0000 7fdda5ffb640 1 -- 172.21.15.37:0/698566651 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdda8088030 con 0x7fddb40d4230 2024-04-25T09:01:45.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.194+0000 7fddbb776640 1 -- 172.21.15.37:0/698566651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fddb40ca4d0 con 0x7fddb40d4230 2024-04-25T09:01:45.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fdda5ffb640 1 -- 172.21.15.37:0/698566651 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdda8001d10 con 0x7fddb40d4230 2024-04-25T09:01:45.186 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:01:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 -- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdd98066f60 msgr2=0x7fdd98069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdd98066f60 0x7fdd98069420 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fddb0007fd0 tx=0x7fddb0009040 comp rx=0 tx=0).stop 2024-04-25T09:01:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 -- 172.21.15.37:0/698566651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb40d4230 msgr2=0x7fddb40b6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb40d4230 0x7fddb40b6b00 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fdda800c760 tx=0x7fdda800cc20 comp rx=0 tx=0).stop 2024-04-25T09:01:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 -- 172.21.15.37:0/698566651 shutdown_connections 2024-04-25T09:01:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdd98066f60 0x7fdd98069420 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fddb415da80 0x7fddb40c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddb40d4230 0x7fddb40b6b00 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 --2- 172.21.15.37:0/698566651 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fddb40d07f0 0x7fddb4156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 -- 172.21.15.37:0/698566651 >> 172.21.15.37:0/698566651 conn(0x7fddb408b000 msgr2=0x7fddb40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:45.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 -- 172.21.15.37:0/698566651 shutdown_connections 2024-04-25T09:01:45.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.198+0000 7fddbb776640 1 -- 172.21.15.37:0/698566651 wait complete. 2024-04-25T09:01:45.205 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:01:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7faa009b0640 1 Processor -- start 2024-04-25T09:01:45.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7faa009b0640 1 -- start start 2024-04-25T09:01:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7faa009b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9fc0d07f0 0x7fa9fc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7faa009b0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9fc06be60 con 0x7fa9fc15da80 2024-04-25T09:01:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7faa009b0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9fc15de60 con 0x7fa9fc0d1cd0 2024-04-25T09:01:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7faa009b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc06c440 con 0x7fa9fc0d07f0 2024-04-25T09:01:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9fa59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa9fc15da80 0x7fa9fc0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41016/0 (socket says 172.21.15.37:41016) 2024-04-25T09:01:45.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9fa59c640 1 -- 172.21.15.37:0/2614979558 learned_addr learned my addr 172.21.15.37:0/2614979558 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:45.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f9d9b640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9fc0d07f0 0x7fa9fc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96086590 0 0) 0x7fa9fc06be60 con 0x7fa9fc15da80 2024-04-25T09:01:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9e4003530 con 0x7fa9fc15da80 2024-04-25T09:01:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1404707490 0 0) 0x7fa9e4003530 con 0x7fa9fc15da80 2024-04-25T09:01:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9fc0d07f0 msgr2=0x7fa9fc0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9fc0d07f0 0x7fa9fc0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 >> v1:172.21.15.203:6789/0 conn(0x7fa9fc0d1cd0 legacy=0x7fa9fc06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc0d2450 con 0x7fa9fc15da80 2024-04-25T09:01:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fa9ec001680 con 0x7fa9fc15da80 2024-04-25T09:01:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 >> v1:172.21.15.37:6789/0 conn(0x7fa9fc15da80 legacy=0x7fa9fc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9fc0d07f0 0x7fa9e4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9e4003fe0 0x7fa9e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e4004d50 0x7fa9e40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc06c440 con 0x7fa9e4004d50 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e400bb30 con 0x7fa9e4003fe0 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e400bd30 con 0x7fa9fc0d07f0 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9fad9d640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e4004d50 0x7fa9e40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9fa59c640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9fc0d07f0 0x7fa9e4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9f9d9b640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9e4003fe0 0x7fa9e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9fad9d640 1 -- 172.21.15.37:0/2614979558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9fc0d07f0 msgr2=0x7fa9e4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9fad9d640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9fc0d07f0 0x7fa9e4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9fad9d640 1 -- 172.21.15.37:0/2614979558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9e4003fe0 msgr2=0x7fa9e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9fad9d640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9e4003fe0 0x7fa9e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.338+0000 7fa9fad9d640 1 -- 172.21.15.37:0/2614979558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e4003530 con 0x7fa9e4004d50 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7fa9fad9d640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e4004d50 0x7fa9e40051d0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4001e70 tx=0x7fa9f401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d49621f84a273e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:45.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4001a90 con 0x7fa9e4004d50 2024-04-25T09:01:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f4003b80 con 0x7fa9e4004d50 2024-04-25T09:01:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7fa9f959a640 1 -- 172.21.15.37:0/2614979558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4005080 con 0x7fa9e4004d50 2024-04-25T09:01:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7faa009b0640 1 -- 172.21.15.37:0/2614979558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e4004d50 msgr2=0x7fa9e40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7faa009b0640 1 --2- 172.21.15.37:0/2614979558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9e4004d50 0x7fa9e40051d0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4001e70 tx=0x7fa9f401c780 comp rx=0 tx=0).stop 2024-04-25T09:01:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7fa9fad9d640 1 -- 172.21.15.37:0/2614979558 reap_dead start 2024-04-25T09:01:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7faa009b0640 1 -- 172.21.15.37:0/2614979558 shutdown_connections 2024-04-25T09:01:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7faa009b0640 1 -- 172.21.15.37:0/2614979558 >> 172.21.15.37:0/2614979558 conn(0x7fa9fc08b000 msgr2=0x7fa9fc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7faa009b0640 1 -- 172.21.15.37:0/2614979558 shutdown_connections 2024-04-25T09:01:45.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7faa009b0640 1 -- 172.21.15.37:0/2614979558 wait complete. 2024-04-25T09:01:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7faa009b0640 1 Processor -- start 2024-04-25T09:01:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.342+0000 7faa009b0640 1 -- start start 2024-04-25T09:01:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7faa009b0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9e4004d50 0x7fa9fc15c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7faa009b0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9fc0cc7d0 0x7fa9fc0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7faa009b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9fc0b6240 0x7fa9fc0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7faa009b0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc0d26f0 con 0x7fa9fc0b6240 2024-04-25T09:01:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7faa009b0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc0d2160 con 0x7fa9fc0cc7d0 2024-04-25T09:01:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7faa009b0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9fc0d2870 con 0x7fa9e4004d50 2024-04-25T09:01:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9fc0b6240 0x7fa9fc0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9fc0b6240 0x7fa9fc0b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51984/0 (socket says 172.21.15.37:51984) 2024-04-25T09:01:45.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9fad9d640 1 -- 172.21.15.37:0/310004371 learned_addr learned my addr 172.21.15.37:0/310004371 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9fa59c640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9e4004d50 0x7fa9fc15c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9f9d9b640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9fc0cc7d0 0x7fa9fc0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9fad9d640 1 -- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9e4004d50 msgr2=0x7fa9fc15c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9fad9d640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9e4004d50 0x7fa9fc15c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9fad9d640 1 -- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9fc0cc7d0 msgr2=0x7fa9fc0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9fad9d640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9fc0cc7d0 0x7fa9fc0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9fad9d640 1 -- 172.21.15.37:0/310004371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc0b6e50 con 0x7fa9fc0b6240 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9f9d9b640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9fc0cc7d0 0x7fa9fc0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9fad9d640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9fc0b6240 0x7fa9fc0b66c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fa9f401cc60 tx=0x7fa9e4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9eaffd640 1 -- 172.21.15.37:0/310004371 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f40b8070 con 0x7fa9fc0b6240 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9eaffd640 1 -- 172.21.15.37:0/310004371 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f4005820 con 0x7fa9fc0b6240 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9eaffd640 1 -- 172.21.15.37:0/310004371 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4001a90 con 0x7fa9fc0b6240 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7faa009b0640 1 -- 172.21.15.37:0/310004371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9fc0b7130 con 0x7fa9fc0b6240 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7faa009b0640 1 -- 172.21.15.37:0/310004371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9fc0b76f0 con 0x7fa9fc0b6240 2024-04-25T09:01:45.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7faa009b0640 1 -- 172.21.15.37:0/310004371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9fc04efa0 con 0x7fa9fc0b6240 2024-04-25T09:01:45.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9eaffd640 1 -- 172.21.15.37:0/310004371 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa9f40bb070 con 0x7fa9fc0b6240 2024-04-25T09:01:45.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9eaffd640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa9dc066f90 0x7fa9dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.346+0000 7fa9eaffd640 1 -- 172.21.15.37:0/310004371 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa9f411e3f0 con 0x7fa9fc0b6240 2024-04-25T09:01:45.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.350+0000 7fa9fa59c640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa9dc066f90 0x7fa9dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.350+0000 7fa9fa59c640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa9dc066f90 0x7fa9dc069450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0b8150 tx=0x7fa9ec000bd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:45.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.350+0000 7fa9eaffd640 1 -- 172.21.15.37:0/310004371 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa9f40ac020 con 0x7fa9fc0b6240 2024-04-25T09:01:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.506+0000 7faa009b0640 1 -- 172.21.15.37:0/310004371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9fc0cdb20 con 0x7fa9fc0b6240 2024-04-25T09:01:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7fa9eaffd640 1 -- 172.21.15.37:0/310004371 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa9f4016490 con 0x7fa9fc0b6240 2024-04-25T09:01:45.498 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:01:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 -- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa9dc066f90 msgr2=0x7fa9dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa9dc066f90 0x7fa9dc069450 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0b8150 tx=0x7fa9ec000bd0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 -- 172.21.15.37:0/310004371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9fc0b6240 msgr2=0x7fa9fc0b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9fc0b6240 0x7fa9fc0b66c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fa9f401cc60 tx=0x7fa9e4005840 comp rx=0 tx=0).stop 2024-04-25T09:01:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 -- 172.21.15.37:0/310004371 shutdown_connections 2024-04-25T09:01:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa9dc066f90 0x7fa9dc069450 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9fc0b6240 0x7fa9fc0b66c0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa9fc0cc7d0 0x7fa9fc0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 --2- 172.21.15.37:0/310004371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa9e4004d50 0x7fa9fc15c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 -- 172.21.15.37:0/310004371 >> 172.21.15.37:0/310004371 conn(0x7fa9fc08b000 msgr2=0x7fa9fc06c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 -- 172.21.15.37:0/310004371 shutdown_connections 2024-04-25T09:01:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.510+0000 7faa009b0640 1 -- 172.21.15.37:0/310004371 wait complete. 2024-04-25T09:01:45.516 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:01:45.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.638+0000 7f9fa6ae2640 1 Processor -- start 2024-04-25T09:01:45.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.638+0000 7f9fa6ae2640 1 -- start start 2024-04-25T09:01:45.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.638+0000 7f9fa6ae2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9fa015da80 0x7f9fa00d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.638+0000 7f9fa6ae2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9fa00d1cd0 0x7f9fa00d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.638+0000 7f9fa6ae2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa006be60 con 0x7f9fa00d0620 2024-04-25T09:01:45.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.638+0000 7f9fa6ae2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa00d0a70 con 0x7f9fa015da80 2024-04-25T09:01:45.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.638+0000 7f9fa6ae2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fa006c440 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.638+0000 7f9fa487e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9fa015da80 0x7f9fa00d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.638+0000 7f9fa487e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f9fa015da80 0x7f9fa00d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51992/0 (socket says 172.21.15.37:51992) 2024-04-25T09:01:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.638+0000 7f9fa487e640 1 -- 172.21.15.37:0/1000582151 learned_addr learned my addr 172.21.15.37:0/1000582151 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa487e640 1 -- 172.21.15.37:0/1000582151 >> v2:172.21.15.203:3300/0 conn(0x7f9fa00d1cd0 msgr2=0x7f9fa00d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9f97fff640 1 --2- 172.21.15.37:0/1000582151 >> v2:172.21.15.203:3300/0 conn(0x7f9fa00d1cd0 0x7f9fa00d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa487e640 1 --2- 172.21.15.37:0/1000582151 >> v2:172.21.15.203:3300/0 conn(0x7f9fa00d1cd0 0x7f9fa00d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa487e640 1 -- 172.21.15.37:0/1000582151 >> v1:172.21.15.203:6789/0 conn(0x7f9fa00d0620 legacy=0x7f9fa00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa487e640 1 -- 172.21.15.37:0/1000582151 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa015de60 con 0x7f9fa015da80 2024-04-25T09:01:45.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9f97fff640 1 --2- 172.21.15.37:0/1000582151 >> v2:172.21.15.203:3300/0 conn(0x7f9fa00d1cd0 0x7f9fa00d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:45.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa487e640 1 --2- 172.21.15.37:0/1000582151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fa015da80 0x7f9fa00d4230 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009870 tx=0x7f9f9c01c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80a7046ca7c379e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:45.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9f977fe640 1 -- 172.21.15.37:0/1000582151 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0b0070 con 0x7f9fa015da80 2024-04-25T09:01:45.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9f977fe640 1 -- 172.21.15.37:0/1000582151 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9c0ab600 con 0x7f9fa015da80 2024-04-25T09:01:45.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9f977fe640 1 -- 172.21.15.37:0/1000582151 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0ab9d0 con 0x7f9fa015da80 2024-04-25T09:01:45.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/1000582151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fa015da80 msgr2=0x7f9fa00d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa6ae2640 1 --2- 172.21.15.37:0/1000582151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fa015da80 0x7f9fa00d4230 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c009870 tx=0x7f9f9c01c4e0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/1000582151 shutdown_connections 2024-04-25T09:01:45.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa6ae2640 1 --2- 172.21.15.37:0/1000582151 >> v2:172.21.15.203:3300/0 conn(0x7f9fa00d1cd0 0x7f9fa00d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa6ae2640 1 --2- 172.21.15.37:0/1000582151 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fa015da80 0x7f9fa00d4230 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/1000582151 >> 172.21.15.37:0/1000582151 conn(0x7f9fa008b000 msgr2=0x7f9fa00d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:45.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/1000582151 shutdown_connections 2024-04-25T09:01:45.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.642+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/1000582151 wait complete. 2024-04-25T09:01:45.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa6ae2640 1 Processor -- start 2024-04-25T09:01:45.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa6ae2640 1 -- start start 2024-04-25T09:01:45.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa6ae2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa00d0620 0x7f9fa0154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa6ae2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fa00d1cd0 0x7f9fa00b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa6ae2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fa015da80 0x7f9fa00b5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa6ae2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa016a730 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa6ae2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa016a5b0 con 0x7f9fa015da80 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa6ae2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa016a8b0 con 0x7f9fa00d0620 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f97fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fa00d1cd0 0x7f9fa00b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f97fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fa00d1cd0 0x7f9fa00b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51994/0 (socket says 172.21.15.37:51994) 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f97fff640 1 -- 172.21.15.37:0/2169989027 learned_addr learned my addr 172.21.15.37:0/2169989027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa487e640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa00d0620 0x7f9fa0154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa507f640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fa015da80 0x7f9fa00b5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f97fff640 1 -- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa00d0620 msgr2=0x7f9fa0154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f97fff640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa00d0620 0x7f9fa0154db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f97fff640 1 -- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fa015da80 msgr2=0x7f9fa00b5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f97fff640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fa015da80 0x7f9fa00b5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f97fff640 1 -- 172.21.15.37:0/2169989027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa015f120 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f97fff640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fa00d1cd0 0x7f9fa00b51c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000c7f0 tx=0x7f9f9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:45.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f957fa640 1 -- 172.21.15.37:0/2169989027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90005ba0 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f957fa640 1 -- 172.21.15.37:0/2169989027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f90005d40 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/2169989027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa015f400 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9f957fa640 1 -- 172.21.15.37:0/2169989027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90007a50 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/2169989027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa015f9c0 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.646+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/2169989027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa004efa0 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.650+0000 7f9f957fa640 1 -- 172.21.15.37:0/2169989027 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9f90007bf0 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.650+0000 7f9f957fa640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9f8c066f60 0x7f9f8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.650+0000 7f9f957fa640 1 -- 172.21.15.37:0/2169989027 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9f90085260 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.650+0000 7f9fa487e640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9f8c066f60 0x7f9f8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.650+0000 7f9fa487e640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9f8c066f60 0x7f9f8c069420 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0ad000 tx=0x7f9f9c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:45.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.654+0000 7f9f957fa640 1 -- 172.21.15.37:0/2169989027 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9f90015020 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/2169989027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9fa00c6c10 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9f957fa640 1 -- 172.21.15.37:0/2169989027 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9f90056170 con 0x7f9fa00d1cd0 2024-04-25T09:01:45.810 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9f8c066f60 msgr2=0x7f9f8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9f8c066f60 0x7f9f8c069420 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0ad000 tx=0x7f9f9c008040 comp rx=0 tx=0).stop 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/2169989027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fa00d1cd0 msgr2=0x7f9fa00b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fa00d1cd0 0x7f9fa00b51c0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000c7f0 tx=0x7f9f9000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/2169989027 shutdown_connections 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9f8c066f60 0x7f9f8c069420 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9fa015da80 0x7f9fa00b5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9fa00d1cd0 0x7f9fa00b51c0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 --2- 172.21.15.37:0/2169989027 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9fa00d0620 0x7f9fa0154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/2169989027 >> 172.21.15.37:0/2169989027 conn(0x7f9fa008b000 msgr2=0x7f9fa0160920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/2169989027 shutdown_connections 2024-04-25T09:01:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.822+0000 7f9fa6ae2640 1 -- 172.21.15.37:0/2169989027 wait complete. 2024-04-25T09:01:45.828 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:01:45.829 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:45.835 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:45.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.998+0000 7f586eb1e640 1 Processor -- start 2024-04-25T09:01:45.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:45.998+0000 7f586eb1e640 1 -- start start 2024-04-25T09:01:45.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586eb1e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5868144ca0 0x7f586806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586eb1e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f586806ec60 0x7f5868142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586eb1e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58680ccc70 con 0x7f58680ce510 2024-04-25T09:01:45.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586eb1e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58680cc2b0 con 0x7f586806ec60 2024-04-25T09:01:45.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586eb1e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586806e290 con 0x7f5868144ca0 2024-04-25T09:01:45.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586d0bb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f586806ec60 0x7f5868142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586c8ba640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f58680ce510 0x7f58680ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41028/0 (socket says 172.21.15.37:41028) 2024-04-25T09:01:45.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586c8ba640 1 -- 172.21.15.37:0/3291308369 learned_addr learned my addr 172.21.15.37:0/3291308369 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:45.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586d0bb640 1 -- 172.21.15.37:0/3291308369 >> v2:172.21.15.203:3300/0 conn(0x7f5868144ca0 msgr2=0x7f586806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:45.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586d0bb640 1 --2- 172.21.15.37:0/3291308369 >> v2:172.21.15.203:3300/0 conn(0x7f5868144ca0 0x7f586806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586d0bb640 1 -- 172.21.15.37:0/3291308369 >> v1:172.21.15.37:6789/0 conn(0x7f58680ce510 legacy=0x7f58680ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586d0bb640 1 -- 172.21.15.37:0/3291308369 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586806dc40 con 0x7f586806ec60 2024-04-25T09:01:45.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f58677fe640 1 -- 172.21.15.37:0/3291308369 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079639848 0 0) 0x7f58680ccc70 con 0x7f58680ce510 2024-04-25T09:01:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f586d0bb640 1 --2- 172.21.15.37:0/3291308369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f586806ec60 0x7f5868142680 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f5868070fe0 tx=0x7f586001f110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75aa3e3b210feefe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f58677fe640 1 -- 172.21.15.37:0/3291308369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58600b4070 con 0x7f586806ec60 2024-04-25T09:01:45.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f58677fe640 1 -- 172.21.15.37:0/3291308369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58600af600 con 0x7f586806ec60 2024-04-25T09:01:45.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.002+0000 7f58677fe640 1 -- 172.21.15.37:0/3291308369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58600af9d0 con 0x7f586806ec60 2024-04-25T09:01:45.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- 172.21.15.37:0/3291308369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f586806ec60 msgr2=0x7f5868142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 --2- 172.21.15.37:0/3291308369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f586806ec60 0x7f5868142680 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f5868070fe0 tx=0x7f586001f110 comp rx=0 tx=0).stop 2024-04-25T09:01:45.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- 172.21.15.37:0/3291308369 shutdown_connections 2024-04-25T09:01:45.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 --2- 172.21.15.37:0/3291308369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f586806ec60 0x7f5868142680 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 --2- 172.21.15.37:0/3291308369 >> v2:172.21.15.203:3300/0 conn(0x7f5868144ca0 0x7f586806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- 172.21.15.37:0/3291308369 >> 172.21.15.37:0/3291308369 conn(0x7f58680b11a0 msgr2=0x7f58680b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:45.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- 172.21.15.37:0/3291308369 shutdown_connections 2024-04-25T09:01:45.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- 172.21.15.37:0/3291308369 wait complete. 2024-04-25T09:01:45.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 Processor -- start 2024-04-25T09:01:45.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- start start 2024-04-25T09:01:45.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f586806ec60 0x7f5868130da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58680ce510 0x7f586812eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5868144ca0 0x7f58681e1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58681451f0 con 0x7f5868144ca0 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58680cc900 con 0x7f58680ce510 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5868145370 con 0x7f586806ec60 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586d0bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5868144ca0 0x7f58681e1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586d0bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5868144ca0 0x7f58681e1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52014/0 (socket says 172.21.15.37:52014) 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586d0bb640 1 -- 172.21.15.37:0/49348758 learned_addr learned my addr 172.21.15.37:0/49348758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586c8ba640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f586806ec60 0x7f5868130da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586d0bb640 1 -- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f586806ec60 msgr2=0x7f5868130da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f5867fff640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58680ce510 0x7f586812eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586d0bb640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f586806ec60 0x7f5868130da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586d0bb640 1 -- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58680ce510 msgr2=0x7f586812eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586d0bb640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58680ce510 0x7f586812eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586d0bb640 1 -- 172.21.15.37:0/49348758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586812f570 con 0x7f5868144ca0 2024-04-25T09:01:45.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586c8ba640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f586806ec60 0x7f5868130da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:45.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586d0bb640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5868144ca0 0x7f58681e1f20 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f586001bd20 tx=0x7f586001beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:45.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f58657fa640 1 -- 172.21.15.37:0/49348758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58600b4040 con 0x7f5868144ca0 2024-04-25T09:01:45.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f58657fa640 1 -- 172.21.15.37:0/49348758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58600c2070 con 0x7f5868144ca0 2024-04-25T09:01:45.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f58657fa640 1 -- 172.21.15.37:0/49348758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58600ba660 con 0x7f5868144ca0 2024-04-25T09:01:45.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- 172.21.15.37:0/49348758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586812f850 con 0x7f5868144ca0 2024-04-25T09:01:45.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- 172.21.15.37:0/49348758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586812fe10 con 0x7f5868144ca0 2024-04-25T09:01:45.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.006+0000 7f586eb1e640 1 -- 172.21.15.37:0/49348758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586804efa0 con 0x7f5868144ca0 2024-04-25T09:01:45.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.010+0000 7f58657fa640 1 -- 172.21.15.37:0/49348758 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58600ba800 con 0x7f5868144ca0 2024-04-25T09:01:45.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.010+0000 7f58657fa640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f584c066f60 0x7f584c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:45.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.010+0000 7f58657fa640 1 -- 172.21.15.37:0/49348758 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5860123990 con 0x7f5868144ca0 2024-04-25T09:01:45.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.010+0000 7f586c8ba640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f584c066f60 0x7f584c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:45.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.010+0000 7f586c8ba640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f584c066f60 0x7f584c069420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f585c000f70 tx=0x7f585c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:46.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.014+0000 7f58657fa640 1 -- 172.21.15.37:0/49348758 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5860018c70 con 0x7f5868144ca0 2024-04-25T09:01:46.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.170+0000 7f586eb1e640 1 -- 172.21.15.37:0/49348758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f586806c030 con 0x7f5868144ca0 2024-04-25T09:01:46.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f58657fa640 1 -- 172.21.15.37:0/49348758 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16848 (secure 0 0 0) 0x7f58600f4810 con 0x7f5868144ca0 2024-04-25T09:01:46.162 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:46.162 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:01:36.569159+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:33.212328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:33.895814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:46.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 -- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f584c066f60 msgr2=0x7f584c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f584c066f60 0x7f584c069420 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f585c000f70 tx=0x7f585c009040 comp rx=0 tx=0).stop 2024-04-25T09:01:46.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 -- 172.21.15.37:0/49348758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5868144ca0 msgr2=0x7f58681e1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5868144ca0 0x7f58681e1f20 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f586001bd20 tx=0x7f586001beb0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 -- 172.21.15.37:0/49348758 shutdown_connections 2024-04-25T09:01:46.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f584c066f60 0x7f584c069420 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5868144ca0 0x7f58681e1f20 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f58680ce510 0x7f586812eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 --2- 172.21.15.37:0/49348758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f586806ec60 0x7f5868130da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 -- 172.21.15.37:0/49348758 >> 172.21.15.37:0/49348758 conn(0x7f58680b11a0 msgr2=0x7f5868071030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:46.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 -- 172.21.15.37:0/49348758 shutdown_connections 2024-04-25T09:01:46.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.174+0000 7f586eb1e640 1 -- 172.21.15.37:0/49348758 wait complete. 2024-04-25T09:01:46.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:01:33.212328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:01:33.895814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:01:46.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:01:46.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6edc7f4640 1 Processor -- start 2024-04-25T09:01:46.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6edc7f4640 1 -- start start 2024-04-25T09:01:46.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6edc7f4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6ed40cf590 0x7f6ed415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6edc7f4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f6ed40ce170 0x7f6ed40ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6edc7f4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed40ca4c0 con 0x7f6ed40cf970 2024-04-25T09:01:46.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6edc7f4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed40ca780 con 0x7f6ed40cf590 2024-04-25T09:01:46.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6edc7f4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed40cd440 con 0x7f6ed40ce170 2024-04-25T09:01:46.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6eda590640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6ed40cf590 0x7f6ed415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6eda590640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6ed40cf590 0x7f6ed415ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52026/0 (socket says 172.21.15.37:52026) 2024-04-25T09:01:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6eda590640 1 -- 172.21.15.37:0/3181672224 learned_addr learned my addr 172.21.15.37:0/3181672224 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6edad91640 1 --2- 172.21.15.37:0/3181672224 >> v2:172.21.15.203:3300/0 conn(0x7f6ed40ce170 0x7f6ed40ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6eda590640 1 -- 172.21.15.37:0/3181672224 >> v2:172.21.15.203:3300/0 conn(0x7f6ed40ce170 msgr2=0x7f6ed40ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6eda590640 1 --2- 172.21.15.37:0/3181672224 >> v2:172.21.15.203:3300/0 conn(0x7f6ed40ce170 0x7f6ed40ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6ed958e640 1 -- 172.21.15.37:0/3181672224 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3494833398 0 0) 0x7f6ed40ca4c0 con 0x7f6ed40cf970 2024-04-25T09:01:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6eda590640 1 -- 172.21.15.37:0/3181672224 >> v1:172.21.15.37:6789/0 conn(0x7f6ed40cf970 legacy=0x7f6ed40c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.306+0000 7f6eda590640 1 -- 172.21.15.37:0/3181672224 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed40cd690 con 0x7f6ed40cf590 2024-04-25T09:01:46.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6eda590640 1 --2- 172.21.15.37:0/3181672224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed40cf590 0x7f6ed415ecb0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40c9940 tx=0x7f6ed001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e73e1d93a1206ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:46.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6ed958e640 1 -- 172.21.15.37:0/3181672224 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed00ab430 con 0x7f6ed40cf590 2024-04-25T09:01:46.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6ed958e640 1 -- 172.21.15.37:0/3181672224 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed00b0070 con 0x7f6ed40cf590 2024-04-25T09:01:46.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6ed958e640 1 -- 172.21.15.37:0/3181672224 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed00ab7f0 con 0x7f6ed40cf590 2024-04-25T09:01:46.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 -- 172.21.15.37:0/3181672224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed40cf590 msgr2=0x7f6ed415ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 --2- 172.21.15.37:0/3181672224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed40cf590 0x7f6ed415ecb0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40c9940 tx=0x7f6ed001c520 comp rx=0 tx=0).stop 2024-04-25T09:01:46.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 -- 172.21.15.37:0/3181672224 shutdown_connections 2024-04-25T09:01:46.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 --2- 172.21.15.37:0/3181672224 >> v2:172.21.15.203:3300/0 conn(0x7f6ed40ce170 0x7f6ed40ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 --2- 172.21.15.37:0/3181672224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed40cf590 0x7f6ed415ecb0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 -- 172.21.15.37:0/3181672224 >> 172.21.15.37:0/3181672224 conn(0x7f6ed406ae60 msgr2=0x7f6ed406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:46.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 -- 172.21.15.37:0/3181672224 shutdown_connections 2024-04-25T09:01:46.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 -- 172.21.15.37:0/3181672224 wait complete. 2024-04-25T09:01:46.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 Processor -- start 2024-04-25T09:01:46.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 -- start start 2024-04-25T09:01:46.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed40ce170 0x7f6ed4097910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed40cf590 0x7f6ed4098020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ed40cf970 0x7f6ed4098560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed40ca240 con 0x7f6ed40ce170 2024-04-25T09:01:46.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed40ca0c0 con 0x7f6ed40cf590 2024-04-25T09:01:46.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edc7f4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed40ac170 con 0x7f6ed40cf970 2024-04-25T09:01:46.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6eda590640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed40ce170 0x7f6ed4097910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6eda590640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed40ce170 0x7f6ed4097910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52030/0 (socket says 172.21.15.37:52030) 2024-04-25T09:01:46.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6eda590640 1 -- 172.21.15.37:0/863592917 learned_addr learned my addr 172.21.15.37:0/863592917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:46.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6ed9d8f640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed40cf590 0x7f6ed4098020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6edad91640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ed40cf970 0x7f6ed4098560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6eda590640 1 -- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ed40cf970 msgr2=0x7f6ed4098560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6eda590640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ed40cf970 0x7f6ed4098560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6eda590640 1 -- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed40cf590 msgr2=0x7f6ed4098020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6eda590640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed40cf590 0x7f6ed4098020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6eda590640 1 -- 172.21.15.37:0/863592917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed4159e10 con 0x7f6ed40ce170 2024-04-25T09:01:46.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6eda590640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed40ce170 0x7f6ed4097910 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00ad040 tx=0x7f6ed001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:46.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.310+0000 7f6ecaffd640 1 -- 172.21.15.37:0/863592917 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed00be5f0 con 0x7f6ed40ce170 2024-04-25T09:01:46.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.314+0000 7f6ecaffd640 1 -- 172.21.15.37:0/863592917 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed00be7b0 con 0x7f6ed40ce170 2024-04-25T09:01:46.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.314+0000 7f6edc7f4640 1 -- 172.21.15.37:0/863592917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed4158f00 con 0x7f6ed40ce170 2024-04-25T09:01:46.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.314+0000 7f6ecaffd640 1 -- 172.21.15.37:0/863592917 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed00beba0 con 0x7f6ed40ce170 2024-04-25T09:01:46.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.314+0000 7f6edc7f4640 1 -- 172.21.15.37:0/863592917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed4159aa0 con 0x7f6ed40ce170 2024-04-25T09:01:46.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.314+0000 7f6edc7f4640 1 -- 172.21.15.37:0/863592917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed404efa0 con 0x7f6ed40ce170 2024-04-25T09:01:46.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.314+0000 7f6ecaffd640 1 -- 172.21.15.37:0/863592917 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6ed00c7430 con 0x7f6ed40ce170 2024-04-25T09:01:46.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.314+0000 7f6ecaffd640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6eb8066f60 0x7f6eb8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.314+0000 7f6ecaffd640 1 -- 172.21.15.37:0/863592917 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ed00a8030 con 0x7f6ed40ce170 2024-04-25T09:01:46.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.314+0000 7f6ed9d8f640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6eb8066f60 0x7f6eb8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.314+0000 7f6ed9d8f640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6eb8066f60 0x7f6eb8069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40c9060 tx=0x7f6ec4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:46.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.318+0000 7f6ecaffd640 1 -- 172.21.15.37:0/863592917 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6ed0016110 con 0x7f6ed40ce170 2024-04-25T09:01:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.470+0000 7f6edc7f4640 1 -- 172.21.15.37:0/863592917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6ed40ca4c0 con 0x7f6ed40ce170 2024-04-25T09:01:46.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6ecaffd640 1 -- 172.21.15.37:0/863592917 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ed00f0e00 con 0x7f6ed40ce170 2024-04-25T09:01:46.462 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:01:46.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 -- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6eb8066f60 msgr2=0x7f6eb8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6eb8066f60 0x7f6eb8069420 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40c9060 tx=0x7f6ec4009040 comp rx=0 tx=0).stop 2024-04-25T09:01:46.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 -- 172.21.15.37:0/863592917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed40ce170 msgr2=0x7f6ed4097910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed40ce170 0x7f6ed4097910 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f6ed00ad040 tx=0x7f6ed001ad60 comp rx=0 tx=0).stop 2024-04-25T09:01:46.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 -- 172.21.15.37:0/863592917 shutdown_connections 2024-04-25T09:01:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6eb8066f60 0x7f6eb8069420 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6ed40cf970 0x7f6ed4098560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6ed40cf590 0x7f6ed4098020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 --2- 172.21.15.37:0/863592917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ed40ce170 0x7f6ed4097910 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 -- 172.21.15.37:0/863592917 >> 172.21.15.37:0/863592917 conn(0x7f6ed406ae60 msgr2=0x7f6ed406cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 -- 172.21.15.37:0/863592917 shutdown_connections 2024-04-25T09:01:46.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.474+0000 7f6edc7f4640 1 -- 172.21.15.37:0/863592917 wait complete. 2024-04-25T09:01:46.479 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:01:46.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f09cde640 1 Processor -- start 2024-04-25T09:01:46.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f09cde640 1 -- start start 2024-04-25T09:01:46.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f09cde640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f0415da80 0x7f3f040d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f09cde640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d07f0 0x7f3f0406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f09cde640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f0406be60 con 0x7f3f040d1cd0 2024-04-25T09:01:46.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f09cde640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0406c440 con 0x7f3f0415da80 2024-04-25T09:01:46.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f09cde640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0415de60 con 0x7f3f040d07f0 2024-04-25T09:01:46.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f037fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3f0415da80 0x7f3f040d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d07f0 0x7f3f0406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f03fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d07f0 0x7f3f0406c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:59886/0 (socket says 172.21.15.37:59886) 2024-04-25T09:01:46.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f03fff640 1 -- 172.21.15.37:0/2885304332 learned_addr learned my addr 172.21.15.37:0/2885304332 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f03fff640 1 -- 172.21.15.37:0/2885304332 >> v2:172.21.15.203:3300/0 conn(0x7f3f0415da80 msgr2=0x7f3f040d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f03fff640 1 --2- 172.21.15.37:0/2885304332 >> v2:172.21.15.203:3300/0 conn(0x7f3f0415da80 0x7f3f040d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f03fff640 1 -- 172.21.15.37:0/2885304332 >> v1:172.21.15.203:6789/0 conn(0x7f3f040d1cd0 legacy=0x7f3f040d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.614+0000 7f3f03fff640 1 -- 172.21.15.37:0/2885304332 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f040d2450 con 0x7f3f040d07f0 2024-04-25T09:01:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f03fff640 1 --2- 172.21.15.37:0/2885304332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d07f0 0x7f3f0406c8d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009e80 tx=0x7f3efc01c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b8e8c06e8de7a21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:46.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f027fc640 1 -- 172.21.15.37:0/2885304332 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc0a5810 con 0x7f3f040d07f0 2024-04-25T09:01:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f027fc640 1 -- 172.21.15.37:0/2885304332 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3efc0a59b0 con 0x7f3f040d07f0 2024-04-25T09:01:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f027fc640 1 -- 172.21.15.37:0/2885304332 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3efc0a5d80 con 0x7f3f040d07f0 2024-04-25T09:01:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f09cde640 1 -- 172.21.15.37:0/2885304332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d07f0 msgr2=0x7f3f0406c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f09cde640 1 --2- 172.21.15.37:0/2885304332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d07f0 0x7f3f0406c8d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3efc009e80 tx=0x7f3efc01c9e0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f09cde640 1 -- 172.21.15.37:0/2885304332 shutdown_connections 2024-04-25T09:01:46.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f09cde640 1 --2- 172.21.15.37:0/2885304332 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d07f0 0x7f3f0406c8d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f09cde640 1 --2- 172.21.15.37:0/2885304332 >> v2:172.21.15.203:3300/0 conn(0x7f3f0415da80 0x7f3f040d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f09cde640 1 -- 172.21.15.37:0/2885304332 >> 172.21.15.37:0/2885304332 conn(0x7f3f0408b000 msgr2=0x7f3f040d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:46.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f09cde640 1 -- 172.21.15.37:0/2885304332 shutdown_connections 2024-04-25T09:01:46.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f09cde640 1 -- 172.21.15.37:0/2885304332 wait complete. 2024-04-25T09:01:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.618+0000 7f3f09cde640 1 Processor -- start 2024-04-25T09:01:46.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f09cde640 1 -- start start 2024-04-25T09:01:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f09cde640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f040d07f0 0x7f3f0415cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f09cde640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d1cd0 0x7f3f040ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f09cde640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f0415da80 0x7f3f040b6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f09cde640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0406d7a0 con 0x7f3f040d07f0 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f09cde640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0406d620 con 0x7f3f0415da80 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f09cde640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f0406d920 con 0x7f3f040d1cd0 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f040d07f0 0x7f3f0415cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f040d07f0 0x7f3f0415cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52036/0 (socket says 172.21.15.37:52036) 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f037fe640 1 -- 172.21.15.37:0/132061153 learned_addr learned my addr 172.21.15.37:0/132061153 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f02ffd640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d1cd0 0x7f3f040ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f03fff640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f0415da80 0x7f3f040b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f037fe640 1 -- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d1cd0 msgr2=0x7f3f040ccf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f037fe640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d1cd0 0x7f3f040ccf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f037fe640 1 -- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f0415da80 msgr2=0x7f3f040b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f037fe640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f0415da80 0x7f3f040b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f037fe640 1 -- 172.21.15.37:0/132061153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f040cab90 con 0x7f3f040d07f0 2024-04-25T09:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f037fe640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f040d07f0 0x7f3f0415cef0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400e870 tx=0x7f3ef400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3eebfff640 1 -- 172.21.15.37:0/132061153 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef4017070 con 0x7f3f040d07f0 2024-04-25T09:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3eebfff640 1 -- 172.21.15.37:0/132061153 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef400c940 con 0x7f3f040d07f0 2024-04-25T09:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f09cde640 1 -- 172.21.15.37:0/132061153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f040cae70 con 0x7f3f040d07f0 2024-04-25T09:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3eebfff640 1 -- 172.21.15.37:0/132061153 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef400ccf0 con 0x7f3f040d07f0 2024-04-25T09:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f09cde640 1 -- 172.21.15.37:0/132061153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f040cb430 con 0x7f3f040d07f0 2024-04-25T09:01:46.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3f09cde640 1 -- 172.21.15.37:0/132061153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f0404efa0 con 0x7f3f040d07f0 2024-04-25T09:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3eebfff640 1 -- 172.21.15.37:0/132061153 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ef401a070 con 0x7f3f040d07f0 2024-04-25T09:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3eebfff640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ef0066f60 0x7f3ef0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.622+0000 7f3eebfff640 1 -- 172.21.15.37:0/132061153 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ef4085310 con 0x7f3f040d07f0 2024-04-25T09:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.626+0000 7f3f02ffd640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ef0066f60 0x7f3ef0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.626+0000 7f3f02ffd640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ef0066f60 0x7f3ef0069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f04156d00 tx=0x7f3ef8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:46.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.626+0000 7f3eebfff640 1 -- 172.21.15.37:0/132061153 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ef4055bd0 con 0x7f3f040d07f0 2024-04-25T09:01:46.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.782+0000 7f3f09cde640 1 -- 172.21.15.37:0/132061153 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3f0415a5b0 con 0x7f3f040d07f0 2024-04-25T09:01:46.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.782+0000 7f3eebfff640 1 -- 172.21.15.37:0/132061153 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ef4059c70 con 0x7f3f040d07f0 2024-04-25T09:01:46.771 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:01:46.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 -- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ef0066f60 msgr2=0x7f3ef0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ef0066f60 0x7f3ef0069420 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3f04156d00 tx=0x7f3ef8009040 comp rx=0 tx=0).stop 2024-04-25T09:01:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 -- 172.21.15.37:0/132061153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f040d07f0 msgr2=0x7f3f0415cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f040d07f0 0x7f3f0415cef0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f3ef400e870 tx=0x7f3ef400ec70 comp rx=0 tx=0).stop 2024-04-25T09:01:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 -- 172.21.15.37:0/132061153 shutdown_connections 2024-04-25T09:01:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ef0066f60 0x7f3ef0069420 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3f0415da80 0x7f3f040b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3f040d1cd0 0x7f3f040ccf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 --2- 172.21.15.37:0/132061153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f040d07f0 0x7f3f0415cef0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 -- 172.21.15.37:0/132061153 >> 172.21.15.37:0/132061153 conn(0x7f3f0408b000 msgr2=0x7f3f0415e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 -- 172.21.15.37:0/132061153 shutdown_connections 2024-04-25T09:01:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.786+0000 7f3f09cde640 1 -- 172.21.15.37:0/132061153 wait complete. 2024-04-25T09:01:46.789 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:01:46.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5e608640 1 Processor -- start 2024-04-25T09:01:46.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5e608640 1 -- start start 2024-04-25T09:01:46.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5e608640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5815da80 0x7f6c580d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5e608640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c580d07f0 0x7f6c5806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5e608640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5806be60 con 0x7f6c580d1cd0 2024-04-25T09:01:46.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5e608640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5806c440 con 0x7f6c580d07f0 2024-04-25T09:01:46.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5e608640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5815de60 con 0x7f6c5815da80 2024-04-25T09:01:46.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5cba5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c580d07f0 0x7f6c5806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5cba5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c580d07f0 0x7f6c5806c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52040/0 (socket says 172.21.15.37:52040) 2024-04-25T09:01:46.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5cba5640 1 -- 172.21.15.37:0/3318799030 learned_addr learned my addr 172.21.15.37:0/3318799030 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:46.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c57fff640 1 --2- 172.21.15.37:0/3318799030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5815da80 0x7f6c580d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5cba5640 1 -- 172.21.15.37:0/3318799030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5815da80 msgr2=0x7f6c580d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:46.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5cba5640 1 --2- 172.21.15.37:0/3318799030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5815da80 0x7f6c580d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5cba5640 1 -- 172.21.15.37:0/3318799030 >> v1:172.21.15.203:6789/0 conn(0x7f6c580d1cd0 legacy=0x7f6c580d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:46.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.918+0000 7f6c5cba5640 1 -- 172.21.15.37:0/3318799030 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580d2450 con 0x7f6c580d07f0 2024-04-25T09:01:46.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5cba5640 1 --2- 172.21.15.37:0/3318799030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c580d07f0 0x7f6c5806c8d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009e50 tx=0x7f6c5001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fd0671e37399d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:46.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c577fe640 1 -- 172.21.15.37:0/3318799030 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c500a5800 con 0x7f6c580d07f0 2024-04-25T09:01:46.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c577fe640 1 -- 172.21.15.37:0/3318799030 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c500a59a0 con 0x7f6c580d07f0 2024-04-25T09:01:46.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c577fe640 1 -- 172.21.15.37:0/3318799030 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c500a5d70 con 0x7f6c580d07f0 2024-04-25T09:01:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 -- 172.21.15.37:0/3318799030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c580d07f0 msgr2=0x7f6c5806c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 --2- 172.21.15.37:0/3318799030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c580d07f0 0x7f6c5806c8d0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009e50 tx=0x7f6c5001ca00 comp rx=0 tx=0).stop 2024-04-25T09:01:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 -- 172.21.15.37:0/3318799030 shutdown_connections 2024-04-25T09:01:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 --2- 172.21.15.37:0/3318799030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c580d07f0 0x7f6c5806c8d0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 --2- 172.21.15.37:0/3318799030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5815da80 0x7f6c580d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 -- 172.21.15.37:0/3318799030 >> 172.21.15.37:0/3318799030 conn(0x7f6c5808b000 msgr2=0x7f6c580d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 -- 172.21.15.37:0/3318799030 shutdown_connections 2024-04-25T09:01:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 -- 172.21.15.37:0/3318799030 wait complete. 2024-04-25T09:01:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 Processor -- start 2024-04-25T09:01:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 -- start start 2024-04-25T09:01:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c580d07f0 0x7f6c5815c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c580d1cd0 0x7f6c58160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5815da80 0x7f6c580b3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5806d7a0 con 0x7f6c580d1cd0 2024-04-25T09:01:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5806d620 con 0x7f6c580d07f0 2024-04-25T09:01:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5806d920 con 0x7f6c5815da80 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c4ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c580d1cd0 0x7f6c58160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c4ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c580d1cd0 0x7f6c58160870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52044/0 (socket says 172.21.15.37:52044) 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c4ffff640 1 -- 172.21.15.37:0/302763437 learned_addr learned my addr 172.21.15.37:0/302763437 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c57fff640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c580d07f0 0x7f6c5815c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5cba5640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5815da80 0x7f6c580b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c4ffff640 1 -- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5815da80 msgr2=0x7f6c580b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c4ffff640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5815da80 0x7f6c580b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c4ffff640 1 -- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c580d07f0 msgr2=0x7f6c5815c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c4ffff640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c580d07f0 0x7f6c5815c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c4ffff640 1 -- 172.21.15.37:0/302763437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5815e200 con 0x7f6c580d1cd0 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c4ffff640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c580d1cd0 0x7f6c58160870 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000c430 tx=0x7f6c4000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c557fa640 1 -- 172.21.15.37:0/302763437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400052d0 con 0x7f6c580d1cd0 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c557fa640 1 -- 172.21.15.37:0/302763437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c400054b0 con 0x7f6c580d1cd0 2024-04-25T09:01:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 -- 172.21.15.37:0/302763437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5815e4e0 con 0x7f6c580d1cd0 2024-04-25T09:01:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c557fa640 1 -- 172.21.15.37:0/302763437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40006a70 con 0x7f6c580d1cd0 2024-04-25T09:01:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.922+0000 7f6c5e608640 1 -- 172.21.15.37:0/302763437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5815eaa0 con 0x7f6c580d1cd0 2024-04-25T09:01:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.926+0000 7f6c5e608640 1 -- 172.21.15.37:0/302763437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5804efa0 con 0x7f6c580d1cd0 2024-04-25T09:01:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.926+0000 7f6c557fa640 1 -- 172.21.15.37:0/302763437 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c40006cd0 con 0x7f6c580d1cd0 2024-04-25T09:01:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.926+0000 7f6c557fa640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6c3c066f60 0x7f6c3c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.926+0000 7f6c557fa640 1 -- 172.21.15.37:0/302763437 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c400847d0 con 0x7f6c580d1cd0 2024-04-25T09:01:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.926+0000 7f6c57fff640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6c3c066f60 0x7f6c3c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.926+0000 7f6c57fff640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6c3c066f60 0x7f6c3c069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6c5814b8d0 tx=0x7f6c48008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:46.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:46.930+0000 7f6c557fa640 1 -- 172.21.15.37:0/302763437 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c400555b0 con 0x7f6c580d1cd0 2024-04-25T09:01:47.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 -- 172.21.15.37:0/302763437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c580b19c0 con 0x7f6c580d1cd0 2024-04-25T09:01:47.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c557fa640 1 -- 172.21.15.37:0/302763437 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c40054eb0 con 0x7f6c580d1cd0 2024-04-25T09:01:47.070 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:01:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 -- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6c3c066f60 msgr2=0x7f6c3c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6c3c066f60 0x7f6c3c069420 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f6c5814b8d0 tx=0x7f6c48008040 comp rx=0 tx=0).stop 2024-04-25T09:01:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 -- 172.21.15.37:0/302763437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c580d1cd0 msgr2=0x7f6c58160870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c580d1cd0 0x7f6c58160870 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000c430 tx=0x7f6c4000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 -- 172.21.15.37:0/302763437 shutdown_connections 2024-04-25T09:01:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6c3c066f60 0x7f6c3c069420 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c5815da80 0x7f6c580b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c580d1cd0 0x7f6c58160870 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 --2- 172.21.15.37:0/302763437 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c580d07f0 0x7f6c5815c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 -- 172.21.15.37:0/302763437 >> 172.21.15.37:0/302763437 conn(0x7f6c5808b000 msgr2=0x7f6c580d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 -- 172.21.15.37:0/302763437 shutdown_connections 2024-04-25T09:01:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.082+0000 7f6c5e608640 1 -- 172.21.15.37:0/302763437 wait complete. 2024-04-25T09:01:47.087 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:01:47.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.210+0000 7fae9b537640 1 Processor -- start 2024-04-25T09:01:47.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.210+0000 7fae9b537640 1 -- start start 2024-04-25T09:01:47.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.210+0000 7fae9b537640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fae940cfb80 0x7fae940cff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:47.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.210+0000 7fae9b537640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae940d04d0 con 0x7fae9409a480 2024-04-25T09:01:47.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.210+0000 7fae9b537640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae9415bd30 con 0x7fae9415b950 2024-04-25T09:01:47.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.210+0000 7fae9b537640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae940cc0d0 con 0x7fae940cfb80 2024-04-25T09:01:47.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.210+0000 7fae98ad2640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fae9409a480 0x7fae940cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:41046/0 (socket says 172.21.15.37:41046) 2024-04-25T09:01:47.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.210+0000 7fae98ad2640 1 -- 172.21.15.37:0/3730600480 learned_addr learned my addr 172.21.15.37:0/3730600480 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:47.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae99ad4640 1 --2- 172.21.15.37:0/3730600480 >> v2:172.21.15.203:3300/0 conn(0x7fae940cfb80 0x7fae940cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:47.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2687609585 0 0) 0x7fae84001120 con 0x7fae9409a480 2024-04-25T09:01:47.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae80003530 con 0x7fae9409a480 2024-04-25T09:01:47.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 9946240 0 0) 0x7fae80003530 con 0x7fae9409a480 2024-04-25T09:01:47.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 >> v2:172.21.15.203:3300/0 conn(0x7fae940cfb80 msgr2=0x7fae940cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:47.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 --2- 172.21.15.37:0/3730600480 >> v2:172.21.15.203:3300/0 conn(0x7fae940cfb80 0x7fae940cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 >> v1:172.21.15.203:6789/0 conn(0x7fae9415b950 legacy=0x7fae940cf420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae9409ab10 con 0x7fae9409a480 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fae84001850 con 0x7fae9409a480 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 >> v1:172.21.15.37:6789/0 conn(0x7fae9409a480 legacy=0x7fae940cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 --2- 172.21.15.37:0/3730600480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae80003a30 0x7fae80003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 --2- 172.21.15.37:0/3730600480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae80004810 0x7fae80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 --2- 172.21.15.37:0/3730600480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae800051d0 0x7fae8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae940cc0d0 con 0x7fae800051d0 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8000c100 con 0x7fae80004810 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8000c300 con 0x7fae80003a30 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae99ad4640 1 --2- 172.21.15.37:0/3730600480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae800051d0 0x7fae8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae992d3640 1 --2- 172.21.15.37:0/3730600480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae80003a30 0x7fae80003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae99ad4640 1 -- 172.21.15.37:0/3730600480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae80003a30 msgr2=0x7fae80003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae99ad4640 1 --2- 172.21.15.37:0/3730600480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae80003a30 0x7fae80003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae99ad4640 1 -- 172.21.15.37:0/3730600480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae80004810 msgr2=0x7fae80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:47.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae99ad4640 1 --2- 172.21.15.37:0/3730600480 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae80004810 0x7fae80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae99ad4640 1 -- 172.21.15.37:0/3730600480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80003530 con 0x7fae800051d0 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae992d3640 1 --2- 172.21.15.37:0/3730600480 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae80003a30 0x7fae80003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae99ad4640 1 -- 172.21.15.37:0/3730600480 reap_dead start 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae99ad4640 1 --2- 172.21.15.37:0/3730600480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae800051d0 0x7fae8000b9d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009fc0 tx=0x7fae8c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d538cd89b4517f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c009c10 con 0x7fae800051d0 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c0a5d10 con 0x7fae800051d0 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae8bfff640 1 -- 172.21.15.37:0/3730600480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c003be0 con 0x7fae800051d0 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae9b537640 1 -- 172.21.15.37:0/3730600480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae800051d0 msgr2=0x7fae8000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.214+0000 7fae9b537640 1 --2- 172.21.15.37:0/3730600480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae800051d0 0x7fae8000b9d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009fc0 tx=0x7fae8c01ccc0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 -- 172.21.15.37:0/3730600480 shutdown_connections 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 --2- 172.21.15.37:0/3730600480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae800051d0 0x7fae8000b9d0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 -- 172.21.15.37:0/3730600480 >> 172.21.15.37:0/3730600480 conn(0x7fae940b30b0 msgr2=0x7fae940b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 -- 172.21.15.37:0/3730600480 shutdown_connections 2024-04-25T09:01:47.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 -- 172.21.15.37:0/3730600480 wait complete. 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 Processor -- start 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 -- start start 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae800051d0 0x7fae941f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae9415bf30 0x7fae940c50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae940c5630 0x7fae940af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae940cc650 con 0x7fae940c5630 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae9409a910 con 0x7fae9415bf30 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae9b537640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae940cc930 con 0x7fae800051d0 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae99ad4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae940c5630 0x7fae940af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae99ad4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae940c5630 0x7fae940af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52054/0 (socket says 172.21.15.37:52054) 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae99ad4640 1 -- 172.21.15.37:0/820571458 learned_addr learned my addr 172.21.15.37:0/820571458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:47.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae98ad2640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae9415bf30 0x7fae940c50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae992d3640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae800051d0 0x7fae941f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae99ad4640 1 -- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae800051d0 msgr2=0x7fae941f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae99ad4640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae800051d0 0x7fae941f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae99ad4640 1 -- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae9415bf30 msgr2=0x7fae940c50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae99ad4640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae9415bf30 0x7fae940c50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae99ad4640 1 -- 172.21.15.37:0/820571458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae940c5c60 con 0x7fae940c5630 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae98ad2640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae9415bf30 0x7fae940c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae99ad4640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae940c5630 0x7fae940af5d0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0987f0 tx=0x7fae8c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae89ffb640 1 -- 172.21.15.37:0/820571458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0aed50 con 0x7fae940c5630 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae89ffb640 1 -- 172.21.15.37:0/820571458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c0bd070 con 0x7fae940c5630 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.218+0000 7fae89ffb640 1 -- 172.21.15.37:0/820571458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c003ba0 con 0x7fae940c5630 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.222+0000 7fae9b537640 1 -- 172.21.15.37:0/820571458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae940c5f40 con 0x7fae940c5630 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.222+0000 7fae9b537640 1 -- 172.21.15.37:0/820571458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae940c6c20 con 0x7fae940c5630 2024-04-25T09:01:47.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.222+0000 7fae9b537640 1 -- 172.21.15.37:0/820571458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae9404efa0 con 0x7fae940c5630 2024-04-25T09:01:47.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.222+0000 7fae89ffb640 1 -- 172.21.15.37:0/820571458 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fae8c004030 con 0x7fae940c5630 2024-04-25T09:01:47.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.222+0000 7fae89ffb640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fae78066f90 0x7fae78069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:47.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.222+0000 7fae89ffb640 1 -- 172.21.15.37:0/820571458 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fae8c120d80 con 0x7fae940c5630 2024-04-25T09:01:47.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.222+0000 7fae992d3640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fae78066f90 0x7fae78069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:47.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.222+0000 7fae992d3640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fae78066f90 0x7fae78069450 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fae90000c10 tx=0x7fae90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:47.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.226+0000 7fae89ffb640 1 -- 172.21.15.37:0/820571458 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fae8c0169d0 con 0x7fae940c5630 2024-04-25T09:01:47.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.390+0000 7fae9b537640 1 -- 172.21.15.37:0/820571458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fae940d13a0 con 0x7fae940c5630 2024-04-25T09:01:47.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.390+0000 7fae89ffb640 1 -- 172.21.15.37:0/820571458 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7fae8c0f1d20 con 0x7fae940c5630 2024-04-25T09:01:47.380 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T09:01:47.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 -- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fae78066f90 msgr2=0x7fae78069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:47.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fae78066f90 0x7fae78069450 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fae90000c10 tx=0x7fae90009040 comp rx=0 tx=0).stop 2024-04-25T09:01:47.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 -- 172.21.15.37:0/820571458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae940c5630 msgr2=0x7fae940af5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:47.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae940c5630 0x7fae940af5d0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fae8c0987f0 tx=0x7fae8c009750 comp rx=0 tx=0).stop 2024-04-25T09:01:47.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 -- 172.21.15.37:0/820571458 shutdown_connections 2024-04-25T09:01:47.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fae78066f90 0x7fae78069450 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae940c5630 0x7fae940af5d0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fae9415bf30 0x7fae940c50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 --2- 172.21.15.37:0/820571458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fae800051d0 0x7fae941f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:47.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 -- 172.21.15.37:0/820571458 >> 172.21.15.37:0/820571458 conn(0x7fae940b30b0 msgr2=0x7fae9415ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:47.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 -- 172.21.15.37:0/820571458 shutdown_connections 2024-04-25T09:01:47.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:47.394+0000 7fae9b537640 1 -- 172.21.15.37:0/820571458 wait complete. 2024-04-25T09:01:47.398 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T09:01:47.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:47.398 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:47.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[51611]: fuse finished with error 0 and tester_r 0 2024-04-25T09:01:47.412 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:01:53.514 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:01:53.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:53.515 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:01:53.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:53.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c9dbfd640 1 Processor -- start 2024-04-25T09:01:53.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c9dbfd640 1 -- start start 2024-04-25T09:01:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c9dbfd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9806c3b0 0x7f4c98168730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c9dbfd640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9815e2e0 con 0x7f4c980af390 2024-04-25T09:01:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c9dbfd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9815e5c0 con 0x7f4c9815dd50 2024-04-25T09:01:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c9dbfd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980b2f10 con 0x7f4c9806c3b0 2024-04-25T09:01:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c977fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f4c980af390 0x7f4c98066cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38320/0 (socket says 172.21.15.37:38320) 2024-04-25T09:01:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c977fe640 1 -- 172.21.15.37:0/897021080 learned_addr learned my addr 172.21.15.37:0/897021080 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c97fff640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9806c3b0 0x7f4c98168730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676069975 0 0) 0x7f4c9815e2e0 con 0x7f4c980af390 2024-04-25T09:01:53.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c80003530 con 0x7f4c980af390 2024-04-25T09:01:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1129896764 0 0) 0x7f4c80003530 con 0x7f4c980af390 2024-04-25T09:01:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9806c3b0 msgr2=0x7f4c98168730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9806c3b0 0x7f4c98168730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 >> v1:172.21.15.203:6789/0 conn(0x7f4c9815dd50 legacy=0x7f4c9806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9815e8a0 con 0x7f4c980af390 2024-04-25T09:01:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f4c88001680 con 0x7f4c980af390 2024-04-25T09:01:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 >> v1:172.21.15.37:6789/0 conn(0x7f4c980af390 legacy=0x7f4c98066cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c80003a60 0x7f4c80003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:53.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c80004810 0x7f4c80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9806c3b0 0x7f4c8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c80005560 con 0x7f4c80003a60 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8000bf10 con 0x7f4c80004810 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c8000c090 con 0x7f4c9806c3b0 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c977fe640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c80003a60 0x7f4c80003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c96ffd640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c80004810 0x7f4c80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c97fff640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9806c3b0 0x7f4c8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c977fe640 1 -- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9806c3b0 msgr2=0x7f4c8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c977fe640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9806c3b0 0x7f4c8000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c977fe640 1 -- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c80004810 msgr2=0x7f4c80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c977fe640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c80004810 0x7f4c80004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c977fe640 1 -- 172.21.15.37:0/897021080 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c80003530 con 0x7f4c80003a60 2024-04-25T09:01:53.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.710+0000 7f4c97fff640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c9806c3b0 0x7f4c8000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c977fe640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c80003a60 0x7f4c80003e80 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800c160 tx=0x7f4c8801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a13773b5fa642e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88007c70 con 0x7f4c80003a60 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c880a9070 con 0x7f4c80003a60 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c967fc640 1 -- 172.21.15.37:0/897021080 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88005d20 con 0x7f4c80003a60 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/897021080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c80003a60 msgr2=0x7f4c80003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c9dbfd640 1 --2- 172.21.15.37:0/897021080 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c80003a60 0x7f4c80003e80 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800c160 tx=0x7f4c8801cbc0 comp rx=0 tx=0).stop 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c97fff640 1 -- 172.21.15.37:0/897021080 reap_dead start 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/897021080 shutdown_connections 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/897021080 >> 172.21.15.37:0/897021080 conn(0x7f4c980717f0 msgr2=0x7f4c98071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/897021080 shutdown_connections 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/897021080 wait complete. 2024-04-25T09:01:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c9dbfd640 1 Processor -- start 2024-04-25T09:01:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.714+0000 7f4c9dbfd640 1 -- start start 2024-04-25T09:01:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c9dbfd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c80003a60 0x7f4c981f9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c9dbfd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c981fac20 0x7f4c981fa530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c9dbfd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c981564e0 0x7f4c98140d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c9dbfd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c981692d0 con 0x7f4c981fac20 2024-04-25T09:01:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c9dbfd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c98169150 con 0x7f4c981564e0 2024-04-25T09:01:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c9dbfd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c98169450 con 0x7f4c80003a60 2024-04-25T09:01:53.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c96ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c981fac20 0x7f4c981fa530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c96ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c981fac20 0x7f4c981fa530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52650/0 (socket says 172.21.15.37:52650) 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c96ffd640 1 -- 172.21.15.37:0/791753123 learned_addr learned my addr 172.21.15.37:0/791753123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c977fe640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c80003a60 0x7f4c981f9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c97fff640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c981564e0 0x7f4c98140d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c96ffd640 1 -- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c80003a60 msgr2=0x7f4c981f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c96ffd640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c80003a60 0x7f4c981f9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c96ffd640 1 -- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c981564e0 msgr2=0x7f4c98140d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c96ffd640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c981564e0 0x7f4c98140d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c96ffd640 1 -- 172.21.15.37:0/791753123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c98163260 con 0x7f4c981fac20 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c97fff640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c981564e0 0x7f4c98140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c96ffd640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c981fac20 0x7f4c981fa530 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00f5f0 tx=0x7f4c8c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c7ffff640 1 -- 172.21.15.37:0/791753123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c0052e0 con 0x7f4c981fac20 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c7ffff640 1 -- 172.21.15.37:0/791753123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c018070 con 0x7f4c981fac20 2024-04-25T09:01:53.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/791753123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c98163540 con 0x7f4c981fac20 2024-04-25T09:01:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c7ffff640 1 -- 172.21.15.37:0/791753123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c003820 con 0x7f4c981fac20 2024-04-25T09:01:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/791753123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c981412c0 con 0x7f4c981fac20 2024-04-25T09:01:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/791753123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9804efa0 con 0x7f4c981fac20 2024-04-25T09:01:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c7ffff640 1 -- 172.21.15.37:0/791753123 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4c8c0039c0 con 0x7f4c981fac20 2024-04-25T09:01:53.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c7ffff640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4c84066f90 0x7f4c84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:53.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c7ffff640 1 -- 172.21.15.37:0/791753123 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c8c084f20 con 0x7f4c981fac20 2024-04-25T09:01:53.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.718+0000 7f4c977fe640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4c84066f90 0x7f4c84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:53.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.722+0000 7f4c977fe640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4c84066f90 0x7f4c84069450 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4c88007fd0 tx=0x7f4c880b1040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:53.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.722+0000 7f4c7ffff640 1 -- 172.21.15.37:0/791753123 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4c8c005480 con 0x7f4c981fac20 2024-04-25T09:01:53.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.918+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/791753123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c980cec40 con 0x7f4c981fac20 2024-04-25T09:01:53.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.918+0000 7f4c7ffff640 1 -- 172.21.15.37:0/791753123 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f4c8c0556b0 con 0x7f4c981fac20 2024-04-25T09:01:53.908 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:53.908 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:01:37.945874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8727":{"gid":8727,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1266122577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1266122577},{"type":"v1","addr":"172.21.15.37:6835","nonce":1266122577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:01:53.908 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 81 2024-04-25T09:01:53.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.918+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4c84066f90 msgr2=0x7f4c84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:53.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.918+0000 7f4c9dbfd640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4c84066f90 0x7f4c84069450 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4c88007fd0 tx=0x7f4c880b1040 comp rx=0 tx=0).stop 2024-04-25T09:01:53.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.922+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/791753123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c981fac20 msgr2=0x7f4c981fa530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:53.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.922+0000 7f4c9dbfd640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c981fac20 0x7f4c981fa530 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00f5f0 tx=0x7f4c8c00fab0 comp rx=0 tx=0).stop 2024-04-25T09:01:53.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.922+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/791753123 shutdown_connections 2024-04-25T09:01:53.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.922+0000 7f4c9dbfd640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4c84066f90 0x7f4c84069450 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:53.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.922+0000 7f4c9dbfd640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f4c981564e0 0x7f4c98140d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:53.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.922+0000 7f4c9dbfd640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c981fac20 0x7f4c981fa530 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:53.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.922+0000 7f4c9dbfd640 1 --2- 172.21.15.37:0/791753123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f4c80003a60 0x7f4c981f9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:53.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.922+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/791753123 >> 172.21.15.37:0/791753123 conn(0x7f4c980717f0 msgr2=0x7f4c980cf3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:53.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.922+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/791753123 shutdown_connections 2024-04-25T09:01:53.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:53.922+0000 7f4c9dbfd640 1 -- 172.21.15.37:0/791753123 wait complete. 2024-04-25T09:01:53.927 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T09:01:54.079 INFO:teuthology.orchestra.run.smithi037.stdout:{"message":"","return_code":0} 2024-04-25T09:01:54.080 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:01:54.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:54.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.210+0000 7fd6b3601640 1 Processor -- start 2024-04-25T09:01:54.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.210+0000 7fd6b3601640 1 -- start start 2024-04-25T09:01:54.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.210+0000 7fd6b3601640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd6ac15a280 0x7fd6ac15a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.210+0000 7fd6b3601640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ac15ae30 con 0x7fd6ac159ea0 2024-04-25T09:01:54.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.210+0000 7fd6b3601640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ac15b110 con 0x7fd6ac0718c0 2024-04-25T09:01:54.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.210+0000 7fd6b3601640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ac0b2990 con 0x7fd6ac15a280 2024-04-25T09:01:54.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.210+0000 7fd6b0b9c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd6ac15a280 0x7fd6ac15a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b139d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd6ac159ea0 0x7fd6ac0b2e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38336/0 (socket says 172.21.15.37:38336) 2024-04-25T09:01:54.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b139d640 1 -- 172.21.15.37:0/982240680 learned_addr learned my addr 172.21.15.37:0/982240680 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:54.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b0b9c640 1 -- 172.21.15.37:0/982240680 >> v1:172.21.15.203:6789/0 conn(0x7fd6ac0718c0 legacy=0x7fd6ac06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:54.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b0b9c640 1 -- 172.21.15.37:0/982240680 >> v1:172.21.15.37:6789/0 conn(0x7fd6ac159ea0 legacy=0x7fd6ac0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b0b9c640 1 -- 172.21.15.37:0/982240680 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac0b2620 con 0x7fd6ac15a280 2024-04-25T09:01:54.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b0b9c640 1 --2- 172.21.15.37:0/982240680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15a280 0x7fd6ac15a700 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00ba60 tx=0x7fd69c01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cd7f36ab80003e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6a3fff640 1 -- 172.21.15.37:0/982240680 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0b4070 con 0x7fd6ac15a280 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6a3fff640 1 -- 172.21.15.37:0/982240680 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c0a7b10 con 0x7fd6ac15a280 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6a3fff640 1 -- 172.21.15.37:0/982240680 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c0af440 con 0x7fd6ac15a280 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b3601640 1 -- 172.21.15.37:0/982240680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15a280 msgr2=0x7fd6ac15a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b3601640 1 --2- 172.21.15.37:0/982240680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15a280 0x7fd6ac15a700 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00ba60 tx=0x7fd69c01eb30 comp rx=0 tx=0).stop 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b3601640 1 -- 172.21.15.37:0/982240680 shutdown_connections 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b3601640 1 --2- 172.21.15.37:0/982240680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15a280 0x7fd6ac15a700 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b3601640 1 -- 172.21.15.37:0/982240680 >> 172.21.15.37:0/982240680 conn(0x7fd6ac0b11a0 msgr2=0x7fd6ac0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b3601640 1 -- 172.21.15.37:0/982240680 shutdown_connections 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.214+0000 7fd6b3601640 1 -- 172.21.15.37:0/982240680 wait complete. 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b3601640 1 Processor -- start 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b3601640 1 -- start start 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b3601640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6ac0718c0 0x7fd6ac1455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b3601640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6ac159ea0 0x7fd6ac1592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b3601640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15a280 0x7fd6ac14abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b3601640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac167bd0 con 0x7fd6ac15a280 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b3601640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0b3530 con 0x7fd6ac159ea0 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b3601640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac167d50 con 0x7fd6ac0718c0 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b1b9e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15a280 0x7fd6ac14abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b1b9e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15a280 0x7fd6ac14abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52660/0 (socket says 172.21.15.37:52660) 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b1b9e640 1 -- 172.21.15.37:0/467405321 learned_addr learned my addr 172.21.15.37:0/467405321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b139d640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6ac0718c0 0x7fd6ac1455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b0b9c640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6ac159ea0 0x7fd6ac1592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b1b9e640 1 -- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6ac0718c0 msgr2=0x7fd6ac1455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b1b9e640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6ac0718c0 0x7fd6ac1455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b1b9e640 1 -- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6ac159ea0 msgr2=0x7fd6ac1592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b1b9e640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6ac159ea0 0x7fd6ac1592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b1b9e640 1 -- 172.21.15.37:0/467405321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac153f30 con 0x7fd6ac15a280 2024-04-25T09:01:54.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b1b9e640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15a280 0x7fd6ac14abd0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400c780 tx=0x7fd6a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:54.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6a1ffb640 1 -- 172.21.15.37:0/467405321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4004b70 con 0x7fd6ac15a280 2024-04-25T09:01:54.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6a1ffb640 1 -- 172.21.15.37:0/467405321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a4004d10 con 0x7fd6ac15a280 2024-04-25T09:01:54.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6a1ffb640 1 -- 172.21.15.37:0/467405321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a400aa30 con 0x7fd6ac15a280 2024-04-25T09:01:54.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b3601640 1 -- 172.21.15.37:0/467405321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac154210 con 0x7fd6ac15a280 2024-04-25T09:01:54.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.218+0000 7fd6b3601640 1 -- 172.21.15.37:0/467405321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac14c040 con 0x7fd6ac15a280 2024-04-25T09:01:54.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.222+0000 7fd6b3601640 1 -- 172.21.15.37:0/467405321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac04efa0 con 0x7fd6ac15a280 2024-04-25T09:01:54.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.222+0000 7fd6a1ffb640 1 -- 172.21.15.37:0/467405321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd6a400abd0 con 0x7fd6ac15a280 2024-04-25T09:01:54.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.222+0000 7fd6a1ffb640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd690066f60 0x7fd690069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.222+0000 7fd6a1ffb640 1 -- 172.21.15.37:0/467405321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6a4085490 con 0x7fd6ac15a280 2024-04-25T09:01:54.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.222+0000 7fd6b139d640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd690066f60 0x7fd690069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.222+0000 7fd6b139d640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd690066f60 0x7fd690069420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac1f6280 tx=0x7fd6a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:54.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.226+0000 7fd6a1ffb640 1 -- 172.21.15.37:0/467405321 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd6a4020070 con 0x7fd6ac15a280 2024-04-25T09:01:54.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.422+0000 7fd6b3601640 1 -- 172.21.15.37:0/467405321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6ac073d90 con 0x7fd6ac15a280 2024-04-25T09:01:54.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.422+0000 7fd6a1ffb640 1 -- 172.21.15.37:0/467405321 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fd6a4056300 con 0x7fd6ac15a280 2024-04-25T09:01:54.412 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:54.412 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:01:37.945874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8727":{"gid":8727,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1266122577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1266122577},{"type":"v1","addr":"172.21.15.37:6835","nonce":1266122577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:01:54.412 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 81 2024-04-25T09:01:54.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 -- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd690066f60 msgr2=0x7fd690069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd690066f60 0x7fd690069420 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd6ac1f6280 tx=0x7fd6a8009040 comp rx=0 tx=0).stop 2024-04-25T09:01:54.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 -- 172.21.15.37:0/467405321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15a280 msgr2=0x7fd6ac14abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15a280 0x7fd6ac14abd0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fd6a400c780 tx=0x7fd6a400cc40 comp rx=0 tx=0).stop 2024-04-25T09:01:54.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 -- 172.21.15.37:0/467405321 shutdown_connections 2024-04-25T09:01:54.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd690066f60 0x7fd690069420 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd6ac15a280 0x7fd6ac14abd0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd6ac159ea0 0x7fd6ac1592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 --2- 172.21.15.37:0/467405321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd6ac0718c0 0x7fd6ac1455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 -- 172.21.15.37:0/467405321 >> 172.21.15.37:0/467405321 conn(0x7fd6ac0b11a0 msgr2=0x7fd6ac15bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:54.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 -- 172.21.15.37:0/467405321 shutdown_connections 2024-04-25T09:01:54.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.426+0000 7fd6b3601640 1 -- 172.21.15.37:0/467405321 wait complete. 2024-04-25T09:01:54.430 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-04-25T09:01:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2ec90080 1 Processor -- start 2024-04-25T09:01:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2ec90080 1 -- start start 2024-04-25T09:01:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2ec90080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 0x5624881eade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2ec90080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56248815d110 0x562488133560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2ec90080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56248815d4f0 0x562488202bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2ec90080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x562488207480 con 0x562488130330 2024-04-25T09:01:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2ec90080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624880d79f0 con 0x56248815d110 2024-04-25T09:01:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2ec90080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x562488207720 con 0x56248815d4f0 2024-04-25T09:01:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2c7ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 0x5624881eade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2c7ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 0x5624881eade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52668/0 (socket says 172.21.15.37:52668) 2024-04-25T09:01:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2c7ae640 1 -- 172.21.15.37:0/314131955 learned_addr learned my addr 172.21.15.37:0/314131955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2bfad640 1 --2- 172.21.15.37:0/314131955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56248815d110 0x562488133560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2c7ae640 1 -- 172.21.15.37:0/314131955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56248815d4f0 msgr2=0x562488202bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2cfaf640 1 --2- 172.21.15.37:0/314131955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56248815d4f0 0x562488202bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2c7ae640 1 --2- 172.21.15.37:0/314131955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56248815d4f0 0x562488202bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2c7ae640 1 -- 172.21.15.37:0/314131955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56248815d110 msgr2=0x562488133560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:54.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2c7ae640 1 --2- 172.21.15.37:0/314131955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56248815d110 0x562488133560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2c7ae640 1 -- 172.21.15.37:0/314131955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624881eb860 con 0x562488130330 2024-04-25T09:01:54.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2bfad640 1 --2- 172.21.15.37:0/314131955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56248815d110 0x562488133560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:54.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2cfaf640 1 --2- 172.21.15.37:0/314131955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56248815d4f0 0x562488202bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:54.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2c7ae640 1 --2- 172.21.15.37:0/314131955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 0x5624881eade0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0097a0 tx=0x7f4b1c009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1a818f5df5fdc88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:54.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2afab640 1 -- 172.21.15.37:0/314131955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c017070 con 0x562488130330 2024-04-25T09:01:54.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2afab640 1 -- 172.21.15.37:0/314131955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c009e50 con 0x562488130330 2024-04-25T09:01:54.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.478+0000 7f4b2afab640 1 -- 172.21.15.37:0/314131955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c006b10 con 0x562488130330 2024-04-25T09:01:54.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 -- 172.21.15.37:0/314131955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 msgr2=0x5624881eade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 --2- 172.21.15.37:0/314131955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 0x5624881eade0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0097a0 tx=0x7f4b1c009c60 comp rx=0 tx=0).stop 2024-04-25T09:01:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 -- 172.21.15.37:0/314131955 shutdown_connections 2024-04-25T09:01:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 --2- 172.21.15.37:0/314131955 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56248815d4f0 0x562488202bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 --2- 172.21.15.37:0/314131955 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56248815d110 0x562488133560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 --2- 172.21.15.37:0/314131955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 0x5624881eade0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 -- 172.21.15.37:0/314131955 >> 172.21.15.37:0/314131955 conn(0x56248815ba80 msgr2=0x562488134260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 -- 172.21.15.37:0/314131955 shutdown_connections 2024-04-25T09:01:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 -- 172.21.15.37:0/314131955 wait complete. 2024-04-25T09:01:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 Processor -- start 2024-04-25T09:01:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 -- start start 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 0x5624881760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56248815d110 0x5624882094d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56248815d4f0 0x5624881746a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624882021d0 con 0x562488130330 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x562488202470 con 0x56248815d4f0 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x562488202710 con 0x56248815d110 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2c7ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 0x5624881760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2c7ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 0x5624881760c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52672/0 (socket says 172.21.15.37:52672) 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2c7ae640 1 -- 172.21.15.37:0/674149572 learned_addr learned my addr 172.21.15.37:0/674149572 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2c7ae640 1 -- 172.21.15.37:0/674149572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56248815d110 msgr2=0x5624882094d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2c7ae640 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56248815d110 0x5624882094d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2c7ae640 1 -- 172.21.15.37:0/674149572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56248815d4f0 msgr2=0x5624881746a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2cfaf640 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56248815d4f0 0x5624881746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2c7ae640 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56248815d4f0 0x5624881746a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2c7ae640 1 -- 172.21.15.37:0/674149572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562488176a40 con 0x562488130330 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2c7ae640 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 0x5624881760c0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c015040 tx=0x7f4b1c003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b29fa9640 1 -- 172.21.15.37:0/674149572 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c021020 con 0x562488130330 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b29fa9640 1 -- 172.21.15.37:0/674149572 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c0126b0 con 0x562488130330 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 -- 172.21.15.37:0/674149572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562488176ce0 con 0x562488130330 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b29fa9640 1 -- 172.21.15.37:0/674149572 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c017040 con 0x562488130330 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b2ec90080 1 -- 172.21.15.37:0/674149572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562488177220 con 0x562488130330 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.482+0000 7f4b29fa9640 1 -- 172.21.15.37:0/674149572 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f4b1c024020 con 0x562488130330 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b29fa9640 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4b18065970 0x7f4b18067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b29fa9640 1 -- 172.21.15.37:0/674149572 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b1c0851d0 con 0x562488130330 2024-04-25T09:01:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x562488209a10 0x5624882a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 -- 172.21.15.37:0/674149572 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v9 -- 0x562488176600 con 0x562488209a10 2024-04-25T09:01:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2cfaf640 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x562488209a10 0x5624882a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2bfad640 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4b18065970 0x7f4b18067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2cfaf640 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x562488209a10 0x5624882a2f20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:54.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2cfaf640 1 -- 172.21.15.37:0/674149572 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x7f4b24002730 con 0x562488209a10 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 -- 172.21.15.37:0/674149572 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x562488209a10 msgr2=0x5624882a2f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x562488209a10 0x5624882a2f20 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 -- 172.21.15.37:0/674149572 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4b18065970 msgr2=0x7f4b18067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4b18065970 0x7f4b18067e30 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f4b200ac000 tx=0x7f4b20003030 comp rx=0 tx=0).stop 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 -- 172.21.15.37:0/674149572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 msgr2=0x5624881760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 --2- 172.21.15.37:0/674149572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562488130330 0x5624881760c0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c015040 tx=0x7f4b1c003030 comp rx=0 tx=0).stop 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2cfaf640 1 -- 172.21.15.37:0/674149572 reap_dead start 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 -- 172.21.15.37:0/674149572 shutdown_connections 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 -- 172.21.15.37:0/674149572 >> 172.21.15.37:0/674149572 conn(0x56248815ba80 msgr2=0x5624881737c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 -- 172.21.15.37:0/674149572 shutdown_connections 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.486+0000 7f4b2ec90080 1 -- 172.21.15.37:0/674149572 wait complete. 2024-04-25T09:01:54.481 INFO:teuthology.orchestra.run.smithi037.stdout:����� 2024-04-25T09:01:54.482 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:01:54.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e92f2640 1 Processor -- start 2024-04-25T09:01:54.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e92f2640 1 -- start start 2024-04-25T09:01:54.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e92f2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb9e415da80 0x7fb9e40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e92f2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e406be60 con 0x7fb9e40d1cd0 2024-04-25T09:01:54.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e92f2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e415de60 con 0x7fb9e40d07f0 2024-04-25T09:01:54.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e92f2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e406c440 con 0x7fb9e415da80 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e27fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb9e40d1cd0 0x7fb9e40d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38346/0 (socket says 172.21.15.37:38346) 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e27fc640 1 -- 172.21.15.37:0/1592250915 learned_addr learned my addr 172.21.15.37:0/1592250915 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551354263 0 0) 0x7fb9e406be60 con 0x7fb9e40d1cd0 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9cc003530 con 0x7fb9e40d1cd0 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e2ffd640 1 --2- 172.21.15.37:0/1592250915 >> v2:172.21.15.203:3300/0 conn(0x7fb9e415da80 0x7fb9e40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3363030649 0 0) 0x7fb9cc003530 con 0x7fb9e40d1cd0 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 >> v2:172.21.15.203:3300/0 conn(0x7fb9e415da80 msgr2=0x7fb9e40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 --2- 172.21.15.37:0/1592250915 >> v2:172.21.15.203:3300/0 conn(0x7fb9e415da80 0x7fb9e40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 >> v1:172.21.15.203:6789/0 conn(0x7fb9e40d07f0 legacy=0x7fb9e406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e40d2450 con 0x7fb9e40d1cd0 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fb9d8001800 con 0x7fb9e40d1cd0 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 >> v1:172.21.15.37:6789/0 conn(0x7fb9e40d1cd0 legacy=0x7fb9e40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 --2- 172.21.15.37:0/1592250915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9cc003a30 0x7fb9cc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 --2- 172.21.15.37:0/1592250915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9cc004810 0x7fb9cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 --2- 172.21.15.37:0/1592250915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9cc005240 0x7fb9cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e406c440 con 0x7fb9cc004810 2024-04-25T09:01:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc00c1a0 con 0x7fb9cc003a30 2024-04-25T09:01:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc00c3a0 con 0x7fb9cc005240 2024-04-25T09:01:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e27fc640 1 --2- 172.21.15.37:0/1592250915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9cc004810 0x7fb9cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e2ffd640 1 --2- 172.21.15.37:0/1592250915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9cc003a30 0x7fb9cc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e27fc640 1 -- 172.21.15.37:0/1592250915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9cc005240 msgr2=0x7fb9cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e27fc640 1 --2- 172.21.15.37:0/1592250915 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9cc005240 0x7fb9cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e27fc640 1 -- 172.21.15.37:0/1592250915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9cc003a30 msgr2=0x7fb9cc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e27fc640 1 --2- 172.21.15.37:0/1592250915 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9cc003a30 0x7fb9cc003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e27fc640 1 -- 172.21.15.37:0/1592250915 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc003530 con 0x7fb9cc004810 2024-04-25T09:01:54.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.658+0000 7fb9e37fe640 1 -- 172.21.15.37:0/1592250915 reap_dead start 2024-04-25T09:01:54.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e27fc640 1 --2- 172.21.15.37:0/1592250915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9cc004810 0x7fb9cc004c90 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800c240 tx=0x7fb9d801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aac7410f53e9ced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8007d30 con 0x7fb9cc004810 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d8007270 con 0x7fb9cc004810 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e1ffb640 1 -- 172.21.15.37:0/1592250915 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80057b0 con 0x7fb9cc004810 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 -- 172.21.15.37:0/1592250915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9cc004810 msgr2=0x7fb9cc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 --2- 172.21.15.37:0/1592250915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9cc004810 0x7fb9cc004c90 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fb9d800c240 tx=0x7fb9d801f3f0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 -- 172.21.15.37:0/1592250915 shutdown_connections 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 --2- 172.21.15.37:0/1592250915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9cc004810 0x7fb9cc004c90 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 -- 172.21.15.37:0/1592250915 >> 172.21.15.37:0/1592250915 conn(0x7fb9e408b000 msgr2=0x7fb9e40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 -- 172.21.15.37:0/1592250915 shutdown_connections 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 -- 172.21.15.37:0/1592250915 wait complete. 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 Processor -- start 2024-04-25T09:01:54.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 -- start start 2024-04-25T09:01:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9cc004810 0x7fb9e41f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9e415f0b0 0x7fb9e40c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9e4159a80 0x7fb9e4159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40d26f0 con 0x7fb9e415f0b0 2024-04-25T09:01:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40d2120 con 0x7fb9cc004810 2024-04-25T09:01:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e92f2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e40d2870 con 0x7fb9e4159a80 2024-04-25T09:01:54.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9e415f0b0 0x7fb9e40c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9e415f0b0 0x7fb9e40c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52678/0 (socket says 172.21.15.37:52678) 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e27fc640 1 -- 172.21.15.37:0/140524789 learned_addr learned my addr 172.21.15.37:0/140524789 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e2ffd640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9cc004810 0x7fb9e41f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e37fe640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9e4159a80 0x7fb9e4159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e27fc640 1 -- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9e4159a80 msgr2=0x7fb9e4159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e27fc640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9e4159a80 0x7fb9e4159f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e27fc640 1 -- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9cc004810 msgr2=0x7fb9e41f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e27fc640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9cc004810 0x7fb9e41f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.662+0000 7fb9e27fc640 1 -- 172.21.15.37:0/140524789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e415f900 con 0x7fb9e415f0b0 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9e27fc640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9e415f0b0 0x7fb9e40c98b0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8001b00 tx=0x7fb9cc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9cb7fe640 1 -- 172.21.15.37:0/140524789 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80015c0 con 0x7fb9e415f0b0 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9cb7fe640 1 -- 172.21.15.37:0/140524789 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d80b6070 con 0x7fb9e415f0b0 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9e92f2640 1 -- 172.21.15.37:0/140524789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e415fbe0 con 0x7fb9e415f0b0 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9cb7fe640 1 -- 172.21.15.37:0/140524789 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d80057b0 con 0x7fb9e415f0b0 2024-04-25T09:01:54.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9e92f2640 1 -- 172.21.15.37:0/140524789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e41601a0 con 0x7fb9e415f0b0 2024-04-25T09:01:54.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9e92f2640 1 -- 172.21.15.37:0/140524789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e404efa0 con 0x7fb9e415f0b0 2024-04-25T09:01:54.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9cb7fe640 1 -- 172.21.15.37:0/140524789 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb9d80b8020 con 0x7fb9e415f0b0 2024-04-25T09:01:54.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9cb7fe640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9d0066f60 0x7fb9d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:54.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9cb7fe640 1 -- 172.21.15.37:0/140524789 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb9d8122290 con 0x7fb9e415f0b0 2024-04-25T09:01:54.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9e2ffd640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9d0066f60 0x7fb9d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:54.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.666+0000 7fb9e2ffd640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9d0066f60 0x7fb9d0069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400c880 tx=0x7fb9d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:54.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.670+0000 7fb9cb7fe640 1 -- 172.21.15.37:0/140524789 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb9d8019100 con 0x7fb9e415f0b0 2024-04-25T09:01:54.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.866+0000 7fb9e92f2640 1 -- 172.21.15.37:0/140524789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb9e4073d90 con 0x7fb9e415f0b0 2024-04-25T09:01:54.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.866+0000 7fb9cb7fe640 1 -- 172.21.15.37:0/140524789 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb9d80f3280 con 0x7fb9e415f0b0 2024-04-25T09:01:54.855 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:01:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.866+0000 7fb9e92f2640 1 -- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9d0066f60 msgr2=0x7fb9d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.866+0000 7fb9e92f2640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9d0066f60 0x7fb9d0069420 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fb9d400c880 tx=0x7fb9d4008040 comp rx=0 tx=0).stop 2024-04-25T09:01:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.866+0000 7fb9e92f2640 1 -- 172.21.15.37:0/140524789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9e415f0b0 msgr2=0x7fb9e40c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:54.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.866+0000 7fb9e92f2640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9e415f0b0 0x7fb9e40c98b0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8001b00 tx=0x7fb9cc005840 comp rx=0 tx=0).stop 2024-04-25T09:01:54.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.866+0000 7fb9e92f2640 1 -- 172.21.15.37:0/140524789 shutdown_connections 2024-04-25T09:01:54.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.866+0000 7fb9e92f2640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb9d0066f60 0x7fb9d0069420 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.866+0000 7fb9e92f2640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb9e4159a80 0x7fb9e4159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.866+0000 7fb9e92f2640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9e415f0b0 0x7fb9e40c98b0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.866+0000 7fb9e92f2640 1 --2- 172.21.15.37:0/140524789 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb9cc004810 0x7fb9e41f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:54.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.870+0000 7fb9e92f2640 1 -- 172.21.15.37:0/140524789 >> 172.21.15.37:0/140524789 conn(0x7fb9e408b000 msgr2=0x7fb9e40d10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:54.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.870+0000 7fb9e92f2640 1 -- 172.21.15.37:0/140524789 shutdown_connections 2024-04-25T09:01:54.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:54.870+0000 7fb9e92f2640 1 -- 172.21.15.37:0/140524789 wait complete. 2024-04-25T09:01:54.873 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:01:54.873 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:01:54.873 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T09:01:54.873 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:01:54.873 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:54.873 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:01:54.873 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:01:54.873 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:01:54.873 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:01:54.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:54.874 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T09:01:54.880 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:01:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:01:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:01:54.880 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41926sec preferred_lft 41926sec 2024-04-25T09:01:54.881 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:54.883 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:01:54.883 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:54.883 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:01:54.883 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:54.883 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:01:54.883 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:54.883 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:01:54.883 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:54.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:54.883 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:01:54.933 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T09:01:54.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:54.935 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T09:01:54.985 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T09:01:54.986 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:01:54.986 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T09:01:54.986 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:01:54.986 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T09:01:54.986 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:01:54.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:54.987 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T09:01:54.987 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T09:01:54.987 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:01:54.987 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:01:54.987 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:01:54.987 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T09:01:55.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:55.058 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T09:01:55.106 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:01:55.106 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:01:55.106 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:01:55.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:55.107 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T09:01:55.164 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:01:55.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:01:55.165 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:55.214 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:01:55.214 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:01:55.214 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:55.262 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:01:55.320 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:01:55.371 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:01:55.371 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:01:55.371 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:01:55.371 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:01:55.371 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:01:55.371 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:01:55.371 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:01:55.371 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:01:55.371 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:01:55.371 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:01:55.372 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:01:55.373 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:01:55.373 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:01:55.373 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:01:55.373 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:01:55.373 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:55.373 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:55.373 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:55.373 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:55.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:55.374 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:01:55.422 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:01:55.423 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:01:55.467 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:01:55.481 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:01:55.531 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:01:55.531 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:01:55.531 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:01:55.531 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:01:55.532 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:01:55.532 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:01:55.532 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:01:55.532 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:01:55.532 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:01:55.532 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:01:55.532 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:01:55.532 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:01:55.532 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:01:55.532 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:01:55.534 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:55.534 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:55.534 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:55.534 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:55.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:55.534 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:01:55.551 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T09:01:55.562+0000 7fedcbbb4580 -1 init, newargv = 0x564cedc83f90 newargc=15 2024-04-25T09:01:55.552 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[52070]: starting ceph client 2024-04-25T09:01:55.557 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[52070]: starting fuse 2024-04-25T09:01:56.579 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:56.582 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:56.657 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:01:56.683 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:01:56.732 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:01:56.732 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:01:56.732 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:01:56.732 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:01:56.732 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:01:56.733 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:56.734 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:56.735 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:56.735 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:56.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:56.736 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:01:56.782 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T09:01:56.782 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:01:56.782 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:01:56.782 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T09:01:56.782 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:01:56.783 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:01:56.784 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:01:56.784 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T09:01:56.785 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T09:01:56.785 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T09:01:56.785 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:01:56.785 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:56.785 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T09:01:56.785 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:01:56.909 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.52070.asok 2024-04-25T09:01:56.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:01:56.921 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.52070.asok 2024-04-25T09:01:56.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:56.921 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.52070.asok status 2024-04-25T09:01:57.083 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:01:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T09:01:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:01:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:01:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T09:01:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T09:01:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:01:57.083 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "52070", 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 8991, 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 8991 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4047056867 2024-04-25T09:01:57.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4047056867 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.8991 192.168.0.1:0/4047056867", 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/4047056867", 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 81, 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 110, 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T09:01:57.085 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:01:57.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:57.097 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T09:01:57.097 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:57.104 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T09:01:57.104 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:57.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:57.104 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:57.159 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:01:57.159 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:57.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:57.207 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T09:01:57.323 INFO:teuthology.orchestra.run.smithi037.stderr:6+0 records in 2024-04-25T09:01:57.323 INFO:teuthology.orchestra.run.smithi037.stderr:6+0 records out 2024-04-25T09:01:57.323 INFO:teuthology.orchestra.run.smithi037.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0616699 s, 102 MB/s 2024-04-25T09:01:57.324 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:01:57.325 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:57.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:57.330 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T09:01:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:6+0 records in 2024-04-25T09:01:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:6+0 records out 2024-04-25T09:01:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.06644 s, 94.7 MB/s 2024-04-25T09:01:57.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:57.451 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi037.stdout:./ 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi037.stdout:./file2_sixmegs 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi037.stdout:./file3_sixmegs 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi037.stdout:./file1_sixmegs 2024-04-25T09:01:57.465 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:01:57.465 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:01:57.465 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:01:57.465 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:57.465 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T09:01:57.465 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:01:57.578 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T09:01:57.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:01:57.588 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:01:57.588 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:01:57.588 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:01:57.588 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:57.588 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-04-25T09:01:57.588 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:01:57.694 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627777 2024-04-25T09:01:57.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:01:57.704 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:01:57.704 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:01:57.704 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:01:57.704 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:57.704 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-04-25T09:01:57.704 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:01:57.806 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627778 2024-04-25T09:01:57.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:01:57.816 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:01:57.816 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:01:57.816 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:01:57.817 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:01:57.817 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-04-25T09:01:57.817 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:01:57.922 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627776 2024-04-25T09:01:57.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:01:57.932 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:01:57.932 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:57.979 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:58.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.162+0000 7ff8077f1640 1 Processor -- start 2024-04-25T09:01:58.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.162+0000 7ff8077f1640 1 -- start start 2024-04-25T09:01:58.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff8077f1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff80006b820 0x7ff8000b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff8077f1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8000afb20 con 0x7ff8000b1ac0 2024-04-25T09:01:58.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff8077f1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8000afd90 con 0x7ff80006b440 2024-04-25T09:01:58.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff8077f1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8000b06c0 con 0x7ff80006b820 2024-04-25T09:01:58.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff805d8e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff8000b1ac0 0x7ff8000cf650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38360/0 (socket says 172.21.15.37:38360) 2024-04-25T09:01:58.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff805d8e640 1 -- 172.21.15.37:0/2332815234 learned_addr learned my addr 172.21.15.37:0/2332815234 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff804d8c640 1 --2- 172.21.15.37:0/2332815234 >> v2:172.21.15.203:3300/0 conn(0x7ff80006b820 0x7ff8000b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122347774 0 0) 0x7ff8000afb20 con 0x7ff8000b1ac0 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7ec003530 con 0x7ff8000b1ac0 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 300835162 0 0) 0x7ff7ec003530 con 0x7ff8000b1ac0 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 >> v2:172.21.15.203:3300/0 conn(0x7ff80006b820 msgr2=0x7ff8000b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 --2- 172.21.15.37:0/2332815234 >> v2:172.21.15.203:3300/0 conn(0x7ff80006b820 0x7ff8000b0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 >> v1:172.21.15.203:6789/0 conn(0x7ff80006b440 legacy=0x7ff8000b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8000b0070 con 0x7ff8000b1ac0 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7ff7f8001a00 con 0x7ff8000b1ac0 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 >> v1:172.21.15.37:6789/0 conn(0x7ff8000b1ac0 legacy=0x7ff8000cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 --2- 172.21.15.37:0/2332815234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 0x7ff7ec003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 --2- 172.21.15.37:0/2332815234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7ec004810 0x7ff7ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 --2- 172.21.15.37:0/2332815234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7ec0051d0 0x7ff7ec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000b06c0 con 0x7ff7ec003a30 2024-04-25T09:01:58.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec00c100 con 0x7ff7ec004810 2024-04-25T09:01:58.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ec00c300 con 0x7ff7ec0051d0 2024-04-25T09:01:58.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff80558d640 1 --2- 172.21.15.37:0/2332815234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 0x7ff7ec003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff805d8e640 1 --2- 172.21.15.37:0/2332815234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7ec0051d0 0x7ff7ec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff80558d640 1 -- 172.21.15.37:0/2332815234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7ec0051d0 msgr2=0x7ff7ec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:58.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff80558d640 1 --2- 172.21.15.37:0/2332815234 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff7ec0051d0 0x7ff7ec00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff80558d640 1 -- 172.21.15.37:0/2332815234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7ec004810 msgr2=0x7ff7ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:58.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff80558d640 1 --2- 172.21.15.37:0/2332815234 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff7ec004810 0x7ff7ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff80558d640 1 -- 172.21.15.37:0/2332815234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec003530 con 0x7ff7ec003a30 2024-04-25T09:01:58.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.166+0000 7ff805d8e640 1 -- 172.21.15.37:0/2332815234 reap_dead start 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff80558d640 1 --2- 172.21.15.37:0/2332815234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 0x7ff7ec003e50 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4001780 tx=0x7ff7f401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e9a8f2e502f04a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40013a0 con 0x7ff7ec003a30 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f40052f0 con 0x7ff7ec003a30 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff7f3fff640 1 -- 172.21.15.37:0/2332815234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4003840 con 0x7ff7ec003a30 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 -- 172.21.15.37:0/2332815234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 msgr2=0x7ff7ec003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 --2- 172.21.15.37:0/2332815234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 0x7ff7ec003e50 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4001780 tx=0x7ff7f401c3d0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 -- 172.21.15.37:0/2332815234 shutdown_connections 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 --2- 172.21.15.37:0/2332815234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 0x7ff7ec003e50 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 -- 172.21.15.37:0/2332815234 >> 172.21.15.37:0/2332815234 conn(0x7ff8000b1f10 msgr2=0x7ff8000b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 -- 172.21.15.37:0/2332815234 shutdown_connections 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 -- 172.21.15.37:0/2332815234 wait complete. 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 Processor -- start 2024-04-25T09:01:58.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 -- start start 2024-04-25T09:01:58.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 0x7ff8001f9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff800157ca0 0x7ff800158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8001f6b60 0x7ff8001f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000cff10 con 0x7ff7ec003a30 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000cfd90 con 0x7ff800157ca0 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff8077f1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000d0090 con 0x7ff8001f6b60 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff80558d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 0x7ff8001f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff80558d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 0x7ff8001f9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52690/0 (socket says 172.21.15.37:52690) 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.170+0000 7ff80558d640 1 -- 172.21.15.37:0/2474892822 learned_addr learned my addr 172.21.15.37:0/2474892822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff804d8c640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff800157ca0 0x7ff800158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff805d8e640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8001f6b60 0x7ff8001f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff80558d640 1 -- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8001f6b60 msgr2=0x7ff8001f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff80558d640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8001f6b60 0x7ff8001f6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff80558d640 1 -- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff800157ca0 msgr2=0x7ff800158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff80558d640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff800157ca0 0x7ff800158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff80558d640 1 -- 172.21.15.37:0/2474892822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff800157120 con 0x7ff7ec003a30 2024-04-25T09:01:58.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff805d8e640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8001f6b60 0x7ff8001f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:58.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff80558d640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 0x7ff8001f9670 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7ff7f401c200 tx=0x7ff7ec005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff7f1ffb640 1 -- 172.21.15.37:0/2474892822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4007a40 con 0x7ff7ec003a30 2024-04-25T09:01:58.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff7f1ffb640 1 -- 172.21.15.37:0/2474892822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f40ba070 con 0x7ff7ec003a30 2024-04-25T09:01:58.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff8077f1640 1 -- 172.21.15.37:0/2474892822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8001f9bb0 con 0x7ff7ec003a30 2024-04-25T09:01:58.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff7f1ffb640 1 -- 172.21.15.37:0/2474892822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f40013a0 con 0x7ff7ec003a30 2024-04-25T09:01:58.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff8077f1640 1 -- 172.21.15.37:0/2474892822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff800156640 con 0x7ff7ec003a30 2024-04-25T09:01:58.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff8077f1640 1 -- 172.21.15.37:0/2474892822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff80004efa0 con 0x7ff7ec003a30 2024-04-25T09:01:58.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff7f1ffb640 1 -- 172.21.15.37:0/2474892822 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff7f40052f0 con 0x7ff7ec003a30 2024-04-25T09:01:58.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff7f1ffb640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7e4066f90 0x7ff7e4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff7f1ffb640 1 -- 172.21.15.37:0/2474892822 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7f411f640 con 0x7ff7ec003a30 2024-04-25T09:01:58.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.174+0000 7ff804d8c640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7e4066f90 0x7ff7e4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.178+0000 7ff804d8c640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7e4066f90 0x7ff7e4069450 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff8000af800 tx=0x7ff7fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.178+0000 7ff7f1ffb640 1 -- 172.21.15.37:0/2474892822 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff7f4016110 con 0x7ff7ec003a30 2024-04-25T09:01:58.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.342+0000 7ff8077f1640 1 -- 172.21.15.37:0/2474892822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff80006e8a0 con 0x7ff7ec003a30 2024-04-25T09:01:58.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.342+0000 7ff7f1ffb640 1 -- 172.21.15.37:0/2474892822 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16848 (secure 0 0 0) 0x7ff7f40f0420 con 0x7ff7ec003a30 2024-04-25T09:01:58.332 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:01:58.332 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:01:36.569159+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:33.212328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:33.895814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 -- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7e4066f90 msgr2=0x7ff7e4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7e4066f90 0x7ff7e4069450 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7ff8000af800 tx=0x7ff7fc008040 comp rx=0 tx=0).stop 2024-04-25T09:01:58.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 -- 172.21.15.37:0/2474892822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 msgr2=0x7ff8001f9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 0x7ff8001f9670 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7ff7f401c200 tx=0x7ff7ec005840 comp rx=0 tx=0).stop 2024-04-25T09:01:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 -- 172.21.15.37:0/2474892822 shutdown_connections 2024-04-25T09:01:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff7e4066f90 0x7ff7e4069450 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8001f6b60 0x7ff8001f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff800157ca0 0x7ff800158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 --2- 172.21.15.37:0/2474892822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7ec003a30 0x7ff8001f9670 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 -- 172.21.15.37:0/2474892822 >> 172.21.15.37:0/2474892822 conn(0x7ff8000b1f10 msgr2=0x7ff80015f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 -- 172.21.15.37:0/2474892822 shutdown_connections 2024-04-25T09:01:58.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.346+0000 7ff8077f1640 1 -- 172.21.15.37:0/2474892822 wait complete. 2024-04-25T09:01:58.351 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:01:33.212328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:01:33.895814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:01:58.352 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:01:58.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e812d640 1 Processor -- start 2024-04-25T09:01:58.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e812d640 1 -- start start 2024-04-25T09:01:58.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e812d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f49e015b950 0x7f49e009a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e812d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e015bd30 con 0x7f49e00cf420 2024-04-25T09:01:58.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e812d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e00cc0d0 con 0x7f49e00cfb80 2024-04-25T09:01:58.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e812d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e009ab10 con 0x7f49e015b950 2024-04-25T09:01:58.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e56c8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f49e00cf420 0x7f49e00cce50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38366/0 (socket says 172.21.15.37:38366) 2024-04-25T09:01:58.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e56c8640 1 -- 172.21.15.37:0/3237546744 learned_addr learned my addr 172.21.15.37:0/3237546744 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:58.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e5ec9640 1 --2- 172.21.15.37:0/3237546744 >> v2:172.21.15.203:3300/0 conn(0x7f49e015b950 0x7f49e009a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566057414 0 0) 0x7f49e015bd30 con 0x7f49e00cf420 2024-04-25T09:01:58.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49cc003530 con 0x7f49e00cf420 2024-04-25T09:01:58.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2649070692 0 0) 0x7f49cc003530 con 0x7f49e00cf420 2024-04-25T09:01:58.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 >> v2:172.21.15.203:3300/0 conn(0x7f49e015b950 msgr2=0x7f49e009a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:58.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 --2- 172.21.15.37:0/3237546744 >> v2:172.21.15.203:3300/0 conn(0x7f49e015b950 0x7f49e009a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 >> v1:172.21.15.203:6789/0 conn(0x7f49e00cfb80 legacy=0x7f49e00cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e00cc370 con 0x7f49e00cf420 2024-04-25T09:01:58.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f49d4001680 con 0x7f49e00cf420 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 >> v1:172.21.15.37:6789/0 conn(0x7f49e00cf420 legacy=0x7f49e00cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49cc003a30 0x7f49cc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49cc004810 0x7f49cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49cc0051d0 0x7f49cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e009ab10 con 0x7f49cc003a30 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc00c100 con 0x7f49cc004810 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc00c300 con 0x7f49cc0051d0 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e5ec9640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49cc003a30 0x7f49cc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e56c8640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49cc004810 0x7f49cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e66ca640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49cc0051d0 0x7f49cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e5ec9640 1 -- 172.21.15.37:0/3237546744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49cc0051d0 msgr2=0x7f49cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e5ec9640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49cc0051d0 0x7f49cc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e5ec9640 1 -- 172.21.15.37:0/3237546744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49cc004810 msgr2=0x7f49cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:58.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e5ec9640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49cc004810 0x7f49cc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e66ca640 1 -- 172.21.15.37:0/3237546744 reap_dead start 2024-04-25T09:01:58.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e5ec9640 1 -- 172.21.15.37:0/3237546744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc003530 con 0x7f49cc003a30 2024-04-25T09:01:58.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.494+0000 7f49e56c8640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49cc004810 0x7f49cc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:58.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e5ec9640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49cc003a30 0x7f49cc003e50 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009ac0 tx=0x7f49dc01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588b9b8588f1a683 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc0b2070 con 0x7f49cc003a30 2024-04-25T09:01:58.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49dc0a5520 con 0x7f49cc003a30 2024-04-25T09:01:58.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e4ec7640 1 -- 172.21.15.37:0/3237546744 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49dc009690 con 0x7f49cc003a30 2024-04-25T09:01:58.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e812d640 1 -- 172.21.15.37:0/3237546744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49cc003a30 msgr2=0x7f49cc003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e812d640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49cc003a30 0x7f49cc003e50 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009ac0 tx=0x7f49dc01ca40 comp rx=0 tx=0).stop 2024-04-25T09:01:58.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e812d640 1 -- 172.21.15.37:0/3237546744 shutdown_connections 2024-04-25T09:01:58.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e812d640 1 --2- 172.21.15.37:0/3237546744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49cc003a30 0x7f49cc003e50 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e812d640 1 -- 172.21.15.37:0/3237546744 >> 172.21.15.37:0/3237546744 conn(0x7f49e00b30b0 msgr2=0x7f49e00b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:58.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e812d640 1 -- 172.21.15.37:0/3237546744 shutdown_connections 2024-04-25T09:01:58.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e812d640 1 -- 172.21.15.37:0/3237546744 wait complete. 2024-04-25T09:01:58.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e812d640 1 Processor -- start 2024-04-25T09:01:58.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.498+0000 7f49e812d640 1 -- start start 2024-04-25T09:01:58.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e812d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49cc003a30 0x7f49e01e9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e812d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49e0155f70 0x7f49e01e9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e812d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e006c7c0 0x7f49e006cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e812d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00d0520 con 0x7f49e0155f70 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e812d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e00d03a0 con 0x7f49e006c7c0 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e812d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e006f290 con 0x7f49cc003a30 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e56c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49e0155f70 0x7f49e01e9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e56c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49e0155f70 0x7f49e01e9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52700/0 (socket says 172.21.15.37:52700) 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e56c8640 1 -- 172.21.15.37:0/1722571563 learned_addr learned my addr 172.21.15.37:0/1722571563 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e5ec9640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49cc003a30 0x7f49e01e9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e66ca640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e006c7c0 0x7f49e006cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e56c8640 1 -- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49cc003a30 msgr2=0x7f49e01e9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e56c8640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49cc003a30 0x7f49e01e9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e56c8640 1 -- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e006c7c0 msgr2=0x7f49e006cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e56c8640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e006c7c0 0x7f49e006cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e56c8640 1 -- 172.21.15.37:0/1722571563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e01eae90 con 0x7f49e0155f70 2024-04-25T09:01:58.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e66ca640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e006c7c0 0x7f49e006cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e56c8640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49e0155f70 0x7f49e01e9dc0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f49d400e740 tx=0x7f49d400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49d27fc640 1 -- 172.21.15.37:0/1722571563 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d400ee50 con 0x7f49e0155f70 2024-04-25T09:01:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49d27fc640 1 -- 172.21.15.37:0/1722571563 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d4018070 con 0x7f49e0155f70 2024-04-25T09:01:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e812d640 1 -- 172.21.15.37:0/1722571563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e01ea650 con 0x7f49e0155f70 2024-04-25T09:01:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49d27fc640 1 -- 172.21.15.37:0/1722571563 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d400caf0 con 0x7f49e0155f70 2024-04-25T09:01:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e812d640 1 -- 172.21.15.37:0/1722571563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e00c5190 con 0x7f49e0155f70 2024-04-25T09:01:58.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e812d640 1 -- 172.21.15.37:0/1722571563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e004efa0 con 0x7f49e0155f70 2024-04-25T09:01:58.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49d27fc640 1 -- 172.21.15.37:0/1722571563 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f49d400cc90 con 0x7f49e0155f70 2024-04-25T09:01:58.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49d27fc640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f49c4067130 0x7f49c40695f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49d27fc640 1 -- 172.21.15.37:0/1722571563 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f49d4084ff0 con 0x7f49e0155f70 2024-04-25T09:01:58.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.502+0000 7f49e5ec9640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f49c4067130 0x7f49c40695f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.506+0000 7f49e5ec9640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f49c4067130 0x7f49c40695f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0af000 tx=0x7f49dc0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.506+0000 7f49d27fc640 1 -- 172.21.15.37:0/1722571563 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f49e01ea650 con 0x7f49e0155f70 2024-04-25T09:01:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.670+0000 7f49e812d640 1 -- 172.21.15.37:0/1722571563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49e00d13a0 con 0x7f49e0155f70 2024-04-25T09:01:58.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.670+0000 7f49d27fc640 1 -- 172.21.15.37:0/1722571563 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f49d4055f00 con 0x7f49e0155f70 2024-04-25T09:01:58.659 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 -- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f49c4067130 msgr2=0x7f49c40695f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f49c4067130 0x7f49c40695f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0af000 tx=0x7f49dc0046d0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 -- 172.21.15.37:0/1722571563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49e0155f70 msgr2=0x7f49e01e9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49e0155f70 0x7f49e01e9dc0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f49d400e740 tx=0x7f49d400ec00 comp rx=0 tx=0).stop 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 -- 172.21.15.37:0/1722571563 shutdown_connections 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f49c4067130 0x7f49c40695f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f49e006c7c0 0x7f49e006cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49e0155f70 0x7f49e01e9dc0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 --2- 172.21.15.37:0/1722571563 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f49cc003a30 0x7f49e01e9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 -- 172.21.15.37:0/1722571563 >> 172.21.15.37:0/1722571563 conn(0x7f49e00b30b0 msgr2=0x7f49e00d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 -- 172.21.15.37:0/1722571563 shutdown_connections 2024-04-25T09:01:58.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.674+0000 7f49e812d640 1 -- 172.21.15.37:0/1722571563 wait complete. 2024-04-25T09:01:58.679 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:01:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f16a70a2640 1 Processor -- start 2024-04-25T09:01:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f16a70a2640 1 -- start start 2024-04-25T09:01:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f16a70a2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16a015b420 0x7f16a00ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f16a70a2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f16a015b800 0x7f16a00cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f16a70a2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a00cbd20 con 0x7f16a00cfad0 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f16a70a2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a00cff20 con 0x7f16a015b800 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f16a70a2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00cc930 con 0x7f16a015b420 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f169ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f16a015b800 0x7f16a00cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f169ffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f16a015b800 0x7f16a00cf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52710/0 (socket says 172.21.15.37:52710) 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f169ffff640 1 -- 172.21.15.37:0/3880455112 learned_addr learned my addr 172.21.15.37:0/3880455112 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f16a563f640 1 --1- 172.21.15.37:0/3880455112 >> v1:172.21.15.37:6789/0 conn(0x7f16a00cfad0 0x7f16a015f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38374/0 (socket says 172.21.15.37:38374) 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f16a4e3e640 1 --2- 172.21.15.37:0/3880455112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16a015b420 0x7f16a00ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f169ffff640 1 -- 172.21.15.37:0/3880455112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16a015b420 msgr2=0x7f16a00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f169ffff640 1 --2- 172.21.15.37:0/3880455112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16a015b420 0x7f16a00ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f169f7fe640 1 -- 172.21.15.37:0/3880455112 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403206907 0 0) 0x7f16a00cbd20 con 0x7f16a00cfad0 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f169ffff640 1 -- 172.21.15.37:0/3880455112 >> v1:172.21.15.37:6789/0 conn(0x7f16a00cfad0 legacy=0x7f16a015f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f169ffff640 1 -- 172.21.15.37:0/3880455112 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a00cc000 con 0x7f16a015b800 2024-04-25T09:01:58.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.822+0000 7f16a4e3e640 1 --2- 172.21.15.37:0/3880455112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16a015b420 0x7f16a00ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:58.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f169ffff640 1 --2- 172.21.15.37:0/3880455112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16a015b800 0x7f16a00cf590 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1690009e50 tx=0x7f169001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ef2553bf3de442 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f169f7fe640 1 -- 172.21.15.37:0/3880455112 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900a58d0 con 0x7f16a015b800 2024-04-25T09:01:58.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f169f7fe640 1 -- 172.21.15.37:0/3880455112 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16900a5a70 con 0x7f16a015b800 2024-04-25T09:01:58.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f169f7fe640 1 -- 172.21.15.37:0/3880455112 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900a5e40 con 0x7f16a015b800 2024-04-25T09:01:58.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 -- 172.21.15.37:0/3880455112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16a015b800 msgr2=0x7f16a00cf590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 --2- 172.21.15.37:0/3880455112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16a015b800 0x7f16a00cf590 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1690009e50 tx=0x7f169001ca00 comp rx=0 tx=0).stop 2024-04-25T09:01:58.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 -- 172.21.15.37:0/3880455112 shutdown_connections 2024-04-25T09:01:58.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 --2- 172.21.15.37:0/3880455112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16a015b800 0x7f16a00cf590 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 --2- 172.21.15.37:0/3880455112 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16a015b420 0x7f16a00ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 -- 172.21.15.37:0/3880455112 >> 172.21.15.37:0/3880455112 conn(0x7f16a006ae60 msgr2=0x7f16a006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:58.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 -- 172.21.15.37:0/3880455112 shutdown_connections 2024-04-25T09:01:58.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 -- 172.21.15.37:0/3880455112 wait complete. 2024-04-25T09:01:58.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 Processor -- start 2024-04-25T09:01:58.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 -- start start 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16a00cfad0 0x7f16a0156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16a015b420 0x7f16a006d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16a015b800 0x7f16a006dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00caa80 con 0x7f16a00cfad0 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00ca900 con 0x7f16a015b420 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a70a2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00cadb0 con 0x7f16a015b800 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a4e3e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16a00cfad0 0x7f16a0156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a4e3e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16a00cfad0 0x7f16a0156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52718/0 (socket says 172.21.15.37:52718) 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a4e3e640 1 -- 172.21.15.37:0/4078732403 learned_addr learned my addr 172.21.15.37:0/4078732403 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a4e3e640 1 -- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16a015b800 msgr2=0x7f16a006dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a563f640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16a015b800 0x7f16a006dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a4e3e640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16a015b800 0x7f16a006dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f169ffff640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16a015b420 0x7f16a006d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.826+0000 7f16a4e3e640 1 -- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16a015b420 msgr2=0x7f16a006d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f16a4e3e640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16a015b420 0x7f16a006d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f16a4e3e640 1 -- 172.21.15.37:0/4078732403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a0157da0 con 0x7f16a00cfad0 2024-04-25T09:01:58.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f16a4e3e640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16a00cfad0 0x7f16a0156970 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f168800e810 tx=0x7f168800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f169d7fa640 1 -- 172.21.15.37:0/4078732403 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1688017070 con 0x7f16a00cfad0 2024-04-25T09:01:58.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f169d7fa640 1 -- 172.21.15.37:0/4078732403 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168800c8d0 con 0x7f16a00cfad0 2024-04-25T09:01:58.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f169d7fa640 1 -- 172.21.15.37:0/4078732403 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168800f040 con 0x7f16a00cfad0 2024-04-25T09:01:58.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f16a70a2640 1 -- 172.21.15.37:0/4078732403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a0158080 con 0x7f16a00cfad0 2024-04-25T09:01:58.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f16a70a2640 1 -- 172.21.15.37:0/4078732403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a015d330 con 0x7f16a00cfad0 2024-04-25T09:01:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f16a70a2640 1 -- 172.21.15.37:0/4078732403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a004efa0 con 0x7f16a00cfad0 2024-04-25T09:01:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f169d7fa640 1 -- 172.21.15.37:0/4078732403 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f168801a070 con 0x7f16a00cfad0 2024-04-25T09:01:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f169d7fa640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1680066f60 0x7f1680069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f169d7fa640 1 -- 172.21.15.37:0/4078732403 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1688084e80 con 0x7f16a00cfad0 2024-04-25T09:01:58.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f169ffff640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1680066f60 0x7f1680069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.830+0000 7f169ffff640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1680066f60 0x7f1680069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f16900ae000 tx=0x7f16900a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.834+0000 7f169d7fa640 1 -- 172.21.15.37:0/4078732403 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1688055c60 con 0x7f16a00cfad0 2024-04-25T09:01:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.994+0000 7f16a70a2640 1 -- 172.21.15.37:0/4078732403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f16a00c9690 con 0x7f16a00cfad0 2024-04-25T09:01:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.994+0000 7f169d7fa640 1 -- 172.21.15.37:0/4078732403 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1688055560 con 0x7f16a00cfad0 2024-04-25T09:01:58.984 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:01:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 -- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1680066f60 msgr2=0x7f1680069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1680066f60 0x7f1680069420 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f16900ae000 tx=0x7f16900a7070 comp rx=0 tx=0).stop 2024-04-25T09:01:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 -- 172.21.15.37:0/4078732403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16a00cfad0 msgr2=0x7f16a0156970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16a00cfad0 0x7f16a0156970 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f168800e810 tx=0x7f168800ecd0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 -- 172.21.15.37:0/4078732403 shutdown_connections 2024-04-25T09:01:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1680066f60 0x7f1680069420 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16a015b800 0x7f16a006dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16a015b420 0x7f16a006d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 --2- 172.21.15.37:0/4078732403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16a00cfad0 0x7f16a0156970 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 -- 172.21.15.37:0/4078732403 >> 172.21.15.37:0/4078732403 conn(0x7f16a006ae60 msgr2=0x7f16a00b0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 -- 172.21.15.37:0/4078732403 shutdown_connections 2024-04-25T09:01:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:58.998+0000 7f16a70a2640 1 -- 172.21.15.37:0/4078732403 wait complete. 2024-04-25T09:01:59.003 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:01:59.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa016b3c640 1 Processor -- start 2024-04-25T09:01:59.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa016b3c640 1 -- start start 2024-04-25T09:01:59.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa016b3c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 0x7fa0100cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa016b3c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0100cb8c0 0x7fa0100cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa016b3c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0100cc2b0 con 0x7fa010159bd0 2024-04-25T09:01:59.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa016b3c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa010072780 con 0x7fa0100cb8c0 2024-04-25T09:01:59.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa016b3c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa010071b70 con 0x7fa010159fb0 2024-04-25T09:01:59.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa00ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 0x7fa0100cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa00ffff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 0x7fa0100cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:48806/0 (socket says 172.21.15.37:48806) 2024-04-25T09:01:59.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa00ffff640 1 -- 172.21.15.37:0/3070259345 learned_addr learned my addr 172.21.15.37:0/3070259345 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:59.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa0150d9640 1 --2- 172.21.15.37:0/3070259345 >> v2:172.21.15.203:3300/0 conn(0x7fa0100cb8c0 0x7fa0100cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa00ffff640 1 -- 172.21.15.37:0/3070259345 >> v2:172.21.15.203:3300/0 conn(0x7fa0100cb8c0 msgr2=0x7fa0100cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:59.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa00ffff640 1 --2- 172.21.15.37:0/3070259345 >> v2:172.21.15.203:3300/0 conn(0x7fa0100cb8c0 0x7fa0100cbd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa00ffff640 1 -- 172.21.15.37:0/3070259345 >> v1:172.21.15.203:6789/0 conn(0x7fa010159bd0 legacy=0x7fa0100c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.146+0000 7fa00ffff640 1 -- 172.21.15.37:0/3070259345 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa010071e50 con 0x7fa010159fb0 2024-04-25T09:01:59.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa00ffff640 1 --2- 172.21.15.37:0/3070259345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 0x7fa0100cb380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0040098a0 tx=0x7fa00401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee37c0e863d88332 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa00f7fe640 1 -- 172.21.15.37:0/3070259345 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0040aa5f0 con 0x7fa010159fb0 2024-04-25T09:01:59.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa00f7fe640 1 -- 172.21.15.37:0/3070259345 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0040af070 con 0x7fa010159fb0 2024-04-25T09:01:59.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa00f7fe640 1 -- 172.21.15.37:0/3070259345 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0040aa9b0 con 0x7fa010159fb0 2024-04-25T09:01:59.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa016b3c640 1 -- 172.21.15.37:0/3070259345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 msgr2=0x7fa0100cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa016b3c640 1 --2- 172.21.15.37:0/3070259345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 0x7fa0100cb380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa0040098a0 tx=0x7fa00401c510 comp rx=0 tx=0).stop 2024-04-25T09:01:59.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa016b3c640 1 -- 172.21.15.37:0/3070259345 shutdown_connections 2024-04-25T09:01:59.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa016b3c640 1 --2- 172.21.15.37:0/3070259345 >> v2:172.21.15.203:3300/0 conn(0x7fa0100cb8c0 0x7fa0100cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa016b3c640 1 --2- 172.21.15.37:0/3070259345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 0x7fa0100cb380 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa016b3c640 1 -- 172.21.15.37:0/3070259345 >> 172.21.15.37:0/3070259345 conn(0x7fa01006ae60 msgr2=0x7fa01006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:59.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa016b3c640 1 -- 172.21.15.37:0/3070259345 shutdown_connections 2024-04-25T09:01:59.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa016b3c640 1 -- 172.21.15.37:0/3070259345 wait complete. 2024-04-25T09:01:59.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa016b3c640 1 Processor -- start 2024-04-25T09:01:59.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.150+0000 7fa016b3c640 1 -- start start 2024-04-25T09:01:59.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa016b3c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0100cb8c0 0x7fa010153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa016b3c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa010159bd0 0x7fa01015b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa016b3c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 0x7fa010162660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa016b3c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa010165af0 con 0x7fa0100cb8c0 2024-04-25T09:01:59.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa016b3c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa010165970 con 0x7fa010159bd0 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa016b3c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa010165c70 con 0x7fa010159fb0 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa0148d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0100cb8c0 0x7fa010153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa0148d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0100cb8c0 0x7fa010153ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52722/0 (socket says 172.21.15.37:52722) 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa0148d8640 1 -- 172.21.15.37:0/621434817 learned_addr learned my addr 172.21.15.37:0/621434817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa0150d9640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 0x7fa010162660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa00ffff640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa010159bd0 0x7fa01015b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa0148d8640 1 -- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 msgr2=0x7fa010162660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa0148d8640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 0x7fa010162660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa0148d8640 1 -- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa010159bd0 msgr2=0x7fa01015b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa0148d8640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa010159bd0 0x7fa01015b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa0148d8640 1 -- 172.21.15.37:0/621434817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01015cbb0 con 0x7fa0100cb8c0 2024-04-25T09:01:59.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa0150d9640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 0x7fa010162660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:59.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa0148d8640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0100cb8c0 0x7fa010153ba0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa00000c370 tx=0x7fa00000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa00d7fa640 1 -- 172.21.15.37:0/621434817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0000052d0 con 0x7fa0100cb8c0 2024-04-25T09:01:59.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa00d7fa640 1 -- 172.21.15.37:0/621434817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa000005470 con 0x7fa0100cb8c0 2024-04-25T09:01:59.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa016b3c640 1 -- 172.21.15.37:0/621434817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0101540e0 con 0x7fa0100cb8c0 2024-04-25T09:01:59.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa00d7fa640 1 -- 172.21.15.37:0/621434817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa000007260 con 0x7fa0100cb8c0 2024-04-25T09:01:59.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa016b3c640 1 -- 172.21.15.37:0/621434817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa01015c0d0 con 0x7fa0100cb8c0 2024-04-25T09:01:59.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa016b3c640 1 -- 172.21.15.37:0/621434817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa01004efa0 con 0x7fa0100cb8c0 2024-04-25T09:01:59.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa00d7fa640 1 -- 172.21.15.37:0/621434817 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa000007400 con 0x7fa0100cb8c0 2024-04-25T09:01:59.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa00d7fa640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9ffc066f90 0x7f9ffc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa00d7fa640 1 -- 172.21.15.37:0/621434817 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa000084b10 con 0x7fa0100cb8c0 2024-04-25T09:01:59.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.154+0000 7fa00ffff640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9ffc066f90 0x7f9ffc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.158+0000 7fa00ffff640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9ffc066f90 0x7f9ffc069450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa0040ae000 tx=0x7fa0040a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.158+0000 7fa00d7fa640 1 -- 172.21.15.37:0/621434817 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa0101540e0 con 0x7fa0100cb8c0 2024-04-25T09:01:59.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.318+0000 7fa016b3c640 1 -- 172.21.15.37:0/621434817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0100ae460 con 0x7fa0100cb8c0 2024-04-25T09:01:59.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa00d7fa640 1 -- 172.21.15.37:0/621434817 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa01015cbb0 con 0x7fa0100cb8c0 2024-04-25T09:01:59.310 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:01:59.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 -- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9ffc066f90 msgr2=0x7f9ffc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9ffc066f90 0x7f9ffc069450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fa0040ae000 tx=0x7fa0040a7000 comp rx=0 tx=0).stop 2024-04-25T09:01:59.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 -- 172.21.15.37:0/621434817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0100cb8c0 msgr2=0x7fa010153ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0100cb8c0 0x7fa010153ba0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa00000c370 tx=0x7fa00000c830 comp rx=0 tx=0).stop 2024-04-25T09:01:59.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 -- 172.21.15.37:0/621434817 shutdown_connections 2024-04-25T09:01:59.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9ffc066f90 0x7f9ffc069450 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa010159fb0 0x7fa010162660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa010159bd0 0x7fa01015b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 --2- 172.21.15.37:0/621434817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0100cb8c0 0x7fa010153ba0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 -- 172.21.15.37:0/621434817 >> 172.21.15.37:0/621434817 conn(0x7fa01006ae60 msgr2=0x7fa0100ad0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:59.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 -- 172.21.15.37:0/621434817 shutdown_connections 2024-04-25T09:01:59.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.322+0000 7fa016b3c640 1 -- 172.21.15.37:0/621434817 wait complete. 2024-04-25T09:01:59.330 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:01:59.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.462+0000 7f651cc15640 1 Processor -- start 2024-04-25T09:01:59.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.462+0000 7f651cc15640 1 -- start start 2024-04-25T09:01:59.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.462+0000 7f651cc15640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65180b2c30 0x7f651815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.462+0000 7f651cc15640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6518066c30 con 0x7f65180af280 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.462+0000 7f651cc15640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65180af660 con 0x7f651806c3b0 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.462+0000 7f651cc15640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651806daa0 con 0x7f65180b2c30 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f651659c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f65180af280 0x7f651806bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46830/0 (socket says 172.21.15.37:46830) 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f651659c640 1 -- 172.21.15.37:0/2640532135 learned_addr learned my addr 172.21.15.37:0/2640532135 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6516d9d640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65180b2c30 0x7f651815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689286240 0 0) 0x7f6518066c30 con 0x7f65180af280 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6504003530 con 0x7f65180af280 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2086490271 0 0) 0x7f6504003530 con 0x7f65180af280 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65180b2c30 msgr2=0x7f651815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65180b2c30 0x7f651815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 >> v1:172.21.15.203:6789/0 conn(0x7f651806c3b0 legacy=0x7f651806df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651815e640 con 0x7f65180af280 2024-04-25T09:01:59.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f6508001680 con 0x7f65180af280 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 >> v1:172.21.15.37:6789/0 conn(0x7f65180af280 legacy=0x7f651806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6504003a60 0x7f6504003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65180b2c30 0x7f6504004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 0x7f65040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650400bae0 con 0x7f6504004d50 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f650400bc90 con 0x7f6504003a60 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f650400be90 con 0x7f65180b2c30 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6516d9d640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 0x7f65040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f651659c640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6504003a60 0x7f6504003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6516d9d640 1 -- 172.21.15.37:0/2640532135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65180b2c30 msgr2=0x7f6504004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6516d9d640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65180b2c30 0x7f6504004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6516d9d640 1 -- 172.21.15.37:0/2640532135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6504003a60 msgr2=0x7f6504003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6516d9d640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6504003a60 0x7f6504003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6516d9d640 1 -- 172.21.15.37:0/2640532135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6504003530 con 0x7f6504004d50 2024-04-25T09:01:59.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f651659c640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6504003a60 0x7f6504003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:59.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6516d9d640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 0x7f65040051d0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6510009f90 tx=0x7f651001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b9e10dcfe7ae9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65100b2070 con 0x7f6504004d50 2024-04-25T09:01:59.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65100a5c90 con 0x7f6504004d50 2024-04-25T09:01:59.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f6515d9b640 1 -- 172.21.15.37:0/2640532135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510009c10 con 0x7f6504004d50 2024-04-25T09:01:59.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.466+0000 7f651cc15640 1 -- 172.21.15.37:0/2640532135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 msgr2=0x7f65040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 --2- 172.21.15.37:0/2640532135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 0x7f65040051d0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6510009f90 tx=0x7f651001cc80 comp rx=0 tx=0).stop 2024-04-25T09:01:59.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f6516d9d640 1 -- 172.21.15.37:0/2640532135 reap_dead start 2024-04-25T09:01:59.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 -- 172.21.15.37:0/2640532135 shutdown_connections 2024-04-25T09:01:59.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 -- 172.21.15.37:0/2640532135 >> 172.21.15.37:0/2640532135 conn(0x7f6518071810 msgr2=0x7f6518071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:59.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 -- 172.21.15.37:0/2640532135 shutdown_connections 2024-04-25T09:01:59.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 -- 172.21.15.37:0/2640532135 wait complete. 2024-04-25T09:01:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 Processor -- start 2024-04-25T09:01:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 -- start start 2024-04-25T09:01:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 0x7f6518154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f651815b000 0x7f651815b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65181586c0 0x7f6518158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651815df10 con 0x7f6504004d50 2024-04-25T09:01:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651815ec00 con 0x7f651815b000 2024-04-25T09:01:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651815e090 con 0x7f65181586c0 2024-04-25T09:01:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 0x7f6518154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 0x7f6518154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59860/0 (socket says 172.21.15.37:59860) 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651659c640 1 -- 172.21.15.37:0/2409054315 learned_addr learned my addr 172.21.15.37:0/2409054315 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f6516d9d640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65181586c0 0x7f6518158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f650ffff640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f651815b000 0x7f651815b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651659c640 1 -- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65181586c0 msgr2=0x7f6518158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651659c640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65181586c0 0x7f6518158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651659c640 1 -- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f651815b000 msgr2=0x7f651815b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651659c640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f651815b000 0x7f651815b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651659c640 1 -- 172.21.15.37:0/2409054315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6518159430 con 0x7f6504004d50 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651659c640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 0x7f6518154630 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f6508007fd0 tx=0x7f650800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f650effd640 1 -- 172.21.15.37:0/2409054315 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508019070 con 0x7f6504004d50 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f650effd640 1 -- 172.21.15.37:0/2409054315 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650800de50 con 0x7f6504004d50 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 -- 172.21.15.37:0/2409054315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6518159710 con 0x7f6504004d50 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f650effd640 1 -- 172.21.15.37:0/2409054315 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65080047f0 con 0x7f6504004d50 2024-04-25T09:01:59.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.470+0000 7f651cc15640 1 -- 172.21.15.37:0/2409054315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6518159cd0 con 0x7f6504004d50 2024-04-25T09:01:59.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.474+0000 7f651cc15640 1 -- 172.21.15.37:0/2409054315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f651804efa0 con 0x7f6504004d50 2024-04-25T09:01:59.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.474+0000 7f650effd640 1 -- 172.21.15.37:0/2409054315 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f650801c070 con 0x7f6504004d50 2024-04-25T09:01:59.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.474+0000 7f650effd640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64f8066f90 0x7f64f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.474+0000 7f650effd640 1 -- 172.21.15.37:0/2409054315 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6508084ae0 con 0x7f6504004d50 2024-04-25T09:01:59.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.474+0000 7f650ffff640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64f8066f90 0x7f64f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.474+0000 7f650ffff640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64f8066f90 0x7f64f8069450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f65180b0d80 tx=0x7f6500009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.478+0000 7f650effd640 1 -- 172.21.15.37:0/2409054315 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6508055310 con 0x7f6504004d50 2024-04-25T09:01:59.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.654+0000 7f651cc15640 1 -- 172.21.15.37:0/2409054315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f65180727c0 con 0x7f6504004d50 2024-04-25T09:01:59.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.654+0000 7f650effd640 1 -- 172.21.15.37:0/2409054315 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7f65080593b0 con 0x7f6504004d50 2024-04-25T09:01:59.644 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T09:01:59.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 -- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64f8066f90 msgr2=0x7f64f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64f8066f90 0x7f64f8069450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f65180b0d80 tx=0x7f6500009040 comp rx=0 tx=0).stop 2024-04-25T09:01:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 -- 172.21.15.37:0/2409054315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 msgr2=0x7f6518154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 0x7f6518154630 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f6508007fd0 tx=0x7f650800dc00 comp rx=0 tx=0).stop 2024-04-25T09:01:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 -- 172.21.15.37:0/2409054315 shutdown_connections 2024-04-25T09:01:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f64f8066f90 0x7f64f8069450 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f65181586c0 0x7f6518158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f651815b000 0x7f651815b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 --2- 172.21.15.37:0/2409054315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6504004d50 0x7f6518154630 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 -- 172.21.15.37:0/2409054315 >> 172.21.15.37:0/2409054315 conn(0x7f6518071810 msgr2=0x7f651806bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 -- 172.21.15.37:0/2409054315 shutdown_connections 2024-04-25T09:01:59.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.658+0000 7f651cc15640 1 -- 172.21.15.37:0/2409054315 wait complete. 2024-04-25T09:01:59.664 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:01:59.664 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:59.669 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e94c0b640 1 Processor -- start 2024-04-25T09:01:59.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e94c0b640 1 -- start start 2024-04-25T09:01:59.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e94c0b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e900ce510 0x7f2e900ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e94c0b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2e901451f0 0x7f2e90142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e94c0b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e900cee30 con 0x7f2e90144ca0 2024-04-25T09:01:59.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e94c0b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e900cc2b0 con 0x7f2e901451f0 2024-04-25T09:01:59.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e94c0b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9006e290 con 0x7f2e900ce510 2024-04-25T09:01:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e8e59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e900ce510 0x7f2e900ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e8ed9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2e901451f0 0x7f2e90142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e8e59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e900ce510 0x7f2e900ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:50398/0 (socket says 172.21.15.37:50398) 2024-04-25T09:01:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e8e59c640 1 -- 172.21.15.37:0/3190592568 learned_addr learned my addr 172.21.15.37:0/3190592568 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e8e59c640 1 -- 172.21.15.37:0/3190592568 >> v2:172.21.15.203:3300/0 conn(0x7f2e901451f0 msgr2=0x7f2e90142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:59.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e8e59c640 1 --2- 172.21.15.37:0/3190592568 >> v2:172.21.15.203:3300/0 conn(0x7f2e901451f0 0x7f2e90142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e8e59c640 1 -- 172.21.15.37:0/3190592568 >> v1:172.21.15.203:6789/0 conn(0x7f2e90144ca0 legacy=0x7f2e9006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.850+0000 7f2e8e59c640 1 -- 172.21.15.37:0/3190592568 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e900ccc70 con 0x7f2e900ce510 2024-04-25T09:01:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e8e59c640 1 --2- 172.21.15.37:0/3190592568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e900ce510 0x7f2e900ce8f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009870 tx=0x7f2e8001ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=747a6c3b50c77b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e8d59a640 1 -- 172.21.15.37:0/3190592568 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e800b5070 con 0x7f2e900ce510 2024-04-25T09:01:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e8d59a640 1 -- 172.21.15.37:0/3190592568 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e800a8a50 con 0x7f2e900ce510 2024-04-25T09:01:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e8d59a640 1 -- 172.21.15.37:0/3190592568 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e800a8e20 con 0x7f2e900ce510 2024-04-25T09:01:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e94c0b640 1 -- 172.21.15.37:0/3190592568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e900ce510 msgr2=0x7f2e900ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e94c0b640 1 --2- 172.21.15.37:0/3190592568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e900ce510 0x7f2e900ce8f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009870 tx=0x7f2e8001ed50 comp rx=0 tx=0).stop 2024-04-25T09:01:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e94c0b640 1 -- 172.21.15.37:0/3190592568 shutdown_connections 2024-04-25T09:01:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e94c0b640 1 --2- 172.21.15.37:0/3190592568 >> v2:172.21.15.203:3300/0 conn(0x7f2e901451f0 0x7f2e90142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e94c0b640 1 --2- 172.21.15.37:0/3190592568 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e900ce510 0x7f2e900ce8f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e94c0b640 1 -- 172.21.15.37:0/3190592568 >> 172.21.15.37:0/3190592568 conn(0x7f2e900b11a0 msgr2=0x7f2e900b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e94c0b640 1 -- 172.21.15.37:0/3190592568 shutdown_connections 2024-04-25T09:01:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e94c0b640 1 -- 172.21.15.37:0/3190592568 wait complete. 2024-04-25T09:01:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e94c0b640 1 Processor -- start 2024-04-25T09:01:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.854+0000 7f2e94c0b640 1 -- start start 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e94c0b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e900ce510 0x7f2e90143870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e94c0b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e90144ca0 0x7f2e901e5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e94c0b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e901451f0 0x7f2e901e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e94c0b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9006ee30 con 0x7f2e901451f0 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e94c0b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9006df20 con 0x7f2e900ce510 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e94c0b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e900cc900 con 0x7f2e90144ca0 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e901451f0 0x7f2e901e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e901451f0 0x7f2e901e5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59862/0 (socket says 172.21.15.37:59862) 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8ed9d640 1 -- 172.21.15.37:0/53815763 learned_addr learned my addr 172.21.15.37:0/53815763 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8e59c640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e900ce510 0x7f2e90143870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8dd9b640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e90144ca0 0x7f2e901e5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8ed9d640 1 -- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e90144ca0 msgr2=0x7f2e901e5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8ed9d640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e90144ca0 0x7f2e901e5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8ed9d640 1 -- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e900ce510 msgr2=0x7f2e90143870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8ed9d640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e900ce510 0x7f2e90143870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8ed9d640 1 -- 172.21.15.37:0/53815763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e901e66a0 con 0x7f2e901451f0 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8ed9d640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e901451f0 0x7f2e901e5e80 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800efc0 tx=0x7f2e8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e7effd640 1 -- 172.21.15.37:0/53815763 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88018020 con 0x7f2e901451f0 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e7effd640 1 -- 172.21.15.37:0/53815763 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e88014480 con 0x7f2e901451f0 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e94c0b640 1 -- 172.21.15.37:0/53815763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e901f1710 con 0x7f2e901451f0 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e7effd640 1 -- 172.21.15.37:0/53815763 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e88014870 con 0x7f2e901451f0 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e94c0b640 1 -- 172.21.15.37:0/53815763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e901f1cd0 con 0x7f2e901451f0 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e94c0b640 1 -- 172.21.15.37:0/53815763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e9004efa0 con 0x7f2e901451f0 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e7effd640 1 -- 172.21.15.37:0/53815763 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2e8801b020 con 0x7f2e901451f0 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e7effd640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e70066f60 0x7f2e70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e7effd640 1 -- 172.21.15.37:0/53815763 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2e88085ba0 con 0x7f2e901451f0 2024-04-25T09:01:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.858+0000 7f2e8e59c640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e70066f60 0x7f2e70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.862+0000 7f2e8e59c640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e70066f60 0x7f2e70069420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2e800b2000 tx=0x7f2e80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:01:59.862+0000 7f2e7effd640 1 -- 172.21.15.37:0/53815763 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2e88022070 con 0x7f2e901451f0 2024-04-25T09:02:00.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.026+0000 7f2e94c0b640 1 -- 172.21.15.37:0/53815763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e900b1cc0 con 0x7f2e901451f0 2024-04-25T09:02:00.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.026+0000 7f2e7effd640 1 -- 172.21.15.37:0/53815763 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16848 (secure 0 0 0) 0x7f2e88010040 con 0x7f2e901451f0 2024-04-25T09:02:00.015 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:00.015 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:01:36.569159+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:33.212328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:33.895814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.026+0000 7f2e94c0b640 1 -- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e70066f60 msgr2=0x7f2e70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.026+0000 7f2e94c0b640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e70066f60 0x7f2e70069420 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2e800b2000 tx=0x7f2e80008040 comp rx=0 tx=0).stop 2024-04-25T09:02:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.026+0000 7f2e94c0b640 1 -- 172.21.15.37:0/53815763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e901451f0 msgr2=0x7f2e901e5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.026+0000 7f2e94c0b640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e901451f0 0x7f2e901e5e80 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800efc0 tx=0x7f2e8800c490 comp rx=0 tx=0).stop 2024-04-25T09:02:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.026+0000 7f2e94c0b640 1 -- 172.21.15.37:0/53815763 shutdown_connections 2024-04-25T09:02:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.026+0000 7f2e94c0b640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2e70066f60 0x7f2e70069420 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.026+0000 7f2e94c0b640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e901451f0 0x7f2e901e5e80 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.030+0000 7f2e94c0b640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2e90144ca0 0x7f2e901e5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.030+0000 7f2e94c0b640 1 --2- 172.21.15.37:0/53815763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2e900ce510 0x7f2e90143870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.030+0000 7f2e94c0b640 1 -- 172.21.15.37:0/53815763 >> 172.21.15.37:0/53815763 conn(0x7f2e900b11a0 msgr2=0x7f2e900b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.030+0000 7f2e94c0b640 1 -- 172.21.15.37:0/53815763 shutdown_connections 2024-04-25T09:02:00.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.030+0000 7f2e94c0b640 1 -- 172.21.15.37:0/53815763 wait complete. 2024-04-25T09:02:00.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:01:33.212328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:01:33.895814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:02:00.035 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:02:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79be514640 1 Processor -- start 2024-04-25T09:02:00.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79be514640 1 -- start start 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79be514640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f79b815aac0 0x7f79b80ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79be514640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f79b80cf0e0 0x7f79b80cd180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79be514640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b80ce320 con 0x7f79b815aea0 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79be514640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b80ce0d0 con 0x7f79b815aac0 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79be514640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b80ce600 con 0x7f79b80cf0e0 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79b7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f79b815aac0 0x7f79b80ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79b7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f79b815aac0 0x7f79b80ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59878/0 (socket says 172.21.15.37:59878) 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79b7fff640 1 -- 172.21.15.37:0/74371318 learned_addr learned my addr 172.21.15.37:0/74371318 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79b77fe640 1 --2- 172.21.15.37:0/74371318 >> v2:172.21.15.203:3300/0 conn(0x7f79b80cf0e0 0x7f79b80cd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79b7fff640 1 -- 172.21.15.37:0/74371318 >> v2:172.21.15.203:3300/0 conn(0x7f79b80cf0e0 msgr2=0x7f79b80cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79b7fff640 1 --2- 172.21.15.37:0/74371318 >> v2:172.21.15.203:3300/0 conn(0x7f79b80cf0e0 0x7f79b80cd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79b7fff640 1 -- 172.21.15.37:0/74371318 >> v1:172.21.15.203:6789/0 conn(0x7f79b815aea0 legacy=0x7f79b815b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:00.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.170+0000 7f79b7fff640 1 -- 172.21.15.37:0/74371318 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b80ce8e0 con 0x7f79b815aac0 2024-04-25T09:02:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b7fff640 1 --2- 172.21.15.37:0/74371318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b815aac0 0x7f79b80ccc40 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f79b815e1b0 tx=0x7f79a801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f7940f74556640 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b6ffd640 1 -- 172.21.15.37:0/74371318 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a80b0070 con 0x7f79b815aac0 2024-04-25T09:02:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b6ffd640 1 -- 172.21.15.37:0/74371318 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a80ab600 con 0x7f79b815aac0 2024-04-25T09:02:00.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b6ffd640 1 -- 172.21.15.37:0/74371318 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a80ab9d0 con 0x7f79b815aac0 2024-04-25T09:02:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 -- 172.21.15.37:0/74371318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b815aac0 msgr2=0x7f79b80ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 --2- 172.21.15.37:0/74371318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b815aac0 0x7f79b80ccc40 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f79b815e1b0 tx=0x7f79a801c4e0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 -- 172.21.15.37:0/74371318 shutdown_connections 2024-04-25T09:02:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 --2- 172.21.15.37:0/74371318 >> v2:172.21.15.203:3300/0 conn(0x7f79b80cf0e0 0x7f79b80cd180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 --2- 172.21.15.37:0/74371318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b815aac0 0x7f79b80ccc40 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 -- 172.21.15.37:0/74371318 >> 172.21.15.37:0/74371318 conn(0x7f79b8099a70 msgr2=0x7f79b80d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 -- 172.21.15.37:0/74371318 shutdown_connections 2024-04-25T09:02:00.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 -- 172.21.15.37:0/74371318 wait complete. 2024-04-25T09:02:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 Processor -- start 2024-04-25T09:02:00.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 -- start start 2024-04-25T09:02:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b80cf0e0 0x7f79b80c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b815aac0 0x7f79b814f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79b815aea0 0x7f79b80c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b806b830 con 0x7f79b815aac0 2024-04-25T09:02:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b806b6b0 con 0x7f79b80cf0e0 2024-04-25T09:02:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79be514640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b806b9b0 con 0x7f79b815aea0 2024-04-25T09:02:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b815aac0 0x7f79b814f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b815aac0 0x7f79b814f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59894/0 (socket says 172.21.15.37:59894) 2024-04-25T09:02:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b77fe640 1 -- 172.21.15.37:0/4174394590 learned_addr learned my addr 172.21.15.37:0/4174394590 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b7fff640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b80cf0e0 0x7f79b80c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79bcab1640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79b815aea0 0x7f79b80c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b77fe640 1 -- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79b815aea0 msgr2=0x7f79b80c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b77fe640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79b815aea0 0x7f79b80c4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b77fe640 1 -- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b80cf0e0 msgr2=0x7f79b80c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b77fe640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b80cf0e0 0x7f79b80c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b77fe640 1 -- 172.21.15.37:0/4174394590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b80c8e80 con 0x7f79b815aac0 2024-04-25T09:02:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.174+0000 7f79b77fe640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b815aac0 0x7f79b814f9f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c7f0 tx=0x7f79a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.178+0000 7f79b4ff9640 1 -- 172.21.15.37:0/4174394590 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0015070 con 0x7f79b815aac0 2024-04-25T09:02:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.178+0000 7f79b4ff9640 1 -- 172.21.15.37:0/4174394590 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a0004240 con 0x7f79b815aac0 2024-04-25T09:02:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.178+0000 7f79be514640 1 -- 172.21.15.37:0/4174394590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b80c9160 con 0x7f79b815aac0 2024-04-25T09:02:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.178+0000 7f79b4ff9640 1 -- 172.21.15.37:0/4174394590 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0003620 con 0x7f79b815aac0 2024-04-25T09:02:00.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.178+0000 7f79be514640 1 -- 172.21.15.37:0/4174394590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b80c9720 con 0x7f79b815aac0 2024-04-25T09:02:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.178+0000 7f79be514640 1 -- 172.21.15.37:0/4174394590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b804efa0 con 0x7f79b815aac0 2024-04-25T09:02:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.178+0000 7f79b4ff9640 1 -- 172.21.15.37:0/4174394590 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f79a00039d0 con 0x7f79b815aac0 2024-04-25T09:02:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.178+0000 7f79b4ff9640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7998066f60 0x7f7998069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.178+0000 7f79b4ff9640 1 -- 172.21.15.37:0/4174394590 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79a00852d0 con 0x7f79b815aac0 2024-04-25T09:02:00.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.178+0000 7f79b7fff640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7998066f60 0x7f7998069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.178+0000 7f79b7fff640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7998066f60 0x7f7998069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f79a80ad000 tx=0x7f79a80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:00.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.182+0000 7f79b4ff9640 1 -- 172.21.15.37:0/4174394590 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f79a00560d0 con 0x7f79b815aac0 2024-04-25T09:02:00.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 -- 172.21.15.37:0/4174394590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79b80b6480 con 0x7f79b815aac0 2024-04-25T09:02:00.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79b4ff9640 1 -- 172.21.15.37:0/4174394590 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79a00850f0 con 0x7f79b815aac0 2024-04-25T09:02:00.334 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 -- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7998066f60 msgr2=0x7f7998069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7998066f60 0x7f7998069420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f79a80ad000 tx=0x7f79a80a6000 comp rx=0 tx=0).stop 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 -- 172.21.15.37:0/4174394590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b815aac0 msgr2=0x7f79b814f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b815aac0 0x7f79b814f9f0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f79a000c7f0 tx=0x7f79a000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 -- 172.21.15.37:0/4174394590 shutdown_connections 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7998066f60 0x7f7998069420 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f79b815aea0 0x7f79b80c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79b815aac0 0x7f79b814f9f0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 --2- 172.21.15.37:0/4174394590 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f79b80cf0e0 0x7f79b80c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 -- 172.21.15.37:0/4174394590 >> 172.21.15.37:0/4174394590 conn(0x7f79b8099a70 msgr2=0x7f79b80aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 -- 172.21.15.37:0/4174394590 shutdown_connections 2024-04-25T09:02:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.346+0000 7f79be514640 1 -- 172.21.15.37:0/4174394590 wait complete. 2024-04-25T09:02:00.354 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:02:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c5bf85640 1 Processor -- start 2024-04-25T09:02:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c5bf85640 1 -- start start 2024-04-25T09:02:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c5bf85640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1c5415da80 0x7f1c540d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c5bf85640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c540d1cd0 0x7f1c540d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c5bf85640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5406be60 con 0x7f1c540d0620 2024-04-25T09:02:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c5bf85640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5406c440 con 0x7f1c5415da80 2024-04-25T09:02:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c5bf85640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5415de60 con 0x7f1c540d1cd0 2024-04-25T09:02:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c5a522640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1c540d0620 0x7f1c540ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46844/0 (socket says 172.21.15.37:46844) 2024-04-25T09:02:00.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c5a522640 1 -- 172.21.15.37:0/3767240097 learned_addr learned my addr 172.21.15.37:0/3767240097 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c59d21640 1 --2- 172.21.15.37:0/3767240097 >> v2:172.21.15.37:3300/0 conn(0x7f1c5415da80 0x7f1c540d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c59520640 1 --2- 172.21.15.37:0/3767240097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c540d1cd0 0x7f1c540d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c58d1f640 1 -- 172.21.15.37:0/3767240097 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3757591134 0 0) 0x7f1c5406be60 con 0x7f1c540d0620 2024-04-25T09:02:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c58d1f640 1 -- 172.21.15.37:0/3767240097 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c40003530 con 0x7f1c540d0620 2024-04-25T09:02:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c59d21640 1 -- 172.21.15.37:0/3767240097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c540d1cd0 msgr2=0x7f1c540d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c59d21640 1 --2- 172.21.15.37:0/3767240097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c540d1cd0 0x7f1c540d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c59d21640 1 -- 172.21.15.37:0/3767240097 >> v1:172.21.15.37:6789/0 conn(0x7f1c540d0620 legacy=0x7f1c540ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c59d21640 1 -- 172.21.15.37:0/3767240097 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c540d2450 con 0x7f1c5415da80 2024-04-25T09:02:00.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.490+0000 7f1c59520640 1 --2- 172.21.15.37:0/3767240097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c540d1cd0 0x7f1c540d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c59d21640 1 --2- 172.21.15.37:0/3767240097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c5415da80 0x7f1c540d4230 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1c500097c0 tx=0x7f1c5001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79c289cbbef8209 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c58d1f640 1 -- 172.21.15.37:0/3767240097 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c500b0070 con 0x7f1c5415da80 2024-04-25T09:02:00.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c58d1f640 1 -- 172.21.15.37:0/3767240097 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c500ab600 con 0x7f1c5415da80 2024-04-25T09:02:00.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c58d1f640 1 -- 172.21.15.37:0/3767240097 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c500ab9d0 con 0x7f1c5415da80 2024-04-25T09:02:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c5bf85640 1 -- 172.21.15.37:0/3767240097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c5415da80 msgr2=0x7f1c540d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c5bf85640 1 --2- 172.21.15.37:0/3767240097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c5415da80 0x7f1c540d4230 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f1c500097c0 tx=0x7f1c5001c4a0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c5bf85640 1 -- 172.21.15.37:0/3767240097 shutdown_connections 2024-04-25T09:02:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c5bf85640 1 --2- 172.21.15.37:0/3767240097 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c540d1cd0 0x7f1c540d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c5bf85640 1 --2- 172.21.15.37:0/3767240097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c5415da80 0x7f1c540d4230 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c5bf85640 1 -- 172.21.15.37:0/3767240097 >> 172.21.15.37:0/3767240097 conn(0x7f1c5408b000 msgr2=0x7f1c540d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c5bf85640 1 -- 172.21.15.37:0/3767240097 shutdown_connections 2024-04-25T09:02:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c5bf85640 1 -- 172.21.15.37:0/3767240097 wait complete. 2024-04-25T09:02:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c5bf85640 1 Processor -- start 2024-04-25T09:02:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.494+0000 7f1c5bf85640 1 -- start start 2024-04-25T09:02:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c5bf85640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c540d0620 0x7f1c54156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c5bf85640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c540d1cd0 0x7f1c540b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c5bf85640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c5415da80 0x7f1c5415bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c5bf85640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5416a730 con 0x7f1c540d1cd0 2024-04-25T09:02:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c5bf85640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5416a5b0 con 0x7f1c5415da80 2024-04-25T09:02:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c5bf85640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5416a8b0 con 0x7f1c540d0620 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c59520640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c540d1cd0 0x7f1c540b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c59520640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c540d1cd0 0x7f1c540b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59912/0 (socket says 172.21.15.37:59912) 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c59520640 1 -- 172.21.15.37:0/504423840 learned_addr learned my addr 172.21.15.37:0/504423840 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c59d21640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c540d0620 0x7f1c54156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c59520640 1 -- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c540d0620 msgr2=0x7f1c54156510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c5a522640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c5415da80 0x7f1c5415bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c59520640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c540d0620 0x7f1c54156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c59520640 1 -- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c5415da80 msgr2=0x7f1c5415bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c59520640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c5415da80 0x7f1c5415bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c59520640 1 -- 172.21.15.37:0/504423840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c540b6db0 con 0x7f1c540d1cd0 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c59520640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c540d1cd0 0x7f1c540b6310 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400e7e0 tx=0x7f1c4400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c4a7fc640 1 -- 172.21.15.37:0/504423840 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4400f040 con 0x7f1c540d1cd0 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c4a7fc640 1 -- 172.21.15.37:0/504423840 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c440059a0 con 0x7f1c540d1cd0 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c5bf85640 1 -- 172.21.15.37:0/504423840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c540b7090 con 0x7f1c540d1cd0 2024-04-25T09:02:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c4a7fc640 1 -- 172.21.15.37:0/504423840 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44005d70 con 0x7f1c540d1cd0 2024-04-25T09:02:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c5bf85640 1 -- 172.21.15.37:0/504423840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c540b7e00 con 0x7f1c540d1cd0 2024-04-25T09:02:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c5bf85640 1 -- 172.21.15.37:0/504423840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5404efa0 con 0x7f1c540d1cd0 2024-04-25T09:02:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c4a7fc640 1 -- 172.21.15.37:0/504423840 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1c44004030 con 0x7f1c540d1cd0 2024-04-25T09:02:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c4a7fc640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c38066f60 0x7f1c38069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c4a7fc640 1 -- 172.21.15.37:0/504423840 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1c44085390 con 0x7f1c540d1cd0 2024-04-25T09:02:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.498+0000 7f1c59d21640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c38066f60 0x7f1c38069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.502+0000 7f1c59d21640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c38066f60 0x7f1c38069420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1c500ad000 tx=0x7f1c50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.502+0000 7f1c4a7fc640 1 -- 172.21.15.37:0/504423840 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1c44020020 con 0x7f1c540d1cd0 2024-04-25T09:02:00.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.666+0000 7f1c5bf85640 1 -- 172.21.15.37:0/504423840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1c540b0510 con 0x7f1c540d1cd0 2024-04-25T09:02:00.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.666+0000 7f1c4a7fc640 1 -- 172.21.15.37:0/504423840 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1c44017030 con 0x7f1c540d1cd0 2024-04-25T09:02:00.655 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:02:00.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.666+0000 7f1c5bf85640 1 -- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c38066f60 msgr2=0x7f1c38069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.666+0000 7f1c5bf85640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c38066f60 0x7f1c38069420 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f1c500ad000 tx=0x7f1c50008040 comp rx=0 tx=0).stop 2024-04-25T09:02:00.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.666+0000 7f1c5bf85640 1 -- 172.21.15.37:0/504423840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c540d1cd0 msgr2=0x7f1c540b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.666+0000 7f1c5bf85640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c540d1cd0 0x7f1c540b6310 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f1c4400e7e0 tx=0x7f1c4400eca0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.670+0000 7f1c5bf85640 1 -- 172.21.15.37:0/504423840 shutdown_connections 2024-04-25T09:02:00.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.670+0000 7f1c5bf85640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1c38066f60 0x7f1c38069420 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.670+0000 7f1c5bf85640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1c5415da80 0x7f1c5415bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.670+0000 7f1c5bf85640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c540d1cd0 0x7f1c540b6310 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.670+0000 7f1c5bf85640 1 --2- 172.21.15.37:0/504423840 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1c540d0620 0x7f1c54156510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.670+0000 7f1c5bf85640 1 -- 172.21.15.37:0/504423840 >> 172.21.15.37:0/504423840 conn(0x7f1c5408b000 msgr2=0x7f1c540b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:00.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.670+0000 7f1c5bf85640 1 -- 172.21.15.37:0/504423840 shutdown_connections 2024-04-25T09:02:00.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.670+0000 7f1c5bf85640 1 -- 172.21.15.37:0/504423840 wait complete. 2024-04-25T09:02:00.674 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:02:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073ec1e640 1 Processor -- start 2024-04-25T09:02:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073ec1e640 1 -- start start 2024-04-25T09:02:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073ec1e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f07380cf590 0x7f07380cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073ec1e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0738072000 0x7f0738072430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073ec1e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07380cbd20 con 0x7f073815b6b0 2024-04-25T09:02:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073ec1e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073815ba90 con 0x7f0738072000 2024-04-25T09:02:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073ec1e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07380cc000 con 0x7f07380cf590 2024-04-25T09:02:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073d1bb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0738072000 0x7f0738072430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073d1bb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0738072000 0x7f0738072430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59920/0 (socket says 172.21.15.37:59920) 2024-04-25T09:02:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073d1bb640 1 -- 172.21.15.37:0/3053473419 learned_addr learned my addr 172.21.15.37:0/3053473419 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f0737fff640 1 --2- 172.21.15.37:0/3053473419 >> v2:172.21.15.203:3300/0 conn(0x7f07380cf590 0x7f07380cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073d1bb640 1 -- 172.21.15.37:0/3053473419 >> v2:172.21.15.203:3300/0 conn(0x7f07380cf590 msgr2=0x7f07380cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073d1bb640 1 --2- 172.21.15.37:0/3053473419 >> v2:172.21.15.203:3300/0 conn(0x7f07380cf590 0x7f07380cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073d1bb640 1 -- 172.21.15.37:0/3053473419 >> v1:172.21.15.203:6789/0 conn(0x7f073815b6b0 legacy=0x7f07380ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.802+0000 7f073d1bb640 1 -- 172.21.15.37:0/3053473419 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07380cc2e0 con 0x7f0738072000 2024-04-25T09:02:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073d1bb640 1 --2- 172.21.15.37:0/3053473419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0738072000 0x7f0738072430 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f07380b4430 tx=0x7f073001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10aab089a9adb6fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f07377fe640 1 -- 172.21.15.37:0/3053473419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300a5760 con 0x7f0738072000 2024-04-25T09:02:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f07377fe640 1 -- 172.21.15.37:0/3053473419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07300a5900 con 0x7f0738072000 2024-04-25T09:02:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f07377fe640 1 -- 172.21.15.37:0/3053473419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07300a5cd0 con 0x7f0738072000 2024-04-25T09:02:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 -- 172.21.15.37:0/3053473419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0738072000 msgr2=0x7f0738072430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 --2- 172.21.15.37:0/3053473419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0738072000 0x7f0738072430 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f07380b4430 tx=0x7f073001c9b0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 -- 172.21.15.37:0/3053473419 shutdown_connections 2024-04-25T09:02:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 --2- 172.21.15.37:0/3053473419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0738072000 0x7f0738072430 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 --2- 172.21.15.37:0/3053473419 >> v2:172.21.15.203:3300/0 conn(0x7f07380cf590 0x7f07380cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 -- 172.21.15.37:0/3053473419 >> 172.21.15.37:0/3053473419 conn(0x7f073806ae60 msgr2=0x7f073806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 -- 172.21.15.37:0/3053473419 shutdown_connections 2024-04-25T09:02:00.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 -- 172.21.15.37:0/3053473419 wait complete. 2024-04-25T09:02:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 Processor -- start 2024-04-25T09:02:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 -- start start 2024-04-25T09:02:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0738072000 0x7f07381565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07380cf590 0x7f07381596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f073815b6b0 0x7f07381533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380caf30 con 0x7f07380cf590 2024-04-25T09:02:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380cadb0 con 0x7f0738072000 2024-04-25T09:02:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f073ec1e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07381682a0 con 0x7f073815b6b0 2024-04-25T09:02:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f0737fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07380cf590 0x7f07381596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f0737fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07380cf590 0x7f07381596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59926/0 (socket says 172.21.15.37:59926) 2024-04-25T09:02:00.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.806+0000 7f0737fff640 1 -- 172.21.15.37:0/142279495 learned_addr learned my addr 172.21.15.37:0/142279495 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f073d1bb640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f073815b6b0 0x7f07381533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f073c9ba640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0738072000 0x7f07381565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f0737fff640 1 -- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f073815b6b0 msgr2=0x7f07381533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f0737fff640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f073815b6b0 0x7f07381533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f0737fff640 1 -- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0738072000 msgr2=0x7f07381565f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f0737fff640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0738072000 0x7f07381565f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f0737fff640 1 -- 172.21.15.37:0/142279495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0738158f00 con 0x7f07380cf590 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f073d1bb640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f073815b6b0 0x7f07381533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f073c9ba640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0738072000 0x7f07381565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f0737fff640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07380cf590 0x7f07381596a0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f072c00d830 tx=0x7f072c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f07357fa640 1 -- 172.21.15.37:0/142279495 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00b6c0 con 0x7f07380cf590 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f07357fa640 1 -- 172.21.15.37:0/142279495 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c019070 con 0x7f07380cf590 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f073ec1e640 1 -- 172.21.15.37:0/142279495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07381591e0 con 0x7f07380cf590 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f07357fa640 1 -- 172.21.15.37:0/142279495 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c00ba60 con 0x7f07380cf590 2024-04-25T09:02:00.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f073ec1e640 1 -- 172.21.15.37:0/142279495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0738155a40 con 0x7f07380cf590 2024-04-25T09:02:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f073ec1e640 1 -- 172.21.15.37:0/142279495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073804efa0 con 0x7f07380cf590 2024-04-25T09:02:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f07357fa640 1 -- 172.21.15.37:0/142279495 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f072c01e460 con 0x7f07380cf590 2024-04-25T09:02:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f07357fa640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0724066f60 0x7f0724069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:00.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f07357fa640 1 -- 172.21.15.37:0/142279495 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f072c0858a0 con 0x7f07380cf590 2024-04-25T09:02:00.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f073c9ba640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0724066f60 0x7f0724069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:00.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.810+0000 7f073c9ba640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0724066f60 0x7f0724069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f072800a850 tx=0x7f0728008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:00.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.814+0000 7f07357fa640 1 -- 172.21.15.37:0/142279495 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f072c056720 con 0x7f07380cf590 2024-04-25T09:02:00.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.966+0000 7f073ec1e640 1 -- 172.21.15.37:0/142279495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0738158c90 con 0x7f07380cf590 2024-04-25T09:02:00.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f07357fa640 1 -- 172.21.15.37:0/142279495 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f072c056020 con 0x7f07380cf590 2024-04-25T09:02:00.959 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:02:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 -- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0724066f60 msgr2=0x7f0724069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0724066f60 0x7f0724069420 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f072800a850 tx=0x7f0728008040 comp rx=0 tx=0).stop 2024-04-25T09:02:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 -- 172.21.15.37:0/142279495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07380cf590 msgr2=0x7f07381596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07380cf590 0x7f07381596a0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f072c00d830 tx=0x7f072c00dcf0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 -- 172.21.15.37:0/142279495 shutdown_connections 2024-04-25T09:02:00.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0724066f60 0x7f0724069420 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f073815b6b0 0x7f07381533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07380cf590 0x7f07381596a0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 --2- 172.21.15.37:0/142279495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0738072000 0x7f07381565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 -- 172.21.15.37:0/142279495 >> 172.21.15.37:0/142279495 conn(0x7f073806ae60 msgr2=0x7f07380b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 -- 172.21.15.37:0/142279495 shutdown_connections 2024-04-25T09:02:00.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:00.970+0000 7f073ec1e640 1 -- 172.21.15.37:0/142279495 wait complete. 2024-04-25T09:02:00.975 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:02:01.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711f3bf640 1 Processor -- start 2024-04-25T09:02:01.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711f3bf640 1 -- start start 2024-04-25T09:02:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711f3bf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f71180d0870 0x7f71180d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711f3bf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f711815dc30 0x7f71180d2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711f3bf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71180ce6b0 con 0x7f711815d850 2024-04-25T09:02:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711f3bf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71180ce990 con 0x7f711815dc30 2024-04-25T09:02:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711f3bf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71180cec70 con 0x7f71180d0870 2024-04-25T09:02:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711d15b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f711815d850 0x7f71180d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46856/0 (socket says 172.21.15.37:46856) 2024-04-25T09:02:01.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711d15b640 1 -- 172.21.15.37:0/2983795137 learned_addr learned my addr 172.21.15.37:0/2983795137 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711d95c640 1 --2- 172.21.15.37:0/2983795137 >> v2:172.21.15.37:3300/0 conn(0x7f711815dc30 0x7f71180d2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711c95a640 1 --2- 172.21.15.37:0/2983795137 >> v2:172.21.15.203:3300/0 conn(0x7f71180d0870 0x7f71180d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f710ffff640 1 -- 172.21.15.37:0/2983795137 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807411361 0 0) 0x7f71180ce6b0 con 0x7f711815d850 2024-04-25T09:02:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f710ffff640 1 -- 172.21.15.37:0/2983795137 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7104003530 con 0x7f711815d850 2024-04-25T09:02:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711d95c640 1 -- 172.21.15.37:0/2983795137 >> v2:172.21.15.203:3300/0 conn(0x7f71180d0870 msgr2=0x7f71180d1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711d95c640 1 --2- 172.21.15.37:0/2983795137 >> v2:172.21.15.203:3300/0 conn(0x7f71180d0870 0x7f71180d1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711d95c640 1 -- 172.21.15.37:0/2983795137 >> v1:172.21.15.37:6789/0 conn(0x7f711815d850 legacy=0x7f71180d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.102+0000 7f711d95c640 1 -- 172.21.15.37:0/2983795137 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180cef50 con 0x7f711815dc30 2024-04-25T09:02:01.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f711d95c640 1 --2- 172.21.15.37:0/2983795137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f711815dc30 0x7f71180d2320 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f711808b930 tx=0x7f711001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85106b46dda669e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:01.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f710ffff640 1 -- 172.21.15.37:0/2983795137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71100b1070 con 0x7f711815dc30 2024-04-25T09:02:01.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f710ffff640 1 -- 172.21.15.37:0/2983795137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71100ac5f0 con 0x7f711815dc30 2024-04-25T09:02:01.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f710ffff640 1 -- 172.21.15.37:0/2983795137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71100ac9c0 con 0x7f711815dc30 2024-04-25T09:02:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f711f3bf640 1 -- 172.21.15.37:0/2983795137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f711815dc30 msgr2=0x7f71180d2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f711f3bf640 1 --2- 172.21.15.37:0/2983795137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f711815dc30 0x7f71180d2320 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f711808b930 tx=0x7f711001c480 comp rx=0 tx=0).stop 2024-04-25T09:02:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f711f3bf640 1 -- 172.21.15.37:0/2983795137 shutdown_connections 2024-04-25T09:02:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f711f3bf640 1 --2- 172.21.15.37:0/2983795137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f711815dc30 0x7f71180d2320 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f711f3bf640 1 --2- 172.21.15.37:0/2983795137 >> v2:172.21.15.203:3300/0 conn(0x7f71180d0870 0x7f71180d1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f711f3bf640 1 -- 172.21.15.37:0/2983795137 >> 172.21.15.37:0/2983795137 conn(0x7f711808b000 msgr2=0x7f71180d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:01.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f711f3bf640 1 -- 172.21.15.37:0/2983795137 shutdown_connections 2024-04-25T09:02:01.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f711f3bf640 1 -- 172.21.15.37:0/2983795137 wait complete. 2024-04-25T09:02:01.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f711f3bf640 1 Processor -- start 2024-04-25T09:02:01.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.106+0000 7f711f3bf640 1 -- start start 2024-04-25T09:02:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711f3bf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71180d0870 0x7f7118154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711f3bf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f711815d850 0x7f71180b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711f3bf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f711815dc30 0x7f7118154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711f3bf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711816a4c0 con 0x7f711815d850 2024-04-25T09:02:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711f3bf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711816a340 con 0x7f711815dc30 2024-04-25T09:02:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711f3bf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711816a640 con 0x7f71180d0870 2024-04-25T09:02:01.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711c95a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f711815d850 0x7f71180b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711c95a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f711815d850 0x7f71180b8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59950/0 (socket says 172.21.15.37:59950) 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711c95a640 1 -- 172.21.15.37:0/1218956933 learned_addr learned my addr 172.21.15.37:0/1218956933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711d15b640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71180d0870 0x7f7118154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711c95a640 1 -- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71180d0870 msgr2=0x7f7118154c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711d95c640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f711815dc30 0x7f7118154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711c95a640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71180d0870 0x7f7118154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711c95a640 1 -- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f711815dc30 msgr2=0x7f7118154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711c95a640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f711815dc30 0x7f7118154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711c95a640 1 -- 172.21.15.37:0/1218956933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180cbb50 con 0x7f711815d850 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711c95a640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f711815d850 0x7f71180b8dd0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f710800d5f0 tx=0x7f710800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f710dffb640 1 -- 172.21.15.37:0/1218956933 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71080125f0 con 0x7f711815d850 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f710dffb640 1 -- 172.21.15.37:0/1218956933 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108019070 con 0x7f711815d850 2024-04-25T09:02:01.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711f3bf640 1 -- 172.21.15.37:0/1218956933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7118155240 con 0x7f711815d850 2024-04-25T09:02:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f710dffb640 1 -- 172.21.15.37:0/1218956933 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71080129b0 con 0x7f711815d850 2024-04-25T09:02:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711f3bf640 1 -- 172.21.15.37:0/1218956933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7118155ae0 con 0x7f711815d850 2024-04-25T09:02:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711f3bf640 1 -- 172.21.15.37:0/1218956933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7118155630 con 0x7f711815d850 2024-04-25T09:02:01.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f710dffb640 1 -- 172.21.15.37:0/1218956933 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7108012b50 con 0x7f711815d850 2024-04-25T09:02:01.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f710dffb640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70fc066f60 0x7f70fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f710dffb640 1 -- 172.21.15.37:0/1218956933 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71080855e0 con 0x7f711815d850 2024-04-25T09:02:01.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.110+0000 7f711d15b640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70fc066f60 0x7f70fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.114+0000 7f711d15b640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70fc066f60 0x7f70fc069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f71140012f0 tx=0x7f7114009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:01.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.114+0000 7f710dffb640 1 -- 172.21.15.37:0/1218956933 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7108056460 con 0x7f711815d850 2024-04-25T09:02:01.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.286+0000 7f711f3bf640 1 -- 172.21.15.37:0/1218956933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f71180d0e60 con 0x7f711815d850 2024-04-25T09:02:01.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.286+0000 7f710dffb640 1 -- 172.21.15.37:0/1218956933 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7108055d60 con 0x7f711815d850 2024-04-25T09:02:01.276 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T09:02:01.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 -- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70fc066f60 msgr2=0x7f70fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70fc066f60 0x7f70fc069420 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f71140012f0 tx=0x7f7114009040 comp rx=0 tx=0).stop 2024-04-25T09:02:01.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 -- 172.21.15.37:0/1218956933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f711815d850 msgr2=0x7f71180b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f711815d850 0x7f71180b8dd0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f710800d5f0 tx=0x7f710800dab0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 -- 172.21.15.37:0/1218956933 shutdown_connections 2024-04-25T09:02:01.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f70fc066f60 0x7f70fc069420 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f711815dc30 0x7f7118154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f711815d850 0x7f71180b8dd0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 --2- 172.21.15.37:0/1218956933 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f71180d0870 0x7f7118154c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 -- 172.21.15.37:0/1218956933 >> 172.21.15.37:0/1218956933 conn(0x7f711808b000 msgr2=0x7f711806bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:01.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 -- 172.21.15.37:0/1218956933 shutdown_connections 2024-04-25T09:02:01.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:01.290+0000 7f711f3bf640 1 -- 172.21.15.37:0/1218956933 wait complete. 2024-04-25T09:02:01.294 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T09:02:01.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:01.295 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:02:01.305 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[52070]: fuse finished with error 0 and tester_r 0 2024-04-25T09:02:01.307 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:02:07.409 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:02:07.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:07.410 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:02:07.417 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:07.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.606+0000 7fe428616640 1 Processor -- start 2024-04-25T09:02:07.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.606+0000 7fe428616640 1 -- start start 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe428616640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe42015da80 0x7fe4200d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe428616640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42006be60 con 0x7fe4200d1cd0 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe428616640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe42015de60 con 0x7fe4200d07f0 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe428616640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42006c440 con 0x7fe42015da80 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe425bb1640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe4200d1cd0 0x7fe4200d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46860/0 (socket says 172.21.15.37:46860) 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe425bb1640 1 -- 172.21.15.37:0/1605393012 learned_addr learned my addr 172.21.15.37:0/1605393012 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4263b2640 1 --2- 172.21.15.37:0/1605393012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe42015da80 0x7fe4200d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3028143770 0 0) 0x7fe42006be60 con 0x7fe4200d1cd0 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe40c003530 con 0x7fe4200d1cd0 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2844461133 0 0) 0x7fe40c003530 con 0x7fe4200d1cd0 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe42015da80 msgr2=0x7fe4200d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 --2- 172.21.15.37:0/1605393012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe42015da80 0x7fe4200d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 >> v1:172.21.15.203:6789/0 conn(0x7fe4200d07f0 legacy=0x7fe42006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4200d2450 con 0x7fe4200d1cd0 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fe414001800 con 0x7fe4200d1cd0 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 >> v1:172.21.15.37:6789/0 conn(0x7fe4200d1cd0 legacy=0x7fe4200d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 --2- 172.21.15.37:0/1605393012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 0x7fe40c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 --2- 172.21.15.37:0/1605393012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe42015da80 0x7fe40c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 --2- 172.21.15.37:0/1605393012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe40c004d50 0x7fe40c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42006c440 con 0x7fe40c003a60 2024-04-25T09:02:07.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c00bb00 con 0x7fe40c004d50 2024-04-25T09:02:07.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c00bd00 con 0x7fe42015da80 2024-04-25T09:02:07.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4263b2640 1 --2- 172.21.15.37:0/1605393012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 0x7fe40c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe426bb3640 1 --2- 172.21.15.37:0/1605393012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe40c004d50 0x7fe40c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4263b2640 1 -- 172.21.15.37:0/1605393012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe42015da80 msgr2=0x7fe40c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:07.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4263b2640 1 --2- 172.21.15.37:0/1605393012 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe42015da80 0x7fe40c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4263b2640 1 -- 172.21.15.37:0/1605393012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe40c004d50 msgr2=0x7fe40c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:07.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4263b2640 1 --2- 172.21.15.37:0/1605393012 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe40c004d50 0x7fe40c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.610+0000 7fe4263b2640 1 -- 172.21.15.37:0/1605393012 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c003530 con 0x7fe40c003a60 2024-04-25T09:02:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4263b2640 1 --2- 172.21.15.37:0/1605393012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 0x7fe40c003ea0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009ae0 tx=0x7fe41c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd3731d1f0a7ef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c0096b0 con 0x7fe40c003a60 2024-04-25T09:02:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41c0a64b0 con 0x7fe40c003a60 2024-04-25T09:02:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4253b0640 1 -- 172.21.15.37:0/1605393012 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c0a68a0 con 0x7fe40c003a60 2024-04-25T09:02:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 -- 172.21.15.37:0/1605393012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 msgr2=0x7fe40c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 --2- 172.21.15.37:0/1605393012 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 0x7fe40c003ea0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009ae0 tx=0x7fe41c01c9c0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe426bb3640 1 -- 172.21.15.37:0/1605393012 reap_dead start 2024-04-25T09:02:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 -- 172.21.15.37:0/1605393012 shutdown_connections 2024-04-25T09:02:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 -- 172.21.15.37:0/1605393012 >> 172.21.15.37:0/1605393012 conn(0x7fe42008b000 msgr2=0x7fe4200d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 -- 172.21.15.37:0/1605393012 shutdown_connections 2024-04-25T09:02:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 -- 172.21.15.37:0/1605393012 wait complete. 2024-04-25T09:02:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 Processor -- start 2024-04-25T09:02:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 -- start start 2024-04-25T09:02:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 0x7fe420157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4200b5d20 0x7fe4200b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4200b3e80 0x7fe4200c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42006d970 con 0x7fe40c003a60 2024-04-25T09:02:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42006d7f0 con 0x7fe4200b3e80 2024-04-25T09:02:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe428616640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42006daf0 con 0x7fe4200b5d20 2024-04-25T09:02:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4263b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 0x7fe420157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4263b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 0x7fe420157d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59972/0 (socket says 172.21.15.37:59972) 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4263b2640 1 -- 172.21.15.37:0/2651435291 learned_addr learned my addr 172.21.15.37:0/2651435291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe425bb1640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4200b5d20 0x7fe4200b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe426bb3640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4200b3e80 0x7fe4200c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4263b2640 1 -- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4200b5d20 msgr2=0x7fe4200b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4263b2640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4200b5d20 0x7fe4200b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4263b2640 1 -- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4200b3e80 msgr2=0x7fe4200c71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4263b2640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4200b3e80 0x7fe4200c71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.614+0000 7fe4263b2640 1 -- 172.21.15.37:0/2651435291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4200b1e90 con 0x7fe40c003a60 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe4263b2640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 0x7fe420157d30 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0b0040 tx=0x7fe40c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe412ffd640 1 -- 172.21.15.37:0/2651435291 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c004020 con 0x7fe40c003a60 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe412ffd640 1 -- 172.21.15.37:0/2651435291 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41c0041c0 con 0x7fe40c003a60 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe428616640 1 -- 172.21.15.37:0/2651435291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4200b10f0 con 0x7fe40c003a60 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe412ffd640 1 -- 172.21.15.37:0/2651435291 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c0043b0 con 0x7fe40c003a60 2024-04-25T09:02:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe428616640 1 -- 172.21.15.37:0/2651435291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4200b1690 con 0x7fe40c003a60 2024-04-25T09:02:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe428616640 1 -- 172.21.15.37:0/2651435291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe42004efa0 con 0x7fe40c003a60 2024-04-25T09:02:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe412ffd640 1 -- 172.21.15.37:0/2651435291 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe41c0c1020 con 0x7fe40c003a60 2024-04-25T09:02:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe412ffd640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe412ffd640 1 -- 172.21.15.37:0/2651435291 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe41c11f1a0 con 0x7fe40c003a60 2024-04-25T09:02:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe425bb1640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.618+0000 7fe425bb1640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe404066f60 0x7fe404069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe414001140 tx=0x7fe414009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.622+0000 7fe412ffd640 1 -- 172.21.15.37:0/2651435291 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe41c016700 con 0x7fe40c003a60 2024-04-25T09:02:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.818+0000 7fe428616640 1 -- 172.21.15.37:0/2651435291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4200b4260 con 0x7fe40c003a60 2024-04-25T09:02:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.818+0000 7fe412ffd640 1 -- 172.21.15.37:0/2651435291 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fe41c0ef9d0 con 0x7fe40c003a60 2024-04-25T09:02:07.808 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:07.808 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:01:37.945874+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8727},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8727":{"gid":8727,"name":"c","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/1266122577","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":1266122577},{"type":"v1","addr":"172.21.15.37:6835","nonce":1266122577}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 81 2024-04-25T09:02:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 -- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe404066f60 msgr2=0x7fe404069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe404066f60 0x7fe404069420 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe414001140 tx=0x7fe414009040 comp rx=0 tx=0).stop 2024-04-25T09:02:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 -- 172.21.15.37:0/2651435291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 msgr2=0x7fe420157d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 0x7fe420157d30 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fe41c0b0040 tx=0x7fe40c005840 comp rx=0 tx=0).stop 2024-04-25T09:02:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 -- 172.21.15.37:0/2651435291 shutdown_connections 2024-04-25T09:02:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe404066f60 0x7fe404069420 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe4200b3e80 0x7fe4200c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe4200b5d20 0x7fe4200b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 --2- 172.21.15.37:0/2651435291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe40c003a60 0x7fe420157d30 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 -- 172.21.15.37:0/2651435291 >> 172.21.15.37:0/2651435291 conn(0x7fe42008b000 msgr2=0x7fe4200b22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 -- 172.21.15.37:0/2651435291 shutdown_connections 2024-04-25T09:02:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:07.822+0000 7fe428616640 1 -- 172.21.15.37:0/2651435291 wait complete. 2024-04-25T09:02:07.828 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T09:02:07.973 INFO:teuthology.orchestra.run.smithi037.stdout:{"message":"","return_code":0} 2024-04-25T09:02:07.973 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T09:02:07.974 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:02:07.974 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:07.980 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:02:07.980 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi037.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-25T09:02:07.980 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:02:07.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:07.981 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:02:08.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:02:08.026 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T09:02:08.027 DEBUG:teuthology.orchestra.run.smithi037:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T09:02:08.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:08.071 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T09:02:08.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.218+0000 7f95c60e1640 1 Processor -- start 2024-04-25T09:02:08.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.218+0000 7f95c60e1640 1 -- start start 2024-04-25T09:02:08.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.218+0000 7f95c60e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c00d21f0 0x7f95c00b0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.218+0000 7f95c60e1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f95c008b4c0 0x7f95c00b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.218+0000 7f95c60e1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95c00ce870 con 0x7f95c0161050 2024-04-25T09:02:08.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.218+0000 7f95c60e1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95c0161430 con 0x7f95c008b4c0 2024-04-25T09:02:08.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.218+0000 7f95c60e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00ceb50 con 0x7f95c00d21f0 2024-04-25T09:02:08.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c00d21f0 0x7f95c00b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95bffff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f95c008b4c0 0x7f95c00b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95bf7fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f95c0161050 0x7f95c00d1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:36198/0 (socket says 172.21.15.37:36198) 2024-04-25T09:02:08.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95bf7fe640 1 -- 172.21.15.37:0/1177422539 learned_addr learned my addr 172.21.15.37:0/1177422539 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:08.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95b7fff640 1 -- 172.21.15.37:0/1177422539 >> v2:172.21.15.203:3300/0 conn(0x7f95c008b4c0 msgr2=0x7f95c00b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95b7fff640 1 --2- 172.21.15.37:0/1177422539 >> v2:172.21.15.203:3300/0 conn(0x7f95c008b4c0 0x7f95c00b07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95b7fff640 1 -- 172.21.15.37:0/1177422539 >> v1:172.21.15.203:6789/0 conn(0x7f95c0161050 legacy=0x7f95c00d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95b7fff640 1 -- 172.21.15.37:0/1177422539 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00cee30 con 0x7f95c00d21f0 2024-04-25T09:02:08.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95b7fff640 1 --2- 172.21.15.37:0/1177422539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c00d21f0 0x7f95c00b0280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f95b00097f0 tx=0x7f95b001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88487471f88a564a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95beffd640 1 -- 172.21.15.37:0/1177422539 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b00b2070 con 0x7f95c00d21f0 2024-04-25T09:02:08.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95beffd640 1 -- 172.21.15.37:0/1177422539 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b00ad640 con 0x7f95c00d21f0 2024-04-25T09:02:08.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.222+0000 7f95beffd640 1 -- 172.21.15.37:0/1177422539 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b00ada10 con 0x7f95c00d21f0 2024-04-25T09:02:08.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 -- 172.21.15.37:0/1177422539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c00d21f0 msgr2=0x7f95c00b0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 --2- 172.21.15.37:0/1177422539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c00d21f0 0x7f95c00b0280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f95b00097f0 tx=0x7f95b001c4a0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 -- 172.21.15.37:0/1177422539 shutdown_connections 2024-04-25T09:02:08.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 --2- 172.21.15.37:0/1177422539 >> v2:172.21.15.203:3300/0 conn(0x7f95c008b4c0 0x7f95c00b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 --2- 172.21.15.37:0/1177422539 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c00d21f0 0x7f95c00b0280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 -- 172.21.15.37:0/1177422539 >> 172.21.15.37:0/1177422539 conn(0x7f95c008b000 msgr2=0x7f95c00d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 -- 172.21.15.37:0/1177422539 shutdown_connections 2024-04-25T09:02:08.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 -- 172.21.15.37:0/1177422539 wait complete. 2024-04-25T09:02:08.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 Processor -- start 2024-04-25T09:02:08.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 -- start start 2024-04-25T09:02:08.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95c008b4c0 0x7f95c0155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00d21f0 0x7f95c015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c0161050 0x7f95c015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00b0e80 con 0x7f95c00d21f0 2024-04-25T09:02:08.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00b0d00 con 0x7f95c008b4c0 2024-04-25T09:02:08.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95c60e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00b1000 con 0x7f95c0161050 2024-04-25T09:02:08.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00d21f0 0x7f95c015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00d21f0 0x7f95c015a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59976/0 (socket says 172.21.15.37:59976) 2024-04-25T09:02:08.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95b7fff640 1 -- 172.21.15.37:0/1399196885 learned_addr learned my addr 172.21.15.37:0/1399196885 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:08.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95bffff640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c0161050 0x7f95c015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95c008b4c0 0x7f95c0155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95b7fff640 1 -- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c0161050 msgr2=0x7f95c015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95b7fff640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c0161050 0x7f95c015a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95b7fff640 1 -- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95c008b4c0 msgr2=0x7f95c0155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95b7fff640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95c008b4c0 0x7f95c0155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95b7fff640 1 -- 172.21.15.37:0/1399196885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c00cd9d0 con 0x7f95c00d21f0 2024-04-25T09:02:08.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.226+0000 7f95b7fff640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00d21f0 0x7f95c015a070 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f95b00b0040 tx=0x7f95b0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95bcff9640 1 -- 172.21.15.37:0/1399196885 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b00ba020 con 0x7f95c00d21f0 2024-04-25T09:02:08.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95bcff9640 1 -- 172.21.15.37:0/1399196885 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b00b6410 con 0x7f95c00d21f0 2024-04-25T09:02:08.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95c60e1640 1 -- 172.21.15.37:0/1399196885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c015abe0 con 0x7f95c00d21f0 2024-04-25T09:02:08.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95bcff9640 1 -- 172.21.15.37:0/1399196885 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b00b2040 con 0x7f95c00d21f0 2024-04-25T09:02:08.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95c60e1640 1 -- 172.21.15.37:0/1399196885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c015b1a0 con 0x7f95c00d21f0 2024-04-25T09:02:08.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95bcff9640 1 -- 172.21.15.37:0/1399196885 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f95b00bd020 con 0x7f95c00d21f0 2024-04-25T09:02:08.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95bcff9640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95a0066f60 0x7f95a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95bcff9640 1 -- 172.21.15.37:0/1399196885 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f95b011efe0 con 0x7f95c00d21f0 2024-04-25T09:02:08.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95a0066f60 0x7f95a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95c924e000 1 Processor -- start 2024-04-25T09:02:08.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95c924e000 1 -- start start 2024-04-25T09:02:08.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95a0066f60 0x7f95a0069420 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f95a8000f70 tx=0x7f95a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95c924e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x5586981e4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95c924e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586981e45b0 0x558698199d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95c924e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55869819a290 0x55869819a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95c924e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558698080f40 con 0x5586981e3c90 2024-04-25T09:02:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95c924e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558697fe8070 con 0x5586981e45b0 2024-04-25T09:02:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.230+0000 7f95c924e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558697f701a0 con 0x55869819a290 2024-04-25T09:02:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x5586981e4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x5586981e4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59986/0 (socket says 172.21.15.37:59986) 2024-04-25T09:02:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 -- 172.21.15.37:0/2913984119 learned_addr learned my addr 172.21.15.37:0/2913984119 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95b7fff640 1 --2- 172.21.15.37:0/2913984119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586981e45b0 0x558698199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bffff640 1 --2- 172.21.15.37:0/2913984119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55869819a290 0x55869819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 -- 172.21.15.37:0/2913984119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55869819a290 msgr2=0x55869819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/2913984119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55869819a290 0x55869819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 -- 172.21.15.37:0/2913984119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586981e45b0 msgr2=0x558698199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/2913984119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586981e45b0 0x558698199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 -- 172.21.15.37:0/2913984119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558697f6e510 con 0x5586981e3c90 2024-04-25T09:02:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/2913984119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x5586981e4070 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005aa0 tx=0x7f95a8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0972ba0c7086fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95b57fa640 1 -- 172.21.15.37:0/2913984119 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8007710 con 0x5586981e3c90 2024-04-25T09:02:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95b57fa640 1 -- 172.21.15.37:0/2913984119 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a800e6c0 con 0x5586981e3c90 2024-04-25T09:02:08.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95b57fa640 1 -- 172.21.15.37:0/2913984119 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a800ea90 con 0x5586981e3c90 2024-04-25T09:02:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 -- 172.21.15.37:0/2913984119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 msgr2=0x5586981e4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 --2- 172.21.15.37:0/2913984119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x5586981e4070 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005aa0 tx=0x7f95a8011000 comp rx=0 tx=0).stop 2024-04-25T09:02:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 -- 172.21.15.37:0/2913984119 shutdown_connections 2024-04-25T09:02:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 --2- 172.21.15.37:0/2913984119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55869819a290 0x55869819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 --2- 172.21.15.37:0/2913984119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586981e45b0 0x558698199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 --2- 172.21.15.37:0/2913984119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x5586981e4070 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 -- 172.21.15.37:0/2913984119 >> 172.21.15.37:0/2913984119 conn(0x558698196240 msgr2=0x558698198630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 -- 172.21.15.37:0/2913984119 shutdown_connections 2024-04-25T09:02:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 -- 172.21.15.37:0/2913984119 wait complete. 2024-04-25T09:02:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 -- start start 2024-04-25T09:02:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f997b17c640 1 Processor -- start 2024-04-25T09:02:08.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 Processor -- start 2024-04-25T09:02:08.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f997b17c640 1 -- start start 2024-04-25T09:02:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55869819a290 0x5586981a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586981e3c90 0x5586981a5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 0x5586981a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558697fa26c0 con 0x55869819a290 2024-04-25T09:02:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x558697fe4110 con 0x5586981e3c90 2024-04-25T09:02:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586981a3200 con 0x5586981e45b0 2024-04-25T09:02:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f997b17c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f99740cd690 0x7f997406ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95c924e000 10 client.0 _ll_register_callbacks cb 0x558698185570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f997b17c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997406ee20 con 0x7f99740af5f0 2024-04-25T09:02:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f997b17c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740d0f40 con 0x7f99740cda70 2024-04-25T09:02:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f997b17c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99740d1220 con 0x7f99740cd690 2024-04-25T09:02:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55869819a290 0x5586981a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55869819a290 0x5586981a4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59988/0 (socket says 172.21.15.37:59988) 2024-04-25T09:02:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 -- 172.21.15.37:0/3285071320 learned_addr learned my addr 172.21.15.37:0/3285071320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f9978f18640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f99740cd690 0x7f997406ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f9979719640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f99740af5f0 0x7f997406e710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46868/0 (socket says 172.21.15.37:46868) 2024-04-25T09:02:08.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f9979719640 1 -- 172.21.15.37:0/3470222788 learned_addr learned my addr 172.21.15.37:0/3470222788 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95b7fff640 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586981e3c90 0x5586981a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bffff640 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 0x5586981a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 -- 172.21.15.37:0/3285071320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 msgr2=0x5586981a58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 0x5586981a58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 -- 172.21.15.37:0/3285071320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586981e3c90 msgr2=0x5586981a5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586981e3c90 0x5586981a5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bf7fe640 1 -- 172.21.15.37:0/3285071320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558697f703b0 con 0x55869819a290 2024-04-25T09:02:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f95bffff640 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 0x5586981a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f9978f18640 1 -- 172.21.15.37:0/3470222788 >> v1:172.21.15.203:6789/0 conn(0x7f99740cda70 legacy=0x7f997406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f9978f18640 1 -- 172.21.15.37:0/3470222788 >> v1:172.21.15.37:6789/0 conn(0x7f99740af5f0 legacy=0x7f997406e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.234+0000 7f9978f18640 1 -- 172.21.15.37:0/3470222788 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997406f5d0 con 0x7f99740cd690 2024-04-25T09:02:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55869819a290 0x5586981a4e30 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f95a8001e70 tx=0x7f95a8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b18204c8d98e3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f959e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f959e7fc640 1 -- 172.21.15.37:0/3285071320 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a800ee30 con 0x55869819a290 2024-04-25T09:02:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f959e7fc640 1 -- 172.21.15.37:0/3285071320 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f95a8010070 con 0x55869819a290 2024-04-25T09:02:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 -- 172.21.15.37:0/3285071320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95a800e6c0 con 0x55869819a290 2024-04-25T09:02:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f959e7fc640 1 -- 172.21.15.37:0/3285071320 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95a8031050 con 0x55869819a290 2024-04-25T09:02:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 10 client.9069 fetch_fsmap learned FSMap version 81 2024-04-25T09:02:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 -- 172.21.15.37:0/3285071320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x558698069e00 con 0x55869819a290 2024-04-25T09:02:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f959e7fc640 1 -- 172.21.15.37:0/3285071320 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a800e6c0 con 0x55869819a290 2024-04-25T09:02:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f959e7fc640 1 -- 172.21.15.37:0/3285071320 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f95a8034020 con 0x55869819a290 2024-04-25T09:02:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 10 client.9069 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T09:02:08.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 10 client.9069 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 20 client.9069 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x5586981a6180 0x5586981f5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 4 client.9069 mds_command: new command op to 8727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 -- 172.21.15.37:0/3285071320 --> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558697f83d60 con 0x5586981a6180 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95b7fff640 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x5586981a6180 0x5586981f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f959e7fc640 10 client.9069 ms_handle_connect on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95b7fff640 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x5586981a6180 0x5586981f5410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c4db0464fbfe74 server_cookie=bb4bf4760212ed4 in_seq=0 out_seq=0 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f959e7fc640 1 -- 172.21.15.37:0/3285071320 <== mds.0 v2:172.21.15.37:6834/1266122577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558697f83d60 con 0x5586981a6180 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f959e7fc640 10 client.9069 handle_command_reply: tid=0 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 client.9069 shutdown 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 -- 172.21.15.37:0/3285071320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55869819a290 msgr2=0x5586981a4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55869819a290 0x5586981a4e30 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f95a8001e70 tx=0x7f95a8000bd0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 -- 172.21.15.37:0/3285071320 shutdown_connections 2024-04-25T09:02:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x5586981a6180 0x5586981f5410 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 0x5586981a58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5586981e3c90 0x5586981a5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 --2- 172.21.15.37:0/3285071320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55869819a290 0x5586981a4e30 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f959e7fc640 0 client.9069 ms_handle_reset on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:08.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 -- 172.21.15.37:0/3285071320 >> 172.21.15.37:0/3285071320 conn(0x558698196240 msgr2=0x558698198ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 -- 172.21.15.37:0/3285071320 shutdown_connections 2024-04-25T09:02:08.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f9978f18640 1 --2- 172.21.15.37:0/3470222788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99740cd690 0x7f997406ac90 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f99680097c0 tx=0x7f996801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62d15c91b3268b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 1 -- 172.21.15.37:0/3285071320 wait complete. 2024-04-25T09:02:08.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f95c924e000 20 client.9069 trim_cache size 0 max 16384 2024-04-25T09:02:08.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f99737fe640 1 -- 172.21.15.37:0/3470222788 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99680b0070 con 0x7f99740cd690 2024-04-25T09:02:08.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f99737fe640 1 -- 172.21.15.37:0/3470222788 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99680ab600 con 0x7f99740cd690 2024-04-25T09:02:08.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f99737fe640 1 -- 172.21.15.37:0/3470222788 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99680a4e60 con 0x7f99740cd690 2024-04-25T09:02:08.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f997b17c640 1 -- 172.21.15.37:0/3470222788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99740cd690 msgr2=0x7f997406ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f997b17c640 1 --2- 172.21.15.37:0/3470222788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99740cd690 0x7f997406ac90 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f99680097c0 tx=0x7f996801c3d0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f997b17c640 1 -- 172.21.15.37:0/3470222788 shutdown_connections 2024-04-25T09:02:08.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f997b17c640 1 --2- 172.21.15.37:0/3470222788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99740cd690 0x7f997406ac90 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f997b17c640 1 -- 172.21.15.37:0/3470222788 >> 172.21.15.37:0/3470222788 conn(0x7f99740d0420 msgr2=0x7f99740d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f997b17c640 1 -- 172.21.15.37:0/3470222788 shutdown_connections 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.238+0000 7f997b17c640 1 -- 172.21.15.37:0/3470222788 wait complete. 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f997b17c640 1 Processor -- start 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f997b17c640 1 -- start start 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f997b17c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99740af5f0 0x7f99741f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f997b17c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99740cd690 0x7f997414f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f997b17c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99740cda70 0x7f99740c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f997b17c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740ccc90 con 0x7f99740cd690 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f997b17c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997406b5e0 con 0x7f99740cda70 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f997b17c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99740ce240 con 0x7f99740af5f0 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9973fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99740cd690 0x7f997414f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9973fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99740cd690 0x7f997414f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60014/0 (socket says 172.21.15.37:60014) 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9973fff640 1 -- 172.21.15.37:0/2656033505 learned_addr learned my addr 172.21.15.37:0/2656033505 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:08.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9978f18640 1 --2- 172.21.15.37:0/2656033505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99740af5f0 0x7f99741f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9979719640 1 --2- 172.21.15.37:0/2656033505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99740cda70 0x7f99740c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9973fff640 1 -- 172.21.15.37:0/2656033505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99740af5f0 msgr2=0x7f99741f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9973fff640 1 --2- 172.21.15.37:0/2656033505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99740af5f0 0x7f99741f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9973fff640 1 -- 172.21.15.37:0/2656033505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99740cda70 msgr2=0x7f99740c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9973fff640 1 --2- 172.21.15.37:0/2656033505 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99740cda70 0x7f99740c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9973fff640 1 -- 172.21.15.37:0/2656033505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740c7110 con 0x7f99740cd690 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9978f18640 1 --2- 172.21.15.37:0/2656033505 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99740af5f0 0x7f99741f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f9973fff640 1 --2- 172.21.15.37:0/2656033505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99740cd690 0x7f997414f220 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f996000c450 tx=0x7f996000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f99717fa640 1 -- 172.21.15.37:0/2656033505 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960012830 con 0x7f99740cd690 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f99717fa640 1 -- 172.21.15.37:0/2656033505 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9960017070 con 0x7f99740cd690 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f997b17c640 1 -- 172.21.15.37:0/2656033505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99740b3ea0 con 0x7f99740cd690 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f99717fa640 1 -- 172.21.15.37:0/2656033505 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960012c30 con 0x7f99740cd690 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f997b17c640 1 -- 172.21.15.37:0/2656033505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99740b4460 con 0x7f99740cd690 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f997b17c640 1 -- 172.21.15.37:0/2656033505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f9974068340 con 0x7f99740cd690 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.242+0000 7f99717fa640 1 -- 172.21.15.37:0/2656033505 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f996001c460 con 0x7f99740cd690 2024-04-25T09:02:08.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.246+0000 7f99717fa640 1 --2- 172.21.15.37:0/2656033505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9958066f60 0x7f9958069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.246+0000 7f99717fa640 1 -- 172.21.15.37:0/2656033505 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f99600859a0 con 0x7f99740cd690 2024-04-25T09:02:08.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.246+0000 7f99717fa640 1 -- 172.21.15.37:0/2656033505 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+512 (secure 0 0 0) 0x7f99600862b0 con 0x7f99740cd690 2024-04-25T09:02:08.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.246+0000 7f9978f18640 1 --2- 172.21.15.37:0/2656033505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9958066f60 0x7f9958069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.246+0000 7f997b17c640 1 -- 172.21.15.37:0/2656033505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f99740cc280 con 0x7f99740cd690 2024-04-25T09:02:08.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.246+0000 7f9978f18640 1 --2- 172.21.15.37:0/2656033505 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9958066f60 0x7f9958069420 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f99680ad000 tx=0x7f9968008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95c924e000 1 Processor -- start 2024-04-25T09:02:08.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95c924e000 1 -- start start 2024-04-25T09:02:08.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95c924e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55869819a290 0x558698199640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95c924e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e3c90 0x558698199b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95c924e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e45b0 0x5586981f5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95c924e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0203840 con 0x5586981e45b0 2024-04-25T09:02:08.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95c924e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c0202a20 con 0x55869819a290 2024-04-25T09:02:08.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95c924e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00cb710 con 0x5586981e3c90 2024-04-25T09:02:08.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e45b0 0x5586981f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e45b0 0x5586981f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60018/0 (socket says 172.21.15.37:60018) 2024-04-25T09:02:08.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95bffff640 1 -- 172.21.15.37:0/858149081 learned_addr learned my addr 172.21.15.37:0/858149081 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95b7fff640 1 --2- 172.21.15.37:0/858149081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e3c90 0x558698199b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95bffff640 1 -- 172.21.15.37:0/858149081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e3c90 msgr2=0x558698199b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/858149081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55869819a290 0x558698199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95bffff640 1 --2- 172.21.15.37:0/858149081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e3c90 0x558698199b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95bffff640 1 -- 172.21.15.37:0/858149081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55869819a290 msgr2=0x558698199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95bffff640 1 --2- 172.21.15.37:0/858149081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55869819a290 0x558698199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95bffff640 1 -- 172.21.15.37:0/858149081 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55869803fc50 con 0x5586981e45b0 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.254+0000 7f95b7fff640 1 --2- 172.21.15.37:0/858149081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e3c90 0x558698199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95bffff640 1 --2- 172.21.15.37:0/858149081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e45b0 0x5586981f5410 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f95b8007680 tx=0x7f95b8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3986664fe735416e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b4ff9640 1 -- 172.21.15.37:0/858149081 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b801d070 con 0x5586981e45b0 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b4ff9640 1 -- 172.21.15.37:0/858149081 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b80148d0 con 0x5586981e45b0 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b4ff9640 1 -- 172.21.15.37:0/858149081 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b8014cc0 con 0x5586981e45b0 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 -- 172.21.15.37:0/858149081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e45b0 msgr2=0x5586981f5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 --2- 172.21.15.37:0/858149081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e45b0 0x5586981f5410 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f95b8007680 tx=0x7f95b8009750 comp rx=0 tx=0).stop 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 -- 172.21.15.37:0/858149081 shutdown_connections 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 --2- 172.21.15.37:0/858149081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e45b0 0x5586981f5410 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 --2- 172.21.15.37:0/858149081 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e3c90 0x558698199b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 --2- 172.21.15.37:0/858149081 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55869819a290 0x558698199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 -- 172.21.15.37:0/858149081 >> 172.21.15.37:0/858149081 conn(0x558698196240 msgr2=0x55869819ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 -- 172.21.15.37:0/858149081 shutdown_connections 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 -- 172.21.15.37:0/858149081 wait complete. 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 -- start start 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 Processor -- start 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55869819a290 0x5586982002d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x558698200810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 0x558698200d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55869809ef20 con 0x5586981e3c90 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95c00cbd10 con 0x55869819a290 2024-04-25T09:02:08.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558697f8abd0 con 0x5586981e45b0 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 10 client.0 _ll_register_callbacks cb 0x7f95a8001330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x558698200810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x558698200810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60034/0 (socket says 172.21.15.37:60034) 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b7fff640 1 -- 172.21.15.37:0/85746376 learned_addr learned my addr 172.21.15.37:0/85746376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b7fff640 1 -- 172.21.15.37:0/85746376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 msgr2=0x558698200d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95bffff640 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 0x558698200d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55869819a290 0x5586982002d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b7fff640 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 0x558698200d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b7fff640 1 -- 172.21.15.37:0/85746376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55869819a290 msgr2=0x5586982002d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b7fff640 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55869819a290 0x5586982002d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b7fff640 1 -- 172.21.15.37:0/85746376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558697fb0120 con 0x5586981e3c90 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95bffff640 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 0x558698200d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95b7fff640 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x558698200810 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f95b00f7930 tx=0x7f95b00a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17182539080746d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f959e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:08.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f959e7fc640 1 -- 172.21.15.37:0/85746376 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b00f38a0 con 0x5586981e3c90 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f959e7fc640 1 -- 172.21.15.37:0/85746376 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f95b00eda50 con 0x5586981e3c90 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 -- 172.21.15.37:0/85746376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95b80148d0 con 0x5586981e3c90 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f959e7fc640 1 -- 172.21.15.37:0/85746376 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95b00f7e60 con 0x5586981e3c90 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 10 client.9081 fetch_fsmap learned FSMap version 81 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 -- 172.21.15.37:0/85746376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x558697f8f170 con 0x5586981e3c90 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f959e7fc640 1 -- 172.21.15.37:0/85746376 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b80148d0 con 0x5586981e3c90 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f959e7fc640 1 -- 172.21.15.37:0/85746376 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f95b00b5050 con 0x5586981e3c90 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 10 client.9081 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 10 client.9081 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 20 client.9081 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.258+0000 7f95c924e000 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x558698249e70 0x55869824c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 4 client.9081 mds_command: new command op to 8727 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 -- 172.21.15.37:0/85746376 --> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x558697ff55b0 con 0x558698249e70 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x558698249e70 0x55869824c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95bf7fe640 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x558698249e70 0x55869824c260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5e4cdc869748159 server_cookie=a20adfb1679eef91 in_seq=0 out_seq=0 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f959e7fc640 10 client.9081 ms_handle_connect on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f959e7fc640 1 -- 172.21.15.37:0/85746376 <== mds.0 v2:172.21.15.37:6834/1266122577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558697ff55b0 con 0x558698249e70 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f959e7fc640 10 client.9081 handle_command_reply: tid=0 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 client.9081 shutdown 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 -- 172.21.15.37:0/85746376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 msgr2=0x558698200810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x558698200810 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f95b00f7930 tx=0x7f95b00a7000 comp rx=0 tx=0).stop 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 -- 172.21.15.37:0/85746376 shutdown_connections 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x558698249e70 0x55869824c260 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5586981e45b0 0x558698200d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5586981e3c90 0x558698200810 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 --2- 172.21.15.37:0/85746376 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55869819a290 0x5586982002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f959e7fc640 0 client.9081 ms_handle_reset on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 -- 172.21.15.37:0/85746376 >> 172.21.15.37:0/85746376 conn(0x558698196240 msgr2=0x558698198630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 -- 172.21.15.37:0/85746376 shutdown_connections 2024-04-25T09:02:08.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 1 -- 172.21.15.37:0/85746376 wait complete. 2024-04-25T09:02:08.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c924e000 20 client.9081 trim_cache size 0 max 16384 2024-04-25T09:02:08.262 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:02:08.262 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-04-25T09:02:08.262 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "64cffc93-1d47-4fa1-bad4-afafa8943510", 2024-04-25T09:02:08.262 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-04-25T09:02:08.262 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 -- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95a0066f60 msgr2=0x7f95a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95a0066f60 0x7f95a0069420 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f95a8000f70 tx=0x7f95a8009040 comp rx=0 tx=0).stop 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 -- 172.21.15.37:0/1399196885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00d21f0 msgr2=0x7f95c015a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00d21f0 0x7f95c015a070 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f95b00b0040 tx=0x7f95b0019240 comp rx=0 tx=0).stop 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 -- 172.21.15.37:0/1399196885 shutdown_connections 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f95a0066f60 0x7f95a0069420 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f95c0161050 0x7f95c015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95c00d21f0 0x7f95c015a070 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 --2- 172.21.15.37:0/1399196885 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f95c008b4c0 0x7f95c0155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 -- 172.21.15.37:0/1399196885 >> 172.21.15.37:0/1399196885 conn(0x7f95c008b000 msgr2=0x7f95c006ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 -- 172.21.15.37:0/1399196885 shutdown_connections 2024-04-25T09:02:08.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.262+0000 7f95c60e1640 1 -- 172.21.15.37:0/1399196885 wait complete. 2024-04-25T09:02:08.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:08.267 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T09:02:08.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.390+0000 7fbc35701640 1 Processor -- start 2024-04-25T09:02:08.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.390+0000 7fbc35701640 1 -- start start 2024-04-25T09:02:08.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.390+0000 7fbc35701640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbc300b5270 0x7fbc300b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc35701640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc3006d090 0x7fbc3006d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc35701640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3006bff0 con 0x7fbc300b5650 2024-04-25T09:02:08.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc35701640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc3006cc00 con 0x7fbc300b5270 2024-04-25T09:02:08.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc35701640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3006c2d0 con 0x7fbc3006d090 2024-04-25T09:02:08.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2e7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbc300b5650 0x7fbc300d3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46872/0 (socket says 172.21.15.37:46872) 2024-04-25T09:02:08.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2e7fc640 1 -- 172.21.15.37:0/849307033 learned_addr learned my addr 172.21.15.37:0/849307033 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:08.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc3006d090 0x7fbc3006d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2effd640 1 --2- 172.21.15.37:0/849307033 >> v2:172.21.15.203:3300/0 conn(0x7fbc300b5270 0x7fbc300b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369882030 0 0) 0x7fbc3006bff0 con 0x7fbc300b5650 2024-04-25T09:02:08.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc18003530 con 0x7fbc300b5650 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1822796345 0 0) 0x7fbc18003530 con 0x7fbc300b5650 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc3006d090 msgr2=0x7fbc3006d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc3006d090 0x7fbc3006d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 >> v2:172.21.15.203:3300/0 conn(0x7fbc300b5270 msgr2=0x7fbc300b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 --2- 172.21.15.37:0/849307033 >> v2:172.21.15.203:3300/0 conn(0x7fbc300b5270 0x7fbc300b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3006c5b0 con 0x7fbc300b5650 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fbc24001800 con 0x7fbc300b5650 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 >> v1:172.21.15.37:6789/0 conn(0x7fbc300b5650 legacy=0x7fbc300d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc18003a30 0x7fbc18003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc3006d090 0x7fbc18004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc18004d50 0x7fbc180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3006cc00 con 0x7fbc18003a30 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3006c2d0 con 0x7fbc18004d50 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1800b9d0 con 0x7fbc3006d090 2024-04-25T09:02:08.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2effd640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc18003a30 0x7fbc18003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2effd640 1 -- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc3006d090 msgr2=0x7fbc18004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc3006d090 0x7fbc18004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc18004d50 0x7fbc180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2effd640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc3006d090 0x7fbc18004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2effd640 1 -- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc18004d50 msgr2=0x7fbc180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2effd640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc18004d50 0x7fbc180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2effd640 1 -- 172.21.15.37:0/849307033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc18003530 con 0x7fbc18003a30 2024-04-25T09:02:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc3006d090 0x7fbc18004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:08.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc18004d50 0x7fbc180051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:08.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2effd640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc18003a30 0x7fbc18003e50 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000e170 tx=0x7fbc2001d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f754ef5df62a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc20009c60 con 0x7fbc18003a30 2024-04-25T09:02:08.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc200a5d10 con 0x7fbc18003a30 2024-04-25T09:02:08.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.394+0000 7fbc2dffb640 1 -- 172.21.15.37:0/849307033 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc200ad630 con 0x7fbc18003a30 2024-04-25T09:02:08.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 -- 172.21.15.37:0/849307033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc18003a30 msgr2=0x7fbc18003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 --2- 172.21.15.37:0/849307033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc18003a30 0x7fbc18003e50 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000e170 tx=0x7fbc2001d760 comp rx=0 tx=0).stop 2024-04-25T09:02:08.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2f7fe640 1 -- 172.21.15.37:0/849307033 reap_dead start 2024-04-25T09:02:08.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 -- 172.21.15.37:0/849307033 shutdown_connections 2024-04-25T09:02:08.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 -- 172.21.15.37:0/849307033 >> 172.21.15.37:0/849307033 conn(0x7fbc3008b000 msgr2=0x7fbc300d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 -- 172.21.15.37:0/849307033 shutdown_connections 2024-04-25T09:02:08.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 -- 172.21.15.37:0/849307033 wait complete. 2024-04-25T09:02:08.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 Processor -- start 2024-04-25T09:02:08.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 -- start start 2024-04-25T09:02:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc18003a30 0x7fbc300b6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc30148a10 0x7fbc300b7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc300b7fd0 0x7fbc30156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300b5be0 con 0x7fbc30148a10 2024-04-25T09:02:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc300b5a60 con 0x7fbc300b7fd0 2024-04-25T09:02:08.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3015de20 con 0x7fbc18003a30 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc30148a10 0x7fbc300b7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2effd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc18003a30 0x7fbc300b6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc30148a10 0x7fbc300b7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60046/0 (socket says 172.21.15.37:60046) 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2e7fc640 1 -- 172.21.15.37:0/3868180849 learned_addr learned my addr 172.21.15.37:0/3868180849 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc300b7fd0 0x7fbc30156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2e7fc640 1 -- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc18003a30 msgr2=0x7fbc300b6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc18003a30 0x7fbc300b6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2e7fc640 1 -- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc300b7fd0 msgr2=0x7fbc30156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc300b7fd0 0x7fbc30156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2e7fc640 1 -- 172.21.15.37:0/3868180849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc300cd310 con 0x7fbc30148a10 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc300b7fd0 0x7fbc30156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc30148a10 0x7fbc300b7a90 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fbc24007fd0 tx=0x7fbc2400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc177fe640 1 -- 172.21.15.37:0/3868180849 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24007480 con 0x7fbc30148a10 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc177fe640 1 -- 172.21.15.37:0/3868180849 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc24017070 con 0x7fbc30148a10 2024-04-25T09:02:08.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc177fe640 1 -- 172.21.15.37:0/3868180849 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240059b0 con 0x7fbc30148a10 2024-04-25T09:02:08.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 -- 172.21.15.37:0/3868180849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc300b8840 con 0x7fbc30148a10 2024-04-25T09:02:08.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.398+0000 7fbc35701640 1 -- 172.21.15.37:0/3868180849 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc30148e90 con 0x7fbc30148a10 2024-04-25T09:02:08.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc177fe640 1 -- 172.21.15.37:0/3868180849 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbc24005b50 con 0x7fbc30148a10 2024-04-25T09:02:08.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc177fe640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc1c066f60 0x7fbc1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc177fe640 1 -- 172.21.15.37:0/3868180849 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc240851e0 con 0x7fbc30148a10 2024-04-25T09:02:08.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc2effd640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc1c066f60 0x7fbc1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc3886e000 1 Processor -- start 2024-04-25T09:02:08.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc3886e000 1 -- start start 2024-04-25T09:02:08.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x55829678fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296745b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc2effd640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc1c066f60 0x7fbc1c069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbc200ae000 tx=0x7fbc2001cd00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967460a0 0x558296746480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc3886e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558296592ff0 con 0x5582967902f0 2024-04-25T09:02:08.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc3886e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829651b150 con 0x55829678f9d0 2024-04-25T09:02:08.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc3886e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55829658f090 con 0x5582967460a0 2024-04-25T09:02:08.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc2e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296745b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc2e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296745b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60052/0 (socket says 172.21.15.37:60052) 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc2e7fc640 1 -- 172.21.15.37:0/2689606205 learned_addr learned my addr 172.21.15.37:0/2689606205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc2effd640 1 --2- 172.21.15.37:0/2689606205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x55829678fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/2689606205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967460a0 0x558296746480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc2e7fc640 1 -- 172.21.15.37:0/2689606205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967460a0 msgr2=0x558296746480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/2689606205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967460a0 0x558296746480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc2e7fc640 1 -- 172.21.15.37:0/2689606205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 msgr2=0x55829678fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.402+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/2689606205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x55829678fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2e7fc640 1 -- 172.21.15.37:0/2689606205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582967290d0 con 0x5582967902f0 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/2689606205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296745b60 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fbc2405da20 tx=0x7fbc24005f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c054d0cb01897b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc14ff9640 1 -- 172.21.15.37:0/2689606205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24059990 con 0x5582967902f0 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc14ff9640 1 -- 172.21.15.37:0/2689606205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2405df50 con 0x5582967902f0 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc14ff9640 1 -- 172.21.15.37:0/2689606205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24051eb0 con 0x5582967902f0 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 -- 172.21.15.37:0/2689606205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 msgr2=0x558296745b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 --2- 172.21.15.37:0/2689606205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296745b60 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fbc2405da20 tx=0x7fbc24005f80 comp rx=0 tx=0).stop 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 -- 172.21.15.37:0/2689606205 shutdown_connections 2024-04-25T09:02:08.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 --2- 172.21.15.37:0/2689606205 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967460a0 0x558296746480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 --2- 172.21.15.37:0/2689606205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296745b60 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 --2- 172.21.15.37:0/2689606205 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x55829678fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 -- 172.21.15.37:0/2689606205 >> 172.21.15.37:0/2689606205 conn(0x558296742020 msgr2=0x558296744440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 -- 172.21.15.37:0/2689606205 shutdown_connections 2024-04-25T09:02:08.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 -- 172.21.15.37:0/2689606205 wait complete. 2024-04-25T09:02:08.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 -- start start 2024-04-25T09:02:08.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 Processor -- start 2024-04-25T09:02:08.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967460a0 0x5582967747f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x558296774d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296775270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x558296557250 con 0x5582967902f0 2024-04-25T09:02:08.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829654d690 con 0x55829678f9d0 2024-04-25T09:02:08.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55829651ba40 con 0x5582967460a0 2024-04-25T09:02:08.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 10 client.0 _ll_register_callbacks cb 0x558296730e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296775270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296775270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60060/0 (socket says 172.21.15.37:60060) 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2f7fe640 1 -- 172.21.15.37:0/2374552472 learned_addr learned my addr 172.21.15.37:0/2374552472 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x558296774d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2effd640 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967460a0 0x5582967747f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2f7fe640 1 -- 172.21.15.37:0/2374552472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967460a0 msgr2=0x5582967747f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967460a0 0x5582967747f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2f7fe640 1 -- 172.21.15.37:0/2374552472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 msgr2=0x558296774d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x558296774d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2f7fe640 1 -- 172.21.15.37:0/2374552472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582966401c0 con 0x5582967902f0 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2effd640 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967460a0 0x5582967747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296775270 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fbc300cdcb0 tx=0x7fbc280146c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4541e069bae228a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc0dffb640 1 -- 172.21.15.37:0/2374552472 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2801b030 con 0x5582967902f0 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc0dffb640 1 -- 172.21.15.37:0/2374552472 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc28004020 con 0x5582967902f0 2024-04-25T09:02:08.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 -- 172.21.15.37:0/2374552472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc2405df50 con 0x5582967902f0 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc0dffb640 1 -- 172.21.15.37:0/2374552472 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc2800f040 con 0x5582967902f0 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 10 client.9093 fetch_fsmap learned FSMap version 81 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc3886e000 1 -- 172.21.15.37:0/2374552472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55829663c170 con 0x5582967902f0 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.406+0000 7fbc0dffb640 1 -- 172.21.15.37:0/2374552472 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2405df50 con 0x5582967902f0 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc0dffb640 1 -- 172.21.15.37:0/2374552472 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbc28029080 con 0x5582967902f0 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 10 client.9093 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 10 client.9093 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 20 client.9093 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55829679fc80 0x5582967a2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 4 client.9093 mds_command: new command op to 8727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 -- 172.21.15.37:0/2374552472 --> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55829652ec60 con 0x55829679fc80 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc2effd640 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55829679fc80 0x5582967a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc2effd640 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55829679fc80 0x5582967a2070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f78b7673ebf4c0 server_cookie=ee58ea8b65bcf572 in_seq=0 out_seq=0 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc0dffb640 10 client.9093 ms_handle_connect on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc0dffb640 1 -- 172.21.15.37:0/2374552472 <== mds.0 v2:172.21.15.37:6834/1266122577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55829652ec60 con 0x55829679fc80 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc0dffb640 10 client.9093 handle_command_reply: tid=0 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 client.9093 shutdown 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 -- 172.21.15.37:0/2374552472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 msgr2=0x558296775270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296775270 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fbc300cdcb0 tx=0x7fbc280146c0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 -- 172.21.15.37:0/2374552472 shutdown_connections 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967902f0 0x558296775270 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x558296774d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55829679fc80 0x5582967a2070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 --2- 172.21.15.37:0/2374552472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967460a0 0x5582967747f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc0dffb640 0 client.9093 ms_handle_reset on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 -- 172.21.15.37:0/2374552472 >> 172.21.15.37:0/2374552472 conn(0x558296742020 msgr2=0x558296744ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 -- 172.21.15.37:0/2374552472 shutdown_connections 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 1 -- 172.21.15.37:0/2374552472 wait complete. 2024-04-25T09:02:08.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.410+0000 7fbc3886e000 20 client.9093 trim_cache size 0 max 16384 2024-04-25T09:02:08.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 Processor -- start 2024-04-25T09:02:08.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 -- start start 2024-04-25T09:02:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x55829678edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x558296745420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967902f0 0x558296745960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc30203950 con 0x5582967460a0 2024-04-25T09:02:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc30202b10 con 0x55829678f9d0 2024-04-25T09:02:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc30202060 con 0x5582967902f0 2024-04-25T09:02:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x55829678edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x55829678edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60062/0 (socket says 172.21.15.37:60062) 2024-04-25T09:02:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2effd640 1 -- 172.21.15.37:0/1301737472 learned_addr learned my addr 172.21.15.37:0/1301737472 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:08.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/1301737472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x558296745420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/1301737472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967902f0 0x558296745960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2effd640 1 -- 172.21.15.37:0/1301737472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967902f0 msgr2=0x558296745960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2effd640 1 --2- 172.21.15.37:0/1301737472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967902f0 0x558296745960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2effd640 1 -- 172.21.15.37:0/1301737472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 msgr2=0x558296745420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2effd640 1 --2- 172.21.15.37:0/1301737472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x558296745420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2effd640 1 -- 172.21.15.37:0/1301737472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582965d9de0 con 0x5582967460a0 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/1301737472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967902f0 0x558296745960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/1301737472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x558296745420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc2effd640 1 --2- 172.21.15.37:0/1301737472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x55829678edb0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000e170 tx=0x7fbc2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f42d7b15dd7cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc0ffff640 1 -- 172.21.15.37:0/1301737472 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc200cb230 con 0x5582967460a0 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc0ffff640 1 -- 172.21.15.37:0/1301737472 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc200cb3d0 con 0x5582967460a0 2024-04-25T09:02:08.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc0ffff640 1 -- 172.21.15.37:0/1301737472 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc200cb580 con 0x5582967460a0 2024-04-25T09:02:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 -- 172.21.15.37:0/1301737472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 msgr2=0x55829678edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 --2- 172.21.15.37:0/1301737472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x55829678edb0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fbc2000e170 tx=0x7fbc2000b040 comp rx=0 tx=0).stop 2024-04-25T09:02:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 -- 172.21.15.37:0/1301737472 shutdown_connections 2024-04-25T09:02:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 --2- 172.21.15.37:0/1301737472 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967902f0 0x558296745960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 --2- 172.21.15.37:0/1301737472 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x558296745420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 --2- 172.21.15.37:0/1301737472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x55829678edb0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 -- 172.21.15.37:0/1301737472 >> 172.21.15.37:0/1301737472 conn(0x558296742020 msgr2=0x55829678e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 -- 172.21.15.37:0/1301737472 shutdown_connections 2024-04-25T09:02:08.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.426+0000 7fbc3886e000 1 -- 172.21.15.37:0/1301737472 wait complete. 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- start start 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 Processor -- start 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x5582967f0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x5582967f0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967902f0 0x5582967a2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55829652de00 con 0x5582967460a0 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc30202510 con 0x55829678f9d0 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x558296592300 con 0x5582967902f0 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 10 client.0 _ll_register_callbacks cb 0x558296711040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x5582967f0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x5582967f0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60066/0 (socket says 172.21.15.37:60066) 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2effd640 1 -- 172.21.15.37:0/881061250 learned_addr learned my addr 172.21.15.37:0/881061250 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2f7fe640 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967902f0 0x5582967a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x5582967f0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2effd640 1 -- 172.21.15.37:0/881061250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967902f0 msgr2=0x5582967a2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2effd640 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967902f0 0x5582967a2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2effd640 1 -- 172.21.15.37:0/881061250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 msgr2=0x5582967f0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2effd640 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x5582967f0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2effd640 1 -- 172.21.15.37:0/881061250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582965348e0 con 0x5582967460a0 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x5582967f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2effd640 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x5582967f0520 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fbc2001d1e0 tx=0x7fbc18005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6585c33acb930eca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc0dffb640 1 -- 172.21.15.37:0/881061250 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc200cb780 con 0x5582967460a0 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc0dffb640 1 -- 172.21.15.37:0/881061250 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc200b8500 con 0x5582967460a0 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- 172.21.15.37:0/881061250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc200cb3d0 con 0x5582967460a0 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc0dffb640 1 -- 172.21.15.37:0/881061250 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc200e1050 con 0x5582967460a0 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 10 client.9099 fetch_fsmap learned FSMap version 81 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- 172.21.15.37:0/881061250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x5582965ec8f0 con 0x5582967460a0 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc0dffb640 1 -- 172.21.15.37:0/881061250 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc200cb3d0 con 0x5582967460a0 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc0dffb640 1 -- 172.21.15.37:0/881061250 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbc200e4020 con 0x5582967460a0 2024-04-25T09:02:08.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 10 client.9099 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 10 client.9099 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 20 client.9099 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x5582967a3280 0x5582967a3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 4 client.9099 mds_command: new command op to 8727 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- 172.21.15.37:0/881061250 --> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5582965a0530 con 0x5582967a3280 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x5582967a3280 0x5582967a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc2e7fc640 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x5582967a3280 0x5582967a3660 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d9261c8945b9618 server_cookie=8b7f5d1a7f78da7 in_seq=0 out_seq=0 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc0dffb640 10 client.9099 ms_handle_connect on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc0dffb640 1 -- 172.21.15.37:0/881061250 <== mds.0 v2:172.21.15.37:6834/1266122577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5582965a0530 con 0x5582967a3280 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc0dffb640 10 client.9099 handle_command_reply: tid=0 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 client.9099 shutdown 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- 172.21.15.37:0/881061250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 msgr2=0x5582967f0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x5582967f0520 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7fbc2001d1e0 tx=0x7fbc18005840 comp rx=0 tx=0).stop 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- 172.21.15.37:0/881061250 shutdown_connections 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x5582967a3280 0x5582967a3660 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5582967902f0 0x5582967a2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55829678f9d0 0x5582967f0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc0dffb640 0 client.9099 ms_handle_reset on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:08.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 --2- 172.21.15.37:0/881061250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5582967460a0 0x5582967f0520 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- 172.21.15.37:0/881061250 >> 172.21.15.37:0/881061250 conn(0x558296742020 msgr2=0x55829678e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- 172.21.15.37:0/881061250 shutdown_connections 2024-04-25T09:02:08.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 1 -- 172.21.15.37:0/881061250 wait complete. 2024-04-25T09:02:08.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.430+0000 7fbc3886e000 20 client.9099 trim_cache size 0 max 16384 2024-04-25T09:02:08.426 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:02:08.426 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:02:08.427 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-04-25T09:02:08.427 INFO:teuthology.orchestra.run.smithi037.stdout: "64cffc93-1d47-4fa1-bad4-afafa8943510": { 2024-04-25T09:02:08.427 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-04-25T09:02:08.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "64cffc93-1d47-4fa1-bad4-afafa8943510", 2024-04-25T09:02:08.427 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,repair" 2024-04-25T09:02:08.427 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:02:08.427 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:02:08.427 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 -- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc1c066f60 msgr2=0x7fbc1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc1c066f60 0x7fbc1c069420 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbc200ae000 tx=0x7fbc2001cd00 comp rx=0 tx=0).stop 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 -- 172.21.15.37:0/3868180849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc30148a10 msgr2=0x7fbc300b7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc30148a10 0x7fbc300b7a90 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fbc24007fd0 tx=0x7fbc2400dc00 comp rx=0 tx=0).stop 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 -- 172.21.15.37:0/3868180849 shutdown_connections 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbc1c066f60 0x7fbc1c069420 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbc300b7fd0 0x7fbc30156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc30148a10 0x7fbc300b7a90 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 --2- 172.21.15.37:0/3868180849 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbc18003a30 0x7fbc300b6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 -- 172.21.15.37:0/3868180849 >> 172.21.15.37:0/3868180849 conn(0x7fbc3008b000 msgr2=0x7fbc30159430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 -- 172.21.15.37:0/3868180849 shutdown_connections 2024-04-25T09:02:08.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:08.434+0000 7fbc35701640 1 -- 172.21.15.37:0/3868180849 wait complete. 2024-04-25T09:02:08.438 INFO:tasks.cephfs.filesystem:scrub status for tag:64cffc93-1d47-4fa1-bad4-afafa8943510 - {'path': '/', 'tag': '64cffc93-1d47-4fa1-bad4-afafa8943510', 'options': 'recursive,repair'} 2024-04-25T09:02:09.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:09.434+0000 7f99717fa640 1 -- 172.21.15.37:0/2656033505 <== mon.0 v2:172.21.15.37:3300/0 7 ==== log(4 entries from seq 1 at 2024-04-25T09:02:08.263956+0000) v1 ==== 695+0+0 (secure 0 0 0) 0x7f9960083e50 con 0x7f99740cd690 2024-04-25T09:02:09.423 INFO:teuthology.orchestra.run.smithi037.stdout: cluster: 2024-04-25T09:02:09.423 INFO:teuthology.orchestra.run.smithi037.stdout: id: 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T09:02:09.423 INFO:teuthology.orchestra.run.smithi037.stdout: health: HEALTH_OK 2024-04-25T09:02:09.423 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:09.423 INFO:teuthology.orchestra.run.smithi037.stdout: services: 2024-04-25T09:02:09.423 INFO:teuthology.orchestra.run.smithi037.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-25T09:02:09.423 INFO:teuthology.orchestra.run.smithi037.stdout: mgr: x(active, since 9m), standbys: y 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: osd: 8 osds: 8 up (since 9m), 8 in (since 9m) 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: data: 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: volumes: 1/1 healthy 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: pools: 3 pools, 97 pgs 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: objects: 30 objects, 19 MiB 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: pgs: 97 active+clean 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: io: 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: client: 85 B/s rd, 1.0 MiB/s wr, 0 op/s rd, 2 op/s wr 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:08.263956+0000 mds.c [INF] scrub queued for path: / 2024-04-25T09:02:09.424 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:08.263960+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T09:02:09.425 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:08.263967+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-25T09:02:09.425 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:08.265657+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T09:02:11.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:11.442+0000 7f99717fa640 1 -- 172.21.15.37:0/2656033505 <== mon.0 v2:172.21.15.37:3300/0 8 ==== log(1 entries from seq 5 at 2024-04-25T09:02:10.559880+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f9960086000 con 0x7f99740cd690 2024-04-25T09:02:11.433 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:10.559880+0000 mds.c [INF] scrub summary: idle 2024-04-25T09:02:12.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:12.446+0000 7f99717fa640 1 -- 172.21.15.37:0/2656033505 <== mon.0 v2:172.21.15.37:3300/0 9 ==== log(4 entries from seq 1118 at 2024-04-25T09:02:12.411327+0000) v1 ==== 1138+0+0 (secure 0 0 0) 0x7f99600541c0 con 0x7f99740cd690 2024-04-25T09:02:13.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:13.470+0000 7f99717fa640 1 -- 172.21.15.37:0/2656033505 <== mon.0 v2:172.21.15.37:3300/0 10 ==== log(2 entries from seq 1120 at 2024-04-25T09:02:12.454996+0000) v1 ==== 566+0+0 (secure 0 0 0) 0x7f9960054420 con 0x7f99740cd690 2024-04-25T09:02:26.633 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:26.636 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:38.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:38.439 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T09:02:38.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.594+0000 7f3746134640 1 Processor -- start 2024-04-25T09:02:38.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.594+0000 7f3746134640 1 -- start start 2024-04-25T09:02:38.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f3746134640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f374015b800 0x7f37400cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f3746134640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37400cbd20 con 0x7f37400cfad0 2024-04-25T09:02:38.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f3746134640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37400cc000 con 0x7f374015b420 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f3746134640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37400cff40 con 0x7f374015b800 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f37400cfad0 0x7f3740072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49138/0 (socket says 172.21.15.37:49138) 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373ffff640 1 -- 172.21.15.37:0/3637982383 learned_addr learned my addr 172.21.15.37:0/3637982383 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373effd640 1 --2- 172.21.15.37:0/3637982383 >> v2:172.21.15.203:3300/0 conn(0x7f374015b800 0x7f37400cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233271341 0 0) 0x7f37400cbd20 con 0x7f37400cfad0 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3728003530 con 0x7f37400cfad0 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 952237756 0 0) 0x7f3728003530 con 0x7f37400cfad0 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 >> v2:172.21.15.203:3300/0 conn(0x7f374015b800 msgr2=0x7f37400cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 --2- 172.21.15.37:0/3637982383 >> v2:172.21.15.203:3300/0 conn(0x7f374015b800 0x7f37400cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 >> v1:172.21.15.203:6789/0 conn(0x7f374015b420 legacy=0x7f37400ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37400cc2e0 con 0x7f37400cfad0 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3738001a00 con 0x7f37400cfad0 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 >> v1:172.21.15.37:6789/0 conn(0x7f37400cfad0 legacy=0x7f3740072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 --2- 172.21.15.37:0/3637982383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3728003a30 0x7f3728003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 --2- 172.21.15.37:0/3637982383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3728004810 0x7f3728004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 --2- 172.21.15.37:0/3637982383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37280051d0 0x7f372800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37400cff40 con 0x7f37280051d0 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372800c100 con 0x7f3728003a30 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372800c300 con 0x7f3728004810 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373ffff640 1 --2- 172.21.15.37:0/3637982383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37280051d0 0x7f372800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373effd640 1 --2- 172.21.15.37:0/3637982383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3728004810 0x7f3728004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373ffff640 1 -- 172.21.15.37:0/3637982383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3728004810 msgr2=0x7f3728004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373ffff640 1 --2- 172.21.15.37:0/3637982383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3728004810 0x7f3728004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373ffff640 1 -- 172.21.15.37:0/3637982383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3728003a30 msgr2=0x7f3728003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373f7fe640 1 --2- 172.21.15.37:0/3637982383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3728003a30 0x7f3728003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373ffff640 1 --2- 172.21.15.37:0/3637982383 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3728003a30 0x7f3728003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373ffff640 1 -- 172.21.15.37:0/3637982383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3728003530 con 0x7f37280051d0 2024-04-25T09:02:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373ffff640 1 -- 172.21.15.37:0/3637982383 reap_dead start 2024-04-25T09:02:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373ffff640 1 --2- 172.21.15.37:0/3637982383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37280051d0 0x7f372800b9d0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f373800a840 tx=0x7f373801d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d1c78385bc519f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373800a460 con 0x7f37280051d0 2024-04-25T09:02:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37380ac640 con 0x7f37280051d0 2024-04-25T09:02:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.598+0000 7f373e7fc640 1 -- 172.21.15.37:0/3637982383 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37380aca30 con 0x7f37280051d0 2024-04-25T09:02:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 -- 172.21.15.37:0/3637982383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37280051d0 msgr2=0x7f372800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 --2- 172.21.15.37:0/3637982383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37280051d0 0x7f372800b9d0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f373800a840 tx=0x7f373801d360 comp rx=0 tx=0).stop 2024-04-25T09:02:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 -- 172.21.15.37:0/3637982383 shutdown_connections 2024-04-25T09:02:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 --2- 172.21.15.37:0/3637982383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37280051d0 0x7f372800b9d0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 -- 172.21.15.37:0/3637982383 >> 172.21.15.37:0/3637982383 conn(0x7f374006ae60 msgr2=0x7f374006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 -- 172.21.15.37:0/3637982383 shutdown_connections 2024-04-25T09:02:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 -- 172.21.15.37:0/3637982383 wait complete. 2024-04-25T09:02:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 Processor -- start 2024-04-25T09:02:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 -- start start 2024-04-25T09:02:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37280051d0 0x7f374006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37400c67a0 0x7f37400c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37400c1dd0 0x7f37400c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37400cb9f0 con 0x7f37400c1dd0 2024-04-25T09:02:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37400cc930 con 0x7f37280051d0 2024-04-25T09:02:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3746134640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37400cbb70 con 0x7f37400c67a0 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f373ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37400c1dd0 0x7f37400c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f373ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37400c1dd0 0x7f37400c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51224/0 (socket says 172.21.15.37:51224) 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f373ffff640 1 -- 172.21.15.37:0/2914849390 learned_addr learned my addr 172.21.15.37:0/2914849390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f373f7fe640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37280051d0 0x7f374006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f373effd640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37400c67a0 0x7f37400c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f373ffff640 1 -- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37400c67a0 msgr2=0x7f37400c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f373ffff640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37400c67a0 0x7f37400c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f373ffff640 1 -- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37280051d0 msgr2=0x7f374006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f373ffff640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37280051d0 0x7f374006d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f373ffff640 1 -- 172.21.15.37:0/2914849390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37400b32d0 con 0x7f37400c1dd0 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f373ffff640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37400c1dd0 0x7f37400c2260 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f373801dc00 tx=0x7f3728005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3727fff640 1 -- 172.21.15.37:0/2914849390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3738001940 con 0x7f37400c1dd0 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3727fff640 1 -- 172.21.15.37:0/2914849390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3738001ae0 con 0x7f37400c1dd0 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.602+0000 7f3727fff640 1 -- 172.21.15.37:0/2914849390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37380ac9f0 con 0x7f37400c1dd0 2024-04-25T09:02:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f3746134640 1 -- 172.21.15.37:0/2914849390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37400cf690 con 0x7f37400c1dd0 2024-04-25T09:02:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f3746134640 1 -- 172.21.15.37:0/2914849390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37400cdda0 con 0x7f37400c1dd0 2024-04-25T09:02:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f3727fff640 1 -- 172.21.15.37:0/2914849390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37380acb90 con 0x7f37400c1dd0 2024-04-25T09:02:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f3727fff640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f372c066f90 0x7f372c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f3727fff640 1 -- 172.21.15.37:0/2914849390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6167+0+0 (secure 0 0 0) 0x7f373811ecd0 con 0x7f37400c1dd0 2024-04-25T09:02:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f373f7fe640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f372c066f90 0x7f372c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f37492a1000 1 Processor -- start 2024-04-25T09:02:38.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f37492a1000 1 -- start start 2024-04-25T09:02:38.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f373f7fe640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f372c066f90 0x7f372c069450 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3730001660 tx=0x7f3730009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:38.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f37492a1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 0x55a7c288adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f37492a1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c2840b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f37492a1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c2841480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f37492a1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c268dff0 con 0x55a7c28410a0 2024-04-25T09:02:38.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f37492a1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2616150 con 0x55a7c288a9d0 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f37492a1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7c268a090 con 0x55a7c288b2f0 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f373ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c2841480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f373ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c2841480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51234/0 (socket says 172.21.15.37:51234) 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.606+0000 7f373ffff640 1 -- 172.21.15.37:0/3466200579 learned_addr learned my addr 172.21.15.37:0/3466200579 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373effd640 1 --2- 172.21.15.37:0/3466200579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c2840b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373f7fe640 1 --2- 172.21.15.37:0/3466200579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 0x55a7c288adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373ffff640 1 -- 172.21.15.37:0/3466200579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 msgr2=0x55a7c2840b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373ffff640 1 --2- 172.21.15.37:0/3466200579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c2840b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373ffff640 1 -- 172.21.15.37:0/3466200579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 msgr2=0x55a7c288adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373ffff640 1 --2- 172.21.15.37:0/3466200579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 0x55a7c288adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373ffff640 1 -- 172.21.15.37:0/3466200579 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c28240d0 con 0x55a7c28410a0 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373f7fe640 1 --2- 172.21.15.37:0/3466200579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 0x55a7c288adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373ffff640 1 --2- 172.21.15.37:0/3466200579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c2841480 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f37380f7550 tx=0x7f3738007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db08cfecd319b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37257fa640 1 -- 172.21.15.37:0/3466200579 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37380bc070 con 0x55a7c28410a0 2024-04-25T09:02:38.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37257fa640 1 -- 172.21.15.37:0/3466200579 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37380ef370 con 0x55a7c28410a0 2024-04-25T09:02:38.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37257fa640 1 -- 172.21.15.37:0/3466200579 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37380f34c0 con 0x55a7c28410a0 2024-04-25T09:02:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 -- 172.21.15.37:0/3466200579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 msgr2=0x55a7c2841480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 --2- 172.21.15.37:0/3466200579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c2841480 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f37380f7550 tx=0x7f3738007040 comp rx=0 tx=0).stop 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 -- 172.21.15.37:0/3466200579 shutdown_connections 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 --2- 172.21.15.37:0/3466200579 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c2841480 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 --2- 172.21.15.37:0/3466200579 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c2840b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 --2- 172.21.15.37:0/3466200579 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 0x55a7c288adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 -- 172.21.15.37:0/3466200579 >> 172.21.15.37:0/3466200579 conn(0x55a7c283d020 msgr2=0x55a7c283f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 -- 172.21.15.37:0/3466200579 shutdown_connections 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 -- 172.21.15.37:0/3466200579 wait complete. 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 -- start start 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 Processor -- start 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c286f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 0x55a7c286fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c2870270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2652250 con 0x55a7c28410a0 2024-04-25T09:02:38.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2648690 con 0x55a7c288a9d0 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2616a40 con 0x55a7c288b2f0 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 10 client.0 _ll_register_callbacks cb 0x55a7c282be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c286f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c286f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51236/0 (socket says 172.21.15.37:51236) 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373f7fe640 1 -- 172.21.15.37:0/1619790639 learned_addr learned my addr 172.21.15.37:0/1619790639 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373ffff640 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c2870270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373effd640 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 0x55a7c286fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373f7fe640 1 -- 172.21.15.37:0/1619790639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 msgr2=0x55a7c2870270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373f7fe640 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c2870270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373f7fe640 1 -- 172.21.15.37:0/1619790639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 msgr2=0x55a7c286fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373f7fe640 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 0x55a7c286fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373f7fe640 1 -- 172.21.15.37:0/1619790639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7c273b1c0 con 0x55a7c28410a0 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373effd640 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 0x55a7c286fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f373f7fe640 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c286f7f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f3730003b80 tx=0x7f3730001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a565faa6fed9c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f371e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:38.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f371e7fc640 1 -- 172.21.15.37:0/1619790639 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730019070 con 0x55a7c28410a0 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f371e7fc640 1 -- 172.21.15.37:0/1619790639 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6167+0+0 (secure 0 0 0) 0x7f3730010040 con 0x55a7c28410a0 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.610+0000 7f37492a1000 1 -- 172.21.15.37:0/1619790639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37380ef370 con 0x55a7c28410a0 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f371e7fc640 1 -- 172.21.15.37:0/1619790639 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f373000fd70 con 0x55a7c28410a0 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 10 client.9111 fetch_fsmap learned FSMap version 81 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 -- 172.21.15.37:0/1619790639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55a7c2737170 con 0x55a7c28410a0 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f371e7fc640 1 -- 172.21.15.37:0/1619790639 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37380ef370 con 0x55a7c28410a0 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f371e7fc640 1 -- 172.21.15.37:0/1619790639 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f373001f070 con 0x55a7c28410a0 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 10 client.9111 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 10 client.9111 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 20 client.9111 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55a7c289ac80 0x55a7c289d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 4 client.9111 mds_command: new command op to 8727 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 -- 172.21.15.37:0/1619790639 --> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7c2629c60 con 0x55a7c289ac80 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f373effd640 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55a7c289ac80 0x55a7c289d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f373effd640 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55a7c289ac80 0x55a7c289d070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6577a05d8bccde7 server_cookie=3cccf2257327f101 in_seq=0 out_seq=0 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f371e7fc640 10 client.9111 ms_handle_connect on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f371e7fc640 1 -- 172.21.15.37:0/1619790639 <== mds.0 v2:172.21.15.37:6834/1266122577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a7c2629c60 con 0x55a7c289ac80 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f371e7fc640 10 client.9111 handle_command_reply: tid=0 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 client.9111 shutdown 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 -- 172.21.15.37:0/1619790639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 msgr2=0x55a7c286f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c286f7f0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f3730003b80 tx=0x7f3730001130 comp rx=0 tx=0).stop 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 -- 172.21.15.37:0/1619790639 shutdown_connections 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55a7c289ac80 0x55a7c289d070 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c2870270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c288a9d0 0x55a7c286fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f371e7fc640 0 client.9111 ms_handle_reset on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 --2- 172.21.15.37:0/1619790639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c28410a0 0x55a7c286f7f0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 -- 172.21.15.37:0/1619790639 >> 172.21.15.37:0/1619790639 conn(0x55a7c283d020 msgr2=0x55a7c283fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 -- 172.21.15.37:0/1619790639 shutdown_connections 2024-04-25T09:02:38.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 1 -- 172.21.15.37:0/1619790639 wait complete. 2024-04-25T09:02:38.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.614+0000 7f37492a1000 20 client.9111 trim_cache size 0 max 16384 2024-04-25T09:02:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 Processor -- start 2024-04-25T09:02:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 -- start start 2024-04-25T09:02:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c28410a0 0x55a7c28e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288a9d0 0x55a7c28e7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288b2f0 0x55a7c28e7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37402007f0 con 0x55a7c288b2f0 2024-04-25T09:02:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37401ff990 con 0x55a7c28410a0 2024-04-25T09:02:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37401fed90 con 0x55a7c288a9d0 2024-04-25T09:02:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288b2f0 0x55a7c28e7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288b2f0 0x55a7c28e7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51242/0 (socket says 172.21.15.37:51242) 2024-04-25T09:02:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373ffff640 1 -- 172.21.15.37:0/244900187 learned_addr learned my addr 172.21.15.37:0/244900187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373effd640 1 --2- 172.21.15.37:0/244900187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288a9d0 0x55a7c28e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373f7fe640 1 --2- 172.21.15.37:0/244900187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c28410a0 0x55a7c28e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373ffff640 1 -- 172.21.15.37:0/244900187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288a9d0 msgr2=0x55a7c28e7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373ffff640 1 --2- 172.21.15.37:0/244900187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288a9d0 0x55a7c28e7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373ffff640 1 -- 172.21.15.37:0/244900187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c28410a0 msgr2=0x55a7c28e6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373ffff640 1 --2- 172.21.15.37:0/244900187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c28410a0 0x55a7c28e6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373ffff640 1 -- 172.21.15.37:0/244900187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7c26d4de0 con 0x55a7c288b2f0 2024-04-25T09:02:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373effd640 1 --2- 172.21.15.37:0/244900187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288a9d0 0x55a7c28e7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f373ffff640 1 --2- 172.21.15.37:0/244900187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288b2f0 0x55a7c28e7960 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f37380acfd0 tx=0x7f37380b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648da6a20d38c037 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f3724ff9640 1 -- 172.21.15.37:0/244900187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37380f7a80 con 0x55a7c288b2f0 2024-04-25T09:02:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f3724ff9640 1 -- 172.21.15.37:0/244900187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37380f34c0 con 0x55a7c288b2f0 2024-04-25T09:02:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f3724ff9640 1 -- 172.21.15.37:0/244900187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37380eb9e0 con 0x55a7c288b2f0 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 -- 172.21.15.37:0/244900187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288b2f0 msgr2=0x55a7c28e7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 --2- 172.21.15.37:0/244900187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288b2f0 0x55a7c28e7960 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f37380acfd0 tx=0x7f37380b8000 comp rx=0 tx=0).stop 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 -- 172.21.15.37:0/244900187 shutdown_connections 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 --2- 172.21.15.37:0/244900187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288b2f0 0x55a7c28e7960 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 --2- 172.21.15.37:0/244900187 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288a9d0 0x55a7c28e7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 --2- 172.21.15.37:0/244900187 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c28410a0 0x55a7c28e6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.630+0000 7f37492a1000 1 -- 172.21.15.37:0/244900187 >> 172.21.15.37:0/244900187 conn(0x55a7c283d020 msgr2=0x55a7c2897fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- 172.21.15.37:0/244900187 shutdown_connections 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- 172.21.15.37:0/244900187 wait complete. 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- start start 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 Processor -- start 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c28410a0 0x55a7c28eab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288a9d0 0x55a7c28eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c289cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7c2628e00 con 0x55a7c288a9d0 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37401ff390 con 0x55a7c28410a0 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7c268d300 con 0x55a7c288b2f0 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 10 client.0 _ll_register_callbacks cb 0x55a7c280c040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288a9d0 0x55a7c28eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288a9d0 0x55a7c28eb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51252/0 (socket says 172.21.15.37:51252) 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373effd640 1 -- 172.21.15.37:0/317195507 learned_addr learned my addr 172.21.15.37:0/317195507 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373ffff640 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c289cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373f7fe640 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c28410a0 0x55a7c28eab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373effd640 1 -- 172.21.15.37:0/317195507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 msgr2=0x55a7c289cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373effd640 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c289cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373effd640 1 -- 172.21.15.37:0/317195507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c28410a0 msgr2=0x55a7c28eab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373effd640 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c28410a0 0x55a7c28eab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373effd640 1 -- 172.21.15.37:0/317195507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7c262f8e0 con 0x55a7c288a9d0 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373ffff640 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c289cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:38.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373effd640 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288a9d0 0x55a7c28eb080 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f3734009c90 tx=0x7f373401eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a815667789a9f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f371e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f371e7fc640 1 -- 172.21.15.37:0/317195507 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373400f040 con 0x55a7c288a9d0 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f371e7fc640 1 -- 172.21.15.37:0/317195507 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 6167+0+0 (secure 0 0 0) 0x7f3734005ba0 con 0x55a7c288a9d0 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- 172.21.15.37:0/317195507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37380f34c0 con 0x55a7c288a9d0 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f371e7fc640 1 -- 172.21.15.37:0/317195507 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3734005da0 con 0x55a7c288a9d0 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 10 client.9117 fetch_fsmap learned FSMap version 81 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- 172.21.15.37:0/317195507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55a7c26e78f0 con 0x55a7c288a9d0 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f371e7fc640 1 -- 172.21.15.37:0/317195507 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3734022030 con 0x55a7c288a9d0 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f371e7fc640 1 -- 172.21.15.37:0/317195507 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 81) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3734015040 con 0x55a7c288a9d0 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 10 client.9117 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 10 client.9117 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 20 client.9117 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55a7c289dd10 0x55a7c289e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 4 client.9117 mds_command: new command op to 8727 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:02:38.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- 172.21.15.37:0/317195507 --> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7c269b530 con 0x55a7c289dd10 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373f7fe640 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55a7c289dd10 0x55a7c289e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f373f7fe640 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55a7c289dd10 0x55a7c289e120 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83b47c0b6de206 server_cookie=33e1955aeea375d2 in_seq=0 out_seq=0 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f371e7fc640 10 client.9117 ms_handle_connect on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f371e7fc640 1 -- 172.21.15.37:0/317195507 <== mds.0 v2:172.21.15.37:6834/1266122577 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a7c269b530 con 0x55a7c289dd10 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f371e7fc640 10 client.9117 handle_command_reply: tid=0 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 client.9117 shutdown 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- 172.21.15.37:0/317195507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288a9d0 msgr2=0x55a7c28eb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288a9d0 0x55a7c28eb080 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f3734009c90 tx=0x7f373401eed0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- 172.21.15.37:0/317195507 shutdown_connections 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.37:6834/1266122577,v1:172.21.15.37:6835/1266122577] conn(0x55a7c289dd10 0x55a7c289e120 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a7c288b2f0 0x55a7c289cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a7c288a9d0 0x55a7c28eb080 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f371e7fc640 0 client.9117 ms_handle_reset on v2:172.21.15.37:6834/1266122577 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 --2- 172.21.15.37:0/317195507 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a7c28410a0 0x55a7c28eab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- 172.21.15.37:0/317195507 >> 172.21.15.37:0/317195507 conn(0x55a7c283d020 msgr2=0x55a7c28890a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- 172.21.15.37:0/317195507 shutdown_connections 2024-04-25T09:02:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 1 -- 172.21.15.37:0/317195507 wait complete. 2024-04-25T09:02:38.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.634+0000 7f37492a1000 20 client.9117 trim_cache size 0 max 16384 2024-04-25T09:02:38.630 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:02:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-04-25T09:02:38.630 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-04-25T09:02:38.630 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:02:38.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 -- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f372c066f90 msgr2=0x7f372c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f372c066f90 0x7f372c069450 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3730001660 tx=0x7f3730009040 comp rx=0 tx=0).stop 2024-04-25T09:02:38.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 -- 172.21.15.37:0/2914849390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37400c1dd0 msgr2=0x7f37400c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37400c1dd0 0x7f37400c2260 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f373801dc00 tx=0x7f3728005840 comp rx=0 tx=0).stop 2024-04-25T09:02:38.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 -- 172.21.15.37:0/2914849390 shutdown_connections 2024-04-25T09:02:38.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f372c066f90 0x7f372c069450 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37400c1dd0 0x7f37400c2260 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37400c67a0 0x7f37400c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 --2- 172.21.15.37:0/2914849390 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37280051d0 0x7f374006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 -- 172.21.15.37:0/2914849390 >> 172.21.15.37:0/2914849390 conn(0x7f374006ae60 msgr2=0x7f374006e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:38.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 -- 172.21.15.37:0/2914849390 shutdown_connections 2024-04-25T09:02:38.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.638+0000 7f3746134640 1 -- 172.21.15.37:0/2914849390 wait complete. 2024-04-25T09:02:38.642 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:02:38.743 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:02:38.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47d0c31640 1 Processor -- start 2024-04-25T09:02:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47d0c31640 1 -- start start 2024-04-25T09:02:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47d0c31640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47cc071030 0x7f47cc161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47d0c31640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f47cc071e90 0x7f47cc0b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47d0c31640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47cc0b29e0 con 0x7f47cc071410 2024-04-25T09:02:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47d0c31640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47cc162680 con 0x7f47cc071e90 2024-04-25T09:02:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47d0c31640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc162960 con 0x7f47cc071030 2024-04-25T09:02:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47c9d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f47cc071e90 0x7f47cc0b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47cad9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f47cc071410 0x7f47cc165920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:49152/0 (socket says 172.21.15.37:49152) 2024-04-25T09:02:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47cad9d640 1 -- 172.21.15.37:0/2249711729 learned_addr learned my addr 172.21.15.37:0/2249711729 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47ca59c640 1 --2- 172.21.15.37:0/2249711729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47cc071030 0x7f47cc161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47c959a640 1 -- 172.21.15.37:0/2249711729 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161910270 0 0) 0x7f47cc0b29e0 con 0x7f47cc071410 2024-04-25T09:02:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47c959a640 1 -- 172.21.15.37:0/2249711729 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47b8003530 con 0x7f47cc071410 2024-04-25T09:02:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47c9d9b640 1 -- 172.21.15.37:0/2249711729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47cc071030 msgr2=0x7f47cc161fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47c9d9b640 1 --2- 172.21.15.37:0/2249711729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47cc071030 0x7f47cc161fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47c9d9b640 1 -- 172.21.15.37:0/2249711729 >> v1:172.21.15.37:6789/0 conn(0x7f47cc071410 legacy=0x7f47cc165920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.886+0000 7f47c9d9b640 1 -- 172.21.15.37:0/2249711729 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc162c40 con 0x7f47cc071e90 2024-04-25T09:02:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47c9d9b640 1 --2- 172.21.15.37:0/2249711729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47cc071e90 0x7f47cc0b24a0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0097c0 tx=0x7f47bc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8a09c2e2e8493 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47c959a640 1 -- 172.21.15.37:0/2249711729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0b0070 con 0x7f47cc071e90 2024-04-25T09:02:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47c959a640 1 -- 172.21.15.37:0/2249711729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc0ab600 con 0x7f47cc071e90 2024-04-25T09:02:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47c959a640 1 -- 172.21.15.37:0/2249711729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0ab9d0 con 0x7f47cc071e90 2024-04-25T09:02:38.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47d0c31640 1 -- 172.21.15.37:0/2249711729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47cc071e90 msgr2=0x7f47cc0b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47d0c31640 1 --2- 172.21.15.37:0/2249711729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47cc071e90 0x7f47cc0b24a0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f47bc0097c0 tx=0x7f47bc01c410 comp rx=0 tx=0).stop 2024-04-25T09:02:38.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47d0c31640 1 -- 172.21.15.37:0/2249711729 shutdown_connections 2024-04-25T09:02:38.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47d0c31640 1 --2- 172.21.15.37:0/2249711729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47cc071e90 0x7f47cc0b24a0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47d0c31640 1 --2- 172.21.15.37:0/2249711729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47cc071030 0x7f47cc161fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47d0c31640 1 -- 172.21.15.37:0/2249711729 >> 172.21.15.37:0/2249711729 conn(0x7f47cc0b11a0 msgr2=0x7f47cc0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:38.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47d0c31640 1 -- 172.21.15.37:0/2249711729 shutdown_connections 2024-04-25T09:02:38.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.890+0000 7f47d0c31640 1 -- 172.21.15.37:0/2249711729 wait complete. 2024-04-25T09:02:38.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47d0c31640 1 Processor -- start 2024-04-25T09:02:38.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47d0c31640 1 -- start start 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47d0c31640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47cc071030 0x7f47cc15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47d0c31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47cc071410 0x7f47cc0cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47d0c31640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47cc071e90 0x7f47cc0cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47d0c31640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc16f050 con 0x7f47cc071410 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47d0c31640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc16eed0 con 0x7f47cc071030 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47d0c31640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc16f1d0 con 0x7f47cc071e90 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47cc071410 0x7f47cc0cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47cc071410 0x7f47cc0cf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51270/0 (socket says 172.21.15.37:51270) 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c9d9b640 1 -- 172.21.15.37:0/2798940736 learned_addr learned my addr 172.21.15.37:0/2798940736 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47cad9d640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47cc071e90 0x7f47cc0cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47ca59c640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47cc071030 0x7f47cc15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c9d9b640 1 -- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47cc071e90 msgr2=0x7f47cc0cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c9d9b640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47cc071e90 0x7f47cc0cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c9d9b640 1 -- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47cc071030 msgr2=0x7f47cc15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c9d9b640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47cc071030 0x7f47cc15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c9d9b640 1 -- 172.21.15.37:0/2798940736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc0cfec0 con 0x7f47cc071410 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47cad9d640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47cc071e90 0x7f47cc0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47ca59c640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47cc071030 0x7f47cc15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c9d9b640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47cc071410 0x7f47cc0cf4b0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f47bc01aac0 tx=0x7f47bc0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c2ffd640 1 -- 172.21.15.37:0/2798940736 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0ba070 con 0x7f47cc071410 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c2ffd640 1 -- 172.21.15.37:0/2798940736 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47bc0b5460 con 0x7f47cc071410 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47d0c31640 1 -- 172.21.15.37:0/2798940736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47cc0d01a0 con 0x7f47cc071410 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47c2ffd640 1 -- 172.21.15.37:0/2798940736 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47bc0b0040 con 0x7f47cc071410 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.894+0000 7f47d0c31640 1 -- 172.21.15.37:0/2798940736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47cc1565c0 con 0x7f47cc071410 2024-04-25T09:02:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.898+0000 7f47d0c31640 1 -- 172.21.15.37:0/2798940736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47cc04efa0 con 0x7f47cc071410 2024-04-25T09:02:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.898+0000 7f47c2ffd640 1 -- 172.21.15.37:0/2798940736 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f47bc0bd070 con 0x7f47cc071410 2024-04-25T09:02:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.898+0000 7f47c2ffd640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f47ac066f90 0x7f47ac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.898+0000 7f47c2ffd640 1 -- 172.21.15.37:0/2798940736 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6167+0+0 (secure 0 0 0) 0x7f47bc11f5b0 con 0x7f47cc071410 2024-04-25T09:02:38.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.898+0000 7f47ca59c640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f47ac066f90 0x7f47ac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:38.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.898+0000 7f47ca59c640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f47ac066f90 0x7f47ac069450 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009840 tx=0x7f47b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:38.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:38.902+0000 7f47c2ffd640 1 -- 172.21.15.37:0/2798940736 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f47bc016090 con 0x7f47cc071410 2024-04-25T09:02:39.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.098+0000 7f47d0c31640 1 -- 172.21.15.37:0/2798940736 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f47cc0b1dc0 con 0x7f47cc071410 2024-04-25T09:02:39.089 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T09:02:39.102+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:02:39.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.102+0000 7f47c2ffd640 1 -- 172.21.15.37:0/2798940736 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) v1 ==== 161+0+0 (secure 0 0 0) 0x7f47bc0f03f0 con 0x7f47cc071410 2024-04-25T09:02:39.092 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:02:39.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 -- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f47ac066f90 msgr2=0x7f47ac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f47ac066f90 0x7f47ac069450 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009840 tx=0x7f47b4008040 comp rx=0 tx=0).stop 2024-04-25T09:02:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 -- 172.21.15.37:0/2798940736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47cc071410 msgr2=0x7f47cc0cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47cc071410 0x7f47cc0cf4b0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f47bc01aac0 tx=0x7f47bc0191b0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 -- 172.21.15.37:0/2798940736 shutdown_connections 2024-04-25T09:02:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f47ac066f90 0x7f47ac069450 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f47cc071e90 0x7f47cc0cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47cc071410 0x7f47cc0cf4b0 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 --2- 172.21.15.37:0/2798940736 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f47cc071030 0x7f47cc15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 -- 172.21.15.37:0/2798940736 >> 172.21.15.37:0/2798940736 conn(0x7f47cc0b11a0 msgr2=0x7f47cc0b2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:39.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 -- 172.21.15.37:0/2798940736 shutdown_connections 2024-04-25T09:02:39.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.106+0000 7f47d0c31640 1 -- 172.21.15.37:0/2798940736 wait complete. 2024-04-25T09:02:39.107 INFO:tasks.ceph.mds.c.smithi037.stderr: -6495> 2024-04-25T09:01:36.522+0000 7fc23f839640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:02:39.114 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-04-25T09:02:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab1287400 1 -- start start 2024-04-25T09:02:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab1287400 1 Processor -- start 2024-04-25T09:02:39.147 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab1287400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555565c45800 0x555565dffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab1287400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555565c45400 0x555565dff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab1287400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555565c45000 0x555565dff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab1287400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555566a16180 con 0x555565c45400 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab1287400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555565ca4900 con 0x555565c45800 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab1287400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555565ca4480 con 0x555565c45000 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aaf245640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555565c45400 0x555565dff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aaf245640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555565c45400 0x555565dff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51302/0 (socket says 172.21.15.37:51302) 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aaf245640 1 -- 172.21.15.37:0/3841122760 learned_addr learned my addr 172.21.15.37:0/3841122760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab0247640 1 --2- 172.21.15.37:0/3841122760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555565c45000 0x555565dff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aafa46640 1 --2- 172.21.15.37:0/3841122760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555565c45800 0x555565dffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aaf245640 1 -- 172.21.15.37:0/3841122760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555565c45000 msgr2=0x555565dff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:39.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aaf245640 1 --2- 172.21.15.37:0/3841122760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555565c45000 0x555565dff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aaf245640 1 -- 172.21.15.37:0/3841122760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555565c45800 msgr2=0x555565dffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aaf245640 1 --2- 172.21.15.37:0/3841122760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555565c45800 0x555565dffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aaf245640 1 -- 172.21.15.37:0/3841122760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555669f2b40 con 0x555565c45400 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab0247640 1 --2- 172.21.15.37:0/3841122760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555565c45000 0x555565dff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aaf245640 1 --2- 172.21.15.37:0/3841122760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555565c45400 0x555565dff600 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x555566a9aa20 tx=0x555566a00000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63058678736cc6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aad241640 1 -- 172.21.15.37:0/3841122760 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555565c7a9c0 con 0x555565c45400 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aad241640 1 -- 172.21.15.37:0/3841122760 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555565c8e1c0 con 0x555565c45400 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5aad241640 1 -- 172.21.15.37:0/3841122760 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555565c7a4e0 con 0x555565c45400 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab1287400 1 -- 172.21.15.37:0/3841122760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555669f2d20 con 0x555565c45400 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab1287400 1 -- 172.21.15.37:0/3841122760 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5555669f2f00 con 0x555565c45400 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.158+0000 7f5ab1287400 4 waiting for MDS map... 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aad241640 1 -- 172.21.15.37:0/3841122760 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x555566b5e200 con 0x555565c45400 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aad241640 1 -- 172.21.15.37:0/3841122760 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 82) v1 ==== 2723+0+0 (secure 0 0 0) 0x555565c9b180 con 0x555565c45400 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 4 Got MDS map 82 2024-04-25T09:02:39.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 10 main: JournalTool::main 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 4 main: JournalTool: connecting to RADOS... 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 Processor -- start 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 -- start start 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5555669fd800 0x555565dfeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 0x555565dfe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555566bb6000 0x555565e00100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555565ca4180 con 0x5555669fdc00 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555566a16d80 con 0x555566bb6000 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555566a16f00 con 0x5555669fd800 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aafa46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 0x555565dfe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aafa46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 0x555565dfe580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51320/0 (socket says 172.21.15.37:51320) 2024-04-25T09:02:39.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aafa46640 1 -- 172.21.15.37:0/1322061179 learned_addr learned my addr 172.21.15.37:0/1322061179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab0247640 1 --2- 172.21.15.37:0/1322061179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5555669fd800 0x555565dfeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aaf245640 1 --2- 172.21.15.37:0/1322061179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555566bb6000 0x555565e00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aafa46640 1 -- 172.21.15.37:0/1322061179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5555669fd800 msgr2=0x555565dfeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aafa46640 1 --2- 172.21.15.37:0/1322061179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5555669fd800 0x555565dfeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aafa46640 1 -- 172.21.15.37:0/1322061179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555566bb6000 msgr2=0x555565e00100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aafa46640 1 --2- 172.21.15.37:0/1322061179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555566bb6000 0x555565e00100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aafa46640 1 -- 172.21.15.37:0/1322061179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555669f21e0 con 0x5555669fdc00 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aafa46640 1 --2- 172.21.15.37:0/1322061179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 0x555565dfe580 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x555566a106c0 tx=0x555566a0ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=302dfa99f487f33c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aab23d640 1 -- 172.21.15.37:0/1322061179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555565c7a9c0 con 0x5555669fdc00 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aab23d640 1 -- 172.21.15.37:0/1322061179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555565c8ec40 con 0x5555669fdc00 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5aab23d640 1 -- 172.21.15.37:0/1322061179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555565c7a4e0 con 0x5555669fdc00 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 -- 172.21.15.37:0/1322061179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 msgr2=0x555565dfe580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 --2- 172.21.15.37:0/1322061179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 0x555565dfe580 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x555566a106c0 tx=0x555566a0ec00 comp rx=0 tx=0).stop 2024-04-25T09:02:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 -- 172.21.15.37:0/1322061179 shutdown_connections 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 --2- 172.21.15.37:0/1322061179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555566bb6000 0x555565e00100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 --2- 172.21.15.37:0/1322061179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 0x555565dfe580 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 --2- 172.21.15.37:0/1322061179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5555669fd800 0x555565dfeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 -- 172.21.15.37:0/1322061179 >> 172.21.15.37:0/1322061179 conn(0x555565c44c00 msgr2=0x555565c73800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 -- 172.21.15.37:0/1322061179 shutdown_connections 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 -- 172.21.15.37:0/1322061179 wait complete. 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 Processor -- start 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.162+0000 7f5ab1287400 1 -- start start 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 0x555565dfeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555669fd800 0x555565dfe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555566bb7800 0x555565e00100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555566a17200 con 0x5555669fdc00 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555566a17080 con 0x5555669fd800 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555566a17380 con 0x555566bb7800 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab0247640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 0x555565dfeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab0247640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 0x555565dfeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51336/0 (socket says 172.21.15.37:51336) 2024-04-25T09:02:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab0247640 1 -- 172.21.15.37:0/2929672737 learned_addr learned my addr 172.21.15.37:0/2929672737 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5aaf245640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555566bb7800 0x555565e00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5aafa46640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555669fd800 0x555565dfe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab0247640 1 -- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555566bb7800 msgr2=0x555565e00100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab0247640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555566bb7800 0x555565e00100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab0247640 1 -- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555669fd800 msgr2=0x555565dfe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab0247640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555669fd800 0x555565dfe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab0247640 1 -- 172.21.15.37:0/2929672737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5555669f3a40 con 0x5555669fdc00 2024-04-25T09:02:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5aafa46640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5555669fd800 0x555565dfe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab0247640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 0x555565dfeb00 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x555566a113b0 tx=0x555566a0ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5aa9a3a640 1 -- 172.21.15.37:0/2929672737 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555565c7a9c0 con 0x5555669fdc00 2024-04-25T09:02:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5aa9a3a640 1 -- 172.21.15.37:0/2929672737 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555565c8e540 con 0x5555669fdc00 2024-04-25T09:02:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5555669f3c20 con 0x5555669fdc00 2024-04-25T09:02:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5aa9a3a640 1 -- 172.21.15.37:0/2929672737 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555565c7a4e0 con 0x5555669fdc00 2024-04-25T09:02:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5555669f3a40 con 0x5555669fdc00 2024-04-25T09:02:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 4 main: JournalTool: resolving pool 15 2024-04-25T09:02:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5aa9a3a640 1 -- 172.21.15.37:0/2929672737 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x555566bb0c00 con 0x5555669fdc00 2024-04-25T09:02:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5aa9a3a640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x555565f6d400 0x55556605c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5aa9a3a640 1 -- 172.21.15.37:0/2929672737 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x555565f76a00 con 0x5555669fdc00 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 4 main: JournalTool: creating IoCtx.. 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 4 main: Executing for rank 0 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 4 Filtering by inode '1099511627777' 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x555566bb7400 0x555565e00680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v9 -- 0x555566bb7000 con 0x555566bb7400 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5aafa46640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x555565f6d400 0x55556605c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.166+0000 7f5aaf245640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x555566bb7400 0x555565e00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5aafa46640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x555565f6d400 0x55556605c000 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x555566a106c0 tx=0x555566a00120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5aaf245640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x555566bb7400 0x555565e00680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5aaf245640 1 -- 172.21.15.37:0/2929672737 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x555565dc0000 con 0x555566bb7400 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 4 Pointer 400.00000000 is readable 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T09:02:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v9 -- 0x555566bb6c00 con 0x555566bb7400 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5aaf245640 1 -- 172.21.15.37:0/2929672737 <== osd.6 v2:172.21.15.203:6800/467003745 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x555565dc0000 con 0x555566bb7400 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 10 Header 0x400000 0x405f0e 0x40628a 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 10 Starting journal scan from offset 0x405f0e 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v9 -- 0x555566bb6800 con 0x555566bb7400 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5aaf245640 1 -- 172.21.15.37:0/2929672737 <== osd.6 v2:172.21.15.203:6800/467003745 3 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x555565dc0000 con 0x555566bb7400 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 10 Parsing data, 0x37c bytes available 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 10 Attempting decode at 0x405f0e 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 10 Consumed 0x37c bytes 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 10 Valid entry at 0x405f0e 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x555566bb6400 0x555565e00c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v9 -- 0x555566bb6000 con 0x555566bb6400 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.170+0000 7f5ab0247640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x555566bb6400 0x555565e00c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.174+0000 7f5ab0247640 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x555566bb6400 0x555565e00c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.167 INFO:teuthology.orchestra.run.smithi037.stdout:Events by type: 2024-04-25T09:02:39.167 INFO:teuthology.orchestra.run.smithi037.stdout:Errors: 0 2024-04-25T09:02:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab0247640 1 -- 172.21.15.37:0/2929672737 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x555565dc0500 con 0x555566bb6400 2024-04-25T09:02:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 4 Reached end of journal objects 2024-04-25T09:02:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 4 Scanned objects, 0 missing, 1 valid 2024-04-25T09:02:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 4 Events scanned, 0 gaps 2024-04-25T09:02:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 4 Found 1 valid events 2024-04-25T09:02:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 4 Selected 0 events events for processing 2024-04-25T09:02:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 4 0 events 2024-04-25T09:02:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x555566bb7400 msgr2=0x555565e00680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x555566bb7400 0x555565e00680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x555566bb6400 msgr2=0x555565e00c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x555566bb6400 0x555565e00c00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x555565f6d400 msgr2=0x55556605c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x555565f6d400 0x55556605c000 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x555566a106c0 tx=0x555566a00120 comp rx=0 tx=0).stop 2024-04-25T09:02:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 msgr2=0x555565dfeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 --2- 172.21.15.37:0/2929672737 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555669fdc00 0x555565dfeb00 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x555566a113b0 tx=0x555566a0ed20 comp rx=0 tx=0).stop 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5aafa46640 1 -- 172.21.15.37:0/2929672737 reap_dead start 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5aafa46640 1 -- 172.21.15.37:0/2929672737 reap_dead start 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 shutdown_connections 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 >> 172.21.15.37:0/2929672737 conn(0x555565c44c00 msgr2=0x555565c73800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 shutdown_connections 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/2929672737 wait complete. 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/3841122760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555565c45400 msgr2=0x555565dff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 --2- 172.21.15.37:0/3841122760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555565c45400 0x555565dff600 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x555566a9aa20 tx=0x555566a00000 comp rx=0 tx=0).stop 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/3841122760 shutdown_connections 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 --2- 172.21.15.37:0/3841122760 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555565c45000 0x555565dff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 --2- 172.21.15.37:0/3841122760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555565c45400 0x555565dff600 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 --2- 172.21.15.37:0/3841122760 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555565c45800 0x555565dffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/3841122760 >> 172.21.15.37:0/3841122760 conn(0x555565c45c00 msgr2=0x555565c73000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/3841122760 shutdown_connections 2024-04-25T09:02:39.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.178+0000 7f5ab1287400 1 -- 172.21.15.37:0/3841122760 wait complete. 2024-04-25T09:02:39.172 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.058895, result: Events by type: Errors: 0 2024-04-25T09:02:39.173 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-04-25T09:02:39.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855eb34400 1 -- start start 2024-04-25T09:02:39.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855eb34400 1 Processor -- start 2024-04-25T09:02:39.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855eb34400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f306f800 0x5639f3228580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855eb34400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f306f400 0x5639f3228b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855eb34400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f306f000 0x5639f3229080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855eb34400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f3e3e600 con 0x5639f306f400 2024-04-25T09:02:39.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855eb34400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f3e3e480 con 0x5639f306f000 2024-04-25T09:02:39.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855eb34400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639f3e3e300 con 0x5639f306f800 2024-04-25T09:02:39.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855caf2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f306f400 0x5639f3228b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855caf2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f306f400 0x5639f3228b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51356/0 (socket says 172.21.15.37:51356) 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855caf2640 1 -- 172.21.15.37:0/2907095747 learned_addr learned my addr 172.21.15.37:0/2907095747 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855d2f3640 1 --2- 172.21.15.37:0/2907095747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f306f800 0x5639f3228580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855daf4640 1 --2- 172.21.15.37:0/2907095747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f306f000 0x5639f3229080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855caf2640 1 -- 172.21.15.37:0/2907095747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f306f800 msgr2=0x5639f3228580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855caf2640 1 --2- 172.21.15.37:0/2907095747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f306f800 0x5639f3228580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855caf2640 1 -- 172.21.15.37:0/2907095747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f306f000 msgr2=0x5639f3229080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855caf2640 1 --2- 172.21.15.37:0/2907095747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f306f000 0x5639f3229080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855caf2640 1 -- 172.21.15.37:0/2907095747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639f3106960 con 0x5639f306f400 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855daf4640 1 --2- 172.21.15.37:0/2907095747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f306f000 0x5639f3229080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855d2f3640 1 --2- 172.21.15.37:0/2907095747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f306f800 0x5639f3228580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855caf2640 1 --2- 172.21.15.37:0/2907095747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f306f400 0x5639f3228b00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x5639f3ed2960 tx=0x5639f3162000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946110ba0fa06782 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855aaee640 1 -- 172.21.15.37:0/2907095747 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639f30a49c0 con 0x5639f306f400 2024-04-25T09:02:39.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855aaee640 1 -- 172.21.15.37:0/2907095747 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639f30b8540 con 0x5639f306f400 2024-04-25T09:02:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855aaee640 1 -- 172.21.15.37:0/2907095747 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639f30a44e0 con 0x5639f306f400 2024-04-25T09:02:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855eb34400 1 -- 172.21.15.37:0/2907095747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639f3106d20 con 0x5639f306f400 2024-04-25T09:02:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855eb34400 1 -- 172.21.15.37:0/2907095747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5639f3106f00 con 0x5639f306f400 2024-04-25T09:02:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.258+0000 7f855eb34400 4 waiting for MDS map... 2024-04-25T09:02:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855aaee640 1 -- 172.21.15.37:0/2907095747 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x5639f3f88200 con 0x5639f306f400 2024-04-25T09:02:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855aaee640 1 -- 172.21.15.37:0/2907095747 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 82) v1 ==== 2723+0+0 (secure 0 0 0) 0x5639f30c5180 con 0x5639f306f400 2024-04-25T09:02:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 4 Got MDS map 82 2024-04-25T09:02:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 10 main: JournalTool::main 2024-04-25T09:02:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 4 main: JournalTool: connecting to RADOS... 2024-04-25T09:02:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 Processor -- start 2024-04-25T09:02:39.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- start start 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f3e45800 0x5639f3229600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 0x5639f3229b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f3fe4000 0x5639f322a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f3e3e180 con 0x5639f3e45c00 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f30ce480 con 0x5639f3e45800 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639f30ce900 con 0x5639f3fe4000 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855d2f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 0x5639f3229b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855d2f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 0x5639f3229b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51364/0 (socket says 172.21.15.37:51364) 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855d2f3640 1 -- 172.21.15.37:0/3309969188 learned_addr learned my addr 172.21.15.37:0/3309969188 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855caf2640 1 --2- 172.21.15.37:0/3309969188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f3fe4000 0x5639f322a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855daf4640 1 --2- 172.21.15.37:0/3309969188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f3e45800 0x5639f3229600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855d2f3640 1 -- 172.21.15.37:0/3309969188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f3fe4000 msgr2=0x5639f322a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:39.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855d2f3640 1 --2- 172.21.15.37:0/3309969188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f3fe4000 0x5639f322a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855d2f3640 1 -- 172.21.15.37:0/3309969188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f3e45800 msgr2=0x5639f3229600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855d2f3640 1 --2- 172.21.15.37:0/3309969188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f3e45800 0x5639f3229600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855d2f3640 1 -- 172.21.15.37:0/3309969188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639f31061e0 con 0x5639f3e45c00 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855caf2640 1 --2- 172.21.15.37:0/3309969188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f3fe4000 0x5639f322a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855d2f3640 1 --2- 172.21.15.37:0/3309969188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 0x5639f3229b80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x5639f3fdc6f0 tx=0x5639f3f98cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d4ab91cd476267 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f8558aea640 1 -- 172.21.15.37:0/3309969188 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639f30a49c0 con 0x5639f3e45c00 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f8558aea640 1 -- 172.21.15.37:0/3309969188 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639f30b8c40 con 0x5639f3e45c00 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f8558aea640 1 -- 172.21.15.37:0/3309969188 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639f30a44e0 con 0x5639f3e45c00 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- 172.21.15.37:0/3309969188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 msgr2=0x5639f3229b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 --2- 172.21.15.37:0/3309969188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 0x5639f3229b80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x5639f3fdc6f0 tx=0x5639f3f98cc0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- 172.21.15.37:0/3309969188 shutdown_connections 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 --2- 172.21.15.37:0/3309969188 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f3fe4000 0x5639f322a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 --2- 172.21.15.37:0/3309969188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 0x5639f3229b80 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 --2- 172.21.15.37:0/3309969188 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f3e45800 0x5639f3229600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- 172.21.15.37:0/3309969188 >> 172.21.15.37:0/3309969188 conn(0x5639f306ec00 msgr2=0x5639f309b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- 172.21.15.37:0/3309969188 shutdown_connections 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- 172.21.15.37:0/3309969188 wait complete. 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 Processor -- start 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- start start 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 0x5639f322b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f3e45800 0x5639f322ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f3fe5800 0x5639f322a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f3e3f380 con 0x5639f3e45c00 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.262+0000 7f855eb34400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639f3e3e180 con 0x5639f3fe5800 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855eb34400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639f3e3f500 con 0x5639f3e45800 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855daf4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 0x5639f322b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855daf4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 0x5639f322b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51378/0 (socket says 172.21.15.37:51378) 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855daf4640 1 -- 172.21.15.37:0/1032489341 learned_addr learned my addr 172.21.15.37:0/1032489341 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855d2f3640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f3e45800 0x5639f322ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855caf2640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f3fe5800 0x5639f322a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855daf4640 1 -- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f3e45800 msgr2=0x5639f322ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855daf4640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f3e45800 0x5639f322ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855daf4640 1 -- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f3fe5800 msgr2=0x5639f322a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855daf4640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f3fe5800 0x5639f322a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855daf4640 1 -- 172.21.15.37:0/1032489341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639f3107a40 con 0x5639f3e45c00 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855d2f3640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f3e45800 0x5639f322ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855daf4640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 0x5639f322b180 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x5639f3fdd3b0 tx=0x5639f3f98ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f85572e7640 1 -- 172.21.15.37:0/1032489341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639f30a49c0 con 0x5639f3e45c00 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f85572e7640 1 -- 172.21.15.37:0/1032489341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639f30b9340 con 0x5639f3e45c00 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5639f3107c20 con 0x5639f3e45c00 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f85572e7640 1 -- 172.21.15.37:0/1032489341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5639f30a44e0 con 0x5639f3e45c00 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639f3107a40 con 0x5639f3e45c00 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855eb34400 4 main: JournalTool: resolving pool 15 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f85572e7640 1 -- 172.21.15.37:0/1032489341 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x5639f3fdec00 con 0x5639f3e45c00 2024-04-25T09:02:39.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f85572e7640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x5639f3ff3400 0x5639f322b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f85572e7640 1 -- 172.21.15.37:0/1032489341 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x5639f33a0a00 con 0x5639f3e45c00 2024-04-25T09:02:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855eb34400 4 main: JournalTool: creating IoCtx.. 2024-04-25T09:02:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855eb34400 4 main: Executing for rank 0 2024-04-25T09:02:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855eb34400 4 Filtering by inode '1099511627778' 2024-04-25T09:02:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855eb34400 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5639f3fe5400 0x5639f322a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:1 15.f 15:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v9 -- 0x5639f3fe5000 con 0x5639f3fe5400 2024-04-25T09:02:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855d2f3640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x5639f3ff3400 0x5639f322b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855caf2640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5639f3fe5400 0x5639f322a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855caf2640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5639f3fe5400 0x5639f322a100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.266+0000 7f855d2f3640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x5639f3ff3400 0x5639f322b700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x5639f3fdc6f0 tx=0x5639f3f98c00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855caf2640 1 -- 172.21.15.37:0/1032489341 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+22 (crc 0 0 0) 0x5639f31ea000 con 0x5639f3fe5400 2024-04-25T09:02:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 4 Pointer 400.00000000 is readable 2024-04-25T09:02:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 4 JournalScanner::scan: reading header object '200.00000000' 2024-04-25T09:02:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:2 15.14 15:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v9 -- 0x5639f3fe4c00 con 0x5639f3fe5400 2024-04-25T09:02:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855caf2640 1 -- 172.21.15.37:0/1032489341 <== osd.6 v2:172.21.15.203:6800/467003745 2 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) v8 ==== 156+0+90 (crc 0 0 0) 0x5639f31ea000 con 0x5639f3fe5400 2024-04-25T09:02:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 10 Header 0x400000 0x405f0e 0x40628a 2024-04-25T09:02:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 10 Starting journal scan from offset 0x405f0e 2024-04-25T09:02:39.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:3 15.14 15:2e2fa760:::200.00000001:head [read 24334~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v9 -- 0x5639f3fe4800 con 0x5639f3fe5400 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855caf2640 1 -- 172.21.15.37:0/1032489341 <== osd.6 v2:172.21.15.203:6800/467003745 3 ==== osd_op_reply(3 200.00000001 [read 24334~892 out=892b] v0'0 uv28 ondisk = 0) v8 ==== 156+0+892 (crc 0 0 0) 0x5639f31ea000 con 0x5639f3fe5400 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 4 Read 0x37c bytes from 200.00000001 gap=0 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 10 Parsing data, 0x37c bytes available 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 10 Attempting decode at 0x405f0e 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 10 Consumed 0x37c bytes 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 10 Valid entry at 0x405f0e 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5639f3fe4400 0x5639f3229b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:4 15.1b 15:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e113) v9 -- 0x5639f3fe4000 con 0x5639f3fe4400 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855daf4640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5639f3fe4400 0x5639f3229b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855daf4640 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5639f3fe4400 0x5639f3229b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855daf4640 1 -- 172.21.15.37:0/1032489341 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5639f31ea500 con 0x5639f3fe4400 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 4 Reached end of journal objects 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 4 Scanned objects, 0 missing, 1 valid 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 4 Events scanned, 0 gaps 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 4 Found 1 valid events 2024-04-25T09:02:39.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 4 Selected 0 events events for processing 2024-04-25T09:02:39.264 INFO:teuthology.orchestra.run.smithi037.stdout:Events by type: 2024-04-25T09:02:39.264 INFO:teuthology.orchestra.run.smithi037.stdout:Errors: 0 2024-04-25T09:02:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 4 0 events 2024-04-25T09:02:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5639f3fe5400 msgr2=0x5639f322a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x5639f3fe5400 0x5639f322a100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5639f3fe4400 msgr2=0x5639f3229b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x5639f3fe4400 0x5639f3229b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x5639f3ff3400 msgr2=0x5639f322b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x5639f3ff3400 0x5639f322b700 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x5639f3fdc6f0 tx=0x5639f3f98c00 comp rx=0 tx=0).stop 2024-04-25T09:02:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 msgr2=0x5639f322b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 --2- 172.21.15.37:0/1032489341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f3e45c00 0x5639f322b180 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x5639f3fdd3b0 tx=0x5639f3f98ea0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855d2f3640 1 -- 172.21.15.37:0/1032489341 reap_dead start 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 shutdown_connections 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855d2f3640 1 -- 172.21.15.37:0/1032489341 reap_dead start 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 >> 172.21.15.37:0/1032489341 conn(0x5639f306ec00 msgr2=0x5639f309b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 shutdown_connections 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/1032489341 wait complete. 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/2907095747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f306f400 msgr2=0x5639f3228b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 --2- 172.21.15.37:0/2907095747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f306f400 0x5639f3228b00 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x5639f3ed2960 tx=0x5639f3162000 comp rx=0 tx=0).stop 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/2907095747 shutdown_connections 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 --2- 172.21.15.37:0/2907095747 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639f306f000 0x5639f3229080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 --2- 172.21.15.37:0/2907095747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639f306f400 0x5639f3228b00 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 --2- 172.21.15.37:0/2907095747 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639f306f800 0x5639f3228580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.270+0000 7f855eb34400 1 -- 172.21.15.37:0/2907095747 >> 172.21.15.37:0/2907095747 conn(0x5639f306fc00 msgr2=0x5639f309b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.274+0000 7f855eb34400 1 -- 172.21.15.37:0/2907095747 shutdown_connections 2024-04-25T09:02:39.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.274+0000 7f855eb34400 1 -- 172.21.15.37:0/2907095747 wait complete. 2024-04-25T09:02:39.268 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.095032, result: Events by type: Errors: 0 2024-04-25T09:02:39.268 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-04-25T09:02:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 Processor -- start 2024-04-25T09:02:39.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 -- start start 2024-04-25T09:02:39.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a331f570 0x55b0a33f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0a336d360 0x55b0a334f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b0a33f71f0 0x55b0a33f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a3351b70 con 0x55b0a331f570 2024-04-25T09:02:39.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a33519f0 con 0x55b0a336d360 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0a3351cf0 con 0x55b0a33f71f0 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedf1c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a331f570 0x55b0a33f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedf1c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a331f570 0x55b0a33f5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51524/0 (socket says 172.21.15.37:51524) 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedf1c7640 1 -- 172.21.15.37:0/2412899129 learned_addr learned my addr 172.21.15.37:0/2412899129 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdede9c6640 1 --2- 172.21.15.37:0/2412899129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0a336d360 0x55b0a334f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedf1c7640 1 -- 172.21.15.37:0/2412899129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b0a33f71f0 msgr2=0x55b0a33f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedf9c8640 1 --2- 172.21.15.37:0/2412899129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b0a33f71f0 0x55b0a33f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedf1c7640 1 --2- 172.21.15.37:0/2412899129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b0a33f71f0 0x55b0a33f1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedf1c7640 1 -- 172.21.15.37:0/2412899129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0a336d360 msgr2=0x55b0a334f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedf1c7640 1 --2- 172.21.15.37:0/2412899129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0a336d360 0x55b0a334f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedf1c7640 1 -- 172.21.15.37:0/2412899129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0a33f35c0 con 0x55b0a331f570 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedf9c8640 1 --2- 172.21.15.37:0/2412899129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b0a33f71f0 0x55b0a33f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdede9c6640 1 --2- 172.21.15.37:0/2412899129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0a336d360 0x55b0a334f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedf1c7640 1 --2- 172.21.15.37:0/2412899129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a331f570 0x55b0a33f5ca0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fded0019850 tx=0x7fded00033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7701805e2d8f59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedd9c4640 1 -- 172.21.15.37:0/2412899129 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00aab60 con 0x55b0a331f570 2024-04-25T09:02:39.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedd9c4640 1 -- 172.21.15.37:0/2412899129 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded00b9070 con 0x55b0a331f570 2024-04-25T09:02:39.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdedd9c4640 1 -- 172.21.15.37:0/2412899129 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded00b4460 con 0x55b0a331f570 2024-04-25T09:02:39.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 -- 172.21.15.37:0/2412899129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a331f570 msgr2=0x55b0a33f5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 --2- 172.21.15.37:0/2412899129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a331f570 0x55b0a33f5ca0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fded0019850 tx=0x7fded00033b0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 -- 172.21.15.37:0/2412899129 shutdown_connections 2024-04-25T09:02:39.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 --2- 172.21.15.37:0/2412899129 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b0a33f71f0 0x55b0a33f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 --2- 172.21.15.37:0/2412899129 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0a336d360 0x55b0a334f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 --2- 172.21.15.37:0/2412899129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a331f570 0x55b0a33f5ca0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 -- 172.21.15.37:0/2412899129 >> 172.21.15.37:0/2412899129 conn(0x55b0a3325c90 msgr2=0x55b0a331ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:39.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 -- 172.21.15.37:0/2412899129 shutdown_connections 2024-04-25T09:02:39.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.354+0000 7fdee16a9080 1 -- 172.21.15.37:0/2412899129 wait complete. 2024-04-25T09:02:39.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdee16a9080 1 Processor -- start 2024-04-25T09:02:39.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdee16a9080 1 -- start start 2024-04-25T09:02:39.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdee16a9080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0a331f570 0x55b0a33458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdee16a9080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a336d360 0x55b0a33da210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdee16a9080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b0a33f71f0 0x55b0a33db9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdee16a9080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a33f3b00 con 0x55b0a336d360 2024-04-25T09:02:39.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdee16a9080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b0a33f3860 con 0x55b0a331f570 2024-04-25T09:02:39.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdee16a9080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b0a33f3da0 con 0x55b0a33f71f0 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdede9c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a336d360 0x55b0a33da210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdede9c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a336d360 0x55b0a33da210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51532/0 (socket says 172.21.15.37:51532) 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdede9c6640 1 -- 172.21.15.37:0/983719763 learned_addr learned my addr 172.21.15.37:0/983719763 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdedf1c7640 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0a331f570 0x55b0a33458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdedf9c8640 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b0a33f71f0 0x55b0a33db9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdede9c6640 1 -- 172.21.15.37:0/983719763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b0a33f71f0 msgr2=0x55b0a33db9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdede9c6640 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b0a33f71f0 0x55b0a33db9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdede9c6640 1 -- 172.21.15.37:0/983719763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0a331f570 msgr2=0x55b0a33458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdede9c6640 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b0a331f570 0x55b0a33458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdede9c6640 1 -- 172.21.15.37:0/983719763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b0a33daab0 con 0x55b0a336d360 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdede9c6640 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a336d360 0x55b0a33da210 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0076e0 tx=0x7fdecc007ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdedc9c2640 1 -- 172.21.15.37:0/983719763 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc00b0d0 con 0x55b0a336d360 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdedc9c2640 1 -- 172.21.15.37:0/983719763 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdecc00b270 con 0x55b0a336d360 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdee16a9080 1 -- 172.21.15.37:0/983719763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b0a33dad50 con 0x55b0a336d360 2024-04-25T09:02:39.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdedc9c2640 1 -- 172.21.15.37:0/983719763 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdecc006b10 con 0x55b0a336d360 2024-04-25T09:02:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdee16a9080 1 -- 172.21.15.37:0/983719763 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b0a33db290 con 0x55b0a336d360 2024-04-25T09:02:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdedc9c2640 1 -- 172.21.15.37:0/983719763 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdecc006cb0 con 0x55b0a336d360 2024-04-25T09:02:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.358+0000 7fdedc9c2640 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdec8065970 0x7fdec8067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.362+0000 7fdedc9c2640 1 -- 172.21.15.37:0/983719763 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fdecc083e30 con 0x55b0a336d360 2024-04-25T09:02:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.362+0000 7fdedf1c7640 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdec8065970 0x7fdec8067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.362+0000 7fdee16a9080 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b0a33dc8e0 0x55b0a3367400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.362+0000 7fdee16a9080 1 -- 172.21.15.37:0/983719763 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e113) v9 -- 0x55b0a3364c20 con 0x55b0a33dc8e0 2024-04-25T09:02:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.362+0000 7fdedf9c8640 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b0a33dc8e0 0x55b0a3367400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.362+0000 7fdedf9c8640 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b0a33dc8e0 0x55b0a3367400 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.362+0000 7fdedf1c7640 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdec8065970 0x7fdec8067e30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fded00b8000 tx=0x7fded00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdedf9c8640 1 -- 172.21.15.37:0/983719763 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v113'5 uv5 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fded8007680 con 0x55b0a33dc8e0 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdee16a9080 1 -- 172.21.15.37:0/983719763 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b0a33dc8e0 msgr2=0x55b0a3367400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdee16a9080 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b0a33dc8e0 0x55b0a3367400 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdee16a9080 1 -- 172.21.15.37:0/983719763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdec8065970 msgr2=0x7fdec8067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdee16a9080 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdec8065970 0x7fdec8067e30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fded00b8000 tx=0x7fded00a6000 comp rx=0 tx=0).stop 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdee16a9080 1 -- 172.21.15.37:0/983719763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a336d360 msgr2=0x55b0a33da210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdee16a9080 1 --2- 172.21.15.37:0/983719763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a336d360 0x55b0a33da210 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fdecc0076e0 tx=0x7fdecc007ba0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdedf9c8640 1 -- 172.21.15.37:0/983719763 reap_dead start 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdee16a9080 1 -- 172.21.15.37:0/983719763 shutdown_connections 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdee16a9080 1 -- 172.21.15.37:0/983719763 >> 172.21.15.37:0/983719763 conn(0x55b0a3325c90 msgr2=0x55b0a334d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdee16a9080 1 -- 172.21.15.37:0/983719763 shutdown_connections 2024-04-25T09:02:39.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.366+0000 7fdee16a9080 1 -- 172.21.15.37:0/983719763 wait complete. 2024-04-25T09:02:39.356 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T09:02:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.542+0000 7f3ab1d55640 1 Processor -- start 2024-04-25T09:02:39.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.542+0000 7f3ab1d55640 1 -- start start 2024-04-25T09:02:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3ab1d55640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac15b6b0 0x7f3aac0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3ab1d55640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3aac073010 0x7f3aac0cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3ab1d55640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aac0cbd20 con 0x7f3aac0cf590 2024-04-25T09:02:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3ab1d55640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aac15ba90 con 0x7f3aac073010 2024-04-25T09:02:39.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3ab1d55640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0cff30 con 0x7f3aac15b6b0 2024-04-25T09:02:39.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac15b6b0 0x7f3aac0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac15b6b0 0x7f3aac0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:44264/0 (socket says 172.21.15.37:44264) 2024-04-25T09:02:39.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3aab7fe640 1 -- 172.21.15.37:0/4088068758 learned_addr learned my addr 172.21.15.37:0/4088068758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:39.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3aabfff640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3aac073010 0x7f3aac0cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3aab7fe640 1 -- 172.21.15.37:0/4088068758 >> v2:172.21.15.203:3300/0 conn(0x7f3aac073010 msgr2=0x7f3aac0cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:39.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3aab7fe640 1 --2- 172.21.15.37:0/4088068758 >> v2:172.21.15.203:3300/0 conn(0x7f3aac073010 0x7f3aac0cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3aab7fe640 1 -- 172.21.15.37:0/4088068758 >> v1:172.21.15.203:6789/0 conn(0x7f3aac0cf590 legacy=0x7f3aac15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.546+0000 7f3aab7fe640 1 -- 172.21.15.37:0/4088068758 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac0cc000 con 0x7f3aac15b6b0 2024-04-25T09:02:39.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3aab7fe640 1 --2- 172.21.15.37:0/4088068758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac15b6b0 0x7f3aac0ca3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009e50 tx=0x7f3a9c01ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333a722cb752bfdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3aaa7fc640 1 -- 172.21.15.37:0/4088068758 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0a57e0 con 0x7f3aac15b6b0 2024-04-25T09:02:39.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3aaa7fc640 1 -- 172.21.15.37:0/4088068758 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c0a5980 con 0x7f3aac15b6b0 2024-04-25T09:02:39.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3aaa7fc640 1 -- 172.21.15.37:0/4088068758 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0a5d50 con 0x7f3aac15b6b0 2024-04-25T09:02:39.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3ab1d55640 1 -- 172.21.15.37:0/4088068758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac15b6b0 msgr2=0x7f3aac0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3ab1d55640 1 --2- 172.21.15.37:0/4088068758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac15b6b0 0x7f3aac0ca3c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c009e50 tx=0x7f3a9c01ca00 comp rx=0 tx=0).stop 2024-04-25T09:02:39.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3ab1d55640 1 -- 172.21.15.37:0/4088068758 shutdown_connections 2024-04-25T09:02:39.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3ab1d55640 1 --2- 172.21.15.37:0/4088068758 >> v2:172.21.15.203:3300/0 conn(0x7f3aac073010 0x7f3aac0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3ab1d55640 1 --2- 172.21.15.37:0/4088068758 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac15b6b0 0x7f3aac0ca3c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3ab1d55640 1 -- 172.21.15.37:0/4088068758 >> 172.21.15.37:0/4088068758 conn(0x7f3aac06ae60 msgr2=0x7f3aac06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:39.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3ab1d55640 1 -- 172.21.15.37:0/4088068758 shutdown_connections 2024-04-25T09:02:39.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3ab1d55640 1 -- 172.21.15.37:0/4088068758 wait complete. 2024-04-25T09:02:39.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3ab1d55640 1 Processor -- start 2024-04-25T09:02:39.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.550+0000 7f3ab1d55640 1 -- start start 2024-04-25T09:02:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3ab1d55640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aac073010 0x7f3aac0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3ab1d55640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac0cf590 0x7f3aac157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3ab1d55640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aac15b6b0 0x7f3aac1655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3ab1d55640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0ca900 con 0x7f3aac073010 2024-04-25T09:02:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3ab1d55640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0cba30 con 0x7f3aac15b6b0 2024-04-25T09:02:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3ab1d55640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac0cadb0 con 0x7f3aac0cf590 2024-04-25T09:02:39.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aac073010 0x7f3aac0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aac073010 0x7f3aac0c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51544/0 (socket says 172.21.15.37:51544) 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aab7fe640 1 -- 172.21.15.37:0/1524626396 learned_addr learned my addr 172.21.15.37:0/1524626396 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aabfff640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aac15b6b0 0x7f3aac1655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aaaffd640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac0cf590 0x7f3aac157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aab7fe640 1 -- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac0cf590 msgr2=0x7f3aac157af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aab7fe640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac0cf590 0x7f3aac157af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aab7fe640 1 -- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aac15b6b0 msgr2=0x7f3aac1655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aab7fe640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aac15b6b0 0x7f3aac1655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aab7fe640 1 -- 172.21.15.37:0/1524626396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aac0c7d60 con 0x7f3aac073010 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aabfff640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aac15b6b0 0x7f3aac1655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aab7fe640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aac073010 0x7f3aac0c8e00 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0af040 tx=0x7f3a9c0197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aa3fff640 1 -- 172.21.15.37:0/1524626396 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0be020 con 0x7f3aac073010 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aa3fff640 1 -- 172.21.15.37:0/1524626396 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9c0b8640 con 0x7f3aac073010 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3ab1d55640 1 -- 172.21.15.37:0/1524626396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac0c8040 con 0x7f3aac073010 2024-04-25T09:02:39.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aa3fff640 1 -- 172.21.15.37:0/1524626396 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9c0b89f0 con 0x7f3aac073010 2024-04-25T09:02:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3ab1d55640 1 -- 172.21.15.37:0/1524626396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac0c8600 con 0x7f3aac073010 2024-04-25T09:02:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3ab1d55640 1 -- 172.21.15.37:0/1524626396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac04efa0 con 0x7f3aac073010 2024-04-25T09:02:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aa3fff640 1 -- 172.21.15.37:0/1524626396 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3a9c0c1070 con 0x7f3aac073010 2024-04-25T09:02:39.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aa3fff640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a8c066f60 0x7f3a8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:39.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aa3fff640 1 -- 172.21.15.37:0/1524626396 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7f3a9c11f3a0 con 0x7f3aac073010 2024-04-25T09:02:39.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.554+0000 7f3aaaffd640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a8c066f60 0x7f3a8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:39.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.558+0000 7f3aaaffd640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a8c066f60 0x7f3a8c069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0b3ae0 tx=0x7f3a94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:39.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.558+0000 7f3aa3fff640 1 -- 172.21.15.37:0/1524626396 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3a9c016620 con 0x7f3aac073010 2024-04-25T09:02:39.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:39.758+0000 7f3ab1d55640 1 -- 172.21.15.37:0/1524626396 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f3aac073630 con 0x7f3aac073010 2024-04-25T09:02:40.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.106+0000 7f3aa3fff640 1 -- 172.21.15.37:0/1524626396 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v83) v1 ==== 162+0+0 (secure 0 0 0) 0x7f3a9c0f0190 con 0x7f3aac073010 2024-04-25T09:02:40.095 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T09:02:40.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 -- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a8c066f60 msgr2=0x7f3a8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:40.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a8c066f60 0x7f3a8c069420 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0b3ae0 tx=0x7f3a94009040 comp rx=0 tx=0).stop 2024-04-25T09:02:40.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 -- 172.21.15.37:0/1524626396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aac073010 msgr2=0x7f3aac0c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:40.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aac073010 0x7f3aac0c8e00 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0af040 tx=0x7f3a9c0197a0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 -- 172.21.15.37:0/1524626396 shutdown_connections 2024-04-25T09:02:40.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3a8c066f60 0x7f3a8c069420 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3aac15b6b0 0x7f3aac1655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3aac0cf590 0x7f3aac157af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 --2- 172.21.15.37:0/1524626396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3aac073010 0x7f3aac0c8e00 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 -- 172.21.15.37:0/1524626396 >> 172.21.15.37:0/1524626396 conn(0x7f3aac06ae60 msgr2=0x7f3aac0ac730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:40.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 -- 172.21.15.37:0/1524626396 shutdown_connections 2024-04-25T09:02:40.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.110+0000 7f3ab1d55640 1 -- 172.21.15.37:0/1524626396 wait complete. 2024-04-25T09:02:40.125 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.254+0000 7fe7749d4640 1 Processor -- start 2024-04-25T09:02:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.254+0000 7fe7749d4640 1 -- start start 2024-04-25T09:02:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe7749d4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7700b3f50 0x7fe77015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe7749d4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe7700b24a0 0x7fe7700b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe7749d4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7700b2090 con 0x7fe77015a700 2024-04-25T09:02:40.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe7749d4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe77015f200 con 0x7fe7700b24a0 2024-04-25T09:02:40.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe7749d4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700b43e0 con 0x7fe7700b3f50 2024-04-25T09:02:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe76dd9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7700b3f50 0x7fe77015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe76ed9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe7700b24a0 0x7fe7700b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe76dd9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7700b3f50 0x7fe77015ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:44284/0 (socket says 172.21.15.37:44284) 2024-04-25T09:02:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe76dd9b640 1 -- 172.21.15.37:0/1685668894 learned_addr learned my addr 172.21.15.37:0/1685668894 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe76dd9b640 1 -- 172.21.15.37:0/1685668894 >> v2:172.21.15.203:3300/0 conn(0x7fe7700b24a0 msgr2=0x7fe7700b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe76dd9b640 1 --2- 172.21.15.37:0/1685668894 >> v2:172.21.15.203:3300/0 conn(0x7fe7700b24a0 0x7fe7700b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe76dd9b640 1 -- 172.21.15.37:0/1685668894 >> v1:172.21.15.203:6789/0 conn(0x7fe77015a700 legacy=0x7fe7700cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe76dd9b640 1 -- 172.21.15.37:0/1685668894 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7700b46c0 con 0x7fe7700b3f50 2024-04-25T09:02:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.258+0000 7fe76dd9b640 1 --2- 172.21.15.37:0/1685668894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7700b3f50 0x7fe77015ecc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe764009870 tx=0x7fe76401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b9ce4d34473c435 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76d59a640 1 -- 172.21.15.37:0/1685668894 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7640b0070 con 0x7fe7700b3f50 2024-04-25T09:02:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76d59a640 1 -- 172.21.15.37:0/1685668894 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7640ab600 con 0x7fe7700b3f50 2024-04-25T09:02:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76d59a640 1 -- 172.21.15.37:0/1685668894 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7640ab9d0 con 0x7fe7700b3f50 2024-04-25T09:02:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 -- 172.21.15.37:0/1685668894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7700b3f50 msgr2=0x7fe77015ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 --2- 172.21.15.37:0/1685668894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7700b3f50 0x7fe77015ecc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe764009870 tx=0x7fe76401c440 comp rx=0 tx=0).stop 2024-04-25T09:02:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 -- 172.21.15.37:0/1685668894 shutdown_connections 2024-04-25T09:02:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 --2- 172.21.15.37:0/1685668894 >> v2:172.21.15.203:3300/0 conn(0x7fe7700b24a0 0x7fe7700b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 --2- 172.21.15.37:0/1685668894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe7700b3f50 0x7fe77015ecc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 -- 172.21.15.37:0/1685668894 >> 172.21.15.37:0/1685668894 conn(0x7fe7700b11a0 msgr2=0x7fe7700b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 -- 172.21.15.37:0/1685668894 shutdown_connections 2024-04-25T09:02:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 -- 172.21.15.37:0/1685668894 wait complete. 2024-04-25T09:02:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 Processor -- start 2024-04-25T09:02:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 -- start start 2024-04-25T09:02:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7700b24a0 0x7fe7701642d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77015a700 0x7fe770153c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe770154190 0x7fe7701529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700cd310 con 0x7fe7700b24a0 2024-04-25T09:02:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700ce600 con 0x7fe77015a700 2024-04-25T09:02:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe7749d4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7700cd490 con 0x7fe770154190 2024-04-25T09:02:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7700b24a0 0x7fe7701642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7700b24a0 0x7fe7701642d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51552/0 (socket says 172.21.15.37:51552) 2024-04-25T09:02:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76e59c640 1 -- 172.21.15.37:0/1069874953 learned_addr learned my addr 172.21.15.37:0/1069874953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76dd9b640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77015a700 0x7fe770153c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76ed9d640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe770154190 0x7fe7701529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76e59c640 1 -- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe770154190 msgr2=0x7fe7701529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76e59c640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe770154190 0x7fe7701529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76e59c640 1 -- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77015a700 msgr2=0x7fe770153c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.262+0000 7fe76e59c640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77015a700 0x7fe770153c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe76e59c640 1 -- 172.21.15.37:0/1069874953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe770162e80 con 0x7fe7700b24a0 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe76ed9d640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe770154190 0x7fe7701529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe76dd9b640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77015a700 0x7fe770153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe76e59c640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7700b24a0 0x7fe7701642d0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c7c0 tx=0x7fe76000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe75effd640 1 -- 172.21.15.37:0/1069874953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760016020 con 0x7fe7700b24a0 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe75effd640 1 -- 172.21.15.37:0/1069874953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe760006d60 con 0x7fe7700b24a0 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe7749d4640 1 -- 172.21.15.37:0/1069874953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe770164b90 con 0x7fe7700b24a0 2024-04-25T09:02:40.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe75effd640 1 -- 172.21.15.37:0/1069874953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760003bc0 con 0x7fe7700b24a0 2024-04-25T09:02:40.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe7749d4640 1 -- 172.21.15.37:0/1069874953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe770165150 con 0x7fe7700b24a0 2024-04-25T09:02:40.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe7749d4640 1 -- 172.21.15.37:0/1069874953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7701545a0 con 0x7fe7700b24a0 2024-04-25T09:02:40.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe75effd640 1 -- 172.21.15.37:0/1069874953 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe76000a850 con 0x7fe7700b24a0 2024-04-25T09:02:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe75effd640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe750066f60 0x7fe750069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe75effd640 1 -- 172.21.15.37:0/1069874953 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fe760084f70 con 0x7fe7700b24a0 2024-04-25T09:02:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe76dd9b640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe750066f60 0x7fe750069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:40.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.266+0000 7fe76dd9b640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe750066f60 0x7fe750069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe7640ad000 tx=0x7fe764008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:40.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.270+0000 7fe75effd640 1 -- 172.21.15.37:0/1069874953 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe7701545a0 con 0x7fe7700b24a0 2024-04-25T09:02:40.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.470+0000 7fe7749d4640 1 -- 172.21.15.37:0/1069874953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe77006f260 con 0x7fe7700b24a0 2024-04-25T09:02:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.470+0000 7fe75effd640 1 -- 172.21.15.37:0/1069874953 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fe760020020 con 0x7fe7700b24a0 2024-04-25T09:02:40.459 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:40.460 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":84,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:02:40.113632+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"c","rank":0,"incarnation":84,"state":"up:replay","state_seq":1,"addr":"172.21.15.37:6835/2017261543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2017261543},{"type":"v1","addr":"172.21.15.37:6835","nonce":2017261543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:40.460 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 84 2024-04-25T09:02:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 -- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe750066f60 msgr2=0x7fe750069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe750066f60 0x7fe750069420 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe7640ad000 tx=0x7fe764008040 comp rx=0 tx=0).stop 2024-04-25T09:02:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 -- 172.21.15.37:0/1069874953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7700b24a0 msgr2=0x7fe7701642d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:40.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7700b24a0 0x7fe7701642d0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fe76000c7c0 tx=0x7fe76000cc80 comp rx=0 tx=0).stop 2024-04-25T09:02:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 -- 172.21.15.37:0/1069874953 shutdown_connections 2024-04-25T09:02:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe750066f60 0x7fe750069420 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe770154190 0x7fe7701529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe77015a700 0x7fe770153c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 --2- 172.21.15.37:0/1069874953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe7700b24a0 0x7fe7701642d0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 -- 172.21.15.37:0/1069874953 >> 172.21.15.37:0/1069874953 conn(0x7fe7700b11a0 msgr2=0x7fe770071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 -- 172.21.15.37:0/1069874953 shutdown_connections 2024-04-25T09:02:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:40.474+0000 7fe7749d4640 1 -- 172.21.15.37:0/1069874953 wait complete. 2024-04-25T09:02:40.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:01:34.915809+0000', 'modified': '2024-04-25T09:02:40.113632+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9135': {'gid': 9135, 'name': 'c', 'rank': 0, 'incarnation': 84, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.37:6835/2017261543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2017261543}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2017261543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:02:40.882 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9135:up:replay 2024-04-25T09:02:41.884 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:42.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.030+0000 7fc598573640 1 Processor -- start 2024-04-25T09:02:42.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.030+0000 7fc598573640 1 -- start start 2024-04-25T09:02:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc598573640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5900b0b00 0x7fc5900cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc598573640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b1ae0 0x7fc590151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc598573640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc590160310 con 0x7fc5900cc650 2024-04-25T09:02:42.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc598573640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59015cfa0 con 0x7fc5900b0b00 2024-04-25T09:02:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc598573640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59015cdf0 con 0x7fc5900b1ae0 2024-04-25T09:02:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc596b10640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b1ae0 0x7fc590151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc59630f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5900b0b00 0x7fc5900cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc596b10640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b1ae0 0x7fc590151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:44296/0 (socket says 172.21.15.37:44296) 2024-04-25T09:02:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc59630f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc5900b0b00 0x7fc5900cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:51158/0 (socket says 172.21.15.37:51158) 2024-04-25T09:02:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc596b10640 1 -- 172.21.15.37:0/3154143755 learned_addr learned my addr 172.21.15.37:0/3154143755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc595b0e640 1 --1- 172.21.15.37:0/3154143755 >> v1:172.21.15.203:6789/0 conn(0x7fc5900cc650 0x7fc5900b13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:46792/0 (socket says 172.21.15.37:46792) 2024-04-25T09:02:42.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc596b10640 1 -- 172.21.15.37:0/3154143755 >> v2:172.21.15.203:3300/0 conn(0x7fc5900b0b00 msgr2=0x7fc5900cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:42.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc596b10640 1 --2- 172.21.15.37:0/3154143755 >> v2:172.21.15.203:3300/0 conn(0x7fc5900b0b00 0x7fc5900cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:42.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc596b10640 1 -- 172.21.15.37:0/3154143755 >> v1:172.21.15.203:6789/0 conn(0x7fc5900cc650 legacy=0x7fc5900b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:42.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.034+0000 7fc596b10640 1 -- 172.21.15.37:0/3154143755 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59015f550 con 0x7fc5900b1ae0 2024-04-25T09:02:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc596b10640 1 --2- 172.21.15.37:0/3154143755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b1ae0 0x7fc590151fd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc588009e50 tx=0x7fc58801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f09d3447ff48762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc59530d640 1 -- 172.21.15.37:0/3154143755 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880a5510 con 0x7fc5900b1ae0 2024-04-25T09:02:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc59530d640 1 -- 172.21.15.37:0/3154143755 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5880a56b0 con 0x7fc5900b1ae0 2024-04-25T09:02:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc59530d640 1 -- 172.21.15.37:0/3154143755 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880a5a80 con 0x7fc5900b1ae0 2024-04-25T09:02:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 -- 172.21.15.37:0/3154143755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b1ae0 msgr2=0x7fc590151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:42.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 --2- 172.21.15.37:0/3154143755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b1ae0 0x7fc590151fd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc588009e50 tx=0x7fc58801c850 comp rx=0 tx=0).stop 2024-04-25T09:02:42.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 -- 172.21.15.37:0/3154143755 shutdown_connections 2024-04-25T09:02:42.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 --2- 172.21.15.37:0/3154143755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b1ae0 0x7fc590151fd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:42.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 --2- 172.21.15.37:0/3154143755 >> v2:172.21.15.203:3300/0 conn(0x7fc5900b0b00 0x7fc5900cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:42.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 -- 172.21.15.37:0/3154143755 >> 172.21.15.37:0/3154143755 conn(0x7fc5900b1f10 msgr2=0x7fc5900b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:42.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 -- 172.21.15.37:0/3154143755 shutdown_connections 2024-04-25T09:02:42.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 -- 172.21.15.37:0/3154143755 wait complete. 2024-04-25T09:02:42.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 Processor -- start 2024-04-25T09:02:42.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 -- start start 2024-04-25T09:02:42.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b0b00 0x7fc5901585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:42.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5900b1ae0 0x7fc59015ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:42.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5900cc650 0x7fc5901564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:42.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5900b1060 con 0x7fc5900cc650 2024-04-25T09:02:42.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5900b0ee0 con 0x7fc5900b1ae0 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc598573640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59015bde0 con 0x7fc5900b0b00 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.038+0000 7fc596b10640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5900cc650 0x7fc5901564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc596b10640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5900cc650 0x7fc5901564a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51562/0 (socket says 172.21.15.37:51562) 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc596b10640 1 -- 172.21.15.37:0/384320288 learned_addr learned my addr 172.21.15.37:0/384320288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc59630f640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b0b00 0x7fc5901585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc595b0e640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5900b1ae0 0x7fc59015ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc596b10640 1 -- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b0b00 msgr2=0x7fc5901585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc596b10640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b0b00 0x7fc5901585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc596b10640 1 -- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5900b1ae0 msgr2=0x7fc59015ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc596b10640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5900b1ae0 0x7fc59015ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc596b10640 1 -- 172.21.15.37:0/384320288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5900ae110 con 0x7fc5900cc650 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc596b10640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5900cc650 0x7fc5901564a0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc588009e50 tx=0x7fc5880195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc582ffd640 1 -- 172.21.15.37:0/384320288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5880a5510 con 0x7fc5900cc650 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc582ffd640 1 -- 172.21.15.37:0/384320288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5880be070 con 0x7fc5900cc650 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc598573640 1 -- 172.21.15.37:0/384320288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5900ad590 con 0x7fc5900cc650 2024-04-25T09:02:42.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc582ffd640 1 -- 172.21.15.37:0/384320288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588009510 con 0x7fc5900cc650 2024-04-25T09:02:42.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc598573640 1 -- 172.21.15.37:0/384320288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5900adb50 con 0x7fc5900cc650 2024-04-25T09:02:42.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc598573640 1 -- 172.21.15.37:0/384320288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59004efa0 con 0x7fc5900cc650 2024-04-25T09:02:42.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc582ffd640 1 -- 172.21.15.37:0/384320288 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc5880b88d0 con 0x7fc5900cc650 2024-04-25T09:02:42.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc582ffd640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 0x7fc574069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:42.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc582ffd640 1 -- 172.21.15.37:0/384320288 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fc588120330 con 0x7fc5900cc650 2024-04-25T09:02:42.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc59630f640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 0x7fc574069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:42.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.042+0000 7fc59630f640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 0x7fc574069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009af0 tx=0x7fc58c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:42.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.046+0000 7fc582ffd640 1 -- 172.21.15.37:0/384320288 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc590152d20 con 0x7fc5900cc650 2024-04-25T09:02:42.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.246+0000 7fc598573640 1 -- 172.21.15.37:0/384320288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5900b6380 con 0x7fc5900cc650 2024-04-25T09:02:42.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.246+0000 7fc582ffd640 1 -- 172.21.15.37:0/384320288 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4942 (secure 0 0 0) 0x7fc5880f1240 con 0x7fc5900cc650 2024-04-25T09:02:42.237 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:42.237 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":86,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:02:41.123965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"c","rank":0,"incarnation":84,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.37:6835/2017261543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2017261543},{"type":"v1","addr":"172.21.15.37:6835","nonce":2017261543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:42.237 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 86 2024-04-25T09:02:42.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 -- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 msgr2=0x7fc574069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:42.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 0x7fc574069420 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009af0 tx=0x7fc58c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 -- 172.21.15.37:0/384320288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5900cc650 msgr2=0x7fc5901564a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5900cc650 0x7fc5901564a0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fc588009e50 tx=0x7fc5880195f0 comp rx=0 tx=0).stop 2024-04-25T09:02:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 -- 172.21.15.37:0/384320288 shutdown_connections 2024-04-25T09:02:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc574066f60 0x7fc574069420 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc5900cc650 0x7fc5901564a0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc5900b1ae0 0x7fc59015ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 --2- 172.21.15.37:0/384320288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc5900b0b00 0x7fc5901585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 -- 172.21.15.37:0/384320288 >> 172.21.15.37:0/384320288 conn(0x7fc5900b1f10 msgr2=0x7fc59015d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:42.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 -- 172.21.15.37:0/384320288 shutdown_connections 2024-04-25T09:02:42.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:42.250+0000 7fc598573640 1 -- 172.21.15.37:0/384320288 wait complete. 2024-04-25T09:02:42.659 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:01:34.915809+0000', 'modified': '2024-04-25T09:02:41.123965+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9135': {'gid': 9135, 'name': 'c', 'rank': 0, 'incarnation': 84, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.37:6835/2017261543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2017261543}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2017261543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:02:42.659 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9135:up:rejoin 2024-04-25T09:02:43.660 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f900a8c5640 1 Processor -- start 2024-04-25T09:02:43.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f900a8c5640 1 -- start start 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f900a8c5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f90040ce510 0x7f90040ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f900a8c5640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90040cee30 con 0x7f9004144ca0 2024-04-25T09:02:43.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f900a8c5640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90040ccc70 con 0x7f90041451f0 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f900a8c5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90040cc2b0 con 0x7f90040ce510 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f90037fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f9004144ca0 0x7f900406e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56586/0 (socket says 172.21.15.37:56586) 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f90037fe640 1 -- 172.21.15.37:0/2720233813 learned_addr learned my addr 172.21.15.37:0/2720233813 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:43.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9003fff640 1 --2- 172.21.15.37:0/2720233813 >> v2:172.21.15.203:3300/0 conn(0x7f90040ce510 0x7f90040ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774284323 0 0) 0x7f90040cee30 con 0x7f9004144ca0 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fec003530 con 0x7f9004144ca0 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3882151099 0 0) 0x7f8fec003530 con 0x7f9004144ca0 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 >> v2:172.21.15.203:3300/0 conn(0x7f90040ce510 msgr2=0x7f90040ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 --2- 172.21.15.37:0/2720233813 >> v2:172.21.15.203:3300/0 conn(0x7f90040ce510 0x7f90040ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 >> v1:172.21.15.203:6789/0 conn(0x7f90041451f0 legacy=0x7f9004142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900406dc40 con 0x7f9004144ca0 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f8ff8001a00 con 0x7f9004144ca0 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 >> v1:172.21.15.37:6789/0 conn(0x7f9004144ca0 legacy=0x7f900406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec003a30 0x7f8fec003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec004810 0x7f8fec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 0x7f8fec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90040cc2b0 con 0x7f8fec0051d0 2024-04-25T09:02:43.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec00c0e0 con 0x7f8fec004810 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec00c2e0 con 0x7f8fec003a30 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9008e62640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 0x7f8fec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9003fff640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec003a30 0x7f8fec003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f90037fe640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec004810 0x7f8fec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9008e62640 1 -- 172.21.15.37:0/2720233813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec003a30 msgr2=0x7f8fec003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9008e62640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fec003a30 0x7f8fec003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9008e62640 1 -- 172.21.15.37:0/2720233813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec004810 msgr2=0x7f8fec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9008e62640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec004810 0x7f8fec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f9008e62640 1 -- 172.21.15.37:0/2720233813 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec003530 con 0x7f8fec0051d0 2024-04-25T09:02:43.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.806+0000 7f90037fe640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec004810 0x7f8fec004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f9008e62640 1 -- 172.21.15.37:0/2720233813 reap_dead start 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f9008e62640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 0x7f8fec00b9d0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc001750 tx=0x7f8ffc01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cab40dbebc803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc0013a0 con 0x7f8fec0051d0 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc0052d0 con 0x7f8fec0051d0 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f9002ffd640 1 -- 172.21.15.37:0/2720233813 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc003820 con 0x7f8fec0051d0 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f900a8c5640 1 -- 172.21.15.37:0/2720233813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 msgr2=0x7f8fec00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f900a8c5640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 0x7f8fec00b9d0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc001750 tx=0x7f8ffc01c3e0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f900a8c5640 1 -- 172.21.15.37:0/2720233813 shutdown_connections 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f900a8c5640 1 --2- 172.21.15.37:0/2720233813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 0x7f8fec00b9d0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f900a8c5640 1 -- 172.21.15.37:0/2720233813 >> 172.21.15.37:0/2720233813 conn(0x7f90040b11a0 msgr2=0x7f90040b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f900a8c5640 1 -- 172.21.15.37:0/2720233813 shutdown_connections 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f900a8c5640 1 -- 172.21.15.37:0/2720233813 wait complete. 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f900a8c5640 1 Processor -- start 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.810+0000 7f900a8c5640 1 -- start start 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f900a8c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 0x7f90041f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f900a8c5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900415b170 0x7f90040abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f900a8c5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040abfe0 0x7f90040b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f900a8c5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90040cc900 con 0x7f8fec0051d0 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f900a8c5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900406e290 con 0x7f90040abfe0 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f900a8c5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90040cca80 con 0x7f900415b170 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9003fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 0x7f90041f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9003fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 0x7f90041f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51592/0 (socket says 172.21.15.37:51592) 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9003fff640 1 -- 172.21.15.37:0/3115725613 learned_addr learned my addr 172.21.15.37:0/3115725613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:43.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f90037fe640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900415b170 0x7f90040abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9008e62640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040abfe0 0x7f90040b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9003fff640 1 -- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900415b170 msgr2=0x7f90040abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9003fff640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900415b170 0x7f90040abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9003fff640 1 -- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040abfe0 msgr2=0x7f90040b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9003fff640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040abfe0 0x7f90040b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9003fff640 1 -- 172.21.15.37:0/3115725613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90040ac8b0 con 0x7f8fec0051d0 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9008e62640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040abfe0 0x7f90040b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9003fff640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 0x7f90041f3a40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4011590 tx=0x7f8ff4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9000ff9640 1 -- 172.21.15.37:0/3115725613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4015870 con 0x7f8fec0051d0 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9000ff9640 1 -- 172.21.15.37:0/3115725613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff4015a10 con 0x7f8fec0051d0 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f900a8c5640 1 -- 172.21.15.37:0/3115725613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90040ae2c0 con 0x7f8fec0051d0 2024-04-25T09:02:43.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9000ff9640 1 -- 172.21.15.37:0/3115725613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4015dc0 con 0x7f8fec0051d0 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f900a8c5640 1 -- 172.21.15.37:0/3115725613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90040ae880 con 0x7f8fec0051d0 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f900a8c5640 1 -- 172.21.15.37:0/3115725613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900404efa0 con 0x7f8fec0051d0 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9000ff9640 1 -- 172.21.15.37:0/3115725613 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8ff4013400 con 0x7f8fec0051d0 2024-04-25T09:02:43.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9000ff9640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ff0066f60 0x7f8ff0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f9000ff9640 1 -- 172.21.15.37:0/3115725613 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7f8ff4085b30 con 0x7f8fec0051d0 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.814+0000 7f90037fe640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ff0066f60 0x7f8ff0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:43.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.818+0000 7f90037fe640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ff0066f60 0x7f8ff0069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8001380 tx=0x7f8ff8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:43.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:43.818+0000 7f9000ff9640 1 -- 172.21.15.37:0/3115725613 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8ff4019030 con 0x7f8fec0051d0 2024-04-25T09:02:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.014+0000 7f900a8c5640 1 -- 172.21.15.37:0/3115725613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f900406ef00 con 0x7f8fec0051d0 2024-04-25T09:02:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f9000ff9640 1 -- 172.21.15.37:0/3115725613 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4942 (secure 0 0 0) 0x7f8ff4056920 con 0x7f8fec0051d0 2024-04-25T09:02:44.005 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:44.005 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":87,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:02:42.135056+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":113,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9135},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9135":{"gid":9135,"name":"c","rank":0,"incarnation":84,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/2017261543","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2017261543},{"type":"v1","addr":"172.21.15.37:6835","nonce":2017261543}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 87 2024-04-25T09:02:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 -- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ff0066f60 msgr2=0x7f8ff0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ff0066f60 0x7f8ff0069420 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8001380 tx=0x7f8ff8009040 comp rx=0 tx=0).stop 2024-04-25T09:02:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 -- 172.21.15.37:0/3115725613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 msgr2=0x7f90041f3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 0x7f90041f3a40 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4011590 tx=0x7f8ff4011a50 comp rx=0 tx=0).stop 2024-04-25T09:02:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 -- 172.21.15.37:0/3115725613 shutdown_connections 2024-04-25T09:02:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8ff0066f60 0x7f8ff0069420 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f90040abfe0 0x7f90040b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f900415b170 0x7f90040abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 --2- 172.21.15.37:0/3115725613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec0051d0 0x7f90041f3a40 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 -- 172.21.15.37:0/3115725613 >> 172.21.15.37:0/3115725613 conn(0x7f90040b11a0 msgr2=0x7f90040a84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 -- 172.21.15.37:0/3115725613 shutdown_connections 2024-04-25T09:02:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.018+0000 7f900a8c5640 1 -- 172.21.15.37:0/3115725613 wait complete. 2024-04-25T09:02:44.427 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:01:34.915809+0000', 'modified': '2024-04-25T09:02:42.135056+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 113, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9135}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9135': {'gid': 9135, 'name': 'c', 'rank': 0, 'incarnation': 84, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.37:6835/2017261543', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2017261543}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2017261543}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:02:44.427 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:02:44.427 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:02:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.578+0000 7fdf80c9f640 1 Processor -- start 2024-04-25T09:02:44.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.578+0000 7fdf80c9f640 1 -- start start 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.578+0000 7fdf80c9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c15ecc0 0x7fdf7c06e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.578+0000 7fdf80c9f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7c0b2090 con 0x7fdf7c15a700 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.578+0000 7fdf80c9f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf7c0b43e0 con 0x7fdf7c0b3f50 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.578+0000 7fdf80c9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0b46c0 con 0x7fdf7c15ecc0 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.578+0000 7fdf7a59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fdf7c15a700 0x7fdf7c0cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56596/0 (socket says 172.21.15.37:56596) 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.578+0000 7fdf7a59c640 1 -- 172.21.15.37:0/2322073838 learned_addr learned my addr 172.21.15.37:0/2322073838 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.578+0000 7fdf7ad9d640 1 --2- 172.21.15.37:0/2322073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c15ecc0 0x7fdf7c06e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.578+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378427033 0 0) 0x7fdf7c0b2090 con 0x7fdf7c15a700 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.578+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf68003530 con 0x7fdf7c15a700 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3558893785 0 0) 0x7fdf68003530 con 0x7fdf7c15a700 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c15ecc0 msgr2=0x7fdf7c06e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 --2- 172.21.15.37:0/2322073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c15ecc0 0x7fdf7c06e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 >> v1:172.21.15.203:6789/0 conn(0x7fdf7c0b3f50 legacy=0x7fdf7c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c0cd6f0 con 0x7fdf7c15a700 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fdf6c001a20 con 0x7fdf7c15a700 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 >> v1:172.21.15.37:6789/0 conn(0x7fdf7c15a700 legacy=0x7fdf7c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 --2- 172.21.15.37:0/2322073838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf68003a60 0x7fdf68003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 --2- 172.21.15.37:0/2322073838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68004810 0x7fdf68004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 --2- 172.21.15.37:0/2322073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c15ecc0 0x7fdf6800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0b46c0 con 0x7fdf68003a60 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68005560 con 0x7fdf68004810 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6800e120 con 0x7fdf7c15ecc0 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/2322073838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf68003a60 0x7fdf68003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/2322073838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68004810 0x7fdf68004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7a59c640 1 -- 172.21.15.37:0/2322073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c15ecc0 msgr2=0x7fdf6800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/2322073838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c15ecc0 0x7fdf6800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7a59c640 1 -- 172.21.15.37:0/2322073838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68004810 msgr2=0x7fdf68004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/2322073838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68004810 0x7fdf68004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7a59c640 1 -- 172.21.15.37:0/2322073838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf68003530 con 0x7fdf68003a60 2024-04-25T09:02:44.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/2322073838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf68003a60 0x7fdf68003e80 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0b2260 tx=0x7fdf6c01d290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3a38dabdeff3b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c00a410 con 0x7fdf68003a60 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c0ad630 con 0x7fdf68003a60 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7959a640 1 -- 172.21.15.37:0/2322073838 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0ada20 con 0x7fdf68003a60 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf80c9f640 1 -- 172.21.15.37:0/2322073838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf68003a60 msgr2=0x7fdf68003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf80c9f640 1 --2- 172.21.15.37:0/2322073838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf68003a60 0x7fdf68003e80 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0b2260 tx=0x7fdf6c01d290 comp rx=0 tx=0).stop 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf7ad9d640 1 -- 172.21.15.37:0/2322073838 reap_dead start 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf80c9f640 1 -- 172.21.15.37:0/2322073838 shutdown_connections 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf80c9f640 1 -- 172.21.15.37:0/2322073838 >> 172.21.15.37:0/2322073838 conn(0x7fdf7c0b11a0 msgr2=0x7fdf7c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf80c9f640 1 -- 172.21.15.37:0/2322073838 shutdown_connections 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.582+0000 7fdf80c9f640 1 -- 172.21.15.37:0/2322073838 wait complete. 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf80c9f640 1 Processor -- start 2024-04-25T09:02:44.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf80c9f640 1 -- start start 2024-04-25T09:02:44.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf80c9f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68003a60 0x7fdf7c14b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf80c9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf7c14b9c0 0x7fdf7c158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf80c9f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c06d8a0 0x7fdf7c06dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf80c9f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0cd310 con 0x7fdf7c14b9c0 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf80c9f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c15f0d0 con 0x7fdf68003a60 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf80c9f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c0cd490 con 0x7fdf7c06d8a0 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf7c14b9c0 0x7fdf7c158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf7c14b9c0 0x7fdf7c158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51610/0 (socket says 172.21.15.37:51610) 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf79d9b640 1 -- 172.21.15.37:0/3029370564 learned_addr learned my addr 172.21.15.37:0/3029370564 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf7ad9d640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c06d8a0 0x7fdf7c06dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68003a60 0x7fdf7c14b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf79d9b640 1 -- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c06d8a0 msgr2=0x7fdf7c06dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c06d8a0 0x7fdf7c06dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf79d9b640 1 -- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68003a60 msgr2=0x7fdf7c14b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68003a60 0x7fdf7c14b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf79d9b640 1 -- 172.21.15.37:0/3029370564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf7c14d250 con 0x7fdf7c14b9c0 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf7ad9d640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c06d8a0 0x7fdf7c06dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68003a60 0x7fdf7c14b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf7c14b9c0 0x7fdf7c158960 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400e500 tx=0x7fdf6400e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf72ffd640 1 -- 172.21.15.37:0/3029370564 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640053a0 con 0x7fdf7c14b9c0 2024-04-25T09:02:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf72ffd640 1 -- 172.21.15.37:0/3029370564 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64018070 con 0x7fdf7c14b9c0 2024-04-25T09:02:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf80c9f640 1 -- 172.21.15.37:0/3029370564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c14b7b0 con 0x7fdf7c14b9c0 2024-04-25T09:02:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf72ffd640 1 -- 172.21.15.37:0/3029370564 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64003820 con 0x7fdf7c14b9c0 2024-04-25T09:02:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.586+0000 7fdf80c9f640 1 -- 172.21.15.37:0/3029370564 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7c14bf80 con 0x7fdf7c14b9c0 2024-04-25T09:02:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf72ffd640 1 -- 172.21.15.37:0/3029370564 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdf640039c0 con 0x7fdf7c14b9c0 2024-04-25T09:02:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf72ffd640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf5c066f90 0x7fdf5c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf72ffd640 1 -- 172.21.15.37:0/3029370564 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fdf64084bc0 con 0x7fdf7c14b9c0 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf5c066f90 0x7fdf5c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf83e0c000 1 Processor -- start 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf83e0c000 1 -- start start 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf5c066f90 0x7fdf5c069450 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0ad000 tx=0x7fdf6c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e801d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e7b79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e7b7f30 0x55ef2e7b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef2e69f5f0 con 0x55ef2e801920 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef2e606da0 con 0x55ef2e7b7f30 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef2e58f150 con 0x55ef2e802240 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e801d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e801d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51616/0 (socket says 172.21.15.37:51616) 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7a59c640 1 -- 172.21.15.37:0/3459983570 learned_addr learned my addr 172.21.15.37:0/3459983570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7ad9d640 1 --2- 172.21.15.37:0/3459983570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e7b7f30 0x55ef2e7b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/3459983570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e7b79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7a59c640 1 -- 172.21.15.37:0/3459983570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 msgr2=0x55ef2e7b79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/3459983570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e7b79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7a59c640 1 -- 172.21.15.37:0/3459983570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e7b7f30 msgr2=0x55ef2e7b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/3459983570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e7b7f30 0x55ef2e7b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7a59c640 1 -- 172.21.15.37:0/3459983570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef2e5a8ca0 con 0x55ef2e801920 2024-04-25T09:02:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/3459983570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e801d00 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0013c0 tx=0x7fdf6c001d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9bc449787a1479 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf5bfff640 1 -- 172.21.15.37:0/3459983570 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0c7070 con 0x55ef2e801920 2024-04-25T09:02:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf5bfff640 1 -- 172.21.15.37:0/3459983570 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c0b3040 con 0x55ef2e801920 2024-04-25T09:02:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.590+0000 7fdf5bfff640 1 -- 172.21.15.37:0/3459983570 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0bd040 con 0x55ef2e801920 2024-04-25T09:02:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- 172.21.15.37:0/3459983570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 msgr2=0x55ef2e801d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/3459983570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e801d00 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0013c0 tx=0x7fdf6c001d20 comp rx=0 tx=0).stop 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- 172.21.15.37:0/3459983570 shutdown_connections 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/3459983570 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e7b7f30 0x55ef2e7b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/3459983570 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e7b79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/3459983570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e801d00 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- 172.21.15.37:0/3459983570 >> 172.21.15.37:0/3459983570 conn(0x55ef2e7be2a0 msgr2=0x55ef2e7b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- 172.21.15.37:0/3459983570 shutdown_connections 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- 172.21.15.37:0/3459983570 wait complete. 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- start start 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 Processor -- start 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e808c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e801920 0x55ef2e8091c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e802240 0x55ef2e7bbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef2e5c1990 con 0x55ef2e7b7f30 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef2e602e40 con 0x55ef2e802240 2024-04-25T09:02:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef2e592900 con 0x55ef2e801920 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 10 client.0 _ll_register_callbacks cb 0x55ef2e6ba1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e808c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e808c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51622/0 (socket says 172.21.15.37:51622) 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf7a59c640 1 -- 172.21.15.37:0/2479331562 learned_addr learned my addr 172.21.15.37:0/2479331562 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf7ad9d640 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e802240 0x55ef2e7bbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e801920 0x55ef2e8091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf7a59c640 1 -- 172.21.15.37:0/2479331562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e801920 msgr2=0x55ef2e8091c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e801920 0x55ef2e8091c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf7a59c640 1 -- 172.21.15.37:0/2479331562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e802240 msgr2=0x55ef2e7bbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e802240 0x55ef2e7bbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf7a59c640 1 -- 172.21.15.37:0/2479331562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef2e5bf630 con 0x55ef2e7b7f30 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e808c80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0bc040 tx=0x7fdf6c01bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1912206105cf5a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf597fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf597fa640 1 -- 172.21.15.37:0/2479331562 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0c7070 con 0x55ef2e7b7f30 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf597fa640 1 -- 172.21.15.37:0/2479331562 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fdf6c0cc640 con 0x55ef2e7b7f30 2024-04-25T09:02:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- 172.21.15.37:0/2479331562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf6c0b3040 con 0x55ef2e7b7f30 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf597fa640 1 -- 172.21.15.37:0/2479331562 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf6c0d9050 con 0x55ef2e7b7f30 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 10 client.9192 fetch_fsmap learned FSMap version 87 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- 172.21.15.37:0/2479331562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55ef2e58f3a0 con 0x55ef2e7b7f30 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf597fa640 1 -- 172.21.15.37:0/2479331562 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c0b3040 con 0x55ef2e7b7f30 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf597fa640 1 -- 172.21.15.37:0/2479331562 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdf6c0dc020 con 0x55ef2e7b7f30 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 10 client.9192 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 10 client.9192 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 20 client.9192 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55ef2e7bd160 0x55ef2e812ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 4 client.9192 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf83e0c000 1 -- 172.21.15.37:0/2479331562 --> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef2e5a2a50 con 0x55ef2e7bd160 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55ef2e7bd160 0x55ef2e812ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55ef2e7bd160 0x55ef2e812ce0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=469ac2b49a5ff4b2 server_cookie=81fe555539317a74 in_seq=0 out_seq=0 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.594+0000 7fdf597fa640 10 client.9192 ms_handle_connect on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf597fa640 1 -- 172.21.15.37:0/2479331562 <== mds.0 v2:172.21.15.37:6834/2017261543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ef2e5a2a50 con 0x55ef2e7bd160 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf597fa640 10 client.9192 handle_command_reply: tid=0 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 1 client.9192 shutdown 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 1 -- 172.21.15.37:0/2479331562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 msgr2=0x55ef2e808c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e808c80 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0bc040 tx=0x7fdf6c01bad0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 1 -- 172.21.15.37:0/2479331562 shutdown_connections 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55ef2e7bd160 0x55ef2e812ce0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e802240 0x55ef2e7bbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e801920 0x55ef2e8091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf597fa640 0 client.9192 ms_handle_reset on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/2479331562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e808c80 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 1 -- 172.21.15.37:0/2479331562 >> 172.21.15.37:0/2479331562 conn(0x55ef2e7be2a0 msgr2=0x55ef2e7b69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 1 -- 172.21.15.37:0/2479331562 shutdown_connections 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 1 -- 172.21.15.37:0/2479331562 wait complete. 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.598+0000 7fdf83e0c000 20 client.9192 trim_cache size 0 max 16384 2024-04-25T09:02:44.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 Processor -- start 2024-04-25T09:02:44.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 -- start start 2024-04-25T09:02:44.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e85c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e801920 0x55ef2e85cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e85d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c1fe660 con 0x55ef2e7b7f30 2024-04-25T09:02:44.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c155d00 con 0x55ef2e801920 2024-04-25T09:02:44.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c1fe060 con 0x55ef2e802240 2024-04-25T09:02:44.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e85c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf7a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e85c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51630/0 (socket says 172.21.15.37:51630) 2024-04-25T09:02:44.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf7a59c640 1 -- 172.21.15.37:0/953671175 learned_addr learned my addr 172.21.15.37:0/953671175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/953671175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e801920 0x55ef2e85cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf7a59c640 1 -- 172.21.15.37:0/953671175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 msgr2=0x55ef2e85d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/953671175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e85d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf7a59c640 1 -- 172.21.15.37:0/953671175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e801920 msgr2=0x55ef2e85cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/953671175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e801920 0x55ef2e85cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf7a59c640 1 -- 172.21.15.37:0/953671175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef2e802070 con 0x55ef2e7b7f30 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/953671175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e801920 0x55ef2e85cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/953671175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e85c9c0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c098bf0 tx=0x7fdf6c0c1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7782a1f164a32724 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf5b7fe640 1 -- 172.21.15.37:0/953671175 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c0bd070 con 0x55ef2e7b7f30 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf5b7fe640 1 -- 172.21.15.37:0/953671175 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c0d1070 con 0x55ef2e7b7f30 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf5b7fe640 1 -- 172.21.15.37:0/953671175 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6c004b70 con 0x55ef2e7b7f30 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 -- 172.21.15.37:0/953671175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 msgr2=0x55ef2e85c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/953671175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e85c9c0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c098bf0 tx=0x7fdf6c0c1000 comp rx=0 tx=0).stop 2024-04-25T09:02:44.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 -- 172.21.15.37:0/953671175 shutdown_connections 2024-04-25T09:02:44.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/953671175 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e85d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/953671175 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e801920 0x55ef2e85cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/953671175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e7b7f30 0x55ef2e85c9c0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 -- 172.21.15.37:0/953671175 >> 172.21.15.37:0/953671175 conn(0x55ef2e7be2a0 msgr2=0x55ef2e7bb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 -- 172.21.15.37:0/953671175 shutdown_connections 2024-04-25T09:02:44.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 -- 172.21.15.37:0/953671175 wait complete. 2024-04-25T09:02:44.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 -- start start 2024-04-25T09:02:44.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.614+0000 7fdf83e0c000 1 Processor -- start 2024-04-25T09:02:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e7b7f30 0x55ef2e80e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e85d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e85da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef2e60b3d0 con 0x55ef2e801920 2024-04-25T09:02:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef2e6c2d50 con 0x55ef2e7b7f30 2024-04-25T09:02:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef2e5a1980 con 0x55ef2e802240 2024-04-25T09:02:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 10 client.0 _ll_register_callbacks cb 0x7fdf68003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e85d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf79d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e85d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51636/0 (socket says 172.21.15.37:51636) 2024-04-25T09:02:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf79d9b640 1 -- 172.21.15.37:0/4123939530 learned_addr learned my addr 172.21.15.37:0/4123939530 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e7b7f30 0x55ef2e80e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf7ad9d640 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e85da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf79d9b640 1 -- 172.21.15.37:0/4123939530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 msgr2=0x55ef2e85da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e85da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf79d9b640 1 -- 172.21.15.37:0/4123939530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e7b7f30 msgr2=0x55ef2e80e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e7b7f30 0x55ef2e80e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf79d9b640 1 -- 172.21.15.37:0/4123939530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef2e688600 con 0x55ef2e801920 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf7ad9d640 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e85da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf79d9b640 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e85d530 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fdf6405d410 tx=0x7fdf640550c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7917c02f2837a0b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf597fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf597fa640 1 -- 172.21.15.37:0/4123939530 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64059380 con 0x55ef2e801920 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 -- 172.21.15.37:0/4123939530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf6c0d1070 con 0x55ef2e801920 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf597fa640 1 -- 172.21.15.37:0/4123939530 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fdf64053530 con 0x55ef2e801920 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf597fa640 1 -- 172.21.15.37:0/4123939530 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf6405d940 con 0x55ef2e801920 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 10 client.9198 fetch_fsmap learned FSMap version 87 2024-04-25T09:02:44.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 -- 172.21.15.37:0/4123939530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x55ef2e59b9b0 con 0x55ef2e801920 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf597fa640 1 -- 172.21.15.37:0/4123939530 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf6c0d1070 con 0x55ef2e801920 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf597fa640 1 -- 172.21.15.37:0/4123939530 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdf64084040 con 0x55ef2e801920 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 10 client.9198 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 10 client.9198 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 20 client.9198 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55ef2e85dfb0 0x55ef2e85e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 4 client.9198 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 -- 172.21.15.37:0/4123939530 --> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ef2e6142e0 con 0x55ef2e85dfb0 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55ef2e85dfb0 0x55ef2e85e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf7a59c640 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55ef2e85dfb0 0x55ef2e85e390 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3244af702da1bd3c server_cookie=e66a9728a62d29b1 in_seq=0 out_seq=0 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf597fa640 10 client.9198 ms_handle_connect on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf597fa640 1 -- 172.21.15.37:0/4123939530 <== mds.0 v2:172.21.15.37:6834/2017261543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55ef2e6142e0 con 0x55ef2e85dfb0 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf597fa640 10 client.9198 handle_command_reply: tid=0 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 client.9198 shutdown 2024-04-25T09:02:44.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 -- 172.21.15.37:0/4123939530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 msgr2=0x55ef2e85d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e85d530 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fdf6405d410 tx=0x7fdf640550c0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 -- 172.21.15.37:0/4123939530 shutdown_connections 2024-04-25T09:02:44.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55ef2e85dfb0 0x55ef2e85e390 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55ef2e802240 0x55ef2e85da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef2e801920 0x55ef2e85d530 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf597fa640 0 client.9198 ms_handle_reset on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 --2- 172.21.15.37:0/4123939530 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55ef2e7b7f30 0x55ef2e80e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 -- 172.21.15.37:0/4123939530 >> 172.21.15.37:0/4123939530 conn(0x55ef2e7be2a0 msgr2=0x55ef2e8096e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 -- 172.21.15.37:0/4123939530 shutdown_connections 2024-04-25T09:02:44.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 1 -- 172.21.15.37:0/4123939530 wait complete. 2024-04-25T09:02:44.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf83e0c000 20 client.9198 trim_cache size 0 max 16384 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 9135, 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 4.5016356230000003, 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 87, 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 113, 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 113, 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 5.4585114380000004 2024-04-25T09:02:44.615 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:02:44.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 -- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf5c066f90 msgr2=0x7fdf5c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf5c066f90 0x7fdf5c069450 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0ad000 tx=0x7fdf6c0a7000 comp rx=0 tx=0).stop 2024-04-25T09:02:44.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 -- 172.21.15.37:0/3029370564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf7c14b9c0 msgr2=0x7fdf7c158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf7c14b9c0 0x7fdf7c158960 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400e500 tx=0x7fdf6400e9c0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 -- 172.21.15.37:0/3029370564 shutdown_connections 2024-04-25T09:02:44.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdf5c066f90 0x7fdf5c069450 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdf7c06d8a0 0x7fdf7c06dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdf7c14b9c0 0x7fdf7c158960 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 --2- 172.21.15.37:0/3029370564 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdf68003a60 0x7fdf7c14b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 -- 172.21.15.37:0/3029370564 >> 172.21.15.37:0/3029370564 conn(0x7fdf7c0b11a0 msgr2=0x7fdf7c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 -- 172.21.15.37:0/3029370564 shutdown_connections 2024-04-25T09:02:44.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.618+0000 7fdf80c9f640 1 -- 172.21.15.37:0/3029370564 wait complete. 2024-04-25T09:02:44.626 DEBUG:teuthology.orchestra.run.smithi037:> sudo daemon-helper kill ceph --cluster ceph -w 2024-04-25T09:02:44.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:44.628 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-04-25T09:02:44.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f82331fb640 1 Processor -- start 2024-04-25T09:02:44.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f82331fb640 1 -- start start 2024-04-25T09:02:44.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f82331fb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f822c155b70 0x7f822c066bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f82331fb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f822c0b2c30 0x7f822c0b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f82331fb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f822c06d870 0x7f822c06dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f82331fb640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822c06f500 con 0x7f822c155b70 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f82331fb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822c15bc50 con 0x7f822c0b2c30 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f82331fb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0cc250 con 0x7f822c06d870 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f8230f97640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f822c155b70 0x7f822c066bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f8230f97640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f822c155b70 0x7f822c066bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51644/0 (socket says 172.21.15.37:51644) 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f8230f97640 1 -- 172.21.15.37:0/669148336 learned_addr learned my addr 172.21.15.37:0/669148336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f822bfff640 1 --2- 172.21.15.37:0/669148336 >> v2:172.21.15.203:3300/0 conn(0x7f822c0b2c30 0x7f822c0b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f8231798640 1 --2- 172.21.15.37:0/669148336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f822c06d870 0x7f822c06dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f8230f97640 1 -- 172.21.15.37:0/669148336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f822c06d870 msgr2=0x7f822c06dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f8230f97640 1 --2- 172.21.15.37:0/669148336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f822c06d870 0x7f822c06dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f8230f97640 1 -- 172.21.15.37:0/669148336 >> v2:172.21.15.203:3300/0 conn(0x7f822c0b2c30 msgr2=0x7f822c0b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f8230f97640 1 --2- 172.21.15.37:0/669148336 >> v2:172.21.15.203:3300/0 conn(0x7f822c0b2c30 0x7f822c0b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.762+0000 7f8230f97640 1 -- 172.21.15.37:0/669148336 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c15bf10 con 0x7f822c155b70 2024-04-25T09:02:44.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f8230f97640 1 --2- 172.21.15.37:0/669148336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f822c155b70 0x7f822c066bc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f822c06f7f0 tx=0x7f821801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20fe63f9b6ed749 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f822b7fe640 1 -- 172.21.15.37:0/669148336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180b0070 con 0x7f822c155b70 2024-04-25T09:02:44.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f822b7fe640 1 -- 172.21.15.37:0/669148336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82180ab600 con 0x7f822c155b70 2024-04-25T09:02:44.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f822b7fe640 1 -- 172.21.15.37:0/669148336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180ab9d0 con 0x7f822c155b70 2024-04-25T09:02:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f82331fb640 1 -- 172.21.15.37:0/669148336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f822c155b70 msgr2=0x7f822c066bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f82331fb640 1 --2- 172.21.15.37:0/669148336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f822c155b70 0x7f822c066bc0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f822c06f7f0 tx=0x7f821801c490 comp rx=0 tx=0).stop 2024-04-25T09:02:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f82331fb640 1 -- 172.21.15.37:0/669148336 shutdown_connections 2024-04-25T09:02:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f82331fb640 1 --2- 172.21.15.37:0/669148336 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f822c06d870 0x7f822c06dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f82331fb640 1 --2- 172.21.15.37:0/669148336 >> v2:172.21.15.203:3300/0 conn(0x7f822c0b2c30 0x7f822c0b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f82331fb640 1 --2- 172.21.15.37:0/669148336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f822c155b70 0x7f822c066bc0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f82331fb640 1 -- 172.21.15.37:0/669148336 >> 172.21.15.37:0/669148336 conn(0x7f822c0cc680 msgr2=0x7f822c0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f82331fb640 1 -- 172.21.15.37:0/669148336 shutdown_connections 2024-04-25T09:02:44.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f82331fb640 1 -- 172.21.15.37:0/669148336 wait complete. 2024-04-25T09:02:44.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f82331fb640 1 Processor -- start 2024-04-25T09:02:44.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.766+0000 7f82331fb640 1 -- start start 2024-04-25T09:02:44.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82331fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f822c06d870 0x7f822c0b0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82331fb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f822c0b2c30 0x7f822c1f4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82331fb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c155b70 0x7f822c1f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82331fb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c1560d0 con 0x7f822c06d870 2024-04-25T09:02:44.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82331fb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c155f50 con 0x7f822c155b70 2024-04-25T09:02:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82331fb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c15c9f0 con 0x7f822c0b2c30 2024-04-25T09:02:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f8230f97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f822c06d870 0x7f822c0b0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f8230f97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f822c06d870 0x7f822c0b0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51650/0 (socket says 172.21.15.37:51650) 2024-04-25T09:02:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f8230f97640 1 -- 172.21.15.37:0/3424731433 learned_addr learned my addr 172.21.15.37:0/3424731433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f822bfff640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f822c0b2c30 0x7f822c1f4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f8231798640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c155b70 0x7f822c1f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f8230f97640 1 -- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f822c0b2c30 msgr2=0x7f822c1f4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f8230f97640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f822c0b2c30 0x7f822c1f4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f8230f97640 1 -- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c155b70 msgr2=0x7f822c1f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f8230f97640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c155b70 0x7f822c1f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f8230f97640 1 -- 172.21.15.37:0/3424731433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822c144fa0 con 0x7f822c06d870 2024-04-25T09:02:44.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f8230f97640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f822c06d870 0x7f822c0b0290 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f82180ae040 tx=0x7f821801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82297fa640 1 -- 172.21.15.37:0/3424731433 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180ba020 con 0x7f822c06d870 2024-04-25T09:02:44.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82297fa640 1 -- 172.21.15.37:0/3424731433 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82180b6420 con 0x7f822c06d870 2024-04-25T09:02:44.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82331fb640 1 -- 172.21.15.37:0/3424731433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822c145280 con 0x7f822c06d870 2024-04-25T09:02:44.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82297fa640 1 -- 172.21.15.37:0/3424731433 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180b0040 con 0x7f822c06d870 2024-04-25T09:02:44.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82331fb640 1 -- 172.21.15.37:0/3424731433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c145840 con 0x7f822c06d870 2024-04-25T09:02:44.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82297fa640 1 -- 172.21.15.37:0/3424731433 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f82180bd020 con 0x7f822c06d870 2024-04-25T09:02:44.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82297fa640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8210066f60 0x7f8210069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.770+0000 7f82297fa640 1 -- 172.21.15.37:0/3424731433 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7f821811f220 con 0x7f822c06d870 2024-04-25T09:02:44.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f822bfff640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8210066f60 0x7f8210069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 Processor -- start 2024-04-25T09:02:44.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 -- start start 2024-04-25T09:02:44.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f822bfff640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8210066f60 0x7f8210069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f822c15ef80 tx=0x7f8220008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f20f0 0x5570956f24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f2a10 0x5570956a8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55709558af40 con 0x5570956a86a0 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570954f2070 con 0x5570956f20f0 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55709547a1a0 con 0x5570956f2a10 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8231798640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8231798640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51658/0 (socket says 172.21.15.37:51658) 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8231798640 1 -- 172.21.15.37:0/3894126500 learned_addr learned my addr 172.21.15.37:0/3894126500 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8230f97640 1 --2- 172.21.15.37:0/3894126500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f20f0 0x5570956f24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f822bfff640 1 --2- 172.21.15.37:0/3894126500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f2a10 0x5570956a8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8231798640 1 -- 172.21.15.37:0/3894126500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f2a10 msgr2=0x5570956a8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8231798640 1 --2- 172.21.15.37:0/3894126500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f2a10 0x5570956a8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8231798640 1 -- 172.21.15.37:0/3894126500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f20f0 msgr2=0x5570956f24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8231798640 1 --2- 172.21.15.37:0/3894126500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f20f0 0x5570956f24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8231798640 1 -- 172.21.15.37:0/3894126500 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557095478510 con 0x5570956a86a0 2024-04-25T09:02:44.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8231798640 1 --2- 172.21.15.37:0/3894126500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a8a80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f82240074f0 tx=0x7f822400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e569b9986a8c6d0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f82167fc640 1 -- 172.21.15.37:0/3894126500 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224015780 con 0x5570956a86a0 2024-04-25T09:02:44.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f82167fc640 1 -- 172.21.15.37:0/3894126500 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822401b070 con 0x5570956a86a0 2024-04-25T09:02:44.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f82167fc640 1 -- 172.21.15.37:0/3894126500 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224015b40 con 0x5570956a86a0 2024-04-25T09:02:44.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f89458d4640 1 Processor -- start 2024-04-25T09:02:44.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f89458d4640 1 -- start start 2024-04-25T09:02:44.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 -- 172.21.15.37:0/3894126500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 msgr2=0x5570956a8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 --2- 172.21.15.37:0/3894126500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a8a80 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f82240074f0 tx=0x7f822400b040 comp rx=0 tx=0).stop 2024-04-25T09:02:44.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 -- 172.21.15.37:0/3894126500 shutdown_connections 2024-04-25T09:02:44.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 --2- 172.21.15.37:0/3894126500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a8a80 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 --2- 172.21.15.37:0/3894126500 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f2a10 0x5570956a8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 --2- 172.21.15.37:0/3894126500 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f20f0 0x5570956f24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 -- 172.21.15.37:0/3894126500 >> 172.21.15.37:0/3894126500 conn(0x5570956a0240 msgr2=0x5570956a2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 -- 172.21.15.37:0/3894126500 shutdown_connections 2024-04-25T09:02:44.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 -- 172.21.15.37:0/3894126500 wait complete. 2024-04-25T09:02:44.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f89458d4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f894015a7e0 0x7f89400af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f89458d4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f89400ae210 0x7f89400b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f89458d4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89400afbe0 con 0x7f894015abc0 2024-04-25T09:02:44.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f89458d4640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89400aeed0 con 0x7f894015a7e0 2024-04-25T09:02:44.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f89458d4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894006ff20 con 0x7f89400ae210 2024-04-25T09:02:44.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f893e7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f894015abc0 0x7f89400adb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56604/0 (socket says 172.21.15.37:56604) 2024-04-25T09:02:44.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f893e7fc640 1 -- 172.21.15.37:0/3921531692 learned_addr learned my addr 172.21.15.37:0/3921531692 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:44.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f893effd640 1 --2- 172.21.15.37:0/3921531692 >> v2:172.21.15.37:3300/0 conn(0x7f894015a7e0 0x7f89400af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f893f7fe640 1 --2- 172.21.15.37:0/3921531692 >> v2:172.21.15.203:3300/0 conn(0x7f89400ae210 0x7f89400b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 -- start start 2024-04-25T09:02:44.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f893dffb640 1 -- 172.21.15.37:0/3921531692 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 850717986 0 0) 0x7f89400afbe0 con 0x7f894015abc0 2024-04-25T09:02:44.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f893dffb640 1 -- 172.21.15.37:0/3921531692 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8928003530 con 0x7f894015abc0 2024-04-25T09:02:44.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f893effd640 1 -- 172.21.15.37:0/3921531692 >> v2:172.21.15.203:3300/0 conn(0x7f89400ae210 msgr2=0x7f89400b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f893effd640 1 --2- 172.21.15.37:0/3921531692 >> v2:172.21.15.203:3300/0 conn(0x7f89400ae210 0x7f89400b3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f893effd640 1 -- 172.21.15.37:0/3921531692 >> v1:172.21.15.37:6789/0 conn(0x7f894015abc0 legacy=0x7f89400adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f893effd640 1 -- 172.21.15.37:0/3921531692 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89400b4900 con 0x7f894015a7e0 2024-04-25T09:02:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.774+0000 7f8236368000 1 Processor -- start 2024-04-25T09:02:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956a86a0 0x5570956f9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f20f0 0x5570956f9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f2a10 0x5570956dfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570954ac6c0 con 0x5570956f2a10 2024-04-25T09:02:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570954ee110 con 0x5570956a86a0 2024-04-25T09:02:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557095494bd0 con 0x5570956f20f0 2024-04-25T09:02:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 10 client.0 _ll_register_callbacks cb 0x55709568f570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8231798640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f2a10 0x5570956dfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8231798640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f2a10 0x5570956dfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51664/0 (socket says 172.21.15.37:51664) 2024-04-25T09:02:44.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8231798640 1 -- 172.21.15.37:0/883646150 learned_addr learned my addr 172.21.15.37:0/883646150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8230f97640 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956a86a0 0x5570956f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f822bfff640 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f20f0 0x5570956f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8231798640 1 -- 172.21.15.37:0/883646150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f20f0 msgr2=0x5570956f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8231798640 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f20f0 0x5570956f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8231798640 1 -- 172.21.15.37:0/883646150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956a86a0 msgr2=0x5570956f9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8231798640 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956a86a0 0x5570956f9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8231798640 1 -- 172.21.15.37:0/883646150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55709547a3b0 con 0x5570956f2a10 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8231798640 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f2a10 0x5570956dfd20 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f8224008010 tx=0x7f8224004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5698f91b4fb273e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f820b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f820b7fe640 1 -- 172.21.15.37:0/883646150 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8224009970 con 0x5570956f2a10 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 -- 172.21.15.37:0/883646150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f822401b070 con 0x5570956f2a10 2024-04-25T09:02:44.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f820b7fe640 1 -- 172.21.15.37:0/883646150 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7f8224029610 con 0x5570956f2a10 2024-04-25T09:02:44.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f820b7fe640 1 -- 172.21.15.37:0/883646150 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8224031050 con 0x5570956f2a10 2024-04-25T09:02:44.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 10 client.9216 fetch_fsmap learned FSMap version 87 2024-04-25T09:02:44.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 -- 172.21.15.37:0/883646150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x557095573e00 con 0x5570956f2a10 2024-04-25T09:02:44.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f893effd640 1 --2- 172.21.15.37:0/3921531692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f894015a7e0 0x7f89400af270 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f89400b30a0 tx=0x7f893001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b78bc526144c3872 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f820b7fe640 1 -- 172.21.15.37:0/883646150 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822401b070 con 0x5570956f2a10 2024-04-25T09:02:44.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f893dffb640 1 -- 172.21.15.37:0/3921531692 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300ab650 con 0x7f894015a7e0 2024-04-25T09:02:44.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f820b7fe640 1 -- 172.21.15.37:0/883646150 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8224034020 con 0x5570956f2a10 2024-04-25T09:02:44.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f893dffb640 1 -- 172.21.15.37:0/3921531692 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89300b0070 con 0x7f894015a7e0 2024-04-25T09:02:44.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 10 client.9216 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T09:02:44.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f893dffb640 1 -- 172.21.15.37:0/3921531692 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89300aba10 con 0x7f894015a7e0 2024-04-25T09:02:44.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 10 client.9216 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:02:44.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 20 client.9216 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:44.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x5570956e0a90 0x5570956e0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 4 client.9216 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:44.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 -- 172.21.15.37:0/883646150 --> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55709548dd60 con 0x5570956e0a90 2024-04-25T09:02:44.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8230f97640 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x5570956e0a90 0x5570956e0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f820b7fe640 10 client.9216 ms_handle_connect on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8230f97640 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x5570956e0a90 0x5570956e0e70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b5d8154e236915a server_cookie=ede9a2e453e7c9ba in_seq=0 out_seq=0 2024-04-25T09:02:44.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f89458d4640 1 -- 172.21.15.37:0/3921531692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f894015a7e0 msgr2=0x7f89400af270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f89458d4640 1 --2- 172.21.15.37:0/3921531692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f894015a7e0 0x7f89400af270 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f89400b30a0 tx=0x7f893001c450 comp rx=0 tx=0).stop 2024-04-25T09:02:44.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f89458d4640 1 -- 172.21.15.37:0/3921531692 shutdown_connections 2024-04-25T09:02:44.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f89458d4640 1 --2- 172.21.15.37:0/3921531692 >> v2:172.21.15.203:3300/0 conn(0x7f89400ae210 0x7f89400b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f89458d4640 1 --2- 172.21.15.37:0/3921531692 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f894015a7e0 0x7f89400af270 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f89458d4640 1 -- 172.21.15.37:0/3921531692 >> 172.21.15.37:0/3921531692 conn(0x7f89400b5830 msgr2=0x7f89400b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f89458d4640 1 -- 172.21.15.37:0/3921531692 shutdown_connections 2024-04-25T09:02:44.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f89458d4640 1 -- 172.21.15.37:0/3921531692 wait complete. 2024-04-25T09:02:44.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f820b7fe640 1 -- 172.21.15.37:0/883646150 <== mds.0 v2:172.21.15.37:6834/2017261543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55709548dd60 con 0x5570956e0a90 2024-04-25T09:02:44.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f820b7fe640 10 client.9216 handle_command_reply: tid=0 2024-04-25T09:02:44.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 client.9216 shutdown 2024-04-25T09:02:44.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 -- 172.21.15.37:0/883646150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f2a10 msgr2=0x5570956dfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f2a10 0x5570956dfd20 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f8224008010 tx=0x7f8224004ac0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 -- 172.21.15.37:0/883646150 shutdown_connections 2024-04-25T09:02:44.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x5570956e0a90 0x5570956e0e70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f2a10 0x5570956dfd20 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f20f0 0x5570956f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f820b7fe640 0 client.9216 ms_handle_reset on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 --2- 172.21.15.37:0/883646150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956a86a0 0x5570956f9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 -- 172.21.15.37:0/883646150 >> 172.21.15.37:0/883646150 conn(0x5570956a0240 msgr2=0x5570956a9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 -- 172.21.15.37:0/883646150 shutdown_connections 2024-04-25T09:02:44.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.778+0000 7f8236368000 1 -- 172.21.15.37:0/883646150 wait complete. 2024-04-25T09:02:44.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f8236368000 20 client.9216 trim_cache size 0 max 16384 2024-04-25T09:02:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f89458d4640 1 Processor -- start 2024-04-25T09:02:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f89458d4640 1 -- start start 2024-04-25T09:02:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f89458d4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89400ae210 0x7f89400b2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f89458d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f894015a7e0 0x7f894013db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f89458d4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894015abc0 0x7f8940140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f89458d4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89400b4060 con 0x7f894015a7e0 2024-04-25T09:02:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f89458d4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89400ae5f0 con 0x7f89400ae210 2024-04-25T09:02:44.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f89458d4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89400b41e0 con 0x7f894015abc0 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f893e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f894015a7e0 0x7f894013db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f893e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f894015a7e0 0x7f894013db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51678/0 (socket says 172.21.15.37:51678) 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f893e7fc640 1 -- 172.21.15.37:0/2358069916 learned_addr learned my addr 172.21.15.37:0/2358069916 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f893f7fe640 1 --2- 172.21.15.37:0/2358069916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894015abc0 0x7f8940140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f893effd640 1 --2- 172.21.15.37:0/2358069916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89400ae210 0x7f89400b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f893e7fc640 1 -- 172.21.15.37:0/2358069916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894015abc0 msgr2=0x7f8940140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f893e7fc640 1 --2- 172.21.15.37:0/2358069916 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f894015abc0 0x7f8940140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f893e7fc640 1 -- 172.21.15.37:0/2358069916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89400ae210 msgr2=0x7f89400b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f893e7fc640 1 --2- 172.21.15.37:0/2358069916 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f89400ae210 0x7f89400b2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f893e7fc640 1 -- 172.21.15.37:0/2358069916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89400b19d0 con 0x7f894015a7e0 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f893e7fc640 1 --2- 172.21.15.37:0/2358069916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f894015a7e0 0x7f894013db90 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f893400c3c0 tx=0x7f893400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f8927fff640 1 -- 172.21.15.37:0/2358069916 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934017070 con 0x7f894015a7e0 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f8927fff640 1 -- 172.21.15.37:0/2358069916 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8934005cf0 con 0x7f894015a7e0 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f89458d4640 1 -- 172.21.15.37:0/2358069916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89400b0e50 con 0x7f894015a7e0 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f8927fff640 1 -- 172.21.15.37:0/2358069916 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89340072f0 con 0x7f894015a7e0 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f89458d4640 1 -- 172.21.15.37:0/2358069916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89400b1410 con 0x7f894015a7e0 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f89458d4640 1 -- 172.21.15.37:0/2358069916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f8940068340 con 0x7f894015a7e0 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.782+0000 7f8927fff640 1 -- 172.21.15.37:0/2358069916 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f893401a070 con 0x7f894015a7e0 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.786+0000 7f8927fff640 1 --2- 172.21.15.37:0/2358069916 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f892c066f60 0x7f892c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.786+0000 7f8927fff640 1 -- 172.21.15.37:0/2358069916 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7f89340846e0 con 0x7f894015a7e0 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.786+0000 7f8927fff640 1 -- 172.21.15.37:0/2358069916 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+447 (secure 0 0 0) 0x7f8934055450 con 0x7f894015a7e0 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.786+0000 7f893effd640 1 --2- 172.21.15.37:0/2358069916 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f892c066f60 0x7f892c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.786+0000 7f89458d4640 1 -- 172.21.15.37:0/2358069916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7f894014a500 con 0x7f894015a7e0 2024-04-25T09:02:44.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.786+0000 7f8927fff640 1 -- 172.21.15.37:0/2358069916 <== mon.0 v2:172.21.15.37:3300/0 7 ==== log(2 entries from seq 1144 at 2024-04-25T09:02:43.133209+0000) v1 ==== 389+0+0 (secure 0 0 0) 0x7f8934082b00 con 0x7f894015a7e0 2024-04-25T09:02:44.788 INFO:teuthology.orchestra.run.smithi037.stdout: cluster: 2024-04-25T09:02:44.788 INFO:teuthology.orchestra.run.smithi037.stdout: id: 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T09:02:44.788 INFO:teuthology.orchestra.run.smithi037.stdout: health: HEALTH_OK 2024-04-25T09:02:44.788 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:44.788 INFO:teuthology.orchestra.run.smithi037.stdout: services: 2024-04-25T09:02:44.788 INFO:teuthology.orchestra.run.smithi037.stdout: mon: 3 daemons, quorum a,b,c (age 10m) 2024-04-25T09:02:44.788 INFO:teuthology.orchestra.run.smithi037.stdout: mgr: x(active, since 10m), standbys: y 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout: mds: 1/1 daemons up, 3 standby 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout: osd: 8 osds: 8 up (since 10m), 8 in (since 10m) 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout: data: 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout: volumes: 1/1 healthy 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout: pools: 3 pools, 97 pgs 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout: objects: 30 objects, 19 MiB 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout: usage: 291 MiB used, 720 GiB / 720 GiB avail 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout: pgs: 97 active+clean 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:43.133209+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T09:02:44.789 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:43.133261+0000 mon.a [INF] Cluster is now healthy 2024-04-25T09:02:44.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.786+0000 7f893effd640 1 --2- 172.21.15.37:0/2358069916 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f892c066f60 0x7f892c069420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f89300af000 tx=0x7f89300a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 Processor -- start 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 -- start start 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f20f0 0x5570956a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f2a10 0x5570956a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c1ff2b0 con 0x5570956a86a0 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0cefe0 con 0x5570956f20f0 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822c15d610 con 0x5570956f2a10 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8230f97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8230f97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51684/0 (socket says 172.21.15.37:51684) 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8230f97640 1 -- 172.21.15.37:0/785850210 learned_addr learned my addr 172.21.15.37:0/785850210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8231798640 1 --2- 172.21.15.37:0/785850210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f2a10 0x5570956a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f822bfff640 1 --2- 172.21.15.37:0/785850210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f20f0 0x5570956a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8230f97640 1 -- 172.21.15.37:0/785850210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f2a10 msgr2=0x5570956a2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8230f97640 1 --2- 172.21.15.37:0/785850210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f2a10 0x5570956a2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8230f97640 1 -- 172.21.15.37:0/785850210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f20f0 msgr2=0x5570956a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8230f97640 1 --2- 172.21.15.37:0/785850210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f20f0 0x5570956a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8230f97640 1 -- 172.21.15.37:0/785850210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570956f2840 con 0x5570956a86a0 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8230f97640 1 --2- 172.21.15.37:0/785850210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a3790 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f821801c970 tx=0x7f82180f2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab30df817e9f970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8215ffb640 1 -- 172.21.15.37:0/785850210 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180f01c0 con 0x5570956a86a0 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8215ffb640 1 -- 172.21.15.37:0/785850210 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82180efac0 con 0x5570956a86a0 2024-04-25T09:02:44.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8215ffb640 1 -- 172.21.15.37:0/785850210 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82180f3c10 con 0x5570956a86a0 2024-04-25T09:02:44.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 -- 172.21.15.37:0/785850210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 msgr2=0x5570956a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 --2- 172.21.15.37:0/785850210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a3790 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f821801c970 tx=0x7f82180f2040 comp rx=0 tx=0).stop 2024-04-25T09:02:44.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 -- 172.21.15.37:0/785850210 shutdown_connections 2024-04-25T09:02:44.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 --2- 172.21.15.37:0/785850210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956f2a10 0x5570956a2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 --2- 172.21.15.37:0/785850210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f20f0 0x5570956a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 --2- 172.21.15.37:0/785850210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956a86a0 0x5570956a3790 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 -- 172.21.15.37:0/785850210 >> 172.21.15.37:0/785850210 conn(0x5570956a0240 msgr2=0x5570956f0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 -- 172.21.15.37:0/785850210 shutdown_connections 2024-04-25T09:02:44.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 -- 172.21.15.37:0/785850210 wait complete. 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 -- start start 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.798+0000 7f8236368000 1 Processor -- start 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956a86a0 0x5570956f7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f20f0 0x5570956f8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f2a10 0x5570956f8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570955ae6a0 con 0x5570956f20f0 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c0ce9e0 con 0x5570956f2a10 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5570954ee110 con 0x5570956a86a0 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 10 client.0 _ll_register_callbacks cb 0x7f8224009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f822bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f20f0 0x5570956f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f822bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f20f0 0x5570956f8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51698/0 (socket says 172.21.15.37:51698) 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f822bfff640 1 -- 172.21.15.37:0/1219837943 learned_addr learned my addr 172.21.15.37:0/1219837943 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8230f97640 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956a86a0 0x5570956f7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8231798640 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f2a10 0x5570956f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f822bfff640 1 -- 172.21.15.37:0/1219837943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956a86a0 msgr2=0x5570956f7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f822bfff640 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956a86a0 0x5570956f7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f822bfff640 1 -- 172.21.15.37:0/1219837943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f2a10 msgr2=0x5570956f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f822bfff640 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f2a10 0x5570956f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f822bfff640 1 -- 172.21.15.37:0/1219837943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557095549c50 con 0x5570956f20f0 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8231798640 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f2a10 0x5570956f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:44.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f822bfff640 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f20f0 0x5570956f8050 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f82200157b0 tx=0x7f82200046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80938badab227815 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f820b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f820b7fe640 1 -- 172.21.15.37:0/1219837943 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8220024990 con 0x5570956f20f0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f820b7fe640 1 -- 172.21.15.37:0/1219837943 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7f822002ed10 con 0x5570956f20f0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 -- 172.21.15.37:0/1219837943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f82180efac0 con 0x5570956f20f0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f820b7fe640 1 -- 172.21.15.37:0/1219837943 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8220024b30 con 0x5570956f20f0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 10 client.9225 fetch_fsmap learned FSMap version 87 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 -- 172.21.15.37:0/1219837943 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x5570954ba120 con 0x5570956f20f0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f820b7fe640 1 -- 172.21.15.37:0/1219837943 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82180efac0 con 0x5570956f20f0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f820b7fe640 1 -- 172.21.15.37:0/1219837943 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8220035050 con 0x5570956f20f0 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 10 client.9225 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 10 client.9225 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 20 client.9225 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:44.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55709570fb10 0x55709570fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 4 client.9225 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 -- 172.21.15.37:0/1219837943 --> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x5570954ff5b0 con 0x55709570fb10 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8230f97640 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55709570fb10 0x55709570fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8230f97640 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55709570fb10 0x55709570fef0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ed9ee54501505 server_cookie=4d8aefd482e46710 in_seq=0 out_seq=0 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f820b7fe640 10 client.9225 ms_handle_connect on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f820b7fe640 1 -- 172.21.15.37:0/1219837943 <== mds.0 v2:172.21.15.37:6834/2017261543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5570954ff5b0 con 0x55709570fb10 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f820b7fe640 10 client.9225 handle_command_reply: tid=0 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 client.9225 shutdown 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 -- 172.21.15.37:0/1219837943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f20f0 msgr2=0x5570956f8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f20f0 0x5570956f8050 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f82200157b0 tx=0x7f82200046d0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.803 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T09:02:44.802+0000 7faee09c0640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T09:02:44.803 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T09:02:44.802+0000 7faee09c0640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T09:02:44.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 -- 172.21.15.37:0/1219837943 shutdown_connections 2024-04-25T09:02:44.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55709570fb10 0x55709570fef0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5570956f2a10 0x5570956f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570956f20f0 0x5570956f8050 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 --2- 172.21.15.37:0/1219837943 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5570956a86a0 0x5570956f7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f820b7fe640 0 client.9225 ms_handle_reset on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 -- 172.21.15.37:0/1219837943 >> 172.21.15.37:0/1219837943 conn(0x5570956a0240 msgr2=0x5570956f1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 -- 172.21.15.37:0/1219837943 shutdown_connections 2024-04-25T09:02:44.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 1 -- 172.21.15.37:0/1219837943 wait complete. 2024-04-25T09:02:44.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.802+0000 7f8236368000 20 client.9225 trim_cache size 0 max 16384 2024-04-25T09:02:44.805 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T09:02:44.802+0000 7faee09c0640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T09:02:44.805 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T09:02:44.802+0000 7faee09c0640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T09:02:44.805 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:02:44.805 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-04-25T09:02:44.805 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "ee695888-91eb-4a32-bf66-a652f165a606", 2024-04-25T09:02:44.805 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-04-25T09:02:44.806 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:02:44.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 -- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8210066f60 msgr2=0x7f8210069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8210066f60 0x7f8210069420 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f822c15ef80 tx=0x7f8220008040 comp rx=0 tx=0).stop 2024-04-25T09:02:44.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 -- 172.21.15.37:0/3424731433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f822c06d870 msgr2=0x7f822c0b0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f822c06d870 0x7f822c0b0290 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f82180ae040 tx=0x7f821801acd0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 -- 172.21.15.37:0/3424731433 shutdown_connections 2024-04-25T09:02:44.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8210066f60 0x7f8210069420 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f822c155b70 0x7f822c1f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f822c0b2c30 0x7f822c1f4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 --2- 172.21.15.37:0/3424731433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f822c06d870 0x7f822c0b0290 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 -- 172.21.15.37:0/3424731433 >> 172.21.15.37:0/3424731433 conn(0x7f822c0cc680 msgr2=0x7f822c149990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 -- 172.21.15.37:0/3424731433 shutdown_connections 2024-04-25T09:02:44.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.806+0000 7f82331fb640 1 -- 172.21.15.37:0/3424731433 wait complete. 2024-04-25T09:02:44.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:44.810 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T09:02:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.934+0000 7fbe715f1640 1 Processor -- start 2024-04-25T09:02:44.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.934+0000 7fbe715f1640 1 -- start start 2024-04-25T09:02:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe715f1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbe6c159bd0 0x7fbe6c0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe715f1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe6c0c9ef0 con 0x7fbe6c0b28c0 2024-04-25T09:02:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe715f1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe6c071b70 con 0x7fbe6c159fb0 2024-04-25T09:02:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe715f1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6c072780 con 0x7fbe6c159bd0 2024-04-25T09:02:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fbe6c0b28c0 0x7fbe6c163960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56606/0 (socket says 172.21.15.37:56606) 2024-04-25T09:02:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6b7fe640 1 -- 172.21.15.37:0/2497722772 learned_addr learned my addr 172.21.15.37:0/2497722772 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:02:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2497722772 >> v2:172.21.15.203:3300/0 conn(0x7fbe6c159bd0 0x7fbe6c0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538291704 0 0) 0x7fbe6c0c9ef0 con 0x7fbe6c0b28c0 2024-04-25T09:02:44.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe58003530 con 0x7fbe6c0b28c0 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3715422487 0 0) 0x7fbe58003530 con 0x7fbe6c0b28c0 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 >> v2:172.21.15.203:3300/0 conn(0x7fbe6c159bd0 msgr2=0x7fbe6c0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 --2- 172.21.15.37:0/2497722772 >> v2:172.21.15.203:3300/0 conn(0x7fbe6c159bd0 0x7fbe6c0c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 >> v1:172.21.15.203:6789/0 conn(0x7fbe6c159fb0 legacy=0x7fbe6c0b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c071e50 con 0x7fbe6c0b28c0 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fbe5c001680 con 0x7fbe6c0b28c0 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 >> v1:172.21.15.37:6789/0 conn(0x7fbe6c0b28c0 legacy=0x7fbe6c163960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 --2- 172.21.15.37:0/2497722772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 0x7fbe58003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 --2- 172.21.15.37:0/2497722772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe58004810 0x7fbe58004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 --2- 172.21.15.37:0/2497722772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe580051d0 0x7fbe5800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c072780 con 0x7fbe58003a30 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5800c100 con 0x7fbe580051d0 2024-04-25T09:02:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5800c300 con 0x7fbe58004810 2024-04-25T09:02:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2497722772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 0x7fbe58003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6b7fe640 1 --2- 172.21.15.37:0/2497722772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe580051d0 0x7fbe5800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6affd640 1 -- 172.21.15.37:0/2497722772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe58004810 msgr2=0x7fbe58004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2497722772 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe58004810 0x7fbe58004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6affd640 1 -- 172.21.15.37:0/2497722772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe580051d0 msgr2=0x7fbe5800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2497722772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe580051d0 0x7fbe5800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6affd640 1 -- 172.21.15.37:0/2497722772 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe58003530 con 0x7fbe58003a30 2024-04-25T09:02:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6b7fe640 1 --2- 172.21.15.37:0/2497722772 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe580051d0 0x7fbe5800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:44.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.938+0000 7fbe6b7fe640 1 -- 172.21.15.37:0/2497722772 reap_dead start 2024-04-25T09:02:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2497722772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 0x7fbe58003e50 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009fc0 tx=0x7fbe6401cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88748ed896cc6b98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe640b2070 con 0x7fbe58003a30 2024-04-25T09:02:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe640a5c40 con 0x7fbe58003a30 2024-04-25T09:02:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe69ffb640 1 -- 172.21.15.37:0/2497722772 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64009c10 con 0x7fbe58003a30 2024-04-25T09:02:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 -- 172.21.15.37:0/2497722772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 msgr2=0x7fbe58003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 --2- 172.21.15.37:0/2497722772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 0x7fbe58003e50 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fbe64009fc0 tx=0x7fbe6401cc50 comp rx=0 tx=0).stop 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 -- 172.21.15.37:0/2497722772 shutdown_connections 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 --2- 172.21.15.37:0/2497722772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 0x7fbe58003e50 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 -- 172.21.15.37:0/2497722772 >> 172.21.15.37:0/2497722772 conn(0x7fbe6c06ae60 msgr2=0x7fbe6c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 -- 172.21.15.37:0/2497722772 shutdown_connections 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 -- 172.21.15.37:0/2497722772 wait complete. 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 Processor -- start 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 -- start start 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 0x7fbe6c06db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe6c0c40e0 0x7fbe6c0c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe6c0c4540 0x7fbe6c1f4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0b2330 con 0x7fbe58003a30 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0b21b0 con 0x7fbe6c0c40e0 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe715f1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0b24b0 con 0x7fbe6c0c4540 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 0x7fbe6c06db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 0x7fbe6c06db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51704/0 (socket says 172.21.15.37:51704) 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6affd640 1 -- 172.21.15.37:0/2818479969 learned_addr learned my addr 172.21.15.37:0/2818479969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6b7fe640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe6c0c4540 0x7fbe6c1f4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe6c0c40e0 0x7fbe6c0c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6affd640 1 -- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe6c0c4540 msgr2=0x7fbe6c1f4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe6c0c4540 0x7fbe6c1f4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6affd640 1 -- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe6c0c40e0 msgr2=0x7fbe6c0c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe6c0c40e0 0x7fbe6c0c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6affd640 1 -- 172.21.15.37:0/2818479969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c0b3290 con 0x7fbe58003a30 2024-04-25T09:02:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.942+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 0x7fbe6c06db30 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fbe64098760 tx=0x7fbe64009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe637fe640 1 -- 172.21.15.37:0/2818479969 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe640b2040 con 0x7fbe58003a30 2024-04-25T09:02:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe637fe640 1 -- 172.21.15.37:0/2818479969 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe640a5a40 con 0x7fbe58003a30 2024-04-25T09:02:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe715f1640 1 -- 172.21.15.37:0/2818479969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6c0d0060 con 0x7fbe58003a30 2024-04-25T09:02:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe637fe640 1 -- 172.21.15.37:0/2818479969 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe640adce0 con 0x7fbe58003a30 2024-04-25T09:02:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe715f1640 1 -- 172.21.15.37:0/2818479969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6c0c8ca0 con 0x7fbe58003a30 2024-04-25T09:02:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe637fe640 1 -- 172.21.15.37:0/2818479969 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbe640bf030 con 0x7fbe58003a30 2024-04-25T09:02:44.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe637fe640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe4c066f60 0x7fbe4c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe637fe640 1 -- 172.21.15.37:0/2818479969 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fbe6411f5b0 con 0x7fbe58003a30 2024-04-25T09:02:44.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe4c066f60 0x7fbe4c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe4c066f60 0x7fbe4c069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c09c600 tx=0x7fbe54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe7475e000 1 Processor -- start 2024-04-25T09:02:44.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.946+0000 7fbe7475e000 1 -- start start 2024-04-25T09:02:44.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x563946049db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x563945fffb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x563946000480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563945e4cff0 con 0x5639460499d0 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563945dd5150 con 0x5639460000a0 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563945e49090 con 0x56394604a2f0 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x563946049db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x563946049db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51720/0 (socket says 172.21.15.37:51720) 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 -- 172.21.15.37:0/896181897 learned_addr learned my addr 172.21.15.37:0/896181897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/896181897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x563945fffb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6b7fe640 1 --2- 172.21.15.37:0/896181897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x563946000480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 -- 172.21.15.37:0/896181897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 msgr2=0x563945fffb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 --2- 172.21.15.37:0/896181897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x563945fffb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 -- 172.21.15.37:0/896181897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 msgr2=0x563946000480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 --2- 172.21.15.37:0/896181897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x563946000480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 -- 172.21.15.37:0/896181897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563945fe30d0 con 0x5639460499d0 2024-04-25T09:02:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6b7fe640 1 --2- 172.21.15.37:0/896181897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x563946000480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 --2- 172.21.15.37:0/896181897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x563946049db0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fbe640f7e80 tx=0x7fbe64004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4157385a85e0f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe60ff9640 1 -- 172.21.15.37:0/896181897 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe640a5c10 con 0x5639460499d0 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe60ff9640 1 -- 172.21.15.37:0/896181897 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe640efca0 con 0x5639460499d0 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe60ff9640 1 -- 172.21.15.37:0/896181897 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe640f3df0 con 0x5639460499d0 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- 172.21.15.37:0/896181897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 msgr2=0x563946049db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 --2- 172.21.15.37:0/896181897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x563946049db0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fbe640f7e80 tx=0x7fbe64004a50 comp rx=0 tx=0).stop 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- 172.21.15.37:0/896181897 shutdown_connections 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 --2- 172.21.15.37:0/896181897 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x563946000480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 --2- 172.21.15.37:0/896181897 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x563945fffb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 --2- 172.21.15.37:0/896181897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x563946049db0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- 172.21.15.37:0/896181897 >> 172.21.15.37:0/896181897 conn(0x563945ffc020 msgr2=0x563945ffe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- 172.21.15.37:0/896181897 shutdown_connections 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- 172.21.15.37:0/896181897 wait complete. 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- start start 2024-04-25T09:02:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 Processor -- start 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460000a0 0x56394602e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460499d0 0x56394602ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x56394602f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563945e11250 con 0x5639460000a0 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563945e07690 con 0x5639460499d0 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563945dd5a40 con 0x56394604a2f0 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe7475e000 10 client.0 _ll_register_callbacks cb 0x563945feae90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460000a0 0x56394602e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460000a0 0x56394602e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51732/0 (socket says 172.21.15.37:51732) 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 -- 172.21.15.37:0/965436872 learned_addr learned my addr 172.21.15.37:0/965436872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6b7fe640 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x56394602f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460499d0 0x56394602ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 -- 172.21.15.37:0/965436872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 msgr2=0x56394602f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x56394602f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 -- 172.21.15.37:0/965436872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460499d0 msgr2=0x56394602ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460499d0 0x56394602ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 -- 172.21.15.37:0/965436872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563945efa1c0 con 0x5639460000a0 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460499d0 0x56394602ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe6affd640 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460000a0 0x56394602e7f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fbe640ec100 tx=0x7fbe64000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9dda107637ad2a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.950+0000 7fbe49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe49ffb640 1 -- 172.21.15.37:0/965436872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe640f83b0 con 0x5639460000a0 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe49ffb640 1 -- 172.21.15.37:0/965436872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fbe6411d940 con 0x5639460000a0 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 -- 172.21.15.37:0/965436872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe640efca0 con 0x5639460000a0 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe49ffb640 1 -- 172.21.15.37:0/965436872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe640f3df0 con 0x5639460000a0 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 10 client.9237 fetch_fsmap learned FSMap version 87 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 -- 172.21.15.37:0/965436872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x563945ef6170 con 0x5639460000a0 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe49ffb640 1 -- 172.21.15.37:0/965436872 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe640efca0 con 0x5639460000a0 2024-04-25T09:02:44.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe49ffb640 1 -- 172.21.15.37:0/965436872 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbe6411dce0 con 0x5639460000a0 2024-04-25T09:02:44.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 10 client.9237 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T09:02:44.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 10 client.9237 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:02:44.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 20 client.9237 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:44.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x563946059c80 0x56394605c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 4 client.9237 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:44.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 -- 172.21.15.37:0/965436872 --> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563945de8c60 con 0x563946059c80 2024-04-25T09:02:44.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x563946059c80 0x56394605c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe49ffb640 10 client.9237 ms_handle_connect on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x563946059c80 0x56394605c070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27d18b323ccd7e98 server_cookie=f0d66405532b8344 in_seq=0 out_seq=0 2024-04-25T09:02:44.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe49ffb640 1 -- 172.21.15.37:0/965436872 <== mds.0 v2:172.21.15.37:6834/2017261543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563945de8c60 con 0x563946059c80 2024-04-25T09:02:44.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe49ffb640 10 client.9237 handle_command_reply: tid=0 2024-04-25T09:02:44.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 client.9237 shutdown 2024-04-25T09:02:44.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 -- 172.21.15.37:0/965436872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460000a0 msgr2=0x56394602e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460000a0 0x56394602e7f0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fbe640ec100 tx=0x7fbe64000f50 comp rx=0 tx=0).stop 2024-04-25T09:02:44.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 -- 172.21.15.37:0/965436872 shutdown_connections 2024-04-25T09:02:44.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x563946059c80 0x56394605c070 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x56394602f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460499d0 0x56394602ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 --2- 172.21.15.37:0/965436872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460000a0 0x56394602e7f0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe49ffb640 0 client.9237 ms_handle_reset on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 -- 172.21.15.37:0/965436872 >> 172.21.15.37:0/965436872 conn(0x563945ffc020 msgr2=0x563945ffeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 -- 172.21.15.37:0/965436872 shutdown_connections 2024-04-25T09:02:44.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 1 -- 172.21.15.37:0/965436872 wait complete. 2024-04-25T09:02:44.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.954+0000 7fbe7475e000 20 client.9237 trim_cache size 0 max 16384 2024-04-25T09:02:44.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 Processor -- start 2024-04-25T09:02:44.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 -- start start 2024-04-25T09:02:44.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x5639460a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639460499d0 0x5639460a6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56394604a2f0 0x5639460a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c201ea0 con 0x56394604a2f0 2024-04-25T09:02:44.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c2010a0 con 0x5639460000a0 2024-04-25T09:02:44.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c2004d0 con 0x5639460499d0 2024-04-25T09:02:44.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56394604a2f0 0x5639460a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56394604a2f0 0x5639460a6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51734/0 (socket says 172.21.15.37:51734) 2024-04-25T09:02:44.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6b7fe640 1 -- 172.21.15.37:0/2717236708 learned_addr learned my addr 172.21.15.37:0/2717236708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/2717236708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639460499d0 0x5639460a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2717236708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x5639460a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6b7fe640 1 -- 172.21.15.37:0/2717236708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639460499d0 msgr2=0x5639460a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6b7fe640 1 --2- 172.21.15.37:0/2717236708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639460499d0 0x5639460a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6b7fe640 1 -- 172.21.15.37:0/2717236708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 msgr2=0x5639460a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6b7fe640 1 --2- 172.21.15.37:0/2717236708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x5639460a60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6b7fe640 1 -- 172.21.15.37:0/2717236708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563945e93de0 con 0x56394604a2f0 2024-04-25T09:02:44.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/2717236708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639460499d0 0x5639460a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe6b7fe640 1 --2- 172.21.15.37:0/2717236708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56394604a2f0 0x5639460a6b70 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c001060 tx=0x7fbe5c004bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c929a295743ede8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe4bfff640 1 -- 172.21.15.37:0/2717236708 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c01b070 con 0x56394604a2f0 2024-04-25T09:02:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe4bfff640 1 -- 172.21.15.37:0/2717236708 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c004db0 con 0x56394604a2f0 2024-04-25T09:02:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe4bfff640 1 -- 172.21.15.37:0/2717236708 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c0119a0 con 0x56394604a2f0 2024-04-25T09:02:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 -- 172.21.15.37:0/2717236708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56394604a2f0 msgr2=0x5639460a6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 --2- 172.21.15.37:0/2717236708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56394604a2f0 0x5639460a6b70 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c001060 tx=0x7fbe5c004bc0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 -- 172.21.15.37:0/2717236708 shutdown_connections 2024-04-25T09:02:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 --2- 172.21.15.37:0/2717236708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56394604a2f0 0x5639460a6b70 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 --2- 172.21.15.37:0/2717236708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5639460499d0 0x5639460a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 --2- 172.21.15.37:0/2717236708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x5639460a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 -- 172.21.15.37:0/2717236708 >> 172.21.15.37:0/2717236708 conn(0x563945ffc020 msgr2=0x563946056fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 -- 172.21.15.37:0/2717236708 shutdown_connections 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 -- 172.21.15.37:0/2717236708 wait complete. 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.970+0000 7fbe7475e000 1 -- start start 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 Processor -- start 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x56394605c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x56394605c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x56394605ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563945de7e00 con 0x5639460499d0 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c200aa0 con 0x5639460000a0 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563945e4c300 con 0x56394604a2f0 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 10 client.0 _ll_register_callbacks cb 0x563945fcb040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x56394605c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x56394605c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51746/0 (socket says 172.21.15.37:51746) 2024-04-25T09:02:44.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6a7fc640 1 -- 172.21.15.37:0/2017075924 learned_addr learned my addr 172.21.15.37:0/2017075924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6b7fe640 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x56394605ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x56394605c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6a7fc640 1 -- 172.21.15.37:0/2017075924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 msgr2=0x56394605ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x56394605ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6a7fc640 1 -- 172.21.15.37:0/2017075924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 msgr2=0x56394605c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x56394605c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6a7fc640 1 -- 172.21.15.37:0/2017075924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563945dee8e0 con 0x5639460499d0 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6b7fe640 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x56394605ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6a7fc640 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x56394605c8c0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fbe54001580 tx=0x7fbe54007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c28262bcf23fbaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe49ffb640 1 -- 172.21.15.37:0/2017075924 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5401e040 con 0x5639460499d0 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 -- 172.21.15.37:0/2017075924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe5c004db0 con 0x5639460499d0 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe49ffb640 1 -- 172.21.15.37:0/2017075924 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fbe5400ec30 con 0x5639460499d0 2024-04-25T09:02:44.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe49ffb640 1 -- 172.21.15.37:0/2017075924 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe54034050 con 0x5639460499d0 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 10 client.9243 fetch_fsmap learned FSMap version 87 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 -- 172.21.15.37:0/2017075924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x563945ea68f0 con 0x5639460499d0 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe49ffb640 1 -- 172.21.15.37:0/2017075924 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c004db0 con 0x5639460499d0 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe49ffb640 1 -- 172.21.15.37:0/2017075924 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbe54037020 con 0x5639460499d0 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 10 client.9243 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 10 client.9243 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 20 client.9243 populate_metadata read hostname 'smithi037' 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x56394605d9a0 0x5639460b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 4 client.9243 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 -- 172.21.15.37:0/2017075924 --> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563945e5a530 con 0x56394605d9a0 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x56394605d9a0 0x5639460b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe6affd640 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x56394605d9a0 0x5639460b1f60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49a043704ad7207e server_cookie=7084cf04fc55221b in_seq=0 out_seq=0 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe49ffb640 10 client.9243 ms_handle_connect on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe49ffb640 1 -- 172.21.15.37:0/2017075924 <== mds.0 v2:172.21.15.37:6834/2017261543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x563945e5a530 con 0x56394605d9a0 2024-04-25T09:02:44.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe49ffb640 10 client.9243 handle_command_reply: tid=0 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 client.9243 shutdown 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 -- 172.21.15.37:0/2017075924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 msgr2=0x56394605c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x56394605c8c0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fbe54001580 tx=0x7fbe54007b20 comp rx=0 tx=0).stop 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 -- 172.21.15.37:0/2017075924 shutdown_connections 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x56394605d9a0 0x5639460b1f60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56394604a2f0 0x56394605ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5639460499d0 0x56394605c8c0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe49ffb640 0 client.9243 ms_handle_reset on v2:172.21.15.37:6834/2017261543 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 --2- 172.21.15.37:0/2017075924 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5639460000a0 0x56394605c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 -- 172.21.15.37:0/2017075924 >> 172.21.15.37:0/2017075924 conn(0x563945ffc020 msgr2=0x5639460480d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 -- 172.21.15.37:0/2017075924 shutdown_connections 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 1 -- 172.21.15.37:0/2017075924 wait complete. 2024-04-25T09:02:44.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe7475e000 20 client.9243 trim_cache size 0 max 16384 2024-04-25T09:02:44.978 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:02:44.978 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:02:44.978 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-04-25T09:02:44.978 INFO:teuthology.orchestra.run.smithi037.stdout: "ee695888-91eb-4a32-bf66-a652f165a606": { 2024-04-25T09:02:44.978 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-04-25T09:02:44.978 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "ee695888-91eb-4a32-bf66-a652f165a606", 2024-04-25T09:02:44.978 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,repair" 2024-04-25T09:02:44.978 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:02:44.978 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:02:44.978 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:02:44.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 -- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe4c066f60 msgr2=0x7fbe4c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe4c066f60 0x7fbe4c069420 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c09c600 tx=0x7fbe54009040 comp rx=0 tx=0).stop 2024-04-25T09:02:44.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 -- 172.21.15.37:0/2818479969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 msgr2=0x7fbe6c06db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:44.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 0x7fbe6c06db30 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fbe64098760 tx=0x7fbe64009750 comp rx=0 tx=0).stop 2024-04-25T09:02:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 -- 172.21.15.37:0/2818479969 shutdown_connections 2024-04-25T09:02:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbe4c066f60 0x7fbe4c069420 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbe6c0c4540 0x7fbe6c1f4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbe6c0c40e0 0x7fbe6c0c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 --2- 172.21.15.37:0/2818479969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe58003a30 0x7fbe6c06db30 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 -- 172.21.15.37:0/2818479969 >> 172.21.15.37:0/2818479969 conn(0x7fbe6c06ae60 msgr2=0x7fbe6c0aee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 -- 172.21.15.37:0/2818479969 shutdown_connections 2024-04-25T09:02:44.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:44.974+0000 7fbe715f1640 1 -- 172.21.15.37:0/2818479969 wait complete. 2024-04-25T09:02:44.981 INFO:tasks.cephfs.filesystem:scrub status for tag:ee695888-91eb-4a32-bf66-a652f165a606 - {'path': '/', 'tag': 'ee695888-91eb-4a32-bf66-a652f165a606', 'options': 'recursive,repair'} 2024-04-25T09:02:46.107 INFO:tasks.ceph.mds.c.smithi037.stderr:2024-04-25T09:02:46.118+0000 7faee49c8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:02:46.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:46.154+0000 7f8927fff640 1 -- 172.21.15.37:0/2358069916 <== mon.0 v2:172.21.15.37:3300/0 8 ==== log(8 entries from seq 1 at 2024-04-25T09:02:44.805528+0000) v1 ==== 1407+0+0 (secure 0 0 0) 0x7f8934052df0 con 0x7f894015a7e0 2024-04-25T09:02:46.142 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:44.805528+0000 mds.c [INF] scrub queued for path: / 2024-04-25T09:02:46.142 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:44.805531+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T09:02:46.142 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:44.805536+0000 mds.c [INF] scrub summary: active paths [/] 2024-04-25T09:02:46.142 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:44.806216+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-04-25T09:02:46.142 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:44.806224+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-04-25T09:02:46.142 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:44.806845+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-04-25T09:02:46.142 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:44.806858+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-04-25T09:02:46.142 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:44.807566+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-04-25T09:02:50.385 INFO:teuthology.orchestra.run.smithi037.stdout:2024-04-25T09:02:49.165706+0000 mds.c [INF] scrub summary: idle 2024-04-25T09:02:50.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:02:50.398+0000 7f8927fff640 1 -- 172.21.15.37:0/2358069916 <== mon.0 v2:172.21.15.37:3300/0 9 ==== log(1 entries from seq 9 at 2024-04-25T09:02:49.165706+0000) v1 ==== 189+0+0 (secure 0 0 0) 0x7f8934053050 con 0x7f894015a7e0 2024-04-25T09:02:56.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:56.741 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:14.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:14.983 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T09:03:15.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.133+0000 7fda1e728640 1 Processor -- start 2024-04-25T09:03:15.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.133+0000 7fda1e728640 1 -- start start 2024-04-25T09:03:15.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1e728640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda180d07f0 0x7fda180d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1e728640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1806be60 con 0x7fda180d1cd0 2024-04-25T09:03:15.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1e728640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda1815de60 con 0x7fda1815da80 2024-04-25T09:03:15.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1e728640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda1806c440 con 0x7fda180d07f0 2024-04-25T09:03:15.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1ccc5640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fda180d1cd0 0x7fda1806c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42644/0 (socket says 172.21.15.37:42644) 2024-04-25T09:03:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1ccc5640 1 -- 172.21.15.37:0/550937127 learned_addr learned my addr 172.21.15.37:0/550937127 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:15.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda177fe640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda180d07f0 0x7fda180d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926648990 0 0) 0x7fda1806be60 con 0x7fda180d1cd0 2024-04-25T09:03:15.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda04003530 con 0x7fda180d1cd0 2024-04-25T09:03:15.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2396274702 0 0) 0x7fda04003530 con 0x7fda180d1cd0 2024-04-25T09:03:15.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda180d07f0 msgr2=0x7fda180d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda180d07f0 0x7fda180d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 >> v1:172.21.15.203:6789/0 conn(0x7fda1815da80 legacy=0x7fda180d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda180d2450 con 0x7fda180d1cd0 2024-04-25T09:03:15.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fda10001a00 con 0x7fda180d1cd0 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 >> v1:172.21.15.37:6789/0 conn(0x7fda180d1cd0 legacy=0x7fda1806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda04003a60 0x7fda04003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda180d07f0 0x7fda04004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda04004d50 0x7fda040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda1806c440 con 0x7fda04004d50 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0400ba80 con 0x7fda04003a60 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0400bc80 con 0x7fda180d07f0 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda04004d50 0x7fda040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda17fff640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda04003a60 0x7fda04003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda177fe640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda180d07f0 0x7fda04004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1ccc5640 1 -- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda180d07f0 msgr2=0x7fda04004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda180d07f0 0x7fda04004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1ccc5640 1 -- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda04003a60 msgr2=0x7fda04003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda04003a60 0x7fda04003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda1ccc5640 1 -- 172.21.15.37:0/550937127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda04003530 con 0x7fda04004d50 2024-04-25T09:03:15.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.137+0000 7fda17fff640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda04003a60 0x7fda04003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda04004d50 0x7fda040051d0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fda1000a800 tx=0x7fda1001fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba8b1bef1b9d4e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1000a450 con 0x7fda04004d50 2024-04-25T09:03:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda100aa6b0 con 0x7fda04004d50 2024-04-25T09:03:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda16ffd640 1 -- 172.21.15.37:0/550937127 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda100aaaa0 con 0x7fda04004d50 2024-04-25T09:03:15.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 -- 172.21.15.37:0/550937127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda04004d50 msgr2=0x7fda040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 --2- 172.21.15.37:0/550937127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda04004d50 0x7fda040051d0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fda1000a800 tx=0x7fda1001fa60 comp rx=0 tx=0).stop 2024-04-25T09:03:15.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1ccc5640 1 -- 172.21.15.37:0/550937127 reap_dead start 2024-04-25T09:03:15.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 -- 172.21.15.37:0/550937127 shutdown_connections 2024-04-25T09:03:15.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 -- 172.21.15.37:0/550937127 >> 172.21.15.37:0/550937127 conn(0x7fda1808b000 msgr2=0x7fda180d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 -- 172.21.15.37:0/550937127 shutdown_connections 2024-04-25T09:03:15.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 -- 172.21.15.37:0/550937127 wait complete. 2024-04-25T09:03:15.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 Processor -- start 2024-04-25T09:03:15.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 -- start start 2024-04-25T09:03:15.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda04004d50 0x7fda18152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda1815eda0 0x7fda18160a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda18154b10 0x7fda1815b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda180d26f0 con 0x7fda18154b10 2024-04-25T09:03:15.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda180d2160 con 0x7fda1815eda0 2024-04-25T09:03:15.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1e728640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda180d2870 con 0x7fda04004d50 2024-04-25T09:03:15.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1ccc5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda18154b10 0x7fda1815b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1ccc5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda18154b10 0x7fda1815b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32940/0 (socket says 172.21.15.37:32940) 2024-04-25T09:03:15.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1ccc5640 1 -- 172.21.15.37:0/2290943469 learned_addr learned my addr 172.21.15.37:0/2290943469 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:15.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda17fff640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda04004d50 0x7fda18152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda177fe640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda1815eda0 0x7fda18160a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1ccc5640 1 -- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda04004d50 msgr2=0x7fda18152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda04004d50 0x7fda18152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1ccc5640 1 -- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda1815eda0 msgr2=0x7fda18160a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda1815eda0 0x7fda18160a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda1ccc5640 1 -- 172.21.15.37:0/2290943469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1815f9b0 con 0x7fda18154b10 2024-04-25T09:03:15.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.141+0000 7fda17fff640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda04004d50 0x7fda18152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:15.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda18154b10 0x7fda1815b020 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fda1000a800 tx=0x7fda04005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda14ff9640 1 -- 172.21.15.37:0/2290943469 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1000a450 con 0x7fda18154b10 2024-04-25T09:03:15.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda14ff9640 1 -- 172.21.15.37:0/2290943469 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda100aa490 con 0x7fda18154b10 2024-04-25T09:03:15.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda1e728640 1 -- 172.21.15.37:0/2290943469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1815fc90 con 0x7fda18154b10 2024-04-25T09:03:15.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda14ff9640 1 -- 172.21.15.37:0/2290943469 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda100011d0 con 0x7fda18154b10 2024-04-25T09:03:15.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda1e728640 1 -- 172.21.15.37:0/2290943469 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda18160250 con 0x7fda18154b10 2024-04-25T09:03:15.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda14ff9640 1 -- 172.21.15.37:0/2290943469 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda100aaa60 con 0x7fda18154b10 2024-04-25T09:03:15.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda14ff9640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9f8066f90 0x7fd9f8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda14ff9640 1 -- 172.21.15.37:0/2290943469 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fda10122b20 con 0x7fda18154b10 2024-04-25T09:03:15.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda17fff640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9f8066f90 0x7fd9f8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda21895000 1 Processor -- start 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda21895000 1 -- start start 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.145+0000 7fda17fff640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9f8066f90 0x7fd9f8069450 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fda08005540 tx=0x7fda08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223d5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d62f0 0x55982238bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x55982238c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598221d8ff0 con 0x5598223d59d0 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559822161150 con 0x55982238c0a0 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598221d5090 con 0x5598223d62f0 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223d5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223d5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32942/0 (socket says 172.21.15.37:32942) 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda17fff640 1 -- 172.21.15.37:0/747913083 learned_addr learned my addr 172.21.15.37:0/747913083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda177fe640 1 --2- 172.21.15.37:0/747913083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d62f0 0x55982238bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/747913083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x55982238c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda17fff640 1 -- 172.21.15.37:0/747913083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d62f0 msgr2=0x55982238bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda17fff640 1 --2- 172.21.15.37:0/747913083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d62f0 0x55982238bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda17fff640 1 -- 172.21.15.37:0/747913083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 msgr2=0x55982238c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda17fff640 1 --2- 172.21.15.37:0/747913083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x55982238c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda17fff640 1 -- 172.21.15.37:0/747913083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982236f0d0 con 0x5598223d59d0 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/747913083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x55982238c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda17fff640 1 --2- 172.21.15.37:0/747913083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223d5db0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fda08001790 tx=0x7fda080055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a5d016a96f1dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda0dffb640 1 -- 172.21.15.37:0/747913083 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08027070 con 0x5598223d59d0 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda0dffb640 1 -- 172.21.15.37:0/747913083 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda08014980 con 0x5598223d59d0 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda0dffb640 1 -- 172.21.15.37:0/747913083 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08014d30 con 0x5598223d59d0 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- 172.21.15.37:0/747913083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 msgr2=0x5598223d5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 --2- 172.21.15.37:0/747913083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223d5db0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fda08001790 tx=0x7fda080055f0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- 172.21.15.37:0/747913083 shutdown_connections 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 --2- 172.21.15.37:0/747913083 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x55982238c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 --2- 172.21.15.37:0/747913083 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d62f0 0x55982238bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 --2- 172.21.15.37:0/747913083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223d5db0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- 172.21.15.37:0/747913083 >> 172.21.15.37:0/747913083 conn(0x559822388020 msgr2=0x55982238a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- 172.21.15.37:0/747913083 shutdown_connections 2024-04-25T09:03:15.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- 172.21.15.37:0/747913083 wait complete. 2024-04-25T09:03:15.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- start start 2024-04-25T09:03:15.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 Processor -- start 2024-04-25T09:03:15.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55982238c0a0 0x5598223ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223bad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598223d62f0 0x5598223bb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55982219d250 con 0x5598223d59d0 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x559822193690 con 0x5598223d62f0 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x559822161a40 con 0x55982238c0a0 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda21895000 10 client.0 _ll_register_callbacks cb 0x559822376e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223bad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223bad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32948/0 (socket says 172.21.15.37:32948) 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda177fe640 1 -- 172.21.15.37:0/641003053 learned_addr learned my addr 172.21.15.37:0/641003053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda17fff640 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55982238c0a0 0x5598223ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598223d62f0 0x5598223bb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda177fe640 1 -- 172.21.15.37:0/641003053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55982238c0a0 msgr2=0x5598223ba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda177fe640 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55982238c0a0 0x5598223ba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda177fe640 1 -- 172.21.15.37:0/641003053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598223d62f0 msgr2=0x5598223bb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda177fe640 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598223d62f0 0x5598223bb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.149+0000 7fda177fe640 1 -- 172.21.15.37:0/641003053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598222861c0 con 0x5598223d59d0 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda177fe640 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223bad30 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fda00016450 tx=0x7fda00016910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c73c9abb9ac1107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fd9f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/641003053 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00004220 con 0x5598223d59d0 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 -- 172.21.15.37:0/641003053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda08014980 con 0x5598223d59d0 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/641003053 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fda000043c0 con 0x5598223d59d0 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/641003053 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda00028050 con 0x5598223d59d0 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 10 client.9255 fetch_fsmap learned FSMap version 87 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 -- 172.21.15.37:0/641003053 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x559822282170 con 0x5598223d59d0 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/641003053 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda08014980 con 0x5598223d59d0 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/641003053 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fda0002b020 con 0x5598223d59d0 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 10 client.9255 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 10 client.9255 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 20 client.9255 populate_metadata read hostname 'smithi037' 2024-04-25T09:03:15.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x5598223e5c80 0x5598223e8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 4 client.9255 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 -- 172.21.15.37:0/641003053 --> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559822174c60 con 0x5598223e5c80 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda17fff640 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x5598223e5c80 0x5598223e8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda17fff640 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x5598223e5c80 0x5598223e8070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9f2ad53aedb8655 server_cookie=bf0e186d7095e951 in_seq=0 out_seq=0 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fd9f6ffd640 10 client.9255 ms_handle_connect on v2:172.21.15.37:6834/2017261543 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/641003053 <== mds.0 v2:172.21.15.37:6834/2017261543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559822174c60 con 0x5598223e5c80 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fd9f6ffd640 10 client.9255 handle_command_reply: tid=0 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 client.9255 shutdown 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 -- 172.21.15.37:0/641003053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 msgr2=0x5598223bad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223bad30 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fda00016450 tx=0x7fda00016910 comp rx=0 tx=0).stop 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 -- 172.21.15.37:0/641003053 shutdown_connections 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x5598223e5c80 0x5598223e8070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5598223d62f0 0x5598223bb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d59d0 0x5598223bad30 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 --2- 172.21.15.37:0/641003053 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55982238c0a0 0x5598223ba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fd9f6ffd640 0 client.9255 ms_handle_reset on v2:172.21.15.37:6834/2017261543 2024-04-25T09:03:15.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 -- 172.21.15.37:0/641003053 >> 172.21.15.37:0/641003053 conn(0x559822388020 msgr2=0x55982238aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 -- 172.21.15.37:0/641003053 shutdown_connections 2024-04-25T09:03:15.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 1 -- 172.21.15.37:0/641003053 wait complete. 2024-04-25T09:03:15.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.153+0000 7fda21895000 20 client.9255 trim_cache size 0 max 16384 2024-04-25T09:03:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 Processor -- start 2024-04-25T09:03:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 -- start start 2024-04-25T09:03:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x559822431ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d59d0 0x559822432420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x559822432960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda18200640 con 0x5598223d62f0 2024-04-25T09:03:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda181ff7a0 con 0x55982238c0a0 2024-04-25T09:03:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda181540c0 con 0x5598223d59d0 2024-04-25T09:03:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda1ccc5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x559822432960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda1ccc5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x559822432960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32960/0 (socket says 172.21.15.37:32960) 2024-04-25T09:03:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda1ccc5640 1 -- 172.21.15.37:0/4034808925 learned_addr learned my addr 172.21.15.37:0/4034808925 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda17fff640 1 --2- 172.21.15.37:0/4034808925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x559822431ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda177fe640 1 --2- 172.21.15.37:0/4034808925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d59d0 0x559822432420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda1ccc5640 1 -- 172.21.15.37:0/4034808925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d59d0 msgr2=0x559822432420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/4034808925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d59d0 0x559822432420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda1ccc5640 1 -- 172.21.15.37:0/4034808925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 msgr2=0x559822431ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/4034808925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x559822431ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda1ccc5640 1 -- 172.21.15.37:0/4034808925 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55982221fde0 con 0x5598223d62f0 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda17fff640 1 --2- 172.21.15.37:0/4034808925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x559822431ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/4034808925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x559822432960 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fda1001f890 tx=0x7fda100f39d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a976d93ccd8d61f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda0d7fa640 1 -- 172.21.15.37:0/4034808925 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda100c4040 con 0x5598223d62f0 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda0d7fa640 1 -- 172.21.15.37:0/4034808925 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda100fb920 con 0x5598223d62f0 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda0d7fa640 1 -- 172.21.15.37:0/4034808925 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda100f7160 con 0x5598223d62f0 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 -- 172.21.15.37:0/4034808925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 msgr2=0x559822432960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 --2- 172.21.15.37:0/4034808925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x559822432960 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fda1001f890 tx=0x7fda100f39d0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 -- 172.21.15.37:0/4034808925 shutdown_connections 2024-04-25T09:03:15.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 --2- 172.21.15.37:0/4034808925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x559822432960 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 --2- 172.21.15.37:0/4034808925 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d59d0 0x559822432420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 --2- 172.21.15.37:0/4034808925 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x559822431ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 -- 172.21.15.37:0/4034808925 >> 172.21.15.37:0/4034808925 conn(0x559822388020 msgr2=0x5598223d40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 -- 172.21.15.37:0/4034808925 shutdown_connections 2024-04-25T09:03:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.169+0000 7fda21895000 1 -- 172.21.15.37:0/4034808925 wait complete. 2024-04-25T09:03:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- start start 2024-04-25T09:03:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 Processor -- start 2024-04-25T09:03:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x5598223e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d59d0 0x5598223e9110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x5598223e9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x559822173e00 con 0x5598223d62f0 2024-04-25T09:03:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda181ff1a0 con 0x55982238c0a0 2024-04-25T09:03:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598221d8300 con 0x5598223d59d0 2024-04-25T09:03:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 10 client.0 _ll_register_callbacks cb 0x559822357040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda1ccc5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x5598223e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda1ccc5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x5598223e9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32968/0 (socket says 172.21.15.37:32968) 2024-04-25T09:03:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda1ccc5640 1 -- 172.21.15.37:0/1565179755 learned_addr learned my addr 172.21.15.37:0/1565179755 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda177fe640 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d59d0 0x5598223e9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda17fff640 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x5598223e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda1ccc5640 1 -- 172.21.15.37:0/1565179755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d59d0 msgr2=0x5598223e9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d59d0 0x5598223e9110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda1ccc5640 1 -- 172.21.15.37:0/1565179755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 msgr2=0x5598223e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x5598223e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda1ccc5640 1 -- 172.21.15.37:0/1565179755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55982217a8e0 con 0x5598223d62f0 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda17fff640 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x5598223e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda1ccc5640 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x5598223e9650 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fda1001f890 tx=0x7fda1001fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926c78dd5536aa44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fd9f6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/1565179755 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda100c4040 con 0x5598223d62f0 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/1565179755 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7fda100aad50 con 0x5598223d62f0 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- 172.21.15.37:0/1565179755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda100fb920 con 0x5598223d62f0 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/1565179755 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda10110050 con 0x5598223d62f0 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 10 client.9261 fetch_fsmap learned FSMap version 87 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- 172.21.15.37:0/1565179755 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x5598222328f0 con 0x5598223d62f0 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/1565179755 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda100fb920 con 0x5598223d62f0 2024-04-25T09:03:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/1565179755 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fda100fb490 con 0x5598223d62f0 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 10 client.9261 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 10 client.9261 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 20 client.9261 populate_metadata read hostname 'smithi037' 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55982243bf50 0x55982243e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 4 client.9261 mds_command: new command op to 9135 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- 172.21.15.37:0/1565179755 --> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5598221e6530 con 0x55982243bf50 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda17fff640 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55982243bf50 0x55982243e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda17fff640 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55982243bf50 0x55982243e340 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3e777e593467247 server_cookie=c4de34ab1d3214db in_seq=0 out_seq=0 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fd9f6ffd640 10 client.9261 ms_handle_connect on v2:172.21.15.37:6834/2017261543 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fd9f6ffd640 1 -- 172.21.15.37:0/1565179755 <== mds.0 v2:172.21.15.37:6834/2017261543 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5598221e6530 con 0x55982243bf50 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fd9f6ffd640 10 client.9261 handle_command_reply: tid=0 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 client.9261 shutdown 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- 172.21.15.37:0/1565179755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 msgr2=0x5598223e9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x5598223e9650 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fda1001f890 tx=0x7fda1001fa60 comp rx=0 tx=0).stop 2024-04-25T09:03:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- 172.21.15.37:0/1565179755 shutdown_connections 2024-04-25T09:03:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.37:6834/2017261543,v1:172.21.15.37:6835/2017261543] conn(0x55982243bf50 0x55982243e340 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5598223d62f0 0x5598223e9650 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5598223d59d0 0x5598223e9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 --2- 172.21.15.37:0/1565179755 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55982238c0a0 0x5598223e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fd9f6ffd640 0 client.9261 ms_handle_reset on v2:172.21.15.37:6834/2017261543 2024-04-25T09:03:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- 172.21.15.37:0/1565179755 >> 172.21.15.37:0/1565179755 conn(0x559822388020 msgr2=0x5598223d4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- 172.21.15.37:0/1565179755 shutdown_connections 2024-04-25T09:03:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.173+0000 7fda21895000 1 -- 172.21.15.37:0/1565179755 wait complete. 2024-04-25T09:03:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda21895000 20 client.9261 trim_cache size 0 max 16384 2024-04-25T09:03:15.173 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:03:15.174 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-04-25T09:03:15.174 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-04-25T09:03:15.174 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 -- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9f8066f90 msgr2=0x7fd9f8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9f8066f90 0x7fd9f8069450 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fda08005540 tx=0x7fda08009040 comp rx=0 tx=0).stop 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 -- 172.21.15.37:0/2290943469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda18154b10 msgr2=0x7fda1815b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda18154b10 0x7fda1815b020 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fda1000a800 tx=0x7fda04005840 comp rx=0 tx=0).stop 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 -- 172.21.15.37:0/2290943469 shutdown_connections 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd9f8066f90 0x7fd9f8069450 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda18154b10 0x7fda1815b020 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda1815eda0 0x7fda18160a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 --2- 172.21.15.37:0/2290943469 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda04004d50 0x7fda18152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 -- 172.21.15.37:0/2290943469 >> 172.21.15.37:0/2290943469 conn(0x7fda1808b000 msgr2=0x7fda180b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 -- 172.21.15.37:0/2290943469 shutdown_connections 2024-04-25T09:03:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.177+0000 7fda1e728640 1 -- 172.21.15.37:0/2290943469 wait complete. 2024-04-25T09:03:15.182 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:03:15.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:03:15.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.425+0000 7f584d4fe640 1 Processor -- start 2024-04-25T09:03:15.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.425+0000 7f584d4fe640 1 -- start start 2024-04-25T09:03:15.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.425+0000 7f584d4fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f584815d970 0x7f58480d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.425+0000 7f584d4fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58480d3de0 0x7f584806d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.425+0000 7f584d4fe640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480cf460 con 0x7f584806d1f0 2024-04-25T09:03:15.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.425+0000 7f584d4fe640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f584806c710 con 0x7f584815d970 2024-04-25T09:03:15.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.425+0000 7f584d4fe640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584806c9f0 con 0x7f58480d3de0 2024-04-25T09:03:15.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.425+0000 7f5846ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f584815d970 0x7f58480d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.425+0000 7f5846ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f584815d970 0x7f58480d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32984/0 (socket says 172.21.15.37:32984) 2024-04-25T09:03:15.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.425+0000 7f5846ffd640 1 -- 172.21.15.37:0/2043517085 learned_addr learned my addr 172.21.15.37:0/2043517085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:15.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.425+0000 7f58477fe640 1 --2- 172.21.15.37:0/2043517085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58480d3de0 0x7f584806d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.429+0000 7f5846ffd640 1 -- 172.21.15.37:0/2043517085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58480d3de0 msgr2=0x7f584806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.429+0000 7f5846ffd640 1 --2- 172.21.15.37:0/2043517085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58480d3de0 0x7f584806d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.429+0000 7f5846ffd640 1 -- 172.21.15.37:0/2043517085 >> v1:172.21.15.203:6789/0 conn(0x7f584806d1f0 legacy=0x7f58480d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.429+0000 7f5846ffd640 1 -- 172.21.15.37:0/2043517085 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58480cf740 con 0x7f584815d970 2024-04-25T09:03:15.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.429+0000 7f5846ffd640 1 --2- 172.21.15.37:0/2043517085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f584815d970 0x7f58480d1340 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f5838009870 tx=0x7f583801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39d56180e8c8414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.429+0000 7f58467fc640 1 -- 172.21.15.37:0/2043517085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58380b0070 con 0x7f584815d970 2024-04-25T09:03:15.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.429+0000 7f58467fc640 1 -- 172.21.15.37:0/2043517085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5838004230 con 0x7f584815d970 2024-04-25T09:03:15.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.429+0000 7f58467fc640 1 -- 172.21.15.37:0/2043517085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5838003620 con 0x7f584815d970 2024-04-25T09:03:15.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 -- 172.21.15.37:0/2043517085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f584815d970 msgr2=0x7f58480d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 --2- 172.21.15.37:0/2043517085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f584815d970 0x7f58480d1340 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f5838009870 tx=0x7f583801c440 comp rx=0 tx=0).stop 2024-04-25T09:03:15.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 -- 172.21.15.37:0/2043517085 shutdown_connections 2024-04-25T09:03:15.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 --2- 172.21.15.37:0/2043517085 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f58480d3de0 0x7f584806d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 --2- 172.21.15.37:0/2043517085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f584815d970 0x7f58480d1340 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 -- 172.21.15.37:0/2043517085 >> 172.21.15.37:0/2043517085 conn(0x7f584809cae0 msgr2=0x7f584808b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 -- 172.21.15.37:0/2043517085 shutdown_connections 2024-04-25T09:03:15.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 -- 172.21.15.37:0/2043517085 wait complete. 2024-04-25T09:03:15.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 Processor -- start 2024-04-25T09:03:15.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 -- start start 2024-04-25T09:03:15.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584806d1f0 0x7f5848156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58480d3de0 0x7f58481f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f584815d970 0x7f58480b3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480cf110 con 0x7f58480d3de0 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584815df60 con 0x7f584815d970 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58480d1a90 con 0x7f584806d1f0 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58480d3de0 0x7f58481f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58480d3de0 0x7f58481f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32986/0 (socket says 172.21.15.37:32986) 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583e7fc640 1 -- 172.21.15.37:0/1908786913 learned_addr learned my addr 172.21.15.37:0/1908786913 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f5846ffd640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584806d1f0 0x7f5848156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f58477fe640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f584815d970 0x7f58480b3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583e7fc640 1 -- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584806d1f0 msgr2=0x7f5848156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583e7fc640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584806d1f0 0x7f5848156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583e7fc640 1 -- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f584815d970 msgr2=0x7f58480b3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583e7fc640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f584815d970 0x7f58480b3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583e7fc640 1 -- 172.21.15.37:0/1908786913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58481f73e0 con 0x7f58480d3de0 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f58477fe640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f584815d970 0x7f58480b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:15.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583e7fc640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58480d3de0 0x7f58481f6cf0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f5830001ec0 tx=0x7f583000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583ffff640 1 -- 172.21.15.37:0/1908786913 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58300052d0 con 0x7f58480d3de0 2024-04-25T09:03:15.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583ffff640 1 -- 172.21.15.37:0/1908786913 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5830005490 con 0x7f58480d3de0 2024-04-25T09:03:15.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 -- 172.21.15.37:0/1908786913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58481f76c0 con 0x7f58480d3de0 2024-04-25T09:03:15.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f583ffff640 1 -- 172.21.15.37:0/1908786913 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830007220 con 0x7f58480d3de0 2024-04-25T09:03:15.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.433+0000 7f584d4fe640 1 -- 172.21.15.37:0/1908786913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5848159290 con 0x7f58480d3de0 2024-04-25T09:03:15.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.437+0000 7f583ffff640 1 -- 172.21.15.37:0/1908786913 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f58300073c0 con 0x7f58480d3de0 2024-04-25T09:03:15.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.437+0000 7f584d4fe640 1 -- 172.21.15.37:0/1908786913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f584804efa0 con 0x7f58480d3de0 2024-04-25T09:03:15.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.437+0000 7f583ffff640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5828066f60 0x7f5828069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.437+0000 7f583ffff640 1 -- 172.21.15.37:0/1908786913 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6253+0+0 (secure 0 0 0) 0x7f58300845c0 con 0x7f58480d3de0 2024-04-25T09:03:15.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.437+0000 7f5846ffd640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5828066f60 0x7f5828069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.437+0000 7f5846ffd640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5828066f60 0x7f5828069420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f58380ad000 tx=0x7f58380a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.441+0000 7f583ffff640 1 -- 172.21.15.37:0/1908786913 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5830054df0 con 0x7f58480d3de0 2024-04-25T09:03:15.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.637+0000 7f584d4fe640 1 -- 172.21.15.37:0/1908786913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f58480cfd30 con 0x7f58480d3de0 2024-04-25T09:03:15.627 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T09:03:15.637+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:03:15.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.641+0000 7f583ffff640 1 -- 172.21.15.37:0/1908786913 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5830058e90 con 0x7f58480d3de0 2024-04-25T09:03:15.631 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:03:15.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 -- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5828066f60 msgr2=0x7f5828069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5828066f60 0x7f5828069420 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f58380ad000 tx=0x7f58380a6000 comp rx=0 tx=0).stop 2024-04-25T09:03:15.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 -- 172.21.15.37:0/1908786913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58480d3de0 msgr2=0x7f58481f6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58480d3de0 0x7f58481f6cf0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f5830001ec0 tx=0x7f583000c820 comp rx=0 tx=0).stop 2024-04-25T09:03:15.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 -- 172.21.15.37:0/1908786913 shutdown_connections 2024-04-25T09:03:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5828066f60 0x7f5828069420 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f584815d970 0x7f58480b3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f58480d3de0 0x7f58481f6cf0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 --2- 172.21.15.37:0/1908786913 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f584806d1f0 0x7f5848156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 -- 172.21.15.37:0/1908786913 >> 172.21.15.37:0/1908786913 conn(0x7f584809cae0 msgr2=0x7f584815ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 -- 172.21.15.37:0/1908786913 shutdown_connections 2024-04-25T09:03:15.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.645+0000 7f584d4fe640 1 -- 172.21.15.37:0/1908786913 wait complete. 2024-04-25T09:03:15.644 INFO:tasks.ceph.mds.c.smithi037.stderr: -1147> 2024-04-25T09:02:44.802+0000 7faee09c0640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-04-25T09:03:15.644 INFO:tasks.ceph.mds.c.smithi037.stderr: -1144> 2024-04-25T09:02:44.802+0000 7faee09c0640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-04-25T09:03:15.644 INFO:tasks.ceph.mds.c.smithi037.stderr: -1104> 2024-04-25T09:02:44.802+0000 7faee09c0640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-04-25T09:03:15.644 INFO:tasks.ceph.mds.c.smithi037.stderr: -1101> 2024-04-25T09:02:44.802+0000 7faee09c0640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-04-25T09:03:15.644 INFO:tasks.ceph.mds.c.smithi037.stderr: -957> 2024-04-25T09:02:46.118+0000 7faee49c8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:03:15.654 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-04-25T09:03:15.682 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T09:03:15.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.693+0000 7f5781336400 1 -- start start 2024-04-25T09:03:15.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.693+0000 7f5781336400 1 Processor -- start 2024-04-25T09:03:15.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c622e9800 0x557c624a2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c622e9400 0x557c624a2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c622e9000 0x557c624a3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c630c2600 con 0x557c622e9400 2024-04-25T09:03:15.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c630c2480 con 0x557c622e9800 2024-04-25T09:03:15.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c630c2300 con 0x557c622e9000 2024-04-25T09:03:15.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577f2f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c622e9400 0x557c624a2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577f2f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c622e9400 0x557c624a2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33022/0 (socket says 172.21.15.37:33022) 2024-04-25T09:03:15.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577f2f4640 1 -- 172.21.15.37:0/1539288852 learned_addr learned my addr 172.21.15.37:0/1539288852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:15.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577faf5640 1 --2- 172.21.15.37:0/1539288852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c622e9800 0x557c624a2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f57802f6640 1 --2- 172.21.15.37:0/1539288852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c622e9000 0x557c624a3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577f2f4640 1 -- 172.21.15.37:0/1539288852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c622e9000 msgr2=0x557c624a3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577f2f4640 1 --2- 172.21.15.37:0/1539288852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c622e9000 0x557c624a3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577f2f4640 1 -- 172.21.15.37:0/1539288852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c622e9800 msgr2=0x557c624a2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577f2f4640 1 --2- 172.21.15.37:0/1539288852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c622e9800 0x557c624a2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577f2f4640 1 -- 172.21.15.37:0/1539288852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c630aa960 con 0x557c622e9400 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577f2f4640 1 --2- 172.21.15.37:0/1539288852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c622e9400 0x557c624a2b00 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x557c63157200 tx=0x557c623ab740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddcff9820acf5b3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577d2f0640 1 -- 172.21.15.37:0/1539288852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557c6231e4e0 con 0x557c622e9400 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577d2f0640 1 -- 172.21.15.37:0/1539288852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557c62332540 con 0x557c622e9400 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577d2f0640 1 -- 172.21.15.37:0/1539288852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557c6231eb60 con 0x557c622e9400 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 -- 172.21.15.37:0/1539288852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c630aad20 con 0x557c622e9400 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 -- 172.21.15.37:0/1539288852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557c630aaf00 con 0x557c622e9400 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 4 waiting for MDS map... 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577d2f0640 1 -- 172.21.15.37:0/1539288852 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x557c63208600 con 0x557c622e9400 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577d2f0640 1 -- 172.21.15.37:0/1539288852 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 88) v1 ==== 2723+0+0 (secure 0 0 0) 0x557c6233f180 con 0x557c622e9400 2024-04-25T09:03:15.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 4 Got MDS map 88 2024-04-25T09:03:15.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 10 main: main 2024-04-25T09:03:15.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 4 main: connecting to RADOS... 2024-04-25T09:03:15.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 Processor -- start 2024-04-25T09:03:15.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 -- start start 2024-04-25T09:03:15.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c4c00 0x557c624a3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c630c5000 0x557c624a3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c630c5400 0x557c624a4100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c630c2180 con 0x557c630c4c00 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c62348480 con 0x557c630c5400 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f5781336400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c62348900 con 0x557c630c5000 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f57802f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c4c00 0x557c624a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f57802f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c4c00 0x557c624a3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33030/0 (socket says 172.21.15.37:33030) 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f57802f6640 1 -- 172.21.15.37:0/1454803278 learned_addr learned my addr 172.21.15.37:0/1454803278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577faf5640 1 --2- 172.21.15.37:0/1454803278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c630c5000 0x557c624a3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f57802f6640 1 -- 172.21.15.37:0/1454803278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c630c5000 msgr2=0x557c624a3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f57802f6640 1 --2- 172.21.15.37:0/1454803278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c630c5000 0x557c624a3b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f57802f6640 1 -- 172.21.15.37:0/1454803278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c630c5400 msgr2=0x557c624a4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577f2f4640 1 --2- 172.21.15.37:0/1454803278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c630c5400 0x557c624a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f57802f6640 1 --2- 172.21.15.37:0/1454803278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c630c5400 0x557c624a4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f57802f6640 1 -- 172.21.15.37:0/1454803278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c630aad20 con 0x557c630c4c00 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f57802f6640 1 --2- 172.21.15.37:0/1454803278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c4c00 0x557c624a3600 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x557c63156960 tx=0x557c630b8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1085fa0f55ed0ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577b2ec640 1 -- 172.21.15.37:0/1454803278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557c6231e9c0 con 0x557c630c4c00 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577b2ec640 1 -- 172.21.15.37:0/1454803278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557c62332c40 con 0x557c630c4c00 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.697+0000 7f577b2ec640 1 -- 172.21.15.37:0/1454803278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557c6231e4e0 con 0x557c630c4c00 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 -- 172.21.15.37:0/1454803278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c4c00 msgr2=0x557c624a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 --2- 172.21.15.37:0/1454803278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c4c00 0x557c624a3600 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x557c63156960 tx=0x557c630b8000 comp rx=0 tx=0).stop 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 -- 172.21.15.37:0/1454803278 shutdown_connections 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 --2- 172.21.15.37:0/1454803278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c630c5400 0x557c624a4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 --2- 172.21.15.37:0/1454803278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c630c5000 0x557c624a3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 --2- 172.21.15.37:0/1454803278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c4c00 0x557c624a3600 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 -- 172.21.15.37:0/1454803278 >> 172.21.15.37:0/1454803278 conn(0x557c622e8c00 msgr2=0x557c62315000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 -- 172.21.15.37:0/1454803278 shutdown_connections 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 -- 172.21.15.37:0/1454803278 wait complete. 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 Processor -- start 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 -- start start 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c630c4c00 0x557c624a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c5000 0x557c624a4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c630c5400 0x557c624a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c630c3200 con 0x557c630c5000 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c630c2180 con 0x557c630c4c00 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c630c3380 con 0x557c630c5400 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f577faf5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c5000 0x557c624a4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f577faf5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c5000 0x557c624a4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33038/0 (socket says 172.21.15.37:33038) 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f577faf5640 1 -- 172.21.15.37:0/2201398339 learned_addr learned my addr 172.21.15.37:0/2201398339 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f57802f6640 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c630c4c00 0x557c624a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f577f2f4640 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c630c5400 0x557c624a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f577faf5640 1 -- 172.21.15.37:0/2201398339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c630c5400 msgr2=0x557c624a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f577faf5640 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c630c5400 0x557c624a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f577faf5640 1 -- 172.21.15.37:0/2201398339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c630c4c00 msgr2=0x557c624a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f577faf5640 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c630c4c00 0x557c624a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f577faf5640 1 -- 172.21.15.37:0/2201398339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c630abc20 con 0x557c630c5000 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f577faf5640 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c5000 0x557c624a4c00 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x557c6326f2f0 tx=0x557c6321ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5779ae9640 1 -- 172.21.15.37:0/2201398339 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557c6231e9c0 con 0x557c630c5000 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5779ae9640 1 -- 172.21.15.37:0/2201398339 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557c62332a80 con 0x557c630c5000 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 -- 172.21.15.37:0/2201398339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557c630abe00 con 0x557c630c5000 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5779ae9640 1 -- 172.21.15.37:0/2201398339 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557c6231e4e0 con 0x557c630c5000 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 1 -- 172.21.15.37:0/2201398339 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c630aa1e0 con 0x557c630c5000 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5781336400 4 main: resolving pool 15 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.701+0000 7f5779ae9640 1 -- 172.21.15.37:0/2201398339 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x557c63204c00 con 0x557c630c5000 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5779ae9640 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x557c63270c00 0x557c624a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5779ae9640 1 -- 172.21.15.37:0/2201398339 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x557c6261aa00 con 0x557c630c5000 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 4 main: creating IoCtx.. 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x557c630c5800 0x557c624a4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 -- 172.21.15.37:0/2201398339 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:1 15.13 15:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e114) v9 -- 0x557c630c5c00 con 0x557c630c5800 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f577f2f4640 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x557c630c5800 0x557c624a4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f57802f6640 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x557c63270c00 0x557c624a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f577f2f4640 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x557c630c5800 0x557c624a4100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f57802f6640 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x557c63270c00 0x557c624a5700 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x557c63156960 tx=0x557c630b82a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f577f2f4640 1 -- 172.21.15.37:0/2201398339 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x557c623f4000 con 0x557c630c5800 2024-04-25T09:03:15.699 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:03:15.699 INFO:teuthology.orchestra.run.smithi037.stdout: "0": { 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: "data": { 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 5, 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: "inotable": { 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: "projected_free": [ 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: "start": 1099511627779, 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: "len": 1099511627773 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: "free": [ 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: "start": 1099511627779, 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: "len": 1099511627773 2024-04-25T09:03:15.700 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi037.stdout: "result": 0 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 -- 172.21.15.37:0/2201398339 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x557c630c5800 msgr2=0x557c624a4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x557c630c5800 0x557c624a4100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 -- 172.21.15.37:0/2201398339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x557c63270c00 msgr2=0x557c624a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x557c63270c00 0x557c624a5700 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x557c63156960 tx=0x557c630b82a0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 -- 172.21.15.37:0/2201398339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c5000 msgr2=0x557c624a4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 --2- 172.21.15.37:0/2201398339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c630c5000 0x557c624a4c00 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x557c6326f2f0 tx=0x557c6321ac00 comp rx=0 tx=0).stop 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f577faf5640 1 -- 172.21.15.37:0/2201398339 reap_dead start 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 -- 172.21.15.37:0/2201398339 shutdown_connections 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 -- 172.21.15.37:0/2201398339 >> 172.21.15.37:0/2201398339 conn(0x557c622e8c00 msgr2=0x557c62315000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 -- 172.21.15.37:0/2201398339 shutdown_connections 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 -- 172.21.15.37:0/2201398339 wait complete. 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 -- 172.21.15.37:0/1539288852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c622e9400 msgr2=0x557c624a2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 --2- 172.21.15.37:0/1539288852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c622e9400 0x557c624a2b00 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x557c63157200 tx=0x557c623ab740 comp rx=0 tx=0).stop 2024-04-25T09:03:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 -- 172.21.15.37:0/1539288852 shutdown_connections 2024-04-25T09:03:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 --2- 172.21.15.37:0/1539288852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557c622e9000 0x557c624a3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 --2- 172.21.15.37:0/1539288852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c622e9400 0x557c624a2b00 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 --2- 172.21.15.37:0/1539288852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557c622e9800 0x557c624a2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.705+0000 7f5781336400 1 -- 172.21.15.37:0/1539288852 >> 172.21.15.37:0/1539288852 conn(0x557c622e9c00 msgr2=0x557c62315800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.709+0000 7f5781336400 1 -- 172.21.15.37:0/1539288852 shutdown_connections 2024-04-25T09:03:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:15.709+0000 7f5781336400 1 -- 172.21.15.37:0/1539288852 wait complete. 2024-04-25T09:03:15.704 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.050061, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-04-25T09:03:15.704 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:03:15.705 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:15.778 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:15.826 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:03:15.827 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:03:15.827 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:15.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:03:15.833 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:03:15.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:15.833 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:03:15.881 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T09:03:15.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:15.882 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:03:15.882 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:15.888 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:15.888 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T09:03:15.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:15.888 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:03:15.937 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:03:15.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:15.938 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:03:15.938 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:15.944 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:03:15.944 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:03:15.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:15.944 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:03:15.993 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:03:15.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:15.994 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:03:15.994 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:16.000 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:16.000 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T09:03:16.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:16.000 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:03:16.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:16.050 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:03:16.051 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:16.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9be5789640 1 Processor -- start 2024-04-25T09:03:16.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9be5789640 1 -- start start 2024-04-25T09:03:16.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9be5789640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00c4590 0x7f9be0070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9be5789640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9be00b0740 0x7f9be00c4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9be5789640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be00c4eb0 con 0x7f9be009a510 2024-04-25T09:03:16.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9be5789640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be0165670 con 0x7f9be00b0740 2024-04-25T09:03:16.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9be5789640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be006ddb0 con 0x7f9be00c4590 2024-04-25T09:03:16.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9bde7fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9be00b0740 0x7f9be00c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9bdeffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00c4590 0x7f9be0070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9bde7fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f9be00b0740 0x7f9be00c4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:51244/0 (socket says 172.21.15.37:51244) 2024-04-25T09:03:16.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9bdeffd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00c4590 0x7f9be0070f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:38994/0 (socket says 172.21.15.37:38994) 2024-04-25T09:03:16.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9bdf7fe640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f9be009a510 0x7f9be006bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:60336/0 (socket says 172.21.15.37:60336) 2024-04-25T09:03:16.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.197+0000 7f9bde7fc640 1 -- 172.21.15.37:0/293657705 learned_addr learned my addr 172.21.15.37:0/293657705 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:16.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9bde7fc640 1 -- 172.21.15.37:0/293657705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00c4590 msgr2=0x7f9be0070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:16.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9bde7fc640 1 --2- 172.21.15.37:0/293657705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00c4590 0x7f9be0070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9bde7fc640 1 -- 172.21.15.37:0/293657705 >> v1:172.21.15.203:6789/0 conn(0x7f9be009a510 legacy=0x7f9be006bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9bde7fc640 1 -- 172.21.15.37:0/293657705 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be006d7f0 con 0x7f9be00b0740 2024-04-25T09:03:16.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9bdeffd640 1 --2- 172.21.15.37:0/293657705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00c4590 0x7f9be0070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:16.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9bde7fc640 1 --2- 172.21.15.37:0/293657705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be00b0740 0x7f9be00c4970 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009870 tx=0x7f9bd401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd2542583dbb464a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9bddffb640 1 -- 172.21.15.37:0/293657705 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd40b0070 con 0x7f9be00b0740 2024-04-25T09:03:16.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9bddffb640 1 -- 172.21.15.37:0/293657705 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd40ab600 con 0x7f9be00b0740 2024-04-25T09:03:16.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9bddffb640 1 -- 172.21.15.37:0/293657705 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd40ab9d0 con 0x7f9be00b0740 2024-04-25T09:03:16.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9be5789640 1 -- 172.21.15.37:0/293657705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be00b0740 msgr2=0x7f9be00c4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9be5789640 1 --2- 172.21.15.37:0/293657705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be00b0740 0x7f9be00c4970 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4009870 tx=0x7f9bd401c440 comp rx=0 tx=0).stop 2024-04-25T09:03:16.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9be5789640 1 -- 172.21.15.37:0/293657705 shutdown_connections 2024-04-25T09:03:16.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.201+0000 7f9be5789640 1 --2- 172.21.15.37:0/293657705 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be00b0740 0x7f9be00c4970 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 --2- 172.21.15.37:0/293657705 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00c4590 0x7f9be0070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 -- 172.21.15.37:0/293657705 >> 172.21.15.37:0/293657705 conn(0x7f9be00b30b0 msgr2=0x7f9be00b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:16.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 -- 172.21.15.37:0/293657705 shutdown_connections 2024-04-25T09:03:16.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 -- 172.21.15.37:0/293657705 wait complete. 2024-04-25T09:03:16.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 Processor -- start 2024-04-25T09:03:16.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 -- start start 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be009a510 0x7f9be015bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00b0740 0x7f9be015d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be00c4590 0x7f9be015db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00d0f20 con 0x7f9be009a510 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be00d0da0 con 0x7f9be00c4590 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be016f440 con 0x7f9be00b0740 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bdeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be009a510 0x7f9be015bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bdeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be009a510 0x7f9be015bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33050/0 (socket says 172.21.15.37:33050) 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bdeffd640 1 -- 172.21.15.37:0/2626596934 learned_addr learned my addr 172.21.15.37:0/2626596934 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bdf7fe640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be00c4590 0x7f9be015db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bde7fc640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00b0740 0x7f9be015d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bdeffd640 1 -- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00b0740 msgr2=0x7f9be015d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bdeffd640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00b0740 0x7f9be015d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bdeffd640 1 -- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be00c4590 msgr2=0x7f9be015db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bdeffd640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be00c4590 0x7f9be015db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bdeffd640 1 -- 172.21.15.37:0/2626596934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be00c9860 con 0x7f9be009a510 2024-04-25T09:03:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bdf7fe640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be00c4590 0x7f9be015db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bdeffd640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be009a510 0x7f9be015bb40 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000e9b0 tx=0x7f9bd000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bc77fe640 1 -- 172.21.15.37:0/2626596934 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd0017070 con 0x7f9be009a510 2024-04-25T09:03:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bc77fe640 1 -- 172.21.15.37:0/2626596934 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd000ca00 con 0x7f9be009a510 2024-04-25T09:03:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 -- 172.21.15.37:0/2626596934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be00c9b40 con 0x7f9be009a510 2024-04-25T09:03:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9bc77fe640 1 -- 172.21.15.37:0/2626596934 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd000cdd0 con 0x7f9be009a510 2024-04-25T09:03:16.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.205+0000 7f9be5789640 1 -- 172.21.15.37:0/2626596934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be00ca100 con 0x7f9be009a510 2024-04-25T09:03:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.209+0000 7f9be5789640 1 -- 172.21.15.37:0/2626596934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be004efa0 con 0x7f9be009a510 2024-04-25T09:03:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.209+0000 7f9bc77fe640 1 -- 172.21.15.37:0/2626596934 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9bd001a070 con 0x7f9be009a510 2024-04-25T09:03:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.209+0000 7f9bc77fe640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bcc066f60 0x7f9bcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.209+0000 7f9bc77fe640 1 -- 172.21.15.37:0/2626596934 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f9bd0085180 con 0x7f9be009a510 2024-04-25T09:03:16.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.209+0000 7f9bde7fc640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bcc066f60 0x7f9bcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.209+0000 7f9bde7fc640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bcc066f60 0x7f9bcc069420 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40ad000 tx=0x7f9bd4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.213+0000 7f9bc77fe640 1 -- 172.21.15.37:0/2626596934 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9bd0055f10 con 0x7f9be009a510 2024-04-25T09:03:16.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.405+0000 7f9be5789640 1 -- 172.21.15.37:0/2626596934 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9be00b0310 con 0x7f9be009a510 2024-04-25T09:03:16.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.405+0000 7f9bc77fe640 1 -- 172.21.15.37:0/2626596934 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) v1 ==== 94+0+4143 (secure 0 0 0) 0x7f9bd0055810 con 0x7f9be009a510 2024-04-25T09:03:16.396 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:16.396 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":88,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:03:15.641847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:03:16.396 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 88 2024-04-25T09:03:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 -- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bcc066f60 msgr2=0x7f9bcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bcc066f60 0x7f9bcc069420 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40ad000 tx=0x7f9bd4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 -- 172.21.15.37:0/2626596934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be009a510 msgr2=0x7f9be015bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be009a510 0x7f9be015bb40 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f9bd000e9b0 tx=0x7f9bd000ee70 comp rx=0 tx=0).stop 2024-04-25T09:03:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 -- 172.21.15.37:0/2626596934 shutdown_connections 2024-04-25T09:03:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9bcc066f60 0x7f9bcc069420 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9be00c4590 0x7f9be015db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9be00b0740 0x7f9be015d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 --2- 172.21.15.37:0/2626596934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9be009a510 0x7f9be015bb40 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 -- 172.21.15.37:0/2626596934 >> 172.21.15.37:0/2626596934 conn(0x7f9be00b30b0 msgr2=0x7f9be0162310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 -- 172.21.15.37:0/2626596934 shutdown_connections 2024-04-25T09:03:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.409+0000 7f9be5789640 1 -- 172.21.15.37:0/2626596934 wait complete. 2024-04-25T09:03:16.417 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:16.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.553+0000 7fb791b9b640 1 Processor -- start 2024-04-25T09:03:16.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.553+0000 7fb791b9b640 1 -- start start 2024-04-25T09:03:16.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb791b9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb78c1451f0 0x7fb78c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb791b9b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78c0ccc70 con 0x7fb78c0ce510 2024-04-25T09:03:16.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb791b9b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb78c06dc40 con 0x7fb78c144ca0 2024-04-25T09:03:16.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb791b9b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c0cc2b0 con 0x7fb78c1451f0 2024-04-25T09:03:16.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78b7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb78c0ce510 0x7fb78c0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42684/0 (socket says 172.21.15.37:42684) 2024-04-25T09:03:16.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78b7fe640 1 -- 172.21.15.37:0/2915067785 learned_addr learned my addr 172.21.15.37:0/2915067785 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:16.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108991993 0 0) 0x7fb78c0ccc70 con 0x7fb78c0ce510 2024-04-25T09:03:16.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb774003530 con 0x7fb78c0ce510 2024-04-25T09:03:16.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78bfff640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb78c1451f0 0x7fb78c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4232770272 0 0) 0x7fb774003530 con 0x7fb78c0ce510 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb78c1451f0 msgr2=0x7fb78c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb78c1451f0 0x7fb78c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 >> v1:172.21.15.203:6789/0 conn(0x7fb78c144ca0 legacy=0x7fb78c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c06df20 con 0x7fb78c0ce510 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fb77c001680 con 0x7fb78c0ce510 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 >> v1:172.21.15.37:6789/0 conn(0x7fb78c0ce510 legacy=0x7fb78c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb774003a60 0x7fb774003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb774004810 0x7fb774004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7740051d0 0x7fb77400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c0cc2b0 con 0x7fb7740051d0 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77400c0d0 con 0x7fb774004810 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77400c2d0 con 0x7fb774003a60 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78bfff640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7740051d0 0x7fb77400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78affd640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb774004810 0x7fb774004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78b7fe640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb774003a60 0x7fb774003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78bfff640 1 -- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb774003a60 msgr2=0x7fb774003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:16.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78bfff640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb774003a60 0x7fb774003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78bfff640 1 -- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb774004810 msgr2=0x7fb774004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:16.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78bfff640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb774004810 0x7fb774004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78bfff640 1 -- 172.21.15.37:0/2915067785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb774003530 con 0x7fb7740051d0 2024-04-25T09:03:16.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.557+0000 7fb78affd640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb774004810 0x7fb774004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:16.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78bfff640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7740051d0 0x7fb77400b9d0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fb784009ed0 tx=0x7fb78401cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4575ccf15c22b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7840b2070 con 0x7fb7740051d0 2024-04-25T09:03:16.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7840a5c00 con 0x7fb7740051d0 2024-04-25T09:03:16.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78a7fc640 1 -- 172.21.15.37:0/2915067785 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784009b20 con 0x7fb7740051d0 2024-04-25T09:03:16.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 -- 172.21.15.37:0/2915067785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7740051d0 msgr2=0x7fb77400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 --2- 172.21.15.37:0/2915067785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7740051d0 0x7fb77400b9d0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fb784009ed0 tx=0x7fb78401cd20 comp rx=0 tx=0).stop 2024-04-25T09:03:16.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78bfff640 1 -- 172.21.15.37:0/2915067785 reap_dead start 2024-04-25T09:03:16.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 -- 172.21.15.37:0/2915067785 shutdown_connections 2024-04-25T09:03:16.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 -- 172.21.15.37:0/2915067785 >> 172.21.15.37:0/2915067785 conn(0x7fb78c0b11a0 msgr2=0x7fb78c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:16.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 -- 172.21.15.37:0/2915067785 shutdown_connections 2024-04-25T09:03:16.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 -- 172.21.15.37:0/2915067785 wait complete. 2024-04-25T09:03:16.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 Processor -- start 2024-04-25T09:03:16.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 -- start start 2024-04-25T09:03:16.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7740051d0 0x7fb78c15b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb78c13d660 0x7fb78c13dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78c0c4410 0x7fb78c0c4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c0cc900 con 0x7fb78c13d660 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c06e290 con 0x7fb78c0c4410 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb78c142680 con 0x7fb7740051d0 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb78c13d660 0x7fb78c13dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb78c13d660 0x7fb78c13dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33080/0 (socket says 172.21.15.37:33080) 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78affd640 1 -- 172.21.15.37:0/3995273650 learned_addr learned my addr 172.21.15.37:0/3995273650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78b7fe640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7740051d0 0x7fb78c15b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78bfff640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78c0c4410 0x7fb78c0c4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78affd640 1 -- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7740051d0 msgr2=0x7fb78c15b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78affd640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7740051d0 0x7fb78c15b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78affd640 1 -- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78c0c4410 msgr2=0x7fb78c0c4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78affd640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78c0c4410 0x7fb78c0c4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78affd640 1 -- 172.21.15.37:0/3995273650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb78c13e130 con 0x7fb78c13d660 2024-04-25T09:03:16.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78bfff640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78c0c4410 0x7fb78c0c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:16.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb78affd640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb78c13d660 0x7fb78c13dae0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fb78000e580 tx=0x7fb78000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb773fff640 1 -- 172.21.15.37:0/3995273650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7800052e0 con 0x7fb78c13d660 2024-04-25T09:03:16.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb773fff640 1 -- 172.21.15.37:0/3995273650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb780018070 con 0x7fb78c13d660 2024-04-25T09:03:16.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb791b9b640 1 -- 172.21.15.37:0/3995273650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78c13e410 con 0x7fb78c13d660 2024-04-25T09:03:16.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.561+0000 7fb773fff640 1 -- 172.21.15.37:0/3995273650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780005480 con 0x7fb78c13d660 2024-04-25T09:03:16.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.565+0000 7fb791b9b640 1 -- 172.21.15.37:0/3995273650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78c13e9d0 con 0x7fb78c13d660 2024-04-25T09:03:16.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.565+0000 7fb791b9b640 1 -- 172.21.15.37:0/3995273650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78c04efa0 con 0x7fb78c13d660 2024-04-25T09:03:16.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.565+0000 7fb773fff640 1 -- 172.21.15.37:0/3995273650 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb780003840 con 0x7fb78c13d660 2024-04-25T09:03:16.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.565+0000 7fb773fff640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb778066f90 0x7fb778069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.565+0000 7fb773fff640 1 -- 172.21.15.37:0/3995273650 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7fb7800850c0 con 0x7fb78c13d660 2024-04-25T09:03:16.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.565+0000 7fb78b7fe640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb778066f90 0x7fb778069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.565+0000 7fb78b7fe640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb778066f90 0x7fb778069450 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000f70 tx=0x7fb77c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.569+0000 7fb773fff640 1 -- 172.21.15.37:0/3995273650 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb780055dc0 con 0x7fb78c13d660 2024-04-25T09:03:16.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.737+0000 7fb791b9b640 1 -- 172.21.15.37:0/3995273650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb78c0b1cc0 con 0x7fb78c13d660 2024-04-25T09:03:16.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.737+0000 7fb773fff640 1 -- 172.21.15.37:0/3995273650 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+17223 (secure 0 0 0) 0x7fb7800556c0 con 0x7fb78c13d660 2024-04-25T09:03:16.727 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:16.727 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":114,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:15.641872+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:33.212328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:33.895814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.29","mappings":[{"from":5,"to":4}]},{"pgid":"16.39","mappings":[{"from":1,"to":4},{"from":6,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2017261543":"2024-04-26T09:03:15.641825+0000","172.21.15.37:6834/2017261543":"2024-04-26T09:03:15.641825+0000","172.21.15.37:6835/1266122577":"2024-04-26T09:02:39.103789+0000","172.21.15.37:6834/1266122577":"2024-04-26T09:02:39.103789+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:16.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 -- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb778066f90 msgr2=0x7fb778069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb778066f90 0x7fb778069450 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb77c000f70 tx=0x7fb77c009040 comp rx=0 tx=0).stop 2024-04-25T09:03:16.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 -- 172.21.15.37:0/3995273650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb78c13d660 msgr2=0x7fb78c13dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb78c13d660 0x7fb78c13dae0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fb78000e580 tx=0x7fb78000ea40 comp rx=0 tx=0).stop 2024-04-25T09:03:16.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 -- 172.21.15.37:0/3995273650 shutdown_connections 2024-04-25T09:03:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb778066f90 0x7fb778069450 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb78c0c4410 0x7fb78c0c4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb78c13d660 0x7fb78c13dae0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 --2- 172.21.15.37:0/3995273650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb7740051d0 0x7fb78c15b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 -- 172.21.15.37:0/3995273650 >> 172.21.15.37:0/3995273650 conn(0x7fb78c0b11a0 msgr2=0x7fb78c142b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 -- 172.21.15.37:0/3995273650 shutdown_connections 2024-04-25T09:03:16.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.741+0000 7fb791b9b640 1 -- 172.21.15.37:0/3995273650 wait complete. 2024-04-25T09:03:16.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:01:33.212328+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:01:33.895814+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:16.748 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:16.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f789b15c640 1 Processor -- start 2024-04-25T09:03:16.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f789b15c640 1 -- start start 2024-04-25T09:03:16.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f789b15c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940b0b00 0x7f78940cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f789b15c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f78940cc650 0x7f78940b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f789b15c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7894160310 con 0x7f78940b1910 2024-04-25T09:03:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f789b15c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789415cfa0 con 0x7f78940cc650 2024-04-25T09:03:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f789b15c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789415cdf0 con 0x7f78940b0b00 2024-04-25T09:03:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f7898ef8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940b0b00 0x7f78940cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f7898ef8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940b0b00 0x7f78940cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:39044/0 (socket says 172.21.15.37:39044) 2024-04-25T09:03:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f7898ef8640 1 -- 172.21.15.37:0/4234459293 learned_addr learned my addr 172.21.15.37:0/4234459293 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:16.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f7893fff640 1 --2- 172.21.15.37:0/4234459293 >> v2:172.21.15.203:3300/0 conn(0x7f78940cc650 0x7f78940b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f7898ef8640 1 -- 172.21.15.37:0/4234459293 >> v2:172.21.15.203:3300/0 conn(0x7f78940cc650 msgr2=0x7f78940b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f7898ef8640 1 --2- 172.21.15.37:0/4234459293 >> v2:172.21.15.203:3300/0 conn(0x7f78940cc650 0x7f78940b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f7898ef8640 1 -- 172.21.15.37:0/4234459293 >> v1:172.21.15.203:6789/0 conn(0x7f78940b1910 legacy=0x7f7894151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.893+0000 7f7898ef8640 1 -- 172.21.15.37:0/4234459293 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789415f550 con 0x7f78940b0b00 2024-04-25T09:03:16.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f7898ef8640 1 --2- 172.21.15.37:0/4234459293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940b0b00 0x7f78940cc110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7884009870 tx=0x7f788401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65b87652cbe01bee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f78937fe640 1 -- 172.21.15.37:0/4234459293 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78840b2070 con 0x7f78940b0b00 2024-04-25T09:03:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f78937fe640 1 -- 172.21.15.37:0/4234459293 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78840a9070 con 0x7f78940b0b00 2024-04-25T09:03:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f78937fe640 1 -- 172.21.15.37:0/4234459293 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78840ad880 con 0x7f78940b0b00 2024-04-25T09:03:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f789b15c640 1 -- 172.21.15.37:0/4234459293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940b0b00 msgr2=0x7f78940cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f789b15c640 1 --2- 172.21.15.37:0/4234459293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940b0b00 0x7f78940cc110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7884009870 tx=0x7f788401c560 comp rx=0 tx=0).stop 2024-04-25T09:03:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f789b15c640 1 -- 172.21.15.37:0/4234459293 shutdown_connections 2024-04-25T09:03:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f789b15c640 1 --2- 172.21.15.37:0/4234459293 >> v2:172.21.15.203:3300/0 conn(0x7f78940cc650 0x7f78940b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f789b15c640 1 --2- 172.21.15.37:0/4234459293 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940b0b00 0x7f78940cc110 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f789b15c640 1 -- 172.21.15.37:0/4234459293 >> 172.21.15.37:0/4234459293 conn(0x7f78940b1f10 msgr2=0x7f78940b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f789b15c640 1 -- 172.21.15.37:0/4234459293 shutdown_connections 2024-04-25T09:03:16.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f789b15c640 1 -- 172.21.15.37:0/4234459293 wait complete. 2024-04-25T09:03:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f789b15c640 1 Processor -- start 2024-04-25T09:03:16.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.897+0000 7f789b15c640 1 -- start start 2024-04-25T09:03:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f789b15c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78940b0b00 0x7f7894165960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f789b15c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78940b1910 0x7f7894165ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f789b15c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940cc650 0x7f78940c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f789b15c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78940b0ee0 con 0x7f78940b0b00 2024-04-25T09:03:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f789b15c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78940b1d20 con 0x7f78940b1910 2024-04-25T09:03:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f789b15c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78940b1060 con 0x7f78940cc650 2024-04-25T09:03:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7898ef8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78940b0b00 0x7f7894165960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7898ef8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78940b0b00 0x7f7894165960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33094/0 (socket says 172.21.15.37:33094) 2024-04-25T09:03:16.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7898ef8640 1 -- 172.21.15.37:0/2833244881 learned_addr learned my addr 172.21.15.37:0/2833244881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7898ef8640 1 -- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940cc650 msgr2=0x7f78940c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f78996f9640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940cc650 0x7f78940c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7893fff640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78940b1910 0x7f7894165ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7898ef8640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940cc650 0x7f78940c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7898ef8640 1 -- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78940b1910 msgr2=0x7f7894165ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7898ef8640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78940b1910 0x7f7894165ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7898ef8640 1 -- 172.21.15.37:0/2833244881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940c7570 con 0x7f78940b0b00 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7893fff640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78940b1910 0x7f7894165ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f78996f9640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940cc650 0x7f78940c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7898ef8640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78940b0b00 0x7f7894165960 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f78840b0040 tx=0x7f788401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f78917fa640 1 -- 172.21.15.37:0/2833244881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78840bc020 con 0x7f78940b0b00 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f78917fa640 1 -- 172.21.15.37:0/2833244881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78840add80 con 0x7f78940b0b00 2024-04-25T09:03:16.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f789b15c640 1 -- 172.21.15.37:0/2833244881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78940c7850 con 0x7f78940b0b00 2024-04-25T09:03:16.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f78917fa640 1 -- 172.21.15.37:0/2833244881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78840b2040 con 0x7f78940b0b00 2024-04-25T09:03:16.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f789b15c640 1 -- 172.21.15.37:0/2833244881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78940c7b30 con 0x7f78940b0b00 2024-04-25T09:03:16.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f789b15c640 1 -- 172.21.15.37:0/2833244881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789404efa0 con 0x7f78940b0b00 2024-04-25T09:03:16.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f78917fa640 1 -- 172.21.15.37:0/2833244881 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f78840bf020 con 0x7f78940b0b00 2024-04-25T09:03:16.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f78917fa640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7880066f90 0x7f7880069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f78917fa640 1 -- 172.21.15.37:0/2833244881 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f788411f400 con 0x7f78940b0b00 2024-04-25T09:03:16.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.901+0000 7f7893fff640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7880066f90 0x7f7880069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.905+0000 7f7893fff640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7880066f90 0x7f7880069450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f78941fa230 tx=0x7f7888008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:16.905+0000 7f78917fa640 1 -- 172.21.15.37:0/2833244881 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f78840c6030 con 0x7f78940b0b00 2024-04-25T09:03:17.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.069+0000 7f789b15c640 1 -- 172.21.15.37:0/2833244881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7894159790 con 0x7f78940b0b00 2024-04-25T09:03:17.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f78917fa640 1 -- 172.21.15.37:0/2833244881 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7884016150 con 0x7f78940b0b00 2024-04-25T09:03:17.061 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:03:17.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 -- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7880066f90 msgr2=0x7f7880069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7880066f90 0x7f7880069450 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f78941fa230 tx=0x7f7888008040 comp rx=0 tx=0).stop 2024-04-25T09:03:17.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 -- 172.21.15.37:0/2833244881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78940b0b00 msgr2=0x7f7894165960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78940b0b00 0x7f7894165960 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f78840b0040 tx=0x7f788401ada0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 -- 172.21.15.37:0/2833244881 shutdown_connections 2024-04-25T09:03:17.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7880066f90 0x7f7880069450 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f78940cc650 0x7f78940c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f78940b1910 0x7f7894165ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 --2- 172.21.15.37:0/2833244881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f78940b0b00 0x7f7894165960 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 -- 172.21.15.37:0/2833244881 >> 172.21.15.37:0/2833244881 conn(0x7f78940b1f10 msgr2=0x7f789415bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 -- 172.21.15.37:0/2833244881 shutdown_connections 2024-04-25T09:03:17.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.073+0000 7f789b15c640 1 -- 172.21.15.37:0/2833244881 wait complete. 2024-04-25T09:03:17.081 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:17.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.225+0000 7f36cbf52640 1 Processor -- start 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.225+0000 7f36cbf52640 1 -- start start 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.225+0000 7f36cbf52640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f36c415b6b0 0x7f36c40ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.225+0000 7f36cbf52640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36c4073010 0x7f36c40cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.225+0000 7f36cbf52640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c40cbd20 con 0x7f36c40cf590 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.225+0000 7f36cbf52640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c415ba90 con 0x7f36c415b6b0 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.225+0000 7f36cbf52640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40cc000 con 0x7f36c4073010 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.225+0000 7f36c9cee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f36c415b6b0 0x7f36c40ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.225+0000 7f36c9cee640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f36c415b6b0 0x7f36c40ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33108/0 (socket says 172.21.15.37:33108) 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.225+0000 7f36c9cee640 1 -- 172.21.15.37:0/252022694 learned_addr learned my addr 172.21.15.37:0/252022694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.225+0000 7f36ca4ef640 1 --2- 172.21.15.37:0/252022694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36c4073010 0x7f36c40cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.229+0000 7f36c9cee640 1 -- 172.21.15.37:0/252022694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36c4073010 msgr2=0x7f36c40cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.229+0000 7f36c9cee640 1 --2- 172.21.15.37:0/252022694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36c4073010 0x7f36c40cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.229+0000 7f36c9cee640 1 -- 172.21.15.37:0/252022694 >> v1:172.21.15.203:6789/0 conn(0x7f36c40cf590 legacy=0x7f36c415f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.229+0000 7f36c9cee640 1 -- 172.21.15.37:0/252022694 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c40cc2e0 con 0x7f36c415b6b0 2024-04-25T09:03:17.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.229+0000 7f36ca4ef640 1 --2- 172.21.15.37:0/252022694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36c4073010 0x7f36c40cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:17.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.229+0000 7f36c9cee640 1 --2- 172.21.15.37:0/252022694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c415b6b0 0x7f36c40ca3c0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009870 tx=0x7f36b801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af07579159fe161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.229+0000 7f36c8cec640 1 -- 172.21.15.37:0/252022694 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b80b0070 con 0x7f36c415b6b0 2024-04-25T09:03:17.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c8cec640 1 -- 172.21.15.37:0/252022694 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b80ab600 con 0x7f36c415b6b0 2024-04-25T09:03:17.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c8cec640 1 -- 172.21.15.37:0/252022694 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b80ab9d0 con 0x7f36c415b6b0 2024-04-25T09:03:17.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 -- 172.21.15.37:0/252022694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c415b6b0 msgr2=0x7f36c40ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 --2- 172.21.15.37:0/252022694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c415b6b0 0x7f36c40ca3c0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f36b8009870 tx=0x7f36b801c560 comp rx=0 tx=0).stop 2024-04-25T09:03:17.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 -- 172.21.15.37:0/252022694 shutdown_connections 2024-04-25T09:03:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 --2- 172.21.15.37:0/252022694 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36c4073010 0x7f36c40cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 --2- 172.21.15.37:0/252022694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c415b6b0 0x7f36c40ca3c0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 -- 172.21.15.37:0/252022694 >> 172.21.15.37:0/252022694 conn(0x7f36c406ae60 msgr2=0x7f36c406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 -- 172.21.15.37:0/252022694 shutdown_connections 2024-04-25T09:03:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 -- 172.21.15.37:0/252022694 wait complete. 2024-04-25T09:03:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 Processor -- start 2024-04-25T09:03:17.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 -- start start 2024-04-25T09:03:17.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36c4073010 0x7f36c40ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40cf590 0x7f36c41648d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36c415b6b0 0x7f36c40c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40cadb0 con 0x7f36c40cf590 2024-04-25T09:03:17.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40ca900 con 0x7f36c4073010 2024-04-25T09:03:17.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36cbf52640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40caf30 con 0x7f36c415b6b0 2024-04-25T09:03:17.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c94ed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40cf590 0x7f36c41648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c94ed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40cf590 0x7f36c41648d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33112/0 (socket says 172.21.15.37:33112) 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c94ed640 1 -- 172.21.15.37:0/739329327 learned_addr learned my addr 172.21.15.37:0/739329327 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c9cee640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36c4073010 0x7f36c40ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36ca4ef640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36c415b6b0 0x7f36c40c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c94ed640 1 -- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36c415b6b0 msgr2=0x7f36c40c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c94ed640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36c415b6b0 0x7f36c40c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c94ed640 1 -- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36c4073010 msgr2=0x7f36c40ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c94ed640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36c4073010 0x7f36c40ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c94ed640 1 -- 172.21.15.37:0/739329327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c40cd880 con 0x7f36c40cf590 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36c94ed640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40cf590 0x7f36c41648d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f36c000c500 tx=0x7f36c000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.233+0000 7f36b67fc640 1 -- 172.21.15.37:0/739329327 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0016020 con 0x7f36c40cf590 2024-04-25T09:03:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.237+0000 7f36b67fc640 1 -- 172.21.15.37:0/739329327 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c0005220 con 0x7f36c40cf590 2024-04-25T09:03:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.237+0000 7f36b67fc640 1 -- 172.21.15.37:0/739329327 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c0007030 con 0x7f36c40cf590 2024-04-25T09:03:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.237+0000 7f36cbf52640 1 -- 172.21.15.37:0/739329327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c40cdb20 con 0x7f36c40cf590 2024-04-25T09:03:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.237+0000 7f36cbf52640 1 -- 172.21.15.37:0/739329327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c40ce3c0 con 0x7f36c40cf590 2024-04-25T09:03:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.237+0000 7f36cbf52640 1 -- 172.21.15.37:0/739329327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c40cdcf0 con 0x7f36c40cf590 2024-04-25T09:03:17.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.237+0000 7f36b67fc640 1 -- 172.21.15.37:0/739329327 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f36c00071f0 con 0x7f36c40cf590 2024-04-25T09:03:17.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.237+0000 7f36b67fc640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f36a8066f60 0x7f36a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.237+0000 7f36b67fc640 1 -- 172.21.15.37:0/739329327 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f36c0084ba0 con 0x7f36c40cf590 2024-04-25T09:03:17.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.237+0000 7f36c9cee640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f36a8066f60 0x7f36a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.237+0000 7f36c9cee640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f36a8066f60 0x7f36a8069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f36b80ad000 tx=0x7f36b80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.241+0000 7f36b67fc640 1 -- 172.21.15.37:0/739329327 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f36c0020020 con 0x7f36c40cf590 2024-04-25T09:03:17.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.405+0000 7f36cbf52640 1 -- 172.21.15.37:0/739329327 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f36c40ce5c0 con 0x7f36c40cf590 2024-04-25T09:03:17.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.405+0000 7f36b67fc640 1 -- 172.21.15.37:0/739329327 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7f36c40ce5c0 con 0x7f36c40cf590 2024-04-25T09:03:17.395 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:03:17.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 -- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f36a8066f60 msgr2=0x7f36a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f36a8066f60 0x7f36a8069420 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f36b80ad000 tx=0x7f36b80a6000 comp rx=0 tx=0).stop 2024-04-25T09:03:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 -- 172.21.15.37:0/739329327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40cf590 msgr2=0x7f36c41648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40cf590 0x7f36c41648d0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f36c000c500 tx=0x7f36c000c9c0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 -- 172.21.15.37:0/739329327 shutdown_connections 2024-04-25T09:03:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f36a8066f60 0x7f36a8069420 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f36c415b6b0 0x7f36c40c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f36c40cf590 0x7f36c41648d0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 --2- 172.21.15.37:0/739329327 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f36c4073010 0x7f36c40ce8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 -- 172.21.15.37:0/739329327 >> 172.21.15.37:0/739329327 conn(0x7f36c406ae60 msgr2=0x7f36c40b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 -- 172.21.15.37:0/739329327 shutdown_connections 2024-04-25T09:03:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.409+0000 7f36cbf52640 1 -- 172.21.15.37:0/739329327 wait complete. 2024-04-25T09:03:17.415 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.545+0000 7f734851d640 1 Processor -- start 2024-04-25T09:03:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.545+0000 7f734851d640 1 -- start start 2024-04-25T09:03:17.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f734851d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f734015e1e0 0x7f734015e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f734851d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734015eb30 con 0x7f73400b2ac0 2024-04-25T09:03:17.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f734851d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73400ce070 con 0x7f734015a600 2024-04-25T09:03:17.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f734851d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73400ce350 con 0x7f734015e1e0 2024-04-25T09:03:17.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f7345ab8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f73400b2ac0 0x7f73400cb9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42690/0 (socket says 172.21.15.37:42690) 2024-04-25T09:03:17.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f7345ab8640 1 -- 172.21.15.37:0/3632485785 learned_addr learned my addr 172.21.15.37:0/3632485785 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:17.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f7346aba640 1 --2- 172.21.15.37:0/3632485785 >> v2:172.21.15.37:3300/0 conn(0x7f734015e1e0 0x7f734015e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f73452b7640 1 -- 172.21.15.37:0/3632485785 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 803254698 0 0) 0x7f7330001120 con 0x7f73400b2ac0 2024-04-25T09:03:17.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f73452b7640 1 -- 172.21.15.37:0/3632485785 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f732c003530 con 0x7f73400b2ac0 2024-04-25T09:03:17.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f7346aba640 1 -- 172.21.15.37:0/3632485785 >> v1:172.21.15.203:6789/0 conn(0x7f734015a600 legacy=0x7f73400cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f7346aba640 1 -- 172.21.15.37:0/3632485785 >> v1:172.21.15.37:6789/0 conn(0x7f73400b2ac0 legacy=0x7f73400cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.549+0000 7f7346aba640 1 -- 172.21.15.37:0/3632485785 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73400ce630 con 0x7f734015e1e0 2024-04-25T09:03:17.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f7346aba640 1 --2- 172.21.15.37:0/3632485785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734015e1e0 0x7f734015e5f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f7338009d90 tx=0x7f733801f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12da75b2bd60f3bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f73452b7640 1 -- 172.21.15.37:0/3632485785 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73380b4070 con 0x7f734015e1e0 2024-04-25T09:03:17.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f73452b7640 1 -- 172.21.15.37:0/3632485785 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73380af600 con 0x7f734015e1e0 2024-04-25T09:03:17.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f73452b7640 1 -- 172.21.15.37:0/3632485785 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73380af9d0 con 0x7f734015e1e0 2024-04-25T09:03:17.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 -- 172.21.15.37:0/3632485785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734015e1e0 msgr2=0x7f734015e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 --2- 172.21.15.37:0/3632485785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734015e1e0 0x7f734015e5f0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f7338009d90 tx=0x7f733801f150 comp rx=0 tx=0).stop 2024-04-25T09:03:17.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 -- 172.21.15.37:0/3632485785 shutdown_connections 2024-04-25T09:03:17.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 --2- 172.21.15.37:0/3632485785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734015e1e0 0x7f734015e5f0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 -- 172.21.15.37:0/3632485785 >> 172.21.15.37:0/3632485785 conn(0x7f73400b1f10 msgr2=0x7f73400b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 -- 172.21.15.37:0/3632485785 shutdown_connections 2024-04-25T09:03:17.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 -- 172.21.15.37:0/3632485785 wait complete. 2024-04-25T09:03:17.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 Processor -- start 2024-04-25T09:03:17.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 -- start start 2024-04-25T09:03:17.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73400b2ac0 0x7f7340154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734015a600 0x7f73400c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f734015e1e0 0x7f7340151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734015a9e0 con 0x7f734015a600 2024-04-25T09:03:17.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73400cc1e0 con 0x7f734015e1e0 2024-04-25T09:03:17.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f734851d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734015ab60 con 0x7f73400b2ac0 2024-04-25T09:03:17.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f7345ab8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734015a600 0x7f73400c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f7345ab8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734015a600 0x7f73400c6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33122/0 (socket says 172.21.15.37:33122) 2024-04-25T09:03:17.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f7345ab8640 1 -- 172.21.15.37:0/2620398281 learned_addr learned my addr 172.21.15.37:0/2620398281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f73462b9640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73400b2ac0 0x7f7340154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.553+0000 7f7346aba640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f734015e1e0 0x7f7340151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7345ab8640 1 -- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73400b2ac0 msgr2=0x7f7340154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7345ab8640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73400b2ac0 0x7f7340154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7345ab8640 1 -- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f734015e1e0 msgr2=0x7f7340151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7345ab8640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f734015e1e0 0x7f7340151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7345ab8640 1 -- 172.21.15.37:0/2620398281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73401503e0 con 0x7f734015a600 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7345ab8640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734015a600 0x7f73400c6d10 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f733000c760 tx=0x7f733000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7336ffd640 1 -- 172.21.15.37:0/2620398281 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330017070 con 0x7f734015a600 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7336ffd640 1 -- 172.21.15.37:0/2620398281 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7330005ac0 con 0x7f734015a600 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f734851d640 1 -- 172.21.15.37:0/2620398281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73400c7250 con 0x7f734015a600 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7336ffd640 1 -- 172.21.15.37:0/2620398281 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7330007070 con 0x7f734015a600 2024-04-25T09:03:17.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f734851d640 1 -- 172.21.15.37:0/2620398281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7340150650 con 0x7f734015a600 2024-04-25T09:03:17.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f734851d640 1 -- 172.21.15.37:0/2620398281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734004efa0 con 0x7f734015a600 2024-04-25T09:03:17.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7336ffd640 1 -- 172.21.15.37:0/2620398281 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f733001a070 con 0x7f734015a600 2024-04-25T09:03:17.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7336ffd640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7324066f60 0x7f7324069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f7336ffd640 1 -- 172.21.15.37:0/2620398281 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f73300846e0 con 0x7f734015a600 2024-04-25T09:03:17.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f73462b9640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7324066f60 0x7f7324069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.557+0000 7f73462b9640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7324066f60 0x7f7324069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f73400b51b0 tx=0x7f733c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.561+0000 7f7336ffd640 1 -- 172.21.15.37:0/2620398281 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f734015eb30 con 0x7f734015a600 2024-04-25T09:03:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.721+0000 7f734851d640 1 -- 172.21.15.37:0/2620398281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f73400b6380 con 0x7f734015a600 2024-04-25T09:03:17.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.721+0000 7f7336ffd640 1 -- 172.21.15.37:0/2620398281 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7330055460 con 0x7f734015a600 2024-04-25T09:03:17.711 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 -- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7324066f60 msgr2=0x7f7324069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7324066f60 0x7f7324069420 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f73400b51b0 tx=0x7f733c009040 comp rx=0 tx=0).stop 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 -- 172.21.15.37:0/2620398281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734015a600 msgr2=0x7f73400c6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734015a600 0x7f73400c6d10 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f733000c760 tx=0x7f733000cc20 comp rx=0 tx=0).stop 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 -- 172.21.15.37:0/2620398281 shutdown_connections 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7324066f60 0x7f7324069420 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f734015e1e0 0x7f7340151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734015a600 0x7f73400c6d10 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 --2- 172.21.15.37:0/2620398281 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f73400b2ac0 0x7f7340154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 -- 172.21.15.37:0/2620398281 >> 172.21.15.37:0/2620398281 conn(0x7f73400b1f10 msgr2=0x7f73400b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 -- 172.21.15.37:0/2620398281 shutdown_connections 2024-04-25T09:03:17.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.725+0000 7f734851d640 1 -- 172.21.15.37:0/2620398281 wait complete. 2024-04-25T09:03:17.730 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.857+0000 7eff0fe14640 1 Processor -- start 2024-04-25T09:03:17.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.857+0000 7eff0fe14640 1 -- start start 2024-04-25T09:03:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0fe14640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7eff080b3f50 0x7eff0815ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0fe14640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff080b2090 con 0x7eff0815a700 2024-04-25T09:03:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0fe14640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff080b43e0 con 0x7eff080b24a0 2024-04-25T09:03:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0fe14640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0815f200 con 0x7eff080b3f50 2024-04-25T09:03:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0d3af640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7eff080b3f50 0x7eff0815ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0dbb0640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7eff0815a700 0x7eff080cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42702/0 (socket says 172.21.15.37:42702) 2024-04-25T09:03:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0dbb0640 1 -- 172.21.15.37:0/2494183193 learned_addr learned my addr 172.21.15.37:0/2494183193 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0d3af640 1 -- 172.21.15.37:0/2494183193 >> v1:172.21.15.203:6789/0 conn(0x7eff080b24a0 legacy=0x7eff0806e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0d3af640 1 -- 172.21.15.37:0/2494183193 >> v1:172.21.15.37:6789/0 conn(0x7eff0815a700 legacy=0x7eff080cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0d3af640 1 -- 172.21.15.37:0/2494183193 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff080b46c0 con 0x7eff080b3f50 2024-04-25T09:03:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0d3af640 1 --2- 172.21.15.37:0/2494183193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff080b3f50 0x7eff0815ecc0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7efef80097c0 tx=0x7efef801e520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30abddda049447 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0cbae640 1 -- 172.21.15.37:0/2494183193 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef80ad430 con 0x7eff080b3f50 2024-04-25T09:03:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0cbae640 1 -- 172.21.15.37:0/2494183193 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef80b2070 con 0x7eff080b3f50 2024-04-25T09:03:17.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.861+0000 7eff0cbae640 1 -- 172.21.15.37:0/2494183193 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef80ad7f0 con 0x7eff080b3f50 2024-04-25T09:03:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 -- 172.21.15.37:0/2494183193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff080b3f50 msgr2=0x7eff0815ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 --2- 172.21.15.37:0/2494183193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff080b3f50 0x7eff0815ecc0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7efef80097c0 tx=0x7efef801e520 comp rx=0 tx=0).stop 2024-04-25T09:03:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 -- 172.21.15.37:0/2494183193 shutdown_connections 2024-04-25T09:03:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 --2- 172.21.15.37:0/2494183193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff080b3f50 0x7eff0815ecc0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 -- 172.21.15.37:0/2494183193 >> 172.21.15.37:0/2494183193 conn(0x7eff080b11a0 msgr2=0x7eff080b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 -- 172.21.15.37:0/2494183193 shutdown_connections 2024-04-25T09:03:17.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 -- 172.21.15.37:0/2494183193 wait complete. 2024-04-25T09:03:17.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 Processor -- start 2024-04-25T09:03:17.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 -- start start 2024-04-25T09:03:17.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff080b24a0 0x7eff08159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0815a700 0x7eff080aa620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff08154ef0 0x7eff080aab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff080cd310 con 0x7eff08154ef0 2024-04-25T09:03:17.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff080ce600 con 0x7eff080b24a0 2024-04-25T09:03:17.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff080cd490 con 0x7eff0815a700 2024-04-25T09:03:17.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0e3b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff08154ef0 0x7eff080aab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0e3b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff08154ef0 0x7eff080aab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33138/0 (socket says 172.21.15.37:33138) 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0e3b1640 1 -- 172.21.15.37:0/1297399072 learned_addr learned my addr 172.21.15.37:0/1297399072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0d3af640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0815a700 0x7eff080aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0e3b1640 1 -- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0815a700 msgr2=0x7eff080aa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0e3b1640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0815a700 0x7eff080aa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0dbb0640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff080b24a0 0x7eff08159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0e3b1640 1 -- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff080b24a0 msgr2=0x7eff08159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0e3b1640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff080b24a0 0x7eff08159440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0e3b1640 1 -- 172.21.15.37:0/1297399072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff08159a10 con 0x7eff08154ef0 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0d3af640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0815a700 0x7eff080aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0e3b1640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff08154ef0 0x7eff080aab60 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7eff0000b9a0 tx=0x7eff0000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7efefe7fc640 1 -- 172.21.15.37:0/1297399072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00019070 con 0x7eff08154ef0 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7efefe7fc640 1 -- 172.21.15.37:0/1297399072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0000cba0 con 0x7eff08154ef0 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 -- 172.21.15.37:0/1297399072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff08159cf0 con 0x7eff08154ef0 2024-04-25T09:03:17.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7efefe7fc640 1 -- 172.21.15.37:0/1297399072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00014440 con 0x7eff08154ef0 2024-04-25T09:03:17.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.865+0000 7eff0fe14640 1 -- 172.21.15.37:0/1297399072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff08158c40 con 0x7eff08154ef0 2024-04-25T09:03:17.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.869+0000 7eff0fe14640 1 -- 172.21.15.37:0/1297399072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0806f260 con 0x7eff08154ef0 2024-04-25T09:03:17.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.869+0000 7efefe7fc640 1 -- 172.21.15.37:0/1297399072 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7eff0001c070 con 0x7eff08154ef0 2024-04-25T09:03:17.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.869+0000 7efefe7fc640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efeec066f60 0x7efeec069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.869+0000 7efefe7fc640 1 -- 172.21.15.37:0/1297399072 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7eff00084fe0 con 0x7eff08154ef0 2024-04-25T09:03:17.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.869+0000 7eff0dbb0640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efeec066f60 0x7efeec069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.869+0000 7eff0dbb0640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efeec066f60 0x7efeec069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7eff04000f70 tx=0x7eff04009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:17.873+0000 7efefe7fc640 1 -- 172.21.15.37:0/1297399072 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7eff00055d70 con 0x7eff08154ef0 2024-04-25T09:03:18.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.069+0000 7eff0fe14640 1 -- 172.21.15.37:0/1297399072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff0804ef80 con 0x7eff08154ef0 2024-04-25T09:03:18.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.069+0000 7efefe7fc640 1 -- 172.21.15.37:0/1297399072 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4934 (secure 0 0 0) 0x7eff00055670 con 0x7eff08154ef0 2024-04-25T09:03:18.061 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:18.061 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2524942703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2524942703},{"type":"v1","addr":"172.21.15.37:6835","nonce":2524942703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:03:15.641847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:03:18.061 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 89 2024-04-25T09:03:18.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 -- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efeec066f60 msgr2=0x7efeec069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efeec066f60 0x7efeec069420 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7eff04000f70 tx=0x7eff04009040 comp rx=0 tx=0).stop 2024-04-25T09:03:18.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 -- 172.21.15.37:0/1297399072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff08154ef0 msgr2=0x7eff080aab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff08154ef0 0x7eff080aab60 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7eff0000b9a0 tx=0x7eff0000be60 comp rx=0 tx=0).stop 2024-04-25T09:03:18.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 -- 172.21.15.37:0/1297399072 shutdown_connections 2024-04-25T09:03:18.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efeec066f60 0x7efeec069420 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff08154ef0 0x7eff080aab60 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7eff0815a700 0x7eff080aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 --2- 172.21.15.37:0/1297399072 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7eff080b24a0 0x7eff08159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 -- 172.21.15.37:0/1297399072 >> 172.21.15.37:0/1297399072 conn(0x7eff080b11a0 msgr2=0x7eff080b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:18.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 -- 172.21.15.37:0/1297399072 shutdown_connections 2024-04-25T09:03:18.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.073+0000 7eff0fe14640 1 -- 172.21.15.37:0/1297399072 wait complete. 2024-04-25T09:03:18.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f260bf4d640 1 Processor -- start 2024-04-25T09:03:18.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f260bf4d640 1 -- start start 2024-04-25T09:03:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f260bf4d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f260415da10 0x7f2604071030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f260bf4d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2604162310 con 0x7f26040b24a0 2024-04-25T09:03:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f260bf4d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26040adbd0 con 0x7f2604071e90 2024-04-25T09:03:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f260bf4d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26040ae860 con 0x7f260415da10 2024-04-25T09:03:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f2609ce9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f260415da10 0x7f2604071030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f26094e8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f26040b24a0 0x7f26041614f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42706/0 (socket says 172.21.15.37:42706) 2024-04-25T09:03:18.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f26094e8640 1 -- 172.21.15.37:0/839113406 learned_addr learned my addr 172.21.15.37:0/839113406 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:18.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f2609ce9640 1 -- 172.21.15.37:0/839113406 >> v1:172.21.15.203:6789/0 conn(0x7f2604071e90 legacy=0x7f2604161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f2609ce9640 1 -- 172.21.15.37:0/839113406 >> v1:172.21.15.37:6789/0 conn(0x7f26040b24a0 legacy=0x7f26041614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f2608ce7640 1 -- 172.21.15.37:0/839113406 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1820298401 0 0) 0x7f2604162310 con 0x7f26040b24a0 2024-04-25T09:03:18.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.209+0000 7f2609ce9640 1 -- 172.21.15.37:0/839113406 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260415e210 con 0x7f260415da10 2024-04-25T09:03:18.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f2609ce9640 1 --2- 172.21.15.37:0/839113406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f260415da10 0x7f2604071030 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f26000097c0 tx=0x7f260001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ca407ab776f11a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f2608ce7640 1 -- 172.21.15.37:0/839113406 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26000b0070 con 0x7f260415da10 2024-04-25T09:03:18.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f2608ce7640 1 -- 172.21.15.37:0/839113406 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26000ab600 con 0x7f260415da10 2024-04-25T09:03:18.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f2608ce7640 1 -- 172.21.15.37:0/839113406 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26000ab9d0 con 0x7f260415da10 2024-04-25T09:03:18.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f260bf4d640 1 -- 172.21.15.37:0/839113406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f260415da10 msgr2=0x7f2604071030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f260bf4d640 1 --2- 172.21.15.37:0/839113406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f260415da10 0x7f2604071030 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f26000097c0 tx=0x7f260001c380 comp rx=0 tx=0).stop 2024-04-25T09:03:18.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f260bf4d640 1 -- 172.21.15.37:0/839113406 shutdown_connections 2024-04-25T09:03:18.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f260bf4d640 1 --2- 172.21.15.37:0/839113406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f260415da10 0x7f2604071030 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f260bf4d640 1 -- 172.21.15.37:0/839113406 >> 172.21.15.37:0/839113406 conn(0x7f26040b11a0 msgr2=0x7f26040b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:18.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f260bf4d640 1 -- 172.21.15.37:0/839113406 shutdown_connections 2024-04-25T09:03:18.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f260bf4d640 1 -- 172.21.15.37:0/839113406 wait complete. 2024-04-25T09:03:18.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f260bf4d640 1 Processor -- start 2024-04-25T09:03:18.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.213+0000 7f260bf4d640 1 -- start start 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f260bf4d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2604071e90 0x7f2604157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f260bf4d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26040b24a0 0x7f2604158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f260bf4d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f260415da10 0x7f260415b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f260bf4d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260415ee70 con 0x7f2604071e90 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f260bf4d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260415ecf0 con 0x7f26040b24a0 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f260bf4d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f260416c4c0 con 0x7f260415da10 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f2609ce9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2604071e90 0x7f2604157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f2609ce9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2604071e90 0x7f2604157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33164/0 (socket says 172.21.15.37:33164) 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f2609ce9640 1 -- 172.21.15.37:0/2120835708 learned_addr learned my addr 172.21.15.37:0/2120835708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f260a4ea640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f260415da10 0x7f260415b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f2609ce9640 1 -- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f260415da10 msgr2=0x7f260415b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f2609ce9640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f260415da10 0x7f260415b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f2609ce9640 1 -- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26040b24a0 msgr2=0x7f2604158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f2609ce9640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26040b24a0 0x7f2604158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f2609ce9640 1 -- 172.21.15.37:0/2120835708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2604156c80 con 0x7f2604071e90 2024-04-25T09:03:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f2609ce9640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2604071e90 0x7f2604157ce0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f2600018f90 tx=0x7f2600019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f25fa7fc640 1 -- 172.21.15.37:0/2120835708 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26000b8030 con 0x7f2604071e90 2024-04-25T09:03:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f25fa7fc640 1 -- 172.21.15.37:0/2120835708 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26000b4410 con 0x7f2604071e90 2024-04-25T09:03:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f260bf4d640 1 -- 172.21.15.37:0/2120835708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2604156f20 con 0x7f2604071e90 2024-04-25T09:03:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f25fa7fc640 1 -- 172.21.15.37:0/2120835708 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26000b0040 con 0x7f2604071e90 2024-04-25T09:03:18.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f260bf4d640 1 -- 172.21.15.37:0/2120835708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26041577c0 con 0x7f2604071e90 2024-04-25T09:03:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f260bf4d640 1 -- 172.21.15.37:0/2120835708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26041570f0 con 0x7f2604071e90 2024-04-25T09:03:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f25fa7fc640 1 -- 172.21.15.37:0/2120835708 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26000bb070 con 0x7f2604071e90 2024-04-25T09:03:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f25fa7fc640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f25e8066f60 0x7f25e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f25fa7fc640 1 -- 172.21.15.37:0/2120835708 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f260011eb90 con 0x7f2604071e90 2024-04-25T09:03:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.217+0000 7f26094e8640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f25e8066f60 0x7f25e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.221+0000 7f26094e8640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f25e8066f60 0x7f25e8069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f25f4000c10 tx=0x7f25f4007910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.221+0000 7f25fa7fc640 1 -- 172.21.15.37:0/2120835708 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f26000c2070 con 0x7f2604071e90 2024-04-25T09:03:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.385+0000 7f260bf4d640 1 -- 172.21.15.37:0/2120835708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f260404efa0 con 0x7f2604071e90 2024-04-25T09:03:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.385+0000 7f25fa7fc640 1 -- 172.21.15.37:0/2120835708 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+17223 (secure 0 0 0) 0x7f2600016090 con 0x7f2604071e90 2024-04-25T09:03:18.376 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:18.376 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":114,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:15.641872+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:33.212328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:33.895814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.29","mappings":[{"from":5,"to":4}]},{"pgid":"16.39","mappings":[{"from":1,"to":4},{"from":6,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2017261543":"2024-04-26T09:03:15.641825+0000","172.21.15.37:6834/2017261543":"2024-04-26T09:03:15.641825+0000","172.21.15.37:6835/1266122577":"2024-04-26T09:02:39.103789+0000","172.21.15.37:6834/1266122577":"2024-04-26T09:02:39.103789+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 -- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f25e8066f60 msgr2=0x7f25e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f25e8066f60 0x7f25e8069420 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f25f4000c10 tx=0x7f25f4007910 comp rx=0 tx=0).stop 2024-04-25T09:03:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 -- 172.21.15.37:0/2120835708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2604071e90 msgr2=0x7f2604157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2604071e90 0x7f2604157ce0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f2600018f90 tx=0x7f2600019120 comp rx=0 tx=0).stop 2024-04-25T09:03:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 -- 172.21.15.37:0/2120835708 shutdown_connections 2024-04-25T09:03:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f25e8066f60 0x7f25e8069420 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f260415da10 0x7f260415b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f26040b24a0 0x7f2604158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 --2- 172.21.15.37:0/2120835708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2604071e90 0x7f2604157ce0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 -- 172.21.15.37:0/2120835708 >> 172.21.15.37:0/2120835708 conn(0x7f26040b11a0 msgr2=0x7f260406e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 -- 172.21.15.37:0/2120835708 shutdown_connections 2024-04-25T09:03:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.389+0000 7f260bf4d640 1 -- 172.21.15.37:0/2120835708 wait complete. 2024-04-25T09:03:18.396 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:03:18.396 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.525+0000 7fa8b9c1d640 1 Processor -- start 2024-04-25T09:03:18.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.525+0000 7fa8b9c1d640 1 -- start start 2024-04-25T09:03:18.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b9c1d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa8b4144ca0 0x7fa8b406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b9c1d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa8b406ec60 0x7fa8b4142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b9c1d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8b40ccc70 con 0x7fa8b40ce510 2024-04-25T09:03:18.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b9c1d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b40cc2b0 con 0x7fa8b406ec60 2024-04-25T09:03:18.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b9c1d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8b406e290 con 0x7fa8b4144ca0 2024-04-25T09:03:18.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa8b406ec60 0x7fa8b4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa8b406ec60 0x7fa8b4142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33178/0 (socket says 172.21.15.37:33178) 2024-04-25T09:03:18.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b3fff640 1 -- 172.21.15.37:0/2685143369 learned_addr learned my addr 172.21.15.37:0/2685143369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:18.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b2ffd640 1 --2- 172.21.15.37:0/2685143369 >> v2:172.21.15.203:3300/0 conn(0x7fa8b4144ca0 0x7fa8b406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b3fff640 1 -- 172.21.15.37:0/2685143369 >> v2:172.21.15.203:3300/0 conn(0x7fa8b4144ca0 msgr2=0x7fa8b406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b3fff640 1 --2- 172.21.15.37:0/2685143369 >> v2:172.21.15.203:3300/0 conn(0x7fa8b4144ca0 0x7fa8b406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b3fff640 1 -- 172.21.15.37:0/2685143369 >> v1:172.21.15.203:6789/0 conn(0x7fa8b40ce510 legacy=0x7fa8b40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.529+0000 7fa8b3fff640 1 -- 172.21.15.37:0/2685143369 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b406dc40 con 0x7fa8b406ec60 2024-04-25T09:03:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b3fff640 1 --2- 172.21.15.37:0/2685143369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b406ec60 0x7fa8b4142680 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4070fe0 tx=0x7fa8ac01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f345a9fef8246c62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b27fc640 1 -- 172.21.15.37:0/2685143369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0b4070 con 0x7fa8b406ec60 2024-04-25T09:03:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b27fc640 1 -- 172.21.15.37:0/2685143369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac0af400 con 0x7fa8b406ec60 2024-04-25T09:03:18.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b27fc640 1 -- 172.21.15.37:0/2685143369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0af7d0 con 0x7fa8b406ec60 2024-04-25T09:03:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/2685143369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b406ec60 msgr2=0x7fa8b4142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 --2- 172.21.15.37:0/2685143369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b406ec60 0x7fa8b4142680 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fa8b4070fe0 tx=0x7fa8ac01f1e0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/2685143369 shutdown_connections 2024-04-25T09:03:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 --2- 172.21.15.37:0/2685143369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b406ec60 0x7fa8b4142680 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 --2- 172.21.15.37:0/2685143369 >> v2:172.21.15.203:3300/0 conn(0x7fa8b4144ca0 0x7fa8b406e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/2685143369 >> 172.21.15.37:0/2685143369 conn(0x7fa8b40b11a0 msgr2=0x7fa8b40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/2685143369 shutdown_connections 2024-04-25T09:03:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/2685143369 wait complete. 2024-04-25T09:03:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 Processor -- start 2024-04-25T09:03:18.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 -- start start 2024-04-25T09:03:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8b406ec60 0x7fa8b4141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8b40ce510 0x7fa8b41425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b4144ca0 0x7fa8b4130e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b41451f0 con 0x7fa8b4144ca0 2024-04-25T09:03:18.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b40cc900 con 0x7fa8b406ec60 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b9c1d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b4145370 con 0x7fa8b40ce510 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b4144ca0 0x7fa8b4130e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b4144ca0 0x7fa8b4130e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33180/0 (socket says 172.21.15.37:33180) 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.533+0000 7fa8b3fff640 1 -- 172.21.15.37:0/871530745 learned_addr learned my addr 172.21.15.37:0/871530745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b37fe640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8b406ec60 0x7fa8b4141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b2ffd640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8b40ce510 0x7fa8b41425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b3fff640 1 -- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8b40ce510 msgr2=0x7fa8b41425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b3fff640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8b40ce510 0x7fa8b41425e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b3fff640 1 -- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8b406ec60 msgr2=0x7fa8b4141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b3fff640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8b406ec60 0x7fa8b4141670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b3fff640 1 -- 172.21.15.37:0/871530745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b41f2370 con 0x7fa8b4144ca0 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b3fff640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b4144ca0 0x7fa8b4130e90 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0b2040 tx=0x7fa8ac01bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa89bfff640 1 -- 172.21.15.37:0/871530745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0be020 con 0x7fa8b4144ca0 2024-04-25T09:03:18.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa89bfff640 1 -- 172.21.15.37:0/871530745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac0090d0 con 0x7fa8b4144ca0 2024-04-25T09:03:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/871530745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b41313d0 con 0x7fa8b4144ca0 2024-04-25T09:03:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa89bfff640 1 -- 172.21.15.37:0/871530745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0b4040 con 0x7fa8b4144ca0 2024-04-25T09:03:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/871530745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b41f1ad0 con 0x7fa8b4144ca0 2024-04-25T09:03:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/871530745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b404efa0 con 0x7fa8b4144ca0 2024-04-25T09:03:18.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa89bfff640 1 -- 172.21.15.37:0/871530745 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa8ac0c1020 con 0x7fa8b4144ca0 2024-04-25T09:03:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa89bfff640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa8a0066f60 0x7fa8a0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa89bfff640 1 -- 172.21.15.37:0/871530745 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7fa8ac1237c0 con 0x7fa8b4144ca0 2024-04-25T09:03:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.537+0000 7fa8b37fe640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa8a0066f60 0x7fa8a0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.541+0000 7fa8b37fe640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa8a0066f60 0x7fa8a0069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4001650 tx=0x7fa8a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.541+0000 7fa89bfff640 1 -- 172.21.15.37:0/871530745 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8ac0c8030 con 0x7fa8b4144ca0 2024-04-25T09:03:18.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.737+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/871530745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8b41e0d70 con 0x7fa8b4144ca0 2024-04-25T09:03:18.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa89bfff640 1 -- 172.21.15.37:0/871530745 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fa8ac018d40 con 0x7fa8b4144ca0 2024-04-25T09:03:18.729 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:18.729 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2524942703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2524942703},{"type":"v1","addr":"172.21.15.37:6835","nonce":2524942703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:03:15.641847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:03:18.730 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 89 2024-04-25T09:03:18.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa8a0066f60 msgr2=0x7fa8a0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa8a0066f60 0x7fa8a0069420 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4001650 tx=0x7fa8a4009040 comp rx=0 tx=0).stop 2024-04-25T09:03:18.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/871530745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b4144ca0 msgr2=0x7fa8b4130e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b4144ca0 0x7fa8b4130e90 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac0b2040 tx=0x7fa8ac01bf80 comp rx=0 tx=0).stop 2024-04-25T09:03:18.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/871530745 shutdown_connections 2024-04-25T09:03:18.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa8a0066f60 0x7fa8a0069420 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8b4144ca0 0x7fa8b4130e90 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8b40ce510 0x7fa8b41425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 --2- 172.21.15.37:0/871530745 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa8b406ec60 0x7fa8b4141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/871530745 >> 172.21.15.37:0/871530745 conn(0x7fa8b40b11a0 msgr2=0x7fa8b40b0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:18.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/871530745 shutdown_connections 2024-04-25T09:03:18.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.741+0000 7fa8b9c1d640 1 -- 172.21.15.37:0/871530745 wait complete. 2024-04-25T09:03:18.748 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:18.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.889+0000 7f514c9bc640 1 Processor -- start 2024-04-25T09:03:18.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.889+0000 7f514c9bc640 1 -- start start 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.889+0000 7f514c9bc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480af280 0x7f514815df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.889+0000 7f514c9bc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f51480b2c30 0x7f514806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.889+0000 7f514c9bc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f514806c1e0 0x7f5148161500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.889+0000 7f514c9bc640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514806daa0 con 0x7f514806c1e0 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.889+0000 7f514c9bc640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514806c640 con 0x7f51480b2c30 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.889+0000 7f514c9bc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5148066c30 con 0x7f51480af280 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.889+0000 7f5146d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f514806c1e0 0x7f5148161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f5146d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f514806c1e0 0x7f5148161500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33196/0 (socket says 172.21.15.37:33196) 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f5146d9d640 1 -- 172.21.15.37:0/2217359927 learned_addr learned my addr 172.21.15.37:0/2217359927 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514659c640 1 --2- 172.21.15.37:0/2217359927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480af280 0x7f514815df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f5145d9b640 1 --2- 172.21.15.37:0/2217359927 >> v2:172.21.15.203:3300/0 conn(0x7f51480b2c30 0x7f514806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f5146d9d640 1 -- 172.21.15.37:0/2217359927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480af280 msgr2=0x7f514815df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:18.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f5146d9d640 1 --2- 172.21.15.37:0/2217359927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480af280 0x7f514815df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f5146d9d640 1 -- 172.21.15.37:0/2217359927 >> v2:172.21.15.203:3300/0 conn(0x7f51480b2c30 msgr2=0x7f514806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:18.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f5146d9d640 1 --2- 172.21.15.37:0/2217359927 >> v2:172.21.15.203:3300/0 conn(0x7f51480b2c30 0x7f514806bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f5146d9d640 1 -- 172.21.15.37:0/2217359927 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51480af660 con 0x7f514806c1e0 2024-04-25T09:03:18.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f5146d9d640 1 --2- 172.21.15.37:0/2217359927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f514806c1e0 0x7f5148161500 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f514806d7c0 tx=0x7f514001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5f4e6d4eb028e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514559a640 1 -- 172.21.15.37:0/2217359927 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51400ac070 con 0x7f514806c1e0 2024-04-25T09:03:18.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514559a640 1 -- 172.21.15.37:0/2217359927 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51400a8420 con 0x7f514806c1e0 2024-04-25T09:03:18.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514559a640 1 -- 172.21.15.37:0/2217359927 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51400a87f0 con 0x7f514806c1e0 2024-04-25T09:03:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514c9bc640 1 -- 172.21.15.37:0/2217359927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f514806c1e0 msgr2=0x7f5148161500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514c9bc640 1 --2- 172.21.15.37:0/2217359927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f514806c1e0 0x7f5148161500 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f514806d7c0 tx=0x7f514001c730 comp rx=0 tx=0).stop 2024-04-25T09:03:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514c9bc640 1 -- 172.21.15.37:0/2217359927 shutdown_connections 2024-04-25T09:03:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514c9bc640 1 --2- 172.21.15.37:0/2217359927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f514806c1e0 0x7f5148161500 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514c9bc640 1 --2- 172.21.15.37:0/2217359927 >> v2:172.21.15.203:3300/0 conn(0x7f51480b2c30 0x7f514806bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514c9bc640 1 --2- 172.21.15.37:0/2217359927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f51480af280 0x7f514815df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514c9bc640 1 -- 172.21.15.37:0/2217359927 >> 172.21.15.37:0/2217359927 conn(0x7f5148071810 msgr2=0x7f5148071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514c9bc640 1 -- 172.21.15.37:0/2217359927 shutdown_connections 2024-04-25T09:03:18.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.893+0000 7f514c9bc640 1 -- 172.21.15.37:0/2217359927 wait complete. 2024-04-25T09:03:18.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514c9bc640 1 Processor -- start 2024-04-25T09:03:18.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514c9bc640 1 -- start start 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514c9bc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f514806c1e0 0x7f5148152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514c9bc640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51480af280 0x7f514814d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514c9bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480b2c30 0x7f51481512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514c9bc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514806e090 con 0x7f51480b2c30 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514c9bc640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514806df10 con 0x7f51480af280 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514c9bc640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514806e210 con 0x7f514806c1e0 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480b2c30 0x7f51481512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5146d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480b2c30 0x7f51481512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33212/0 (socket says 172.21.15.37:33212) 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5146d9d640 1 -- 172.21.15.37:0/3809535278 learned_addr learned my addr 172.21.15.37:0/3809535278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514659c640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f514806c1e0 0x7f5148152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5145d9b640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51480af280 0x7f514814d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5146d9d640 1 -- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f514806c1e0 msgr2=0x7f5148152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5146d9d640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f514806c1e0 0x7f5148152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5146d9d640 1 -- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51480af280 msgr2=0x7f514814d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5146d9d640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51480af280 0x7f514814d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5146d9d640 1 -- 172.21.15.37:0/3809535278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5148154530 con 0x7f51480b2c30 2024-04-25T09:03:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5146d9d640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480b2c30 0x7f51481512a0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f514001ade0 tx=0x7f51400194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5136ffd640 1 -- 172.21.15.37:0/3809535278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51400ac040 con 0x7f51480b2c30 2024-04-25T09:03:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5136ffd640 1 -- 172.21.15.37:0/3809535278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51400bd450 con 0x7f51480b2c30 2024-04-25T09:03:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514c9bc640 1 -- 172.21.15.37:0/3809535278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5148153490 con 0x7f51480b2c30 2024-04-25T09:03:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f5136ffd640 1 -- 172.21.15.37:0/3809535278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5140009510 con 0x7f51480b2c30 2024-04-25T09:03:18.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514c9bc640 1 -- 172.21.15.37:0/3809535278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5148153a50 con 0x7f51480b2c30 2024-04-25T09:03:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.897+0000 7f514c9bc640 1 -- 172.21.15.37:0/3809535278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514804efa0 con 0x7f51480b2c30 2024-04-25T09:03:18.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.901+0000 7f5136ffd640 1 -- 172.21.15.37:0/3809535278 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f51400af070 con 0x7f51480b2c30 2024-04-25T09:03:18.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.901+0000 7f5136ffd640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5128066f60 0x7f5128069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.901+0000 7f5136ffd640 1 -- 172.21.15.37:0/3809535278 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f514011fa70 con 0x7f51480b2c30 2024-04-25T09:03:18.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.901+0000 7f514659c640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5128066f60 0x7f5128069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.901+0000 7f514659c640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5128066f60 0x7f5128069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f51480cc550 tx=0x7f513c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:18.905+0000 7f5136ffd640 1 -- 172.21.15.37:0/3809535278 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f51400165e0 con 0x7f51480b2c30 2024-04-25T09:03:19.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.097+0000 7f514c9bc640 1 -- 172.21.15.37:0/3809535278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51480727c0 con 0x7f51480b2c30 2024-04-25T09:03:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.097+0000 7f5136ffd640 1 -- 172.21.15.37:0/3809535278 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f51400f0890 con 0x7f51480b2c30 2024-04-25T09:03:19.088 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:19.088 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":89,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2524942703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2524942703},{"type":"v1","addr":"172.21.15.37:6835","nonce":2524942703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[{"mdsmap":{"epoch":88,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:34.915809+0000","modified":"2024-04-25T09:03:15.641847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":114,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:03:19.088 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 89 2024-04-25T09:03:19.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 -- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5128066f60 msgr2=0x7f5128069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5128066f60 0x7f5128069420 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f51480cc550 tx=0x7f513c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 -- 172.21.15.37:0/3809535278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480b2c30 msgr2=0x7f51481512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480b2c30 0x7f51481512a0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f514001ade0 tx=0x7f51400194d0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 -- 172.21.15.37:0/3809535278 shutdown_connections 2024-04-25T09:03:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5128066f60 0x7f5128069420 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51480b2c30 0x7f51481512a0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f51480af280 0x7f514814d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 --2- 172.21.15.37:0/3809535278 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f514806c1e0 0x7f5148152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 -- 172.21.15.37:0/3809535278 >> 172.21.15.37:0/3809535278 conn(0x7f5148071810 msgr2=0x7f514806b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 -- 172.21.15.37:0/3809535278 shutdown_connections 2024-04-25T09:03:19.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.101+0000 7f514c9bc640 1 -- 172.21.15.37:0/3809535278 wait complete. 2024-04-25T09:03:19.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:19.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.237+0000 7fe5cfbc0640 1 Processor -- start 2024-04-25T09:03:19.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.237+0000 7fe5cfbc0640 1 -- start start 2024-04-25T09:03:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe5c80d07f0 0x7fe5c80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe5c80d1cd0 0x7fe5c806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c806be60 con 0x7fe5c815da80 2024-04-25T09:03:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c806c440 con 0x7fe5c80d1cd0 2024-04-25T09:03:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c815de60 con 0x7fe5c80d07f0 2024-04-25T09:03:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5ce15d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe5c80d1cd0 0x7fe5c806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cd95c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe5c815da80 0x7fe5c80d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:42716/0 (socket says 172.21.15.37:42716) 2024-04-25T09:03:19.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cd95c640 1 -- 172.21.15.37:0/4051497005 learned_addr learned my addr 172.21.15.37:0/4051497005 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cd15b640 1 --2- 172.21.15.37:0/4051497005 >> v2:172.21.15.203:3300/0 conn(0x7fe5c80d07f0 0x7fe5c80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5ce15d640 1 -- 172.21.15.37:0/4051497005 >> v2:172.21.15.203:3300/0 conn(0x7fe5c80d07f0 msgr2=0x7fe5c80d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5ce15d640 1 --2- 172.21.15.37:0/4051497005 >> v2:172.21.15.203:3300/0 conn(0x7fe5c80d07f0 0x7fe5c80d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5ce15d640 1 -- 172.21.15.37:0/4051497005 >> v1:172.21.15.37:6789/0 conn(0x7fe5c815da80 legacy=0x7fe5c80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5ce15d640 1 -- 172.21.15.37:0/4051497005 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c80d2450 con 0x7fe5c80d1cd0 2024-04-25T09:03:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5ce15d640 1 --2- 172.21.15.37:0/4051497005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c80d1cd0 0x7fe5c806c8d0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe5c808d880 tx=0x7fe5c001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29dee6c12a842509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cc95a640 1 -- 172.21.15.37:0/4051497005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00b2070 con 0x7fe5c80d1cd0 2024-04-25T09:03:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cc95a640 1 -- 172.21.15.37:0/4051497005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c00a59e0 con 0x7fe5c80d1cd0 2024-04-25T09:03:19.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cc95a640 1 -- 172.21.15.37:0/4051497005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00a5db0 con 0x7fe5c80d1cd0 2024-04-25T09:03:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/4051497005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c80d1cd0 msgr2=0x7fe5c806c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 --2- 172.21.15.37:0/4051497005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c80d1cd0 0x7fe5c806c8d0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe5c808d880 tx=0x7fe5c001c900 comp rx=0 tx=0).stop 2024-04-25T09:03:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/4051497005 shutdown_connections 2024-04-25T09:03:19.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 --2- 172.21.15.37:0/4051497005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c80d1cd0 0x7fe5c806c8d0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 --2- 172.21.15.37:0/4051497005 >> v2:172.21.15.203:3300/0 conn(0x7fe5c80d07f0 0x7fe5c80d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/4051497005 >> 172.21.15.37:0/4051497005 conn(0x7fe5c808b000 msgr2=0x7fe5c80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/4051497005 shutdown_connections 2024-04-25T09:03:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.241+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/4051497005 wait complete. 2024-04-25T09:03:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cfbc0640 1 Processor -- start 2024-04-25T09:03:19.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cfbc0640 1 -- start start 2024-04-25T09:03:19.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cfbc0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c80d07f0 0x7fe5c8155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cfbc0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5c80d1cd0 0x7fe5c8156780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cfbc0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c815da80 0x7fe5c81594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cfbc0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c806d7a0 con 0x7fe5c80d07f0 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cfbc0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c806d620 con 0x7fe5c80d1cd0 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cfbc0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c806d920 con 0x7fe5c815da80 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cd95c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c80d07f0 0x7fe5c8155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cd95c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c80d07f0 0x7fe5c8155380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33224/0 (socket says 172.21.15.37:33224) 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cd95c640 1 -- 172.21.15.37:0/2931428287 learned_addr learned my addr 172.21.15.37:0/2931428287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cd95c640 1 -- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c815da80 msgr2=0x7fe5c81594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5ce15d640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c815da80 0x7fe5c81594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cd15b640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5c80d1cd0 0x7fe5c8156780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cd95c640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c815da80 0x7fe5c81594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cd95c640 1 -- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5c80d1cd0 msgr2=0x7fe5c8156780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cd95c640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5c80d1cd0 0x7fe5c8156780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cd95c640 1 -- 172.21.15.37:0/2931428287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c815f710 con 0x7fe5c80d07f0 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cd95c640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c80d07f0 0x7fe5c8155380 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c3a0 tx=0x7fe5c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5be7fc640 1 -- 172.21.15.37:0/2931428287 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4017070 con 0x7fe5c80d07f0 2024-04-25T09:03:19.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5be7fc640 1 -- 172.21.15.37:0/2931428287 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c4005cf0 con 0x7fe5c80d07f0 2024-04-25T09:03:19.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5be7fc640 1 -- 172.21.15.37:0/2931428287 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40072f0 con 0x7fe5c80d07f0 2024-04-25T09:03:19.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/2931428287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c815f9f0 con 0x7fe5c80d07f0 2024-04-25T09:03:19.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/2931428287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c815ffb0 con 0x7fe5c80d07f0 2024-04-25T09:03:19.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/2931428287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c804efa0 con 0x7fe5c80d07f0 2024-04-25T09:03:19.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.245+0000 7fe5be7fc640 1 -- 172.21.15.37:0/2931428287 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5c401a070 con 0x7fe5c80d07f0 2024-04-25T09:03:19.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.249+0000 7fe5be7fc640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe5ac066f60 0x7fe5ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.249+0000 7fe5be7fc640 1 -- 172.21.15.37:0/2931428287 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7fe5c4084a60 con 0x7fe5c80d07f0 2024-04-25T09:03:19.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.249+0000 7fe5cd15b640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe5ac066f60 0x7fe5ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.249+0000 7fe5cd15b640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe5ac066f60 0x7fe5ac069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe5c815a830 tx=0x7fe5b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.249+0000 7fe5be7fc640 1 -- 172.21.15.37:0/2931428287 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5c4021020 con 0x7fe5c80d07f0 2024-04-25T09:03:19.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.405+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/2931428287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5c80cf380 con 0x7fe5c80d07f0 2024-04-25T09:03:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.409+0000 7fe5be7fc640 1 -- 172.21.15.37:0/2931428287 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+17223 (secure 0 0 0) 0x7fe5c4055880 con 0x7fe5c80d07f0 2024-04-25T09:03:19.399 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:19.399 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":114,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:15.641872+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:33.212328+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:33.895814+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.29","mappings":[{"from":5,"to":4}]},{"pgid":"16.39","mappings":[{"from":1,"to":4},{"from":6,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2017261543":"2024-04-26T09:03:15.641825+0000","172.21.15.37:6834/2017261543":"2024-04-26T09:03:15.641825+0000","172.21.15.37:6835/1266122577":"2024-04-26T09:02:39.103789+0000","172.21.15.37:6834/1266122577":"2024-04-26T09:02:39.103789+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.409+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe5ac066f60 msgr2=0x7fe5ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.409+0000 7fe5cfbc0640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe5ac066f60 0x7fe5ac069420 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe5c815a830 tx=0x7fe5b8008040 comp rx=0 tx=0).stop 2024-04-25T09:03:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.413+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/2931428287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c80d07f0 msgr2=0x7fe5c8155380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.413+0000 7fe5cfbc0640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c80d07f0 0x7fe5c8155380 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c3a0 tx=0x7fe5c400c860 comp rx=0 tx=0).stop 2024-04-25T09:03:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.413+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/2931428287 shutdown_connections 2024-04-25T09:03:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.413+0000 7fe5cfbc0640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe5ac066f60 0x7fe5ac069420 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.413+0000 7fe5cfbc0640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe5c815da80 0x7fe5c81594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.413+0000 7fe5cfbc0640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe5c80d1cd0 0x7fe5c8156780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.413+0000 7fe5cfbc0640 1 --2- 172.21.15.37:0/2931428287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5c80d07f0 0x7fe5c8155380 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.413+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/2931428287 >> 172.21.15.37:0/2931428287 conn(0x7fe5c808b000 msgr2=0x7fe5c80b4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.413+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/2931428287 shutdown_connections 2024-04-25T09:03:19.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.413+0000 7fe5cfbc0640 1 -- 172.21.15.37:0/2931428287 wait complete. 2024-04-25T09:03:19.417 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:03:19.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.541+0000 7f88b6fa2640 1 Processor -- start 2024-04-25T09:03:19.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.541+0000 7f88b6fa2640 1 -- start start 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.541+0000 7f88b6fa2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f88b00ce510 0x7f88b00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.541+0000 7f88b6fa2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f88b01451f0 0x7f88b0142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.541+0000 7f88b6fa2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b00cee30 con 0x7f88b0144ca0 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.541+0000 7f88b6fa2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b00cc2b0 con 0x7f88b00ce510 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.541+0000 7f88b6fa2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b006e290 con 0x7f88b01451f0 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.541+0000 7f88b4d3e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f88b00ce510 0x7f88b00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b4d3e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f88b00ce510 0x7f88b00ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43580/0 (socket says 172.21.15.37:43580) 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b4d3e640 1 -- 172.21.15.37:0/674948394 learned_addr learned my addr 172.21.15.37:0/674948394 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b553f640 1 --2- 172.21.15.37:0/674948394 >> v2:172.21.15.203:3300/0 conn(0x7f88b01451f0 0x7f88b0142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b4d3e640 1 -- 172.21.15.37:0/674948394 >> v2:172.21.15.203:3300/0 conn(0x7f88b01451f0 msgr2=0x7f88b0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b4d3e640 1 --2- 172.21.15.37:0/674948394 >> v2:172.21.15.203:3300/0 conn(0x7f88b01451f0 0x7f88b0142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b4d3e640 1 -- 172.21.15.37:0/674948394 >> v1:172.21.15.203:6789/0 conn(0x7f88b0144ca0 legacy=0x7f88b006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b4d3e640 1 -- 172.21.15.37:0/674948394 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b00ccc70 con 0x7f88b00ce510 2024-04-25T09:03:19.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b553f640 1 --2- 172.21.15.37:0/674948394 >> v2:172.21.15.203:3300/0 conn(0x7f88b01451f0 0x7f88b0142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b4d3e640 1 --2- 172.21.15.37:0/674948394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b00ce510 0x7f88b00ce8f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f88b00b48f0 tx=0x7f88a001ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a146002971f9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88af7fe640 1 -- 172.21.15.37:0/674948394 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a00b5070 con 0x7f88b00ce510 2024-04-25T09:03:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88af7fe640 1 -- 172.21.15.37:0/674948394 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a00a89d0 con 0x7f88b00ce510 2024-04-25T09:03:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88af7fe640 1 -- 172.21.15.37:0/674948394 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a00a8da0 con 0x7f88b00ce510 2024-04-25T09:03:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b6fa2640 1 -- 172.21.15.37:0/674948394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b00ce510 msgr2=0x7f88b00ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b6fa2640 1 --2- 172.21.15.37:0/674948394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b00ce510 0x7f88b00ce8f0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f88b00b48f0 tx=0x7f88a001ecd0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b6fa2640 1 -- 172.21.15.37:0/674948394 shutdown_connections 2024-04-25T09:03:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b6fa2640 1 --2- 172.21.15.37:0/674948394 >> v2:172.21.15.203:3300/0 conn(0x7f88b01451f0 0x7f88b0142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b6fa2640 1 --2- 172.21.15.37:0/674948394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b00ce510 0x7f88b00ce8f0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b6fa2640 1 -- 172.21.15.37:0/674948394 >> 172.21.15.37:0/674948394 conn(0x7f88b00b11a0 msgr2=0x7f88b00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:19.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b6fa2640 1 -- 172.21.15.37:0/674948394 shutdown_connections 2024-04-25T09:03:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.545+0000 7f88b6fa2640 1 -- 172.21.15.37:0/674948394 wait complete. 2024-04-25T09:03:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b6fa2640 1 Processor -- start 2024-04-25T09:03:19.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b6fa2640 1 -- start start 2024-04-25T09:03:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b6fa2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b00ce510 0x7f88b013f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b6fa2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88b0144ca0 0x7f88b0142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b6fa2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b01451f0 0x7f88b00c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b6fa2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b006ee30 con 0x7f88b01451f0 2024-04-25T09:03:19.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b6fa2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b006df20 con 0x7f88b00ce510 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b6fa2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b00cc900 con 0x7f88b0144ca0 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b553f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b01451f0 0x7f88b00c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b553f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b01451f0 0x7f88b00c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43586/0 (socket says 172.21.15.37:43586) 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b553f640 1 -- 172.21.15.37:0/4156161729 learned_addr learned my addr 172.21.15.37:0/4156161729 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88affff640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88b0144ca0 0x7f88b0142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b4d3e640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b00ce510 0x7f88b013f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b553f640 1 -- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88b0144ca0 msgr2=0x7f88b0142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b553f640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88b0144ca0 0x7f88b0142430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b553f640 1 -- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b00ce510 msgr2=0x7f88b013f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b553f640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b00ce510 0x7f88b013f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b553f640 1 -- 172.21.15.37:0/4156161729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b0140370 con 0x7f88b01451f0 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b553f640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b01451f0 0x7f88b00c8c20 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f88a800cd80 tx=0x7f88a800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88ad7fa640 1 -- 172.21.15.37:0/4156161729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8003ba0 con 0x7f88b01451f0 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88ad7fa640 1 -- 172.21.15.37:0/4156161729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a8003d60 con 0x7f88b01451f0 2024-04-25T09:03:19.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b6fa2640 1 -- 172.21.15.37:0/4156161729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b0140650 con 0x7f88b01451f0 2024-04-25T09:03:19.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88ad7fa640 1 -- 172.21.15.37:0/4156161729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a8004260 con 0x7f88b01451f0 2024-04-25T09:03:19.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b6fa2640 1 -- 172.21.15.37:0/4156161729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b0140c10 con 0x7f88b01451f0 2024-04-25T09:03:19.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.549+0000 7f88b6fa2640 1 -- 172.21.15.37:0/4156161729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b0141100 con 0x7f88b01451f0 2024-04-25T09:03:19.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.553+0000 7f88ad7fa640 1 -- 172.21.15.37:0/4156161729 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88a8004400 con 0x7f88b01451f0 2024-04-25T09:03:19.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.553+0000 7f88ad7fa640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f889c066f60 0x7f889c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.553+0000 7f88ad7fa640 1 -- 172.21.15.37:0/4156161729 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f88a80860c0 con 0x7f88b01451f0 2024-04-25T09:03:19.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.553+0000 7f88b4d3e640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f889c066f60 0x7f889c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.553+0000 7f88b4d3e640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f889c066f60 0x7f889c069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f88a00b2000 tx=0x7f88a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.557+0000 7f88ad7fa640 1 -- 172.21.15.37:0/4156161729 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88a8014030 con 0x7f88b01451f0 2024-04-25T09:03:19.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:19.749+0000 7f88b6fa2640 1 -- 172.21.15.37:0/4156161729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f88b0142210 con 0x7f88b01451f0 2024-04-25T09:03:20.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.677+0000 7f88ad7fa640 1 -- 172.21.15.37:0/4156161729 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) v1 ==== 161+0+0 (secure 0 0 0) 0x7f88a8056fc0 con 0x7f88b01451f0 2024-04-25T09:03:20.668 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:03:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 -- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f889c066f60 msgr2=0x7f889c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f889c066f60 0x7f889c069420 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f88a00b2000 tx=0x7f88a0008040 comp rx=0 tx=0).stop 2024-04-25T09:03:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 -- 172.21.15.37:0/4156161729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b01451f0 msgr2=0x7f88b00c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b01451f0 0x7f88b00c8c20 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f88a800cd80 tx=0x7f88a800a650 comp rx=0 tx=0).stop 2024-04-25T09:03:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 -- 172.21.15.37:0/4156161729 shutdown_connections 2024-04-25T09:03:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f889c066f60 0x7f889c069420 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88b01451f0 0x7f88b00c8c20 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88b0144ca0 0x7f88b0142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 --2- 172.21.15.37:0/4156161729 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88b00ce510 0x7f88b013f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 -- 172.21.15.37:0/4156161729 >> 172.21.15.37:0/4156161729 conn(0x7f88b00b11a0 msgr2=0x7f88b00b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 -- 172.21.15.37:0/4156161729 shutdown_connections 2024-04-25T09:03:20.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.681+0000 7f88b6fa2640 1 -- 172.21.15.37:0/4156161729 wait complete. 2024-04-25T09:03:20.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:03:20.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.837+0000 7f9264140640 1 Processor -- start 2024-04-25T09:03:20.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.837+0000 7f9264140640 1 -- start start 2024-04-25T09:03:20.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9264140640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f925c0cfa20 0x7f925c072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9264140640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c0cfe90 con 0x7f925c15b6b0 2024-04-25T09:03:20.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9264140640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c0cbd20 con 0x7f925c0cf590 2024-04-25T09:03:20.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9264140640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925c15ba90 con 0x7f925c0cfa20 2024-04-25T09:03:20.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9261edc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f925c15b6b0 0x7f925c0ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45874/0 (socket says 172.21.15.37:45874) 2024-04-25T09:03:20.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9261edc640 1 -- 172.21.15.37:0/3023951565 learned_addr learned my addr 172.21.15.37:0/3023951565 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:20.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f92626dd640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f925c0cfa20 0x7f925c072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875082386 0 0) 0x7f925c0cfe90 con 0x7f925c15b6b0 2024-04-25T09:03:20.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9248003530 con 0x7f925c15b6b0 2024-04-25T09:03:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1521752848 0 0) 0x7f9248003530 con 0x7f925c15b6b0 2024-04-25T09:03:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f925c0cfa20 msgr2=0x7f925c072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f925c0cfa20 0x7f925c072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 >> v1:172.21.15.203:6789/0 conn(0x7f925c0cf590 legacy=0x7f925c072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c0cc000 con 0x7f925c15b6b0 2024-04-25T09:03:20.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f9258001680 con 0x7f925c15b6b0 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 >> v1:172.21.15.37:6789/0 conn(0x7f925c15b6b0 legacy=0x7f925c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9248003a60 0x7f9248003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f925c0cfa20 0x7f9248004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9248004d50 0x7f92480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c15ba90 con 0x7f9248004d50 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924800ba80 con 0x7f9248003a60 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924800bc30 con 0x7f925c0cfa20 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f92626dd640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9248004d50 0x7f92480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9261edc640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9248003a60 0x7f9248003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f92616db640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f925c0cfa20 0x7f9248004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f92626dd640 1 -- 172.21.15.37:0/3023951565 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f925c0cfa20 msgr2=0x7f9248004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f92626dd640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f925c0cfa20 0x7f9248004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f92626dd640 1 -- 172.21.15.37:0/3023951565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9248003a60 msgr2=0x7f9248003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:20.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f92626dd640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9248003a60 0x7f9248003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f92626dd640 1 -- 172.21.15.37:0/3023951565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9248003530 con 0x7f9248004d50 2024-04-25T09:03:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f92626dd640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9248004d50 0x7f92480051d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f9254009fc0 tx=0x7f925401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6d2df960165675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92540b2070 con 0x7f9248004d50 2024-04-25T09:03:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92540a5b80 con 0x7f9248004d50 2024-04-25T09:03:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.841+0000 7f9260eda640 1 -- 172.21.15.37:0/3023951565 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254009c10 con 0x7f9248004d50 2024-04-25T09:03:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 -- 172.21.15.37:0/3023951565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9248004d50 msgr2=0x7f92480051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 --2- 172.21.15.37:0/3023951565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9248004d50 0x7f92480051d0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f9254009fc0 tx=0x7f925401cc80 comp rx=0 tx=0).stop 2024-04-25T09:03:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92626dd640 1 -- 172.21.15.37:0/3023951565 reap_dead start 2024-04-25T09:03:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 -- 172.21.15.37:0/3023951565 shutdown_connections 2024-04-25T09:03:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 -- 172.21.15.37:0/3023951565 >> 172.21.15.37:0/3023951565 conn(0x7f925c06ae60 msgr2=0x7f925c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:20.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 -- 172.21.15.37:0/3023951565 shutdown_connections 2024-04-25T09:03:20.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 -- 172.21.15.37:0/3023951565 wait complete. 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 Processor -- start 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 -- start start 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9248004d50 0x7f925c0adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f925c0c2e70 0x7f925c15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c1f4250 0x7f925c1f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0cc8a0 con 0x7f925c0c2e70 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0cc5c0 con 0x7f925c1f4250 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0cba30 con 0x7f9248004d50 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92616db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f925c0c2e70 0x7f925c15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9261edc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9248004d50 0x7f925c0adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9261edc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9248004d50 0x7f925c0adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:46722/0 (socket says 172.21.15.37:46722) 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9261edc640 1 -- 172.21.15.37:0/2137874638 learned_addr learned my addr 172.21.15.37:0/2137874638 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:20.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92616db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f925c0c2e70 0x7f925c15a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43612/0 (socket says 172.21.15.37:43612) 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92616db640 1 -- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9248004d50 msgr2=0x7f925c0adfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92616db640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9248004d50 0x7f925c0adfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92626dd640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c1f4250 0x7f925c1f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92616db640 1 -- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c1f4250 msgr2=0x7f925c1f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92616db640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c1f4250 0x7f925c1f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92616db640 1 -- 172.21.15.37:0/2137874638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c0c32f0 con 0x7f925c0c2e70 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92626dd640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c1f4250 0x7f925c1f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92616db640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f925c0c2e70 0x7f925c15a530 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e490 tx=0x7f924c00e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92527fc640 1 -- 172.21.15.37:0/2137874638 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c019070 con 0x7f925c0c2e70 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92527fc640 1 -- 172.21.15.37:0/2137874638 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c0052e0 con 0x7f925c0c2e70 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 -- 172.21.15.37:0/2137874638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c0c53e0 con 0x7f925c0c2e70 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f92527fc640 1 -- 172.21.15.37:0/2137874638 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c003840 con 0x7f925c0c2e70 2024-04-25T09:03:20.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.845+0000 7f9264140640 1 -- 172.21.15.37:0/2137874638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c0c5960 con 0x7f925c0c2e70 2024-04-25T09:03:20.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.849+0000 7f9264140640 1 -- 172.21.15.37:0/2137874638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925c04efa0 con 0x7f925c0c2e70 2024-04-25T09:03:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.849+0000 7f92527fc640 1 -- 172.21.15.37:0/2137874638 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f924c01c070 con 0x7f925c0c2e70 2024-04-25T09:03:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.849+0000 7f92527fc640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9240066f60 0x7f9240069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.849+0000 7f92527fc640 1 -- 172.21.15.37:0/2137874638 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f924c0849d0 con 0x7f925c0c2e70 2024-04-25T09:03:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.849+0000 7f9261edc640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9240066f60 0x7f9240069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:20.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.849+0000 7f9261edc640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9240066f60 0x7f9240069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f92580017f0 tx=0x7f9258001780 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:20.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:20.853+0000 7f92527fc640 1 -- 172.21.15.37:0/2137874638 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f925c0c32f0 con 0x7f925c0c2e70 2024-04-25T09:03:21.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.045+0000 7f9264140640 1 -- 172.21.15.37:0/2137874638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f925c15fe40 con 0x7f925c0c2e70 2024-04-25T09:03:21.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.681+0000 7f92527fc640 1 -- 172.21.15.37:0/2137874638 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v91) v1 ==== 104+0+0 (secure 0 0 0) 0x7f924c0556c0 con 0x7f925c0c2e70 2024-04-25T09:03:21.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 -- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9240066f60 msgr2=0x7f9240069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9240066f60 0x7f9240069420 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f92580017f0 tx=0x7f9258001780 comp rx=0 tx=0).stop 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 -- 172.21.15.37:0/2137874638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f925c0c2e70 msgr2=0x7f925c15a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f925c0c2e70 0x7f925c15a530 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f924c00e490 tx=0x7f924c00e890 comp rx=0 tx=0).stop 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 -- 172.21.15.37:0/2137874638 shutdown_connections 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9240066f60 0x7f9240069420 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f925c1f4250 0x7f925c1f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f925c0c2e70 0x7f925c15a530 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 --2- 172.21.15.37:0/2137874638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9248004d50 0x7f925c0adfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 -- 172.21.15.37:0/2137874638 >> 172.21.15.37:0/2137874638 conn(0x7f925c06ae60 msgr2=0x7f925c0b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 -- 172.21.15.37:0/2137874638 shutdown_connections 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.685+0000 7f9264140640 1 -- 172.21.15.37:0/2137874638 wait complete. 2024-04-25T09:03:21.698 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:03:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.841+0000 7fcbf04d8640 1 Processor -- start 2024-04-25T09:03:21.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.841+0000 7fcbf04d8640 1 -- start start 2024-04-25T09:03:21.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.841+0000 7fcbf04d8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80cfc20 0x7fcbe80ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.841+0000 7fcbf04d8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcbe80ce790 0x7fcbe80b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.841+0000 7fcbf04d8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe80b3f70 con 0x7fcbe80b3b90 2024-04-25T09:03:21.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.841+0000 7fcbf04d8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe80cef80 con 0x7fcbe80ce790 2024-04-25T09:03:21.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.841+0000 7fcbf04d8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe806de70 con 0x7fcbe80cfc20 2024-04-25T09:03:21.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbeda73640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80cfc20 0x7fcbe80ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbeea75640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcbe80ce790 0x7fcbe80b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbeda73640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80cfc20 0x7fcbe80ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:46730/0 (socket says 172.21.15.37:46730) 2024-04-25T09:03:21.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbeda73640 1 -- 172.21.15.37:0/2785418581 learned_addr learned my addr 172.21.15.37:0/2785418581 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:21.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbeea75640 1 -- 172.21.15.37:0/2785418581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80cfc20 msgr2=0x7fcbe80ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:21.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbeea75640 1 --2- 172.21.15.37:0/2785418581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80cfc20 0x7fcbe80ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbeea75640 1 -- 172.21.15.37:0/2785418581 >> v1:172.21.15.203:6789/0 conn(0x7fcbe80b3b90 legacy=0x7fcbe80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbeea75640 1 -- 172.21.15.37:0/2785418581 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe806e150 con 0x7fcbe80ce790 2024-04-25T09:03:21.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbeda73640 1 --2- 172.21.15.37:0/2785418581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80cfc20 0x7fcbe80ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:21.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbeea75640 1 --2- 172.21.15.37:0/2785418581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80ce790 0x7fcbe80b2190 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009d90 tx=0x7fcbe001c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cbfb9e7ab91cdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:21.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbed272640 1 -- 172.21.15.37:0/2785418581 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00aa670 con 0x7fcbe80ce790 2024-04-25T09:03:21.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbed272640 1 -- 172.21.15.37:0/2785418581 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe00af070 con 0x7fcbe80ce790 2024-04-25T09:03:21.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.845+0000 7fcbed272640 1 -- 172.21.15.37:0/2785418581 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00aaa30 con 0x7fcbe80ce790 2024-04-25T09:03:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 -- 172.21.15.37:0/2785418581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80ce790 msgr2=0x7fcbe80b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 --2- 172.21.15.37:0/2785418581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80ce790 0x7fcbe80b2190 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fcbe0009d90 tx=0x7fcbe001c8e0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 -- 172.21.15.37:0/2785418581 shutdown_connections 2024-04-25T09:03:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 --2- 172.21.15.37:0/2785418581 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80ce790 0x7fcbe80b2190 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 --2- 172.21.15.37:0/2785418581 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80cfc20 0x7fcbe80ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 -- 172.21.15.37:0/2785418581 >> 172.21.15.37:0/2785418581 conn(0x7fcbe806ae60 msgr2=0x7fcbe806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:21.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 -- 172.21.15.37:0/2785418581 shutdown_connections 2024-04-25T09:03:21.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 -- 172.21.15.37:0/2785418581 wait complete. 2024-04-25T09:03:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 Processor -- start 2024-04-25T09:03:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 -- start start 2024-04-25T09:03:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80b3b90 0x7fcbe8098560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbe80ce790 0x7fcbe815cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80cfc20 0x7fcbe815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe806edb0 con 0x7fcbe80ce790 2024-04-25T09:03:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe806ec30 con 0x7fcbe80cfc20 2024-04-25T09:03:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe816a780 con 0x7fcbe80b3b90 2024-04-25T09:03:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbeda73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbe80ce790 0x7fcbe815cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbeda73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbe80ce790 0x7fcbe815cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43618/0 (socket says 172.21.15.37:43618) 2024-04-25T09:03:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbeda73640 1 -- 172.21.15.37:0/3888086201 learned_addr learned my addr 172.21.15.37:0/3888086201 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:21.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbee274640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80b3b90 0x7fcbe8098560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbeea75640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80cfc20 0x7fcbe815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbeda73640 1 -- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80b3b90 msgr2=0x7fcbe8098560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbeda73640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80b3b90 0x7fcbe8098560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbeda73640 1 -- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80cfc20 msgr2=0x7fcbe815d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:21.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbeda73640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80cfc20 0x7fcbe815d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:21.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbeda73640 1 -- 172.21.15.37:0/3888086201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe8098aa0 con 0x7fcbe80ce790 2024-04-25T09:03:21.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbeda73640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbe80ce790 0x7fcbe815cd60 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800ea00 tx=0x7fcbd800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:21.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbdeffd640 1 -- 172.21.15.37:0/3888086201 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd8017070 con 0x7fcbe80ce790 2024-04-25T09:03:21.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbdeffd640 1 -- 172.21.15.37:0/3888086201 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbd800ca60 con 0x7fcbe80ce790 2024-04-25T09:03:21.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 -- 172.21.15.37:0/3888086201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe809ae80 con 0x7fcbe80ce790 2024-04-25T09:03:21.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbdeffd640 1 -- 172.21.15.37:0/3888086201 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbd800ce10 con 0x7fcbe80ce790 2024-04-25T09:03:21.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.849+0000 7fcbf04d8640 1 -- 172.21.15.37:0/3888086201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe809b440 con 0x7fcbe80ce790 2024-04-25T09:03:21.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.853+0000 7fcbf04d8640 1 -- 172.21.15.37:0/3888086201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe8067c30 con 0x7fcbe80ce790 2024-04-25T09:03:21.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.853+0000 7fcbdeffd640 1 -- 172.21.15.37:0/3888086201 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcbd801a070 con 0x7fcbe80ce790 2024-04-25T09:03:21.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.853+0000 7fcbdeffd640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbcc066f60 0x7fcbcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:21.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.853+0000 7fcbdeffd640 1 -- 172.21.15.37:0/3888086201 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6339+0+0 (secure 0 0 0) 0x7fcbd8085a00 con 0x7fcbe80ce790 2024-04-25T09:03:21.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.853+0000 7fcbee274640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbcc066f60 0x7fcbcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:21.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.853+0000 7fcbee274640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbcc066f60 0x7fcbcc069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcbe815dc00 tx=0x7fcbe4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:21.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:21.857+0000 7fcbdeffd640 1 -- 172.21.15.37:0/3888086201 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcbd8056770 con 0x7fcbe80ce790 2024-04-25T09:03:22.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.017+0000 7fcbf04d8640 1 -- 172.21.15.37:0/3888086201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcbe806d860 con 0x7fcbe80ce790 2024-04-25T09:03:22.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.693+0000 7fcbdeffd640 1 -- 172.21.15.37:0/3888086201 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcbd8056070 con 0x7fcbe80ce790 2024-04-25T09:03:22.684 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:03:22.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.697+0000 7fcbf04d8640 1 -- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbcc066f60 msgr2=0x7fcbcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.697+0000 7fcbf04d8640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbcc066f60 0x7fcbcc069420 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcbe815dc00 tx=0x7fcbe4009040 comp rx=0 tx=0).stop 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.697+0000 7fcbf04d8640 1 -- 172.21.15.37:0/3888086201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbe80ce790 msgr2=0x7fcbe815cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.697+0000 7fcbf04d8640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbe80ce790 0x7fcbe815cd60 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fcbd800ea00 tx=0x7fcbd800eec0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.697+0000 7fcbf04d8640 1 -- 172.21.15.37:0/3888086201 shutdown_connections 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.697+0000 7fcbf04d8640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbcc066f60 0x7fcbcc069420 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.697+0000 7fcbf04d8640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbe80cfc20 0x7fcbe815d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.697+0000 7fcbf04d8640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbe80ce790 0x7fcbe815cd60 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.697+0000 7fcbf04d8640 1 --2- 172.21.15.37:0/3888086201 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbe80b3b90 0x7fcbe8098560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.697+0000 7fcbf04d8640 1 -- 172.21.15.37:0/3888086201 >> 172.21.15.37:0/3888086201 conn(0x7fcbe806ae60 msgr2=0x7fcbe80ac170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.701+0000 7fcbf04d8640 1 -- 172.21.15.37:0/3888086201 shutdown_connections 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.701+0000 7fcbf04d8640 1 -- 172.21.15.37:0/3888086201 wait complete. 2024-04-25T09:03:22.711 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:03:22.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde76105640 1 Processor -- start 2024-04-25T09:03:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde76105640 1 -- start start 2024-04-25T09:03:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde76105640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fde7009a940 0x7fde70099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde76105640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde7009a3b0 con 0x7fde7015b840 2024-04-25T09:03:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde76105640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde700cd400 con 0x7fde70099fb0 2024-04-25T09:03:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde76105640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde700b08e0 con 0x7fde7009a940 2024-04-25T09:03:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde6f7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fde7015b840 0x7fde700ccbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45884/0 (socket says 172.21.15.37:45884) 2024-04-25T09:03:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde6f7fe640 1 -- 172.21.15.37:0/3369308421 learned_addr learned my addr 172.21.15.37:0/3369308421 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:22.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde6effd640 1 --2- 172.21.15.37:0/3369308421 >> v2:172.21.15.37:3300/0 conn(0x7fde7009a940 0x7fde70099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde6effd640 1 -- 172.21.15.37:0/3369308421 >> v1:172.21.15.203:6789/0 conn(0x7fde70099fb0 legacy=0x7fde700b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde6effd640 1 -- 172.21.15.37:0/3369308421 >> v1:172.21.15.37:6789/0 conn(0x7fde7015b840 legacy=0x7fde700ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:22.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.857+0000 7fde6effd640 1 -- 172.21.15.37:0/3369308421 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde700b0b30 con 0x7fde7009a940 2024-04-25T09:03:22.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.861+0000 7fde6effd640 1 --2- 172.21.15.37:0/3369308421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde7009a940 0x7fde70099a70 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fde64009d90 tx=0x7fde6401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395b5ffdd689778e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:22.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.861+0000 7fde6e7fc640 1 -- 172.21.15.37:0/3369308421 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde640b2070 con 0x7fde7009a940 2024-04-25T09:03:22.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.861+0000 7fde6e7fc640 1 -- 172.21.15.37:0/3369308421 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde640a57f0 con 0x7fde7009a940 2024-04-25T09:03:22.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.861+0000 7fde6e7fc640 1 -- 172.21.15.37:0/3369308421 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde640a5bc0 con 0x7fde7009a940 2024-04-25T09:03:22.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.861+0000 7fde76105640 1 -- 172.21.15.37:0/3369308421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde7009a940 msgr2=0x7fde70099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.861+0000 7fde76105640 1 --2- 172.21.15.37:0/3369308421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde7009a940 0x7fde70099a70 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fde64009d90 tx=0x7fde6401c8a0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.861+0000 7fde76105640 1 -- 172.21.15.37:0/3369308421 shutdown_connections 2024-04-25T09:03:22.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.861+0000 7fde76105640 1 --2- 172.21.15.37:0/3369308421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde7009a940 0x7fde70099a70 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.861+0000 7fde76105640 1 -- 172.21.15.37:0/3369308421 >> 172.21.15.37:0/3369308421 conn(0x7fde700b3030 msgr2=0x7fde700b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:22.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.861+0000 7fde76105640 1 -- 172.21.15.37:0/3369308421 shutdown_connections 2024-04-25T09:03:22.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.861+0000 7fde76105640 1 -- 172.21.15.37:0/3369308421 wait complete. 2024-04-25T09:03:22.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde76105640 1 Processor -- start 2024-04-25T09:03:22.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde76105640 1 -- start start 2024-04-25T09:03:22.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde76105640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70099fb0 0x7fde700b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde76105640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde7009a940 0x7fde70159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde76105640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde7015b840 0x7fde7015a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde76105640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde700b2940 con 0x7fde7015b840 2024-04-25T09:03:22.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde76105640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde700b27c0 con 0x7fde70099fb0 2024-04-25T09:03:22.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde76105640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde70169410 con 0x7fde7009a940 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde7015b840 0x7fde7015a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde7015b840 0x7fde7015a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43634/0 (socket says 172.21.15.37:43634) 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6ffff640 1 -- 172.21.15.37:0/2823208820 learned_addr learned my addr 172.21.15.37:0/2823208820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6f7fe640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70099fb0 0x7fde700b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6effd640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde7009a940 0x7fde70159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6ffff640 1 -- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde7009a940 msgr2=0x7fde70159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6ffff640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde7009a940 0x7fde70159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6ffff640 1 -- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70099fb0 msgr2=0x7fde700b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6ffff640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70099fb0 0x7fde700b5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6ffff640 1 -- 172.21.15.37:0/2823208820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde700b4160 con 0x7fde7015b840 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6f7fe640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70099fb0 0x7fde700b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde6ffff640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde7015b840 0x7fde7015a410 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fde6800c370 tx=0x7fde6800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde57fff640 1 -- 172.21.15.37:0/2823208820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68005220 con 0x7fde7015b840 2024-04-25T09:03:22.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde57fff640 1 -- 172.21.15.37:0/2823208820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde680053c0 con 0x7fde7015b840 2024-04-25T09:03:22.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde76105640 1 -- 172.21.15.37:0/2823208820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde700b4440 con 0x7fde7015b840 2024-04-25T09:03:22.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde57fff640 1 -- 172.21.15.37:0/2823208820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68007270 con 0x7fde7015b840 2024-04-25T09:03:22.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde76105640 1 -- 172.21.15.37:0/2823208820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7006cd50 con 0x7fde7015b840 2024-04-25T09:03:22.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde76105640 1 -- 172.21.15.37:0/2823208820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde70068340 con 0x7fde7015b840 2024-04-25T09:03:22.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.865+0000 7fde57fff640 1 -- 172.21.15.37:0/2823208820 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fde68019020 con 0x7fde7015b840 2024-04-25T09:03:22.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.869+0000 7fde57fff640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde5c066f60 0x7fde5c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:22.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.869+0000 7fde57fff640 1 -- 172.21.15.37:0/2823208820 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5901+0+0 (secure 0 0 0) 0x7fde68084360 con 0x7fde7015b840 2024-04-25T09:03:22.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.869+0000 7fde6f7fe640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde5c066f60 0x7fde5c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:22.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.869+0000 7fde6f7fe640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde5c066f60 0x7fde5c069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fde6000a850 tx=0x7fde60008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:22.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:22.873+0000 7fde57fff640 1 -- 172.21.15.37:0/2823208820 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fde68055320 con 0x7fde7015b840 2024-04-25T09:03:23.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.037+0000 7fde76105640 1 -- 172.21.15.37:0/2823208820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fde70156ed0 con 0x7fde7015b840 2024-04-25T09:03:23.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.717+0000 7fde57fff640 1 -- 172.21.15.37:0/2823208820 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) v1 ==== 169+0+0 (secure 0 0 0) 0x7fde68054c20 con 0x7fde7015b840 2024-04-25T09:03:23.708 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T09:03:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.721+0000 7fde76105640 1 -- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde5c066f60 msgr2=0x7fde5c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.721+0000 7fde76105640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde5c066f60 0x7fde5c069420 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fde6000a850 tx=0x7fde60008040 comp rx=0 tx=0).stop 2024-04-25T09:03:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.721+0000 7fde76105640 1 -- 172.21.15.37:0/2823208820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde7015b840 msgr2=0x7fde7015a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.721+0000 7fde76105640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde7015b840 0x7fde7015a410 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fde6800c370 tx=0x7fde6800c830 comp rx=0 tx=0).stop 2024-04-25T09:03:23.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.721+0000 7fde76105640 1 -- 172.21.15.37:0/2823208820 shutdown_connections 2024-04-25T09:03:23.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.721+0000 7fde76105640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fde5c066f60 0x7fde5c069420 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.721+0000 7fde76105640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde7015b840 0x7fde7015a410 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.721+0000 7fde76105640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fde7009a940 0x7fde70159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.721+0000 7fde76105640 1 --2- 172.21.15.37:0/2823208820 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fde70099fb0 0x7fde700b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.721+0000 7fde76105640 1 -- 172.21.15.37:0/2823208820 >> 172.21.15.37:0/2823208820 conn(0x7fde700b3030 msgr2=0x7fde700cdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.725+0000 7fde76105640 1 -- 172.21.15.37:0/2823208820 shutdown_connections 2024-04-25T09:03:23.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.725+0000 7fde76105640 1 -- 172.21.15.37:0/2823208820 wait complete. 2024-04-25T09:03:23.745 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:03:23.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.877+0000 7fa7707ae640 1 Processor -- start 2024-04-25T09:03:23.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.877+0000 7fa7707ae640 1 -- start start 2024-04-25T09:03:23.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.877+0000 7fa7707ae640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa7680cfa20 0x7fa768072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.877+0000 7fa7707ae640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7680cbd20 con 0x7fa7680cf590 2024-04-25T09:03:23.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.877+0000 7fa7707ae640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7680cc000 con 0x7fa76815b6b0 2024-04-25T09:03:23.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.877+0000 7fa7707ae640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76815ba90 con 0x7fa7680cfa20 2024-04-25T09:03:23.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.877+0000 7fa76dd49640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa7680cf590 0x7fa768072000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45898/0 (socket says 172.21.15.37:45898) 2024-04-25T09:03:23.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.877+0000 7fa76dd49640 1 -- 172.21.15.37:0/2440449025 learned_addr learned my addr 172.21.15.37:0/2440449025 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:23.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76ed4b640 1 --2- 172.21.15.37:0/2440449025 >> v2:172.21.15.203:3300/0 conn(0x7fa7680cfa20 0x7fa768072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800945589 0 0) 0x7fa7680cbd20 con 0x7fa7680cf590 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa754003530 con 0x7fa7680cf590 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939225506 0 0) 0x7fa754003530 con 0x7fa7680cf590 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 >> v2:172.21.15.203:3300/0 conn(0x7fa7680cfa20 msgr2=0x7fa768072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 --2- 172.21.15.37:0/2440449025 >> v2:172.21.15.203:3300/0 conn(0x7fa7680cfa20 0x7fa768072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 >> v1:172.21.15.203:6789/0 conn(0x7fa76815b6b0 legacy=0x7fa7680ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7680cc2e0 con 0x7fa7680cf590 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fa75c001a00 con 0x7fa7680cf590 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 >> v1:172.21.15.37:6789/0 conn(0x7fa7680cf590 legacy=0x7fa768072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 --2- 172.21.15.37:0/2440449025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa754003a30 0x7fa754003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 --2- 172.21.15.37:0/2440449025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa754004810 0x7fa754004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 --2- 172.21.15.37:0/2440449025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7540051d0 0x7fa75400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76815ba90 con 0x7fa754004810 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75400c100 con 0x7fa7540051d0 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75400c300 con 0x7fa754003a30 2024-04-25T09:03:23.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76dd49640 1 --2- 172.21.15.37:0/2440449025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa754004810 0x7fa754004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76e54a640 1 --2- 172.21.15.37:0/2440449025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa754003a30 0x7fa754003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76dd49640 1 -- 172.21.15.37:0/2440449025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa754003a30 msgr2=0x7fa754003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:23.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76dd49640 1 --2- 172.21.15.37:0/2440449025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa754003a30 0x7fa754003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76dd49640 1 -- 172.21.15.37:0/2440449025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7540051d0 msgr2=0x7fa75400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:23.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76dd49640 1 --2- 172.21.15.37:0/2440449025 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa7540051d0 0x7fa75400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76dd49640 1 -- 172.21.15.37:0/2440449025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa754003530 con 0x7fa754004810 2024-04-25T09:03:23.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76ed4b640 1 -- 172.21.15.37:0/2440449025 reap_dead start 2024-04-25T09:03:23.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76e54a640 1 --2- 172.21.15.37:0/2440449025 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa754003a30 0x7fa754003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:23.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76dd49640 1 --2- 172.21.15.37:0/2440449025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa754004810 0x7fa754004c90 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00a850 tx=0x7fa75c01fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c42c94c187894a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:23.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00a4a0 con 0x7fa754004810 2024-04-25T09:03:23.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c0ad070 con 0x7fa754004810 2024-04-25T09:03:23.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa76d548640 1 -- 172.21.15.37:0/2440449025 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0a99c0 con 0x7fa754004810 2024-04-25T09:03:23.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa7707ae640 1 -- 172.21.15.37:0/2440449025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa754004810 msgr2=0x7fa754004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa7707ae640 1 --2- 172.21.15.37:0/2440449025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa754004810 0x7fa754004c90 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fa75c00a850 tx=0x7fa75c01fb00 comp rx=0 tx=0).stop 2024-04-25T09:03:23.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa7707ae640 1 -- 172.21.15.37:0/2440449025 shutdown_connections 2024-04-25T09:03:23.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa7707ae640 1 --2- 172.21.15.37:0/2440449025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa754004810 0x7fa754004c90 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa7707ae640 1 -- 172.21.15.37:0/2440449025 >> 172.21.15.37:0/2440449025 conn(0x7fa76806ae60 msgr2=0x7fa76806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:23.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa7707ae640 1 -- 172.21.15.37:0/2440449025 shutdown_connections 2024-04-25T09:03:23.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.881+0000 7fa7707ae640 1 -- 172.21.15.37:0/2440449025 wait complete. 2024-04-25T09:03:23.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa7707ae640 1 Processor -- start 2024-04-25T09:03:23.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa7707ae640 1 -- start start 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa7707ae640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa754004810 0x7fa7680ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa7707ae640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa76815ac10 0x7fa7680c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa7707ae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7680b25b0 0x7fa7680c3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa7707ae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680cba30 con 0x7fa7680b25b0 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa7707ae640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680cc8a0 con 0x7fa76815ac10 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa7707ae640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7680cfe50 con 0x7fa754004810 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76ed4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7680b25b0 0x7fa7680c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76ed4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7680b25b0 0x7fa7680c3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43650/0 (socket says 172.21.15.37:43650) 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76ed4b640 1 -- 172.21.15.37:0/552343945 learned_addr learned my addr 172.21.15.37:0/552343945 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76e54a640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa754004810 0x7fa7680ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76dd49640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa76815ac10 0x7fa7680c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76ed4b640 1 -- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa754004810 msgr2=0x7fa7680ad420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76ed4b640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa754004810 0x7fa7680ad420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76ed4b640 1 -- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa76815ac10 msgr2=0x7fa7680c2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76ed4b640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa76815ac10 0x7fa7680c2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76ed4b640 1 -- 172.21.15.37:0/552343945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7680c4000 con 0x7fa7680b25b0 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76dd49640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa76815ac10 0x7fa7680c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa76ed4b640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7680b25b0 0x7fa7680c3370 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fa76815f070 tx=0x7fa76000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa75affd640 1 -- 172.21.15.37:0/552343945 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760017020 con 0x7fa7680b25b0 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa75affd640 1 -- 172.21.15.37:0/552343945 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa760004030 con 0x7fa7680b25b0 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa7707ae640 1 -- 172.21.15.37:0/552343945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7680c5520 con 0x7fa7680b25b0 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa75affd640 1 -- 172.21.15.37:0/552343945 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760004400 con 0x7fa7680b25b0 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa7707ae640 1 -- 172.21.15.37:0/552343945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7680c4520 con 0x7fa7680b25b0 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.885+0000 7fa7707ae640 1 -- 172.21.15.37:0/552343945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76804efa0 con 0x7fa7680b25b0 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.889+0000 7fa75affd640 1 -- 172.21.15.37:0/552343945 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa76001a020 con 0x7fa7680b25b0 2024-04-25T09:03:23.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.889+0000 7fa75affd640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa74c066f90 0x7fa74c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:23.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.889+0000 7fa75affd640 1 -- 172.21.15.37:0/552343945 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa760085050 con 0x7fa7680b25b0 2024-04-25T09:03:23.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.889+0000 7fa76e54a640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa74c066f90 0x7fa74c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:23.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.889+0000 7fa76e54a640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa74c066f90 0x7fa74c069450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa764000c10 tx=0x7fa764009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:23.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:23.893+0000 7fa75affd640 1 -- 172.21.15.37:0/552343945 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa760021020 con 0x7fa7680b25b0 2024-04-25T09:03:24.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.049+0000 7fa7707ae640 1 -- 172.21.15.37:0/552343945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa7680ae200 con 0x7fa7680b25b0 2024-04-25T09:03:24.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.061+0000 7fa75affd640 1 -- 172.21.15.37:0/552343945 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa760056210 con 0x7fa7680b25b0 2024-04-25T09:03:24.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 -- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa74c066f90 msgr2=0x7fa74c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa74c066f90 0x7fa74c069450 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fa764000c10 tx=0x7fa764009040 comp rx=0 tx=0).stop 2024-04-25T09:03:24.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 -- 172.21.15.37:0/552343945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7680b25b0 msgr2=0x7fa7680c3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7680b25b0 0x7fa7680c3370 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fa76815f070 tx=0x7fa76000ec00 comp rx=0 tx=0).stop 2024-04-25T09:03:24.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 -- 172.21.15.37:0/552343945 shutdown_connections 2024-04-25T09:03:24.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa74c066f90 0x7fa74c069450 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7680b25b0 0x7fa7680c3370 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa76815ac10 0x7fa7680c2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 --2- 172.21.15.37:0/552343945 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa754004810 0x7fa7680ad420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 -- 172.21.15.37:0/552343945 >> 172.21.15.37:0/552343945 conn(0x7fa76806ae60 msgr2=0x7fa768156130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 -- 172.21.15.37:0/552343945 shutdown_connections 2024-04-25T09:03:24.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.065+0000 7fa7707ae640 1 -- 172.21.15.37:0/552343945 wait complete. 2024-04-25T09:03:24.072 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-04-25T09:03:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efeed981640 1 Processor -- start 2024-04-25T09:03:24.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efeed981640 1 -- start start 2024-04-25T09:03:24.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efeed981640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8159ea0 0x7efee80b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efeed981640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efee8071fd0 0x7efee815a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efeed981640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee815ae30 con 0x7efee815a280 2024-04-25T09:03:24.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efeed981640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee80b2990 con 0x7efee8071fd0 2024-04-25T09:03:24.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efeed981640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee815b110 con 0x7efee8159ea0 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efee77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7efee8071fd0 0x7efee815a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efee67fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7efee815a280 0x7efee80718c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45902/0 (socket says 172.21.15.37:45902) 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efee67fc640 1 -- 172.21.15.37:0/1033837959 learned_addr learned my addr 172.21.15.37:0/1033837959 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efee6ffd640 1 --2- 172.21.15.37:0/1033837959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8159ea0 0x7efee80b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efee77fe640 1 -- 172.21.15.37:0/1033837959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8159ea0 msgr2=0x7efee80b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efee77fe640 1 --2- 172.21.15.37:0/1033837959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8159ea0 0x7efee80b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efee77fe640 1 -- 172.21.15.37:0/1033837959 >> v1:172.21.15.37:6789/0 conn(0x7efee815a280 legacy=0x7efee80718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efee5ffb640 1 -- 172.21.15.37:0/1033837959 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881288757 0 0) 0x7efee815ae30 con 0x7efee815a280 2024-04-25T09:03:24.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.201+0000 7efee77fe640 1 -- 172.21.15.37:0/1033837959 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee80b2620 con 0x7efee8071fd0 2024-04-25T09:03:24.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee77fe640 1 --2- 172.21.15.37:0/1033837959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efee8071fd0 0x7efee815a700 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7efee0009d90 tx=0x7efee001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf5fd083f96dd51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee5ffb640 1 -- 172.21.15.37:0/1033837959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee00b2070 con 0x7efee8071fd0 2024-04-25T09:03:24.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee5ffb640 1 -- 172.21.15.37:0/1033837959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee00a57b0 con 0x7efee8071fd0 2024-04-25T09:03:24.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee5ffb640 1 -- 172.21.15.37:0/1033837959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee00a5b80 con 0x7efee8071fd0 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 -- 172.21.15.37:0/1033837959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efee8071fd0 msgr2=0x7efee815a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 --2- 172.21.15.37:0/1033837959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efee8071fd0 0x7efee815a700 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7efee0009d90 tx=0x7efee001c970 comp rx=0 tx=0).stop 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 -- 172.21.15.37:0/1033837959 shutdown_connections 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 --2- 172.21.15.37:0/1033837959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efee8071fd0 0x7efee815a700 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 --2- 172.21.15.37:0/1033837959 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8159ea0 0x7efee80b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 -- 172.21.15.37:0/1033837959 >> 172.21.15.37:0/1033837959 conn(0x7efee80b11a0 msgr2=0x7efee80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 -- 172.21.15.37:0/1033837959 shutdown_connections 2024-04-25T09:03:24.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 -- 172.21.15.37:0/1033837959 wait complete. 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 Processor -- start 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 -- start start 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8071fd0 0x7efee8142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee8159ea0 0x7efee80cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efee815a280 0x7efee8167a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee806d420 con 0x7efee815a280 2024-04-25T09:03:24.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee806d2a0 con 0x7efee8159ea0 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee806d5a0 con 0x7efee8071fd0 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efee815a280 0x7efee8167a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efee815a280 0x7efee8167a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43660/0 (socket says 172.21.15.37:43660) 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee77fe640 1 -- 172.21.15.37:0/2903495028 learned_addr learned my addr 172.21.15.37:0/2903495028 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee6ffd640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8071fd0 0x7efee8142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee67fc640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee8159ea0 0x7efee80cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee77fe640 1 -- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8071fd0 msgr2=0x7efee8142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee77fe640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8071fd0 0x7efee8142770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee77fe640 1 -- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee8159ea0 msgr2=0x7efee80cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee77fe640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee8159ea0 0x7efee80cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee77fe640 1 -- 172.21.15.37:0/2903495028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee8168120 con 0x7efee815a280 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efee77fe640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efee815a280 0x7efee8167a20 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7efee001b020 tx=0x7efee0019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efecffff640 1 -- 172.21.15.37:0/2903495028 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee00bd020 con 0x7efee815a280 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efecffff640 1 -- 172.21.15.37:0/2903495028 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee00b8470 con 0x7efee815a280 2024-04-25T09:03:24.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efecffff640 1 -- 172.21.15.37:0/2903495028 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee00b2040 con 0x7efee815a280 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 -- 172.21.15.37:0/2903495028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee8168400 con 0x7efee815a280 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.205+0000 7efeed981640 1 -- 172.21.15.37:0/2903495028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee81689c0 con 0x7efee815a280 2024-04-25T09:03:24.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.209+0000 7efeed981640 1 -- 172.21.15.37:0/2903495028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee804efa0 con 0x7efee815a280 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.209+0000 7efecffff640 1 -- 172.21.15.37:0/2903495028 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efee00c0020 con 0x7efee815a280 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.209+0000 7efecffff640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efed4066f60 0x7efed4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.209+0000 7efecffff640 1 -- 172.21.15.37:0/2903495028 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5421+0+0 (secure 0 0 0) 0x7efee011fbf0 con 0x7efee815a280 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.209+0000 7efee6ffd640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efed4066f60 0x7efed4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.209+0000 7efee6ffd640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efed4066f60 0x7efed4069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efee806eb50 tx=0x7efedc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.213+0000 7efecffff640 1 -- 172.21.15.37:0/2903495028 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efee00c7030 con 0x7efee815a280 2024-04-25T09:03:24.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.433+0000 7efeed981640 1 -- 172.21.15.37:0/2903495028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) v1 -- 0x7efee806e6b0 con 0x7efee815a280 2024-04-25T09:03:24.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.721+0000 7efecffff640 1 -- 172.21.15.37:0/2903495028 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v509) v1 ==== 146+0+0 (secure 0 0 0) 0x7efee0016560 con 0x7efee815a280 2024-04-25T09:03:24.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 -- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efed4066f60 msgr2=0x7efed4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efed4066f60 0x7efed4069420 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efee806eb50 tx=0x7efedc008040 comp rx=0 tx=0).stop 2024-04-25T09:03:24.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 -- 172.21.15.37:0/2903495028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efee815a280 msgr2=0x7efee8167a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efee815a280 0x7efee8167a20 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7efee001b020 tx=0x7efee0019710 comp rx=0 tx=0).stop 2024-04-25T09:03:24.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 -- 172.21.15.37:0/2903495028 shutdown_connections 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efed4066f60 0x7efed4069420 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efee815a280 0x7efee8167a20 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efee8159ea0 0x7efee80cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 --2- 172.21.15.37:0/2903495028 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efee8071fd0 0x7efee8142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 -- 172.21.15.37:0/2903495028 >> 172.21.15.37:0/2903495028 conn(0x7efee80b11a0 msgr2=0x7efee80cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 -- 172.21.15.37:0/2903495028 shutdown_connections 2024-04-25T09:03:24.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.725+0000 7efeed981640 1 -- 172.21.15.37:0/2903495028 wait complete. 2024-04-25T09:03:24.737 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T09:03:24.737 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-04-25T09:03:24.737 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:24.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.877+0000 7efe879e3640 1 Processor -- start 2024-04-25T09:03:24.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.877+0000 7efe879e3640 1 -- start start 2024-04-25T09:03:24.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe879e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe80071e90 0x7efe80161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe879e3640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efe80071410 0x7efe8006d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe879e3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe80162680 con 0x7efe80071030 2024-04-25T09:03:24.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe879e3640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe80162960 con 0x7efe80071410 2024-04-25T09:03:24.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe879e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe80162c40 con 0x7efe80071e90 2024-04-25T09:03:24.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe84f7e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe80071e90 0x7efe80161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe84f7e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe80071e90 0x7efe80161fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:46790/0 (socket says 172.21.15.37:46790) 2024-04-25T09:03:24.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe84f7e640 1 -- 172.21.15.37:0/4230134388 learned_addr learned my addr 172.21.15.37:0/4230134388 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:24.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe85f80640 1 --2- 172.21.15.37:0/4230134388 >> v2:172.21.15.203:3300/0 conn(0x7efe80071410 0x7efe8006d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe84f7e640 1 -- 172.21.15.37:0/4230134388 >> v2:172.21.15.203:3300/0 conn(0x7efe80071410 msgr2=0x7efe8006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:24.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe84f7e640 1 --2- 172.21.15.37:0/4230134388 >> v2:172.21.15.203:3300/0 conn(0x7efe80071410 0x7efe8006d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe84f7e640 1 -- 172.21.15.37:0/4230134388 >> v1:172.21.15.203:6789/0 conn(0x7efe80071030 legacy=0x7efe800b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe84f7e640 1 -- 172.21.15.37:0/4230134388 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe80162f20 con 0x7efe80071e90 2024-04-25T09:03:24.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe85f80640 1 --2- 172.21.15.37:0/4230134388 >> v2:172.21.15.203:3300/0 conn(0x7efe80071410 0x7efe8006d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe84f7e640 1 --2- 172.21.15.37:0/4230134388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe80071e90 0x7efe80161fb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efe70009870 tx=0x7efe7001c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2daf049bc42180b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe77fff640 1 -- 172.21.15.37:0/4230134388 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700b1070 con 0x7efe80071e90 2024-04-25T09:03:24.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.881+0000 7efe77fff640 1 -- 172.21.15.37:0/4230134388 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe700a8050 con 0x7efe80071e90 2024-04-25T09:03:24.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe77fff640 1 -- 172.21.15.37:0/4230134388 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe700ac860 con 0x7efe80071e90 2024-04-25T09:03:24.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 -- 172.21.15.37:0/4230134388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe80071e90 msgr2=0x7efe80161fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 --2- 172.21.15.37:0/4230134388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe80071e90 0x7efe80161fb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7efe70009870 tx=0x7efe7001c4d0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 -- 172.21.15.37:0/4230134388 shutdown_connections 2024-04-25T09:03:24.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 --2- 172.21.15.37:0/4230134388 >> v2:172.21.15.203:3300/0 conn(0x7efe80071410 0x7efe8006d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 --2- 172.21.15.37:0/4230134388 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe80071e90 0x7efe80161fb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 -- 172.21.15.37:0/4230134388 >> 172.21.15.37:0/4230134388 conn(0x7efe800b11a0 msgr2=0x7efe800b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:24.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 -- 172.21.15.37:0/4230134388 shutdown_connections 2024-04-25T09:03:24.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 -- 172.21.15.37:0/4230134388 wait complete. 2024-04-25T09:03:24.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 Processor -- start 2024-04-25T09:03:24.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 -- start start 2024-04-25T09:03:24.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe80071030 0x7efe8015a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80071410 0x7efe80153e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe801543c0 0x7efe80144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8006e170 con 0x7efe80071030 2024-04-25T09:03:24.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8006dff0 con 0x7efe80071410 2024-04-25T09:03:24.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe879e3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8006e2f0 con 0x7efe801543c0 2024-04-25T09:03:24.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe8577f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe80071030 0x7efe8015a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe8577f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe80071030 0x7efe8015a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43672/0 (socket says 172.21.15.37:43672) 2024-04-25T09:03:24.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.885+0000 7efe8577f640 1 -- 172.21.15.37:0/1801631116 learned_addr learned my addr 172.21.15.37:0/1801631116 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:24.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe85f80640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe801543c0 0x7efe80144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe84f7e640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80071410 0x7efe80153e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe8577f640 1 -- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe801543c0 msgr2=0x7efe80144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe8577f640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe801543c0 0x7efe80144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe8577f640 1 -- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80071410 msgr2=0x7efe80153e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:24.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe8577f640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80071410 0x7efe80153e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:24.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe8577f640 1 -- 172.21.15.37:0/1801631116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8015dd00 con 0x7efe80071030 2024-04-25T09:03:24.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe8577f640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe80071030 0x7efe8015a930 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7efe7800c790 tx=0x7efe7800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe75ffb640 1 -- 172.21.15.37:0/1801631116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78016020 con 0x7efe80071030 2024-04-25T09:03:24.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe75ffb640 1 -- 172.21.15.37:0/1801631116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe78006d60 con 0x7efe80071030 2024-04-25T09:03:24.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe879e3640 1 -- 172.21.15.37:0/1801631116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8015dfe0 con 0x7efe80071030 2024-04-25T09:03:24.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe75ffb640 1 -- 172.21.15.37:0/1801631116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe78003bc0 con 0x7efe80071030 2024-04-25T09:03:24.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe879e3640 1 -- 172.21.15.37:0/1801631116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8015e5a0 con 0x7efe80071030 2024-04-25T09:03:24.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe879e3640 1 -- 172.21.15.37:0/1801631116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe801547d0 con 0x7efe80071030 2024-04-25T09:03:24.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe75ffb640 1 -- 172.21.15.37:0/1801631116 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe7800a850 con 0x7efe80071030 2024-04-25T09:03:24.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe75ffb640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe64066f60 0x7efe64069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:24.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe75ffb640 1 -- 172.21.15.37:0/1801631116 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5421+0+0 (secure 0 0 0) 0x7efe78084be0 con 0x7efe80071030 2024-04-25T09:03:24.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe84f7e640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe64066f60 0x7efe64069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:24.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.889+0000 7efe84f7e640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe64066f60 0x7efe64069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efe700ae000 tx=0x7efe70008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:24.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:24.893+0000 7efe75ffb640 1 -- 172.21.15.37:0/1801631116 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe801547d0 con 0x7efe80071030 2024-04-25T09:03:25.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.053+0000 7efe879e3640 1 -- 172.21.15.37:0/1801631116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe800ae860 con 0x7efe80071030 2024-04-25T09:03:25.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe75ffb640 1 -- 172.21.15.37:0/1801631116 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+13397 (secure 0 0 0) 0x7efe78020020 con 0x7efe80071030 2024-04-25T09:03:25.046 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:25.046 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":116,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:23.713657+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2017261543":"2024-04-26T09:03:15.641825+0000","172.21.15.37:6834/2017261543":"2024-04-26T09:03:15.641825+0000","172.21.15.37:6835/1266122577":"2024-04-26T09:02:39.103789+0000","172.21.15.37:6834/1266122577":"2024-04-26T09:02:39.103789+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 -- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe64066f60 msgr2=0x7efe64069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe64066f60 0x7efe64069420 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efe700ae000 tx=0x7efe70008040 comp rx=0 tx=0).stop 2024-04-25T09:03:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 -- 172.21.15.37:0/1801631116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe80071030 msgr2=0x7efe8015a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe80071030 0x7efe8015a930 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7efe7800c790 tx=0x7efe7800cc50 comp rx=0 tx=0).stop 2024-04-25T09:03:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 -- 172.21.15.37:0/1801631116 shutdown_connections 2024-04-25T09:03:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe64066f60 0x7efe64069420 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe801543c0 0x7efe80144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe80071410 0x7efe80153e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 --2- 172.21.15.37:0/1801631116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe80071030 0x7efe8015a930 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 -- 172.21.15.37:0/1801631116 >> 172.21.15.37:0/1801631116 conn(0x7efe800b11a0 msgr2=0x7efe80155610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 -- 172.21.15.37:0/1801631116 shutdown_connections 2024-04-25T09:03:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.057+0000 7efe879e3640 1 -- 172.21.15.37:0/1801631116 wait complete. 2024-04-25T09:03:25.066 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:25.067 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08cabd640 1 Processor -- start 2024-04-25T09:03:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08cabd640 1 -- start start 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08cabd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b5270 0x7fc0880b0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08cabd640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08806bff0 con 0x7fc0880b5650 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08cabd640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08806c2d0 con 0x7fc08806d090 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08cabd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08806cc00 con 0x7fc0880b5270 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc085d9b640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc0880b5650 0x7fc0880d3f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45918/0 (socket says 172.21.15.37:45918) 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc085d9b640 1 -- 172.21.15.37:0/3627480340 learned_addr learned my addr 172.21.15.37:0/3627480340 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08659c640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b5270 0x7fc0880b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069514569 0 0) 0x7fc08806bff0 con 0x7fc0880b5650 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc070003530 con 0x7fc0880b5650 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2865272041 0 0) 0x7fc070003530 con 0x7fc0880b5650 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b5270 msgr2=0x7fc0880b0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b5270 0x7fc0880b0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 >> v1:172.21.15.203:6789/0 conn(0x7fc08806d090 legacy=0x7fc08806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08806c5b0 con 0x7fc0880b5650 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fc0780016d0 con 0x7fc0880b5650 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 >> v1:172.21.15.37:6789/0 conn(0x7fc0880b5650 legacy=0x7fc0880d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 0x7fc070003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b5270 0x7fc070004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc070004d50 0x7fc0700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08806cc00 con 0x7fc070003a60 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc07000bb00 con 0x7fc070004d50 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc07000bd00 con 0x7fc0880b5270 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08659c640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 0x7fc070003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc085d9b640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b5270 0x7fc070004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc086d9d640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc070004d50 0x7fc0700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08659c640 1 -- 172.21.15.37:0/3627480340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b5270 msgr2=0x7fc070004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08659c640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b5270 0x7fc070004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08659c640 1 -- 172.21.15.37:0/3627480340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc070004d50 msgr2=0x7fc0700051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08659c640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc070004d50 0x7fc0700051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.209+0000 7fc08659c640 1 -- 172.21.15.37:0/3627480340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc070003530 con 0x7fc070003a60 2024-04-25T09:03:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08659c640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 0x7fc070003ea0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009ac0 tx=0x7fc07c01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec52ea5bbde8557e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0ac070 con 0x7fc070003a60 2024-04-25T09:03:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c0059a0 con 0x7fc070003a60 2024-04-25T09:03:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08559a640 1 -- 172.21.15.37:0/3627480340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c009690 con 0x7fc070003a60 2024-04-25T09:03:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08cabd640 1 -- 172.21.15.37:0/3627480340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 msgr2=0x7fc070003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08cabd640 1 --2- 172.21.15.37:0/3627480340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 0x7fc070003ea0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fc07c009ac0 tx=0x7fc07c01c990 comp rx=0 tx=0).stop 2024-04-25T09:03:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc086d9d640 1 -- 172.21.15.37:0/3627480340 reap_dead start 2024-04-25T09:03:25.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08cabd640 1 -- 172.21.15.37:0/3627480340 shutdown_connections 2024-04-25T09:03:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08cabd640 1 -- 172.21.15.37:0/3627480340 >> 172.21.15.37:0/3627480340 conn(0x7fc08808b000 msgr2=0x7fc0880d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08cabd640 1 -- 172.21.15.37:0/3627480340 shutdown_connections 2024-04-25T09:03:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08cabd640 1 -- 172.21.15.37:0/3627480340 wait complete. 2024-04-25T09:03:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08cabd640 1 Processor -- start 2024-04-25T09:03:25.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.213+0000 7fc08cabd640 1 -- start start 2024-04-25T09:03:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08cabd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 0x7fc088168280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08cabd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b6150 0x7fc0880b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08cabd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0881647a0 0x7fc08815ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08cabd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0880b5bb0 con 0x7fc070003a60 2024-04-25T09:03:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08cabd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0880b5a30 con 0x7fc0881647a0 2024-04-25T09:03:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08cabd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08815de20 con 0x7fc0880b6150 2024-04-25T09:03:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 0x7fc088168280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 0x7fc088168280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43684/0 (socket says 172.21.15.37:43684) 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08659c640 1 -- 172.21.15.37:0/1182486180 learned_addr learned my addr 172.21.15.37:0/1182486180 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc085d9b640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b6150 0x7fc0880b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08659c640 1 -- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b6150 msgr2=0x7fc0880b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08659c640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b6150 0x7fc0880b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc086d9d640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0881647a0 0x7fc08815ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08659c640 1 -- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0881647a0 msgr2=0x7fc08815ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08659c640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0881647a0 0x7fc08815ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08659c640 1 -- 172.21.15.37:0/1182486180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0880b6910 con 0x7fc070003a60 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08659c640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 0x7fc088168280 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fc07c098290 tx=0x7fc070005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc076ffd640 1 -- 172.21.15.37:0/1182486180 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c0ac040 con 0x7fc070003a60 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc076ffd640 1 -- 172.21.15.37:0/1182486180 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07c0ba070 con 0x7fc070003a60 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08cabd640 1 -- 172.21.15.37:0/1182486180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0880b6bf0 con 0x7fc070003a60 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc076ffd640 1 -- 172.21.15.37:0/1182486180 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07c006430 con 0x7fc070003a60 2024-04-25T09:03:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08cabd640 1 -- 172.21.15.37:0/1182486180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0880b7490 con 0x7fc070003a60 2024-04-25T09:03:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc08cabd640 1 -- 172.21.15.37:0/1182486180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08804efa0 con 0x7fc070003a60 2024-04-25T09:03:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc076ffd640 1 -- 172.21.15.37:0/1182486180 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc07c0065d0 con 0x7fc070003a60 2024-04-25T09:03:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc076ffd640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc068066f60 0x7fc068069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc076ffd640 1 -- 172.21.15.37:0/1182486180 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc07c11f4c0 con 0x7fc070003a60 2024-04-25T09:03:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.217+0000 7fc085d9b640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc068066f60 0x7fc068069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.221+0000 7fc085d9b640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc068066f60 0x7fc068069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc078001980 tx=0x7fc0780018f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.221+0000 7fc076ffd640 1 -- 172.21.15.37:0/1182486180 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc07c0166a0 con 0x7fc070003a60 2024-04-25T09:03:25.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.381+0000 7fc08cabd640 1 -- 172.21.15.37:0/1182486180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0880d1b20 con 0x7fc070003a60 2024-04-25T09:03:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.381+0000 7fc076ffd640 1 -- 172.21.15.37:0/1182486180 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc07c0f0540 con 0x7fc070003a60 2024-04-25T09:03:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:03:25.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 -- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc068066f60 msgr2=0x7fc068069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc068066f60 0x7fc068069420 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fc078001980 tx=0x7fc0780018f0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 -- 172.21.15.37:0/1182486180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 msgr2=0x7fc088168280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 0x7fc088168280 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7fc07c098290 tx=0x7fc070005840 comp rx=0 tx=0).stop 2024-04-25T09:03:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 -- 172.21.15.37:0/1182486180 shutdown_connections 2024-04-25T09:03:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc068066f60 0x7fc068069420 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0881647a0 0x7fc08815ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0880b6150 0x7fc0880b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 --2- 172.21.15.37:0/1182486180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc070003a60 0x7fc088168280 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 -- 172.21.15.37:0/1182486180 >> 172.21.15.37:0/1182486180 conn(0x7fc08808b000 msgr2=0x7fc0880b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 -- 172.21.15.37:0/1182486180 shutdown_connections 2024-04-25T09:03:25.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.385+0000 7fc08cabd640 1 -- 172.21.15.37:0/1182486180 wait complete. 2024-04-25T09:03:25.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T09:03:25.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5b072c640 1 Processor -- start 2024-04-25T09:03:25.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5b072c640 1 -- start start 2024-04-25T09:03:25.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5b072c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5a80b0b00 0x7fa5a80cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5b072c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa5a80cc650 0x7fa5a80b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5b072c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a8160310 con 0x7fa5a80b1910 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5b072c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a815cfa0 con 0x7fa5a80cc650 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5b072c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a815cdf0 con 0x7fa5a80b0b00 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5aecc9640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa5a80b1910 0x7fa5a8151fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45932/0 (socket says 172.21.15.37:45932) 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5aecc9640 1 -- 172.21.15.37:0/4276721235 learned_addr learned my addr 172.21.15.37:0/4276721235 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5adcc7640 1 --2- 172.21.15.37:0/4276721235 >> v2:172.21.15.203:3300/0 conn(0x7fa5a80cc650 0x7fa5a80b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ae4c8640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5a80b0b00 0x7fa5a80cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804595911 0 0) 0x7fa5a8160310 con 0x7fa5a80b1910 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa594003530 con 0x7fa5a80b1910 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1951531917 0 0) 0x7fa594003530 con 0x7fa5a80b1910 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5a80b0b00 msgr2=0x7fa5a80cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5a80b0b00 0x7fa5a80cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 >> v2:172.21.15.203:3300/0 conn(0x7fa5a80cc650 msgr2=0x7fa5a80b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 --2- 172.21.15.37:0/4276721235 >> v2:172.21.15.203:3300/0 conn(0x7fa5a80cc650 0x7fa5a80b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a815f550 con 0x7fa5a80b1910 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fa5a0001a00 con 0x7fa5a80b1910 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 >> v1:172.21.15.37:6789/0 conn(0x7fa5a80b1910 legacy=0x7fa5a8151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594003a30 0x7fa594003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5a80b0b00 0x7fa594004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa594004d50 0x7fa5940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59400b9d0 con 0x7fa594004d50 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59400bb50 con 0x7fa594003a30 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59400bcd0 con 0x7fa5a80b0b00 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5aecc9640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa594004d50 0x7fa5940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5ae4c8640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594003a30 0x7fa594003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5adcc7640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5a80b0b00 0x7fa594004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5aecc9640 1 -- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5a80b0b00 msgr2=0x7fa594004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5aecc9640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5a80b0b00 0x7fa594004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5aecc9640 1 -- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594003a30 msgr2=0x7fa594003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5aecc9640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa594003a30 0x7fa594003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5aecc9640 1 -- 172.21.15.37:0/4276721235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa594003530 con 0x7fa594004d50 2024-04-25T09:03:25.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.517+0000 7fa5adcc7640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa5a80b0b00 0x7fa594004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:25.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5aecc9640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa594004d50 0x7fa5940051d0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000a8a0 tx=0x7fa5a001d3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0491fb0408112f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a000a4c0 con 0x7fa594004d50 2024-04-25T09:03:25.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a00a9020 con 0x7fa594004d50 2024-04-25T09:03:25.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5ad4c6640 1 -- 172.21.15.37:0/4276721235 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a00072f0 con 0x7fa594004d50 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 -- 172.21.15.37:0/4276721235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa594004d50 msgr2=0x7fa5940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 --2- 172.21.15.37:0/4276721235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa594004d50 0x7fa5940051d0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000a8a0 tx=0x7fa5a001d3c0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5aecc9640 1 -- 172.21.15.37:0/4276721235 reap_dead start 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 -- 172.21.15.37:0/4276721235 shutdown_connections 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 -- 172.21.15.37:0/4276721235 >> 172.21.15.37:0/4276721235 conn(0x7fa5a80b1f10 msgr2=0x7fa5a80b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 -- 172.21.15.37:0/4276721235 shutdown_connections 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 -- 172.21.15.37:0/4276721235 wait complete. 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 Processor -- start 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 -- start start 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa594004d50 0x7fa5a814a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5a80b25e0 0x7fa5a80b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a806e9c0 0x7fa5a806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a815fb10 con 0x7fa5a806e9c0 2024-04-25T09:03:25.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a815fe80 con 0x7fa5a80b25e0 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5b072c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a806b950 con 0x7fa594004d50 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5aecc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a806e9c0 0x7fa5a806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5aecc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a806e9c0 0x7fa5a806ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43694/0 (socket says 172.21.15.37:43694) 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.521+0000 7fa5aecc9640 1 -- 172.21.15.37:0/808389268 learned_addr learned my addr 172.21.15.37:0/808389268 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5adcc7640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5a80b25e0 0x7fa5a80b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5ae4c8640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa594004d50 0x7fa5a814a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5aecc9640 1 -- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa594004d50 msgr2=0x7fa5a814a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5aecc9640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa594004d50 0x7fa5a814a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5aecc9640 1 -- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5a80b25e0 msgr2=0x7fa5a80b5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5aecc9640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5a80b25e0 0x7fa5a80b5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5aecc9640 1 -- 172.21.15.37:0/808389268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8066e60 con 0x7fa5a806e9c0 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5aecc9640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a806e9c0 0x7fa5a806ee70 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000a8a0 tx=0x7fa594005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa59effd640 1 -- 172.21.15.37:0/808389268 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a000a4c0 con 0x7fa5a806e9c0 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa59effd640 1 -- 172.21.15.37:0/808389268 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a0004d80 con 0x7fa5a806e9c0 2024-04-25T09:03:25.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5b072c640 1 -- 172.21.15.37:0/808389268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a8073d40 con 0x7fa5a806e9c0 2024-04-25T09:03:25.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa59effd640 1 -- 172.21.15.37:0/808389268 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a0002020 con 0x7fa5a806e9c0 2024-04-25T09:03:25.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5b072c640 1 -- 172.21.15.37:0/808389268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a80c7190 con 0x7fa5a806e9c0 2024-04-25T09:03:25.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5b072c640 1 -- 172.21.15.37:0/808389268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a804efa0 con 0x7fa5a806e9c0 2024-04-25T09:03:25.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa59effd640 1 -- 172.21.15.37:0/808389268 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa5a0007520 con 0x7fa5a806e9c0 2024-04-25T09:03:25.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa59effd640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa58c066f90 0x7fa58c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:25.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa59effd640 1 -- 172.21.15.37:0/808389268 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa5a011f330 con 0x7fa5a806e9c0 2024-04-25T09:03:25.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.525+0000 7fa5ae4c8640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa58c066f90 0x7fa58c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:25.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.529+0000 7fa5ae4c8640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa58c066f90 0x7fa58c069450 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa5a809e540 tx=0x7fa5a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:25.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.529+0000 7fa59effd640 1 -- 172.21.15.37:0/808389268 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa5a00170d0 con 0x7fa5a806e9c0 2024-04-25T09:03:25.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:25.737+0000 7fa5b072c640 1 -- 172.21.15.37:0/808389268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7fa5a80b29c0 con 0x7fa5a806e9c0 2024-04-25T09:03:26.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa59effd640 1 -- 172.21.15.37:0/808389268 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v511) v1 ==== 147+0+0 (secure 0 0 0) 0x7fa5a00f0330 con 0x7fa5a806e9c0 2024-04-25T09:03:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 -- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa58c066f90 msgr2=0x7fa58c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa58c066f90 0x7fa58c069450 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa5a809e540 tx=0x7fa5a4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 -- 172.21.15.37:0/808389268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a806e9c0 msgr2=0x7fa5a806ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a806e9c0 0x7fa5a806ee70 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fa5a000a8a0 tx=0x7fa594005840 comp rx=0 tx=0).stop 2024-04-25T09:03:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 -- 172.21.15.37:0/808389268 shutdown_connections 2024-04-25T09:03:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa58c066f90 0x7fa58c069450 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5a806e9c0 0x7fa5a806ee70 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa5a80b25e0 0x7fa5a80b5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 --2- 172.21.15.37:0/808389268 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa594004d50 0x7fa5a814a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 -- 172.21.15.37:0/808389268 >> 172.21.15.37:0/808389268 conn(0x7fa5a80b1f10 msgr2=0x7fa5a806e0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 -- 172.21.15.37:0/808389268 shutdown_connections 2024-04-25T09:03:26.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.737+0000 7fa5b072c640 1 -- 172.21.15.37:0/808389268 wait complete. 2024-04-25T09:03:26.749 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:03:26.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b0c750640 1 Processor -- start 2024-04-25T09:03:26.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b0c750640 1 -- start start 2024-04-25T09:03:26.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b0c750640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7b040cf840 0x7f7b040cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b0c750640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b0415d180 0x7f7b0415d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b0c750640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0406b710 con 0x7f7b0406fc50 2024-04-25T09:03:26.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b0c750640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b04161560 con 0x7f7b040cf840 2024-04-25T09:03:26.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b0c750640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b041617b0 con 0x7f7b0415d180 2024-04-25T09:03:26.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b0a4ec640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7b0406fc50 0x7f7b0406ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45944/0 (socket says 172.21.15.37:45944) 2024-04-25T09:03:26.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b0a4ec640 1 -- 172.21.15.37:0/2951225069 learned_addr learned my addr 172.21.15.37:0/2951225069 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:26.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b09ceb640 1 --2- 172.21.15.37:0/2951225069 >> v2:172.21.15.37:3300/0 conn(0x7f7b040cf840 0x7f7b040cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b0aced640 1 --2- 172.21.15.37:0/2951225069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b0415d180 0x7f7b0415d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b094ea640 1 -- 172.21.15.37:0/2951225069 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330975770 0 0) 0x7f7b0406b710 con 0x7f7b0406fc50 2024-04-25T09:03:26.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b094ea640 1 -- 172.21.15.37:0/2951225069 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7aec003530 con 0x7f7b0406fc50 2024-04-25T09:03:26.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b09ceb640 1 -- 172.21.15.37:0/2951225069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b0415d180 msgr2=0x7f7b0415d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:26.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b09ceb640 1 --2- 172.21.15.37:0/2951225069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b0415d180 0x7f7b0415d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b09ceb640 1 -- 172.21.15.37:0/2951225069 >> v1:172.21.15.37:6789/0 conn(0x7f7b0406fc50 legacy=0x7f7b0406ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.889+0000 7f7b09ceb640 1 -- 172.21.15.37:0/2951225069 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04161a90 con 0x7f7b040cf840 2024-04-25T09:03:26.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b09ceb640 1 --2- 172.21.15.37:0/2951225069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b040cf840 0x7f7b040cfcd0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f7af80097c0 tx=0x7f7af801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43a84536aa4753b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b094ea640 1 -- 172.21.15.37:0/2951225069 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af80b0070 con 0x7f7b040cf840 2024-04-25T09:03:26.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b094ea640 1 -- 172.21.15.37:0/2951225069 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af80ab600 con 0x7f7b040cf840 2024-04-25T09:03:26.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b094ea640 1 -- 172.21.15.37:0/2951225069 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af80ab9d0 con 0x7f7b040cf840 2024-04-25T09:03:26.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b0c750640 1 -- 172.21.15.37:0/2951225069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b040cf840 msgr2=0x7f7b040cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b0c750640 1 --2- 172.21.15.37:0/2951225069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b040cf840 0x7f7b040cfcd0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f7af80097c0 tx=0x7f7af801c470 comp rx=0 tx=0).stop 2024-04-25T09:03:26.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b0c750640 1 -- 172.21.15.37:0/2951225069 shutdown_connections 2024-04-25T09:03:26.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b0c750640 1 --2- 172.21.15.37:0/2951225069 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b0415d180 0x7f7b0415d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b0c750640 1 --2- 172.21.15.37:0/2951225069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b040cf840 0x7f7b040cfcd0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b0c750640 1 -- 172.21.15.37:0/2951225069 >> 172.21.15.37:0/2951225069 conn(0x7f7b04099a70 msgr2=0x7f7b040d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:26.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b0c750640 1 -- 172.21.15.37:0/2951225069 shutdown_connections 2024-04-25T09:03:26.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b0c750640 1 -- 172.21.15.37:0/2951225069 wait complete. 2024-04-25T09:03:26.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b0c750640 1 Processor -- start 2024-04-25T09:03:26.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.893+0000 7f7b0c750640 1 -- start start 2024-04-25T09:03:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0c750640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0406fc50 0x7f7b04152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0c750640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b040cf840 0x7f7b04154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0c750640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0415d180 0x7f7b04155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0c750640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04070030 con 0x7f7b0406fc50 2024-04-25T09:03:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0c750640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040d0e30 con 0x7f7b0415d180 2024-04-25T09:03:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0c750640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b040701b0 con 0x7f7b040cf840 2024-04-25T09:03:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0a4ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0406fc50 0x7f7b04152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0a4ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0406fc50 0x7f7b04152940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43716/0 (socket says 172.21.15.37:43716) 2024-04-25T09:03:26.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0a4ec640 1 -- 172.21.15.37:0/3335750039 learned_addr learned my addr 172.21.15.37:0/3335750039 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b09ceb640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b040cf840 0x7f7b04154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0a4ec640 1 -- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b040cf840 msgr2=0x7f7b04154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0a4ec640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b040cf840 0x7f7b04154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0aced640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0415d180 0x7f7b04155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0a4ec640 1 -- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0415d180 msgr2=0x7f7b04155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0a4ec640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0415d180 0x7f7b04155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0a4ec640 1 -- 172.21.15.37:0/3335750039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04152e80 con 0x7f7b0406fc50 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0a4ec640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0406fc50 0x7f7b04152940 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00cad0 tx=0x7f7afc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b02ffd640 1 -- 172.21.15.37:0/3335750039 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc017070 con 0x7f7b0406fc50 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b02ffd640 1 -- 172.21.15.37:0/3335750039 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc00ac30 con 0x7f7b0406fc50 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b02ffd640 1 -- 172.21.15.37:0/3335750039 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc012400 con 0x7f7b0406fc50 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0c750640 1 -- 172.21.15.37:0/3335750039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b040c4d20 con 0x7f7b0406fc50 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0c750640 1 -- 172.21.15.37:0/3335750039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b040c52e0 con 0x7f7b0406fc50 2024-04-25T09:03:26.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b0c750640 1 -- 172.21.15.37:0/3335750039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0404efa0 con 0x7f7b0406fc50 2024-04-25T09:03:26.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b02ffd640 1 -- 172.21.15.37:0/3335750039 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7afc01a070 con 0x7f7b0406fc50 2024-04-25T09:03:26.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b02ffd640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7af0066f60 0x7f7af0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:26.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b02ffd640 1 -- 172.21.15.37:0/3335750039 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7afc084d40 con 0x7f7b0406fc50 2024-04-25T09:03:26.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.897+0000 7f7b09ceb640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7af0066f60 0x7f7af0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:26.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.901+0000 7f7b09ceb640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7af0066f60 0x7f7af0069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7af80ad000 tx=0x7f7af8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:26.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:26.901+0000 7f7b02ffd640 1 -- 172.21.15.37:0/3335750039 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7afc055dd0 con 0x7f7b0406fc50 2024-04-25T09:03:27.044 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:27.047 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:27.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.069+0000 7f7b0c750640 1 -- 172.21.15.37:0/3335750039 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7b0406ae30 con 0x7f7b0406fc50 2024-04-25T09:03:27.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b02ffd640 1 -- 172.21.15.37:0/3335750039 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7afc0556d0 con 0x7f7b0406fc50 2024-04-25T09:03:27.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 -- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7af0066f60 msgr2=0x7f7af0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7af0066f60 0x7f7af0069420 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f7af80ad000 tx=0x7f7af8008040 comp rx=0 tx=0).stop 2024-04-25T09:03:27.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 -- 172.21.15.37:0/3335750039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0406fc50 msgr2=0x7f7b04152940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0406fc50 0x7f7b04152940 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00cad0 tx=0x7f7afc00cf90 comp rx=0 tx=0).stop 2024-04-25T09:03:27.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 -- 172.21.15.37:0/3335750039 shutdown_connections 2024-04-25T09:03:27.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7af0066f60 0x7f7af0069420 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7b0415d180 0x7f7b04155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7b040cf840 0x7f7b04154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 --2- 172.21.15.37:0/3335750039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b0406fc50 0x7f7b04152940 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 -- 172.21.15.37:0/3335750039 >> 172.21.15.37:0/3335750039 conn(0x7f7b04099a70 msgr2=0x7f7b040ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 -- 172.21.15.37:0/3335750039 shutdown_connections 2024-04-25T09:03:27.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.073+0000 7f7b0c750640 1 -- 172.21.15.37:0/3335750039 wait complete. 2024-04-25T09:03:27.080 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:03:27.081 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:27.198 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:27.242 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:03:27.242 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:03:27.242 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:27.248 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:03:27.248 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:03:27.248 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:27.290 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:27.290 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:03:27.290 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:27.298 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:03:27.298 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:03:27.298 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:27.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:27.338 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:03:27.339 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:27.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.481+0000 7ffaf42cd640 1 Processor -- start 2024-04-25T09:03:27.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.481+0000 7ffaf42cd640 1 -- start start 2024-04-25T09:03:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf42cd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffaec144ca0 0x7ffaec06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf42cd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ffaec06ec60 0x7ffaec142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf42cd640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaec0ccc70 con 0x7ffaec0ce510 2024-04-25T09:03:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf42cd640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaec0cc2b0 con 0x7ffaec144ca0 2024-04-25T09:03:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf42cd640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaec06e290 con 0x7ffaec06ec60 2024-04-25T09:03:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf1868640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ffaec144ca0 0x7ffaec06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf2069640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ffaec0ce510 0x7ffaec0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45948/0 (socket says 172.21.15.37:45948) 2024-04-25T09:03:27.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf2069640 1 -- 172.21.15.37:0/121204840 learned_addr learned my addr 172.21.15.37:0/121204840 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf286a640 1 --2- 172.21.15.37:0/121204840 >> v2:172.21.15.203:3300/0 conn(0x7ffaec06ec60 0x7ffaec142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf1868640 1 -- 172.21.15.37:0/121204840 >> v2:172.21.15.203:3300/0 conn(0x7ffaec06ec60 msgr2=0x7ffaec142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf1868640 1 --2- 172.21.15.37:0/121204840 >> v2:172.21.15.203:3300/0 conn(0x7ffaec06ec60 0x7ffaec142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf1868640 1 -- 172.21.15.37:0/121204840 >> v1:172.21.15.37:6789/0 conn(0x7ffaec0ce510 legacy=0x7ffaec0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf1868640 1 -- 172.21.15.37:0/121204840 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec06dc40 con 0x7ffaec144ca0 2024-04-25T09:03:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.485+0000 7ffaf1868640 1 --2- 172.21.15.37:0/121204840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec144ca0 0x7ffaec06e720 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0b48f0 tx=0x7ffadc01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fbc4e2e88a2870 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf1067640 1 -- 172.21.15.37:0/121204840 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0b5070 con 0x7ffaec144ca0 2024-04-25T09:03:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf1067640 1 -- 172.21.15.37:0/121204840 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc0a8970 con 0x7ffaec144ca0 2024-04-25T09:03:27.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf1067640 1 -- 172.21.15.37:0/121204840 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc0a8d40 con 0x7ffaec144ca0 2024-04-25T09:03:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 -- 172.21.15.37:0/121204840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec144ca0 msgr2=0x7ffaec06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 --2- 172.21.15.37:0/121204840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec144ca0 0x7ffaec06e720 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7ffaec0b48f0 tx=0x7ffadc01ec70 comp rx=0 tx=0).stop 2024-04-25T09:03:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 -- 172.21.15.37:0/121204840 shutdown_connections 2024-04-25T09:03:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 --2- 172.21.15.37:0/121204840 >> v2:172.21.15.203:3300/0 conn(0x7ffaec06ec60 0x7ffaec142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 --2- 172.21.15.37:0/121204840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec144ca0 0x7ffaec06e720 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 -- 172.21.15.37:0/121204840 >> 172.21.15.37:0/121204840 conn(0x7ffaec0b11a0 msgr2=0x7ffaec0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 -- 172.21.15.37:0/121204840 shutdown_connections 2024-04-25T09:03:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 -- 172.21.15.37:0/121204840 wait complete. 2024-04-25T09:03:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 Processor -- start 2024-04-25T09:03:27.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 -- start start 2024-04-25T09:03:27.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaec06ec60 0x7ffaec15b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffaec0ce510 0x7ffaec0ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec144ca0 0x7ffaec0adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec1451f0 con 0x7ffaec144ca0 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec0cc900 con 0x7ffaec06ec60 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaec145370 con 0x7ffaec0ce510 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf286a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec144ca0 0x7ffaec0adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf286a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec144ca0 0x7ffaec0adf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43726/0 (socket says 172.21.15.37:43726) 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf286a640 1 -- 172.21.15.37:0/3058429313 learned_addr learned my addr 172.21.15.37:0/3058429313 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf1868640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffaec0ce510 0x7ffaec0ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf2069640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaec06ec60 0x7ffaec15b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf286a640 1 -- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffaec0ce510 msgr2=0x7ffaec0ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf286a640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffaec0ce510 0x7ffaec0ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf286a640 1 -- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaec06ec60 msgr2=0x7ffaec15b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf286a640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaec06ec60 0x7ffaec15b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf286a640 1 -- 172.21.15.37:0/3058429313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaec143de0 con 0x7ffaec144ca0 2024-04-25T09:03:27.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf286a640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec144ca0 0x7ffaec0adf20 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7ffae4007e40 tx=0x7ffae400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffae2ffd640 1 -- 172.21.15.37:0/3058429313 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4014620 con 0x7ffaec144ca0 2024-04-25T09:03:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffae2ffd640 1 -- 172.21.15.37:0/3058429313 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae4019070 con 0x7ffaec144ca0 2024-04-25T09:03:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 -- 172.21.15.37:0/3058429313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaec135660 con 0x7ffaec144ca0 2024-04-25T09:03:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffae2ffd640 1 -- 172.21.15.37:0/3058429313 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae4014a20 con 0x7ffaec144ca0 2024-04-25T09:03:27.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.489+0000 7ffaf42cd640 1 -- 172.21.15.37:0/3058429313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaec132a90 con 0x7ffaec144ca0 2024-04-25T09:03:27.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.493+0000 7ffaf42cd640 1 -- 172.21.15.37:0/3058429313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaec04efa0 con 0x7ffaec144ca0 2024-04-25T09:03:27.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.493+0000 7ffae2ffd640 1 -- 172.21.15.37:0/3058429313 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffae4014bc0 con 0x7ffaec144ca0 2024-04-25T09:03:27.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.493+0000 7ffae2ffd640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffad0066f60 0x7ffad0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.493+0000 7ffae2ffd640 1 -- 172.21.15.37:0/3058429313 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ffae40864f0 con 0x7ffaec144ca0 2024-04-25T09:03:27.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.493+0000 7ffaf2069640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffad0066f60 0x7ffad0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.493+0000 7ffaf2069640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffad0066f60 0x7ffad0069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffae8000f70 tx=0x7ffae8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.497+0000 7ffae2ffd640 1 -- 172.21.15.37:0/3058429313 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffae4057790 con 0x7ffaec144ca0 2024-04-25T09:03:27.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 -- 172.21.15.37:0/3058429313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaec0b1cc0 con 0x7ffaec144ca0 2024-04-25T09:03:27.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffae2ffd640 1 -- 172.21.15.37:0/3058429313 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+3699 (secure 0 0 0) 0x7ffae4057090 con 0x7ffaec144ca0 2024-04-25T09:03:27.674 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:27.674 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6356,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6835/1763102794","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":1763102794},{"type":"v1","addr":"172.21.15.203:6835","nonce":1763102794}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":7192,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/64431494","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6836","nonce":64431494},{"type":"v1","addr":"172.21.15.203:6837","nonce":64431494}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":8712,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/4101859214","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":4101859214},{"type":"v1","addr":"172.21.15.37:6833","nonce":4101859214}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":9282,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2524942703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2524942703},{"type":"v1","addr":"172.21.15.37:6835","nonce":2524942703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89}],"filesystems":[]} 2024-04-25T09:03:27.674 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 91 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 -- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffad0066f60 msgr2=0x7ffad0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffad0066f60 0x7ffad0069420 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ffae8000f70 tx=0x7ffae8009040 comp rx=0 tx=0).stop 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 -- 172.21.15.37:0/3058429313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec144ca0 msgr2=0x7ffaec0adf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec144ca0 0x7ffaec0adf20 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7ffae4007e40 tx=0x7ffae400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 -- 172.21.15.37:0/3058429313 shutdown_connections 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffad0066f60 0x7ffad0069420 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec144ca0 0x7ffaec0adf20 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffaec0ce510 0x7ffaec0ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 --2- 172.21.15.37:0/3058429313 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffaec06ec60 0x7ffaec15b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 -- 172.21.15.37:0/3058429313 >> 172.21.15.37:0/3058429313 conn(0x7ffaec0b11a0 msgr2=0x7ffaec0a8770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 -- 172.21.15.37:0/3058429313 shutdown_connections 2024-04-25T09:03:27.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.685+0000 7ffaf42cd640 1 -- 172.21.15.37:0/3058429313 wait complete. 2024-04-25T09:03:27.692 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:03:27.692 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:03:27.692 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:03:27.692 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:27.693 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:03:27.693 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:03:27.693 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:03:27.693 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:27.693 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:03:27.693 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:03:27.693 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:03:27.694 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:27.694 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:03:27.694 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:03:27.694 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:03:27.694 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:27.703 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:03:27.703 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:03:27.704 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:03:27.704 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:03:27.706 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:03:27.706 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:03:27.707 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:03:27.707 DEBUG:teuthology.orchestra.run.smithi203:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:03:27.749 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:03:27.750 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:27.750 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:03:27.751 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:27.754 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:03:27.754 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:27.755 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:03:27.755 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:27.755 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:03:27.816 INFO:tasks.ceph.mds.c.smithi037.stdout:starting mds.c at 2024-04-25T09:03:27.819 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T09:03:27.826 INFO:tasks.ceph.mds.d.smithi203.stdout:starting mds.d at 2024-04-25T09:03:27.826 INFO:tasks.ceph.mds.b.smithi203.stdout:starting mds.b at 2024-04-25T09:03:27.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18cd1e2640 1 Processor -- start 2024-04-25T09:03:27.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18cd1e2640 1 -- start start 2024-04-25T09:03:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18cd1e2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f18c815da80 0x7f18c80d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18cd1e2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18c80d07f0 0x7f18c806c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18cd1e2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c806be60 con 0x7f18c80d1cd0 2024-04-25T09:03:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18cd1e2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c806c440 con 0x7f18c815da80 2024-04-25T09:03:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18cd1e2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c815de60 con 0x7f18c80d07f0 2024-04-25T09:03:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18c6d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f18c815da80 0x7f18c80d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18c759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18c80d07f0 0x7f18c806c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18c6d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f18c815da80 0x7f18c80d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43766/0 (socket says 172.21.15.37:43766) 2024-04-25T09:03:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18c759e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18c80d07f0 0x7f18c806c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:46914/0 (socket says 172.21.15.37:46914) 2024-04-25T09:03:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18be59c640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f18c80d1cd0 0x7f18c80d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:42818/0 (socket says 172.21.15.37:42818) 2024-04-25T09:03:27.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18c6d9d640 1 -- 172.21.15.37:0/4013694391 learned_addr learned my addr 172.21.15.37:0/4013694391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:27.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18c6d9d640 1 -- 172.21.15.37:0/4013694391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18c80d07f0 msgr2=0x7f18c806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:27.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18c6d9d640 1 --2- 172.21.15.37:0/4013694391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18c80d07f0 0x7f18c806c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18c659c640 1 -- 172.21.15.37:0/4013694391 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880425568 0 0) 0x7f18c806be60 con 0x7f18c80d1cd0 2024-04-25T09:03:27.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18c6d9d640 1 -- 172.21.15.37:0/4013694391 >> v1:172.21.15.203:6789/0 conn(0x7f18c80d1cd0 legacy=0x7f18c80d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.881+0000 7f18c6d9d640 1 -- 172.21.15.37:0/4013694391 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c80d2450 con 0x7f18c815da80 2024-04-25T09:03:27.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18c6d9d640 1 --2- 172.21.15.37:0/4013694391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c815da80 0x7f18c80d4230 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f18b80098a0 tx=0x7f18b801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b29c7eb9b75170 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18c659c640 1 -- 172.21.15.37:0/4013694391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b80a7070 con 0x7f18c815da80 2024-04-25T09:03:27.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18c659c640 1 -- 172.21.15.37:0/4013694391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b80b0070 con 0x7f18c815da80 2024-04-25T09:03:27.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18c659c640 1 -- 172.21.15.37:0/4013694391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b80ab890 con 0x7f18c815da80 2024-04-25T09:03:27.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18cd1e2640 1 -- 172.21.15.37:0/4013694391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c815da80 msgr2=0x7f18c80d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18cd1e2640 1 --2- 172.21.15.37:0/4013694391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c815da80 0x7f18c80d4230 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f18b80098a0 tx=0x7f18b801c510 comp rx=0 tx=0).stop 2024-04-25T09:03:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18cd1e2640 1 -- 172.21.15.37:0/4013694391 shutdown_connections 2024-04-25T09:03:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18cd1e2640 1 --2- 172.21.15.37:0/4013694391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18c80d07f0 0x7f18c806c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18cd1e2640 1 --2- 172.21.15.37:0/4013694391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c815da80 0x7f18c80d4230 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18cd1e2640 1 -- 172.21.15.37:0/4013694391 >> 172.21.15.37:0/4013694391 conn(0x7f18c808b000 msgr2=0x7f18c80d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18cd1e2640 1 -- 172.21.15.37:0/4013694391 shutdown_connections 2024-04-25T09:03:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18cd1e2640 1 -- 172.21.15.37:0/4013694391 wait complete. 2024-04-25T09:03:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.885+0000 7f18cd1e2640 1 Processor -- start 2024-04-25T09:03:27.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18cd1e2640 1 -- start start 2024-04-25T09:03:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18cd1e2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18c80d07f0 0x7f18c80c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18cd1e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c80d1cd0 0x7f18c815f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18cd1e2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18c815da80 0x7f18c8157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18cd1e2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c806d7a0 con 0x7f18c80d1cd0 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18cd1e2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c806d620 con 0x7f18c80d07f0 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18cd1e2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c806d920 con 0x7f18c815da80 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18be59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c80d1cd0 0x7f18c815f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18be59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c80d1cd0 0x7f18c815f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43774/0 (socket says 172.21.15.37:43774) 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18be59c640 1 -- 172.21.15.37:0/412948496 learned_addr learned my addr 172.21.15.37:0/412948496 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18c759e640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18c815da80 0x7f18c8157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18c6d9d640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18c80d07f0 0x7f18c80c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18be59c640 1 -- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18c815da80 msgr2=0x7f18c8157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18be59c640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18c815da80 0x7f18c8157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18be59c640 1 -- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18c80d07f0 msgr2=0x7f18c80c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18be59c640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18c80d07f0 0x7f18c80c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18be59c640 1 -- 172.21.15.37:0/412948496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c80ca130 con 0x7f18c80d1cd0 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18c6d9d640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18c80d07f0 0x7f18c80c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:27.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18be59c640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c80d1cd0 0x7f18c815f310 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f18b000c480 tx=0x7f18b000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18bffff640 1 -- 172.21.15.37:0/412948496 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b0017070 con 0x7f18c80d1cd0 2024-04-25T09:03:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18bffff640 1 -- 172.21.15.37:0/412948496 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b0005ca0 con 0x7f18c80d1cd0 2024-04-25T09:03:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18cd1e2640 1 -- 172.21.15.37:0/412948496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c80ca860 con 0x7f18c80d1cd0 2024-04-25T09:03:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18bffff640 1 -- 172.21.15.37:0/412948496 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b00072b0 con 0x7f18c80d1cd0 2024-04-25T09:03:27.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18cd1e2640 1 -- 172.21.15.37:0/412948496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c80cae20 con 0x7f18c80d1cd0 2024-04-25T09:03:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18cd1e2640 1 -- 172.21.15.37:0/412948496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c804efa0 con 0x7f18c80d1cd0 2024-04-25T09:03:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18bffff640 1 -- 172.21.15.37:0/412948496 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f18b001a070 con 0x7f18c80d1cd0 2024-04-25T09:03:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18bffff640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18a8066f60 0x7f18a8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18bffff640 1 -- 172.21.15.37:0/412948496 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f18b0084280 con 0x7f18c80d1cd0 2024-04-25T09:03:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.889+0000 7f18c6d9d640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18a8066f60 0x7f18a8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:27.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.893+0000 7f18c6d9d640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18a8066f60 0x7f18a8069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f18b80af000 tx=0x7f18b80a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:27.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:27.897+0000 7f18bffff640 1 -- 172.21.15.37:0/412948496 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f18b0055440 con 0x7f18c80d1cd0 2024-04-25T09:03:28.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:28.065+0000 7f18cd1e2640 1 -- 172.21.15.37:0/412948496 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f18c80d05b0 con 0x7f18c80d1cd0 2024-04-25T09:03:29.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.093+0000 7f18bffff640 1 -- 172.21.15.37:0/412948496 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v121) v1 ==== 97+0+0 (secure 0 0 0) 0x7f18b0054d40 con 0x7f18c80d1cd0 2024-04-25T09:03:29.084 INFO:teuthology.orchestra.run.smithi037.stderr: removed all blocklist entries 2024-04-25T09:03:29.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 -- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18a8066f60 msgr2=0x7f18a8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18a8066f60 0x7f18a8069420 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f18b80af000 tx=0x7f18b80a7000 comp rx=0 tx=0).stop 2024-04-25T09:03:29.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 -- 172.21.15.37:0/412948496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c80d1cd0 msgr2=0x7f18c815f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c80d1cd0 0x7f18c815f310 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f18b000c480 tx=0x7f18b000c940 comp rx=0 tx=0).stop 2024-04-25T09:03:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 -- 172.21.15.37:0/412948496 shutdown_connections 2024-04-25T09:03:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f18a8066f60 0x7f18a8069420 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f18c815da80 0x7f18c8157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c80d1cd0 0x7f18c815f310 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 --2- 172.21.15.37:0/412948496 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f18c80d07f0 0x7f18c80c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 -- 172.21.15.37:0/412948496 >> 172.21.15.37:0/412948496 conn(0x7f18c808b000 msgr2=0x7f18c80d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 -- 172.21.15.37:0/412948496 shutdown_connections 2024-04-25T09:03:29.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.097+0000 7f18cd1e2640 1 -- 172.21.15.37:0/412948496 wait complete. 2024-04-25T09:03:29.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:03:29.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a0d19a640 1 Processor -- start 2024-04-25T09:03:29.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a0d19a640 1 -- start start 2024-04-25T09:03:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a0d19a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0a080b0380 0x7f0a080d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a0d19a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0806bff0 con 0x7f0a080b5270 2024-04-25T09:03:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a0d19a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a0806c2d0 con 0x7f0a080d4440 2024-04-25T09:03:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a0d19a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0806cc00 con 0x7f0a080b0380 2024-04-25T09:03:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a06d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0a080b5270 0x7f0a080b5650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:45952/0 (socket says 172.21.15.37:45952) 2024-04-25T09:03:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a06d9d640 1 -- 172.21.15.37:0/3632060110 learned_addr learned my addr 172.21.15.37:0/3632060110 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415766770 0 0) 0x7f0a0806bff0 con 0x7f0a080b5270 2024-04-25T09:03:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f0003530 con 0x7f0a080b5270 2024-04-25T09:03:29.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a0659c640 1 --2- 172.21.15.37:0/3632060110 >> v2:172.21.15.203:3300/0 conn(0x7f0a080b0380 0x7f0a080d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4148333032 0 0) 0x7f09f0003530 con 0x7f0a080b5270 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 >> v2:172.21.15.203:3300/0 conn(0x7f0a080b0380 msgr2=0x7f0a080d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 --2- 172.21.15.37:0/3632060110 >> v2:172.21.15.203:3300/0 conn(0x7f0a080b0380 0x7f0a080d3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 >> v1:172.21.15.203:6789/0 conn(0x7f0a080d4440 legacy=0x7f0a0806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0806c5b0 con 0x7f0a080b5270 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f09f8001800 con 0x7f0a080b5270 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 >> v1:172.21.15.37:6789/0 conn(0x7f0a080b5270 legacy=0x7f0a080b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09f0003a30 0x7f09f0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09f0004810 0x7f09f0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f00051d0 0x7f09f000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0806cc00 con 0x7f09f0003a30 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f000c100 con 0x7f09f00051d0 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f000c300 con 0x7f09f0004810 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a06d9d640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09f0003a30 0x7f09f0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a0759e640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f00051d0 0x7f09f000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a0659c640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09f0004810 0x7f09f0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a06d9d640 1 -- 172.21.15.37:0/3632060110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09f0004810 msgr2=0x7f09f0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a06d9d640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09f0004810 0x7f09f0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a06d9d640 1 -- 172.21.15.37:0/3632060110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f00051d0 msgr2=0x7f09f000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a06d9d640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f00051d0 0x7f09f000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a06d9d640 1 -- 172.21.15.37:0/3632060110 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f0003530 con 0x7f09f0003a30 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a0759e640 1 -- 172.21.15.37:0/3632060110 reap_dead start 2024-04-25T09:03:29.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.245+0000 7f0a0659c640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09f0004810 0x7f09f0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a06d9d640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09f0003a30 0x7f09f0003e50 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f09f800c240 tx=0x7f09f801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8104daa0caee0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8007cf0 con 0x7f09f0003a30 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f80a9020 con 0x7f09f0003a30 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a05d9b640 1 -- 172.21.15.37:0/3632060110 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f80072f0 con 0x7f09f0003a30 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a0d19a640 1 -- 172.21.15.37:0/3632060110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09f0003a30 msgr2=0x7f09f0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a0d19a640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09f0003a30 0x7f09f0003e50 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f09f800c240 tx=0x7f09f801cc60 comp rx=0 tx=0).stop 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a0d19a640 1 -- 172.21.15.37:0/3632060110 shutdown_connections 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a0d19a640 1 --2- 172.21.15.37:0/3632060110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09f0003a30 0x7f09f0003e50 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a0d19a640 1 -- 172.21.15.37:0/3632060110 >> 172.21.15.37:0/3632060110 conn(0x7f0a0808b000 msgr2=0x7f0a080d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a0d19a640 1 -- 172.21.15.37:0/3632060110 shutdown_connections 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a0d19a640 1 -- 172.21.15.37:0/3632060110 wait complete. 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a0d19a640 1 Processor -- start 2024-04-25T09:03:29.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.249+0000 7f0a0d19a640 1 -- start start 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0d19a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f0003a30 0x7f0a08151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0d19a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a08150d70 0x7f0a081506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0d19a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080cc740 0x7f0a080c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0d19a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0815de20 con 0x7f0a080cc740 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0d19a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a080b0760 con 0x7f09f0003a30 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0d19a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0815dfa0 con 0x7f0a08150d70 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080cc740 0x7f0a080c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080cc740 0x7f0a080c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43796/0 (socket says 172.21.15.37:43796) 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0759e640 1 -- 172.21.15.37:0/819183108 learned_addr learned my addr 172.21.15.37:0/819183108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a06d9d640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f0003a30 0x7f0a08151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0659c640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a08150d70 0x7f0a081506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0759e640 1 -- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a08150d70 msgr2=0x7f0a081506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0759e640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a08150d70 0x7f0a081506e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0759e640 1 -- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f0003a30 msgr2=0x7f0a08151c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0759e640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f0003a30 0x7f0a08151c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0759e640 1 -- 172.21.15.37:0/819183108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a080cb6a0 con 0x7f0a080cc740 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a06d9d640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f0003a30 0x7f0a08151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0759e640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080cc740 0x7f0a080c7290 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f0a080b30a0 tx=0x7f0a0000e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f09ef7fe640 1 -- 172.21.15.37:0/819183108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00019070 con 0x7f0a080cc740 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f09ef7fe640 1 -- 172.21.15.37:0/819183108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a00006e30 con 0x7f0a080cc740 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0d19a640 1 -- 172.21.15.37:0/819183108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a080cb980 con 0x7f0a080cc740 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f09ef7fe640 1 -- 172.21.15.37:0/819183108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00003ba0 con 0x7f0a080cc740 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0d19a640 1 -- 172.21.15.37:0/819183108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a080cbf40 con 0x7f0a080cc740 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a0d19a640 1 -- 172.21.15.37:0/819183108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0804efa0 con 0x7f0a080cc740 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f09ef7fe640 1 -- 172.21.15.37:0/819183108 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0a0001c070 con 0x7f0a080cc740 2024-04-25T09:03:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f09ef7fe640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f4066f60 0x7f09f4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f09ef7fe640 1 -- 172.21.15.37:0/819183108 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0a00084420 con 0x7f0a080cc740 2024-04-25T09:03:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.253+0000 7f0a06d9d640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f4066f60 0x7f09f4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.257+0000 7f0a06d9d640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f4066f60 0x7f09f4069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f09f80aa000 tx=0x7f09f80b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.257+0000 7f09ef7fe640 1 -- 172.21.15.37:0/819183108 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0a000555f0 con 0x7f0a080cc740 2024-04-25T09:03:29.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 -- 172.21.15.37:0/819183108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0a080c6a60 con 0x7f0a080cc740 2024-04-25T09:03:29.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f09ef7fe640 1 -- 172.21.15.37:0/819183108 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f0a00054ef0 con 0x7f0a080cc740 2024-04-25T09:03:29.458 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:29.458 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:03:29.458 INFO:teuthology.orchestra.run.smithi037.stdout: "auth_dump": [ 2024-04-25T09:03:29.458 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.458 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.a", 2024-04-25T09:03:29.458 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpm2pLfABAAJbn2dGKdl8SHS8pApIWC1Q==", 2024-04-25T09:03:29.458 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.458 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:03:29.458 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T09:03:29.458 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.b", 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmqB+jDxAAmRnBzN3vqO8HIf/cd3QoJQ==", 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.c", 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmiXRqCRAACtY8cFxwqOT4WNzj9WmeKA==", 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mds.d", 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmX3ARGBAAW/ZUr85NsKtd8HzxE+Yefw==", 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:03:29.460 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.0", 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCSGSpmwhMiJhAAbmQz36LujHeYXaKIOdLLSA==", 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.461 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.1", 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCVGSpm/OLPKxAA9ZmrrWxvt4QDj2lHATnzSg==", 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.2", 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCYGSpmq7tEMxAA/wjwgkrtnTkrMoC6G+YIlQ==", 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:29.462 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.3", 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCbGSpmReB1ORAAkXje/r11I3ArZHErIvijgw==", 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.4", 2024-04-25T09:03:29.463 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQChGSpm4XfLORAAGj+R1Nk/EGmvEdpt8kQJlg==", 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.5", 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQClGSpmd5f1ARAAKF3XG//v3Uqx4bckcD4JMw==", 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.6", 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCoGSpmi9OECBAAJBskMWGCNlAaej9xUc6g8w==", 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "osd.7", 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCrGSpmunQiDhAA3UZo2vzT44HEYUBUia75NA==", 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile osd", 2024-04-25T09:03:29.465 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.0", 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA==", 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow r", 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.1", 2024-04-25T09:03:29.466 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmu8UoJhAAyutD5hzKLZmr57WuAWScuQ==", 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.2", 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmGVVdIxAAY7/VOuqF5OoomqqVxE1Ifw==", 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T09:03:29.467 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.3", 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCPGSpmjFNDKxAAmo6DB9W17TZaGlvkdqkSuQ==", 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow", 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow r", 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow rw", 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow rwx" 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "client.admin", 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm+mGgJRAA9oTRTlZpGBdCZ0/bj580Bg==", 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr": "allow *", 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow *", 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.x", 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpmLChTORAAo6fs/9PDJ5YqqcbfHfPO9Q==", 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout: "entity": "mgr.y", 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout: "key": "AQCOGSpm3NLHNhAAeqSA658L6P9OcrSO5JVzLg==", 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout: "caps": { 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout: "mds": "allow *", 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout: "mon": "allow profile mgr", 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout: "osd": "allow *" 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-25T09:03:29.470 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:03:29.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 -- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f4066f60 msgr2=0x7f09f4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f4066f60 0x7f09f4069420 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f09f80aa000 tx=0x7f09f80b0040 comp rx=0 tx=0).stop 2024-04-25T09:03:29.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 -- 172.21.15.37:0/819183108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080cc740 msgr2=0x7f0a080c7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080cc740 0x7f0a080c7290 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f0a080b30a0 tx=0x7f0a0000e930 comp rx=0 tx=0).stop 2024-04-25T09:03:29.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 -- 172.21.15.37:0/819183108 shutdown_connections 2024-04-25T09:03:29.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f09f4066f60 0x7f09f4069420 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0a080cc740 0x7f0a080c7290 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0a08150d70 0x7f0a081506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 --2- 172.21.15.37:0/819183108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09f0003a30 0x7f0a08151c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 -- 172.21.15.37:0/819183108 >> 172.21.15.37:0/819183108 conn(0x7f0a0808b000 msgr2=0x7f0a080b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 -- 172.21.15.37:0/819183108 shutdown_connections 2024-04-25T09:03:29.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.469+0000 7f0a0d19a640 1 -- 172.21.15.37:0/819183108 wait complete. 2024-04-25T09:03:29.476 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:29.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecc6684640 1 Processor -- start 2024-04-25T09:03:29.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecc6684640 1 -- start start 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecc6684640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fecc00af280 0x7fecc015df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecc6684640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecc00b2c30 0x7fecc006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecc6684640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fecc006c1e0 0x7fecc0161500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecc6684640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc006daa0 con 0x7fecc00af280 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecc6684640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecc0066c30 con 0x7fecc006c1e0 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecc6684640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc00af660 con 0x7fecc00b2c30 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecbffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fecc00af280 0x7fecc015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecbffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fecc00af280 0x7fecc015df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50830/0 (socket says 172.21.15.37:50830) 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecbffff640 1 -- 172.21.15.37:0/1656795360 learned_addr learned my addr 172.21.15.37:0/1656795360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecc4c21640 1 --2- 172.21.15.37:0/1656795360 >> v2:172.21.15.203:3300/0 conn(0x7fecc006c1e0 0x7fecc0161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/1656795360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecc00b2c30 0x7fecc006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecbffff640 1 -- 172.21.15.37:0/1656795360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecc00b2c30 msgr2=0x7fecc006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecbffff640 1 --2- 172.21.15.37:0/1656795360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecc00b2c30 0x7fecc006bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecbffff640 1 -- 172.21.15.37:0/1656795360 >> v2:172.21.15.203:3300/0 conn(0x7fecc006c1e0 msgr2=0x7fecc0161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecbffff640 1 --2- 172.21.15.37:0/1656795360 >> v2:172.21.15.203:3300/0 conn(0x7fecc006c1e0 0x7fecc0161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecbffff640 1 -- 172.21.15.37:0/1656795360 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc015e640 con 0x7fecc00af280 2024-04-25T09:03:29.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.609+0000 7fecc4c21640 1 --2- 172.21.15.37:0/1656795360 >> v2:172.21.15.203:3300/0 conn(0x7fecc006c1e0 0x7fecc0161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:29.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecbffff640 1 --2- 172.21.15.37:0/1656795360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc00af280 0x7fecc015df10 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fecc0072c30 tx=0x7fecb001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9259bcf239db81f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecbeffd640 1 -- 172.21.15.37:0/1656795360 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00b0070 con 0x7fecc00af280 2024-04-25T09:03:29.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecbeffd640 1 -- 172.21.15.37:0/1656795360 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb00ab600 con 0x7fecc00af280 2024-04-25T09:03:29.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecbeffd640 1 -- 172.21.15.37:0/1656795360 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00ab9d0 con 0x7fecc00af280 2024-04-25T09:03:29.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecc6684640 1 -- 172.21.15.37:0/1656795360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc00af280 msgr2=0x7fecc015df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecc6684640 1 --2- 172.21.15.37:0/1656795360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc00af280 0x7fecc015df10 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fecc0072c30 tx=0x7fecb001c260 comp rx=0 tx=0).stop 2024-04-25T09:03:29.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecc6684640 1 -- 172.21.15.37:0/1656795360 shutdown_connections 2024-04-25T09:03:29.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecc6684640 1 --2- 172.21.15.37:0/1656795360 >> v2:172.21.15.203:3300/0 conn(0x7fecc006c1e0 0x7fecc0161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecc6684640 1 --2- 172.21.15.37:0/1656795360 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecc00b2c30 0x7fecc006bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecc6684640 1 --2- 172.21.15.37:0/1656795360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc00af280 0x7fecc015df10 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecc6684640 1 -- 172.21.15.37:0/1656795360 >> 172.21.15.37:0/1656795360 conn(0x7fecc0071810 msgr2=0x7fecc0071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecc6684640 1 -- 172.21.15.37:0/1656795360 shutdown_connections 2024-04-25T09:03:29.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecc6684640 1 -- 172.21.15.37:0/1656795360 wait complete. 2024-04-25T09:03:29.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecc6684640 1 Processor -- start 2024-04-25T09:03:29.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.613+0000 7fecc6684640 1 -- start start 2024-04-25T09:03:29.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecc6684640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc006c1e0 0x7fecc014d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecc6684640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecc00af280 0x7fecc00adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecc6684640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecc00b2c30 0x7fecc015ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecc6684640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc006e0b0 con 0x7fecc006c1e0 2024-04-25T09:03:29.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecc6684640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc006df30 con 0x7fecc00af280 2024-04-25T09:03:29.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecc6684640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc006e230 con 0x7fecc00b2c30 2024-04-25T09:03:29.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc006c1e0 0x7fecc014d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc006c1e0 0x7fecc014d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50840/0 (socket says 172.21.15.37:50840) 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbffff640 1 -- 172.21.15.37:0/2631873405 learned_addr learned my addr 172.21.15.37:0/2631873405 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecc00af280 0x7fecc00adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecc4c21640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecc00b2c30 0x7fecc015ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbffff640 1 -- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecc00b2c30 msgr2=0x7fecc015ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbffff640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecc00b2c30 0x7fecc015ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbffff640 1 -- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecc00af280 msgr2=0x7fecc00adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbffff640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecc00af280 0x7fecc00adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbffff640 1 -- 172.21.15.37:0/2631873405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc0152be0 con 0x7fecc006c1e0 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbffff640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc006c1e0 0x7fecc014d9c0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fecb00097c0 tx=0x7fecb001aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbcff9640 1 -- 172.21.15.37:0/2631873405 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00ba020 con 0x7fecc006c1e0 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbcff9640 1 -- 172.21.15.37:0/2631873405 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb00b6620 con 0x7fecc006c1e0 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecc6684640 1 -- 172.21.15.37:0/2631873405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc0152ec0 con 0x7fecc006c1e0 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbcff9640 1 -- 172.21.15.37:0/2631873405 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb00b0040 con 0x7fecc006c1e0 2024-04-25T09:03:29.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecc6684640 1 -- 172.21.15.37:0/2631873405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc0153480 con 0x7fecc006c1e0 2024-04-25T09:03:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecc6684640 1 -- 172.21.15.37:0/2631873405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc004efa0 con 0x7fecc006c1e0 2024-04-25T09:03:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbcff9640 1 -- 172.21.15.37:0/2631873405 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fecb00bd020 con 0x7fecc006c1e0 2024-04-25T09:03:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbcff9640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fecac066f60 0x7fecac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbcff9640 1 -- 172.21.15.37:0/2631873405 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fecb011f240 con 0x7fecc006c1e0 2024-04-25T09:03:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.617+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fecac066f60 0x7fecac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.621+0000 7fecbf7fe640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fecac066f60 0x7fecac069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fecc015dad0 tx=0x7fecb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.621+0000 7fecbcff9640 1 -- 172.21.15.37:0/2631873405 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fecb00c4030 con 0x7fecc006c1e0 2024-04-25T09:03:29.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.777+0000 7fecc6684640 1 -- 172.21.15.37:0/2631873405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecc00727c0 con 0x7fecc006c1e0 2024-04-25T09:03:29.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.777+0000 7fecbcff9640 1 -- 172.21.15.37:0/2631873405 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) v1 ==== 74+0+13138 (secure 0 0 0) 0x7fecb0016110 con 0x7fecc006c1e0 2024-04-25T09:03:29.769 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:29.769 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":121,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:29.095344+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 -- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fecac066f60 msgr2=0x7fecac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fecac066f60 0x7fecac069420 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fecc015dad0 tx=0x7fecb4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 -- 172.21.15.37:0/2631873405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc006c1e0 msgr2=0x7fecc014d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc006c1e0 0x7fecc014d9c0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fecb00097c0 tx=0x7fecb001aaa0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 -- 172.21.15.37:0/2631873405 shutdown_connections 2024-04-25T09:03:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fecac066f60 0x7fecac069420 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fecc00b2c30 0x7fecc015ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fecc00af280 0x7fecc00adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 --2- 172.21.15.37:0/2631873405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fecc006c1e0 0x7fecc014d9c0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 -- 172.21.15.37:0/2631873405 >> 172.21.15.37:0/2631873405 conn(0x7fecc0071810 msgr2=0x7fecc0073620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 -- 172.21.15.37:0/2631873405 shutdown_connections 2024-04-25T09:03:29.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.781+0000 7fecc6684640 1 -- 172.21.15.37:0/2631873405 wait complete. 2024-04-25T09:03:29.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:29.787 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.921+0000 7f8ded520640 1 Processor -- start 2024-04-25T09:03:29.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8ded520640 1 -- start start 2024-04-25T09:03:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8ded520640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8de80cf590 0x7f8de815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8ded520640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8de80cf970 0x7f8de80c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8ded520640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80c9ef0 0x7f8de80b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8ded520640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de80ca780 con 0x7f8de80cf590 2024-04-25T09:03:29.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8ded520640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de80cd440 con 0x7f8de80cf970 2024-04-25T09:03:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8ded520640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80cdce0 con 0x7f8de80c9ef0 2024-04-25T09:03:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8de80cf590 0x7f8de815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de6ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8de80cf590 0x7f8de815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50846/0 (socket says 172.21.15.37:50846) 2024-04-25T09:03:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de6ffd640 1 -- 172.21.15.37:0/2294311048 learned_addr learned my addr 172.21.15.37:0/2294311048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de67fc640 1 --2- 172.21.15.37:0/2294311048 >> v2:172.21.15.203:3300/0 conn(0x7f8de80cf970 0x7f8de80c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de77fe640 1 --2- 172.21.15.37:0/2294311048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80c9ef0 0x7f8de80b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de6ffd640 1 -- 172.21.15.37:0/2294311048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80c9ef0 msgr2=0x7f8de80b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de6ffd640 1 --2- 172.21.15.37:0/2294311048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80c9ef0 0x7f8de80b21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de6ffd640 1 -- 172.21.15.37:0/2294311048 >> v2:172.21.15.203:3300/0 conn(0x7f8de80cf970 msgr2=0x7f8de80c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de6ffd640 1 --2- 172.21.15.37:0/2294311048 >> v2:172.21.15.203:3300/0 conn(0x7f8de80cf970 0x7f8de80c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de6ffd640 1 -- 172.21.15.37:0/2294311048 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de80ca4c0 con 0x7f8de80cf590 2024-04-25T09:03:29.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de6ffd640 1 --2- 172.21.15.37:0/2294311048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de80cf590 0x7f8de815ecb0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f8de80d00e0 tx=0x7f8dd801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=949fe6e4c9625131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de5ffb640 1 -- 172.21.15.37:0/2294311048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80b0070 con 0x7f8de80cf590 2024-04-25T09:03:29.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de5ffb640 1 -- 172.21.15.37:0/2294311048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd80ab600 con 0x7f8de80cf590 2024-04-25T09:03:29.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.925+0000 7f8de5ffb640 1 -- 172.21.15.37:0/2294311048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80ab9d0 con 0x7f8de80cf590 2024-04-25T09:03:29.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 -- 172.21.15.37:0/2294311048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de80cf590 msgr2=0x7f8de815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 --2- 172.21.15.37:0/2294311048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de80cf590 0x7f8de815ecb0 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f8de80d00e0 tx=0x7f8dd801c230 comp rx=0 tx=0).stop 2024-04-25T09:03:29.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 -- 172.21.15.37:0/2294311048 shutdown_connections 2024-04-25T09:03:29.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 --2- 172.21.15.37:0/2294311048 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80c9ef0 0x7f8de80b21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 --2- 172.21.15.37:0/2294311048 >> v2:172.21.15.203:3300/0 conn(0x7f8de80cf970 0x7f8de80c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 --2- 172.21.15.37:0/2294311048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de80cf590 0x7f8de815ecb0 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 -- 172.21.15.37:0/2294311048 >> 172.21.15.37:0/2294311048 conn(0x7f8de806ae60 msgr2=0x7f8de806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:29.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 -- 172.21.15.37:0/2294311048 shutdown_connections 2024-04-25T09:03:29.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 -- 172.21.15.37:0/2294311048 wait complete. 2024-04-25T09:03:29.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 Processor -- start 2024-04-25T09:03:29.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 -- start start 2024-04-25T09:03:29.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80c9ef0 0x7f8de806e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de80cf590 0x7f8de80b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de80cf970 0x7f8de8098880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80ac2f0 con 0x7f8de80cf970 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80ac170 con 0x7f8de80cf590 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80cbde0 con 0x7f8de80c9ef0 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de80cf970 0x7f8de8098880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de80cf970 0x7f8de8098880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50850/0 (socket says 172.21.15.37:50850) 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de77fe640 1 -- 172.21.15.37:0/152881156 learned_addr learned my addr 172.21.15.37:0/152881156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de67fc640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de80cf590 0x7f8de80b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de6ffd640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80c9ef0 0x7f8de806e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de77fe640 1 -- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80c9ef0 msgr2=0x7f8de806e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de77fe640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80c9ef0 0x7f8de806e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de77fe640 1 -- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de80cf590 msgr2=0x7f8de80b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de77fe640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de80cf590 0x7f8de80b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:29.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de77fe640 1 -- 172.21.15.37:0/152881156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de81566e0 con 0x7f8de80cf970 2024-04-25T09:03:29.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de6ffd640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80c9ef0 0x7f8de806e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:29.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8de77fe640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de80cf970 0x7f8de8098880 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f8de0007e90 tx=0x7f8de000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/152881156 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0014420 con 0x7f8de80cf970 2024-04-25T09:03:29.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/152881156 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0019070 con 0x7f8de80cf970 2024-04-25T09:03:29.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 -- 172.21.15.37:0/152881156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de80b0950 con 0x7f8de80cf970 2024-04-25T09:03:29.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/152881156 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0014820 con 0x7f8de80cf970 2024-04-25T09:03:29.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.929+0000 7f8ded520640 1 -- 172.21.15.37:0/152881156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de80b0f10 con 0x7f8de80cf970 2024-04-25T09:03:29.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.933+0000 7f8ded520640 1 -- 172.21.15.37:0/152881156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de80b1110 con 0x7f8de80cf970 2024-04-25T09:03:29.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.933+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/152881156 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8de00149c0 con 0x7f8de80cf970 2024-04-25T09:03:29.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.933+0000 7f8ddf7fe640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:29.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.933+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/152881156 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8de0085640 con 0x7f8de80cf970 2024-04-25T09:03:29.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.933+0000 7f8de6ffd640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:29.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.933+0000 7f8de6ffd640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 0x7f8dc8069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80ad000 tx=0x7f8dd8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:29.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:29.937+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/152881156 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8de80b1110 con 0x7f8de80cf970 2024-04-25T09:03:30.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.097+0000 7f8ded520640 1 -- 172.21.15.37:0/152881156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8de804efa0 con 0x7f8de80cf970 2024-04-25T09:03:30.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.097+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/152881156 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8de00568a0 con 0x7f8de80cf970 2024-04-25T09:03:30.085 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:03:30.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 -- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 msgr2=0x7f8dc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 0x7f8dc8069420 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80ad000 tx=0x7f8dd8008040 comp rx=0 tx=0).stop 2024-04-25T09:03:30.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 -- 172.21.15.37:0/152881156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de80cf970 msgr2=0x7f8de8098880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de80cf970 0x7f8de8098880 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f8de0007e90 tx=0x7f8de000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 -- 172.21.15.37:0/152881156 shutdown_connections 2024-04-25T09:03:30.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de80cf970 0x7f8de8098880 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de80cf590 0x7f8de80b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 --2- 172.21.15.37:0/152881156 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80c9ef0 0x7f8de806e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 -- 172.21.15.37:0/152881156 >> 172.21.15.37:0/152881156 conn(0x7f8de806ae60 msgr2=0x7f8de806daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:30.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 -- 172.21.15.37:0/152881156 shutdown_connections 2024-04-25T09:03:30.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.101+0000 7f8ded520640 1 -- 172.21.15.37:0/152881156 wait complete. 2024-04-25T09:03:30.104 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:03:30.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f09584da640 1 Processor -- start 2024-04-25T09:03:30.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f09584da640 1 -- start start 2024-04-25T09:03:30.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f09584da640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f095015da80 0x7f09500d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f09584da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 0x7f09500d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f09584da640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095006be60 con 0x7f09500d0620 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f09584da640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095006c440 con 0x7f095015da80 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f09584da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095015de60 con 0x7f09500d1cd0 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f0956276640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f095015da80 0x7f09500d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f0956a77640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f09500d0620 0x7f09500ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38004/0 (socket says 172.21.15.37:38004) 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f0956a77640 1 -- 172.21.15.37:0/3412330341 learned_addr learned my addr 172.21.15.37:0/3412330341 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f0955a75640 1 --2- 172.21.15.37:0/3412330341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 0x7f09500d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f0956276640 1 -- 172.21.15.37:0/3412330341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 msgr2=0x7f09500d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f0956276640 1 --2- 172.21.15.37:0/3412330341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 0x7f09500d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f0956276640 1 -- 172.21.15.37:0/3412330341 >> v1:172.21.15.37:6789/0 conn(0x7f09500d0620 legacy=0x7f09500ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f0956276640 1 -- 172.21.15.37:0/3412330341 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500d2450 con 0x7f095015da80 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f0955274640 1 -- 172.21.15.37:0/3412330341 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1302800748 0 0) 0x7f095006be60 con 0x7f09500d0620 2024-04-25T09:03:30.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.229+0000 7f0955a75640 1 --2- 172.21.15.37:0/3412330341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 0x7f09500d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:30.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f0956276640 1 --2- 172.21.15.37:0/3412330341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f095015da80 0x7f09500d4230 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f094c0097c0 tx=0x7f094c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad10f6f03546f016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f0955274640 1 -- 172.21.15.37:0/3412330341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0b0070 con 0x7f095015da80 2024-04-25T09:03:30.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f0955274640 1 -- 172.21.15.37:0/3412330341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c0ab600 con 0x7f095015da80 2024-04-25T09:03:30.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f0955274640 1 -- 172.21.15.37:0/3412330341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0ab9d0 con 0x7f095015da80 2024-04-25T09:03:30.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 -- 172.21.15.37:0/3412330341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f095015da80 msgr2=0x7f09500d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 --2- 172.21.15.37:0/3412330341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f095015da80 0x7f09500d4230 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f094c0097c0 tx=0x7f094c01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 -- 172.21.15.37:0/3412330341 shutdown_connections 2024-04-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 --2- 172.21.15.37:0/3412330341 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 0x7f09500d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 --2- 172.21.15.37:0/3412330341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f095015da80 0x7f09500d4230 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 -- 172.21.15.37:0/3412330341 >> 172.21.15.37:0/3412330341 conn(0x7f095008b000 msgr2=0x7f09500d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 -- 172.21.15.37:0/3412330341 shutdown_connections 2024-04-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 -- 172.21.15.37:0/3412330341 wait complete. 2024-04-25T09:03:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 Processor -- start 2024-04-25T09:03:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 -- start start 2024-04-25T09:03:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09500d0620 0x7f0950155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 0x7f09500b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095015da80 0x7f09500b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095016a730 con 0x7f09500d0620 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095016a5b0 con 0x7f095015da80 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f09584da640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095016a8b0 con 0x7f09500d1cd0 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.233+0000 7f0956276640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09500d0620 0x7f0950155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0956276640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09500d0620 0x7f0950155f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50860/0 (socket says 172.21.15.37:50860) 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0956276640 1 -- 172.21.15.37:0/2821779874 learned_addr learned my addr 172.21.15.37:0/2821779874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0955a75640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 0x7f09500b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0956276640 1 -- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 msgr2=0x7f09500b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0956a77640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095015da80 0x7f09500b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0956276640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 0x7f09500b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0956276640 1 -- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095015da80 msgr2=0x7f09500b5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0956276640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095015da80 0x7f09500b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0956276640 1 -- 172.21.15.37:0/2821779874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950156470 con 0x7f09500d0620 2024-04-25T09:03:30.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0955a75640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 0x7f09500b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0956276640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09500d0620 0x7f0950155f30 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f094c0097c0 tx=0x7f094c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0946ffd640 1 -- 172.21.15.37:0/2821779874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0b0040 con 0x7f09500d0620 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0946ffd640 1 -- 172.21.15.37:0/2821779874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c0b43f0 con 0x7f09500d0620 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f09584da640 1 -- 172.21.15.37:0/2821779874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09501601e0 con 0x7f09500d0620 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0946ffd640 1 -- 172.21.15.37:0/2821779874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0b47c0 con 0x7f09500d0620 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f09584da640 1 -- 172.21.15.37:0/2821779874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09501604a0 con 0x7f09500d0620 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f09584da640 1 -- 172.21.15.37:0/2821779874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095004efa0 con 0x7f09500d0620 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0946ffd640 1 -- 172.21.15.37:0/2821779874 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f094c0b4960 con 0x7f09500d0620 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0946ffd640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0934066f60 0x7f0934069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0946ffd640 1 -- 172.21.15.37:0/2821779874 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f094c11e9a0 con 0x7f09500d0620 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0955a75640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0934066f60 0x7f0934069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.237+0000 7f0955a75640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0934066f60 0x7f0934069420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0940006fd0 tx=0x7f0940008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.241+0000 7f0946ffd640 1 -- 172.21.15.37:0/2821779874 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f094c016090 con 0x7f09500d0620 2024-04-25T09:03:30.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 -- 172.21.15.37:0/2821779874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f09500b2b80 con 0x7f09500d0620 2024-04-25T09:03:30.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f0946ffd640 1 -- 172.21.15.37:0/2821779874 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v121) v1 ==== 84+0+65 (secure 0 0 0) 0x7f094c0efc00 con 0x7f09500d0620 2024-04-25T09:03:30.390 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:30.390 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-25T09:03:30.390 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-25T09:03:30.390 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-25T09:03:30.390 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-25T09:03:30.390 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:30.390 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-25T09:03:30.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 -- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0934066f60 msgr2=0x7f0934069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0934066f60 0x7f0934069420 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0940006fd0 tx=0x7f0940008040 comp rx=0 tx=0).stop 2024-04-25T09:03:30.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 -- 172.21.15.37:0/2821779874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09500d0620 msgr2=0x7f0950155f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09500d0620 0x7f0950155f30 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f094c0097c0 tx=0x7f094c019240 comp rx=0 tx=0).stop 2024-04-25T09:03:30.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 -- 172.21.15.37:0/2821779874 shutdown_connections 2024-04-25T09:03:30.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0934066f60 0x7f0934069420 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f095015da80 0x7f09500b5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500d1cd0 0x7f09500b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 --2- 172.21.15.37:0/2821779874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09500d0620 0x7f0950155f30 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 -- 172.21.15.37:0/2821779874 >> 172.21.15.37:0/2821779874 conn(0x7f095008b000 msgr2=0x7f09500b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:30.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 -- 172.21.15.37:0/2821779874 shutdown_connections 2024-04-25T09:03:30.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.401+0000 7f09584da640 1 -- 172.21.15.37:0/2821779874 wait complete. 2024-04-25T09:03:30.408 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:03:30.408 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:03:30.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ef33c640 1 Processor -- start 2024-04-25T09:03:30.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ef33c640 1 -- start start 2024-04-25T09:03:30.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ef33c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e8161120 0x7f37e80b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ef33c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f37e80b1140 0x7f37e80b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ef33c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e8161500 con 0x7f37e80b1ae0 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ef33c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e80b0770 con 0x7f37e80b1140 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ef33c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80d1ff0 con 0x7f37e8161120 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ed8d9640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f37e80b1ae0 0x7f37e8166790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38012/0 (socket says 172.21.15.37:38012) 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ed8d9640 1 -- 172.21.15.37:0/575559762 learned_addr learned my addr 172.21.15.37:0/575559762 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076145679 0 0) 0x7f37e8161500 con 0x7f37e80b1ae0 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ed0d8640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e8161120 0x7f37e80b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ec8d7640 1 --2- 172.21.15.37:0/575559762 >> v2:172.21.15.203:3300/0 conn(0x7f37e80b1140 0x7f37e80b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37d4003530 con 0x7f37e80b1ae0 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 666252906 0 0) 0x7f37d4003530 con 0x7f37e80b1ae0 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e8161120 msgr2=0x7f37e80b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e8161120 0x7f37e80b0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 >> v2:172.21.15.203:3300/0 conn(0x7f37e80b1140 msgr2=0x7f37e80b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 --2- 172.21.15.37:0/575559762 >> v2:172.21.15.203:3300/0 conn(0x7f37e80b1140 0x7f37e80b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e80d22d0 con 0x7f37e80b1ae0 2024-04-25T09:03:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f37e0001a00 con 0x7f37e80b1ae0 2024-04-25T09:03:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ec8d7640 1 --2- 172.21.15.37:0/575559762 >> v2:172.21.15.203:3300/0 conn(0x7f37e80b1140 0x7f37e80b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:30.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 >> v1:172.21.15.37:6789/0 conn(0x7f37e80b1ae0 legacy=0x7f37e8166790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37d40039c0 0x7f37d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37d4004810 0x7f37d4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e8161120 0x7f37d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80b0770 con 0x7f37d4004810 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80d1ff0 con 0x7f37d40039c0 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d400ba80 con 0x7f37e8161120 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ec8d7640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37d4004810 0x7f37d4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ed0d8640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37d40039c0 0x7f37d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ed8d9640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e8161120 0x7f37d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ec8d7640 1 -- 172.21.15.37:0/575559762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e8161120 msgr2=0x7f37d40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ec8d7640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e8161120 0x7f37d40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ec8d7640 1 -- 172.21.15.37:0/575559762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37d40039c0 msgr2=0x7f37d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:30.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ec8d7640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37d40039c0 0x7f37d4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.545+0000 7f37ec8d7640 1 -- 172.21.15.37:0/575559762 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d4003530 con 0x7f37d4004810 2024-04-25T09:03:30.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37ec8d7640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37d4004810 0x7f37d4004c90 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f37e40098a0 tx=0x7f37e401c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fabb370c4767cde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e400f040 con 0x7f37d4004810 2024-04-25T09:03:30.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e40a8660 con 0x7f37d4004810 2024-04-25T09:03:30.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37dffff640 1 -- 172.21.15.37:0/575559762 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e40a8a10 con 0x7f37d4004810 2024-04-25T09:03:30.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37ef33c640 1 -- 172.21.15.37:0/575559762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37d4004810 msgr2=0x7f37d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37ef33c640 1 --2- 172.21.15.37:0/575559762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37d4004810 0x7f37d4004c90 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f37e40098a0 tx=0x7f37e401c5f0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37ed8d9640 1 -- 172.21.15.37:0/575559762 reap_dead start 2024-04-25T09:03:30.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37ef33c640 1 -- 172.21.15.37:0/575559762 shutdown_connections 2024-04-25T09:03:30.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37ef33c640 1 -- 172.21.15.37:0/575559762 >> 172.21.15.37:0/575559762 conn(0x7f37e808b000 msgr2=0x7f37e80d36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:30.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37ef33c640 1 -- 172.21.15.37:0/575559762 shutdown_connections 2024-04-25T09:03:30.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37ef33c640 1 -- 172.21.15.37:0/575559762 wait complete. 2024-04-25T09:03:30.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37ef33c640 1 Processor -- start 2024-04-25T09:03:30.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.549+0000 7f37ef33c640 1 -- start start 2024-04-25T09:03:30.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ef33c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37d4004810 0x7f37e80ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ef33c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e814ffe0 0x7f37e8150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ef33c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e814d230 0x7f37e814d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ef33c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80d2890 con 0x7f37e814d230 2024-04-25T09:03:30.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ef33c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80b0400 con 0x7f37d4004810 2024-04-25T09:03:30.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ef33c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e80b2780 con 0x7f37e814ffe0 2024-04-25T09:03:30.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed8d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e814d230 0x7f37e814d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed8d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e814d230 0x7f37e814d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50886/0 (socket says 172.21.15.37:50886) 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed8d9640 1 -- 172.21.15.37:0/1926928483 learned_addr learned my addr 172.21.15.37:0/1926928483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed0d8640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37d4004810 0x7f37e80ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ec8d7640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e814ffe0 0x7f37e8150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed8d9640 1 -- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e814ffe0 msgr2=0x7f37e8150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed8d9640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e814ffe0 0x7f37e8150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed8d9640 1 -- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37d4004810 msgr2=0x7f37e80ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed8d9640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37d4004810 0x7f37e80ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed8d9640 1 -- 172.21.15.37:0/1926928483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e80b7e00 con 0x7f37e814d230 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed0d8640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37d4004810 0x7f37e80ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ec8d7640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e814ffe0 0x7f37e8150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed8d9640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e814d230 0x7f37e814d6b0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f37e0000f90 tx=0x7f37e0010cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ddffb640 1 -- 172.21.15.37:0/1926928483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e001a070 con 0x7f37e814d230 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ddffb640 1 -- 172.21.15.37:0/1926928483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37e0011040 con 0x7f37e814d230 2024-04-25T09:03:30.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ef33c640 1 -- 172.21.15.37:0/1926928483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e80b80e0 con 0x7f37e814d230 2024-04-25T09:03:30.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ddffb640 1 -- 172.21.15.37:0/1926928483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37e000eca0 con 0x7f37e814d230 2024-04-25T09:03:30.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ef33c640 1 -- 172.21.15.37:0/1926928483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e80cacf0 con 0x7f37e814d230 2024-04-25T09:03:30.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ef33c640 1 -- 172.21.15.37:0/1926928483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e8067c30 con 0x7f37e814d230 2024-04-25T09:03:30.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ddffb640 1 -- 172.21.15.37:0/1926928483 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f37e001d070 con 0x7f37e814d230 2024-04-25T09:03:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ddffb640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f37cc066f60 0x7f37cc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ddffb640 1 -- 172.21.15.37:0/1926928483 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f37e0087400 con 0x7f37e814d230 2024-04-25T09:03:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.553+0000 7f37ed0d8640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f37cc066f60 0x7f37cc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.557+0000 7f37ed0d8640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f37cc066f60 0x7f37cc069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f37d800e810 tx=0x7f37d8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.557+0000 7f37ddffb640 1 -- 172.21.15.37:0/1926928483 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f37e0058660 con 0x7f37e814d230 2024-04-25T09:03:30.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:30.717+0000 7f37ef33c640 1 -- 172.21.15.37:0/1926928483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f37e80d0b40 con 0x7f37e814d230 2024-04-25T09:03:31.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.101+0000 7f37ddffb640 1 -- 172.21.15.37:0/1926928483 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v122) v1 ==== 138+0+0 (secure 0 0 0) 0x7f37e0057f60 con 0x7f37e814d230 2024-04-25T09:03:31.089 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-25T09:03:31.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 -- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f37cc066f60 msgr2=0x7f37cc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f37cc066f60 0x7f37cc069420 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f37d800e810 tx=0x7f37d8008040 comp rx=0 tx=0).stop 2024-04-25T09:03:31.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 -- 172.21.15.37:0/1926928483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e814d230 msgr2=0x7f37e814d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e814d230 0x7f37e814d6b0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f37e0000f90 tx=0x7f37e0010cf0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 -- 172.21.15.37:0/1926928483 shutdown_connections 2024-04-25T09:03:31.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f37cc066f60 0x7f37cc069420 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37e814d230 0x7f37e814d6b0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f37e814ffe0 0x7f37e8150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 --2- 172.21.15.37:0/1926928483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f37d4004810 0x7f37e80ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 -- 172.21.15.37:0/1926928483 >> 172.21.15.37:0/1926928483 conn(0x7f37e808b000 msgr2=0x7f37e806d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 -- 172.21.15.37:0/1926928483 shutdown_connections 2024-04-25T09:03:31.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.105+0000 7f37ef33c640 1 -- 172.21.15.37:0/1926928483 wait complete. 2024-04-25T09:03:31.122 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:03:31.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.265+0000 7f8fda158640 1 Processor -- start 2024-04-25T09:03:31.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.265+0000 7f8fda158640 1 -- start start 2024-04-25T09:03:31.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.265+0000 7f8fda158640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd40cfa20 0x7f8fd4072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.265+0000 7f8fda158640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd40cfe90 con 0x7f8fd415b6b0 2024-04-25T09:03:31.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.265+0000 7f8fda158640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fd40cbd20 con 0x7f8fd40cf590 2024-04-25T09:03:31.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.265+0000 7f8fda158640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd415ba90 con 0x7f8fd40cfa20 2024-04-25T09:03:31.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.265+0000 7f8fd37fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8fd415b6b0 0x7f8fd40ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38018/0 (socket says 172.21.15.37:38018) 2024-04-25T09:03:31.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.265+0000 7f8fd37fe640 1 -- 172.21.15.37:0/591494653 learned_addr learned my addr 172.21.15.37:0/591494653 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:31.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.265+0000 7f8fd3fff640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd40cfa20 0x7f8fd4072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.265+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1281508940 0 0) 0x7f8fd40cfe90 con 0x7f8fd415b6b0 2024-04-25T09:03:31.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.265+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fc0003530 con 0x7f8fd415b6b0 2024-04-25T09:03:31.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4153446959 0 0) 0x7f8fc0003530 con 0x7f8fd415b6b0 2024-04-25T09:03:31.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd40cfa20 msgr2=0x7f8fd4072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd40cfa20 0x7f8fd4072710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 >> v1:172.21.15.203:6789/0 conn(0x7f8fd40cf590 legacy=0x7f8fd4072000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd40cc000 con 0x7f8fd415b6b0 2024-04-25T09:03:31.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f8fc4001680 con 0x7f8fd415b6b0 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 >> v1:172.21.15.37:6789/0 conn(0x7f8fd415b6b0 legacy=0x7f8fd40ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd40cfa20 0x7f8fc0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0003fe0 0x7f8fc0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc0004d50 0x7f8fc00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd415ba90 con 0x7f8fc0004d50 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc000bb30 con 0x7f8fc0003fe0 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc000bd30 con 0x7f8fd40cfa20 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd3fff640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc0004d50 0x7f8fc00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fcaffd640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0003fe0 0x7f8fc0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd37fe640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd40cfa20 0x7f8fc0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd3fff640 1 -- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd40cfa20 msgr2=0x7f8fc0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd3fff640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd40cfa20 0x7f8fc0003a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd3fff640 1 -- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0003fe0 msgr2=0x7f8fc0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd3fff640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0003fe0 0x7f8fc0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd3fff640 1 -- 172.21.15.37:0/591494653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0003530 con 0x7f8fc0004d50 2024-04-25T09:03:31.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fcaffd640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc0003fe0 0x7f8fc0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd3fff640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc0004d50 0x7f8fc00051d0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009fc0 tx=0x7f8fcc01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bfc04046d202f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc0b2070 con 0x7f8fc0004d50 2024-04-25T09:03:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fcc0a5c90 con 0x7f8fc0004d50 2024-04-25T09:03:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd2ffd640 1 -- 172.21.15.37:0/591494653 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc009c10 con 0x7f8fc0004d50 2024-04-25T09:03:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fda158640 1 -- 172.21.15.37:0/591494653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc0004d50 msgr2=0x7f8fc00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fda158640 1 --2- 172.21.15.37:0/591494653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc0004d50 0x7f8fc00051d0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc009fc0 tx=0x7f8fcc01cc80 comp rx=0 tx=0).stop 2024-04-25T09:03:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fd3fff640 1 -- 172.21.15.37:0/591494653 reap_dead start 2024-04-25T09:03:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fda158640 1 -- 172.21.15.37:0/591494653 shutdown_connections 2024-04-25T09:03:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fda158640 1 -- 172.21.15.37:0/591494653 >> 172.21.15.37:0/591494653 conn(0x7f8fd406ae60 msgr2=0x7f8fd406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fda158640 1 -- 172.21.15.37:0/591494653 shutdown_connections 2024-04-25T09:03:31.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.269+0000 7f8fda158640 1 -- 172.21.15.37:0/591494653 wait complete. 2024-04-25T09:03:31.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fda158640 1 Processor -- start 2024-04-25T09:03:31.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fda158640 1 -- start start 2024-04-25T09:03:31.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fda158640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fc0004d50 0x7f8fd4098920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fda158640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4159b90 0x7f8fd40c45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fda158640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd4158cb0 0x7f8fd40c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fda158640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40cc8a0 con 0x7f8fd4159b90 2024-04-25T09:03:31.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fda158640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40cc5c0 con 0x7f8fd4158cb0 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fda158640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd40cba30 con 0x7f8fc0004d50 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fcaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4159b90 0x7f8fd40c45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fcaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4159b90 0x7f8fd40c45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50910/0 (socket says 172.21.15.37:50910) 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fcaffd640 1 -- 172.21.15.37:0/1520595727 learned_addr learned my addr 172.21.15.37:0/1520595727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fd3fff640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd4158cb0 0x7f8fd40c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fd37fe640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fc0004d50 0x7f8fd4098920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fcaffd640 1 -- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fc0004d50 msgr2=0x7f8fd4098920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fcaffd640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fc0004d50 0x7f8fd4098920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fcaffd640 1 -- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd4158cb0 msgr2=0x7f8fd40c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fcaffd640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd4158cb0 0x7f8fd40c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fcaffd640 1 -- 172.21.15.37:0/1520595727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd415a0d0 con 0x7f8fd4159b90 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fcaffd640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4159b90 0x7f8fd40c45e0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00d7d0 tx=0x7f8fbc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fd0ff9640 1 -- 172.21.15.37:0/1520595727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc005270 con 0x7f8fd4159b90 2024-04-25T09:03:31.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fd0ff9640 1 -- 172.21.15.37:0/1520595727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc017070 con 0x7f8fd4159b90 2024-04-25T09:03:31.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fda158640 1 -- 172.21.15.37:0/1520595727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd415a3b0 con 0x7f8fd4159b90 2024-04-25T09:03:31.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fd0ff9640 1 -- 172.21.15.37:0/1520595727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00b650 con 0x7f8fd4159b90 2024-04-25T09:03:31.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fda158640 1 -- 172.21.15.37:0/1520595727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd415a970 con 0x7f8fd4159b90 2024-04-25T09:03:31.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.273+0000 7f8fda158640 1 -- 172.21.15.37:0/1520595727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd404efa0 con 0x7f8fd4159b90 2024-04-25T09:03:31.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.277+0000 7f8fd0ff9640 1 -- 172.21.15.37:0/1520595727 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fbc00b7f0 con 0x7f8fd4159b90 2024-04-25T09:03:31.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.277+0000 7f8fd0ff9640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fb4066f60 0x7f8fb4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.277+0000 7f8fd0ff9640 1 -- 172.21.15.37:0/1520595727 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f8fbc084e60 con 0x7f8fd4159b90 2024-04-25T09:03:31.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.277+0000 7f8fd37fe640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fb4066f60 0x7f8fb4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.277+0000 7f8fd37fe640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fb4066f60 0x7f8fb4069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4000fc0 tx=0x7f8fc4001540 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.281+0000 7f8fd0ff9640 1 -- 172.21.15.37:0/1520595727 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fbc055e30 con 0x7f8fd4159b90 2024-04-25T09:03:31.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:31.445+0000 7f8fda158640 1 -- 172.21.15.37:0/1520595727 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8fd40b3e30 con 0x7f8fd4159b90 2024-04-25T09:03:32.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.109+0000 7f8fd0ff9640 1 -- 172.21.15.37:0/1520595727 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v123) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8fbc055730 con 0x7f8fd4159b90 2024-04-25T09:03:32.099 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 -- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fb4066f60 msgr2=0x7f8fb4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fb4066f60 0x7f8fb4069420 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f8fc4000fc0 tx=0x7f8fc4001540 comp rx=0 tx=0).stop 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 -- 172.21.15.37:0/1520595727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4159b90 msgr2=0x7f8fd40c45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4159b90 0x7f8fd40c45e0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00d7d0 tx=0x7f8fbc00dc90 comp rx=0 tx=0).stop 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 -- 172.21.15.37:0/1520595727 shutdown_connections 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fb4066f60 0x7f8fb4069420 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd4158cb0 0x7f8fd40c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4159b90 0x7f8fd40c45e0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 --2- 172.21.15.37:0/1520595727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fc0004d50 0x7f8fd4098920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 -- 172.21.15.37:0/1520595727 >> 172.21.15.37:0/1520595727 conn(0x7f8fd406ae60 msgr2=0x7f8fd415f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 -- 172.21.15.37:0/1520595727 shutdown_connections 2024-04-25T09:03:32.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.113+0000 7f8fda158640 1 -- 172.21.15.37:0/1520595727 wait complete. 2024-04-25T09:03:32.135 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:03:32.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f759779c640 1 Processor -- start 2024-04-25T09:03:32.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f759779c640 1 -- start start 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f759779c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f759015da80 0x7f75900d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f759779c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75900d1cd0 0x7f75900d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f759779c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f75900d0620 0x7f75900ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f759779c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759006c440 con 0x7f759015da80 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f759779c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759006be60 con 0x7f75900d0620 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f759779c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759015de60 con 0x7f75900d1cd0 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f7595538640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f759015da80 0x7f75900d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f7595538640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f759015da80 0x7f75900d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50922/0 (socket says 172.21.15.37:50922) 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f7595538640 1 -- 172.21.15.37:0/1442041103 learned_addr learned my addr 172.21.15.37:0/1442041103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f7595d39640 1 --2- 172.21.15.37:0/1442041103 >> v2:172.21.15.203:3300/0 conn(0x7f75900d0620 0x7f75900ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f7594d37640 1 --2- 172.21.15.37:0/1442041103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75900d1cd0 0x7f75900d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f7595538640 1 -- 172.21.15.37:0/1442041103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75900d1cd0 msgr2=0x7f75900d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f7595538640 1 --2- 172.21.15.37:0/1442041103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75900d1cd0 0x7f75900d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f7595538640 1 -- 172.21.15.37:0/1442041103 >> v2:172.21.15.203:3300/0 conn(0x7f75900d0620 msgr2=0x7f75900ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f7595538640 1 --2- 172.21.15.37:0/1442041103 >> v2:172.21.15.203:3300/0 conn(0x7f75900d0620 0x7f75900ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.281+0000 7f7595538640 1 -- 172.21.15.37:0/1442041103 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75900d2450 con 0x7f759015da80 2024-04-25T09:03:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f7595538640 1 --2- 172.21.15.37:0/1442041103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759015da80 0x7f75900d4230 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f759006de60 tx=0x7f758c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a21aaa4c82b197 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f7587fff640 1 -- 172.21.15.37:0/1442041103 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c0b0070 con 0x7f759015da80 2024-04-25T09:03:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f7587fff640 1 -- 172.21.15.37:0/1442041103 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758c0ab600 con 0x7f759015da80 2024-04-25T09:03:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f7587fff640 1 -- 172.21.15.37:0/1442041103 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c0ab9d0 con 0x7f759015da80 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f759779c640 1 -- 172.21.15.37:0/1442041103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759015da80 msgr2=0x7f75900d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f759779c640 1 --2- 172.21.15.37:0/1442041103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759015da80 0x7f75900d4230 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f759006de60 tx=0x7f758c01c520 comp rx=0 tx=0).stop 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f759779c640 1 -- 172.21.15.37:0/1442041103 shutdown_connections 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f759779c640 1 --2- 172.21.15.37:0/1442041103 >> v2:172.21.15.203:3300/0 conn(0x7f75900d0620 0x7f75900ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f759779c640 1 --2- 172.21.15.37:0/1442041103 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75900d1cd0 0x7f75900d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f759779c640 1 --2- 172.21.15.37:0/1442041103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759015da80 0x7f75900d4230 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f759779c640 1 -- 172.21.15.37:0/1442041103 >> 172.21.15.37:0/1442041103 conn(0x7f759008b000 msgr2=0x7f75900d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f759779c640 1 -- 172.21.15.37:0/1442041103 shutdown_connections 2024-04-25T09:03:32.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f759779c640 1 -- 172.21.15.37:0/1442041103 wait complete. 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f759779c640 1 Processor -- start 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.285+0000 7f759779c640 1 -- start start 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f759779c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75900d0620 0x7f75900c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f759779c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75900d1cd0 0x7f75901514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f759779c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759015da80 0x7f7590151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f759779c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759006ca50 con 0x7f759015da80 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f759779c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759006c8d0 con 0x7f75900d1cd0 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f759779c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759006cbd0 con 0x7f75900d0620 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7595d39640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759015da80 0x7f7590151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7595d39640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759015da80 0x7f7590151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50926/0 (socket says 172.21.15.37:50926) 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7595d39640 1 -- 172.21.15.37:0/1540098315 learned_addr learned my addr 172.21.15.37:0/1540098315 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7594d37640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75900d1cd0 0x7f75901514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7595538640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75900d0620 0x7f75900c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7595d39640 1 -- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75900d0620 msgr2=0x7f75900c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7595d39640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75900d0620 0x7f75900c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7595d39640 1 -- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75900d1cd0 msgr2=0x7f75901514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7595d39640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75900d1cd0 0x7f75901514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7595d39640 1 -- 172.21.15.37:0/1540098315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759015f6d0 con 0x7f759015da80 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7595d39640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759015da80 0x7f7590151a30 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f7588007e60 tx=0x7f758800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7585ffb640 1 -- 172.21.15.37:0/1540098315 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588014620 con 0x7f759015da80 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7585ffb640 1 -- 172.21.15.37:0/1540098315 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7588019070 con 0x7f759015da80 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f759779c640 1 -- 172.21.15.37:0/1540098315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759015f9b0 con 0x7f759015da80 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7585ffb640 1 -- 172.21.15.37:0/1540098315 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588014a20 con 0x7f759015da80 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f759779c640 1 -- 172.21.15.37:0/1540098315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759015ff70 con 0x7f759015da80 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f759779c640 1 -- 172.21.15.37:0/1540098315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7590160170 con 0x7f759015da80 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7585ffb640 1 -- 172.21.15.37:0/1540098315 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7588014bc0 con 0x7f759015da80 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7585ffb640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7574066f60 0x7f7574069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7585ffb640 1 -- 172.21.15.37:0/1540098315 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f75880867b0 con 0x7f759015da80 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.289+0000 7f7595538640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7574066f60 0x7f7574069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.293+0000 7f7595538640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7574066f60 0x7f7574069420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f758c0ad000 tx=0x7f758c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.293+0000 7f7585ffb640 1 -- 172.21.15.37:0/1540098315 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7588057770 con 0x7f759015da80 2024-04-25T09:03:32.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:32.489+0000 7f759779c640 1 -- 172.21.15.37:0/1540098315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f759004efa0 con 0x7f759015da80 2024-04-25T09:03:33.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.113+0000 7f7585ffb640 1 -- 172.21.15.37:0/1540098315 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v93) v1 ==== 174+0+0 (secure 0 0 0) 0x7f7588057070 con 0x7f759015da80 2024-04-25T09:03:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T09:03:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.113+0000 7f759779c640 1 -- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7574066f60 msgr2=0x7f7574069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.117+0000 7f759779c640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7574066f60 0x7f7574069420 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f758c0ad000 tx=0x7f758c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.117+0000 7f759779c640 1 -- 172.21.15.37:0/1540098315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759015da80 msgr2=0x7f7590151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.117+0000 7f759779c640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759015da80 0x7f7590151a30 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f7588007e60 tx=0x7f758800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.117+0000 7f759779c640 1 -- 172.21.15.37:0/1540098315 shutdown_connections 2024-04-25T09:03:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.117+0000 7f759779c640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7574066f60 0x7f7574069420 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.117+0000 7f759779c640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759015da80 0x7f7590151a30 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.117+0000 7f759779c640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f75900d1cd0 0x7f75901514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.117+0000 7f759779c640 1 --2- 172.21.15.37:0/1540098315 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f75900d0620 0x7f75900c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.117+0000 7f759779c640 1 -- 172.21.15.37:0/1540098315 >> 172.21.15.37:0/1540098315 conn(0x7f759008b000 msgr2=0x7f7590073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:33.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.117+0000 7f759779c640 1 -- 172.21.15.37:0/1540098315 shutdown_connections 2024-04-25T09:03:33.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.117+0000 7f759779c640 1 -- 172.21.15.37:0/1540098315 wait complete. 2024-04-25T09:03:33.134 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:33.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.265+0000 7f6c17441640 1 Processor -- start 2024-04-25T09:03:33.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.265+0000 7f6c17441640 1 -- start start 2024-04-25T09:03:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c17441640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c1015a280 0x7f6c1015a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c17441640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c100718c0 0x7f6c10071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c17441640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c1015ae30 con 0x7f6c10159ea0 2024-04-25T09:03:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c17441640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c100b2990 con 0x7f6c1015a280 2024-04-25T09:03:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c17441640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c100b3530 con 0x7f6c100718c0 2024-04-25T09:03:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c149dc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c1015a280 0x7f6c1015a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c149dc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6c1015a280 0x7f6c1015a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50936/0 (socket says 172.21.15.37:50936) 2024-04-25T09:03:33.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c149dc640 1 -- 172.21.15.37:0/3600272071 learned_addr learned my addr 172.21.15.37:0/3600272071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c159de640 1 --2- 172.21.15.37:0/3600272071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c100718c0 0x7f6c10071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c149dc640 1 -- 172.21.15.37:0/3600272071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c100718c0 msgr2=0x7f6c10071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c149dc640 1 --2- 172.21.15.37:0/3600272071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c100718c0 0x7f6c10071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c149dc640 1 -- 172.21.15.37:0/3600272071 >> v1:172.21.15.37:6789/0 conn(0x7f6c10159ea0 legacy=0x7f6c100b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c149dc640 1 -- 172.21.15.37:0/3600272071 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1015b110 con 0x7f6c1015a280 2024-04-25T09:03:33.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c07fff640 1 -- 172.21.15.37:0/3600272071 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712179807 0 0) 0x7f6c1015ae30 con 0x7f6c10159ea0 2024-04-25T09:03:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c149dc640 1 --2- 172.21.15.37:0/3600272071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c1015a280 0x7f6c1015a700 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009870 tx=0x7f6c0c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5e4b53a7ce337f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c07fff640 1 -- 172.21.15.37:0/3600272071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0ab5f0 con 0x7f6c1015a280 2024-04-25T09:03:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c07fff640 1 -- 172.21.15.37:0/3600272071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0c0b0070 con 0x7f6c1015a280 2024-04-25T09:03:33.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c07fff640 1 -- 172.21.15.37:0/3600272071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c0ab9b0 con 0x7f6c1015a280 2024-04-25T09:03:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c17441640 1 -- 172.21.15.37:0/3600272071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c1015a280 msgr2=0x7f6c1015a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.269+0000 7f6c17441640 1 --2- 172.21.15.37:0/3600272071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c1015a280 0x7f6c1015a700 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c009870 tx=0x7f6c0c01c560 comp rx=0 tx=0).stop 2024-04-25T09:03:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 -- 172.21.15.37:0/3600272071 shutdown_connections 2024-04-25T09:03:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 --2- 172.21.15.37:0/3600272071 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c100718c0 0x7f6c10071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 --2- 172.21.15.37:0/3600272071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c1015a280 0x7f6c1015a700 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 -- 172.21.15.37:0/3600272071 >> 172.21.15.37:0/3600272071 conn(0x7f6c100b11a0 msgr2=0x7f6c100b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 -- 172.21.15.37:0/3600272071 shutdown_connections 2024-04-25T09:03:33.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 -- 172.21.15.37:0/3600272071 wait complete. 2024-04-25T09:03:33.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 Processor -- start 2024-04-25T09:03:33.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 -- start start 2024-04-25T09:03:33.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c100718c0 0x7f6c1014b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c10159ea0 0x7f6c1014bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c1015a280 0x7f6c1013bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1006d420 con 0x7f6c100718c0 2024-04-25T09:03:33.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1006d2a0 con 0x7f6c10159ea0 2024-04-25T09:03:33.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1006d5a0 con 0x7f6c1015a280 2024-04-25T09:03:33.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c151dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c100718c0 0x7f6c1014b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c151dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c100718c0 0x7f6c1014b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50944/0 (socket says 172.21.15.37:50944) 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c151dd640 1 -- 172.21.15.37:0/4050317380 learned_addr learned my addr 172.21.15.37:0/4050317380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c149dc640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c10159ea0 0x7f6c1014bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c159de640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c1015a280 0x7f6c1013bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c151dd640 1 -- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c1015a280 msgr2=0x7f6c1013bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c151dd640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c1015a280 0x7f6c1013bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c151dd640 1 -- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c10159ea0 msgr2=0x7f6c1014bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c151dd640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c10159ea0 0x7f6c1014bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c151dd640 1 -- 172.21.15.37:0/4050317380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1013c310 con 0x7f6c100718c0 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c151dd640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c100718c0 0x7f6c1014b2e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000c4d0 tx=0x7f6c0000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c05ffb640 1 -- 172.21.15.37:0/4050317380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00017070 con 0x7f6c100718c0 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c05ffb640 1 -- 172.21.15.37:0/4050317380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c00005480 con 0x7f6c100718c0 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c05ffb640 1 -- 172.21.15.37:0/4050317380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00003840 con 0x7f6c100718c0 2024-04-25T09:03:33.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 -- 172.21.15.37:0/4050317380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c10159090 con 0x7f6c100718c0 2024-04-25T09:03:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.273+0000 7f6c17441640 1 -- 172.21.15.37:0/4050317380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c10151730 con 0x7f6c100718c0 2024-04-25T09:03:33.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.277+0000 7f6c17441640 1 -- 172.21.15.37:0/4050317380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c1004efa0 con 0x7f6c100718c0 2024-04-25T09:03:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.277+0000 7f6c05ffb640 1 -- 172.21.15.37:0/4050317380 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6c0001a070 con 0x7f6c100718c0 2024-04-25T09:03:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.277+0000 7f6c05ffb640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bf4066f60 0x7f6bf4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.277+0000 7f6c05ffb640 1 -- 172.21.15.37:0/4050317380 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6c00084510 con 0x7f6c100718c0 2024-04-25T09:03:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.277+0000 7f6c149dc640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bf4066f60 0x7f6bf4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.277+0000 7f6c149dc640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bf4066f60 0x7f6bf4069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0af000 tx=0x7f6c0c0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.281+0000 7f6c05ffb640 1 -- 172.21.15.37:0/4050317380 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6c000554e0 con 0x7f6c100718c0 2024-04-25T09:03:33.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.445+0000 7f6c17441640 1 -- 172.21.15.37:0/4050317380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c1006e6b0 con 0x7f6c100718c0 2024-04-25T09:03:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.445+0000 7f6c05ffb640 1 -- 172.21.15.37:0/4050317380 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16929 (secure 0 0 0) 0x7f6c00054de0 con 0x7f6c100718c0 2024-04-25T09:03:33.436 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:33.436 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":124,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:32.493406+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:33.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 -- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bf4066f60 msgr2=0x7f6bf4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bf4066f60 0x7f6bf4069420 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c0af000 tx=0x7f6c0c0a7000 comp rx=0 tx=0).stop 2024-04-25T09:03:33.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 -- 172.21.15.37:0/4050317380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c100718c0 msgr2=0x7f6c1014b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c100718c0 0x7f6c1014b2e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000c4d0 tx=0x7f6c0000c990 comp rx=0 tx=0).stop 2024-04-25T09:03:33.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 -- 172.21.15.37:0/4050317380 shutdown_connections 2024-04-25T09:03:33.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bf4066f60 0x7f6bf4069420 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6c1015a280 0x7f6c1013bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6c10159ea0 0x7f6c1014bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 --2- 172.21.15.37:0/4050317380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c100718c0 0x7f6c1014b2e0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 -- 172.21.15.37:0/4050317380 >> 172.21.15.37:0/4050317380 conn(0x7f6c100b11a0 msgr2=0x7f6c100b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 -- 172.21.15.37:0/4050317380 shutdown_connections 2024-04-25T09:03:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.449+0000 7f6c17441640 1 -- 172.21.15.37:0/4050317380 wait complete. 2024-04-25T09:03:33.456 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:33.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6d67d6640 1 Processor -- start 2024-04-25T09:03:33.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6d67d6640 1 -- start start 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6d67d6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6d015b950 0x7fe6d009a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6d67d6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6d00cfb80 0x7fe6d00cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6d67d6640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6d015bd30 con 0x7fe6d00cf420 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6d67d6640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d00d0510 con 0x7fe6d015b950 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6d67d6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00cc0d0 con 0x7fe6d00cfb80 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6cffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6d015b950 0x7fe6d009a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6cffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6d015b950 0x7fe6d009a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50950/0 (socket says 172.21.15.37:50950) 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6cffff640 1 -- 172.21.15.37:0/1929506935 learned_addr learned my addr 172.21.15.37:0/1929506935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6d4d73640 1 --2- 172.21.15.37:0/1929506935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6d00cfb80 0x7fe6d00cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6cffff640 1 -- 172.21.15.37:0/1929506935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6d00cfb80 msgr2=0x7fe6d00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6cffff640 1 --2- 172.21.15.37:0/1929506935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6d00cfb80 0x7fe6d00cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6cffff640 1 -- 172.21.15.37:0/1929506935 >> v1:172.21.15.203:6789/0 conn(0x7fe6d00cf420 legacy=0x7fe6d00cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:33.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.589+0000 7fe6cffff640 1 -- 172.21.15.37:0/1929506935 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d009ab10 con 0x7fe6d015b950 2024-04-25T09:03:33.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6cffff640 1 --2- 172.21.15.37:0/1929506935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d015b950 0x7fe6d009a480 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009e50 tx=0x7fe6c001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81eef0a0d470789 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6ceffd640 1 -- 172.21.15.37:0/1929506935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00a5760 con 0x7fe6d015b950 2024-04-25T09:03:33.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6ceffd640 1 -- 172.21.15.37:0/1929506935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c00a5900 con 0x7fe6d015b950 2024-04-25T09:03:33.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6ceffd640 1 -- 172.21.15.37:0/1929506935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00a5cd0 con 0x7fe6d015b950 2024-04-25T09:03:33.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 -- 172.21.15.37:0/1929506935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d015b950 msgr2=0x7fe6d009a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 --2- 172.21.15.37:0/1929506935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d015b950 0x7fe6d009a480 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009e50 tx=0x7fe6c001c9f0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 -- 172.21.15.37:0/1929506935 shutdown_connections 2024-04-25T09:03:33.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 --2- 172.21.15.37:0/1929506935 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6d00cfb80 0x7fe6d00cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 --2- 172.21.15.37:0/1929506935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d015b950 0x7fe6d009a480 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 -- 172.21.15.37:0/1929506935 >> 172.21.15.37:0/1929506935 conn(0x7fe6d00b30b0 msgr2=0x7fe6d00b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:33.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 -- 172.21.15.37:0/1929506935 shutdown_connections 2024-04-25T09:03:33.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 -- 172.21.15.37:0/1929506935 wait complete. 2024-04-25T09:03:33.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 Processor -- start 2024-04-25T09:03:33.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 -- start start 2024-04-25T09:03:33.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d00cf420 0x7fe6d01516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6d00cfb80 0x7fe6d0140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d015b950 0x7fe6d01578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d006faa0 con 0x7fe6d015b950 2024-04-25T09:03:33.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00cf8b0 con 0x7fe6d00cf420 2024-04-25T09:03:33.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d67d6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d006fc20 con 0x7fe6d00cfb80 2024-04-25T09:03:33.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d4d73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d015b950 0x7fe6d01578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d4d73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d015b950 0x7fe6d01578f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50956/0 (socket says 172.21.15.37:50956) 2024-04-25T09:03:33.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d4d73640 1 -- 172.21.15.37:0/4217971852 learned_addr learned my addr 172.21.15.37:0/4217971852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:33.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6cf7fe640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6d00cfb80 0x7fe6d0140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6cffff640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d00cf420 0x7fe6d01516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d4d73640 1 -- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6d00cfb80 msgr2=0x7fe6d0140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d4d73640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6d00cfb80 0x7fe6d0140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d4d73640 1 -- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d00cf420 msgr2=0x7fe6d01516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d4d73640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d00cf420 0x7fe6d01516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6d4d73640 1 -- 172.21.15.37:0/4217971852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d01410d0 con 0x7fe6d015b950 2024-04-25T09:03:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.593+0000 7fe6cffff640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d00cf420 0x7fe6d01516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6d4d73640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d015b950 0x7fe6d01578f0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800afd0 tx=0x7fe6c800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6ccff9640 1 -- 172.21.15.37:0/4217971852 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8015070 con 0x7fe6d015b950 2024-04-25T09:03:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6ccff9640 1 -- 172.21.15.37:0/4217971852 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c8004220 con 0x7fe6d015b950 2024-04-25T09:03:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6ccff9640 1 -- 172.21.15.37:0/4217971852 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8003620 con 0x7fe6d015b950 2024-04-25T09:03:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6d67d6640 1 -- 172.21.15.37:0/4217971852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d01413b0 con 0x7fe6d015b950 2024-04-25T09:03:33.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6d67d6640 1 -- 172.21.15.37:0/4217971852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d013ea80 con 0x7fe6d015b950 2024-04-25T09:03:33.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6d67d6640 1 -- 172.21.15.37:0/4217971852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d013ec80 con 0x7fe6d015b950 2024-04-25T09:03:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6ccff9640 1 -- 172.21.15.37:0/4217971852 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6c80037c0 con 0x7fe6d015b950 2024-04-25T09:03:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6ccff9640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6b0066f60 0x7fe6b0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6ccff9640 1 -- 172.21.15.37:0/4217971852 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe6c8084fd0 con 0x7fe6d015b950 2024-04-25T09:03:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6cffff640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6b0066f60 0x7fe6b0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.597+0000 7fe6cffff640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6b0066f60 0x7fe6b0069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00ae000 tx=0x7fe6c00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.601+0000 7fe6ccff9640 1 -- 172.21.15.37:0/4217971852 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6d013ec80 con 0x7fe6d015b950 2024-04-25T09:03:33.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.765+0000 7fe6d67d6640 1 -- 172.21.15.37:0/4217971852 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6d009a150 con 0x7fe6d015b950 2024-04-25T09:03:33.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.765+0000 7fe6ccff9640 1 -- 172.21.15.37:0/4217971852 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16929 (secure 0 0 0) 0x7fe6c8055e50 con 0x7fe6d015b950 2024-04-25T09:03:33.756 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:33.757 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":124,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:32.493406+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:33.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 -- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6b0066f60 msgr2=0x7fe6b0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6b0066f60 0x7fe6b0069420 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00ae000 tx=0x7fe6c00a7070 comp rx=0 tx=0).stop 2024-04-25T09:03:33.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 -- 172.21.15.37:0/4217971852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d015b950 msgr2=0x7fe6d01578f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d015b950 0x7fe6d01578f0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800afd0 tx=0x7fe6c800ca20 comp rx=0 tx=0).stop 2024-04-25T09:03:33.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 -- 172.21.15.37:0/4217971852 shutdown_connections 2024-04-25T09:03:33.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6b0066f60 0x7fe6b0069420 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d015b950 0x7fe6d01578f0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6d00cfb80 0x7fe6d0140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 --2- 172.21.15.37:0/4217971852 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d00cf420 0x7fe6d01516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 -- 172.21.15.37:0/4217971852 >> 172.21.15.37:0/4217971852 conn(0x7fe6d00b30b0 msgr2=0x7fe6d0070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:33.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 -- 172.21.15.37:0/4217971852 shutdown_connections 2024-04-25T09:03:33.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.769+0000 7fe6d67d6640 1 -- 172.21.15.37:0/4217971852 wait complete. 2024-04-25T09:03:33.778 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:03:33.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.909+0000 7f8ff1a10640 1 Processor -- start 2024-04-25T09:03:33.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8ff1a10640 1 -- start start 2024-04-25T09:03:33.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8ff1a10640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8fec0cf590 0x7f8fec0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8ff1a10640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec0cbd20 con 0x7f8fec15b6b0 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8ff1a10640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec0cc000 con 0x7f8fec072000 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8ff1a10640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec15ba90 con 0x7f8fec0cf590 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8feaffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8fec15b6b0 0x7f8fec0ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38034/0 (socket says 172.21.15.37:38034) 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8feaffd640 1 -- 172.21.15.37:0/714041237 learned_addr learned my addr 172.21.15.37:0/714041237 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fea7fc640 1 --2- 172.21.15.37:0/714041237 >> v2:172.21.15.203:3300/0 conn(0x7f8fec0cf590 0x7f8fec0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771132697 0 0) 0x7f8fec0cbd20 con 0x7f8fec15b6b0 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd4003530 con 0x7f8fec15b6b0 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2918736559 0 0) 0x7f8fd4003530 con 0x7f8fec15b6b0 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 >> v2:172.21.15.203:3300/0 conn(0x7f8fec0cf590 msgr2=0x7f8fec0cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 --2- 172.21.15.37:0/714041237 >> v2:172.21.15.203:3300/0 conn(0x7f8fec0cf590 0x7f8fec0cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 >> v1:172.21.15.203:6789/0 conn(0x7f8fec072000 legacy=0x7f8fec072430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec0cc2e0 con 0x7f8fec15b6b0 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f8fdc001680 con 0x7f8fec15b6b0 2024-04-25T09:03:33.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 >> v1:172.21.15.37:6789/0 conn(0x7f8fec15b6b0 legacy=0x7f8fec0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd4003a80 0x7f8fd4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4004810 0x7f8fd4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd4005220 0x7f8fd400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec15ba90 con 0x7f8fd4004810 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd400c150 con 0x7f8fd4003a80 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd400c350 con 0x7f8fd4005220 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fea7fc640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4004810 0x7f8fd4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8feaffd640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd4003a80 0x7f8fd4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8feb7fe640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd4005220 0x7f8fd400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fea7fc640 1 -- 172.21.15.37:0/714041237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd4005220 msgr2=0x7f8fd400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fea7fc640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd4005220 0x7f8fd400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fea7fc640 1 -- 172.21.15.37:0/714041237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd4003a80 msgr2=0x7f8fd4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fea7fc640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd4003a80 0x7f8fd4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8fea7fc640 1 -- 172.21.15.37:0/714041237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd4003530 con 0x7f8fd4004810 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8feaffd640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fd4003a80 0x7f8fd4003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:33.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.913+0000 7f8feb7fe640 1 -- 172.21.15.37:0/714041237 reap_dead start 2024-04-25T09:03:33.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8fea7fc640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4004810 0x7f8fd4004c90 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009ae0 tx=0x7f8fe001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d60a3f8321411b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00b2070 con 0x7f8fd4004810 2024-04-25T09:03:33.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe00a5550 con 0x7f8fd4004810 2024-04-25T09:03:33.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8fe9ffb640 1 -- 172.21.15.37:0/714041237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00096b0 con 0x7f8fd4004810 2024-04-25T09:03:33.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 -- 172.21.15.37:0/714041237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4004810 msgr2=0x7f8fd4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4004810 0x7f8fd4004c90 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009ae0 tx=0x7f8fe001ca70 comp rx=0 tx=0).stop 2024-04-25T09:03:33.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 -- 172.21.15.37:0/714041237 shutdown_connections 2024-04-25T09:03:33.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 --2- 172.21.15.37:0/714041237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fd4004810 0x7f8fd4004c90 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 -- 172.21.15.37:0/714041237 >> 172.21.15.37:0/714041237 conn(0x7f8fec06ae60 msgr2=0x7f8fec06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:33.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 -- 172.21.15.37:0/714041237 shutdown_connections 2024-04-25T09:03:33.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 -- 172.21.15.37:0/714041237 wait complete. 2024-04-25T09:03:33.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 Processor -- start 2024-04-25T09:03:33.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 -- start start 2024-04-25T09:03:33.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd4004810 0x7f8fec1f2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec14cbf0 0x7f8fec06d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec06da50 0x7f8fec0ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.917+0000 7f8ff1a10640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0cba30 con 0x7f8fec06da50 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8ff1a10640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0cc8a0 con 0x7f8fec14cbf0 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8ff1a10640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0cadb0 con 0x7f8fd4004810 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8feb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec06da50 0x7f8fec0ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8feb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec06da50 0x7f8fec0ac170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50972/0 (socket says 172.21.15.37:50972) 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8feb7fe640 1 -- 172.21.15.37:0/3076730826 learned_addr learned my addr 172.21.15.37:0/3076730826 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8fea7fc640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec14cbf0 0x7f8fec06d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8feaffd640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd4004810 0x7f8fec1f2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8feb7fe640 1 -- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd4004810 msgr2=0x7f8fec1f2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8feb7fe640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd4004810 0x7f8fec1f2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8feb7fe640 1 -- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec14cbf0 msgr2=0x7f8fec06d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8feb7fe640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec14cbf0 0x7f8fec06d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:33.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8feb7fe640 1 -- 172.21.15.37:0/3076730826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec159d00 con 0x7f8fec06da50 2024-04-25T09:03:33.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8feb7fe640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec06da50 0x7f8fec0ac170 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d770 tx=0x7f8fe400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8fd3fff640 1 -- 172.21.15.37:0/3076730826 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4018070 con 0x7f8fec06da50 2024-04-25T09:03:33.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8fd3fff640 1 -- 172.21.15.37:0/3076730826 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe4003b80 con 0x7f8fec06da50 2024-04-25T09:03:33.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8ff1a10640 1 -- 172.21.15.37:0/3076730826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec159fe0 con 0x7f8fec06da50 2024-04-25T09:03:33.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8fd3fff640 1 -- 172.21.15.37:0/3076730826 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe4005070 con 0x7f8fec06da50 2024-04-25T09:03:33.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8ff1a10640 1 -- 172.21.15.37:0/3076730826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec15a5a0 con 0x7f8fec06da50 2024-04-25T09:03:33.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8ff1a10640 1 -- 172.21.15.37:0/3076730826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec04efa0 con 0x7f8fec06da50 2024-04-25T09:03:33.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8fd3fff640 1 -- 172.21.15.37:0/3076730826 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fe401b070 con 0x7f8fec06da50 2024-04-25T09:03:33.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8fd3fff640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fd8066f60 0x7f8fd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:33.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8fd3fff640 1 -- 172.21.15.37:0/3076730826 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8fe4084870 con 0x7f8fec06da50 2024-04-25T09:03:33.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.921+0000 7f8feaffd640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fd8066f60 0x7f8fd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:33.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.925+0000 7f8feaffd640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fd8066f60 0x7f8fd8069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc001ec0 tx=0x7f8fdc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:33.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:33.925+0000 7f8fd3fff640 1 -- 172.21.15.37:0/3076730826 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fe4055720 con 0x7f8fec06da50 2024-04-25T09:03:34.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:34.121+0000 7f8ff1a10640 1 -- 172.21.15.37:0/3076730826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8fec0b1920 con 0x7f8fec06da50 2024-04-25T09:03:35.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.133+0000 7f8fd3fff640 1 -- 172.21.15.37:0/3076730826 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v96) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8fe4055020 con 0x7f8fec06da50 2024-04-25T09:03:35.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.133+0000 7f8ff1a10640 1 -- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fd8066f60 msgr2=0x7f8fd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.133+0000 7f8ff1a10640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fd8066f60 0x7f8fd8069420 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc001ec0 tx=0x7f8fdc009040 comp rx=0 tx=0).stop 2024-04-25T09:03:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.133+0000 7f8ff1a10640 1 -- 172.21.15.37:0/3076730826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec06da50 msgr2=0x7f8fec0ac170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.133+0000 7f8ff1a10640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec06da50 0x7f8fec0ac170 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d770 tx=0x7f8fe400dc30 comp rx=0 tx=0).stop 2024-04-25T09:03:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.137+0000 7f8ff1a10640 1 -- 172.21.15.37:0/3076730826 shutdown_connections 2024-04-25T09:03:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.137+0000 7f8ff1a10640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fd8066f60 0x7f8fd8069420 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.137+0000 7f8ff1a10640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec06da50 0x7f8fec0ac170 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.137+0000 7f8ff1a10640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fec14cbf0 0x7f8fec06d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.137+0000 7f8ff1a10640 1 --2- 172.21.15.37:0/3076730826 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fd4004810 0x7f8fec1f2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.137+0000 7f8ff1a10640 1 -- 172.21.15.37:0/3076730826 >> 172.21.15.37:0/3076730826 conn(0x7f8fec06ae60 msgr2=0x7f8fec0affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.137+0000 7f8ff1a10640 1 -- 172.21.15.37:0/3076730826 shutdown_connections 2024-04-25T09:03:35.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.137+0000 7f8ff1a10640 1 -- 172.21.15.37:0/3076730826 wait complete. 2024-04-25T09:03:35.147 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:03:35.148 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:03:35.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd042339640 1 Processor -- start 2024-04-25T09:03:35.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd042339640 1 -- start start 2024-04-25T09:03:35.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd042339640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd03c0d07f0 0x7fd03c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd042339640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03c06be60 con 0x7fd03c15da80 2024-04-25T09:03:35.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd042339640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03c15de60 con 0x7fd03c0d1cd0 2024-04-25T09:03:35.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd042339640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c06c440 con 0x7fd03c0d07f0 2024-04-25T09:03:35.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd03c15da80 0x7fd03c0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38048/0 (socket says 172.21.15.37:38048) 2024-04-25T09:03:35.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03bfff640 1 -- 172.21.15.37:0/1832638632 learned_addr learned my addr 172.21.15.37:0/1832638632 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:35.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03b7fe640 1 --2- 172.21.15.37:0/1832638632 >> v2:172.21.15.203:3300/0 conn(0x7fd03c0d07f0 0x7fd03c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127192168 0 0) 0x7fd03c06be60 con 0x7fd03c15da80 2024-04-25T09:03:35.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd024003530 con 0x7fd03c15da80 2024-04-25T09:03:35.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4286909439 0 0) 0x7fd024003530 con 0x7fd03c15da80 2024-04-25T09:03:35.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 >> v2:172.21.15.203:3300/0 conn(0x7fd03c0d07f0 msgr2=0x7fd03c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:35.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 --2- 172.21.15.37:0/1832638632 >> v2:172.21.15.203:3300/0 conn(0x7fd03c0d07f0 0x7fd03c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 >> v1:172.21.15.203:6789/0 conn(0x7fd03c0d1cd0 legacy=0x7fd03c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c0d2450 con 0x7fd03c15da80 2024-04-25T09:03:35.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fd02c001680 con 0x7fd03c15da80 2024-04-25T09:03:35.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 >> v1:172.21.15.37:6789/0 conn(0x7fd03c15da80 legacy=0x7fd03c0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 --2- 172.21.15.37:0/1832638632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 0x7fd024003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 --2- 172.21.15.37:0/1832638632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd024004810 0x7fd024004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 --2- 172.21.15.37:0/1832638632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd024005220 0x7fd02400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c06c440 con 0x7fd024003a80 2024-04-25T09:03:35.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02400c150 con 0x7fd024004810 2024-04-25T09:03:35.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02400c350 con 0x7fd024005220 2024-04-25T09:03:35.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03bfff640 1 --2- 172.21.15.37:0/1832638632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 0x7fd024003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03b7fe640 1 --2- 172.21.15.37:0/1832638632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd024004810 0x7fd024004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03bfff640 1 -- 172.21.15.37:0/1832638632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd024005220 msgr2=0x7fd02400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:35.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd0408d6640 1 --2- 172.21.15.37:0/1832638632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd024005220 0x7fd02400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03bfff640 1 --2- 172.21.15.37:0/1832638632 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd024005220 0x7fd02400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03bfff640 1 -- 172.21.15.37:0/1832638632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd024004810 msgr2=0x7fd024004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:35.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03bfff640 1 --2- 172.21.15.37:0/1832638632 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd024004810 0x7fd024004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd03bfff640 1 -- 172.21.15.37:0/1832638632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd024003530 con 0x7fd024003a80 2024-04-25T09:03:35.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.293+0000 7fd0408d6640 1 -- 172.21.15.37:0/1832638632 reap_dead start 2024-04-25T09:03:35.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd03bfff640 1 --2- 172.21.15.37:0/1832638632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 0x7fd024003ea0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00c190 tx=0x7fd02c01cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54ce66dea24ef567 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c007c70 con 0x7fd024003a80 2024-04-25T09:03:35.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c0a9070 con 0x7fd024003a80 2024-04-25T09:03:35.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd03affd640 1 -- 172.21.15.37:0/1832638632 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0a5e00 con 0x7fd024003a80 2024-04-25T09:03:35.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd042339640 1 -- 172.21.15.37:0/1832638632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 msgr2=0x7fd024003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd042339640 1 --2- 172.21.15.37:0/1832638632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 0x7fd024003ea0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00c190 tx=0x7fd02c01cbb0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd042339640 1 -- 172.21.15.37:0/1832638632 shutdown_connections 2024-04-25T09:03:35.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd042339640 1 --2- 172.21.15.37:0/1832638632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 0x7fd024003ea0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd042339640 1 -- 172.21.15.37:0/1832638632 >> 172.21.15.37:0/1832638632 conn(0x7fd03c08b000 msgr2=0x7fd03c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:35.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd042339640 1 -- 172.21.15.37:0/1832638632 shutdown_connections 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd042339640 1 -- 172.21.15.37:0/1832638632 wait complete. 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd042339640 1 Processor -- start 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.297+0000 7fd042339640 1 -- start start 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd042339640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 0x7fd03c0c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd042339640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd03c0b1360 0x7fd03c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd042339640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd03c0cb160 0x7fd03c15b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd042339640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0d2870 con 0x7fd024003a80 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd042339640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0d26f0 con 0x7fd03c0b1360 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd042339640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0d00e0 con 0x7fd03c0cb160 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 0x7fd03c0c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 0x7fd03c0c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50986/0 (socket says 172.21.15.37:50986) 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03bfff640 1 -- 172.21.15.37:0/1048432209 learned_addr learned my addr 172.21.15.37:0/1048432209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd0408d6640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd03c0cb160 0x7fd03c15b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03b7fe640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd03c0b1360 0x7fd03c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03bfff640 1 -- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd03c0cb160 msgr2=0x7fd03c15b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03bfff640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd03c0cb160 0x7fd03c15b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03bfff640 1 -- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd03c0b1360 msgr2=0x7fd03c15a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03bfff640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd03c0b1360 0x7fd03c15a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03bfff640 1 -- 172.21.15.37:0/1048432209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c0c74b0 con 0x7fd024003a80 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03b7fe640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd03c0b1360 0x7fd03c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd0408d6640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd03c0cb160 0x7fd03c15b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03bfff640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 0x7fd03c0c6f70 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00c190 tx=0x7fd024005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd038ff9640 1 -- 172.21.15.37:0/1048432209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0a5b50 con 0x7fd024003a80 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd038ff9640 1 -- 172.21.15.37:0/1048432209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c0a5cf0 con 0x7fd024003a80 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd042339640 1 -- 172.21.15.37:0/1048432209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c15ae70 con 0x7fd024003a80 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd038ff9640 1 -- 172.21.15.37:0/1048432209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0073c0 con 0x7fd024003a80 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd042339640 1 -- 172.21.15.37:0/1048432209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c0c7c00 con 0x7fd024003a80 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd042339640 1 -- 172.21.15.37:0/1048432209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03c0c7e40 con 0x7fd024003a80 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd038ff9640 1 -- 172.21.15.37:0/1048432209 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd02c007560 con 0x7fd024003a80 2024-04-25T09:03:35.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd038ff9640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd028066f60 0x7fd028069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:35.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd038ff9640 1 -- 172.21.15.37:0/1048432209 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd02c11ebf0 con 0x7fd024003a80 2024-04-25T09:03:35.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.301+0000 7fd03b7fe640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd028066f60 0x7fd028069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:35.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.305+0000 7fd03b7fe640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd028066f60 0x7fd028069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0b97a0 tx=0x7fd030008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:35.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.305+0000 7fd038ff9640 1 -- 172.21.15.37:0/1048432209 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd03c0c7e40 con 0x7fd024003a80 2024-04-25T09:03:35.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:35.509+0000 7fd042339640 1 -- 172.21.15.37:0/1048432209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd03c073730 con 0x7fd024003a80 2024-04-25T09:03:36.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.141+0000 7fd038ff9640 1 -- 172.21.15.37:0/1048432209 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v97) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd02c0168f0 con 0x7fd024003a80 2024-04-25T09:03:36.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 -- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd028066f60 msgr2=0x7fd028069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd028066f60 0x7fd028069420 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0b97a0 tx=0x7fd030008040 comp rx=0 tx=0).stop 2024-04-25T09:03:36.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 -- 172.21.15.37:0/1048432209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 msgr2=0x7fd03c0c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 0x7fd03c0c6f70 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fd02c00c190 tx=0x7fd024005840 comp rx=0 tx=0).stop 2024-04-25T09:03:36.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 -- 172.21.15.37:0/1048432209 shutdown_connections 2024-04-25T09:03:36.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd028066f60 0x7fd028069420 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd03c0cb160 0x7fd03c15b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd03c0b1360 0x7fd03c15a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 --2- 172.21.15.37:0/1048432209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd024003a80 0x7fd03c0c6f70 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 -- 172.21.15.37:0/1048432209 >> 172.21.15.37:0/1048432209 conn(0x7fd03c08b000 msgr2=0x7fd03c06c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 -- 172.21.15.37:0/1048432209 shutdown_connections 2024-04-25T09:03:36.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.149+0000 7fd042339640 1 -- 172.21.15.37:0/1048432209 wait complete. 2024-04-25T09:03:36.161 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:03:36.162 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:36.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.301+0000 7f3ae692b640 1 Processor -- start 2024-04-25T09:03:36.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.301+0000 7f3ae692b640 1 -- start start 2024-04-25T09:03:36.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3ae692b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f3ae00ce510 0x7f3ae00ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3ae692b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae00cee30 con 0x7f3ae01451f0 2024-04-25T09:03:36.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3ae692b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae00ccc70 con 0x7f3ae0144ca0 2024-04-25T09:03:36.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3ae692b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae00cc2b0 con 0x7f3ae00ce510 2024-04-25T09:03:36.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3ae4ec8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3ae01451f0 0x7f3ae0142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38050/0 (socket says 172.21.15.37:38050) 2024-04-25T09:03:36.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3ae4ec8640 1 -- 172.21.15.37:0/1782414725 learned_addr learned my addr 172.21.15.37:0/1782414725 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:36.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adffff640 1 --2- 172.21.15.37:0/1782414725 >> v2:172.21.15.203:3300/0 conn(0x7f3ae00ce510 0x7f3ae00ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164517185 0 0) 0x7f3ae00cee30 con 0x7f3ae01451f0 2024-04-25T09:03:36.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ac8003530 con 0x7f3ae01451f0 2024-04-25T09:03:36.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3452815610 0 0) 0x7f3ac8003530 con 0x7f3ae01451f0 2024-04-25T09:03:36.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 >> v2:172.21.15.203:3300/0 conn(0x7f3ae00ce510 msgr2=0x7f3ae00ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 --2- 172.21.15.37:0/1782414725 >> v2:172.21.15.203:3300/0 conn(0x7f3ae00ce510 0x7f3ae00ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 >> v1:172.21.15.203:6789/0 conn(0x7f3ae0144ca0 legacy=0x7f3ae006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae006dc40 con 0x7f3ae01451f0 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3ad8001a00 con 0x7f3ae01451f0 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 >> v1:172.21.15.37:6789/0 conn(0x7f3ae01451f0 legacy=0x7f3ae0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8003a30 0x7f3ac8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac8004810 0x7f3ac8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ac80051d0 0x7f3ac800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00cc2b0 con 0x7f3ac8003a30 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac800c0e0 con 0x7f3ac80051d0 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac800c2e0 con 0x7f3ac8004810 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adffff640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8003a30 0x7f3ac8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adf7fe640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac8004810 0x7f3ac8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3ae4ec8640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ac80051d0 0x7f3ac800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adffff640 1 -- 172.21.15.37:0/1782414725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac8004810 msgr2=0x7f3ac8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adffff640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac8004810 0x7f3ac8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adffff640 1 -- 172.21.15.37:0/1782414725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ac80051d0 msgr2=0x7f3ac800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adffff640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ac80051d0 0x7f3ac800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adffff640 1 -- 172.21.15.37:0/1782414725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac8003530 con 0x7f3ac8003a30 2024-04-25T09:03:36.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3ae4ec8640 1 -- 172.21.15.37:0/1782414725 reap_dead start 2024-04-25T09:03:36.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.305+0000 7f3adf7fe640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ac8004810 0x7f3ac8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3adffff640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8003a30 0x7f3ac8003e50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009ac0 tx=0x7f3ad001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8ede9b5b39b350 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00b2070 con 0x7f3ac8003a30 2024-04-25T09:03:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad00a5640 con 0x7f3ac8003a30 2024-04-25T09:03:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3adeffd640 1 -- 172.21.15.37:0/1782414725 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad0009690 con 0x7f3ac8003a30 2024-04-25T09:03:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 -- 172.21.15.37:0/1782414725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8003a30 msgr2=0x7f3ac8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8003a30 0x7f3ac8003e50 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009ac0 tx=0x7f3ad001ca50 comp rx=0 tx=0).stop 2024-04-25T09:03:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 -- 172.21.15.37:0/1782414725 shutdown_connections 2024-04-25T09:03:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 --2- 172.21.15.37:0/1782414725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ac8003a30 0x7f3ac8003e50 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 -- 172.21.15.37:0/1782414725 >> 172.21.15.37:0/1782414725 conn(0x7f3ae00b11a0 msgr2=0x7f3ae00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 -- 172.21.15.37:0/1782414725 shutdown_connections 2024-04-25T09:03:36.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 -- 172.21.15.37:0/1782414725 wait complete. 2024-04-25T09:03:36.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 Processor -- start 2024-04-25T09:03:36.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 -- start start 2024-04-25T09:03:36.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ac8003a30 0x7f3ae01f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ae00aa550 0x7f3ae013def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ae0071030 0x7f3ae00714c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00cc900 con 0x7f3ae0071030 2024-04-25T09:03:36.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae006e290 con 0x7f3ac8003a30 2024-04-25T09:03:36.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae692b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00ce8f0 con 0x7f3ae00aa550 2024-04-25T09:03:36.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae4ec8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ae0071030 0x7f3ae00714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae4ec8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ae0071030 0x7f3ae00714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51006/0 (socket says 172.21.15.37:51006) 2024-04-25T09:03:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae4ec8640 1 -- 172.21.15.37:0/648498365 learned_addr learned my addr 172.21.15.37:0/648498365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3adffff640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ac8003a30 0x7f3ae01f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3adf7fe640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ae00aa550 0x7f3ae013def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae4ec8640 1 -- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ae00aa550 msgr2=0x7f3ae013def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae4ec8640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ae00aa550 0x7f3ae013def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae4ec8640 1 -- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ac8003a30 msgr2=0x7f3ae01f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae4ec8640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ac8003a30 0x7f3ae01f1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3ae4ec8640 1 -- 172.21.15.37:0/648498365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae01f24e0 con 0x7f3ae0071030 2024-04-25T09:03:36.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3adffff640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ac8003a30 0x7f3ae01f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:36.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.309+0000 7f3adf7fe640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ae00aa550 0x7f3ae013def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:36.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3ae4ec8640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ae0071030 0x7f3ae00714c0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f3ae006cea0 tx=0x7f3ad8010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3adcff9640 1 -- 172.21.15.37:0/648498365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad801a070 con 0x7f3ae0071030 2024-04-25T09:03:36.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3adcff9640 1 -- 172.21.15.37:0/648498365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad8011040 con 0x7f3ae0071030 2024-04-25T09:03:36.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3ae692b640 1 -- 172.21.15.37:0/648498365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae00aaa10 con 0x7f3ae0071030 2024-04-25T09:03:36.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3adcff9640 1 -- 172.21.15.37:0/648498365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8012990 con 0x7f3ae0071030 2024-04-25T09:03:36.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3ae692b640 1 -- 172.21.15.37:0/648498365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae00aafd0 con 0x7f3ae0071030 2024-04-25T09:03:36.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3ae692b640 1 -- 172.21.15.37:0/648498365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae01f2790 con 0x7f3ae0071030 2024-04-25T09:03:36.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3adcff9640 1 -- 172.21.15.37:0/648498365 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ad801d070 con 0x7f3ae0071030 2024-04-25T09:03:36.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3adcff9640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3acc066f90 0x7f3acc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3adcff9640 1 -- 172.21.15.37:0/648498365 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ad8087620 con 0x7f3ae0071030 2024-04-25T09:03:36.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3adffff640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3acc066f90 0x7f3acc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.313+0000 7f3adffff640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3acc066f90 0x7f3acc069450 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00af000 tx=0x7f3ad0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.317+0000 7f3adcff9640 1 -- 172.21.15.37:0/648498365 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ae01f24e0 con 0x7f3ae0071030 2024-04-25T09:03:36.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.513+0000 7f3ae692b640 1 -- 172.21.15.37:0/648498365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ae004efa0 con 0x7f3ae0071030 2024-04-25T09:03:36.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.513+0000 7f3adcff9640 1 -- 172.21.15.37:0/648498365 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f3ad8058530 con 0x7f3ae0071030 2024-04-25T09:03:36.502 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:36.502 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":97,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:03:35.513718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9447":{"gid":9447,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/2748919477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2748919477},{"type":"v1","addr":"172.21.15.37:6835","nonce":2748919477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:03:36.502 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 97 2024-04-25T09:03:36.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.513+0000 7f3ae692b640 1 -- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3acc066f90 msgr2=0x7f3acc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.513+0000 7f3ae692b640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3acc066f90 0x7f3acc069450 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00af000 tx=0x7f3ad0008040 comp rx=0 tx=0).stop 2024-04-25T09:03:36.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.513+0000 7f3ae692b640 1 -- 172.21.15.37:0/648498365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ae0071030 msgr2=0x7f3ae00714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.513+0000 7f3ae692b640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ae0071030 0x7f3ae00714c0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f3ae006cea0 tx=0x7f3ad8010a00 comp rx=0 tx=0).stop 2024-04-25T09:03:36.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.513+0000 7f3ae692b640 1 -- 172.21.15.37:0/648498365 shutdown_connections 2024-04-25T09:03:36.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.513+0000 7f3ae692b640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3acc066f90 0x7f3acc069450 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.513+0000 7f3ae692b640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ae0071030 0x7f3ae00714c0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.513+0000 7f3ae692b640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ae00aa550 0x7f3ae013def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.513+0000 7f3ae692b640 1 --2- 172.21.15.37:0/648498365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ac8003a30 0x7f3ae01f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.517+0000 7f3ae692b640 1 -- 172.21.15.37:0/648498365 >> 172.21.15.37:0/648498365 conn(0x7f3ae00b11a0 msgr2=0x7f3ae00b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.517+0000 7f3ae692b640 1 -- 172.21.15.37:0/648498365 shutdown_connections 2024-04-25T09:03:36.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.517+0000 7f3ae692b640 1 -- 172.21.15.37:0/648498365 wait complete. 2024-04-25T09:03:36.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:36.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8f613640 1 Processor -- start 2024-04-25T09:03:36.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8f613640 1 -- start start 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8f613640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2c8815a750 0x7f2c8806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8f613640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2c88066bb0 0x7f2c8806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8f613640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8815ab30 con 0x7f2c8806c1e0 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8f613640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c88067100 con 0x7f2c8815a750 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8f613640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c880cf560 con 0x7f2c88066bb0 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8d3af640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2c8815a750 0x7f2c8806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8d3af640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2c8815a750 0x7f2c8806ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51008/0 (socket says 172.21.15.37:51008) 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8d3af640 1 -- 172.21.15.37:0/184555826 learned_addr learned my addr 172.21.15.37:0/184555826 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8cbae640 1 --2- 172.21.15.37:0/184555826 >> v2:172.21.15.203:3300/0 conn(0x7f2c88066bb0 0x7f2c8806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8d3af640 1 -- 172.21.15.37:0/184555826 >> v2:172.21.15.203:3300/0 conn(0x7f2c88066bb0 msgr2=0x7f2c8806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8d3af640 1 --2- 172.21.15.37:0/184555826 >> v2:172.21.15.203:3300/0 conn(0x7f2c88066bb0 0x7f2c8806bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8d3af640 1 -- 172.21.15.37:0/184555826 >> v1:172.21.15.203:6789/0 conn(0x7f2c8806c1e0 legacy=0x7f2c8815e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.649+0000 7f2c8d3af640 1 -- 172.21.15.37:0/184555826 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c880cf840 con 0x7f2c8815a750 2024-04-25T09:03:36.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8d3af640 1 --2- 172.21.15.37:0/184555826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c8815a750 0x7f2c8806ee60 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2c880b5ef0 tx=0x7f2c8401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c388c6448301b65c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c7ffff640 1 -- 172.21.15.37:0/184555826 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840ac020 con 0x7f2c8815a750 2024-04-25T09:03:36.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c7ffff640 1 -- 172.21.15.37:0/184555826 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c840a6480 con 0x7f2c8815a750 2024-04-25T09:03:36.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c7ffff640 1 -- 172.21.15.37:0/184555826 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c840a6850 con 0x7f2c8815a750 2024-04-25T09:03:36.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 -- 172.21.15.37:0/184555826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c8815a750 msgr2=0x7f2c8806ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 --2- 172.21.15.37:0/184555826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c8815a750 0x7f2c8806ee60 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f2c880b5ef0 tx=0x7f2c8401c890 comp rx=0 tx=0).stop 2024-04-25T09:03:36.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 -- 172.21.15.37:0/184555826 shutdown_connections 2024-04-25T09:03:36.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 --2- 172.21.15.37:0/184555826 >> v2:172.21.15.203:3300/0 conn(0x7f2c88066bb0 0x7f2c8806bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 --2- 172.21.15.37:0/184555826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c8815a750 0x7f2c8806ee60 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 -- 172.21.15.37:0/184555826 >> 172.21.15.37:0/184555826 conn(0x7f2c88071810 msgr2=0x7f2c88071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 -- 172.21.15.37:0/184555826 shutdown_connections 2024-04-25T09:03:36.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 -- 172.21.15.37:0/184555826 wait complete. 2024-04-25T09:03:36.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 Processor -- start 2024-04-25T09:03:36.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 -- start start 2024-04-25T09:03:36.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c88066bb0 0x7f2c880b08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c8806c1e0 0x7f2c880b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c8815a750 0x7f2c88167ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88165920 con 0x7f2c8806c1e0 2024-04-25T09:03:36.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c881657a0 con 0x7f2c8815a750 2024-04-25T09:03:36.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8f613640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c88165aa0 con 0x7f2c88066bb0 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.653+0000 7f2c8cbae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c8806c1e0 0x7f2c880b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8cbae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c8806c1e0 0x7f2c880b0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51014/0 (socket says 172.21.15.37:51014) 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8cbae640 1 -- 172.21.15.37:0/1791148602 learned_addr learned my addr 172.21.15.37:0/1791148602 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8d3af640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c88066bb0 0x7f2c880b08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8dbb0640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c8815a750 0x7f2c88167ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8cbae640 1 -- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c88066bb0 msgr2=0x7f2c880b08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8cbae640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c88066bb0 0x7f2c880b08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8cbae640 1 -- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c8815a750 msgr2=0x7f2c88167ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8cbae640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c8815a750 0x7f2c88167ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8cbae640 1 -- 172.21.15.37:0/1791148602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8814ed30 con 0x7f2c8806c1e0 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8cbae640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c8806c1e0 0x7f2c880b0e30 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800d7d0 tx=0x7f2c7800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c7dffb640 1 -- 172.21.15.37:0/1791148602 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78004020 con 0x7f2c8806c1e0 2024-04-25T09:03:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c7dffb640 1 -- 172.21.15.37:0/1791148602 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c780041c0 con 0x7f2c8806c1e0 2024-04-25T09:03:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8f613640 1 -- 172.21.15.37:0/1791148602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8814efd0 con 0x7f2c8806c1e0 2024-04-25T09:03:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c7dffb640 1 -- 172.21.15.37:0/1791148602 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c780043b0 con 0x7f2c8806c1e0 2024-04-25T09:03:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8f613640 1 -- 172.21.15.37:0/1791148602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8814f590 con 0x7f2c8806c1e0 2024-04-25T09:03:36.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8f613640 1 -- 172.21.15.37:0/1791148602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8815c640 con 0x7f2c8806c1e0 2024-04-25T09:03:36.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c7dffb640 1 -- 172.21.15.37:0/1791148602 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2c7801b020 con 0x7f2c8806c1e0 2024-04-25T09:03:36.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c7dffb640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c6c066f60 0x7f2c6c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c7dffb640 1 -- 172.21.15.37:0/1791148602 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2c78084fc0 con 0x7f2c8806c1e0 2024-04-25T09:03:36.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8d3af640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c6c066f60 0x7f2c6c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.657+0000 7f2c8d3af640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c6c066f60 0x7f2c6c069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2c840ad000 tx=0x7f2c840b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.661+0000 7f2c7dffb640 1 -- 172.21.15.37:0/1791148602 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2c8815c640 con 0x7f2c8806c1e0 2024-04-25T09:03:36.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.825+0000 7f2c8f613640 1 -- 172.21.15.37:0/1791148602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c880727c0 con 0x7f2c8806c1e0 2024-04-25T09:03:36.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.825+0000 7f2c7dffb640 1 -- 172.21.15.37:0/1791148602 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16916 (secure 0 0 0) 0x7f2c78055e30 con 0x7f2c8806c1e0 2024-04-25T09:03:36.816 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:36.817 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":125,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:33.770823+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:36.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 -- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c6c066f60 msgr2=0x7f2c6c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c6c066f60 0x7f2c6c069420 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f2c840ad000 tx=0x7f2c840b3040 comp rx=0 tx=0).stop 2024-04-25T09:03:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 -- 172.21.15.37:0/1791148602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c8806c1e0 msgr2=0x7f2c880b0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c8806c1e0 0x7f2c880b0e30 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f2c7800d7d0 tx=0x7f2c7800dc90 comp rx=0 tx=0).stop 2024-04-25T09:03:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 -- 172.21.15.37:0/1791148602 shutdown_connections 2024-04-25T09:03:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2c6c066f60 0x7f2c6c069420 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2c8815a750 0x7f2c88167ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2c8806c1e0 0x7f2c880b0e30 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 --2- 172.21.15.37:0/1791148602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2c88066bb0 0x7f2c880b08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 -- 172.21.15.37:0/1791148602 >> 172.21.15.37:0/1791148602 conn(0x7f2c88071810 msgr2=0x7f2c88073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 -- 172.21.15.37:0/1791148602 shutdown_connections 2024-04-25T09:03:36.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.829+0000 7f2c8f613640 1 -- 172.21.15.37:0/1791148602 wait complete. 2024-04-25T09:03:36.836 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:03:36.836 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:36.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.957+0000 7f5e3fb92640 1 Processor -- start 2024-04-25T09:03:36.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.957+0000 7f5e3fb92640 1 -- start start 2024-04-25T09:03:36.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3fb92640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5e380ce510 0x7f5e380ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3fb92640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e381451f0 0x7f5e38142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3fb92640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e380cee30 con 0x7f5e38144ca0 2024-04-25T09:03:36.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3fb92640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e380cc2b0 con 0x7f5e380ce510 2024-04-25T09:03:36.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3fb92640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3806e290 con 0x7f5e381451f0 2024-04-25T09:03:36.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3d92e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f5e380ce510 0x7f5e380ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3e12f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e381451f0 0x7f5e38142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3e12f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e381451f0 0x7f5e38142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36132/0 (socket says 172.21.15.37:36132) 2024-04-25T09:03:36.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3e12f640 1 -- 172.21.15.37:0/697441927 learned_addr learned my addr 172.21.15.37:0/697441927 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:36.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3d12d640 1 --1- 172.21.15.37:0/697441927 >> v1:172.21.15.203:6789/0 conn(0x7f5e38144ca0 0x7f5e3806e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:56458/0 (socket says 172.21.15.37:56458) 2024-04-25T09:03:36.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3d92e640 1 -- 172.21.15.37:0/697441927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e381451f0 msgr2=0x7f5e38142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:36.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3d92e640 1 --2- 172.21.15.37:0/697441927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e381451f0 0x7f5e38142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3d92e640 1 -- 172.21.15.37:0/697441927 >> v1:172.21.15.203:6789/0 conn(0x7f5e38144ca0 legacy=0x7f5e3806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.961+0000 7f5e3d92e640 1 -- 172.21.15.37:0/697441927 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e380ccc70 con 0x7f5e380ce510 2024-04-25T09:03:36.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3d92e640 1 --2- 172.21.15.37:0/697441927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e380ce510 0x7f5e380ce8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5e340097c0 tx=0x7f5e3401ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cda03e52705179 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3c92c640 1 -- 172.21.15.37:0/697441927 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340b5070 con 0x7f5e380ce510 2024-04-25T09:03:36.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3c92c640 1 -- 172.21.15.37:0/697441927 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e340a8990 con 0x7f5e380ce510 2024-04-25T09:03:36.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3c92c640 1 -- 172.21.15.37:0/697441927 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340a8d60 con 0x7f5e380ce510 2024-04-25T09:03:36.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 -- 172.21.15.37:0/697441927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e380ce510 msgr2=0x7f5e380ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 --2- 172.21.15.37:0/697441927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e380ce510 0x7f5e380ce8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5e340097c0 tx=0x7f5e3401ec90 comp rx=0 tx=0).stop 2024-04-25T09:03:36.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 -- 172.21.15.37:0/697441927 shutdown_connections 2024-04-25T09:03:36.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 --2- 172.21.15.37:0/697441927 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e381451f0 0x7f5e38142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 --2- 172.21.15.37:0/697441927 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e380ce510 0x7f5e380ce8f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 -- 172.21.15.37:0/697441927 >> 172.21.15.37:0/697441927 conn(0x7f5e380b11a0 msgr2=0x7f5e380b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:36.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 -- 172.21.15.37:0/697441927 shutdown_connections 2024-04-25T09:03:36.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 -- 172.21.15.37:0/697441927 wait complete. 2024-04-25T09:03:36.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 Processor -- start 2024-04-25T09:03:36.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 -- start start 2024-04-25T09:03:36.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e380ce510 0x7f5e380cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e38144ca0 0x7f5e3815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e381451f0 0x7f5e3815b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3806ee30 con 0x7f5e38144ca0 2024-04-25T09:03:36.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3806df20 con 0x7f5e381451f0 2024-04-25T09:03:36.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3fb92640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e380cc900 con 0x7f5e380ce510 2024-04-25T09:03:36.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3d12d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e38144ca0 0x7f5e3815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3d12d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e38144ca0 0x7f5e3815ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51028/0 (socket says 172.21.15.37:51028) 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3d12d640 1 -- 172.21.15.37:0/2953944119 learned_addr learned my addr 172.21.15.37:0/2953944119 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3d92e640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e380ce510 0x7f5e380cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3e12f640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e381451f0 0x7f5e3815b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3d12d640 1 -- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e380ce510 msgr2=0x7f5e380cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3d12d640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e380ce510 0x7f5e380cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3d12d640 1 -- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e381451f0 msgr2=0x7f5e3815b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3d12d640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e381451f0 0x7f5e3815b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3d12d640 1 -- 172.21.15.37:0/2953944119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e381e1980 con 0x7f5e38144ca0 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e3d12d640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e38144ca0 0x7f5e3815ace0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c430 tx=0x7f5e2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.965+0000 7f5e2e7fc640 1 -- 172.21.15.37:0/2953944119 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280074e0 con 0x7f5e38144ca0 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.969+0000 7f5e2e7fc640 1 -- 172.21.15.37:0/2953944119 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e28007680 con 0x7f5e38144ca0 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.969+0000 7f5e3fb92640 1 -- 172.21.15.37:0/2953944119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e381e1c60 con 0x7f5e38144ca0 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.969+0000 7f5e2e7fc640 1 -- 172.21.15.37:0/2953944119 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28004a30 con 0x7f5e38144ca0 2024-04-25T09:03:36.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.969+0000 7f5e3fb92640 1 -- 172.21.15.37:0/2953944119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e381e2220 con 0x7f5e38144ca0 2024-04-25T09:03:36.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.969+0000 7f5e3fb92640 1 -- 172.21.15.37:0/2953944119 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3804efa0 con 0x7f5e38144ca0 2024-04-25T09:03:36.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.969+0000 7f5e2e7fc640 1 -- 172.21.15.37:0/2953944119 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e2801a020 con 0x7f5e38144ca0 2024-04-25T09:03:36.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.969+0000 7f5e2e7fc640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e1c066f60 0x7f5e1c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:36.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.969+0000 7f5e2e7fc640 1 -- 172.21.15.37:0/2953944119 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e280849b0 con 0x7f5e38144ca0 2024-04-25T09:03:36.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.969+0000 7f5e3d92e640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e1c066f60 0x7f5e1c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:36.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.969+0000 7f5e3d92e640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e1c066f60 0x7f5e1c069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5e340b2000 tx=0x7f5e34008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:36.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:36.973+0000 7f5e2e7fc640 1 -- 172.21.15.37:0/2953944119 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5e28010300 con 0x7f5e38144ca0 2024-04-25T09:03:37.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.125+0000 7f5e3fb92640 1 -- 172.21.15.37:0/2953944119 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e380b1620 con 0x7f5e1c066f60 2024-04-25T09:03:37.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.141+0000 7f5e2e7fc640 1 -- 172.21.15.37:0/2953944119 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252416 (secure 0 0 0) 0x7f5e380b1620 con 0x7f5e1c066f60 2024-04-25T09:03:37.135 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:37.136 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 -- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e1c066f60 msgr2=0x7f5e1c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e1c066f60 0x7f5e1c069420 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5e340b2000 tx=0x7f5e34008040 comp rx=0 tx=0).stop 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 -- 172.21.15.37:0/2953944119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e38144ca0 msgr2=0x7f5e3815ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e38144ca0 0x7f5e3815ace0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800c430 tx=0x7f5e2800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 -- 172.21.15.37:0/2953944119 shutdown_connections 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e1c066f60 0x7f5e1c069420 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5e381451f0 0x7f5e3815b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e38144ca0 0x7f5e3815ace0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 --2- 172.21.15.37:0/2953944119 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5e380ce510 0x7f5e380cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 -- 172.21.15.37:0/2953944119 >> 172.21.15.37:0/2953944119 conn(0x7f5e380b11a0 msgr2=0x7f5e380b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 -- 172.21.15.37:0/2953944119 shutdown_connections 2024-04-25T09:03:37.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.145+0000 7f5e3fb92640 1 -- 172.21.15.37:0/2953944119 wait complete. 2024-04-25T09:03:37.152 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-04-25T09:03:35.356889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262680,"kb_used_data":13216,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712040,"statfs":{"total":773094113280,"available":772825128960,"internally_reserved":0,"allocated":13533184,"data_stored":6956236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.247850"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.808221+0000","last_change":"2024-04-25T09:03:32.803679+0000","last_active":"2024-04-25T09:03:32.808221+0000","last_peered":"2024-04-25T09:03:32.808221+0000","last_clean":"2024-04-25T09:03:32.808221+0000","last_became_active":"2024-04-25T09:03:32.802944+0000","last_became_peered":"2024-04-25T09:03:32.802944+0000","last_unstale":"2024-04-25T09:03:32.808221+0000","last_undegraded":"2024-04-25T09:03:32.808221+0000","last_fullsized":"2024-04-25T09:03:32.808221+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:24.835744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831973+0000","last_change":"2024-04-25T09:03:32.829719+0000","last_active":"2024-04-25T09:03:32.831973+0000","last_peered":"2024-04-25T09:03:32.831973+0000","last_clean":"2024-04-25T09:03:32.831973+0000","last_became_active":"2024-04-25T09:03:32.829183+0000","last_became_peered":"2024-04-25T09:03:32.829183+0000","last_unstale":"2024-04-25T09:03:32.831973+0000","last_undegraded":"2024-04-25T09:03:32.831973+0000","last_fullsized":"2024-04-25T09:03:32.831973+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:45.806170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.832011+0000","last_change":"2024-04-25T09:03:32.829788+0000","last_active":"2024-04-25T09:03:32.832011+0000","last_peered":"2024-04-25T09:03:32.832011+0000","last_clean":"2024-04-25T09:03:32.832011+0000","last_became_active":"2024-04-25T09:03:32.829143+0000","last_became_peered":"2024-04-25T09:03:32.829143+0000","last_unstale":"2024-04-25T09:03:32.832011+0000","last_undegraded":"2024-04-25T09:03:32.832011+0000","last_fullsized":"2024-04-25T09:03:32.832011+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:17.420562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.827573+0000","last_change":"2024-04-25T09:03:32.822942+0000","last_active":"2024-04-25T09:03:32.827573+0000","last_peered":"2024-04-25T09:03:32.827573+0000","last_clean":"2024-04-25T09:03:32.827573+0000","last_became_active":"2024-04-25T09:03:32.821904+0000","last_became_peered":"2024-04-25T09:03:32.821904+0000","last_unstale":"2024-04-25T09:03:32.827573+0000","last_undegraded":"2024-04-25T09:03:32.827573+0000","last_fullsized":"2024-04-25T09:03:32.827573+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:35.087962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.805059+0000","last_change":"2024-04-25T09:03:32.791004+0000","last_active":"2024-04-25T09:03:32.805059+0000","last_peered":"2024-04-25T09:03:32.805059+0000","last_clean":"2024-04-25T09:03:32.805059+0000","last_became_active":"2024-04-25T09:03:32.788710+0000","last_became_peered":"2024-04-25T09:03:32.788710+0000","last_unstale":"2024-04-25T09:03:32.805059+0000","last_undegraded":"2024-04-25T09:03:32.805059+0000","last_fullsized":"2024-04-25T09:03:32.805059+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:32.238618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.813756+0000","last_change":"2024-04-25T09:03:32.802590+0000","last_active":"2024-04-25T09:03:32.813756+0000","last_peered":"2024-04-25T09:03:32.813756+0000","last_clean":"2024-04-25T09:03:32.813756+0000","last_became_active":"2024-04-25T09:03:32.801840+0000","last_became_peered":"2024-04-25T09:03:32.801840+0000","last_unstale":"2024-04-25T09:03:32.813756+0000","last_undegraded":"2024-04-25T09:03:32.813756+0000","last_fullsized":"2024-04-25T09:03:32.813756+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:48.861040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.820844+0000","last_change":"2024-04-25T09:03:32.816495+0000","last_active":"2024-04-25T09:03:32.820844+0000","last_peered":"2024-04-25T09:03:32.820844+0000","last_clean":"2024-04-25T09:03:32.820844+0000","last_became_active":"2024-04-25T09:03:32.815726+0000","last_became_peered":"2024-04-25T09:03:32.815726+0000","last_unstale":"2024-04-25T09:03:32.820844+0000","last_undegraded":"2024-04-25T09:03:32.820844+0000","last_fullsized":"2024-04-25T09:03:32.820844+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:02.577084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831634+0000","last_change":"2024-04-25T09:03:32.825527+0000","last_active":"2024-04-25T09:03:32.831634+0000","last_peered":"2024-04-25T09:03:32.831634+0000","last_clean":"2024-04-25T09:03:32.831634+0000","last_became_active":"2024-04-25T09:03:32.824881+0000","last_became_peered":"2024-04-25T09:03:32.824881+0000","last_unstale":"2024-04-25T09:03:32.831634+0000","last_undegraded":"2024-04-25T09:03:32.831634+0000","last_fullsized":"2024-04-25T09:03:32.831634+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:47.542161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.836572+0000","last_change":"2024-04-25T09:03:32.834980+0000","last_active":"2024-04-25T09:03:32.836572+0000","last_peered":"2024-04-25T09:03:32.836572+0000","last_clean":"2024-04-25T09:03:32.836572+0000","last_became_active":"2024-04-25T09:03:32.833455+0000","last_became_peered":"2024-04-25T09:03:32.833455+0000","last_unstale":"2024-04-25T09:03:32.836572+0000","last_undegraded":"2024-04-25T09:03:32.836572+0000","last_fullsized":"2024-04-25T09:03:32.836572+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:53.171753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.812162+0000","last_change":"2024-04-25T09:03:32.806470+0000","last_active":"2024-04-25T09:03:32.812162+0000","last_peered":"2024-04-25T09:03:32.812162+0000","last_clean":"2024-04-25T09:03:32.812162+0000","last_became_active":"2024-04-25T09:03:32.805738+0000","last_became_peered":"2024-04-25T09:03:32.805738+0000","last_unstale":"2024-04-25T09:03:32.812162+0000","last_undegraded":"2024-04-25T09:03:32.812162+0000","last_fullsized":"2024-04-25T09:03:32.812162+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:32.238618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.810761+0000","last_change":"2024-04-25T09:03:32.806525+0000","last_active":"2024-04-25T09:03:32.810761+0000","last_peered":"2024-04-25T09:03:32.810761+0000","last_clean":"2024-04-25T09:03:32.810761+0000","last_became_active":"2024-04-25T09:03:32.805776+0000","last_became_peered":"2024-04-25T09:03:32.805776+0000","last_unstale":"2024-04-25T09:03:32.810761+0000","last_undegraded":"2024-04-25T09:03:32.810761+0000","last_fullsized":"2024-04-25T09:03:32.810761+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:35.562353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.818869+0000","last_change":"2024-04-25T09:03:32.813816+0000","last_active":"2024-04-25T09:03:32.818869+0000","last_peered":"2024-04-25T09:03:32.818869+0000","last_clean":"2024-04-25T09:03:32.818869+0000","last_became_active":"2024-04-25T09:03:32.808191+0000","last_became_peered":"2024-04-25T09:03:32.808191+0000","last_unstale":"2024-04-25T09:03:32.818869+0000","last_undegraded":"2024-04-25T09:03:32.818869+0000","last_fullsized":"2024-04-25T09:03:32.818869+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:26.449846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.818678+0000","last_change":"2024-04-25T09:03:32.813433+0000","last_active":"2024-04-25T09:03:32.818678+0000","last_peered":"2024-04-25T09:03:32.818678+0000","last_clean":"2024-04-25T09:03:32.818678+0000","last_became_active":"2024-04-25T09:03:32.812403+0000","last_became_peered":"2024-04-25T09:03:32.812403+0000","last_unstale":"2024-04-25T09:03:32.818678+0000","last_undegraded":"2024-04-25T09:03:32.818678+0000","last_fullsized":"2024-04-25T09:03:32.818678+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.420083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831894+0000","last_change":"2024-04-25T09:03:32.826779+0000","last_active":"2024-04-25T09:03:32.831894+0000","last_peered":"2024-04-25T09:03:32.831894+0000","last_clean":"2024-04-25T09:03:32.831894+0000","last_became_active":"2024-04-25T09:03:32.825162+0000","last_became_peered":"2024-04-25T09:03:32.825162+0000","last_unstale":"2024-04-25T09:03:32.831894+0000","last_undegraded":"2024-04-25T09:03:32.831894+0000","last_fullsized":"2024-04-25T09:03:32.831894+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:44.265870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.826409+0000","last_change":"2024-04-25T09:03:32.817108+0000","last_active":"2024-04-25T09:03:32.826409+0000","last_peered":"2024-04-25T09:03:32.826409+0000","last_clean":"2024-04-25T09:03:32.826409+0000","last_became_active":"2024-04-25T09:03:32.816312+0000","last_became_peered":"2024-04-25T09:03:32.816312+0000","last_unstale":"2024-04-25T09:03:32.826409+0000","last_undegraded":"2024-04-25T09:03:32.826409+0000","last_fullsized":"2024-04-25T09:03:32.826409+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:35.087962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.816684+0000","last_change":"2024-04-25T09:03:32.807820+0000","last_active":"2024-04-25T09:03:32.816684+0000","last_peered":"2024-04-25T09:03:32.816684+0000","last_clean":"2024-04-25T09:03:32.816684+0000","last_became_active":"2024-04-25T09:03:32.807045+0000","last_became_peered":"2024-04-25T09:03:32.807045+0000","last_unstale":"2024-04-25T09:03:32.816684+0000","last_undegraded":"2024-04-25T09:03:32.816684+0000","last_fullsized":"2024-04-25T09:03:32.816684+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:38.673985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.835810+0000","last_change":"2024-04-25T09:03:32.834896+0000","last_active":"2024-04-25T09:03:32.835810+0000","last_peered":"2024-04-25T09:03:32.835810+0000","last_clean":"2024-04-25T09:03:32.835810+0000","last_became_active":"2024-04-25T09:03:32.833709+0000","last_became_peered":"2024-04-25T09:03:32.833709+0000","last_unstale":"2024-04-25T09:03:32.835810+0000","last_undegraded":"2024-04-25T09:03:32.835810+0000","last_fullsized":"2024-04-25T09:03:32.835810+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:24.835744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831774+0000","last_change":"2024-04-25T09:03:32.826063+0000","last_active":"2024-04-25T09:03:32.831774+0000","last_peered":"2024-04-25T09:03:32.831774+0000","last_clean":"2024-04-25T09:03:32.831774+0000","last_became_active":"2024-04-25T09:03:32.825276+0000","last_became_peered":"2024-04-25T09:03:32.825276+0000","last_unstale":"2024-04-25T09:03:32.831774+0000","last_undegraded":"2024-04-25T09:03:32.831774+0000","last_fullsized":"2024-04-25T09:03:32.831774+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:18.118297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.827563+0000","last_change":"2024-04-25T09:03:32.825143+0000","last_active":"2024-04-25T09:03:32.827563+0000","last_peered":"2024-04-25T09:03:32.827563+0000","last_clean":"2024-04-25T09:03:32.827563+0000","last_became_active":"2024-04-25T09:03:32.824349+0000","last_became_peered":"2024-04-25T09:03:32.824349+0000","last_unstale":"2024-04-25T09:03:32.827563+0000","last_undegraded":"2024-04-25T09:03:32.827563+0000","last_fullsized":"2024-04-25T09:03:32.827563+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:35.562353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.815948+0000","last_change":"2024-04-25T09:03:32.811954+0000","last_active":"2024-04-25T09:03:32.815948+0000","last_peered":"2024-04-25T09:03:32.815948+0000","last_clean":"2024-04-25T09:03:32.815948+0000","last_became_active":"2024-04-25T09:03:32.810436+0000","last_became_peered":"2024-04-25T09:03:32.810436+0000","last_unstale":"2024-04-25T09:03:32.815948+0000","last_undegraded":"2024-04-25T09:03:32.815948+0000","last_fullsized":"2024-04-25T09:03:32.815948+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.764315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.825222+0000","last_change":"2024-04-25T09:03:32.821432+0000","last_active":"2024-04-25T09:03:32.825222+0000","last_peered":"2024-04-25T09:03:32.825222+0000","last_clean":"2024-04-25T09:03:32.825222+0000","last_became_active":"2024-04-25T09:03:32.818141+0000","last_became_peered":"2024-04-25T09:03:32.818141+0000","last_unstale":"2024-04-25T09:03:32.825222+0000","last_undegraded":"2024-04-25T09:03:32.825222+0000","last_fullsized":"2024-04-25T09:03:32.825222+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:35.562353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.839203+0000","last_change":"2024-04-25T09:03:32.836447+0000","last_active":"2024-04-25T09:03:32.839203+0000","last_peered":"2024-04-25T09:03:32.839203+0000","last_clean":"2024-04-25T09:03:32.839203+0000","last_became_active":"2024-04-25T09:03:32.835792+0000","last_became_peered":"2024-04-25T09:03:32.835792+0000","last_unstale":"2024-04-25T09:03:32.839203+0000","last_undegraded":"2024-04-25T09:03:32.839203+0000","last_fullsized":"2024-04-25T09:03:32.839203+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:53.171753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.824901+0000","last_change":"2024-04-25T09:03:32.818744+0000","last_active":"2024-04-25T09:03:32.824901+0000","last_peered":"2024-04-25T09:03:32.824901+0000","last_clean":"2024-04-25T09:03:32.824901+0000","last_became_active":"2024-04-25T09:03:32.817822+0000","last_became_peered":"2024-04-25T09:03:32.817822+0000","last_unstale":"2024-04-25T09:03:32.824901+0000","last_undegraded":"2024-04-25T09:03:32.824901+0000","last_fullsized":"2024-04-25T09:03:32.824901+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:32.238618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.829964+0000","last_change":"2024-04-25T09:03:32.819525+0000","last_active":"2024-04-25T09:03:32.829964+0000","last_peered":"2024-04-25T09:03:32.829964+0000","last_clean":"2024-04-25T09:03:32.829964+0000","last_became_active":"2024-04-25T09:03:32.818801+0000","last_became_peered":"2024-04-25T09:03:32.818801+0000","last_unstale":"2024-04-25T09:03:32.829964+0000","last_undegraded":"2024-04-25T09:03:32.829964+0000","last_fullsized":"2024-04-25T09:03:32.829964+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:05.258773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.829971+0000","last_change":"2024-04-25T09:03:32.815934+0000","last_active":"2024-04-25T09:03:32.829971+0000","last_peered":"2024-04-25T09:03:32.829971+0000","last_clean":"2024-04-25T09:03:32.829971+0000","last_became_active":"2024-04-25T09:03:32.815119+0000","last_became_peered":"2024-04-25T09:03:32.815119+0000","last_unstale":"2024-04-25T09:03:32.829971+0000","last_undegraded":"2024-04-25T09:03:32.829971+0000","last_fullsized":"2024-04-25T09:03:32.829971+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:52.306518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.805453+0000","last_change":"2024-04-25T09:03:32.799937+0000","last_active":"2024-04-25T09:03:32.805453+0000","last_peered":"2024-04-25T09:03:32.805453+0000","last_clean":"2024-04-25T09:03:32.805453+0000","last_became_active":"2024-04-25T09:03:32.799164+0000","last_became_peered":"2024-04-25T09:03:32.799164+0000","last_unstale":"2024-04-25T09:03:32.805453+0000","last_undegraded":"2024-04-25T09:03:32.805453+0000","last_fullsized":"2024-04-25T09:03:32.805453+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:53.561488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.805380+0000","last_change":"2024-04-25T09:03:32.801476+0000","last_active":"2024-04-25T09:03:32.805380+0000","last_peered":"2024-04-25T09:03:32.805380+0000","last_clean":"2024-04-25T09:03:32.805380+0000","last_became_active":"2024-04-25T09:03:32.799001+0000","last_became_peered":"2024-04-25T09:03:32.799001+0000","last_unstale":"2024-04-25T09:03:32.805380+0000","last_undegraded":"2024-04-25T09:03:32.805380+0000","last_fullsized":"2024-04-25T09:03:32.805380+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.420083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.820747+0000","last_change":"2024-04-25T09:03:32.816795+0000","last_active":"2024-04-25T09:03:32.820747+0000","last_peered":"2024-04-25T09:03:32.820747+0000","last_clean":"2024-04-25T09:03:32.820747+0000","last_became_active":"2024-04-25T09:03:32.816106+0000","last_became_peered":"2024-04-25T09:03:32.816106+0000","last_unstale":"2024-04-25T09:03:32.820747+0000","last_undegraded":"2024-04-25T09:03:32.820747+0000","last_fullsized":"2024-04-25T09:03:32.820747+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:59.022260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.810571+0000","last_change":"2024-04-25T09:03:32.805491+0000","last_active":"2024-04-25T09:03:32.810571+0000","last_peered":"2024-04-25T09:03:32.810571+0000","last_clean":"2024-04-25T09:03:32.810571+0000","last_became_active":"2024-04-25T09:03:32.803874+0000","last_became_peered":"2024-04-25T09:03:32.803874+0000","last_unstale":"2024-04-25T09:03:32.810571+0000","last_undegraded":"2024-04-25T09:03:32.810571+0000","last_fullsized":"2024-04-25T09:03:32.810571+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:43.761302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831945+0000","last_change":"2024-04-25T09:03:32.827018+0000","last_active":"2024-04-25T09:03:32.831945+0000","last_peered":"2024-04-25T09:03:32.831945+0000","last_clean":"2024-04-25T09:03:32.831945+0000","last_became_active":"2024-04-25T09:03:32.825977+0000","last_became_peered":"2024-04-25T09:03:32.825977+0000","last_unstale":"2024-04-25T09:03:32.831945+0000","last_undegraded":"2024-04-25T09:03:32.831945+0000","last_fullsized":"2024-04-25T09:03:32.831945+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:31.684282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.836489+0000","last_change":"2024-04-25T09:03:32.828507+0000","last_active":"2024-04-25T09:03:32.836489+0000","last_peered":"2024-04-25T09:03:32.836489+0000","last_clean":"2024-04-25T09:03:32.836489+0000","last_became_active":"2024-04-25T09:03:32.822088+0000","last_became_peered":"2024-04-25T09:03:32.822088+0000","last_unstale":"2024-04-25T09:03:32.836489+0000","last_undegraded":"2024-04-25T09:03:32.836489+0000","last_fullsized":"2024-04-25T09:03:32.836489+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:28.186515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.836620+0000","last_change":"2024-04-25T09:03:32.819181+0000","last_active":"2024-04-25T09:03:32.836620+0000","last_peered":"2024-04-25T09:03:32.836620+0000","last_clean":"2024-04-25T09:03:32.836620+0000","last_became_active":"2024-04-25T09:03:32.818211+0000","last_became_peered":"2024-04-25T09:03:32.818211+0000","last_unstale":"2024-04-25T09:03:32.836620+0000","last_undegraded":"2024-04-25T09:03:32.836620+0000","last_fullsized":"2024-04-25T09:03:32.836620+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:26.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.812033+0000","last_change":"2024-04-25T09:03:32.805909+0000","last_active":"2024-04-25T09:03:32.812033+0000","last_peered":"2024-04-25T09:03:32.812033+0000","last_clean":"2024-04-25T09:03:32.812033+0000","last_became_active":"2024-04-25T09:03:32.805113+0000","last_became_peered":"2024-04-25T09:03:32.805113+0000","last_unstale":"2024-04-25T09:03:32.812033+0000","last_undegraded":"2024-04-25T09:03:32.812033+0000","last_fullsized":"2024-04-25T09:03:32.812033+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:08.376454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"124'4","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.141561+0000","last_change":"2024-04-25T09:03:32.141549+0000","last_active":"2024-04-25T09:03:33.141561+0000","last_peered":"2024-04-25T09:03:33.141561+0000","last_clean":"2024-04-25T09:03:33.141561+0000","last_became_active":"2024-04-25T09:03:32.140817+0000","last_became_peered":"2024-04-25T09:03:32.140817+0000","last_unstale":"2024-04-25T09:03:33.141561+0000","last_undegraded":"2024-04-25T09:03:33.141561+0000","last_fullsized":"2024-04-25T09:03:33.141561+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:27.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.827350+0000","last_change":"2024-04-25T09:03:32.822879+0000","last_active":"2024-04-25T09:03:32.827350+0000","last_peered":"2024-04-25T09:03:32.827350+0000","last_clean":"2024-04-25T09:03:32.827350+0000","last_became_active":"2024-04-25T09:03:32.822263+0000","last_became_peered":"2024-04-25T09:03:32.822263+0000","last_unstale":"2024-04-25T09:03:32.827350+0000","last_undegraded":"2024-04-25T09:03:32.827350+0000","last_fullsized":"2024-04-25T09:03:32.827350+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:46.508470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.134225+0000","last_change":"2024-04-25T09:03:32.137716+0000","last_active":"2024-04-25T09:03:33.134225+0000","last_peered":"2024-04-25T09:03:33.134225+0000","last_clean":"2024-04-25T09:03:33.134225+0000","last_became_active":"2024-04-25T09:03:32.137271+0000","last_became_peered":"2024-04-25T09:03:32.137271+0000","last_unstale":"2024-04-25T09:03:33.134225+0000","last_undegraded":"2024-04-25T09:03:33.134225+0000","last_fullsized":"2024-04-25T09:03:33.134225+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:51.302679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.836290+0000","last_change":"2024-04-25T09:03:32.821025+0000","last_active":"2024-04-25T09:03:32.836290+0000","last_peered":"2024-04-25T09:03:32.836290+0000","last_clean":"2024-04-25T09:03:32.836290+0000","last_became_active":"2024-04-25T09:03:32.820293+0000","last_became_peered":"2024-04-25T09:03:32.820293+0000","last_unstale":"2024-04-25T09:03:32.836290+0000","last_undegraded":"2024-04-25T09:03:32.836290+0000","last_fullsized":"2024-04-25T09:03:32.836290+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:16.529254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.143866+0000","last_change":"2024-04-25T09:03:32.149094+0000","last_active":"2024-04-25T09:03:33.143866+0000","last_peered":"2024-04-25T09:03:33.143866+0000","last_clean":"2024-04-25T09:03:33.143866+0000","last_became_active":"2024-04-25T09:03:32.148337+0000","last_became_peered":"2024-04-25T09:03:32.148337+0000","last_unstale":"2024-04-25T09:03:33.143866+0000","last_undegraded":"2024-04-25T09:03:33.143866+0000","last_fullsized":"2024-04-25T09:03:33.143866+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:27.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.825001+0000","last_change":"2024-04-25T09:03:32.820614+0000","last_active":"2024-04-25T09:03:32.825001+0000","last_peered":"2024-04-25T09:03:32.825001+0000","last_clean":"2024-04-25T09:03:32.825001+0000","last_became_active":"2024-04-25T09:03:32.819217+0000","last_became_peered":"2024-04-25T09:03:32.819217+0000","last_unstale":"2024-04-25T09:03:32.825001+0000","last_undegraded":"2024-04-25T09:03:32.825001+0000","last_fullsized":"2024-04-25T09:03:32.825001+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.420083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.130411+0000","last_change":"2024-04-25T09:03:32.151168+0000","last_active":"2024-04-25T09:03:33.130411+0000","last_peered":"2024-04-25T09:03:33.130411+0000","last_clean":"2024-04-25T09:03:33.130411+0000","last_became_active":"2024-04-25T09:03:32.150755+0000","last_became_peered":"2024-04-25T09:03:32.150755+0000","last_unstale":"2024-04-25T09:03:33.130411+0000","last_undegraded":"2024-04-25T09:03:33.130411+0000","last_fullsized":"2024-04-25T09:03:33.130411+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:44.666207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.796701+0000","last_change":"2024-04-25T09:03:32.787090+0000","last_active":"2024-04-25T09:03:32.796701+0000","last_peered":"2024-04-25T09:03:32.796701+0000","last_clean":"2024-04-25T09:03:32.796701+0000","last_became_active":"2024-04-25T09:03:32.786254+0000","last_became_peered":"2024-04-25T09:03:32.786254+0000","last_unstale":"2024-04-25T09:03:32.796701+0000","last_undegraded":"2024-04-25T09:03:32.796701+0000","last_fullsized":"2024-04-25T09:03:32.796701+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:16.529254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.138923+0000","last_change":"2024-04-25T09:03:32.154970+0000","last_active":"2024-04-25T09:03:33.138923+0000","last_peered":"2024-04-25T09:03:33.138923+0000","last_clean":"2024-04-25T09:03:33.138923+0000","last_became_active":"2024-04-25T09:03:32.154321+0000","last_became_peered":"2024-04-25T09:03:32.154321+0000","last_unstale":"2024-04-25T09:03:33.138923+0000","last_undegraded":"2024-04-25T09:03:33.138923+0000","last_fullsized":"2024-04-25T09:03:33.138923+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:27.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.805280+0000","last_change":"2024-04-25T09:03:32.801022+0000","last_active":"2024-04-25T09:03:32.805280+0000","last_peered":"2024-04-25T09:03:32.805280+0000","last_clean":"2024-04-25T09:03:32.805280+0000","last_became_active":"2024-04-25T09:03:32.800313+0000","last_became_peered":"2024-04-25T09:03:32.800313+0000","last_unstale":"2024-04-25T09:03:32.805280+0000","last_undegraded":"2024-04-25T09:03:32.805280+0000","last_fullsized":"2024-04-25T09:03:32.805280+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.420083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.136819+0000","last_change":"2024-04-25T09:03:32.158494+0000","last_active":"2024-04-25T09:03:33.136819+0000","last_peered":"2024-04-25T09:03:33.136819+0000","last_clean":"2024-04-25T09:03:33.136819+0000","last_became_active":"2024-04-25T09:03:32.158211+0000","last_became_peered":"2024-04-25T09:03:32.158211+0000","last_unstale":"2024-04-25T09:03:33.136819+0000","last_undegraded":"2024-04-25T09:03:33.136819+0000","last_fullsized":"2024-04-25T09:03:33.136819+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:28.212478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831574+0000","last_change":"2024-04-25T09:03:32.825082+0000","last_active":"2024-04-25T09:03:32.831574+0000","last_peered":"2024-04-25T09:03:32.831574+0000","last_clean":"2024-04-25T09:03:32.831574+0000","last_became_active":"2024-04-25T09:03:32.821175+0000","last_became_peered":"2024-04-25T09:03:32.821175+0000","last_unstale":"2024-04-25T09:03:32.831574+0000","last_undegraded":"2024-04-25T09:03:32.831574+0000","last_fullsized":"2024-04-25T09:03:32.831574+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:36.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.801841+0000","last_change":"2024-04-25T09:03:32.149238+0000","last_active":"2024-04-25T09:03:32.801841+0000","last_peered":"2024-04-25T09:03:32.801841+0000","last_clean":"2024-04-25T09:03:32.801841+0000","last_became_active":"2024-04-25T09:03:32.148593+0000","last_became_peered":"2024-04-25T09:03:32.148593+0000","last_unstale":"2024-04-25T09:03:32.801841+0000","last_undegraded":"2024-04-25T09:03:32.801841+0000","last_fullsized":"2024-04-25T09:03:32.801841+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:09.161775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.808148+0000","last_change":"2024-04-25T09:03:32.803956+0000","last_active":"2024-04-25T09:03:32.808148+0000","last_peered":"2024-04-25T09:03:32.808148+0000","last_clean":"2024-04-25T09:03:32.808148+0000","last_became_active":"2024-04-25T09:03:32.798484+0000","last_became_peered":"2024-04-25T09:03:32.798484+0000","last_unstale":"2024-04-25T09:03:32.808148+0000","last_undegraded":"2024-04-25T09:03:32.808148+0000","last_fullsized":"2024-04-25T09:03:32.808148+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:09.004049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.790938+0000","last_change":"2024-04-25T09:03:32.144190+0000","last_active":"2024-04-25T09:03:32.790938+0000","last_peered":"2024-04-25T09:03:32.790938+0000","last_clean":"2024-04-25T09:03:32.790938+0000","last_became_active":"2024-04-25T09:03:32.143891+0000","last_became_peered":"2024-04-25T09:03:32.143891+0000","last_unstale":"2024-04-25T09:03:32.790938+0000","last_undegraded":"2024-04-25T09:03:32.790938+0000","last_fullsized":"2024-04-25T09:03:32.790938+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:22.612040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.809556+0000","last_change":"2024-04-25T09:03:32.804007+0000","last_active":"2024-04-25T09:03:32.809556+0000","last_peered":"2024-04-25T09:03:32.809556+0000","last_clean":"2024-04-25T09:03:32.809556+0000","last_became_active":"2024-04-25T09:03:32.803115+0000","last_became_peered":"2024-04-25T09:03:32.803115+0000","last_unstale":"2024-04-25T09:03:32.809556+0000","last_undegraded":"2024-04-25T09:03:32.809556+0000","last_fullsized":"2024-04-25T09:03:32.809556+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:24.835744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.782793+0000","last_change":"2024-04-25T09:03:32.149239+0000","last_active":"2024-04-25T09:03:32.782793+0000","last_peered":"2024-04-25T09:03:32.782793+0000","last_clean":"2024-04-25T09:03:32.782793+0000","last_became_active":"2024-04-25T09:03:32.147139+0000","last_became_peered":"2024-04-25T09:03:32.147139+0000","last_unstale":"2024-04-25T09:03:32.782793+0000","last_undegraded":"2024-04-25T09:03:32.782793+0000","last_fullsized":"2024-04-25T09:03:32.782793+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:07.372615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.837054+0000","last_change":"2024-04-25T09:03:32.829900+0000","last_active":"2024-04-25T09:03:32.837054+0000","last_peered":"2024-04-25T09:03:32.837054+0000","last_clean":"2024-04-25T09:03:32.837054+0000","last_became_active":"2024-04-25T09:03:32.829163+0000","last_became_peered":"2024-04-25T09:03:32.829163+0000","last_unstale":"2024-04-25T09:03:32.837054+0000","last_undegraded":"2024-04-25T09:03:32.837054+0000","last_fullsized":"2024-04-25T09:03:32.837054+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:32.238618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.812720+0000","last_change":"2024-04-25T09:03:32.149208+0000","last_active":"2024-04-25T09:03:32.812720+0000","last_peered":"2024-04-25T09:03:32.812720+0000","last_clean":"2024-04-25T09:03:32.812720+0000","last_became_active":"2024-04-25T09:03:32.148445+0000","last_became_peered":"2024-04-25T09:03:32.148445+0000","last_unstale":"2024-04-25T09:03:32.812720+0000","last_undegraded":"2024-04-25T09:03:32.812720+0000","last_fullsized":"2024-04-25T09:03:32.812720+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:44.666207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831466+0000","last_change":"2024-04-25T09:03:32.821118+0000","last_active":"2024-04-25T09:03:32.831466+0000","last_peered":"2024-04-25T09:03:32.831466+0000","last_clean":"2024-04-25T09:03:32.831466+0000","last_became_active":"2024-04-25T09:03:32.820285+0000","last_became_peered":"2024-04-25T09:03:32.820285+0000","last_unstale":"2024-04-25T09:03:32.831466+0000","last_undegraded":"2024-04-25T09:03:32.831466+0000","last_fullsized":"2024-04-25T09:03:32.831466+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:26.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"124'3","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.146856+0000","last_change":"2024-04-25T09:03:32.150827+0000","last_active":"2024-04-25T09:03:33.146856+0000","last_peered":"2024-04-25T09:03:33.146856+0000","last_clean":"2024-04-25T09:03:33.146856+0000","last_became_active":"2024-04-25T09:03:32.150130+0000","last_became_peered":"2024-04-25T09:03:32.150130+0000","last_unstale":"2024-04-25T09:03:33.146856+0000","last_undegraded":"2024-04-25T09:03:33.146856+0000","last_fullsized":"2024-04-25T09:03:33.146856+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:07.941668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.837292+0000","last_change":"2024-04-25T09:03:32.828654+0000","last_active":"2024-04-25T09:03:32.837292+0000","last_peered":"2024-04-25T09:03:32.837292+0000","last_clean":"2024-04-25T09:03:32.837292+0000","last_became_active":"2024-04-25T09:03:32.822245+0000","last_became_peered":"2024-04-25T09:03:32.822245+0000","last_unstale":"2024-04-25T09:03:32.837292+0000","last_undegraded":"2024-04-25T09:03:32.837292+0000","last_fullsized":"2024-04-25T09:03:32.837292+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:09.004049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"124'6","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.139708+0000","last_change":"2024-04-25T09:03:32.158665+0000","last_active":"2024-04-25T09:03:33.139708+0000","last_peered":"2024-04-25T09:03:33.139708+0000","last_clean":"2024-04-25T09:03:33.139708+0000","last_became_active":"2024-04-25T09:03:32.158224+0000","last_became_peered":"2024-04-25T09:03:32.158224+0000","last_unstale":"2024-04-25T09:03:33.139708+0000","last_undegraded":"2024-04-25T09:03:33.139708+0000","last_fullsized":"2024-04-25T09:03:33.139708+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:51.559954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.810123+0000","last_change":"2024-04-25T09:03:32.797523+0000","last_active":"2024-04-25T09:03:32.810123+0000","last_peered":"2024-04-25T09:03:32.810123+0000","last_clean":"2024-04-25T09:03:32.810123+0000","last_became_active":"2024-04-25T09:03:32.796425+0000","last_became_peered":"2024-04-25T09:03:32.796425+0000","last_unstale":"2024-04-25T09:03:32.810123+0000","last_undegraded":"2024-04-25T09:03:32.810123+0000","last_fullsized":"2024-04-25T09:03:32.810123+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:32.238618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.134904+0000","last_change":"2024-04-25T09:03:32.155066+0000","last_active":"2024-04-25T09:03:33.134904+0000","last_peered":"2024-04-25T09:03:33.134904+0000","last_clean":"2024-04-25T09:03:33.134904+0000","last_became_active":"2024-04-25T09:03:32.154618+0000","last_became_peered":"2024-04-25T09:03:32.154618+0000","last_unstale":"2024-04-25T09:03:33.134904+0000","last_undegraded":"2024-04-25T09:03:33.134904+0000","last_fullsized":"2024-04-25T09:03:33.134904+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:31.541020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.800891+0000","last_change":"2024-04-25T09:03:32.793813+0000","last_active":"2024-04-25T09:03:32.800891+0000","last_peered":"2024-04-25T09:03:32.800891+0000","last_clean":"2024-04-25T09:03:32.800891+0000","last_became_active":"2024-04-25T09:03:32.793069+0000","last_became_peered":"2024-04-25T09:03:32.793069+0000","last_unstale":"2024-04-25T09:03:32.800891+0000","last_undegraded":"2024-04-25T09:03:32.800891+0000","last_fullsized":"2024-04-25T09:03:32.800891+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:26.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.820734+0000","last_change":"2024-04-25T09:03:32.158152+0000","last_active":"2024-04-25T09:03:32.820734+0000","last_peered":"2024-04-25T09:03:32.820734+0000","last_clean":"2024-04-25T09:03:32.820734+0000","last_became_active":"2024-04-25T09:03:32.157678+0000","last_became_peered":"2024-04-25T09:03:32.157678+0000","last_unstale":"2024-04-25T09:03:32.820734+0000","last_undegraded":"2024-04-25T09:03:32.820734+0000","last_fullsized":"2024-04-25T09:03:32.820734+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:00.666009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.811689+0000","last_change":"2024-04-25T09:03:32.809552+0000","last_active":"2024-04-25T09:03:32.811689+0000","last_peered":"2024-04-25T09:03:32.811689+0000","last_clean":"2024-04-25T09:03:32.811689+0000","last_became_active":"2024-04-25T09:03:32.808822+0000","last_became_peered":"2024-04-25T09:03:32.808822+0000","last_unstale":"2024-04-25T09:03:32.811689+0000","last_undegraded":"2024-04-25T09:03:32.811689+0000","last_fullsized":"2024-04-25T09:03:32.811689+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:27.333634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.797512+0000","last_change":"2024-04-25T09:03:32.154024+0000","last_active":"2024-04-25T09:03:32.797512+0000","last_peered":"2024-04-25T09:03:32.797512+0000","last_clean":"2024-04-25T09:03:32.797512+0000","last_became_active":"2024-04-25T09:03:32.153707+0000","last_became_peered":"2024-04-25T09:03:32.153707+0000","last_unstale":"2024-04-25T09:03:32.797512+0000","last_undegraded":"2024-04-25T09:03:32.797512+0000","last_fullsized":"2024-04-25T09:03:32.797512+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:29.947679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.836394+0000","last_change":"2024-04-25T09:03:32.821277+0000","last_active":"2024-04-25T09:03:32.836394+0000","last_peered":"2024-04-25T09:03:32.836394+0000","last_clean":"2024-04-25T09:03:32.836394+0000","last_became_active":"2024-04-25T09:03:32.820192+0000","last_became_peered":"2024-04-25T09:03:32.820192+0000","last_unstale":"2024-04-25T09:03:32.836394+0000","last_undegraded":"2024-04-25T09:03:32.836394+0000","last_fullsized":"2024-04-25T09:03:32.836394+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:13.774476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.131404+0000","last_change":"2024-04-25T09:03:32.153853+0000","last_active":"2024-04-25T09:03:33.131404+0000","last_peered":"2024-04-25T09:03:33.131404+0000","last_clean":"2024-04-25T09:03:33.131404+0000","last_became_active":"2024-04-25T09:03:32.153076+0000","last_became_peered":"2024-04-25T09:03:32.153076+0000","last_unstale":"2024-04-25T09:03:33.131404+0000","last_undegraded":"2024-04-25T09:03:33.131404+0000","last_fullsized":"2024-04-25T09:03:33.131404+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.254934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.839179+0000","last_change":"2024-04-25T09:03:32.836323+0000","last_active":"2024-04-25T09:03:32.839179+0000","last_peered":"2024-04-25T09:03:32.839179+0000","last_clean":"2024-04-25T09:03:32.839179+0000","last_became_active":"2024-04-25T09:03:32.835704+0000","last_became_peered":"2024-04-25T09:03:32.835704+0000","last_unstale":"2024-04-25T09:03:32.839179+0000","last_undegraded":"2024-04-25T09:03:32.839179+0000","last_fullsized":"2024-04-25T09:03:32.839179+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:55.685027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.133260+0000","last_change":"2024-04-25T09:03:32.153467+0000","last_active":"2024-04-25T09:03:33.133260+0000","last_peered":"2024-04-25T09:03:33.133260+0000","last_clean":"2024-04-25T09:03:33.133260+0000","last_became_active":"2024-04-25T09:03:32.152138+0000","last_became_peered":"2024-04-25T09:03:32.152138+0000","last_unstale":"2024-04-25T09:03:33.133260+0000","last_undegraded":"2024-04-25T09:03:33.133260+0000","last_fullsized":"2024-04-25T09:03:33.133260+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:52.568160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.818924+0000","last_change":"2024-04-25T09:03:32.814236+0000","last_active":"2024-04-25T09:03:32.818924+0000","last_peered":"2024-04-25T09:03:32.818924+0000","last_clean":"2024-04-25T09:03:32.818924+0000","last_became_active":"2024-04-25T09:03:32.808997+0000","last_became_peered":"2024-04-25T09:03:32.808997+0000","last_unstale":"2024-04-25T09:03:32.818924+0000","last_undegraded":"2024-04-25T09:03:32.818924+0000","last_fullsized":"2024-04-25T09:03:32.818924+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:13.994789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.144111+0000","last_change":"2024-04-25T09:03:32.162761+0000","last_active":"2024-04-25T09:03:33.144111+0000","last_peered":"2024-04-25T09:03:33.144111+0000","last_clean":"2024-04-25T09:03:33.144111+0000","last_became_active":"2024-04-25T09:03:32.162177+0000","last_became_peered":"2024-04-25T09:03:32.162177+0000","last_unstale":"2024-04-25T09:03:33.144111+0000","last_undegraded":"2024-04-25T09:03:33.144111+0000","last_fullsized":"2024-04-25T09:03:33.144111+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:08.000210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.838930+0000","last_change":"2024-04-25T09:03:32.836311+0000","last_active":"2024-04-25T09:03:32.838930+0000","last_peered":"2024-04-25T09:03:32.838930+0000","last_clean":"2024-04-25T09:03:32.838930+0000","last_became_active":"2024-04-25T09:03:32.831215+0000","last_became_peered":"2024-04-25T09:03:32.831215+0000","last_unstale":"2024-04-25T09:03:32.838930+0000","last_undegraded":"2024-04-25T09:03:32.838930+0000","last_fullsized":"2024-04-25T09:03:32.838930+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.420083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.797305+0000","last_change":"2024-04-25T09:03:32.158850+0000","last_active":"2024-04-25T09:03:32.797305+0000","last_peered":"2024-04-25T09:03:32.797305+0000","last_clean":"2024-04-25T09:03:32.797305+0000","last_became_active":"2024-04-25T09:03:32.158190+0000","last_became_peered":"2024-04-25T09:03:32.158190+0000","last_unstale":"2024-04-25T09:03:32.797305+0000","last_undegraded":"2024-04-25T09:03:32.797305+0000","last_fullsized":"2024-04-25T09:03:32.797305+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:12.770637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.818469+0000","last_change":"2024-04-25T09:03:32.813019+0000","last_active":"2024-04-25T09:03:32.818469+0000","last_peered":"2024-04-25T09:03:32.818469+0000","last_clean":"2024-04-25T09:03:32.818469+0000","last_became_active":"2024-04-25T09:03:32.808886+0000","last_became_peered":"2024-04-25T09:03:32.808886+0000","last_unstale":"2024-04-25T09:03:32.818469+0000","last_undegraded":"2024-04-25T09:03:32.818469+0000","last_fullsized":"2024-04-25T09:03:32.818469+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:18.896389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.140666+0000","last_change":"2024-04-25T09:03:32.148444+0000","last_active":"2024-04-25T09:03:33.140666+0000","last_peered":"2024-04-25T09:03:33.140666+0000","last_clean":"2024-04-25T09:03:33.140666+0000","last_became_active":"2024-04-25T09:03:32.147856+0000","last_became_peered":"2024-04-25T09:03:32.147856+0000","last_unstale":"2024-04-25T09:03:33.140666+0000","last_undegraded":"2024-04-25T09:03:33.140666+0000","last_fullsized":"2024-04-25T09:03:33.140666+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:07.928541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831814+0000","last_change":"2024-04-25T09:03:32.826708+0000","last_active":"2024-04-25T09:03:32.831814+0000","last_peered":"2024-04-25T09:03:32.831814+0000","last_clean":"2024-04-25T09:03:32.831814+0000","last_became_active":"2024-04-25T09:03:32.825927+0000","last_became_peered":"2024-04-25T09:03:32.825927+0000","last_unstale":"2024-04-25T09:03:32.831814+0000","last_undegraded":"2024-04-25T09:03:32.831814+0000","last_fullsized":"2024-04-25T09:03:32.831814+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:25.603713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.795516+0000","last_change":"2024-04-25T09:03:32.163640+0000","last_active":"2024-04-25T09:03:32.795516+0000","last_peered":"2024-04-25T09:03:32.795516+0000","last_clean":"2024-04-25T09:03:32.795516+0000","last_became_active":"2024-04-25T09:03:32.162907+0000","last_became_peered":"2024-04-25T09:03:32.162907+0000","last_unstale":"2024-04-25T09:03:32.795516+0000","last_undegraded":"2024-04-25T09:03:32.795516+0000","last_fullsized":"2024-04-25T09:03:32.795516+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.254934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831643+0000","last_change":"2024-04-25T09:03:32.825769+0000","last_active":"2024-04-25T09:03:32.831643+0000","last_peered":"2024-04-25T09:03:32.831643+0000","last_clean":"2024-04-25T09:03:32.831643+0000","last_became_active":"2024-04-25T09:03:32.821572+0000","last_became_peered":"2024-04-25T09:03:32.821572+0000","last_unstale":"2024-04-25T09:03:32.831643+0000","last_undegraded":"2024-04-25T09:03:32.831643+0000","last_fullsized":"2024-04-25T09:03:32.831643+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:07.424424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.797939+0000","last_change":"2024-04-25T09:03:32.153220+0000","last_active":"2024-04-25T09:03:32.797939+0000","last_peered":"2024-04-25T09:03:32.797939+0000","last_clean":"2024-04-25T09:03:32.797939+0000","last_became_active":"2024-04-25T09:03:32.152800+0000","last_became_peered":"2024-04-25T09:03:32.152800+0000","last_unstale":"2024-04-25T09:03:32.797939+0000","last_undegraded":"2024-04-25T09:03:32.797939+0000","last_fullsized":"2024-04-25T09:03:32.797939+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:55.846258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.837066+0000","last_change":"2024-04-25T09:03:32.829572+0000","last_active":"2024-04-25T09:03:32.837066+0000","last_peered":"2024-04-25T09:03:32.837066+0000","last_clean":"2024-04-25T09:03:32.837066+0000","last_became_active":"2024-04-25T09:03:32.828867+0000","last_became_peered":"2024-04-25T09:03:32.828867+0000","last_unstale":"2024-04-25T09:03:32.837066+0000","last_undegraded":"2024-04-25T09:03:32.837066+0000","last_fullsized":"2024-04-25T09:03:32.837066+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:08.376454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.795917+0000","last_change":"2024-04-25T09:03:32.152796+0000","last_active":"2024-04-25T09:03:32.795917+0000","last_peered":"2024-04-25T09:03:32.795917+0000","last_clean":"2024-04-25T09:03:32.795917+0000","last_became_active":"2024-04-25T09:03:32.151086+0000","last_became_peered":"2024-04-25T09:03:32.151086+0000","last_unstale":"2024-04-25T09:03:32.795917+0000","last_undegraded":"2024-04-25T09:03:32.795917+0000","last_fullsized":"2024-04-25T09:03:32.795917+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:15.525415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.815578+0000","last_change":"2024-04-25T09:03:32.808888+0000","last_active":"2024-04-25T09:03:32.815578+0000","last_peered":"2024-04-25T09:03:32.815578+0000","last_clean":"2024-04-25T09:03:32.815578+0000","last_became_active":"2024-04-25T09:03:32.807200+0000","last_became_peered":"2024-04-25T09:03:32.807200+0000","last_unstale":"2024-04-25T09:03:32.815578+0000","last_undegraded":"2024-04-25T09:03:32.815578+0000","last_fullsized":"2024-04-25T09:03:32.815578+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:05.062073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.138086+0000","last_change":"2024-04-25T09:03:32.150818+0000","last_active":"2024-04-25T09:03:33.138086+0000","last_peered":"2024-04-25T09:03:33.138086+0000","last_clean":"2024-04-25T09:03:33.138086+0000","last_became_active":"2024-04-25T09:03:32.150538+0000","last_became_peered":"2024-04-25T09:03:32.150538+0000","last_unstale":"2024-04-25T09:03:33.138086+0000","last_undegraded":"2024-04-25T09:03:33.138086+0000","last_fullsized":"2024-04-25T09:03:33.138086+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:23.585002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.802005+0000","last_change":"2024-04-25T09:03:32.789111+0000","last_active":"2024-04-25T09:03:32.802005+0000","last_peered":"2024-04-25T09:03:32.802005+0000","last_clean":"2024-04-25T09:03:32.802005+0000","last_became_active":"2024-04-25T09:03:32.788355+0000","last_became_peered":"2024-04-25T09:03:32.788355+0000","last_unstale":"2024-04-25T09:03:32.802005+0000","last_undegraded":"2024-04-25T09:03:32.802005+0000","last_fullsized":"2024-04-25T09:03:32.802005+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:13.774476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.780476+0000","last_change":"2024-04-25T09:03:32.149607+0000","last_active":"2024-04-25T09:03:32.780476+0000","last_peered":"2024-04-25T09:03:32.780476+0000","last_clean":"2024-04-25T09:03:32.780476+0000","last_became_active":"2024-04-25T09:03:32.148774+0000","last_became_peered":"2024-04-25T09:03:32.148774+0000","last_unstale":"2024-04-25T09:03:32.780476+0000","last_undegraded":"2024-04-25T09:03:32.780476+0000","last_fullsized":"2024-04-25T09:03:32.780476+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:25.712381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.828060+0000","last_change":"2024-04-25T09:03:32.823047+0000","last_active":"2024-04-25T09:03:32.828060+0000","last_peered":"2024-04-25T09:03:32.828060+0000","last_clean":"2024-04-25T09:03:32.828060+0000","last_became_active":"2024-04-25T09:03:32.820771+0000","last_became_peered":"2024-04-25T09:03:32.820771+0000","last_unstale":"2024-04-25T09:03:32.828060+0000","last_undegraded":"2024-04-25T09:03:32.828060+0000","last_fullsized":"2024-04-25T09:03:32.828060+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:55.685027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.782776+0000","last_change":"2024-04-25T09:03:32.137406+0000","last_active":"2024-04-25T09:03:32.782776+0000","last_peered":"2024-04-25T09:03:32.782776+0000","last_clean":"2024-04-25T09:03:32.782776+0000","last_became_active":"2024-04-25T09:03:32.136585+0000","last_became_peered":"2024-04-25T09:03:32.136585+0000","last_unstale":"2024-04-25T09:03:32.782776+0000","last_undegraded":"2024-04-25T09:03:32.782776+0000","last_fullsized":"2024-04-25T09:03:32.782776+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:08.000210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.835594+0000","last_change":"2024-04-25T09:03:32.834222+0000","last_active":"2024-04-25T09:03:32.835594+0000","last_peered":"2024-04-25T09:03:32.835594+0000","last_clean":"2024-04-25T09:03:32.835594+0000","last_became_active":"2024-04-25T09:03:32.833490+0000","last_became_peered":"2024-04-25T09:03:32.833490+0000","last_unstale":"2024-04-25T09:03:32.835594+0000","last_undegraded":"2024-04-25T09:03:32.835594+0000","last_fullsized":"2024-04-25T09:03:32.835594+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:36.156183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.792523+0000","last_change":"2024-04-25T09:03:32.151390+0000","last_active":"2024-04-25T09:03:32.792523+0000","last_peered":"2024-04-25T09:03:32.792523+0000","last_clean":"2024-04-25T09:03:32.792523+0000","last_became_active":"2024-04-25T09:03:32.149110+0000","last_became_peered":"2024-04-25T09:03:32.149110+0000","last_unstale":"2024-04-25T09:03:32.792523+0000","last_undegraded":"2024-04-25T09:03:32.792523+0000","last_fullsized":"2024-04-25T09:03:32.792523+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:57.582279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.816732+0000","last_change":"2024-04-25T09:03:32.808003+0000","last_active":"2024-04-25T09:03:32.816732+0000","last_peered":"2024-04-25T09:03:32.816732+0000","last_clean":"2024-04-25T09:03:32.816732+0000","last_became_active":"2024-04-25T09:03:32.807272+0000","last_became_peered":"2024-04-25T09:03:32.807272+0000","last_unstale":"2024-04-25T09:03:32.816732+0000","last_undegraded":"2024-04-25T09:03:32.816732+0000","last_fullsized":"2024-04-25T09:03:32.816732+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:56.861333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.783352+0000","last_change":"2024-04-25T09:03:32.137822+0000","last_active":"2024-04-25T09:03:32.783352+0000","last_peered":"2024-04-25T09:03:32.783352+0000","last_clean":"2024-04-25T09:03:32.783352+0000","last_became_active":"2024-04-25T09:03:32.137026+0000","last_became_peered":"2024-04-25T09:03:32.137026+0000","last_unstale":"2024-04-25T09:03:32.783352+0000","last_undegraded":"2024-04-25T09:03:32.783352+0000","last_fullsized":"2024-04-25T09:03:32.783352+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:08.000210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.798830+0000","last_change":"2024-04-25T09:03:32.795561+0000","last_active":"2024-04-25T09:03:32.798830+0000","last_peered":"2024-04-25T09:03:32.798830+0000","last_clean":"2024-04-25T09:03:32.798830+0000","last_became_active":"2024-04-25T09:03:32.793594+0000","last_became_peered":"2024-04-25T09:03:32.793594+0000","last_unstale":"2024-04-25T09:03:32.798830+0000","last_undegraded":"2024-04-25T09:03:32.798830+0000","last_fullsized":"2024-04-25T09:03:32.798830+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:39.750952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.785023+0000","last_change":"2024-04-25T09:03:32.154303+0000","last_active":"2024-04-25T09:03:32.785023+0000","last_peered":"2024-04-25T09:03:32.785023+0000","last_clean":"2024-04-25T09:03:32.785023+0000","last_became_active":"2024-04-25T09:03:32.153400+0000","last_became_peered":"2024-04-25T09:03:32.153400+0000","last_unstale":"2024-04-25T09:03:32.785023+0000","last_undegraded":"2024-04-25T09:03:32.785023+0000","last_fullsized":"2024-04-25T09:03:32.785023+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:07.372615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.824876+0000","last_change":"2024-04-25T09:03:32.818442+0000","last_active":"2024-04-25T09:03:32.824876+0000","last_peered":"2024-04-25T09:03:32.824876+0000","last_clean":"2024-04-25T09:03:32.824876+0000","last_became_active":"2024-04-25T09:03:32.817653+0000","last_became_peered":"2024-04-25T09:03:32.817653+0000","last_unstale":"2024-04-25T09:03:32.824876+0000","last_undegraded":"2024-04-25T09:03:32.824876+0000","last_fullsized":"2024-04-25T09:03:32.824876+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:08.376454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.791091+0000","last_change":"2024-04-25T09:03:32.150314+0000","last_active":"2024-04-25T09:03:32.791091+0000","last_peered":"2024-04-25T09:03:32.791091+0000","last_clean":"2024-04-25T09:03:32.791091+0000","last_became_active":"2024-04-25T09:03:32.148425+0000","last_became_peered":"2024-04-25T09:03:32.148425+0000","last_unstale":"2024-04-25T09:03:32.791091+0000","last_undegraded":"2024-04-25T09:03:32.791091+0000","last_fullsized":"2024-04-25T09:03:32.791091+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:13.576886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.808092+0000","last_change":"2024-04-25T09:03:32.803221+0000","last_active":"2024-04-25T09:03:32.808092+0000","last_peered":"2024-04-25T09:03:32.808092+0000","last_clean":"2024-04-25T09:03:32.808092+0000","last_became_active":"2024-04-25T09:03:32.802048+0000","last_became_peered":"2024-04-25T09:03:32.802048+0000","last_unstale":"2024-04-25T09:03:32.808092+0000","last_undegraded":"2024-04-25T09:03:32.808092+0000","last_fullsized":"2024-04-25T09:03:32.808092+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:28.186515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.804515+0000","last_change":"2024-04-25T09:03:32.153503+0000","last_active":"2024-04-25T09:03:32.804515+0000","last_peered":"2024-04-25T09:03:32.804515+0000","last_clean":"2024-04-25T09:03:32.804515+0000","last_became_active":"2024-04-25T09:03:32.152057+0000","last_became_peered":"2024-04-25T09:03:32.152057+0000","last_unstale":"2024-04-25T09:03:32.804515+0000","last_undegraded":"2024-04-25T09:03:32.804515+0000","last_fullsized":"2024-04-25T09:03:32.804515+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:25.712381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.825162+0000","last_change":"2024-04-25T09:03:32.820828+0000","last_active":"2024-04-25T09:03:32.825162+0000","last_peered":"2024-04-25T09:03:32.825162+0000","last_clean":"2024-04-25T09:03:32.825162+0000","last_became_active":"2024-04-25T09:03:32.818733+0000","last_became_peered":"2024-04-25T09:03:32.818733+0000","last_unstale":"2024-04-25T09:03:32.825162+0000","last_undegraded":"2024-04-25T09:03:32.825162+0000","last_fullsized":"2024-04-25T09:03:32.825162+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:24.835744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"124'8","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.137091+0000","last_change":"2024-04-25T09:03:32.144778+0000","last_active":"2024-04-25T09:03:33.137091+0000","last_peered":"2024-04-25T09:03:33.137091+0000","last_clean":"2024-04-25T09:03:33.137091+0000","last_became_active":"2024-04-25T09:03:32.144394+0000","last_became_peered":"2024-04-25T09:03:32.144394+0000","last_unstale":"2024-04-25T09:03:33.137091+0000","last_undegraded":"2024-04-25T09:03:33.137091+0000","last_fullsized":"2024-04-25T09:03:33.137091+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:35.877384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":365,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.804265+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T09:03:32.804265+0000","last_peered":"2024-04-25T09:03:32.804265+0000","last_clean":"2024-04-25T09:03:32.804265+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T09:03:32.804265+0000","last_undegraded":"2024-04-25T09:03:32.804265+0000","last_fullsized":"2024-04-25T09:03:32.804265+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574986,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2125824,"data_stored":1306656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":724037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1544192,"data_stored":723455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574986,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1556480,"data_stored":725067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574986,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1544192,"data_stored":723455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1544192,"data_stored":723455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1531904,"data_stored":716288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2138112,"data_stored":1313823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:37.157 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:37.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f14b3b640 1 Processor -- start 2024-04-25T09:03:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f14b3b640 1 -- start start 2024-04-25T09:03:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f14b3b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 0x7f0f1015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f14b3b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0f100b24a0 0x7f0f100b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f14b3b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f100b2090 con 0x7f0f100b3f50 2024-04-25T09:03:37.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f14b3b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f1015f200 con 0x7f0f100b24a0 2024-04-25T09:03:37.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f14b3b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100b43e0 con 0x7f0f1015a700 2024-04-25T09:03:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f0e59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 0x7f0f1015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f0e59c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 0x7f0f1015ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:36156/0 (socket says 172.21.15.37:36156) 2024-04-25T09:03:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f0e59c640 1 -- 172.21.15.37:0/2706434468 learned_addr learned my addr 172.21.15.37:0/2706434468 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f0ed9d640 1 --2- 172.21.15.37:0/2706434468 >> v2:172.21.15.203:3300/0 conn(0x7f0f100b24a0 0x7f0f100b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f0e59c640 1 -- 172.21.15.37:0/2706434468 >> v2:172.21.15.203:3300/0 conn(0x7f0f100b24a0 msgr2=0x7f0f100b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f0e59c640 1 --2- 172.21.15.37:0/2706434468 >> v2:172.21.15.203:3300/0 conn(0x7f0f100b24a0 0x7f0f100b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f0e59c640 1 -- 172.21.15.37:0/2706434468 >> v1:172.21.15.203:6789/0 conn(0x7f0f100b3f50 legacy=0x7f0f100cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.329+0000 7f0f0e59c640 1 -- 172.21.15.37:0/2706434468 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f100b46c0 con 0x7f0f1015a700 2024-04-25T09:03:37.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f0e59c640 1 --2- 172.21.15.37:0/2706434468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 0x7f0f1015ecc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009e50 tx=0x7f0f0001c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad70c3afd6be5ff9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:37.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f0d59a640 1 -- 172.21.15.37:0/2706434468 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f000a56c0 con 0x7f0f1015a700 2024-04-25T09:03:37.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f0d59a640 1 -- 172.21.15.37:0/2706434468 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f000a5860 con 0x7f0f1015a700 2024-04-25T09:03:37.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f0d59a640 1 -- 172.21.15.37:0/2706434468 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f000a5c30 con 0x7f0f1015a700 2024-04-25T09:03:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 -- 172.21.15.37:0/2706434468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 msgr2=0x7f0f1015ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 --2- 172.21.15.37:0/2706434468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 0x7f0f1015ecc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f00009e50 tx=0x7f0f0001c910 comp rx=0 tx=0).stop 2024-04-25T09:03:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 -- 172.21.15.37:0/2706434468 shutdown_connections 2024-04-25T09:03:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 --2- 172.21.15.37:0/2706434468 >> v2:172.21.15.203:3300/0 conn(0x7f0f100b24a0 0x7f0f100b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 --2- 172.21.15.37:0/2706434468 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 0x7f0f1015ecc0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 -- 172.21.15.37:0/2706434468 >> 172.21.15.37:0/2706434468 conn(0x7f0f100b11a0 msgr2=0x7f0f100b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:37.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 -- 172.21.15.37:0/2706434468 shutdown_connections 2024-04-25T09:03:37.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 -- 172.21.15.37:0/2706434468 wait complete. 2024-04-25T09:03:37.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 Processor -- start 2024-04-25T09:03:37.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 -- start start 2024-04-25T09:03:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f100b24a0 0x7f0f101559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 0x7f0f101f9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f101fa0d0 0x7f0f101fa560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100cd310 con 0x7f0f101fa0d0 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100ce600 con 0x7f0f100b24a0 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f14b3b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f100cd490 con 0x7f0f1015a700 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.333+0000 7f0f0ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f101fa0d0 0x7f0f101fa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f101fa0d0 0x7f0f101fa560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51036/0 (socket says 172.21.15.37:51036) 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0ed9d640 1 -- 172.21.15.37:0/1367582279 learned_addr learned my addr 172.21.15.37:0/1367582279 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0e59c640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f100b24a0 0x7f0f101559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0dd9b640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 0x7f0f101f9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0ed9d640 1 -- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 msgr2=0x7f0f101f9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0ed9d640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 0x7f0f101f9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0ed9d640 1 -- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f100b24a0 msgr2=0x7f0f101559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0ed9d640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f100b24a0 0x7f0f101559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0ed9d640 1 -- 172.21.15.37:0/1367582279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1014d020 con 0x7f0f101fa0d0 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0dd9b640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 0x7f0f101f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:37.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0ed9d640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f101fa0d0 0x7f0f101fa560 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800e820 tx=0x7f0f0800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:37.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0efeffd640 1 -- 172.21.15.37:0/1367582279 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f08017070 con 0x7f0f101fa0d0 2024-04-25T09:03:37.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0efeffd640 1 -- 172.21.15.37:0/1367582279 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f0800c940 con 0x7f0f101fa0d0 2024-04-25T09:03:37.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f14b3b640 1 -- 172.21.15.37:0/1367582279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f101faaa0 con 0x7f0f101fa0d0 2024-04-25T09:03:37.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0efeffd640 1 -- 172.21.15.37:0/1367582279 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f0800cd10 con 0x7f0f101fa0d0 2024-04-25T09:03:37.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f14b3b640 1 -- 172.21.15.37:0/1367582279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f10158360 con 0x7f0f101fa0d0 2024-04-25T09:03:37.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f14b3b640 1 -- 172.21.15.37:0/1367582279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1004efa0 con 0x7f0f101fa0d0 2024-04-25T09:03:37.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0efeffd640 1 -- 172.21.15.37:0/1367582279 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0f0801a070 con 0x7f0f101fa0d0 2024-04-25T09:03:37.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0efeffd640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ef0066f60 0x7f0ef0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0efeffd640 1 -- 172.21.15.37:0/1367582279 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f08084ee0 con 0x7f0f101fa0d0 2024-04-25T09:03:37.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0e59c640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ef0066f60 0x7f0ef0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.337+0000 7f0f0e59c640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ef0066f60 0x7f0ef0069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0f000ae000 tx=0x7f0f000a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:37.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.341+0000 7f0efeffd640 1 -- 172.21.15.37:0/1367582279 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0f101faaa0 con 0x7f0f101fa0d0 2024-04-25T09:03:37.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.497+0000 7f0f14b3b640 1 -- 172.21.15.37:0/1367582279 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f10156290 con 0x7f0ef0066f60 2024-04-25T09:03:37.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.513+0000 7f0efeffd640 1 -- 172.21.15.37:0/1367582279 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252416 (secure 0 0 0) 0x7f0f000a56c0 con 0x7f0ef0066f60 2024-04-25T09:03:37.506 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:37.508 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T09:03:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 -- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ef0066f60 msgr2=0x7f0ef0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ef0066f60 0x7f0ef0069420 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f0f000ae000 tx=0x7f0f000a7070 comp rx=0 tx=0).stop 2024-04-25T09:03:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 -- 172.21.15.37:0/1367582279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f101fa0d0 msgr2=0x7f0f101fa560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f101fa0d0 0x7f0f101fa560 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800e820 tx=0x7f0f0800ece0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 -- 172.21.15.37:0/1367582279 shutdown_connections 2024-04-25T09:03:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ef0066f60 0x7f0ef0069420 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f101fa0d0 0x7f0f101fa560 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0f1015a700 0x7f0f101f9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 --2- 172.21.15.37:0/1367582279 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0f100b24a0 0x7f0f101559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 -- 172.21.15.37:0/1367582279 >> 172.21.15.37:0/1367582279 conn(0x7f0f100b11a0 msgr2=0x7f0f1006d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 -- 172.21.15.37:0/1367582279 shutdown_connections 2024-04-25T09:03:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.517+0000 7f0f14b3b640 1 -- 172.21.15.37:0/1367582279 wait complete. 2024-04-25T09:03:37.525 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-04-25T09:03:37.357447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":96,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262680,"kb_used_data":13216,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712040,"statfs":{"total":773094113280,"available":772825128960,"internally_reserved":0,"allocated":13533184,"data_stored":6956236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001719"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.808221+0000","last_change":"2024-04-25T09:03:32.803679+0000","last_active":"2024-04-25T09:03:32.808221+0000","last_peered":"2024-04-25T09:03:32.808221+0000","last_clean":"2024-04-25T09:03:32.808221+0000","last_became_active":"2024-04-25T09:03:32.802944+0000","last_became_peered":"2024-04-25T09:03:32.802944+0000","last_unstale":"2024-04-25T09:03:32.808221+0000","last_undegraded":"2024-04-25T09:03:32.808221+0000","last_fullsized":"2024-04-25T09:03:32.808221+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:24.835744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831973+0000","last_change":"2024-04-25T09:03:32.829719+0000","last_active":"2024-04-25T09:03:32.831973+0000","last_peered":"2024-04-25T09:03:32.831973+0000","last_clean":"2024-04-25T09:03:32.831973+0000","last_became_active":"2024-04-25T09:03:32.829183+0000","last_became_peered":"2024-04-25T09:03:32.829183+0000","last_unstale":"2024-04-25T09:03:32.831973+0000","last_undegraded":"2024-04-25T09:03:32.831973+0000","last_fullsized":"2024-04-25T09:03:32.831973+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:45.806170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.832011+0000","last_change":"2024-04-25T09:03:32.829788+0000","last_active":"2024-04-25T09:03:32.832011+0000","last_peered":"2024-04-25T09:03:32.832011+0000","last_clean":"2024-04-25T09:03:32.832011+0000","last_became_active":"2024-04-25T09:03:32.829143+0000","last_became_peered":"2024-04-25T09:03:32.829143+0000","last_unstale":"2024-04-25T09:03:32.832011+0000","last_undegraded":"2024-04-25T09:03:32.832011+0000","last_fullsized":"2024-04-25T09:03:32.832011+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:17.420562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.827573+0000","last_change":"2024-04-25T09:03:32.822942+0000","last_active":"2024-04-25T09:03:32.827573+0000","last_peered":"2024-04-25T09:03:32.827573+0000","last_clean":"2024-04-25T09:03:32.827573+0000","last_became_active":"2024-04-25T09:03:32.821904+0000","last_became_peered":"2024-04-25T09:03:32.821904+0000","last_unstale":"2024-04-25T09:03:32.827573+0000","last_undegraded":"2024-04-25T09:03:32.827573+0000","last_fullsized":"2024-04-25T09:03:32.827573+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:35.087962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.805059+0000","last_change":"2024-04-25T09:03:32.791004+0000","last_active":"2024-04-25T09:03:32.805059+0000","last_peered":"2024-04-25T09:03:32.805059+0000","last_clean":"2024-04-25T09:03:32.805059+0000","last_became_active":"2024-04-25T09:03:32.788710+0000","last_became_peered":"2024-04-25T09:03:32.788710+0000","last_unstale":"2024-04-25T09:03:32.805059+0000","last_undegraded":"2024-04-25T09:03:32.805059+0000","last_fullsized":"2024-04-25T09:03:32.805059+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:32.238618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.813756+0000","last_change":"2024-04-25T09:03:32.802590+0000","last_active":"2024-04-25T09:03:32.813756+0000","last_peered":"2024-04-25T09:03:32.813756+0000","last_clean":"2024-04-25T09:03:32.813756+0000","last_became_active":"2024-04-25T09:03:32.801840+0000","last_became_peered":"2024-04-25T09:03:32.801840+0000","last_unstale":"2024-04-25T09:03:32.813756+0000","last_undegraded":"2024-04-25T09:03:32.813756+0000","last_fullsized":"2024-04-25T09:03:32.813756+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:48.861040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.820844+0000","last_change":"2024-04-25T09:03:32.816495+0000","last_active":"2024-04-25T09:03:32.820844+0000","last_peered":"2024-04-25T09:03:32.820844+0000","last_clean":"2024-04-25T09:03:32.820844+0000","last_became_active":"2024-04-25T09:03:32.815726+0000","last_became_peered":"2024-04-25T09:03:32.815726+0000","last_unstale":"2024-04-25T09:03:32.820844+0000","last_undegraded":"2024-04-25T09:03:32.820844+0000","last_fullsized":"2024-04-25T09:03:32.820844+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:02.577084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831634+0000","last_change":"2024-04-25T09:03:32.825527+0000","last_active":"2024-04-25T09:03:32.831634+0000","last_peered":"2024-04-25T09:03:32.831634+0000","last_clean":"2024-04-25T09:03:32.831634+0000","last_became_active":"2024-04-25T09:03:32.824881+0000","last_became_peered":"2024-04-25T09:03:32.824881+0000","last_unstale":"2024-04-25T09:03:32.831634+0000","last_undegraded":"2024-04-25T09:03:32.831634+0000","last_fullsized":"2024-04-25T09:03:32.831634+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:47.542161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.836572+0000","last_change":"2024-04-25T09:03:32.834980+0000","last_active":"2024-04-25T09:03:32.836572+0000","last_peered":"2024-04-25T09:03:32.836572+0000","last_clean":"2024-04-25T09:03:32.836572+0000","last_became_active":"2024-04-25T09:03:32.833455+0000","last_became_peered":"2024-04-25T09:03:32.833455+0000","last_unstale":"2024-04-25T09:03:32.836572+0000","last_undegraded":"2024-04-25T09:03:32.836572+0000","last_fullsized":"2024-04-25T09:03:32.836572+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:53.171753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.812162+0000","last_change":"2024-04-25T09:03:32.806470+0000","last_active":"2024-04-25T09:03:32.812162+0000","last_peered":"2024-04-25T09:03:32.812162+0000","last_clean":"2024-04-25T09:03:32.812162+0000","last_became_active":"2024-04-25T09:03:32.805738+0000","last_became_peered":"2024-04-25T09:03:32.805738+0000","last_unstale":"2024-04-25T09:03:32.812162+0000","last_undegraded":"2024-04-25T09:03:32.812162+0000","last_fullsized":"2024-04-25T09:03:32.812162+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:32.238618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.810761+0000","last_change":"2024-04-25T09:03:32.806525+0000","last_active":"2024-04-25T09:03:32.810761+0000","last_peered":"2024-04-25T09:03:32.810761+0000","last_clean":"2024-04-25T09:03:32.810761+0000","last_became_active":"2024-04-25T09:03:32.805776+0000","last_became_peered":"2024-04-25T09:03:32.805776+0000","last_unstale":"2024-04-25T09:03:32.810761+0000","last_undegraded":"2024-04-25T09:03:32.810761+0000","last_fullsized":"2024-04-25T09:03:32.810761+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:35.562353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.818869+0000","last_change":"2024-04-25T09:03:32.813816+0000","last_active":"2024-04-25T09:03:32.818869+0000","last_peered":"2024-04-25T09:03:32.818869+0000","last_clean":"2024-04-25T09:03:32.818869+0000","last_became_active":"2024-04-25T09:03:32.808191+0000","last_became_peered":"2024-04-25T09:03:32.808191+0000","last_unstale":"2024-04-25T09:03:32.818869+0000","last_undegraded":"2024-04-25T09:03:32.818869+0000","last_fullsized":"2024-04-25T09:03:32.818869+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:26.449846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.818678+0000","last_change":"2024-04-25T09:03:32.813433+0000","last_active":"2024-04-25T09:03:32.818678+0000","last_peered":"2024-04-25T09:03:32.818678+0000","last_clean":"2024-04-25T09:03:32.818678+0000","last_became_active":"2024-04-25T09:03:32.812403+0000","last_became_peered":"2024-04-25T09:03:32.812403+0000","last_unstale":"2024-04-25T09:03:32.818678+0000","last_undegraded":"2024-04-25T09:03:32.818678+0000","last_fullsized":"2024-04-25T09:03:32.818678+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.420083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831894+0000","last_change":"2024-04-25T09:03:32.826779+0000","last_active":"2024-04-25T09:03:32.831894+0000","last_peered":"2024-04-25T09:03:32.831894+0000","last_clean":"2024-04-25T09:03:32.831894+0000","last_became_active":"2024-04-25T09:03:32.825162+0000","last_became_peered":"2024-04-25T09:03:32.825162+0000","last_unstale":"2024-04-25T09:03:32.831894+0000","last_undegraded":"2024-04-25T09:03:32.831894+0000","last_fullsized":"2024-04-25T09:03:32.831894+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:44.265870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.826409+0000","last_change":"2024-04-25T09:03:32.817108+0000","last_active":"2024-04-25T09:03:32.826409+0000","last_peered":"2024-04-25T09:03:32.826409+0000","last_clean":"2024-04-25T09:03:32.826409+0000","last_became_active":"2024-04-25T09:03:32.816312+0000","last_became_peered":"2024-04-25T09:03:32.816312+0000","last_unstale":"2024-04-25T09:03:32.826409+0000","last_undegraded":"2024-04-25T09:03:32.826409+0000","last_fullsized":"2024-04-25T09:03:32.826409+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:35.087962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.816684+0000","last_change":"2024-04-25T09:03:32.807820+0000","last_active":"2024-04-25T09:03:32.816684+0000","last_peered":"2024-04-25T09:03:32.816684+0000","last_clean":"2024-04-25T09:03:32.816684+0000","last_became_active":"2024-04-25T09:03:32.807045+0000","last_became_peered":"2024-04-25T09:03:32.807045+0000","last_unstale":"2024-04-25T09:03:32.816684+0000","last_undegraded":"2024-04-25T09:03:32.816684+0000","last_fullsized":"2024-04-25T09:03:32.816684+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:38.673985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.835810+0000","last_change":"2024-04-25T09:03:32.834896+0000","last_active":"2024-04-25T09:03:32.835810+0000","last_peered":"2024-04-25T09:03:32.835810+0000","last_clean":"2024-04-25T09:03:32.835810+0000","last_became_active":"2024-04-25T09:03:32.833709+0000","last_became_peered":"2024-04-25T09:03:32.833709+0000","last_unstale":"2024-04-25T09:03:32.835810+0000","last_undegraded":"2024-04-25T09:03:32.835810+0000","last_fullsized":"2024-04-25T09:03:32.835810+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:24.835744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831774+0000","last_change":"2024-04-25T09:03:32.826063+0000","last_active":"2024-04-25T09:03:32.831774+0000","last_peered":"2024-04-25T09:03:32.831774+0000","last_clean":"2024-04-25T09:03:32.831774+0000","last_became_active":"2024-04-25T09:03:32.825276+0000","last_became_peered":"2024-04-25T09:03:32.825276+0000","last_unstale":"2024-04-25T09:03:32.831774+0000","last_undegraded":"2024-04-25T09:03:32.831774+0000","last_fullsized":"2024-04-25T09:03:32.831774+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:18.118297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.827563+0000","last_change":"2024-04-25T09:03:32.825143+0000","last_active":"2024-04-25T09:03:32.827563+0000","last_peered":"2024-04-25T09:03:32.827563+0000","last_clean":"2024-04-25T09:03:32.827563+0000","last_became_active":"2024-04-25T09:03:32.824349+0000","last_became_peered":"2024-04-25T09:03:32.824349+0000","last_unstale":"2024-04-25T09:03:32.827563+0000","last_undegraded":"2024-04-25T09:03:32.827563+0000","last_fullsized":"2024-04-25T09:03:32.827563+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:35.562353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.815948+0000","last_change":"2024-04-25T09:03:32.811954+0000","last_active":"2024-04-25T09:03:32.815948+0000","last_peered":"2024-04-25T09:03:32.815948+0000","last_clean":"2024-04-25T09:03:32.815948+0000","last_became_active":"2024-04-25T09:03:32.810436+0000","last_became_peered":"2024-04-25T09:03:32.810436+0000","last_unstale":"2024-04-25T09:03:32.815948+0000","last_undegraded":"2024-04-25T09:03:32.815948+0000","last_fullsized":"2024-04-25T09:03:32.815948+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.764315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.825222+0000","last_change":"2024-04-25T09:03:32.821432+0000","last_active":"2024-04-25T09:03:32.825222+0000","last_peered":"2024-04-25T09:03:32.825222+0000","last_clean":"2024-04-25T09:03:32.825222+0000","last_became_active":"2024-04-25T09:03:32.818141+0000","last_became_peered":"2024-04-25T09:03:32.818141+0000","last_unstale":"2024-04-25T09:03:32.825222+0000","last_undegraded":"2024-04-25T09:03:32.825222+0000","last_fullsized":"2024-04-25T09:03:32.825222+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:35.562353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.839203+0000","last_change":"2024-04-25T09:03:32.836447+0000","last_active":"2024-04-25T09:03:32.839203+0000","last_peered":"2024-04-25T09:03:32.839203+0000","last_clean":"2024-04-25T09:03:32.839203+0000","last_became_active":"2024-04-25T09:03:32.835792+0000","last_became_peered":"2024-04-25T09:03:32.835792+0000","last_unstale":"2024-04-25T09:03:32.839203+0000","last_undegraded":"2024-04-25T09:03:32.839203+0000","last_fullsized":"2024-04-25T09:03:32.839203+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:53.171753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.824901+0000","last_change":"2024-04-25T09:03:32.818744+0000","last_active":"2024-04-25T09:03:32.824901+0000","last_peered":"2024-04-25T09:03:32.824901+0000","last_clean":"2024-04-25T09:03:32.824901+0000","last_became_active":"2024-04-25T09:03:32.817822+0000","last_became_peered":"2024-04-25T09:03:32.817822+0000","last_unstale":"2024-04-25T09:03:32.824901+0000","last_undegraded":"2024-04-25T09:03:32.824901+0000","last_fullsized":"2024-04-25T09:03:32.824901+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:32.238618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.829964+0000","last_change":"2024-04-25T09:03:32.819525+0000","last_active":"2024-04-25T09:03:32.829964+0000","last_peered":"2024-04-25T09:03:32.829964+0000","last_clean":"2024-04-25T09:03:32.829964+0000","last_became_active":"2024-04-25T09:03:32.818801+0000","last_became_peered":"2024-04-25T09:03:32.818801+0000","last_unstale":"2024-04-25T09:03:32.829964+0000","last_undegraded":"2024-04-25T09:03:32.829964+0000","last_fullsized":"2024-04-25T09:03:32.829964+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:05.258773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.829971+0000","last_change":"2024-04-25T09:03:32.815934+0000","last_active":"2024-04-25T09:03:32.829971+0000","last_peered":"2024-04-25T09:03:32.829971+0000","last_clean":"2024-04-25T09:03:32.829971+0000","last_became_active":"2024-04-25T09:03:32.815119+0000","last_became_peered":"2024-04-25T09:03:32.815119+0000","last_unstale":"2024-04-25T09:03:32.829971+0000","last_undegraded":"2024-04-25T09:03:32.829971+0000","last_fullsized":"2024-04-25T09:03:32.829971+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:52.306518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.805453+0000","last_change":"2024-04-25T09:03:32.799937+0000","last_active":"2024-04-25T09:03:32.805453+0000","last_peered":"2024-04-25T09:03:32.805453+0000","last_clean":"2024-04-25T09:03:32.805453+0000","last_became_active":"2024-04-25T09:03:32.799164+0000","last_became_peered":"2024-04-25T09:03:32.799164+0000","last_unstale":"2024-04-25T09:03:32.805453+0000","last_undegraded":"2024-04-25T09:03:32.805453+0000","last_fullsized":"2024-04-25T09:03:32.805453+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:53.561488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.805380+0000","last_change":"2024-04-25T09:03:32.801476+0000","last_active":"2024-04-25T09:03:32.805380+0000","last_peered":"2024-04-25T09:03:32.805380+0000","last_clean":"2024-04-25T09:03:32.805380+0000","last_became_active":"2024-04-25T09:03:32.799001+0000","last_became_peered":"2024-04-25T09:03:32.799001+0000","last_unstale":"2024-04-25T09:03:32.805380+0000","last_undegraded":"2024-04-25T09:03:32.805380+0000","last_fullsized":"2024-04-25T09:03:32.805380+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.420083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.820747+0000","last_change":"2024-04-25T09:03:32.816795+0000","last_active":"2024-04-25T09:03:32.820747+0000","last_peered":"2024-04-25T09:03:32.820747+0000","last_clean":"2024-04-25T09:03:32.820747+0000","last_became_active":"2024-04-25T09:03:32.816106+0000","last_became_peered":"2024-04-25T09:03:32.816106+0000","last_unstale":"2024-04-25T09:03:32.820747+0000","last_undegraded":"2024-04-25T09:03:32.820747+0000","last_fullsized":"2024-04-25T09:03:32.820747+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:59.022260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.810571+0000","last_change":"2024-04-25T09:03:32.805491+0000","last_active":"2024-04-25T09:03:32.810571+0000","last_peered":"2024-04-25T09:03:32.810571+0000","last_clean":"2024-04-25T09:03:32.810571+0000","last_became_active":"2024-04-25T09:03:32.803874+0000","last_became_peered":"2024-04-25T09:03:32.803874+0000","last_unstale":"2024-04-25T09:03:32.810571+0000","last_undegraded":"2024-04-25T09:03:32.810571+0000","last_fullsized":"2024-04-25T09:03:32.810571+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:43.761302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831945+0000","last_change":"2024-04-25T09:03:32.827018+0000","last_active":"2024-04-25T09:03:32.831945+0000","last_peered":"2024-04-25T09:03:32.831945+0000","last_clean":"2024-04-25T09:03:32.831945+0000","last_became_active":"2024-04-25T09:03:32.825977+0000","last_became_peered":"2024-04-25T09:03:32.825977+0000","last_unstale":"2024-04-25T09:03:32.831945+0000","last_undegraded":"2024-04-25T09:03:32.831945+0000","last_fullsized":"2024-04-25T09:03:32.831945+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:31.684282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.836489+0000","last_change":"2024-04-25T09:03:32.828507+0000","last_active":"2024-04-25T09:03:32.836489+0000","last_peered":"2024-04-25T09:03:32.836489+0000","last_clean":"2024-04-25T09:03:32.836489+0000","last_became_active":"2024-04-25T09:03:32.822088+0000","last_became_peered":"2024-04-25T09:03:32.822088+0000","last_unstale":"2024-04-25T09:03:32.836489+0000","last_undegraded":"2024-04-25T09:03:32.836489+0000","last_fullsized":"2024-04-25T09:03:32.836489+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:28.186515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.836620+0000","last_change":"2024-04-25T09:03:32.819181+0000","last_active":"2024-04-25T09:03:32.836620+0000","last_peered":"2024-04-25T09:03:32.836620+0000","last_clean":"2024-04-25T09:03:32.836620+0000","last_became_active":"2024-04-25T09:03:32.818211+0000","last_became_peered":"2024-04-25T09:03:32.818211+0000","last_unstale":"2024-04-25T09:03:32.836620+0000","last_undegraded":"2024-04-25T09:03:32.836620+0000","last_fullsized":"2024-04-25T09:03:32.836620+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:26.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.812033+0000","last_change":"2024-04-25T09:03:32.805909+0000","last_active":"2024-04-25T09:03:32.812033+0000","last_peered":"2024-04-25T09:03:32.812033+0000","last_clean":"2024-04-25T09:03:32.812033+0000","last_became_active":"2024-04-25T09:03:32.805113+0000","last_became_peered":"2024-04-25T09:03:32.805113+0000","last_unstale":"2024-04-25T09:03:32.812033+0000","last_undegraded":"2024-04-25T09:03:32.812033+0000","last_fullsized":"2024-04-25T09:03:32.812033+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:08.376454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"124'4","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.141561+0000","last_change":"2024-04-25T09:03:32.141549+0000","last_active":"2024-04-25T09:03:33.141561+0000","last_peered":"2024-04-25T09:03:33.141561+0000","last_clean":"2024-04-25T09:03:33.141561+0000","last_became_active":"2024-04-25T09:03:32.140817+0000","last_became_peered":"2024-04-25T09:03:32.140817+0000","last_unstale":"2024-04-25T09:03:33.141561+0000","last_undegraded":"2024-04-25T09:03:33.141561+0000","last_fullsized":"2024-04-25T09:03:33.141561+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:27.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.827350+0000","last_change":"2024-04-25T09:03:32.822879+0000","last_active":"2024-04-25T09:03:32.827350+0000","last_peered":"2024-04-25T09:03:32.827350+0000","last_clean":"2024-04-25T09:03:32.827350+0000","last_became_active":"2024-04-25T09:03:32.822263+0000","last_became_peered":"2024-04-25T09:03:32.822263+0000","last_unstale":"2024-04-25T09:03:32.827350+0000","last_undegraded":"2024-04-25T09:03:32.827350+0000","last_fullsized":"2024-04-25T09:03:32.827350+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:46.508470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.134225+0000","last_change":"2024-04-25T09:03:32.137716+0000","last_active":"2024-04-25T09:03:33.134225+0000","last_peered":"2024-04-25T09:03:33.134225+0000","last_clean":"2024-04-25T09:03:33.134225+0000","last_became_active":"2024-04-25T09:03:32.137271+0000","last_became_peered":"2024-04-25T09:03:32.137271+0000","last_unstale":"2024-04-25T09:03:33.134225+0000","last_undegraded":"2024-04-25T09:03:33.134225+0000","last_fullsized":"2024-04-25T09:03:33.134225+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:51.302679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.836290+0000","last_change":"2024-04-25T09:03:32.821025+0000","last_active":"2024-04-25T09:03:32.836290+0000","last_peered":"2024-04-25T09:03:32.836290+0000","last_clean":"2024-04-25T09:03:32.836290+0000","last_became_active":"2024-04-25T09:03:32.820293+0000","last_became_peered":"2024-04-25T09:03:32.820293+0000","last_unstale":"2024-04-25T09:03:32.836290+0000","last_undegraded":"2024-04-25T09:03:32.836290+0000","last_fullsized":"2024-04-25T09:03:32.836290+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:16.529254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.143866+0000","last_change":"2024-04-25T09:03:32.149094+0000","last_active":"2024-04-25T09:03:33.143866+0000","last_peered":"2024-04-25T09:03:33.143866+0000","last_clean":"2024-04-25T09:03:33.143866+0000","last_became_active":"2024-04-25T09:03:32.148337+0000","last_became_peered":"2024-04-25T09:03:32.148337+0000","last_unstale":"2024-04-25T09:03:33.143866+0000","last_undegraded":"2024-04-25T09:03:33.143866+0000","last_fullsized":"2024-04-25T09:03:33.143866+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:27.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.825001+0000","last_change":"2024-04-25T09:03:32.820614+0000","last_active":"2024-04-25T09:03:32.825001+0000","last_peered":"2024-04-25T09:03:32.825001+0000","last_clean":"2024-04-25T09:03:32.825001+0000","last_became_active":"2024-04-25T09:03:32.819217+0000","last_became_peered":"2024-04-25T09:03:32.819217+0000","last_unstale":"2024-04-25T09:03:32.825001+0000","last_undegraded":"2024-04-25T09:03:32.825001+0000","last_fullsized":"2024-04-25T09:03:32.825001+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.420083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.130411+0000","last_change":"2024-04-25T09:03:32.151168+0000","last_active":"2024-04-25T09:03:33.130411+0000","last_peered":"2024-04-25T09:03:33.130411+0000","last_clean":"2024-04-25T09:03:33.130411+0000","last_became_active":"2024-04-25T09:03:32.150755+0000","last_became_peered":"2024-04-25T09:03:32.150755+0000","last_unstale":"2024-04-25T09:03:33.130411+0000","last_undegraded":"2024-04-25T09:03:33.130411+0000","last_fullsized":"2024-04-25T09:03:33.130411+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:44.666207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.796701+0000","last_change":"2024-04-25T09:03:32.787090+0000","last_active":"2024-04-25T09:03:32.796701+0000","last_peered":"2024-04-25T09:03:32.796701+0000","last_clean":"2024-04-25T09:03:32.796701+0000","last_became_active":"2024-04-25T09:03:32.786254+0000","last_became_peered":"2024-04-25T09:03:32.786254+0000","last_unstale":"2024-04-25T09:03:32.796701+0000","last_undegraded":"2024-04-25T09:03:32.796701+0000","last_fullsized":"2024-04-25T09:03:32.796701+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:16.529254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.138923+0000","last_change":"2024-04-25T09:03:32.154970+0000","last_active":"2024-04-25T09:03:33.138923+0000","last_peered":"2024-04-25T09:03:33.138923+0000","last_clean":"2024-04-25T09:03:33.138923+0000","last_became_active":"2024-04-25T09:03:32.154321+0000","last_became_peered":"2024-04-25T09:03:32.154321+0000","last_unstale":"2024-04-25T09:03:33.138923+0000","last_undegraded":"2024-04-25T09:03:33.138923+0000","last_fullsized":"2024-04-25T09:03:33.138923+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:27.182676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.805280+0000","last_change":"2024-04-25T09:03:32.801022+0000","last_active":"2024-04-25T09:03:32.805280+0000","last_peered":"2024-04-25T09:03:32.805280+0000","last_clean":"2024-04-25T09:03:32.805280+0000","last_became_active":"2024-04-25T09:03:32.800313+0000","last_became_peered":"2024-04-25T09:03:32.800313+0000","last_unstale":"2024-04-25T09:03:32.805280+0000","last_undegraded":"2024-04-25T09:03:32.805280+0000","last_fullsized":"2024-04-25T09:03:32.805280+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.420083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.136819+0000","last_change":"2024-04-25T09:03:32.158494+0000","last_active":"2024-04-25T09:03:33.136819+0000","last_peered":"2024-04-25T09:03:33.136819+0000","last_clean":"2024-04-25T09:03:33.136819+0000","last_became_active":"2024-04-25T09:03:32.158211+0000","last_became_peered":"2024-04-25T09:03:32.158211+0000","last_unstale":"2024-04-25T09:03:33.136819+0000","last_undegraded":"2024-04-25T09:03:33.136819+0000","last_fullsized":"2024-04-25T09:03:33.136819+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:28.212478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831574+0000","last_change":"2024-04-25T09:03:32.825082+0000","last_active":"2024-04-25T09:03:32.831574+0000","last_peered":"2024-04-25T09:03:32.831574+0000","last_clean":"2024-04-25T09:03:32.831574+0000","last_became_active":"2024-04-25T09:03:32.821175+0000","last_became_peered":"2024-04-25T09:03:32.821175+0000","last_unstale":"2024-04-25T09:03:32.831574+0000","last_undegraded":"2024-04-25T09:03:32.831574+0000","last_fullsized":"2024-04-25T09:03:32.831574+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:36.605060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.801841+0000","last_change":"2024-04-25T09:03:32.149238+0000","last_active":"2024-04-25T09:03:32.801841+0000","last_peered":"2024-04-25T09:03:32.801841+0000","last_clean":"2024-04-25T09:03:32.801841+0000","last_became_active":"2024-04-25T09:03:32.148593+0000","last_became_peered":"2024-04-25T09:03:32.148593+0000","last_unstale":"2024-04-25T09:03:32.801841+0000","last_undegraded":"2024-04-25T09:03:32.801841+0000","last_fullsized":"2024-04-25T09:03:32.801841+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:09.161775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.808148+0000","last_change":"2024-04-25T09:03:32.803956+0000","last_active":"2024-04-25T09:03:32.808148+0000","last_peered":"2024-04-25T09:03:32.808148+0000","last_clean":"2024-04-25T09:03:32.808148+0000","last_became_active":"2024-04-25T09:03:32.798484+0000","last_became_peered":"2024-04-25T09:03:32.798484+0000","last_unstale":"2024-04-25T09:03:32.808148+0000","last_undegraded":"2024-04-25T09:03:32.808148+0000","last_fullsized":"2024-04-25T09:03:32.808148+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:09.004049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.790938+0000","last_change":"2024-04-25T09:03:32.144190+0000","last_active":"2024-04-25T09:03:32.790938+0000","last_peered":"2024-04-25T09:03:32.790938+0000","last_clean":"2024-04-25T09:03:32.790938+0000","last_became_active":"2024-04-25T09:03:32.143891+0000","last_became_peered":"2024-04-25T09:03:32.143891+0000","last_unstale":"2024-04-25T09:03:32.790938+0000","last_undegraded":"2024-04-25T09:03:32.790938+0000","last_fullsized":"2024-04-25T09:03:32.790938+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:22.612040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.809556+0000","last_change":"2024-04-25T09:03:32.804007+0000","last_active":"2024-04-25T09:03:32.809556+0000","last_peered":"2024-04-25T09:03:32.809556+0000","last_clean":"2024-04-25T09:03:32.809556+0000","last_became_active":"2024-04-25T09:03:32.803115+0000","last_became_peered":"2024-04-25T09:03:32.803115+0000","last_unstale":"2024-04-25T09:03:32.809556+0000","last_undegraded":"2024-04-25T09:03:32.809556+0000","last_fullsized":"2024-04-25T09:03:32.809556+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:24.835744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.782793+0000","last_change":"2024-04-25T09:03:32.149239+0000","last_active":"2024-04-25T09:03:32.782793+0000","last_peered":"2024-04-25T09:03:32.782793+0000","last_clean":"2024-04-25T09:03:32.782793+0000","last_became_active":"2024-04-25T09:03:32.147139+0000","last_became_peered":"2024-04-25T09:03:32.147139+0000","last_unstale":"2024-04-25T09:03:32.782793+0000","last_undegraded":"2024-04-25T09:03:32.782793+0000","last_fullsized":"2024-04-25T09:03:32.782793+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:07.372615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.837054+0000","last_change":"2024-04-25T09:03:32.829900+0000","last_active":"2024-04-25T09:03:32.837054+0000","last_peered":"2024-04-25T09:03:32.837054+0000","last_clean":"2024-04-25T09:03:32.837054+0000","last_became_active":"2024-04-25T09:03:32.829163+0000","last_became_peered":"2024-04-25T09:03:32.829163+0000","last_unstale":"2024-04-25T09:03:32.837054+0000","last_undegraded":"2024-04-25T09:03:32.837054+0000","last_fullsized":"2024-04-25T09:03:32.837054+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:32.238618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.812720+0000","last_change":"2024-04-25T09:03:32.149208+0000","last_active":"2024-04-25T09:03:32.812720+0000","last_peered":"2024-04-25T09:03:32.812720+0000","last_clean":"2024-04-25T09:03:32.812720+0000","last_became_active":"2024-04-25T09:03:32.148445+0000","last_became_peered":"2024-04-25T09:03:32.148445+0000","last_unstale":"2024-04-25T09:03:32.812720+0000","last_undegraded":"2024-04-25T09:03:32.812720+0000","last_fullsized":"2024-04-25T09:03:32.812720+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:44.666207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831466+0000","last_change":"2024-04-25T09:03:32.821118+0000","last_active":"2024-04-25T09:03:32.831466+0000","last_peered":"2024-04-25T09:03:32.831466+0000","last_clean":"2024-04-25T09:03:32.831466+0000","last_became_active":"2024-04-25T09:03:32.820285+0000","last_became_peered":"2024-04-25T09:03:32.820285+0000","last_unstale":"2024-04-25T09:03:32.831466+0000","last_undegraded":"2024-04-25T09:03:32.831466+0000","last_fullsized":"2024-04-25T09:03:32.831466+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:26.167402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"124'3","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.146856+0000","last_change":"2024-04-25T09:03:32.150827+0000","last_active":"2024-04-25T09:03:33.146856+0000","last_peered":"2024-04-25T09:03:33.146856+0000","last_clean":"2024-04-25T09:03:33.146856+0000","last_became_active":"2024-04-25T09:03:32.150130+0000","last_became_peered":"2024-04-25T09:03:32.150130+0000","last_unstale":"2024-04-25T09:03:33.146856+0000","last_undegraded":"2024-04-25T09:03:33.146856+0000","last_fullsized":"2024-04-25T09:03:33.146856+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:07.941668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.837292+0000","last_change":"2024-04-25T09:03:32.828654+0000","last_active":"2024-04-25T09:03:32.837292+0000","last_peered":"2024-04-25T09:03:32.837292+0000","last_clean":"2024-04-25T09:03:32.837292+0000","last_became_active":"2024-04-25T09:03:32.822245+0000","last_became_peered":"2024-04-25T09:03:32.822245+0000","last_unstale":"2024-04-25T09:03:32.837292+0000","last_undegraded":"2024-04-25T09:03:32.837292+0000","last_fullsized":"2024-04-25T09:03:32.837292+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:09.004049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"124'6","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.139708+0000","last_change":"2024-04-25T09:03:32.158665+0000","last_active":"2024-04-25T09:03:33.139708+0000","last_peered":"2024-04-25T09:03:33.139708+0000","last_clean":"2024-04-25T09:03:33.139708+0000","last_became_active":"2024-04-25T09:03:32.158224+0000","last_became_peered":"2024-04-25T09:03:32.158224+0000","last_unstale":"2024-04-25T09:03:33.139708+0000","last_undegraded":"2024-04-25T09:03:33.139708+0000","last_fullsized":"2024-04-25T09:03:33.139708+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:51.559954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.810123+0000","last_change":"2024-04-25T09:03:32.797523+0000","last_active":"2024-04-25T09:03:32.810123+0000","last_peered":"2024-04-25T09:03:32.810123+0000","last_clean":"2024-04-25T09:03:32.810123+0000","last_became_active":"2024-04-25T09:03:32.796425+0000","last_became_peered":"2024-04-25T09:03:32.796425+0000","last_unstale":"2024-04-25T09:03:32.810123+0000","last_undegraded":"2024-04-25T09:03:32.810123+0000","last_fullsized":"2024-04-25T09:03:32.810123+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:32.238618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.134904+0000","last_change":"2024-04-25T09:03:32.155066+0000","last_active":"2024-04-25T09:03:33.134904+0000","last_peered":"2024-04-25T09:03:33.134904+0000","last_clean":"2024-04-25T09:03:33.134904+0000","last_became_active":"2024-04-25T09:03:32.154618+0000","last_became_peered":"2024-04-25T09:03:32.154618+0000","last_unstale":"2024-04-25T09:03:33.134904+0000","last_undegraded":"2024-04-25T09:03:33.134904+0000","last_fullsized":"2024-04-25T09:03:33.134904+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:12:31.541020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.800891+0000","last_change":"2024-04-25T09:03:32.793813+0000","last_active":"2024-04-25T09:03:32.800891+0000","last_peered":"2024-04-25T09:03:32.800891+0000","last_clean":"2024-04-25T09:03:32.800891+0000","last_became_active":"2024-04-25T09:03:32.793069+0000","last_became_peered":"2024-04-25T09:03:32.793069+0000","last_unstale":"2024-04-25T09:03:32.800891+0000","last_undegraded":"2024-04-25T09:03:32.800891+0000","last_fullsized":"2024-04-25T09:03:32.800891+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:26.716220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.820734+0000","last_change":"2024-04-25T09:03:32.158152+0000","last_active":"2024-04-25T09:03:32.820734+0000","last_peered":"2024-04-25T09:03:32.820734+0000","last_clean":"2024-04-25T09:03:32.820734+0000","last_became_active":"2024-04-25T09:03:32.157678+0000","last_became_peered":"2024-04-25T09:03:32.157678+0000","last_unstale":"2024-04-25T09:03:32.820734+0000","last_undegraded":"2024-04-25T09:03:32.820734+0000","last_fullsized":"2024-04-25T09:03:32.820734+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:30:00.666009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.811689+0000","last_change":"2024-04-25T09:03:32.809552+0000","last_active":"2024-04-25T09:03:32.811689+0000","last_peered":"2024-04-25T09:03:32.811689+0000","last_clean":"2024-04-25T09:03:32.811689+0000","last_became_active":"2024-04-25T09:03:32.808822+0000","last_became_peered":"2024-04-25T09:03:32.808822+0000","last_unstale":"2024-04-25T09:03:32.811689+0000","last_undegraded":"2024-04-25T09:03:32.811689+0000","last_fullsized":"2024-04-25T09:03:32.811689+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:27.333634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.797512+0000","last_change":"2024-04-25T09:03:32.154024+0000","last_active":"2024-04-25T09:03:32.797512+0000","last_peered":"2024-04-25T09:03:32.797512+0000","last_clean":"2024-04-25T09:03:32.797512+0000","last_became_active":"2024-04-25T09:03:32.153707+0000","last_became_peered":"2024-04-25T09:03:32.153707+0000","last_unstale":"2024-04-25T09:03:32.797512+0000","last_undegraded":"2024-04-25T09:03:32.797512+0000","last_fullsized":"2024-04-25T09:03:32.797512+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:29.947679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.836394+0000","last_change":"2024-04-25T09:03:32.821277+0000","last_active":"2024-04-25T09:03:32.836394+0000","last_peered":"2024-04-25T09:03:32.836394+0000","last_clean":"2024-04-25T09:03:32.836394+0000","last_became_active":"2024-04-25T09:03:32.820192+0000","last_became_peered":"2024-04-25T09:03:32.820192+0000","last_unstale":"2024-04-25T09:03:32.836394+0000","last_undegraded":"2024-04-25T09:03:32.836394+0000","last_fullsized":"2024-04-25T09:03:32.836394+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:13.774476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.131404+0000","last_change":"2024-04-25T09:03:32.153853+0000","last_active":"2024-04-25T09:03:33.131404+0000","last_peered":"2024-04-25T09:03:33.131404+0000","last_clean":"2024-04-25T09:03:33.131404+0000","last_became_active":"2024-04-25T09:03:32.153076+0000","last_became_peered":"2024-04-25T09:03:32.153076+0000","last_unstale":"2024-04-25T09:03:33.131404+0000","last_undegraded":"2024-04-25T09:03:33.131404+0000","last_fullsized":"2024-04-25T09:03:33.131404+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.254934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.839179+0000","last_change":"2024-04-25T09:03:32.836323+0000","last_active":"2024-04-25T09:03:32.839179+0000","last_peered":"2024-04-25T09:03:32.839179+0000","last_clean":"2024-04-25T09:03:32.839179+0000","last_became_active":"2024-04-25T09:03:32.835704+0000","last_became_peered":"2024-04-25T09:03:32.835704+0000","last_unstale":"2024-04-25T09:03:32.839179+0000","last_undegraded":"2024-04-25T09:03:32.839179+0000","last_fullsized":"2024-04-25T09:03:32.839179+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:55.685027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.133260+0000","last_change":"2024-04-25T09:03:32.153467+0000","last_active":"2024-04-25T09:03:33.133260+0000","last_peered":"2024-04-25T09:03:33.133260+0000","last_clean":"2024-04-25T09:03:33.133260+0000","last_became_active":"2024-04-25T09:03:32.152138+0000","last_became_peered":"2024-04-25T09:03:32.152138+0000","last_unstale":"2024-04-25T09:03:33.133260+0000","last_undegraded":"2024-04-25T09:03:33.133260+0000","last_fullsized":"2024-04-25T09:03:33.133260+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:52.568160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.818924+0000","last_change":"2024-04-25T09:03:32.814236+0000","last_active":"2024-04-25T09:03:32.818924+0000","last_peered":"2024-04-25T09:03:32.818924+0000","last_clean":"2024-04-25T09:03:32.818924+0000","last_became_active":"2024-04-25T09:03:32.808997+0000","last_became_peered":"2024-04-25T09:03:32.808997+0000","last_unstale":"2024-04-25T09:03:32.818924+0000","last_undegraded":"2024-04-25T09:03:32.818924+0000","last_fullsized":"2024-04-25T09:03:32.818924+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:13.994789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"124'1","reported_seq":13,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.144111+0000","last_change":"2024-04-25T09:03:32.162761+0000","last_active":"2024-04-25T09:03:33.144111+0000","last_peered":"2024-04-25T09:03:33.144111+0000","last_clean":"2024-04-25T09:03:33.144111+0000","last_became_active":"2024-04-25T09:03:32.162177+0000","last_became_peered":"2024-04-25T09:03:32.162177+0000","last_unstale":"2024-04-25T09:03:33.144111+0000","last_undegraded":"2024-04-25T09:03:33.144111+0000","last_fullsized":"2024-04-25T09:03:33.144111+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:08.000210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.838930+0000","last_change":"2024-04-25T09:03:32.836311+0000","last_active":"2024-04-25T09:03:32.838930+0000","last_peered":"2024-04-25T09:03:32.838930+0000","last_clean":"2024-04-25T09:03:32.838930+0000","last_became_active":"2024-04-25T09:03:32.831215+0000","last_became_peered":"2024-04-25T09:03:32.831215+0000","last_unstale":"2024-04-25T09:03:32.838930+0000","last_undegraded":"2024-04-25T09:03:32.838930+0000","last_fullsized":"2024-04-25T09:03:32.838930+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.420083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.797305+0000","last_change":"2024-04-25T09:03:32.158850+0000","last_active":"2024-04-25T09:03:32.797305+0000","last_peered":"2024-04-25T09:03:32.797305+0000","last_clean":"2024-04-25T09:03:32.797305+0000","last_became_active":"2024-04-25T09:03:32.158190+0000","last_became_peered":"2024-04-25T09:03:32.158190+0000","last_unstale":"2024-04-25T09:03:32.797305+0000","last_undegraded":"2024-04-25T09:03:32.797305+0000","last_fullsized":"2024-04-25T09:03:32.797305+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:12.770637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.818469+0000","last_change":"2024-04-25T09:03:32.813019+0000","last_active":"2024-04-25T09:03:32.818469+0000","last_peered":"2024-04-25T09:03:32.818469+0000","last_clean":"2024-04-25T09:03:32.818469+0000","last_became_active":"2024-04-25T09:03:32.808886+0000","last_became_peered":"2024-04-25T09:03:32.808886+0000","last_unstale":"2024-04-25T09:03:32.818469+0000","last_undegraded":"2024-04-25T09:03:32.818469+0000","last_fullsized":"2024-04-25T09:03:32.818469+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:18.896389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.140666+0000","last_change":"2024-04-25T09:03:32.148444+0000","last_active":"2024-04-25T09:03:33.140666+0000","last_peered":"2024-04-25T09:03:33.140666+0000","last_clean":"2024-04-25T09:03:33.140666+0000","last_became_active":"2024-04-25T09:03:32.147856+0000","last_became_peered":"2024-04-25T09:03:32.147856+0000","last_unstale":"2024-04-25T09:03:33.140666+0000","last_undegraded":"2024-04-25T09:03:33.140666+0000","last_fullsized":"2024-04-25T09:03:33.140666+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:07.928541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831814+0000","last_change":"2024-04-25T09:03:32.826708+0000","last_active":"2024-04-25T09:03:32.831814+0000","last_peered":"2024-04-25T09:03:32.831814+0000","last_clean":"2024-04-25T09:03:32.831814+0000","last_became_active":"2024-04-25T09:03:32.825927+0000","last_became_peered":"2024-04-25T09:03:32.825927+0000","last_unstale":"2024-04-25T09:03:32.831814+0000","last_undegraded":"2024-04-25T09:03:32.831814+0000","last_fullsized":"2024-04-25T09:03:32.831814+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:25.603713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.795516+0000","last_change":"2024-04-25T09:03:32.163640+0000","last_active":"2024-04-25T09:03:32.795516+0000","last_peered":"2024-04-25T09:03:32.795516+0000","last_clean":"2024-04-25T09:03:32.795516+0000","last_became_active":"2024-04-25T09:03:32.162907+0000","last_became_peered":"2024-04-25T09:03:32.162907+0000","last_unstale":"2024-04-25T09:03:32.795516+0000","last_undegraded":"2024-04-25T09:03:32.795516+0000","last_fullsized":"2024-04-25T09:03:32.795516+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.254934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.831643+0000","last_change":"2024-04-25T09:03:32.825769+0000","last_active":"2024-04-25T09:03:32.831643+0000","last_peered":"2024-04-25T09:03:32.831643+0000","last_clean":"2024-04-25T09:03:32.831643+0000","last_became_active":"2024-04-25T09:03:32.821572+0000","last_became_peered":"2024-04-25T09:03:32.821572+0000","last_unstale":"2024-04-25T09:03:32.831643+0000","last_undegraded":"2024-04-25T09:03:32.831643+0000","last_fullsized":"2024-04-25T09:03:32.831643+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:07.424424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.797939+0000","last_change":"2024-04-25T09:03:32.153220+0000","last_active":"2024-04-25T09:03:32.797939+0000","last_peered":"2024-04-25T09:03:32.797939+0000","last_clean":"2024-04-25T09:03:32.797939+0000","last_became_active":"2024-04-25T09:03:32.152800+0000","last_became_peered":"2024-04-25T09:03:32.152800+0000","last_unstale":"2024-04-25T09:03:32.797939+0000","last_undegraded":"2024-04-25T09:03:32.797939+0000","last_fullsized":"2024-04-25T09:03:32.797939+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:55.846258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.837066+0000","last_change":"2024-04-25T09:03:32.829572+0000","last_active":"2024-04-25T09:03:32.837066+0000","last_peered":"2024-04-25T09:03:32.837066+0000","last_clean":"2024-04-25T09:03:32.837066+0000","last_became_active":"2024-04-25T09:03:32.828867+0000","last_became_peered":"2024-04-25T09:03:32.828867+0000","last_unstale":"2024-04-25T09:03:32.837066+0000","last_undegraded":"2024-04-25T09:03:32.837066+0000","last_fullsized":"2024-04-25T09:03:32.837066+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:08.376454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.795917+0000","last_change":"2024-04-25T09:03:32.152796+0000","last_active":"2024-04-25T09:03:32.795917+0000","last_peered":"2024-04-25T09:03:32.795917+0000","last_clean":"2024-04-25T09:03:32.795917+0000","last_became_active":"2024-04-25T09:03:32.151086+0000","last_became_peered":"2024-04-25T09:03:32.151086+0000","last_unstale":"2024-04-25T09:03:32.795917+0000","last_undegraded":"2024-04-25T09:03:32.795917+0000","last_fullsized":"2024-04-25T09:03:32.795917+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:15.525415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.815578+0000","last_change":"2024-04-25T09:03:32.808888+0000","last_active":"2024-04-25T09:03:32.815578+0000","last_peered":"2024-04-25T09:03:32.815578+0000","last_clean":"2024-04-25T09:03:32.815578+0000","last_became_active":"2024-04-25T09:03:32.807200+0000","last_became_peered":"2024-04-25T09:03:32.807200+0000","last_unstale":"2024-04-25T09:03:32.815578+0000","last_undegraded":"2024-04-25T09:03:32.815578+0000","last_fullsized":"2024-04-25T09:03:32.815578+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:05.062073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"124'2","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.138086+0000","last_change":"2024-04-25T09:03:32.150818+0000","last_active":"2024-04-25T09:03:33.138086+0000","last_peered":"2024-04-25T09:03:33.138086+0000","last_clean":"2024-04-25T09:03:33.138086+0000","last_became_active":"2024-04-25T09:03:32.150538+0000","last_became_peered":"2024-04-25T09:03:32.150538+0000","last_unstale":"2024-04-25T09:03:33.138086+0000","last_undegraded":"2024-04-25T09:03:33.138086+0000","last_fullsized":"2024-04-25T09:03:33.138086+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:23.585002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.802005+0000","last_change":"2024-04-25T09:03:32.789111+0000","last_active":"2024-04-25T09:03:32.802005+0000","last_peered":"2024-04-25T09:03:32.802005+0000","last_clean":"2024-04-25T09:03:32.802005+0000","last_became_active":"2024-04-25T09:03:32.788355+0000","last_became_peered":"2024-04-25T09:03:32.788355+0000","last_unstale":"2024-04-25T09:03:32.802005+0000","last_undegraded":"2024-04-25T09:03:32.802005+0000","last_fullsized":"2024-04-25T09:03:32.802005+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:13.774476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.780476+0000","last_change":"2024-04-25T09:03:32.149607+0000","last_active":"2024-04-25T09:03:32.780476+0000","last_peered":"2024-04-25T09:03:32.780476+0000","last_clean":"2024-04-25T09:03:32.780476+0000","last_became_active":"2024-04-25T09:03:32.148774+0000","last_became_peered":"2024-04-25T09:03:32.148774+0000","last_unstale":"2024-04-25T09:03:32.780476+0000","last_undegraded":"2024-04-25T09:03:32.780476+0000","last_fullsized":"2024-04-25T09:03:32.780476+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:25.712381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.828060+0000","last_change":"2024-04-25T09:03:32.823047+0000","last_active":"2024-04-25T09:03:32.828060+0000","last_peered":"2024-04-25T09:03:32.828060+0000","last_clean":"2024-04-25T09:03:32.828060+0000","last_became_active":"2024-04-25T09:03:32.820771+0000","last_became_peered":"2024-04-25T09:03:32.820771+0000","last_unstale":"2024-04-25T09:03:32.828060+0000","last_undegraded":"2024-04-25T09:03:32.828060+0000","last_fullsized":"2024-04-25T09:03:32.828060+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:55.685027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.782776+0000","last_change":"2024-04-25T09:03:32.137406+0000","last_active":"2024-04-25T09:03:32.782776+0000","last_peered":"2024-04-25T09:03:32.782776+0000","last_clean":"2024-04-25T09:03:32.782776+0000","last_became_active":"2024-04-25T09:03:32.136585+0000","last_became_peered":"2024-04-25T09:03:32.136585+0000","last_unstale":"2024-04-25T09:03:32.782776+0000","last_undegraded":"2024-04-25T09:03:32.782776+0000","last_fullsized":"2024-04-25T09:03:32.782776+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:08.000210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.835594+0000","last_change":"2024-04-25T09:03:32.834222+0000","last_active":"2024-04-25T09:03:32.835594+0000","last_peered":"2024-04-25T09:03:32.835594+0000","last_clean":"2024-04-25T09:03:32.835594+0000","last_became_active":"2024-04-25T09:03:32.833490+0000","last_became_peered":"2024-04-25T09:03:32.833490+0000","last_unstale":"2024-04-25T09:03:32.835594+0000","last_undegraded":"2024-04-25T09:03:32.835594+0000","last_fullsized":"2024-04-25T09:03:32.835594+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:36.156183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.792523+0000","last_change":"2024-04-25T09:03:32.151390+0000","last_active":"2024-04-25T09:03:32.792523+0000","last_peered":"2024-04-25T09:03:32.792523+0000","last_clean":"2024-04-25T09:03:32.792523+0000","last_became_active":"2024-04-25T09:03:32.149110+0000","last_became_peered":"2024-04-25T09:03:32.149110+0000","last_unstale":"2024-04-25T09:03:32.792523+0000","last_undegraded":"2024-04-25T09:03:32.792523+0000","last_fullsized":"2024-04-25T09:03:32.792523+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:57.582279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.816732+0000","last_change":"2024-04-25T09:03:32.808003+0000","last_active":"2024-04-25T09:03:32.816732+0000","last_peered":"2024-04-25T09:03:32.816732+0000","last_clean":"2024-04-25T09:03:32.816732+0000","last_became_active":"2024-04-25T09:03:32.807272+0000","last_became_peered":"2024-04-25T09:03:32.807272+0000","last_unstale":"2024-04-25T09:03:32.816732+0000","last_undegraded":"2024-04-25T09:03:32.816732+0000","last_fullsized":"2024-04-25T09:03:32.816732+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:56.861333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.783352+0000","last_change":"2024-04-25T09:03:32.137822+0000","last_active":"2024-04-25T09:03:32.783352+0000","last_peered":"2024-04-25T09:03:32.783352+0000","last_clean":"2024-04-25T09:03:32.783352+0000","last_became_active":"2024-04-25T09:03:32.137026+0000","last_became_peered":"2024-04-25T09:03:32.137026+0000","last_unstale":"2024-04-25T09:03:32.783352+0000","last_undegraded":"2024-04-25T09:03:32.783352+0000","last_fullsized":"2024-04-25T09:03:32.783352+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:08.000210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.798830+0000","last_change":"2024-04-25T09:03:32.795561+0000","last_active":"2024-04-25T09:03:32.798830+0000","last_peered":"2024-04-25T09:03:32.798830+0000","last_clean":"2024-04-25T09:03:32.798830+0000","last_became_active":"2024-04-25T09:03:32.793594+0000","last_became_peered":"2024-04-25T09:03:32.793594+0000","last_unstale":"2024-04-25T09:03:32.798830+0000","last_undegraded":"2024-04-25T09:03:32.798830+0000","last_fullsized":"2024-04-25T09:03:32.798830+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:39.750952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.785023+0000","last_change":"2024-04-25T09:03:32.154303+0000","last_active":"2024-04-25T09:03:32.785023+0000","last_peered":"2024-04-25T09:03:32.785023+0000","last_clean":"2024-04-25T09:03:32.785023+0000","last_became_active":"2024-04-25T09:03:32.153400+0000","last_became_peered":"2024-04-25T09:03:32.153400+0000","last_unstale":"2024-04-25T09:03:32.785023+0000","last_undegraded":"2024-04-25T09:03:32.785023+0000","last_fullsized":"2024-04-25T09:03:32.785023+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:07.372615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.824876+0000","last_change":"2024-04-25T09:03:32.818442+0000","last_active":"2024-04-25T09:03:32.824876+0000","last_peered":"2024-04-25T09:03:32.824876+0000","last_clean":"2024-04-25T09:03:32.824876+0000","last_became_active":"2024-04-25T09:03:32.817653+0000","last_became_peered":"2024-04-25T09:03:32.817653+0000","last_unstale":"2024-04-25T09:03:32.824876+0000","last_undegraded":"2024-04-25T09:03:32.824876+0000","last_fullsized":"2024-04-25T09:03:32.824876+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:08.376454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.791091+0000","last_change":"2024-04-25T09:03:32.150314+0000","last_active":"2024-04-25T09:03:32.791091+0000","last_peered":"2024-04-25T09:03:32.791091+0000","last_clean":"2024-04-25T09:03:32.791091+0000","last_became_active":"2024-04-25T09:03:32.148425+0000","last_became_peered":"2024-04-25T09:03:32.148425+0000","last_unstale":"2024-04-25T09:03:32.791091+0000","last_undegraded":"2024-04-25T09:03:32.791091+0000","last_fullsized":"2024-04-25T09:03:32.791091+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:13.576886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.808092+0000","last_change":"2024-04-25T09:03:32.803221+0000","last_active":"2024-04-25T09:03:32.808092+0000","last_peered":"2024-04-25T09:03:32.808092+0000","last_clean":"2024-04-25T09:03:32.808092+0000","last_became_active":"2024-04-25T09:03:32.802048+0000","last_became_peered":"2024-04-25T09:03:32.802048+0000","last_unstale":"2024-04-25T09:03:32.808092+0000","last_undegraded":"2024-04-25T09:03:32.808092+0000","last_fullsized":"2024-04-25T09:03:32.808092+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:28.186515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.804515+0000","last_change":"2024-04-25T09:03:32.153503+0000","last_active":"2024-04-25T09:03:32.804515+0000","last_peered":"2024-04-25T09:03:32.804515+0000","last_clean":"2024-04-25T09:03:32.804515+0000","last_became_active":"2024-04-25T09:03:32.152057+0000","last_became_peered":"2024-04-25T09:03:32.152057+0000","last_unstale":"2024-04-25T09:03:32.804515+0000","last_undegraded":"2024-04-25T09:03:32.804515+0000","last_fullsized":"2024-04-25T09:03:32.804515+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:25.712381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.825162+0000","last_change":"2024-04-25T09:03:32.820828+0000","last_active":"2024-04-25T09:03:32.825162+0000","last_peered":"2024-04-25T09:03:32.825162+0000","last_clean":"2024-04-25T09:03:32.825162+0000","last_became_active":"2024-04-25T09:03:32.818733+0000","last_became_peered":"2024-04-25T09:03:32.818733+0000","last_unstale":"2024-04-25T09:03:32.825162+0000","last_undegraded":"2024-04-25T09:03:32.825162+0000","last_fullsized":"2024-04-25T09:03:32.825162+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:32.104061+0000","last_clean_scrub_stamp":"2024-04-25T09:03:32.104061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:24.835744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"124'8","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:33.137091+0000","last_change":"2024-04-25T09:03:32.144778+0000","last_active":"2024-04-25T09:03:33.137091+0000","last_peered":"2024-04-25T09:03:33.137091+0000","last_clean":"2024-04-25T09:03:33.137091+0000","last_became_active":"2024-04-25T09:03:32.144394+0000","last_became_peered":"2024-04-25T09:03:32.144394+0000","last_unstale":"2024-04-25T09:03:33.137091+0000","last_undegraded":"2024-04-25T09:03:33.137091+0000","last_fullsized":"2024-04-25T09:03:33.137091+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:31.100222+0000","last_clean_scrub_stamp":"2024-04-25T09:03:31.100222+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:35.877384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":365,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T09:03:32.804265+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T09:03:32.804265+0000","last_peered":"2024-04-25T09:03:32.804265+0000","last_clean":"2024-04-25T09:03:32.804265+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T09:03:32.804265+0000","last_undegraded":"2024-04-25T09:03:32.804265+0000","last_fullsized":"2024-04-25T09:03:32.804265+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574986,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2125824,"data_stored":1306656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574987,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1548288,"data_stored":724037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574987,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1544192,"data_stored":723455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574986,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1556480,"data_stored":725067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574986,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32692,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339148,"statfs":{"total":96636764160,"available":96603287552,"internally_reserved":0,"allocated":1544192,"data_stored":723455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32688,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339152,"statfs":{"total":96636764160,"available":96603291648,"internally_reserved":0,"allocated":1544192,"data_stored":723455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574987,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32684,"kb_used_data":1496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339156,"statfs":{"total":96636764160,"available":96603295744,"internally_reserved":0,"allocated":1531904,"data_stored":716288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574988,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2138112,"data_stored":1313823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:37.531 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:03:37.531 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:03:37.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.713+0000 7fc059f4c640 1 Processor -- start 2024-04-25T09:03:37.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.713+0000 7fc059f4c640 1 -- start start 2024-04-25T09:03:37.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.713+0000 7fc059f4c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc0540d3df0 0x7fc054168590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.713+0000 7fc059f4c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0540d1950 con 0x7fc05415d9c0 2024-04-25T09:03:37.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.713+0000 7fc059f4c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0540d1bf0 con 0x7fc0540d1320 2024-04-25T09:03:37.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.713+0000 7fc059f4c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0540d1720 con 0x7fc0540d3df0 2024-04-25T09:03:37.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.713+0000 7fc0537fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc05415d9c0 0x7fc0540cf270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38060/0 (socket says 172.21.15.37:38060) 2024-04-25T09:03:37.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.713+0000 7fc0537fe640 1 -- 172.21.15.37:0/3559698108 learned_addr learned my addr 172.21.15.37:0/3559698108 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:37.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.713+0000 7fc053fff640 1 --2- 172.21.15.37:0/3559698108 >> v2:172.21.15.203:3300/0 conn(0x7fc0540d3df0 0x7fc054168590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.713+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483398933 0 0) 0x7fc0540d1950 con 0x7fc05415d9c0 2024-04-25T09:03:37.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.713+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc040003530 con 0x7fc05415d9c0 2024-04-25T09:03:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 672637647 0 0) 0x7fc040003530 con 0x7fc05415d9c0 2024-04-25T09:03:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 >> v2:172.21.15.203:3300/0 conn(0x7fc0540d3df0 msgr2=0x7fc054168590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 --2- 172.21.15.37:0/3559698108 >> v2:172.21.15.203:3300/0 conn(0x7fc0540d3df0 0x7fc054168590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 >> v1:172.21.15.203:6789/0 conn(0x7fc0540d1320 legacy=0x7fc0540d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0540d1ed0 con 0x7fc05415d9c0 2024-04-25T09:03:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fc03c0016a0 con 0x7fc05415d9c0 2024-04-25T09:03:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 >> v1:172.21.15.37:6789/0 conn(0x7fc05415d9c0 legacy=0x7fc0540cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc040003a60 0x7fc040003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 0x7fc040004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0400051d0 0x7fc04000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04000c140 con 0x7fc040004810 2024-04-25T09:03:37.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04000c2f0 con 0x7fc0400051d0 2024-04-25T09:03:37.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc04000c4f0 con 0x7fc040003a60 2024-04-25T09:03:37.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc052ffd640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 0x7fc040004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0537fe640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc040003a60 0x7fc040003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc053fff640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0400051d0 0x7fc04000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc052ffd640 1 -- 172.21.15.37:0/3559698108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc040003a60 msgr2=0x7fc040003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:37.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc052ffd640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc040003a60 0x7fc040003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc052ffd640 1 -- 172.21.15.37:0/3559698108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0400051d0 msgr2=0x7fc04000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc052ffd640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0400051d0 0x7fc04000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc053fff640 1 -- 172.21.15.37:0/3559698108 reap_dead start 2024-04-25T09:03:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc052ffd640 1 -- 172.21.15.37:0/3559698108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc040003530 con 0x7fc040004810 2024-04-25T09:03:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0537fe640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc040003a60 0x7fc040003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc052ffd640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 0x7fc040004c90 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fc0440017b0 tx=0x7fc04401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bd03669763c96c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0440013d0 con 0x7fc040004810 2024-04-25T09:03:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0440052d0 con 0x7fc040004810 2024-04-25T09:03:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc0527fc640 1 -- 172.21.15.37:0/3559698108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c001d50 con 0x7fc040004810 2024-04-25T09:03:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc059f4c640 1 -- 172.21.15.37:0/3559698108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 msgr2=0x7fc040004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc059f4c640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 0x7fc040004c90 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fc0440017b0 tx=0x7fc04401c3e0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc059f4c640 1 -- 172.21.15.37:0/3559698108 shutdown_connections 2024-04-25T09:03:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc059f4c640 1 --2- 172.21.15.37:0/3559698108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 0x7fc040004c90 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc059f4c640 1 -- 172.21.15.37:0/3559698108 >> 172.21.15.37:0/3559698108 conn(0x7fc05409cad0 msgr2=0x7fc05408b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc059f4c640 1 -- 172.21.15.37:0/3559698108 shutdown_connections 2024-04-25T09:03:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.717+0000 7fc059f4c640 1 -- 172.21.15.37:0/3559698108 wait complete. 2024-04-25T09:03:37.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc059f4c640 1 Processor -- start 2024-04-25T09:03:37.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc059f4c640 1 -- start start 2024-04-25T09:03:37.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc059f4c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 0x7fc054159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc059f4c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc05415cb90 0x7fc0540b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc059f4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0540c9430 0x7fc0540c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc059f4c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05415dda0 con 0x7fc040004810 2024-04-25T09:03:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc059f4c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0540d2520 con 0x7fc05415cb90 2024-04-25T09:03:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc059f4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0540d36e0 con 0x7fc0540c9430 2024-04-25T09:03:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc0537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 0x7fc054159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc0537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 0x7fc054159ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51058/0 (socket says 172.21.15.37:51058) 2024-04-25T09:03:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc0537fe640 1 -- 172.21.15.37:0/4021622001 learned_addr learned my addr 172.21.15.37:0/4021622001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc053fff640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0540c9430 0x7fc0540c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc052ffd640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc05415cb90 0x7fc0540b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc0537fe640 1 -- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0540c9430 msgr2=0x7fc0540c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc0537fe640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0540c9430 0x7fc0540c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc0537fe640 1 -- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc05415cb90 msgr2=0x7fc0540b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc0537fe640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc05415cb90 0x7fc0540b6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc0537fe640 1 -- 172.21.15.37:0/4021622001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0540b7240 con 0x7fc040004810 2024-04-25T09:03:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc0537fe640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 0x7fc054159ef0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0107b0 tx=0x7fc03c010c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc04bfff640 1 -- 172.21.15.37:0/4021622001 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00e8f0 con 0x7fc040004810 2024-04-25T09:03:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc04bfff640 1 -- 172.21.15.37:0/4021622001 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc03c011040 con 0x7fc040004810 2024-04-25T09:03:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc059f4c640 1 -- 172.21.15.37:0/4021622001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0540b6c80 con 0x7fc040004810 2024-04-25T09:03:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc04bfff640 1 -- 172.21.15.37:0/4021622001 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03c00ecb0 con 0x7fc040004810 2024-04-25T09:03:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc059f4c640 1 -- 172.21.15.37:0/4021622001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0540b79a0 con 0x7fc040004810 2024-04-25T09:03:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.721+0000 7fc059f4c640 1 -- 172.21.15.37:0/4021622001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05404efa0 con 0x7fc040004810 2024-04-25T09:03:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.725+0000 7fc04bfff640 1 -- 172.21.15.37:0/4021622001 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc03c01d020 con 0x7fc040004810 2024-04-25T09:03:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.725+0000 7fc04bfff640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc034066f60 0x7fc034069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.725+0000 7fc04bfff640 1 -- 172.21.15.37:0/4021622001 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc03c087e10 con 0x7fc040004810 2024-04-25T09:03:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.725+0000 7fc052ffd640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc034066f60 0x7fc034069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.725+0000 7fc052ffd640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc034066f60 0x7fc034069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc0440ac000 tx=0x7fc0440011f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.729+0000 7fc04bfff640 1 -- 172.21.15.37:0/4021622001 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc03c058c00 con 0x7fc040004810 2024-04-25T09:03:37.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.941+0000 7fc059f4c640 1 -- 172.21.15.37:0/4021622001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc05406d2f0 con 0x7fc040004810 2024-04-25T09:03:37.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc04bfff640 1 -- 172.21.15.37:0/4021622001 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fc03c058500 con 0x7fc040004810 2024-04-25T09:03:37.933 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-25T09:03:37.933 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCPGSpmtET7HRAAoSF0YWVDy6Y/5iYb+82wgA== 2024-04-25T09:03:37.933 INFO:teuthology.orchestra.run.smithi037.stdout: caps mds = "allow" 2024-04-25T09:03:37.933 INFO:teuthology.orchestra.run.smithi037.stdout: caps mon = "allow r" 2024-04-25T09:03:37.933 INFO:teuthology.orchestra.run.smithi037.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:03:37.933 INFO:teuthology.orchestra.run.smithi037.stderr:updated caps for client.0 2024-04-25T09:03:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 -- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc034066f60 msgr2=0x7fc034069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc034066f60 0x7fc034069420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc0440ac000 tx=0x7fc0440011f0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 -- 172.21.15.37:0/4021622001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 msgr2=0x7fc054159ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 0x7fc054159ef0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fc03c0107b0 tx=0x7fc03c010c70 comp rx=0 tx=0).stop 2024-04-25T09:03:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 -- 172.21.15.37:0/4021622001 shutdown_connections 2024-04-25T09:03:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc034066f60 0x7fc034069420 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0540c9430 0x7fc0540c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc05415cb90 0x7fc0540b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 --2- 172.21.15.37:0/4021622001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc040004810 0x7fc054159ef0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:37.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 -- 172.21.15.37:0/4021622001 >> 172.21.15.37:0/4021622001 conn(0x7fc05409cad0 msgr2=0x7fc05415ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:37.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 -- 172.21.15.37:0/4021622001 shutdown_connections 2024-04-25T09:03:37.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:37.945+0000 7fc059f4c640 1 -- 172.21.15.37:0/4021622001 wait complete. 2024-04-25T09:03:37.951 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:38.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.085+0000 7fb4bb6a9640 1 Processor -- start 2024-04-25T09:03:38.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.085+0000 7fb4bb6a9640 1 -- start start 2024-04-25T09:03:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b4099a70 0x7fb4b40cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb4b4099e50 0x7fb4b406c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b40d0050 con 0x7fb4b4161510 2024-04-25T09:03:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b40cfe20 con 0x7fb4b4099e50 2024-04-25T09:03:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b40d0330 con 0x7fb4b4099a70 2024-04-25T09:03:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4b8c44640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb4b4099e50 0x7fb4b406c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4b8c44640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb4b4099e50 0x7fb4b406c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51068/0 (socket says 172.21.15.37:51068) 2024-04-25T09:03:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4b8c44640 1 -- 172.21.15.37:0/1672596866 learned_addr learned my addr 172.21.15.37:0/1672596866 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4b9c46640 1 --1- 172.21.15.37:0/1672596866 >> v1:172.21.15.37:6789/0 conn(0x7fb4b4161510 0x7fb4b41619a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:38062/0 (socket says 172.21.15.37:38062) 2024-04-25T09:03:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4b8c44640 1 -- 172.21.15.37:0/1672596866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b4099a70 msgr2=0x7fb4b40cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4b8c44640 1 --2- 172.21.15.37:0/1672596866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b4099a70 0x7fb4b40cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4b8c44640 1 -- 172.21.15.37:0/1672596866 >> v1:172.21.15.37:6789/0 conn(0x7fb4b4161510 legacy=0x7fb4b41619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4b8c44640 1 -- 172.21.15.37:0/1672596866 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b40d0610 con 0x7fb4b4099e50 2024-04-25T09:03:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4b8c44640 1 --2- 172.21.15.37:0/1672596866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b4099e50 0x7fb4b406c9e0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009870 tx=0x7fb4a401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbe27c6bd55d7da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4abfff640 1 -- 172.21.15.37:0/1672596866 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40ab650 con 0x7fb4b4099e50 2024-04-25T09:03:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4abfff640 1 -- 172.21.15.37:0/1672596866 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a40b0070 con 0x7fb4b4099e50 2024-04-25T09:03:38.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4abfff640 1 -- 172.21.15.37:0/1672596866 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40aba10 con 0x7fb4b4099e50 2024-04-25T09:03:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/1672596866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b4099e50 msgr2=0x7fb4b406c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 --2- 172.21.15.37:0/1672596866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b4099e50 0x7fb4b406c9e0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009870 tx=0x7fb4a401c560 comp rx=0 tx=0).stop 2024-04-25T09:03:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/1672596866 shutdown_connections 2024-04-25T09:03:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 --2- 172.21.15.37:0/1672596866 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b4099e50 0x7fb4b406c9e0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 --2- 172.21.15.37:0/1672596866 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b4099a70 0x7fb4b40cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/1672596866 >> 172.21.15.37:0/1672596866 conn(0x7fb4b40b3100 msgr2=0x7fb4b40b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/1672596866 shutdown_connections 2024-04-25T09:03:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.089+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/1672596866 wait complete. 2024-04-25T09:03:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4bb6a9640 1 Processor -- start 2024-04-25T09:03:38.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4bb6a9640 1 -- start start 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4bb6a9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b4099a70 0x7fb4b4144d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4bb6a9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b4099e50 0x7fb4b40ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4bb6a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b4161510 0x7fb4b40ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4bb6a9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b40b2da0 con 0x7fb4b4161510 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4bb6a9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b409a2b0 con 0x7fb4b4099e50 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4bb6a9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b40b2f20 con 0x7fb4b4099a70 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b9c46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b4161510 0x7fb4b40ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b9c46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b4161510 0x7fb4b40ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51070/0 (socket says 172.21.15.37:51070) 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b9c46640 1 -- 172.21.15.37:0/309948183 learned_addr learned my addr 172.21.15.37:0/309948183 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b9445640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b4099a70 0x7fb4b4144d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b8c44640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b4099e50 0x7fb4b40ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b9c46640 1 -- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b4099a70 msgr2=0x7fb4b4144d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b9c46640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b4099a70 0x7fb4b4144d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b9c46640 1 -- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b4099e50 msgr2=0x7fb4b40ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b9c46640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b4099e50 0x7fb4b40ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b9c46640 1 -- 172.21.15.37:0/309948183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b4145570 con 0x7fb4b4161510 2024-04-25T09:03:38.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b9445640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b4099a70 0x7fb4b4144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:38.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4b9c46640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b4161510 0x7fb4b40ce5d0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c740 tx=0x7fb4ac00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4a9ffb640 1 -- 172.21.15.37:0/309948183 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac017070 con 0x7fb4b4161510 2024-04-25T09:03:38.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4a9ffb640 1 -- 172.21.15.37:0/309948183 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ac00ce50 con 0x7fb4b4161510 2024-04-25T09:03:38.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/309948183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b4145850 con 0x7fb4b4161510 2024-04-25T09:03:38.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4a9ffb640 1 -- 172.21.15.37:0/309948183 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ac00a860 con 0x7fb4b4161510 2024-04-25T09:03:38.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/309948183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b4142cb0 con 0x7fb4b4161510 2024-04-25T09:03:38.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.093+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/309948183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b404efa0 con 0x7fb4b4161510 2024-04-25T09:03:38.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.097+0000 7fb4a9ffb640 1 -- 172.21.15.37:0/309948183 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb4ac01a070 con 0x7fb4b4161510 2024-04-25T09:03:38.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.097+0000 7fb4a9ffb640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb498066f60 0x7fb498069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.097+0000 7fb4a9ffb640 1 -- 172.21.15.37:0/309948183 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb4ac0847a0 con 0x7fb4b4161510 2024-04-25T09:03:38.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.097+0000 7fb4b9445640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb498066f60 0x7fb498069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.097+0000 7fb4b9445640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb498066f60 0x7fb498069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40b62b0 tx=0x7fb4b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.101+0000 7fb4a9ffb640 1 -- 172.21.15.37:0/309948183 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb4ac055620 con 0x7fb4b4161510 2024-04-25T09:03:38.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.289+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/309948183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4b40b3cc0 con 0x7fb4b4161510 2024-04-25T09:03:38.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.289+0000 7fb4a9ffb640 1 -- 172.21.15.37:0/309948183 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb4ac054f20 con 0x7fb4b4161510 2024-04-25T09:03:38.279 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:38.279 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":97,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:03:35.513718+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9447},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9447":{"gid":9447,"name":"a","rank":0,"incarnation":93,"state":"up:active","state_seq":3,"addr":"172.21.15.37:6835/2748919477","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2748919477},{"type":"v1","addr":"172.21.15.37:6835","nonce":2748919477}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:03:38.280 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 97 2024-04-25T09:03:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb498066f60 msgr2=0x7fb498069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb498066f60 0x7fb498069420 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fb4b40b62b0 tx=0x7fb4b0008040 comp rx=0 tx=0).stop 2024-04-25T09:03:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/309948183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b4161510 msgr2=0x7fb4b40ce5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b4161510 0x7fb4b40ce5d0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac00c740 tx=0x7fb4ac00cc00 comp rx=0 tx=0).stop 2024-04-25T09:03:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/309948183 shutdown_connections 2024-04-25T09:03:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb498066f60 0x7fb498069420 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b4161510 0x7fb4b40ce5d0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb4b4099e50 0x7fb4b40ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 --2- 172.21.15.37:0/309948183 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb4b4099a70 0x7fb4b4144d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/309948183 >> 172.21.15.37:0/309948183 conn(0x7fb4b40b3100 msgr2=0x7fb4b415e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/309948183 shutdown_connections 2024-04-25T09:03:38.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.293+0000 7fb4bb6a9640 1 -- 172.21.15.37:0/309948183 wait complete. 2024-04-25T09:03:38.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:03:32.494159+0000', 'modified': '2024-04-25T09:03:35.513718+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9447}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9447': {'gid': 9447, 'name': 'a', 'rank': 0, 'incarnation': 93, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.37:6835/2748919477', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2748919477}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2748919477}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:03:38.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:03:38.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:03:38.747 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T09:03:38.757+0000 7f57569d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:03:38.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.845+0000 7f3ccbc4c640 1 Processor -- start 2024-04-25T09:03:38.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.845+0000 7f3ccbc4c640 1 -- start start 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.845+0000 7f3ccbc4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc40ce510 0x7f3cc40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.845+0000 7f3ccbc4c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3cc4144ca0 0x7f3cc406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.845+0000 7f3ccbc4c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc40cee30 con 0x7f3cc406ec60 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.845+0000 7f3ccbc4c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc40cc2b0 con 0x7f3cc4144ca0 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.845+0000 7f3ccbc4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc406e290 con 0x7f3cc40ce510 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.845+0000 7f3cc91e7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3cc4144ca0 0x7f3cc406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.845+0000 7f3cc91e7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3cc4144ca0 0x7f3cc406e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51078/0 (socket says 172.21.15.37:51078) 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.845+0000 7f3cc91e7640 1 -- 172.21.15.37:0/3525281583 learned_addr learned my addr 172.21.15.37:0/3525281583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/3525281583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc40ce510 0x7f3cc40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3cc91e7640 1 -- 172.21.15.37:0/3525281583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc40ce510 msgr2=0x7f3cc40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3cc91e7640 1 --2- 172.21.15.37:0/3525281583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc40ce510 0x7f3cc40ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3cc91e7640 1 -- 172.21.15.37:0/3525281583 >> v1:172.21.15.203:6789/0 conn(0x7f3cc406ec60 legacy=0x7f3cc4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3cc91e7640 1 -- 172.21.15.37:0/3525281583 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc40ccc70 con 0x7f3cc4144ca0 2024-04-25T09:03:38.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/3525281583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc40ce510 0x7f3cc40ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3cc91e7640 1 --2- 172.21.15.37:0/3525281583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc4144ca0 0x7f3cc406e720 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009870 tx=0x7f3cc001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f2316e2750ef54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3cc89e6640 1 -- 172.21.15.37:0/3525281583 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00b5070 con 0x7f3cc4144ca0 2024-04-25T09:03:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3cc89e6640 1 -- 172.21.15.37:0/3525281583 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc00a89d0 con 0x7f3cc4144ca0 2024-04-25T09:03:38.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3cc89e6640 1 -- 172.21.15.37:0/3525281583 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00a8da0 con 0x7f3cc4144ca0 2024-04-25T09:03:38.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/3525281583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc4144ca0 msgr2=0x7f3cc406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3ccbc4c640 1 --2- 172.21.15.37:0/3525281583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc4144ca0 0x7f3cc406e720 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009870 tx=0x7f3cc001ed10 comp rx=0 tx=0).stop 2024-04-25T09:03:38.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/3525281583 shutdown_connections 2024-04-25T09:03:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3ccbc4c640 1 --2- 172.21.15.37:0/3525281583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc4144ca0 0x7f3cc406e720 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3ccbc4c640 1 --2- 172.21.15.37:0/3525281583 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc40ce510 0x7f3cc40ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/3525281583 >> 172.21.15.37:0/3525281583 conn(0x7f3cc40b11a0 msgr2=0x7f3cc40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/3525281583 shutdown_connections 2024-04-25T09:03:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.849+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/3525281583 wait complete. 2024-04-25T09:03:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3ccbc4c640 1 Processor -- start 2024-04-25T09:03:38.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3ccbc4c640 1 -- start start 2024-04-25T09:03:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3ccbc4c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc406ec60 0x7f3cc4140370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3ccbc4c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cc40ce510 0x7f3cc413d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3ccbc4c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc4144ca0 0x7f3cc41440e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3ccbc4c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40cc900 con 0x7f3cc406ec60 2024-04-25T09:03:38.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3ccbc4c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc406df20 con 0x7f3cc40ce510 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3ccbc4c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc4142d90 con 0x7f3cc4144ca0 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cc99e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc406ec60 0x7f3cc4140370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cc99e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc406ec60 0x7f3cc4140370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51094/0 (socket says 172.21.15.37:51094) 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cc99e8640 1 -- 172.21.15.37:0/36189272 learned_addr learned my addr 172.21.15.37:0/36189272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cc91e7640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cc40ce510 0x7f3cc413d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc4144ca0 0x7f3cc41440e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cc99e8640 1 -- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc4144ca0 msgr2=0x7f3cc41440e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc4144ca0 0x7f3cc41440e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cc99e8640 1 -- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cc40ce510 msgr2=0x7f3cc413d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cc40ce510 0x7f3cc413d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cc99e8640 1 -- 172.21.15.37:0/36189272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc413b580 con 0x7f3cc406ec60 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc406ec60 0x7f3cc4140370 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c7c0 tx=0x7f3cb800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cb67fc640 1 -- 172.21.15.37:0/36189272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8007a30 con 0x7f3cc406ec60 2024-04-25T09:03:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cb67fc640 1 -- 172.21.15.37:0/36189272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb8007bd0 con 0x7f3cc406ec60 2024-04-25T09:03:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3cb67fc640 1 -- 172.21.15.37:0/36189272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80113f0 con 0x7f3cc406ec60 2024-04-25T09:03:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/36189272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc413b860 con 0x7f3cc406ec60 2024-04-25T09:03:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.853+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/36189272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc413be20 con 0x7f3cc406ec60 2024-04-25T09:03:38.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cb67fc640 1 -- 172.21.15.37:0/36189272 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3cb8011590 con 0x7f3cc406ec60 2024-04-25T09:03:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3ccedb9000 1 Processor -- start 2024-04-25T09:03:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3ccedb9000 1 -- start start 2024-04-25T09:03:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cb67fc640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca8066f60 0x7f3ca8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cb67fc640 1 -- 172.21.15.37:0/36189272 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3cb80854a0 con 0x7f3cc406ec60 2024-04-25T09:03:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca8066f60 0x7f3ca8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b5240 0x563b0d06a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d06b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b0cf525f0 con 0x563b0d0b4920 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b0ceb9da0 con 0x563b0d06af30 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b0ce42150 con 0x563b0d0b5240 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cc99e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cc99e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51100/0 (socket says 172.21.15.37:51100) 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cc99e8640 1 -- 172.21.15.37:0/1422306351 learned_addr learned my addr 172.21.15.37:0/1422306351 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cc91e7640 1 --2- 172.21.15.37:0/1422306351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b5240 0x563b0d06a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/1422306351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d06b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cc99e8640 1 -- 172.21.15.37:0/1422306351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b5240 msgr2=0x563b0d06a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/1422306351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b5240 0x563b0d06a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cc99e8640 1 -- 172.21.15.37:0/1422306351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 msgr2=0x563b0d06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/1422306351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d06b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cc99e8640 1 -- 172.21.15.37:0/1422306351 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b0ce5bca0 con 0x563b0d0b4920 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/1422306351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d06b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/1422306351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0b4d00 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8055b10 tx=0x7f3cb80563c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f486a02c12e49dad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca8066f60 0x7f3ca8069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4137e00 tx=0x7f3cbc0058d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3ca77fe640 1 -- 172.21.15.37:0/1422306351 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb801e070 con 0x563b0d0b4920 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3ca77fe640 1 -- 172.21.15.37:0/1422306351 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb805e290 con 0x563b0d0b4920 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.857+0000 7f3ca77fe640 1 -- 172.21.15.37:0/1422306351 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb80521f0 con 0x563b0d0b4920 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- 172.21.15.37:0/1422306351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 msgr2=0x563b0d0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/1422306351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0b4d00 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f3cb8055b10 tx=0x7f3cb80563c0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- 172.21.15.37:0/1422306351 shutdown_connections 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/1422306351 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d06b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/1422306351 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b5240 0x563b0d06a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/1422306351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0b4d00 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- 172.21.15.37:0/1422306351 >> 172.21.15.37:0/1422306351 conn(0x563b0d0712a0 msgr2=0x563b0d0692d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- 172.21.15.37:0/1422306351 shutdown_connections 2024-04-25T09:03:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- 172.21.15.37:0/1422306351 wait complete. 2024-04-25T09:03:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- start start 2024-04-25T09:03:38.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 Processor -- start 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d0bbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b4920 0x563b0d0bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d06eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b0ce74990 con 0x563b0d0b5240 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b0ceb5e40 con 0x563b0d06af30 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b0ce45900 con 0x563b0d0b4920 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 10 client.0 _ll_register_callbacks cb 0x563b0cf6d1c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cca1e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d06eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cca1e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d06eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51112/0 (socket says 172.21.15.37:51112) 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cca1e9640 1 -- 172.21.15.37:0/110259967 learned_addr learned my addr 172.21.15.37:0/110259967 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cc91e7640 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b4920 0x563b0d0bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d0bbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cca1e9640 1 -- 172.21.15.37:0/110259967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b4920 msgr2=0x563b0d0bc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b4920 0x563b0d0bc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cca1e9640 1 -- 172.21.15.37:0/110259967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 msgr2=0x563b0d0bbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d0bbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cca1e9640 1 -- 172.21.15.37:0/110259967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b0ce72630 con 0x563b0d0b5240 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d06eaa0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0199a0 tx=0x7f3cbc003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1896f9f0dd8aed8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/110259967 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc019e60 con 0x563b0d0b5240 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- 172.21.15.37:0/110259967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cb805e290 con 0x563b0d0b5240 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/110259967 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3cbc010070 con 0x563b0d0b5240 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/110259967 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cbc003b90 con 0x563b0d0b5240 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 10 client.9576 fetch_fsmap learned FSMap version 97 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- 172.21.15.37:0/110259967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) v3 -- 0x563b0ce423a0 con 0x563b0d0b5240 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/110259967 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb805e290 con 0x563b0d0b5240 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/110259967 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 97) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3cbc006b60 con 0x563b0d0b5240 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 10 client.9576 fetch_fsmap finished waiting for FSMap version 97 2024-04-25T09:03:38.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 10 client.9576 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 20 client.9576 populate_metadata read hostname 'smithi037' 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x563b0d070160 0x563b0d0c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 4 client.9576 mds_command: new command op to 9447 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ccedb9000 1 -- 172.21.15.37:0/110259967 --> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b0ce55a50 con 0x563b0d070160 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x563b0d070160 0x563b0d0c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3ca4ff9640 10 client.9576 ms_handle_connect on v2:172.21.15.37:6834/2748919477 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.861+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x563b0d070160 0x563b0d0c5dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45e566c617e85587 server_cookie=1f622bf54edfb74f in_seq=0 out_seq=0 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/110259967 <== mds.0 v2:172.21.15.37:6834/2748919477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b0ce55a50 con 0x563b0d070160 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ca4ff9640 10 client.9576 handle_command_reply: tid=0 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 1 client.9576 shutdown 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 1 -- 172.21.15.37:0/110259967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 msgr2=0x563b0d06eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d06eaa0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc0199a0 tx=0x7f3cbc003a80 comp rx=0 tx=0).stop 2024-04-25T09:03:38.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 1 -- 172.21.15.37:0/110259967 shutdown_connections 2024-04-25T09:03:38.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x563b0d070160 0x563b0d0c5dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d06eaa0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b4920 0x563b0d0bc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/110259967 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d0bbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ca4ff9640 0 client.9576 ms_handle_reset on v2:172.21.15.37:6834/2748919477 2024-04-25T09:03:38.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 1 -- 172.21.15.37:0/110259967 >> 172.21.15.37:0/110259967 conn(0x563b0d0712a0 msgr2=0x563b0d0699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 1 -- 172.21.15.37:0/110259967 shutdown_connections 2024-04-25T09:03:38.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 1 -- 172.21.15.37:0/110259967 wait complete. 2024-04-25T09:03:38.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.865+0000 7f3ccedb9000 20 client.9576 trim_cache size 0 max 16384 2024-04-25T09:03:38.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 Processor -- start 2024-04-25T09:03:38.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 -- start start 2024-04-25T09:03:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d06af30 0x563b0d118a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d0b4920 0x563b0d118fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d119500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc41ff460 con 0x563b0d0b5240 2024-04-25T09:03:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc41394e0 con 0x563b0d0b4920 2024-04-25T09:03:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc41fee60 con 0x563b0d06af30 2024-04-25T09:03:38.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3cca1e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d119500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3cca1e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d119500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51116/0 (socket says 172.21.15.37:51116) 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3cca1e9640 1 -- 172.21.15.37:0/2436628181 learned_addr learned my addr 172.21.15.37:0/2436628181 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/2436628181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d06af30 0x563b0d118a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3cc91e7640 1 --2- 172.21.15.37:0/2436628181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d0b4920 0x563b0d118fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3cca1e9640 1 -- 172.21.15.37:0/2436628181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d06af30 msgr2=0x563b0d118a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/2436628181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d06af30 0x563b0d118a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3cca1e9640 1 -- 172.21.15.37:0/2436628181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d0b4920 msgr2=0x563b0d118fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/2436628181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d0b4920 0x563b0d118fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3cca1e9640 1 -- 172.21.15.37:0/2436628181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b0d0b5070 con 0x563b0d0b5240 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/2436628181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d119500 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009010 tx=0x7f3cbc01b050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4e6b1045752467 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ca6ffd640 1 -- 172.21.15.37:0/2436628181 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc019e60 con 0x563b0d0b5240 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ca6ffd640 1 -- 172.21.15.37:0/2436628181 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc0157a0 con 0x563b0d0b5240 2024-04-25T09:03:38.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ca6ffd640 1 -- 172.21.15.37:0/2436628181 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc015b70 con 0x563b0d0b5240 2024-04-25T09:03:38.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 -- 172.21.15.37:0/2436628181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 msgr2=0x563b0d119500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/2436628181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d119500 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc009010 tx=0x7f3cbc01b050 comp rx=0 tx=0).stop 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 -- 172.21.15.37:0/2436628181 shutdown_connections 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/2436628181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b5240 0x563b0d119500 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/2436628181 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d0b4920 0x563b0d118fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/2436628181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d06af30 0x563b0d118a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 -- 172.21.15.37:0/2436628181 >> 172.21.15.37:0/2436628181 conn(0x563b0d0712a0 msgr2=0x563b0d06e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 -- 172.21.15.37:0/2436628181 shutdown_connections 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 -- 172.21.15.37:0/2436628181 wait complete. 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 -- start start 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 Processor -- start 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d0c1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0d8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b5240 0x563b0d0d8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.881+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b0cebe3d0 con 0x563b0d0b4920 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b0cf75d50 con 0x563b0d06af30 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b0ce54980 con 0x563b0d0b5240 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 10 client.0 _ll_register_callbacks cb 0x7f3cb8051e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc91e7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0d8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc91e7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0d8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51132/0 (socket says 172.21.15.37:51132) 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc91e7640 1 -- 172.21.15.37:0/3092210911 learned_addr learned my addr 172.21.15.37:0/3092210911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cca1e9640 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b5240 0x563b0d0d8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d0c1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc91e7640 1 -- 172.21.15.37:0/3092210911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b5240 msgr2=0x563b0d0d8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc91e7640 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b5240 0x563b0d0d8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc91e7640 1 -- 172.21.15.37:0/3092210911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 msgr2=0x563b0d0c1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc91e7640 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d0c1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc91e7640 1 -- 172.21.15.37:0/3092210911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b0cf3b600 con 0x563b0d0b4920 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc91e7640 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0d8050 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009870 tx=0x7f3cc001d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5c2456ef3c5a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:38.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ca4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/3092210911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc00c1020 con 0x563b0d0b4920 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/3092210911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3cc00ba440 con 0x563b0d0b4920 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 -- 172.21.15.37:0/3092210911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3cbc0157a0 con 0x563b0d0b4920 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/3092210911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3cc00cb050 con 0x563b0d0b4920 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 10 client.9582 fetch_fsmap learned FSMap version 97 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 -- 172.21.15.37:0/3092210911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) v3 -- 0x563b0ce4e9b0 con 0x563b0d0b4920 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/3092210911 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc0157a0 con 0x563b0d0b4920 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/3092210911 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 97) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3cc00ce020 con 0x563b0d0b4920 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 10 client.9582 fetch_fsmap finished waiting for FSMap version 97 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 10 client.9582 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 20 client.9582 populate_metadata read hostname 'smithi037' 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x563b0d070e50 0x563b0d0d8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 4 client.9582 mds_command: new command op to 9447 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:03:38.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 -- 172.21.15.37:0/3092210911 --> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563b0cec72e0 con 0x563b0d070e50 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x563b0d070e50 0x563b0d0d8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3cc99e8640 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x563b0d070e50 0x563b0d0d8ad0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c7c2e2d6788592d server_cookie=d4513533edb6674e in_seq=0 out_seq=0 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ca4ff9640 10 client.9582 ms_handle_connect on v2:172.21.15.37:6834/2748919477 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ca4ff9640 1 -- 172.21.15.37:0/3092210911 <== mds.0 v2:172.21.15.37:6834/2748919477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+320 (crc 0 0 0) 0x563b0cec72e0 con 0x563b0d070e50 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ca4ff9640 10 client.9582 handle_command_reply: tid=0 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 client.9582 shutdown 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 -- 172.21.15.37:0/3092210911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 msgr2=0x563b0d0d8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0d8050 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009870 tx=0x7f3cc001d550 comp rx=0 tx=0).stop 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 -- 172.21.15.37:0/3092210911 shutdown_connections 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x563b0d070e50 0x563b0d0d8ad0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x563b0d0b5240 0x563b0d0d8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b0d0b4920 0x563b0d0d8050 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ca4ff9640 0 client.9582 ms_handle_reset on v2:172.21.15.37:6834/2748919477 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 --2- 172.21.15.37:0/3092210911 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x563b0d06af30 0x563b0d0c1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 -- 172.21.15.37:0/3092210911 >> 172.21.15.37:0/3092210911 conn(0x563b0d0712a0 msgr2=0x563b0d0bc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 -- 172.21.15.37:0/3092210911 shutdown_connections 2024-04-25T09:03:38.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 1 -- 172.21.15.37:0/3092210911 wait complete. 2024-04-25T09:03:38.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccedb9000 20 client.9582 trim_cache size 0 max 16384 2024-04-25T09:03:38.880 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:03:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T09:03:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T09:03:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 9447, 2024-04-25T09:03:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T09:03:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T09:03:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T09:03:38.880 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 5.771643901, 2024-04-25T09:03:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 97, 2024-04-25T09:03:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 125, 2024-04-25T09:03:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 125, 2024-04-25T09:03:38.881 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 11.052694983 2024-04-25T09:03:38.881 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:03:38.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca8066f60 msgr2=0x7f3ca8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca8066f60 0x7f3ca8069420 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3cc4137e00 tx=0x7f3cbc0058d0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/36189272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc406ec60 msgr2=0x7f3cc4140370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:38.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc406ec60 0x7f3cc4140370 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c7c0 tx=0x7f3cb800cc80 comp rx=0 tx=0).stop 2024-04-25T09:03:38.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/36189272 shutdown_connections 2024-04-25T09:03:38.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3ca8066f60 0x7f3ca8069420 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3cc4144ca0 0x7f3cc41440e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3cc40ce510 0x7f3cc413d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 --2- 172.21.15.37:0/36189272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3cc406ec60 0x7f3cc4140370 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:38.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/36189272 >> 172.21.15.37:0/36189272 conn(0x7f3cc40b11a0 msgr2=0x7f3cc40b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:38.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/36189272 shutdown_connections 2024-04-25T09:03:38.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:38.885+0000 7f3ccbc4c640 1 -- 172.21.15.37:0/36189272 wait complete. 2024-04-25T09:03:38.891 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:03:39.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f74943f9640 1 Processor -- start 2024-04-25T09:03:39.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f74943f9640 1 -- start start 2024-04-25T09:03:39.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f74943f9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f748c15b950 0x7f748c09a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f74943f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c0cf420 0x7f748c0cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f74943f9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f748c0cfb80 0x7f748c0d0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f74943f9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c15bd30 con 0x7f748c0cfb80 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f74943f9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c0cc0d0 con 0x7f748c15b950 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f74943f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c09ab10 con 0x7f748c0cf420 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f7492996640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f748c0cfb80 0x7f748c0d0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f7492996640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f748c0cfb80 0x7f748c0d0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51148/0 (socket says 172.21.15.37:51148) 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f7492996640 1 -- 172.21.15.37:0/3489543625 learned_addr learned my addr 172.21.15.37:0/3489543625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f7491994640 1 --2- 172.21.15.37:0/3489543625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c0cf420 0x7f748c0cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f7492195640 1 --2- 172.21.15.37:0/3489543625 >> v2:172.21.15.203:3300/0 conn(0x7f748c15b950 0x7f748c09a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f7492996640 1 -- 172.21.15.37:0/3489543625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c0cf420 msgr2=0x7f748c0cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f7492996640 1 --2- 172.21.15.37:0/3489543625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c0cf420 0x7f748c0cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f7492996640 1 -- 172.21.15.37:0/3489543625 >> v2:172.21.15.203:3300/0 conn(0x7f748c15b950 msgr2=0x7f748c09a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:39.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f7492996640 1 --2- 172.21.15.37:0/3489543625 >> v2:172.21.15.203:3300/0 conn(0x7f748c15b950 0x7f748c09a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f7492996640 1 -- 172.21.15.37:0/3489543625 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0cc370 con 0x7f748c0cfb80 2024-04-25T09:03:39.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.017+0000 7f7491994640 1 --2- 172.21.15.37:0/3489543625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c0cf420 0x7f748c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:39.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f7492996640 1 --2- 172.21.15.37:0/3489543625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f748c0cfb80 0x7f748c0d0010 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f748c072480 tx=0x7f748401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c62ced8e46a8d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:39.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f7491193640 1 -- 172.21.15.37:0/3489543625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74840a5760 con 0x7f748c0cfb80 2024-04-25T09:03:39.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f7491193640 1 -- 172.21.15.37:0/3489543625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74840a5900 con 0x7f748c0cfb80 2024-04-25T09:03:39.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f7491193640 1 -- 172.21.15.37:0/3489543625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74840a5cd0 con 0x7f748c0cfb80 2024-04-25T09:03:39.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f74943f9640 1 -- 172.21.15.37:0/3489543625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f748c0cfb80 msgr2=0x7f748c0d0010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f74943f9640 1 --2- 172.21.15.37:0/3489543625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f748c0cfb80 0x7f748c0d0010 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f748c072480 tx=0x7f748401c9f0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f74943f9640 1 -- 172.21.15.37:0/3489543625 shutdown_connections 2024-04-25T09:03:39.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f74943f9640 1 --2- 172.21.15.37:0/3489543625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f748c0cfb80 0x7f748c0d0010 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f74943f9640 1 --2- 172.21.15.37:0/3489543625 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c0cf420 0x7f748c0cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f74943f9640 1 --2- 172.21.15.37:0/3489543625 >> v2:172.21.15.203:3300/0 conn(0x7f748c15b950 0x7f748c09a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f74943f9640 1 -- 172.21.15.37:0/3489543625 >> 172.21.15.37:0/3489543625 conn(0x7f748c0b30b0 msgr2=0x7f748c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:39.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f74943f9640 1 -- 172.21.15.37:0/3489543625 shutdown_connections 2024-04-25T09:03:39.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f74943f9640 1 -- 172.21.15.37:0/3489543625 wait complete. 2024-04-25T09:03:39.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f74943f9640 1 Processor -- start 2024-04-25T09:03:39.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.021+0000 7f74943f9640 1 -- start start 2024-04-25T09:03:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f74943f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f748c0cf420 0x7f748c0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f74943f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f748c0cfb80 0x7f748c0b65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f74943f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c15b950 0x7f748c0b6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f74943f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c0cd390 con 0x7f748c0cf420 2024-04-25T09:03:39.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f74943f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c06faa0 con 0x7f748c0cfb80 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f74943f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c0c65a0 con 0x7f748c15b950 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7492195640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f748c0cf420 0x7f748c0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7492195640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f748c0cf420 0x7f748c0b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51152/0 (socket says 172.21.15.37:51152) 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7492195640 1 -- 172.21.15.37:0/2130441334 learned_addr learned my addr 172.21.15.37:0/2130441334 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7492996640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c15b950 0x7f748c0b6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7491994640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f748c0cfb80 0x7f748c0b65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7492195640 1 -- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c15b950 msgr2=0x7f748c0b6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7492195640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c15b950 0x7f748c0b6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7492195640 1 -- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f748c0cfb80 msgr2=0x7f748c0b65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7492195640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f748c0cfb80 0x7f748c0b65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7492195640 1 -- 172.21.15.37:0/2130441334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c0b4620 con 0x7f748c0cf420 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7492996640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c15b950 0x7f748c0b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7491994640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f748c0cfb80 0x7f748c0b65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7492195640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f748c0cf420 0x7f748c0b40e0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f748800d620 tx=0x7f748800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7482ffd640 1 -- 172.21.15.37:0/2130441334 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748800b620 con 0x7f748c0cf420 2024-04-25T09:03:39.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7482ffd640 1 -- 172.21.15.37:0/2130441334 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7488019070 con 0x7f748c0cf420 2024-04-25T09:03:39.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f74943f9640 1 -- 172.21.15.37:0/2130441334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c0b48a0 con 0x7f748c0cf420 2024-04-25T09:03:39.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7482ffd640 1 -- 172.21.15.37:0/2130441334 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748800b9e0 con 0x7f748c0cf420 2024-04-25T09:03:39.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f74943f9640 1 -- 172.21.15.37:0/2130441334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c0b4e60 con 0x7f748c0cf420 2024-04-25T09:03:39.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f74943f9640 1 -- 172.21.15.37:0/2130441334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c04efa0 con 0x7f748c0cf420 2024-04-25T09:03:39.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7482ffd640 1 -- 172.21.15.37:0/2130441334 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f748800bb80 con 0x7f748c0cf420 2024-04-25T09:03:39.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7482ffd640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7470066f60 0x7f7470069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:39.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7482ffd640 1 -- 172.21.15.37:0/2130441334 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7488085540 con 0x7f748c0cf420 2024-04-25T09:03:39.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.025+0000 7f7491994640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7470066f60 0x7f7470069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:39.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.029+0000 7f7491994640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7470066f60 0x7f7470069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f748c0c6390 tx=0x7f747c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:39.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.029+0000 7f7482ffd640 1 -- 172.21.15.37:0/2130441334 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f748c0b4620 con 0x7f748c0cf420 2024-04-25T09:03:39.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.225+0000 7f74943f9640 1 -- 172.21.15.37:0/2130441334 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f748c0b63b0 con 0x7f748c0cf420 2024-04-25T09:03:39.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.225+0000 7f7482ffd640 1 -- 172.21.15.37:0/2130441334 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) v1 ==== 53+0+73 (secure 0 0 0) 0x7f74880563c0 con 0x7f748c0cf420 2024-04-25T09:03:39.216 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:03:39.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 -- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7470066f60 msgr2=0x7f7470069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7470066f60 0x7f7470069420 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f748c0c6390 tx=0x7f747c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:39.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 -- 172.21.15.37:0/2130441334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f748c0cf420 msgr2=0x7f748c0b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f748c0cf420 0x7f748c0b40e0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f748800d620 tx=0x7f748800dae0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 -- 172.21.15.37:0/2130441334 shutdown_connections 2024-04-25T09:03:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7470066f60 0x7f7470069420 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f748c15b950 0x7f748c0b6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f748c0cfb80 0x7f748c0b65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 --2- 172.21.15.37:0/2130441334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f748c0cf420 0x7f748c0b40e0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 -- 172.21.15.37:0/2130441334 >> 172.21.15.37:0/2130441334 conn(0x7f748c0b30b0 msgr2=0x7f748c070780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:39.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 -- 172.21.15.37:0/2130441334 shutdown_connections 2024-04-25T09:03:39.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:39.229+0000 7f74943f9640 1 -- 172.21.15.37:0/2130441334 wait complete. 2024-04-25T09:03:39.234 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:03:39.234 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:03:39.234 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T09:03:39.234 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:03:39.234 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:39.234 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:03:39.234 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:03:39.234 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:03:39.234 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:39.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.235 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T09:03:39.241 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:39.241 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T09:03:39.242 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41821sec preferred_lft 41821sec 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.243 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:03:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.244 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:03:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:03:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.244 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:03:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:03:39.244 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:39.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.244 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:03:39.296 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T09:03:39.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.297 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T09:03:39.346 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T09:03:39.346 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:03:39.346 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T09:03:39.346 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:03:39.346 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T09:03:39.347 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:03:39.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.347 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T09:03:39.347 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T09:03:39.347 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:03:39.347 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:03:39.347 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:03:39.347 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T09:03:39.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.421 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T09:03:39.469 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:03:39.469 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:03:39.470 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:39.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T09:03:39.526 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:03:39.526 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:39.526 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:39.574 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:39.574 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:39.574 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:39.621 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:03:39.679 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:03:39.727 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:03:39.728 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:03:39.728 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:03:39.728 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:03:39.728 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:03:39.728 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:03:39.728 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:03:39.728 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:03:39.728 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:03:39.728 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:03:39.728 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:03:39.728 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:03:39.729 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:03:39.730 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:03:39.730 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:03:39.730 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:39.730 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:39.730 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:39.730 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:39.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.731 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:03:39.777 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:03:39.777 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:03:39.822 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:03:39.837 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:03:39.885 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:03:39.885 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:03:39.886 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:39.887 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:39.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:39.888 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:03:39.907 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T09:03:39.917+0000 7fb1843bb580 -1 init, newargv = 0x5557b3356d70 newargc=15 2024-04-25T09:03:39.908 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[54358]: starting ceph client 2024-04-25T09:03:39.914 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[54358]: starting fuse 2024-04-25T09:03:41.013 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:03:41.031 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:03:41.080 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:03:41.080 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:03:41.080 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:03:41.080 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:03:41.080 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:03:41.080 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:03:41.080 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:03:41.080 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:03:41.081 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:03:41.081 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:03:41.081 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:03:41.081 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:03:41.081 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:03:41.081 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:03:41.081 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:03:41.081 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:03:41.081 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:03:41.081 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:41.082 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:41.083 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:41.083 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:41.083 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:41.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:41.084 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:03:41.130 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T09:03:41.130 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:03:41.130 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:03:41.130 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T09:03:41.130 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T09:03:41.130 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:03:41.130 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T09:03:41.130 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T09:03:41.131 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T09:03:41.132 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T09:03:41.132 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:03:41.132 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:41.132 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T09:03:41.132 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:03:41.258 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.54358.asok 2024-04-25T09:03:41.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:41.270 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54358.asok 2024-04-25T09:03:41.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:41.270 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54358.asok status 2024-04-25T09:03:41.430 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:03:41.430 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T09:03:41.430 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:03:41.430 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:03:41.430 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T09:03:41.430 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T09:03:41.430 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:03:41.430 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "54358", 2024-04-25T09:03:41.430 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T09:03:41.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:41.430 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 9594, 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 9594 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 637997003 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:03:41.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 637997003 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.9594 192.168.0.1:0/637997003", 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/637997003", 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 97, 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 125, 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T09:03:41.432 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:03:41.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:41.445 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T09:03:41.445 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:41.451 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T09:03:41.451 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:41.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:41.451 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:41.504 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:03:41.504 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:03:41.504 DEBUG:teuthology.orchestra.run.smithi037:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:03:41.504 DEBUG:teuthology.orchestra.run.smithi037:> parent = os.path.join(mount_point, "parent") 2024-04-25T09:03:41.504 DEBUG:teuthology.orchestra.run.smithi037:> os.mkdir(parent) 2024-04-25T09:03:41.504 DEBUG:teuthology.orchestra.run.smithi037:> flushed = os.path.join(parent, "flushed") 2024-04-25T09:03:41.504 DEBUG:teuthology.orchestra.run.smithi037:> os.mkdir(flushed) 2024-04-25T09:03:41.504 DEBUG:teuthology.orchestra.run.smithi037:> for f in ["alpha", "bravo", "charlie"]: 2024-04-25T09:03:41.505 DEBUG:teuthology.orchestra.run.smithi037:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-04-25T09:03:41.505 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:03:41.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:41.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:41.646 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T09:03:41.709 INFO:teuthology.orchestra.run.smithi037.stdout:./ 2024-04-25T09:03:41.709 INFO:teuthology.orchestra.run.smithi037.stdout:./parent 2024-04-25T09:03:41.709 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/flushed 2024-04-25T09:03:41.709 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/flushed/charlie 2024-04-25T09:03:41.709 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/flushed/alpha 2024-04-25T09:03:41.709 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/flushed/bravo 2024-04-25T09:03:41.709 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:03:41.709 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:03:41.710 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:03:41.710 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:41.710 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T09:03:41.710 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:03:41.823 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T09:03:41.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:41.833 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:03:41.833 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:03:41.833 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:03:41.833 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:41.833 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T09:03:41.834 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:03:41.944 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627776 2024-04-25T09:03:41.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:41.954 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:03:41.954 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:03:41.954 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:03:41.954 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:41.954 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T09:03:41.954 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:03:42.057 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627777 2024-04-25T09:03:42.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:42.067 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:03:42.067 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:03:42.067 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:03:42.067 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:42.067 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T09:03:42.068 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:03:42.170 INFO:teuthology.orchestra.run.smithi037.stdout:1099511628280 2024-04-25T09:03:42.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:42.181 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:03:42.181 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:03:42.181 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:03:42.181 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:42.181 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T09:03:42.181 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:03:42.285 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627778 2024-04-25T09:03:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:42.295 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:03:42.295 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:03:42.295 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:03:42.295 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:42.295 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T09:03:42.296 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:03:42.405 INFO:teuthology.orchestra.run.smithi037.stdout:1099511628279 2024-04-25T09:03:42.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:42.415 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:03:42.415 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:42.461 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:42.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0acea1a640 1 Processor -- start 2024-04-25T09:03:42.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0acea1a640 1 -- start start 2024-04-25T09:03:42.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0acea1a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ac8144ca0 0x7f0ac806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:42.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0acea1a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac806ec60 0x7f0ac8142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:42.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0acea1a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac80ccc70 con 0x7f0ac80ce510 2024-04-25T09:03:42.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0acea1a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac80cc2b0 con 0x7f0ac8144ca0 2024-04-25T09:03:42.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0acea1a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac806e290 con 0x7f0ac806ec60 2024-04-25T09:03:42.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0accfb7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac806ec60 0x7f0ac8142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:42.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0ac77fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ac8144ca0 0x7f0ac806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:42.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0accfb7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac806ec60 0x7f0ac8142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:33140/0 (socket says 172.21.15.37:33140) 2024-04-25T09:03:42.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0ac77fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0ac8144ca0 0x7f0ac806e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:44830/0 (socket says 172.21.15.37:44830) 2024-04-25T09:03:42.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0accfb7640 1 -- 172.21.15.37:0/1232925944 learned_addr learned my addr 172.21.15.37:0/1232925944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:42.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0accfb7640 1 -- 172.21.15.37:0/1232925944 >> v2:172.21.15.203:3300/0 conn(0x7f0ac8144ca0 msgr2=0x7f0ac806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:42.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0accfb7640 1 --2- 172.21.15.37:0/1232925944 >> v2:172.21.15.203:3300/0 conn(0x7f0ac8144ca0 0x7f0ac806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0accfb7640 1 -- 172.21.15.37:0/1232925944 >> v1:172.21.15.203:6789/0 conn(0x7f0ac80ce510 legacy=0x7f0ac80ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0accfb7640 1 -- 172.21.15.37:0/1232925944 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac806dc40 con 0x7f0ac806ec60 2024-04-25T09:03:42.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.653+0000 7f0ac77fe640 1 --2- 172.21.15.37:0/1232925944 >> v2:172.21.15.203:3300/0 conn(0x7f0ac8144ca0 0x7f0ac806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:42.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0accfb7640 1 --2- 172.21.15.37:0/1232925944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac806ec60 0x7f0ac8142680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009e50 tx=0x7f0ac001f210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31a039eed5f6c24a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:42.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0ac6ffd640 1 -- 172.21.15.37:0/1232925944 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00b4070 con 0x7f0ac806ec60 2024-04-25T09:03:42.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0ac6ffd640 1 -- 172.21.15.37:0/1232925944 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac00af400 con 0x7f0ac806ec60 2024-04-25T09:03:42.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0ac6ffd640 1 -- 172.21.15.37:0/1232925944 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00af7d0 con 0x7f0ac806ec60 2024-04-25T09:03:42.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0acea1a640 1 -- 172.21.15.37:0/1232925944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac806ec60 msgr2=0x7f0ac8142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0acea1a640 1 --2- 172.21.15.37:0/1232925944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac806ec60 0x7f0ac8142680 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0009e50 tx=0x7f0ac001f210 comp rx=0 tx=0).stop 2024-04-25T09:03:42.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0acea1a640 1 -- 172.21.15.37:0/1232925944 shutdown_connections 2024-04-25T09:03:42.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0acea1a640 1 --2- 172.21.15.37:0/1232925944 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac806ec60 0x7f0ac8142680 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0acea1a640 1 --2- 172.21.15.37:0/1232925944 >> v2:172.21.15.203:3300/0 conn(0x7f0ac8144ca0 0x7f0ac806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0acea1a640 1 -- 172.21.15.37:0/1232925944 >> 172.21.15.37:0/1232925944 conn(0x7f0ac80b11a0 msgr2=0x7f0ac80b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:42.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0acea1a640 1 -- 172.21.15.37:0/1232925944 shutdown_connections 2024-04-25T09:03:42.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0acea1a640 1 -- 172.21.15.37:0/1232925944 wait complete. 2024-04-25T09:03:42.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0acea1a640 1 Processor -- start 2024-04-25T09:03:42.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.657+0000 7f0acea1a640 1 -- start start 2024-04-25T09:03:42.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0acea1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac806ec60 0x7f0ac80c7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:42.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0acea1a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac80ce510 0x7f0ac8133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:42.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0acea1a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ac8144ca0 0x7f0ac80c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:42.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0acea1a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac81451f0 con 0x7f0ac806ec60 2024-04-25T09:03:42.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0acea1a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac80cc900 con 0x7f0ac8144ca0 2024-04-25T09:03:42.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0acea1a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac8145370 con 0x7f0ac80ce510 2024-04-25T09:03:42.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac806ec60 0x7f0ac80c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:42.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac806ec60 0x7f0ac80c7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50436/0 (socket says 172.21.15.37:50436) 2024-04-25T09:03:42.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac7fff640 1 -- 172.21.15.37:0/1342399675 learned_addr learned my addr 172.21.15.37:0/1342399675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:42.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac77fe640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac80ce510 0x7f0ac8133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:42.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0accfb7640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ac8144ca0 0x7f0ac80c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:42.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac7fff640 1 -- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac80ce510 msgr2=0x7f0ac8133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac7fff640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac80ce510 0x7f0ac8133ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac7fff640 1 -- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ac8144ca0 msgr2=0x7f0ac80c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac7fff640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ac8144ca0 0x7f0ac80c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac7fff640 1 -- 172.21.15.37:0/1342399675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac80c7be0 con 0x7f0ac806ec60 2024-04-25T09:03:42.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac7fff640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac806ec60 0x7f0ac80c7470 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c450 tx=0x7f0ab800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:42.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac4ff9640 1 -- 172.21.15.37:0/1342399675 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8016020 con 0x7f0ac806ec60 2024-04-25T09:03:42.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac4ff9640 1 -- 172.21.15.37:0/1342399675 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab80052d0 con 0x7f0ac806ec60 2024-04-25T09:03:42.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0acea1a640 1 -- 172.21.15.37:0/1342399675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac80c7ec0 con 0x7f0ac806ec60 2024-04-25T09:03:42.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac4ff9640 1 -- 172.21.15.37:0/1342399675 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8007260 con 0x7f0ac806ec60 2024-04-25T09:03:42.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0acea1a640 1 -- 172.21.15.37:0/1342399675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac80c8480 con 0x7f0ac806ec60 2024-04-25T09:03:42.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0acea1a640 1 -- 172.21.15.37:0/1342399675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ac804efa0 con 0x7f0ac806ec60 2024-04-25T09:03:42.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac4ff9640 1 -- 172.21.15.37:0/1342399675 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0ab8007400 con 0x7f0ac806ec60 2024-04-25T09:03:42.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac4ff9640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ab4066f60 0x7f0ab4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:42.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac4ff9640 1 -- 172.21.15.37:0/1342399675 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0ab8084b10 con 0x7f0ac806ec60 2024-04-25T09:03:42.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.661+0000 7f0ac77fe640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ab4066f60 0x7f0ab4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:42.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.665+0000 7f0ac77fe640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ab4066f60 0x7f0ab4069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80b2050 tx=0x7f0abc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:42.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.669+0000 7f0ac4ff9640 1 -- 172.21.15.37:0/1342399675 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0ab8020020 con 0x7f0ac806ec60 2024-04-25T09:03:42.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.829+0000 7f0acea1a640 1 -- 172.21.15.37:0/1342399675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ac80b1cc0 con 0x7f0ac806ec60 2024-04-25T09:03:42.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.829+0000 7f0ac4ff9640 1 -- 172.21.15.37:0/1342399675 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16916 (secure 0 0 0) 0x7f0ab8005480 con 0x7f0ac806ec60 2024-04-25T09:03:42.819 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:42.819 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":125,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:33.770823+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 -- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ab4066f60 msgr2=0x7f0ab4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ab4066f60 0x7f0ab4069420 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f0ac80b2050 tx=0x7f0abc008040 comp rx=0 tx=0).stop 2024-04-25T09:03:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 -- 172.21.15.37:0/1342399675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac806ec60 msgr2=0x7f0ac80c7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac806ec60 0x7f0ac80c7470 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c450 tx=0x7f0ab800c910 comp rx=0 tx=0).stop 2024-04-25T09:03:42.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 -- 172.21.15.37:0/1342399675 shutdown_connections 2024-04-25T09:03:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0ab4066f60 0x7f0ab4069420 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0ac8144ca0 0x7f0ac80c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0ac80ce510 0x7f0ac8133ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 --2- 172.21.15.37:0/1342399675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac806ec60 0x7f0ac80c7470 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 -- 172.21.15.37:0/1342399675 >> 172.21.15.37:0/1342399675 conn(0x7f0ac80b11a0 msgr2=0x7f0ac80b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 -- 172.21.15.37:0/1342399675 shutdown_connections 2024-04-25T09:03:42.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.833+0000 7f0acea1a640 1 -- 172.21.15.37:0/1342399675 wait complete. 2024-04-25T09:03:42.838 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:30.763724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:31.488462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:42.839 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:42.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe64e316640 1 Processor -- start 2024-04-25T09:03:42.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe64e316640 1 -- start start 2024-04-25T09:03:42.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe64e316640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe64815b6b0 0x7fe6480ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:42.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe64e316640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe64815ff50 con 0x7fe6480cf590 2024-04-25T09:03:42.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe64e316640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6480cbd20 con 0x7fe648073010 2024-04-25T09:03:42.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe64e316640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe64815ba90 con 0x7fe64815b6b0 2024-04-25T09:03:42.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe63ffff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe6480cf590 0x7fe64815f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53102/0 (socket says 172.21.15.37:53102) 2024-04-25T09:03:42.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe63ffff640 1 -- 172.21.15.37:0/2163242651 learned_addr learned my addr 172.21.15.37:0/2163242651 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:42.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe647fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe64815b6b0 0x7fe6480ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:42.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe6477fe640 1 -- 172.21.15.37:0/2163242651 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 826157095 0 0) 0x7fe64815ff50 con 0x7fe6480cf590 2024-04-25T09:03:42.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe6477fe640 1 -- 172.21.15.37:0/2163242651 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe634003530 con 0x7fe6480cf590 2024-04-25T09:03:42.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe647fff640 1 -- 172.21.15.37:0/2163242651 >> v1:172.21.15.203:6789/0 conn(0x7fe648073010 legacy=0x7fe64815f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe647fff640 1 -- 172.21.15.37:0/2163242651 >> v1:172.21.15.37:6789/0 conn(0x7fe6480cf590 legacy=0x7fe64815f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.973+0000 7fe647fff640 1 -- 172.21.15.37:0/2163242651 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6480cc000 con 0x7fe64815b6b0 2024-04-25T09:03:42.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe647fff640 1 --2- 172.21.15.37:0/2163242651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe64815b6b0 0x7fe6480ca3c0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fe6300097c0 tx=0x7fe63001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a56884d04fc285 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:42.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe6477fe640 1 -- 172.21.15.37:0/2163242651 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6300a7070 con 0x7fe64815b6b0 2024-04-25T09:03:42.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe6477fe640 1 -- 172.21.15.37:0/2163242651 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6300b0070 con 0x7fe64815b6b0 2024-04-25T09:03:42.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe6477fe640 1 -- 172.21.15.37:0/2163242651 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6300ab890 con 0x7fe64815b6b0 2024-04-25T09:03:42.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 -- 172.21.15.37:0/2163242651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe64815b6b0 msgr2=0x7fe6480ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 --2- 172.21.15.37:0/2163242651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe64815b6b0 0x7fe6480ca3c0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7fe6300097c0 tx=0x7fe63001c470 comp rx=0 tx=0).stop 2024-04-25T09:03:42.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 -- 172.21.15.37:0/2163242651 shutdown_connections 2024-04-25T09:03:42.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 --2- 172.21.15.37:0/2163242651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe64815b6b0 0x7fe6480ca3c0 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 -- 172.21.15.37:0/2163242651 >> 172.21.15.37:0/2163242651 conn(0x7fe64806ae60 msgr2=0x7fe64806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:42.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 -- 172.21.15.37:0/2163242651 shutdown_connections 2024-04-25T09:03:42.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 -- 172.21.15.37:0/2163242651 wait complete. 2024-04-25T09:03:42.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 Processor -- start 2024-04-25T09:03:42.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 -- start start 2024-04-25T09:03:42.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe648073010 0x7fe6480c1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:42.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6480cf590 0x7fe648151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe64815b6b0 0x7fe648164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480ca900 con 0x7fe6480cf590 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.977+0000 7fe64e316640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480cba30 con 0x7fe64815b6b0 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe64e316640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6480cfdb0 con 0x7fe648073010 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe63ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6480cf590 0x7fe648151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe63ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6480cf590 0x7fe648151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50448/0 (socket says 172.21.15.37:50448) 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe63ffff640 1 -- 172.21.15.37:0/4258963178 learned_addr learned my addr 172.21.15.37:0/4258963178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe63ffff640 1 -- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe648073010 msgr2=0x7fe6480c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe64c8b3640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe64815b6b0 0x7fe648164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe647fff640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe648073010 0x7fe6480c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe63ffff640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe648073010 0x7fe6480c1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe63ffff640 1 -- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe64815b6b0 msgr2=0x7fe648164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:42.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe63ffff640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe64815b6b0 0x7fe648164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:42.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe63ffff640 1 -- 172.21.15.37:0/4258963178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6480b10e0 con 0x7fe6480cf590 2024-04-25T09:03:42.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe63ffff640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6480cf590 0x7fe648151db0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c4d0 tx=0x7fe63800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:42.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe6457fa640 1 -- 172.21.15.37:0/4258963178 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638017070 con 0x7fe6480cf590 2024-04-25T09:03:42.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe6457fa640 1 -- 172.21.15.37:0/4258963178 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe638005cf0 con 0x7fe6480cf590 2024-04-25T09:03:42.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe6457fa640 1 -- 172.21.15.37:0/4258963178 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638007270 con 0x7fe6480cf590 2024-04-25T09:03:42.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe64e316640 1 -- 172.21.15.37:0/4258963178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6480c1df0 con 0x7fe6480cf590 2024-04-25T09:03:42.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe64e316640 1 -- 172.21.15.37:0/4258963178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6480c23b0 con 0x7fe6480cf590 2024-04-25T09:03:42.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe64e316640 1 -- 172.21.15.37:0/4258963178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe64804efa0 con 0x7fe6480cf590 2024-04-25T09:03:42.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe6457fa640 1 -- 172.21.15.37:0/4258963178 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe63801a070 con 0x7fe6480cf590 2024-04-25T09:03:42.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe6457fa640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe628066f60 0x7fe628069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:42.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe6457fa640 1 -- 172.21.15.37:0/4258963178 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe638084470 con 0x7fe6480cf590 2024-04-25T09:03:42.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.981+0000 7fe647fff640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe628066f60 0x7fe628069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:42.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.985+0000 7fe647fff640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe628066f60 0x7fe628069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe6300af000 tx=0x7fe6300a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:42.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:42.985+0000 7fe6457fa640 1 -- 172.21.15.37:0/4258963178 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe638054c90 con 0x7fe6480cf590 2024-04-25T09:03:43.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.145+0000 7fe64e316640 1 -- 172.21.15.37:0/4258963178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6480b4090 con 0x7fe6480cf590 2024-04-25T09:03:43.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.145+0000 7fe6457fa640 1 -- 172.21.15.37:0/4258963178 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe638058d30 con 0x7fe6480cf590 2024-04-25T09:03:43.136 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:03:43.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 -- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe628066f60 msgr2=0x7fe628069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe628066f60 0x7fe628069420 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fe6300af000 tx=0x7fe6300a7000 comp rx=0 tx=0).stop 2024-04-25T09:03:43.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 -- 172.21.15.37:0/4258963178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6480cf590 msgr2=0x7fe648151db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6480cf590 0x7fe648151db0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fe63800c4d0 tx=0x7fe63800c990 comp rx=0 tx=0).stop 2024-04-25T09:03:43.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 -- 172.21.15.37:0/4258963178 shutdown_connections 2024-04-25T09:03:43.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe628066f60 0x7fe628069420 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe64815b6b0 0x7fe648164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6480cf590 0x7fe648151db0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 --2- 172.21.15.37:0/4258963178 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe648073010 0x7fe6480c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 -- 172.21.15.37:0/4258963178 >> 172.21.15.37:0/4258963178 conn(0x7fe64806ae60 msgr2=0x7fe64806e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 -- 172.21.15.37:0/4258963178 shutdown_connections 2024-04-25T09:03:43.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.149+0000 7fe64e316640 1 -- 172.21.15.37:0/4258963178 wait complete. 2024-04-25T09:03:43.155 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:43.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfc2bb640 1 Processor -- start 2024-04-25T09:03:43.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfc2bb640 1 -- start start 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfc2bb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8bf40d07f0 0x7f8bf40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfc2bb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40d1cd0 0x7f8bf406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfc2bb640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf406be60 con 0x7f8bf415da80 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfc2bb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf406c440 con 0x7f8bf40d07f0 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfc2bb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf415de60 con 0x7f8bf40d1cd0 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfa057640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8bf415da80 0x7f8bf40d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53114/0 (socket says 172.21.15.37:53114) 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfa057640 1 -- 172.21.15.37:0/12698116 learned_addr learned my addr 172.21.15.37:0/12698116 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfa858640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40d1cd0 0x7f8bf406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9856640 1 --2- 172.21.15.37:0/12698116 >> v2:172.21.15.203:3300/0 conn(0x7f8bf40d07f0 0x7f8bf40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1684362483 0 0) 0x7f8bf406be60 con 0x7f8bf415da80 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8be0003530 con 0x7f8bf415da80 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2965955033 0 0) 0x7f8be0003530 con 0x7f8bf415da80 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40d1cd0 msgr2=0x7f8bf406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40d1cd0 0x7f8bf406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 >> v2:172.21.15.203:3300/0 conn(0x7f8bf40d07f0 msgr2=0x7f8bf40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 --2- 172.21.15.37:0/12698116 >> v2:172.21.15.203:3300/0 conn(0x7f8bf40d07f0 0x7f8bf40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf40d2450 con 0x7f8bf415da80 2024-04-25T09:03:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f8bf0001a20 con 0x7f8bf415da80 2024-04-25T09:03:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 >> v1:172.21.15.37:6789/0 conn(0x7f8bf415da80 legacy=0x7f8bf40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40d1cd0 0x7f8be0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 0x7f8be0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8be0004d50 0x7f8be00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf406c440 con 0x7f8be0003fb0 2024-04-25T09:03:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf415de60 con 0x7f8be0004d50 2024-04-25T09:03:43.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be000ba00 con 0x7f8bf40d1cd0 2024-04-25T09:03:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9856640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 0x7f8be0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfa057640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40d1cd0 0x7f8be0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bfa858640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8be0004d50 0x7f8be00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9856640 1 -- 172.21.15.37:0/12698116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40d1cd0 msgr2=0x7f8be0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9856640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40d1cd0 0x7f8be0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9856640 1 -- 172.21.15.37:0/12698116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8be0004d50 msgr2=0x7f8be00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9856640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8be0004d50 0x7f8be00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.289+0000 7f8bf9856640 1 -- 172.21.15.37:0/12698116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be0003530 con 0x7f8be0003fb0 2024-04-25T09:03:43.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bf9856640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 0x7f8be0004810 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009ac0 tx=0x7f8be401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3ff841d08b4ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4009690 con 0x7f8be0003fb0 2024-04-25T09:03:43.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be40059a0 con 0x7f8be0003fb0 2024-04-25T09:03:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bf9055640 1 -- 172.21.15.37:0/12698116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be4005d90 con 0x7f8be0003fb0 2024-04-25T09:03:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/12698116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 msgr2=0x7f8be0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bfc2bb640 1 --2- 172.21.15.37:0/12698116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 0x7f8be0004810 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f8be4009ac0 tx=0x7f8be401ca90 comp rx=0 tx=0).stop 2024-04-25T09:03:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bfa858640 1 -- 172.21.15.37:0/12698116 reap_dead start 2024-04-25T09:03:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/12698116 shutdown_connections 2024-04-25T09:03:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/12698116 >> 172.21.15.37:0/12698116 conn(0x7f8bf408b000 msgr2=0x7f8bf40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/12698116 shutdown_connections 2024-04-25T09:03:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/12698116 wait complete. 2024-04-25T09:03:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bfc2bb640 1 Processor -- start 2024-04-25T09:03:43.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.293+0000 7f8bfc2bb640 1 -- start start 2024-04-25T09:03:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfc2bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 0x7f8bf40c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfc2bb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bf40c82b0 0x7f8bf40a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfc2bb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40a0dd0 0x7f8bf409d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfc2bb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf40d26f0 con 0x7f8be0003fb0 2024-04-25T09:03:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfc2bb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf40d2140 con 0x7f8bf40c82b0 2024-04-25T09:03:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfc2bb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf40d2870 con 0x7f8bf40a0dd0 2024-04-25T09:03:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfa057640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 0x7f8bf40c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfa057640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 0x7f8bf40c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50464/0 (socket says 172.21.15.37:50464) 2024-04-25T09:03:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfa057640 1 -- 172.21.15.37:0/1117205321 learned_addr learned my addr 172.21.15.37:0/1117205321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:43.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bf9856640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bf40c82b0 0x7f8bf40a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfa858640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40a0dd0 0x7f8bf409d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfa057640 1 -- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40a0dd0 msgr2=0x7f8bf409d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfa057640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40a0dd0 0x7f8bf409d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfa057640 1 -- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bf40c82b0 msgr2=0x7f8bf40a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfa057640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bf40c82b0 0x7f8bf40a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfa057640 1 -- 172.21.15.37:0/1117205321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf409f7f0 con 0x7f8be0003fb0 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfa057640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 0x7f8bf40c7d70 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0011840 tx=0x7f8bf0011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8beaffd640 1 -- 172.21.15.37:0/1117205321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0010040 con 0x7f8be0003fb0 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8beaffd640 1 -- 172.21.15.37:0/1117205321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf001a070 con 0x7f8be0003fb0 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/1117205321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf409fad0 con 0x7f8be0003fb0 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8beaffd640 1 -- 172.21.15.37:0/1117205321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000fda0 con 0x7f8be0003fb0 2024-04-25T09:03:43.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/1117205321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf40a0090 con 0x7f8be0003fb0 2024-04-25T09:03:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/1117205321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf404efa0 con 0x7f8be0003fb0 2024-04-25T09:03:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8beaffd640 1 -- 172.21.15.37:0/1117205321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8bf001f460 con 0x7f8be0003fb0 2024-04-25T09:03:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8beaffd640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8bd8066f60 0x7f8bd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.297+0000 7f8beaffd640 1 -- 172.21.15.37:0/1117205321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8bf0088e00 con 0x7f8be0003fb0 2024-04-25T09:03:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.301+0000 7f8bf9856640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8bd8066f60 0x7f8bd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.301+0000 7f8bf9856640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8bd8066f60 0x7f8bd8069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8be40ae000 tx=0x7f8be40046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.301+0000 7f8beaffd640 1 -- 172.21.15.37:0/1117205321 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8bf0059cf0 con 0x7f8be0003fb0 2024-04-25T09:03:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.461+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/1117205321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8bf40d44f0 con 0x7f8be0003fb0 2024-04-25T09:03:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.461+0000 7f8beaffd640 1 -- 172.21.15.37:0/1117205321 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8bf00595f0 con 0x7f8be0003fb0 2024-04-25T09:03:43.452 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:03:43.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8bd8066f60 msgr2=0x7f8bd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8bd8066f60 0x7f8bd8069420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8be40ae000 tx=0x7f8be40046d0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/1117205321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 msgr2=0x7f8bf40c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 0x7f8bf40c7d70 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0011840 tx=0x7f8bf0011d00 comp rx=0 tx=0).stop 2024-04-25T09:03:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/1117205321 shutdown_connections 2024-04-25T09:03:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8bd8066f60 0x7f8bd8069420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8bf40a0dd0 0x7f8bf409d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8bf40c82b0 0x7f8bf40a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 --2- 172.21.15.37:0/1117205321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8be0003fb0 0x7f8bf40c7d70 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/1117205321 >> 172.21.15.37:0/1117205321 conn(0x7f8bf408b000 msgr2=0x7f8bf40b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/1117205321 shutdown_connections 2024-04-25T09:03:43.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.465+0000 7f8bfc2bb640 1 -- 172.21.15.37:0/1117205321 wait complete. 2024-04-25T09:03:43.471 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda753ed640 1 Processor -- start 2024-04-25T09:03:43.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda753ed640 1 -- start start 2024-04-25T09:03:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda753ed640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda700cf590 0x7fda700cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda753ed640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fda70072000 0x7fda70072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda753ed640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda700cbd20 con 0x7fda7015b6b0 2024-04-25T09:03:43.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda753ed640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7015ba90 con 0x7fda70072000 2024-04-25T09:03:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda753ed640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda700cff30 con 0x7fda700cf590 2024-04-25T09:03:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6effd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fda7015b6b0 0x7fda700ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53120/0 (socket says 172.21.15.37:53120) 2024-04-25T09:03:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6effd640 1 -- 172.21.15.37:0/2922010695 learned_addr learned my addr 172.21.15.37:0/2922010695 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6f7fe640 1 --2- 172.21.15.37:0/2922010695 >> v2:172.21.15.203:3300/0 conn(0x7fda70072000 0x7fda70072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda67fff640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda700cf590 0x7fda700cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2673993173 0 0) 0x7fda700cbd20 con 0x7fda7015b6b0 2024-04-25T09:03:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda5c003530 con 0x7fda7015b6b0 2024-04-25T09:03:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 250891847 0 0) 0x7fda5c003530 con 0x7fda7015b6b0 2024-04-25T09:03:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda700cf590 msgr2=0x7fda700cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda700cf590 0x7fda700cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 >> v2:172.21.15.203:3300/0 conn(0x7fda70072000 msgr2=0x7fda70072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 --2- 172.21.15.37:0/2922010695 >> v2:172.21.15.203:3300/0 conn(0x7fda70072000 0x7fda70072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda700cc000 con 0x7fda7015b6b0 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fda60001680 con 0x7fda7015b6b0 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 >> v1:172.21.15.37:6789/0 conn(0x7fda7015b6b0 legacy=0x7fda700ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda5c003a30 0x7fda5c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda5c004810 0x7fda5c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda700cf590 0x7fda5c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda7015ba90 con 0x7fda5c004810 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c00b9d0 con 0x7fda5c003a30 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c00bb50 con 0x7fda700cf590 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda67fff640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda5c004810 0x7fda5c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6effd640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda5c003a30 0x7fda5c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda67fff640 1 -- 172.21.15.37:0/2922010695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda700cf590 msgr2=0x7fda5c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda67fff640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda700cf590 0x7fda5c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda67fff640 1 -- 172.21.15.37:0/2922010695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda5c003a30 msgr2=0x7fda5c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda67fff640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda5c003a30 0x7fda5c003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda67fff640 1 -- 172.21.15.37:0/2922010695 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c003530 con 0x7fda5c004810 2024-04-25T09:03:43.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.605+0000 7fda6effd640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda5c003a30 0x7fda5c003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:43.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda67fff640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda5c004810 0x7fda5c004c90 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fda58009af0 tx=0x7fda5801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858ea7751aef9b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda580b2070 con 0x7fda5c004810 2024-04-25T09:03:43.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda580a5580 con 0x7fda5c004810 2024-04-25T09:03:43.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda6e7fc640 1 -- 172.21.15.37:0/2922010695 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58009690 con 0x7fda5c004810 2024-04-25T09:03:43.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 -- 172.21.15.37:0/2922010695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda5c004810 msgr2=0x7fda5c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 --2- 172.21.15.37:0/2922010695 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda5c004810 0x7fda5c004c90 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fda58009af0 tx=0x7fda5801ca70 comp rx=0 tx=0).stop 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda6f7fe640 1 -- 172.21.15.37:0/2922010695 reap_dead start 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 -- 172.21.15.37:0/2922010695 shutdown_connections 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 -- 172.21.15.37:0/2922010695 >> 172.21.15.37:0/2922010695 conn(0x7fda7006ae60 msgr2=0x7fda7006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 -- 172.21.15.37:0/2922010695 shutdown_connections 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 -- 172.21.15.37:0/2922010695 wait complete. 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 Processor -- start 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 -- start start 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda5c004810 0x7fda700ac170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda7015c600 0x7fda700c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda700c6970 0x7fda700ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda700cc8a0 con 0x7fda7015c600 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda700cc5c0 con 0x7fda700c6970 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda753ed640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda700cba30 con 0x7fda5c004810 2024-04-25T09:03:43.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda7015c600 0x7fda700c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda67fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda7015c600 0x7fda700c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50484/0 (socket says 172.21.15.37:50484) 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.609+0000 7fda67fff640 1 -- 172.21.15.37:0/1444084030 learned_addr learned my addr 172.21.15.37:0/1444084030 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda6f7fe640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda700c6970 0x7fda700ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda6effd640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda5c004810 0x7fda700ac170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda67fff640 1 -- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda5c004810 msgr2=0x7fda700ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda67fff640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda5c004810 0x7fda700ac170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda67fff640 1 -- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda700c6970 msgr2=0x7fda700ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda67fff640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda700c6970 0x7fda700ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda67fff640 1 -- 172.21.15.37:0/1444084030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda700ac6b0 con 0x7fda7015c600 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda67fff640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda7015c600 0x7fda700c6430 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fda58009af0 tx=0x7fda5c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda677fe640 1 -- 172.21.15.37:0/1444084030 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda580be070 con 0x7fda7015c600 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda677fe640 1 -- 172.21.15.37:0/1444084030 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda580b7460 con 0x7fda7015c600 2024-04-25T09:03:43.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda753ed640 1 -- 172.21.15.37:0/1444084030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda700ac960 con 0x7fda7015c600 2024-04-25T09:03:43.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda677fe640 1 -- 172.21.15.37:0/1444084030 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda580b2040 con 0x7fda7015c600 2024-04-25T09:03:43.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda753ed640 1 -- 172.21.15.37:0/1444084030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda700acf20 con 0x7fda7015c600 2024-04-25T09:03:43.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda753ed640 1 -- 172.21.15.37:0/1444084030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda7004efa0 con 0x7fda7015c600 2024-04-25T09:03:43.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda677fe640 1 -- 172.21.15.37:0/1444084030 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fda580c1070 con 0x7fda7015c600 2024-04-25T09:03:43.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda677fe640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fda50066f90 0x7fda50069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda677fe640 1 -- 172.21.15.37:0/1444084030 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda5811ed80 con 0x7fda7015c600 2024-04-25T09:03:43.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda6effd640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fda50066f90 0x7fda50069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.613+0000 7fda6effd640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fda50066f90 0x7fda50069450 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fda60004880 tx=0x7fda60001520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.617+0000 7fda677fe640 1 -- 172.21.15.37:0/1444084030 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fda580167c0 con 0x7fda7015c600 2024-04-25T09:03:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.773+0000 7fda753ed640 1 -- 172.21.15.37:0/1444084030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda700b1ee0 con 0x7fda7015c600 2024-04-25T09:03:43.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.773+0000 7fda677fe640 1 -- 172.21.15.37:0/1444084030 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda580ef510 con 0x7fda7015c600 2024-04-25T09:03:43.762 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:03:43.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.773+0000 7fda753ed640 1 -- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fda50066f90 msgr2=0x7fda50069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.773+0000 7fda753ed640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fda50066f90 0x7fda50069450 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fda60004880 tx=0x7fda60001520 comp rx=0 tx=0).stop 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.773+0000 7fda753ed640 1 -- 172.21.15.37:0/1444084030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda7015c600 msgr2=0x7fda700c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.773+0000 7fda753ed640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda7015c600 0x7fda700c6430 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fda58009af0 tx=0x7fda5c005840 comp rx=0 tx=0).stop 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.773+0000 7fda753ed640 1 -- 172.21.15.37:0/1444084030 shutdown_connections 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.773+0000 7fda753ed640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fda50066f90 0x7fda50069450 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.773+0000 7fda753ed640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fda700c6970 0x7fda700ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.777+0000 7fda753ed640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda7015c600 0x7fda700c6430 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.777+0000 7fda753ed640 1 --2- 172.21.15.37:0/1444084030 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fda5c004810 0x7fda700ac170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.777+0000 7fda753ed640 1 -- 172.21.15.37:0/1444084030 >> 172.21.15.37:0/1444084030 conn(0x7fda7006ae60 msgr2=0x7fda70157350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.777+0000 7fda753ed640 1 -- 172.21.15.37:0/1444084030 shutdown_connections 2024-04-25T09:03:43.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.777+0000 7fda753ed640 1 -- 172.21.15.37:0/1444084030 wait complete. 2024-04-25T09:03:43.780 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:03:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab415c640 1 Processor -- start 2024-04-25T09:03:43.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab415c640 1 -- start start 2024-04-25T09:03:43.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab415c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aac0ce510 0x7f0aac0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab415c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0aac1451f0 0x7f0aac142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab415c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aac0cee30 con 0x7f0aac144ca0 2024-04-25T09:03:43.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab415c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aac0cc2b0 con 0x7f0aac1451f0 2024-04-25T09:03:43.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab415c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac06e290 con 0x7f0aac0ce510 2024-04-25T09:03:43.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab26f9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0aac1451f0 0x7f0aac142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab26f9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f0aac1451f0 0x7f0aac142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50494/0 (socket says 172.21.15.37:50494) 2024-04-25T09:03:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab26f9640 1 -- 172.21.15.37:0/1843272727 learned_addr learned my addr 172.21.15.37:0/1843272727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab26f9640 1 -- 172.21.15.37:0/1843272727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aac0ce510 msgr2=0x7f0aac0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab1ef8640 1 --2- 172.21.15.37:0/1843272727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aac0ce510 0x7f0aac0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab26f9640 1 --2- 172.21.15.37:0/1843272727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aac0ce510 0x7f0aac0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab26f9640 1 -- 172.21.15.37:0/1843272727 >> v1:172.21.15.203:6789/0 conn(0x7f0aac144ca0 legacy=0x7f0aac06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:43.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.905+0000 7f0ab26f9640 1 -- 172.21.15.37:0/1843272727 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac0ccc70 con 0x7f0aac1451f0 2024-04-25T09:03:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab26f9640 1 --2- 172.21.15.37:0/1843272727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0aac1451f0 0x7f0aac142680 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009e50 tx=0x7f0aa401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22da305e39d2bc38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab0ef6640 1 -- 172.21.15.37:0/1843272727 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40b4070 con 0x7f0aac1451f0 2024-04-25T09:03:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab0ef6640 1 -- 172.21.15.37:0/1843272727 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa40af400 con 0x7f0aac1451f0 2024-04-25T09:03:43.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab0ef6640 1 -- 172.21.15.37:0/1843272727 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40af7d0 con 0x7f0aac1451f0 2024-04-25T09:03:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 -- 172.21.15.37:0/1843272727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0aac1451f0 msgr2=0x7f0aac142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 --2- 172.21.15.37:0/1843272727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0aac1451f0 0x7f0aac142680 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009e50 tx=0x7f0aa401f1e0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 -- 172.21.15.37:0/1843272727 shutdown_connections 2024-04-25T09:03:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 --2- 172.21.15.37:0/1843272727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0aac1451f0 0x7f0aac142680 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 --2- 172.21.15.37:0/1843272727 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aac0ce510 0x7f0aac0ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 -- 172.21.15.37:0/1843272727 >> 172.21.15.37:0/1843272727 conn(0x7f0aac0b11a0 msgr2=0x7f0aac0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:43.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 -- 172.21.15.37:0/1843272727 shutdown_connections 2024-04-25T09:03:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 -- 172.21.15.37:0/1843272727 wait complete. 2024-04-25T09:03:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 Processor -- start 2024-04-25T09:03:43.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 -- start start 2024-04-25T09:03:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0aac0ce510 0x7f0aac144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aac144ca0 0x7f0aac135d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aac1451f0 0x7f0aac1e1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac06ee30 con 0x7f0aac0ce510 2024-04-25T09:03:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac06df20 con 0x7f0aac144ca0 2024-04-25T09:03:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab415c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0cc900 con 0x7f0aac1451f0 2024-04-25T09:03:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab1ef8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0aac0ce510 0x7f0aac144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab1ef8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0aac0ce510 0x7f0aac144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50508/0 (socket says 172.21.15.37:50508) 2024-04-25T09:03:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab1ef8640 1 -- 172.21.15.37:0/476954630 learned_addr learned my addr 172.21.15.37:0/476954630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab26f9640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aac1451f0 0x7f0aac1e1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab16f7640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aac144ca0 0x7f0aac135d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab1ef8640 1 -- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aac1451f0 msgr2=0x7f0aac1e1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.909+0000 7f0ab1ef8640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aac1451f0 0x7f0aac1e1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0ab1ef8640 1 -- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aac144ca0 msgr2=0x7f0aac135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0ab1ef8640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aac144ca0 0x7f0aac135d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0ab1ef8640 1 -- 172.21.15.37:0/476954630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac1e2250 con 0x7f0aac0ce510 2024-04-25T09:03:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0ab1ef8640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0aac0ce510 0x7f0aac144240 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800c820 tx=0x7f0aa800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0aa27fc640 1 -- 172.21.15.37:0/476954630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8015070 con 0x7f0aac0ce510 2024-04-25T09:03:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0aa27fc640 1 -- 172.21.15.37:0/476954630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa8007a30 con 0x7f0aac0ce510 2024-04-25T09:03:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0ab415c640 1 -- 172.21.15.37:0/476954630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aac1e2520 con 0x7f0aac0ce510 2024-04-25T09:03:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0aa27fc640 1 -- 172.21.15.37:0/476954630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa8007e20 con 0x7f0aac0ce510 2024-04-25T09:03:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0ab415c640 1 -- 172.21.15.37:0/476954630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac1e2ae0 con 0x7f0aac0ce510 2024-04-25T09:03:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0ab415c640 1 -- 172.21.15.37:0/476954630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aac04efa0 con 0x7f0aac0ce510 2024-04-25T09:03:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0aa27fc640 1 -- 172.21.15.37:0/476954630 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0aa8018070 con 0x7f0aac0ce510 2024-04-25T09:03:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0aa27fc640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a90066f60 0x7f0a90069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0aa27fc640 1 -- 172.21.15.37:0/476954630 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0aa80850e0 con 0x7f0aac0ce510 2024-04-25T09:03:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0ab16f7640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a90066f60 0x7f0a90069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.913+0000 7f0ab16f7640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a90066f60 0x7f0a90069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0b10d0 tx=0x7f0a9c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:43.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:43.917+0000 7f0aa27fc640 1 -- 172.21.15.37:0/476954630 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0aa8055f60 con 0x7f0aac0ce510 2024-04-25T09:03:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.085+0000 7f0ab415c640 1 -- 172.21.15.37:0/476954630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0aac0cd8e0 con 0x7f0aac0ce510 2024-04-25T09:03:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.085+0000 7f0aa27fc640 1 -- 172.21.15.37:0/476954630 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0aa8055860 con 0x7f0aac0ce510 2024-04-25T09:03:44.074 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T09:03:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 -- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a90066f60 msgr2=0x7f0a90069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a90066f60 0x7f0a90069420 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0b10d0 tx=0x7f0a9c009040 comp rx=0 tx=0).stop 2024-04-25T09:03:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 -- 172.21.15.37:0/476954630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0aac0ce510 msgr2=0x7f0aac144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0aac0ce510 0x7f0aac144240 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f0aa800c820 tx=0x7f0aa800cce0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 -- 172.21.15.37:0/476954630 shutdown_connections 2024-04-25T09:03:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0a90066f60 0x7f0a90069420 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0aac1451f0 0x7f0aac1e1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0aac144ca0 0x7f0aac135d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 --2- 172.21.15.37:0/476954630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0aac0ce510 0x7f0aac144240 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 -- 172.21.15.37:0/476954630 >> 172.21.15.37:0/476954630 conn(0x7f0aac0b11a0 msgr2=0x7f0aac0c5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 -- 172.21.15.37:0/476954630 shutdown_connections 2024-04-25T09:03:44.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.089+0000 7f0ab415c640 1 -- 172.21.15.37:0/476954630 wait complete. 2024-04-25T09:03:44.092 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:03:44.092 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:44.098 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:44.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.273+0000 7fb2e9dbf640 1 Processor -- start 2024-04-25T09:03:44.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.273+0000 7fb2e9dbf640 1 -- start start 2024-04-25T09:03:44.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e9dbf640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb2e415da80 0x7fb2e40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e9dbf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb2e40d1cd0 0x7fb2e40d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e9dbf640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e406be60 con 0x7fb2e40d0620 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e9dbf640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e40d0a70 con 0x7fb2e40d1cd0 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e9dbf640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2e406c440 con 0x7fb2e415da80 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e3fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb2e40d0620 0x7fb2e40ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53134/0 (socket says 172.21.15.37:53134) 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e3fff640 1 -- 172.21.15.37:0/443109781 learned_addr learned my addr 172.21.15.37:0/443109781 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e2ffd640 1 --2- 172.21.15.37:0/443109781 >> v2:172.21.15.37:3300/0 conn(0x7fb2e40d1cd0 0x7fb2e40d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e37fe640 1 --2- 172.21.15.37:0/443109781 >> v2:172.21.15.203:3300/0 conn(0x7fb2e415da80 0x7fb2e40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e27fc640 1 -- 172.21.15.37:0/443109781 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 949567487 0 0) 0x7fb2e406be60 con 0x7fb2e40d0620 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e27fc640 1 -- 172.21.15.37:0/443109781 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2cc003530 con 0x7fb2e40d0620 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e2ffd640 1 -- 172.21.15.37:0/443109781 >> v2:172.21.15.203:3300/0 conn(0x7fb2e415da80 msgr2=0x7fb2e40d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e2ffd640 1 --2- 172.21.15.37:0/443109781 >> v2:172.21.15.203:3300/0 conn(0x7fb2e415da80 0x7fb2e40d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e2ffd640 1 -- 172.21.15.37:0/443109781 >> v1:172.21.15.37:6789/0 conn(0x7fb2e40d0620 legacy=0x7fb2e40ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e2ffd640 1 -- 172.21.15.37:0/443109781 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e415de60 con 0x7fb2e40d1cd0 2024-04-25T09:03:44.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e2ffd640 1 --2- 172.21.15.37:0/443109781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e40d1cd0 0x7fb2e40d00e0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80097b0 tx=0x7fb2d801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc835d8cf71722e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e27fc640 1 -- 172.21.15.37:0/443109781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80b0070 con 0x7fb2e40d1cd0 2024-04-25T09:03:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e27fc640 1 -- 172.21.15.37:0/443109781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d80ab600 con 0x7fb2e40d1cd0 2024-04-25T09:03:44.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.277+0000 7fb2e27fc640 1 -- 172.21.15.37:0/443109781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80ab9d0 con 0x7fb2e40d1cd0 2024-04-25T09:03:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/443109781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e40d1cd0 msgr2=0x7fb2e40d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 --2- 172.21.15.37:0/443109781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e40d1cd0 0x7fb2e40d00e0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80097b0 tx=0x7fb2d801c460 comp rx=0 tx=0).stop 2024-04-25T09:03:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/443109781 shutdown_connections 2024-04-25T09:03:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 --2- 172.21.15.37:0/443109781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e40d1cd0 0x7fb2e40d00e0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 --2- 172.21.15.37:0/443109781 >> v2:172.21.15.203:3300/0 conn(0x7fb2e415da80 0x7fb2e40d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/443109781 >> 172.21.15.37:0/443109781 conn(0x7fb2e408b000 msgr2=0x7fb2e40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/443109781 shutdown_connections 2024-04-25T09:03:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/443109781 wait complete. 2024-04-25T09:03:44.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 Processor -- start 2024-04-25T09:03:44.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- start start 2024-04-25T09:03:44.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40d0620 0x7fb2e40b7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e40d1cd0 0x7fb2e4153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e415da80 0x7fb2e40b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e416a730 con 0x7fb2e415da80 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e416a5b0 con 0x7fb2e40d0620 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e416a8b0 con 0x7fb2e40d1cd0 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e415da80 0x7fb2e40b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e415da80 0x7fb2e40b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50524/0 (socket says 172.21.15.37:50524) 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e3fff640 1 -- 172.21.15.37:0/2096910391 learned_addr learned my addr 172.21.15.37:0/2096910391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e3fff640 1 -- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e40d1cd0 msgr2=0x7fb2e4153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e3fff640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e40d1cd0 0x7fb2e4153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e3fff640 1 -- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40d0620 msgr2=0x7fb2e40b7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e37fe640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40d0620 0x7fb2e40b7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e3fff640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40d0620 0x7fb2e40b7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e3fff640 1 -- 172.21.15.37:0/2096910391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e4154050 con 0x7fb2e415da80 2024-04-25T09:03:44.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e3fff640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e415da80 0x7fb2e40b6310 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00cb40 tx=0x7fb2dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2cbfff640 1 -- 172.21.15.37:0/2096910391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc017070 con 0x7fb2e415da80 2024-04-25T09:03:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2cbfff640 1 -- 172.21.15.37:0/2096910391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2dc00aca0 con 0x7fb2e415da80 2024-04-25T09:03:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/2096910391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e40b6ee0 con 0x7fb2e415da80 2024-04-25T09:03:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2cbfff640 1 -- 172.21.15.37:0/2096910391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc012620 con 0x7fb2e415da80 2024-04-25T09:03:44.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/2096910391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e40b7720 con 0x7fb2e415da80 2024-04-25T09:03:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.281+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/2096910391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e404efa0 con 0x7fb2e415da80 2024-04-25T09:03:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.285+0000 7fb2cbfff640 1 -- 172.21.15.37:0/2096910391 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb2dc01a070 con 0x7fb2e415da80 2024-04-25T09:03:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.285+0000 7fb2cbfff640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2d0066f60 0x7fb2d0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.285+0000 7fb2cbfff640 1 -- 172.21.15.37:0/2096910391 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb2dc084fd0 con 0x7fb2e415da80 2024-04-25T09:03:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.285+0000 7fb2e37fe640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2d0066f60 0x7fb2d0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.285+0000 7fb2e37fe640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2d0066f60 0x7fb2d0069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40096f0 tx=0x7fb2d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.289+0000 7fb2cbfff640 1 -- 172.21.15.37:0/2096910391 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fb2e40b6ee0 con 0x7fb2e415da80 2024-04-25T09:03:44.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.441+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/2096910391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2e40d10f0 con 0x7fb2e415da80 2024-04-25T09:03:44.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2cbfff640 1 -- 172.21.15.37:0/2096910391 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16916 (secure 0 0 0) 0x7fb2dc055e50 con 0x7fb2e415da80 2024-04-25T09:03:44.434 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:44.434 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":125,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:33.770823+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:44.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2d0066f60 msgr2=0x7fb2d0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2d0066f60 0x7fb2d0069420 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb2d40096f0 tx=0x7fb2d4008040 comp rx=0 tx=0).stop 2024-04-25T09:03:44.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/2096910391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e415da80 msgr2=0x7fb2e40b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e415da80 0x7fb2e40b6310 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00cb40 tx=0x7fb2dc00a060 comp rx=0 tx=0).stop 2024-04-25T09:03:44.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/2096910391 shutdown_connections 2024-04-25T09:03:44.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb2d0066f60 0x7fb2d0069420 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e415da80 0x7fb2e40b6310 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb2e40d1cd0 0x7fb2e4153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 --2- 172.21.15.37:0/2096910391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb2e40d0620 0x7fb2e40b7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/2096910391 >> 172.21.15.37:0/2096910391 conn(0x7fb2e408b000 msgr2=0x7fb2e40d14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/2096910391 shutdown_connections 2024-04-25T09:03:44.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.445+0000 7fb2e9dbf640 1 -- 172.21.15.37:0/2096910391 wait complete. 2024-04-25T09:03:44.452 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:30.763724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:31.488462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:44.452 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:44.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa15324c640 1 Processor -- start 2024-04-25T09:03:44.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa15324c640 1 -- start start 2024-04-25T09:03:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa15324c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c0cd350 0x7fa14c15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa15324c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa14c0cf0e0 0x7fa14c15b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa15324c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14c0ce320 con 0x7fa14c15aac0 2024-04-25T09:03:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa15324c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14c0ce0d0 con 0x7fa14c0cf0e0 2024-04-25T09:03:44.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa15324c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0ce600 con 0x7fa14c0cd350 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa150fe8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa14c15aac0 0x7fa14c0ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53138/0 (socket says 172.21.15.37:53138) 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa150fe8640 1 -- 172.21.15.37:0/3787301938 learned_addr learned my addr 172.21.15.37:0/3787301938 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa1517e9640 1 --2- 172.21.15.37:0/3787301938 >> v2:172.21.15.203:3300/0 conn(0x7fa14c0cf0e0 0x7fa14c15b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14bfff640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c0cd350 0x7fa14c15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3555920862 0 0) 0x7fa14c0ce320 con 0x7fa14c15aac0 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa138003530 con 0x7fa14c15aac0 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1156586271 0 0) 0x7fa138003530 con 0x7fa14c15aac0 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c0cd350 msgr2=0x7fa14c15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c0cd350 0x7fa14c15aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 >> v2:172.21.15.203:3300/0 conn(0x7fa14c0cf0e0 msgr2=0x7fa14c15b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 --2- 172.21.15.37:0/3787301938 >> v2:172.21.15.203:3300/0 conn(0x7fa14c0cf0e0 0x7fa14c15b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0ce8e0 con 0x7fa14c15aac0 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fa144001a00 con 0x7fa14c15aac0 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 >> v1:172.21.15.37:6789/0 conn(0x7fa14c15aac0 legacy=0x7fa14c0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c0cd350 0x7fa138003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa138003fb0 0x7fa138004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 0x7fa1380051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0ce0d0 con 0x7fa138004d50 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0ce600 con 0x7fa138003fb0 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa13800ba00 con 0x7fa14c0cd350 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa1517e9640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 0x7fa1380051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa14bfff640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa138003fb0 0x7fa138004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa150fe8640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c0cd350 0x7fa138003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa1517e9640 1 -- 172.21.15.37:0/3787301938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c0cd350 msgr2=0x7fa138003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa1517e9640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c0cd350 0x7fa138003a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa1517e9640 1 -- 172.21.15.37:0/3787301938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa138003fb0 msgr2=0x7fa138004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa1517e9640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa138003fb0 0x7fa138004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.589+0000 7fa1517e9640 1 -- 172.21.15.37:0/3787301938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa138003530 con 0x7fa138004d50 2024-04-25T09:03:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa1517e9640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 0x7fa1380051d0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009ae0 tx=0x7fa13c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f597640175ff46d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0b2070 con 0x7fa138004d50 2024-04-25T09:03:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa13c0a56a0 con 0x7fa138004d50 2024-04-25T09:03:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa14b7fe640 1 -- 172.21.15.37:0/3787301938 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0096b0 con 0x7fa138004d50 2024-04-25T09:03:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa15324c640 1 -- 172.21.15.37:0/3787301938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 msgr2=0x7fa1380051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa15324c640 1 --2- 172.21.15.37:0/3787301938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 0x7fa1380051d0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009ae0 tx=0x7fa13c01cab0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa1517e9640 1 -- 172.21.15.37:0/3787301938 reap_dead start 2024-04-25T09:03:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa15324c640 1 -- 172.21.15.37:0/3787301938 shutdown_connections 2024-04-25T09:03:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa15324c640 1 -- 172.21.15.37:0/3787301938 >> 172.21.15.37:0/3787301938 conn(0x7fa14c099a70 msgr2=0x7fa14c0d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa15324c640 1 -- 172.21.15.37:0/3787301938 shutdown_connections 2024-04-25T09:03:44.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa15324c640 1 -- 172.21.15.37:0/3787301938 wait complete. 2024-04-25T09:03:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.593+0000 7fa15324c640 1 Processor -- start 2024-04-25T09:03:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa15324c640 1 -- start start 2024-04-25T09:03:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa15324c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 0x7fa14c0b5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa15324c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c14cce0 0x7fa14c0b5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa15324c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14c1666e0 0x7fa14c166b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa15324c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c15baa0 con 0x7fa138004d50 2024-04-25T09:03:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa15324c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c15b920 con 0x7fa14c1666e0 2024-04-25T09:03:44.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa15324c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c15aea0 con 0x7fa14c14cce0 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa150fe8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 0x7fa14c0b5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa150fe8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 0x7fa14c0b5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50542/0 (socket says 172.21.15.37:50542) 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa150fe8640 1 -- 172.21.15.37:0/1671351997 learned_addr learned my addr 172.21.15.37:0/1671351997 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa14bfff640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c14cce0 0x7fa14c0b5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa1517e9640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14c1666e0 0x7fa14c166b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa150fe8640 1 -- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c14cce0 msgr2=0x7fa14c0b5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa150fe8640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c14cce0 0x7fa14c0b5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa150fe8640 1 -- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14c1666e0 msgr2=0x7fa14c166b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa150fe8640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14c1666e0 0x7fa14c166b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa150fe8640 1 -- 172.21.15.37:0/1671351997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c165640 con 0x7fa138004d50 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa150fe8640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 0x7fa14c0b5420 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fa144002220 tx=0x7fa14400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa1497fa640 1 -- 172.21.15.37:0/1671351997 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144019070 con 0x7fa138004d50 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa1497fa640 1 -- 172.21.15.37:0/1671351997 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa144004d70 con 0x7fa138004d50 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa1497fa640 1 -- 172.21.15.37:0/1671351997 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1440072d0 con 0x7fa138004d50 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa15324c640 1 -- 172.21.15.37:0/1671351997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14c165920 con 0x7fa138004d50 2024-04-25T09:03:44.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa15324c640 1 -- 172.21.15.37:0/1671351997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c165ee0 con 0x7fa138004d50 2024-04-25T09:03:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa15324c640 1 -- 172.21.15.37:0/1671351997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14c04efa0 con 0x7fa138004d50 2024-04-25T09:03:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa1497fa640 1 -- 172.21.15.37:0/1671351997 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa14401c070 con 0x7fa138004d50 2024-04-25T09:03:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa1497fa640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa12c066f90 0x7fa12c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa1497fa640 1 -- 172.21.15.37:0/1671351997 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1440853b0 con 0x7fa138004d50 2024-04-25T09:03:44.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.597+0000 7fa14bfff640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa12c066f90 0x7fa12c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.601+0000 7fa14bfff640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa12c066f90 0x7fa12c069450 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0c71f0 tx=0x7fa134008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.601+0000 7fa1497fa640 1 -- 172.21.15.37:0/1671351997 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa144056230 con 0x7fa138004d50 2024-04-25T09:03:44.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.761+0000 7fa15324c640 1 -- 172.21.15.37:0/1671351997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa14c154900 con 0x7fa138004d50 2024-04-25T09:03:44.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.761+0000 7fa1497fa640 1 -- 172.21.15.37:0/1671351997 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa144055b30 con 0x7fa138004d50 2024-04-25T09:03:44.751 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:03:44.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 -- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa12c066f90 msgr2=0x7fa12c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa12c066f90 0x7fa12c069450 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fa14c0c71f0 tx=0x7fa134008040 comp rx=0 tx=0).stop 2024-04-25T09:03:44.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 -- 172.21.15.37:0/1671351997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 msgr2=0x7fa14c0b5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 0x7fa14c0b5420 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fa144002220 tx=0x7fa14400b560 comp rx=0 tx=0).stop 2024-04-25T09:03:44.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 -- 172.21.15.37:0/1671351997 shutdown_connections 2024-04-25T09:03:44.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa12c066f90 0x7fa12c069450 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa14c1666e0 0x7fa14c166b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa14c14cce0 0x7fa14c0b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 --2- 172.21.15.37:0/1671351997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa138004d50 0x7fa14c0b5420 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 -- 172.21.15.37:0/1671351997 >> 172.21.15.37:0/1671351997 conn(0x7fa14c099a70 msgr2=0x7fa14c0ae640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 -- 172.21.15.37:0/1671351997 shutdown_connections 2024-04-25T09:03:44.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.765+0000 7fa15324c640 1 -- 172.21.15.37:0/1671351997 wait complete. 2024-04-25T09:03:44.770 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6f591e640 1 Processor -- start 2024-04-25T09:03:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6f591e640 1 -- start start 2024-04-25T09:03:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6f591e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00cf590 0x7fe6f00cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6f591e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6f0072000 0x7fe6f0072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6f591e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f00cbd20 con 0x7fe6f015b6b0 2024-04-25T09:03:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6f591e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f015ba90 con 0x7fe6f0072000 2024-04-25T09:03:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6f591e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00cff30 con 0x7fe6f00cf590 2024-04-25T09:03:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6eeffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe6f015b6b0 0x7fe6f00ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:53144/0 (socket says 172.21.15.37:53144) 2024-04-25T09:03:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6eeffd640 1 -- 172.21.15.37:0/763350434 learned_addr learned my addr 172.21.15.37:0/763350434 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6ef7fe640 1 --2- 172.21.15.37:0/763350434 >> v2:172.21.15.203:3300/0 conn(0x7fe6f0072000 0x7fe6f0072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6ee7fc640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00cf590 0x7fe6f00cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459104555 0 0) 0x7fe6f00cbd20 con 0x7fe6f015b6b0 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6d8003530 con 0x7fe6f015b6b0 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 387232301 0 0) 0x7fe6d8003530 con 0x7fe6f015b6b0 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00cf590 msgr2=0x7fe6f00cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00cf590 0x7fe6f00cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 >> v2:172.21.15.203:3300/0 conn(0x7fe6f0072000 msgr2=0x7fe6f0072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 --2- 172.21.15.37:0/763350434 >> v2:172.21.15.203:3300/0 conn(0x7fe6f0072000 0x7fe6f0072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f00cc000 con 0x7fe6f015b6b0 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fe6e0001680 con 0x7fe6f015b6b0 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 >> v1:172.21.15.37:6789/0 conn(0x7fe6f015b6b0 legacy=0x7fe6f00ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d8003a30 0x7fe6d8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00cf590 0x7fe6d8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d8004d50 0x7fe6d80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f015ba90 con 0x7fe6d8004d50 2024-04-25T09:03:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d800b9d0 con 0x7fe6d8003a30 2024-04-25T09:03:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d800bb50 con 0x7fe6f00cf590 2024-04-25T09:03:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6ef7fe640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d8004d50 0x7fe6d80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6eeffd640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d8003a30 0x7fe6d8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6ee7fc640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00cf590 0x7fe6d8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6ef7fe640 1 -- 172.21.15.37:0/763350434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00cf590 msgr2=0x7fe6d8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6ef7fe640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00cf590 0x7fe6d8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6ef7fe640 1 -- 172.21.15.37:0/763350434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d8003a30 msgr2=0x7fe6d8003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6ef7fe640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d8003a30 0x7fe6d8003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.905+0000 7fe6ef7fe640 1 -- 172.21.15.37:0/763350434 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d8003530 con 0x7fe6d8004d50 2024-04-25T09:03:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6ef7fe640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d8004d50 0x7fe6d80051d0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009fc0 tx=0x7fe6e801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3ba58078f2d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e80b2070 con 0x7fe6d8004d50 2024-04-25T09:03:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e80a5cd0 con 0x7fe6d8004d50 2024-04-25T09:03:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6edffb640 1 -- 172.21.15.37:0/763350434 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e8009c10 con 0x7fe6d8004d50 2024-04-25T09:03:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6f591e640 1 -- 172.21.15.37:0/763350434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d8004d50 msgr2=0x7fe6d80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6f591e640 1 --2- 172.21.15.37:0/763350434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6d8004d50 0x7fe6d80051d0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8009fc0 tx=0x7fe6e801ccc0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6ef7fe640 1 -- 172.21.15.37:0/763350434 reap_dead start 2024-04-25T09:03:44.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6f591e640 1 -- 172.21.15.37:0/763350434 shutdown_connections 2024-04-25T09:03:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6f591e640 1 -- 172.21.15.37:0/763350434 >> 172.21.15.37:0/763350434 conn(0x7fe6f006ae60 msgr2=0x7fe6f006b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6f591e640 1 -- 172.21.15.37:0/763350434 shutdown_connections 2024-04-25T09:03:44.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6f591e640 1 -- 172.21.15.37:0/763350434 wait complete. 2024-04-25T09:03:44.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6f591e640 1 Processor -- start 2024-04-25T09:03:44.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.909+0000 7fe6f591e640 1 -- start start 2024-04-25T09:03:44.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6f591e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d8004d50 0x7fe6f015e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6f591e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f006e520 0x7fe6f00c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6f591e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00b35d0 0x7fe6f00b3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6f591e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00cc8a0 con 0x7fe6f006e520 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6f591e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00cc5c0 con 0x7fe6d8004d50 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6f591e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00cba30 con 0x7fe6f00b35d0 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6ee7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f006e520 0x7fe6f00c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6ee7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f006e520 0x7fe6f00c48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50568/0 (socket says 172.21.15.37:50568) 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6ee7fc640 1 -- 172.21.15.37:0/159418936 learned_addr learned my addr 172.21.15.37:0/159418936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6ee7fc640 1 -- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00b35d0 msgr2=0x7fe6f00b3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6eeffd640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d8004d50 0x7fe6f015e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6ef7fe640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00b35d0 0x7fe6f00b3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6ee7fc640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00b35d0 0x7fe6f00b3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6ee7fc640 1 -- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d8004d50 msgr2=0x7fe6f015e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6ee7fc640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d8004d50 0x7fe6f015e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:44.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6ee7fc640 1 -- 172.21.15.37:0/159418936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f015c560 con 0x7fe6f006e520 2024-04-25T09:03:44.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6ee7fc640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f006e520 0x7fe6f00c48d0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4010890 tx=0x7fe6e4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6d7fff640 1 -- 172.21.15.37:0/159418936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400f040 con 0x7fe6f006e520 2024-04-25T09:03:44.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6d7fff640 1 -- 172.21.15.37:0/159418936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e400eb60 con 0x7fe6f006e520 2024-04-25T09:03:44.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6f591e640 1 -- 172.21.15.37:0/159418936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f00c5240 con 0x7fe6f006e520 2024-04-25T09:03:44.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6d7fff640 1 -- 172.21.15.37:0/159418936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e401c400 con 0x7fe6f006e520 2024-04-25T09:03:44.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6f591e640 1 -- 172.21.15.37:0/159418936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f00c5800 con 0x7fe6f006e520 2024-04-25T09:03:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6f591e640 1 -- 172.21.15.37:0/159418936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f004efa0 con 0x7fe6f006e520 2024-04-25T09:03:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6d7fff640 1 -- 172.21.15.37:0/159418936 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6e401c5a0 con 0x7fe6f006e520 2024-04-25T09:03:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6d7fff640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6dc066f90 0x7fe6dc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6d7fff640 1 -- 172.21.15.37:0/159418936 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe6e4085790 con 0x7fe6f006e520 2024-04-25T09:03:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.913+0000 7fe6eeffd640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6dc066f90 0x7fe6dc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.917+0000 7fe6eeffd640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6dc066f90 0x7fe6dc069450 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0001ec0 tx=0x7fe6e0000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:44.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:44.917+0000 7fe6d7fff640 1 -- 172.21.15.37:0/159418936 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6f015c560 con 0x7fe6f006e520 2024-04-25T09:03:45.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.077+0000 7fe6f591e640 1 -- 172.21.15.37:0/159418936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe6f0073170 con 0x7fe6f006e520 2024-04-25T09:03:45.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6d7fff640 1 -- 172.21.15.37:0/159418936 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe6e4018030 con 0x7fe6f006e520 2024-04-25T09:03:45.069 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:03:45.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 -- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6dc066f90 msgr2=0x7fe6dc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6dc066f90 0x7fe6dc069450 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0001ec0 tx=0x7fe6e0000ee0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 -- 172.21.15.37:0/159418936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f006e520 msgr2=0x7fe6f00c48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f006e520 0x7fe6f00c48d0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4010890 tx=0x7fe6e4010d50 comp rx=0 tx=0).stop 2024-04-25T09:03:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 -- 172.21.15.37:0/159418936 shutdown_connections 2024-04-25T09:03:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6dc066f90 0x7fe6dc069450 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6f00b35d0 0x7fe6f00b3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f006e520 0x7fe6f00c48d0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 --2- 172.21.15.37:0/159418936 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6d8004d50 0x7fe6f015e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 -- 172.21.15.37:0/159418936 >> 172.21.15.37:0/159418936 conn(0x7fe6f006ae60 msgr2=0x7fe6f0073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 -- 172.21.15.37:0/159418936 shutdown_connections 2024-04-25T09:03:45.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.081+0000 7fe6f591e640 1 -- 172.21.15.37:0/159418936 wait complete. 2024-04-25T09:03:45.089 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:45.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe5635db640 1 Processor -- start 2024-04-25T09:03:45.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe5635db640 1 -- start start 2024-04-25T09:03:45.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe5635db640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe55c0d0130 0x7fe55c0cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe5635db640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe55c0d1fe0 0x7fe55c161c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe5635db640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55c0d2530 con 0x7fe55c0d0510 2024-04-25T09:03:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe5635db640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c0d2810 con 0x7fe55c0d1fe0 2024-04-25T09:03:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe5635db640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55c0b3160 con 0x7fe55c0d0130 2024-04-25T09:03:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe560b76640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe55c0d1fe0 0x7fe55c161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe560b76640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe55c0d1fe0 0x7fe55c161c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50580/0 (socket says 172.21.15.37:50580) 2024-04-25T09:03:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe560b76640 1 -- 172.21.15.37:0/4058386503 learned_addr learned my addr 172.21.15.37:0/4058386503 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe561377640 1 --2- 172.21.15.37:0/4058386503 >> v2:172.21.15.203:3300/0 conn(0x7fe55c0d0130 0x7fe55c0cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe560b76640 1 -- 172.21.15.37:0/4058386503 >> v2:172.21.15.203:3300/0 conn(0x7fe55c0d0130 msgr2=0x7fe55c0cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe560b76640 1 --2- 172.21.15.37:0/4058386503 >> v2:172.21.15.203:3300/0 conn(0x7fe55c0d0130 0x7fe55c0cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe560b76640 1 -- 172.21.15.37:0/4058386503 >> v1:172.21.15.203:6789/0 conn(0x7fe55c0d0510 legacy=0x7fe55c162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.233+0000 7fe560b76640 1 -- 172.21.15.37:0/4058386503 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c0b3440 con 0x7fe55c0d1fe0 2024-04-25T09:03:45.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe560b76640 1 --2- 172.21.15.37:0/4058386503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c0d1fe0 0x7fe55c161c20 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fe55c08baa0 tx=0x7fe54c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeed5efe883c71b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe553fff640 1 -- 172.21.15.37:0/4058386503 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0b0070 con 0x7fe55c0d1fe0 2024-04-25T09:03:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe553fff640 1 -- 172.21.15.37:0/4058386503 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c0ab600 con 0x7fe55c0d1fe0 2024-04-25T09:03:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe553fff640 1 -- 172.21.15.37:0/4058386503 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0ab9d0 con 0x7fe55c0d1fe0 2024-04-25T09:03:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 -- 172.21.15.37:0/4058386503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c0d1fe0 msgr2=0x7fe55c161c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 --2- 172.21.15.37:0/4058386503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c0d1fe0 0x7fe55c161c20 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fe55c08baa0 tx=0x7fe54c01c520 comp rx=0 tx=0).stop 2024-04-25T09:03:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 -- 172.21.15.37:0/4058386503 shutdown_connections 2024-04-25T09:03:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 --2- 172.21.15.37:0/4058386503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c0d1fe0 0x7fe55c161c20 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 --2- 172.21.15.37:0/4058386503 >> v2:172.21.15.203:3300/0 conn(0x7fe55c0d0130 0x7fe55c0cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 -- 172.21.15.37:0/4058386503 >> 172.21.15.37:0/4058386503 conn(0x7fe55c08b000 msgr2=0x7fe55c0d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 -- 172.21.15.37:0/4058386503 shutdown_connections 2024-04-25T09:03:45.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 -- 172.21.15.37:0/4058386503 wait complete. 2024-04-25T09:03:45.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 Processor -- start 2024-04-25T09:03:45.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 -- start start 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe55c0d0130 0x7fe55c1539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c0d0510 0x7fe55c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe55c0d1fe0 0x7fe55c0cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c0b4450 con 0x7fe55c0d0510 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c162a80 con 0x7fe55c0d1fe0 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe5635db640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55c0b45d0 con 0x7fe55c0d0130 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe560b76640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c0d0510 0x7fe55c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe560b76640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c0d0510 0x7fe55c0cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50596/0 (socket says 172.21.15.37:50596) 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe560b76640 1 -- 172.21.15.37:0/1937141770 learned_addr learned my addr 172.21.15.37:0/1937141770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe561377640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe55c0d0130 0x7fe55c1539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe561b78640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe55c0d1fe0 0x7fe55c0cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe560b76640 1 -- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe55c0d0130 msgr2=0x7fe55c1539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe560b76640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe55c0d0130 0x7fe55c1539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe560b76640 1 -- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe55c0d1fe0 msgr2=0x7fe55c0cce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe560b76640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe55c0d1fe0 0x7fe55c0cce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe560b76640 1 -- 172.21.15.37:0/1937141770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55c153ee0 con 0x7fe55c0d0510 2024-04-25T09:03:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.237+0000 7fe561377640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe55c0d0130 0x7fe55c1539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe560b76640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c0d0510 0x7fe55c0cc410 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fe54c01abd0 tx=0x7fe54c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe551ffb640 1 -- 172.21.15.37:0/1937141770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0ba020 con 0x7fe55c0d0510 2024-04-25T09:03:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe551ffb640 1 -- 172.21.15.37:0/1937141770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c0b6420 con 0x7fe55c0d0510 2024-04-25T09:03:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe551ffb640 1 -- 172.21.15.37:0/1937141770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c0b0040 con 0x7fe55c0d0510 2024-04-25T09:03:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe5635db640 1 -- 172.21.15.37:0/1937141770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55c0b7af0 con 0x7fe55c0d0510 2024-04-25T09:03:45.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe5635db640 1 -- 172.21.15.37:0/1937141770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55c0b6d30 con 0x7fe55c0d0510 2024-04-25T09:03:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe5635db640 1 -- 172.21.15.37:0/1937141770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55c04efa0 con 0x7fe55c0d0510 2024-04-25T09:03:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe551ffb640 1 -- 172.21.15.37:0/1937141770 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe54c0bd020 con 0x7fe55c0d0510 2024-04-25T09:03:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe551ffb640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe540066f60 0x7fe540069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe551ffb640 1 -- 172.21.15.37:0/1937141770 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe54c11f3a0 con 0x7fe55c0d0510 2024-04-25T09:03:45.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe561377640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe540066f60 0x7fe540069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.241+0000 7fe561377640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe540066f60 0x7fe540069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe558009840 tx=0x7fe558008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.245+0000 7fe551ffb640 1 -- 172.21.15.37:0/1937141770 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe54c0c4030 con 0x7fe55c0d0510 2024-04-25T09:03:45.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 -- 172.21.15.37:0/1937141770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe55c06ad40 con 0x7fe55c0d0510 2024-04-25T09:03:45.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe551ffb640 1 -- 172.21.15.37:0/1937141770 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe54c016110 con 0x7fe55c0d0510 2024-04-25T09:03:45.393 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:03:45.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 -- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe540066f60 msgr2=0x7fe540069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe540066f60 0x7fe540069420 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe558009840 tx=0x7fe558008040 comp rx=0 tx=0).stop 2024-04-25T09:03:45.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 -- 172.21.15.37:0/1937141770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c0d0510 msgr2=0x7fe55c0cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c0d0510 0x7fe55c0cc410 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7fe54c01abd0 tx=0x7fe54c0192c0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 -- 172.21.15.37:0/1937141770 shutdown_connections 2024-04-25T09:03:45.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe540066f60 0x7fe540069420 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe55c0d1fe0 0x7fe55c0cce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe55c0d0510 0x7fe55c0cc410 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 --2- 172.21.15.37:0/1937141770 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe55c0d0130 0x7fe55c1539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 -- 172.21.15.37:0/1937141770 >> 172.21.15.37:0/1937141770 conn(0x7fe55c08b000 msgr2=0x7fe55c15a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 -- 172.21.15.37:0/1937141770 shutdown_connections 2024-04-25T09:03:45.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.405+0000 7fe5635db640 1 -- 172.21.15.37:0/1937141770 wait complete. 2024-04-25T09:03:45.411 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:03:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.541+0000 7fa059ffb640 1 Processor -- start 2024-04-25T09:03:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.541+0000 7fa059ffb640 1 -- start start 2024-04-25T09:03:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa059ffb640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa05415da80 0x7fa0540d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa059ffb640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa0540d1cd0 0x7fa0540d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa059ffb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0540d0620 0x7fa0540ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa059ffb640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05406c440 con 0x7fa05415da80 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa059ffb640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05406be60 con 0x7fa0540d1cd0 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa059ffb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05415de60 con 0x7fa0540d0620 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa0537fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa05415da80 0x7fa0540d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa0537fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fa05415da80 0x7fa0540d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50602/0 (socket says 172.21.15.37:50602) 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa0537fe640 1 -- 172.21.15.37:0/2425746459 learned_addr learned my addr 172.21.15.37:0/2425746459 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa052ffd640 1 --2- 172.21.15.37:0/2425746459 >> v2:172.21.15.203:3300/0 conn(0x7fa0540d1cd0 0x7fa0540d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa053fff640 1 --2- 172.21.15.37:0/2425746459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0540d0620 0x7fa0540ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa0537fe640 1 -- 172.21.15.37:0/2425746459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0540d0620 msgr2=0x7fa0540ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa0537fe640 1 --2- 172.21.15.37:0/2425746459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0540d0620 0x7fa0540ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa0537fe640 1 -- 172.21.15.37:0/2425746459 >> v2:172.21.15.203:3300/0 conn(0x7fa0540d1cd0 msgr2=0x7fa0540d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa0537fe640 1 --2- 172.21.15.37:0/2425746459 >> v2:172.21.15.203:3300/0 conn(0x7fa0540d1cd0 0x7fa0540d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa0537fe640 1 -- 172.21.15.37:0/2425746459 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540d2450 con 0x7fa05415da80 2024-04-25T09:03:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa052ffd640 1 --2- 172.21.15.37:0/2425746459 >> v2:172.21.15.203:3300/0 conn(0x7fa0540d1cd0 0x7fa0540d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:45.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa053fff640 1 --2- 172.21.15.37:0/2425746459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0540d0620 0x7fa0540ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.545+0000 7fa0537fe640 1 --2- 172.21.15.37:0/2425746459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05415da80 0x7fa0540d4230 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fa044009870 tx=0x7fa04401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575f39427bd4f4f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0527fc640 1 -- 172.21.15.37:0/2425746459 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0440b0070 con 0x7fa05415da80 2024-04-25T09:03:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0527fc640 1 -- 172.21.15.37:0/2425746459 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0440ab600 con 0x7fa05415da80 2024-04-25T09:03:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0527fc640 1 -- 172.21.15.37:0/2425746459 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0440ab9d0 con 0x7fa05415da80 2024-04-25T09:03:45.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 -- 172.21.15.37:0/2425746459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05415da80 msgr2=0x7fa0540d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 --2- 172.21.15.37:0/2425746459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05415da80 0x7fa0540d4230 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fa044009870 tx=0x7fa04401c520 comp rx=0 tx=0).stop 2024-04-25T09:03:45.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 -- 172.21.15.37:0/2425746459 shutdown_connections 2024-04-25T09:03:45.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 --2- 172.21.15.37:0/2425746459 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa0540d0620 0x7fa0540ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 --2- 172.21.15.37:0/2425746459 >> v2:172.21.15.203:3300/0 conn(0x7fa0540d1cd0 0x7fa0540d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 --2- 172.21.15.37:0/2425746459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa05415da80 0x7fa0540d4230 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 -- 172.21.15.37:0/2425746459 >> 172.21.15.37:0/2425746459 conn(0x7fa05408b000 msgr2=0x7fa0540d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 -- 172.21.15.37:0/2425746459 shutdown_connections 2024-04-25T09:03:45.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 -- 172.21.15.37:0/2425746459 wait complete. 2024-04-25T09:03:45.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 Processor -- start 2024-04-25T09:03:45.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 -- start start 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0540d0620 0x7fa054150fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0540d1cd0 0x7fa0540b6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa05415da80 0x7fa0540b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05406ca50 con 0x7fa0540d0620 2024-04-25T09:03:45.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05406c8d0 con 0x7fa0540d1cd0 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05406cbd0 con 0x7fa05415da80 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0540d0620 0x7fa054150fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0540d0620 0x7fa054150fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50612/0 (socket says 172.21.15.37:50612) 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0537fe640 1 -- 172.21.15.37:0/343818018 learned_addr learned my addr 172.21.15.37:0/343818018 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa052ffd640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0540d1cd0 0x7fa0540b6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa053fff640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa05415da80 0x7fa0540b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0537fe640 1 -- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa05415da80 msgr2=0x7fa0540b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0537fe640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa05415da80 0x7fa0540b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0537fe640 1 -- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0540d1cd0 msgr2=0x7fa0540b6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0537fe640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0540d1cd0 0x7fa0540b6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0537fe640 1 -- 172.21.15.37:0/343818018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0540b7190 con 0x7fa0540d0620 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa0537fe640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0540d0620 0x7fa054150fa0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fa0440ae040 tx=0x7fa0440192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa03bfff640 1 -- 172.21.15.37:0/343818018 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0440b0040 con 0x7fa0540d0620 2024-04-25T09:03:45.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa03bfff640 1 -- 172.21.15.37:0/343818018 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0440b9070 con 0x7fa0540d0620 2024-04-25T09:03:45.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 -- 172.21.15.37:0/343818018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0540b7470 con 0x7fa0540d0620 2024-04-25T09:03:45.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa03bfff640 1 -- 172.21.15.37:0/343818018 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0440b48a0 con 0x7fa0540d0620 2024-04-25T09:03:45.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.549+0000 7fa059ffb640 1 -- 172.21.15.37:0/343818018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0540b7a30 con 0x7fa0540d0620 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.553+0000 7fa059ffb640 1 -- 172.21.15.37:0/343818018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05404efa0 con 0x7fa0540d0620 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.553+0000 7fa03bfff640 1 -- 172.21.15.37:0/343818018 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa0440ab4c0 con 0x7fa0540d0620 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.553+0000 7fa03bfff640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa040066f90 0x7fa040069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.553+0000 7fa03bfff640 1 -- 172.21.15.37:0/343818018 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa04411efa0 con 0x7fa0540d0620 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.553+0000 7fa052ffd640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa040066f90 0x7fa040069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.553+0000 7fa052ffd640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa040066f90 0x7fa040069450 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa0540a1250 tx=0x7fa048008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.557+0000 7fa03bfff640 1 -- 172.21.15.37:0/343818018 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa044016110 con 0x7fa0540d0620 2024-04-25T09:03:45.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.721+0000 7fa059ffb640 1 -- 172.21.15.37:0/343818018 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa0540b2390 con 0x7fa0540d0620 2024-04-25T09:03:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa03bfff640 1 -- 172.21.15.37:0/343818018 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa0440efd90 con 0x7fa0540d0620 2024-04-25T09:03:45.713 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T09:03:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 -- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa040066f90 msgr2=0x7fa040069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa040066f90 0x7fa040069450 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fa0540a1250 tx=0x7fa048008040 comp rx=0 tx=0).stop 2024-04-25T09:03:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 -- 172.21.15.37:0/343818018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0540d0620 msgr2=0x7fa054150fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0540d0620 0x7fa054150fa0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fa0440ae040 tx=0x7fa0440192c0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 -- 172.21.15.37:0/343818018 shutdown_connections 2024-04-25T09:03:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa040066f90 0x7fa040069450 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa05415da80 0x7fa0540b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa0540d1cd0 0x7fa0540b6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 --2- 172.21.15.37:0/343818018 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0540d0620 0x7fa054150fa0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 -- 172.21.15.37:0/343818018 >> 172.21.15.37:0/343818018 conn(0x7fa05408b000 msgr2=0x7fa0540b1100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 -- 172.21.15.37:0/343818018 shutdown_connections 2024-04-25T09:03:45.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:45.725+0000 7fa059ffb640 1 -- 172.21.15.37:0/343818018 wait complete. 2024-04-25T09:03:45.730 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T09:03:45.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:45.730 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:45.741 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[54358]: fuse finished with error 0 and tester_r 0 2024-04-25T09:03:45.742 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:51.844 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:03:51.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:51.844 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:03:51.852 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:03:51.853 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T09:03:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.041+0000 7f91de635640 1 Processor -- start 2024-04-25T09:03:52.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.041+0000 7f91de635640 1 -- start start 2024-04-25T09:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91de635640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f91d815b920 0x7f91d8099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91de635640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d80b0440 0x7f91d8099fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91de635640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d809a950 con 0x7f91d80aeb80 2024-04-25T09:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91de635640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d809ac30 con 0x7f91d815b920 2024-04-25T09:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91de635640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d80ae380 con 0x7f91d80b0440 2024-04-25T09:03:52.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91d7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f91d815b920 0x7f91d8099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91dcbd2640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f91d80aeb80 0x7f91d80aeff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50738/0 (socket says 172.21.15.37:50738) 2024-04-25T09:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91dcbd2640 1 -- 172.21.15.37:0/3737619383 learned_addr learned my addr 172.21.15.37:0/3737619383 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91d77fe640 1 --2- 172.21.15.37:0/3737619383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d80b0440 0x7f91d8099fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91d7fff640 1 -- 172.21.15.37:0/3737619383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d80b0440 msgr2=0x7f91d8099fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91d7fff640 1 --2- 172.21.15.37:0/3737619383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d80b0440 0x7f91d8099fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91d7fff640 1 -- 172.21.15.37:0/3737619383 >> v1:172.21.15.37:6789/0 conn(0x7f91d80aeb80 legacy=0x7f91d80aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.045+0000 7f91d7fff640 1 -- 172.21.15.37:0/3737619383 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d80ae660 con 0x7f91d815b920 2024-04-25T09:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d7fff640 1 --2- 172.21.15.37:0/3737619383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91d815b920 0x7f91d8099a70 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0097c0 tx=0x7f91cc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a479112b7ef124a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d6ffd640 1 -- 172.21.15.37:0/3737619383 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0b1070 con 0x7f91d815b920 2024-04-25T09:03:52.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d6ffd640 1 -- 172.21.15.37:0/3737619383 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc0ac410 con 0x7f91d815b920 2024-04-25T09:03:52.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d6ffd640 1 -- 172.21.15.37:0/3737619383 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0ac7e0 con 0x7f91d815b920 2024-04-25T09:03:52.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 -- 172.21.15.37:0/3737619383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91d815b920 msgr2=0x7f91d8099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 --2- 172.21.15.37:0/3737619383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91d815b920 0x7f91d8099a70 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0097c0 tx=0x7f91cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 -- 172.21.15.37:0/3737619383 shutdown_connections 2024-04-25T09:03:52.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 --2- 172.21.15.37:0/3737619383 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d80b0440 0x7f91d8099fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 --2- 172.21.15.37:0/3737619383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91d815b920 0x7f91d8099a70 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 -- 172.21.15.37:0/3737619383 >> 172.21.15.37:0/3737619383 conn(0x7f91d80b3010 msgr2=0x7f91d80b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 -- 172.21.15.37:0/3737619383 shutdown_connections 2024-04-25T09:03:52.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 -- 172.21.15.37:0/3737619383 wait complete. 2024-04-25T09:03:52.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 Processor -- start 2024-04-25T09:03:52.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 -- start start 2024-04-25T09:03:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91d80aeb80 0x7f91d81529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d80b0440 0x7f91d81587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d815b920 0x7f91d8151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8168850 con 0x7f91d80aeb80 2024-04-25T09:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d81686d0 con 0x7f91d815b920 2024-04-25T09:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91de635640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d81689d0 con 0x7f91d80b0440 2024-04-25T09:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91d80aeb80 0x7f91d81529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91d80aeb80 0x7f91d81529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45468/0 (socket says 172.21.15.37:45468) 2024-04-25T09:03:52.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d7fff640 1 -- 172.21.15.37:0/1814363930 learned_addr learned my addr 172.21.15.37:0/1814363930 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91dcbd2640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d815b920 0x7f91d8151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d77fe640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d80b0440 0x7f91d81587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d7fff640 1 -- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d80b0440 msgr2=0x7f91d81587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d7fff640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d80b0440 0x7f91d81587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d7fff640 1 -- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d815b920 msgr2=0x7f91d8151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d7fff640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d815b920 0x7f91d8151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d7fff640 1 -- 172.21.15.37:0/1814363930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8151dc0 con 0x7f91d80aeb80 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.049+0000 7f91d7fff640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91d80aeb80 0x7f91d81529d0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f91cc01a0d0 tx=0x7f91cc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91d4ff9640 1 -- 172.21.15.37:0/1814363930 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0b1040 con 0x7f91d80aeb80 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91d4ff9640 1 -- 172.21.15.37:0/1814363930 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc0bc070 con 0x7f91d80aeb80 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91d4ff9640 1 -- 172.21.15.37:0/1814363930 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0b5630 con 0x7f91d80aeb80 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91de635640 1 -- 172.21.15.37:0/1814363930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d81530d0 con 0x7f91d80aeb80 2024-04-25T09:03:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91de635640 1 -- 172.21.15.37:0/1814363930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d80b59f0 con 0x7f91d80aeb80 2024-04-25T09:03:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91d4ff9640 1 -- 172.21.15.37:0/1814363930 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91cc0b57d0 con 0x7f91d80aeb80 2024-04-25T09:03:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91d4ff9640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91c4066f60 0x7f91c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91d4ff9640 1 -- 172.21.15.37:0/1814363930 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91cc11ed70 con 0x7f91d80aeb80 2024-04-25T09:03:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91d77fe640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91c4066f60 0x7f91c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91e17a2000 1 Processor -- start 2024-04-25T09:03:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91e17a2000 1 -- start start 2024-04-25T09:03:52.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91d77fe640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91c4066f60 0x7f91c4069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f91d8157c50 tx=0x7f91d000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d06e30 0x55cb55d07210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d07750 0x55cb55cbcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55cbd4b0 0x55cb55cbd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91e17a2000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb55b05ff0 con 0x55cb55d06e30 2024-04-25T09:03:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91e17a2000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb55a8e150 con 0x55cb55d07750 2024-04-25T09:03:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.053+0000 7f91e17a2000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb55b02090 con 0x55cb55cbd4b0 2024-04-25T09:03:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d06e30 0x55cb55d07210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d06e30 0x55cb55d07210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45470/0 (socket says 172.21.15.37:45470) 2024-04-25T09:03:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 -- 172.21.15.37:0/4249272143 learned_addr learned my addr 172.21.15.37:0/4249272143 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d77fe640 1 --2- 172.21.15.37:0/4249272143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d07750 0x55cb55cbcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91dcbd2640 1 --2- 172.21.15.37:0/4249272143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55cbd4b0 0x55cb55cbd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 -- 172.21.15.37:0/4249272143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55cbd4b0 msgr2=0x55cb55cbd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 --2- 172.21.15.37:0/4249272143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55cbd4b0 0x55cb55cbd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 -- 172.21.15.37:0/4249272143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d07750 msgr2=0x55cb55cbcf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 --2- 172.21.15.37:0/4249272143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d07750 0x55cb55cbcf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 -- 172.21.15.37:0/4249272143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb55c9c0d0 con 0x55cb55d06e30 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 --2- 172.21.15.37:0/4249272143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d06e30 0x55cb55d07210 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0ef3e0 tx=0x7f91cc0eedb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb6dfca4b640f6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91bdffb640 1 -- 172.21.15.37:0/4249272143 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0ef450 con 0x55cb55d06e30 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91bdffb640 1 -- 172.21.15.37:0/4249272143 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc0f35a0 con 0x55cb55d06e30 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91bdffb640 1 -- 172.21.15.37:0/4249272143 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0f7b60 con 0x55cb55d06e30 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 -- 172.21.15.37:0/4249272143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d06e30 msgr2=0x55cb55d07210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 --2- 172.21.15.37:0/4249272143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d06e30 0x55cb55d07210 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0ef3e0 tx=0x7f91cc0eedb0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 -- 172.21.15.37:0/4249272143 shutdown_connections 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 --2- 172.21.15.37:0/4249272143 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55cbd4b0 0x55cb55cbd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 --2- 172.21.15.37:0/4249272143 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d07750 0x55cb55cbcf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 --2- 172.21.15.37:0/4249272143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d06e30 0x55cb55d07210 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 -- 172.21.15.37:0/4249272143 >> 172.21.15.37:0/4249272143 conn(0x55cb55cb5020 msgr2=0x55cb55cb7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 -- 172.21.15.37:0/4249272143 shutdown_connections 2024-04-25T09:03:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 -- 172.21.15.37:0/4249272143 wait complete. 2024-04-25T09:03:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 -- start start 2024-04-25T09:03:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 Processor -- start 2024-04-25T09:03:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55cbd4b0 0x55cb55d067e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55d06e30 0x55cb55cc3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d07750 0x55cb55cc38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb55b0a620 con 0x55cb55cbd4b0 2024-04-25T09:03:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb55ac0690 con 0x55cb55d07750 2024-04-25T09:03:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb55a8ea40 con 0x55cb55d06e30 2024-04-25T09:03:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91e17a2000 10 client.0 _ll_register_callbacks cb 0x55cb55ca3e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55cbd4b0 0x55cb55d067e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55cbd4b0 0x55cb55d067e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45472/0 (socket says 172.21.15.37:45472) 2024-04-25T09:03:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 -- 172.21.15.37:0/3042247850 learned_addr learned my addr 172.21.15.37:0/3042247850 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d77fe640 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55d06e30 0x55cb55cc3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 -- 172.21.15.37:0/3042247850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55d06e30 msgr2=0x55cb55cc3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55d06e30 0x55cb55cc3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91dcbd2640 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d07750 0x55cb55cc38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 -- 172.21.15.37:0/3042247850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d07750 msgr2=0x55cb55cc38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d07750 0x55cb55cc38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 -- 172.21.15.37:0/3042247850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb55bb31c0 con 0x55cb55cbd4b0 2024-04-25T09:03:52.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d77fe640 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55d06e30 0x55cb55cc3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:52.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91d7fff640 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55cbd4b0 0x55cb55d067e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f91cc097e80 tx=0x7f91cc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f6867dd815ae80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:03:52.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91b6ffd640 1 -- 172.21.15.37:0/3042247850 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc0f3370 con 0x55cb55cbd4b0 2024-04-25T09:03:52.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.057+0000 7f91b6ffd640 1 -- 172.21.15.37:0/3042247850 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91cc0f76d0 con 0x55cb55cbd4b0 2024-04-25T09:03:52.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 -- 172.21.15.37:0/3042247850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91cc0f35a0 con 0x55cb55cbd4b0 2024-04-25T09:03:52.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91b6ffd640 1 -- 172.21.15.37:0/3042247850 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91cc0ef450 con 0x55cb55cbd4b0 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 10 client.9672 fetch_fsmap learned FSMap version 97 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 -- 172.21.15.37:0/3042247850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) v3 -- 0x55cb55baf170 con 0x55cb55cbd4b0 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91b6ffd640 1 -- 172.21.15.37:0/3042247850 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc0f35a0 con 0x55cb55cbd4b0 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91b6ffd640 1 -- 172.21.15.37:0/3042247850 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 97) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91cc0b4050 con 0x55cb55cbd4b0 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 10 client.9672 fetch_fsmap finished waiting for FSMap version 97 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 10 client.9672 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 20 client.9672 populate_metadata read hostname 'smithi037' 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x55cb55cc45e0 0x55cb55d190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 4 client.9672 mds_command: new command op to 9447 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 -- 172.21.15.37:0/3042247850 --> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb55aa1c60 con 0x55cb55cc45e0 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91d77fe640 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x55cb55cc45e0 0x55cb55d190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91d77fe640 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x55cb55cc45e0 0x55cb55d190f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d9a90b2c6b9dd2c server_cookie=b09740dbf1ec15ec in_seq=0 out_seq=0 2024-04-25T09:03:52.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91b6ffd640 10 client.9672 ms_handle_connect on v2:172.21.15.37:6834/2748919477 2024-04-25T09:03:52.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91b6ffd640 1 -- 172.21.15.37:0/3042247850 <== mds.0 v2:172.21.15.37:6834/2748919477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cb55aa1c60 con 0x55cb55cc45e0 2024-04-25T09:03:52.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91b6ffd640 10 client.9672 handle_command_reply: tid=0 2024-04-25T09:03:52.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 client.9672 shutdown 2024-04-25T09:03:52.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 -- 172.21.15.37:0/3042247850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55cbd4b0 msgr2=0x55cb55d067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55cbd4b0 0x55cb55d067e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f91cc097e80 tx=0x7f91cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 -- 172.21.15.37:0/3042247850 shutdown_connections 2024-04-25T09:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x55cb55cc45e0 0x55cb55d190f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d07750 0x55cb55cc38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55d06e30 0x55cb55cc3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 --2- 172.21.15.37:0/3042247850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55cbd4b0 0x55cb55d067e0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91b6ffd640 0 client.9672 ms_handle_reset on v2:172.21.15.37:6834/2748919477 2024-04-25T09:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 -- 172.21.15.37:0/3042247850 >> 172.21.15.37:0/3042247850 conn(0x55cb55cb5020 msgr2=0x55cb55cbe120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 -- 172.21.15.37:0/3042247850 shutdown_connections 2024-04-25T09:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 1 -- 172.21.15.37:0/3042247850 wait complete. 2024-04-25T09:03:52.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.061+0000 7f91e17a2000 20 client.9672 trim_cache size 0 max 16384 2024-04-25T09:03:52.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.073+0000 7f91e17a2000 1 Processor -- start 2024-04-25T09:03:52.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.073+0000 7f91e17a2000 1 -- start start 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55cbd4b0 0x55cb55ce5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55d06e30 0x55cb55ce6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d8157580 con 0x55cb55d07750 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d806dcd0 con 0x55cb55cbd4b0 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d806d0d0 con 0x55cb55d06e30 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d142a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45486/0 (socket says 172.21.15.37:45486) 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 -- 172.21.15.37:0/2974665303 learned_addr learned my addr 172.21.15.37:0/2974665303 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91d7fff640 1 --2- 172.21.15.37:0/2974665303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55cbd4b0 0x55cb55ce5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91d77fe640 1 --2- 172.21.15.37:0/2974665303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55d06e30 0x55cb55ce6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 -- 172.21.15.37:0/2974665303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55d06e30 msgr2=0x55cb55ce6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 --2- 172.21.15.37:0/2974665303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55d06e30 0x55cb55ce6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 -- 172.21.15.37:0/2974665303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55cbd4b0 msgr2=0x55cb55ce5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 --2- 172.21.15.37:0/2974665303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55cbd4b0 0x55cb55ce5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 -- 172.21.15.37:0/2974665303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb55b4cde0 con 0x55cb55d07750 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91d7fff640 1 --2- 172.21.15.37:0/2974665303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55cbd4b0 0x55cb55ce5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 --2- 172.21.15.37:0/2974665303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d142a0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006c00 tx=0x7f91c8007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35432a554414b726 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91bd7fa640 1 -- 172.21.15.37:0/2974665303 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c801b070 con 0x55cb55d07750 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91bd7fa640 1 -- 172.21.15.37:0/2974665303 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c8007740 con 0x55cb55d07750 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91bd7fa640 1 -- 172.21.15.37:0/2974665303 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8006350 con 0x55cb55d07750 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- 172.21.15.37:0/2974665303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 msgr2=0x55cb55d142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 --2- 172.21.15.37:0/2974665303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d142a0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f91c8006c00 tx=0x7f91c8007410 comp rx=0 tx=0).stop 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- 172.21.15.37:0/2974665303 shutdown_connections 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 --2- 172.21.15.37:0/2974665303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d142a0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 --2- 172.21.15.37:0/2974665303 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55d06e30 0x55cb55ce6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 --2- 172.21.15.37:0/2974665303 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55cbd4b0 0x55cb55ce5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- 172.21.15.37:0/2974665303 >> 172.21.15.37:0/2974665303 conn(0x55cb55cb5020 msgr2=0x55cb55d13b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- 172.21.15.37:0/2974665303 shutdown_connections 2024-04-25T09:03:52.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- 172.21.15.37:0/2974665303 wait complete. 2024-04-25T09:03:52.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- start start 2024-04-25T09:03:52.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 Processor -- start 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55cbd4b0 0x55cb55d190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d06e30 0x55cb55d19630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d19b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb55aa0e00 con 0x55cb55d07750 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d806d6d0 con 0x55cb55d06e30 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cb55b05300 con 0x55cb55cbd4b0 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91e17a2000 10 client.0 _ll_register_callbacks cb 0x55cb55c84040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d19b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d19b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45496/0 (socket says 172.21.15.37:45496) 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 -- 172.21.15.37:0/465726440 learned_addr learned my addr 172.21.15.37:0/465726440 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91d7fff640 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55cbd4b0 0x55cb55d190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91d77fe640 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d06e30 0x55cb55d19630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 -- 172.21.15.37:0/465726440 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55cbd4b0 msgr2=0x55cb55d190f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55cbd4b0 0x55cb55d190f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 -- 172.21.15.37:0/465726440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d06e30 msgr2=0x55cb55d19630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d06e30 0x55cb55d19630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91dcbd2640 1 -- 172.21.15.37:0/465726440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cb55aa78e0 con 0x55cb55d07750 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91d7fff640 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55cbd4b0 0x55cb55d190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.077+0000 7f91d77fe640 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d06e30 0x55cb55d19630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91dcbd2640 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d19b70 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f91c8007450 tx=0x7f91c8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e374805e5d633f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91b6ffd640 1 -- 172.21.15.37:0/465726440 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8027060 con 0x55cb55d07750 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91b6ffd640 1 -- 172.21.15.37:0/465726440 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91c80047f0 con 0x55cb55d07750 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91e17a2000 1 -- 172.21.15.37:0/465726440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb55b13530 con 0x55cb55d07750 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91b6ffd640 1 -- 172.21.15.37:0/465726440 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=97) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91c802d050 con 0x55cb55d07750 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91e17a2000 10 client.9678 fetch_fsmap learned FSMap version 97 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91e17a2000 1 -- 172.21.15.37:0/465726440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=97,monmap=2+}) v3 -- 0x55cb55b5f8f0 con 0x55cb55d07750 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91b6ffd640 1 -- 172.21.15.37:0/465726440 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cb55b13530 con 0x55cb55d07750 2024-04-25T09:03:52.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91b6ffd640 1 -- 172.21.15.37:0/465726440 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 97) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f91c8030020 con 0x55cb55d07750 2024-04-25T09:03:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91e17a2000 10 client.9678 fetch_fsmap finished waiting for FSMap version 97 2024-04-25T09:03:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91e17a2000 10 client.9678 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:03:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91e17a2000 20 client.9678 populate_metadata read hostname 'smithi037' 2024-04-25T09:03:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91e17a2000 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x55cb55d1a6f0 0x55cb55d6efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91e17a2000 4 client.9678 mds_command: new command op to 9447 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T09:03:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91e17a2000 1 -- 172.21.15.37:0/465726440 --> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb55b0d880 con 0x55cb55d1a6f0 2024-04-25T09:03:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91d7fff640 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x55cb55d1a6f0 0x55cb55d6efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91d7fff640 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x55cb55d1a6f0 0x55cb55d6efe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be552ac999868cd1 server_cookie=755b553b290594ad in_seq=0 out_seq=0 2024-04-25T09:03:52.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.081+0000 7f91b6ffd640 10 client.9678 ms_handle_connect on v2:172.21.15.37:6834/2748919477 2024-04-25T09:03:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91b6ffd640 1 -- 172.21.15.37:0/465726440 <== mds.0 v2:172.21.15.37:6834/2748919477 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb55b0d880 con 0x55cb55d1a6f0 2024-04-25T09:03:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91b6ffd640 10 client.9678 handle_command_reply: tid=0 2024-04-25T09:03:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91e17a2000 1 client.9678 shutdown 2024-04-25T09:03:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91e17a2000 1 -- 172.21.15.37:0/465726440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 msgr2=0x55cb55d19b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91e17a2000 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d19b70 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f91c8007450 tx=0x7f91c8000bd0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91e17a2000 1 -- 172.21.15.37:0/465726440 shutdown_connections 2024-04-25T09:03:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91e17a2000 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.37:6834/2748919477,v1:172.21.15.37:6835/2748919477] conn(0x55cb55d1a6f0 0x55cb55d6efe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91e17a2000 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb55d07750 0x55cb55d19b70 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91b6ffd640 0 client.9678 ms_handle_reset on v2:172.21.15.37:6834/2748919477 2024-04-25T09:03:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91e17a2000 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55cb55d06e30 0x55cb55d19630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91e17a2000 1 --2- 172.21.15.37:0/465726440 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55cb55cbd4b0 0x55cb55d190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.093+0000 7f91e17a2000 1 -- 172.21.15.37:0/465726440 >> 172.21.15.37:0/465726440 conn(0x55cb55cb5020 msgr2=0x55cb55d14420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91e17a2000 1 -- 172.21.15.37:0/465726440 shutdown_connections 2024-04-25T09:03:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91e17a2000 1 -- 172.21.15.37:0/465726440 wait complete. 2024-04-25T09:03:52.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91e17a2000 20 client.9678 trim_cache size 0 max 16384 2024-04-25T09:03:52.087 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:03:52.087 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-04-25T09:03:52.087 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-04-25T09:03:52.087 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 -- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91c4066f60 msgr2=0x7f91c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91c4066f60 0x7f91c4069420 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f91d8157c50 tx=0x7f91d000b040 comp rx=0 tx=0).stop 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 -- 172.21.15.37:0/1814363930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91d80aeb80 msgr2=0x7f91d81529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91d80aeb80 0x7f91d81529d0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f91cc01a0d0 tx=0x7f91cc019240 comp rx=0 tx=0).stop 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 -- 172.21.15.37:0/1814363930 shutdown_connections 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f91c4066f60 0x7f91c4069420 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91d815b920 0x7f91d8151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f91d80b0440 0x7f91d81587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 --2- 172.21.15.37:0/1814363930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91d80aeb80 0x7f91d81529d0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 -- 172.21.15.37:0/1814363930 >> 172.21.15.37:0/1814363930 conn(0x7f91d80b3010 msgr2=0x7f91d80cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 -- 172.21.15.37:0/1814363930 shutdown_connections 2024-04-25T09:03:52.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.097+0000 7f91de635640 1 -- 172.21.15.37:0/1814363930 wait complete. 2024-04-25T09:03:52.102 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:03:52.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc496a73640 1 Processor -- start 2024-04-25T09:03:52.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc496a73640 1 -- start start 2024-04-25T09:03:52.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc496a73640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc49015ed40 0x7fc4900b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc496a73640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc4900b2a70 0x7fc4900cc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc496a73640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4900b1000 con 0x7fc49015f120 2024-04-25T09:03:52.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc496a73640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4900b1610 con 0x7fc49015ed40 2024-04-25T09:03:52.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc496a73640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4900d0500 con 0x7fc4900b2a70 2024-04-25T09:03:52.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc49480f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc49015ed40 0x7fc4900b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc49480f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc49015ed40 0x7fc4900b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45500/0 (socket says 172.21.15.37:45500) 2024-04-25T09:03:52.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc49480f640 1 -- 172.21.15.37:0/4146904116 learned_addr learned my addr 172.21.15.37:0/4146904116 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:52.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc495010640 1 --2- 172.21.15.37:0/4146904116 >> v2:172.21.15.203:3300/0 conn(0x7fc4900b2a70 0x7fc4900cc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc49480f640 1 -- 172.21.15.37:0/4146904116 >> v2:172.21.15.203:3300/0 conn(0x7fc4900b2a70 msgr2=0x7fc4900cc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:52.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc49480f640 1 --2- 172.21.15.37:0/4146904116 >> v2:172.21.15.203:3300/0 conn(0x7fc4900b2a70 0x7fc4900cc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc49480f640 1 -- 172.21.15.37:0/4146904116 >> v1:172.21.15.203:6789/0 conn(0x7fc49015f120 legacy=0x7fc4900b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.229+0000 7fc49480f640 1 -- 172.21.15.37:0/4146904116 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4900d0220 con 0x7fc49015ed40 2024-04-25T09:03:52.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc49480f640 1 --2- 172.21.15.37:0/4146904116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc49015ed40 0x7fc4900b1aa0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fc490073450 tx=0x7fc48001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7447b2c9cd38cbe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc48f7fe640 1 -- 172.21.15.37:0/4146904116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800aa670 con 0x7fc49015ed40 2024-04-25T09:03:52.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc48f7fe640 1 -- 172.21.15.37:0/4146904116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4800af070 con 0x7fc49015ed40 2024-04-25T09:03:52.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc48f7fe640 1 -- 172.21.15.37:0/4146904116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4800aaa30 con 0x7fc49015ed40 2024-04-25T09:03:52.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 -- 172.21.15.37:0/4146904116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc49015ed40 msgr2=0x7fc4900b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 --2- 172.21.15.37:0/4146904116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc49015ed40 0x7fc4900b1aa0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fc490073450 tx=0x7fc48001c4f0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 -- 172.21.15.37:0/4146904116 shutdown_connections 2024-04-25T09:03:52.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 --2- 172.21.15.37:0/4146904116 >> v2:172.21.15.203:3300/0 conn(0x7fc4900b2a70 0x7fc4900cc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 --2- 172.21.15.37:0/4146904116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc49015ed40 0x7fc4900b1aa0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 -- 172.21.15.37:0/4146904116 >> 172.21.15.37:0/4146904116 conn(0x7fc4900b3020 msgr2=0x7fc4900b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 -- 172.21.15.37:0/4146904116 shutdown_connections 2024-04-25T09:03:52.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 -- 172.21.15.37:0/4146904116 wait complete. 2024-04-25T09:03:52.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 Processor -- start 2024-04-25T09:03:52.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 -- start start 2024-04-25T09:03:52.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4900b2a70 0x7fc490154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc49015ed40 0x7fc49006d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc49015f120 0x7fc49006bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc490168780 con 0x7fc49015f120 2024-04-25T09:03:52.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc490168600 con 0x7fc4900b2a70 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc496a73640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc490168900 con 0x7fc49015ed40 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc495010640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc49015f120 0x7fc49006bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc495010640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc49015f120 0x7fc49006bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45508/0 (socket says 172.21.15.37:45508) 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc495010640 1 -- 172.21.15.37:0/2894828612 learned_addr learned my addr 172.21.15.37:0/2894828612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.233+0000 7fc48ffff640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc49015ed40 0x7fc49006d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc49480f640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4900b2a70 0x7fc490154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc495010640 1 -- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc49015ed40 msgr2=0x7fc49006d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc495010640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc49015ed40 0x7fc49006d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc495010640 1 -- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4900b2a70 msgr2=0x7fc490154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc495010640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4900b2a70 0x7fc490154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc495010640 1 -- 172.21.15.37:0/2894828612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc490155000 con 0x7fc49015f120 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc495010640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc49015f120 0x7fc49006bda0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fc488007e60 tx=0x7fc48800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc48d7fa640 1 -- 172.21.15.37:0/2894828612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488014420 con 0x7fc49015f120 2024-04-25T09:03:52.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc48d7fa640 1 -- 172.21.15.37:0/2894828612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc488019070 con 0x7fc49015f120 2024-04-25T09:03:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc48d7fa640 1 -- 172.21.15.37:0/2894828612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488014820 con 0x7fc49015f120 2024-04-25T09:03:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc496a73640 1 -- 172.21.15.37:0/2894828612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc490165300 con 0x7fc49015f120 2024-04-25T09:03:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc496a73640 1 -- 172.21.15.37:0/2894828612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4900cb010 con 0x7fc49015f120 2024-04-25T09:03:52.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc496a73640 1 -- 172.21.15.37:0/2894828612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49004efa0 con 0x7fc49015f120 2024-04-25T09:03:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc48d7fa640 1 -- 172.21.15.37:0/2894828612 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4880149c0 con 0x7fc49015f120 2024-04-25T09:03:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc48d7fa640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc47c066f60 0x7fc47c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc48d7fa640 1 -- 172.21.15.37:0/2894828612 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc488085880 con 0x7fc49015f120 2024-04-25T09:03:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.237+0000 7fc49480f640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc47c066f60 0x7fc47c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:52.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.241+0000 7fc49480f640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc47c066f60 0x7fc47c069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc4800ae000 tx=0x7fc4800a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:52.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.241+0000 7fc48d7fa640 1 -- 172.21.15.37:0/2894828612 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc490155000 con 0x7fc49015f120 2024-04-25T09:03:52.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 -- 172.21.15.37:0/2894828612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc4900b2070 con 0x7fc49015f120 2024-04-25T09:03:52.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc48d7fa640 1 -- 172.21.15.37:0/2894828612 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v97) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc4880560b0 con 0x7fc49015f120 2024-04-25T09:03:52.422 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:03:52.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 -- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc47c066f60 msgr2=0x7fc47c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc47c066f60 0x7fc47c069420 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc4800ae000 tx=0x7fc4800a7000 comp rx=0 tx=0).stop 2024-04-25T09:03:52.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 -- 172.21.15.37:0/2894828612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc49015f120 msgr2=0x7fc49006bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc49015f120 0x7fc49006bda0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fc488007e60 tx=0x7fc48800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 -- 172.21.15.37:0/2894828612 shutdown_connections 2024-04-25T09:03:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc47c066f60 0x7fc47c069420 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc49015f120 0x7fc49006bda0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc49015ed40 0x7fc49006d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 --2- 172.21.15.37:0/2894828612 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4900b2a70 0x7fc490154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 -- 172.21.15.37:0/2894828612 >> 172.21.15.37:0/2894828612 conn(0x7fc4900b3020 msgr2=0x7fc4901553c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.433+0000 7fc496a73640 1 -- 172.21.15.37:0/2894828612 shutdown_connections 2024-04-25T09:03:52.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:52.437+0000 7fc496a73640 1 -- 172.21.15.37:0/2894828612 wait complete. 2024-04-25T09:03:52.440 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:03:52.440 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:03:52.440 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T09:03:52.440 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:03:52.440 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:52.440 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:03:52.440 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:03:52.440 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:03:52.440 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:52.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:52.441 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T09:03:52.447 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:52.447 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:52.447 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:52.447 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:52.447 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:52.447 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:52.447 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41808sec preferred_lft 41808sec 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:03:52.448 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:52.449 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:03:52.450 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:52.450 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:03:52.450 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:52.450 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:03:52.450 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:52.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:52.450 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:03:52.497 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T09:03:52.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:52.498 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T09:03:52.546 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T09:03:52.547 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:03:52.547 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T09:03:52.547 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:03:52.547 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T09:03:52.547 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:03:52.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:52.547 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T09:03:52.547 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T09:03:52.547 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:03:52.547 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:03:52.547 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:03:52.548 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T09:03:52.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:52.619 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T09:03:52.664 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:03:52.665 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:03:52.665 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:52.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:52.665 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T09:03:52.723 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:03:52.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:52.724 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:52.768 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:52.768 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:52.769 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:52.816 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:03:52.875 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:03:52.926 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:03:52.926 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:03:52.926 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:03:52.926 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:03:52.926 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:03:52.926 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:03:52.926 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:03:52.926 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:03:52.927 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:03:52.928 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:03:52.928 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:03:52.928 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:03:52.928 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:03:52.928 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:03:52.928 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:03:52.928 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:52.928 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:52.928 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:52.928 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:52.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:52.929 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:03:52.977 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:03:52.977 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:03:53.021 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:03:53.036 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:03:53.089 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:03:53.089 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:03:53.089 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:03:53.089 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:03:53.089 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:03:53.089 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:03:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:03:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:03:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:03:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:03:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:03:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:03:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:03:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:03:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:03:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:03:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:53.091 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:53.092 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:53.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:53.092 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:03:53.105 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T09:03:53.117+0000 7f03da8dd580 -1 init, newargv = 0x555906860d30 newargc=15 2024-04-25T09:03:53.105 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[54802]: starting ceph client 2024-04-25T09:03:53.110 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[54802]: starting fuse 2024-04-25T09:03:54.210 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:03:54.227 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:03:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:03:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:03:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:03:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:03:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:03:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:03:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:03:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:03:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:03:54.279 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:03:54.280 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:03:54.281 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:03:54.281 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:03:54.281 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:03:54.281 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:03:54.281 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:54.281 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:54.281 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:54.281 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:54.281 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:54.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:54.282 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:03:54.330 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T09:03:54.330 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:54.331 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T09:03:54.332 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:03:54.332 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:03:54.332 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:03:54.332 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T09:03:54.332 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T09:03:54.332 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T09:03:54.332 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:03:54.332 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:54.332 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T09:03:54.332 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:03:54.455 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.54802.asok 2024-04-25T09:03:54.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:54.468 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.54802.asok 2024-04-25T09:03:54.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:54.468 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.54802.asok status 2024-04-25T09:03:54.625 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:03:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T09:03:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:03:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:03:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T09:03:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T09:03:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:03:54.625 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "54802", 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 9687, 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 9687 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:03:54.626 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 769893760 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 769893760 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.9687 192.168.0.1:0/769893760", 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/769893760", 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 97, 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 125, 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:03:54.627 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T09:03:54.628 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T09:03:54.628 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:03:54.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:54.639 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T09:03:54.639 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:54.645 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T09:03:54.645 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:54.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:54.645 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:54.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:54.700 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-04-25T09:03:54.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:54.756 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-04-25T09:03:54.867 INFO:teuthology.orchestra.run.smithi037.stderr:8+0 records in 2024-04-25T09:03:54.868 INFO:teuthology.orchestra.run.smithi037.stderr:8+0 records out 2024-04-25T09:03:54.868 INFO:teuthology.orchestra.run.smithi037.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0595528 s, 141 MB/s 2024-04-25T09:03:54.869 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:03:54.869 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:03:54.869 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:03:54.869 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:54.869 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T09:03:54.869 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:03:54.940 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627779 2024-04-25T09:03:54.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:54.950 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:03:54.950 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:03:54.950 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:03:54.950 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:03:54.950 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T09:03:54.950 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:03:55.058 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627780 2024-04-25T09:03:55.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:55.068 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:03:55.069 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:55.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:55.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efeaa7fa640 1 Processor -- start 2024-04-25T09:03:55.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efeaa7fa640 1 -- start start 2024-04-25T09:03:55.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efeaa7fa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7efea40ce510 0x7efea40ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efeaa7fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efea4144ca0 0x7efea406e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efeaa7fa640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea40cee30 con 0x7efea406ec60 2024-04-25T09:03:55.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efeaa7fa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea40cc2b0 con 0x7efea40ce510 2024-04-25T09:03:55.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efeaa7fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea406e290 con 0x7efea4144ca0 2024-04-25T09:03:55.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea8d97640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7efea406ec60 0x7efea4142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50746/0 (socket says 172.21.15.37:50746) 2024-04-25T09:03:55.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea8d97640 1 -- 172.21.15.37:0/1072892958 learned_addr learned my addr 172.21.15.37:0/1072892958 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:55.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efe9bfff640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efea4144ca0 0x7efea406e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586072368 0 0) 0x7efea40cee30 con 0x7efea406ec60 2024-04-25T09:03:55.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea3fff640 1 --2- 172.21.15.37:0/1072892958 >> v2:172.21.15.203:3300/0 conn(0x7efea40ce510 0x7efea40ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe90003530 con 0x7efea406ec60 2024-04-25T09:03:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3893236037 0 0) 0x7efe90003530 con 0x7efea406ec60 2024-04-25T09:03:55.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efea4144ca0 msgr2=0x7efea406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efea4144ca0 0x7efea406e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 >> v2:172.21.15.203:3300/0 conn(0x7efea40ce510 msgr2=0x7efea40ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 --2- 172.21.15.37:0/1072892958 >> v2:172.21.15.203:3300/0 conn(0x7efea40ce510 0x7efea40ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea40ccc70 con 0x7efea406ec60 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7efe9c001a00 con 0x7efea406ec60 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 >> v1:172.21.15.37:6789/0 conn(0x7efea406ec60 legacy=0x7efea4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe90003a30 0x7efe90003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe90004810 0x7efe90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efea4144ca0 0x7efe900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea40cc2b0 con 0x7efe90004810 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea406e290 con 0x7efe90003a30 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9000ba20 con 0x7efea4144ca0 2024-04-25T09:03:55.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efe9bfff640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe90004810 0x7efe90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea3fff640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe90003a30 0x7efe90003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efea8d97640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efea4144ca0 0x7efe900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efe9bfff640 1 -- 172.21.15.37:0/1072892958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efea4144ca0 msgr2=0x7efe900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efe9bfff640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efea4144ca0 0x7efe900051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efe9bfff640 1 -- 172.21.15.37:0/1072892958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe90003a30 msgr2=0x7efe90003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efe9bfff640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efe90003a30 0x7efe90003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.305+0000 7efe9bfff640 1 -- 172.21.15.37:0/1072892958 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe90003530 con 0x7efe90004810 2024-04-25T09:03:55.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efe9bfff640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe90004810 0x7efe90004c90 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009ab0 tx=0x7efe8c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=833a896fe3421ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0b2070 con 0x7efe90004810 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c0a5560 con 0x7efe90004810 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efea37fe640 1 -- 172.21.15.37:0/1072892958 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c0096b0 con 0x7efe90004810 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1072892958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe90004810 msgr2=0x7efe90004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efeaa7fa640 1 --2- 172.21.15.37:0/1072892958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe90004810 0x7efe90004c90 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7efe8c009ab0 tx=0x7efe8c01ca50 comp rx=0 tx=0).stop 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efea8d97640 1 -- 172.21.15.37:0/1072892958 reap_dead start 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1072892958 shutdown_connections 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1072892958 >> 172.21.15.37:0/1072892958 conn(0x7efea40b11a0 msgr2=0x7efea40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1072892958 shutdown_connections 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1072892958 wait complete. 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efeaa7fa640 1 Processor -- start 2024-04-25T09:03:55.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.309+0000 7efeaa7fa640 1 -- start start 2024-04-25T09:03:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efeaa7fa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe90004810 0x7efea40af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efeaa7fa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efea40ae990 0x7efea40ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efeaa7fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efea415aed0 0x7efea40b39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efeaa7fa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea40cc900 con 0x7efea415aed0 2024-04-25T09:03:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efeaa7fa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea406df20 con 0x7efea40ae990 2024-04-25T09:03:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efeaa7fa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea4142d90 con 0x7efe90004810 2024-04-25T09:03:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea8d97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efea415aed0 0x7efea40b39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea8d97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efea415aed0 0x7efea40b39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45516/0 (socket says 172.21.15.37:45516) 2024-04-25T09:03:55.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea8d97640 1 -- 172.21.15.37:0/1405139427 learned_addr learned my addr 172.21.15.37:0/1405139427 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efe9bfff640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efea40ae990 0x7efea40ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea3fff640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe90004810 0x7efea40af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea8d97640 1 -- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe90004810 msgr2=0x7efea40af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea8d97640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe90004810 0x7efea40af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea8d97640 1 -- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efea40ae990 msgr2=0x7efea40ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea8d97640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efea40ae990 0x7efea40ad0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea8d97640 1 -- 172.21.15.37:0/1405139427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea40ad8f0 con 0x7efea415aed0 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efe9bfff640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efea40ae990 0x7efea40ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea8d97640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efea415aed0 0x7efea40b39c0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7efea406f0c0 tx=0x7efe9c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea17fa640 1 -- 172.21.15.37:0/1405139427 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c00ebf0 con 0x7efea415aed0 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea17fa640 1 -- 172.21.15.37:0/1405139427 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c00edb0 con 0x7efea415aed0 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1405139427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea40adbd0 con 0x7efea415aed0 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea17fa640 1 -- 172.21.15.37:0/1405139427 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c014670 con 0x7efea415aed0 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1405139427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea40ae190 con 0x7efea415aed0 2024-04-25T09:03:55.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1405139427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea404efa0 con 0x7efea415aed0 2024-04-25T09:03:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea17fa640 1 -- 172.21.15.37:0/1405139427 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7efe9c014810 con 0x7efea415aed0 2024-04-25T09:03:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea17fa640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe84066f90 0x7efe84069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea17fa640 1 -- 172.21.15.37:0/1405139427 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe9c084e60 con 0x7efea415aed0 2024-04-25T09:03:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.313+0000 7efea3fff640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe84066f90 0x7efe84069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.317+0000 7efea3fff640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe84066f90 0x7efe84069450 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7efe940099f0 tx=0x7efe94008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.317+0000 7efea17fa640 1 -- 172.21.15.37:0/1405139427 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7efe9c055d70 con 0x7efea415aed0 2024-04-25T09:03:55.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.481+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1405139427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efea40b1720 con 0x7efea415aed0 2024-04-25T09:03:55.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.481+0000 7efea17fa640 1 -- 172.21.15.37:0/1405139427 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16916 (secure 0 0 0) 0x7efe9c055670 con 0x7efea415aed0 2024-04-25T09:03:55.471 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:55.471 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":125,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:33.770823+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe84066f90 msgr2=0x7efe84069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe84066f90 0x7efe84069450 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7efe940099f0 tx=0x7efe94008040 comp rx=0 tx=0).stop 2024-04-25T09:03:55.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1405139427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efea415aed0 msgr2=0x7efea40b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efea415aed0 0x7efea40b39c0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7efea406f0c0 tx=0x7efe9c007bc0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1405139427 shutdown_connections 2024-04-25T09:03:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7efe84066f90 0x7efe84069450 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efea415aed0 0x7efea40b39c0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7efea40ae990 0x7efea40ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 --2- 172.21.15.37:0/1405139427 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7efe90004810 0x7efea40af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1405139427 >> 172.21.15.37:0/1405139427 conn(0x7efea40b11a0 msgr2=0x7efea40ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1405139427 shutdown_connections 2024-04-25T09:03:55.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.485+0000 7efeaa7fa640 1 -- 172.21.15.37:0/1405139427 wait complete. 2024-04-25T09:03:55.491 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:30.763724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:31.488462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:55.491 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:55.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.629+0000 7f3da342d640 1 Processor -- start 2024-04-25T09:03:55.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.629+0000 7f3da342d640 1 -- start start 2024-04-25T09:03:55.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d9c0cfca0 0x7f3d9c15dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3d9c15e4f0 0x7f3d9c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d9c0cbf10 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d9c0cc1f0 con 0x7f3d9c15e4f0 2024-04-25T09:03:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0cc4d0 con 0x7f3d9c0cfca0 2024-04-25T09:03:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da19ca640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3d9c15e4f0 0x7f3d9c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da19ca640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3d9c15e4f0 0x7f3d9c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45522/0 (socket says 172.21.15.37:45522) 2024-04-25T09:03:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da19ca640 1 -- 172.21.15.37:0/752014495 learned_addr learned my addr 172.21.15.37:0/752014495 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da09c8640 1 --2- 172.21.15.37:0/752014495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d9c0cfca0 0x7f3d9c15dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da19ca640 1 -- 172.21.15.37:0/752014495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d9c0cfca0 msgr2=0x7f3d9c15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da19ca640 1 --2- 172.21.15.37:0/752014495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d9c0cfca0 0x7f3d9c15dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da19ca640 1 -- 172.21.15.37:0/752014495 >> v1:172.21.15.37:6789/0 conn(0x7f3d9c0b3cf0 legacy=0x7f3d9c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3d8ffff640 1 -- 172.21.15.37:0/752014495 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797377005 0 0) 0x7f3d9c0cbf10 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da19ca640 1 -- 172.21.15.37:0/752014495 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c0cc7b0 con 0x7f3d9c15e4f0 2024-04-25T09:03:55.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da19ca640 1 --2- 172.21.15.37:0/752014495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d9c15e4f0 0x7f3d9c0caaf0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009e50 tx=0x7f3d9401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd6a7568db1051b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3d8ffff640 1 -- 172.21.15.37:0/752014495 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940a5700 con 0x7f3d9c15e4f0 2024-04-25T09:03:55.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3d8ffff640 1 -- 172.21.15.37:0/752014495 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d940a58a0 con 0x7f3d9c15e4f0 2024-04-25T09:03:55.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3d8ffff640 1 -- 172.21.15.37:0/752014495 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d940a5c70 con 0x7f3d9c15e4f0 2024-04-25T09:03:55.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 -- 172.21.15.37:0/752014495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d9c15e4f0 msgr2=0x7f3d9c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 --2- 172.21.15.37:0/752014495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d9c15e4f0 0x7f3d9c0caaf0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009e50 tx=0x7f3d9401ca30 comp rx=0 tx=0).stop 2024-04-25T09:03:55.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 -- 172.21.15.37:0/752014495 shutdown_connections 2024-04-25T09:03:55.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 --2- 172.21.15.37:0/752014495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d9c15e4f0 0x7f3d9c0caaf0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 --2- 172.21.15.37:0/752014495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d9c0cfca0 0x7f3d9c15dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 -- 172.21.15.37:0/752014495 >> 172.21.15.37:0/752014495 conn(0x7f3d9c06ae60 msgr2=0x7f3d9c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 -- 172.21.15.37:0/752014495 shutdown_connections 2024-04-25T09:03:55.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.633+0000 7f3da342d640 1 -- 172.21.15.37:0/752014495 wait complete. 2024-04-25T09:03:55.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da342d640 1 Processor -- start 2024-04-25T09:03:55.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da342d640 1 -- start start 2024-04-25T09:03:55.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da342d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d9c0b3cf0 0x7f3d9c0c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da342d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d9c0cfca0 0x7f3d9c1586b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da342d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d9c15e4f0 0x7f3d9c156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da342d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0cb9c0 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da342d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0cb840 con 0x7f3d9c0cfca0 2024-04-25T09:03:55.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da342d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9c0cbb40 con 0x7f3d9c15e4f0 2024-04-25T09:03:55.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da11c9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d9c0b3cf0 0x7f3d9c0c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da11c9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d9c0b3cf0 0x7f3d9c0c2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45538/0 (socket says 172.21.15.37:45538) 2024-04-25T09:03:55.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da11c9640 1 -- 172.21.15.37:0/2124639371 learned_addr learned my addr 172.21.15.37:0/2124639371 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:55.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da19ca640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d9c15e4f0 0x7f3d9c156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da09c8640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d9c0cfca0 0x7f3d9c1586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da11c9640 1 -- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d9c15e4f0 msgr2=0x7f3d9c156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da11c9640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d9c15e4f0 0x7f3d9c156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da11c9640 1 -- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d9c0cfca0 msgr2=0x7f3d9c1586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da11c9640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d9c0cfca0 0x7f3d9c1586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da11c9640 1 -- 172.21.15.37:0/2124639371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c0c34c0 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da11c9640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d9c0b3cf0 0x7f3d9c0c2de0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000c4d0 tx=0x7f3d9000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3d8dffb640 1 -- 172.21.15.37:0/2124639371 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90017070 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3d8dffb640 1 -- 172.21.15.37:0/2124639371 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d90005480 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da342d640 1 -- 172.21.15.37:0/2124639371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9c0c37a0 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3d8dffb640 1 -- 172.21.15.37:0/2124639371 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90003840 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da342d640 1 -- 172.21.15.37:0/2124639371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9c0c3d60 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.637+0000 7f3da342d640 1 -- 172.21.15.37:0/2124639371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9c04efa0 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.641+0000 7f3d8dffb640 1 -- 172.21.15.37:0/2124639371 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3d9001a070 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.641+0000 7f3d8dffb640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d80066f60 0x7f3d80069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.641+0000 7f3d8dffb640 1 -- 172.21.15.37:0/2124639371 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d90084550 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.641+0000 7f3da09c8640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d80066f60 0x7f3d80069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.641+0000 7f3da09c8640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d80066f60 0x7f3d80069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c160e30 tx=0x7f3d98008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.645+0000 7f3d8dffb640 1 -- 172.21.15.37:0/2124639371 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3d90054da0 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 -- 172.21.15.37:0/2124639371 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d9c0ccf10 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3d8dffb640 1 -- 172.21.15.37:0/2124639371 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d90058e40 con 0x7f3d9c0b3cf0 2024-04-25T09:03:55.793 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 -- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d80066f60 msgr2=0x7f3d80069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d80066f60 0x7f3d80069420 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c160e30 tx=0x7f3d98008040 comp rx=0 tx=0).stop 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 -- 172.21.15.37:0/2124639371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d9c0b3cf0 msgr2=0x7f3d9c0c2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d9c0b3cf0 0x7f3d9c0c2de0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000c4d0 tx=0x7f3d9000c990 comp rx=0 tx=0).stop 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 -- 172.21.15.37:0/2124639371 shutdown_connections 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3d80066f60 0x7f3d80069420 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3d9c15e4f0 0x7f3d9c156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3d9c0cfca0 0x7f3d9c1586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 --2- 172.21.15.37:0/2124639371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3d9c0b3cf0 0x7f3d9c0c2de0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 -- 172.21.15.37:0/2124639371 >> 172.21.15.37:0/2124639371 conn(0x7f3d9c06ae60 msgr2=0x7f3d9c06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 -- 172.21.15.37:0/2124639371 shutdown_connections 2024-04-25T09:03:55.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.805+0000 7f3da342d640 1 -- 172.21.15.37:0/2124639371 wait complete. 2024-04-25T09:03:55.812 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:55.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc93139e640 1 Processor -- start 2024-04-25T09:03:55.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc93139e640 1 -- start start 2024-04-25T09:03:55.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc93139e640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc92c06f4a0 0x7fc92c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc93139e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc92c099fb0 0x7fc92c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc93139e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92c0b0e10 con 0x7fc92c0b0a30 2024-04-25T09:03:55.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc93139e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c0d0d90 con 0x7fc92c099fb0 2024-04-25T09:03:55.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc93139e640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c0d0180 con 0x7fc92c06f4a0 2024-04-25T09:03:55.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc92affd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc92c0b0a30 0x7fc92c0cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:50756/0 (socket says 172.21.15.37:50756) 2024-04-25T09:03:55.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc92affd640 1 -- 172.21.15.37:0/3093922838 learned_addr learned my addr 172.21.15.37:0/3093922838 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:55.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc92b7fe640 1 --2- 172.21.15.37:0/3093922838 >> v2:172.21.15.37:3300/0 conn(0x7fc92c099fb0 0x7fc92c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc929ffb640 1 -- 172.21.15.37:0/3093922838 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966224399 0 0) 0x7fc92c0b0e10 con 0x7fc92c0b0a30 2024-04-25T09:03:55.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc929ffb640 1 -- 172.21.15.37:0/3093922838 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc918003530 con 0x7fc92c0b0a30 2024-04-25T09:03:55.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc92a7fc640 1 --2- 172.21.15.37:0/3093922838 >> v2:172.21.15.203:3300/0 conn(0x7fc92c06f4a0 0x7fc92c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc929ffb640 1 -- 172.21.15.37:0/3093922838 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 603514491 0 0) 0x7fc918003530 con 0x7fc92c0b0a30 2024-04-25T09:03:55.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc92b7fe640 1 -- 172.21.15.37:0/3093922838 >> v2:172.21.15.203:3300/0 conn(0x7fc92c06f4a0 msgr2=0x7fc92c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:55.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc92b7fe640 1 --2- 172.21.15.37:0/3093922838 >> v2:172.21.15.203:3300/0 conn(0x7fc92c06f4a0 0x7fc92c099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc92b7fe640 1 -- 172.21.15.37:0/3093922838 >> v1:172.21.15.37:6789/0 conn(0x7fc92c0b0a30 legacy=0x7fc92c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.941+0000 7fc92b7fe640 1 -- 172.21.15.37:0/3093922838 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c0d0460 con 0x7fc92c099fb0 2024-04-25T09:03:55.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc92b7fe640 1 --2- 172.21.15.37:0/3093922838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc92c099fb0 0x7fc92c15d230 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0b7670 tx=0x7fc92401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca7805e1f1cecde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc929ffb640 1 -- 172.21.15.37:0/3093922838 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9240ac070 con 0x7fc92c099fb0 2024-04-25T09:03:55.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc929ffb640 1 -- 172.21.15.37:0/3093922838 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9240a64a0 con 0x7fc92c099fb0 2024-04-25T09:03:55.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc929ffb640 1 -- 172.21.15.37:0/3093922838 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914001680 con 0x7fc92c099fb0 2024-04-25T09:03:55.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 -- 172.21.15.37:0/3093922838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc92c099fb0 msgr2=0x7fc92c15d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 --2- 172.21.15.37:0/3093922838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc92c099fb0 0x7fc92c15d230 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fc92c0b7670 tx=0x7fc92401c850 comp rx=0 tx=0).stop 2024-04-25T09:03:55.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 -- 172.21.15.37:0/3093922838 shutdown_connections 2024-04-25T09:03:55.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 --2- 172.21.15.37:0/3093922838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc92c099fb0 0x7fc92c15d230 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 --2- 172.21.15.37:0/3093922838 >> v2:172.21.15.203:3300/0 conn(0x7fc92c06f4a0 0x7fc92c099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 -- 172.21.15.37:0/3093922838 >> 172.21.15.37:0/3093922838 conn(0x7fc92c0b30b0 msgr2=0x7fc92c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:55.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 -- 172.21.15.37:0/3093922838 shutdown_connections 2024-04-25T09:03:55.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 -- 172.21.15.37:0/3093922838 wait complete. 2024-04-25T09:03:55.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 Processor -- start 2024-04-25T09:03:55.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 -- start start 2024-04-25T09:03:55.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc92c06f4a0 0x7fc92c160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c099fb0 0x7fc92c14d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc92c0b0a30 0x7fc92c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0cbfa0 con 0x7fc92c0b0a30 2024-04-25T09:03:55.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0cbe20 con 0x7fc92c099fb0 2024-04-25T09:03:55.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.945+0000 7fc93139e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0cc120 con 0x7fc92c06f4a0 2024-04-25T09:03:55.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc92c0b0a30 0x7fc92c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc92c0b0a30 0x7fc92c06d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45560/0 (socket says 172.21.15.37:45560) 2024-04-25T09:03:55.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92b7fe640 1 -- 172.21.15.37:0/2882676719 learned_addr learned my addr 172.21.15.37:0/2882676719 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92b7fe640 1 -- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc92c06f4a0 msgr2=0x7fc92c160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92a7fc640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c099fb0 0x7fc92c14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92affd640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc92c06f4a0 0x7fc92c160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92b7fe640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc92c06f4a0 0x7fc92c160b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92b7fe640 1 -- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c099fb0 msgr2=0x7fc92c14d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92b7fe640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c099fb0 0x7fc92c14d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92b7fe640 1 -- 172.21.15.37:0/2882676719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c1610a0 con 0x7fc92c0b0a30 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92b7fe640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc92c0b0a30 0x7fc92c06d2a0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fc92401af00 tx=0x7fc9240195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc9237fe640 1 -- 172.21.15.37:0/2882676719 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9240c0070 con 0x7fc92c0b0a30 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc9237fe640 1 -- 172.21.15.37:0/2882676719 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9240a7480 con 0x7fc92c0b0a30 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc93139e640 1 -- 172.21.15.37:0/2882676719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c15bc80 con 0x7fc92c0b0a30 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc9237fe640 1 -- 172.21.15.37:0/2882676719 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9240ac050 con 0x7fc92c0b0a30 2024-04-25T09:03:55.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc93139e640 1 -- 172.21.15.37:0/2882676719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c15c520 con 0x7fc92c0b0a30 2024-04-25T09:03:55.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc93139e640 1 -- 172.21.15.37:0/2882676719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c04efa0 con 0x7fc92c0b0a30 2024-04-25T09:03:55.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc9237fe640 1 -- 172.21.15.37:0/2882676719 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc9240c3070 con 0x7fc92c0b0a30 2024-04-25T09:03:55.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc9237fe640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc90c066f60 0x7fc90c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:55.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc9237fe640 1 -- 172.21.15.37:0/2882676719 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc92411f890 con 0x7fc92c0b0a30 2024-04-25T09:03:55.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92affd640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc90c066f60 0x7fc90c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:55.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.949+0000 7fc92affd640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc90c066f60 0x7fc90c069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc9140011f0 tx=0x7fc914009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:55.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:55.953+0000 7fc9237fe640 1 -- 172.21.15.37:0/2882676719 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc924016440 con 0x7fc92c0b0a30 2024-04-25T09:03:56.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.105+0000 7fc93139e640 1 -- 172.21.15.37:0/2882676719 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc92c0b3ee0 con 0x7fc92c0b0a30 2024-04-25T09:03:56.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc9237fe640 1 -- 172.21.15.37:0/2882676719 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc9240f0830 con 0x7fc92c0b0a30 2024-04-25T09:03:56.097 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 -- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc90c066f60 msgr2=0x7fc90c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc90c066f60 0x7fc90c069420 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc9140011f0 tx=0x7fc914009040 comp rx=0 tx=0).stop 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 -- 172.21.15.37:0/2882676719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc92c0b0a30 msgr2=0x7fc92c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc92c0b0a30 0x7fc92c06d2a0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fc92401af00 tx=0x7fc9240195f0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 -- 172.21.15.37:0/2882676719 shutdown_connections 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc90c066f60 0x7fc90c069420 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc92c0b0a30 0x7fc92c06d2a0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc92c099fb0 0x7fc92c14d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 --2- 172.21.15.37:0/2882676719 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc92c06f4a0 0x7fc92c160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 -- 172.21.15.37:0/2882676719 >> 172.21.15.37:0/2882676719 conn(0x7fc92c0b30b0 msgr2=0x7fc92c0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 -- 172.21.15.37:0/2882676719 shutdown_connections 2024-04-25T09:03:56.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.109+0000 7fc93139e640 1 -- 172.21.15.37:0/2882676719 wait complete. 2024-04-25T09:03:56.115 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:56.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.245+0000 7f5392fb1640 1 Processor -- start 2024-04-25T09:03:56.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.245+0000 7f5392fb1640 1 -- start start 2024-04-25T09:03:56.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f538c15da80 0x7f538c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f538c0d1cd0 0x7f538c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f538c0d0620 0x7f538c0ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538c06c440 con 0x7f538c0d1cd0 2024-04-25T09:03:56.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538c06be60 con 0x7f538c0d0620 2024-04-25T09:03:56.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538c15de60 con 0x7f538c15da80 2024-04-25T09:03:56.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f538c0d1cd0 0x7f538c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538bfff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f538c0d1cd0 0x7f538c0d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45570/0 (socket says 172.21.15.37:45570) 2024-04-25T09:03:56.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538bfff640 1 -- 172.21.15.37:0/3709941493 learned_addr learned my addr 172.21.15.37:0/3709941493 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:56.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f539154e640 1 --2- 172.21.15.37:0/3709941493 >> v2:172.21.15.203:3300/0 conn(0x7f538c0d0620 0x7f538c0ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5390d4d640 1 --2- 172.21.15.37:0/3709941493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f538c15da80 0x7f538c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538bfff640 1 -- 172.21.15.37:0/3709941493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f538c15da80 msgr2=0x7f538c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538bfff640 1 --2- 172.21.15.37:0/3709941493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f538c15da80 0x7f538c0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538bfff640 1 -- 172.21.15.37:0/3709941493 >> v2:172.21.15.203:3300/0 conn(0x7f538c0d0620 msgr2=0x7f538c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538bfff640 1 --2- 172.21.15.37:0/3709941493 >> v2:172.21.15.203:3300/0 conn(0x7f538c0d0620 0x7f538c0ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538bfff640 1 -- 172.21.15.37:0/3709941493 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c0d2450 con 0x7f538c0d1cd0 2024-04-25T09:03:56.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538bfff640 1 --2- 172.21.15.37:0/3709941493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538c0d1cd0 0x7f538c0d00e0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f538c06de60 tx=0x7f537801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe83e8c91249e24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538b7fe640 1 -- 172.21.15.37:0/3709941493 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53780ab5f0 con 0x7f538c0d1cd0 2024-04-25T09:03:56.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538b7fe640 1 -- 172.21.15.37:0/3709941493 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53780b0070 con 0x7f538c0d1cd0 2024-04-25T09:03:56.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f538b7fe640 1 -- 172.21.15.37:0/3709941493 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53780ab9b0 con 0x7f538c0d1cd0 2024-04-25T09:03:56.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 -- 172.21.15.37:0/3709941493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538c0d1cd0 msgr2=0x7f538c0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 --2- 172.21.15.37:0/3709941493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538c0d1cd0 0x7f538c0d00e0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f538c06de60 tx=0x7f537801c520 comp rx=0 tx=0).stop 2024-04-25T09:03:56.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 -- 172.21.15.37:0/3709941493 shutdown_connections 2024-04-25T09:03:56.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 --2- 172.21.15.37:0/3709941493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538c0d1cd0 0x7f538c0d00e0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 --2- 172.21.15.37:0/3709941493 >> v2:172.21.15.203:3300/0 conn(0x7f538c0d0620 0x7f538c0ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 --2- 172.21.15.37:0/3709941493 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f538c15da80 0x7f538c0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 -- 172.21.15.37:0/3709941493 >> 172.21.15.37:0/3709941493 conn(0x7f538c08b000 msgr2=0x7f538c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 -- 172.21.15.37:0/3709941493 shutdown_connections 2024-04-25T09:03:56.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.249+0000 7f5392fb1640 1 -- 172.21.15.37:0/3709941493 wait complete. 2024-04-25T09:03:56.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5392fb1640 1 Processor -- start 2024-04-25T09:03:56.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5392fb1640 1 -- start start 2024-04-25T09:03:56.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5392fb1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f538c0d0620 0x7f538c0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5392fb1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538c0d1cd0 0x7f538c157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5392fb1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f538c15da80 0x7f538c0b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5392fb1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c06ca30 con 0x7f538c0d1cd0 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5392fb1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c06c8b0 con 0x7f538c0d0620 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5392fb1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538c06cbb0 con 0x7f538c15da80 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f538bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538c0d1cd0 0x7f538c157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f538bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538c0d1cd0 0x7f538c157360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45576/0 (socket says 172.21.15.37:45576) 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f538bfff640 1 -- 172.21.15.37:0/3980898571 learned_addr learned my addr 172.21.15.37:0/3980898571 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5390d4d640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f538c0d0620 0x7f538c0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f538bfff640 1 -- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f538c15da80 msgr2=0x7f538c0b74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f538bfff640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f538c15da80 0x7f538c0b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f538bfff640 1 -- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f538c0d0620 msgr2=0x7f538c0b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f538bfff640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f538c0d0620 0x7f538c0b8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f538bfff640 1 -- 172.21.15.37:0/3980898571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c0b9200 con 0x7f538c0d1cd0 2024-04-25T09:03:56.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5390d4d640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f538c0d0620 0x7f538c0b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:56.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f538bfff640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538c0d1cd0 0x7f538c157360 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f537801abd0 tx=0x7f53780192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f53897fa640 1 -- 172.21.15.37:0/3980898571 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5378003ab0 con 0x7f538c0d1cd0 2024-04-25T09:03:56.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f53897fa640 1 -- 172.21.15.37:0/3980898571 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5378003c50 con 0x7f538c0d1cd0 2024-04-25T09:03:56.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5392fb1640 1 -- 172.21.15.37:0/3980898571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c1583e0 con 0x7f538c0d1cd0 2024-04-25T09:03:56.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f53897fa640 1 -- 172.21.15.37:0/3980898571 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53780bd400 con 0x7f538c0d1cd0 2024-04-25T09:03:56.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5392fb1640 1 -- 172.21.15.37:0/3980898571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c1589a0 con 0x7f538c0d1cd0 2024-04-25T09:03:56.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.253+0000 7f5392fb1640 1 -- 172.21.15.37:0/3980898571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538c158ce0 con 0x7f538c0d1cd0 2024-04-25T09:03:56.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.257+0000 7f53897fa640 1 -- 172.21.15.37:0/3980898571 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f53780bd5a0 con 0x7f538c0d1cd0 2024-04-25T09:03:56.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.257+0000 7f53897fa640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5370066f60 0x7f5370069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.257+0000 7f53897fa640 1 -- 172.21.15.37:0/3980898571 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f537811f970 con 0x7f538c0d1cd0 2024-04-25T09:03:56.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.257+0000 7f5390d4d640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5370066f60 0x7f5370069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.257+0000 7f5390d4d640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5370066f60 0x7f5370069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f53800097a0 tx=0x7f5380008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.261+0000 7f53897fa640 1 -- 172.21.15.37:0/3980898571 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f538c158ce0 con 0x7f538c0d1cd0 2024-04-25T09:03:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.413+0000 7f5392fb1640 1 -- 172.21.15.37:0/3980898571 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f538c160cd0 con 0x7f538c0d1cd0 2024-04-25T09:03:56.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f53897fa640 1 -- 172.21.15.37:0/3980898571 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f53780b9030 con 0x7f538c0d1cd0 2024-04-25T09:03:56.404 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:03:56.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 -- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5370066f60 msgr2=0x7f5370069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5370066f60 0x7f5370069420 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f53800097a0 tx=0x7f5380008040 comp rx=0 tx=0).stop 2024-04-25T09:03:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 -- 172.21.15.37:0/3980898571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538c0d1cd0 msgr2=0x7f538c157360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538c0d1cd0 0x7f538c157360 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f537801abd0 tx=0x7f53780192c0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 -- 172.21.15.37:0/3980898571 shutdown_connections 2024-04-25T09:03:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5370066f60 0x7f5370069420 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f538c15da80 0x7f538c0b74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f538c0d1cd0 0x7f538c157360 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 --2- 172.21.15.37:0/3980898571 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f538c0d0620 0x7f538c0b8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 -- 172.21.15.37:0/3980898571 >> 172.21.15.37:0/3980898571 conn(0x7f538c08b000 msgr2=0x7f538c0b1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 -- 172.21.15.37:0/3980898571 shutdown_connections 2024-04-25T09:03:56.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.417+0000 7f5392fb1640 1 -- 172.21.15.37:0/3980898571 wait complete. 2024-04-25T09:03:56.422 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:03:56.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.557+0000 7f117165c640 1 Processor -- start 2024-04-25T09:03:56.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.557+0000 7f117165c640 1 -- start start 2024-04-25T09:03:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f116c15da80 0x7f116c0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f116c0d1cd0 0x7f116c0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116c06be60 con 0x7f116c0d0620 2024-04-25T09:03:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c0d0a70 con 0x7f116c0d1cd0 2024-04-25T09:03:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116c06c440 con 0x7f116c15da80 2024-04-25T09:03:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f116a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f116c0d1cd0 0x7f116c0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f116a7fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f116c0d1cd0 0x7f116c0d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45580/0 (socket says 172.21.15.37:45580) 2024-04-25T09:03:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f116a7fc640 1 -- 172.21.15.37:0/1763342848 learned_addr learned my addr 172.21.15.37:0/1763342848 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f116a7fc640 1 -- 172.21.15.37:0/1763342848 >> v2:172.21.15.203:3300/0 conn(0x7f116c15da80 msgr2=0x7f116c0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f116a7fc640 1 --2- 172.21.15.37:0/1763342848 >> v2:172.21.15.203:3300/0 conn(0x7f116c15da80 0x7f116c0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f116a7fc640 1 -- 172.21.15.37:0/1763342848 >> v1:172.21.15.203:6789/0 conn(0x7f116c0d0620 legacy=0x7f116c0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f116a7fc640 1 -- 172.21.15.37:0/1763342848 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c15de60 con 0x7f116c0d1cd0 2024-04-25T09:03:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f116a7fc640 1 --2- 172.21.15.37:0/1763342848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116c0d1cd0 0x7f116c0d00e0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f1160009860 tx=0x7f116001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afddeff2e7f376d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f1169ffb640 1 -- 172.21.15.37:0/1763342848 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11600b0070 con 0x7f116c0d1cd0 2024-04-25T09:03:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f1169ffb640 1 -- 172.21.15.37:0/1763342848 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1160004230 con 0x7f116c0d1cd0 2024-04-25T09:03:56.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f1169ffb640 1 -- 172.21.15.37:0/1763342848 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160003620 con 0x7f116c0d1cd0 2024-04-25T09:03:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 -- 172.21.15.37:0/1763342848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116c0d1cd0 msgr2=0x7f116c0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 --2- 172.21.15.37:0/1763342848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116c0d1cd0 0x7f116c0d00e0 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f1160009860 tx=0x7f116001c4c0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 -- 172.21.15.37:0/1763342848 shutdown_connections 2024-04-25T09:03:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 --2- 172.21.15.37:0/1763342848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116c0d1cd0 0x7f116c0d00e0 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 --2- 172.21.15.37:0/1763342848 >> v2:172.21.15.203:3300/0 conn(0x7f116c15da80 0x7f116c0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 -- 172.21.15.37:0/1763342848 >> 172.21.15.37:0/1763342848 conn(0x7f116c08b000 msgr2=0x7f116c0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 -- 172.21.15.37:0/1763342848 shutdown_connections 2024-04-25T09:03:56.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.561+0000 7f117165c640 1 -- 172.21.15.37:0/1763342848 wait complete. 2024-04-25T09:03:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f117165c640 1 Processor -- start 2024-04-25T09:03:56.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f117165c640 1 -- start start 2024-04-25T09:03:56.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f117165c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116c0d0620 0x7f116c157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f117165c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116c0d1cd0 0x7f116c15ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f117165c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116c15da80 0x7f116c1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f117165c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c16a730 con 0x7f116c15da80 2024-04-25T09:03:56.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f117165c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c16a5b0 con 0x7f116c0d1cd0 2024-04-25T09:03:56.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f117165c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116c16a8b0 con 0x7f116c0d0620 2024-04-25T09:03:56.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116c15da80 0x7f116c1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116c15da80 0x7f116c1581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45586/0 (socket says 172.21.15.37:45586) 2024-04-25T09:03:56.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116b7fe640 1 -- 172.21.15.37:0/3594741918 learned_addr learned my addr 172.21.15.37:0/3594741918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:56.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116affd640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116c0d0620 0x7f116c157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116b7fe640 1 -- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116c0d0620 msgr2=0x7f116c157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116b7fe640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116c0d0620 0x7f116c157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116a7fc640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116c0d1cd0 0x7f116c15ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116b7fe640 1 -- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116c0d1cd0 msgr2=0x7f116c15ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116b7fe640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116c0d1cd0 0x7f116c15ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116b7fe640 1 -- 172.21.15.37:0/3594741918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c157740 con 0x7f116c15da80 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116affd640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116c0d0620 0x7f116c157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f116b7fe640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116c15da80 0x7f116c1581e0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f116400c7b0 tx=0x7f116400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f11537fe640 1 -- 172.21.15.37:0/3594741918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164017070 con 0x7f116c15da80 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f11537fe640 1 -- 172.21.15.37:0/3594741918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1164003db0 con 0x7f116c15da80 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f117165c640 1 -- 172.21.15.37:0/3594741918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116c0c7dd0 con 0x7f116c15da80 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f11537fe640 1 -- 172.21.15.37:0/3594741918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11640052b0 con 0x7f116c15da80 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f117165c640 1 -- 172.21.15.37:0/3594741918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c0c8390 con 0x7f116c15da80 2024-04-25T09:03:56.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.565+0000 7f117165c640 1 -- 172.21.15.37:0/3594741918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116c04efa0 con 0x7f116c15da80 2024-04-25T09:03:56.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.569+0000 7f11537fe640 1 -- 172.21.15.37:0/3594741918 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f116401a070 con 0x7f116c15da80 2024-04-25T09:03:56.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.569+0000 7f11537fe640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1158066f60 0x7f1158069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.569+0000 7f11537fe640 1 -- 172.21.15.37:0/3594741918 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1164084780 con 0x7f116c15da80 2024-04-25T09:03:56.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.569+0000 7f116affd640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1158066f60 0x7f1158069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.569+0000 7f116affd640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1158066f60 0x7f1158069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f116c157c30 tx=0x7f115c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.573+0000 7f11537fe640 1 -- 172.21.15.37:0/3594741918 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1164055690 con 0x7f116c15da80 2024-04-25T09:03:56.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.737+0000 7f117165c640 1 -- 172.21.15.37:0/3594741918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f116c0b9570 con 0x7f116c15da80 2024-04-25T09:03:56.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.737+0000 7f11537fe640 1 -- 172.21.15.37:0/3594741918 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1164054f90 con 0x7f116c15da80 2024-04-25T09:03:56.728 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T09:03:56.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 -- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1158066f60 msgr2=0x7f1158069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1158066f60 0x7f1158069420 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f116c157c30 tx=0x7f115c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:56.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 -- 172.21.15.37:0/3594741918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116c15da80 msgr2=0x7f116c1581e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116c15da80 0x7f116c1581e0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f116400c7b0 tx=0x7f116400cc70 comp rx=0 tx=0).stop 2024-04-25T09:03:56.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 -- 172.21.15.37:0/3594741918 shutdown_connections 2024-04-25T09:03:56.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1158066f60 0x7f1158069420 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f116c15da80 0x7f116c1581e0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f116c0d1cd0 0x7f116c15ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 --2- 172.21.15.37:0/3594741918 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f116c0d0620 0x7f116c157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 -- 172.21.15.37:0/3594741918 >> 172.21.15.37:0/3594741918 conn(0x7f116c08b000 msgr2=0x7f116c0b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 -- 172.21.15.37:0/3594741918 shutdown_connections 2024-04-25T09:03:56.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.741+0000 7f117165c640 1 -- 172.21.15.37:0/3594741918 wait complete. 2024-04-25T09:03:56.747 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:03:56.747 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:56.752 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.929+0000 7fe7014f8640 1 Processor -- start 2024-04-25T09:03:56.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.929+0000 7fe7014f8640 1 -- start start 2024-04-25T09:03:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe7014f8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe6fc15da80 0x7fe6fc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe7014f8640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6fc0d07f0 0x7fe6fc06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe7014f8640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6fc06be60 con 0x7fe6fc0d1cd0 2024-04-25T09:03:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe7014f8640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6fc06c440 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe7014f8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6fc15de60 con 0x7fe6fc15da80 2024-04-25T09:03:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6fb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6fc0d07f0 0x7fe6fc06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6fb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe6fc0d07f0 0x7fe6fc06c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45600/0 (socket says 172.21.15.37:45600) 2024-04-25T09:03:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6fb7fe640 1 -- 172.21.15.37:0/2128505088 learned_addr learned my addr 172.21.15.37:0/2128505088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6faffd640 1 --2- 172.21.15.37:0/2128505088 >> v2:172.21.15.203:3300/0 conn(0x7fe6fc15da80 0x7fe6fc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6fb7fe640 1 -- 172.21.15.37:0/2128505088 >> v2:172.21.15.203:3300/0 conn(0x7fe6fc15da80 msgr2=0x7fe6fc0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6fb7fe640 1 --2- 172.21.15.37:0/2128505088 >> v2:172.21.15.203:3300/0 conn(0x7fe6fc15da80 0x7fe6fc0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6fb7fe640 1 -- 172.21.15.37:0/2128505088 >> v1:172.21.15.203:6789/0 conn(0x7fe6fc0d1cd0 legacy=0x7fe6fc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6fb7fe640 1 -- 172.21.15.37:0/2128505088 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc0d2450 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6fb7fe640 1 --2- 172.21.15.37:0/2128505088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6fc0d07f0 0x7fe6fc06c8d0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009e50 tx=0x7fe6f401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fded415b4f0a8d8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6fa7fc640 1 -- 172.21.15.37:0/2128505088 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40a57e0 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6fa7fc640 1 -- 172.21.15.37:0/2128505088 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f40a5980 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.933+0000 7fe6fa7fc640 1 -- 172.21.15.37:0/2128505088 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f40a5d50 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 -- 172.21.15.37:0/2128505088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6fc0d07f0 msgr2=0x7fe6fc06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 --2- 172.21.15.37:0/2128505088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6fc0d07f0 0x7fe6fc06c8d0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009e50 tx=0x7fe6f401c9c0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 -- 172.21.15.37:0/2128505088 shutdown_connections 2024-04-25T09:03:56.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 --2- 172.21.15.37:0/2128505088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6fc0d07f0 0x7fe6fc06c8d0 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 --2- 172.21.15.37:0/2128505088 >> v2:172.21.15.203:3300/0 conn(0x7fe6fc15da80 0x7fe6fc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 -- 172.21.15.37:0/2128505088 >> 172.21.15.37:0/2128505088 conn(0x7fe6fc08b000 msgr2=0x7fe6fc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 -- 172.21.15.37:0/2128505088 shutdown_connections 2024-04-25T09:03:56.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 -- 172.21.15.37:0/2128505088 wait complete. 2024-04-25T09:03:56.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 Processor -- start 2024-04-25T09:03:56.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 -- start start 2024-04-25T09:03:56.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6fc0d07f0 0x7fe6fc155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6fc0d1cd0 0x7fe6fc160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6fc15da80 0x7fe6fc0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc06d7a0 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc06d620 con 0x7fe6fc0d1cd0 2024-04-25T09:03:56.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc06d920 con 0x7fe6fc15da80 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6faffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6fc0d07f0 0x7fe6fc155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6faffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6fc0d07f0 0x7fe6fc155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45616/0 (socket says 172.21.15.37:45616) 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6faffd640 1 -- 172.21.15.37:0/3270124317 learned_addr learned my addr 172.21.15.37:0/3270124317 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6f27fc640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6fc0d1cd0 0x7fe6fc160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6fb7fe640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6fc15da80 0x7fe6fc0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6faffd640 1 -- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6fc15da80 msgr2=0x7fe6fc0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6faffd640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6fc15da80 0x7fe6fc0cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6faffd640 1 -- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6fc0d1cd0 msgr2=0x7fe6fc160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6faffd640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6fc0d1cd0 0x7fe6fc160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6faffd640 1 -- 172.21.15.37:0/3270124317 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc155cf0 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6faffd640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6fc0d07f0 0x7fe6fc155050 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00d5f0 tx=0x7fe6ec00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6f3fff640 1 -- 172.21.15.37:0/3270124317 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0125f0 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6f3fff640 1 -- 172.21.15.37:0/3270124317 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec019070 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 -- 172.21.15.37:0/3270124317 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6fc155fd0 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe6f3fff640 1 -- 172.21.15.37:0/3270124317 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0129d0 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.937+0000 7fe7014f8640 1 -- 172.21.15.37:0/3270124317 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6fc156590 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.941+0000 7fe7014f8640 1 -- 172.21.15.37:0/3270124317 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6fc04efa0 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.941+0000 7fe6f3fff640 1 -- 172.21.15.37:0/3270124317 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe6ec012b70 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.941+0000 7fe6f3fff640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6e0066f60 0x7fe6e0069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.941+0000 7fe6f3fff640 1 -- 172.21.15.37:0/3270124317 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe6ec085700 con 0x7fe6fc0d07f0 2024-04-25T09:03:56.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.941+0000 7fe6f27fc640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6e0066f60 0x7fe6e0069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.941+0000 7fe6f27fc640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6e0066f60 0x7fe6e0069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0b11f0 tx=0x7fe6e4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:56.945+0000 7fe6f3fff640 1 -- 172.21.15.37:0/3270124317 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe6ec0564e0 con 0x7fe6fc0d07f0 2024-04-25T09:03:57.091 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.101+0000 7fe7014f8640 1 -- 172.21.15.37:0/3270124317 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6fc0b2220 con 0x7fe6fc0d07f0 2024-04-25T09:03:57.094 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:57.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.105+0000 7fe6f3fff640 1 -- 172.21.15.37:0/3270124317 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16916 (secure 0 0 0) 0x7fe6ec083e20 con 0x7fe6fc0d07f0 2024-04-25T09:03:57.095 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:57.095 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":125,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:33.770823+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.105+0000 7fe7014f8640 1 -- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6e0066f60 msgr2=0x7fe6e0069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.105+0000 7fe7014f8640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6e0066f60 0x7fe6e0069420 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc0b11f0 tx=0x7fe6e4009040 comp rx=0 tx=0).stop 2024-04-25T09:03:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.105+0000 7fe7014f8640 1 -- 172.21.15.37:0/3270124317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6fc0d07f0 msgr2=0x7fe6fc155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.105+0000 7fe7014f8640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6fc0d07f0 0x7fe6fc155050 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00d5f0 tx=0x7fe6ec00dab0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.105+0000 7fe7014f8640 1 -- 172.21.15.37:0/3270124317 shutdown_connections 2024-04-25T09:03:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.105+0000 7fe7014f8640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe6e0066f60 0x7fe6e0069420 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.105+0000 7fe7014f8640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe6fc15da80 0x7fe6fc0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.105+0000 7fe7014f8640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe6fc0d1cd0 0x7fe6fc160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.105+0000 7fe7014f8640 1 --2- 172.21.15.37:0/3270124317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6fc0d07f0 0x7fe6fc155050 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.105+0000 7fe7014f8640 1 -- 172.21.15.37:0/3270124317 >> 172.21.15.37:0/3270124317 conn(0x7fe6fc08b000 msgr2=0x7fe6fc0d2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.109+0000 7fe7014f8640 1 -- 172.21.15.37:0/3270124317 shutdown_connections 2024-04-25T09:03:57.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.109+0000 7fe7014f8640 1 -- 172.21.15.37:0/3270124317 wait complete. 2024-04-25T09:03:57.113 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:30.763724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:31.488462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:57.114 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:57.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f2902aa3640 1 Processor -- start 2024-04-25T09:03:57.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f2902aa3640 1 -- start start 2024-04-25T09:03:57.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f2902aa3640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f28fc15b6b0 0x7f28fc0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f2902aa3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28fc073010 0x7f28fc0cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f2902aa3640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28fc0cbd20 con 0x7f28fc0cf590 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f2902aa3640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28fc15ba90 con 0x7f28fc15b6b0 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f2902aa3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0cc000 con 0x7f28fc073010 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f290083f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f28fc15b6b0 0x7f28fc0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f290083f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f28fc15b6b0 0x7f28fc0ca3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45628/0 (socket says 172.21.15.37:45628) 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f290083f640 1 -- 172.21.15.37:0/535972378 learned_addr learned my addr 172.21.15.37:0/535972378 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f2901040640 1 --2- 172.21.15.37:0/535972378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28fc073010 0x7f28fc0cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f290083f640 1 -- 172.21.15.37:0/535972378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28fc073010 msgr2=0x7f28fc0cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f290083f640 1 --2- 172.21.15.37:0/535972378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28fc073010 0x7f28fc0cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f290083f640 1 -- 172.21.15.37:0/535972378 >> v1:172.21.15.203:6789/0 conn(0x7f28fc0cf590 legacy=0x7f28fc15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.321+0000 7f290083f640 1 -- 172.21.15.37:0/535972378 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc0cc2e0 con 0x7f28fc15b6b0 2024-04-25T09:03:57.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f290083f640 1 --2- 172.21.15.37:0/535972378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28fc15b6b0 0x7f28fc0ca3c0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009870 tx=0x7f28ec01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431d079dc14b1777 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f28fb7fe640 1 -- 172.21.15.37:0/535972378 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0b0070 con 0x7f28fc15b6b0 2024-04-25T09:03:57.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f28fb7fe640 1 -- 172.21.15.37:0/535972378 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ec0ab600 con 0x7f28fc15b6b0 2024-04-25T09:03:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f28fb7fe640 1 -- 172.21.15.37:0/535972378 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0ab9d0 con 0x7f28fc15b6b0 2024-04-25T09:03:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f2902aa3640 1 -- 172.21.15.37:0/535972378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28fc15b6b0 msgr2=0x7f28fc0ca3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f2902aa3640 1 --2- 172.21.15.37:0/535972378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28fc15b6b0 0x7f28fc0ca3c0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009870 tx=0x7f28ec01c560 comp rx=0 tx=0).stop 2024-04-25T09:03:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f2902aa3640 1 -- 172.21.15.37:0/535972378 shutdown_connections 2024-04-25T09:03:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f2902aa3640 1 --2- 172.21.15.37:0/535972378 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28fc073010 0x7f28fc0cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f2902aa3640 1 --2- 172.21.15.37:0/535972378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28fc15b6b0 0x7f28fc0ca3c0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f2902aa3640 1 -- 172.21.15.37:0/535972378 >> 172.21.15.37:0/535972378 conn(0x7f28fc06ae60 msgr2=0x7f28fc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f2902aa3640 1 -- 172.21.15.37:0/535972378 shutdown_connections 2024-04-25T09:03:57.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.325+0000 7f2902aa3640 1 -- 172.21.15.37:0/535972378 wait complete. 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2902aa3640 1 Processor -- start 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2902aa3640 1 -- start start 2024-04-25T09:03:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2902aa3640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28fc073010 0x7f28fc0957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2902aa3640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28fc0cf590 0x7f28fc164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2902aa3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28fc15b6b0 0x7f28fc164bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2902aa3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0cadb0 con 0x7f28fc15b6b0 2024-04-25T09:03:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2902aa3640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0ca900 con 0x7f28fc0cf590 2024-04-25T09:03:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2902aa3640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28fc0caf30 con 0x7f28fc073010 2024-04-25T09:03:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2901040640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28fc15b6b0 0x7f28fc164bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2901040640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28fc15b6b0 0x7f28fc164bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45632/0 (socket says 172.21.15.37:45632) 2024-04-25T09:03:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2901040640 1 -- 172.21.15.37:0/3887535792 learned_addr learned my addr 172.21.15.37:0/3887535792 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f28fbfff640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28fc0cf590 0x7f28fc164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f290083f640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28fc073010 0x7f28fc0957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2901040640 1 -- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28fc073010 msgr2=0x7f28fc0957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2901040640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28fc073010 0x7f28fc0957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2901040640 1 -- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28fc0cf590 msgr2=0x7f28fc164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2901040640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28fc0cf590 0x7f28fc164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2901040640 1 -- 172.21.15.37:0/3887535792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc0aff00 con 0x7f28fc15b6b0 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2901040640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28fc15b6b0 0x7f28fc164bb0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f28f4007e30 tx=0x7f28f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f28f97fa640 1 -- 172.21.15.37:0/3887535792 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4014620 con 0x7f28fc15b6b0 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f28f97fa640 1 -- 172.21.15.37:0/3887535792 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4019070 con 0x7f28fc15b6b0 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2902aa3640 1 -- 172.21.15.37:0/3887535792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28fc0988c0 con 0x7f28fc15b6b0 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f28f97fa640 1 -- 172.21.15.37:0/3887535792 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4014a00 con 0x7f28fc15b6b0 2024-04-25T09:03:57.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2902aa3640 1 -- 172.21.15.37:0/3887535792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28fc1582f0 con 0x7f28fc15b6b0 2024-04-25T09:03:57.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f2902aa3640 1 -- 172.21.15.37:0/3887535792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28fc04efa0 con 0x7f28fc15b6b0 2024-04-25T09:03:57.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.329+0000 7f28f97fa640 1 -- 172.21.15.37:0/3887535792 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f28f4014ba0 con 0x7f28fc15b6b0 2024-04-25T09:03:57.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.333+0000 7f28f97fa640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f28e8066f60 0x7f28e8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.333+0000 7f28f97fa640 1 -- 172.21.15.37:0/3887535792 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f28f4085ae0 con 0x7f28fc15b6b0 2024-04-25T09:03:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.333+0000 7f290083f640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f28e8066f60 0x7f28e8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.333+0000 7f290083f640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f28e8066f60 0x7f28e8069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0ad000 tx=0x7f28ec008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.337+0000 7f28f97fa640 1 -- 172.21.15.37:0/3887535792 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f28f40568c0 con 0x7f28fc15b6b0 2024-04-25T09:03:57.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.497+0000 7f2902aa3640 1 -- 172.21.15.37:0/3887535792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f28fc06bfc0 con 0x7f28fc15b6b0 2024-04-25T09:03:57.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.497+0000 7f28f97fa640 1 -- 172.21.15.37:0/3887535792 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) v1 ==== 93+0+10 (secure 0 0 0) 0x7f28f40561c0 con 0x7f28fc15b6b0 2024-04-25T09:03:57.488 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:03:57.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 -- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f28e8066f60 msgr2=0x7f28e8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f28e8066f60 0x7f28e8069420 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0ad000 tx=0x7f28ec008040 comp rx=0 tx=0).stop 2024-04-25T09:03:57.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 -- 172.21.15.37:0/3887535792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28fc15b6b0 msgr2=0x7f28fc164bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28fc15b6b0 0x7f28fc164bb0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f28f4007e30 tx=0x7f28f400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 -- 172.21.15.37:0/3887535792 shutdown_connections 2024-04-25T09:03:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f28e8066f60 0x7f28e8069420 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28fc15b6b0 0x7f28fc164bb0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f28fc0cf590 0x7f28fc164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 --2- 172.21.15.37:0/3887535792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f28fc073010 0x7f28fc0957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 -- 172.21.15.37:0/3887535792 >> 172.21.15.37:0/3887535792 conn(0x7f28fc06ae60 msgr2=0x7f28fc0b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 -- 172.21.15.37:0/3887535792 shutdown_connections 2024-04-25T09:03:57.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.501+0000 7f2902aa3640 1 -- 172.21.15.37:0/3887535792 wait complete. 2024-04-25T09:03:57.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:57.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf72b1640 1 Processor -- start 2024-04-25T09:03:57.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf72b1640 1 -- start start 2024-04-25T09:03:57.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf72b1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf00b2c30 0x7fcbf00b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf72b1640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcbf009a7b0 0x7fcbf0099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf72b1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf00cbec0 con 0x7fcbf0099fb0 2024-04-25T09:03:57.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf72b1640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf00cc1a0 con 0x7fcbf009a7b0 2024-04-25T09:03:57.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf72b1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00cc480 con 0x7fcbf00b2c30 2024-04-25T09:03:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf484c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcbf009a7b0 0x7fcbf0099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf504d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf00b2c30 0x7fcbf00b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf484c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcbf009a7b0 0x7fcbf0099a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:43780/0 (socket says 172.21.15.37:43780) 2024-04-25T09:03:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf484c640 1 -- 172.21.15.37:0/1135635584 learned_addr learned my addr 172.21.15.37:0/1135635584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf584e640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fcbf0099fb0 0x7fcbf00ccf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:53930/0 (socket says 172.21.15.37:53930) 2024-04-25T09:03:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf504d640 1 -- 172.21.15.37:0/1135635584 >> v2:172.21.15.203:3300/0 conn(0x7fcbf009a7b0 msgr2=0x7fcbf0099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf504d640 1 --2- 172.21.15.37:0/1135635584 >> v2:172.21.15.203:3300/0 conn(0x7fcbf009a7b0 0x7fcbf0099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf504d640 1 -- 172.21.15.37:0/1135635584 >> v1:172.21.15.203:6789/0 conn(0x7fcbf0099fb0 legacy=0x7fcbf00ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.633+0000 7fcbf504d640 1 -- 172.21.15.37:0/1135635584 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00cc760 con 0x7fcbf00b2c30 2024-04-25T09:03:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf504d640 1 --2- 172.21.15.37:0/1135635584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf00b2c30 0x7fcbf00b0ba0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009d90 tx=0x7fcbec01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a8adfc85ca9fefd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbe7fff640 1 -- 172.21.15.37:0/1135635584 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0b2070 con 0x7fcbf00b2c30 2024-04-25T09:03:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbe7fff640 1 -- 172.21.15.37:0/1135635584 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbec0a5910 con 0x7fcbf00b2c30 2024-04-25T09:03:57.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbe7fff640 1 -- 172.21.15.37:0/1135635584 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0a5ce0 con 0x7fcbf00b2c30 2024-04-25T09:03:57.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 -- 172.21.15.37:0/1135635584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf00b2c30 msgr2=0x7fcbf00b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 --2- 172.21.15.37:0/1135635584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf00b2c30 0x7fcbf00b0ba0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcbec009d90 tx=0x7fcbec01c940 comp rx=0 tx=0).stop 2024-04-25T09:03:57.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 -- 172.21.15.37:0/1135635584 shutdown_connections 2024-04-25T09:03:57.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 --2- 172.21.15.37:0/1135635584 >> v2:172.21.15.203:3300/0 conn(0x7fcbf009a7b0 0x7fcbf0099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 --2- 172.21.15.37:0/1135635584 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf00b2c30 0x7fcbf00b0ba0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 -- 172.21.15.37:0/1135635584 >> 172.21.15.37:0/1135635584 conn(0x7fcbf00b3010 msgr2=0x7fcbf00b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 -- 172.21.15.37:0/1135635584 shutdown_connections 2024-04-25T09:03:57.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 -- 172.21.15.37:0/1135635584 wait complete. 2024-04-25T09:03:57.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 Processor -- start 2024-04-25T09:03:57.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 -- start start 2024-04-25T09:03:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf0099fb0 0x7fcbf0151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbf009a7b0 0x7fcbf0151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf00b2c30 0x7fcbf00b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf009ad10 con 0x7fcbf00b2c30 2024-04-25T09:03:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf009ab90 con 0x7fcbf009a7b0 2024-04-25T09:03:57.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf72b1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf015d190 con 0x7fcbf0099fb0 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf584e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf00b2c30 0x7fcbf00b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf584e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf00b2c30 0x7fcbf00b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45646/0 (socket says 172.21.15.37:45646) 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.637+0000 7fcbf584e640 1 -- 172.21.15.37:0/3872185495 learned_addr learned my addr 172.21.15.37:0/3872185495 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf484c640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbf009a7b0 0x7fcbf0151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf504d640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf0099fb0 0x7fcbf0151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf584e640 1 -- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf0099fb0 msgr2=0x7fcbf0151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf584e640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf0099fb0 0x7fcbf0151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf584e640 1 -- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbf009a7b0 msgr2=0x7fcbf0151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf584e640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbf009a7b0 0x7fcbf0151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf584e640 1 -- 172.21.15.37:0/3872185495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf01598f0 con 0x7fcbf00b2c30 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf504d640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf0099fb0 0x7fcbf0151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf584e640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf00b2c30 0x7fcbf00b4140 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800c390 tx=0x7fcbe800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbe5ffb640 1 -- 172.21.15.37:0/3872185495 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe80074e0 con 0x7fcbf00b2c30 2024-04-25T09:03:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbe5ffb640 1 -- 172.21.15.37:0/3872185495 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8007680 con 0x7fcbf00b2c30 2024-04-25T09:03:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf72b1640 1 -- 172.21.15.37:0/3872185495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf0159bd0 con 0x7fcbf00b2c30 2024-04-25T09:03:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbe5ffb640 1 -- 172.21.15.37:0/3872185495 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8004a10 con 0x7fcbf00b2c30 2024-04-25T09:03:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf72b1640 1 -- 172.21.15.37:0/3872185495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf015a190 con 0x7fcbf00b2c30 2024-04-25T09:03:57.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbe5ffb640 1 -- 172.21.15.37:0/3872185495 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcbe801a020 con 0x7fcbf00b2c30 2024-04-25T09:03:57.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf72b1640 1 -- 172.21.15.37:0/3872185495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf004efa0 con 0x7fcbf00b2c30 2024-04-25T09:03:57.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbe5ffb640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbd4066f60 0x7fcbd4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbe5ffb640 1 -- 172.21.15.37:0/3872185495 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcbe80849b0 con 0x7fcbf00b2c30 2024-04-25T09:03:57.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf504d640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbd4066f60 0x7fcbd4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.641+0000 7fcbf504d640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbd4066f60 0x7fcbd4069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0af000 tx=0x7fcbec0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.645+0000 7fcbe5ffb640 1 -- 172.21.15.37:0/3872185495 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcbe8055830 con 0x7fcbf00b2c30 2024-04-25T09:03:57.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.797+0000 7fcbf72b1640 1 -- 172.21.15.37:0/3872185495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcbf006ba80 con 0x7fcbf00b2c30 2024-04-25T09:03:57.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.797+0000 7fcbe5ffb640 1 -- 172.21.15.37:0/3872185495 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcbe8055130 con 0x7fcbf00b2c30 2024-04-25T09:03:57.787 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:03:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 -- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbd4066f60 msgr2=0x7fcbd4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbd4066f60 0x7fcbd4069420 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fcbec0af000 tx=0x7fcbec0a7070 comp rx=0 tx=0).stop 2024-04-25T09:03:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 -- 172.21.15.37:0/3872185495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf00b2c30 msgr2=0x7fcbf00b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf00b2c30 0x7fcbf00b4140 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800c390 tx=0x7fcbe800c850 comp rx=0 tx=0).stop 2024-04-25T09:03:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 -- 172.21.15.37:0/3872185495 shutdown_connections 2024-04-25T09:03:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcbd4066f60 0x7fcbd4069420 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcbf00b2c30 0x7fcbf00b4140 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcbf009a7b0 0x7fcbf0151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 --2- 172.21.15.37:0/3872185495 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcbf0099fb0 0x7fcbf0151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 -- 172.21.15.37:0/3872185495 >> 172.21.15.37:0/3872185495 conn(0x7fcbf00b3010 msgr2=0x7fcbf0158fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 -- 172.21.15.37:0/3872185495 shutdown_connections 2024-04-25T09:03:57.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.801+0000 7fcbf72b1640 1 -- 172.21.15.37:0/3872185495 wait complete. 2024-04-25T09:03:57.805 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:57.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.929+0000 7f633d8aa640 1 Processor -- start 2024-04-25T09:03:57.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.929+0000 7f633d8aa640 1 -- start start 2024-04-25T09:03:57.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f633d8aa640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f63380cf590 0x7f633815ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f633d8aa640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f63380cf970 0x7f63380c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f633d8aa640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63380ca4c0 con 0x7f63380c9ef0 2024-04-25T09:03:57.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f633d8aa640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63380ca780 con 0x7f63380cf590 2024-04-25T09:03:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f633d8aa640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63380cd440 con 0x7f63380cf970 2024-04-25T09:03:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f6336ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f63380cf590 0x7f633815ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f6336ffd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f63380cf590 0x7f633815ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45660/0 (socket says 172.21.15.37:45660) 2024-04-25T09:03:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f6336ffd640 1 -- 172.21.15.37:0/4196314819 learned_addr learned my addr 172.21.15.37:0/4196314819 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f632e7fc640 1 --2- 172.21.15.37:0/4196314819 >> v2:172.21.15.203:3300/0 conn(0x7f63380cf970 0x7f63380c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f6336ffd640 1 -- 172.21.15.37:0/4196314819 >> v2:172.21.15.203:3300/0 conn(0x7f63380cf970 msgr2=0x7f63380c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f6336ffd640 1 --2- 172.21.15.37:0/4196314819 >> v2:172.21.15.203:3300/0 conn(0x7f63380cf970 0x7f63380c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f6336ffd640 1 -- 172.21.15.37:0/4196314819 >> v1:172.21.15.203:6789/0 conn(0x7f63380c9ef0 legacy=0x7f63380b21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f6336ffd640 1 -- 172.21.15.37:0/4196314819 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380cd690 con 0x7f63380cf590 2024-04-25T09:03:57.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f6336ffd640 1 --2- 172.21.15.37:0/4196314819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63380cf590 0x7f633815ecb0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f63380d00e0 tx=0x7f632801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f66549c90e24540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f63367fc640 1 -- 172.21.15.37:0/4196314819 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280b1070 con 0x7f63380cf590 2024-04-25T09:03:57.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f63367fc640 1 -- 172.21.15.37:0/4196314819 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63280ac640 con 0x7f63380cf590 2024-04-25T09:03:57.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f63367fc640 1 -- 172.21.15.37:0/4196314819 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280aca10 con 0x7f63380cf590 2024-04-25T09:03:57.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f633d8aa640 1 -- 172.21.15.37:0/4196314819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63380cf590 msgr2=0x7f633815ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.933+0000 7f633d8aa640 1 --2- 172.21.15.37:0/4196314819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63380cf590 0x7f633815ecb0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f63380d00e0 tx=0x7f632801c4e0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 -- 172.21.15.37:0/4196314819 shutdown_connections 2024-04-25T09:03:57.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 --2- 172.21.15.37:0/4196314819 >> v2:172.21.15.203:3300/0 conn(0x7f63380cf970 0x7f63380c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 --2- 172.21.15.37:0/4196314819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63380cf590 0x7f633815ecb0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 -- 172.21.15.37:0/4196314819 >> 172.21.15.37:0/4196314819 conn(0x7f633806ae60 msgr2=0x7f633806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 -- 172.21.15.37:0/4196314819 shutdown_connections 2024-04-25T09:03:57.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 -- 172.21.15.37:0/4196314819 wait complete. 2024-04-25T09:03:57.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 Processor -- start 2024-04-25T09:03:57.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 -- start start 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63380c9ef0 0x7f633806d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63380cf590 0x7f633815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63380cf970 0x7f633815c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6338167660 con 0x7f63380cf970 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63381674e0 con 0x7f63380c9ef0 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63381677e0 con 0x7f63380cf590 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f63377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63380cf970 0x7f633815c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f63377fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63380cf970 0x7f633815c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45662/0 (socket says 172.21.15.37:45662) 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f63377fe640 1 -- 172.21.15.37:0/1187666456 learned_addr learned my addr 172.21.15.37:0/1187666456 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f6336ffd640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63380c9ef0 0x7f633806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f632e7fc640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63380cf590 0x7f633815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f63377fe640 1 -- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63380cf590 msgr2=0x7f633815bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f63377fe640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63380cf590 0x7f633815bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f63377fe640 1 -- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63380c9ef0 msgr2=0x7f633806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f63377fe640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63380c9ef0 0x7f633806d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f63377fe640 1 -- 172.21.15.37:0/1187666456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380cca20 con 0x7f63380cf970 2024-04-25T09:03:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f6336ffd640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63380c9ef0 0x7f633806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f63377fe640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63380cf970 0x7f633815c160 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f633000b9a0 tx=0x7f633000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f6334ff9640 1 -- 172.21.15.37:0/1187666456 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330019070 con 0x7f63380cf970 2024-04-25T09:03:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f6334ff9640 1 -- 172.21.15.37:0/1187666456 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f633000cba0 con 0x7f63380cf970 2024-04-25T09:03:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 -- 172.21.15.37:0/1187666456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633806e9d0 con 0x7f63380cf970 2024-04-25T09:03:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f6334ff9640 1 -- 172.21.15.37:0/1187666456 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330014440 con 0x7f63380cf970 2024-04-25T09:03:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.937+0000 7f633d8aa640 1 -- 172.21.15.37:0/1187666456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633806ec70 con 0x7f63380cf970 2024-04-25T09:03:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.941+0000 7f633d8aa640 1 -- 172.21.15.37:0/1187666456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633804efa0 con 0x7f63380cf970 2024-04-25T09:03:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.941+0000 7f6334ff9640 1 -- 172.21.15.37:0/1187666456 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f633001c070 con 0x7f63380cf970 2024-04-25T09:03:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.941+0000 7f6334ff9640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6318066f60 0x7f6318069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.941+0000 7f6334ff9640 1 -- 172.21.15.37:0/1187666456 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6330084d30 con 0x7f63380cf970 2024-04-25T09:03:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.941+0000 7f6336ffd640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6318066f60 0x7f6318069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.941+0000 7f6336ffd640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6318066f60 0x7f6318069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f63280ae000 tx=0x7f6328008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:57.945+0000 7f6334ff9640 1 -- 172.21.15.37:0/1187666456 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6330055c40 con 0x7f63380cf970 2024-04-25T09:03:58.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.101+0000 7f633d8aa640 1 -- 172.21.15.37:0/1187666456 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63380b4170 con 0x7f63380cf970 2024-04-25T09:03:58.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.101+0000 7f6334ff9640 1 -- 172.21.15.37:0/1187666456 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6330055540 con 0x7f63380cf970 2024-04-25T09:03:58.091 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:03:58.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 -- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6318066f60 msgr2=0x7f6318069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6318066f60 0x7f6318069420 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f63280ae000 tx=0x7f6328008040 comp rx=0 tx=0).stop 2024-04-25T09:03:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 -- 172.21.15.37:0/1187666456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63380cf970 msgr2=0x7f633815c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63380cf970 0x7f633815c160 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f633000b9a0 tx=0x7f633000be60 comp rx=0 tx=0).stop 2024-04-25T09:03:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 -- 172.21.15.37:0/1187666456 shutdown_connections 2024-04-25T09:03:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6318066f60 0x7f6318069420 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63380cf970 0x7f633815c160 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f63380cf590 0x7f633815bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 --2- 172.21.15.37:0/1187666456 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f63380c9ef0 0x7f633806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 -- 172.21.15.37:0/1187666456 >> 172.21.15.37:0/1187666456 conn(0x7f633806ae60 msgr2=0x7f633815e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 -- 172.21.15.37:0/1187666456 shutdown_connections 2024-04-25T09:03:58.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.105+0000 7f633d8aa640 1 -- 172.21.15.37:0/1187666456 wait complete. 2024-04-25T09:03:58.110 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:03:58.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e87f6640 1 Processor -- start 2024-04-25T09:03:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e87f6640 1 -- start start 2024-04-25T09:03:58.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e87f6640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f45e015a700 0x7f45e015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e87f6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e00b3f50 0x7f45e00cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e87f6640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f45e00ce430 0x7f45e00b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e87f6640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e015f200 con 0x7f45e00ce430 2024-04-25T09:03:58.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e87f6640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e00b2090 con 0x7f45e015a700 2024-04-25T09:03:58.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e87f6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00b43e0 con 0x7f45e00b3f50 2024-04-25T09:03:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e6d93640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f45e00ce430 0x7f45e00b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e6d93640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f45e00ce430 0x7f45e00b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45672/0 (socket says 172.21.15.37:45672) 2024-04-25T09:03:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e6d93640 1 -- 172.21.15.37:0/4154871016 learned_addr learned my addr 172.21.15.37:0/4154871016 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e6592640 1 --2- 172.21.15.37:0/4154871016 >> v2:172.21.15.203:3300/0 conn(0x7f45e015a700 0x7f45e015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.237+0000 7f45e5d91640 1 --2- 172.21.15.37:0/4154871016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e00b3f50 0x7f45e00cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.241+0000 7f45e6d93640 1 -- 172.21.15.37:0/4154871016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e00b3f50 msgr2=0x7f45e00cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.241+0000 7f45e6d93640 1 --2- 172.21.15.37:0/4154871016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e00b3f50 0x7f45e00cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.241+0000 7f45e6d93640 1 -- 172.21.15.37:0/4154871016 >> v2:172.21.15.203:3300/0 conn(0x7f45e015a700 msgr2=0x7f45e015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.241+0000 7f45e6d93640 1 --2- 172.21.15.37:0/4154871016 >> v2:172.21.15.203:3300/0 conn(0x7f45e015a700 0x7f45e015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.241+0000 7f45e6d93640 1 -- 172.21.15.37:0/4154871016 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e00b46c0 con 0x7f45e00ce430 2024-04-25T09:03:58.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.241+0000 7f45e5d91640 1 --2- 172.21.15.37:0/4154871016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e00b3f50 0x7f45e00cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:58.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.241+0000 7f45e6d93640 1 --2- 172.21.15.37:0/4154871016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45e00ce430 0x7f45e00b24a0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f45e006bd20 tx=0x7f45d801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d473fb9ec3ef9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.241+0000 7f45e5590640 1 -- 172.21.15.37:0/4154871016 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d80a5630 con 0x7f45e00ce430 2024-04-25T09:03:58.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.241+0000 7f45e5590640 1 -- 172.21.15.37:0/4154871016 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d80a57d0 con 0x7f45e00ce430 2024-04-25T09:03:58.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.241+0000 7f45e5590640 1 -- 172.21.15.37:0/4154871016 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d80a5ba0 con 0x7f45e00ce430 2024-04-25T09:03:58.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 -- 172.21.15.37:0/4154871016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45e00ce430 msgr2=0x7f45e00b24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 --2- 172.21.15.37:0/4154871016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45e00ce430 0x7f45e00b24a0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f45e006bd20 tx=0x7f45d801c8a0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 -- 172.21.15.37:0/4154871016 shutdown_connections 2024-04-25T09:03:58.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 --2- 172.21.15.37:0/4154871016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45e00ce430 0x7f45e00b24a0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 --2- 172.21.15.37:0/4154871016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e00b3f50 0x7f45e00cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 --2- 172.21.15.37:0/4154871016 >> v2:172.21.15.203:3300/0 conn(0x7f45e015a700 0x7f45e015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 -- 172.21.15.37:0/4154871016 >> 172.21.15.37:0/4154871016 conn(0x7f45e00b11a0 msgr2=0x7f45e00b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 -- 172.21.15.37:0/4154871016 shutdown_connections 2024-04-25T09:03:58.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 -- 172.21.15.37:0/4154871016 wait complete. 2024-04-25T09:03:58.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 Processor -- start 2024-04-25T09:03:58.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 -- start start 2024-04-25T09:03:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e00ce430 0x7f45e014fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45e015a700 0x7f45e01399b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e0139ef0 0x7f45e0153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00cd490 con 0x7f45e015a700 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e00cd310 con 0x7f45e00ce430 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e0071030 con 0x7f45e0139ef0 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e5d91640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45e015a700 0x7f45e01399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e5d91640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45e015a700 0x7f45e01399b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45688/0 (socket says 172.21.15.37:45688) 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e5d91640 1 -- 172.21.15.37:0/884390708 learned_addr learned my addr 172.21.15.37:0/884390708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e6592640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e00ce430 0x7f45e014fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e6d93640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e0139ef0 0x7f45e0153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e5d91640 1 -- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e0139ef0 msgr2=0x7f45e0153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e5d91640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e0139ef0 0x7f45e0153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e5d91640 1 -- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e00ce430 msgr2=0x7f45e014fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e5d91640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e00ce430 0x7f45e014fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e5d91640 1 -- 172.21.15.37:0/884390708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e01549d0 con 0x7f45e015a700 2024-04-25T09:03:58.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e6d93640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e0139ef0 0x7f45e0153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:58.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e5d91640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45e015a700 0x7f45e01399b0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f45d000d800 tx=0x7f45d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45d6ffd640 1 -- 172.21.15.37:0/884390708 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0018070 con 0x7f45e015a700 2024-04-25T09:03:58.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45d6ffd640 1 -- 172.21.15.37:0/884390708 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d0004030 con 0x7f45e015a700 2024-04-25T09:03:58.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45e87f6640 1 -- 172.21.15.37:0/884390708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e0154cb0 con 0x7f45e015a700 2024-04-25T09:03:58.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.245+0000 7f45d6ffd640 1 -- 172.21.15.37:0/884390708 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0004400 con 0x7f45e015a700 2024-04-25T09:03:58.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.249+0000 7f45e87f6640 1 -- 172.21.15.37:0/884390708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e0155270 con 0x7f45e015a700 2024-04-25T09:03:58.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.249+0000 7f45e87f6640 1 -- 172.21.15.37:0/884390708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e013a300 con 0x7f45e015a700 2024-04-25T09:03:58.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.249+0000 7f45d6ffd640 1 -- 172.21.15.37:0/884390708 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f45d00096a0 con 0x7f45e015a700 2024-04-25T09:03:58.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.249+0000 7f45d6ffd640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45c4066f60 0x7f45c4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.249+0000 7f45d6ffd640 1 -- 172.21.15.37:0/884390708 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f45d0085570 con 0x7f45e015a700 2024-04-25T09:03:58.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.249+0000 7f45e6592640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45c4066f60 0x7f45c4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.249+0000 7f45e6592640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45c4066f60 0x7f45c4069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009af0 tx=0x7f45dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.253+0000 7f45d6ffd640 1 -- 172.21.15.37:0/884390708 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f45d00563f0 con 0x7f45e015a700 2024-04-25T09:03:58.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.421+0000 7f45e87f6640 1 -- 172.21.15.37:0/884390708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f45e004efa0 con 0x7f45e015a700 2024-04-25T09:03:58.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45d6ffd640 1 -- 172.21.15.37:0/884390708 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) v1 ==== 80+0+0 (secure 0 0 0) 0x7f45d0055cf0 con 0x7f45e015a700 2024-04-25T09:03:58.412 INFO:teuthology.orchestra.run.smithi037.stderr:listed 0 entries 2024-04-25T09:03:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 -- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45c4066f60 msgr2=0x7f45c4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45c4066f60 0x7f45c4069420 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f45dc009af0 tx=0x7f45dc008040 comp rx=0 tx=0).stop 2024-04-25T09:03:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 -- 172.21.15.37:0/884390708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45e015a700 msgr2=0x7f45e01399b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45e015a700 0x7f45e01399b0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f45d000d800 tx=0x7f45d000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 -- 172.21.15.37:0/884390708 shutdown_connections 2024-04-25T09:03:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f45c4066f60 0x7f45c4069420 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f45e0139ef0 0x7f45e0153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45e015a700 0x7f45e01399b0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 --2- 172.21.15.37:0/884390708 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f45e00ce430 0x7f45e014fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 -- 172.21.15.37:0/884390708 >> 172.21.15.37:0/884390708 conn(0x7f45e00b11a0 msgr2=0x7f45e00b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 -- 172.21.15.37:0/884390708 shutdown_connections 2024-04-25T09:03:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.425+0000 7f45e87f6640 1 -- 172.21.15.37:0/884390708 wait complete. 2024-04-25T09:03:58.431 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T09:03:58.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:58.431 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:58.441 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[54802]: fuse finished with error 0 and tester_r 0 2024-04-25T09:03:58.444 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:58.503 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:03:58.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:58.503 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:03:58.507 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:03:58.508 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:58.553 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:03:58.553 DEBUG:tasks.cephfs.fuse_mount:ceph-fuse client.0 is not mounted at ubuntu@smithi037.front.sepia.ceph.com /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:58.553 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:03:58.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:58.553 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:03:58.599 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-04-25T09:03:58.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:58.600 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:03:58.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d3186640 1 Processor -- start 2024-04-25T09:03:58.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d3186640 1 -- start start 2024-04-25T09:03:58.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d3186640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f02cc143790 0x7f02cc06d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d3186640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 0x7f02cc0b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d3186640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02cc143b70 con 0x7f02cc15da40 2024-04-25T09:03:58.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d3186640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02cc06e170 con 0x7f02cc143790 2024-04-25T09:03:58.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d3186640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc06e450 con 0x7f02cc06c3b0 2024-04-25T09:03:58.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d0f22640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f02cc143790 0x7f02cc06d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d0f22640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f02cc143790 0x7f02cc06d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45702/0 (socket says 172.21.15.37:45702) 2024-04-25T09:03:58.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d0f22640 1 -- 172.21.15.37:0/3425529070 learned_addr learned my addr 172.21.15.37:0/3425529070 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d1723640 1 --2- 172.21.15.37:0/3425529070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 0x7f02cc0b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d0f22640 1 -- 172.21.15.37:0/3425529070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 msgr2=0x7f02cc0b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d0f22640 1 --2- 172.21.15.37:0/3425529070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 0x7f02cc0b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d0f22640 1 -- 172.21.15.37:0/3425529070 >> v1:172.21.15.203:6789/0 conn(0x7f02cc15da40 legacy=0x7f02cc06bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d0f22640 1 -- 172.21.15.37:0/3425529070 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc06e730 con 0x7f02cc143790 2024-04-25T09:03:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.773+0000 7f02d1723640 1 --2- 172.21.15.37:0/3425529070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 0x7f02cc0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d0f22640 1 --2- 172.21.15.37:0/3425529070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc143790 0x7f02cc06d870 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00af60 tx=0x7f02bc01daf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08221e49cf98263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02cb7fe640 1 -- 172.21.15.37:0/3425529070 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc00a2d0 con 0x7f02cc143790 2024-04-25T09:03:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02cb7fe640 1 -- 172.21.15.37:0/3425529070 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02bc0a6710 con 0x7f02cc143790 2024-04-25T09:03:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02cb7fe640 1 -- 172.21.15.37:0/3425529070 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02bc0a6ae0 con 0x7f02cc143790 2024-04-25T09:03:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 -- 172.21.15.37:0/3425529070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc143790 msgr2=0x7f02cc06d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 --2- 172.21.15.37:0/3425529070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc143790 0x7f02cc06d870 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f02bc00af60 tx=0x7f02bc01daf0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 -- 172.21.15.37:0/3425529070 shutdown_connections 2024-04-25T09:03:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 --2- 172.21.15.37:0/3425529070 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 0x7f02cc0b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 --2- 172.21.15.37:0/3425529070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc143790 0x7f02cc06d870 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 -- 172.21.15.37:0/3425529070 >> 172.21.15.37:0/3425529070 conn(0x7f02cc071810 msgr2=0x7f02cc071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 -- 172.21.15.37:0/3425529070 shutdown_connections 2024-04-25T09:03:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 -- 172.21.15.37:0/3425529070 wait complete. 2024-04-25T09:03:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 Processor -- start 2024-04-25T09:03:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 -- start start 2024-04-25T09:03:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 0x7f02cc142b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02cc143790 0x7f02cc1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc15da40 0x7f02cc140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc0b3b00 con 0x7f02cc15da40 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc0b3980 con 0x7f02cc143790 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d3186640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02cc0b3c80 con 0x7f02cc06c3b0 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d1723640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc15da40 0x7f02cc140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d1723640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc15da40 0x7f02cc140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45712/0 (socket says 172.21.15.37:45712) 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.777+0000 7f02d1723640 1 -- 172.21.15.37:0/770917783 learned_addr learned my addr 172.21.15.37:0/770917783 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d0f22640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 0x7f02cc142b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02cbfff640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02cc143790 0x7f02cc1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d1723640 1 -- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 msgr2=0x7f02cc142b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d1723640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 0x7f02cc142b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d1723640 1 -- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02cc143790 msgr2=0x7f02cc1590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d1723640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02cc143790 0x7f02cc1590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d1723640 1 -- 172.21.15.37:0/770917783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02cc140440 con 0x7f02cc15da40 2024-04-25T09:03:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d0f22640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 0x7f02cc142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02cbfff640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02cc143790 0x7f02cc1590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d1723640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc15da40 0x7f02cc140b00 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f02c400c7f0 tx=0x7f02c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02c97fa640 1 -- 172.21.15.37:0/770917783 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c4015070 con 0x7f02cc15da40 2024-04-25T09:03:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02c97fa640 1 -- 172.21.15.37:0/770917783 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c4007860 con 0x7f02cc15da40 2024-04-25T09:03:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d3186640 1 -- 172.21.15.37:0/770917783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02cc13cdc0 con 0x7f02cc15da40 2024-04-25T09:03:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02c97fa640 1 -- 172.21.15.37:0/770917783 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c4007c50 con 0x7f02cc15da40 2024-04-25T09:03:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d3186640 1 -- 172.21.15.37:0/770917783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02cc140720 con 0x7f02cc15da40 2024-04-25T09:03:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d3186640 1 -- 172.21.15.37:0/770917783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02cc04efa0 con 0x7f02cc15da40 2024-04-25T09:03:58.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02c97fa640 1 -- 172.21.15.37:0/770917783 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f02c4018070 con 0x7f02cc15da40 2024-04-25T09:03:58.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02c97fa640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02ac066f60 0x7f02ac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02c97fa640 1 -- 172.21.15.37:0/770917783 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f02c4085790 con 0x7f02cc15da40 2024-04-25T09:03:58.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d0f22640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02ac066f60 0x7f02ac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.781+0000 7f02d0f22640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02ac066f60 0x7f02ac069420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0af000 tx=0x7f02bc0a8070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.785+0000 7f02c97fa640 1 -- 172.21.15.37:0/770917783 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f02c4056680 con 0x7f02cc15da40 2024-04-25T09:03:58.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.973+0000 7f02d3186640 1 -- 172.21.15.37:0/770917783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f02cc072670 con 0x7f02cc15da40 2024-04-25T09:03:58.963 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T09:03:58.973+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:03:58.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.977+0000 7f02c97fa640 1 -- 172.21.15.37:0/770917783 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v98) v1 ==== 161+0+0 (secure 0 0 0) 0x7f02c4055f80 con 0x7f02cc15da40 2024-04-25T09:03:58.966 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:03:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.977+0000 7f02d3186640 1 -- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02ac066f60 msgr2=0x7f02ac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.977+0000 7f02d3186640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02ac066f60 0x7f02ac069420 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f02bc0af000 tx=0x7f02bc0a8070 comp rx=0 tx=0).stop 2024-04-25T09:03:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.977+0000 7f02d3186640 1 -- 172.21.15.37:0/770917783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc15da40 msgr2=0x7f02cc140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.977+0000 7f02d3186640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc15da40 0x7f02cc140b00 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f02c400c7f0 tx=0x7f02c400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.981+0000 7f02d3186640 1 -- 172.21.15.37:0/770917783 shutdown_connections 2024-04-25T09:03:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.981+0000 7f02d3186640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f02ac066f60 0x7f02ac069420 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.981+0000 7f02d3186640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02cc15da40 0x7f02cc140b00 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.981+0000 7f02d3186640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02cc143790 0x7f02cc1590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.981+0000 7f02d3186640 1 --2- 172.21.15.37:0/770917783 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02cc06c3b0 0x7f02cc142b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.981+0000 7f02d3186640 1 -- 172.21.15.37:0/770917783 >> 172.21.15.37:0/770917783 conn(0x7f02cc071810 msgr2=0x7f02cc0cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.981+0000 7f02d3186640 1 -- 172.21.15.37:0/770917783 shutdown_connections 2024-04-25T09:03:58.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:58.981+0000 7f02d3186640 1 -- 172.21.15.37:0/770917783 wait complete. 2024-04-25T09:03:58.982 INFO:tasks.ceph.mds.a.smithi037.stderr: -7584> 2024-04-25T09:03:38.757+0000 7f57569d9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:03:58.986 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:03:58.988 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:03:59.010 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-25T09:03:59.010 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-04-25T09:03:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-04-25T09:03:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:03:59.013 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-04-25T09:03:59.013 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:03:59.013 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:03:59.013 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-04-25T09:03:59.013 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.013 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:03:59.013 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.013 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-04-25T09:03:59.013 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.013 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-04-25T09:03:59.014 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-04-25T09:03:59.015 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-04-25T09:03:59.016 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 20 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 20 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 10 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T09:03:59.017 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:03:59.018 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:03:59.019 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:03:59.020 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.020 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:03:59.020 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:03:59.020 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:03:59.020 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.020 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-04-25T09:03:59.020 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:03:59.020 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.020 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:03:59.020 INFO:teuthology.orchestra.run.smithi037.stdout: mon op complaint time = 120 2024-04-25T09:03:59.020 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout: client mount timeout = 600 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T09:03:59.021 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout:[mds] 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds = 20 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds balancer = 20 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug frag = True 2024-04-25T09:03:59.022 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug scatterstat = True 2024-04-25T09:03:59.023 INFO:teuthology.orchestra.run.smithi037.stdout: mds op complaint time = 180 2024-04-25T09:03:59.023 INFO:teuthology.orchestra.run.smithi037.stdout: mds verify scatter = False 2024-04-25T09:03:59.023 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T09:03:59.023 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T09:03:59.023 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T09:03:59.023 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2024-04-25T09:03:59.024 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:03:59.025 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.026 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2024-04-25T09:03:59.027 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.028 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2024-04-25T09:03:59.028 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2024-04-25T09:03:59.028 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2024-04-25T09:03:59.028 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T09:03:59.028 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T09:03:59.028 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.028 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2024-04-25T09:03:59.028 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2024-04-25T09:03:59.029 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.029 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2024-04-25T09:03:59.029 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2024-04-25T09:03:59.029 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 20 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 20 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 10 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T09:03:59.035 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = True 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = False 2024-04-25T09:03:59.036 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T09:03:59.037 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T09:03:59.037 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T09:03:59.038 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T09:03:59.039 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:03:59.086 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:03:59.109 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.110 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-04-25T09:03:59.111 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.112 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.113 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-04-25T09:03:59.114 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 20 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 20 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 10 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-04-25T09:03:59.115 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T09:03:59.116 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T09:03:59.116 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:03:59.116 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:03:59.116 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-04-25T09:03:59.116 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.116 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-04-25T09:03:59.116 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:03:59.116 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T09:03:59.116 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-04-25T09:03:59.116 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T09:03:59.116 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-04-25T09:03:59.117 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: mon op complaint time = 120 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: client mount timeout = 600 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-04-25T09:03:59.118 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout:[mds] 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds = 20 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds balancer = 20 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug frag = True 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug scatterstat = False 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout: mds op complaint time = 180 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout: mds verify scatter = False 2024-04-25T09:03:59.119 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T09:03:59.120 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T09:03:59.120 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T09:03:59.120 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.121 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:03:59.122 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2024-04-25T09:03:59.123 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2024-04-25T09:03:59.124 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 20 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 20 2024-04-25T09:03:59.125 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 10 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:03:59.126 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:03:59.127 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2024-04-25T09:03:59.128 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2024-04-25T09:03:59.129 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2024-04-25T09:03:59.130 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2024-04-25T09:03:59.130 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:03:59.130 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2024-04-25T09:03:59.130 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = False 2024-04-25T09:03:59.130 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2024-04-25T09:03:59.130 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = False 2024-04-25T09:03:59.130 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T09:03:59.130 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T09:03:59.130 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T09:03:59.132 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-04-25T09:03:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 Processor -- start 2024-04-25T09:03:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- start start 2024-04-25T09:03:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82600028d0 0x56025e76e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56025e84bbf0 0x56025e84afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56025e84b4f0 0x56025e83df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e7129f0 con 0x56025e84b4f0 2024-04-25T09:03:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e716950 con 0x7f82600028d0 2024-04-25T09:03:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56025e76aff0 con 0x56025e84bbf0 2024-04-25T09:03:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56025e84b4f0 0x56025e83df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56025e84b4f0 0x56025e83df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45762/0 (socket says 172.21.15.37:45762) 2024-04-25T09:03:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 -- 172.21.15.37:0/697781838 learned_addr learned my addr 172.21.15.37:0/697781838 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f8272bce640 1 --2- 172.21.15.37:0/697781838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82600028d0 0x56025e76e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82723cd640 1 --2- 172.21.15.37:0/697781838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56025e84bbf0 0x56025e84afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 -- 172.21.15.37:0/697781838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56025e84bbf0 msgr2=0x56025e84afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 --2- 172.21.15.37:0/697781838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56025e84bbf0 0x56025e84afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 -- 172.21.15.37:0/697781838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82600028d0 msgr2=0x56025e76e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 --2- 172.21.15.37:0/697781838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82600028d0 0x56025e76e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 -- 172.21.15.37:0/697781838 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56025e83a5a0 con 0x56025e84b4f0 2024-04-25T09:03:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 --2- 172.21.15.37:0/697781838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56025e84b4f0 0x56025e83df50 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x56025e796a80 tx=0x7f82640098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10b4dff9631a0dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82713cb640 1 -- 172.21.15.37:0/697781838 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264019070 con 0x56025e84b4f0 2024-04-25T09:03:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82713cb640 1 -- 172.21.15.37:0/697781838 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8264008900 con 0x56025e84b4f0 2024-04-25T09:03:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82713cb640 1 -- 172.21.15.37:0/697781838 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264008cf0 con 0x56025e84b4f0 2024-04-25T09:03:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- 172.21.15.37:0/697781838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56025e84b4f0 msgr2=0x56025e83df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 --2- 172.21.15.37:0/697781838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56025e84b4f0 0x56025e83df50 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x56025e796a80 tx=0x7f82640098b0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- 172.21.15.37:0/697781838 shutdown_connections 2024-04-25T09:03:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 --2- 172.21.15.37:0/697781838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56025e84b4f0 0x56025e83df50 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 --2- 172.21.15.37:0/697781838 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56025e84bbf0 0x56025e84afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 --2- 172.21.15.37:0/697781838 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f82600028d0 0x56025e76e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- 172.21.15.37:0/697781838 >> 172.21.15.37:0/697781838 conn(0x56025e796ab0 msgr2=0x56025e76ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- 172.21.15.37:0/697781838 shutdown_connections 2024-04-25T09:03:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- 172.21.15.37:0/697781838 wait complete. 2024-04-25T09:03:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 Processor -- start 2024-04-25T09:03:59.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- start start 2024-04-25T09:03:59.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56025e84b4f0 0x56025e83f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56025e84bbf0 0x56025e824cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82600028d0 0x56025e825210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e83a300 con 0x7f82600028d0 2024-04-25T09:03:59.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56025e796400 con 0x56025e84bbf0 2024-04-25T09:03:59.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82750b0080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56025e796660 con 0x56025e84b4f0 2024-04-25T09:03:59.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82600028d0 0x56025e825210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82600028d0 0x56025e825210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45768/0 (socket says 172.21.15.37:45768) 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.221+0000 7f82733cf640 1 -- 172.21.15.37:0/643606380 learned_addr learned my addr 172.21.15.37:0/643606380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82723cd640 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56025e84bbf0 0x56025e824cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f8272bce640 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56025e84b4f0 0x56025e83f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82733cf640 1 -- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56025e84b4f0 msgr2=0x56025e83f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82733cf640 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56025e84b4f0 0x56025e83f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82733cf640 1 -- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56025e84bbf0 msgr2=0x56025e824cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82733cf640 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56025e84bbf0 0x56025e824cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82733cf640 1 -- 172.21.15.37:0/643606380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56025e827980 con 0x7f82600028d0 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f8272bce640 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56025e84b4f0 0x56025e83f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82733cf640 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82600028d0 0x56025e825210 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f8264006010 tx=0x7f8264002fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f826bfff640 1 -- 172.21.15.37:0/643606380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264019040 con 0x7f82600028d0 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f826bfff640 1 -- 172.21.15.37:0/643606380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8264024070 con 0x7f82600028d0 2024-04-25T09:03:59.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82750b0080 1 -- 172.21.15.37:0/643606380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56025e828200 con 0x7f82600028d0 2024-04-25T09:03:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f826bfff640 1 -- 172.21.15.37:0/643606380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264014b60 con 0x7f82600028d0 2024-04-25T09:03:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82750b0080 1 -- 172.21.15.37:0/643606380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56025e83f650 con 0x7f82600028d0 2024-04-25T09:03:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f826bfff640 1 -- 172.21.15.37:0/643606380 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f826401c070 con 0x7f82600028d0 2024-04-25T09:03:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f826bfff640 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8254065970 0x7f8254067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f826bfff640 1 -- 172.21.15.37:0/643606380 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f82640853e0 con 0x7f82600028d0 2024-04-25T09:03:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82750b0080 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56025e8290f0 0x56025e829500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82750b0080 1 -- 172.21.15.37:0/643606380 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:1 17.7 17:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e126) v9 -- 0x56025e829a40 con 0x56025e8290f0 2024-04-25T09:03:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f8272bce640 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8254065970 0x7f8254067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82723cd640 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56025e8290f0 0x56025e829500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f82723cd640 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56025e8290f0 0x56025e829500 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.225+0000 7f8272bce640 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8254065970 0x7f8254067e30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f826c004610 tx=0x7f826c006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.229+0000 7f82723cd640 1 -- 172.21.15.37:0/643606380 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v126'5 uv5 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f825c0b3040 con 0x56025e8290f0 2024-04-25T09:03:59.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.229+0000 7f82750b0080 1 -- 172.21.15.37:0/643606380 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56025e8290f0 msgr2=0x56025e829500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.229+0000 7f82750b0080 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56025e8290f0 0x56025e829500 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.229+0000 7f82750b0080 1 -- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8254065970 msgr2=0x7f8254067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.229+0000 7f82750b0080 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8254065970 0x7f8254067e30 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f826c004610 tx=0x7f826c006040 comp rx=0 tx=0).stop 2024-04-25T09:03:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.229+0000 7f82750b0080 1 -- 172.21.15.37:0/643606380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82600028d0 msgr2=0x56025e825210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.229+0000 7f82750b0080 1 --2- 172.21.15.37:0/643606380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82600028d0 0x56025e825210 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f8264006010 tx=0x7f8264002fc0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.229+0000 7f82733cf640 1 -- 172.21.15.37:0/643606380 reap_dead start 2024-04-25T09:03:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.229+0000 7f82750b0080 1 -- 172.21.15.37:0/643606380 shutdown_connections 2024-04-25T09:03:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.229+0000 7f82750b0080 1 -- 172.21.15.37:0/643606380 >> 172.21.15.37:0/643606380 conn(0x56025e796ab0 msgr2=0x56025e84c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.233+0000 7f82750b0080 1 -- 172.21.15.37:0/643606380 shutdown_connections 2024-04-25T09:03:59.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.233+0000 7f82750b0080 1 -- 172.21.15.37:0/643606380 wait complete. 2024-04-25T09:03:59.223 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T09:03:59.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f01987d9640 1 Processor -- start 2024-04-25T09:03:59.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f01987d9640 1 -- start start 2024-04-25T09:03:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f01987d9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f019015dc10 0x7f0190071110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f01987d9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0190071960 con 0x7f019015d830 2024-04-25T09:03:59.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f01987d9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0190071c40 con 0x7f0190143870 2024-04-25T09:03:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f01987d9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0190071f20 con 0x7f019015dc10 2024-04-25T09:03:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f0196575640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f019015d830 0x7f019015aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48716/0 (socket says 172.21.15.37:48716) 2024-04-25T09:03:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f0196575640 1 -- 172.21.15.37:0/2185740776 learned_addr learned my addr 172.21.15.37:0/2185740776 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:03:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f0195d74640 1 --2- 172.21.15.37:0/2185740776 >> v2:172.21.15.203:3300/0 conn(0x7f019015dc10 0x7f0190071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f0195573640 1 -- 172.21.15.37:0/2185740776 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400832457 0 0) 0x7f0190071960 con 0x7f019015d830 2024-04-25T09:03:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f0195573640 1 -- 172.21.15.37:0/2185740776 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f017c003530 con 0x7f019015d830 2024-04-25T09:03:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f0195d74640 1 -- 172.21.15.37:0/2185740776 >> v1:172.21.15.203:6789/0 conn(0x7f0190143870 legacy=0x7f019006e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f0195d74640 1 -- 172.21.15.37:0/2185740776 >> v1:172.21.15.37:6789/0 conn(0x7f019015d830 legacy=0x7f019015aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.405+0000 7f0195d74640 1 -- 172.21.15.37:0/2185740776 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0190071650 con 0x7f019015dc10 2024-04-25T09:03:59.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f0195d74640 1 --2- 172.21.15.37:0/2185740776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f019015dc10 0x7f0190071110 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f01800097f0 tx=0x7f018001c380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa554132fd05c4a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f0195573640 1 -- 172.21.15.37:0/2185740776 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01800b0070 con 0x7f019015dc10 2024-04-25T09:03:59.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f0195573640 1 -- 172.21.15.37:0/2185740776 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01800ab600 con 0x7f019015dc10 2024-04-25T09:03:59.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f0195573640 1 -- 172.21.15.37:0/2185740776 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01800ab9b0 con 0x7f019015dc10 2024-04-25T09:03:59.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f01987d9640 1 -- 172.21.15.37:0/2185740776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f019015dc10 msgr2=0x7f0190071110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f01987d9640 1 --2- 172.21.15.37:0/2185740776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f019015dc10 0x7f0190071110 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f01800097f0 tx=0x7f018001c380 comp rx=0 tx=0).stop 2024-04-25T09:03:59.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f01987d9640 1 -- 172.21.15.37:0/2185740776 shutdown_connections 2024-04-25T09:03:59.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f01987d9640 1 --2- 172.21.15.37:0/2185740776 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f019015dc10 0x7f0190071110 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f01987d9640 1 -- 172.21.15.37:0/2185740776 >> 172.21.15.37:0/2185740776 conn(0x7f0190066fa0 msgr2=0x7f019006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f01987d9640 1 -- 172.21.15.37:0/2185740776 shutdown_connections 2024-04-25T09:03:59.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f01987d9640 1 -- 172.21.15.37:0/2185740776 wait complete. 2024-04-25T09:03:59.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f01987d9640 1 Processor -- start 2024-04-25T09:03:59.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.409+0000 7f01987d9640 1 -- start start 2024-04-25T09:03:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f01987d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0190143870 0x7f01901597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f01987d9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f019015d830 0x7f0190156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f01987d9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01900ceb80 0x7f0190155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f01987d9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01900b3cc0 con 0x7f0190143870 2024-04-25T09:03:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f01987d9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019006eb60 con 0x7f019015d830 2024-04-25T09:03:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f01987d9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01900b3e40 con 0x7f01900ceb80 2024-04-25T09:03:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0196575640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0190143870 0x7f01901597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0196575640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0190143870 0x7f01901597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43592/0 (socket says 172.21.15.37:43592) 2024-04-25T09:03:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0196575640 1 -- 172.21.15.37:0/4236219458 learned_addr learned my addr 172.21.15.37:0/4236219458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:03:59.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0195d74640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f019015d830 0x7f0190156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0196d76640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01900ceb80 0x7f0190155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0196575640 1 -- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01900ceb80 msgr2=0x7f0190155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0196575640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01900ceb80 0x7f0190155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0196575640 1 -- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f019015d830 msgr2=0x7f0190156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0196575640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f019015d830 0x7f0190156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0196575640 1 -- 172.21.15.37:0/4236219458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0190135d80 con 0x7f0190143870 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0196d76640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01900ceb80 0x7f0190155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0195d74640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f019015d830 0x7f0190156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0196575640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0190143870 0x7f01901597b0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f018c00c4d0 tx=0x7f018c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0186ffd640 1 -- 172.21.15.37:0/4236219458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c017070 con 0x7f0190143870 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0186ffd640 1 -- 172.21.15.37:0/4236219458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018c005cf0 con 0x7f0190143870 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f01987d9640 1 -- 172.21.15.37:0/4236219458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01900af280 con 0x7f0190143870 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0186ffd640 1 -- 172.21.15.37:0/4236219458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c007270 con 0x7f0190143870 2024-04-25T09:03:59.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f01987d9640 1 -- 172.21.15.37:0/4236219458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01900af840 con 0x7f0190143870 2024-04-25T09:03:59.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f01987d9640 1 -- 172.21.15.37:0/4236219458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0190135b10 con 0x7f0190143870 2024-04-25T09:03:59.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0186ffd640 1 -- 172.21.15.37:0/4236219458 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f018c01a070 con 0x7f0190143870 2024-04-25T09:03:59.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0186ffd640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0174066f60 0x7f0174069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0186ffd640 1 -- 172.21.15.37:0/4236219458 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f018c084570 con 0x7f0190143870 2024-04-25T09:03:59.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.413+0000 7f0195d74640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0174066f60 0x7f0174069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.417+0000 7f0195d74640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0174066f60 0x7f0174069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f01800ad000 tx=0x7f01800a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.417+0000 7f0186ffd640 1 -- 172.21.15.37:0/4236219458 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f018c054d40 con 0x7f0190143870 2024-04-25T09:03:59.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.617+0000 7f01987d9640 1 -- 172.21.15.37:0/4236219458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f0190067c40 con 0x7f0190143870 2024-04-25T09:03:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.981+0000 7f0186ffd640 1 -- 172.21.15.37:0/4236219458 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v99) v1 ==== 162+0+0 (secure 0 0 0) 0x7f018c058de0 con 0x7f0190143870 2024-04-25T09:03:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T09:03:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 -- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0174066f60 msgr2=0x7f0174069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0174066f60 0x7f0174069420 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f01800ad000 tx=0x7f01800a6000 comp rx=0 tx=0).stop 2024-04-25T09:03:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 -- 172.21.15.37:0/4236219458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0190143870 msgr2=0x7f01901597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0190143870 0x7f01901597b0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f018c00c4d0 tx=0x7f018c00c990 comp rx=0 tx=0).stop 2024-04-25T09:03:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 -- 172.21.15.37:0/4236219458 shutdown_connections 2024-04-25T09:03:59.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0174066f60 0x7f0174069420 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f01900ceb80 0x7f0190155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f019015d830 0x7f0190156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 --2- 172.21.15.37:0/4236219458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0190143870 0x7f01901597b0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 -- 172.21.15.37:0/4236219458 >> 172.21.15.37:0/4236219458 conn(0x7f0190066fa0 msgr2=0x7f019006d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 -- 172.21.15.37:0/4236219458 shutdown_connections 2024-04-25T09:03:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:03:59.985+0000 7f01987d9640 1 -- 172.21.15.37:0/4236219458 wait complete. 2024-04-25T09:04:00.000 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.137+0000 7f2b9279d640 1 Processor -- start 2024-04-25T09:04:00.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.137+0000 7f2b9279d640 1 -- start start 2024-04-25T09:04:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b9279d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f2b8c071e90 0x7f2b8c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b9279d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f2b8c06e260 0x7f2b8c06e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b9279d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8c0b3010 con 0x7f2b8c143f50 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b9279d640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8c15ef10 con 0x7f2b8c06e260 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b9279d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8c0b2dc0 con 0x7f2b8c071e90 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b8bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f2b8c143f50 0x7f2b8c15e800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48720/0 (socket says 172.21.15.37:48720) 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b8bfff640 1 -- 172.21.15.37:0/542860666 learned_addr learned my addr 172.21.15.37:0/542860666 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b90d3a640 1 --2- 172.21.15.37:0/542860666 >> v2:172.21.15.37:3300/0 conn(0x7f2b8c06e260 0x7f2b8c06e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b8affd640 1 -- 172.21.15.37:0/542860666 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832715108 0 0) 0x7f2b8c0b3010 con 0x7f2b8c143f50 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b8b7fe640 1 --2- 172.21.15.37:0/542860666 >> v2:172.21.15.203:3300/0 conn(0x7f2b8c071e90 0x7f2b8c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b8affd640 1 -- 172.21.15.37:0/542860666 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b74003530 con 0x7f2b8c143f50 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b90d3a640 1 -- 172.21.15.37:0/542860666 >> v2:172.21.15.203:3300/0 conn(0x7f2b8c071e90 msgr2=0x7f2b8c0b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b90d3a640 1 --2- 172.21.15.37:0/542860666 >> v2:172.21.15.203:3300/0 conn(0x7f2b8c071e90 0x7f2b8c0b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b90d3a640 1 -- 172.21.15.37:0/542860666 >> v1:172.21.15.37:6789/0 conn(0x7f2b8c143f50 legacy=0x7f2b8c15e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b90d3a640 1 -- 172.21.15.37:0/542860666 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c0b32f0 con 0x7f2b8c06e260 2024-04-25T09:04:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b90d3a640 1 --2- 172.21.15.37:0/542860666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8c06e260 0x7f2b8c06e710 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009d90 tx=0x7f2b8401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c98805c2dc68a5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.141+0000 7f2b8affd640 1 -- 172.21.15.37:0/542860666 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840b2070 con 0x7f2b8c06e260 2024-04-25T09:04:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8affd640 1 -- 172.21.15.37:0/542860666 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b840a57f0 con 0x7f2b8c06e260 2024-04-25T09:04:00.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8affd640 1 -- 172.21.15.37:0/542860666 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840a5bc0 con 0x7f2b8c06e260 2024-04-25T09:04:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 -- 172.21.15.37:0/542860666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8c06e260 msgr2=0x7f2b8c06e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 --2- 172.21.15.37:0/542860666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8c06e260 0x7f2b8c06e710 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009d90 tx=0x7f2b8401c930 comp rx=0 tx=0).stop 2024-04-25T09:04:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 -- 172.21.15.37:0/542860666 shutdown_connections 2024-04-25T09:04:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 --2- 172.21.15.37:0/542860666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8c06e260 0x7f2b8c06e710 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 --2- 172.21.15.37:0/542860666 >> v2:172.21.15.203:3300/0 conn(0x7f2b8c071e90 0x7f2b8c0b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 -- 172.21.15.37:0/542860666 >> 172.21.15.37:0/542860666 conn(0x7f2b8c066fa0 msgr2=0x7f2b8c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 -- 172.21.15.37:0/542860666 shutdown_connections 2024-04-25T09:04:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 -- 172.21.15.37:0/542860666 wait complete. 2024-04-25T09:04:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 Processor -- start 2024-04-25T09:04:00.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 -- start start 2024-04-25T09:04:00.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8c06e260 0x7f2b8c143130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b8c071e90 0x7f2b8c163650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b8c143f50 0x7f2b8c163b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c15b6a0 con 0x7f2b8c06e260 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c15b520 con 0x7f2b8c071e90 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c15b820 con 0x7f2b8c143f50 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8c06e260 0x7f2b8c143130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8c06e260 0x7f2b8c143130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43600/0 (socket says 172.21.15.37:43600) 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8bfff640 1 -- 172.21.15.37:0/1946282454 learned_addr learned my addr 172.21.15.37:0/1946282454 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8b7fe640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b8c071e90 0x7f2b8c163650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b90d3a640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b8c143f50 0x7f2b8c163b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8bfff640 1 -- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b8c143f50 msgr2=0x7f2b8c163b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8bfff640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b8c143f50 0x7f2b8c163b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8bfff640 1 -- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b8c071e90 msgr2=0x7f2b8c163650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8bfff640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b8c071e90 0x7f2b8c163650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8bfff640 1 -- 172.21.15.37:0/1946282454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c15f770 con 0x7f2b8c06e260 2024-04-25T09:04:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8b7fe640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b8c071e90 0x7f2b8c163650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b90d3a640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b8c143f50 0x7f2b8c163b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b8bfff640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8c06e260 0x7f2b8c143130 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c450 tx=0x7f2b7c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b88ff9640 1 -- 172.21.15.37:0/1946282454 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c017070 con 0x7f2b8c06e260 2024-04-25T09:04:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b88ff9640 1 -- 172.21.15.37:0/1946282454 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7c005cf0 con 0x7f2b8c06e260 2024-04-25T09:04:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.145+0000 7f2b9279d640 1 -- 172.21.15.37:0/1946282454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c15fa50 con 0x7f2b8c06e260 2024-04-25T09:04:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.149+0000 7f2b88ff9640 1 -- 172.21.15.37:0/1946282454 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7c007290 con 0x7f2b8c06e260 2024-04-25T09:04:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.149+0000 7f2b9279d640 1 -- 172.21.15.37:0/1946282454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c160010 con 0x7f2b8c06e260 2024-04-25T09:04:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.149+0000 7f2b9279d640 1 -- 172.21.15.37:0/1946282454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8c067c40 con 0x7f2b8c06e260 2024-04-25T09:04:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.149+0000 7f2b88ff9640 1 -- 172.21.15.37:0/1946282454 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f2b7c01a070 con 0x7f2b8c06e260 2024-04-25T09:04:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.149+0000 7f2b88ff9640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b78066f60 0x7f2b78069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.149+0000 7f2b88ff9640 1 -- 172.21.15.37:0/1946282454 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2b7c084530 con 0x7f2b8c06e260 2024-04-25T09:04:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.149+0000 7f2b8b7fe640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b78066f60 0x7f2b78069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.149+0000 7f2b8b7fe640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b78066f60 0x7f2b78069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0b1d30 tx=0x7f2b80008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.153+0000 7f2b88ff9640 1 -- 172.21.15.37:0/1946282454 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f2b7c0553f0 con 0x7f2b8c06e260 2024-04-25T09:04:00.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.345+0000 7f2b9279d640 1 -- 172.21.15.37:0/1946282454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b8c0b30d0 con 0x7f2b8c06e260 2024-04-25T09:04:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b88ff9640 1 -- 172.21.15.37:0/1946282454 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f2b7c054cf0 con 0x7f2b8c06e260 2024-04-25T09:04:00.337 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:00.337 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":100,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:03:59.988364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"a","rank":0,"incarnation":100,"state":"up:replay","state_seq":1,"addr":"172.21.15.37:6835/2159545482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2159545482},{"type":"v1","addr":"172.21.15.37:6835","nonce":2159545482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:04:00.337 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 100 2024-04-25T09:04:00.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 -- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b78066f60 msgr2=0x7f2b78069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b78066f60 0x7f2b78069420 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0b1d30 tx=0x7f2b80008040 comp rx=0 tx=0).stop 2024-04-25T09:04:00.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 -- 172.21.15.37:0/1946282454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8c06e260 msgr2=0x7f2b8c143130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8c06e260 0x7f2b8c143130 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f2b7c00c450 tx=0x7f2b7c00c910 comp rx=0 tx=0).stop 2024-04-25T09:04:00.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 -- 172.21.15.37:0/1946282454 shutdown_connections 2024-04-25T09:04:00.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2b78066f60 0x7f2b78069420 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f2b8c143f50 0x7f2b8c163b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f2b8c071e90 0x7f2b8c163650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 --2- 172.21.15.37:0/1946282454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8c06e260 0x7f2b8c143130 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 -- 172.21.15.37:0/1946282454 >> 172.21.15.37:0/1946282454 conn(0x7f2b8c066fa0 msgr2=0x7f2b8c15d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 -- 172.21.15.37:0/1946282454 shutdown_connections 2024-04-25T09:04:00.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:00.349+0000 7f2b9279d640 1 -- 172.21.15.37:0/1946282454 wait complete. 2024-04-25T09:04:00.758 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:03:32.494159+0000', 'modified': '2024-04-25T09:03:59.988364+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9762}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9762': {'gid': 9762, 'name': 'a', 'rank': 0, 'incarnation': 100, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.37:6835/2159545482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2159545482}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2159545482}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:00.759 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9762:up:replay 2024-04-25T09:04:01.760 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:01.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd87910e640 1 Processor -- start 2024-04-25T09:04:01.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd87910e640 1 -- start start 2024-04-25T09:04:01.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd87910e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd874071590 0x7fd87415c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd87910e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87415f240 con 0x7fd87415edc0 2024-04-25T09:04:01.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd87910e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87415f520 con 0x7fd874071130 2024-04-25T09:04:01.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd87910e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87415f800 con 0x7fd874071590 2024-04-25T09:04:01.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd87359e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd874071590 0x7fd87415c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd872d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd87415edc0 0x7fd874144910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48724/0 (socket says 172.21.15.37:48724) 2024-04-25T09:04:01.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd872d9d640 1 -- 172.21.15.37:0/4059823721 learned_addr learned my addr 172.21.15.37:0/4059823721 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:01.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd87359e640 1 -- 172.21.15.37:0/4059823721 >> v1:172.21.15.203:6789/0 conn(0x7fd874071130 legacy=0x7fd87415bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd87359e640 1 -- 172.21.15.37:0/4059823721 >> v1:172.21.15.37:6789/0 conn(0x7fd87415edc0 legacy=0x7fd874144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.909+0000 7fd87359e640 1 -- 172.21.15.37:0/4059823721 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87415fae0 con 0x7fd874071590 2024-04-25T09:04:01.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87359e640 1 --2- 172.21.15.37:0/4059823721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd874071590 0x7fd87415c610 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009d90 tx=0x7fd86c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85fb3f07cbe363a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd871d9b640 1 -- 172.21.15.37:0/4059823721 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0b2070 con 0x7fd874071590 2024-04-25T09:04:01.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd871d9b640 1 -- 172.21.15.37:0/4059823721 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86c0a58e0 con 0x7fd874071590 2024-04-25T09:04:01.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd871d9b640 1 -- 172.21.15.37:0/4059823721 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86c0a5cb0 con 0x7fd874071590 2024-04-25T09:04:01.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 -- 172.21.15.37:0/4059823721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd874071590 msgr2=0x7fd87415c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 --2- 172.21.15.37:0/4059823721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd874071590 0x7fd87415c610 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7fd86c009d90 tx=0x7fd86c01c930 comp rx=0 tx=0).stop 2024-04-25T09:04:01.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 -- 172.21.15.37:0/4059823721 shutdown_connections 2024-04-25T09:04:01.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 --2- 172.21.15.37:0/4059823721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd874071590 0x7fd87415c610 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 -- 172.21.15.37:0/4059823721 >> 172.21.15.37:0/4059823721 conn(0x7fd874066fa0 msgr2=0x7fd87406ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:01.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 -- 172.21.15.37:0/4059823721 shutdown_connections 2024-04-25T09:04:01.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 -- 172.21.15.37:0/4059823721 wait complete. 2024-04-25T09:04:01.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 Processor -- start 2024-04-25T09:04:01.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 -- start start 2024-04-25T09:04:01.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd874071130 0x7fd87415b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd874071590 0x7fd8740b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd87415edc0 0x7fd8740b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8741451a0 con 0x7fd874071590 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd874145020 con 0x7fd87415edc0 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87910e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8741695f0 con 0x7fd874071130 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd874071590 0x7fd8740b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd874071590 0x7fd8740b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43606/0 (socket says 172.21.15.37:43606) 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87259c640 1 -- 172.21.15.37:0/3209259100 learned_addr learned my addr 172.21.15.37:0/3209259100 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87359e640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd87415edc0 0x7fd8740b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd872d9d640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd874071130 0x7fd87415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87259c640 1 -- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd874071130 msgr2=0x7fd87415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87259c640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd874071130 0x7fd87415b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87259c640 1 -- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd87415edc0 msgr2=0x7fd8740b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87259c640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd87415edc0 0x7fd8740b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87259c640 1 -- 172.21.15.37:0/3209259100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8740b0990 con 0x7fd874071590 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87359e640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd87415edc0 0x7fd8740b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.913+0000 7fd87259c640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd874071590 0x7fd8740b1a30 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fd86400c480 tx=0x7fd86400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.917+0000 7fd86b7fe640 1 -- 172.21.15.37:0/3209259100 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864012830 con 0x7fd874071590 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.917+0000 7fd86b7fe640 1 -- 172.21.15.37:0/3209259100 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd864017070 con 0x7fd874071590 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.917+0000 7fd87910e640 1 -- 172.21.15.37:0/3209259100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8740b0c70 con 0x7fd874071590 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.917+0000 7fd86b7fe640 1 -- 172.21.15.37:0/3209259100 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864012c30 con 0x7fd874071590 2024-04-25T09:04:01.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.917+0000 7fd87910e640 1 -- 172.21.15.37:0/3209259100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8740b1230 con 0x7fd874071590 2024-04-25T09:04:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.917+0000 7fd87910e640 1 -- 172.21.15.37:0/3209259100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd874067c40 con 0x7fd874071590 2024-04-25T09:04:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.917+0000 7fd86b7fe640 1 -- 172.21.15.37:0/3209259100 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd86401c460 con 0x7fd874071590 2024-04-25T09:04:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.917+0000 7fd86b7fe640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd854066f60 0x7fd854069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.917+0000 7fd86b7fe640 1 -- 172.21.15.37:0/3209259100 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd8640850d0 con 0x7fd874071590 2024-04-25T09:04:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.917+0000 7fd872d9d640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd854066f60 0x7fd854069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.917+0000 7fd872d9d640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd854066f60 0x7fd854069420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd85c000f70 tx=0x7fd85c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:01.921+0000 7fd86b7fe640 1 -- 172.21.15.37:0/3209259100 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd864055ef0 con 0x7fd874071590 2024-04-25T09:04:02.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.117+0000 7fd87910e640 1 -- 172.21.15.37:0/3209259100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd874072220 con 0x7fd874071590 2024-04-25T09:04:02.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.117+0000 7fd86b7fe640 1 -- 172.21.15.37:0/3209259100 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+4951 (secure 0 0 0) 0x7fd8640557f0 con 0x7fd874071590 2024-04-25T09:04:02.107 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:02.108 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":102,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:04:00.997785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"a","rank":0,"incarnation":100,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.37:6835/2159545482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2159545482},{"type":"v1","addr":"172.21.15.37:6835","nonce":2159545482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:04:02.108 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 102 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 -- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd854066f60 msgr2=0x7fd854069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd854066f60 0x7fd854069420 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd85c000f70 tx=0x7fd85c009040 comp rx=0 tx=0).stop 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 -- 172.21.15.37:0/3209259100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd874071590 msgr2=0x7fd8740b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd874071590 0x7fd8740b1a30 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7fd86400c480 tx=0x7fd86400c940 comp rx=0 tx=0).stop 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 -- 172.21.15.37:0/3209259100 shutdown_connections 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd854066f60 0x7fd854069420 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd87415edc0 0x7fd8740b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd874071590 0x7fd8740b1a30 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 --2- 172.21.15.37:0/3209259100 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd874071130 0x7fd87415b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 -- 172.21.15.37:0/3209259100 >> 172.21.15.37:0/3209259100 conn(0x7fd874066fa0 msgr2=0x7fd874142220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 -- 172.21.15.37:0/3209259100 shutdown_connections 2024-04-25T09:04:02.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:02.121+0000 7fd87910e640 1 -- 172.21.15.37:0/3209259100 wait complete. 2024-04-25T09:04:02.529 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:03:32.494159+0000', 'modified': '2024-04-25T09:04:00.997785+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9762}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9762': {'gid': 9762, 'name': 'a', 'rank': 0, 'incarnation': 100, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.37:6835/2159545482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2159545482}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2159545482}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:02.529 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9762:up:rejoin 2024-04-25T09:04:03.530 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:03.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096f41d640 1 Processor -- start 2024-04-25T09:04:03.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096f41d640 1 -- start start 2024-04-25T09:04:03.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096f41d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f096815edc0 0x7f0968144910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096f41d640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096815f240 con 0x7f0968071840 2024-04-25T09:04:03.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096f41d640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096815f520 con 0x7f0968144e50 2024-04-25T09:04:03.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096f41d640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096815f800 con 0x7f096815edc0 2024-04-25T09:04:03.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096d9ba640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0968071840 0x7f096815bf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48736/0 (socket says 172.21.15.37:48736) 2024-04-25T09:04:03.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096d9ba640 1 -- 172.21.15.37:0/3448653974 learned_addr learned my addr 172.21.15.37:0/3448653974 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:03.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096d1b9640 1 --2- 172.21.15.37:0/3448653974 >> v2:172.21.15.203:3300/0 conn(0x7f096815edc0 0x7f0968144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436817515 0 0) 0x7f096815f240 con 0x7f0968071840 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0950003530 con 0x7f0968071840 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1131841594 0 0) 0x7f0950003530 con 0x7f0968071840 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 >> v2:172.21.15.203:3300/0 conn(0x7f096815edc0 msgr2=0x7f0968144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 --2- 172.21.15.37:0/3448653974 >> v2:172.21.15.203:3300/0 conn(0x7f096815edc0 0x7f0968144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 >> v1:172.21.15.203:6789/0 conn(0x7f0968144e50 legacy=0x7f0968071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096815fae0 con 0x7f0968071840 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0960001680 con 0x7f0968071840 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 >> v1:172.21.15.37:6789/0 conn(0x7f0968071840 legacy=0x7f096815bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 --2- 172.21.15.37:0/3448653974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500039c0 0x7f0950003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 --2- 172.21.15.37:0/3448653974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0950004810 0x7f0950004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 --2- 172.21.15.37:0/3448653974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09500051d0 0x7f095000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096815f800 con 0x7f0950004810 2024-04-25T09:04:03.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095000c070 con 0x7f09500051d0 2024-04-25T09:04:03.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095000c270 con 0x7f09500039c0 2024-04-25T09:04:03.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096c9b8640 1 --2- 172.21.15.37:0/3448653974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0950004810 0x7f0950004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096d1b9640 1 --2- 172.21.15.37:0/3448653974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500039c0 0x7f0950003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096d9ba640 1 --2- 172.21.15.37:0/3448653974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09500051d0 0x7f095000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096c9b8640 1 -- 172.21.15.37:0/3448653974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500039c0 msgr2=0x7f0950003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:03.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096c9b8640 1 --2- 172.21.15.37:0/3448653974 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f09500039c0 0x7f0950003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096c9b8640 1 -- 172.21.15.37:0/3448653974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09500051d0 msgr2=0x7f095000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:03.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096c9b8640 1 --2- 172.21.15.37:0/3448653974 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f09500051d0 0x7f095000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096d9ba640 1 -- 172.21.15.37:0/3448653974 reap_dead start 2024-04-25T09:04:03.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.677+0000 7f096c9b8640 1 -- 172.21.15.37:0/3448653974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950003530 con 0x7f0950004810 2024-04-25T09:04:03.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096c9b8640 1 --2- 172.21.15.37:0/3448653974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0950004810 0x7f0950004c90 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f095c001780 tx=0x7f095c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8397fa52cf6f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c0013a0 con 0x7f0950004810 2024-04-25T09:04:03.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095c0052f0 con 0x7f0950004810 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f0967fff640 1 -- 172.21.15.37:0/3448653974 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095c003840 con 0x7f0950004810 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 -- 172.21.15.37:0/3448653974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0950004810 msgr2=0x7f0950004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 --2- 172.21.15.37:0/3448653974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0950004810 0x7f0950004c90 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f095c001780 tx=0x7f095c01c400 comp rx=0 tx=0).stop 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 -- 172.21.15.37:0/3448653974 shutdown_connections 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 --2- 172.21.15.37:0/3448653974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0950004810 0x7f0950004c90 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 -- 172.21.15.37:0/3448653974 >> 172.21.15.37:0/3448653974 conn(0x7f0968066fa0 msgr2=0x7f096806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 -- 172.21.15.37:0/3448653974 shutdown_connections 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 -- 172.21.15.37:0/3448653974 wait complete. 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 Processor -- start 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 -- start start 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0950004810 0x7f09680ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f096815b120 0x7f09680add20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09680b33d0 0x7f09680b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0968144910 con 0x7f09680b33d0 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09681600a0 con 0x7f0950004810 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.681+0000 7f096f41d640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0968144a90 con 0x7f096815b120 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096d9ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09680b33d0 0x7f09680b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096d9ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09680b33d0 0x7f09680b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43632/0 (socket says 172.21.15.37:43632) 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096d9ba640 1 -- 172.21.15.37:0/3976695613 learned_addr learned my addr 172.21.15.37:0/3976695613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096d1b9640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0950004810 0x7f09680ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096c9b8640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f096815b120 0x7f09680add20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096d9ba640 1 -- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f096815b120 msgr2=0x7f09680add20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096d9ba640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f096815b120 0x7f09680add20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096d9ba640 1 -- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0950004810 msgr2=0x7f09680ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096d9ba640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0950004810 0x7f09680ad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096d9ba640 1 -- 172.21.15.37:0/3976695613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0968139c50 con 0x7f09680b33d0 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096d9ba640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09680b33d0 0x7f09680b3830 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f0968071d10 tx=0x7f09600119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f0965ffb640 1 -- 172.21.15.37:0/3976695613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960010040 con 0x7f09680b33d0 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f0965ffb640 1 -- 172.21.15.37:0/3976695613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096001a070 con 0x7f09680b33d0 2024-04-25T09:04:03.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096f41d640 1 -- 172.21.15.37:0/3976695613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0968139f30 con 0x7f09680b33d0 2024-04-25T09:04:03.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f0965ffb640 1 -- 172.21.15.37:0/3976695613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960013a20 con 0x7f09680b33d0 2024-04-25T09:04:03.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096f41d640 1 -- 172.21.15.37:0/3976695613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096813a4f0 con 0x7f09680b33d0 2024-04-25T09:04:03.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096f41d640 1 -- 172.21.15.37:0/3976695613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0968067c40 con 0x7f09680b33d0 2024-04-25T09:04:03.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f0965ffb640 1 -- 172.21.15.37:0/3976695613 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0960013bc0 con 0x7f09680b33d0 2024-04-25T09:04:03.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f0965ffb640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0954066f60 0x7f0954069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f0965ffb640 1 -- 172.21.15.37:0/3976695613 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f0960087f50 con 0x7f09680b33d0 2024-04-25T09:04:03.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.685+0000 7f096d1b9640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0954066f60 0x7f0954069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.689+0000 7f096d1b9640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0954066f60 0x7f0954069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0968143c00 tx=0x7f095800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.689+0000 7f0965ffb640 1 -- 172.21.15.37:0/3976695613 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0960058d90 con 0x7f09680b33d0 2024-04-25T09:04:03.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.885+0000 7f096f41d640 1 -- 172.21.15.37:0/3976695613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09680711d0 con 0x7f09680b33d0 2024-04-25T09:04:03.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.885+0000 7f0965ffb640 1 -- 172.21.15.37:0/3976695613 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f0960058690 con 0x7f09680b33d0 2024-04-25T09:04:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:03.874 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:04:02.008068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/2159545482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2159545482},{"type":"v1","addr":"172.21.15.37:6835","nonce":2159545482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:04:03.875 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 103 2024-04-25T09:04:03.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.885+0000 7f096f41d640 1 -- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0954066f60 msgr2=0x7f0954069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.885+0000 7f096f41d640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0954066f60 0x7f0954069420 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f0968143c00 tx=0x7f095800b040 comp rx=0 tx=0).stop 2024-04-25T09:04:03.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.885+0000 7f096f41d640 1 -- 172.21.15.37:0/3976695613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09680b33d0 msgr2=0x7f09680b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.885+0000 7f096f41d640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09680b33d0 0x7f09680b3830 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f0968071d10 tx=0x7f09600119f0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.889+0000 7f096f41d640 1 -- 172.21.15.37:0/3976695613 shutdown_connections 2024-04-25T09:04:03.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.889+0000 7f096f41d640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0954066f60 0x7f0954069420 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.889+0000 7f096f41d640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09680b33d0 0x7f09680b3830 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.889+0000 7f096f41d640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f096815b120 0x7f09680add20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.889+0000 7f096f41d640 1 --2- 172.21.15.37:0/3976695613 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0950004810 0x7f09680ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.889+0000 7f096f41d640 1 -- 172.21.15.37:0/3976695613 >> 172.21.15.37:0/3976695613 conn(0x7f0968066fa0 msgr2=0x7f09680af030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:03.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.889+0000 7f096f41d640 1 -- 172.21.15.37:0/3976695613 shutdown_connections 2024-04-25T09:04:03.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:03.889+0000 7f096f41d640 1 -- 172.21.15.37:0/3976695613 wait complete. 2024-04-25T09:04:04.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 103, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:03:32.494159+0000', 'modified': '2024-04-25T09:04:02.008068+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 126, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9762}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9762': {'gid': 9762, 'name': 'a', 'rank': 0, 'incarnation': 100, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.37:6835/2159545482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2159545482}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2159545482}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:04.297 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:04:04.297 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:04:04.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.445+0000 7f297d457640 1 Processor -- start 2024-04-25T09:04:04.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.445+0000 7f297d457640 1 -- start start 2024-04-25T09:04:04.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f297d457640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f29780ceeb0 0x7f297806cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f297d457640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 0x7f297815e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f297d457640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f297815f050 con 0x7f29780d32d0 2024-04-25T09:04:04.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f297d457640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29780b6ae0 con 0x7f29780ceeb0 2024-04-25T09:04:04.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f297d457640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297806c1c0 con 0x7f297815e2e0 2024-04-25T09:04:04.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f29767fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f29780ceeb0 0x7f297806cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f29777fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 0x7f297815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f2976ffd640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f29780d32d0 0x7f29780ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:46346/0 (socket says 172.21.15.37:46346) 2024-04-25T09:04:04.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f29767fc640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f29780ceeb0 0x7f297806cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:50418/0 (socket says 172.21.15.37:50418) 2024-04-25T09:04:04.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f2976ffd640 1 -- 172.21.15.37:0/1666021665 learned_addr learned my addr 172.21.15.37:0/1666021665 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:04.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f29767fc640 1 -- 172.21.15.37:0/1666021665 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 msgr2=0x7f297815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f29767fc640 1 --2- 172.21.15.37:0/1666021665 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 0x7f297815e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f29767fc640 1 -- 172.21.15.37:0/1666021665 >> v1:172.21.15.203:6789/0 conn(0x7f29780d32d0 legacy=0x7f29780ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f29767fc640 1 -- 172.21.15.37:0/1666021665 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f297806c410 con 0x7f29780ceeb0 2024-04-25T09:04:04.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.449+0000 7f29777fe640 1 --2- 172.21.15.37:0/1666021665 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 0x7f297815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:04.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f29767fc640 1 --2- 172.21.15.37:0/1666021665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29780ceeb0 0x7f297806cef0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f296c0097a0 tx=0x7f296c01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b2c672973caaba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f2975ffb640 1 -- 172.21.15.37:0/1666021665 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0b5070 con 0x7f29780ceeb0 2024-04-25T09:04:04.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f2975ffb640 1 -- 172.21.15.37:0/1666021665 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296c0a8970 con 0x7f29780ceeb0 2024-04-25T09:04:04.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f2975ffb640 1 -- 172.21.15.37:0/1666021665 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0a8d40 con 0x7f29780ceeb0 2024-04-25T09:04:04.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f297d457640 1 -- 172.21.15.37:0/1666021665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29780ceeb0 msgr2=0x7f297806cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f297d457640 1 --2- 172.21.15.37:0/1666021665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29780ceeb0 0x7f297806cef0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f296c0097a0 tx=0x7f296c01ec70 comp rx=0 tx=0).stop 2024-04-25T09:04:04.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f297d457640 1 -- 172.21.15.37:0/1666021665 shutdown_connections 2024-04-25T09:04:04.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f297d457640 1 --2- 172.21.15.37:0/1666021665 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 0x7f297815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f297d457640 1 --2- 172.21.15.37:0/1666021665 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29780ceeb0 0x7f297806cef0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f297d457640 1 -- 172.21.15.37:0/1666021665 >> 172.21.15.37:0/1666021665 conn(0x7f29780d39f0 msgr2=0x7f29780d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f297d457640 1 -- 172.21.15.37:0/1666021665 shutdown_connections 2024-04-25T09:04:04.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.453+0000 7f297d457640 1 -- 172.21.15.37:0/1666021665 wait complete. 2024-04-25T09:04:04.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f297d457640 1 Processor -- start 2024-04-25T09:04:04.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f297d457640 1 -- start start 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f297d457640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29780ceeb0 0x7f29780cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f297d457640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29780d32d0 0x7f29781f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f297d457640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 0x7f2978146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f297d457640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29780d36b0 con 0x7f29780ceeb0 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f297d457640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297806c9d0 con 0x7f29780d32d0 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f297d457640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f297815ecb0 con 0x7f297815e2e0 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29780ceeb0 0x7f29780cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29780ceeb0 0x7f29780cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43636/0 (socket says 172.21.15.37:43636) 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f2976ffd640 1 -- 172.21.15.37:0/2861673888 learned_addr learned my addr 172.21.15.37:0/2861673888 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f29767fc640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29780d32d0 0x7f29781f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f29777fe640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 0x7f2978146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f2976ffd640 1 -- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 msgr2=0x7f2978146b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f2976ffd640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 0x7f2978146b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f2976ffd640 1 -- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29780d32d0 msgr2=0x7f29781f32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f2976ffd640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29780d32d0 0x7f29781f32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f2976ffd640 1 -- 172.21.15.37:0/2861673888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29780cdc90 con 0x7f29780ceeb0 2024-04-25T09:04:04.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f29777fe640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 0x7f2978146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:04.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f29767fc640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29780d32d0 0x7f29781f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:04.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f2976ffd640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29780ceeb0 0x7f29780cc9e0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f296800ca70 tx=0x7f296800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f295f7fe640 1 -- 172.21.15.37:0/2861673888 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968017070 con 0x7f29780ceeb0 2024-04-25T09:04:04.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f295f7fe640 1 -- 172.21.15.37:0/2861673888 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296800ab70 con 0x7f29780ceeb0 2024-04-25T09:04:04.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f297d457640 1 -- 172.21.15.37:0/2861673888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29781f3800 con 0x7f29780ceeb0 2024-04-25T09:04:04.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f295f7fe640 1 -- 172.21.15.37:0/2861673888 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968012400 con 0x7f29780ceeb0 2024-04-25T09:04:04.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.457+0000 7f297d457640 1 -- 172.21.15.37:0/2861673888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29780cd080 con 0x7f29780ceeb0 2024-04-25T09:04:04.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f295f7fe640 1 -- 172.21.15.37:0/2861673888 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f296801a070 con 0x7f29780ceeb0 2024-04-25T09:04:04.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f295f7fe640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2964066f60 0x7f2964069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f295f7fe640 1 -- 172.21.15.37:0/2861673888 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2968084e00 con 0x7f29780ceeb0 2024-04-25T09:04:04.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29767fc640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2964066f60 0x7f2964069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 Processor -- start 2024-04-25T09:04:04.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 -- start start 2024-04-25T09:04:04.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29767fc640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2964066f60 0x7f2964069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b2000 tx=0x7f296c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e5c920 0x55a723e5cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5d240 0x55a723e129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723e13310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a723cfa5f0 con 0x55a723e12f30 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a723c61da0 con 0x55a723e5c920 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a723bea150 con 0x55a723e5d240 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723e13310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723e13310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43648/0 (socket says 172.21.15.37:43648) 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29777fe640 1 -- 172.21.15.37:0/1331608309 learned_addr learned my addr 172.21.15.37:0/1331608309 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29767fc640 1 --2- 172.21.15.37:0/1331608309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5d240 0x55a723e129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f2976ffd640 1 --2- 172.21.15.37:0/1331608309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e5c920 0x55a723e5cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29777fe640 1 -- 172.21.15.37:0/1331608309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5d240 msgr2=0x55a723e129f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29777fe640 1 --2- 172.21.15.37:0/1331608309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5d240 0x55a723e129f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29777fe640 1 -- 172.21.15.37:0/1331608309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e5c920 msgr2=0x55a723e5cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29777fe640 1 --2- 172.21.15.37:0/1331608309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e5c920 0x55a723e5cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29777fe640 1 -- 172.21.15.37:0/1331608309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a723c03ca0 con 0x55a723e12f30 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29777fe640 1 --2- 172.21.15.37:0/1331608309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723e13310 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f2970009be0 tx=0x7f2970009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cb53cf86ac1a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f295cff9640 1 -- 172.21.15.37:0/1331608309 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970019070 con 0x55a723e12f30 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f295cff9640 1 -- 172.21.15.37:0/1331608309 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2970011cd0 con 0x55a723e12f30 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f295cff9640 1 -- 172.21.15.37:0/1331608309 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970015660 con 0x55a723e12f30 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 -- 172.21.15.37:0/1331608309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 msgr2=0x55a723e13310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 --2- 172.21.15.37:0/1331608309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723e13310 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f2970009be0 tx=0x7f2970009190 comp rx=0 tx=0).stop 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 -- 172.21.15.37:0/1331608309 shutdown_connections 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 --2- 172.21.15.37:0/1331608309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723e13310 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 --2- 172.21.15.37:0/1331608309 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5d240 0x55a723e129f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 --2- 172.21.15.37:0/1331608309 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e5c920 0x55a723e5cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 -- 172.21.15.37:0/1331608309 >> 172.21.15.37:0/1331608309 conn(0x55a723e192a0 msgr2=0x55a723e112d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 -- 172.21.15.37:0/1331608309 shutdown_connections 2024-04-25T09:04:04.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.461+0000 7f29805c4000 1 -- 172.21.15.37:0/1331608309 wait complete. 2024-04-25T09:04:04.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 -- start start 2024-04-25T09:04:04.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 Processor -- start 2024-04-25T09:04:04.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e12f30 0x55a723e63c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5c920 0x55a723e641c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5d240 0x55a723e16aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a723c1c990 con 0x55a723e5c920 2024-04-25T09:04:04.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a723c5de40 con 0x55a723e12f30 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a723bed900 con 0x55a723e5d240 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 10 client.0 _ll_register_callbacks cb 0x55a723d151c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29767fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5c920 0x55a723e641c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29767fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5c920 0x55a723e641c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43662/0 (socket says 172.21.15.37:43662) 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29767fc640 1 -- 172.21.15.37:0/1942220220 learned_addr learned my addr 172.21.15.37:0/1942220220 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29777fe640 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5d240 0x55a723e16aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f2976ffd640 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e12f30 0x55a723e63c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29767fc640 1 -- 172.21.15.37:0/1942220220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5d240 msgr2=0x55a723e16aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29767fc640 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5d240 0x55a723e16aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29767fc640 1 -- 172.21.15.37:0/1942220220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e12f30 msgr2=0x55a723e63c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29767fc640 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e12f30 0x55a723e63c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29767fc640 1 -- 172.21.15.37:0/1942220220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a723c1a630 con 0x55a723e5c920 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29777fe640 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5d240 0x55a723e16aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:04.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29767fc640 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5c920 0x55a723e641c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b3040 tx=0x7f296c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd4b5608450936f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f2955ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f2955ffb640 1 -- 172.21.15.37:0/1942220220 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296c0b0a50 con 0x55a723e5c920 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f2955ffb640 1 -- 172.21.15.37:0/1942220220 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f296c0b0bf0 con 0x55a723e5c920 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 -- 172.21.15.37:0/1942220220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2970011cd0 con 0x55a723e5c920 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f2955ffb640 1 -- 172.21.15.37:0/1942220220 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f296c0d3050 con 0x55a723e5c920 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 10 client.9810 fetch_fsmap learned FSMap version 103 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 -- 172.21.15.37:0/1942220220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x55a723bea3a0 con 0x55a723e5c920 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f2955ffb640 1 -- 172.21.15.37:0/1942220220 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2970011cd0 con 0x55a723e5c920 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f2955ffb640 1 -- 172.21.15.37:0/1942220220 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 103) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f296c0d6020 con 0x55a723e5c920 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 10 client.9810 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 10 client.9810 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 20 client.9810 populate_metadata read hostname 'smithi037' 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55a723e18160 0x55a723e6dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 4 client.9810 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:04:04.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f29805c4000 1 -- 172.21.15.37:0/1942220220 --> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a723bfda50 con 0x55a723e18160 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f2976ffd640 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55a723e18160 0x55a723e6dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f2955ffb640 10 client.9810 ms_handle_connect on v2:172.21.15.37:6834/2159545482 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.465+0000 7f2976ffd640 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55a723e18160 0x55a723e6dc70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=776eea0247d0aefa server_cookie=3c6647cfcc50f361 in_seq=0 out_seq=0 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f2955ffb640 1 -- 172.21.15.37:0/1942220220 <== mds.0 v2:172.21.15.37:6834/2159545482 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a723bfda50 con 0x55a723e18160 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f2955ffb640 10 client.9810 handle_command_reply: tid=0 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 1 client.9810 shutdown 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 1 -- 172.21.15.37:0/1942220220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5c920 msgr2=0x55a723e641c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5c920 0x55a723e641c0 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b3040 tx=0x7f296c0046d0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 1 -- 172.21.15.37:0/1942220220 shutdown_connections 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55a723e18160 0x55a723e6dc70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5d240 0x55a723e16aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5c920 0x55a723e641c0 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f2955ffb640 0 client.9810 ms_handle_reset on v2:172.21.15.37:6834/2159545482 2024-04-25T09:04:04.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 1 --2- 172.21.15.37:0/1942220220 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e12f30 0x55a723e63c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 1 -- 172.21.15.37:0/1942220220 >> 172.21.15.37:0/1942220220 conn(0x55a723e192a0 msgr2=0x55a723e119e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 1 -- 172.21.15.37:0/1942220220 shutdown_connections 2024-04-25T09:04:04.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 1 -- 172.21.15.37:0/1942220220 wait complete. 2024-04-25T09:04:04.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.469+0000 7f29805c4000 20 client.9810 trim_cache size 0 max 16384 2024-04-25T09:04:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 Processor -- start 2024-04-25T09:04:04.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 -- start start 2024-04-25T09:04:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e12f30 0x55a723eb8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5c920 0x55a723eb8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5d240 0x55a723eb8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781ff690 con 0x55a723e5d240 2024-04-25T09:04:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2978145180 con 0x55a723e12f30 2024-04-25T09:04:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29781ff160 con 0x55a723e5c920 2024-04-25T09:04:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5d240 0x55a723eb8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5d240 0x55a723eb8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43668/0 (socket says 172.21.15.37:43668) 2024-04-25T09:04:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29777fe640 1 -- 172.21.15.37:0/3426519922 learned_addr learned my addr 172.21.15.37:0/3426519922 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f2976ffd640 1 --2- 172.21.15.37:0/3426519922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e12f30 0x55a723eb8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29767fc640 1 --2- 172.21.15.37:0/3426519922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5c920 0x55a723eb8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29777fe640 1 -- 172.21.15.37:0/3426519922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5c920 msgr2=0x55a723eb8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29777fe640 1 --2- 172.21.15.37:0/3426519922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5c920 0x55a723eb8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29777fe640 1 -- 172.21.15.37:0/3426519922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e12f30 msgr2=0x55a723eb8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29777fe640 1 --2- 172.21.15.37:0/3426519922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e12f30 0x55a723eb8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29777fe640 1 -- 172.21.15.37:0/3426519922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a723e5d070 con 0x55a723e5d240 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29777fe640 1 --2- 172.21.15.37:0/3426519922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5d240 0x55a723eb8db0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f2970018000 tx=0x7f2970016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4065aeeda5e961ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f2957fff640 1 -- 172.21.15.37:0/3426519922 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297002e070 con 0x55a723e5d240 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f2957fff640 1 -- 172.21.15.37:0/3426519922 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2970009df0 con 0x55a723e5d240 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f2957fff640 1 -- 172.21.15.37:0/3426519922 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2970019040 con 0x55a723e5d240 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 -- 172.21.15.37:0/3426519922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5d240 msgr2=0x55a723eb8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 --2- 172.21.15.37:0/3426519922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5d240 0x55a723eb8db0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f2970018000 tx=0x7f2970016000 comp rx=0 tx=0).stop 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 -- 172.21.15.37:0/3426519922 shutdown_connections 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 --2- 172.21.15.37:0/3426519922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e5d240 0x55a723eb8db0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 --2- 172.21.15.37:0/3426519922 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5c920 0x55a723eb8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 --2- 172.21.15.37:0/3426519922 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e12f30 0x55a723eb8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 -- 172.21.15.37:0/3426519922 >> 172.21.15.37:0/3426519922 conn(0x55a723e192a0 msgr2=0x55a723e16220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 -- 172.21.15.37:0/3426519922 shutdown_connections 2024-04-25T09:04:04.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 -- 172.21.15.37:0/3426519922 wait complete. 2024-04-25T09:04:04.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 -- start start 2024-04-25T09:04:04.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.485+0000 7f29805c4000 1 Processor -- start 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723ebd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5c920 0x55a723ebddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e5d240 0x55a723ea16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a723c663d0 con 0x55a723e12f30 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a723d1dd50 con 0x55a723e5d240 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a723bfc980 con 0x55a723e5c920 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 10 client.0 _ll_register_callbacks cb 0x7f2970009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723ebd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2976ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723ebd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43686/0 (socket says 172.21.15.37:43686) 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2976ffd640 1 -- 172.21.15.37:0/1295461476 learned_addr learned my addr 172.21.15.37:0/1295461476 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29767fc640 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5c920 0x55a723ebddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29777fe640 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e5d240 0x55a723ea16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2976ffd640 1 -- 172.21.15.37:0/1295461476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5c920 msgr2=0x55a723ebddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2976ffd640 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5c920 0x55a723ebddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2976ffd640 1 -- 172.21.15.37:0/1295461476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e5d240 msgr2=0x55a723ea16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2976ffd640 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e5d240 0x55a723ea16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2976ffd640 1 -- 172.21.15.37:0/1295461476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a723ce3600 con 0x55a723e12f30 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29767fc640 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5c920 0x55a723ebddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29777fe640 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e5d240 0x55a723ea16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2976ffd640 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723ebd880 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f29680552c0 tx=0x7f29680597d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6433be6c71683fe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2955ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2955ffb640 1 -- 172.21.15.37:0/1295461476 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2968051c30 con 0x55a723e12f30 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2955ffb640 1 -- 172.21.15.37:0/1295461476 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f296805d840 con 0x55a723e12f30 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 -- 172.21.15.37:0/1295461476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2970009df0 con 0x55a723e12f30 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2955ffb640 1 -- 172.21.15.37:0/1295461476 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2968051dd0 con 0x55a723e12f30 2024-04-25T09:04:04.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 10 client.9816 fetch_fsmap learned FSMap version 103 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 -- 172.21.15.37:0/1295461476 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x55a723bf69b0 con 0x55a723e12f30 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2955ffb640 1 -- 172.21.15.37:0/1295461476 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2970009df0 con 0x55a723e12f30 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2955ffb640 1 -- 172.21.15.37:0/1295461476 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 103) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f296807cb40 con 0x55a723e12f30 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 10 client.9816 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 10 client.9816 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 20 client.9816 populate_metadata read hostname 'smithi037' 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55a723ea1c00 0x55a723ea1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 4 client.9816 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 -- 172.21.15.37:0/1295461476 --> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a723c6f2e0 con 0x55a723ea1c00 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29767fc640 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55a723ea1c00 0x55a723ea1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29767fc640 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55a723ea1c00 0x55a723ea1fe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f4a3c11a8340bf5 server_cookie=cefdadca2545ff33 in_seq=0 out_seq=0 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2955ffb640 10 client.9816 ms_handle_connect on v2:172.21.15.37:6834/2159545482 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2955ffb640 1 -- 172.21.15.37:0/1295461476 <== mds.0 v2:172.21.15.37:6834/2159545482 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55a723c6f2e0 con 0x55a723ea1c00 2024-04-25T09:04:04.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2955ffb640 10 client.9816 handle_command_reply: tid=0 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 client.9816 shutdown 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 -- 172.21.15.37:0/1295461476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 msgr2=0x55a723ebd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723ebd880 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7f29680552c0 tx=0x7f29680597d0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 -- 172.21.15.37:0/1295461476 shutdown_connections 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55a723ea1c00 0x55a723ea1fe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55a723e5d240 0x55a723ea16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55a723e5c920 0x55a723ebddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f2955ffb640 0 client.9816 ms_handle_reset on v2:172.21.15.37:6834/2159545482 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 --2- 172.21.15.37:0/1295461476 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a723e12f30 0x55a723ebd880 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 -- 172.21.15.37:0/1295461476 >> 172.21.15.37:0/1295461476 conn(0x55a723e192a0 msgr2=0x55a723e647f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 -- 172.21.15.37:0/1295461476 shutdown_connections 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 1 -- 172.21.15.37:0/1295461476 wait complete. 2024-04-25T09:04:04.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f29805c4000 20 client.9816 trim_cache size 0 max 16384 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 9762, 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 4.4974069300000004, 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 103, 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 126, 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 126, 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 5.4507305199999996 2024-04-25T09:04:04.486 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:04.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 -- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2964066f60 msgr2=0x7f2964069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2964066f60 0x7f2964069420 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f296c0b2000 tx=0x7f296c008040 comp rx=0 tx=0).stop 2024-04-25T09:04:04.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 -- 172.21.15.37:0/2861673888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29780ceeb0 msgr2=0x7f29780cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29780ceeb0 0x7f29780cc9e0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f296800ca70 tx=0x7f296800cf30 comp rx=0 tx=0).stop 2024-04-25T09:04:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 -- 172.21.15.37:0/2861673888 shutdown_connections 2024-04-25T09:04:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2964066f60 0x7f2964069420 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f297815e2e0 0x7f2978146b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f29780d32d0 0x7f29781f32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 --2- 172.21.15.37:0/2861673888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29780ceeb0 0x7f29780cc9e0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 -- 172.21.15.37:0/2861673888 >> 172.21.15.37:0/2861673888 conn(0x7f29780d39f0 msgr2=0x7f297815f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 -- 172.21.15.37:0/2861673888 shutdown_connections 2024-04-25T09:04:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.489+0000 7f297d457640 1 -- 172.21.15.37:0/2861673888 wait complete. 2024-04-25T09:04:04.497 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:04:04.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc769a640 1 Processor -- start 2024-04-25T09:04:04.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc769a640 1 -- start start 2024-04-25T09:04:04.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc769a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8fc00d05d0 0x7f8fc00cee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc769a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc006cc70 con 0x7f8fc006bf00 2024-04-25T09:04:04.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc769a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc00ce090 con 0x7f8fc015ecf0 2024-04-25T09:04:04.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc769a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc015f0d0 con 0x7f8fc00d05d0 2024-04-25T09:04:04.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc4c35640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8fc00d05d0 0x7f8fc00cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc5c37640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8fc006bf00 0x7f8fc006c310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48740/0 (socket says 172.21.15.37:48740) 2024-04-25T09:04:04.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc5c37640 1 -- 172.21.15.37:0/2310595522 learned_addr learned my addr 172.21.15.37:0/2310595522 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:04.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc4c35640 1 -- 172.21.15.37:0/2310595522 >> v1:172.21.15.203:6789/0 conn(0x7f8fc015ecf0 legacy=0x7f8fc00cfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc4c35640 1 -- 172.21.15.37:0/2310595522 >> v1:172.21.15.37:6789/0 conn(0x7f8fc006bf00 legacy=0x7f8fc006c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fc4c35640 1 -- 172.21.15.37:0/2310595522 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00ce370 con 0x7f8fc00d05d0 2024-04-25T09:04:04.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.625+0000 7f8fb7fff640 1 -- 172.21.15.37:0/2310595522 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3480309978 0 0) 0x7f8fc006cc70 con 0x7f8fc006bf00 2024-04-25T09:04:04.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc4c35640 1 --2- 172.21.15.37:0/2310595522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc00d05d0 0x7f8fc00cee50 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00097c0 tx=0x7f8fb001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb25d39cf47480 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fb7fff640 1 -- 172.21.15.37:0/2310595522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00b1070 con 0x7f8fc00d05d0 2024-04-25T09:04:04.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fb7fff640 1 -- 172.21.15.37:0/2310595522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb00ac5f0 con 0x7f8fc00d05d0 2024-04-25T09:04:04.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fb7fff640 1 -- 172.21.15.37:0/2310595522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb00ac9c0 con 0x7f8fc00d05d0 2024-04-25T09:04:04.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 -- 172.21.15.37:0/2310595522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc00d05d0 msgr2=0x7f8fc00cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 --2- 172.21.15.37:0/2310595522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc00d05d0 0x7f8fc00cee50 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00097c0 tx=0x7f8fb001c4a0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 -- 172.21.15.37:0/2310595522 shutdown_connections 2024-04-25T09:04:04.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 --2- 172.21.15.37:0/2310595522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc00d05d0 0x7f8fc00cee50 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 -- 172.21.15.37:0/2310595522 >> 172.21.15.37:0/2310595522 conn(0x7f8fc00b52a0 msgr2=0x7f8fc00b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 -- 172.21.15.37:0/2310595522 shutdown_connections 2024-04-25T09:04:04.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 -- 172.21.15.37:0/2310595522 wait complete. 2024-04-25T09:04:04.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 Processor -- start 2024-04-25T09:04:04.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 -- start start 2024-04-25T09:04:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc006bf00 0x7f8fc0158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fc00d05d0 0x7f8fc015f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc015ecf0 0x7f8fc015ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc016ba00 con 0x7f8fc015ecf0 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc016b880 con 0x7f8fc006bf00 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc769a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc016bb80 con 0x7f8fc00d05d0 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc5c37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc015ecf0 0x7f8fc015ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc5c37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc015ecf0 0x7f8fc015ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43700/0 (socket says 172.21.15.37:43700) 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc5c37640 1 -- 172.21.15.37:0/1795866812 learned_addr learned my addr 172.21.15.37:0/1795866812 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc5436640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc006bf00 0x7f8fc0158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc4c35640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fc00d05d0 0x7f8fc015f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc5c37640 1 -- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fc00d05d0 msgr2=0x7f8fc015f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc5c37640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fc00d05d0 0x7f8fc015f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc5c37640 1 -- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc006bf00 msgr2=0x7f8fc0158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc5c37640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc006bf00 0x7f8fc0158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc5c37640 1 -- 172.21.15.37:0/1795866812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00d0d40 con 0x7f8fc015ecf0 2024-04-25T09:04:04.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc5436640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc006bf00 0x7f8fc0158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:04.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.629+0000 7f8fc5c37640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc015ecf0 0x7f8fc015ff30 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800c7a0 tx=0x7f8fb800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.633+0000 7f8fb5ffb640 1 -- 172.21.15.37:0/1795866812 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8018070 con 0x7f8fc015ecf0 2024-04-25T09:04:04.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.633+0000 7f8fb5ffb640 1 -- 172.21.15.37:0/1795866812 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb800a640 con 0x7f8fc015ecf0 2024-04-25T09:04:04.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.633+0000 7f8fc769a640 1 -- 172.21.15.37:0/1795866812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc014fc90 con 0x7f8fc015ecf0 2024-04-25T09:04:04.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.633+0000 7f8fb5ffb640 1 -- 172.21.15.37:0/1795866812 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800aab0 con 0x7f8fc015ecf0 2024-04-25T09:04:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.633+0000 7f8fc769a640 1 -- 172.21.15.37:0/1795866812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc015b1c0 con 0x7f8fc015ecf0 2024-04-25T09:04:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.633+0000 7f8fc769a640 1 -- 172.21.15.37:0/1795866812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc0067c40 con 0x7f8fc015ecf0 2024-04-25T09:04:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.633+0000 7f8fb5ffb640 1 -- 172.21.15.37:0/1795866812 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8fb801b070 con 0x7f8fc015ecf0 2024-04-25T09:04:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.633+0000 7f8fb5ffb640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fa4066f60 0x7f8fa4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.633+0000 7f8fb5ffb640 1 -- 172.21.15.37:0/1795866812 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8fb8084a90 con 0x7f8fc015ecf0 2024-04-25T09:04:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.633+0000 7f8fc5436640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fa4066f60 0x7f8fa4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.633+0000 7f8fc5436640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fa4066f60 0x7f8fa4069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc000f70 tx=0x7f8fbc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.637+0000 7f8fb5ffb640 1 -- 172.21.15.37:0/1795866812 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8fb8055a30 con 0x7f8fc015ecf0 2024-04-25T09:04:04.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 -- 172.21.15.37:0/1795866812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8fc00d3570 con 0x7f8fc015ecf0 2024-04-25T09:04:04.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fb5ffb640 1 -- 172.21.15.37:0/1795866812 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v103) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8fb8055330 con 0x7f8fc015ecf0 2024-04-25T09:04:04.826 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:04:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 -- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fa4066f60 msgr2=0x7f8fa4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fa4066f60 0x7f8fa4069420 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc000f70 tx=0x7f8fbc009040 comp rx=0 tx=0).stop 2024-04-25T09:04:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 -- 172.21.15.37:0/1795866812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc015ecf0 msgr2=0x7f8fc015ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc015ecf0 0x7f8fc015ff30 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f8fb800c7a0 tx=0x7f8fb800cc60 comp rx=0 tx=0).stop 2024-04-25T09:04:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 -- 172.21.15.37:0/1795866812 shutdown_connections 2024-04-25T09:04:04.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8fa4066f60 0x7f8fa4069420 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc015ecf0 0x7f8fc015ff30 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8fc00d05d0 0x7f8fc015f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 --2- 172.21.15.37:0/1795866812 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8fc006bf00 0x7f8fc0158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 -- 172.21.15.37:0/1795866812 >> 172.21.15.37:0/1795866812 conn(0x7f8fc00b52a0 msgr2=0x7f8fc00d2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 -- 172.21.15.37:0/1795866812 shutdown_connections 2024-04-25T09:04:04.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:04.837+0000 7f8fc769a640 1 -- 172.21.15.37:0/1795866812 wait complete. 2024-04-25T09:04:04.844 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:04:04.844 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:04:04.844 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T09:04:04.844 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:04:04.844 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:04.844 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:04:04.844 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:04:04.844 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:04:04.844 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:04.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:04.845 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T09:04:04.851 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:04:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:04:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:04:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T09:04:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:04.851 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:04.851 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T09:04:04.851 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41796sec preferred_lft 41796sec 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T09:04:04.852 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:04.853 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:04.854 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:04:04.854 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:04.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:04.854 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:04:04.900 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T09:04:04.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:04.902 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T09:04:04.954 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T09:04:04.954 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:04:04.954 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T09:04:04.954 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:04:04.954 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T09:04:04.954 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:04:04.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:04.955 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T09:04:04.955 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T09:04:04.955 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:04:04.955 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:04:04.955 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:04:04.955 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T09:04:04.984 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T09:04:04.993+0000 7fbfbe817640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:04:05.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:05.014 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T09:04:05.065 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:04:05.065 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:04:05.065 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:05.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:05.065 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T09:04:05.120 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:04:05.120 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:05.121 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:05.169 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:05.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:05.169 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:05.217 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:04:05.275 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:05.322 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:04:05.322 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:04:05.322 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:04:05.323 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:05.324 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:05.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:05.326 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:04:05.373 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:04:05.374 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:04:05.418 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:04:05.434 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:04:05.481 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:04:05.482 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:05.483 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:05.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:05.484 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:04:05.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T09:04:05.513+0000 7f8eb6b31580 -1 init, newargv = 0x555ddc84eb30 newargc=15 2024-04-25T09:04:05.502 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[55444]: starting ceph client 2024-04-25T09:04:05.509 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[55444]: starting fuse 2024-04-25T09:04:06.607 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:04:06.624 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:06.675 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:04:06.675 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:04:06.675 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:04:06.675 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:04:06.676 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:06.677 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:06.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:06.679 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:04:06.729 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T09:04:06.730 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:04:06.730 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:04:06.730 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T09:04:06.730 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T09:04:06.730 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:06.730 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T09:04:06.730 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:06.730 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T09:04:06.730 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:04:06.730 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T09:04:06.731 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T09:04:06.732 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:04:06.732 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:06.732 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T09:04:06.732 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:06.858 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.55444.asok 2024-04-25T09:04:06.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:06.870 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.55444.asok 2024-04-25T09:04:06.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:06.870 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.55444.asok status 2024-04-25T09:04:07.025 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "55444", 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 9834, 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T09:04:07.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 9834 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 572936589 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 572936589 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.9834 192.168.0.1:0/572936589", 2024-04-25T09:04:07.027 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/572936589", 2024-04-25T09:04:07.028 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T09:04:07.028 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 103, 2024-04-25T09:04:07.028 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 126, 2024-04-25T09:04:07.028 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:04:07.028 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T09:04:07.028 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T09:04:07.028 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:07.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:07.040 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T09:04:07.040 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:07.047 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T09:04:07.047 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:07.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:07.047 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:07.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:07.106 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T09:04:07.173 INFO:teuthology.orchestra.run.smithi037.stdout:./ 2024-04-25T09:04:07.173 INFO:teuthology.orchestra.run.smithi037.stdout:./parent 2024-04-25T09:04:07.173 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/flushed 2024-04-25T09:04:07.173 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/flushed/charlie 2024-04-25T09:04:07.173 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/flushed/alpha 2024-04-25T09:04:07.173 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/unflushed 2024-04-25T09:04:07.173 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/unflushed/jfile 2024-04-25T09:04:07.174 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:07.174 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:07.174 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:07.174 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:07.174 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T09:04:07.174 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:07.285 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T09:04:07.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:07.295 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:07.295 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:07.295 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:07.295 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:07.295 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T09:04:07.295 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:07.400 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627776 2024-04-25T09:04:07.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:07.411 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:07.411 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:07.411 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:07.411 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:07.411 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T09:04:07.411 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:07.520 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627777 2024-04-25T09:04:07.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:07.530 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:07.530 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:07.530 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:07.530 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:07.531 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T09:04:07.531 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:07.636 INFO:teuthology.orchestra.run.smithi037.stdout:1099511628280 2024-04-25T09:04:07.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:07.646 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:07.646 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:07.646 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:07.646 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:07.646 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T09:04:07.646 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:07.754 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627778 2024-04-25T09:04:07.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:07.764 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:07.764 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:07.765 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:07.765 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:07.765 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T09:04:07.765 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:07.877 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627779 2024-04-25T09:04:07.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:07.887 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:07.887 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:07.888 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:07.888 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:07.888 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T09:04:07.888 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:07.992 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627780 2024-04-25T09:04:07.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:08.003 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:04:08.003 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:08.050 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.237+0000 7fd0ed01e640 1 Processor -- start 2024-04-25T09:04:08.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.237+0000 7fd0ed01e640 1 -- start start 2024-04-25T09:04:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0ed01e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd0e8071590 0x7fd0e815c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0ed01e640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e815f240 con 0x7fd0e815edc0 2024-04-25T09:04:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0ed01e640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e815f520 con 0x7fd0e8071130 2024-04-25T09:04:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0ed01e640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e815f800 con 0x7fd0e8071590 2024-04-25T09:04:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e759e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd0e8071590 0x7fd0e815c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e6d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd0e815edc0 0x7fd0e8144910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48748/0 (socket says 172.21.15.37:48748) 2024-04-25T09:04:08.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e6d9d640 1 -- 172.21.15.37:0/618809237 learned_addr learned my addr 172.21.15.37:0/618809237 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:08.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e5d9b640 1 -- 172.21.15.37:0/618809237 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573483135 0 0) 0x7fd0e815f240 con 0x7fd0e815edc0 2024-04-25T09:04:08.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e5d9b640 1 -- 172.21.15.37:0/618809237 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0d0003530 con 0x7fd0e815edc0 2024-04-25T09:04:08.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e759e640 1 -- 172.21.15.37:0/618809237 >> v1:172.21.15.203:6789/0 conn(0x7fd0e8071130 legacy=0x7fd0e815bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e759e640 1 -- 172.21.15.37:0/618809237 >> v1:172.21.15.37:6789/0 conn(0x7fd0e815edc0 legacy=0x7fd0e8144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e759e640 1 -- 172.21.15.37:0/618809237 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e815fae0 con 0x7fd0e8071590 2024-04-25T09:04:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e759e640 1 --2- 172.21.15.37:0/618809237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0e8071590 0x7fd0e815c610 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00097c0 tx=0x7fd0e001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4b9ce31f258c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e5d9b640 1 -- 172.21.15.37:0/618809237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e00b0070 con 0x7fd0e8071590 2024-04-25T09:04:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e5d9b640 1 -- 172.21.15.37:0/618809237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0e0004230 con 0x7fd0e8071590 2024-04-25T09:04:08.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.241+0000 7fd0e5d9b640 1 -- 172.21.15.37:0/618809237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0003620 con 0x7fd0e8071590 2024-04-25T09:04:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 -- 172.21.15.37:0/618809237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0e8071590 msgr2=0x7fd0e815c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 --2- 172.21.15.37:0/618809237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0e8071590 0x7fd0e815c610 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fd0e00097c0 tx=0x7fd0e001c460 comp rx=0 tx=0).stop 2024-04-25T09:04:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 -- 172.21.15.37:0/618809237 shutdown_connections 2024-04-25T09:04:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 --2- 172.21.15.37:0/618809237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0e8071590 0x7fd0e815c610 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 -- 172.21.15.37:0/618809237 >> 172.21.15.37:0/618809237 conn(0x7fd0e8066fa0 msgr2=0x7fd0e806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 -- 172.21.15.37:0/618809237 shutdown_connections 2024-04-25T09:04:08.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 -- 172.21.15.37:0/618809237 wait complete. 2024-04-25T09:04:08.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 Processor -- start 2024-04-25T09:04:08.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 -- start start 2024-04-25T09:04:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0e8071130 0x7fd0e815a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0e8071590 0x7fd0e80ad020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0e815edc0 0x7fd0e8138c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e81451a0 con 0x7fd0e8071130 2024-04-25T09:04:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e8145020 con 0x7fd0e8071590 2024-04-25T09:04:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e81695f0 con 0x7fd0e815edc0 2024-04-25T09:04:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0e6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0e8071130 0x7fd0e815a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0e6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0e8071130 0x7fd0e815a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43732/0 (socket says 172.21.15.37:43732) 2024-04-25T09:04:08.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0e6d9d640 1 -- 172.21.15.37:0/3412328184 learned_addr learned my addr 172.21.15.37:0/3412328184 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0e6d9d640 1 -- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0e815edc0 msgr2=0x7fd0e8138c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0e6d9d640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0e815edc0 0x7fd0e8138c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0e6d9d640 1 -- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0e8071590 msgr2=0x7fd0e80ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0e6d9d640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0e8071590 0x7fd0e80ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0e6d9d640 1 -- 172.21.15.37:0/3412328184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e80ae410 con 0x7fd0e8071130 2024-04-25T09:04:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0e6d9d640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0e8071130 0x7fd0e815a900 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800cad0 tx=0x7fd0d800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0cf7fe640 1 -- 172.21.15.37:0/3412328184 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8017070 con 0x7fd0e8071130 2024-04-25T09:04:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0cf7fe640 1 -- 172.21.15.37:0/3412328184 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d800ac30 con 0x7fd0e8071130 2024-04-25T09:04:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 -- 172.21.15.37:0/3412328184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e80ade50 con 0x7fd0e8071130 2024-04-25T09:04:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0cf7fe640 1 -- 172.21.15.37:0/3412328184 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8012400 con 0x7fd0e8071130 2024-04-25T09:04:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.245+0000 7fd0ed01e640 1 -- 172.21.15.37:0/3412328184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e8142780 con 0x7fd0e8071130 2024-04-25T09:04:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.249+0000 7fd0ed01e640 1 -- 172.21.15.37:0/3412328184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e80b3740 con 0x7fd0e8071130 2024-04-25T09:04:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.249+0000 7fd0cf7fe640 1 -- 172.21.15.37:0/3412328184 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0d801a070 con 0x7fd0e8071130 2024-04-25T09:04:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.249+0000 7fd0cf7fe640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0d4066f90 0x7fd0d4069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.249+0000 7fd0cf7fe640 1 -- 172.21.15.37:0/3412328184 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd0d8084f80 con 0x7fd0e8071130 2024-04-25T09:04:08.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.249+0000 7fd0e659c640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0d4066f90 0x7fd0d4069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.249+0000 7fd0e659c640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0d4066f90 0x7fd0d4069450 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8144c40 tx=0x7fd0dc0079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.253+0000 7fd0cf7fe640 1 -- 172.21.15.37:0/3412328184 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd0d8055db0 con 0x7fd0e8071130 2024-04-25T09:04:08.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.413+0000 7fd0ed01e640 1 -- 172.21.15.37:0/3412328184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0e8067c40 con 0x7fd0e8071130 2024-04-25T09:04:08.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0cf7fe640 1 -- 172.21.15.37:0/3412328184 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+17045 (secure 0 0 0) 0x7fd0d80556b0 con 0x7fd0e8071130 2024-04-25T09:04:08.405 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:08.405 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":126,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:58.978382+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2748919477":"2024-04-26T09:03:58.978336+0000","172.21.15.37:6834/2748919477":"2024-04-26T09:03:58.978336+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:08.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 -- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0d4066f90 msgr2=0x7fd0d4069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0d4066f90 0x7fd0d4069450 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8144c40 tx=0x7fd0dc0079e0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 -- 172.21.15.37:0/3412328184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0e8071130 msgr2=0x7fd0e815a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0e8071130 0x7fd0e815a900 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800cad0 tx=0x7fd0d800cf90 comp rx=0 tx=0).stop 2024-04-25T09:04:08.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 -- 172.21.15.37:0/3412328184 shutdown_connections 2024-04-25T09:04:08.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd0d4066f90 0x7fd0d4069450 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0e815edc0 0x7fd0e8138c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0e8071590 0x7fd0e80ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 --2- 172.21.15.37:0/3412328184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0e8071130 0x7fd0e815a900 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 -- 172.21.15.37:0/3412328184 >> 172.21.15.37:0/3412328184 conn(0x7fd0e8066fa0 msgr2=0x7fd0e8143c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 -- 172.21.15.37:0/3412328184 shutdown_connections 2024-04-25T09:04:08.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.417+0000 7fd0ed01e640 1 -- 172.21.15.37:0/3412328184 wait complete. 2024-04-25T09:04:08.425 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:30.763724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:31.488462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:08.425 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cc6e3b640 1 Processor -- start 2024-04-25T09:04:08.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cc6e3b640 1 -- start start 2024-04-25T09:04:08.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cc6e3b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5cc015c980 0x7f5cc015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cc6e3b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc00cda10 con 0x7f5cc0071110 2024-04-25T09:04:08.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cc6e3b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc00cdcf0 con 0x7f5cc0144e40 2024-04-25T09:04:08.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cc6e3b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc015f260 con 0x7f5cc015c980 2024-04-25T09:04:08.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cbffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5cc015c980 0x7f5cc015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cc53d8640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5cc0071110 0x7f5cc015ce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48764/0 (socket says 172.21.15.37:48764) 2024-04-25T09:04:08.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cc53d8640 1 -- 172.21.15.37:0/2406266214 learned_addr learned my addr 172.21.15.37:0/2406266214 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:08.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cbffff640 1 -- 172.21.15.37:0/2406266214 >> v1:172.21.15.203:6789/0 conn(0x7f5cc0144e40 legacy=0x7f5cc00cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cbf7fe640 1 -- 172.21.15.37:0/2406266214 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562645847 0 0) 0x7f5cc00cda10 con 0x7f5cc0071110 2024-04-25T09:04:08.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cbffff640 1 -- 172.21.15.37:0/2406266214 >> v1:172.21.15.37:6789/0 conn(0x7f5cc0071110 legacy=0x7f5cc015ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.569+0000 7f5cbffff640 1 -- 172.21.15.37:0/2406266214 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc015f4b0 con 0x7f5cc015c980 2024-04-25T09:04:08.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.573+0000 7f5cbffff640 1 --2- 172.21.15.37:0/2406266214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc015c980 0x7f5cc015fcb0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097c0 tx=0x7f5cb401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a8c46e5d4e62ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.573+0000 7f5cbf7fe640 1 -- 172.21.15.37:0/2406266214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40b0070 con 0x7f5cc015c980 2024-04-25T09:04:08.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.573+0000 7f5cbf7fe640 1 -- 172.21.15.37:0/2406266214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb40ab600 con 0x7f5cc015c980 2024-04-25T09:04:08.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.573+0000 7f5cbf7fe640 1 -- 172.21.15.37:0/2406266214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40ab9d0 con 0x7f5cc015c980 2024-04-25T09:04:08.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.573+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/2406266214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc015c980 msgr2=0x7f5cc015fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.573+0000 7f5cc6e3b640 1 --2- 172.21.15.37:0/2406266214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc015c980 0x7f5cc015fcb0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40097c0 tx=0x7f5cb401c4a0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.573+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/2406266214 shutdown_connections 2024-04-25T09:04:08.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.573+0000 7f5cc6e3b640 1 --2- 172.21.15.37:0/2406266214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc015c980 0x7f5cc015fcb0 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.573+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/2406266214 >> 172.21.15.37:0/2406266214 conn(0x7f5cc0066fa0 msgr2=0x7f5cc006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.573+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/2406266214 shutdown_connections 2024-04-25T09:04:08.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.573+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/2406266214 wait complete. 2024-04-25T09:04:08.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc6e3b640 1 Processor -- start 2024-04-25T09:04:08.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc6e3b640 1 -- start start 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc6e3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc0071110 0x7f5cc00ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc6e3b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cc0144e40 0x7f5cc00ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc6e3b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cc015c980 0x7f5cc013ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc6e3b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc01601f0 con 0x7f5cc0071110 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc6e3b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc015f820 con 0x7f5cc0144e40 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc6e3b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc0169e20 con 0x7f5cc015c980 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc4bd7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc0071110 0x7f5cc00ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc4bd7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc0071110 0x7f5cc00ae940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43744/0 (socket says 172.21.15.37:43744) 2024-04-25T09:04:08.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc4bd7640 1 -- 172.21.15.37:0/3494504620 learned_addr learned my addr 172.21.15.37:0/3494504620 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cbffff640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cc0144e40 0x7f5cc00ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc53d8640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cc015c980 0x7f5cc013ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc4bd7640 1 -- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cc015c980 msgr2=0x7f5cc013ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc4bd7640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cc015c980 0x7f5cc013ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc4bd7640 1 -- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cc0144e40 msgr2=0x7f5cc00ad330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc4bd7640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cc0144e40 0x7f5cc00ad330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc4bd7640 1 -- 172.21.15.37:0/3494504620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc0142870 con 0x7f5cc0071110 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cbffff640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cc0144e40 0x7f5cc00ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc53d8640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cc015c980 0x7f5cc013ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc4bd7640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc0071110 0x7f5cc00ae940 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000c450 tx=0x7f5cb000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cbd7fa640 1 -- 172.21.15.37:0/3494504620 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb00052d0 con 0x7f5cc0071110 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cbd7fa640 1 -- 172.21.15.37:0/3494504620 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb0005470 con 0x7f5cc0071110 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/3494504620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc00ad870 con 0x7f5cc0071110 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cbd7fa640 1 -- 172.21.15.37:0/3494504620 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb0007260 con 0x7f5cc0071110 2024-04-25T09:04:08.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/3494504620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc00b4290 con 0x7f5cc0071110 2024-04-25T09:04:08.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/3494504620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc0142a40 con 0x7f5cc0071110 2024-04-25T09:04:08.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.577+0000 7f5cbd7fa640 1 -- 172.21.15.37:0/3494504620 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5cb0007400 con 0x7f5cc0071110 2024-04-25T09:04:08.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.581+0000 7f5cbd7fa640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cac066f60 0x7f5cac069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.581+0000 7f5cbd7fa640 1 -- 172.21.15.37:0/3494504620 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5cb0084700 con 0x7f5cc0071110 2024-04-25T09:04:08.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.581+0000 7f5cbffff640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cac066f60 0x7f5cac069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.581+0000 7f5cbffff640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cac066f60 0x7f5cac069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40ad000 tx=0x7f5cb4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.585+0000 7f5cbd7fa640 1 -- 172.21.15.37:0/3494504620 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5cb0055520 con 0x7f5cc0071110 2024-04-25T09:04:08.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.745+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/3494504620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5cc0067c40 con 0x7f5cc0071110 2024-04-25T09:04:08.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.745+0000 7f5cbd7fa640 1 -- 172.21.15.37:0/3494504620 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5cb0054e20 con 0x7f5cc0071110 2024-04-25T09:04:08.735 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:04:08.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cac066f60 msgr2=0x7f5cac069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cac066f60 0x7f5cac069420 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40ad000 tx=0x7f5cb4008040 comp rx=0 tx=0).stop 2024-04-25T09:04:08.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/3494504620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc0071110 msgr2=0x7f5cc00ae940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc0071110 0x7f5cc00ae940 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f5cb000c450 tx=0x7f5cb000c910 comp rx=0 tx=0).stop 2024-04-25T09:04:08.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/3494504620 shutdown_connections 2024-04-25T09:04:08.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5cac066f60 0x7f5cac069420 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5cc015c980 0x7f5cc013ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5cc0144e40 0x7f5cc00ad330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 --2- 172.21.15.37:0/3494504620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cc0071110 0x7f5cc00ae940 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/3494504620 >> 172.21.15.37:0/3494504620 conn(0x7f5cc0066fa0 msgr2=0x7f5cc0140500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/3494504620 shutdown_connections 2024-04-25T09:04:08.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.749+0000 7f5cc6e3b640 1 -- 172.21.15.37:0/3494504620 wait complete. 2024-04-25T09:04:08.754 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:08.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba2d954640 1 Processor -- start 2024-04-25T09:04:08.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba2d954640 1 -- start start 2024-04-25T09:04:08.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba2d954640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fba2815d830 0x7fba2815aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba2d954640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba28143870 0x7fba2806e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba2d954640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2815af80 con 0x7fba2815dc10 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba2d954640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba28071960 con 0x7fba2815d830 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba2d954640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba28071c40 con 0x7fba28143870 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba267fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fba2815dc10 0x7fba28071110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48774/0 (socket says 172.21.15.37:48774) 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba267fc640 1 -- 172.21.15.37:0/3012849739 learned_addr learned my addr 172.21.15.37:0/3012849739 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba26ffd640 1 --2- 172.21.15.37:0/3012849739 >> v2:172.21.15.203:3300/0 conn(0x7fba2815d830 0x7fba2815aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2326932564 0 0) 0x7fba2815af80 con 0x7fba2815dc10 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba10003530 con 0x7fba2815dc10 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba277fe640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba28143870 0x7fba2806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3769995775 0 0) 0x7fba10003530 con 0x7fba2815dc10 2024-04-25T09:04:08.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba28143870 msgr2=0x7fba2806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba28143870 0x7fba2806e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 >> v2:172.21.15.203:3300/0 conn(0x7fba2815d830 msgr2=0x7fba2815aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 --2- 172.21.15.37:0/3012849739 >> v2:172.21.15.203:3300/0 conn(0x7fba2815d830 0x7fba2815aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba28071f20 con 0x7fba2815dc10 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fba18001680 con 0x7fba2815dc10 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 >> v1:172.21.15.37:6789/0 conn(0x7fba2815dc10 legacy=0x7fba28071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba28143870 0x7fba100039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10003f40 0x7fba10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba10004d50 0x7fba100051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba28071960 con 0x7fba10003f40 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba28071c40 con 0x7fba10004d50 2024-04-25T09:04:08.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba1000ba20 con 0x7fba28143870 2024-04-25T09:04:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba267fc640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10003f40 0x7fba10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba26ffd640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba28143870 0x7fba100039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba277fe640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba10004d50 0x7fba100051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba267fc640 1 -- 172.21.15.37:0/3012849739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba28143870 msgr2=0x7fba100039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba267fc640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba28143870 0x7fba100039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba267fc640 1 -- 172.21.15.37:0/3012849739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba10004d50 msgr2=0x7fba100051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba267fc640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba10004d50 0x7fba100051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.897+0000 7fba267fc640 1 -- 172.21.15.37:0/3012849739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10003530 con 0x7fba10003f40 2024-04-25T09:04:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba267fc640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10003f40 0x7fba10004810 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fba1800c270 tx=0x7fba1801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248a138537e862e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18007d40 con 0x7fba10003f40 2024-04-25T09:04:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba180a9020 con 0x7fba10003f40 2024-04-25T09:04:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba25ffb640 1 -- 172.21.15.37:0/3012849739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18005b00 con 0x7fba10003f40 2024-04-25T09:04:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba2d954640 1 -- 172.21.15.37:0/3012849739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10003f40 msgr2=0x7fba10004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba2d954640 1 --2- 172.21.15.37:0/3012849739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba10003f40 0x7fba10004810 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7fba1800c270 tx=0x7fba1801cca0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba277fe640 1 -- 172.21.15.37:0/3012849739 reap_dead start 2024-04-25T09:04:08.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba2d954640 1 -- 172.21.15.37:0/3012849739 shutdown_connections 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba2d954640 1 -- 172.21.15.37:0/3012849739 >> 172.21.15.37:0/3012849739 conn(0x7fba28066fa0 msgr2=0x7fba2806ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba2d954640 1 -- 172.21.15.37:0/3012849739 shutdown_connections 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba2d954640 1 -- 172.21.15.37:0/3012849739 wait complete. 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba2d954640 1 Processor -- start 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.901+0000 7fba2d954640 1 -- start start 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba2d954640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba10003f40 0x7fba281f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba2d954640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba28142670 0x7fba281f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba2d954640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba28139610 0x7fba281405d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba2d954640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba280b4050 con 0x7fba28139610 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba2d954640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba280b3ed0 con 0x7fba28142670 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba2d954640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba281650b0 con 0x7fba10003f40 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba28139610 0x7fba281405d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba28139610 0x7fba281405d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43758/0 (socket says 172.21.15.37:43758) 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba277fe640 1 -- 172.21.15.37:0/1015930518 learned_addr learned my addr 172.21.15.37:0/1015930518 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba26ffd640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba10003f40 0x7fba281f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba267fc640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba28142670 0x7fba281f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba277fe640 1 -- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba10003f40 msgr2=0x7fba281f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba277fe640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba10003f40 0x7fba281f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba277fe640 1 -- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba28142670 msgr2=0x7fba281f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba277fe640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba28142670 0x7fba281f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba277fe640 1 -- 172.21.15.37:0/1015930518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba28142e60 con 0x7fba28139610 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba277fe640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba28139610 0x7fba281405d0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fba2806f340 tx=0x7fba200115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba0ffff640 1 -- 172.21.15.37:0/1015930518 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba20011e40 con 0x7fba28139610 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba0ffff640 1 -- 172.21.15.37:0/1015930518 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2001b070 con 0x7fba28139610 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba0ffff640 1 -- 172.21.15.37:0/1015930518 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba200039a0 con 0x7fba28139610 2024-04-25T09:04:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba2d954640 1 -- 172.21.15.37:0/1015930518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba2813a580 con 0x7fba28139610 2024-04-25T09:04:08.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba2d954640 1 -- 172.21.15.37:0/1015930518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2813ab40 con 0x7fba28139610 2024-04-25T09:04:08.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba2d954640 1 -- 172.21.15.37:0/1015930518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba28067c40 con 0x7fba28139610 2024-04-25T09:04:08.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba0ffff640 1 -- 172.21.15.37:0/1015930518 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fba20003b40 con 0x7fba28139610 2024-04-25T09:04:08.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba0ffff640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba14066f90 0x7fba14069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba0ffff640 1 -- 172.21.15.37:0/1015930518 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fba200859e0 con 0x7fba28139610 2024-04-25T09:04:08.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.905+0000 7fba26ffd640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba14066f90 0x7fba14069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.909+0000 7fba26ffd640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba14066f90 0x7fba14069450 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009a30 tx=0x7fba1c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:08.913+0000 7fba0ffff640 1 -- 172.21.15.37:0/1015930518 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fba20025020 con 0x7fba28139610 2024-04-25T09:04:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.073+0000 7fba2d954640 1 -- 172.21.15.37:0/1015930518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fba2806c110 con 0x7fba28139610 2024-04-25T09:04:09.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba0ffff640 1 -- 172.21.15.37:0/1015930518 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7fba20056810 con 0x7fba28139610 2024-04-25T09:04:09.065 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:04:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 -- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba14066f90 msgr2=0x7fba14069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba14066f90 0x7fba14069450 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009a30 tx=0x7fba1c008040 comp rx=0 tx=0).stop 2024-04-25T09:04:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 -- 172.21.15.37:0/1015930518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba28139610 msgr2=0x7fba281405d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba28139610 0x7fba281405d0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7fba2806f340 tx=0x7fba200115a0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 -- 172.21.15.37:0/1015930518 shutdown_connections 2024-04-25T09:04:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fba14066f90 0x7fba14069450 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba28139610 0x7fba281405d0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fba28142670 0x7fba281f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 --2- 172.21.15.37:0/1015930518 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fba10003f40 0x7fba281f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 -- 172.21.15.37:0/1015930518 >> 172.21.15.37:0/1015930518 conn(0x7fba28066fa0 msgr2=0x7fba2813fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 -- 172.21.15.37:0/1015930518 shutdown_connections 2024-04-25T09:04:09.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.077+0000 7fba2d954640 1 -- 172.21.15.37:0/1015930518 wait complete. 2024-04-25T09:04:09.084 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:09.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.225+0000 7f1bc6768640 1 Processor -- start 2024-04-25T09:04:09.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.225+0000 7f1bc6768640 1 -- start start 2024-04-25T09:04:09.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bc6768640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1bc0144e40 0x7f1bc015fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bc6768640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc00cda10 con 0x7f1bc015c980 2024-04-25T09:04:09.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bc6768640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bc00cdcf0 con 0x7f1bc0071110 2024-04-25T09:04:09.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bc6768640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bc015f260 con 0x7f1bc0144e40 2024-04-25T09:04:09.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbf7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1bc015c980 0x7f1bc00cd130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48782/0 (socket says 172.21.15.37:48782) 2024-04-25T09:04:09.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbf7fe640 1 -- 172.21.15.37:0/3253950638 learned_addr learned my addr 172.21.15.37:0/3253950638 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:09.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbffff640 1 --2- 172.21.15.37:0/3253950638 >> v2:172.21.15.203:3300/0 conn(0x7f1bc0144e40 0x7f1bc015fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 372642186 0 0) 0x7f1bc00cda10 con 0x7f1bc015c980 2024-04-25T09:04:09.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ba8003530 con 0x7f1bc015c980 2024-04-25T09:04:09.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1273807754 0 0) 0x7f1ba8003530 con 0x7f1bc015c980 2024-04-25T09:04:09.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 >> v2:172.21.15.203:3300/0 conn(0x7f1bc0144e40 msgr2=0x7f1bc015fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 --2- 172.21.15.37:0/3253950638 >> v2:172.21.15.203:3300/0 conn(0x7f1bc0144e40 0x7f1bc015fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 >> v1:172.21.15.203:6789/0 conn(0x7f1bc0071110 legacy=0x7f1bc015ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc015f4b0 con 0x7f1bc015c980 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1bb4001680 con 0x7f1bc015c980 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 >> v1:172.21.15.37:6789/0 conn(0x7f1bc015c980 legacy=0x7f1bc00cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 --2- 172.21.15.37:0/3253950638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ba80039c0 0x7f1ba8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 --2- 172.21.15.37:0/3253950638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba8004810 0x7f1ba8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 --2- 172.21.15.37:0/3253950638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 0x7f1ba800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc015f260 con 0x7f1ba80051d0 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba800c070 con 0x7f1ba80039c0 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba800c270 con 0x7f1ba8004810 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bc4d05640 1 --2- 172.21.15.37:0/3253950638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 0x7f1ba800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbf7fe640 1 --2- 172.21.15.37:0/3253950638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba8004810 0x7f1ba8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bbffff640 1 --2- 172.21.15.37:0/3253950638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ba80039c0 0x7f1ba8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bc4d05640 1 -- 172.21.15.37:0/3253950638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba8004810 msgr2=0x7f1ba8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bc4d05640 1 --2- 172.21.15.37:0/3253950638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1ba8004810 0x7f1ba8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bc4d05640 1 -- 172.21.15.37:0/3253950638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ba80039c0 msgr2=0x7f1ba8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bc4d05640 1 --2- 172.21.15.37:0/3253950638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1ba80039c0 0x7f1ba8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.229+0000 7f1bc4d05640 1 -- 172.21.15.37:0/3253950638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba8003530 con 0x7f1ba80051d0 2024-04-25T09:04:09.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.233+0000 7f1bc4d05640 1 -- 172.21.15.37:0/3253950638 reap_dead start 2024-04-25T09:04:09.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.233+0000 7f1bc4d05640 1 --2- 172.21.15.37:0/3253950638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 0x7f1ba800b9d0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8001e70 tx=0x7f1bb801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b22e0e0912a3ca8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.233+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8001a90 con 0x7f1ba80051d0 2024-04-25T09:04:09.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.233+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb8003bc0 con 0x7f1ba80051d0 2024-04-25T09:04:09.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.233+0000 7f1bbeffd640 1 -- 172.21.15.37:0/3253950638 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb8005020 con 0x7f1ba80051d0 2024-04-25T09:04:09.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.233+0000 7f1bc6768640 1 -- 172.21.15.37:0/3253950638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 msgr2=0x7f1ba800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.233+0000 7f1bc6768640 1 --2- 172.21.15.37:0/3253950638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 0x7f1ba800b9d0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f1bb8001e70 tx=0x7f1bb801c780 comp rx=0 tx=0).stop 2024-04-25T09:04:09.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.233+0000 7f1bc6768640 1 -- 172.21.15.37:0/3253950638 shutdown_connections 2024-04-25T09:04:09.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.233+0000 7f1bc6768640 1 --2- 172.21.15.37:0/3253950638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 0x7f1ba800b9d0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.233+0000 7f1bc6768640 1 -- 172.21.15.37:0/3253950638 >> 172.21.15.37:0/3253950638 conn(0x7f1bc0066fa0 msgr2=0x7f1bc006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 -- 172.21.15.37:0/3253950638 shutdown_connections 2024-04-25T09:04:09.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 -- 172.21.15.37:0/3253950638 wait complete. 2024-04-25T09:04:09.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 Processor -- start 2024-04-25T09:04:09.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 -- start start 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 0x7f1bc01f08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc01f4ec0 0x7f1bc0134d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc01352d0 0x7f1bc0132d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc015fcb0 con 0x7f1ba80051d0 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc015f820 con 0x7f1bc01352d0 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bc015fe30 con 0x7f1bc01f4ec0 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 0x7f1bc01f08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 0x7f1bc01f08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43770/0 (socket says 172.21.15.37:43770) 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbffff640 1 -- 172.21.15.37:0/4077307157 learned_addr learned my addr 172.21.15.37:0/4077307157 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbf7fe640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc01f4ec0 0x7f1bc0134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc4d05640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc01352d0 0x7f1bc0132d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbffff640 1 -- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc01f4ec0 msgr2=0x7f1bc0134d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbffff640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc01f4ec0 0x7f1bc0134d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbffff640 1 -- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc01352d0 msgr2=0x7f1bc0132d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbffff640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc01352d0 0x7f1bc0132d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbffff640 1 -- 172.21.15.37:0/4077307157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bc0131cf0 con 0x7f1ba80051d0 2024-04-25T09:04:09.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc4d05640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc01352d0 0x7f1bc0132d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:09.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbffff640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 0x7f1bc01f08d0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0011560 tx=0x7f1bb0011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbcff9640 1 -- 172.21.15.37:0/4077307157 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0015930 con 0x7f1ba80051d0 2024-04-25T09:04:09.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbcff9640 1 -- 172.21.15.37:0/4077307157 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bb0015ad0 con 0x7f1ba80051d0 2024-04-25T09:04:09.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 -- 172.21.15.37:0/4077307157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bc0131fd0 con 0x7f1ba80051d0 2024-04-25T09:04:09.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bbcff9640 1 -- 172.21.15.37:0/4077307157 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb001d400 con 0x7f1ba80051d0 2024-04-25T09:04:09.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.237+0000 7f1bc6768640 1 -- 172.21.15.37:0/4077307157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bc0132590 con 0x7f1ba80051d0 2024-04-25T09:04:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.241+0000 7f1bc6768640 1 -- 172.21.15.37:0/4077307157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc0067c40 con 0x7f1ba80051d0 2024-04-25T09:04:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.241+0000 7f1bbcff9640 1 -- 172.21.15.37:0/4077307157 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1bb001d5a0 con 0x7f1ba80051d0 2024-04-25T09:04:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.241+0000 7f1bbcff9640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bac066f90 0x7f1bac069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.241+0000 7f1bbcff9640 1 -- 172.21.15.37:0/4077307157 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f1bb0085a80 con 0x7f1ba80051d0 2024-04-25T09:04:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.241+0000 7f1bbf7fe640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bac066f90 0x7f1bac069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.241+0000 7f1bbf7fe640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bac066f90 0x7f1bac069450 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00cca20 tx=0x7f1bb4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.245+0000 7f1bbcff9640 1 -- 172.21.15.37:0/4077307157 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1bb0087e20 con 0x7f1ba80051d0 2024-04-25T09:04:09.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.401+0000 7f1bc6768640 1 -- 172.21.15.37:0/4077307157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1bc0072220 con 0x7f1ba80051d0 2024-04-25T09:04:09.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bbcff9640 1 -- 172.21.15.37:0/4077307157 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1bb0019030 con 0x7f1ba80051d0 2024-04-25T09:04:09.393 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:04:09.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 -- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bac066f90 msgr2=0x7f1bac069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bac066f90 0x7f1bac069450 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1bc00cca20 tx=0x7f1bb4009040 comp rx=0 tx=0).stop 2024-04-25T09:04:09.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 -- 172.21.15.37:0/4077307157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 msgr2=0x7f1bc01f08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 0x7f1bc01f08d0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0011560 tx=0x7f1bb0011a20 comp rx=0 tx=0).stop 2024-04-25T09:04:09.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 -- 172.21.15.37:0/4077307157 shutdown_connections 2024-04-25T09:04:09.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1bac066f90 0x7f1bac069450 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1bc01352d0 0x7f1bc0132d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1bc01f4ec0 0x7f1bc0134d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 --2- 172.21.15.37:0/4077307157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ba80051d0 0x7f1bc01f08d0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 -- 172.21.15.37:0/4077307157 >> 172.21.15.37:0/4077307157 conn(0x7f1bc0066fa0 msgr2=0x7f1bc00b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 -- 172.21.15.37:0/4077307157 shutdown_connections 2024-04-25T09:04:09.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.405+0000 7f1bc6768640 1 -- 172.21.15.37:0/4077307157 wait complete. 2024-04-25T09:04:09.413 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:04:09.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f314a6ec640 1 Processor -- start 2024-04-25T09:04:09.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f314a6ec640 1 -- start start 2024-04-25T09:04:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f314a6ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31441441e0 0x7f31440ad460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f314a6ec640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f31440b1640 0x7f31440b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f314a6ec640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31440cd370 con 0x7f3144143e00 2024-04-25T09:04:09.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f314a6ec640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31440b0300 con 0x7f31440b1640 2024-04-25T09:04:09.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f314a6ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31440cd640 con 0x7f31441441e0 2024-04-25T09:04:09.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f3148c89640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f31440b1640 0x7f31440b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f3143fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3144143e00 0x7f31440ccc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48236/0 (socket says 172.21.15.37:48236) 2024-04-25T09:04:09.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f3148c89640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f31440b1640 0x7f31440b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42596/0 (socket says 172.21.15.37:42596) 2024-04-25T09:04:09.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f3143fff640 1 -- 172.21.15.37:0/2110191161 learned_addr learned my addr 172.21.15.37:0/2110191161 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:09.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f31437fe640 1 --2- 172.21.15.37:0/2110191161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31441441e0 0x7f31440ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f3148c89640 1 -- 172.21.15.37:0/2110191161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31441441e0 msgr2=0x7f31440ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f3148c89640 1 --2- 172.21.15.37:0/2110191161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31441441e0 0x7f31440ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f3148c89640 1 -- 172.21.15.37:0/2110191161 >> v1:172.21.15.37:6789/0 conn(0x7f3144143e00 legacy=0x7f31440ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.553+0000 7f3148c89640 1 -- 172.21.15.37:0/2110191161 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31440cd920 con 0x7f31440b1640 2024-04-25T09:04:09.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f3148c89640 1 --2- 172.21.15.37:0/2110191161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31440b1640 0x7f31440b1ad0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f31340097c0 tx=0x7f313401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d1d67b396af453 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f3142ffd640 1 -- 172.21.15.37:0/2110191161 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31340b0070 con 0x7f31440b1640 2024-04-25T09:04:09.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f3142ffd640 1 -- 172.21.15.37:0/2110191161 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31340ab600 con 0x7f31440b1640 2024-04-25T09:04:09.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f3142ffd640 1 -- 172.21.15.37:0/2110191161 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31340ab9d0 con 0x7f31440b1640 2024-04-25T09:04:09.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f314a6ec640 1 -- 172.21.15.37:0/2110191161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31440b1640 msgr2=0x7f31440b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f314a6ec640 1 --2- 172.21.15.37:0/2110191161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31440b1640 0x7f31440b1ad0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7f31340097c0 tx=0x7f313401c4a0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f314a6ec640 1 -- 172.21.15.37:0/2110191161 shutdown_connections 2024-04-25T09:04:09.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f314a6ec640 1 --2- 172.21.15.37:0/2110191161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31440b1640 0x7f31440b1ad0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f314a6ec640 1 --2- 172.21.15.37:0/2110191161 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31441441e0 0x7f31440ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f314a6ec640 1 -- 172.21.15.37:0/2110191161 >> 172.21.15.37:0/2110191161 conn(0x7f31440b2180 msgr2=0x7f31440b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f314a6ec640 1 -- 172.21.15.37:0/2110191161 shutdown_connections 2024-04-25T09:04:09.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f314a6ec640 1 -- 172.21.15.37:0/2110191161 wait complete. 2024-04-25T09:04:09.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f314a6ec640 1 Processor -- start 2024-04-25T09:04:09.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.557+0000 7f314a6ec640 1 -- start start 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f314a6ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31440b1640 0x7f3144150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f314a6ec640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3144143e00 0x7f314415c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f314a6ec640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31441441e0 0x7f31440ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f314a6ec640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31440b0f80 con 0x7f31440b1640 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f314a6ec640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31440b2fd0 con 0x7f3144143e00 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f314a6ec640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31440b1100 con 0x7f31441441e0 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3143fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31440b1640 0x7f3144150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3143fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31440b1640 0x7f3144150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42612/0 (socket says 172.21.15.37:42612) 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3143fff640 1 -- 172.21.15.37:0/3841542640 learned_addr learned my addr 172.21.15.37:0/3841542640 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3143fff640 1 -- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31441441e0 msgr2=0x7f31440ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f31437fe640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3144143e00 0x7f314415c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3143fff640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31441441e0 0x7f31440ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3143fff640 1 -- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3144143e00 msgr2=0x7f314415c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3143fff640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3144143e00 0x7f314415c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3143fff640 1 -- 172.21.15.37:0/3841542640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31440cb4d0 con 0x7f31440b1640 2024-04-25T09:04:09.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3143fff640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31440b1640 0x7f3144150d40 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f313c00cb10 tx=0x7f313c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3140ff9640 1 -- 172.21.15.37:0/3841542640 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c017070 con 0x7f31440b1640 2024-04-25T09:04:09.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3140ff9640 1 -- 172.21.15.37:0/3841542640 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313c00ac70 con 0x7f31440b1640 2024-04-25T09:04:09.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f314a6ec640 1 -- 172.21.15.37:0/3841542640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31440cb7b0 con 0x7f31440b1640 2024-04-25T09:04:09.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3140ff9640 1 -- 172.21.15.37:0/3841542640 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313c012620 con 0x7f31440b1640 2024-04-25T09:04:09.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f314a6ec640 1 -- 172.21.15.37:0/3841542640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31440cbd70 con 0x7f31440b1640 2024-04-25T09:04:09.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f314a6ec640 1 -- 172.21.15.37:0/3841542640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3144068350 con 0x7f31440b1640 2024-04-25T09:04:09.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3140ff9640 1 -- 172.21.15.37:0/3841542640 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f313c01a070 con 0x7f31440b1640 2024-04-25T09:04:09.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3140ff9640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3124066f60 0x7f3124069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f3140ff9640 1 -- 172.21.15.37:0/3841542640 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f313c0850c0 con 0x7f31440b1640 2024-04-25T09:04:09.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.561+0000 7f31437fe640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3124066f60 0x7f3124069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.565+0000 7f31437fe640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3124066f60 0x7f3124069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f314415c2a0 tx=0x7f312c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.565+0000 7f3140ff9640 1 -- 172.21.15.37:0/3841542640 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f313c055e50 con 0x7f31440b1640 2024-04-25T09:04:09.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.741+0000 7f314a6ec640 1 -- 172.21.15.37:0/3841542640 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f314415e630 con 0x7f31440b1640 2024-04-25T09:04:09.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.741+0000 7f3140ff9640 1 -- 172.21.15.37:0/3841542640 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v126) v1 ==== 80+0+122 (secure 0 0 0) 0x7f313c055750 con 0x7f31440b1640 2024-04-25T09:04:09.731 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/2748919477 2024-04-26T09:03:58.978336+0000 2024-04-25T09:04:09.731 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6834/2748919477 2024-04-26T09:03:58.978336+0000 2024-04-25T09:04:09.731 INFO:teuthology.orchestra.run.smithi037.stderr:listed 2 entries 2024-04-25T09:04:09.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 -- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3124066f60 msgr2=0x7f3124069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3124066f60 0x7f3124069420 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f314415c2a0 tx=0x7f312c008040 comp rx=0 tx=0).stop 2024-04-25T09:04:09.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 -- 172.21.15.37:0/3841542640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31440b1640 msgr2=0x7f3144150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31440b1640 0x7f3144150d40 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f313c00cb10 tx=0x7f313c00a060 comp rx=0 tx=0).stop 2024-04-25T09:04:09.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 -- 172.21.15.37:0/3841542640 shutdown_connections 2024-04-25T09:04:09.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3124066f60 0x7f3124069420 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f31441441e0 0x7f31440ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3144143e00 0x7f314415c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 --2- 172.21.15.37:0/3841542640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31440b1640 0x7f3144150d40 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 -- 172.21.15.37:0/3841542640 >> 172.21.15.37:0/3841542640 conn(0x7f31440b2180 msgr2=0x7f3144073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 -- 172.21.15.37:0/3841542640 shutdown_connections 2024-04-25T09:04:09.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.745+0000 7f314a6ec640 1 -- 172.21.15.37:0/3841542640 wait complete. 2024-04-25T09:04:09.750 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:04:09.751 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:09.756 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:09.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b7906640 1 Processor -- start 2024-04-25T09:04:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b7906640 1 -- start start 2024-04-25T09:04:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b7906640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd2b0143f50 0x7fd2b00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b7906640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fd2b0071e90 0x7fd2b015e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b7906640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b00b3010 con 0x7fd2b006e260 2024-04-25T09:04:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b7906640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b00b2dc0 con 0x7fd2b0143f50 2024-04-25T09:04:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b7906640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b00b3940 con 0x7fd2b0071e90 2024-04-25T09:04:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b56a2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd2b0143f50 0x7fd2b00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b56a2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd2b0143f50 0x7fd2b00b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42616/0 (socket says 172.21.15.37:42616) 2024-04-25T09:04:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b56a2640 1 -- 172.21.15.37:0/18643874 learned_addr learned my addr 172.21.15.37:0/18643874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b4ea1640 1 --2- 172.21.15.37:0/18643874 >> v2:172.21.15.203:3300/0 conn(0x7fd2b0071e90 0x7fd2b015e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b56a2640 1 -- 172.21.15.37:0/18643874 >> v2:172.21.15.203:3300/0 conn(0x7fd2b0071e90 msgr2=0x7fd2b015e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b56a2640 1 --2- 172.21.15.37:0/18643874 >> v2:172.21.15.203:3300/0 conn(0x7fd2b0071e90 0x7fd2b015e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b56a2640 1 -- 172.21.15.37:0/18643874 >> v1:172.21.15.203:6789/0 conn(0x7fd2b006e260 legacy=0x7fd2b015ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.937+0000 7fd2b56a2640 1 -- 172.21.15.37:0/18643874 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00b32f0 con 0x7fd2b0143f50 2024-04-25T09:04:09.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b56a2640 1 --2- 172.21.15.37:0/18643874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b0143f50 0x7fd2b00b3dd0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009e50 tx=0x7fd2ac01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f20b9e35e95d9e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2a7fff640 1 -- 172.21.15.37:0/18643874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0a57e0 con 0x7fd2b0143f50 2024-04-25T09:04:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2a7fff640 1 -- 172.21.15.37:0/18643874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac0a5980 con 0x7fd2b0143f50 2024-04-25T09:04:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2a7fff640 1 -- 172.21.15.37:0/18643874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0a5d50 con 0x7fd2b0143f50 2024-04-25T09:04:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 -- 172.21.15.37:0/18643874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b0143f50 msgr2=0x7fd2b00b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 --2- 172.21.15.37:0/18643874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b0143f50 0x7fd2b00b3dd0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac009e50 tx=0x7fd2ac01c9c0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 -- 172.21.15.37:0/18643874 shutdown_connections 2024-04-25T09:04:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 --2- 172.21.15.37:0/18643874 >> v2:172.21.15.203:3300/0 conn(0x7fd2b0071e90 0x7fd2b015e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 --2- 172.21.15.37:0/18643874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b0143f50 0x7fd2b00b3dd0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 -- 172.21.15.37:0/18643874 >> 172.21.15.37:0/18643874 conn(0x7fd2b0066fa0 msgr2=0x7fd2b006ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 -- 172.21.15.37:0/18643874 shutdown_connections 2024-04-25T09:04:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 -- 172.21.15.37:0/18643874 wait complete. 2024-04-25T09:04:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 Processor -- start 2024-04-25T09:04:09.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 -- start start 2024-04-25T09:04:09.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b006e260 0x7fd2b013ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b0071e90 0x7fd2b0142340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2b0143f50 0x7fd2b0155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b0165570 con 0x7fd2b006e260 2024-04-25T09:04:09.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b01653f0 con 0x7fd2b0071e90 2024-04-25T09:04:09.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b7906640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b01656f0 con 0x7fd2b0143f50 2024-04-25T09:04:09.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b56a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b006e260 0x7fd2b013ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b56a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b006e260 0x7fd2b013ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42632/0 (socket says 172.21.15.37:42632) 2024-04-25T09:04:09.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b56a2640 1 -- 172.21.15.37:0/1724412204 learned_addr learned my addr 172.21.15.37:0/1724412204 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b56a2640 1 -- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2b0143f50 msgr2=0x7fd2b0155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b56a2640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2b0143f50 0x7fd2b0155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b56a2640 1 -- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b0071e90 msgr2=0x7fd2b0142340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b56a2640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b0071e90 0x7fd2b0142340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b56a2640 1 -- 172.21.15.37:0/1724412204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b0142880 con 0x7fd2b006e260 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.941+0000 7fd2b56a2640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b006e260 0x7fd2b013ff20 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0195d0 tx=0x7fd2ac019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.945+0000 7fd2a5ffb640 1 -- 172.21.15.37:0/1724412204 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0b8470 con 0x7fd2b006e260 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.945+0000 7fd2a5ffb640 1 -- 172.21.15.37:0/1724412204 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ac0b8630 con 0x7fd2b006e260 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.945+0000 7fd2a5ffb640 1 -- 172.21.15.37:0/1724412204 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0b8a20 con 0x7fd2b006e260 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.945+0000 7fd2b7906640 1 -- 172.21.15.37:0/1724412204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b0142b20 con 0x7fd2b006e260 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.945+0000 7fd2b7906640 1 -- 172.21.15.37:0/1724412204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b01430e0 con 0x7fd2b006e260 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.945+0000 7fd2b7906640 1 -- 172.21.15.37:0/1724412204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b01437a0 con 0x7fd2b006e260 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.945+0000 7fd2a5ffb640 1 -- 172.21.15.37:0/1724412204 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd2ac0c0020 con 0x7fd2b006e260 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.945+0000 7fd2a5ffb640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd294066f60 0x7fd294069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.945+0000 7fd2a5ffb640 1 -- 172.21.15.37:0/1724412204 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd2ac11fbe0 con 0x7fd2b006e260 2024-04-25T09:04:09.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.945+0000 7fd2b4ea1640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd294066f60 0x7fd294069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.945+0000 7fd2b4ea1640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd294066f60 0x7fd294069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00b23d0 tx=0x7fd2a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:09.949+0000 7fd2a5ffb640 1 -- 172.21.15.37:0/1724412204 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd2b01437a0 con 0x7fd2b006e260 2024-04-25T09:04:10.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.109+0000 7fd2b7906640 1 -- 172.21.15.37:0/1724412204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2b0067c40 con 0x7fd2b006e260 2024-04-25T09:04:10.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.113+0000 7fd2a5ffb640 1 -- 172.21.15.37:0/1724412204 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+17045 (secure 0 0 0) 0x7fd2ac0c7020 con 0x7fd2b006e260 2024-04-25T09:04:10.103 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:10.103 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":126,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:03:58.978382+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2748919477":"2024-04-26T09:03:58.978336+0000","172.21.15.37:6834/2748919477":"2024-04-26T09:03:58.978336+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:10.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 -- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd294066f60 msgr2=0x7fd294069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd294066f60 0x7fd294069420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fd2b00b23d0 tx=0x7fd2a0008040 comp rx=0 tx=0).stop 2024-04-25T09:04:10.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 -- 172.21.15.37:0/1724412204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b006e260 msgr2=0x7fd2b013ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b006e260 0x7fd2b013ff20 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0195d0 tx=0x7fd2ac019760 comp rx=0 tx=0).stop 2024-04-25T09:04:10.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 -- 172.21.15.37:0/1724412204 shutdown_connections 2024-04-25T09:04:10.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd294066f60 0x7fd294069420 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd2b0143f50 0x7fd2b0155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd2b0071e90 0x7fd2b0142340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 --2- 172.21.15.37:0/1724412204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2b006e260 0x7fd2b013ff20 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 -- 172.21.15.37:0/1724412204 >> 172.21.15.37:0/1724412204 conn(0x7fd2b0066fa0 msgr2=0x7fd2b015d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 -- 172.21.15.37:0/1724412204 shutdown_connections 2024-04-25T09:04:10.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.117+0000 7fd2b7906640 1 -- 172.21.15.37:0/1724412204 wait complete. 2024-04-25T09:04:10.122 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:30.763724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:31.488462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:10.123 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:10.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c45e1640 1 Processor -- start 2024-04-25T09:04:10.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c45e1640 1 -- start start 2024-04-25T09:04:10.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c45e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc0b19c0 0x7f27bc0cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c45e1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27bc15d4b0 con 0x7f27bc0cec10 2024-04-25T09:04:10.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c45e1640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27bc15d790 con 0x7f27bc15bc60 2024-04-25T09:04:10.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c45e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc15e0c0 con 0x7f27bc0b19c0 2024-04-25T09:04:10.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c1b7c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f27bc0cec10 0x7f27bc0cafa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48252/0 (socket says 172.21.15.37:48252) 2024-04-25T09:04:10.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c1b7c640 1 -- 172.21.15.37:0/3086359212 learned_addr learned my addr 172.21.15.37:0/3086359212 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:10.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c2b7e640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc0b19c0 0x7f27bc0cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388680589 0 0) 0x7f27bc15d4b0 con 0x7f27bc0cec10 2024-04-25T09:04:10.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27a8003530 con 0x7f27bc0cec10 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1350673294 0 0) 0x7f27a8003530 con 0x7f27bc0cec10 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc0b19c0 msgr2=0x7f27bc0cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc0b19c0 0x7f27bc0cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 >> v1:172.21.15.203:6789/0 conn(0x7f27bc15bc60 legacy=0x7f27bc0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc15da70 con 0x7f27bc0cec10 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f27ac0016a0 con 0x7f27bc0cec10 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 >> v1:172.21.15.37:6789/0 conn(0x7f27bc0cec10 legacy=0x7f27bc0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a80039f0 0x7f27a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a8004810 0x7f27a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc0b19c0 0x7f27a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc15e0c0 con 0x7f27a80039f0 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a800bb30 con 0x7f27a8004810 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a800df40 con 0x7f27bc0b19c0 2024-04-25T09:04:10.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c237d640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a80039f0 0x7f27a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c1b7c640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a8004810 0x7f27a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c2b7e640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc0b19c0 0x7f27a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c237d640 1 -- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc0b19c0 msgr2=0x7f27a8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c237d640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc0b19c0 0x7f27a8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c237d640 1 -- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a8004810 msgr2=0x7f27a8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c237d640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a8004810 0x7f27a8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c237d640 1 -- 172.21.15.37:0/3086359212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a8003530 con 0x7f27a80039f0 2024-04-25T09:04:10.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.265+0000 7f27c1b7c640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a8004810 0x7f27a8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:10.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c237d640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a80039f0 0x7f27a8003e10 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f27b8001780 tx=0x7f27b801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c127b1412557d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b80013a0 con 0x7f27a80039f0 2024-04-25T09:04:10.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b80052f0 con 0x7f27a80039f0 2024-04-25T09:04:10.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c137b640 1 -- 172.21.15.37:0/3086359212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8003840 con 0x7f27a80039f0 2024-04-25T09:04:10.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c45e1640 1 -- 172.21.15.37:0/3086359212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a80039f0 msgr2=0x7f27a8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c45e1640 1 --2- 172.21.15.37:0/3086359212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27a80039f0 0x7f27a8003e10 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f27b8001780 tx=0x7f27b801c420 comp rx=0 tx=0).stop 2024-04-25T09:04:10.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c2b7e640 1 -- 172.21.15.37:0/3086359212 reap_dead start 2024-04-25T09:04:10.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c45e1640 1 -- 172.21.15.37:0/3086359212 shutdown_connections 2024-04-25T09:04:10.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c45e1640 1 -- 172.21.15.37:0/3086359212 >> 172.21.15.37:0/3086359212 conn(0x7f27bc0b1f30 msgr2=0x7f27bc0b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c45e1640 1 -- 172.21.15.37:0/3086359212 shutdown_connections 2024-04-25T09:04:10.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c45e1640 1 -- 172.21.15.37:0/3086359212 wait complete. 2024-04-25T09:04:10.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c45e1640 1 Processor -- start 2024-04-25T09:04:10.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.269+0000 7f27c45e1640 1 -- start start 2024-04-25T09:04:10.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c45e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a80039f0 0x7f27bc0ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c45e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc14b060 0x7f27bc14a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c45e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc06e150 0x7f27bc06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c45e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc0cc250 con 0x7f27bc14b060 2024-04-25T09:04:10.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c45e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc0cc0d0 con 0x7f27a80039f0 2024-04-25T09:04:10.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c45e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27bc06ac90 con 0x7f27bc06e150 2024-04-25T09:04:10.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c1b7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc14b060 0x7f27bc14a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c1b7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc14b060 0x7f27bc14a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42662/0 (socket says 172.21.15.37:42662) 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c1b7c640 1 -- 172.21.15.37:0/1081540647 learned_addr learned my addr 172.21.15.37:0/1081540647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c237d640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a80039f0 0x7f27bc0ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c2b7e640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc06e150 0x7f27bc06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c1b7c640 1 -- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc06e150 msgr2=0x7f27bc06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c1b7c640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc06e150 0x7f27bc06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c1b7c640 1 -- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a80039f0 msgr2=0x7f27bc0ca1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c1b7c640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a80039f0 0x7f27bc0ca1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c1b7c640 1 -- 172.21.15.37:0/1081540647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27bc06eb10 con 0x7f27bc14b060 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c1b7c640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc14b060 0x7f27bc14a7c0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0011f0 tx=0x7f27ac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27b2ffd640 1 -- 172.21.15.37:0/1081540647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac019070 con 0x7f27bc14b060 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27b2ffd640 1 -- 172.21.15.37:0/1081540647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ac00de50 con 0x7f27bc14b060 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c45e1640 1 -- 172.21.15.37:0/1081540647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27bc0c7870 con 0x7f27bc14b060 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27b2ffd640 1 -- 172.21.15.37:0/1081540647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0047f0 con 0x7f27bc14b060 2024-04-25T09:04:10.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c45e1640 1 -- 172.21.15.37:0/1081540647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27bc162c30 con 0x7f27bc14b060 2024-04-25T09:04:10.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c45e1640 1 -- 172.21.15.37:0/1081540647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27bc068350 con 0x7f27bc14b060 2024-04-25T09:04:10.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27b2ffd640 1 -- 172.21.15.37:0/1081540647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f27ac004990 con 0x7f27bc14b060 2024-04-25T09:04:10.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27b2ffd640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27a0066f90 0x7f27a0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27b2ffd640 1 -- 172.21.15.37:0/1081540647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f27ac084be0 con 0x7f27bc14b060 2024-04-25T09:04:10.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.273+0000 7f27c237d640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27a0066f90 0x7f27a0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.277+0000 7f27c237d640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27a0066f90 0x7f27a0069450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f27b8001750 tx=0x7f27b80011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.277+0000 7f27b2ffd640 1 -- 172.21.15.37:0/1081540647 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f27ac055a10 con 0x7f27bc14b060 2024-04-25T09:04:10.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.441+0000 7f27c45e1640 1 -- 172.21.15.37:0/1081540647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27bc0722e0 con 0x7f27bc14b060 2024-04-25T09:04:10.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.441+0000 7f27b2ffd640 1 -- 172.21.15.37:0/1081540647 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v126) v1 ==== 93+0+10 (secure 0 0 0) 0x7f27ac055310 con 0x7f27bc14b060 2024-04-25T09:04:10.431 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:04:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 -- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27a0066f90 msgr2=0x7f27a0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27a0066f90 0x7f27a0069450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f27b8001750 tx=0x7f27b80011a0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 -- 172.21.15.37:0/1081540647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc14b060 msgr2=0x7f27bc14a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc14b060 0x7f27bc14a7c0 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0011f0 tx=0x7f27ac00dc00 comp rx=0 tx=0).stop 2024-04-25T09:04:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 -- 172.21.15.37:0/1081540647 shutdown_connections 2024-04-25T09:04:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f27a0066f90 0x7f27a0069450 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f27bc06e150 0x7f27bc06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc14b060 0x7f27bc14a7c0 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 --2- 172.21.15.37:0/1081540647 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f27a80039f0 0x7f27bc0ca1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 -- 172.21.15.37:0/1081540647 >> 172.21.15.37:0/1081540647 conn(0x7f27bc0b1f30 msgr2=0x7f27bc0cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 -- 172.21.15.37:0/1081540647 shutdown_connections 2024-04-25T09:04:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.445+0000 7f27c45e1640 1 -- 172.21.15.37:0/1081540647 wait complete. 2024-04-25T09:04:10.449 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:10.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.589+0000 7f8a320d0640 1 Processor -- start 2024-04-25T09:04:10.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.589+0000 7f8a320d0640 1 -- start start 2024-04-25T09:04:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a320d0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8a2c15c980 0x7f8a2c15fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a320d0640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2c0cda10 con 0x7f8a2c071110 2024-04-25T09:04:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a320d0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a2c0cdcf0 con 0x7f8a2c144e40 2024-04-25T09:04:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a320d0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a2c15f260 con 0x7f8a2c15c980 2024-04-25T09:04:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2affd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8a2c15c980 0x7f8a2c15fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2affd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8a2c15c980 0x7f8a2c15fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42676/0 (socket says 172.21.15.37:42676) 2024-04-25T09:04:10.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2bfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8a2c071110 0x7f8a2c15ce30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48260/0 (socket says 172.21.15.37:48260) 2024-04-25T09:04:10.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2affd640 1 -- 172.21.15.37:0/3857116360 learned_addr learned my addr 172.21.15.37:0/3857116360 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:10.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2affd640 1 -- 172.21.15.37:0/3857116360 >> v1:172.21.15.203:6789/0 conn(0x7f8a2c144e40 legacy=0x7f8a2c0cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2affd640 1 -- 172.21.15.37:0/3857116360 >> v1:172.21.15.37:6789/0 conn(0x7f8a2c071110 legacy=0x7f8a2c15ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2a7fc640 1 -- 172.21.15.37:0/3857116360 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880987092 0 0) 0x7f8a2c0cda10 con 0x7f8a2c071110 2024-04-25T09:04:10.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2affd640 1 -- 172.21.15.37:0/3857116360 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c15f4b0 con 0x7f8a2c15c980 2024-04-25T09:04:10.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2affd640 1 --2- 172.21.15.37:0/3857116360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2c15c980 0x7f8a2c15fcb0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009870 tx=0x7f8a2001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48705f69714fe1e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2a7fc640 1 -- 172.21.15.37:0/3857116360 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a200b0070 con 0x7f8a2c15c980 2024-04-25T09:04:10.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2a7fc640 1 -- 172.21.15.37:0/3857116360 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a200ab600 con 0x7f8a2c15c980 2024-04-25T09:04:10.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.593+0000 7f8a2a7fc640 1 -- 172.21.15.37:0/3857116360 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a200ab9d0 con 0x7f8a2c15c980 2024-04-25T09:04:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 -- 172.21.15.37:0/3857116360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2c15c980 msgr2=0x7f8a2c15fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 --2- 172.21.15.37:0/3857116360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2c15c980 0x7f8a2c15fcb0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009870 tx=0x7f8a2001c520 comp rx=0 tx=0).stop 2024-04-25T09:04:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 -- 172.21.15.37:0/3857116360 shutdown_connections 2024-04-25T09:04:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 --2- 172.21.15.37:0/3857116360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2c15c980 0x7f8a2c15fcb0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 -- 172.21.15.37:0/3857116360 >> 172.21.15.37:0/3857116360 conn(0x7f8a2c066fa0 msgr2=0x7f8a2c06ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 -- 172.21.15.37:0/3857116360 shutdown_connections 2024-04-25T09:04:10.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 -- 172.21.15.37:0/3857116360 wait complete. 2024-04-25T09:04:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 Processor -- start 2024-04-25T09:04:10.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 -- start start 2024-04-25T09:04:10.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a2c071110 0x7f8a2c13f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2c144e40 0x7f8a2c0a9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a2c15c980 0x7f8a2c1f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c1601f0 con 0x7f8a2c144e40 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c15f820 con 0x7f8a2c15c980 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a2c169e20 con 0x7f8a2c071110 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a2affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2c144e40 0x7f8a2c0a9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a2affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2c144e40 0x7f8a2c0a9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42682/0 (socket says 172.21.15.37:42682) 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a2affd640 1 -- 172.21.15.37:0/3790547664 learned_addr learned my addr 172.21.15.37:0/3790547664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a2bfff640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a2c15c980 0x7f8a2c1f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a2affd640 1 -- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a2c071110 msgr2=0x7f8a2c13f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a2affd640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a2c071110 0x7f8a2c13f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a2affd640 1 -- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a2c15c980 msgr2=0x7f8a2c1f84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a2affd640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a2c15c980 0x7f8a2c1f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a2affd640 1 -- 172.21.15.37:0/3790547664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2c13fbe0 con 0x7f8a2c144e40 2024-04-25T09:04:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a2affd640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2c144e40 0x7f8a2c0a9220 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f8a2001abd0 tx=0x7f8a2001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a13fff640 1 -- 172.21.15.37:0/3790547664 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a200ba020 con 0x7f8a2c144e40 2024-04-25T09:04:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a13fff640 1 -- 172.21.15.37:0/3790547664 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a200b6640 con 0x7f8a2c144e40 2024-04-25T09:04:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 -- 172.21.15.37:0/3790547664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a2c13fec0 con 0x7f8a2c144e40 2024-04-25T09:04:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a13fff640 1 -- 172.21.15.37:0/3790547664 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a200b0040 con 0x7f8a2c144e40 2024-04-25T09:04:10.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.597+0000 7f8a320d0640 1 -- 172.21.15.37:0/3790547664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a2c140480 con 0x7f8a2c144e40 2024-04-25T09:04:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.601+0000 7f8a320d0640 1 -- 172.21.15.37:0/3790547664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2c067c40 con 0x7f8a2c144e40 2024-04-25T09:04:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.601+0000 7f8a13fff640 1 -- 172.21.15.37:0/3790547664 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8a200bd020 con 0x7f8a2c144e40 2024-04-25T09:04:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.601+0000 7f8a13fff640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a18066f60 0x7f8a18069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.601+0000 7f8a13fff640 1 -- 172.21.15.37:0/3790547664 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8a2011f4b0 con 0x7f8a2c144e40 2024-04-25T09:04:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.601+0000 7f8a2b7fe640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a18066f60 0x7f8a18069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.601+0000 7f8a2b7fe640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a18066f60 0x7f8a18069420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c003cf0 tx=0x7f8a1c0079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.605+0000 7f8a13fff640 1 -- 172.21.15.37:0/3790547664 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a200c4030 con 0x7f8a2c144e40 2024-04-25T09:04:10.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 -- 172.21.15.37:0/3790547664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8a2c1673e0 con 0x7f8a2c144e40 2024-04-25T09:04:10.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a13fff640 1 -- 172.21.15.37:0/3790547664 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v126) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8a20016110 con 0x7f8a2c144e40 2024-04-25T09:04:10.753 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 -- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a18066f60 msgr2=0x7f8a18069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a18066f60 0x7f8a18069420 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c003cf0 tx=0x7f8a1c0079c0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 -- 172.21.15.37:0/3790547664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2c144e40 msgr2=0x7f8a2c0a9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2c144e40 0x7f8a2c0a9220 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f8a2001abd0 tx=0x7f8a2001ad60 comp rx=0 tx=0).stop 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 -- 172.21.15.37:0/3790547664 shutdown_connections 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a18066f60 0x7f8a18069420 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a2c15c980 0x7f8a2c1f84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2c144e40 0x7f8a2c0a9220 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 --2- 172.21.15.37:0/3790547664 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a2c071110 0x7f8a2c13f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 -- 172.21.15.37:0/3790547664 >> 172.21.15.37:0/3790547664 conn(0x7f8a2c066fa0 msgr2=0x7f8a2c159ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 -- 172.21.15.37:0/3790547664 shutdown_connections 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.765+0000 7f8a320d0640 1 -- 172.21.15.37:0/3790547664 wait complete. 2024-04-25T09:04:10.774 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:10.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f4424fc7640 1 Processor -- start 2024-04-25T09:04:10.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f4424fc7640 1 -- start start 2024-04-25T09:04:10.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f4424fc7640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44200b4430 0x7f44200b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f4424fc7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200cce90 0x7f44200cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f4424fc7640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f442009ab90 con 0x7f44200b1f20 2024-04-25T09:04:10.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f4424fc7640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44200b28f0 con 0x7f44200b4430 2024-04-25T09:04:10.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f4424fc7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44200cc2f0 con 0x7f44200cce90 2024-04-25T09:04:10.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f441dd9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44200b4430 0x7f44200b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f441ed9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200cce90 0x7f44200cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f441dd9b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f44200b4430 0x7f44200b23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:48492/0 (socket says 172.21.15.37:48492) 2024-04-25T09:04:10.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f441dd9b640 1 -- 172.21.15.37:0/2299277017 learned_addr learned my addr 172.21.15.37:0/2299277017 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:10.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f441ed9d640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200cce90 0x7f44200cf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:42092/0 (socket says 172.21.15.37:42092) 2024-04-25T09:04:10.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f441e59c640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f44200b1f20 0x7f44200cc780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:50946/0 (socket says 172.21.15.37:50946) 2024-04-25T09:04:10.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f441dd9b640 1 -- 172.21.15.37:0/2299277017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200cce90 msgr2=0x7f44200cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f441dd9b640 1 --2- 172.21.15.37:0/2299277017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200cce90 0x7f44200cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f441dd9b640 1 -- 172.21.15.37:0/2299277017 >> v1:172.21.15.203:6789/0 conn(0x7f44200b1f20 legacy=0x7f44200cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f441dd9b640 1 -- 172.21.15.37:0/2299277017 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200b48c0 con 0x7f44200b4430 2024-04-25T09:04:10.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.905+0000 7f441ed9d640 1 --2- 172.21.15.37:0/2299277017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200cce90 0x7f44200cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:10.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f441dd9b640 1 --2- 172.21.15.37:0/2299277017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44200b4430 0x7f44200b23b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4408009e50 tx=0x7f440801c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7c2767280d69476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f441d59a640 1 -- 172.21.15.37:0/2299277017 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44080ac070 con 0x7f44200b4430 2024-04-25T09:04:10.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f441d59a640 1 -- 172.21.15.37:0/2299277017 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44080a8410 con 0x7f44200b4430 2024-04-25T09:04:10.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f441d59a640 1 -- 172.21.15.37:0/2299277017 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44080a87e0 con 0x7f44200b4430 2024-04-25T09:04:10.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f4424fc7640 1 -- 172.21.15.37:0/2299277017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44200b4430 msgr2=0x7f44200b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f4424fc7640 1 --2- 172.21.15.37:0/2299277017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44200b4430 0x7f44200b23b0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4408009e50 tx=0x7f440801c6e0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f4424fc7640 1 -- 172.21.15.37:0/2299277017 shutdown_connections 2024-04-25T09:04:10.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f4424fc7640 1 --2- 172.21.15.37:0/2299277017 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200cce90 0x7f44200cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f4424fc7640 1 --2- 172.21.15.37:0/2299277017 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44200b4430 0x7f44200b23b0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f4424fc7640 1 -- 172.21.15.37:0/2299277017 >> 172.21.15.37:0/2299277017 conn(0x7f44200d15e0 msgr2=0x7f44200d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f4424fc7640 1 -- 172.21.15.37:0/2299277017 shutdown_connections 2024-04-25T09:04:10.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.909+0000 7f4424fc7640 1 -- 172.21.15.37:0/2299277017 wait complete. 2024-04-25T09:04:10.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4424fc7640 1 Processor -- start 2024-04-25T09:04:10.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4424fc7640 1 -- start start 2024-04-25T09:04:10.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4424fc7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44200b1f20 0x7f44201f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4424fc7640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200b4430 0x7f44201519e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4424fc7640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44200cce90 0x7f4420152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4424fc7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200cd420 con 0x7f44200b1f20 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4424fc7640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200cd2a0 con 0x7f44200cce90 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4424fc7640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4420168790 con 0x7f44200b4430 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f441e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44200b1f20 0x7f44201f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f441e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44200b1f20 0x7f44201f4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42696/0 (socket says 172.21.15.37:42696) 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f441e59c640 1 -- 172.21.15.37:0/2054460330 learned_addr learned my addr 172.21.15.37:0/2054460330 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f441dd9b640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200b4430 0x7f44201519e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f441ed9d640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44200cce90 0x7f4420152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f441e59c640 1 -- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200b4430 msgr2=0x7f44201519e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f441e59c640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200b4430 0x7f44201519e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f441e59c640 1 -- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44200cce90 msgr2=0x7f4420152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f441e59c640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44200cce90 0x7f4420152a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f441e59c640 1 -- 172.21.15.37:0/2054460330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200c9aa0 con 0x7f44200b1f20 2024-04-25T09:04:10.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f441e59c640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44200b1f20 0x7f44201f4be0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f441000c370 tx=0x7f441000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4416ffd640 1 -- 172.21.15.37:0/2054460330 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44100074e0 con 0x7f44200b1f20 2024-04-25T09:04:10.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4416ffd640 1 -- 172.21.15.37:0/2054460330 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4410007680 con 0x7f44200b1f20 2024-04-25T09:04:10.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4424fc7640 1 -- 172.21.15.37:0/2054460330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44200c9d80 con 0x7f44200b1f20 2024-04-25T09:04:10.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4416ffd640 1 -- 172.21.15.37:0/2054460330 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4410004a30 con 0x7f44200b1f20 2024-04-25T09:04:10.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4424fc7640 1 -- 172.21.15.37:0/2054460330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44200ca340 con 0x7f44200b1f20 2024-04-25T09:04:10.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4424fc7640 1 -- 172.21.15.37:0/2054460330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4420068350 con 0x7f44200b1f20 2024-04-25T09:04:10.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4416ffd640 1 -- 172.21.15.37:0/2054460330 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f441001a020 con 0x7f44200b1f20 2024-04-25T09:04:10.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4416ffd640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4400066f60 0x7f4400069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.913+0000 7f4416ffd640 1 -- 172.21.15.37:0/2054460330 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f44100849d0 con 0x7f44200b1f20 2024-04-25T09:04:10.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.917+0000 7f441dd9b640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4400066f60 0x7f4400069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.917+0000 7f441dd9b640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4400066f60 0x7f4400069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f44080ae000 tx=0x7f44080a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:10.921+0000 7f4416ffd640 1 -- 172.21.15.37:0/2054460330 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4410055800 con 0x7f44200b1f20 2024-04-25T09:04:11.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 -- 172.21.15.37:0/2054460330 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f44200d01c0 con 0x7f44200b1f20 2024-04-25T09:04:11.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4416ffd640 1 -- 172.21.15.37:0/2054460330 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v126) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4410055100 con 0x7f44200b1f20 2024-04-25T09:04:11.062 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:04:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 -- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4400066f60 msgr2=0x7f4400069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4400066f60 0x7f4400069420 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f44080ae000 tx=0x7f44080a7000 comp rx=0 tx=0).stop 2024-04-25T09:04:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 -- 172.21.15.37:0/2054460330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44200b1f20 msgr2=0x7f44201f4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44200b1f20 0x7f44201f4be0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f441000c370 tx=0x7f441000c830 comp rx=0 tx=0).stop 2024-04-25T09:04:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 -- 172.21.15.37:0/2054460330 shutdown_connections 2024-04-25T09:04:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4400066f60 0x7f4400069420 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f44200cce90 0x7f4420152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f44200b4430 0x7f44201519e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 --2- 172.21.15.37:0/2054460330 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44200b1f20 0x7f44201f4be0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 -- 172.21.15.37:0/2054460330 >> 172.21.15.37:0/2054460330 conn(0x7f44200d15e0 msgr2=0x7f44201557b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 -- 172.21.15.37:0/2054460330 shutdown_connections 2024-04-25T09:04:11.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.073+0000 7f4424fc7640 1 -- 172.21.15.37:0/2054460330 wait complete. 2024-04-25T09:04:11.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6bf2875640 1 Processor -- start 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6bf2875640 1 -- start start 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6bf2875640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6bec0ceeb0 0x7f6bec06cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6bf2875640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec15f050 con 0x7f6bec0d32d0 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6bf2875640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec06c410 con 0x7f6bec15e2e0 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6bf2875640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bec0b6ae0 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6bebfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6bec0d32d0 0x7f6bec0ce7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48272/0 (socket says 172.21.15.37:48272) 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6beb7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6bec0ceeb0 0x7f6bec06cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6bebfff640 1 -- 172.21.15.37:0/1746062571 learned_addr learned my addr 172.21.15.37:0/1746062571 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6beb7fe640 1 -- 172.21.15.37:0/1746062571 >> v1:172.21.15.203:6789/0 conn(0x7f6bec15e2e0 legacy=0x7f6bec15e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6beb7fe640 1 -- 172.21.15.37:0/1746062571 >> v1:172.21.15.37:6789/0 conn(0x7f6bec0d32d0 legacy=0x7f6bec0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6beaffd640 1 -- 172.21.15.37:0/1746062571 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256739238 0 0) 0x7f6bec15f050 con 0x7f6bec0d32d0 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.205+0000 7f6beb7fe640 1 -- 172.21.15.37:0/1746062571 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec06c6f0 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6beb7fe640 1 --2- 172.21.15.37:0/1746062571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bec0ceeb0 0x7f6bec06cef0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40097a0 tx=0x7f6bd401ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65ed946c09cebc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6beaffd640 1 -- 172.21.15.37:0/1746062571 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd40b5070 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6beaffd640 1 -- 172.21.15.37:0/1746062571 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd40a8930 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6beaffd640 1 -- 172.21.15.37:0/1746062571 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd40a8d00 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 -- 172.21.15.37:0/1746062571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bec0ceeb0 msgr2=0x7f6bec06cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 --2- 172.21.15.37:0/1746062571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bec0ceeb0 0x7f6bec06cef0 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40097a0 tx=0x7f6bd401ec30 comp rx=0 tx=0).stop 2024-04-25T09:04:11.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 -- 172.21.15.37:0/1746062571 shutdown_connections 2024-04-25T09:04:11.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 --2- 172.21.15.37:0/1746062571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bec0ceeb0 0x7f6bec06cef0 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 -- 172.21.15.37:0/1746062571 >> 172.21.15.37:0/1746062571 conn(0x7f6bec0d39f0 msgr2=0x7f6bec0d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 -- 172.21.15.37:0/1746062571 shutdown_connections 2024-04-25T09:04:11.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 -- 172.21.15.37:0/1746062571 wait complete. 2024-04-25T09:04:11.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 Processor -- start 2024-04-25T09:04:11.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 -- start start 2024-04-25T09:04:11.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bec0ceeb0 0x7f6bec1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec0d32d0 0x7f6bec1f4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bec15e2e0 0x7f6bec0cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0d36b0 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec06c1c0 con 0x7f6bec0d32d0 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0d1bc0 con 0x7f6bec15e2e0 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bec0ceeb0 0x7f6bec1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bec0ceeb0 0x7f6bec1572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42708/0 (socket says 172.21.15.37:42708) 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bebfff640 1 -- 172.21.15.37:0/2623118073 learned_addr learned my addr 172.21.15.37:0/2623118073 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bebfff640 1 -- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bec15e2e0 msgr2=0x7f6bec0cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bebfff640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bec15e2e0 0x7f6bec0cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bebfff640 1 -- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec0d32d0 msgr2=0x7f6bec1f4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bebfff640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec0d32d0 0x7f6bec1f4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bebfff640 1 -- 172.21.15.37:0/2623118073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec1f4660 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bebfff640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bec0ceeb0 0x7f6bec1572d0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f6be4001570 tx=0x7f6be400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6be8ff9640 1 -- 172.21.15.37:0/2623118073 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4018070 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6be8ff9640 1 -- 172.21.15.37:0/2623118073 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be400a640 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6be8ff9640 1 -- 172.21.15.37:0/2623118073 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be400aa30 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.209+0000 7f6bf2875640 1 -- 172.21.15.37:0/2623118073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec156230 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.213+0000 7f6bf2875640 1 -- 172.21.15.37:0/2623118073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec1567f0 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.213+0000 7f6bf2875640 1 -- 172.21.15.37:0/2623118073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec067c40 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.213+0000 7f6be8ff9640 1 -- 172.21.15.37:0/2623118073 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6be400abd0 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.213+0000 7f6be8ff9640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bcc066f60 0x7f6bcc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.213+0000 7f6be8ff9640 1 -- 172.21.15.37:0/2623118073 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f6be40848c0 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.213+0000 7f6beb7fe640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bcc066f60 0x7f6bcc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.213+0000 7f6beb7fe640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bcc066f60 0x7f6bcc069420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40b2000 tx=0x7f6bd4004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.217+0000 7f6be8ff9640 1 -- 172.21.15.37:0/2623118073 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6bec1f4660 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.389+0000 7f6bf2875640 1 -- 172.21.15.37:0/2623118073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6bec06d020 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.389+0000 7f6be8ff9640 1 -- 172.21.15.37:0/2623118073 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v126) v1 ==== 80+0+122 (secure 0 0 0) 0x7f6be4055780 con 0x7f6bec0ceeb0 2024-04-25T09:04:11.379 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/2748919477 2024-04-26T09:03:58.978336+0000 2024-04-25T09:04:11.379 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6834/2748919477 2024-04-26T09:03:58.978336+0000 2024-04-25T09:04:11.380 INFO:teuthology.orchestra.run.smithi037.stderr:listed 2 entries 2024-04-25T09:04:11.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 -- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bcc066f60 msgr2=0x7f6bcc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bcc066f60 0x7f6bcc069420 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40b2000 tx=0x7f6bd4004740 comp rx=0 tx=0).stop 2024-04-25T09:04:11.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 -- 172.21.15.37:0/2623118073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bec0ceeb0 msgr2=0x7f6bec1572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bec0ceeb0 0x7f6bec1572d0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f6be4001570 tx=0x7f6be400cd20 comp rx=0 tx=0).stop 2024-04-25T09:04:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 -- 172.21.15.37:0/2623118073 shutdown_connections 2024-04-25T09:04:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6bcc066f60 0x7f6bcc069420 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6bec15e2e0 0x7f6bec0cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6bec0d32d0 0x7f6bec1f4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 --2- 172.21.15.37:0/2623118073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6bec0ceeb0 0x7f6bec1572d0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 -- 172.21.15.37:0/2623118073 >> 172.21.15.37:0/2623118073 conn(0x7f6bec0d39f0 msgr2=0x7f6bec15f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 -- 172.21.15.37:0/2623118073 shutdown_connections 2024-04-25T09:04:11.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.393+0000 7f6bf2875640 1 -- 172.21.15.37:0/2623118073 wait complete. 2024-04-25T09:04:11.398 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T09:04:11.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:11.398 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:11.409 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[55444]: fuse finished with error 0 and tester_r 0 2024-04-25T09:04:11.410 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:11.495 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:04:11.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:11.495 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:04:11.501 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:11.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f38968f9640 1 Processor -- start 2024-04-25T09:04:11.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f38968f9640 1 -- start start 2024-04-25T09:04:11.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f38968f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c4810 0x7f38900b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f38968f9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f38900c4bf0 0x7f38901621f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f38968f9640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38901628d0 con 0x7f389006bf50 2024-04-25T09:04:11.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f38968f9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3890099a20 con 0x7f38900c4bf0 2024-04-25T09:04:11.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f38968f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3890162bb0 con 0x7f38900c4810 2024-04-25T09:04:11.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f3894e96640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f389006bf50 0x7f3890099eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48278/0 (socket says 172.21.15.37:48278) 2024-04-25T09:04:11.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f3894e96640 1 -- 172.21.15.37:0/4049291346 learned_addr learned my addr 172.21.15.37:0/4049291346 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:11.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388f7fe640 1 --2- 172.21.15.37:0/4049291346 >> v2:172.21.15.203:3300/0 conn(0x7f38900c4bf0 0x7f38901621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534841275 0 0) 0x7f38901628d0 con 0x7f389006bf50 2024-04-25T09:04:11.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388ffff640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c4810 0x7f38900b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3878003530 con 0x7f389006bf50 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3737563712 0 0) 0x7f3878003530 con 0x7f389006bf50 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c4810 msgr2=0x7f38900b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c4810 0x7f38900b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 >> v2:172.21.15.203:3300/0 conn(0x7f38900c4bf0 msgr2=0x7f38901621f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 --2- 172.21.15.37:0/4049291346 >> v2:172.21.15.203:3300/0 conn(0x7f38900c4bf0 0x7f38901621f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3890162e90 con 0x7f389006bf50 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f3888001a00 con 0x7f389006bf50 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388f7fe640 1 --2- 172.21.15.37:0/4049291346 >> v2:172.21.15.203:3300/0 conn(0x7f38900c4bf0 0x7f38901621f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 >> v1:172.21.15.37:6789/0 conn(0x7f389006bf50 legacy=0x7f3890099eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c4810 0x7f38780039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3878003f40 0x7f3878004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3878004d50 0x7f38780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3890099a20 con 0x7f3878003f40 2024-04-25T09:04:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3890162bb0 con 0x7f3878004d50 2024-04-25T09:04:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387800bb60 con 0x7f38900c4810 2024-04-25T09:04:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388f7fe640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3878003f40 0x7f3878004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388f7fe640 1 -- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c4810 msgr2=0x7f38780039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388ffff640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c4810 0x7f38780039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f3894e96640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3878004d50 0x7f38780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388f7fe640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c4810 0x7f38780039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388f7fe640 1 -- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3878004d50 msgr2=0x7f38780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388f7fe640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3878004d50 0x7f38780051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388f7fe640 1 -- 172.21.15.37:0/4049291346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3878003530 con 0x7f3878003f40 2024-04-25T09:04:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.681+0000 7f388ffff640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900c4810 0x7f38780039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:11.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f388f7fe640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3878003f40 0x7f3878004810 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f38800098d0 tx=0x7f388001c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acd67b01b700a23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800ac070 con 0x7f3878003f40 2024-04-25T09:04:11.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38800a6630 con 0x7f3878003f40 2024-04-25T09:04:11.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f388effd640 1 -- 172.21.15.37:0/4049291346 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388000f040 con 0x7f3878003f40 2024-04-25T09:04:11.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 -- 172.21.15.37:0/4049291346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3878003f40 msgr2=0x7f3878004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 --2- 172.21.15.37:0/4049291346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3878003f40 0x7f3878004810 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f38800098d0 tx=0x7f388001c620 comp rx=0 tx=0).stop 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f3894e96640 1 -- 172.21.15.37:0/4049291346 reap_dead start 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 -- 172.21.15.37:0/4049291346 shutdown_connections 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 -- 172.21.15.37:0/4049291346 >> 172.21.15.37:0/4049291346 conn(0x7f38900d15e0 msgr2=0x7f38900d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 -- 172.21.15.37:0/4049291346 shutdown_connections 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 -- 172.21.15.37:0/4049291346 wait complete. 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 Processor -- start 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 -- start start 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3878003f40 0x7f389015dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38900b4bc0 0x7f38900b69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900b5b50 0x7f38900b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389016cfb0 con 0x7f38900b4bc0 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f389016ce30 con 0x7f3878003f40 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.685+0000 7f38968f9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f389016d130 con 0x7f38900b5b50 2024-04-25T09:04:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38900b4bc0 0x7f38900b69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38900b4bc0 0x7f38900b69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42716/0 (socket says 172.21.15.37:42716) 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388f7fe640 1 -- 172.21.15.37:0/3912307061 learned_addr learned my addr 172.21.15.37:0/3912307061 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388f7fe640 1 -- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900b5b50 msgr2=0x7f38900b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388f7fe640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900b5b50 0x7f38900b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388f7fe640 1 -- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3878003f40 msgr2=0x7f389015dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388f7fe640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3878003f40 0x7f389015dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388f7fe640 1 -- 172.21.15.37:0/3912307061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38900b05f0 con 0x7f38900b4bc0 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388f7fe640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38900b4bc0 0x7f38900b69f0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f38800097f0 tx=0x7f38800193c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388cff9640 1 -- 172.21.15.37:0/3912307061 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800ac040 con 0x7f38900b4bc0 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388cff9640 1 -- 172.21.15.37:0/3912307061 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388000f000 con 0x7f38900b4bc0 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f38968f9640 1 -- 172.21.15.37:0/3912307061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38900c6890 con 0x7f38900b4bc0 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388cff9640 1 -- 172.21.15.37:0/3912307061 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38800b1880 con 0x7f38900b4bc0 2024-04-25T09:04:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f38968f9640 1 -- 172.21.15.37:0/3912307061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38900b53b0 con 0x7f38900b4bc0 2024-04-25T09:04:11.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f38968f9640 1 -- 172.21.15.37:0/3912307061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3890068350 con 0x7f38900b4bc0 2024-04-25T09:04:11.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388cff9640 1 -- 172.21.15.37:0/3912307061 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f38800af070 con 0x7f38900b4bc0 2024-04-25T09:04:11.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388cff9640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f387c066f60 0x7f387c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388cff9640 1 -- 172.21.15.37:0/3912307061 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f388011f220 con 0x7f38900b4bc0 2024-04-25T09:04:11.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388ffff640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f387c066f60 0x7f387c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.689+0000 7f388ffff640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f387c066f60 0x7f387c069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3884009930 tx=0x7f3884008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.693+0000 7f388cff9640 1 -- 172.21.15.37:0/3912307061 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3880016360 con 0x7f38900b4bc0 2024-04-25T09:04:11.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.885+0000 7f38968f9640 1 -- 172.21.15.37:0/3912307061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38900d2310 con 0x7f38900b4bc0 2024-04-25T09:04:11.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.885+0000 7f388cff9640 1 -- 172.21.15.37:0/3912307061 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f38800f0170 con 0x7f38900b4bc0 2024-04-25T09:04:11.874 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:11.874 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":103,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":103,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:04:02.008068+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":126,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9762},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"a","rank":0,"incarnation":100,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/2159545482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2159545482},{"type":"v1","addr":"172.21.15.37:6835","nonce":2159545482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:04:11.875 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 103 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 -- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f387c066f60 msgr2=0x7f387c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f387c066f60 0x7f387c069420 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3884009930 tx=0x7f3884008040 comp rx=0 tx=0).stop 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 -- 172.21.15.37:0/3912307061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38900b4bc0 msgr2=0x7f38900b69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38900b4bc0 0x7f38900b69f0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f38800097f0 tx=0x7f38800193c0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 -- 172.21.15.37:0/3912307061 shutdown_connections 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f387c066f60 0x7f387c069420 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f38900b5b50 0x7f38900b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38900b4bc0 0x7f38900b69f0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 --2- 172.21.15.37:0/3912307061 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3878003f40 0x7f389015dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 -- 172.21.15.37:0/3912307061 >> 172.21.15.37:0/3912307061 conn(0x7f38900d15e0 msgr2=0x7f389016bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 -- 172.21.15.37:0/3912307061 shutdown_connections 2024-04-25T09:04:11.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:11.889+0000 7f38968f9640 1 -- 172.21.15.37:0/3912307061 wait complete. 2024-04-25T09:04:11.892 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json tag path /parent mytag123 2024-04-25T09:04:12.036 INFO:teuthology.orchestra.run.smithi037.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-04-25T09:04:12.036 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-04-25T09:04:12.037 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-04-25T09:04:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.085+0000 7f667bbe4080 1 Processor -- start 2024-04-25T09:04:12.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.085+0000 7f667bbe4080 1 -- start start 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f86042bca0 0x55f86042f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f860504230 0x55f8604717b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f860504610 0x55f860500ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f860476c00 con 0x55f860504610 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8603d39f0 con 0x55f860504230 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f860476d80 con 0x55f86042bca0 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679f03640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f860504610 0x55f860500ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679f03640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f860504610 0x55f860500ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42728/0 (socket says 172.21.15.37:42728) 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679f03640 1 -- 172.21.15.37:0/977092326 learned_addr learned my addr 172.21.15.37:0/977092326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6678f01640 1 --2- 172.21.15.37:0/977092326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f860504230 0x55f8604717b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679702640 1 --2- 172.21.15.37:0/977092326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f86042bca0 0x55f86042f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679f03640 1 -- 172.21.15.37:0/977092326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f86042bca0 msgr2=0x55f86042f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679f03640 1 --2- 172.21.15.37:0/977092326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f86042bca0 0x55f86042f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679f03640 1 -- 172.21.15.37:0/977092326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f860504230 msgr2=0x55f8604717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679f03640 1 --2- 172.21.15.37:0/977092326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f860504230 0x55f8604717b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679f03640 1 -- 172.21.15.37:0/977092326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f860472180 con 0x55f860504610 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679f03640 1 --2- 172.21.15.37:0/977092326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f860504610 0x55f860500ab0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f667400a590 tx=0x7f6674003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fad9b6a26b67c5eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f66737fe640 1 -- 172.21.15.37:0/977092326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66740af040 con 0x55f860504610 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f66737fe640 1 -- 172.21.15.37:0/977092326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66740b9070 con 0x55f860504610 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f66737fe640 1 -- 172.21.15.37:0/977092326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66740aabb0 con 0x55f860504610 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- 172.21.15.37:0/977092326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f860504610 msgr2=0x55f860500ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 --2- 172.21.15.37:0/977092326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f860504610 0x55f860500ab0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f667400a590 tx=0x7f6674003030 comp rx=0 tx=0).stop 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- 172.21.15.37:0/977092326 shutdown_connections 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 --2- 172.21.15.37:0/977092326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f860504610 0x55f860500ab0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 --2- 172.21.15.37:0/977092326 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f860504230 0x55f8604717b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 --2- 172.21.15.37:0/977092326 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f86042bca0 0x55f86042f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- 172.21.15.37:0/977092326 >> 172.21.15.37:0/977092326 conn(0x55f860457ab0 msgr2=0x55f86045cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- 172.21.15.37:0/977092326 shutdown_connections 2024-04-25T09:04:12.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- 172.21.15.37:0/977092326 wait complete. 2024-04-25T09:04:12.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 Processor -- start 2024-04-25T09:04:12.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- start start 2024-04-25T09:04:12.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f86042bca0 0x55f860501bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f860504230 0x55f860506cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f860504610 0x55f8604fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f860430450 con 0x55f86042bca0 2024-04-25T09:04:12.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8604578f0 con 0x55f860504230 2024-04-25T09:04:12.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f667bbe4080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f8604306b0 con 0x55f860504610 2024-04-25T09:04:12.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679702640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f86042bca0 0x55f860501bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679702640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f86042bca0 0x55f860501bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42740/0 (socket says 172.21.15.37:42740) 2024-04-25T09:04:12.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679702640 1 -- 172.21.15.37:0/478996082 learned_addr learned my addr 172.21.15.37:0/478996082 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6678f01640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f860504230 0x55f860506cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679f03640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f860504610 0x55f8604fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.089+0000 7f6679702640 1 -- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f860504610 msgr2=0x55f8604fd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6679702640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f860504610 0x55f8604fd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6679702640 1 -- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f860504230 msgr2=0x55f860506cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6679702640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f860504230 0x55f860506cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6679702640 1 -- 172.21.15.37:0/478996082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8605023a0 con 0x55f86042bca0 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6678f01640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55f860504230 0x55f860506cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6679f03640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55f860504610 0x55f8604fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6679702640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f86042bca0 0x55f860501bb0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f666c002c00 tx=0x7f666c00a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f66727fc640 1 -- 172.21.15.37:0/478996082 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c010680 con 0x55f86042bca0 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f66727fc640 1 -- 172.21.15.37:0/478996082 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666c010840 con 0x55f86042bca0 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 -- 172.21.15.37:0/478996082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f860502640 con 0x55f86042bca0 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f66727fc640 1 -- 172.21.15.37:0/478996082 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c010c30 con 0x55f86042bca0 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 -- 172.21.15.37:0/478996082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f860502b80 con 0x55f86042bca0 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f66727fc640 1 -- 172.21.15.37:0/478996082 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f666c00f3e0 con 0x55f86042bca0 2024-04-25T09:04:12.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f66727fc640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f665c065940 0x7f665c067e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f66727fc640 1 -- 172.21.15.37:0/478996082 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f666c0840d0 con 0x55f86042bca0 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55f860507200 0x55f86050a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6678f01640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f665c065940 0x7f665c067e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 -- 172.21.15.37:0/478996082 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:1 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) v9 -- 0x55f86050a7c0 con 0x55f860507200 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6679f03640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55f860507200 0x55f86050a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6679f03640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55f860507200 0x55f86050a280 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6679f03640 1 -- 172.21.15.37:0/478996082 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f66740b2040 con 0x55f860507200 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6678f01640 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f665c065940 0x7f665c067e00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f666400b620 tx=0x7f6664007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 -- 172.21.15.37:0/478996082 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55f860507200 msgr2=0x55f86050a280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55f860507200 0x55f86050a280 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 -- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f665c065940 msgr2=0x7f665c067e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f665c065940 0x7f665c067e00 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f666400b620 tx=0x7f6664007040 comp rx=0 tx=0).stop 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 -- 172.21.15.37:0/478996082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f86042bca0 msgr2=0x55f860501bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 --2- 172.21.15.37:0/478996082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f86042bca0 0x55f860501bb0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f666c002c00 tx=0x7f666c00a5b0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f6679f03640 1 -- 172.21.15.37:0/478996082 reap_dead start 2024-04-25T09:04:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 -- 172.21.15.37:0/478996082 shutdown_connections 2024-04-25T09:04:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 -- 172.21.15.37:0/478996082 >> 172.21.15.37:0/478996082 conn(0x55f860457ab0 msgr2=0x55f8604fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 -- 172.21.15.37:0/478996082 shutdown_connections 2024-04-25T09:04:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.093+0000 7f667bbe4080 1 -- 172.21.15.37:0/478996082 wait complete. 2024-04-25T09:04:12.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:04:12.089 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-04-25T09:04:12.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.173+0000 7f269831c080 1 Processor -- start 2024-04-25T09:04:12.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.173+0000 7f269831c080 1 -- start start 2024-04-25T09:04:12.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.173+0000 7f269831c080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5564fa1a35b0 0x5564fa179d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.173+0000 7f269831c080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 0x5564fa179400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.173+0000 7f269831c080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564fa24d670 0x5564fa242ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.173+0000 7f269831c080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564fa11c9f0 con 0x5564fa24d290 2024-04-25T09:04:12.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.173+0000 7f269831c080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564fa120950 con 0x5564fa24d670 2024-04-25T09:04:12.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.173+0000 7f269831c080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564fa1a7460 con 0x5564fa1a35b0 2024-04-25T09:04:12.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 0x5564fa179400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 0x5564fa179400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42764/0 (socket says 172.21.15.37:42764) 2024-04-25T09:04:12.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 -- 172.21.15.37:0/2068284824 learned_addr learned my addr 172.21.15.37:0/2068284824 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695e3a640 1 --2- 172.21.15.37:0/2068284824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5564fa1a35b0 0x5564fa179d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269663b640 1 --2- 172.21.15.37:0/2068284824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564fa24d670 0x5564fa242ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 -- 172.21.15.37:0/2068284824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5564fa1a35b0 msgr2=0x5564fa179d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 --2- 172.21.15.37:0/2068284824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5564fa1a35b0 0x5564fa179d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 -- 172.21.15.37:0/2068284824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564fa24d670 msgr2=0x5564fa242ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 --2- 172.21.15.37:0/2068284824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564fa24d670 0x5564fa242ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 -- 172.21.15.37:0/2068284824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564fa17a3f0 con 0x5564fa24d290 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695e3a640 1 --2- 172.21.15.37:0/2068284824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5564fa1a35b0 0x5564fa179d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 --2- 172.21.15.37:0/2068284824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 0x5564fa179400 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f268000a500 tx=0x7f2680003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22440547b65dc39d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f268ffff640 1 -- 172.21.15.37:0/2068284824 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26800af040 con 0x5564fa24d290 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f268ffff640 1 -- 172.21.15.37:0/2068284824 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26800b9070 con 0x5564fa24d290 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f268ffff640 1 -- 172.21.15.37:0/2068284824 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26800aadd0 con 0x5564fa24d290 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 -- 172.21.15.37:0/2068284824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 msgr2=0x5564fa179400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 --2- 172.21.15.37:0/2068284824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 0x5564fa179400 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f268000a500 tx=0x7f2680003030 comp rx=0 tx=0).stop 2024-04-25T09:04:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 -- 172.21.15.37:0/2068284824 shutdown_connections 2024-04-25T09:04:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 --2- 172.21.15.37:0/2068284824 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564fa24d670 0x5564fa242ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 --2- 172.21.15.37:0/2068284824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 0x5564fa179400 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 --2- 172.21.15.37:0/2068284824 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5564fa1a35b0 0x5564fa179d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 -- 172.21.15.37:0/2068284824 >> 172.21.15.37:0/2068284824 conn(0x5564fa1a1c50 msgr2=0x5564fa24ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 -- 172.21.15.37:0/2068284824 shutdown_connections 2024-04-25T09:04:12.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 -- 172.21.15.37:0/2068284824 wait complete. 2024-04-25T09:04:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 Processor -- start 2024-04-25T09:04:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 -- start start 2024-04-25T09:04:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5564fa1a35b0 0x5564fa24e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 0x5564fa230180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564fa24d670 0x5564fa2306c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564fa17a930 con 0x5564fa24d290 2024-04-25T09:04:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564fa17a690 con 0x5564fa24d670 2024-04-25T09:04:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269831c080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5564fa24b550 con 0x5564fa1a35b0 2024-04-25T09:04:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 0x5564fa230180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 0x5564fa230180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42774/0 (socket says 172.21.15.37:42774) 2024-04-25T09:04:12.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 -- 172.21.15.37:0/244451407 learned_addr learned my addr 172.21.15.37:0/244451407 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695e3a640 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5564fa1a35b0 0x5564fa24e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269663b640 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564fa24d670 0x5564fa2306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 -- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5564fa1a35b0 msgr2=0x5564fa24e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5564fa1a35b0 0x5564fa24e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 -- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564fa24d670 msgr2=0x5564fa2306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564fa24d670 0x5564fa2306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 -- 172.21.15.37:0/244451407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564fa245090 con 0x5564fa24d290 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f269663b640 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5564fa24d670 0x5564fa2306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.177+0000 7f2695639640 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 0x5564fa230180 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f268001b5e0 tx=0x7f2680017ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f268effd640 1 -- 172.21.15.37:0/244451407 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26800c1020 con 0x5564fa24d290 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f268effd640 1 -- 172.21.15.37:0/244451407 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f268000a2f0 con 0x5564fa24d290 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 -- 172.21.15.37:0/244451407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5564fa245330 con 0x5564fa24d290 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f268effd640 1 -- 172.21.15.37:0/244451407 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26800af070 con 0x5564fa24d290 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 -- 172.21.15.37:0/244451407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564fa245870 con 0x5564fa24d290 2024-04-25T09:04:12.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f268effd640 1 -- 172.21.15.37:0/244451407 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f26800c4020 con 0x5564fa24d290 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f268effd640 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2678065970 0x7f2678067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f268effd640 1 -- 172.21.15.37:0/244451407 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f268011faf0 con 0x5564fa24d290 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x5564fa24e7d0 0x5564fa245fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 -- 172.21.15.37:0/244451407 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:1 18.12 18:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) v9 -- 0x5564fa2464f0 con 0x5564fa24e7d0 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f2695e3a640 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2678065970 0x7f2678067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269663b640 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x5564fa24e7d0 0x5564fa245fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269663b640 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x5564fa24e7d0 0x5564fa245fb0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269663b640 1 -- 172.21.15.37:0/244451407 <== osd.2 v2:172.21.15.37:6808/332967247 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f2688006d30 con 0x5564fa24e7d0 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f2695e3a640 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2678065970 0x7f2678067e30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2690004180 tx=0x7f26900015e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 -- 172.21.15.37:0/244451407 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x5564fa24e7d0 msgr2=0x5564fa245fb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x5564fa24e7d0 0x5564fa245fb0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 -- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2678065970 msgr2=0x7f2678067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2678065970 0x7f2678067e30 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f2690004180 tx=0x7f26900015e0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 -- 172.21.15.37:0/244451407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 msgr2=0x5564fa230180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 --2- 172.21.15.37:0/244451407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5564fa24d290 0x5564fa230180 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f268001b5e0 tx=0x7f2680017ea0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269663b640 1 -- 172.21.15.37:0/244451407 reap_dead start 2024-04-25T09:04:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 -- 172.21.15.37:0/244451407 shutdown_connections 2024-04-25T09:04:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 -- 172.21.15.37:0/244451407 >> 172.21.15.37:0/244451407 conn(0x5564fa1a1c50 msgr2=0x5564fa24bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 -- 172.21.15.37:0/244451407 shutdown_connections 2024-04-25T09:04:12.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.181+0000 7f269831c080 1 -- 172.21.15.37:0/244451407 wait complete. 2024-04-25T09:04:12.179 INFO:teuthology.orchestra.run.smithi037.stdout:mytag123 2024-04-25T09:04:12.179 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-04-25T09:04:12.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 Processor -- start 2024-04-25T09:04:12.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 -- start start 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5653c4efd510 0x5653c4ef4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4eff620 0x5653c4ef5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5653c4f891f0 0x5653c4eb0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c4ee57c0 con 0x5653c4eff620 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c4e589f0 con 0x5653c4f891f0 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653c4ee5a60 con 0x5653c4efd510 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfffc2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4eff620 0x5653c4ef5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfffc2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4eff620 0x5653c4ef5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42796/0 (socket says 172.21.15.37:42796) 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfffc2640 1 -- 172.21.15.37:0/759836540 learned_addr learned my addr 172.21.15.37:0/759836540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfffc2640 1 -- 172.21.15.37:0/759836540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5653c4efd510 msgr2=0x5653c4ef4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d00fc4640 1 --2- 172.21.15.37:0/759836540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5653c4f891f0 0x5653c4eb0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfffc2640 1 --2- 172.21.15.37:0/759836540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5653c4efd510 0x5653c4ef4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfffc2640 1 -- 172.21.15.37:0/759836540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5653c4f891f0 msgr2=0x5653c4eb0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfffc2640 1 --2- 172.21.15.37:0/759836540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5653c4f891f0 0x5653c4eb0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfffc2640 1 -- 172.21.15.37:0/759836540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653c4f87f20 con 0x5653c4eff620 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfffc2640 1 --2- 172.21.15.37:0/759836540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4eff620 0x5653c4ef5930 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40ad040 tx=0x7f3cf4003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57fe0a80eed06d25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfefc0640 1 -- 172.21.15.37:0/759836540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40af040 con 0x5653c4eff620 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfefc0640 1 -- 172.21.15.37:0/759836540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf40b9070 con 0x5653c4eff620 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3cfefc0640 1 -- 172.21.15.37:0/759836540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40aae10 con 0x5653c4eff620 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 -- 172.21.15.37:0/759836540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4eff620 msgr2=0x5653c4ef5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 --2- 172.21.15.37:0/759836540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4eff620 0x5653c4ef5930 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f3cf40ad040 tx=0x7f3cf4003030 comp rx=0 tx=0).stop 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 -- 172.21.15.37:0/759836540 shutdown_connections 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 --2- 172.21.15.37:0/759836540 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5653c4f891f0 0x5653c4eb0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 --2- 172.21.15.37:0/759836540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4eff620 0x5653c4ef5930 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 --2- 172.21.15.37:0/759836540 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5653c4efd510 0x5653c4ef4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 -- 172.21.15.37:0/759836540 >> 172.21.15.37:0/759836540 conn(0x5653c4eb8a80 msgr2=0x5653c4f67250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 -- 172.21.15.37:0/759836540 shutdown_connections 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 -- 172.21.15.37:0/759836540 wait complete. 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 Processor -- start 2024-04-25T09:04:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.265+0000 7f3d02ca5080 1 -- start start 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d02ca5080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5653c4efd510 0x5653c4efa350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d02ca5080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5653c4eff620 0x5653c4efb980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d02ca5080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4f891f0 0x5653c4f851c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d02ca5080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c4ef6050 con 0x5653c4f891f0 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d02ca5080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5653c4f680a0 con 0x5653c4efd510 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d02ca5080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5653c4f87a30 con 0x5653c4eff620 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d00fc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4f891f0 0x5653c4f851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d00fc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4f891f0 0x5653c4f851c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42800/0 (socket says 172.21.15.37:42800) 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d00fc4640 1 -- 172.21.15.37:0/375423001 learned_addr learned my addr 172.21.15.37:0/375423001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d007c3640 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5653c4efd510 0x5653c4efa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3cfffc2640 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5653c4eff620 0x5653c4efb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d00fc4640 1 -- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5653c4eff620 msgr2=0x5653c4efb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d00fc4640 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5653c4eff620 0x5653c4efb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d00fc4640 1 -- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5653c4efd510 msgr2=0x5653c4efa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d00fc4640 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5653c4efd510 0x5653c4efa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d00fc4640 1 -- 172.21.15.37:0/375423001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5653c4f85700 con 0x5653c4f891f0 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d007c3640 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5653c4efd510 0x5653c4efa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d00fc4640 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4f891f0 0x5653c4f851c0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000cdc0 tx=0x7f3cf000a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3cfdfbe640 1 -- 172.21.15.37:0/375423001 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000f410 con 0x5653c4f891f0 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3cfdfbe640 1 -- 172.21.15.37:0/375423001 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf0014020 con 0x5653c4f891f0 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3cfdfbe640 1 -- 172.21.15.37:0/375423001 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000f810 con 0x5653c4f891f0 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d02ca5080 1 -- 172.21.15.37:0/375423001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5653c4efaa60 con 0x5653c4f891f0 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d02ca5080 1 -- 172.21.15.37:0/375423001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5653c4efafa0 con 0x5653c4f891f0 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3cfdfbe640 1 -- 172.21.15.37:0/375423001 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3cf000f9b0 con 0x5653c4f891f0 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3cfdfbe640 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cec065970 0x7f3cec067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3cfdfbe640 1 -- 172.21.15.37:0/375423001 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f3cf00842e0 con 0x5653c4f891f0 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d02ca5080 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x5653c4f806f0 0x5653c4f68780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.269+0000 7f3d02ca5080 1 -- 172.21.15.37:0/375423001 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:1 18.19 18:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) v9 -- 0x5653c4ef70e0 con 0x5653c4f806f0 2024-04-25T09:04:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d007c3640 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cec065970 0x7f3cec067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3cfffc2640 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x5653c4f806f0 0x5653c4f68780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d007c3640 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cec065970 0x7f3cec067e30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8005fd0 tx=0x7f3cf8007040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3cfffc2640 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x5653c4f806f0 0x5653c4f68780 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3cfffc2640 1 -- 172.21.15.37:0/375423001 <== osd.4 v2:172.21.15.203:6802/714177927 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f3cf40b2040 con 0x5653c4f806f0 2024-04-25T09:04:12.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d02ca5080 1 -- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x5653c4f806f0 msgr2=0x5653c4f68780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d02ca5080 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x5653c4f806f0 0x5653c4f68780 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d02ca5080 1 -- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cec065970 msgr2=0x7f3cec067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d02ca5080 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3cec065970 0x7f3cec067e30 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8005fd0 tx=0x7f3cf8007040 comp rx=0 tx=0).stop 2024-04-25T09:04:12.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d02ca5080 1 -- 172.21.15.37:0/375423001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4f891f0 msgr2=0x5653c4f851c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d02ca5080 1 --2- 172.21.15.37:0/375423001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5653c4f891f0 0x5653c4f851c0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000cdc0 tx=0x7f3cf000a5e0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d00fc4640 1 -- 172.21.15.37:0/375423001 reap_dead start 2024-04-25T09:04:12.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d02ca5080 1 -- 172.21.15.37:0/375423001 shutdown_connections 2024-04-25T09:04:12.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d02ca5080 1 -- 172.21.15.37:0/375423001 >> 172.21.15.37:0/375423001 conn(0x5653c4eb8a80 msgr2=0x5653c4f85b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d02ca5080 1 -- 172.21.15.37:0/375423001 shutdown_connections 2024-04-25T09:04:12.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.273+0000 7f3d02ca5080 1 -- 172.21.15.37:0/375423001 wait complete. 2024-04-25T09:04:12.265 INFO:teuthology.orchestra.run.smithi037.stdout:mytag123 2024-04-25T09:04:12.266 DEBUG:teuthology.orchestra.run.smithi037:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-04-25T09:04:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 Processor -- start 2024-04-25T09:04:12.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- start start 2024-04-25T09:04:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e16ec9fee0 0x55e16ed3d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e16ecbc7e0 0x55e16eca3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ed47f70 0x55e16eca4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16eca4c10 con 0x55e16ed47f70 2024-04-25T09:04:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ec1a9f0 con 0x55e16ec9fee0 2024-04-25T09:04:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e16ed48d70 con 0x55e16ecbc7e0 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac29ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ed47f70 0x55e16eca4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac29ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ed47f70 0x55e16eca4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42820/0 (socket says 172.21.15.37:42820) 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac29ea640 1 -- 172.21.15.37:0/8356566 learned_addr learned my addr 172.21.15.37:0/8356566 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac19e8640 1 --2- 172.21.15.37:0/8356566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e16ecbc7e0 0x55e16eca3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac21e9640 1 --2- 172.21.15.37:0/8356566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e16ec9fee0 0x55e16ed3d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac29ea640 1 -- 172.21.15.37:0/8356566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e16ecbc7e0 msgr2=0x55e16eca3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac29ea640 1 --2- 172.21.15.37:0/8356566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e16ecbc7e0 0x55e16eca3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac29ea640 1 -- 172.21.15.37:0/8356566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e16ec9fee0 msgr2=0x55e16ed3d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac29ea640 1 --2- 172.21.15.37:0/8356566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e16ec9fee0 0x55e16ed3d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac29ea640 1 -- 172.21.15.37:0/8356566 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e16ed46d60 con 0x55e16ed47f70 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac19e8640 1 --2- 172.21.15.37:0/8356566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e16ecbc7e0 0x55e16eca3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac21e9640 1 --2- 172.21.15.37:0/8356566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e16ec9fee0 0x55e16ed3d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac29ea640 1 --2- 172.21.15.37:0/8356566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ed47f70 0x55e16eca4240 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4001ea0 tx=0x7f9ab400b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8aa4abe85a08cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac09e6640 1 -- 172.21.15.37:0/8356566 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400a040 con 0x55e16ed47f70 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac09e6640 1 -- 172.21.15.37:0/8356566 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab400f620 con 0x55e16ed47f70 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac09e6640 1 -- 172.21.15.37:0/8356566 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400f9f0 con 0x55e16ed47f70 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- 172.21.15.37:0/8356566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ed47f70 msgr2=0x55e16eca4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 --2- 172.21.15.37:0/8356566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ed47f70 0x55e16eca4240 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4001ea0 tx=0x7f9ab400b680 comp rx=0 tx=0).stop 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- 172.21.15.37:0/8356566 shutdown_connections 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 --2- 172.21.15.37:0/8356566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ed47f70 0x55e16eca4240 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 --2- 172.21.15.37:0/8356566 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e16ecbc7e0 0x55e16eca3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 --2- 172.21.15.37:0/8356566 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e16ec9fee0 0x55e16ed3d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- 172.21.15.37:0/8356566 >> 172.21.15.37:0/8356566 conn(0x55e16ec9eac0 msgr2=0x55e16ec737b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- 172.21.15.37:0/8356566 shutdown_connections 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- 172.21.15.37:0/8356566 wait complete. 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 Processor -- start 2024-04-25T09:04:12.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- start start 2024-04-25T09:04:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e16ec9fee0 0x55e16ede6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ecbc7e0 0x55e16ed41da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e16ed47f70 0x55e16ed422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ec775f0 con 0x55e16ecbc7e0 2024-04-25T09:04:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e16ec77810 con 0x55e16ed47f70 2024-04-25T09:04:12.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.353+0000 7f9ac46cb080 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e16ed536f0 con 0x55e16ec9fee0 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac19e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ecbc7e0 0x55e16ed41da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac19e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ecbc7e0 0x55e16ed41da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42824/0 (socket says 172.21.15.37:42824) 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac19e8640 1 -- 172.21.15.37:0/2504823973 learned_addr learned my addr 172.21.15.37:0/2504823973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac29ea640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e16ed47f70 0x55e16ed422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac21e9640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e16ec9fee0 0x55e16ede6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac19e8640 1 -- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e16ec9fee0 msgr2=0x55e16ede6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac19e8640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e16ec9fee0 0x55e16ede6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac19e8640 1 -- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e16ed47f70 msgr2=0x55e16ed422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac19e8640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e16ed47f70 0x55e16ed422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac19e8640 1 -- 172.21.15.37:0/2504823973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e16ed53990 con 0x55e16ecbc7e0 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac29ea640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e16ed47f70 0x55e16ed422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac21e9640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e16ec9fee0 0x55e16ede6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac19e8640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ecbc7e0 0x55e16ed41da0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80066c0 tx=0x7f9ab8006b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ab37fe640 1 -- 172.21.15.37:0/2504823973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab800c6c0 con 0x55e16ecbc7e0 2024-04-25T09:04:12.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ab37fe640 1 -- 172.21.15.37:0/2504823973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab800c860 con 0x55e16ecbc7e0 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac46cb080 1 -- 172.21.15.37:0/2504823973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e16ed53c30 con 0x55e16ecbc7e0 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ab37fe640 1 -- 172.21.15.37:0/2504823973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab800cc50 con 0x55e16ecbc7e0 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac46cb080 1 -- 172.21.15.37:0/2504823973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e16ede7270 con 0x55e16ecbc7e0 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ab37fe640 1 -- 172.21.15.37:0/2504823973 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9ab8019020 con 0x55e16ecbc7e0 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ab37fe640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9aa4065970 0x7f9aa4067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ab37fe640 1 -- 172.21.15.37:0/2504823973 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9ab8083c10 con 0x55e16ecbc7e0 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac46cb080 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55e16ed54670 0x55e16ede4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac46cb080 1 -- 172.21.15.37:0/2504823973 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:1 18.5 18:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e126) v9 -- 0x55e16ecb3dc0 con 0x55e16ed54670 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac21e9640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9aa4065970 0x7f9aa4067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac29ea640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55e16ed54670 0x55e16ede4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac29ea640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55e16ed54670 0x55e16ede4b40 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac21e9640 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9aa4065970 0x7f9aa4067e30 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0ab000 tx=0x7f9abc0a5070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac29ea640 1 -- 172.21.15.37:0/2504823973 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+12 (crc 0 0 0) 0x7f9ab400d040 con 0x55e16ed54670 2024-04-25T09:04:12.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac46cb080 1 -- 172.21.15.37:0/2504823973 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55e16ed54670 msgr2=0x55e16ede4b40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac46cb080 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55e16ed54670 0x55e16ede4b40 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac46cb080 1 -- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9aa4065970 msgr2=0x7f9aa4067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.357+0000 7f9ac46cb080 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9aa4065970 0x7f9aa4067e30 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0ab000 tx=0x7f9abc0a5070 comp rx=0 tx=0).stop 2024-04-25T09:04:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.361+0000 7f9ac46cb080 1 -- 172.21.15.37:0/2504823973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ecbc7e0 msgr2=0x55e16ed41da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.361+0000 7f9ac46cb080 1 --2- 172.21.15.37:0/2504823973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e16ecbc7e0 0x55e16ed41da0 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f9ab80066c0 tx=0x7f9ab8006b80 comp rx=0 tx=0).stop 2024-04-25T09:04:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.361+0000 7f9ac29ea640 1 -- 172.21.15.37:0/2504823973 reap_dead start 2024-04-25T09:04:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.361+0000 7f9ac46cb080 1 -- 172.21.15.37:0/2504823973 shutdown_connections 2024-04-25T09:04:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.361+0000 7f9ac46cb080 1 -- 172.21.15.37:0/2504823973 >> 172.21.15.37:0/2504823973 conn(0x55e16ec9eac0 msgr2=0x55e16ec75ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.361+0000 7f9ac46cb080 1 -- 172.21.15.37:0/2504823973 shutdown_connections 2024-04-25T09:04:12.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.361+0000 7f9ac46cb080 1 -- 172.21.15.37:0/2504823973 wait complete. 2024-04-25T09:04:12.353 INFO:teuthology.orchestra.run.smithi037.stdout:mytag123 2024-04-25T09:04:12.353 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:04:12.353 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-04-25T09:04:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.537+0000 7fe04d04a640 1 Processor -- start 2024-04-25T09:04:12.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.537+0000 7fe04d04a640 1 -- start start 2024-04-25T09:04:12.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04d04a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe0480b4430 0x7fe0480b23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04d04a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0480cce90 0x7fe0480cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04d04a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04809ab90 con 0x7fe0480b1f20 2024-04-25T09:04:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04d04a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0480b28f0 con 0x7fe0480b4430 2024-04-25T09:04:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04d04a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480cc2f0 con 0x7fe0480cce90 2024-04-25T09:04:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04659c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe0480b4430 0x7fe0480b23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe046d9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe0480b1f20 0x7fe0480cc780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48318/0 (socket says 172.21.15.37:48318) 2024-04-25T09:04:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe046d9d640 1 -- 172.21.15.37:0/1851218068 learned_addr learned my addr 172.21.15.37:0/1851218068 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04659c640 1 -- 172.21.15.37:0/1851218068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0480cce90 msgr2=0x7fe0480cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04759e640 1 --2- 172.21.15.37:0/1851218068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0480cce90 0x7fe0480cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04659c640 1 --2- 172.21.15.37:0/1851218068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0480cce90 0x7fe0480cf1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04659c640 1 -- 172.21.15.37:0/1851218068 >> v1:172.21.15.37:6789/0 conn(0x7fe0480b1f20 legacy=0x7fe0480cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04659c640 1 -- 172.21.15.37:0/1851218068 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0480b48c0 con 0x7fe0480b4430 2024-04-25T09:04:12.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe045d9b640 1 -- 172.21.15.37:0/1851218068 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221974858 0 0) 0x7fe04809ab90 con 0x7fe0480b1f20 2024-04-25T09:04:12.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe04659c640 1 --2- 172.21.15.37:0/1851218068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0480b4430 0x7fe0480b23b0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fe0300097c0 tx=0x7fe03001c150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979590a45a426415 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe045d9b640 1 -- 172.21.15.37:0/1851218068 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300b1070 con 0x7fe0480b4430 2024-04-25T09:04:12.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe045d9b640 1 -- 172.21.15.37:0/1851218068 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0300ac440 con 0x7fe0480b4430 2024-04-25T09:04:12.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.541+0000 7fe045d9b640 1 -- 172.21.15.37:0/1851218068 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300ac810 con 0x7fe0480b4430 2024-04-25T09:04:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 -- 172.21.15.37:0/1851218068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0480b4430 msgr2=0x7fe0480b23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 --2- 172.21.15.37:0/1851218068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0480b4430 0x7fe0480b23b0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fe0300097c0 tx=0x7fe03001c150 comp rx=0 tx=0).stop 2024-04-25T09:04:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 -- 172.21.15.37:0/1851218068 shutdown_connections 2024-04-25T09:04:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 --2- 172.21.15.37:0/1851218068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0480cce90 0x7fe0480cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 --2- 172.21.15.37:0/1851218068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0480b4430 0x7fe0480b23b0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 -- 172.21.15.37:0/1851218068 >> 172.21.15.37:0/1851218068 conn(0x7fe0480d15e0 msgr2=0x7fe0480d19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 -- 172.21.15.37:0/1851218068 shutdown_connections 2024-04-25T09:04:12.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 -- 172.21.15.37:0/1851218068 wait complete. 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 Processor -- start 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 -- start start 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0480b1f20 0x7fe048159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0480b4430 0x7fe04806cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0480cce90 0x7fe048152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480cd420 con 0x7fe0480b1f20 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480cd2a0 con 0x7fe0480b4430 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe048168790 con 0x7fe0480cce90 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe046d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0480b1f20 0x7fe048159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe046d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0480b1f20 0x7fe048159040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42836/0 (socket says 172.21.15.37:42836) 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe046d9d640 1 -- 172.21.15.37:0/886370422 learned_addr learned my addr 172.21.15.37:0/886370422 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe046d9d640 1 -- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0480cce90 msgr2=0x7fe048152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04759e640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0480cce90 0x7fe048152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04659c640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0480b4430 0x7fe04806cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe046d9d640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0480cce90 0x7fe048152400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe046d9d640 1 -- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0480b4430 msgr2=0x7fe04806cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe046d9d640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0480b4430 0x7fe04806cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe046d9d640 1 -- 172.21.15.37:0/886370422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04806d2c0 con 0x7fe0480b1f20 2024-04-25T09:04:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe046d9d640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0480b1f20 0x7fe048159040 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c480 tx=0x7fe03800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe03f7fe640 1 -- 172.21.15.37:0/886370422 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038018070 con 0x7fe0480b1f20 2024-04-25T09:04:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe03f7fe640 1 -- 172.21.15.37:0/886370422 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe038007500 con 0x7fe0480b1f20 2024-04-25T09:04:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 -- 172.21.15.37:0/886370422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe048159740 con 0x7fe0480b1f20 2024-04-25T09:04:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe03f7fe640 1 -- 172.21.15.37:0/886370422 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038004a30 con 0x7fe0480b1f20 2024-04-25T09:04:12.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.545+0000 7fe04d04a640 1 -- 172.21.15.37:0/886370422 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe048159d00 con 0x7fe0480b1f20 2024-04-25T09:04:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe03f7fe640 1 -- 172.21.15.37:0/886370422 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe03801b070 con 0x7fe0480b1f20 2024-04-25T09:04:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe03f7fe640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe028066f60 0x7fe028069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe03f7fe640 1 -- 172.21.15.37:0/886370422 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe0380844b0 con 0x7fe0480b1f20 2024-04-25T09:04:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04659c640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe028066f60 0x7fe028069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe0501b7000 1 Processor -- start 2024-04-25T09:04:12.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe0501b7000 1 -- start start 2024-04-25T09:04:12.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04659c640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe028066f60 0x7fe028069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe0300ae000 tx=0x7fe030008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b979d0 0x55e306b97db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b982f0 0x55e306b4db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306b4e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe0501b7000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e30699aff0 con 0x55e306b4e0a0 2024-04-25T09:04:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe0501b7000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e306923150 con 0x55e306b982f0 2024-04-25T09:04:12.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe0501b7000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e306997090 con 0x55e306b979d0 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306b4e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306b4e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42838/0 (socket says 172.21.15.37:42838) 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04759e640 1 -- 172.21.15.37:0/1395754287 learned_addr learned my addr 172.21.15.37:0/1395754287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe046d9d640 1 --2- 172.21.15.37:0/1395754287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b979d0 0x55e306b97db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04659c640 1 --2- 172.21.15.37:0/1395754287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b982f0 0x55e306b4db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04759e640 1 -- 172.21.15.37:0/1395754287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b979d0 msgr2=0x55e306b97db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04759e640 1 --2- 172.21.15.37:0/1395754287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b979d0 0x55e306b97db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04759e640 1 -- 172.21.15.37:0/1395754287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b982f0 msgr2=0x55e306b4db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04759e640 1 --2- 172.21.15.37:0/1395754287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b982f0 0x55e306b4db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04759e640 1 -- 172.21.15.37:0/1395754287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e306b310d0 con 0x55e306b4e0a0 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe04759e640 1 --2- 172.21.15.37:0/1395754287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306b4e480 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fe040008d40 tx=0x7fe04000d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf739d3ea0e5d4d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe03cff9640 1 -- 172.21.15.37:0/1395754287 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040015020 con 0x55e306b4e0a0 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe03cff9640 1 -- 172.21.15.37:0/1395754287 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0400039a0 con 0x55e306b4e0a0 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.549+0000 7fe03cff9640 1 -- 172.21.15.37:0/1395754287 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040003d70 con 0x55e306b4e0a0 2024-04-25T09:04:12.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- 172.21.15.37:0/1395754287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 msgr2=0x55e306b4e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 --2- 172.21.15.37:0/1395754287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306b4e480 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fe040008d40 tx=0x7fe04000d6b0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- 172.21.15.37:0/1395754287 shutdown_connections 2024-04-25T09:04:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 --2- 172.21.15.37:0/1395754287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306b4e480 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 --2- 172.21.15.37:0/1395754287 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b982f0 0x55e306b4db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 --2- 172.21.15.37:0/1395754287 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b979d0 0x55e306b97db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- 172.21.15.37:0/1395754287 >> 172.21.15.37:0/1395754287 conn(0x55e306b4a020 msgr2=0x55e306b4c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- 172.21.15.37:0/1395754287 shutdown_connections 2024-04-25T09:04:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- 172.21.15.37:0/1395754287 wait complete. 2024-04-25T09:04:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- start start 2024-04-25T09:04:12.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 Processor -- start 2024-04-25T09:04:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b4e0a0 0x55e306b7c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b979d0 0x55e306b7cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b982f0 0x55e306b7d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e30695f250 con 0x55e306b979d0 2024-04-25T09:04:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e306955690 con 0x55e306b4e0a0 2024-04-25T09:04:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e306923a40 con 0x55e306b982f0 2024-04-25T09:04:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 10 client.0 _ll_register_callbacks cb 0x55e306b38e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe04659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b979d0 0x55e306b7cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe04659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b979d0 0x55e306b7cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42840/0 (socket says 172.21.15.37:42840) 2024-04-25T09:04:12.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe04659c640 1 -- 172.21.15.37:0/2004216385 learned_addr learned my addr 172.21.15.37:0/2004216385 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe04759e640 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b982f0 0x55e306b7d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe046d9d640 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b4e0a0 0x55e306b7c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe04659c640 1 -- 172.21.15.37:0/2004216385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b982f0 msgr2=0x55e306b7d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe04659c640 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b982f0 0x55e306b7d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe04659c640 1 -- 172.21.15.37:0/2004216385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b4e0a0 msgr2=0x55e306b7c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe04659c640 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b4e0a0 0x55e306b7c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe04659c640 1 -- 172.21.15.37:0/2004216385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e306a481c0 con 0x55e306b979d0 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe04659c640 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b979d0 0x55e306b7cd30 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fe030019d80 tx=0x7fe0300046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f28e8fe110e0718 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe025ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe025ffb640 1 -- 172.21.15.37:0/2004216385 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300b8450 con 0x55e306b979d0 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- 172.21.15.37:0/2004216385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0400039a0 con 0x55e306b979d0 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe025ffb640 1 -- 172.21.15.37:0/2004216385 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe0300b85f0 con 0x55e306b979d0 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe025ffb640 1 -- 172.21.15.37:0/2004216385 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe0300b87f0 con 0x55e306b979d0 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 10 client.9972 fetch_fsmap learned FSMap version 103 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- 172.21.15.37:0/2004216385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x55e306a44170 con 0x55e306b979d0 2024-04-25T09:04:12.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe025ffb640 1 -- 172.21.15.37:0/2004216385 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0400039a0 con 0x55e306b979d0 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe025ffb640 1 -- 172.21.15.37:0/2004216385 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 103) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0300b8990 con 0x55e306b979d0 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 10 client.9972 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 10 client.9972 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 20 client.9972 populate_metadata read hostname 'smithi037' 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55e306ba7c80 0x55e306baa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 4 client.9972 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe0501b7000 1 -- 172.21.15.37:0/2004216385 --> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e306936c60 con 0x55e306ba7c80 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe046d9d640 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55e306ba7c80 0x55e306baa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe046d9d640 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55e306ba7c80 0x55e306baa070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf817d1ddeb2f59 server_cookie=a05ea36848fc25c0 in_seq=0 out_seq=0 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.553+0000 7fe025ffb640 10 client.9972 ms_handle_connect on v2:172.21.15.37:6834/2159545482 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe025ffb640 1 -- 172.21.15.37:0/2004216385 <== mds.0 v2:172.21.15.37:6834/2159545482 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e306936c60 con 0x55e306ba7c80 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe025ffb640 10 client.9972 handle_command_reply: tid=0 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 1 client.9972 shutdown 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 1 -- 172.21.15.37:0/2004216385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b979d0 msgr2=0x55e306b7cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b979d0 0x55e306b7cd30 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fe030019d80 tx=0x7fe0300046d0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 1 -- 172.21.15.37:0/2004216385 shutdown_connections 2024-04-25T09:04:12.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55e306ba7c80 0x55e306baa070 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b982f0 0x55e306b7d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b979d0 0x55e306b7cd30 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2004216385 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b4e0a0 0x55e306b7c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe025ffb640 0 client.9972 ms_handle_reset on v2:172.21.15.37:6834/2159545482 2024-04-25T09:04:12.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 1 -- 172.21.15.37:0/2004216385 >> 172.21.15.37:0/2004216385 conn(0x55e306b4a020 msgr2=0x55e306b4cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 1 -- 172.21.15.37:0/2004216385 shutdown_connections 2024-04-25T09:04:12.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 1 -- 172.21.15.37:0/2004216385 wait complete. 2024-04-25T09:04:12.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.557+0000 7fe0501b7000 20 client.9972 trim_cache size 0 max 16384 2024-04-25T09:04:12.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe0501b7000 1 Processor -- start 2024-04-25T09:04:12.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe0501b7000 1 -- start start 2024-04-25T09:04:12.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306bf40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b979d0 0x55e306bf4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b982f0 0x55e306bf4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe0501b7000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0482033f0 con 0x55e306b4e0a0 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe0501b7000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe048202580 con 0x55e306b979d0 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe0501b7000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480af320 con 0x55e306b982f0 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe046d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306bf40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe046d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306bf40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42844/0 (socket says 172.21.15.37:42844) 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe046d9d640 1 -- 172.21.15.37:0/53324630 learned_addr learned my addr 172.21.15.37:0/53324630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe046d9d640 1 -- 172.21.15.37:0/53324630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b982f0 msgr2=0x55e306bf4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe046d9d640 1 --2- 172.21.15.37:0/53324630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b982f0 0x55e306bf4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe046d9d640 1 -- 172.21.15.37:0/53324630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b979d0 msgr2=0x55e306bf4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe046d9d640 1 --2- 172.21.15.37:0/53324630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b979d0 0x55e306bf4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe046d9d640 1 -- 172.21.15.37:0/53324630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3069e1de0 con 0x55e306b4e0a0 2024-04-25T09:04:12.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe046d9d640 1 --2- 172.21.15.37:0/53324630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306bf40f0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fe038054970 tx=0x7fe038058040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c90245d24794fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.569+0000 7fe027fff640 1 -- 172.21.15.37:0/53324630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038052f70 con 0x55e306b4e0a0 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe027fff640 1 -- 172.21.15.37:0/53324630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0380512e0 con 0x55e306b4e0a0 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe027fff640 1 -- 172.21.15.37:0/53324630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038004dd0 con 0x55e306b4e0a0 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- 172.21.15.37:0/53324630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 msgr2=0x55e306bf40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 --2- 172.21.15.37:0/53324630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306bf40f0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fe038054970 tx=0x7fe038058040 comp rx=0 tx=0).stop 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- 172.21.15.37:0/53324630 shutdown_connections 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 --2- 172.21.15.37:0/53324630 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b982f0 0x55e306bf4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 --2- 172.21.15.37:0/53324630 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b979d0 0x55e306bf4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 --2- 172.21.15.37:0/53324630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b4e0a0 0x55e306bf40f0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- 172.21.15.37:0/53324630 >> 172.21.15.37:0/53324630 conn(0x55e306b4a020 msgr2=0x55e306ba4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- 172.21.15.37:0/53324630 shutdown_connections 2024-04-25T09:04:12.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- 172.21.15.37:0/53324630 wait complete. 2024-04-25T09:04:12.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- start start 2024-04-25T09:04:12.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 Processor -- start 2024-04-25T09:04:12.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b4e0a0 0x55e306baa380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b979d0 0x55e306baa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b982f0 0x55e306baae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e306935e00 con 0x55e306b982f0 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0480af920 con 0x55e306b979d0 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e30699a300 con 0x55e306b4e0a0 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 10 client.0 _ll_register_callbacks cb 0x55e306b19040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe04759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b982f0 0x55e306baae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe04759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b982f0 0x55e306baae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42846/0 (socket says 172.21.15.37:42846) 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe04759e640 1 -- 172.21.15.37:0/2745254724 learned_addr learned my addr 172.21.15.37:0/2745254724 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe046d9d640 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b4e0a0 0x55e306baa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe04659c640 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b979d0 0x55e306baa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe04759e640 1 -- 172.21.15.37:0/2745254724 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b4e0a0 msgr2=0x55e306baa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe04759e640 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b4e0a0 0x55e306baa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe04759e640 1 -- 172.21.15.37:0/2745254724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b979d0 msgr2=0x55e306baa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe04759e640 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b979d0 0x55e306baa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe04759e640 1 -- 172.21.15.37:0/2745254724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e30693c8e0 con 0x55e306b982f0 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe04759e640 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b982f0 0x55e306baae00 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fe04000d6f0 tx=0x7fe040004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c73cb27da708d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe025ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe025ffb640 1 -- 172.21.15.37:0/2745254724 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040015040 con 0x55e306b982f0 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe025ffb640 1 -- 172.21.15.37:0/2745254724 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe040026cd0 con 0x55e306b982f0 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- 172.21.15.37:0/2745254724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0380512e0 con 0x55e306b982f0 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe025ffb640 1 -- 172.21.15.37:0/2745254724 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=103) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe040008520 con 0x55e306b982f0 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 10 client.9978 fetch_fsmap learned FSMap version 103 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- 172.21.15.37:0/2745254724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=103,monmap=2+}) v3 -- 0x55e3069f48f0 con 0x55e306b982f0 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe025ffb640 1 -- 172.21.15.37:0/2745254724 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0380512e0 con 0x55e306b982f0 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe025ffb640 1 -- 172.21.15.37:0/2745254724 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 103) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe040023070 con 0x55e306b982f0 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 10 client.9978 fetch_fsmap finished waiting for FSMap version 103 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 10 client.9978 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.a 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 20 client.9978 populate_metadata read hostname 'smithi037' 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55e306baba50 0x55e306bfff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 4 client.9978 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T09:04:12.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe0501b7000 1 -- 172.21.15.37:0/2745254724 --> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e3069a8530 con 0x55e306baba50 2024-04-25T09:04:12.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.573+0000 7fe046d9d640 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55e306baba50 0x55e306bfff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.577+0000 7fe046d9d640 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55e306baba50 0x55e306bfff60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b76280aa6a533a15 server_cookie=916630f5d29a69cc in_seq=0 out_seq=0 2024-04-25T09:04:12.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.577+0000 7fe025ffb640 10 client.9978 ms_handle_connect on v2:172.21.15.37:6834/2159545482 2024-04-25T09:04:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe025ffb640 1 -- 172.21.15.37:0/2745254724 <== mds.0 v2:172.21.15.37:6834/2159545482 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e3069a8530 con 0x55e306baba50 2024-04-25T09:04:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe025ffb640 10 client.9978 handle_command_reply: tid=0 2024-04-25T09:04:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 1 client.9978 shutdown 2024-04-25T09:04:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 1 -- 172.21.15.37:0/2745254724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b982f0 msgr2=0x55e306baae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b982f0 0x55e306baae00 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7fe04000d6f0 tx=0x7fe040004a50 comp rx=0 tx=0).stop 2024-04-25T09:04:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 1 -- 172.21.15.37:0/2745254724 shutdown_connections 2024-04-25T09:04:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.37:6834/2159545482,v1:172.21.15.37:6835/2159545482] conn(0x55e306baba50 0x55e306bfff60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e306b982f0 0x55e306baae00 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55e306b979d0 0x55e306baa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 1 --2- 172.21.15.37:0/2745254724 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55e306b4e0a0 0x55e306baa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe025ffb640 0 client.9978 ms_handle_reset on v2:172.21.15.37:6834/2159545482 2024-04-25T09:04:12.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 1 -- 172.21.15.37:0/2745254724 >> 172.21.15.37:0/2745254724 conn(0x55e306b4a020 msgr2=0x55e306b960d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 1 -- 172.21.15.37:0/2745254724 shutdown_connections 2024-04-25T09:04:12.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 1 -- 172.21.15.37:0/2745254724 wait complete. 2024-04-25T09:04:12.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe0501b7000 20 client.9978 trim_cache size 0 max 16384 2024-04-25T09:04:12.580 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:12.580 INFO:teuthology.orchestra.run.smithi037.stdout: "message": "", 2024-04-25T09:04:12.580 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0 2024-04-25T09:04:12.580 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:12.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 -- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe028066f60 msgr2=0x7fe028069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe028066f60 0x7fe028069420 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe0300ae000 tx=0x7fe030008040 comp rx=0 tx=0).stop 2024-04-25T09:04:12.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 -- 172.21.15.37:0/886370422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0480b1f20 msgr2=0x7fe048159040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0480b1f20 0x7fe048159040 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fe03800c480 tx=0x7fe03800c940 comp rx=0 tx=0).stop 2024-04-25T09:04:12.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 -- 172.21.15.37:0/886370422 shutdown_connections 2024-04-25T09:04:12.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe028066f60 0x7fe028069420 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe0480cce90 0x7fe048152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe0480b4430 0x7fe04806cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 --2- 172.21.15.37:0/886370422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0480b1f20 0x7fe048159040 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 -- 172.21.15.37:0/886370422 >> 172.21.15.37:0/886370422 conn(0x7fe0480d15e0 msgr2=0x7fe0480cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 -- 172.21.15.37:0/886370422 shutdown_connections 2024-04-25T09:04:12.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.589+0000 7fe04d04a640 1 -- 172.21.15.37:0/886370422 wait complete. 2024-04-25T09:04:12.596 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:04:12.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76fa25c640 1 Processor -- start 2024-04-25T09:04:12.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76fa25c640 1 -- start start 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76fa25c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f76f40d32d0 0x7f76f406cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76fa25c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f76f415e9f0 0x7f76f40cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76fa25c640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f415f050 con 0x7f76f40ce7a0 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76fa25c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f40b6ae0 con 0x7f76f40d32d0 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76fa25c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f406c1c0 con 0x7f76f415e9f0 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76f37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f76f40d32d0 0x7f76f406cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76f37fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f76f40d32d0 0x7f76f406cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42854/0 (socket says 172.21.15.37:42854) 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76f37fe640 1 -- 172.21.15.37:0/1696933514 learned_addr learned my addr 172.21.15.37:0/1696933514 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76f37fe640 1 -- 172.21.15.37:0/1696933514 >> v2:172.21.15.203:3300/0 conn(0x7f76f415e9f0 msgr2=0x7f76f40cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76f37fe640 1 --2- 172.21.15.37:0/1696933514 >> v2:172.21.15.203:3300/0 conn(0x7f76f415e9f0 0x7f76f40cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76f37fe640 1 -- 172.21.15.37:0/1696933514 >> v1:172.21.15.203:6789/0 conn(0x7f76f40ce7a0 legacy=0x7f76f415e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.721+0000 7f76f37fe640 1 -- 172.21.15.37:0/1696933514 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f406c410 con 0x7f76f40d32d0 2024-04-25T09:04:12.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f37fe640 1 --2- 172.21.15.37:0/1696933514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f40d32d0 0x7f76f406cef0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f76f40d1960 tx=0x7f76e401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb37cb86cb4a0951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f27fc640 1 -- 172.21.15.37:0/1696933514 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e40b5070 con 0x7f76f40d32d0 2024-04-25T09:04:12.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f27fc640 1 -- 172.21.15.37:0/1696933514 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e40a89d0 con 0x7f76f40d32d0 2024-04-25T09:04:12.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f27fc640 1 -- 172.21.15.37:0/1696933514 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e40a8da0 con 0x7f76f40d32d0 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 -- 172.21.15.37:0/1696933514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f40d32d0 msgr2=0x7f76f406cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 --2- 172.21.15.37:0/1696933514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f40d32d0 0x7f76f406cef0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f76f40d1960 tx=0x7f76e401ecd0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 -- 172.21.15.37:0/1696933514 shutdown_connections 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 --2- 172.21.15.37:0/1696933514 >> v2:172.21.15.203:3300/0 conn(0x7f76f415e9f0 0x7f76f40cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 --2- 172.21.15.37:0/1696933514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f40d32d0 0x7f76f406cef0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 -- 172.21.15.37:0/1696933514 >> 172.21.15.37:0/1696933514 conn(0x7f76f40d39f0 msgr2=0x7f76f40d3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 -- 172.21.15.37:0/1696933514 shutdown_connections 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 -- 172.21.15.37:0/1696933514 wait complete. 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 Processor -- start 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 -- start start 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76f40ce7a0 0x7f76f41f3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f40d32d0 0x7f76f4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f415e9f0 0x7f76f41587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40d36b0 con 0x7f76f415e9f0 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f406c9d0 con 0x7f76f40d32d0 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76fa25c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f40d1bc0 con 0x7f76f40ce7a0 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f415e9f0 0x7f76f41587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f415e9f0 0x7f76f41587a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42866/0 (socket says 172.21.15.37:42866) 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f3fff640 1 -- 172.21.15.37:0/2143687345 learned_addr learned my addr 172.21.15.37:0/2143687345 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f2ffd640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f40d32d0 0x7f76f4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f37fe640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76f40ce7a0 0x7f76f41f3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f3fff640 1 -- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76f40ce7a0 msgr2=0x7f76f41f3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f3fff640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76f40ce7a0 0x7f76f41f3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f3fff640 1 -- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f40d32d0 msgr2=0x7f76f4157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f3fff640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f40d32d0 0x7f76f4157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f3fff640 1 -- 172.21.15.37:0/2143687345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f4146670 con 0x7f76f415e9f0 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.725+0000 7f76f37fe640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76f40ce7a0 0x7f76f41f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:12.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76f3fff640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f415e9f0 0x7f76f41587a0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00b390 tx=0x7f76ec00b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76ebfff640 1 -- 172.21.15.37:0/2143687345 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec015020 con 0x7f76f415e9f0 2024-04-25T09:04:12.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76ebfff640 1 -- 172.21.15.37:0/2143687345 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ec011830 con 0x7f76f415e9f0 2024-04-25T09:04:12.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76fa25c640 1 -- 172.21.15.37:0/2143687345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f4146950 con 0x7f76f415e9f0 2024-04-25T09:04:12.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76ebfff640 1 -- 172.21.15.37:0/2143687345 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ec011c20 con 0x7f76f415e9f0 2024-04-25T09:04:12.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76fa25c640 1 -- 172.21.15.37:0/2143687345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f4146f10 con 0x7f76f415e9f0 2024-04-25T09:04:12.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76fa25c640 1 -- 172.21.15.37:0/2143687345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f415a670 con 0x7f76f415e9f0 2024-04-25T09:04:12.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76ebfff640 1 -- 172.21.15.37:0/2143687345 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f76ec018020 con 0x7f76f415e9f0 2024-04-25T09:04:12.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76ebfff640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76d8066f60 0x7f76d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76ebfff640 1 -- 172.21.15.37:0/2143687345 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f76ec0850b0 con 0x7f76f415e9f0 2024-04-25T09:04:12.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76f37fe640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76d8066f60 0x7f76d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.729+0000 7f76f37fe640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76d8066f60 0x7f76d8069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f76e40b2000 tx=0x7f76e4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.733+0000 7f76ebfff640 1 -- 172.21.15.37:0/2143687345 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f76ec01f020 con 0x7f76f415e9f0 2024-04-25T09:04:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.933+0000 7f76fa25c640 1 -- 172.21.15.37:0/2143687345 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f76f40cdd70 con 0x7f76f415e9f0 2024-04-25T09:04:12.923 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T09:04:12.933+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:04:12.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.937+0000 7f76ebfff640 1 -- 172.21.15.37:0/2143687345 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v104) v1 ==== 161+0+0 (secure 0 0 0) 0x7f76ec0172c0 con 0x7f76f415e9f0 2024-04-25T09:04:12.926 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:04:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 -- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76d8066f60 msgr2=0x7f76d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76d8066f60 0x7f76d8069420 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f76e40b2000 tx=0x7f76e4008040 comp rx=0 tx=0).stop 2024-04-25T09:04:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 -- 172.21.15.37:0/2143687345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f415e9f0 msgr2=0x7f76f41587a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f415e9f0 0x7f76f41587a0 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f76ec00b390 tx=0x7f76ec00b850 comp rx=0 tx=0).stop 2024-04-25T09:04:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 -- 172.21.15.37:0/2143687345 shutdown_connections 2024-04-25T09:04:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f76d8066f60 0x7f76d8069420 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f76f415e9f0 0x7f76f41587a0 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f76f40d32d0 0x7f76f4157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 --2- 172.21.15.37:0/2143687345 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f76f40ce7a0 0x7f76f41f3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 -- 172.21.15.37:0/2143687345 >> 172.21.15.37:0/2143687345 conn(0x7f76f40d39f0 msgr2=0x7f76f40b8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 -- 172.21.15.37:0/2143687345 shutdown_connections 2024-04-25T09:04:12.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.941+0000 7f76fa25c640 1 -- 172.21.15.37:0/2143687345 wait complete. 2024-04-25T09:04:12.938 INFO:tasks.ceph.mds.a.smithi037.stderr: -2124> 2024-04-25T09:04:04.993+0000 7fbfbe817640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:04:12.947 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2024-04-25T09:04:12.971 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T09:04:12.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 -- start start 2024-04-25T09:04:12.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 Processor -- start 2024-04-25T09:04:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebdf89800 0x557ebe142580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebdf89400 0x557ebe142b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebdf89000 0x557ebe143080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7a180 con 0x557ebdf89800 2024-04-25T09:04:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7a300 con 0x557ebdf89000 2024-04-25T09:04:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7a480 con 0x557ebdf89400 2024-04-25T09:04:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebdf89800 0x557ebe142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebdf89800 0x557ebe142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42908/0 (socket says 172.21.15.37:42908) 2024-04-25T09:04:12.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 -- 172.21.15.37:0/3373344881 learned_addr learned my addr 172.21.15.37:0/3373344881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 -- 172.21.15.37:0/3373344881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebdf89400 msgr2=0x557ebe142b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e2689640 1 --2- 172.21.15.37:0/3373344881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebdf89000 0x557ebe143080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1687640 1 --2- 172.21.15.37:0/3373344881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebdf89400 0x557ebe142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 --2- 172.21.15.37:0/3373344881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebdf89400 0x557ebe142b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 -- 172.21.15.37:0/3373344881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebdf89000 msgr2=0x557ebe143080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 --2- 172.21.15.37:0/3373344881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebdf89000 0x557ebe143080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 -- 172.21.15.37:0/3373344881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ebed36b40 con 0x557ebdf89800 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 --2- 172.21.15.37:0/3373344881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebdf89800 0x557ebe142580 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x557ebeddf3b0 tx=0x557ebe04b080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=176917790b640da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22df683640 1 -- 172.21.15.37:0/3373344881 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557ebdfbe4e0 con 0x557ebdf89800 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22df683640 1 -- 172.21.15.37:0/3373344881 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ebdfd21c0 con 0x557ebdf89800 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22df683640 1 -- 172.21.15.37:0/3373344881 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557ebdfbeb60 con 0x557ebdf89800 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 -- 172.21.15.37:0/3373344881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ebed36d20 con 0x557ebdf89800 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 -- 172.21.15.37:0/3373344881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557ebed36f00 con 0x557ebdf89800 2024-04-25T09:04:12.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 4 waiting for MDS map... 2024-04-25T09:04:12.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22df683640 1 -- 172.21.15.37:0/3373344881 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x557ebee88600 con 0x557ebdf89800 2024-04-25T09:04:12.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22df683640 1 -- 172.21.15.37:0/3373344881 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 104) v1 ==== 2723+0+0 (secure 0 0 0) 0x557ebdfdf180 con 0x557ebdf89800 2024-04-25T09:04:12.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 4 Got MDS map 104 2024-04-25T09:04:12.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 10 main: JournalTool::main 2024-04-25T09:04:12.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 4 main: JournalTool: connecting to RADOS... 2024-04-25T09:04:12.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 Processor -- start 2024-04-25T09:04:12.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 -- start start 2024-04-25T09:04:12.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8ec00 0x557ebe143600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f000 0x557ebe143b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8f400 0x557ebe144100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7a600 con 0x557ebee8f000 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7a780 con 0x557ebee8ec00 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e36c9400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7a900 con 0x557ebee8f400 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f000 0x557ebe143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f000 0x557ebe143b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42922/0 (socket says 172.21.15.37:42922) 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 -- 172.21.15.37:0/1634622363 learned_addr learned my addr 172.21.15.37:0/1634622363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e2689640 1 --2- 172.21.15.37:0/1634622363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8ec00 0x557ebe143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1687640 1 --2- 172.21.15.37:0/1634622363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8f400 0x557ebe144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 -- 172.21.15.37:0/1634622363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8f400 msgr2=0x557ebe144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 --2- 172.21.15.37:0/1634622363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8f400 0x557ebe144100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 -- 172.21.15.37:0/1634622363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8ec00 msgr2=0x557ebe143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 --2- 172.21.15.37:0/1634622363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8ec00 0x557ebe143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1e88640 1 -- 172.21.15.37:0/1634622363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ebed36d20 con 0x557ebee8f000 2024-04-25T09:04:12.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e2689640 1 --2- 172.21.15.37:0/1634622363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8ec00 0x557ebe143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:12.997+0000 7f22e1687640 1 --2- 172.21.15.37:0/1634622363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8f400 0x557ebe144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e1e88640 1 --2- 172.21.15.37:0/1634622363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f000 0x557ebe143b80 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x557ebeea2660 tx=0x557ebe04ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64110bfc261119bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22dd67f640 1 -- 172.21.15.37:0/1634622363 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557ebdfbe340 con 0x557ebee8f000 2024-04-25T09:04:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22dd67f640 1 -- 172.21.15.37:0/1634622363 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ebdfd2e00 con 0x557ebee8f000 2024-04-25T09:04:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22dd67f640 1 -- 172.21.15.37:0/1634622363 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557ebdfbe4e0 con 0x557ebee8f000 2024-04-25T09:04:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 -- 172.21.15.37:0/1634622363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f000 msgr2=0x557ebe143b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 --2- 172.21.15.37:0/1634622363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f000 0x557ebe143b80 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x557ebeea2660 tx=0x557ebe04ca80 comp rx=0 tx=0).stop 2024-04-25T09:04:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 -- 172.21.15.37:0/1634622363 shutdown_connections 2024-04-25T09:04:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 --2- 172.21.15.37:0/1634622363 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8f400 0x557ebe144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 --2- 172.21.15.37:0/1634622363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f000 0x557ebe143b80 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 --2- 172.21.15.37:0/1634622363 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8ec00 0x557ebe143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 -- 172.21.15.37:0/1634622363 >> 172.21.15.37:0/1634622363 conn(0x557ebdf88c00 msgr2=0x557ebdfb7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 -- 172.21.15.37:0/1634622363 shutdown_connections 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 -- 172.21.15.37:0/1634622363 wait complete. 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 Processor -- start 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 -- start start 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8ec00 0x557ebe145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8f000 0x557ebe144c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f400 0x557ebe144680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7bc80 con 0x557ebee8f400 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7bb00 con 0x557ebee8f000 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7b980 con 0x557ebee8ec00 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e1687640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f400 0x557ebe144680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e1687640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f400 0x557ebe144680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42926/0 (socket says 172.21.15.37:42926) 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e1687640 1 -- 172.21.15.37:0/3759408322 learned_addr learned my addr 172.21.15.37:0/3759408322 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e2689640 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8ec00 0x557ebe145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e1e88640 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8f000 0x557ebe144c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e1687640 1 -- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8ec00 msgr2=0x557ebe145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e1687640 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8ec00 0x557ebe145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e1687640 1 -- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8f000 msgr2=0x557ebe144c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e1687640 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8f000 0x557ebe144c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e1687640 1 -- 172.21.15.37:0/3759408322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ebed37c20 con 0x557ebee8f400 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e2689640 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8ec00 0x557ebe145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e1687640 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f400 0x557ebe144680 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x557ebedde9c0 tx=0x557ebed40000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22dbe7c640 1 -- 172.21.15.37:0/3759408322 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557ebdfbe9c0 con 0x557ebee8f400 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22dbe7c640 1 -- 172.21.15.37:0/3759408322 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ebdfd2a80 con 0x557ebee8f400 2024-04-25T09:04:13.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 -- 172.21.15.37:0/3759408322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557ebed37e00 con 0x557ebee8f400 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22dbe7c640 1 -- 172.21.15.37:0/3759408322 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557ebdfbe340 con 0x557ebee8f400 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 1 -- 172.21.15.37:0/3759408322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ebed361e0 con 0x557ebee8f400 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.001+0000 7f22e36c9400 4 main: JournalTool: resolving pool 17 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22dbe7c640 1 -- 172.21.15.37:0/3759408322 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x557ebef02c00 con 0x557ebee8f400 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22dbe7c640 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x557ebe396000 0x557ebe145700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22dbe7c640 1 -- 172.21.15.37:0/3759408322 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x557ebe2b8600 con 0x557ebee8f400 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 4 main: JournalTool: creating IoCtx.. 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 4 main: Executing for rank 0 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e2689640 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x557ebe396000 0x557ebe145700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 -- start start 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 Processor -- start 2024-04-25T09:04:13.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e2689640 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x557ebe396000 0x557ebe145700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x557ebe096780 tx=0x557ebe3a3800 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8fc00 0x557ebe144100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebe397000 0x557ebe143b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebe397400 0x557ebe143600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7a780 con 0x557ebe397400 2024-04-25T09:04:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7a900 con 0x557ebee8fc00 2024-04-25T09:04:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x557ebee7b680 con 0x557ebe397000 2024-04-25T09:04:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e1687640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebe397400 0x557ebe143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e1687640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebe397400 0x557ebe143600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42934/0 (socket says 172.21.15.37:42934) 2024-04-25T09:04:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e1687640 1 -- 172.21.15.37:0/2401449288 learned_addr learned my addr 172.21.15.37:0/2401449288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e2689640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8fc00 0x557ebe144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e1e88640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebe397000 0x557ebe143b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e1687640 1 -- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebe397000 msgr2=0x557ebe143b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e1687640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebe397000 0x557ebe143b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e1687640 1 -- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8fc00 msgr2=0x557ebe144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e1687640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8fc00 0x557ebe144100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e1687640 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ebe3ee780 con 0x557ebe397400 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e1e88640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebe397000 0x557ebe143b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e2689640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8fc00 0x557ebe144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e1687640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebe397400 0x557ebe143600 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x557ebe2fb1d0 tx=0x557ebeef9320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8a824676b46bd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22d8e76640 1 -- 172.21.15.37:0/2401449288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557ebe32da00 con 0x557ebe397400 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22d8e76640 1 -- 172.21.15.37:0/2401449288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557ebdfd3340 con 0x557ebe397400 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22d8e76640 1 -- 172.21.15.37:0/2401449288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557ebe32dba0 con 0x557ebe397400 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557ebe3ee960 con 0x557ebe397400 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557ebe3eeb40 con 0x557ebe397400 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 4 waiting for MDS map... 2024-04-25T09:04:13.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22d8e76640 1 -- 172.21.15.37:0/2401449288 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x557ebe3fc800 con 0x557ebe397400 2024-04-25T09:04:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22d8e76640 1 -- 172.21.15.37:0/2401449288 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 104) v1 ==== 2723+0+0 (secure 0 0 0) 0x557ebe408000 con 0x557ebe397400 2024-04-25T09:04:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 4 Got MDS map 104 2024-04-25T09:04:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x557ebe397800 0x557ebe41c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:1 17.f 17:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x557ebe397c00 con 0x557ebe397800 2024-04-25T09:04:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e2689640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x557ebe397800 0x557ebe41c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.005+0000 7f22e2689640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x557ebe397800 0x557ebe41c000 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=a90ab4395755880 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.009+0000 7f22e2689640 1 -- 172.21.15.37:0/2401449288 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v127'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x557ebe104000 con 0x557ebe397800 2024-04-25T09:04:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.009+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x557ebe405400 0x557ebe41c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.009+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:2 17.14 17:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x557ebe405800 con 0x557ebe405400 2024-04-25T09:04:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.009+0000 7f22e1e88640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x557ebe405400 0x557ebe41c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.009+0000 7f22e1e88640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x557ebe405400 0x557ebe41c580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=150c0938be332cb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e1e88640 1 -- 172.21.15.37:0/2401449288 <== osd.3 v2:172.21.15.37:6800/4126478460 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v127'41 uv41 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x557ebe104500 con 0x557ebe405400 2024-04-25T09:04:13.007 INFO:teuthology.orchestra.run.smithi037.stdout:writing EResetJournal entry 2024-04-25T09:04:13.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:3 17.14 17:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x557ebe405c00 con 0x557ebe405400 2024-04-25T09:04:13.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x557ebe434000 0x557ebe41cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:4 17.1b 17:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x557ebe434400 con 0x557ebe434000 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x557ebe434800 0x557ebe41d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:5 17.b 17:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x557ebe434c00 con 0x557ebe434800 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x557ebe435000 0x557ebe41d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:6 17.1f 17:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x557ebe435400 con 0x557ebe435000 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:7 17.1a 17:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x557ebe435800 con 0x557ebe434800 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:8 17.1e 17:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x557ebe435c00 con 0x557ebe397800 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e1e88640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x557ebe434800 0x557ebe41d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e2689640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x557ebe434000 0x557ebe41cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e1687640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x557ebe435000 0x557ebe41d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e1e88640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x557ebe434800 0x557ebe41d080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=a5682e449fcd2428 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e1687640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x557ebe435000 0x557ebe41d600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=9f35522e4e7d0fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.013+0000 7f22e2689640 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x557ebe434000 0x557ebe41cb00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=9d4a862b1a3577f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.017+0000 7f22e2689640 1 -- 172.21.15.37:0/2401449288 <== osd.1 v2:172.21.15.37:6816/2944303862 2 ==== osd_op_reply(8 200.00000006 [delete] v127'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x557ebe104000 con 0x557ebe397800 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.017+0000 7f22e1e88640 1 -- 172.21.15.37:0/2401449288 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== osd_op_reply(7 200.00000005 [delete] v127'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x557ebe104500 con 0x557ebe434800 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.017+0000 7f22e1e88640 1 -- 172.21.15.37:0/2401449288 <== osd.3 v2:172.21.15.37:6800/4126478460 2 ==== osd_op_reply(3 200.00000001 [delete] v127'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x557ebe104500 con 0x557ebe405400 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.017+0000 7f22e1e88640 1 -- 172.21.15.37:0/2401449288 <== osd.0 v2:172.21.15.37:6824/1116141266 2 ==== osd_op_reply(5 200.00000003 [delete] v127'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x557ebe104500 con 0x557ebe434800 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.017+0000 7f22e2689640 1 -- 172.21.15.37:0/2401449288 <== osd.4 v2:172.21.15.203:6802/714177927 1 ==== osd_op_reply(4 200.00000002 [delete] v127'3 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x557ebe104000 con 0x557ebe434000 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.017+0000 7f22d7673640 1 -- 172.21.15.37:0/2401449288 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:9 17.14 17:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x557ebe396c00 con 0x557ebe405400 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.017+0000 7f22e1687640 1 -- 172.21.15.37:0/2401449288 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(6 200.00000004 [delete] v127'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x557ebe104780 con 0x557ebe435000 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e1e88640 1 -- 172.21.15.37:0/2401449288 <== osd.3 v2:172.21.15.37:6800/4126478460 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v127'43 uv43 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x557ebe104500 con 0x557ebe405400 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 4 Successfully wrote new journal pointer and header for rank 8:0 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x557ebe434800 msgr2=0x557ebe41d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x557ebe434800 0x557ebe41d080 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x557ebe397800 msgr2=0x557ebe41c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x557ebe397800 0x557ebe41c000 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x557ebe405400 msgr2=0x557ebe41c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x557ebe405400 0x557ebe41c580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x557ebe434000 msgr2=0x557ebe41cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x557ebe434000 0x557ebe41cb00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x557ebe435000 msgr2=0x557ebe41d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x557ebe435000 0x557ebe41d600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e1e88640 1 -- 172.21.15.37:0/2401449288 reap_dead start 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e1e88640 1 -- 172.21.15.37:0/2401449288 reap_dead start 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e1e88640 1 -- 172.21.15.37:0/2401449288 reap_dead start 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebe397400 msgr2=0x557ebe143600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebe397400 0x557ebe143600 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x557ebe2fb1d0 tx=0x557ebeef9320 comp rx=0 tx=0).stop 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 shutdown_connections 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/2401449288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebe397400 0x557ebe143600 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 >> 172.21.15.37:0/2401449288 conn(0x557ebee8f800 msgr2=0x557ebe3ea000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 shutdown_connections 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/2401449288 wait complete. 2024-04-25T09:04:13.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x557ebe396000 msgr2=0x557ebe145700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x557ebe396000 0x557ebe145700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x557ebe096780 tx=0x557ebe3a3800 comp rx=0 tx=0).stop 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/3759408322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f400 msgr2=0x557ebe144680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f400 0x557ebe144680 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x557ebedde9c0 tx=0x557ebed40000 comp rx=0 tx=0).stop 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/3759408322 shutdown_connections 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x557ebe396000 0x557ebe145700 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebee8f400 0x557ebe144680 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebee8f000 0x557ebe144c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/3759408322 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebee8ec00 0x557ebe145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/3759408322 >> 172.21.15.37:0/3759408322 conn(0x557ebdf88c00 msgr2=0x557ebdfb7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/3759408322 shutdown_connections 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/3759408322 wait complete. 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/3373344881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebdf89800 msgr2=0x557ebe142580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/3373344881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebdf89800 0x557ebe142580 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x557ebeddf3b0 tx=0x557ebe04b080 comp rx=0 tx=0).stop 2024-04-25T09:04:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/3373344881 shutdown_connections 2024-04-25T09:04:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/3373344881 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x557ebdf89000 0x557ebe143080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/3373344881 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x557ebdf89400 0x557ebe142b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 --2- 172.21.15.37:0/3373344881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ebdf89800 0x557ebe142580 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/3373344881 >> 172.21.15.37:0/3373344881 conn(0x557ebdf89c00 msgr2=0x557ebdfb7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/3373344881 shutdown_connections 2024-04-25T09:04:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.021+0000 7f22e36c9400 1 -- 172.21.15.37:0/3373344881 wait complete. 2024-04-25T09:04:13.017 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.070291, result: writing EResetJournal entry 2024-04-25T09:04:13.017 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-04-25T09:04:13.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 -- start start 2024-04-25T09:04:13.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 Processor -- start 2024-04-25T09:04:13.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333c507000 0x56333c6c0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333c507400 0x56333c6c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333c507800 0x56333c6c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56333d2e2600 con 0x56333c507400 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56333d2e2480 con 0x56333c507800 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56333d2e2300 con 0x56333c507000 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333c507400 0x56333c6c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333c507400 0x56333c6c0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42954/0 (socket says 172.21.15.37:42954) 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 -- 172.21.15.37:0/2019501117 learned_addr learned my addr 172.21.15.37:0/2019501117 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc78409640 1 --2- 172.21.15.37:0/2019501117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333c507800 0x56333c6c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 -- 172.21.15.37:0/2019501117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333c507000 msgr2=0x56333c6c0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77c08640 1 --2- 172.21.15.37:0/2019501117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333c507000 0x56333c6c0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 --2- 172.21.15.37:0/2019501117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333c507000 0x56333c6c0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 -- 172.21.15.37:0/2019501117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333c507800 msgr2=0x56333c6c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 --2- 172.21.15.37:0/2019501117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333c507800 0x56333c6c1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 -- 172.21.15.37:0/2019501117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56333c52c1e0 con 0x56333c507400 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc78409640 1 --2- 172.21.15.37:0/2019501117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333c507800 0x56333c6c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 --2- 172.21.15.37:0/2019501117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333c507400 0x56333c6c0b00 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x56333d355410 tx=0x56333d2d4000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18654753e0c6f4f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc75403640 1 -- 172.21.15.37:0/2019501117 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56333c53c9c0 con 0x56333c507400 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc75403640 1 -- 172.21.15.37:0/2019501117 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56333c550540 con 0x56333c507400 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc75403640 1 -- 172.21.15.37:0/2019501117 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56333c53c4e0 con 0x56333c507400 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 -- 172.21.15.37:0/2019501117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56333c52d2c0 con 0x56333c507400 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 -- 172.21.15.37:0/2019501117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56333c52c3c0 con 0x56333c507400 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 4 waiting for MDS map... 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc75403640 1 -- 172.21.15.37:0/2019501117 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x56333d414200 con 0x56333c507400 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc75403640 1 -- 172.21.15.37:0/2019501117 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 104) v1 ==== 2723+0+0 (secure 0 0 0) 0x56333c55d180 con 0x56333c507400 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 4 Got MDS map 104 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 4 datascan.main: Using metadata pool output 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 4 datascan.main: connecting to RADOS... 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 Processor -- start 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 -- start start 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333d2e7800 0x56333c6c1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333d2e7c00 0x56333c6c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d482000 0x56333c6c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56333d2e2180 con 0x56333d482000 2024-04-25T09:04:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56333c566480 con 0x56333d2e7c00 2024-04-25T09:04:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc794498c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56333c566900 con 0x56333d2e7800 2024-04-25T09:04:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d482000 0x56333c6c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d482000 0x56333c6c2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42956/0 (socket says 172.21.15.37:42956) 2024-04-25T09:04:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 -- 172.21.15.37:0/3972704441 learned_addr learned my addr 172.21.15.37:0/3972704441 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77c08640 1 --2- 172.21.15.37:0/3972704441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333d2e7c00 0x56333c6c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc78409640 1 --2- 172.21.15.37:0/3972704441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333d2e7800 0x56333c6c1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 -- 172.21.15.37:0/3972704441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333d2e7800 msgr2=0x56333c6c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 --2- 172.21.15.37:0/3972704441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333d2e7800 0x56333c6c1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 -- 172.21.15.37:0/3972704441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333d2e7c00 msgr2=0x56333c6c1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 --2- 172.21.15.37:0/3972704441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333d2e7c00 0x56333c6c1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 -- 172.21.15.37:0/3972704441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56333c52c1e0 con 0x56333d482000 2024-04-25T09:04:13.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc78409640 1 --2- 172.21.15.37:0/3972704441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333d2e7800 0x56333c6c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.109+0000 7fcc77407640 1 --2- 172.21.15.37:0/3972704441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d482000 0x56333c6c2100 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x56333d355e90 tx=0x56333d2d4300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd302508d2e22572 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc733ff640 1 -- 172.21.15.37:0/3972704441 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56333c53cb60 con 0x56333d482000 2024-04-25T09:04:13.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc733ff640 1 -- 172.21.15.37:0/3972704441 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56333c550c40 con 0x56333d482000 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc733ff640 1 -- 172.21.15.37:0/3972704441 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56333c53c340 con 0x56333d482000 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 -- 172.21.15.37:0/3972704441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d482000 msgr2=0x56333c6c2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3972704441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d482000 0x56333c6c2100 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x56333d355e90 tx=0x56333d2d4300 comp rx=0 tx=0).stop 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 -- 172.21.15.37:0/3972704441 shutdown_connections 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3972704441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d482000 0x56333c6c2100 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3972704441 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333d2e7c00 0x56333c6c1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3972704441 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333d2e7800 0x56333c6c1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 -- 172.21.15.37:0/3972704441 >> 172.21.15.37:0/3972704441 conn(0x56333d2e7400 msgr2=0x56333c534000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 -- 172.21.15.37:0/3972704441 shutdown_connections 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 -- 172.21.15.37:0/3972704441 wait complete. 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 Processor -- start 2024-04-25T09:04:13.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 -- start start 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333d2e7c00 0x56333c6c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d2e7800 0x56333c6c2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333d483000 0x56333c6c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x56333d2e3380 con 0x56333d2e7800 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x56333d2e3200 con 0x56333d483000 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x56333d2e3500 con 0x56333d2e7c00 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc77c08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d2e7800 0x56333c6c2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc77c08640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d2e7800 0x56333c6c2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42966/0 (socket says 172.21.15.37:42966) 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 learned_addr learned my addr 172.21.15.37:0/3918058397 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc77407640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333d483000 0x56333c6c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333d2e7c00 msgr2=0x56333c6c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc78409640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333d2e7c00 0x56333c6c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc77c08640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333d2e7c00 0x56333c6c3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333d483000 msgr2=0x56333c6c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc77c08640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333d483000 0x56333c6c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56333c52d4a0 con 0x56333d2e7800 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc77407640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333d483000 0x56333c6c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc77c08640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d2e7800 0x56333c6c2c00 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x56333d2d3e90 tx=0x56333c83a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc71bfc640 1 -- 172.21.15.37:0/3918058397 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56333c53cb60 con 0x56333d2e7800 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc71bfc640 1 -- 172.21.15.37:0/3918058397 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56333c551340 con 0x56333d2e7800 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc71bfc640 1 -- 172.21.15.37:0/3918058397 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56333c53c340 con 0x56333d2e7800 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56333c52da40 con 0x56333d2e7800 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56333c52dc20 con 0x56333d2e7800 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc794498c0 4 datascan.init: resolving metadata pool 17 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.113+0000 7fcc71bfc640 1 -- 172.21.15.37:0/3918058397 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x56333c834c00 con 0x56333d2e7800 2024-04-25T09:04:13.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc71bfc640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56333c83cc00 0x56333c6c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc71bfc640 1 -- 172.21.15.37:0/3918058397 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x56333d485000 con 0x56333d2e7800 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc794498c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc794498c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc794498c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56333d482c00 0x56333c6c2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:1 18.0 18:00000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d482800 con 0x56333d482c00 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc77407640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56333d482c00 0x56333c6c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc78409640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56333c83cc00 0x56333c6c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc77407640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56333d482c00 0x56333c6c2100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== osd_op_reply(1 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333d482c00 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc72bfe640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x56333c83dc00 0x56333c940000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:2 18.20 18:04000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c944000 con 0x56333c83dc00 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc78409640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56333c83cc00 0x56333c6c3700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x56333c5e2fc0 tx=0x56333c83b680 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc78409640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x56333c83dc00 0x56333c940000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.117+0000 7fcc78409640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x56333c83dc00 0x56333c940000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.121+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== osd_op_reply(2 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c83dc00 2024-04-25T09:04:13.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.121+0000 7fcc733ff640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x56333c944800 0x56333c940580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.121+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:3 18.10 18:08000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c944c00 con 0x56333c944800 2024-04-25T09:04:13.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.121+0000 7fcc77c08640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x56333c944800 0x56333c940580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.121+0000 7fcc77c08640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x56333c944800 0x56333c940580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(3 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616780 con 0x56333c944800 2024-04-25T09:04:13.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:4 18.30 18:0c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c944400 con 0x56333d482c00 2024-04-25T09:04:13.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.0 v2:172.21.15.37:6824/1116141266 2 ==== osd_op_reply(4 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333d482c00 2024-04-25T09:04:13.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:5 18.8 18:10000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945000 con 0x56333d482c00 2024-04-25T09:04:13.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.0 v2:172.21.15.37:6824/1116141266 3 ==== osd_op_reply(5 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333d482c00 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:6 18.28 18:14000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d483c00 con 0x56333c83dc00 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.5 v2:172.21.15.203:6808/213476662 2 ==== osd_op_reply(6 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c83dc00 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc733ff640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x56333c944400 0x56333c940b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:7 18.18 18:18000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d483400 con 0x56333c944400 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc77407640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x56333c944400 0x56333c940b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc77407640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x56333c944400 0x56333c940b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(7 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc72bfe640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x56333d483800 0x56333c941600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:8 18.38 18:1c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d2e7000 con 0x56333d483800 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc78409640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x56333d483800 0x56333c941600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc78409640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x56333d483800 0x56333c941600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.3 v2:172.21.15.37:6800/4126478460 1 ==== osd_op_reply(8 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333d483800 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:9 18.4 18:20000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d482800 con 0x56333d482c00 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.0 v2:172.21.15.37:6824/1116141266 4 ==== osd_op_reply(9 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333d482c00 2024-04-25T09:04:13.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:10 18.24 18:24000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c944000 con 0x56333d482c00 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.0 v2:172.21.15.37:6824/1116141266 5 ==== osd_op_reply(10 [pgnls start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333d482c00 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.125+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:11 18.14 18:28000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945400 con 0x56333c944800 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 <== osd.6 v2:172.21.15.203:6800/467003745 2 ==== osd_op_reply(11 [pgnls start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616780 con 0x56333c944800 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:12 18.34 18:2c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d2e7000 con 0x56333d482c00 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.0 v2:172.21.15.37:6824/1116141266 6 ==== osd_op_reply(12 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333d482c00 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:13 18.c 18:30000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945800 con 0x56333c83dc00 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.5 v2:172.21.15.203:6808/213476662 3 ==== osd_op_reply(13 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c83dc00 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc72bfe640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x56333d483c00 0x56333c941b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:14 18.2c 18:34000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c83d800 con 0x56333d483c00 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc77c08640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x56333d483c00 0x56333c941b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc77c08640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x56333d483c00 0x56333c941b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 <== osd.2 v2:172.21.15.37:6808/332967247 1 ==== osd_op_reply(14 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616780 con 0x56333d483c00 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:15 18.1c 18:38000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945c00 con 0x56333c944400 2024-04-25T09:04:13.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 2 ==== osd_op_reply(15 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc72bfe640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x56333c95c000 0x56333c942100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:16 18.3c 18:3c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95c400 con 0x56333c95c000 2024-04-25T09:04:13.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc77407640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x56333c95c000 0x56333c942100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.129+0000 7fcc77407640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x56333c95c000 0x56333c942100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== osd_op_reply(16 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c95c000 2024-04-25T09:04:13.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:17 18.2 18:40000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945000 con 0x56333c95c000 2024-04-25T09:04:13.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.7 v2:172.21.15.203:6824/2957998038 2 ==== osd_op_reply(17 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c95c000 2024-04-25T09:04:13.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:18 18.22 18:44000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95c800 con 0x56333d483800 2024-04-25T09:04:13.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.3 v2:172.21.15.37:6800/4126478460 2 ==== osd_op_reply(18 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333d483800 2024-04-25T09:04:13.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:19 18.12 18:48000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c944000 con 0x56333d483c00 2024-04-25T09:04:13.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 <== osd.2 v2:172.21.15.37:6808/332967247 2 ==== osd_op_reply(19 [pgnls start_epoch 127 out=81b] v126'22 uv22 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56333c616780 con 0x56333d483c00 2024-04-25T09:04:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:20 18.12 18:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d482400 con 0x56333d483c00 2024-04-25T09:04:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 <== osd.2 v2:172.21.15.37:6808/332967247 3 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x56333c616780 con 0x56333d483c00 2024-04-25T09:04:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc794498c0 10 datascan.operator(): handling object 1099511627778.0 2024-04-25T09:04:13.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:21 18.12 18:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d482000 con 0x56333d483c00 2024-04-25T09:04:13.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 <== osd.2 v2:172.21.15.37:6808/332967247 4 ==== osd_op_reply(21 10000000002.00000000 [call] v127'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56333c616780 con 0x56333d483c00 2024-04-25T09:04:13.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:22 18.32 18:4c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95c400 con 0x56333d483800 2024-04-25T09:04:13.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.3 v2:172.21.15.37:6800/4126478460 3 ==== osd_op_reply(22 [pgnls start_epoch 127 out=49b] v125'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333d483800 2024-04-25T09:04:13.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.133+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:23 18.a 18:50000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d2e7000 con 0x56333d483800 2024-04-25T09:04:13.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.3 v2:172.21.15.37:6800/4126478460 4 ==== osd_op_reply(23 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333d483800 2024-04-25T09:04:13.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc72bfe640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x56333c95cc00 0x56333c942680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:24 18.2a 18:54000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95d000 con 0x56333c95cc00 2024-04-25T09:04:13.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc78409640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x56333c95cc00 0x56333c942680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc78409640 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x56333c95cc00 0x56333c942680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.4 v2:172.21.15.203:6802/714177927 1 ==== osd_op_reply(24 [pgnls start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c95cc00 2024-04-25T09:04:13.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:25 18.1a 18:58000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d483400 con 0x56333d483800 2024-04-25T09:04:13.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.3 v2:172.21.15.37:6800/4126478460 5 ==== osd_op_reply(25 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333d483800 2024-04-25T09:04:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:26 18.3a 18:5c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95c800 con 0x56333d482c00 2024-04-25T09:04:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.0 v2:172.21.15.37:6824/1116141266 7 ==== osd_op_reply(26 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333d482c00 2024-04-25T09:04:13.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:27 18.6 18:60000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d482800 con 0x56333c95c000 2024-04-25T09:04:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.7 v2:172.21.15.203:6824/2957998038 3 ==== osd_op_reply(27 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c95c000 2024-04-25T09:04:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:28 18.26 18:64000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95c400 con 0x56333d483c00 2024-04-25T09:04:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 <== osd.2 v2:172.21.15.37:6808/332967247 5 ==== osd_op_reply(28 [pgnls start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616780 con 0x56333d483c00 2024-04-25T09:04:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:29 18.16 18:68000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96e000 con 0x56333d483800 2024-04-25T09:04:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.3 v2:172.21.15.37:6800/4126478460 6 ==== osd_op_reply(29 [pgnls start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333d483800 2024-04-25T09:04:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:30 18.36 18:6c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95d000 con 0x56333c944400 2024-04-25T09:04:13.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 3 ==== osd_op_reply(30 [pgnls start_epoch 127 out=81b] v125'33 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.137+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:31 18.36 18:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95d400 con 0x56333c944400 2024-04-25T09:04:13.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 4 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc794498c0 10 datascan.operator(): handling object 1099511628279.0 2024-04-25T09:04:13.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:32 18.36 18:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95d800 con 0x56333c944400 2024-04-25T09:04:13.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 5 ==== osd_op_reply(32 100000001f7.00000000 [call] v127'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:33 18.e 18:70000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c944c00 con 0x56333d482c00 2024-04-25T09:04:13.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.0 v2:172.21.15.37:6824/1116141266 8 ==== osd_op_reply(33 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333d482c00 2024-04-25T09:04:13.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:34 18.2e 18:74000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95dc00 con 0x56333d483800 2024-04-25T09:04:13.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.3 v2:172.21.15.37:6800/4126478460 7 ==== osd_op_reply(34 [pgnls start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333d483800 2024-04-25T09:04:13.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:35 18.1e 18:78000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96e400 con 0x56333c83dc00 2024-04-25T09:04:13.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.5 v2:172.21.15.203:6808/213476662 4 ==== osd_op_reply(35 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c83dc00 2024-04-25T09:04:13.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:36 18.3e 18:7c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d2e7000 con 0x56333d483800 2024-04-25T09:04:13.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.3 v2:172.21.15.37:6800/4126478460 8 ==== osd_op_reply(36 [pgnls start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333d483800 2024-04-25T09:04:13.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:37 18.1 18:80000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96e800 con 0x56333d483800 2024-04-25T09:04:13.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.3 v2:172.21.15.37:6800/4126478460 9 ==== osd_op_reply(37 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333d483800 2024-04-25T09:04:13.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:38 18.21 18:84000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945800 con 0x56333c83dc00 2024-04-25T09:04:13.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.5 v2:172.21.15.203:6808/213476662 5 ==== osd_op_reply(38 [pgnls start_epoch 127 out=49b] v125'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c83dc00 2024-04-25T09:04:13.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:39 18.11 18:88000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96ec00 con 0x56333c944400 2024-04-25T09:04:13.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.141+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 6 ==== osd_op_reply(39 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.145+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:40 18.31 18:8c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95c800 con 0x56333c95cc00 2024-04-25T09:04:13.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.145+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.4 v2:172.21.15.203:6802/714177927 2 ==== osd_op_reply(40 [pgnls start_epoch 127 out=81b] v125'15 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56333c616500 con 0x56333c95cc00 2024-04-25T09:04:13.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.145+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:41 18.31 18:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945400 con 0x56333c95cc00 2024-04-25T09:04:13.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.145+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.4 v2:172.21.15.203:6802/714177927 3 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x56333c616500 con 0x56333c95cc00 2024-04-25T09:04:13.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.145+0000 7fcc794498c0 10 datascan.operator(): handling object 1099511627780.1 2024-04-25T09:04:13.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.145+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:42 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c944000 con 0x56333c944400 2024-04-25T09:04:13.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.145+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 7 ==== osd_op_reply(42 10000000004.00000000 [call] v127'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.145+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:43 18.9 18:90000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945c00 con 0x56333d483800 2024-04-25T09:04:13.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.145+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.3 v2:172.21.15.37:6800/4126478460 10 ==== osd_op_reply(43 [pgnls start_epoch 127 out=49b] v125'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333d483800 2024-04-25T09:04:13.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.145+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:44 18.29 18:94000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95d000 con 0x56333c95c000 2024-04-25T09:04:13.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.149+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.7 v2:172.21.15.203:6824/2957998038 4 ==== osd_op_reply(44 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c95c000 2024-04-25T09:04:13.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.149+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:45 18.19 18:98000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96f000 con 0x56333c95cc00 2024-04-25T09:04:13.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.149+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.4 v2:172.21.15.203:6802/714177927 4 ==== osd_op_reply(45 [pgnls start_epoch 127 out=81b] v126'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56333c616500 con 0x56333c95cc00 2024-04-25T09:04:13.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.149+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:46 18.19 18:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c83d800 con 0x56333c95cc00 2024-04-25T09:04:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.149+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.4 v2:172.21.15.203:6802/714177927 5 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x56333c616500 con 0x56333c95cc00 2024-04-25T09:04:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.149+0000 7fcc794498c0 10 datascan.operator(): handling object 1099511628280.0 2024-04-25T09:04:13.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.149+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:47 18.19 18:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96ec00 con 0x56333c95cc00 2024-04-25T09:04:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.153+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.4 v2:172.21.15.203:6802/714177927 6 ==== osd_op_reply(47 100000001f8.00000000 [call] v127'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56333c616500 con 0x56333c95cc00 2024-04-25T09:04:13.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.153+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:48 18.39 18:9c000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96e000 con 0x56333d482c00 2024-04-25T09:04:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.153+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.0 v2:172.21.15.37:6824/1116141266 9 ==== osd_op_reply(48 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333d482c00 2024-04-25T09:04:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.153+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:49 18.5 18:a0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96f400 con 0x56333c944400 2024-04-25T09:04:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.153+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 8 ==== osd_op_reply(49 [pgnls start_epoch 127 out=81b] v127'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.153+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:50 18.5 18:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96e400 con 0x56333c944400 2024-04-25T09:04:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.153+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 9 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.153+0000 7fcc794498c0 10 datascan.operator(): handling object 1099511627780.0 2024-04-25T09:04:13.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.153+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:51 18.5 18:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96e800 con 0x56333c944400 2024-04-25T09:04:13.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.153+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 10 ==== osd_op_reply(51 10000000004.00000000 [call] v127'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.153+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:52 18.25 18:a4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96f800 con 0x56333c944800 2024-04-25T09:04:13.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.157+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 <== osd.6 v2:172.21.15.203:6800/467003745 3 ==== osd_op_reply(52 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616780 con 0x56333c944800 2024-04-25T09:04:13.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.157+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:53 18.15 18:a8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96f000 con 0x56333c83dc00 2024-04-25T09:04:13.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.157+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.5 v2:172.21.15.203:6808/213476662 6 ==== osd_op_reply(53 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c83dc00 2024-04-25T09:04:13.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.157+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:54 18.35 18:ac000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95d400 con 0x56333c95cc00 2024-04-25T09:04:13.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.157+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.4 v2:172.21.15.203:6802/714177927 7 ==== osd_op_reply(54 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c95cc00 2024-04-25T09:04:13.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.157+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:55 18.d 18:b0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96ec00 con 0x56333c95cc00 2024-04-25T09:04:13.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.157+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.4 v2:172.21.15.203:6802/714177927 8 ==== osd_op_reply(55 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c95cc00 2024-04-25T09:04:13.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.157+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:56 18.2d 18:b4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d482800 con 0x56333c944800 2024-04-25T09:04:13.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.157+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 <== osd.6 v2:172.21.15.203:6800/467003745 4 ==== osd_op_reply(56 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616780 con 0x56333c944800 2024-04-25T09:04:13.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:57 18.1d 18:b8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c83d800 con 0x56333d482c00 2024-04-25T09:04:13.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.0 v2:172.21.15.37:6824/1116141266 10 ==== osd_op_reply(57 [pgnls start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333d482c00 2024-04-25T09:04:13.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:58 18.3d 18:bc000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d483400 con 0x56333d483800 2024-04-25T09:04:13.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.3 v2:172.21.15.37:6800/4126478460 11 ==== osd_op_reply(58 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333d483800 2024-04-25T09:04:13.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:59 18.3 18:c0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945800 con 0x56333c944400 2024-04-25T09:04:13.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 11 ==== osd_op_reply(59 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:60 18.23 18:c4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945000 con 0x56333d483c00 2024-04-25T09:04:13.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 <== osd.2 v2:172.21.15.37:6808/332967247 6 ==== osd_op_reply(60 [pgnls start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616780 con 0x56333d483c00 2024-04-25T09:04:13.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:61 18.13 18:c8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945400 con 0x56333c95c000 2024-04-25T09:04:13.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.7 v2:172.21.15.203:6824/2957998038 5 ==== osd_op_reply(61 [pgnls start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c95c000 2024-04-25T09:04:13.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:62 18.33 18:cc000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c944c00 con 0x56333c944400 2024-04-25T09:04:13.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 12 ==== osd_op_reply(62 [pgnls start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:63 18.b 18:d0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945c00 con 0x56333c83dc00 2024-04-25T09:04:13.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.5 v2:172.21.15.203:6808/213476662 7 ==== osd_op_reply(63 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c83dc00 2024-04-25T09:04:13.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:64 18.2b 18:d4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96e000 con 0x56333c95c000 2024-04-25T09:04:13.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.7 v2:172.21.15.203:6824/2957998038 6 ==== osd_op_reply(64 [pgnls start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c95c000 2024-04-25T09:04:13.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.161+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:65 18.1b 18:d8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95c800 con 0x56333c944400 2024-04-25T09:04:13.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.165+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 13 ==== osd_op_reply(65 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.165+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:66 18.3b 18:dc000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96f400 con 0x56333c95cc00 2024-04-25T09:04:13.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.165+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.4 v2:172.21.15.203:6802/714177927 9 ==== osd_op_reply(66 [pgnls start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c95cc00 2024-04-25T09:04:13.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.165+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:67 18.7 18:e0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d482400 con 0x56333d483c00 2024-04-25T09:04:13.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.165+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 <== osd.2 v2:172.21.15.37:6808/332967247 7 ==== osd_op_reply(67 [pgnls start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616780 con 0x56333d483c00 2024-04-25T09:04:13.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.165+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:68 18.27 18:e4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95d000 con 0x56333c83dc00 2024-04-25T09:04:13.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.165+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.5 v2:172.21.15.203:6808/213476662 8 ==== osd_op_reply(68 [pgnls start_epoch 127 out=49b] v125'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c83dc00 2024-04-25T09:04:13.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.165+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:69 18.17 18:e8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d482000 con 0x56333c83dc00 2024-04-25T09:04:13.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.165+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.5 v2:172.21.15.203:6808/213476662 9 ==== osd_op_reply(69 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c83dc00 2024-04-25T09:04:13.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.165+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:70 18.37 18:ec000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c95c400 con 0x56333c95c000 2024-04-25T09:04:13.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.7 v2:172.21.15.203:6824/2957998038 7 ==== osd_op_reply(70 [pgnls start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c95c000 2024-04-25T09:04:13.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:71 18.f 18:f0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333d482800 con 0x56333c83dc00 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.5 v2:172.21.15.203:6808/213476662 10 ==== osd_op_reply(71 [pgnls start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c83dc00 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:72 18.2f 18:f4000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c944000 con 0x56333c944400 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.1 v2:172.21.15.37:6816/2944303862 14 ==== osd_op_reply(72 [pgnls start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c944400 2024-04-25T09:04:13.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc733ff640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:73 18.1f 18:f8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c96e400 con 0x56333c95c000 2024-04-25T09:04:13.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc77407640 1 -- 172.21.15.37:0/3918058397 <== osd.7 v2:172.21.15.203:6824/2957998038 8 ==== osd_op_reply(73 [pgnls start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616000 con 0x56333c95c000 2024-04-25T09:04:13.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc72bfe640 1 -- 172.21.15.37:0/3918058397 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:74 18.3f 18:fc000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x56333c945800 con 0x56333c95cc00 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc78409640 1 -- 172.21.15.37:0/3918058397 <== osd.4 v2:172.21.15.203:6802/714177927 10 ==== osd_op_reply(74 [pgnls start_epoch 127 out=49b] v125'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56333c616500 con 0x56333c95cc00 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56333d482c00 msgr2=0x56333c6c2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x56333d482c00 0x56333c6c2100 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x56333c944400 msgr2=0x56333c940b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x56333c944400 0x56333c940b00 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x56333d483c00 msgr2=0x56333c941b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x56333d483c00 0x56333c941b80 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x56333d483800 msgr2=0x56333c941600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x56333d483800 0x56333c941600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x56333c95cc00 msgr2=0x56333c942680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x56333c95cc00 0x56333c942680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 reap_dead start 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x56333c83dc00 msgr2=0x56333c940000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x56333c83dc00 0x56333c940000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x56333c944800 msgr2=0x56333c940580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x56333c944800 0x56333c940580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 reap_dead start 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x56333c95c000 msgr2=0x56333c942100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc77c08640 1 -- 172.21.15.37:0/3918058397 reap_dead start 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x56333c95c000 0x56333c942100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56333c83cc00 msgr2=0x56333c6c3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56333c83cc00 0x56333c6c3700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x56333c5e2fc0 tx=0x56333c83b680 comp rx=0 tx=0).stop 2024-04-25T09:04:13.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d2e7800 msgr2=0x56333c6c2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d2e7800 0x56333c6c2c00 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x56333d2d3e90 tx=0x56333c83a0c0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 shutdown_connections 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x56333c83cc00 0x56333c6c3700 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x56333c95c000 0x56333c942100 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 --2- 172.21.15.37:0/3918058397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333d2e7800 0x56333c6c2c00 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 >> 172.21.15.37:0/3918058397 conn(0x56333d2e7400 msgr2=0x56333c534000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.169+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 shutdown_connections 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.173+0000 7fcc794498c0 1 -- 172.21.15.37:0/3918058397 wait complete. 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.173+0000 7fcc794498c0 1 -- 172.21.15.37:0/2019501117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333c507400 msgr2=0x56333c6c0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.173+0000 7fcc794498c0 1 --2- 172.21.15.37:0/2019501117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333c507400 0x56333c6c0b00 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x56333d355410 tx=0x56333d2d4000 comp rx=0 tx=0).stop 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.173+0000 7fcc794498c0 1 -- 172.21.15.37:0/2019501117 shutdown_connections 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.173+0000 7fcc794498c0 1 --2- 172.21.15.37:0/2019501117 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x56333c507800 0x56333c6c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.173+0000 7fcc794498c0 1 --2- 172.21.15.37:0/2019501117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56333c507400 0x56333c6c0b00 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.173+0000 7fcc794498c0 1 --2- 172.21.15.37:0/2019501117 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x56333c507000 0x56333c6c0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.173+0000 7fcc794498c0 1 -- 172.21.15.37:0/2019501117 >> 172.21.15.37:0/2019501117 conn(0x56333c506c00 msgr2=0x56333c533800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.173+0000 7fcc794498c0 1 -- 172.21.15.37:0/2019501117 shutdown_connections 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.173+0000 7fcc794498c0 1 -- 172.21.15.37:0/2019501117 wait complete. 2024-04-25T09:04:13.164 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.147178, result: 2024-04-25T09:04:13.165 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-04-25T09:04:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 -- start start 2024-04-25T09:04:13.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 Processor -- start 2024-04-25T09:04:13.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b94ed0000 0x555b942bb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b94101c00 0x555b942bab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b94ed1c00 0x555b942bbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9500c000 con 0x555b94101c00 2024-04-25T09:04:13.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9500c180 con 0x555b94ed1c00 2024-04-25T09:04:13.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b9500c300 con 0x555b94ed0000 2024-04-25T09:04:13.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba29359640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b94101c00 0x555b942bab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba29359640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b94101c00 0x555b942bab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42990/0 (socket says 172.21.15.37:42990) 2024-04-25T09:04:13.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba29359640 1 -- 172.21.15.37:0/1060046793 learned_addr learned my addr 172.21.15.37:0/1060046793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba29b5a640 1 --2- 172.21.15.37:0/1060046793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b94ed0000 0x555b942bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2a35b640 1 --2- 172.21.15.37:0/1060046793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b94ed1c00 0x555b942bbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba29359640 1 -- 172.21.15.37:0/1060046793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b94ed0000 msgr2=0x555b942bb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba29359640 1 --2- 172.21.15.37:0/1060046793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b94ed0000 0x555b942bb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba29359640 1 -- 172.21.15.37:0/1060046793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b94ed1c00 msgr2=0x555b942bbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba29359640 1 --2- 172.21.15.37:0/1060046793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b94ed1c00 0x555b942bbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba29359640 1 -- 172.21.15.37:0/1060046793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b941261e0 con 0x555b94101c00 2024-04-25T09:04:13.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba29b5a640 1 --2- 172.21.15.37:0/1060046793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b94ed0000 0x555b942bb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:13.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba29359640 1 --2- 172.21.15.37:0/1060046793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b94101c00 0x555b942bab00 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x555b94f558c0 tx=0x555b941bb1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa57b268bda92047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba27355640 1 -- 172.21.15.37:0/1060046793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b941361a0 con 0x555b94101c00 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba27355640 1 -- 172.21.15.37:0/1060046793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b9414a540 con 0x555b94101c00 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba27355640 1 -- 172.21.15.37:0/1060046793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b941364e0 con 0x555b94101c00 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/1060046793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b941272c0 con 0x555b94101c00 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/1060046793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555b94127e00 con 0x555b94101c00 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 4 waiting for MDS map... 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba27355640 1 -- 172.21.15.37:0/1060046793 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x555b95026600 con 0x555b94101c00 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba27355640 1 -- 172.21.15.37:0/1060046793 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 104) v1 ==== 2723+0+0 (secure 0 0 0) 0x555b94157180 con 0x555b94101c00 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 4 Got MDS map 104 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 4 datascan.main: Using metadata pool output 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 4 datascan.main: connecting to RADOS... 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 Processor -- start 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 -- start start 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b95029000 0x555b942ba580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029400 0x555b942bc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b95029800 0x555b942bc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9500c480 con 0x555b95029400 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9500c600 con 0x555b95029000 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.257+0000 7fba2b39b8c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b9500c780 con 0x555b95029800 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29b5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029400 0x555b942bc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29b5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029400 0x555b942bc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43002/0 (socket says 172.21.15.37:43002) 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29b5a640 1 -- 172.21.15.37:0/2694916210 learned_addr learned my addr 172.21.15.37:0/2694916210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2a35b640 1 --2- 172.21.15.37:0/2694916210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b95029000 0x555b942ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29b5a640 1 -- 172.21.15.37:0/2694916210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b95029800 msgr2=0x555b942bc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29b5a640 1 --2- 172.21.15.37:0/2694916210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b95029800 0x555b942bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29b5a640 1 -- 172.21.15.37:0/2694916210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b95029000 msgr2=0x555b942ba580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29b5a640 1 --2- 172.21.15.37:0/2694916210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b95029000 0x555b942ba580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29b5a640 1 -- 172.21.15.37:0/2694916210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b941261e0 con 0x555b95029400 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2a35b640 1 --2- 172.21.15.37:0/2694916210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b95029000 0x555b942ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29b5a640 1 --2- 172.21.15.37:0/2694916210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029400 0x555b942bc100 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x555b95034b40 tx=0x555b941fe180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe9680221b4dab3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba25351640 1 -- 172.21.15.37:0/2694916210 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b94136b60 con 0x555b95029400 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba25351640 1 -- 172.21.15.37:0/2694916210 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b9414ac40 con 0x555b95029400 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba25351640 1 -- 172.21.15.37:0/2694916210 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b941361a0 con 0x555b95029400 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/2694916210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029400 msgr2=0x555b942bc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/2694916210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029400 0x555b942bc100 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x555b95034b40 tx=0x555b941fe180 comp rx=0 tx=0).stop 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/2694916210 shutdown_connections 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/2694916210 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b95029800 0x555b942bc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/2694916210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029400 0x555b942bc100 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/2694916210 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b95029000 0x555b942ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/2694916210 >> 172.21.15.37:0/2694916210 conn(0x555b95028c00 msgr2=0x555b9412f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/2694916210 shutdown_connections 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/2694916210 wait complete. 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 Processor -- start 2024-04-25T09:04:13.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 -- start start 2024-04-25T09:04:13.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b95029000 0x555b942bd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b95029400 0x555b942bd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029800 0x555b942bcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9500dc80 con 0x555b95029800 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x555b9500db00 con 0x555b95029000 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x555b9500d980 con 0x555b95029400 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29359640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029800 0x555b942bcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29359640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029800 0x555b942bcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43012/0 (socket says 172.21.15.37:43012) 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 learned_addr learned my addr 172.21.15.37:0/3986654307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b95029400 msgr2=0x555b942bd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29359640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b95029400 0x555b942bd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b95029000 msgr2=0x555b942bd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29359640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b95029000 0x555b942bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555b941272c0 con 0x555b95029800 2024-04-25T09:04:13.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba29359640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029800 0x555b942bcc00 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x555b95035830 tx=0x555b941baae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba23b4e640 1 -- 172.21.15.37:0/3986654307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b94136340 con 0x555b95029800 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba23b4e640 1 -- 172.21.15.37:0/3986654307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555b9414aa80 con 0x555b95029800 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba23b4e640 1 -- 172.21.15.37:0/3986654307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555b94136b60 con 0x555b95029800 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555b94127c20 con 0x555b95029800 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555b941274a0 con 0x555b95029800 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.261+0000 7fba2b39b8c0 4 datascan.init: resolving metadata pool 17 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba23b4e640 1 -- 172.21.15.37:0/3986654307 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x555b9443ec00 con 0x555b95029800 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba23b4e640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x555b94440c00 0x555b94526000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba23b4e640 1 -- 172.21.15.37:0/3986654307 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x555b9443ca00 con 0x555b95029800 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2b39b8c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2b39b8c0 4 datascan.main: data pool 'cephfs_data' has ID 18 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2b39b8c0 4 datascan.main: opening data pool 'cephfs_data' 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x555b95029c00 0x555b942bc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:1 17.14 17:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94441000 con 0x555b95029c00 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba29b5a640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x555b95029c00 0x555b942bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2a35b640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x555b94440c00 0x555b94526000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba29b5a640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x555b95029c00 0x555b942bc680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv36 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x555b94441400 0x555b942bc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:2 17.1f 17:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94441800 con 0x555b94441400 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2a35b640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x555b94440c00 0x555b94526000 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x555b94427470 tx=0x555b95041e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2a35b640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x555b94441400 0x555b942bc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2a35b640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x555b94441400 0x555b942bc100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x555b941b4500 con 0x555b94441400 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x555b94441c00 0x555b942ba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:3 18.0 18:00000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94441000 con 0x555b94441c00 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.265+0000 7fba29b5a640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x555b94441c00 0x555b942ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba29b5a640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x555b94441c00 0x555b942ba580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba24b50640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x555b9453a000 0x555b94526580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:4 18.20 18:04000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453a400 con 0x555b9453a000 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba29359640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x555b9453a000 0x555b94526580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba29359640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x555b9453a000 0x555b94526580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:5 18.10 18:08000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453ac00 con 0x555b94441400 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.6 v2:172.21.15.203:6800/467003745 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b94441400 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:6 18.30 18:0c000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453a800 con 0x555b94441c00 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:7 18.8 18:10000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453b000 con 0x555b94441c00 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:8 18.28 18:14000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453b800 con 0x555b9453a000 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.269+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba25351640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x555b9453b400 0x555b94526b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:9 18.18 18:18000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c000 con 0x555b9453b400 2024-04-25T09:04:13.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba2a35b640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x555b9453b400 0x555b94526b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba2a35b640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x555b9453b400 0x555b94526b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:10 18.38 18:1c000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453bc00 con 0x555b95029c00 2024-04-25T09:04:13.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:11 18.4 18:20000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c400 con 0x555b94441c00 2024-04-25T09:04:13.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:12 18.24 18:24000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c000 con 0x555b94441c00 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 5 ==== osd_op_reply(12 [pgnls-filter start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:13 18.14 18:28000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c800 con 0x555b94441400 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.6 v2:172.21.15.203:6800/467003745 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b94441400 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:14 18.34 18:2c000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454cc00 con 0x555b94441c00 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:15 18.c 18:30000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453a800 con 0x555b9453a000 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba24b50640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x555b9453ac00 0x555b94527080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:16 18.2c 18:34000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94441800 con 0x555b9453ac00 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba29b5a640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x555b9453ac00 0x555b94527080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba29b5a640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x555b9453ac00 0x555b94527080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.2 v2:172.21.15.37:6808/332967247 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b9453ac00 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.273+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:17 18.1c 18:38000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453b000 con 0x555b9453b400 2024-04-25T09:04:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba24b50640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x555b9454c400 0x555b94527600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:18 18.3c 18:3c000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c000 con 0x555b9454c400 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba29359640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x555b9454c400 0x555b94527600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba29359640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x555b9454c400 0x555b94527600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9454c400 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:19 18.2 18:40000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453bc00 con 0x555b9454c400 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.7 v2:172.21.15.203:6824/2957998038 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9454c400 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:20 18.22 18:44000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d000 con 0x555b95029c00 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:21 18.12 18:48000000::::head [pgnls-filter start_epoch 127 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94441000 con 0x555b9453ac00 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.2 v2:172.21.15.37:6808/332967247 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 127 out=81b] v127'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b941b4000 con 0x555b9453ac00 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:22 18.0 18:00000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94ed1800 con 0x555b94441c00 2024-04-25T09:04:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 7 ==== osd_op_reply(22 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.277+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:23 18.20 18:04000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c000 con 0x555b9453a000 2024-04-25T09:04:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 4 ==== osd_op_reply(23 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:24 18.10 18:08000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d400 con 0x555b94441400 2024-04-25T09:04:13.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.6 v2:172.21.15.203:6800/467003745 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b94441400 2024-04-25T09:04:13.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:25 18.30 18:0c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453a400 con 0x555b94441c00 2024-04-25T09:04:13.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 8 ==== osd_op_reply(25 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:26 18.8 18:10000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d800 con 0x555b94441c00 2024-04-25T09:04:13.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 9 ==== osd_op_reply(26 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:27 18.28 18:14000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453b800 con 0x555b9453a000 2024-04-25T09:04:13.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 5 ==== osd_op_reply(27 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:28 18.18 18:18000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454cc00 con 0x555b9453b400 2024-04-25T09:04:13.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:29 18.38 18:1c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453a800 con 0x555b95029c00 2024-04-25T09:04:13.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:30 18.4 18:20000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d000 con 0x555b94441c00 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 10 ==== osd_op_reply(30 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:31 18.24 18:24000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94440800 con 0x555b94441c00 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 11 ==== osd_op_reply(31 [pgnls-filter start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:32 18.14 18:28000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453a400 con 0x555b94441400 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.6 v2:172.21.15.203:6800/467003745 5 ==== osd_op_reply(32 [pgnls-filter start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b94441400 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:33 18.34 18:2c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c800 con 0x555b94441c00 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.281+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 12 ==== osd_op_reply(33 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:34 18.c 18:30000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94ed1800 con 0x555b9453a000 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:35 18.2c 18:34000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d400 con 0x555b9453ac00 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.2 v2:172.21.15.37:6808/332967247 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b9453ac00 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:36 18.1c 18:38000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94441800 con 0x555b9453b400 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:37 18.3c 18:3c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454cc00 con 0x555b9454c400 2024-04-25T09:04:13.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.7 v2:172.21.15.203:6824/2957998038 3 ==== osd_op_reply(37 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9454c400 2024-04-25T09:04:13.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:38 18.2 18:40000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94441000 con 0x555b9454c400 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.7 v2:172.21.15.203:6824/2957998038 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9454c400 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:39 18.22 18:44000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d800 con 0x555b95029c00 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:40 18.12 18:48000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94441000 con 0x555b9453ac00 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.2 v2:172.21.15.37:6808/332967247 4 ==== osd_op_reply(40 [pgnls-filter start_epoch 127 out=49b] v127'23 uv23 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b9453ac00 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:41 18.32 18:4c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454dc00 con 0x555b95029c00 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 127 out=49b] v125'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:42 18.a 18:50000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c000 con 0x555b95029c00 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 7 ==== osd_op_reply(42 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba25351640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x555b9453b000 0x555b94527b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.285+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:43 18.2a 18:54000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d000 con 0x555b9453b000 2024-04-25T09:04:13.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2a35b640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x555b9453b000 0x555b94527b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2a35b640 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x555b9453b000 0x555b94527b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.4 v2:172.21.15.203:6802/714177927 1 ==== osd_op_reply(43 [pgnls-filter start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b000 2024-04-25T09:04:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:44 18.1a 18:58000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454cc00 con 0x555b95029c00 2024-04-25T09:04:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 8 ==== osd_op_reply(44 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:45 18.3a 18:5c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c800 con 0x555b94441c00 2024-04-25T09:04:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 13 ==== osd_op_reply(45 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:46 18.6 18:60000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94ed1800 con 0x555b9454c400 2024-04-25T09:04:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.7 v2:172.21.15.203:6824/2957998038 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9454c400 2024-04-25T09:04:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:47 18.26 18:64000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d400 con 0x555b9453ac00 2024-04-25T09:04:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.2 v2:172.21.15.37:6808/332967247 5 ==== osd_op_reply(47 [pgnls-filter start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b9453ac00 2024-04-25T09:04:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:48 18.16 18:68000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453bc00 con 0x555b95029c00 2024-04-25T09:04:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 9 ==== osd_op_reply(48 [pgnls-filter start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:49 18.36 18:6c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d800 con 0x555b9453b400 2024-04-25T09:04:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 127 out=81b] v127'34 uv34 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2b39b8c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2b39b8c0 10 datascan.operator(): handling object 100000001f7.0 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:50 18.36 18:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94ed1000 con 0x555b9453b400 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 6 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) v8 ==== 416+0+194 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2b39b8c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2b39b8c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2b39b8c0 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.289+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:51 17.7 17:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94ed0c00 con 0x555b94441c00 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 14 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:52 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d000 con 0x555b95029c00 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 10 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=524b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+524 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-04-25T09:04:13.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t() 2024-04-25T09:04:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 20 datascan.get_frag_of: frag is * 2024-04-25T09:04:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:53 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d800 con 0x555b94441c00 2024-04-25T09:04:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 15 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 10 datascan.find_or_create_dirfrag: read_version = 7 2024-04-25T09:04:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-04-25T09:04:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:54 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454dc00 con 0x555b94441c00 2024-04-25T09:04:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 16 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T09:04:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 20 datascan.read_dentry: bravo_head not found in result 2024-04-25T09:04:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-04-25T09:04:13.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:55 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94ed1000 con 0x555b94441c00 2024-04-25T09:04:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 17 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v127'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-25T09:04:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:56 18.e 18:70000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94441800 con 0x555b94441c00 2024-04-25T09:04:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 18 ==== osd_op_reply(56 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:57 18.2e 18:74000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94440800 con 0x555b95029c00 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 11 ==== osd_op_reply(57 [pgnls-filter start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.293+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:58 18.1e 18:78000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453b800 con 0x555b9453a000 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:59 18.3e 18:7c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94441000 con 0x555b95029c00 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 12 ==== osd_op_reply(59 [pgnls-filter start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:60 18.1 18:80000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c000 con 0x555b95029c00 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:61 18.21 18:84000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453a800 con 0x555b9453a000 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 8 ==== osd_op_reply(61 [pgnls-filter start_epoch 127 out=49b] v125'24 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:62 18.11 18:88000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454cc00 con 0x555b9453b400 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 7 ==== osd_op_reply(62 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:63 18.31 18:8c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454dc00 con 0x555b9453b000 2024-04-25T09:04:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.4 v2:172.21.15.203:6802/714177927 2 ==== osd_op_reply(63 [pgnls-filter start_epoch 127 out=49b] v125'15 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b000 2024-04-25T09:04:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:64 18.9 18:90000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c800 con 0x555b95029c00 2024-04-25T09:04:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 14 ==== osd_op_reply(64 [pgnls-filter start_epoch 127 out=49b] v125'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:65 18.29 18:94000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9457a000 con 0x555b9454c400 2024-04-25T09:04:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.7 v2:172.21.15.203:6824/2957998038 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9454c400 2024-04-25T09:04:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.297+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:66 18.19 18:98000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d400 con 0x555b9453b000 2024-04-25T09:04:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.4 v2:172.21.15.203:6802/714177927 3 ==== osd_op_reply(66 [pgnls-filter start_epoch 127 out=49b] v127'24 uv24 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b000 2024-04-25T09:04:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:67 18.39 18:9c000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9457a400 con 0x555b94441c00 2024-04-25T09:04:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 19 ==== osd_op_reply(67 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:68 18.5 18:a0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d000 con 0x555b9453b400 2024-04-25T09:04:13.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 127 out=49b] v127'15 uv15 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:69 18.25 18:a4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9457a800 con 0x555b94441400 2024-04-25T09:04:13.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.6 v2:172.21.15.203:6800/467003745 6 ==== osd_op_reply(69 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b94441400 2024-04-25T09:04:13.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:70 18.15 18:a8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d800 con 0x555b9453a000 2024-04-25T09:04:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 9 ==== osd_op_reply(70 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:71 18.35 18:ac000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9457ac00 con 0x555b9453b000 2024-04-25T09:04:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.4 v2:172.21.15.203:6802/714177927 4 ==== osd_op_reply(71 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b000 2024-04-25T09:04:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.301+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:72 18.d 18:b0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94ed0c00 con 0x555b9453b000 2024-04-25T09:04:13.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.4 v2:172.21.15.203:6802/714177927 5 ==== osd_op_reply(72 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b000 2024-04-25T09:04:13.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:73 18.2d 18:b4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9457b000 con 0x555b94441400 2024-04-25T09:04:13.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.6 v2:172.21.15.203:6800/467003745 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b94441400 2024-04-25T09:04:13.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:74 18.1d 18:b8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453bc00 con 0x555b94441c00 2024-04-25T09:04:13.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.0 v2:172.21.15.37:6824/1116141266 20 ==== osd_op_reply(74 [pgnls-filter start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b94441c00 2024-04-25T09:04:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:75 18.3d 18:bc000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9457b400 con 0x555b95029c00 2024-04-25T09:04:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.3 v2:172.21.15.37:6800/4126478460 15 ==== osd_op_reply(75 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b95029c00 2024-04-25T09:04:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:76 18.3 18:c0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9457a800 con 0x555b9453b400 2024-04-25T09:04:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:77 18.23 18:c4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9457b800 con 0x555b9453ac00 2024-04-25T09:04:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.2 v2:172.21.15.37:6808/332967247 6 ==== osd_op_reply(77 [pgnls-filter start_epoch 127 out=49b] v125'14 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b9453ac00 2024-04-25T09:04:13.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:78 18.13 18:c8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454cc00 con 0x555b9454c400 2024-04-25T09:04:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.7 v2:172.21.15.203:6824/2957998038 7 ==== osd_op_reply(78 [pgnls-filter start_epoch 127 out=49b] v125'30 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9454c400 2024-04-25T09:04:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:79 18.33 18:cc000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9457bc00 con 0x555b9453b400 2024-04-25T09:04:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 127 out=49b] v125'20 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.305+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:80 18.b 18:d0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454dc00 con 0x555b9453a000 2024-04-25T09:04:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 10 ==== osd_op_reply(80 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:81 18.2b 18:d4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d800 con 0x555b9454c400 2024-04-25T09:04:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.7 v2:172.21.15.203:6824/2957998038 8 ==== osd_op_reply(81 [pgnls-filter start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9454c400 2024-04-25T09:04:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:82 18.1b 18:d8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d400 con 0x555b9453b400 2024-04-25T09:04:13.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:83 18.3b 18:dc000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454cc00 con 0x555b9453b000 2024-04-25T09:04:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.4 v2:172.21.15.203:6802/714177927 6 ==== osd_op_reply(83 [pgnls-filter start_epoch 127 out=49b] v125'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b000 2024-04-25T09:04:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:84 18.7 18:e0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454d000 con 0x555b9453ac00 2024-04-25T09:04:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 <== osd.2 v2:172.21.15.37:6808/332967247 7 ==== osd_op_reply(84 [pgnls-filter start_epoch 127 out=49b] v125'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4000 con 0x555b9453ac00 2024-04-25T09:04:13.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:85 18.27 18:e4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454dc00 con 0x555b9453a000 2024-04-25T09:04:13.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 127 out=49b] v125'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.309+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:86 18.17 18:e8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94ed1400 con 0x555b9453a000 2024-04-25T09:04:13.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:87 18.37 18:ec000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c000 con 0x555b9454c400 2024-04-25T09:04:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.7 v2:172.21.15.203:6824/2957998038 9 ==== osd_op_reply(87 [pgnls-filter start_epoch 127 out=49b] v125'22 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9454c400 2024-04-25T09:04:13.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:88 18.f 18:f0000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453b800 con 0x555b9453a000 2024-04-25T09:04:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.5 v2:172.21.15.203:6808/213476662 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 127 out=49b] v125'18 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9453a000 2024-04-25T09:04:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:89 18.2f 18:f4000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9454c800 con 0x555b9453b400 2024-04-25T09:04:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.1 v2:172.21.15.37:6816/2944303862 12 ==== osd_op_reply(89 [pgnls-filter start_epoch 127 out=49b] v125'16 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b400 2024-04-25T09:04:13.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba24b50640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:90 18.1f 18:f8000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b9453a800 con 0x555b9454c400 2024-04-25T09:04:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba29359640 1 -- 172.21.15.37:0/3986654307 <== osd.7 v2:172.21.15.203:6824/2957998038 10 ==== osd_op_reply(90 [pgnls-filter start_epoch 127 out=49b] v125'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4780 con 0x555b9454c400 2024-04-25T09:04:13.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba25351640 1 -- 172.21.15.37:0/3986654307 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:91 18.3f 18:fc000000::::head [pgnls-filter start_epoch 127 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x555b94441800 con 0x555b9453b000 2024-04-25T09:04:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2a35b640 1 -- 172.21.15.37:0/3986654307 <== osd.4 v2:172.21.15.203:6802/714177927 7 ==== osd_op_reply(91 [pgnls-filter start_epoch 127 out=49b] v125'17 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555b941b4500 con 0x555b9453b000 2024-04-25T09:04:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x555b94441c00 msgr2=0x555b942ba580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x555b94441c00 0x555b942ba580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x555b9453b400 msgr2=0x555b94526b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x555b9453b400 0x555b94526b00 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x555b9453ac00 msgr2=0x555b94527080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x555b9453ac00 0x555b94527080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x555b95029c00 msgr2=0x555b942bc680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x555b95029c00 0x555b942bc680 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x555b9453b000 msgr2=0x555b94527b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 reap_dead start 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x555b9453b000 0x555b94527b80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 reap_dead start 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x555b9453a000 msgr2=0x555b94526580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x555b9453a000 0x555b94526580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x555b94441400 msgr2=0x555b942bc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x555b94441400 0x555b942bc100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x555b9454c400 msgr2=0x555b94527600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x555b9454c400 0x555b94527600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x555b94440c00 msgr2=0x555b94526000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x555b94440c00 0x555b94526000 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x555b94427470 tx=0x555b95041e60 comp rx=0 tx=0).stop 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029800 msgr2=0x555b942bcc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.313+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/3986654307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b95029800 0x555b942bcc00 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x555b95035830 tx=0x555b941baae0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba29b5a640 1 -- 172.21.15.37:0/3986654307 reap_dead start 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 shutdown_connections 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 >> 172.21.15.37:0/3986654307 conn(0x555b95028c00 msgr2=0x555b9412f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 shutdown_connections 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/3986654307 wait complete. 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/1060046793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b94101c00 msgr2=0x555b942bab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/1060046793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b94101c00 0x555b942bab00 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x555b94f558c0 tx=0x555b941bb1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/1060046793 shutdown_connections 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/1060046793 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x555b94ed1c00 0x555b942bbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/1060046793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b94101c00 0x555b942bab00 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 --2- 172.21.15.37:0/1060046793 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x555b94ed0000 0x555b942bb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/1060046793 >> 172.21.15.37:0/1060046793 conn(0x555b94100c00 msgr2=0x555b9412e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/1060046793 shutdown_connections 2024-04-25T09:04:13.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.317+0000 7fba2b39b8c0 1 -- 172.21.15.37:0/1060046793 wait complete. 2024-04-25T09:04:13.308 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.143723, result: 2024-04-25T09:04:13.309 DEBUG:teuthology.orchestra.run.smithi037:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 1 -- start start 2024-04-25T09:04:13.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 1 Processor -- start 2024-04-25T09:04:13.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120800 0x55b812517b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b81235dc00 0x55b812516b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b81235d800 0x55b812517600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b813264000 con 0x55b81235dc00 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b813264180 con 0x55b813120800 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b813264300 con 0x55b81235d800 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f65804640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b81235dc00 0x55b812516b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f65804640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b81235dc00 0x55b812516b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43036/0 (socket says 172.21.15.37:43036) 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f65804640 1 -- 172.21.15.37:0/2824249262 learned_addr learned my addr 172.21.15.37:0/2824249262 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f66005640 1 --2- 172.21.15.37:0/2824249262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120800 0x55b812517b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f66806640 1 --2- 172.21.15.37:0/2824249262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b81235d800 0x55b812517600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f65804640 1 -- 172.21.15.37:0/2824249262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b81235d800 msgr2=0x55b812517600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f65804640 1 --2- 172.21.15.37:0/2824249262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b81235d800 0x55b812517600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f65804640 1 -- 172.21.15.37:0/2824249262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120800 msgr2=0x55b812517b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f65804640 1 --2- 172.21.15.37:0/2824249262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120800 0x55b812517b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f65804640 1 -- 172.21.15.37:0/2824249262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8123821e0 con 0x55b81235dc00 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f65804640 1 --2- 172.21.15.37:0/2824249262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b81235dc00 0x55b812516b00 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x55b8131a9b90 tx=0x55b812417020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c8e85b7cfcc187 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f63800640 1 -- 172.21.15.37:0/2824249262 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b8123921a0 con 0x55b81235dc00 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f63800640 1 -- 172.21.15.37:0/2824249262 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b8123a61c0 con 0x55b81235dc00 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f63800640 1 -- 172.21.15.37:0/2824249262 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b8123924e0 con 0x55b81235dc00 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 1 -- 172.21.15.37:0/2824249262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8123832c0 con 0x55b81235dc00 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 1 -- 172.21.15.37:0/2824249262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b812383c20 con 0x55b81235dc00 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 4 waiting for MDS map... 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f63800640 1 -- 172.21.15.37:0/2824249262 <== mon.0 v2:172.21.15.37:3300/0 4 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x55b81327c600 con 0x55b81235dc00 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f63800640 1 -- 172.21.15.37:0/2824249262 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 104) v1 ==== 2723+0+0 (secure 0 0 0) 0x55b8123b3180 con 0x55b81235dc00 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 4 Got MDS map 104 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 4 datascan.main: Using metadata pool output 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.393+0000 7f7f678468c0 4 datascan.main: connecting to RADOS... 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 Processor -- start 2024-04-25T09:04:13.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- start start 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b813120400 0x55b812517080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120000 0x55b812518100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813280800 0x55b812518680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b813264480 con 0x55b813280800 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b813264600 con 0x55b813120000 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b813264780 con 0x55b813120400 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813280800 0x55b812518680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813280800 0x55b812518680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43038/0 (socket says 172.21.15.37:43038) 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 -- 172.21.15.37:0/83672150 learned_addr learned my addr 172.21.15.37:0/83672150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 -- 172.21.15.37:0/83672150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b813120400 msgr2=0x55b812517080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 --2- 172.21.15.37:0/83672150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b813120400 0x55b812517080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 -- 172.21.15.37:0/83672150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120000 msgr2=0x55b812518100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 --2- 172.21.15.37:0/83672150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120000 0x55b812518100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 -- 172.21.15.37:0/83672150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b812383e00 con 0x55b813280800 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 --2- 172.21.15.37:0/83672150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813280800 0x55b812518680 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x55b813273020 tx=0x55b812420e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e39636cfa79e1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f617fc640 1 -- 172.21.15.37:0/83672150 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b812392b60 con 0x55b813280800 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f617fc640 1 -- 172.21.15.37:0/83672150 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b8123a6a80 con 0x55b813280800 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f617fc640 1 -- 172.21.15.37:0/83672150 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b8123921a0 con 0x55b813280800 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- 172.21.15.37:0/83672150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813280800 msgr2=0x55b812518680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 --2- 172.21.15.37:0/83672150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813280800 0x55b812518680 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x55b813273020 tx=0x55b812420e40 comp rx=0 tx=0).stop 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- 172.21.15.37:0/83672150 shutdown_connections 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 --2- 172.21.15.37:0/83672150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813280800 0x55b812518680 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 --2- 172.21.15.37:0/83672150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120000 0x55b812518100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 --2- 172.21.15.37:0/83672150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b813120400 0x55b812517080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- 172.21.15.37:0/83672150 >> 172.21.15.37:0/83672150 conn(0x55b813121c00 msgr2=0x55b8132de000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- 172.21.15.37:0/83672150 shutdown_connections 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- 172.21.15.37:0/83672150 wait complete. 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 Processor -- start 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- start start 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b813120400 0x55b812519700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120000 0x55b812519180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813281800 0x55b812518c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b813265e00 con 0x55b813281800 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b813265c80 con 0x55b813120000 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f678468c0 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b813265b00 con 0x55b813120400 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813281800 0x55b812518c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813281800 0x55b812518c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43040/0 (socket says 172.21.15.37:43040) 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 learned_addr learned my addr 172.21.15.37:0/2322448888 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f66806640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b813120400 0x55b812519700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120000 0x55b812519180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b813120400 msgr2=0x55b812519700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b813120400 0x55b812519700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120000 msgr2=0x55b812519180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120000 0x55b812519180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.397+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8123821e0 con 0x55b813281800 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f65804640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813281800 0x55b812518c00 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x55b813273290 tx=0x55b81329ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f5fff9640 1 -- 172.21.15.37:0/2322448888 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b8123924e0 con 0x55b813281800 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f5fff9640 1 -- 172.21.15.37:0/2322448888 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b8123a7340 con 0x55b813281800 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b8123832c0 con 0x55b813281800 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f5fff9640 1 -- 172.21.15.37:0/2322448888 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b812392340 con 0x55b813281800 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b812383c20 con 0x55b813281800 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f678468c0 4 datascan.init: resolving metadata pool 17 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f5fff9640 1 -- 172.21.15.37:0/2322448888 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x55b81268ec00 con 0x55b813281800 2024-04-25T09:04:13.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f5fff9640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55b81268d000 0x55b812778000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f5fff9640 1 -- 172.21.15.37:0/2322448888 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x55b812692600 con 0x55b813281800 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f678468c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f678468c0 4 datascan.main: resolving metadata pool 17 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55b813281400 0x55b812518680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b813281000 con 0x55b813281400 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f66806640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55b81268d000 0x55b812778000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55b813281400 0x55b812518680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f66806640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55b81268d000 0x55b812778000 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55b812439140 tx=0x55b81245a180 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55b813281400 0x55b812518680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== osd_op_reply(1 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55b813281000 0x55b812778580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:2 17.10 17:08000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268d400 con 0x55b813281000 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.401+0000 7f7f66806640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55b813281000 0x55b812778580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66806640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55b813281000 0x55b812778580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 1 ==== osd_op_reply(2 [pgnls start_epoch 127 out=177b] v126'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:3 17.8 17:10000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268d400 con 0x55b813281400 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 2 ==== osd_op_reply(3 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55b81268d400 0x55b812778b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:4 17.18 17:18000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268d800 con 0x55b81268d400 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55b81268d400 0x55b812778b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55b81268d400 0x55b812778b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== osd_op_reply(4 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:5 17.4 17:20000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268d800 con 0x55b81268d400 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 2 ==== osd_op_reply(5 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:6 17.14 17:28000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268d800 con 0x55b813281000 2024-04-25T09:04:13.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 2 ==== osd_op_reply(6 [pgnls start_epoch 127 out=149b] v127'43 uv43 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:7 17.c 17:30000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268d800 con 0x55b813281400 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 3 ==== osd_op_reply(7 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b81268d800 0x55b812779080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:8 17.1c 17:38000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268dc00 con 0x55b81268d800 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f65804640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b81268d800 0x55b812779080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f65804640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b81268d800 0x55b812779080 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== osd_op_reply(8 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b812410780 con 0x55b81268d800 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f65804640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55b81268dc00 0x55b812779600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.405+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:9 17.2 17:40000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268c800 con 0x55b81268dc00 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66806640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55b81268dc00 0x55b812779600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66806640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55b81268dc00 0x55b812779600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.4 v2:172.21.15.203:6802/714177927 1 ==== osd_op_reply(9 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410500 con 0x55b81268dc00 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:10 17.12 17:48000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268c800 con 0x55b813281000 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 3 ==== osd_op_reply(10 [pgnls start_epoch 127 out=81b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:11 17.a 17:50000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268c800 con 0x55b813281400 2024-04-25T09:04:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 4 ==== osd_op_reply(11 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:12 17.1a 17:58000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268c800 con 0x55b81268d400 2024-04-25T09:04:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 3 ==== osd_op_reply(12 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55b81268c800 0x55b812779b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:13 17.6 17:60000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812792000 con 0x55b81268c800 2024-04-25T09:04:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55b81268c800 0x55b812779b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.409+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55b81268c800 0x55b812779b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== osd_op_reply(13 [pgnls start_epoch 127 out=77b] v126'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55b812410000 con 0x55b81268c800 2024-04-25T09:04:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66005640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55b812792000 0x55b81277a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:14 17.16 17:68000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812792400 con 0x55b812792000 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f65804640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55b812792000 0x55b81277a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f65804640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55b812792000 0x55b81277a100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== osd_op_reply(14 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410780 con 0x55b812792000 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f65804640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55b812792400 0x55b81277ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:15 17.e 17:70000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b812792400 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66806640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55b812792400 0x55b81277ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66806640 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55b812792400 0x55b81277ac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.2 v2:172.21.15.37:6808/332967247 1 ==== osd_op_reply(15 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b812410500 con 0x55b812792400 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:16 17.1e 17:78000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b81268d800 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.1 v2:172.21.15.37:6816/2944303862 2 ==== osd_op_reply(16 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410780 con 0x55b81268d800 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:17 17.1 17:80000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b81268d400 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 4 ==== osd_op_reply(17 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:18 17.11 17:88000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b813281000 2024-04-25T09:04:13.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 4 ==== osd_op_reply(18 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:19 17.9 17:90000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b813281000 2024-04-25T09:04:13.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 5 ==== osd_op_reply(19 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.413+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:20 17.19 17:98000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b812792000 2024-04-25T09:04:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.417+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.5 v2:172.21.15.203:6808/213476662 2 ==== osd_op_reply(20 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410780 con 0x55b812792000 2024-04-25T09:04:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.417+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:21 17.5 17:a0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b81268c800 2024-04-25T09:04:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.417+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.6 v2:172.21.15.203:6800/467003745 2 ==== osd_op_reply(21 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268c800 2024-04-25T09:04:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.417+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:22 17.15 17:a8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b812792000 2024-04-25T09:04:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.417+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.5 v2:172.21.15.203:6808/213476662 3 ==== osd_op_reply(22 [pgnls start_epoch 127 out=97b] v126'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55b812410780 con 0x55b812792000 2024-04-25T09:04:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.417+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:23 17.d 17:b0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b813281400 2024-04-25T09:04:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.417+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 5 ==== osd_op_reply(23 [pgnls start_epoch 127 out=74b] v124'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.417+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:24 17.1d 17:b8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b813281400 2024-04-25T09:04:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.417+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 6 ==== osd_op_reply(24 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.417+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:25 17.3 17:c0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b81268dc00 2024-04-25T09:04:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.4 v2:172.21.15.203:6802/714177927 2 ==== osd_op_reply(25 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410500 con 0x55b81268dc00 2024-04-25T09:04:13.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:26 17.13 17:c8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b813281000 2024-04-25T09:04:13.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 6 ==== osd_op_reply(26 [pgnls start_epoch 127 out=98b] v126'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:27 17.b 17:d0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b81268d400 2024-04-25T09:04:13.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 5 ==== osd_op_reply(27 [pgnls start_epoch 127 out=76b] v127'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:28 17.1b 17:d8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b81268dc00 2024-04-25T09:04:13.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.4 v2:172.21.15.203:6802/714177927 3 ==== osd_op_reply(28 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410500 con 0x55b81268dc00 2024-04-25T09:04:13.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:29 17.7 17:e0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b81268d400 2024-04-25T09:04:13.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 6 ==== osd_op_reply(29 [pgnls start_epoch 127 out=105b] v127'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:30 17.17 17:e8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b81268c800 2024-04-25T09:04:13.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.6 v2:172.21.15.203:6800/467003745 3 ==== osd_op_reply(30 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268c800 2024-04-25T09:04:13.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:31 17.f 17:f0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b81268d800 2024-04-25T09:04:13.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.1 v2:172.21.15.37:6816/2944303862 3 ==== osd_op_reply(31 [pgnls start_epoch 127 out=73b] v127'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b812410780 con 0x55b81268d800 2024-04-25T09:04:13.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:32 17.1f 17:f8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b81268c800 2024-04-25T09:04:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.6 v2:172.21.15.203:6800/467003745 4 ==== osd_op_reply(32 [pgnls start_epoch 127 out=101b] v127'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55b812410000 con 0x55b81268c800 2024-04-25T09:04:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 601.00000000 2024-04-25T09:04:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:33 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b813280c00 con 0x55b813281000 2024-04-25T09:04:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 7 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 602.00000000 2024-04-25T09:04:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.421+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:34 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b813280800 con 0x55b813281000 2024-04-25T09:04:13.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 8 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:35 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812792c00 con 0x55b813281000 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 9 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 600.00000000 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:36 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812793000 con 0x55b813281000 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 10 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 603.00000000 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:37 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812793400 con 0x55b813281000 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 11 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 200.00000000 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 200.00000001 2024-04-25T09:04:13.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 606.00000000 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:38 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812793800 con 0x55b813281000 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 12 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 607.00000000 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:39 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812793c00 con 0x55b81268d800 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.1 v2:172.21.15.37:6816/2944303862 4 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410780 con 0x55b81268d800 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:40 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812792c00 con 0x55b813281000 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 13 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 608.00000000 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:41 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812793000 con 0x55b812792400 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.2 v2:172.21.15.37:6808/332967247 2 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b812792400 2024-04-25T09:04:13.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 604.00000000 2024-04-25T09:04:13.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:42 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812793400 con 0x55b812792000 2024-04-25T09:04:13.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.5 v2:172.21.15.203:6808/213476662 4 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410780 con 0x55b812792000 2024-04-25T09:04:13.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 500.00000000 2024-04-25T09:04:13.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T09:04:13.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-04-25T09:04:13.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 605.00000000 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:43 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b813280c00 con 0x55b813281400 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 7 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 100.00000000 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:44 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b813280800 con 0x55b813281000 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 14 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 609.00000000 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.425+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:45 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81235d000 con 0x55b81268d400 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 7 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-04-25T09:04:13.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:46 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6000 con 0x55b81268d400 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 8 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f678468c0 20 datascan.scan_links: injected ino detected 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 400.00000000 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f678468c0 10 datascan.scan_links: step 1: handling object 1.00000000 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:47 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6400 con 0x55b81268c800 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.6 v2:172.21.15.203:6800/467003745 5 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x55b812410000 con 0x55b81268c800 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:48 17.0 17:00000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281400 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 8 ==== osd_op_reply(48 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:49 17.10 17:08000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281000 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 15 ==== osd_op_reply(49 [pgnls start_epoch 127 out=177b] v126'16 uv16 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:50 17.8 17:10000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281400 2024-04-25T09:04:13.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 9 ==== osd_op_reply(50 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:51 17.18 17:18000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268d400 2024-04-25T09:04:13.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 9 ==== osd_op_reply(51 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:52 17.4 17:20000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268d400 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 10 ==== osd_op_reply(52 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.429+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:53 17.14 17:28000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281000 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 16 ==== osd_op_reply(53 [pgnls start_epoch 127 out=149b] v127'43 uv43 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:54 17.c 17:30000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281400 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 10 ==== osd_op_reply(54 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:55 17.1c 17:38000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268d800 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.1 v2:172.21.15.37:6816/2944303862 5 ==== osd_op_reply(55 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b812410780 con 0x55b81268d800 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:56 17.2 17:40000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268dc00 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.4 v2:172.21.15.203:6802/714177927 4 ==== osd_op_reply(56 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410500 con 0x55b81268dc00 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:57 17.12 17:48000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281000 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 17 ==== osd_op_reply(57 [pgnls start_epoch 127 out=81b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:58 17.a 17:50000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281400 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 11 ==== osd_op_reply(58 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:59 17.1a 17:58000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268d400 2024-04-25T09:04:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 11 ==== osd_op_reply(59 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:60 17.6 17:60000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268c800 2024-04-25T09:04:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.6 v2:172.21.15.203:6800/467003745 6 ==== osd_op_reply(60 [pgnls start_epoch 127 out=77b] v126'9 uv9 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55b812410000 con 0x55b81268c800 2024-04-25T09:04:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.433+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:61 17.16 17:68000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b812792000 2024-04-25T09:04:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.5 v2:172.21.15.203:6808/213476662 5 ==== osd_op_reply(61 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410780 con 0x55b812792000 2024-04-25T09:04:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:62 17.e 17:70000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b812792400 2024-04-25T09:04:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.2 v2:172.21.15.37:6808/332967247 3 ==== osd_op_reply(62 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b812410500 con 0x55b812792400 2024-04-25T09:04:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:63 17.1e 17:78000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268d800 2024-04-25T09:04:13.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.1 v2:172.21.15.37:6816/2944303862 6 ==== osd_op_reply(63 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410780 con 0x55b81268d800 2024-04-25T09:04:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:64 17.1 17:80000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268d400 2024-04-25T09:04:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 12 ==== osd_op_reply(64 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:65 17.11 17:88000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281000 2024-04-25T09:04:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 18 ==== osd_op_reply(65 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:66 17.9 17:90000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281000 2024-04-25T09:04:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 19 ==== osd_op_reply(66 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:67 17.19 17:98000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b812792000 2024-04-25T09:04:13.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.5 v2:172.21.15.203:6808/213476662 6 ==== osd_op_reply(67 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410780 con 0x55b812792000 2024-04-25T09:04:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268c800 2024-04-25T09:04:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.6 v2:172.21.15.203:6800/467003745 7 ==== osd_op_reply(68 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268c800 2024-04-25T09:04:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.437+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:69 17.15 17:a8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b812792000 2024-04-25T09:04:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.5 v2:172.21.15.203:6808/213476662 7 ==== osd_op_reply(69 [pgnls start_epoch 127 out=97b] v126'4 uv4 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x55b812410780 con 0x55b812792000 2024-04-25T09:04:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:70 17.d 17:b0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281400 2024-04-25T09:04:13.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 12 ==== osd_op_reply(70 [pgnls start_epoch 127 out=74b] v124'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:71 17.1d 17:b8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281400 2024-04-25T09:04:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 13 ==== osd_op_reply(71 [pgnls start_epoch 127 out=73b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:72 17.3 17:c0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268dc00 2024-04-25T09:04:13.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.4 v2:172.21.15.203:6802/714177927 5 ==== osd_op_reply(72 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410500 con 0x55b81268dc00 2024-04-25T09:04:13.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:73 17.13 17:c8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281000 2024-04-25T09:04:13.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 20 ==== osd_op_reply(73 [pgnls start_epoch 127 out=98b] v126'5 uv5 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:74 17.b 17:d0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268d400 2024-04-25T09:04:13.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 13 ==== osd_op_reply(74 [pgnls start_epoch 127 out=76b] v127'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- osd_op(unknown.0.0:75 17.1b 17:d8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268dc00 2024-04-25T09:04:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.4 v2:172.21.15.203:6802/714177927 6 ==== osd_op_reply(75 [pgnls start_epoch 127 out=49b] v127'3 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410500 con 0x55b81268dc00 2024-04-25T09:04:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:76 17.7 17:e0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268d400 2024-04-25T09:04:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 14 ==== osd_op_reply(76 [pgnls start_epoch 127 out=105b] v127'9 uv9 ondisk = 1) v8 ==== 144+0+105 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.441+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:77 17.17 17:e8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268c800 2024-04-25T09:04:13.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.6 v2:172.21.15.203:6800/467003745 8 ==== osd_op_reply(77 [pgnls start_epoch 127 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b812410000 con 0x55b81268c800 2024-04-25T09:04:13.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:78 17.f 17:f0000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268d800 2024-04-25T09:04:13.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.1 v2:172.21.15.37:6816/2944303862 7 ==== osd_op_reply(78 [pgnls start_epoch 127 out=73b] v127'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b812410780 con 0x55b81268d800 2024-04-25T09:04:13.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:79 17.1f 17:f8000000::::head [pgnls start_epoch 127 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b81268c800 2024-04-25T09:04:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.6 v2:172.21.15.203:6800/467003745 9 ==== osd_op_reply(79 [pgnls start_epoch 127 out=101b] v127'9 uv8 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55b812410000 con 0x55b81268c800 2024-04-25T09:04:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 601.00000000 2024-04-25T09:04:13.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:80 17.10 17:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6c00 con 0x55b813281000 2024-04-25T09:04:13.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 21 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 602.00000000 2024-04-25T09:04:13.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:81 17.10 17:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b7000 con 0x55b813281000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 22 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:82 17.10 17:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b7400 con 0x55b813281000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 23 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1047b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+1047 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 600.00000000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:83 17.10 17:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b7800 con 0x55b813281000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 24 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 603.00000000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:84 17.10 17:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b7c00 con 0x55b813281000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 25 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 200.00000000 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-04-25T09:04:13.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 200.00000001 2024-04-25T09:04:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-04-25T09:04:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 606.00000000 2024-04-25T09:04:13.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:85 17.14 17:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6c00 con 0x55b813281000 2024-04-25T09:04:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 26 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv38 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 607.00000000 2024-04-25T09:04:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.445+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- osd_op(unknown.0.0:86 17.1c 17:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b7000 con 0x55b81268d800 2024-04-25T09:04:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.1 v2:172.21.15.37:6816/2944303862 8 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410780 con 0x55b81268d800 2024-04-25T09:04:13.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:87 17.12 17:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b7400 con 0x55b813281000 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 27 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=523b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+523 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 608.00000000 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- osd_op(unknown.0.0:88 17.e 17:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81235d000 con 0x55b812792400 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.2 v2:172.21.15.37:6808/332967247 4 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410500 con 0x55b812792400 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 604.00000000 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- osd_op(unknown.0.0:89 17.15 17:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b813280800 con 0x55b812792000 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f65804640 1 -- 172.21.15.37:0/2322448888 <== osd.5 v2:172.21.15.203:6808/213476662 8 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410780 con 0x55b812792000 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 500.00000000 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-04-25T09:04:13.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 605.00000000 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:90 17.1d 17:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b813280c00 con 0x55b813281400 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 14 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 100.00000000 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:91 17.13 17:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b81268cc00 con 0x55b813281000 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 28 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 609.00000000 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:92 17.7 17:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812793800 con 0x55b81268d400 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 15 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-04-25T09:04:13.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:93 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812792c00 con 0x55b81268d400 2024-04-25T09:04:13.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 16 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1545b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+1545 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 20 datascan.scan_links: injected ino detected 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 400.00000000 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 10 datascan.scan_links: step 2: handling object 1.00000000 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.449+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- osd_op(unknown.0.0:94 17.1f 17:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812793000 con 0x55b81268c800 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.6 v2:172.21.15.203:6800/467003745 10 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=524b] v0'0 uv8 ondisk = 0) v8 ==== 154+0+524 (crc 0 0 0) 0x55b812410000 con 0x55b81268c800 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 10 datascan.scan_links: removing dup dentries from 0 objects 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 10 datascan.scan_links: processing 1 injected_inos 2024-04-25T09:04:13.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:95 17.7 17:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812792800 con 0x55b81268d400 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 17 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+506 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 20 datascan.scan_links: first is now 2 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- osd_op(unknown.0.0:96 17.7 17:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6400 con 0x55b81268d400 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.0 v2:172.21.15.37:6824/1116141266 18 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v127'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b812410000 con 0x55b81268d400 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 10 datascan.scan_links: updating inotable 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:97 17.13 17:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6800 con 0x55b813281000 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 29 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-04-25T09:04:13.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.453+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- osd_op(unknown.0.0:98 17.13 17:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b8127b6000 con 0x55b813281000 2024-04-25T09:04:13.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f66806640 1 -- 172.21.15.37:0/2322448888 <== osd.3 v2:172.21.15.37:6800/4126478460 30 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v127'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55b812410500 con 0x55b813281000 2024-04-25T09:04:13.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 10 datascan.scan_links: updating snaptable 2024-04-25T09:04:13.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- osd_op(unknown.0.0:99 17.d 17:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e127) v9 -- 0x55b812793800 con 0x55b813281400 2024-04-25T09:04:13.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 <== osd.7 v2:172.21.15.203:6824/2957998038 15 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55b812410000 con 0x55b813281400 2024-04-25T09:04:13.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55b81268d400 msgr2=0x55b812778b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x55b81268d400 0x55b812778b00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b81268d800 msgr2=0x55b812779080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x55b81268d800 0x55b812779080 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55b812792400 msgr2=0x55b81277ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x55b812792400 0x55b81277ac00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55b813281000 msgr2=0x55b812778580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 reap_dead start 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x55b813281000 0x55b812778580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55b81268dc00 msgr2=0x55b812779600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x55b81268dc00 0x55b812779600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55b812792000 msgr2=0x55b81277a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x55b812792000 0x55b81277a100 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55b81268c800 msgr2=0x55b812779b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x55b81268c800 0x55b812779b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55b813281400 msgr2=0x55b812518680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x55b813281400 0x55b812518680 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f66005640 1 -- 172.21.15.37:0/2322448888 reap_dead start 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55b81268d000 msgr2=0x55b812778000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55b81268d000 0x55b812778000 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55b812439140 tx=0x55b81245a180 comp rx=0 tx=0).stop 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813281800 msgr2=0x55b812518c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813281800 0x55b812518c00 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x55b813273290 tx=0x55b81329ec00 comp rx=0 tx=0).stop 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 shutdown_connections 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x55b81268d000 0x55b812778000 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2322448888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b813281800 0x55b812518c00 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 >> 172.21.15.37:0/2322448888 conn(0x55b813121c00 msgr2=0x55b8132de000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 shutdown_connections 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2322448888 wait complete. 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2824249262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b81235dc00 msgr2=0x55b812516b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2824249262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b81235dc00 0x55b812516b00 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x55b8131a9b90 tx=0x55b812417020 comp rx=0 tx=0).stop 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2824249262 shutdown_connections 2024-04-25T09:04:13.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2824249262 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x55b81235d800 0x55b812517600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2824249262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b81235dc00 0x55b812516b00 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 --2- 172.21.15.37:0/2824249262 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x55b813120800 0x55b812517b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2824249262 >> 172.21.15.37:0/2824249262 conn(0x55b81235cc00 msgr2=0x55b81238a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2824249262 shutdown_connections 2024-04-25T09:04:13.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.457+0000 7f7f678468c0 1 -- 172.21.15.37:0/2824249262 wait complete. 2024-04-25T09:04:13.452 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.142922, result: 2024-04-25T09:04:13.453 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:04:13.498 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:04:13.523 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-25T09:04:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-04-25T09:04:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:04:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-04-25T09:04:13.523 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:04:13.524 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:04:13.525 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.526 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.527 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 20 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 20 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:04:13.528 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 10 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:04:13.529 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:04:13.530 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: mon op complaint time = 120 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.531 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: client mount timeout = 600 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-04-25T09:04:13.532 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout:[mds] 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds = 20 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds balancer = 20 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug frag = True 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug scatterstat = False 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: mds op complaint time = 180 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: mds verify scatter = True 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T09:04:13.533 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T09:04:13.534 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2024-04-25T09:04:13.534 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2024-04-25T09:04:13.534 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:04:13.534 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2024-04-25T09:04:13.534 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.534 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2024-04-25T09:04:13.534 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2024-04-25T09:04:13.535 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2024-04-25T09:04:13.536 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.537 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.538 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 20 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 20 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 10 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:04:13.539 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2024-04-25T09:04:13.540 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:04:13.541 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:04:13.542 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2024-04-25T09:04:13.543 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:04:13.544 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2024-04-25T09:04:13.544 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = False 2024-04-25T09:04:13.544 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2024-04-25T09:04:13.544 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = True 2024-04-25T09:04:13.544 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T09:04:13.544 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T09:04:13.544 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T09:04:13.546 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:04:13.593 DEBUG:teuthology.orchestra.run.smithi203:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T09:04:13.618 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-25T09:04:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: chdir = "" 2024-04-25T09:04:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:04:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: auth supported = cephx 2024-04-25T09:04:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.618 INFO:teuthology.orchestra.run.smithi037.stdout: filestore xattr use omap = true 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: mon clock drift allowed = 1.000 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: osd crush chooseleaf type = 0 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: auth debug = true 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on old message = true 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: ms die on bug = true 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: mon pg warn max object skew = 0 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:04:13.619 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default size = 2 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow primary affinity = true 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd allow pg remap = true 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on no sortbitwise = false 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: mon warn on too few osds = false 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: mon_allow_pool_size_one = true 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:04:13.620 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: osd default data pool replay window = 5 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: mon allow pool delete = true 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: mon cluster log file level = debug 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: debug asserts on shutdown = true 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: mon health detail to clog = false 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout:[osd] 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: osd journal size = 100 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub load threshold = 5.0 2024-04-25T09:04:13.621 INFO:teuthology.orchestra.run.smithi037.stdout: osd scrub max interval = 600 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: osd recover clone overlap = true 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: osd recovery max chunk = 1048576 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug shutdown = true 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug op order = true 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug verify stray on activate = true 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug trim objects = true 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: osd open classes on start = true 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug pg log writeout = true 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:04:13.622 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: osd map max advance = 10 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: journal zero on create = true 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: filestore apply finisher threads = 3 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: bdev debug aio = true 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: osd debug misdirected ops = true 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore block size = 96636764160 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: bluestore fsck on mount = True 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluefs = 20 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: debug bluestore = 20 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: debug osd = 20 2024-04-25T09:04:13.623 INFO:teuthology.orchestra.run.smithi037.stdout: debug rocksdb = 10 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd full ratio = 0.9 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: osd objectstore = bluestore 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout:[mgr] 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: debug mgr = 20 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:04:13.624 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: mgr/telemetry/nag = false 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout:[mon] 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: debug mon = 20 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: debug paxos = 20 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: debug auth = 20 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: mon data avail warn = 5 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: mon mgr mkfs grace = 240 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd reporter subtree level = osd 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: mon osd prime pg temp = true 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:04:13.625 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: # 1m isn't quite enough 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: mon op complaint time = 120 2024-04-25T09:04:13.626 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout:[client] 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: rgw cache enabled = true 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable ops log = true 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: rgw enable usage log = true 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: client mount timeout = 600 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: debug client = 20 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: pid file = "" 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.a] 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.b] 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout:[mon.c] 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a] 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c] 2024-04-25T09:04:13.627 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b] 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d] 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout:[mds] 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds = 20 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: debug mds balancer = 20 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: debug ms = 1 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug frag = True 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: mds debug scatterstat = True 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: mds op complaint time = 180 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: mds verify scatter = True 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: osd op complaint time = 180 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: rados mon op timeout = 900 2024-04-25T09:04:13.628 INFO:teuthology.orchestra.run.smithi037.stdout: rados osd op timeout = 900 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout:[global] 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: chdir = "" 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: auth supported = cephx 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: filestore xattr use omap = true 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: mon clock drift allowed = 1.000 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: osd crush chooseleaf type = 0 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: auth debug = true 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on old message = true 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: ms die on bug = true 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.629 INFO:teuthology.orchestra.run.smithi203.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: mon pg warn max object skew = 0 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default size = 2 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow primary affinity = true 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd allow pg remap = true 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on legacy crush tunables = false 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on crush straw calc version zero = false 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on no sortbitwise = false 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on osd down out interval zero = false 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: mon warn on too few osds = false 2024-04-25T09:04:13.630 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: mon_allow_pool_size_one = true 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: osd default data pool replay window = 5 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: mon allow pool delete = true 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: mon cluster log file level = debug 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: debug asserts on shutdown = true 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: mon health detail to clog = false 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: mon host = "172.21.15.37,172.21.15.203,[v2:172.21.15.203:3301,v1:172.21.15.203:6790]" 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: fsid = 3ecc7551-d5a4-47e7-a9cf-6943742cae5b 2024-04-25T09:04:13.631 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout:[osd] 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd journal size = 100 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub load threshold = 5.0 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd scrub max interval = 600 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd mclock profile = high_recovery_ops 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd recover clone overlap = true 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd recovery max chunk = 1048576 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug shutdown = true 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug op order = true 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug verify stray on activate = true 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug trim objects = true 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd open classes on start = true 2024-04-25T09:04:13.632 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug pg log writeout = true 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: osd deep scrub update digest min age = 30 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: osd map max advance = 10 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: journal zero on create = true 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: filestore ondisk finisher threads = 3 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: filestore apply finisher threads = 3 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: bdev debug aio = true 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: osd debug misdirected ops = true 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore block size = 96636764160 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: bluestore fsck on mount = True 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluefs = 20 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: debug bluestore = 20 2024-04-25T09:04:13.633 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: debug osd = 20 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: debug rocksdb = 10 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd full ratio = 0.9 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd nearfull ratio = 0.8 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: osd failsafe full ratio = 0.95 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: osd objectstore = bluestore 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout:[mgr] 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: debug mgr = 20 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T09:04:13.634 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: mgr/telemetry/nag = false 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout:[mon] 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: debug mon = 20 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: debug paxos = 20 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: debug auth = 20 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: mon data avail warn = 5 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: mon mgr mkfs grace = 240 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min pgs per osd = 4 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd reporter subtree level = osd 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: mon osd prime pg temp = true 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: mon reweight min bytes per osd = 10 2024-04-25T09:04:13.635 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: # 1m isn't quite enough 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: mon_down_mkfs_grace = 2m 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: mon_warn_on_filestore_osds = false 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: mon op complaint time = 120 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout:[client] 2024-04-25T09:04:13.636 INFO:teuthology.orchestra.run.smithi203.stdout: rgw cache enabled = true 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable ops log = true 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout: rgw enable usage log = true 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout: client mount timeout = 600 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout: debug client = 20 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout: pid file = "" 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.a] 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.b] 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout:[mon.c] 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.a] 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.c] 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.b] 2024-04-25T09:04:13.637 INFO:teuthology.orchestra.run.smithi203.stdout:[mds.d] 2024-04-25T09:04:13.638 INFO:teuthology.orchestra.run.smithi203.stdout:[mds] 2024-04-25T09:04:13.638 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds = 20 2024-04-25T09:04:13.638 INFO:teuthology.orchestra.run.smithi203.stdout: debug mds balancer = 20 2024-04-25T09:04:13.638 INFO:teuthology.orchestra.run.smithi203.stdout: debug ms = 1 2024-04-25T09:04:13.638 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug frag = True 2024-04-25T09:04:13.638 INFO:teuthology.orchestra.run.smithi203.stdout: mds debug scatterstat = True 2024-04-25T09:04:13.638 INFO:teuthology.orchestra.run.smithi203.stdout: mds op complaint time = 180 2024-04-25T09:04:13.638 INFO:teuthology.orchestra.run.smithi203.stdout: mds verify scatter = True 2024-04-25T09:04:13.638 INFO:teuthology.orchestra.run.smithi203.stdout: osd op complaint time = 180 2024-04-25T09:04:13.638 INFO:teuthology.orchestra.run.smithi203.stdout: rados mon op timeout = 900 2024-04-25T09:04:13.638 INFO:teuthology.orchestra.run.smithi203.stdout: rados osd op timeout = 900 2024-04-25T09:04:13.640 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T09:04:13.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e84673640 1 Processor -- start 2024-04-25T09:04:13.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e84673640 1 -- start start 2024-04-25T09:04:13.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e84673640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f0e7c06f4a0 0x7f0e7c099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e84673640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c0b0e10 con 0x7f0e7c0b0a30 2024-04-25T09:04:13.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e84673640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e7c0d0180 con 0x7f0e7c099fb0 2024-04-25T09:04:13.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e84673640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7c0d0d90 con 0x7f0e7c06f4a0 2024-04-25T09:04:13.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8240f640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f0e7c0b0a30 0x7f0e7c0cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48368/0 (socket says 172.21.15.37:48368) 2024-04-25T09:04:13.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8240f640 1 -- 172.21.15.37:0/1708066232 learned_addr learned my addr 172.21.15.37:0/1708066232 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:13.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e81c0e640 1 --2- 172.21.15.37:0/1708066232 >> v2:172.21.15.203:3300/0 conn(0x7f0e7c06f4a0 0x7f0e7c099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276906414 0 0) 0x7f0e7c0b0e10 con 0x7f0e7c0b0a30 2024-04-25T09:04:13.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e68003530 con 0x7f0e7c0b0a30 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3341481471 0 0) 0x7f0e68003530 con 0x7f0e7c0b0a30 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 >> v2:172.21.15.203:3300/0 conn(0x7f0e7c06f4a0 msgr2=0x7f0e7c099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 --2- 172.21.15.37:0/1708066232 >> v2:172.21.15.203:3300/0 conn(0x7f0e7c06f4a0 0x7f0e7c099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 >> v1:172.21.15.203:6789/0 conn(0x7f0e7c099fb0 legacy=0x7f0e7c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c0d0460 con 0x7f0e7c0b0a30 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0e70001680 con 0x7f0e7c0b0a30 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 >> v1:172.21.15.37:6789/0 conn(0x7f0e7c0b0a30 legacy=0x7f0e7c0cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 --2- 172.21.15.37:0/1708066232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e68003a30 0x7f0e68003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 --2- 172.21.15.37:0/1708066232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 0x7f0e68004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 --2- 172.21.15.37:0/1708066232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e680051d0 0x7f0e6800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0d0d90 con 0x7f0e68004810 2024-04-25T09:04:13.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6800c100 con 0x7f0e680051d0 2024-04-25T09:04:13.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6800c300 con 0x7f0e68003a30 2024-04-25T09:04:13.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e81c0e640 1 --2- 172.21.15.37:0/1708066232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 0x7f0e68004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e8240f640 1 --2- 172.21.15.37:0/1708066232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e68003a30 0x7f0e68003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e81c0e640 1 -- 172.21.15.37:0/1708066232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e68003a30 msgr2=0x7f0e68003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e81c0e640 1 --2- 172.21.15.37:0/1708066232 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e68003a30 0x7f0e68003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e81c0e640 1 -- 172.21.15.37:0/1708066232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e680051d0 msgr2=0x7f0e6800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e81c0e640 1 --2- 172.21.15.37:0/1708066232 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e680051d0 0x7f0e6800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e81c0e640 1 -- 172.21.15.37:0/1708066232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e68003530 con 0x7f0e68004810 2024-04-25T09:04:13.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.825+0000 7f0e82c10640 1 -- 172.21.15.37:0/1708066232 reap_dead start 2024-04-25T09:04:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.829+0000 7f0e81c0e640 1 --2- 172.21.15.37:0/1708066232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 0x7f0e68004c90 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009fc0 tx=0x7f0e7801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caad04bb1f944a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.829+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e780b2070 con 0x7f0e68004810 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.829+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e780a5ab0 con 0x7f0e68004810 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.829+0000 7f0e8140d640 1 -- 172.21.15.37:0/1708066232 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e78009c10 con 0x7f0e68004810 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.829+0000 7f0e84673640 1 -- 172.21.15.37:0/1708066232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 msgr2=0x7f0e68004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.829+0000 7f0e84673640 1 --2- 172.21.15.37:0/1708066232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 0x7f0e68004c90 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009fc0 tx=0x7f0e7801cca0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.829+0000 7f0e84673640 1 -- 172.21.15.37:0/1708066232 shutdown_connections 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.829+0000 7f0e84673640 1 --2- 172.21.15.37:0/1708066232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 0x7f0e68004c90 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.829+0000 7f0e84673640 1 -- 172.21.15.37:0/1708066232 >> 172.21.15.37:0/1708066232 conn(0x7f0e7c0b30b0 msgr2=0x7f0e7c0b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.829+0000 7f0e84673640 1 -- 172.21.15.37:0/1708066232 shutdown_connections 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.829+0000 7f0e84673640 1 -- 172.21.15.37:0/1708066232 wait complete. 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e84673640 1 Processor -- start 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e84673640 1 -- start start 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e84673640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 0x7f0e7c0c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e84673640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e7c0b6a20 0x7f0e7c151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e84673640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c0ca290 0x7f0e7c0ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e84673640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c099a70 con 0x7f0e68004810 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e84673640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c0d0a20 con 0x7f0e7c0ca290 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e84673640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7c099bf0 con 0x7f0e7c0b6a20 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e8240f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 0x7f0e7c0c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e8240f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 0x7f0e7c0c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43066/0 (socket says 172.21.15.37:43066) 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e8240f640 1 -- 172.21.15.37:0/2730695377 learned_addr learned my addr 172.21.15.37:0/2730695377 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e8240f640 1 -- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e7c0b6a20 msgr2=0x7f0e7c151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e8240f640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e7c0b6a20 0x7f0e7c151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e8240f640 1 -- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c0ca290 msgr2=0x7f0e7c0ca710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e8240f640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c0ca290 0x7f0e7c0ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e8240f640 1 -- 172.21.15.37:0/2730695377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c0c5ba0 con 0x7f0e68004810 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e8240f640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 0x7f0e7c0c90a0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f0e70010b40 tx=0x7f0e70005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e6effd640 1 -- 172.21.15.37:0/2730695377 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7001b070 con 0x7f0e68004810 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e6effd640 1 -- 172.21.15.37:0/2730695377 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e70011040 con 0x7f0e68004810 2024-04-25T09:04:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e84673640 1 -- 172.21.15.37:0/2730695377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7c0c9680 con 0x7f0e68004810 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e6effd640 1 -- 172.21.15.37:0/2730695377 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7000ed70 con 0x7f0e68004810 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e84673640 1 -- 172.21.15.37:0/2730695377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7c0b4350 con 0x7f0e68004810 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e84673640 1 -- 172.21.15.37:0/2730695377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7c04efa0 con 0x7f0e68004810 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e6effd640 1 -- 172.21.15.37:0/2730695377 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f0e7001e070 con 0x7f0e68004810 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e6effd640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 0x7f0e60069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e6effd640 1 -- 172.21.15.37:0/2730695377 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f0e70087c90 con 0x7f0e68004810 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.833+0000 7f0e81c0e640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 0x7f0e60069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.837+0000 7f0e81c0e640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 0x7f0e60069420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0e780af000 tx=0x7f0e7801b4e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:13.837+0000 7f0e6effd640 1 -- 172.21.15.37:0/2730695377 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f0e7c0c5ba0 con 0x7f0e68004810 2024-04-25T09:04:14.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.041+0000 7f0e84673640 1 -- 172.21.15.37:0/2730695377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f0e7c06fa80 con 0x7f0e68004810 2024-04-25T09:04:14.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.949+0000 7f0e6effd640 1 -- 172.21.15.37:0/2730695377 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v106) v1 ==== 162+0+0 (secure 0 0 0) 0x7f0e70058af0 con 0x7f0e68004810 2024-04-25T09:04:14.939 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T09:04:14.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 -- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 msgr2=0x7f0e60069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 0x7f0e60069420 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0e780af000 tx=0x7f0e7801b4e0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 -- 172.21.15.37:0/2730695377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 msgr2=0x7f0e7c0c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 0x7f0e7c0c90a0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f0e70010b40 tx=0x7f0e70005740 comp rx=0 tx=0).stop 2024-04-25T09:04:14.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 -- 172.21.15.37:0/2730695377 shutdown_connections 2024-04-25T09:04:14.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0e60066f60 0x7f0e60069420 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f0e7c0ca290 0x7f0e7c0ca710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f0e7c0b6a20 0x7f0e7c151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 --2- 172.21.15.37:0/2730695377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e68004810 0x7f0e7c0c90a0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 -- 172.21.15.37:0/2730695377 >> 172.21.15.37:0/2730695377 conn(0x7f0e7c0b30b0 msgr2=0x7f0e7c0cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 -- 172.21.15.37:0/2730695377 shutdown_connections 2024-04-25T09:04:14.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:14.953+0000 7f0e84673640 1 -- 172.21.15.37:0/2730695377 wait complete. 2024-04-25T09:04:14.968 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:15.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732c66a640 1 Processor -- start 2024-04-25T09:04:15.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732c66a640 1 -- start start 2024-04-25T09:04:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732c66a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f732415a700 0x7f732415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732c66a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f73240b24a0 0x7f73240b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732c66a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73240b2090 con 0x7f73240b3f50 2024-04-25T09:04:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732c66a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f732415f200 con 0x7f732415a700 2024-04-25T09:04:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732c66a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73240b43e0 con 0x7f73240b24a0 2024-04-25T09:04:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732a406640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f732415a700 0x7f732415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732a406640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f732415a700 0x7f732415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43074/0 (socket says 172.21.15.37:43074) 2024-04-25T09:04:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732a406640 1 -- 172.21.15.37:0/86279797 learned_addr learned my addr 172.21.15.37:0/86279797 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732ac07640 1 --2- 172.21.15.37:0/86279797 >> v2:172.21.15.203:3300/0 conn(0x7f73240b24a0 0x7f73240b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732a406640 1 -- 172.21.15.37:0/86279797 >> v2:172.21.15.203:3300/0 conn(0x7f73240b24a0 msgr2=0x7f73240b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732a406640 1 --2- 172.21.15.37:0/86279797 >> v2:172.21.15.203:3300/0 conn(0x7f73240b24a0 0x7f73240b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732a406640 1 -- 172.21.15.37:0/86279797 >> v1:172.21.15.203:6789/0 conn(0x7f73240b3f50 legacy=0x7f73240cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732a406640 1 -- 172.21.15.37:0/86279797 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73240b46c0 con 0x7f732415a700 2024-04-25T09:04:15.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.101+0000 7f732ac07640 1 --2- 172.21.15.37:0/86279797 >> v2:172.21.15.203:3300/0 conn(0x7f73240b24a0 0x7f73240b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:15.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f732a406640 1 --2- 172.21.15.37:0/86279797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a700 0x7f732415ecc0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f732406bd20 tx=0x7f732001ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed46698e9e623436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f7329404640 1 -- 172.21.15.37:0/86279797 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73200b4070 con 0x7f732415a700 2024-04-25T09:04:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f7329404640 1 -- 172.21.15.37:0/86279797 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73200a7a30 con 0x7f732415a700 2024-04-25T09:04:15.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f7329404640 1 -- 172.21.15.37:0/86279797 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73200a7e00 con 0x7f732415a700 2024-04-25T09:04:15.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f732c66a640 1 -- 172.21.15.37:0/86279797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a700 msgr2=0x7f732415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f732c66a640 1 --2- 172.21.15.37:0/86279797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a700 0x7f732415ecc0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f732406bd20 tx=0x7f732001ea70 comp rx=0 tx=0).stop 2024-04-25T09:04:15.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f732c66a640 1 -- 172.21.15.37:0/86279797 shutdown_connections 2024-04-25T09:04:15.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f732c66a640 1 --2- 172.21.15.37:0/86279797 >> v2:172.21.15.203:3300/0 conn(0x7f73240b24a0 0x7f73240b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f732c66a640 1 --2- 172.21.15.37:0/86279797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732415a700 0x7f732415ecc0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f732c66a640 1 -- 172.21.15.37:0/86279797 >> 172.21.15.37:0/86279797 conn(0x7f73240b11a0 msgr2=0x7f73240b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:15.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f732c66a640 1 -- 172.21.15.37:0/86279797 shutdown_connections 2024-04-25T09:04:15.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f732c66a640 1 -- 172.21.15.37:0/86279797 wait complete. 2024-04-25T09:04:15.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f732c66a640 1 Processor -- start 2024-04-25T09:04:15.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.105+0000 7f732c66a640 1 -- start start 2024-04-25T09:04:15.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732c66a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73240b24a0 0x7f7324155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732c66a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415a700 0x7f73241f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732c66a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732406db00 0x7f73240ab860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732c66a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73240cd310 con 0x7f732406db00 2024-04-25T09:04:15.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732c66a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73240ce600 con 0x7f73240b24a0 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732c66a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73240cd490 con 0x7f732415a700 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732ac07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732406db00 0x7f73240ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732ac07640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732406db00 0x7f73240ab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43082/0 (socket says 172.21.15.37:43082) 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732ac07640 1 -- 172.21.15.37:0/3348797883 learned_addr learned my addr 172.21.15.37:0/3348797883 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732a406640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73240b24a0 0x7f7324155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f7329c05640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415a700 0x7f73241f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732ac07640 1 -- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415a700 msgr2=0x7f73241f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732ac07640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415a700 0x7f73241f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732ac07640 1 -- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73240b24a0 msgr2=0x7f7324155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732ac07640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73240b24a0 0x7f7324155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732ac07640 1 -- 172.21.15.37:0/3348797883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73241f5ea0 con 0x7f732406db00 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732a406640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73240b24a0 0x7f7324155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732ac07640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732406db00 0x7f73240ab860 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f731c00cd50 tx=0x7f731c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:15.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f731affd640 1 -- 172.21.15.37:0/3348797883 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c00f680 con 0x7f732406db00 2024-04-25T09:04:15.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f731affd640 1 -- 172.21.15.37:0/3348797883 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c00f840 con 0x7f732406db00 2024-04-25T09:04:15.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f731affd640 1 -- 172.21.15.37:0/3348797883 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c00fc30 con 0x7f732406db00 2024-04-25T09:04:15.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732c66a640 1 -- 172.21.15.37:0/3348797883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73241f6180 con 0x7f732406db00 2024-04-25T09:04:15.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732c66a640 1 -- 172.21.15.37:0/3348797883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73240ab340 con 0x7f732406db00 2024-04-25T09:04:15.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732c66a640 1 -- 172.21.15.37:0/3348797883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73240ab540 con 0x7f732406db00 2024-04-25T09:04:15.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f731affd640 1 -- 172.21.15.37:0/3348797883 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f731c00e3e0 con 0x7f732406db00 2024-04-25T09:04:15.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f731affd640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7308066f90 0x7f7308069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f731affd640 1 -- 172.21.15.37:0/3348797883 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f731c085ff0 con 0x7f732406db00 2024-04-25T09:04:15.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.109+0000 7f732a406640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7308066f90 0x7f7308069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.113+0000 7f732a406640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7308066f90 0x7f7308069450 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f73200b1000 tx=0x7f73200a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:15.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.117+0000 7f731affd640 1 -- 172.21.15.37:0/3348797883 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f73240ab540 con 0x7f732406db00 2024-04-25T09:04:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.313+0000 7f732c66a640 1 -- 172.21.15.37:0/3348797883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f732404efa0 con 0x7f732406db00 2024-04-25T09:04:15.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.313+0000 7f731affd640 1 -- 172.21.15.37:0/3348797883 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f731c014030 con 0x7f732406db00 2024-04-25T09:04:15.303 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:15.303 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":107,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:04:14.958475+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":107,"state":"up:replay","state_seq":1,"addr":"172.21.15.37:6835/2361387213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2361387213},{"type":"v1","addr":"172.21.15.37:6835","nonce":2361387213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:04:15.303 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 107 2024-04-25T09:04:15.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 -- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7308066f90 msgr2=0x7f7308069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7308066f90 0x7f7308069450 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f73200b1000 tx=0x7f73200a9070 comp rx=0 tx=0).stop 2024-04-25T09:04:15.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 -- 172.21.15.37:0/3348797883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732406db00 msgr2=0x7f73240ab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732406db00 0x7f73240ab860 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f731c00cd50 tx=0x7f731c00a650 comp rx=0 tx=0).stop 2024-04-25T09:04:15.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 -- 172.21.15.37:0/3348797883 shutdown_connections 2024-04-25T09:04:15.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7308066f90 0x7f7308069450 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f732406db00 0x7f73240ab860 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f732415a700 0x7f73241f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 --2- 172.21.15.37:0/3348797883 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f73240b24a0 0x7f7324155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 -- 172.21.15.37:0/3348797883 >> 172.21.15.37:0/3348797883 conn(0x7f73240b11a0 msgr2=0x7f732415cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:15.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 -- 172.21.15.37:0/3348797883 shutdown_connections 2024-04-25T09:04:15.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:15.317+0000 7f732c66a640 1 -- 172.21.15.37:0/3348797883 wait complete. 2024-04-25T09:04:15.726 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:03:32.494159+0000', 'modified': '2024-04-25T09:04:14.958475+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9990': {'gid': 9990, 'name': 'a', 'rank': 0, 'incarnation': 107, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.37:6835/2361387213', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2361387213}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2361387213}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:15.726 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9990:up:replay 2024-04-25T09:04:16.726 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:16.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f4021350640 1 Processor -- start 2024-04-25T09:04:16.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f4021350640 1 -- start start 2024-04-25T09:04:16.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f4021350640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f401c0ce510 0x7f401c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f4021350640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f401c1451f0 0x7f401c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f4021350640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f401c0cee30 con 0x7f401c144ca0 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f4021350640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c0cc2b0 con 0x7f401c0ce510 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f4021350640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c06e290 con 0x7f401c1451f0 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f401affd640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f401c0ce510 0x7f401c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f401a7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f401c144ca0 0x7f401c06e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:48370/0 (socket says 172.21.15.37:48370) 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f401a7fc640 1 -- 172.21.15.37:0/4037671412 learned_addr learned my addr 172.21.15.37:0/4037671412 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f401b7fe640 1 --2- 172.21.15.37:0/4037671412 >> v2:172.21.15.203:3300/0 conn(0x7f401c1451f0 0x7f401c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f401affd640 1 -- 172.21.15.37:0/4037671412 >> v2:172.21.15.203:3300/0 conn(0x7f401c1451f0 msgr2=0x7f401c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f401affd640 1 --2- 172.21.15.37:0/4037671412 >> v2:172.21.15.203:3300/0 conn(0x7f401c1451f0 0x7f401c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f401affd640 1 -- 172.21.15.37:0/4037671412 >> v1:172.21.15.37:6789/0 conn(0x7f401c144ca0 legacy=0x7f401c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.873+0000 7f401affd640 1 -- 172.21.15.37:0/4037671412 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0ccc70 con 0x7f401c0ce510 2024-04-25T09:04:16.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f401affd640 1 --2- 172.21.15.37:0/4037671412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0ce510 0x7f401c0ce8f0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f401c0b48f0 tx=0x7f400c01c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dcee7907e74358 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4019ffb640 1 -- 172.21.15.37:0/4037671412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c0b0070 con 0x7f401c0ce510 2024-04-25T09:04:16.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4019ffb640 1 -- 172.21.15.37:0/4037671412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c0ab600 con 0x7f401c0ce510 2024-04-25T09:04:16.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4019ffb640 1 -- 172.21.15.37:0/4037671412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c0ab9d0 con 0x7f401c0ce510 2024-04-25T09:04:16.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4021350640 1 -- 172.21.15.37:0/4037671412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0ce510 msgr2=0x7f401c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4021350640 1 --2- 172.21.15.37:0/4037671412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0ce510 0x7f401c0ce8f0 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7f401c0b48f0 tx=0x7f400c01c3a0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4021350640 1 -- 172.21.15.37:0/4037671412 shutdown_connections 2024-04-25T09:04:16.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4021350640 1 --2- 172.21.15.37:0/4037671412 >> v2:172.21.15.203:3300/0 conn(0x7f401c1451f0 0x7f401c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4021350640 1 --2- 172.21.15.37:0/4037671412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c0ce510 0x7f401c0ce8f0 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4021350640 1 -- 172.21.15.37:0/4037671412 >> 172.21.15.37:0/4037671412 conn(0x7f401c0b11a0 msgr2=0x7f401c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:16.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4021350640 1 -- 172.21.15.37:0/4037671412 shutdown_connections 2024-04-25T09:04:16.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4021350640 1 -- 172.21.15.37:0/4037671412 wait complete. 2024-04-25T09:04:16.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4021350640 1 Processor -- start 2024-04-25T09:04:16.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.877+0000 7f4021350640 1 -- start start 2024-04-25T09:04:16.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f4021350640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f401c0ce510 0x7f401c1f5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f4021350640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f401c144ca0 0x7f401c0ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f4021350640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1451f0 0x7f401c13db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f4021350640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c06ee30 con 0x7f401c1451f0 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f4021350640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c06df20 con 0x7f401c0ce510 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f4021350640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c0cc900 con 0x7f401c144ca0 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1451f0 0x7f401c13db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1451f0 0x7f401c13db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43098/0 (socket says 172.21.15.37:43098) 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401b7fe640 1 -- 172.21.15.37:0/2190351428 learned_addr learned my addr 172.21.15.37:0/2190351428 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401affd640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f401c0ce510 0x7f401c1f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401a7fc640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f401c144ca0 0x7f401c0ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401b7fe640 1 -- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f401c144ca0 msgr2=0x7f401c0ca2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401b7fe640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f401c144ca0 0x7f401c0ca2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401b7fe640 1 -- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f401c0ce510 msgr2=0x7f401c1f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401b7fe640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f401c0ce510 0x7f401c1f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401b7fe640 1 -- 172.21.15.37:0/2190351428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c1f5950 con 0x7f401c1451f0 2024-04-25T09:04:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401b7fe640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1451f0 0x7f401c13db70 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f4014009fd0 tx=0x7f401400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f40037fe640 1 -- 172.21.15.37:0/2190351428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014014420 con 0x7f401c1451f0 2024-04-25T09:04:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f40037fe640 1 -- 172.21.15.37:0/2190351428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4014019070 con 0x7f401c1451f0 2024-04-25T09:04:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f4021350640 1 -- 172.21.15.37:0/2190351428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c1e0e80 con 0x7f401c1451f0 2024-04-25T09:04:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f40037fe640 1 -- 172.21.15.37:0/2190351428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014014800 con 0x7f401c1451f0 2024-04-25T09:04:16.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f4021350640 1 -- 172.21.15.37:0/2190351428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c1e1440 con 0x7f401c1451f0 2024-04-25T09:04:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f4021350640 1 -- 172.21.15.37:0/2190351428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c04efa0 con 0x7f401c1451f0 2024-04-25T09:04:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f40037fe640 1 -- 172.21.15.37:0/2190351428 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f40140149a0 con 0x7f401c1451f0 2024-04-25T09:04:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f40037fe640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 0x7f4008069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f40037fe640 1 -- 172.21.15.37:0/2190351428 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f4014085ae0 con 0x7f401c1451f0 2024-04-25T09:04:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.881+0000 7f401affd640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 0x7f4008069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.885+0000 7f401affd640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 0x7f4008069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f401c070fe0 tx=0x7f400c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:16.885+0000 7f40037fe640 1 -- 172.21.15.37:0/2190351428 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f4014056890 con 0x7f401c1451f0 2024-04-25T09:04:17.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 -- 172.21.15.37:0/2190351428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f401c0b1cc0 con 0x7f401c1451f0 2024-04-25T09:04:17.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f40037fe640 1 -- 172.21.15.37:0/2190351428 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f4014056190 con 0x7f401c1451f0 2024-04-25T09:04:17.070 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:17.070 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":109,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:04:15.975744+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":107,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.37:6835/2361387213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2361387213},{"type":"v1","addr":"172.21.15.37:6835","nonce":2361387213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:04:17.070 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 109 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 -- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 msgr2=0x7f4008069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 0x7f4008069420 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f401c070fe0 tx=0x7f400c008040 comp rx=0 tx=0).stop 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 -- 172.21.15.37:0/2190351428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1451f0 msgr2=0x7f401c13db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1451f0 0x7f401c13db70 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f4014009fd0 tx=0x7f401400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 -- 172.21.15.37:0/2190351428 shutdown_connections 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f4008066f60 0x7f4008069420 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f401c1451f0 0x7f401c13db70 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f401c144ca0 0x7f401c0ca2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 --2- 172.21.15.37:0/2190351428 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f401c0ce510 0x7f401c1f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 -- 172.21.15.37:0/2190351428 >> 172.21.15.37:0/2190351428 conn(0x7f401c0b11a0 msgr2=0x7f401c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 -- 172.21.15.37:0/2190351428 shutdown_connections 2024-04-25T09:04:17.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:17.081+0000 7f4021350640 1 -- 172.21.15.37:0/2190351428 wait complete. 2024-04-25T09:04:17.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 109, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:03:32.494159+0000', 'modified': '2024-04-25T09:04:15.975744+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9990': {'gid': 9990, 'name': 'a', 'rank': 0, 'incarnation': 107, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.37:6835/2361387213', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2361387213}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2361387213}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:17.490 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_9990:up:rejoin 2024-04-25T09:04:18.492 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:18.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.637+0000 7fbef901a640 1 Processor -- start 2024-04-25T09:04:18.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef901a640 1 -- start start 2024-04-25T09:04:18.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef901a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbef415a700 0x7fbef415ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef901a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbef40b24a0 0x7fbef40b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef901a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbef40b2090 con 0x7fbef40b3f50 2024-04-25T09:04:18.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef901a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbef415f200 con 0x7fbef415a700 2024-04-25T09:04:18.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef901a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40b43e0 con 0x7fbef40b24a0 2024-04-25T09:04:18.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef2d9d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fbef415a700 0x7fbef415ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef359e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbef40b24a0 0x7fbef40b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef259c640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fbef40b3f50 0x7fbef40cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:51046/0 (socket says 172.21.15.37:51046) 2024-04-25T09:04:18.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef259c640 1 -- 172.21.15.37:0/2595087832 learned_addr learned my addr 172.21.15.37:0/2595087832 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:18.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef2d9d640 1 --2- 172.21.15.37:0/2595087832 >> v2:172.21.15.203:3300/0 conn(0x7fbef415a700 0x7fbef415ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:48990/0 (socket says 172.21.15.37:48990) 2024-04-25T09:04:18.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef2d9d640 1 -- 172.21.15.37:0/2595087832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbef40b24a0 msgr2=0x7fbef40b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:18.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef2d9d640 1 --2- 172.21.15.37:0/2595087832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbef40b24a0 0x7fbef40b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef2d9d640 1 -- 172.21.15.37:0/2595087832 >> v1:172.21.15.203:6789/0 conn(0x7fbef40b3f50 legacy=0x7fbef40cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.641+0000 7fbef2d9d640 1 -- 172.21.15.37:0/2595087832 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef40b46c0 con 0x7fbef415a700 2024-04-25T09:04:18.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef2d9d640 1 --2- 172.21.15.37:0/2595087832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbef415a700 0x7fbef415ecc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009d90 tx=0x7fbee401c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=667a1abf1afd2992 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef1d9b640 1 -- 172.21.15.37:0/2595087832 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee40b2070 con 0x7fbef415a700 2024-04-25T09:04:18.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef1d9b640 1 -- 172.21.15.37:0/2595087832 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee40a5800 con 0x7fbef415a700 2024-04-25T09:04:18.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef1d9b640 1 -- 172.21.15.37:0/2595087832 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee40a5bd0 con 0x7fbef415a700 2024-04-25T09:04:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef901a640 1 -- 172.21.15.37:0/2595087832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbef415a700 msgr2=0x7fbef415ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef901a640 1 --2- 172.21.15.37:0/2595087832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbef415a700 0x7fbef415ecc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009d90 tx=0x7fbee401c850 comp rx=0 tx=0).stop 2024-04-25T09:04:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef901a640 1 -- 172.21.15.37:0/2595087832 shutdown_connections 2024-04-25T09:04:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef901a640 1 --2- 172.21.15.37:0/2595087832 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbef40b24a0 0x7fbef40b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef901a640 1 --2- 172.21.15.37:0/2595087832 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbef415a700 0x7fbef415ecc0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef901a640 1 -- 172.21.15.37:0/2595087832 >> 172.21.15.37:0/2595087832 conn(0x7fbef40b11a0 msgr2=0x7fbef40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef901a640 1 -- 172.21.15.37:0/2595087832 shutdown_connections 2024-04-25T09:04:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.645+0000 7fbef901a640 1 -- 172.21.15.37:0/2595087832 wait complete. 2024-04-25T09:04:18.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef901a640 1 Processor -- start 2024-04-25T09:04:18.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef901a640 1 -- start start 2024-04-25T09:04:18.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef901a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbef40b24a0 0x7fbef414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef901a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbef415a700 0x7fbef414fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef901a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbef41445e0 0x7fbef414e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef901a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40cd310 con 0x7fbef40b24a0 2024-04-25T09:04:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef901a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40ce600 con 0x7fbef41445e0 2024-04-25T09:04:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef901a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40cd490 con 0x7fbef415a700 2024-04-25T09:04:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbef40b24a0 0x7fbef414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbef40b24a0 0x7fbef414bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43112/0 (socket says 172.21.15.37:43112) 2024-04-25T09:04:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef2d9d640 1 -- 172.21.15.37:0/4105043015 learned_addr learned my addr 172.21.15.37:0/4105043015 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef259c640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbef415a700 0x7fbef414fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef359e640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbef41445e0 0x7fbef414e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef2d9d640 1 -- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbef415a700 msgr2=0x7fbef414fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef2d9d640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbef415a700 0x7fbef414fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef2d9d640 1 -- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbef41445e0 msgr2=0x7fbef414e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef2d9d640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbef41445e0 0x7fbef414e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef2d9d640 1 -- 172.21.15.37:0/4105043015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef414ed10 con 0x7fbef40b24a0 2024-04-25T09:04:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef2d9d640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbef40b24a0 0x7fbef414bc50 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fbee401a480 tx=0x7fbee40195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/4105043015 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee40bd020 con 0x7fbef40b24a0 2024-04-25T09:04:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/4105043015 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee40b8670 con 0x7fbef40b24a0 2024-04-25T09:04:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef901a640 1 -- 172.21.15.37:0/4105043015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef414eff0 con 0x7fbef40b24a0 2024-04-25T09:04:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/4105043015 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee40b2040 con 0x7fbef40b24a0 2024-04-25T09:04:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef901a640 1 -- 172.21.15.37:0/4105043015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef414f5b0 con 0x7fbef40b24a0 2024-04-25T09:04:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.649+0000 7fbef901a640 1 -- 172.21.15.37:0/4105043015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef404efa0 con 0x7fbef40b24a0 2024-04-25T09:04:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.653+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/4105043015 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fbee40c0020 con 0x7fbef40b24a0 2024-04-25T09:04:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.653+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbed4066f60 0x7fbed4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.653+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/4105043015 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fbee4120220 con 0x7fbef40b24a0 2024-04-25T09:04:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.653+0000 7fbef259c640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbed4066f60 0x7fbed4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.653+0000 7fbef259c640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbed4066f60 0x7fbed4069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbef415ddb0 tx=0x7fbedc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.657+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/4105043015 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fbee40c7030 con 0x7fbef40b24a0 2024-04-25T09:04:18.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.849+0000 7fbef901a640 1 -- 172.21.15.37:0/4105043015 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbef406f260 con 0x7fbef40b24a0 2024-04-25T09:04:18.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.849+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/4105043015 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4951 (secure 0 0 0) 0x7fbee4016560 con 0x7fbef40b24a0 2024-04-25T09:04:18.840 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:18.840 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:04:16.988456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/2361387213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2361387213},{"type":"v1","addr":"172.21.15.37:6835","nonce":2361387213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:04:18.840 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 110 2024-04-25T09:04:18.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 -- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbed4066f60 msgr2=0x7fbed4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbed4066f60 0x7fbed4069420 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbef415ddb0 tx=0x7fbedc009040 comp rx=0 tx=0).stop 2024-04-25T09:04:18.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 -- 172.21.15.37:0/4105043015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbef40b24a0 msgr2=0x7fbef414bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbef40b24a0 0x7fbef414bc50 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fbee401a480 tx=0x7fbee40195f0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 -- 172.21.15.37:0/4105043015 shutdown_connections 2024-04-25T09:04:18.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fbed4066f60 0x7fbed4069420 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fbef41445e0 0x7fbef414e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fbef415a700 0x7fbef414fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 --2- 172.21.15.37:0/4105043015 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbef40b24a0 0x7fbef414bc50 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 -- 172.21.15.37:0/4105043015 >> 172.21.15.37:0/4105043015 conn(0x7fbef40b11a0 msgr2=0x7fbef40b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 -- 172.21.15.37:0/4105043015 shutdown_connections 2024-04-25T09:04:18.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:18.853+0000 7fbef901a640 1 -- 172.21.15.37:0/4105043015 wait complete. 2024-04-25T09:04:18.945 INFO:tasks.ceph.mds.a.smithi037.stderr:2024-04-25T09:04:18.957+0000 7f95d617b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:04:19.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:03:32.494159+0000', 'modified': '2024-04-25T09:04:16.988456+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 127, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9990': {'gid': 9990, 'name': 'a', 'rank': 0, 'incarnation': 107, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.37:6835/2361387213', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.37:6834', 'nonce': 2361387213}, {'type': 'v1', 'addr': '172.21.15.37:6835', 'nonce': 2361387213}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:19.263 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:04:19.263 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:04:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.409+0000 7f163b794640 1 Processor -- start 2024-04-25T09:04:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.409+0000 7f163b794640 1 -- start start 2024-04-25T09:04:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.409+0000 7f163b794640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f163415da80 0x7f16340d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.409+0000 7f163b794640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16340d1cd0 0x7f16340d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.409+0000 7f163b794640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f16340d0620 0x7f16340ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.409+0000 7f163b794640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163406c440 con 0x7f163415da80 2024-04-25T09:04:19.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.409+0000 7f163b794640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163406be60 con 0x7f16340d0620 2024-04-25T09:04:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.409+0000 7f163b794640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163415de60 con 0x7f16340d1cd0 2024-04-25T09:04:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f1639530640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f163415da80 0x7f16340d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f1639530640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f163415da80 0x7f16340d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51324/0 (socket says 172.21.15.37:51324) 2024-04-25T09:04:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f1639530640 1 -- 172.21.15.37:0/1895851089 learned_addr learned my addr 172.21.15.37:0/1895851089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f1639d31640 1 --2- 172.21.15.37:0/1895851089 >> v2:172.21.15.203:3300/0 conn(0x7f16340d0620 0x7f16340ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f1638d2f640 1 --2- 172.21.15.37:0/1895851089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16340d1cd0 0x7f16340d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f1639530640 1 -- 172.21.15.37:0/1895851089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16340d1cd0 msgr2=0x7f16340d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f1639530640 1 --2- 172.21.15.37:0/1895851089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16340d1cd0 0x7f16340d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f1639530640 1 -- 172.21.15.37:0/1895851089 >> v2:172.21.15.203:3300/0 conn(0x7f16340d0620 msgr2=0x7f16340ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f1639530640 1 --2- 172.21.15.37:0/1895851089 >> v2:172.21.15.203:3300/0 conn(0x7f16340d0620 0x7f16340ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f1639530640 1 -- 172.21.15.37:0/1895851089 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340d2450 con 0x7f163415da80 2024-04-25T09:04:19.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f1639530640 1 --2- 172.21.15.37:0/1895851089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f163415da80 0x7f16340d4230 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f163406de60 tx=0x7f163001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d577d7c508119966 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f162bfff640 1 -- 172.21.15.37:0/1895851089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16300b0070 con 0x7f163415da80 2024-04-25T09:04:19.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f162bfff640 1 -- 172.21.15.37:0/1895851089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16300ab600 con 0x7f163415da80 2024-04-25T09:04:19.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f162bfff640 1 -- 172.21.15.37:0/1895851089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16300ab9d0 con 0x7f163415da80 2024-04-25T09:04:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f163b794640 1 -- 172.21.15.37:0/1895851089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f163415da80 msgr2=0x7f16340d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f163b794640 1 --2- 172.21.15.37:0/1895851089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f163415da80 0x7f16340d4230 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f163406de60 tx=0x7f163001c520 comp rx=0 tx=0).stop 2024-04-25T09:04:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f163b794640 1 -- 172.21.15.37:0/1895851089 shutdown_connections 2024-04-25T09:04:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f163b794640 1 --2- 172.21.15.37:0/1895851089 >> v2:172.21.15.203:3300/0 conn(0x7f16340d0620 0x7f16340ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f163b794640 1 --2- 172.21.15.37:0/1895851089 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16340d1cd0 0x7f16340d00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f163b794640 1 --2- 172.21.15.37:0/1895851089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f163415da80 0x7f16340d4230 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f163b794640 1 -- 172.21.15.37:0/1895851089 >> 172.21.15.37:0/1895851089 conn(0x7f163408b000 msgr2=0x7f16340d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f163b794640 1 -- 172.21.15.37:0/1895851089 shutdown_connections 2024-04-25T09:04:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.413+0000 7f163b794640 1 -- 172.21.15.37:0/1895851089 wait complete. 2024-04-25T09:04:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f163b794640 1 Processor -- start 2024-04-25T09:04:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f163b794640 1 -- start start 2024-04-25T09:04:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f163b794640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16340d0620 0x7f1634156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f163b794640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16340d1cd0 0x7f16341502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f163b794640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f163415da80 0x7f163415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f163b794640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163406ca50 con 0x7f163415da80 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f163b794640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163406c8d0 con 0x7f16340d1cd0 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f163b794640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163406cbd0 con 0x7f16340d0620 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1639d31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f163415da80 0x7f163415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1639d31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f163415da80 0x7f163415c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51334/0 (socket says 172.21.15.37:51334) 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1639d31640 1 -- 172.21.15.37:0/2918806310 learned_addr learned my addr 172.21.15.37:0/2918806310 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1638d2f640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16340d1cd0 0x7f16341502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1639530640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16340d0620 0x7f1634156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1639d31640 1 -- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16340d0620 msgr2=0x7f1634156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1639d31640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16340d0620 0x7f1634156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1639d31640 1 -- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16340d1cd0 msgr2=0x7f16341502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1639d31640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16340d1cd0 0x7f16341502b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1639d31640 1 -- 172.21.15.37:0/2918806310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1634156b40 con 0x7f163415da80 2024-04-25T09:04:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1639d31640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f163415da80 0x7f163415c6f0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f162c007e90 tx=0x7f162c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1629ffb640 1 -- 172.21.15.37:0/2918806310 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c014420 con 0x7f163415da80 2024-04-25T09:04:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1629ffb640 1 -- 172.21.15.37:0/2918806310 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c019070 con 0x7f163415da80 2024-04-25T09:04:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f163b794640 1 -- 172.21.15.37:0/2918806310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1634156de0 con 0x7f163415da80 2024-04-25T09:04:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f1629ffb640 1 -- 172.21.15.37:0/2918806310 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c014820 con 0x7f163415da80 2024-04-25T09:04:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.417+0000 7f163b794640 1 -- 172.21.15.37:0/2918806310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163415e060 con 0x7f163415da80 2024-04-25T09:04:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1629ffb640 1 -- 172.21.15.37:0/2918806310 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f162c0149c0 con 0x7f163415da80 2024-04-25T09:04:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1629ffb640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1618066f60 0x7f1618069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1629ffb640 1 -- 172.21.15.37:0/2918806310 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f162c085bd0 con 0x7f163415da80 2024-04-25T09:04:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1639530640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1618066f60 0x7f1618069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f163e901000 1 Processor -- start 2024-04-25T09:04:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f163e901000 1 -- start start 2024-04-25T09:04:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1639530640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1618066f60 0x7f1618069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f16300ad000 tx=0x7f1630008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f163e901000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4e9920 0x5638ed4e9d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f163e901000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed49f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f163e901000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed49ff30 0x5638ed4a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f163e901000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ed3875f0 con 0x5638ed4ea240 2024-04-25T09:04:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f163e901000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ed2eeda0 con 0x5638ed49ff30 2024-04-25T09:04:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f163e901000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638ed277150 con 0x5638ed4e9920 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1638d2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed49f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1638d2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed49f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51340/0 (socket says 172.21.15.37:51340) 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1638d2f640 1 -- 172.21.15.37:0/1615458903 learned_addr learned my addr 172.21.15.37:0/1615458903 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1639530640 1 --2- 172.21.15.37:0/1615458903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4e9920 0x5638ed4e9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1639d31640 1 --2- 172.21.15.37:0/1615458903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed49ff30 0x5638ed4a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1638d2f640 1 -- 172.21.15.37:0/1615458903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4e9920 msgr2=0x5638ed4e9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1638d2f640 1 --2- 172.21.15.37:0/1615458903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4e9920 0x5638ed4e9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1638d2f640 1 -- 172.21.15.37:0/1615458903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed49ff30 msgr2=0x5638ed4a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1638d2f640 1 --2- 172.21.15.37:0/1615458903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed49ff30 0x5638ed4a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1638d2f640 1 -- 172.21.15.37:0/1615458903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638ed290ca0 con 0x5638ed4ea240 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f1638d2f640 1 --2- 172.21.15.37:0/1615458903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed49f9f0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f1624006fd0 tx=0x7f162400bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a82b7506db914 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f161effd640 1 -- 172.21.15.37:0/1615458903 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624004230 con 0x5638ed4ea240 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f161effd640 1 -- 172.21.15.37:0/1615458903 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16240043d0 con 0x5638ed4ea240 2024-04-25T09:04:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.421+0000 7f161effd640 1 -- 172.21.15.37:0/1615458903 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162400f9e0 con 0x5638ed4ea240 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- 172.21.15.37:0/1615458903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 msgr2=0x5638ed49f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 --2- 172.21.15.37:0/1615458903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed49f9f0 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7f1624006fd0 tx=0x7f162400bd10 comp rx=0 tx=0).stop 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- 172.21.15.37:0/1615458903 shutdown_connections 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 --2- 172.21.15.37:0/1615458903 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed49ff30 0x5638ed4a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 --2- 172.21.15.37:0/1615458903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed49f9f0 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 --2- 172.21.15.37:0/1615458903 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4e9920 0x5638ed4e9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- 172.21.15.37:0/1615458903 >> 172.21.15.37:0/1615458903 conn(0x5638ed4a62a0 msgr2=0x5638ed49e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- 172.21.15.37:0/1615458903 shutdown_connections 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- 172.21.15.37:0/1615458903 wait complete. 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- start start 2024-04-25T09:04:19.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 Processor -- start 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed49ff30 0x5638ed4f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed4e9920 0x5638ed4f1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed4a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ed2a9990 con 0x5638ed4ea240 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ed2eae40 con 0x5638ed4e9920 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638ed27a900 con 0x5638ed49ff30 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 10 client.0 _ll_register_callbacks cb 0x5638ed3a21c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639d31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed4a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639d31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed4a3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51352/0 (socket says 172.21.15.37:51352) 2024-04-25T09:04:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639d31640 1 -- 172.21.15.37:0/1238091068 learned_addr learned my addr 172.21.15.37:0/1238091068 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639530640 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed49ff30 0x5638ed4f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1638d2f640 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed4e9920 0x5638ed4f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639d31640 1 -- 172.21.15.37:0/1238091068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed49ff30 msgr2=0x5638ed4f0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639d31640 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed49ff30 0x5638ed4f0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639d31640 1 -- 172.21.15.37:0/1238091068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed4e9920 msgr2=0x5638ed4f1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639d31640 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed4e9920 0x5638ed4f1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639d31640 1 -- 172.21.15.37:0/1238091068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638ed2a7630 con 0x5638ed4ea240 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639530640 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed49ff30 0x5638ed4f0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639d31640 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed4a3aa0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f162c0561a0 tx=0x7f162c0562d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2a13cb404d491b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1613fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1613fff640 1 -- 172.21.15.37:0/1238091068 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c082060 con 0x5638ed4ea240 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1613fff640 1 -- 172.21.15.37:0/1238091068 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f162c083af0 con 0x5638ed4ea240 2024-04-25T09:04:19.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- 172.21.15.37:0/1238091068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16240043d0 con 0x5638ed4ea240 2024-04-25T09:04:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1613fff640 1 -- 172.21.15.37:0/1238091068 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7f162c052880 con 0x5638ed4ea240 2024-04-25T09:04:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 10 client.10080 fetch_fsmap learned FSMap version 110 2024-04-25T09:04:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- 172.21.15.37:0/1238091068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x5638ed2773a0 con 0x5638ed4ea240 2024-04-25T09:04:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1613fff640 1 -- 172.21.15.37:0/1238091068 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16240043d0 con 0x5638ed4ea240 2024-04-25T09:04:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1613fff640 1 -- 172.21.15.37:0/1238091068 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 110) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f162c0742a0 con 0x5638ed4ea240 2024-04-25T09:04:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 10 client.10080 fetch_fsmap finished waiting for FSMap version 110 2024-04-25T09:04:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 10 client.10080 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:04:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 20 client.10080 populate_metadata read hostname 'smithi037' 2024-04-25T09:04:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.37:6834/2361387213,v1:172.21.15.37:6835/2361387213] conn(0x5638ed4a5160 0x5638ed4fad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 4 client.10080 mds_command: new command op to 9990 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:04:19.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f163e901000 1 -- 172.21.15.37:0/1238091068 --> [v2:172.21.15.37:6834/2361387213,v1:172.21.15.37:6835/2361387213] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638ed28aa50 con 0x5638ed4a5160 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639530640 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.37:6834/2361387213,v1:172.21.15.37:6835/2361387213] conn(0x5638ed4a5160 0x5638ed4fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1639530640 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.37:6834/2361387213,v1:172.21.15.37:6835/2361387213] conn(0x5638ed4a5160 0x5638ed4fad50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=781ab277610c807c server_cookie=b57715b1e48922c8 in_seq=0 out_seq=0 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.425+0000 7f1613fff640 10 client.10080 ms_handle_connect on v2:172.21.15.37:6834/2361387213 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f1613fff640 1 -- 172.21.15.37:0/1238091068 <== mds.0 v2:172.21.15.37:6834/2361387213 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5638ed28aa50 con 0x5638ed4a5160 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f1613fff640 10 client.10080 handle_command_reply: tid=0 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 1 client.10080 shutdown 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 1 -- 172.21.15.37:0/1238091068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 msgr2=0x5638ed4a3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed4a3aa0 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 crypto rx=0x7f162c0561a0 tx=0x7f162c0562d0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 1 -- 172.21.15.37:0/1238091068 shutdown_connections 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.37:6834/2361387213,v1:172.21.15.37:6835/2361387213] conn(0x5638ed4a5160 0x5638ed4fad50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4ea240 0x5638ed4a3aa0 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed4e9920 0x5638ed4f1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 1 --2- 172.21.15.37:0/1238091068 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed49ff30 0x5638ed4f0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f1613fff640 0 client.10080 ms_handle_reset on v2:172.21.15.37:6834/2361387213 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 1 -- 172.21.15.37:0/1238091068 >> 172.21.15.37:0/1238091068 conn(0x5638ed4a62a0 msgr2=0x5638ed49e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 1 -- 172.21.15.37:0/1238091068 shutdown_connections 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 1 -- 172.21.15.37:0/1238091068 wait complete. 2024-04-25T09:04:19.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.429+0000 7f163e901000 20 client.10080 trim_cache size 0 max 16384 2024-04-25T09:04:19.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 Processor -- start 2024-04-25T09:04:19.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 -- start start 2024-04-25T09:04:19.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed49ff30 0x5638ed545410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4e9920 0x5638ed545950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4ea240 0x5638ed545e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16342035e0 con 0x5638ed4e9920 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16342027c0 con 0x5638ed49ff30 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16342021c0 con 0x5638ed4ea240 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f1638d2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4e9920 0x5638ed545950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f1638d2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4e9920 0x5638ed545950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51362/0 (socket says 172.21.15.37:51362) 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f1638d2f640 1 -- 172.21.15.37:0/2086352501 learned_addr learned my addr 172.21.15.37:0/2086352501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f1639530640 1 --2- 172.21.15.37:0/2086352501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed49ff30 0x5638ed545410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f1638d2f640 1 -- 172.21.15.37:0/2086352501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4ea240 msgr2=0x5638ed545e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f1639d31640 1 --2- 172.21.15.37:0/2086352501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4ea240 0x5638ed545e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f1638d2f640 1 --2- 172.21.15.37:0/2086352501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4ea240 0x5638ed545e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f1638d2f640 1 -- 172.21.15.37:0/2086352501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed49ff30 msgr2=0x5638ed545410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f1638d2f640 1 --2- 172.21.15.37:0/2086352501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed49ff30 0x5638ed545410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f1638d2f640 1 -- 172.21.15.37:0/2086352501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638ed4ea070 con 0x5638ed4e9920 2024-04-25T09:04:19.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f1638d2f640 1 --2- 172.21.15.37:0/2086352501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4e9920 0x5638ed545950 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x5638ed292200 tx=0x7f1624009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19dc2e52a55a392c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f161e7fc640 1 -- 172.21.15.37:0/2086352501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16240059a0 con 0x5638ed4e9920 2024-04-25T09:04:19.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f161e7fc640 1 -- 172.21.15.37:0/2086352501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624024020 con 0x5638ed4e9920 2024-04-25T09:04:19.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f161e7fc640 1 -- 172.21.15.37:0/2086352501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1624005da0 con 0x5638ed4e9920 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 -- 172.21.15.37:0/2086352501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4e9920 msgr2=0x5638ed545950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 --2- 172.21.15.37:0/2086352501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4e9920 0x5638ed545950 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x5638ed292200 tx=0x7f1624009000 comp rx=0 tx=0).stop 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 -- 172.21.15.37:0/2086352501 shutdown_connections 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 --2- 172.21.15.37:0/2086352501 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4ea240 0x5638ed545e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 --2- 172.21.15.37:0/2086352501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed4e9920 0x5638ed545950 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 --2- 172.21.15.37:0/2086352501 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed49ff30 0x5638ed545410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 -- 172.21.15.37:0/2086352501 >> 172.21.15.37:0/2086352501 conn(0x5638ed4a62a0 msgr2=0x5638ed4a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 -- 172.21.15.37:0/2086352501 shutdown_connections 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 -- 172.21.15.37:0/2086352501 wait complete. 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 -- start start 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.445+0000 7f163e901000 1 Processor -- start 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed49ff30 0x5638ed4f6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4e9920 0x5638ed54acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed4ea240 0x5638ed54b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ed2f33d0 con 0x5638ed49ff30 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ed3aad50 con 0x5638ed4ea240 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x5638ed289980 con 0x5638ed4e9920 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 10 client.0 _ll_register_callbacks cb 0x7f162400b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1639530640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed49ff30 0x5638ed4f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1639530640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed49ff30 0x5638ed4f6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51372/0 (socket says 172.21.15.37:51372) 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1639530640 1 -- 172.21.15.37:0/2473995392 learned_addr learned my addr 172.21.15.37:0/2473995392 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1639530640 1 -- 172.21.15.37:0/2473995392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4e9920 msgr2=0x5638ed54acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1639530640 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4e9920 0x5638ed54acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1639d31640 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed4ea240 0x5638ed54b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1639530640 1 -- 172.21.15.37:0/2473995392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed4ea240 msgr2=0x5638ed54b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1639530640 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed4ea240 0x5638ed54b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1639530640 1 -- 172.21.15.37:0/2473995392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638ed370600 con 0x5638ed49ff30 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1639530640 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed49ff30 0x5638ed4f6190 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f1630008010 tx=0x7f163001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e38195c91955a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1613fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1613fff640 1 -- 172.21.15.37:0/2473995392 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16300b0040 con 0x5638ed49ff30 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1613fff640 1 -- 172.21.15.37:0/2473995392 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f16300c7150 con 0x5638ed49ff30 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 -- 172.21.15.37:0/2473995392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1624024020 con 0x5638ed49ff30 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1613fff640 1 -- 172.21.15.37:0/2473995392 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=110) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16300d5050 con 0x5638ed49ff30 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 10 client.10086 fetch_fsmap learned FSMap version 110 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 -- 172.21.15.37:0/2473995392 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=110,monmap=2+}) v3 -- 0x5638ed2839b0 con 0x5638ed49ff30 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1613fff640 1 -- 172.21.15.37:0/2473995392 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1624024020 con 0x5638ed49ff30 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1613fff640 1 -- 172.21.15.37:0/2473995392 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 110) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f16300d8020 con 0x5638ed49ff30 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 10 client.10086 fetch_fsmap finished waiting for FSMap version 110 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 10 client.10086 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 20 client.10086 populate_metadata read hostname 'smithi037' 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.37:6834/2361387213,v1:172.21.15.37:6835/2361387213] conn(0x5638ed5047d0 0x5638ed504bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 4 client.10086 mds_command: new command op to 9990 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 -- 172.21.15.37:0/2473995392 --> [v2:172.21.15.37:6834/2361387213,v1:172.21.15.37:6835/2361387213] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5638ed2fc2e0 con 0x5638ed5047d0 2024-04-25T09:04:19.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1638d2f640 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.37:6834/2361387213,v1:172.21.15.37:6835/2361387213] conn(0x5638ed5047d0 0x5638ed504bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1613fff640 10 client.10086 ms_handle_connect on v2:172.21.15.37:6834/2361387213 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1638d2f640 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.37:6834/2361387213,v1:172.21.15.37:6835/2361387213] conn(0x5638ed5047d0 0x5638ed504bb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbfe1d35206a441d server_cookie=e2bf64d4fbefdf4b in_seq=0 out_seq=0 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1613fff640 1 -- 172.21.15.37:0/2473995392 <== mds.0 v2:172.21.15.37:6834/2361387213 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5638ed2fc2e0 con 0x5638ed5047d0 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1613fff640 10 client.10086 handle_command_reply: tid=0 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 client.10086 shutdown 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 -- 172.21.15.37:0/2473995392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed49ff30 msgr2=0x5638ed4f6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed49ff30 0x5638ed4f6190 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f1630008010 tx=0x7f163001ad60 comp rx=0 tx=0).stop 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 -- 172.21.15.37:0/2473995392 shutdown_connections 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x5638ed4ea240 0x5638ed54b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x5638ed4e9920 0x5638ed54acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.37:6834/2361387213,v1:172.21.15.37:6835/2361387213] conn(0x5638ed5047d0 0x5638ed504bb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 --2- 172.21.15.37:0/2473995392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5638ed49ff30 0x5638ed4f6190 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f1613fff640 0 client.10086 ms_handle_reset on v2:172.21.15.37:6834/2361387213 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 -- 172.21.15.37:0/2473995392 >> 172.21.15.37:0/2473995392 conn(0x5638ed4a62a0 msgr2=0x5638ed4f18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 -- 172.21.15.37:0/2473995392 shutdown_connections 2024-04-25T09:04:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 1 -- 172.21.15.37:0/2473995392 wait complete. 2024-04-25T09:04:19.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163e901000 20 client.10086 trim_cache size 0 max 16384 2024-04-25T09:04:19.444 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:19.444 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "3ecc7551-d5a4-47e7-a9cf-6943742cae5b", 2024-04-25T09:04:19.444 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-25T09:04:19.444 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 9990, 2024-04-25T09:04:19.444 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-25T09:04:19.444 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-25T09:04:19.444 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-25T09:04:19.444 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 4.4907270099999996, 2024-04-25T09:04:19.444 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 110, 2024-04-25T09:04:19.445 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 127, 2024-04-25T09:04:19.445 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 127, 2024-04-25T09:04:19.445 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 6.4667985870000004 2024-04-25T09:04:19.445 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:19.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 -- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1618066f60 msgr2=0x7f1618069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1618066f60 0x7f1618069420 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f16300ad000 tx=0x7f1630008040 comp rx=0 tx=0).stop 2024-04-25T09:04:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 -- 172.21.15.37:0/2918806310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f163415da80 msgr2=0x7f163415c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f163415da80 0x7f163415c6f0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f162c007e90 tx=0x7f162c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 -- 172.21.15.37:0/2918806310 shutdown_connections 2024-04-25T09:04:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1618066f60 0x7f1618069420 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f163415da80 0x7f163415c6f0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f16340d1cd0 0x7f16341502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 --2- 172.21.15.37:0/2918806310 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f16340d0620 0x7f1634156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 -- 172.21.15.37:0/2918806310 >> 172.21.15.37:0/2918806310 conn(0x7f163408b000 msgr2=0x7f16340d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 -- 172.21.15.37:0/2918806310 shutdown_connections 2024-04-25T09:04:19.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.449+0000 7f163b794640 1 -- 172.21.15.37:0/2918806310 wait complete. 2024-04-25T09:04:19.456 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:04:19.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.581+0000 7f529f26b640 1 Processor -- start 2024-04-25T09:04:19.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.581+0000 7f529f26b640 1 -- start start 2024-04-25T09:04:19.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529f26b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980b2ac0 0x7f52980cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529f26b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52980ce070 con 0x7f529815e1e0 2024-04-25T09:04:19.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529f26b640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52980ce350 con 0x7f529815a600 2024-04-25T09:04:19.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529f26b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52980ce630 con 0x7f52980b2ac0 2024-04-25T09:04:19.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529d808640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f529815e1e0 0x7f52980b3250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51948/0 (socket says 172.21.15.37:51948) 2024-04-25T09:04:19.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529d808640 1 -- 172.21.15.37:0/2564067557 learned_addr learned my addr 172.21.15.37:0/2564067557 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529c806640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980b2ac0 0x7f52980cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1515814485 0 0) 0x7f52980ce070 con 0x7f529815e1e0 2024-04-25T09:04:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5284003530 con 0x7f529815e1e0 2024-04-25T09:04:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2069227821 0 0) 0x7f5284003530 con 0x7f529815e1e0 2024-04-25T09:04:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980b2ac0 msgr2=0x7f52980cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980b2ac0 0x7f52980cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 >> v1:172.21.15.203:6789/0 conn(0x7f529815a600 legacy=0x7f52980cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52980ce910 con 0x7f529815e1e0 2024-04-25T09:04:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f5290001a00 con 0x7f529815e1e0 2024-04-25T09:04:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 >> v1:172.21.15.37:6789/0 conn(0x7f529815e1e0 legacy=0x7f52980b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980b2ac0 0x7f5284003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5284003fe0 0x7f5284004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5284004d50 0x7f52840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52980ce630 con 0x7f5284004d50 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528400ddd0 con 0x7f5284003fe0 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f528400dfd0 con 0x7f52980b2ac0 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529d808640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5284004d50 0x7f52840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529c806640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5284003fe0 0x7f5284004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529d007640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980b2ac0 0x7f5284003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529d808640 1 -- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980b2ac0 msgr2=0x7f5284003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529d808640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980b2ac0 0x7f5284003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529d808640 1 -- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5284003fe0 msgr2=0x7f5284004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529d808640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5284003fe0 0x7f5284004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529d808640 1 -- 172.21.15.37:0/2564067557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5284003530 con 0x7f5284004d50 2024-04-25T09:04:19.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.585+0000 7f529c806640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5284003fe0 0x7f5284004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:19.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529d808640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5284004d50 0x7f52840051d0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f529000a860 tx=0x7f529001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3edee8783d9b38b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529000a480 con 0x7f5284004d50 2024-04-25T09:04:19.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52900ad630 con 0x7f5284004d50 2024-04-25T09:04:19.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f528ffff640 1 -- 172.21.15.37:0/2564067557 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52900adaa0 con 0x7f5284004d50 2024-04-25T09:04:19.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 -- 172.21.15.37:0/2564067557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5284004d50 msgr2=0x7f52840051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 --2- 172.21.15.37:0/2564067557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5284004d50 0x7f52840051d0 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f529000a860 tx=0x7f529001d380 comp rx=0 tx=0).stop 2024-04-25T09:04:19.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529d808640 1 -- 172.21.15.37:0/2564067557 reap_dead start 2024-04-25T09:04:19.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 -- 172.21.15.37:0/2564067557 shutdown_connections 2024-04-25T09:04:19.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 -- 172.21.15.37:0/2564067557 >> 172.21.15.37:0/2564067557 conn(0x7f52980b1f10 msgr2=0x7f52980b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 -- 172.21.15.37:0/2564067557 shutdown_connections 2024-04-25T09:04:19.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 -- 172.21.15.37:0/2564067557 wait complete. 2024-04-25T09:04:19.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 Processor -- start 2024-04-25T09:04:19.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 -- start start 2024-04-25T09:04:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f529809dd10 0x7f52980c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529809e0f0 0x7f52980c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980c6750 0x7f52980c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529815ab60 con 0x7f529809e0f0 2024-04-25T09:04:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529815a9e0 con 0x7f529809dd10 2024-04-25T09:04:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529f26b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529815e650 con 0x7f52980c6750 2024-04-25T09:04:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.589+0000 7f529c806640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529809e0f0 0x7f52980c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529c806640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529809e0f0 0x7f52980c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51392/0 (socket says 172.21.15.37:51392) 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529c806640 1 -- 172.21.15.37:0/2631007160 learned_addr learned my addr 172.21.15.37:0/2631007160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529d007640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f529809dd10 0x7f52980c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529c806640 1 -- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980c6750 msgr2=0x7f52980c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529c806640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980c6750 0x7f52980c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529c806640 1 -- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f529809dd10 msgr2=0x7f52980c47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529c806640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f529809dd10 0x7f52980c47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529c806640 1 -- 172.21.15.37:0/2631007160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52980b57a0 con 0x7f529809e0f0 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529d007640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f529809dd10 0x7f52980c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529c806640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529809e0f0 0x7f52980c6210 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f52880115f0 tx=0x7f5288011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f528dffb640 1 -- 172.21.15.37:0/2631007160 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288015870 con 0x7f529809e0f0 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f528dffb640 1 -- 172.21.15.37:0/2631007160 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288015a10 con 0x7f529809e0f0 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f528dffb640 1 -- 172.21.15.37:0/2631007160 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288015dc0 con 0x7f529809e0f0 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529f26b640 1 -- 172.21.15.37:0/2631007160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52980b5a80 con 0x7f529809e0f0 2024-04-25T09:04:19.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529f26b640 1 -- 172.21.15.37:0/2631007160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52980c87c0 con 0x7f529809e0f0 2024-04-25T09:04:19.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529f26b640 1 -- 172.21.15.37:0/2631007160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529804efa0 con 0x7f529809e0f0 2024-04-25T09:04:19.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f528dffb640 1 -- 172.21.15.37:0/2631007160 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f528800f460 con 0x7f529809e0f0 2024-04-25T09:04:19.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f528dffb640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f527c066f90 0x7f527c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:19.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f528dffb640 1 -- 172.21.15.37:0/2631007160 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f5288085ba0 con 0x7f529809e0f0 2024-04-25T09:04:19.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.593+0000 7f529d007640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f527c066f90 0x7f527c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:19.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.597+0000 7f529d007640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f527c066f90 0x7f527c069450 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5294001640 tx=0x7f5294009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:19.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.597+0000 7f528dffb640 1 -- 172.21.15.37:0/2631007160 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f5288019030 con 0x7f529809e0f0 2024-04-25T09:04:19.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.789+0000 7f529f26b640 1 -- 172.21.15.37:0/2631007160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f52980b6380 con 0x7f529809e0f0 2024-04-25T09:04:19.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.789+0000 7f528dffb640 1 -- 172.21.15.37:0/2631007160 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v110) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5288056970 con 0x7f529809e0f0 2024-04-25T09:04:19.778 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:04:19.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 -- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f527c066f90 msgr2=0x7f527c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f527c066f90 0x7f527c069450 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5294001640 tx=0x7f5294009040 comp rx=0 tx=0).stop 2024-04-25T09:04:19.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 -- 172.21.15.37:0/2631007160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529809e0f0 msgr2=0x7f52980c6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:19.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529809e0f0 0x7f52980c6210 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f52880115f0 tx=0x7f5288011ab0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 -- 172.21.15.37:0/2631007160 shutdown_connections 2024-04-25T09:04:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f527c066f90 0x7f527c069450 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f52980c6750 0x7f52980c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f529809e0f0 0x7f52980c6210 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 --2- 172.21.15.37:0/2631007160 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f529809dd10 0x7f52980c47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 -- 172.21.15.37:0/2631007160 >> 172.21.15.37:0/2631007160 conn(0x7f52980b1f10 msgr2=0x7f529806eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 -- 172.21.15.37:0/2631007160 shutdown_connections 2024-04-25T09:04:19.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:19.793+0000 7f529f26b640 1 -- 172.21.15.37:0/2631007160 wait complete. 2024-04-25T09:04:19.798 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:04:19.798 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:04:19.798 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-25T09:04:19.798 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:04:19.798 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:19.798 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:04:19.798 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:04:19.798 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:04:19.798 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:19.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.799 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-25T09:04:19.805 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:04:19.805 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:04:19.805 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:04:19.806 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.806 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-25T09:04:19.806 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.806 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:19.806 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.806 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-25T09:04:19.806 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:19.806 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.806 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-25T09:04:19.806 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:04:19.806 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.807 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-25T09:04:19.807 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T09:04:19.807 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41781sec preferred_lft 41781sec 2024-04-25T09:04:19.807 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-04-25T09:04:19.807 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.807 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T09:04:19.807 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.807 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-25T09:04:19.807 INFO:teuthology.orchestra.run.smithi037.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T09:04:19.807 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 02:42:8e:42:38:10 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.807 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T09:04:19.808 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.808 INFO:teuthology.orchestra.run.smithi037.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:04:19.808 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:19.808 INFO:teuthology.orchestra.run.smithi037.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:04:19.808 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.808 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T09:04:19.808 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.808 INFO:teuthology.orchestra.run.smithi037.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:19.808 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:19.808 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T09:04:19.808 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.809 INFO:teuthology.orchestra.run.smithi037.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:19.809 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:19.809 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T09:04:19.809 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:19.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.809 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:04:19.857 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T09:04:19.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.859 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-25T09:04:19.910 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-25T09:04:19.911 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:04:19.911 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-04-25T09:04:19.911 INFO:teuthology.orchestra.run.smithi037.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T09:04:19.911 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-04-25T09:04:19.911 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:04:19.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.911 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec bash -c ' 2024-04-25T09:04:19.911 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-25T09:04:19.911 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:04:19.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:04:19.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:04:19.912 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-25T09:04:19.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:19.982 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T09:04:20.029 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:04:20.029 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:04:20.029 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:20.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:20.029 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-25T09:04:20.086 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:04:20.086 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:20.086 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:20.134 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:20.134 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:20.134 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:20.181 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:04:20.239 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:20.290 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:04:20.290 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:04:20.290 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:04:20.290 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:04:20.290 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:04:20.290 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:04:20.291 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:04:20.291 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:04:20.291 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:04:20.291 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:04:20.291 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:04:20.291 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:04:20.291 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:20.291 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:04:20.291 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:04:20.291 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:04:20.291 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:04:20.292 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:20.293 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:20.293 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:20.293 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:20.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:20.294 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:04:20.342 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:04:20.342 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:04:20.386 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:04:20.401 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:20.449 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:04:20.450 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:04:20.450 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:04:20.450 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:04:20.450 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:04:20.450 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:04:20.450 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:04:20.450 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:04:20.451 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:04:20.451 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:04:20.451 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:04:20.451 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:04:20.451 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:20.451 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:04:20.451 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:04:20.451 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:04:20.451 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:04:20.451 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:04:20.452 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:20.452 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:04:20.453 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:20.453 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:04:20.453 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:04:20.453 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:04:20.453 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:20.453 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:20.454 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:20.454 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:20.454 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:04:20.454 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:20.454 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:20.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:20.455 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:04:20.464 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:2024-04-25T09:04:20.473+0000 7f8c2ed67580 -1 init, newargv = 0x55f634e694f0 newargc=15 2024-04-25T09:04:20.464 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[56351]: starting ceph client 2024-04-25T09:04:20.469 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[56351]: starting fuse 2024-04-25T09:04:21.569 DEBUG:teuthology.orchestra.run.smithi037:> sudo modprobe fuse 2024-04-25T09:04:21.586 DEBUG:teuthology.orchestra.run.smithi037:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:21.635 INFO:teuthology.orchestra.run.smithi037.stdout:/sys 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/proc 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/dev 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/pts 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/run 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/ 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/security 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/shm 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/run/lock 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/cgroup 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/pstore 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/bpf 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/hugepages 2024-04-25T09:04:21.636 INFO:teuthology.orchestra.run.smithi037.stdout:/dev/mqueue 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/debug 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/tracing 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/proc/fs/nfsd 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/sys/kernel/config 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/run/rpc_pipefs 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/run/user/1000 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:21.637 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:21.638 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:21.638 INFO:teuthology.orchestra.run.smithi037.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:21.638 INFO:teuthology.orchestra.run.smithi037.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:21.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:21.639 DEBUG:teuthology.orchestra.run.smithi037:> ls /sys/fs/fuse/connections 2024-04-25T09:04:21.685 INFO:teuthology.orchestra.run.smithi037.stdout:45 2024-04-25T09:04:21.686 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> import glob 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> import re 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> import subprocess 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> def _find_admin_socket(client_name): 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> files = glob.glob(asok_path) 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:21.686 DEBUG:teuthology.orchestra.run.smithi037:> # Given a non-glob path, it better be there 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> if "*" not in asok_path: 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> assert(len(files) == 1) 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> return files[0] 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> for f in files: 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> contents = proc_f.read() 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> if mountpoint in contents: 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> return f 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> print(_find_admin_socket("client.0")) 2024-04-25T09:04:21.687 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:21.814 INFO:teuthology.orchestra.run.smithi037.stdout:/var/run/ceph/ceph-client.0.56351.asok 2024-04-25T09:04:21.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:21.824 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.56351.asok 2024-04-25T09:04:21.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:21.824 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.56351.asok status 2024-04-25T09:04:21.985 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:21.985 INFO:teuthology.orchestra.run.smithi037.stdout: "metadata": { 2024-04-25T09:04:21.985 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:04:21.985 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:04:21.985 INFO:teuthology.orchestra.run.smithi037.stdout: "entity_id": "0", 2024-04-25T09:04:21.985 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": "smithi037", 2024-04-25T09:04:21.985 INFO:teuthology.orchestra.run.smithi037.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:04:21.985 INFO:teuthology.orchestra.run.smithi037.stdout: "pid": "56351", 2024-04-25T09:04:21.985 INFO:teuthology.orchestra.run.smithi037.stdout: "root": "/" 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_count": 0, 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: "dentry_pinned_count": 0, 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 10098, 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: "inst": { 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: "name": { 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "client", 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: "num": 10098 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2734354549 2024-04-25T09:04:21.986 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": { 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2734354549 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "inst_str": "client.10098 192.168.0.1:0/2734354549", 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "addr_str": "192.168.0.1:0/2734354549", 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "inode_count": 1, 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_epoch": 110, 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch": 127, 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "blocklisted": false, 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs" 2024-04-25T09:04:21.987 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:21.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:21.999 DEBUG:teuthology.orchestra.run.smithi037:> stat --file-system '--printf=%T 2024-04-25T09:04:21.999 DEBUG:teuthology.orchestra.run.smithi037:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:22.005 INFO:teuthology.orchestra.run.smithi037.stdout:fuseblk 2024-04-25T09:04:22.005 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:22.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:22.006 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:22.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:22.061 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-04-25T09:04:22.126 INFO:teuthology.orchestra.run.smithi037.stdout:./ 2024-04-25T09:04:22.126 INFO:teuthology.orchestra.run.smithi037.stdout:./parent 2024-04-25T09:04:22.126 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/flushed 2024-04-25T09:04:22.126 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/flushed/charlie 2024-04-25T09:04:22.126 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/flushed/alpha 2024-04-25T09:04:22.127 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/flushed/bravo 2024-04-25T09:04:22.127 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/unflushed 2024-04-25T09:04:22.127 INFO:teuthology.orchestra.run.smithi037.stdout:./parent/unflushed/jfile 2024-04-25T09:04:22.128 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:22.128 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:22.128 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:22.128 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:22.128 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-04-25T09:04:22.128 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:22.241 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-25T09:04:22.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:22.252 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:22.253 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:22.253 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:22.253 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:22.253 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-04-25T09:04:22.253 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:22.364 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627776 2024-04-25T09:04:22.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:22.374 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:22.375 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:22.375 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:22.375 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:22.375 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-04-25T09:04:22.375 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:22.484 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627777 2024-04-25T09:04:22.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:22.496 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:22.496 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:22.496 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:22.496 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:22.496 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-04-25T09:04:22.496 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:22.605 INFO:teuthology.orchestra.run.smithi037.stdout:1099511628280 2024-04-25T09:04:22.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:22.616 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:22.616 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:22.617 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:22.617 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:22.617 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-04-25T09:04:22.617 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:22.723 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627778 2024-04-25T09:04:22.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:22.734 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:22.734 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:22.734 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:22.734 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:22.734 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-04-25T09:04:22.734 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:22.844 INFO:teuthology.orchestra.run.smithi037.stdout:1099511628279 2024-04-25T09:04:22.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:22.854 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:22.854 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:22.854 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:22.855 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:22.855 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-04-25T09:04:22.855 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:22.962 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627779 2024-04-25T09:04:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:22.973 DEBUG:teuthology.orchestra.run.smithi037:> stdin-killer -- python3 -c ' 2024-04-25T09:04:22.973 DEBUG:teuthology.orchestra.run.smithi037:> import os 2024-04-25T09:04:22.973 DEBUG:teuthology.orchestra.run.smithi037:> import stat 2024-04-25T09:04:22.973 DEBUG:teuthology.orchestra.run.smithi037:> 2024-04-25T09:04:22.973 DEBUG:teuthology.orchestra.run.smithi037:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-04-25T09:04:22.973 DEBUG:teuthology.orchestra.run.smithi037:> ' 2024-04-25T09:04:23.088 INFO:teuthology.orchestra.run.smithi037.stdout:1099511627780 2024-04-25T09:04:23.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:23.098 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:04:23.098 DEBUG:teuthology.orchestra.run.smithi037:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:04:23.182 DEBUG:teuthology.orchestra.run.smithi203:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:04:23.234 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:04:23.234 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:04:23.234 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:23.241 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:23.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.429+0000 7ff01e906640 1 Processor -- start 2024-04-25T09:04:23.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.429+0000 7ff01e906640 1 -- start start 2024-04-25T09:04:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.429+0000 7ff01e906640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff01815da60 0x7ff01806bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.429+0000 7ff01e906640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff01806c1e0 0x7ff0180cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.429+0000 7ff01e906640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01806c5f0 con 0x7ff018143810 2024-04-25T09:04:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.429+0000 7ff01e906640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0180b32d0 con 0x7ff01815da60 2024-04-25T09:04:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.429+0000 7ff01e906640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0180b3530 con 0x7ff01806c1e0 2024-04-25T09:04:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.429+0000 7ff0177fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff01815da60 0x7ff01806bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.429+0000 7ff0177fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff01815da60 0x7ff01806bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51402/0 (socket says 172.21.15.37:51402) 2024-04-25T09:04:23.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.429+0000 7ff0177fe640 1 -- 172.21.15.37:0/3225095770 learned_addr learned my addr 172.21.15.37:0/3225095770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:23.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.429+0000 7ff01cea3640 1 --2- 172.21.15.37:0/3225095770 >> v2:172.21.15.203:3300/0 conn(0x7ff01806c1e0 0x7ff0180cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff0177fe640 1 -- 172.21.15.37:0/3225095770 >> v2:172.21.15.203:3300/0 conn(0x7ff01806c1e0 msgr2=0x7ff0180cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff0177fe640 1 --2- 172.21.15.37:0/3225095770 >> v2:172.21.15.203:3300/0 conn(0x7ff01806c1e0 0x7ff0180cd050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff0177fe640 1 -- 172.21.15.37:0/3225095770 >> v1:172.21.15.203:6789/0 conn(0x7ff018143810 legacy=0x7ff0180b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff0177fe640 1 -- 172.21.15.37:0/3225095770 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0180b3810 con 0x7ff01815da60 2024-04-25T09:04:23.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff01cea3640 1 --2- 172.21.15.37:0/3225095770 >> v2:172.21.15.203:3300/0 conn(0x7ff01806c1e0 0x7ff0180cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:23.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff0177fe640 1 --2- 172.21.15.37:0/3225095770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff01815da60 0x7ff01806bca0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7ff01815e9a0 tx=0x7ff00801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6785049efdd5bae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff016ffd640 1 -- 172.21.15.37:0/3225095770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080b0070 con 0x7ff01815da60 2024-04-25T09:04:23.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff016ffd640 1 -- 172.21.15.37:0/3225095770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0080ab600 con 0x7ff01815da60 2024-04-25T09:04:23.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff016ffd640 1 -- 172.21.15.37:0/3225095770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080ab9d0 con 0x7ff01815da60 2024-04-25T09:04:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff01e906640 1 -- 172.21.15.37:0/3225095770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff01815da60 msgr2=0x7ff01806bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff01e906640 1 --2- 172.21.15.37:0/3225095770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff01815da60 0x7ff01806bca0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7ff01815e9a0 tx=0x7ff00801c520 comp rx=0 tx=0).stop 2024-04-25T09:04:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff01e906640 1 -- 172.21.15.37:0/3225095770 shutdown_connections 2024-04-25T09:04:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff01e906640 1 --2- 172.21.15.37:0/3225095770 >> v2:172.21.15.203:3300/0 conn(0x7ff01806c1e0 0x7ff0180cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff01e906640 1 --2- 172.21.15.37:0/3225095770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff01815da60 0x7ff01806bca0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.433+0000 7ff01e906640 1 -- 172.21.15.37:0/3225095770 >> 172.21.15.37:0/3225095770 conn(0x7ff0180717f0 msgr2=0x7ff018071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 -- 172.21.15.37:0/3225095770 shutdown_connections 2024-04-25T09:04:23.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 -- 172.21.15.37:0/3225095770 wait complete. 2024-04-25T09:04:23.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 Processor -- start 2024-04-25T09:04:23.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 -- start start 2024-04-25T09:04:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff01806c1e0 0x7ff01813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff018143810 0x7ff018138520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff01815da60 0x7ff01813bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180cd710 con 0x7ff01806c1e0 2024-04-25T09:04:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180cd590 con 0x7ff01815da60 2024-04-25T09:04:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0180cd890 con 0x7ff018143810 2024-04-25T09:04:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff017fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff01806c1e0 0x7ff01813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff017fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff01806c1e0 0x7ff01813f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51408/0 (socket says 172.21.15.37:51408) 2024-04-25T09:04:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff017fff640 1 -- 172.21.15.37:0/2863508203 learned_addr learned my addr 172.21.15.37:0/2863508203 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff0177fe640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff018143810 0x7ff018138520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01cea3640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff01815da60 0x7ff01813bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff017fff640 1 -- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff018143810 msgr2=0x7ff018138520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff017fff640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff018143810 0x7ff018138520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff017fff640 1 -- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff01815da60 msgr2=0x7ff01813bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff017fff640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff01815da60 0x7ff01813bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff017fff640 1 -- 172.21.15.37:0/2863508203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01813fbf0 con 0x7ff01806c1e0 2024-04-25T09:04:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff0177fe640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff018143810 0x7ff018138520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff017fff640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff01806c1e0 0x7ff01813f6b0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7ff00000c450 tx=0x7ff00000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff014ff9640 1 -- 172.21.15.37:0/2863508203 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000016020 con 0x7ff01806c1e0 2024-04-25T09:04:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff014ff9640 1 -- 172.21.15.37:0/2863508203 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0000052d0 con 0x7ff01806c1e0 2024-04-25T09:04:23.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff014ff9640 1 -- 172.21.15.37:0/2863508203 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000007260 con 0x7ff01806c1e0 2024-04-25T09:04:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 -- 172.21.15.37:0/2863508203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff018137480 con 0x7ff01806c1e0 2024-04-25T09:04:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.437+0000 7ff01e906640 1 -- 172.21.15.37:0/2863508203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff018137a40 con 0x7ff01806c1e0 2024-04-25T09:04:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.441+0000 7ff01e906640 1 -- 172.21.15.37:0/2863508203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff01804efa0 con 0x7ff01806c1e0 2024-04-25T09:04:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.441+0000 7ff014ff9640 1 -- 172.21.15.37:0/2863508203 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff000007400 con 0x7ff01806c1e0 2024-04-25T09:04:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.441+0000 7ff014ff9640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feff8066f60 0x7feff8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.441+0000 7ff014ff9640 1 -- 172.21.15.37:0/2863508203 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff000084c50 con 0x7ff01806c1e0 2024-04-25T09:04:23.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.441+0000 7ff0177fe640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feff8066f60 0x7feff8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.441+0000 7ff0177fe640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feff8066f60 0x7feff8069420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ad000 tx=0x7ff008008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.445+0000 7ff014ff9640 1 -- 172.21.15.37:0/2863508203 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff000020020 con 0x7ff01806c1e0 2024-04-25T09:04:23.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.617+0000 7ff01e906640 1 -- 172.21.15.37:0/2863508203 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff01806ee10 con 0x7ff01806c1e0 2024-04-25T09:04:23.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.617+0000 7ff014ff9640 1 -- 172.21.15.37:0/2863508203 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17175 (secure 0 0 0) 0x7ff01806c5f0 con 0x7ff01806c1e0 2024-04-25T09:04:23.608 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:23.608 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":127,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:04:12.937853+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2159545482":"2024-04-26T09:04:12.937783+0000","172.21.15.37:6834/2159545482":"2024-04-26T09:04:12.937783+0000","172.21.15.37:6835/2748919477":"2024-04-26T09:03:58.978336+0000","172.21.15.37:6834/2748919477":"2024-04-26T09:03:58.978336+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:23.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 -- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feff8066f60 msgr2=0x7feff8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feff8066f60 0x7feff8069420 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff0080ad000 tx=0x7ff008008040 comp rx=0 tx=0).stop 2024-04-25T09:04:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 -- 172.21.15.37:0/2863508203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff01806c1e0 msgr2=0x7ff01813f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff01806c1e0 0x7ff01813f6b0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7ff00000c450 tx=0x7ff00000c910 comp rx=0 tx=0).stop 2024-04-25T09:04:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 -- 172.21.15.37:0/2863508203 shutdown_connections 2024-04-25T09:04:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7feff8066f60 0x7feff8069420 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff01815da60 0x7ff01813bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff018143810 0x7ff018138520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 --2- 172.21.15.37:0/2863508203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff01806c1e0 0x7ff01813f6b0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 -- 172.21.15.37:0/2863508203 >> 172.21.15.37:0/2863508203 conn(0x7ff0180717f0 msgr2=0x7ff0181635e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 -- 172.21.15.37:0/2863508203 shutdown_connections 2024-04-25T09:04:23.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.621+0000 7ff01e906640 1 -- 172.21.15.37:0/2863508203 wait complete. 2024-04-25T09:04:23.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:30.763724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:31.488462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:23.628 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:23.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a6b6f640 1 Processor -- start 2024-04-25T09:04:23.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a6b6f640 1 -- start start 2024-04-25T09:04:23.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a6b6f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15a00b0b00 0x7f15a00cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a6b6f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00cc650 0x7f15a00b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a6b6f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15a0160310 con 0x7f15a00b1910 2024-04-25T09:04:23.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a6b6f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15a015cfa0 con 0x7f15a00b0b00 2024-04-25T09:04:23.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a6b6f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a015cdf0 con 0x7f15a00cc650 2024-04-25T09:04:23.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a490b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f15a00b0b00 0x7f15a00cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a510c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f15a00b1910 0x7f15a0151fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51960/0 (socket says 172.21.15.37:51960) 2024-04-25T09:04:23.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a510c640 1 -- 172.21.15.37:0/1201583023 learned_addr learned my addr 172.21.15.37:0/1201583023 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:23.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f159ffff640 1 --2- 172.21.15.37:0/1201583023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00cc650 0x7f15a00b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f159f7fe640 1 -- 172.21.15.37:0/1201583023 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369652143 0 0) 0x7f15a0160310 con 0x7f15a00b1910 2024-04-25T09:04:23.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f159f7fe640 1 -- 172.21.15.37:0/1201583023 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1588003530 con 0x7f15a00b1910 2024-04-25T09:04:23.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a490b640 1 -- 172.21.15.37:0/1201583023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00cc650 msgr2=0x7f15a00b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a490b640 1 --2- 172.21.15.37:0/1201583023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00cc650 0x7f15a00b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a490b640 1 -- 172.21.15.37:0/1201583023 >> v1:172.21.15.37:6789/0 conn(0x7f15a00b1910 legacy=0x7f15a0151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.765+0000 7f15a490b640 1 -- 172.21.15.37:0/1201583023 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a015f550 con 0x7f15a00b0b00 2024-04-25T09:04:23.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a490b640 1 --2- 172.21.15.37:0/1201583023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00b0b00 0x7f15a00cc110 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f15900097c0 tx=0x7f159001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56d1c0a26e5efc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f159f7fe640 1 -- 172.21.15.37:0/1201583023 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15900b0070 con 0x7f15a00b0b00 2024-04-25T09:04:23.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f159f7fe640 1 -- 172.21.15.37:0/1201583023 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15900ab600 con 0x7f15a00b0b00 2024-04-25T09:04:23.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f159f7fe640 1 -- 172.21.15.37:0/1201583023 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15900ab9b0 con 0x7f15a00b0b00 2024-04-25T09:04:23.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 -- 172.21.15.37:0/1201583023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00b0b00 msgr2=0x7f15a00cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 --2- 172.21.15.37:0/1201583023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00b0b00 0x7f15a00cc110 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7f15900097c0 tx=0x7f159001c4a0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 -- 172.21.15.37:0/1201583023 shutdown_connections 2024-04-25T09:04:23.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 --2- 172.21.15.37:0/1201583023 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00cc650 0x7f15a00b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 --2- 172.21.15.37:0/1201583023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00b0b00 0x7f15a00cc110 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 -- 172.21.15.37:0/1201583023 >> 172.21.15.37:0/1201583023 conn(0x7f15a00b1f10 msgr2=0x7f15a00b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 -- 172.21.15.37:0/1201583023 shutdown_connections 2024-04-25T09:04:23.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 -- 172.21.15.37:0/1201583023 wait complete. 2024-04-25T09:04:23.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 Processor -- start 2024-04-25T09:04:23.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 -- start start 2024-04-25T09:04:23.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b0b00 0x7f15a0152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a00b1910 0x7f15a00c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00cc650 0x7f15a014df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00b0ee0 con 0x7f15a00cc650 2024-04-25T09:04:23.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00b1d20 con 0x7f15a00b1910 2024-04-25T09:04:23.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a6b6f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a00b1060 con 0x7f15a00b0b00 2024-04-25T09:04:23.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.769+0000 7f15a510c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00cc650 0x7f15a014df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a510c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00cc650 0x7f15a014df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51426/0 (socket says 172.21.15.37:51426) 2024-04-25T09:04:23.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a510c640 1 -- 172.21.15.37:0/4233079086 learned_addr learned my addr 172.21.15.37:0/4233079086 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:23.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f159ffff640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a00b1910 0x7f15a00c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a490b640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b0b00 0x7f15a0152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a510c640 1 -- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b0b00 msgr2=0x7f15a0152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a510c640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b0b00 0x7f15a0152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a510c640 1 -- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a00b1910 msgr2=0x7f15a00c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a510c640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a00b1910 0x7f15a00c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a510c640 1 -- 172.21.15.37:0/4233079086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a0153f10 con 0x7f15a00cc650 2024-04-25T09:04:23.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a490b640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b0b00 0x7f15a0152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:23.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a510c640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00cc650 0x7f15a014df40 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f159800cb40 tx=0x7f159800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f159d7fa640 1 -- 172.21.15.37:0/4233079086 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598017070 con 0x7f15a00cc650 2024-04-25T09:04:23.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f159d7fa640 1 -- 172.21.15.37:0/4233079086 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f159800aca0 con 0x7f15a00cc650 2024-04-25T09:04:23.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a6b6f640 1 -- 172.21.15.37:0/4233079086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a014ec60 con 0x7f15a00cc650 2024-04-25T09:04:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f159d7fa640 1 -- 172.21.15.37:0/4233079086 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1598012620 con 0x7f15a00cc650 2024-04-25T09:04:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a6b6f640 1 -- 172.21.15.37:0/4233079086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a0153950 con 0x7f15a00cc650 2024-04-25T09:04:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a6b6f640 1 -- 172.21.15.37:0/4233079086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15a004efa0 con 0x7f15a00cc650 2024-04-25T09:04:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f159d7fa640 1 -- 172.21.15.37:0/4233079086 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f159801a070 con 0x7f15a00cc650 2024-04-25T09:04:23.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f159d7fa640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f158c066f90 0x7f158c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f159d7fa640 1 -- 172.21.15.37:0/4233079086 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f1598084fe0 con 0x7f15a00cc650 2024-04-25T09:04:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a490b640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f158c066f90 0x7f158c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.773+0000 7f15a490b640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f158c066f90 0x7f158c069450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f15900ad000 tx=0x7f15900a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.777+0000 7f159d7fa640 1 -- 172.21.15.37:0/4233079086 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1598055db0 con 0x7f15a00cc650 2024-04-25T09:04:23.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.937+0000 7f15a6b6f640 1 -- 172.21.15.37:0/4233079086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f15a01540e0 con 0x7f15a00cc650 2024-04-25T09:04:23.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f159d7fa640 1 -- 172.21.15.37:0/4233079086 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f15980556b0 con 0x7f15a00cc650 2024-04-25T09:04:23.929 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:04:23.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 -- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f158c066f90 msgr2=0x7f158c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f158c066f90 0x7f158c069450 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f15900ad000 tx=0x7f15900a6000 comp rx=0 tx=0).stop 2024-04-25T09:04:23.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 -- 172.21.15.37:0/4233079086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00cc650 msgr2=0x7f15a014df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00cc650 0x7f15a014df40 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7f159800cb40 tx=0x7f159800a060 comp rx=0 tx=0).stop 2024-04-25T09:04:23.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 -- 172.21.15.37:0/4233079086 shutdown_connections 2024-04-25T09:04:23.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f158c066f90 0x7f158c069450 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15a00cc650 0x7f15a014df40 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f15a00b1910 0x7f15a00c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 --2- 172.21.15.37:0/4233079086 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f15a00b0b00 0x7f15a0152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 -- 172.21.15.37:0/4233079086 >> 172.21.15.37:0/4233079086 conn(0x7f15a00b1f10 msgr2=0x7f15a015bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 -- 172.21.15.37:0/4233079086 shutdown_connections 2024-04-25T09:04:23.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:23.941+0000 7f15a6b6f640 1 -- 172.21.15.37:0/4233079086 wait complete. 2024-04-25T09:04:23.948 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:24.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8ce6f4640 1 Processor -- start 2024-04-25T09:04:24.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8ce6f4640 1 -- start start 2024-04-25T09:04:24.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8ce6f4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80cf590 0x7ff8c80cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8ce6f4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff8c8072000 0x7ff8c8072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8ce6f4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c80cbd20 con 0x7ff8c815b6b0 2024-04-25T09:04:24.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8ce6f4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c815ba90 con 0x7ff8c8072000 2024-04-25T09:04:24.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8ce6f4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80cff30 con 0x7ff8c80cf590 2024-04-25T09:04:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ff8c815b6b0 0x7ff8c80ca3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51974/0 (socket says 172.21.15.37:51974) 2024-04-25T09:04:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c7fff640 1 -- 172.21.15.37:0/1808889989 learned_addr learned my addr 172.21.15.37:0/1808889989 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c77fe640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80cf590 0x7ff8c80cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8ccc91640 1 --2- 172.21.15.37:0/1808889989 >> v2:172.21.15.203:3300/0 conn(0x7ff8c8072000 0x7ff8c8072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129387179 0 0) 0x7ff8c80cbd20 con 0x7ff8c815b6b0 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b4003530 con 0x7ff8c815b6b0 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3395692065 0 0) 0x7ff8b4003530 con 0x7ff8c815b6b0 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80cf590 msgr2=0x7ff8c80cf9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80cf590 0x7ff8c80cf9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 >> v2:172.21.15.203:3300/0 conn(0x7ff8c8072000 msgr2=0x7ff8c8072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 --2- 172.21.15.37:0/1808889989 >> v2:172.21.15.203:3300/0 conn(0x7ff8c8072000 0x7ff8c8072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c80cc000 con 0x7ff8c815b6b0 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7ff8b0001680 con 0x7ff8c815b6b0 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 >> v1:172.21.15.37:6789/0 conn(0x7ff8c815b6b0 legacy=0x7ff8c80ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80cf590 0x7ff8b4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b4003fb0 0x7ff8b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8b4004d50 0x7ff8b40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c815ba90 con 0x7ff8b4003fb0 2024-04-25T09:04:24.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b400b9d0 con 0x7ff8b4004d50 2024-04-25T09:04:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b400bb80 con 0x7ff8c80cf590 2024-04-25T09:04:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c77fe640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b4003fb0 0x7ff8b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c7fff640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80cf590 0x7ff8b4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8ccc91640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8b4004d50 0x7ff8b40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c77fe640 1 -- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80cf590 msgr2=0x7ff8b4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c77fe640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80cf590 0x7ff8b4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c77fe640 1 -- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8b4004d50 msgr2=0x7ff8b40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c77fe640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8b4004d50 0x7ff8b40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c77fe640 1 -- 172.21.15.37:0/1808889989 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b4003530 con 0x7ff8b4003fb0 2024-04-25T09:04:24.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.085+0000 7ff8c7fff640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80cf590 0x7ff8b4003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:24.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.089+0000 7ff8c77fe640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b4003fb0 0x7ff8b4004810 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009ae0 tx=0x7ff8b801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c578e25b70c99bff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.089+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80b2070 con 0x7ff8b4003fb0 2024-04-25T09:04:24.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.089+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b80a5660 con 0x7ff8b4003fb0 2024-04-25T09:04:24.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.089+0000 7ff8c6ffd640 1 -- 172.21.15.37:0/1808889989 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80096b0 con 0x7ff8b4003fb0 2024-04-25T09:04:24.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.089+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1808889989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b4003fb0 msgr2=0x7ff8b4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.089+0000 7ff8ce6f4640 1 --2- 172.21.15.37:0/1808889989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8b4003fb0 0x7ff8b4004810 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009ae0 tx=0x7ff8b801ca70 comp rx=0 tx=0).stop 2024-04-25T09:04:24.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.089+0000 7ff8ccc91640 1 -- 172.21.15.37:0/1808889989 reap_dead start 2024-04-25T09:04:24.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.089+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1808889989 shutdown_connections 2024-04-25T09:04:24.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.089+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1808889989 >> 172.21.15.37:0/1808889989 conn(0x7ff8c806ae60 msgr2=0x7ff8c806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.089+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1808889989 shutdown_connections 2024-04-25T09:04:24.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.089+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1808889989 wait complete. 2024-04-25T09:04:24.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ce6f4640 1 Processor -- start 2024-04-25T09:04:24.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ce6f4640 1 -- start start 2024-04-25T09:04:24.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ce6f4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8b4003fb0 0x7ff8c81596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ce6f4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8c81f0ae0 0x7ff8c81f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ce6f4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80723e0 0x7ff8c8072860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ce6f4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80cc8a0 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ce6f4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80cc5c0 con 0x7ff8b4003fb0 2024-04-25T09:04:24.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ce6f4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c80cba30 con 0x7ff8c80723e0 2024-04-25T09:04:24.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8c81f0ae0 0x7ff8c81f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8c81f0ae0 0x7ff8c81f0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51446/0 (socket says 172.21.15.37:51446) 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c77fe640 1 -- 172.21.15.37:0/1426955337 learned_addr learned my addr 172.21.15.37:0/1426955337 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c7fff640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8b4003fb0 0x7ff8c81596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ccc91640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80723e0 0x7ff8c8072860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c77fe640 1 -- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80723e0 msgr2=0x7ff8c8072860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c77fe640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80723e0 0x7ff8c8072860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c77fe640 1 -- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8b4003fb0 msgr2=0x7ff8c81596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c77fe640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8b4003fb0 0x7ff8c81596d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c77fe640 1 -- 172.21.15.37:0/1426955337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c81f15d0 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c77fe640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8c81f0ae0 0x7ff8c81f0f60 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009ae0 tx=0x7ff8b4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c4ff9640 1 -- 172.21.15.37:0/1426955337 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80bc030 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c4ff9640 1 -- 172.21.15.37:0/1426955337 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b8004020 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1426955337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c81f18b0 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c4ff9640 1 -- 172.21.15.37:0/1426955337 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80b2040 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1426955337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c8159c10 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1426955337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c804efa0 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c4ff9640 1 -- 172.21.15.37:0/1426955337 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff8b80bf070 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c4ff9640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff8a8066f90 0x7ff8a8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.093+0000 7ff8c4ff9640 1 -- 172.21.15.37:0/1426955337 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7ff8b811ee60 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.097+0000 7ff8c7fff640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff8a8066f90 0x7ff8a8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.097+0000 7ff8c7fff640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff8a8066f90 0x7ff8a8069450 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0000fc0 tx=0x7ff8b0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.101+0000 7ff8c4ff9640 1 -- 172.21.15.37:0/1426955337 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff8b8016780 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.257+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1426955337 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff8c81f1a80 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8c4ff9640 1 -- 172.21.15.37:0/1426955337 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff8b80ef790 con 0x7ff8c81f0ae0 2024-04-25T09:04:24.249 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:04:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff8a8066f90 msgr2=0x7ff8a8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff8a8066f90 0x7ff8a8069450 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0000fc0 tx=0x7ff8b0009040 comp rx=0 tx=0).stop 2024-04-25T09:04:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1426955337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8c81f0ae0 msgr2=0x7ff8c81f0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8c81f0ae0 0x7ff8c81f0f60 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009ae0 tx=0x7ff8b4005840 comp rx=0 tx=0).stop 2024-04-25T09:04:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1426955337 shutdown_connections 2024-04-25T09:04:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff8a8066f90 0x7ff8a8069450 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff8c80723e0 0x7ff8c8072860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8c81f0ae0 0x7ff8c81f0f60 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 --2- 172.21.15.37:0/1426955337 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff8b4003fb0 0x7ff8c81596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1426955337 >> 172.21.15.37:0/1426955337 conn(0x7ff8c806ae60 msgr2=0x7ff8c80b3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1426955337 shutdown_connections 2024-04-25T09:04:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.261+0000 7ff8ce6f4640 1 -- 172.21.15.37:0/1426955337 wait complete. 2024-04-25T09:04:24.268 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:24.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7cfe02f640 1 Processor -- start 2024-04-25T09:04:24.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7cfe02f640 1 -- start start 2024-04-25T09:04:24.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7cfe02f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cf815d9e0 0x7f7cf806c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7cfe02f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7cf8066a80 0x7f7cf80cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7cfe02f640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf80ca3e0 con 0x7f7cf815ddc0 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7cfe02f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf806c630 con 0x7f7cf8066a80 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7cfe02f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf806b6a0 con 0x7f7cf815d9e0 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7ceffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7cf8066a80 0x7f7cf80cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7ceffff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f7cf8066a80 0x7f7cf80cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51460/0 (socket says 172.21.15.37:51460) 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7cf7fff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f7cf815ddc0 0x7f7cf80cefc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51986/0 (socket says 172.21.15.37:51986) 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7ceffff640 1 -- 172.21.15.37:0/443519547 learned_addr learned my addr 172.21.15.37:0/443519547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7cf77fe640 1 --2- 172.21.15.37:0/443519547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cf815d9e0 0x7f7cf806c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7ceffff640 1 -- 172.21.15.37:0/443519547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cf815d9e0 msgr2=0x7f7cf806c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7ceffff640 1 --2- 172.21.15.37:0/443519547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cf815d9e0 0x7f7cf806c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7ceffff640 1 -- 172.21.15.37:0/443519547 >> v1:172.21.15.37:6789/0 conn(0x7f7cf815ddc0 legacy=0x7f7cf80cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7ceffff640 1 -- 172.21.15.37:0/443519547 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf80ca6c0 con 0x7f7cf8066a80 2024-04-25T09:04:24.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.413+0000 7f7cf6ffd640 1 -- 172.21.15.37:0/443519547 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2353347095 0 0) 0x7f7cf80ca3e0 con 0x7f7cf815ddc0 2024-04-25T09:04:24.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7ceffff640 1 --2- 172.21.15.37:0/443519547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cf8066a80 0x7f7cf80cbd30 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009870 tx=0x7f7ce001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a988875adebd6c65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf6ffd640 1 -- 172.21.15.37:0/443519547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00ab650 con 0x7f7cf8066a80 2024-04-25T09:04:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf6ffd640 1 -- 172.21.15.37:0/443519547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ce00b0070 con 0x7f7cf8066a80 2024-04-25T09:04:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf6ffd640 1 -- 172.21.15.37:0/443519547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce00aba10 con 0x7f7cf8066a80 2024-04-25T09:04:24.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 -- 172.21.15.37:0/443519547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cf8066a80 msgr2=0x7f7cf80cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 --2- 172.21.15.37:0/443519547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cf8066a80 0x7f7cf80cbd30 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009870 tx=0x7f7ce001c560 comp rx=0 tx=0).stop 2024-04-25T09:04:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 -- 172.21.15.37:0/443519547 shutdown_connections 2024-04-25T09:04:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 --2- 172.21.15.37:0/443519547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cf8066a80 0x7f7cf80cbd30 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 --2- 172.21.15.37:0/443519547 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cf815d9e0 0x7f7cf806c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 -- 172.21.15.37:0/443519547 >> 172.21.15.37:0/443519547 conn(0x7f7cf80cc670 msgr2=0x7f7cf80cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 -- 172.21.15.37:0/443519547 shutdown_connections 2024-04-25T09:04:24.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 -- 172.21.15.37:0/443519547 wait complete. 2024-04-25T09:04:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 Processor -- start 2024-04-25T09:04:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 -- start start 2024-04-25T09:04:24.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8066a80 0x7f7cf80c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cf815d9e0 0x7f7cf80c5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cf815ddc0 0x7f7cf8157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80cf850 con 0x7f7cf815ddc0 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf80cf6d0 con 0x7f7cf8066a80 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cfe02f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf806e020 con 0x7f7cf815d9e0 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cf815ddc0 0x7f7cf8157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cf815ddc0 0x7f7cf8157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51466/0 (socket says 172.21.15.37:51466) 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf7fff640 1 -- 172.21.15.37:0/3505242379 learned_addr learned my addr 172.21.15.37:0/3505242379 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7ceffff640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cf815d9e0 0x7f7cf80c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf77fe640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8066a80 0x7f7cf80c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf7fff640 1 -- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cf815d9e0 msgr2=0x7f7cf80c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf7fff640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cf815d9e0 0x7f7cf80c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf7fff640 1 -- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8066a80 msgr2=0x7f7cf80c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf7fff640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8066a80 0x7f7cf80c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf7fff640 1 -- 172.21.15.37:0/3505242379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf80c75f0 con 0x7f7cf815ddc0 2024-04-25T09:04:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.417+0000 7f7cf77fe640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8066a80 0x7f7cf80c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cf7fff640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cf815ddc0 0x7f7cf8157760 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c6d0 tx=0x7f7cf000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cf4ff9640 1 -- 172.21.15.37:0/3505242379 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf0018070 con 0x7f7cf815ddc0 2024-04-25T09:04:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cf4ff9640 1 -- 172.21.15.37:0/3505242379 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf0004d70 con 0x7f7cf815ddc0 2024-04-25T09:04:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cf4ff9640 1 -- 172.21.15.37:0/3505242379 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf000a880 con 0x7f7cf815ddc0 2024-04-25T09:04:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cfe02f640 1 -- 172.21.15.37:0/3505242379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf815b030 con 0x7f7cf815ddc0 2024-04-25T09:04:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cfe02f640 1 -- 172.21.15.37:0/3505242379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf809e890 con 0x7f7cf815ddc0 2024-04-25T09:04:24.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cfe02f640 1 -- 172.21.15.37:0/3505242379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf804efa0 con 0x7f7cf815ddc0 2024-04-25T09:04:24.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cf4ff9640 1 -- 172.21.15.37:0/3505242379 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7cf000ac30 con 0x7f7cf815ddc0 2024-04-25T09:04:24.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cf4ff9640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cd8066f60 0x7f7cd8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cf4ff9640 1 -- 172.21.15.37:0/3505242379 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f7cf0084850 con 0x7f7cf815ddc0 2024-04-25T09:04:24.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cf77fe640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cd8066f60 0x7f7cd8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.421+0000 7f7cf77fe640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cd8066f60 0x7f7cd8069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7cf81f9980 tx=0x7f7ce8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.425+0000 7f7cf4ff9640 1 -- 172.21.15.37:0/3505242379 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7cf00556b0 con 0x7f7cf815ddc0 2024-04-25T09:04:24.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.585+0000 7f7cfe02f640 1 -- 172.21.15.37:0/3505242379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7cf806e3e0 con 0x7f7cf815ddc0 2024-04-25T09:04:24.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.585+0000 7f7cf4ff9640 1 -- 172.21.15.37:0/3505242379 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7cf0054fb0 con 0x7f7cf815ddc0 2024-04-25T09:04:24.576 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:04:24.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 -- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cd8066f60 msgr2=0x7f7cd8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cd8066f60 0x7f7cd8069420 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f7cf81f9980 tx=0x7f7ce8008040 comp rx=0 tx=0).stop 2024-04-25T09:04:24.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 -- 172.21.15.37:0/3505242379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cf815ddc0 msgr2=0x7f7cf8157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cf815ddc0 0x7f7cf8157760 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 crypto rx=0x7f7cf000c6d0 tx=0x7f7cf000cb90 comp rx=0 tx=0).stop 2024-04-25T09:04:24.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 -- 172.21.15.37:0/3505242379 shutdown_connections 2024-04-25T09:04:24.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cd8066f60 0x7f7cd8069420 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cf815ddc0 0x7f7cf8157760 unknown :-1 s=CLOSED pgs=2186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cf815d9e0 0x7f7cf80c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 --2- 172.21.15.37:0/3505242379 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cf8066a80 0x7f7cf80c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 -- 172.21.15.37:0/3505242379 >> 172.21.15.37:0/3505242379 conn(0x7f7cf80cc670 msgr2=0x7f7cf806e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 -- 172.21.15.37:0/3505242379 shutdown_connections 2024-04-25T09:04:24.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.589+0000 7f7cfe02f640 1 -- 172.21.15.37:0/3505242379 wait complete. 2024-04-25T09:04:24.596 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:04:24.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa88ba6c640 1 Processor -- start 2024-04-25T09:04:24.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa88ba6c640 1 -- start start 2024-04-25T09:04:24.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa88ba6c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa88415da80 0x7fa8840d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa88ba6c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d07f0 0x7fa88406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa88ba6c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88406be60 con 0x7fa8840d1cd0 2024-04-25T09:04:24.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa88ba6c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88406c440 con 0x7fa88415da80 2024-04-25T09:04:24.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa88ba6c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88415de60 con 0x7fa8840d07f0 2024-04-25T09:04:24.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa889007640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fa8840d1cd0 0x7fa8840d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:51988/0 (socket says 172.21.15.37:51988) 2024-04-25T09:04:24.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa889007640 1 -- 172.21.15.37:0/3000329603 learned_addr learned my addr 172.21.15.37:0/3000329603 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:24.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa88a009640 1 --2- 172.21.15.37:0/3000329603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d07f0 0x7fa88406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa889808640 1 --2- 172.21.15.37:0/3000329603 >> v2:172.21.15.203:3300/0 conn(0x7fa88415da80 0x7fa8840d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052812337 0 0) 0x7fa88406be60 con 0x7fa8840d1cd0 2024-04-25T09:04:24.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa870003530 con 0x7fa8840d1cd0 2024-04-25T09:04:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2709573554 0 0) 0x7fa870003530 con 0x7fa8840d1cd0 2024-04-25T09:04:24.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d07f0 msgr2=0x7fa88406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 --2- 172.21.15.37:0/3000329603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d07f0 0x7fa88406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 >> v2:172.21.15.203:3300/0 conn(0x7fa88415da80 msgr2=0x7fa8840d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 --2- 172.21.15.37:0/3000329603 >> v2:172.21.15.203:3300/0 conn(0x7fa88415da80 0x7fa8840d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8840d2450 con 0x7fa8840d1cd0 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fa8740016f0 con 0x7fa8840d1cd0 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 >> v1:172.21.15.37:6789/0 conn(0x7fa8840d1cd0 legacy=0x7fa8840d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 --2- 172.21.15.37:0/3000329603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa870003a30 0x7fa870003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 --2- 172.21.15.37:0/3000329603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa870004810 0x7fa870004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 --2- 172.21.15.37:0/3000329603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d07f0 0x7fa8700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88406c440 con 0x7fa870004810 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88415de60 con 0x7fa870003a30 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87000ba00 con 0x7fa8840d07f0 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa889007640 1 --2- 172.21.15.37:0/3000329603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa870004810 0x7fa870004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa889808640 1 --2- 172.21.15.37:0/3000329603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa870003a30 0x7fa870003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa889007640 1 -- 172.21.15.37:0/3000329603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d07f0 msgr2=0x7fa8700051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa889007640 1 --2- 172.21.15.37:0/3000329603 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8840d07f0 0x7fa8700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa889007640 1 -- 172.21.15.37:0/3000329603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa870003a30 msgr2=0x7fa870003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:24.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa889007640 1 --2- 172.21.15.37:0/3000329603 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa870003a30 0x7fa870003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.729+0000 7fa889007640 1 -- 172.21.15.37:0/3000329603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa870003530 con 0x7fa870004810 2024-04-25T09:04:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa889007640 1 --2- 172.21.15.37:0/3000329603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa870004810 0x7fa870004c90 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fa874001ec0 tx=0x7fa87401f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f857f9f6ce6eb2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874007d40 con 0x7fa870004810 2024-04-25T09:04:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa874005ae0 con 0x7fa870004810 2024-04-25T09:04:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa888806640 1 -- 172.21.15.37:0/3000329603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874007070 con 0x7fa870004810 2024-04-25T09:04:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa88ba6c640 1 -- 172.21.15.37:0/3000329603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa870004810 msgr2=0x7fa870004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa88ba6c640 1 --2- 172.21.15.37:0/3000329603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa870004810 0x7fa870004c90 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fa874001ec0 tx=0x7fa87401f310 comp rx=0 tx=0).stop 2024-04-25T09:04:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa88a009640 1 -- 172.21.15.37:0/3000329603 reap_dead start 2024-04-25T09:04:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa88ba6c640 1 -- 172.21.15.37:0/3000329603 shutdown_connections 2024-04-25T09:04:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa88ba6c640 1 -- 172.21.15.37:0/3000329603 >> 172.21.15.37:0/3000329603 conn(0x7fa88408b000 msgr2=0x7fa8840d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa88ba6c640 1 -- 172.21.15.37:0/3000329603 shutdown_connections 2024-04-25T09:04:24.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa88ba6c640 1 -- 172.21.15.37:0/3000329603 wait complete. 2024-04-25T09:04:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa88ba6c640 1 Processor -- start 2024-04-25T09:04:24.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.733+0000 7fa88ba6c640 1 -- start start 2024-04-25T09:04:24.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa88ba6c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa870004810 0x7fa884151d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa88ba6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415ad80 0x7fa8840c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa88ba6c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8841606c0 0x7fa884160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa88ba6c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840d26f0 con 0x7fa88415ad80 2024-04-25T09:04:24.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa88ba6c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840d2120 con 0x7fa870004810 2024-04-25T09:04:24.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa88ba6c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8840d2870 con 0x7fa8841606c0 2024-04-25T09:04:24.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889007640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415ad80 0x7fa8840c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889007640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415ad80 0x7fa8840c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51492/0 (socket says 172.21.15.37:51492) 2024-04-25T09:04:24.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889007640 1 -- 172.21.15.37:0/1739586123 learned_addr learned my addr 172.21.15.37:0/1739586123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889808640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa870004810 0x7fa884151d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa88a009640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8841606c0 0x7fa884160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889007640 1 -- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8841606c0 msgr2=0x7fa884160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889007640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8841606c0 0x7fa884160b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889007640 1 -- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa870004810 msgr2=0x7fa884151d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889007640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa870004810 0x7fa884151d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889007640 1 -- 172.21.15.37:0/1739586123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8840b73c0 con 0x7fa88415ad80 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889808640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa870004810 0x7fa884151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889007640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415ad80 0x7fa8840c71e0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fa8740019f0 tx=0x7fa870005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa87a7fc640 1 -- 172.21.15.37:0/1739586123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8740047f0 con 0x7fa88415ad80 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa87a7fc640 1 -- 172.21.15.37:0/1739586123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8740049b0 con 0x7fa88415ad80 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa88ba6c640 1 -- 172.21.15.37:0/1739586123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8840b7660 con 0x7fa88415ad80 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa87a7fc640 1 -- 172.21.15.37:0/1739586123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa874004da0 con 0x7fa88415ad80 2024-04-25T09:04:24.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa88ba6c640 1 -- 172.21.15.37:0/1739586123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8840b7c20 con 0x7fa88415ad80 2024-04-25T09:04:24.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa88ba6c640 1 -- 172.21.15.37:0/1739586123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa88404efa0 con 0x7fa88415ad80 2024-04-25T09:04:24.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa87a7fc640 1 -- 172.21.15.37:0/1739586123 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa874007070 con 0x7fa88415ad80 2024-04-25T09:04:24.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa87a7fc640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa868066f60 0x7fa868069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:24.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa87a7fc640 1 -- 172.21.15.37:0/1739586123 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fa874122230 con 0x7fa88415ad80 2024-04-25T09:04:24.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.737+0000 7fa889808640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa868066f60 0x7fa868069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:24.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.741+0000 7fa889808640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa868066f60 0x7fa868069420 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa8800099a0 tx=0x7fa880008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:24.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.745+0000 7fa87a7fc640 1 -- 172.21.15.37:0/1739586123 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa8740b7070 con 0x7fa88415ad80 2024-04-25T09:04:24.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.913+0000 7fa88ba6c640 1 -- 172.21.15.37:0/1739586123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa88406dac0 con 0x7fa88415ad80 2024-04-25T09:04:24.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa87a7fc640 1 -- 172.21.15.37:0/1739586123 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v127) v1 ==== 80+0+244 (secure 0 0 0) 0x7fa874019080 con 0x7fa88415ad80 2024-04-25T09:04:24.904 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/2159545482 2024-04-26T09:04:12.937783+0000 2024-04-25T09:04:24.904 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6834/2159545482 2024-04-26T09:04:12.937783+0000 2024-04-25T09:04:24.904 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6835/2748919477 2024-04-26T09:03:58.978336+0000 2024-04-25T09:04:24.905 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6834/2748919477 2024-04-26T09:03:58.978336+0000 2024-04-25T09:04:24.905 INFO:teuthology.orchestra.run.smithi037.stderr:listed 4 entries 2024-04-25T09:04:24.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 -- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa868066f60 msgr2=0x7fa868069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa868066f60 0x7fa868069420 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa8800099a0 tx=0x7fa880008040 comp rx=0 tx=0).stop 2024-04-25T09:04:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 -- 172.21.15.37:0/1739586123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415ad80 msgr2=0x7fa8840c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415ad80 0x7fa8840c71e0 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 crypto rx=0x7fa8740019f0 tx=0x7fa870005840 comp rx=0 tx=0).stop 2024-04-25T09:04:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 -- 172.21.15.37:0/1739586123 shutdown_connections 2024-04-25T09:04:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa868066f60 0x7fa868069420 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa8841606c0 0x7fa884160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa88415ad80 0x7fa8840c71e0 unknown :-1 s=CLOSED pgs=2189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 --2- 172.21.15.37:0/1739586123 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa870004810 0x7fa884151d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 -- 172.21.15.37:0/1739586123 >> 172.21.15.37:0/1739586123 conn(0x7fa88408b000 msgr2=0x7fa8840b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 -- 172.21.15.37:0/1739586123 shutdown_connections 2024-04-25T09:04:24.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:24.917+0000 7fa88ba6c640 1 -- 172.21.15.37:0/1739586123 wait complete. 2024-04-25T09:04:24.923 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi037.front.sepia.ceph.com... 2024-04-25T09:04:24.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:24.923 DEBUG:teuthology.orchestra.run.smithi037:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:24.935 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi037.stderr:ceph-fuse[56351]: fuse finished with error 0 and tester_r 0 2024-04-25T09:04:24.936 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:04:24.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:24.937 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:04:25.059 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:04:25.059 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:25.066 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:25.066 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T09:04:25.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:25.066 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:04:25.113 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:04:25.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:04:25.115 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:04:25.115 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:25.121 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:04:25.155 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:04:25.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:25.156 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:04:25.168 INFO:teuthology.orchestra.run.smithi037.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:04:25.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:04:25.169 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:04:25.169 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:25.175 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:25.175 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T09:04:25.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:25.175 DEBUG:teuthology.orchestra.run.smithi203:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:04:25.224 INFO:teuthology.orchestra.run.smithi203.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:04:25.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:04:25.225 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:25.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07b5ea5640 1 Processor -- start 2024-04-25T09:04:25.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07b5ea5640 1 -- start start 2024-04-25T09:04:25.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07b5ea5640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f07b015d850 0x7f07b00d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07b5ea5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07b015dc30 0x7f07b00d06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07b5ea5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f07b00d1de0 0x7f07b00d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07b5ea5640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b00d27b0 con 0x7f07b00d1de0 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07b5ea5640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07b00ce6b0 con 0x7f07b015d850 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07b5ea5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b00ce990 con 0x7f07b015dc30 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07affff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f07b00d1de0 0x7f07b00d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07affff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f07b00d1de0 0x7f07b00d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51506/0 (socket says 172.21.15.37:51506) 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07affff640 1 -- 172.21.15.37:0/3327858225 learned_addr learned my addr 172.21.15.37:0/3327858225 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07aeffd640 1 --2- 172.21.15.37:0/3327858225 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07b015dc30 0x7f07b00d06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07af7fe640 1 --2- 172.21.15.37:0/3327858225 >> v2:172.21.15.203:3300/0 conn(0x7f07b015d850 0x7f07b00d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07affff640 1 -- 172.21.15.37:0/3327858225 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07b015dc30 msgr2=0x7f07b00d06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07affff640 1 --2- 172.21.15.37:0/3327858225 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07b015dc30 0x7f07b00d06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07affff640 1 -- 172.21.15.37:0/3327858225 >> v2:172.21.15.203:3300/0 conn(0x7f07b015d850 msgr2=0x7f07b00d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:25.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07affff640 1 --2- 172.21.15.37:0/3327858225 >> v2:172.21.15.203:3300/0 conn(0x7f07b015d850 0x7f07b00d0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.369+0000 7f07affff640 1 -- 172.21.15.37:0/3327858225 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b00cec70 con 0x7f07b00d1de0 2024-04-25T09:04:25.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07affff640 1 --2- 172.21.15.37:0/3327858225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b00d1de0 0x7f07b00d2270 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f07b0067970 tx=0x7f07a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0df73d8b8f5653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:25.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07ae7fc640 1 -- 172.21.15.37:0/3327858225 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80ab640 con 0x7f07b00d1de0 2024-04-25T09:04:25.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07ae7fc640 1 -- 172.21.15.37:0/3327858225 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a80b0070 con 0x7f07b00d1de0 2024-04-25T09:04:25.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07ae7fc640 1 -- 172.21.15.37:0/3327858225 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a80aba00 con 0x7f07b00d1de0 2024-04-25T09:04:25.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 -- 172.21.15.37:0/3327858225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b00d1de0 msgr2=0x7f07b00d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 --2- 172.21.15.37:0/3327858225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b00d1de0 0x7f07b00d2270 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f07b0067970 tx=0x7f07a801c520 comp rx=0 tx=0).stop 2024-04-25T09:04:25.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 -- 172.21.15.37:0/3327858225 shutdown_connections 2024-04-25T09:04:25.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 --2- 172.21.15.37:0/3327858225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b00d1de0 0x7f07b00d2270 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 --2- 172.21.15.37:0/3327858225 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07b015dc30 0x7f07b00d06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 --2- 172.21.15.37:0/3327858225 >> v2:172.21.15.203:3300/0 conn(0x7f07b015d850 0x7f07b00d0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 -- 172.21.15.37:0/3327858225 >> 172.21.15.37:0/3327858225 conn(0x7f07b008b000 msgr2=0x7f07b00d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:25.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 -- 172.21.15.37:0/3327858225 shutdown_connections 2024-04-25T09:04:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 -- 172.21.15.37:0/3327858225 wait complete. 2024-04-25T09:04:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 Processor -- start 2024-04-25T09:04:25.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 -- start start 2024-04-25T09:04:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00d1de0 0x7f07b0159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b015d850 0x7f07b014e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07b015dc30 0x7f07b014e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0161f70 con 0x7f07b015d850 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b0161df0 con 0x7f07b00d1de0 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07b5ea5640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b01620f0 con 0x7f07b015dc30 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07aeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b015d850 0x7f07b014e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07aeffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b015d850 0x7f07b014e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51520/0 (socket says 172.21.15.37:51520) 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.373+0000 7f07aeffd640 1 -- 172.21.15.37:0/2739488643 learned_addr learned my addr 172.21.15.37:0/2739488643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07af7fe640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00d1de0 0x7f07b0159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07affff640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07b015dc30 0x7f07b014e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07aeffd640 1 -- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07b015dc30 msgr2=0x7f07b014e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07aeffd640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07b015dc30 0x7f07b014e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07aeffd640 1 -- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00d1de0 msgr2=0x7f07b0159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07aeffd640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00d1de0 0x7f07b0159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07aeffd640 1 -- 172.21.15.37:0/2739488643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b0158870 con 0x7f07b015d850 2024-04-25T09:04:25.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07af7fe640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00d1de0 0x7f07b0159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:25.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07aeffd640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b015d850 0x7f07b014e480 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f07a400d620 tx=0x7f07a400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:25.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f0797fff640 1 -- 172.21.15.37:0/2739488643 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400b620 con 0x7f07b015d850 2024-04-25T09:04:25.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f0797fff640 1 -- 172.21.15.37:0/2739488643 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a4019070 con 0x7f07b015d850 2024-04-25T09:04:25.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07b5ea5640 1 -- 172.21.15.37:0/2739488643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07b0158b50 con 0x7f07b015d850 2024-04-25T09:04:25.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f0797fff640 1 -- 172.21.15.37:0/2739488643 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400b9e0 con 0x7f07b015d850 2024-04-25T09:04:25.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07b5ea5640 1 -- 172.21.15.37:0/2739488643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07b0159110 con 0x7f07b015d850 2024-04-25T09:04:25.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07b5ea5640 1 -- 172.21.15.37:0/2739488643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07b004efa0 con 0x7f07b015d850 2024-04-25T09:04:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f0797fff640 1 -- 172.21.15.37:0/2739488643 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f07a400bb80 con 0x7f07b015d850 2024-04-25T09:04:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f0797fff640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f079c066f90 0x7f079c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f0797fff640 1 -- 172.21.15.37:0/2739488643 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f07a40856f0 con 0x7f07b015d850 2024-04-25T09:04:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07af7fe640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f079c066f90 0x7f079c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.377+0000 7f07af7fe640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f079c066f90 0x7f079c069450 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009f70 tx=0x7f07a000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:25.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.381+0000 7f0797fff640 1 -- 172.21.15.37:0/2739488643 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f07a4056430 con 0x7f07b015d850 2024-04-25T09:04:25.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 -- 172.21.15.37:0/2739488643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07b006c6f0 con 0x7f07b015d850 2024-04-25T09:04:25.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f0797fff640 1 -- 172.21.15.37:0/2739488643 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f07a4055d30 con 0x7f07b015d850 2024-04-25T09:04:25.565 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:25.565 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:04:16.988456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/2361387213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2361387213},{"type":"v1","addr":"172.21.15.37:6835","nonce":2361387213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:04:25.566 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 110 2024-04-25T09:04:25.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 -- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f079c066f90 msgr2=0x7f079c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f079c066f90 0x7f079c069450 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f07a0009f70 tx=0x7f07a000b040 comp rx=0 tx=0).stop 2024-04-25T09:04:25.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 -- 172.21.15.37:0/2739488643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b015d850 msgr2=0x7f07b014e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b015d850 0x7f07b014e480 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f07a400d620 tx=0x7f07a400dae0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 -- 172.21.15.37:0/2739488643 shutdown_connections 2024-04-25T09:04:25.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f079c066f90 0x7f079c069450 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f07b015dc30 0x7f07b014e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07b015d850 0x7f07b014e480 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 --2- 172.21.15.37:0/2739488643 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f07b00d1de0 0x7f07b0159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 -- 172.21.15.37:0/2739488643 >> 172.21.15.37:0/2739488643 conn(0x7f07b008b000 msgr2=0x7f07b006cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:25.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 -- 172.21.15.37:0/2739488643 shutdown_connections 2024-04-25T09:04:25.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.577+0000 7f07b5ea5640 1 -- 172.21.15.37:0/2739488643 wait complete. 2024-04-25T09:04:25.584 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:25.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.717+0000 7f91b3d46640 1 Processor -- start 2024-04-25T09:04:25.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.717+0000 7f91b3d46640 1 -- start start 2024-04-25T09:04:25.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b3d46640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f91ac144ca0 0x7f91ac06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b3d46640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91ac0ccc70 con 0x7f91ac06ec60 2024-04-25T09:04:25.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b3d46640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91ac06dc40 con 0x7f91ac0ce510 2024-04-25T09:04:25.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b3d46640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91ac0cc2b0 con 0x7f91ac144ca0 2024-04-25T09:04:25.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b22e3640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f91ac06ec60 0x7f91ac142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52002/0 (socket says 172.21.15.37:52002) 2024-04-25T09:04:25.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b22e3640 1 -- 172.21.15.37:0/3234653645 learned_addr learned my addr 172.21.15.37:0/3234653645 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:25.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b12e1640 1 --2- 172.21.15.37:0/3234653645 >> v2:172.21.15.203:3300/0 conn(0x7f91ac144ca0 0x7f91ac06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536197778 0 0) 0x7f91ac0ccc70 con 0x7f91ac06ec60 2024-04-25T09:04:25.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9194003530 con 0x7f91ac06ec60 2024-04-25T09:04:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1548571188 0 0) 0x7f9194003530 con 0x7f91ac06ec60 2024-04-25T09:04:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 >> v2:172.21.15.203:3300/0 conn(0x7f91ac144ca0 msgr2=0x7f91ac06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 --2- 172.21.15.37:0/3234653645 >> v2:172.21.15.203:3300/0 conn(0x7f91ac144ca0 0x7f91ac06e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 >> v1:172.21.15.203:6789/0 conn(0x7f91ac0ce510 legacy=0x7f91ac0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac06df20 con 0x7f91ac06ec60 2024-04-25T09:04:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f91a4001a00 con 0x7f91ac06ec60 2024-04-25T09:04:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 >> v1:172.21.15.37:6789/0 conn(0x7f91ac06ec60 legacy=0x7f91ac142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9194003a30 0x7f9194003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9194004810 0x7f9194004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9194005240 0x7f919400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac0cc2b0 con 0x7f9194005240 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f919400c190 con 0x7f9194004810 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f919400c390 con 0x7f9194003a30 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b22e3640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9194005240 0x7f919400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b12e1640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9194004810 0x7f9194004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b22e3640 1 -- 172.21.15.37:0/3234653645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9194003a30 msgr2=0x7f9194003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b1ae2640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9194003a30 0x7f9194003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b22e3640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9194003a30 0x7f9194003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b22e3640 1 -- 172.21.15.37:0/3234653645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9194004810 msgr2=0x7f9194004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b22e3640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9194004810 0x7f9194004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b22e3640 1 -- 172.21.15.37:0/3234653645 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9194003530 con 0x7f9194005240 2024-04-25T09:04:25.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.721+0000 7f91b12e1640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f9194004810 0x7f9194004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b22e3640 1 -- 172.21.15.37:0/3234653645 reap_dead start 2024-04-25T09:04:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b22e3640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9194005240 0x7f919400b9d0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f91a400a850 tx=0x7f91a401d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35ee79649f46eb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a400a4a0 con 0x7f9194005240 2024-04-25T09:04:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a40ad650 con 0x7f9194005240 2024-04-25T09:04:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b0ae0640 1 -- 172.21.15.37:0/3234653645 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a40ada40 con 0x7f9194005240 2024-04-25T09:04:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 -- 172.21.15.37:0/3234653645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9194005240 msgr2=0x7f919400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9194005240 0x7f919400b9d0 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f91a400a850 tx=0x7f91a401d3a0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 -- 172.21.15.37:0/3234653645 shutdown_connections 2024-04-25T09:04:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 --2- 172.21.15.37:0/3234653645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9194005240 0x7f919400b9d0 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 -- 172.21.15.37:0/3234653645 >> 172.21.15.37:0/3234653645 conn(0x7f91ac0b11a0 msgr2=0x7f91ac0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:25.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 -- 172.21.15.37:0/3234653645 shutdown_connections 2024-04-25T09:04:25.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 -- 172.21.15.37:0/3234653645 wait complete. 2024-04-25T09:04:25.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 Processor -- start 2024-04-25T09:04:25.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 -- start start 2024-04-25T09:04:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9194005240 0x7f91ac15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91ac0ad750 0x7f91ac0adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91ac1504a0 0x7f91ac150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac1451f0 con 0x7f91ac1504a0 2024-04-25T09:04:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac06e290 con 0x7f91ac0ad750 2024-04-25T09:04:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91ac145370 con 0x7f9194005240 2024-04-25T09:04:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b22e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91ac1504a0 0x7f91ac150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b22e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91ac1504a0 0x7f91ac150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51538/0 (socket says 172.21.15.37:51538) 2024-04-25T09:04:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b22e3640 1 -- 172.21.15.37:0/661221150 learned_addr learned my addr 172.21.15.37:0/661221150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b1ae2640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9194005240 0x7f91ac15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b12e1640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91ac0ad750 0x7f91ac0adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b22e3640 1 -- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9194005240 msgr2=0x7f91ac15b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b22e3640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9194005240 0x7f91ac15b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b22e3640 1 -- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91ac0ad750 msgr2=0x7f91ac0adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b22e3640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91ac0ad750 0x7f91ac0adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b22e3640 1 -- 172.21.15.37:0/661221150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91ac152750 con 0x7f91ac1504a0 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b22e3640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91ac1504a0 0x7f91ac150920 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f91a401dc40 tx=0x7f9194005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91aa7fc640 1 -- 172.21.15.37:0/661221150 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4001940 con 0x7f91ac1504a0 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91aa7fc640 1 -- 172.21.15.37:0/661221150 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a40b4070 con 0x7f91ac1504a0 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 -- 172.21.15.37:0/661221150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91ac1529c0 con 0x7f91ac1504a0 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91aa7fc640 1 -- 172.21.15.37:0/661221150 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a40ada00 con 0x7f91ac1504a0 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.725+0000 7f91b3d46640 1 -- 172.21.15.37:0/661221150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91ac152f80 con 0x7f91ac1504a0 2024-04-25T09:04:25.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.729+0000 7f91b3d46640 1 -- 172.21.15.37:0/661221150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91ac04efa0 con 0x7f91ac1504a0 2024-04-25T09:04:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.729+0000 7f91aa7fc640 1 -- 172.21.15.37:0/661221150 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f91a40b6070 con 0x7f91ac1504a0 2024-04-25T09:04:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.729+0000 7f91aa7fc640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9198066f90 0x7f9198069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.729+0000 7f91aa7fc640 1 -- 172.21.15.37:0/661221150 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f91a411f100 con 0x7f91ac1504a0 2024-04-25T09:04:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.729+0000 7f91b1ae2640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9198066f90 0x7f9198069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:25.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.729+0000 7f91b1ae2640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9198066f90 0x7f9198069450 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f919c001660 tx=0x7f919c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:25.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.733+0000 7f91aa7fc640 1 -- 172.21.15.37:0/661221150 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f91a40170b0 con 0x7f91ac1504a0 2024-04-25T09:04:25.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.893+0000 7f91b3d46640 1 -- 172.21.15.37:0/661221150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91ac0b1cc0 con 0x7f91ac1504a0 2024-04-25T09:04:25.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91aa7fc640 1 -- 172.21.15.37:0/661221150 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17175 (secure 0 0 0) 0x7f91a40efe40 con 0x7f91ac1504a0 2024-04-25T09:04:25.885 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:25.885 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":127,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:04:12.937853+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2159545482":"2024-04-26T09:04:12.937783+0000","172.21.15.37:6834/2159545482":"2024-04-26T09:04:12.937783+0000","172.21.15.37:6835/2748919477":"2024-04-26T09:03:58.978336+0000","172.21.15.37:6834/2748919477":"2024-04-26T09:03:58.978336+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:25.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 -- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9198066f90 msgr2=0x7f9198069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9198066f90 0x7f9198069450 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f919c001660 tx=0x7f919c009040 comp rx=0 tx=0).stop 2024-04-25T09:04:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 -- 172.21.15.37:0/661221150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91ac1504a0 msgr2=0x7f91ac150920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91ac1504a0 0x7f91ac150920 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7f91a401dc40 tx=0x7f9194005840 comp rx=0 tx=0).stop 2024-04-25T09:04:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 -- 172.21.15.37:0/661221150 shutdown_connections 2024-04-25T09:04:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9198066f90 0x7f9198069450 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91ac1504a0 0x7f91ac150920 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f91ac0ad750 0x7f91ac0adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 --2- 172.21.15.37:0/661221150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f9194005240 0x7f91ac15b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 -- 172.21.15.37:0/661221150 >> 172.21.15.37:0/661221150 conn(0x7f91ac0b11a0 msgr2=0x7f91ac0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 -- 172.21.15.37:0/661221150 shutdown_connections 2024-04-25T09:04:25.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:25.897+0000 7f91b3d46640 1 -- 172.21.15.37:0/661221150 wait complete. 2024-04-25T09:04:25.905 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:52:19.193245+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:30.763724+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:31.488462+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:25.905 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:26.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc5013ac640 1 Processor -- start 2024-04-25T09:04:26.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc5013ac640 1 -- start start 2024-04-25T09:04:26.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc5013ac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc0d07f0 0x7fc4fc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc5013ac640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc4fc0d1cd0 0x7fc4fc06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc5013ac640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4fc06be60 con 0x7fc4fc15da80 2024-04-25T09:04:26.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc5013ac640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4fc06c440 con 0x7fc4fc0d1cd0 2024-04-25T09:04:26.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc5013ac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc15de60 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc4fa7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc0d07f0 0x7fc4fc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc4fb7fe640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc4fc0d1cd0 0x7fc4fc06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc4fa7fc640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc0d07f0 0x7fc4fc0d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3301/0 says I am v2:172.21.15.37:56862/0 (socket says 172.21.15.37:56862) 2024-04-25T09:04:26.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc4fa7fc640 1 -- 172.21.15.37:0/1353563872 learned_addr learned my addr 172.21.15.37:0/1353563872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:26.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc4fa7fc640 1 -- 172.21.15.37:0/1353563872 >> v2:172.21.15.203:3300/0 conn(0x7fc4fc0d1cd0 msgr2=0x7fc4fc06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:26.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc4fa7fc640 1 --2- 172.21.15.37:0/1353563872 >> v2:172.21.15.203:3300/0 conn(0x7fc4fc0d1cd0 0x7fc4fc06c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc4fa7fc640 1 -- 172.21.15.37:0/1353563872 >> v1:172.21.15.203:6789/0 conn(0x7fc4fc15da80 legacy=0x7fc4fc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc4fa7fc640 1 -- 172.21.15.37:0/1353563872 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc0d2450 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.045+0000 7fc4fb7fe640 1 --2- 172.21.15.37:0/1353563872 >> v2:172.21.15.203:3300/0 conn(0x7fc4fc0d1cd0 0x7fc4fc06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:26.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc4fa7fc640 1 --2- 172.21.15.37:0/1353563872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc0d07f0 0x7fc4fc0d4230 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0006fd0 tx=0x7fc4f001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8dbcc0c88f6bfa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc4f9ffb640 1 -- 172.21.15.37:0/1353563872 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00a7070 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc4f9ffb640 1 -- 172.21.15.37:0/1353563872 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f00b0070 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc4f9ffb640 1 -- 172.21.15.37:0/1353563872 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f00ab890 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc5013ac640 1 -- 172.21.15.37:0/1353563872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc0d07f0 msgr2=0x7fc4fc0d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc5013ac640 1 --2- 172.21.15.37:0/1353563872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc0d07f0 0x7fc4fc0d4230 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0006fd0 tx=0x7fc4f001c510 comp rx=0 tx=0).stop 2024-04-25T09:04:26.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc5013ac640 1 -- 172.21.15.37:0/1353563872 shutdown_connections 2024-04-25T09:04:26.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc5013ac640 1 --2- 172.21.15.37:0/1353563872 >> v2:172.21.15.203:3300/0 conn(0x7fc4fc0d1cd0 0x7fc4fc06c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc5013ac640 1 --2- 172.21.15.37:0/1353563872 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc0d07f0 0x7fc4fc0d4230 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc5013ac640 1 -- 172.21.15.37:0/1353563872 >> 172.21.15.37:0/1353563872 conn(0x7fc4fc08b000 msgr2=0x7fc4fc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:26.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc5013ac640 1 -- 172.21.15.37:0/1353563872 shutdown_connections 2024-04-25T09:04:26.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.049+0000 7fc5013ac640 1 -- 172.21.15.37:0/1353563872 wait complete. 2024-04-25T09:04:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc5013ac640 1 Processor -- start 2024-04-25T09:04:26.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc5013ac640 1 -- start start 2024-04-25T09:04:26.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc5013ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4fc0d07f0 0x7fc4fc1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc5013ac640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4fc0d1cd0 0x7fc4fc0b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc5013ac640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc15da80 0x7fc4fc0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc5013ac640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc06d7a0 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc5013ac640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc06d620 con 0x7fc4fc0d1cd0 2024-04-25T09:04:26.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc5013ac640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4fc06d920 con 0x7fc4fc15da80 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4faffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4fc0d07f0 0x7fc4fc1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4faffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4fc0d07f0 0x7fc4fc1555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51540/0 (socket says 172.21.15.37:51540) 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4faffd640 1 -- 172.21.15.37:0/601374628 learned_addr learned my addr 172.21.15.37:0/601374628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4fa7fc640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4fc0d1cd0 0x7fc4fc0b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4fb7fe640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc15da80 0x7fc4fc0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4faffd640 1 -- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc15da80 msgr2=0x7fc4fc0b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4faffd640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc15da80 0x7fc4fc0b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4faffd640 1 -- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4fc0d1cd0 msgr2=0x7fc4fc0b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4faffd640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4fc0d1cd0 0x7fc4fc0b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4faffd640 1 -- 172.21.15.37:0/601374628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4fc15e270 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4faffd640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4fc0d07f0 0x7fc4fc1555c0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00c370 tx=0x7fc4ec00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4e37fe640 1 -- 172.21.15.37:0/601374628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec0052d0 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4e37fe640 1 -- 172.21.15.37:0/601374628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ec005470 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc5013ac640 1 -- 172.21.15.37:0/601374628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4fc15e550 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc5013ac640 1 -- 172.21.15.37:0/601374628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4fc15eb10 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4e37fe640 1 -- 172.21.15.37:0/601374628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ec007260 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc5013ac640 1 -- 172.21.15.37:0/601374628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4dc0052a0 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4e37fe640 1 -- 172.21.15.37:0/601374628 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc4ec007400 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4e37fe640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f90 0x7fc4e8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.053+0000 7fc4e37fe640 1 -- 172.21.15.37:0/601374628 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fc4ec0846e0 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.057+0000 7fc4fa7fc640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f90 0x7fc4e8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.057+0000 7fc4fa7fc640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f90 0x7fc4e8069450 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00af000 tx=0x7fc4f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.061+0000 7fc4e37fe640 1 -- 172.21.15.37:0/601374628 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc4ec055420 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.217+0000 7fc5013ac640 1 -- 172.21.15.37:0/601374628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4dc005860 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc4e37fe640 1 -- 172.21.15.37:0/601374628 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4ec054d20 con 0x7fc4fc0d07f0 2024-04-25T09:04:26.208 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-25T09:04:26.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 -- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f90 msgr2=0x7fc4e8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f90 0x7fc4e8069450 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fc4f00af000 tx=0x7fc4f0009040 comp rx=0 tx=0).stop 2024-04-25T09:04:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 -- 172.21.15.37:0/601374628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4fc0d07f0 msgr2=0x7fc4fc1555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4fc0d07f0 0x7fc4fc1555c0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec00c370 tx=0x7fc4ec00c830 comp rx=0 tx=0).stop 2024-04-25T09:04:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 -- 172.21.15.37:0/601374628 shutdown_connections 2024-04-25T09:04:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc4e8066f90 0x7fc4e8069450 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc4fc15da80 0x7fc4fc0b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc4fc0d1cd0 0x7fc4fc0b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 --2- 172.21.15.37:0/601374628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4fc0d07f0 0x7fc4fc1555c0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 -- 172.21.15.37:0/601374628 >> 172.21.15.37:0/601374628 conn(0x7fc4fc08b000 msgr2=0x7fc4fc0b02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 -- 172.21.15.37:0/601374628 shutdown_connections 2024-04-25T09:04:26.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.221+0000 7fc5013ac640 1 -- 172.21.15.37:0/601374628 wait complete. 2024-04-25T09:04:26.227 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d14709640 1 Processor -- start 2024-04-25T09:04:26.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d14709640 1 -- start start 2024-04-25T09:04:26.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d14709640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c0b3cf0 0x7f1d0c0cf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d14709640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0c0cbf10 con 0x7f1d0c0cfad0 2024-04-25T09:04:26.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d14709640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d0c0cc1f0 con 0x7f1d0c15e6c0 2024-04-25T09:04:26.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d14709640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0cff10 con 0x7f1d0c0b3cf0 2024-04-25T09:04:26.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d11ca4640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f1d0c0cfad0 0x7f1d0c15dfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52014/0 (socket says 172.21.15.37:52014) 2024-04-25T09:04:26.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d11ca4640 1 -- 172.21.15.37:0/3684748483 learned_addr learned my addr 172.21.15.37:0/3684748483 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:26.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d124a5640 1 --2- 172.21.15.37:0/3684748483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c0b3cf0 0x7f1d0c0cf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901190641 0 0) 0x7f1d0c0cbf10 con 0x7f1d0c0cfad0 2024-04-25T09:04:26.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cf8003530 con 0x7f1d0c0cfad0 2024-04-25T09:04:26.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2283126425 0 0) 0x7f1cf8003530 con 0x7f1d0c0cfad0 2024-04-25T09:04:26.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c0b3cf0 msgr2=0x7f1d0c0cf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:26.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 --2- 172.21.15.37:0/3684748483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c0b3cf0 0x7f1d0c0cf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 >> v1:172.21.15.203:6789/0 conn(0x7f1d0c15e6c0 legacy=0x7f1d0c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c0cc4d0 con 0x7f1d0c0cfad0 2024-04-25T09:04:26.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f1cfc001680 con 0x7f1d0c0cfad0 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 >> v1:172.21.15.37:6789/0 conn(0x7f1d0c0cfad0 legacy=0x7f1d0c15dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 --2- 172.21.15.37:0/3684748483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf8003a60 0x7f1cf8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 --2- 172.21.15.37:0/3684748483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c0b3cf0 0x7f1cf8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 --2- 172.21.15.37:0/3684748483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cf8004d50 0x7f1cf80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0cff10 con 0x7f1cf8004d50 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf800ba80 con 0x7f1cf8003a60 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf800bc80 con 0x7f1d0c0b3cf0 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d12ca6640 1 --2- 172.21.15.37:0/3684748483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cf8004d50 0x7f1cf80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d11ca4640 1 --2- 172.21.15.37:0/3684748483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c0b3cf0 0x7f1cf8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d12ca6640 1 -- 172.21.15.37:0/3684748483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c0b3cf0 msgr2=0x7f1cf8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d12ca6640 1 --2- 172.21.15.37:0/3684748483 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c0b3cf0 0x7f1cf8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d12ca6640 1 -- 172.21.15.37:0/3684748483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf8003a60 msgr2=0x7f1cf8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d12ca6640 1 --2- 172.21.15.37:0/3684748483 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf8003a60 0x7f1cf8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.369+0000 7f1d12ca6640 1 -- 172.21.15.37:0/3684748483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf8003530 con 0x7f1cf8004d50 2024-04-25T09:04:26.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d12ca6640 1 --2- 172.21.15.37:0/3684748483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cf8004d50 0x7f1cf80051d0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f1d08007fd0 tx=0x7f1d0801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655283b68342266e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d080b1070 con 0x7f1cf8004d50 2024-04-25T09:04:26.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d080ac640 con 0x7f1cf8004d50 2024-04-25T09:04:26.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d114a3640 1 -- 172.21.15.37:0/3684748483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d08001460 con 0x7f1cf8004d50 2024-04-25T09:04:26.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d14709640 1 -- 172.21.15.37:0/3684748483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cf8004d50 msgr2=0x7f1cf80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d14709640 1 --2- 172.21.15.37:0/3684748483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1cf8004d50 0x7f1cf80051d0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f1d08007fd0 tx=0x7f1d0801c4e0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d12ca6640 1 -- 172.21.15.37:0/3684748483 reap_dead start 2024-04-25T09:04:26.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d14709640 1 -- 172.21.15.37:0/3684748483 shutdown_connections 2024-04-25T09:04:26.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d14709640 1 -- 172.21.15.37:0/3684748483 >> 172.21.15.37:0/3684748483 conn(0x7f1d0c06ae60 msgr2=0x7f1d0c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:26.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d14709640 1 -- 172.21.15.37:0/3684748483 shutdown_connections 2024-04-25T09:04:26.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d14709640 1 -- 172.21.15.37:0/3684748483 wait complete. 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d14709640 1 Processor -- start 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.373+0000 7f1d14709640 1 -- start start 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d14709640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf8004d50 0x7f1d0c155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d14709640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c15c7b0 0x7f1d0c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d14709640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d0c071f10 0x7f1d0c072390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d14709640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0ac170 con 0x7f1d0c071f10 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d14709640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c15ead0 con 0x7f1cf8004d50 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d14709640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0b40d0 con 0x7f1d0c15c7b0 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d12ca6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d0c071f10 0x7f1d0c072390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d12ca6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d0c071f10 0x7f1d0c072390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51558/0 (socket says 172.21.15.37:51558) 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d12ca6640 1 -- 172.21.15.37:0/3468593602 learned_addr learned my addr 172.21.15.37:0/3468593602 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d11ca4640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c15c7b0 0x7f1d0c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d124a5640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf8004d50 0x7f1d0c155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d12ca6640 1 -- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c15c7b0 msgr2=0x7f1d0c15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d12ca6640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c15c7b0 0x7f1d0c15b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d12ca6640 1 -- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf8004d50 msgr2=0x7f1d0c155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d12ca6640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf8004d50 0x7f1d0c155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d12ca6640 1 -- 172.21.15.37:0/3468593602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c15bba0 con 0x7f1d0c071f10 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d11ca4640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c15c7b0 0x7f1d0c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d12ca6640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d0c071f10 0x7f1d0c072390 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f1d080af040 tx=0x7f1cf8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d02ffd640 1 -- 172.21.15.37:0/3468593602 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d080bd070 con 0x7f1d0c071f10 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d02ffd640 1 -- 172.21.15.37:0/3468593602 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d080ace50 con 0x7f1d0c071f10 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d14709640 1 -- 172.21.15.37:0/3468593602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0c15be80 con 0x7f1d0c071f10 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d02ffd640 1 -- 172.21.15.37:0/3468593602 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d080b1040 con 0x7f1d0c071f10 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d14709640 1 -- 172.21.15.37:0/3468593602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0c0af0e0 con 0x7f1d0c071f10 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d14709640 1 -- 172.21.15.37:0/3468593602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0c04efa0 con 0x7f1d0c071f10 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d02ffd640 1 -- 172.21.15.37:0/3468593602 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1d080c0070 con 0x7f1d0c071f10 2024-04-25T09:04:26.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d02ffd640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1cf0066f90 0x7f1cf0069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.377+0000 7f1d02ffd640 1 -- 172.21.15.37:0/3468593602 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f1d0811e550 con 0x7f1d0c071f10 2024-04-25T09:04:26.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.381+0000 7f1d124a5640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1cf0066f90 0x7f1cf0069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.381+0000 7f1d124a5640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1cf0066f90 0x7f1cf0069450 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009a40 tx=0x7f1d0400d040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.381+0000 7f1d02ffd640 1 -- 172.21.15.37:0/3468593602 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1d080161f0 con 0x7f1d0c071f10 2024-04-25T09:04:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.541+0000 7f1d14709640 1 -- 172.21.15.37:0/3468593602 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d0c06eb90 con 0x7f1d0c071f10 2024-04-25T09:04:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.541+0000 7f1d02ffd640 1 -- 172.21.15.37:0/3468593602 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d080eecc0 con 0x7f1d0c071f10 2024-04-25T09:04:26.532 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-25T09:04:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 -- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1cf0066f90 msgr2=0x7f1cf0069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1cf0066f90 0x7f1cf0069450 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009a40 tx=0x7f1d0400d040 comp rx=0 tx=0).stop 2024-04-25T09:04:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 -- 172.21.15.37:0/3468593602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d0c071f10 msgr2=0x7f1d0c072390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d0c071f10 0x7f1d0c072390 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f1d080af040 tx=0x7f1cf8005840 comp rx=0 tx=0).stop 2024-04-25T09:04:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 -- 172.21.15.37:0/3468593602 shutdown_connections 2024-04-25T09:04:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1cf0066f90 0x7f1cf0069450 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d0c071f10 0x7f1d0c072390 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1d0c15c7b0 0x7f1d0c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 --2- 172.21.15.37:0/3468593602 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1cf8004d50 0x7f1d0c155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 -- 172.21.15.37:0/3468593602 >> 172.21.15.37:0/3468593602 conn(0x7f1d0c06ae60 msgr2=0x7f1d0c15f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 -- 172.21.15.37:0/3468593602 shutdown_connections 2024-04-25T09:04:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.545+0000 7f1d14709640 1 -- 172.21.15.37:0/3468593602 wait complete. 2024-04-25T09:04:26.550 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:26.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.677+0000 7fd0b3e69640 1 Processor -- start 2024-04-25T09:04:26.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.677+0000 7fd0b3e69640 1 -- start start 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b3e69640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac15da80 0x7fd0ac0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b3e69640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd0ac0d1cd0 0x7fd0ac0d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b3e69640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0ac06be60 con 0x7fd0ac0d0620 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b3e69640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0ac06c440 con 0x7fd0ac0d1cd0 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b3e69640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac15de60 con 0x7fd0ac15da80 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b1404640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd0ac0d1cd0 0x7fd0ac0d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b2406640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fd0ac0d0620 0x7fd0ac0ce3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52020/0 (socket says 172.21.15.37:52020) 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b2406640 1 -- 172.21.15.37:0/632959147 learned_addr learned my addr 172.21.15.37:0/632959147 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b1c05640 1 --2- 172.21.15.37:0/632959147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac15da80 0x7fd0ac0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b1404640 1 -- 172.21.15.37:0/632959147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac15da80 msgr2=0x7fd0ac0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b1404640 1 --2- 172.21.15.37:0/632959147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac15da80 0x7fd0ac0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b1404640 1 -- 172.21.15.37:0/632959147 >> v1:172.21.15.37:6789/0 conn(0x7fd0ac0d0620 legacy=0x7fd0ac0ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b0c03640 1 -- 172.21.15.37:0/632959147 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996776655 0 0) 0x7fd0ac06be60 con 0x7fd0ac0d0620 2024-04-25T09:04:26.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b1404640 1 -- 172.21.15.37:0/632959147 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac0d2450 con 0x7fd0ac0d1cd0 2024-04-25T09:04:26.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b1404640 1 --2- 172.21.15.37:0/632959147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ac0d1cd0 0x7fd0ac0d00e0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0097a0 tx=0x7fd09c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734bb2bd105a2024 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b0c03640 1 -- 172.21.15.37:0/632959147 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0b2070 con 0x7fd0ac0d1cd0 2024-04-25T09:04:26.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b0c03640 1 -- 172.21.15.37:0/632959147 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c0ad640 con 0x7fd0ac0d1cd0 2024-04-25T09:04:26.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.681+0000 7fd0b0c03640 1 -- 172.21.15.37:0/632959147 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c0ada10 con 0x7fd0ac0d1cd0 2024-04-25T09:04:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 -- 172.21.15.37:0/632959147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ac0d1cd0 msgr2=0x7fd0ac0d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 --2- 172.21.15.37:0/632959147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ac0d1cd0 0x7fd0ac0d00e0 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7fd09c0097a0 tx=0x7fd09c01c480 comp rx=0 tx=0).stop 2024-04-25T09:04:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 -- 172.21.15.37:0/632959147 shutdown_connections 2024-04-25T09:04:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 --2- 172.21.15.37:0/632959147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ac0d1cd0 0x7fd0ac0d00e0 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 --2- 172.21.15.37:0/632959147 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac15da80 0x7fd0ac0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 -- 172.21.15.37:0/632959147 >> 172.21.15.37:0/632959147 conn(0x7fd0ac08b000 msgr2=0x7fd0ac0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 -- 172.21.15.37:0/632959147 shutdown_connections 2024-04-25T09:04:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 -- 172.21.15.37:0/632959147 wait complete. 2024-04-25T09:04:26.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 Processor -- start 2024-04-25T09:04:26.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 -- start start 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0d0620 0x7fd0ac153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d1cd0 0x7fd0ac15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ac15da80 0x7fd0ac156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac16a730 con 0x7fd0ac15da80 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac16a5b0 con 0x7fd0ac0d1cd0 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0ac16a8b0 con 0x7fd0ac0d0620 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b2406640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ac15da80 0x7fd0ac156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b2406640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ac15da80 0x7fd0ac156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51580/0 (socket says 172.21.15.37:51580) 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b2406640 1 -- 172.21.15.37:0/3541576739 learned_addr learned my addr 172.21.15.37:0/3541576739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b1c05640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0d0620 0x7fd0ac153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b1404640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d1cd0 0x7fd0ac15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b2406640 1 -- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0d0620 msgr2=0x7fd0ac153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b2406640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0d0620 0x7fd0ac153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b2406640 1 -- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d1cd0 msgr2=0x7fd0ac15a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b2406640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d1cd0 0x7fd0ac15a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b2406640 1 -- 172.21.15.37:0/3541576739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0ac158fa0 con 0x7fd0ac15da80 2024-04-25T09:04:26.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b1404640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d1cd0 0x7fd0ac15a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:26.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b2406640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ac15da80 0x7fd0ac156100 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400c7a0 tx=0x7fd0a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0a27fc640 1 -- 172.21.15.37:0/3541576739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a4018070 con 0x7fd0ac15da80 2024-04-25T09:04:26.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0a27fc640 1 -- 172.21.15.37:0/3541576739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0a400a640 con 0x7fd0ac15da80 2024-04-25T09:04:26.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 -- 172.21.15.37:0/3541576739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0ac1592e0 con 0x7fd0ac15da80 2024-04-25T09:04:26.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0a27fc640 1 -- 172.21.15.37:0/3541576739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0a400aab0 con 0x7fd0ac15da80 2024-04-25T09:04:26.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.685+0000 7fd0b3e69640 1 -- 172.21.15.37:0/3541576739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0ac159b20 con 0x7fd0ac15da80 2024-04-25T09:04:26.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.689+0000 7fd0b3e69640 1 -- 172.21.15.37:0/3541576739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0ac04efa0 con 0x7fd0ac15da80 2024-04-25T09:04:26.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.689+0000 7fd0a27fc640 1 -- 172.21.15.37:0/3541576739 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd0a401b070 con 0x7fd0ac15da80 2024-04-25T09:04:26.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.689+0000 7fd0a27fc640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd090066f60 0x7fd090069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.689+0000 7fd0a27fc640 1 -- 172.21.15.37:0/3541576739 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd0a40848c0 con 0x7fd0ac15da80 2024-04-25T09:04:26.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.689+0000 7fd0b1c05640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd090066f60 0x7fd090069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.689+0000 7fd0b1c05640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd090066f60 0x7fd090069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800b930 tx=0x7fd0a8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.693+0000 7fd0a27fc640 1 -- 172.21.15.37:0/3541576739 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd0ac1592e0 con 0x7fd0ac15da80 2024-04-25T09:04:26.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.845+0000 7fd0b3e69640 1 -- 172.21.15.37:0/3541576739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0ac0d4140 con 0x7fd0ac15da80 2024-04-25T09:04:26.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.845+0000 7fd0a27fc640 1 -- 172.21.15.37:0/3541576739 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0a4055690 con 0x7fd0ac15da80 2024-04-25T09:04:26.834 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-25T09:04:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.845+0000 7fd0b3e69640 1 -- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd090066f60 msgr2=0x7fd090069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.849+0000 7fd0b3e69640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd090066f60 0x7fd090069420 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd0a800b930 tx=0x7fd0a8008040 comp rx=0 tx=0).stop 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.849+0000 7fd0b3e69640 1 -- 172.21.15.37:0/3541576739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ac15da80 msgr2=0x7fd0ac156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.849+0000 7fd0b3e69640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ac15da80 0x7fd0ac156100 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fd0a400c7a0 tx=0x7fd0a400cc60 comp rx=0 tx=0).stop 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.849+0000 7fd0b3e69640 1 -- 172.21.15.37:0/3541576739 shutdown_connections 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.849+0000 7fd0b3e69640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd090066f60 0x7fd090069420 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.849+0000 7fd0b3e69640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd0ac15da80 0x7fd0ac156100 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.849+0000 7fd0b3e69640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd0ac0d1cd0 0x7fd0ac15a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.849+0000 7fd0b3e69640 1 --2- 172.21.15.37:0/3541576739 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd0ac0d0620 0x7fd0ac153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.849+0000 7fd0b3e69640 1 -- 172.21.15.37:0/3541576739 >> 172.21.15.37:0/3541576739 conn(0x7fd0ac08b000 msgr2=0x7fd0ac160840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.849+0000 7fd0b3e69640 1 -- 172.21.15.37:0/3541576739 shutdown_connections 2024-04-25T09:04:26.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.849+0000 7fd0b3e69640 1 -- 172.21.15.37:0/3541576739 wait complete. 2024-04-25T09:04:26.852 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:26.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.973+0000 7f7ce1695640 1 Processor -- start 2024-04-25T09:04:26.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.973+0000 7f7ce1695640 1 -- start start 2024-04-25T09:04:26.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7ce1695640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7cdc15da80 0x7f7cdc0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7ce1695640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cdc0d07f0 0x7f7cdc06c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7ce1695640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cdc06be60 con 0x7f7cdc0d1cd0 2024-04-25T09:04:26.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7ce1695640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cdc06c440 con 0x7f7cdc15da80 2024-04-25T09:04:26.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7ce1695640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc15de60 con 0x7f7cdc0d07f0 2024-04-25T09:04:26.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7cdaffd640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f7cdc15da80 0x7f7cdc0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cdc0d07f0 0x7f7cdc06c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7cda7fc640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f7cdc0d1cd0 0x7f7cdc0d00e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:53576/0 (socket says 172.21.15.37:53576) 2024-04-25T09:04:26.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7cda7fc640 1 -- 172.21.15.37:0/1608098181 learned_addr learned my addr 172.21.15.37:0/1608098181 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:26.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/1608098181 >> v2:172.21.15.203:3300/0 conn(0x7f7cdc15da80 msgr2=0x7f7cdc0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:26.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/1608098181 >> v2:172.21.15.203:3300/0 conn(0x7f7cdc15da80 0x7f7cdc0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/1608098181 >> v1:172.21.15.203:6789/0 conn(0x7f7cdc0d1cd0 legacy=0x7f7cdc0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/1608098181 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc0d2450 con 0x7f7cdc0d07f0 2024-04-25T09:04:26.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.977+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/1608098181 >> v2:172.21.15.203:3300/0 conn(0x7f7cdc15da80 0x7f7cdc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:26.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/1608098181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cdc0d07f0 0x7f7cdc06c8d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009dc0 tx=0x7f7cd401c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be0e6398cd22e55e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cd9ffb640 1 -- 172.21.15.37:0/1608098181 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40b2070 con 0x7f7cdc0d07f0 2024-04-25T09:04:26.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cd9ffb640 1 -- 172.21.15.37:0/1608098181 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd40a5940 con 0x7f7cdc0d07f0 2024-04-25T09:04:26.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cd9ffb640 1 -- 172.21.15.37:0/1608098181 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40a5d10 con 0x7f7cdc0d07f0 2024-04-25T09:04:26.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 -- 172.21.15.37:0/1608098181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cdc0d07f0 msgr2=0x7f7cdc06c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 --2- 172.21.15.37:0/1608098181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cdc0d07f0 0x7f7cdc06c8d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009dc0 tx=0x7f7cd401c920 comp rx=0 tx=0).stop 2024-04-25T09:04:26.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 -- 172.21.15.37:0/1608098181 shutdown_connections 2024-04-25T09:04:26.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 --2- 172.21.15.37:0/1608098181 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cdc0d07f0 0x7f7cdc06c8d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 --2- 172.21.15.37:0/1608098181 >> v2:172.21.15.203:3300/0 conn(0x7f7cdc15da80 0x7f7cdc0d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 -- 172.21.15.37:0/1608098181 >> 172.21.15.37:0/1608098181 conn(0x7f7cdc08b000 msgr2=0x7f7cdc0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:26.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 -- 172.21.15.37:0/1608098181 shutdown_connections 2024-04-25T09:04:26.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 -- 172.21.15.37:0/1608098181 wait complete. 2024-04-25T09:04:26.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 Processor -- start 2024-04-25T09:04:26.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 -- start start 2024-04-25T09:04:26.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cdc0d07f0 0x7f7cdc156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cdc0d1cd0 0x7f7cdc0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc15da80 0x7f7cdc158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc06d7a0 con 0x7f7cdc15da80 2024-04-25T09:04:26.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc06d620 con 0x7f7cdc0d07f0 2024-04-25T09:04:26.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7ce1695640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cdc06d920 con 0x7f7cdc0d1cd0 2024-04-25T09:04:26.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc15da80 0x7f7cdc158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cdb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc15da80 0x7f7cdc158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51594/0 (socket says 172.21.15.37:51594) 2024-04-25T09:04:26.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/2257503752 learned_addr learned my addr 172.21.15.37:0/2257503752 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:26.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cda7fc640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cdc0d1cd0 0x7f7cdc0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cdc0d07f0 0x7f7cdc156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cdc0d1cd0 msgr2=0x7f7cdc0c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cdc0d1cd0 0x7f7cdc0c83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cdc0d07f0 msgr2=0x7f7cdc156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:26.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cdc0d07f0 0x7f7cdc156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.981+0000 7f7cdb7fe640 1 -- 172.21.15.37:0/2257503752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cdc0c8ca0 con 0x7f7cdc15da80 2024-04-25T09:04:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7cdb7fe640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc15da80 0x7f7cdc158ce0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40b0040 tx=0x7f7cd40196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7cc37fe640 1 -- 172.21.15.37:0/2257503752 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40b2050 con 0x7f7cdc15da80 2024-04-25T09:04:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7cc37fe640 1 -- 172.21.15.37:0/2257503752 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd40be070 con 0x7f7cdc15da80 2024-04-25T09:04:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7ce1695640 1 -- 172.21.15.37:0/2257503752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cdc0c8f80 con 0x7f7cdc15da80 2024-04-25T09:04:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7cc37fe640 1 -- 172.21.15.37:0/2257503752 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd40ada90 con 0x7f7cdc15da80 2024-04-25T09:04:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7ce1695640 1 -- 172.21.15.37:0/2257503752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cdc0c9540 con 0x7f7cdc15da80 2024-04-25T09:04:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7ce1695640 1 -- 172.21.15.37:0/2257503752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cdc04efa0 con 0x7f7cdc15da80 2024-04-25T09:04:26.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7cc37fe640 1 -- 172.21.15.37:0/2257503752 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f7cd40adc30 con 0x7f7cdc15da80 2024-04-25T09:04:26.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7cc37fe640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cc8066f60 0x7f7cc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7cc37fe640 1 -- 172.21.15.37:0/2257503752 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f7cd4120500 con 0x7f7cdc15da80 2024-04-25T09:04:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cc8066f60 0x7f7cc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.985+0000 7f7cdaffd640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cc8066f60 0x7f7cc8069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0d1a40 tx=0x7f7ccc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:26.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:26.989+0000 7f7cc37fe640 1 -- 172.21.15.37:0/2257503752 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f7cd4016540 con 0x7f7cdc15da80 2024-04-25T09:04:27.145 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:27.147 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:27.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.181+0000 7f7ce1695640 1 -- 172.21.15.37:0/2257503752 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cdc0d4270 con 0x7f7cdc15da80 2024-04-25T09:04:27.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7cc37fe640 1 -- 172.21.15.37:0/2257503752 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f7cd40ab340 con 0x7f7cdc15da80 2024-04-25T09:04:27.174 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:27.174 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:04:16.988456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/2361387213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2361387213},{"type":"v1","addr":"172.21.15.37:6835","nonce":2361387213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:04:27.174 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 110 2024-04-25T09:04:27.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 -- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cc8066f60 msgr2=0x7f7cc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cc8066f60 0x7f7cc8069420 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc0d1a40 tx=0x7f7ccc008040 comp rx=0 tx=0).stop 2024-04-25T09:04:27.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 -- 172.21.15.37:0/2257503752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc15da80 msgr2=0x7f7cdc158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc15da80 0x7f7cdc158ce0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f7cd40b0040 tx=0x7f7cd40196c0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 -- 172.21.15.37:0/2257503752 shutdown_connections 2024-04-25T09:04:27.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f7cc8066f60 0x7f7cc8069420 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7cdc15da80 0x7f7cdc158ce0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f7cdc0d1cd0 0x7f7cdc0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 --2- 172.21.15.37:0/2257503752 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f7cdc0d07f0 0x7f7cdc156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 -- 172.21.15.37:0/2257503752 >> 172.21.15.37:0/2257503752 conn(0x7f7cdc08b000 msgr2=0x7f7cdc0b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:27.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 -- 172.21.15.37:0/2257503752 shutdown_connections 2024-04-25T09:04:27.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.185+0000 7f7ce1695640 1 -- 172.21.15.37:0/2257503752 wait complete. 2024-04-25T09:04:27.190 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:27.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.369+0000 7f8dee3ff640 1 Processor -- start 2024-04-25T09:04:27.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.369+0000 7f8dee3ff640 1 -- start start 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8dee3ff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8de806fc50 0x7f8de806ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8dee3ff640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de806b710 con 0x7f8de806b1d0 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8dee3ff640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de81617b0 con 0x7f8de80cff50 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8dee3ff640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de8161560 con 0x7f8de806fc50 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8ddf7fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8de806b1d0 0x7f8de80cf840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52032/0 (socket says 172.21.15.37:52032) 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8de7fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8de806fc50 0x7f8de806ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/2541244309 learned_addr learned my addr 172.21.15.37:0/2541244309 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8de77fe640 1 -- 172.21.15.37:0/2541244309 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923254247 0 0) 0x7f8de806b710 con 0x7f8de806b1d0 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8de77fe640 1 -- 172.21.15.37:0/2541244309 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dd4003530 con 0x7f8de806b1d0 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8de7fff640 1 -- 172.21.15.37:0/2541244309 >> v1:172.21.15.203:6789/0 conn(0x7f8de80cff50 legacy=0x7f8de815d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8de7fff640 1 -- 172.21.15.37:0/2541244309 >> v1:172.21.15.37:6789/0 conn(0x7f8de806b1d0 legacy=0x7f8de80cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.373+0000 7f8de7fff640 1 -- 172.21.15.37:0/2541244309 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de8161a90 con 0x7f8de806fc50 2024-04-25T09:04:27.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8de7fff640 1 --2- 172.21.15.37:0/2541244309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de806fc50 0x7f8de806ac90 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80097c0 tx=0x7f8dd801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe24a0ac34d28e30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:27.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8de77fe640 1 -- 172.21.15.37:0/2541244309 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80b0070 con 0x7f8de806fc50 2024-04-25T09:04:27.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8de77fe640 1 -- 172.21.15.37:0/2541244309 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd80ab600 con 0x7f8de806fc50 2024-04-25T09:04:27.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8de77fe640 1 -- 172.21.15.37:0/2541244309 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80ab9d0 con 0x7f8de806fc50 2024-04-25T09:04:27.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2541244309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de806fc50 msgr2=0x7f8de806ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 --2- 172.21.15.37:0/2541244309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de806fc50 0x7f8de806ac90 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80097c0 tx=0x7f8dd801c460 comp rx=0 tx=0).stop 2024-04-25T09:04:27.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2541244309 shutdown_connections 2024-04-25T09:04:27.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 --2- 172.21.15.37:0/2541244309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de806fc50 0x7f8de806ac90 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2541244309 >> 172.21.15.37:0/2541244309 conn(0x7f8de8099a70 msgr2=0x7f8de80d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:27.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2541244309 shutdown_connections 2024-04-25T09:04:27.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2541244309 wait complete. 2024-04-25T09:04:27.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 Processor -- start 2024-04-25T09:04:27.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 -- start start 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de806b1d0 0x7f8de8154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de806fc50 0x7f8de815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80cff50 0x7f8de815c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de8162410 con 0x7f8de806fc50 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de8162290 con 0x7f8de806b1d0 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dee3ff640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de815dca0 con 0x7f8de80cff50 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8ddf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de806fc50 0x7f8de815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8ddf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de806fc50 0x7f8de815bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51626/0 (socket says 172.21.15.37:51626) 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/2935672292 learned_addr learned my addr 172.21.15.37:0/2935672292 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8de7fff640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de806b1d0 0x7f8de8154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8dec99c640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80cff50 0x7f8de815c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80cff50 msgr2=0x7f8de815c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8ddf7fe640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80cff50 0x7f8de815c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de806b1d0 msgr2=0x7f8de8154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8ddf7fe640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de806b1d0 0x7f8de8154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8ddf7fe640 1 -- 172.21.15.37:0/2935672292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de815c6e0 con 0x7f8de806fc50 2024-04-25T09:04:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.377+0000 7f8ddf7fe640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de806fc50 0x7f8de815bb60 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000c3c0 tx=0x7f8dd000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.381+0000 7f8de57fa640 1 -- 172.21.15.37:0/2935672292 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0017070 con 0x7f8de806fc50 2024-04-25T09:04:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.381+0000 7f8de57fa640 1 -- 172.21.15.37:0/2935672292 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd0005d10 con 0x7f8de806fc50 2024-04-25T09:04:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.381+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2935672292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de815c9c0 con 0x7f8de806fc50 2024-04-25T09:04:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.381+0000 7f8de57fa640 1 -- 172.21.15.37:0/2935672292 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0007310 con 0x7f8de806fc50 2024-04-25T09:04:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.381+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2935672292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de80b3610 con 0x7f8de806fc50 2024-04-25T09:04:27.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.381+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2935672292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de80b3120 con 0x7f8de806fc50 2024-04-25T09:04:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.381+0000 7f8de57fa640 1 -- 172.21.15.37:0/2935672292 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8dd001a070 con 0x7f8de806fc50 2024-04-25T09:04:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.381+0000 7f8de57fa640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.381+0000 7f8de57fa640 1 -- 172.21.15.37:0/2935672292 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f8dd0084610 con 0x7f8de806fc50 2024-04-25T09:04:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.381+0000 7f8de7fff640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.381+0000 7f8de7fff640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 0x7f8dc8069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80ad000 tx=0x7f8dd80a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:27.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.385+0000 7f8de57fa640 1 -- 172.21.15.37:0/2935672292 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8dd00554a0 con 0x7f8de806fc50 2024-04-25T09:04:27.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.537+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2935672292 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8de804efa0 con 0x7f8de806fc50 2024-04-25T09:04:27.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.541+0000 7f8de57fa640 1 -- 172.21.15.37:0/2935672292 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17175 (secure 0 0 0) 0x7f8dd0054da0 con 0x7f8de806fc50 2024-04-25T09:04:27.529 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:27.529 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":127,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:04:12.937853+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2159545482":"2024-04-26T09:04:12.937783+0000","172.21.15.37:6834/2159545482":"2024-04-26T09:04:12.937783+0000","172.21.15.37:6835/2748919477":"2024-04-26T09:03:58.978336+0000","172.21.15.37:6834/2748919477":"2024-04-26T09:03:58.978336+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 msgr2=0x7f8dc8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 0x7f8dc8069420 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80ad000 tx=0x7f8dd80a6000 comp rx=0 tx=0).stop 2024-04-25T09:04:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2935672292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de806fc50 msgr2=0x7f8de815bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de806fc50 0x7f8de815bb60 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000c3c0 tx=0x7f8dd000c880 comp rx=0 tx=0).stop 2024-04-25T09:04:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2935672292 shutdown_connections 2024-04-25T09:04:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8dc8066f60 0x7f8dc8069420 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8de80cff50 0x7f8de815c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8de806fc50 0x7f8de815bb60 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 --2- 172.21.15.37:0/2935672292 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8de806b1d0 0x7f8de8154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2935672292 >> 172.21.15.37:0/2935672292 conn(0x7f8de8099a70 msgr2=0x7f8de8073590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:27.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2935672292 shutdown_connections 2024-04-25T09:04:27.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.545+0000 7f8dee3ff640 1 -- 172.21.15.37:0/2935672292 wait complete. 2024-04-25T09:04:27.548 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:04:27.548 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:27.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.669+0000 7f7820a00640 1 Processor -- start 2024-04-25T09:04:27.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.669+0000 7f7820a00640 1 -- start start 2024-04-25T09:04:27.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f781c0b3f50 0x7f781c15ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f781c0b24a0 0x7f781c0b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f781c0b2090 con 0x7f781c15a700 2024-04-25T09:04:27.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f781c15f200 con 0x7f781c0b24a0 2024-04-25T09:04:27.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f781c0b43e0 con 0x7f781c0b3f50 2024-04-25T09:04:27.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781a59c640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f781c15a700 0x7f781c0cdef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52042/0 (socket says 172.21.15.37:52042) 2024-04-25T09:04:27.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781a59c640 1 -- 172.21.15.37:0/4043999030 learned_addr learned my addr 172.21.15.37:0/4043999030 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:27.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781ad9d640 1 --2- 172.21.15.37:0/4043999030 >> v2:172.21.15.37:3300/0 conn(0x7f781c0b24a0 0x7f781c0b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781959a640 1 -- 172.21.15.37:0/4043999030 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4031400405 0 0) 0x7f781c0b2090 con 0x7f781c15a700 2024-04-25T09:04:27.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7819d9b640 1 --2- 172.21.15.37:0/4043999030 >> v2:172.21.15.203:3300/0 conn(0x7f781c0b3f50 0x7f781c15ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781959a640 1 -- 172.21.15.37:0/4043999030 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7804003530 con 0x7f781c15a700 2024-04-25T09:04:27.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781ad9d640 1 -- 172.21.15.37:0/4043999030 >> v2:172.21.15.203:3300/0 conn(0x7f781c0b3f50 msgr2=0x7f781c15ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:27.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781ad9d640 1 --2- 172.21.15.37:0/4043999030 >> v2:172.21.15.203:3300/0 conn(0x7f781c0b3f50 0x7f781c15ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781ad9d640 1 -- 172.21.15.37:0/4043999030 >> v1:172.21.15.37:6789/0 conn(0x7f781c15a700 legacy=0x7f781c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781ad9d640 1 -- 172.21.15.37:0/4043999030 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c0b46c0 con 0x7f781c0b24a0 2024-04-25T09:04:27.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7819d9b640 1 --2- 172.21.15.37:0/4043999030 >> v2:172.21.15.203:3300/0 conn(0x7f781c0b3f50 0x7f781c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781ad9d640 1 --2- 172.21.15.37:0/4043999030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781c0b24a0 0x7f781c0b28f0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f781c0b3e20 tx=0x7f781401c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931650c6efce632 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781959a640 1 -- 172.21.15.37:0/4043999030 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78140b2070 con 0x7f781c0b24a0 2024-04-25T09:04:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781959a640 1 -- 172.21.15.37:0/4043999030 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78140a57c0 con 0x7f781c0b24a0 2024-04-25T09:04:27.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f781959a640 1 -- 172.21.15.37:0/4043999030 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78140a5b90 con 0x7f781c0b24a0 2024-04-25T09:04:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 -- 172.21.15.37:0/4043999030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781c0b24a0 msgr2=0x7f781c0b28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 --2- 172.21.15.37:0/4043999030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781c0b24a0 0x7f781c0b28f0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f781c0b3e20 tx=0x7f781401c810 comp rx=0 tx=0).stop 2024-04-25T09:04:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 -- 172.21.15.37:0/4043999030 shutdown_connections 2024-04-25T09:04:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 --2- 172.21.15.37:0/4043999030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781c0b24a0 0x7f781c0b28f0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 --2- 172.21.15.37:0/4043999030 >> v2:172.21.15.203:3300/0 conn(0x7f781c0b3f50 0x7f781c15ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 -- 172.21.15.37:0/4043999030 >> 172.21.15.37:0/4043999030 conn(0x7f781c0b11a0 msgr2=0x7f781c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 -- 172.21.15.37:0/4043999030 shutdown_connections 2024-04-25T09:04:27.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.673+0000 7f7820a00640 1 -- 172.21.15.37:0/4043999030 wait complete. 2024-04-25T09:04:27.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f7820a00640 1 Processor -- start 2024-04-25T09:04:27.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f7820a00640 1 -- start start 2024-04-25T09:04:27.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f7820a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781c0b24a0 0x7f781c15aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f7820a00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f781c15a700 0x7f781c1595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f7820a00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f781c1531c0 0x7f781c144d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f7820a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c0cd310 con 0x7f781c0b24a0 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f7820a00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f781c0ce600 con 0x7f781c1531c0 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f7820a00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c0cd490 con 0x7f781c15a700 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f781a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781c0b24a0 0x7f781c15aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f781a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781c0b24a0 0x7f781c15aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51650/0 (socket says 172.21.15.37:51650) 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f781a59c640 1 -- 172.21.15.37:0/1616487984 learned_addr learned my addr 172.21.15.37:0/1616487984 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f781a59c640 1 -- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f781c15a700 msgr2=0x7f781c1595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f781a59c640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f781c15a700 0x7f781c1595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f781a59c640 1 -- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f781c1531c0 msgr2=0x7f781c144d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f781a59c640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f781c1531c0 0x7f781c144d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f781a59c640 1 -- 172.21.15.37:0/1616487984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c157a90 con 0x7f781c0b24a0 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f781a59c640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781c0b24a0 0x7f781c15aed0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f780c00c460 tx=0x7f780c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f780affd640 1 -- 172.21.15.37:0/1616487984 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c017070 con 0x7f781c0b24a0 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f780affd640 1 -- 172.21.15.37:0/1616487984 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780c005d10 con 0x7f781c0b24a0 2024-04-25T09:04:27.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f7820a00640 1 -- 172.21.15.37:0/1616487984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f781c15d2f0 con 0x7f781c0b24a0 2024-04-25T09:04:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f780affd640 1 -- 172.21.15.37:0/1616487984 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780c0072b0 con 0x7f781c0b24a0 2024-04-25T09:04:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f7820a00640 1 -- 172.21.15.37:0/1616487984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f781c15db30 con 0x7f781c0b24a0 2024-04-25T09:04:27.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f7820a00640 1 -- 172.21.15.37:0/1616487984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f781c157c60 con 0x7f781c0b24a0 2024-04-25T09:04:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.677+0000 7f780affd640 1 -- 172.21.15.37:0/1616487984 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f780c01a070 con 0x7f781c0b24a0 2024-04-25T09:04:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.681+0000 7f780affd640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77fc066f60 0x7f77fc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.681+0000 7f780affd640 1 -- 172.21.15.37:0/1616487984 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f780c0845f0 con 0x7f781c0b24a0 2024-04-25T09:04:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.681+0000 7f7819d9b640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77fc066f60 0x7f77fc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:27.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.681+0000 7f7819d9b640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77fc066f60 0x7f77fc069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f781c15a0c0 tx=0x7f7810008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:27.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.685+0000 7f780affd640 1 -- 172.21.15.37:0/1616487984 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f781c15d2f0 con 0x7f781c0b24a0 2024-04-25T09:04:27.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.873+0000 7f7820a00640 1 -- 172.21.15.37:0/1616487984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f781c04efa0 con 0x7f781c0b24a0 2024-04-25T09:04:27.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.873+0000 7f780affd640 1 -- 172.21.15.37:0/1616487984 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4951 (secure 0 0 0) 0x7f780c0553c0 con 0x7f781c0b24a0 2024-04-25T09:04:27.862 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:27.862 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:04:16.988456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/2361387213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2361387213},{"type":"v1","addr":"172.21.15.37:6835","nonce":2361387213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:04:27.862 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 110 2024-04-25T09:04:27.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.873+0000 7f7820a00640 1 -- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77fc066f60 msgr2=0x7f77fc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.873+0000 7f7820a00640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77fc066f60 0x7f77fc069420 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f781c15a0c0 tx=0x7f7810008040 comp rx=0 tx=0).stop 2024-04-25T09:04:27.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.873+0000 7f7820a00640 1 -- 172.21.15.37:0/1616487984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781c0b24a0 msgr2=0x7f781c15aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:27.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.873+0000 7f7820a00640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781c0b24a0 0x7f781c15aed0 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f780c00c460 tx=0x7f780c00c920 comp rx=0 tx=0).stop 2024-04-25T09:04:27.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.877+0000 7f7820a00640 1 -- 172.21.15.37:0/1616487984 shutdown_connections 2024-04-25T09:04:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.877+0000 7f7820a00640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f77fc066f60 0x7f77fc069420 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.877+0000 7f7820a00640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f781c1531c0 0x7f781c144d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.877+0000 7f7820a00640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f781c15a700 0x7f781c1595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.877+0000 7f7820a00640 1 --2- 172.21.15.37:0/1616487984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f781c0b24a0 0x7f781c15aed0 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.877+0000 7f7820a00640 1 -- 172.21.15.37:0/1616487984 >> 172.21.15.37:0/1616487984 conn(0x7f781c0b11a0 msgr2=0x7f781c164910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.877+0000 7f7820a00640 1 -- 172.21.15.37:0/1616487984 shutdown_connections 2024-04-25T09:04:27.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:27.877+0000 7f7820a00640 1 -- 172.21.15.37:0/1616487984 wait complete. 2024-04-25T09:04:27.880 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:28.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4f64b640 1 Processor -- start 2024-04-25T09:04:28.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4f64b640 1 -- start start 2024-04-25T09:04:28.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4f64b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fec480ce510 0x7fec480ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4f64b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fec48144ca0 0x7fec4806e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4f64b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec4806ec60 0x7fec48142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4f64b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec480cc2b0 con 0x7fec480ce510 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4f64b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4806e290 con 0x7fec48144ca0 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4f64b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec480cee30 con 0x7fec4806ec60 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4d3e7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fec480ce510 0x7fec480ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4d3e7640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fec480ce510 0x7fec480ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51660/0 (socket says 172.21.15.37:51660) 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4d3e7640 1 -- 172.21.15.37:0/1851215339 learned_addr learned my addr 172.21.15.37:0/1851215339 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4dbe8640 1 --2- 172.21.15.37:0/1851215339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec4806ec60 0x7fec48142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4cbe6640 1 --2- 172.21.15.37:0/1851215339 >> v2:172.21.15.203:3300/0 conn(0x7fec48144ca0 0x7fec4806e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4d3e7640 1 -- 172.21.15.37:0/1851215339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec4806ec60 msgr2=0x7fec48142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4d3e7640 1 --2- 172.21.15.37:0/1851215339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec4806ec60 0x7fec48142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4d3e7640 1 -- 172.21.15.37:0/1851215339 >> v2:172.21.15.203:3300/0 conn(0x7fec48144ca0 msgr2=0x7fec4806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4d3e7640 1 --2- 172.21.15.37:0/1851215339 >> v2:172.21.15.203:3300/0 conn(0x7fec48144ca0 0x7fec4806e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.013+0000 7fec4d3e7640 1 -- 172.21.15.37:0/1851215339 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec480ccc70 con 0x7fec480ce510 2024-04-25T09:04:28.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4d3e7640 1 --2- 172.21.15.37:0/1851215339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec480ce510 0x7fec480ce8f0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fec48070fe0 tx=0x7fec4001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=883a0eef166ef55d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec3ffff640 1 -- 172.21.15.37:0/1851215339 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec400b5070 con 0x7fec480ce510 2024-04-25T09:04:28.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec3ffff640 1 -- 172.21.15.37:0/1851215339 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec400a89d0 con 0x7fec480ce510 2024-04-25T09:04:28.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec3ffff640 1 -- 172.21.15.37:0/1851215339 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec400a8da0 con 0x7fec480ce510 2024-04-25T09:04:28.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4f64b640 1 -- 172.21.15.37:0/1851215339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec480ce510 msgr2=0x7fec480ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4f64b640 1 --2- 172.21.15.37:0/1851215339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec480ce510 0x7fec480ce8f0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7fec48070fe0 tx=0x7fec4001ed10 comp rx=0 tx=0).stop 2024-04-25T09:04:28.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4f64b640 1 -- 172.21.15.37:0/1851215339 shutdown_connections 2024-04-25T09:04:28.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4f64b640 1 --2- 172.21.15.37:0/1851215339 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec4806ec60 0x7fec48142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4f64b640 1 --2- 172.21.15.37:0/1851215339 >> v2:172.21.15.203:3300/0 conn(0x7fec48144ca0 0x7fec4806e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4f64b640 1 --2- 172.21.15.37:0/1851215339 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec480ce510 0x7fec480ce8f0 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4f64b640 1 -- 172.21.15.37:0/1851215339 >> 172.21.15.37:0/1851215339 conn(0x7fec480b11a0 msgr2=0x7fec480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:28.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4f64b640 1 -- 172.21.15.37:0/1851215339 shutdown_connections 2024-04-25T09:04:28.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4f64b640 1 -- 172.21.15.37:0/1851215339 wait complete. 2024-04-25T09:04:28.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4f64b640 1 Processor -- start 2024-04-25T09:04:28.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.017+0000 7fec4f64b640 1 -- start start 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4f64b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4806ec60 0x7fec480cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4f64b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec480ce510 0x7fec48133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4f64b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec48144ca0 0x7fec481e2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4f64b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec480cc900 con 0x7fec4806ec60 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4f64b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4806df20 con 0x7fec480ce510 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4f64b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec481451f0 con 0x7fec48144ca0 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4d3e7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4806ec60 0x7fec480cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4d3e7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4806ec60 0x7fec480cac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51666/0 (socket says 172.21.15.37:51666) 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4d3e7640 1 -- 172.21.15.37:0/357066444 learned_addr learned my addr 172.21.15.37:0/357066444 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4cbe6640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec480ce510 0x7fec48133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4d3e7640 1 -- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec48144ca0 msgr2=0x7fec481e2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4d3e7640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec48144ca0 0x7fec481e2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4d3e7640 1 -- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec480ce510 msgr2=0x7fec48133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4d3e7640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec480ce510 0x7fec48133ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4d3e7640 1 -- 172.21.15.37:0/357066444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec48134220 con 0x7fec4806ec60 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4d3e7640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4806ec60 0x7fec480cac20 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fec400b3040 tx=0x7fec4001bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec3dffb640 1 -- 172.21.15.37:0/357066444 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec400b5070 con 0x7fec4806ec60 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec3dffb640 1 -- 172.21.15.37:0/357066444 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec400b0a70 con 0x7fec4806ec60 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4f64b640 1 -- 172.21.15.37:0/357066444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec481e2220 con 0x7fec4806ec60 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec3dffb640 1 -- 172.21.15.37:0/357066444 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec400b0e60 con 0x7fec4806ec60 2024-04-25T09:04:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4f64b640 1 -- 172.21.15.37:0/357066444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec481e8d70 con 0x7fec4806ec60 2024-04-25T09:04:28.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4f64b640 1 -- 172.21.15.37:0/357066444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec481e88c0 con 0x7fec4806ec60 2024-04-25T09:04:28.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec3dffb640 1 -- 172.21.15.37:0/357066444 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fec400c2060 con 0x7fec4806ec60 2024-04-25T09:04:28.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec3dffb640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec2c066f60 0x7fec2c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec3dffb640 1 -- 172.21.15.37:0/357066444 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fec401223f0 con 0x7fec4806ec60 2024-04-25T09:04:28.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4cbe6640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec2c066f60 0x7fec2c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.021+0000 7fec4cbe6640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec2c066f60 0x7fec2c069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fec481f6b20 tx=0x7fec38008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.025+0000 7fec3dffb640 1 -- 172.21.15.37:0/357066444 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fec40018870 con 0x7fec4806ec60 2024-04-25T09:04:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.221+0000 7fec4f64b640 1 -- 172.21.15.37:0/357066444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec4804efa0 con 0x7fec4806ec60 2024-04-25T09:04:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec3dffb640 1 -- 172.21.15.37:0/357066444 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4951 (secure 0 0 0) 0x7fec400c2350 con 0x7fec4806ec60 2024-04-25T09:04:28.213 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:28.213 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":110,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:32.494159+0000","modified":"2024-04-25T09:04:16.988456+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":107,"state":"up:active","state_seq":4,"addr":"172.21.15.37:6835/2361387213","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2361387213},{"type":"v1","addr":"172.21.15.37:6835","nonce":2361387213}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:04:28.214 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 110 2024-04-25T09:04:28.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 -- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec2c066f60 msgr2=0x7fec2c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec2c066f60 0x7fec2c069420 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fec481f6b20 tx=0x7fec38008040 comp rx=0 tx=0).stop 2024-04-25T09:04:28.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 -- 172.21.15.37:0/357066444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4806ec60 msgr2=0x7fec480cac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4806ec60 0x7fec480cac20 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7fec400b3040 tx=0x7fec4001bab0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 -- 172.21.15.37:0/357066444 shutdown_connections 2024-04-25T09:04:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fec2c066f60 0x7fec2c069420 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fec48144ca0 0x7fec481e2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fec480ce510 0x7fec48133ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 --2- 172.21.15.37:0/357066444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec4806ec60 0x7fec480cac20 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 -- 172.21.15.37:0/357066444 >> 172.21.15.37:0/357066444 conn(0x7fec480b11a0 msgr2=0x7fec480b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 -- 172.21.15.37:0/357066444 shutdown_connections 2024-04-25T09:04:28.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.225+0000 7fec4f64b640 1 -- 172.21.15.37:0/357066444 wait complete. 2024-04-25T09:04:28.233 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:28.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd923319640 1 Processor -- start 2024-04-25T09:04:28.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd923319640 1 -- start start 2024-04-25T09:04:28.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd923319640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd91c0ce510 0x7fd91c0ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd923319640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd91c144ca0 0x7fd91c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd923319640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91c0cee30 con 0x7fd91c06ec60 2024-04-25T09:04:28.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd923319640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91c0cc2b0 con 0x7fd91c0ce510 2024-04-25T09:04:28.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd923319640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c06e290 con 0x7fd91c144ca0 2024-04-25T09:04:28.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd9210b5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd91c0ce510 0x7fd91c0ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd9210b5640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fd91c0ce510 0x7fd91c0ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51676/0 (socket says 172.21.15.37:51676) 2024-04-25T09:04:28.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd9210b5640 1 -- 172.21.15.37:0/2695163438 learned_addr learned my addr 172.21.15.37:0/2695163438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:28.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd9210b5640 1 -- 172.21.15.37:0/2695163438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd91c144ca0 msgr2=0x7fd91c06e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:28.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd9210b5640 1 --2- 172.21.15.37:0/2695163438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd91c144ca0 0x7fd91c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd9210b5640 1 -- 172.21.15.37:0/2695163438 >> v1:172.21.15.37:6789/0 conn(0x7fd91c06ec60 legacy=0x7fd91c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.357+0000 7fd9210b5640 1 -- 172.21.15.37:0/2695163438 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c0ccc70 con 0x7fd91c0ce510 2024-04-25T09:04:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd9210b5640 1 --2- 172.21.15.37:0/2695163438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd91c0ce510 0x7fd91c0ce8f0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fd918009870 tx=0x7fd91801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a616ee17be7990d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd913fff640 1 -- 172.21.15.37:0/2695163438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9180b5070 con 0x7fd91c0ce510 2024-04-25T09:04:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd913fff640 1 -- 172.21.15.37:0/2695163438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9180a8a50 con 0x7fd91c0ce510 2024-04-25T09:04:28.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd913fff640 1 -- 172.21.15.37:0/2695163438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9180a8e20 con 0x7fd91c0ce510 2024-04-25T09:04:28.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 -- 172.21.15.37:0/2695163438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd91c0ce510 msgr2=0x7fd91c0ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 --2- 172.21.15.37:0/2695163438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd91c0ce510 0x7fd91c0ce8f0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fd918009870 tx=0x7fd91801ed50 comp rx=0 tx=0).stop 2024-04-25T09:04:28.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 -- 172.21.15.37:0/2695163438 shutdown_connections 2024-04-25T09:04:28.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 --2- 172.21.15.37:0/2695163438 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd91c144ca0 0x7fd91c06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 --2- 172.21.15.37:0/2695163438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd91c0ce510 0x7fd91c0ce8f0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 -- 172.21.15.37:0/2695163438 >> 172.21.15.37:0/2695163438 conn(0x7fd91c0b11a0 msgr2=0x7fd91c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 -- 172.21.15.37:0/2695163438 shutdown_connections 2024-04-25T09:04:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 -- 172.21.15.37:0/2695163438 wait complete. 2024-04-25T09:04:28.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 Processor -- start 2024-04-25T09:04:28.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 -- start start 2024-04-25T09:04:28.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd91c06ec60 0x7fd91c1e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd91c0ce510 0x7fd91c1e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd91c144ca0 0x7fd91c1f4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0cc900 con 0x7fd91c0ce510 2024-04-25T09:04:28.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c06df20 con 0x7fd91c144ca0 2024-04-25T09:04:28.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd923319640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c142d90 con 0x7fd91c06ec60 2024-04-25T09:04:28.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd9208b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd91c0ce510 0x7fd91c1e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd9208b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd91c0ce510 0x7fd91c1e6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51686/0 (socket says 172.21.15.37:51686) 2024-04-25T09:04:28.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.361+0000 7fd9208b4640 1 -- 172.21.15.37:0/1953934912 learned_addr learned my addr 172.21.15.37:0/1953934912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:28.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9218b6640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd91c144ca0 0x7fd91c1f4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9210b5640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd91c06ec60 0x7fd91c1e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9208b4640 1 -- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd91c06ec60 msgr2=0x7fd91c1e64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9208b4640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd91c06ec60 0x7fd91c1e64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9208b4640 1 -- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd91c144ca0 msgr2=0x7fd91c1f4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9208b4640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd91c144ca0 0x7fd91c1f4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9208b4640 1 -- 172.21.15.37:0/1953934912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c1e29c0 con 0x7fd91c0ce510 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9218b6640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd91c144ca0 0x7fd91c1f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9210b5640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd91c06ec60 0x7fd91c1e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9208b4640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd91c0ce510 0x7fd91c1e6a10 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00b3f0 tx=0x7fd90c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd911ffb640 1 -- 172.21.15.37:0/1953934912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c015020 con 0x7fd91c0ce510 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd911ffb640 1 -- 172.21.15.37:0/1953934912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90c011850 con 0x7fd91c0ce510 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd911ffb640 1 -- 172.21.15.37:0/1953934912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90c011c40 con 0x7fd91c0ce510 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd923319640 1 -- 172.21.15.37:0/1953934912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91c1e3c80 con 0x7fd91c0ce510 2024-04-25T09:04:28.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd923319640 1 -- 172.21.15.37:0/1953934912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c1e4240 con 0x7fd91c0ce510 2024-04-25T09:04:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd923319640 1 -- 172.21.15.37:0/1953934912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91c04efa0 con 0x7fd91c0ce510 2024-04-25T09:04:28.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd911ffb640 1 -- 172.21.15.37:0/1953934912 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fd90c018020 con 0x7fd91c0ce510 2024-04-25T09:04:28.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd911ffb640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd900066f60 0x7fd900069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd911ffb640 1 -- 172.21.15.37:0/1953934912 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fd90c085170 con 0x7fd91c0ce510 2024-04-25T09:04:28.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9210b5640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd900066f60 0x7fd900069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.365+0000 7fd9210b5640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd900066f60 0x7fd900069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd9180b2000 tx=0x7fd918008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.369+0000 7fd911ffb640 1 -- 172.21.15.37:0/1953934912 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fd90c01f020 con 0x7fd91c0ce510 2024-04-25T09:04:28.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.525+0000 7fd923319640 1 -- 172.21.15.37:0/1953934912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd91c0c6200 con 0x7fd91c0ce510 2024-04-25T09:04:28.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd911ffb640 1 -- 172.21.15.37:0/1953934912 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17175 (secure 0 0 0) 0x7fd90c0172c0 con 0x7fd91c0ce510 2024-04-25T09:04:28.517 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:28.517 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":127,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:04:12.937853+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:30.763724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:31.488462+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2159545482":"2024-04-26T09:04:12.937783+0000","172.21.15.37:6834/2159545482":"2024-04-26T09:04:12.937783+0000","172.21.15.37:6835/2748919477":"2024-04-26T09:03:58.978336+0000","172.21.15.37:6834/2748919477":"2024-04-26T09:03:58.978336+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:28.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 -- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd900066f60 msgr2=0x7fd900069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd900066f60 0x7fd900069420 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd9180b2000 tx=0x7fd918008040 comp rx=0 tx=0).stop 2024-04-25T09:04:28.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 -- 172.21.15.37:0/1953934912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd91c0ce510 msgr2=0x7fd91c1e6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd91c0ce510 0x7fd91c1e6a10 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00b3f0 tx=0x7fd90c00b8b0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 -- 172.21.15.37:0/1953934912 shutdown_connections 2024-04-25T09:04:28.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fd900066f60 0x7fd900069420 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fd91c144ca0 0x7fd91c1f4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd91c0ce510 0x7fd91c1e6a10 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 --2- 172.21.15.37:0/1953934912 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fd91c06ec60 0x7fd91c1e64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 -- 172.21.15.37:0/1953934912 >> 172.21.15.37:0/1953934912 conn(0x7fd91c0b11a0 msgr2=0x7fd91c0c48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:28.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 -- 172.21.15.37:0/1953934912 shutdown_connections 2024-04-25T09:04:28.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.529+0000 7fd923319640 1 -- 172.21.15.37:0/1953934912 wait complete. 2024-04-25T09:04:28.535 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:04:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc65aa06640 1 Processor -- start 2024-04-25T09:04:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc65aa06640 1 -- start start 2024-04-25T09:04:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc65aa06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc65406c3b0 0x7fc6540b3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc65aa06640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc654067140 con 0x7fc6540cf550 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc65aa06640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6540b3d60 con 0x7fc65415c3a0 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc65aa06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540b4040 con 0x7fc65406c3b0 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc658fa3640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc6540cf550 0x7fc65406d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:52048/0 (socket says 172.21.15.37:52048) 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc658fa3640 1 -- 172.21.15.37:0/2341570016 learned_addr learned my addr 172.21.15.37:0/2341570016 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc6537fe640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc65406c3b0 0x7fc6540b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1778924625 0 0) 0x7fc654067140 con 0x7fc6540cf550 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc63c003530 con 0x7fc6540cf550 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2299765693 0 0) 0x7fc63c003530 con 0x7fc6540cf550 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc65406c3b0 msgr2=0x7fc6540b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc65406c3b0 0x7fc6540b3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 >> v1:172.21.15.203:6789/0 conn(0x7fc65415c3a0 legacy=0x7fc65406bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654162550 con 0x7fc6540cf550 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fc64c001680 con 0x7fc6540cf550 2024-04-25T09:04:28.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 >> v1:172.21.15.37:6789/0 conn(0x7fc6540cf550 legacy=0x7fc65406d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc65406c3b0 0x7fc63c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c003fa0 0x7fc63c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c004d50 0x7fc63c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540b4040 con 0x7fc63c003fa0 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c00bab0 con 0x7fc63c004d50 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c00bcb0 con 0x7fc65406c3b0 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.661+0000 7fc6537fe640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c003fa0 0x7fc63c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc658fa3640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c004d50 0x7fc63c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc653fff640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc65406c3b0 0x7fc63c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc6537fe640 1 -- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc65406c3b0 msgr2=0x7fc63c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc6537fe640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc65406c3b0 0x7fc63c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc6537fe640 1 -- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c004d50 msgr2=0x7fc63c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc6537fe640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc63c004d50 0x7fc63c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc6537fe640 1 -- 172.21.15.37:0/2341570016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c003530 con 0x7fc63c003fa0 2024-04-25T09:04:28.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc653fff640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc65406c3b0 0x7fc63c003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:28.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc6537fe640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c003fa0 0x7fc63c004810 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fc648009ae0 tx=0x7fc64801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d65ac4b66b2e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6480b2070 con 0x7fc63c003fa0 2024-04-25T09:04:28.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6480a5660 con 0x7fc63c003fa0 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc652ffd640 1 -- 172.21.15.37:0/2341570016 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6480096b0 con 0x7fc63c003fa0 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc65aa06640 1 -- 172.21.15.37:0/2341570016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c003fa0 msgr2=0x7fc63c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc65aa06640 1 --2- 172.21.15.37:0/2341570016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc63c003fa0 0x7fc63c004810 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7fc648009ae0 tx=0x7fc64801ca70 comp rx=0 tx=0).stop 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc658fa3640 1 -- 172.21.15.37:0/2341570016 reap_dead start 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc65aa06640 1 -- 172.21.15.37:0/2341570016 shutdown_connections 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc65aa06640 1 -- 172.21.15.37:0/2341570016 >> 172.21.15.37:0/2341570016 conn(0x7fc654071810 msgr2=0x7fc654071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc65aa06640 1 -- 172.21.15.37:0/2341570016 shutdown_connections 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.665+0000 7fc65aa06640 1 -- 172.21.15.37:0/2341570016 wait complete. 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc65aa06640 1 Processor -- start 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc65aa06640 1 -- start start 2024-04-25T09:04:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc65aa06640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc63c003fa0 0x7fc6541fb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc65aa06640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc654146c10 0x7fc65415cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc65aa06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc654143520 0x7fc6541439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc65aa06640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65415c780 con 0x7fc654143520 2024-04-25T09:04:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc65aa06640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc654162ba0 con 0x7fc654146c10 2024-04-25T09:04:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc65aa06640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc654163030 con 0x7fc63c003fa0 2024-04-25T09:04:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc658fa3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc654143520 0x7fc6541439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc658fa3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc654143520 0x7fc6541439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51696/0 (socket says 172.21.15.37:51696) 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc658fa3640 1 -- 172.21.15.37:0/1773512022 learned_addr learned my addr 172.21.15.37:0/1773512022 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc653fff640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc63c003fa0 0x7fc6541fb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc6537fe640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc654146c10 0x7fc65415cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc658fa3640 1 -- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc63c003fa0 msgr2=0x7fc6541fb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc658fa3640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc63c003fa0 0x7fc6541fb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc658fa3640 1 -- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc654146c10 msgr2=0x7fc65415cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc658fa3640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc654146c10 0x7fc65415cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc658fa3640 1 -- 172.21.15.37:0/1773512022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6541e9270 con 0x7fc654143520 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc653fff640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc63c003fa0 0x7fc6541fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc658fa3640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc654143520 0x7fc6541439a0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fc6540731a0 tx=0x7fc64c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc650ff9640 1 -- 172.21.15.37:0/1773512022 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c00c630 con 0x7fc654143520 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc650ff9640 1 -- 172.21.15.37:0/1773512022 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c018070 con 0x7fc654143520 2024-04-25T09:04:28.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc650ff9640 1 -- 172.21.15.37:0/1773512022 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c00ca30 con 0x7fc654143520 2024-04-25T09:04:28.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc65aa06640 1 -- 172.21.15.37:0/1773512022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6541fbba0 con 0x7fc654143520 2024-04-25T09:04:28.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc65aa06640 1 -- 172.21.15.37:0/1773512022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6541fc160 con 0x7fc654143520 2024-04-25T09:04:28.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc65aa06640 1 -- 172.21.15.37:0/1773512022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65404efa0 con 0x7fc654143520 2024-04-25T09:04:28.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.669+0000 7fc650ff9640 1 -- 172.21.15.37:0/1773512022 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc64c00cbd0 con 0x7fc654143520 2024-04-25T09:04:28.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.673+0000 7fc650ff9640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 0x7fc640069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:28.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.673+0000 7fc650ff9640 1 -- 172.21.15.37:0/1773512022 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6273+0+0 (secure 0 0 0) 0x7fc64c084e30 con 0x7fc654143520 2024-04-25T09:04:28.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.673+0000 7fc653fff640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 0x7fc640069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:28.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.673+0000 7fc653fff640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 0x7fc640069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc6440059b0 tx=0x7fc644005290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:28.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.677+0000 7fc650ff9640 1 -- 172.21.15.37:0/1773512022 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc64c055c00 con 0x7fc654143520 2024-04-25T09:04:28.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.873+0000 7fc65aa06640 1 -- 172.21.15.37:0/1773512022 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc654162290 con 0x7fc654143520 2024-04-25T09:04:28.862 INFO:tasks.ceph.mon.a.smithi037.stderr:2024-04-25T09:04:28.873+0000 7ff66e2e8640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:04:28.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.877+0000 7fc650ff9640 1 -- 172.21.15.37:0/1773512022 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v111) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc64c055500 con 0x7fc654143520 2024-04-25T09:04:28.866 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:04:28.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 -- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 msgr2=0x7fc640069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 0x7fc640069420 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fc6440059b0 tx=0x7fc644005290 comp rx=0 tx=0).stop 2024-04-25T09:04:28.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 -- 172.21.15.37:0/1773512022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc654143520 msgr2=0x7fc6541439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:28.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc654143520 0x7fc6541439a0 secure :-1 s=READY pgs=2216 cs=0 l=1 rev1=1 crypto rx=0x7fc6540731a0 tx=0x7fc64c00eb90 comp rx=0 tx=0).stop 2024-04-25T09:04:28.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 -- 172.21.15.37:0/1773512022 shutdown_connections 2024-04-25T09:04:28.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc640066f60 0x7fc640069420 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc654143520 0x7fc6541439a0 unknown :-1 s=CLOSED pgs=2216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc654146c10 0x7fc65415cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 --2- 172.21.15.37:0/1773512022 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc63c003fa0 0x7fc6541fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:28.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 -- 172.21.15.37:0/1773512022 >> 172.21.15.37:0/1773512022 conn(0x7fc654071810 msgr2=0x7fc6540b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:28.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 -- 172.21.15.37:0/1773512022 shutdown_connections 2024-04-25T09:04:28.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:28.881+0000 7fc65aa06640 1 -- 172.21.15.37:0/1773512022 wait complete. 2024-04-25T09:04:28.871 INFO:tasks.ceph.mds.a.smithi037.stderr: -1461> 2024-04-25T09:04:18.957+0000 7f95d617b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:04:28.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:04:28.903 INFO:tasks.ceph.mds.a.smithi037.stdout:starting mds.a at 2024-04-25T09:04:29.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.017+0000 7f1f0ee85640 1 Processor -- start 2024-04-25T09:04:29.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.017+0000 7f1f0ee85640 1 -- start start 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f0ee85640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f1f0815da80 0x7f1f080d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f0ee85640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1f080d1cd0 0x7f1f080d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f0ee85640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0806be60 con 0x7f1f080d0620 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f0ee85640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f080d0a70 con 0x7f1f080d1cd0 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f0ee85640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0806c440 con 0x7f1f0815da80 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f07fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1f080d1cd0 0x7f1f080d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f07fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f1f080d1cd0 0x7f1f080d00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51722/0 (socket says 172.21.15.37:51722) 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f07fff640 1 -- 172.21.15.37:0/185738561 learned_addr learned my addr 172.21.15.37:0/185738561 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f0cc21640 1 --2- 172.21.15.37:0/185738561 >> v2:172.21.15.203:3300/0 conn(0x7f1f0815da80 0x7f1f080d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f07fff640 1 -- 172.21.15.37:0/185738561 >> v2:172.21.15.203:3300/0 conn(0x7f1f0815da80 msgr2=0x7f1f080d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f07fff640 1 --2- 172.21.15.37:0/185738561 >> v2:172.21.15.203:3300/0 conn(0x7f1f0815da80 0x7f1f080d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f07fff640 1 -- 172.21.15.37:0/185738561 >> v1:172.21.15.203:6789/0 conn(0x7f1f080d0620 legacy=0x7f1f080ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:29.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f07fff640 1 -- 172.21.15.37:0/185738561 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0815de60 con 0x7f1f080d1cd0 2024-04-25T09:04:29.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f0cc21640 1 --2- 172.21.15.37:0/185738561 >> v2:172.21.15.203:3300/0 conn(0x7f1f0815da80 0x7f1f080d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f07fff640 1 --2- 172.21.15.37:0/185738561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080d1cd0 0x7f1f080d00e0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f1efc006fd0 tx=0x7f1efc01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c212ce69b939b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f077fe640 1 -- 172.21.15.37:0/185738561 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0b0070 con 0x7f1f080d1cd0 2024-04-25T09:04:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f077fe640 1 -- 172.21.15.37:0/185738561 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc004230 con 0x7f1f080d1cd0 2024-04-25T09:04:29.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.021+0000 7f1f077fe640 1 -- 172.21.15.37:0/185738561 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc003620 con 0x7f1f080d1cd0 2024-04-25T09:04:29.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 -- 172.21.15.37:0/185738561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080d1cd0 msgr2=0x7f1f080d00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 --2- 172.21.15.37:0/185738561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080d1cd0 0x7f1f080d00e0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7f1efc006fd0 tx=0x7f1efc01c4c0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 -- 172.21.15.37:0/185738561 shutdown_connections 2024-04-25T09:04:29.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 --2- 172.21.15.37:0/185738561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080d1cd0 0x7f1f080d00e0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 --2- 172.21.15.37:0/185738561 >> v2:172.21.15.203:3300/0 conn(0x7f1f0815da80 0x7f1f080d4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 -- 172.21.15.37:0/185738561 >> 172.21.15.37:0/185738561 conn(0x7f1f0808b000 msgr2=0x7f1f080d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:29.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 -- 172.21.15.37:0/185738561 shutdown_connections 2024-04-25T09:04:29.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 -- 172.21.15.37:0/185738561 wait complete. 2024-04-25T09:04:29.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 Processor -- start 2024-04-25T09:04:29.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 -- start start 2024-04-25T09:04:29.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f080d0620 0x7f1f080cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080d1cd0 0x7f1f081532a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f0815da80 0x7f1f08158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0816a730 con 0x7f1f080d1cd0 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0816a5b0 con 0x7f1f0815da80 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0ee85640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0816a8b0 con 0x7f1f080d0620 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080d1cd0 0x7f1f081532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080d1cd0 0x7f1f081532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51730/0 (socket says 172.21.15.37:51730) 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f07fff640 1 -- 172.21.15.37:0/1492103638 learned_addr learned my addr 172.21.15.37:0/1492103638 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0cc21640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f080d0620 0x7f1f080cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f0d422640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f0815da80 0x7f1f08158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f07fff640 1 -- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f080d0620 msgr2=0x7f1f080cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f07fff640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f080d0620 0x7f1f080cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f07fff640 1 -- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f0815da80 msgr2=0x7f1f08158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f07fff640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f0815da80 0x7f1f08158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f07fff640 1 -- 172.21.15.37:0/1492103638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f080cb240 con 0x7f1f080d1cd0 2024-04-25T09:04:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.025+0000 7f1f07fff640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080d1cd0 0x7f1f081532a0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f1efc01a0f0 tx=0x7f1efc019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.029+0000 7f1f057fa640 1 -- 172.21.15.37:0/1492103638 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0bb070 con 0x7f1f080d1cd0 2024-04-25T09:04:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.029+0000 7f1f057fa640 1 -- 172.21.15.37:0/1492103638 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc0043e0 con 0x7f1f080d1cd0 2024-04-25T09:04:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.029+0000 7f1f0ee85640 1 -- 172.21.15.37:0/1492103638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f08156ed0 con 0x7f1f080d1cd0 2024-04-25T09:04:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.029+0000 7f1f0ee85640 1 -- 172.21.15.37:0/1492103638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f080cb7c0 con 0x7f1f080d1cd0 2024-04-25T09:04:29.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.029+0000 7f1f057fa640 1 -- 172.21.15.37:0/1492103638 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc0b0040 con 0x7f1f080d1cd0 2024-04-25T09:04:29.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.029+0000 7f1f0ee85640 1 -- 172.21.15.37:0/1492103638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee80052a0 con 0x7f1f080d1cd0 2024-04-25T09:04:29.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.029+0000 7f1f057fa640 1 -- 172.21.15.37:0/1492103638 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f1efc0be070 con 0x7f1f080d1cd0 2024-04-25T09:04:29.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.029+0000 7f1f057fa640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ef4066f60 0x7f1ef4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:29.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.029+0000 7f1f057fa640 1 -- 172.21.15.37:0/1492103638 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6359+0+0 (secure 0 0 0) 0x7f1efc11ebf0 con 0x7f1f080d1cd0 2024-04-25T09:04:29.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.029+0000 7f1f0cc21640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ef4066f60 0x7f1ef4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:29.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.029+0000 7f1f0cc21640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ef4066f60 0x7f1ef4069420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8007510 tx=0x7f1ef80074a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:29.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.033+0000 7f1f057fa640 1 -- 172.21.15.37:0/1492103638 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f1efc0160b0 con 0x7f1f080d1cd0 2024-04-25T09:04:29.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.229+0000 7f1f0ee85640 1 -- 172.21.15.37:0/1492103638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1ee8005860 con 0x7f1f080d1cd0 2024-04-25T09:04:29.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.881+0000 7f1f057fa640 1 -- 172.21.15.37:0/1492103638 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v112) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1efc0ef970 con 0x7f1f080d1cd0 2024-04-25T09:04:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 -- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ef4066f60 msgr2=0x7f1ef4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ef4066f60 0x7f1ef4069420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8007510 tx=0x7f1ef80074a0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 -- 172.21.15.37:0/1492103638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080d1cd0 msgr2=0x7f1f081532a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080d1cd0 0x7f1f081532a0 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f1efc01a0f0 tx=0x7f1efc019260 comp rx=0 tx=0).stop 2024-04-25T09:04:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 -- 172.21.15.37:0/1492103638 shutdown_connections 2024-04-25T09:04:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1ef4066f60 0x7f1ef4069420 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f1f0815da80 0x7f1f08158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f080d1cd0 0x7f1f081532a0 secure :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f1efc01a0f0 tx=0x7f1efc019260 comp rx=0 tx=0).stop 2024-04-25T09:04:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 --2- 172.21.15.37:0/1492103638 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f1f080d0620 0x7f1f080cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 -- 172.21.15.37:0/1492103638 >> 172.21.15.37:0/1492103638 conn(0x7f1f0808b000 msgr2=0x7f1f08160890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 -- 172.21.15.37:0/1492103638 shutdown_connections 2024-04-25T09:04:29.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:29.885+0000 7f1f0ee85640 1 -- 172.21.15.37:0/1492103638 wait complete. 2024-04-25T09:04:29.896 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:04:30.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.081+0000 7ffb93f54640 1 Processor -- start 2024-04-25T09:04:30.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.081+0000 7ffb93f54640 1 -- start start 2024-04-25T09:04:30.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.081+0000 7ffb93f54640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ffb8c0cf980 0x7ffb8c0ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.081+0000 7ffb93f54640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c0b28c0 con 0x7ffb8c0cf510 2024-04-25T09:04:30.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.081+0000 7ffb93f54640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c15bb20 con 0x7ffb8c0adc70 2024-04-25T09:04:30.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.081+0000 7ffb93f54640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b2b60 con 0x7ffb8c0cf980 2024-04-25T09:04:30.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb914ef640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7ffb8c0cf510 0x7ffb8c0ac170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56138/0 (socket says 172.21.15.37:56138) 2024-04-25T09:04:30.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb914ef640 1 -- 172.21.15.37:0/2905111035 learned_addr learned my addr 172.21.15.37:0/2905111035 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:30.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb924f1640 1 --2- 172.21.15.37:0/2905111035 >> v2:172.21.15.203:3300/0 conn(0x7ffb8c0cf980 0x7ffb8c0ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092252041 0 0) 0x7ffb8c0b28c0 con 0x7ffb8c0cf510 2024-04-25T09:04:30.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb78003530 con 0x7ffb8c0cf510 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3653097359 0 0) 0x7ffb78003530 con 0x7ffb8c0cf510 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 >> v2:172.21.15.203:3300/0 conn(0x7ffb8c0cf980 msgr2=0x7ffb8c0ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 --2- 172.21.15.37:0/2905111035 >> v2:172.21.15.203:3300/0 conn(0x7ffb8c0cf980 0x7ffb8c0ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 >> v1:172.21.15.203:6789/0 conn(0x7ffb8c0adc70 legacy=0x7ffb8c06e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0b2e40 con 0x7ffb8c0cf510 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7ffb7c0016a0 con 0x7ffb8c0cf510 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 >> v1:172.21.15.37:6789/0 conn(0x7ffb8c0cf510 legacy=0x7ffb8c0ac170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 --2- 172.21.15.37:0/2905111035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb78003a10 0x7ffb78003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 --2- 172.21.15.37:0/2905111035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 0x7ffb78004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 --2- 172.21.15.37:0/2905111035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb780051d0 0x7ffb7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b2b60 con 0x7ffb78004810 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7800c0d0 con 0x7ffb78003a10 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb7800c2d0 con 0x7ffb780051d0 2024-04-25T09:04:30.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb914ef640 1 --2- 172.21.15.37:0/2905111035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 0x7ffb78004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb91cf0640 1 --2- 172.21.15.37:0/2905111035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb78003a10 0x7ffb78003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb914ef640 1 -- 172.21.15.37:0/2905111035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb780051d0 msgr2=0x7ffb7800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:30.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb914ef640 1 --2- 172.21.15.37:0/2905111035 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb780051d0 0x7ffb7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb914ef640 1 -- 172.21.15.37:0/2905111035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb78003a10 msgr2=0x7ffb78003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:30.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb914ef640 1 --2- 172.21.15.37:0/2905111035 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb78003a10 0x7ffb78003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb914ef640 1 -- 172.21.15.37:0/2905111035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78003530 con 0x7ffb78004810 2024-04-25T09:04:30.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb924f1640 1 -- 172.21.15.37:0/2905111035 reap_dead start 2024-04-25T09:04:30.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb914ef640 1 --2- 172.21.15.37:0/2905111035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 0x7ffb78004c90 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00c160 tx=0x7ffb7c01f2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b74a1661e4e78f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:30.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c007c70 con 0x7ffb78004810 2024-04-25T09:04:30.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c0ac070 con 0x7ffb78004810 2024-04-25T09:04:30.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.085+0000 7ffb90cee640 1 -- 172.21.15.37:0/2905111035 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c005b20 con 0x7ffb78004810 2024-04-25T09:04:30.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- 172.21.15.37:0/2905111035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 msgr2=0x7ffb78004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 --2- 172.21.15.37:0/2905111035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 0x7ffb78004c90 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c00c160 tx=0x7ffb7c01f2b0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- 172.21.15.37:0/2905111035 shutdown_connections 2024-04-25T09:04:30.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 --2- 172.21.15.37:0/2905111035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 0x7ffb78004c90 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- 172.21.15.37:0/2905111035 >> 172.21.15.37:0/2905111035 conn(0x7ffb8c06ae60 msgr2=0x7ffb8c06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:30.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- 172.21.15.37:0/2905111035 shutdown_connections 2024-04-25T09:04:30.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- 172.21.15.37:0/2905111035 wait complete. 2024-04-25T09:04:30.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 Processor -- start 2024-04-25T09:04:30.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- start start 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 0x7ffb8c098a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb8c0c40b0 0x7ffb8c0ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8c0c55e0 0x7ffb8c0c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b36e0 con 0x7ffb78004810 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0b3400 con 0x7ffb8c0c55e0 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c0cfdf0 con 0x7ffb8c0c40b0 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb91cf0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 0x7ffb8c098a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb91cf0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 0x7ffb8c098a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50092/0 (socket says 172.21.15.37:50092) 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb91cf0640 1 -- 172.21.15.37:0/219710792 learned_addr learned my addr 172.21.15.37:0/219710792 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb924f1640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8c0c55e0 0x7ffb8c0c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb914ef640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb8c0c40b0 0x7ffb8c0ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb91cf0640 1 -- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb8c0c40b0 msgr2=0x7ffb8c0ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb91cf0640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb8c0c40b0 0x7ffb8c0ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb91cf0640 1 -- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8c0c55e0 msgr2=0x7ffb8c0c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb91cf0640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8c0c55e0 0x7ffb8c0c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb91cf0640 1 -- 172.21.15.37:0/219710792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0c4540 con 0x7ffb78004810 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb914ef640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb8c0c40b0 0x7ffb8c0ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb924f1640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8c0c55e0 0x7ffb8c0c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb91cf0640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 0x7ffb8c098a00 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800e4e0 tx=0x7ffb8800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb827fc640 1 -- 172.21.15.37:0/219710792 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb880052e0 con 0x7ffb78004810 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb827fc640 1 -- 172.21.15.37:0/219710792 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb88018070 con 0x7ffb78004810 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- 172.21.15.37:0/219710792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c0c4820 con 0x7ffb78004810 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb827fc640 1 -- 172.21.15.37:0/219710792 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88005480 con 0x7ffb78004810 2024-04-25T09:04:30.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- 172.21.15.37:0/219710792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c0c4de0 con 0x7ffb78004810 2024-04-25T09:04:30.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.089+0000 7ffb93f54640 1 -- 172.21.15.37:0/219710792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c067c30 con 0x7ffb78004810 2024-04-25T09:04:30.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.093+0000 7ffb827fc640 1 -- 172.21.15.37:0/219710792 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ffb88003840 con 0x7ffb78004810 2024-04-25T09:04:30.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.093+0000 7ffb827fc640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb70066f60 0x7ffb70069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:30.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.093+0000 7ffb827fc640 1 -- 172.21.15.37:0/219710792 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6359+0+0 (secure 0 0 0) 0x7ffb88084f90 con 0x7ffb78004810 2024-04-25T09:04:30.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.093+0000 7ffb914ef640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb70066f60 0x7ffb70069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:30.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.093+0000 7ffb914ef640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb70066f60 0x7ffb70069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0ae000 tx=0x7ffb7c0aa070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:30.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.097+0000 7ffb827fc640 1 -- 172.21.15.37:0/219710792 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ffb88055c80 con 0x7ffb78004810 2024-04-25T09:04:30.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.257+0000 7ffb93f54640 1 -- 172.21.15.37:0/219710792 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffb8c06d760 con 0x7ffb78004810 2024-04-25T09:04:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.893+0000 7ffb827fc640 1 -- 172.21.15.37:0/219710792 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v129) v1 ==== 181+0+0 (secure 0 0 0) 0x7ffb88055580 con 0x7ffb78004810 2024-04-25T09:04:30.882 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:04:30.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 -- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb70066f60 msgr2=0x7ffb70069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb70066f60 0x7ffb70069420 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c0ae000 tx=0x7ffb7c0aa070 comp rx=0 tx=0).stop 2024-04-25T09:04:30.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 -- 172.21.15.37:0/219710792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 msgr2=0x7ffb8c098a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 0x7ffb8c098a00 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800e4e0 tx=0x7ffb8800e9a0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 -- 172.21.15.37:0/219710792 shutdown_connections 2024-04-25T09:04:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ffb70066f60 0x7ffb70069420 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ffb8c0c55e0 0x7ffb8c0c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ffb8c0c40b0 0x7ffb8c0ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 --2- 172.21.15.37:0/219710792 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb78004810 0x7ffb8c098a00 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 -- 172.21.15.37:0/219710792 >> 172.21.15.37:0/219710792 conn(0x7ffb8c06ae60 msgr2=0x7ffb8c15e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 -- 172.21.15.37:0/219710792 shutdown_connections 2024-04-25T09:04:30.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:30.897+0000 7ffb93f54640 1 -- 172.21.15.37:0/219710792 wait complete. 2024-04-25T09:04:30.914 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:04:31.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.057+0000 7f027d361640 1 Processor -- start 2024-04-25T09:04:31.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.057+0000 7f027d361640 1 -- start start 2024-04-25T09:04:31.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f027d361640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780adc70 0x7f027806e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f027d361640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f027806e9f0 0x7f02780cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f027d361640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02780b28c0 con 0x7f02780cfa50 2024-04-25T09:04:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f027d361640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f027815bb20 con 0x7f027806e9f0 2024-04-25T09:04:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f027d361640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02780b2b60 con 0x7f02780adc70 2024-04-25T09:04:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f02777fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f02780cfa50 0x7f027815f250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56148/0 (socket says 172.21.15.37:56148) 2024-04-25T09:04:31.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f02777fe640 1 -- 172.21.15.37:0/298572409 learned_addr learned my addr 172.21.15.37:0/298572409 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:31.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f02767fc640 1 --2- 172.21.15.37:0/298572409 >> v2:172.21.15.203:3300/0 conn(0x7f027806e9f0 0x7f02780cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742735391 0 0) 0x7f02780b28c0 con 0x7f02780cfa50 2024-04-25T09:04:31.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0260003530 con 0x7f02780cfa50 2024-04-25T09:04:31.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0276ffd640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780adc70 0x7f027806e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3843479462 0 0) 0x7f0260003530 con 0x7f02780cfa50 2024-04-25T09:04:31.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780adc70 msgr2=0x7f027806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:31.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780adc70 0x7f027806e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 >> v2:172.21.15.203:3300/0 conn(0x7f027806e9f0 msgr2=0x7f02780cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 --2- 172.21.15.37:0/298572409 >> v2:172.21.15.203:3300/0 conn(0x7f027806e9f0 0x7f02780cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780b2e40 con 0x7f02780cfa50 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f0270001a00 con 0x7f02780cfa50 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 >> v1:172.21.15.37:6789/0 conn(0x7f02780cfa50 legacy=0x7f027815f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02600039c0 0x7f0260003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780adc70 0x7f0260004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0260004d50 0x7f02600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f027815bb20 con 0x7f0260004d50 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02780b2b60 con 0x7f02600039c0 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026000b9d0 con 0x7f02780adc70 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f02777fe640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0260004d50 0x7f02600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f0276ffd640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02600039c0 0x7f0260003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f02767fc640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780adc70 0x7f0260004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f02777fe640 1 -- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780adc70 msgr2=0x7f0260004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:31.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f02777fe640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780adc70 0x7f0260004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f02777fe640 1 -- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02600039c0 msgr2=0x7f0260003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:31.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f02777fe640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02600039c0 0x7f0260003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f02777fe640 1 -- 172.21.15.37:0/298572409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0260003530 con 0x7f0260004d50 2024-04-25T09:04:31.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.061+0000 7f02767fc640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780adc70 0x7f0260004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:31.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02777fe640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0260004d50 0x7f02600051d0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f027000a7a0 tx=0x7f027001d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41069bf1adb6f19b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027000a3f0 con 0x7f0260004d50 2024-04-25T09:04:31.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02700ad650 con 0x7f0260004d50 2024-04-25T09:04:31.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f0275ffb640 1 -- 172.21.15.37:0/298572409 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02700ada40 con 0x7f0260004d50 2024-04-25T09:04:31.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 -- 172.21.15.37:0/298572409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0260004d50 msgr2=0x7f02600051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 --2- 172.21.15.37:0/298572409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0260004d50 0x7f02600051d0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f027000a7a0 tx=0x7f027001d2c0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02777fe640 1 -- 172.21.15.37:0/298572409 reap_dead start 2024-04-25T09:04:31.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 -- 172.21.15.37:0/298572409 shutdown_connections 2024-04-25T09:04:31.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 -- 172.21.15.37:0/298572409 >> 172.21.15.37:0/298572409 conn(0x7f027806ae60 msgr2=0x7f027806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:31.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 -- 172.21.15.37:0/298572409 shutdown_connections 2024-04-25T09:04:31.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 -- 172.21.15.37:0/298572409 wait complete. 2024-04-25T09:04:31.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 Processor -- start 2024-04-25T09:04:31.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 -- start start 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780ac3d0 0x7f02780ce0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02780ac7b0 0x7f02780ce610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02780cceb0 0x7f02780cd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02780b36e0 con 0x7f02780ac7b0 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02780b3400 con 0x7f02780cceb0 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f027d361640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02780b3c00 con 0x7f02780ac3d0 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02767fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02780ac7b0 0x7f02780ce610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02767fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02780ac7b0 0x7f02780ce610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50102/0 (socket says 172.21.15.37:50102) 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02767fc640 1 -- 172.21.15.37:0/1599744365 learned_addr learned my addr 172.21.15.37:0/1599744365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f0276ffd640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780ac3d0 0x7f02780ce0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02777fe640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02780cceb0 0x7f02780cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02767fc640 1 -- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780ac3d0 msgr2=0x7f02780ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02767fc640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780ac3d0 0x7f02780ce0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02767fc640 1 -- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02780cceb0 msgr2=0x7f02780cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02767fc640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02780cceb0 0x7f02780cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02767fc640 1 -- 172.21.15.37:0/1599744365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02780ae750 con 0x7f02780ac7b0 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02777fe640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02780cceb0 0x7f02780cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.065+0000 7f02767fc640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02780ac7b0 0x7f02780ce610 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f026c0115d0 tx=0x7f026c011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.069+0000 7f025f7fe640 1 -- 172.21.15.37:0/1599744365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c004220 con 0x7f02780ac7b0 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.069+0000 7f025f7fe640 1 -- 172.21.15.37:0/1599744365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f026c0043c0 con 0x7f02780ac7b0 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.069+0000 7f027d361640 1 -- 172.21.15.37:0/1599744365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02780cec40 con 0x7f02780ac7b0 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.069+0000 7f025f7fe640 1 -- 172.21.15.37:0/1599744365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f026c00f910 con 0x7f02780ac7b0 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.069+0000 7f027d361640 1 -- 172.21.15.37:0/1599744365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f027815a120 con 0x7f02780ac7b0 2024-04-25T09:04:31.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.069+0000 7f027d361640 1 -- 172.21.15.37:0/1599744365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0278067c30 con 0x7f02780ac7b0 2024-04-25T09:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.069+0000 7f025f7fe640 1 -- 172.21.15.37:0/1599744365 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f026c00fab0 con 0x7f02780ac7b0 2024-04-25T09:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.069+0000 7f025f7fe640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0264066f90 0x7f0264069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.069+0000 7f025f7fe640 1 -- 172.21.15.37:0/1599744365 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5921+0+0 (secure 0 0 0) 0x7f026c0859e0 con 0x7f02780ac7b0 2024-04-25T09:04:31.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.069+0000 7f0276ffd640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0264066f90 0x7f0264069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:31.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.069+0000 7f0276ffd640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0264066f90 0x7f0264069450 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0268009a60 tx=0x7f0268008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:31.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.073+0000 7f025f7fe640 1 -- 172.21.15.37:0/1599744365 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f026c019030 con 0x7f02780ac7b0 2024-04-25T09:04:31.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.233+0000 7f027d361640 1 -- 172.21.15.37:0/1599744365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f027815b450 con 0x7f02780ac7b0 2024-04-25T09:04:31.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f025f7fe640 1 -- 172.21.15.37:0/1599744365 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v130) v1 ==== 169+0+0 (secure 0 0 0) 0x7f026c0563a0 con 0x7f02780ac7b0 2024-04-25T09:04:31.885 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 -- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0264066f90 msgr2=0x7f0264069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0264066f90 0x7f0264069450 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f0268009a60 tx=0x7f0268008040 comp rx=0 tx=0).stop 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 -- 172.21.15.37:0/1599744365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02780ac7b0 msgr2=0x7f02780ce610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02780ac7b0 0x7f02780ce610 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7f026c0115d0 tx=0x7f026c011a90 comp rx=0 tx=0).stop 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 -- 172.21.15.37:0/1599744365 shutdown_connections 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f0264066f90 0x7f0264069450 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f02780cceb0 0x7f02780cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f02780ac7b0 0x7f02780ce610 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 --2- 172.21.15.37:0/1599744365 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f02780ac3d0 0x7f02780ce0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 -- 172.21.15.37:0/1599744365 >> 172.21.15.37:0/1599744365 conn(0x7f027806ae60 msgr2=0x7f02780cc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 -- 172.21.15.37:0/1599744365 shutdown_connections 2024-04-25T09:04:31.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:31.897+0000 7f027d361640 1 -- 172.21.15.37:0/1599744365 wait complete. 2024-04-25T09:04:31.913 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:04:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.049+0000 7f6d2a65f640 1 Processor -- start 2024-04-25T09:04:32.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.049+0000 7f6d2a65f640 1 -- start start 2024-04-25T09:04:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d2a65f640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6d240d0130 0x7f6d240cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d2a65f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d240d0510 0x7f6d24162330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d2a65f640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d240d2530 con 0x7f6d240d1fe0 2024-04-25T09:04:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d2a65f640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d240d2810 con 0x7f6d240d0130 2024-04-25T09:04:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d2a65f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240b3160 con 0x7f6d240d0510 2024-04-25T09:04:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d23fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6d240d0130 0x7f6d240cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d23fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f6d240d0130 0x7f6d240cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50118/0 (socket says 172.21.15.37:50118) 2024-04-25T09:04:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d23fff640 1 -- 172.21.15.37:0/183876556 learned_addr learned my addr 172.21.15.37:0/183876556 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:32.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d28bfc640 1 --2- 172.21.15.37:0/183876556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d240d0510 0x7f6d24162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d23fff640 1 -- 172.21.15.37:0/183876556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d240d0510 msgr2=0x7f6d24162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d23fff640 1 --2- 172.21.15.37:0/183876556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d240d0510 0x7f6d24162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d23fff640 1 -- 172.21.15.37:0/183876556 >> v1:172.21.15.203:6789/0 conn(0x7f6d240d1fe0 legacy=0x7f6d24161c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d23fff640 1 -- 172.21.15.37:0/183876556 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d240b3440 con 0x7f6d240d0130 2024-04-25T09:04:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d23fff640 1 --2- 172.21.15.37:0/183876556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d240d0130 0x7f6d240cf4c0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f6d2408baa0 tx=0x7f6d1801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b34574aa5ccfa3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d22ffd640 1 -- 172.21.15.37:0/183876556 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d180b0070 con 0x7f6d240d0130 2024-04-25T09:04:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d22ffd640 1 -- 172.21.15.37:0/183876556 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d180ab600 con 0x7f6d240d0130 2024-04-25T09:04:32.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.053+0000 7f6d22ffd640 1 -- 172.21.15.37:0/183876556 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d180ab9d0 con 0x7f6d240d0130 2024-04-25T09:04:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 -- 172.21.15.37:0/183876556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d240d0130 msgr2=0x7f6d240cf4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 --2- 172.21.15.37:0/183876556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d240d0130 0x7f6d240cf4c0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7f6d2408baa0 tx=0x7f6d1801c4e0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 -- 172.21.15.37:0/183876556 shutdown_connections 2024-04-25T09:04:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 --2- 172.21.15.37:0/183876556 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d240d0510 0x7f6d24162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 --2- 172.21.15.37:0/183876556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d240d0130 0x7f6d240cf4c0 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 -- 172.21.15.37:0/183876556 >> 172.21.15.37:0/183876556 conn(0x7f6d2408b000 msgr2=0x7f6d240d36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 -- 172.21.15.37:0/183876556 shutdown_connections 2024-04-25T09:04:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 -- 172.21.15.37:0/183876556 wait complete. 2024-04-25T09:04:32.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 Processor -- start 2024-04-25T09:04:32.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 -- start start 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d240d0130 0x7f6d240ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d240d0510 0x7f6d240c98e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d240d1fe0 0x7f6d241516b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240b4450 con 0x7f6d240d0510 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d24162a80 con 0x7f6d240d0130 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240b45d0 con 0x7f6d240d1fe0 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d240d0510 0x7f6d240c98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d240d0510 0x7f6d240c98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50126/0 (socket says 172.21.15.37:50126) 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d237fe640 1 -- 172.21.15.37:0/3832055401 learned_addr learned my addr 172.21.15.37:0/3832055401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d23fff640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d240d0130 0x7f6d240ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d28bfc640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d240d1fe0 0x7f6d241516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d237fe640 1 -- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d240d1fe0 msgr2=0x7f6d241516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d237fe640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d240d1fe0 0x7f6d241516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d237fe640 1 -- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d240d0130 msgr2=0x7f6d240ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d237fe640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d240d0130 0x7f6d240ca7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d237fe640 1 -- 172.21.15.37:0/3832055401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d241522f0 con 0x7f6d240d0510 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d28bfc640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d240d1fe0 0x7f6d241516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d237fe640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d240d0510 0x7f6d240c98e0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400b410 tx=0x7f6d1400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d20ff9640 1 -- 172.21.15.37:0/3832055401 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14015030 con 0x7f6d240d0510 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d20ff9640 1 -- 172.21.15.37:0/3832055401 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d14004020 con 0x7f6d240d0510 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 -- 172.21.15.37:0/3832055401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d24152560 con 0x7f6d240d0510 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d20ff9640 1 -- 172.21.15.37:0/3832055401 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d14004450 con 0x7f6d240d0510 2024-04-25T09:04:32.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.057+0000 7f6d2a65f640 1 -- 172.21.15.37:0/3832055401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d24152e00 con 0x7f6d240d0510 2024-04-25T09:04:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.061+0000 7f6d2a65f640 1 -- 172.21.15.37:0/3832055401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d2404efa0 con 0x7f6d240d0510 2024-04-25T09:04:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.061+0000 7f6d20ff9640 1 -- 172.21.15.37:0/3832055401 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f6d14018070 con 0x7f6d240d0510 2024-04-25T09:04:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.061+0000 7f6d20ff9640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d10066f60 0x7f6d10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.061+0000 7f6d20ff9640 1 -- 172.21.15.37:0/3832055401 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f6d14084b10 con 0x7f6d240d0510 2024-04-25T09:04:32.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.061+0000 7f6d23fff640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d10066f60 0x7f6d10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.061+0000 7f6d23fff640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d10066f60 0x7f6d10069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6d180ad000 tx=0x7f6d180a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.065+0000 7f6d20ff9640 1 -- 172.21.15.37:0/3832055401 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f6d14055bd0 con 0x7f6d240d0510 2024-04-25T09:04:32.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.233+0000 7f6d2a65f640 1 -- 172.21.15.37:0/3832055401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6d240b2540 con 0x7f6d240d0510 2024-04-25T09:04:32.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.241+0000 7f6d20ff9640 1 -- 172.21.15.37:0/3832055401 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6d140554d0 con 0x7f6d240d0510 2024-04-25T09:04:32.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 -- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d10066f60 msgr2=0x7f6d10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d10066f60 0x7f6d10069420 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6d180ad000 tx=0x7f6d180a6000 comp rx=0 tx=0).stop 2024-04-25T09:04:32.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 -- 172.21.15.37:0/3832055401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d240d0510 msgr2=0x7f6d240c98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d240d0510 0x7f6d240c98e0 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7f6d1400b410 tx=0x7f6d1400b8d0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 -- 172.21.15.37:0/3832055401 shutdown_connections 2024-04-25T09:04:32.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6d10066f60 0x7f6d10069420 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6d240d1fe0 0x7f6d241516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d240d0510 0x7f6d240c98e0 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 --2- 172.21.15.37:0/3832055401 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6d240d0130 0x7f6d240ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 -- 172.21.15.37:0/3832055401 >> 172.21.15.37:0/3832055401 conn(0x7f6d2408b000 msgr2=0x7f6d240b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:32.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 -- 172.21.15.37:0/3832055401 shutdown_connections 2024-04-25T09:04:32.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.245+0000 7f6d2a65f640 1 -- 172.21.15.37:0/3832055401 wait complete. 2024-04-25T09:04:32.251 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-04-25T09:04:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde8e27640 1 Processor -- start 2024-04-25T09:04:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde8e27640 1 -- start start 2024-04-25T09:04:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde8e27640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fcde40b2ac0 0x7fcde40ce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde8e27640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcde40ce880 0x7fcde40cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde8e27640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40d0e30 0x7fcde40d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde8e27640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde406bdc0 con 0x7fcde40ce880 2024-04-25T09:04:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde8e27640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde40ced00 con 0x7fcde40b2ac0 2024-04-25T09:04:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde8e27640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40cefe0 con 0x7fcde40d0e30 2024-04-25T09:04:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde1d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcde40ce880 0x7fcde40cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde1d9b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fcde40ce880 0x7fcde40cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50142/0 (socket says 172.21.15.37:50142) 2024-04-25T09:04:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde1d9b640 1 -- 172.21.15.37:0/2958078389 learned_addr learned my addr 172.21.15.37:0/2958078389 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde259c640 1 --2- 172.21.15.37:0/2958078389 >> v2:172.21.15.203:3300/0 conn(0x7fcde40b2ac0 0x7fcde40ce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde2d9d640 1 --2- 172.21.15.37:0/2958078389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40d0e30 0x7fcde40d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde1d9b640 1 -- 172.21.15.37:0/2958078389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40d0e30 msgr2=0x7fcde40d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde1d9b640 1 --2- 172.21.15.37:0/2958078389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40d0e30 0x7fcde40d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde1d9b640 1 -- 172.21.15.37:0/2958078389 >> v2:172.21.15.203:3300/0 conn(0x7fcde40b2ac0 msgr2=0x7fcde40ce340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde1d9b640 1 --2- 172.21.15.37:0/2958078389 >> v2:172.21.15.203:3300/0 conn(0x7fcde40b2ac0 0x7fcde40ce340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde1d9b640 1 -- 172.21.15.37:0/2958078389 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde40cf2c0 con 0x7fcde40ce880 2024-04-25T09:04:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.393+0000 7fcde259c640 1 --2- 172.21.15.37:0/2958078389 >> v2:172.21.15.203:3300/0 conn(0x7fcde40b2ac0 0x7fcde40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde1d9b640 1 --2- 172.21.15.37:0/2958078389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde40ce880 0x7fcde40cfed0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fcde406bc80 tx=0x7fcdcc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68914f9095b9a4c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde159a640 1 -- 172.21.15.37:0/2958078389 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0ab640 con 0x7fcde40ce880 2024-04-25T09:04:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde159a640 1 -- 172.21.15.37:0/2958078389 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc0b0070 con 0x7fcde40ce880 2024-04-25T09:04:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde159a640 1 -- 172.21.15.37:0/2958078389 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0aba00 con 0x7fcde40ce880 2024-04-25T09:04:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde8e27640 1 -- 172.21.15.37:0/2958078389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde40ce880 msgr2=0x7fcde40cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde8e27640 1 --2- 172.21.15.37:0/2958078389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde40ce880 0x7fcde40cfed0 secure :-1 s=READY pgs=2228 cs=0 l=1 rev1=1 crypto rx=0x7fcde406bc80 tx=0x7fcdcc01c460 comp rx=0 tx=0).stop 2024-04-25T09:04:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde8e27640 1 -- 172.21.15.37:0/2958078389 shutdown_connections 2024-04-25T09:04:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde8e27640 1 --2- 172.21.15.37:0/2958078389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde40ce880 0x7fcde40cfed0 unknown :-1 s=CLOSED pgs=2228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde8e27640 1 --2- 172.21.15.37:0/2958078389 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40d0e30 0x7fcde40d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde8e27640 1 --2- 172.21.15.37:0/2958078389 >> v2:172.21.15.203:3300/0 conn(0x7fcde40b2ac0 0x7fcde40ce340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde8e27640 1 -- 172.21.15.37:0/2958078389 >> 172.21.15.37:0/2958078389 conn(0x7fcde408b000 msgr2=0x7fcde40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde8e27640 1 -- 172.21.15.37:0/2958078389 shutdown_connections 2024-04-25T09:04:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde8e27640 1 -- 172.21.15.37:0/2958078389 wait complete. 2024-04-25T09:04:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde8e27640 1 Processor -- start 2024-04-25T09:04:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.397+0000 7fcde8e27640 1 -- start start 2024-04-25T09:04:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde8e27640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde40b2ac0 0x7fcde4151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde8e27640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde40ce880 0x7fcde40cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde8e27640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40d0e30 0x7fcde40b8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde8e27640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40d2070 con 0x7fcde40ce880 2024-04-25T09:04:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde8e27640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40d1ef0 con 0x7fcde40b2ac0 2024-04-25T09:04:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde8e27640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40d21f0 con 0x7fcde40d0e30 2024-04-25T09:04:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde40ce880 0x7fcde40cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde40ce880 0x7fcde40cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50158/0 (socket says 172.21.15.37:50158) 2024-04-25T09:04:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde1d9b640 1 -- 172.21.15.37:0/1904498653 learned_addr learned my addr 172.21.15.37:0/1904498653 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde259c640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde40b2ac0 0x7fcde4151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde2d9d640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40d0e30 0x7fcde40b8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde1d9b640 1 -- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40d0e30 msgr2=0x7fcde40b8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde1d9b640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40d0e30 0x7fcde40b8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde1d9b640 1 -- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde40b2ac0 msgr2=0x7fcde4151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde1d9b640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde40b2ac0 0x7fcde4151c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde1d9b640 1 -- 172.21.15.37:0/1904498653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde40cb600 con 0x7fcde40ce880 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde259c640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde40b2ac0 0x7fcde4151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde1d9b640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde40ce880 0x7fcde40cc6a0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc01ab10 tx=0x7fcdcc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcddaffd640 1 -- 172.21.15.37:0/1904498653 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0be3f0 con 0x7fcde40ce880 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcddaffd640 1 -- 172.21.15.37:0/1904498653 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdcc0be590 con 0x7fcde40ce880 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde8e27640 1 -- 172.21.15.37:0/1904498653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde40cb8e0 con 0x7fcde40ce880 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcddaffd640 1 -- 172.21.15.37:0/1904498653 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdcc0be940 con 0x7fcde40ce880 2024-04-25T09:04:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde8e27640 1 -- 172.21.15.37:0/1904498653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde40cbea0 con 0x7fcde40ce880 2024-04-25T09:04:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde8e27640 1 -- 172.21.15.37:0/1904498653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde404efa0 con 0x7fcde40ce880 2024-04-25T09:04:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcddaffd640 1 -- 172.21.15.37:0/1904498653 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fcdcc0b43f0 con 0x7fcde40ce880 2024-04-25T09:04:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcddaffd640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdc4066f60 0x7fcdc4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcddaffd640 1 -- 172.21.15.37:0/1904498653 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fcdcc11f540 con 0x7fcde40ce880 2024-04-25T09:04:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.401+0000 7fcde259c640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdc4066f60 0x7fcdc4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.405+0000 7fcde259c640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdc4066f60 0x7fcdc4069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009540 tx=0x7fcdd400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.405+0000 7fcddaffd640 1 -- 172.21.15.37:0/1904498653 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fcdcc0c4060 con 0x7fcde40ce880 2024-04-25T09:04:32.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.625+0000 7fcde8e27640 1 -- 172.21.15.37:0/1904498653 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) v1 -- 0x7fcde406c590 con 0x7fcde40ce880 2024-04-25T09:04:32.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.909+0000 7fcddaffd640 1 -- 172.21.15.37:0/1904498653 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v565) v1 ==== 144+0+0 (secure 0 0 0) 0x7fcdcc016110 con 0x7fcde40ce880 2024-04-25T09:04:32.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.913+0000 7fcde8e27640 1 -- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdc4066f60 msgr2=0x7fcdc4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.913+0000 7fcde8e27640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdc4066f60 0x7fcdc4069420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009540 tx=0x7fcdd400b040 comp rx=0 tx=0).stop 2024-04-25T09:04:32.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.913+0000 7fcde8e27640 1 -- 172.21.15.37:0/1904498653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde40ce880 msgr2=0x7fcde40cc6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:32.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.913+0000 7fcde8e27640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde40ce880 0x7fcde40cc6a0 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7fcdcc01ab10 tx=0x7fcdcc01aca0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.913+0000 7fcde8e27640 1 -- 172.21.15.37:0/1904498653 shutdown_connections 2024-04-25T09:04:32.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.913+0000 7fcde8e27640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fcdc4066f60 0x7fcdc4069420 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.913+0000 7fcde8e27640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fcde40d0e30 0x7fcde40b8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.913+0000 7fcde8e27640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcde40ce880 0x7fcde40cc6a0 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.913+0000 7fcde8e27640 1 --2- 172.21.15.37:0/1904498653 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fcde40b2ac0 0x7fcde4151c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:32.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.913+0000 7fcde8e27640 1 -- 172.21.15.37:0/1904498653 >> 172.21.15.37:0/1904498653 conn(0x7fcde408b000 msgr2=0x7fcde406cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:32.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.917+0000 7fcde8e27640 1 -- 172.21.15.37:0/1904498653 shutdown_connections 2024-04-25T09:04:32.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:32.917+0000 7fcde8e27640 1 -- 172.21.15.37:0/1904498653 wait complete. 2024-04-25T09:04:32.927 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-04-25T09:04:32.928 INFO:tasks.cephfs_test_runner: 2024-04-25T09:04:32.928 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T09:04:32.928 INFO:tasks.cephfs_test_runner:Ran 7 tests in 670.710s 2024-04-25T09:04:32.928 INFO:tasks.cephfs_test_runner: 2024-04-25T09:04:32.928 INFO:tasks.cephfs_test_runner:OK 2024-04-25T09:04:32.928 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T09:04:32.938 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T09:04:32.952 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T09:04:32.952 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:04:32.953 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:32.959 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:04:32.959 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-25T09:04:32.959 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:33.005 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16339604k,nr_inodes=4084901,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15777 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275304k,nr_inodes=818826,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275308k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:04:33.006 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:04:33.006 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:33.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:33.012 DEBUG:teuthology.orchestra.run.smithi203:> set -ex 2024-04-25T09:04:33.012 DEBUG:teuthology.orchestra.run.smithi203:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:33.060 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336428k,nr_inodes=4084107,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=21870 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274668k,nr_inodes=818667,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274672k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:33.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:33.061 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-25T09:04:33.067 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:04:33.067 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:04:33.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:33.068 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:33.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:33.123 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:33.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:33.181 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-25T09:04:33.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:33.370 DEBUG:teuthology.orchestra.run.smithi203:> ip netns list 2024-04-25T09:04:33.377 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T09:04:33.377 INFO:teuthology.orchestra.run.smithi203.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T09:04:33.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:33.377 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T09:04:33.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:33.436 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:04:33.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:33.496 DEBUG:teuthology.orchestra.run.smithi203:> sudo ip link delete ceph-brx 2024-04-25T09:04:33.665 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T09:04:33.675 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T09:04:33.675 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:33.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe2363f8640 1 Processor -- start 2024-04-25T09:04:33.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe2363f8640 1 -- start start 2024-04-25T09:04:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe2363f8640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fe2300ce510 0x7fe2300ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe2363f8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe230144ca0 0x7fe23006e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe2363f8640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2300cee30 con 0x7fe23006ec60 2024-04-25T09:04:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe2363f8640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2300cc2b0 con 0x7fe2300ce510 2024-04-25T09:04:33.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe2363f8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe23006e290 con 0x7fe230144ca0 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe234995640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe23006ec60 0x7fe230142680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56164/0 (socket says 172.21.15.37:56164) 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe234995640 1 -- 172.21.15.37:0/2420304637 learned_addr learned my addr 172.21.15.37:0/2420304637 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22ffff640 1 --2- 172.21.15.37:0/2420304637 >> v2:172.21.15.203:3300/0 conn(0x7fe2300ce510 0x7fe2300ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22f7fe640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe230144ca0 0x7fe23006e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1726090372 0 0) 0x7fe2300cee30 con 0x7fe23006ec60 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe218003530 con 0x7fe23006ec60 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2363445872 0 0) 0x7fe218003530 con 0x7fe23006ec60 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe230144ca0 msgr2=0x7fe23006e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe230144ca0 0x7fe23006e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 >> v2:172.21.15.203:3300/0 conn(0x7fe2300ce510 msgr2=0x7fe2300ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 --2- 172.21.15.37:0/2420304637 >> v2:172.21.15.203:3300/0 conn(0x7fe2300ce510 0x7fe2300ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2300ccc70 con 0x7fe23006ec60 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fe228001680 con 0x7fe23006ec60 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 >> v1:172.21.15.37:6789/0 conn(0x7fe23006ec60 legacy=0x7fe230142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe230144ca0 0x7fe218003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe218003fb0 0x7fe218004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218004d50 0x7fe2180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300cc2b0 con 0x7fe218004d50 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23006e290 con 0x7fe218003fb0 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21800ba80 con 0x7fe230144ca0 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe234995640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218004d50 0x7fe2180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22ffff640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe230144ca0 0x7fe218003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe22f7fe640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe218003fb0 0x7fe218004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe234995640 1 -- 172.21.15.37:0/2420304637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe230144ca0 msgr2=0x7fe218003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe234995640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe230144ca0 0x7fe218003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe234995640 1 -- 172.21.15.37:0/2420304637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe218003fb0 msgr2=0x7fe218004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe234995640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe218003fb0 0x7fe218004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.821+0000 7fe234995640 1 -- 172.21.15.37:0/2420304637 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe218003530 con 0x7fe218004d50 2024-04-25T09:04:33.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe234995640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218004d50 0x7fe2180051d0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fe23006f0c0 tx=0x7fe22801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6114f655ca56052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe228007d60 con 0x7fe218004d50 2024-04-25T09:04:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2280a9020 con 0x7fe218004d50 2024-04-25T09:04:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe22effd640 1 -- 172.21.15.37:0/2420304637 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2280a5b60 con 0x7fe218004d50 2024-04-25T09:04:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe2363f8640 1 -- 172.21.15.37:0/2420304637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218004d50 msgr2=0x7fe2180051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe2363f8640 1 --2- 172.21.15.37:0/2420304637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe218004d50 0x7fe2180051d0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7fe23006f0c0 tx=0x7fe22801cc60 comp rx=0 tx=0).stop 2024-04-25T09:04:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe234995640 1 -- 172.21.15.37:0/2420304637 reap_dead start 2024-04-25T09:04:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe2363f8640 1 -- 172.21.15.37:0/2420304637 shutdown_connections 2024-04-25T09:04:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe2363f8640 1 -- 172.21.15.37:0/2420304637 >> 172.21.15.37:0/2420304637 conn(0x7fe2300b11a0 msgr2=0x7fe2300b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe2363f8640 1 -- 172.21.15.37:0/2420304637 shutdown_connections 2024-04-25T09:04:33.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe2363f8640 1 -- 172.21.15.37:0/2420304637 wait complete. 2024-04-25T09:04:33.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe2363f8640 1 Processor -- start 2024-04-25T09:04:33.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.825+0000 7fe2363f8640 1 -- start start 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe2363f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe23013c010 0x7fe2301f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe2363f8640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe23015af40 0x7fe230140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe2363f8640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe230141350 0x7fe2300a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe2363f8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2300cc900 con 0x7fe23013c010 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe2363f8640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe23006df20 con 0x7fe230141350 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe2363f8640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe230142d90 con 0x7fe23015af40 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe23013c010 0x7fe2301f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe23013c010 0x7fe2301f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50174/0 (socket says 172.21.15.37:50174) 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22ffff640 1 -- 172.21.15.37:0/3257487894 learned_addr learned my addr 172.21.15.37:0/3257487894 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22f7fe640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe23015af40 0x7fe230140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe234995640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe230141350 0x7fe2300a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22ffff640 1 -- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe23015af40 msgr2=0x7fe230140e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22ffff640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe23015af40 0x7fe230140e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22ffff640 1 -- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe230141350 msgr2=0x7fe2300a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22ffff640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe230141350 0x7fe2300a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22ffff640 1 -- 172.21.15.37:0/3257487894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2301f5110 con 0x7fe23013c010 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22ffff640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe23013c010 0x7fe2301f48c0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fe220011b00 tx=0x7fe220009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22cff9640 1 -- 172.21.15.37:0/3257487894 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22000fc10 con 0x7fe23013c010 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22cff9640 1 -- 172.21.15.37:0/3257487894 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22000fdb0 con 0x7fe23013c010 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22cff9640 1 -- 172.21.15.37:0/3257487894 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe220014660 con 0x7fe23013c010 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe2363f8640 1 -- 172.21.15.37:0/3257487894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2301f53f0 con 0x7fe23013c010 2024-04-25T09:04:33.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe2363f8640 1 -- 172.21.15.37:0/3257487894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2300aa210 con 0x7fe23013c010 2024-04-25T09:04:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe2363f8640 1 -- 172.21.15.37:0/3257487894 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23004efa0 con 0x7fe23013c010 2024-04-25T09:04:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22cff9640 1 -- 172.21.15.37:0/3257487894 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe220012040 con 0x7fe23013c010 2024-04-25T09:04:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.829+0000 7fe22cff9640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe21c066f60 0x7fe21c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.833+0000 7fe22cff9640 1 -- 172.21.15.37:0/3257487894 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe220085610 con 0x7fe23013c010 2024-04-25T09:04:33.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.833+0000 7fe22f7fe640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe21c066f60 0x7fe21c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:33.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.833+0000 7fe22f7fe640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe21c066f60 0x7fe21c069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe2300ac8d0 tx=0x7fe22400b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:33.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.837+0000 7fe22cff9640 1 -- 172.21.15.37:0/3257487894 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe220019030 con 0x7fe23013c010 2024-04-25T09:04:33.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.997+0000 7fe2363f8640 1 -- 172.21.15.37:0/3257487894 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe2300b1cc0 con 0x7fe21c066f60 2024-04-25T09:04:33.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:33.997+0000 7fe22cff9640 1 -- 172.21.15.37:0/3257487894 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13412 (secure 0 0 0) 0x7fe2300b1cc0 con 0x7fe21c066f60 2024-04-25T09:04:33.987 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:33.987 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T09:04:33.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 -- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe21c066f60 msgr2=0x7fe21c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe21c066f60 0x7fe21c069420 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe2300ac8d0 tx=0x7fe22400b040 comp rx=0 tx=0).stop 2024-04-25T09:04:33.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 -- 172.21.15.37:0/3257487894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe23013c010 msgr2=0x7fe2301f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:33.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe23013c010 0x7fe2301f48c0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7fe220011b00 tx=0x7fe220009f70 comp rx=0 tx=0).stop 2024-04-25T09:04:33.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 -- 172.21.15.37:0/3257487894 shutdown_connections 2024-04-25T09:04:33.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe21c066f60 0x7fe21c069420 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe230141350 0x7fe2300a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe23015af40 0x7fe230140e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 --2- 172.21.15.37:0/3257487894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe23013c010 0x7fe2301f48c0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:33.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 -- 172.21.15.37:0/3257487894 >> 172.21.15.37:0/3257487894 conn(0x7fe2300b11a0 msgr2=0x7fe2300ce8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:33.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 -- 172.21.15.37:0/3257487894 shutdown_connections 2024-04-25T09:04:33.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.001+0000 7fe2363f8640 1 -- 172.21.15.37:0/3257487894 wait complete. 2024-04-25T09:04:34.006 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-04-25T09:04:33.377195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279400,"kb_used_data":29984,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695320,"statfs":{"total":773094113280,"available":772808007680,"internally_reserved":0,"allocated":30703616,"data_stored":23909269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390584,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-174,"num_write_kb":-8293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001750"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":372,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.026481+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T09:04:13.026481+0000","last_peered":"2024-04-25T09:04:13.026481+0000","last_clean":"2024-04-25T09:04:13.026481+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T09:04:13.026481+0000","last_undegraded":"2024-04-25T09:04:13.026481+0000","last_fullsized":"2024-04-25T09:04:13.026481+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33320,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2195456,"data_stored":1334330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36820,"kb_used_data":5644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":5779456,"data_stored":4938065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36828,"kb_used_data":5648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335012,"statfs":{"total":96636764160,"available":96599052288,"internally_reserved":0,"allocated":5783552,"data_stored":4937643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":5795840,"data_stored":4938797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1597440,"data_stored":746166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":743184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":5648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":5783552,"data_stored":4937510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2183168,"data_stored":1333574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:34.007 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:34.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc242b73640 1 Processor -- start 2024-04-25T09:04:34.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc242b73640 1 -- start start 2024-04-25T09:04:34.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc242b73640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc23c15d970 0x7fc23c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc242b73640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc23c0d3de0 0x7fc23c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc242b73640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc23c0cf460 con 0x7fc23c06d1f0 2024-04-25T09:04:34.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc242b73640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23c06c710 con 0x7fc23c0d3de0 2024-04-25T09:04:34.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc242b73640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c06c9f0 con 0x7fc23c15d970 2024-04-25T09:04:34.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc241110640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc23c0d3de0 0x7fc23c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc24090f640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc23c15d970 0x7fc23c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc23bfff640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fc23c06d1f0 0x7fc23c0d36d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:48036/0 (socket says 172.21.15.37:48036) 2024-04-25T09:04:34.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc23bfff640 1 -- 172.21.15.37:0/4180824687 learned_addr learned my addr 172.21.15.37:0/4180824687 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:34.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc24090f640 1 -- 172.21.15.37:0/4180824687 >> v2:172.21.15.203:3300/0 conn(0x7fc23c0d3de0 msgr2=0x7fc23c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:34.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc24090f640 1 --2- 172.21.15.37:0/4180824687 >> v2:172.21.15.203:3300/0 conn(0x7fc23c0d3de0 0x7fc23c06d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc23b7fe640 1 -- 172.21.15.37:0/4180824687 <== mon.1 v1:172.21.15.203:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 630330828 0 0) 0x7fc23c0cf460 con 0x7fc23c06d1f0 2024-04-25T09:04:34.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc24090f640 1 -- 172.21.15.37:0/4180824687 >> v1:172.21.15.203:6789/0 conn(0x7fc23c06d1f0 legacy=0x7fc23c0d36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.145+0000 7fc24090f640 1 -- 172.21.15.37:0/4180824687 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c0cf740 con 0x7fc23c15d970 2024-04-25T09:04:34.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc24090f640 1 --2- 172.21.15.37:0/4180824687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc23c15d970 0x7fc23c0d1340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2240097c0 tx=0x7fc22401c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37364b773e50b2fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:34.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc23b7fe640 1 -- 172.21.15.37:0/4180824687 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2240b0070 con 0x7fc23c15d970 2024-04-25T09:04:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc23b7fe640 1 -- 172.21.15.37:0/4180824687 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2240ab600 con 0x7fc23c15d970 2024-04-25T09:04:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc23b7fe640 1 -- 172.21.15.37:0/4180824687 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2240ab9d0 con 0x7fc23c15d970 2024-04-25T09:04:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc242b73640 1 -- 172.21.15.37:0/4180824687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc23c15d970 msgr2=0x7fc23c0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc242b73640 1 --2- 172.21.15.37:0/4180824687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc23c15d970 0x7fc23c0d1340 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc2240097c0 tx=0x7fc22401c380 comp rx=0 tx=0).stop 2024-04-25T09:04:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc242b73640 1 -- 172.21.15.37:0/4180824687 shutdown_connections 2024-04-25T09:04:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc242b73640 1 --2- 172.21.15.37:0/4180824687 >> v2:172.21.15.203:3300/0 conn(0x7fc23c0d3de0 0x7fc23c06d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc242b73640 1 --2- 172.21.15.37:0/4180824687 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc23c15d970 0x7fc23c0d1340 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc242b73640 1 -- 172.21.15.37:0/4180824687 >> 172.21.15.37:0/4180824687 conn(0x7fc23c09cae0 msgr2=0x7fc23c08b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc242b73640 1 -- 172.21.15.37:0/4180824687 shutdown_connections 2024-04-25T09:04:34.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc242b73640 1 -- 172.21.15.37:0/4180824687 wait complete. 2024-04-25T09:04:34.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc242b73640 1 Processor -- start 2024-04-25T09:04:34.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.149+0000 7fc242b73640 1 -- start start 2024-04-25T09:04:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc242b73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc23c06d1f0 0x7fc23c14ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc242b73640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc23c0d3de0 0x7fc23c0b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc242b73640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc23c15d970 0x7fc23c1f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc242b73640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c0cf190 con 0x7fc23c06d1f0 2024-04-25T09:04:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc242b73640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c15df60 con 0x7fc23c15d970 2024-04-25T09:04:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc242b73640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23c0d1a90 con 0x7fc23c0d3de0 2024-04-25T09:04:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc24090f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc23c06d1f0 0x7fc23c14ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc24090f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc23c06d1f0 0x7fc23c14ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50188/0 (socket says 172.21.15.37:50188) 2024-04-25T09:04:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc24090f640 1 -- 172.21.15.37:0/2145948982 learned_addr learned my addr 172.21.15.37:0/2145948982 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc23bfff640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc23c0d3de0 0x7fc23c0b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc241110640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc23c15d970 0x7fc23c1f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc24090f640 1 -- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc23c0d3de0 msgr2=0x7fc23c0b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc24090f640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc23c0d3de0 0x7fc23c0b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc24090f640 1 -- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc23c15d970 msgr2=0x7fc23c1f9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc24090f640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc23c15d970 0x7fc23c1f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc24090f640 1 -- 172.21.15.37:0/2145948982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c1503b0 con 0x7fc23c06d1f0 2024-04-25T09:04:34.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc24090f640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc23c06d1f0 0x7fc23c14ca30 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fc224019fb0 tx=0x7fc224019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:34.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc2397fa640 1 -- 172.21.15.37:0/2145948982 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2240b8020 con 0x7fc23c06d1f0 2024-04-25T09:04:34.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc2397fa640 1 -- 172.21.15.37:0/2145948982 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2240b4410 con 0x7fc23c06d1f0 2024-04-25T09:04:34.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc242b73640 1 -- 172.21.15.37:0/2145948982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23c14d6f0 con 0x7fc23c06d1f0 2024-04-25T09:04:34.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc2397fa640 1 -- 172.21.15.37:0/2145948982 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2240b0040 con 0x7fc23c06d1f0 2024-04-25T09:04:34.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc242b73640 1 -- 172.21.15.37:0/2145948982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23c14dcb0 con 0x7fc23c06d1f0 2024-04-25T09:04:34.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc242b73640 1 -- 172.21.15.37:0/2145948982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23c04efa0 con 0x7fc23c06d1f0 2024-04-25T09:04:34.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc2397fa640 1 -- 172.21.15.37:0/2145948982 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc2240bb020 con 0x7fc23c06d1f0 2024-04-25T09:04:34.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc2397fa640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc21c066f60 0x7fc21c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.153+0000 7fc2397fa640 1 -- 172.21.15.37:0/2145948982 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc22411eaf0 con 0x7fc23c06d1f0 2024-04-25T09:04:34.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.157+0000 7fc23bfff640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc21c066f60 0x7fc21c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.157+0000 7fc23bfff640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc21c066f60 0x7fc21c069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc23c15c770 tx=0x7fc22c007ce0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:34.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.157+0000 7fc2397fa640 1 -- 172.21.15.37:0/2145948982 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fc2240c2030 con 0x7fc23c06d1f0 2024-04-25T09:04:34.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.317+0000 7fc242b73640 1 -- 172.21.15.37:0/2145948982 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc23c0cfd30 con 0x7fc21c066f60 2024-04-25T09:04:34.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.317+0000 7fc2397fa640 1 -- 172.21.15.37:0/2145948982 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13412 (secure 0 0 0) 0x7fc23c0cfd30 con 0x7fc21c066f60 2024-04-25T09:04:34.308 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:34.308 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T09:04:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 -- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc21c066f60 msgr2=0x7fc21c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc21c066f60 0x7fc21c069420 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc23c15c770 tx=0x7fc22c007ce0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 -- 172.21.15.37:0/2145948982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc23c06d1f0 msgr2=0x7fc23c14ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc23c06d1f0 0x7fc23c14ca30 secure :-1 s=READY pgs=2233 cs=0 l=1 rev1=1 crypto rx=0x7fc224019fb0 tx=0x7fc224019120 comp rx=0 tx=0).stop 2024-04-25T09:04:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 -- 172.21.15.37:0/2145948982 shutdown_connections 2024-04-25T09:04:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc21c066f60 0x7fc21c069420 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc23c15d970 0x7fc23c1f9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc23c0d3de0 0x7fc23c0b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 --2- 172.21.15.37:0/2145948982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc23c06d1f0 0x7fc23c14ca30 unknown :-1 s=CLOSED pgs=2233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 -- 172.21.15.37:0/2145948982 >> 172.21.15.37:0/2145948982 conn(0x7fc23c09cae0 msgr2=0x7fc23c0d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:34.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 -- 172.21.15.37:0/2145948982 shutdown_connections 2024-04-25T09:04:34.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.321+0000 7fc242b73640 1 -- 172.21.15.37:0/2145948982 wait complete. 2024-04-25T09:04:34.327 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-04-25T09:04:33.377195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279400,"kb_used_data":29984,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695320,"statfs":{"total":773094113280,"available":772808007680,"internally_reserved":0,"allocated":30703616,"data_stored":23909269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390584,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-174,"num_write_kb":-8293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001750"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":372,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.026481+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T09:04:13.026481+0000","last_peered":"2024-04-25T09:04:13.026481+0000","last_clean":"2024-04-25T09:04:13.026481+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T09:04:13.026481+0000","last_undegraded":"2024-04-25T09:04:13.026481+0000","last_fullsized":"2024-04-25T09:04:13.026481+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33320,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2195456,"data_stored":1334330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36820,"kb_used_data":5644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":5779456,"data_stored":4938065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36828,"kb_used_data":5648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335012,"statfs":{"total":96636764160,"available":96599052288,"internally_reserved":0,"allocated":5783552,"data_stored":4937643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":5795840,"data_stored":4938797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1597440,"data_stored":746166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":743184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":5648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":5783552,"data_stored":4937510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2183168,"data_stored":1333574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:34.327 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T09:04:34.328 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:34.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9728e09640 1 Processor -- start 2024-04-25T09:04:34.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9728e09640 1 -- start start 2024-04-25T09:04:34.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9728e09640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f97240ce510 0x7f97240ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9728e09640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97241451f0 0x7f9724142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9728e09640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97240cee30 con 0x7f9724144ca0 2024-04-25T09:04:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9728e09640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97240cc2b0 con 0x7f97241451f0 2024-04-25T09:04:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9728e09640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f972406e290 con 0x7f97240ce510 2024-04-25T09:04:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9722d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97241451f0 0x7f9724142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9722d9d640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f97241451f0 0x7f9724142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50198/0 (socket says 172.21.15.37:50198) 2024-04-25T09:04:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9722d9d640 1 -- 172.21.15.37:0/1383770983 learned_addr learned my addr 172.21.15.37:0/1383770983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f972259c640 1 --2- 172.21.15.37:0/1383770983 >> v2:172.21.15.203:3300/0 conn(0x7f97240ce510 0x7f97240ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9722d9d640 1 -- 172.21.15.37:0/1383770983 >> v2:172.21.15.203:3300/0 conn(0x7f97240ce510 msgr2=0x7f97240ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9722d9d640 1 --2- 172.21.15.37:0/1383770983 >> v2:172.21.15.203:3300/0 conn(0x7f97240ce510 0x7f97240ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9722d9d640 1 -- 172.21.15.37:0/1383770983 >> v1:172.21.15.203:6789/0 conn(0x7f9724144ca0 legacy=0x7f972406e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:34.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.457+0000 7f9722d9d640 1 -- 172.21.15.37:0/1383770983 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240ccc70 con 0x7f97241451f0 2024-04-25T09:04:34.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9722d9d640 1 --2- 172.21.15.37:0/1383770983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97241451f0 0x7f9724142680 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f9724144c50 tx=0x7f971c01f1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba3daeef66959ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f972159a640 1 -- 172.21.15.37:0/1383770983 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c0b4070 con 0x7f97241451f0 2024-04-25T09:04:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f972159a640 1 -- 172.21.15.37:0/1383770983 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971c0af400 con 0x7f97241451f0 2024-04-25T09:04:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f972159a640 1 -- 172.21.15.37:0/1383770983 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971c0af7d0 con 0x7f97241451f0 2024-04-25T09:04:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 -- 172.21.15.37:0/1383770983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97241451f0 msgr2=0x7f9724142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 --2- 172.21.15.37:0/1383770983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97241451f0 0x7f9724142680 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f9724144c50 tx=0x7f971c01f1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 -- 172.21.15.37:0/1383770983 shutdown_connections 2024-04-25T09:04:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 --2- 172.21.15.37:0/1383770983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97241451f0 0x7f9724142680 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 --2- 172.21.15.37:0/1383770983 >> v2:172.21.15.203:3300/0 conn(0x7f97240ce510 0x7f97240ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 -- 172.21.15.37:0/1383770983 >> 172.21.15.37:0/1383770983 conn(0x7f97240b11a0 msgr2=0x7f97240b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 -- 172.21.15.37:0/1383770983 shutdown_connections 2024-04-25T09:04:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 -- 172.21.15.37:0/1383770983 wait complete. 2024-04-25T09:04:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 Processor -- start 2024-04-25T09:04:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 -- start start 2024-04-25T09:04:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97240ce510 0x7f97240ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9724144ca0 0x7f972415b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97241451f0 0x7f97241e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972406ee30 con 0x7f9724144ca0 2024-04-25T09:04:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972406df20 con 0x7f97241451f0 2024-04-25T09:04:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9728e09640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97240cc900 con 0x7f97240ce510 2024-04-25T09:04:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9721d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9724144ca0 0x7f972415b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9721d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9724144ca0 0x7f972415b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50210/0 (socket says 172.21.15.37:50210) 2024-04-25T09:04:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9721d9b640 1 -- 172.21.15.37:0/305359953 learned_addr learned my addr 172.21.15.37:0/305359953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.461+0000 7f9722d9d640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97241451f0 0x7f97241e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f972259c640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97240ce510 0x7f97240ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9721d9b640 1 -- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97240ce510 msgr2=0x7f97240ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9721d9b640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97240ce510 0x7f97240ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9721d9b640 1 -- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97241451f0 msgr2=0x7f97241e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9721d9b640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97241451f0 0x7f97241e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9721d9b640 1 -- 172.21.15.37:0/305359953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97241e1820 con 0x7f9724144ca0 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9722d9d640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97241451f0 0x7f97241e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9721d9b640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9724144ca0 0x7f972415b180 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f971800c410 tx=0x7f971800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9712ffd640 1 -- 172.21.15.37:0/305359953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718011790 con 0x7f9724144ca0 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9712ffd640 1 -- 172.21.15.37:0/305359953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9718017070 con 0x7f9724144ca0 2024-04-25T09:04:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9728e09640 1 -- 172.21.15.37:0/305359953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97241e1b00 con 0x7f9724144ca0 2024-04-25T09:04:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9712ffd640 1 -- 172.21.15.37:0/305359953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718011b70 con 0x7f9724144ca0 2024-04-25T09:04:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9728e09640 1 -- 172.21.15.37:0/305359953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97241e20c0 con 0x7f9724144ca0 2024-04-25T09:04:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9728e09640 1 -- 172.21.15.37:0/305359953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972404efa0 con 0x7f9724144ca0 2024-04-25T09:04:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9712ffd640 1 -- 172.21.15.37:0/305359953 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f9718011d10 con 0x7f9724144ca0 2024-04-25T09:04:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9712ffd640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9704066f60 0x7f9704069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f9712ffd640 1 -- 172.21.15.37:0/305359953 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f9718084bc0 con 0x7f9724144ca0 2024-04-25T09:04:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.465+0000 7f972259c640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9704066f60 0x7f9704069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.469+0000 7f972259c640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9704066f60 0x7f9704069420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f971400a970 tx=0x7f9714008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.469+0000 7f9712ffd640 1 -- 172.21.15.37:0/305359953 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f9718021020 con 0x7f9724144ca0 2024-04-25T09:04:34.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.629+0000 7f9728e09640 1 -- 172.21.15.37:0/305359953 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97240b1cc0 con 0x7f9704066f60 2024-04-25T09:04:34.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.629+0000 7f9712ffd640 1 -- 172.21.15.37:0/305359953 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13412 (secure 0 0 0) 0x7f97240b1cc0 con 0x7f9704066f60 2024-04-25T09:04:34.619 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:34.620 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T09:04:34.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 -- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9704066f60 msgr2=0x7f9704069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9704066f60 0x7f9704069420 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f971400a970 tx=0x7f9714008040 comp rx=0 tx=0).stop 2024-04-25T09:04:34.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 -- 172.21.15.37:0/305359953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9724144ca0 msgr2=0x7f972415b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9724144ca0 0x7f972415b180 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f971800c410 tx=0x7f971800c8d0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 -- 172.21.15.37:0/305359953 shutdown_connections 2024-04-25T09:04:34.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f9704066f60 0x7f9704069420 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f97241451f0 0x7f97241e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9724144ca0 0x7f972415b180 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 --2- 172.21.15.37:0/305359953 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f97240ce510 0x7f97240ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 -- 172.21.15.37:0/305359953 >> 172.21.15.37:0/305359953 conn(0x7f97240b11a0 msgr2=0x7f9724071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:34.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 -- 172.21.15.37:0/305359953 shutdown_connections 2024-04-25T09:04:34.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.633+0000 7f9728e09640 1 -- 172.21.15.37:0/305359953 wait complete. 2024-04-25T09:04:34.638 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-04-25T09:04:33.377195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":279400,"kb_used_data":29984,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754695320,"statfs":{"total":773094113280,"available":772808007680,"internally_reserved":0,"allocated":30703616,"data_stored":23909269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8390584,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-262,"num_write":-174,"num_write_kb":-8293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001750"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":372,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.026481+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T09:04:13.026481+0000","last_peered":"2024-04-25T09:04:13.026481+0000","last_clean":"2024-04-25T09:04:13.026481+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T09:04:13.026481+0000","last_undegraded":"2024-04-25T09:04:13.026481+0000","last_fullsized":"2024-04-25T09:04:13.026481+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33320,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2195456,"data_stored":1334330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36820,"kb_used_data":5644,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":5779456,"data_stored":4938065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36828,"kb_used_data":5648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335012,"statfs":{"total":96636764160,"available":96599052288,"internally_reserved":0,"allocated":5783552,"data_stored":4937643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36836,"kb_used_data":5660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335004,"statfs":{"total":96636764160,"available":96599044096,"internally_reserved":0,"allocated":5795840,"data_stored":4938797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32736,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339104,"statfs":{"total":96636764160,"available":96603242496,"internally_reserved":0,"allocated":1597440,"data_stored":746166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32728,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339112,"statfs":{"total":96636764160,"available":96603250688,"internally_reserved":0,"allocated":1585152,"data_stored":743184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":5648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":5783552,"data_stored":4937510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2183168,"data_stored":1333574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:34.639 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:34.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.773+0000 7f92cd481640 1 Processor -- start 2024-04-25T09:04:34.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.773+0000 7f92cd481640 1 -- start start 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f92c809aa80 0x7f92c80b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f92c80b19f0 0x7f92c806c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92c8099cc0 con 0x7f92c80b24d0 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c809aee0 con 0x7f92c809aa80 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92c8099a70 con 0x7f92c80b19f0 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c67fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f92c809aa80 0x7f92c80b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c67fc640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f92c809aa80 0x7f92c80b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50218/0 (socket says 172.21.15.37:50218) 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c67fc640 1 -- 172.21.15.37:0/1447515398 learned_addr learned my addr 172.21.15.37:0/1447515398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c77fe640 1 --2- 172.21.15.37:0/1447515398 >> v2:172.21.15.203:3300/0 conn(0x7f92c80b19f0 0x7f92c806c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c67fc640 1 -- 172.21.15.37:0/1447515398 >> v2:172.21.15.203:3300/0 conn(0x7f92c80b19f0 msgr2=0x7f92c806c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:34.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c67fc640 1 --2- 172.21.15.37:0/1447515398 >> v2:172.21.15.203:3300/0 conn(0x7f92c80b19f0 0x7f92c806c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c67fc640 1 -- 172.21.15.37:0/1447515398 >> v1:172.21.15.203:6789/0 conn(0x7f92c80b24d0 legacy=0x7f92c80b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:34.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c67fc640 1 -- 172.21.15.37:0/1447515398 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c8099fa0 con 0x7f92c809aa80 2024-04-25T09:04:34.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c77fe640 1 --2- 172.21.15.37:0/1447515398 >> v2:172.21.15.203:3300/0 conn(0x7f92c80b19f0 0x7f92c806c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:34.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c67fc640 1 --2- 172.21.15.37:0/1447515398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c809aa80 0x7f92c80b14b0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f92c8074750 tx=0x7f92b801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc4e1baba6d05c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:34.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c5ffb640 1 -- 172.21.15.37:0/1447515398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b80b2070 con 0x7f92c809aa80 2024-04-25T09:04:34.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c5ffb640 1 -- 172.21.15.37:0/1447515398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b80ad640 con 0x7f92c809aa80 2024-04-25T09:04:34.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92c5ffb640 1 -- 172.21.15.37:0/1447515398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b80ada10 con 0x7f92c809aa80 2024-04-25T09:04:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 -- 172.21.15.37:0/1447515398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c809aa80 msgr2=0x7f92c80b14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 --2- 172.21.15.37:0/1447515398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c809aa80 0x7f92c80b14b0 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7f92c8074750 tx=0x7f92b801c390 comp rx=0 tx=0).stop 2024-04-25T09:04:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 -- 172.21.15.37:0/1447515398 shutdown_connections 2024-04-25T09:04:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 --2- 172.21.15.37:0/1447515398 >> v2:172.21.15.203:3300/0 conn(0x7f92c80b19f0 0x7f92c806c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 --2- 172.21.15.37:0/1447515398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c809aa80 0x7f92c80b14b0 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 -- 172.21.15.37:0/1447515398 >> 172.21.15.37:0/1447515398 conn(0x7f92c80b3020 msgr2=0x7f92c80b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.777+0000 7f92cd481640 1 -- 172.21.15.37:0/1447515398 shutdown_connections 2024-04-25T09:04:34.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 -- 172.21.15.37:0/1447515398 wait complete. 2024-04-25T09:04:34.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 Processor -- start 2024-04-25T09:04:34.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 -- start start 2024-04-25T09:04:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92c809aa80 0x7f92c809d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c80b19f0 0x7f92c815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92c80b24d0 0x7f92c815a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c815e020 con 0x7f92c80b19f0 2024-04-25T09:04:34.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c815dea0 con 0x7f92c80b24d0 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c816a770 con 0x7f92c809aa80 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c80b19f0 0x7f92c815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c80b19f0 0x7f92c815a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50226/0 (socket says 172.21.15.37:50226) 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c67fc640 1 -- 172.21.15.37:0/2479119179 learned_addr learned my addr 172.21.15.37:0/2479119179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c6ffd640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92c809aa80 0x7f92c809d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c77fe640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92c80b24d0 0x7f92c815a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c67fc640 1 -- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92c809aa80 msgr2=0x7f92c809d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c67fc640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92c809aa80 0x7f92c809d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c67fc640 1 -- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92c80b24d0 msgr2=0x7f92c815a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c67fc640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92c80b24d0 0x7f92c815a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c67fc640 1 -- 172.21.15.37:0/2479119179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c809d870 con 0x7f92c80b19f0 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c6ffd640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92c809aa80 0x7f92c809d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92c67fc640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c80b19f0 0x7f92c815a390 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f92b80b0040 tx=0x7f92b8019130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:34.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92af7fe640 1 -- 172.21.15.37:0/2479119179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b80bd070 con 0x7f92c80b19f0 2024-04-25T09:04:34.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92af7fe640 1 -- 172.21.15.37:0/2479119179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b80b6770 con 0x7f92c80b19f0 2024-04-25T09:04:34.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 -- 172.21.15.37:0/2479119179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c809db50 con 0x7f92c80b19f0 2024-04-25T09:04:34.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92af7fe640 1 -- 172.21.15.37:0/2479119179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b80b2070 con 0x7f92c80b19f0 2024-04-25T09:04:34.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 -- 172.21.15.37:0/2479119179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c809e110 con 0x7f92c80b19f0 2024-04-25T09:04:34.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.781+0000 7f92cd481640 1 -- 172.21.15.37:0/2479119179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c804efa0 con 0x7f92c80b19f0 2024-04-25T09:04:34.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.785+0000 7f92af7fe640 1 -- 172.21.15.37:0/2479119179 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f92b80c0070 con 0x7f92c80b19f0 2024-04-25T09:04:34.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.785+0000 7f92af7fe640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f92b4066f60 0x7f92b4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:34.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.785+0000 7f92af7fe640 1 -- 172.21.15.37:0/2479119179 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f92b811ec90 con 0x7f92c80b19f0 2024-04-25T09:04:34.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.785+0000 7f92c6ffd640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f92b4066f60 0x7f92b4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:34.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.785+0000 7f92c6ffd640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f92b4066f60 0x7f92b4069420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00a850 tx=0x7f92bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:34.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.789+0000 7f92af7fe640 1 -- 172.21.15.37:0/2479119179 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f92b80160d0 con 0x7f92c80b19f0 2024-04-25T09:04:34.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.949+0000 7f92cd481640 1 -- 172.21.15.37:0/2479119179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92c81611d0 con 0x7f92c80b19f0 2024-04-25T09:04:34.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.949+0000 7f92af7fe640 1 -- 172.21.15.37:0/2479119179 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v130) v1 ==== 74+0+13527 (secure 0 0 0) 0x7f92b80efdf0 con 0x7f92c80b19f0 2024-04-25T09:04:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:34.940 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":130,"fsid":"3ecc7551-d5a4-47e7-a9cf-6943742cae5b","created":"2024-04-25T08:52:06.325308+0000","modified":"2024-04-25T09:04:31.896959+0000","last_up_change":"2024-04-25T08:52:17.663387+0000","last_in_change":"2024-04-25T08:52:08.607946+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:52:19.193245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7eda0189-e0df-47ea-99d9-17bbe2fe40cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6825","nonce":1116141266}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6827","nonce":1116141266}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6831","nonce":1116141266}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":1116141266},{"type":"v1","addr":"172.21.15.37:6829","nonce":1116141266}]},"public_addr":"172.21.15.37:6825/1116141266","cluster_addr":"172.21.15.37:6827/1116141266","heartbeat_back_addr":"172.21.15.37:6831/1116141266","heartbeat_front_addr":"172.21.15.37:6829/1116141266","state":["exists","up"]},{"osd":1,"uuid":"8b0293a9-db94-43c9-bd5b-fe75890d1599","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6817","nonce":2944303862}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6819","nonce":2944303862}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6823","nonce":2944303862}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2944303862},{"type":"v1","addr":"172.21.15.37:6821","nonce":2944303862}]},"public_addr":"172.21.15.37:6817/2944303862","cluster_addr":"172.21.15.37:6819/2944303862","heartbeat_back_addr":"172.21.15.37:6823/2944303862","heartbeat_front_addr":"172.21.15.37:6821/2944303862","state":["exists","up"]},{"osd":2,"uuid":"9c8f8bd3-2f80-401c-8091-4d29cb03cf4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6809","nonce":332967247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6811","nonce":332967247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6815","nonce":332967247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":332967247},{"type":"v1","addr":"172.21.15.37:6813","nonce":332967247}]},"public_addr":"172.21.15.37:6809/332967247","cluster_addr":"172.21.15.37:6811/332967247","heartbeat_back_addr":"172.21.15.37:6815/332967247","heartbeat_front_addr":"172.21.15.37:6813/332967247","state":["exists","up"]},{"osd":3,"uuid":"38a1085b-a817-4560-8f4e-297ab640dd94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6801","nonce":4126478460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6803","nonce":4126478460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6807","nonce":4126478460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":4126478460},{"type":"v1","addr":"172.21.15.37:6805","nonce":4126478460}]},"public_addr":"172.21.15.37:6801/4126478460","cluster_addr":"172.21.15.37:6803/4126478460","heartbeat_back_addr":"172.21.15.37:6807/4126478460","heartbeat_front_addr":"172.21.15.37:6805/4126478460","state":["exists","up"]},{"osd":4,"uuid":"501d204e-2b71-406d-8505-6fe0dd15cff8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6802","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6804","nonce":714177927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6806","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6809","nonce":714177927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6818","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6820","nonce":714177927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6812","nonce":714177927},{"type":"v1","addr":"172.21.15.203:6816","nonce":714177927}]},"public_addr":"172.21.15.203:6804/714177927","cluster_addr":"172.21.15.203:6809/714177927","heartbeat_back_addr":"172.21.15.203:6820/714177927","heartbeat_front_addr":"172.21.15.203:6816/714177927","state":["exists","up"]},{"osd":5,"uuid":"6b27cd03-a272-441c-95eb-af689e2ffc1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6808","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6811","nonce":213476662}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6815","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6817","nonce":213476662}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6822","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6823","nonce":213476662}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6819","nonce":213476662},{"type":"v1","addr":"172.21.15.203:6821","nonce":213476662}]},"public_addr":"172.21.15.203:6811/213476662","cluster_addr":"172.21.15.203:6817/213476662","heartbeat_back_addr":"172.21.15.203:6823/213476662","heartbeat_front_addr":"172.21.15.203:6821/213476662","state":["exists","up"]},{"osd":6,"uuid":"2eee4199-0713-4ff2-bd6d-d8b2f3e10151","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6800","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6801","nonce":467003745}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6803","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6805","nonce":467003745}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6813","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6814","nonce":467003745}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6807","nonce":467003745},{"type":"v1","addr":"172.21.15.203:6810","nonce":467003745}]},"public_addr":"172.21.15.203:6801/467003745","cluster_addr":"172.21.15.203:6805/467003745","heartbeat_back_addr":"172.21.15.203:6814/467003745","heartbeat_front_addr":"172.21.15.203:6810/467003745","state":["exists","up"]},{"osd":7,"uuid":"bb76d7a8-1a44-4aec-a592-8112bfeab2c8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":123,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6824","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6825","nonce":2957998038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6826","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6827","nonce":2957998038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6830","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6831","nonce":2957998038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6828","nonce":2957998038},{"type":"v1","addr":"172.21.15.203:6829","nonce":2957998038}]},"public_addr":"172.21.15.203:6825/2957998038","cluster_addr":"172.21.15.203:6827/2957998038","heartbeat_back_addr":"172.21.15.203:6831/2957998038","heartbeat_front_addr":"172.21.15.203:6829/2957998038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.084036+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.269596+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.222715+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.954957+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.139055+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:13.980606+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.267025+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:52:14.184934+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6835/2361387213":"2024-04-26T09:04:28.877322+0000","172.21.15.37:6834/2361387213":"2024-04-26T09:04:28.877322+0000","172.21.15.37:6835/2159545482":"2024-04-26T09:04:12.937783+0000","172.21.15.37:6834/2159545482":"2024-04-26T09:04:12.937783+0000","172.21.15.37:6835/2748919477":"2024-04-26T09:03:58.978336+0000","172.21.15.37:6834/2748919477":"2024-04-26T09:03:58.978336+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 -- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f92b4066f60 msgr2=0x7f92b4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f92b4066f60 0x7f92b4069420 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f92bc00a850 tx=0x7f92bc008040 comp rx=0 tx=0).stop 2024-04-25T09:04:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 -- 172.21.15.37:0/2479119179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c80b19f0 msgr2=0x7f92c815a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c80b19f0 0x7f92c815a390 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7f92b80b0040 tx=0x7f92b8019130 comp rx=0 tx=0).stop 2024-04-25T09:04:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 -- 172.21.15.37:0/2479119179 shutdown_connections 2024-04-25T09:04:34.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f92b4066f60 0x7f92b4069420 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f92c80b24d0 0x7f92c815a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92c80b19f0 0x7f92c815a390 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 --2- 172.21.15.37:0/2479119179 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f92c809aa80 0x7f92c809d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 -- 172.21.15.37:0/2479119179 >> 172.21.15.37:0/2479119179 conn(0x7f92c80b3020 msgr2=0x7f92c80b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 -- 172.21.15.37:0/2479119179 shutdown_connections 2024-04-25T09:04:34.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:34.953+0000 7f92cd481640 1 -- 172.21.15.37:0/2479119179 wait complete. 2024-04-25T09:04:35.960 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T09:04:35.960 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:36.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0e14d2640 1 Processor -- start 2024-04-25T09:04:36.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0e14d2640 1 -- start start 2024-04-25T09:04:36.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0e14d2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fc0dc0b36d0 0x7fc0dc0b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0e14d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0dc0cd000 0x7fc0dc0b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0e14d2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0dc0b3bd0 con 0x7fc0dc159f20 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0e14d2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0dc0b3eb0 con 0x7fc0dc0b36d0 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0e14d2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc0b4190 con 0x7fc0dc0cd000 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0daffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc0dc159f20 0x7fc0dc06d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56170/0 (socket says 172.21.15.37:56170) 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0daffd640 1 -- 172.21.15.37:0/1389381036 learned_addr learned my addr 172.21.15.37:0/1389381036 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0da7fc640 1 --2- 172.21.15.37:0/1389381036 >> v2:172.21.15.37:3300/0 conn(0x7fc0dc0b36d0 0x7fc0dc0b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0db7fe640 1 --2- 172.21.15.37:0/1389381036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0dc0cd000 0x7fc0dc0b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0d9ffb640 1 -- 172.21.15.37:0/1389381036 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060175215 0 0) 0x7fc0dc0b3bd0 con 0x7fc0dc159f20 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0d9ffb640 1 -- 172.21.15.37:0/1389381036 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c8003530 con 0x7fc0dc159f20 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0da7fc640 1 -- 172.21.15.37:0/1389381036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0dc0cd000 msgr2=0x7fc0dc0b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0da7fc640 1 --2- 172.21.15.37:0/1389381036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0dc0cd000 0x7fc0dc0b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0da7fc640 1 -- 172.21.15.37:0/1389381036 >> v1:172.21.15.37:6789/0 conn(0x7fc0dc159f20 legacy=0x7fc0dc06d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.113+0000 7fc0da7fc640 1 -- 172.21.15.37:0/1389381036 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc06cf80 con 0x7fc0dc0b36d0 2024-04-25T09:04:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0da7fc640 1 --2- 172.21.15.37:0/1389381036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc0b36d0 0x7fc0dc0b2190 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009d90 tx=0x7fc0cc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4565d9f52e274c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0d9ffb640 1 -- 172.21.15.37:0/1389381036 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc0b2070 con 0x7fc0dc0b36d0 2024-04-25T09:04:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0d9ffb640 1 -- 172.21.15.37:0/1389381036 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0cc0a59a0 con 0x7fc0dc0b36d0 2024-04-25T09:04:36.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0d9ffb640 1 -- 172.21.15.37:0/1389381036 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc0a5d70 con 0x7fc0dc0b36d0 2024-04-25T09:04:36.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0e14d2640 1 -- 172.21.15.37:0/1389381036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc0b36d0 msgr2=0x7fc0dc0b2190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0e14d2640 1 --2- 172.21.15.37:0/1389381036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc0b36d0 0x7fc0dc0b2190 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc009d90 tx=0x7fc0cc01c940 comp rx=0 tx=0).stop 2024-04-25T09:04:36.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0e14d2640 1 -- 172.21.15.37:0/1389381036 shutdown_connections 2024-04-25T09:04:36.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0e14d2640 1 --2- 172.21.15.37:0/1389381036 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0dc0cd000 0x7fc0dc0b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0e14d2640 1 --2- 172.21.15.37:0/1389381036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc0b36d0 0x7fc0dc0b2190 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0e14d2640 1 -- 172.21.15.37:0/1389381036 >> 172.21.15.37:0/1389381036 conn(0x7fc0dc06ae60 msgr2=0x7fc0dc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0e14d2640 1 -- 172.21.15.37:0/1389381036 shutdown_connections 2024-04-25T09:04:36.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.116+0000 7fc0e14d2640 1 -- 172.21.15.37:0/1389381036 wait complete. 2024-04-25T09:04:36.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0e14d2640 1 Processor -- start 2024-04-25T09:04:36.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0e14d2640 1 -- start start 2024-04-25T09:04:36.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0e14d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0dc0b36d0 0x7fc0dc1f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0e14d2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc0cd000 0x7fc0dc0986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0e14d2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0dc159f20 0x7fc0dc0c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0e14d2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc166f50 con 0x7fc0dc0cd000 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0e14d2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc166dd0 con 0x7fc0dc159f20 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0e14d2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc1670d0 con 0x7fc0dc0b36d0 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0da7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc0cd000 0x7fc0dc0986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0da7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc0cd000 0x7fc0dc0986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50258/0 (socket says 172.21.15.37:50258) 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0da7fc640 1 -- 172.21.15.37:0/2296114879 learned_addr learned my addr 172.21.15.37:0/2296114879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0da7fc640 1 -- 172.21.15.37:0/2296114879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0dc0b36d0 msgr2=0x7fc0dc1f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0daffd640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0dc0b36d0 0x7fc0dc1f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0da7fc640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc0dc0b36d0 0x7fc0dc1f4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0da7fc640 1 -- 172.21.15.37:0/2296114879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0dc159f20 msgr2=0x7fc0dc0c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0db7fe640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0dc159f20 0x7fc0dc0c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0da7fc640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc0dc159f20 0x7fc0dc0c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0da7fc640 1 -- 172.21.15.37:0/2296114879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc0c3f20 con 0x7fc0dc0cd000 2024-04-25T09:04:36.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0da7fc640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc0cd000 0x7fc0dc0986f0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc01a570 tx=0x7fc0cc0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0d37fe640 1 -- 172.21.15.37:0/2296114879 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc0b2050 con 0x7fc0dc0cd000 2024-04-25T09:04:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0d37fe640 1 -- 172.21.15.37:0/2296114879 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0cc0be070 con 0x7fc0dc0cd000 2024-04-25T09:04:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0e14d2640 1 -- 172.21.15.37:0/2296114879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0dc0c4200 con 0x7fc0dc0cd000 2024-04-25T09:04:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0d37fe640 1 -- 172.21.15.37:0/2296114879 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0cc0ada70 con 0x7fc0dc0cd000 2024-04-25T09:04:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0e14d2640 1 -- 172.21.15.37:0/2296114879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc0c47c0 con 0x7fc0dc0cd000 2024-04-25T09:04:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0e14d2640 1 -- 172.21.15.37:0/2296114879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0dc067c30 con 0x7fc0dc0cd000 2024-04-25T09:04:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0d37fe640 1 -- 172.21.15.37:0/2296114879 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc0cc0adc10 con 0x7fc0dc0cd000 2024-04-25T09:04:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.120+0000 7fc0d37fe640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc0bc066f60 0x7fc0bc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.124+0000 7fc0d37fe640 1 -- 172.21.15.37:0/2296114879 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc0cc120b40 con 0x7fc0dc0cd000 2024-04-25T09:04:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.124+0000 7fc0d37fe640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x7fc0bc06d540 0x7fc0bc06f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.124+0000 7fc0d37fe640 1 -- 172.21.15.37:0/2296114879 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0cc0c5e70 con 0x7fc0bc06d540 2024-04-25T09:04:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.124+0000 7fc0d37fe640 1 -- 172.21.15.37:0/2296114879 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0cc120fd0 con 0x7fc0dc0cd000 2024-04-25T09:04:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.124+0000 7fc0daffd640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc0bc066f60 0x7fc0bc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.124+0000 7fc0db7fe640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x7fc0bc06d540 0x7fc0bc06f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.124+0000 7fc0db7fe640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x7fc0bc06d540 0x7fc0bc06f980 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.124+0000 7fc0daffd640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc0bc066f60 0x7fc0bc069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4007fd0 tx=0x7fc0c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.124+0000 7fc0d37fe640 1 -- 172.21.15.37:0/2296114879 <== osd.0 v2:172.21.15.37:6824/1116141266 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc0cc0c5e70 con 0x7fc0bc06d540 2024-04-25T09:04:36.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0e14d2640 1 -- 172.21.15.37:0/2296114879 --> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc0dc0c96a0 con 0x7fc0bc06d540 2024-04-25T09:04:36.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0d37fe640 1 -- 172.21.15.37:0/2296114879 <== osd.0 v2:172.21.15.37:6824/1116141266 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc0dc0c96a0 con 0x7fc0bc06d540 2024-04-25T09:04:36.127 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:36.127 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:36.127 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:36.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0e14d2640 1 -- 172.21.15.37:0/2296114879 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x7fc0bc06d540 msgr2=0x7fc0bc06f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0e14d2640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.37:6824/1116141266,v1:172.21.15.37:6825/1116141266] conn(0x7fc0bc06d540 0x7fc0bc06f980 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0e14d2640 1 -- 172.21.15.37:0/2296114879 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc0bc066f60 msgr2=0x7fc0bc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0e14d2640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc0bc066f60 0x7fc0bc069420 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4007fd0 tx=0x7fc0c4009040 comp rx=0 tx=0).stop 2024-04-25T09:04:36.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0e14d2640 1 -- 172.21.15.37:0/2296114879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc0cd000 msgr2=0x7fc0dc0986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0e14d2640 1 --2- 172.21.15.37:0/2296114879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc0dc0cd000 0x7fc0dc0986f0 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc01a570 tx=0x7fc0cc0196e0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0db7fe640 1 -- 172.21.15.37:0/2296114879 reap_dead start 2024-04-25T09:04:36.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0e14d2640 1 -- 172.21.15.37:0/2296114879 shutdown_connections 2024-04-25T09:04:36.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0e14d2640 1 -- 172.21.15.37:0/2296114879 >> 172.21.15.37:0/2296114879 conn(0x7fc0dc06ae60 msgr2=0x7fc0dc0ad500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0e14d2640 1 -- 172.21.15.37:0/2296114879 shutdown_connections 2024-04-25T09:04:36.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.136+0000 7fc0e14d2640 1 -- 172.21.15.37:0/2296114879 wait complete. 2024-04-25T09:04:36.144 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T09:04:36.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a2a04b640 1 Processor -- start 2024-04-25T09:04:36.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a2a04b640 1 -- start start 2024-04-25T09:04:36.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a2a04b640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f8a2406f4a0 0x7f8a24099a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a2a04b640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f8a24099fb0 0x7f8a2415d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a2a04b640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a240b0e10 con 0x7f8a240b0a30 2024-04-25T09:04:36.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a2a04b640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a240d0d90 con 0x7f8a24099fb0 2024-04-25T09:04:36.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a2a04b640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a240d0180 con 0x7f8a2406f4a0 2024-04-25T09:04:36.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a237fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f8a240b0a30 0x7f8a240cf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56182/0 (socket says 172.21.15.37:56182) 2024-04-25T09:04:36.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a237fe640 1 -- 172.21.15.37:0/3957118364 learned_addr learned my addr 172.21.15.37:0/3957118364 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:36.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a23fff640 1 --2- 172.21.15.37:0/3957118364 >> v2:172.21.15.37:3300/0 conn(0x7f8a24099fb0 0x7f8a2415d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a22ffd640 1 --2- 172.21.15.37:0/3957118364 >> v2:172.21.15.203:3300/0 conn(0x7f8a2406f4a0 0x7f8a24099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a23fff640 1 -- 172.21.15.37:0/3957118364 >> v2:172.21.15.203:3300/0 conn(0x7f8a2406f4a0 msgr2=0x7f8a24099a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:36.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a23fff640 1 --2- 172.21.15.37:0/3957118364 >> v2:172.21.15.203:3300/0 conn(0x7f8a2406f4a0 0x7f8a24099a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a23fff640 1 -- 172.21.15.37:0/3957118364 >> v1:172.21.15.37:6789/0 conn(0x7f8a240b0a30 legacy=0x7f8a240cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a23fff640 1 -- 172.21.15.37:0/3957118364 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a240d0460 con 0x7f8a24099fb0 2024-04-25T09:04:36.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a227fc640 1 -- 172.21.15.37:0/3957118364 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2155788409 0 0) 0x7f8a240b0e10 con 0x7f8a240b0a30 2024-04-25T09:04:36.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.272+0000 7f8a22ffd640 1 --2- 172.21.15.37:0/3957118364 >> v2:172.21.15.203:3300/0 conn(0x7f8a2406f4a0 0x7f8a24099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a23fff640 1 --2- 172.21.15.37:0/3957118364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a24099fb0 0x7f8a2415d230 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f8a240b7670 tx=0x7f8a1c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba8771973a53a5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a227fc640 1 -- 172.21.15.37:0/3957118364 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0b2070 con 0x7f8a24099fb0 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a227fc640 1 -- 172.21.15.37:0/3957118364 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a1c0a57f0 con 0x7f8a24099fb0 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a227fc640 1 -- 172.21.15.37:0/3957118364 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0a5bc0 con 0x7f8a24099fb0 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a2a04b640 1 -- 172.21.15.37:0/3957118364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a24099fb0 msgr2=0x7f8a2415d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a2a04b640 1 --2- 172.21.15.37:0/3957118364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a24099fb0 0x7f8a2415d230 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f8a240b7670 tx=0x7f8a1c01c930 comp rx=0 tx=0).stop 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a2a04b640 1 -- 172.21.15.37:0/3957118364 shutdown_connections 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a2a04b640 1 --2- 172.21.15.37:0/3957118364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a24099fb0 0x7f8a2415d230 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a2a04b640 1 --2- 172.21.15.37:0/3957118364 >> v2:172.21.15.203:3300/0 conn(0x7f8a2406f4a0 0x7f8a24099a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a2a04b640 1 -- 172.21.15.37:0/3957118364 >> 172.21.15.37:0/3957118364 conn(0x7f8a240b30b0 msgr2=0x7f8a240b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a2a04b640 1 -- 172.21.15.37:0/3957118364 shutdown_connections 2024-04-25T09:04:36.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a2a04b640 1 -- 172.21.15.37:0/3957118364 wait complete. 2024-04-25T09:04:36.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a2a04b640 1 Processor -- start 2024-04-25T09:04:36.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.276+0000 7f8a2a04b640 1 -- start start 2024-04-25T09:04:36.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a2a04b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2406f4a0 0x7f8a24157ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a2a04b640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a24099fb0 0x7f8a2414ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a2a04b640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a240b0a30 0x7f8a24154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a2a04b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240cbfa0 con 0x7f8a2406f4a0 2024-04-25T09:04:36.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a2a04b640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240cbe20 con 0x7f8a24099fb0 2024-04-25T09:04:36.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a2a04b640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a240cc120 con 0x7f8a240b0a30 2024-04-25T09:04:36.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2406f4a0 0x7f8a24157ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2406f4a0 0x7f8a24157ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50266/0 (socket says 172.21.15.37:50266) 2024-04-25T09:04:36.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a237fe640 1 -- 172.21.15.37:0/2945663567 learned_addr learned my addr 172.21.15.37:0/2945663567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a22ffd640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a24099fb0 0x7f8a2414ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a23fff640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a240b0a30 0x7f8a24154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a237fe640 1 -- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a240b0a30 msgr2=0x7f8a24154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a237fe640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a240b0a30 0x7f8a24154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a237fe640 1 -- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a24099fb0 msgr2=0x7f8a2414ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a237fe640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a24099fb0 0x7f8a2414ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a237fe640 1 -- 172.21.15.37:0/2945663567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a24153a90 con 0x7f8a2406f4a0 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a23fff640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a240b0a30 0x7f8a24154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a237fe640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2406f4a0 0x7f8a24157ae0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c4b0 tx=0x7f8a1400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a0bfff640 1 -- 172.21.15.37:0/2945663567 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14018070 con 0x7f8a2406f4a0 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a0bfff640 1 -- 172.21.15.37:0/2945663567 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a14007500 con 0x7f8a2406f4a0 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a2a04b640 1 -- 172.21.15.37:0/2945663567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a24153d70 con 0x7f8a2406f4a0 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a0bfff640 1 -- 172.21.15.37:0/2945663567 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a14004a30 con 0x7f8a2406f4a0 2024-04-25T09:04:36.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a2a04b640 1 -- 172.21.15.37:0/2945663567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a24154330 con 0x7f8a2406f4a0 2024-04-25T09:04:36.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a2a04b640 1 -- 172.21.15.37:0/2945663567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a2404efa0 con 0x7f8a2406f4a0 2024-04-25T09:04:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a0bfff640 1 -- 172.21.15.37:0/2945663567 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f8a1401b070 con 0x7f8a2406f4a0 2024-04-25T09:04:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a0bfff640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a10066f60 0x7f8a10069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.280+0000 7f8a0bfff640 1 -- 172.21.15.37:0/2945663567 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f8a140842b0 con 0x7f8a2406f4a0 2024-04-25T09:04:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.284+0000 7f8a22ffd640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a10066f60 0x7f8a10069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.284+0000 7f8a22ffd640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a10066f60 0x7f8a10069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8a240b7410 tx=0x7f8a18008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.288+0000 7f8a0bfff640 1 -- 172.21.15.37:0/2945663567 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f8a14055380 con 0x7f8a2406f4a0 2024-04-25T09:04:36.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.444+0000 7f8a2a04b640 1 -- 172.21.15.37:0/2945663567 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a240b3ee0 con 0x7f8a10066f60 2024-04-25T09:04:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.444+0000 7f8a0bfff640 1 -- 172.21.15.37:0/2945663567 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8a240b3ee0 con 0x7f8a10066f60 2024-04-25T09:04:36.434 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T09:04:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 -- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a10066f60 msgr2=0x7f8a10069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a10066f60 0x7f8a10069420 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8a240b7410 tx=0x7f8a18008040 comp rx=0 tx=0).stop 2024-04-25T09:04:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 -- 172.21.15.37:0/2945663567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2406f4a0 msgr2=0x7f8a24157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2406f4a0 0x7f8a24157ae0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400c4b0 tx=0x7f8a1400c970 comp rx=0 tx=0).stop 2024-04-25T09:04:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 -- 172.21.15.37:0/2945663567 shutdown_connections 2024-04-25T09:04:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f8a10066f60 0x7f8a10069420 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f8a240b0a30 0x7f8a24154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f8a24099fb0 0x7f8a2414ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 --2- 172.21.15.37:0/2945663567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a2406f4a0 0x7f8a24157ae0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 -- 172.21.15.37:0/2945663567 >> 172.21.15.37:0/2945663567 conn(0x7f8a240b30b0 msgr2=0x7f8a240b27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 -- 172.21.15.37:0/2945663567 shutdown_connections 2024-04-25T09:04:36.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.448+0000 7f8a2a04b640 1 -- 172.21.15.37:0/2945663567 wait complete. 2024-04-25T09:04:36.453 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T09:04:36.453 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:36.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.580+0000 7ff54c524640 1 Processor -- start 2024-04-25T09:04:36.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.580+0000 7ff54c524640 1 -- start start 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.580+0000 7ff54c524640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff5440ae6c0 0x7ff5440b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.580+0000 7ff54c524640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff5440cf510 0x7ff5440cf940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.580+0000 7ff54c524640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54415e0f0 con 0x7ff54415a3c0 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.580+0000 7ff54c524640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5440cfe80 con 0x7ff5440ae6c0 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.580+0000 7ff54c524640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5440b24a0 con 0x7ff5440cf510 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff549abf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff5440ae6c0 0x7ff5440b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff549abf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff5440ae6c0 0x7ff5440b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50276/0 (socket says 172.21.15.37:50276) 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff549abf640 1 -- 172.21.15.37:0/27084636 learned_addr learned my addr 172.21.15.37:0/27084636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff54aac1640 1 --2- 172.21.15.37:0/27084636 >> v2:172.21.15.203:3300/0 conn(0x7ff5440cf510 0x7ff5440cf940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff549abf640 1 -- 172.21.15.37:0/27084636 >> v2:172.21.15.203:3300/0 conn(0x7ff5440cf510 msgr2=0x7ff5440cf940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff549abf640 1 --2- 172.21.15.37:0/27084636 >> v2:172.21.15.203:3300/0 conn(0x7ff5440cf510 0x7ff5440cf940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff549abf640 1 -- 172.21.15.37:0/27084636 >> v1:172.21.15.203:6789/0 conn(0x7ff54415a3c0 legacy=0x7ff54415a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff549abf640 1 -- 172.21.15.37:0/27084636 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54415b0b0 con 0x7ff5440ae6c0 2024-04-25T09:04:36.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff549abf640 1 --2- 172.21.15.37:0/27084636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5440ae6c0 0x7ff5440b27a0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7ff5440cdd00 tx=0x7ff53401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f979acd860a2921e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff5492be640 1 -- 172.21.15.37:0/27084636 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5340b1070 con 0x7ff5440ae6c0 2024-04-25T09:04:36.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff5492be640 1 -- 172.21.15.37:0/27084636 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5340ac630 con 0x7ff5440ae6c0 2024-04-25T09:04:36.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff5492be640 1 -- 172.21.15.37:0/27084636 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5340a8050 con 0x7ff5440ae6c0 2024-04-25T09:04:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff54c524640 1 -- 172.21.15.37:0/27084636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5440ae6c0 msgr2=0x7ff5440b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff54c524640 1 --2- 172.21.15.37:0/27084636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5440ae6c0 0x7ff5440b27a0 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7ff5440cdd00 tx=0x7ff53401c4e0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff54c524640 1 -- 172.21.15.37:0/27084636 shutdown_connections 2024-04-25T09:04:36.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff54c524640 1 --2- 172.21.15.37:0/27084636 >> v2:172.21.15.203:3300/0 conn(0x7ff5440cf510 0x7ff5440cf940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff54c524640 1 --2- 172.21.15.37:0/27084636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5440ae6c0 0x7ff5440b27a0 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff54c524640 1 -- 172.21.15.37:0/27084636 >> 172.21.15.37:0/27084636 conn(0x7ff54406ae60 msgr2=0x7ff54406b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff54c524640 1 -- 172.21.15.37:0/27084636 shutdown_connections 2024-04-25T09:04:36.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.584+0000 7ff54c524640 1 -- 172.21.15.37:0/27084636 wait complete. 2024-04-25T09:04:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54c524640 1 Processor -- start 2024-04-25T09:04:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54c524640 1 -- start start 2024-04-25T09:04:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54c524640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5440ae6c0 0x7ff5440b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54c524640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5440cf510 0x7ff54409c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54c524640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff54415a3c0 0x7ff5440cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54c524640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5440b2e60 con 0x7ff5440ae6c0 2024-04-25T09:04:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54c524640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5440b2ce0 con 0x7ff5440cf510 2024-04-25T09:04:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54c524640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5441673f0 con 0x7ff54415a3c0 2024-04-25T09:04:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54a2c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5440ae6c0 0x7ff5440b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54a2c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5440ae6c0 0x7ff5440b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50284/0 (socket says 172.21.15.37:50284) 2024-04-25T09:04:36.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54a2c0640 1 -- 172.21.15.37:0/763881150 learned_addr learned my addr 172.21.15.37:0/763881150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff549abf640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5440cf510 0x7ff54409c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54aac1640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff54415a3c0 0x7ff5440cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54a2c0640 1 -- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff54415a3c0 msgr2=0x7ff5440cbfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54a2c0640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff54415a3c0 0x7ff5440cbfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54a2c0640 1 -- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5440cf510 msgr2=0x7ff54409c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54a2c0640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff5440cf510 0x7ff54409c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54a2c0640 1 -- 172.21.15.37:0/763881150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54409cf50 con 0x7ff5440ae6c0 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54aac1640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff54415a3c0 0x7ff5440cbfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54a2c0640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5440ae6c0 0x7ff5440b0110 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7ff54000b410 tx=0x7ff54000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff53affd640 1 -- 172.21.15.37:0/763881150 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540015030 con 0x7ff5440ae6c0 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff53affd640 1 -- 172.21.15.37:0/763881150 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff540004020 con 0x7ff5440ae6c0 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54c524640 1 -- 172.21.15.37:0/763881150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54409d230 con 0x7ff5440ae6c0 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff53affd640 1 -- 172.21.15.37:0/763881150 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540004450 con 0x7ff5440ae6c0 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54c524640 1 -- 172.21.15.37:0/763881150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff544099670 con 0x7ff5440ae6c0 2024-04-25T09:04:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff54c524640 1 -- 172.21.15.37:0/763881150 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff544067c30 con 0x7ff5440ae6c0 2024-04-25T09:04:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff53affd640 1 -- 172.21.15.37:0/763881150 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff540018070 con 0x7ff5440ae6c0 2024-04-25T09:04:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.588+0000 7ff53affd640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff528066f60 0x7ff528069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.592+0000 7ff53affd640 1 -- 172.21.15.37:0/763881150 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ff540084a90 con 0x7ff5440ae6c0 2024-04-25T09:04:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.592+0000 7ff53affd640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x7ff52806d540 0x7ff52806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.592+0000 7ff53affd640 1 -- 172.21.15.37:0/763881150 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff540020e50 con 0x7ff52806d540 2024-04-25T09:04:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.592+0000 7ff53affd640 1 -- 172.21.15.37:0/763881150 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5400892c0 con 0x7ff5440ae6c0 2024-04-25T09:04:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.592+0000 7ff54aac1640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x7ff52806d540 0x7ff52806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.592+0000 7ff549abf640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff528066f60 0x7ff528069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.592+0000 7ff54aac1640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x7ff52806d540 0x7ff52806f980 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.592+0000 7ff549abf640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff528066f60 0x7ff528069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff5340ae000 tx=0x7ff534008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.592+0000 7ff53affd640 1 -- 172.21.15.37:0/763881150 <== osd.1 v2:172.21.15.37:6816/2944303862 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff540020e50 con 0x7ff52806d540 2024-04-25T09:04:36.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54c524640 1 -- 172.21.15.37:0/763881150 --> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff54406edf0 con 0x7ff52806d540 2024-04-25T09:04:36.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff53affd640 1 -- 172.21.15.37:0/763881150 <== osd.1 v2:172.21.15.37:6816/2944303862 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff54406edf0 con 0x7ff52806d540 2024-04-25T09:04:36.594 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:36.595 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:36.595 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54c524640 1 -- 172.21.15.37:0/763881150 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x7ff52806d540 msgr2=0x7ff52806f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54c524640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.37:6816/2944303862,v1:172.21.15.37:6817/2944303862] conn(0x7ff52806d540 0x7ff52806f980 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54c524640 1 -- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff528066f60 msgr2=0x7ff528069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54c524640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff528066f60 0x7ff528069420 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff5340ae000 tx=0x7ff534008040 comp rx=0 tx=0).stop 2024-04-25T09:04:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54c524640 1 -- 172.21.15.37:0/763881150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5440ae6c0 msgr2=0x7ff5440b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54c524640 1 --2- 172.21.15.37:0/763881150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5440ae6c0 0x7ff5440b0110 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7ff54000b410 tx=0x7ff54000b8d0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54aac1640 1 -- 172.21.15.37:0/763881150 reap_dead start 2024-04-25T09:04:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54c524640 1 -- 172.21.15.37:0/763881150 shutdown_connections 2024-04-25T09:04:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54c524640 1 -- 172.21.15.37:0/763881150 >> 172.21.15.37:0/763881150 conn(0x7ff54406ae60 msgr2=0x7ff54415a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54c524640 1 -- 172.21.15.37:0/763881150 shutdown_connections 2024-04-25T09:04:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.604+0000 7ff54c524640 1 -- 172.21.15.37:0/763881150 wait complete. 2024-04-25T09:04:36.610 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T09:04:36.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.732+0000 7f99fc2c0640 1 Processor -- start 2024-04-25T09:04:36.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.732+0000 7f99fc2c0640 1 -- start start 2024-04-25T09:04:36.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.732+0000 7f99fc2c0640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f99f415da80 0x7f99f40d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.732+0000 7f99fc2c0640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f99f40d07f0 0x7f99f406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.732+0000 7f99fc2c0640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f406be60 con 0x7f99f40d1cd0 2024-04-25T09:04:36.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.732+0000 7f99fc2c0640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f406c440 con 0x7f99f415da80 2024-04-25T09:04:36.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.732+0000 7f99fc2c0640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f415de60 con 0x7f99f40d07f0 2024-04-25T09:04:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.732+0000 7f99fa05c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f99f415da80 0x7f99f40d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.732+0000 7f99fa05c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f99f415da80 0x7f99f40d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50288/0 (socket says 172.21.15.37:50288) 2024-04-25T09:04:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.732+0000 7f99fa05c640 1 -- 172.21.15.37:0/2681750167 learned_addr learned my addr 172.21.15.37:0/2681750167 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fa85d640 1 --2- 172.21.15.37:0/2681750167 >> v2:172.21.15.203:3300/0 conn(0x7f99f40d07f0 0x7f99f406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fa05c640 1 -- 172.21.15.37:0/2681750167 >> v2:172.21.15.203:3300/0 conn(0x7f99f40d07f0 msgr2=0x7f99f406c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fa05c640 1 --2- 172.21.15.37:0/2681750167 >> v2:172.21.15.203:3300/0 conn(0x7f99f40d07f0 0x7f99f406c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fa05c640 1 -- 172.21.15.37:0/2681750167 >> v1:172.21.15.203:6789/0 conn(0x7f99f40d1cd0 legacy=0x7f99f40d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fa05c640 1 -- 172.21.15.37:0/2681750167 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f40d2450 con 0x7f99f415da80 2024-04-25T09:04:36.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fa05c640 1 --2- 172.21.15.37:0/2681750167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f415da80 0x7f99f40d4230 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009870 tx=0x7f99f001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0a2fa55d57388b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99f905a640 1 -- 172.21.15.37:0/2681750167 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f00ab650 con 0x7f99f415da80 2024-04-25T09:04:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99f905a640 1 -- 172.21.15.37:0/2681750167 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f00b0070 con 0x7f99f415da80 2024-04-25T09:04:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99f905a640 1 -- 172.21.15.37:0/2681750167 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f00aba10 con 0x7f99f415da80 2024-04-25T09:04:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fc2c0640 1 -- 172.21.15.37:0/2681750167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f415da80 msgr2=0x7f99f40d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fc2c0640 1 --2- 172.21.15.37:0/2681750167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f415da80 0x7f99f40d4230 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009870 tx=0x7f99f001c4f0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fc2c0640 1 -- 172.21.15.37:0/2681750167 shutdown_connections 2024-04-25T09:04:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fc2c0640 1 --2- 172.21.15.37:0/2681750167 >> v2:172.21.15.203:3300/0 conn(0x7f99f40d07f0 0x7f99f406c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fc2c0640 1 --2- 172.21.15.37:0/2681750167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f415da80 0x7f99f40d4230 unknown :-1 s=CLOSED pgs=2246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fc2c0640 1 -- 172.21.15.37:0/2681750167 >> 172.21.15.37:0/2681750167 conn(0x7f99f408b000 msgr2=0x7f99f40d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fc2c0640 1 -- 172.21.15.37:0/2681750167 shutdown_connections 2024-04-25T09:04:36.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.736+0000 7f99fc2c0640 1 -- 172.21.15.37:0/2681750167 wait complete. 2024-04-25T09:04:36.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fc2c0640 1 Processor -- start 2024-04-25T09:04:36.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fc2c0640 1 -- start start 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fc2c0640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99f40d07f0 0x7f99f415c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fc2c0640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99f40d1cd0 0x7f99f4157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fc2c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f415da80 0x7f99f4158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fc2c0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f406d7a0 con 0x7f99f415da80 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fc2c0640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f406d620 con 0x7f99f40d07f0 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fc2c0640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f406d920 con 0x7f99f40d1cd0 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fa85d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f415da80 0x7f99f4158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fa85d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f415da80 0x7f99f4158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50292/0 (socket says 172.21.15.37:50292) 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fa85d640 1 -- 172.21.15.37:0/1012342092 learned_addr learned my addr 172.21.15.37:0/1012342092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fa05c640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99f40d07f0 0x7f99f415c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99f985b640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99f40d1cd0 0x7f99f4157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fa85d640 1 -- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99f40d1cd0 msgr2=0x7f99f4157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fa85d640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99f40d1cd0 0x7f99f4157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fa85d640 1 -- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99f40d07f0 msgr2=0x7f99f415c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fa85d640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99f40d07f0 0x7f99f415c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fa85d640 1 -- 172.21.15.37:0/1012342092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f4153d80 con 0x7f99f415da80 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99f985b640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99f40d1cd0 0x7f99f4157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fa85d640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f415da80 0x7f99f4158190 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f99ec007e40 tx=0x7f99ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99eaffd640 1 -- 172.21.15.37:0/1012342092 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec014620 con 0x7f99f415da80 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99eaffd640 1 -- 172.21.15.37:0/1012342092 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec019070 con 0x7f99f415da80 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fc2c0640 1 -- 172.21.15.37:0/1012342092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f4154060 con 0x7f99f415da80 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99eaffd640 1 -- 172.21.15.37:0/1012342092 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec014a20 con 0x7f99f415da80 2024-04-25T09:04:36.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fc2c0640 1 -- 172.21.15.37:0/1012342092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f4154620 con 0x7f99f415da80 2024-04-25T09:04:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.740+0000 7f99fc2c0640 1 -- 172.21.15.37:0/1012342092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f404efa0 con 0x7f99f415da80 2024-04-25T09:04:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.744+0000 7f99eaffd640 1 -- 172.21.15.37:0/1012342092 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f99ec014bc0 con 0x7f99f415da80 2024-04-25T09:04:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.744+0000 7f99eaffd640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f99d8066f60 0x7f99d8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.744+0000 7f99eaffd640 1 -- 172.21.15.37:0/1012342092 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f99ec085850 con 0x7f99f415da80 2024-04-25T09:04:36.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.744+0000 7f99fa05c640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f99d8066f60 0x7f99d8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:36.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.744+0000 7f99fa05c640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f99d8066f60 0x7f99d8069420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f99f00af000 tx=0x7f99f00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:36.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.748+0000 7f99eaffd640 1 -- 172.21.15.37:0/1012342092 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f99ec056920 con 0x7f99f415da80 2024-04-25T09:04:36.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.900+0000 7f99fc2c0640 1 -- 172.21.15.37:0/1012342092 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f99f4153b30 con 0x7f99d8066f60 2024-04-25T09:04:36.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.900+0000 7f99eaffd640 1 -- 172.21.15.37:0/1012342092 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f99f4153b30 con 0x7f99d8066f60 2024-04-25T09:04:36.891 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T09:04:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 -- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f99d8066f60 msgr2=0x7f99d8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f99d8066f60 0x7f99d8069420 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f99f00af000 tx=0x7f99f00a7000 comp rx=0 tx=0).stop 2024-04-25T09:04:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 -- 172.21.15.37:0/1012342092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f415da80 msgr2=0x7f99f4158190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f415da80 0x7f99f4158190 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f99ec007e40 tx=0x7f99ec00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 -- 172.21.15.37:0/1012342092 shutdown_connections 2024-04-25T09:04:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f99d8066f60 0x7f99d8069420 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99f415da80 0x7f99f4158190 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f99f40d1cd0 0x7f99f4157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 --2- 172.21.15.37:0/1012342092 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f99f40d07f0 0x7f99f415c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 -- 172.21.15.37:0/1012342092 >> 172.21.15.37:0/1012342092 conn(0x7f99f408b000 msgr2=0x7f99f40d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:36.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 -- 172.21.15.37:0/1012342092 shutdown_connections 2024-04-25T09:04:36.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:36.904+0000 7f99fc2c0640 1 -- 172.21.15.37:0/1012342092 wait complete. 2024-04-25T09:04:36.908 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T09:04:36.908 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eb2185640 1 Processor -- start 2024-04-25T09:04:37.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eb2185640 1 -- start start 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eb2185640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f5eac15ddd0 0x7f5eac06bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eb2185640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eac06d5d0 con 0x7f5eac0d3e00 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eb2185640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eac06cd30 con 0x7f5eac15d9f0 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eb2185640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac06d010 con 0x7f5eac15ddd0 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eabfff640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f5eac0d3e00 0x7f5eac06c500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56198/0 (socket says 172.21.15.37:56198) 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eabfff640 1 -- 172.21.15.37:0/1055290203 learned_addr learned my addr 172.21.15.37:0/1055290203 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eaaffd640 1 --2- 172.21.15.37:0/1055290203 >> v2:172.21.15.37:3300/0 conn(0x7f5eac15ddd0 0x7f5eac06bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eaa7fc640 1 -- 172.21.15.37:0/1055290203 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263558190 0 0) 0x7f5eac06d5d0 con 0x7f5eac0d3e00 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eaa7fc640 1 -- 172.21.15.37:0/1055290203 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e98003530 con 0x7f5eac0d3e00 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eaaffd640 1 -- 172.21.15.37:0/1055290203 >> v1:172.21.15.203:6789/0 conn(0x7f5eac15d9f0 legacy=0x7f5eac0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eaaffd640 1 -- 172.21.15.37:0/1055290203 >> v1:172.21.15.37:6789/0 conn(0x7f5eac0d3e00 legacy=0x7f5eac06c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.044+0000 7f5eaaffd640 1 -- 172.21.15.37:0/1055290203 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac06d2f0 con 0x7f5eac15ddd0 2024-04-25T09:04:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eaaffd640 1 --2- 172.21.15.37:0/1055290203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eac15ddd0 0x7f5eac06bfc0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f5e940097c0 tx=0x7f5e9401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea6f540ebc5e821 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eaa7fc640 1 -- 172.21.15.37:0/1055290203 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e940aa5f0 con 0x7f5eac15ddd0 2024-04-25T09:04:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eaa7fc640 1 -- 172.21.15.37:0/1055290203 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e940af070 con 0x7f5eac15ddd0 2024-04-25T09:04:37.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eaa7fc640 1 -- 172.21.15.37:0/1055290203 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e940aa9b0 con 0x7f5eac15ddd0 2024-04-25T09:04:37.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eb2185640 1 -- 172.21.15.37:0/1055290203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eac15ddd0 msgr2=0x7f5eac06bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eb2185640 1 --2- 172.21.15.37:0/1055290203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eac15ddd0 0x7f5eac06bfc0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f5e940097c0 tx=0x7f5e9401c470 comp rx=0 tx=0).stop 2024-04-25T09:04:37.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eb2185640 1 -- 172.21.15.37:0/1055290203 shutdown_connections 2024-04-25T09:04:37.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eb2185640 1 --2- 172.21.15.37:0/1055290203 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eac15ddd0 0x7f5eac06bfc0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eb2185640 1 -- 172.21.15.37:0/1055290203 >> 172.21.15.37:0/1055290203 conn(0x7f5eac09ca80 msgr2=0x7f5eac08b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eb2185640 1 -- 172.21.15.37:0/1055290203 shutdown_connections 2024-04-25T09:04:37.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eb2185640 1 -- 172.21.15.37:0/1055290203 wait complete. 2024-04-25T09:04:37.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eb2185640 1 Processor -- start 2024-04-25T09:04:37.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.048+0000 7f5eb2185640 1 -- start start 2024-04-25T09:04:37.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eb2185640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eac0d3e00 0x7f5eac1537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eb2185640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5eac15d9f0 0x7f5eac14e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eb2185640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5eac15ddd0 0x7f5eac0b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eb2185640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac16b2a0 con 0x7f5eac0d3e00 2024-04-25T09:04:37.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eb2185640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac16b120 con 0x7f5eac15d9f0 2024-04-25T09:04:37.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eb2185640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac16b420 con 0x7f5eac15ddd0 2024-04-25T09:04:37.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eac0d3e00 0x7f5eac1537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eac0d3e00 0x7f5eac1537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50310/0 (socket says 172.21.15.37:50310) 2024-04-25T09:04:37.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eab7fe640 1 -- 172.21.15.37:0/3249136720 learned_addr learned my addr 172.21.15.37:0/3249136720 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:37.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eaaffd640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5eac15d9f0 0x7f5eac14e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eabfff640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5eac15ddd0 0x7f5eac0b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eab7fe640 1 -- 172.21.15.37:0/3249136720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5eac15ddd0 msgr2=0x7f5eac0b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eab7fe640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f5eac15ddd0 0x7f5eac0b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eab7fe640 1 -- 172.21.15.37:0/3249136720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5eac15d9f0 msgr2=0x7f5eac14e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eab7fe640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f5eac15d9f0 0x7f5eac14e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eab7fe640 1 -- 172.21.15.37:0/3249136720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac15cfc0 con 0x7f5eac0d3e00 2024-04-25T09:04:37.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eab7fe640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eac0d3e00 0x7f5eac1537b0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c790 tx=0x7f5e9c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5ea3fff640 1 -- 172.21.15.37:0/3249136720 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c016020 con 0x7f5eac0d3e00 2024-04-25T09:04:37.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5ea3fff640 1 -- 172.21.15.37:0/3249136720 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c006dd0 con 0x7f5eac0d3e00 2024-04-25T09:04:37.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eb2185640 1 -- 172.21.15.37:0/3249136720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac14eb30 con 0x7f5eac0d3e00 2024-04-25T09:04:37.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5ea3fff640 1 -- 172.21.15.37:0/3249136720 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c003c10 con 0x7f5eac0d3e00 2024-04-25T09:04:37.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eb2185640 1 -- 172.21.15.37:0/3249136720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac14db10 con 0x7f5eac0d3e00 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eb2185640 1 -- 172.21.15.37:0/3249136720 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5eac153eb0 con 0x7f5eac0d3e00 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5ea3fff640 1 -- 172.21.15.37:0/3249136720 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f5e9c00a850 con 0x7f5eac0d3e00 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5ea3fff640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e8c066f60 0x7f5e8c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5ea3fff640 1 -- 172.21.15.37:0/3249136720 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f5e9c084c60 con 0x7f5eac0d3e00 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5ea3fff640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x7f5e8c06d540 0x7f5e8c06f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5ea3fff640 1 -- 172.21.15.37:0/3249136720 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e8c070050 con 0x7f5e8c06d540 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5ea3fff640 1 -- 172.21.15.37:0/3249136720 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e9c08a070 con 0x7f5eac0d3e00 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eabfff640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x7f5e8c06d540 0x7f5e8c06f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eaaffd640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e8c066f60 0x7f5e8c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.052+0000 7f5eabfff640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x7f5e8c06d540 0x7f5e8c06f980 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.056+0000 7f5eaaffd640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e8c066f60 0x7f5e8c069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5e940ae000 tx=0x7f5e940a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.056+0000 7f5ea3fff640 1 -- 172.21.15.37:0/3249136720 <== osd.2 v2:172.21.15.37:6808/332967247 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5e8c070050 con 0x7f5e8c06d540 2024-04-25T09:04:37.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eb2185640 1 -- 172.21.15.37:0/3249136720 --> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5eac068340 con 0x7f5e8c06d540 2024-04-25T09:04:37.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5ea3fff640 1 -- 172.21.15.37:0/3249136720 <== osd.2 v2:172.21.15.37:6808/332967247 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f5eac068340 con 0x7f5e8c06d540 2024-04-25T09:04:37.058 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:37.058 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:37.058 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:37.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eb2185640 1 -- 172.21.15.37:0/3249136720 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x7f5e8c06d540 msgr2=0x7f5e8c06f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eb2185640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.37:6808/332967247,v1:172.21.15.37:6809/332967247] conn(0x7f5e8c06d540 0x7f5e8c06f980 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eb2185640 1 -- 172.21.15.37:0/3249136720 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e8c066f60 msgr2=0x7f5e8c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eb2185640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f5e8c066f60 0x7f5e8c069420 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5e940ae000 tx=0x7f5e940a7000 comp rx=0 tx=0).stop 2024-04-25T09:04:37.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eb2185640 1 -- 172.21.15.37:0/3249136720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eac0d3e00 msgr2=0x7f5eac1537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eb2185640 1 --2- 172.21.15.37:0/3249136720 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5eac0d3e00 0x7f5eac1537b0 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c790 tx=0x7f5e9c00cc50 comp rx=0 tx=0).stop 2024-04-25T09:04:37.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eabfff640 1 -- 172.21.15.37:0/3249136720 reap_dead start 2024-04-25T09:04:37.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eb2185640 1 -- 172.21.15.37:0/3249136720 shutdown_connections 2024-04-25T09:04:37.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eb2185640 1 -- 172.21.15.37:0/3249136720 >> 172.21.15.37:0/3249136720 conn(0x7f5eac09ca80 msgr2=0x7f5eac158670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eb2185640 1 -- 172.21.15.37:0/3249136720 shutdown_connections 2024-04-25T09:04:37.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.068+0000 7f5eb2185640 1 -- 172.21.15.37:0/3249136720 wait complete. 2024-04-25T09:04:37.074 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T09:04:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214d1e1640 1 Processor -- start 2024-04-25T09:04:37.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214d1e1640 1 -- start start 2024-04-25T09:04:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214d1e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21480ce510 0x7f21480ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214d1e1640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f21481451f0 0x7f2148142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214d1e1640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21480cee30 con 0x7f2148144ca0 2024-04-25T09:04:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214d1e1640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21480cc2b0 con 0x7f21481451f0 2024-04-25T09:04:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214d1e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214806e290 con 0x7f21480ce510 2024-04-25T09:04:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214759e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f21481451f0 0x7f2148142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214759e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f21481451f0 0x7f2148142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50314/0 (socket says 172.21.15.37:50314) 2024-04-25T09:04:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214759e640 1 -- 172.21.15.37:0/3059112711 learned_addr learned my addr 172.21.15.37:0/3059112711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f2146d9d640 1 --2- 172.21.15.37:0/3059112711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21480ce510 0x7f21480ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214759e640 1 -- 172.21.15.37:0/3059112711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21480ce510 msgr2=0x7f21480ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214759e640 1 --2- 172.21.15.37:0/3059112711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21480ce510 0x7f21480ce8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214759e640 1 -- 172.21.15.37:0/3059112711 >> v1:172.21.15.37:6789/0 conn(0x7f2148144ca0 legacy=0x7f214806e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.200+0000 7f214759e640 1 -- 172.21.15.37:0/3059112711 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21480ccc70 con 0x7f21481451f0 2024-04-25T09:04:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f214759e640 1 --2- 172.21.15.37:0/3059112711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21481451f0 0x7f2148142680 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f2140009870 tx=0x7f214001ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8721e406e77dd52e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f2145d9b640 1 -- 172.21.15.37:0/3059112711 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21400b5070 con 0x7f21481451f0 2024-04-25T09:04:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f2145d9b640 1 -- 172.21.15.37:0/3059112711 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21400a8a50 con 0x7f21481451f0 2024-04-25T09:04:37.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f2145d9b640 1 -- 172.21.15.37:0/3059112711 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21400a8e20 con 0x7f21481451f0 2024-04-25T09:04:37.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f214d1e1640 1 -- 172.21.15.37:0/3059112711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21481451f0 msgr2=0x7f2148142680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f214d1e1640 1 --2- 172.21.15.37:0/3059112711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21481451f0 0x7f2148142680 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f2140009870 tx=0x7f214001ed50 comp rx=0 tx=0).stop 2024-04-25T09:04:37.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f214d1e1640 1 -- 172.21.15.37:0/3059112711 shutdown_connections 2024-04-25T09:04:37.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f214d1e1640 1 --2- 172.21.15.37:0/3059112711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f21481451f0 0x7f2148142680 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f214d1e1640 1 --2- 172.21.15.37:0/3059112711 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21480ce510 0x7f21480ce8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f214d1e1640 1 -- 172.21.15.37:0/3059112711 >> 172.21.15.37:0/3059112711 conn(0x7f21480b11a0 msgr2=0x7f21480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f214d1e1640 1 -- 172.21.15.37:0/3059112711 shutdown_connections 2024-04-25T09:04:37.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f214d1e1640 1 -- 172.21.15.37:0/3059112711 wait complete. 2024-04-25T09:04:37.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f214d1e1640 1 Processor -- start 2024-04-25T09:04:37.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.204+0000 7f214d1e1640 1 -- start start 2024-04-25T09:04:37.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214d1e1640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21480ce510 0x7f21481f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214d1e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2148144ca0 0x7f21481e3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214d1e1640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21481451f0 0x7f21480a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214d1e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214806ee30 con 0x7f2148144ca0 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214d1e1640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214806df20 con 0x7f21481451f0 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214d1e1640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21480cc900 con 0x7f21480ce510 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2148144ca0 0x7f21481e3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2148144ca0 0x7f21481e3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50316/0 (socket says 172.21.15.37:50316) 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214659c640 1 -- 172.21.15.37:0/2394156350 learned_addr learned my addr 172.21.15.37:0/2394156350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f2146d9d640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21480ce510 0x7f21481f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214759e640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21481451f0 0x7f21480a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214659c640 1 -- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21480ce510 msgr2=0x7f21481f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214659c640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21480ce510 0x7f21481f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214659c640 1 -- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21481451f0 msgr2=0x7f21480a84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214659c640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21481451f0 0x7f21480a84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214659c640 1 -- 172.21.15.37:0/2394156350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21481f1800 con 0x7f2148144ca0 2024-04-25T09:04:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f2146d9d640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21480ce510 0x7f21481f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214759e640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21481451f0 0x7f21480a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214659c640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2148144ca0 0x7f21481e3660 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c430 tx=0x7f213c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f212f7fe640 1 -- 172.21.15.37:0/2394156350 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c0052d0 con 0x7f2148144ca0 2024-04-25T09:04:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f212f7fe640 1 -- 172.21.15.37:0/2394156350 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f213c005470 con 0x7f2148144ca0 2024-04-25T09:04:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214d1e1640 1 -- 172.21.15.37:0/2394156350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21481e4100 con 0x7f2148144ca0 2024-04-25T09:04:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f212f7fe640 1 -- 172.21.15.37:0/2394156350 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f213c007260 con 0x7f2148144ca0 2024-04-25T09:04:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214d1e1640 1 -- 172.21.15.37:0/2394156350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2148132f50 con 0x7f2148144ca0 2024-04-25T09:04:37.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f214d1e1640 1 -- 172.21.15.37:0/2394156350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f214804efa0 con 0x7f2148144ca0 2024-04-25T09:04:37.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f212f7fe640 1 -- 172.21.15.37:0/2394156350 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f213c007400 con 0x7f2148144ca0 2024-04-25T09:04:37.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f212f7fe640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2134066f90 0x7f2134069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.208+0000 7f212f7fe640 1 -- 172.21.15.37:0/2394156350 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f213c084b60 con 0x7f2148144ca0 2024-04-25T09:04:37.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.212+0000 7f2146d9d640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2134066f90 0x7f2134069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.212+0000 7f2146d9d640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2134066f90 0x7f2134069450 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2138003fd0 tx=0x7f21380039e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.212+0000 7f212f7fe640 1 -- 172.21.15.37:0/2394156350 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f213c055da0 con 0x7f2148144ca0 2024-04-25T09:04:37.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.372+0000 7f214d1e1640 1 -- 172.21.15.37:0/2394156350 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f2148073150 con 0x7f2134066f90 2024-04-25T09:04:37.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.372+0000 7f212f7fe640 1 -- 172.21.15.37:0/2394156350 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2148073150 con 0x7f2134066f90 2024-04-25T09:04:37.363 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T09:04:37.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 -- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2134066f90 msgr2=0x7f2134069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2134066f90 0x7f2134069450 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2138003fd0 tx=0x7f21380039e0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 -- 172.21.15.37:0/2394156350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2148144ca0 msgr2=0x7f21481e3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2148144ca0 0x7f21481e3660 secure :-1 s=READY pgs=2253 cs=0 l=1 rev1=1 crypto rx=0x7f213c00c430 tx=0x7f213c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 -- 172.21.15.37:0/2394156350 shutdown_connections 2024-04-25T09:04:37.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f2134066f90 0x7f2134069450 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f21481451f0 0x7f21480a84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2148144ca0 0x7f21481e3660 unknown :-1 s=CLOSED pgs=2253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 --2- 172.21.15.37:0/2394156350 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f21480ce510 0x7f21481f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 -- 172.21.15.37:0/2394156350 >> 172.21.15.37:0/2394156350 conn(0x7f21480b11a0 msgr2=0x7f21480c9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 -- 172.21.15.37:0/2394156350 shutdown_connections 2024-04-25T09:04:37.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.376+0000 7f214d1e1640 1 -- 172.21.15.37:0/2394156350 wait complete. 2024-04-25T09:04:37.383 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T09:04:37.383 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19e130c640 1 Processor -- start 2024-04-25T09:04:37.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19e130c640 1 -- start start 2024-04-25T09:04:37.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19e130c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f19dc15a3c0 0x7f19dc0ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19e130c640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f19dc15ac30 0x7f19dc0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19e130c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19dc15e0f0 con 0x7f19dc15a7a0 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19e130c640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19dc0b2eb0 con 0x7f19dc15ac30 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19e130c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19dc0b24a0 con 0x7f19dc15a3c0 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19db7fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f19dc15ac30 0x7f19dc0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19da7fc640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f19dc15a7a0 0x7f19dc0b27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56216/0 (socket says 172.21.15.37:56216) 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19da7fc640 1 -- 172.21.15.37:0/3234046115 learned_addr learned my addr 172.21.15.37:0/3234046115 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19daffd640 1 --2- 172.21.15.37:0/3234046115 >> v2:172.21.15.203:3300/0 conn(0x7f19dc15a3c0 0x7f19dc0ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19db7fe640 1 -- 172.21.15.37:0/3234046115 >> v2:172.21.15.203:3300/0 conn(0x7f19dc15a3c0 msgr2=0x7f19dc0ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19db7fe640 1 --2- 172.21.15.37:0/3234046115 >> v2:172.21.15.203:3300/0 conn(0x7f19dc15a3c0 0x7f19dc0ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19db7fe640 1 -- 172.21.15.37:0/3234046115 >> v1:172.21.15.37:6789/0 conn(0x7f19dc15a7a0 legacy=0x7f19dc0b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19db7fe640 1 -- 172.21.15.37:0/3234046115 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc15b0b0 con 0x7f19dc15ac30 2024-04-25T09:04:37.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.520+0000 7f19d9ffb640 1 -- 172.21.15.37:0/3234046115 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687908206 0 0) 0x7f19dc15e0f0 con 0x7f19dc15a7a0 2024-04-25T09:04:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19db7fe640 1 --2- 172.21.15.37:0/3234046115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc15ac30 0x7f19dc0cf510 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0cbcc0 tx=0x7f19d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6abdf1a60d9181b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19d9ffb640 1 -- 172.21.15.37:0/3234046115 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d40b2070 con 0x7f19dc15ac30 2024-04-25T09:04:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19d9ffb640 1 -- 172.21.15.37:0/3234046115 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d40a9070 con 0x7f19dc15ac30 2024-04-25T09:04:37.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19d9ffb640 1 -- 172.21.15.37:0/3234046115 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d40ad880 con 0x7f19dc15ac30 2024-04-25T09:04:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19e130c640 1 -- 172.21.15.37:0/3234046115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc15ac30 msgr2=0x7f19dc0cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19e130c640 1 --2- 172.21.15.37:0/3234046115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc15ac30 0x7f19dc0cf510 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f19dc0cbcc0 tx=0x7f19d401c450 comp rx=0 tx=0).stop 2024-04-25T09:04:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19e130c640 1 -- 172.21.15.37:0/3234046115 shutdown_connections 2024-04-25T09:04:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19e130c640 1 --2- 172.21.15.37:0/3234046115 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc15ac30 0x7f19dc0cf510 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19e130c640 1 --2- 172.21.15.37:0/3234046115 >> v2:172.21.15.203:3300/0 conn(0x7f19dc15a3c0 0x7f19dc0ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19e130c640 1 -- 172.21.15.37:0/3234046115 >> 172.21.15.37:0/3234046115 conn(0x7f19dc06ae60 msgr2=0x7f19dc06b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19e130c640 1 -- 172.21.15.37:0/3234046115 shutdown_connections 2024-04-25T09:04:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19e130c640 1 -- 172.21.15.37:0/3234046115 wait complete. 2024-04-25T09:04:37.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19e130c640 1 Processor -- start 2024-04-25T09:04:37.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.524+0000 7f19e130c640 1 -- start start 2024-04-25T09:04:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19e130c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19dc15a3c0 0x7f19dc070e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19e130c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc15a7a0 0x7f19dc153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19e130c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19dc15ac30 0x7f19dc072c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19e130c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc167570 con 0x7f19dc15a7a0 2024-04-25T09:04:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19e130c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc1673f0 con 0x7f19dc15a3c0 2024-04-25T09:04:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19e130c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc1676f0 con 0x7f19dc15ac30 2024-04-25T09:04:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19da7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc15a7a0 0x7f19dc153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19da7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc15a7a0 0x7f19dc153d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50330/0 (socket says 172.21.15.37:50330) 2024-04-25T09:04:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19da7fc640 1 -- 172.21.15.37:0/474678880 learned_addr learned my addr 172.21.15.37:0/474678880 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19daffd640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19dc15a3c0 0x7f19dc070e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19db7fe640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19dc15ac30 0x7f19dc072c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19da7fc640 1 -- 172.21.15.37:0/474678880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19dc15ac30 msgr2=0x7f19dc072c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19da7fc640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f19dc15ac30 0x7f19dc072c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19da7fc640 1 -- 172.21.15.37:0/474678880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19dc15a3c0 msgr2=0x7f19dc070e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19da7fc640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f19dc15a3c0 0x7f19dc070e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19da7fc640 1 -- 172.21.15.37:0/474678880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc071390 con 0x7f19dc15a7a0 2024-04-25T09:04:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19da7fc640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc15a7a0 0x7f19dc153d00 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f19cc001570 tx=0x7f19cc00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19c37fe640 1 -- 172.21.15.37:0/474678880 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc018070 con 0x7f19dc15a7a0 2024-04-25T09:04:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19c37fe640 1 -- 172.21.15.37:0/474678880 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc00a640 con 0x7f19dc15a7a0 2024-04-25T09:04:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19e130c640 1 -- 172.21.15.37:0/474678880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19dc153a60 con 0x7f19dc15a7a0 2024-04-25T09:04:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19c37fe640 1 -- 172.21.15.37:0/474678880 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc00aa30 con 0x7f19dc15a7a0 2024-04-25T09:04:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19e130c640 1 -- 172.21.15.37:0/474678880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19dc154c00 con 0x7f19dc15a7a0 2024-04-25T09:04:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19e130c640 1 -- 172.21.15.37:0/474678880 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19dc067c30 con 0x7f19dc15a7a0 2024-04-25T09:04:37.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19c37fe640 1 -- 172.21.15.37:0/474678880 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f19cc01b070 con 0x7f19dc15a7a0 2024-04-25T09:04:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19c37fe640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19c8066f90 0x7f19c8069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19c37fe640 1 -- 172.21.15.37:0/474678880 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f19cc084d80 con 0x7f19dc15a7a0 2024-04-25T09:04:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19c37fe640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x7f19c806d570 0x7f19c806f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19c37fe640 1 -- 172.21.15.37:0/474678880 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f19cc016e70 con 0x7f19c806d570 2024-04-25T09:04:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19c37fe640 1 -- 172.21.15.37:0/474678880 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19cc085130 con 0x7f19dc15a7a0 2024-04-25T09:04:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.528+0000 7f19daffd640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19c8066f90 0x7f19c8069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.532+0000 7f19db7fe640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x7f19c806d570 0x7f19c806f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.532+0000 7f19db7fe640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x7f19c806d570 0x7f19c806f9b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.532+0000 7f19daffd640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19c8066f90 0x7f19c8069450 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f19d000b9d0 tx=0x7f19d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.532+0000 7f19c37fe640 1 -- 172.21.15.37:0/474678880 <== osd.3 v2:172.21.15.37:6800/4126478460 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f19cc016e70 con 0x7f19c806d570 2024-04-25T09:04:37.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19e130c640 1 -- 172.21.15.37:0/474678880 --> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f19dc0aea60 con 0x7f19c806d570 2024-04-25T09:04:37.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19c37fe640 1 -- 172.21.15.37:0/474678880 <== osd.3 v2:172.21.15.37:6800/4126478460 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f19dc0aea60 con 0x7f19c806d570 2024-04-25T09:04:37.535 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:37.535 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:37.535 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:37.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19e130c640 1 -- 172.21.15.37:0/474678880 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x7f19c806d570 msgr2=0x7f19c806f9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19e130c640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.37:6800/4126478460,v1:172.21.15.37:6801/4126478460] conn(0x7f19c806d570 0x7f19c806f9b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19e130c640 1 -- 172.21.15.37:0/474678880 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19c8066f90 msgr2=0x7f19c8069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19e130c640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f19c8066f90 0x7f19c8069450 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f19d000b9d0 tx=0x7f19d0008040 comp rx=0 tx=0).stop 2024-04-25T09:04:37.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19e130c640 1 -- 172.21.15.37:0/474678880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc15a7a0 msgr2=0x7f19dc153d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19e130c640 1 --2- 172.21.15.37:0/474678880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f19dc15a7a0 0x7f19dc153d00 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7f19cc001570 tx=0x7f19cc00cd20 comp rx=0 tx=0).stop 2024-04-25T09:04:37.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19db7fe640 1 -- 172.21.15.37:0/474678880 reap_dead start 2024-04-25T09:04:37.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19e130c640 1 -- 172.21.15.37:0/474678880 shutdown_connections 2024-04-25T09:04:37.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19e130c640 1 -- 172.21.15.37:0/474678880 >> 172.21.15.37:0/474678880 conn(0x7f19dc06ae60 msgr2=0x7f19dc06b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19e130c640 1 -- 172.21.15.37:0/474678880 shutdown_connections 2024-04-25T09:04:37.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.544+0000 7f19e130c640 1 -- 172.21.15.37:0/474678880 wait complete. 2024-04-25T09:04:37.552 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T09:04:37.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbf5d2640 1 Processor -- start 2024-04-25T09:04:37.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbf5d2640 1 -- start start 2024-04-25T09:04:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbf5d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb815aac0 0x7fdcb80ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbf5d2640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdcb80cd180 0x7fdcb815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbf5d2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb80ce320 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbf5d2640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb80cd5d0 con 0x7fdcb80cd180 2024-04-25T09:04:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbf5d2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80ce0d0 con 0x7fdcb815aac0 2024-04-25T09:04:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbd36e640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb815aac0 0x7fdcb80ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbdb6f640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fdcb80cd180 0x7fdcb815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbcb6d640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fdcb80cf0e0 0x7fdcb815aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:48090/0 (socket says 172.21.15.37:48090) 2024-04-25T09:04:37.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbcb6d640 1 -- 172.21.15.37:0/2621009813 learned_addr learned my addr 172.21.15.37:0/2621009813 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:37.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbdb6f640 1 --2- 172.21.15.37:0/2621009813 >> v2:172.21.15.203:3300/0 conn(0x7fdcb80cd180 0x7fdcb815b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.203:3300/0 says I am v2:172.21.15.37:42696/0 (socket says 172.21.15.37:42696) 2024-04-25T09:04:37.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbdb6f640 1 -- 172.21.15.37:0/2621009813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb815aac0 msgr2=0x7fdcb80ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:37.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbdb6f640 1 --2- 172.21.15.37:0/2621009813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb815aac0 0x7fdcb80ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbdb6f640 1 -- 172.21.15.37:0/2621009813 >> v1:172.21.15.203:6789/0 conn(0x7fdcb80cf0e0 legacy=0x7fdcb815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.680+0000 7fdcbdb6f640 1 -- 172.21.15.37:0/2621009813 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80ce600 con 0x7fdcb80cd180 2024-04-25T09:04:37.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcbdb6f640 1 --2- 172.21.15.37:0/2621009813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb80cd180 0x7fdcb815b5b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdca80097c0 tx=0x7fdca801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e438206a30d358df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcb7fff640 1 -- 172.21.15.37:0/2621009813 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca80b2070 con 0x7fdcb80cd180 2024-04-25T09:04:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcb7fff640 1 -- 172.21.15.37:0/2621009813 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdca80ad640 con 0x7fdcb80cd180 2024-04-25T09:04:37.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcb7fff640 1 -- 172.21.15.37:0/2621009813 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdca80ada10 con 0x7fdcb80cd180 2024-04-25T09:04:37.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/2621009813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb80cd180 msgr2=0x7fdcb815b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcbf5d2640 1 --2- 172.21.15.37:0/2621009813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb80cd180 0x7fdcb815b5b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fdca80097c0 tx=0x7fdca801c4a0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/2621009813 shutdown_connections 2024-04-25T09:04:37.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcbf5d2640 1 --2- 172.21.15.37:0/2621009813 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb80cd180 0x7fdcb815b5b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcbf5d2640 1 --2- 172.21.15.37:0/2621009813 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb815aac0 0x7fdcb80ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/2621009813 >> 172.21.15.37:0/2621009813 conn(0x7fdcb8099a70 msgr2=0x7fdcb80d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/2621009813 shutdown_connections 2024-04-25T09:04:37.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.684+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/2621009813 wait complete. 2024-04-25T09:04:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbf5d2640 1 Processor -- start 2024-04-25T09:04:37.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbf5d2640 1 -- start start 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbf5d2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb80cd180 0x7fdcb8152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbf5d2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcb80cf0e0 0x7fdcb81528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbf5d2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb815aac0 0x7fdcb8152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbf5d2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80cb9a0 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbf5d2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80cb820 con 0x7fdcb80cd180 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbf5d2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80cbb20 con 0x7fdcb815aac0 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbcb6d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcb80cf0e0 0x7fdcb81528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbcb6d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcb80cf0e0 0x7fdcb81528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50338/0 (socket says 172.21.15.37:50338) 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbcb6d640 1 -- 172.21.15.37:0/4163909667 learned_addr learned my addr 172.21.15.37:0/4163909667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbd36e640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb80cd180 0x7fdcb8152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbdb6f640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb815aac0 0x7fdcb8152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbcb6d640 1 -- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb815aac0 msgr2=0x7fdcb8152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbcb6d640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb815aac0 0x7fdcb8152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbcb6d640 1 -- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb80cd180 msgr2=0x7fdcb8152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbcb6d640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb80cd180 0x7fdcb8152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbcb6d640 1 -- 172.21.15.37:0/4163909667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80c5300 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbdb6f640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb815aac0 0x7fdcb8152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:37.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbcb6d640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcb80cf0e0 0x7fdcb81528a0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00c420 tx=0x7fdcac00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcb5ffb640 1 -- 172.21.15.37:0/4163909667 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac017070 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcb5ffb640 1 -- 172.21.15.37:0/4163909667 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcac005cf0 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/4163909667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb80c7870 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcb5ffb640 1 -- 172.21.15.37:0/4163909667 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac0072f0 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.688+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/4163909667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb80c55a0 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.692+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/4163909667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb804efa0 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.692+0000 7fdcb5ffb640 1 -- 172.21.15.37:0/4163909667 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fdcac01a070 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.692+0000 7fdcb5ffb640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdca4066f60 0x7fdca4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.692+0000 7fdcb5ffb640 1 -- 172.21.15.37:0/4163909667 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fdcac0843e0 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.692+0000 7fdcbd36e640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdca4066f60 0x7fdca4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.692+0000 7fdcbd36e640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdca4066f60 0x7fdca4069420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fdcb81f77a0 tx=0x7fdcb000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.696+0000 7fdcb5ffb640 1 -- 172.21.15.37:0/4163909667 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fdcac055410 con 0x7fdcb80cf0e0 2024-04-25T09:04:37.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.856+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/4163909667 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcb8158990 con 0x7fdca4066f60 2024-04-25T09:04:37.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.856+0000 7fdcb5ffb640 1 -- 172.21.15.37:0/4163909667 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdcb8158990 con 0x7fdca4066f60 2024-04-25T09:04:37.847 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T09:04:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdca4066f60 msgr2=0x7fdca4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdca4066f60 0x7fdca4069420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fdcb81f77a0 tx=0x7fdcb000b040 comp rx=0 tx=0).stop 2024-04-25T09:04:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/4163909667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcb80cf0e0 msgr2=0x7fdcb81528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcb80cf0e0 0x7fdcb81528a0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00c420 tx=0x7fdcac00c8e0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/4163909667 shutdown_connections 2024-04-25T09:04:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fdca4066f60 0x7fdca4069420 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fdcb815aac0 0x7fdcb8152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdcb80cf0e0 0x7fdcb81528a0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 --2- 172.21.15.37:0/4163909667 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fdcb80cd180 0x7fdcb8152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/4163909667 >> 172.21.15.37:0/4163909667 conn(0x7fdcb8099a70 msgr2=0x7fdcb8154de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/4163909667 shutdown_connections 2024-04-25T09:04:37.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:37.860+0000 7fdcbf5d2640 1 -- 172.21.15.37:0/4163909667 wait complete. 2024-04-25T09:04:37.866 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T09:04:37.866 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:38.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00eddad640 1 Processor -- start 2024-04-25T09:04:38.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00eddad640 1 -- start start 2024-04-25T09:04:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00eddad640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f00e80ce150 0x7f00e80ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00eddad640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f00e815b370 0x7f00e815b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00eddad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e806bd40 0x7f00e806c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00eddad640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e815b0c0 con 0x7f00e80ce150 2024-04-25T09:04:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00eddad640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00e80cea70 con 0x7f00e815b370 2024-04-25T09:04:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00eddad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e806da80 con 0x7f00e806bd40 2024-04-25T09:04:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00e77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f00e80ce150 0x7f00e80ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00e77fe640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f00e80ce150 0x7f00e80ce530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50346/0 (socket says 172.21.15.37:50346) 2024-04-25T09:04:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00e77fe640 1 -- 172.21.15.37:0/2750646721 learned_addr learned my addr 172.21.15.37:0/2750646721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00e7fff640 1 --2- 172.21.15.37:0/2750646721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e806bd40 0x7f00e806c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00e6ffd640 1 --2- 172.21.15.37:0/2750646721 >> v2:172.21.15.203:3300/0 conn(0x7f00e815b370 0x7f00e815b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00e77fe640 1 -- 172.21.15.37:0/2750646721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e806bd40 msgr2=0x7f00e806c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00e77fe640 1 --2- 172.21.15.37:0/2750646721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e806bd40 0x7f00e806c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00e77fe640 1 -- 172.21.15.37:0/2750646721 >> v2:172.21.15.203:3300/0 conn(0x7f00e815b370 msgr2=0x7f00e815b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00e77fe640 1 --2- 172.21.15.37:0/2750646721 >> v2:172.21.15.203:3300/0 conn(0x7f00e815b370 0x7f00e815b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.012+0000 7f00e77fe640 1 -- 172.21.15.37:0/2750646721 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e806dd60 con 0x7f00e80ce150 2024-04-25T09:04:38.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00e77fe640 1 --2- 172.21.15.37:0/2750646721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e80ce150 0x7f00e80ce530 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f00e80cd760 tx=0x7f00d801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf45f3af5620b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00e67fc640 1 -- 172.21.15.37:0/2750646721 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d80ab430 con 0x7f00e80ce150 2024-04-25T09:04:38.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00e67fc640 1 -- 172.21.15.37:0/2750646721 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d80b0070 con 0x7f00e80ce150 2024-04-25T09:04:38.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00e67fc640 1 -- 172.21.15.37:0/2750646721 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d80ab7f0 con 0x7f00e80ce150 2024-04-25T09:04:38.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00eddad640 1 -- 172.21.15.37:0/2750646721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e80ce150 msgr2=0x7f00e80ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00eddad640 1 --2- 172.21.15.37:0/2750646721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e80ce150 0x7f00e80ce530 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7f00e80cd760 tx=0x7f00d801c4f0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00eddad640 1 -- 172.21.15.37:0/2750646721 shutdown_connections 2024-04-25T09:04:38.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00eddad640 1 --2- 172.21.15.37:0/2750646721 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e806bd40 0x7f00e806c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00eddad640 1 --2- 172.21.15.37:0/2750646721 >> v2:172.21.15.203:3300/0 conn(0x7f00e815b370 0x7f00e815b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00eddad640 1 --2- 172.21.15.37:0/2750646721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e80ce150 0x7f00e80ce530 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00eddad640 1 -- 172.21.15.37:0/2750646721 >> 172.21.15.37:0/2750646721 conn(0x7f00e80cc680 msgr2=0x7f00e80cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00eddad640 1 -- 172.21.15.37:0/2750646721 shutdown_connections 2024-04-25T09:04:38.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00eddad640 1 -- 172.21.15.37:0/2750646721 wait complete. 2024-04-25T09:04:38.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00eddad640 1 Processor -- start 2024-04-25T09:04:38.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.016+0000 7f00eddad640 1 -- start start 2024-04-25T09:04:38.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00eddad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e806bd40 0x7f00e815de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00eddad640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80ce150 0x7f00e81575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00eddad640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e815b370 0x7f00e80af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00eddad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e806ee00 con 0x7f00e806bd40 2024-04-25T09:04:38.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00eddad640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e806e8e0 con 0x7f00e815b370 2024-04-25T09:04:38.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00eddad640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e806ef80 con 0x7f00e80ce150 2024-04-25T09:04:38.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e806bd40 0x7f00e815de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e806bd40 0x7f00e815de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50356/0 (socket says 172.21.15.37:50356) 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e77fe640 1 -- 172.21.15.37:0/2812910342 learned_addr learned my addr 172.21.15.37:0/2812910342 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e7fff640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e815b370 0x7f00e80af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e6ffd640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80ce150 0x7f00e81575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e77fe640 1 -- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80ce150 msgr2=0x7f00e81575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e77fe640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f00e80ce150 0x7f00e81575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e77fe640 1 -- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e815b370 msgr2=0x7f00e80af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e77fe640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f00e815b370 0x7f00e80af8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e77fe640 1 -- 172.21.15.37:0/2812910342 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e8158070 con 0x7f00e806bd40 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e77fe640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e806bd40 0x7f00e815de50 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f00d801aba0 tx=0x7f00d8019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00dffff640 1 -- 172.21.15.37:0/2812910342 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d80abd70 con 0x7f00e806bd40 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00dffff640 1 -- 172.21.15.37:0/2812910342 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d80bd070 con 0x7f00e806bd40 2024-04-25T09:04:38.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00eddad640 1 -- 172.21.15.37:0/2812910342 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e8158350 con 0x7f00e806bd40 2024-04-25T09:04:38.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00dffff640 1 -- 172.21.15.37:0/2812910342 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d80b4660 con 0x7f00e806bd40 2024-04-25T09:04:38.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00eddad640 1 -- 172.21.15.37:0/2812910342 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e8158910 con 0x7f00e806bd40 2024-04-25T09:04:38.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00eddad640 1 -- 172.21.15.37:0/2812910342 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f00e8158ce0 con 0x7f00e806bd40 2024-04-25T09:04:38.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00dffff640 1 -- 172.21.15.37:0/2812910342 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f00d80b4800 con 0x7f00e806bd40 2024-04-25T09:04:38.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00dffff640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00c8066f60 0x7f00c8069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00dffff640 1 -- 172.21.15.37:0/2812910342 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f00d811fa00 con 0x7f00e806bd40 2024-04-25T09:04:38.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00dffff640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x7f00c806d540 0x7f00c806f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00dffff640 1 -- 172.21.15.37:0/2812910342 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f00d80a9ec0 con 0x7f00c806d540 2024-04-25T09:04:38.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00dffff640 1 -- 172.21.15.37:0/2812910342 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00d811fe90 con 0x7f00e806bd40 2024-04-25T09:04:38.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e6ffd640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00c8066f60 0x7f00c8069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.020+0000 7f00e7fff640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x7f00c806d540 0x7f00c806f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.024+0000 7f00e6ffd640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00c8066f60 0x7f00c8069420 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f00e81f7230 tx=0x7f00d000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.024+0000 7f00e7fff640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x7f00c806d540 0x7f00c806f980 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.024+0000 7f00dffff640 1 -- 172.21.15.37:0/2812910342 <== osd.4 v2:172.21.15.203:6802/714177927 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f00d80a9ec0 con 0x7f00c806d540 2024-04-25T09:04:38.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.036+0000 7f00eddad640 1 -- 172.21.15.37:0/2812910342 --> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f00e8068340 con 0x7f00c806d540 2024-04-25T09:04:38.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00dffff640 1 -- 172.21.15.37:0/2812910342 <== osd.4 v2:172.21.15.203:6802/714177927 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f00e8068340 con 0x7f00c806d540 2024-04-25T09:04:38.028 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:38.028 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:38.028 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:38.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00eddad640 1 -- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x7f00c806d540 msgr2=0x7f00c806f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00eddad640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:6802/714177927,v1:172.21.15.203:6804/714177927] conn(0x7f00c806d540 0x7f00c806f980 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00eddad640 1 -- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00c8066f60 msgr2=0x7f00c8069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00eddad640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f00c8066f60 0x7f00c8069420 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f00e81f7230 tx=0x7f00d000b040 comp rx=0 tx=0).stop 2024-04-25T09:04:38.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00eddad640 1 -- 172.21.15.37:0/2812910342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e806bd40 msgr2=0x7f00e815de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00eddad640 1 --2- 172.21.15.37:0/2812910342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f00e806bd40 0x7f00e815de50 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7f00d801aba0 tx=0x7f00d8019290 comp rx=0 tx=0).stop 2024-04-25T09:04:38.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00e7fff640 1 -- 172.21.15.37:0/2812910342 reap_dead start 2024-04-25T09:04:38.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00eddad640 1 -- 172.21.15.37:0/2812910342 shutdown_connections 2024-04-25T09:04:38.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00eddad640 1 -- 172.21.15.37:0/2812910342 >> 172.21.15.37:0/2812910342 conn(0x7f00e80cc680 msgr2=0x7f00e80b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00eddad640 1 -- 172.21.15.37:0/2812910342 shutdown_connections 2024-04-25T09:04:38.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.040+0000 7f00eddad640 1 -- 172.21.15.37:0/2812910342 wait complete. 2024-04-25T09:04:38.045 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T09:04:38.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe553d81640 1 Processor -- start 2024-04-25T09:04:38.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe553d81640 1 -- start start 2024-04-25T09:04:38.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe553d81640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c071e90 0x7fe54c161fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe553d81640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe54c071410 0x7fe54c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe553d81640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54c162680 con 0x7fe54c071030 2024-04-25T09:04:38.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe553d81640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54c162960 con 0x7fe54c071410 2024-04-25T09:04:38.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe553d81640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c162c40 con 0x7fe54c071e90 2024-04-25T09:04:38.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe55231e640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fe54c071410 0x7fe54c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe551b1d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fe54c071030 0x7fe54c0b24a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56218/0 (socket says 172.21.15.37:56218) 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe551b1d640 1 -- 172.21.15.37:0/2571541987 learned_addr learned my addr 172.21.15.37:0/2571541987 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe55131c640 1 --2- 172.21.15.37:0/2571541987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c071e90 0x7fe54c161fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe55231e640 1 -- 172.21.15.37:0/2571541987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c071e90 msgr2=0x7fe54c161fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe55231e640 1 --2- 172.21.15.37:0/2571541987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c071e90 0x7fe54c161fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe55231e640 1 -- 172.21.15.37:0/2571541987 >> v1:172.21.15.37:6789/0 conn(0x7fe54c071030 legacy=0x7fe54c0b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.172+0000 7fe55231e640 1 -- 172.21.15.37:0/2571541987 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c162f20 con 0x7fe54c071410 2024-04-25T09:04:38.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe55231e640 1 --2- 172.21.15.37:0/2571541987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c071410 0x7fe54c06d090 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fe5480097c0 tx=0x7fe54801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c60d47980eca93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe550b1b640 1 -- 172.21.15.37:0/2571541987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5480b0070 con 0x7fe54c071410 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe550b1b640 1 -- 172.21.15.37:0/2571541987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5480ab600 con 0x7fe54c071410 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe550b1b640 1 -- 172.21.15.37:0/2571541987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5480ab9d0 con 0x7fe54c071410 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 -- 172.21.15.37:0/2571541987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c071410 msgr2=0x7fe54c06d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 --2- 172.21.15.37:0/2571541987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c071410 0x7fe54c06d090 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fe5480097c0 tx=0x7fe54801c410 comp rx=0 tx=0).stop 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 -- 172.21.15.37:0/2571541987 shutdown_connections 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 --2- 172.21.15.37:0/2571541987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c071410 0x7fe54c06d090 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 --2- 172.21.15.37:0/2571541987 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c071e90 0x7fe54c161fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 -- 172.21.15.37:0/2571541987 >> 172.21.15.37:0/2571541987 conn(0x7fe54c0b11a0 msgr2=0x7fe54c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 -- 172.21.15.37:0/2571541987 shutdown_connections 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 -- 172.21.15.37:0/2571541987 wait complete. 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 Processor -- start 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 -- start start 2024-04-25T09:04:38.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c071030 0x7fe54c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe54c071410 0x7fe54c15c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c071e90 0x7fe54c152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c06e170 con 0x7fe54c071e90 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c06dff0 con 0x7fe54c071410 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe553d81640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c06e2f0 con 0x7fe54c071030 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe55231e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c071e90 0x7fe54c152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe55231e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c071e90 0x7fe54c152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50370/0 (socket says 172.21.15.37:50370) 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe55231e640 1 -- 172.21.15.37:0/100322788 learned_addr learned my addr 172.21.15.37:0/100322788 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe55231e640 1 -- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c071030 msgr2=0x7fe54c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe55231e640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c071030 0x7fe54c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe55231e640 1 -- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe54c071410 msgr2=0x7fe54c15c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe55231e640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe54c071410 0x7fe54c15c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe55231e640 1 -- 172.21.15.37:0/100322788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c154290 con 0x7fe54c071e90 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe55231e640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c071e90 0x7fe54c152580 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fe54801a040 tx=0x7fe5480191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe5427fc640 1 -- 172.21.15.37:0/100322788 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5480b8020 con 0x7fe54c071e90 2024-04-25T09:04:38.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe5427fc640 1 -- 172.21.15.37:0/100322788 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5480b4610 con 0x7fe54c071e90 2024-04-25T09:04:38.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.176+0000 7fe5427fc640 1 -- 172.21.15.37:0/100322788 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5480b0040 con 0x7fe54c071e90 2024-04-25T09:04:38.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.180+0000 7fe553d81640 1 -- 172.21.15.37:0/100322788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54c159830 con 0x7fe54c071e90 2024-04-25T09:04:38.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.180+0000 7fe553d81640 1 -- 172.21.15.37:0/100322788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54c15c7d0 con 0x7fe54c071e90 2024-04-25T09:04:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.180+0000 7fe553d81640 1 -- 172.21.15.37:0/100322788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54c04efa0 con 0x7fe54c071e90 2024-04-25T09:04:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.180+0000 7fe5427fc640 1 -- 172.21.15.37:0/100322788 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fe5480bb020 con 0x7fe54c071e90 2024-04-25T09:04:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.180+0000 7fe5427fc640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe530066f60 0x7fe530069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.180+0000 7fe5427fc640 1 -- 172.21.15.37:0/100322788 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fe54811ee40 con 0x7fe54c071e90 2024-04-25T09:04:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.180+0000 7fe551b1d640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe530066f60 0x7fe530069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.180+0000 7fe551b1d640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe530066f60 0x7fe530069420 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe5440012f0 tx=0x7fe544001640 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.184+0000 7fe5427fc640 1 -- 172.21.15.37:0/100322788 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fe5480c2030 con 0x7fe54c071e90 2024-04-25T09:04:38.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 -- 172.21.15.37:0/100322788 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe54c0b41e0 con 0x7fe530066f60 2024-04-25T09:04:38.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe5427fc640 1 -- 172.21.15.37:0/100322788 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe54c0b41e0 con 0x7fe530066f60 2024-04-25T09:04:38.333 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T09:04:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 -- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe530066f60 msgr2=0x7fe530069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe530066f60 0x7fe530069420 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fe5440012f0 tx=0x7fe544001640 comp rx=0 tx=0).stop 2024-04-25T09:04:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 -- 172.21.15.37:0/100322788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c071e90 msgr2=0x7fe54c152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c071e90 0x7fe54c152580 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fe54801a040 tx=0x7fe5480191b0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 -- 172.21.15.37:0/100322788 shutdown_connections 2024-04-25T09:04:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fe530066f60 0x7fe530069420 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe54c071e90 0x7fe54c152580 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fe54c071410 0x7fe54c15c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 --2- 172.21.15.37:0/100322788 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fe54c071030 0x7fe54c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 -- 172.21.15.37:0/100322788 >> 172.21.15.37:0/100322788 conn(0x7fe54c0b11a0 msgr2=0x7fe54c1555f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 -- 172.21.15.37:0/100322788 shutdown_connections 2024-04-25T09:04:38.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.344+0000 7fe553d81640 1 -- 172.21.15.37:0/100322788 wait complete. 2024-04-25T09:04:38.353 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T09:04:38.354 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.488+0000 7f345f349640 1 Processor -- start 2024-04-25T09:04:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.488+0000 7f345f349640 1 -- start start 2024-04-25T09:04:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.488+0000 7f345f349640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f34580cf510 0x7f34580cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.488+0000 7f345f349640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f34580ac170 0x7f34580ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.488+0000 7f345f349640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34580b28c0 con 0x7f34580adc70 2024-04-25T09:04:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.488+0000 7f345f349640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34580cfec0 con 0x7f34580cf510 2024-04-25T09:04:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.488+0000 7f345f349640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345815bb20 con 0x7f34580ac170 2024-04-25T09:04:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.488+0000 7f345c8e4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f34580cf510 0x7f34580cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.488+0000 7f345c8e4640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f34580cf510 0x7f34580cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50378/0 (socket says 172.21.15.37:50378) 2024-04-25T09:04:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.488+0000 7f345c8e4640 1 -- 172.21.15.37:0/2881177722 learned_addr learned my addr 172.21.15.37:0/2881177722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345d8e6640 1 --2- 172.21.15.37:0/2881177722 >> v2:172.21.15.203:3300/0 conn(0x7f34580ac170 0x7f34580ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345c8e4640 1 -- 172.21.15.37:0/2881177722 >> v2:172.21.15.203:3300/0 conn(0x7f34580ac170 msgr2=0x7f34580ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345c8e4640 1 --2- 172.21.15.37:0/2881177722 >> v2:172.21.15.203:3300/0 conn(0x7f34580ac170 0x7f34580ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345c8e4640 1 -- 172.21.15.37:0/2881177722 >> v1:172.21.15.203:6789/0 conn(0x7f34580adc70 legacy=0x7f345806e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345c8e4640 1 -- 172.21.15.37:0/2881177722 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34580b2b60 con 0x7f34580cf510 2024-04-25T09:04:38.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345c8e4640 1 --2- 172.21.15.37:0/2881177722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580cf510 0x7f34580cf980 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f345806f460 tx=0x7f345401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91a07b333798d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f344ffff640 1 -- 172.21.15.37:0/2881177722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34540b1070 con 0x7f34580cf510 2024-04-25T09:04:38.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f344ffff640 1 -- 172.21.15.37:0/2881177722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34540ac410 con 0x7f34580cf510 2024-04-25T09:04:38.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f344ffff640 1 -- 172.21.15.37:0/2881177722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34540ac7e0 con 0x7f34580cf510 2024-04-25T09:04:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345f349640 1 -- 172.21.15.37:0/2881177722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580cf510 msgr2=0x7f34580cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345f349640 1 --2- 172.21.15.37:0/2881177722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580cf510 0x7f34580cf980 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f345806f460 tx=0x7f345401c4c0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345f349640 1 -- 172.21.15.37:0/2881177722 shutdown_connections 2024-04-25T09:04:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345f349640 1 --2- 172.21.15.37:0/2881177722 >> v2:172.21.15.203:3300/0 conn(0x7f34580ac170 0x7f34580ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345f349640 1 --2- 172.21.15.37:0/2881177722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580cf510 0x7f34580cf980 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345f349640 1 -- 172.21.15.37:0/2881177722 >> 172.21.15.37:0/2881177722 conn(0x7f345806ae60 msgr2=0x7f345806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345f349640 1 -- 172.21.15.37:0/2881177722 shutdown_connections 2024-04-25T09:04:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.492+0000 7f345f349640 1 -- 172.21.15.37:0/2881177722 wait complete. 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345f349640 1 Processor -- start 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345f349640 1 -- start start 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345f349640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580ac170 0x7f34580c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345f349640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34580adc70 0x7f34580b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345f349640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34580cf510 0x7f34580b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345f349640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34580b3c00 con 0x7f34580ac170 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345f349640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34580b36e0 con 0x7f34580adc70 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345f349640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34580b1d00 con 0x7f34580cf510 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345d0e5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580ac170 0x7f34580c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345d0e5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580ac170 0x7f34580c3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50382/0 (socket says 172.21.15.37:50382) 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345d0e5640 1 -- 172.21.15.37:0/1286089391 learned_addr learned my addr 172.21.15.37:0/1286089391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345c8e4640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34580adc70 0x7f34580b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345d8e6640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34580cf510 0x7f34580b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345d0e5640 1 -- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34580cf510 msgr2=0x7f34580b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345d0e5640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34580cf510 0x7f34580b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345d0e5640 1 -- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34580adc70 msgr2=0x7f34580b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345d0e5640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34580adc70 0x7f34580b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345d0e5640 1 -- 172.21.15.37:0/1286089391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34580b1a20 con 0x7f34580ac170 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345c8e4640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f34580adc70 0x7f34580b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345d8e6640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f34580cf510 0x7f34580b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345d0e5640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580ac170 0x7f34580c3c80 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f344800c450 tx=0x7f344800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f344dffb640 1 -- 172.21.15.37:0/1286089391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448012830 con 0x7f34580ac170 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f344dffb640 1 -- 172.21.15.37:0/1286089391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3448017070 con 0x7f34580ac170 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f344dffb640 1 -- 172.21.15.37:0/1286089391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448012c30 con 0x7f34580ac170 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345f349640 1 -- 172.21.15.37:0/1286089391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34580c4ed0 con 0x7f34580ac170 2024-04-25T09:04:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345f349640 1 -- 172.21.15.37:0/1286089391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34580c5490 con 0x7f34580ac170 2024-04-25T09:04:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.496+0000 7f345f349640 1 -- 172.21.15.37:0/1286089391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3458067c30 con 0x7f34580ac170 2024-04-25T09:04:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.500+0000 7f344dffb640 1 -- 172.21.15.37:0/1286089391 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f344801c460 con 0x7f34580ac170 2024-04-25T09:04:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.500+0000 7f344dffb640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f343c066f60 0x7f343c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.500+0000 7f344dffb640 1 -- 172.21.15.37:0/1286089391 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f3448085860 con 0x7f34580ac170 2024-04-25T09:04:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.500+0000 7f344dffb640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x7f343c06d540 0x7f343c06f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.500+0000 7f344dffb640 1 -- 172.21.15.37:0/1286089391 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34480054b0 con 0x7f343c06d540 2024-04-25T09:04:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.500+0000 7f344dffb640 1 -- 172.21.15.37:0/1286089391 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3448085c10 con 0x7f34580ac170 2024-04-25T09:04:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.500+0000 7f345c8e4640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f343c066f60 0x7f343c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.500+0000 7f345d8e6640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x7f343c06d540 0x7f343c06f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.500+0000 7f345d8e6640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x7f343c06d540 0x7f343c06f980 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.500+0000 7f345c8e4640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f343c066f60 0x7f343c069420 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f34540ae000 tx=0x7f34540a6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.500+0000 7f344dffb640 1 -- 172.21.15.37:0/1286089391 <== osd.5 v2:172.21.15.203:6808/213476662 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f34480054b0 con 0x7f343c06d540 2024-04-25T09:04:38.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345f349640 1 -- 172.21.15.37:0/1286089391 --> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f345806d760 con 0x7f343c06d540 2024-04-25T09:04:38.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f344dffb640 1 -- 172.21.15.37:0/1286089391 <== osd.5 v2:172.21.15.203:6808/213476662 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f345806d760 con 0x7f343c06d540 2024-04-25T09:04:38.506 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:38.506 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:38.506 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:38.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345f349640 1 -- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x7f343c06d540 msgr2=0x7f343c06f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345f349640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:6808/213476662,v1:172.21.15.203:6811/213476662] conn(0x7f343c06d540 0x7f343c06f980 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345f349640 1 -- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f343c066f60 msgr2=0x7f343c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345f349640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f343c066f60 0x7f343c069420 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f34540ae000 tx=0x7f34540a6040 comp rx=0 tx=0).stop 2024-04-25T09:04:38.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345f349640 1 -- 172.21.15.37:0/1286089391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580ac170 msgr2=0x7f34580c3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345f349640 1 --2- 172.21.15.37:0/1286089391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34580ac170 0x7f34580c3c80 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f344800c450 tx=0x7f344800c910 comp rx=0 tx=0).stop 2024-04-25T09:04:38.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345d8e6640 1 -- 172.21.15.37:0/1286089391 reap_dead start 2024-04-25T09:04:38.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345f349640 1 -- 172.21.15.37:0/1286089391 shutdown_connections 2024-04-25T09:04:38.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345f349640 1 -- 172.21.15.37:0/1286089391 >> 172.21.15.37:0/1286089391 conn(0x7f345806ae60 msgr2=0x7f34580714d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345f349640 1 -- 172.21.15.37:0/1286089391 shutdown_connections 2024-04-25T09:04:38.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.516+0000 7f345f349640 1 -- 172.21.15.37:0/1286089391 wait complete. 2024-04-25T09:04:38.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T09:04:38.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.648+0000 7ff2a9cc9640 1 Processor -- start 2024-04-25T09:04:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.648+0000 7ff2a9cc9640 1 -- start start 2024-04-25T09:04:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.648+0000 7ff2a9cc9640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7ff2a40b2ac0 0x7ff2a40cb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.648+0000 7ff2a9cc9640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff2a415e1e0 0x7ff2a415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.648+0000 7ff2a9cc9640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a415eb30 con 0x7ff2a415a600 2024-04-25T09:04:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.648+0000 7ff2a9cc9640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a40ce070 con 0x7ff2a415e1e0 2024-04-25T09:04:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.648+0000 7ff2a9cc9640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a40ce350 con 0x7ff2a40b2ac0 2024-04-25T09:04:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.648+0000 7ff2a3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff2a415e1e0 0x7ff2a415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.648+0000 7ff2a3fff640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7ff2a415e1e0 0x7ff2a415e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50384/0 (socket says 172.21.15.37:50384) 2024-04-25T09:04:38.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.648+0000 7ff2a3fff640 1 -- 172.21.15.37:0/1472609404 learned_addr learned my addr 172.21.15.37:0/1472609404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:38.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.648+0000 7ff2a2ffd640 1 --2- 172.21.15.37:0/1472609404 >> v2:172.21.15.203:3300/0 conn(0x7ff2a40b2ac0 0x7ff2a40cb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.652+0000 7ff2a3fff640 1 -- 172.21.15.37:0/1472609404 >> v2:172.21.15.203:3300/0 conn(0x7ff2a40b2ac0 msgr2=0x7ff2a40cb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.652+0000 7ff2a3fff640 1 --2- 172.21.15.37:0/1472609404 >> v2:172.21.15.203:3300/0 conn(0x7ff2a40b2ac0 0x7ff2a40cb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.652+0000 7ff2a3fff640 1 -- 172.21.15.37:0/1472609404 >> v1:172.21.15.203:6789/0 conn(0x7ff2a415a600 legacy=0x7ff2a40cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.652+0000 7ff2a3fff640 1 -- 172.21.15.37:0/1472609404 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40ce630 con 0x7ff2a415e1e0 2024-04-25T09:04:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.652+0000 7ff2a3fff640 1 --2- 172.21.15.37:0/1472609404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2a415e1e0 0x7ff2a415e5f0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4074860 tx=0x7ff29c01f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120102013851ad0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.652+0000 7ff2a27fc640 1 -- 172.21.15.37:0/1472609404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0b4070 con 0x7ff2a415e1e0 2024-04-25T09:04:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.652+0000 7ff2a27fc640 1 -- 172.21.15.37:0/1472609404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c0af400 con 0x7ff2a415e1e0 2024-04-25T09:04:38.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.652+0000 7ff2a27fc640 1 -- 172.21.15.37:0/1472609404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0af7d0 con 0x7ff2a415e1e0 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/1472609404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2a415e1e0 msgr2=0x7ff2a415e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 --2- 172.21.15.37:0/1472609404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2a415e1e0 0x7ff2a415e5f0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4074860 tx=0x7ff29c01f1e0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/1472609404 shutdown_connections 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 --2- 172.21.15.37:0/1472609404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2a415e1e0 0x7ff2a415e5f0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 --2- 172.21.15.37:0/1472609404 >> v2:172.21.15.203:3300/0 conn(0x7ff2a40b2ac0 0x7ff2a40cb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/1472609404 >> 172.21.15.37:0/1472609404 conn(0x7ff2a40b1f10 msgr2=0x7ff2a40b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/1472609404 shutdown_connections 2024-04-25T09:04:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/1472609404 wait complete. 2024-04-25T09:04:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 Processor -- start 2024-04-25T09:04:38.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 -- start start 2024-04-25T09:04:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2a40b2ac0 0x7ff2a4159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2a415a600 0x7ff2a40c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2a415e1e0 0x7ff2a40c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40cc1e0 con 0x7ff2a415e1e0 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a40cbef0 con 0x7ff2a415a600 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a9cc9640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a415a9e0 con 0x7ff2a40b2ac0 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2a415e1e0 0x7ff2a40c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2a415e1e0 0x7ff2a40c9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50390/0 (socket says 172.21.15.37:50390) 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a3fff640 1 -- 172.21.15.37:0/3497367488 learned_addr learned my addr 172.21.15.37:0/3497367488 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a2ffd640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2a415a600 0x7ff2a40c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a3fff640 1 -- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2a40b2ac0 msgr2=0x7ff2a4159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a37fe640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2a40b2ac0 0x7ff2a4159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a3fff640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2a40b2ac0 0x7ff2a4159a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a3fff640 1 -- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2a415a600 msgr2=0x7ff2a40c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a3fff640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2a415a600 0x7ff2a40c5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a3fff640 1 -- 172.21.15.37:0/3497367488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a4159f60 con 0x7ff2a415e1e0 2024-04-25T09:04:38.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff2a3fff640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2a415e1e0 0x7ff2a40c9c80 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0b2040 tx=0x7ff29c01bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff28bfff640 1 -- 172.21.15.37:0/3497367488 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c0b4040 con 0x7ff2a415e1e0 2024-04-25T09:04:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff28bfff640 1 -- 172.21.15.37:0/3497367488 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c0c0070 con 0x7ff2a415e1e0 2024-04-25T09:04:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.656+0000 7ff28bfff640 1 -- 172.21.15.37:0/3497367488 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c009510 con 0x7ff2a415e1e0 2024-04-25T09:04:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.660+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/3497367488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a40c7a40 con 0x7ff2a415e1e0 2024-04-25T09:04:38.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.660+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/3497367488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a40c8000 con 0x7ff2a415e1e0 2024-04-25T09:04:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.660+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/3497367488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a404efa0 con 0x7ff2a415e1e0 2024-04-25T09:04:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.660+0000 7ff28bfff640 1 -- 172.21.15.37:0/3497367488 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7ff29c0c5460 con 0x7ff2a415e1e0 2024-04-25T09:04:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.660+0000 7ff28bfff640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 0x7ff290069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.660+0000 7ff28bfff640 1 -- 172.21.15.37:0/3497367488 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7ff29c122e90 con 0x7ff2a415e1e0 2024-04-25T09:04:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.660+0000 7ff2a37fe640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 0x7ff290069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.660+0000 7ff2a37fe640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 0x7ff290069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff294007fd0 tx=0x7ff294009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.664+0000 7ff28bfff640 1 -- 172.21.15.37:0/3497367488 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7ff29c0ca020 con 0x7ff2a415e1e0 2024-04-25T09:04:38.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/3497367488 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2a415b0d0 con 0x7ff290066f60 2024-04-25T09:04:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff28bfff640 1 -- 172.21.15.37:0/3497367488 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff2a415b0d0 con 0x7ff290066f60 2024-04-25T09:04:38.817 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T09:04:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 msgr2=0x7ff290069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 0x7ff290069420 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff294007fd0 tx=0x7ff294009040 comp rx=0 tx=0).stop 2024-04-25T09:04:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/3497367488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2a415e1e0 msgr2=0x7ff2a40c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2a415e1e0 0x7ff2a40c9c80 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0b2040 tx=0x7ff29c01bf80 comp rx=0 tx=0).stop 2024-04-25T09:04:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/3497367488 shutdown_connections 2024-04-25T09:04:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7ff290066f60 0x7ff290069420 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2a415e1e0 0x7ff2a40c9c80 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7ff2a415a600 0x7ff2a40c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 --2- 172.21.15.37:0/3497367488 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7ff2a40b2ac0 0x7ff2a4159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/3497367488 >> 172.21.15.37:0/3497367488 conn(0x7ff2a40b1f10 msgr2=0x7ff2a415d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/3497367488 shutdown_connections 2024-04-25T09:04:38.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.828+0000 7ff2a9cc9640 1 -- 172.21.15.37:0/3497367488 wait complete. 2024-04-25T09:04:38.836 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T09:04:38.837 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:38.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8b1a9c640 1 Processor -- start 2024-04-25T09:04:38.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8b1a9c640 1 -- start start 2024-04-25T09:04:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8b1a9c640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fc8ac15d9f0 0x7fc8ac0d36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8b1a9c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac06c6d0 0x7fc8ac0d3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8b1a9c640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ac06d5d0 con 0x7fc8ac15ddd0 2024-04-25T09:04:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8b1a9c640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ac06cd30 con 0x7fc8ac15d9f0 2024-04-25T09:04:38.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8b1a9c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac06d010 con 0x7fc8ac06c6d0 2024-04-25T09:04:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8aaffd640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fc8ac15ddd0 0x7fc8ac06bfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56224/0 (socket says 172.21.15.37:56224) 2024-04-25T09:04:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8aaffd640 1 -- 172.21.15.37:0/3751249713 learned_addr learned my addr 172.21.15.37:0/3751249713 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8abfff640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac06c6d0 0x7fc8ac0d3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8ab7fe640 1 --2- 172.21.15.37:0/3751249713 >> v2:172.21.15.203:3300/0 conn(0x7fc8ac15d9f0 0x7fc8ac0d36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793141992 0 0) 0x7fc8ac06d5d0 con 0x7fc8ac15ddd0 2024-04-25T09:04:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.980+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc898003530 con 0x7fc8ac15ddd0 2024-04-25T09:04:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3643695718 0 0) 0x7fc898003530 con 0x7fc8ac15ddd0 2024-04-25T09:04:38.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac06c6d0 msgr2=0x7fc8ac0d3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac06c6d0 0x7fc8ac0d3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 >> v2:172.21.15.203:3300/0 conn(0x7fc8ac15d9f0 msgr2=0x7fc8ac0d36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 --2- 172.21.15.37:0/3751249713 >> v2:172.21.15.203:3300/0 conn(0x7fc8ac15d9f0 0x7fc8ac0d36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac06d2f0 con 0x7fc8ac15ddd0 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7fc89c001a00 con 0x7fc8ac15ddd0 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 >> v1:172.21.15.37:6789/0 conn(0x7fc8ac15ddd0 legacy=0x7fc8ac06bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac06c6d0 0x7fc8980039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc898003f40 0x7fc898004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc898004d50 0x7fc8980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac06cd30 con 0x7fc898003f40 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac06d010 con 0x7fc898004d50 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89800b9d0 con 0x7fc8ac06c6d0 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aaffd640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc898003f40 0x7fc898004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8ab7fe640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac06c6d0 0x7fc8980039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8abfff640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc898004d50 0x7fc8980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aaffd640 1 -- 172.21.15.37:0/3751249713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac06c6d0 msgr2=0x7fc8980039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aaffd640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac06c6d0 0x7fc8980039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aaffd640 1 -- 172.21.15.37:0/3751249713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc898004d50 msgr2=0x7fc8980051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aaffd640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc898004d50 0x7fc8980051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aaffd640 1 -- 172.21.15.37:0/3751249713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc898003530 con 0x7fc898003f40 2024-04-25T09:04:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aaffd640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc898003f40 0x7fc898004810 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00a8a0 tx=0x7fc89c01fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888d55c54bc0f18e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c00a4c0 con 0x7fc898003f40 2024-04-25T09:04:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89c0ad020 con 0x7fc898003f40 2024-04-25T09:04:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8aa7fc640 1 -- 172.21.15.37:0/3751249713 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0a9740 con 0x7fc898003f40 2024-04-25T09:04:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/3751249713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc898003f40 msgr2=0x7fc898004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8b1a9c640 1 --2- 172.21.15.37:0/3751249713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc898003f40 0x7fc898004810 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00a8a0 tx=0x7fc89c01fac0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8abfff640 1 -- 172.21.15.37:0/3751249713 reap_dead start 2024-04-25T09:04:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/3751249713 shutdown_connections 2024-04-25T09:04:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/3751249713 >> 172.21.15.37:0/3751249713 conn(0x7fc8ac09ca80 msgr2=0x7fc8ac08b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/3751249713 shutdown_connections 2024-04-25T09:04:38.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.984+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/3751249713 wait complete. 2024-04-25T09:04:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8b1a9c640 1 Processor -- start 2024-04-25T09:04:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8b1a9c640 1 -- start start 2024-04-25T09:04:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8b1a9c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc898003f40 0x7fc8ac0ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8b1a9c640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ac0b6f00 0x7fc8ac0b7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8b1a9c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac1598f0 0x7fc8ac159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8b1a9c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac0d3db0 con 0x7fc898003f40 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8b1a9c640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac0d3c30 con 0x7fc8ac0b6f00 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8b1a9c640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac0d3f30 con 0x7fc8ac1598f0 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8ab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc898003f40 0x7fc8ac0ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8ab7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc898003f40 0x7fc8ac0ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50408/0 (socket says 172.21.15.37:50408) 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8ab7fe640 1 -- 172.21.15.37:0/2057562193 learned_addr learned my addr 172.21.15.37:0/2057562193 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8aaffd640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ac0b6f00 0x7fc8ac0b7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8abfff640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac1598f0 0x7fc8ac159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8ab7fe640 1 -- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac1598f0 msgr2=0x7fc8ac159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8ab7fe640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fc8ac1598f0 0x7fc8ac159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8ab7fe640 1 -- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ac0b6f00 msgr2=0x7fc8ac0b7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8ab7fe640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fc8ac0b6f00 0x7fc8ac0b7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8ab7fe640 1 -- 172.21.15.37:0/2057562193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac15e570 con 0x7fc898003f40 2024-04-25T09:04:38.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8ab7fe640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc898003f40 0x7fc8ac0ccaa0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fc8940116d0 tx=0x7fc894011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8a3fff640 1 -- 172.21.15.37:0/2057562193 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400f9c0 con 0x7fc898003f40 2024-04-25T09:04:38.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8a3fff640 1 -- 172.21.15.37:0/2057562193 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89400fb60 con 0x7fc898003f40 2024-04-25T09:04:38.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/2057562193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac15e850 con 0x7fc898003f40 2024-04-25T09:04:38.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8a3fff640 1 -- 172.21.15.37:0/2057562193 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894015460 con 0x7fc898003f40 2024-04-25T09:04:38.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/2057562193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac15ee10 con 0x7fc898003f40 2024-04-25T09:04:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/2057562193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc8ac068340 con 0x7fc898003f40 2024-04-25T09:04:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.988+0000 7fc8a3fff640 1 -- 172.21.15.37:0/2057562193 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fc894015600 con 0x7fc898003f40 2024-04-25T09:04:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.992+0000 7fc8a3fff640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc88c066f90 0x7fc88c069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.992+0000 7fc8a3fff640 1 -- 172.21.15.37:0/2057562193 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fc894085e40 con 0x7fc898003f40 2024-04-25T09:04:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.992+0000 7fc8a3fff640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x7fc88c06d570 0x7fc88c06f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.992+0000 7fc8a3fff640 1 -- 172.21.15.37:0/2057562193 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc894004480 con 0x7fc88c06d570 2024-04-25T09:04:38.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.992+0000 7fc8a3fff640 1 -- 172.21.15.37:0/2057562193 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8940862d0 con 0x7fc898003f40 2024-04-25T09:04:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.992+0000 7fc8aaffd640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc88c066f90 0x7fc88c069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.992+0000 7fc8abfff640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x7fc88c06d570 0x7fc88c06f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.992+0000 7fc8aaffd640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc88c066f90 0x7fc88c069450 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0ae000 tx=0x7fc89c001e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.992+0000 7fc8abfff640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x7fc88c06d570 0x7fc88c06f9b0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:38.992+0000 7fc8a3fff640 1 -- 172.21.15.37:0/2057562193 <== osd.6 v2:172.21.15.203:6800/467003745 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc894004480 con 0x7fc88c06d570 2024-04-25T09:04:38.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/2057562193 --> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc8ac0c8900 con 0x7fc88c06d570 2024-04-25T09:04:38.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8a3fff640 1 -- 172.21.15.37:0/2057562193 <== osd.6 v2:172.21.15.203:6800/467003745 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc8ac0c8900 con 0x7fc88c06d570 2024-04-25T09:04:38.997 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:38.997 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:38.997 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:38.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x7fc88c06d570 msgr2=0x7fc88c06f9b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8b1a9c640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:6800/467003745,v1:172.21.15.203:6801/467003745] conn(0x7fc88c06d570 0x7fc88c06f9b0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc88c066f90 msgr2=0x7fc88c069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8b1a9c640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fc88c066f90 0x7fc88c069450 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0ae000 tx=0x7fc89c001e40 comp rx=0 tx=0).stop 2024-04-25T09:04:38.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/2057562193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc898003f40 msgr2=0x7fc8ac0ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8b1a9c640 1 --2- 172.21.15.37:0/2057562193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc898003f40 0x7fc8ac0ccaa0 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fc8940116d0 tx=0x7fc894011b90 comp rx=0 tx=0).stop 2024-04-25T09:04:38.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8abfff640 1 -- 172.21.15.37:0/2057562193 reap_dead start 2024-04-25T09:04:39.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/2057562193 shutdown_connections 2024-04-25T09:04:39.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/2057562193 >> 172.21.15.37:0/2057562193 conn(0x7fc8ac09ca80 msgr2=0x7fc8ac0d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/2057562193 shutdown_connections 2024-04-25T09:04:39.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.008+0000 7fc8b1a9c640 1 -- 172.21.15.37:0/2057562193 wait complete. 2024-04-25T09:04:39.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T09:04:39.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3ef6dc2640 1 Processor -- start 2024-04-25T09:04:39.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3ef6dc2640 1 -- start start 2024-04-25T09:04:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3ef6dc2640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f3ef00d0870 0x7f3ef00d1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3ef6dc2640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef00ce6b0 con 0x7f3ef015d850 2024-04-25T09:04:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3ef6dc2640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef00ce990 con 0x7f3ef015dc30 2024-04-25T09:04:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3ef6dc2640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef00cec70 con 0x7f3ef00d0870 2024-04-25T09:04:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3ef4b5e640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f3ef015d850 0x7f3ef00d0160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:56234/0 (socket says 172.21.15.37:56234) 2024-04-25T09:04:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3ef4b5e640 1 -- 172.21.15.37:0/621837709 learned_addr learned my addr 172.21.15.37:0/621837709 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3eeffff640 1 --2- 172.21.15.37:0/621837709 >> v2:172.21.15.37:3300/0 conn(0x7f3ef00d0870 0x7f3ef00d1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3eef7fe640 1 -- 172.21.15.37:0/621837709 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 356695656 0 0) 0x7f3ef00ce6b0 con 0x7f3ef015d850 2024-04-25T09:04:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3eef7fe640 1 -- 172.21.15.37:0/621837709 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed8003530 con 0x7f3ef015d850 2024-04-25T09:04:39.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3eeffff640 1 -- 172.21.15.37:0/621837709 >> v1:172.21.15.203:6789/0 conn(0x7f3ef015dc30 legacy=0x7f3ef0167910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3eeffff640 1 -- 172.21.15.37:0/621837709 >> v1:172.21.15.37:6789/0 conn(0x7f3ef015d850 legacy=0x7f3ef00d0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.144+0000 7f3eeffff640 1 -- 172.21.15.37:0/621837709 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00cef50 con 0x7f3ef00d0870 2024-04-25T09:04:39.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3eeffff640 1 --2- 172.21.15.37:0/621837709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef00d0870 0x7f3ef00d1de0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d90 tx=0x7f3ee401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c6fd67ad318947 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3eef7fe640 1 -- 172.21.15.37:0/621837709 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b2070 con 0x7f3ef00d0870 2024-04-25T09:04:39.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3eef7fe640 1 -- 172.21.15.37:0/621837709 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee40a59f0 con 0x7f3ef00d0870 2024-04-25T09:04:39.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3eef7fe640 1 -- 172.21.15.37:0/621837709 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40a5da0 con 0x7f3ef00d0870 2024-04-25T09:04:39.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/621837709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef00d0870 msgr2=0x7f3ef00d1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3ef6dc2640 1 --2- 172.21.15.37:0/621837709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef00d0870 0x7f3ef00d1de0 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d90 tx=0x7f3ee401c970 comp rx=0 tx=0).stop 2024-04-25T09:04:39.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/621837709 shutdown_connections 2024-04-25T09:04:39.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3ef6dc2640 1 --2- 172.21.15.37:0/621837709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef00d0870 0x7f3ef00d1de0 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/621837709 >> 172.21.15.37:0/621837709 conn(0x7f3ef008b000 msgr2=0x7f3ef00d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/621837709 shutdown_connections 2024-04-25T09:04:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/621837709 wait complete. 2024-04-25T09:04:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3ef6dc2640 1 Processor -- start 2024-04-25T09:04:39.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.148+0000 7f3ef6dc2640 1 -- start start 2024-04-25T09:04:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef6dc2640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef00d0870 0x7f3ef00cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef6dc2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef015d850 0x7f3ef015c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef6dc2640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef015dc30 0x7f3ef015cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef6dc2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef016b9d0 con 0x7f3ef015d850 2024-04-25T09:04:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef6dc2640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef016b850 con 0x7f3ef00d0870 2024-04-25T09:04:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef6dc2640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef016bb50 con 0x7f3ef015dc30 2024-04-25T09:04:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eeffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef015d850 0x7f3ef015c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eeffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef015d850 0x7f3ef015c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50434/0 (socket says 172.21.15.37:50434) 2024-04-25T09:04:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eeffff640 1 -- 172.21.15.37:0/1993582907 learned_addr learned my addr 172.21.15.37:0/1993582907 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef4b5e640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef00d0870 0x7f3ef00cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef535f640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef015dc30 0x7f3ef015cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eeffff640 1 -- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef015dc30 msgr2=0x7f3ef015cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eeffff640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef015dc30 0x7f3ef015cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eeffff640 1 -- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef00d0870 msgr2=0x7f3ef00cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eeffff640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef00d0870 0x7f3ef00cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eeffff640 1 -- 172.21.15.37:0/1993582907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef00cbdd0 con 0x7f3ef015d850 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef535f640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef015dc30 0x7f3ef015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eeffff640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef015d850 0x7f3ef015c5f0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d90 tx=0x7f3ee4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eed7fa640 1 -- 172.21.15.37:0/1993582907 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee40b2050 con 0x7f3ef015d850 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eed7fa640 1 -- 172.21.15.37:0/1993582907 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee40b3040 con 0x7f3ef015d850 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/1993582907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef00cc0a0 con 0x7f3ef015d850 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eed7fa640 1 -- 172.21.15.37:0/1993582907 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4003db0 con 0x7f3ef015d850 2024-04-25T09:04:39.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/1993582907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef00cc660 con 0x7f3ef015d850 2024-04-25T09:04:39.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/1993582907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef004efa0 con 0x7f3ef015d850 2024-04-25T09:04:39.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eed7fa640 1 -- 172.21.15.37:0/1993582907 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f3ee40c0020 con 0x7f3ef015d850 2024-04-25T09:04:39.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eed7fa640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3edc066f60 0x7f3edc069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3eed7fa640 1 -- 172.21.15.37:0/1993582907 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f3ee411f910 con 0x7f3ef015d850 2024-04-25T09:04:39.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.152+0000 7f3ef4b5e640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3edc066f60 0x7f3edc069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.156+0000 7f3ef4b5e640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3edc066f60 0x7f3edc069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00011f0 tx=0x7f3ee0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.156+0000 7f3eed7fa640 1 -- 172.21.15.37:0/1993582907 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f3ee4016560 con 0x7f3ef015d850 2024-04-25T09:04:39.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.320+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/1993582907 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ef00d0e60 con 0x7f3edc066f60 2024-04-25T09:04:39.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.320+0000 7f3eed7fa640 1 -- 172.21.15.37:0/1993582907 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3ef00d0e60 con 0x7f3edc066f60 2024-04-25T09:04:39.310 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T09:04:39.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3edc066f60 msgr2=0x7f3edc069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3edc066f60 0x7f3edc069420 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00011f0 tx=0x7f3ee0009040 comp rx=0 tx=0).stop 2024-04-25T09:04:39.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/1993582907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef015d850 msgr2=0x7f3ef015c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef015d850 0x7f3ef015c5f0 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4009d90 tx=0x7f3ee4019710 comp rx=0 tx=0).stop 2024-04-25T09:04:39.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/1993582907 shutdown_connections 2024-04-25T09:04:39.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f3edc066f60 0x7f3edc069420 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f3ef015dc30 0x7f3ef015cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ef015d850 0x7f3ef015c5f0 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 --2- 172.21.15.37:0/1993582907 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f3ef00d0870 0x7f3ef00cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/1993582907 >> 172.21.15.37:0/1993582907 conn(0x7f3ef008b000 msgr2=0x7f3ef015e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/1993582907 shutdown_connections 2024-04-25T09:04:39.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.324+0000 7f3ef6dc2640 1 -- 172.21.15.37:0/1993582907 wait complete. 2024-04-25T09:04:39.330 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T09:04:39.330 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.472+0000 7fb0af79a640 1 Processor -- start 2024-04-25T09:04:39.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.472+0000 7fb0af79a640 1 -- start start 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.472+0000 7fb0af79a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fb0a815a3c0 0x7fb0a80ae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.472+0000 7fb0af79a640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb0a815a7a0 0x7fb0a80b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.472+0000 7fb0af79a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0a815e0f0 con 0x7fb0a815ac30 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.472+0000 7fb0af79a640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a80b24a0 con 0x7fb0a815a7a0 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0af79a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0a815b0b0 con 0x7fb0a815a3c0 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0acd35640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7fb0a815a7a0 0x7fb0a80b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0add37640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7fb0a815ac30 0x7fb0a80cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:46334/0 (socket says 172.21.15.37:46334) 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0add37640 1 -- 172.21.15.37:0/1840894795 learned_addr learned my addr 172.21.15.37:0/1840894795 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0ad536640 1 --2- 172.21.15.37:0/1840894795 >> v2:172.21.15.203:3300/0 conn(0x7fb0a815a3c0 0x7fb0a80ae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0acd35640 1 -- 172.21.15.37:0/1840894795 >> v2:172.21.15.203:3300/0 conn(0x7fb0a815a3c0 msgr2=0x7fb0a80ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0acd35640 1 --2- 172.21.15.37:0/1840894795 >> v2:172.21.15.203:3300/0 conn(0x7fb0a815a3c0 0x7fb0a80ae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0acd35640 1 -- 172.21.15.37:0/1840894795 >> v1:172.21.15.37:6789/0 conn(0x7fb0a815ac30 legacy=0x7fb0a80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0acd35640 1 -- 172.21.15.37:0/1840894795 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a815b390 con 0x7fb0a815a7a0 2024-04-25T09:04:39.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0acd35640 1 --2- 172.21.15.37:0/1840894795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a815a7a0 0x7fb0a80b27a0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80cbcc0 tx=0x7fb09801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35bfa0c118a7f538 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb09ffff640 1 -- 172.21.15.37:0/1840894795 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0980a7070 con 0x7fb0a815a7a0 2024-04-25T09:04:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb09ffff640 1 -- 172.21.15.37:0/1840894795 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0980b0070 con 0x7fb0a815a7a0 2024-04-25T09:04:39.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb09ffff640 1 -- 172.21.15.37:0/1840894795 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0980ab890 con 0x7fb0a815a7a0 2024-04-25T09:04:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0af79a640 1 -- 172.21.15.37:0/1840894795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a815a7a0 msgr2=0x7fb0a80b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0af79a640 1 --2- 172.21.15.37:0/1840894795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a815a7a0 0x7fb0a80b27a0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80cbcc0 tx=0x7fb09801c450 comp rx=0 tx=0).stop 2024-04-25T09:04:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0af79a640 1 -- 172.21.15.37:0/1840894795 shutdown_connections 2024-04-25T09:04:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0af79a640 1 --2- 172.21.15.37:0/1840894795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a815a7a0 0x7fb0a80b27a0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0af79a640 1 --2- 172.21.15.37:0/1840894795 >> v2:172.21.15.203:3300/0 conn(0x7fb0a815a3c0 0x7fb0a80ae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0af79a640 1 -- 172.21.15.37:0/1840894795 >> 172.21.15.37:0/1840894795 conn(0x7fb0a806ae60 msgr2=0x7fb0a806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0af79a640 1 -- 172.21.15.37:0/1840894795 shutdown_connections 2024-04-25T09:04:39.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.476+0000 7fb0af79a640 1 -- 172.21.15.37:0/1840894795 wait complete. 2024-04-25T09:04:39.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0af79a640 1 Processor -- start 2024-04-25T09:04:39.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0af79a640 1 -- start start 2024-04-25T09:04:39.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0af79a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0a815a3c0 0x7fb0a8155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0af79a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a815a7a0 0x7fb0a8157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0af79a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a815ac30 0x7fb0a80cbfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0af79a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a815edb0 con 0x7fb0a815ac30 2024-04-25T09:04:39.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0af79a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a80b2ef0 con 0x7fb0a815a7a0 2024-04-25T09:04:39.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0af79a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0a815ef30 con 0x7fb0a815a3c0 2024-04-25T09:04:39.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0add37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a815ac30 0x7fb0a80cbfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0add37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a815ac30 0x7fb0a80cbfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52280/0 (socket says 172.21.15.37:52280) 2024-04-25T09:04:39.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0add37640 1 -- 172.21.15.37:0/412153650 learned_addr learned my addr 172.21.15.37:0/412153650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0ad536640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0a815a3c0 0x7fb0a8155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0acd35640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a815a7a0 0x7fb0a8157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0add37640 1 -- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0a815a3c0 msgr2=0x7fb0a8155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0add37640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0a815a3c0 0x7fb0a8155a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0add37640 1 -- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a815a7a0 msgr2=0x7fb0a8157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0add37640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a815a7a0 0x7fb0a8157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0add37640 1 -- 172.21.15.37:0/412153650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a8156050 con 0x7fb0a815ac30 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0ad536640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fb0a815a3c0 0x7fb0a8155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0acd35640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fb0a815a7a0 0x7fb0a8157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0add37640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a815ac30 0x7fb0a80cbfa0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000cab0 tx=0x7fb0a000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb09dffb640 1 -- 172.21.15.37:0/412153650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0017070 con 0x7fb0a815ac30 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb09dffb640 1 -- 172.21.15.37:0/412153650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a000ac30 con 0x7fb0a815ac30 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb09dffb640 1 -- 172.21.15.37:0/412153650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a0012400 con 0x7fb0a815ac30 2024-04-25T09:04:39.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0af79a640 1 -- 172.21.15.37:0/412153650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a8156330 con 0x7fb0a815ac30 2024-04-25T09:04:39.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0af79a640 1 -- 172.21.15.37:0/412153650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0a8157f50 con 0x7fb0a815ac30 2024-04-25T09:04:39.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.480+0000 7fb0af79a640 1 -- 172.21.15.37:0/412153650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0a8067c30 con 0x7fb0a815ac30 2024-04-25T09:04:39.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.484+0000 7fb09dffb640 1 -- 172.21.15.37:0/412153650 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fb0a001a070 con 0x7fb0a815ac30 2024-04-25T09:04:39.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.484+0000 7fb09dffb640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb08c066f60 0x7fb08c069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.484+0000 7fb09dffb640 1 -- 172.21.15.37:0/412153650 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fb0a0084c30 con 0x7fb0a815ac30 2024-04-25T09:04:39.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.484+0000 7fb09dffb640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x7fb08c06d540 0x7fb08c06f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.484+0000 7fb09dffb640 1 -- 172.21.15.37:0/412153650 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0a0014ec0 con 0x7fb08c06d540 2024-04-25T09:04:39.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.484+0000 7fb09dffb640 1 -- 172.21.15.37:0/412153650 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0a00892a0 con 0x7fb0a815ac30 2024-04-25T09:04:39.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.484+0000 7fb0ad536640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb08c066f60 0x7fb08c069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.484+0000 7fb0acd35640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x7fb08c06d540 0x7fb08c06f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.484+0000 7fb0ad536640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb08c066f60 0x7fb08c069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009840 tx=0x7fb0a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.484+0000 7fb0acd35640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x7fb08c06d540 0x7fb08c06f980 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.484+0000 7fb09dffb640 1 -- 172.21.15.37:0/412153650 <== osd.7 v2:172.21.15.203:6824/2957998038 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb0a0014ec0 con 0x7fb08c06d540 2024-04-25T09:04:39.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0af79a640 1 -- 172.21.15.37:0/412153650 --> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb0a806edf0 con 0x7fb08c06d540 2024-04-25T09:04:39.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb09dffb640 1 -- 172.21.15.37:0/412153650 <== osd.7 v2:172.21.15.203:6824/2957998038 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb0a806edf0 con 0x7fb08c06d540 2024-04-25T09:04:39.490 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-25T09:04:39.490 INFO:teuthology.orchestra.run.smithi037.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:39.490 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-25T09:04:39.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0af79a640 1 -- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x7fb08c06d540 msgr2=0x7fb08c06f980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0af79a640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:6824/2957998038,v1:172.21.15.203:6825/2957998038] conn(0x7fb08c06d540 0x7fb08c06f980 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0af79a640 1 -- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb08c066f60 msgr2=0x7fb08c069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0af79a640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fb08c066f60 0x7fb08c069420 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009840 tx=0x7fb0a4008040 comp rx=0 tx=0).stop 2024-04-25T09:04:39.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0af79a640 1 -- 172.21.15.37:0/412153650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a815ac30 msgr2=0x7fb0a80cbfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0af79a640 1 --2- 172.21.15.37:0/412153650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a815ac30 0x7fb0a80cbfa0 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7fb0a000cab0 tx=0x7fb0a000cf70 comp rx=0 tx=0).stop 2024-04-25T09:04:39.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0add37640 1 -- 172.21.15.37:0/412153650 reap_dead start 2024-04-25T09:04:39.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0af79a640 1 -- 172.21.15.37:0/412153650 shutdown_connections 2024-04-25T09:04:39.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0af79a640 1 -- 172.21.15.37:0/412153650 >> 172.21.15.37:0/412153650 conn(0x7fb0a806ae60 msgr2=0x7fb0a80cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0af79a640 1 -- 172.21.15.37:0/412153650 shutdown_connections 2024-04-25T09:04:39.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.500+0000 7fb0af79a640 1 -- 172.21.15.37:0/412153650 wait complete. 2024-04-25T09:04:39.507 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T09:04:39.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.632+0000 7fa772e5a640 1 Processor -- start 2024-04-25T09:04:39.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.632+0000 7fa772e5a640 1 -- start start 2024-04-25T09:04:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.632+0000 7fa772e5a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c144ca0 0x7fa76c06e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.632+0000 7fa772e5a640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7fa76c06ec60 0x7fa76c142680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.632+0000 7fa772e5a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76c0ccc70 con 0x7fa76c0ce510 2024-04-25T09:04:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.632+0000 7fa772e5a640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c0cc2b0 con 0x7fa76c06ec60 2024-04-25T09:04:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.632+0000 7fa772e5a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c06e290 con 0x7fa76c144ca0 2024-04-25T09:04:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.632+0000 7fa770bf6640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7fa76c0ce510 0x7fa76c0ce8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:43032/0 (socket says 172.21.15.37:43032) 2024-04-25T09:04:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.632+0000 7fa76bfff640 1 --2- 172.21.15.37:0/2269855613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c144ca0 0x7fa76c06e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.632+0000 7fa7713f7640 1 --2- 172.21.15.37:0/2269855613 >> v2:172.21.15.203:3300/0 conn(0x7fa76c06ec60 0x7fa76c142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.632+0000 7fa770bf6640 1 -- 172.21.15.37:0/2269855613 learned_addr learned my addr 172.21.15.37:0/2269855613 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:39.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa76bfff640 1 -- 172.21.15.37:0/2269855613 >> v2:172.21.15.203:3300/0 conn(0x7fa76c06ec60 msgr2=0x7fa76c142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa76bfff640 1 --2- 172.21.15.37:0/2269855613 >> v2:172.21.15.203:3300/0 conn(0x7fa76c06ec60 0x7fa76c142680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa76bfff640 1 -- 172.21.15.37:0/2269855613 >> v1:172.21.15.203:6789/0 conn(0x7fa76c0ce510 legacy=0x7fa76c0ce8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa76bfff640 1 -- 172.21.15.37:0/2269855613 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c06dc40 con 0x7fa76c144ca0 2024-04-25T09:04:39.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa7713f7640 1 --2- 172.21.15.37:0/2269855613 >> v2:172.21.15.203:3300/0 conn(0x7fa76c06ec60 0x7fa76c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:39.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa76bfff640 1 --2- 172.21.15.37:0/2269855613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c144ca0 0x7fa76c06e720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa7580097c0 tx=0x7fa75801ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93436e0092ca703e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa76b7fe640 1 -- 172.21.15.37:0/2269855613 <== mon.2 v2:172.21.15.203:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7580b5070 con 0x7fa76c144ca0 2024-04-25T09:04:39.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa76b7fe640 1 -- 172.21.15.37:0/2269855613 <== mon.2 v2:172.21.15.203:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7580a8990 con 0x7fa76c144ca0 2024-04-25T09:04:39.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa76b7fe640 1 -- 172.21.15.37:0/2269855613 <== mon.2 v2:172.21.15.203:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7580a8d60 con 0x7fa76c144ca0 2024-04-25T09:04:39.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa772e5a640 1 -- 172.21.15.37:0/2269855613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c144ca0 msgr2=0x7fa76c06e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa772e5a640 1 --2- 172.21.15.37:0/2269855613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c144ca0 0x7fa76c06e720 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa7580097c0 tx=0x7fa75801ec90 comp rx=0 tx=0).stop 2024-04-25T09:04:39.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa772e5a640 1 -- 172.21.15.37:0/2269855613 shutdown_connections 2024-04-25T09:04:39.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa772e5a640 1 --2- 172.21.15.37:0/2269855613 >> v2:172.21.15.203:3300/0 conn(0x7fa76c06ec60 0x7fa76c142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa772e5a640 1 --2- 172.21.15.37:0/2269855613 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c144ca0 0x7fa76c06e720 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa772e5a640 1 -- 172.21.15.37:0/2269855613 >> 172.21.15.37:0/2269855613 conn(0x7fa76c0b11a0 msgr2=0x7fa76c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa772e5a640 1 -- 172.21.15.37:0/2269855613 shutdown_connections 2024-04-25T09:04:39.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.636+0000 7fa772e5a640 1 -- 172.21.15.37:0/2269855613 wait complete. 2024-04-25T09:04:39.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa772e5a640 1 Processor -- start 2024-04-25T09:04:39.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa772e5a640 1 -- start start 2024-04-25T09:04:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa772e5a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c06ec60 0x7fa76c15af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa772e5a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa76c0ce510 0x7fa76c0b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa772e5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa76c144ca0 0x7fa76c0aa6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa772e5a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c1451f0 con 0x7fa76c144ca0 2024-04-25T09:04:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa772e5a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c0cc900 con 0x7fa76c0ce510 2024-04-25T09:04:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa772e5a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c145370 con 0x7fa76c06ec60 2024-04-25T09:04:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7713f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa76c144ca0 0x7fa76c0aa6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7713f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa76c144ca0 0x7fa76c0aa6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52286/0 (socket says 172.21.15.37:52286) 2024-04-25T09:04:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7713f7640 1 -- 172.21.15.37:0/3286194779 learned_addr learned my addr 172.21.15.37:0/3286194779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa770bf6640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c06ec60 0x7fa76c15af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa76bfff640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa76c0ce510 0x7fa76c0b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7713f7640 1 -- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c06ec60 msgr2=0x7fa76c15af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7713f7640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c06ec60 0x7fa76c15af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7713f7640 1 -- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa76c0ce510 msgr2=0x7fa76c0b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7713f7640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa76c0ce510 0x7fa76c0b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7713f7640 1 -- 172.21.15.37:0/3286194779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c0af290 con 0x7fa76c144ca0 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa770bf6640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c06ec60 0x7fa76c15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7713f7640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa76c144ca0 0x7fa76c0aa6e0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fa76400efc0 tx=0x7fa76400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7697fa640 1 -- 172.21.15.37:0/3286194779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764018020 con 0x7fa76c144ca0 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7697fa640 1 -- 172.21.15.37:0/3286194779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa764014480 con 0x7fa76c144ca0 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa7697fa640 1 -- 172.21.15.37:0/3286194779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764014870 con 0x7fa76c144ca0 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa772e5a640 1 -- 172.21.15.37:0/3286194779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76c0af570 con 0x7fa76c144ca0 2024-04-25T09:04:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.640+0000 7fa772e5a640 1 -- 172.21.15.37:0/3286194779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c0afb30 con 0x7fa76c144ca0 2024-04-25T09:04:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.644+0000 7fa772e5a640 1 -- 172.21.15.37:0/3286194779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c04efa0 con 0x7fa76c144ca0 2024-04-25T09:04:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.644+0000 7fa7697fa640 1 -- 172.21.15.37:0/3286194779 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7fa76401b020 con 0x7fa76c144ca0 2024-04-25T09:04:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.644+0000 7fa7697fa640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa750066f60 0x7fa750069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.644+0000 7fa7697fa640 1 -- 172.21.15.37:0/3286194779 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7fa764085dc0 con 0x7fa76c144ca0 2024-04-25T09:04:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.644+0000 7fa770bf6640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa750066f60 0x7fa750069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.644+0000 7fa770bf6640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa750066f60 0x7fa750069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa760000f70 tx=0x7fa760009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.648+0000 7fa7697fa640 1 -- 172.21.15.37:0/3286194779 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7fa764022030 con 0x7fa76c144ca0 2024-04-25T09:04:39.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.808+0000 7fa772e5a640 1 -- 172.21.15.37:0/3286194779 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fa76c0b1cc0 con 0x7fa750066f60 2024-04-25T09:04:39.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.808+0000 7fa7697fa640 1 -- 172.21.15.37:0/3286194779 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa76c0b1cc0 con 0x7fa750066f60 2024-04-25T09:04:39.798 INFO:teuthology.orchestra.run.smithi037.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 -- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa750066f60 msgr2=0x7fa750069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa750066f60 0x7fa750069420 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa760000f70 tx=0x7fa760009040 comp rx=0 tx=0).stop 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 -- 172.21.15.37:0/3286194779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa76c144ca0 msgr2=0x7fa76c0aa6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa76c144ca0 0x7fa76c0aa6e0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7fa76400efc0 tx=0x7fa76400c490 comp rx=0 tx=0).stop 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 -- 172.21.15.37:0/3286194779 shutdown_connections 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7fa750066f60 0x7fa750069420 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa76c144ca0 0x7fa76c0aa6e0 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7fa76c0ce510 0x7fa76c0b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 --2- 172.21.15.37:0/3286194779 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7fa76c06ec60 0x7fa76c15af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 -- 172.21.15.37:0/3286194779 >> 172.21.15.37:0/3286194779 conn(0x7fa76c0b11a0 msgr2=0x7fa76c0cce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 -- 172.21.15.37:0/3286194779 shutdown_connections 2024-04-25T09:04:39.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.812+0000 7fa772e5a640 1 -- 172.21.15.37:0/3286194779 wait complete. 2024-04-25T09:04:39.817 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:39.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f8807c00640 1 Processor -- start 2024-04-25T09:04:39.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f8807c00640 1 -- start start 2024-04-25T09:04:39.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f8807c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f880015ba20 0x7f88000cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f8807c00640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f880006b3a0 0x7f88000cf6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f8807c00640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f880015c260 con 0x7f880015fd90 2024-04-25T09:04:39.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f8807c00640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88000d1430 con 0x7f880006b3a0 2024-04-25T09:04:39.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f8807c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88000cc6e0 con 0x7f880015ba20 2024-04-25T09:04:39.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f880599c640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f880015ba20 0x7f88000cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f880619d640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f880006b3a0 0x7f88000cf6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f880519b640 1 --1- >> v1:172.21.15.203:6789/0 conn(0x7f880015fd90 0x7f880006ac90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.203:6789/0 says I am v1:172.21.15.37:43038/0 (socket says 172.21.15.37:43038) 2024-04-25T09:04:39.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f880519b640 1 -- 172.21.15.37:0/3716633395 learned_addr learned my addr 172.21.15.37:0/3716633395 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:04:39.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f880619d640 1 -- 172.21.15.37:0/3716633395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f880015ba20 msgr2=0x7f88000cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f880619d640 1 --2- 172.21.15.37:0/3716633395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f880015ba20 0x7f88000cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f880619d640 1 -- 172.21.15.37:0/3716633395 >> v1:172.21.15.203:6789/0 conn(0x7f880015fd90 legacy=0x7f880006ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f880619d640 1 -- 172.21.15.37:0/3716633395 --> v2:172.21.15.203:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880015c540 con 0x7f880006b3a0 2024-04-25T09:04:39.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.948+0000 7f880599c640 1 --2- 172.21.15.37:0/3716633395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f880015ba20 0x7f88000cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:39.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.952+0000 7f880619d640 1 --2- 172.21.15.37:0/3716633395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f880006b3a0 0x7f88000cf6d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009d90 tx=0x7f87f801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89775975cf495503 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.952+0000 7f880499a640 1 -- 172.21.15.37:0/3716633395 <== mon.1 v2:172.21.15.203:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80b2070 con 0x7f880006b3a0 2024-04-25T09:04:39.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.952+0000 7f880499a640 1 -- 172.21.15.37:0/3716633395 <== mon.1 v2:172.21.15.203:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f80a5920 con 0x7f880006b3a0 2024-04-25T09:04:39.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.952+0000 7f880499a640 1 -- 172.21.15.37:0/3716633395 <== mon.1 v2:172.21.15.203:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80a5cf0 con 0x7f880006b3a0 2024-04-25T09:04:39.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.952+0000 7f8807c00640 1 -- 172.21.15.37:0/3716633395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f880006b3a0 msgr2=0x7f88000cf6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.952+0000 7f8807c00640 1 --2- 172.21.15.37:0/3716633395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f880006b3a0 0x7f88000cf6d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f87f8009d90 tx=0x7f87f801c970 comp rx=0 tx=0).stop 2024-04-25T09:04:39.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.952+0000 7f8807c00640 1 -- 172.21.15.37:0/3716633395 shutdown_connections 2024-04-25T09:04:39.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.952+0000 7f8807c00640 1 --2- 172.21.15.37:0/3716633395 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f880006b3a0 0x7f88000cf6d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.952+0000 7f8807c00640 1 --2- 172.21.15.37:0/3716633395 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f880015ba20 0x7f88000cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.952+0000 7f8807c00640 1 -- 172.21.15.37:0/3716633395 >> 172.21.15.37:0/3716633395 conn(0x7f8800099a70 msgr2=0x7f88000d0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.952+0000 7f8807c00640 1 -- 172.21.15.37:0/3716633395 shutdown_connections 2024-04-25T09:04:39.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f8807c00640 1 -- 172.21.15.37:0/3716633395 wait complete. 2024-04-25T09:04:39.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f8807c00640 1 Processor -- start 2024-04-25T09:04:39.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f8807c00640 1 -- start start 2024-04-25T09:04:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f8807c00640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f880006b3a0 0x7f8800151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f8807c00640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f880015ba20 0x7f88000c5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f8807c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f880015fd90 0x7f8800158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f8807c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880015bf80 con 0x7f880015fd90 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f8807c00640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880015be00 con 0x7f880006b3a0 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f8807c00640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8800168720 con 0x7f880015ba20 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880619d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f880015fd90 0x7f8800158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880619d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f880015fd90 0x7f8800158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52296/0 (socket says 172.21.15.37:52296) 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880619d640 1 -- 172.21.15.37:0/1284488679 learned_addr learned my addr 172.21.15.37:0/1284488679 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880599c640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f880006b3a0 0x7f8800151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880519b640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f880015ba20 0x7f88000c5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880619d640 1 -- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f880015ba20 msgr2=0x7f88000c5b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880619d640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f880015ba20 0x7f88000c5b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880619d640 1 -- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f880006b3a0 msgr2=0x7f8800151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880619d640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f880006b3a0 0x7f8800151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880619d640 1 -- 172.21.15.37:0/1284488679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8800151f50 con 0x7f880015fd90 2024-04-25T09:04:39.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880599c640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f880006b3a0 0x7f8800151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:39.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f880619d640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f880015fd90 0x7f8800158c60 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f87f801b020 tx=0x7f87f8019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f87f67fc640 1 -- 172.21.15.37:0/1284488679 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80b2050 con 0x7f880015fd90 2024-04-25T09:04:39.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f87f67fc640 1 -- 172.21.15.37:0/1284488679 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f80be070 con 0x7f880015fd90 2024-04-25T09:04:39.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f8807c00640 1 -- 172.21.15.37:0/1284488679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88000c4a80 con 0x7f880015fd90 2024-04-25T09:04:39.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f87f67fc640 1 -- 172.21.15.37:0/1284488679 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f80ada70 con 0x7f880015fd90 2024-04-25T09:04:39.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.956+0000 7f8807c00640 1 -- 172.21.15.37:0/1284488679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88000c5040 con 0x7f880015fd90 2024-04-25T09:04:39.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.960+0000 7f8807c00640 1 -- 172.21.15.37:0/1284488679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880004efa0 con 0x7f880015fd90 2024-04-25T09:04:39.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.960+0000 7f87f67fc640 1 -- 172.21.15.37:0/1284488679 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f87f80adc10 con 0x7f880015fd90 2024-04-25T09:04:39.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.960+0000 7f87f67fc640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87e4066f60 0x7f87e4069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.960+0000 7f87f67fc640 1 -- 172.21.15.37:0/1284488679 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f87f8120260 con 0x7f880015fd90 2024-04-25T09:04:39.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.960+0000 7f880599c640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87e4066f60 0x7f87e4069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.960+0000 7f880599c640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87e4066f60 0x7f87e4069420 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009bc0 tx=0x7f87fc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:39.964+0000 7f87f67fc640 1 -- 172.21.15.37:0/1284488679 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f87f8016560 con 0x7f880015fd90 2024-04-25T09:04:40.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.120+0000 7f8807c00640 1 -- 172.21.15.37:0/1284488679 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f880006b7c0 con 0x7f87e4066f60 2024-04-25T09:04:40.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f87f67fc640 1 -- 172.21.15.37:0/1284488679 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13390 (secure 0 0 0) 0x7f880006b7c0 con 0x7f87e4066f60 2024-04-25T09:04:40.112 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:04:40.113 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 -- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87e4066f60 msgr2=0x7f87e4069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87e4066f60 0x7f87e4069420 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009bc0 tx=0x7f87fc008040 comp rx=0 tx=0).stop 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 -- 172.21.15.37:0/1284488679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f880015fd90 msgr2=0x7f8800158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f880015fd90 0x7f8800158c60 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f87f801b020 tx=0x7f87f8019710 comp rx=0 tx=0).stop 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 -- 172.21.15.37:0/1284488679 shutdown_connections 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f87e4066f60 0x7f87e4069420 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f880015fd90 0x7f8800158c60 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f880015ba20 0x7f88000c5b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 --2- 172.21.15.37:0/1284488679 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f880006b3a0 0x7f8800151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 -- 172.21.15.37:0/1284488679 >> 172.21.15.37:0/1284488679 conn(0x7f8800099a70 msgr2=0x7f880015d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 -- 172.21.15.37:0/1284488679 shutdown_connections 2024-04-25T09:04:40.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:04:40.124+0000 7f8807c00640 1 -- 172.21.15.37:0/1284488679 wait complete. 2024-04-25T09:04:40.133 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-04-25T09:04:39.378773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263320,"kb_used_data":13864,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711400,"statfs":{"total":773094113280,"available":772824473600,"internally_reserved":0,"allocated":14196736,"data_stored":7280896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8388625,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":-32,"num_read_kb":-13,"num_write":-29,"num_write_kb":-8195,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.475814"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":378,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:04:32.953960+0000","last_change":"2024-04-25T08:52:21.105839+0000","last_active":"2024-04-25T09:04:32.953960+0000","last_peered":"2024-04-25T09:04:32.953960+0000","last_clean":"2024-04-25T09:04:32.953960+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T09:04:32.953960+0000","last_undegraded":"2024-04-25T09:04:32.953960+0000","last_fullsized":"2024-04-25T09:04:32.953960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:52:19.682109+0000","last_clean_scrub_stamp":"2024-04-25T08:52:19.682109+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.188351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33352,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338488,"statfs":{"total":96636764160,"available":96602611712,"internally_reserved":0,"allocated":2220032,"data_stored":1352888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32772,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32768,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32772,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32768,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32768,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32768,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33352,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338488,"statfs":{"total":96636764160,"available":96602611712,"internally_reserved":0,"allocated":2220032,"data_stored":1352888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:40.136 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T08:52:19.682109+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=8, tm_min=52, tm_sec=19, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=9, tm_min=4, tm_sec=34, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T09:04:40.136 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T09:04:57.354 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:57.357 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:00.137 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:05:00.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148e642640 1 Processor -- start 2024-04-25T09:05:00.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148e642640 1 -- start start 2024-04-25T09:05:00.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148e642640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f14880b0b00 0x7f14880cc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148e642640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f14880b1ae0 0x7f1488151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148e642640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1488160310 con 0x7f14880cc650 2024-04-25T09:05:00.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148e642640 1 -- --> v2:172.21.15.37:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148815cfa0 con 0x7f14880b1ae0 2024-04-25T09:05:00.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148e642640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148815cdf0 con 0x7f14880b0b00 2024-04-25T09:05:00.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148cbdf640 1 --2- >> v2:172.21.15.37:3300/0 conn(0x7f14880b1ae0 0x7f1488151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f14877fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f14880cc650 0x7f14880b13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55002/0 (socket says 172.21.15.37:55002) 2024-04-25T09:05:00.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f14877fe640 1 -- 172.21.15.37:0/1755406793 learned_addr learned my addr 172.21.15.37:0/1755406793 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:05:00.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f1487fff640 1 --2- 172.21.15.37:0/1755406793 >> v2:172.21.15.203:3300/0 conn(0x7f14880b0b00 0x7f14880cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f1486ffd640 1 -- 172.21.15.37:0/1755406793 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2852160113 0 0) 0x7f1488160310 con 0x7f14880cc650 2024-04-25T09:05:00.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f1486ffd640 1 -- 172.21.15.37:0/1755406793 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1470003530 con 0x7f14880cc650 2024-04-25T09:05:00.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148cbdf640 1 -- 172.21.15.37:0/1755406793 >> v2:172.21.15.203:3300/0 conn(0x7f14880b0b00 msgr2=0x7f14880cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:00.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148cbdf640 1 --2- 172.21.15.37:0/1755406793 >> v2:172.21.15.203:3300/0 conn(0x7f14880b0b00 0x7f14880cc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148cbdf640 1 -- 172.21.15.37:0/1755406793 >> v1:172.21.15.37:6789/0 conn(0x7f14880cc650 legacy=0x7f14880b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.284+0000 7f148cbdf640 1 -- 172.21.15.37:0/1755406793 --> v2:172.21.15.37:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148815f550 con 0x7f14880b1ae0 2024-04-25T09:05:00.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148cbdf640 1 --2- 172.21.15.37:0/1755406793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14880b1ae0 0x7f1488151fd0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f14880ccde0 tx=0x7f148001c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b49c9e285ad908 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f1486ffd640 1 -- 172.21.15.37:0/1755406793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14800a6620 con 0x7f14880b1ae0 2024-04-25T09:05:00.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f1486ffd640 1 -- 172.21.15.37:0/1755406793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14800a67c0 con 0x7f14880b1ae0 2024-04-25T09:05:00.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f1486ffd640 1 -- 172.21.15.37:0/1755406793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14800a6b90 con 0x7f14880b1ae0 2024-04-25T09:05:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 -- 172.21.15.37:0/1755406793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14880b1ae0 msgr2=0x7f1488151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 --2- 172.21.15.37:0/1755406793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14880b1ae0 0x7f1488151fd0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f14880ccde0 tx=0x7f148001c750 comp rx=0 tx=0).stop 2024-04-25T09:05:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 -- 172.21.15.37:0/1755406793 shutdown_connections 2024-04-25T09:05:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 --2- 172.21.15.37:0/1755406793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14880b1ae0 0x7f1488151fd0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 --2- 172.21.15.37:0/1755406793 >> v2:172.21.15.203:3300/0 conn(0x7f14880b0b00 0x7f14880cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 -- 172.21.15.37:0/1755406793 >> 172.21.15.37:0/1755406793 conn(0x7f14880b1f10 msgr2=0x7f14880b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 -- 172.21.15.37:0/1755406793 shutdown_connections 2024-04-25T09:05:00.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 -- 172.21.15.37:0/1755406793 wait complete. 2024-04-25T09:05:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 Processor -- start 2024-04-25T09:05:00.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 -- start start 2024-04-25T09:05:00.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14880b0b00 0x7f14880c5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.288+0000 7f148e642640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14880b1ae0 0x7f148814c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148e642640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14880cc650 0x7f1488154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148e642640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14880b1060 con 0x7f14880cc650 2024-04-25T09:05:00.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148e642640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14880b0ee0 con 0x7f14880b0b00 2024-04-25T09:05:00.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148e642640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148815bde0 con 0x7f14880b1ae0 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148cbdf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14880cc650 0x7f1488154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148cbdf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14880cc650 0x7f1488154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52980/0 (socket says 172.21.15.37:52980) 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148cbdf640 1 -- 172.21.15.37:0/1304955166 learned_addr learned my addr 172.21.15.37:0/1304955166 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f1487fff640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14880b0b00 0x7f14880c5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f14877fe640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14880b1ae0 0x7f148814c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148cbdf640 1 -- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14880b1ae0 msgr2=0x7f148814c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148cbdf640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14880b1ae0 0x7f148814c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148cbdf640 1 -- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14880b0b00 msgr2=0x7f14880c5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148cbdf640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14880b0b00 0x7f14880c5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148cbdf640 1 -- 172.21.15.37:0/1304955166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148814c660 con 0x7f14880cc650 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f1487fff640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14880b0b00 0x7f14880c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148cbdf640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14880cc650 0x7f1488154ea0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f1480019360 tx=0x7f14800194f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f1484ff9640 1 -- 172.21.15.37:0/1304955166 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14800b0600 con 0x7f14880cc650 2024-04-25T09:05:00.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f1484ff9640 1 -- 172.21.15.37:0/1304955166 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14800bf070 con 0x7f14880cc650 2024-04-25T09:05:00.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f1484ff9640 1 -- 172.21.15.37:0/1304955166 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14800b09c0 con 0x7f14880cc650 2024-04-25T09:05:00.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148e642640 1 -- 172.21.15.37:0/1304955166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14880c5030 con 0x7f14880cc650 2024-04-25T09:05:00.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148e642640 1 -- 172.21.15.37:0/1304955166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14880c55f0 con 0x7f14880cc650 2024-04-25T09:05:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f148e642640 1 -- 172.21.15.37:0/1304955166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148804efa0 con 0x7f14880cc650 2024-04-25T09:05:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f1484ff9640 1 -- 172.21.15.37:0/1304955166 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f14800a6630 con 0x7f14880cc650 2024-04-25T09:05:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f1484ff9640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1474066f60 0x7f1474069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f1484ff9640 1 -- 172.21.15.37:0/1304955166 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f148011f880 con 0x7f14880cc650 2024-04-25T09:05:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.292+0000 7f1487fff640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1474066f60 0x7f1474069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.296+0000 7f1487fff640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1474066f60 0x7f1474069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1478007a70 tx=0x7f1478008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.296+0000 7f1484ff9640 1 -- 172.21.15.37:0/1304955166 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f14800c8020 con 0x7f14880cc650 2024-04-25T09:05:00.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.456+0000 7f148e642640 1 -- 172.21.15.37:0/1304955166 --> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f14880c6330 con 0x7f1474066f60 2024-04-25T09:05:00.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.456+0000 7f1484ff9640 1 -- 172.21.15.37:0/1304955166 <== mgr.4105 v2:172.21.15.203:6832/3562039738 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13379 (secure 0 0 0) 0x7f147800f920 con 0x7f1474066f60 2024-04-25T09:05:00.446 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:05:00.447 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-25T09:05:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 -- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1474066f60 msgr2=0x7f1474069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1474066f60 0x7f1474069420 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1478007a70 tx=0x7f1478008040 comp rx=0 tx=0).stop 2024-04-25T09:05:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 -- 172.21.15.37:0/1304955166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14880cc650 msgr2=0x7f1488154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14880cc650 0x7f1488154ea0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f1480019360 tx=0x7f14800194f0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 -- 172.21.15.37:0/1304955166 shutdown_connections 2024-04-25T09:05:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f1474066f60 0x7f1474069420 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14880cc650 0x7f1488154ea0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f14880b1ae0 0x7f148814c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 --2- 172.21.15.37:0/1304955166 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f14880b0b00 0x7f14880c5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 -- 172.21.15.37:0/1304955166 >> 172.21.15.37:0/1304955166 conn(0x7f14880b1f10 msgr2=0x7f148815b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:00.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 -- 172.21.15.37:0/1304955166 shutdown_connections 2024-04-25T09:05:00.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.460+0000 7f148e642640 1 -- 172.21.15.37:0/1304955166 wait complete. 2024-04-25T09:05:00.467 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2024-04-25T09:04:59.384487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263304,"kb_used_data":13864,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711416,"statfs":{"total":773094113280,"available":772824489984,"internally_reserved":0,"allocated":14196736,"data_stored":7280896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003468"},"pg_stats":[{"pgid":"1.0","version":"130'119","reported_seq":389,"reported_epoch":130,"state":"active+clean","last_fresh":"2024-04-25T09:04:40.085629+0000","last_change":"2024-04-25T09:04:40.085709+0000","last_active":"2024-04-25T09:04:40.085629+0000","last_peered":"2024-04-25T09:04:40.085629+0000","last_clean":"2024-04-25T09:04:40.085629+0000","last_became_active":"2024-04-25T08:52:21.103892+0000","last_became_peered":"2024-04-25T08:52:21.103892+0000","last_unstale":"2024-04-25T09:04:40.085629+0000","last_undegraded":"2024-04-25T09:04:40.085629+0000","last_fullsized":"2024-04-25T09:04:40.085629+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"130'119","last_scrub_stamp":"2024-04-25T09:04:40.085446+0000","last_deep_scrub":"130'119","last_deep_scrub_stamp":"2024-04-25T09:04:40.085446+0000","last_clean_scrub_stamp":"2024-04-25T09:04:40.085446+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:49.610994+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575003,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33344,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338496,"statfs":{"total":96636764160,"available":96602619904,"internally_reserved":0,"allocated":2220032,"data_stored":1352888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32772,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32768,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32772,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575003,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32768,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575005,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32768,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32768,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339072,"statfs":{"total":96636764160,"available":96603209728,"internally_reserved":0,"allocated":1626112,"data_stored":762520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575005,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33344,"kb_used_data":2168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338496,"statfs":{"total":96636764160,"available":96602619904,"internally_reserved":0,"allocated":2220032,"data_stored":1352888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:05:00.468 DEBUG:teuthology.orchestra.run.smithi037:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T09:05:00.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.600+0000 7f88f0e2a640 1 Processor -- start 2024-04-25T09:05:00.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.600+0000 7f88f0e2a640 1 -- start start 2024-04-25T09:05:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88f0e2a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec15da80 0x7f88ec0d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88f0e2a640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec06be60 con 0x7f88ec0d07f0 2024-04-25T09:05:00.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88f0e2a640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88ec15de60 con 0x7f88ec0d1cd0 2024-04-25T09:05:00.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88f0e2a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec06c440 con 0x7f88ec15da80 2024-04-25T09:05:00.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88ead9d640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f88ec0d07f0 0x7f88ec06c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55010/0 (socket says 172.21.15.37:55010) 2024-04-25T09:05:00.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88ead9d640 1 -- 172.21.15.37:0/2562076730 learned_addr learned my addr 172.21.15.37:0/2562076730 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:05:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88ea59c640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec15da80 0x7f88ec0d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933492401 0 0) 0x7f88ec06be60 con 0x7f88ec0d07f0 2024-04-25T09:05:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d4003530 con 0x7f88ec0d07f0 2024-04-25T09:05:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2615430090 0 0) 0x7f88d4003530 con 0x7f88ec0d07f0 2024-04-25T09:05:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec15da80 msgr2=0x7f88ec0d4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec15da80 0x7f88ec0d4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 >> v1:172.21.15.203:6789/0 conn(0x7f88ec0d1cd0 legacy=0x7f88ec0d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec0d2450 con 0x7f88ec0d07f0 2024-04-25T09:05:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f88e4001a00 con 0x7f88ec0d07f0 2024-04-25T09:05:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 >> v1:172.21.15.37:6789/0 conn(0x7f88ec0d07f0 legacy=0x7f88ec06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec15da80 0x7f88d4003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d4003fc0 0x7f88d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4004d50 0x7f88d40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec06c440 con 0x7f88d4004d50 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d400bab0 con 0x7f88d4003fc0 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d400bcb0 con 0x7f88ec15da80 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88ead9d640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4004d50 0x7f88d40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88ea59c640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec15da80 0x7f88d4003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88e9d9b640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d4003fc0 0x7f88d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88ead9d640 1 -- 172.21.15.37:0/2562076730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec15da80 msgr2=0x7f88d4003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88ead9d640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec15da80 0x7f88d4003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88ead9d640 1 -- 172.21.15.37:0/2562076730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d4003fc0 msgr2=0x7f88d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88ead9d640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d4003fc0 0x7f88d4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.604+0000 7f88ead9d640 1 -- 172.21.15.37:0/2562076730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d4003530 con 0x7f88d4004d50 2024-04-25T09:05:00.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88ead9d640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4004d50 0x7f88d40051d0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f88e400a850 tx=0x7f88e401fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7efa80a4e47c2fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e400a4a0 con 0x7f88d4004d50 2024-04-25T09:05:00.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e40aa710 con 0x7f88d4004d50 2024-04-25T09:05:00.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88e959a640 1 -- 172.21.15.37:0/2562076730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e40aab40 con 0x7f88d4004d50 2024-04-25T09:05:00.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 -- 172.21.15.37:0/2562076730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4004d50 msgr2=0x7f88d40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 --2- 172.21.15.37:0/2562076730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88d4004d50 0x7f88d40051d0 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7f88e400a850 tx=0x7f88e401fac0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88ead9d640 1 -- 172.21.15.37:0/2562076730 reap_dead start 2024-04-25T09:05:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 -- 172.21.15.37:0/2562076730 shutdown_connections 2024-04-25T09:05:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 -- 172.21.15.37:0/2562076730 >> 172.21.15.37:0/2562076730 conn(0x7f88ec08b000 msgr2=0x7f88ec0d36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 -- 172.21.15.37:0/2562076730 shutdown_connections 2024-04-25T09:05:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 -- 172.21.15.37:0/2562076730 wait complete. 2024-04-25T09:05:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 Processor -- start 2024-04-25T09:05:00.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 -- start start 2024-04-25T09:05:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d4004d50 0x7f88ec14bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88ec14e5c0 0x7f88ec14b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec14d270 0x7f88ec0b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec0d26f0 con 0x7f88ec14e5c0 2024-04-25T09:05:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec0d20f0 con 0x7f88d4004d50 2024-04-25T09:05:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88f0e2a640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88ec0d2870 con 0x7f88ec14d270 2024-04-25T09:05:00.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88e9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88ec14e5c0 0x7f88ec14b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88e9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88ec14e5c0 0x7f88ec14b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52994/0 (socket says 172.21.15.37:52994) 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88e9d9b640 1 -- 172.21.15.37:0/1607152305 learned_addr learned my addr 172.21.15.37:0/1607152305 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88ea59c640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d4004d50 0x7f88ec14bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88e9d9b640 1 -- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec14d270 msgr2=0x7f88ec0b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88ead9d640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec14d270 0x7f88ec0b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88e9d9b640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec14d270 0x7f88ec0b7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88e9d9b640 1 -- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d4004d50 msgr2=0x7f88ec14bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88e9d9b640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d4004d50 0x7f88ec14bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.608+0000 7f88e9d9b640 1 -- 172.21.15.37:0/1607152305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ec14c1d0 con 0x7f88ec14e5c0 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88e9d9b640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88ec14e5c0 0x7f88ec14b220 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f88e000e650 tx=0x7f88e000eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88daffd640 1 -- 172.21.15.37:0/1607152305 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00050d0 con 0x7f88ec14e5c0 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88daffd640 1 -- 172.21.15.37:0/1607152305 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e0018070 con 0x7f88ec14e5c0 2024-04-25T09:05:00.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88daffd640 1 -- 172.21.15.37:0/1607152305 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00054b0 con 0x7f88ec14e5c0 2024-04-25T09:05:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88f0e2a640 1 -- 172.21.15.37:0/1607152305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ec14c4b0 con 0x7f88ec14e5c0 2024-04-25T09:05:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88f0e2a640 1 -- 172.21.15.37:0/1607152305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ec14ca70 con 0x7f88ec14e5c0 2024-04-25T09:05:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88f0e2a640 1 -- 172.21.15.37:0/1607152305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ec04efa0 con 0x7f88ec14e5c0 2024-04-25T09:05:00.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88daffd640 1 -- 172.21.15.37:0/1607152305 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f88e000c6c0 con 0x7f88ec14e5c0 2024-04-25T09:05:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88daffd640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88cc066f90 0x7f88cc069450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88daffd640 1 -- 172.21.15.37:0/1607152305 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f88e0084e70 con 0x7f88ec14e5c0 2024-04-25T09:05:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88ea59c640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88cc066f90 0x7f88cc069450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.612+0000 7f88ea59c640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88cc066f90 0x7f88cc069450 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009a30 tx=0x7f88dc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.616+0000 7f88daffd640 1 -- 172.21.15.37:0/1607152305 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f88e0055ea0 con 0x7f88ec14e5c0 2024-04-25T09:05:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.776+0000 7f88f0e2a640 1 -- 172.21.15.37:0/1607152305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f88ec0cda00 con 0x7f88ec14e5c0 2024-04-25T09:05:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.784+0000 7f88daffd640 1 -- 172.21.15.37:0/1607152305 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v24)=0 v24) v1 ==== 123+0+0 (secure 0 0 0) 0x7f88e001be40 con 0x7f88ec14e5c0 2024-04-25T09:05:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.784+0000 7f88daffd640 1 -- 172.21.15.37:0/1607152305 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f88e00557a0 con 0x7f88ec14e5c0 2024-04-25T09:05:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 -- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88cc066f90 msgr2=0x7f88cc069450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88cc066f90 0x7f88cc069450 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f88dc009a30 tx=0x7f88dc008040 comp rx=0 tx=0).stop 2024-04-25T09:05:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 -- 172.21.15.37:0/1607152305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88ec14e5c0 msgr2=0x7f88ec14b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88ec14e5c0 0x7f88ec14b220 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7f88e000e650 tx=0x7f88e000eb10 comp rx=0 tx=0).stop 2024-04-25T09:05:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 -- 172.21.15.37:0/1607152305 shutdown_connections 2024-04-25T09:05:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f88cc066f90 0x7f88cc069450 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f88ec14d270 0x7f88ec0b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88ec14e5c0 0x7f88ec14b220 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 --2- 172.21.15.37:0/1607152305 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f88d4004d50 0x7f88ec14bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 -- 172.21.15.37:0/1607152305 >> 172.21.15.37:0/1607152305 conn(0x7f88ec08b000 msgr2=0x7f88ec0b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 -- 172.21.15.37:0/1607152305 shutdown_connections 2024-04-25T09:05:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.788+0000 7f88f0e2a640 1 -- 172.21.15.37:0/1607152305 wait complete. 2024-04-25T09:05:00.794 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:05:00.794 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:05:00.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f645d3c4640 1 Processor -- start 2024-04-25T09:05:00.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f645d3c4640 1 -- start start 2024-04-25T09:05:00.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f645d3c4640 1 --2- >> v2:172.21.15.203:3300/0 conn(0x7f645806c3b0 0x7f645806ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f645d3c4640 1 -- --> v1:172.21.15.37:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645815ab30 con 0x7f6458066bb0 2024-04-25T09:05:00.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f645d3c4640 1 -- --> v1:172.21.15.203:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64580cf560 con 0x7f645815a750 2024-04-25T09:05:00.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f645d3c4640 1 -- --> v2:172.21.15.203:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6458067100 con 0x7f645806c3b0 2024-04-25T09:05:00.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64577fe640 1 --1- >> v1:172.21.15.37:6789/0 conn(0x7f6458066bb0 0x7f64580b2c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.37:6789/0 says I am v1:172.21.15.37:55012/0 (socket says 172.21.15.37:55012) 2024-04-25T09:05:00.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64577fe640 1 -- 172.21.15.37:0/2673312599 learned_addr learned my addr 172.21.15.37:0/2673312599 (peer_addr_for_me v1:172.21.15.37:0/0) 2024-04-25T09:05:00.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 <== mon.0 v1:172.21.15.37:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210446527 0 0) 0x7f645815ab30 con 0x7f6458066bb0 2024-04-25T09:05:00.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 --> v1:172.21.15.37:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6440003530 con 0x7f6458066bb0 2024-04-25T09:05:00.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64567fc640 1 --2- 172.21.15.37:0/2673312599 >> v2:172.21.15.203:3300/0 conn(0x7f645806c3b0 0x7f645806ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 <== mon.0 v1:172.21.15.37:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 209204742 0 0) 0x7f6440003530 con 0x7f6458066bb0 2024-04-25T09:05:00.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 >> v2:172.21.15.203:3300/0 conn(0x7f645806c3b0 msgr2=0x7f645806ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:00.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 --2- 172.21.15.37:0/2673312599 >> v2:172.21.15.203:3300/0 conn(0x7f645806c3b0 0x7f645806ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 >> v1:172.21.15.203:6789/0 conn(0x7f645815a750 legacy=0x7f645806bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 --> v1:172.21.15.37:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64580cf840 con 0x7f6458066bb0 2024-04-25T09:05:00.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 <== mon.0 v1:172.21.15.37:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 159056366 0 0) 0x7f6450001a00 con 0x7f6458066bb0 2024-04-25T09:05:00.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 >> v1:172.21.15.37:6789/0 conn(0x7f6458066bb0 legacy=0x7f64580b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6440003a30 0x7f6440003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6440004810 0x7f6440004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6440005380 0x7f644000b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6458067100 con 0x7f6440004810 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644000c2d0 con 0x7f6440005380 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644000c4d0 con 0x7f6440003a30 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64567fc640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6440004810 0x7f6440004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f6456ffd640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6440003a30 0x7f6440003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64577fe640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6440005380 0x7f644000b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64567fc640 1 -- 172.21.15.37:0/2673312599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6440003a30 msgr2=0x7f6440003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64567fc640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f6440003a30 0x7f6440003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64567fc640 1 -- 172.21.15.37:0/2673312599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6440005380 msgr2=0x7f644000b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64567fc640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6440005380 0x7f644000b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64567fc640 1 -- 172.21.15.37:0/2673312599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6440003530 con 0x7f6440004810 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64577fe640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6440005380 0x7f644000b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.920+0000 7f64577fe640 1 -- 172.21.15.37:0/2673312599 reap_dead start 2024-04-25T09:05:00.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f64567fc640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6440004810 0x7f6440004c90 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f644800ba30 tx=0x7f644801c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9042bea9db1c47f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480b3070 con 0x7f6440004810 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f64480ae630 con 0x7f6440004810 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f6455ffb640 1 -- 172.21.15.37:0/2673312599 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480aa050 con 0x7f6440004810 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 -- 172.21.15.37:0/2673312599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6440004810 msgr2=0x7f6440004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6440004810 0x7f6440004c90 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f644800ba30 tx=0x7f644801c6a0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 -- 172.21.15.37:0/2673312599 shutdown_connections 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 --2- 172.21.15.37:0/2673312599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6440004810 0x7f6440004c90 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 -- 172.21.15.37:0/2673312599 >> 172.21.15.37:0/2673312599 conn(0x7f6458071810 msgr2=0x7f6458071c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 -- 172.21.15.37:0/2673312599 shutdown_connections 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 -- 172.21.15.37:0/2673312599 wait complete. 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 Processor -- start 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 -- start start 2024-04-25T09:05:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 --2- >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6440004810 0x7f6458152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6458155de0 0x7f6458143770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 --2- >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f645815bcb0 0x7f645815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645815ea10 con 0x7f6458155de0 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 -- --> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64580ced80 con 0x7f6440004810 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.924+0000 7f645d3c4640 1 -- --> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645815eb90 con 0x7f645815bcb0 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f64567fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6458155de0 0x7f6458143770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f64567fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6458155de0 0x7f6458143770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53022/0 (socket says 172.21.15.37:53022) 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f64567fc640 1 -- 172.21.15.37:0/824157242 learned_addr learned my addr 172.21.15.37:0/824157242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f64577fe640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f645815bcb0 0x7f645815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f6456ffd640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6440004810 0x7f6458152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f64567fc640 1 -- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f645815bcb0 msgr2=0x7f645815c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f64567fc640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f645815bcb0 0x7f645815c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f64567fc640 1 -- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6440004810 msgr2=0x7f6458152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f64567fc640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6440004810 0x7f6458152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f64567fc640 1 -- 172.21.15.37:0/824157242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6458144290 con 0x7f6458155de0 2024-04-25T09:05:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f64567fc640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6458155de0 0x7f6458143770 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f644801c4d0 tx=0x7f6440005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f643f7fe640 1 -- 172.21.15.37:0/824157242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64480b3040 con 0x7f6458155de0 2024-04-25T09:05:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f643f7fe640 1 -- 172.21.15.37:0/824157242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f64480ae430 con 0x7f6458155de0 2024-04-25T09:05:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f643f7fe640 1 -- 172.21.15.37:0/824157242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644800b600 con 0x7f6458155de0 2024-04-25T09:05:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f645d3c4640 1 -- 172.21.15.37:0/824157242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6458156260 con 0x7f6458155de0 2024-04-25T09:05:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f645d3c4640 1 -- 172.21.15.37:0/824157242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6458144500 con 0x7f6458155de0 2024-04-25T09:05:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f645d3c4640 1 -- 172.21.15.37:0/824157242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645815c670 con 0x7f6458155de0 2024-04-25T09:05:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f643f7fe640 1 -- 172.21.15.37:0/824157242 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) v1 ==== 86727+0+0 (secure 0 0 0) 0x7f64480c0020 con 0x7f6458155de0 2024-04-25T09:05:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f643f7fe640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6444066f60 0x7f6444069420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f643f7fe640 1 -- 172.21.15.37:0/824157242 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5507+0+0 (secure 0 0 0) 0x7f644811ffe0 con 0x7f6458155de0 2024-04-25T09:05:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.928+0000 7f6456ffd640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6444066f60 0x7f6444069420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.932+0000 7f6456ffd640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6444066f60 0x7f6444069420 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f644c007fd0 tx=0x7f644c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:00.932+0000 7f643f7fe640 1 -- 172.21.15.37:0/824157242 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+175264 (secure 0 0 0) 0x7f64480163b0 con 0x7f6458155de0 2024-04-25T09:05:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.128+0000 7f645d3c4640 1 -- 172.21.15.37:0/824157242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f645804efa0 con 0x7f6458155de0 2024-04-25T09:05:01.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.128+0000 7f643f7fe640 1 -- 172.21.15.37:0/824157242 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+3708 (secure 0 0 0) 0x7f64480f0b30 con 0x7f6458155de0 2024-04-25T09:05:01.119 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-25T09:05:01.120 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":112,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6779,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6836/3380427507","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6834","nonce":3380427507},{"type":"v1","addr":"172.21.15.203:6836","nonce":3380427507}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":6782,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.203:6837/2684459104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.203:6835","nonce":2684459104},{"type":"v1","addr":"172.21.15.203:6837","nonce":2684459104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6833/1209088190","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":1209088190},{"type":"v1","addr":"172.21.15.37:6833","nonce":1209088190}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":10209,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/2932825438","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":2932825438},{"type":"v1","addr":"172.21.15.37:6835","nonce":2932825438}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":112}],"filesystems":[]} 2024-04-25T09:05:01.120 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 112 2024-04-25T09:05:01.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 -- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6444066f60 msgr2=0x7f6444069420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6444066f60 0x7f6444069420 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f644c007fd0 tx=0x7f644c009040 comp rx=0 tx=0).stop 2024-04-25T09:05:01.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 -- 172.21.15.37:0/824157242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6458155de0 msgr2=0x7f6458143770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:01.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6458155de0 0x7f6458143770 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f644801c4d0 tx=0x7f6440005840 comp rx=0 tx=0).stop 2024-04-25T09:05:01.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 -- 172.21.15.37:0/824157242 shutdown_connections 2024-04-25T09:05:01.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:6832/3562039738,v1:172.21.15.203:6833/3562039738] conn(0x7f6444066f60 0x7f6444069420 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:3301/0,v1:172.21.15.203:6790/0] conn(0x7f645815bcb0 0x7f645815c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6458155de0 0x7f6458143770 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 --2- 172.21.15.37:0/824157242 >> [v2:172.21.15.203:3300/0,v1:172.21.15.203:6789/0] conn(0x7f6440004810 0x7f6458152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:01.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 -- 172.21.15.37:0/824157242 >> 172.21.15.37:0/824157242 conn(0x7f6458071810 msgr2=0x7f6458159170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:01.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 -- 172.21.15.37:0/824157242 shutdown_connections 2024-04-25T09:05:01.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-25T09:05:01.132+0000 7f645d3c4640 1 -- 172.21.15.37:0/824157242 wait complete. 2024-04-25T09:05:01.139 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T09:05:01.139 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T09:05:01.140 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:05:01.140 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:01.150 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:05:01.150 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:05:01.150 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:01.205 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:05:01.206 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:05:01.206 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:01.223 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:05:01.224 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:05:01.224 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:01.279 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:05:01.280 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T09:05:01.280 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T09:05:01.280 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:01.379 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T09:05:01.379 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T09:05:01.380 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:07.482 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T09:05:07.482 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T09:05:07.482 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:07.566 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T09:05:07.567 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T09:05:07.567 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:07.664 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T09:05:07.665 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T09:05:07.665 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:07.740 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T09:05:07.741 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T09:05:07.741 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:07.842 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T09:05:07.842 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T09:05:07.842 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:07.923 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T09:05:07.924 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T09:05:07.924 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:08.017 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T09:05:08.017 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T09:05:08.018 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T09:05:08.018 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:08.046 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T09:05:08.046 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T09:05:08.046 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:08.076 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T09:05:08.076 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T09:05:08.076 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T09:05:08.076 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:08.110 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T09:05:08.110 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T09:05:08.110 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:08.132 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T09:05:08.132 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T09:05:08.132 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:08.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-04-25T09:05:08.191 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-25T09:05:08.192 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T09:05:08.192 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T09:05:08.192 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2024-04-25T09:05:08.210 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:05:08.211 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T09:05:08.561 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:05:08.561 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T09:05:08.784 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:05:08.784 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T09:05:08.989 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:05:08.989 DEBUG:teuthology.orchestra.run.smithi037:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T09:05:09.226 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T09:05:09.226 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T09:05:09.511 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T09:05:09.511 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T09:05:09.724 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T09:05:09.724 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T09:05:09.895 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T09:05:09.895 DEBUG:teuthology.orchestra.run.smithi203:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T09:05:10.046 INFO:tasks.ceph:Archiving mon data... 2024-04-25T09:05:10.048 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997/data/mon.a.tgz 2024-04-25T09:05:10.048 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-04-25T09:05:10.054 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.bVYiWi4pwe 2024-04-25T09:05:10.054 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.bVYiWi4pwe 2024-04-25T09:05:10.200 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.bVYiWi4pwe 2024-04-25T09:05:10.260 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.bVYiWi4pwe is 445KB 2024-04-25T09:05:10.322 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.bVYiWi4pwe 2024-04-25T09:05:10.330 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997/data/mon.b.tgz 2024-04-25T09:05:10.330 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-25T09:05:10.336 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.vw1sbGtib4 2024-04-25T09:05:10.336 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.vw1sbGtib4 2024-04-25T09:05:10.491 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.vw1sbGtib4 2024-04-25T09:05:10.554 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.vw1sbGtib4 is 463KB 2024-04-25T09:05:10.617 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.vw1sbGtib4 2024-04-25T09:05:10.622 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997/data/mon.c.tgz 2024-04-25T09:05:10.622 DEBUG:teuthology.orchestra.run.smithi203:> mktemp 2024-04-25T09:05:10.667 INFO:teuthology.orchestra.run.smithi203.stdout:/tmp/tmp.J5AKYp5iDd 2024-04-25T09:05:10.667 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.J5AKYp5iDd 2024-04-25T09:05:10.823 DEBUG:teuthology.orchestra.run.smithi203:> sudo chmod 0666 /tmp/tmp.J5AKYp5iDd 2024-04-25T09:05:10.887 DEBUG:teuthology.orchestra.remote:smithi203:/tmp/tmp.J5AKYp5iDd is 463KB 2024-04-25T09:05:10.949 DEBUG:teuthology.orchestra.run.smithi203:> rm -fr /tmp/tmp.J5AKYp5iDd 2024-04-25T09:05:10.955 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T09:05:10.955 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T09:05:10.957 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T09:05:11.017 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T09:05:11.059 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T09:05:11.059 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T09:05:11.061 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997/remote/smithi037/crash 2024-04-25T09:05:11.062 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T09:05:11.075 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997/remote/smithi203/crash 2024-04-25T09:05:11.076 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T09:05:11.114 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T09:05:11.114 DEBUG:teuthology.orchestra.run.smithi037:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:11.121 DEBUG:teuthology.orchestra.run.smithi203:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:11.200 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:11.214 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:11.254 INFO:tasks.ceph:Compressing logs... 2024-04-25T09:05:11.254 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T09:05:11.257 DEBUG:teuthology.orchestra.run.smithi203:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T09:05:11.276 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51153.log 2024-04-25T09:05:11.277 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38945.log 2024-04-25T09:05:11.277 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34864.log 2024-04-25T09:05:11.278 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51153.log.gz 2024-04-25T09:05:11.278 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54588.log 2024-04-25T09:05:11.278 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38945.log.gz 2024-04-25T09:05:11.279 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45836.log 2024-04-25T09:05:11.279 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34864.log: /var/log/ceph/ceph-client.admin.54588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37678.log 2024-04-25T09:05:11.279 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54588.log.gz 2024-04-25T09:05:11.280 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57157.log 2024-04-25T09:05:11.280 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45836.log.gz 2024-04-25T09:05:11.280 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34864.log.gz 2024-04-25T09:05:11.283 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48348.log 2024-04-25T09:05:11.283 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37678.log.gz 2024-04-25T09:05:11.283 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50689.log 2024-04-25T09:05:11.283 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48348.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57157.log.gz 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48296.log 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48348.log.gz 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51670.log 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38178.log 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50689.log.gz 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48296.log.gz 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46649.log 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38178.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51670.log.gz 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45135.log 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38178.log.gz 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52445.log 2024-04-25T09:05:11.284 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45135.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46649.log.gz 2024-04-25T09:05:11.285 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42275.log 2024-04-25T09:05:11.285 INFO:teuthology.orchestra.run.smithi037.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.45135.log.gz 2024-04-25T09:05:11.285 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.52445.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35202.log 2024-04-25T09:05:11.301 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46381.log 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42275.log: /var/log/ceph/ceph-client.admin.35202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36517.log 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42275.log.gz 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52445.log.gz 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46381.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35202.log.gz 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39773.log 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46381.log.gz 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42705.log 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36517.log.gz 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43361.log 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39773.log: /var/log/ceph/ceph-client.admin.42705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53698.log 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42705.log.gz 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39773.log.gz 2024-04-25T09:05:11.302 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34810.log 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43361.log.gz 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36334.log 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53698.log.gz 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52643.log 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34810.log: /var/log/ceph/ceph-client.admin.36334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51696.log 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36334.log.gz 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34810.log.gz 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57261.log 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56928.log 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.52643.log.gz 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51696.log.gz 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50291.log 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57261.log: /var/log/ceph/ceph-client.admin.56928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40764.log 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57261.log.gz 2024-04-25T09:05:11.303 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56928.log.gz 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55025.log 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40764.log: gzip -5 --verbose 89.1% -- -- replaced with /var/log/ceph/ceph-client.admin.50291.log.gz /var/log/ceph/ceph-client.admin.35204.log 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40764.log.gz 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.55025.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27457.log 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40144.log 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55025.log.gz 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35204.log.gz 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34942.log 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.27457.log: /var/log/ceph/ceph-client.admin.40144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36699.log 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34942.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40144.log.gz 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr: 92.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.27457.log.gz -5 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34942.log.gz 2024-04-25T09:05:11.304 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44705.log 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36699.log: /var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53796.log 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36699.log.gz 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47760.log 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44705.log.gz 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53822.log 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50403.log 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47760.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53796.log.gz 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41780.log 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47760.log.gz 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53822.log.gz 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47916.log 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50403.log.gz 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55974.log 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41780.log.gz 2024-04-25T09:05:11.305 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53928.log 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47916.log: /var/log/ceph/ceph-client.admin.55974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52353.log 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47916.log.gz 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.55974.log.gz 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37390.log 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53928.log.gz 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.52353.log: /var/log/ceph/ceph-client.admin.43700.log 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46201.log 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52353.log.gz 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37390.log.gz 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53034.log 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43700.log.gz 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50760.log 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46201.log: /var/log/ceph/ceph-client.admin.53034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51722.log 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46201.log.gz 2024-04-25T09:05:11.306 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44785.log 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.53034.log.gz 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.50760.log.gz 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42249.log 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44785.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51722.log.gz -5 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54071.log 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42249.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44785.log.gz 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47537.log 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42249.log.gz 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42863.log 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47537.log: 94.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54071.log.gz 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34182.log 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47537.log.gz 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43285.log 2024-04-25T09:05:11.307 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34182.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.54175.log -- replaced with /var/log/ceph/ceph-client.admin.42863.log.gz 2024-04-25T09:05:11.308 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.308 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34182.log.gz 2024-04-25T09:05:11.308 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53956.log 2024-04-25T09:05:11.328 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27790.log 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27742.log 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.27790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27718.log 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27790.log.gz 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.27742.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.27742.log.gz 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.27718.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27718.log.gz 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27838.log 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.27838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33978.log 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27838.log.gz 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T09:05:11.329 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.3.33978.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T09:05:11.330 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T09:05:11.330 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- 91.2% -- replaced with /var/log/ceph/ceph-client.3.33978.log.gz 2024-04-25T09:05:11.330 INFO:teuthology.orchestra.run.smithi203.stderr: /var/log/ceph/ceph.audit.log 2024-04-25T09:05:11.330 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T09:05:11.330 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33862.log 2024-04-25T09:05:11.338 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-mds.d.log: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T09:05:11.338 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T09:05:11.338 INFO:teuthology.orchestra.run.smithi203.stderr: 92.6%/var/log/ceph/ceph-client.1.33862.log: -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T09:05:11.338 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T09:05:11.338 INFO:teuthology.orchestra.run.smithi203.stderr: 90.6%/var/log/ceph/ceph-mds.b.log: -- replaced with /var/log/ceph/ceph-client.1.33862.log.gz 2024-04-25T09:05:11.338 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T09:05:11.338 INFO:teuthology.orchestra.run.smithi203.stderr: 93.7%/var/log/ceph/ceph-mon.b.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T09:05:11.338 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27814.log 2024-04-25T09:05:11.338 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-osd.4.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T09:05:11.341 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27766.log 2024-04-25T09:05:11.341 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.27814.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27814.log.gz 2024-04-25T09:05:11.342 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27670.log 2024-04-25T09:05:11.342 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.27766.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27766.log.gz 2024-04-25T09:05:11.342 INFO:teuthology.orchestra.run.smithi203.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27694.log 2024-04-25T09:05:11.346 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.27670.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.27670.log.gz 2024-04-25T09:05:11.346 INFO:teuthology.orchestra.run.smithi203.stderr:/var/log/ceph/ceph-client.admin.27694.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27694.log.gz 2024-04-25T09:05:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.54175.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.54201.log 2024-04-25T09:05:11.385 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43285.log.gz 2024-04-25T09:05:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53350.log 2024-04-25T09:05:11.385 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54175.log.gz 2024-04-25T09:05:11.385 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.53956.log.gz 2024-04-25T09:05:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35203.log 2024-04-25T09:05:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53350.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54201.log.gz 2024-04-25T09:05:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46813.log 2024-04-25T09:05:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35203.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.53350.log.gz 2024-04-25T09:05:11.385 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37442.log 2024-04-25T09:05:11.385 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35203.log.gz 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54303.log 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39642.log 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46813.log.gz 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37442.log.gz 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53297.log 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.54303.log.gz 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49511.log 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38334.log 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39642.log.gz 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53297.log.gz 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50866.log 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49511.log.gz 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55726.log 2024-04-25T09:05:11.386 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38334.log: /var/log/ceph/ceph-client.admin.50866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27566.log 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.50866.log.gz 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38334.log.gz 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56673.log 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.55726.log.gz 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.27566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50663.log 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56673.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27566.log.gz 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37757.log 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56673.log.gz 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52911.log 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50663.log.gz 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40423.log 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52911.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37757.log.gz 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33831.log 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.52911.log.gz 2024-04-25T09:05:11.387 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50264.log 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40423.log: /var/log/ceph/ceph-client.admin.33831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48139.log 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33831.log.gz 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40423.log.gz 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42197.log 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48139.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.50264.log.gz 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40397.log 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48139.log.gz 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42197.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50611.log 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42197.log.gz 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43890.log 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40397.log.gz 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55951.log 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43890.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50611.log.gzgzip 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44455.log 2024-04-25T09:05:11.388 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55951.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.39053.log 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43890.log.gz 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55951.log.gz 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40658.log 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39616.log 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44455.log.gz 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40658.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39053.log.gz 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36595.log 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40658.log.gz 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52804.log 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36595.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.39616.log.gz 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33859.log 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43726.log 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36595.log.gz 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52804.log.gz 2024-04-25T09:05:11.389 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.33859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53323.log 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33859.log.gz 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46623.log 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53323.log: 89.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.43726.log.gz 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34362.log 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53323.log.gz 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36360.log 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46623.log.gz 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37730.log 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53454.log 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34362.log.gz 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37730.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36360.log.gz 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39564.log 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37730.log.gz 2024-04-25T09:05:11.390 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35198.log 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45292.log 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.53454.log.gz 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35198.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39564.log.gz 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42145.log 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35198.log.gz 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33725.log 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45292.log: /var/log/ceph/ceph-client.admin.42145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44534.log 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42145.log.gz 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45292.log.gz 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.33725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53730.log 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50343.log 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33725.log.gz 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44534.log.gz 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52248.log 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47734.log 2024-04-25T09:05:11.391 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53730.log.gz 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52248.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50343.log.gz -5 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39025.log 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52248.log.gz 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47509.log 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39025.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47734.log.gz 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44273.log 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47509.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39025.log.gz 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27676.log 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47509.log.gz 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37600.log 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.27676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54697.log 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44273.log.gz 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37600.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27676.log.gz 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37338.log 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37600.log.gz 2024-04-25T09:05:11.392 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55569.log 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46701.log 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37338.log.gz 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57313.log 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.54697.log.gz 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2%/var/log/ceph/ceph-client.admin.46701.log: -- replaced with /var/log/ceph/ceph-client.admin.55569.log.gz 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53107.log 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46701.log.gz 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47483.log 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.53107.log: /var/log/ceph/ceph-client.admin.55051.log 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57313.log.gz 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50840.log 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47483.log.gz 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.53107.log.gz 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34784.log 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50840.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55051.log.gz 2024-04-25T09:05:11.393 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53245.log 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50840.log.gz 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34784.log: /var/log/ceph/ceph-client.admin.52858.log 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45783.log 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34784.log.gz 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53245.log.gz 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38490.log 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34132.log 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52858.log.gz 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53900.log 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45783.log.gz 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34132.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38490.log.gz 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38464.log 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36308.log 2024-04-25T09:05:11.394 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.34132.log.gz 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54227.log 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53900.log.gz 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38464.log.gz 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49667.log 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36308.log: /var/log/ceph/ceph-client.admin.54227.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36308.log.gz 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45511.log 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.54227.log.gz 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52015.log 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53428.log 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52015.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45511.log.gz 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54431.log 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- 89.0%/var/log/ceph/ceph-client.admin.53428.log: /var/log/ceph/ceph-client.admin.39930.log 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52015.log.gz 2024-04-25T09:05:11.395 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53428.log.gz 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36994.log 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.49667.log.gz 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48968.log 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54431.log.gz 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39930.log: /var/log/ceph/ceph-client.admin.36994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55700.log 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.39930.log.gz 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36994.log.gz 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38022.log 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54614.log 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48968.log.gz 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38022.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55700.log.gz 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44195.log 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38022.log.gz 2024-04-25T09:05:11.396 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56902.log 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53848.log 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44195.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.54614.log.gz 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56902.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44195.log.gz 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56191.log 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56902.log.gz 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51270.log 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.56191.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33922.log 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.53848.log.gz 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51270.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56191.log.gz 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40475.log 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51270.log.gz 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.33922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35436.log 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42536.log 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33922.log.gz 2024-04-25T09:05:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35436.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40475.log.gz 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37072.log 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35436.log.gz 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45083.log 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42536.log: /var/log/ceph/ceph-client.admin.37072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37020.log 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42536.log.gz 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45083.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37072.log.gz 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55285.log 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45083.log.gz 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41115.log 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39721.log 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37020.log.gz 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41115.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55285.log.gz 2024-04-25T09:05:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52740.log 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41115.log.gz 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42837.log 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5/var/log/ceph/ceph-client.admin.52740.log: --verbose -- /var/log/ceph/ceph-client.admin.34996.log 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39721.log.gz 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42837.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.52740.log.gz 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49599.log 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42837.log.gz 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34981.log 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37046.log 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34996.log.gz 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35488.log 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49599.log.gz 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34981.log.gz 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54944.log 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.35488.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46331.log 2024-04-25T09:05:11.399 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37046.log.gz 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54944.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35488.log.gz 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45940.log 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54944.log.gz 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44677.log 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49947.log 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48165.log 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45940.log.gz 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46331.log.gz 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44677.log.gz 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.49947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36256.log 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51428.log 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48165.log.gz 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45973.log 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- 92.1% /var/log/ceph/ceph-osd.2.log 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.0.49947.log.gz 2024-04-25T09:05:11.400 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51428.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36256.log.gz 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48758.log 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51428.log.gz 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55648.log 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45973.log.gz 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56351.log 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.48758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52275.log 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55648.log: /var/log/ceph/ceph-client.0.56351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37809.log 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55648.log.gz 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38919.log 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37809.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52275.log.gz 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46125.log 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38919.log: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.37809.log.gz 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41970.log 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38919.log.gz 2024-04-25T09:05:11.401 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46761.log 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% 94.0% -- replaced with /var/log/ceph/ceph-client.admin.46125.log.gz -- replaced with /var/log/ceph/ceph-client.0.48758.log.gz 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34597.log 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41970.log: /var/log/ceph/ceph-client.admin.46761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49406.log 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41970.log.gz 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44590.log 88.3% 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46761.log.gz 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34597.log.gz 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37652.log 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49406.log: /var/log/ceph/ceph-client.admin.44590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56457.log 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49406.log.gz 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44590.log.gz 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56296.log 2024-04-25T09:05:11.402 INFO:teuthology.orchestra.run.smithi037.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.56351.log.gz 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53084.log 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37652.log.gz 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36966.log 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56296.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56457.log.gz 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53084.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56296.log.gz 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46943.log 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53084.log.gz 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50637.log 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36966.log.gz 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48018.log 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46943.log.gz 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57547.log 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50637.log: /var/log/ceph/ceph-client.admin.48018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57131.log 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48018.log.gz 2024-04-25T09:05:11.403 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46538.log 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50637.log.gz/var/log/ceph/ceph-client.admin.57547.log: 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45914.log 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57131.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57547.log.gz 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.46538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42360.log 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57131.log.gz 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39804.log 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45914.log: /var/log/ceph/ceph-client.admin.42360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50892.log 87.9% 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45914.log.gz 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42360.log.gz 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56165.log/var/log/ceph/ceph-client.admin.39804.log: 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.0.46538.log.gz 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38787.log 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39804.log.gz 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.50892.log.gz 2024-04-25T09:05:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36725.log 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35384.log 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38787.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56165.log.gz 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36725.log: gzip -5 --verbose 87.8% -- -- replaced with /var/log/ceph/ceph-client.admin.38787.log.gz /var/log/ceph/ceph-client.admin.38074.log 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35384.log: 89.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.36725.log.gz 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55077.log 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38074.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35384.log.gz 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40222.log 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38074.log.gz 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55077.log: /var/log/ceph/ceph-client.admin.40222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45266.log 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.55077.log.gz 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mon.a.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40222.log.gz 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54640.log 2024-04-25T09:05:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56127.log 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45266.log: /var/log/ceph/ceph-client.admin.54640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41702.log 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45266.log.gz 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54640.log.gz 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55103.log 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49380.log 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41702.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56127.log.gz 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37572.log 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41702.log.gz 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44072.log 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55103.log.gz 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49380.log: /var/log/ceph/ceph-client.admin.37572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38438.log 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49380.log.gz 2024-04-25T09:05:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43437.log 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37572.log.gz 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48374.log 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44072.log.gz 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38438.log: /var/log/ceph/ceph-client.admin.43437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42679.log 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38438.log.gz 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41243.log 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42679.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.43437.log.gz 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48374.log.gz 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47196.log 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41243.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42679.log.gz 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40907.log 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41243.log.gz 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35644.log 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40907.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47196.log.gz 2024-04-25T09:05:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35592.log 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40907.log.gz 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34890.log 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37626.log 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35592.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35644.log.gz 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42048.log 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35592.log.gz 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37626.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34890.log.gz 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50232.log 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42048.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37626.log.gz 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44481.log 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42048.log.gz 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49569.log 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42562.log 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50232.log.gz 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49569.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.44481.log.gz 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41892.log 2024-04-25T09:05:11.408 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49569.log.gz 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38282.log 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.42562.log.gz 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54970.log 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48270.log 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38282.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41892.log.gz 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48498.log 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38282.log.gz 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.54970.log.gz 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45161.log 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48270.log: /var/log/ceph/ceph-client.admin.48498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51906.log 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48270.log.gz 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41411.log 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51906.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45161.log.gz 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48498.log.gz 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42510.log 2024-04-25T09:05:11.409 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51906.log.gz 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53874.log 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51802.log 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41411.log.gz 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42510.log.gz 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40371.log 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38126.log 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53874.log.gz 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51802.log.gz 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41167.log 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40371.log.gz 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40196.log 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38126.log: /var/log/ceph/ceph-client.admin.41167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36569.log 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38126.log.gz 2024-04-25T09:05:11.410 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41167.log.gz 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43942.log 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40196.log.gz 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46995.log 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35800.log 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43942.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36569.log.gz 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46995.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45057.log 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43942.log.gz 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46995.log.gz 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48322.log 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36543.log 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45057.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35800.log.gz 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54666.log 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45057.log.gz 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.48322.log.gz 2024-04-25T09:05:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42915.log 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56699.log 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36543.log.gz 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54666.log: /var/log/ceph/ceph-client.admin.42915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40959.log 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.54666.log.gz 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46727.log 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56699.log: /var/log/ceph/ceph-client.admin.40959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37468.log 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42915.log.gz 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56699.log.gz 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39590.log 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40959.log.gz 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54457.log 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37468.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46727.log.gz 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39352.log 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37468.log.gz 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39590.log.gz 2024-04-25T09:05:11.412 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36198.log 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43648.log 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54457.log.gz 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39352.log: /var/log/ceph/ceph-client.admin.36198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38997.log 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:gzip/var/log/ceph/ceph-client.admin.43648.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40853.log 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36198.log.gz 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.39352.log.gz 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38997.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43648.log.gz 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40345.log 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38997.log.gz 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56831.log 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40853.log: /var/log/ceph/ceph-client.admin.40345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 88.1% 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40853.log.gz 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40345.log.gz 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35195.log 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40274.log 2024-04-25T09:05:11.413 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51506.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56831.log.gz/var/log/ceph/ceph-client.admin.35195.log: 2024-04-25T09:05:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52570.log 2024-04-25T09:05:11.414 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35195.log.gz 2024-04-25T09:05:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40274.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2024-04-25T09:05:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47339.log 2024-04-25T09:05:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52570.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40274.log.gz 2024-04-25T09:05:11.414 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43616.log 2024-04-25T09:05:11.474 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43257.log 2024-04-25T09:05:11.474 INFO:teuthology.orchestra.run.smithi037.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52570.log.gz 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35184.log 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47339.log.gz 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43257.log: /var/log/ceph/ceph-client.admin.44299.log 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43616.log.gz 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53271.log 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43257.log.gz 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44299.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.35184.log.gz -5 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46151.log 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44299.log.gz 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36148.log 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38613.log 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46151.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53271.log.gz 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.2.36148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36057.log 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36413.log 2024-04-25T09:05:11.475 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46151.log.gz 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38613.log.gz 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41567.log 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.2.36148.log.gz 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36057.log.gz 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34994.log 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36413.log: /var/log/ceph/ceph-client.admin.41567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39952.log 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36413.log.gz 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37125.log 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39952.log: 91.1%gzip 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34994.log.gz -5 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.49302.log 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41567.log.gz 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51244.log 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.39952.log.gz 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34305.log 2024-04-25T09:05:11.476 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37125.log.gz 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49302.log.gz 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53480.log 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34305.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51244.log.gz 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55901.log 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41361.log 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6%/var/log/ceph/ceph-client.admin.55901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47652.log 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53480.log.gz 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41063.log 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47652.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41361.log.gz 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43994.log 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55901.log.gz 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34706.log 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.34305.log.gz 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47652.log.gz 2024-04-25T09:05:11.477 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41063.log: /var/log/ceph/ceph-client.admin.43994.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41063.log.gz 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43994.log.gz 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56954.log 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43157.log 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34706.log.gz 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43968.log 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34414.log 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43157.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56954.log.gz 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53506.log 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34414.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43968.log.gz 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42653.log 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43157.log.gz 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53506.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34414.log.gz 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56621.log 2024-04-25T09:05:11.478 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42653.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53506.log.gz 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49694.log 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42653.log.gz 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51376.log 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49694.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56621.log.gz 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41644.log 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37364.log 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51376.log.gz 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37286.log 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47708.log 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.41644.log.gz 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37286.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37364.log.gz 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54149.log 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.49694.log.gz 2024-04-25T09:05:11.479 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37286.log.gz 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44994.log 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47708.log: gzip -5 --verbose --/var/log/ceph/ceph-client.admin.54149.log: /var/log/ceph/ceph-client.admin.44364.log 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.44994.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47708.log.gz 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38867.log 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54149.log.gz 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50001.log 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44364.log.gz 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57053.log 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50001.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38867.log.gz 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43752.log 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48627.log 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9%/var/log/ceph/ceph-client.admin.43752.log: -- replaced with /var/log/ceph/ceph-client.admin.57053.log.gz 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50001.log.gz 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44332.log 2024-04-25T09:05:11.480 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43752.log.gz 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48627.log: 92.7% -- replaced with /var/log/ceph/ceph-client.0.44994.log.gz 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48627.log.gz 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55259.log 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53662.log 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44332.log.gz 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38761.log 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50156.log 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53662.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.55259.log.gz 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37861.log 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53662.log.gz 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50156.log: gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38761.log.gz -5 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr: --verbose -- /var/log/ceph/ceph-client.admin.46787.log 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50156.log.gz 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35774.log 2024-04-25T09:05:11.481 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43838.log 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46787.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37861.log.gz 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54483.log 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46787.log.gz 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43838.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35774.log.gz 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52144.log 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43838.log.gz 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38412.log 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52144.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54483.log.gz 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56779.log 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52144.log.gz 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38412.log.gz 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37996.log 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42301.log 2024-04-25T09:05:11.482 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56779.log.gz 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39300.log 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42301.log: gzip 89.2% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.37996.log.gz -- 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/ceph-client.admin.57417.log 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42301.log.gz 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46969.log 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57417.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39300.log.gz 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54017.log 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57417.log.gz 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57521.log 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54043.log 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54017.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46969.log.gz 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46023.log 88.9% 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54017.log.gz 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.57521.log.gz 2024-04-25T09:05:11.483 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40534.log 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54043.log: /var/log/ceph/ceph-client.admin.46023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41089.log 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38100.log 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54043.log.gz 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41089.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40534.log.gz 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46023.log.gz 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52196.log 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41089.log.gz 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57495.log 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38100.log: /var/log/ceph/ceph-client.admin.52196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43916.log 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38100.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56483.log 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.52196.log.gz 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57495.log: /var/log/ceph/ceph-client.admin.43916.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57495.log.gz 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43916.log.gz 2024-04-25T09:05:11.484 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41011.log 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45323.log 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56483.log.gz 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55444.log 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57365.log 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45323.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41011.log.gz 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.55444.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45323.log.gz 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35462.log 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52620.log 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57365.log: /var/log/ceph/ceph-client.admin.35462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48918.log 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52620.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57365.log.gz 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35462.log.gz 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50918.log 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52620.log.gz 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48918.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48918.log.gz 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51611.log 2024-04-25T09:05:11.485 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50585.log 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50918.log: /var/log/ceph/ceph-client.0.51611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51828.log 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.55444.log.gz 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50585.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50918.log.gz 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47600.log 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50585.log.gz 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48599.log 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47600.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51828.log.gz 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34002.log 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55221.log 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47600.log.gz/var/log/ceph/ceph-client.admin.48599.log: 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34002.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48599.log.gz 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34002.log.gz 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45240.log 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42074.log 2024-04-25T09:05:11.486 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6%/var/log/ceph/ceph-client.admin.45240.log: -- replaced with /var/log/ceph/ceph-client.admin.55221.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50106.log 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45240.log.gz 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42074.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42074.log.gz 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.51611.log.gz 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57235.log 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48534.log 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45888.log 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48534.log: 87.5% 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50106.log.gz 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57235.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42759.log 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48534.log.gz 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45888.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45888.log.gz 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45537.log 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51774.log 2024-04-25T09:05:11.487 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40501.log 88.5% 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42759.log.gz 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40736.log 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45537.log.gz 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51774.log.gz 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47170.log 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40501.log: /var/log/ceph/ceph-client.admin.40736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56647.log 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7%/var/log/ceph/ceph-client.admin.47170.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40736.log.gz 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45731.log 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40501.log.gz 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56647.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47170.log.gz 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40827.log 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56647.log.gz 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39170.log 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37704.log 2024-04-25T09:05:11.488 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40827.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45731.log.gz 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38542.log 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr: 87.8%/var/log/ceph/ceph-client.admin.37704.log: -- replaced with /var/log/ceph/ceph-client.admin.40827.log.gz 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38587.log 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39170.log.gz 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37704.log.gz 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38542.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38542.log.gz 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56865.log 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44837.log 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38587.log.gz 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46891.log 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35540.log 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44837.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56865.log.gz 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46891.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44837.log.gz 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49328.log 2024-04-25T09:05:11.489 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44020.log 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4%/var/log/ceph/ceph-client.admin.35540.log: -- replaced with /var/log/ceph/ceph-client.admin.46891.log.gz 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41814.log 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36224.log 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49328.log.gz 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35540.log.gz 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41814.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44020.log.gz 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48994.log 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41814.log.gz 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42590.log 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36224.log: /var/log/ceph/ceph-client.admin.48994.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36224.log.gz 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50814.log 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49354.log 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42590.log: /var/log/ceph/ceph-client.admin.50814.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42590.log.gz 2024-04-25T09:05:11.490 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48994.log.gz 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr: 87.9%/var/log/ceph/ceph-client.admin.49354.log: -- replaced with /var/log/ceph/ceph-client.admin.50814.log.gz 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49354.log.gz 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53584.log 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36673.log 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46457.log 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36673.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53584.log.gz 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52778.log 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40684.log 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36673.log.gz/var/log/ceph/ceph-client.admin.46457.log: 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52778.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46457.log.gz 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52778.log.gz 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48814.log 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48458.log 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4%/var/log/ceph/ceph-client.admin.48814.log: -- replaced with /var/log/ceph/ceph-client.admin.40684.log.gz 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51181.log 2024-04-25T09:05:11.491 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48458.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48814.log.gz 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.48458.log.gz 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51454.log 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49175.log 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51454.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51181.log.gz 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53192.log 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49072.log 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51454.log.gz/var/log/ceph/ceph-client.admin.49175.log: 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53192.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53192.log.gz 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.49175.log.gz 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50507.log 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49537.log 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49072.log: /var/log/ceph/ceph-client.admin.50507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52446.log 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr: 87.7%/var/log/ceph/ceph-client.admin.49537.log: -- replaced with /var/log/ceph/ceph-client.admin.50507.log.gz 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49072.log.gz 2024-04-25T09:05:11.492 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49537.log.gz 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36621.log 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52410.log 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr: 86.2%/var/log/ceph/ceph-client.admin.36621.log: -- replaced with /var/log/ceph/ceph-client.admin.52446.log.gz 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51048.log 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36621.log.gz 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52410.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52410.log.gz 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47565.log 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46839.log 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51048.log: /var/log/ceph/ceph-client.admin.47565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49046.log 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46839.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51048.log.gz 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47565.log.gz 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40170.log 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46839.log.gz 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49046.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.49046.log.gz 2024-04-25T09:05:11.493 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43490.log 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35566.log 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40170.log: /var/log/ceph/ceph-client.admin.43490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34028.log 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35566.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40170.log.gz 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34388.log 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35566.log.gz 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43490.log.gz 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34028.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34028.log.gz 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55832.log 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41617.log 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39747.log 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34388.log: /var/log/ceph/ceph-client.admin.41617.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34388.log.gz 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55832.log.gz 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- 89.4% /var/log/ceph/ceph-client.0.43020.log 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41617.log.gz 2024-04-25T09:05:11.494 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57339.log 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39747.log: /var/log/ceph/ceph-client.0.43020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55876.log 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57339.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39747.log.gz 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.57339.log.gz 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35670.log 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40566.log 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35670.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.55876.log.gz 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35670.log.gz 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51350.log 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.0.43020.log.gz 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49459.log 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40566.log: /var/log/ceph/ceph-client.admin.51350.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40566.log.gz 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51350.log.gz 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43812.log 2024-04-25T09:05:11.495 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56980.log 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49459.log: /var/log/ceph/ceph-client.admin.43812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46227.log 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56980.log: 87.7% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49459.log.gz 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43812.log.gz 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56980.log.gz 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56509.log 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54802.log 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46227.log: /var/log/ceph/ceph-client.admin.56509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57287.log 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.54802.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56509.log.gz 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46227.log.gz 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39908.log 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41438.log 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57287.log: /var/log/ceph/ceph-client.admin.39908.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57287.log.gz 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.39908.log.gz 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37918.log 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41438.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41438.log.gz 2024-04-25T09:05:11.496 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22648.log 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44046.log 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37918.log: /var/log/ceph/ceph.tmp-client.admin.22648.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37918.log.gz 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22648.log.gz 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45109.log 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44247.log 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44046.log: 94.3% -- replaced with /var/log/ceph/ceph-client.0.54802.log.gz 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45109.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44046.log.gz 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45109.log.gz 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44939.log 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43387.log 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44247.log: /var/log/ceph/ceph-client.admin.44939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42618.log 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43387.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44247.log.gz 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44939.log.gz 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57105.log 2024-04-25T09:05:11.497 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44143.log 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1%/var/log/ceph/ceph-client.admin.42618.log: -- replaced with /var/log/ceph/ceph-client.admin.43387.log.gz 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57105.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42618.log.gz 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57105.log.gz 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48435.log 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38639.log 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44143.log: /var/log/ceph/ceph-client.admin.48435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39834.log 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38639.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44143.log.gz 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48435.log.gz 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52301.log 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38639.log.gz 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39834.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39834.log.gz 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56025.log 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41510.log 2024-04-25T09:05:11.498 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52301.log: /var/log/ceph/ceph-client.admin.56025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46407.log 2024-04-25T09:05:11.499 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41510.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52301.log.gz 2024-04-25T09:05:11.499 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34466.log 2024-04-25T09:05:11.499 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46407.log: 90.8% 92.1% -- replaced with /var/log/ceph/ceph-client.admin.56025.log.gz 2024-04-25T09:05:11.499 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41510.log.gz 2024-04-25T09:05:11.499 INFO:teuthology.orchestra.run.smithi037.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.46407.log.gz 2024-04-25T09:05:11.499 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55621.log 2024-04-25T09:05:11.499 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44811.log 2024-04-25T09:05:11.500 INFO:teuthology.orchestra.run.smithi203.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34466.log: /var/log/ceph/ceph-client.admin.55621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38841.log 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44811.log: 88.3% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34466.log.gz 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55621.log.gz 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44811.log.gz 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39886.log 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40118.log 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38841.log: /var/log/ceph/ceph-client.admin.39886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51880.log 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40118.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38841.log.gz 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.39886.log.gz 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 87.7% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.40118.log.gz /var/log/ceph/ceph-client.admin.41840.log 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41754.log 2024-04-25T09:05:11.537 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51880.log: /var/log/ceph/ceph-client.admin.41840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47457.log 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51880.log.gz 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8%/var/log/ceph/ceph-client.admin.41754.log: -- replaced with /var/log/ceph/ceph-client.admin.41840.log.gz 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41754.log.gz 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46675.log 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45587.log 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6%/var/log/ceph/ceph-client.admin.46675.log: -- replaced with /var/log/ceph/ceph-client.admin.47457.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47626.log 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46675.log.gz/var/log/ceph/ceph-client.admin.45587.log: 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45587.log.gz 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33644.log 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35514.log 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46277.log 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.33644.log: /var/log/ceph/ceph-client.admin.35514.log: 89.6% 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47626.log.gz 2024-04-25T09:05:11.538 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.33644.log.gz 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35514.log.gz 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53376.log 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35826.log 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46277.log: /var/log/ceph/ceph-client.admin.53376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56595.log 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35826.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.53376.log.gz 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46277.log.gz 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.35826.log.gz 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37177.log 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55339.log 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56595.log: /var/log/ceph/ceph-client.admin.37177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34545.log 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55339.log: 88.8% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56595.log.gz 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37177.log.gz 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57575.log 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55339.log.gz 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34545.log: /var/log/ceph/ceph.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34545.log.gz 2024-04-25T09:05:11.539 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57209.log 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49622.log 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57575.log: /var/log/ceph/ceph-client.admin.57209.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57575.log.gz 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57209.log.gz 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52688.log 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44508.log 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49622.log: /var/log/ceph/ceph-client.admin.52688.log: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34732.log 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.52688.log.gz 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44508.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.49622.log.gz 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44508.log.gz 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34985.log 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40449.log 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34732.log: /var/log/ceph/ceph-client.admin.34985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36855.log 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40449.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34732.log.gz 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34985.log.gz 2024-04-25T09:05:11.540 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47274.log 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40449.log.gz 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44429.log 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36855.log: /var/log/ceph/ceph-client.admin.47274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49842.log 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44429.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36855.log.gz 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47274.log.gz 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43335.log 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44429.log.gz 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49842.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49842.log.gz 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39669.log 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52327.log 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43335.log: /var/log/ceph/ceph-client.admin.39669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56727.log 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52327.log: 88.4% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43335.log.gz 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39669.log.gz 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52327.log.gz 2024-04-25T09:05:11.541 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53532.log 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54097.log 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56727.log: /var/log/ceph/ceph-client.admin.53532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56569.log 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54097.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56727.log.gz 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.53532.log.gz 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34991.log 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54097.log.gz 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47248.log 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56569.log: /var/log/ceph/ceph-client.admin.34991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42811.log 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47248.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34991.log.gz 89.4% 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56569.log.gz 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47248.log.gz 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51324.log 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37970.log 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42811.log: /var/log/ceph/ceph-client.admin.51324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54509.log 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42811.log.gz 2024-04-25T09:05:11.542 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51324.log.gz 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37970.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37970.log.gz 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52517.log 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37546.log 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52517.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38048.log 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.54509.log.gz 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37546.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.52517.log.gz 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37546.log.gz 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33894.log 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57027.log 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38048.log: /var/log/ceph/ceph-client.admin.33894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51556.log 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57027.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33894.log.gz 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38048.log.gz 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57027.log.gz 2024-04-25T09:05:11.543 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50317.log 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44562.log 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51556.log: /var/log/ceph/ceph-client.admin.50317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41918.log 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44562.log: 88.2% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51556.log.gz 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50317.log.gz 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44562.log.gz 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37887.log 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39222.log 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41918.log: /var/log/ceph/ceph-client.admin.37887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37229.log 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39222.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41918.log.gz 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37887.log.gz 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34758.log 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39222.log.gz 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51100.log 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37229.log: /var/log/ceph/ceph-client.admin.34758.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37229.log.gz 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34758.log.gz 2024-04-25T09:05:11.544 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39090.log 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38735.log 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45642.log 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39090.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51100.log.gz 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39090.log.gz 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38735.log: 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38735.log.gz 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48864.log 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.45642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47092.log 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37520.log 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47092.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.48864.log.gz 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47092.log.gz 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57079.log 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.45642.log.gz 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49485.log 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37520.log: /var/log/ceph/ceph-client.admin.57079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33803.log 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49485.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57079.log.gz 2024-04-25T09:05:11.545 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37520.log.gz 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49485.log.gz 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52832.log 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35410.log 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.33803.log: /var/log/ceph/ceph-client.admin.52832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36465.log 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35410.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52832.log.gz 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33803.log.gz 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35410.log.gz 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54535.log 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34916.log 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42171.log 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54535.log: 88.0% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54535.log.gz 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34916.log: -- replaced with /var/log/ceph/ceph-client.admin.36465.log.gz 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34916.log.gz 2024-04-25T09:05:11.546 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48073.log 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40933.log 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr: 87.3%/var/log/ceph/ceph-client.0.48073.log: -- replaced with /var/log/ceph/ceph-client.admin.42171.log.gzgzip 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36491.log 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40933.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40933.log.gz 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51127.log 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43674.log 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36491.log: /var/log/ceph/ceph-client.admin.51127.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36491.log.gz 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51127.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57625.log 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43674.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43674.log.gz 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.48073.log.gz 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49225.log 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55196.log 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57625.log: /var/log/ceph/ceph-client.admin.49225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50983.log 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57625.log.gz 2024-04-25T09:05:11.547 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55196.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49225.log.gz 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.55196.log.gz 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55854.log 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39248.log 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55854.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50983.log.gz 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37494.log 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44651.log 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55854.log.gz 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39248.log: /var/log/ceph/ceph-client.admin.37494.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39248.log.gz 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37494.log.gz 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45809.log 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56220.log 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44651.log: /var/log/ceph/ceph-client.admin.45809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38256.log 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44651.log.gz 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7%/var/log/ceph/ceph-client.admin.56220.log: -- replaced with /var/log/ceph/ceph-client.admin.45809.log.gz 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56220.log.gz 2024-04-25T09:05:11.548 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43132.log 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41866.log 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38256.log: /var/log/ceph/ceph-client.admin.43132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37151.log 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38256.log.gz 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41866.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43132.log.gz 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41866.log.gz 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54358.log 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35722.log 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52982.log 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.54358.log: 89.1%/var/log/ceph/ceph-client.admin.35722.log: -- replaced with /var/log/ceph/ceph-client.admin.37151.log.gz 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35722.log.gz 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48703.log 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56052.log 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48703.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52982.log.gz 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48703.log.gz 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33751.log 2024-04-25T09:05:11.549 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36647.log 2024-04-25T09:05:11.550 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56052.log: /var/log/ceph/ceph-client.admin.33751.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33751.log.gz 2024-04-25T09:05:11.550 INFO:teuthology.orchestra.run.smithi037.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.56052.log.gz 2024-04-25T09:05:11.550 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34990.log 2024-04-25T09:05:11.550 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57183.log 2024-04-25T09:05:11.550 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36647.log: /var/log/ceph/ceph-client.admin.34990.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36647.log.gz 2024-04-25T09:05:11.550 INFO:teuthology.orchestra.run.smithi037.stderr: 92.9% -- replaced with /var/log/ceph/ceph-client.0.54358.log.gz 2024-04-25T09:05:11.550 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52715.log 2024-04-25T09:05:11.550 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34990.log.gz 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57183.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57183.log.gz 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38204.log 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38893.log 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52715.log: gzip/var/log/ceph/ceph-client.admin.38204.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41388.log 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38893.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38204.log.gz 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.52715.log.gz 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38893.log.gz 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45757.log 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47786.log 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35194.log 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45757.log: 88.3% 89.6%/var/log/ceph/ceph-client.admin.47786.log: -- replaced with /var/log/ceph/ceph-client.admin.41388.log.gz 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45757.log.gz 2024-04-25T09:05:11.567 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47786.log.gz 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56753.log 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45705.log 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34519.log 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35194.log.gz 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45705.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56753.log.gz 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45705.log.gz 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40040.log 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37203.log 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34519.log: /var/log/ceph/ceph-client.admin.40040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41141.log 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37203.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34519.log.gz 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40040.log.gz 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54123.log 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37203.log.gz 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36751.log 2024-04-25T09:05:11.568 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41141.log: /var/log/ceph/ceph-client.admin.54123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46075.log 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36751.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.54123.log.gz 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41141.log.gz 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36751.log.gz 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42889.log 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34649.log 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46075.log: /var/log/ceph/ceph-client.admin.42889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48405.log 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34649.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42889.log.gz 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34649.log.gz 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.46075.log.gz 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33489.log 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35748.log 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48405.log: /var/log/ceph/ceph-client.admin.33489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41996.log 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35748.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48405.log.gz 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.33489.log.gz 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34571.log 2024-04-25T09:05:11.569 INFO:teuthology.orchestra.run.smithi037.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35748.log.gz 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36907.log 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41996.log: /var/log/ceph/ceph-client.admin.34571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41944.log 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36907.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34571.log.gz 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41996.log.gz 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36907.log.gz 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39144.log 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48653.log 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41944.log: /var/log/ceph/ceph-client.admin.39144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40879.log 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48653.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41944.log.gz 88.1% 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39144.log.gz 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36386.log 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55389.log 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48653.log.gz 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36386.log: /var/log/ceph/ceph-client.admin.40879.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40879.log.gz 2024-04-25T09:05:11.570 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36386.log.gz 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39457.log 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43080.log 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50455.log 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.39457.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55389.log.gz/var/log/ceph/ceph-client.admin.43080.log: 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43080.log.gz 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50429.log 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34106.log 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50455.log.gz/var/log/ceph/ceph-client.admin.50429.log: 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50429.log.gz 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41728.log 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51748.log 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34106.log: /var/log/ceph/ceph-client.admin.41728.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34106.log.gz 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41728.log.gz 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40066.log 2024-04-25T09:05:11.571 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51748.log: /var/log/ceph/ceph-client.admin.40066.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51748.log.gz 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40066.log.gz 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45188.log 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.45188.log: 93.1% -- replaced with /var/log/ceph/ceph-client.0.39457.log.gz 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45188.log.gz 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43540.log 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54562.log 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43540.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gz 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43540.log.gz 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53166.log 2024-04-25T09:05:11.572 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.54562.log.gz/var/log/ceph/ceph-mds.a.log: 2024-04-25T09:05:11.573 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54747.log 2024-04-25T09:05:11.573 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53166.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.53166.log.gz 2024-04-25T09:05:11.574 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50788.log 2024-04-25T09:05:11.574 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54747.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.54747.log.gz 2024-04-25T09:05:11.575 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40985.log 2024-04-25T09:05:11.575 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50788.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50788.log.gz 2024-04-25T09:05:11.576 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54997.log 2024-04-25T09:05:11.577 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40985.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40985.log.gz 2024-04-25T09:05:11.577 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57599.log 2024-04-25T09:05:11.578 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54997.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54997.log.gz 2024-04-25T09:05:11.578 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48573.log 2024-04-25T09:05:11.579 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57599.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57599.log.gz 2024-04-25T09:05:11.580 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42733.log 2024-04-25T09:05:11.581 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48573.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48573.log.gz 2024-04-25T09:05:11.581 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39402.log 2024-04-25T09:05:11.582 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42733.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42733.log.gz 2024-04-25T09:05:11.582 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55752.log 2024-04-25T09:05:11.583 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39402.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39402.log.gz 2024-04-25T09:05:11.584 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54892.log 2024-04-25T09:05:11.585 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55752.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55752.log.gz 2024-04-25T09:05:11.585 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38152.log 2024-04-25T09:05:11.586 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54892.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.54892.log.gz 2024-04-25T09:05:11.586 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40792.log 2024-04-25T09:05:11.587 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38152.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38152.log.gz 2024-04-25T09:05:11.588 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42785.log 2024-04-25T09:05:11.589 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40792.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40792.log.gz 2024-04-25T09:05:11.589 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48217.log 2024-04-25T09:05:11.590 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42785.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42785.log.gz 2024-04-25T09:05:11.590 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37098.log 2024-04-25T09:05:11.591 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48217.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48217.log.gz 2024-04-25T09:05:11.592 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45353.log 2024-04-25T09:05:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37098.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37098.log.gz 2024-04-25T09:05:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38815.log 2024-04-25T09:05:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45353.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45353.log.gz 2024-04-25T09:05:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41298.log 2024-04-25T09:05:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38815.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38815.log.gz 2024-04-25T09:05:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49251.log 2024-04-25T09:05:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.41298.log: 93.0% -- replaced with /var/log/ceph/ceph-client.0.41298.log.gz 2024-04-25T09:05:11.599 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53991.log 2024-04-25T09:05:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49251.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49251.log.gz 2024-04-25T09:05:11.601 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56805.log 2024-04-25T09:05:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53991.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.53991.log.gz 2024-04-25T09:05:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36282.log 2024-04-25T09:05:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56805.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56805.log.gz 2024-04-25T09:05:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51967.log 2024-04-25T09:05:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36282.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36282.log.gz 2024-04-25T09:05:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34054.log 2024-04-25T09:05:11.606 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51967.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51967.log.gz 2024-04-25T09:05:11.606 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56079.log 2024-04-25T09:05:11.607 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34054.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34054.log.gz 2024-04-25T09:05:11.608 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48191.log 2024-04-25T09:05:11.610 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56079.log: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.56079.log.gz 2024-04-25T09:05:11.610 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51990.log 2024-04-25T09:05:11.611 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48191.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48191.log.gz 2024-04-25T09:05:11.612 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42392.log 2024-04-25T09:05:11.613 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51990.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.51990.log.gz 2024-04-25T09:05:11.613 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57443.log 2024-04-25T09:05:11.614 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42392.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42392.log.gz 2024-04-25T09:05:11.614 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51209.log 2024-04-25T09:05:11.615 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57443.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57443.log.gz 2024-04-25T09:05:11.616 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55783.log 2024-04-25T09:05:11.616 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51209.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.51209.log.gz 2024-04-25T09:05:11.617 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50369.log 2024-04-25T09:05:11.618 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55783.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55783.log.gz 2024-04-25T09:05:11.618 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42022.log 2024-04-25T09:05:11.619 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50369.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.50369.log.gz 2024-04-25T09:05:11.619 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34440.log 2024-04-25T09:05:11.620 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42022.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42022.log.gz 2024-04-25T09:05:11.621 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48243.log 2024-04-25T09:05:11.622 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34440.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34440.log.gz 2024-04-25T09:05:11.622 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45214.log 2024-04-25T09:05:11.623 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.48243.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48243.log.gz 2024-04-25T09:05:11.623 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50481.log 2024-04-25T09:05:11.624 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45214.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45214.log.gz 2024-04-25T09:05:11.624 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47680.log 2024-04-25T09:05:11.625 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50481.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.50481.log.gz 2024-04-25T09:05:11.626 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54918.log 2024-04-25T09:05:11.627 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47680.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47680.log.gz 2024-04-25T09:05:11.627 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51402.log 2024-04-25T09:05:11.628 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54918.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54918.log.gz 2024-04-25T09:05:11.628 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51298.log 2024-04-25T09:05:11.629 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51402.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.51402.log.gz 2024-04-25T09:05:11.630 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40710.log 2024-04-25T09:05:11.631 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51298.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.51298.log.gz 2024-04-25T09:05:11.631 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46592.log 2024-04-25T09:05:11.632 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40710.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40710.log.gz 2024-04-25T09:05:11.632 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52070.log 2024-04-25T09:05:11.633 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46592.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.46592.log.gz 2024-04-25T09:05:11.634 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49724.log 2024-04-25T09:05:11.642 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.52070.log: 94.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53636.log 2024-04-25T09:05:11.644 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49724.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49724.log.gz 2024-04-25T09:05:11.644 INFO:teuthology.orchestra.run.smithi037.stderr: -- replaced with /var/log/ceph/ceph-client.0.52070.log.gz 2024-04-25T09:05:11.644 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34623.log 2024-04-25T09:05:11.645 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53636.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.53636.log.gz 2024-04-25T09:05:11.645 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37783.log 2024-04-25T09:05:11.646 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34623.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34623.log.gz 2024-04-25T09:05:11.647 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44757.log 2024-04-25T09:05:11.648 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37783.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37783.log.gz 2024-04-25T09:05:11.648 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54866.log 2024-04-25T09:05:11.649 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44757.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44757.log.gz 2024-04-25T09:05:11.649 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52910.log 2024-04-25T09:05:11.650 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54866.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.54866.log.gz 2024-04-25T09:05:11.650 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41037.log 2024-04-25T09:05:11.652 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52910.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.52910.log.gz 2024-04-25T09:05:11.652 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34995.log 2024-04-25T09:05:11.653 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41037.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41037.log.gz 2024-04-25T09:05:11.653 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39988.log 2024-04-25T09:05:11.654 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34995.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34995.log.gz 2024-04-25T09:05:11.655 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37255.log 2024-04-25T09:05:11.656 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39988.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39988.log.gz 2024-04-25T09:05:11.656 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44221.log 2024-04-25T09:05:11.657 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37255.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37255.log.gz 2024-04-25T09:05:11.657 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54253.log 2024-04-25T09:05:11.658 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44221.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44221.log.gz 2024-04-25T09:05:11.658 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44889.log 2024-04-25T09:05:11.660 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.54253.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.54253.log.gz 2024-04-25T09:05:11.660 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44625.log 2024-04-25T09:05:11.662 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44889.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44889.log.gz 2024-04-25T09:05:11.662 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55313.log 2024-04-25T09:05:11.663 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44625.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44625.log.gz 2024-04-25T09:05:11.663 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34680.log 2024-04-25T09:05:11.664 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55313.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55313.log.gz 2024-04-25T09:05:11.664 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40319.log 2024-04-25T09:05:11.665 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34680.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34680.log.gz 2024-04-25T09:05:11.666 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53402.log 2024-04-25T09:05:11.667 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40319.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40319.log.gz 2024-04-25T09:05:11.667 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47118.log 2024-04-25T09:05:11.668 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53402.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.53402.log.gz 2024-04-25T09:05:11.668 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47222.log 2024-04-25T09:05:11.669 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47118.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47118.log.gz 2024-04-25T09:05:11.669 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51854.log 2024-04-25T09:05:11.670 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47222.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47222.log.gz 2024-04-25T09:05:11.671 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33976.log 2024-04-25T09:05:11.672 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51854.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51854.log.gz 2024-04-25T09:05:11.672 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41676.log 2024-04-25T09:05:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.33976.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33976.log.gz 2024-04-25T09:05:11.673 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42965.log 2024-04-25T09:05:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41676.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41676.log.gz 2024-04-25T09:05:11.674 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44169.log 2024-04-25T09:05:11.675 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42965.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42965.log.gz 2024-04-25T09:05:11.676 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47047.log 2024-04-25T09:05:11.677 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44169.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44169.log.gz 2024-04-25T09:05:11.677 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55517.log 2024-04-25T09:05:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47047.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47047.log.gz 2024-04-25T09:05:11.678 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55810.log 2024-04-25T09:05:11.679 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55517.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55517.log.gz 2024-04-25T09:05:11.680 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40014.log 2024-04-25T09:05:11.681 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55810.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.55810.log.gz 2024-04-25T09:05:11.681 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50951.log 2024-04-25T09:05:11.682 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40014.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40014.log.gz 2024-04-25T09:05:11.682 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45862.log 2024-04-25T09:05:11.683 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50951.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50951.log.gz 2024-04-25T09:05:11.684 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47942.log 2024-04-25T09:05:11.685 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45862.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45862.log.gz 2024-04-25T09:05:11.685 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35887.log 2024-04-25T09:05:11.686 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47942.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47942.log.gz 2024-04-25T09:05:11.686 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37835.log 2024-04-25T09:05:11.687 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35887.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35887.log.gz 2024-04-25T09:05:11.688 INFO:teuthology.orchestra.run.smithi203.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T09:05:11.688 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47021.log 2024-04-25T09:05:11.688 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37835.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37835.log.gz 2024-04-25T09:05:11.689 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44731.log 2024-04-25T09:05:11.690 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47021.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47021.log.gz 2024-04-25T09:05:11.690 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33777.log 2024-04-25T09:05:11.691 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44731.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44731.log.gz 2024-04-25T09:05:11.691 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52222.log 2024-04-25T09:05:11.692 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.33777.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33777.log.gz 2024-04-25T09:05:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51480.log 2024-04-25T09:05:11.693 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52222.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.52222.log.gz 2024-04-25T09:05:11.694 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52379.log 2024-04-25T09:05:11.695 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51480.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51480.log.gz 2024-04-25T09:05:11.695 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50182.log 2024-04-25T09:05:11.696 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.52379.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.52379.log.gz 2024-04-25T09:05:11.696 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42458.log 2024-04-25T09:05:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50182.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.50182.log.gz 2024-04-25T09:05:11.698 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47838.log 2024-04-25T09:05:11.699 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42458.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42458.log.gz 2024-04-25T09:05:11.699 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50559.log 2024-04-25T09:05:11.700 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47838.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47838.log.gz 2024-04-25T09:05:11.700 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56535.log 2024-04-25T09:05:11.701 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50559.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.50559.log.gz 2024-04-25T09:05:11.702 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41193.log 2024-04-25T09:05:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56535.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56535.log.gz 2024-04-25T09:05:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33699.log 2024-04-25T09:05:11.705 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41193.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.41193.log.gz 2024-04-25T09:05:11.705 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55595.log 2024-04-25T09:05:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.33699.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33699.log.gz 2024-04-25T09:05:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43109.log 2024-04-25T09:05:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55595.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55595.log.gz 2024-04-25T09:05:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45483.log 2024-04-25T09:05:11.709 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43109.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43109.log.gz 2024-04-25T09:05:11.709 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33546.log 2024-04-25T09:05:11.710 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45483.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45483.log.gz 2024-04-25T09:05:11.710 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49122.log 2024-04-25T09:05:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.33546.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.33546.log.gz 2024-04-25T09:05:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38308.log 2024-04-25T09:05:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49122.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49122.log.gz 2024-04-25T09:05:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39118.log 2024-04-25T09:05:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38308.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38308.log.gz 2024-04-25T09:05:11.714 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36439.log 2024-04-25T09:05:11.715 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39118.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39118.log.gz 2024-04-25T09:05:11.716 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38671.log 2024-04-25T09:05:11.716 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36439.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36439.log.gz 2024-04-25T09:05:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36777.log 2024-04-25T09:05:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38671.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38671.log.gz 2024-04-25T09:05:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40248.log 2024-04-25T09:05:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36777.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36777.log.gz 2024-04-25T09:05:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51074.log 2024-04-25T09:05:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40248.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40248.log.gz 2024-04-25T09:05:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38386.log 2024-04-25T09:05:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51074.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51074.log.gz 2024-04-25T09:05:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34080.log 2024-04-25T09:05:11.723 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38386.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38386.log.gz 2024-04-25T09:05:11.723 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37416.log 2024-04-25T09:05:11.724 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34080.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34080.log.gz 2024-04-25T09:05:11.725 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53219.log 2024-04-25T09:05:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.37416.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37416.log.gz 2024-04-25T09:05:11.726 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42327.log 2024-04-25T09:05:11.727 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53219.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.53219.log.gz 2024-04-25T09:05:11.727 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41439.log 2024-04-25T09:05:11.728 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42327.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42327.log.gz 2024-04-25T09:05:11.728 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45419.log 2024-04-25T09:05:11.729 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.41439.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41439.log.gz 2024-04-25T09:05:11.730 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50734.log 2024-04-25T09:05:11.731 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45419.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45419.log.gz 2024-04-25T09:05:11.731 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47968.log 2024-04-25T09:05:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50734.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.50734.log.gz 2024-04-25T09:05:11.732 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49816.log 2024-04-25T09:05:11.734 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47968.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.47968.log.gz 2024-04-25T09:05:11.734 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49762.log 2024-04-25T09:05:11.735 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49816.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49816.log.gz 2024-04-25T09:05:11.735 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36933.log 2024-04-25T09:05:11.736 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49762.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49762.log.gz 2024-04-25T09:05:11.736 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47144.log 2024-04-25T09:05:11.737 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36933.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36933.log.gz 2024-04-25T09:05:11.738 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55543.log 2024-04-25T09:05:11.739 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47144.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47144.log.gz 2024-04-25T09:05:11.739 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44117.log 2024-04-25T09:05:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55543.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.55543.log.gz 2024-04-25T09:05:11.740 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51937.log 2024-04-25T09:05:11.741 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.44117.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44117.log.gz 2024-04-25T09:05:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47307.log 2024-04-25T09:05:11.743 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.51937.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51937.log.gz 2024-04-25T09:05:11.743 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53558.log 2024-04-25T09:05:11.744 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47307.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47307.log.gz 2024-04-25T09:05:11.744 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40092.log 2024-04-25T09:05:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53558.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.53558.log.gz 2024-04-25T09:05:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56431.log 2024-04-25T09:05:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.40092.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40092.log.gz 2024-04-25T09:05:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53610.log 2024-04-25T09:05:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56431.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56431.log.gz 2024-04-25T09:05:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49788.log 2024-04-25T09:05:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.53610.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.53610.log.gz 2024-04-25T09:05:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34838.log 2024-04-25T09:05:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49788.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49788.log.gz 2024-04-25T09:05:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50051.log 2024-04-25T09:05:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34838.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34838.log.gz 2024-04-25T09:05:11.752 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47405.log 2024-04-25T09:05:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50051.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50051.log.gz 2024-04-25T09:05:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43864.log 2024-04-25T09:05:11.754 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47405.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47405.log.gz 2024-04-25T09:05:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39864.log 2024-04-25T09:05:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43864.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43864.log.gz 2024-04-25T09:05:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39326.log 2024-04-25T09:05:11.757 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39864.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.39864.log.gz 2024-04-25T09:05:11.757 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55137.log 2024-04-25T09:05:11.758 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39326.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39326.log.gz 2024-04-25T09:05:11.759 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39538.log 2024-04-25T09:05:11.760 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.55137.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55137.log.gz 2024-04-25T09:05:11.760 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42484.log 2024-04-25T09:05:11.761 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39538.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39538.log.gz 2024-04-25T09:05:11.761 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36007.log 2024-04-25T09:05:11.762 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42484.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42484.log.gz 2024-04-25T09:05:11.767 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43786.log 2024-04-25T09:05:11.768 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.0.36007.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.36007.log.gz 2024-04-25T09:05:11.768 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38230.log 2024-04-25T09:05:11.768 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43786.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43786.log.gz 2024-04-25T09:05:11.768 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38516.log 2024-04-25T09:05:11.768 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38230.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38230.log.gz 2024-04-25T09:05:11.768 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39274.log 2024-04-25T09:05:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38516.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38516.log.gz 2024-04-25T09:05:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57469.log 2024-04-25T09:05:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39274.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39274.log.gz 2024-04-25T09:05:11.770 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39695.log 2024-04-25T09:05:11.771 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57469.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57469.log.gz 2024-04-25T09:05:11.771 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42100.log 2024-04-25T09:05:11.772 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.39695.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39695.log.gz 2024-04-25T09:05:11.772 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35696.log 2024-04-25T09:05:11.773 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.42100.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42100.log.gz 2024-04-25T09:05:11.774 INFO:teuthology.orchestra.run.smithi037.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T09:05:11.774 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37944.log 2024-04-25T09:05:11.775 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45376.log 2024-04-25T09:05:11.775 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35696.log: /var/log/ceph/ceph-client.admin.37944.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35696.log.gz 2024-04-25T09:05:11.775 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37944.log.gz 2024-04-25T09:05:11.775 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34993.log 2024-04-25T09:05:11.776 INFO:teuthology.orchestra.run.smithi203.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T09:05:11.777 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33579.log 2024-04-25T09:05:11.777 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45376.log: /var/log/ceph/ceph-client.admin.34993.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.45376.log.gz 2024-04-25T09:05:11.777 INFO:teuthology.orchestra.run.smithi037.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34993.log.gz 2024-04-25T09:05:11.777 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52170.log 2024-04-25T09:05:11.777 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34492.log 2024-04-25T09:05:11.777 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.33579.log: /var/log/ceph/ceph-client.admin.52170.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33579.log.gz 2024-04-25T09:05:11.779 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35618.log 2024-04-25T09:05:11.779 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52170.log.gz 2024-04-25T09:05:11.779 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.34492.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34492.log.gz 2024-04-25T09:05:11.779 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43207.log 2024-04-25T09:05:11.779 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49433.log 2024-04-25T09:05:11.779 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35618.log: /var/log/ceph/ceph-client.admin.43207.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35618.log.gz 2024-04-25T09:05:11.780 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47431.log 2024-04-25T09:05:11.780 INFO:teuthology.orchestra.run.smithi037.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43207.log.gz 2024-04-25T09:05:11.781 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.49433.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.49433.log.gz 2024-04-25T09:05:11.781 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55674.log 2024-04-25T09:05:11.781 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50533.log 2024-04-25T09:05:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47431.log: /var/log/ceph/ceph-client.admin.55674.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47431.log.gz 2024-04-25T09:05:11.782 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47864.log 2024-04-25T09:05:11.783 INFO:teuthology.orchestra.run.smithi037.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.55674.log.gz 2024-04-25T09:05:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.50533.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50533.log.gz 2024-04-25T09:05:11.783 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33950.log 2024-04-25T09:05:11.784 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46865.log 2024-04-25T09:05:11.784 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.47864.log: /var/log/ceph/ceph-client.admin.33950.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47864.log.gz 2024-04-25T09:05:11.785 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33950.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43566.log 2024-04-25T09:05:11.785 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:11.785 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46865.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46865.log.gz 2024-04-25T09:05:11.786 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47890.log 2024-04-25T09:05:11.786 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36829.log 2024-04-25T09:05:11.787 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.43566.log: /var/log/ceph/ceph-client.admin.47890.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47890.log.gz 2024-04-25T09:05:11.787 INFO:teuthology.orchestra.run.smithi037.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43566.log.gz 2024-04-25T09:05:11.787 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47812.log 2024-04-25T09:05:11.788 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36881.log 2024-04-25T09:05:11.788 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36829.log: /var/log/ceph/ceph-client.admin.47812.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36829.log.gz 2024-04-25T09:05:11.789 INFO:teuthology.orchestra.run.smithi037.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47812.log.gz 2024-04-25T09:05:11.789 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44863.log 2024-04-25T09:05:11.790 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36803.log 2024-04-25T09:05:11.790 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36881.log: /var/log/ceph/ceph-client.admin.44863.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36881.log.gz 2024-04-25T09:05:11.790 INFO:teuthology.orchestra.run.smithi037.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44863.log.gz 2024-04-25T09:05:11.790 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37312.log 2024-04-25T09:05:11.791 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46483.log 2024-04-25T09:05:11.791 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.36803.log: /var/log/ceph/ceph-client.admin.37312.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36803.log.gz 2024-04-25T09:05:11.791 INFO:teuthology.orchestra.run.smithi037.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37312.log.gz 2024-04-25T09:05:11.791 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39196.log 2024-04-25T09:05:11.792 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57391.log 2024-04-25T09:05:11.792 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.46483.log: /var/log/ceph/ceph-client.admin.39196.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46483.log.gz 2024-04-25T09:05:11.793 INFO:teuthology.orchestra.run.smithi037.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39196.log.gz 2024-04-25T09:05:11.793 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38360.log 2024-04-25T09:05:11.793 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38971.log 2024-04-25T09:05:11.794 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.57391.log: /var/log/ceph/ceph-client.admin.38360.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57391.log.gz 2024-04-25T09:05:11.794 INFO:teuthology.orchestra.run.smithi037.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38360.log.gz 2024-04-25T09:05:11.794 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40632.log 2024-04-25T09:05:11.794 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45457.log 2024-04-25T09:05:11.795 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.38971.log: /var/log/ceph/ceph-client.admin.40632.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38971.log.gz 2024-04-25T09:05:11.795 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40632.log.gz 2024-04-25T09:05:11.795 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46917.log 2024-04-25T09:05:11.796 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35201.log 2024-04-25T09:05:11.796 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.45457.log: /var/log/ceph/ceph-client.admin.46917.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45457.log.gz 2024-04-25T09:05:11.796 INFO:teuthology.orchestra.run.smithi037.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46917.log.gz 2024-04-25T09:05:11.796 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42223.log 2024-04-25T09:05:11.797 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56246.log 2024-04-25T09:05:11.797 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.35201.log: /var/log/ceph/ceph-client.admin.42223.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35201.log.gz 2024-04-25T09:05:11.798 INFO:teuthology.orchestra.run.smithi037.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42223.log.gz 2024-04-25T09:05:11.798 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33670.log 2024-04-25T09:05:11.799 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/ceph-client.admin.56246.log: /var/log/ceph/ceph-client.admin.33670.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33670.log.gz 2024-04-25T09:05:11.799 INFO:teuthology.orchestra.run.smithi037.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.56246.log.gz 2024-04-25T09:05:11.861 INFO:teuthology.orchestra.run.smithi037.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T09:05:12.222 INFO:teuthology.orchestra.run.smithi037.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T09:05:12.569 INFO:teuthology.orchestra.run.smithi203.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T09:05:12.602 INFO:teuthology.orchestra.run.smithi203.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T09:05:12.715 INFO:teuthology.orchestra.run.smithi203.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T09:05:12.719 INFO:teuthology.orchestra.run.smithi037.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T09:05:12.778 INFO:teuthology.orchestra.run.smithi203.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T09:05:12.780 INFO:teuthology.orchestra.run.smithi203.stderr: 2024-04-25T09:05:12.780 INFO:teuthology.orchestra.run.smithi203.stderr:real 0m1.479s 2024-04-25T09:05:12.780 INFO:teuthology.orchestra.run.smithi203.stderr:user 0m6.232s 2024-04-25T09:05:12.780 INFO:teuthology.orchestra.run.smithi203.stderr:sys 0m0.186s 2024-04-25T09:05:12.865 INFO:teuthology.orchestra.run.smithi037.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T09:05:13.017 INFO:teuthology.orchestra.run.smithi037.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T09:05:13.043 INFO:teuthology.orchestra.run.smithi037.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T09:05:13.044 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-25T09:05:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m1.784s 2024-04-25T09:05:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m8.056s 2024-04-25T09:05:13.044 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.507s 2024-04-25T09:05:13.044 INFO:tasks.ceph:Archiving logs... 2024-04-25T09:05:13.045 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997/remote/smithi037/log 2024-04-25T09:05:13.045 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T09:05:13.196 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-04-25T09:05:13.197 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-25T09:05:14.587 DEBUG:teuthology.misc:Transferring archived files from smithi203:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997/remote/smithi203/log 2024-04-25T09:05:14.588 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T09:05:15.349 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T09:05:15.358 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T09:05:15.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T09:05:15.361 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T09:05:15.446 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T09:05:15.543 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T09:05:15.543 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:05:15.546 DEBUG:teuthology.orchestra.run.smithi203:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:05:15.558 INFO:teuthology.orchestra.run.smithi037.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T09:05:15.558 INFO:teuthology.orchestra.run.smithi037.stdout:============================================================================== 2024-04-25T09:05:15.558 INFO:teuthology.orchestra.run.smithi037.stdout:+hv01.front.sepi 44.4.53.2 2 u 8 64 377 0.075 -3.244 0.486 2024-04-25T09:05:15.558 INFO:teuthology.orchestra.run.smithi037.stdout:+hv02.front.sepi 63.231.80.2 3 u 9 64 377 0.080 -0.705 0.446 2024-04-25T09:05:15.558 INFO:teuthology.orchestra.run.smithi037.stdout:*hv03.front.sepi 74.6.168.72 3 u 4 64 377 0.077 -2.908 0.794 2024-04-25T09:05:15.558 INFO:teuthology.orchestra.run.smithi037.stdout: hv04.front.sepi .INIT. 16 u - 512 0 0.000 +0.000 0.000 2024-04-25T09:05:15.560 INFO:teuthology.orchestra.run.smithi203.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T09:05:15.561 INFO:teuthology.orchestra.run.smithi203.stdout:============================================================================== 2024-04-25T09:05:15.561 INFO:teuthology.orchestra.run.smithi203.stdout:+hv01.front.sepi 44.4.53.2 2 u 3 64 377 0.062 -7.796 2.911 2024-04-25T09:05:15.561 INFO:teuthology.orchestra.run.smithi203.stdout:+hv02.front.sepi 63.231.80.2 3 u 3 64 377 0.075 -2.786 1.977 2024-04-25T09:05:15.561 INFO:teuthology.orchestra.run.smithi203.stdout:*hv03.front.sepi 74.6.168.72 3 u 7 64 377 0.088 -2.818 2.690 2024-04-25T09:05:15.561 INFO:teuthology.orchestra.run.smithi203.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-04-25T09:05:15.562 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T09:05:15.572 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T09:05:15.573 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T09:05:15.601 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T09:05:15.626 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T09:05:15.660 INFO:teuthology.task.internal:Duration was 1422.898930 seconds 2024-04-25T09:05:15.661 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T09:05:15.677 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T09:05:15.677 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T09:05:15.680 DEBUG:teuthology.orchestra.run.smithi203:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T09:05:15.717 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T09:05:15.718 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-04-25T09:05:15.718 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T09:05:15.769 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi203.front.sepia.ceph.com 2024-04-25T09:05:15.769 DEBUG:teuthology.orchestra.run.smithi203:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T09:05:15.779 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T09:05:15.780 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T09:05:15.813 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T09:05:15.851 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T09:05:15.852 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T09:05:15.889 DEBUG:teuthology.orchestra.run.smithi203:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T09:05:15.968 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T09:05:15.980 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T09:05:15.980 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T09:05:15.993 DEBUG:teuthology.orchestra.run.smithi203:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T09:05:16.004 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T09:05:16.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:05:16.037 DEBUG:teuthology.orchestra.run.smithi203:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:05:16.046 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-04-25T09:05:16.055 INFO:teuthology.orchestra.run.smithi203.stdout:kernel.core_pattern = core 2024-04-25T09:05:16.067 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:05:16.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:05:16.104 DEBUG:teuthology.orchestra.run.smithi203:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:05:16.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:05:16.114 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T09:05:16.125 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T09:05:16.126 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997/remote/smithi037 2024-04-25T09:05:16.126 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T09:05:16.169 DEBUG:teuthology.misc:Transferring archived files from smithi203:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7672997/remote/smithi203 2024-04-25T09:05:16.169 DEBUG:teuthology.orchestra.run.smithi203:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T09:05:16.190 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T09:05:16.191 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T09:05:16.209 DEBUG:teuthology.orchestra.run.smithi203:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T09:05:16.236 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T09:05:16.248 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T09:05:16.248 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T09:05:16.268 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T09:05:16.268 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T09:05:16.271 DEBUG:teuthology.orchestra.run.smithi203:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T09:05:16.274 INFO:teuthology.orchestra.run.smithi037.stdout: 658628 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 09:05 /home/ubuntu/cephtest 2024-04-25T09:05:16.287 INFO:teuthology.orchestra.run.smithi203.stdout: 658640 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 09:05 /home/ubuntu/cephtest 2024-04-25T09:05:16.289 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T09:05:16.300 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T09:05:16.337 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1422.8989300727844 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T09:05:16.337 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:05:16.407 INFO:teuthology.run:pass